Methodology
How we verify sources
What this site checks about its sources, and what it does not. Every figure below is measured from the data at build time.
Two Layers
Verification has two layers
“Backed by sources” actually means two separate things. Conflating them makes unchecked ground read as checked.
Bibliographic check
A machine can settle this
We query Crossref and PubMed to confirm that a DOI or PubMed ID really points at the paper we say it does. A mismatch in first author, publication year or journal is detected here, as is any retraction or correction on record at the time of the check. That result is a snapshot: a retraction issued afterwards is not reflected until the next check (the check date is shown on each study's page). A new build using a lock older than 365 days will fail, but nothing published is updated until a new build runs. This check is automated, and the site fails to build if it does not pass.
Content check
Matched against the abstract, item by item
Even when the identifier is right, a summary or a “what this study shows” line can claim more than the paper measured. Checking the identifier cannot detect this. It requires matching the record against the original abstract item by item: sample size, measured outcomes, study design and population. That comparison is performed by AI and is not an expert assessment of the study's validity.
Most of the errors found in our July 2026 audit were of the second kind: the DOI was correct, but the text described an outcome the paper never measured. Checking identifiers alone will never surface those.
Current Numbers
Current numbers
All figures below are counted from the data at build time. Numbers we have not reached are shown in the same place as the ones we have.
- Studies with bibliography checked
- 168 / 168
- 100%
- Studies with content checked
- 39 / 168
- 23%
- Supplements with sources attached
- 0 / 90
- Supplement records cannot yet carry sources
- Sources registered as exceptions
- 4
- All reasons are listed below
Content checks by what was verified
- sample size22 / 39
- measured outcomes39 / 39
- study design39 / 39
- population studied24 / 39
Of 168 study records, 129 have not had their content checked. They are published with the bibliographic check alone, and that state is shown on each study's own page.
And to be plain about it: of the 30 pre-existing records whose content we have checked so far, 26 needed correction. Most errors take the form of a correctly cited paper whose stated findings go beyond what that paper actually measured.
Those 30 were not randomly sampled — we are working in order of how often each record is cited by our articles — so the rate may not carry over unchanged to the 129 still unchecked. We have no grounds for assuming it is lower, either. A further 9 content-verified records were written from the abstract during this audit; needing no correction was a foregone conclusion for those, so they are excluded from the rate above.
Badges
What the badges mean
- Checked against the abstract
- The record has been matched against the original abstract and its claims stay within what the paper reports. What was checked is listed under “Verification breakdown” on each page. The comparison is performed by AI and is not an expert judgement of validity.
- Content not checked
- The bibliography (authors, year, journal) has been checked, but the text has not yet been matched against the abstract. This does not mean the record is wrong — it means we have not checked.
Every change to this site is reviewed by the operator before publication. That review covers the change itself; it does not necessarily involve re-reading the abstract. What the badge records is that the AI comparison was made.
Retractions
Withdrawn study records
Records we withdrew because they could not be reconciled with primary sources. We keep what each one claimed, down to the sample size. Without a record of corrections, the claim to verify anything cannot itself be verified. These ids are never reused.
Effects of curcumin supplementation on exercise-induced inflammation and DOMS — meta-analysis
curcumin-inflammation-exercise-meta
Yavari A, Javadi M, Mirmiran P, Bahadoran Z / Asian Journal of Sports Medicine / 2015 / n=857
The cited Yavari et al. (2015) Asian J Sports Med is in fact “Exercise-Induced Oxidative Stress and Dietary Antioxidants”, a narrative review of antioxidants in general, not a meta-analysis of curcumin. The sample size of 857 does not appear in the original paper either.
Effects of HIIT on the preservation of muscle mass and hypertrophy (meta-analysis)
hiit-muscle-preservation-meta
Batacan RB Jr, et al. / British Journal of Sports Medicine / 2017 / n=872
The cited Batacan et al. (2017) BJSM is in fact “Effects of high-intensity interval training on cardiometabolic health”, which assessed VO2max, blood pressure, glucose and lipids. Preservation of muscle mass was never measured.
Effects of plyometric training on muscular power and sports performance (meta-analysis)
plyometric-training-power-meta
Saez de Villarreal E, et al. / Journal of Strength and Conditioning Research / 2012 / n=1015
No meta-analysis on muscular power and sports performance in general by Saez de Villarreal et al. (2012) in JSCR could be found. What does exist in that journal and year is a sprint-specific analysis (PMID 22240550), which is a different paper from what the record described.
Effects of CoQ10 supplementation on cardiovascular function and energy metabolism — meta-analysis
coq10-mitochondria-energy-meta
Mantle D, Hargreaves IP, et al. / Nutrients (MDPI) / 2022 / n=2149
No meta-analysis on cardiovascular function and energy metabolism co-authored by Mantle & Hargreaves (2022) could be found. The registered DOI pointed to Borymska (2022), a study on silymarin.
Withdrawn:
Exceptions
Sources registered as exceptions
Sources that cannot pass the automated check are registered as exceptions with a stated reason. An exception applies only when both the identifier and the target file match, and it records which failure it covers. The build warns about exceptions that are no longer used.
doi:10.1519/jsc.0000000000002848
data/articles/resistance-bands-strength-truth.json
Exception applies to: handling of a retraction or correction
The retraction notice itself is cited deliberately. It is presented alongside the original paper (doi:10.1519/jsc.0000000000000971) so that readers can verify the 2018 retraction for themselves. The article text also states that the paper was retracted.
doi:10.1038/sj.ejcn.1602899
data/studies/zma-testosterone-rct.json
Exception applies to: publication-year mismatch
Koehler 2009, Eur J Clin Nutr 63(1):65-70. Crossref returns 2007, the online-first date, which differs from the 2009 print issue. The paper itself matches.
doi:10.1519/jsc.0000000000000971
data/articles/resistance-bands-strength-truth.json
Exception applies to: handling of a retraction or correction
Soria-Gila 2015. The article deliberately discusses this paper as “a retracted study”; it was retracted in 2018. The citation label also marks it as retracted in 2018.
doi:10.3109/09546631003801619
data/studies/glutathione-skin-pigmentation-rct.json
Exception applies to: publication-year mismatch
Arjinpathana 2012, J Dermatolog Treat 23(2):97-102. Crossref returns 2010, the online-first date, which differs from the 2012 print issue. The paper itself matches.
The full list of sources is on the sources page