How it works
For standard test statistics, the p-value is uniquely determined by the statistic and its degrees of freedom: t = 2.5 with df = 28 implies a two-tailed p of 0.0185…, no wiggle room. When a paper prints the statistic, the df, and the p-value together, the three form a verifiable consistency promise.
Two kinds of inconsistency are flagged:
- Exact-value mismatch: the recomputed p, rounded to the reported precision, does not equal the reported value (e.g., reported p = .05, recomputed .019).
- Significance decision error: the paper claims significance (p < .05) but the recomputed p ≥ .05, or vice versa — the most serious kind, as it changes conclusions.
A worked example
A paper reports: t(20) = 1.9, p = .04.
- Recomputed: exact two-tailed p = 0.0719
- Rounded to 2 decimals: .07 ≠ .04 — exact-value mismatch
- And the paper claims significance (.04 < .05) while 0.0719 > .05 — significance decision error
Using this method, Nuijten et al. (2016) scanned a quarter of a million psychology papers published 1985–2013 and found that roughly half contained at least one p-value inconsistency, and about one in eight contained at least one decision error affecting the significance conclusion.
Applicability & sources of false alarms
- t, r, and z are computed two-tailed by default — one-tailed reports will be misflagged and need manual review
- APA recommends 3-decimal p-values and never writing p = .000; “p = .000” (really < .001) can cause false alarms
- When the statistic itself is rounded (e.g., t to 2 decimals), the recomputed p carries natural error — this tool rounds to the reported precision to tolerate it
- Passages describing results purely in words (“significant”) without statistics cannot be checked
What an inconsistency actually means
A p-value inconsistency is not fabrication. Nuijten et al. concluded the vast majority stem from typos, copy-paste errors, rounding habits, and the trial-and-error traces of the analysis process. But dense, systematic inconsistencies — especially ones that always nudge toward significance — are worth pursuing as a lead, cross-checked with GRIM/SPRITE and the statistical fingerprints.
Source
Nuijten MB, Hartgerink CHJ, van Assen MALM, Epskamp S, Wicherts JM (2016). The prevalence of statistical reporting errors in psychology (1985–2013). Behavior Research Methods, 48, 1205–1226. Original tool: statcheck.io
⚠️ Important: statcheck-lite and every method on this site are screening aids. No single anomaly is sufficient to determine data fabrication. Final determinations must be made by competent institutions through due process.