Insights · Method
The replication problem is not unique to psi_
Psychology, medicine and economics all found the same crack in the foundations. What the psi literature can borrow from how they responded.
- 08 October 2025
- Implausible Research
- 6 min read
When a landmark psi result fails to replicate, the usual reading is that the phenomenon was never there. That reading would be more convincing if replication rates in mainstream psychology, cancer biology and experimental economics were not sitting in the same uncomfortable range.
The honest conclusion is narrower and more useful: small effects measured with flexible analysis pipelines do not survive independent repetition, whatever the field.
What actually fixed things elsewhere
Three changes did most of the work: pre-registration of the analysis plan, adequately powered samples decided in advance, and registered reports where the journal commits to publish before the result is known.
None of them require agreeing on whether an effect exists. They only require that the answer be decided before the data arrives.
How we apply it
Every Implausible series is hashed and time-stamped before the first session runs: hypothesis, sample size, exclusion rules, primary and secondary outcomes. Deviations are logged and reported as deviations.
We also run adversarial replication by default — a second team, given the protocol and no access to the first dataset, repeats the series before we describe any result as a finding.
More insights