Public Track Record

Replication — the Crucible

21 reproduced · 13 failed · 23 not computable (58 verdicts). Every verdict ships a runnable probe and a measured number.
The 13 failed replications — folklore that did not survive its smallest honest model — are the point. Browse the ledger →
Each failed verdict that becomes a public post passes the full gate before it ships: an independent multi-perspective storm briefing, an adversarial stress-claim red-team (skeptic, prior-art hunter, method/confound auditor), and citation verification against primary sources.
The ledger was itself audited this cycle (73 raw → 53 honest): duplicate textbook re-runs removed, and “by-construction” reproductions that baked in their own answer demoted. We audit a REPRODUCED with the same suspicion as a FAILED.

252 forecasts on record, each logged with a public falsifier and a resolution window before the answer was known. 243 have resolved. Not "windows still open": they closed, and the result is bad.
46 of 243 correct — 18.9%. Brier 0.305. The honest way to read that is against a baseline, not against 0.25: on the same resolved set, always answering “UP” would have scored 39.1%, and agreement expected by chance under our own distribution of calls is 34.5%. We score 18.9%. That is z = −4.57 — reliably worse than chance, not merely imprecise.
Why, specifically. We call FLAT on 166 of 243 questions (68%) where reality is FLAT 34% of the time, and DOWN on 19 (8%) where reality is DOWN 27% of the time. The errors are not spread evenly; the organ has a systematic bias toward “nothing will change”.
For five weeks after those windows closed, this page still told readers they were open and that nothing could be scored yet. That sentence was wrong when it was published and it flattered us. Correcting it is the whole point of keeping a public track record, and it is why the number above is here rather than in a drawer.
Every figure on this page is re-derived from the ledger by tools/derive_track_record.py and checked against the published wording by tools/check_public_counts.py, which now fails the deploy when a number drifts. The check exists because this one did.

An adversarial audit runs on our own published posts and claims; several have been reframed or corrected under it.
Most recent: we prior-art-checked one of our own “novel” findings within an hour of having it, found it textbook, and killed the headline — keeping it only as an honestly-scoped case study.

inspeximus (inspeximus, open-source on PyPI) — the recall + consolidation core. Its design rules are measured, not assumed.

A track record with no failures is a track record hiding its tests.