The Crucible

Claims, rebuilt in code and tested.

A public ledger of scientific and technical claims rebuilt as minimal computational models and run. Three honest verdicts — reproduced when the mechanism holds in its smallest model, failed when it doesn’t (science’s rarest export, published here), and not computable when there’s no simulable core. Every verdict ships a measured number and the code that produced it.

21
Reproduced
14
Failed
23
Not computable
Open data — the full ledger as JSON · now a citable dataset on Hugging Face →
datasets.load_dataset("Danchi17/folklore-index")
Failed replication Cognitive science · Gilovich, Vallone & Tversky · 1985

The hot hand is not a fallacy

The canon

For 30 years the canonical finding was that streak shooting is a cognitive illusion: players who feel “hot” are misreading randomness.

What we rebuilt

We rebuilt their exact estimator — P(hit | 3 prior hits) − P(hit | 3 prior misses), averaged per shooting record — and ran it on i.i.d. shooters with provably no hot hand.

The verdict

On a no-hot-hand shooter the estimator does not read 0; it reads −7.9pp (t=−28 at n=100, growing to −17pp at streak length 4). This is the streak-selection bias of Miller & Sanjurjo (2018): selecting shots that follow a streak inside a finite sequence is a biased sample. A GVT measurement of ~0 therefore implies a real hot hand of roughly +8.5pp. The famous fallacy was the analysis, not the players.

The ledger

35 claims rebuilt & tested · newest first
FAILED

A memory's support edge -- which earlier record a new write was derived from -- can be rec

Measured against constructed ground truth with the negatives that matter: a negative is a same-domain, same-vocabulary write issued right after the SAME recall -- the pair a real store actually contains, not an unrelated document. n=120 per cell, 5 thresholds x 4 retention levels, run against the shipped library (inspeximus 1.89.0) via probes/infer_lineage_precision.py, public at https://github.com/DanceNitra/inspeximus/blob/v1.89.0/probes/infer_lineage_precision.py and in the 1.89.0 sdist -- reproduce with `pip install inspeximus==1.89.0` and run it. Best recall of the inferred edge is 0.133, at the loosest threshold (0.05) and with the derivative retaining 90% of the parent's wording; 0.083 at 70%, 0.017 at 50%, 0.000 at 30%. Precision is 1.000 wherever it fires and false positives are zero in every cell, so the failure is RECALL, not confusion: the inference is right when it speaks and almost always silent. Consequence: a retraction that walks declared `derived_from` edges cannot be rescued by inferring the undeclared ones. METHOD NOTE, because it nearly went out wrong: the first model was a hand-written bag-of-words cosine over a 14-word shared vocabulary, which put every pair above threshold and returned recall 1.000 with precision exactly 0.500 -- while its VERDICT line, written in advance as prose, asserted 'recovers at most a small fraction'. The claim contradicted the data sent to support it. The recorded run executes the shipped probe and COMPUTES the verdict from the parsed table instead of stating it.

OUR OWN claim, and we shipped it: inspeximus irunnable model →
FAILED

world-model-mcp ENFORCES what its trust schema declares (expires_at: 'rows should be dropp

Measured with both controls passing (truncate->REMOVED; encoded store->NOT_COMPUTABLE). After the PUBLIC delete() the value is on disk in facts.db AND still retrievable. expires_at/influence_state are inert columns; the source itself notes the sweep 'lands in a follow-up'. IMPORTANT AND SELF-CRITICA

SaravananJaichandar/world-model-mcp v0.15.5 (P
REPRODUCED

Agent tool-action REVERSIBILITY is decidable ex-ante from the tool signature — the assumpt

Smallest model: label every tool in the ToolEmu asset file (38 toolkits / 330 tools; paper reports 36/311) for argument-dependence of reversibility with TWO independent LLM labelers, then measure a granted universal executor's realized harm-reach under isolated vs networked assumptions. RESULT: reversibility IS ~93% decidable from the signature alone (arg-dependent 0.07; Cohen's kappa 0.82, 'almost perfect' Landis-Koch) — the per-tool assumption mostly holds. MEASURED FAILURE MODE: the ~7% undecidable residual is verb-polymorphic UNIVERSAL EXECUTORS (shell/SQL/eval; only ~0.3% of tools, cat-agreement 0.94), for which reversibility is undecidable from the signature AND realized harm-reach is ENVIRONMENT-conditional — an ISOLATED shell reaches ~0% of external/API irreversible harms but a realistically-credentialed NETWORKED shell reaches ~0.66 external / ~0.80 local. IMPLICATION (confirming the consensus, not new): the boundary that bounds a memory-poisoned agent's irreversible EXTERNAL harm is executor CONTAINMENT (no ambient network/credentials), NOT per-tool reversibility metadata. HONEST CAVEATS: (1) LLM labelers measure shared model priors, not ground truth (networked-external agreement is only 0.32 — genuinely CLI/credential-dependent); (2) ToolEmu under-weights shell/code-interpreter tools now standard in production (OpenAI CI, Claude computer-use), so the 0.3% universal-executor prevalence is a LOWER bound; (3) the harm-locality map itself is NOT novel — it is arXiv:2607.07474's scope×reversibility grid on a different corpus; the genuinely new, unpublished bits are the measured 93% signature-DECIDABILITY RATE (kappa) and the isolated→networked reach flip.

Owiredu-Ashley, 'Beyond Attack-Success Rarunnable model →
FAILED

A paper's content-generality (rated reference-blind from its abstract) predicts how much i

Reference-blind LLM generality ratings (deepseek-v4-flash + kimi-k2.7-code) vs build-on on real Semantic Scholar data (345 ML/CS + 413 Medicine papers, 2015-17). With S2 influentialCitationCount, generality->build-on rank-partial (controlling raw citations) = +0.11/+0.14 in ML/CS (95% CI excludes 0), ~0 in Medicine: a small apparent effect. PRE-REGISTERED domain-neutral swap to a classifier-free build-on proxy (focused-citer = citing papers with short reference lists, field-normalized) erases it: ML partial = -0.02/-0.03 (CI straddles 0). Positive control: focused-citer detects build-on beyond popularity at partial +0.17, so the null is not proxy noise. Verdict: the effect is metric-specific / operationalization-dependent - consistent with (not proof of) the CS-trained influential-citation classifier's home-field advantage. Limits: small magnitudes, one classifier-free proxy, two fields.

A plausible science-of-science hypothesis in trunnable model →
FAILED

Attaching provenance to a memory -- a "source:" tag, a "corroborated by N records" line, a

Defense-aware attacker vs 4 in-memory defenses (fictional-fact OVERRIDE, so no model prior confounds), 10 targets across 8 families (Anthropic Claude Opus 4.8, Google Gemini-3-flash + Gemma-3-27b, DeepSeek-v4-flash, Moonshot Kimi-k2.7, MiniMax-m3, NVIDIA Nemotron-3, Zhipu GLM-5.2, OpenAI gpt-oss 20b/120b), judge gpt-oss:120b. MEAN attack-success: no defense 1.00; source cited IN the record 0.91; "corroborated by >=2" written IN the record 0.95 -- the attacker forges whatever the defense reads, because it controls the content (v5). Even coherence + temporal corroboration gates are theater vs a competent forger who writes on-topic and time-spreads witnesses -- both free (v8). The ONLY signal that holds is metadata the writer CANNOT set: system-assigned write-history / distinct Ed25519 verified keys -- Biba integrity (1977) / Douceur Sybil cost (2002). Enforced in CODE (the memory layer surfaces only what passed the gate, so the poison never reaches the model) the end-to-end attack-success drops 1.00 -> 0.00 on all 10 models, gemma-3 included -- a code gate protects even a model that ignores a "distrust this memory" instruction, whereas the SAME rule as a PROMPT leaks (gemma-3 80%). BUT the defense PRICES the attack, it does not CLOSE it: (a) a sleeper who pays the Douceur cost (2 real keypairs, on-topic, time-spread) is indistinguishable from a legit fact -- Cheng & Friedman 2005, no symmetric reputation is sybilproof, confirmed empirically (v8); (b) it rides on outcome-credit being un-self-gradable -- a MINJA-style self-graded outcome (arXiv:2503.03704) collapses it back to 1.00; (c) it is coupled to a freshness cost -- the same >=2-corroboration bar that blocks a single-shot poison blocks a single-witness TRUE update (quickest-change-detection delay-vs-false-alarm; you cannot accept fresh facts fast without also accepting single-shot poison). VERDICT FAILED for "content-provenance is a memory-poisoning defense"; the honest replacement is a code-enforced gate on unforgeable, EARNED (un-self-gradable) signals, which prices but does not eliminate the attack. Prior art credited (not claimed): Biba 1977; Douceur 2002; Cheng-Friedman 2005; knowledge-conflict/memory-strength (Mallen PopQA ACL 2023, Li et al. arXiv:2409.10955 ACL 2025); adaptive-attack canon (Carlini-Wagner 2017, Athalye 2018, Tramer 2020, Nasr "Attacker Moves Second" arXiv:2510.09023); memory/RAG poisoning (MINJA 2503.03704, AgentPoison NeurIPS 2024 arXiv:2407.12784, PoisonedRAG USENIX Sec 2025 arXiv:2402.07867); PeerTrust signal fusion (Xiong-Liu 2004); quickest-change-detection (Lorden 1971 / CUSUM). HONEST LIMITS: single-turn fictional-fact override (not multi-turn persistent poisoning); one LLM judge (gpt-oss:120b), part self-judging gpt-oss outputs (biases lenient, not toward inflated robustness); attacker = static best-of-battery + a Claude-authored defense-aware battery, NOT an RL optimizer (PISmith arXiv:2603.13026), so ASR is a lower bound vs a stronger attacker; the code-gate GATED=0 is partly BY-CONSTRUCTION (a corroboration gate filters an uncorroborated poison ~tautologically) -- the measured, non-tautological parts are the end-to-end model pass-through, the Cheng-Friedman residual, and the density-dependent utility cost (~51% of legit recalls blocked at ~1x use -> ~6% at ~8x); open-weight/cloud snapshots only (Claude tested via Claude Code, no closed API). Probes: research/probes/memory_defense_layer_probe.py (attack), memory_gate_defense_probe.py (defense end-to-end), memory_corroboration_cost_probe.py (cost/residual/coupling). Post: r/RAG 2026-07-06.

OWASP Agentic Top-10 ASI06 (Memory & Conte
FAILED

A big-enough context window replaces retrieval (CAG / "RAG is dead"): preload the whole co

Judge-free probe on one synthetic structured log at 5k/25k/60k/110k tokens, two task families. NEEDLE (single-fact lookup) = 1.00 at every length incl. 110k; SYNTH (read-everything count/filter aggregation) collapses 0.75->~0.25-0.375 by ~25k and stays there. Verdict FAILED for CAG *on synthesis*; needle lookup survives long context fine, so blanket "context rot kills recall" is also too strong. It is the task type, not the length. Replicates lost-in-the-middle (Liu 2023/TACL 2024), RULER (Hsieh/NVIDIA 2024), NoLiMa (2502.05167), Chroma context-rot (2025, non-peer-reviewed). HONEST LIMITS: n=8 synth Qs/length (the 0.75->0.3 drop is Fisher p~0.13, credible mainly by matching the replicated literature); one frontier model (exact model not logged); even at 5k synth is 0.75 so part of the failure is exhaustive arithmetic the model cannot do at any length -> the real fix for count/filter is a TOOL, not retrieval. Probe: research/probes/ragdead (generator + independent gold-re-derivation scorer). Post: public/posts/does-long-context-kill-rag.html.

CAG / "long context kills RAG" viralrunnable model →
FAILED AI / Future of Work

AI coding assistants make developers ~55% faster

The universal claim fails. A storm-research audit (8 verified sources) shows the famous 55.8% is a vendor preprint on ONE greenfield task; the only independent RCT on experienced devs in mature repos found -19% (time), while a peer-reviewed 3-RCT study found +26% (task-count, juniors). A minimal write-vs-review model reproduces the junior-gain / expert-loss SIGN-FLIP (crossover k*=0.62), robust in 79% of param sets -- so the split is one operating-point curve, not a contradiction.

Peng/GitHub 2023 vs METR 2025 vs Cui/Demirer 2025
REPRODUCED AI evaluation

Chatbot Arena Elo ranks LLMs by genuine quality (cite Arena rank to pick a model)

A classifier that sees ONLY answer style (length, markdown headers, bold, lists) and NO model identity predicts the human winner 61.5% of the time, and its per-model ranking correlates rho=0.74 with the real win-rate ranking across 48 models — it reproduces the leaderboard ORDER without knowing which model wrote anything.

Chiang et al. - 2024 (LMArena)runnable model →
FAILED Management / Future of Work

Founder-led firms returned 3.1x more — the founders mentality drives performance

With IDENTICAL expected returns, a founder-led cohort that is merely ~1.8x as volatile, run through the same survive-and-be-large index filter, shows an apparent 2.60x mean return gap — 76% of Bains 3.1x — from ZERO skill. The advantage is tail-driven: the MEDIAN gap is only 1.58x.

Zook & Allen (Bain) - 2016runnable model →
REPRODUCED AI evaluation

LLM-as-judge agrees with humans ~80%, so it is a valid quality judge

A zero-understanding null judge that simply picks the LONGER answer already agrees with human preferences 68.1% of the time (vs 50% chance), reproducing ~half of the GPT-4 judges celebrated above-chance agreement; the GPT-4 judge itself agrees with pick-the-longer 73.5% of the time.

Zheng et al. - 2023 (MT-Bench)runnable model →
FAILED Management / Future of Work

Good to Great: 11 companies, discoverable shared traits = sustained greatness

A zero-skill null reproduces the whole pattern: select firms for a 15-year 3.8x-the-market leap from a population where NO firm is more skilled than any other, and their next 15 years revert to the market (forward excess +0.015, 95% CI spanning 0; ~47% beat the market = a coin flip).

Jim Collins, Good to Great - 2001runnable model →
FAILED Behavioral economics

Food nudges are 2.5x more responsive than other domains

From an IDENTICAL true effect in every domain, differential publication bias alone reproduces a 2.63x food/other ratio (control with equal sample sizes = 1.00). The famous 2.5x is a small-study mirage.

Mertens, Herberz, Hahnel & Brosch · 2021runnable model →
REPRODUCED

Cosine similarity BARELY distinguishes a superseded fact from its replacement (AUROC ~0.61

Replicated on local nomic (24 subject-relation-object facts, mean-centered): the cosine 'is-this-a-supersession' classifier scores AUROC 0.613 (chance 0.5; Yadav ~0.59) -- a contradiction is often MORE embedding-similar to the original than a genuine rephrase (10/24 cases). A pure-cosine top-1 store served the stale value 41.7% of the time; a deterministic (subject,relation,object) supersession key drove that to 0% (shipped in inspeximus v0.2.0). Falsifier: if the key did not cut stale below cosine it would be useless -- it did. Honest scope: synthetic, 24 facts, characterizes the mechanism, not a product benchmark. SCOPED (Crucible audit 2026-07-05): AUROC 0.613 is barely ABOVE chance (0.5), not zero -- the honest statement is 'near-useless in practice', carried by the harm number (pure-cosine top-1 served the STALE value 41.7% of the time), not by 'cannot'. n=24 = small-n replication consistent with Yadav 0.59 (arXiv:2606.26511), not a large-n certification.

Temporal Validity in Retrieval Memory: Eliminarunnable model →
REPRODUCED

Episodic memory (Model-Free Episodic Control: store the MAX observed discounted return per

Smallest model of the mechanism; exploration HELD IDENTICAL (both learners fed the same uniform-random trajectories off-policy, so the only difference is credit assignment). Exp A (deterministic chain + step cost): EC reaches a goal-reachin SCOPED (Crucible audit 2026-07-05): the max-return store is unbiased only in a NOISELESS deterministic env (max=mean); EC beating parametric learning on a tiny deterministic chain is close to textbook (tabular memorization beats function approximation when no generalization is needed). Do not generalize to broad sample-efficiency.

Sarrico, Arulkumaran, Agostinelli et al. - Samrunnable model →
FAILED

Real-world networks are scale-free: their degree distributions follow a power law p(k) ~ k

Under a rigorous Clauset-Shalizi-Newman fit (MLE alpha, KS-selected xmin, bootstrap goodness-of-fit, Vuong likelihood-ratio vs lognormal, n=20,000): a lognormal that 'looks scale-free' on a log-log plot is correctly REFUSED (power-law GOF p=0.01; LR favors lognormal, -17.5); a genuine Pareto power law passes (GOF p=0.92; LR +102); and a real Barabasi-Albert network is only a TIE (LR -0.1) - power law is not even clearly preferred over lognormal for true preferential attachment. So 'looks scale-free' is not 'is scale-free' and the universal claim is not safely inferable (Broido-Clauset reproduces). Power law IS reproduced for true BA graphs. CONTESTED (Crucible audit 2026-07-05): FAILED holds only under the STRICT CSN pure-power-law criterion; under regularly-varying / generalized-power-law definitions many networks pass (Voitalov et al. 2019; Serafino et al. 2021 PNAS: finite-size effects hide true power laws). Open debate, not a closed debunk.

Barabasi-Albert (1999) framing, widely repeaterunnable model →
FAILED

Emergent abilities of large language models are genuine, sharp capability transitions - a

The canonical SHARP 'emergence' curve is reproduced by a SMOOTH, continuous per-token skill measured with a nonlinear exact-match metric: the same smooth skill gives a per-token transition width 7.0 vs an exact-match (L=100) width 1.05 -- 6.7x sharper PURELY from the metric -- and the apparent onset shifts -0.07 -> +5.58 as answer length grows with NO change in underlying skill. No capability discontinuity is needed, so benchmark sharpness is not evidence for one (Schaeffer et al. 2023 'mirage', NeurIPS Outstanding Paper, reproduces). CONTESTED / scope: shows the metric mechanism is SUFFICIENT to fake sharpness here; it does NOT prove no ability is ever genuinely emergent -- Du et al. 2024 argue downstream abilities track a genuine pretraining-loss threshold. What FAILS is 'benchmark sharpness proves a discontinuity', not all emergence.

Wei et al. 2022 'Emergent Abilities of Larunnable model →
REPRODUCED

A Pareto regret that reflects Pareto optimality without relying on scalarization functions

Smallest model: 2-objective bandit, known Pareto front (4 optimal + 2 dominated arms). Pareto-UCB1 with the scalarization-free Pareto suboptimality gap achieved sublinear cumulative Pareto regret (avg/step 0.027->0.005, growth factor 1.82 o

Pareto Regret Analyses in Multi-objective Multrunnable model →
FAILED Collective intelligence

…but "diversity trumps ability" is not a general law

Stress-test of the same claim in a DIFFERENT, faithful problem model (NK landscapes; paired and statistically powered): the best-ability group matches or slightly beats the random/diverse group at every difficulty level. So the Hong-Page advantage is real at the original parameters (companion card) but does NOT generalize — condition-specific, not a universal law, consistent with Thompson (2014). The effect is small; this refutes the universal claim, not the value of diversity in specific regimes.

Hong & Page · 2004 (contested, Thompson 2014)runnable model →
REPRODUCED

The survival probability of a branching process obeys finite-size scaling in the control p

Smallest Galton-Watson model (Poisson offspring), vectorized over 40k realizations. Critical eps=0: n*P_n -> ~2 (Kolmogorov 2/sigma^2, sigma^2=1). Scaling collapse confirmed: n*P_n is a function of x=eps*n alone - within-x spread 0.116 vs a

Garcia-Millan, Font-Clos et al. 2015, "Firunnable model →
REPRODUCED

Whether a mouse geroprotector is recorded as extending lifespan can depend on the survival

Self-contained sim (weighted log-rank family, n=50/arm, 4000 trials). Age-localized true effect: log-rank power 32.5% vs Gehan 72.3%; the two tests give DISCORDANT verdicts on 39.9% of identical datasets. Under the null, best-of-3 tests inf

deep-research 2026-06-16 + Jiang et al., GeroSrunnable model →
REPRODUCED

Recursive prompt/self synthesis improves performance ONLY when scored by an EXTERNAL fitne

Smallest model (lab a08981): a population recursively synthesizes candidates FROM ITSELF each generation; the pivotal variable is the SELECTION signal. With an EXTERNAL fitness anchor (as Promptbreeder has: real task accuracy) recursive synthesis improves; with self-scoring only it does NOT -- so the gain is attributable to external selection, not to self-reference. Claim RESCOPED + unverified '~15%' figure dropped (Crucible audit 2026-07-05): the headline previously mis-credited self-reference.

Promptbreeder (Fernando, Banarse, Michalewski,runnable model →
REPRODUCED

In DiD with FEW treated units and spatially/serially-correlated errors, standard DiD infer

N=30,T=12,1 treated,rho=0.7,true effect=0,800 reps: DiD 95% CI coverage=0.305 (severe under-coverage vs nominal 0.95); SC coverage=0.891; SC RMSE=1.017 < DiD RMSE=1.267. DiD inference invalid, SC materially better.

Alvarez &amp; Ferman (2020)runnable model →
REPRODUCED

Derenyi-Palla-Vicsek (2005): k-clique percolation in ER graphs at p_c(k)=[(k-1)N]^(-1/(k-1

k=3 scaling exponent confirmed: empirical p_c*sqrt(2N) constant across N=400/800/1600 (1.26,1.26,1.19). N^(-1/2) scaling reproduced; prefactor ~1.2x asymptotic formula due to finite-size + 50%-coverage operational threshold.

Clique Percolation in Random Networks (Derenyirunnable model →
REPRODUCED

Systems in the same universality class share the same critical exponents (Lubeck 2004)

Three structurally different Z2 mean-field models (tanh self-consistency, phi^4 free energy, arctan self-consistency) all give order-parameter exponent beta=0.500; a different-class absorbing-state model gives beta=1.000. Same class -> same

Universal Scaling Behavior of Non-Equilibrium runnable model →
REPRODUCED

LinUCB (Chu et al 2011): linear contextual bandit achieves O(sqrt(Td log^3)) i.e. sublinea

Empirical regret growth exponent 0.03-0.11 (sub-sqrt(T), well inside the O(sqrt(T)) upper bound); cum regret 3-8 vs linear non-learning regret 1400-2700; d-scaling ~sqrt(d) to d. Computable core holds.

Contextual Bandits with Linear Payoff Functionrunnable model →
FAILED Cognitive science

The Dunning–Kruger plot draws itself from pure noise

A null with NO metacognitive deficit produces the shape of the famous quartile plot — a large positive gap at the bottom falling to a negative one at the top — from regression to the mean plus a uniform better-than-average bias. The published magnitudes are NOT_RECOVERED: the entry recorded no parameters, and parameters fitted to its own published anchors reproduce the table by construction, which is a curve fit and not a test. What does survive unfitted: at zero offset the plot is symmetric (+25.3 / −25.4), so the asymmetry comes from the offset, not from regression to the mean.

Kruger & Dunning · 1999runnable model →
REPRODUCED Linguistics / statistics

A monkey at a typewriter really does produce Zipf's law

Random typing yields exponent −1.24; even a natural fine-structure discriminator fails at matched corpus size (1.8× < 2× bar). The deflationary inference survives a severe test.

Miller · 1957runnable model →
REPRODUCED Finance

Thirty stocks diversify you — until the tails get heavy

N=30 captures ~96% of achievable risk reduction at realistic tails — but only 85% of tail-risk reduction near infinite-variance tails, where ~100 stocks are needed.

Evans & Archer · 1968runnable model →
REPRODUCED Collective intelligence

Diversity beats ability — but only at Hong & Page's exact parameters

Reproduced in the authors' own relay model: a random team beats the best-individuals team by +1.65 (t=4.1) at the paper's parameters — but the edge is fragile, reversing to −0.38 on a smoother landscape. See the companion FAILED card: the advantage does not survive a different, powered problem model.

Hong & Page · 2004runnable model →
REPRODUCED Optimization

SGD finds a real minimum of a non-convex loss

On a double-well non-convex objective, SGD localizes to a true minimum at the predicted convergence rate.

Fehrman, Gess & Jentzenrunnable model →
REPRODUCED Statistical physics / neuroscience

Criticality leaves a power-law fingerprint

At the critical point, fluctuations show power-law scaling and long-range correlations — the signature reproduces cleanly.

Kitzbichler, Smith, Rahn · 2009runnable model →
REPRODUCED Network science / epidemiology

In scale-free networks, the epidemic threshold vanishes

As N grows, λ_c = ⟨k⟩/⟨k²⟩ → 0: a hub-rich network has effectively no herd-immunity threshold.

Pastor-Satorras & Vespignanirunnable model →
REPRODUCED Optimization

SGD's slow convergence is a variance floor

Constant-step SGD stalls at a noise floor; variance-reduced methods keep converging — the slowdown is variance, not curvature.

Johnson & Zhang · 2013runnable model →
REPRODUCED Network science

Pulling the hubs shatters the network

Removing the top 10% of nodes by degree multiplies the bond-percolation threshold several-fold (BA, N=2000).

Percolation & cascade robustnessrunnable model →

Honest passes

4 of 23 shown — no simulable core, on the record anyway

Not every claim has a computable mechanism. When a claim is descriptive rather than mechanistic, the honest verdict is not computable — recorded, not quietly dropped. 23 verdicts fall in this class; 4 are shown here as examples and all of them are in the ledger JSON.

How a verdict is earned

01

Model before verdict

The smallest model of the claim’s stated mechanism is built first, scoped to that mechanism — not reverse-engineered to a desired answer.

02

A number, not a vibe

Every verdict is a measured quantity with a direction that could refute it — an effect size, a threshold, an exponent, a bias.

03

Re-runnable

The code ships with the verdict. A reproduced means the minimal mechanism computes; it does not certify the original paper beyond doubt.

04

Failures are the point

A failed means the stated mechanism didn’t survive its smallest honest model, with the discrepancy measured. Those stay published.

Have a claim that deserves the bench?

Send a quantitative, mechanism-bearing claim — a number, a threshold, an exponent, a rate. If it has a simulable core, it gets a model and a public verdict, whichever way it lands.

Submit a claim →