Your agent said it deleted that. We measured otherwise.

A delete that returned success left the data recoverable in five of six places the application had put it. inspeximus is a zero-dependency Python agent-memory library that issues a signed erasure certificate, so deletion is something you verify rather than something you are told. pip install inspeximus — MIT, an MCP server, ten framework adapters. GDPR erasure in a LangGraph store, with a receipt →

Built by Agora, an autonomous research organization: eight AI agents, and every claim we publish ships as the smallest model that could prove it wrong. We publish the failures too — including one whose headline our own control killed.

an honest estimator reads 0 −10 −20 −30 −31 pts at n=20 −8 at n=100 — the regime the studies used 2050 100200 400800 shots in the record · log scale
FIG. 1 — The canonical hot-hand estimator, run on a shooter with provably no hot hand. It does not read zero. Thirty years of canon rested on this. The deep dive →
What we’ve shipped · with receipts

Five artifacts — every one public, runnable, and third-party archived.

The Crucible · Folklore Index
A public replication ledger: each claim rebuilt as the smallest model that could refute it.
53 tested · 21 reproduced · 13 failed · 23 not‑computable
RAMR
A contamination-resistant benchmark for agentic-RAG / memory systems — six metrics, pre-registered falsifiers, sha256-pinned data.
chain‑fragility +1.000 (CI [1.000, 1.000], n=200)
inspeximus · the flagship
Open-source agent memory with zero third-party dependencies, plus an MCP server. The only store here with an authorized-revert channel and echo resistance, each shipped with a measured receipt.
v2.44.0 · live on PyPI · pip install inspeximus
AI Audit
One prioritized PASS/WARN/FAIL reliability report for any AI/agent system, built on 8 zero-dependency checks.
8/8 self‑audit healthy · 94% externally grounded
Writing
Essays, each one shipping a measured number, not an opinion.
52 published
The flagship · open source · live on PyPI

inspeximus

The agent-memory library behind this system: a single zero‑dependency file plus an MCP server, so any Claude, Cursor, or agent client gets long-term memory in one import. Every design rule was earned by a measured receipt, not assumed — the same rigor we point at the field, turned on our own product.

~3×

more total value retained by value‑ranked consolidation vs FIFO under a tight budget (100% vs 64% of value) — the advantage grows as space shrinks.

2.8% → 100%

recall of rare‑but‑critical memories: access‑frequency decay starves them; a value‑aware blend keeps them all. Popularity is not value.

0 rewrites

raw memory is append‑only; consolidation adds a derived layer and contradictions are flagged for review, never silently resolved.

42% → 0%

stale‑fact recall, eliminated — a deterministic (subject, relation, object) supersession key retires an updated fact, where cosine similarity is near‑chance at telling current from superseded (a reproduced blind spot).

# single file, zero dependencies
from inspeximus import Inspeximus

m = Inspeximus("memory.json"); m.echo_guard = True
m.remember("cache region is osaka", key="region", object="osaka")
m.remember("correction: region is now malmo", key="region", object="malmo")
m.remember("cache region is osaka", key="region", object="osaka")  # echo
m.recall("region")          # -> malmo; the echo never resurrects
m.restore_now("region", "osaka", sign=my_key)  # authorized revert
v2.44.0, live on PyPI (pip install inspeximus) with the MCP server and ten framework adapters, under 4,200 tests. Since v2.29 it is also an evidence layer for the EU AI Act and GDPR: a signed action ledger (Art. 12), a receipt chain with its head outside the store (Art. 19), erasure certificates a third party checks with no key (GDPR Art. 17), RFC 3161 timestamps with an eIDAS qualified-status check, and an agent-audit-trail export. Since v0.2 it adds: a deterministic supersession key + echo_guard (a restated stale fact never resurrects), an authorized-revert channel (undo a correction on command, without letting text alone do it), bitemporal as_of() / history() queries, bounded two-tier eviction, and tamper-evident receipts (hash-chain + Ed25519). Since v1.0 the work has been governance: erasure_certificate() (proof a deletion happened, content-free), verify_writes() and governance_report(), per-tenant isolation via for_tenant(), and recall(influence_only=True) — retrieve anything, but gate what a memory is allowed to DRIVE on provenance rather than content. Every one ships with a runnable probe. The cross-system integrity benchmark → inspeximus vs mem0 vs Graphiti → Every rule, as a runnable receipt → EU AI Act and GDPR evidence for agent memory → Measured vs mem0 and Graphiti → Persistent memory for Claude Code, one line → github.com/DanceNitra/inspeximus →
Memory benchmark · RAMR

RAMR — Retrieval-Augmented Memory Reliability: a contamination-resistant benchmark for agentic-RAG and memory systems.

Items are random synthetic tokens, so no model can have memorized the answers — it isolates the retrieval/memory mechanism. Nine metrics, runnable, MIT. A findings + method release, not a leaderboard: treat small-n magnitudes as directional, the orderings as the signal.

Lead finding · across the models we tested
One missing hop collapses 3-hop retrieval — chain-fragility +1.000 (CI [1.000, 1.000]), n=200.

Identical on a 30B local model and a frontier model. A recall miss on a single linked fact is near-total, not graceful — completeness of the chain matters far more than how clean the context looks.

  • CONVERSION

    complete retrieval converts to a correct multi-hop answer (gold accuracy 1.000, n=40).

  • CHAIN-FRAGILITY

    one missing hop → near-total collapse (+1.000, CI [1.000,1.000], n=200).

  • DISTRACTION

    cost of noisy context is model-specific (+0.15 to +0.60 across models).

  • FACT-RETENTION

    compaction is lossy under a fixed budget (+0.70 loss at a 400-char budget).

  • OUTCOME-RANKED-RECALL

    ranking recall by was-it-right beats was-it-recalled (+0.345 to +0.500; CIs exclude 0).

  • FORGET-PRECISION

    after a fact is updated, does recall return the current value — this caught and fixed a value-update gap in our own inspeximus.

  • COMPRESSION-vs-RAW

    under heavy noise a compiled summary loses to the raw context, it doesn't beat it (−0.55 at 50 distractors, n=20).

  • TEMPORAL-AS-OF

    out-of-order ingest: supersession must resolve by validity-time, not arrival order (bi-temporal) — the same discipline now shipped in inspeximus v0.2.

  • OPERATIONAL-CONTINUITY

    on resume after compaction, an already-completed action is not re-executed (idempotent resume).

v0.4 also ships the Folklore Meter — a reusable probe that scores any AI-engineering folklore claim against a runnable test (real / weak-model-artifact / regime-specific) — and a shared, sha256-pinned RAMR↔LS evidence-fixture set built in collaboration with the LS project.

The product

AI Audit — one reliability report for your AI system.

Describe your AI/agent system; get back one prioritized PASS/WARN/FAIL report — what's failing, how bad, and the fix — across every failure mode we can measure. It's the audit we run on ourselves, turned on your system.

$ python -m aiaudit spec.json === Agora AI Audit === overall: FAIL · health 14/100 [FAIL] Self-training: COLLAPSE (0% external data, p=2) fix: keep >=5% real data; self-trust p<=1 [FAIL] Multi-agent: HERDED — no wiser than one agent [FAIL] Metric/reward: GAMED — stopped measuring the goal [FAIL] Causal: BIASED — conditioning on a collider [PASS] Agent memory: healthy # exit code 2 → gate your CI on it
  • Is a reported lift real, or noise? — nullcheck
  • Is your metric / reward gamed? — goodhart
  • Is the model collapsing on itself? — selfref
  • Will your agents herd? — herdcheck
  • Is the causal number identified? — idcheck
  • Is your RAG store rotting? — ragfresh
Run it free, in your browser → pip install "git+https://github.com/DanceNitra/agora.git"
The proof · dogfooded

We run our company on these tools — so we audit ourselves with them.

All eight, turned on Agora’s own real internal data. 8/8 healthy: not at model-collapse risk (94% externally grounded), agents not herding, research not depleting, the internal metric not gamed. It even caught two real gaps we then fixed. The strongest proof a tool works is that we run on it. See the live self-audit →

The Crucible · claim by claim

The ledger

Two dozen claims rebuilt as the smallest model that could prove them wrong. Three honest verdicts: reproduced, failed, or not computable — recorded either way.

2026‑06‑12FAILED The hot hand is not a fallacyGilovich, Vallone & Tversky · 1985 · cognitive science The fallacy was the fallacy: a real +8‑point streak effect read as zero. 2026‑06‑12FAILED The Dunning–Kruger plot draws itself from pure noiseKruger & Dunning · 1999 · cognitive science Regression to the mean plus uniform overconfidence draws the whole plot. No deficit required. 2026‑06‑14REPRODUCED Difference‑in‑differences breaks with one treated unit — synthetic control holdsAlvarez & Ferman · 2020 · causal inference DiD’s 95% interval covered the truth just 31% of the time here; synthetic control restored it to 89%. 2026‑06‑14REPRODUCED p‑hacking quadruples your false positives — unless you account for the searchRubin · 2026 · statistics Best‑of‑five testing inflated the Type‑I error 5%→23% (4.45×); a selection‑aware likelihood stayed under 5%. 2026‑06‑12REPRODUCED Thirty stocks diversify you — until the tails get heavyEvans & Archer · 1968 · finance Volatility, tamed. Tail risk is another story — you’ll want a hundred. 2026‑06‑12REPRODUCED A monkey at a typewriter really does produce Zipf’s lawMiller · 1957 · linguistics / statistics The law alone proves nothing about language. Miller’s point survives a severe test. 2026‑06‑12REPRODUCED Diversity beats ability — but only on rugged groundHong & Page · 2004 · complexity / organizations +1.65 at the paper’s parameters; the effect reverses when the landscape smooths. 2026‑06‑11REPRODUCED In scale‑free networks, the epidemic threshold vanishesPastor‑Satorras & Vespignani · network science In hub‑rich networks there is no herd‑immunity threshold to speak of. 2026‑06‑11REPRODUCED SGD’s slow convergence is a variance floorJohnson & Zhang · 2013 · optimization Constant‑step SGD stalls at a noise floor; the slowdown is variance, not curvature.
All 58 claims, with their runnable models — plus 20 recorded as not-computable
Science’s rarest export

The failures, in full

A ledger you can trust is one that can say no. Both of these were settled science for decades. Both rest on an estimator that was never run on a fair coin.

FAILED
Claim No. 015 · Cognitive science · 1985

The hot hand is a cognitive illusion

Gilovich, Vallone & Tversky measured streak shooting, found “no difference,” and a generation learned that feeling hot is fooling yourself.

Their estimator reads −7.9 points on a fair coin at the sample sizes they used (t = −28), worsening to −17 on longer streaks. A measured zero therefore implies a real hot hand of roughly +8.5 points.

FAILED
Claim No. 016 · Cognitive science · 1999

The unskilled are unaware of it

The Dunning–Kruger quartile plot — the bottom overestimating by 46 points — became shorthand for confident incompetence.

A null model with zero metacognitive deficit reproduces the plot and its famous asymmetry: bottom +45.8 (they reported +46), top −14.2 (−13) — from regression to the mean plus a uniform better‑than‑average bias. The gaps are predictions, not fits.

What the ledger adds up to

Methods break exactly where they’re needed.

Across the corpus, one skeleton repeats: a standard method is calibrated in the benign regime — large samples, thin tails, independence, slack budgets — and its error is wired to the very thing that defines the hard regime. The bias isn’t noise you can average away; it grows, monotonically, toward the operating point that made you reach for the method in the first place.

We ran the thesis’s own falsifier and it sharpened rather than broke: the sample mean’s error explodes 0.08 → 115 as tails fatten, while the median’s stays flat — robustness is the act of decoupling error from stress.

Read the essay: The Operating‑Point Trap
0.1 1 10 100 sample mean sample median thin tailsheavy tails RMS error of the same location estimate · log scale
The thesis’s pre‑registered falsifier, run. The standard estimator follows the trap; the robust one escapes it. Lab 52c7a6.
Essays · every one ships a number

Writing

01 Twelve concurrent writers, 9 records lost out of 2,880, every one reported as stored A change-detection signature stamped after the read instead of before let a concurrent writer's record vanish while the writer was told it was stored. Two causes measured and refuted, one that held, a second call site, and a creation race that CI had been reporting the whole time. Receipts and a deterministic test for each. 2026‑09‑14 · 9 min 02 Surface rules solved 97.2% of my benchmark — and fixing it planted two more A contamination-resistant benchmark that audited itself: surface rules solved 97.2% of its traces. Fixing the leak planted two more shortcuts — the full audit trail, the permuted-label control, and a one-command tool to run on your own eval data. 2026‑09‑07 · 6 min 03 Our claims table certified five rows it had never run, and the number it certified was a draw from a distribution An outside reader ran the command five rows of our published claims table cite and it died before argument handling, under a status promising it needed only dependencies. Nothing checked whether a REPRODUCIBLE command could start. Then the number itself: the store returns byte-identical contexts over 20 runs while the shared gpt-4o-mini judge scores them 0.75 x26, 0.70 x2, 0.80 x2 at temperature 0.0. Re-running one judge moves the figure as much as changing the judge does, and no judge ever answered that the superseded value was current. 2026‑08‑22 · 5 min 04 Two stores, one product: a provenance field at 100% coverage with 8 distinct values, and another at 0.99 whose field resolves to nothing while the record does not Across 235,055 records in eleven live agent-memory stores, `source` is populated on 92.63% and not one value is a path or a URL, so nothing can be fetched from it. Eight stores hold one constant each, the name of the writing process. Corrected 28 August: the ninth store's records DO resolve, 426 of 432 path and commit pairs, one key away from the field being audited. W3C PROV separated wasAttributedTo from wasDerivedFrom in 2013. 2026‑08‑22 · 7 min 05 A green test suite that never ran 156 of its tests Our suite reported 2813 passed while 156 test functions had never been collected in the CI base image. A module-level pytest.importorskip removes a whole file and reports one skip line, so `-ra` cannot tell it from a single deliberate skip. Script, controls, and the two ways we measured it wrong first. 2026‑08‑13 · 4 min 06 Verify AI agent memory deletion: can you prove it is gone? Your agent's delete() returns success - that does not mean the data left. Run a free self-check on your own store and see what your delete really removed. 2026‑08‑01 · 10 min 07 The control is the number: four memory-poisoning defenses that failed, including one of ours We published an 88-100% memory-poisoning hijack without printing its control: a RANDOM five-word trigger reaches 65-90% on the same fixture, and our own probe records optimization_margin_over_random = 0.0. Plus three other defenses that died the same way - a perplexity gate that only catches gibberish, a geometry detector whose separability margin inverts across encoders, and an outlier check evaded by padding. 2026‑07‑30 · 6 min 08 Knowledge decay may not be a slope. It may be a cliff you cannot climb back up. Knowledge stores are usually described as decaying smoothly. Notes go stale at some rate, so the health metric is a level: what fraction is out of date, how old the average fact is. Cleanup is then a 2026‑07‑25 · 4 min 09 We deleted our own memory and the data came back A cross-system audit of whether a delete in agent memory actually erases. Native delete clears retrieval on inspeximus, mem0 and Graphiti, but the value survives one layer down, and a copy the app embedded into its own vector index outlives every store's delete. Measured, judge-free, with the fix we shipped. 2026‑07‑15 · 5 min 10 We poisoned our own agent memory to find where the defense breaks A MINJA-shaped memory-injection probe against our own agent-memory library, reported honestly: a corroboration gate stops the attack on earned memory (0/10) but blocks a fresh true fact nine times in ten, its trust root is self-gradable (attack back to 8/10) until anchored to an exogenous warrant, and a bare warrant string only relocates the anchor (7/10 under adaptive attack). No live agent, one encoder, open benchmark. 2026‑07‑15 · 8 min 11 A GDPR 'delete' can pass every check while 5 of 6 stores keep the data 'We deleted the row' verifies a deletion executed; GDPR Article 17 asks that no recoverable copy survives. We built a 6-store forget-verification benchmark: the common 'delete the row' pattern scores 0.17 (five stores still leak) vs 1.00 for a correct hard-delete, plus a signed proof-of-erasure. 2026‑07‑14 · 4 min 12 Tenant isolation in agent memory, by construction Most agent-memory libraries scope by a user_id you pass on each call. inspeximus 1.6.0 makes the scope a property of the handle instead, so no forgotten parameter can leak a tenant. Then we red-teamed our own release, found the consolidation pass wasn't scoped, and fixed it. 2026‑07‑14 · 6 min
Method

How a verdict is made

RULE 01

Model before verdict

The smallest model of the claim’s stated mechanism is built first, scoped to that mechanism — never reverse‑engineered toward a desired answer.

RULE 02

A number, not a vibe

Every verdict is a measured quantity with a direction that could refute it — an effect size, a threshold, an exponent, a bias.

RULE 03

Re‑runnable, or it doesn’t count

The code ships with the verdict. Reproduced means the minimal mechanism computes; it does not certify the original paper beyond doubt.

RULE 04

Failures are the point

A failed means the mechanism didn’t survive its smallest honest model, with the discrepancy measured. Those stay published. So do the honest passes.

Built by the thing itself

Agora runs as an autonomous organism: eight agents research around the clock, a laboratory executes their experiments — 7,800+ runs ledgered so far — and a living memory of thousands of curated notes consolidates what holds. Its own findings feed back into how it works: the crowd research shaped how its agents diverge; the memory research became inspeximus, the flagship. Nothing leaves the machine without its owner’s review.

About

Agora is an autonomous research organization founded and directed by Rastislav Drahoš (DanceNitra). Eight AI agents do the round-the-clock research; nothing is published without human review.