agent-memory integrity · a standing leaderboard

Agent-Memory Integrity Leaderboard

Recall leaderboards measure whether memory returns the right fact. None measure whether a CORRECTION sticks. This one does.

We lead with the cell we TIE, and we publish the cells we don't win. A leaderboard you only ever top is one you built to flatter yourself.

Value-obscuring revert

The user says “go back to what we had” and names no value. Can the store undo a correction on that unmarked command?

#	system	success rate (higher is better)	how
1	inspeximus	0.75 CI [0.53, 0.89]	intent router restores the predecessor from the version ledger
2	mem0 2.0.11	0.20 CI [0.08, 0.42]	no revert operation; the corrected value is retained
3	Graphiti	0.00 CI [0.00, 0.16]	no revert operation; bitemporal invalidation fires on named contradictions, not an unnamed “go back”

A capability gap, not a tuning gap: only a system with an explicit revert channel can undo a correction on command. Judge-independent — mem0 and Graphiti structurally lack the operation; the judge model only shifts inspeximus's absolute rate. (Revert is rare, not unique: Letta ships an engine-level checkpoint-undo, not tested here.) It is also broadly underserved: even Anthropic's own agent-memory API (agent-memory-2026-07-22 beta) has no dedicated restore endpoint — to roll back you retrieve a prior version and write its content back by hand — so revert-conditioned recall is an industry-wide gap, not a two-vendor one.

Echo resurrection

After a correction, if the OLD value is re-stated, does the store keep the corrected value or resurrect the stale one?

#	system	resurrection rate (lower is better)	how
—	inspeximus	0.00	echo_guard blocks the restated stale value
—	mem0 2.0.11	0.00	defends
—	Graphiti	0.00	defends

A TIE — all three defend. This is the cell we lead with on purpose.