We deleted our own memory and the data came back

You tell an agent to forget something. It says it forgot. Then you go looking for the thing you deleted, and it is still there.

We ran that test against three shipping memory systems, our own included, and wrote down where the data actually stops being recoverable. The instrument is deliberately dumb: issue each system's own documented delete, then look for the erased value, verbatim, on every surface the system exposes. No LLM judge, no interpretation. Eight subjects, real sensitive values (a medical condition, an address, a salary).

Native delete clears the front door

The good news first, and it is genuine: on all three systems the retrieval surface comes back clean. Ask for the deleted value through the normal query and enumerate calls, and it is gone, 0 of 8 on inspeximus, mem0 and Graphiti alike. If your only threat is "does a query return the deleted fact," every one of these passes.

The front door is clean, the residue is one room over

The value is off the query path, but it has moved to a room most people do not check.

On mem0, the history API hands the erased content straight back. After a delete, history(memory_id) returns the DELETE event with old_memory set to the value you just removed, verbatim, to anyone with store access, on all 8 of 8. The only native way to clear it is a global reset() that wipes everything; there is no per-record or per-user purge, and delete_all() actually adds a DELETE row rather than removing the history. This is a design trade-off, an audit trail that keeps content versus one that records the deletion event without it, not a bug we are dunking on. But an operator answering a right-to-erasure request needs that room in scope.

The raw storage files carry it too, though this one is not mem0's to own. In the default local Qdrant config the payload text sits in the embedded segment files until a vacuum runs, a property of that substrate shared by any app that uses it, not a defect of the memory layer on top (timing-dependent, we saw it 4 to 5 of 8). Graphiti's explicit remove_episode() measured surprisingly clean, 0 of 8 on all four surfaces, with a per-subject check confirming the value was in the graph before each delete; its documented bitemporal retention is about superseded facts, a different operation.

The row where everyone fails, us included

Then there is the copy the application made itself. A real RAG stack embeds your documents into its own vector index, outside the memory store, and that index never hears about the delete. So the value is recoverable there 8 of 8 after every store's native erase, inspeximus's too. We lead with this row on purpose. A store cannot erase infrastructure it was never told about, and pretending otherwise is how "we deleted it" becomes a false statement.

This is a known failure class, and we credit it: Ghost Vectors (arXiv:2606.18497) reconstructs soft-deleted vectors, and the whole machine-unlearning literature's thesis is that delete does not delete. What this audit adds is a standing, re-runnable measurement of it across systems people actually ship, part of our wider agent-memory integrity work and the full audit report.

The fix, and its honest ceiling

Because no store can close that gap by itself, we shipped the wiring as a first-class operation in inspeximus 1.8.0. You register the app's fan-out stores, the vector index, the caches, the logs, as erasure targets, and forget_subject() cascades the delete through all of them, re-checks recoverability, and returns a hash-chained manifest that reads complete only if every store verified the value gone, naming the leaks otherwise. Measured: the unwired index leaks 8 of 8, wired it erases 0 of 8 for the targets you register, and a deliberately broken wiring cannot produce a clean receipt.

The ceiling is honest and worth stating. The manifest covers only the targets you register; an unknown copy stays unknown, for every system. It attests that content is no longer recoverable at check time, not that a disk was physically wiped, and it does not touch backups or the reconstruction of text from a retained embedding. GDPR Article 17(3) even permits retention for audit and legal purposes, so this is an engineering signal about where erased content stays readable, not a compliance verdict.

FAQ

Does deleting a memory from an agent actually erase it? On the query path, yes: retrieval returns the deleted value 0 of 8 across inspeximus, mem0 and Graphiti. But the value can survive one layer down, in a history/audit API, in raw storage files before compaction, and in any external vector index the application built itself. Test recoverability across the fan-out, not just whether the API returned success.

Why does deleting from a vector store not remove the data? Vector databases like Qdrant soft-delete by default and only reclaim the bytes on a vacuum or compaction pass, so the payload text sits in segment files until then. Separately, a retained embedding can be inverted back toward the original text (Ghost Vectors, arXiv:2606.18497), so deleting the row while keeping the vector does not erase the content.

How do I verify data erasure in agent memory for a GDPR right-to-erasure request? Adversarially re-attempt recovery of the erased value from every store the data fanned into, not just confirm the delete call succeeded. inspeximus 1.8.0 ships this: register the app's fan-out stores and forget_subject() cascades the delete, re-checks recoverability, and returns a hash-chained manifest that reads complete only if every registered store verified the value gone.

Does mem0 keep deleted data? In mem0 OSS 2.0.x the history API returns the deleted content (old_memory) verbatim after a delete, and the only native clear is a global reset(). This is a design trade-off, an audit trail that keeps content, not a defect, but an operator answering an erasure request needs that surface in scope.

The short version: a delete that clears the query is table stakes. The question worth testing on your own stack is whether the deleted value survives one room over, and whether anything you ship can tell you when it does. The harness is open; point it at your store and contradict us.