Your RAG store is rotting — and the fix is what you keep, not better retrieval

The claim. Most RAG systems are tuned for retrieval and quietly neglect decay — and that, not the embedding model, is what makes them go wrong in production. A vector store that keeps every chunk forever surfaces last quarter's pricing, serves a deleted policy, and grows a bill that climbs on unchanged data. But the fix isn't a better retrieval model either: it's what you keep. Rank what to keep by value, and you retain almost all of the value that matters; clean by recency alone and you keep barely more than half.

What we measured. We built the smallest honest model of the decision and ran a fair head-to-head on a 1,000-chunk store with a known true value, at a tight 50% keep-budget (every strategy keeps exactly 500). Scores are the % of the keep-best-by-true-value oracle, averaged over 20 seeds (sd ~1–2%). The primary regime is realistic: content age tracks value (fresher content tends to be more valuable).

The gap is value-awareness, not freshness. A value-aware keep-policy retains roughly 1.5–1.8× as much of the value that matters as a recency-only cleanup (i.e. ~50–80% more). And notice freshness barely helps the keep ranking: value-only (100%) is already at the ceiling, and adding a freshness term (95%) is neutral-to-slightly-negative. In the worst-case regime, where content age is independent of value, recency-only collapses to 56% — statistically tied with random — while value-only stays 100% and the hits-proxy holds ~90%.

Disclose the oracle. The ~95–100% "with labels" numbers assume you can score a chunk's true value. In production you usually can't. So the realistic, observable number is the hits-proxy at ~90–91% — a decayed access-count stand-in, almost as good as the value-only optimum. We were wrong in an earlier version of this post when we called access-frequency "the wrong signal": the benchmark refutes that. When you have no value labels, a decayed hit-count is a strong proxy — this is exactly LFU-with-aging / LFUDA. The honest caveat: hit-count tracks value only when popularity correlates with value (often true, not always).

Where freshness actually earns its keep. Not in the keep-ranking — in two other places. (a) The query-time staleness multiplier: rank fresh chunks above stale ones at retrieval, without deleting anything. (b) Lifecycle: flag orphaned vectors (source deleted) for removal and stale-but-valuable chunks for refresh — failure modes that pure retrieval tuning never catches. Treat freshness as the retrieval-time and lifecycle layer, not as what wins the keep-benchmark.

A second, separate result: schedule the cleanup. Run it as a periodic batch, not a per-write hook. In a separate runnable lab, continuous per-write pruning buys only a ~8% retrieval-quality (signal-vs-clutter) edge but pays ~25× more pruning events, so once a pruning event has any real cost the scheduled pass wins on net (analytic crossover ≈ 0.07 overhead/event). This is measured apart from the keep-ranking benchmark.

FalsifierIf a recency-only policy retained as much true value as a value-aware policy at a tight budget, ranking by value would be pointless. It doesn't: 62% (and 56% in the worst case) vs 100% at a 50% budget. And if continuous per-write pruning beat the periodic batch on net once pruning has overhead, the "schedule it" rule would be wrong — measured, it loses.

Not a new method. The individual ideas are established: GDSF cost-aware caching, LFU-aging for the frequency-vs-value failure, and temporal-RAG / staleness decay. What we add is the integrated zero-dependency tool plus the measured comparison. See the prior art below.

The tool. We packaged this as ragfresh — a single zero-dependency file that takes your store's chunk metadata (when it was updated, last accessed, hit count, whether the source still exists) and returns a per-chunk plan: keep, down-weight, refresh, or prune — plus a query-time staleness multiplier so fresh chunks rank above stale ones without deleting anything. It's the decay/consolidation core that runs our own autonomous research memory over ~5,800 notes, repointed at vector stores. Open-core; the core stays free. The keep-ranking benchmark above (the value / recency / hits arms) reproduces with one command (python ragfresh.py); the batch-vs-hook number is a separate lab. Check the numbers yourself — which is the point.

FAQ

Why does my RAG system degrade in production? Usually decay, not the embedding model. A vector store that keeps every chunk forever surfaces last quarter's pricing, serves a deleted policy, and grows a bill that climbs on unchanged data — orphaned vectors, stale answers, cost bloat.

Is the lever freshness, or a better retrieval model? Neither — it's value-awareness in what you keep. Ranking what to keep by value retains ~100% of the achievable value at a 50% budget; a value-blind recency-only cleanup retains only ~56–62% (close to random when content-age doesn't track value). That's a keep/eviction comparison, not a claim about retrieval or embedding quality. Freshness itself barely changes the keep-ranking (value-only ~100% vs the value+freshness blend ~95%).

Isn't access-frequency the wrong signal? No — that was wrong and we corrected it. A decayed hit-count proxy retains ~90–91% (raw hit-count ~92%), close to the value-only optimum, so when you have no value labels it's a strong stand-in (this is LFU-with-aging / LFUDA). The caveat: hit-count tracks value only when popularity correlates with value — usually true, not always.

What should I actually change? Rank what to keep by value (or a decayed hit-count proxy if you have no value labels) instead of keeping everything or cleaning by recency alone. Use freshness as a query-time staleness multiplier and to flag orphaned vectors (source deleted) and stale-but-valuable chunks for refresh. Note: the ~90–100% with-labels numbers assume you can score chunk value; in production the realistic, observable number is the hit-count proxy ~90–91%.

Related research

• Does long context kill RAG? We measured it
• What should an AI agent forget? Two tiers beat every single rule
• Multi-hop recall on LoCoMo: put the model in the retrieval loop

Prior art

ragfresh's value+freshness eviction is the GreedyDual-Size-Frequency (GDSF) / cost-aware caching family; the "frequency ≠ value" failure is the textbook LFU cache-pollution problem, and the fix is LFU-Aging / LFUDA. See Cao & Irani (1997, GreedyDual-Size, USENIX); Cherkasova (1998, GDSF, HP Labs); Young (2002, GreedyDual); Arlitt et al. (2000, LFU-DA, "Evaluating content management techniques for web proxy caches"). For the freshness/decay layer, see the temporal-RAG / staleness literature — FreshLLMs / FreshQA (Vu et al. 2023, arXiv:2310.03214). ragfresh is a packaged, runnable application of cost-aware (GDSF) eviction + temporal decay to vector stores — not a new method. The individual ideas are established (GDSF cost-aware caching; LFU-aging; temporal-RAG); what we add is the integrated zero-dependency tool plus the measured comparison.