Session Description

Every retrieval system now competes with a simpler-looking alternative: skip the index and let the agent search live over tools with long context. For small or fast-changing corpora that genuinely makes sense. But “just search live” has a cost curve with a very different shape from an index, and most teams pick a side before doing the maths.

This talk makes the maths explicit and reproducible. I’ll walk through recall, latency and memory tradeoffs measured across quantization schemes and dimensionality reduction: how far you can compress an index without trashing recall, and where the refinement step is quietly doing the real work. Then the part most benchmarks skip: where recall stops predicting end-task answer quality. Recall is a proxy, and I’ll show where it breaks.

From there I convert compression into money, against repeated live search and long-context inference over the same corpus. Benchmarks run on public datasets; the methodology and numbers are open.

I’ll ground it in two open-source systems built on exactly this tradeoff: memsearch, an agent-memory layer that indexes plain markdown instead of replaying full transcripts, and claude-context, a semantic code-search tool that avoids loading whole repositories into a model’s context on every query. Both exist because live re-search over large private history or repos gets expensive fast, and both come with measured token and latency deltas.

You’ll leave with a practical framework for the question that actually matters: at what corpus size, update frequency and query volume a compressed index beats live search on cost, and how much answer quality you trade to get there.