Researchers at PricewaterhouseCoopers U.S. publish arXiv paper showing grep literal search outperforms vector retrieval across agent harnesses on LongMemEval fact-recovery tasks

There’s an open question on whether grep is all you need for agentic search.

This recent paper by @PwCUS (Sen et al.) seems to suggest that. It’s titled “Is Grep All You Need? How Agent Harnesses Reshape Agentic Search”. They test various agentic harnesses (in-house, Claude Code, Codex), and equip the agent with both vector search and grep.

They find that grep generally yields higher accuracy than semantic search.

IMO the main gap of the paper is that it tests retrieval over conversational memory, not over a real-world corpus of enterprise documents. Standard enterprise RAG setups involve asking complex questions over a static document corpus (e.g. 10-Ks, legal contracts, SOPs). The corpus here is per-user chat history, which is quite a different document distribution.

I do think that evolving agentic harnesses simplify the problem of retrieval - hence the popularity with file sandboxes and a vector db is “just a database” - but IMO there’s still more work to be done here.

Paper: https://arxiv.org/pdf/2605.15184

Is Grep All You Need?

The surprising result is not that grep is powerful, but that agent design makes it powerful.

The paper says not that grep beats vectors, but that agents fail or win through their harness.

That sounds like a small distinction until you look at what was actually tested.

The authors compare grep-style search and vector retrieval across LongMemEval tasks, where agents must recover facts from long conversation histories full of distractors. Inline grep beats inline vector across every harness-model pair in their main experiment, sometimes by wide margins.

The tempting headline is that vector databases are overbuilt for coding agents.

The better reading is sharper: when the answer is anchored in literal evidence, names, dates, file paths, function names, error strings, user preferences, grep gives the model a clean mechanical advantage.

Embeddings are built to tolerate paraphrase, but tolerance has a cost. They can pull in semantically nearby clutter, especially when a short agent query is vague.

Grep has the opposite failure mode. It is dumb, cheap, and narrow, but when the agent knows the right string to hunt for, dumb becomes a feature.

The deeper finding is that retrieval is not a component you can benchmark in isolation. The same search method behaves differently depending on whether results are injected inline, written to files, routed through a CLI, or wrapped in a custom agent loop.

So the question is not “Do we still need vector databases?”

The question is whether your agent is solving a semantic discovery problem or an evidence-location problem.

For coding agents, a surprising amount of work is evidence-location: find the symbol, trace the call, inspect the diff, read the failing test, recover the exact line.

Vectors still matter at scale and for fuzzy conceptual search, but this paper weakens the lazy default that every serious agent stack begins with embeddings.

Sometimes the upgrade is not a smarter index.

Sometimes it is giving the model primitive tools, clean files, disciplined context, and a harness that lets exact search do exact work.

----

Paper Link – arxiv. org/abs/2605.15184

Paper Title: "Is Grep All You Need? How Agent Harnesses Reshape Agentic Search"

Better search may come less from smarter indexes than from giving agents a richer way to touch text.

Shows that AI agents using basic terminal tools like grep, file reads, and shell commands to search raw data perform far better than conventional retrieval systems on multiple benchmarks.

On BrowseComp-Plus, swapping semantic retrieval for terminal search raised accuracy from 69% to 80% while lowering cost.

The deeper point is not that grep is magically smarter than embeddings.

It is that retrieval is usually treated as a model problem, when it is also an interface problem.

A conventional retriever turns the corpus into a narrow ritual: ask once, receive a ranked list, reason over whatever survived.

That works when the question is close to a document’s semantic center, but it breaks when the answer depends on exact phrases, faint clues, document structure, or a chain of small discoveries.

Direct Corpus Interaction changes the shape of the task.

The agent can search an exact string, inspect nearby context, notice a new entity, constrain the search again, and keep testing its hypothesis against the raw files.

Here’s the part most people miss: the gain was not mainly from finding more gold documents, but from extracting more usable evidence once a promising document was reached.

That makes DCI less like a better search engine and more like giving the model fingers.

The limitation is real: as the corpus grows, the cost of finding the first useful anchor rises quickly, and blunt terminal search will not replace indexes for every large, static collection.

But the paper’s lesson still lands cleanly.

For capable agents, the bottleneck may no longer be only what they know, or even how they reason, but how much of the world their tools allow them to touch.

----

Paper Link – arxiv. org/abs/2605.05242

Paper Title: "Beyond Semantic Similarity: Rethinking Retrieval for Agentic Search via Direct Corpus Interaction"