Papers
arxiv:2605.05242

Beyond Semantic Similarity: Rethinking Retrieval for Agentic Search via Direct Corpus Interaction

Published on May 3
Β· Submitted by
ZhuofengLi
on May 8
Authors:
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,
,

Abstract

Direct corpus interaction enables more effective agentic search by allowing agents to query raw text directly, outperforming traditional retrieval methods in complex tasks.

AI-generated summary

Modern retrieval systems, whether lexical or semantic, expose a corpus through a fixed similarity interface that compresses access into a single top-k retrieval step before reasoning. This abstraction is efficient, but for agentic search, it becomes a bottleneck: exact lexical constraints, sparse clue conjunctions, local context checks, and multi-step hypothesis refinement are difficult to implement by calling a conventional off-the-shelf retriever, and evidence filtered out early cannot be recovered by stronger downstream reasoning. Agentic tasks further exacerbate this limitation because they require agents to orchestrate multiple steps, including discovering intermediate entities, combining weak clues, and revising the plan after observing partial evidence. To tackle the limitation, we study direct corpus interaction (DCI), where an agent searches the raw corpus directly with general-purpose terminal tools (e.g., grep, file reads, shell commands, lightweight scripts), without any embedding model, vector index, or retrieval API. This approach requires no offline indexing and adapts naturally to evolving local corpora. Across IR benchmarks and end-to-end agentic search tasks, this simple setup substantially outperforms strong sparse, dense, and reranking baselines on several BRIGHT and BEIR datasets, and attains strong accuracy on BrowseComp-Plus and multi-hop QA without relying on any conventional semantic retriever. Our results indicate that as language agents become stronger, retrieval quality depends not only on reasoning ability but also on the resolution of the interface through which the model interacts with the corpus, with which DCI opens a broader interface-design space for agentic search.

Community

Paper submitter

πŸ”₯ The best retriever for agentic search … is no retriever. Introducing Direct Corpus Interaction (DCI).

πŸš€ We replaced the entire agentic search pipeline β€” embedding model, vector index, top-k retrieval β€” with only grep and bash. πŸ”§

πŸ’‘The Magic:
The agent searches the raw corpus directly β€” grep, find, bash, shell pipelines β€” exactly like a coding agent navigating a codebase. No preprocess. No embedding model. No vector index. No offline indexing.

πŸ“ŠThe Results:
DCI outperforms top baselines across 13 benchmarks, with average gains of:
πŸ” Agentic Search (BrowseComp-Plus): +11.0%
🧠 Multi-hop QA: +30.7%
πŸ“ˆ IR Ranking: +21.5%

πŸ’‘ Insights:
Beyond accuracy, we conduct a series of controlled ablation studies to pinpoint the sources of DCI’s gains in Section 4. Specifically, we examine trajectory-level search patterns (RQ2), evidence utilization (RQ3), corpus scale (RQ4), context management (RQ5), and tool usage (RQ6).

Try it yourself!
πŸ‘¨β€πŸ’» GitHub: https://github.com/DCI-Agent/DCI-Agent-Lite
πŸš€ Demo: https://huggingface.co/spaces/DCI-Agent/demo
πŸ”Ž Eval logs: https://huggingface.co/datasets/DCI-Agent/eval-logs

Interesting breakdown of this paper on arXivLens: https://arxivlens.com/PaperView/Details/beyond-semantic-similarity-rethinking-retrieval-for-agentic-search-via-direct-corpus-interaction-2755-8403a410
Covers the executive summary, detailed methodology, and practical applications.

Sign up or log in to comment

Models citing this paper 0

No model linking this paper

Cite arxiv.org/abs/2605.05242 in a model README.md to link it from this page.

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2605.05242 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2605.05242 in a Space README.md to link it from this page.

Collections including this paper 0

No Collection including this paper

Add this paper to a collection to link it from this page.