arXiv CS open access

Beyond Top-K: Replacing Black-Box Retrieval with Interpretable Agentic Operations

2026 · arxiv_cs

arXiv CS · Papers · 2026

artificial-intelligence knowledge-representation reasoning

Conceptio Open Archive

Analyze & explore related research in Conceptio

Connect this record directly into your research workflow or AI agent. Explore citations across open repositories, query full text, and export structured proof bundles.

Explore in Search →Connect via MCP / API ↗

Beyond Top-K: Replacing Black-Box Retrieval with Interpretable Agentic Operations

Sagar Tamang * 1 2 Ayush Vyas 2 Tabarakul Hazarika 2

15.7 chunk financial report

arXiv:2608.06305v1 [cs.AI] 6 Aug 2026

780-page financial report 86.8% table rows

embed the chunks

select top-k

Conventional RAG the unit header is on the other side of the boundary.

58.8

grep normalized line search

outline headings + line numbers

read bounded span, cited lines

agent chooses what to read next

READ (ours)

Figure 1. Two retrieval interfaces over the same document. The conventional pipeline commits to a partition before the query is known, so a figure can arrive without the header that states its unit and fiscal year — on this document that separation survives every chunk size we tried (Section 4). R EAD leaves the document intact and lets the agent choose what to read next, so each step is a citable line range rather than a similarity score. Figures at right are accuracy, in percent correct, over the same 51 verified questions: 15.7% for dense top-k against 58.8% for R EAD (Section 8). Tuning the dense baseline’s chunk size and retrieval depth raises it to 35.3% at best, which R EAD still leads by 23.5 points (Section D).

Abstract

ker built as a steelman fixes the unit problem but leaves 27–30% of numeric chunks with no fiscalyear header at every chunk size we tried. We propose R EAD (Reliable Embedding-free Agentic Document-search), in which an agent reads the raw document through three deterministic operations — normalized lexical search, structural navigation, and bounded span reads — exposed over the Model Context Protocol, so a trajectory is a replayable audit trail, not an opaque similarity score. On 51 verified questions R EAD answers 58.8% against dense retrieval’s 15.7% (pHolm = 2 × 10−5 ) — or 35.3% tuned, which R EAD still leads by 23.5 points (pHolm = 0.017). An agent given the same loop but a top-k tool reaches only 27.5%, locating the gain in the interface rather than in iteration. We also report what the evidence does not support: BM25 is statistically indistinguishable from R EAD, so our result separates embedding-based from embedding-free retrieval, not agentic from lexical search.

Retrieval-augmented generation over long documents is dominated by one design: chunk the text, embed the chunks, and surface the top-k nearest neighbours of the query. We argue that for an important class of documents — financial statements, audit reports, regulatory returns — this design is structurally unsound, and we make the argument measurable. On a 780-page government financial report, 86.8% of content lines are table rows, thousands of near-identical figures compete in one embedding space, and a figure inherits its unit from a header a median of 13 lines above it — so a chunk boundary routinely separates a number from whether it is in lakh or crore, an error of two orders of magnitude. A table-aware chun1

Indian Institute of Technology Patna, Bihta, India TwoSpoon, India. Correspondence to: Sagar Tamang <sagar pa2508mch184@iitp.ac.in>. 2

Preprint. August 7, 2026.

Beyond Top-K: Interpretable Agentic Operations for Document Retrieval

1. Introduction

including a unit-inheritance analysis we have not seen reported elsewhere (Section 4).

Retrieval-augmented generation (RAG) is the default recipe for grounding language models in external knowledge (Lewis et al., 2020; Gao et al., 2024). A corpus is split into chunks, each is mapped to a dense vector (Karpukhin et al., 2020), and at query time the top-k nearest neighbours are concatenated into the prompt. A decade of work has refined every stage — chunking policies, embedding models, sparse–dense fusion, reranking (Wang et al., 2024; Cormack et al., 2009; Khattab & Zaharia, 2020) — but the contract has not changed: the model sees a small, similarity-ranked slice of the corpus, selected by a mechanism it cannot inspect, steer, or verify.

• A steelmanned baseline, and the residue it cannot fix. Table-aware chunking cuts unitless chunks from 18.0% to 0.3%, but 28.9% of numeric chunks still carry no fiscalyear header (Table 2). We report the strong configuration throughout. • R EAD, an embedding-free retrieval interface, released as an MCP server, together with the finding that normalized lexical matching is what makes it viable on converted PDFs — and a measurement of exactly which conversion artifacts make it so (Section 5.2).

This paper is about a class of documents for which that contract fails, and about why the failure is structural rather than a matter of tuning. The class is large: financial statements, audit reports, statutory filings, and regulatory returns, which share three properties — they are overwhelmingly tabular, they contain thousands of near-duplicate numbers, and they carry meaning in layout, where a figure’s units and fiscal year are declared in a header some distance above it.

• A conversion-fidelity ceiling. We quantify the damage the PDF converter inflicts (Table 4) and show it is a bound shared by every retrieval method, requiring its own question category rather than being attributed to a retriever (Section 5.3). • An evaluation protocol with three independent grading signals, including a mechanical groundedness check that a fluent-but-unsupported answer cannot pass (Section 6.3).

Consider the question that motivated this work, asked of the Gujarat Finance Accounts 2024–25, Volume I: what is the difference between the Fiscal Deficit and the Revenue Surplus for 2024–25? The two operands sit 47 lines apart in the same statement, and their difference appears nowhere in the document — it must be computed. In a pilot run (Section 7), a tuned dense retriever failed to place both operands in the same top-k context on any trial, and the generator correctly reported that it could not answer. This is not a hallucination; it is retrieval declining to deliver, and it is the best case. The worse case is a retriever that surfaces one plausible figure and a fluent generator that uses it.

2. Related Work 2.1. RAG and Its Moving Parts RAG couples a retriever with a generator so outputs are conditioned on retrieved evidence (Lewis et al., 2020). Dense passage retrieval established learned dual encoders as the dominant retriever (Karpukhin et al., 2020), refined by late-interaction models (Khattab & Zaharia, 2020), learned sparse expansion (Formal et al., 2021), and unsupervised contrastive encoders (Izacard et al., 2022). The BEIR benchmark showed that BM25 (Robertson et al., 1995; Thakur et al., 2021) remains a strong zero-shot baseline against far heavier neural retrievers — an early signal that lexical matching is harder to beat than assumed.

We make three moves. First, we measure the setting rather than assert it: Section 4 quantifies tabularity, numeric nearduplication, and the distance between a figure and its governing unit header, and then measures how much of that context chunking destroys. Second, we build the strongest chunker we can — one that carries unit and fiscal-year headers across every boundary — so that what remains is a property of the paradigm rather than of a lazy implementation. Third, we describe R EAD — Reliable Embedding-free Agentic Document-search — an interface in which a toolcalling agent reads the document directly, and we compare it against dense, sparse, and hybrid baselines under a paired protocol.

In deployment the retriever is one of many coupled components. Wang et al. (2024) catalogue the design space and find end-to-end quality highly sensitive to each choice. Two fragilities matter here. Position effects: models under-use evidence in the middle of long contexts (Liu et al., 2024), so a retriever that surfaces the right chunk may still not deliver it usefully. Behavioural drift: the services powering embedding and generation change opaquely over time (Chen et al., 2024), so an index built against one model version silently degrades against another. Adaptive schemes such as Self-RAG teach the model when to retrieve and to critique what it retrieved (Asai et al., 2023), but keep the index and its maintenance burden.

Our contributions: • A measured characterisation of exactness-critical documents. We quantify the properties that defeat chunkand-embed retrieval on a real 780-page financial report, 2

Beyond Top-K: Interpretable Agentic Operations for Document Retrieval

2.2. Long and Structured Documents

and introduce retrieval interface resolution as the explanatory lens. Practitioner accounts report the same pattern (Yu, 2025).

Long-document QA exposes the weakest link of chunkbased RAG: the chunker. Financial statements encode meaning in layout — row/column adjacency, multi-page tables, header inheritance — which fixed-size chunking destroys. Work on spreadsheets illustrates the escalating machinery required to compensate: SpreadsheetLLM compresses sheets to fit context windows (Dong et al., 2024), while FRTR builds multi-granular row, column, and block embeddings fused by reciprocal rank fusion to reach usable accuracy on enterprise workbooks (Gulati et al., 2026). These systems show that structure-aware retrieval can be built, at the price of ever more elaborate corpus-specific indexing. We take the opposite route: leave the document intact and give the agent operations expressive enough to exploit its structure directly.

We differ in four ways. Domain. Prior comparisons target multi-document corpora — web snapshots, conversation logs, repositories — where the task is finding the right file. We target a single, very long, densely tabular document, where the task is navigating structure within one file. Task. Our benchmark is exactness-critical: answers are specific figures whose correctness is binary, and several must be computed rather than retrieved. Corpus condition. Prior work greps clean text. We grep the output of a PDF converter, and show in Section 5.2 that naive lexical matching is materially degraded by conversion artifacts — a failure mode that does not arise on clean corpora and that, to our knowledge, has not been reported. Framing. Beyond accuracy and cost we foreground interpretability: because every operation is deterministic, the trajectory is an audit log, which is a requirement in the financial-reporting settings we study.

An alternative response is to dispense with retrieval and rely on long context. Our document is 1.68 MB and fits a 1M-token window, so this is feasible here; but it remains costly, exhibits positional degradation (Liu et al., 2024), and does not scale to corpora that keep growing. We include it as a baseline rather than dismissing it.

3. Two Retrieval Interfaces We frame the comparison as one between two interfaces to the same document, consumed by the same backbone. Let D be a document rendered to text as an ordered sequence of lines D = (ℓ1 , . . . , ℓL ) with a structural map S(D) and a unit map u : {1..L} → U giving the unit in force at each line.

2.3. Agentic Retrieval and Tool Use A separate line makes retrieval iterative. Multi-hop systems learn to retrieve reasoning paths rather than isolated passages (Asai et al., 2020; Trivedi et al., 2023), and ReActstyle agents interleave reasoning with search (Yao et al., 2023; Schick et al., 2023). The competence of models at operating software (Chen et al., 2021) underwrites a stronger version: SWE-agent showed a bash-equipped agent navigates million-line repositories (Yang et al., 2024), and Agentless showed a fixed grep-and-read pipeline rivals full agent frameworks (Xia et al., 2024). Memory-augmented conversational agents such as Chronos likewise replace opaque recall with structured, searchable stores (Sen et al., 2026b). The Model Context Protocol (Anthropic, 2024) standardises how such tools are exposed. R EAD is an MCP server whose tools target a single long document rather than a repository.

Interface A: embedding-mediated top-k. An offline pipeline partitions D into chunks {cj }M j=1 , embeds each as vj = E(cj ) ∈ Rd , and returns Rk (q) = arg top-k j∈[M ]

E(q) · vj . ∥E(q)∥ ∥vj ∥

(1)

Three properties follow. Lossy projection: all evidence available to selection is compressed into d floats, so distinctions below the resolution of E are unrecoverable at selection time. Fixed budget: exactly k chunks are surfaced whether the question needs one cell or forty pages. Opacity: the only account of why cj was chosen is a scalar.

2.4. Direct Corpus Access versus Dense Retrieval

A fourth property is specific to structured documents and is the one we quantify. Chunking is a partition of D, but u is defined by inheritance from preceding lines. A chunk cj spanning lines [a, b] is unit-preserving only if some line in [a, b] declares u(b). Nothing in Equation (1) enforces this, and Section 4 shows how often it fails.

Closest to this work are two recent comparisons. Sen et al. (2026a) evaluate grep against vector search across custom and provider-native agent harnesses on conversationalmemory QA, finding inline grep beats vector retrieval for every harness–model pair tested, and that the harness shifts accuracy as much as the retriever does. Li et al. (2026) formalise direct corpus interaction — replacing the retrieval API with terminal primitives over the raw corpus — improving accuracy on BrowseComp-Plus while reducing cost,

Interface B: interpretable agentic operations. exposes D through deterministic operations:

R EAD

• SEARCH(p) → {(i, ℓi , loc(i))}: all lines matching pat3

Beyond Top-K: Interpretable Agentic Operations for Document Retrieval

tern p under a normalization ν (Section 5.2), with line number and structural location;

Property Printed pages Pages with recoverable folio Lines (non-empty) Characters Statement / section headers Table rows Table-row fraction Table columns (p50 / p90) Numeric tokens Distinct numeric tokens Numeric repetition ratio

• OUTLINE() → S(D): statements and sections with their line and page extents; • READ(a, b) → (ℓa , . . . , ℓb ): a bounded contiguous span addressed by location. The agent composes these in a closed loop: hypothesise a lexical anchor, search, inspect hits against the outline, read a span wide enough to capture the governing headers, and either extract the answer or refine. A trajectory is a sequence τ = (o1 , r1 , . . . , oT , rT ) terminating in an answer with cited locations.

Value 780 745 19,198 1,683,730 189 16,664 86.8% 7 / 10 58,791 15,960 3.68×

Table 1. Corpus profile. The document is overwhelmingly tabular, and its numbers are heavily repeated: 58,791 numeric tokens take only 15,960 distinct values.

Crucially, READ is span-addressed, not chunk-addressed: the agent chooses [a, b] after seeing where its evidence lies, so it can extend upward to include a unit header. Interface A cannot, because the partition was fixed offline.

table rows, with a median of 7 columns. Retrieval methods tuned on prose benchmarks are being applied here to a fundamentally different object.

Determinism and auditability. Every ot is a pure function of (D, args), so re-executing τ reproduces the same results and any third party can replay the evidence chain. We call an answer grounded if each figure it contains appears verbatim in some span surfaced in τ ; this is mechanically checkable (Section 6.3). Under Interface A the analogous check cannot localise: one can verify the answer string occurs in the retrieved chunks, but not interrogate why competing chunks were excluded.

Its numbers are near-duplicates. 58,791 numeric tokens collapse to 15,960 distinct values, a repetition ratio of 3.68. Thousands of similar magnitudes, in similar row contexts, compete in a single embedding space. The two operands of our motivating question — Rs. 48,964.89 crore and Rs. 18,942.94 crore — sit in adjacent rows of the same statement. Its units are inherited, not local. This is the property we have not seen reported, and it is the most consequential. The book declares units in headers of the form “(Rs. in lakh)”: 666 lines declare lakh and 50 declare crore. A figure inherits the unit of the nearest declaration above it, at a distance of median 13 lines, p90 26 lines, max 143 lines (Figure 2). Lakh and crore differ by a factor of 100. A retrieval unit that does not contain the governing header therefore yields figures that are ambiguous by two orders of magnitude — and, unlike a missing answer, this failure is invisible to the generator, which sees a plausible number.

4. The Setting, Measured 4.1. Testbed Our document is the Gujarat Finance Accounts 2024–25, Volume I (Comptroller and Auditor General of India, 2025), the audited annual financial statement of an Indian state government — a public record, 780 printed pages, 3.97 MB of PDF. It is representative of the class we target: statutory, tabular, exactness-critical, and consumed by people who must be able to check where a number came from.

4.3. What Chunking Destroys

We convert to Markdown with pymupdf4llm, selected over three alternatives in a prior comparison for one reason: it is the only converter tested that keeps a row label bound to its figures across columns. The alternatives fail in ways that make any retrieval comparison meaningless — one collapses thirty table rows into a single cell, another severs row labels from their numbers entirely. Section A records the comparison.

We make the consequence precise. For a chunk c containing at least one numeric table row, define c unitless if no line in c declares a unit, and yearless if no line matches a fiscal-year column header. Table 2 reports both under two chunkers at matched size. Three readings matter.

Table 1 reports the profile. Three quantities carry the argument.

First, naive chunking degrades catastrophically at small chunk sizes: at 800 characters, 67.4% of numeric chunks carry no unit declaration. Smaller chunks are usually recommended for precise retrieval; here they are precisely what destroys the context that makes a figure interpretable.

It is a table, not prose. 86.8% of non-empty lines are

Second, the fix works, and works cheaply. Our table-aware

4.2. What the Document Is

Beyond Top-K: Interpretable Agentic Operations for Document Retrieval

Numeric chunks affected (%)

Numeric table rows

No unit declaration

median = 13 p90 = 26

1200 1000 800 600 200 0

800 1,500 2,000 2,500 4,000

30 20 14.7

Fixed-size Table-aware

Lines below the governing unit header

Fixed-size

30.1

36.4

28.7

45.7 28.9

43.4 28.3

40.5 27.2

1,500

2,000

2,500 3 × 103

4,000

18.1

1.0

1,500

0.3

11.6 0.2

0.4 5.4

2,000

2,500 3 × 103

4,000

Chunk size (characters)

800

Chunk size (characters)

Figure 3. The same data. Fixed-size chunking (squares) trades one failure for another as size grows: fewer unitless chunks, but larger retrieval units. Table-aware chunking (circles) removes the unit problem entirely and leaves the year problem untouched at every size.

Figure 2. Distance from a numeric table row to the header declaring its unit. A chunk shorter than this distance cannot contain both, so the figures it carries are ambiguous between lakh and crore — a factor of 100.

Chunk

800

400

No fiscal-year header

table row can exceed 3,000 characters, and splitting it would sever the row. We report p95 chunk length, which tracks the target closely, rather than the maximum, which does not.

Table-aware

Unitless

Yearless

Unitless

Yearless

5. R EAD

67.4% 36.3% 18.0% 11.6% 5.4%

73.8% 55.8% 45.9% 43.5% 40.5%

14.7% 1.0% 0.3% 0.2% 0.4%

30.1% 28.7% 28.9% 28.3% 27.2%

5.1. Implementation R EAD — Reliable Embedding-free Agentic Documentsearch — is a read-only MCP server (Anthropic, 2024) over a directory of Markdown. It exposes the three operation classes of Section 3 as four tools: grep (normalized line search); list (files with sizes) and outline (headings with line numbers), which together provide structural navigation; and read (numbered lines, capped at 400 per call). Each is jailed to a root directory and hard-capped so no single call can flood the context. There is no embedding model, no vector index, and no learned component; the server’s entire state is the document text.

Table 2. Context destroyed by chunking, over a 5× range of chunk size. Fixed-size chunking degrades sharply as chunks shrink. Our table-aware chunker carries the governing unit and fiscal-year rows across every boundary and nearly eliminates unitless chunks — but the yearless residue is invariant to chunk size, spanning only 2.9 points across the whole sweep.

chunker searches backwards up to 200 lines for the governing unit and year rows and prepends them, subject to a budget so a carried header cannot consume the chunk. At 2,000 characters it reduces unitless chunks from 18.0% to 0.3%, a factor of 60.

5.2. Why Normalized Matching A literal grep over converted PDF text is materially weaker than over clean text, and this appears not to have been reported in prior agentic-search work, which evaluates on already-clean corpora. Our converter introduces: 4,685 rows with a spurious leading empty column; 32,404 emphasis markers welded to tokens; 1,719 words split across table cells; 1,032 repeated page headers; and 24 distinct spellings of statement numbers across 654 mentions. Most consequentially, 14,269 numbers are printed with digit grouping.

Third — and this is the load-bearing observation — the yearless residue does not respond to tuning. Across a five-fold range of chunk size it moves between 27.2% and 30.1%, a spread of 2.9 points, with no trend that tuning could exploit. Roughly 28% of numeric chunks cannot be attributed to a fiscal year no matter how the partition is drawn, because in this document the year is declared once for a table that runs across many pages. A partition fixed before the query is known has no way to recover it. An agent that reads a span after seeing where its evidence lies does.

R EAD’s grep therefore matches against two normalized shadows of each line — one with emphasis and cell boundaries replaced by spaces, one with all separators and thousands-commas removed — while reporting original line numbers, so results chain into read.

We use the table-aware chunker for every retrieval baseline in this paper. A baseline that fails for a fixable reason proves nothing, and we would rather argue against the strongest version of the paradigm.

We measured which artifacts actually matter rather than assuming, probing twelve realistic queries under normalized and literal matching. Normalization changes the outcome for exactly two:

Remark 4.1. Chunk size is a soft target: a single converted 5

Beyond Top-K: Interpretable Agentic Operations for Document Retrieval Query

Kind

Norm.

Literal

48964.89 Statement Subject

figure, ungrouped split across cells

✓ ✓

✗ ✗

48,964.89 Fiscal Deficit fiscal deficit Statement No. 17 Co-operation

figure, as printed inside **. . . ** lowercase statement number row label

✓ ✓ ✓ ✓ ✓

Table 3. Matching sensitivity (7 of 12 probes shown). Normalization is load-bearing for digit grouping and for words split across table cells. It is not needed for case, for emphasis wrapping a whole phrase, or for statement-number punctuation — we had assumed the latter mattered and it does not.

The first row is the important one. On this document the agent’s queries are figures — reverse lookups, cross-checks, verification of a computed value — and an agent that has derived or recalled a number writes it without separators. Digit grouping is precisely what breaks that search, and 14,269 of the document’s numbers carry it. We report this narrower mechanism rather than the broader one we first assumed, and we retain read naive grep as an ablation that isolates it.

Class Suspect bare integer (missing decimal) Orphaned separator cell Word split across cells Detached separator (owner identified) Decimal detached from its digits

174 139 133 22 15

Total instances Table rows affected

483 440 (2.6%)

Numeric rows affected Numeric rows with a magnitude error

381 (3.0%) 11 (0.09%)

Table 4. Conversion damage in the corpus. The book’s nil convention (“..”, 17,734 cells) is excluded — counting it as damage overstates the total by roughly 40×, an error we made on a first pass. Damage is rare but concentrated: the 15 detached decimals sit almost entirely in two statements.

5.3. The Conversion Ceiling Every retrieval comparison on a converted PDF silently assumes the text is a faithful rendering of the document. It is not, and the gap is neither uniform noise nor negligible: it is structured, it is measurable, and where it bites it changes a figure by two orders of magnitude. The clearest case is Statement No. 18, which lists loans sanctioned in perpetuity. The printed interest rates are 11.00 and 10.00 per cent. The conversion places the digits in one table cell and the decimal point in the cell below: 16458: 16459: 16460: 16461:

Count

It is a ceiling, not a confound. No retriever — dense, sparse, or agentic — can recover a decimal point that is in a different cell from its digits. Accuracy measured on such a question is measuring the converter, not the retrieval interface. We therefore give these questions their own category (conversion limited) rather than charging their failures to whichever system was under test, and we report them separately.

...|500.00|**1100**| ...|471.50|**.**| ...|500.00|**1000**| ...|500.00|**.**|

Table 4 quantifies this across the document. Three points follow.

It reframes what a retrieval failure means. In an informal evaluation of this document, a system was judged to have failed on the Statement No. 18 question. Inspection showed it had retrieved exactly the right lines and read them faithfully; the source text was wrong. Distinguishing “did not find the evidence” from “found evidence that the conversion had already destroyed” requires the trajectory, which is available for R EAD and for top-k systems alike but not for a system that reports only an answer.

It is rare but concentrated. Only 3.0% of numeric rows are affected and 0.09% carry an outright magnitude error, so this is not a pervasive degradation. But the affected rows cluster: a question that happens to target Statement No. 18 is far more likely to be unanswerable-as-converted than the aggregate rate suggests.

A second observation from the same evaluation is worth recording because it is not a failure. Statement No. 18 is denominated in lakh while most aggregate statements are in crore, so the same amount is correctly reported as either Rs. 10,597.61 lakh or Rs. 105.98 crore. A grader that does not normalise units scores one of these wrong.

An agent that retrieves these lines perfectly still reads 1100 where the document prints 11.00. The same pattern appears in Statement No. 2, where the Consolidated Fund deficit Rs. 26,837.65 crore is split into the digit run 2683765 on one line and the separators ,. on the next, so the wellformed figure occurs nowhere in the corpus.

Beyond Top-K: Interpretable Agentic Operations for Document Retrieval System

Description

R EAD R EADlit R EAD−out

Agent + list/outline/grep/read Same, literal grep (ablation) grep + read only (ablation)

Dense BM25 Hybrid

Chunk → embed → top-k Sparse top-k RRF(dense, BM25) + encoder

AgenticVec Oracle LongCtx

Agent + vector-search tool Chunks overlapping gold lines Whole document in context

pear anywhere in the corpus — otherwise the question is a lookup in disguise. No model generates or grades a gold answer. The unanswerable probes include false-premise items, in which the figures exist but not the quantity the question presupposes — for instance asking for a rate of interest from a statement that reports only interest amounts. These detect a failure mode accuracy alone cannot: a system that accepts the premise and fabricates a matching citation. We have no clean instance of that failure in our data; the one candidate in the pilot turned out to be an artifact of our own analysis code (Section 7).

cross-

Table 5. Systems under comparison. R EADlit isolates normalized matching; AgenticVec isolates interface from iteration.

6.3. Metrics Our numeric matcher normalises lakh, crore, and thousand before comparison for exactly this reason (Section 6.3).

Three independent signals, because no one of them is trustworthy alone. Exact-figure accuracy. Deterministic numeric comparison after normalizing Indian digit grouping, parenthesised and (-) negatives, and lakh/crore units, at 0.5% relative tolerance — enough to equate 48,964.89 and 48,964.9 while separating 48,964 from 18,942.

6. Evaluation Protocol 6.1. Systems All systems share one backbone, one answer format, and one accounting of tokens, cost, and latency, so a difference between them is a difference of interface.

LLM judge. A grader with the reference answer in hand, held fixed across systems, emitting a structured verdict. We report agreement with the deterministic signal; disagreements are read by hand.

Two entries carry the argument. R EADlit isolates normalized matching (Section 5.2). AgenticVec is the critical control: an agent with the same loop and budget but a vector-search tool instead of R EAD’s operations. If R EAD’s advantage came merely from being able to iterate, AgenticVec would close the gap; if it comes from the interface, it will not. Without this control, any result is attributable to iteration alone.

Groundedness. Does every figure in the answer appear in text the system actually retrieved or read? This is purely mechanical, and it is the metric a fluent-but-unsupported answer cannot pass. It is computable for R EAD and for top-k systems alike, since both expose exactly what they surfaced.

Retrieval baselines use the table-aware chunker (Table 2) and a local bge-base-en-v1.5 encoder. Embeddings run locally so the baseline does not drift when a hosted embedding endpoint updates (Chen et al., 2024); Section D reports the chunk-size and k sweep, and we report the best dense configuration found rather than the first.

We additionally report hallucination rate on unanswerable probes, page-level evidence coverage, tool-call counts, tokens, cost, and latency. Comparisons are paired: exact McNemar on discordant pairs, bootstrap confidence intervals on the delta, Holm-corrected across the family of baselines. We report the benchmark’s minimum detectable difference so that a null result is reported as underpowered rather than as no difference.

6.2. Benchmark Questions span six categories: single-figure lookup, multirow aggregation, cross-statement arithmetic, structural navigation, unanswerable probes, and conversion-limited items whose answer the converter destroyed (Section 5.3). Gold answers are transcribed from the document by the authors and then checked mechanically: every expected figure must appear within one line of its declared gold line, and the column mapping of each source table is verified against an arithmetic identity the table itself satisfies rather than against our reading of its header (Section C). Arithmetic questions carry both the derived answer and its operands, and a validator asserts that the derived value does not ap-

7. Pilot Observations Before the full evaluation we ran a two-question pilot to exercise the harness. We report it because it makes the mechanism concrete, and we state plainly that n=2 supports no quantitative claim. The dense baseline abstained on every cell, reporting that the Fiscal Deficit was not present in its retrieved excerpts. The two operands lie on lines 385 and 432 — 47 lines apart, in the same statement — and no top-8 slice of 2,000-character 7

Beyond Top-K: Interpretable Agentic Operations for Document Retrieval Backbone

System

Turns

In-tok

Outcome

2.5-pro

R EAD Dense

4 1

9,428 6,702

correct abstained

3.6-flash

R EAD Dense

7 1

58,320 6,702

correct† abstained

budget, differing only in that its retrieval tool returns top-k chunks instead of R EAD’s four operations. It scores 27.5%. Iteration alone therefore does not close the gap — an agent allowed to query a vector index repeatedly remains 31.4 points behind the same agent given a lexical, structural interface (p = 0.012). The gain belongs to the interface, not to the loop.

Table 6. Pilot on the motivating question, two backbones. † We first recorded this cell as citing the wrong statement. That was our own parsing bug, not the model’s error, and the retraction is described below. This is n=2 and is not a result; it is reported to show the failure mode concretely.

8.2. What does not support a stronger claim BM25 is not beaten. At 51.0% it sits 7.8 points below R EAD with a confidence interval spanning zero (p = 1.00). With 51 questions the smallest difference this design can resolve at 80% power is 27.7 points, so this is a null result, not evidence of equivalence. That threshold is an a priori, unpaired approximation at a base rate of one half; the paired test is more sensitive, because it conditions on discordant pairs alone — in Section D a 23.5-point gap does reach significance. It is therefore a guide to what this benchmark can be expected to see, not a decision rule. What makes the BM25 comparison uninformative is more specific: of 22 discordant questions, 13 favour R EAD and 9 favour BM25, which is close to a coin toss. The honest reading is that our evidence separates embedding-based retrieval from embedding-free retrieval, and does not separate the two embedding-free systems from each other. That is a narrower claim than “agentic search wins”, and it happens to be the claim the paper set out to test: on this document, the vector index is not merely unnecessary — it is the component doing the damage. BM25 also costs a third as much as R EAD and runs in a third the latency, which matters for anyone choosing a system rather than testing a hypothesis.

chunks contained both. R EAD located both and computed the difference, Rs. 30,021.95 crore, a value that appears nowhere in the document. Two further observations shaped the protocol. First, iteration is expensive: R EAD consumed 58–78k input tokens against the dense baseline’s 6.7–8.2k, roughly an order of magnitude, and any cost-accuracy claim must carry this. Second, a methodological caution we record because it nearly became a result. We initially read one backbone’s answer as citing the wrong statement — a provenance failure invisible to exact-figure accuracy. On inspection the model was right and our analysis was wrong: our sectionattribution code recognised only Markdown headings, while the converter renders each statement’s running header as a table row with the title split across cells. Roughly a hundred lines were therefore attributed to the preceding statement. The lesson is not about the model. It is that a parsing bug in the analysis layer can manufacture a confident, plausible finding about model behaviour, and that such a finding is most likely to survive review when it is the one the authors expected to see. We caught it by reading the raw lines, not by any metric.

Both ablations score above R EAD. Removing memory outline, or replacing normalized matching with raw regular expressions, each yields 66.7% against R EAD’s 58.8%. Neither difference approaches significance (p = 1.00), and at n = 51 a 7.8-point gap is four questions. We investigated rather than dismissed it (Section 7): the six questions driving the inversion decompose into one harness failure, one genuine sign error, one correct answer buried in hedging, and three real search failures. We therefore cannot claim the full operation set is the best configuration — only that it is not detectably worse. A larger benchmark may well show that outline access is dead weight on a document whose headings the converter mangles.

8. Results All systems use gemini-2.5-pro and see the same 51 questions. Every comparison is paired on the question, tested with an exact McNemar test, and Holm-corrected across the family of six baselines. Confidence intervals on accuracy are Wilson; intervals on differences are paired bootstrap.

R EAD is not the most grounded system. At 58.0% it trails BM25 (67.3%) and hybrid (72.0%). A system that reads more text has more opportunity to state a figure that its own retrieved text does not contain, and R EAD exposes far more lines per question than a top-k retriever does. This cuts against the interpretability argument and we report it as such.

8.1. Main comparison R EAD answers 58.8% of the benchmark against dense retrieval’s 15.7%, a paired difference of 43.1 points that survives correction at pHolm = 2 × 10−5 (Table 8). The same holds against the hybrid pipeline (+29.4, p = 0.016). The comparison we consider most informative is agentic RAG: the same agent, the same backbone, the same turn

The dense baseline can be tuned well above the figure we 8

Beyond Top-K: Interpretable Agentic Operations for Document Retrieval Table 7. Accuracy and behaviour over 51 questions. Grounded is the fraction of answers in which every figure stated appears in text the system actually retrieved or read. Halluc. is the fraction of the five unanswerable probes answered rather than declined. Intervals are Wilson 95%. system

acc. (%)

95% CI

grounded

halluc.

$/q

R EAD and its ablations no-outline 66.7 naive-grep 66.7 R EAD 58.8

[53.0, 78.0] [53.0, 78.0] [45.2, 71.2]

66.7 60.4 58.0

40 0 40

0.040 0.043 0.058

retrieval baselines BM25 hybrid agentic RAG dense

[37.7, 64.1] [18.7, 43.0] [17.1, 40.9] [ 8.2, 28.0]

67.3 72.0 42.6 47.9

40 60 0 80

0.020 0.022 0.050 0.023

51.0 29.4 27.5 15.7

Table 8. Paired comparisons against R EAD. Positive ∆ favours R EAD. p is exact McNemar, Holm-corrected over the six comparisons. dense here is the configuration of Section 6 (2,000-character table-aware chunks, k=8); Section D sweeps that configuration and reports the comparison against dense’s best setting. vs.

∆ (pp)

95% CI

pHolm

dense agentic RAG hybrid BM25 naive-grep no-outline

+43.1 +31.4 +29.4 +7.8 −7.8 −7.8

[27.5, 56.9] [13.7, 49.0] [11.8, 47.1] [−9.8, 25.5] [−23.5, 7.8] [−21.6, 5.9]

0.00002 0.012 0.016 1.00 1.00 1.00

retrieval is not uniformly weak; it is weak exactly where the document is numeric. Table 10 isolates the mechanism. Dense retrieval fails to place the gold evidence in context on 13 questions and hybrid on 15, against R EAD’s 4. The consequence is visible in the next column: dense declines to answer 11 answerable questions and hybrid 12, against R EAD’s 1. This is the failure mode from Section 1 — retrieval declining to deliver — and it is the common case rather than a curiosity. Across the whole benchmark, dense retrieval surfaces a gold page in only 4.3% of questions, against R EAD’s 41.3%.

∗ ∗ ∗

Two further observations. First, on the two conversionlimited questions only the R EAD family and hybrid score at all; the purely lexical and purely dense retrievers score zero. Second, agentic RAG shows the opposite profile to dense: it rarely fails to surface something (5 never-surfaced) but produces 21 ungrounded answers, the worst of any system — an agent that keeps querying a vector index accumulates plausible context and then reasons over it past the point the evidence supports.

report. Table 7 lists dense at 15.7%, which is its accuracy at k=8 — the depth every retrieval baseline in this paper uses, so that the comparison is matched. It is not dense’s best. Sweeping chunk size and depth (Section D) finds 35.3% at 2,000 characters and k=16, more than double. The mechanism is the one Section 4 predicts: a top-k system cannot ensure a chunk arrives with the header that gives its figures a unit and a year, so its only recourse is to retrieve more chunks and cover the header by chance. Doubling k buys roughly that. We therefore do not claim dense retrieval scores 15.7% on this document — we claim it scores 35.3% when tuned, at 41% higher cost per question, and that R EAD still beats that configuration by 23.5 points (pHolm = 0.017). Every one of the six configurations we swept is beaten significantly.

8.4. Behaviour on unanswerable questions Five questions have no answer in the document. Dense retrieval answers four of them anyway (80%), hybrid three, R EAD and BM25 two. The interesting pair is the two systems that never fabricated: naive-grep and agentic RAG, both at 0%. For agentic RAG this is not a virtue — it also abstains on 8 answerable questions, so it is simply a reticent system. With only five probes, none of these differences is individually meaningful; we report the counts rather than rates for that reason, and treat the pattern as suggestive at best.

8.3. Where the difference comes from The gap is concentrated in lookup, the largest category: 70.8% for R EAD against 12.5% for dense (Table 9, Figure 4). These are the questions that ask for one printed figure from a named row and column, which is precisely where thousands of near-identical numbers compete in one embedding space (Section 4). On navigation — questions about which statement, which page, which unit — every system does comparatively well, because the answer is carried by prose and headings rather than by digits. Dense

8.5. Cost R EAD is the most expensive system per question ($0.058 against dense’s $0.023) and the slowest (31.4 s against 12.7 s), because it makes 4.8 tool calls per question and 9

Beyond Top-K: Interpretable Agentic Operations for Document Retrieval Table 9. Accuracy (%) by category. Conv.-lim. questions have evidence damaged in PDF conversion (Section 5.3); no system can be expected to reach 100%. system

look. (24)

arith. (9)

agg. (5)

nav. (6)

conv.-lim. (2)

unans. (5)

no-outline naive-grep R EAD BM25 hybrid agentic RAG dense

70.8 75.0 70.8 62.5 20.8 16.7 12.5

33.3 33.3 44.4 33.3 0.0 11.1 11.1

100.0 60.0 40.0 20.0 60.0 20.0 0.0

83.3 66.7 50.0 66.7 66.7 50.0 50.0

50.0 50.0 50.0 0.0 50.0 0.0 0.0

60.0 100.0 60.0 60.0 40.0 100.0 20.0

Table 10. Failure taxonomy: counts of failing cells by cause, over 51 questions. Never surfaced means the gold evidence never entered the model’s context. system R EAD naive-grep no-outline BM25 dense hybrid agentic RAG

never surfaced

abstained (answerable)

ungrounded

4 6 5 7 13 15 5

1 0 0 7 11 12 8

11 9 7 8 14 5 21

reads substantially more text. Against BM25 — the baseline it does not significantly beat — it costs 2.9× as much for 7.8 points that may be noise. A practitioner reading only Table 7 should take BM25 seriously. The case for R EAD rests on the failure taxonomy and on auditability, not on the cost–accuracy frontier.

their evidence from printed pages 2–18 of a 745-page volume. Both defects were identified from the document rather than from any system’s results, and questions were drafted without running them first. But both happen to fall where chunk-based retrieval is weak, and we knew that when we chose to fix them. A reader should treat the n=51 result as one we sought and found, not one we stumbled into; the pre-registered version of this study would have fixed the benchmark first. We report the accuracy of dense retrieval at each stage — 24.0%, 16.7%, 15.7% — so the drift is visible rather than hidden.

9. Limitations One document, one backbone, 51 questions. Every number here comes from a single financial report read by a single model. Nothing establishes that the result transfers to another document class, another accounting convention, or a weaker backbone — R EAD presumes a model that can plan a search, and we have not measured where that ability gives out. At this size the benchmark is expected to resolve paired differences of roughly 28 points, so the 7.8-point R EAD–BM25 gap is unresolved rather than absent.

R EAD is not the most grounded system, trailing BM25 and hybrid on the fraction of answers whose every figure appears in retrieved text (Section 8). An interpretability argument that cannot demonstrate superior groundedness is weaker for it. Scope. Claims are limited to single, long, structured documents consumed by capable tool-calling models. Lexical anchoring presumes the answer’s vocabulary is guessable from the question; where queries and evidence share little surface vocabulary, dense retrieval’s paraphrase tolerance may retain an edge and hybrid designs remain sensible.

The benchmark was built by the authors of the system under test. Gold answers are validated mechanically against the document rather than by the drafter’s own reading, and the categories where R EAD is weakest are reported rather than dropped (Section C). Neither of those removes the conflict.

One document, one converter. All measurements come from a single report converted by a single tool. The conversion ceiling in Section 5.3 is a property of pymupdf4llm on this PDF, not a universal constant; a different converter would move it, and a commercial layout model would likely lower it at a cost we did not pay. What we claim is that the ceiling exists and must be measured, not that this particular

The benchmark grew while the experiment ran, and the growth was not blind. An earlier 25-question version produced no significant difference between any pair of systems. Two further batches were added after inspection revealed coverage defects — first that no question exercised column selection in a wide table, then that 31 of 36 questions drew 10

Beyond Top-K: Interpretable Agentic Operations for Document Retrieval

BM25 top-k

READ (ours)

Accuracy (%)

100 80

67 67

60 62

60 60 40 20

44 20

Dense top-k

Hybrid + rerank

20 0

Single-figure lookup

Multi-row aggregation

Cross-statement arithmetic

Structural / navigational

Unanswerable probes

Figure 4. Accuracy by question category, R EAD against the three retrieval baselines. Bars are Wilson 95% intervals; the categories carry 24, 5, 9, 6 and 5 questions respectively, so the intervals are wide and most single-category differences are not individually significant. The pattern that matters is the first group: on single-figure lookup, where the document is densest in near-identical numbers, dense retrieval reaches 12% and R EAD 71%. On structural questions, which are carried by prose rather than digits, the four systems are indistinguishable.

value generalises. The unit-inheritance property is common to Indian government financial statements but its distances are specific to this document; the artifact counts are specific to pymupdf4llm. Generality requires more documents, more converters, and other domains.

leaves up to 67.4% of numeric chunks unitless; a tableaware chunker built as a steelman reduces that to 0.3%, but 27–30% still lack a fiscal-year header at every chunk size tested. That residue is flat across a five-fold range of chunk size: it is not a tuning failure but what happens when a partition is fixed before the query is known.

Cost. R EAD’s per-query cost is variable and can exceed a fixed top-k budget substantially, as the pilot shows. R EAD trades worst-case cost for correctness and auditability, which is the right trade in high-stakes settings and not obviously the right one in high-volume consumer search.

R EAD replaces the chunk–embed–index pipeline with deterministic operations, turning retrieval from an opaque ranking into a replayable procedure. On 51 questions it answers 58.8% against dense retrieval’s 15.7%, and the mechanism behind the gap is visible rather than inferred: dense retrieval never places the required evidence in context on 13 questions, where R EAD fails to on 4. An agent handed the same loop and backbone but a top-k tool reaches 27.5%, so the gain is attributable to the interface and not to iteration.

What this should not be read as claiming. Not that embeddings are obsolete, and not that dense retrieval is weak in general — BEIR-style benchmarks measure something real. The claim is narrower: for documents whose meaning depends on inherited layout context, a partition fixed before the query is the wrong primitive.

We are equally clear about the boundary of the claim. A BM25 baseline is statistically indistinguishable from R EAD at a third of the cost. What the evidence separates is embedding-based from embedding-free retrieval on this document — not agentic search from lexical search. The vector index was not merely unnecessary here; it was the component doing the damage, and a practitioner with a lexical index and no agent would already have captured most of the available gain.

10. Conclusion We challenged the assumption that question answering over long documents requires embedding-based retrieval, and we grounded the challenge in measurement. On a 780-page financial report, 86.8% of content is tabular, numbers repeat 3.68-fold, and units are inherited from headers a median of 13 lines above the figures they govern. Fixed-size chunking

As models become competent agents, the retrieval question 11

Beyond Top-K: Interpretable Agentic Operations for Document Retrieval

shifts from which retriever ranks best to which interface lets the agent see, verify, and account for the evidence. For long structured documents we argue the answer is the simplest one: let the model read.

758–759, New York, NY, USA, 2009. Association for Computing Machinery. ISBN 9781605584836. doi: 10.1145/1571941.1572114. URL https://doi. org/10.1145/1571941.1572114. Dong, H., Zhao, J., Tian, Y., Xiong, J., Zhou, M., Lin, Y., Cambronero, J., He, Y., Han, S., and Zhang, D. Encoding spreadsheets for large language models. In Al-Onaizan, Y., Bansal, M., and Chen, Y.-N. (eds.), Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp. 20728–20748, Miami, Florida, USA, November 2024. Association for Computational Linguistics. doi: 10.18653/v1/2024.emnlp-main.1154. URL https://aclanthology.org/2024.em nlp-main.1154/.

Acknowledgements References Anthropic. Introducing the Model Context Protocol. http s://www.anthropic.com/news/model-con text-protocol, 2024. Accessed 2026-08-03. Asai, A., Hashimoto, K., Hajishirzi, H., Socher, R., and Xiong, C. Learning to retrieve reasoning paths over wikipedia graph for question answering, 2020. URL https://arxiv.org/abs/1911.10470.

Formal, T., Piwowarski, B., and Clinchant, S. Splade: Sparse lexical and expansion model for first stage ranking. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’21, pp. 2288–2292, New York, NY, USA, 2021. Association for Computing Machinery. ISBN 9781450380379. doi: 10.1145/3404835.3463098. URL https://doi.org/10.1145/3404835. 3463098.

Asai, A., Wu, Z., Wang, Y., Sil, A., and Hajishirzi, H. Selfrag: Learning to retrieve, generate, and critique through self-reflection, 2023. URL https://arxiv.org/ abs/2310.11511. Chen, L., Zaharia, M., and Zou, J. How is ChatGPT’s behavior changing over time? Harvard Data Science Review, 6(2), mar 12 2024. https://hdsr.mitpress.mit.edu/pub/y95zitmz.

Gao, Y., Xiong, Y., Gao, X., Jia, K., Pan, J., Bi, Y., Dai, Y., Sun, J., Wang, M., and Wang, H. Retrieval-augmented generation for large language models: A survey, 2024. URL https://arxiv.org/abs/2312.10997.

Chen, M., Tworek, J., Jun, H., Yuan, Q., de Oliveira Pinto, H. P., Kaplan, J., Edwards, H., Burda, Y., Joseph, N., Brockman, G., Ray, A., Puri, R., Krueger, G., Petrov, M., Khlaaf, H., Sastry, G., Mishkin, P., Chan, B., Gray, S., Ryder, N., Pavlov, M., Power, A., Kaiser, L., Bavarian, M., Winter, C., Tillet, P., Such, F. P., Cummings, D., Plappert, M., Chantzis, F., Barnes, E., Herbert-Voss, A., Guss, W. H., Nichol, A., Paino, A., Tezak, N., Tang, J., Babuschkin, I., Balaji, S., Jain, S., Saunders, W., Hesse, C., Carr, A. N., Leike, J., Achiam, J., Misra, V., Morikawa, E., Radford, A., Knight, M., Brundage, M., Murati, M., Mayer, K., Welinder, P., McGrew, B., Amodei, D., McCandlish, S., Sutskever, I., and Zaremba, W. Evaluating large language models trained on code, 2021. URL https://arxiv.org/abs/2107.0 3374.

Gulati, A., Sen, S., Sarguroh, W., and Paul, K. From rows to reasoning: A retrieval-augmented multimodal framework for spreadsheet understanding, 2026. URL https:// arxiv.org/abs/2601.08741. Izacard, G., Caron, M., Hosseini, L., Riedel, S., Bojanowski, P., Joulin, A., and Grave, E. Unsupervised dense information retrieval with contrastive learning, 2022. URL https://arxiv.org/abs/2112.09118.

Comptroller and Auditor General of India. Finance accounts 2024–25, volume i. Office of the Principal Accountant General (A&E), Gujarat, 2025. URL https://cag. gov.in/uploads/state_accounts_report /account-report-FA-VOL-I-2024-25-069 c52aa2b34bf9-63690940.pdf. Cormack, G. V., Clarke, C. L. A., and Buettcher, S. Reciprocal rank fusion outperforms condorcet and individual rank learning methods. In Proceedings of the 32nd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’09, pp. 12

Karpukhin, V., Oguz, B., Min, S., Lewis, P., Wu, L., Edunov, S., Chen, D., and Yih, W.-t. Dense passage retrieval for open-domain question answering. In Webber, B., Cohn, T., He, Y., and Liu, Y. (eds.), Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 6769–6781, Online, November 2020. Association for Computational Linguistics. doi: 10.18653/v1/2020.emnlp-main.550. URL https://ac lanthology.org/2020.emnlp-main.550/. Khattab, O. and Zaharia, M. Colbert: Efficient and effective passage search via contextualized late interaction over bert. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’20, pp. 39–48, New York, NY, USA, 2020. Association for Computing Machinery.

Beyond Top-K: Interpretable Agentic Operations for Document Retrieval

ISBN 9781450380164. doi: 10.1145/3397271.3401075. URL https://doi.org/10.1145/3397271. 3401075. Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W.-t., Rocktäschel, T., Riedel, S., and Kiela, D. Retrieval-augmented generation for knowledge-intensive nlp tasks. In Proceedings of the 34th International Conference on Neural Information Processing Systems, NIPS ’20, Red Hook, NY, USA, 2020. Curran Associates Inc. ISBN 9781713829546.

Trivedi, H., Balasubramanian, N., Khot, T., and Sabharwal, A. Interleaving retrieval with chain-of-thought reasoning for knowledge-intensive multi-step questions, 2023. URL https://arxiv.org/abs/2212.10509. Wang, X., Wang, Z., Gao, X., Zhang, F., Wu, Y., Xu, Z., Shi, T., Wang, Z., Li, S., Qian, Q., Yin, R., Lv, C., Zheng, X., and Huang, X. Searching for best practices in retrievalaugmented generation. In Al-Onaizan, Y., Bansal, M., and Chen, Y.-N. (eds.), Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp. 17716–17736, Miami, Florida, USA, November 2024. Association for Computational Linguistics. doi: 10.18653/v1/2024.emnlp-main.981. URL https://ac lanthology.org/2024.emnlp-main.981/.

Li, Z., Zhang, H., Wei, C., Lu, P., Nie, P., Lu, Y., Bai, Y., Feng, S., Zhu, H., Zhong, M., Zhang, Y., Xie, J., Choi, Y., Zou, J., Han, J., Chen, W., Lin, J., Jiang, D., and Zhang, Y. Beyond semantic similarity: Rethinking retrieval for agentic search via direct corpus interaction, 2026. URL https://arxiv.org/abs/2605.05242.

Xia, C. S., Deng, Y., Dunn, S., and Zhang, L. Agentless: Demystifying llm-based software engineering agents, 2024. URL https://arxiv.org/abs/2407.01489.

Liu, N. F., Lin, K., Hewitt, J., Paranjape, A., Bevilacqua, M., Petroni, F., and Liang, P. Lost in the middle: How language models use long contexts. Transactions of the Association for Computational Linguistics, 12:157–173, 2024. doi: 10.1162/tacl a 00638. URL https://ac lanthology.org/2024.tacl-1.9/.

Yang, J., Jimenez, C., Wettig, A., Lieret, K., Yao, S., Narasimhan, K., and Press, O. Swe-agent: Agentcomputer interfaces enable automated software engineering. Advances in Neural Information Processing Systems, 37, 2024. ISSN 1049-5258. Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K., and Cao, Y. React: Synergizing reasoning and acting in language models, 2023. URL https://arxiv.or g/abs/2210.03629.

Robertson, S., Walker, S., Jones, S., Hancock-Beaulieu, M. M., and Gatford, M. Okapi at trec-3. In Overview of the Third Text REtrieval Conference (TREC-3), pp. 109–126. Gaithersburg, MD: NIST, January 1995. URL https://www.microsoft.com/en-us/rese arch/publication/okapi-at-trec-3/.

Yu, F. J. Grep vs graph: Agentic search is powerful, but enterprise AI needs governed knowledge. https://me dium.com/@yu-joshua/grep-vs-graph-age ntic-search-is-powerful-but-enterpris e-ai-needs-governed-knowledge-8de709c 31451, 2025. Accessed 2026-08-03.

Schick, T., Dwivedi-Yu, J., Dessı́, R., Raileanu, R., Lomeli, M., Hambro, E., Zettlemoyer, L., Cancedda, N., and Scialom, T. Toolformer: language models can teach themselves to use tools. In Proceedings of the 37th International Conference on Neural Information Processing Systems, NIPS ’23, Red Hook, NY, USA, 2023. Curran Associates Inc. Sen, S., Kasturi, A., Lumer, E., Gulati, A., and Subbiah, V. K. Is grep all you need? how agent harnesses reshape agentic search, 2026a. URL https://arxiv.org/ abs/2605.15184. Sen, S., Lumer, E., Gulati, A., and Subbiah, V. K. Chronos: Temporal-aware conversational agents with structured event retrieval for long-term memory, 2026b. URL ht tps://arxiv.org/abs/2603.16862. Thakur, N., Reimers, N., Rücklé, A., Srivastava, A., and Gurevych, I. Beir: A heterogenous benchmark for zeroshot evaluation of information retrieval models, 2021. URL https://arxiv.org/abs/2104.08663. 13

Beyond Top-K: Interpretable Agentic Operations for Document Retrieval

A. Converter Comparison Section 5.3 argues that PDF→Markdown conversion sets a ceiling on what any retriever can recover. That ceiling is a property of the converter, so the converter was chosen before any retrieval system was built, and the choice is documented here rather than asserted. Four backends were run over the full 780-page source. Three completed and reached the back cover; docling never finished (below). Table 11. Converter comparison on the full document. The rating is our judgement; the counts beneath it are what drove it.

backend

verdict

failure mode

pymupdf4llm

chosen

cloudconvert

rejected

markitdown

rejected

docling

unrated

Keeps table columns aligned. Known defects accepted: a mangled table of contents and a leading empty column on every row. Neither touches the financial statements. Collapses whole tables into a single -joined cell — up to 30 printed rows in one cell. Salvageable in principle, not usable as given. Severs row labels from their figures. The loss is unrecoverable: no downstream process can reattach a label to the right row. Never completed a run.

The deciding evidence. pymupdf4llm cloudconvert markitdown

The same statement row, as rendered by each backend:

||Co-operation|26.23|15.93| columns intact |**(B)** Housing 74.48 18.48 ...| 30 rows in one cell Co-operation | | | | 26.23 | 15.93 | label orphaned from its figures

Supporting counts. cloudconvert emits 83,535 tokens and 2,459 collapsed mega-cells against pymupdf4llm’s 18,319 and 842; 1,506 of its rows are only two columns wide, which is the collapse signature. markitdown fragments 5,547 data rows across 2,589 tables, about 2.1 rows per table. pymupdf4llm’s column counts peak at 5–10, matching the real statements. Two measures that look decisive and are not. Digit counts mislead: cloudconvert carries roughly 39,000 more digit characters, but all three backends recover the same ≈15,940 distinct numbers — the surplus is repetition introduced by cell collapse, not extra information. Encoding fidelity is a wash: 2 bad glyphs in pymupdf4llm, 2 in markitdown, 0 in cloudconvert, across about 2 MB. Had we scored converters on either metric we would have chosen cloudconvert, whose output is the least usable of the three that completed. On docling. Every run died with std::bad alloc during preprocessing. The cause is its default v1 parse backend compounded by Windows commit-limit exhaustion: at the point of failure the machine had 15.8 GB of free RAM but only 0.4 GB of commit headroom. The failing page drifts with available memory — page 4 on one run, page 7 on another — so this is an environment interaction, not a poisoned page, and we record it as unrated rather than as a quality judgement. Caveat. These ratings are ours, applied to one document, by readers who knew which backend they had already been using. They are reported so the conversion ceiling in Section 5.3 can be attributed to a specific artifact, not offered as a converter benchmark. 14

Beyond Top-K: Interpretable Agentic Operations for Document Retrieval

B. Implementation Details B.1. The four operations R EAD exposes four tools over the Model Context Protocol. The server is read-only and jailed to a root directory: every path is resolved and rejected if it escapes that root, so no trajectory can read outside the corpus. memory list Enumerate files under an optional subdirectory. memory outline Return the heading structure of a file, each heading with its line number. memory grep Search line by line. Takes query and optional path, context (0–10 lines, default 2), and max matches (default 25). Returns line no: prefixed matches with surrounding context. memory read Return lines start line to end line of a file, 1-indexed and inclusive. The design point is that memory grep returns line numbers and memory read accepts them, so the agent composes locate-then-read without any intermediate representation the user cannot inspect. Line numbers are 1-indexed everywhere, matching the released MCP server, so a trajectory can be replayed against the document by hand. B.2. Normalized matching A query is matched against two shadow forms of every line, and succeeds if either contains it. Both start from a base form: Unicode NFKC normalization, emphasis markers (*, ) stripped, case folded. spaced form Table-cell boundaries (|) replaced by spaces and whitespace collapsed. This makes cell structure invisible to matching while keeping word boundaries. squashed form All cell boundaries, whitespace, and thousands separators removed. This is what lets "Fiscal Deficit" match the converted |**Fiscal**| **Defi**|**cit**|, where the converter split the word across two cells, and "48964.89" match the printed 48,964.89. An empty query after normalization is an error rather than a match-everything. Section 5.2 measures what each form contributes. B.3. The table-aware chunker The chunker is our steelman for the dense and hybrid baselines, not part of R EAD. It packs lines up to chunk chars, and at every boundary prepends a context header recovered by scanning backwards from the boundary for two things: the nearest unit declaration ((Rs. in lakh) and variants) and the nearest fiscal-year or as-at-date header. Both are emitted in document order so the carried header reads as it does in the book. Two details matter. First, the header search looks backwards rather than carrying the table run’s first rows: in this corpus the unit declaration usually sits above the table run, so carrying the run’s own first rows propagates decoration and not meaning. Fixing this moved the fraction of unitless numeric chunks from 18.0% to 0.3% and is the difference between the steelman working and appearing to work. Second, the seed budget is capped at max(200, chunk chars/2) and each carried line at max(120, chunk chars/4), so a wide header cannot crowd out the chunk body it is meant to explain. The chunker is forward-only. An earlier rewind-based implementation of overlap could fail to make forward progress on long table runs and exhaust memory; the current one always advances. The chunker version participates in the index cache key, so a change to the algorithm cannot be silently served from a stale index. B.4. Models and decoding All systems use gemini-2.5-pro as the generator through the google-genai SDK, with a manual tool loop rather than the SDK’s automatic one so that every call, its arguments, and the exact text it exposed are logged. Dense retrieval uses BAAI/bge-base-en-v1.5; hybrid adds cross-encoder/ms-marco-MiniLM-L-6-v2 as a reranker over the fused candidate set. One decoding detail is not cosmetic. Gemini charges thinking tokens against max output tokens. Under a dynamic 15

Beyond Top-K: Interpretable Agentic Operations for Document Retrieval

thinking budget the model can spend the entire allowance reasoning and emit zero output tokens, returning an empty answer with stop reason=MAX TOKENS — which a grader scores as a wrong answer rather than as a harness failure. This affected 2 of 175 cells in an early run before we found it. Thinking is now capped at half the output budget. We report it because the failure is silent, is invisible in accuracy numbers, and would bias whichever system is given the least generous budget.

C. Benchmark Details C.1. Composition The benchmark holds 51 questions with verified gold answers, drawn from six categories (Table 12). Gold evidence is recorded as line numbers into the converted Markdown, from which printed page numbers are derived. Table 12. Benchmark composition. “Derived” questions have answers that appear nowhere in the document and must be computed.

Beyond Top-K: Replacing Black-Box Retrieval with Interpretable Agentic Operations

Analyze & explore related research in Conceptio

Related documents