From Research Questions to Columns: Operationalization-Aware Data Discovery

From Research Questions to Columns: Operationalization-Aware Data Discovery Houming Chen

H. V. Jagadish

University of Michigan Ann Arbor, Michigan, USA houmingc@umich.edu

University of Michigan Ann Arbor, Michigan, USA jag@umich.edu

arXiv:2608.04536v1 [cs.DB] 5 Aug 2026

Abstract Researchers often approach a data repository with an abstract concept and ask which columns can measure it. Useful columns may not resemble the query; they may matter only as complementary indicators in a defensible measure. This need differs from schema linking and column retrieval, which begin from more explicit needs and reward direct relevance. We define operationalization-aware data discovery (OADD): given a broad question and a database, optionally under a scope constraint, OADD jointly determines how focal concepts can be measured with available data and identifies supporting columns. Developing OADD methods requires examples for design and evaluation, but asking researchers to supply conceptual questions and their columns is impractical. We construct OADD-Bench by treating empirical papers as records of schema in use. A question miner extracts and reframes a paper-supported question; a paper-conditioned column miner reconstructs its measurements and grounds them to database identifiers. We admit only mappings supported by the publication and database documentation. OADD-Bench contains 160 questions from 111 papers and 4,682 question–column labels. Each target records a measurement used in published research; the paper supplies the precedent, while the miners extract and ground it. We evaluate lexical and neural retrieval, adapted schema-linking systems, and large language model (LLM) OADD agents. Each method receives only a question, permitted years, and dataset metadata; source papers are used only to construct and document benchmark labels. At the largest output limit, direct retrieval reaches at most 0.185 recall. The strongest schema-linking adaptation reaches 0.401 but remains optimized for a different objective; an OADDdirected agent performs best at 0.465. Even this agent covers less than half the target columns, showing that OADD remains an open problem.

CCS Concepts • Information systems → Data management systems; Information retrieval.

consider the concretization of an abstract concept, the triangulation to it by means of related other variables, or other such adjustments needed to “operationalize” the concept of interest to the researcher. As such, there is an operationalization gap: deciding which concrete observations can represent an abstract concept, whether or not any of them directly resemble the target concept syntactically or semantically. Motivating example. Consider the famous Health and Retirement Study (HRS), a longitudinal study of health, cognition, work, wealth, family, and social life [22]. Most HRS variables record recurring survey responses, alongside many other types of data. Imagine a researcher investigating whether social isolation is related to healthy aging. Searching for “social isolation” finds no ready-made field. A fuzzy search may return variables about Social Security; a semantic search may return a question about feeling lonely. Neither reveals a defensible social-isolation measure. Yet Cenzer et al. [3] construct one by combining marital status, household size, proximity to children, religious attendance, and volunteering—all variables available in HRS. Individually, none of these variables states “social isolation”; together, they form a measure of it. Finding this bundle requires crossing the operationalization gap, not merely improving lexical or semantic matching. Figure 1 summarizes this indirect mapping. We call this task operationalization-aware data discovery (OADD). It is a common task for a substantial class of database users. Yet it has not been studied by the data management community. Its closest existing problem is schema linking, commonly studied in text-to-SQL [29]. Both map a natural-language query to database columns; they differ in what the query has already specified. A user asking “How many flights went from Los Angeles to New York yesterday?” may not know the schema names, but has specified the event, endpoints, time, and desired count. None of this is specified by a user seeking measures of social isolation. An OADD system

CONCEPT OF INTEREST

direct search no ready-made field

Social isolation

×

DATABASE

can be measured by

exist in

Keywords data discovery, operationalization, scientific data, benchmark

1

Introduction

A database organizes concrete facts, while a researcher approaching a data source has conceptual questions. Having the relevant observations in a database does not make them discoverable from a scientific idea. Most column-search methods look for a direct lexical or semantic relation between a query and a field. They do not

HRS

OBSERVABLE INDICATORS household size

proximity to children

religious attendance

volunteering

Figure 1: Operationalization connects a concept to available columns through observable indicators, even when direct search finds no field. Components follow Cenzer et al. [3].

Houming Chen and H. V. Jagadish

must therefore bridge the logical gap between a focal concept and the observations that can measure it, using only data that actually exist in the database. A scientifically sensible measure that requires unavailable observations is not an answer. Neither is a list of fields that fails to measure the focal concept. Without this step, a usable database may appear not to contain the concept, or a researcher may settle for a superficially related proxy; either failure can change which studies appear feasible and how their concepts are measured. In this paper, we identify the OADD problem and lay the groundwork for the data management community to address it. OADD formalizes work that users perform manually when they reconcile scientific concepts with available fields. It targets an early point in the research workflow: broad question → focal-column candidates → researcher validation and coding → analysis. Section 2.1 gives the formal problem definition. Schema-linking methods typically assume that users know what data they need even if they do not know the exact schema names. They therefore seek direct relevance and ignore fields valuable only within a measurement bundle. Consequently, standard schemalinking methods tend to perform poorly on OADD (see Section 4), motivating new methods for OADD. Before we can develop new methods, though, we need to specify the OADD task more completely. What constitutes a good answer, and what should an OADD technique optimize? The desired answer is a set of columns that can operationalize the focal concept, often through relationships requiring domain expertise to recognize. In short, we need a shared benchmark to make OADD concrete and give future methods a common basis for comparison. To create such a benchmark, we could directly ask human annotators to perform OADD: interpret the idea, choose a defensible measure, and locate its exact columns. This is expensive and requires multiple annotators with expertise in both the scientific domain and the database. Our idea instead is to leverage records already produced through scientific practice. Peer-reviewed papers usually record how domain researchers translated questions into measurements used in the reported analyses. (This record may appear only in supplementary materials.) Created by domain researchers for real studies and documented contemporaneously, these mappings are strong external positives rather than labels invented for evaluation. Crucially, they are independent of any OADD system or retrieval architecture. Our task then becomes to mine these implicit question-to-measurement mappings and ground them in exact column identifiers. As we describe in Section 3, this is still much easier than recruiting large numbers of expert annotators. Reconstructing the mapping from published papers is non-trivial because measurement details may be scattered across the main text and supplementary materials, while exact identifiers are often omitted. Construction must therefore synthesize paper evidence while navigating documentation (for more than 122,000 fields in the case of HRS). Recent large language model (LLM) agents make this construction strategy practical: they can combine document understanding with multi-step reasoning and tool use [14, 18]. The published study supplies the scientific precedent; the LLM finds its scattered measurement description and connects it to HRS documentation. A question miner extracts a paper-supported question and reframes

it for OADD; a column miner performs the more demanding measurement reconstruction and grounding. Having constructed the benchmark, we use it (in Section 4) to ask whether familiar search and schema-linking techniques already suffice. The OADD-directed agent achieves the highest recall, but even its strongest tested model covers less than half of the targets. To summarize, we make three main contributions: • OADD. We formulate a problem new to the data management community but familiar to researchers: jointly determining a scientific measurement and the available columns that can realize it. • OADD-Bench with publication-grounded construction. We build and release OADD-Bench,1 a benchmark for OADD comprising 160 publication-grounded questions with 4,682 labels from 111 papers in a 122,324-column scientific database. Our evidence-constrained method grounds paper-reported measurements, preserves publication and identifier provenance, and abstains when a complete mapping is unsupported. • Diagnostic evaluation. Target-relative recall across direct retrieval, five schema-linking adaptations, and one OADDagent design with three model configurations exposes the gap between direct relevance and coverage of publicationgrounded measurement bundles.

2 Operationalization-Aware Data Discovery 2.1 Problem definition Let 𝐷 be a database and Col(𝐷) its uniquely identified columns. A broad research question 𝑞 names one or more focal measurement roles—constructs or quantities examined as exposures, outcomes, mediators, or moderators—without fully specifying how to observe them. An optional scope constraint 𝐶 admits columns by time period, product, access tier, or explicit subset, inducing Col𝐶 (𝐷) = {𝑐 ∈ Col(𝐷) : 𝐶 (𝑐)}; without a restriction, all columns are eligible. An operationalization 𝑀 maps each focal role 𝑟 to a nonempty set 𝑀 (𝑟 ) ⊆ Col𝐶 (𝐷) whose fields, individually or jointly, provide a scientifically defensible measure of 𝑟 in the context of 𝑞. Its column Ð set is 𝑉 (𝑀) = 𝑟 𝑀 (𝑟 ). OADD problem. Given (𝑞, 𝐷, 𝐶), find the column set 𝑉 (𝑀) for a scientifically defensible operationalization 𝑀. A system may expose or rank its interpretations, but the required result is a set of column identifiers. Scientific defensibility is domain-dependent rather than a schema property. Moreover, 𝑀 and 𝑉 (𝑀) must be chosen jointly: an unrealizable measure and a related-but-insufficient column list are both invalid. This operationalization gap is intrinsic rather than a vocabulary mismatch: a useful column need not resemble 𝑞, and its role may emerge only within a bundle. OADD is the column-discovery stage of operationalization, not a complete study design. It ends once the question has been grounded in a tractable set of fields that a researcher can inspect. Coding, aggregation, modeling, and causal identification remain downstream; covariates, weights, join keys, and sample-flow rules are excluded unless the question makes them focal. 1 https://github.com/umich-dbgroup/OADD-Bench

From Research Questions to Columns

Table 1: Representative OADD question–column mappings. Permitted years are shown with each question; explanations make the intervening measurement decisions explicit, and target sets are shown in full. Research question 𝑞 and scope constraint 𝐶

Target columns 𝑉

Explanation

Could experiences of everyday discrimination be related to immune health in older adults? [12] [permitted years: 2016]

PLB029A–PLB029F; PCD4T_COUNT, PCD8T_COUNT, PBCELL_COUNT; PCD4N_COUNT, PCD4TEMRA_COUNT, PCD8N_COUNT, PCD8TEMRA_COUNT; PNAIVEB_COUNT, PIGD_PLUS_MEMB_COUNT, PIGD_MINUS_MEMB_COUNT

PLB029A–PLB029F record frequencies of less respectful treatment, poorer service, being regarded as unintelligent or feared, threats or harassment, and worse medical treatment. Averaging them operationalizes everyday discrimination. The ten blood-assay fields count total, naive, and terminally differentiated T- and B-cell populations, operationalizing immune-cell composition.

Could more positive views of aging protect everyday independence in later life? [23] [permitted years: 2014, 2016]

OLB028B01, OLB028B02, OLB028B3–OLB028B8; PG041, PG044, PG047, PG050, PG059

OLB028B01–OLB028B8 ask about perceived decline, energy, usefulness, happiness, expectations, satisfaction with aging, and discontinued activities and disliked aspects of aging. Coding the responses in a common direction and averaging them operationalizes views of aging. The five PG fields record difficulty preparing meals, shopping, making phone calls, taking medications, and managing money; their sum is a 0–5 measure of impaired everyday independence.

2.2

Benchmark formulation

OADD-Bench turns this task into a controlled empirical target. Each OADD-Bench example is 𝑏 = (𝑞, 𝐶, 𝑉 ), where 𝐶 specifies permitted survey years and 𝑉 records one database-realizable operationalization of the focal roles used in a published study. Documented use makes 𝑉 a strong positive target. Other defensible measurements may exist, but the available database constrains plausible alternatives: HRS social-isolation measures vary in coverage but repeatedly use household ties, family contact, and social participation [3, 20]. The publication establishes the scientific use and serves only as construction provenance. At evaluation time, the publication is absent: the system addresses OADD directly by selecting columns from the question and database. 𝑉 supplies a known positive set for reproducible scoring: overlap shows that the system found columns with demonstrated scientific use. A benchmark target must be objectively checkable. Construct names or prose plans would require semantic judgment at scoring time and leave the data unresolved. We therefore ground 𝑉 in exact HRS identifiers, enabling direct set comparison and field inspection. A reported recode or composite is traced to source fields or withheld. The metadata covers 122,324 identifiers across 169 products and releases. During prediction, evaluated systems receive the question, scope, output budget, and HRS metadata; the paper-derived measurement plan, target columns, and explanation are not method inputs. The released benchmark CSV provides these targets and a concise operationalization explanation for scoring and inspection. Companion JSON preserves detailed paper-use and identifier-grounding records for audits without redistributing paper text.

2.3

Recommended metrics

Exact-column recall measures coverage of these demonstrated-use targets under controlled output budgets, not exhaustive correctness over every possible operationalization. For 𝑅𝑞 = |𝑉𝑞 |, we report macro Recall@𝑅, Recall@2𝑅, and Recall@5𝑅, allowing at most 𝑅𝑞 , 2𝑅𝑞 , or 5𝑅𝑞 returned columns. These are evaluation budgets, not a deployment assumption that target size is known. They normalize

different target sizes, prevent return-all strategies, and let 5𝑅 accommodate broader candidate sets. Methods may return fewer columns, and scoring uses their actual output. Across many questions, low recall at 5𝑅 means repeatedly missing fields already demonstrated to support the focal concepts.

3 Constructing OADD-Bench 3.1 Construction overview Published studies provide scientifically grounded labels, but their measurement descriptions must be mapped to exact HRS identifiers. Rosenberg et al., for example, describe social isolation as a 15-question composite and place its components in a supplementary figure [20]. One is “Number of children within 10 miles,” but neither source states that this maps to QE012 in 2018 and KE012– PE012 in earlier waves. Grounding it requires joining scattered paper evidence with HRS documentation. Modern LLMs make this supervision extractable at scale through scientific interpretation and iterative metadata navigation [16, 33]. We use a question miner and a paper-conditioned column miner, retaining only evidencecomplete mappings (Figure 2).

3.2

Question miner

The question miner extracts the questions investigated in a paper and expresses each as a broad, early-stage question, preserving its focal concepts, relation, population, and scope. A paper may contribute multiple questions. This document-understanding task is related to scientific summarization and research-question extraction [2, 24].

Published HRS study and supplements Question miner: paper →

𝑞

Column miner: 𝑞 + paper + HRS metadata → 𝑉 Benchmark example (𝑞, 𝐶, 𝑉 ) , or unresolved mapping

Figure 2: Papers provide construction evidence; evaluated systems later receive only 𝑞, 𝐶, and the database metadata.

Houming Chen and H. V. Jagadish

It also removes measurement details that would reveal the answer. For example, the first question in Table 1 asks whether everyday discrimination is related to immune health without naming the six treatment-report fields or the T- and B-cell counts. Including those names would reduce the task to direct schema linking. The relation and population remain; the measurements have not yet been chosen.

3.3

Column miner

The column miner receives the question, the paper and its supplements, and official HRS metadata. It uses a coarse-to-fine process to turn scattered narrative evidence into a small evidence packet and then exact identifiers. All LLM passes use GPT-5.5 with schemaconstrained outputs. 1. Reconstruct the measurement. The miner identifies the focal roles in the question and synthesizes how the paper observes each one, including its components, scope, product clues, and supporting passages across the main text and supplements. It excludes nonfocal covariates and cannot propose identifiers at this stage. 2. Retrieve candidate evidence. Scale names, observable indicators, and codebook wording may differ sharply, so the miner issues separate queries for named measures, observable components, and likely codebook phrasing. Lexical retrieval is combined across identifiers, labels, question text, and product metadata; scope and repeated-wave structure recover compatible fields and companion items. Per-measurement capacity prevents an easy lexical match from displacing less obvious components. 3. Build a reviewable evidence packet. Retrieval is too brittle to select labels, while presenting every field with full codebook evidence is impractical. An LLM screening pass removes clearly unrelated candidates but retains plausible raw items, released scores, routing fields, companions, and repeated-wave variants. Full evidence is loaded only for this shortlist. 4. Ground or abstain. A final pass jointly examines the measurement record and shortlisted codebook evidence to resolve scope, units, routing, and raw versus released fields. It may select only packet-supported identifiers. If a focal component or derived-field lineage remains uncertain, the mapping stays unresolved. Deterministic checks reject identifiers absent from the evidence, non-HRS targets, inconsistent mappings, and duplicates.

3.4

Evidence-based selection and scale

Our publication frame is a June 2026 crawl of the public HRS bibliography, yielding 9,954 heterogeneous records [10]. We pursued full text through two routes: a systematic 500-record sample, screened for likely analytic HRS use and public availability, and unambiguous open-access Europe PMC records title-matched to the bibliography. Through public or authorized routes, we attempted 861 distinct records and obtained 450 full-text documents usable for construction. An HRS bibliography entry does not by itself yield an auditable label: some papers do not report an empirical HRS analysis, while others omit the details needed to reconstruct their focal measurements. We therefore select the acquired papers and their candidate questions in two stages.

The first stage determines paper eligibility. An accessible, peerreviewed article must report an empirical HRS analysis and describe an analytically central measure and its HRS scope well enough to reconstruct its observable components. Eligibility does not depend on miner success, metadata coverage, or retrieval difficulty. Of 450 acquired full texts, 364 pass. The second stage operates on candidate question–column examples; one paper may support several. To focus this challenge set on cases where a bundle must be discovered, an admitted example must ground at least two HRS columns. This selection defines the benchmark, not OADD itself, which may have a single-column answer. Each example must support every focal measurement in the released question with publication evidence. When a paper studies several relationships, a candidate question may focus on one supported relationship rather than summarize the entire paper; it cannot introduce a construct or guess an unresolved identifier or derived-variable lineage. Each selected identifier must be supported by loaded HRS documentation. The miners produce 448 candidate mappings; 140 remain after the two selection stages and a final consistency screening. We additionally include 20 annotated examples from distinct papers under the same evidence and grounding requirements. Together they form an evidence-complete 160-example release, not a sample of all HRS studies. Secondary consistency check. For these 20 examples, an annotator reviews and edits structured evidence prefills created from the papers, supplements, and HRS documentation. We compare the resulting annotations with the column-miner reconstructions across 640 labels. The miner covers 0.942 of the annotated labels, while 0.946 of its labels are retained by the annotations, yielding 0.933 macro set F1. Human annotations are not treated as gold: this reconstruction itself requires scientific and HRS expertise. Agreement therefore checks consistency between two reconstructions; the publication supplies the scientific precedent. The resulting benchmark contains 160 questions from 111 papers. Its 4,682 question–column label incidences cover 2,032 distinct HRS identifiers and 873 paper-documented measurement components. Target sets range from 2 to 136 columns (median 21; mean 29.3).

4 Testing OADD Systems 4.1 Evaluation design We compare direct retrieval, adapted schema linking, and LLMbased discovery. Every system receives only the question, permitted years, an output limit, and HRS metadata; paper identifiers, provenance, and measurement descriptions are hidden. This fixes a reproducible experimental boundary rather than restricting OADD generally. LLM conditions use metadata-search tools but no paper or Web retrieval. The accompanying artifact includes the benchmark CSV, detailed JSON audit records, raw HRS documentation, baselines, and scoring code. HRS representation. Each searchable record contains its identifier, label, codebook question text, and product metadata. Retrieval groups repeated-wave equivalents and expands permittedyear columns up to the output limit. Schema linkers receive the same metadata as 169 wide product or release tables with 126,091

From Research Questions to Columns

Table 2: Question-only macro exact-column recall on all 160 questions. Macro exact-column recall Method Direct column retrieval BM25 [19] TF–IDF [21] BGE-base [32] SPLADE++ [8] Four-way rank fusion [4] Schema linking RESDSQL ranker [13] CRUSH4SQL [11] Bidirectional [17] LinkAlign [30] AutoLink [31] LLM agents GPT-5.4 nano GPT-5.4 mini GPT-5.5

Recall@𝑅 Recall@2𝑅 Recall@5𝑅 0.039 0.038 0.051 0.056 0.068

0.069 0.071 0.104 0.087 0.111

0.129 0.127 0.185 0.159 0.179

0.012 0.020 0.132 0.156 0.185

0.021 0.028 0.173 0.177 0.282

0.054 0.048 0.198 0.203 0.401

0.232 0.259 0.399

0.270 0.305 0.458

0.272 0.319 0.465

table–column occurrences. Both representations resolve to official identifiers. Retrieval systems. BM25 and TF–IDF test lexical matching; SPLADE++ and BGE-base test learned sparse and dense retrieval. Four-way reciprocal-rank fusion combines their rankings. Schema-linking systems. We adapt five text-to-SQL methods. The released Spider-trained RESDSQL schema-item classifier reranks dense-prefiltered HRS packets without its SQL decoder [13]. Four LLM adaptations use CRUSH4SQL’s hypothetical-schema retrieval, LinkAlign’s gap-directed rewrite, AutoLink’s iterative exploration, and bidirectional table/column linking [11, 17, 30, 31]. We retain these linking mechanisms, map outputs to HRS identifiers, and omit SQL and value tools. These are OADD adaptations, not native text-to-SQL reproductions. CRUSH4SQL, LinkAlign, and AutoLink use GPT-5.5; bidirectional uses GPT-5.4 mini. OADD agents. The agent iteratively formulates metadata queries, inspects hybrid retrieval results, and returns exact identifiers. Its prompt requests coherent measurement bundles without prescribing an operationalization. We run this design with GPT-5.4 nano, GPT-5.4 mini, and GPT-5.5. Independent 𝑅, 2𝑅, and 5𝑅 runs may return fewer than their respective limits, so recall need not be monotone. Outputs are validated, deduplicated, and charged to the same budget.

4.2

Results and analysis

Direct similarity covers few benchmark targets. Across the lexical, dense, learned-sparse, and fused rankings in Table 2, no direct retrieval method exceeds 0.185 at 5𝑅. Thus, even with an allowance five times the reference-set size, direct question–field matching recovers less than one fifth of the documented target columns on average. This is consistent with the operationalization gap: useful fields need not directly resemble the question. Schema linking optimizes a different target. Schema linking is a closer comparison because it also maps language to schema elements, but a text-to-SQL question typically already expresses the required quantities. It therefore optimizes direct query–column relevance, whereas OADD must determine which observations could

measure a concept whose operationalization remains unspecified. The adaptations vary widely: the released RESDSQL ranker reaches 0.054 at 5𝑅, while the strongest LLM adaptation, AutoLink, reaches 0.401, compared with 0.465 for the OADD agent, whose performance we analyze in the following paragraph. A paired bootstrap over source papers estimates this mean difference at 0.063 (95% CI [0.004, 0.121]). These scores measure how schema-linking mechanisms transfer to OADD, not their native text-to-SQL performance. OADD-directed agents perform best, but remain incomplete. Higher column-level recall does not imply complete measurement bundles. At 5𝑅, the GPT-5.5 agent retrieves at least one field for 61.0% of a question’s measurement components on average and every field for 37.0%. The corresponding partial and complete rates are 59.0% and 29.6% for AutoLink, and 21.5% and 9.8% for BGE-base. The agent retrieves at least one field for every component in only 50 of 160 questions (AutoLink: 43). The remaining gap is not simply an output-budget effect. From 2𝑅 to 5𝑅, the strongest agent’s average output rises only from 33.3 to 39.9 columns, against an average 5𝑅 allowance of 146.3, while recall moves from 0.458 to 0.465. Model choice also matters within the fixed agent design: Recall@5𝑅 is 0.272 with GPT-5.4 nano, 0.319 with GPT-5.4 mini, and 0.465 with GPT-5.5. Yet even the strongest tested system misses more than half of the documented target columns.

5

Related Work

Schema linking and data discovery. Schema linking grounds natural-language requirements in tables and columns, usually for text-to-SQL [29]. OADD shares the question-to-schema interface, but begins before the required observations have been specified. This logical gap is the object of the task: useful columns may be indirect indicators whose role emerges only within a bundle, rather than columns that directly resemble the query. RESDSQL, CRUSH4SQL, LinkAlign, AutoLink, and bidirectional retrieval are therefore natural mechanisms to test, but their benchmarks begin from more explicit requirements [11, 13, 17, 30, 31]. Data-discovery systems retrieve tables and fields from metadata, values, relations, and learned representations [6, 7]; recent LLM systems extend semantic dataset and field search [1, 9, 28]. Their usual item-wise relevance objectives do not test whether a set of fields jointly realizes a scientific measurement. SDRQuerier maps text to predefined variables in one harmonized survey database and visualizes availability [27]; it leaves measurement assembly to users rather than discovering indirect bundles. Survey linking and scientific measurement. Prior work links a named phenomenon or publication passage to survey items [5, 25, 26]. OADD instead begins with a broad question, must infer the intervening measurement for its focal roles, and grounds the resulting bundle to exact columns. LLMs also induce measures and derive workflows [16, 33]; these can supply components, but OADD makes their joint realizability in the available database part of the measurement choice. Concurrent work catalogs operationalization failures in executable agent workflows [15]; OADD instead asks which fields can measure an unspecified scientific concept.

Houming Chen and H. V. Jagadish

6

Conclusion and Remarks

We introduced OADD, the data-discovery problem of jointly reasoning about scientific measurement and the database columns that can realize it. OADD-Bench makes the problem testable with 160 publication-grounded questions from 111 papers and 4,682 question–column labels. At 5𝑅, direct retrieval reaches only 0.185 recall. The strongest schema-linking adaptation reaches 0.401 despite optimizing a different target; the OADD-directed agent reaches 0.465 but still covers less than half of the target columns. OADDBench provides a common basis for developing stronger systems, including bundle-level discovery over repeated waves, provenanceaware grounding, and interactive researcher validation.

References [1] Muhammad Imam Luthfi Balaka, David Alexander, Qiming Wang, Yue Gong, Adila Krisnadhi, and Raul Castro Fernandez. 2025. Pneuma: Leveraging LLMs for Tabular Data Representation and Retrieval in an End-to-End System. Proceedings of the ACM on Management of Data 3, 3, Article 200 (2025), 28 pages. doi:10.1145/ 3725337 [2] Isabel Cachola, Kyle Lo, Arman Cohan, and Daniel Weld. 2020. TLDR: Extreme Summarization of Scientific Documents. In Findings of the Association for Computational Linguistics: EMNLP 2020. Association for Computational Linguistics, 4766–4777. doi:10.18653/v1/2020.findings-emnlp.428 [3] Irena Cenzer, Jacqueline M. Torres, Yulin Yang, Karla Renata Flores Romero, Mary C. Thoma, and Ashwin A. Kotwal. 2025. Development and Evaluation of a Brief Social Isolation Measure in the Nationally-Representative Health and Retirement Study. Journal of the American Geriatrics Society 73, 12 (2025), 3807– 3813. doi:10.1111/jgs.70056 [4] Gordon V. Cormack, Charles L. A. Clarke, and Stefan Buettcher. 2009. Reciprocal Rank Fusion Outperforms Condorcet and Individual Rank Learning Methods. In Proceedings of the 32nd International ACM SIGIR Conference on Research and Development in Information Retrieval. 758–759. doi:10.1145/1571941.1572114 [5] Nadine Dulisch, Andreas Oskar Kempf, and Philipp Schaer. 2015. Query Expansion for Survey Question Retrieval in the Social Sciences. In Research and Advanced Technology for Digital Libraries: TPDL 2015. Springer, 28–39. doi:10.1007/978-3-319-24592-8_3 [6] Grace Fan, Jin Wang, Yuliang Li, Dan Zhang, and Renée J. Miller. 2023. Semanticsaware Dataset Discovery from Data Lakes with Contextualized Column-based Representation Learning. Proceedings of the VLDB Endowment 16, 7 (2023). [7] Raul Castro Fernandez, Ziawasch Abedjan, Samuel Madden, and Michael Stonebraker. 2018. Aurum: A Data Discovery System. In 2018 IEEE 34th International Conference on Data Engineering. 1001–1012. doi:10.1109/ICDE.2018.00094 [8] Thibault Formal, Carlos Lassance, Benjamin Piwowarski, and Stéphane Clinchant. 2022. From Distillation to Hard Negative Sampling: Making Sparse Neural IR Models More Effective. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval. Association for Computing Machinery, 2353–2359. doi:10.1145/3477495.3531857 [9] Mark Green, Maura Halstead, Caroline Jay, Richard Kingston, Alex Singleton, and David Topping. 2026. Comparing How Large Language Models Perform against Keyword-Based Searches for Social Science Research Data Discovery. arXiv preprint arXiv:2601.19559 (2026). doi:10.48550/arXiv.2601.19559 [10] Health and Retirement Study. 2026. HRS Bibliography. Accessed June 2026. https://hrs.isr.umich.edu/publications/biblio/ [11] Mayank Kothyari, Dhruva Dhingra, Sunita Sarawagi, and Soumen Chakrabarti. 2023. CRUSH4SQL: Collective Retrieval Using Schema Hallucination for Text2SQL. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. 14054–14066. doi:10.18653/v1/2023.emnlp-main.868 [12] Emiko O. Kranz, Jemar R. Bather, Xiaoyan Zhang, Virginia W. Chang, Steven W. Cole, and Adolfo G. Cuevas. 2026. Discrimination Exposure and Lymphocyte Differentiation: Results from the Health and Retirement Study. Brain, Behavior, & Immunity - Health 52 (2026), 101170. doi:10.1016/j.bbih.2026.101170 [13] Haoyang Li, Jing Zhang, Cuiping Li, and Hong Chen. 2023. RESDSQL: Decoupling Schema Linking and Skeleton Parsing for Text-to-SQL. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 37. 13067–13075. doi:10.1609/aaai.v37i11. 26535 [14] Xiao Liu, Hao Yu, Hanchen Zhang, et al. 2024. AgentBench: Evaluating LLMs as Agents. In International Conference on Learning Representations.

[15] Jalal Mahmud and Eser Kandogan. 2026. Exploring the Semantic Gap in Agentic Data Systems: A Formative Study of Operationalization Failures in Analytical Workflows. arXiv preprint arXiv:2607.00828 (2026). doi:10.48550/arXiv.2607.00828 [16] Bodhisattwa Prasad Majumder, Harshit Surana, Dhruv Agarwal, Bhavana Dalvi Mishra, Abhijeetsingh Meena, Aryan Prakhar, Tirth Vora, Tushar Khot, Ashish Sabharwal, and Peter Clark. 2025. DiscoveryBench: Towards Data-Driven Discovery with Large Language Models. In International Conference on Learning Representations. [17] Md Mahadi Hasan Nahid, Davood Rafiei, Weiwei Zhang, and Yong Zhang. 2026. Rethinking Schema Linking: A Context-Aware Bidirectional Retrieval Approach for Text-to-SQL. In Findings of the Association for Computational Linguistics: EACL 2026. Association for Computational Linguistics, 4516–4546. doi:10.18653/ v1/2026.findings-eacl.236 [18] OpenAI. 2023. GPT-4 Technical Report. arXiv preprint arXiv:2303.08774 (2023). doi:10.48550/arXiv.2303.08774 [19] Stephen Robertson and Hugo Zaragoza. 2009. The Probabilistic Relevance Framework: BM25 and Beyond. Foundations and Trends in Information Retrieval 3, 4 (2009), 333–389. doi:10.1561/1500000019 [20] Mara Rosenberg, Irena Cenzer, Alexander K. Smith, and Ashwin A. Kotwal. 2026. End-of-Life Loneliness, Social Isolation, and Symptom Burden: A NationallyRepresentative Study. Journal of the American Geriatrics Society (2026). doi:10. 1111/jgs.70398 [21] Gerard Salton and Christopher Buckley. 1988. Term-Weighting Approaches in Automatic Text Retrieval. Information Processing & Management 24, 5 (1988), 513–523. doi:10.1016/0306-4573(88)90021-0 [22] Amanda Sonnega, Jessica D. Faul, Mary Beth Ofstedal, Kenneth M. Langa, John W. R. Phillips, and David R. Weir. 2014. Cohort Profile: the Health and Retirement Study (HRS). International Journal of Epidemiology 43, 2 (2014), 576–585. doi:10. 1093/ije/dyu067 [23] Chen Su, Sen Zhang, Qiandan Zheng, Jie Miao, and Junhong Guo. 2025. SelfPerceptions of Aging and Sarcopenia in Older Adults: The Mediating Role of IADL. Frontiers in Medicine 12 (2025), 1693158. doi:10.3389/fmed.2025.1693158 [24] Sina Taslimi, Artemis Capari, Hosein Azarbonyad, Zi Long Zhu, Zubair Afzal, Evangelos Kanoulas, and George Tsatsaronis. 2025. Extracting, Detecting, and Generating Research Questions for Scientific Articles. In Proceedings of the 31st International Conference on Computational Linguistics. Association for Computational Linguistics, 8573–8588. [25] Tornike Tsereteli, Yavuz Selim Kartal, Simone Paolo Ponzetto, Andrea Zielinski, Kai Eckert, and Philipp Mayr. 2022. Overview of the SV-Ident 2022 Shared Task on Survey Variable Identification in Social Science Publications. In Proceedings of the Third Workshop on Scholarly Document Processing. doi:10.18653/v1/2022.sdp-1.29 [26] Tornike Tsereteli, Daniel Ruffinelli, and Simone Paolo Ponzetto. 2024. Enriching Social Science Research via Survey Item Linking. arXiv preprint arXiv:2412.15831 (2024). doi:10.48550/arXiv.2412.15831 [27] Yamei Tu, Olga Li, Junpeng Wang, Han-Wei Shen, Przemek Powałko, Irina Tomescu-Dubrow, Kazimierz M. Słomczyński, Spyros Blanas, and J. Craig Jenkins. 2023. SDRQuerier: A Visual Querying Framework for Cross-National Survey Data Recycling. IEEE Transactions on Visualization and Computer Graphics 29, 6 (2023), 2862–2874. doi:10.1109/TVCG.2023.3261944 [28] Vijay Viswanathan, Kiril Gashteovski, Carolin Lawrence, Tongshuang Wu, and Graham Neubig. 2023. DataFinder: Scientific Dataset Recommendation from Natural Language Descriptions. In Proceedings of ACL. 10288–10303. doi:10. 18653/v1/2023.acl-long.573 [29] Bailin Wang, Richard Shin, Xiaodong Liu, Oleksandr Polozov, and Matthew Richardson. 2020. RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parsers. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 6778–6788. doi:10.18653/v1/2020.acl-main.677 [30] Yihan Wang, Peiyu Liu, and Xin Yang. 2025. LinkAlign: Scalable Schema Linking for Real-World Large-Scale Multi-Database Text-to-SQL. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 977–991. doi:10.18653/v1/2025.emnlp-main.51 [31] Ziyang Wang, Yuanlei Zheng, Zhenbiao Cao, Xiaojin Zhang, Zhongyu Wei, Pei Fu, Zhenbo Luo, Wei Chen, and Xiang Bai. 2026. AutoLink: Autonomous Schema Exploration and Expansion for Scalable Schema Linking in Text-to-SQL at Scale. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 40. 33809–33817. doi:10.1609/aaai.v40i40.40672 [32] Shitao Xiao, Zheng Liu, Peitian Zhang, Niklas Muennighoff, Defu Lian, and Jian-Yun Nie. 2024. C-Pack: Packed Resources for General Chinese Embeddings. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval. Association for Computing Machinery, 641–649. doi:10.1145/3626772.3657878 [33] Yi Yang, Hanyu Duan, Jiaxin Liu, and Kar Yan Tam. 2024. LLM-Measure: Generating Valid, Consistent, and Reproducible Text-Based Measures for Social Science Research. arXiv preprint arXiv:2409.12722 (2024). doi:10.48550/arXiv.2409.12722

Related documents

• MSET 8103 - Data science concepts#938842
• D1.1 Data and tools foundation: Inventory, Integration, and Identification Expertise#418006
• JET: Judge-Guided Evolution at Test Time for Agent Programs#1119199
• JET: Judge-Guided Evolution at Test Time for Agent Programs#1108842
• Navigating data sharing in research.#275982