arXiv CS open access

Polaris: Learning to Generate Table Descriptions from Retrieval Feedback

2026 · arxiv_cs

arXiv CS · Papers · 2026

Conceptio Open Archive

Analyze & explore related research in Conceptio

Connect this record directly into your research workflow or AI agent. Explore citations across open repositories, query full text, and export structured proof bundles.

Explore in Search →Connect via MCP / API ↗

Polaris: Learning to Generate Table Descriptions from Retrieval Feedback Ting Cai, Tuan Minh Phan, AnHai Doan University of Wisconsin-Madison {tingcai, minhrua, anhai}@cs.wisc.edu

ACTEMPS(EID, FN, LN, SAL)

arXiv:2608.17171v1 [cs.CL] 17 Aug 2026

Abstract

This table stores the records of currently active employees, where each row gives an employee's first name, last name, and salary. It supports HR and payroll analyses such as staff compensation, wage distribution, and headcount reporting.

Many table-centric NLP tasks such as NL2SQL first retrieve relevant tables from large collections using keyword search. Recent work uses LLMs to generate natural-language table descriptions to improve retrieval, but they are typically optimized for fluency rather than retrieval effectiveness. We present Polaris, a system that trains an LLM to generate table descriptions directly from retrieval feedback. Our key insight is that existing table retrieval benchmarks already contain the supervision needed for this task: given query–table relevance judgments, we generate multiple candidate descriptions for each table, rank them by their BM25 retrieval effectiveness, and use the resulting preference pairs to fine-tune the LLM with Direct Preference Optimization (DPO). Polaris further expands abbreviated table and column names before generation to reduce vocabulary mismatch. Extensive experiments show that Polaris outperforms the state-of-the-art AutoDDG solution, often by a significant margin. More broadly, our results demonstrate that retrieval benchmarks can be repurposed as supervision for training LLMs to generate retrieval-oriented metadata.

Figure 1: A tiny example of table description.

these tables to find a relatively small set of relevant tables, before we can perform task-specific reasoning. Consider for example NL2SQL. Given a natural language query such as “show quarterly sales for 2025”, many NL2SQL solutions perform KWS to retrieve the relevant tables, then generate a SQL query over these tables (Liu et al., 2025). While important, KWS over tables is difficult because the table and column names are often cryptic and short (e.g., “DTPh”, “ActEmployees”), making it hard to search on. In response, many recent works have proposed using LLMs to generate table descriptions in natural language, then performing KWS over these descriptions to find relevant tables (see Figure 1) (Zhang et al., 2026; Balaka et al., 2025; Singh et al., 2025). As far as we can tell, the most advanced work in this direction is AutoDDG, which is shown empirically to outperform six other solutions (Zhang et al., 2026). While promising, the AutoDDG work has several major limitations, as we discuss below. In this paper we describe Polaris, which addresses these limitations and significantly advances the state of the art. First, AutoDDG employs a relatively complex pipeline with multiple LLM calls to analyze a table’s metadata (e.g., table name, column names) and tuples before generating its description. We show that a much simpler pipeline, requiring only a single LLM call, achieves comparable or even better performance, and we explain why. Second, AutoDDG uses the table names and column names “as is”. In practice, these names are

Introduction

Tabular data is ubiquitous in companies, sciences, government agencies, and others (Zhang et al., 2023; Hanson, 2025). As a result, the emerging direction of NLP tasks for tabular data has grown significantly in the past ten years. Examples of such tasks include NL2SQL, table QA, schemabased relation detection, and more (Liu et al., 2025; Zhou et al., 2026; Koutras et al., 2021). Many of these tasks rely on a fundamental primitive, which is keyword search over tables. This is because the set of tables under consideration is often very large in practice. So given the user intent, often expressed in natural language or keywords, we must first perform keyword search (KWS) over 1

often cryptic or abbreviated, e.g., DTPh for “daytime phone” (Cai et al., 2025). Such names give the LLM little to work with, producing generic descriptions that miss important vocabulary. We show that by expanding these names into full English phrases, using the solution in (Cai et al., 2025), we can provide richer input to the LLM and can substantially improve the quality of the generated descriptions. Third, AutoDDG uses the LLM “as is”, without fine tuning. We show how to fine tune the LLM to produce better table descriptions for keyword search. Our approach is inspired by reinforcement learning with human feedback (RLHF) (Ziegler et al., 2019), in which a provider such as ChatGPT asks the user to rank alternative answers to the same question, uses the rankings to learn a reward function, then uses the reward function in a reinforcement learning process to fine tune the LLM to produce better answers. We adapt this paradigm in two ways. First, we do not have access to many human users, and even if we do, they typically cannot tell which table description is better for KWS. So we use labeled data, in which given a set of tables and a set of keyword queries, we know which tables are relevant to which queries. Several such labeled datasets already exist, and are widely used to train and evaluate KWS solutions (see the experiment section). We show how to use these labeled datasets to rank table descriptions. Then instead of RLHF, we use the rankings to fine tune the LLM via direct preference optimization (DPO), a recent popular technique (Rafailov et al., 2023). The fourth limitation is that AutoDDG uses only the generated table descriptions in KWS over tables. We show that, while useful, these descriptions are not perfect, so using them together with other table metadata (e.g., table/column name expansions, contexts, etc.) can significantly improve KWS accuracy. Finally, the AutoDDG work uses only two labeled datasets for evaluation. We have cleaned 3 major labeled datasets that have been used in research on KWS over tables, and created 3 new labeled datasets. This produces a benchmark of 6 datasets consisting of (table, query, relevance score) triplets. As far as we can tell, this is the largest benchmark to date for evaluating KWS over tables. In summary, we make the following contributions:

cantly improves upon AutoDDG. Polaris uses a simpler pipeline to generate table descriptions, expands cryptic table and column names into full English phrases, uses labeled data to fine tune LLMs to produce better descriptions, and exploits these descriptions as well as other table metadata for KWS. • We develop a benchmark of six labeled datasets, the largest benchmark to date for evaluating KWS solutions over tables. • Extensive experiments show that Polaris outperforms AutoDDG on the above benchmark, often by a large margin (e.g., up to 44.6% increase of NDCG@100). Further, Polaris generates much shorter table descriptions than AutoDDG, reducing the average description length by 47-59%. More broadly, our results suggest that retrieval benchmarks can be reused as supervision for training LLMs to generate retrievaloriented metadata. The code is available at github.com/anhaidgroup/polaris, and the datasets at hf.co/collections/anhaidgroup/polaris-v1.

Preliminaries

In this section we describe AutoDDG then build on that to describe the problem setting of Polaris. AutoDDG: This solution uses a complex multistep pipeline to generate table descriptions (Zhang et al., 2026). For a particular table T , the pipeline takes as input the table name and column names, additional metadata if any, sample tuple values (if available). It generates table statistics, table topics, and column summaries, then uses these to generate a table description d(T ) geared toward KWS. This pipeline makes at least four LLM calls, often many more. Given a keyword query Q, AutoDDG performs KWS to return a ranked list of tables. Here each table T is ranked based on a similarity score between its description d(T ) and query Q. This similarity score is the BM25 TF/IDF score widely used in today’s KWS engines (Robertson and Zaragoza, 2009). The paper (Zhang et al., 2026) shows that AutoDDG outperforms six solutions, including two that are pre-trained data-to-text models (Tang et al., 2023; Zhao et al., 2022), and Pneuma (Balaka et al., 2025), which uses LLMs to summarize the schema and sampled rows for RAG-style table discovery.

• We develop Polaris, a solution that signifi2

Tables T

Table Context (optional), Untrained LLM M

Labeled Table-Query Pairs

Table/Column Name Expansion

Initial Description Generation

KWS-based Ranking

DPO Training

Expanded Table/Column Names

Candidate Descriptions

Preference Pairs

Trained LLM M

Trained LLM M Tables N

Table/Column Name Expansion

Trained Description Generation

Descriptions F

Figure 2: The overall architecture of Polaris.

Table Description Generation in Polaris: Similar to AutoDDG, to generate a description d(T ) for a table T , we also use its metadata: table name, column names, and additional context if any (e.g., table caption, Web page title). Unlike AutoDDG, however, we do not use the tuples of T in description generation. Such tuples are often not available, for privacy and legal reasons. Furthermore, we show later that even without using the tuples Polaris already outperforms AutoDDG with tuples (see Section 4). As discussed in the introduction, another distinguishing aspect of Polaris is that we use labeled datasets, each of which consists of tables, queries, and relevance scores, to fine tune the LLM to produce better descriptions for KWS (see Section 3).

salary”). An untrained LLM then uses these expansions and table contexts (if any) to generate for each table multiple candidate descriptions. Next, we use labeled datasets to rank these descriptions (regarding their quality for KWS over tables). Finally, we use these rankings to train, i.e., fine tune the LLM, using direct preference optimization (DPO). In the second stage (see the bottom part of Figure 2), given a corpus of tables N (over which we want to enable KWS), we expand the table/column names. The trained LLM then uses these expansions and table contexts (if any) to generate for each table a description. We now describe the main components of Polaris in detail. 3.1

We assume Polaris has access to a set of labeled datasets D = {D1 , . . . , Dn }. Each dataset consists of (table, query, relevance score) triplets. Higher score indicates more relevance, and a score of 0 indicates no relevance. Several such labeled datasets already exist and have been widely used in research on KWS over tables. Let T be the set of all tables in D. Later we will generate alternative descriptions for these tables, then rank them. To generate good descriptions, here we will expand the table/column names in T into English phrases (e.g., “DTPh” to “daytime phone”, “russ_ind” to “russell index”). This is because the original names are often short and cryptic. So when fed these names, the LLM often produces vague descriptions, hurting KWS. To expand the table/column names, we use Columbo (Cai et al., 2025), a state-of-the-art solution that uses an LLM on the original names. Appendix A.1 describes how we adapt Columbo to the Polaris context.

Keyword Search in Polaris: We adopt the same KWS setting as AutoDDG. Specifically, given a keyword query Q, we return a ranked list of tables, where each table T is ranked based on the popular BM25 TF/IDF score between its description d(T ) and query Q. Considering other similarity scores (e.g., embedding vectors, hybrid (Karpukhin et al., 2020; Bruch et al., 2023; Formal et al., 2021)) is left for future work. Since the generated table descriptions are not perfect (see Section 4), unlike AutoDDG, we will also consider a KWS setting where the BM25 score uses both the descriptions and other table metadata. We show that this new score can significantly improve KWS accuracy.

Table/Column Name Expansion

The Polaris Solution

We now describe how Polaris generates table descriptions in two stages: fine tuning an LLM and using the LLM to generate descriptions. In the first stage (see the top part of Figure 2), given a set of tables (used for training, as we discuss later), we expand the table/column names into full English phrases (e.g., “eSal” into “employee

3.2

Initial Table Description Generation

Next, we generate descriptions for the tables in the labeled datasets. For each table T , we send a 3

corpora C1 and C2 , as shown in Figure 3. These corpora are identical, except that table T1 uses different descriptions: d11 in C1 , and d12 in C2 . All other tables are non-gold, each of them uses the single description that we have generated. Suppose performing query Q1 on corpus C1 and corpus C2 produces the ranking R1 = ⟨T2 , T1 , T3 ⟩ and R2 = ⟨T2 , T3 , T1 ⟩, respectively (see Figure 3). Then T1 is ranked higher in R1 compared to R2 , suggesting that in this case description d11 is better than description d12 for table T1 .

Corpus C1 Q1 Dataset S1 T1 T2 T3 Q1: T1

T1 → d11, d12 T2 → d2 T3 → d3

T1 (d11) T2 (d2) T3 (d3)

T2 T1 T3

Corpus C2 Q1 T1 (d12) T2 (d2) T3 (d3)

d11 > d12 for T1

T2 T3 T1

Figure 3: Using a labeled dataset to generate rankings.

We realize the above idea as follows. First, for each gold table we combine all of its corpora into one corpus. For example, for gold table T1 in Figure 3, we combine the two corpora C1 and C2 into a single corpus C12 = {T1 (d11 ), T1 (d12 ), T2 (d2 ), T3 (d3 )}. This corpus has 2 copies for table T1 , each with a different description, and 1 copy for each remaining table, each with a randomly selected description. Second, we score the two descriptions of the gold table and rank them based on their scores. Continuing with the above example, the score of description d11 (for table T1 ) is the BM25 score between query Q1 (for which T1 is a gold answer) and d11 . If T1 is a gold answer for multiple queries, then the score for d11 is the average of its scores for these queries. We compute the score of description d12 in a similar fashion. Suppose the score of d11 is greater than that of d12 . Intuitively this means that for the queries for which T1 is a correct answer, using description d11 will place T1 higher in the ranked list of tables, compared to using description d12 . As a result, we rank d11 higher, denoted as d11 > d12 . Finally, we generate 3 descriptions for each gold table, not 2. For example, for gold table T1 , we generate d11 , d12 , d13 . Now suppose we can rank d13 > d11 > d12 , as previously discussed. Then we keep only the ranking that we judge to be the most reliable, namely d13 > d12 , to be used in fine tuning the LLM. (Thus, we do not use the other two rankings d13 > d11 , d11 > d12 .) Algorithm 1 provides the pseudo code of our complete solution. As discussed earlier, this solution uses one “unified” corpus for each gold table, not multiple “individual” corpora. Using multiple corpora incurs a significant cost in creating the corpora and indexing them for querying. So we approximate this process with just one corpus. Since this “unified” corpus is identical to each “individ-

prompt to an LLM M (with no fine tuning). The prompt provides M with the table/column name expansions, the context (if any), and an one-shot example adapted from (Zhang et al., 2026). Inspired by the descriptions in AutoDDG, we ask the LLM to produce similar descriptions, which are JSON objects with six fields: data set overview, key themes/topics, applications/use cases, concepts/synonyms, keywords/themes, and additional context. See Appendix A.2 for the prompt and a sample description. Recall that each labeled dataset consists of (table, query, score) triplets. If a table has a relevance score greater than 0, we call it a gold table, because it is a “gold” answer to at least one query. For each gold table we generate k ≥ 2 table descriptions, by sampling with non-zero temperature and top-p sampling, so that the LLM produces diverse outputs from the same prompt. For each remaining (non-gold) table, we generate a single description. The reason why we treat these two kinds of tables differently will be clear in the next subsection. 3.3

Ranking the Table Descriptions

We now rank the table descriptions. Human users typically cannot tell which description is better for a table KWS-wise. So our key idea is to use the labeled datasets to rank. The following example illustrates this idea. Example 1 Consider the labeled dataset S1 in Figure 3, which has 3 tables T1 , T2 , T3 and 1 query Q1 . Suppose T1 is the only “gold” answer to Q1 (i.e., having a relevance score greater than 0). Suppose that we have used an LLM to generate 2 descriptions d11 , d12 for the gold table T1 and 1 description each for the non-gold tables T2 and T3 (see Figure 3). Then our goal is to rank descriptions d11 and d12 , i.e., deciding which one is better KWS-wise for the gold table T1 . To do so, we construct two 4

Algorithm 1 The training procedure of Polaris.

3.4

Input:

We now use the rankings of the descriptions to fine tune the base LLM via DPO. Recall that in the previous steps, for each gold table T , we have generated and ranked 3 descriptions dπ1 > dπ2 > dπ3 . This yields the following preference pair: prompt(T ), dwin = dπ1 , dlose = dπ3 ,

D — labeled datasets, each D = (T , Q, R): T — tables Q — keyword queries R : Q×T → {0, 1, . . . } — relevance judgments (0 = not relevant) M — untrained LLM K — number of candidate descriptions per table Output: M∗ — trained LLM 1: P ← ∅ ▷ preference pairs 2: For each dataset D ∈ D do 3: Expand table and column names in D 4: G ← tables with a relevant query (∃q : R(q, T ) > 0) 5: C0 ← ∅ ▷ default corpus 6: For each table T ∈ T do 7: If T ∈ G then 8: Generate K candidate descriptions CT with M 9: Add a random description from CT to C0 10: Else 11: Generate one description with M 12: Add it to C0 13: For each table T ∈ G do 14: C ← C0 ∪ C T ▷ table T ’s corpus 15: For each candidate description c ∈ CT do 16: s(c) ← mean BM25C (q, c) over relevant queries 17: cbest ← arg maxc∈CT s(c) ▷ best description 18: cworst ← arg minc∈CT s(c) ▷ worst description 19: Add (prompt(T ), cbest , cworst ) to P 20: M∗ ← M trained with DPO on P 21: Return M∗

DPO Training with the Rankings

where prompt(T ) is the prompt used for generating the description of table T . Let P be the set of all preference pairs, one per gold table (from all labeled datasets). We then fine tune the base LLM M using direct preference optimization (DPO) (Rafailov et al., 2023), which trains the LLM to increase the likelihood of dwin relative to dlose without a separate reward model. The win |x) DPO loss is LDPO = −E[log σ(β log ππrefθ (d (dwin |x) − lose |x) β log ππrefθ (d (dlose |x) )], where πθ is the model being trained, πref is the frozen base model, x is the prompt, and β controls regularization strength. We use Llama 3.1 8B Instruct as the base model, with 4-bit quantization and LoRA (Hu et al., 2022) for parameter-efficient fine-tuning. The adapter targets all attention and feed-forward projections (q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, and down_proj), with rank r = 64 and α = 64. This gives 167M trainable parameters out of 8.2B total (2.05%). We train for one epoch with β = 0.1, learning rate 5 × 10−6 , a linear schedule with 10% warmup, effective batch size 8, and the AdamW 8-bit optimizer. See Appendix A.4 for details.

ual” corpus, except that it has 2 extra copies for the target gold table, we found the approximation to produce negligible loss in BM25 scores. Further, our solution assigns a random description to each non-target table in the corpus (e.g., assigning d2 to T2 and d3 to T3 in Figure 3). Ideally, we should run this solution multiple times. Each time we assign the best description (i.e., the one with top rank) found so far to each non-target table in the corpus.

3.5

Description Generation for New Tables

Once fine tuned, the LLM M is ready to use. Given a new corpus of tables on which we want to perform KWS, for each table in the corpus, we first expand the table/column names, then feed these together with other metadata into the LLM M to generate a table description, using the same prompt described in Section 3.2.

For example, suppose we have determined d11 > d12 for T1 (see Figure 3). Now suppose that T2 is also a gold table. Then when creating a corpus for T2 , we should assign description d11 to T1 . We should repeat this process until a convergence condition. Doing this however is very time consuming. As a result, currently we run the above solution only once for each gold table (in which all remaining tables in the same corpus are assigned random descriptions). The experiments show that this is already effective in generating good descriptions for KWS. Exploring running the above solution multiple times is ongoing work.

Experiments

We now evaluate Polaris, using the six labeled datasets described in Table 1. We created the first three datasets, while the last three have been used in prior research on KWS over tables. Appendix A.5 describes these datasets, as well as the LLM, DPO, and KWS settings of our experiments. We perform each experiment in a leave-one-out fashion. For example, for the AW dataset, Polaris 5

Dataset AW Arctic LTER ECIR WTR WikiTables

Domain

#Tables

Enterprise Science Science Government Web Web

#Queries

96 251 2,015 2,100 4,634 3,361

15 20 15 12 60 57

#Gold

Dataset

Method

@10

@20

@50

@100

41 121 374 1,411 2,190 845

AutoDDG Polaris

.49 .66

.64 .77

.76 .83

.83 .83

.85 .83

Arctic

AutoDDG Polaris

.24 .29

.34 .41

.48 .58

.68 .77

.77 .82

LTER

AutoDDG Polaris

.08 .18

.14 .25

.22 .32

.32 .39

.35 .43

ECIR

AutoDDG Polaris

.01 .01

.02 .03

.04 .04

.08 .08

.12 .13

WTR

AutoDDG Polaris

.08 .08

.14 .15

.25 .26

.43 .48

.59 .66

WikiTables

AutoDDG Polaris

.21 .21

.29 .32

.41 .42

.66 .67

.83 .81

Table 1: Datasets for our experiments. #Gold is the number of gold tables. Dataset

Method

@10

@20

@50

@100

AutoDDG Polaris

.5551 .7312

.6192 .7582

.6692 .7869

.6969 .7869

.7030 .7869

Arctic

AutoDDG Polaris

.6274 .7133

.5942 .6743

.5942 .6939

.6360 .7416

.6812 .7678

LTER

AutoDDG Polaris

.4372 .7218

.4194 .6546

.4011 .6013

.3656 .5239

.3559 .5145

ECIR

AutoDDG Polaris

.4215 .4946

.4429 .5050

.4512 .4981

.4408 .4873

.4180 .4672

WTR

AutoDDG Polaris

.3748 .3840

.3548 .3840

.3767 .3894

.4159 .4449

.4884 .5309

WikiTables

AutoDDG Polaris

.4139 .4111

.4085 .4215

.4398 .4407

.5176 .5171

.5811 .5727

Table 3: Recall@K of AutoDDG vs. Polaris.

table/column name expansion, by modifying the prompt (see Appendix A.2) to use only the original table/column names, then use this prompt on the base LLM (without fine tuning) to generate table descriptions. The results show that the “disabled” Polaris still substantially outperforms AutoDDG on 4 datasets (AW, LTER, Arctic, ECIR), and performs similarly on 2 datasets (WTR, WikiTables). See Tables 9-10 in Appendix A.7. This suggests our simpler prompt can already perform better or competitive with AutoDDG. This is likely because of the multi-step “lossy” nature of AutoDDG’s prompt. Specifically, AutoDDG (1) uses the LLM to generate column summaries, table topics, etc., then (2) feeds these into the LLM again to generate a table description. Step 1 is “lossy” due to the imperfection of the LLM and the length limitations of the summaries/topics. So the description generated from the output of Step 1 is not as accurate as it can be. Effect of DPO Training: Tables 11-12 in Appendix A.7 compare the full Polaris with “P-DPO”, the Polaris version without DPO training (i.e., base LLM with name expansion). The tables show that DPO training improves AW (+9.4% at @5), LTER (+10.4% at @5), and WTR (+5.0% at @5). The exception is ECIR, where accuracy decreases (e.g., .495 vs. .570 at @5). This is probably due to the very high density of relevant tables per query: almost any reasonable description retrieves a large fraction of relevant tables, so the preference signal from DPO adds little in this low-discrimination regime. Effect of Table/Column Name Expansion: Tables 11-12 in Appendix A.7 compare the full Polaris with “P-exp”, the Polaris version without name expansion (i.e., using DPO-trained LLM,

Table 2: NDCG@K of AutoDDG vs. Polaris

fine tunes the LLM using the remaining five labeled datasets, uses it to generate descriptions for the tables in AW, performs KWS for the queries in AW (using the generated descriptions), and reports the aggregated accuracies over these queries (using their gold tables). We consider two accuracy measures widely used in KWS research: NDCG@K (ranking quality with graded relevance) and Recall@K (Järvelin and Kekäläinen, 2002; Manning et al., 2008). Overall Performance: Tables 2-3 compare Polaris with AutoDDG in terms of NDCG@K and Recall@K, respectively. They show that Polaris outperforms AutoDDG on the first five datasets for nearly all K-s and for both measures, often by a significant margin (e.g., NDCG@100 on LTER: 0.356 → 0.515, +44.6% relative, and NDCG@5 on AW: 0.555 → 0.731, +31.7% relative). On the last dataset, WikiTables, both methods perform similarly (e.g., AutoDDG leads at R@100 by 0.02 while Polaris leads at NDCG@10 by 0.01). Interestingly, we found that Polaris generates much shorter table descriptions than AutoDDG, reducing the average description length by 47-59% (see Appendix A.6). Ablation Studies: We now evaluate the major components of Polaris. Prompt Quality: First, we disable DPO training and 6

Multi-field Polaris

with only original table/column names). The tables show that expansion improves LTER (+21.2% at @5), AW (+5.4% at @5), Arctic (+6.6% at @5), and WTR (+8.0% at @5). The benefit for ECIR and WikiTables (which rely on external context rather than table names) exists but is small.

0.9

0.9 0.8

0.8

0.6

0.6 @5

@10 @20 @50 @100

ECIR

0.6

0.5 @5

@10 @20 @50 @100

WTR

0.6 0.5

0.5

0.3 @5

@10 @20 @50 @100

WikiTables

0.6 0.5

0.4

LTER

0.8 0.7

0.7

Sensitivity Analysis: Next we examine how Polaris’s accuracy changes regarding four major design choices: the LLM, the DPO regularization parameter β, the number of candidate descriptions per table, and the number of training epochs. For space reasons we only briefly summarize the findings here (see Appendices A.8-A.11 for detail). Regarding LLMs, we replace Llama 3.1 8B Instruct with Qwen2.5-7B-Instruct for both Polaris and AutoDDG, keeping all other settings identical. Table 13 shows that Polaris with Qwen outperforms Polaris with Llama on AW, Arctic, and WikiTables, but underperforms on LTER, while ECIR and WTR show mixed results across cut-offs. Table 15 shows that Polaris with Qwen outperforms AutoDDG with Qwen on all datasets, confirming that Polaris remains superior to AutoDDG under a different LLM. Next, we increase the DPO regularization parameter β from 0.1 (default) to 0.5, making the DPO objective more conservative (closer to the reference model). Tables 17-18 show that Polaris with β = 0.5 improves AW (the smallest dataset) but hurts LTER and has negligible effect elsewhere. The default β = 0.1 provides a better balance across datasets. Next, we increase the number of candidate descriptions per gold table from K = 3 (default) to K = 5. Even with K = 5, we still select only the best and worst performing descriptions and form one training pair per table. Tables 19-20 show that the new Polaris substantially improves AW (+9.7% at @5) and WikiTables (+5.3% at @5) but hurts LTER (−11.8% at @5) and Arctic (−4.1% at @5). More candidates provide a wider range of preference pairs but may also introduce noise when the best and worst candidates are harder to distinguish. Finally, we increase training from 1 epoch (default) to 3 epochs. Tables 21-22 show that the new Polaris substantially improves Arctic (+13.3% at @5) but hurts ECIR, WTR, and WikiTables. AW shows mixed results (worse at early cut-offs, better at deep ones). Additional training helps datasets with many gold tables in training (Arctic contributes many preference pairs) but risks overfit-

Single-field Polaris

Arctic

0.4 @5

@10 @20 @50 @100

Figure 4: NDCG@K of multi-field Polaris vs. singlefield Polaris.

ting on smaller training sets, supporting our default of 1 epoch. Multi-Field KWS with Polaris: So far Polaris uses only the generated table descriptions for KWS. Since these descriptions are not perfect, we hypothesize that using more fields can improve KWS. So we create P(multi), a Polaris version that uses 3 fields: the description, the table/column name expansions (concatenated into a string), and the table context. P(multi) uses the following BM25 score BM25(q, d) = maxf ∈F BM25f (q, d) + P 0.1 f ′ ∈F \{f ∗ }BM25f ′ (q, d), where F is the set of the above 3 fields, f ∗ = arg maxf BM25f (q, d) is the best-matching field, and the second term acts as a tiebreaker (weighted by 0.1) over the remaining fields. This “best field plus tiebreaker” formulation lets the strongest signal dominate while still rewarding tables that match across multiple fields. Figure 4 shows that P(multi) outperforms Polaris using a single field on 5 datasets, often by a large margin, and is competitive on 1 dataset. In a detailed ablation study, we found that the table description remains the most critical field (removing it causes significant NDCG drop). We also found that the table/column name expansions are critical for datasets with a lot of abbreviated table/column names (e.g., AW, LTER, Arctic). Finally, we found that the table context matters for Web and government data, e.g., removing it substantially hurts WTR, where the page-level metadata provides important topical signals. See Appendix A.12 for detail.

Related Work

Metadata Generation for Tables: A growing body of work has been focusing on generating metadata for tables, such as table and dataset descriptions (Zhang et al., 2026; Balaka et al., 2025; 7

Preference Learning and DPO: Reinforcement Learning from Human Feedback (RLHF) (Ziegler et al., 2019) aligns language models with human preferences by learning a reward model and optimizing the policy via reinforcement learning. Direct Preference Optimization (DPO) (Rafailov et al., 2023) simplifies this process by directly training on preference pairs, achieving comparable performance without a reward model. DPO has since been widely applied to tasks such as dialogue alignment (Tunstall et al., 2024) and summarization, while RLAIF (Lee et al., 2024) shows that AIgenerated preference labels can effectively replace human annotations. Our work applies DPO in a different setting: preference pairs are derived automatically from keyword retrieval performance rather than human or AI judgments. The closest concurrent work is Coelho et al. (2025), who optimize query generation using retrieval feedback. Other work similarly uses retrieval signals to optimize query generation (Zhou et al., 2023; Yao et al., 2025) or document rewriting (Uzan et al., 2026). In contrast, Polaris optimizes table description generation. Preference pairs are constructed from human relevance judgments in table retrieval benchmarks, using BM25 retrieval effectiveness as the optimization signal.

Singh et al., 2025; Gan et al., 2026; Giancotti et al., 2026), column and database descriptions (Wretblad et al., 2024; Gao and Luo, 2025; Cai et al., 2026), table or column name expansion (Cai et al., 2025; Zhang et al., 2023; Luoma and Kumar, 2025; Xia et al., 2025), semantic column types (Hulsebos et al., 2019; Zhang et al., 2020; Feuer et al., 2024; Kayali et al., 2024), and spatial and temporal coverage summaries for columns (Castelo et al., 2021). This is to target a variety of downstream tasks, including KWS, NL2SQL, data catalog, and more. Among these works, the closest to Polaris is AutoDDG (Zhang et al., 2026), which also generates table descriptions. But AutoDDG (and many prior works) does not train the LLM using retrieval feedback, nor uses table/column name expansion, as Polaris does. Keyword Search over Tables: Prior Keyword Search work has largely focused on improving retrieval and ranking models. Zhang and Balog (2018) propose semantic matching between queries and tables using sparse and dense representations within a learning-to-rank framework. Chen et al. (2020b) train a BERT-based neural ranker with content selection to encode tables under BERT’s input constraints, while StruBERT (Trabelsi et al., 2022) further incorporates row and column structure into the matching model. Chen et al. (2020a) generate schema labels from table content and incorporate the labels into a ranking model to improve dataset search. In contrast, Polaris improves the table representation by generating descriptions optimized for keyword search. Our approach is complementary to these retrieval models, since the generated descriptions can serve as input to any of them. Existing benchmarks for table retrieval have been discussed in (Zhang and Balog, 2018; Bhagavatula et al., 2015; Chen et al., 2021, 2020a), providing the labeled relevance data that enables our DPO training. More recently, TARGET (Ji et al., 2025) shows that dense retrievers substantially outperform BM25 but remain highly sensitive to metadata quality, while PIPER (Terrenzi et al., 2026) addresses poor metadata by generating pseudo-queries from table contents for dense retrieval. Similar to Polaris, PIPER generates textual representations of tables, but it targets dense retrieval and does not optimize the generator using retrieval feedback.

Conclusion

We presented Polaris, a system for generating table descriptions optimized for keyword search. Our key idea is to close the loop between metadata generation and retrieval: instead of relying on human or AI judgments, we use retrieval performance on existing table retrieval benchmarks to automatically construct preference pairs and fine-tune an LLM via DPO. Combined with table and column name expansion, this produces descriptions that better match users’ search vocabulary. Experiments on six datasets spanning enterprise, scientific, government, and web domains show that Polaris consistently outperforms the state of the art while using a simpler generation pipeline. More broadly, our results suggest that retrieval benchmarks can serve not only to evaluate retrieval systems, but also to train metadata generators. We hope this idea will prove useful beyond keyword search, for example in optimizing metadata for dense retrieval, NL2SQL, table question answering, and other downstream tasks.

Limitations

References Muhammad Imam Luthfi Balaka, David Alexander, Qiming Wang, Yue Gong, Adila Krisnadhi, and Raul Castro Fernandez. 2025. Pneuma: Leveraging LLMs for tabular data representation and retrieval in an end-to-end system. Proceedings of the ACM on Management of Data, 3(3).

Polaris requires query–table relevance judgments to construct preference pairs for DPO training. While several table retrieval benchmarks already provide such labels and new ones can often be collected from user logs or relevance annotation, our approach is less applicable to domains where no retrieval supervision is available. In such settings, the untrained variant of Polaris remains applicable, although it does not benefit from retrieval-guided optimization.

Chandra Sekhar Bhagavatula, Thanapon Noraset, and Doug Downey. 2015. TabEL: Entity linking in web tables. In Proceedings of the International Semantic Web Conference (ISWC), pages 425–441. Sebastian Bruch, Siyu Gai, and Amir Ingber. 2023. An analysis of fusion functions for hybrid retrieval. ACM Transactions on Information Systems, 42(1):20:1– 20:35.

Our current training objective derives preferences solely from BM25 retrieval scores. Consequently, the learned descriptions are optimized for lexical retrieval and may not transfer to retrieval systems based primarily on dense embeddings or hybrid ranking. Extending our preference construction framework to optimize for dense or hybrid retrieval objectives is an important direction for future work.

Ting Cai, Rakesh R. Menon, Yiru Chen, Zifan Liu, Yuan Tian, Fei Wu, Anudeep Chimakurthi, Prashanthi Ramamurthy, Sunav Choudhary, Kun Qian, and Yunyao Li. 2026. TACO: Task-aware column description generation using LLMs. arXiv preprint arXiv:2606.21685. Ting Cai, Stephen Sheen, and AnHai Doan. 2025. Columbo: Expanding abbreviated column names for tabular data using large language models. In Findings of the Association for Computational Linguistics: EMNLP 2025, pages 24774–24792.

More generally, Polaris optimizes the metadata generation component rather than the retrieval model itself. As such, it is complementary to advances in retrieval architectures, including dense retrievers and learned ranking models, which could potentially benefit from improved table descriptions.

Sonia Castelo, Rémi Rampin, Aécio Santos, Aline Bessa, Fernando Chirigati, and Juliana Freire. 2021. Auctus: A dataset search engine for data discovery and augmentation. Proceedings of the VLDB Endowment, 14(12):2791–2794.

Polaris generates descriptions using only table schemas and metadata. This design targets practical settings where table contents are unavailable because of privacy, legal, or computational constraints. However, when representative table rows are available, incorporating table contents may yield richer descriptions for certain datasets. Developing methods that selectively exploit row-level data while preserving the simplicity and broad applicability of Polaris is an interesting direction for future work.

Zhiyu Chen, Haiyan Jia, Jeff Heflin, and Brian D. Davison. 2020a. Leveraging schema labels to enhance dataset search. In Proceedings of the European Conference on Information Retrieval (ECIR), pages 267– 280. Zhiyu Chen, Mohamed Trabelsi, Jeff Heflin, Yinan Xu, and Brian D. Davison. 2020b. Table search using a deep contextualized language model. In Proceedings of the ACM SIGIR Conference on Research and Development in Information Retrieval, pages 589–598. Zhiyu Chen, Shuo Zhang, and Brian D. Davison. 2021. WTR: A test collection for web table retrieval. In Proceedings of the ACM SIGIR Conference on Research and Development in Information Retrieval, pages 2514–2520.

Finally, our experiments cover six benchmark datasets spanning enterprise, scientific, government, and Web domains, and leave-one-datasetout evaluation demonstrates encouraging crossdomain generalization. Nevertheless, the benefit of retrieval-guided training varies across datasets, suggesting that the effectiveness of retrieval-based preference learning depends on characteristics of the underlying corpus. Better understanding when retrieval feedback is most informative remains an important direction for future work.

João Coelho, Bruno Martins, João Magalhães, and Chenyan Xiong. 2025. Aligning web query generation with ranking objectives via direct preference optimization. In Proceedings of the ACM SIGIR Conference on Research and Development in Information Retrieval, pages 2982–2986. Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. 2023. QLoRA: Efficient finetuning of quantized LLMs. In Advances in Neural Information Processing Systems, volume 36.

Benjamin Feuer, Yurong Liu, Chinmay Hegde, and Juliana Freire. 2024. ArcheType: A novel framework for open-source column type annotation using large language models. Proceedings of the VLDB Endowment, 17(9):2279–2292.

Christos Koutras, George Siachamis, Andra Ionescu, Kyriakos Psarakis, Jerry Brons, Marios Fragkoulis, Christoph Lofi, Angela Bonifati, and Asterios Katsifodimos. 2021. Valentine: Evaluating matching techniques for dataset discovery. In Proceedings of the IEEE International Conference on Data Engineering (ICDE), pages 468–479.

Thibault Formal, Benjamin Piwowarski, and Stéphane Clinchant. 2021. SPLADE: Sparse lexical and expansion model for first stage ranking. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR), pages 2288–2292.

Harrison Lee, Samrat Phatale, Hassan Mansoor, Thomas Mesnard, Johan Ferret, Kellie Lu, Colton Bishop, Ethan Hall, Victor Carbune, Abhinav Rastogi, and Sushant Prakash. 2024. RLAIF vs. RLHF: Scaling reinforcement learning from human feedback with AI feedback. In Proceedings of the 41st International Conference on Machine Learning (ICML), volume 235, pages 26874–26901.

Lisa-Yao Gan, Arunav Das, Johanna Walker, Klaus Diepold, and Elena Simperl. 2026. Less is more? when dataset context hurts LLM-generated dataset descriptions. arXiv preprint arXiv:2606.02334.

Xinyu Liu, Shuyu Shen, Boyan Li, Peixian Ma, Runzhi Jiang, Yuxin Zhang, Ju Fan, Guoliang Li, Nan Tang, and Yuyu Luo. 2025. A survey of text-to-SQL in the era of LLMs: Where are we, and where are we going? IEEE Transactions on Knowledge and Data Engineering, 37(10):5735–5754.

Yingqi Gao and Zhiling Luo. 2025. Automatic database description generation for text-to-SQL. arXiv preprint arXiv:2502.20657. Raffaele Giancotti, Rajna Fani, Rafi Al Attrach, Pierangelo Veltri, and Tom Pollard. 2026. DataAtlas: Automatic generation of data dictionaries using large language models. JAMIA Open, 9(3):ooag119.

Llama Team, AI @ Meta. 2024. The Llama 3 herd of models. arXiv preprint arXiv:2407.21783. Kyle Luoma and Arun Kumar. 2025. SNAILS: Schema naming assessments for improved LLM-based SQL inference. Proceedings of the ACM on Management of Data, 3(1):77:1–77:26.

Paul Hanson. 2025. Environmental data initiative. Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations.

Christopher D. Manning, Prabhakar Raghavan, and Hinrich Schütze. 2008. Introduction to Information Retrieval. Cambridge University Press.

Madelon Hulsebos, Kevin Hu, Michiel Bakker, Emanuel Zgraggen, Arvind Satyanarayan, Tim Kraska, Cagatay Demiralp, and Cesar Hidalgo. 2019. Sherlock: A deep learning approach to semantic data type detection. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.

Qwen Team. 2024. Qwen2.5 technical report. arXiv preprint arXiv:2412.15115. Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D. Manning, Stefano Ermon, and Chelsea Finn. 2023. Direct preference optimization: Your language model is secretly a reward model. In Advances in Neural Information Processing Systems, volume 36, pages 53728–53741.

Kalervo Järvelin and Jaana Kekäläinen. 2002. Cumulated gain-based evaluation of IR techniques. ACM Transactions on Information Systems, 20(4):422– 446.

Stephen Robertson and Hugo Zaragoza. 2009. The probabilistic relevance framework: BM25 and beyond. Foundations and Trends in Information Retrieval, 3(4):333–389.

Xingyu Ji, Parker Glenn, Aditya G. Parameswaran, and Madelon Hulsebos. 2025. TARGET: Benchmarking table retrieval for generative tasks. arXiv preprint arXiv:2505.11545.

Mayank Singh, Abhijeet Kumar, Sasidhar Donaparthi, and Gayatri Karambelkar. 2025. Leveraging retrieval augmented generative LLMs for automated metadata description generation to enhance data catalogs. arXiv preprint arXiv:2503.09003.

Vladimir Karpukhin, Barlas Oğuz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. Dense passage retrieval for opendomain question answering. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6769–6781.

Tianyi Tang, Junyi Li, Wayne Xin Zhao, and Ji-Rong Wen. 2023. MVP: Multi-task supervised pre-training for natural language generation. In Findings of the Association for Computational Linguistics: ACL 2023, pages 8758–8794.

Moe Kayali, Anton Lykov, Ilias Fountalis, Nikolaos Vasiloglou, Dan Olteanu, and Dan Suciu. 2024. CHORUS: Foundation models for unified data discovery and exploration. Proceedings of the VLDB Endowment, 17(8):2104–2114.

Riccardo Terrenzi, Matteo Falconi, Serkan Ayvaz, and Pierluigi Plebani. 2026. PIPER: Contentbased table search via profiling and LLM-generated

pseudoqueries. In International Conference on Database and Expert Systems Applications (DEXA). ArXiv:2605.18199.

Yilun Zhao, Linyong Nan, Zhenting Qi, Rui Zhang, and Dragomir Radev. 2022. ReasTAP: Injecting table reasoning skills during pre-training via synthetic reasoning examples. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 9006–9018.

Mohamed Trabelsi, Zhiyu Chen, Shuo Zhang, Brian D. Davison, and Jeff Heflin. 2022. StruBERT: Structureaware BERT for table search and matching. In Proceedings of the Web Conference (WWW), pages 442– 451.

Wei Zhou, Bolei Ma, Annemarie Friedrich, and Mohsen Mesgar. 2026. Table question answering in the era of large language models: A comprehensive survey of tasks, methods, and evaluation. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (ACL), pages 12161–12189.

Lewis Tunstall, Edward Beeching, Nathan Lambert, Nazneen Rajani, Kashif Rasul, Younes Belkada, Shengyi Huang, Leandro von Werra, Clémentine Fourrier, Nathan Habib, Nathan Sarrazin, Omar Sanseviero, Alexander M. Rush, and Thomas Wolf. 2024. Zephyr: Direct distillation of LM alignment. In Conference on Language Modeling (COLM).

Yujia Zhou, Zhicheng Dou, and Ji-Rong Wen. 2023. Enhancing generative retrieval with reinforcement learning from relevance feedback. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 12481– 12490.

Omri Uzan, Ron Polonsky, Douwe Kiela, and Christopher Potts. 2026. Document optimization for blackbox retrieval via reinforcement learning. arXiv preprint arXiv:2604.05087.

Daniel M. Ziegler, Nisan Stiennon, Jeffrey Wu, Tom B. Brown, Alec Radford, Dario Amodei, Paul Christiano, and Geoffrey Irving. 2019. Fine-tuning language models from human preferences. arXiv preprint arXiv:1909.08593.

Niklas Wretblad, Oskar Holmström, Erik Larsson, Axel Wiksäter, Oscar Söderlund, Hjalmar Öhman, Ture Pontén, Martin Forsberg, Martin Sörme, and Fredrik Heintz. 2024. Synthetic SQL column descriptions and their impact on text-to-SQL performance. arXiv preprint arXiv:2408.04691.

A.1

Table/Column Name Expansion

This section provides additional detail on the name expansion step (Section 3.1).

Yikuan Xia, Jiazun Chen, Sujian Li, and Jun Gao. 2025. Realistic training data generation and rule enhanced decoding in LLM for NameGuess. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6990– 7007. Sijia Yao, Pengcheng Huang, Zhenghao Liu, Yu Gu, Yukun Yan, Shi Yu, and Ge Yu. 2025. ExpandR: Teaching dense retrievers beyond queries with LLM guidance. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 19036–19054. Dan Zhang, Yoshihiko Suhara, Jinfeng Li, Madelon Hulsebos, Cagatay Demiralp, and Wang-Chiew Tan. 2020. SATO: Contextual semantic type detection in tables. Proceedings of the VLDB Endowment, 13(11):1835–1848. Haoxiang Zhang, Yurong Liu, Aécio Santos, Wei-Lun Hung, and Juliana Freire. 2026. AutoDDG: Automated dataset description generation using large language models. Proceedings of the ACM on Management of Data, 4(1):1–27. Jiani Zhang, Zhengyuan Shen, Balasubramaniam Srinivasan, Shen Wang, Huzefa Rangwala, and George Karypis. 2023. NameGuess: Column name expansion for tabular data. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 13276–13290. Shuo Zhang and Krisztian Balog. 2018. Ad hoc table retrieval using semantic similarity. In Proceedings of the Web Conference (WWW), pages 1553–1562.

(a) Table name: Address

(b) Table name: FactCallCenter

Figure 5: Example column names and expansions.

Figure 5 shows examples of original column names and their gold expansions.

as few-shot examples. The difference is in the input: each question supplies the table name together with its column names, so the model can use the columns as evidence for what the table is about (e.g., hh_id, age_grp, edu_lvl support reading Pop_census as “Population Census”). At inference we pass the already-expanded column names rather than the original ones, and include at most the first 25 of them to bound prompt length. The output is again a token-level mapping, e.g., Prchs_info: Prchs → Purchase, info → Information, concatenated into one expanded table name. Tables without a name skip this step.

Column Name Expansion: Table 4 shows the prompt for column name expansion. It has three parts. First, a system message that fixes the assistant role and asks the model to reply in the same format as the examples. Second, a task statement with eight guidelines: expand every abbreviation, including chemical symbols and units; leave numbers untouched; add nothing beyond the expansion; preserve token order; keep expansions short; commit to exactly one expansion per token rather than hedging with alternatives; and leave non-abbreviated tokens unchanged. Third, four few-shot examples (Prchs_info, Emp_info, AirQ_data, and Pop_census), each written as a question, a chain-of-thought reasoning block, and a final answer. The reasoning block first reads the table name for domain context, then splits each column name into tokens and expands them one at a time. The final answer is a token-level mapping, e.g., sal_amt: sal → Salary, amt → Amount, which we concatenate into a single expanded string per column. The target table’s name and a batch of its column names are appended in the same question format, and the model continues from [Answer]. We use the default batch size of 10 column names per prompt. Table Name Expansion: Table 5 shows the prompt for table name expansion. It reuses the system message and the eight guidelines verbatim, changing only “column names” to “table names” in the task statement, and it uses the same four tables 12

Column Name Expansion Prompt System: You are a helpful assistant, answer the question from the user and reply in the same format. User: Your task is to expand abbreviated column names into full-form phrases. You should return the tokens of each column name and their associated expansions. First reason step by step and then return your final answer. Follow the guidelines below when you expand: 1. Expand all abbreviations in the column names. 2. Expand chemical symbols and units of measure to their full names. 3. Do not expand or mutate numbers. 4. Do not add extra words or explanations. 5. Maintain the original order of tokens in the expansion. 6. The tokens should be as concise and simple as possible. 7. Only provide 1 expansion for each token, even if you are uncertain, output the most possible one. Do not put ambiguous or need context after your expansion. 8. If the token is not abbreviated, the expansion should be itself, do not paraphrase or add other words around it. [Question] As abbreviations of column names from a table named Prchs_info, c_name | pCd | dt stand for? Think step by step [Answer] ### Reasoning 1. Identify the table name context: • The table is named “Prchs_info”, which likely stands for “Purchase Information”. • This suggests that the column names are related to purchase details. 2. Break down each column name into tokens and infer meanings: • c_name → c, name – c is commonly used for Customer in business contexts. – name clearly refers to Name. – Together, c_name most likely means Customer Name. • pCd → p, Cd – p is frequently used for Product in sales or inventory tables. – Cd is a common abbreviation for Code. – Together, pCd most likely means Product Code. • dt → dt – dt is a standard abbreviation for Date. – In the context of a purchase table, it likely refers to Purchase Date. ### Final Answer c_name: c → Customer, name → Name pCd: p → Product, Cd → Code dt: dt → Date (3 more few-shot examples omitted for brevity) [Question] As abbreviations of column names from a table named {table_name}, {column_name_1} | {column_name_2} | . . . | {column_name_k} stand for [Answer]

Table 4: Column name expansion prompt. The prompt uses four few-shot examples (first shown in full above; three omitted for brevity) followed by the target input. Input placeholders are shown in red.

Table Name Expansion Prompt System: You are a helpful assistant, answer the question from the user and reply in the same format. User: Your task is to expand abbreviated table names into full-form phrases. You should return the tokens of each table name and their associated expansions. First reason step by step and then return your final answer. Follow the guidelines below when you expand: 1. Expand all abbreviations in the table names. 2. Expand chemical symbols and units of measure to their full names. 3. Do not expand or mutate numbers. 4. Do not add extra words or explanations. 5. Maintain the original order of tokens in the expansion. 6. The tokens should be as concise and simple as possible. 7. Only provide 1 expansion for each token, even if you are uncertain, output the most possible one. Do not put ambiguous or need context after your expansion. 8. If the token is not abbreviated, the expansion should be itself, do not paraphrase or add other words around it. [Question] Given the following table information: Table name: Prchs_info Column names: c_name | pCd | dt What is the full form of the table name Prchs_info? Think step by step [Answer] ### Reasoning 1. Understand the context from the table name: • The table name is “Prchs_info”. • The suffix _info clearly suggests “Information”. • The prefix “Prchs” is a shortened form, likely representing “Purchase”. 2. Tokenize the table name: • Tokens: Prchs, info • Prchs → common abbreviation for Purchase. • info → abbreviation for Information. ### Final Answer Prchs_info: Prchs → Purchase, info → Information (3 more few-shot examples omitted for brevity) [Question] Given the following table information: Table name: {table_name} Column names: {expanded_column_name_1} | {expanded_column_name_2} | . . . | {expanded_column_name_k} What is the full form of the table name {table_name}? Think step by step [Answer]

Table 5: Table name expansion prompt. The prompt uses four few-shot examples (first shown in full above; three omitted for brevity) with expanded column names as additional context. Input placeholders are shown in red.

A.2

Initial Table Description Generation Description Generation: Table 6 shows the description generation prompt.

This section provides additional detail on the initial table description generation step: the prompt used to generate descriptions, a sample generated description, and the LLM and decoding configuration used at this step. Description Generation Prompt

System: You are a careful, concise data documentation assistant. You ONLY use the provided table information, which may include table name, column names and table context, to infer high-level information. If something is not obvious from the schema, say ‘unknown’ instead of guessing. You do NOT need to restate the table name, column names, or table context in the JSON. Your output MUST be valid JSON, wrapped in triple backticks with the language tag json. Do NOT include any text outside the JSON code fence. Follow the schema exactly—no extra keys. JSON schema (instructions, not to be echoed verbatim): {"dataset_overview": str, "key_themes_or_topics": [str], "applications_and_use_cases": [str], "concepts_and_synonyms": [{"concept": str, "synonyms": [str]}], "keywords_and_themes": [str], "additional_context": [str]} Formatting rules: 1. Return only a JSON object inside a “‘json code fence. 2. Include no explanations, headers, or markdown outside the code fence. 3. Maintain the key order in the schema if possible. 4. For concepts_and_synonyms, include 3–6 items, each with 1–4 synonyms. User (one-shot example): Given the table information below: Table Name: insurers Column Names: company_name, year, insurer_type, total_assets, liabilities, premiums_written Produce documentation following the JSON schema in the system message. Keep it concise and truthful. If unsure, use ‘unknown’. Assistant (one-shot example): “‘json {"dataset_overview": "Financial attributes of health insurers by company and year, including insurer type, assets, liabilities, and written premiums.", "key_themes_or_topics": ["Health insurance finance", "Insurer types", "Financial metrics"], "applications_and_use_cases": ["Evaluate insurer financial health", "Benchmark insurer types", "Analyze market trends"], "concepts_and_synonyms": [{"concept": "Premiums Written", "synonyms": ["Premium Income", "Gross Written Premium"]}, {"concept": "Liabilities", "synonyms": ["Financial Obligations", "Reserves"]}, {"concept": "Assets", "synonyms": ["Total Assets", "Asset Valuation"]}], "keywords_and_themes": ["health insurance", "premiums", "assets", "liabilities", "yearly financials"], "additional_context": ["Values are organized by company and year."]} “‘ User (actual query): Using the table information below, generate JSON that follows the schema in the system message. Write 2–5 sentences for dataset_overview. Provide specific, schema-derived items; avoid speculation. If information is unclear, use ‘unknown’. Output JSON only, wrapped in “‘json fences. Table Name: {expanded_table_name} (optional) Column Names: {expanded_col_1}, {expanded_col_2}, . . . , {expanded_col_k} Table Context: {table_context} (optional)

Table 6: Prompt for initial table-description generation. Input placeholders are shown in red.

Sample Generated Table Description { "dataset_overview": "Quarterly language-preference data for Social Security Disability Insurance claimants from fiscal year 2016 onward. The table supports analysis of how claimant language preferences change across quarters and years.", "key_themes_or_topics": ["Social Security Disability Insurance", "Language preferences", "Quarterly data"], "applications_and_use_cases": ["Analyze quarterly language trends", "Compare claimant language preferences across fiscal years"], "concepts_and_synonyms": [ {"concept": "Language Preference", "synonyms": ["Spoken Language", "Preferred Language"]}, {"concept": "Disability Insurance", "synonyms": ["SSDI", "Disability Benefits"]}, {"concept": "Quarterly Data", "synonyms": ["Periodic Data", "Time Series"]}], "keywords_and_themes": ["language preferences", "SSDI", "quarterly data", "claimant demographics", "fiscal year"], "additional_context": ["Data is organized by fiscal year and quarter."] }

Table 7: Example output of the initial description-generation stage.

Sample Description: Table 7 shows an output description produced by the prompt. The input schema contains fiscal-year, quarter, and language-preference columns for Disability Insurance claimants.

ℓ(d) = 1 − b + b BM25 then assigns

tf(t, d)(k1 + 1) , tf(t, d) + k ℓ(d) 1 t∈q N − nt + 0.5 IDF(t) = log 1 + , nt + 0.5

The Generator: We use Llama 3.1 8B Instruct as the base model. At this initial stage, the model has not yet been fine-tuned with DPO. For each table, the model receives the expanded table name, expanded column names, and table context, and returns the six-field description illustrated above.

BM25CT (q, d) =

IDF(t)

where tf(t, d) is the frequency of term t in d, |d| is the description length, avgdl is the average document length in CT , N = |CT |, and nt is the number of documents containing t. We use Elasticsearch’s BM25 defaults, k1 = 1.2 and b = 0.75. The score of candidate description dj is its BM25 score averaged over all queries for which T is relevant:

For each gold table—a table relevant to at least one training query— we sample K = 3 descriptions with temperature 0.5, top_p = 0.9, top_k = 50, and seed 42. The non-zero temperature gives the candidates enough variation to create a useful ranking signal. For every other table, we generate one description using the same model, prompt, and decoding settings.

sT (dj ) =

A.3

|d| . avgdl

X 1 BM25CT (q, dj ). |Q(T )| q∈Q(T )

Ranking the Table Descriptions

This averaging gives equal weight to every relevant query and produces one retrieval-oriented score per description. We order the candidates as dπ1 ≻ dπ2 ≻ · · · ≻ dπK , where sT (dπ1 ) ≥ sT (dπ2 ) ≥ · · · ≥ sT (dπK ). Intuitively, a higherranked description uses language that makes the correct table score more highly for its relevant queries.

We rank the K candidate descriptions of each gold table by how well they support retrieval for the queries related to that table. Let Q(T ) = {q : R(q, T ) > 0} be the set of queries for which table T is relevant. We construct a unified corpus CT containing K copies of T , one with each candidate description, and one copy of every other table. A gold table other than T is represented by one randomly selected candidate. Thus, the K copies of T differ only in their descriptions.

A.4

DPO Training with the Rankings

With K = 3, each gold table has ranked candidates dπ1 ≻ dπ2 ≻ dπ3 . We retain the most separated pair and construct one training example

For query q and candidate description d, define the BM25 length-normalization factor 16

WikiTables: Derived from a large corpus of Wikipedia tables used for entity linking (Bhagavatula et al., 2015). The original benchmark contains 60 queries and 1.6 million tables, with graded relevance on a 0–2 scale (0: irrelevant, 1: partially relevant, 2: fully relevant). As these are web tables, they lack meaningful table names; each table consists of column names and table context (page title, caption, and surrounding text). Generating descriptions for all 1.6M tables is computationally prohibitive, so we apply a BM25based downsampling strategy (see below). After downsampling and removing queries with no confirmed positive tables, we retain 3,361 tables and 57 queries.

prompt(T ), dwin = dπ1 , dlose = dπ3 . Direct Preference Optimization trains the model to increase the likelihood of the winning description relative to the losing one, without fitting a separate reward model. Its loss is

πθ (dwin | x) LDPO = −E log σ β log πref (dwin | x) πθ (dlose | x) − β log , πref (dlose | x) where x = prompt(T ), πθ is the model being trained, πref is the frozen base model, and β controls regularization. We use Llama 3.1 8B Instruct with 4-bit quantization and LoRA for parameter-efficient DPO training. The adapter targets all attention and feedforward projections (q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, and down_proj), with rank r = 64 and α = 64. This gives 167M trainable parameters out of 8.2B total (2.05%). We train for one epoch with β = 0.1, learning rate 5 × 10−6 , a linear schedule with 10% warmup, effective batch size 8, and the AdamW 8-bit optimizer. Training uses leave-one-dataset-out crossvalidation: for each held-out dataset, preference pairs from the other five datasets form the training set. Parameter

Value

Base model Quantization LoRA rank / α Trainable parameters DPO β Learning rate Epochs Effective batch size Optimizer LR schedule

Llama 3.1 8B Instruct 4-bit 64 / 64 167M / 8.2B (2.05%) 0.1 5 × 10−6 1 8 AdamW 8-bit Linear; 10% warmup

WTR: Built over English web tables from CommonCrawl (Chen et al., 2021), using the same 60 queries as WikiTables but with different table coverage. The original benchmark contains approximately 3 million tables, also with graded relevance on a 0–2 scale. Tables include page titles, captions, and surrounding text as context but no table names. We apply the same downsampling strategy, yielding 4,634 tables across 60 queries. Downsampling Strategy: Both WikiTables and WTR contain millions of tables, making it impractical to generate descriptions for all of them. We downsample each corpus deterministically using a BM25-based strategy. For each query, we retrieve the top-50 tables from the full corpus via BM25, using only the available metadata fields (column names with the trigram analyzer and table context with the word analyzer, both with equal weight); no descriptions or table names are used, as descriptions have not yet been generated at this stage and these datasets do not have table names. We then take the union of these retrieved tables with all gold-relevant tables (relevance score > 0). This strategy does not reduce task difficulty; rather, it focuses the corpus on hard cases by retaining the most competitive non-relevant tables (hard negatives that BM25 would actually surface) alongside all truly relevant tables.

Table 8: DPO training configuration.

A.5

Datasets and Experimental Settings

ECIR: Built from the U.S. government open data portal (Chen et al., 2020a). The original benchmark defines 6 search tasks, each with 20 queries, and provides task–table relevance judgments on a 0–3 scale from multiple annotators. Prior work (Zhang et al., 2026) treats all 120 queries independently. Upon careful inspection, we identify two issues:

We evaluate on six datasets spanning four domains. The datasets vary in scale (96 to 4,634 tables), domain (enterprise, scientific, government, web), and the number of relevant tables per query. Three datasets are curated from prior work and three are newly created. 17

The Generators: We use Llama 3.1 8B Instruct (Llama Team, AI @ Meta, 2024) as the base model for description generation, with 4-bit quantization (QLoRA (Dettmers et al., 2023)) for parameter-efficient DPO fine-tuning. For name expansion, we use Llama 4 Maverick 17B Instruct via AWS Bedrock with temperature 0.2, top_p 0.9, and max_tokens 4196. Table 8 summarizes the DPO training hyperparameters. The LoRA adapter targets all attention and feed-forward projections (q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj), with rank r = 64 and α = 64, resulting in 167M trainable parameters out of 8.2B total (2.05%). We train for one epoch with DPO β = 0.1, learning rate 5 × 10−6 with linear schedule and 10% warmup, effective batch size 8, and AdamW 8-bit optimizer. For each held-out dataset, we train on preference pairs from the remaining five datasets (leave-one-dataset-out cross-validation).

(1) the 20 queries within each task are highly similar and lack diversity, with some queries unrelated to the task; and (2) the relevance labels suffer from false positives—many table–task pairs labeled as highly relevant (≥ 2) are not actually relevant upon manual inspection, while pairs labeled as irrelevant (0) are generally correct. To address these issues, we manually re-label all table–query pairs originally scored as highly relevant (≥ 2), keeping labels of 0 as-is. To reduce redundancy across queries within a task, we select 2 representative and maximally distinct queries per task. We also remove tables that are not in CSV format and cannot be opened, yielding 12 queries across 2,100 parseable tables with relevance on a 0–3 scale. AW: Based on the AdventureWorks sample database (Microsoft), a well-known enterprise schema with 96 tables that use many abbreviated names (e.g., BusEntId, DTPh, STRGUID). We created 15 queries for enterprise data discovery tasks, such as “employee phone number” and “discounted products”, and manually labeled the relevance of all 96 tables for each query. The relevance score is binary (0 or 1).

Keyword Search: We use Elasticsearch with BM25 scoring (k1 = 1.2, b = 0.75). The wordlevel analyzer applies standard tokenization (splitting on whitespace and punctuation), lowercasing, and English stemming (Porter), so that query and document terms are matched after normalization. Unless otherwise noted (§A.12), we index each table as a single description field with this analyzer, isolating the effect of description quality from indexing configuration differences.

Arctic: A random sample of 251 tables from the Environmental Data Initiative (EDI), a scientific data repository storing long-term ecological and environmental research data. We created 20 queries covering topics such as “precipitation,” “species diversity,” and “nitrous oxide” and manually labeled the table–query pairs. Relevance is binary (0 or 1).

Evaluation Metrics: We report NDCG@K (ranking quality with graded relevance) and Recall@K (fraction of relevant tables retrieved in the top K) for K ∈ {5, 10, 20, 50, 100}. For Recall, all tables with relevance score > 0 are treated as relevant.

LTER: A larger sample of 2,015 tables from the top-downloaded EDI collections, representing a more focused subset of frequently used scientific datasets. We created 15 queries and manually labeled relevance as in Arctic. Relevance is binary (0 or 1). Relevance Annotation: For all three new datasets, we use a pooling-based annotation workflow: for each query, we run multiple retrieval configurations (varying BM25 field weights, tokenization strategies, and dense retrieval) and collect the union of candidate tables. A domain expert then manually verifies whether each candidate is relevant to the query, producing binary relevance labels used for both training and evaluation. This pooling approach reduces the chance of missing relevant tables that any single retrieval method would overlook.

A.6

Overall Performance

Figure 6 shows the average length of the descriptions generated by AutoDDG and Polaris.

A.7

Ablation Studies

Table 9 shows NDCG@K and Table 10 shows Recall@K for prompt quality evaluation. Table 11 reports NDCG@K and Table 12 reports Recall@K for the effect of DPO training and the effect of name expansion. 18

AutoDDG

Polaris 3,206

3,099

3,000

3,036

2,977

2,964

2,000 1,561

1,535

1,618

3,000

0 Arctic

LTER

ECIR

WTR

Dataset

Method

@10

@20

@50

@100

AutoDDG P −(exp+DPO)

.5551 .7667

.6192 .7948

.6692 .8129

.6969 .8129

.7030 .8129

Arctic

AutoDDG P −(exp+DPO)

.6274 .7041

.5942 .6651

.5942 .6700

.6360 .7008

.6812 .7226

LTER

AutoDDG P −(exp+DPO)

.4372 .6042

.4194 .5856

.4011 .5353

.3656 .4358

.3559 .4188

ECIR

AutoDDG P −(exp+DPO)

.4215 .5390

.4429 .5555

.4512 .5385

.4408 .5141

.4180 .4927

WTR

AutoDDG P −(exp+DPO)

.3748 .3693

.3548 .3688

.3767 .3684

.4159 .4335

.4884 .5155

WikiTables

AutoDDG P −(exp+DPO)

.4139 .3999

.4085 .4027

.4398 .4456

.5176 .5189

.5811 .5721

@20

@50

@100

AutoDDG P −(exp+DPO)

.49 .69

.64 .78

.76 .82

.83 .82

.85 .82

Arctic

AutoDDG P −(exp+DPO)

.24 .29

.34 .40

.48 .55

.68 .74

.77 .78

LTER

AutoDDG P −(exp+DPO)

.08 .12

.14 .18

.22 .26

.32 .33

.35 .37

ECIR

AutoDDG P −(exp+DPO)

.01 .02

.02 .02

.04 .04

.08 .09

.12 .14

WTR

AutoDDG P −(exp+DPO)

.08 .08

.14 .14

.25 .24

.43 .47

.59 .64

WikiTables

AutoDDG P −(exp+DPO)

.21 .20

.29 .28

.41 .44

.66 .68

.83 .82

Table 10: Prompt quality ablation (Recall@K): AutoDDG vs. P −(exp+DPO). P −(exp+DPO) = Polaris without DPO training and table/column name expansion. Blue = better per cell.

Table 9: Prompt quality ablation (NDCG@K): AutoDDG vs. P −(exp+DPO). P −(exp+DPO) = Polaris without DPO training and table/column name expansion. Blue = better per cell.

Sensitivity Analysis with Different LLMs

Table 13 shows NDCG@K and Table 14 shows Recall@K for the effect of replacing the underlying LLM with Qwen2.5-7B-Instruct (Qwen Team, 2024). Table 15 reports NDCG@K and Table 16 reports Recall@K for the comparison between Polaris and AutoDDG, both using Qwen2.5-7B-Instruct.

Dataset

Method

@10

@20

@50

@100

Polaris P −DPO P −exp

.7312 .6681 .6940

.7582 .7287 .7485

.7869 .7794 .7847

Arctic

Polaris P −DPO P −exp

.7133 .7209 .6690

.6743 .6785 .6493

.6939 .6891 .6737

.7416 .7334 .6872

.7678 .7535 .7106

LTER

Polaris P −DPO P −exp

.7218 .6538 .5957

.6546 .6187 .5750

.6013 .5639 .5257

.5239 .4743 .4434

.5145 .4488 .4230

ECIR

Polaris P −DPO P −exp

.4946 .5696 .4912

.5050 .5894 .4968

.4981 .5502 .4954

.4873 .5114 .5048

.4672 .4941 .4726

WTR

Polaris P −DPO P −exp

.3840 .3657 .3554

.3840 .3635 .3781

.3894 .3740 .3820

.4449 .4402 .4390

.5309 .5226 .5208

WikiTables

Polaris P −DPO P −exp

.4111 .4162 .3818

.4215 .4094 .3930

.4407 .4396 .4377

.5171 .5229 .5200

.5727 .5722 .5725

Table 11: Effect of DPO training and table/column name expansion (NDCG@K): Polaris vs. P −DPO vs. P −exp. P −DPO = Polaris without DPO training, P −exp = Polaris without name expansion. Blue = best per cell.

A.11

Sensitivity Analysis with Different DPO β

Sensitivity Analysis with Different Numbers of Training Epochs

Table 21 shows NDCG@K and Table 22 shows Recall@K for the effect of increasing DPO training from 1 epoch to 3 epochs.

Table 17 shows NDCG@K and Table 18 shows Recall@K for the effect of increasing the DPO regularization parameter β from 0.1 to 0.5.

A.10

@10

WikiTables

Figure 6: Average length of the descriptions generated by AutoDDG and Polaris on each dataset.

A.9

1,243

1,000

A.8

Method

1,458

1,288

Dataset

A.12

Multi-Field Keyword Search

Table 23 shows NDCG@K and Table 24 shows Recall@K for multi-field Polaris, compared to singlefield Polaris and AutoDDG. Table 25 shows NDCG@K for the effect of removing each of the three fields (description, table/column name expansions, and table context) from multi-field Polaris.

Sensitivity Analysis with Different Numbers of Descriptions

Table 19 shows NDCG@K and Table 20 shows Recall@K for the effect of increasing the number of candidate descriptions per gold table from 3 to 5. 19

Dataset

Method

@10

@20

@50

@100

Polaris P −DPO P −exp

.66 .61 .65

.77 .76 .79

.83 .89 .87

Arctic

Polaris P −DPO P −exp

.29 .30 .27

.41 .40 .39

.58 .56 .57

.77 .76 .72

.82 .80 .77

LTER

Polaris P −DPO P −exp

.18 .12 .13

.25 .19 .18

.32 .28 .25

.39 .36 .33

.43 .38 .36

ECIR

Polaris P −DPO P −exp

.01 .01 .01

.03 .03 .02

.04 .04 .04

.08 .09 .09

.13 .14 .13

WTR

Polaris P −DPO P −exp

.08 .08 .08

.15 .14 .15

.26 .25 .26

.48 .47 .48

.66 .65 .65

WikiTables

Polaris P −DPO P −exp

.21 .20 .21

.32 .28 .29

.42 .41 .45

.67 .68 .70

.81 .81 .86

Method

@10

@20

@50

@100

P (Qwen) Polaris

.8265 .7312

.8390 .7582

.8733 .7869

Arctic

P (Qwen) Polaris

.7303 .7133

.6952 .6743

.7071 .6939

.7425 .7416

.7690 .7678

LTER

P (Qwen) Polaris

.6269 .7218

.5964 .6546

.5393 .6013

.4439 .5239

.4259 .5145

ECIR

P (Qwen) Polaris

.5188 .4946

.5117 .5050

.4935 .4981

.4834 .4873

.4577 .4672

WTR

P (Qwen) Polaris

.4346 .3840

.3961 .3840

.3957 .3894

.4430 .4449

.5123 .5309

WikiTables

P (Qwen) Polaris

.4250 .4111

.4383 .4215

.4692 .4407

.5403 .5171

.5971 .5727

Table 13: Polaris with different LLMs (NDCG@K): P (Qwen) vs. Polaris. P (Qwen) = Polaris with Qwen2.5-7B, Polaris = Polaris with Llama 3.1 8B. Blue = better per cell. Dataset

Method

@10

@20

@50

@100

P (Qwen) Polaris

.73 .66

.81 .77

.88 .83

Arctic

P (Qwen) Polaris

.29 .29

.42 .41

.60 .58

.77 .77

.82 .82

LTER

P (Qwen) Polaris

.11 .18

.18 .25

.27 .32

.34 .39

.36 .43

ECIR

P (Qwen) Polaris

.01 .01

.02 .03

.04 .04

.08 .08

.12 .13

WTR

P (Qwen) Polaris

.09 .08

.15 .15

.25 .26

.46 .48

.61 .66

WikiTables

P (Qwen) Polaris

.19 .21

.30 .32

.45 .42

.69 .67

.84 .81

Method

@10

@20

@50

@100

A(Qwen) P (Qwen)

.4853 .8265

.5286 .8390

.5946 .8733

.6306 .8733

Arctic

A(Qwen) P (Qwen)

.6861 .7303

.6586 .6952

.6786 .7071

.7055 .7425

.7437 .7690

LTER

A(Qwen) P (Qwen)

.3379 .6269

.3269 .5964

.3275 .5393

.3380 .4439

.3440 .4259

ECIR

A(Qwen) P (Qwen)

.3660 .5188

.4175 .5117

.4200 .4935

.4033 .4834

.3989 .4577

WTR

A(Qwen) P (Qwen)

.3541 .4346

.3714 .3961

.3691 .3957

.4164 .4430

.5010 .5123

WikiTables

A(Qwen) P (Qwen)

.4414 .4250

.4493 .4383

.4718 .4692

.5479 .5403

.6036 .5971

Table 15: Polaris vs. AutoDDG under Qwen2.5-7B (NDCG@K): A(Qwen) vs. P (Qwen). A(Qwen) = AutoDDG with Qwen2.5-7B, P (Qwen) = Polaris with Qwen2.5-7B. Blue = better per cell.

Table 12: Effect of DPO training and table/column name expansion (Recall@K): Polaris vs. P −DPO vs. P −exp. P −DPO = Polaris without DPO training, P −exp = Polaris without name expansion. Blue = best per cell. Dataset

Dataset

Method

@10

@20

@50

@100

A(Qwen) AutoDDG P (Qwen) Polaris

.42 .49 .73 .66

.55 .64 .81 .77

.73 .76 .88 .83

.83 .83 .88 .83

.83 .85 .88 .83

Arctic

A(Qwen) AutoDDG P (Qwen) Polaris

.29 .24 .29 .29

.41 .34 .42 .41

.58 .48 .60 .58

.74 .68 .77 .77

.81 .77 .82 .82

LTER

A(Qwen) AutoDDG P (Qwen) Polaris

.05 .08 .11 .18

.08 .14 .18 .25

.17 .22 .27 .32

.31 .32 .34 .39

.36 .35 .36 .43

ECIR

A(Qwen) AutoDDG P (Qwen) Polaris

.01 .01 .01 .01

.02 .02 .02 .03

.04 .04 .04 .04

.07 .08 .08 .08

.11 .12 .12 .13

WTR

A(Qwen) AutoDDG P (Qwen) Polaris

.08 .08 .09 .08

.15 .14 .15 .15

.24 .25 .25 .26

.44 .43 .46 .48

.62 .59 .61 .66

WikiTables

A(Qwen) AutoDDG P (Qwen) Polaris

.24 .21 .19 .21

.34 .29 .30 .32

.47 .41 .45 .42

.70 .66 .69 .67

.86 .83 .84 .81

Table 16: Polaris vs. AutoDDG under different LLMs (Recall@K): A(Qwen) vs. AutoDDG vs. P (Qwen) vs. Polaris. A(Qwen) = AutoDDG with Qwen2.5-7B, P (Qwen) = Polaris with Qwen2.5-7B; AutoDDG and Polaris use Llama 3.1 8B. Blue = best per cell.

Table 14: Polaris with different LLMs (Recall@K): P (Qwen) vs. Polaris. P (Qwen) = Polaris with Qwen2.5-7B, Polaris = Polaris with Llama 3.1 8B. Blue = better per cell.

Dataset

Method

@10

@20

@50

@100

Dataset

Method

@10

@20

@50

@100

P (β=0.5) Polaris

.7398 .7312

.7662 .7582

.8051 .7869

P (5-desc) Polaris

.73 .66

.84 .77

.90 .83

Arctic

P (β=0.5) Polaris

.7266 .7133

.6874 .6743

.6837 .6939

.7380 .7416

.7585 .7678

Arctic

P (5-desc) Polaris

.29 .29

.41 .41

.61 .58

.80 .77

.84 .82

LTER

P (β=0.5) Polaris

.6760 .7218

.6151 .6546

.5806 .6013

.4929 .5239

.4732 .5145

LTER

P (5-desc) Polaris

.13 .18

.20 .25

.28 .32

.35 .39

.39 .43

ECIR

P (β=0.5) Polaris

.4914 .4946

.5158 .5050

.4837 .4981

.4956 .4873

.4784 .4672

ECIR

P (5-desc) Polaris

.01 .01

.02 .03

.04 .04

.08 .08

.13 .13

WTR

P (β=0.5) Polaris

.3728 .3840

.3820 .3840

.3868 .3894

.4455 .4449

.5275 .5309

WTR

WikiTables

P (β=0.5) Polaris

.4040 .4111

.4116 .4215

.4368 .4407

.5123 .5171

.5728 .5727

P (5-desc) Polaris

.08 .08

.16 .15

.27 .26

.50 .48

.67 .66

WikiTables

P (5-desc) Polaris

.21 .21

.32 .32

.43 .42

.66 .67

.83 .81

Table 17: Effect of the DPO regularization parameter β (NDCG@K): P (β=0.5) vs. Polaris. P (β=0.5) = Polaris with β = 0.5; Polaris uses the default β = 0.1. Blue = better per cell.

Dataset

Method

@10

@20

@50

@100

P (β=0.5) Polaris

.68 .66

.77 .77

.86 .83

Arctic

P (β=0.5) Polaris

.30 .29

.42 .41

.56 .58

.78 .77

LTER

P (β=0.5) Polaris

.13 .18

.20 .25

.29 .32

ECIR

P (β=0.5) Polaris

.01 .01

.02 .03

WTR

P (β=0.5) Polaris

.08 .08

WikiTables

P (β=0.5) Polaris

.20 .21

Table 20: Effect of the number of candidate descriptions (Recall@K): P (5-desc) vs. Polaris. P (5-desc) = Polaris with 5 candidate descriptions per gold table; Polaris has 3 by default. Blue = better per cell.

Dataset

Method

@10

@20

@50

@100

P (3-epoch) Polaris

.6908 .7312

.7522 .7582

.7881 .7869

.7968 .7869

.82 .82

Arctic

P (3-epoch) Polaris

.8080 .7133

.7131 .6743

.7059 .6939

.7595 .7416

.7951 .7678

.37 .39

.40 .43

LTER

P (3-epoch) Polaris

.7064 .7218

.6535 .6546

.6020 .6013

.5105 .5239

.4940 .5145

.04 .04

.08 .08

.13 .13

ECIR

P (3-epoch) Polaris

.4532 .4946

.4813 .5050

.4763 .4981

.4702 .4873

.4584 .4672

.15 .15

.25 .26

.47 .48

.65 .66

WTR

P (3-epoch) Polaris

.3810 .3840

.3505 .3840

.3571 .3894

.4095 .4449

.4859 .5309

.30 .32

.42 .42

.67 .67

.83 .81

WikiTables

P (3-epoch) Polaris

.4069 .4111

.4002 .4215

.4359 .4407

.5142 .5171

.5753 .5727

Table 21: Effect of the number of training epochs (NDCG@K): P (3-epoch) vs. Polaris. P (3-epoch) = Polaris trained for 3 epochs; Polaris uses 1 epoch by default. Blue = better per cell.

Table 18: Effect of the DPO regularization parameter β (Recall@K): P (β=0.5) vs. Polaris. P (β=0.5) = Polaris with β = 0.5; Polaris uses the default β = 0.1. Blue = better per cell.

Dataset

Method

@10

@20

@50

@100

Dataset

Method

@10

@20

@50

@100

P (3-epoch) Polaris

.65 .66

.83 .77

.91 .83

.93 .83

P (5-desc) Polaris

.8019 .7312

.8313 .7582

.8601 .7869

Arctic

P (5-desc) Polaris

.6843 .7133

.6593 .6743

.7013 .6939

.7446 .7416

.7645 .7678

Arctic

P (3-epoch) Polaris

.32 .29

.42 .41

.59 .58

.80 .77

.86 .82

LTER

P (5-desc) Polaris

.6364 .7218

.5987 .6546

.5567 .6013

.4805 .5239

.4694 .5145

LTER

P (3-epoch) Polaris

.16 .18

.23 .25

.31 .32

.38 .39

.42 .43

ECIR

P (5-desc) Polaris

.4728 .4946

.4673 .5050

.4832 .4981

.4897 .4873

.4731 .4672

ECIR

P (3-epoch) Polaris

.01 .01

.02 .03

.04 .04

.08 .08

.13 .13

WTR

P (5-desc) Polaris

.3723 .3840

.3842 .3840

.3982 .3894

.4623 .4449

.5391 .5309

WTR

P (3-epoch) Polaris

.08 .08

.13 .15

.22 .26

.42 .48

.59 .66

WikiTables

P (5-desc) Polaris

.4328 .4111

.4364 .4215

.4550 .4407

.5248 .5171

.5834 .5727

WikiTables

P (3-epoch) Polaris

.20 .21

.28 .32

.44 .42

.67 .67

.84 .81

Table 19: Effect of the number of candidate descriptions (NDCG@K): P (5-desc) vs. Polaris. P (5-desc) = Polaris with 5 candidate descriptions per gold table; Polaris has 3 by default. Blue = better per cell.

Table 22: Effect of the number of training epochs (Recall@K): P (3-epoch) vs. Polaris. P (3-epoch) = Polaris trained for 3 epochs; Polaris uses 1 epoch by default. Blue = better per cell.

Dataset

Method

@10

@20

@50

@100

AutoDDG Polaris P (multi)

.5551 .7312 .8133

.6192 .7582 .8282

.6692 .7869 .8677

.6969 .7869 .8677

.7030 .7869 .8677

Arctic

AutoDDG Polaris P (multi)

.6274 .7133 .7303

.5942 .6743 .7146

.5942 .6939 .7330

.6360 .7416 .7755

.6812 .7678 .8058

LTER

AutoDDG Polaris P (multi)

.4372 .7218 .7962

.4194 .6546 .7074

.4011 .6013 .6567

.3656 .5239 .6278

.3559 .5145 .6384

ECIR

AutoDDG Polaris P (multi)

.4215 .4946 .5575

.4429 .5050 .5347

.4512 .4981 .5049

.4408 .4873 .4965

.4180 .4672 .4757

WTR

AutoDDG Polaris P (multi)

.3748 .3840 .3802

.3548 .3840 .3786

.3767 .3894 .3858

.4159 .4449 .4657

.4884 .5309 .5912

WikiTables

AutoDDG Polaris P (multi)

.4139 .4111 .4268

.4085 .4215 .4155

.4398 .4407 .4419

.5176 .5171 .5172

.5811 .5727 .5849

Table 23: Multi-field KWS (NDCG@K): AutoDDG vs. Polaris vs. P (multi). P (multi) = Polaris using multiple fields, Polaris = single field (description), AutoDDG = single field (description). Blue = best per cell.

Dataset

Method

@10

@20

@50

@100

AutoDDG Polaris P (multi)

.49 .66 .75

.64 .77 .83

.76 .83 .92

.83 .83 .92

.85 .83 .92

Arctic

AutoDDG Polaris P (multi)

.24 .29 .32

.34 .41 .46

.48 .58 .63

.68 .77 .82

.77 .82 .88

LTER

AutoDDG Polaris P (multi)

.08 .18 .24

.14 .25 .31

.22 .32 .39

.32 .39 .53

.35 .43 .61

ECIR

AutoDDG Polaris P (multi)

.01 .01 .01

.02 .03 .02

.04 .04 .04

.08 .08 .08

.12 .13 .13

WTR

AutoDDG Polaris P (multi)

.08 .08 .09

.14 .15 .16

.25 .26 .27

.43 .48 .52

.59 .66 .78

WikiTables

AutoDDG Polaris P (multi)

.21 .21 .20

.29 .32 .29

.41 .42 .41

.66 .67 .64

.83 .81 .84

Dataset

Method

@10

@20

@50

@100

P (multi) −desc −exp

.8133 .7594 .7312

.8282 .7668 .7582

.8677 .7895 .7869

Arctic

P (multi) −desc −exp

.7303 .7602 .7133

.7146 .7162 .6743

.7330 .6726 .6939

.7755 .7139 .7416

.8058 .7255 .7678

LTER

P (multi) −desc −exp

.7962 .5938 .7218

.7074 .5639 .6546

.6567 .5236 .6013

.6278 .5084 .5239

.6384 .4959 .5145

ECIR

P (multi) −desc −exp −ctx

.5575 .4478 .5053 .5474

.5347 .4737 .5231 .5333

.5049 .4474 .5159 .4925

.4965 .4614 .4996 .4816

.4757 .4422 .4770 .4548

WTR

P (multi) −desc −exp −ctx

.3802 .3698 .4053 .3583

.3786 .3734 .4003 .3581

.3858 .3811 .4150 .3559

.4657 .4632 .4816 .4296

.5912 .5844 .5993 .5254

WikiTables

P (multi) −desc −exp −ctx

.4268 .3179 .4214 .4168

.4155 .3187 .4146 .4135

.4419 .3476 .4403 .4369

.5172 .4207 .5233 .5091

.5849 .4946 .5812 .5706

Table 25: Field ablation (NDCG@K) for multi-field P (multi). −desc = w/o description, −exp = w/o table/column name expansions, −ctx = w/o table context (ECIR, WTR, WikiTables only). Blue = best per cell.

Table 24: Multi-field KWS (Recall@K): AutoDDG vs. Polaris vs. P (multi). P (multi) = Polaris using multiple fields, Polaris = single field (description), AutoDDG = single field (description). Blue = best per cell.

Polaris: Learning to Generate Table Descriptions from Retrieval Feedback

Analyze & explore related research in Conceptio

Related documents