arXiv CSopen access

TRL-Bench: Standardizing Cross-Paradigm Representation-Level Evaluation of Tabular Encoders

TRL-B ENCH: Standardizing Cross-Paradigm Representation-Level Evaluation of Tabular Encoders

Wei Pang1,∗ , Xiangru Jian2,∗ , Hehan Li1,∗ , Zhixuan Yu1,∗ , Alex Xue2,∗ Jinyang Li3 , Zhengyuan Dong2 , Xinjian Zhao1 , Hao Xu4 Chao Zhang5 , Reynold Cheng3 , M. Tamer Özsu2 , Tianshu Yu1,†

arXiv:2606.09323v1 [cs.AI] 8 Jun 2026

1

3

The Chinese University of Hong Kong, Shenzhen 2 University of Waterloo The University of Hong Kong 4 The University of Sydney 5 Université Lyon 1 ∗

Core contributors

†

Corresponding author: [email protected]

Abstract Tabular encoders are usually evaluated inside task-specific end-to-end pipelines, so models from different training paradigms are difficult to compare directly even when they operate on similar tabular signals. We introduce TRL-B ENCH, a multigranular tabular representation learning (TRL) benchmark that standardizes crossparadigm representation-level evaluation: each encoder exports row-, column-, or table embeddings through its supported wrapper, and shared lightweight heads probe them across three suites: TRL-CT BENCH (column/table), TRL-R BENCH (row), and TRL-DLTE (compositional Data-Lake Table Enrichment spanning all three granularities). To support this standardized setting, we release curated benchmark assets and task reformulations, including 50 OpenML tables with 123 verified targets, 16 row-pair linkage rewrites, and a 47,772-table DLTE lake derived from 1,379 parent tables. Across 20 models and 16 tasks, TRL-B ENCH shows that once downstream conditions are standardized, encoder quality is capability-specific rather than captured by a single leaderboard. In TRL-CT BENCH, generic text encoders often lead on tasks with strong surface-text signal, while tabular specialists win where their pretraining objective aligns with the task. In TRL-R BENCH, within-table prediction and cross-table linkage favor different training regimes, with atomic linkage performance correlating strongly with the row-matching stage of DLTE pipelines. In TRL-DLTE, the strongest pipelines combine capabilitymatched specialists rather than reuse a single encoder, and top end-to-end quality depends on non-additive compositional fit rather than per-stage marginal rank alone. TRL-B ENCH provides a common protocol for measuring reusable signal in exported tabular representations under shared downstream conditions. Code: https://github.com/LOGO-CUHKSZ/TRL-Bench; data is released on Hugging Face.1

1

Introduction

Tables have long been recognized as the fundamental data structures for storing structured data, and there has been considerable work on using them across a wide range of analytical workloads. Recent work has produced strong row-, column-, and table-level encoders for reasoning over tabular data. Many of these are useful as reusable components: tables can be encoded once and their 1 TRL-CT BENCH : https://huggingface.co/datasets/logo-lab/trl-ctbench

TRL-R BENCH: https://huggingface.co/datasets/logo-lab/trl-rbench TRL-DLTE: https://huggingface.co/datasets/logo-lab/trl-dlte Preprint.

DATA

ENCODERS

2

TRL-CTBENCH

20 datasets 13 tasks

Generic Text

TRL-RBENCH 50 OpenML tables 123 targets 16 record-linkage datasets Hand-verified targets Leakage-audited pairs

Table Structure

Col Centric

Source tables (encoded once)

TransferBased

PriorBased

TASKS

4

Table embeddings

5

TRL-DLTE

TRL-CTBENCH

Retrieve candidate tables using table embeddings

…

FINDINGS

Transfer is capability-specific

Row signal is not single-faceted

Capability-matched hybrids beat singleencoder reuse

Compositional fit matters

...

Row regimes

TRL-DLTE

Lake of 47,772 tables

Table Text

EMBEDDINGS

3

Column/Table families

Column embeddings

Target-Table Learners

Schema

Joinability

… Tabular Encoder

Legend

Row embeddings

…

Reusable across tasks and datasets

Benchmark suites (by color) Blue = TRL-CTBENCH Purple = TRL-DLTE Yellow = TRL-RBENCH

Unionability Grounding

TRL-RBENCH

Embedding granularities (by color / icon) Table embedding Column embedding Row embedding

Align columns + decide union / join / none using column embeddings ...

2

Match rows + merge using row embeddings ...

Record Row prediction linkage

End-to-end metric: UJ-H

Figure 1: TRL-Bench at a glance. Each model is processed once through its supported wrapper to export row-, column-, or table embeddings, and shared lightweight modules then evaluate those embeddings across TRL-CT BENCH (schema, joinability, unionability, grounding), TRL-R BENCH (row prediction, record linkage), and TRL-DLTE (multi-stage data-lake enrichment). embeddings indexed and reused across tasks and large multi-table corpora such as data lakes, where per-task fine-tuning is often impractical [5, 24, 23]. In such encode-once, reuse-many settings, the representation itself, not the task-specific wrapper, is the object of evaluation. Yet these encoders are still mostly evaluated inside task-specific end-to-end pipelines, so models from different training paradigms are difficult to compare directly: a strong result may come from the wrapped predictor, training budget, and task-specific adaptation as much as from the encoder itself. This motivates a comparability question: under one shared evaluation protocol over the exported representations, how do heterogeneous tabular encoders actually differ? TRL-B ENCH is designed around that question and complements end-to-end task benchmarks by isolating reusable representation quality under shared downstream conditions (Figure 1). Each model is run once through its supported wrapper to export the row-, column-, or table embeddings it exposes, and shared lightweight downstream modules evaluate those embeddings across tasks rather than re-optimizing the encoder end to end. Throughout, we use “encoder” operationally to denote any tabular model that exposes reusable row-, column-, or table-level embeddings. To make this evaluation comprehensive rather than task-specific, TRL-B ENCH treats retrieval, schema alignment, linkage, prediction, and grounding as atomic capabilities that serve as reusable building blocks for downstream tabular systems in the encode-once, reuse-many setting. The three suites measure these capabilities at the granularities where embeddings are reused: TRL-CT BENCH for column/table transfer, TRL-R BENCH for row transfer, and TRL-DLTE for compositional data-lake table enrichment. When the benchmark is used to test 20 models and 16 tasks, three empirical findings emerge. First, once downstream conditions are standardized, transfer is capability-specific: in TRL-CT BENCH, generic text encoders often lead on tasks with strong surface-text signal, while the remaining wins are better explained by pretraining–task alignment than by any single dominant encoder class. Second, row signal is not single-faceted: within-table prediction and noisy cross-table linkage separate model families by training scope. Third, compositional fit shapes pipeline quality: in TRL-DLTE, the best pipelines are capability-matched hybrids that consistently outperform single-encoder reuse. Per-stage marginals are informative but do not determine the top pipelines. End-to-end quality depends on how well retrieval, column alignment, and row matching compose, not on per-stage rank in isolation. Contributions. 1. Standardized cross-paradigm protocol: heterogeneous encoders export row-, column-, or table-level embeddings, and shared lightweight readouts evaluate them under common task definitions, enabling direct comparison without end-to-end fine-tuning. 2. Comprehensive benchmark for reusable tabular signal: TRL-CT BENCH, TRL-R BENCH, and TRL-DLTE cover column/table transfer, row transfer, and compositional enrichment over 16 tasks and 87 datasets from SATO [82], SOTAB [43], WikiCT [17], Spider [81], Valentine [45], OpenML [73], and DeepMatcher/WDC [53, 63] (Appendix F). 3. Curated assets and task reformulations: we contribute (a) 20 column/table datasets standardized for representation-level evaluation, (b) 50 OpenML-derived row-prediction tables with 123 hand-verified targets, (c) 16 record-linkage datasets rewritten as explicit row-pair matching tasks, and (d) a 47,772-table enrichment lake built from 1,379 parent tables, together with representationcentric rewrites of heterogeneous source tasks such as WTQ [60] and DeepMatcher. 2

Table 1: Comparison with prior tabular evaluation resources (✓ = supported). Cross-paradigm: multiple training paradigms under one protocol. Repr.-level eval.: representation-level evaluation as primary intended use. Downstream transfer: downstream task performance vs. intrinsic properties. Task family: broad problem class (e.g., row prediction, semantic typing). Observatory reports intrinsic properties only, hence “–”. Cross- Repr.-level Downstream # Task paradigm eval. transfer fam.

Benchmark

Col. Table Row Comp.

OpenML suites [73, 8, 25] TabArena [21] DeepMatcher [53] LakeBench [68, 18] SANTOS / TUS [54, 39] Valentine [45] SemTab / SOTAB [36, 43] Observatory [15]

✗ ✗ ✗ ✓ ✓ ✓ ✓ ✓

✗ ✗ ✗ ✓ ✓ ✗ ✗ ✓

✓ ✓ ✓ ✗ ✗ ✗ ✗ ✓

✗ ✗ ✗ ✗ ✗ ✗ ✗ ✗

✗ ✗ ✗ ✗ ✗ ✗ ✗ ✓

✗ ✗ ✗ ✗ ✗ ✗ ✗ ✓

✓ ✓ ✓ ✓ ✓ ✓ ✓ ✗

1 1 1 2 1 1 1 –

TRL-B ENCH (ours)

✓

✓

✓

✓

✓

✓

✓

7

4. A cross-paradigm empirical study: across 20 models and 16 tasks, we show that no single pretraining recipe behaves as a universal tabular representation, and we identify structural gaps in model choice, transfer scope, and pipeline composition that single-paradigm or single-granularity evaluations cannot isolate.

2

Related Work and Positioning

We situate TRL-B ENCH relative to both prior tabular model families and existing benchmark resources. Appendix B gives additional citations and comparisons. Related work: model families and evaluation traditions. Prior tabular work is fragmented by granularity. Row-level models focus on supervised prediction and transfer [33, 66, 72, 48, 80, 6, 74, 76, 31, 32, 64, 27, 12, 28, 41, 87], while column/table models target schema semantics, grounding, retrieval, and discovery [19, 50, 29, 78, 17, 75, 34, 24, 40]. These families are usually evaluated in task-specific settings rather than under a shared multi-granular representation-level protocol. Observatory [15] is the closest prior resource that compares frozen tabular embeddings across model families, but it measures perturbation- and invariance-style intrinsic properties such as sample fidelity and order insignificance rather than downstream task performance. Positioning of TRL-B ENCH. Prior benchmarks each target narrow task scopes (Table 1) [73, 8, 25, 21, 53, 63, 36, 43, 45, 54, 39, 68, 18] and usually compare models within a single task family or end-to-end pipeline rather than under a shared representation-level protocol. TRLB ENCH complements these resources by standardizing heterogeneous tabular encoders into a shared representation-level evaluation protocol. Its main distinctions are: (i) multi-granular evaluation across columns, rows, and tables, (ii) direct cross-paradigm comparison under common task definitions and lightweight downstream heads, and (iii) a compositional DLTE benchmark testing whether strong atomic capabilities compose into an end-to-end pipeline. Because this standardized comparison operates on exported representations, it applies to models that expose reusable row-, column-, or table-level embeddings, either natively or via a natural extraction point in the architecture. Generative table LLMs [85, 69] generally do not provide such an interface, while heavily task-specific fine-tuned systems [49, 70, 35] are formulated as end-to-end predictors rather than reusable representations.

3

Benchmark Design

TRL-B ENCH asks a comparability question: once heterogeneous tabular encoders are evaluated under one shared representation-level protocol, how do they differ across rows, columns, and tables, and how do they compose end-to-end? The benchmark has three suites, TRL-CT BENCH, TRL-R BENCH, and TRL-DLTE. Figure 1 gives the high-level view. 3.1

Problem Setting and Standardized Representation-Level Protocol

Throughout, we adopt two established properties of good representations from the representationlearning literature: recoverability under simple, capacity-limited readouts (the probing tradition [2]), 3

(a) ROW PREDICTION CURATION

TRL-Rbench

(b) DLTE LAKE CONSTRUCTION

sc

INPUT

re en 1

158 candidate tables

RULE SCREEN ≥ 2 valid targets/table

audit

2

review

DEGENERACY AUDIT drop ID, nearunique, leaked, constant targets

3

e as le re

OUTPUT

INPUT

HUMAN REVIEW & REPAIR

50

1,379

verify reg/cls fix labels

curated tables

parent tables

77 classification

989 TabFact

46 regression

390 WTQ

TabArena OpenML-CC18 OpenML-CTR23

fra gm

en t

1

compose

FRAGMENT GENERATION

TRL-DLTE

2

+ distractors

3

m co

ne bi

OUTPUT

COMPOSITIONAL TARGETS

HARD DISTRACTORS

11,032

36,740

targets · distractors

CKAN tables

47,772

seed · union · join

union + join targets

shared retrieval lake 827/207/345 train/dev/test

>=2 targets/table

Figure 2: Curation of TRL-R BENCH row-prediction tables and assembly of the TRL-DLTE lake. (a) Row prediction curation: 158 candidate tables filtered through rule screening, degeneracy audit, and human review with label repair into 50 tables with 123 targets. (b) DLTE lake assembly: 1,379 TabFact/WTQ parents fragmented into seed queries and union/join targets at four noise tiers. 11,032 targets are embedded alongside 36,740 CKAN distractors in a 47,772-table lake. and transferability across many downstream tasks [7]. We take a reusable tabular representation to be good to the extent that a single exported embedding satisfies both. TRL-B ENCH standardizes heterogeneous tabular models at the level of exported representations, not by forcing a single input serialization. For a table T with columns C(T ) = (c1 , . . . , cM ) and rows R(T ) = (r1 , . . . , rN ), a tabular encoder fθ may expose column, row, or table representations, col E col (T ) = (ecol 1 , . . . , eM ),

row E row (T ) = (erow 1 , . . . , eN ),

etbl (T ).

For each task, write e for the relevant exported encoder output(s) (a column, row, table, or pair thereof). A downstream module r then maps these embeddings to the task output, and TRL-B ENCH uses three downstream-module types. Training-free modules r(e) have no task-specific learned parameters and operate directly on embedding geometry, e.g., cosine ranking for schema matching or union search, or k-means for column clustering. Learned modules rψ (e) are lightweight supervised probes trained on exported encoder embeddings, e.g., column type prediction, join/union classification, join search, row prediction, and record linkage. Query-conditioned modules rψ (q, e) additionally consume a frozen text-query embedding q = ftext (query), e.g., a dual-projection head for table retrieval or a decoder for table QA. Operationalizing recoverability, all downstream heads are intentionally lightweight and held fixed across encoder families, so comparisons reflect the choice of exported embedding under common readouts rather than the choice of downstream predictor. Transferability is then tested by reusing each exported embedding across the multiple tasks within its suite, and, for models that expose multiple granularities, across suites as well (Sec. 3.2–3.4). The learned-module category follows a unified supervised-probe protocol. For every supervised probe task, we train both a linear head and a one-hidden-layer MLP (hidden size 256, robust to head size and depth, Appendix Table 24) on exported encoder embeddings with Adam [42] under standardized settings (Appendix R). We use the arithmetic average of the two as the canonical score: the linear head tests linearly accessible signal, and the MLP tests whether a small nonlinear readout can recover it. Per-head diagnostics, including a cosine reference for record linkage, are reported in Appendices K.2 and L.3. For table-level tasks, if a model supports multiple table aggregations (e.g., cls, col-mean, tok-mean), we apply this protocol to each aggregation and report the strongest (per-model, full ablations in Appendix K.3). Training-free tasks (Column Clustering, Union Search, Schema Matching), query-conditioned tasks (Table QA, Table Retrieval), and the DLTE pipeline (Sec. 3.4) follow their corresponding module types defined above. Pairwise tasks use fixed combination operators, typically concatenation. Appendix Tables 7–8 give per-task details. A model is evaluated only when the required granularity is natively exposed or obtainable by supported pooling. Appendix C summarizes supported granularities, and Appendix D states the benchmark’s wrapper policy. 3.2

TRL-CT BENCH: Column- and Table-Level Transfer

TRL-CT BENCH contains 13 tasks: 8 column-level and 5 table-level, grouped into schema understanding, joinability, unionability, and grounding. Schema understanding. These tasks consume a single column embedding or an ordered intra-table column pair and test whether exported column representations expose semantic type and intra-table structure. They include column type prediction, column clustering, and column relation prediction. 4

Joinability. Joinability asks whether two tables are complementary, i.e., one can add attributes to the other through overlapping columns. Tasks are join search, column overlap, and table-level join classification. Since raw cosine between column embeddings does not directly model directional value containment (i.e., whether one column’s values are contained in another’s), the main join-search setting uses a minimal learned projection head on top of the exported embeddings. Unionability. Unionability asks whether two tables are stackable after schema alignment. Tasks are union search, schema matching, union classification, union regression, and table subset. Union search uses SANTOS [39], UGEN [58], and TUS [54], with an added low-overlap TUS-hard variant (Appendix K.5) that removes positive pairs whose directed column containment is ≥ 0.70, creating an explicit contrast between the original TUS setting and one without high value-containment positives. Grounding. Grounding tests whether a representation can ground a natural-language query in structured table content. These tasks are query-conditioned: a frozen text encoder embeds the question, and a lightweight head combines that query embedding with table-side representations exported by the evaluated model. Table QA uses question and column embeddings with a lightweight decoder, while table retrieval trains a dual projection head over query and table embeddings. Task adaptation and curation. Most source tasks were not originally formulated for standardized representation-level evaluation. We therefore standardize table identifiers, align label schemas, rewrite end-to-end datasets into representation-centric variants, and clean dataset layouts where needed. Across the benchmark, this rewriting is concrete: for WikiTableQuestions [60], we replace joint encoder fine-tuning with exported column/question embeddings plus a lightweight decoder. For the four pairwise tasks whose original splits exhibit table-level overlap (join classification, column overlap, union classification, and union regression), we enforce table-disjoint train/dev/test splits that prevent any test table from appearing during training. The remaining supervised table-pair task (table subset) already has table-disjoint splits in the source data. Appendix Tables 7–8 list per-task evaluation modes, split types, and metrics. Appendix E.1 summarizes the benchmark’s shortcut and leakage mitigations.

3.3

TRL-R BENCH: Row-Level Transfer

TRL-R BENCH asks whether exported row embeddings transfer both within a table and across tables. It contains row prediction and record linkage. Row prediction. For row prediction, the encoder sees only the observed columns X, produces one target-agnostic embedding per row, and that same embedding is reused to predict each curated target column yk ∈ Y = {y1 , . . . , yK } (K ≥ 2) with a lightweight probe under the protocol of Sec. 3.1. This asks whether a single row embedding can be reused across multiple targets from the same table, including tables with mixed classification and regression targets. The suite contains 50 OpenML-derived tables with 123 curated targets (77 classification, 46 regression), filtered from 158 candidates from TabArena [21], OpenML-CC18 [8], and OpenML-CTR23 [25]. Every released table has 2–3 targets (mean 2.46). Human curators selected target columns, repaired label issues where needed, verified classification-versus-regression typing, and removed degenerate or leaked targets such as constant columns, near-duplicate targets, and label columns recoverable from the input (Figure 2). All target columns are excluded from encoder input. Record linkage. Record linkage complements intra-table prediction with inter-table matching: given a pair of rows from two tables, predict whether they refer to the same entity. We adopt 16 datasets from two entity-matching benchmark families [53, 63, 62]: 8 clean DeepMatcher [53] benchmarks, 4 dirty DeepMatcher variants with synthetic schema noise, and 4 size variants of the WDC Products Large-Scale Product Matching (LSPM) benchmark. For analysis, we split these into Clean Linkage (the 8 clean DeepMatcher datasets) and Robust Linkage (the 4 dirty DeepMatcher + 4 WDC datasets). We rewrite these sources as explicit row tables with labeled row pairs, retain the original source pair-disjoint splits, which are the canonical evaluation protocol in the entity-matching literature, and feed paired exported row embeddings via concatenation to a lightweight supervised probe under the protocol of Sec. 3.1. Appendix L.4 reports per-source train/test pair and row-overlap statistics, and documents the removal of label-equivalent columns before any encoder consumes a row. 5

3.4

TRL-DLTE: Multi-Stage Data Lake Table Enrichment

Atomic tasks test local transfer, but not whether row-, column-, and table-level representations compose into a full multi-stage pipeline. TRL-DLTE addresses this gap. We start from a complete parent table, the ground-truth table to be reconstructed. From it, we remove a block of rows and a block of columns. The remaining subtable is the seed query. The removed rows form the union target (same schema as the seed, additional rows), and the removed columns form the join target (same rows as the seed, additional attributes). Given only the seed and a data lake, the system must recover both targets by retrieving relevant tables, deciding whether each candidate contributes by union, join, or neither, aligning columns, matching rows, and merging the result. We build TRL-DLTE from filtered parent tables drawn from TabFact [13] and WTQ [60]. Figure 2(b) summarizes the construction and counts: each parent is fragmented at four cumulative noise tiers (clean, schema, cell, hard) into a seed query, a union target, and a join target, and the targets are inserted into a shared retrieval lake together with CKAN distractors. Seeds serve only as queries and are not lake members. Parent tables are split before fragmentation so that train/dev/test remain parent-disjoint. Evaluation has three stages. Stage 1 retrieves candidate tables using table embeddings. Stage 2 aligns columns and predicts union/join/none using column embeddings. Stage 3 matches rows and merges content using row embeddings. Pipelines can use a single multi-granular model or combine different specialists across stages, making DLTE a composition test for the benchmark as a whole. Full stage-wise operator specifications, including the Stage-2 threshold calibration procedure and the dev-based selection of the headline pipeline reported in Sec. 4.4, are in Appendix M. We introduce UJ-H as the primary end-to-end score, which summarizes recovery of both the union and join targets. Let Runion be the fraction of removed-row-block cells recovered in the seed columns, and Rjoin the fraction of removed-column-block cells recovered for the seed rows. UJ-H is the perquery harmonic mean of these two recalls, averaged over queries (zero when both recalls vanish): 2 Runion Rjoin UJ-H = Runion + Rjoin penalizing pipelines that succeed on only one enrichment path. We additionally report Cell F1 in Appendix N.1. It is a multiset F1 over recovered cells pooled across the removed-row and removed-column blocks, used as a complementary diagnostic of pooled cell-recovery yield.

4

Experiments

We now use the standardized representation-level protocol of Sec. 3.1 to compare heterogeneous tabular encoders across the three benchmark suites introduced in Sec. 3: TRL-CT BENCH, TRLR BENCH, and TRL-DLTE. 4.1

Experimental Setup

We follow the standardized representation-level protocol in Sec. 3.1. Here we summarize only the choices needed to read the result tables. Compared models. We evaluate 20 models spanning generic text encoders, table-aware and structureaware encoders, column-specialized models, target-table self-supervised learners, and meta-pretrained priors. Appendix Table 5 lists the full inventory. Baselines and controls. Each task reports the applicable simple non-neural or analytical baselines alongside learned encoders (e.g., TF-IDF for column-level tasks, value overlap for search). “Best∗ ” in Table 2 denotes the strongest applicable non-neural baseline, with markers a–d identifying which one. Full specifications and 5-seed per-dataset results are in Appendix G. Reporting and Metrics. CTBench reports raw metrics plus per-family normalized-rank (NR, lower is better) aggregates. For table-level tasks, the main comparison uses the strongest supported aggregation (Appendix K.3). Row prediction averages over 123 targets (77 classification and 46 regression, with TAB T RANSFORMER covering 63 due to its categorical-feature requirement), with linkage split into Clean Linkage (DM-C) and Robust Linkage (DM-D + WDC NR aggregate). DLTE uses UJ-H as the primary end-to-end score. To avoid selection bias from reporting the maximum over 1,120 test evaluations, headline DLTE pipelines are selected on the development split by UJ-H 6

Table 2: Column- and table-level results on 13 TRL-CT BENCH tasks spanning four capability families. Join, Union, and Grounding are evaluated at both column and table granularity (superscripts c t / ). NR reports the mean normalized rank within each family. Dashes indicate unsupported granularities. Formula of metrics in Appendix H. Schema Family

Model Best∗

Baseline

Generic Text

Join

Col Typec

Col Clustc

Col Relc

F1 ↑

NMI↑

F1 ↑

0.813a

0.400a

0.015

±.007

±.001

±.001

BERT

0.926 ±.001

±.000

±.002

GTE

0.922

0.466

0.516

0.811

0.826

NR ↓

—

Union

Join Searchc

Col Overlapc†

Join Class.t†

MAP↑

nRMSE↓

F1 ↑

0.155b

1.014

0.516

±.000

±.003

±.014

0.000

0.434 ±.000

±.001

±.010

0.190

0.469

0.817

0.535

±.002

±.002

±.019

0.381

0.406

0.855

±.002

±.002

±.034

0.476

0.320

0.823

0.544

±.001

±.002

±.011

0.786

0.553

±.002

±.001

±.002

Tabular-Pretrained TaBERT

0.874

0.514

0.760

±.003

±.003

±.002

TAPAS

0.868

0.448

0.769

±.001

±.001

±.003

TAPEX

—

—

—

—

—

—

0.538

TABBIE

0.892

0.262

0.785

0.476

0.208

0.862

0.542

±.004

±.007

±.002

±.001

±.002

±.022

TURL

0.814

0.406

0.758

0.667

0.299

0.809

0.532

±.001

±.004

±.002

±.000

±.001

±.014

—

—

—

—

—

0.468

Table-Text

0.498

NR ↓

—

Schema Matchc

Union Class.t†

Union Reg.t†

Tbl Subsett

MAP↑

R@GT↑

F1 ↑

nRMSE↓

F1 ↑

0.574c

0.473d

0.500

1.138

0.458

±.000

±.000

±.004

±.024

±.027

0.048

0.564 ±.000

±.000

±.002

±.009

±.005

0.243

0.608

0.417

0.423

0.843

0.600

0.544

±.000

±.000

±.002

±.009

±.002

0.760

0.615

0.540

0.430

TUTA

—

Col.-Centric

0.789

0.404

0.698

±.004

±.000

±.001

TabSketchFM

0.566

0.252

0.373

±.001

±.007

±.003

0.810

0.316

0.847

0.510

±.001

±.001

±.019

1.000

0.265

0.946

0.516

±.001

±.001

±.015

0.592

0.545

0.619 ±.000

±.000

±.004

±.009

±.004

0.323

0.529

0.415

0.837

0.607

0.567

±.000

±.000

±.004

±.005

±.004

0.333

—

—

0.854

0.609

0.558

±.002

±.006

±.003

0.693

0.410

0.205

0.833

0.663

0.546

±.000

±.000

±.002

±.001

±.004

0.471

0.575

0.340

0.814

0.657

0.507

±.000

±.001

±.001

±.009

±.004

—

—

0.810

0.652

0.447

±.003

±.007

±.008

0.853

0.662

0.539

1.000

±.010

Starmie

0.857

0.630

±.021

Table-Struct.

Grounding

Union Searchc

0.640

0.662

0.764

±.000

±.000

±.002

±.003

±.005

0.841

0.531

0.155

0.737

0.668

0.553

±.000

±.000

±.002

±.010

±.005

Tbl QAc

Tbl Ret.t

Acc↑

MRR↑

0.204

0.131

±.004

±.120

0.260

0.255

0.367

±.004

±.008

0.343

0.245

0.476

±.002

±.003

NR ↓

—

0.457

0.267 ±.005

±.013

0.419

0.254

0.295

±.003

0.372 ±.006

0.185

—

0.332

0.727

0.276

0.170

±.004

±.004

0.673

0.277

0.199

±.005

±.010

—

0.260

NR ↓

— 0.397 0.429

0.198 0.579 0.333

±.005

0.778

0.516 0.389 0.556

±.013

0.356

0.266

0.018

±.005

±.002

0.787

0.235

0.218

±.005

±.011

0.714 0.833

∗ Best baseline per task: unmarked = Random; a TF-IDF, b,c value overlap, d Jaccard. Table-level results report, for each model,

the best-performing supported table aggregation among CLS, COL - MEAN, and TOK - MEAN under the unified supervised-probe protocol of Sec. 3.1; full aggregation ablations in Tables 17 and 18.

within the relevant candidate set and evaluated once on test. Top-50 frequencies, per-stage marginals, and Oracle-RA (Stages 1–2 replaced by ground truth) are test-set descriptive analyses over the full pipeline space. Stage-2 thresholds are calibrated on development with macro-F1 , independently of headline pipeline selection (Appendix M). NR averages a model’s normalized rank over units of an aggregate (tasks within a CTBench family, target columns, or linkage datasets), excluding missing units. Formulas and standard metric definitions are in Appendix H. In all tables, ↓ marks lower-is-better. The colors orange / blue / light purple highlight ranks 1/2/3 per column, and † marks table-disjoint splits. 4.2

Column- and Table-Level Results

Table 2 reports the main column/table results across 13 TRL-CT BENCH tasks, evaluated on the 10 of 20 models that natively expose column or table embeddings, spanning schema understanding, joinability, unionability, and grounding. Two empirical patterns stand out. Generic-text rankings track surface-text signal. Family-level NR (lower is better) for BERT and GTE worsens from Schema through Grounding (BERT 0.000 → 0.048 → 0.260 → 0.397, and GTE 0.190 → 0.243 → 0.343 → 0.429). A task-level surface-text audit (Appendix K.1) is consistent with this interpretation: generic text encoders are strongest on tasks where headers and short cell strings carry most of the signal, and the CTBench tasks where a tabular specialist beats them (Union Search, Schema Matching, Table Subset, and Table QA) all sit in the Union and Grounding families, which instead reward cross-table alignment geometry and grounded table understanding. Pretraining alignment matters beyond surface cues. The four CTBench tasks won by tabular specialists are each consistent with their winner’s pretraining design. Column-centric S TARMIE’s contrastive objective matches the cosine-scoring setup for Union Search (0.662 MAP) and Schema Matching (0.764 R@GT). On Table Subset, the top three are all tabular specialists (TAPAS 0.567 F1 , TAPEX 0.558, TAB S KETCH FM 0.553), placing structural pretraining above text serialization on this task. In Grounding, the Table-Text family leads at the family level: TA BERT’s joint text-table pretraining delivers the best Grounding NR (0.198), while task-level wins split between structureaware TURL (Table QA, 0.277 Acc) and generic-text GTE (Table Retrieval, 0.476 MRR). Although GTE is classified as a generic text model, it is pretrained with a retrieval-contrastive objective [50]. Its Table Retrieval win is itself an instance of pretraining-task alignment rather than an exception. 4.3

Row-Level Results

Table 3 reports row-level transfer under the shared probe protocol of Sec. 3.1. Three empirical patterns stand out. 7

Table 3: Row-level results across four evaluation categories. Row prediction averages over 77 classification and 46 regression targets (TAB T RANSFORMER: 63 targets due to categorical-feature requirement). SGM is SGM0.01 (nRMSE). Linkage columns report binary F1 (match class) on DM-C, DM-D, and WDC. See Appendix H for the convention. Rank columns aggregate ranks over individual targets (classification, regression) or datasets (linkage). Formula of metrics in Appendix H. Classification

Regression

Clean Linkage

Robust Linkage

Family

Model

AUROC ↑

Macro F1 ↑

NR ↓

SGM ↓

NR ↓

DM-C F1 ↑

NR ↓

DM-D F1 ↑

WDC F1 ↑

NR ↓

Baseline

Dummy Random

0.500 ±.000 0.506 ±.001

0.304 ±.000 0.348 ±.001

— —

1.004 ±.000 1.103 ±.000

— —

0.000 ±.000 0.179 ±.007

— —

0.000 ±.000 0.223 ±.003

0.000 ±.000 0.128 ±.003

— —

TransferBased

BERT GTE TABBIE TUTA

0.791 ±.000 0.770 ±.000 0.770 ±.001 0.720 ±.000

0.635 ±.000 0.610 ±.001 0.599 ±.001 0.553 ±.002

0.378 0.544 0.541 0.551

0.704 ±.002 0.765 ±.001 0.766 ±.003 0.725 ±.003

0.559 0.714 0.643 0.632

0.418 ±.004 0.392 ±.006 0.365 ±.005 0.377 ±.011

0.096 0.173 0.250 0.231

0.464 ±.005 0.516 ±.005 0.330 ±.011 0.451 ±.006

0.236 ±.003 0.311 ±.001 0.140 ±.009 0.227 ±.005

0.163 0.048 0.404 0.154

PriorBased

TabICL TabPFN

0.816 ±.001 0.793 ±.001

0.671 ±.002 0.621 ±.002

0.164 0.492

0.505 ±.001 0.607 ±.002

0.139 0.499

0.316 ±.011 0.254 ±.008

0.423 0.596

0.318 ±.006 0.251 ±.023

0.147 ±.006 0.087 ±.010

0.394 0.663

Target-Table Learners

VIME SCARF DAE TabBinning SAINT SubTab TabTransformer TransTab

0.794 ±.000 0.794 ±.001 0.793 ±.001 0.792 ±.001 0.768 ±.002 0.731 ±.001 0.768 ±.003 0.778 ±.001

0.640 ±.001 0.642 ±.001 0.643 ±.001 0.640 ±.001 0.595 ±.004 0.550 ±.002 0.594 ±.004 0.608 ±.001

0.385 0.371 0.379 0.396 0.543 0.798 0.497 0.477

0.556 ±.003 0.571 ±.002 0.575 ±.002 0.573 ±.002 0.617 ±.009 0.782 ±.003 0.666 ±.003 0.611 ±.020

0.367 0.399 0.392 0.397 0.561 0.779 0.447 0.441

0.257 ±.026 0.266 ±.010 0.241 ±.013 0.256 ±.011 0.167 ±.047 0.094 ±.005 0.083 ±.017 0.339 ±.009

0.529 0.510 0.644 0.615 0.712 0.933 0.942 0.346

0.259 ±.027 0.258 ±.021 0.252 ±.022 0.279 ±.025 0.176 ±.048 0.121 ±.005 0.089 ±.041 0.423 ±.032

0.099 ±.006 0.069 ±.008 0.098 ±.004 0.068 ±.004 0.133 ±.004 0.009 ±.005 0.020 ±.004 0.400 ±.025

0.596 0.683 0.615 0.673 0.606 0.962 0.942 0.096

Prediction and linkage decouple by model family. Prior-based TAB ICL leads prediction (AUROC 0.816, Macro-F1 0.671, SGM 0.505), while linkage leaders are dominated by transfer-oriented encoders: BERT on Clean Linkage (F1 0.418, NR 0.096) and GTE on Robust Linkage (NR 0.048), with T RANS TAB second on Robust Linkage (NR 0.096) via its cross-table contrastive objective. Target-table SSL methods are generally competitive on prediction but weak on linkage. Intra- and inter-table transfer follows training scope. Row prediction operates within a table (intra-table transfer), while record linkage operates across tables (inter-table transfer). Target-table SSL methods, trained from scratch on each target, fit locally: they are competitive on prediction (mean NR 0.48/0.47 on classification/regression) but trail on linkage (0.65/0.65 on Clean/Robust). Transfer-based encoders, applying one shared model to every table, produce comparable row spaces: they lead linkage (mean NR 0.19/0.19) but sit mid-pack on prediction (0.50/0.64). Two designs combine both axes. T RANS TAB layers a cross-table contrastive objective onto per-table SSL, taking second on Robust Linkage (NR 0.096) while staying competitive on prediction. TAB ICL layers target-table adaptation onto a shared meta-pretrained prior, leading prediction and ranking 5th of 14 on Robust Linkage (NR 0.394). Combining intra-table and inter-table transfer is thus hard but achievable through different design paths, an open direction for further study. Geometric diagnostics cross-validate the task-based rankings. Alongside the task-based protocol, we also evaluate exported row embeddings through task-free geometric diagnostics (Appendix L.5). The two axes agree: row-linkage utility correlates with embedding anisotropy at |ρ̄| ≈ 0.80 (Linear head, αreq ), and regression utility correlates with effective rank at |ρ̄| ≈ 0.36 (MLP head, RankMe⋆ ). See Figure 9. Intrinsic embedding geometry thus offers a task-free lens on row-level transfer, with broadly consistent agreement across the full diagnostic family. 4.4

Compositional Results on TRL-DLTE

We analyze the full 10 × 8 × 14 = 1120 pipeline space (Stage-1 table encoders × Stage-2 column models × Stage-3 row models, with pool composition given in Appendix N). Table 4 and Figure 3 summarize the resulting top-50 memberships, per-stage marginals, and full pipeline landscape. Three findings matter most. Capability-matched hybrids beat single-encoder reuse. Under this dev-selection protocol, the best hybrid TUTA/GTE/GTE reaches 0.229 UJ-H, 0.090 above the best dev-selected monolithic BERT/BERT/BERT (0.139). Development and test rankings are similar over the 1,120 pipelines (Spearman ρ = 0.96, top-50 overlap 42/50, see Appendix N), so the dev-selected hybrid result is consistent with the broader test landscape. Frontier presence tracks atomic-task leadership at Stages 2–3 (Table 4): the Stage-2 frontier picks all lead at least one CTBench task (e.g., BERT on Schema NR and Join NR, GTE on Join Search and Table Retrieval, TURL on Table QA), and the Stage-3 top-three frontier picks (GTE, T RANS TAB, TAB ICL) are exactly the top three row models 8

on Oracle-RA (Appendix N.4). GTE and T RANS TAB additionally take the top two slots on Robust Linkage NR, with TUTA third. Table 4: Per-stage view of the DLTE Compositional fit shapes pipeline quality. Atomic pipeline space (5-round avg, test set). Top 5 strength is thus necessary for frontier presence but not models per stage by top-50 frequency. U Jsufficient for the best assembly. Per-stage marginals H is the per-stage marginal over all 1,120 summarize average main effects, not optimal com- pipelines. Bottom row lists marginal leaders. positions. The test Stage-1/2/3 marginal leaders asStage Model # % U J-H semble to S TARMIE/TABBIE/T RANS TAB at 0.134 Starmie 19 38% 0.144 UJ-H (Fig. 3), well below both the test rank-1 TUTA 18 36% 0.138 pipeline S TARMIE/GTE/GTE (0.253) and the dev4 8% 0.127 Stage 1 (Tbl) TAPEX GTE 3 6% 0.129 selected hybrid TUTA/GTE/GTE (0.229). DevelBERT 3 6% 0.128 opment marginals assemble to a different pipeline, TURL 16 32% 0.143 S TARMIE/BERT/T RANS TAB, scoring 0.231 on test GTE 16 32% 0.141 10 20% 0.135 Stage 2 (Col) BERT (Fig. 3). Marginals therefore carry signal, but the TAPAS 7 14% 0.132 0.097 swing between the two marginal assemblies, deTaBERT 1 2% 0.128 spite this split stability, shows that top marginal ranks TransTab 12 24% 0.132 GTE 12 24% 0.131 are a lossy selection rule. Two decouplings in Table 4 12 24% 0.130 Stage 3 (Row) TabICL explain why. At Stage 1, atomic retrieval strength TUTA 7 14% 0.130 BERT 6 12% 0.128 decouples from compositional utility: GTE leads CTBench Table Retrieval (0.476 MRR, Table 2) and Per-stage marginal leaders (U J-H, unrounded means): Starmie 0.144 / TABBIE 0.143 / TransTab 0.132. DLTE target recall@100 (0.801), yet ranks only third on the Stage-1 UJ-H marginal, behind S TARMIE and TUTA. At Stage 2, average marginal strength decouples from top-pipeline membership: TABBIE leads the Stage-2 marginal (unrounded) but never enters the top-50, where GTE and TURL dominate. DLTE therefore rewards compositional fit, defined here as non-additive compatibility among retrieval, column alignment, and row matching, not independent per-stage rank alone. These patterns hold separately on TabFact-only and WTQ-only parents (Spearman ρ = 0.871 cross-source rank agreement across all 1,120 canonical pipelines, see Appendix N.8). A shared identity-resolution capability across RBench and DLTE. DLTE Stage 3 (row matching) is the compositional counterpart of RBench’s Stage 1 Record Linkage task: both test whether exported Stage 2 row embeddings can resolve cross-table row identity under noise. Oracle-RA (Appendix N.4) shows that the two settings share a strong rowStage 3 GTE model rank signal: Stage-3 row-model rankings TURL agree with the RBench Robust Linkage NR rankTABBIE Star./BERT/TrTab=0.231 ing at Spearman |ρ| = 0.80 (p = 6.3×10−4 ). The agreement is strongest at the top two modBERT / BERT / BERT = 0.139 els, GTE and T RANS TAB, with only a small middle-pack shift at TAB ICL (third on OracleTUTA / GTE / GTE = 0.229 Star./TABBIE/TrTab=0.134 RA versus fifth on Robust Linkage NR). This Figure 3: DLTE pipeline landscape (test split). identifies a shared identity-resolution capability Axes sort by per-stage marginal U J-H. Color of frozen row embeddings, surfaced consistently encodes end-to-end U J-H. Halo boxes mark by both atomic record linkage (RBench) and compositional row matching (DLTE Stage 3). This marginal and dev-selected compositions. Stage-3 row-model ranking persists across all four DLTE noise tiers (Appendix N). 4.5

Three Open Gaps Across the Suites

Read jointly, the three suites expose three structural gaps that single-paradigm or single-granularity evaluations cannot isolate. At the level of model choice, TRL-CT BENCH (Sec. 4.2) reveals a specialization gap, where no pretraining recipe behaves like a universal representation. At the level of transfer, TRL-R BENCH (Sec. 4.3) reveals a transfer-scope gap, where intra-table adaptation and cross-table comparability pull learning in different directions. At the level of deployment, TRLDLTE (Sec. 4.4) reveals a composition gap, where granularity choices interact rather than stack independently. Together, these mark where tabular representation research has yet to converge on a unified account. 9

5

Conclusion

TRL-B ENCH reframes tabular encoder evaluation around the artifact many downstream systems actually reuse: exported embeddings. Under a single representation-level protocol, heterogeneous encoders become comparable without conflating their embeddings with task-specific wrappers, retraining budgets, or adaptation. By making this setting measurable across diverse encoders, datasets, and downstream uses, TRL-B ENCH provides a common reference point for building tabular models as portable representation learners rather than one-off task solvers.

References [1] Kumar Krishna Agrawal, Arnab Kumar Mondal, Arna Ghosh, and Blake Richards. α-ReQ: Assessing representation quality in self-supervised learning by measuring eigenspectrum decay. In NeurIPS, 2022. [2] Guillaume Alain and Yoshua Bengio. Understanding intermediate layers using linear classifier probes. arXiv preprint arXiv:1610.01644, 2016. [3] Alcoholrithm. TabularS3L: A PyTorch Lightning-based library for self- and semi-supervised learning on tabular data. https://github.com/Alcoholrithm/TabularS3L, 2024. [4] Alessio Ansuini, Alessandro Laio, Jakob H. Macke, and Davide Zoccolan. Intrinsic dimension of data representations in deep neural networks. In NeurIPS, 2019. [5] Gilbert Badaro, Mohammed Saeed, and Paolo Papotti. Transformers for tabular data representation: A survey of models and applications. Transactions of the Association for Computational Linguistics, 11:227–249, 2023. [6] Dara Bahri, Heinrich Jiang, Yi Tay, and Donald Metzler. SCARF: Self-supervised contrastive learning using random feature corruption. In ICLR, 2022. [7] Yoshua Bengio, Aaron Courville, and Pascal Vincent. Representation learning: A review and new perspectives. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(8): 1798–1828, 2013. [8] Bernd Bischl, Giuseppe Casalicchio, Matthias Feurer, Pieter Gijsbers, Frank Hutter, Michel Lang, Rafael Gomes Mantovani, Jan N. van Rijn, and Joaquin Vanschoren. OpenML benchmarking suites. In NeurIPS Datasets and Benchmarks Track, 2021. [9] Alex Bogatu, Alvaro A. A. Fernandes, Norman W. Paton, and Nikolaos Konstantinou. Dataset discovery in data lakes. In ICDE, pages 709–720, 2020. [10] Vadim Borisov, Kathrin Seßler, Tobias Leemann, Martin Pawelczyk, and Gjergji Kasneci. Language models are realistic tabular data generators. In ICLR, 2023. [11] Vadim Borisov, Tobias Leemann, Kathrin Seßler, Johannes Haug, Martin Pawelczyk, and Gjergji Kasneci. Deep neural networks and tabular data: A survey. IEEE Transactions on Neural Networks and Learning Systems, 35(6):7499–7519, 2024. [12] Jintai Chen, Jiahuan Yan, Qiyuan Chen, Danny Ziyi Chen, Jian Wu, and Jimeng Sun. ExcelFormer: A neural network surpassing GBDTs on tabular data. arXiv preprint arXiv:2301.02819, 2023. [13] Wenhu Chen, Hongmin Wang, Jianshu Chen, Yunkai Zhang, Hong Wang, Shiyang Li, Xiyou Zhou, and William Yang Wang. TabFact: A large-scale dataset for table-based fact verification. In ICLR, 2020. [14] Victor Chernozhukov, Denis Chetverikov, Mert Demirer, Esther Duflo, Christian Hansen, Whitney Newey, and James Robins. Double/debiased machine learning for treatment and structural parameters. The Econometrics Journal, 2018. [15] Tianji Cong, Madelon Hulsebos, Zhenjie Sun, Paul Groth, and H. V. Jagadish. Observatory: Characterizing embeddings of relational tables. Proceedings of the VLDB Endowment, 17(4): 849–862, 2023. 10

[16] Janez Demšar. Statistical comparisons of classifiers over multiple data sets. Journal of Machine Learning Research, 7:1–30, 2006. [17] Xiang Deng, Huan Sun, Alyssa Lees, You Wu, and Cong Yu. TURL: Table understanding through representation learning. Proceedings of the VLDB Endowment, 14(3):307–319, 2020. [18] Yuhao Deng, Chengliang Chai, Lei Cao, Qin Yuan, Siyuan Chen, Yanrui Yu, Zhaoze Sun, Junyi Wang, Jiajun Li, Ziqi Cao, Kaisen Jin, Chi Zhang, Yuqing Jiang, Yuanfang Zhang, Yuping Wang, Ye Yuan, Guoren Wang, and Nan Tang. LakeBench: A benchmark for discovering joinable and unionable tables in data lakes. Proceedings of the VLDB Endowment, 17(8):1925–1938, 2024. doi: 10.14778/3659437.3659448. [19] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In NAACL, 2019. [20] Nick Erickson, Jonas Mueller, Alexander Shirkov, Hang Zhang, Pedro Larroy, Mu Li, and Alexander Smola. AutoGluon-Tabular: Robust and accurate AutoML for structured data. arXiv preprint arXiv:2003.06505, 2020. [21] Nick Erickson, Lennart Purucker, Andrej Tschalzev, David Holzmüller, Prateek Mutalik Desai, David Salinas, and Frank Hutter. TabArena: A living benchmark for machine learning on tabular data. In NeurIPS Datasets and Benchmarks Track, 2025. [22] Elena Facco, Maria d’Errico, Alex Rodriguez, and Alessandro Laio. Estimating the intrinsic dimension of datasets by a minimal neighborhood information. Scientific Reports, 7(1):12140, 2017. [23] Grace Fan, Jin Wang, Yuliang Li, and Renée J. Miller. Table discovery in data lakes: State-ofthe-art and future directions. In SIGMOD Companion, 2023. [24] Grace Fan, Jin Wang, Yuliang Li, Dan Zhang, and Renée J. Miller. Semantics-aware dataset discovery from data lakes with contextualized column-based representation learning. Proceedings of the VLDB Endowment, 16(7):1726–1739, 2023. [25] Sebastian Fischer, Liana Harutyunyan, Matthias Feurer, and Bernd Bischl. OpenML-CTR23: A curated tabular regression benchmarking suite. In AutoML Conference, 2023. [26] Quentin Garrido, Randall Balestriero, Laurent Najman, and Yann LeCun. RankMe: Assessing the downstream performance of pretrained self-supervised representations by their rank. In ICML, 2023. [27] Yury Gorishniy, Ivan Rubachev, Valentin Khrulkov, and Artem Babenko. Revisiting deep learning models for tabular data. In NeurIPS, 2021. [28] Yury Gorishniy, Ivan Rubachev, Nikolay Kartashev, Daniil Shlenskii, Akim Kotelnikov, and Artem Babenko. TabR: Tabular deep learning meets nearest neighbors. In ICLR, 2024. [29] Jonathan Herzig, Pawel Krzysztof Nowak, Thomas Müller, Francesco Piccinno, and Julian Martin Eisenschlos. TaPas: Weakly supervised table parsing via pre-training. In ACL, 2020. [30] Jonathan Herzig, Thomas Müller, Syrine Krichene, and Julian Eisenschlos. Open domain question answering over tables via dense retrieval. In NAACL, 2021. [31] Noah Hollmann, Samuel Müller, Katharina Eggensperger, and Frank Hutter. TabPFN: A transformer That solves small tabular classification problems in a second. In ICLR, 2023. [32] Noah Hollmann, Samuel Müller, Lennart Purucker, Arjun Krishnakumar, Max Körfer, Shi Bin Hoo, Robin Tibor Schirrmeister, and Frank Hutter. Accurate predictions on small data with a tabular foundation model. Nature, 2025. [33] Xin Huang, Ashish Khetan, Milan Cvitkovic, and Zohar Karnin. TabTransformer: Tabular data modeling using contextual embeddings. arXiv preprint arXiv:2012.06678, 2020. [34] Hiroshi Iida, Dung Thai, Varun Manjunatha, and Mohit Iyyer. TABBIE: Pretrained representations of tabular data. In NAACL, 2021. 11

[35] Zhengbao Jiang, Yi Mao, Pengcheng He, Graham Neubig, and Weizhu Chen. OmniTab: Pretraining with natural and synthetic data for few-shot table-based question answering. In NAACL, 2022. [36] Ernesto Jiménez-Ruiz, Oktie Hassanzadeh, Vasilis Efthymiou, Jiaoyan Chen, and Kavitha Srinivas. SemTab 2019: Resources to benchmark tabular data to knowledge graph matching systems. In ESWC, pages 514–530, 2020. doi: 10.1007/978-3-030-49461-2_30. [37] Jeff Johnson, Matthijs Douze, and Hervé Jégou. Billion-scale similarity search with GPUs. IEEE Transactions on Big Data, 7(3):535–547, 2021. [38] James Jordon, Jinsung Yoon, and Mihaela van der Schaar. PATE-GAN: Generating synthetic data with differential privacy guarantees. In ICLR, 2019. [39] Aamod Khatiwada, Grace Fan, Roee Shraga, Zixuan Chen, Wolfgang Gatterbauer, Renée J. Miller, and Mirek Riedewald. SANTOS: Relationship-based semantic table union search. In SIGMOD, 2023. [40] Aamod Khatiwada, Harsha Kokel, Ibrahim Abdelaziz, Subhajit Chaudhury, Julian Dolby, Oktie Hassanzadeh, Zhenhan Huang, Tejaswini Pedapati, Horst Samulowitz, and Kavitha Srinivas. TabSketchFM: Sketch-based tabular representation learning for data discovery over data lakes. In ICDE, 2025. [41] Myung Jun Kim, Léo Grinsztajn, and Gaël Varoquaux. CARTE: Pretraining and transfer for tabular learning. In ICML, 2024. [42] Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In ICLR, 2015. [43] Keti Korini, Ralph Peeters, and Christian Bizer. SOTAB: The WDC Schema.org table annotation benchmark. In SemTab @ ISWC, 2022. [44] Akim Kotelnikov, Dmitry Baranchuk, Ivan Rubachev, and Artem Babenko. TabDDPM: Modelling tabular data with diffusion models. In ICML, 2023. [45] Christos Koutras, George Siachamis, Andra Ionescu, Kyriakos Psarakis, Jerry Brons, Marios Fragkoulis, Christoph Lofi, Angela Bonifati, and Asterios Katsifodimos. Valentine: Evaluating matching techniques for dataset discovery. In ICDE, 2021. [46] Harold W. Kuhn. The Hungarian method for the assignment problem. Naval Research Logistics Quarterly, 2(1–2):83–97, 1955. [47] Guillaume Lample, Alexis Conneau, Marc’Aurelio Ranzato, Ludovic Denoyer, and Hervé Jégou. Word translation without parallel data. In ICLR, 2018. [48] Kyungeun Lee, Ye Seul Sim, Hye-Seung Cho, Moonjung Eo, Suhee Yoon, Sanghyu Yoon, and Woohyung Lim. Binning as a pretext task: Improving self-supervised learning in tabular domains. In ICML, 2024. [49] Yuliang Li, Jinfeng Li, Yoshihiko Suhara, AnHai Doan, and Wang-Chiew Tan. Deep entity matching with pre-trained language models. Proceedings of the VLDB Endowment, 14(1): 50–60, 2020. [50] Zehan Li, Xin Zhang, Yanzhao Zhang, Dingkun Long, Pengjun Xie, and Meishan Zhang. Towards general text embeddings with multi-stage contrastive learning. arXiv preprint arXiv:2308.03281, 2023. [51] Fei Tony Liu, Kai Ming Ting, and Zhi-Hua Zhou. Isolation forest. In ICDM, 2008. [52] Qian Liu, Bei Chen, Jiaqi Guo, Morteza Ziyadi, Zeqi Lin, Weizhu Chen, and Jian-Guang Lou. TAPEX: Table pre-training via learning a neural SQL executor. In ICLR, 2022. [53] Sidharth Mudgal, Han Li, Theodoros Rekatsinas, AnHai Doan, Youngchoon Park, Ganesh Krishnan, Rohit Deep, Esteban Arcaute, and Vijay Raghavendra. Deep learning for entity matching: A design space exploration. In SIGMOD, 2018. 12

[54] Fatemeh Nargesian, Erkang Zhu, Ken Q. Pu, and Renée J. Miller. Table union search on open data. Proceedings of the VLDB Endowment, 11(7):813–825, 2018. [55] Arvind Neelakantan, Tao Xu, Raul Puri, Alec Radford, Jesse Michael Han, Jerry Tworek, Qiming Yuan, Nikolas Tezak, Jong Wook Kim, Chris Hallacy, Johannes Heidecke, Pranav Shyam, Boris Power, Tyna Eloundou Nekoul, Girish Sastry, Gretchen Krueger, David Schnurr, Felipe Petroski Such, Kenny Hsu, Madeleine Thompson, Tabarak Khan, Toki Sherbakov, Joanne Jang, Peter Welinder, and Lilian Weng. Text and code embeddings by contrastive pre-training. arXiv preprint arXiv:2201.10005, 2022. [56] Jianmo Ni, Gustavo Hernández Ábrego, Noah Constant, Ji Ma, Keith B. Hall, Daniel Cer, and Yinfei Yang. Sentence-T5: Scalable sentence encoders from pre-trained text-to-text models. In Findings of ACL, 2022. [57] OpenAI. New embedding models and API updates. https://openai.com/index/ new-embedding-models-and-api-updates/, 2024. Released January 25, 2024. [58] Koyena Pal, Aamod Khatiwada, Roee Shraga, and Renée J. Miller. Generative benchmark creation for table union search. arXiv preprint arXiv:2308.03883, 2023. [59] Wei Pang, Masoumeh Shafieinejad, Lucy Liu, Stephanie Hazlewood, and Xi He. ClavaDDPM: Multi-relational data synthesis with cluster-guided diffusion models. In NeurIPS, 2024. [60] Panupong Pasupat and Percy Liang. Compositional semantic parsing on semi-structured tables. In ACL, 2015. [61] Neha Patki, Roy Wedge, and Kalyan Veeramachaneni. The synthetic data vault. In DSAA, 2016. [62] Ralph Peeters, Anna Primpeli, Benedikt Wichtlhuber, and Christian Bizer. Using schema.org annotations for training and maintaining product matchers. In WIMS, 2020. [63] Anna Primpeli, Ralph Peeters, and Christian Bizer. The WDC training dataset and gold standard for large-scale product matching. In Companion of The 2019 World Wide Web Conference (WWW ’19 Companion), ECNLP Workshop, 2019. [64] Jingang Qu, David Holzmüller, Gaël Varoquaux, and Marine Le Morvan. TabICL: A tabular foundation model for in-context learning on large data. In ICML, 2025. [65] Ravid Shwartz-Ziv and Amitai Armon. Tabular data: Deep learning is not all you need. Information Fusion, 81:84–90, 2022. [66] Gowthami Somepalli, Micah Goldblum, Avi Schwarzschild, C. Bayan Bruss, and Tom Goldstein. SAINT: Improved neural networks for tabular data via row attention and contrastive pre-training. arXiv preprint arXiv:2106.01342, 2021. [67] Kaitao Song, Xu Tan, Tao Qin, Jianfeng Lu, and Tie-Yan Liu. MPNet: Masked and permuted pre-training for language understanding. In NeurIPS, 2020. [68] Kavitha Srinivas, Julian Dolby, Ibrahim Abdelaziz, Oktie Hassanzadeh, Harsha Kokel, Aamod Khatiwada, Tejaswini Pedapati, Subhajit Chaudhury, and Horst Samulowitz. LakeBench: Benchmarks for data discovery over data lakes. arXiv preprint arXiv:2307.04217, 2023. [69] Aofeng Su, Aowen Wang, Chao Ye, et al. TableGPT2: A large multimodal model with tabular data integration. arXiv preprint arXiv:2411.02059, 2024. [70] Yoshihiko Suhara, Jinfeng Li, Yuliang Li, Dan Zhang, Çağatay Demiralp, Chen Chen, and Wang-Chiew Tan. Annotating columns with pre-trained language models. In SIGMOD, 2022. [71] Anton Tsitsulin, Marina Munkhoeva, and Bryan Perozzi. Unsupervised embedding quality evaluation. arXiv preprint arXiv:2305.16562, 2023. [72] Talip Ucar, Ehsan Hajiramezanali, and Lindsay Edwards. SubTab: Subsetting features of tabular data for self-supervised representation learning. In NeurIPS, 2021. 13

[73] Joaquin Vanschoren, Jan N. van Rijn, Bernd Bischl, and Luis Torgo. OpenML: Networked science in machine learning. ACM SIGKDD Explorations Newsletter, 15(2):49–60, 2014. [74] Pascal Vincent, Hugo Larochelle, Yoshua Bengio, and Pierre-Antoine Manzagol. Extracting and composing robust features with denoising autoencoders. In ICML, 2008. [75] Zhiruo Wang, Haoyu Dong, Ran Jia, Jia Li, Zhiyi Fu, Shi Han, and Dongmei Zhang. TUTA: Tree-based transformers for generally structured table pre-training. In KDD, 2021. [76] Zifeng Wang and Jimeng Sun. TransTab: Learning transferable tabular transformers across tables. In NeurIPS, 2022. [77] Lei Xu, Maria Skoularidou, Alfredo Cuesta-Infante, and Kalyan Veeramachaneni. Modeling tabular data using conditional GAN. In NeurIPS, 2019. [78] Pengcheng Yin, Graham Neubig, Wen tau Yih, and Sebastian Riedel. TaBERT: Pretraining for joint understanding of textual and tabular data. In ACL, 2020. [79] Jinsung Yoon, James Jordon, and Mihaela van der Schaar. GAIN: Missing data imputation using generative adversarial nets. In ICML, 2018. [80] Jinsung Yoon, Yao Zhang, James Jordon, and Mihaela van der Schaar. VIME: Extending the success of self- and semi-supervised learning to tabular domain. In NeurIPS, 2020. [81] Tao Yu, Rui Zhang, Kai Yang, Michihiro Yasunaga, Dongxu Wang, Zifan Li, James Ma, Irene Li, Qingning Yao, Shanelle Roman, Zilin Zhang, and Dragomir Radev. Spider: A large-scale human-labeled dataset for complex and cross-domain semantic parsing and text-to-SQL task. In EMNLP, 2018. [82] Dan Zhang, Yoshihiko Suhara, Jinfeng Li, Madelon Hulsebos, Çağatay Demiralp, and WangChiew Tan. Sato: Contextual semantic type detection in tables. Proceedings of the VLDB Endowment, 13(11):1835–1848, 2020. [83] Hengrui Zhang, Jiani Zhang, Balasubramaniam Srinivasan, Zhengyuan Shen, Xiao Qin, Christos Faloutsos, Huzefa Rangwala, and George Karypis. Mixed-type tabular data synthesis with score-based diffusion in latent space. In ICLR, 2024. [84] Jun Zhang, Graham Cormode, Cecilia M. Procopiuc, Divesh Srivastava, and Xiaokui Xiao. PrivBayes: Private data release via bayesian networks. ACM Transactions on Database Systems, 2017. [85] Tianshu Zhang, Xiang Yue, Yifei Li, and Huan Sun. TableLlama: Towards open large generalist models for tables. In NAACL, 2024. [86] Yi Zhang and Zachary G. Ives. Finding related tables in data lakes for interactive data science. In SIGMOD, 2020. [87] Bingzhao Zhu, Xingjian Shi, Nick Erickson, Mu Li, George Karypis, and Mahsa Shoaran. XTab: Cross-table pretraining for tabular transformers. In ICML, 2023. [88] Erkang Zhu, Dong Deng, Fatemeh Nargesian, and Renée J. Miller. JOSIE: Overlap set similarity search for finding joinable tables in data lakes. In SIGMOD, 2019.

14

Appendix A

Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

B

Extended Related Work and Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

C

Model Inventory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

D

Model Input Policy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

E

Appendix Task Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 E.1

Benchmark Protocol Adaptations under Frozen Multi-Granular Transfer . . . . . . 23

F

Full Dataset Inventory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

G

Task-Local Baselines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

H

Metric Definitions and Normalized Rank . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

I

Table-Footprint Coverage Across Suites . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

J

Family-Level Performance Summary Figure . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

K

CTBench Diagnostics and Ablations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

L

M

K.1

Observational Lexical-Accessibility Proxies for CTBench . . . . . . . . . . . . . . . . 36

K.2

Probe Head Complexity for Column/Table-Level Tasks . . . . . . . . . . . . . . . . . 36

K.3

Aggregation Ablation for Table-Level Embeddings . . . . . . . . . . . . . . . . . . . . 37

K.4

Join Search: Direct Cosine vs. Learned Projection . . . . . . . . . . . . . . . . . . . . . 37

K.5

Union Search: TUS vs. TUS-hard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

K.6

Table Retrieval: Model-Only vs. Hybrid Mode . . . . . . . . . . . . . . . . . . . . . . . 38

K.7

Query Encoder Sensitivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

K.8

Pair-Level Random vs. Table-Disjoint Split Ablation . . . . . . . . . . . . . . . . . . . 41

RBench Diagnostics and Ablations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 L.1

Row-Prediction Probe Diagnostics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

L.2

Embedding Dimension for Record Linkage . . . . . . . . . . . . . . . . . . . . . . . . . 45

L.3

Probe Head for Record Linkage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

L.4

Record Linkage Split and Leakage Audit . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

L.5

Intrinsic Embedding-Geometry Diagnostics for Row Encoders . . . . . . . . . . . . . 49

L.6

Intrinsic-Geometry Diagnostics: Per-Head Breakdowns . . . . . . . . . . . . . . . . . 53

DLTE Operator Specification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 M.1

Stage-1 Retrieval: Scoring and Pool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

M.2

Stage-2 Alignment and Classification: Thresholds and Grid . . . . . . . . . . . . . . 55

M.3

Stage-3 Row Matching: CSLS and Profiles . . . . . . . . . . . . . . . . . . . . . . . . . 56

M.4

Stage-3 Second-Pass Join on Union-Appended Rows . . . . . . . . . . . . . . . . . . 57

M.5

What is and is not tuned . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 15

N

DLTE Detailed Rankings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58 N.1

Cell F1 as a Complementary Diagnostic . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

N.2

Pipeline Component Sensitivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

N.3

Per-Stage Marginal Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

N.4

Oracle-RA Row-Model Diagnostic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62

N.5

Full Per-Stage Model Rankings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

N.6

Stage 3 Row Matching: Full Pipeline Rankings . . . . . . . . . . . . . . . . . . . . . . . 65

N.7

Top-20 Pipeline Combinations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67

N.8

Source Split: TabFact vs. WTQ . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68

O

Proprietary Embedding Ablation: Retrieval vs. Structural Grounding . . . . . . . . 70

P

Robustness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71 P.1

Unified Aggregation Protocol . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71

P.2

Task and Dataset Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72

P.3

Sample Fidelity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72

P.4

Perturbation Robustness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73

P.5

Row/Column Order Insignificance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73

P.6

Cross-Task Comparison of Shared Models . . . . . . . . . . . . . . . . . . . . . . . . . . 74

P.7

Implementation Notes and Caveats . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75

Q

Computational Efficiency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76

R

Reproducibility Details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78

S

Statistical Reporting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79

T

Dataset Counting Protocol . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82

U

Broader Impact . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83

V

Licenses and Asset Documentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84

16

A

Limitations

TRL-B ENCH is designed to standardize cross-paradigm comparison at the representation level, not to report the best end-to-end system for each task. Its scores therefore answer a narrower question, namely what common lightweight readouts can extract from exported embeddings, rather than replacing fully adapted-system benchmarks. The protocol standardizes task definitions and downstream evaluation, not raw model preprocessing: each encoder is run in its documented operating regime through its standard supported wrapper rather than a single forced serialization, which improves fidelity to the original models but leaves some wrapper-induced variation. Appendix D summarizes this policy, and the exact wrapper settings are documented in the released code. Finally, dataset counts and normalized-rank summaries are convenience aggregates and should be read alongside the per-task results.

17

B

Extended Related Work and Scope

This section provides the fuller narrative context omitted from the compressed main-text related-work discussion. Table 1 in the main paper remains the compact structural summary. Row-level tabular learning and evaluation. A large body of work in tabular deep learning focuses on row-level predictive modeling, using feature-token architectures, denoising losses, contrastive selfsupervision, cross-table transfer, or prior-based/meta-pretrained predictors [33, 66, 72, 48, 80, 6, 74, 76, 31, 32, 64]. See Borisov et al. [11] and Shwartz-Ziv and Armon [65] for surveys. Recent strong backbones such as FT-Transformer, ExcelFormer, TabR, CARTE, and XTab further strengthen this row-level tradition [27, 12, 28, 41, 87]. Because these methods are usually presented as end-to-end row predictors rather than as encoders exposing reusable row embeddings under a shared multigranular representation-level protocol, we treat them as important row-level references rather than as direct baselines for TRL-B ENCH’s encode-once comparison. These methods have substantially improved supervised tabular prediction, but their evaluation is still centered on row-level predictive benchmarks, which do not test whether one exported row representation can be reused across multiple targets from the same table or transferred to inter-table tasks such as record linkage. Column- and table-centric representation learning. A separate line of work studies schema semantics, table-language grounding, retrieval, and discovery. Generic text encoders can be applied to serialized tables [19, 50], while table-aware and column-aware models [29, 78, 52, 17, 75, 34, 24, 40] inject structural or contrastive inductive bias through pretraining. Their evaluations are typically task-specific (semantic typing, relation prediction, question answering, table retrieval, or data discovery) rather than unified transfer across columns, rows, and tables. Across both rowlevel and column/table-level work, evaluation remains largely single-granularity and task-specific, which makes cross-granularity transfer hard to assess. A notable exception is Observatory [15], which characterizes learned representations along perturbation- and invariance-style properties such as sample fidelity, perturbation robustness, and order insignificance, but does not relate those measurements to downstream transfer on the same encoder outputs. Our robustness appendix directly applies Observatory’s three diagnostics to the models in our benchmark pool, and Appendix L.5 additionally pairs downstream row-level evaluation with intrinsic embedding-geometry diagnostics (spectral spread, spectral shape, spatial structure) from the broader representation-learning literature on the same exported row embeddings used by the benchmark protocol, cross-validating the two views. The main benchmark suites themselves are organized around downstream task transfer under a shared representation-level protocol. Task-specific benchmarks and broader evaluation suites. The tabular ecosystem already contains strong benchmarks for semantic typing and schema understanding [36, 82, 43], schema matching [45], join and union discovery [39, 68, 18, 24], entity matching [53, 63], table question answering [60], question-to-table retrieval [30], and row prediction [73, 8, 25, 21]. LakeBench [68] is the closest prior resource, covering both join and union discovery with systematic model comparisons, but it does not evaluate row-level transfer, record linkage, or multi-stage enrichment composition, and it does not separate training regimes. TabArena [21] provides carefully curated row-level benchmarking with living maintenance, but evaluates supervised end-to-end prediction rather than frozen-representation transfer. These resources are indispensable ingredients for TRL-B ENCH, but they differ substantially in split design, task formulation, supervision, and permitted adaptation, and they generally do not combine representation-level downstream evaluation with explicit separation of training regimes under a common protocol. Prior work on data lakes and table augmentation, including discovery systems such as D3L [9] and Juneau [86], typically evaluates retrieval, join recommendation, or schema alignment stages in isolation [39, 68, 18, 24]. TRL-DLTE instead benchmarks their composition under a common representation-centric protocol. Scope and model selection. TRL-B ENCH complements these model papers and task-specific benchmarks rather than replacing them. Rather than reproducing each task family’s strongest end-toend stack, it standardizes comparison around transferable signal already present in a representation. Accordingly, the main leaderboard focuses on models that can participate in the shared representationlevel protocol and keeps scale roughly controlled to the ∼1M–1B parameter band studied in this paper. End-to-end generative table LLMs and heavily task-specific fine-tuned systems (e.g., TableLlama, TableGPT2, Ditto, DODUO, OmniTab) are complementary but out of scope for this comparison [85, 18

69, 49, 70, 35]: the former typically operate at 7B+ scale and do not expose compatible stable multigranular embeddings, while the latter belong to a separate task-specific end-to-end paradigm. The benchmark differs from prior work in three ways. First, it is explicitly multi-granular across columns, rows, and tables. Second, it is explicitly cross-paradigm, comparing externally pretrained transfer, target-table self-supervision, and prior-based predictors under one interface. Third, it includes a compositional data-lake enrichment benchmark that tests whether strong atomic representations actually compose into a useful end-to-end system. Other tabular ML threads. Tabular data is studied across many research threads in addition to representation learning. Tabular synthesis spans GANs, diffusion models, language models, and relational generators [77, 44, 83, 10, 61, 59], with privacy-preserving variants targeting differentialprivacy guarantees [38, 84]. Other threads include AutoML and hyperparameter-optimization stacks for tabular prediction [20], causal inference and treatment-effect estimation on tabular covariates [14], anomaly and outlier detection [51], and missing-value imputation [79]. Although diverse in goal, several of these threads increasingly intersect with representation learning, whether through learned latent spaces in generative models or embedding-based scoring in anomaly and outlier detection, making the quality of reusable encoders a cross-cutting concern that TRL-B ENCH is designed to characterize.

19

C

Model Inventory

Table 5: Model inventory in TRL-B ENCH. Parameter counts come from the loaded checkpoint (frozen encoders), the official Python package (TabICL, TabPFN), or the trained model under the default training configuration (target-table SSL, per-model architectures in Appendix D). ‡ Retrained per dataset, reported at the median input-feature count (31) across the 50 TRL-R BENCH OpenML tables. Model

Granularity

Family

Adaptation

Pretraining source

Dim

Params

BERT GTE TabSketchFM TAPAS TaBERT TABBIE Starmie TURL TUTA TAPEX TabICL TabPFN SCARF SAINT SubTab TabTransformer TabBinning TransTab VIME DAE

Column, Row, Table Column, Row, Table Column, Table Column, Table Column, Table Column, Row, Table Column, Table Column, Table Table, Row Table Row Row Row Row Row Row Row Row Row Row

Generic text encoder Generic text encoder Column-specialized Table-Text Table-Text Structure-aware Column-specialized Structure-aware Structure-aware Table-Text Prior-based Prior-based Target-table SSL Target-table SSL Target-table SSL Target-table SSL Target-table SSL Target-table SSL Target-table SSL Target-table SSL

Frozen Frozen Frozen Frozen Frozen Frozen Frozen Frozen Frozen Frozen Meta-pretrained Meta-pretrained Target-table SSL Target-table SSL Target-table SSL Target-table SSL Target-table SSL Target-table SSL Target-table SSL Target-table SSL

General text General text Tables only Tables + NL Tables + NL Tables only Tables only (contrastive) Entity-linked tables Tables only Tables + SQL execution Tabular tasks/examples Synthetic priors Target-table unlabeled data Target-table unlabeled data Target-table unlabeled data Target-table unlabeled data Target-table unlabeled data Target-table unlabeled data Target-table unlabeled data Target-table unlabeled data

768 768 768 768 768 768 768 312 768 768 512 192 512 512 512 512 512 512 512 512

110M 110M 135M 111M 181M 170M 125M 314M 134M 139M 27M 11M 0.7M‡ 38M 0.8M‡ 0.7M‡ 0.7M‡ 19M 0.7M‡ 0.6M‡

20

D

Model Input Policy

Tabular encoders expose different native interfaces, for example serialized text, structured cells, or direct feature tensors, and therefore come with model-specific preprocessing and context regimes. In TRL-B ENCH, each encoder is evaluated under its standard public operating regime: the preprocessing and context configuration described in its source paper or released with its official code, using the paper-default or released-default settings whenever available. When several supported settings are possible, we choose the most permissive deterministic setting that stays within the model’s documented operating regime and preserves benchmark coverage. We deliberately do not force a single common input serialization or context length, because doing so would push most models outside the regime in which they were designed and validated, and would distort rather than equalize the comparison. The benchmark should therefore be read as a comparison of encoder packages under their supported interfaces, with the per-model launch scripts (exact tokenization, truncation, and configuration) provided in the codebase. Of the 20 models, 7 target-table SSL methods (VIME, SCARF, S UB TAB, TAB B INNING, TAB T RANSFORMER, SAINT, DAE) are implemented through the TabularS3L framework [3], and 2 generic text encoders (BERT, GTE) are applied to serialized tabular input. Wrappers also introduce model-specific choices that we note here. TABBIE and TUTA row embeddings are synthesized via per-row mini-tables (no published row-extraction head exists in either model). TAB PFN is run with ignore_pretraining_limits=True and TURL with max_entities=12000, both librarysupported settings that admit inputs beyond the models’ pretrained sizes. DAE’s wrapper uses TabularS3L’s tabular Swap corruption [80]; see also [74] for the original image-domain DAE. Permodel citations are in Table 6. Per-model launch scripts (exact tokenization, truncation, output extraction) live in the released codebase. For target-table SSL methods, the Table 5 parameter counts are measured from the trained model under the default training configuration: hidden size 512 with 3 hidden layers for the MLP-based learners; SAINT uses a 6-layer d=512 feature-token transformer; T RANS TAB uses a 2-layer d=512 encoder whose size is dominated by a ∼16M shared token-embedding table. Because these models are retrained per dataset, the backbone grows approximately linearly with input dimension (feature counts range from 7 to 1,777 across the 50 TRL-R BENCH OpenML tables), yielding an empirical upper bound of roughly 1.6–2.0M parameters for the largest-feature dataset.

21

Table 6: Per-model source provenance. The Source column lists the model’s source paper and (where relevant) the implementation library. Wrapper scripts (exact tokenization, truncation, output extraction) live in models/<model>/ in the released codebase. Wrapper-introduced choices are documented in the policy paragraph above. Model

Family

Source

BERT GTE

Generic text Generic text

[19] [50]

TAPAS TaBERT TAPEX

Table-Text Table-Text Table-Text

[29] [78] [52]

Starmie TabSketchFM

Column-spec. Column-spec.

[24] [40]

TABBIE TURL TUTA

Struct.-aware Struct.-aware Struct.-aware

[34] [17] [75]

TabICL TabPFN

Prior-based Prior-based

[64] [31, 32]

TransTab VIME SCARF SubTab TabBinning TabTransformer SAINT DAE

Target-table SSL Target-table SSL Target-table SSL Target-table SSL Target-table SSL Target-table SSL Target-table SSL Target-table SSL

[76] [80]; [3] [6]; [3] [72]; [3] [48]; [3] [33]; [3] [66]; [3] [74]; [80]; [3]

22

Table 7: Summary of column- and table-level tasks in TRL-B ENCH. Module form is the fine-grained downstream-module instantiation. These map to the three downstream-module types of Sec. 3.1: Geometry → training-free, Probe and Learned proj. → learned, and Dual proj. and Decoder → query-conditioned. DLTE’s Pipeline module (Table 8) is a stage-wise composition specified in Sec. 3.4. Split: † = table-disjoint (re-split by this project), orig. = table-disjoint in source data, – = no split (training-free task). Task

Level

Column Type Pred.

Column Probe

Module form Split

Column Clustering

Column Geometry

Protocol summary

Metric

Frozen column embeddings F1 with linear / MLP probe – Training-free clustering on NMI frozen column embeddings Original Ordered column-pair probe F1 over concatenated frozen embeddings Query-disjoint Retrieval over frozen col- MAP umn embeddings with a small learned projection Table-disjoint† Frozen column-pair regres- nRMSE sion probe – Retrieval under one-to-one MAP column alignment over frozen embeddings Training-free ranking of cross- R@GT – table column pairs by cosine similarity Table QA with frozen column Accuracy Original representations on the table side † Frozen table-pair probe with F1 Table-disjoint lightweight head † Frozen table-pair probe with F1 Table-disjoint lightweight head † Table-disjoint Frozen table-pair regression nRMSE probe Tbl-disjoint (orig.) Frozen table-pair probe with F1 lightweight head Original Dual-projection question-to- MRR table retrieval over frozen table embeddings Original

Column Relation Pred. Column Probe

Join Search

Column Learned proj.

Column Overlap

Column Probe

Union Search

Column Geometry

Schema Matching

Column Geometry

Table QA

Column Decoder

Join Classification

Table

Probe

Union Classification

Table

Probe

Union Regression

Table

Probe

Table Subset

Table

Probe

Table Retrieval

Table

Dual proj.

Sources SATO, SOTAB SATO, SOTAB WikiCT (relation)

OpenData variants

Wiki Containment SANTOS, UGEN, TUS

Valentine

WTQ

Spider Join Wiki Union ECB Union CKAN Subset NQ-Tables

Table 8: Summary of row-level and compositional tasks in TRL-B ENCH. See Table 7 caption for column definitions. Task

Level

Row Prediction

Module form Split

Row

Probe

Record Linkage Row

Probe

DLTE

Tbl + Col Pipeline + Row

Protocol summary

Metric

Macro-F1 , Original (OpenML) One frozen row embedding per AUROC, SGM record, reused across multiple targets from the same table Original (source) Pair classification over concate- F1 nated frozen row embeddings; headline averages linear and MLP probe heads Parent-disjoint

Sources 50 OpenML tables

DeepMatcher, WDC

Three-stage retrieval → align- UJ-H, Cell F1 TabFact, WTQ ment → merge over frozen encoder outputs; operators in Appendix M

E

Appendix Task Summary

E.1

Benchmark Protocol Adaptations under Frozen Multi-Granular Transfer

The protocol of Section 3.1 fixes the common evaluation infrastructure (frozen embeddings, shared lightweight readouts), but several reused source tasks need targeted protocol adaptations to remain meaningful tests of multi-granular representation transfer. Each paragraph below states one consideration, the choice we adopt in TRL-B ENCH, and the empirical evidence motivating that choice within this setting. Cross-table generalization in reused pair tasks. Affected tasks: Join Classification, Column Overlap, Union Classification, Union Regression. Choice: table-disjoint train/dev/test splits. Why: These source tasks were created for different objectives; for our transfer-oriented use, table-disjoint 23

Table 9: Grouping of the 16 benchmark tasks by downstream-module type (Sec. 3.1). The three primary module types are training-free, learned, and query-conditioned. DLTE is handled separately as a multi-stage pipeline (Sec. 3.4). The fine-grained Module form entries in Tables 7–8 refine this taxonomy. Downstream-module type

Tasks

Training-free Learned

Column Clustering; Union Search; Schema Matching Column Type Pred.; Column Relation Pred.; Join Search; Column Overlap; Join Classification; Union Classification; Union Regression; Table Subset; Row Prediction; Record Linkage Table QA; Table Retrieval DLTE

Query-conditioned Pipeline

splits ensure test tables are unseen during training and lower Join Classification F1 by 0.212 on average relative to pair-random splits (Table 23). High-overlap positives in union search. Affected task: Union Search (TUS). Choice: TUS-hard variant (containment ≥ 0.70 removed). Why: In our frozen retrieval setting, removing the highestoverlap positives helps distinguish lexical overlap from broader union signal; the value-overlap baseline drops from 1.000 to 0.008 and rankings shift substantially (Table 20). Degenerate or mislabeled targets. Affected task: Row Prediction. Choice: human review, label repair, and degeneracy audits on 158 candidate tables, retaining 50 for release. Why: Removes constant-column targets, near-duplicate targets, and labeling issues from the OpenML candidate pool before reusable row-transfer evaluation. Label-equivalent columns leaking match identity. Affected datasets: Record Linkage on WDC and Fodors–Zagats. Choice: remove cluster_id and identifiers (WDC) and class (Fodors– Zagats) before any encoder consumes a row. Why: These columns are deterministic functions of the match label; without removal, frozen text encoders trivially reach near-perfect F1 on these sources (Appendix L.4). Train/test row overlap in reused linkage sources. Affected task: Record Linkage. Choice: retain source pair-disjoint splits, audit per-source row overlap, and report a strict row-disjoint ablation on the 10 viable sources. Why: Pair-disjoint splits are the entity-matching canon; the strict ablation confirms rankings are stable across all 14 row models (Spearman ρ = 0.94, p = 5.6 × 10−7 ; Appendix L.4). End-to-end scoring across removed blocks. Affected task: DLTE. Choice: UJ-H (harmonic mean of union and join recall) as the primary metric. Why: UJ-H directly tracks recovery of both removed blocks; Cell F1 (pooled cell-recovery yield) is reported as a complementary diagnostic in Appendix N.1. Retrieval difficulty in DLTE. Affected task: DLTE (Stage 1). Choice: include 36,740 CKAN distractor tables in the lake. Why: A large distractor pool makes retrieval a meaningful tablerepresentation test; the lake contains 11,032 targets among 47,772 tables total. Shared query-side signal in hybrid retrieval. Affected task: Table Retrieval. Choice: model-only mode (no query-encoder table embedding concatenated on the table side). Why: Hybrid mode adds a strong common signal from the query encoder and compresses model differences into a narrow MRR band (Table 21), whereas model-only better isolates table-side transfer.

24

F

Full Dataset Inventory

CTBench datasets (20). Schema Understanding: SATO, SOTAB, WikiCT (relation). Joinability: OpenData (main), OpenData CAN, OpenData USA, OpenData UK/SG, Wiki Containment (wiki_containment), Spider Join (spider_join). Unionability: SANTOS, UGEN-v1, UGEN-v2, TUS, TUS-hard, Valentine, Wiki Union (wiki_union), ECB Union (ecb_union), CKAN Subset (ckan_subset). Grounding: WTQ (WikiTableQuestions), NQ-Tables. Row-Prediction dataset inventory real statistics from the 50 OpenML tables (a) Subject domain of the 50 source datasets

(b) Table-level task profile (50 tables)

Games & Other Education

Software & Security

Classification-only

Finance & Economics

4 2 (8.0%) (4.0%)

11 (22.0%)

12 (24.0%)

6 (12.0%)

3 (6.0%) Engineering & Industrial

6 (12.0%)

8 (16.0%) 6 (12.0%)

Natural Sciences

36 (72.0%)

Business & Marketing

6 (12.0%)

Regression-only

Mixed (CLF + REG)

Healthcare & Medicine

Figure 4: Row-Prediction dataset inventory. Real statistics computed from the 50 source datasets and their per-target metadata. (a) Subject-domain distribution of the 50 OpenML tables, hand-curated from each dataset’s public OpenML description: 12 Finance & Economics, 8 Business & Marketing, 6 Healthcare & Medicine, 6 Natural Sciences, 6 Engineering & Industrial, 6 Software & Security, 2 Education, and 4 Games & Other. (b) Table-level task profile of the 50 tables: 11 host only classification targets, 3 host only regression targets, and 36 host both classification and regression targets on the same table, enabling the “one frozen row embedding reused across multiple targets” protocol of TRL-B ENCH. Row prediction (50 OpenML tables, 123 targets). OpenML dataset IDs: 3, 38, 458, 1063, 1486, 4534, 6332, 40668, 40966, 40978, 44958, 44967, 44975, 44984, 44992, 46906, 46907, 46908, 46910, 46911, 46912, 46915, 46916, 46918, 46919, 46920, 46922, 46923, 46927, 46929, 46930, 46932, 46933, 46934, 46935, 46937, 46939, 46940, 46950, 46952, 46955, 46956, 46958, 46960, 46961, 46963, 46964, 46969, 46979, 46980. Per-table target counts range from 2 to 3 (77 classification + 46 regression). Sourced from TabArena, OpenML-CC18, and OpenML-CTR23; filtered from 158 candidates. Record linkage (16 datasets). DeepMatcher clean (8): amazon-google, beer, dblp-acm, dblpscholar, fodors-zagats, itunes-amazon, walmart-amazon, abt-buy. DeepMatcher dirty (4): dblp-acm, dblp-scholar, itunes-amazon, walmart-amazon. WDC Products (4 sizes): small (∼2.5K pairs), medium (∼8K pairs), large (∼18K pairs), xlarge (∼30K pairs). DLTE. Parent tables: 1,379 (989 from TabFact, 390 from WTQ). Split: 827/207/345 train/dev/test. Fragments: 5,516 seeds + 5,516 union targets + 5,516 join targets = 16,548 total. Distractors: 36,740 CKAN tables. Total lake: 47,772 tables. Noise tiers: clean, schema, cell, hard.

25

Table 10: Per-task applicability of task-local baselines in TRL-B ENCH. ✓ = baseline is evaluated on the task. Blank = not applicable. Coverage rationale is in Appendix G. Rnd. TF-IDF Chance Dum.

Inv. Hung. Cos. Jacc. Dist. Idx. Set Thr.

Task Column Clustering Column Type Prediction Column Relation Prediction Join Search (cosine) Join Search (learned) Column Overlap Union Search Schema Matching Table QA Join Classification Table Subset Union Classification Union Regression Table Retrieval Record Linkage Row Prediction

G

✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓

✓ ✓

✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓

✓ ✓ ✓ ✓ ✓ ✓

✓

✓ ✓ ✓ ✓ ✓ ✓

✓

Task-Local Baselines

Each downstream task in TRL-B ENCH includes simple task-local baselines in addition to learned encoders. These baselines serve three distinct roles. Embedding baselines (Random, TF-IDF) produce frozen vectors that flow through the same downstream pipeline as neural encoders. They test whether the pipeline itself drives performance rather than the embedding. Embedding-free baselines (InvertedIndex Containment, Hungarian Set Match, Jaccard and Distribution matchers from Valentine) bypass the pipeline and operate on raw table data. They represent classical, task-specific methods and provide a reference for what is achievable without learned representations. Analytical baselines (Chance, Dummy) compute expected performance from dataset statistics alone. Table 10 lists per-task applicability. The baselines themselves are documented below. Random embeddings. Random vectors of the same dimension as the neural encoder replace learned embeddings throughout the downstream pipeline, at every supported granularity (column, row, table). Any benchmark entry that significantly under-performs Random indicates a failure mode. Entries that match Random indicate that the probe head, not the encoder, is doing the work. Chance and Dummy. Chance is an analytical floor: for classification tasks, the expected score from random class assignment proportional to class frequencies; for retrieval, the expected recall from uniform random ranking. Dummy is the strictly stronger majority-class (classification) or mean (regression) predictor trained on frozen embeddings. It exposes cases where the label distribution alone is enough. Every supervised probe task reports a dummy head alongside the linear and MLP heads. TF-IDF embeddings. A character-n-gram TF-IDF vectorizer (charwb analyzer, range [3, 5], 256 dimensions) produces a per-column embedding from the column header concatenated with up to 50 sampled cell values. Fit per dataset. Restricted to the two column-level tasks with compatible serialization (column type prediction and column clustering) on the sato and SOTAB datasets. TF-IDF row embeddings. A row-level analogue of the column TF-IDF baseline, used as the string-similarity reference for record linkage. Each row is serialized as col: val | col: val | . . . (the same template used by the GTE / BERT row encoders), then character-n-gram TF-IDF (charwb , range [3, 5], 512 dimensions) is fit per dataset on the union of tableA and tableB rows so that paired rows live in a shared vocabulary. The resulting row vectors flow through the same downstream record-linkage probe as the neural row encoders, including all four heads (cosine threshold, linear, MLP, dummy). Because record linkage is dominated by surface character overlap on many sources 26

(Sec. 3.3), this baseline is the appropriate floor for a learned row encoder: an encoder that does not beat TF-IDF row at matching head type is not capturing match signal beyond raw character overlap. Jaccard token-overlap row embeddings. A second row-level non-neural baseline that complements char-TF-IDF row at the token level. Each row is serialized identically and tokenized into word unigrams. Rows are then represented as L2-normalized binary token-presence vectors (TfidfVectorizer(analyzer=’word’, binary=True, use_idf=False, norm=’l2’), 512 p dimensions). Cosine of two such vectors equals the Ochiai coefficient |A ∩ B|/ |A| · |B|, a token-overlap similarity that is a near-monotone transform of the token Jaccard |A ∩ B|/|A ∪ B|, so under the cosine-threshold head this baseline reads as a token-overlap threshold close to a Jaccard threshold. Under the learned MLP / linear heads the per-token presence indicators remain directly accessible to the probe. Together, char-TF-IDF row (sub-word) and Jaccard row (token-level) form the two-way string-similarity floor against which a neural row encoder must compete on linkage. On the canonical avg(MLP, linear) probe protocol, TF-IDF row reaches F1 = 0.380/0.495/0.227 on DM-C / DM-D / WDC, and Jaccard row reaches F1 = 0.353/0.481/0.255 on the same subsets, both well above the Random / Dummy floors (Table 12) and below the strongest learned encoders (Table 3). Inverted-Index Containment. An embedding-free join-search baseline. Given a query column Q and a lake column C, the score is the containment |Q ∩ C|/|Q| computed via an inverted index over normalized cell values (posting-list prune at 10,000 to remove ubiquitous values). This is a strong baseline in the JOSIE [88] lineage of work, since LakeBench-style ground truth is defined by value overlap. Hungarian Set Match. The union-search counterpart to Inverted-Index Containment. Columnto-column containment scores are assembled into a bipartite matrix and solved with the Hungarian algorithm [46]. The table-level score is the mean of matched column scores. Identical to the embedding-based union-search pipeline except cosine similarity is replaced by value containment. Valentine matchers (Jaccard, Distribution). Two embedding-free schema-matching baselines from the Valentine library. Jaccard scores column pairs by a weighted combination of header character 3-gram Jaccard and value-set Jaccard. Distribution scores by comparing value distributions (KS/EMD for numerical columns, frequency for categorical). Both are evaluated on the Valentine benchmark under the same Recall@GT metric as embedding-based schema matching. Cosine-threshold (record linkage). An unsupervised head that replaces the learned linear/MLP probe with a single cosine-similarity threshold between paired row embeddings, tuned on the validation split. It is reported as a fourth head in Table 27 and consolidated in Table 13 here. It is the only baseline that specifically exercises geometry rather than supervised readout. Per-dataset results. Tables 11–13 report 5-seed means (± std) for every baseline on every dataset where it applies. Table 11 gives the embedding-free matchers. Table 12 summarizes Random, Dummy, and TF-IDF by task-level means (probe tasks) or by classification/regression/linkage block (row-level tasks). Table 13 gives Cosine-threshold across the 14 row models used in Table 3. Overlap with the main results tables is intentional and serves as a cross-validation check: e.g., Jaccard’s Valentine R@GT matches the Best∗ marker (d) of Table 2, and Cosine-threshold values match the Cos column of Table 27.

27

Table 11: Non-neural matching baselines, per-dataset 5-seed means. Values are deterministic given fixed inputs. Reported ± indicates seed-to-seed variation in the upstream data pipeline. Metrics: col-MAP for Inv.-Index Containment, MAP@10 for Hungarian Set Match, Recall@GT for Valentine matchers. nseeds

Baseline

Dataset

Inv.-Index Cont.

opendata opendata_CAN opendata_USA opendata_UK_SG

0.148 ± 0.000 0.182 ± 0.000 0.138 ± 0.000 0.152 ± 0.000

Value

5 5 5 5

Hungarian Set Match

santos tus tus_hard ugen_v1 ugen_v2

0.975 ± 0.000 1.000 ± 0.000 0.008 ± 0.000 0.647 ± 0.000 0.239 ± 0.000

5 5 5 5 5

Jaccard (Valentine)

valentine

0.473 ± 0.000

5

Distribution (Valentine)

valentine

0.394 ± 0.002

5

Table 12: Random, Dummy, and TF-IDF baselines per probe task (5-seed mean ± std). Values are computed under the paper’s main convention (Tables 2 and 3): avg(MLP, linear) probe where both heads exist, best table-aggregation per task, strict (table-disjoint) splits for the †-marked pairwise tasks (C OL OVERLAP, J OIN C LS, U NION C LS, U NION R EG), T BL R ET via the model_only retrieval pipeline, and binary match-class F1 for record linkage. NA = baseline not applicable to the task per Table 10. Task (metric)

Random

Dummy

ColType (F1 ) ColClust (NMI) ColRel (F1 ) ColOverlap (nRMSE) JoinCls (F1 ) UnionCls (F1 ) UnionReg (nRMSE) TblSubset (F1 ) TblQA (Acc) TblRet (MRR)

0.132 ± 0.003 0.032 ± 0.001 0.015 ± 0.001 1.014 ± 0.003 0.516 ± 0.014 0.500 ± 0.004 1.138 ± 0.024 0.458 ± 0.027 0.204 ± 0.004 0.131 ± 0.120

0.178 ± 0.000 0.813 ± 0.007 NA 0.400 ± 0.001 0.000 ± 0.000 NA 1.000 ± 0.000 NA 0.414 ± 0.000 NA 0.331 ± 0.000 NA 1.001 ± 0.000 NA 0.369 ± 0.000 NA

RowPred (AUROC) RowPred (Macro-F1 ) RowPred Reg. (SGM↓)

0.506 ± 0.001 0.500 ± 0.000 0.348 ± 0.001 0.304 ± 0.000 1.103 ± 0.000 1.004 ± 0.000

NA NA NA

RecLink DM-C (F1 ) RecLink DM-D (F1 ) RecLink WDC (F1 )

0.179 ± 0.007 0.000 ± 0.000 0.223 ± 0.003 0.000 ± 0.000 0.128 ± 0.003 0.000 ± 0.000

NA NA NA

28

NA NA

TF-IDF

NA NA

Table 13: Cosine-threshold baseline on record linkage (binary match-class F1 , 5-seed mean ± std). An unsupervised baseline that thresholds cosine similarity between frozen row embeddings. The same numbers appear in the “Cos” column of Table 27. Model BERT GTE TABBIE TUTA TabICL TabPFN VIME SCARF DAE TabBinning SAINT SubTab TabTransformer TransTab

DM-C

DM-D

WDC

All

0.390 ± 0.000 0.698 ± 0.000 0.309 ± 0.000 0.363 ± 0.000 0.377 ± 0.000 0.260 ± 0.007 0.242 ± 0.000 0.350 ± 0.001 0.265 ± 0.000 0.387 ± 0.002 0.251 ± 0.002 0.275 ± 0.000 0.257 ± 0.000 0.410 ± 0.001

0.315 ± 0.000 0.728 ± 0.000 0.296 ± 0.000 0.397 ± 0.000 0.328 ± 0.000 0.285 ± 0.000 0.295 ± 0.000 0.352 ± 0.002 0.297 ± 0.000 0.421 ± 0.002 0.286 ± 0.008 0.331 ± 0.000 0.301 ± 0.000 0.311 ± 0.000

0.390 ± 0.000 0.511 ± 0.000 0.389 ± 0.000 0.354 ± 0.000 0.341 ± 0.000 0.387 ± 0.000 0.423 ± 0.000 0.357 ± 0.000 0.429 ± 0.000 0.383 ± 0.000 0.429 ± 0.000 0.428 ± 0.000 0.426 ± 0.000 0.567 ± 0.004

0.371 ± 0.000 0.659 ± 0.000 0.326 ± 0.000 0.369 ± 0.000 0.356 ± 0.000 0.298 ± 0.004 0.301 ± 0.000 0.352 ± 0.001 0.314 ± 0.000 0.395 ± 0.002 0.304 ± 0.003 0.327 ± 0.000 0.310 ± 0.000 0.425 ± 0.001

29

H

Metric Definitions and Normalized Rank

Normalized rank (NR). NR denotes the mean normalized rank of a model across the finest evaluation unit u available for a given aggregate: X ranku (m) − 1 1 NR(m) = , |U(m)| Nu − 1 u∈U (m)

where Nu is the number of models with a score on unit u, U(m) is the subset of units on which m is scored, and ties are broken by min. Missing units are excluded from m’s own average and do not penalize other models’ per-unit ranks. Lower is better. The unit u differs by aggregate: CTB ENCH family NRs average ranks over the individual tasks in a family (e.g., Schema NR over ColType, ColClust, ColRel); row-prediction NRs average ranks over individual target columns (77 classification, 46 regression; classification ranks are averaged separately over AUROC and Macro F1 ); and Clean/Robust Linkage NRs average ranks over individual datasets (8 clean DM; 4 dirty DM + 4 WDC). Because dynamic range widens when fewer units are averaged, absolute NR magnitudes should be read within a column and not compared across suites. Task metrics. F1 scores are macro-averaged over classes by default. This covers all multi-class tasks in the benchmark, including column type prediction, row-prediction classification, and the table-pair classification tasks (join classification, union classification, table subset). Record linkage is the sole exception: because it is a binary classification task, we follow the entity-matching convention established by DeepMatcher [53] and WDC Products [63] and report binary F1 on the match (positive) class, equivalent to sklearn.metrics.f1_score(..., average=’binary’, pos_label=1). Group-level linkage scores (DM-C, DM-D, WDC, and the “All (16 pairs)” columns of Tables 26 and√27) are the unweighted mean of per-dataset binary F1 . For regression tasks, we use nRMSE = 1 − R2 , a monotone transform of the coefficient of determination (lower is better; values above 1 correspond to negative R2 ). To aggregate nRMSE across the 46 regression targets, we report the shifted geometric mean SGMε with ε = 0.01: SGMε (x1 , . . . , xK ) =

K Y

(xi + ε)

1/K

− ε,

i=1

which reduces sensitivity to outlier targets while penalizing consistently poor performance. The shift prevents the product from collapsing when any xi is exactly zero. AUROC for row-prediction classification is the area under the ROC curve, computed per target column and then averaged across the 77 classification targets; binary AUROC is used for binary targets and weighted one-vs-rest AUROC is used for multi-class targets, following the standard sklearn.metrics.roc_auc_score convention. MAP denotes mean average precision computed over the full ranked list (not truncated to a fixed K). R@GT (Recall at Ground Truth) follows the Valentine convention [45]: all m × n candidate column pairs are ranked by cosine similarity, the top k pairs are retained with k = |ground truth|, and recall is the fraction of true correspondences among them. R@k (Recall at k) is the fraction of ground-truth targets present in the top-k retrieved items; we report k=100 as the Stage-1 retrieval diagnostic in DLTE. NMI is standard normalized mutual information with arithmetic averaging. MRR is mean reciprocal rank. Acc on Table QA is exact-match denotation accuracy on WikiTableQuestions, following the source convention [60]. UJ-H is defined in Sec. 3.4, and Cell F1 , a complementary DLTE diagnostic, is defined in Appendix N.1.

30

I

Table-Footprint Coverage Across Suites

For each table T , let nrow (T ) and ncol (T ) denote its row and column counts (equivalently, |R(T )| and |C(T )| in the notation of Sec. 3.1), and define its cell footprint as Fcell (T ) = nrow (T ) ncol (T ). A loadable table input is a concrete table object returned by a benchmark evaluation loader for a table-valued role (feature table, left/right entity table, query table, or lake/corpus table), before model-specific serialization or truncation. We count each distinct table once within each role; if the same physical table appears under multiple roles, it is counted once per role. Thus row-prediction datasets contribute one feature table each, record-linkage datasets contribute their left and right entity tables, column- and table-CTBench datasets contribute their table-valued query/lake/corpus inputs as applicable, and TRL-DLTE is summarized by its 47,772 retrieval-lake candidates. Labeled table pairs, their labels, and train/valid/test split indices are metadata rather than additional table inputs: a table referenced by many pairs or splits is still counted once in its role. Figure 5 shows the joint (nrow , ncol ) density per suite, Figure 6 marginalizes to Fcell on a logarithmic axis with 50 common bins, and Table 14 lists per-dataset summary statistics for the 87 dataset-source entries, grouped into seven benchmark categories: Schema, Joinability, Unionability, and Grounding from CTBench, Row Prediction and Record Linkage from RBench, and DLTE on its own. These statistics are descriptive only: benchmark scores are computed per task and are not weighted by table count or cell footprint.

Joint rows-cols footprint of TRL-Bench table inputs (dashed diagonals: constant Fcell = nrow ncol) Table-CTBench (5 datasets; 267,705 table inputs) Fcell {102 106} 100

101 100

% of suite

102

10 1

10 1

10 2

10 2

10 3

Row-Rbench (66 datasets; 82 table inputs)

10 3

104 F {102 106} cell

6 × 100

103

4 × 100

Fcell {102 106}

100

% of suite

3 × 100

102

10 1

2 × 100

101 100

DLTE (1 dataset; 47,772 table inputs)

% of suite

Cols per table ncol(T) (log scale)

100

103 % of suite

Cols per table ncol(T) (log scale)

Column-CTBench (15 datasets; 284,377 table inputs)

104 F {102 106} cell

100

101

102

103

104

105

Rows per table nrow(T) (log scale)

106

107

10 2

100

101

102

103

104

105

Rows per table nrow(T) (log scale)

106

107

Figure 5: Joint rows-columns footprint of TRL-Bench table inputs. Each panel plots a 2D density of (nrow (T ), ncol (T )) over the counted loadable table inputs of one suite, on log–log axes. Bin intensities are normalized within each suite to “% of suite”, so a smaller suite is not visually dominated by a larger one. Gray dashed diagonals mark constant-footprint contours Fcell ∈ {102 , 103 , 104 , 105 , 106 }. The black × in each panel sits at the suite-wise median of nrow and the suite-wise median of ncol (computed independently along each axis). 31

Per-suite cell-footprint distributions of TRL-Bench table inputs Column-CTBench (15 datasets; 284,377 tables)

25000

p50=72

20000

# tables

Table-CTBench (5 datasets; 267,705 tables) p50=40

p95=290,304

15000

20000

10000

10000

5000 0

0

Row-Rbench (66 datasets; 82 tables) p50=95,702

# tables

12.5

DLTE (1 dataset; 47,772 tables)

p95=5,051,753

10.0 7.5 5.0 2.5 0.0 100

p95=28,410

30000

101

102

103

104

105

106

107

Cells per table C = Rows Cols (log scale)

108

109

2500 2000 1500 1000 500 0 100

p50=2,532

101

102

103

104

p95=112,500

105

106

107

Cells per table C = Rows Cols (log scale)

108

109

Figure 6: Per-suite cell-footprint distributions of TRL-Bench table inputs. Each panel histograms the footprint values Fcell (T ) = nrow (T ) ncol (T ) for the counted loadable table inputs in one suite, using 50 common logarithmically spaced bins over 100 –109 cells. The black dashed line marks the median footprint within that suite, and the gray dotted line marks the corresponding 95th percentile. Each panel’s y-axis reports the number of counted table inputs per bin and is scaled independently because the four suites differ by orders of magnitude in the number of counted inputs.

Table 14: Per-dataset cell-footprint statistics across TRL-Bench suites. Each row is one of the 87 dataset-source entries in the benchmark, grouped by benchmark category (Schema, Joinability, Unionability, Grounding, Row Prediction, Record Linkage, DLTE). Cell footprint Fcell (T ) = nrow (T ) ncol (T ), i.e. row count times column count, is computed once per counted loadable table input under its dataset role. Labeled pairs and train/valid/test split indices are not expanded into additional table inputs. Mean, median, and Std. are computed over the counted inputs for that dataset. Std. uses the population convention (ddof=0) when at least two inputs are present. Row Prediction entries contain one counted feature table each, so Mean equals Median. Their Std. cells are shown with a dash because there is no within-dataset dispersion to summarize. Dataset

Category

# table inputs

Mean (cells)

Median (cells)

Std. (cells)

Schema SOTAB WikiCT (rel.) sato

Schema Schema Schema

(col) 72,629 53,567 78,733

1,887 42 29

260 22 10

9,922 105 120

Joinability OpenData (main) OpenData CAN OpenData UK/SG OpenData USA Spider Join Wiki Containment

Joinability Joinability Joinability Joinability Joinability Joinability

(col/tbl) 16,823 4,960 3,090 5,165 15,996 39,084

1,025,435 1,245,433 272,228 1,087,988 104,286 134

258,792 359,628 54,610 263,340 1,000 110

3,585,105 3,858,684 2,082,714 4,209,460 852,833 89

Unionability CKAN Subset ECB Union SANTOS TUS TUS-hard UGEN v1 UGEN v2 Valentine Wiki Union

Unionability Unionability Unionability Unionability Unionability Unionability Unionability Unionability Unionability

(col/tbl) 36,846 4,226 600 1,651 2,769 1,050 1,050 1,098 40,752

33,039 11,457 96,975 41,815 44,109 77 307 315,128 133

6,696 1,134 17,116 37,746 41,850 70 140 210,000 110

87,250 49,893 244,244 30,518 29,579 44 437 248,934 89

(col/tbl)

Grounding

(continued on next page)

32

Dataset

Category

# table inputs

Mean (cells)

Median (cells)

Std. (cells)

NQ-Tables WTQ

Grounding Grounding

169,885 2,108

47 173

18 90

155 292

Row Prediction kc2 nomao kr-vs-kp sick connect-4 MiceProtein Internet-Ads auction-verification student-perf-por wave-energy cps88wages fps-benchmark PhishingWebsites analcatdata-authorship anneal Fiat-500 APSFailure bank-marketing Bank-Churn Bioresponse churn coil2000 credit-g credit-card-default airline-satisfaction Diabetes130US diamonds Fitness-Club GiveMeSomeCredit hazelnut-spread heloc hiva-agnostic houses HR-Analytics in-vehicle-coupon kddcup09-appetency Marketing-Campaign polish-bankruptcy qsar-biodeg SDSS17 seismic-bumps splice students-dropout superconductivity website-phishing wine-quality NATICUSdroid jm1 MIC cylinder-bands

Row Prediction Row Prediction Row Prediction Row Prediction Row Prediction Row Prediction Row Prediction Row Prediction Row Prediction Row Prediction Row Prediction Row Prediction Row Prediction Row Prediction Row Prediction Row Prediction Row Prediction Row Prediction Row Prediction Row Prediction Row Prediction Row Prediction Row Prediction Row Prediction Row Prediction Row Prediction Row Prediction Row Prediction Row Prediction Row Prediction Row Prediction Row Prediction Row Prediction Row Prediction Row Prediction Row Prediction Row Prediction Row Prediction Row Prediction Row Prediction Row Prediction Row Prediction Row Prediction Row Prediction Row Prediction Row Prediction Row Prediction Row Prediction Row Prediction Row Prediction

(row) 1 10,440 1 4,032,405 1 108,664 1 94,300 1 2,769,837 1 81,000 1 5,105,403 1 10,215 1 18,172 1 3,384,000 1 140,775 1 935,712 1 309,540 1 58,029 1 26,042 1 9,228 1 12,692,000 1 497,321 1 90,000 1 6,658,025 1 85,000 1 815,226 1 18,000 1 630,000 1 2,467,720 1 3,075,274 1 431,520 1 7,500 1 1,350,000 1 69,600 1 230,098 1 6,213,520 1 123,840 1 191,580 1 291,732 1 10,400,000 1 51,520 1 366,420 1 41,106 1 780,530 1 33,592 1 188,210 1 150,416 1 1,701,040 1 9,471 1 64,970 1 629,244 1 217,700 1 185,191 1 17,820

10,440 4,032,405 108,664 94,300 2,769,837 81,000 5,105,403 10,215 18,172 3,384,000 140,775 935,712 309,540 58,029 26,042 9,228 12,692,000 497,321 90,000 6,658,025 85,000 815,226 18,000 630,000 2,467,720 3,075,274 431,520 7,500 1,350,000 69,600 230,098 6,213,520 123,840 191,580 291,732 10,400,000 51,520 366,420 41,106 780,530 33,592 188,210 150,416 1,701,040 9,471 64,970 629,244 217,700 185,191 17,820

– – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – – –

Record Linkage DM-abt_buy DM-amazon_google DM-beer DM-dblp_acm DM-dblp_acm_dirty DM-dblp_scholar DM-dblp_scholar_dirty

Record Linkage Record Linkage Record Linkage Record Linkage Record Linkage Record Linkage Record Linkage

(row) 2 2 2 2 2 2 2

3,260 6,884 14,690 9,820 9,820 133,758 133,758

16 2,794 2,690 644 644 123,294 123,294

3,260 6,884 14,690 9,820 9,820 133,758 133,758

(continued on next page)

33

Dataset

Category

DM-fodors_zagats DM-itunes_amazon DM-itunes_amazon_dirty DM-walmart_amazon DM-walmart_amazon_dirty WDC-large WDC-medium WDC-small WDC-xlarge

Record Linkage Record Linkage Record Linkage Record Linkage Record Linkage Record Linkage Record Linkage Record Linkage Record Linkage

DLTE DLTE-Lake

DLTE

# table inputs

Mean (cells)

Median (cells)

Std. (cells)

2 2 2 2 2 2 2 2 2

2,160 251,320 251,320 61,570 61,570 97,136 79,688 56,948 100,380

2,160 251,320 251,320 61,570 61,570 97,136 79,688 56,948 100,380

505 196,064 196,064 48,800 48,800 32 245 3,440 798

(dlte) 47,772

23,967

2,532

63,592

34

J

Family-Level Performance Summary Figure

(a) Column / Table

(b) Row

Schema

Grounding

Classification

Robust Linkage

Join

Union BERT

GTE

TaBERT

TAPAS

TABBIE

TURL

Starmie

TabSFM

Regression

Clean Linkage

TabICL

TransTab

DAE

Figure 7: Granularity-dependent transfer profiles. Radar plots summarize family-level performance for representative models on the two atomic suites. Panel (a) compares column/table encoders across the four TRL-CT BENCH capability families: Schema, Join, Union, and Grounding. Panel (b) compares row encoders across Classification, Regression, Clean Linkage, and Robust Linkage in TRL-R BENCH. BERT, GTE, and TABBIE appear in both panels because they expose both columnand row-level embeddings. The remaining models in each panel are granularity specialists and are evaluated only within their supported suite. Values are rank-normalized within each axis (farther from center is better). No single model dominates all axes, and strengths shift substantially with representation granularity and task family. This figure is a qualitative summary of Tables 2 and 3.

35

K

CTBench Diagnostics and Ablations

K.1

Observational Lexical-Accessibility Proxies for CTBench

To support the Sec. 4.2 reading that generic text encoders remain strong on many column/table tasks through lexical accessibility, we report two observational proxies in Table 15 computed directly from the main CTBench results. The first is the gap between the strongest applicable non-neural baseline and the best neural encoder on the task: small gaps (after direction correction) indicate that surface lexical statistics, such as Jaccard, TF-IDF, value overlap, and Valentine matchers, already recover most of the task signal. The second is the direction-corrected generic-text advantage, the mean score of BERT/GTE minus the mean of tabular specialists (TA BERT/TAPAS/S TARMIE/TURL) on the same task. Generic text encoders lead on 10 of 13 CTBench tasks, with the strongest advantages on tasks whose input is dominated by short natural-language text (table retrieval +0.200, join search +0.116, column type prediction +0.088, column relation prediction +0.072, and column clustering +0.048), consistent with these tasks being accessible from headers and short cell strings. The three exceptions where the four-specialist mean beats the two-generic-text mean are exactly the tasks whose pretraining objective is tightly aligned: S TARMIE’s column-level contrastive objective wins schema matching (−0.067) and union search (−0.010), and TURL’s table-language modeling wins Table QA (−0.016). Sec. 4.2 additionally counts Table Subset as a specialist-won task because the individual best model is TAPAS (0.567 F1 , beating both BERT and GTE); on this mean-vs.-mean proxy Table Subset is borderline (+0.006 to generic text) since the four-specialist mean dilutes a single-specialist win. This is an observational hint, not a causal test. A direct header-masking ablation would provide stronger evidence and is left as future work. Within the current data, however, the ordering aligns with the main-text reading: CTBench task families differ systematically in whether text- or structure-pretrained encoders provide the dominant signal. Table 15: Observational proxies for surface-text signal in TRL-CT BENCH tasks, ordered by descending generic-text advantage. Best baseline is the strongest task-local non-neural baseline from Table 2 (TF-IDF, Jaccard, Hungarian/Valentine, value-overlap, etc.). Best neural is the strongest CTBench encoder on that task. Gap is the metric-direction-corrected difference (best-neural − best-baseline, with positive meaning neural is better). Generic-text advantage is the mean of BERT, GTE minus the mean of tabular specialists TA BERT, TAPAS, S TARMIE, TURL on the same task (direction-corrected). Tasks near the top of the list are those most consistent with the claim that generic text encoders remain competitive on CTBench through surface-text signal. Tasks near the bottom are those where tabular pretraining provides a substantial objective-specific gain.

K.2

Task

Dir. Best baseline Best neural (model)

Gap

Generic-text adv.

Table Retrieval Join Search Col Type Col Rel Col Clust Union Reg. Union Class Col Overlap Join Class Table Subset Union Search Table QA Schema Match

↑ ↑ ↑ ↑ ↑ ↓ ↑ ↓ ↑ ↑ ↑ ↑ ↑

+0.345 +0.314 +0.113 +0.811 +0.116 +0.546 +0.357 +0.228 +0.037 +0.109 +0.088 +0.073 +0.291

+0.200 +0.116 +0.088 +0.072 +0.048 +0.039 +0.034 +0.032 +0.023 +0.006 −0.010 −0.016 −0.067

0.131 0.155 0.813 0.015 0.400 1.138 0.500 1.014 0.516 0.458 0.574 0.204 0.473

0.476 (GTE) 0.469 (GTE) 0.926 (BERT) 0.826 (BERT) 0.516 (BERT) 0.592 (BERT) 0.857 (BERT) 0.786 (BERT) 0.553 (BERT) 0.567 (TAPAS) 0.662 (Starmie) 0.277 (TURL) 0.764 (Starmie)

Probe Head Complexity for Column/Table-Level Tasks

Main column- and table-level results follow the unified supervised-probe protocol of Sec. 3.1 (averaging linear and MLP heads). This appendix reports the linear and MLP components separately to show per-task head sensitivity. Table 16 compares MLP, linear, and dummy probes across five column/table-level tasks. Frozen embeddings carry most of the signal. The ∆L-D column shows that the linear probe already far outperforms the dummy baseline on every task: averages of +0.65 (ColType), +0.72 36

Table 16: Ablation: Probe head complexity for column/table-level tasks. MLP = two-layer MLP head. Linear = logistic regression / ridge. Dummy = majority-class or mean prediction. Best embedding per model, 5-seed average. ∆M-L = MLP − Linear (positive = MLP is better). ∆L-D = Linear − Dummy (capacity of the frozen embedding itself). Col Typec

Col Relc

Join Class.t†

Union Class.t†

F1 ↑

F1 ↑

F1 ↑

F1 ↑

Model

MLP

Lin

Dum

BERT GTE TaBERT TAPAS TAPEX TABBIE TURL TUTA Starmie TabSketchFM

0.928 0.922 0.874 0.869 — 0.881 0.836 — 0.767 0.583

0.924 0.923 0.874 0.867 — 0.904 0.792 — 0.811 0.550

0.178 0.178 0.178 0.178 — 0.178 0.178 — — 0.178

Avg.

0.832

0.830

0.178

Tbl Subsett F1 ↑

∆M-L

∆L-D

MLP

Lin

Dum

∆M-L

∆L-D

MLP

Lin

Dum

∆M-L

∆L-D

MLP

Lin

Dum

∆M-L

∆L-D

MLP

Lin

Dum

∆M-L

∆L-D

0.819 0.797 0.749 0.755 — 0.793 0.740 — 0.701 0.356

0.000 0.000 0.000 0.000 — 0.000 0.000 — — 0.000

+0.01

+0.82

+0.06

+0.11

+0.12

+0.37

+0.27

+0.03

+0.02

+0.12

+0.19

+0.41

+0.26

+0.07

+0.03

+0.08

+0.19

+0.43

+0.32

+0.03

−0.02

+0.79

−0.04

+0.15

+0.20

+0.40

+0.30

+0.03

+0.04

+0.74

−0.04

+0.14

+0.26

+0.36

+0.26

+0.01

—

−0.01

+0.06

+0.19

+0.38

+0.05

+0.05

−0.01

— —

+0.00

—

+0.19

—

+0.03

+0.36

−0.07

+0.14

+0.10

+0.36

0.369 0.369 0.369 0.369 0.369 0.369 0.369 0.369 — 0.369

+0.06

−0.06

0.424 0.425 0.403 0.439 0.396 0.396 0.379 0.421 0.407 0.443

+0.24

—

0.665 0.662 0.677 0.695 0.718 0.697 0.635 0.472 0.672 0.663

+0.24

+0.41

+0.76

0.331 0.331 0.331 0.331 0.331 0.331 0.331 0.331 — 0.331

+0.42

+0.75

0.752 0.736 0.702 0.741 0.757 0.736 0.686 0.713 0.757 0.689

+0.21

—

0.962 0.950 0.818 0.931 0.951 0.931 0.942 0.906 0.949 0.785

+0.21

+0.14

+0.03

0.414 0.414 0.414 0.414 0.414 0.414 0.414 0.414 — 0.414

−0.03

+0.02

0.578 0.551 0.527 0.532 0.497 0.560 0.554 0.470 0.509 0.550

+0.16

+0.80

0.529 0.519 0.468 0.555 0.530 0.523 0.510 0.465 0.512 0.482

−0.05

+0.03

0.714

0.000

+0.02

+0.72

0.509

0.533

0.414

−0.02

+0.12

0.913

0.727

0.331

+0.19

+0.39

0.656

0.413

0.369

+0.00

+0.75

−0.00

+0.74

+0.00

+0.70

+0.00

+0.69

—

—

−0.02

+0.73

+0.04

+0.61

— −0.04

— —

+0.03

+0.37

0.834 0.826 0.771 0.782 — 0.777 0.777 — 0.695 0.390

+0.00

+0.65

0.731

+0.26

—

+0.22

+0.07

+0.24

+0.05

(ColRel), +0.12 (JoinCls), +0.39 (UnionCls), and +0.05 (TblSubset). This confirms that the frozen representations encode task-relevant structure without any task-specific training. MLP adds little over linear on column tasks. For ColType and ColRel, the average ∆M-L is +0.00 and +0.02 respectively, indicating that a linear probe is sufficient. For UnionCls and TblSubset, the MLP gains are more consistent (+0.19 and +0.24 on average), suggesting that nonlinear separation helps when table-pair geometry is more complex. JoinCls favors linear probes. The average ∆M-L for JoinCls is −0.02, and several models show negative gaps (e.g., BERT −0.05, TaBERT −0.06). This indicates that the MLP head overfits on the relatively small join classification training sets, and a linear probe is the more reliable choice for this task. K.3

Aggregation Ablation for Table-Level Embeddings

For models that expose multiple candidate table embeddings, we compare three aggregation strategies: CLS (the [CLS] token of the linearized table), COL - MEAN (the mean of per-column embeddings), and TOK - MEAN (the mean of all non-padding token hidden states). Tables 17 and 18 report the full MLP- and linear-probe breakdowns, respectively. Task-level averages. With an MLP probe, TOK - MEAN gives the best average on UnionCls, UnionReg (lower is better), and TblRet, while COL - MEAN is narrowly best on JoinCls and TblSubset. With a linear probe, COL - MEAN is best on JoinCls and UnionReg, while TOK - MEAN is best on UnionCls and TblSubset. However, several gaps are tiny (e.g., 0.494/0.500/0.497 on MLP JoinCls for CLS/ COL - MEAN/ TOK - MEAN and 0.724/0.725/0.734 on linear UnionCls), so these averages should be read as rough tendencies rather than definitive rankings. Per-model variation. The per-model tables show some encoder-specific preferences: TABBIE often peaks with CLS under MLP, whereas TAPAS and several text-pretrained models more often favor COL - MEAN or TOK - MEAN. Nevertheless, the model-dependent variation is modest relative to the task-dependent variation, so we do not treat aggregation choice as a major axis of analysis. The main comparison in Table 2 reports each encoder’s strongest supported aggregation averaged over MLP and linear probes. K.4

Join Search: Direct Cosine vs. Learned Projection

Interpretation. Direct cosine is an informative no-training control, but in our frozen-transfer reuse of join search the relation of interest is directional containment rather than pure semantic similarity. The learned projection is therefore the canonical main-table setting: it remains a minimal probe over frozen embeddings while matching the transfer objective more closely. K.5

Union Search: TUS vs. TUS-hard

Table 20 compares model performance on TUS (original) and TUS-hard (low-overlap variant). TUShard filters out positive pairs whose directed column containment is at least 0.70, removing the 37

Table 17: Effect of table-level embedding aggregation (MLP probe). For each model we evaluate every supported aggregation strategy: CLS ([CLS] token from the linearized table), COL - MEAN (mean of per-column embeddings), and TOK - MEAN√(mean of all non-padding token hidden states). Metrics match Table 2: F1 = macro F1 , nRMSE = 1 − R2 . Table 2 reports avg(MLP, linear) for the best supported aggregation. Tables 17–18 break down the per-probe results. Values are mean ± std over 5 random seeds. Bold / underlined values highlight best/second-best aggregation for each model and metric (shown only when ≥2 variants exist). † Table-disjoint split. Dashes indicate the model does not produce that embedding variant. Family

Model

BERT

CLS COL - MEAN TOK - MEAN

GTE

CLS COL - MEAN TOK - MEAN

TaBERT

COL - MEAN

TAPAS

CLS COL - MEAN TOK - MEAN

TAPEX

CLS TOK - MEAN

TABBIE

CLS COL - MEAN

Generic Text

Table-Text

Table-Struct.

Col.-Centric

Agg.

JoinCls†

UnionCls†

UnionReg†

TblSubset

TblRet

F1 ↑

F1 ↑

nRMSE ↓

F1 ↑

MRR ↑

0.503±0.033 0.493±0.040 0.529±0.019 0.491±0.029 0.519±0.037 0.453±0.035

0.956±0.003 0.961±0.002 0.962±0.004 0.946±0.005 0.944±0.002 0.950±0.004

0.509±0.009 0.454±0.009 0.449±0.018 0.640±0.011 0.475±0.019 0.578±0.018

0.615±0.010 0.665±0.009 0.653±0.008 0.567±0.014 0.662±0.004 0.543±0.044

0.321±0.013 0.357±0.009 0.367±0.008 0.476±0.003 0.450±0.013 0.473±0.008

0.468±0.068 0.460±0.035 0.555±0.022 0.503±0.031 0.530±0.046 0.526±0.043

0.818±0.007 0.856±0.006 0.929±0.007 0.931±0.006 0.942±0.002 0.951±0.004

0.536±0.017 0.576±0.018 0.494±0.008 0.488±0.010 0.468±0.012 0.471±0.010

0.677±0.007 0.689±0.007 0.695±0.009 0.693±0.011 0.718±0.009 0.704±0.007

0.372±0.013 0.265±0.009 0.285±0.013 0.295±0.006 — —

0.523±0.043 0.474±0.034 0.510±0.028 0.465±0.019

0.931±0.003 0.919±0.002 0.942±0.002 0.906±0.006

0.542±0.002 0.665±0.011 0.499±0.018 0.511±0.013

0.697±0.007 0.681±0.009 0.635±0.007 0.472±0.016

0.170±0.004 0.102±0.002 0.199±0.010 0.260±0.013

TURL TUTA

COL - MEAN

Starmie

COL - MEAN

TabSketchFM

CLS COL - MEAN TOK - MEAN

0.512±0.017 0.482±0.032 0.471±0.026 0.476±0.021

0.949±0.004 0.779±0.004 0.785±0.002 0.783±0.004

0.536±0.006 0.549±0.024 0.512±0.021 0.513±0.013

0.672±0.010 0.659±0.015 0.647±0.004 0.663±0.009

0.018±0.002 0.218±0.011 0.197±0.014 0.193±0.035

CLS COL - MEAN TOK - MEAN

0.494 0.500 0.497

0.902 0.906 0.915

0.542 0.521 0.500

0.631 0.667 0.651

0.285 0.248 0.332

Avg. across models

CLS

highest-overlap 36% of positives so that our frozen-transfer evaluation can better separate lexical overlap from broader union signal. Interpretation. On original TUS, the value-overlap baseline achieves MAP 1.000 and BERT, GTE, and TURL all score above 0.95. On TUS-hard, the baseline drops to 0.008 and the ranking shifts substantially (Spearman ρ = −0.67). The models most robust under this low-overlap variant are S TARMIE and TA BERT, whose pretraining objectives emphasize cross-table structure. We therefore report TUS-hard alongside TUS in TRL-B ENCH not as a replacement for the original resource, but as a complementary variant for the frozen-transfer setting, where it is useful to distinguish overlap-driven retrieval from transfer that remains helpful when overlap is limited. K.6

Table Retrieval: Model-Only vs. Hybrid Mode

Table 21 compares model-only and hybrid retrieval modes. In model-only mode, the projection head operates solely on the model’s own table embedding. In hybrid mode, the model’s table embedding is concatenated with the query encoder’s (MPNet [67] or sentence-T5 [56]) table embedding before projection, bridging the modality gap between the model’s table space and the query space. Interpretation. Hybrid mode uniformly improves MRR for every model, with gains ranging from +0.057 (GTE) to +0.509 (S TARMIE). However, all hybrid MRRs converge to a narrow band of 0.509–0.555, regardless of the model’s own retrieval quality (0.018–0.476 in model-only mode). This indicates that the added query-encoder table embedding contributes most of the shared signal in the augmented setting. For TRL-B ENCH’s main comparison we therefore use model-only mode, because it isolates the transfer quality of the model’s own table representation rather than performance in a pipeline with added query-side table evidence.

38

Table 18: Effect of table-level embedding aggregation (LINEAR probe). For each model we evaluate every supported aggregation strategy: CLS ([CLS] token from the linearized table), COL - MEAN (mean of per-column embeddings), and TOK - MEAN √ (mean of all non-padding token hidden states). Metrics match Table 2: F1 = macro F1 , nRMSE = 1 − R2 . Table 2 reports the strongest supported aggregation in the corresponding main comparison. Values are mean ± std over 4–5 random seeds. Bold / underlined values highlight best/second-best aggregation for each model and metric (shown only when ≥2 variants exist). † Table-disjoint split. Dashes indicate the model does not produce that embedding variant. Family

Model

BERT

CLS COL - MEAN TOK - MEAN

GTE

CLS COL - MEAN TOK - MEAN

TaBERT

COL - MEAN

TAPAS

CLS COL - MEAN TOK - MEAN

TAPEX

CLS TOK - MEAN

TABBIE

CLS COL - MEAN

Generic Text

Table-Text

Table-Struct.

Col.-Centric

Agg.

JoinCls†

UnionCls†

UnionReg†

TblSubset

F1 ↑

F1 ↑

nRMSE ↓

F1 ↑

0.526±0.000 0.500±0.000 0.578±0.000 0.503±0.000 0.551±0.000 0.478±0.000

0.745±0.000 0.745±0.000 0.752±0.000 0.726±0.000 0.735±0.000 0.736±0.000

0.770±0.000 0.736±0.000 0.735±0.000 0.830±0.000 0.726±0.000 0.754±0.000

0.436±0.000 0.424±0.000 0.408±0.000 0.421±0.000 0.425±0.000 0.416±0.000

0.527±0.000 0.531±0.000 0.532±0.000 0.516±0.000 0.497±0.000 0.550±0.000

0.702±0.000 0.707±0.000 0.746±0.000 0.741±0.000 0.747±0.000 0.757±0.000

0.694±0.000 0.760±0.000 0.731±0.000 0.726±0.000 0.750±0.000 0.772±0.000

0.403±0.001 0.432±0.000 0.439±0.001 0.436±0.001 0.396±0.000 0.411±0.000

0.560±0.000 0.581±0.000 0.554±0.000 0.470±0.000

0.736±0.000 0.737±0.000 0.686±0.000 0.713±0.000

0.784±0.000 0.757±0.000 0.814±0.000 0.792±0.000

0.396±0.000 0.387±0.000 0.379±0.000 0.421±0.001

TURL TUTA

COL - MEAN

Starmie

COL - MEAN

TabSketchFM

CLS COL - MEAN TOK - MEAN

0.509±0.031 0.550±0.007 0.513±0.013 0.533±0.015

0.757±0.000 0.695±0.000 0.689±0.000 0.684±0.000

0.789±0.000 0.864±0.001 0.825±0.001 0.830±0.001

0.407±0.000 0.432±0.002 0.445±0.003 0.443±0.001

CLS COL - MEAN TOK - MEAN

0.520 0.533 0.531

0.724 0.725 0.734

0.793 0.759 0.763

0.419 0.414 0.423

Avg. across models

CLS

Table 19: Join search MAP: direct cosine similarity vs. learned linear projection, averaged across 5 embedding rounds. The learned projection trains a shared Linear(d, d) head with multi-positive InfoNCE loss on a fixed 20%/80% query-role-disjoint split and evaluates on the held-out 80%. Results are column-level MAP, macro-averaged over queries. Bold orange / Underlined blue / Light purple highlights indicate best/second-best/third-best per column (non-baseline). All

CAN

USA

UK+SG

Family

Model

Cos.

Proj.

Cos.

Proj.

Cos.

Proj.

Cos.

Proj.

Generic Text

BERT GTE

0.387 0.411

0.431 0.461

0.300 0.331

0.328 0.370

0.447 0.460

0.508 0.535

0.431 0.478

0.470 0.508

Tabular-Pretrained TaBERT Table-Text TAPAS

0.326 0.246

0.458 0.332

0.295 0.230

0.346 0.295

0.377 0.285

0.525 0.383

0.262 0.208

0.296 0.272

Table-Struct.

TURL TABBIE

0.271 0.189

0.292 0.218

0.248 0.174

0.267 0.176

0.299 0.207

0.341 0.231

0.269 0.206

0.295 0.206

Col.-Centric

Starmie TabSketchFM

0.248 0.231

0.298 0.277

0.230 0.195

0.295 0.227

0.290 0.276

0.379 0.339

0.206 0.188

0.293 0.218

39

Table 20: Union search MAP on TUS vs. TUS-hard (5-seed average). TUS-hard filters out positive pairs with high directed containment (≥ 0.70), creating a low-overlap variant that better separates lexical overlap from broader union signal in our frozen-transfer setting. Ranks are among neural models only. Bold orange / Underlined blue / Light purple highlights indicate best/secondbest/third-best on TUS-hard. Spearman ρ = −0.67 between the two rankings. Type

Model

MAP TUS ↑

Rank TUS

MAP TUS-hard ↑

Rank Hard

Drop (%)

Baseline

Random Val. Ovlp

0.209 1.000

— —

0.079 0.008

— —

62.3 99.2

BERT GTE TaBERT TAPAS TABBIE TURL Starmie TabSketchFM

0.959 0.954 0.926 0.891 0.700 0.953 0.844 0.941

1 2 5 6 8 3 7 4

0.307 0.293 0.436 0.376 0.317 0.304 0.523 0.376

6 8 2 3 5 7 1 4

68.0 69.2 52.9 57.8 54.8 68.1 38.1 60.1

Generic Text Table-Text Table-Struct. Col.-Centric

Table 21: Table retrieval MRR: model-only vs. hybrid mode. In model-only mode, the projection head operates on the model’s own table embedding. In hybrid mode, the model’s table embedding is concatenated with the query encoder’s (mpnet or sentence-t5) table embedding before projection. For each model, we report the best aggregation × sentence-encoder combination. Values are mean ± std over 5 seeds. Family Generic Text Table-Text

Table-Struct. Col.-Centric

Model

Model-Only MRR ↑

Hybrid MRR ↑

∆

BERT GTE TaBERT TAPAS TAPEX TABBIE TURL TUTA Starmie TabSketchFM

0.367±0.008 0.476±0.003 0.372±0.013 0.295±0.006 0.332±0.005 0.170±0.004 0.199±0.010 0.260±0.013 0.018±0.002 0.218±0.011

0.553±0.007 0.533±0.008 0.555±0.006 0.526±0.008 0.536±0.028 0.516±0.007 0.521±0.031 0.509±0.012 0.527±0.012 0.522±0.008

+0.186 +0.057 +0.183 +0.231 +0.204 +0.347 +0.322 +0.249 +0.509 +0.304

40

K.7

Query Encoder Sensitivity

Table 22: Ablation: Query encoder for grounding tasks. Table retrieval uses MRR (↑, model_only). Semantic parsing uses accuracy (↑). Each table model is paired with two query encoders. ∆ = MPNet − ST5. 5-seed average. Table retrieval uses best embedding per model.

Model BERT GTE TaBERT TAPAS TAPEX TABBIE TURL TUTA Starmie

Tbl Ret.t

Tbl QAc

MRR↑

Acc↑

ST5

∆

0.368

0.352

+0.02

± 0.007

± 0.007

MPNet

0.478

0.440

± 0.004

± 0.005

0.372

0.324

± 0.013

± 0.007

0.296

0.276

± 0.006

± 0.007

0.376

0.341

± 0.097

± 0.073

0.170

0.145

± 0.004

± 0.002

+0.05 +0.02 +0.03 +0.02

0.271 −0.03 ± 0.004

0.233

0.256 −0.02

± 0.004

± 0.005

0.252

0.281 −0.03

± 0.007

± 0.005

0.240

0.269 −0.03

± 0.005

± 0.008

—

—

0.261

0.290 −0.03 ± 0.008

0.199 −0.00 0.274 0.235 ± 0.013

0.018

0.012

± 0.002

± 0.002

0.191

± 0.013

± 0.006

0.276

0.252

+0.03 +0.01 +0.03 +0.02

—

± 0.004

± 0.010

0.260

∆

0.238

0.197 ± 0.013

ST5

± 0.005

± 0.014

TabSketchFM 0.221 Avg.

+0.04

MPNet

0.281 −0.01

± 0.006

± 0.004

—

—

—

0.248

0.283 −0.03

± 0.008

± 0.008

0.216

0.253 −0.04

± 0.006

± 0.006

0.245

0.273 −0.03

Table 22 compares MPNet [67] and sentence-T5 [56] (ST5) as query encoders across table retrieval (MRR) and table QA (accuracy). MPNet consistently outperforms ST5 on table retrieval for almost every model (average ∆=+0.02), with the largest gain for TaBERT (+0.05). The pattern reverses for table QA: ST5 is better for all models with available results (average ∆=−0.03), suggesting ST5’s longer context pretraining better supports semantic parsing. In both tasks the absolute differences are small (≤0.05), indicating that query encoder choice has limited sensitivity on these tasks. MPNet is used as the default query encoder in the main evaluation. K.8

Pair-Level Random vs. Table-Disjoint Split Ablation

Four table-pair tasks (join classification, column overlap, union classification, and union regression) support both the original pair-level random splits and the table-disjoint splits used in our frozentransfer evaluation, where training and test tables are separated so the task measures cross-table generalization. Table 23 compares the two protocols. Interpretation. Under the table-disjoint protocol, performance is uniformly lower. The effect is largest on join classification, where the mean per-model F1 drop is 0.212 (aggregate means: 0.736 → 0.523), and union regression shows the next-largest change (average nRMSE increases by 0.110). This is consistent with our transfer-oriented setting being harder: models must generalize to unseen tables rather than to new pairs drawn from already observed tables. Relative rankings are largely preserved, suggesting that table-disjoint evaluation mainly changes the difficulty level and the degree of cross-table separation required by our protocol. We therefore use table-disjoint splits as the default when repurposing these tasks for frozen cross-table transfer.

41

Table 23: Pair-level random vs. table-disjoint split comparison on the four tasks that support both protocols. Each cell reports the 5-seed average (avg. of MLP and linear probes, best aggregation for table-level tasks). ∆ = table-disjoint − pair-random. Negative ∆ for F1 and positive ∆ for nRMSE both indicate that table-disjoint evaluation is harder. Values are mean ± std over 5 seeds. Dashes indicate the model does not produce that embedding variant. Avg. is computed over all models with available data per task.

Type

Model

Baseline

Random

P-R

JoinCls

ColOverlap

UnionCls

F1 ↑

nRMSE↓

F1 ↑

T-D

∆

P-R

T-D

∆

P-R

T-D

UnionReg nRMSE↓

∆

P-R

T-D

∆

0.704 0.516 −0.189 0.972 1.012 +0.040 0.666 0.500 −0.166 0.868 1.138 +0.270

BERT

0.768 0.553 −0.215 0.758 0.786 +0.028 0.877 0.857 −0.020 0.485 0.592 +0.107

Generic Text GTE

0.760 0.535 −0.225 0.775 0.817 +0.042 0.868 0.843 −0.025 0.504 0.600 +0.096

± 0.013 ± 0.027

TaBERT

± 0.001

± 0.002 ± 0.002

± 0.004 ± 0.005

± 0.009

± 0.034

± 0.000

± 0.002

± 0.001

± 0.004

± 0.003

± 0.009 ± 0.009

TAPEX

0.784 0.538 −0.246

± 0.010

± 0.011

± 0.000

± 0.002

—

—

± 0.021

± 0.001

± 0.004

± 0.002

± 0.005

— 0.877 0.854 −0.023 0.494 0.609 +0.115 ± 0.000

± 0.002

± 0.002

± 0.006

TABBIE

0.763 0.542 −0.221 0.832 0.862 +0.030 0.861 0.833 −0.028 0.540 0.663 +0.123

TURL

0.626 0.532 −0.094 0.778 0.809 +0.031 0.842 0.814 −0.028 0.549 0.657 +0.108

TUTA

0.692 0.468 −0.224

± 0.014 ± 0.028

Starmie

± 0.022 ± 0.014

± 0.001

± 0.002

± 0.001

± 0.001

—

—

± 0.010

± 0.001 ± 0.001

± 0.002 ± 0.001

± 0.003 ± 0.003

± 0.001 ± 0.009

— 0.850 0.810 −0.041 0.531 0.652 +0.121 ± 0.001

± 0.003

± 0.005

± 0.007

0.715 0.510 −0.205 0.808 0.847 +0.039 0.876 0.853 −0.023 0.560 0.662 +0.102 ± 0.005

± 0.019

± 0.000

± 0.001

± 0.000

± 0.002

± 0.003

± 0.003

TabSketchFM 0.750 0.516 −0.234 0.887 0.946 +0.059 0.794 0.737 −0.057 0.545 0.668 +0.123 ± 0.015

Avg.

± 0.002

± 0.001

0.740 0.544 −0.196 0.782 0.823 +0.041 0.867 0.837 −0.030 0.502 0.607 +0.105

± 0.009

Col.-Centric

± 0.000

± 0.001

TAPAS

± 0.010

Table-Struct.

± 0.019

± 0.001

0.759 0.498 −0.261 0.812 0.855 +0.044 0.819 0.760 −0.059 0.520 0.615 +0.095 ± 0.021

Table-Text

± 0.010

± 0.015

± 0.001

± 0.001

± 0.001

± 0.002

± 0.002

± 0.010

0.736 0.523 −0.212 0.804 0.843 +0.039 0.853 0.820 −0.033 0.523 0.633 +0.110

42

Table 24: Row-prediction probe-head sweep. Here d counts linear layers. The default one-hiddenlayer MLP (h=256, d=2) is near-optimal for classification. Deeper/wider heads do not provide a consistent regression benefit. Head config

Avg Macro-F1

Avg SGM↓

0.6173 0.6173 0.6144 0.6064 0.5983

0.757 0.750 0.772 0.755 0.750

h=256, d=2 (default) h=128, d=3 h=256, d=3 h=512, d=4 h=512, d=5

L

RBench Diagnostics and Ablations

L.1

Row-Prediction Probe Diagnostics

Representative linear-vs.-MLP cases. For classification, several models are already strongest with a linear probe: the text-transfer encoders BERT (0.6322 MLP → 0.6360 linear) and GTE (0.6004 → 0.6200), as well as the target-table contrastive learner T RANS TAB (0.6016 → 0.6151). By contrast, the feature-corruption target-table SSL models benefit primarily on regression, e.g., DAE improves from 0.5984 to 0.5357 SGM, SCARF from 0.5917 to 0.5462, and TAB B INNING from 0.5929 to 0.5448 when moving from linear to MLP. Dimensionality check. Dimensionality alone does not explain the row ranking. Even after upgrading SSL row encoders to 768 dimensions, TAB ICL (512-d) remains strongest on both classification and regression in this dim-controlled comparison (0.6744 Macro-F1 and 0.4873 SGM, computed on the dim-ablation subset; the main-table values are 0.671 and 0.505 under the standard probe protocol). Embedding dimensions range from 192 (TAB PFN) to 768 (generic text and table-aware models), with most SSL row models at 512. Because the MLP probe uses a fixed hidden size of 256, the first-layer parameter count scales linearly with input dimension (e.g., 768 × 256 vs. 192 × 256). The linear-probe comparison above serves as a dimension-proportional control: linear probes have exactly d × C parameters (where d is the embedding dimension and C the number of classes), so they do not introduce a fixed-width bottleneck. The main row ranking is consistent across linear and MLP probes, indicating that the ranking reflects embedding quality rather than probe-capacity artifacts. For pairwise tasks (record linkage), concatenation doubles the input dimension, producing 1536-d inputs for 768-d models and 1024-d for 512-d models with the same hidden-size-256 MLP. The same linear-probe consistency check applies. Per-target comparison. Figure 8 plots per-target scores for TAB ICL against the strongest comparator on each regime: BERT for classification (AUROC) and DAE for regression (nRMSE). TAB ICL wins on 57/77 classification targets and 38/46 regression targets, confirming that its advantage is broad rather than driven by a few outlier tasks. Adaptation regime explains a substantial fraction of row-level variance. Table 25 quantifies the Sec. 4.3 claim that training regime (externally pretrained transfer vs. target-table self-supervision vs. prior-based meta-pretraining) materially shapes row-level rankings, using the per-sub-task normalizedrank aggregates from Table 3. The decomposition uses the classical 1-way ANOVA sums-of-squares with regime identity as the factor, so between-regime variance and within-regime variance sum to the total sub-task variance (an η 2 -style fraction). The regime effect is largest on the two linkage sub-tasks: between-regime variance accounts for 64% of cross-model NR variance on clean linkage and 48% on robust linkage. The Kruskal–Wallis test rejects the null of identical regime distributions at α = 0.05 on clean linkage (p = 0.014) and is borderline on robust linkage (p = 0.064). Regression shows a smaller regime effect (41% between/total) with a marginally significant Kruskal–Wallis statistic (p = 0.090), while classification has the weakest effect (17% between/total, p = 0.373), consistent with the observation that frozen text encoders and target-table learners both reach competitive classification AUROC for different reasons. The direction of the regime effect also depends on the sub-task: Transfer and Prior-based encoders dominate linkage, whereas Target-Table learners are more competitive on prediction regression. This supports keeping the three regimes separately rather than flattening them into one leaderboard. (We note that the Prior-based regime contains only 43

(a) Classification 1.0

TabICL wins 57/77 targets

1.50 1.25

TabICL nRMSE

TabICL AUROC

0.9

(b) Regression

1.75

0.8 0.7 0.6

1.00 0.75 0.50 0.25

0.5

TabICL wins 38/46 targets

0.00 0.5

0.6

0.7

0.8

BERT AUROC

0.9

1.0

0.0

0.5

1.0

1.5

DAE nRMSE

Figure 8: Per-target pairwise comparison of TAB ICL against the strongest comparator. (a) Classification AUROC: TAB ICL vs. BERT across 77 targets. (b) Regression nRMSE: TAB ICL vs. DAE across 46 targets. Points above the diagonal in (a) and below it in (b) indicate TAB ICL wins. Each color represents a different source dataset. two models, so the Kruskal–Wallis χ2 approximation is borderline at the α = 0.05 level for robust linkage. The between/total η 2 summary does not share this small-sample limitation.) Table 25: Regime-wise normalized-rank summary for TRL-RB ENCH. For each of the four sub-tasks we report mean ± std normalized rank within each adaptation regime (lower is better), together with a Kruskal–Wallis H-test for the null that all regimes have equal NR distributions and the fraction of total sub-task variance explained by between-regime variance. Values are computed from the per-model NR aggregates in Table 3. The “between / total” column quantifies how much of the cross-model variation is captured by regime identity alone. Sub-task

Transfer

Target-Table

Prior-Based

Kruskal–Wallis H (p)

Between / Total

Classification Regression Clean Linkage Robust Linkage

0.504±0.073 0.637±0.055 0.188±0.060 0.192±0.130

0.481±0.134 0.473±0.129 0.654±0.193 0.647±0.249

0.328±0.164 0.319±0.180 0.509±0.086 0.528±0.135

2.0 (0.373) 4.8 (0.090) 8.6 (0.014) 5.5 (0.064)

0.17 0.41 0.64 0.48

44

L.2

Embedding Dimension for Record Linkage

Table 26 reports binary F1 (match class) for eight target-table learners across five embedding dimensions (64, 128, 256, 512, 768) on the record linkage task. Performance increases monotonically with dimension for most models: the average All-pairs F1 rises from 0.058 at d=64 to 0.139 at d=768. The WDC group is consistently the hardest across all dimensions, reflecting the greater heterogeneity of product-matching pairs. TransTab improves substantially with larger embeddings (0.139 → 0.254 overall), while SubTab remains near zero regardless of dimension, suggesting its representations lack pairwise match signal at any scale. These results justify using the native embedding size (512-d for target-table SSL models) as the default throughout the main evaluation. Table 26: Ablation: Embedding dimension for record linkage. Binary F1 (match class, see Appendix H), 5-seed average, linear probe head. Results are broken down by dataset group: DM-C = 8 clean DeepMatcher pairs, DM-D = 4 dirty DeepMatcher pairs, WDC = 4 WDC-Products pairs, All = unweighted mean over all 16 pairs. Bold = best dimension per model, underline = second best. DM-C F1 ↑

DM-D F1 ↑

WDC F1 ↑

All (16 pairs) F1 ↑

Model

64

128

256

512

768

64

128

256

512

768

64

128

256

512

768

64

128

256

512

768

VIME

0.095

0.101

0.112

0.150

0.191

0.043

0.110

0.130

0.162

0.225

0.007

0.009

0.018

0.047

0.075

0.060

0.081

0.093

0.127

0.171

± 0.001

± 0.026

± 0.000

± 0.002

± 0.000

± 0.000

± 0.008

± 0.000

± 0.001

± 0.000

± 0.000

± 0.001

± 0.000

± 0.002

± 0.000

± 0.000

± 0.011

± 0.000

± 0.002

± 0.000

SCARF

0.097

0.150

0.158

0.159

0.175

0.041

0.073

0.130

0.120

0.143

0.009

0.008

0.021

0.040

0.056

0.061

0.095

0.117

0.120

0.138

± 0.001

± 0.006

± 0.000

± 0.006

± 0.000

± 0.000

± 0.000

± 0.000

± 0.010

± 0.000

± 0.000

± 0.000

± 0.000

± 0.001

± 0.001

± 0.000

± 0.003

± 0.000

± 0.000

± 0.000

DAE

0.078

0.126

0.119

0.160

0.183

0.072

0.092

0.127

0.157

0.200

0.007

0.008

0.019

0.049

0.075

0.059

0.088

0.096

0.131

0.160

± 0.000

± 0.018

± 0.000

± 0.002

± 0.000

± 0.000

± 0.009

± 0.000

± 0.004

± 0.000

± 0.000

± 0.000

± 0.000

± 0.001

± 0.000

± 0.000

± 0.011

± 0.000

± 0.000

± 0.000

0.087

0.124

0.117

0.161

0.149

0.099

0.087

0.125

0.152

0.161

0.008

0.012

0.025

0.046

0.064

0.070

0.087

0.096

0.130

0.131

± 0.000

± 0.000

± 0.000

± 0.001

± 0.000

± 0.000

± 0.001

± 0.000

± 0.010

± 0.000

± 0.000

± 0.000

± 0.000

± 0.000

± 0.000

± 0.000

± 0.000

± 0.000

± 0.003

± 0.000

0.058

0.078

0.144

0.187

0.204

0.058

0.061

0.093

0.194

0.168

0.000

0.000

0.011

0.034

0.064

0.044

0.054

0.098

0.150

0.160

± 0.000

± 0.000

± 0.000

± 0.003

± 0.000

± 0.000

± 0.000

± 0.000

± 0.013

± 0.000

± 0.000

± 0.000

± 0.000

± 0.003

± 0.000

± 0.000

± 0.000

± 0.000

± 0.001

± 0.000

0.000

0.033

0.014

0.035

0.043

0.000

0.010

0.017

0.017

0.043

0.000

0.000

0.000

0.000

0.001

0.000

0.019

0.011

0.022

0.032

± 0.000

± 0.004

± 0.000

± 0.003

± 0.000

± 0.000

± 0.022

± 0.000

± 0.000

± 0.000

± 0.000

± 0.000

± 0.000

± 0.000

± 0.000

± 0.000

± 0.003

± 0.000

± 0.001

± 0.000

0.064

0.075

0.071

0.103

0.100

0.000

0.010

0.042

0.062

0.056

0.002

0.002

0.008

0.000

0.010

0.032

0.041

0.048

0.067

0.067

± 0.000

± 0.000

± 0.000

± 0.012

± 0.000

± 0.000

± 0.000

± 0.000

± 0.006

± 0.000

± 0.000

± 0.001

± 0.000

± 0.000

± 0.000

± 0.000

± 0.000

± 0.000

± 0.004

± 0.000

0.179

0.205

0.271

0.266

0.286

0.198

0.239

0.310

0.393

0.369

0.000

0.000

0.016

0.052

0.075

0.139

0.162

0.217

0.245

0.254

± 0.003

± 0.011

± 0.001

± 0.002

± 0.013

± 0.003

± 0.003

± 0.008

± 0.007

± 0.001

± 0.000

± 0.000

± 0.001

± 0.002

± 0.001

± 0.002

± 0.006

± 0.002

± 0.003

± 0.006

0.082

0.112

0.126

0.153

0.166

0.064

0.085

0.122

0.157

0.171

0.004

0.005

0.015

0.034

0.052

0.058

0.078

0.097

0.124

0.139

TabBinning SAINT SubTab TabTransf. TransTab Avg.

45

L.3

Probe Head for Record Linkage

Table 27: Ablation: Probe head for record linkage. Four evaluation protocols on frozen row embeddings (base 768-dim): Cosine = cosine-similarity thresholding (unsupervised). Linear = logistic regression probe. MLP = one-hidden-layer MLP probe (hidden size 256). Dummy = majorityclass baseline. Binary F1 (match class, see Appendix H), 5-seed average, grouped by dataset family. ∆L-C = Linear − Cosine (gain from supervised probing). DM-C (8 pairs) F1 ↑

DM-D (4 pairs) F1 ↑

WDC (4 pairs) F1 ↑

Model

Cos

Lin

MLP

Dum

∆L-C

Cos

Lin

MLP

Dum

BERT GTE TUTA TABBIE TabICL TabPFN

0.390 0.698 0.363 0.309 0.377 0.260

0.349 0.334 0.317 0.309 0.187 0.165

0.487 0.451 0.437 0.421 0.444 0.343

0.000 0.000 0.000 0.000 0.000 0.000

−0.04

0.315 0.728 0.397 0.296 0.328 0.285

0.388 0.442 0.370 0.302 0.193 0.153

0.541 0.589 0.532 0.359 0.442 0.348

0.000 0.000 0.000 0.000 0.000 0.000

TransTab VIME SCARF DAE TabBinning SAINT SubTab TabTransf.

0.410 0.242 0.350 0.265 0.387 0.251 0.275 0.257

0.258 0.150 0.148 0.152 0.141 0.169 0.034 0.097

0.419 0.364 0.384 0.330 0.371 0.165 0.154 0.068

0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000

−0.15

−0.16

0.311 0.295 0.352 0.297 0.421 0.286 0.331 0.301

0.363 0.148 0.107 0.145 0.141 0.163 0.014 0.066

0.482 0.370 0.408 0.360 0.417 0.189 0.229 0.111

0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000

Avg.

0.345

0.201

0.346

0.000

−0.14

0.353

0.214

0.384

0.000

−0.36 −0.05 +0.00 −0.19 −0.10

−0.09 −0.20 −0.11 −0.25 −0.08 −0.24

All (16 pairs) F1 ↑

∆L-C

Cos

Lin

MLP

Dum

∆L-C

Cos

Lin

MLP

Dum

∆L-C

+0.07

0.390 0.511 0.354 0.389 0.341 0.387

0.093 0.072 0.090 0.092 0.048 0.023

0.379 0.550 0.365 0.188 0.245 0.152

0.000 0.000 0.000 0.000 0.000 0.000

−0.30

0.371 0.659 0.369 0.326 0.356 0.298

0.295 0.295 0.273 0.253 0.154 0.126

0.473 0.510 0.443 0.347 0.394 0.296

0.000 0.000 0.000 0.000 0.000 0.000

−0.08

0.041 0.050 0.037 0.051 0.041 0.032 0.000 0.022

0.760 0.149 0.102 0.145 0.095 0.235 0.017 0.018

0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000

−0.53

−0.40

0.425 0.301 0.352 0.314 0.395 0.304 0.327 0.310

0.230 0.125 0.110 0.125 0.116 0.133 0.020 0.071

0.520 0.311 0.320 0.291 0.313 0.188 0.138 0.066

0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000

−0.19

−0.24

0.567 0.423 0.357 0.429 0.383 0.429 0.428 0.426

−0.14

0.415

0.049

0.243

0.000

−0.37

0.365

0.166

0.330

0.000

−0.20

−0.29 −0.03 +0.01 −0.14 −0.13 +0.05 −0.15 −0.24 −0.15 −0.28 −0.12 −0.32

−0.44 −0.26 −0.30 −0.29 −0.36

−0.37 −0.32 −0.38 −0.34 −0.40 −0.43

−0.36 −0.10 −0.07 −0.20 −0.17

−0.18 −0.24 −0.19 −0.28 −0.17 −0.31 −0.24

Table 27 compares cosine similarity, linear probing, an MLP probe, and a dummy baseline across 14 models on the record linkage task. Cosine outperforms linear for most models. The average ∆L-C is −0.20 overall, meaning supervised linear probing hurts relative to unsupervised cosine matching. This holds across all three dataset families and is most severe on WDC (−0.37), where linear probes average only F1 =0.049 versus cosine’s 0.415. A nonlinear head recovers most of what the linear probe leaves on the table. Averaged across all 16 pairs, the MLP head lifts F1 by +0.164 over the linear probe (0.330 vs. 0.166). The gap is largest on WDC (+0.194) where linear probes nearly collapse, and still substantial on DM-D (+0.170) and DM-C (+0.145). At the model level, WDC produces the most dramatic reversals for T RANS TAB (0.041 Lin → 0.760 MLP), GTE (0.072 → 0.550), and BERT (0.093 → 0.379). These models encode usable entity-matching structure that a linear probe cannot access in 5-seed training. Overall, the MLP score 0.330 (All) sits below cosine’s 0.365, with MLP beating cosine on DM-D (+0.031) and matching it on DM-C (+0.001). Cosine retains its lead on WDC (−0.172). WDC is the hardest for linear probes, and still hard for some MLP probes. The linear probe nearly collapses on WDC for target-table SSL models (e.g., SubTab: 0.000, TabTransf.: 0.022), whereas cosine similarity remains non-trivial (∼0.38–0.43). The MLP head partially rescues this for T RANS TAB (0.760, target-table SSL with cross-table contrastive objective), GTE (0.550), BERT (0.379), and TAB ICL (0.245), but it stays near-zero for the target-table SSL encoders on WDC (SubTab: 0.017, TabTransf.: 0.018, TabBinning: 0.095, SCARF: 0.102). These embeddings appear to lack WDC-relevant entity-matching signal rather than merely hiding it in non-linear form. Implication for the main evaluation. Main record linkage results follow the unified supervisedprobe protocol of Sec. 3.1, which averages the linear and MLP probe heads. We headline avg(linear, MLP) rather than cosine because it is the only head that applies uniformly to every row sub-task (cosine is undefined for prediction) and because the linear and MLP heads span the linear-vs-nonlinear capacity axis at a fixed supervision level. Averaging does not privilege either regime, which matters because some encoder/source combinations carry linkage signal that is linearly accessible (text encoders on DeepMatcher) while others need a nonlinear readout (T RANS TAB, GTE, BERT on WDC). This appendix disentangles the two: the MLP head recovers a meaningful slice of the transferable matching signal that cosine can extract, while the linear probe under-reads it, especially on WDC. Averaging the two therefore pulls headline numbers downward for models with strong nonlinear structure (T RANS TAB, GTE, BERT on WDC), and the cosine column here is best read as a training-free reference rather than an upper bound. The relative strength of learned versus training-free matching varies substantially across dataset families, with cosine stronger on WDC and the MLP head competitive on the DeepMatcher benchmarks.

46

Table 28: Source-split row overlap audit for the 16 record-linkage datasets in TRL-R BENCH. Pair overlap is the fraction of test pairs that also appear in the train pair list. Row overlap reports the fraction of distinct test-side tableA / tableB rows that already appear in the train+valid pair lists. The original DeepMatcher [53] and WDC LSPM [62] splits are pair-disjoint by construction (last column), but most sources keep individual rows across splits because each row participates in many candidate pairs. Beer, iTunes-Amazon, iTunes-Amazon-D, WDC-medium, and WDC-small are the only sources where fewer than half of the test-side rows are seen during training on at least one side. Test rows in train+valid Dataset

L.4

train

valid

test

tableA (%)

tableB (%)

Pair overlap (%)

DeepMatcher Clean (DM-C) Abt-Buy 5,743 Amazon-Google 6,874 Beer 268 DBLP-ACM 7,417 DBLP-Scholar 17,223 Fodors-Zagats 567 iTunes-Amazon 321 Walmart-Amazon 6,144

1,916 2,293 91 2,473 5,742 190 109 2,049

1,916 2,293 91 2,473 5,742 189 109 2,049

94.2 90.9 40.8 87.0 95.8 81.0 25.0 87.9

95.0 85.5 51.8 87.7 69.5 86.8 15.1 54.8

0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00

DeepMatcher Dirty (DM-D) DBLP-ACM-D 7,417 DBLP-Scholar-D 17,223 iTunes-Amazon-D 321 Walmart-Amazon-D 6,144

2,473 5,742 109 2,049

2,473 5,742 109 2,049

87.0 95.8 25.0 87.9

87.7 69.5 15.1 54.8

0.00 0.00 0.00 0.00

WDC Products LSPM v2 WDC-small 7,230 WDC-medium 20,453 WDC-large 82,714 WDC-xlarge 171,714

1,808 5,114 20,683 42,947

4,398 4,398 4,398 4,398

22.0 53.4 74.2 75.0

21.5 49.1 68.9 70.8

0.00 0.00 0.00 0.02

Record Linkage Split and Leakage Audit

The 16 record-linkage sources keep their original DeepMatcher [53] and WDC LSPM [62] pairdisjoint splits, so essentially no pair appears in both train and test (pair overlap ≤ 0.02% across all 16 datasets in Table 28). Because each tableA / tableB row participates in many candidate pairs, however, the same row can appear on both sides of the split: in 11 of 16 sources, more than half of the test-side rows on both tableA and tableB already appear in the train+valid pair lists (e.g., Abt-Buy: 94.2% / 95.0%; DBLP-Scholar: 95.8% / 69.5%). Beer, iTunes-Amazon, iTunes-Amazon-D, WDC-medium, and WDC-small are the five sources with cross-split row overlap below 50% on at least one side. We keep all sources to remain comparable with the entity-matching literature. Removal of label-equivalent columns. WDC LSPM v2 raw records ship with three columns that do not belong in feature input: cluster_id is the gold product cluster identifier (column equality reproduces the test label at 99.5% precision and 99.6% recall on every WDC size), and identifiers contains GTIN/MPN unique product IDs (99.8% precision and 37.5% recall) that are excluded from features in the standard WDC LSPM evaluation protocol. Fodors-Zagats analogously exposes a class column whose value is the entity cluster ID (100% precision, 100% recall on the test split). TRL-B ENCH removes cluster_id and identifiers from the four WDC tables and class from Fodors-Zagats before any encoder serializes a row, so the gold label cannot enter the row representation as a feature. The retained columns are brand, category, description, keyValuePairs, price, specTableContent, title for WDC, and name, addr, city, phone, type for Fodors-Zagats. Row-disjoint strict-test ablation. To check that the row-overlap reported above does not distort cross-model rankings, we build a row-disjoint variant of each source: train+valid stay as-is, and the test pair list is filtered to pairs whose tableA and tableB rows do not appear in train+valid. The filter strips most pairs from the high-overlap sources, so we report the strict ablation only on the 10 sources whose strict-test stays ≥ 30 pairs and keeps a ≥ 10% minority class: 6 DeepMatcher viable sources (Amazon-Google, Beer, iTunes-Amazon / iTunes-Amazon-Dirty, WalmartAmazon / Walmart-Amazon-Dirty) and 4 WDC sizes; the 6 skipped sources (Abt-Buy, Fodors-Zagats, DBLP-ACM / -Dirty, DBLP-Scholar / -Dirty) lose either viable pair count or label balance. Rerunning the unified probe protocol of Sec. 3.1 on the strict-test subset at seed 42 and comparing per-model strict-test NR rankings with canonical 5-seed-mean original-protocol rankings on the same 47

10 viable strict-test sources, the strict-vs-original linkage rankings are highly correlated across all 14 row models: Spearman ρ = 0.94 (p = 5.6 × 10−7 ) over the full 10-source set, ρ = 0.97 on DM-viable (6 sources), and ρ = 0.95 on WDC (4 sources). The model-family conclusions of Sec. 4.3 (Transfer-Based encoders dominate Robust Linkage; Target-Table SSL trail) hold under the strict-test ablation. Row overlap does not change the qualitative reading. Table 29 reports per-source strict-test pair counts, positive rates, and absolute F1 for the two top-ranked Robust Linkage models (GTE and T RANS TAB). Table 29: Strict-test row-disjoint ablation at seed 42: per-source pair counts and the per-model linkage F1 (avg of MLP and linear probes) of the two top-ranked Robust Linkage row models (GTE and T RANS TAB), compared with the legacy 5-seed avg. on the same source. Strict-test rows are absent from train+valid by construction (Sec. L.4). “Strict F1 ” is single-seed (seed 42). GTE Source Amazon-Google Beer iTunes-Amazon iTunes-Amazon-D Walmart-Amazon Walmart-Amazon-D WDC-small WDC-medium WDC-large WDC-xlarge

Strict pairs Retained % pos Strict F1 Legacy F1 50 30 70 70 105 105 2718 1195 587 577

2.2% 33.0% 64.2% 64.2% 5.1% 5.1% 61.8% 27.2% 13.3% 13.1%

74.0% 33.3% 30.0% 30.0% 45.7% 45.7% 26.9% 23.8% 15.8% 15.3%

0.700 0.502 0.565 0.640 0.389 0.415 0.289 0.306 0.303 0.309

48

0.310 0.411 0.665 0.677 0.199 0.204 0.313 0.307 0.314 0.311

T RANS TAB ∆ +0.389 +0.092 -0.100 -0.037 +0.190 +0.210 -0.025 -0.001 -0.011 -0.002

Strict F1 Legacy F1 0.636 0.297 0.493 0.648 0.401 0.258 0.264 0.228 0.262 0.190

0.281 0.285 0.574 0.649 0.169 0.171 0.402 0.391 0.406 0.401

∆ +0.354 +0.011 -0.080 -0.001 +0.232 +0.087 -0.139 -0.163 -0.144 -0.212

L.5

Intrinsic Embedding-Geometry Diagnostics for Row Encoders

We complement TRL-R BENCH’s downstream scores with an intrinsic embedding-geometry analysis of the same exported row embeddings used by the standardized protocol. Using eight established diagnostics from the broader representation-learning literature, covering spectral spread, spectral shape, and spatial structure (“task-free” metrics in the prior literature), we ask which geometric properties of row-embedding spaces co-rank with downstream utility on row prediction and record linkage. To our knowledge, prior tabular benchmark resources do not pair representation-level downstream evaluation with intrinsic embedding-geometry diagnostics on the same encoder outputs under a common protocol. Diagnostic families and formulas. We group the eight task-free diagnostics used in this analysis into three complementary families that we introduce here to organise the discussion: Spectral Spread, Spectral Shape, and Spatial Structure. Each underlying metric is drawn from prior work (cited at its definition below). The three-family taxonomy itself is our framing. For each (model, dataset) we form a single frozen embedding matrix X ∈ Rn×d , the row-embedding matrix for that table (for target-table SSL encoders this is the same matrix the encoder was trained on; for frozen transfer and prior-based encoders it is the inference-time row matrix), and evaluate all eight diagnostics from its singular value decomposition X = U Σ V ⊤, (1) and from the eigenspectrum λ1 ≥ · · · ≥ λd ≥ 0 of the centred covariance C = n1 Xc⊤ Xc ,

Xc = X − 1 x̄⊤ ,

x̄ = n1 X ⊤ 1.

(2)

Let σ1 ≥ σ2 ≥ · · · denote the singular values of X and r its numerical rank. All eight diagnostics are deterministic functions of X. Spectral Spread. If variance in an embedding matrix concentrates in only a handful of singular directions, most of the ambient dimensions are redundant: the representation effectively lives on a low-dimensional subspace, and the remaining capacity is unavailable to the downstream head. Spectral Spread diagnostics measure how evenly variance is allocated across the spectrum. Higher values mean more independent directions actively carry information, a necessary condition for rich, transferable embeddings. RankMe [26].  min(n,d)  X RankMe(X) = exp − pi log pi , i=1

σi pi = P . j σj

(3)

The exponentiated entropy of the singular-value distribution. It equals 1 for a rank-one spectrum and min(n, d) when all singular values are equal. RankMe⋆ [26, 71]. RankMe(X) ∈ [0, 1]. (4) min(n, d) A dimension-normalised variant: the fraction of the available embedding dimensions the representation actually uses. RankMe⋆ (X) =

NESum. d

NESum(X) =

1 X λi . λ1 i=1

(5)

Total variance divided by leading variance. It equals 1 under rank-one collapse and approaches min(n, d) as the covariance spectrum flattens. Spectral Shape. Two embeddings can have the same effective rank and still look very different along the spectrum: one may drop off sharply after the top few singular directions, while another decays as a slow power-law that keeps weak but non-trivial signal in many more directions. Spectral Shape diagnostics capture this tail profile, which determines how much low-variance structure survives to support tasks whose discriminative signal is not confined to the dominant directions. Pseudo κ. κ(X) =

σmax . σmin nonzero 49

(6)

A coarse condition-number proxy. Large κ signals a near-degenerate spectrum in which a few directions dominate. αreq [1]. αreq (X) = −β̂1 ,

log λi = β0 + β1 log i + ϵi ,

i = 1, . . . , r,

(7)

where (β̂0 , β̂1 ) is the ordinary least-squares estimator. That is, αreq is the decay exponent of a power-law fit λi ∝ i−α to the centred-covariance spectrum. Larger αreq means a faster-decaying, more heavy-tailed spectrum. Convention note on αreq . We follow the original Agrawal et al. [1] definition, which fits the slope on covariance eigenvalues λi . Tsitsulin et al. [71] restate the same metric on singular values σi . Because λi = σi2 /n, the two fitted slopes differ by exactly a factor of two, so our reported αreq values live on the Agrawal scale and should be divided by 2 to compare against Tsitsulin-scale results. To prevent power-law fits through float-precision noise on (nearly) collapsed spectra we drop eigenvalues below ε · d · λ1 (numpy’s default rank tolerance) before the fit. When fewer than two eigenvalues survive we record αreq = NaN rather than a spurious finite slope. Spatial Structure. Two embeddings can share an identical singular-value spectrum yet lay their points out very differently: one spreading them uniformly, another concentrating them on a lowdimensional manifold or in tight clusters. Spatial Structure diagnostics measure these point-cloudlevel properties, which are invisible to the spectrum alone and reveal whether a representation has acquired a meaningful geometric organisation (beneficial for retrieval- and clustering-style downstream tasks) or has instead collapsed structure that a purely spectral view would not detect. dˆTwoNN [22, 4]. dˆTwoNN (X) = β̂1 ,

k − log 1 − N

= β0 + β1 log µ(k) + ϵk ,

k = 1, . . . , ⌊0.9N ⌋,

(8)

where (β̂0 , β̂1 ) is the ordinary least-squares estimator, µi = d2,i /d1,i with d1,i , d2,i the distances from row i to its first and second nearest neighbours, µ(k) the sorted statistic, and N the sample size. This is the manifold-hypothesis intrinsic dimension: the number of independent directions the data actually spans. We average over 20 random 90% subsets of at most 2000 rows (seed 42). Coherence µ0 .   n d 2 2 max ∥Ui,: ∥2 , max ∥Vj,: ∥2 , (9) µ0 (X) = max r i r j computed on the top-r singular vectors. Coherence is high when a small number of rows or dimensions disproportionately drive the representation, and low when energy is spread evenly across all of them. Self-Cluster [71]. SC(X) =

d ∥X̃ X̃ ⊤ ∥2F − n(d + n − 1) , (d − 1)(n − 1) n

X̃i,: =

Xi,: . ∥Xi,: ∥2

(10)

Zero in expectation when the row-normalised vectors are i.i.d. isotropic, positive under clustering, so it measures how much the embedding departs from a uniform distribution on the sphere. Rows with zero norm (from upstream non-finite sanitisation) are dropped before normalisation. Table 30 reports the Spearman rank correlation (ρ) between eight diagnostics, grouped into the three families above, and downstream performance on the two row-level task categories. Each cell is a per-task average: we compute Spearman ρ within every task (across 13–14 row-capable models per task after task-specific filtering, with RANDOM excluded throughout), then average the per-task ρ’s. The p-values come from Wilcoxon signed-rank tests of the per-task ρ distribution against zero. This matches the aggregation used in the pertask breakdown tables (Tabs. 32–34), so the headline numbers and the per-head detail tell a single consistent story. The Row Prediction column combines classification and regression tasks, using macro-F1 as the performance-oriented score for classification rows and −nRMSE for regression rows. The per-task-type breakdowns (pure-classification vs. pure-regression, per-head) are in Tables 33 and 34. Table 31 provides the per-task breakdown for Row Prediction (Regression) with MLP and linear heads side by side. Tables 32–34 report per-head detail for all three task types, combining MLP and Linear heads side by side for direct comparison. Spectral Spread is the most predictive family. RankMe (ρ̄ = +0.485, p < 0.001), RankMe* (+0.471), and NESum (+0.460) are the strongest observed positive correlates: embeddings with 50

4 5 3 5 6 2 4 3 6 8 6 5 7 4 5 3

1 1 5 6 5 4 9 5 9 7 7 4 7 3 3

2 1 3 5 3 5 3 3 4 3 4 5 3 6 3 6 7 7 4 6 8 4 7 8 10 10 7 3 4 1 4 3

1 1 4 4 2 2 2 3 3 2 2 3 1 3 2 3 5 3 4 1 1 2 5 5 7 7 4 7 5 2 9 5 2 8 5 3 9 8 1 8 22 1 6 1 10

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

20.0 17.5 15.0 12.5 10.0 7.5 5.0 2.5 0.0

Rank of NESum across models (1 = highest metric value)

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

6 6 5 5 3 4 2 1 1 5 2 4

4 5 6 2 5 4 5 4 1 1 4 1 1 3

2 3 3 6 9 7 4 1 3 3 2 1 1 1

5 4 4 7 3 4 4 7 1 2 1 1 1 2

2

1 4 8 6 5 6 2 2 3 3 1 3 1 1

6 2 3 3 5 5 1 1 3 4 5 1 5 2

8 3 2 1 4 5 3 2 1 2 2 3 1 2 6 1

4 4 1 2 2 3 3 4 4 1 1 4 4 5 3 1

2 2 3 3

3 5 4 3 4 2 7 2 5 1

1 2 1 2 5 2 5 4 3 5 4 7 4 1

1 1 1 2 1 1 4 4 3 5 6 4 2 5 3 3

3 2 2 3

2 4 3 6 5 4 2 8 2

2 2 4 1 2 1 3 3 2 4 8 2 3 7 2

1 4 1 2 1 1 3 3 5 3 2 2 8 5 3 2

1 1 2 4 2 3 5 6 3 4 3 3 4 5

1 2 1 1 2 1 4

Record Linkage (linear head) Best metric: req ( Spearman = 0.80, 16 tasks, 276 pairs) 9 8 7

Count of (task, model) pairs

5 5 3 3 1 8 9 4 6 2 7 3 8 2 8 6 5 5 7 5 8 6 7 5 1 6 4 2 7 4 5 4 3 6 5 6 8 4 5 10 3 1 6 5 1 3 1

6 5 4

2 1

3 2

3

Rank of performance across models (1 = best after direction correction)

4 14 5 6 6 6 9 8 8 6 4 7 8 7 8 7 2 4 5 9 4 6 2 1 7 3 4 5 5 2 2 3 8 3 1 9 3 1 1 3 4 2 2 5 3 2 4

Count of (task, model) pairs

10 15 6 7 8 7 4 7 8 6 9 8 5 4 7 6 6 10 4 5 4 8 3 8 6 4 3 3 3 4 2 5 2 1 3 3 2 1 4 3 3 4 4 2 1

Rank of performance across models (1 = best after direction correction)

Rank of performance across models (1 = best after direction correction)

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

1

3

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

0

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

1 1

5 1 1

1 1 1 1 2 1 4 1

1 1 1 1

2

2

1 5 1 4

1 1 1 3 3 3

2 2 1 1 4

2 1 5 1 2 1 3

2 1 3 1 3 2 2

1 2

3 2 3 2 2 2 2 1 2 1

2 3 3 2

1 2 1 3 2 3 1 1 1

4 2 1

3 4 3

2 3

3 4 2 2 1

2 1 4 1 2 1

7 3 1 1 1

2 1

2

2

1 2

1 2

5 2 3 1

1 5 1 1 4 1

7 4 1 2 1 2

6 5

Count of (task, model) pairs

Regression (mlp head) Best metric: RankMe ( Spearman = +0.36, 46 tasks, 711 pairs)

Classification (mlp head) Best metric: NESum ( Spearman = +0.32, 76 tasks, 1182 pairs)

4 2

1

3 1

2

2 1

1 1

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

0

Rank of req across models (1 = highest metric value)

Rank of RankMe across models (1 = highest metric value)

(a) Classification (MLP). Best- (b) Regression (MLP). Best- (c) Record Linkage (Linear). Bestcorrelated prior metric: NESum correlated prior metric: RankMe* correlated prior metric: αreq (ρ̄ = (ρ̄ = +0.32, 76 tasks, with one of (ρ̄ = +0.36, 46 tasks). Density on −0.80, 16 tasks). Density on the 77 classification targets dropped for the main diagonal. anti-diagonal. constant performance across models on MLP). Density on the main diagonal.

Figure 9: Rank–rank density heatmaps for row tasks. For every task, models are ranked by the best-correlated intrinsic-geometry diagnostic (x-axis, 1 = highest value) and by direction-corrected downstream performance (y-axis, 1 = best). Cell numbers count (task, model) pairs per rank bin. Diagonal concentration indicates positive rank agreement. Anti-diagonal concentration indicates negative agreement. Table 30: Spearman rank correlation (ρ) between embedding quality metrics and downstream task performance, reported as per-task averages so the numbers align with the pertask breakdown tables (Tabs. 32–34). Record Linkage: Spearman ρ computed per (dataset, head) across 16 models, then averaged over n = 32 (dataset, head) tasks (16 datasets × {MLP, linear}). Row Prediction: Spearman ρ per (dataset, label, head) across models, averaged over n = 241 task-head cells: classification (76 MLP + 73 linear) and regression (46 MLP + 46 linear) from 77 classification and 46 regression targets crossed with {MLP, linear} probe heads, less 5 classification rows dropped for near-constant performance (undefined ρ). Overall: unweighted mean of Record Linkage and Row Prediction ρ. Reported p is a two-sided Wilcoxon signed-rank test of the per-task ρ distribution against zero. Overall p is the smaller of the two (anti-conservative). Bold ρ indicates p < 0.05. The RANDOM baseline encoder is excluded throughout.

Record Linkage Row Prediction ρ

p

Overall

Family

Metric

ρ

p

ρ̄

p

Spec. Spread

RankMe RankMe* NESum

+0.714 +0.684 +0.657

<.001 <.001 <.001

+0.256 <.001 +0.485 <.001 +0.258 <.001 +0.471 <.001 +0.262 <.001 +0.460 <.001

Spec. Shape

Pseudo κ αreq

+0.116 -0.746

0.036 <.001

+0.032 +0.003

0.039 0.939

dˆTwoNN +0.398 Spatial Struct. Coherence µ0 -0.549 Self-Cluster -0.182

<.001 <.001 <.001

-0.042 -0.030 -0.237

0.095 +0.178 <.001 0.235 -0.289 <.001 <.001 -0.210 <.001

+0.074 0.036 -0.372 <.001

more uniform singular-value distributions (i.e., higher effective rank) consistently rank higher on both tasks. The αreq metric in Spectral Shape is the strongest observed negative correlate overall (ρ̄ = −0.372), but its signal is concentrated on Record Linkage (ρ = −0.746, p < 0.001). On Row Prediction the per-task mean is essentially zero (ρ = +0.003, p = 0.94) because the classification and regression signs partially cancel (see the per-head breakdown in Tab. 31 and the companion classification table). Heavy-tailed spectral decay therefore predicts entity-matching performance but is an ambivalent signal for feature-based prediction. Within Spatial Structure, dˆTwoNN is informative 51

for Record Linkage (ρ = +0.398, p < 0.001) but shows no significant signal for Row Prediction (ρ = −0.042, p = 0.095), indicating task-specific utility. Correlations are uniformly stronger for Record Linkage than Row Prediction, implying that intrinsic quality metrics are better proxies for entity-matching tasks than for feature-based prediction. Table 31: Row Prediction, Regression: per-task correlation between embedding metrics and −nRMSE, computed across models within each (dataset, label), then aggregated. MLP Head (46 tasks) Pearson

Spearman

Linear Head (46 tasks) Dir. Rate

Pearson

Spearman

Dir. Rate

Family

Metric

Mean

Spec. Spread

RankMe RankMe* NESum

+0.317 +0.410 +0.328 +0.480 80.4% 78.3% +0.235 +0.395 +0.257 +0.418 78.3% 76.1% +0.341 +0.462 +0.356 +0.502 80.4% 78.3% +0.261 +0.415 +0.287 +0.472 78.3% 73.9% +0.248 +0.346 +0.250 +0.296 78.3% 76.1% +0.225 +0.303 +0.231 +0.315 76.1% 80.4%

Spec. Shape

Pseudo κ αreq

+0.118 -0.021 +0.036 +0.118 50.0% 56.5% +0.093 -0.038 -0.006 +0.054 56.5% 54.3% +0.012 +0.014 +0.080 +0.144 52.2% 63.0% +0.119 +0.141 +0.186 +0.229 63.0% 76.1%

dˆTwoNN Spatial Struct. Coherence µ0 Self-Cluster

-0.120 -0.042 -0.316

Med.

Mean

Med.

P

S

Mean

Med.

Mean

Med.

P

S

-0.288 -0.105 -0.230 65.2% 63.0% -0.165 -0.212 -0.174 -0.266 65.2% 69.6% -0.044 +0.034 +0.079 52.2% 58.7% +0.011 +0.031 +0.097 +0.137 56.5% 67.4% -0.374 -0.327 -0.414 80.4% 82.6% -0.281 -0.408 -0.296 -0.368 73.9% 80.4%

Row Prediction (Regression) is the most variable. Spectral Spread is the strongest positive signal: RankMe and RankMe* reach MLP Pearson +0.317/+0.341 (Spearman +0.328/+0.356, ∼80% directional), with the linear head slightly weaker (Pearson +0.235/+0.261). NESum follows closely (MLP Pearson +0.248, Spearman +0.250). Spectral Shape is much weaker: Pseudo κ registers MLP Pearson +0.118 (Spearman +0.036), and αreq is essentially flat on the MLP head (Pearson +0.012, Spearman +0.080), emerging only on the linear head (Pearson +0.119, Spearman +0.186). The dominant negative signal is in Spatial Structure: Self-Cluster reaches MLP Pearson −0.316 and Spearman −0.327 (∼80%/83% directional), meaning that tighter intra-model clustering of row embeddings co-varies with worse regression. dˆTwoNN is consistently weakly negative (MLP Pearson −0.120, Spearman −0.105), and Coherence µ0 is near-zero. Relative to classification, regression amplifies both the Spectral Spread positive signal and the Self-Cluster / dˆTwoNN negative signals. αreq flips from weakly negative (classification, MLP Pearson −0.105) to flat or weakly positive (regression). Note: the per-task correlations here (and in Tables 32–34) exclude the RANDOM baseline, matching Table 30. RANDOM is a high-leverage NESum outlier (NESum ≈ 409 vs. ≈4.1 for real encoders, roughly 102 × on this aggregation basis) and can exert undue leverage on scale-sensitive Pearson fits.

52

L.6

Intrinsic-Geometry Diagnostics: Per-Head Breakdowns

The following three tables combine MLP and Linear heads side by side for each task type, enabling direct comparison of how probe complexity modulates the predictive power of each quality metric. Table 32: Per-task correlation between embedding metrics and record linkage performance (F1 , higher is better). We report MLP, Linear, and Cosine-Threshold heads. MLP head Metric NESum RankMe RankMe⋆ αreq Pseudo κ µ0 -coherence Self-cluster TwoNN ID

Spearman

Pearson

0.67

0.58

[0.53, 0.76]

[0.44, 0.69]

0.69

0.59

[0.54, 0.80]

[0.42, 0.71]

0.68

0.60

[0.53, 0.79]

[0.44, 0.71]

−0.69

−0.69

Linear head SC 0.94 0.94 0.94

[−0.80, −0.54] [−0.77, −0.59]

0.11

0.16

[−0.01, 0.25]

[−0.04, 0.36]

−0.57

−0.44

0.62

[−0.67, −0.45] [−0.53, −0.35]

−0.25

−0.22

[−0.33, −0.17] [−0.29, −0.15]

0.33

0.13

[0.20, 0.46]

[−0.06, 0.31]

0.94

0.94 0.94 0.81

Cosine Thr. head

Spearman

Pearson

0.64

0.59

[0.58, 0.71]

[0.52, 0.66]

0.74

0.70

[0.67, 0.79]

[0.64, 0.74]

0.69

0.63

[0.63, 0.74]

[0.57, 0.70]

−0.80

−0.59

[−0.85, −0.73] [−0.71, −0.47]

0.12

0.05

[−0.02, 0.24]

[−0.11, 0.20]

−0.53

−0.40

[−0.61, −0.46] [−0.48, −0.32]

−0.11

−0.05

[−0.19, −0.03]

[−0.12, 0.04]

0.46

0.32

[0.35, 0.57]

[0.18, 0.47]

SC 1.00 1.00 1.00 1.00 0.75 1.00 0.81 1.00

Spearman

Pearson

0.54

0.73

[0.45, 0.63]

[0.62, 0.83]

0.56

0.83

[0.48, 0.64]

[0.78, 0.88]

0.52

0.77

[0.43, 0.61]

[0.69, 0.84]

−0.52

−0.44

[−0.60, −0.45] [−0.51, −0.37]

−0.02

−0.11

[−0.13, 0.10]

[−0.22, 0.00]

−0.54

−0.38

[−0.61, −0.47] [−0.44, −0.32]

−0.18

−0.18

[−0.24, −0.12] [−0.24, −0.12]

0.07

0.10

[−0.08, 0.20]

[−0.04, 0.25]

SC 1.00 1.00 1.00 1.00 0.56 1.00 0.94 0.53

Table 33: Per-task correlation between embedding metrics and classification performance (macro-F1 , higher is better). Spearman / Pearson cells show mean with bootstrap 95% CI in brackets over per-task correlations. SC is the sign-consistency fraction (fraction of tasks whose Spearman is in the dominant-sign direction). MLP head Metric NESum RankMe RankMe⋆ αreq Pseudo κ µ0 -coherence Self-cluster TwoNN ID

Linear head

Spearman

Pearson

0.32

0.29

SC

[0.25, 0.38]

[0.23, 0.35]

0.28

0.26

[0.20, 0.35]

[0.20, 0.32]

0.27

0.26

[0.19, 0.34]

[0.20, 0.32]

−0.08

−0.11

[−0.15, −0.02] [−0.19, −0.02]

0.06

0.14

[−0.01, 0.13]

[0.07, 0.22]

−0.11

−0.21

[−0.18, −0.04] [−0.29, −0.13]

−0.23

−0.22

[−0.31, −0.16] [−0.28, −0.15]

0.05

−0.01

[−0.02, 0.12]

[−0.10, 0.09]

53

0.86 0.79 0.76 0.61 0.60 0.64 0.76 0.53

Spearman

Pearson

0.23

0.27

[0.17, 0.29]

[0.22, 0.32]

0.19

0.23

[0.11, 0.26]

[0.17, 0.29]

0.17

0.22

[0.09, 0.25]

[0.15, 0.28]

−0.07

−0.09

[−0.14, 0.00]

[−0.17, −0.01]

0.02

0.16

[−0.04, 0.09]

[0.08, 0.24]

−0.07

−0.19

[−0.14, 0.00]

[−0.28, −0.10]

−0.15

−0.16

[−0.23, −0.07] [−0.23, −0.10]

−0.02

−0.05

[−0.10, 0.07]

[−0.14, 0.04]

SC 0.79 0.74 0.66 0.60 0.55 0.61 0.67 0.58

Table 34: Per-task correlation between embedding metrics and regression performance (nRMSE = √ 1 − R2 , lower is better, with per-task SGM(nRMSE) = nRMSE). We correlate against −nRMSE so positive values mean “higher metric → better performance.” MLP head Metric NESum RankMe RankMe⋆ αreq Pseudo κ µ0 -coherence Self-cluster TwoNN ID

Spearman

Linear head

Pearson

SC

0.25

0.25

[0.14, 0.36]

[0.14, 0.35]

0.33

0.32

[0.22, 0.43]

[0.22, 0.41]

0.36

0.34

[0.24, 0.46]

[0.24, 0.43]

0.08

0.01

[−0.01, 0.17]

[−0.09, 0.11]

0.04

0.12

[−0.06, 0.13]

[0.02, 0.22]

0.03

−0.04

[−0.09, 0.15]

[−0.16, 0.07]

−0.33

−0.32

[−0.42, −0.23] [−0.41, −0.22]

−0.11

−0.12

[−0.23, 0.02]

[−0.25, 0.02]

54

0.76 0.78 0.78 0.63 0.57 0.59 0.83 0.63

Spearman

Pearson

0.23

0.23

[0.12, 0.33]

[0.12, 0.33]

0.26

0.23

[0.13, 0.38]

[0.11, 0.35]

0.29

0.26

[0.16, 0.41]

[0.14, 0.37]

0.19

0.12

[0.09, 0.28]

[0.02, 0.22]

−0.01

0.09

[−0.09, 0.08]

[−0.00, 0.19]

0.10

0.01

[−0.02, 0.22]

[−0.11, 0.12]

−0.30

−0.28

[−0.41, −0.18] [−0.40, −0.16]

−0.17

−0.17

[−0.31, −0.04] [−0.31, −0.02]

SC 0.80 0.76 0.74 0.76 0.54 0.67 0.80 0.70

M

DLTE Operator Specification

This section specifies the complete DLTE pipeline operators summarized in Sec. 3.4. Algorithm 1 is the end-to-end procedure. The following subsections give the CSLS formula, Stage-2 threshold grid, Stage-3 match-profile scalars, and the union-appended-row second-pass in Stage 3. Nothing below is trained end-to-end: Stage-2 calibrates five scalars per (Stage-1 backbone, column model) pair on the dev split by grid search over macro-F1 . Stage-1 and Stage-3 have no tunable parameters beyond the fixed profile scalars reported here. Algorithm 1 TRL-DLTE pipeline (per query q, lake L). Require: Frozen encoder outputs etbl , ecol , erow ; retrieval depth K = 100; Stage-2 thresholds τfloor , τu , τus , τjm , τks calibrated per (Stage-1 backbone, column model) pair (Table 35); Stage-3 match profiles Πu (union) and Πj (join) from Table 36. 1: Stage 1 (table retrieval). L2-normalize etbl (q) and query the pre-built FAISS [37] inner-product index over the lake’s normalized table embeddings; return top-K candidates CK (q). 2: Stage 2 (column alignment + relation classification). 3: for all c ∈ CK (q) do col 4: Form cost Dij = 1−cos(ecol q,i , ec,j ); solve Hungarian assignment on D to get matched cosine similarities S = (s1 , . . . , sL ). 5: Let n⋆ = |{sk ≥ τfloor }|, r = n⋆ /|C(q)|, µ = mean{sk : sk ≥ τfloor }, m = max S. 6: if r ≥ τu and µ ≥ τus then 7: ŷ(q, c) ← UNION. 8: else if r ≤ τjm and m ≥ τks and n⋆ ∈ {1, 2, 3} then 9: ŷ(q, c) ← JOIN; key pair ← arg maxk sk . 10: else 11: ŷ(q, c) ← NONE. 12: end if 13: end for 14: Stage 3 (row matching + merge). 15: Pick cu ← arg maxŷ(q,c)=UNION κu (q, c) and cj ← arg maxŷ(q,c)=JOIN κj (q, c) over CK (q), if any; ties broken by Stage-1 rank (see Sec. M.3 for κ). 16: Initialize q ′ ← q. 17: if cu exists then 18: Dedup union. Reciprocal-match R(cu ) against R(q) via CSLS + profile Πu ; append unmatched rows of cu to q ′ using the Stage-2 column alignment. 19: end if 20: if cj exists then Join. Iteratively reciprocal-match seed rows R(q) against R(cj ) via CSLS + Πj for up to I j = 10 21: rounds; copy non-key join columns into q ′ . 22: Second pass. Reciprocal-match union-appended rows against as-yet-unmatched R(cj ) (Sec. M.4). 23: end if 24: return enriched query q ′ .

M.1

Stage-1 Retrieval: Scoring and Pool

Given table embeddings etbl (t) ∈ Rd , we L2-normalize each vector and build a FAISS IndexFlatIP over the 47,772-table lake. Inner-product search on unit vectors returns cosine-ranked candidates without approximation error. For the Stage-1 retrieval pool specifically, column-capable encoders are pooled to a table embedding via column mean (variant column_mean). The two native table encoders use their native variants (TAPEX: table_embedding, TUTA: cls_embedding). We retrieve the top K = 100 candidates and pass all of them to Stage 2. M.2

Stage-2 Alignment and Classification: Thresholds and Grid

Hungarian assignment is solved with scipy.optimize.linear_sum_assignment on cost Dij =
col 1 − cos ecol , e with L = min(|C(q)|, |C(c)|) matched pairs. Column embeddings are L2q,i c,j normalized before cosine. NONE is the default label and the per-pair statistics used for classification are the matched similarities S and derived scalars n⋆ , r, µ, m defined in Algorithm 1. When n⋆ = 0 (no matched pair clears τfloor ) we set µ ← 0 by convention. 55

Five thresholds are grid-searched per (Stage-1 backbone, column model) pair on the dev split to maximize three-way macro-F1 over {UNION, JOIN, NONE}. The objective is macro-F1 because NONE dominates (∼98% of pairs) and accuracy optimization collapses to the majority class. Calibration is per-pair because different Stage-1 backbones yield different candidate distributions entering Stage-2, so a per-pair operating point isolates Stage-2 classification quality conditional on the retrieval geometry rather than conflating the two. The grid (Table 35) visits 5 × 6 × 6 × 5 × 5 = 4500 combinations per pair across 10 × 8 = 80 (Stage-1, Stage-2) pairs; total calibration cost is small because per-pair alignments are computed once and only the threshold-dependent per-pair statistics are revisited across the grid. The resulting threshold vector is held fixed across all 14 Stage-3 row models paired with that (Stage-1, Stage-2) pair, and is stored alongside the predictions in the released code. Both calibrated thresholds and downstream metrics are deterministic given the embeddings. Table 35: Stage-2 threshold search space. Reported results always use the dev-selected threshold vector of the corresponding (Stage-1, Stage-2) pair. No shared default vector is used in evaluation.

M.3

Symbol

Interpretation

Grid range (step)

τfloor τu τus τjm τks

Min. per-pair similarity to count as matched Min. match ratio r for UNION Min. mean matched similarity µ for UNION Max. match ratio r for JOIN Min. key-column similarity m for JOIN

[0.70, 0.90] (0.05) [0.50, 1.00] (0.10) [0.70, 0.95] (0.05) [0.20, 0.60] (0.10) [0.75, 0.95] (0.05)

Stage-3 Row Matching: CSLS and Profiles

Candidate-selection scores. Stage 3 consumes, for each query q, the highest-confidence UNION and JOIN candidate from the K Stage-2 outputs. We use a class-conditional score κ matched to the decision rule: for a candidate classified as UNION we set κu (q, c) = min(r, µ) (the binding scalar of the union rule), and for JOIN we set κj (q, c) = m (the key-column similarity). Ties are broken by Stage-1 retrieval rank. All Stage-3 operations reuse the precomputed frozen row embeddings of the source tables; after union append or join merge, the enriched table q ′ is never re-encoded. CSLS similarity. Let M ∈ R|A|×|B| be the raw cosine matrix between two row sets A, B. For kcsls = 5, let ri be the mean of the top-kcsls entries of row i of M and cj the mean of the top-kcsls entries of column j. The CSLS-normalized similarity is sCSLS (i, j) = 2 Mij − ri − cj .

(11)

CSLS [47] discounts hub-like rows/columns whose neighborhoods are dense and makes mutual top-1 pairs more robustly reciprocal. Reciprocal matching with local confidence filters. Given sCSLS , a pair (i, j) is mutual top-1 if j = arg maxj ′ sCSLS (i, j ′ ) and i = arg maxi′ sCSLS (i′ , j). For each row i, let si(1) ≥ si(2) denote its top-two CSLS scores and let µi , σi be the mean and standard deviation of the i-th row of sCSLS . We define the standardized signals  si(1) − si(2) si(1) − µi σi if σi ≥ ϵ margin + zibest = , z = , σ ≡ (12) i i 1 otherwise σi+ σi+ with ϵ = 10−12 (the second case only fires on degenerate rows where every CSLS score is numerically identical), and the analogous candidate-side quantities zjbest , zjmargin computed over the best column axis of sCSLS . A mutual top-1 pair (i, j) is accepted iff min(zibest , zjbest ) ≥ zmin and margin margin margin min(zi , zj ) ≥ zmin . Iterative matching removes accepted pairs and re-computes sCSLS on the remaining rows/columns for up to a profile-specific number of rounds. Profiles Πu and Πj . The two enrichment paths have asymmetric costs: false union appends (duplicate rows) are costlier than missed joins (unmatched new columns on some rows). We therefore use two fixed profile vectors (Table 36) rather than a single shared scalar. Profiles are not tuned per model. They are held fixed across all Stage-3 row encoders, so the reported Stage-3 variation reflects row-embedding geometry rather than operator calibration. 56

Table 36: Stage-3 match profiles. Both profiles use CSLS with kcsls = 5. Profiles are fixed across all row models. Scalar Imax (iterations) best zmin margin zmin smin (absolute floor)

M.4

Πu (union, precision-first)

Πj (join, recall-first)

3 1.00 0.25 disabled

10 0.75 0.10 disabled

Stage-3 Second-Pass Join on Union-Appended Rows

The join phase first matches the seed rows of q against R(cj ). This leaves union-appended rows (new rows introduced by the union path) without join-side coverage. A second reciprocal pass then matches those appended rows against the remaining rows of cj (those not yet consumed by the first pass) under the same profile Πj . Cells are filled for newly matched pairs using the Stage-2 key-pair alignment, and the second-pass match count is logged separately. This mechanism is what produces the hard-region recall (Table 39) of the enriched-table quadrant where new rows meet new columns; without it, hard-region recall collapses to zero for all row models. M.5

What is and is not tuned

Stages 1 and 3 are fully fixed across models and splits: all scalars in Table 36 and the FAISS retrieval depth K are constants, independent of the encoder under evaluation. Stage 2 calibrates a five-scalar operating point per (Stage-1 backbone, column model) pair (80 calibration runs in total) on the dev split, using Stage-2 three-way macro-F1 over {UNION, JOIN, NONE} as the sole objective. Neither test labels nor end-to-end Cell-F1 / UJ-H, and no Stage-3 row-model choice, enter this calibration. The resulting dev-selected threshold vector of each pair is held fixed across all 14 Stage-3 row models paired with that (Stage-1, Stage-2) pair throughout the full pipeline evaluation. Headline pipeline selection (Sec. 4.4) is a separate model-selection step that uses dev UJ-H as the sole criterion and does not enter this Stage-2 calibration.

57

DLTE Detailed Rankings

0.20

0.20

TabTransf.

TabTransf. 0.18

SAINT

SubTab

Stage 3 (row model)

VIME

TabPFN SCARF DAE

TABBIE TabBinning

0.14

VIME

UJ-H (test set)

TABBIE TabBinning

TabPFN

0.12

SCARF DAE

BERT TUTA

TUTA

GTE

(tab

le m

ode l)

A TUT ie Starm

TA TURL BBIE GT BERT E TA TaBERT PAS Ta odel) Starm bSketchFM lumn m ie (co

0.14

0.12

0.10

TabICL GTE

TransTab

GTE

0.16

BERT 0.10

TabICL

BIE TAB hFM ketc TAPAS L TabS TUR RT Stag TaBE TAPEX T e1 BER

0.18

SubTab 0.16

Stage 3 (row model)

SAINT

UJ-H (test set)

N

BIE TAB hFM ketc TAPAS L TabS TUR RT Stag TaBE TAPEX T e1 BER

0.08

(tab

le m

e2

Stag

ode

TransTab

GTE

l)

A TUT ie Starm

TA TURL BBIE GT BERT E TA TaBERT PAS Ta odel) Starm bSketchFM lumn m ie (co

0.08

e2

Stag

(a) Full cube: all 10 × 8 × 14 = 1120 pipelines. Low- (b) Top 40% pipelines only (UJ-H ≥ 60th percentile), UJ-H voxels fade into light blue, high-UJ-H voxels highlighting the strong-performing regions of the cube. pop in deep purple.

Figure 10: Voxel visualisation of the DLTE Stage-3 pipeline space over UJ-H. Axes: Stage 1 (table model, 10) × Stage 2 (column model, 8) × Stage 3 (row model, 14). Colour encodes UJ-H (light blue → deep purple). Axes are reordered by marginal-mean UJ-H so the best-performing corner is contiguous. See the full per-pipeline breakdown in Table 44 (Sec. N.6). Dev/test rank stability. The dev-selection protocol of Sec. 4.4 relies on dev-test pipeline rank similarity. Across all 1,120 canonical pipelines (5-round mean UJ-H per pipeline), Spearman ρ(dev, test) = 0.96 (p ≪ 10−100 , n = 1,120; Kendall τ = 0.84), and the top-50 by dev UJ-H and the top-50 by test UJ-H share 42 of 50 pipelines. This justifies treating dev-selected pipelines as descriptive of the broader test landscape.

58

N.1

Cell F1 as a Complementary Diagnostic

Cell F1 is the multiset F1 score over recovered cells, pooling the removed-row and removed-column blocks for each query. With Cp (q) and Cg (q) the multisets of cells in the pipeline’s predicted enrichment and the ground-truth blocks for query q, Cell F1 (q) =

2 |Cp (q) ∩ Cg (q)| , |Cp (q)| + |Cg (q)|

averaged over queries. It measures how well a pipeline reconstructs parent-table cells in raw cell terms, regardless of how that recovery is distributed between the union and join paths. This makes Cell F1 a complement to UJ-H, the primary end-to-end score for joint recovery of the union and join targets, and we use it here to diagnose stage behavior and high-volume recovery modes in the same 1,120-pipeline space. Per-stage observations. At Stage 1 (table retrieval), the Cell F1 marginal identifies S TARMIE as the strongest retriever (0.601), followed by TUTA (0.593) and GTE (0.591). This agrees with the top of the UJ-H marginal ranking and shows that retrieval quality is a shared driver under both pooled-cell and joint-recovery views (Table 38). At Stage 2 (column alignment plus union/join/none decisions), TA BERT leads on Cell F1 (0.628), ahead of GTE (0.601) and TAPAS (0.600). The Stage 2 Cell F1 span is 0.084, the largest of the three stages, indicating that pooled cell recovery is especially sensitive to the column-side model. At Stage 3 (row matching and merge), the Cell F1 marginals are compressed (span 0.026), with TAB T RANSFORMER (0.591), S UB TAB (0.591), and SAINT (0.590) leading. Oracle-RA (Table 39) clarifies the mechanism: these row models obtain their Cell F1 primarily from near-complete union-side recovery when retrieval and alignment are supplied, separating a union-preservation behavior from the identity-resolution behavior surfaced by Robust Linkage and Oracle-RA UJ-H. Pipeline-level signature. The highest-Cell F1 pipelines share a consistent composition: all top-20 use TA BERT at Stage 2, and Stage 3 concentrates on TAB T RANSFORMER, S UB TAB, SAINT, and TABBIE (Table 45). The best Cell F1 pipeline is S TARMIE/TA BERT/TAB T RANSFORMER at 0.679. The signature is therefore strong table retrieval, TA BERT column-side decisions, and row models that preserve union-side cells. This is an auxiliary lens for workloads that prioritize total recovered cell yield, or for diagnosing which stage limits pooled cell recovery. Overall, Cell F1 adds a practical diagnostic layer: it confirms the Stage 1 retrieval signal, identifies pooled-cell yield as most sensitive to Stage 2, and exposes a Stage 3 union-preservation mode. The full marginal, pipeline, Oracle-RA, and source-split tables report both metrics so the joint-recovery and pooled-cell views can be read directly (Tables 38, 41–43, 45, 44, 39; Appendix N.8).

59

N.2

Pipeline Component Sensitivity

Table 37: Ablation: DLTE pipeline component sensitivity. End-to-end cell F1 (↑) on the test set as a function of Stage 1 table retrieval model (rows) and Stage 2 column alignment model (columns). Stage 3 row matching is held fixed to the best-performing row model ( TabTransf. , selected as the Stage 3 model with the highest mean cell F1 across all (Stage 1, Stage 2) pairs). Each cell shows the full pipeline performance for that (retrieval, alignment) pair. Bold = best retrieval model per column alignment. underline = second. Stage 2: Column Alignment Model Stage 1: Retrieval

BERT

GTE

TaBERT

TAPAS

TURL

Starmie

TabSketchFM

TABBIE

Avg.

BERT GTE TaBERT TAPAS TURL Starmie TabSketchFM TABBIE TAPEX TUTA

0.607 0.608 0.608 0.608 0.606 0.609 0.627 0.586 0.575 0.602

0.608 0.609 0.611 0.609 0.605 0.639 0.601 0.571 0.573 0.628

0.673 0.677 0.676 0.676 0.667 0.679 0.650 0.575 0.575 0.650

0.612 0.612 0.626 0.609 0.610 0.645 0.615 0.573 0.565 0.629

0.603 0.604 0.614 0.605 0.599 0.615 0.602 0.579 0.584 0.615

0.544 0.544 0.544 0.544 0.544 0.544 0.544 0.544 0.544 0.544

0.554 0.555 0.554 0.553 0.557 0.566 0.552 0.550 0.547 0.554

0.570 0.579 0.567 0.565 0.575 0.573 0.556 0.541 0.512 0.567

0.597 0.598 0.600 0.596 0.595 0.609 0.593 0.565 0.559 0.599

Avg.

0.604

0.605

0.650

0.610

0.602

0.544

0.554

0.560

0.591

Table 37 reports end-to-end Cell F1 as a function of Stage 1 (retrieval) and Stage 2 (column alignment) model choices, with Stage 3 fixed to the best Cell F1 row model in this ablation, TAB T RANSFORMER. Among Stage 1 models, Starmie leads (row average 0.609), followed by TaBERT (0.600) and the native table encoder TUTA (0.599). TAPEX trails at 0.559 and TABBIE at 0.565, indicating that weak retrieval creates a hard ceiling for downstream performance. The 10-model Stage 1 pool covers all table-capable encoders: the 8 column-capable models (whose column embeddings are pooled to a table embedding) plus the two native table encoders TAPEX and TUTA. Among Stage 2 models, TaBERT dominates column alignment (column average 0.650), far ahead of the next-best TAPAS (0.610). Starmie at Stage 2 collapses to a flat 0.544 regardless of retrieval model. The wide Stage 2 spread (0.106) confirms that column alignment choice has the largest downstream effect. These results isolate column alignment under a fixed Stage 3 model (TAB T RANSF.). They are conditional, not the unconditional Stage-2 marginal used in Sec. 4.4 (which leads with TABBIE on test and BERT on dev). The discrepancy is expected under non-additive composition: a column model’s apparent strength depends on the upstream retriever and downstream row matcher with which it is paired. N.3

Per-Stage Marginal Analysis

Table 38 decomposes end-to-end performance into per-stage marginal contributions. For a fixed table model, scores are averaged over all 112 compatible pipelines, for a fixed column model over all 140, and for a fixed row model over all 80. Stage 2 (column model) exhibits the widest UJ-H span (0.060) and the widest Cell F1 span (0.084), confirming it has the largest average downstream effect under the current pipeline. Stage 1 (table model) shows a notable disconnect between retrieval recall and end-to-end contribution: S TARMIE ranks 1st on marginal Cell F1 despite only 3rd-best recall@100, while GTE achieves the highest recall (0.801) but ranks 3rd on Cell F1 , behind TUTA (2nd at 0.593). TAPEX further illustrates the disconnect in the opposite direction: it has the lowest marginal Cell F1 (0.558) and nearlowest recall (0.247 R@100, above only TABBIE’s 0.108), consistent with poor retrieval limiting downstream quality. Stage 3 (row model) has the narrowest span (0.026 Cell F1 ) and an even smaller UJ-H span (0.013), confirming that upstream errors largely mask row-model differences in the full pipeline. Marginal rankings are main-effect summaries rather than globally optimal compositions. For a pipeline p = (t, c, r) with end-to-end score y(p), the per-stage marginals are mT (t) = Ec,r [y(t, c, r)], mC (c) = Et,r [y(t, c, r)], and mR (r) = Et,c [y(t, c, r)], and the additive 60

Table 38: Per-stage marginal contributions in TRL-DLTE (5-round average, test set). For a fixed table model, scores are averaged over all 112 compatible pipelines. For a fixed column model, over all 140. For a fixed row model, over all 80. Bold orange / Underlined blue / Light purple highlights indicate best/second-best/third-best per column within each stage panel. Stage 1 additionally reports target recall@100 (mean fraction of the two relevant targets recovered among top-100 candidates) for reference. Performance span = best − worst marginal score within the stage. Stage 1: Table Model

Stage 2: Column Model

Model

Cell U J-H Tgt. F1 ↑ ↑ R@100↑

Starmie TUTA GTE TaBERT

.601 .593 .591 .590

.144 .138 .129 .124

BERT TAPAS TURL TabSketchFM TABBIE TAPEX

.589 .587 .587 .584 .560 .558

.128 .118 .124 .116 .109 .127

Stage 3: Row Model

Model

Cell U J-H F1 ↑ ↑

Model

Cell U J-H F1 ↑ ↑

.740 .585 .801 .720

TaBERT GTE TAPAS TURL

.628 .601 .600 .597

.128 .141 .132 .143

TabTransf. SubTab SAINT TABBIE

.591 .591 .590 .589

.119 .121 .119 .122

.763 .615 .597 .413 .108 .247

BERT TABBIE TabSketchFM Starmie

.595 .554 .553 .544

.135 .143 .100 .084

BERT TransTab GTE TabICL

.581 .574 .574 .566

.128 .132 .131 .130

Span .043 .036 .693 Span .084 .060 Span .026 .013 Note. Stage 3 shows the top 4 and bottom 4 (by Cell F1 ) of 14 row models, separated by a rule; the full ranking is in Appendix N. Stage 2 has the widest average downstream U J-H span (0.060) and the widest Cell F1 span (0.084) under the current pipeline, indicating that column-model choice has the largest mean effect among the three stages. Stage 1’s target recall@100 ranking does not match its downstream Cell F1 ranking (e.g., GTE has the best retrieval score but S TARMIE yields the best downstream Cell F1 marginal), showing that retrieval quality alone does not determine end-to-end enrichment.

Stage 1 (Table Model)

Stage 3: Transfer-Based (overall avg = 0.1277)

Stage 3: Prior-Based (overall avg = 0.1279)

Stage 3: Target-Table Learners (overall avg = 0.1241)

Generic Text

0.138

0.137

0.157

0.093

0.138

0.135

0.157

0.094

0.130

0.131

0.152

0.093

Table-Text

0.137

0.125

0.142

0.090

0.140

0.125

0.143

0.090

0.128

0.128

0.143

0.090

Table-Struct.

0.149

0.129

0.136

0.091

0.148

0.129

0.136

0.091

0.133

0.129

0.135

0.091

Col.-Centric

0.157

0.134

0.149

0.095

0.156

0.133

0.149

0.096

0.140

0.132

0.145

0.095

Generic Text Table-Text Table-Struct. Col.-Centric 0.09

0.10

0.11

Generic Text Table-Text Table-Struct. Col.-Centric

Generic Text Table-Text Table-Struct. Col.-Centric

Stage 2 (Col Model)

0.12

UJ-H

0.13

0.14

0.15

0.16

Figure 11: DLTE category-level UJ-H heatmap (5-round average, test set). Panels correspond to Stage 3 (row model) families. Rows = Stage 1 (table model), columns = Stage 2 (column model). The column-driven gradient confirms Stage 2’s dominant effect. Near-identical panels show Stage 3 differences are largely masked end to end.

main-effect score mT (t) + mC (c) + mR (r) − 2ȳ is maximized by the per-stage rank-1 assembly. On test, this assembly is S TARMIE/TABBIE/T RANS TAB and scores 0.134 UJ-H, while the test rank-1 pipeline S TARMIE/GTE/GTE scores 0.253 and the dev-selected headline TUTA/GTE/GTE scores 0.229. On development, the marginal-leader assembly changes to S TARMIE/BERT/T RANS TAB and is competitive on test (0.231), confirming that marginal main effects carry signal. The change in the leader assembly across splits, TABBIE’s absence from the top-50, and the gap between the marginal-leader assembly and the end-to-end optima together show that the top of the DLTE space is shaped by residual non-additive stage interactions. We refer to this residual structure as compositional fit. 61

N.4

Oracle-RA Row-Model Diagnostic

Table 39 reports Oracle-RA results on the test split. Oracle-RA bypasses Stage 1 (retrieval) and Stage 2 (alignment) with ground-truth data, isolating Stage 3 row matching quality. The UJ-H spread across row models is 0.546, compared with a marginal end-to-end span of just 0.013 (Table 38), confirming that upstream errors mask most row-model differences in the full pipeline. Table 39: Oracle-RA row-model diagnostic (test set). Stages 1–2 use ground truth. Only Stage 3 row matching varies. #

Row Model

UJ-H↑ Cell F1 ↑ Union↑ Join↑ Hard↑

1 2 3 4 5 6 7 8 9 10 11 12 13 14

GTE TransTab TabICL TUTA BERT SCARF DAE VIME TabPFN TabBinning TABBIE SAINT SubTab TabTransformer

0.683 0.658 0.606 0.487 0.454 0.340 0.333 0.318 0.293 0.259 0.231 0.168 0.164 0.137

0.802 0.802 0.789 0.743 0.735 0.709 0.713 0.710 0.706 0.697 0.695 0.684 0.685 0.680

0.696 0.698 0.609 0.829 0.817 0.864 0.893 0.900 0.925 0.902 0.935 0.940 0.957 0.961

0.763 0.721 0.664 0.403 0.386 0.246 0.234 0.223 0.196 0.181 0.154 0.116 0.109 0.092

0.550 0.607 0.737 0.418 0.376 0.318 0.305 0.289 0.256 0.232 0.197 0.144 0.133 0.111

Key observations. Row models divide into two groups: identity-resolving models (GTE, TransTab, TabICL) that achieve balanced union/join recovery, and union-dedup specialists (TabTransformer, SAINT, SubTab) with near-perfect union recall but near-zero join recall. This distinction is reflected in the Cell F1 / UJ-H pipeline-level pattern: Cell F1 captures pooled cell-recovery yield (well-served by high union recall), while UJ-H captures balanced recovery of both removed blocks and therefore requires both recalls to be high (Appendix N.1). TabICL attains the best hard-region recall (0.737), showing that the second-pass join mechanism can recover cells in the new-rows×new-columns quadrant. Per-noise-tier breakdown. Table 40 decomposes Oracle-RA UJ-H across the four TRL-DLTE noise tiers (cumulative clean → schema → cell → hard; see Sec. 3.4). The key observation is that the cross-row-model UJ-H span is large and largely tier-invariant (0.562/0.563/0.506/0.553 for clean/schema/cell/hard), and the top/bottom row-model families are stable at every corruption level: the identity-resolving row models (GTE, T RANS TAB, TAB ICL) occupy the top three positions in every tier, and the union-dedup specialists (TAB T RANSFORMER, SAINT, S UB TAB) occupy the bottom three in every tier. The Stage 3 separability exposed by Oracle-RA is therefore not a noise-sensitivity artifact: it persists across all levels of upstream corruption. The fact that uniondedup specialists rank strongly on end-to-end Cell F1 (Table 43) but rank last under Oracle-RA on both UJ-H and join recall indicates that their end-to-end Cell F1 advantage reflects union-side recovery behavior under upstream error, not strong identity-resolution Stage 3 behavior on balanced enrichment. Cross-validation against RBench robust linkage. The identity-resolving/union-dedup split identified by Oracle-RA is not an artifact of the DLTE pipeline: the same taxonomy is visible in atomic cross-table record linkage. Over all 14 row models, the Oracle-RA UJ-H ranking and the RBench Robust Linkage NR ranking (aggregated over DM-D and WDC; Table 3) are strongly correlated: |ρSpearman | = 0.80 (p = 6.3×10−4 ) and |τKendall | = 0.63 (p = 1.2×10−3 ). The top two row models agree across both views (GTE and T RANS TAB: Robust Linkage NR = 0.048 and 0.096, and Oracle-RA UJ-H also leads with these two), with TAB ICL third on Oracle-RA but fifth on Robust Linkage NR (= 0.394), trailing GTE and T RANS TAB as well as TUTA (= 0.154) and BERT (= 0.163). The union-dedup specialists highlighted by Oracle-RA anchor the other end: S UB TAB and TAB T RANSFORMER occupy the bottom two positions of the Robust Linkage NR column (NR = 0.962, 0.942), with their near-zero join recall in Oracle-RA mirrored by near-zero 62

Table 40: Oracle-RA per-noise-tier U J-H for each row model (test split, 5-round mean). Tiers are the cumulative noise levels used in TRL-DLTE construction (clean → schema → cell → hard). The final row reports the cross-row-model span within each tier, summarizing how Stage 3 separability varies with upstream noise. Row models are sorted by mean U J-H across tiers. Row model

Clean Schema

Cell

Hard

GTE T RANS TAB TAB ICL TUTA BERT SCARF DAE VIME TAB PFN TAB B INNING TABBIE SAINT S UB TAB TAB T RANSFORMER

0.703 0.652 0.613 0.502 0.480 0.351 0.323 0.307 0.306 0.256 0.236 0.171 0.170 0.141

0.704 0.675 0.612 0.499 0.480 0.338 0.323 0.318 0.300 0.256 0.229 0.173 0.172 0.141

0.633 0.612 0.546 0.449 0.431 0.308 0.296 0.275 0.267 0.240 0.207 0.150 0.151 0.127

0.692 0.691 0.653 0.499 0.425 0.363 0.392 0.373 0.297 0.283 0.255 0.177 0.164 0.139

Span (max−min)

0.562

0.563

0.506 0.553

WDC F1 (e.g., TAB T RANSFORMER WDC F1 = 0.020). SAINT sits at NR = 0.606, mid-pack on Robust Linkage despite its union-heavy profile. Rank orderings within each family differ for the same reason: DM-C (clean linkage) does not fully separate identity-resolution behavior, which is why BERT leads there but drops to 5th in Oracle-RA. Even so, the two tests agree at both ends of the ranking. This is consistent with a shared identity-resolution capability of frozen row embeddings, surfaced consistently by both entity matching (RBench) and compositional enrichment (DLTE). N.5

Full Per-Stage Model Rankings

Tables 41–43 report the complete marginal rankings for each DLTE stage (test split, 5-round mean ± std) over the full 10 × 8 × 14 = 1120 table × column × row search space. The 10-model Stage 1 pool covers the 8 column-capable encoders (whose column embeddings are pooled to a table embedding) plus the two native table encoders TAPEX and TUTA. Stage 2 remains the 8 column-capable encoders since TAPEX/TUTA do not expose compatible column embeddings. For a fixed table model, scores are averaged over all 8 × 14 = 112 compatible pipelines, for a fixed column model over all 10 × 14 = 140 pipelines, and for a fixed row model over all 10 × 8 = 80 pipelines. Table 41: Stage 1 (table model) marginal rankings, sorted by Cell F1 (mean ± std over 5 rounds). R@100 is Stage 1 retrieval recall against any gold candidate. Rank Model (Family)

Cell F1 ↑

UJ-H ↑

R@100 ↑

1 2 3 4 5 6 7 8 9 10

0.601±0.001 0.593±0.001 0.591±0.002 0.590±0.001 0.589±0.002 0.587±0.001 0.587±0.000 0.584±0.001 0.560±0.000 0.558±0.001

0.144±0.003 0.138±0.001 0.129±0.001 0.124±0.001 0.128±0.002 0.118±0.001 0.124±0.001 0.116±0.002 0.109±0.001 0.127±0.001

0.740±0.000 0.585±0.000 0.801±0.000 0.720±0.000 0.763±0.000 0.615±0.000 0.597±0.007 0.413±0.001 0.108±0.000 0.247±0.000

Starmie (Col.-Centric) TUTA (Table-Struct.) GTE (Generic Text) TaBERT (Table-Text) BERT (Generic Text) TAPAS (Table-Text) TURL (Table-Struct.) TabSketchFM (Col.-Centric) TABBIE (Table-Struct.) TAPEX (Table-Text)

Retrieval recall does not dictate downstream contribution. Table 41 exposes a non-monotone relationship between raw Stage 1 retrieval quality and downstream end-to-end contribution. GTE attains the highest recall-at-100 (0.801) among the ten table models but does not lead downstream. S TARMIE has lower recall-at-100 (0.740) yet attains the best downstream Cell F1 (0.601) and UJ-H (0.144). The ordering is neither retrieval-dominated nor inverse: TABBIE is worst on recall-at-100 (0.108) while TAPEX is lowest on Cell F1 (0.558), and TA BERT and BERT sit in the middle of 63

Table 42: Stage 2 (column model) marginal rankings, sorted by Cell F1 (mean ± std over 5 rounds). Rank Model (Family) 1 2 3 4 5 6 7 8

TaBERT (Table-Text) GTE (Generic Text) TAPAS (Table-Text) TURL (Table-Struct.) BERT (Generic Text) TABBIE (Table-Struct.) TabSketchFM (Col.-Centric) Starmie (Col.-Centric)

Cell F1 ↑

UJ-H ↑

0.628±0.001 0.601±0.001 0.600±0.001 0.597±0.004 0.595±0.000 0.554±0.001 0.553±0.001 0.544±0.000

0.128±0.000 0.141±0.002 0.132±0.001 0.143±0.004 0.135±0.001 0.143±0.000 0.100±0.001 0.084±0.000

Table 43: Stage 3 (row model) marginal rankings, sorted by Cell F1 (mean ± std over 5 rounds). Rank Model (Family)

Cell F1 ↑

UJ-H ↑

1 2 3 4 5 6 7 8 9 10 11 12 13 14

0.591±0.001 0.591±0.001 0.590±0.002 0.589±0.000 0.589±0.000 0.587±0.000 0.587±0.003 0.587±0.001 0.585±0.001 0.584±0.000 0.581±0.000 0.574±0.001 0.574±0.000 0.566±0.000

0.119±0.001 0.121±0.001 0.119±0.001 0.122±0.000 0.126±0.000 0.127±0.001 0.123±0.003 0.125±0.001 0.127±0.001 0.130±0.001 0.128±0.001 0.132±0.001 0.131±0.001 0.130±0.001

TabTransformer (Target-Table) SubTab (Target-Table) SAINT (Target-Table) TABBIE (Table-Struct.) TabPFN (Prior-Based) DAE (Target-Table) TabBinning (Target-Table) VIME (Target-Table) SCARF (Target-Table) TUTA (Table-Struct.) BERT (Generic Text) TransTab (Target-Table) GTE (Generic Text) TabICL (Prior-Based)

both orderings. This non-monotone pattern shows that Stage 1’s downstream contribution is not determined by recall volume alone. It also depends on the structure of the retrieved candidate set and on whether those candidates are usable by the downstream column-alignment and row-matching models. This is a Stage-1 instance of compositional fit beyond per-stage marginal rank.

64

N.6

Stage 3 Row Matching: Full Pipeline Rankings

Table 44: Ablation: DLTE Stage 3 (row matching). Rows are (Stage 1, Stage 2) configurations. Columns are Stage 3 row models. All ten Stage 1 models are included (8 column-capable + TAPEX and TUTA native table encoders). Cell F1 (↑) on test set. Column groupings by adaptation regime: Transfer = externally pretrained, used frozen. Prior = meta-pretrained prior-fitted. Learner = target-table feature-corruption SSL. Transfer

Prior BIE

Learner ab nsT

ng

sf.

T IN

ab bT Su

n Tra Tab

g. Av

0.663 0.635 0.636 0.612 0.606 0.564 0.568 0.544

0.674 0.638 0.643 0.612 0.609 0.565 0.572 0.544

0.677 0.639 0.645 0.615 0.609 0.566 0.573 0.544

0.679 0.639 0.645 0.615 0.609 0.566 0.573 0.544

0.650 0.635 0.630 0.613 0.608 0.564 0.564 0.544

0.661 0.606 0.604 0.604 0.600 0.560 0.553 0.544

0.662 0.604 0.604 0.603 0.599 0.560 0.553 0.544

0.672 0.608 0.608 0.608 0.603 0.563 0.553 0.544

0.675 0.609 0.608 0.609 0.605 0.564 0.553 0.544

0.676 0.609 0.608 0.609 0.605 0.565 0.553 0.544

0.650 0.601 0.599 0.598 0.595 0.557 0.552 0.544

0.653 0.603 0.603 0.604 0.600 0.572 0.555 0.544

0.660 0.604 0.607 0.606 0.602 0.573 0.555 0.544

0.662 0.603 0.606 0.605 0.602 0.575 0.554 0.544

0.672 0.603 0.610 0.608 0.607 0.577 0.554 0.544

0.676 0.604 0.611 0.610 0.607 0.579 0.555 0.544

0.677 0.604 0.612 0.609 0.608 0.579 0.555 0.544

0.649 0.604 0.603 0.602 0.600 0.570 0.554 0.544

0.661 0.619 0.608 0.607 0.600 0.562 0.552 0.544

0.653 0.613 0.605 0.605 0.598 0.560 0.553 0.544

0.660 0.619 0.609 0.609 0.600 0.562 0.553 0.544

0.662 0.618 0.608 0.607 0.600 0.563 0.553 0.544

0.671 0.624 0.611 0.610 0.606 0.566 0.553 0.544

0.674 0.626 0.613 0.611 0.606 0.567 0.554 0.544

0.676 0.626 0.614 0.611 0.608 0.567 0.554 0.544

0.649 0.612 0.604 0.604 0.595 0.559 0.552 0.544

0.608 0.594 0.605 0.588 0.588 0.551 0.554 0.544

0.658 0.608 0.602 0.604 0.602 0.565 0.554 0.544

0.650 0.604 0.601 0.602 0.600 0.563 0.555 0.544

0.657 0.609 0.602 0.605 0.602 0.565 0.555 0.544

0.659 0.607 0.601 0.604 0.602 0.565 0.554 0.544

0.669 0.611 0.602 0.606 0.607 0.569 0.555 0.544

0.672 0.612 0.604 0.608 0.607 0.570 0.555 0.544

0.673 0.612 0.603 0.608 0.607 0.570 0.554 0.544

0.646 0.604 0.602 0.600 0.599 0.562 0.555 0.544

0.660 0.605 0.606 0.601 0.597 0.572 0.557 0.544

0.610 0.603 0.579 0.576 0.578 0.555 0.552 0.544

0.654 0.604 0.603 0.599 0.595 0.570 0.556 0.544

0.646 0.603 0.599 0.596 0.593 0.568 0.556 0.544

0.654 0.603 0.604 0.599 0.596 0.570 0.556 0.544

0.654 0.603 0.603 0.599 0.595 0.570 0.556 0.544

0.663 0.604 0.609 0.605 0.597 0.573 0.557 0.544

0.666 0.605 0.610 0.606 0.599 0.575 0.558 0.544

0.667 0.605 0.610 0.606 0.599 0.575 0.557 0.544

0.643 0.604 0.597 0.594 0.591 0.566 0.556 0.544

0.583 0.575 0.571 0.570 0.568 0.548 0.537 0.544

0.644 0.623 0.611 0.599 0.597 0.552 0.554 0.544

0.600 0.590 0.582 0.580 0.578 0.549 0.542 0.544

0.638 0.619 0.606 0.597 0.595 0.551 0.552 0.544

0.634 0.615 0.602 0.594 0.593 0.551 0.550 0.544

0.639 0.619 0.608 0.598 0.596 0.551 0.552 0.544

0.640 0.620 0.607 0.597 0.596 0.552 0.553 0.544

0.647 0.624 0.613 0.600 0.598 0.552 0.555 0.544

0.649 0.626 0.614 0.602 0.601 0.552 0.556 0.544

0.650 0.627 0.615 0.602 0.601 0.552 0.556 0.544

0.629 0.612 0.601 0.593 0.591 0.551 0.549 0.544

0.624 0.628 0.619 0.618 0.610 0.559 0.554 0.544

0.584 0.603 0.591 0.598 0.589 0.543 0.550 0.544

0.643 0.627 0.626 0.616 0.602 0.565 0.554 0.544

0.600 0.621 0.605 0.609 0.600 0.550 0.551 0.544

0.639 0.622 0.622 0.613 0.599 0.562 0.553 0.544

0.633 0.624 0.618 0.613 0.600 0.561 0.554 0.544

0.638 0.625 0.621 0.615 0.601 0.563 0.554 0.544

0.639 0.624 0.621 0.613 0.600 0.564 0.554 0.544

0.646 0.627 0.626 0.613 0.602 0.566 0.554 0.544

0.649 0.628 0.629 0.616 0.602 0.568 0.554 0.544

0.650 0.628 0.629 0.615 0.602 0.567 0.554 0.544

0.629 0.624 0.618 0.613 0.601 0.560 0.553 0.544

0.575 0.583 0.573 0.573 0.565 0.547 0.544 0.511

0.589 0.582 0.573 0.570 0.565 0.547 0.544 0.512

0.587 0.568 0.555 0.556 0.557 0.546 0.544 0.516

0.577 0.584 0.573 0.573 0.565 0.547 0.544 0.512

0.595 0.577 0.565 0.561 0.562 0.545 0.544 0.514

0.574 0.583 0.571 0.571 0.563 0.546 0.544 0.511

0.578 0.582 0.570 0.571 0.563 0.547 0.544 0.511

0.578 0.583 0.571 0.572 0.564 0.546 0.544 0.511

0.574 0.583 0.571 0.572 0.563 0.546 0.544 0.512

0.574 0.583 0.574 0.574 0.564 0.546 0.544 0.512

0.574 0.584 0.574 0.574 0.565 0.547 0.544 0.512

0.573 0.584 0.575 0.575 0.565 0.547 0.544 0.512

0.581 0.581 0.570 0.569 0.563 0.546 0.544 0.512

0.559 0.562 0.557 0.561 0.560 0.546 0.544 0.532

0.584 0.577 0.574 0.573 0.570 0.550 0.544 0.540

0.574 0.571 0.567 0.566 0.567 0.551 0.544 0.537

0.549 0.557 0.552 0.556 0.558 0.543 0.544 0.533

0.580 0.577 0.573 0.572 0.569 0.549 0.544 0.540

0.559 0.563 0.559 0.562 0.562 0.543 0.544 0.534

0.580 0.576 0.571 0.570 0.568 0.548 0.544 0.539

0.576 0.574 0.570 0.569 0.567 0.548 0.544 0.538

0.579 0.576 0.572 0.571 0.568 0.549 0.544 0.538

0.579 0.576 0.572 0.571 0.568 0.549 0.544 0.539

0.584 0.577 0.574 0.573 0.570 0.550 0.544 0.540

0.585 0.579 0.575 0.573 0.571 0.550 0.544 0.541

0.586 0.579 0.575 0.573 0.571 0.550 0.544 0.541

0.575 0.572 0.568 0.568 0.567 0.548 0.544 0.537

0.573

0.588

0.583

0.566

0.588

0.575

0.587

0.585

0.587

0.587

0.590

0.591

0.591

0.584

Stage 2

BE

RT

E GT

B TA

TU

Ta

P Tab

Tra

VI

ME

Starmie

TaBERT GTE TAPAS TURL BERT TabSketchFM TABBIE Starmie

0.638 0.637 0.624 0.616 0.609 0.565 0.559 0.544

0.607 0.632 0.608 0.612 0.606 0.562 0.550 0.544

0.671 0.639 0.641 0.614 0.610 0.567 0.570 0.544

0.643 0.639 0.631 0.616 0.613 0.566 0.563 0.544

0.589 0.614 0.592 0.601 0.595 0.561 0.543 0.544

0.670 0.638 0.641 0.616 0.610 0.567 0.569 0.544

0.609 0.634 0.608 0.613 0.610 0.561 0.550 0.544

0.662 0.636 0.637 0.613 0.606 0.564 0.567 0.544

0.654 0.633 0.631 0.612 0.607 0.565 0.565 0.544

0.661 0.636 0.636 0.615 0.608 0.564 0.567 0.544

TAPAS

TaBERT GTE BERT TAPAS TURL TABBIE TabSketchFM Starmie

0.637 0.597 0.595 0.593 0.592 0.552 0.550 0.544

0.610 0.589 0.584 0.582 0.582 0.546 0.551 0.544

0.669 0.609 0.607 0.606 0.602 0.563 0.554 0.544

0.644 0.600 0.598 0.596 0.595 0.557 0.553 0.544

0.593 0.576 0.573 0.572 0.570 0.542 0.549 0.544

0.668 0.607 0.606 0.607 0.602 0.562 0.553 0.544

0.613 0.587 0.585 0.583 0.583 0.548 0.550 0.544

0.662 0.605 0.605 0.603 0.599 0.560 0.552 0.544

0.654 0.602 0.601 0.599 0.596 0.558 0.553 0.544

GTE

TaBERT TURL TAPAS GTE BERT TABBIE TabSketchFM Starmie

0.636 0.604 0.599 0.601 0.600 0.565 0.553 0.544

0.607 0.607 0.594 0.592 0.589 0.557 0.552 0.544

0.669 0.603 0.608 0.608 0.606 0.577 0.556 0.544

0.643 0.607 0.603 0.601 0.602 0.569 0.555 0.544

0.589 0.593 0.580 0.581 0.575 0.552 0.552 0.544

0.668 0.605 0.611 0.607 0.606 0.576 0.555 0.544

0.611 0.608 0.594 0.591 0.591 0.558 0.552 0.544

0.662 0.604 0.607 0.605 0.601 0.574 0.554 0.544

TaBERT

TaBERT TAPAS TURL GTE BERT TABBIE TabSketchFM Starmie

0.636 0.605 0.600 0.600 0.593 0.554 0.550 0.544

0.610 0.593 0.592 0.594 0.577 0.546 0.550 0.544

0.669 0.622 0.611 0.611 0.604 0.565 0.554 0.544

0.644 0.609 0.605 0.602 0.595 0.557 0.553 0.544

0.592 0.580 0.579 0.582 0.565 0.541 0.549 0.544

0.667 0.623 0.611 0.608 0.604 0.564 0.553 0.544

0.612 0.594 0.593 0.592 0.579 0.547 0.550 0.544

BERT

TaBERT TAPAS TURL GTE BERT TABBIE TabSketchFM Starmie

0.634 0.599 0.603 0.599 0.599 0.557 0.554 0.544

0.606 0.594 0.603 0.589 0.586 0.548 0.555 0.544

0.666 0.610 0.602 0.607 0.605 0.568 0.556 0.544

0.641 0.606 0.606 0.599 0.599 0.561 0.556 0.544

0.587 0.581 0.591 0.578 0.574 0.544 0.553 0.544

0.664 0.612 0.603 0.605 0.605 0.567 0.555 0.544

TURL

TaBERT GTE TAPAS BERT TURL TABBIE TabSketchFM Starmie

0.632 0.607 0.591 0.592 0.588 0.561 0.555 0.544

0.608 0.607 0.578 0.575 0.578 0.553 0.553 0.544

0.661 0.605 0.606 0.602 0.597 0.573 0.558 0.544

0.638 0.607 0.594 0.592 0.589 0.565 0.556 0.544

0.590 0.591 0.566 0.564 0.569 0.548 0.550 0.544

TabSketchFM

TaBERT BERT TAPAS TURL GTE TabSketchFM TABBIE Starmie

0.618 0.606 0.595 0.589 0.587 0.550 0.543 0.544

0.598 0.590 0.582 0.580 0.578 0.549 0.540 0.544

0.645 0.623 0.611 0.599 0.598 0.553 0.554 0.544

0.626 0.610 0.598 0.591 0.591 0.552 0.549 0.544

TUTA

TaBERT GTE TAPAS TURL BERT TABBIE TabSketchFM Starmie

0.619 0.625 0.616 0.614 0.604 0.555 0.553 0.544

0.598 0.621 0.604 0.611 0.603 0.548 0.552 0.544

0.645 0.627 0.626 0.614 0.600 0.566 0.555 0.544

TAPEX

GTE TURL BERT TaBERT TAPAS TabSketchFM Starmie TABBIE

0.589 0.581 0.571 0.567 0.565 0.546 0.544 0.508

0.599 0.574 0.563 0.558 0.561 0.546 0.544 0.511

TABBIE

BERT TURL TaBERT TAPAS GTE TabSketchFM Starmie TABBIE

0.574 0.571 0.564 0.565 0.566 0.551 0.544 0.533 0.580

Global Avg.

i inn

SA

Stage 1

F AR SC

TA

L bIC

FN

E DA

B Tab

Table 44 shows Cell F1 for all (Stage 1, Stage 2) configurations across all ten Stage 1 models, crossed with all 14 Stage 3 row models. The best pipelines use TaBERT at Stage 2 (top rows of each group), confirming Stage 2 dominance regardless of Stage 1 choice. Starmie leads as Stage 1 (best avg 0.650), 65

while TAPEX has the lowest average Cell F1 as Stage 1 (0.558), narrowly below TABBIE (0.560), with both lagging the rest of the pool by a clear margin. Within any given (Stage 1, Stage 2) pair, the Stage 3 spread is narrow: for the best configuration (Starmie → TaBERT), Cell F1 ranges from 0.589–0.679, a span of only ∼0.090. The global average Cell F1 by Stage 3 column is nearly flat (0.566–0.591), with SAINT, SubTab, and TabTransf. consistently at the top and TabICL at the bottom, confirming that Stage 3 row model choice has only marginal end-to-end impact and that, within this Cell F1 ablation, the dominant lever is the Stage 2 column alignment model.

66

N.7

Top-20 Pipeline Combinations

Tables 45 and 46 list the top-20 pipeline combinations by Cell F1 and UJ-H, respectively. The two lists rank pipelines along different axes, consistent with their definitions (Appendix N.1): Cell F1 captures pooled cell-recovery yield, while UJ-H captures balanced recovery of both removed blocks. Table 45: Top-20 DLTE combinations by Cell F1 (5-round average, test set). All entries use TaBERT as Stage 2 column model. Their UJ-H ranks (#513–#671) reflect the different axis Cell F1 captures relative to UJ-H (Appendix N.1). #

Stage 1 (Tbl) Fam.

Stage 2 (Col) Fam.

Stage 3 (Row) Fam.

Cell F1 UJ-H UJ-H #

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Starmie Starmie GTE TAPAS GTE TaBERT TAPAS Starmie TaBERT BERT GTE BERT TAPAS Starmie TaBERT Starmie GTE TAPAS TaBERT BERT

TaBERT TaBERT TaBERT TaBERT TaBERT TaBERT TaBERT TaBERT TaBERT TaBERT TaBERT TaBERT TaBERT TaBERT TaBERT TaBERT TaBERT TaBERT TaBERT TaBERT

TabTransf. SubTab TabTransf. TabTransf. SubTab TabTransf. SubTab SAINT SubTab TabTransf. SAINT SubTab SAINT TABBIE SAINT TabPFN TABBIE TABBIE TABBIE SAINT

0.679 0.677 0.677 0.676 0.676 0.676 0.675 0.674 0.674 0.673 0.672 0.672 0.672 0.671 0.671 0.670 0.669 0.669 0.669 0.669

Col-Cen Col-Cen Gen. Txt Tbl-Txt Gen. Txt Tbl-Txt Tbl-Txt Col-Cen Tbl-Txt Gen. Txt Gen. Txt Gen. Txt Tbl-Txt Col-Cen Tbl-Txt Col-Cen Gen. Txt Tbl-Txt Tbl-Txt Gen. Txt

Tbl-Txt Tbl-Txt Tbl-Txt Tbl-Txt Tbl-Txt Tbl-Txt Tbl-Txt Tbl-Txt Tbl-Txt Tbl-Txt Tbl-Txt Tbl-Txt Tbl-Txt Tbl-Txt Tbl-Txt Tbl-Txt Tbl-Txt Tbl-Txt Tbl-Txt Tbl-Txt

Tgt-Tbl Tgt-Tbl Tgt-Tbl Tgt-Tbl Tgt-Tbl Tgt-Tbl Tgt-Tbl Tgt-Tbl Tgt-Tbl Tgt-Tbl Tgt-Tbl Tgt-Tbl Tgt-Tbl Transfer Tgt-Tbl Prior Transfer Transfer Transfer Tgt-Tbl

0.128 0.128 0.125 0.127 0.125 0.125 0.127 0.126 0.125 0.126 0.123 0.126 0.126 0.127 0.123 0.129 0.124 0.125 0.123 0.124

#532 #523 #589 #544 #597 #608 #549 #573 #616 #575 #656 #577 #581 #541 #671 #513 #651 #596 #659 #620

Table 46: Top-20 DLTE combinations by UJ-H (5-round average, test set). Stage 2 is dominated by GTE, BERT, and TURL. Stage 1 is led by Starmie (11 entries), with TUTA appearing 7 times (from the native table-encoder pool), TURL once (#17), and TAPEX once (#19). Zero overlap with the Cell F1 top-20 list (Table 45). #

Stage 1 (Tbl) Fam.

Stage 2 (Col) Fam.

Stage 3 (Row) Fam.

Cell F1 UJ-H CF1 #

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Starmie Starmie Starmie Starmie TUTA Starmie Starmie Starmie Starmie Starmie TUTA TUTA Starmie Starmie TUTA TUTA TURL TUTA TAPEX TUTA

GTE GTE GTE BERT GTE BERT BERT TURL TURL TURL GTE BERT GTE GTE BERT GTE GTE GTE GTE BERT

GTE TransTab TabICL TransTab GTE GTE TabICL GTE TransTab TabICL TransTab GTE TUTA BERT TransTab TabICL GTE TUTA GTE TabICL

0.632 0.634 0.614 0.610 0.621 0.606 0.595 0.612 0.613 0.601 0.621 0.603 0.639 0.637 0.600 0.603 0.607 0.628 0.599 0.589

Col-Cen Col-Cen Col-Cen Col-Cen Tbl-Struct Col-Cen Col-Cen Col-Cen Col-Cen Col-Cen Tbl-Struct Tbl-Struct Col-Cen Col-Cen Tbl-Struct Tbl-Struct Tbl-Struct Tbl-Struct Tbl-Txt Tbl-Struct

Gen. Txt Gen. Txt Gen. Txt Gen. Txt Gen. Txt Gen. Txt Gen. Txt Tbl-Struct Tbl-Struct Tbl-Struct Gen. Txt Gen. Txt Gen. Txt Gen. Txt Gen. Txt Gen. Txt Gen. Txt Gen. Txt Gen. Txt Gen. Txt

67

Transfer Tgt-Tbl Prior Tgt-Tbl Transfer Transfer Prior Transfer Tgt-Tbl Prior Tgt-Tbl Transfer Transfer Transfer Tgt-Tbl Prior Transfer Transfer Transfer Prior

0.253 0.251 0.236 0.231 0.229 0.228 0.228 0.225 0.225 0.220 0.218 0.216 0.215 0.211 0.210 0.210 0.205 0.204 0.204 0.204

#114 #108 #184 #226 #149 #313 #504 #201 #186 #429 #153 #380 #85 #97 #437 #386 #284 #120 #461 #545

N.8

Source Split: TabFact vs. WTQ

To test whether the TRL-DLTE findings are tied to a particular parent source, we partition the 345 test parents into TabFact (246) and WTQ (99) and recompute every pipeline-level metric separately on each partition. All 1,120 canonical pipelines (10 Stage-1 × 8 Stage-2 × 14 Stage-3 models) are evaluated under the same 5 rounds, so the source-split numbers are directly comparable to the main-text aggregates. Table 47: Source-split UJ-H for headline DLTE pipelines (5-round mean over the test split). The hybrid-vs-monolith gap holds separately on TabFact (246 test parents) and WTQ (99 test parents), and is wider on WTQ. Dev-selected pipelines are the rank-1 hybrid and rank-1 monolithic by dev UJ-H. The test-set unconstrained block is the rank-1 across all 1,120 pipelines on test, included as a sensitivity reference. Pipeline notation: Stage 1 (Tbl) / Stage 2 (Col) / Stage 3 (Row). Pipeline

TabFact

WTQ

All

Dev-selected (top-1 by dev UJ-H, evaluated on test): Best hybrid: TUTA/GTE/GTE 0.225 0.238 Best monolith: BERT/BERT/BERT 0.139 0.140 Hybrid − monolith +0.086 +0.099

0.229 0.139 +0.090

Test-set unconstrained reference (max over 1,120 pipelines): Best hybrid: S TARMIE/GTE/GTE 0.242 0.281 Best monolith: GTE/GTE/GTE 0.140 0.157 Hybrid − monolith +0.102 +0.125

0.253 0.145 +0.108

Table 48: Per-stage marginal UJ-H top-5 under each source, averaged over all 1,120 canonical pipelines. Stage 1 is S TARMIE-led in both sources. Stage 2 has a TABBIE ↔ TURL swap at rank 1 but the same top models recur. Stage 3 is T RANS TAB-led in both. Stage

Source

Rank 1

Rank 2

Rank 3

Rank 4

Rank 5

Stage 1 (Tbl)

TabFact WTQ All

S TARMIE (0.140) S TARMIE (0.155) S TARMIE (0.144)

TUTA (0.137) TUTA (0.140) TUTA (0.138)

GTE (0.125) TAPEX (0.138) GTE (0.129)

BERT (0.124) GTE (0.138) BERT (0.128)

TAPEX (0.122) TURL (0.137) TAPEX (0.127)

Stage 2 (Col)

TabFact WTQ All

TABBIE (0.140) TURL (0.159) TABBIE (0.143)

GTE (0.137) TABBIE (0.153) TURL (0.143)

TURL (0.137) GTE (0.150) GTE (0.141)

BERT (0.132) TAPAS (0.132) TA BERT (0.149) BERT (0.140) BERT (0.135) TAPAS (0.132)

TabFact T RANS TAB (0.128) Stage 3 (Row) WTQ T RANS TAB (0.142) All T RANS TAB (0.132)

GTE (0.127) TUTA (0.140) GTE (0.131)

TAB ICL (0.127) GTE (0.140) TAB ICL (0.130)

TUTA (0.126) TAB ICL (0.139) TUTA (0.130)

BERT (0.123) BERT (0.138) BERT (0.128)

Headline pipelines hold in both sources. Table 47 shows that the hybrid-vs-monolith gap persists separately on TabFact and WTQ: the dev-selected best hybrid (TUTA/GTE/GTE) beats the devselected best monolith (BERT/BERT/BERT) by +0.086 UJ-H on TabFact and by +0.099 on WTQ. The gap is if anything wider on WTQ, so the hybrid advantage is not a TabFact-specific artifact. The unconstrained test maximum (S TARMIE/GTE/GTE, dev rank 8) reaches 0.242 / 0.281 on TabFact / WTQ with the same wider-on-WTQ pattern. The unconstrained test-set max monolith shifts to GTE/GTE/GTE (0.140 / 0.157, pooled 0.145), still well below every hybrid pipeline reported here. Per-stage marginal top models are largely source-invariant. Table 48 shows the top-5 models at each stage under each source. Stage 1 is S TARMIE-led in both sources, with TUTA second. Stage 2 has a TABBIE↔TURL swap at rank 1, and four of the top-5 column models (TABBIE, TURL, GTE, BERT) are common to both sources. Stage 3 is T RANS TAB-led in both, with GTE, TAB ICL, and TUTA occupying ranks 2–4 in both sources (in varying order). No stage changes leader family under the source split. Strong rank agreement across sources. Across all 1,120 canonical pipelines, Spearman ρ(TabFact, WTQ) = 0.871 (p ≪ 10−100 ), confirming that pipeline rankings are largely sourceagnostic. For Oracle-RA, which isolates Stage 3 identity resolution by replacing Stages 1–2 with ground truth (Table 39), the row-model ranking agrees at Spearman ρ = 0.987 (p = 7.4 × 10−11 , 68

n = 14): GTE, T RANS TAB, TAB ICL occupy the top three positions in both sources, and TAB T RANSFORMER, S UB TAB, SAINT the bottom three in both. Table 49 reports the full per-source Oracle-RA UJ-H for every row model. Combined with the per-noise-tier breakdown in Table 40, the Stage-3 row-model ranking is stable on both the noise-tier axis (cross-tier spans 0.506–0.563) and the parent-source axis (cross-source spans 0.533–0.551), so the identity-resolving/union-dedup split exposed at Stage 3 does not depend on either the particular noise regime or the particular parent source. Table 49: Oracle-RA per-source UJ-H for each row model (test split, 5-round mean). Sources are the two TRL-DLTE parent pools (TabFact: 246 test parents, WTQ: 99 test parents). The “All” column is the pooled mean over the 345 test parents × 4 noise tiers (1,380 query-tier evaluations). The final row reports the cross-row-model span within each source, comparable in magnitude to the per-tier spans (0.506–0.563) in Table 40. Row models are sorted by pooled UJ-H. Row model

TabFact WTQ

All

GTE T RANS TAB TAB ICL TUTA BERT SCARF DAE VIME TAB PFN TAB B INNING TABBIE SAINT S UB TAB TAB T RANSFORMER

0.684 0.677 0.615 0.499 0.467 0.361 0.346 0.336 0.304 0.252 0.237 0.161 0.166 0.133

0.680 0.609 0.585 0.457 0.421 0.288 0.302 0.275 0.265 0.274 0.219 0.185 0.161 0.147

0.683 0.658 0.606 0.487 0.454 0.340 0.333 0.318 0.293 0.259 0.231 0.168 0.164 0.137

Span (max−min)

0.551

0.533 0.546

69

Table 50: Proprietary embedding ablation across column/table and row tasks. All three OpenAI variants dominate matching tasks (TblRet ranks 1–3, RecLink ranks 1–3) but are mid-pack on structural grounding (TblQA ranks 5–7) and behind the task-adaptive model on row prediction (RowPred ranks 2, 3, 5). Row metrics average MLP and linear probes. RecLink reports binary F1 (match class), averaged unweighted over all 16 linkage datasets (the main table reports the same metric per group). Values are mean ± std over 5 seeds. Bold = best, underline = second best. Dashes indicate the model does not support the required embedding granularity. Column / Table Type

Model

BERT GTE TaBERT Table-Text TAPAS TAPEX TABBIE Table-Struct. TURL TUTA Starmie Col.-Centric TabSketchFM

Generic Text

O

Meta-Pretr. Tgt-Tbl

TabICL TransTab

Proprietary

TE3-Small TE3-Large Ada-002

Row

TblRet

TblQA

RowPred

RecLink

MRR ↑

Acc ↑

AUROC ↑

F1 ↑

0.367±0.008 0.476±0.003 0.372±0.013 0.295±0.006 0.376±0.097 0.170±0.004 0.199±0.010 0.260±0.013 0.018±0.002 0.218±0.011

0.255±0.004 0.245±0.002 0.267±0.005 0.254±0.003 — 0.276±0.004 0.277±0.005 — 0.266±0.005 0.235±0.005

0.791±0.000 0.770±0.000 — — — 0.770±0.001 — 0.720±0.000 — —

0.384±0.002 0.403±0.003 — — — 0.300±0.005 — 0.358±0.005 — —

— —

— —

0.816±0.001 0.274±0.005 0.778±0.001 0.375±0.017

0.490±0.007 0.260±0.005 0.801±0.000 0.412±0.004 0.511±0.006 0.265±0.003 0.797±0.001 0.426±0.004 0.540±0.007 0.266±0.004 0.789±0.001 0.423±0.002

Proprietary Embedding Ablation: Retrieval vs. Structural Grounding

We evaluate three OpenAI embedding variants [55, 57]: TE3-S MALL (text-embedding-3-small, 768d), TE3-L ARGE (text-embedding-3-large, 768-d), and A DA -002 (text-embedding-ada-002, 1536-d). These cover four representative tasks spanning column/table and row levels: Table Retrieval, Table QA, Row Prediction, and Record Linkage. TE3-Small and TE3-Large are requested at 768-d via OpenAI’s dimensions API parameter for dimensional parity with the 768-d open-source encoders in our pool. Ada-002 does not expose this option and retains its native 1536-d. Table 50 reports the results alongside the open-source models from the main tables. Column/table level. All three proprietary variants lead Table Retrieval, occupying ranks 1–3 (A DA -002: 0.540 MRR, vs. GTE: 0.476). This is consistent with their training objective: large-scale semantic retrieval over diverse text, which transfers directly to table-to-query matching. On Table QA, however, the same models rank only 5–7 out of 11, behind TURL (0.277), TABBIE (0.276), and S TARMIE (0.266), models whose pretraining encodes table structure. The contrast within a single task family shows that retrieval quality and structural understanding are distinct capabilities. Row level. The pattern extends to rows. On Record Linkage, an entity-matching task driven by text similarity, the proprietary models again occupy the top three ranks (TE3-L ARGE: 0.426 F1 , vs. GTE: 0.403). On Row Prediction, however, TE3-S MALL reaches 0.801 AUROC (rank 2) but trails TAB ICL (0.816), a meta-pretrained model whose in-context conditioning adapts to each task. Across both granularities, proprietary embeddings lead matching tasks (retrieval, linkage) but do not displace specialized models on understanding tasks (QA, prediction), reinforcing that no single model family, open or proprietary, is universally dominant. Scope and model selection. The general benchmark scope rationale is discussed in Appendix B. The proprietary OpenAI ablation above should therefore be read as a controlled scaling study within the embedding-only regime, not as a proxy for evaluating 7B–70B generative systems.

70

P

Robustness

This section reports three auxiliary stability diagnostics introduced by Observatory [15], namely Sample Fidelity, Perturbation Robustness, and Row/Column Order Insignificance, applied directly to the models in our pool. The diagnostic definitions, evaluation datasets, and headline metrics all follow the Observatory originals. These diagnostics are not part of our contribution and are not counted toward the 16 benchmark tasks. All numerical results below are recomputed from our finalized embeddings. All headline statistics use a unified table-first aggregation protocol. For Sample Fidelity and Row/Column Order Insignificance, we compute two canonical metrics, table_cosine_similarity and table_mcv, and then report the dataset-level mean and standard deviation over valid tables. For Sample Fidelity, each table cell below follows the format cosine mean ± std / MCV mean ± std. For Row/Column Order Insignificance, cosine similarity and MCV are reported in separate tables. For Perturbation Robustness, following the original Observatory protocol, we report changed-only cosine similarity only. Cosine similarity is the primary metric for cross-task comparison. P.1

Unified Aggregation Protocol

Let T (0) be the original table and {T (1) , . . . , T (K) } be its transformed variants. For a table item i (a (k) column, a row, or the whole table depending on the task) with embedding zi under variant k, we compute   1 X (0) (k) si = cos zi , zi , (13) |Vi | k∈Vi   (0) (k) ci = MCV {zi } ∪ {zi | k ∈ Vi } , (14) where Vi is the set of valid transformed variants for item i. The table-level metrics are then m

table_cosine =

1 X si , m i=1

table_mcv =

1 X ci , m i=1

(15)

m

(16)

with m being the number of valid items in the table. Finally, the appendix tables report the mean and standard deviation of these table-level metrics over the evaluation pset. MCV is computed with the Observatory multivariate coefficient of variation implementation µ⊤ Σµ/(µ⊤ µ)2 . For Perturbation Robustness, we use a strict changed-only protocol: only columns that were actually modified by a perturbation are included in the headline cosine metric; tables without changed columns are excluded from the dataset-level aggregate.

71

P.2

Task and Dataset Summary Table 51: Overview of the three appendix diagnostics.

Diagnostic

Core question

Data and scale

Embeddings and model coverage

Sample Fidelity

Whether a table representation remains stable after observing only a subset of rows.

wiki_tables; 4,964 base tables; sampling ratios 0.25, 0.50, and 0.75.

Column embeddings; the 8 column-capable models from our pool (shared across all three robustness diagnostics): bert, gte, starmie, tabbie, tabert, tabsketchfm, tapas, turl.

Perturbation Robustness

Whether column representations remain stable under schema and content perturbations.

Database tables; 80 Column embeddings; base tables; three the same 8 perturbation families: column-capable DB_schema_synonym, models as above. DB_schema_abbreviation, and DB_DBcontent_equivalence.

Row/Column Order Insignificance

Whether row, column, and table representations are invariant to row-order and column-order permutations.

wiki_tables; 4,964 base tables; each table has either 11 or 6 shuffle variants depending on how many unique permutations are available.

P.3

Column, row, and table embeddings; the 8 column-capable models above, 14 row-capable models, and 10 table-capable models from our pool.

Sample Fidelity

The Sample Fidelity diagnostic measures whether the semantic representation of a table remains stable when only a subset of rows is observed. Following the Observatory Sample Fidelity protocol [15], we construct subsampled variants at ratios 0.25, 0.50, and 0.75, and compare each sampled table against the original table through column embeddings. For each column, we compute the average cosine similarity between the original column embedding and all sampled variants, and the MCV over the set consisting of the original column embedding plus all sampled versions. The final table score is the mean over all columns in the same table. Table 52: Dataset parameters for Sample Fidelity. The 0.25 setting has fewer unique subsamples for small tables. The 0.50 and 0.75 settings use 11 versions per base table throughout. Sampling ratio 0.25 0.50 0.75

Base tables

11 variants

8 variants

7/6 variants

4,964 4,964 4,964

3,816 4,964 4,964

335 0 0

475 / 338 0/0 0/0

Analysis. Three patterns are especially clear. First, all eight models become more stable as the retained row fraction grows from 0.25 to 0.75, which means that sample fidelity is strongly tied to how much of the original row distribution remains visible. Second, tabert is the most stable model at every sampling ratio, indicating that its column representations are highly insensitive to the removal of rows. Third, tabbie is the most balanced non-tabert model, combining high cosine similarity 72

Table 53: Full Sample Fidelity results. Each cell reports cosine mean ± std on the first line and MCV mean ± std on the second line. 0.25

0.50

0.75

0.9138 ± 0.0284 / 0.0211 ± 0.0083 0.8290 ± 0.0564 / 0.0316 ± 0.0134 0.9198 ± 0.1161 / 0.0306 ± 0.0407 0.9567 ± 0.0363 / 0.0099 ± 0.0087 0.9933 ± 0.0029 / 0.0025 ± 0.0012 0.4692 ± 0.0984 / 0.9489 ± 0.1555 0.7961 ± 0.0668 / 0.3975 ± 0.1023 0.8679 ± 0.0736 / 0.4658 ± 0.1899

0.9619 ± 0.0175 / 0.0144 ± 0.0073 0.9180 ± 0.0342 / 0.0212 ± 0.0116 0.9846 ± 0.0291 / 0.0123 ± 0.0175 0.9873 ± 0.0136 / 0.0032 ± 0.0032 0.9965 ± 0.0015 / 0.0017 ± 0.0007 0.6596 ± 0.0916 / 0.7524 ± 0.1485 0.9146 ± 0.0407 / 0.2619 ± 0.0666 0.9506 ± 0.0313 / 0.2669 ± 0.0988

0.9828 ± 0.0108 / 0.0110 ± 0.0076 0.9620 ± 0.0199 / 0.0175 ± 0.0124 0.9923 ± 0.0194 / 0.0084 ± 0.0146 0.9959 ± 0.0052 / 0.0014 ± 0.0017 0.9982 ± 0.0009 / 0.0013 ± 0.0006 0.8084 ± 0.0720 / 0.5623 ± 0.1356 0.9602 ± 0.0260 / 0.1827 ± 0.0533 0.9782 ± 0.0197 / 0.1735 ± 0.0726

Model bert gte starmie tabbie tabert tabsketchfm tapas turl

Table 54: Dataset parameters for Perturbation Robustness. “Valid tables” are tables with at least one changed column under the given perturbation type. Perturbation type

Original tables

Valid tables

Skipped tables

Changed pairs

Unchanged pairs

Changed columns

80 80 80

29 76 69

51 4 11

238 691 453

1,320 1,456 1,708

101 232 202

Content equivalence Schema abbreviation Schema synonym

with low variance across all three settings, whereas tabsketchfm is much more sample-sensitive and degrades substantially when only 25% of rows are retained. P.4

Perturbation Robustness

The Perturbation Robustness diagnostic evaluates whether column representations remain stable after semantically valid modifications to the table schema or content. Following the Observatory Perturbation Robustness protocol [15], we use the same three perturbation families: DB_schema_synonym (schema names replaced by synonyms), DB_schema_abbreviation (schema names replaced by abbreviations), and DB_DBcontent_equivalence (schema or content rewritten in a semantically equivalent form). The analysis uses a strict changed-only headline protocol. For each original table and each column that is modified by a perturbation, we compute the average cosine similarity between the original column embedding and all changed variants. The table-level score is the mean over all changed columns in that table. Analysis. The hardest perturbation is content equivalence, followed by the two schema-only perturbations. This shows that semantic re-expression of column content is much more difficult than renaming a schema attribute with a synonym or abbreviation. tabbie is the strongest overall model and remains the most robust on the hardest content-equivalence setting. starmie is nearly invariant to schema-level changes, but drops more than tabbie under content-level perturbations, suggesting that its robustness is particularly strong at the schema level. By contrast, gte and especially tabsketchfm are clearly more sensitive to semantic perturbations. Note that the content equivalence evaluation uses only 29 valid tables (Table 54), so these comparisons should be treated as indicative rather than definitive. P.5

Row/Column Order Insignificance

The Row/Column Order Insignificance diagnostic evaluates whether learned representations are invariant to row-order and column-order permutations. Following the Observatory Row/Column Order Insignificance protocol [15], each base table is paired with multiple shuffled variants from wiki_tables. In our data, 4,451 base tables have 11 variants (the original table plus 10 shuffles), while 513 tables have 6 variants. We study six evaluation conditions: column/column, column/row, row/column, row/row, table/column, and table/row. The first term denotes the embedding granularity and the second term denotes the applied shuffle type. Thus, column/column measures the invariance of column embeddings to column permutation, column/row tests whether column semantics change under row 73

Table 55: Full Perturbation Robustness results. Each cell reports changed-only cosine mean with small-font standard deviation. Model

Content equivalence Schema abbreviation Schema synonym

bert gte starmie tabbie tabert tabsketchfm tapas turl

0.8420 ± 0.0358 0.5777 ± 0.0961 0.9261 ± 0.0715 0.9783 ± 0.0224 0.9451 ± 0.0181 0.2939 ± 0.1748 0.6605 ± 0.0866 0.7551 ± 0.0965

0.9453 ± 0.0294 0.7577 ± 0.0914 0.9998 ± 0.0008 0.9982 ± 0.0028 0.9507 ± 0.0160 0.6471 ± 0.2820 0.9545 ± 0.0342 0.9857 ± 0.0425

0.9524 ± 0.0277 0.8283 ± 0.0644 0.9998 ± 0.0009 0.9987 ± 0.0017 0.9635 ± 0.0134 0.5459 ± 0.2646 0.9584 ± 0.0309 0.9905 ± 0.0344

Table 56: Cosine similarity results for the Row/Column Order Insignificance diagnostic. The six metric columns are grouped by embedding granularity and shuffle type. NA means that the corresponding embedding granularity is not available for that model. Column embeddings row shuffle

Row embeddings

Model

column shuffle

bert dae gte saint scarf starmie subtab tabbie tabert tabicl tabpfn tabsketchfm tabtransformer tabular_binning tapas tapex transtab turl tuta vime

1.0000 ± 0.0000 0.9809 ± 0.0147 0.9754 ± 0.0102 NA NA 0.5015 ± 0.0750 1.0000 ± 0.0000 0.9704 ± 0.0190 0.9556 ± 0.0204 NA NA 0.7533 ± 0.1345 NA NA 0.5371 ± 0.0700 0.9988 ± 0.0027 0.9708 ± 0.0346 NA NA NA 0.8233 ± 0.0762 0.9966 ± 0.0039 0.9986 ± 0.0021 0.9941 ± 0.0029 0.9706 ± 0.0056 0.9947 ± 0.0021 NA NA NA 0.6155 ± 0.0854 NA NA 0.9998 ± 0.0001 0.9598 ± 0.0255 1.0000 ± 0.0000 NA NA NA 0.4226 ± 0.0808 NA NA 0.5587 ± 0.0592 0.9277 ± 0.0242 0.9821 ± 0.0108 NA

column shuffle

row shuffle

Table embeddings column shuffle

1.0000 ± 0.0000 0.9731 ± 0.0144 0.9770 ± 0.0082 NA 1.0000 ± 0.0000 0.9590 ± 0.0174 0.8857 ± 0.0823 NA 0.9741 ± 0.0092 NA NA 0.9988 ± 0.0027 0.9245 ± 0.0282 NA 1.0000 ± 0.0000 0.9975 ± 0.0028 NA 0.9958 ± 0.0012 1.0000 ± 0.0000 NA 1.0000 ± 0.0000 NA NA 0.9902 ± 0.0063 0.4370 ± 0.0761 NA 0.9725 ± 0.0084 NA NA 0.9461 ± 0.0680 NA NA NA NA 0.9944 ± 0.0045 NA NA 0.8712 ± 0.0621 0.9627 ± 0.0349 NA 0.9974 ± 0.0125 0.9970 ± 0.0128 NA NA 0.9974 ± 0.0128 NA NA 0.9279 ± 0.0262 1.0000 ± 0.0000 0.9278 ± 0.0547 NA NA 0.5365 ± 0.0757 0.9783 ± 0.0078 NA

row shuffle 0.9824 ± 0.0168 NA

0.9648 ± 0.0199 NA NA

0.9580 ± 0.0518 NA

0.9988 ± 0.0021 0.9984 ± 0.0009 NA NA

1.0000 ± 0.0000 NA NA

0.9705 ± 0.0624 0.9988 ± 0.0029 NA

0.9972 ± 0.0130 0.9387 ± 0.0549 NA

shuffling, row/column and row/row evaluate the stability of row embeddings under column-order and row-order changes, respectively, and table/column and table/row test whole-table invariance to column and row permutations. When needed, shuffled items are realigned to their original indices before comparison; models that already canonicalize the relevant order do not require this extra step. Analysis. The strongest qualitative pattern is that many models are almost perfectly stable on row/row, whereas the more difficult condition is often row/column, which requires row semantics to survive a change in column order. Among shared column/table models, tabbie is the most balanced model across all available conditions, while turl and tabert also show very strong order robustness. tabsketchfm is especially notable because it is extremely stable under order perturbations even though it is much weaker on the other two tasks. For row representations, tabpfn is almost ideal, tabbie, bert, and gte form a strong second tier, and tabtransformer is the least robust model in this setting, consistent with its use of learned column positional embeddings that make row representations inherently sensitive to column order. P.6

Cross-Task Comparison of Shared Models

To avoid confounding caused by different model coverage across tasks, the cross-task comparison uses only the eight models that appear in all three evaluations: bert, gte, starmie, tabbie, tabert, tabsketchfm, tapas, and turl. We rank models within each task by their average cosine performance and then compute the average rank across tasks (an ordinal summary that reflects relative positioning rather than the magnitude of score differences). Analysis. tabbie is the most balanced model across tasks. It is not always the single best model on every individual condition, but it remains near the top under sampling, semantic perturbation, and 74

Table 57: MCV results for the Row/Column Order Insignificance task. The six metric columns are grouped by embedding granularity and shuffle type. NA means that the corresponding embedding granularity is not available for that model. Column embeddings

Row embeddings

Model

column shuffle

row shuffle

column shuffle

bert dae gte saint scarf starmie subtab tabbie tabert tabicl tabpfn tabsketchfm tabtransformer tabular_binning tapas tapex transtab turl tuta vime

0.0000 ± 0.0000 0.0097 ± 0.0054 0.0127 ± 0.0054 NA NA 0.1699 ± 0.0371 0.0000 ± 0.0000 0.0052 ± 0.0031 0.0074 ± 0.0035 NA NA 0.0545 ± 0.0357 NA NA 0.1551 ± 0.0317 0.0028 ± 0.0039 0.0222 ± 0.0158 NA NA NA 0.3267 ± 0.1011 0.0010 ± 0.0012 0.0004 ± 0.0005 0.0037 ± 0.0013 0.0038 ± 0.0009 0.0019 ± 0.0008 NA NA NA 0.0439 ± 0.0167 NA NA 0.0001 ± 0.0000 0.0066 ± 0.0034 0.0000 ± 0.0000 NA NA NA 0.2524 ± 0.0582 NA NA 0.1514 ± 0.0326 0.0276 ± 0.0088 0.0142 ± 0.0048 NA

Table embeddings

row shuffle

column shuffle

row shuffle

0.0000 ± 0.0000 0.0129 ± 0.0063 0.0412 ± 0.0135 NA 0.0000 ± 0.0000 0.0064 ± 0.0033 0.0289 ± 0.0247 NA 0.0324 ± 0.0086 NA NA 0.0028 ± 0.0039 0.2524 ± 0.0550 NA 0.0000 ± 0.0000 0.0014 ± 0.0013 NA 0.0010 ± 0.0004 0.0000 ± 0.0000 NA 0.0000 ± 0.0000 NA NA 0.0013 ± 0.0008 0.2470 ± 0.0540 NA 0.0394 ± 0.0111 NA NA 0.0288 ± 0.0245 NA NA NA NA 0.0191 ± 0.0096 NA NA 0.0279 ± 0.0204 0.0136 ± 0.0100 NA 0.0027 ± 0.0091 0.0029 ± 0.0089 NA NA 0.0044 ± 0.0148 NA NA 0.0587 ± 0.0190 0.0000 ± 0.0000 0.0514 ± 0.0348 NA NA 0.1638 ± 0.0378 0.0430 ± 0.0144 NA

0.0102 ± 0.0057 NA

0.0052 ± 0.0031 NA NA

0.0250 ± 0.0236 NA

0.0009 ± 0.0009 0.0006 ± 0.0003 NA NA

0.0000 ± 0.0000 NA NA

0.0197 ± 0.0230 0.0046 ± 0.0036 NA

0.0046 ± 0.0143 0.0430 ± 0.0307 NA

Table 58: Cross-task ranking over the eight column-capable models from our pool that appear in all three robustness evaluations. Lower average rank means better overall robustness. Model tabbie tabert starmie turl bert tabsketchfm gte tapas

Sample rank

Perturbation rank

Order rank

Average rank

2 1 3 5 4 8 6 7

1 3 2 5 4 8 7 6

1 3 6 2 5 4 7 8

1.33 2.33 3.67 4.00 4.33 6.67 6.67 7.00

structural reordering, which makes it the strongest all-round choice in our study. tabert is the best model for sample fidelity, while starmie is especially strong for schema-level perturbations. turl is consistently robust without being the single best model in any one dimension. Finally, tabsketchfm is the clearest example of a task-specialized behavior: it is highly order-invariant, but much weaker under row subsampling and semantic perturbation. P.7

Implementation Notes and Caveats

All numbers in this section are recomputed from finalized embeddings using a unified table-first headline protocol rather than copied from older logs. This recomputation includes three practical decisions that are important for interpretation: • Table-first aggregation. All three tasks first aggregate to per-table scores and only then compute dataset-level mean and standard deviation. • Changed-only perturbation scoring. For Perturbation Robustness, only changed columns contribute to the headline cosine metric. We do not report MCV for this task, matching the original Observatory perturbation protocol. • MCV comparability. MCV is computed from different numbers of embedding variants in the sample-fidelity and order-insignificance tasks (up to 11 for order shuffles), and the covariance estimate is rank-deficient in all cases (K+1 samples in d-dimensional space, K+1 ≪ d). MCV magnitudes should therefore be compared only within the same task. These caveats do not change the main qualitative conclusions, but they matter for correct interpretation of the absolute numbers and for reproducible comparison across properties.

75

Q

Computational Efficiency

Table 59: Column embedding generation cost (median wall-clock seconds ± IQR across the efficiency test suite). All models are frozen inference. Model BERT GTE TABBIE TAPAS TURL TA BERT TAB S KETCH FM

Time (s)

Datasets

5.3 ± 4.9 5.7 ± 1.3 7.4 ± 3.2 7.8 ± 3.9 11.2 ± 1.8 13.2 ± 9.7 14.1 ± 5.4

58 58 59 58 55 58 58

Table 60: Table embedding generation cost (median wall-clock seconds ± IQR). Both models are frozen inference. Model TAPEX TUTA

Time (s)

Datasets

6.1 ± 3.3 119.6 ± 138.8

58 58

We report per-model embedding generation cost across three workloads (column, table, row) on a controlled efficiency test suite. All measurements use a single NVIDIA L40S GPU with 32 GB RAM per job. Efficiency test suite. The suite comprises: (i) Eff-Real: 8 anchor tables selected from the 50 TRL-R BENCH OpenML tables via metadata-space clustering (covering 1 000–71 518 rows and 18–1 775 columns with diverse type mixes and missingness rates); (ii) Eff-Scale: 47 semi-synthetic tables generated by varying one factor at a time from a baseline (row track: N ∈ {500, . . . , 100k}, D ∈ {8, . . . , 256}, categorical share, cardinality, missingness; column track: C ∈ {4, . . . , 128}, context rows, cell token length, type mix); and (iii) Bridge: 3 tables valid for both row and column workloads. Each model is timed using its unmodified production embedding script via a thin wall-clock wrapper, ensuring that the measured cost exactly matches the actual benchmark pipeline. Column and table embedding cost. Table 59 reports column-level results. Generic text encoders are fastest (BERT 5.3 s, GTE 5.7 s median), while TA BERT (13.2 s) and TAB S KETCH FM (14.1 s) are slowest, a 2.7× spread. For table embeddings (Table 60), TAPEX (6.1 s) is 20× faster than TUTA (119.6 s), reflecting TUTA’s cell-level tokenization overhead. Row embedding cost. Table 61 reports row-level results. The 101× spread between the fastest (TAB ICL, 8.7 s) and slowest (T RANS TAB, 875 s) models is driven primarily by the frozen-vs.-trained regime distinction: all target-table self-supervised models include per-table training, which dominates their wall-clock cost. Within the frozen regime, TAB ICL and TAB PFN (20.7 s) are fastest despite requiring a fit step, because their meta-learned priors avoid gradient-based training. Scaling behavior. Figure 12 shows how row embedding cost scales with table size. Training-based models (T RANS TAB, SAINT) scale super-linearly with row count, while frozen inference models scale approximately linearly. Feature count scaling is more uniform across the models in the sweep. Column-embedding cost (Table 59) is dominated by per-column tokenization overhead: TA BERT and TAB S KETCH FM are roughly 2.5–2.7× slower per job than BERT/GTE. The column track of E FF -S CALE (C ∈ {4, . . . , 128}) confirms that this gap widens approximately linearly with column count (full curves in the supplementary material). Support envelope. Not all models can handle all scales under the 1-hour budget. TURL runs out of memory on tables with >208 columns or >256 features. TAB PFN cost rises sharply with feature count on E FF -R EAL anchors (18 s at 28 features, 263 s at 208 features, and timeout at 1 775 features), roughly a 15× slowdown over a 7× increase in feature count. SAINT and TAB T RANSFORMER fail 76

Table 61: Row embedding generation cost (median wall-clock seconds ± IQR). “Train” models include per-table self-supervised training. “Infer” models are frozen or meta-pretrained.

103

BERT TABBIE TabularBinning SCARF SwitchTab DAE

Type

Time (s)

Datasets

TAB ICL TAB B INNING TAB PFN SCARF GTE DAE BERT TABBIE S UB TAB VIME TAB T RANSFORMER SAINT T RANS TAB

Infer Train Infer Train Infer Train Infer Infer Train Train Train Train Train

8.7 ± 8.8 10.2 ± 1.4 20.7 ± 125.0 30.9 ± 6.1 31.1 ± 14.5 44.8 ± 27.1 50.1 ± 17.4 57.1 ± 24.6 62.9 ± 20.0 71.8 ± 32.2 86.5 ± 38.0 167.9 ± 64.3 875.1 ± 512.1

8 38 7 38 38 38 38 38 38 38 36 37 38

SubTab VIME TabTransformer GTE SAINT TransTab

Wall-clock time (s)

Wall-clock time (s)

104

Model

102 101

103

BERT TABBIE TabularBinning SCARF SwitchTab DAE

SubTab VIME TabTransformer GTE SAINT TransTab

102

101 103

104 Number of rows (N)

105

101

(a) Row scaling (varying N )

102 Number of features (D)

(b) Feature scaling (varying D)

Figure 12: Embedding generation cost vs. table size (log-log scale). Training-based models scale super-linearly with rows. on the widest anchor table (1 775 columns). These limits are important for practitioners selecting models for large-scale deployment. Methodology. TAB ICL and TAB PFN are timed only on E FF -R EAL anchor tables. Their context-fit step is most representative of production usage when run on real labeled splits, while the synthetic E FF S CALE suite is unlabeled. This labeled-split timing reflects deployment cost. The row-embedding matrix consumed by every TRL-R BENCH task is extracted from a target-agnostic forward pass conditioned only on the unlabeled X rows, so the same matrix is reused across all curated targets within a table. Each measurement times the unmodified production script end-to-end (including model loading, preprocessing, and any per-table training) via a subprocess wrapper that records wallclock time and polls nvidia-smi for peak GPU VRAM at 0.5 s intervals. Results are recorded as individual JSON files with full provenance (hostname, GPU type, SLURM job ID, return code, output verification). The complete 994-run result set and analysis code are included in the supplementary material.

77

R

Reproducibility Details

Data access. All source datasets are publicly available. OpenML tables are accessed via the OpenML API using the dataset IDs listed in Appendix F. DeepMatcher datasets are from the original DeepMatcher release. WDC Products data is from the WDC Product Data Corpus (the LSPM v2 release). CTBench datasets (SATO, SOTAB, SANTOS, Valentine, etc.) are from their respective original releases. DLTE parent tables are derived from TabFact and WTQ. Splits. Row prediction uses the canonical OpenML train/test splits. Record linkage retains the original DeepMatcher (3:1:1 train/valid/test) and WDC splits; both benchmarks define exactly two tables per dataset, so table-disjoint evaluation does not apply. For the four CTBench pairwise tasks marked † (join classification, column overlap, union classification, union regression), we generate table-disjoint splits as follows. All pairs from the source pair-level random splits are pooled, and the set of unique tables is partitioned randomly (seed 42) into disjoint train/valid/test sets at a 70/15/15 ratio. Each pair is then assigned to the split containing both of its tables, and cross-partition pairs are discarded. For spider_join, tables are grouped by database prefix before partitioning, so all tables from the same database land in the same split. Retention rates range from 53% to 100% of original pairs depending on the dataset. The remaining supervised table-pair task (table subset) already has table-disjoint splits in the source data. DLTE uses a parent-table-level 827/207/345 train/dev/test split fixed at benchmark construction (before any pipeline evaluation) by two successive calls to train_test_split: first 75/25 train+dev vs. test, then 80/20 train vs. dev within the train+dev portion, yielding an effective 60/15/25 split. The split is random, not stratified by source (TabFact vs. WTQ). Given the 989/390 source composition, each split is expected to contain roughly 72% TabFact and 28% WTQ parents. Split manifests are included in the released code. All 1,120 canonical pipelines completed across all 5 rounds. Hyperparameters. Supervised probe training follows the unified protocol of Sec. 3.1: for each supervised probe task, we train both a linear head and a one-hidden-layer MLP with hidden size 256 using Adam (learning rate 10−3 ) for up to 100 epochs with early stopping on the validation set, and the headline score is the arithmetic average of the two heads. Seeds. All supervised probe results, including record linkage, are averaged over 5 random seeds: 42, 52, 62, 72, 82. Model wrappers and launch configurations. Exact model-specific wrapper parameters, truncation rules, and embedding-generation launch configurations are provided in the released codebase. Appendix D states the benchmark-level policy. The codebase contains the executable per-model settings used to produce the released embeddings. Third-party model packages. TabPFN and TabICL are used as provided by their respective authors. Their internal weights are not modified during evaluation. The benchmark consumes only the exported intermediate row representations used by the shared probe protocol of Sec. 3.1, never their task predictions as final benchmark outputs.

78

S

Statistical Reporting

For column/table tasks, we report means over 5 random seeds for all supervised probe results. Standard deviations are reported per cell in the main results tables (Tables 2, 3) and in the aggregation ablation tables (Tables 17, 18). Training-free tasks (column clustering, schema matching) are deterministic given fixed embeddings. For row prediction, the reported metrics are macro-averaged across all 123 targets. The sole exception is TAB T RANSFORMER, which covers 63 targets due to its categorical-feature requirement. For DLTE, end-to-end results are 5-round averages. We do not report confidence intervals for the normalized-rank aggregates, as these are summary statistics over heterogeneous per-task metrics. Per-task raw scores with standard deviations are the appropriate unit of statistical comparison. Significance tests. To assess whether model differences are statistically meaningful, we apply Friedman omnibus tests and Holm-corrected pairwise Wilcoxon signed-rank tests, using per-task mean scores (averaged over 5 seeds) as the unit of analysis. Table 62 summarizes the results. Table 62: Friedman omnibus tests and Holm-corrected pairwise Wilcoxon signed-rank results. Each row uses per-task means (averaged over 5 seeds) as the unit of analysis. “Sig. pairs” reports the
number of pairwise comparisons with padj < 0.05 after Holm correction, out of k2 total pairs for k models. TAB T RANSFORMER is excluded from the RBench rows because its partial target coverage (63 of 123) breaks the matched-sample requirement of Friedman/Wilcoxon tests, leaving 13 of the 14 non-baseline row models. Scope

Suite

All tasks Schema Join Union Classification Regression

CTBench CTBench CTBench CTBench RBench RBench

Tasks

Models

Friedman χ2 (p)

Sig. pairs (Holm)

13 3 3 5 77 46

8 8 8 8 13 13

32.1 (<0.0001) 18.0 (0.012) 13.0 (0.072) 11.0 (0.139) 209.4 (3.7×10−38 ) 154.6 (6.5×10−27 )

6 / 28 — — — 26 / 78 38 / 78

On TRL-CT BENCH (13 tasks, 8 fully supported models), the Friedman test rejects the null hypothesis that all models perform equally (χ2 = 32.1, p < 0.0001). However, only 6 of 28 pairwise Wilcoxon signed-rank comparisons are significant after Holm correction (all padj < 0.05), and all six involve TAB S KETCH FM being significantly weaker than the top models. The remaining models (BERT, GTE, TA BERT, TAPAS, TURL, TABBIE, and S TARMIE) are not significantly different from each other across the full task set. Per-family Friedman tests are significant for Schema (p = 0.012) but not for Join (p = 0.072) or Union (p = 0.139), consistent with the observation that Union-family differences are especially narrow. Figure 13 renders the corresponding Demšar critical-difference (CD) diagram [16] using post-hoc Nemenyi at α = 0.05: BERT holds the best mean rank (2.23), and under the CD threshold only pairs involving TAB S KETCH FM (vs. BERT or GTE) and BERT vs. TABBIE clear significance. Most CTBench models form a single overlapping clique, visually confirming the near-tie pattern. On TRL-R BENCH, the larger number of targets provides substantially more statistical power. Classification (77 targets, 13 models; TAB T RANSFORMER excluded due to partial target coverage) yields χ2 = 209.4, p = 3.7 × 10−38 , with 26/78 pairwise comparisons significant. Regression (46 targets, 13 models) yields χ2 = 154.6, p = 6.5 × 10−27 , with 38/78 pairs significant. TAB ICL’s advantage over all other models is confirmed on both classification and regression (all padj < 0.001). Among the remaining transfer-based and target-table learners, most pairwise differences are non-significant (padj = 1.0), supporting the paper’s conclusion that training regime and task family matter more than individual model choice within a regime. TRL-DLTE per-stage significance. For DLTE we test whether each stage’s model identity has a significant end-to-end effect, blocking by the remaining two stages and using the aggregate of 5-round mean scores per pipeline as the unit of analysis. All three stages have balanced complete-block designs: Stage 1 has 112 blocks (8 column × 14 row other-stage combinations) with 10 target models, Stage 2 has 140 blocks (10 tables × 14 rows) with 8 target models, and Stage 3 has 80 79

CTBench critical-difference diagram (k=8 models, N=13 tasks, Friedman 2=32.1, p=3.97e-05; Nemenyi CD = 0.05=2.91) CD = 2.91

2

3

4

5

BERT (2.23) GTE (3.08) TAPAS (4.08) TaBERT (4.15)

6

7

8 TabSFM (6.92) TABBIE (5.38) Starmie (5.08) TURL (5.08)

Figure 13: CTBench critical-difference diagram (Demšar style). The horizontal axis shows mean rank across 13 CTBench tasks for the 8 fully-supported models. Horizontal bars group models whose mean-rank differences are below the Nemenyi critical difference (CD= 2.91), so models sharing a bar are not significantly different at α = 0.05. Only pairs involving TAB S KETCH FM and the two best-ranked generic-text encoders (BERT, GTE) or BERT vs. TABBIE clear the CD threshold. Most of the top-eight CTBench models live in a single overlapping clique. blocks (10 tables × 8 columns) with 14 target models. Table 63 reports Friedman omnibus statistics with Kendall’s concordance W as an effect-size summary, together with Holm-corrected pairwise Wilcoxon signed-rank tests for all three stages on the primary metric UJ-H and the complementary Cell F1 diagnostic. All six stage-metric combinations reject the null at p ≤ 1.1×10−13 (Stage 3 UJ-H), with the other five well below 10−16 , but the interesting picture is the effect-size pattern rather than the p-values. Stage 2 (column alignment) carries the single largest effect on both metrics (Kendall’s W = 0.79 on Cell F1 , 0.66 on UJ-H, with 26/28 and 24/28 Holm-significant pairs), consistent with the finding that column alignment is the most sensitive lever in the pipeline. Stage 1 (retrieval) has a moderate effect on both metrics (W = 0.57 on Cell F1 and W = 0.33 on UJ-H, with 38/45 and 38/45 Holm-significant pairs). Stage 3 exhibits a striking metric-dependent asymmetry. On Cell F1 the effect is strong (W = 0.61, χ2 = 638.7) with 87 of 91 pairwise comparisons Holmsignificant, reflecting the sharp separation between union-dedup specialists (TAB T RANSFORMER, S UB TAB, SAINT) and the other row models on union-side raw cell recovery. On UJ-H, by contrast, the effect collapses to W = 0.09 (weak) and only 31 of 91 pairwise comparisons survive Holm correction, even though the omnibus remains significant. This is not evidence of absent Stage 3 signal: the Oracle-RA diagnostic (Table 39) shows a latent cross-row-model UJ-H spread of 0.546, far above the end-to-end marginal span of ∼ 0.013. The metric-dependent asymmetry instead reflects the DLTE composition bottleneck quantified in Sec. 4.4: Cell F1 is sensitive to Stage 3 union-side behavior, which passes through the pipeline largely intact, while UJ-H additionally requires join-side recovery, which is masked by upstream retrieval/alignment error and therefore compresses the visible Stage 3 effect within the end-to-end pipeline. These tests quantify average stage effects, not the additivity of best compositions. Large average effects do not by themselves determine which model combination is best, and top DLTE quality depends on non-additive compositional fit.

80

Table 63: DLTE per-stage statistical significance on the primary metric U J-H and the complementary Cell F1 diagnostic. Each row tests whether stage-specific model identity affects end-to-end performance, blocking by the remaining two stages (one block per (other-stage-a, other-stage-b) combination, with the aggregate 5-round mean score per pipeline as the unit of analysis). Kendall’s W is the effect-size summary of Friedman, W = χ2 /(n(k − 1)), bounded in [0, 1]. W > 0.5 is a strong effect, W < 0.1 is weak. “Sig. pairs
(Holm)” counts Holm-corrected pairwise Wilcoxon signed-rank tests with padj < 0.05, out of k2 total pairs for k models. All three stages have complete block designs and yield highly significant Friedman omnibus tests on both metrics. The most informative contrast is the large metric-dependent Stage 3 effect-size gap (W = 0.61 on Cell F1 vs. W = 0.09 on U J-H), which reflects how upstream retrieval/alignment errors mask the join-side Stage 3 signal in the balanced U J-H metric. Target stage Stage 1 (table model) Stage 2 (column model) Stage 3 (row model)

Metric

Blocks

k

Friedman χ2 (p)

Kendall’s W

Sig. pairs (Holm, < 0.05)

Cell F1 U J-H Cell F1 U J-H Cell F1 U J-H

112 112 140 140 80 80

10 10 8 8 14 14

569.6 (7.1×10−117 ) 332.6 (3.2×10−66 ) 776.7 (2.0×10−163 ) 649.9 (4.3×10−136 ) 638.7 (4.3×10−128 ) 90.6 (1.1×10−13 )

0.57 0.33 0.79 0.66 0.61 0.09

38 / 45 38 / 45 26 / 28 24 / 28 87 / 91 31 / 91

81

T

Dataset Counting Protocol

The total dataset count depends on the grouping granularity. At the finest level, treating each distinct dataset source as one entry, the benchmark contains 87 datasets (20 CTBench + 50 OpenML rowprediction + 16 record-linkage + 1 DLTE enrichment lake). Alternative aggregations include: 84 (grouping the 4 OpenData regional splits as one source), 77 (additionally grouping the 4 WDC Products size variants as one entry and pairing each DeepMatcher dirty variant with its clean counterpart), and 28 (additionally grouping all 50 OpenML row-prediction tables as one curated collection). Throughout the main paper, we use the dataset-source level as the default unless otherwise noted.

82

U

Broader Impact

A benchmark for tabular representations can improve scientific comparability and reduce evaluation fragmentation. It can also help practitioners identify when specialized table-aware models are necessary and when simpler frozen encoders suffice. At the same time, stronger table representations can be used in settings involving sensitive records, schema inference, or large-scale data linkage. Benchmark releases should therefore carefully respect data licenses, privacy constraints, and documentation requirements. In particular, record linkage and data lake retrieval settings can raise concerns around surveillance, re-identification, and inappropriate dataset fusion. A responsible release should document dataset provenance and usage restrictions.

83

V

Licenses and Asset Documentation

All source datasets are used under their original licenses. TRL-B ENCH creates derived benchmark assets from public source data, including curated target selections, rewritten row-pair tasks, new table-disjoint split manifests, and TRL-DLTE table fragments. For each source, we document the transformation applied (label repairs, split regeneration, row-pair rewrites, fragmentation) and respect the redistribution terms of the original license: where source licenses permit redistribution we release the derived files directly, and where they do not we release reconstruction scripts and manifests rather than mirrored raw files. For platform-hosted collections (e.g., OpenML, CKAN / open government portals), licenses can vary by dataset. The release therefore includes a per-asset manifest with the original source, URL, license, and redistribution status. The benchmark code will be released under an open-source license.

84

Table 64: Licenses and provenance for assets used in TRL-B ENCH. License entries reflect the upstream code/data licenses observed at the official source as of preparation. Where the source repository or release page does not state an explicit dataset license, we mark the cell accordingly. “Code” and “data” licenses are listed separately when they differ. For benchmarks redistributed through LakeBench, both the LakeBench redistribution license and the upstream license are noted. Asset

Citation License

OpenML tables

[73]

SATO

[82]

SOTAB

[43]

WikiCT (relation)

[17]

Wiki Containment / Wiki Union

[68]

SANTOS

[39]

UGEN TUS / TUS-hard

[58] [54]

Valentine

[45]

Spider Join

[81, 68]

OpenData (main / CAN / USA / UK / SG)

[18]

ECB Union / CKAN Subset DeepMatcher

[68]

WDC Products

[63, 62]

WTQ NQ-Tables

[60] [30]

TabFact CKAN distractors

[13] [68]

LakeBench (Srinivas et al.)

[68]

TabArena

[21]

[53]

Notes

Per-dataset (commonly CC0 / CC-BY)

Via OpenML API; licenses are set per dataset by the uploader Apache 2.0 Public release; tables from the WebTables corpus within VizNet Not explicitly stated on dataset page Public release on Web Data Commons; benchmark data must not appear in training (CEUR paper itself: CC-BY 4.0) corpora (per WDC notice) Apache 2.0 (TURL code); CC-BY-SA Relation-extraction split from TURL; inherited from upstream Wikipedia content tables from the WikiTables / TabEL corpus CC-BY 4.0 (LakeBench Zenodo record); Derived from Wikipedia tables; splits from CC-BY-SA 4.0 (label files and Wikipedia LakeBench (Zenodo archives wiki-containment and wiki-union) content) BSD-3-Clause (code); CC-BY 4.0 (data on Public release Zenodo) MIT GitHub: northeastern-datalab/gen Unspecified (source repo has no LICENSE TUS from Nargesian et al.; TUS-hard is a file) repo-derived low-overlap subset (Hungarian-matched containment < 0.70) Apache 2.0 Public release; benchmark assets in a separate (also Apache 2.0) repo CC-BY-SA 4.0 (Spider upstream; Databases from Spider; join-classification benchmark from LakeBench ShareAlike propagates to Spider-derived artifacts); CC-BY 4.0 (LakeBench-original metadata) Unspecified (BIT-DataLab/LakeBench Join- and union-search benchmarks from source repo has no LICENSE; data hosted the Deng et al. LakeBench; per-portal on Google Drive) open-data licenses for source tables (Canadian / UK / US / Singapore open-data portals) CC-BY 4.0 (LakeBench Zenodo); Benchmarks from LakeBench; source data per-portal open licenses for source tables from open govt. portals BSD-3-Clause (code); benchmarks Public release released without explicit dataset license Not explicitly stated on Web Data Training/gold-standard release at WWW Commons; publicly released ’19 Companion (ECNLP); LSPM v2 extension from WIMS 2020 CC-BY-SA 4.0 Table QA + DLTE parent tables Apache 2.0 (TAPAS release artifact, NQ Public release source repo); CC-BY-SA 3.0 (Wikipedia text upstream, pre-2023 NQ era) CC-BY 4.0 (data); MIT (code) DLTE parent tables CC-BY 4.0 (LakeBench Zenodo); TRL-B ENCH-derived selection from the per-portal open licenses for source tables LakeBench CKAN Subset pool; not a separately published LakeBench split CC BY-NC-ND 4.0 (code at Public release; TRL-B ENCH derives only IBM/tabsketchfm); CC-BY 4.0 / from the CC-BY 4.0 Zenodo data, not the CC-BY-SA 4.0 (data on Zenodo) NC-ND code Apache 2.0 Public release; per-table licenses inherited from upstream sources (UCI, Kaggle, OpenML, etc.)

85

Related documents

• String Diagrams for Process Mining#978502
• ZigZag Trie: A Novel Index for Contextual Queries#978503
• A Functional Pilot for Certified Freshness-Aware Semantic--Spatial Range Retrieval#978504
• Efficiently Linking Unstructured Data for Multi-step Reasoning#978505