arXiv CSopen access

LimiX-2: A Contextual Mechanism Network Towards General Structured-Data Intelligence

· arxiv_cs

arXiv CS · Papers · License: Open Access

artificial intelligence, reasoning, knowledge representation

2026-09-16

LimiX-2: A Contextual Mechanism Network Towards General Structured-Data Intelligence LimiX Team Stable AI & Tsinghua University https://github.com/limix-ldm/LimiX/ https://huggingface.co/stableai-org/ https://modelscope.cn/organization/stable-ai/

arXiv:2609.17488v1 [cs.AI] 15 Sep 2026

Abstract We introduce LimiX-2, a new model in the LimiX family, developed through model and data scaling guided by our previously established scaling laws. LimiX-2 adopts the Contextual Mechanism Networks (CMNs) paradigm and is pretrained with ContextConditional Masked Modeling (CCMM). CMNs shifts the organizing principle of incontext learning from target-centric prediction to mechanism-oriented joint modeling. Rather than centering the network on the p(y | x, Dcontext ) objective of conventional tabular PFNs, it is designed around learning p( x, y | Dcontext ), a context-dependent representation of the joint structure underlying data generation. Pretraining uses synthetic datasets generated by structural causal models (SCMs) spanning diverse graph structures, functional mechanisms, and observation processes. Evaluations on TabArena, TALENT, and BCCO show that LimiX-2 outperforms current dataset-specific models and tabular foundation models. Beyond predictive performance, the CMN paradigm also promotes causal awareness in LimiX-2: its feature attention encodes direct causal relationships, enabling accurate causal skeleton recovery.

Figure 1: Performance overview on evaluated benchmarks. LimiX-2 achieves Elo scores of 1935, 1506, and 1432 on TabArena (Erickson et al., 2025), TALENT (Ye et al., 2024; Liu et al., 2024), and BCCO (LimiX Team, 2025), respectively, outperforming all compared foundation models and AutoGluon 1.6 (Erickson et al., 2020). For TabArena, only the best AutoGluon setting is shown. Bar ends and labels indicate Elo point estimates, while shaded regions with terminal markers show 95% bootstrap confidence intervals.

Contents 1

Introduction

2 Architecture 2.1 Embedding of Tabular Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.2 Discriminative Feature Encoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.3 Model Backbone Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2.4 Prediction Heads . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

4 4 4 5 7

Pretraining 3.1 Context-Conditional Masked Modeling for Joint Distribution Learning . . . . . . . . . . . 3.2 Mask Pattern Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3 Mask Embedding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

7 7 8 8

Pretraining Data Generation 4.1 Hyperparameter Sampling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2 Directed Acyclic Graph Generation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3 Functional Mechanisms of SCM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.4 Feature and Target Sampling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.5 Task Adaptation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

8 8 8 8 9 10

Evaluation 5.1 Predictive Performance Evaluation Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.2 Results on TabArena . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.3 Results on TALENT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.4 Results on BCCO . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5.5 Causal Skeleton Recovery Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

10 10 11 15 18 21

Scaling Law 6.1 Experimental Setup and Scaling Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6.2 Scaling Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

21 22 23

Conclusions

Contribution

Introduction

Progress toward general-purpose machine intelligence can be organized around three complementary frontiers: language, the physical world, and structured data (LimiX Team, 2025). Large-scale next-token pretraining and post-training have enabled large language models (LLMs) to follow instructions, use tools, and reason over text and visual inputs (Ouyang et al., 2022; Schick et al., 2023; Team et al., 2023; Guo et al., 2025). Embodied agents and world models pursue physical-world intelligence by learning to model and interact with environments (Ha & Schmidhuber, 2018; Kim et al., 2024). In contrast, general-purpose learning and reasoning over structured data remain comparatively underdeveloped. Structured data supports prediction and decision-making in healthcare (Johnson et al., 2016), finance (Gu et al., 2020), and scientific discovery (Baldi et al., 2014). For tabular prediction, gradient-boosted trees (Chen & Guestrin, 2016; Ke et al., 2017; Dorogush et al., 2018), deep neural networks (Gorishniy et al., 2021; 2024), and automated ensemble pipelines (Erickson et al., 2020) have achieved strong task-specific performance. However, these methods typically require separate training and model selection for each dataset, with limited reuse of knowledge across tasks. This motivates foundation models that learn from diverse datasets through pretraining and transfer to new prediction tasks. Motivated by this gap, we advocate the development of large structured-data models (LDMs)1 : large-scale pretrained models for inference over structured data. We characterize LDMs by three properties: (i) pretraining on large-scale data that covers a wide distribution of tasks; (ii) a unified modeling paradigm over structured variables without task-specific design; and (iii) the ability to perform new tasks at inference time without any model updates. Classification and regression are the canonical instances of tabular data inference, yet they do not exhaust the choice of such tasks. One line of work pretrains tabular predictors on synthetic tasks using Prior-Data Fitted Networks (PFNs) (Müller et al., 2021; Hollmann et al., 2022; 2025). Subsequent models, including TabICL (Qu et al., 2025; 2026), TabFM (Kong & Das, 2026), and Mitra (Zhang et al., 2025; Tao et al., 2026), follow this approach of pretraining on synthetic tasks for supervised in-context prediction. In their standard supervised formulation, PFNs approximate p(y | x, Dcontext ), the posterior predictive distribution of a designated target y given query features x and a labeled context set (Hollmann et al., 2022). This enables prediction on new datasets through in-context learning without parameter updates. However, within each task, direct supervision is confined to predicting a single target column rather than explicitly modeling the joint distribution over all variables, which may significantly limit the ability on variant data reasoning tasks. We introduce Contextual Mechanism Networks (CMNs), a new paradigm for structured-data intelligence that shifts the modeling focus from a designated target to the system of predictive dependencies among variables. Unlike PFNs, CMNs learn from multiple conditional prediction tasks over the same dataset, using context to capture the dependencies shared across them and target the joint dependency structure of p(x, y | Dcontext ). Supervised prediction is thus a special case of a broader framework for inferring unobserved quantities from available evidence. This paradigm builds on the context-conditional modeling principles introduced in LimiX (LimiX Team, 2025). We instantiate CMNs in LimiX-2 and advance this design through model and data scaling. Pretraining uses Context-Conditional Masked Modeling (CCMM) (LimiX Team, 2025), which integrates target prediction and feature reconstruction under varied observation patterns. By organizing supervision across variables, CCMM makes inter-variable inference an explicit pretraining objective rather than an auxiliary capability. This provides a unified basis for supervised prediction, missing-value imputation, and broader conditional reasoning within a single pretrained model. LimiX-2 refines the Transformer-based architecture of LimiX (Vaswani et al., 2017; LimiX Team, 2025) while retaining cell-level representations. It is pretrained exclusively on synthetic datasets produced by an expanded generation engine based on structural causal models (SCMs) (Pearl et al., 2003). Compared with its predecessor, the engine spans a broader range of graph structures, functional mechanisms, and observation processes. Our evaluations demonstrate the effectiveness of the CMN paradigm: a single pretrained LimiX-2 model supports classification, regression, and missing-value imputation, as well as causal discovery, without task-specific parameter updates. We empirically evaluate the prediction performance of LimiX-2 model on three benchmarks widely adopted by the community of tabular machine learning: TabArena, TALENT and BCCO. These benchmarks span broad regimes of sample size, feature dimensionality, class number, categorical–to-numerical feature ratio, missingness and sample-to-feature ratios. The results demonstrate that our LimiX-2 model outperforms current models, including tabular foundation models and traditional models trained specifically on each dataset. Notably, LimiX-2 surpasses TabFM despite being 4 times smaller in model 1 Throughout this report, LDMs refers to large structured-data models.

parameter size. Furthermore, we also conduct causal skeleton recovery evaluations on several typical causal discovery datasets. The results demonstrate that LimiX-2 outperforms other tabular foundation models, tree-based feature importance methods and dedicated causal discovery methods, indicating that its feature attention encodes causal structural information.

Architecture

LimiX-2 continues the cell-level design of the previous generation (LimiX Team, 2025): it does not compress feature information at the row level, instead encodes each cell into separate representation2 , supporting conditional reasoning across variables. Local relations among features within a row can therefore be modeled directly, while dataset-level statistics can be derived from the corresponding cells in context. Contrastive to representations that collapse a whole row into a single vector, cell-level representations better preserve the fine-grained structure tabular data. On this basis, LimiX-2 instantiates cell-level modeling at a larger scale. Every cell must keep its own representation, and masked prediction further requires these vectors to yield consistent conditionals under different visibility patterns. The parameter budget is increased so that each cell has a richer representation space and fine-grained relations across columns and samples can be captured more stably. The extra capacity is not applied as a uniform width multiplier. It is allocated mainly to the subsequent task pathway, which organizes the evidence needed for prediction. 2.1

Embedding of Tabular Data

R Suppose a table has N rows and F columns, we denote raw cell in the i-th row and j-th column as xi,j R into feature representation space and raw targets of the i-th row as yiR . We firstly map each raw cell xi,j

xi,j ∈ Rd . In the previous version LimiX, we set d = 192 for LimiX-16M (LimiX Team, 2025) and d = 96 for LimiX-2M (Wang et al., 2026b). LimiX-2 extends the embedding dimension to d = 256. Missing cells share a single learnable embedding, while column identity is provided separately by discriminative feature encoding (DFE). ( xi,j =

Emiss , R ), Enum ( xi,j

R is missing, if xi,j otherwise.

Enum is a two-layer MLP with RMSNorm (Zhang & Sennrich, 2019) and GELU (Hendrycks & Gimpel, 2016). Emiss is one learnable vector shared by all missing cells, irrelevant to column index and type. Therefore, the entire feature representation tensor can be denoted as x ∈ R N × F×d . Targets are encoded as yi ∈ RKd (we set K = 4 in LimiX-2) according to task type: numerical regression Y , while categorical classification targets are mapped through targets are mapped through an encoder Enum Y . Context rows retain their observed labels, whereas the an orthogonally initialized embedding table Ecat target position of each query row is filled with a learnable MASK embedding. While LimiX uses the task embedding as a whole per sample, LimiX-2 splits the embedding into K task-embedding slots, each of dimension d. Formally, yi = yi,1 , . . . , yi,K , yi,k ∈ Rd . Furthermore, a task-type embedding Etype (τ ), where τ ∈ {cls, reg}, is added to each of the four task embeddings. 2.2

Discriminative Feature Encoding

In LimiX-2, each feature shares the same numerical MLP, so the distinct columns of similar marginal distribution become indistinguishable from their value representations alone. Therefore, an explicit column identity is therefore required. LimiX-2 adopts low-rank DFE to produce column identity representation. The j-th column is associated with an s-dimensional code u j ∈ Rs , where s = d/4 by default. A transformation matrix E ∈ Rs×d then maps the codes into the embedding space, and the mapped codes (i.e. column identity embedding) are added with feature representation. 2 For brevity, the terms representation and embedding are used interchangeably throughout this report and refer to the same concept.

The column identity embedding e j distinguish the columns without encoding sequential proximity: when columns are permuted together with their codes, attention should not depend on an accidental order. The low rank confines column identity to a compact subspace so that statistical strength can be shared across columns. Since the representation dimension is set as a larger value d = 256 than previous generation of LimiX, it becomes easier to memorize column-index shortcuts. Hence, compressing column identity into an s-dimensional code constrains the model recognizing columns rather than positions. 2.3

Model Backbone Architecture

The backbone remains a stack of dual-axis transformer blocks: the feature-axis blocks models variable relations within rows, and the sample-axis blocks use context samples to form a prior for the current table. In LimiX, feature-axis and sample-axis attention and the Feed-Forward Network (FFN) were shared within a block (LimiX Team, 2025). LimiX-2 keeps this two-dimensional factorization, but no longer routes feature and task representations through the same computation path. The model architecture is (l )

shown in Figure 2. We denote xi,j and y(l ) as the output representation of the l-th dual-axis transformer (0)

(0)

block. Specifically, xi,j and yi (0)

are the original representation of the initial embedding components. (0)

Formally, we have xi,j = xi,j and yi

= yi .

Sample-axis attention. For each dual-axis transformer block, the representations produced by the previous block are firstly fed into sample-axis attention components. The sample-axis attention components propagate information across samples on each feature position as well as target position. For the target position, LimiX-2 firstly concatenates the task embeddings into a unified one before feeding into attention process, ( l −1)

( l −1)

= (yi,1

( l −1)

, . . . , yi,K

) ∈ RKd .

In the attention component, context rows are visible to each other, while query rows can only attend to context. The query/key/value mapping functions are shared among features, but not between features and target. A query prediction therefore depends only on the sample’s own features and the context, not on which other test samples share the batch. After the calculation, the target representaions are then split back into K embeddings. The immediate representation of features and target produced by the l th (l )

(l )

sample-axis attention are denoted as xei,j and yei respectively. Asymmetric feature-axis attention. Feature representations may attend to the target and other feature representations, while target representations can only attend to features representations. Formally we have, x(l ) = Attnfeat Q X (e x(l ) ), KX ([e x( l ) , y e(l ) ]), VX ([e x( l ) , y e(l ) ]) , X feat y(l ) = AttnY QY ( y e(l ) ), KY (e x(l ) ), VY (e x( l ) ) , where Attnfeat X/Y (·, ·, ·) is multi-head attention components for feature/target representations and Q X/Y (·), KX/Y (·) and VX/Y (·) are query mapping function, key mapping function and value mapping function of feature and target representation attention respectively. The Q/K/V mapping functions for feature and target representation attention are not shared, so the model implicitly distinguish their roles inside a mixed stream. Independent SwiGLU. The shared MLP is replaced by a gated FFN (Shazeer, 2020), instantiated separately for feature and target representations. Formally, SwiGLU(z) = Wo SiLU(Wg z + bg ) ⊙ (Wv z + bv ) + bo , where Wg/v and bg/v/o are learnable projection matrices and bias parameters. The FFNs of feature representations operates in the space of R p , and the FFNs of target representations operates on the concatenated slot space of RKd .

Feature Reconstruction Head

Classiﬁcation Head

p(y|x)

N×F

(reconstruction feautres )

p(y|x)

(class logits)

Shallow Feature Embeddings

Regression Head

(5000-bin logits)

Final-Layer Task Embeddings

N×F×d

N×K×d

Transformer Block Sample-axis attention

Feature-axis attention X SwiGLU FFN

Y SwiGLU FFN

K Target Embedding per row ﬂatten K×d

concat

Cell Embedding

K x Target Embedding

N×F×d

Mask

N×K×d

MLP

Mask Embedding

REG Target Encoder

Target Embedding

Feature Embedding MLP

CLS Target Embedding

Feature Encoder

𝑥₁₁

𝑥₁₂

...

𝑥₁m

y₁₁

...

𝑥₂₁

Inf

...

y₂₁

NaN

N×F

N×1

Figure 2: Overall structure of LimiX-2. Features are encoded by a MLP-based encoder, and missing cell is imputed by a shared learnable vector. Targets are encoded into K = 4 embedding slots. Each block successively applies sample-axis attention and SwiGLU on the feature and target representations, followed by asymmetric feature-axis attention. Feature embeddings produced at the shallow depth are used for masked-feature reconstruction. Final-layer task embeddings are used for both classification and regression tasks. Multi-head Attention and length stability of multi-head attention. In LimiX, cross-attention used only the one key/value head. In contrast, LimiX-2 applies all the K/V heads. Before attention scores are computed, Q and K are normalized to control the magnitude of the attention logits in deep stacks. Queries are then rescaled per head by a length-dependent factor sh = (1 + wh log n) β h , where n is the current sequence length and wh and β h are learnable and softly truncated by the tanh 6

function. This is conceptually related to length-aware softmax scaling for variable context lengths (Chiang & Cholak, 2022; Nakanishi, 2025). All sublayers use pre-normalized RMSNorm (Zhang & Sennrich, 2019). The computation order inside a block is as follows: independent X/Y sample-axis attention, independent SwiGLU, asymmetric feature-axis attention, and residual connections. The blocks are stacked M = 24 layers deep. 2.4

Prediction Heads

In LimiX-2, the prediction heads of different tasks (i.e. classification, regression and masked-feature reconstruction) are attached to the output representation of different depths. Masked-feature reconstruction needs local details of data and the corresponding prediction head is attached to the shallow depth representations x(lmask ) (lmask < M). In contrast, classification and regression tasks are decoded from the representations of the last layer y( M) . Each head is preceded by an independent bottleneck post-adapter (Post Adapter): Amask , Acls , or Areg . For classification and regression, the post-adapter is applied to each of the K target embeddings slots, which are then concatenated into the space of RKd . For C-way classification, the head emits logits in RC and is trained with cross-entropy. Learning objective of regression does not adopt mean square error (MSE) loss as in LimiX. Instead, LimiX-2 partitions the target range into B = 5000 ordered bins, predicts probability of each bin p ∈ ∆ B−1 , and derive the regression value as following B

ŷ = ∑ pi ci , i =1

where ci is the center value of the ith bin.

Pretraining

3.1

Context-Conditional Masked Modeling for Joint Distribution Learning

Pretraining aims to capture the joint dependency structure of table variables through conditional prediction under varied observation patterns. Following LimiX (LimiX Team, 2025), LimiX-2 adopts ContextConditional Masked Modeling (CCMM), which combines target prediction with masked-feature reconstruction. Whereas standard supervised objectives in PFNs including TabPFN and TabICL concentrate on p(y | x, Dct ) within each task (Hollmann et al., 2025; Qu et al., 2025), CCMM extends direct supervision across variables and conditioning sets. Each pretraining episode partitions a table into disjoint context and query row sets, Ict and Ite . The context Dct = (Xct , yct ) retains available observations, providing evidence about the table’s marginal distributions and inter-variable dependencies. For each query row i ∈ Ite , let πi index its masked feature columns. The model estimates qθ xi,j | xi,−πi , Xct , yct ,

i ∈ Ite , j ∈ πi ,

where j ranges over the masked columns of row i, and xi,−πi denotes the observed query features. The query target yi is predicted from the same conditioning information through the task heads. Along the sample axis, query rows attend only to context rows, and context representations are computed without access to queries. This prevents both direct and context-mediated information exchange between query rows. For fixed input representations and context, predictions are therefore invariant to query-batch composition. The same conditional interface supports classification, regression, and masked-feature reconstruction without task-specific parameter updates. Conditional likelihoods can additionally be used to score potentially anomalous entries. LimiX-2 retains CCMM while scaling an architecture with separate feature and task pathways. The feature pathway models inter-variable dependencies through cell-level representations, while the task pathway uses K = 4 embeddings to aggregate prediction-relevant information. Asymmetric attention directs information from feature representations to the task readout. We scale the backbone and widen the task FFN to provide additional capacity for conditional prediction over wider tables, longer contexts, and more diverse observation patterns.

3.2

Mask Pattern Design

A fixed masking pattern restricts the range of conditional prediction tasks encountered during pretraining. LimiX-2 therefore combines three masking schemes to vary the granularity of prediction targets and the available conditioning information. Masks are applied to individual entries, selected columns across query rows, or blocks of entries, exposing the model to prediction tasks at different granularities. The resulting tasks range from recovering isolated values to predicting target columns and reconstructing groups of missing entries, all conditioned on the remaining observations and the context set. Interleaving these schemes across episodes broadens the coverage of observation patterns and discourages specialization to a single reconstruction setting. 3.3

Mask Embedding

For each cell masked during pretraining, the value embedding is replaced by the shared missing-value embedding Emiss defined in Section 2.1 and added to the DFE column code e j . Masked and naturally missing cells thus share a missingness encoding while retaining column identity. The resulting representations pass through the same dual-axis attention layers as observed-cell embeddings, allowing the model to integrate evidence across columns and context samples and produce distributional predictions through the corresponding output heads.

Pretraining Data Generation

We construct large-scale pretraining data following the SCM framework (Pearl et al., 2003). By varying the components at different stages of the data-generation process, we synthesize a large number of datasets with diverse variable dependencies, feature distributions, and task properties. Inherited from the previous version of LimiX (LimiX Team, 2025), the overall data-generation pipeline consists of five main stages-hyperparameter sampling, directed acyclic graph (DAG) generation, SCM propagation, data sampling, and task adaptation-as illustrated in Figure 3. Building upon this pipeline, LimiX-2 further expands the space of graph structures, functional mechanisms, and variable observation processes, thereby increasing the structural and statistical diversity of the pretraining tasks. 4.1

Hyperparameter Sampling

For each pretraining dataset, we sample a set of hyperparameters that characterize its global properties, including the sample size, the feature dimension (specified separately as the numbers of continuous and categorical features), and the task type (i.e., classification or regression). Given the sampled sample size, we then randomly draw an evaluation position that splits the dataset into a context part and a query part. The sampling distribution of each hyperparameter is randomly chosen from a family of distributions, such as the normal, uniform, and beta distributions. 4.2

Directed Acyclic Graph Generation

We generate DAGs that depict the structural dependencies among variables in a hierarchical manner. The overall DAG is composed of multiple local causal structures (LCSs), which is specified as causal motifs (Barjašić et al., 2021) in this practice. Each causal motif may contain multiple input and output nodes and encodes directed dependencies among variables, such as chain, confounding, and collider structures (Peters et al., 2017; Pearl et al., 2016). Through recursive expansion of causal motifs at multiple granularities, the induced DAG can simultaneously capture macro- and micro-level dependencies with complex local topologies. In addition, we allow topology-constrained graph transformation (Maslov & Sneppen, 2002; Sanfeliu & Fu, 1983) on the DAG, where operations such as edge redirection, local path replacement, and node-level structural transformations are randomly selected and applied. These operations further enrich the local topology while preserving the acyclicity of the graph, allowing the resulting DAGs to exhibit diverse connectivity patterns and information-propagation pathways. 4.3

Functional Mechanisms of SCM

While the DAG of an SCM determines the dependencies among variables, the functional mechanisms determine the statistical relationships underlying these dependencies.

Step 1 Hyperparameter Sampling n=2

n=3

n=4

Node Number

Step 2

Step 3

DAG Generation

SCM Propagation

Causal Motifs

n=...

...

......

Feature Dimension

Sample Size

...

Root Initial

Context

Eval Pos

Task Type

...

…

Classification

Edge Function MLP

Query

Regression

Tree

Directed Acyclic Graph

Step 4

Step 5

Feature & Target Sampling

Task Adaptation

Task Descriptor graph-structure feature-redundancy

Classification

2 Class Balance

Discretization

Aggregation CNN

• weighted sum • logsumexp • average • max • neural network

Multi Class Imbalance

Frequency Param

Regression Original Value

Numerical Transformation

Figure 3: Schematic overview of the synthetic data generation process for pretraining. The pipeline consists of five phases, that are hyperparameter sampling, directed acyclic graph (DAG) generation, SCM propagation, data sampling and task adaptation respectively. For each DAG, we first sample the values of root nodes from distributions with randomly chosen types and parameters. The values of the remaining nodes are then determined by propagating functional computations along the topological order of the DAG. For a node Xi , the value-generation process can be expressed as Xi = f i { gi,j ( X j )} j∈PA(Xi ) , ϵi , where PA( Xi ) denotes the set of parent nodes of Xi , gi,j is the edge function associated with the parent node X j , f i denotes the aggregation function that combines the mapped parent values, and ϵi denotes stochastic noise. LimiX-2 retains the three main types of edge functions from the previous version, including MLPs, CNNs, and decision trees (LimiX Team, 2025). Furthermore, we incorporate additional mechanisms such as linear mappings, kernel functions, piecewise functions, periodic functions, and multiplicative interactions. Moreover, these basic functions can be composed, enabling the SCM to depict more complex variable relationships. For nodes with multiple parents, we design several aggregation strategies, e.g., simple averaging, weighted aggregation, and neural aggregation, producing diverse forms of multivariate interactions. 4.4

Feature and Target Sampling

A complete SCM characterizes the joint states of all variables, whereas in practice only a subset of them is typically observable. To make the generated data resemble real-world scenarios, we retrieve only a subset of variables as sample features and prediction targets to construct each dataset. Specifically, LimiX-2 formulates variable sampling as a multi-attribute selection problem. For each dataset, coarse-grained 9

task descriptors are constructed based on the assigned design with respect to its subgraph structure and feature redundancy. Candidate tasks are then filtered through a multi-objective selection mechanism, ensuring that the resulting tasks not only differ in graph structure but also cover prediction problems with diverse statistical properties, thereby broadening the coverage of the final pretraining tasks in both structural complexity and statistical characteristics. 4.5

Task Adaptation

We apply stochastic observation transformations to the features and target variables for task adaptation. Examples of such transformations include linear scaling, monotonic nonlinear transformations, logarithmic transformations, exponential transformations, and multiple operators can be randomly composed. For the target variable, the transformation is specified according to the task type of each dataset (i.e., classification or regression). For classification tasks, since the initial target variable is generally continuous, we convert it into a categorical target via random discretization: LimiX-2 randomly partitions the value space of the target into several intervals and varies the class frequencies and discretization parameters, allowing different synthetic datasets to yield classification tasks with varying numbers of classes and degrees of class imbalance. For regression tasks, targets may undergo random scale transformations as well as adjustments to their skewness and tail behavior, thereby covering continuous prediction tasks with diverse functional relationships.

Evaluation

In this section, we conduct a comprehensive predictive performance evaluation of LimiX-2 on several public tabular benchmarks, which encompass a diverse collection of real-world classification and regression tasks. As shown in Figure 1, LimiX-2 achieves the highest Elo rating on all three benchmarks. We further evaluate causal skeleton recovery with LimiX-2 on causal discovery benchmarks. The results show that the recovered skeletons based on LimiX-2 feature attention surpass those of other tabular foundation models, XGBoost-based feature importance methods, and dedicated causal discovery methods. 5.1

Predictive Performance Evaluation Setup

Benchmarks. Three widely adopted benchmarks, TabArena (Erickson et al., 2025), TALENT (Ye et al., 2024; Liu et al., 2024), and BCCO (LimiX Team, 2025), are used for evaluation. TabArena focuses on evaluating predictive performance under practical protocols, TALENT evaluates generalization across diverse task types, and BCCO evaluates robustness under challenging and incomplete datasets. These benchmarks provide a broad assessment of LimiX-2 in terms of predictive performance, generalization, scalability, and robustness. • TabArena is an actively maintained benchmark designed to evaluate the practical predictive performance of tabular learning methods. It comprises 51 manually curated real-world tabular datasets, covering a diverse range of classification and regression tasks. • TALENT is a large-scale and systematic benchmark for studying tabular learning methods across a broad spectrum of datasets. It consists of 300 datasets, including 120 binary classification datasets, 80 multiclass classification datasets, and 100 regression datasets. Excluding 12 classification datasets with more than 10 target classes, we perform evaluation on the remaining 288 datasets. • BCCO provides a robustness-focused benchmark for tabular models, designed to evaluate model performance on challenging and imperfect data. It contains 106 classification datasets in BCCOCLS and 50 regression datasets in BCCO-REG, focusing on scenarios involving missing and incomplete features. Benchmark Protocol. Table 1 summarizes the composition of task types in the tabular benchmarks and the main evaluation metrics reported in this technical report. The detailed evaluation protocols for TabArena, TALENT, and BCCO are described as follows. • TabArena Protocol: For TabArena, we fully follow the benchmark’s evaluation configuration, including its dataset collection, data splits, metrics, and result aggregation procedure. We take the published leaderboard scores as the reference for existing baselines (accessed September 15, 2026) and then report the Elo rating computed by the official TabArena evaluation pipeline, ensuring direct comparability with the leaderboard results.

Table 1: Task composition and evaluation metrics of the 3 tabular benchmarks. Twelve TALENT datasets with more than 10 target classes are excluded. Benchmark

# Datasets

Task Type

# Tasks

TabArena

Binary CLS Multiclass CLS REG

30 8 13

TALENT

288

Binary CLS Multiclass CLS REG

120 68 100

BCCO

156

Binary CLS Multiclass CLS REG

71 35 50

Main Metrics

Elo, Rank, Win Rate

• TALENT and BCCO Protocol: For TALENT and BCCO, we follow the official evaluation pipeline provided by TALENT, conducting each experiment with 15 random seeds. For TALENT, we adopt the official fixed 64%/16%/20% train/validation/test split. For BCCO, we randomly hold out 20% of the training set for validation, leaving the official test set unchanged. For models integrated into the TALENT library, we use their default configurations, including model-specific hyperparameter settings. Regarding tabular foundation models such as EXAONE Tabular (Eo et al., 2026), Xiaomi-TabLDM (Wang et al., 2026a), and Mitra-v2 (Tao et al., 2026), we implement the evaluation following the preprocessing and inference procedures specified in their respective official repositories, without tuning the hyperparameters for fair comparisons. For each dataset, models are ranked by accuracy for classification tasks and by RMSE for regression tasks. Ranking is performed for each random seed, with ranks first averaged across seeds and then equally across datasets to obtain the mean rank. In addition to ranks, we aggregate model performance by converting per-dataset results into pairwise comparisons and fitting a Bradley–Terry Elo model, following TabArena, with Random Forest anchored at 1000. For reported Elo, we show 95% confidence intervals from 2000 bootstrap rounds, using the 2.5% and 97.5% quantiles. Baselines. We compare LimiX-2 against a range of state-of-the-art baseline models, including tree-based models, auto-ML frameworks, neural networks, and recent tabular foundation models. • Tree-based models. We include XGBoost (Chen & Guestrin, 2016), LightGBM (Ke et al., 2017), CatBoost (Dorogush et al., 2018), and Random Forest (Breiman, 2001). • Auto-ML frameworks. We compare LimiX-2 against AutoGluon (Erickson et al., 2020), an automated machine learning framework that performs model selection, hyperparameter optimization, and ensemble learning for tabular data. • Neural networks. We evaluate LimiX-2 against neural baselines, including FT-Transformer (Gorishniy et al., 2024), RealMLP (Holzmüller et al., 2024), TabR (Gorishniy et al., 2024), TabM (Gorishniy et al., 2025), and ModernNCA (Ye et al., 2025). • Tabular foundation models. We compare LimiX-2 with recent tabular foundation models, including Mitra-v2 (Tao et al., 2026), TabFM (Kong & Das, 2026), TabPFN-3 (Grinsztajn et al., 2026), TabICLv2 (Qu et al., 2026), TabDPT (Ma et al., 2026), EXAONE Tabular (Eo et al., 2026), Xiaomi-TabLDM (Wang et al., 2026a) and LimiX-16M (LimiX Team, 2025). These models cover a range of recent approaches to pretraining and in-context learning for tabular data. 5.2

Results on TabArena

Overall performance. Figure 4 and Table 2 report performance on the full TabArena benchmark. Across all four predictive metrics reported in the table, LimiX-2 ranks first among the compared methods. It attains the highest Elo score of 1935, exceeding the Elo runner-up, TabFM+, by 117.4 points before rounding, and yields an improvability of 3.3% versus 6.2% for TabFM+. Consistently, LimiX-2 achieves an average rank of 5.5, compared with 9.0 for TabFM+, and records an aggregated win count of 18.9 against 5.3 for TabFM+, corresponding to approximately 3.6 times as many aggregated wins. Win rate averages pairwise comparisons against all other methods in the corresponding table, first across evaluation splits within each dataset and then equally across datasets, with ties counted as half a win. 11

Default

2000

Tuned

Tuned + Ensembled

System / Portfolio

Elo

1800 1600 1400 1200 1000

t t NCA tBoos htGBM oos XGB Modern Ca Lig

DPT ealMLP bICLv2 FN-2.5 FN-2.6 abLDM bPFN-3 EX, 4h) abular itra-v2 TabFM C, 4h) T Tab N P P T M R Ta Ta .6 ( NE .6 ( Tab iaomilTab on 1 EXAO on 1 X Rea u u l l g G o o Aut Aut

Cau

silo

M+

F Tab

iX-2

Lim

Figure 4: Performance on the TabArena benchmark. Baseline results are reported under the default, tuned, and tuned-plus-ensembled configurations. LimiX-2 with default configuration achieves an Elo score of 1935, outperforming all compared foundation models, including TabFM, TabPFN-3, and EXAONE Tabular. LimiX-2 also surpasses AutoGluon under its noncommercial 4h configuration. The aggregated win count instead assigns one unit of credit per split to the best-performing method, shares it equally among tied winners, averages within each dataset, and sums across datasets. Elo point estimates are rounded to integers in the tables. Elo differences and relative comparisons are computed from the unrounded results. Classification performance. Table 3 reports results on the 38 classification datasets. LimiX-2 ranks first across all four predictive metrics reported in the table. It attains an Elo of 1917, an improvability of 4.3%, an average rank of 6.0, an aggregated win count of 10.6, and an average pairwise win rate of 94.5%. These results indicate strong aggregate performance across the classification tasks. Regression performance. Table 4 reports results on the 13 regression datasets. LimiX-2 again ranks first across all four predictive metrics reported in the table, achieving an Elo of 2206, an improvability of 0.6%, an average rank of 3.8, an aggregated win count of 8.3, and an average pairwise win rate of 96.9%. TabFM+ is the Elo runner-up at 2063, followed by AutoGluon 1.6 (noncommercial, 4h) at 2060. These results indicate strong performance on the regression subset. Table 2: Performance on the TabArena benchmark. Models are ranked by Elo, improvability, average rank, and aggregated win count are also reported. LimiX-2 ranks first on all four metrics, with an Elo of 1935 (117.4 points above TabFM+ before rounding). Compared with TabFM+, its improvability is 3.3% versus 6.2%, its average rank is 5.5 versus 9.0, and its aggregated win count is 18.9 versus 5.3. Model

Elo ↑

LimiX-2 (D) 1935−77,+111 TabFM+ 1818−84,+109 Causilo (D) 1790−57,+92 AutoGluon 1.6 (NC, 4h) 1789−60,+99 TabFM (D) 1774−98,+101 Mitra-v2 (D) 1769−74,+102 EXAONE Tabular (D) 1749−55,+72 AutoGluon 1.6 (EX, 4h) 1738−54,+85 AutoGluon 1.5 (EX, 4h) 1648−60,+69 TabPFN-3 (D) 1632−49,+69 Xiaomi-TabLDM (D) 1586−59,+65 TabPFN-2.6 (D) 1580−43,+59 RealTabPFN-2.5 (T+E) 1564−48,+59 TabICLv2 (D) 1563−54,+62 RealTabPFN-2.5 (T) 1528−44,+54 RealTabPFN-2.5 (D) 1502−40,+50 AutoGluon 1.4 (best, 4h) 1479−46,+44 RealMLP (T+E) 1475−42,+45 TabDPT-Turbo (D) 1433−47,+50 TabDPT (T+E) 1431−44,+56 TabM (T+E) 1422−37,+45 LightGBM (T+E) 1406−27,+30

Improv- Avg. rank #wins ability ↓ ↓ ↑

Model

3.3% 6.2% 8.9% 8.6% 6.5% 8.3% 9.5% 9.2% 10.2% 11.6% 12.1% 12.9% 12.6% 12.4% 13.3% 13.8% 14.8% 14.9% 16.0% 15.8% 16.1% 17.0%

TabM (D) iLTM (T) BetaTabPFN (D) TabPFNv2 (T) SAP-RPT-OSS (D) TorchMLP (T+E) EBM (T+E) TabPFNv2 (D) ModernNCA (D) EBM (T) RealMLP (D) XGBoost (D) ExtraTrees (T+E) TorchMLP (T) FastaiMLP (T+E) EBM (D) APLR (T+E) LightGBM (D) ExtraTrees (T) CTBoost (D) RandomForest (T+E) Nori-30M (D)

5.5 9.0 10.1 10.1 10.7 10.9 11.8 12.3 16.9 17.9 20.8 21.2 22.3 22.4 25.0 27.1 28.9 29.3 33.0 33.1 34.0 35.5

18.9 5.3 1.7 1.2 5.9 3.2 2.9 1.2 1.3 0.4 0.5 0.1 0.1 0.1 0.1 0.0 0.0 0.2 0.2 0.4 0.0 0.0

Elo ↑ 1283−42,+43 1280−32,+35 1272−57,+58 1272−57,+57 1268−58,+57 1268−45,+46 1253−36,+36 1245−63,+65 1238−40,+37 1220−40,+44 1217−38,+32 1205−36,+34 1201−43,+46 1200−42,+41 1192−59,+55 1192−48,+46 1186−57,+41 1180−30,+32 1177−53,+43 1171−66,+60 1169−44,+53 1156−82,+73

Improv- Avg. rank #wins ability ↓ ↓ ↑ 18.9% 19.0% 20.3% 20.0% 20.2% 19.0% 20.5% 20.8% 21.0% 21.1% 20.4% 20.6% 21.5% 20.7% 21.4% 22.0% 22.1% 21.1% 22.3% 23.0% 22.5% 24.8%

47.7 47.9 48.8 48.8 49.1 49.1 50.7 51.5 52.2 54.0 54.3 55.6 55.9 56.1 56.8 56.8 57.4 58.0 58.3 58.9 59.1 60.4

0.1 0.1 0.0 0.1 0.6 0.0 0.0 0.3 0.2 0.0 0.1 0.0 0.0 0.0 0.0 0.1 0.0 0.0 0.2 0.1 0.1 0.2

Continued on next page

Table 2 (continued) Model RealMLP (T) CatBoost (T+E) CatBoost (T) TabDPT (T) iLTM (T+E) TabM (T) ModernNCA (T+E) LightGBM (T) ChimeraBoost (T+E) CatBoost (D) XGBoost (T+E) LimiX-16M (D) ModernNCA (T) ChimeraBoost (T) TabSwift (D) XGBoost (T) xRFM (T+E) TabPFNv2 (T+E) CTBoost (T+E) Mitra (D) TabDPT (D) TabICL (D) ChimeraBoost (D) CTBoost (T) xRFM (T)

Elo ↑ 1402−46,+39 1396−32,+37 1384−33,+33 1380−52,+53 1379−38,+39 1371−37,+43 1364−52,+66 1363−27,+24 1360−54,+45 1357−38,+38 1354−31,+30 1345−57,+72 1336−37,+36 1333−51,+43 1332−49,+59 1330−31,+29 1330−44,+44 1325−66,+64 1324−43,+39 1315−64,+63 1308−53,+62 1306−57,+49 1299−55,+42 1294−44,+37 1286−43,+42

Improv- Avg. rank #wins ability ↓ ↓ ↑

Model

16.2% 16.6% 16.8% 16.9% 17.2% 16.9% 17.5% 17.6% 17.9% 17.4% 17.7% 17.5% 17.9% 18.3% 18.0% 18.0% 18.2% 18.6% 18.6% 19.1% 19.1% 19.0% 19.1% 19.1% 19.3%

Nori (D) 1145−77,+70 APLR (T) 1145−56,+45 FastaiMLP (T) 1137−57,+53 RandomForest (T) 1134−45,+47 TabSTAR (T) 1088−83,+75 iLTM (D) 1088−57,+47 TabSTAR (T+E) 1087−83,+79 PerpetualBooster (T+E) 1086−45,+46 OrionMSP (D) 1086−52,+53 APLR (D) 1079−80,+51 TorchMLP (D) 1071−49,+38 PerpetualBooster (T) 1052−46,+47 xRFM (D) 1039−69,+55 TabFlex (D) 1010−74,+65 ExtraTrees (D) 1008−61,+48 FastaiMLP (D) 1005−61,+60 RandomForest (D) 1000−47,+43 KNN (T+E) 991−81,+62 TabSTAR (D) 989−101,+89 Linear (T+E) 956−103,+61 PerpetualBooster (D) 934−61,+41 Linear (T) 932−110,+67 KNN (T) 887−102,+61 Linear (D) 858−117,+68 KNN (D) 646−100,+79

35.8 36.4 37.5 37.9 38.0 38.8 39.5 39.6 39.8 40.2 40.5 41.4 42.2 42.5 42.7 42.8 42.9 43.3 43.5 44.4 45.1 45.3 46.0 46.5 47.3

0.0 0.1 0.0 0.1 0.0 0.1 0.1 0.0 0.8 0.1 0.0 0.3 0.3 0.3 0.0 0.0 0.1 0.0 0.0 0.1 0.1 0.0 0.0 0.0 0.1

Elo ↑

Improv- Avg. rank #wins ability ↓ ↓ ↑ 25.0% 23.2% 22.8% 23.2% 27.1% 24.7% 27.2% 28.0% 25.3% 26.1% 24.5% 29.2% 28.0% 29.2% 27.7% 27.3% 27.8% 29.3% 31.9% 35.1% 33.1% 35.6% 34.1% 38.1% 47.0%

61.4 61.4 62.2 62.5 66.8 66.8 66.8 66.9 66.9 67.5 68.2 69.9 70.9 73.2 73.4 73.6 73.9 74.6 74.7 77.1 78.5 78.6 81.3 82.8 90.2

0.0 0.0 0.0 0.2 0.4 0.1 0.5 0.0 0.0 0.0 0.0 0.0 0.0 0.1 0.0 0.1 0.0 0.1 0.3 0.0 0.1 0.0 0.1 0.1 0.0

Table 3: Performance on classification tasks in the TabArena benchmark. Models are ranked by Elo, improvability, average rank, and aggregated win count are also reported. LimiX-2 ranks first on all four metrics, with an Elo of 1917 (121.1 points above TabFM+ before rounding). Its improvability is 4.3%, approximately 42.7% lower than that of TabFM+ (7.5%). Its average rank is 6.0 versus 9.9, and its aggregated win count is 10.6 versus 4.7. Model

Elo ↑

LimiX-2 (D) 1917−77,+127 TabFM+ 1796−86,+121 Causilo (D) 1768−70,+90 AutoGluon 1.6 (NC, 4h) 1761−53,+88 TabFM (D) 1760−114,+125 Mitra-v2 (D) 1754−85,+126 EXAONE Tabular (D) 1754−57,+85 AutoGluon 1.6 (EX, 4h) 1711−43,+62 AutoGluon 1.5 (EX, 4h) 1652−71,+81 TabPFN-3 (D) 1627−65,+76 Xiaomi-TabLDM (D) 1577−56,+70 TabPFN-2.6 (D) 1576−55,+59 TabICLv2 (D) 1569−63,+70 RealTabPFN-2.5 (T+E) 1557−66,+64 RealTabPFN-2.5 (T) 1537−58,+67 RealTabPFN-2.5 (D) 1525−52,+52 AutoGluon 1.4 (best, 4h) 1481−51,+64 RealMLP (T+E) 1463−38,+52 TabM (T+E) 1442−42,+53 TabDPT-Turbo (D) 1419−56,+71 LightGBM (T+E) 1412−30,+46 TabICL (D) 1406−54,+49 CatBoost (T+E) 1397−44,+53 LimiX-16M (D) 1396−73,+83 TabM (T) 1395−42,+49 TabDPT (T+E) 1393−48,+70 RealMLP (T) 1393−43,+51 iLTM (T+E) 1388−41,+45

Improv- Avg. rank #wins ability ↓ ↓ ↑

Model

4.3% 7.5% 11.0% 10.5% 7.5% 10.1% 11.1% 11.2% 11.7% 14.0% 14.5% 15.2% 14.5% 15.1% 15.7% 16.0% 17.0% 17.6% 18.5% 19.0% 19.2% 19.5% 18.9% 19.3% 19.3% 19.1% 19.0% 19.3%

TabPFNv2 (T) TabM (D) EBM (T+E) TorchMLP (T+E) SAP-RPT-OSS (D) iLTM (T) TabPFNv2 (D) xRFM (T) EBM (T) TabDPT (D) FastaiMLP (T+E) EBM (D) ModernNCA (D) APLR (T+E) CTBoost (D) RealMLP (D) XGBoost (D) TorchMLP (T) ExtraTrees (T+E) APLR (T) FastaiMLP (T) LightGBM (D) ExtraTrees (T) RandomForest (T+E) iLTM (D) RandomForest (T) APLR (D) TabSTAR (T)

6.0 9.9 11.0 11.4 11.4 11.6 11.6 13.7 17.0 18.6 22.0 22.1 22.6 23.5 25.0 25.9 29.6 31.2 33.1 35.4 36.0 36.6 37.4 37.5 37.6 37.8 37.8 38.3

10.6 4.7 1.3 1.1 5.6 2.9 2.8 0.8 1.0 0.4 0.5 0.1 0.1 0.1 0.1 0.0 0.0 0.1 0.0 0.1 0.0 0.0 0.1 0.3 0.0 0.1 0.0 0.0

Elo ↑ 1304−77,+69 1304−43,+52 1301−36,+38 1297−39,+41 1291−71,+53 1285−38,+42 1281−82,+70 1280−62,+56 1268−39,+43 1263−58,+62 1250−67,+59 1246−49,+42 1244−42,+41 1239−59,+45 1233−65,+60 1232−38,+38 1222−46,+42 1220−38,+47 1206−53,+56 1195−62,+50 1188−68,+56 1184−42,+49 1180−61,+61 1180−66,+63 1146−56,+57 1146−68,+68 1141−67,+59 1135−95,+74

Improv- Avg. rank #wins ability ↓ ↓ ↑ 22.9% 21.5% 22.0% 21.2% 22.8% 21.7% 23.5% 22.4% 22.6% 23.0% 22.9% 23.6% 23.9% 23.7% 24.0% 23.1% 23.0% 23.1% 23.9% 24.9% 24.6% 23.9% 24.8% 24.9% 26.1% 25.7% 27.2% 27.9%

46.7 46.7 47.1 47.5 48.1 48.7 49.1 49.2 50.4 50.9 52.2 52.7 52.8 53.3 53.9 54.1 55.0 55.2 56.6 57.6 58.3 58.7 59.1 59.1 62.2 62.3 62.7 63.3

0.1 0.1 0.0 0.0 0.5 0.1 0.3 0.1 0.0 0.1 0.0 0.1 0.2 0.0 0.1 0.1 0.0 0.0 0.0 0.0 0.0 0.0 0.2 0.1 0.1 0.2 0.0 0.3

Continued on next page

Table 3 (continued) Model CatBoost (T) CatBoost (D) LightGBM (T) CTBoost (T+E) ChimeraBoost (T+E) XGBoost (T+E) BetaTabPFN (D) Mitra (D) TabPFNv2 (T+E) ModernNCA (T) ModernNCA (T+E) ChimeraBoost (T) CTBoost (T) TabDPT (T) TabSwift (D) XGBoost (T) xRFM (T+E) ChimeraBoost (D)

Elo ↑ 1388−41,+48 1374−44,+45 1372−31,+36 1371−56,+51 1370−56,+52 1361−46,+43 1360−47,+48 1352−79,+76 1352−72,+85 1349−37,+53 1347−70,+73 1344−53,+53 1341−52,+47 1339−50,+60 1337−57,+54 1336−38,+39 1316−48,+51 1315−65,+52

Improv- Avg. rank #wins ability ↓ ↓ ↑

Model

19.1% 19.5% 19.9% 20.1% 20.2% 20.0% 21.3% 21.5% 21.3% 20.3% 20.4% 20.7% 20.7% 20.4% 21.0% 20.4% 21.2% 21.5%

TabSTAR (T+E) 1131−93,+73 OrionMSP (D) 1117−66,+61 TorchMLP (D) 1087−54,+44 PerpetualBooster (T+E) 1066−62,+56 FastaiMLP (D) 1044−73,+69 Linear (T+E) 1038−89,+87 TabSTAR (D) 1034−117,+85 PerpetualBooster (T) 1032−58,+55 KNN (T+E) 1025−89,+74 TabFlex (D) 1014−103,+82 Linear (T) 1012−94,+87 xRFM (D) 1009−86,+68 RandomForest (D) 1000−66,+59 ExtraTrees (D) 992−71,+73 Linear (D) 937−108,+84 PerpetualBooster (D) 928−79,+58 KNN (T) 913−115,+84 KNN (D) 582−106,+110

38.3 39.7 39.9 39.9 40.0 40.9 41.0 41.9 41.9 42.1 42.4 42.7 43.0 43.2 43.3 43.4 45.5 45.6

0.0 0.1 0.0 0.0 0.0 0.0 0.0 0.1 0.0 0.3 0.0 0.0 0.0 0.1 0.0 0.0 0.1 0.0

Elo ↑

Improv- Avg. rank #wins ability ↓ ↓ ↑ 27.9% 28.0% 27.1% 32.4% 29.2% 33.9% 33.3% 33.8% 31.7% 33.1% 34.5% 32.3% 31.3% 31.3% 37.2% 37.8% 37.3% 52.3%

63.7 64.9 67.5 69.2 70.9 71.4 71.7 71.8 72.4 73.1 73.3 73.5 74.1 74.6 78.1 78.6 79.5 89.7

0.4 0.0 0.0 0.0 0.1 0.0 0.2 0.0 0.1 0.1 0.0 0.0 0.0 0.0 0.1 0.1 0.1 0.0

Table 4: Performance on regression tasks in the TabArena benchmark. Models are ranked by Elo, improvability, average rank, and aggregated win count are also reported. LimiX-2 ranks first on all four metrics, with an Elo of 2206, exceeding TabFM+ and AutoGluon 1.6 (noncommercial, 4h) by 143.2 and 146.3 points, respectively, before rounding. Compared with those two methods, respectively, its improvability is 0.6% versus 2.6% and 3.1%, its average rank is 3.8 versus 6.4 and 6.5, and its aggregated win count is 8.3 versus 0.5 and 0.1. Model

Elo ↑

LimiX-2 (D) 2206−188,+318 TabFM+ 2063−139,+212 AutoGluon 1.6 (NC, 4h) 2060−141,+193 Causilo (D) 2027−131,+237 AutoGluon 1.6 (EX, 4h) 2002−105,+165 TabFM (D) 1976−96,+176 Mitra-v2 (D) 1974−124,+225 EXAONE Tabular (D) 1870−90,+134 TabPFN-3 (D) 1782−124,+209 AutoGluon 1.5 (EX, 4h) 1765−77,+129 Xiaomi-TabLDM (D) 1759−172,+239 Nori-30M (D) 1750−81,+125 TabPFN-2.6 (D) 1719−47,+92 RealTabPFN-2.5 (T+E) 1719−87,+117 TabDPT (T+E) 1708−85,+155 Nori (D) 1668−78,+127 TabICLv2 (D) 1668−137,+225 TabDPT (T) 1657−65,+123 RealMLP (T+E) 1633−59,+104 RealTabPFN-2.5 (T) 1616−98,+132 TabDPT-Turbo (D) 1594−93,+188 AutoGluon 1.4 (best, 4h) 1581−80,+93 TabDPT (D) 1575−67,+139 RealMLP (T) 1537−66,+97 RealTabPFN-2.5 (D) 1529−100,+125 ModernNCA (T+E) 1524−117,+127 CatBoost (T+E) 1479−57,+89 LightGBM (T+E) 1472−71,+76 CatBoost (T) 1456−61,+91 xRFM (T+E) 1456−100,+105 TabM (T+E) 1441−76,+109 iLTM (T+E) 1428−50,+67 LightGBM (T) 1410−73,+79 ChimeraBoost (T+E) 1408−107,+108

Improv- Avg. rank #wins ability ↓ ↓ ↑

Model

0.6% 2.6% 3.1% 2.9% 3.5% 3.7% 3.0% 4.8% 4.5% 6.0% 4.9% 5.5% 6.0% 5.5% 6.3% 6.5% 6.0% 6.7% 7.0% 6.2% 7.2% 8.4% 7.6% 8.0% 7.6% 9.2% 9.9% 10.3% 10.0% 9.4% 8.9% 11.2% 10.8% 11.1%

TabM (D) 1274−94,+114 ModernNCA (D) 1268−73,+81 Mitra (D) 1264−112,+127 SAP-RPT-OSS (D) 1260−147,+158 LimiX-16M (D) 1247−156,+177 TabPFNv2 (T) 1233−129,+146 CTBoost (T+E) 1230−118,+81 TorchMLP (T+E) 1230−107,+103 ExtraTrees (T+E) 1221−101,+109 RealMLP (D) 1218−87,+100 LightGBM (D) 1203−40,+30 ExtraTrees (T) 1200−102,+103 CTBoost (T) 1191−120,+87 TabPFNv2 (D) 1184−145,+126 XGBoost (D) 1182−86,+87 TorchMLP (T) 1178−106,+87 PerpetualBooster (T+E) 1174−94,+62 RandomForest (T+E) 1150−78,+69 xRFM (D) 1145−112,+113 EBM (T+E) 1145−171,+126 PerpetualBooster (T) 1122−95,+50 RandomForest (T) 1114−91,+70 EBM (T) 1103−168,+125 ExtraTrees (D) 1056−118,+88 EBM (D) 1044−163,+116 APLR (T+E) 1029−117,+99 TorchMLP (D) 1027−133,+82 FastaiMLP (T+E) 1024−117,+101 RandomForest (D) 1000−78,+47 APLR (T) 988−137,+100 FastaiMLP (T) 980−118,+104 CTBoost (D) 974−156,+104 TabSTAR (T+E) 956−308,+230 TabSTAR (T) 943−316,+231

3.8 6.4 6.5 7.3 7.9 8.6 8.7 12.1 15.7 16.5 16.7 17.1 18.6 18.7 19.2 21.4 21.4 22.0 23.3 24.3 25.7 26.5 26.9 29.3 29.8 30.1 33.2 33.7 34.8 34.8 35.9 36.8 38.2 38.4

8.3 0.5 0.1 0.3 0.4 0.4 0.3 0.1 0.0 0.2 0.0 0.2 0.0 0.0 0.3 0.0 0.0 0.0 0.0 0.0 0.1 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.8

Elo ↑

Improv- Avg. rank #wins ability ↓ ↓ ↑ 11.3% 12.5% 12.1% 12.6% 12.2% 11.6% 14.4% 12.8% 14.7% 12.4% 13.2% 14.9% 14.7% 13.0% 13.8% 13.5% 14.9% 15.6% 15.5% 16.1% 15.9% 16.1% 16.7% 17.2% 17.5% 17.6% 16.8% 17.0% 17.6% 18.4% 17.5% 20.0% 24.9% 25.1%

48.7 49.2 49.5 49.8 50.8 51.9 52.2 52.2 52.9 53.1 54.2 54.5 55.2 55.7 55.9 56.2 56.5 58.3 58.7 58.7 60.4 60.9 61.7 64.9 65.7 66.6 66.8 67.0 68.5 69.2 69.7 70.0 71.1 71.9

0.0 0.0 0.0 0.1 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.1 0.1

Continued on next page

Table 4 (continued) Elo ↑

Model XGBoost (T+E) TabSwift (D) XGBoost (T) CatBoost (D) xRFM (T) ChimeraBoost (T) TabM (T) ModernNCA (T) iLTM (T) ChimeraBoost (D) TabPFNv2 (T+E)

Improv- Avg. rank #wins ability ↓ ↓ ↑

Model

10.8% 9.2% 10.9% 11.4% 10.0% 11.5% 9.7% 11.1% 11.1% 12.2% 10.6%

PerpetualBooster (D) KNN (T+E) iLTM (D) FastaiMLP (D) APLR (D) TabSTAR (D) KNN (T) KNN (D) Linear (T+E) Linear (T) Linear (D)

1401−39,+52 1392−108,+144 1381−47,+62 1377−85,+85 1375−85,+77 1374−108,+103 1370−89,+102 1365−89,+102 1326−59,+71 1317−108,+109 1316−119,+161

38.9 39.6 40.5 40.8 40.9 41.0 41.3 41.7 44.7 45.4 45.5

0.0 0.0 0.0 0.0 0.0 0.2 0.0 0.0 0.0 0.0 0.0

Elo ↑ 938−120,+82 888−172,+147 874−108,+68 868−157,+103 845−211,+118 834−364,+254 798−186,+142 696−250,+171 479−370,+129 447−424,+149 289−391,+150

19.3% 22.5% 20.5% 21.5% 22.8% 27.9% 24.8% 31.6% 38.5% 38.7% 41.0%

72.1 74.7 75.5 75.8 76.8 77.3 78.8 82.2 86.7 87.2 88.9

0.0 0.0 0.0 0.0 0.0 0.1 0.0 0.0 0.0 0.0 0.0

Model B: Loser

Aut

Rea EXAAutogl Xiao uon Lig C Mod X ON uo T lTabP R m 1 Lim .6 ( Cau MitraE Tabnu 1.6 (TabPF i-TabLTabICabPFN-FN-2.e5alMLTabDPThtGBaMtBoosernNCGA Boos T a T b iX-2 abF FM NC, silo -v2 lar EX, N-3 DM Lv2 2.6 (T P (T (T (T t (T (T t (T (D) M+ (D) 4h) (D) (D) (D) 4h) (D) (D) (D) (D) +E) +E) +E) +E) +E) +E) +E) 65 68 78 79 81 79 81 90 91 91 93 93 97 95 97 97 98 98 LimiX-2 (D) oGl

60 63 64 62 71 66 82 86 85 88 87 90 90 93 92 94 95 TabFM+ 35 58 62 59 68 63 79 80 84 83 83 87 88 91 90 91 93 TabFM (D) 32 40 49 44 59 71 76 76 80 84 83 89 91 96 94 93 96 AutoGluon 1.6 (NC, 4h) 22 37 42 47 55 61 79 80 85 85 83 91 93 94 91 94 95 Causilo (D) 21 36 38 51 61 60 76 77 78 85 83 89 88 94 91 94 94 Mitra-v2 (D) 19 38 41 56 53

Model A: Winner

Improv- Avg. rank #wins ability ↓ ↓ ↑

50 62 70 71 83 80 89 81 93 94 94 96 EXAONE Tabular (D) 21 29 32 41 45 39 65 71 75 78 75 87 89 93 91 91 94 Autogluon 1.6 (EX, 4h) 19 34 37 29 39 40 50 59 60 63 65 76 75 81 82 85 85 TabPFN-3 (D) 10 18 21 24 21 24 38 35

Win Rate

100%

80%

60%

56 56 59 72 77 77 78 80 80 Xiaomi-TabLDM (D) 9 14 20 24 20 23 30 29 41 53 55 70 74 77 77 79 79 TabICLv2 (D) 9 15 16 20 15 22 29 25 40 44 50 72 69 79 80 82 83 TabPFN-2.6 (D) 7 12 17 16 15 15 17 22 37 44 47 69 66 77 77 75 81 RealTabPFN-2.5 (T+E) 7 13 17 17 17 17 20 25 35 41 45 50 50 68 68 64 77 RealMLP (T+E) 3 10 13 11 9 11 11 13 24 28 30 28 31 TabDPT (T+E) 5 10 12 9 LightGBM (T+E) 3 7 9 4 CatBoost (T+E) 3 8 10 6

7 12 19 11 25 23 26 31 34 50

ModernNCA (T+E) 2 XGBoost (T+E) 2

59 59 59 61

7 19 23 23 21 23 32 41

9 18 22 23 20 23 32 41 49

51 50 71

40%

20%

53 63

9 15 20 21 18 25 36 41 50 47

6 15 20 21 17 19 23 39 29 37 43

57 0%

Figure 5: Pairwise win rates on the TabArena benchmark. Entry (i, j) is the pairwise win rate of method i against method j, computed within each dataset and then averaged equally across datasets, with ties counted as half a win. All off-diagonal entries in the row corresponding to LimiX-2 exceed 60%. Pairwise comparisons. Figure 5 reports pairwise win rates for the displayed methods and configurations. Each entry (i, j) is computed by first calculating the split-level win rate of method i against method j within each dataset, with ties counted as half a win, and then averaging equally across datasets. All offdiagonal entries in the row corresponding to LimiX-2 exceed 60%, indicating that LimiX-2 holds a majority win rate against every displayed competitor. In particular, its win rates against TabFM+, TabFM, AutoGluon 1.6 (noncommercial, 4h), and EXAONE Tabular are 65.2%, 67.7%, 78.3%, and 79.0%, respectively. It also achieves 79.3% against Causilo (Nums AI Inc., 2026) and 81.3% against Mitra-v2. Its win rates exceed 90% against the remaining displayed tabular foundation models, including Xiaomi-TabLDM, and reach at least 95% against the displayed tuned-and-ensembled tree-based and neural-network baselines. 5.3

Results on TALENT

Overall performance. Table 5 reports performance on the TALENT benchmark. LimiX-2 achieves the highest Elo across all five evaluation categories, together with the lowest overall improvability and the largest aggregated win count among the compared methods. Its overall Elo reaches 1506, exceeding TabFM and AutoGluon 1.6 (EX, 4h) by 35 and 68 points, respectively. The improvability decreases to 6.75%, compared with 9.17% for TabFM, corresponding to a 26.4% relative reduction. LimiX-2 also records an aggregated win count of 84.3, versus 50.1 for TabFM, approximately 1.7× as many wins. Consistent with these aggregate results, Figure 6 shows that LimiX-2 attains the lowest average rank in

Table 5: Performance on the TALENT benchmark. Models are ranked by overall Elo score. We also report Elo scores for classification and regression tasks, with classification further split into binary and multiclass settings. Red boldface highlights the best result in each metric. LimiX-2 achieves the highest Elo across all five evaluation categories, attaining an overall Elo of 1506 and exceeding the runner-up, TabFM, by 35 points. Its classification and regression Elo scores reach 1475 and 1584, surpassing TabFM by 26 and 55 points, respectively. LimiX-2 also leads in both binary and multiclass classification, with Elo scores of 1455 and 1520, respectively, demonstrating consistent model performance across task types. Elo (↑) Model

LimiX-2 TabFM AutoGluon 1.6 (EX, 4h) EXAONE Tabular TabPFN-3 TabICLv2 Xiaomi-TabLDM TabDPT Mitra-v2 LimiX-16M CatBoost TabR RealMLP ModernNCA FT-Transformer RandomForest XGBoost LightGBM TabM

Improvability

#wins

Overall

Classification

Regression

Binary

Multiclass

(↓)

(↑)

1506−32,+37 1471−28,+30 1438−27,+30 1393−22,+23 1363−22,+26 1344−22,+23 1341−20,+21 1272−25,+24 1269−22,+24 1227−21,+21 1094−22,+23 1075−24,+24 1069−23,+22 1055−26,+24 1032−25,+25 1000−30,+28 999−26,+24 973−30,+29 953−32,+29

1475−35,+38 1449−34,+36 1379−32,+33 1358−26,+25 1331−26,+27 1338−27,+28 1303−22,+24 1279−28,+31 1263−28,+30 1210−23,+23 1094−27,+26 1078−29,+29 1046−27,+26 1053−29,+27 1024−31,+28 1000−37,+35 1007−27,+27 978−38,+35 977−36,+33

1584−71,+92 1529−53,+60 1581−53,+65 1477−36,+44 1441−48,+55 1365−33,+39 1431−37,+44 1262−42,+46 1287−44,+46 1268−42,+43 1094−44,+37 1071−39,+36 1115−42,+39 1058−51,+48 1048−45,+42 1000−53,+48 980−58,+49 964−58,+48 899−61,+53

1455−43,+51 1418−38,+46 1340−38,+40 1370−31,+37 1333−31,+36 1338−35,+39 1305−28,+30 1312−35,+34 1267−31,+34 1217−26,+28 1107−32,+32 1077−35,+31 1023−34,+30 1062−39,+33 1027−39,+36 1000−49,+43 999−32,+29 977−45,+41 967−47,+43

1520−51,+63 1517−64,+78 1465−43,+53 1342−37,+41 1331−42,+49 1343−41,+51 1301−34,+40 1224−51,+52 1258−48,+51 1198−39,+39 1073−51,+46 1080−52,+50 1087−46,+43 1039−49,+44 1018−56,+46 1000−68,+59 1022−53,+44 979−62,+59 996−56,+49

6.75% 9.17% 11.21% 14.74% 15.98% 15.63% 16.48% 19.47% 17.72% 20.52% 26.84% 26.25% 26.62% 26.65% 29.27% 30.06% 29.37% 29.80% 32.37%

84.3 50.1 48.0 12.9 15.7 10.2 7.4 11.2 7.4 6.1 4.9 4.1 4.5 5.4 4.1 3.8 2.3 2.6 3.0

binary classification, multiclass classification, and regression. Together, these results demonstrate that its strong relative performance extends beyond TabArena to the TALENT benchmark. Classification performance. Table 5 shows that LimiX-2 achieves an Elo rating of 1475 on classification tasks, surpassing TabFM by 26 points. This advantage holds in both binary and multiclass classification, with Elo scores of 1455 and 1520, compared with 1418 and 1517 for TabFM, respectively. Figure 6 provides a complementary dataset-level comparison based on accuracy, where LimiX-2 achieves the lowest average ranks of 4.62 and 3.91 on binary and multiclass classification, respectively. The agreement between Elo and average-rank evaluations demonstrates consistently strong classification performance across the two task settings. Regression performance. On regression, LimiX-2 attains the highest Elo of 1584, slightly ahead of AutoGluon 1.6 (EX, 4h) at 1581 and exceeding TabFM and EXAONE Tabular by 55 and 107 points, respectively, as shown in Table 5. Figure 6 provides a complementary comparison based on RMSE, where LimiX-2 again achieves the lowest average rank, at 3.88. The consistent advantage under both Elo and dataset-level average rank further supports the strong regression performance of LimiX-2. Pairwise comparisons. Figure 7 reports pairwise win rates across evaluation tasks. Each entry (i, j) denotes the percentage of tasks on which method i outperforms method j. LimiX-2 wins 57% of tasks against TabFM, 62% against AutoGluon 1.6 (EX, 4h), and 75% against EXAONE Tabular. Against the remaining tabular foundation models, its win rates range from 74% to 85%, while they reach 89–96% against the remaining baselines. Overall, LimiX-2 achieves a win rate above 50% against every competing method, showing that its leading aggregate performance is supported by broad pairwise advantages rather than gains concentrated on a small subset of datasets. Meta-feature subgroups. Figure 8 examines how model rankings vary with dataset size and dimensionality. LimiX-2 maintains among the lowest fitted ranks across most sample-size and feature-count ranges, with particularly strong performance on medium-to-large datasets. Across feature counts, its fitted rank also remains consistently competitive and generally improves as dimensionality increases. These trends indicate that the aggregate advantage of LimiX-2 is preserved across datasets with substantially different numbers of samples and features, rather than being driven by a narrow dataset regime. 16

Binary classification

Multiclass classification

4.62

LimiX-2

Regression

3.91

3.88

TabFM AutoGluon 1.6 (EX, 4h) EXAONE Tabular TabPFN-3 TabICLv2 Xiaomi-TabLDM Mitra-v2 TabDPT LimiX-16M CatBoost TabR RealMLP ModernNCA FT-Transformer XGBoost RandomForest LightGBM TabM

Figure 6: Average-rank comparison on the TALENT benchmark across binary classification, multiclass classification, and regression tasks. Each bar represents a method’s mean rank over datasets in the corresponding task, using accuracy for classification and RMSE for regression. Error bars denote 95% bootstrap CIs from 1000 dataset resamples. For consistency with Table 5, methods are displayed in the same order across all panels. LimiX-2 is highlighted in pink, while baseline models are shown in dark blue. LimiX-2 achieves the lowest average rank in all three task categories, with ranks of 4.62, 3.91, and 3.88 for binary classification, multiclass classification, and regression, respectively. Model B: Loser

EXA Xiao O M FT-Tr Ran m Lim TaAutoGNl E Tab TabPF TabIC i-TabL Tab MitrLaimiX-1CatBo T RealodernaNnsfordomFo XGBoLightG Ta iX-2 bFM uon ular N-3 Lv2 DM DPT -v2 6M ost abR MLP CA mer rest ost BM bM 57 62 75 74 75 78 78 83 85 89 92 93 93 92 93 94 93 96 LimiX-2 TabFM 43

100%

56 66 68 71 70 77 79 82 90 92 93 91 92 92 93 94 95

AutoGluon 38 44

59 62 71 71 73 73 76 87 91 93 92 91 89 94 92 92

EXAONE Tabular 25 34 41

51 60 59 67 71 80 90 90 92 90 91 91 94 93 95

TabPFN-3 26 32 38 49

55 58 64 65 70 84 86 90 90 89 87 91 92 91

TabICLv2 25 29 29 40 45

80%

49 65 63 71 84 87 90 88 88 89 89 90 90

Xiaomi-TabLDM 22 30 29 41 42 51

Model A: Winner

Win Rate

61 64 71 85 88 90 88 90 88 89 92 91

TabDPT 22 23 27 33 36 35 39

51 61 76 79 82 81 85 81 84 84 89

Mitra-v2 17 21 27 29 35 37 36 49

60%

63 78 79 82 79 83 82 83 88 89

LimiX-16M 15 18 24 20 30 29 29 39 37

74 75 78 75 79 81 83 85 87

CatBoost 11 10 13 10 16 16 15 24 22 26 TabR 8

10 14 13 12 21 21 25 43

RealMLP 7

57 57 59 63 70 74 62 75 52 56 58 62 64 66 66

10 10 10 18 18 22 43 48

40%

54 60 58 59 62 67

ModernNCA 7

10 10 12 12 19 21 25 41 44 46

FT-Transformer 8

11 12 10 15 17 21 37 42 40 49

51 59 61 66 63

RandomForest 7

13 11 12 19 18 19 30 38 42 41 47

XGBoost 6

11 11 16 17 17 26 36 41 39 47 50

LightGBM 7

16 12 15 38 34 38 34 45 46 53

TabM 4

11 11 13 25 34 33 37 33 38 38 47

53 53 55 67 50 54 62

20%

47 62 53 0%

Figure 7: Pairwise win rates on the TALENT benchmark. Entry (i, j) is the percentage of evaluation tasks on which method i beats method j. Cells range from purple (row loses) to white (tie) to green (row wins). LimiX-2 achieves a win rate above 50% against every competing method, ranging from 57% against TabFM to 96% against TabM.

TabFM

AutoGluon 1.6 (EX, 4h)

EXAONE Tabular

TabPFN-3

TabICLv2

Xiaomi-TabLDM

TabDPT

Mitra-v2

LimiX-16M

CatBoost

TabR

RealMLP

ModernNCA

FT-Transformer

RandomForest

XGBoost

LightGBM

TabM

20.0

17.5

15.0

12.5

← Rank

LimiX-2

10.0

7.5

5.0

2.5

0.0

10³

10⁴

0.0

10⁵

10¹

Number of samples

10²

Number of features

Figure 8: Model rankings across dataset meta-features on TALENT. Lines show median piecewise-linear fits over 300 bootstrap rounds, with 10th–90th percentile bands (Qu et al., 2026). LimiX-2 achieves among the lowest fitted ranks across most sample-size and feature-count ranges, with particularly strong performance on medium-to-large datasets. These results show that its strong aggregate performance remains robust across datasets with varying scales and dimensionalities. Table 6: Performance on the BCCO benchmark. LimiX-2 ranks first overall, with an Elo of 1432 (56 / 63 / 202 above AutoGluon 1.6 (EX, 4h), TabFM, and LimiX-16M). Its lead holds on classification (1321) and regression (1859), and on binary (1284) tasks. AutoGluon 1.6 (EX, 4h) leads on multiclass (1443). Elo (↑) Model

LimiX-2 AutoGluon 1.6 (EX, 4h) TabFM EXAONE Tabular TabPFN-3 Xiaomi-TabLDM TabICLv2 Mitra-v2 TabDPT LimiX-16M CatBoost TabR RealMLP FT-Transformer ModernNCA RandomForest XGBoost LightGBM TabM

5.4

Improvability

#wins

Overall

Classification

Regression

Binary

Multiclass

(↓)

(↑)

1432−38,+45 1376−37,+39 1369−37,+40 1345−29,+32 1295−26,+29 1277−27,+29 1275−25,+27 1239−31,+32 1238−28,+28 1230−24,+26 1140−28,+25 1045−30,+29 1044−29,+27 1020−32,+30 1019−30,+28 1000−41,+36 997−36,+32 969−44,+40 957−41,+38

1321−35,+42 1269−37,+42 1260−40,+43 1255−32,+36 1188−29,+30 1176−29,+29 1205−30,+34 1169−36,+38 1197−34,+38 1195−30,+31 1101−30,+31 1001−38,+34 973−32,+31 973−40,+34 986−34,+33 1000−44,+39 994−33,+31 953−51,+50 950−47,+42

1859−112,+158 1782−71,+101 1785−75,+101 1689−40,+54 1691−67,+84 1646−60,+72 1542−49,+64 1502−70,+83 1410−60,+70 1385−61,+63 1287−67,+56 1194−70,+59 1269−70,+68 1176−76,+64 1133−74,+62 1000−102,+74 1008−120,+88 1023−93,+67 978−98,+73

1284−44,+51 1199−44,+47 1213−44,+51 1224−43,+46 1153−38,+40 1136−36,+35 1161−34,+35 1129−41,+43 1151−42,+47 1173−37,+37 1096−35,+37 976−41,+41 943−41,+40 942−46,+41 952−45,+41 1000−51,+50 981−41,+38 958−63,+63 953−53,+47

1414−67,+87 1443−56,+76 1374−74,+93 1333−60,+73 1275−44,+48 1273−50,+57 1313−64,+73 1266−73,+82 1308−53,+53 1252−57,+66 1118−66,+58 1061−76,+75 1042−61,+56 1043−81,+64 1064−62,+53 1000−86,+72 1024−64,+53 936−87,+72 941−93,+77

6.97% 9.77% 12.24% 13.40% 14.92% 15.58% 15.67% 16.86% 16.96% 16.94% 21.64% 23.52% 24.58% 25.89% 25.14% 26.75% 26.35% 27.73% 29.45%

50.4 24.5 16.7 7.8 5.7 3.5 3.3 3.0 8.4 6.1 4.6 2.6 3.8 2.1 2.0 2.3 3.0 3.7 2.6

Results on BCCO

Overall performance. Table 6 reports performance on the BCCO benchmark. LimiX-2 achieves the highest overall Elo, along with the lowest overall improvability and the largest aggregated win count among the compared methods. Its overall Elo reaches 1432, exceeding AutoGluon 1.6 (EX, 4h), TabFM, and LimiX-16M by 56, 63, and 202 points, respectively. The improvability decreases to 6.97%, compared with 12.24% for TabFM, corresponding to a 43.1% relative reduction. In addition, LimiX-2 records an aggregated win count of 50.4, versus 16.7 for TabFM, approximately 3.0× as many wins. These results demonstrate the leading predictive performance of LimiX-2 across the heterogeneous tasks within the BCCO benchmark.

Binary classification

Multiclass classification

5.45

LimiX-2

5.17

Regression 3.10

AutoGluon 1.6 (EX, 4h) TabFM EXAONE Tabular TabPFN-3 Xiaomi-TabLDM TabICLv2 Mitra-v2 TabDPT LimiX-16M CatBoost RealMLP TabR FT-Transformer ModernNCA XGBoost RandomForest LightGBM TabM

Figure 9: Average-rank comparison on the BCCO benchmark across binary classification, multiclass classification, and regression tasks. Each bar represents a method’s mean rank over datasets in the corresponding task, using accuracy for classification and RMSE for regression. Error bars denote 95% bootstrap CIs from 1000 dataset resamples. For consistency with Table 6, methods are shown in the same order across all panels. LimiX-2 is highlighted in pink, and baseline models are shown in dark blue. LimiX-2 achieves a low average rank in all three task categories, with ranks of 5.45, 5.17, and 3.10 for binary classification, multiclass classification, and regression, respectively. Classification performance. Table 6 reports a classification Elo of 1321 for LimiX-2, exceeding AutoGluon 1.6 (EX, 4h) by 52 points and TabFM by 61 points. It also achieves the highest Elo in binary classification, reaching 1284. On multiclass classification, LimiX-2 remains highly competitive with an Elo of 1414, 40 points above TabFM and within 29 points of the best result. Figure 9 provides complementary comparisons based on accuracy across 71 binary and 35 multiclass classification datasets. On binary classification, LimiX-2 achieves the best average rank of 5.45, followed by EXAONE Tabular at 6.69 and TabFM at 6.93. On multiclass classification, LimiX-2 attains a competitive average rank of 5.17, ahead of TabFM at 5.93 and within 0.49 of the best result. The agreement between Elo and accuracy-based rankings demonstrates the classification strength of LimiX-2 across both binary and multiclass prediction tasks. Regression performance. On regression, LimiX-2 attains an Elo of 1859, exceeding TabFM by 74 points and TabPFN-3 by 168 points, as shown in Table 6. Figure 9 compares RMSE-based average ranks across 50 regression datasets. LimiX-2 achieves the lowest average rank of 3.10, followed by TabFM at 3.93 and AutoGluon 1.6 (EX, 4h) at 3.97, while TabPFN-3 and EXAONE Tabular attain 5.16 and 5.20. It also improves substantially over LimiX-16M, whose average rank is 9.96. The leading position of LimiX2 under both Elo and RMSE-based rankings reinforces the strong regression performance established on TabArena and TALENT, demonstrating its effectiveness across the regression tasks in the BCCO benchmark. Pairwise comparisons. Figure 10 reports pairwise win rates across evaluation tasks. Each entry (i, j) denotes the percentage of tasks on which method i outperforms method j. For LimiX-2, the off-diagonal values span 62% to 94%, showing that it defeats every baseline in the majority of tasks. Specifically, LimiX-2 wins 65% of tasks against TabFM and 67% against EXAONE Tabular. Against the other tabular foundation models, its win rates fall between 73% and 79%. For tree-based and neural baselines, the win rates are even higher, ranging from 81% to 94%. These pairwise results align with the overall Elo and average-rank findings, confirming that LimiX-2 maintains a consistent predictive advantage across the various model families.

Model B: Loser

EXA Xiao FT-T Ran M O r m LimAutoGl TaNE Tab TabPF TabIC i-TabL Tab MitrLaimiX-1CatBo T RealansfordomFoodernN XGBoLightG Ta iX-2 uon bFM ular N-3 Lv2 DM DPT -v2 6M ost abR MLP mer rest CA ost BM bM 62 65 67 74 75 73 73 79 79 81 93 91 94 89 94 92 92 93 LimiX-2 AutoGluon 38

58 68 67 71 70 72 72 78 86 89 85 87 90 91 87 89

EXAONE Tabular 33 43 42

57 63 63 67 67 71 80 87 87 89 87 89 90 91 89

TabPFN-3 26 34 32 43 TabICLv2 25 26 33 37 43

49 62 55 60 74 81 88 84 84 88 88 86 88 56 55 58 74 82 86 85 81 87 85 87 84

Model A: Winner

TabDPT 27 29 30 33 41 38 44

47 56 70 79 79 85 83 83 84 83 86

Mitra-v2 21 30 28 33 38 45 45 53

72 77 78 83 82 83 84 85 86

CatBoost 19 19 22 20 26 26 26 30 33 28

FT-Transformer 6

70 69 75 79 71 87 76 78

13 14 13 15 19 18 21 24 23 30

40%

51 54 54 58 56 59 64

11 13 12 12 14 21 17 22 31 49

55 54 53 52 60 66

14 15 11 12 16 15 15 21 17 25 46 45

51 55 53 53 68

RandomForest 11 12 13 13 18 16 19 17 21 18 21 46 46 49

47 52 58 59

ModernNCA 6

XGBoost 8

10 12 12 15 16 18 16 13 44 48 47 48 44

LightGBM 8

TabM 7

60%

53 67 76 83 79 79 82 82 84 83

LimiX-16M 21 29 28 29 41 40 42 44 47

80%

57 57 59 62 59 74 85 88 88 82 87 88 86 90

Xiaomi-TabLDM 27 28 29 37 43 51

RealMLP 9

100%

51 57 66 74 72 71 70 71 81 87 93 86 88 92 92 92 88

TabFM 35 49

TabR 7

Win Rate

10 11 13 12 13 17 18 17 29 42 47 45 53

20%

56 60 60 54 58

14 14 13 17 16 15 24 41 40 47 42 40 46

12 11 11 10 12 16 14 17 14 22 36 34 32 41 40 42 48

LimiX-2

AutoGluon 1.6 (EX, 4h)

TabFM

EXAONE Tabular

TabPFN-3

Xiaomi-TabLDM

TabICLv2

Mitra-v2

TabDPT

LimiX-16M

CatBoost

TabR

RealMLP

FT-Transformer

ModernNCA

RandomForest

XGBoost

LightGBM

TabM

20.0

17.5

15.0

12.5

← Rank

Figure 10: Pairwise win rates on the BCCO benchmark. Entry (i, j) is the percentage of evaluation tasks on which method i beats method j. Cells range from purple (row loses) to white (tie) to green (row wins). LimiX-2 wins the majority of tasks against every compared method, achieving win rates of 65% against TabFM and 67% against EXAONE Tabular. Its win rates reach 73–79% against the remaining tabular foundation models, including 79% against LimiX-16M, and 81–94% against tree-based and neuralnetwork baselines, demonstrating broad predictive advantages across the benchmark.

10.0

7.5

5.0

2.5

0.0

10³

0.0

10⁴

Number of samples

10⁰

10¹

10²

Number of features

Figure 11: Model rankings as a function of meta-features on the BCCO benchmark. Each curve is a piecewise-linear fit. The line is the median over 300 bootstrap rounds and the band is the 10th–90th percentile (Qu et al., 2026). Lower is better. Meta-feature subgroups. Figure 11 examines model rankings across dataset characteristics, including sample size and feature dimensionality. LimiX-2 maintains one of the lowest fitted ranks over much of the sample-size range, with particularly strong relative performance around 104 samples. It also achieves low fitted ranks on datasets with approximately 10–100 features, highlighting its effectiveness across a broad range of feature dimensionalities. These results identify advantages across multiple dataset characteristics and complement the leading aggregate performance of LimiX-2.

5.5

Causal Skeleton Recovery Evaluation

We evaluate the causal awareness of tabular foundation models through probing the structural information encoded in their internal feature attention score. Specifically, we investigate whether feature attention scores can distinguish causal relationships (i.e. adjacency in causal graph) from non-causal ones (i.e. non-adjacency). When predicting a target variable, a model with genuine causal awareness is expected to concentrate its attention on the features causally adjacent to the target. The presence of a causal edge between a feature and the target can therefore be identified by thresholding the corresponding attention score: the stronger a model’s causal awareness, the more accurate the causal skeleton induced by this separation should be. We thus adopt causal skeleton recovery performance as an operational measurement of causal awareness for tabular foundation models. Benchmark Protocol Given observational data over F variables, we designate each variable X j in turn as the prediction target, with the remaining variables serving as features for tabular foundation models. Owing to the design of cell-level representations, we can measure the feature attention exclusive to each feature as the causal relationship strength between the feature and target for models retaining cell-level representations. However, some tabular foundation models, such as TabPFN-3, TabICLv2, TabFM and Xiaomi-TabLDM, aggregate feature information into row-level representation to predict targets. Thereby, the feature attention score attended by target can be calculated only at the feature group level. For these models, we distribute the attention score of each feature group uniformly among its member features to obtain the measurement of causal relationship strength. The tree-based XGBoost can also produce feature importance scores (Chen & Guestrin, 2016), for which we adopt the gain-based variant as its measure of causal strength. By retaining the variable relations with high scores, determined by a threshold over bskel of the causal graph. For causal the overall score distribution, we yield a sparse causal skeleton G discovery methods, we convert the produced directed causal graphs into undirected ones to obtain causal skeletons. We compare the estimated skeletons against the ground truths using skeleton F1 score and structural Hamming distance (SHD). Baselines The baselines include tabular foundation models (TabPFN-3, TabICLv2, TabFM, EXAONE Tabular, Xiaomi-TabLDM), XGBoost with gain-based feature importance, and causal discovery methods across major families: pretrained/amortized structure learning (TabCausal (Li et al., 2026), AVICI (Lorch et al., 2022)), nonlinear continuous-optimization (NOTEARS-MLP (Zheng et al., 2020), DAG-GNN (Yu et al., 2019)), conditional independence constraint-based PC (Spirtes et al., 2000), graph score-based GES (Chickering, 2002), and identifiable functional causal models LiNGAM (Shimizu et al., 2006). Benchmarks We evaluate on six causal discovery datasets: three continuous datasets and three discrete datasets. The continuous datasets span the Sachs protein signaling network(Sachs et al., 2005), the UF food production process(Menegozzo et al., 2022), and the CausalChamber light tunnel system(Gamella et al., 2025). The discrete benchmarks are the PATHFINDER network for lymph node pathology(Heckerman et al., 1992; Scutari, 2026), the DIABETES network for blood glucose regulation and insulin adjustment(Andreassen et al., 1991; Scutari, 2026), and the PIGS genetic pedigree network(Cowell et al., 1999; Scutari, 2026). Table 7 shows that the feature attention scores in tabular foundation models encode the structural information of direct causal relationship, which can be converted into a causal skeleton. LimiX-2 attains a mean F1 score of 0.7972 across the six datasets, ranking first in F1 on six datasets and achieving the lowest SHD on five. This improvement suggests that LimiX-2 extracts predictive signal from the direct causal relations, i.e., its feature attention scores encode causal information. EXAONE Tabular ranks second with F1 score of 0.6591. In contrast, the tabular foundation models that compute feature attention at the group level can not distinguish causal relationship strength among features at fine-granularity. Consequently, the causal skeleton recovery performances of TabFM, TabICLv2, TabPFN-3 and XiaomiTabLDM deteriorate markedly, highlighting the importance of CMNs and cell-level representations for causal structure awareness in tabular foundation models. Furthermore, the skeletons recovered by LimiX-2 match or surpass those of methods designed exclusively for causal discovery in several settings, indicating that LimiX-2 can effectively internalize causal structure information into its internal representations.

Scaling Law

Scaling laws have only recently begun to be studied for foundation models operating on structured data. LimiX (LimiX Team, 2025) introduced the first explicit scaling-law study for large structured-data models (LDMs), characterizing how model capacity and pretraining scale affect both training loss and downstream performance. Subsequent work broadened scaling analyses for LDMs along complementary 21

Table 7: Comparison of causal skeleton recovery among tabular foundation models, XGBoost, and dedicated causal discovery methods. LiNGAM assumes continuous variables and is therefore not applicable to the discrete networks (PATHFINDER, DIABETES, and PIGS). These results are marked as –. PC, GES, and NOTEARS-MLP do not finish within the 12-hour time limit on some datasets and are marked as TIMEOUT. Model

Metric

Sachs

CausalChamber

PATHFINDER

DIABETES

PIGS

LimiX-2

F1 score ↑ SHD ↓ F1 score ↑ SHD ↓ F1 score ↑ SHD ↓ F1 score ↑ SHD ↓ F1 score ↑ SHD ↓ F1 score ↑ SHD ↓ F1 score ↑ SHD ↓ F1 score ↑ SHD ↓ F1 score ↑ SHD ↓ F1 score ↑ SHD ↓ F1 score ↑ SHD ↓ F1 score ↑ SHD ↓ F1 score ↑ SHD ↓ F1 score ↑ SHD ↓

0.7143 8 0.6429 10 0.4286 16 0.6429 10 0.5714 12 0.3704 17 0.5714 12 0.5600 11 0.6400 9 0.6400 9 0.5600 11 0.5000 12 0.5600 11 0.4167 14

0.8617 26 0.6492 67 0.2727 80 0.1818 90 0.6845 59 0.2364 84 0.2545 82 0.4000 81 0.3817 81 0.6044 72 0.5521 73 0.4783 72 0.4030 80 0.0860 85

0.7013 23 0.5195 37 0.4156 45 0.2857 55 0.3636 49 0.2078 61 0.2632 56 0.5970 27 0.4000 39 0.6234 29 0.6667 21 0.2917 34 0.4571 38 0.2642 39

0.7829 76 0.5486 158 0.2400 266 0.1600 294 0.1429 300 0.0114 346 0.2914 248 0.0000 189 0.3659 156 TIMEOUT TIMEOUT – – 0.0485 196 0.6146 158 0.3846 144

0.7846 263 0.6994 367 0.6028 485 0.2277 943 0.0426 1169 0.2211 951 0.4111 719 0.1855 685 0.4674 449 TIMEOUT TIMEOUT – – 0.0251 622 TIMEOUT TIMEOUT 0.4320 497

0.9385 77 0.8955 131 0.6895 389 0.1612 1051 0.0271 1219 0.2426 949 0.8125 235 0.0000 592 TIMEOUT TIMEOUT TIMEOUT TIMEOUT – – 0.0000 592 0.8907 145 0.1529 543

EXAONE Tabular TabFM TabICLv2 TabPFN-3 Xiaomi-TabLDM XGBoost TabCausal PC GES LiNGAM AVICI NOTEARS-MLP DAG-GNN

axes. PluRel established power-law scaling of relational-foundation-model pretraining loss with the number of synthetic databases and training tokens. MaskTab reported one-axis-at-a-time scaling trends over unlabeled data volume, feature dimensionality, and model capacity. TabPFN-3 explored context-size and test-time compute scaling for tabular prediction (Kothapalli et al., 2026; Zheng et al., 2026; Grinsztajn et al., 2026). Building on this emerging line of research, LimiX-2 conducts a finer-grained and more controlled scaling study along the model-capacity axis, covering multiple downstream benchmarks. LimiX-2 is pretrained on generated data, allowing the training corpus to be expanded on demand, while its model sizes remain far below those of modern large language models, keeping controlled scaling experiments computationally manageable. This setting allows us to focus on model capacity as the primary scaling variable and ask a targeted question: under a fixed data-generation, optimization, and inference recipe, how does downstream performance scale with the number of trainable parameters? We evaluate LimiX-2 configurations ranging from 12.5M to 406.2M parameters and extrapolate the fitted scaling trend toward the billion-parameter regime. 6.1

Experimental Setup and Scaling Model

For each evaluation series, let Ni denote the number of parameters in millions and Ei the observed Elo score. We fit the following log-linear scaling law: Ni Ei = α + β log2 + εi. (1) 100 Here, α denotes the fitted Elo at 100M parameters, while β measures the expected Elo improvement from each doubling of model size.Both coefficients are estimated by ordinary least squares over the six model sizes for each evaluation series: Ni ∑ ( x − x̄ )( Ei − Ē) bx̄, xi = log b βb = i i , α = Ē − β (2) 2 100 . ∑i ( xi − x̄ )2 22

Baselines are shown only for reference and are not included in the scaling-law fit. We report R2 and residual RMSE to characterize the goodness of fit. Table 8: Log-linear scaling fits. α is the fitted Elo at 100M parameters, and β is the Elo gained per parameter doubling. α

RMSE

1863.88 1427.25 1545.12 1295.86 1795.89

34.68 22.16 18.26 11.24 30.06

0.9808 0.9792 0.9680 0.9617 0.9702

8.31 5.53 5.69 3.84 9.03

Evaluation task TabArena TALENT classification TALENT regression BCCO classification BCCO regression 6.2

Scaling Results

Across all five evaluation series, downstream performance follows a clear log-linear trend with model size. The fitted models achieve R2 values between 0.9617 and 0.9808, with residual RMSE ranging from 3.84 to 9.03 Elo (Table 8). The consistently positive slopes indicate robust gains from increasing model capacity, while their different magnitudes reveal substantial task-dependent variation in the returns to scaling.

Figure 12: Parameter scaling on TabArena. The solid line connects the observed LimiX-2 scale points, while the dashed line shows the log-linear OLS fit extrapolated to 2B parameters. On TabArena, Elo increases from 1766 at 12.5M parameters to 1935 at 406.2M, yielding a gain of 169 Elo over a roughly 32.5× increase in model size. The fitted slope reaches 34.68 Elo per parameter doubling, the steepest among the five evaluation series. On TALENT, classification and regression improve by approximately 109 and 89 Elo, respectively, over the same range. The trend generalizes to the disjoint BCCO benchmark collection, where classification gains 56 Elo and regression gains 147 Elo. Notably, BCCO regression exhibits the second-largest scaling coefficient, at 30.06 Elo per doubling. Within the measured parameter range, LimiX-2 also compares favorably with existing structured-data foundation models at comparable model scales. These comparisons provide evidence of strong empirical parameter efficiency under the reported evaluation protocols, although differences in architectures, pretraining distributions, and training compute preclude attributing the gaps solely to model design. The extrapolated 2B results should be interpreted as forecasts rather than measured performance: larger models may enter different optimization or data regimes, and Elo scores also depend on the benchmark comparison pool. Nevertheless, the consistency of the scaling trends across three benchmark suites and both classification and regression supports a clear conclusion: model capacity is a robust and predictable 23

Figure 13: Parameter scaling on TALENT classification and regression.

Figure 14: Parameter scaling on BCCO classification and regression. scaling axis for LimiX-2 within the regime studied. We observe no clear evidence of performance saturation up to 406.2M parameters, providing strong empirical motivation for extending LimiX-2 into the billion-parameter regime.

Conclusions

In this report, we have advocated the development of Large Data Models (LDMs) and identified a key limitation of the prevailing Prior-Fitted Network (PFN) paradigm, which is the confinement to label prediction, thereby introduce Contextual Mechanism Networks (CMNs), a design paradigm that generalizes label prediction to the imputation of arbitrary masked columns. CMNs are trained with Context-Conditional Masked Modeling (CCMM), an objective that provides substantially denser supervision than label prediction alone and steers the model toward stronger data reasoning capabilities. We instantiated this paradigm with LimiX-2, a transformer-based tabular foundation model with cell-level representations, pretrained exclusively on synthetic data from an SCM-based generation engine. Without any task-specific training, LimiX-2 performs classification, regression, missing-value imputation, and causal inference in a single forward pass. The evaluation results on TabArena, TALENT, and BCCO reveal that Limix-2 outperform a wide range of tabular foundation models and dataset-specific models, and justify the superiority of CMNs.

Contribution

Project Design and Lead Xingxuan Zhang, Peng Cui Core Contributors Gang Ren, Hao Yuan, Hao Zou, Hongze Tan, Hui Wang, Jianhao Song, Jiansheng Li, Jiayao Zhang, Jinghan Zhang, Kaifang Li, Lang Mo, Li Mao, Mingchao Hao, Nuo Xu, Rui Ding, Ruiji Zhang, Shuyang Li, Siyu Mei, Tianyang Zhang, Weiyang Mu, Yancheng Dong, Yongxian Wei, Yuan Xue, Yuanrui Wang, Yue He, Zijia Yang, Ziyun Li Contributors Dongzhe Li, Fuqiang Wang, Jiandong Liu, Jiawei Chen, Jiaxin Du, Kaijie Cheng, Kehan Li, Lei Sun, Linjun Zhou, Ningbo Dai, Qi Wang, Renzhe Xu, Shaoxing Du, Shumeng Yang, Wang Lu, Wenjing Chu, Xiannan Huang, Xiaoyu Lin, Xing Ai, Xinyan Han, Xuanyue Li, Xuanyue Su, Xukun Zhang, Yan Lu, Yaxin Zhang, Yi Qin, Yifei Huang, Yihan Xu, Yongle Lv, Yuanyuan Jiang, Yushan Han

References Steen Andreassen, Roman Hovorka, John Benn, Kristian G. Olesen, and Ewart R. Carson. A model-based approach to insulin adjustment. In Proceedings of the Third Conference on Artificial Intelligence in Medicine, pp. 239–249, 1991. Pierre Baldi, Peter Sadowski, and Daniel Whiteson. Searching for exotic particles in high-energy physics with deep learning. Nature communications, 5(1):4308, 2014. Irena Barjašić, Hrvoje Štefančić, Vedrana Pribičević, and Vinko Zlatić. Causal motifs and existence of endogenous cascades in directed networks with application to company defaults. Scientific reports, 11 (1):24028, 2021. Leo Breiman. Random forests. Machine learning, 45(1):5–32, 2001. Tianqi Chen and Carlos Guestrin. Xgboost: A scalable tree boosting system. In Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining, pp. 785–794, 2016. David Chiang and Peter Cholak. Overcoming a theoretical limitation of self-attention. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 7654–7664, Dublin, Ireland, May 2022. Association for Computational Linguistics. doi: 10.18653/v1/20 22.acl-long.527. URL https://aclanthology.org/2022.acl-long.527/. David Maxwell Chickering. Optimal structure identification with greedy search. Journal of Machine Learning Research, 3:507–554, 2002. Robert G. Cowell, A. Philip Dawid, Steffen L. Lauritzen, and David J. Spiegelhalter. Probabilistic Networks and Expert Systems. Springer, 1999. Anna Veronika Dorogush, Vasily Ershov, and Andrey Gulin. Catboost: gradient boosting with categorical features support. arXiv preprint arXiv:1810.11363, 2018. Moonjung Eo, Min-Kook Suh, Hye-Seung Cho, Jiwon Kim, Seoyoon Kim, Sangjun Nam, and Soonyoung Lee. EXAONE Tabular 1.0: Technical report. arXiv preprint arXiv:2608.25774, 2026. URL https: //arxiv.org/abs/2608.25774. Nick Erickson, Jonas Mueller, Alexander Shirkov, Hang Zhang, Pedro Larroy, Mu Li, and Alexander Smola. Autogluon-tabular: Robust and accurate automl for structured data. arXiv preprint arXiv:2003.06505, 2020. Nick Erickson, Lennart Purucker, Andrej Tschalzev, David Holzmüller, Prateek Mutalik Desai, David Salinas, and Frank Hutter. Tabarena: A living benchmark for machine learning on tabular data. arXiv preprint arXiv:2506.16791, 2025. Juan L. Gamella, Jonas Peters, and Peter Bühlmann. Causal chambers as a real-world physical testbed for AI methodology. Nature Machine Intelligence, 7:107–118, 2025. Yury Gorishniy, Ivan Rubachev, Valentin Khrulkov, and Artem Babenko. Revisiting deep learning models for tabular data. Advances in neural information processing systems, 34:18932–18943, 2021. Yury Gorishniy, Ivan Rubachev, Nikolay Kartashev, Daniil Shlenskii, Akim Kotelnikov, and Artem Babenko. Tabr: Tabular deep learning meets nearest neighbors. In International Conference on Learning Representations, volume 2024, pp. 18209–18249, 2024. Yury Gorishniy, Akim Kotelnikov, and Artem Babenko. Tabm: Advancing tabular deep learning with parameter-efficient ensembling. arxiv 2024. arXiv preprint arXiv:2410.24210, 2025. Léo Grinsztajn, Klemens Flöge, Oscar Key, et al. TabPFN-3: Technical report. arXiv:2605.13986, 2026. URL https://arxiv.org/abs/2605.13986.

arXiv preprint

Shihao Gu, Bryan Kelly, and Dacheng Xiu. Empirical asset pricing via machine learning. The review of financial studies, 33(5):2223–2273, 2020. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Peiyi Wang, Qihao Zhu, Runxin Xu, Ruoyu Zhang, Shirong Ma, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025. David Ha and Jürgen Schmidhuber. World models. arXiv preprint arXiv:1803.10122, 2(3):440, 2018.

David E. Heckerman, Eric J. Horvitz, and Bharat N. Nathwani. Toward normative expert systems: Part i. the pathfinder project. Methods of Information in Medicine, 31(2):90–105, 1992. Dan Hendrycks and Kevin Gimpel. Gaussian error linear units (GELUs). arXiv preprint arXiv:1606.08415, 2016. URL https://arxiv.org/abs/1606.08415. Noah Hollmann, Samuel Müller, Katharina Eggensperger, and Frank Hutter. Tabpfn: A transformer that solves small tabular classification problems in a second. In The Eleventh International Conference on Learning Representations, 2022. Noah Hollmann, Samuel Müller, Lennart Purucker, et al. Accurate predictions on small data with a tabular foundation model. Nature, 637(8045):319–326, 2025. doi: 10.1038/s41586-024-08328-6. URL https://www.nature.com/articles/s41586-024-08328-6. David Holzmüller, Léo Grinsztajn, and Ingo Steinwart. Better by default: Strong pre-tuned mlps and boosted trees on tabular data. Advances in Neural Information Processing Systems, 37:26577–26658, 2024. Alistair EW Johnson, Tom J Pollard, Lu Shen, Li-wei H Lehman, Mengling Feng, Mohammad Ghassemi, Benjamin Moody, Peter Szolovits, Leo Anthony Celi, and Roger G Mark. Mimic-iii, a freely accessible critical care database. Scientific data, 3(1):1–9, 2016. Guolin Ke, Qi Meng, Thomas Finley, Taifeng Wang, Wei Chen, Weidong Ma, Qiwei Ye, and Tie-Yan Liu. Lightgbm: A highly efficient gradient boosting decision tree. Advances in neural information processing systems, 30, 2017. Moo Jin Kim, Karl Pertsch, Siddharth Karamcheti, Ted Xiao, Ashwin Balakrishna, Suraj Nair, Rafael Rafailov, Ethan Foster, Grace Lam, Pannag Sanketi, et al. Openvla: An open-source vision-languageaction model. arXiv preprint arXiv:2406.09246, 2024. Weihao Kong and Abhimanyu Das. Introducing TabFM: A zero-shot foundation model for tabular data. Google Research Blog, June 2026. URL https://research.google/blog/introducing-tabfm-a-zer o-shot-foundation-model-for-tabular-data/. Vignesh Kothapalli, Rishabh Ranjan, Valter Hudovernik, Vijay Prakash Dwivedi, Johannes Hoffart, Carlos Guestrin, and Jure Leskovec. PluRel: Synthetic data unlocks scaling laws for relational foundation models. arXiv preprint arXiv:2602.04029, 2026. URL https://arxiv.org/abs/2602.04029. Zi-Rong Li, Si-Yang Liu, Tian-Zuo Wang, and Han-Jia Ye. TabCausal: Pretraining across causal environments for tabular causal discovery, 2026. URL https://arxiv.org/abs/2605.31156. LimiX Team. LimiX: Unleashing structured-data modeling capability for generalist intelligence. arXiv preprint arXiv:2509.03505, 2025. URL https://arxiv.org/abs/2509.03505. Si-Yang Liu, Hao-Run Cai, Qi-Le Zhou, and Han-Jia Ye. Talent: A tabular analytics and learning toolbox. arXiv preprint arXiv:2407.04057, 2024. Lars Lorch, Scott Sussex, Jonas Rothfuss, Andreas Krause, and Bernhard Schölkopf. Amortized inference for causal structure learning. In Advances in Neural Information Processing Systems, volume 35, 2022. Junwei Ma, Valentin Thomas, Rasa Hosseinzadeh, Alex Labach, Jesse Cresswell, Keyvan Golestan, Guangwei Yu, Anthony L Caterini, and Maks Volkovs. Tabdpt: Scaling tabular foundation models on real data. Advances in Neural Information Processing Systems, 38:172692–172722, 2026. Sergei Maslov and Kim Sneppen. Specificity and stability in topology of protein networks. Science, 296 (5569):910–913, 2002. Giovanni Menegozzo, Diego Dall’Alba, and Paolo Fiorini. CIPCaD-Bench: Continuous industrial process datasets for benchmarking causal discovery methods. In 2022 IEEE 18th International Conference on Automation Science and Engineering, pp. 2124–2131, 2022. Samuel Müller, Noah Hollmann, Sebastian Pineda Arango, Josif Grabocka, and Frank Hutter. Transformers can do bayesian inference. arXiv preprint arXiv:2112.10510, 2021. Ken M. Nakanishi. Scalable-Softmax is superior for attention. arXiv preprint arXiv:2501.19399, 2025. doi: 10.48550/arXiv.2501.19399. URL https://arxiv.org/abs/2501.19399. Nums AI Inc. Causilo: A pretrained tabular foundation model for classification and regression. https: //github.com/nums-ai/causilo, 2026. GitHub repository.

Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:27730–27744, 2022. Judea Pearl, Madelyn Glymour, and Nicholas P Jewell. Causal inference in statistics: A primer. John Wiley & Sons, 2016. Judea Pearl et al. Causality: models, reasoning, and inference. Econometric Theory, 19(675-685):46, 2003. Jonas Peters, Dominik Janzing, and Bernhard Schölkopf. Elements of causal inference: foundations and learning algorithms. The MIT press, 2017. Jingang Qu, David Holzmüller, Gaël Varoquaux, and Marine Le Morvan. TabICL: A tabular foundation model for in-context learning on large data. In Proceedings of the 42nd International Conference on Machine Learning, volume 267 of Proceedings of Machine Learning Research, pp. 50817–50847. PMLR, 2025. URL https://proceedings.mlr.press/v267/qu25d.html. Jingang Qu, David HolzmÃžller, Gaël Varoquaux, and Marine Le Morvan. Tabiclv2: A better, faster, scalable, and open tabular foundation model. In Forty-third International Conference on Machine Learning, 2026. Karen Sachs, Omar Perez, Dana Pe’er, Douglas A. Lauffenburger, and Garry P. Nolan. Causal proteinsignaling networks derived from multiparameter single-cell data. Science, 308(5721):523–529, 2005. Alberto Sanfeliu and King-Sun Fu. A distance measure between attributed relational graphs for pattern recognition. IEEE transactions on systems, man, and cybernetics, (3):353–362, 1983. Timo Schick, Jane Dwivedi-Yu, Roberto Dessı̀, Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. Toolformer: Language models can teach themselves to use tools. Advances in neural information processing systems, 36:68539–68551, 2023. Marco Scutari. Bayesian network repository, 2026. URL https://www.bnlearn.com/bnrepository/. Accessed 2026-09-14. Noam Shazeer. GLU variants improve transformer. arXiv preprint arXiv:2002.05202, 2020. URL https: //arxiv.org/abs/2002.05202. Shohei Shimizu, Patrik O. Hoyer, Aapo Hyvärinen, and Antti Kerminen. A linear non-gaussian acyclic model for causal discovery. Journal of Machine Learning Research, 7:2003–2030, 2006. Peter Spirtes, Clark Glymour, and Richard Scheines. Causation, Prediction, and Search. MIT Press, 2 edition, 2000. Yefan Tao, Xiyuan Zhang, Xinyi Liu, Boran Han, Danielle Maddix, Haoyang Fang, Zhen Han, Jiading Gai, Xuanqing Liu, Michael Bohlke-Schneider, Yuyang (Bernie) Wang, Gerald Friedland, Kevan Mah, Chris Lee, and Chris Kong. Mitra-v2 technical report. arXiv preprint arXiv:2609.04540, 2026. URL https://arxiv.org/abs/2609.04540. Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, et al. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805, 2023. Ashish Vaswani, Noam Shazeer, Niki Parmar, et al. Attention is all you need. In Advances in Neural Information Processing Systems, 2017. URL https://papers.nips.cc/paper/7181-attention-is-all -you-need. Penghui Wang, Wei Liu, Hong Wang, Chengyue Huang, Yuxi Sun, Zirui Wang, Hongming Huang, Quan Wang, Zhenwei Xin, Ping Hou, Jie Yu, Chunxiao Liu, Erli Meng, and Bin Wang. XiaomiTabLDM: A tabular foundation model technical report. arXiv preprint arXiv:2609.03880, 2026a. URL https://arxiv.org/abs/2609.03880. Yuanrui Wang, Xingxuan Zhang, Han Yu, Mingchao Hao, Gang Ren, Hao Yuan, Li Mao, Yunjia Zhang, Chun Yuan, and Peng Cui. Limix-2m: Mitigating low-rank collapse and attention bottlenecks in tabular foundation models. arXiv preprint arXiv:2606.04485, 2026b. Han-Jia Ye, Si-Yang Liu, Hao-Run Cai, Qi-Le Zhou, and De-Chuan Zhan. A closer look at deep learning on tabular data. arXiv preprint arXiv:2407.00956, 2024. URL https://arxiv.org/abs/2407.00956v1.

Han-Jia Ye, Huai-Hong Yin, De-Chuan Zhan, and Wei-Lun Chao. Revisiting nearest neighbor for tabular data: A deep tabular baseline two decades later. In The Thirteenth International Conference on Learning Representations, 2025. Yue Yu, Jie Chen, Tian Gao, and Mo Yu. DAG-GNN: DAG structure learning with graph neural networks. In Proceedings of the 36th International Conference on Machine Learning, pp. 7154–7163, 2019. Biao Zhang and Rico Sennrich. Root mean square layer normalization. In Advances in Neural Information Processing Systems, 2019. URL https://arxiv.org/abs/1910.07467. Xiyuan Zhang, Danielle Maddix Robinson, Junming Yin, Nick Erickson, Abdul Fatir Ansari, Boran Han, Shuai Zhang, Leman Akoglu, Christos Faloutsos, Michael W. Mahoney, Tony Hu, Huzefa Rangwala, George Karypis, and Yuyang (Bernie) Wang. Mitra: Mixed synthetic priors for enhancing tabular foundation models. In Advances in Neural Information Processing Systems, volume 38, 2025. doi: 10.52202/085713-0535. URL https://proceedings.neurips.cc/paper files/paper/2025/hash/1 77d68f4adef163b7b123b5c5adb3c60-Abstract-Conference.html. Bo Zheng, Yudong Chen, Zihua Xiong, Shuai Fang, Peidong He, Yang Yang, and Sheng Guo. MaskTab: Scalable masked tabular pretraining with scaling laws and distillation for industrial classification. arXiv preprint arXiv:2605.11408, 2026. URL https://arxiv.org/abs/2605.11408. Xun Zheng, Chen Dan, Bryon Aragam, Pradeep Ravikumar, and Eric Xing. Learning sparse nonparametric DAGs. In Silvia Chiappa and Roberto Calandra (eds.), Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics, volume 108 of Proceedings of Machine Learning Research, pp. 3414–3425. PMLR, 2020. URL https://proceedings.mlr.press/v108/zheng20a.html.

Record · ID 919427 · SHA-256 81cac82427da6f6b
Retrieved via Conceptio — every document is proof-bundled with source, license, and retrieval metadata.