arXiv CSopen access

Tabular foundation models for in-context prediction of molecular properties

2026 · arxiv_cs

arXiv CS · Papers · License: Open Access · 2026

machine learning, deep learning, neural networks

Tabular foundation models for in-context prediction of molecular properties Karim K. Ben Hicham1 , Jan G. Rittig1 , Martin Grohe2 , Alexander Mitsos4,1,3*

arXiv:2604.16123v1 [cs.LG] 17 Apr 2026

RWTH Aachen University, Process Systems Engineering (AVT.SVT), Aachen, Germany. 2 Lehrstuhl Informatik 7, RWTH Aachen University, Aachen, Germany. 3 Forschungszentrum Jülich GmbH, Institute of Climate and Energy Systems ICE-1: Energy Systems Engineering, Jülich, Germany. 4 JARA-CSD, Aachen, Germany.

*Corresponding author(s). E-mail(s): [email protected]; Abstract Accurate molecular property prediction is central to drug discovery, catalysis, and process design, yet real-world applications are often limited by small datasets. Molecular foundation models provide a promising direction by learning transferable molecular representations; however, they typically involve task-specific fine-tuning, require machine learning expertise, and often fail to outperform classical baselines. Tabular foundation models (TFMs) offer a fundamentally different paradigm: they perform predictions through in-context learning, enabling inference without task-specific training. Here, we evaluate TFMs in the low- to medium-data regime across both standardized pharmaceutical benchmarks and chemical engineering datasets. We evaluate both frozen molecular foundation model representations, as well as classical descriptors and fingerprints. Across the benchmarks, the approach shows excellent predictive performance while reducing computational cost, compared to fine-tuning, with these advantages also transferring to practical engineering data settings. In particular, combining TFMs with CheMeleon embeddings yields up to 100% win rates on 30 MoleculeACE tasks, while compact RDKit2d and Mordred descriptors provide strong descriptorbased alternatives. Molecular representation emerges as a key determinant in TFM performance, with molecular foundation model embeddings and 2D descriptor sets both providing substantial gains over classic molecular fingerprints on many tasks. These results suggest that in-context learning with TFMs provides a

highly accurate and cost-efficient alternative for property prediction in practical applications. Keywords: tabular foundation models, descriptors, chemistry

1 Introduction The ability to predict molecular properties is essential for data-driven decision-making in product, process, and catalyst design, where reliable estimates of biological, physicochemical, or quantum-mechanical properties are needed to prioritize candidates before costly experiments [1–3]. In practice, however, many relevant prediction problems are defined by small- to medium-sized datasets rather than by the large-scale regimes in which deep learning has achieved its greatest successes [2, 4, 5]. This data-limited setting remains one of the main challenges in molecular machine learning. Molecular foundation models have been proposed, in part, to address this challenge of low data availability: by pretraining on large, relatively inexpensive collections of unlabeled molecular strings, structures, or related datasets, they aim to learn transferable chemical representations that can support accurate prediction on heterogeneous downstream tasks with only limited labeled data [6–9]. Yet in practice, their downstream use depends on task-specific fine-tuning, which is prone to overfitting [10], can be sensitive to hyperparameter choices, and still often fails to consistently outperform strong classical baselines such as random forests or gradient boosting tree models trained on fixed molecular fingerprints [9, 11, 12]. The problem is therefore not only how to pretrain informative molecular foundation models, but how to harness the information they contain robustly and efficiently for small downstream datasets. Another, arguably more practical, limiting factor for broad real-world adoption is the required ML expertise and computational budget. In this context, recent advances in tabular foundation models (TFMs), such as TabPFN [13, 14] and TabICL [15, 16], are highly promising. TFMs are pretrained on a large variety of purely synthetic datasets generated via structural causal models (SCMs) and then perform in-context learning: training-free, direct prediction of missing labels in newly provided tabular datasets, enabling inference without task-specific fine-tuning. These models have achieved state-of-the-art performance on comprehensive benchmarks of small- to medium-sized tabular datasets [17], and exhibit substantial generalization capabilities [13, 15, 18]. This improved performance-per-compute trade-off suggests a promising alternative to conventional fine-tuning workflows in small- to medium-sized molecular datasets. Molecular representations can be computed as descriptors or fingerprints, or obtained as pretrained embeddings; TFMs can then serve as downstream predictors with extremely low computational cost and state-of-the-art performance [13, 16, 19]. This avoids the computational cost and complexity of fine-tuning molecular foundation models for every new task.

b. CheMeleon, CLAMP, SMI -TED

Dataset

Pretrained Embeddings 𝑦test = TFM

xtrain

𝑦train

xtest

? Sampled Functions

1D column attention

1D sample attention

ABCIndex, PSA, xLogP, … Fingerprint bits

Sampled Causal Graphs

test

Mordred, Morgan

train

Classical Features

Pretrained on synthetic tabular tasks to perform in-context Bayesian prediction.

Fig. 1 Combining tabular foundation models (TFMs) with diverse molecular representations for property prediction: (a) Molecular representations used in this study, including pretrained embeddings from frozen foundation models (CheMeleon, CLAMP, and SMI-TED) and classical features (RDKit2d and Mordred descriptors, and Morgan fingerprints). (b) In-context prediction with a tabular foundation model (TFM). For each task, the training inputs and labels are provided together with the unlabeled test input at inference time, and the model predicts the missing target without task-specific training or fine-tuning. (c) Schematic of the TFM architecture. The model first applies one-dimensional attention over feature columns and then over samples, enabling joint processing of training and test examples. The resulting representations are passed to a multilayer perceptron (MLP; not shown), which outputs either class probabilities for classification or predictive quantiles for regression. (d) Synthetic-data pretraining paradigm for TFMs. TFMs used in this work are pretrained exclusively on large collections of synthetic tabular tasks generated from sampled functions and structural causal models. These expose the model to diverse statistical structures during pretraining, enabling amortized Bayesian inference on new datasets through in-context prediction alone. Subplots in d adapted from [15].

We investigate the use of TFMs for molecular property prediction in the low- to medium-data regime by combining them with a diverse set of molecular representations, illustrated in Figure 1. Specifically, we combine both TabPFN and TabICL with compact RDKit2d descriptors [20], larger Mordred descriptors [21], Morgan fingerprints [20], and expressive pretrained embeddings from CheMeleon [9], SMI-TED [22], and CLAMP [23], thereby yielding 12 featurizer-model configurations, covering both mechanistic and foundation-model-derived feature spaces. We benchmark against the best prior molecular baselines [7, 9, 24, 25] and strong classical ML models, including XGBoost [26] and CatBoost [27] on 58 datasets from the Polaris and MoleculeACE benchmark suites [9, 12, 28, 29], with a particular focus on the small- to medium-sized (up to ∼6000 samples) datasets that characterize many realistic applications, where data collection is expensive. To further test TFMs in practical scenarios, we evaluate and compare them with highly tuned and domain specialized, state-of-the-art literature baselines across 11 practical engineering datasets, including fuel ignition properties [30, 31], polymer properties [32], and polymer-solvent interactions [33], thereby extending the analysis beyond predominantly pharmaceutical and bioactivity-focused public benchmarks.

This is the first demonstration that frozen molecular foundation model embeddings combined with tabular foundation models can outperform both classical machine learning and advanced fine-tuned deep learning foundation models across diverse molecular property prediction benchmarks. TabPFN combined with expressive representations consistently outperforms prior state-of-the-art methods, including fine-tuned molecular foundation models such as CheMeleon, with up to 100% win rates, defined as best or statistically indistinguishable from the best. In contrast, prior work [34], which used an older version of TabPFN primarily as a probe to identify pretrained foundation model layers suitable for fine-tuning, found that although its performance was generally correlated with downstream results across most tasks, it remained substantially weaker than fine-tuning in absolute terms. We further show that, in contrast to recent work evaluating TabPFN for drug discovery [19], which indicated that “TabPFN is largely invariant to representation choice”, we find that the choice of molecular representation is a major driver of performance, with CheMeleon embeddings and 2D descriptor sets performing most favorably and Morgan fingerprints performing substantially worse. At the same time, TFMs require substantially less compute time in our runtime case study, with speedups of up to 27× on CPU and 46× on GPU, depending on dataset size and using 2048-dimensional CheMeleon embeddings (the largest feature set considered). The same performance trend also extends to the engineering datasets, where TFM-based models remain competitive with highly hyperparameter-tuned literature baselines. Our contribution demonstrates that in-context prediction with TFMs and suitable molecular representations provides a simple, accurate, and cost-efficient approach to real-world molecular property prediction.

2 Results 2.1 Benchmarking on Polaris and MoleculeACE The Polaris [35] and MoleculeACE [12] benchmark suites provide highly representative property prediction tasks in the low- to medium-data regime, comprising 58 tasks in total. Polaris covers a broad range of practically relevant endpoints, including solubility, physiology, and biophysics, whereas MoleculeACE focuses on activity cliffs, a particularly challenging setting in which structurally similar molecules exhibit large differences in activity [12]. We evaluate tabular foundation models (TFMs) with molecular descriptors and pretrained embeddings in these public benchmarks and compare them to various baselines, reporting win rates and average ranks across tasks. All tasks are evaluated on expert-curated fixed train-test splits across five random seeds; see Section 4 for details. Across the combined benchmark suite, shown in Table 1, TFM-based approaches achieve stronger aggregate performance than the compared classical baselines and previously reported molecular foundation models. The best overall configuration, TabPFN-CheMeleonFP (TabPFN with frozen CheMeleon embeddings), achieves 50 wins out of 58 tasks, corresponding to an 86.2% win rate and an average rank of 4.52. Among descriptor-based methods, TabPFN-RDKit2d achieves the next-best average 4

Table 1 Aggregate performance across the Polaris and MoleculeACE benchmark suites (58 tasks total) for a subset of models evaluated on both benchmarks. Model TabPFN-CheMeleonFP TabPFN-RDKit2d TabPFN-Mordred TabICL-RDKit2d TabICL-CheMeleonFP TabICL-Mordred CheMeleon TabICL-CLAMP TabPFN-CLAMP minimol CatBoost-Mordred TabPFN-Morgan RF Morgan fastprop MLP PLR Pretrained RF Mordred TabICL-Morgan MoLFormer PCA MLP Prefitted XGBoost-Mordred Chemprop TabICL-SMI-TED TabPFN-SMI-TED MolCLR

Win Count

Win Rate (%)

Average Rank

50 33 39 32 43 22 24 25 22 19 11 13 10 5 6 6 8 6 5 6 2 4 3 1

86.2 56.9 67.2 55.2 74.1 37.9 41.4 43.1 37.9 32.8 19.0 22.4 17.2 8.6 10.3 10.3 13.8 10.3 8.6 10.3 3.4 6.9 5.2 1.7

4.52 5.31 5.71 5.90 6.31 8.35 8.84 8.86 8.91 11.43 12.05 12.98 15.12 15.98 16.60 17.50 17.64 18.36 18.74 19.53 21.74 24.09 24.45 25.81

Wins denote tasks on which a model is best or statistically indistinguishable from the best by Tukey’s HSD test over five random seeds; Win Rate gives the corresponding percentage of tasks. Average Rank is obtained by averaging within-dataset model ranks across all tasks, with lower values indicating stronger performance. All evaluations use the original expert-curated train-test splits and benchmark-specific metrics. See Section 4 for evaluation and statistical details.

rank, with a 56.9% win rate and average rank of 5.31, while TabPFN-Mordred achieves a higher 67.2% win rate and an average rank of 5.71. TabICL-CheMeleonFP also remains strong, with a 74.1% win rate and average rank of 6.31. By comparison, the fine-tuned CheMeleon model reaches a win rate of 41.4% and an average rank of 8.84 under the matched split and evaluation protocol [9]. Thus, pairing TFMs with frozen molecular representations not only matches but substantially exceeds the performance of task-specific fine-tuning on these benchmarks. The benchmark-wise analyses in Fig. 2 show that this aggregate result is driven by gains on both suites. On Polaris, the highest win rates are achieved by TabPFNCheMeleonFP and TabICL-CheMeleonFP, each with 20 wins out of 28 tasks (71.4%) under the win-rate definition above, followed by TabPFN-Mordred with 19 wins (67.9%) and TabPFN-RDKit2d with 18 wins (64.3%). By average rank, the leading methods are TabPFN-RDKit2d (5.43), TabICL-RDKit2d (6.54), TabPFN-Mordred (6.75), and TabPFN-CheMeleonFP (7.11), compared with 10.25 for CheMeleon and

11.00 for minimol [9]. On MoleculeACE, the separation is even clearer: TabPFNCheMeleonFP is best or statistically tied for best on all 30 tasks (100.0%) and achieves an average rank of 2.10, followed by TabICL-CheMeleonFP at a 76.7% win rate and average rank of 4.00, and TabPFN-Mordred at 66.7% and 4.73. RDKit2d variants are also competitive on MoleculeACE, with TabICL-RDKit2d reaching a 53.3% win rate and average rank of 5.30 and TabPFN-RDKit2d reaching 50.0% and 5.20. CheMeleon reaches a 36.7% win rate and average rank of 7.53 on MoleculeACE, despite having been reported as highly competitive relative to earlier baselines [9]. Notably, twelve methods record zero wins on MoleculeACE, highlighting the difficulty of this benchmark. Despite TFMs strong general performance compared to CheMeleon, they still do not perform equally well for cliff and non-cliff molecules, with details given in Supplementary Fig. S3. Benchmark-specific aggregate tables are provided in Supplementary Tables S3 and S4, with the corresponding Tukey HSD summaries in Supplementary Figs. S1 and S2. Furthermore, consistent trends emerge in relation to TFM model choice and molecular representation. TabPFN often outperforms TabICL for the same representation, most clearly for CheMeleonFP, RDKit2d, and Mordred, in agreement with recent results on general tabular benchmarks [17]. Nevertheless, TabICL remains highly competitive: when paired with CheMeleon embeddings, it still outperforms or matches most previously reported baselines and remains close to the best-performing methods overall. The molecular representation is another major determinant of downstream performance. The strongest results are obtained with CheMeleon-derived embeddings and 2D descriptor sets, including both RDKit2d and Mordred, whereas Morgan fingerprint variants perform clearly worse. For example, TabPFN-Morgan reaches only a 22.4% win rate in the combined benchmark and has substantially lower aggregate performance than TabPFN-CheMeleonFP, TabPFN-RDKit2d, and TabPFN-Mordred. Thus, the gains of TFMs are not representation-agnostic but depend strongly on sufficiently expressive molecular features. Restricted feature-set comparisons, including RDKit2d-MACCS variants, are reported separately for Polaris and MoleculeACE in Supplementary Tables S6 and S5. It is particularly noteworthy that the configurations combining TFMs with CheMeleon-derived embeddings outperform the fine-tuned version of the CheMeleon model itself. This emphasizes the predictive capabilities of TFMs when paired with expressive representations, further suggesting that task-specific fine-tuning of large foundation models is not always necessary to obtain strong results on small- to medium-sized downstream datasets. The runtime analysis in Fig. 2b further shows that these performance gains are accompanied by substantially lower runtime cost. Across representative Polaris datasets spanning the smallest (adme-fang-rppb-1, ntrain = 111, ntest = 24), median (caco2-wang, ntrain = 728, ntest = 182), and largest (ld50-zhu, ntrain = 5907, ntest = 1478) training-set sizes, TabPFN-CheMeleonFP is consistently faster than finetuned CheMeleon on both GPU and CPU, yielding speedups of 4.8×–27.3× on CPU and 18.3×–46.0× on GPU. This is consistent with previously reported computational efficiency of TFMs [15, 18].

Polaris

MoleculeACE

Fig. 2 Performance ranking, win rates, and runtime across molecular property prediction benchmarks. (a) Win rates for Polaris and MoleculeACE, defined as the percentage of tasks on which a model is best or statistically indistinguishable from the best using Tukey’s honestly significant difference test. On Polaris, TabPFN-CheMeleonFP and TabICL-CheMeleonFP each reach 71.4%, followed by TabPFN-Mordred (67.9%) and TabPFN-RDKit2d (64.3%), versus 46.4% for CheMeleon; on MoleculeACE, TabPFN-CheMeleonFP reaches 100.0%, followed by TabICL-CheMeleonFP (76.7%), TabPFN-Mordred (66.7%), TabICL-RDKit2d (53.3%), TabPFN-RDKit2d (50.0%), and CheMeleon (36.7%). TabPFN models are shown in dark blue, TabICL models in light blue, and previously reported baselines from the benchmark-matched comparison of [9] are shown in orange. (b) Runtime decomposition for three representative Polaris datasets used as an illustrative case study. Total runtime is separated into feature computation, fitting or training, and prediction, and compared between TabPFN-CheMeleonFP and fine-tuned CheMeleon on both GPU and CPU, with speedups of 4.8×– 27.3× on CPU and 18.3×–46.0× on GPU. (c) Critical-difference diagram using the Nemenyi post hoc test following Friedman’s test at a significance level of α = 0.05 for Polaris (28 tasks). Lower rank indicates better mean performance. Horizontal bars span models whose ranks are not significantly different in the post hoc comparison. (d) Critical-difference diagram for MoleculeACE (30 tasks), with the same interpretation as in c.

Overall, the benchmark results establish TFMs, especially TabPFN paired with strong molecular representations, as a simple and effective alternative to classical molecular property prediction methods and recent fine-tuning approaches based on molecular foundation models.

2.2 Chemical Engineering Datasets Real-world chemical engineering datasets differ in important ways from the benchmarks considered above. In particular, Polaris and MoleculeACE, while fairly broad and highly valuable as controlled test beds, are centered largely on pharmaceutical and bioactivity-related endpoints. To test whether the strong benchmark performance of TFMs transfers beyond these applications, we therefore evaluate the same approach on three representative engineering-oriented applications: fuel ignition-property prediction, single-property polymer prediction, and polymer-solvent interaction prediction. These datasets certainly also fall in the low-data regime similar to the previous benchmarks. Moreover, they introduce practical challenges that are common in engineering problems, including diverse physicochemical targets, inputs involving more than one molecule (i.e., molecular mixtures), and additional numerical input features, such as temperature. As above, we combine out-of-the-box TFMs with fixed molecular representations and compare them with highly tuned and specialized, state-of-the-art literature baselines under the evaluation protocols used in the respective studies, with results summarized in Table 2; detailed fuel, polymer, and polymer-solvent tables are provided in Supplementary Tables S7, S8, and S9. Reported literature performances are referenced with their respective sources in the tables. Because several of the used literature baselines were not released with reliable fold-wise raw metrics, we interpret these experiments as contextual transfer tests rather than as a fully controlled benchmark. Aggregate performance on the engineering tasks and per-task Tukey HSD plots are provided in Supplementary Table S10 and Supplementary Fig. S5. The fuel ignition benchmarks evaluate how well models can predict experimentally measured combustion-quality indicators such as DCN, RON, and MON, which are central to fuel design because they directly reflect ignition and anti-knock performance in engines [30]. On these tasks, TabPFN-Mordred achieves the lowest mean RMSE across ten folds on the DCN and RON benchmarks. TabICL-Mordred achieves the lowest mean RMSE on the MON task. The GNN MTL-Train [31] exploits correlations between fuel targets, whereas our methods are evaluated as single-task models, and it performs slightly worse on average than CheMeleon. The polymer-property benchmarks cover a diverse set of electronic (Eea, Egb, Egc, Ei), dielectric (EPS, Nc), and structural (Xc) targets, which are frequently required in the design of novel polymers [32, 36, 37]. This makes them important test cases for assessing whether models can generalize across practically relevant material properties in low-data settings. Polymers are featurized via their repeat unit SMILES. Here, TFM-based models, especially TabPFN-Mordred and the RDKit2d variants, match or exceed the strongest reported baselines on several targets, yielding the best mean performance, or tying the best reported literature value, on Eea, Egb, Egc, Ei, and

Table 2 Model performance on the chemical engineering datasets. Fuels

Polymers RMSE ↓ MON

R2 ↑ Eea

R2 ↑ Egb

R2 ↑ Egc

RMSE ↓ DCN

RMSE ↓ RON

TabPFN-Mordred TabPFN-RDKit2d TabICL-RDKit2d TabICL-Mordred TabPFN-CheMeleonFP CatBoost-Mordred CheMeleon TabICL-CheMeleonFP CatBoost-RDKit2d CatBoost-CheMeleonFP TabPFN-Morgan TabICL-Morgan RF-Mordred XGBoost-Mordred RF-RDKit2d CatBoost-Morgan XGBoost-RDKit2d RF-CheMeleonFP RF-Morgan XGBoost-Morgan XGBoost-CheMeleonFP Chemprop GNN

5.97 ± 1.69 7.39 ± 1.75 6.81 ± 1.68 6.59 ± 1.30 8.22 ± 1.97 9.06 ± 1.76 7.35 ± 1.95 8.60 ± 1.43 9.56 ± 2.01 11.11 ± 2.32 11.24 ± 2.47 11.77 ± 2.40 11.83 ± 1.34 10.68 ± 2.16 12.22 ± 2.03 11.73 ± 2.55 11.13 ± 1.71 12.50 ± 1.97 13.38 ± 2.10 12.30 ± 2.45 12.83 ± 1.92 13.39 ± 2.12

8.57 ± 1.78 9.43 ± 1.99 9.25 ± 1.82 9.02 ± 2.26 9.68 ± 2.16 9.12 ± 1.70 9.48 ± 1.80 9.89 ± 1.91 10.03 ± 1.82 10.13 ± 2.26 11.81 ± 2.27 11.99 ± 2.85 10.44 ± 1.64 10.77 ± 1.35 10.84 ± 1.84 12.01 ± 2.09 11.54 ± 1.75 11.22 ± 2.42 13.03 ± 1.91 12.87 ± 1.88 12.33 ± 3.03 11.89 ± 2.10

7.35 ± 1.62 0.93 ± 0.01 0.93 ± 0.01 0.93 ± 0.00 7.88 ± 2.42 0.92 ± 0.01 0.93 ± 0.01 0.92 ± 0.01 7.67 ± 2.59 0.93 ± 0.01 0.92 ± 0.02 0.92 ± 0.01 7.20 ± 1.69 0.93 ± 0.01 0.92 ± 0.02 0.91 ± 0.01 8.01 ± 2.22 0.91 ± 0.01 0.91 ± 0.02 0.92 ± 0.01 8.12 ± 1.97 0.90 ± 0.02 0.91 ± 0.01 0.91 ± 0.01 7.94 ± 2.36 0.91 ± 0.01 0.92 ± 0.01 0.91 ± 0.01 8.28 ± 2.00 0.91 ± 0.01 0.91 ± 0.02 0.90 ± 0.01 8.39 ± 2.09 0.87 ± 0.02 0.89 ± 0.03 0.90 ± 0.01 9.31 ± 2.43 0.87 ± 0.02 0.88 ± 0.03 0.89 ± 0.01 10.25 ± 2.72 0.90 ± 0.01 0.88 ± 0.02 0.87 ± 0.01 10.02 ± 3.11 0.90 ± 0.02 0.89 ± 0.01 0.84 ± 0.02 9.26 ± 2.14 0.86 ± 0.03 0.88 ± 0.03 0.89 ± 0.01 9.86 ± 2.04 0.84 ± 0.04 0.88 ± 0.03 0.89 ± 0.01 9.88 ± 2.19 0.82 ± 0.02 0.89 ± 0.02 0.88 ± 0.01 10.91 ± 2.50 0.88 ± 0.02 0.87 ± 0.03 0.87 ± 0.01 10.18 ± 2.08 0.83 ± 0.01 0.88 ± 0.03 0.88 ± 0.01 10.47 ± 2.50 0.82 ± 0.04 0.86 ± 0.05 0.87 ± 0.01 11.86 ± 2.92 0.83 ± 0.02 0.85 ± 0.05 0.87 ± 0.01 11.89 ± 1.93 0.85 ± 0.02 0.87 ± 0.04 0.87 ± 0.01 10.75 ± 2.75 0.80 ± 0.06 0.87 ± 0.04 0.88 ± 0.01 12.50 ± 3.13 0.82 ± 0.04 0.86 ± 0.02 0.90 ± 0.01

Literature baselines MTL-Train [31] STL [31] PolyCL [36] TransPolymer [37] PolyBERT [32] D-MPNN-TC [33]

7.52 ± 1.84 7.51 ± 1.86 – – – –

9.46 ± 1.72 10.29 ± 2.16 – – – –

8.47 ± 1.95 8.83 ± 2.19 – – – –

Metric Model

– – 0.91 0.89 0.91 –

– – 0.89 0.90 0.88 –

– – 0.88 0.88 0.88 –

PolySolv

R2 ↑ Ei

R2 ↑ EPS

R2 ↑ Nc

0.81 ± 0.02 0.80 ± 0.03 0.81 ± 0.03 0.81 ± 0.03 0.79 ± 0.01 0.80 ± 0.03 0.76 ± 0.01 0.78 ± 0.03 0.78 ± 0.02 0.77 ± 0.03 0.78 ± 0.03 0.76 ± 0.03 0.76 ± 0.05 0.76 ± 0.06 0.76 ± 0.05 0.77 ± 0.03 0.74 ± 0.06 0.75 ± 0.04 0.76 ± 0.03 0.73 ± 0.03 0.75 ± 0.02 0.67 ± 0.05

0.78 ± 0.08 0.77 ± 0.07 0.75 ± 0.06 0.77 ± 0.06 0.77 ± 0.07 0.75 ± 0.08 0.72 ± 0.07 0.75 ± 0.05 0.72 ± 0.08 0.73 ± 0.05 0.76 ± 0.05 0.77 ± 0.04 0.71 ± 0.08 0.66 ± 0.09 0.69 ± 0.08 0.75 ± 0.05 0.67 ± 0.10 0.69 ± 0.08 0.74 ± 0.04 0.72 ± 0.05 0.66 ± 0.07 0.65 ± 0.10

0.85 ± 0.06 0.85 ± 0.06 0.84 ± 0.04 0.84 ± 0.03 0.84 ± 0.05 0.83 ± 0.05 0.82 ± 0.04 0.82 ± 0.04 0.80 ± 0.06 0.81 ± 0.03 0.83 ± 0.04 0.83 ± 0.02 0.79 ± 0.04 0.77 ± 0.07 0.77 ± 0.06 0.82 ± 0.05 0.75 ± 0.08 0.78 ± 0.03 0.79 ± 0.04 0.78 ± 0.05 0.77 ± 0.03 0.72 ± 0.06

– – 0.81 0.79 0.77 –

– – 0.79 0.76 0.77 –

– – 0.85 0.81 0.80 –

R2 ↑ Xc

R2 ↑ χ

0.39 ± 0.16 0.88 ± 0.05 0.40 ± 0.13 0.93 ± 0.03 0.41 ± 0.14 0.93 ± 0.03 0.39 ± 0.12 0.82 ± 0.06 0.41 ± 0.11 0.81 ± 0.08 0.38 ± 0.16 0.90 ± 0.03 0.22 ± 0.18 0.83 ± 0.08 0.40 ± 0.11 0.71 ± 0.10 0.36 ± 0.18 0.90 ± 0.03 0.41 ± 0.09 0.88 ± 0.04 0.38 ± 0.12 0.88 ± 0.04 0.37 ± 0.14 0.87 ± 0.04 0.37 ± 0.10 0.82 ± 0.05 0.25 ± 0.20 0.87 ± 0.04 0.38 ± 0.13 0.85 ± 0.03 0.35 ± 0.14 0.79 ± 0.06 0.30 ± 0.19 0.88 ± 0.04 0.39 ± 0.06 0.74 ± 0.08 0.33 ± 0.16 0.79 ± 0.04 0.29 ± 0.18 0.77 ± 0.07 0.29 ± 0.10 0.80 ± 0.06 0.22 ± 0.06 0.72 ± 0.09 – – 0.40 0.46 0.44 –

– – – – – 0.93 ± 0.03

Values are given as mean ± standard deviation where available, aggregated over ten folds for the fuels and PolySolv datasets and over five folds for the polymer dataset. Fuel tasks are evaluated by RMSE, where lower values are better; polymer and PolySolv tasks are evaluated by R2 , where higher values are better. Bold entries indicate the best displayed mean performance in each task; if displayed means are tied, the displayed standard deviation is used as a tie-breaker, and entries with identical displayed mean and standard deviation are all bold. Literature values without fold-wise standard deviations are treated as having the lowest standard deviation for this tie-breaking only.

Nc while remaining competitive on EPS. Xc is the only target on which the TFMbased models do not match the best reported literature performance, although they still remain competitive with it. The polymer–solvent interaction benchmark (PolySolv) focuses on predicting Flory–Huggins χ interaction parameters for polymer–solvent pairs, a challenging and practically important task because miscibility strongly influences formulation, processing, and polymer product design. In addition to the repeat-unit SMILES and the solvent SMILES, the models also receive temperature and volume fraction as additional input features. On the PolySolv benchmark, the highly tuned and specialized D-MPNN-TC baseline of Liao et al. [33] achieves the top reported R2 , while TabPFNRDKit2d matches this value within rounding and the untuned CatBoost-Mordred baseline and several other TFM variants achieve accuracy close to it. Figure 3 shows a Pareto plot of the relative RMSE gap to the best model on a task, best versus the average runtime per 1,000 total train and test defined as RMSE−RMSE RMSE samples, averaged across the 11 engineering tasks. Both runtime and relative RMSE gap were computed fold-wise under the respective cross-validation protocol and then averaged across folds. The x-axis reports runtime per 1,000 samples, defined as model fitting or training plus prediction, excluding feature computation. Feature computation is especially costly for Mordred descriptors, which places the frozen foundation model embeddings and the smaller RDKit2d descriptor set at a practical advantage.

Fig. 3 Average runtime per 1,000 samples across the 11 engineering tasks and folds, measured as training or fitting plus inference for 1,000 samples and excluding feature computation, plotted against relative RMSE gap. Relative RMSE gap is defined as the RMSE reduction relative to the best method on that task.

The resulting plot reveals a clear Pareto front in the trade-off between computational cost and relative predictive performance. In particular, XGBoost occupies the extreme low-cost end, delivering very fast predictions, whereas TabPFN combined with RDKit2d descriptors and, especially, Mordred descriptors achieves the strongest overall performance. By contrast, the fine-tuned CheMeleon baseline has an average relative RMSE gap that is about 10 percentage points larger while also being an order of magnitude slower. Overall, our results suggest that the strong performance of TFMs is not limited to the controlled Polaris and MoleculeACE benchmarks with a predominantly biological or pharmaceutical focus, but can also carry over to engineering-relevant molecular prediction tasks. Across polymer, polymer-solvent, and fuel datasets, the TFM-based pipeline remains competitive with substantially more specialized baselines, supporting its use as a simple and effective approach in realistic engineering applications.

3 Discussion We investigate a new paradigm for molecular property prediction: TFMs paired with frozen molecular foundation model embeddings and 2D descriptor sets, thereby enabling in-context learning. Across two representative benchmark suites including datasets from a few hundred to several thousand property data points, the TFMbased approach clearly outperforms the evaluated classical prediction methods and

fine-tuned molecular foundation models, in terms of aggregate predictive performance, while being substantially faster. The strong performance also transfers to practical chemical engineering datasets, where TFMs are competitive with extensively hyperparameter-tuned, domain-specialized models from the literature. The practical significance of our results is straightforward: because small- to medium-sized datasets are not only the most common but also extremely practically/industrially relevant setting in molecular property prediction, TFMs combined with in-context learning and rich molecular representations provide an important advance by improving both predictive performance and efficiency while also being easy to deploy. In many real-world projects, the bottleneck is not only predictive performance, but the time, compute, and specialized expertise needed to build and maintain task-specific deep learning pipelines. The presented approach reduces this burden to a simple two-step procedure: compute a high-quality molecular representation once, then use a pretrained tabular foundation model as the predictor. This yields a cheaper, faster, and more accessible workflow for applications such as drug discovery, catalysis, and process design. Our findings also have broader implications for how molecular foundation models should be used and evaluated. Much recent work has focused on end-to-end downstream fine-tuning as the default way to extract value from pretrained models. Our results suggest that, at least in the small- to medium-data regime, the most effective way to leverage molecular foundation model representations may not be through gradient-based fine-tuning, which prior work has shown can overfit [5], but rather through pairing frozen representations with in-context learning and TFMs. This shifts the emphasis from task-specific adaptation toward representation quality, predictor robustness, and workflow efficiency. In that sense, TFMs provide a practical mechanism for turning molecular foundation model embeddings into strong predictors without requiring the additional cost and fragility of per-task optimization. At the same time, our study has substantial room for improvement. First, our benchmarks focus primarily on small- to medium-sized supervised datasets, and it remains to be established how well the observed advantages persist for substantially larger datasets, where end-to-end training becomes more competitive. Although we also explored larger-scale datasets, these experiments became hardware-constrained for larger feature sets, making them difficult to benchmark, even though the smaller RDKit2d features continued to run without issues. Specifically, they became memorylimited without disk offloading and disk-limited with disk offloading (∼ 50 GB per fold), such that the practical benefits of TFM-based pipelines over standard deep learning fine-tuning, especially simplicity, ease of use, and speed, were substantially reduced. As detailed in Supplementary Information Section S3.1, the Brouwer activity-coefficient dataset comprises approximately 21,000 measurements, and performance remains strong for compact pair representations such as RDKit2d, whereas higher-dimensional representations start to approach practical limits. Second, the present work considers primarily single-molecule prediction tasks; extensions to chemically more complex settings, such as many-component mixtures, remain open and potentially challenging, since TFM computation scales considerably with feature dimension [13, 15]. Third, we have focused on out-of-the-box use of TFMs without

changing any inference hyperparameters. A simple way to boost performance is to increase the number of estimators for TabPFN, because TabPFN subsamples features for each estimator; see Supplementary Information Section S1 and Supplementary Table S1. These limitations define several promising directions for future research. It will be important to test the approach systematically on larger datasets, multimolecular systems, and tasks where structured inductive biases or physical constraints matter [3]. In addition, we have only considered a single set of frozen representations per model; hybrid strategies combining in-context prediction with ensembling [38] and fine-tuning where necessary could yield further gains, mostly at the cost of extra compute. Another interesting extension is multi-task prediction. TabPFN and TabICL are designed and pretrained for single-task prediction and can only indirectly exploit cross-task dependencies without again resorting to gradient-based fine-tuning [39]. Native support for in-context multitask learning [40] in future TFMs would therefore be highly valuable, because many applications require to predict multiple targets at the same time with shared underlying physics or biology. Prior work has shown that uncertainty estimates from TFMs can be surprisingly well calibrated out-of-the-box [14]. This opens the door to combining TFMs with uncertainty-aware decision-making strategies, such as active learning and Bayesian optimization [41], in applications like solvent design, where experimental data are limited and data-efficient exploration is especially valuable. More broadly, our results motivate the development of molecular ML pipelines that optimize not only for raw accuracy but also for computational cost, simplicity, and reliability under realistic deployment constraints. We expect TFMs to be valuable both as a practical route to deployment in realworld applications and as a strong baseline for assessing the predictive performance of pretrained foundation models in chemistry.

4 Methods 4.1 Tabular In-Context Learning TFMs are transformer-based models that, in the versions used here, are pretrained exclusively on large collections of synthetic tabular tasks generated from randomly sampled functions and structural causal models (SCMs) [14, 15, 42]. During pretraining, tasks are sampled continuously rather than reused, exposing the model to a broad range of statistical dependencies and causal structures. Formally, if a prediction task T = (Dtrain , Dtest ) is sampled from an SCM-induced prior p(T ), the model parameters are optimized according to

L(θ) = ET ∼p(T ) [− log qθ (ytest | xtest , Dtrain )] .

(1)

This objective trains the model to predict the label of a query example conditional on a set of in-context training examples, rather than through task-specific parameter updates. At inference, the labeled training inputs and outputs for a benchmark task are presented together with an unlabeled test input, and the TFM predicts the missing target directly in context, without retraining or fine-tuning. 12

TFMs process each input table using one-dimensional sequential attention over feature columns and then over samples. The resulting latent representations are passed to a multilayer perceptron head, which outputs class probabilities for classification tasks and predictive quantiles or binned densities for regression tasks. This synthetic pretraining paradigm can be interpreted as amortizing or learning Bayesian inference over a broad prior family of tabular problems, enabling the model to adapt to new datasets from examples alone [42]. For architectural and theoretical details, we refer the reader to [13, 15, 43]. Recent iterations have substantially increased the scale and diversity of pretraining tasks, leading to strong empirical performance on public tabular benchmarks such as TabArena [15, 17]. In the present study, the TFMs are used strictly in in-context inference mode with fixed molecular feature vectors; the TFMs are never retrained or fine-tuned on any molecular benchmark and are not exposed to molecule-specific data during pretraining.

4.2 Molecular Representations The classical molecular representations used in this study are RDKit2d descriptors [20], 2D Mordred descriptors [21], and Morgan fingerprints (radius 2, 2048 bits). In supplementary feature comparisons, we additionally evaluate RDKit2d descriptors concatenated with MACCS keys. The frozen pretrained embeddings are computed with CheMeleon, SMI-TED [22], and CLAMP [23]. CheMeleon is a descriptor-pretrained molecular foundation model based on a directed message-passing neural network and trained to predict deterministic Mordred descriptors, thereby learning transferable molecular embeddings for downstream property prediction [9]. Throughout the manuscript, CheMeleonFP denotes the fixed embedding returned by the released CheMeleon message-passing network before task-specific fine-tuning. SMI-TED is a large-scale SMILES encoder-decoder foundation model trained with self-supervision on PubChem molecules to produce general-purpose molecular embeddings [22]. CLAMP is a contrastive multimodal model that aligns molecular structure with assay text descriptions and yields transferable molecular embeddings for low-data assay activity prediction [23]. The corresponding feature dimensions of each model are listed in Supplementary Table S2.

4.3 Models We evaluate TabPFN-2.6 [13, 18, 44] and TabICL [15] with the above-mentioned classical and frozen foundation model representations. We compare these results with previously reported classical baselines and fine-tuned molecular foundation models, including CheMeleon [9], minimol [25], MolFormer [7], and others evaluated by [9], using the released predictions from their GitHub repository. In addition, we extend the benchmarks to include untuned XGBoost and CatBoost with Mordred descriptors [26, 27]. In the engineering datasets, we also combine the classical tabular ML baselines with foundation model embeddings.

4.4 Preprocessing All featurization and scaling steps are fit on the training portion of each dataset only and then applied unchanged to the test set. Descriptor features and frozen embeddings are all standardized using the training-set feature means and standard deviations and then clamped to the range [−6, 6] to limit the effect of extreme values, following [9]. The considered TFMs apply further normalization or outlier removal steps, which we leave in place to assess out-of-the-box performance without changes to hyperparameters. RDKit2d descriptors are computed from the RDKit descriptor list. Mordred descriptors are computed with the 2D Mordred descriptor set (ignore 3D=True); Morgan fingerprints are computed directly from the parsed RDKit molecule. CheMeleonFP features are generated from the released frozen CheMeleon model, with Chemprop version 2.2.1 [9, 24]. For regression tasks, targets are standardized using the training-set mean and variance and predictions are transformed back to the original scale before benchmark evaluation. For Polaris classification tasks, predicted probabilities are used directly for scoring, with hard labels obtained by thresholding at 0.5 when required by the benchmark interface [9].

4.5 Benchmarks To enable a direct and fair comparison with prior work, we evaluate our approach across the same benchmark suites used in previous studies, namely Polaris and MoleculeACE [9, 12, 28, 29]. These comprise 28 and 30 datasets, respectively, and are representative of practical molecular property prediction tasks. The Polaris suite covers curated endpoints spanning solubility, ADMET, biophysics, and physiology from Polaris and Therapeutics Data Commons, and each dataset is evaluated with its benchmark-specific primary metric [28, 29]. MoleculeACE consists of 30 ChEMBL activity prediction assays with predefined expert-guided train-test splits based on ECFP similarity and activity, thereby explicitly probing generalization across activity cliffs [12].

4.5.1 Performance Evaluation Following [9], we consider only single-task prediction problems and strictly reuse the original train-test splits. Each newly run benchmark experiment is repeated over five random seeds to quantify stochastic variation. We reuse the train-test splits and evaluation setup from [9], and therefore report their main CheMeleon results together with the other published baseline results to enable a split- and metric-matched comparison. We do not perform task-specific hyperparameter optimization for any of the classical tabular ML models, because the goal is to assess practical out-of-the-box performance. The TFMs used are TabPFN-2.6 [13, 18, 44] and TabICLv2 [15]. For MoleculeACE, we compute the overall test RMSE as well as cliff and non-cliff subset RMSEs from the same predictions; the overall test RMSE is used for the aggregated model ranking shown in Fig. 2. For Polaris, benchmark scores are taken directly from the Polaris evaluation interface using the dataset-specific main metric. When the Polaris metric is an error quantity, such as mean squared error or mean absolute error, we rescale

scores within each dataset to the unit interval and invert them so that larger values always indicate better performance before rank aggregation [9]. We use Tukey’s honest significant difference (HSD) test with α = 0.05 on the five seed-specific benchmark scores to determine whether a model is statistically indistinguishable from the best model on a given dataset, following prior work [9, 28]. Because these repeated seeds share the same fixed train-test split, this test quantifies variation due to model stochasticity rather than uncertainty over alternative molecular test sets. The top model and all models not significantly different from it are counted as wins, and win counts are aggregated into the reported win rates. In parallel, we compute within-dataset model ranks and average them across datasets. The critical-difference diagrams in Fig. 2c,d are based on these average ranks and use the Nemenyi post hoc comparison following Friedman’s test at α = 0.05.

4.5.2 Runtime Case Study To quantify computational cost, we perform an illustrative runtime case study comparing TabPFN-CheMeleonFP with fine-tuned CheMeleon on three Polaris datasets selected to represent the smallest, median, and largest training-set sizes: adme-fang-rppb-1 (ntrain = 111), caco2-wang (ntrain = 728), and ld50-zhu (ntrain = 5907). For the TabPFN-CheMeleonFP pipeline, total runtime is decomposed into CheMeleon embedding generation on the training and test molecules, TFM fitting, and test-set prediction. For the fine-tuned CheMeleon baseline, total runtime is decomposed into Chemprop fine-tuning from the CheMeleon foundation weights and prediction on the test set. Both CPU and GPU runtimes are measured with the same execution path and a fixed random seed; CPU measurements are obtained on an Intel Xeon 8468 Sapphire system using 24 cores, and GPU measurements are obtained on a single NVIDIA H100.

4.6 Chemical Engineering Datasets 4.6.1 Data To assess transfer beyond the Polaris and MoleculeACE benchmark suites, we additionally evaluate the methods on 11 chemical engineering regression datasets spanning fuel-property prediction, polymer-property prediction, and polymer-solvent interaction tasks. Fuels. The fuel ignition dataset comprises 505 molecules annotated with up to three experimentally measured fuel-performance properties: derived cetane number (DCN), research octane number (RON), and motor octane number (MON) [30, 31]. These properties probe different aspects of ignition and knocking behavior, and the dataset is incomplete because not every molecule has measurements for all three targets. Polymers. We consider seven polymer-property datasets covering chain band gap (Egc), bulk band gap (Egb), dielectric constant (EPS), refractive index (Nc), ionization energy (Ei), electron affinity (Eea), and crystallization tendency (Xc) [32]. Together, these datasets span optoelectronic, dielectric, and structural polymer properties. All targets except Xc are obtained from density functional theory calculations, whereas 15

Xc is derived from experimental heats of fusion combined with a group-contribution method [32]. PolySolv. In addition, we consider the PolySolv dataset, which contains 1,208 experimental Flory-Huggins interaction parameters (χ) for 38 polymers and 150 solvents [33, 45]. This dataset captures experimentally measured and regressed polymer–solvent interactions across a chemically diverse set of combinations.

4.6.2 Accuracy Evaluation and Literature Baselines All considered TFMs are evaluated with RDKit2d, Morgan, Mordred, and CheMeleonFP representations, alongside graph-based Chemprop baselines, fine-tuned CheMeleon, and untuned random forest, XGBoost, and CatBoost baselines on the same feature spaces. Here, Chemprop GNN denotes a standard Chemprop messagepassing network, and CheMeleon denotes fine-tuning the CheMeleon foundation model through the Chemprop training pipeline. To remain comparable with the respective literature baselines, we repeat the benchmark-specific cross-validation protocols. Fuels. For the fuel tasks, we report the mean and standard deviation of RMSE across folds. We compare against the literature baselines GNN STL and GNN MTLTrain from [31], with matched random-split 10-fold cross-validation. Polymers. Polymers are passed to the respective featurizers using repeat-unit SMILES notation. For the polymer-property benchmarks, we report the mean and standard deviation of R2 across folds. For comparison, we use the published PolyCL, TransPolymer, and PolyBERT results reported in the corresponding studies [32, 36, 37], with matched random-split 5-fold cross-validation. These do not include fold-wise standard deviations. PolySolv. For the PolySolv benchmark, polymer and solvent molecules are featurized separately, and their feature vectors are concatenated with the numerical covariates temperature and volume fraction before regression. For Chemprop GNN and CheMeleon, the same two SMILES columns and numerical covariates are passed to the model. We report the mean and standard deviation of R2 across folds. We compare against the published D-MPNN-TC baseline from [33].

Data Availability No new phyiscal experimental data were generated in this work. The datasets used are available from their respective authors via Polaris or GitHub. Processed benchmark inputs, predictions, and analysis outputs, as well as predictions from the CheMeleon repository [9], are available in our Git repository at https://git.rwth-aachen.de/ avt-svt/public/tabpfn-molprop. A snapshot of the repository is also available on Zenodo [46].

Code Availability The full code for this work including all implementations, experiments and analysis scripts is available under the open-source Eclipse Public License at https://git. rwth-aachen.de/avt-svt/public/tabpfn-molprop. 16

References [1] Yu, M. et al. Deep learning large-scale drug discovery and repurposing. Nature Computational Science 4, 600–614 (2024). [2] Eraqi, B. A., Khizbullin, D., Nagaraja, S. S. & Sarathy, S. M. Molecular property prediction in the ultra-low data regime. Communications Chemistry 8, 201 (2025). [3] Rittig, J. G., Dahmen, M., Grohe, M., Schwaller, P. & Mitsos, A. Molecular machine learning in chemical process design. Current Opinion in Chemical Engineering 52, 101239 (2026). [4] Altae-Tran, H., Ramsundar, B., Pappu, A. S. & Pande, V. Low Data Drug Discovery with One-Shot Learning. ACS Central Science 3, 283–293 (2017). [5] Tetko, I. V., van Deursen, R. & Godin, G. Be aware of overfitting by hyperparameter optimization! Journal of Cheminformatics 16, 139 (2024). [6] Praski, M., Adamczyk, J. & Czech, W. Benchmarking Pretrained Molecular Embedding Models For Molecular Representation Learning (2025). arXiv:2508.06199. [7] Ross, J. et al. Large-scale chemical language representations capture molecular structure and properties. Nature Machine Intelligence 4, 1256–1264 (2022). [8] Wang, Y., Wang, J., Cao, Z. & Farimani, A. B. Molecular Contrastive Learning of Representations via Graph Neural Networks. Nature Machine Intelligence 4, 279–287 (2022). [9] Burns, J. W. et al. Deep Learning Foundation Models from Classical Molecular Descriptors (2026). arXiv:2506.15792. [10] Kumar, A., Raghunathan, A., Jones, R., Ma, T. & Liang, P. Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distribution (2022). arXiv:2202.10054. [11] Grinsztajn, L., Oyallon, E. & Varoquaux, G. Why do tree-based models still outperform deep learning on typical tabular data? Advances in Neural Information Processing Systems 35, 507–520 (2022). [12] van Tilborg, D., Alenicheva, A. & Grisoni, F. Exposing the Limitations of Molecular Machine Learning with Activity Cliffs. Journal of Chemical Information and Modeling 62, 5938–5951 (2022). [13] Hollmann, N. et al. Accurate predictions on small data with a tabular foundation model. Nature 637, 319–326 (2025).

[14] Hollmann, N., Müller, S., Eggensperger, K. & Hutter, F. TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Second (2023). arXiv:2207.01848. [15] Qu, J., Holzmüller, D., Varoquaux, G. & Morvan, M. L. TabICLv2: A better, faster, scalable, and open tabular foundation model (2026). arXiv:2602.11139. [16] Qu, J., Holzmüller, D., Varoquaux, G. & Morvan, M. L. TabICL: A Tabular Foundation Model for In-Context Learning on Large Data (2025). arXiv:2502.05564. [17] Erickson, N. et al. TabArena: A Living Benchmark for Machine Learning on Tabular Data (2025). arXiv:2506.16791. [18] Grinsztajn, L. et al. Tabpfn-2.5: Advancing the state of the art in tabular foundation models. arXiv preprint arXiv:2511.08667 (2025). [19] Chen, W. et al. TabPFN Opens New Avenues for Small-Data Tabular Learning in Drug Discovery. Journal of Chemical Information and Modeling acs.jcim.5c02823 (2026). [20] Landrum, G., Tosco, P., Kelley, B. et al. Rdkit: Open-source cheminformatics (2026). URL https://doi.org/10.5281/zenodo.18797641. [21] Moriwaki, H., Tian, Y.-S., Kawashita, N. & Takagi, T. Mordred: A molecular descriptor calculator. Journal of Cheminformatics 10, 4 (2018). [22] Soares, E. et al. An open-source family of large encoder-decoder foundation models for chemistry. Communications Chemistry 8, 193 (2025). [23] Seidl, P., Vall, A., Hochreiter, S. & Klambauer, G. Enhancing Activity Prediction Models in Drug Discovery with the Ability to Understand Human Language (2023). arXiv:2303.03363. [24] Heid, E. et al. Chemprop: a machine learning package for chemical property prediction. Journal of chemical information and modeling 64, 9–17 (2023). [25] Kläser, K. et al. MiniMol: A Parameter-Efficient Foundation Model for Molecular Learning (2024). arXiv:2404.14986. [26] Sheridan, R. P., Wang, W. M., Liaw, A., Ma, J. & Gifford, E. M. Extreme Gradient Boosting as a Method for Quantitative Structure–Activity Relationships. Journal of Chemical Information and Modeling 56, 2353–2360 (2016). [27] Prokhorenkova, L., Gusev, G., Vorobev, A., Dorogush, A. V. & Gulin, A. Catboost: unbiased boosting with categorical features. Advances in neural information processing systems 31 (2018).

[28] Ash, J. R. et al. Practically Significant Method Comparison Protocols for Machine Learning in Small Molecule Drug Discovery. Journal of Chemical Information and Modeling 65, 9398–9411 (2025). [29] Huang, K. et al. Therapeutics Data Commons: Machine Learning Datasets and Tasks for Drug Discovery and Development (2021). arXiv:2102.09548. [30] Schweidtmann, A. M. et al. Graph neural networks for prediction of fuel ignition quality. Energy & fuels 34, 11395–11407 (2020). [31] bin Javaid, M., Gervens, T., Mitsos, A., Grohe, M. & Rittig, J. G. Exploring data augmentation: Multi-task methods for molecular property prediction. Computers & Chemical Engineering 201, 109253 (2025). [32] Kuenneth, C. & Ramprasad, R. polybert: a chemical language model to enable fully machine-driven ultrafast polymer informatics. Nature communications 14, 4099 (2023). [33] Liao, H.-C., Lin, Y.-H., Peng, C.-H. & Li, Y.-P. Directed message passing neural networks for accurate prediction of polymer–solvent interaction parameters. ACS Engineering Au 5, 530–539 (2025). [34] Pinto, L. Superior molecular representations from intermediate encoder layers. arXiv preprint arXiv:2506.06443 (2025). [35] Wognum, C. et al. A call for an industry-led initiative to critically assess machine learning for real-world drug discovery. Nature Machine Intelligence 6, 1120–1121 (2024). [36] Zhou, J., Yang, Y., Mroz, A. M. & Jelfs, K. E. Polycl: contrastive learning for polymer representation learning via explicit and implicit augmentations. Digital Discovery 4, 149–160 (2025). [37] Xu, C., Wang, Y. & Barati Farimani, A. Transpolymer: a transformer-based language model for polymer property predictions. npj Computational Materials 9, 64 (2023). [38] Erickson, N. et al. Autogluon-tabular: Robust and accurate automl for structured data. arXiv preprint arXiv:2003.06505 (2020). [39] Sinodinos, D. et al. Multitask-informed prior for in-context learning on tabular data: Application to steel property prediction. arXiv preprint arXiv:2603.22738 (2026). [40] Sinodinos, D., Wei, J. Y. & Armanfard, N. Multitab: A scalable foundation for multitask learning on tabular data. Proceedings of the AAAI Conference on Artificial Intelligence (2026).

[41] Yu, R. T.-Y., Picard, C. & Ahmed, F. GIT-BO: High-Dimensional Bayesian Optimization with Tabular Foundation Models (2026). arXiv:2505.20685. [42] Müller, S., Hollmann, N., Arango, S. P., Grabocka, J. & Hutter, F. Transformers Can Do Bayesian Inference (2024). arXiv:2112.10510. [43] Garg, A. et al. Real-tabpfn: Improving tabular foundation models via continued pre-training with real-world data. arXiv preprint arXiv:2507.03971 (2025). [44] PriorLabs. TabPFN (2026). URL https://github.com/PriorLabs/TabPFN. GitHub repository, accessed 2026-04-09. [45] Mark, J. E. & Mark, J. E. Physical properties of polymers handbook Vol. 1076 (Springer, 2007). [46] Ben Hicham, K. K., Rittig, J. G., Grohe, M. & Mitsos, A. Tabular foundation models for in-context prediction of molecular properties (2026). URL https:// doi.org/10.5281/zenodo.19605631. [47] Medina, E. I. S., Linke, S., Stoll, M. & Sundmacher, K. Gibbs–helmholtz graph neural network: capturing the temperature dependency of activity coefficients at infinite dilution. Digital Discovery 2, 781–798 (2023). [48] Brouwer, T., Kersten, S. R., Bargeman, G. & Schuur, B. Trends in solvent impact on infinite dilution activity coefficients of solutes reviewed and visualized using an algorithm to support selection of solvents for greener fluid separations. Separation and Purification Technology 272, 118727 (2021).

Author Contributions Karim K. Ben Hicham: Conceptualization, Methodology, Formal analysis, Investigation, Writing – original draft. Jan G. Rittig: Conceptualization, Supervision, Writing – original draft. Martin Grohe: Methodology, Funding acquisition, Writing – review & editing. Alexander Mitsos: Conceptualization, Supervision, Funding acquisition, Writing – review & editing.

Acknowledgements We acknowledge support of the Werner Siemens Foundation in the frame of the WSS Research Center “catalaix”. We also thank the authors of CheMeleon [9] for releasing the full code and predictions for their work.

Competing Interests The authors declare no competing interests.

S1 Featurization and Model Details TFM Inference Hyperparameter Defaults. Tabular foundation models (TFMs) expose a range of inference-time hyperparameters. For TabICL, these include in particular the number of estimators, preprocessing and normalization choices, feature shuffling, and outlier-handling options. For TabPFN, relevant hyperparameters likewise include the number of estimators, normalization settings, handling of categorical features, softmax temperature, and outlier-removal steps. In this work, however, we deliberately do not tune any of these hyperparameters and instead use both TFMs strictly with their default settings throughout. This choice is motivated by the central aim of this study: to evaluate TFMs as simple, out-ofthe-box downstream predictors for molecular property prediction. We therefore focus on the user experience and practical performance that can be achieved without taskspecific optimization, manual intervention, or additional machine-learning expertise. This choice should yield conservative performance estimates. In particular, further improvements are likely possible through simple changes in the hyperparameters. For example, in the high-dimensional feature regimes considered here, increasing the number of estimators is recommended, especially for TabPFN, where individual estimators operate on subsets of 680 features for categorical features and 500 for numeric features, and predictions are aggregated across estimators. Such tuning could therefore improve feature coverage and downstream predictive performance for very high-dimensional molecular representations. Selected values are shown in Table S1; details can be found in the respective works and GitHub repositories. Classical Features. RDKit2d descriptors [20], Mordred descriptors [21], and Morgan fingerprints [20] are three common molecular representations. RDKit2d descriptors provide a compact set of physicochemical and topological features and are widely used as a simple, robust baseline. Mordred descriptors extend this to a much larger and more diverse descriptor set, offering broader structural coverage at the cost of higher dimensionality. Morgan fingerprints, by contrast, encode local chemical substructures as binary or count-based patterns rather than continuous descriptors. The RDKit2dMACCS variant used in supplementary analyses augments RDKit2d descriptors with MACCS structural keys. In practice, RDKit2d is the most compact descriptor baseline, Mordred is the most expressive descriptor representation, and Morgan fingerprints provide a sparse substructure-based alternative. Table S1 Default TFM inference hyperparameters.

Hyperparameter

TabICLv2

TabPFN-2.6

n estimators outlier threshold softmax temperature prediction threshold max features per estimator feat shuffle method

8 4.0 0.9 0.5 all unique latin

8 None 0.9 0.5 680/500 shuffle

Table S2 Molecular representations used in the Polaris and MoleculeACE benchmark experiments. Featurizer

Dimension

CheMeleonFP [9] CLAMP [23] SMI-TED [22] RDKit2d [20] RDKit2d-MACCS [20] Mordred [21] Morgan (radius 2) [20]

2048 768 768 217 383 1613 2048

S2 Supplementary Benchmark Results Activity Cliffs. Across MoleculeACE ChEMBL assays, the TabPFN-CheMeleonFP model showed particularly strong predictive performance overall, frequently matching or exceeding the best-performing methods at the assay level. Nevertheless, despite this strong overall performance, it still exhibited predominantly positive values of RMSEcliff − RMSEnoncliff , indicating that activity-cliff compounds remained systematically more difficult to predict than non-cliff compounds, as shown in Figure S3. TabPFN-CheMeleonFP displayed a limitation comparable to that observed for CheMeleon, with neither approach overcoming the prediction difficulty associated with activity cliffs. Accordingly, only 2 of 30 tasks for TabPFN-CheMeleonFP showed no significant difference between cliff and non-cliff RMSE, compared with 4 of 30 tasks for CheMeleon. The magnitude of this cliff-associated error increase varied across assays and models. Filled markers denote assays for which a model was either the best-performing method or statistically indistinguishable from the best-performing method, whereas blue markers denote assays for which RMSEcliff − RMSEnoncliff was not significantly different from zero.

tdcommons ld50-zhu Train Size: 5,907 Metric: scaled_mae minimol fastprop XGBoost-Mordred TabPFN-SMI-TED TabPFN-RDKit2D TabPFN-Morgan TabPFN-Mordred TabPFN-CheMeleonFP TabPFN-CLAMP TabICL-SMI-TED TabICL-RDKit2D TabICL-Morgan TabICL-Mordred TabICL-CheMeleonFP TabICL-CLAMP RF Morgan Physchem RF Morgan RF Mordred RF PCA MLP Prefitted PCA MLP MolCLR MoLFormer MLP PLR Pretrained Chemprop Mordred Chemprop CheMeleon CatBoost-Mordred 0.5 1.0 tdcommons bbb-martins Train Size: 1,624 Metric: roc_auc

tdcommons tdcommons tdcommons polaris polaris polaris ames lipophilicity-astrazeneca ppbr-az adme-fang-hclint-1 adme-fang-rclint-1 adme-fang-perm-1 Train Size: 5,821 Train Size: 3,360 Train Size: 2,231 Train Size: 2,229 Train Size: 2,218 Train Size: 1,919 Metric: roc_auc Metric: scaled_mae Metric: scaled_mae Metric: pearsonr Metric: pearsonr Metric: pearsonr

0.7 0.8 0.9 polaris adme-fang-solu-1 Train Size: 1,578 Metric: pearsonr

0 1 0 1 0.25 0.50 0.75 0.25 0.50 0.75 0.50 0.75 tdcommons tdcommons tdcommons tdcommons tdcommons pgp-broccatelli clearance-hepatocyte-az vdss-lombardo clearance-microsome-az caco2-wang Train Size: 973 Train Size: 970 Train Size: 904 Train Size: 881 Train Size: 728 Metric: roc_auc Metric: spearmanr Metric: spearmanr Metric: spearmanr Metric: scaled_mae

minimol fastprop XGBoost-Mordred TabPFN-SMI-TED TabPFN-RDKit2D TabPFN-Morgan TabPFN-Mordred TabPFN-CheMeleonFP TabPFN-CLAMP TabICL-SMI-TED TabICL-RDKit2D TabICL-Morgan TabICL-Mordred TabICL-CheMeleonFP TabICL-CLAMP RF Morgan Physchem RF Morgan RF Mordred RF PCA MLP Prefitted PCA MLP MolCLR MoLFormer MLP PLR Pretrained Chemprop Mordred Chemprop CheMeleon CatBoost-Mordred 0.8 0.9 0.25 0.50 0.75 0.8 0.9 0.2 0.4 0.25 0.50 0.75 0.25 0.50 0.75 0.5 1.0 tdcommons tdcommons polaris polaris tdcommons tdcommons polaris cyp3a4-substrate-carbonmangels cyp2c9-substrate-carbonmangels pkis2-ret-wt-cls-v2 pkis2-ret-wt-reg-v2 half-life-obach cyp2d6-substrate-carbonmangels pkis2-kit-wt-reg-v2 Train Size: 535 Train Size: 534 Train Size: 534 Train Size: 534 Train Size: 532 Train Size: 532 Train Size: 524 Metric: roc_auc Metric: pr_auc Metric: pr_auc Metric: scaled_mse Metric: spearmanr Metric: pr_auc Metric: scaled_mse minimol fastprop XGBoost-Mordred TabPFN-SMI-TED TabPFN-RDKit2D TabPFN-Morgan TabPFN-Mordred TabPFN-CheMeleonFP TabPFN-CLAMP TabICL-SMI-TED TabICL-RDKit2D TabICL-Morgan TabICL-Mordred TabICL-CheMeleonFP TabICL-CLAMP RF Morgan Physchem RF Morgan RF Mordred RF PCA MLP Prefitted PCA MLP MolCLR MoLFormer MLP PLR Pretrained Chemprop Mordred Chemprop CheMeleon CatBoost-Mordred 0.6 0.7 0.3 0.4 0.5 0.50 0.75 0 1 0.0 0.5 0.6 0.7 0 1 polaris tdcommons tdcommons polaris tdcommons polaris polaris pkis2-kit-wt-cls-v2 herg bioavailability-ma pkis2-egfr-wt-reg-v2 dili adme-fang-hppb-1 adme-fang-rppb-1 Train Size: 524 Train Size: 523 Train Size: 512 Train Size: 496 Train Size: 379 Train Size: 126 Train Size: 111 Metric: pr_auc Metric: roc_auc Metric: roc_auc Metric: scaled_mse Metric: roc_auc Metric: pearsonr Metric: pearsonr minimol fastprop XGBoost-Mordred TabPFN-SMI-TED TabPFN-RDKit2D TabPFN-Morgan TabPFN-Mordred TabPFN-CheMeleonFP TabPFN-CLAMP TabICL-SMI-TED TabICL-RDKit2D TabICL-Morgan TabICL-Mordred TabICL-CheMeleonFP TabICL-CLAMP RF Morgan Physchem RF Morgan RF Mordred RF PCA MLP Prefitted PCA MLP MolCLR MoLFormer MLP PLR Pretrained Chemprop Mordred Chemprop CheMeleon CatBoost-Mordred 0.25 0.50 0.75 0.7 0.8 0.9 0.6 0.8 0 1 0.8 0.9 0 1 0 1

Fig. S1 Performance for all models for the 28 Polaris tasks. Models highlighted in blue are the top performers on a given benchmark, whereas models shown in gray are not significantly different from the top performer according to Tukey’s honestly significant difference (HSD) test (α = 0.05) across five repetitions. Models shown in red performed significantly worse and were considered to have lost on that benchmark.

1862_Ki

1871_Ki

0.8 1.0 214_Ki

0.8 234_Ki

0.6

0.8 239_EC50

0.6

0.8 2971_Ki

1.0

0.8

1.0

0.6

0.8 218_EC50

0.6

2034_Ki

2047_EC50

0.8 219_Ki

0.6

0.8 1.0 235_EC50

0.8 1.0 236_Ki

0.6

0.8 244_Ki

0.75 1.00 3979_EC50

0.6

0.8

0.6

204_Ki

0.8

0.75 1.00 1.25 0.50 0.75 1.00 231_Ki 233_Ki

228_Ki

0.6

0.8 1.0 237_Ki

0.8 1.0 262_Ki

0.75 1.00 1.25 0.6 264_Ki

0.8 1.0 2835_Ki

0.8 1.0 4005_Ki

0.6

0.4 0.6 4616_EC50

0.8

0.8 1.0 237_EC50

0.8 4203_Ki

0.9

1.0

2147_Ki

0.7 0.8 0.9

0.8 1.0 238_Ki

0.6

0.8 1.0 287_Ki

0.8 4792_Ki

0.8

1.0

Fig. S2 Performance for all models for the 30 MoleculeACE tasks. Models highlighted in blue are the top performers on a given benchmark, whereas models shown in gray are not significantly different from the top performer according to Tukey’s honestly significant difference (HSD) test (α = 0.05) across five repetitions. Models shown in red performed significantly worse and were considered to have lost on that benchmark.

Table S3 Win rates and average ranks across the Polaris benchmark tasks. Wins count models that are best or statistically indistinguishable from the best model on a task. Model TabPFN-RDKit2d TabICL-RDKit2d TabPFN-Mordred TabPFN-CheMeleonFP TabICL-CheMeleonFP TabICL-Mordred TabPFN-CLAMP TabICL-CLAMP CheMeleon [9] minimol [9] RF Morgan Physchem [9] CatBoost-Mordred RF [9] MLP PLR Pretrained [9] fastprop [9] TabPFN-Morgan RF Mordred [9] MoLFormer [9] PCA MLP Prefitted [9] PCA MLP [9] XGBoost-Mordred Chemprop Mordred [9] TabICL-Morgan RF Morgan [9] Chemprop [9] TabICL-SMI-TED TabPFN-SMI-TED MolCLR [9]

Win Count

Win Rate (%)

Average Rank

18 16 19 20 20 12 15 16 13 15 13 10 14 6 5 7 6 6 5 5 6 4 5 4 2 4 3 1

64.3 57.1 67.9 71.4 71.4 42.9 53.6 57.1 46.4 53.6 46.4 35.7 50.0 21.4 17.9 25.0 21.4 21.4 17.9 17.9 21.4 14.3 17.9 14.3 7.1 14.3 10.7 3.6

5.43 6.54 6.75 7.11 8.79 9.39 9.54 9.79 10.25 11.00 11.04 12.25 12.29 14.39 15.57 16.07 17.86 18.21 18.36 18.61 18.64 18.79 19.21 19.25 19.25 23.18 23.86 24.61

S3 Supplementary Engineering Benchmark Results S3.1 Brouwer Infinite-Dilution Activity-Coefficient Dataset Additionally we compared a subset of all models on a larger engineering dataset, e.g., the Brouwer infinite-dilution activity-coefficient dataset [47, 48]. The dataset contains 20,870 measurements of the logarithmic infinite-dilution activity coefficient, ln γ ∞ , for binary solute–solvent systems. Each observation is defined by a solute SMILES, a solvent SMILES, and a temperature, with ln γ ∞ as the regression target. After filtering to valid SMILES, the processed dataset contains 373 solutes, 349 solvents, and 6,416 distinct solute–solvent pairs, with temperatures ranging from −23.15 to 282.45 ◦ C. We used a 10-fold group split by solute–solvent pair, corresponding to a non-stratified discrete-interpolation setting, so that all temperature measurements for a given binary pair remain in the same fold. Each fold therefore contains 18,783 training observations and 2,087 test observations, making this benchmark substantially larger than the other engineering datasets considered in the main text. The compact RDKit2d pair representation (435 features in total) with TabPFN performs strongest and gives the 25

Table S4 Win rates and average ranks across the MoleculeACE benchmark tasks. Wins count models that are best or statistically indistinguishable from the best model on a task. Model TabPFN-CheMeleonFP TabICL-CheMeleonFP TabPFN-Mordred TabPFN-RDKit2d TabICL-RDKit2d TabICL-Mordred CheMeleon [9] TabICL-CLAMP TabPFN-CLAMP TabPFN-Morgan RF Morgan [9] minimol [9] CatBoost-Mordred RF [9] TabICL-Morgan fastprop [9] RF Mordred [9] MoLFormer [9] MLP PLR Pretrained [9] PCA MLP Prefitted [9] PCA MLP [9] XGBoost-Mordred Chemprop Mordred [9] Chemprop [9] TabICL-SMI-TED TabPFN-SMI-TED MolCLR [9]

Win Count

Win Rate (%)

Average Rank

30 23 20 15 16 10 11 9 7 6 6 4 1 2 3 0 0 0 0 0 0 0 0 0 0 0 0

100.0 76.7 66.7 50.0 53.3 33.3 36.7 30.0 23.3 20.0 20.0 13.3 3.3 6.7 10.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0

2.10 4.00 4.73 5.20 5.30 7.37 7.53 8.00 8.33 10.10 11.27 11.83 11.87 12.63 16.17 16.37 17.17 18.50 18.67 19.10 19.73 20.37 20.73 24.07 24.93 25.00 26.93

lowest mean RMSE, with RMSE = 0.294, MAE = 0.137, and R2 = 0.980. The default Chemprop GNN performs very strongly on this larger activity-coefficient benchmark and achieves strong mean 10-fold performance, with RMSE = 0.299, MAE = 0.160, and R2 = 0.979. The CheMeleon model is also strong, with an RMSE = 0.316, but worse than the model trained from scratch. At the same time, TabPFN-Mordred remains close to the CheMeleon result, with RMSE = 0.327, MAE = 0.160, and R2 = 0.975. A Tukey HSD plot is provided in Figure S4. The runtime comparison remains favorable for TabPFN: TabPFN-RDKit2d requires 61 s per fold and TabPFNMordred requires 92 s per fold on average for fitting and prediction (without feature computation), whereas Chemprop GNN requires 515 s per fold. TabICL experiments with the larger feature sets ran out of GPU memory on a single NVIDIA H100.

Table S5 Restricted feature-set comparison across the MoleculeACE benchmark tasks, including RDKit2d-MACCS variants. Wins count models that are best or statistically indistinguishable from the best model within this comparison on a task. Model TabPFN-CheMeleonFP TabICL-CheMeleonFP TabPFN-Mordred TabPFN-RDKit2d-MACCS TabICL-RDKit2d-MACCS TabPFN-RDKit2d TabICL-RDKit2d CheMeleon TabICL-Mordred TabICL-CLAMP TabPFN-CLAMP TabPFN-Morgan RF TabICL-Morgan XGBoost-Mordred TabICL-SMI-TED TabPFN-SMI-TED

Win Count

Win Rate (%)

Average Rank

20 11 6 13 5 12 5 6 2 2 4 1 0 1 0 0 0

66.7 36.7 20.0 43.3 16.7 40.0 16.7 20.0 6.7 6.7 13.3 3.3 0.0 3.3 0.0 0.0 0.0

2.53 4.43 5.37 5.60 5.97 6.07 6.07 7.83 8.27 8.77 8.87 10.53 11.97 13.03 14.80 16.43 16.47

Table S6 Restricted feature-set comparison across the Polaris benchmark tasks, including RDKit2d-MACCS variants. Wins count models that are best or statistically indistinguishable from the best model within this comparison on a task. Model TabPFN-RDKit2d TabPFN-RDKit2d-MACCS TabPFN-Mordred TabICL-RDKit2d TabPFN-CheMeleonFP TabICL-RDKit2d-MACCS TabICL-CheMeleonFP TabPFN-CLAMP TabICL-CLAMP CheMeleon [9] TabICL-Mordred RF [9] TabPFN-Morgan XGBoost-Mordred TabICL-Morgan TabICL-SMI-TED TabPFN-SMI-TED

Win Count

Win Rate (%)

Average Rank

15 11 11 9 10 11 8 8 9 8 4 6 3 1 3 1 1

53.6 39.3 39.3 32.1 35.7 39.3 28.6 28.6 32.1 28.6 14.3 21.4 10.7 3.6 10.7 3.6 3.6

5.29 5.50 5.86 6.14 6.21 6.75 7.54 7.61 7.75 8.32 8.46 9.71 11.32 13.00 13.07 15.11 15.36

ChEMBL Assay Number

TabPFN-CheMeleonFP _Ki 2823053_KKii 4871_Ki 1034_Ki 2204_ 0 EC5 23490_05C_5K0i E _ 5 3 2 2114__KKii 297EC50 2372_364__KKii 2431_Kii 233_Ki 237_Ki 2 K 26149__Kii 228_K 2_EC50i 6 461 23887__KKi 2 EC50 218__EC50i 397929622__KKi 47_EC50i 2041786427__KKii 21234_K

0.0

RMSEcliff

0.5

RMSEnoncliff

TabPFN-RDKit2D

0.0

RMSEcliff

0.5

RMSEnoncliff

TabPFN-Mordred

0.0

RMSEcliff

0.5

RMSEnoncliff

CheMeleon

0.0

RMSEcliff

0.5

RMSEnoncliff

Fig. S3 Assay-level difference between prediction error on activity-cliff and non-cliff compounds across MoleculeACE benchmarks. Shown are values of RMSEcliff − RMSEnoncliff for TabPFNCheMeleonFP, TabPFN-RDKit2d, TabPFN-Mordred, and CheMeleon across individual ChEMBL assays. Positive values indicate higher prediction error on activity-cliff compounds relative to noncliff compounds, whereas values near zero indicate comparable performance across the two subsets. Filled markers indicate assays for which the corresponding model was the best-performing method or statistically indistinguishable from the best-performing method. Blue markers indicate assays for which RMSEcliff − RMSEnoncliff was not significantly different from zero.

Brouwer_2021 Train Size: 18,783 Metric: RMSE (inverted) XGBoost-Morgan XGBoost-Mordred XGBoost-CheMeleonFP TabPFN-Rdkit2D TabPFN-Morgan TabPFN-Mordred TabPFN-CheMeleonFP TabICL-Rdkit2D RF-Morgan RF-Mordred RF-CheMeleonFP ChempropGNN ChempropCheMeleon CatBoost-Morgan CatBoost-Mordred CatBoost-CheMeleonFP 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 Fig. S4 Model performance for 10-fold cross-validation with random system-wise splits for the infinite-dilution activity-coefficient Brouwer dataset [47, 48]. This corresponds to a non-stratified ”discrete-interpolation” [47] setting. TabPFN-RDKit2d highlighted in blue is the top performer, whereas the models shown in gray are not significantly different from it according to Tukey’s honestly significant difference (HSD) test (α = 0.05) across ten folds. Models shown in red performed significantly worse and were considered to have lost on that benchmark.

RON Train Size: 301 Metric: RMSE (inverted)

MON Train Size: 286 Metric: RMSE (inverted)

DCN Train Size: 212 Metric: RMSE (inverted)

XGBoost-Rdkit2D XGBoost-Morgan XGBoost-Mordred XGBoost-CheMeleonFP TabPFN-Rdkit2D TabPFN-Morgan TabPFN-Mordred TabPFN-CheMeleonFP TabICL-Rdkit2D TabICL-Morgan TabICL-Mordred TabICL-CheMeleonFP RF-Rdkit2D RF-Morgan RF-Mordred RF-CheMeleonFP ChempropGNN ChempropCheMeleon CatBoost-Rdkit2D CatBoost-Morgan CatBoost-Mordred CatBoost-CheMeleonFP 14

12 10 8 Egc Train Size: 2,704 Metric: RMSE (inverted)

10 8 Egb Train Size: 449 Metric: RMSE (inverted)

0.65 0.60 0.55 0.50 0.45 0.40 EPS Train Size: 306 Metric: RMSE (inverted)

0.8

0.6 0.5 Nc Train Size: 306 Metric: RMSE (inverted)

0.4

0.14

0.12 0.10 0.08 PolySolv Train Size: 1,087 Metric: RMSE (inverted)

0.50 0.45 0.40 0.35 0.30 0.25

0.35

0.30

10 8 6 Xc Train Size: 346 Metric: RMSE (inverted)

20 18 Ei Train Size: 296 Metric: RMSE (inverted)

0.4

0.6 0.5 Eea Train Size: 294 Metric: RMSE (inverted)

0.60 0.55 0.50 0.45 0.40 0.35

0.20

0.15

0.10

Fig. S5 Tukey’s HSD plots (α = 0.05) for all 11 engineering tasks.

Table S7 Fuel benchmark 10-fold cross-validation RMSE results across datasets (mean ± standard deviation). Lower values indicate better performance.

Model TabPFN-Mordred TabICL-Mordred TabICL-RDKit2d TabPFN-RDKit2d CheMeleon CatBoost-Mordred TabPFN-CheMeleonFP GNN MTL-Train [31] TabICL-CheMeleonFP GNN STL [31] CatBoost-RDKit2d CatBoost-CheMeleonFP XGBoost-Mordred RF-Mordred XGBoost-RDKit2d RF-RDKit2d TabPFN-Morgan TabICL-Morgan RF-CheMeleonFP CatBoost-Morgan XGBoost-CheMeleonFP XGBoost-Morgan Chemprop GNN RF-Morgan

DCN

RON

MON

5.97 ± 1.69 6.59 ± 1.30 6.81 ± 1.68 7.39 ± 1.75 7.35 ± 1.95 9.06 ± 1.76 8.22 ± 1.97 7.52 ± 1.84 8.60 ± 1.43 7.51 ± 1.86 9.56 ± 2.01 11.11 ± 2.32 10.68 ± 2.16 11.83 ± 1.34 11.13 ± 1.71 12.22 ± 2.03 11.24 ± 2.47 11.77 ± 2.40 12.50 ± 1.97 11.73 ± 2.55 12.83 ± 1.92 12.30 ± 2.45 13.39 ± 2.12 13.38 ± 2.10

8.57 ± 1.78 9.02 ± 2.26 9.25 ± 1.82 9.43 ± 1.99 9.48 ± 1.80 9.12 ± 1.70 9.68 ± 2.16 9.46 ± 1.72 9.89 ± 1.91 10.29 ± 2.16 10.03 ± 1.82 10.13 ± 2.26 10.77 ± 1.35 10.44 ± 1.64 11.54 ± 1.75 10.84 ± 1.84 11.81 ± 2.27 11.99 ± 2.85 11.22 ± 2.42 12.01 ± 2.09 12.33 ± 3.03 12.87 ± 1.88 11.89 ± 2.10 13.03 ± 1.91

7.35 ± 1.62 7.20 ± 1.69 7.67 ± 2.59 7.88 ± 2.42 7.94 ± 2.36 8.12 ± 1.97 8.01 ± 2.22 8.47 ± 1.95 8.28 ± 2.00 8.83 ± 2.19 8.39 ± 2.09 9.31 ± 2.43 9.86 ± 2.04 9.26 ± 2.14 10.18 ± 2.08 9.88 ± 2.19 10.25 ± 2.72 10.02 ± 3.11 10.47 ± 2.50 10.91 ± 2.50 10.75 ± 2.75 11.89 ± 1.93 12.50 ± 3.13 11.86 ± 2.92

Table S8 Polymer benchmark 5-fold cross-validation R2 results across datasets (mean ± standard deviation) with baseline rows. Boldface follows the rounded-mean and standard-deviation tie-breaking rule from Table 2. Model

Eea

Egb

Egc

EPS

TabPFN-Mordred TabPFN-RDKit2d TabICL-RDKit2d TabICL-Mordred TabPFN-CheMeleonFP PolyCL [36] CatBoost-Mordred TabICL-CheMeleonFP TransPolymer [37] PolyBERT [32] CheMeleon TabPFN-Morgan CatBoost-CheMeleonFP TabICL-Morgan CatBoost-RDKit2d RF-Mordred CatBoost-Morgan RF-RDKit2d RF-Morgan XGBoost-Mordred RF-CheMeleonFP XGBoost-RDKit2d XGBoost-Morgan XGBoost-CheMeleonFP Chemprop GNN

0.93 ± 0.01 0.92 ± 0.01 0.93 ± 0.01 0.93 ± 0.01 0.91 ± 0.01 0.91 0.90 ± 0.02 0.91 ± 0.01 0.89 0.91 0.91 ± 0.01 0.90 ± 0.01 0.87 ± 0.02 0.90 ± 0.02 0.87 ± 0.02 0.86 ± 0.03 0.88 ± 0.02 0.82 ± 0.02 0.83 ± 0.02 0.84 ± 0.04 0.82 ± 0.04 0.83 ± 0.01 0.85 ± 0.02 0.80 ± 0.06 0.82 ± 0.04

0.93 ± 0.01 0.93 ± 0.01 0.92 ± 0.02 0.92 ± 0.02 0.91 ± 0.02 0.89 0.91 ± 0.01 0.91 ± 0.02 0.90 0.88 0.92 ± 0.01 0.88 ± 0.02 0.88 ± 0.03 0.89 ± 0.01 0.89 ± 0.03 0.88 ± 0.03 0.87 ± 0.03 0.89 ± 0.02 0.85 ± 0.05 0.88 ± 0.03 0.86 ± 0.05 0.88 ± 0.03 0.87 ± 0.04 0.87 ± 0.04 0.86 ± 0.02

0.93 ± 0.00 0.92 ± 0.01 0.92 ± 0.01 0.91 ± 0.01 0.92 ± 0.01 0.88 0.91 ± 0.01 0.90 ± 0.01 0.88 0.88 0.91 ± 0.01 0.87 ± 0.01 0.89 ± 0.01 0.84 ± 0.02 0.90 ± 0.01 0.89 ± 0.01 0.87 ± 0.01 0.88 ± 0.01 0.87 ± 0.01 0.89 ± 0.01 0.87 ± 0.01 0.88 ± 0.01 0.87 ± 0.01 0.88 ± 0.01 0.90 ± 0.01

0.81 ± 0.02 0.80 ± 0.03 0.81 ± 0.03 0.81 ± 0.03 0.79 ± 0.01 0.81 0.80 ± 0.03 0.78 ± 0.03 0.79 0.77 0.76 ± 0.01 0.78 ± 0.03 0.77 ± 0.03 0.76 ± 0.03 0.78 ± 0.02 0.76 ± 0.05 0.77 ± 0.03 0.76 ± 0.05 0.76 ± 0.03 0.76 ± 0.06 0.75 ± 0.04 0.74 ± 0.06 0.73 ± 0.03 0.75 ± 0.02 0.67 ± 0.05

0.78 ± 0.08 0.77 ± 0.07 0.75 ± 0.06 0.77 ± 0.06 0.77 ± 0.07 0.79 0.75 ± 0.08 0.75 ± 0.05 0.76 0.77 0.72 ± 0.07 0.76 ± 0.05 0.73 ± 0.05 0.77 ± 0.04 0.72 ± 0.08 0.71 ± 0.08 0.75 ± 0.05 0.69 ± 0.08 0.74 ± 0.04 0.66 ± 0.09 0.69 ± 0.08 0.67 ± 0.10 0.72 ± 0.05 0.66 ± 0.07 0.65 ± 0.10

0.85 ± 0.06 0.85 ± 0.06 0.84 ± 0.04 0.84 ± 0.03 0.84 ± 0.05 0.85 0.83 ± 0.05 0.82 ± 0.04 0.81 0.80 0.82 ± 0.04 0.83 ± 0.04 0.81 ± 0.03 0.83 ± 0.02 0.80 ± 0.06 0.79 ± 0.04 0.82 ± 0.05 0.77 ± 0.06 0.79 ± 0.04 0.77 ± 0.07 0.78 ± 0.03 0.75 ± 0.08 0.78 ± 0.05 0.77 ± 0.03 0.72 ± 0.06

0.39 ± 0.16 0.40 ± 0.13 0.41 ± 0.14 0.39 ± 0.12 0.41 ± 0.11 0.40 0.38 ± 0.16 0.40 ± 0.11 0.46 0.44 0.22 ± 0.18 0.38 ± 0.12 0.41 ± 0.09 0.37 ± 0.14 0.36 ± 0.18 0.37 ± 0.10 0.35 ± 0.14 0.38 ± 0.13 0.33 ± 0.16 0.25 ± 0.20 0.39 ± 0.06 0.30 ± 0.19 0.29 ± 0.18 0.29 ± 0.10 0.22 ± 0.06

Table S9 Polymer-solvent interaction benchmark 10-fold cross-validation R2 results (mean ± standard deviation) with baseline rows. Boldface follows the rounded-mean and standard-deviation tie-breaking rule from Table 2.

Model

PolySolv

TabPFN-RDKit2d 0.93 ± 0.03 TabICL-RDKit2d 0.93 ± 0.03 D-MPNN-TC [33] 0.93 ± 0.03 CatBoost-RDKit2d 0.90 ± 0.03 CatBoost-Mordred 0.90 ± 0.03 TabPFN-Morgan 0.88 ± 0.04 TabPFN-Mordred 0.88 ± 0.05 CatBoost-CheMeleonFP 0.88 ± 0.04 XGBoost-RDKit2d 0.88 ± 0.04 TabICL-Morgan 0.87 ± 0.04 XGBoost-Mordred 0.87 ± 0.04 RF-RDKit2d 0.85 ± 0.03 CheMeleon 0.83 ± 0.08 RF-Mordred 0.82 ± 0.05 TabICL-Mordred 0.82 ± 0.06 TabPFN-CheMeleonFP 0.81 ± 0.08 XGBoost-CheMeleonFP 0.80 ± 0.06 CatBoost-Morgan 0.79 ± 0.06 RF-Morgan 0.79 ± 0.04 XGBoost-Morgan 0.77 ± 0.07 RF-CheMeleonFP 0.74 ± 0.08 Chemprop GNN 0.72 ± 0.09 TabICL-CheMeleonFP 0.71 ± 0.10

Table S10 Win rates and ranks across the 11 engineering datasets. Wins count models that are best or statistically indistinguishable from the best model on a task. Model TabPFN-Mordred TabPFN-RDKit2d TabICL-RDKit2d TabICL-Mordred TabPFN-CheMeleonFP CatBoost-Mordred CheMeleon TabICL-CheMeleonFP CatBoost-RDKit2d CatBoost-CheMeleonFP TabPFN-Morgan TabICL-Morgan RF-Mordred XGBoost-Mordred RF-RDKit2d CatBoost-Morgan XGBoost-RDKit2d RF-CheMeleonFP RF-Morgan XGBoost-Morgan XGBoost-CheMeleonFP Chemprop GNN

Win Count 11 11 11 10 10 10 10 9 8 7 9 8 7 8 7 5 8 6 4 4 5 4

Win Rate (%)

Average Rank

100.0 100.0 100.0 90.9 90.9 90.9 90.9 81.8 72.7 63.6 81.8 72.7 63.6 72.7 63.6 45.5 72.7 54.5 36.4 36.4 45.5 36.4

2.00 3.18 3.45 4.27 5.73 6.73 8.91 9.27 10.09 10.64 11.09 12.27 13.27 14.55 14.82 15.18 15.73 17.27 17.64 18.09 19.09 19.73

Record · ID 31269 · SHA-256 c81b1f03cb3aa630
Retrieved via Conceptio — every document is proof-bundled with source, license, and retrieval metadata.