AGENTIC - IMODELS: Evolving agentic interpretability tools via autoresearch
Chandan Singh1
Yan Shuo Tan2
arXiv:2605.03808v1 [cs.AI] 5 May 2026
Weiwei Yang1
Weijia Xu1
Michel Galley1
2
Zelalem Gero1 Jianfeng Gao1
1 Microsoft Research National University of Singapore
Abstract Agentic data science (ADS) systems are rapidly improving their capability to autonomously analyze, fit, and interpret data, potentially moving towards a future where agents conduct the vast majority of data-science work. However, current ADS systems use statistical tools designed to be interpretable by humans, rather than interpretable by agents. To address this, we introduce AGENTIC - IMODELS, an agentic autoresearch loop that evolves data-science tools designed to be interpretable by agents. Specifically, it develops a library of scikit-learn-compatible regressors for tabular data that are optimized for both predictive performance and a novel LLM-based interpretability metric. The metric measures a suite of LLMgraded tests that probe whether a fitted model’s string representation is “simulatable” by an LLM, i.e. whether the LLM can answer questions about the model’s behavior by reading its string output alone. We find that the evolved models jointly improve predictive performance and agent-facing interpretability, generalizing to new datasets and new interpretability tests. Furthermore, these evolved models improve downstream end-to-end ADS, increasing performance for Copilot CLI, Claude Code, and Codex on the BLADE benchmark by up to 73%.1
1
Introduction
The data-science lifecycle has historically been a human-centered process end-to-end: a human defines the question, carries out the analysis using interpretable tools, and communicates the findings. As agentic data science (ADS) systems grow more capable, intermediate analytical steps are increasingly delegated to agents, e.g. selecting models, interpreting coefficients, and drawing conclusions [1–3]. This improvement in capabilities may be critical in accelerating scientific discovery [4, 5], and in tackling emerging issues such as scalable oversight of AI systems [6, 7]. However, the performance of ADS systems is limited by their tools for working with and interpreting data: existing interpretability tools/models and their accompanying implementations produce outputs designed to be read by humans, often containing visualizations or intervenable components that may be difficult for agents to parse [8–13]. This mismatch can derail ADS systems, leading to unreliable analysis [14, 15] and obscured analytical choices [16, 17]. 1 All code for reproducing and adapting AGENTIC - IMODELS is made available at github.com/csinva/agentic-imodels. We additionally release the final library of AGENTIC - IMODELS we develop as a Python package / agent skill for easy integration into ADS workflows.
Preprint.
b
Agentic autoresearch loop
Agentic-imodels library
Performance tests
Interpretability tests
Normalized prediction rank (lower is better)
a
0.8
Black-box Additive Tree imodels Agentic-imodels
0.7 0.6
MLP
FIGS (large)
0.5 PyGAM
0.4
GBM
0.3 0.2
EBM TabPFN
0.2
DT (large) HSTree (large) Ridge OLS SmartAdditive TinyDTDepth2 WinsorizedSparseOLS HingeGAM SparseSignedBasisPursuit TeacherStudentRuleSpline HybridGAM DualPathSparseSymbolic DistilledTreeBlendAtlas HingeEBM
0.3 0.4 0.5 0.6 0.7 0.8 0.9 Agent interpretability score (held-out tests)
1.0
Figure 1: (a) Overview of the AGENTIC - IMODELS autoresearch loop, which optimizes a Python class for predictive performance and agent interpretability (evaluated through LLM-based simulatability tests). (b) The discovered AGENTIC - IMODELS (blue points) improve the Pareto frontier of predictive performance and interpretability over baselines from the literature. See evaluation details in Sec. 4.2. This emerging problem leads us to rethink interpretability in the modern era: rather than designing data-science tools to be interpretable by humans, we aim to design tools that are interpretable by agents. Doing this requires adapting the rich literature on human-centered interpretable machine learning [18–20], which has yielded interpretable models such as decision trees, generalized additive models, and rule lists [8]. This field has traditionally quantified interpretability in terms of concepts such as simulatability, sparsity, and modularity [21, 8, 9], ideally followed by quantitative evaluation with human experiments [22, 20]. In our setting, we replace these human experiments with tests that measure whether an LLM can accurately simulate the model’s behavior in terms of predictions, feature effects, and counterfactuals solely by reading its string representation. These LLM-based tests provide a key practical advantage: they enable computing an agent interpretability score for any model without the need for human experiments. We use this agent interpretability score to guide the automated design of new interpretable models. To do so, we propose AGENTIC - IMODELS, an autoresearch loop that prompts a coding agent (e.g. Claude Code) to iteratively modify a Python class so it simultaneously achieves a high interpretability score and strong predictive performance (Fig. 1a). The loop consists of the coding agent generating candidate model modifications, which are then evaluated using the interpretability score and predictive performance metrics across a large suite of tabular datasets. The coding agent uses these metrics to refine the model, generating new candidate modifications and repeating the evaluation process until the model achieves a user-specified balance of predictive performance and interpretability. In our experiments, we use AGENTIC - IMODELS to build scikit-learn-compatible regressors for tabular data. We run the loop on a suite of 65 tabular datasets with both Claude Code and Codex, and find that it discovers new model classes that push the interpretability-performance frontier beyond a wide range of existing baselines (Fig. 1b), even for held-out datasets and new interpretability tests (Sec. 4.2). They are also useful downstream: equipping four different ADS agents with the evolved models improves their performance on the BLADE end-to-end benchmark [23] by 8%–73% over standard interpretability tools (Sec. 4.3). Qualitatively, we find that the evolved AGENTIC - IMODELS combine existing algorithms in novel ways and present themselves in forms amenable to agent interpretation (Sec. 4.4).
2
Related work
Interpretable machine learning. Interpretable models, those whose reasoning can be directly understood by humans, have long been advocated as preferable to post-hoc explanations of blackbox models [8, 24, 9]. Prominent model families include decision trees [10, 25], rule lists [26], generalized additive models (GAMs) [27–29], and sparse linear models [30], implemented in popular 2
packages such as imodels [12] and interpretML [13]. More recent approaches use LLMs to build interpretable models, e.g. Aug-imodels hand-designs methods to build an interpretable model from an LLM [31], and alternative works train transformers to directly produce an interpretable model, e.g. [32, 33]. Our work extends this ecosystem by using agents to discover new agent-interpretable model classes. Agentic data science (ADS). A rapidly growing literature studies how well coding and reasoning agents can conduct end-to-end data analysis, primarily by measuring agent capability on new benchmarks [2, 3, 34–36, 1], and by documenting ADS failure modes such as LLM-assisted phacking [14], missing sanity checks [15, 17, 37], and unreliable LLM-driven annotation of data and findings [38, 39, 16]. A parallel thread design methods that use agents to directly describe data [40–43] or describes features of a pre-trained foundation model [44–47]. In both threads the agent relies upon human-designed interpretability methods. In contrast, we treat the coding agent as a designer of those methods, by having it directly optimize a model for LLM-graded interpretability. Automated model discovery and autoresearch. Agents that generate and refine code for scientific purposes are a growing area of interest [5, 48–50], particularly agentic “autoresearch” loops against a verifiable outcome, e.g. AlphaEvolve [51] and related works [52–55]. A related line of work focuses on reusable evolving text prompts (i.e. “skills”) that can be used to improve agent performance, e.g. evolving skills collectively across users [56], using multi-agent systems [57, 58], or from scientific resources [59]. Alternative works more broadly seek to optimize agent harnesses end-to-end [60, 61]. Recent test-time adaptation methods also improve agent capabilities through iterative self-refinement, potentially creating reusable skills as a byproduct [62–65]. Our work applies the autoresearch approach specifically to discovering ML models that are interpretable to agents.
3
Methods
The goal of AGENTIC - IMODELS is to automatically discover model classes that are both accurate and interpretable to LLMs. It does this using an agentic autoresearch loop that simultaneously optimizes predictive performance and a novel interpretability metric, using a coding agent that iteratively proposes and refines model implementations (Fig. 1a). In our instantiation of AGENTIC - IMODELS, each model class is a scikit-learn-compatible [66] Python class with fit, predict, and __str__. Predictive performance is measured by taking the average regression performance across a suite of tabular regression datasets. Interpretability is measured by LLM-graded tests that probe whether an LLM can answer quantitative questions about the fitted model by reading only its __str__ output. The coding agent (e.g. Claude Code) is prompted to iteratively modify the model class, evaluate both metrics, and refine based on feedback. We describe our choices for each component in detail below, but note that the AGENTIC - IMODELS loop can be generalized to optimize a broader set of interpretability tools. 3.1
Evaluating predictive performance
Each model’s predictive performance is measured using a suite of regression datasets. For each dataset, each model is evaluated by fitting the model to a training split (and potentially tuning hyperparameters via cross-validation splits on the training set), and then computing the test performance on the dataset. In our case we compute the test-set root mean squared error (RMSE) for each dataset in the suite. To avoid sensitivity to noisy datasets, we rank models on each dataset and then average the ranks across datasets (rather than averaging RMSE directly). 3.2
Evaluating agent interpretability via LLM-graded interpretability tests
For the AGENTIC - IMODELS autoresearch loop, we require a metric that serves as a proxy for the extent to which a model’s representation enables a downstream agent to reliably reason about its behavior. We additionally desire the metric to be (1) automated, requiring no human evaluation and (2) model-agnostic, unlike existing proxies such as tree depth or number of coefficients which do not generalize across model families [9, 20, 18]. We simultaneously satisfy both criteria by defining the Agent interpretability score of a fitted model as the pass rate of LLM-based interpretability tests, where each test evaluates whether an LLM can correctly answer a quantitative question about 3
Generate synthetic data: y = 3x₀ + 2x₁ + ε, ε ~ N(0, 0.3), n = 500, x₀, x₁, x₂ ~ N(0, 1) Fitted model Extract __str__ output:
Ridge Regression (α=1.0 chosen by CV): y = 2.98*x0 + 1.97*x1 + 0.03*x2 + 0.01 Coefficients: x0: 2.9812
x1: 1.9734
x2: 0.0285
intercept: 0.0102
Prompt LLM with __str__ output + query: Point prediction
Feature ranking
Direction of change
Counterfactual target
What does the model predict for x0=2, x1=0, x2=0?
Rank the 3 most important features from most to least
By how much does the prediction change when x0 goes from 0 to 1?
When x1=1 & x2=0, what value of x0 would make the model predict 8.5?
Grade against ground truth: Truth: 5.97 LLM: 5.97
✓
Truth: x0, x1, x2 LLM: x0, x1, x2
Truth: +2.98 ✓ LLM: +2.98
✓
Truth: 2.18 ✗ LLM: 2.83
Figure 2: The interpretability test protocol, illustrated on Ridge regression with four of the 43 tests. Synthetic data is generated and the model is fit. The LLM receives only the __str__ output and a query. The response is graded against the ground truth, which is obtained either from evaluating the fitted function or from knowledge of the ground-truth function. In some cases, the question itself requires evaluating the prediction function, e.g. in the counterfactual target test the value 8.50 is obtained by evaluating the model when setting x0 = 2.18, x1 = 0.0, x2 = 0.0. Here the LLM passes three tests but fails the counterfactual target test. Even though the information to compute the counterfactual target is present in the __str__ output, the representation does not make it easy for the LLM to solve the inverse problem compared to other model representations (e.g. a decision tree that makes the predicted value easily apparent in leaf nodes).
the model’s predictions, given only the model’s __str__ output (note that it is impossible to score highly on the tests without the model’s __str__ providing the necessary information). Every test is broken down into a few steps, illustrated in Fig. 2 for a Ridge regression model. First, synthetic data is generated from a known ground-truth function, here y = 3x0 + 2x1 + ε with an irrelevant third feature x2 . Second, the model is fit to this data, producing a fitted model whose __str__ method returns a human-readable summary (in this case, the coefficient equation y = 2.98*x0 + 1.97*x1 + 0.03*x2 + 0.01, along with some extra printed information such as the regularization parameter and cross-validation score). Third, an LLM receives only this string and a quantitative query about the model, e.g. “What does this model predict for x0 =2, x1 =0, x2 =0?”. Each response is then graded against the ground truth (with numerical tolerance), yielding a binary pass/fail per test. For simplicity, Fig. 2 shows four tests using the same synthetic data / model, but in actuality, the tests span different synthetic datasets / models to cover diverse scenarios. Drawing from prior work on evaluating interpretability via human experiments [20, 22, 18], we develop a total of 200 tests grouped into six categories: • Feature attribution (32 tests) asks which features matter: identifying the most important feature, ranking features, detecting irrelevant ones, and determining the sign of effects. • Point simulation (43 tests) asks the LLM to predict the model’s output for a specific input, ranging from simple single-feature queries to 20-feature sparse problems. • Sensitivity analysis (32 tests) asks how predictions change when inputs change: direction of change, unit sensitivity, and nonlinear threshold detection. • Counterfactual reasoning (28 tests) poses inverse problems: given a target output, find the input value that produces it. • Structural understanding (28 tests) asks meta-questions about the model itself: whether it is compact and what its decision regions look like. 4
• Complex function simulation (37 tests) tests simulation on nonlinear synthetic data including interactions, piecewise functions, exponential decay, and nested thresholds. The tests are randomly split into a development set (43 tests) and a held-out set (157 tests), which can be used to verify that the model strings are genuinely simulatable rather than simply reciting the answers to the original tests. The full list of the tests with descriptions and pass rates is in Table A1 (development set) and Table A2 (held-out set). 3.3
Fully autonomous agentic autoresearch loop
Given the two evaluation metrics above, we use a coding agent to search the space of model implementations, similar to recent work on “autoresearch” [51–55]2 . We instruct the agent to modify only a single file, interpretable_regressor.py, containing a starter InterpretableRegressor class. It then follows an iterative loop (Fig. 1a): (1) edit the model code with a new idea, (2) run both evaluations, (3) check whether metrics improved, (4) save the resulting metrics / model code, and (5) repeat. Throughout this autoresearch loop, the agent persists its memory through a csv file that records each model’s name, basic idea, and resulting metrics. Results for baseline models are recorded in this csv file before starting the loop, providing a reference point for the agent’s search. Prompting choices in the autoresearch loop. The agentic loop is highly configurable to various choices the user has in mind. We prompt the coding agent to explore new ideas, i.e. the prompt ends with Do not simply import a known interpretable model and change its hyperparameters — build your own from scratch using basic building blocks or substantially modify an existing one. The goal is to discover new models, not just find the best hyperparameters for known models...BE CREATIVE! To ensure the agent continues, we prompt it to continue, i.e. Once the experiment loop has begun, do NOT pause to ask the human if you should continue. Run until manually stopped. Always keep going. . If the agent does stop, it is automatically prompted with the command Continue. until the user has determined that sufficient iterations have been completed (in our experiments typically 50-200 iterations). We additionally prompt the agent to avoid simply memorizing and reciting the answers to the given interpretability tests, i.e. Do not simply write the answers to the interpretability tests without actually building a model. See the full prompts used in the agentic loop in the Github repo. The token usage for running the autoresearch loop varies by model and thinking effort; the total generated tokens across all our final experiments is approximately 70 million tokens (see detailed breakdown in Table B2). While this is a substantial number of tokens, the cost to replicate these experiments is relatively modest (as of Spring 2026, replicating the experiments that use Claude Code can be done with 3 months of a $100 Claude Max subscription).
4
Results
4.1
Experimental setup
Datasets. We evaluate predictive performance during the AGENTIC - IMODELS loop on 65 regression datasets, consisting of all the regression datasets from the OpenML TabArena suite [67] (7 datasets) and all the regression datasets from PMLB excluding the duplicated synthetic Friedman datasets [68] (58 datasets). For evaluation on held-out data, we use the 16 OpenML regression datasets from a recent study [69] that do not overlap with the development datasets. Each dataset is preprocessed with an 80%-20% train-test split, subsampled to at most 1,000 samples and 50 features, and the outcome variable is normalized to zero mean and unit standard deviation on the training set. Categorical features are ordinal-encoded and missing values are imputed with medians. See details for all development datasets in Table A3 and held-out datasets in Table A4. Baselines. We compare evolved models against 16 baselines spanning 5 families: Linear: linear regression (OLS), Ridge regression, the Lasso [11]; Tree: DecisionTree with 8 and 20 leaf nodes (DT mini and DT large), HSTree with 8 and 20 leaf nodes (HSTree mini and HSTree large) [70]; Additive: PyGAM [71], Explainable boosting machine (EBM) [29, 28]; Rule-based: FIGS (8 and 20 rules) [72], RuleFit [73]; Black-box: RandomForest (RF) [74], Gradient-boosted decision trees (GBM), multilayer 2 See sample implementation at https://github.com/karpathy/autoresearch.
5
perceptron (MLP), TabPFN-v2.5 [75]. These models are run with their default hyperparameters in their respective packages (scikit-learn [66], imodels [12], interpretML [13]), including default settings for selecting hyperparameters via cross-validation. Each baseline provides a __str__ method that renders its internal structure as text (e.g., tree diagrams, coefficient tables, or feature importance rankings). Agent configurations. We run AGENTIC - IMODELS with two coding agents, Claude Code v2.1.118 (Opus-4.6/Opus-4.7) and Codex v0.118.0 (GPT-5.3), at 3 reasoning-effort levels each (medium, high, extra-high), for 6 runs total producing 42-103 working models per run. We additionally run Claude Code Opus-4.7 at medium reasoning effort 3 times, each with minor prompt variations, e.g. adding Be creative! or Try out-of-the-box ideas!, producing 44-63 working models per run. All runs use GPT-4o [76] as the LLM evaluator unless otherwise noted. End-to-end ADS setup. To test whether the evolved models help in practice, we evaluate four AI agents on the BLADE benchmark [23], which consists of 13 datasets, each paired with a datagrounded question: GitHub Copilot CLI v1.0.40 (powered by Gemini-2.5-pro or Sonnet-4.5), Claude Code v2.1.117 (Sonnet-4.6), and Codex v0.118.0 (GPT-5.3). For each question, BLADE provides a gold-standard analysis from expert data scientists and researchers. We run BLADE with 4 conditions. In the first condition (standard tools), the agent is prompted to use widespread interpretable models from scikit-learn, imodels, statsmodels, and scipy.stats. In the second condition (AGENTIC - IMODELS), the agent is additionally given access to the evolved AGENTIC - IMODELS formatted as a Python package: we curate the library generated by topperforming models across our runs (the highlighted models in Fig. 1b) into a package with 10 evolved regressors spanning the interpretability-performance frontier, and point the agent to a README.md that documents the API and how to use the models. The third and fourth conditions are designed to test whether any observed improvement in the AGENTIC - IMODELS condition is due to explicitly emphasizing a particular package rather than the substance of the models in that package. They use the same prompt wording as in the AGENTIC - IMODELS condition, but change the line pointing to the AGENTIC - IMODELS package: in one control the prompt points to the imodels package [12] and in the other it points to the interpretML package [13]. Every ADS agent then provides analyses for each dataset under all four conditions. Each resulting analysis is compared to the gold-standard human analysis using a rubric to assess (i) correctness, (ii) completeness, and (iii) clarity, each on a scale of 1 (worst) to 10 (best). Each analysis is run 3 times and the resulting analysis is judged 3 times by GPT-4o, resulting in 9 evaluations per dataset per condition per agent; see prompts for the judge rubric and for running the agent on GitHub. 4.2
AGENTIC - IMODELS push the interpretability-performance frontier
Fig. 3a-c shows the interpretability-predictive performance frontier across the different models (breaking down the results shown in Fig. 1b). Predictive performance is measured via the normalized prediction rank, i.e. we pool the models from all runs (16 baselines plus 467 evolved models from the 9 AGENTIC - IMODELS runs), rank them by RMSE on each of the 65 tabular datasets, average the per-dataset ranks for each model, and divide by the pool size so that ranks lie in [0, 1] with lower being better. Baselines (gray crosses) show a clear tradeoff: TabPFN (rank 0.16, agent interpretability score 0.17) predicts best but suffers from weak interpretability while the most interpretable baselines (OLS interpretability 0.69, HSTree-mini at 0.66) have normalized ranks of 0.60-0.82. AGENTIC - IMODELS yield Pareto improvements. Evolved AGENTIC - IMODELS populate the previously empty low-rank, high-interpretability region across different coding agents and reasoning efforts. For example, Claude Code (Fig. 3a) produces HingeEBM (5bag) at rank 0.19 with agent interpretability score 0.71 (nearly matching TabPFN’s rank at over 4 times its agent interpretability score). Codex (Fig. 3b) reaches comparable interpretability with models such as TeacherStudentRuleSpline (v1) (rank 0.36, agent interpretability score 0.80), and tends toward simpler sparse/linear modifications. Even at a fixed agent and reasoning effort, runs display high variability (Fig. 3c); this variability is on par with the differences observed across different reasoning efforts and agents. 6
0.7
MLP HSTree (mini) Ridge OLS
0.6 0.5
RF
0.4
GBM
0.3 0.2
EBM TabPFN
0.0 0.2 0.4 0.6 0.8 Agent interpretability score (held-out tests) (c) Opus-4.7 medium: 3 repeated runs 0.9 0.8 0.7
Baselines Run 1 Run 2 Run 3
MLP HSTree (mini) Ridge OLS
0.6 0.5
RF
0.4
GBM
0.3 0.2
EBM TabPFN
0.0 0.2 0.4 0.6 0.8 Agent interpretability score (held-out tests)
(b) Codex (GPT-5.3) 0.9 0.8 0.7
Baselines medium effort high effort extra-high effort
MLP HSTree (mini) Ridge OLS
0.6 0.5
RF
0.4
GBM
0.3 0.2
1.0 0.8
EBM TabPFN
0.0 0.2 0.4 0.6 0.8 Agent interpretability score (held-out tests) (d) Interpretability generalization
Po t ha rewaentia cki rd l ng
0.8
Normalized prediction rank (lower is better)
0.9
Agent interpretability score (development tests)
Normalized prediction rank (lower is better) Normalized prediction rank (lower is better)
(a) Claude Opus-4 Baselines Opus-4.6 medium effort Opus-4.6 high effort Opus-4.7 extra-high effort
OLS HSTree (large)
0.6 EBM PyGAM
0.4 0.2 0.0
MLP TabPFN
Baselines Opus-4.6 medium Opus-4.7 extra-high Codex medium Codex extra-high
0.0 0.2 0.4 0.6 0.8 Agent interpretability score (held-out tests)
Figure 3: AGENTIC - IMODELS versus baselines (gray crosses) in terms of both predictive performance (the RMSE mean rank: each model’s mean rank is computed across datasets, then normalized to [0, 1] with lower being better) and agent interpretability scores (fraction of tests passed from the 157-test held-out generalization suite (Table A2)). Across different settings, AGENTIC - IMODELS achieve Pareto improvements in terms of predictive performance and interpretability: (a) Claude Opus-4 models across three effort levels, (b) Codex (GPT-5.3) models across three effort levels, (c) Claude Opus-4.7 at medium effort with 3 random repetitions. (d) Agent interpretability scores on the development set of tests versus the held-out set for four of the runs shown in (a)/(b) (using matching colors/markers). Some models exhibit held-out agent interpretability scores significantly below their development scores, suggesting potential reward hacking. Excluding the points in this shaded region, the remaining points show a strong positive correlation (r = 0.65).
Reward hacking of development interpretability tests. Fig. 3d compares each model’s interpretability on the development interpretability tests versus the held-out tests. Some evolved models (most prominently Codex extra-high) score high on development tests but substantially lower on held-out tests (shaded region). Manually inspecting these models reveals that they often engage in reward hacking, e.g. reciting answers to the development interpretability tests directly in their model strings. Nevertheless, most evolved models successfully generalize: after excluding the shaded region, the two agent interpretability scores are strongly correlated (r = 0.65, 228 points). Validating the agent interpretability metric via ablations. Beyond the held-out interpretability tests above, we ask whether the evolved models’ gains are stable under two ablations of the evaluation pipeline. First, we test perturbations to the evaluation of the agent interpretability tests. We evaluate swapping the LLM evaluator from GPT-4o to GPT-5.4 or Claude Haiku-4.5, also adding minor variations to the prompt used for evaluating tests. We find that results are consistent across the evaluator LLM and the prompt variationsin all cases, including when using GPT-5.4 for post-hoc evaluation (correlation of scores between evaluators is 0.83), when Claude Haiku-4.5 is used for posthoc evaluation (correlation of 0.85), or when the evaluator is changed during the AGENTIC - IMODELS 7
10 Score (1-10)
9
Copilot-Gemini (standard tools) Copilot-Sonnet (standard tools) Claude (standard tools) Codex (standard tools)
Copilot-Gemini (+imodels) Copilot-Sonnet (+imodels) Claude (+imodels) Codex (+imodels)
Copilot-Gemini (+agentic-imodels) Copilot-Sonnet (+agentic-imodels) Claude (+agentic-imodels) Codex (+agentic-imodels)
Copilot-Gemini (+interpretml) Copilot-Sonnet (+interpretml) Claude (+interpretml) Codex (+interpretml)
b
Avg score (+agentic-imodels)
a
9 8
8 7
7
6 5
Copilot-Gemini Copilot-Sonnet Claude Codex
6
4
Correctness
Completeness
Clarity
Avg
5
4
6
Figure 4: Including AGENTIC - IMODELS improves performance on the BLADE benchmark across 4 ADS agents: GitHub Copilot CLI (gemini-2.5-pro), GitHub Copilot CLI (sonnet-4.5), Claude Code (sonnet-4.6), and Codex CLI (GPT-5.3). (a) Aggregate scores across the 13 BLADE datasets, with four prompt conditions per agent: standard tools (no explicit package emphasis), prompt emphasizing the imodels package, prompt emphasizing the interpretML package, and prompt emphasizing our AGENTIC - IMODELS package. Including AGENTIC - IMODELS yields substantial improvements over all the other conditions, across different agents and evaluation axes, particularly for the weaker ADS systems where the margin for improvement is larger. (b) Per-dataset average score (mean of correctness, completeness, clarity) with performance using standard tools versus AGENTIC - IMODELS; points above the diagonal indicate improvement. Error bars show standard error of the mean across agent seeds and judge seeds (9 evaluations per dataset per condition: 3 agent runs × 3 judge runs).
optimization loop (where the AGENTIC - IMODELS loop continues to achieve Pareto improvements); see Sec. B.1 for extended results. Validating predictive performance on held-out datasets. Second, we test whether the evolved models’ predictive performance generalizes to held-out datasets. We re-evaluate predictive performance for the 103 evolved models from the Claude Opus-4.6 medium-effort main run (blue points in Fig. 3a) on 16 held-out OpenML datasets in addition to the 157 new held-out interpretability tests, finding that the same conclusions hold and evolved models continue to dominate the Pareto frontier (see Sec. B.2). The best-ranked evolved model, SmartAdditive (v1) yields 69% agent interpretability (beating EBM’s 49% interpretability) while remaining competitive on rank, and the evolved Pareto frontier extends up to HingeGAM (10bp) at 78% interpretability. 4.3
End-to-end ADS performance improves using AGENTIC - IMODELS
In the end-to-end ADS evaluation on the BLADE datasets, we find that providing every agent with the evolved AGENTIC - IMODELS improves its performance across all 3 rubric dimensions (Fig. 4a). Improvements are largest for the ADS systems with weaker base models than for the strongest one (Codex): Copilot CLI driven by gemini-2.5-pro improves its average score by 72.5% with 13/13 datasets improved, Copilot CLI driven by sonnet-4.5 improves by 47.0% with 13/13 improved, Claude Code (sonnet-4.6) improves by 32.3% with 13/13 improved, and Codex (GPT-5.3) improves by 7.9% with 13/13 improved (Fig. 4b). Explicitly emphasizing the imodels or interpretML packages in the prompt does not yield significant improvements for the weaker ADS systems (Copilot-Gemini shifts by −1.8%/+2.1% and Copilot-Sonnet by −2.4%/+3.4% for imodels/interpretML, all within the SE bars of the no-emphasis baseline) and yields a relatively small improvement for Codex (2.7% for imodels and 2.8% for interpretML compared to 7.9% for AGENTIC - IMODELS). The gains in performance are attributable to different properties of the provided evolved models. For example, one qualitative gain comes from the per-feature linear coefficients that HingeEBM prints in its __str__, which give the agent a compact magnitude-and-sign summary for every input. On the mortgage dataset [77], this surfaces gender’s small but non-zero effect (average score 4.6 → 8.6 Copilot-Gemini, 5.9 → 7.8 Claude, 7.2 → 8.0 Codex); on the Crofoot dataset [78], it lets the agent report a directional effect for specific variables rather than a binary significance judgement, raising completeness by +2.7 for both Codex and Claude. 8
8
Avg score (standard tools)
4.4
Qualitative analysis of selected AGENTIC - IMODELS
In this section we analyze patterns across 10 of the high-performing evolved models (the models highlighted in Fig. 1b). See details and example __str__ outputs for some of the models in Sec. C. Pattern 1: bounded display complexity by construction. Across the 10 models, the agents often bound the size of __str__ through hard architectural caps rather than soft regularization on a flexible model. The caps take many forms: a fixed tree depth (e.g., TinyDTDepth2), a top-k feature budget, a fixed number of quantile knots or breakpoints in a hinge basis, a forward-selected basis with rounded coefficients, or a single-line symbolic equation distilled from a stronger teacher. This contrasts with strong baselines such as EBM, FIGS, and PyGAM, whose displays grow with the data and which sit at the worst end of the held-out interpretability axis despite predicting well (Fig. 1b). This intrinsic interpretability principle echoes classical interpretable-ML works that treat complexity as a model constraint [8, 30, 10], and it keeps the printed string short enough for the LLM evaluator to simulate end-to-end. Pattern 2: display optimization as a first-class design axis. Eight of the ten models pair a standard predictor with a non-trivial display strategy that is optimized separately from the architecture itself. Examples include (i) per-feature gating that renders a learned shape function as a single coefficient when an R2 test passes and a short piecewise table otherwise (e.g. SmartAdditive); (ii) collapsing a hinge or basis expansion into an effective linear slope for the printed equation while keeping the precise predictor underneath (e.g. HingeEBM); and (iii) compiling a distilled student into a single-row symbolic equation written specifically for __str__. The choice of display also varies based on the underlying agent: Codex with GPT-5.3 tends toward sparse symbolic displays, while Claude with Opus-4.7 tends toward GAM-style displays.
5
Discussion
Decades of interpretability research have optimized models for a human reader. In the coming decades, with the increasing capabilities of AI agents, building tools for agents to reliably interpret data will be critical to ensuring that insights from data are correctly understood and acted upon. AGENTIC - IMODELS is a first step toward taking this shift seriously, and may be the first of many works that adapt the structural properties that aid human interpretability (e.g., sparsity, additivity, monotonicity) into LLM-suitable properties. Limitations. One limitation of the study here is that our end-to-end ADS evaluations rely on LLM-as-judge for scoring, which may introduce some bias or artifacts (although notably the underlying ground-truth analysis is conducted by expert humans, making the judging task easier). Similarly, our interpretability tests use LLMs for evaluation. While this is effective for evaluating the agent-interpretability of the models being tested, it is unclear how well this measures the humaninterpretability of representations, a question that may be explored in future user studies. As LLM user simulators become more accurate [79, 80], they could be used to automatically evaluate the interpretability of models to provide more reliable assessments that match human judgements. Additionally, the interpretability tests we design do not cover all possible aspects of model interpretability, and particular applications may require specialized tests beyond our current suite. Another limitation is the presence of reward hacking in the experiments, which highlights that our metric is not a perfect proxy. However, the strong correlation between held-out agent interpretability scores and downstream performance suggests that the metric captures useful signal despite being imperfect. Future works may want to design the interpretability tests and the agentic loop to be more robust to increasingly sophisticated agents. For example, tests could be exposed only through an API so the agent never sees the questions or expected answers in plain text. Finally, the agent loop is expensive in LLM API calls, both for the coding agent and for the interpretability evaluator, although these costs may decrease as LLM inference becomes cheaper and more efficient over time. Future work. A natural next step is to broaden the scope of the framework along two axes: across tasks (e.g. classification, time series, text data, larger datasets) and across more general tools (e.g. text explanation pipelines, causal interventions). The same test suite could also be repurposed as an optimization target for improving existing model packages without re-architecting them [12, 13]. 9
Downstream, incorporating the evolved models into real-world agentic workflows (e.g., scientific discovery pipelines) would test whether the improvements transfer to more complex analyses. On the human side, our framework could be modified to optimize human-AI collaborative workflows, rather than focusing on agents or humans in isolation [81, 82]. As LLMs themselves improve at parsing structured models [83], AGENTIC - IMODELS will be increasingly powerful and both the evolved tools and the metric they optimize against will enable more sophisticated, potentially superhuman interpretation of complex data.
References [1] Siyuan Guo, Cheng Deng, Ying Wen, Hechang Chen, Yi Chang, and Jun Wang. Ds-agent: Automated data science by empowering large language models with case-based reasoning. arXiv preprint arXiv:2402.17453, 2024. [2] Fan Nie, Junlin Wang, Harper Hua, Federico Bianchi, Yongchan Kwon, Zhenting Qi, Owen Queen, Shang Zhu, and James Zou. Dsgym: A holistic framework for evaluating and training data science agents. arXiv preprint arXiv:2601.16344, 2026. [3] Ziru Chen, Shijie Chen, Yuting Ning, Qianheng Zhang, Boshi Wang, Botao Yu, Yifei Li, Zeyi Liao, Chen Wei, Zitong Lu, et al. Scienceagentbench: Toward rigorous assessment of language agents for data-driven scientific discovery. arXiv preprint arXiv:2410.05080, 2024. [4] Hanchen Wang, Tianfan Fu, Yuanqi Du, Wenhao Gao, Kexin Huang, Ziming Liu, Payal Chandak, Shengchao Liu, Peter Van Katwyk, Andreea Deac, et al. Scientific discovery in the age of artificial intelligence. Nature, 620(7972):47–60, 2023. [5] Chris Lu, Cong Lu, Robert Tjarko Lange, Jakob Foerster, Jeff Clune, and David Ha. The ai scientist: Towards fully automated open-ended scientific discovery. arXiv preprint arXiv:2408.06292, 2024. [6] Samuel R Bowman, Jeeyoon Hyun, Ethan Perez, Edwin Chen, Craig Pettit, Scott Heiner, Kamilė Lukošiūtė, Amanda Askell, Andy Jones, Anna Chen, et al. Measuring progress on scalable oversight for large language models. arXiv preprint arXiv:2211.03540, 2022. [7] Balint Gyevnar and Atoosa Kasirzadeh. Ai safety for everyone. Nature Machine Intelligence, 7(4):531–542, 2025. [8] Cynthia Rudin. Please stop explaining black box models for high stakes decisions. arXiv preprint arXiv:1811.10154, 2018. [9] W. James Murdoch, Chandan Singh, Karl Kumbier, Reza Abbasi-Asl, and Bin Yu. Definitions, methods, and applications in interpretable machine learning. Proceedings of the National Academy of Sciences of the United States of America, 116(44):22071–22080, 2019. [10] L. Breiman, J. H. Friedman, R. A. Olshen, and C. J. Stone. Classification and Regression Trees. Wadsworth and Brooks, Monterey, CA, 1984. [11] Robert Tibshirani. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society. Series B (Methodological), pages 267–288, 1996. [12] Chandan Singh, Keyan Nasseri, Yan Shuo Tan, Tiffany Tang, and Bin Yu. imodels: a python package for fitting interpretable models. Journal of Open Source Software, 6(61):3192, 2021. [13] Harsha Nori, Samuel Jenkins, Paul Koch, and Rich Caruana. Interpretml: A unified framework for machine learning interpretability. arXiv preprint arXiv:1909.09223, 2019. [14] Samuel G. Z. Asher, Janet Malzahn, Jessica M. Persano, Elliot J. Paschal, Andrew C. W. Myers, and Andrew B. Hall. Do claude code and codex p-hack? sycophancy and statistical analysis in large language models, February 2026. Preprint. [15] Zachary T. Rewolinski, Austin V. Zane, Hao Huang, Chandan Singh, Chenglong Wang, Jianfeng Gao, and Bin Yu. Sanity checks for agentic data science, 2026. [16] Ziming Luo, Atoosa Kasirzadeh, and Nihar B Shah. The more you automate, the less you see: Hidden pitfalls of ai scientist systems. arXiv preprint arXiv:2509.08713, 2025. [17] Venkatesh Sivaraman, Patrick Vossler, Adam Perer, Julian Hong, and Jean Feng. More than "means to an end": Supporting reasoning with transparently designed ai data science processes, 2026.
10
[18] Harmanpreet Kaur, Harsha Nori, Samuel Jenkins, Rich Caruana, Hanna Wallach, and Jennifer Wortman Vaughan. Interpreting interpretability: understanding data scientists’ use of interpretability tools for machine learning. In Proceedings of the 2020 CHI conference on human factors in computing systems, pages 1–14, 2020. [19] Sungsoo Ray Hong, Jessica Hullman, and Enrico Bertini. Human factors in model interpretability: Industry practices, challenges, and needs. Proceedings of the ACM on Human-Computer Interaction, 4(CSCW1):1–26, 2020. [20] Finale Doshi-Velez and Been Kim. A roadmap for a rigorous science of interpretability. arXiv preprint arXiv:1702.08608, 2017. [21] Zachary C Lipton. The mythos of model interpretability. arXiv preprint arXiv:1606.03490, 2016. [22] Isaac Lage, Emily Chen, Jeffrey He, Menaka Narayanan, Been Kim, Sam Gershman, and Finale DoshiVelez. An evaluation of the human-interpretability of explanation. arXiv preprint arXiv:1902.00006, 2019. [23] Ken Gu, Ruoxi Shang, Ruien Jiang, Keying Kuang, Richard-John Lin, Donghe Lyu, Yue Mao, Youran Pan, Teng Wu, Jiaqian Yu, et al. Blade: Benchmarking language model agents for data-driven science. arXiv preprint arXiv:2408.09667, 2024. [24] Cynthia Rudin, Chaofan Chen, Zhi Chen, Haiyang Huang, Lesia Semenova, and Chudi Zhong. Interpretable machine learning: Fundamental principles and 10 grand challenges. arXiv preprint arXiv:2103.11251, 2021. [25] J. Ross Quinlan. Induction of decision trees. Machine learning, 1(1):81–106, 1986. [26] Benjamin Letham, Cynthia Rudin, Tyler H McCormick, and David Madigan. Interpretable classifiers using rules and bayesian analysis: Building a better stroke prediction model. 2015. [27] Trevor Hastie and Robert Tibshirani. Generalized additive models. Statistical Science, 1(3):297–318, 1986. [28] Yin Lou, Rich Caruana, Johannes Gehrke, and Giles Hooker. Accurate intelligible models with pairwise interactions. In Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 623–631, 2013. [29] Rich Caruana, Yin Lou, Johannes Gehrke, Paul Koch, Marc Sturm, and Noemie Elhadad. Intelligible models for healthcare: Predicting pneumonia risk and hospital 30-day readmission. In Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining, pages 1721–1730, 2015. [30] Berk Ustun and Cynthia Rudin. Supersparse linear integer models for optimized medical scoring systems. Machine Learning, 102:349–391, 2016. [31] Chandan Singh, Armin Askari, Rich Caruana, and Jianfeng Gao. Augmenting interpretable models with large language models during training. Nature Communications, 14(1):7913, 2023. [32] Andreas Mueller, Julien Siems, Harsha Nori, David Salinas, Arber Zela, Rich Caruana, and Frank Hutter. Gamformer: In-context learning for generalized additive models. arXiv preprint arXiv:2410.04560, 2024. [33] Yufan Zhuang, Liyuan Liu, Chandan Singh, Jingbo Shang, and Jianfeng Gao. Learning a decision tree algorithm with transformers. arXiv preprint arXiv:2402.03774, 2024. [34] Hanyu Li, Haoyu Liu, Tingyu Zhu, Tianyu Guo, Zeyu Zheng, Xiaotie Deng, and Michael I Jordan. Ida-bench: Evaluating llms on interactive guided data analysis. arXiv preprint arXiv:2505.18223, 2025. [35] Zhangde Song, Jieyu Lu, Yuanqi Du, Botao Yu, Thomas M Pruyn, Yue Huang, Kehan Guo, Xiuzhe Luo, Yuanhao Qu, Yi Qu, et al. Evaluating large language models in scientific discovery. arXiv preprint arXiv:2512.15567, 2025. [36] Yifei Li, Hanane Nour Moussa, Ziru Chen, Shijie Chen, Botao Yu, Mingyi Xue, Benjamin Burns, Tzu-Yao Chiu, Vishal Dey, Zitong Lu, et al. Autosdt: Scaling data-driven discovery tasks toward open co-scientists. arXiv preprint arXiv:2506.08140, 2025. [37] Jaehyun Nam, Jinsung Yoon, Jiefeng Chen, and Tomas Pfister. Ds-star: Data science agent via iterative planning and verification. arXiv preprint arXiv:2509.21825, 2025.
11
[38] Joachim Baumann, Paul Röttger, Aleksandra Urman, Albert Wendsjö, Flor Miriam Plaza-del Arco, Johannes B Gruber, and Dirk Hovy. Large language model hacking: Quantifying the hidden risks of using llms for text annotation. arXiv preprint arXiv:2509.08825, 2025. [39] Yu-Min Tseng, Wei-Lin Chen, Chung-Chi Chen, and Hsin-Hsi Chen. Evaluating large language models as expert annotators. arXiv preprint arXiv:2508.07827, 2025. [40] Ruiqi Zhong, Peter Zhang, Steve Li, Jinwoo Ahn, Dan Klein, and Jacob Steinhardt. Goal driven discovery of distributional differences via language descriptions. ArXiv, abs/2302.14233, 2023. [41] Varun Babbar, Zhicheng Guo, and Cynthia Rudin. " what is different between these datasets?" a framework for explaining data distribution shifts. Journal of Machine Learning Research, 26(180):1–64, 2025. [42] Zhiying Zhu, Weixin Liang, and James Zou. Gsclip: A framework for explaining distribution shifts in natural language. arXiv preprint arXiv:2206.15007, 2022. [43] Rakesh R Menon, Kerem Zaman, and Shashank Srivastava. MaNtLE: Model-agnostic natural language explainer. arXiv preprint arXiv:2305.12995, 2023. [44] Steven Bills, Nick Cammarata, Dan Mossing, Henk Tillman, Leo Gao, Gabriel Goh, Ilya Sutskever, Jan Leike, Jeff Wu, and William Saunders. Language models can explain neurons in language models, 2023. [45] Chandan Singh, Aliyah R Hsu, Richard Antonello, Shailee Jain, Alexander G Huth, Bin Yu, and Jianfeng Gao. Explaining black box text modules in natural language with language models. arXiv preprint arXiv:2305.09863, 2023. [46] Jiaojiao Han, Wujiang Xu, Mingyu Jin, and Mengnan Du. Sage: An agentic explainer framework for interpreting sae features in language models. In Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 5: Industry Track), pages 483–495, 2026. [47] Tamar Rott Shaham, Sarah Schwettmann, Franklin Wang, Achyuta Rajaram, Evan Hernandez, Jacob Andreas, and Antonio Torralba. A multimodal automated interpretability agent. In Forty-first International Conference on Machine Learning, 2024. [48] Samuel Schmidgall, Yusheng Su, Ze Wang, Ximeng Sun, Jialian Wu, Xiaodong Yu, Jiang Liu, Michael Moor, Zicheng Liu, and Emad Barsoum. Agent laboratory: Using llm agents as research assistants. Findings of the Association for Computational Linguistics: EMNLP 2025, pages 5977–6043, 2025. [49] Kyle Swanson, Wesley Wu, Nash L Bulaong, John E Pak, and James Zou. The virtual lab of ai agents designs new sars-cov-2 nanobodies. Nature, 646(8085):716–723, 2025. [50] Zonglin Yang, Xinya Du, Junxian Li, Jie Zheng, Soujanya Poria, and Erik Cambria. Large language models for automated open-domain scientific hypotheses discovery. In Findings of the Association for Computational Linguistics: ACL 2024, pages 13545–13565, 2024. [51] Alexander Novikov, Ngân Vũ, Marvin Eisenberger, Emilien Dupont, Po-Sen Huang, Adam Zsolt Wagner, Sergey Shirobokov, Borislav Kozlovskii, Francisco JR Ruiz, Abbas Mehrabian, et al. Alphaevolve: A coding agent for scientific and algorithmic discovery. arXiv preprint arXiv:2506.13131, 2025. [52] Bernardino Romera-Paredes, Mohammadamin Barekatain, Alexander Novikov, Matej Balog, M Pawan Kumar, Emilien Dupont, Francisco JR Ruiz, Jordan S Ellenberg, Pengming Wang, Omar Fawzi, et al. Mathematical discoveries from program search with large language models. Nature, pages 1–3, 2023. [53] Chandan Singh, John X. Morris, Jyoti Aneja, Alexander M. Rush, and Jianfeng Gao. Explaining patterns in data with language models via interpretable autoprompting, 2023. [54] Yiping Wang, Shao-Rong Su, Zhiyuan Zeng, Eva Xu, Liliang Ren, Xinyu Yang, Zeyi Huang, Xuehai He, Luyao Ma, Baolin Peng, et al. Thetaevolve: Test-time learning on open problems. arXiv preprint arXiv:2511.23473, 2025. [55] Mert Yuksekgonul, Daniel Koceja, Xinhao Li, Federico Bianchi, Jed McCaleb, Xiaolong Wang, Jan Kautz, Yejin Choi, James Zou, Carlos Guestrin, et al. Learning to discover at test time. arXiv preprint arXiv:2601.16175, 2026. [56] Ziyu Ma, Shidong Yang, Yuxiang Ji, Xucong Wang, Yong Wang, Yiming Hu, Tongwen Huang, and Xiangxiang Chu. Skillclaw: Let skills evolve collectively with agentic evolver. arXiv preprint arXiv:2604.08377, 2026.
12
[57] Huichi Zhou, Siyuan Guo, Anjie Liu, Zhongwei Yu, Ziqin Gong, Bowen Zhao, Zhixun Chen, Menglong Zhang, Yihang Chen, Jinsong Li, et al. Memento-skills: Let agents design agents. arXiv preprint arXiv:2603.18743, 2026. [58] Salaheddin Alzubi, Noah Provenzano, Jaydon Bingham, Weiyuan Chen, and Tu Vu. Evoskill: Automated skill discovery for multi-agent systems. arXiv preprint arXiv:2603.02766, 2026. [59] Shuaike Shen, Wenduo Cheng, Mingqian Ma, Alistair Turcan, Martin Jinye Zhang, and Jian Ma. Skillfoundry: Building self-evolving agent skill libraries from heterogeneous scientific resources. arXiv preprint arXiv:2604.03964, 2026. [60] Yoonho Lee, Roshen Nair, Qizheng Zhang, Kangwook Lee, Omar Khattab, and Chelsea Finn. Meta-harness: End-to-end optimization of model harnesses. arXiv preprint arXiv:2603.28052, 2026. [61] Biswa Sengupta and Jinhua Wang. arXiv:2604.20938, 2026.
Harbor: Automated harness optimization.
arXiv preprint
[62] Mirac Suzgun, Mert Yuksekgonul, Federico Bianchi, Dan Jurafsky, and James Zou. Dynamic cheatsheet: Test-time learning with adaptive memory. In Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages 7080–7106, 2026. [63] Tianxin Wei, Noveen Sachdeva, Benjamin Coleman, Zhankui He, Yuanchen Bei, Xuying Ning, Mengting Ai, Yunzhe Li, Jingrui He, Ed H Chi, et al. Evo-memory: Benchmarking llm agent test-time learning with self-evolving memory. arXiv preprint arXiv:2511.20857, 2025. [64] Jenny Zhang, Shengran Hu, Cong Lu, Robert Lange, and Jeff Clune. Darwin godel machine: Open-ended evolution of self-improving agents. arXiv preprint arXiv:2505.22954, 2025. [65] Yufan Zhuang, Chandan Singh, Liyuan Liu, Yelong Shen, Dinghuai Zhang, Jingbo Shang, Jianfeng Gao, and Weizhu Chen. Test-time recursive thinking: Self-improvement without external feedback. arXiv preprint arXiv:2602.03094, 2026. [66] Fabian Pedregosa, Ga ë l Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, et al. Scikit-learn: Machine learning in python. the Journal of machine Learning research, 12(Oct):2825–2830, 2011. [67] Nick Erickson, Lennart Purucker, Andrej Tschalzev, David Holzmüller, Prateek Mutalik Desai, David Salinas, and Frank Hutter. Tabarena: A living benchmark for machine learning on tabular data. arXiv preprint arXiv:2506.16791, 2025. [68] Randal S Olson, William La Cava, Patryk Orzechowski, Ryan J Urbanowicz, and Jason H Moore. Pmlb: a large benchmark suite for machine learning evaluation and comparison. BioData mining, 10(1):36, 2017. [69] Léo Grinsztajn, Edouard Oyallon, and Gaël Varoquaux. Why do tree-based models still outperform deep learning on typical tabular data? Advances in neural information processing systems, 35:507–520, 2022. [70] Abhineet Agarwal, Yan Shuo Tan, Omer Ronen, Chandan Singh, and Bin Yu. Hierarchical shrinkage: improving the accuracy and interpretability of tree-based methods. arXiv:2202.00858, 2 2022. arXiv: 2202.00858. [71] Trevor J Hastie. Generalized additive models. Routledge, 2017. [72] Yan Shuo Tan, Chandan Singh, Keyan Nasseri, Abhineet Agarwal, and Bin Yu. Fast interpretable greedytree sums (figs). arXiv:2201.11931 [cs, stat], 1 2022. arXiv: 2201.11931. [73] J. H. Friedman and B. E. Popescu. Predictive learning via rule ensembles. The Annals of Applied Statistics, 2(3):916–954, 2008. [74] Leo Breiman. Random forests. Machine Learning, 45(1):5–32, 10 2001. [75] Noah Hollmann, Samuel Müller, Lennart Purucker, Arjun Krishnakumar, Max Körfer, Shi Bin Hoo, Robin Tibor Schirrmeister, and Frank Hutter. Accurate predictions on small data with a tabular foundation model. Nature, 637(8045):319–326, 2025. [76] Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276, 2024. [77] Alicia H Munnell, Geoffrey M.B Tootell, Lynn E Browne, and James McEneaney. Mortgage lending in boston: interpreting hmda data. The American economic review, 86(1):25 – 53, 1996.
13
[78] Margaret C. Crofoot, Ian C. Gilby, Martin C. Wikelski, and Roland W. Kays. Interaction location outweighs the competitive advantage of numerical superiority in <i>cebus capucinus</i> intergroup contests. Proceedings of the National Academy of Sciences, 105(2):577–581, 2008. [79] Tarek Naous, Philippe Laban, Wei Xu, and Jennifer Neville. Flipping the dialogue: Training and evaluating user language models. arXiv preprint arXiv:2510.06552, 2025. [80] Shirley Wu, Evelyn Choi, Arpandeep Khatua, Zhanghan Wang, Joy He-Yueya, Tharindu Cyril Weerasooriya, Wei Wei, Diyi Yang, Jure Leskovec, and James Zou. Humanlm: Simulating users with state alignment beats response imitation. arXiv preprint arXiv:2603.03303, 2026. [81] Gagan Bansal, Tongshuang Wu, Joyce Zhou, Raymond Fok, Besmira Nushi, Ece Kamar, Marco Tulio Ribeiro, and Daniel Weld. Does the whole exceed its parts? The effect of AI explanations on complementary team performance. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, pages 1–16, 2021. [82] Jean Feng, Avni Kothari, Patrick Vossler, Andrew Bishara, Lucas Zier, Newton Addo, Aaron Kornblith, Yan Shuo Tan, and Chandan Singh. Human-ai co-design for clinical prediction models. arXiv preprint arXiv:2601.09072, 2026. [83] Benjamin J Lengerich, Sebastian Bordt, Harsha Nori, Mark E Nunnally, Yin Aphinyanaphongs, Manolis Kellis, and Rich Caruana. LLMs understand glass-box models, discover surprises, and suggest repairs. arXiv preprint arXiv:2308.01157, 2023.
A
Experimental setup details
A.1
Interpretability test details
Table A1 lists all 43 development interpretability tests (the set used inside the AGENTIC - IMODELS optimization loop) with descriptions and pass rates pooled across all models (baselines + evolved candidates) from the 9 AGENTIC - IMODELS runs mentioned in Sec. 4.2. Table A2 lists the 157 held-out tests used for post-hoc evaluation, with pass rates pooled the same way. A.2
Dataset details
Table A3 lists all 65 datasets used in the main experiment, including 7 from OpenML and 58 from PMLB [68]. Each dataset is subsampled to at most 1,000 samples and 50 features during evaluation. Table A4 lists the 16 held-out OpenML datasets used in the generalization experiment (Sec. B.2).
14
Table A1: All 43 interpretability tests used inside the AGENTIC - IMODELS optimization loop, grouped by cognitive operation. Pass rates are pooled across the 9 AGENTIC - IMODELS runs mentioned in Sec. 4.2; each run contributes its full set of evaluated models (baselines + evolved candidates). Category
Test
Description
Pass %
Feature attribution
most important feature feature ranking irrelevant features sparse feature set dominant feature sample sign of effect
Identify the single most important feature Rank top 3 features by importance Identify features with negligible effect Identify active features in 10-feature data Identify dominant feature for a specific sample Determine sign and magnitude of a feature effect
97% 98% 87% 82% 100% 76%
Point simulation
point prediction counterfactual prediction all features active two feature perturbation mixed sign goes negative pairwise anti intuitive predict above threshold predict below threshold simulate mixed sign simulate all active simulatability eight features fifteen features sparse twenty features sparse twelve features all active quadratic friedman1
Predict output for a specific input Predict output for a different input Predict with all features non-zero Predict under simultaneous two-feature changes Handle opposite-signed features Compare two complex samples Predict a sample above a threshold Predict a sample below a threshold Trace 6 features with mixed-sign coefficients Simulate 5-feature linear with mixed signs Simulate complex 4-feature prediction 8-feature linear mixed-sign 15 features, only 3 active 20 features, only 4 active 12 features all contributing y = 3x20 − 2x21 + x2 Classic Friedman-1 nonlinear benchmark
90% 89% 66% 68% 41% 13% 72% 85% 53% 27% 44% 46% 23% 35% 24% 59% 63%
Sensitivity analysis
direction of change quantitative sensitivity unit sensitivity nonlinear direction threshold identification nonlinear threshold
Direction of output change when a feature increases Quantify sensitivity over larger ranges Exact unit change prediction (tight tolerance) Predict output under nonlinearity Identify threshold values separating predictions Detect hockey-stick/ReLU-like behavior
82% 88% 79% 66% 75% 52%
Counterfactual reasoning
counterfactual target quadratic counterfactual
Solve inverse problem: find input for target output Inverse problem on nonlinear function
25% 52%
Structural understanding
compactness decision region
Can the model be computed in ≤10 operations? Identify decision boundaries
81% 87%
Complex fn. simulation
simulate double threshold simulate additive nonlinear simulate interaction triple interaction cascading threshold exponential decay piecewise three segment sinusoidal abs value nested threshold
Two step-thresholds on one feature Simulate y = 3 max(0, x0 ) + 2 sin(x1 ) + x2 Simulate y = 3x0 + 2x1 + 1.5x0 x1 Multi-way interactions If-then cascading structure y = 5 exp(−x0 ) + 2x1 3-segment piecewise linear Trigonometric nonlinearity y = 3|x0 | − 2|x1 | + x2 Nested if-then logic
66% 60% 50% 79% 74% 48% 83% 46% 52% 59%
15
Table A2: All 157 new interpretability tests used in the generalization experiment, aggregate. N is the number of distinct tests in each sub-category; pass rates are pooled across the 9 AGENTIC - IMODELS runs mentioned in Sec. 4.2. Sub-category
Description
N
Pass %
top feature zero-effect feature top-2 ranking irrelevant set sign + unit effect
3 3 3 3 3
88% 7% 48% 44% 75%
dominant at sample second feature count active pair importance
Most influential feature; answer varies across tests Identify the single inert feature List the two most influential features in order Name every feature whose effect is negligible Signed change from a unit increase in a chosen feature Dominant contributing feature for a given sample Identify the second-most-important feature Count of features with non-negligible effect Between two named features, which matters more
3 3 3 2
94% 77% 48% 89%
Point simulation (26 tests)
small linear all-features active sparse many-feature alternating sign tail query irrational inputs partial input non-zero intercept compare two points
Predict output on 2–4 feature linear data Predict with every feature non-zero (7–9 feats) Predict on 10–18 feats with only 3–4 active Predict when coefficients alternate in sign Predict at feature values with |xi | ≥ 1.6 Predict at irregular, non-round query points Predict when only a subset of features is specified Predict under a shifted intercept Compare predictions at two input samples
3 3 3 3 3 3 3 3 2
29% 90% 58% 51% 60% 47% 59% 26% 50%
Sensitivity analysis (26 tests)
direction of change wide-range tight unit change crossing point two-feature change decrease phrasing multi-unit step non-zero base small step
Signed delta for a one-unit change at varied bases Quantify sensitivity over a large interval Unit change with tight numerical tolerance Find xk at which prediction crosses a target Delta when two features change simultaneously Signed delta phrased as a decrease Delta under a multi-unit change Unit change from a non-zero baseline Sub-unit step sensitivity
3 3 3 3 3 3 3 3 2
88% 62% 76% 81% 27% 33% 19% 70% 95%
Counterfactual reasoning (26 tests)
inverse on linear inverse, varied feat inverse, sparse data inverse, nonlinear with intercept reverse direction large change mid-index feature negative target
Value of xk that attains a target prediction Counterfactual targeting a rotated feature Counterfactual with many inert features Counterfactual on ReLU/quad/piecewise/abs data Counterfactual under a non-zero intercept Counterfactual requiring a downward change Counterfactual requiring a sizeable feature move Counterfactual on a feature with middle index Counterfactual to a negative target prediction
3 3 3 3 3 3 3 3 2
64% 60% 26% 63% 24% 35% 55% 54% 42%
Structural understanding (26 tests)
decision region compactness argmax argmin monotonic direction output range decision mid-output argmax nonlinear plateau region
Threshold of xk for a specified prediction level Can the model be evaluated in ≤ N operations? Value of xk that maximises prediction Value of xk that minimises prediction Monotonic direction of a chosen feature Approximate range of the model’s predictions Threshold of xk for a mid-range prediction Argmax on a nonlinear data-generating process Region where predictions are approximately constant
3 3 3 3 3 3 3 3 2
71% 67% 33% 34% 56% 45% 76% 46% 86%
Complex fn. simulation (27 tests)
quadratic pairwise interaction triple interaction Friedman-1 cascading threshold exponential decay 3-segment piecewise sinusoidal absolute value
Simulate y = c0 + c1 xk + c2 x2k Simulate linear +xi · xj interaction Simulate multi-way feature products Simulate the Friedman-1 nonlinear benchmark If-then cascading data-generating process Simulate y = c exp(−xk ) + linear Simulate 3-segment piecewise linear function Simulate trigonometric nonlinearity Simulate y = c1 |xi | + c2 |xj | + linear
3 3 3 3 3 3 3 3 3
92% 74% 93% 22% 84% 25% 91% 89% 92%
Category
Feature attribution (26 tests)
16
Table A3: The 65 datasets used in the main experiment (7 OpenML, 58 PMLB). n = samples, p = features (before subsampling). n
p
Dataset
4176 20634 8192 16599 22784 8192 15000 488 1000 1000 1000 47 50 1000000 31104 1000000 17496 116640 177147 1025010 52 8192 15000 108 40768 22784 8192 8192 55 200 209 6435 40768
8 8 21 16 16 8 26 4 10 4 4 13 4 18 9 10 9 9 10 10 2 21 48 5 10 8 8 12 2 10 6 36 10
4544_GeographicalOr.. 485_analcatdata_ve.. 503_wind 505_tecator 519_vinnie 522_pm10 523_analcatdata_ne.. 527_analcatdata_el.. 529_pollen 537_houses 542_pollution 547_no2 556_analcatdata_ap.. 557_analcatdata_ap.. 560_bodyfat 561_cpu 562_cpu_small 564_fried 573_cpu_act 574_house_16H 659_sleuth_ex1714 663_rabe_266 665_sleuth_case2002 666_rmftsa_ladata 678_visualizing_en.. 687_sleuth_ex1605 690_visualizing_ga.. 695_chatfield_4 706_sleuth_case1202 712_chscase_geyser1 banana titanic
Dataset abalone california cpu_act elevators house_16H kin8nm pol 1027_ESL 1028_SWD 1029_LEV 1030_ERA 1089_USCrime 1096_FacultySalaries 1191_BNG_pbc 1193_BNG_lowbwt 1196_BNG_pharynx 1199_BNG_echoMonths 1201_BNG_breastTumor 1203_BNG_pwLinear 1595_poker 192_vineyard 197_cpu_act 201_pol 210_cloud 215_2dplanes 218_house_8L 225_puma8NH 227_cpu_small 228_elusage 229_pwLinear 230_machine_cpu 294_satellite_image 344_mv
n
p
1059 48 6574 240 380 500 100 67 3848 20640 60 500 475 475 252 209 8192 40768 8192 22784 47 120 147 508 111 62 323 235 93 222 5300 2099
117 4 14 124 2 7 2 14 4 8 15 7 3 3 14 7 12 10 21 16 7 2 6 10 3 5 4 12 6 2 2 8
Table A4: The 16 held-out OpenML datasets (suite 335) used in the generalization experiment. n = samples, p = features (before subsampling). ID 44055 44056 44059 44061 44062 44063 44065 44066
Dataset analcatdata_sup.. visualizing_soil diamonds Mercedes_Benz_.. Brazilian_houses Bike_Sharing_D.. nyc-taxi-green-d.. house_sales
n
p
ID
4052 8641 53940 4209 10692 17379 581835 21613
7 4 9 359 11 11 16 17
44068 44069 45041 45043 45045 45046 45047 45048
17
Dataset particulate-matt.. SGEMM_GPU_kern.. topo_2_1 seattlecrime6 delays_zurich_.. Allstate_Claims.. Airlines_DepDel.. medical_charges
n
p
394299 241600 8885 52031 5465575 188318 1000000 163065
6 9 255 4 11 124 5 3
B
Extended results and ablations
B.1
Ablation: alternative LLM evaluators for the interpretability tests
GPT-5.4 evaluator used post-hoc after models have been optimized. We reevaluate the 41 baseline + evolved models from the Claude Opus-4.6 medium-effort main run under GPT-5.4 instead of GPT-4o on the 43 development-set interpretability tests. Only the LLM evaluator changes, so prediction ranks are unaffected. Fig. B1 shows two views of the same 41-model pool. Fig. B1a shows where each model lands in the predictive performance versus interpretability plot under GPT-5.4 (predictive performance is the same as is used in Fig. 3a-c). Despite the score compression for baselines (EBM and TabPFN drop to ≈ 0; GBM from 0.53 to 0.15), the evolved Pareto frontier is preserved: HingeEBM (5bag) retains a rank of 13 at 50% interpretability, well outside the baseline Pareto region. Fig. B1b plots each model’s GPT-5.4 score (x-axis) against its GPT-4o score (y-axis). The two scores are strongly correlated (r = 0.83), and within-evaluator consistency across three random seeds is high (standard deviation < 0.03 for both evaluators), so the between-evaluator divergence reflects genuine differences in evaluation criteria rather than noise. Most points fall above the diagonal, indicating GPT-5.4 is the stricter evaluator overall, with the largest drops on baselines that have complex string representations (EBM, FIGS large, PyGAM, RuleFit) and the largest evolved-model drop on HingeEBM (v1); simple decision trees (DT mini, HSTree mini) lie below the diagonal. The drops are often attributable to GPT-5.4 correctly abstaining from answering when the information is not present whereas GPT-4o would sometimes attempt to answer regardless, occasionally correctly answering some tests by chance. Claude Haiku-4.5 evaluator used post-hoc after models have been optimized. To check that the post-hoc results are not specific to a single evaluator family or to the exact prompt template used for evaluation, we additionally re-evaluate the same 41-model pool with Claude Haiku-4.5 and a slightly perturbed prompt (re-worded preamble, explicit MODEL:/QUESTION: separators, and an instruction to use only the printed model). Fig. B1c-d show the result: again, AGENTIC - IMODELS expands the Pareto frontier, and Haiku scores correlate with GPT-4o at r = 0.85. This is comparable to the GPT-5.4-vs-GPT-4o correlation, although now most models lie below the diagonal.
18
Normalized prediction rank (lower is better)
0.9
b
Agent interpretability score (GPT-4o evaluator)
Normalized prediction rank (lower is better)
0.9a
HSTree (mini) DT (mini)
0.8
HSTree (large) OLS SklearnGAM (hybrid)
0.7 0.6 PyGAM
0.5 0.4
SmartAdditive (v1)
RF
SmoothGAM (RF)
GBM
EBM
0.3
Black-box baseline Additive baseline Tree baseline HingeEBM (5bag) imodels baseline Evolved (25 from 1 run)
HingeEBM (v1) TabPFN
0.2
0.0
0.2 0.4 0.6 0.8 Agent interpretability score (GPT-5.4, 157-test suite)
d Black-box baseline Additive baseline Tree baseline imodels baseline Evolved (25 from 1 run)
0.8 0.7 0.6
DTHSTree (mini) (mini)
HSTree (large) OLS SmoothGAM (noES) SmartAdditive (v1)
RF
0.5
PyGAM
GBM
SmoothGAM (RF)
0.4 EBM
0.3 0.2
TabPFN
0.0
Agent interpretability score (GPT-4o evaluator)
c
0.8 r = 0.83 SmoothGAM (RF sqrt) OLS DT HSTree (mini) (mini) 0.7 HingeEBM (v1) 0.6 FIGS (large) 0.5 0.4 PyGAM RuleFit 0.3 0.2 0.1 Baselines EBM TabPFN Evolved 0.0 0.0 0.2 0.4 0.6 0.8 Agent interpretability score (GPT-5.4 evaluator)
HingeEBM (v1) HingeEBM (5bag)
0.2 0.4 0.6 0.8 Agent interpretability score (Claude Haiku-4.5)
1.0
0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0.0
r = 0.85
SmoothGAM (RF sqrt) OLS DTHSTree (mini) (mini) HingeEBM (v1) FIGS (large)
PyGAM
RuleFit
TabPFN
EBM
Baselines Evolved
0.0 0.2 0.4 0.6 0.8 1.0 Agent interpretability score (Claude Haiku-4.5 evaluator)
Figure B1: Post-hoc evaluator-sensitivity analysis on the held-out interpretability tests (41 baseline + evolved models from the Claude Opus-4.6 medium-effort run). Top row: GPT-5.4 evaluator (same prompt template as the in-loop GPT-4o evaluator). Bottom row: Claude Haiku-4.5 evaluator with a slightly perturbed prompt. (a, c) Normalized prediction rank on the 65 original training datasets (y-axis matching Fig. 3a–c) versus agent interpretability score under each evaluator; baselines are colored crosses by category and evolved models are blue. The evolved Pareto frontier is preserved under both evaluators. (b, d) Per-model agent interpretability score under each held-out evaluator (x-axis) versus the original GPT-4o score (y-axis); baselines are gray, evolved models are blue. Scores correlate strongly with GPT-4o under both evaluators (r = 0.83 for GPT-5.4, r = 0.85 for Haiku), but with opposite biases: GPT-5.4 is the stricter evaluator (most points above the diagonal), whereas Haiku is more lenient (most points below the diagonal). GPT-5.4 evaluator used during the optimization loop. As a stronger test of evaluator stability, we also run the full AGENTIC - IMODELS optimization loop with GPT-5.4 (rather than GPT-4o) as the in-loop interpretability evaluator. We run with Claude Code Opus-4.6, medium reasoning effort, which yields 87 evolved models. Fig. B2 shows the models (and the 16 baselines) on the same rank versus interpretability axes as Fig. B1a, with interpretability measured on the held-out tests (also graded by GPT-5.4). We find that the evolved models extend the Pareto frontier under the stricter evaluator: the best evolved models reach agent interpretability scores in the 0.71 to 0.73 range (e.g., WinsorElasticNet at normalized rank 0.09 and interp 0.73, StepByStepAdaptive at rank 0.07 and interp 0.71), above the best baseline (OLS at rank 0.09 / interp 0.69), while MegaTeacher_a06 (rank 0.06 / interp 0.68) matches or improves on every additive baseline in both dimensions. The AGENTIC - IMODELS identified with the GPT-5.4 evaluator differ from the GPT-4o-driven main runs: rather than hingebased GAMs and additive boosted stumps, the GPT-5.4 loop favors a mix of regularized linear models (WinsorElasticNet, SparseBayesRidge, BayesianRidgeLinear), per-dataset adaptive selectors (AdaptiveElastic, QuantizedAdaptive), and compact distilled trees (MegaTeacher, TinyRecalTree, ElasticCompactTree).
19
Normalized prediction rank (lower is better)
DT (mini)
Black-box baseline Additive baseline Tree baseline imodels baseline Evolved (87 from 1 run)
0.18 0.16 0.14 0.12 0.10 0.08 0.06 0.04
QuantizedAdaptive (v2) HSTree (mini) ElasticCompactTree HSTree (large) WinsorElasticNet OLS
PyGAM GBM EBM
AdaptiveElastic (v2)
TabPFN
0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 Agent interpretability score (held-out tests, GPT-5.4 evaluator)
MegaTeacher (a06)
Figure B2: In-loop GPT-5.4 optimization run: normalized prediction rank on the 65 training datasets versus agent interpretability score on the 157 held-out tests graded by GPT-5.4. The evolved models (blue) continue to improve over the Pareto frontier defined by the baselines (colored crosses). B.2
Ablation: using held-out datasets to evaluate predictive performance b Black-box Additive Tree imodels Agentic-imodels
120 100
OLS
MLP DT (large)
80
Ridge
SmartAdditive
FIGS (large) PyGAM HSTree (large)
60
RF
40
HingeGAM
GBM EBM
20 0.2
HingeEBM
0.3 0.4 0.5 0.6 0.7 0.8 0.9 Agent interpretability score (held-out tests)
Rank among matched models (held-out)
Mean prediction rank on held-out datasets
a
1.0
40
Predictive performance generalization OLS
r = 0.78
MLP
30 SmoothGAM (RF sqrt)
HSTree (large)
20 10 0
GBM HingeEBM (5bag) EBM TabPFN
Baselines Evolved
0 10 20 30 40 Rank among matched models (original)
Figure B3: (a) Interpretability versus prediction rank on 16 held-out datasets and 157 new interpretability tests for the Claude Opus-4.6 medium-effort main run (103 evolved models, blue) together with the 16 baselines. Ranks are computed globally across this 119-model pool. Evolved families such as hinge-based (HingeEBM, HingeGAM) and smooth additive (SmartAdditive) models continue to dominate the Pareto frontier; selected class labels correspond to the highlighted models in Fig. 1b. (b) Prediction ranks on the original versus held-out evaluation for the 41 matched models, re-ranked independently within each evaluation. Ranks correlate at r = 0.78; variance is higher than for the interpretability comparison (Fig. 3d) because the held-out datasets have different characteristics. B.3
Extended results on the BLADE end-to-end ADS analysis
Table B1 shows the per-dataset scores for the BLADE end-to-end evaluation.
20
Table B1: Per-dataset BLADE average scores breaking down the results in Fig. 4. Scores are averaged over the three rubric dimensions and error bars show the standard error over 9 evaluations per agent per condition. Std = standard interpretable tools, AGENTIC - IMODELS = with the agentic-imodels package. Bold indicates the higher score per agent when error bars do not overlap. Copilot (Gemini)
Copilot (Sonnet)
Claude Code
Codex CLI
Dataset
Std
AGENTIC - IMODELS
Std
AGENTIC - IMODELS
Std
AGENTIC - IMODELS
Std
AGENTIC - IMODELS
affairs amtl boxes caschools crofoot fertility fish hurricane mortgage panda_nuts reading soccer teachingratings
2.9±0.2 4.4±0.2 3.9±0.3 5.0±0.2 3.3±0.3 5.1±0.8 4.2±0.3 5.1±0.5 4.6±0.3 4.6±0.2 3.7±0.2 4.1±0.4 5.8±0.5
5.9±0.8 8.5±0.2 6.7±0.8 8.0±0.2 7.3±0.7 7.0±0.7 8.4±0.1 8.6±0.1 8.6±0.1 8.6±0.0 6.1±0.6 5.3±0.7 8.8±0.1
4.4±0.1 6.9±0.6 4.1±0.4 7.9±0.2 4.0±0.2 4.7±0.3 5.4±0.2 5.8±0.8 5.3±0.7 4.5±0.1 4.6±0.5 5.0±0.1 7.3±0.4
6.9±0.5 8.3±0.2 7.7±0.4 7.7±0.4 5.6±0.4 8.6±0.2 8.7±0.1 8.5±0.2 7.6±0.3 8.3±0.2 8.4±0.2 7.5±0.5 8.8±0.1
5.4±0.5 6.4±0.5 4.2±0.4 7.7±0.4 3.9±0.2 7.9±0.2 5.3±0.1 7.3±0.3 5.9±0.3 6.5±0.6 4.6±0.3 6.9±0.3 8.0±0.2
6.7±0.9 8.7±0.0 8.6±0.1 8.1±0.2 6.5±0.6 8.5±0.1 8.6±0.1 8.7±0.1 7.8±0.4 7.8±0.1 8.4±0.1 8.8±0.1 8.8±0.1
7.7±0.5 8.6±0.2 8.7±0.1 7.5±0.3 5.9±0.4 8.4±0.1 8.4±0.2 8.7±0.1 7.2±0.5 8.4±0.1 8.7±0.1 8.4±0.1 8.5±0.1
9.0±0.2 8.7±0.1 9.0±0.1 8.9±0.1 8.3±0.2 8.8±0.2 8.6±0.1 8.8±0.1 8.0±0.4 8.8±0.1 9.3±0.1 8.6±0.1 8.8±0.1
Average
4.36±0.09
7.52±0.06
5.37±0.09
7.90±0.09
6.16±0.09
8.15±0.11
8.09±0.05
8.73±0.05
B.4
Token consumption
Table B2 summarizes approximate token consumption per AGENTIC - IMODELS run. We split the agent column into Fresh tokens (input the model processes for the first time, plus cache-creation writes and outputs) and Cache reads (the same conversational context being re-read from the prompt cache on each follow-up turn): cache reads dominate the raw byte count of an agentic loop but are billed at roughly 10× less per token than fresh input on both Anthropic and OpenAI APIs, so the two columns are not directly comparable as “compute consumed.” Interpretability-test token counts are estimated by multiplying the number of GPT-4o calls (one row per test per candidate model) by an average of 1,050 tokens per call (prompt plus short response). Table B2: Approximate token consumption per AGENTIC - IMODELS run, broken down by what the model actually processed for the first time (Fresh) versus what was re-read from the prompt cache on follow-up turns (Cache reads). Fresh sums fresh input, cache-creation, and output tokens (Anthropic) or non-cached input plus output (OpenAI); Cache reads is the cache-replay traffic, which is billed at ≈10× less per token and so dominates the raw token totals without representing comparable compute. Interpretability-test tokens are estimated assuming ≈1,050 tokens per GPT-4o call (prompt plus response). For the GPT-5.4 evaluator run, we double the per-call estimate to ≈2,100 tokens to account for the larger reasoning traces produced by GPT-5.4. Run
Models tried
Fresh (M)
Cache reads (M)
Interp. calls
Interp. tokens (M)
Total (M)
Claude Opus-4.6 (medium) Claude Opus-4.7 (medium, run 1) Claude Opus-4.7 (medium, run 2) Claude Opus-4.7 (medium, run 3) Claude Opus-4.6 (high) Claude Opus-4.7 (xhigh) Claude Opus-4.6 (medium, GPT-5.4 evaluator) Codex GPT-5.3 (default) Codex GPT-5.3 (high) Codex GPT-5.3 (xhigh)
165 46 50 64 87 65 89 51 42 57
7.4 0.8 0.5 1.2 1.2 4.1 7.6 4.3 27.1 14.9
749.7 34.8 29.3 54.2 185.4 212.2 803.7 43.3 343.5 389.8
7,611 2,666 2,795 3,440 4,214 3,569 4,515 2,881 2,494 3,914
8.0 2.8 2.9 3.6 4.4 3.7 9.4 3.0 2.6 4.1
765.1 38.4 32.7 59.0 191.0 220.0 820.7 50.6 373.2 408.8
Total
716
69.1
2845.9
38,099
44.5
2959.5
C
Descriptions of selected evolved models
Table C1 summarizes the 10 evolved models highlighted in Fig. 1b. Below, we describe in detail the three of these model classes that exemplify the two architectural patterns identified in Sec. 4.4. All models are scikit-learn-compatible and available in the released library. To illustrate each model’s 21
Table C1: The 10 evolved models highlighted in Fig. 1b. Source indicates which agent (Claude Code or Codex) discovered the model. Rank is the normalized mean RMSE rank across the 65 development datasets in the global pool of evolved + baseline models in Fig. 1b (lower is better). Interp. is the held-out interpretability score (fraction of 157 tests passed). Model
Source
Rank ↓
Interp. ↑
HingeEBM
Claude
0.19
0.71
DistilledTreeBlendAtlas
Codex
0.25
0.71
DualPathSparseSymbolic
Codex
0.29
0.71
HybridGAM
Claude
0.29
0.68
TeacherStudentRuleSpline
Codex
0.36
0.80
SparseSignedBasisPursuit
Codex
0.48
0.76
HingeGAM
Claude
0.49
0.78
WinsorizedSparseOLS
Claude
0.58
0.73
TinyDTDepth2 SmartAdditive
Claude Claude
0.59 0.62
0.71 0.73
Description LassoCV on a hinge (piecewise-linear) basis; hidden EBM corrects residuals (Sec. C.1). Ridge student distilled from GBM/RF teachers with validation-calibrated nonnegative blending. Blended GBM/RF/Ridge teacher for batch prediction; sparse symbolic single-row equation for display. SmartAdditive display + hidden depth-5 random-forest residual, shrunk by 0.7 (Sec. C.3). GBM teacher for batch path; sparse symbolic student over linear/spline/rule/gated terms for display. Forward-selected signed basis (linear, hinge, square, interaction) with ridge refit and light coefficient rounding. Lasso on a hinge basis at 10 quantile knots, with adaptive SmartAdditive-style display. LassoCV-selected top-8 features after clipping inputs to the [p1 , p99 ] range; OLS refit. Depth-2 (4-leaf) decision tree. Greedy boosted-stump GAM with adaptive per-feature linearization (Sec. C.2).
__str__ output concretely, we fit each on the same synthetic dataset: y = 2x0 + 0.5x1 + f (x2 ) + ε, where f (x2 ) = 1.5x2 for x2 > 0 and 0.3x2 otherwise, x3 is irrelevant, and ε ∼ N (0, 0.3), with n = 300. C.1
Describing the HingeEBM_5bag model
Architecture. HingeEBM_5bag fits a two-stage model. In stage 1, for each feature j, it constructs positive hinge terms max(0, xj − t) and negative hinge terms max(0, t − xj ) at K = 2 quantile knots, yielding a piecewise-linear basis expansion. Lasso selects a sparse subset, producing the stage-1 prediction: i X Xh + − ŷ (1) = β0 + βj x j + αjk max(0, xj − tjk ) + αjk max(0, tjk − xj ) . (1) j
j,k (1)
In stage 2, if the residuals y − ŷ explain more than 10% of remaining variance, an EBM with 5 (2) outer bags and 1,000 rounds is fitted on the residuals. The final prediction is ŷ = ŷ (1) + ŷEBM , but __str__ displays only the stage-1 component. Display logic. Rather than showing the hinge equation (1) directly, __str__ collapses each feature’s hinge contributions into an effective linear slope. For a positive hinge α+ max(0, xj − t), the slope α+ is added to the coefficient of xj and −α+ t is added to the intercept. Negative hinges are handled symmetrically. The result is a compact linear equation that is easy for an LLM to simulate, though it is only exact above the highest active knot. Example output. Ridge Regression (L2 regularization, alpha=0.01043 chosen by CV): y = 1.9688*x0 + 0.5194*x1 + 1.5351*x2 + -0.1190 Coefficients: x0: 1.9688 x2: 1.5351 x1: 0.5194 intercept: -0.1190 Features with zero coefficients (excluded): x3 22
The display correctly recovers x0 and x1 but shows a single slope for x2 (1.54), which approximates the average effect of x2 ’s piecewise-linear shape. The hidden EBM captures the threshold nonlinearity. C.2
Describing the SmoothAdditiveGAM model
Architecture. SmoothAdditiveGAM (SmartAdditiveRegressor, variant smooth_gam_v1) fits a greedy additive boosted-stump model. In each of 200 rounds, it selects the feature j and threshold τ that most reduces residual SSE via a depth-1 tree, updates the per-feature shape function fj , and subtracts the fitted step. After boosting, 3 passes of Laplacian smoothing (weights 0.6/0.2/0.2) are applied to each fj . The prediction is: X ŷ = µ + fj (xj ), (2) j
where each fj is a piecewise-constant function over the thresholds chosen during boosting. Display logic. For each feature j, a linear approximation fˆj (xj ) ≈ sj xj + bj is computed on training data. If R2 > 0.90 and the feature contributes at least 1% of total importance, it is rendered as a coefficient sj ; otherwise the full piecewise table is shown. The Laplacian smoothing step is critical: it flattens irregular shape functions, increasing the fraction of features that pass the R2 threshold and thus appear as simple linear terms. (A later variant, smooth_gam_r70hyb, lowers this threshold to R2 > 0.70 and adds a hybrid predict step; the core architecture is the same.) Example output. Ridge Regression (L2 regularization, alpha=1.0000 chosen by CV): y = 1.8974*x0 + 0.3974*x1 + 0.5368 Coefficients: x0: 1.8974 x1: 0.3974 intercept: 0.5368 Nonlinear feature effects (piecewise corrections to add to above): f(x2): x2 <= -1.3747: -0.9201 -1.3747 < x2 <= -0.9901: -0.7977 -0.9901 < x2 <= -0.5740: -0.7116 -0.5740 < x2 <= -0.3322: -0.6463 ... x2 > 2.2444: +2.8050 The display correctly identifies x0 and x1 as linear (recovering coefficients close to the ground truth of 2.0 and 0.5), and shows x2 as a piecewise table that captures the slope change at 0. C.3
Describing the RidgeRFResid model
To illustrate that display–predict decoupling was discovered independently across runs, we briefly describe RidgeRFResid. Its architecture is analogous to HingeEBM but uses a different backbone and corrector: ŷ = ŷRidge (x) + λ · ŷRF (x) , (3) | {z } | {z } hidden residual
displayed
where ŷRidge is a Ridge regression model shown in __str__ and a 100-tree depth-5 RandomForest fitted on residuals contributes to predict with shrinkage λ = 0.6. This model achieved mean rank 6.6 with 67% interpretability (within two rank positions of HingeEBM_5bag) while displaying only a linear equation. The same pattern re-emerged as the HybridGAM family in subsequent Claude runs, which used a SmartAdditiveGAM display instead of Ridge.
23