One Run Is Not Enough: A Monte Carlo Protocol for Evaluating LLM Reliability on Data Science Tasks

O NE RUN I S N OT E NOUGH : A M ONTE C ARLO P ROTOCOL FOR E VALUATING LLM R ELIABILITY ON DATA S CIENCE TASKS

arXiv:2611.09729v1 [cs.LG] 6 Nov 2026

Wenqi Ni ∗ University of Edinburgh wenqi.ni@ed.ac.uk

Ozan Evkaya University of Edinburgh Ozan.Evkaya@ed.ac.uk

Miguel de Carvalho University of Edinburgh Miguel.deCarvalho@ed.ac.uk

November 9, 2026

A BSTRACT Large language models (LLMs) have shown promise for automating data science workflows. Benchmarks such as DataSciBench (2026) and StatQA (2024) have advanced the evaluation of their capabilities across data science and statistical tasks. Assessing their reliability requires examining both the quality of their responses and their consistency across repeated runs, task types, and difficulty levels. With this aim, we introduce DS-QA, a benchmark comprising 100 data science and statistics questions drawn from UK university courses, spanning 16 datasets and three difficulty levels. We develop a three-layer evaluation framework that combines general response properties (runtime, verbosity and text similarity), coarse-grained metrics (accuracy, code executability and completeness), and task-specific metrics. We evaluate OpenAI models using a Monte Carlo protocol with 100 repeated runs. The results reveal substantial variability in responses to data visualisation and supervised learning tasks, particularly at medium and hard difficulty levels, highlighting the importance of evaluating consistency alongside response quality. We also find that an increase of 0.1 in the word verbosity ratio corresponds to an increase of about 0.013 in the token verbosity ratio. Keywords Large Language Models, LLM Evaluation, Monte Carlo Study, Data Science, Benchmark

1

Introduction

The recent development of large language models (LLMs) has demonstrated impressive capabilities across a range of scientific tasks including mathematics, physics, and chemistry. Despite their successes, the effectiveness of LLMs in handling complex statistical tasks remains systematically under-explored. While ChatGPT’s Advanced Data Analysis (ADA) can often match or even outperform human analysts in basic tasks, it may struggle with contextually complex or nuanced analyses (Ping et al., 2020). LLMs are unable to make careful assumptions and claims when providing insights, unlike professional human data analysts (Bayne, 2015; Zhu et al., 2024; Jing et al., 2025). Moreover, the inherited randomness of these LLMs can add a certain level of uncertainty to the created responses even if the same prompt is asked repeatedly (Evkaya and de Carvalho, 2026). To fill this gap, our work aims to apply a Monte Carlo assessment to measure the stability of LLM-generated responses. Each question or request is tested 100 times under an identical configuration and then evaluated with the use of our proposed evaluation framework. Contributions. • A university-level data science question-answer benchmark: DS-QA includes 100 data science questionanswer pairs over 16 different datasets across 9 question dimensions and 3 difficulty levels. The questions were mainly extracted from the UK university curriculum. The question dimensions mainly include: 1. Data Understanding and Wrangling (14%), 2. Data Cleaning, Pre-processing and Transformations (5%), 3. Data Summary Statistics and Interpretations (15%), 4. Data visualisation (19%), 5. Statistical Models (44%), and ∗

Corresponding email: wenqi.ni@ed.ac.uk

A PREPRINT - N OVEMBER 9, 2026

Figure 1: The proposed DS-QA contains three key components (cf. Sections 3 and 4). (1) A university-level data science benchmark, which covers 9 task types and 3 difficulty levels. (2) A multilayer evaluation framework, incorporating 4 general properties, 3 coarse-grained metrics, and in total 35 task-specific metrics. (3). A Monte Carlo analysis is used to study the reliability of LLM (text, code and image) outputs. 6. Statistical Hypothesis Testing (3%). DS-QA enriches the task type of Statistical Models to be Linear Regression, Logistic Regression, Principal Component Analysis, and Clustering. See Figure 2 for the difficulty level distribution. • A multi-layer evaluation framework for common data science tasks: The framework consists of general properties, coarse-grained, and task-specific metrics. See Figure 3 for an overview. • A Monte Carlo assessment of response reliability: We examine the response reliability of LLMs across repeated runs per task type and per difficulty level. The key findings are: (1) LLMs show extremely high response deviations when facing Data Visualisation and Supervised Learning tasks, especially at medium and hard difficulty levels. Refer to Figure 4. (2) Word verbosity ratio has a measurable effect on token verbosity ratio: an increase of 0.1 in the word verbosity ratio brings an increase of about 0.013 in the token verbosity ratio (see Figure 6), which helps to evaluate LLM redundancy length from the perspective of computational or API costs. (3) GPT-4 O - MINI shows the highest level of text similarity with reasonable token usage, compared to GPT-4 O and GPT-4.1- NANO. (4) For GPT-4.1- MINI, a temperature larger than 1 is more likely to bring random changes to responses. These findings have direct implications for researchers deploying LLMs in data science pipelines. The benchmark and code are available at: https://github.com/niwq15/LLM-DS-Reliability.git.

2

Related Literature

LLMs on Data Science. The systematic evaluation of LLM has attracted attention due to the importance of ensuring the reliability of AI responses. Early benchmarks on mathematics such as FrontierMath Glazer et al. (2024) and IMProofBench Schmitt et al. (2025) established the foundations of LLM evaluation on expert-level mathematics, moving the focus from final-answer questions to research-level questions requiring a detailed proof. More recent work has focused on data science workflows, such as DS-1000 (Lai et al., 2022), DataSciBench (Zhang et al., 2026), and U-MATH Chernyshev et al. (2025). DS-1000 Lai et al. (2022) proposes a benchmark of 1,000 programming and coding generation problems from StackOverflow. U-MATH Chernyshev et al. (2025) introduces a benchmark of 1,100 university-level problems from current US curriculum, with 20% of problems involving visual elements. DataSciBench (Zhang et al., 2026) proposes a Task–Function–Code (TFC) framework across 6 data science task types with precisely defined metrics and programmatic rules. They also collect prompts from CodeGeeX, BigCodeBench (BCB), LLM-synthesis and human experts. However, the existing benchmarks evaluate performance mostly under single-pass prompting without taking response variability and consistency into account; and the main problem sources, except U-MATH Chernyshev et al. (2025), 2

A PREPRINT - N OVEMBER 9, 2026

Benchmark

Problem Origin

Task Coverage

MLAgentBench (Huang et al., 2023) LiveCodeBench (Jain et al., 2024)

Kaggle

5 question types: Classification, Re- Acc. + Success Rate, gression, Segmentation, Language Human Modeling, Code Speed Improvement. Focus on Competitive Programming Test Cases + Pass and Code Generation Rate

✗

Focus on Programming and Code Generation

Test Cases + Surface-Form Constraints Reproducibility

✗

Executable Code Ratio, Acc., Regression Scores Accuracy

✗

DS-1000 (Lai et al., 2022) AIRepr (Zeng et al., 2025) Text2Analysis (He et al., 2023) InfiAgentDABench (Hu et al., 2024) StatQA (Zhu et al., 2024) U-MATH (Chernyshev et al., 2025) DataSciBench (Zhang et al., 2026) DS-QA (O URS )

LeetCode & AtCoder & CodeForces StackOverflow

Evaluation Metrics

DiscoveryBench & QRData & StatQA Human&LLM

6 question types: Quantitative Reasoning, Statistical Analysis from (Zhu et al., 2024). Tabular tasks

LLM

Tabular tasks

Human

5 question types: Descriptive statistics, Accuracy and ReCorrelation Analysis, Contingency Ta- producibility ble Tests, Distribution Compliance Tests, and Variance Tests Human (Univer- 6 subjects: Algebra, Differential Cal- Accuracy sity Curriculum) culus, Integral Calculus, Multivariable Calculus, Precalculus, and Sequences and Series. CodeGeeX & 5 question types: Data Cleaning, Data 25 Aggregate MetBCB & Human Understanding, Visualisation, Predic- rics and Programtive Modelling, Data Mining and Pat- matic Rules tern Recognition, Interpretability and Report generation. Human (Univer- 16 datasets, 9 question types, 3 dif- 3-layer framework sity Curriculum) ficulty levels; new: Logistic Regres- with 42 metrics sion

Consistency Analysis

✗

✗

✗ ✗

✗

✗

✓(100 trials)

Table 1: Comparison with related work across key dimensions

are platform crawling or LLM synthesis, lacking academic stratification based on real university courses. Our DS-QA benchmark comes from a university-level statistics curriculum and covers 9 common data-science task types with three difficulty levels, filling in the above gaps. Data Science Task Automation. An influential theoretical framework that characterises the data science (DS) task categories is De Bie et al. De Bie et al. (2022). The work introduces four quadrants of DS, which are Data Engineering, Data Exploration, Model Building, and Exploitation. On this basis, subsequent work has enriched the DS task types and examined their LLM-assisted implementation, such as exploratory data analysis (e.g. StatQA Zhu et al. (2024) and DataSciBench Zhang et al. (2026)), code generation for statistical modelling (e.g. DS-1000 Lai et al. (2022) and MLAgentBench (Huang et al., 2023)). LLM tools such as OpenAI’s Advanced Data Analytics (ADA) have extended these capabilities and enabled end-to-end DS task completion. Despite this development, systematic evaluation of these tools under realistic DS conditions remains underdeveloped. Existing studies tend to assess task completion on individual problems without enough attention to explore whether performance keeps consistent across iterations or how model behaviour varies by task types and difficulty levels. The absence of a principled evaluation framework also makes it hard for practitioners to compare across benchmarks on the same task types and in the meanwhile limit actionable guidance on how prompting structuring can help to mitigate failure modes. 3

A PREPRINT - N OVEMBER 9, 2026

By adopting the four-quadrant DS taxonomy from De Bie et al. (2022) as the basis, our work fills the gaps mentioned above by providing both a structured question-answer benchmark and a three-layer evaluation framework (including general properties, coarse-grained metrics, and task-specific metrics) in which the criteria also serve as guidance for prompt design. LLM Consistency and Reliability. A growing number of studies such as PromptRobust (Zhu et al., 2023) and Sclar et al. Sclar et al. (2024) have investigated prompt robustness and showed that contemporary LLMs are very sensitive to semantically equivalent promptings with subtle changes. This raises the question of reliability of LLMs obtained from single evaluations in few-shot settings. Wang et al. Wang et al. (2023) develop a self-consistency decoding strategy to improve accuracy on arithmetic and common reasoning tasks. While it works effectively for structured reasoning, this approach is not deliberately designed for open-ended data science problems, whose outcomes may span numerical results, reasoning texts, code, and charts. (Wadi and Fredette, 2025) uses a Monte Carlo based method to explore the reliability of LLM cognitive bias in social sciences and business. Most of the existing studies (as summarized in Table 1) do not explore the distribution of LLM behaviour and responses over repeated trials. To address this gap, we adapt Monte Carlo (MC) methods from classical statistics to LLM evaluation. By repeating each prompt about 100 times under the same conditions, we obtain empirical response distributions of task completion, accuracy, and code executability, which reveal systematic instability patterns that single-pass evaluation misses entirely.

3

The DS-QA Benchmark

Figure 2: Difficulty level distribution per macro question category. The proposed DS-QA benchmark consists of 100 data science question-answer pairs. The DS-QA pairs were from different undergraduate level statistical modules at the University of Edinburgh, including Introduction to Data Science (IDS) and Statistics Year 2 (StatY2). All these questions are collected in a jsonl file format. Our curated questions extend the four data science quadrants mentioned earlier by De Bie et al. (2022) to six specific question concepts, including: • Data Understanding and Wrangling • Data Cleaning, Pre-processing and Transformations • Data Summary Statistics and Interpretations • Data visualisation - EDA • Statistical Models – Supervised Learning (Linear Regression modeling, Variations of regression (i.e. Lasso or Ridge), Classification models such as Logistic Regression model, Tree based model, Model comparison type questions, Bayesian Regression) – Unsupervised Learning (i.e. Principal Component Analysis (PCA), Clustering methods) 4

A PREPRINT - N OVEMBER 9, 2026

• Statistical hypothesis testing-related questions Beyond these macro question concepts, we further define three difficulty levels (easy, medium vs. hard), which are categorised by clarity and complexity. The specific definitions are: • A question is classified as easy if it can be quickly read from the dataset with only one or a few computational steps. • A question is classified as medium if it may require general reasoning and multiple computational steps. • A hard question requires strong logical reasoning and complicated computational steps. Overall, the created questions range on the above list and vary by easy, medium vs. hard difficulty levels. Figure 2 gives an overview of question categories and difficulty levels. Most DS-QA questions are associated with a specific dataset and the benchmark contains 16 datasets in total. Further details of task types, question IDs and their associated difficulty levels can be found in Appendix A.

4

Evaluation Framework

Figure 3: A summary of the proposed multilayer evaluation framework, including general, coarse-grained and taskspecific metrics. Regarding task-specific metrics, we introduce 5 functional task types on the basis of 4 elementary task types, which are Data Understanding, Data Summary, Data Pre-processing, and Data Visualisation. The pie chart on the right shows a summary of 5 functional task types and their task-specific metrics. Figure 3 gives a summary of our multilayer evaluation framework. General Properties. We define four general properties (Verbosity, Verbosity Ratio, Text Similarity, and Run Time) to measure LLMs. • Verbosity: General length for the number of tokens or words to count the verbosity of the generated text response in general except the coding component. • Verbosity Ratio (input / output): It is the ratio of number of input tokens (words) and number of output tokens (words). • Text Similarity: Similarity measures such as Jaccard Index for the text part of the generated response to compare with the first response as reference or ground truth for some tasks, when it is applicable. • Run Time: It measures the total runtime of the response creation per iteration. Coarse-Grained Metrics. In addition to general properties, we further define three coarse-grained metrics (Completion Ratio, Response Accuracy, and Code Executability). One of the given items below (Response Accuracy) will be combined later with the task specific cases to cover the nuances in generated responses across multiple promptings. • Completion Ratio (CR): It is equal to 0 if the executed thread is failed/not completed, or 1 if the executed thread completed with an acceptable outcome (either matching with ground truth or not). 5

A PREPRINT - N OVEMBER 9, 2026

• Response Accuracy (RA): It takes either 0 or 1 if the reported result is matching with the ground truth (GT) or not, The GT can be numeric, string, or vector. If the output is decimal value, or if the match appears partially, the output will be automatically rounded to the closest value. • Code Executability: This tests whether the generated code is directly executable in a different environment or not, either taking 2 (it is executable and has solved the problem successfully), taking 1 (it is available but not executable directly) or 0 (no code). Task-Specific Metrics. In this level, we look into 9 popular task types, which are Data Understanding, Data Summary, Data Pre-processing, Data visualisation, Linear Regression, Logistic Regression, Principal Component Analysis, Clustering, and Hypothesis testing. The set of criteria for Data visualisation is defined below. For the other task types, a varying set of criteria can be found in Appendix B. The task-specific metrics for specific questions are designed not only to focus on a single numerical answer. For the sake of simplicity, out of 5, each response can be evaluated based on the created metrics individually and repeatedly using binary marking. Data visualisation • Data visualisation completeness: It equals 1 if the requested data visualisation is generated, otherwise 0 instead. If this takes 0 there is nothing further to compute indeed. • Aesthetic mapping: Each data variable is unambiguously mapped to an appropriate aesthetic (e.g., a continuous variable to position, a categorical variable to color), and the mappings are clearly documented in a legend or caption. • Geometric objects/layers: The chosen geometric objects and functions (e.g., geom_bar() for counts, geom_point() for scatter) match the data structure and analytical intent given by the question, with no misuse or unsuitable geometric selection • Scales and coordinate system: Axes use appropriate, non-truncated scales with meaningful breaks; and any coordinate transform (e.g., log scale, coord_flip()) is applied suitably, and all components are fully readable. • Visualisation-interpretation matching: The correct use of words and abbreviations in the created plot and its associated interpretations. Final Score. To quantify the numerical differences between LLM-generated responses across different task types and difficulty levels, we use the following formula: Score = Completion Ratio + Response Accuracy + Code Executability.

(1)

Recall that both Completion Ratio and Response Accuracy metrics return binary values. We normalize the Code Executability metric to be within [0, 1] and hence a score ranges from 0 to 3. To measure the instability of LLM-generated responses for each question and each difficulty level, we take the standard deviation of the final scores into account; that is: rP 2 i (Scorei − Mean_Score) Instability(σ) = (2) N Let σ̄ ∈ [0, 1] denote the normalized standard deviation of scores. The numerical difference is considered as significant if σ̄ ≥ 0.3.

5

Results

In this study, we evaluated a subset of 41 questions that were selected to cover all task categories. All the results and analyses reported below are based on this subset. This section will look into the overall instability score distribution, a correlation heatmap of evaluation metrics, the statistical relationship between token verbosity ratios and word verbosity ratios, and initial insights with regard to different OpenAI models and temperatures. Instability Score Distribution. Figure 4 summarises the distribution of score standard deviation across 100 iterations for GPT-4 O. It is noticeable that for each task type except Model Evaluation, the instability increases with a higher difficulty level. Regarding Data 6

A PREPRINT - N OVEMBER 9, 2026

Figure 4: Instability distribution by category and difficulty level for GPT-4 O at a temperature of 1 (on the same 41-question subset with 100 runs per question). Pre-processing, GPT-4 O performs stably in easy-level tasks (0%), but its performance is highly volatile in medium tasks (66%) and hard tasks (81%). Moreover, GPT-4 O is highly unstable in medium and hard difficulty levels, especially involving the task types of Data Pre-processing, Data Visualisation, and Supervised Learning. The medium-level Data Visualisation tasks reach an instability of 0.99, while the hard-level Supervised Learning tasks even reach 1.00. These imply that repeated runs of an identical question return fully correct and failed answers almost evenly, and hence a single run on these questions is close to a coin toss. One illustrative example for Data Visualisation will be discussed later in Section 6 and more examples can be found in Appendix C. Metric Correlation. Figure 5 presents the correlation heatmap across metrics from the first two layers of our evaluation benchmark in Section 4. The evaluation metrics generally exhibit weak pairwise correlations, suggesting that they may capture complementary aspects of model performance. Verbosity shows weak positive correlations with code executability, text similarity, and runtime, with coefficients no greater than 0.30. This can be interpreted as a natural correlation: responses with a higher level of verbosity take longer runtime and possibly lead to more logical statements, which support more accurate code. Overall, the designed metrics from our evaluation benchmark are not redundant and capture multiple sources of instability. Verbosity Analysis. Previous studies on verbosity have measured verbosity by default word count. However, for complicated data science tasks, token verbosity ratio is not equivalent to word verbosity ratio, even with similar trend. Let x denote word verbosity ratios and y for token verbosity ratios. Figure 6 (left) plots token verbosity ratios against word verbosity ratios on log-log axes and we fit the model with the simulation outcomes of 45 questions with 100 iterations per question. The Spearman coefficient is 0.953 and the Pearson correlation is 0.956. The points fall close to a straight line in log–log space, indicating a power-law relationship. The fitted exponent is 1.01 with a 95% Confidence Interval [0.95, 1.07], which is statistically indistinguishable from 1. Therefore, it is consistent with the raw-scale linear fit and the fitted relationship is: Token Verbosity Ratio = −0.01 + 0.13 × Word Verbosity Ratio For the residual plot, most points (> 95%) fall within the ±2 standard deviation (SD) interval, implying that the residuals have a roughly constant spread in log space. Moreover, the results still hold if we extend the single-model setting (GPT-4 O) to a set of models (GPT-4 O,GPT-4.1- MINI,GPT-4.1- NANO). 7

A PREPRINT - N OVEMBER 9, 2026

Figure 5: Correlation heatmap for reliability indices, including both general properties and coarse-grained metrics.

Figure 6: Relationship between token verbosity ratio and word verbosity ratio for GPT-4 O at a temperature of 1. The left chart visualises the log-log model fitting and the right chart shows the residual plot with 4.5% points beyond a ±2 SD interval.

Different models. Figure 7 shows the text similarity distribution of GPT-4 O, GPT-4 O - MINI, and GPT-4.1- NANO at a temperature of 1. The model GPT-4 O - MINI has the highest average Jaccard similarity level (0.83), followed by GPT-4 O (0.72) and GPT-4.1- NANO (0.65). For this question, GPT-4 O shows the most divergent token usage distribution ranging from 480 to 1500, while the other models GPT-4 O - MINI and GPT-4.1- NANO converge to consume about 500 and 650 tokens, respectively. It is surprising that GPT-4.1- NANO with similar tokens returns responses with moderately volatile levels of text similarity, while GPT-4 O with significantly different tokens tends to produce responses with a higher level of text similarity on average. 8

A PREPRINT - N OVEMBER 9, 2026

Figure 7: Text similarity distribution by model with a temperature of 1 on id=56.

Different Temperatures. Figure 8 shows the performance of GPT-4.1- MINI at a set of temperatures (0.2, 0.7, 1.0, and 1.2). Regarding token usage distribution, the average token remains between 259 and 265 when T ≤ 1.0 and steeply increases to 295 at a temperature of 1.2. When T = 0.2, there are clearly two clusters and the model uses 180-200 tokens in many iterations. The two clusters gradually converge from T = 0.2 to T = 1.0. When T = 1.0 the token values are relatively evenly distributed within 170-400. The widest distribution occurs at T = 1.2, continuous from 140 to 630. One interpretation is that a higher temperature (T > 1) brings more possibilities of random changes. A similar phenomenon can be found in the chart of text similarity. The value of text similarity becomes larger when text responses are more similar with less random changes. When T =1.2, the widest distribution corresponds to the lowest text similarity. From T = 0.2 to T = 1.0, the values tend to converge and have a higher level of text similarity.

6

Illustrative example

Here we provide one inconsistency example from the most unstable categories in the sense of (2), i.e. Data Visualisation. Consider a hard-level example for the evals.csv dataset: "Visualize the distribution of score in the dataframe evals. Is the distribution skewed? What does that tell you about how students rate courses? Is this what you expected to see? Why, or why not?" The average score of aesthetic mapping is 87% and the main reason for losing this point is that frequency information is not used for y-axis as anticipated. Additionally, due to similar inconsistencies or incorrect usage, the average scores of geometric object and visualisation-Interpretation matching are 76% and 68%, respectively. A few unexpected results emerge within 100 iterations, including the addition of a boxplot (see Figure 9b) and the occurrence of odd gaps in the created histograms (see Figure 9c). As identified from Figure 9, the Scales & Coordinate System metric has literally had zero success because the values are somehow trimmed in the created histogram, meaning it never matches the GT image in our comparison.

7

Limitation

Our work focuses on data science tasks in English, and there is scope to further enrich the types and difficulty levels of data science tasks. Our evaluation covers 41 of the 100 benchmark questions and extending it to the full benchmark is left for future work. The experiments were run in the period from March to June 2025 on four OpenAI models (GPT-4 O,GPT-4 O - MINI,GPT-4.1- MINI,GPT-4.1- NANO), which have since been succeeded by the newer frontier models. We do not claim that the instability findings we report transfer to those models. Our protocol is designed for answering whether newer models are more stable. The central finding is that a single run and low dimensional evaluation criteria 9

A PREPRINT - N OVEMBER 9, 2026

Figure 8: Token usage distribution (top) and text similarity distribution (bottom) by temperature for GPT-4.1- MINI on id=21. are unreliable bases for evaluating LLMs, which does not depend on a particular model. The next step is to apply the proposed protocol to the latest frontier models from several providers.

8

Conclusion

In this paper, we introduce a three-layer framework for evaluating LLM performance on data science tasks across a wide range of topics and three difficulty levels. The correlation heatmap also validates that the designed metrics can capture multiple sources of instability. Furthermore, our results show that the tested LLMs often fail to produce the elements identified as required by our evaluation framework. This suggests that the framework’s criteria could benefit prompt engineering directly, for example by explicitly requesting these elements in the prompt. To study the reliability of LLMs over repeated runs, we propose a Monte Carlo assessment and our experimental results reveal substantial variability in data visualisation and hypothesis testing tasks, particularly at medium and hard difficulty levels, highlighting the importance of assessing consistency across repeated runs alongside response quality. Data Visualisation (medium-level) and Supervised Learning (hard-level) have an instability level of no less than 0.99. Moreover, we conduct a verbosity analysis and find that the token verbosity ratio increases by approximately 0.013 for every 0.1 increase in the word verbosity ratio, which can help measure LLM redundancy from the perspective of API costs. Regarding model comparison, we find that GPT-4 O - MINI has the highest level of text similarity with reasonable token usage, compared to GPT-4 O and GPT-4.1- NANO. With regard to different temperatures, the analysis of GPT-4.1- MINI responses shows that a temperature larger than 1 is more likely to bring random changes to responses. Some comments on future work are in order. First, we can further examine how sampling temperature affects both the accuracy and variability of responses. Second, it would be natural to use the proposed framework for comparing performance across a broad range of frontier LLMs. Finally, such a comparison would raise the question of which frontier model performs best and how strong the statistical evidence for its superiority is, given that it could simply have performed better by chance on the particular tasks considered. We leave such open problems for future analysis. 10

A PREPRINT - N OVEMBER 9, 2026

(a) GT Plot

(b) Output at round 22

(c) Output at round 41

(d) Output at round 51

Figure 9: An illustrative example for instability in Data Visualisation, including the Ground Truth chart and three generated images for question id=21 over the evals dataset.

9

Acknowledgements

This work was supported by the Generative AI Lab (GAIL) Seed Fund at the University of Edinburgh during the 2024-2025 round, for the period of January-June 2025.

Ethics and Privacy Statement Our work does not involve human participants and no personal data. The benchmark questions are adapted from university statistics teaching materials and all model outputs were obtained through commercial API under the providers’ terms of service. The main societal risk associated with this study is user over-reliance on answers from a Web-served LLM. Our work is intended to reduce this risk by making multi-round instability visible and measurable. Repeating each prompt 100 times consumes compute power and energy costs, which we consider justified for an evaluation study, but practitioners should consider how many repetitions are required for practical use.

References Sian Bayne. 2015. Teacherbot: interventions in automated teaching. Teaching in Higher Education 20, 4 (April 2015), 455–467. doi:10.1080/13562517.2015.1020783 Konstantin Chernyshev, Vitaliy Polshkov, Vlad Stepanov, Alex Myasnikov, Ekaterina Artemova, Alexei Miasnikov, and Sergei Tilga. 2025. U-MATH: A University-Level Benchmark for Evaluating Mathematical Skills in Large Language Models. In Proceedings of the Fourth Workshop on Generation, Evaluation and Metrics (GEM2 ). Association for Computational Linguistics, Vienna, Austria and virtual meeting, 974–1001. doi:10.18653/v1/2025.gem-1.77 Tijl De Bie, Luc De Raedt, José Hernández-Orallo, Holger H. Hoos, Padhraic Smyth, and Christopher K. I. Williams. 2022. Automating data science. Commun. ACM 65, 3 (Feb. 2022), 76–87. doi:10.1145/3495256 Ozan Evkaya and Miguel de Carvalho. 2026. Using ChatGPT for Data Science Analyses. Harvard Data Science Review 8, 1 (Jan. 2026). doi:10.1162/99608f92.c9429f07 Elliot Glazer, Ege Erdil, Tamay Besiroglu, Diego Chicharro, Evan Chen, Alex Gunning, Caroline Falkman Olsson, Jean-Stanislas Denain, Anson Ho, Emily de Oliveira Santos, Olli Järviniemi, Matthew Barnett, Robert Sandler, Matej Vrzala, Jaime Sevilla, Qiuyu Ren, Elizabeth Pratt, Lionel Levine, Grant Barkley, Natalie Stewart, Bogdan Grechuk, Tetiana Grechuk, Shreepranav Varma Enugandla, and Mark Wildon. 2024. FrontierMath: A Benchmark for Evaluating 11

A PREPRINT - N OVEMBER 9, 2026

Advanced Mathematical Reasoning in AI. arXiv preprint arXiv:2411.04872 (2024). arXiv:2411.04872 [cs.AI] doi:10.48550/arXiv.2411.04872 Xinyi He, Mengyu Zhou, Xinrun Xu, Xiaojun Ma, Rui Ding, Lun Du, Yan Gao, Ran Jia, Xu Chen, Shi Han, Zejian Yuan, and Dongmei Zhang. 2023. Text2Analysis: A Benchmark of Table Question Answering with Advanced Data Analysis and Unclear Queries. doi:10.48550/ARXIV.2312.13671 Xueyu Hu, Ziyu Zhao, Shuang Wei, Ziwei Chai, Qianli Ma, Guoyin Wang, Xuwu Wang, Jing Su, Jingjing Xu, Ming Zhu, Yao Cheng, Jianbo Yuan, Jiwei Li, Kun Kuang, Yang Yang, Hongxia Yang, and Fei Wu. 2024. InfiAgent-DABench: Evaluating Agents on Data Analysis Tasks. doi:10.48550/ARXIV.2401.05507 Qian Huang, Jian Vora, Percy Liang, and Jure Leskovec. 2023. MLAgentBench: Evaluating Language Agents on Machine Learning Experimentation. doi:10.48550/ARXIV.2310.03302 Naman Jain, King Han, Alex Gu, Wen-Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang, Armando Solar-Lezama, Koushik Sen, and Ion Stoica. 2024. LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code. doi:10.48550/ARXIV.2403.07974 Liqiang Jing, Zhehui Huang, Xiaoyang Wang, Wenlin Yao, Wenhao Yu, Kaixin Ma, Hongming Zhang, Xinya Du, and Dong Yu. 2025. DSBench: How Far Are Data Science Agents from Becoming Data Science Experts?. In The Thirteenth International Conference on Learning Representations (ICLR). Yuhang Lai, Chengxi Li, Yiming Wang, Tianyi Zhang, Ruiqi Zhong, Luke Zettlemoyer, Scott Wen-tau Yih, Daniel Fried, Sida Wang, and Tao Yu. 2022. DS-1000: A Natural and Reliable Benchmark for Data Science Code Generation. doi:10.48550/ARXIV.2211.11501 Qing Ping, Feiyang Niu, Govind Thattai, Joel Chengottusseriyil, Qiaozi Gao, Aishwarya Reganti, Prashanth Rajagopal, Gokhan Tur, Dilek Hakkani-Tur, and Prem Nataraja. 2020. Interactive Teaching for Conversational AI. doi:10. 48550/ARXIV.2012.00958 Johannes Schmitt, Gergely Bérczi, Jasper Dekoninck, Jeremy Feusi, Tim Gehrunger, et al. 2025. IMProofBench: Benchmarking AI on Research-Level Mathematical Proof Generation. arXiv preprint arXiv:2509.26076 (2025). Latest version: v2, July 2026. arXiv:2509.26076 [cs.CL] doi:10.48550/arXiv.2509.26076 Melanie Sclar, Yejin Choi, Yulia Tsvetkov, and Alane Suhr. 2024. Quantifying Language Models’ Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formatting. In ICLR. Davood Wadi and Marc Fredette. 2025. A Monte-Carlo Sampling Framework For Reliable Evaluation of Large Language Models Using Behavioral Analysis. In Findings of the Association for Computational Linguistics: EMNLP 2025, Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, and Violet Peng (Eds.). Association for Computational Linguistics, Suzhou, China, 9414–9432. doi:10.18653/v1/2025.findings-emnlp.500 Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2023. Self-Consistency Improves Chain of Thought Reasoning in Language Models. In International Conference on Learning Representations. Qiuhai Zeng, Claire Jin, Xinyue Wang, Yuhan Zheng, and Qunhua Li. 2025. Airepr: An analyst-inspector framework for evaluating reproducibility of llms in data science. In Findings of the Association for Computational Linguistics: EMNLP 2025. doi:10.18653/v1/2025.findings-emnlp.539 Dan Zhang, Sining Zhoubian, Min Cai, Fengzu Li, Lekang Yang, Wei Wang, Tianjiao Dong, Ziniu Hu, Jie Tang, and Yisong Yue. 2026. Datascibench: An llm agent benchmark for data science. In Findings of the Association for Computational Linguistics: ACL 2026. Kaijie Zhu, Jindong Wang, Jiaheng Zhou, Zichen Wang, Hao Chen, Yidong Wang, Linyi Yang, Wei Ye, Neil Zhenqiang Gong, Yue Zhang, et al. 2023. PromptRobust: Towards Evaluating the Robustness of Large Language Models on Adversarial Prompts. In Proceedings of the 1st ACM Workshop on Large AI Systems and Models with Privacy and Safety Analysis. Yizhang Zhu, Shiyi Du, Boyan Li, Yuyu Luo, and Nan Tang. 2024. Are large language models good statisticians?. In Advances in Neural Information Processing Systems (NeurIPS 2024).

12

A PREPRINT - N OVEMBER 9, 2026

A

DS-QA Data Construction

Format of Question-Answer pairs One example question is {"id": 0, "question": "How many tourist attractions are there in the data set?", "concepts": ["Data understanding"], "format": "@count_visitor_value[count_visitor_value] where \"count_visitor_value\" is given in integer format.", "file_name": "UK-visitor-numbers.csv", "level": "easy"}

and the corresponding answer is {"id": 0, "question": "How many tourist attractions are there in the data set?", "concepts": ["Data understanding"], "format": "@count_visitor_value[count_visitor_value] where \"count_visitor_value\" is given in ,→ integer format.", "file_name": "UK-visitor-numbers.csv", "level": "easy", "common_answers": [["count_visitor_value", "348"]]}

All the question-answer pairs can be found in the shared repository. See Table 2 for a summary of task types and their corresponding question IDs at different difficulty levels. Task type

Easy

Medium

Hard

Total

Data Understanding Data Summary

24.0, 44.0, 66.0 (5) 12.0, 13.0, 16.1, 18.0 (6)

(2) 17.0, 19.0, 20.0, 33.0 (4)

14 15

Data Pre-processing Data visualisation

11.0, 31.0, 41.0 (7) 10.0, 32.0, 42.0, 46.0, 54.0 (5) 45.0 (2) 14.0, 55.0 (3) 68.0 (1) –

PCA Clustering Hypothesis Testing Statistical Models

– – – 40.0, 52.0, 68.1 (4)

67.0 (1) – (2) 43.0, 51.0, 72.0, 73.0 (4)

37.0 (1) 15.0, 34.1, 35.0, 36.0, 39.1 (5) 38.1, 48.0, 48.1, 49.0 (7) 39.0, 69.0, 69.1, 69.2, 69.3 (7) (1) 67.1 (2) (2) 50.0, 61.1, 61.2, 64.0, 65.2, 67.2, 68.2 (8)

5 19

Linear Regression Logistic Regression

15.1, 16.0 (2) 14.1, 14.2, 22.0, 34.0, 47.0, 47.1, 53.0, 70.0 (11) 25.0, 38.0, 61.0, 62.0 (5) 69.4 (3)

13 10 2 2 4 16

Table 2: Summary of task types, question IDs and their associated difficulty levels.

B

Definitions of Task-Specific Metrics

Data Summary • Ratio of Accuracy: It will be automatically compared with GT values. • Data context recognition: Clearly define each variable, its units of measurement, and data source to ground the summary in context. • Appropriateness of chosen statistics: Were the most relevant statistics reported for the data type within the context of question (e.g. median for skewed data, mode for categorical)? • Accuracy of interpretation: Do the interpretations logically follow from the statistics? (e.g. not saying data is "normally distributed" based on wrong evidence) • Clarity of interpretation: Are the interpretations understandable, concise, and easy to follow, or is there any confusing word selection or terminology usage? 13

A PREPRINT - N OVEMBER 9, 2026

Linear Regression • Ratio of Accuracy: (will be calculated automatically via comparing GT value) • Correct model coefficient interpretation: Understanding the magnitude and direction of the relationship between independent and dependent variables • p-value recognition: Recognizing statistical significance on the model coefficients by reporting of p-values for each coefficient against a pre-specified α-level (commonly 0.05). • Evaluation Metric Reported and Interpreted: Assessing the model’s goodness of fit and the proportion of variance explained by the model via R2 , and/or supplemented by predictive error metrics such as RMSE or others • Interpretation of Results: Interpreting the results within the context of the specific research question and the problem domain. Logistic Regression • Ratio of Accuracy (will be calculated automatically via comparing GT value) • Correct model coefficient interpretation: Understanding the magnitude and direction of the relationship between independent and dependent variables • p-value recognition: Recognizing statistical significance on the model coefficients by reporting of p-values for each coefficient against a pre-specified α-level (commonly 0.05). • Confusion Matrix recognition: The confusion matrix details and derive class-specific error rates • Interpretation of Results: Interpreting the results within the context of the specific research question and the problem domain. Principal Component Analysis (PCA) • Suitable Variable Selection: Identifying the suitable type of variables in the data by selecting. • Number of Components Justified: Is the decision on how many PCs to keep justified correctly? • Standardization of Input Data: Was scaling recognised and applied where needed before PCA? • Correct Variance/Cum. Variance Explanation: Are explained variance / cumulative variance ratios reported and understood? • Visual evidences: Is the decision on the number of PCs supported by certain visuals like loading, biplot or scree plot. Clustering • Number of clusters justified: Is the final number of obtained clusters justified correctly? • The methods consistency: Is the requested method applied properly with the specification of certain parameters • Cluster characteristics: Are the obtained cluster characteristics such as cluster size, cluster profile mentioned clearly • Validation metrics: Are the certain report metrics like silhouette scores, Davies-Bouldin index, or other relevant statistics are recognised • Visual evidence: Is the decision on the number of clusters supported by certain visuals like scatter plots, dendrograms. Hypothesis testing • Correct Test/Test name Chosen: Recognition of the correct statistical testing / test name • Null and Alternative Hypotheses Clearly Stated: Clearly and correctly stated hypothesis test components with correct notation • Accuracy on test statistics/p-value and/or Critical region Having the correct final decision based on the testing procedure • Correct Interpretation of test statistics/p-value and/or Critical region Having the correct final decision based on the testing procedure • Contextual Conclusion: Interpreting the final decision within the context of the specific research question 14

A PREPRINT - N OVEMBER 9, 2026

C

Additional Illustrative Examples

Hypothesis Testing. Consider a medium-level example with the aeroplane.txt dataset: "For now consider only the data stored in the vector first. We assume that the data are independent observations from N (µ, σ 2 ) and wish to conduct the following hypothesis test: H0 : µ equals 10 vs H1 : µ not equal 10. Define a suitable test statistic and state its distribution, assuming that the null hypothesis is true. Calculate the associated observed test statistic. Conduct the hypothesis test and clearly state your conclusions." "Correct Test/Test name Chosen" has 100% success, whereas there is less consistency on the items such as "H0 and H1 are clearly stated", "Accuracy on Test Stat / p-value / Critical Reg" and "Contextual Conclusion" having values of 73%, 72% and 76%, respectively. Even if for some cases the test numerical results are not matching with the GT, the final interpretation seems reasonable so we had "Correct Interpretation of Test Stat / p-value / Critical Reg" as 82% of the time over 100 times repeated prompting. There were some cases that the solution is incomplete (asking the user to continue), no specific numerical outcome or using the sample mean in the interpretation rather than the population mean, as one of the challenging misuse for learners in general. Linear Regression. Consider another hard-level example from evals.csv, with an overall performance of 4.34 out of 5 scores: "Fit a linear model called score_bty_fit to predict average professor evaluation score from average beauty rating (bty_avg).Print the regression output using tidy(). Based on the regression output, write down the linear model." Our results show that GPT models achieve large success over the sub-items of Accuracy on results - 92%, Correct model coefficient interpretation - 99%, Evaluation Metric reporting - 99% and Interpretation of Results - 93% beyond looking at accuracy only. However, the recognition of the p-value is missing in 51% of 100 cases. PCA One example with an overall performance of 2.92/5 is from the duke_forest.xlsx dataset and at the hard level: "Run PCA for this data, with all suitable variables. Discuss whether it is appropriate to run scaled or unscaled PCA for this data." The LLM-generated response rarely includes the justification of the selected number of components or the numerical explanation of the explained variance or cumulative variance. Even if the code structure seems following the correct recipe mostly, the generated text response is mostly limited. To illustrate, it is observed that "No of components justified" - 16% and "Correct Variance/ Cum Var explanation" - 26% lower values compared to other items such as detecting the reasonable subset of numerical variables to apply PCA ("Suitable Variable Selection" - 98%), necessary scaling for the variables have different ranges ("Scaling Recognition" - 99%) and even the specific supportive visualization in half of the cases ("Visual Evidence" - 53%).

15

Related documents

• GardenBench: A lightweight, daily evaluation of LLM capabilities#1388433
• Well-Specified and Well-Tested? Assessing SWE-bench-Style Datasets with a Self-Evolving Agent#1490812
• Well-Specified and Well-Tested? Assessing SWE-bench-Style Datasets with a Self-Evolving Agent#1483072
• DSEffi-Bench: Demystifying Large Language Models' Capability in Efficient Data Science Code Generation#799464
• DSEffi-Bench: Demystifying Large Language Models' Capability in Efficient Data Science Code Generation#798041