arXiv:2605.12376v1 [cs.AI] 12 May 2026
ProfiliTable: Profiling-Driven Tabular Data Processing via Agentic Workflows WEI LIU, Peking University, China YANG GU, Peking University, China XI YAN, Institute of Computing Technology, Chinese Academy of Sciences, China ZIHAN NAN, Peking University, China BEICHENG XU, Peking University, China KEYAO DING, Peking University, China BIN CUI, Peking University, China WENTAO ZHANG, Peking University, China Table processing—including cleaning, transformation, augmentation, and matching—is a foundational yet error-prone stage in real-world data pipelines. While recent LLM-based approaches show promise for automating such tasks, they often struggle in practice due to ambiguous instructions, complex task structures, and the lack of structured feedback, resulting in syntactically correct but semantically flawed code. To address these challenges, we propose ProfiliTable, an autonomous multi-agent framework centered on dynamic profiling, which constructs and iteratively refines a unified execution context through interactive exploration, knowledge-augmented synthesis, and feedback-driven refinement. ProfiliTable integrates (i) a Profiler that performs ReAct-style data exploration to build semantic understanding, (ii) a Generator that retrieves curated operators to synthesize task-aware code, and (iii) an Evaluator–Summarizer loop that injects execution scores and diagnostic insights to enable closed-loop refinement. Extensive experiments on a diverse benchmark covering 18 tabular task types demonstrate that ProfiliTable consistently outperforms strong baselines, particularly in complex multi-step scenarios. These results highlight the critical role of dynamic profiling in reliably translating ambiguous user intents into robust and governance-compliant table transformations. CCS Concepts: • Information systems → Information integration; • Computing methodologies → Artificial intelligence. Additional Key Words and Phrases: Table Processing, Large Language Models, Autonomous Agents, Dynamic Profiling
1
Introduction
Tabular data constitutes the backbone of modern data-driven decision making and underpins a wide range of machine learning applications, especially in high-stakes domains such as finance [31], healthcare [23], and government [24]. However, real-world tabular data is rarely analysis-ready: it is often noisy, incomplete, and semantically inconsistent, requiring extensive processing before reliable modeling can begin [18]. Moreover, the diversity of table structures and domain semantics precludes the existence of a single, fixed algorithm that can handle all data wrangling scenarios [21]. As a result, rule-based table processing tools such as OpenRefine [19] demand substantial manual configuration and expert intervention, making tabular data processing one of the most labor-intensive and error-prone stages in the data science pipeline [25, 28, 29]. These limitations have motivated growing interest in more flexible and automated solutions that can adapt to diverse tabular structures and task requirements with minimal manual intervention. The advent of large language models (LLMs) has opened new avenues for automating data-centric tasks [20]. Recent works have demonstrated LLMs’ ability to generate executable code for table manipulation from natural language, enabling Data Science Agents that produce Authors’ Contact Information: Wei Liu, Peking University, China; Yang Gu, Peking University, China; Xi Yan, Institute of Computing Technology, Chinese Academy of Sciences, China; Zihan Nan, Peking University, China; Beicheng Xu, Peking University, China; Keyao Ding, Peking University, China; Bin Cui, Peking University, China; Wentao Zhang, Peking University, China. Paper under review
1
2
Wei Liu, Yang Gu, Xi Yan, Zihan Nan, Beicheng Xu, Keyao Ding, Bin Cui, and Wentao Zhang
pandas or SQL scripts [16, 35, 43]. However, existing approaches fall short in delivering robust, general-purpose table curation for three key reasons: First, many methods are narrowly scoped, focusing on isolated operations such as imputation [3, 6, 32], error correction [1, 2, 22], or schema matching [4], and thus cannot handle composite workflows [15]. Second, broader frameworks often prioritize other data science tasks—such as automated machine learning (AutoML) [36, 45] or table-based question answering [33, 39]—over the full spectrum of table processing. Third, even those systems explicitly targeting table processing either rely on generic multi-agent scaffolds, which lack table-specific data understanding [8, 27], or employ hand-crafted pipelines that do not emphasize adaptive, feedback-driven profiling [17, 26]. This limitation is evident in current tools. For example, CleanAgent performs column-type annotation but does not actively sample or inspect the actual cell values during code generation [26]. As a result, when given an ambiguous instruction such as “standardize the currency column,” it has no knowledge of the concrete values present in the column and therefore cannot determine how to construct an appropriate mapping table for standardization. In contrast, a rule-based system like DataGovAgent typically samples only a few rows per column to estimate basic properties [17]. However, this limited inspection often fails to capture the full range of values needed to interpret ambiguous instructions—especially when the relevant column cannot be identified without examining its actual content. To handle such cases reliably, it would need to compute full value statistics such as unique values for that column. But since the target column is unknown in advance, the only safe option is to compute unique values for all columns. This leads to significant redundancy, especially in wide tables, and floods the prompt with irrelevant information. The root issue is that static, rule-driven profiling cannot decide adaptively which column to explore based on the instruction, and therefore must over-provision to avoid failure. These shortcomings highlight why table processing demands a flexible and hypothesis-driven profiling mechanism that explores only what is necessary and does so at the right time. Therefore, we propose ProfiliTable, whose core insight is that robust table processing depends on an active, iterative notion of profiling—not as passive metadata consumption, but as a dynamic process of semantic construction that evolves through interaction, feedback, and knowledge integration. Our profiling framework operates through three synergistic mechanisms: (1) Interactive exploration: The agent actively interrogates tables via a ReAct loop [41], executing lightweight actions (e.g., sampling) to formulate and test hypotheses about data quality and structure, going beyond the fixed, one-size-fits-all rules used in prior works [17]; (2) Knowledge augmented synthesis: For complex operations, the agent decomposes tasks into subgoals and retrieves relevant, pre-validated operator templates from a curated library via Retrieval Augmented Generation (RAG) [5], ensuring that generated code is based on reliable, domain-specific primitives. (3) Feedback-driven refinement: After each code generation attempt, an evaluator computes a task-specific score using ground-truth-aligned logic. A dedicated Summarizer agent then performs a ReAct-style loop over the processed table—formulating verification hypotheses, executing lightweight validation actions, and observing outcomes—to assess alignment with the original intent. The resulting insight is fed to the Profiler and Generator, guiding more accurate profiling and code generation. To evaluate table processing tasks, existing benchmarks for LLM-based table processing often misalign with realworld needs: they either focus on end-to-end analytics or isolated code correctness [34, 42], or lack support for multi-step table transformation workflows and semantics required in real production systems [11, 13, 38]. DataGovBench [17] introduces a hierarchical evaluation with rigorous metrics, but undercovers classical processing operations. We extend it into a more comprehensive benchmark with fine-grained categories—covering cleaning, transformation, augmentation, and matching—to better reflect real-world table curation challenges. For performance, ProfiliTable consistently achieves state-of-the-art performance across both single-step and multi-step tasks under GPT-4o and GPT-5.2, outperforming all Paper under review
ProfiliTable: Profiling-Driven Tabular Data Processing via Agentic Workflows
3
baselines in correctness, completeness, and execution reliability. Notably, it achieves a 100% task-wise runnable rate—a key requirement for production deployment. Our main contributions can be summarized as follows: • We propose ProfiliTable, an autonomous multi-agent framework that unifies interactive exploration, feedback-driven refinement, and knowledge-augmented synthesis under a dynamic profiling paradigm to overcome the brittleness of one-shot LLM code generation. • We construct a benchmark comprising 18 distinct types of table processing tasks—spanning cleaning, transformation, augmentation, and matching. • Extensive experiments demonstrate ProfiliTable’s strong performance across our benchmark, achieving state-of-theart results in both single-step and complex multi-step settings, with consistent gains in correctness, completeness, and execution reliability over strong baselines. 2
Related work
Table-oriented Agents. Large language models have evolved into autonomous agents specialized for tabular data tasks. Recent systems focus explicitly on understanding, cleaning, and transforming tables: Data Interpreter [7] enables end-to-end analysis of structured data through code generation; DataGovAgent [17] enforces data governance via contract-guided planning; and CleanAgent [26] automates standardization of messy tables using declarative APIs. Complementing these, LLM-TabAD [12] treats anomaly detection as a zero-shot batch-level reasoning task, leveraging LLMs’ ability to identify low-density regions in numerical tables without model retraining. These approaches collectively demonstrate a shift toward agents that treat tables as first-class citizens—reasoning over their schema, semantics, quality, and statistical structure. Multi-Agent LLM Frameworks. Beyond single-agent architectures, recent works explore collaborative multi-agent systems where specialized agents interact to solve complex tasks. Minimum-function agents—such as those in CAMEL [14] and ChatDev [27]—assign narrow roles (e.g., coder, tester) to reduce cognitive load, while client-server frameworks employ a central controller to plan workflows and delegate subtasks to client agents, improving modularity and robustness [8, 40, 45]. More adaptive approaches include dynamic agent systems that create or reconfigure agents at runtime: hierarchical generation spawns child agents from a parent controller for structured decomposition [10], while iterative feedback-driven methods (e.g., EvoMAC [9]) refine agent behavior through cross-agent error signals. Although dynamic designs offer flexibility, they often sacrifice stability and reproducibility. Our work uses role-specialized agents such as Interpreter, Profiler, Generator, and Summarizer. These agents operate within a closed-loop, feedback-driven pipeline that balances structured collaboration and adaptive refinement for table processing. 3
ProfiliTable
3.1
Problem Formulation
We study the problem of automated table processing: given a natural language instruction 𝑖 ∈ 𝐼 (e.g., “impute missing values”) and raw input tables Draw , the goal is to produce a processed table 𝑇 that correctly fulfills the intent while satisfying syntactic and semantic validity as well as domain-specific constraints. We aim to build an agentic workflow that, given a natural language instruction and a raw table, automatically produces a correctly processed output table. The workflow should maximize performance across real-world table processing tasks by generating outputs that closely match expert-validated ground-truth results. Paper under review
4
Wei Liu, Yang Gu, Xi Yan, Zihan Nan, Beicheng Xu, Keyao Ding, Bin Cui, and Wentao Zhang In practice, real-world tables are often too large to be loaded entirely into memory. The workflow therefore does
not receive direct access to Draw ; instead, it is only provided with file paths and must interact with the data through a controlled programming interface—such as sampling rows. Consequently, the workflow generates executable code 𝑐 such that 𝑇 = exec(𝑐, Draw ), rather than producing the output table 𝑇 directly. However, due to the ambiguity of natural language instructions and the complexity of tabular semantics, a one-shot code generation strategy often fails to yield correct or robust results. To address this, we model the workflow as an iterative refinement system that engages in closed-loop interaction with the execution environment, progressively improving its solution through feedback. Let 𝑡 = 1, 2, . . . ,𝑇max denote the iteration index. At each step 𝑡, the workflow produces a candidate program 𝑐 (𝑡 ) based on the original instruction 𝐼 , the raw data Draw , local memory M (𝑡 ) , and a unified dynamic profiling context P (𝑡 ) , defined as: P (𝑡 ) =
𝑝 (𝑡 ) |{z}
,
R (𝑡 ) |{z}
,
F (𝑡 ) |{z}
,
(1)
current profiling retrieved operators feedback history
where: • 𝑝 (𝑡 ) is the active profiling summary generated by the Profiler through ReAct-style exploration of Draw ; • R (𝑡 ) denotes operator templates retrieved from a curated knowledge base via RAG; • F (𝑡 ) = {𝑓 (1) , . . . , 𝑓 (𝑡 −1) } is the feedback history from previous iterations, with each feedback signal 𝑓 (𝑖 ) = 𝑠 (𝑖 ) , 𝑒 (𝑖 ) , 𝜏 (𝑖 ) , 𝑝 (𝑖 ) (2) comprising: (i) execution score 𝑠 (𝑖 ) = Eval exec(𝑐 (𝑖 ) , Draw ) , (ii) error trace 𝑒 (𝑖 ) (if any), (iii) diagnostic insight 𝜏 (𝑖 ) from the Summarizer’s ReAct-based validation, and (iv) the profiling summary 𝑝 (𝑖 ) used at step 𝑖. The dynamic profiling context P (𝑡 ) evolves iteratively, as illustrated in Figure 1, becoming increasingly informative and better aligned with the user’s task-specific intent. The code generation policy is thus profiling-aware: 𝑐 (𝑡 ) = 𝑔 𝐼, Draw, P (𝑡 ) , M (𝑡 ) ,
(3)
where the function 𝑔 is implemented by a multi-agent collaboration pipeline. The overall self-improving workflow is depicted in figure 2, highlighting the closed-loop refinement loop and the roles of each agent across iterations 𝑡 = 1, 2, . . . ,𝑇max . 3.2
Autonomous Multi-Agent Framework
As illustrated in Figure 2 and algorithm 1, our table processing workflow implements a closed-loop, feedback-driven framework through several specialized LLM-powered agents: Interpreter, Profiler, Decompositer, Generator, Evaluator, Summarizer and Finalizer. This architecture enables robust, interpretable, and iterative transformation of raw tables into governance-compliant outputs. Interpreter Agent. The Interpreter is the first component in the workflow. It clarifies user’s intent and identifies the task type and complexity—determining whether the request is single-step or multi-step. This decision directly controls operator retrieval: for single-step tasks, it selects operators that match the task type; for multi-step tasks, it routes the request to the Decompositer, which breaks it into subtasks, each of which then retrieves its own matching operators. Paper under review
ProfiliTable: Profiling-Driven Tabular Data Processing via Agentic Workflows
5
Algorithm 1 ProfiliTable: Dynamic Profiling Workflow 1: Parse instruction 𝑖 with Interpreter to get task complexity. 2: Initialize P (0) ← ( ∅, ∅, ∅ ), F (1) ← ∅, M (0) ← ∅, best_score ← 0. 3: for 𝑡 = 1 to 𝑇max do 4: 𝑝 (𝑡 ) ← Profiler(Draw , P (𝑡 −1) ) ⊲ explore raw table(ReAct) 5: if task is multi-step then 6: {𝑖𝑚 } ← Decompositer(𝑖) ⊲ decompose task Ð 7: R (𝑡 ) ← 𝑚 top-𝑘 operators for 𝑖𝑚 8: else 9: R (𝑡 ) ← top-𝑘 operators for 𝑖 ⊲ retrieve operators 10: end if 11: P (𝑡 ) ← (𝑝 (𝑡 ) , R (𝑡 ) , F (𝑡 ) ) ⊲ build profiling context 12: 𝑐 (𝑡 ) ← Generator(𝑖, Draw , P (𝑡 ) , M (𝑡 ) ) ⊲ generate code 13: M (𝑡 +1) ← UpdateMemory(M (𝑡 ) , 𝑐 (𝑡 ) ) 14: 𝑇 (𝑡 ) ← exec(𝑐 (𝑡 ) , Draw ) ⊲ execute code 15: 𝑠 (𝑡 ) ← Evaluator(𝑇 (𝑡 ) , 𝑖) ⊲ evaluate output 16: 𝑒 (𝑡 ) ← GetErrorTrace(𝑐 (𝑡 ) , 𝑇 (𝑡 ) ) 17: 𝜏 (𝑡 ) ← Summarizer(𝑇 (𝑡 ) , 𝑖, 𝑠 (𝑡 ) , 𝑒 (𝑡 ) ) ⊲ validate result(ReAct) 18: 𝑓 (𝑡 ) ← (𝑠 (𝑡 ) , 𝑒 (𝑡 ) , 𝜏 (𝑡 ) , 𝑝 (𝑡 ) ) 19: F (𝑡 +1) ← F (𝑡 ) ∪ { 𝑓 (𝑡 ) } ⊲ update feedback 20: if 𝑠 (𝑡 ) > best_score then 21: best_score ← 𝑠 (𝑡 ) , best_code ← 𝑐 (𝑡 ) 22: end if 23: if 𝑠 (𝑡 ) ≥ 𝜃 then ⊲ The score surpasses the threshold 24: return 𝑇 (𝑡 ) ⊲ return current output 25: end if 26: end for 27: 𝑇 ∗ ← exec(best_code, Draw ) ⊲ return best 28: return 𝑇 ∗
Fig. 1. Profiling reveals ambiguous instructions and recovers concrete currency symbols for accurate ISO 4217 mapping.
By accurately recognizing user intent, the Interpreter ensures that downstream modules act with the right focus and level of detail. Paper under review
6
Wei Liu, Yang Gu, Xi Yan, Zihan Nan, Beicheng Xu, Keyao Ding, Bin Cui, and Wentao Zhang
Fig. 2. The autonomous workflow of ProfiliTable: a self-improving, closed-loop pipeline centered around a dynamic profiling context. An Interpreter parses user intent; a Profiler conducts ReAct-style data exploration to initialize the context; a Decompositer decomposes multi-step tasks; a Generator synthesizes code by grounding in retrieved operators and the current profiling summary; an Evaluator and Summarizer jointly enrich the context with execution feedback and diagnostic insights; and a Finalizer selects the best candidate upon convergence—enabling robust and governance-compliant table processing.
Profiler Agent. The Profiler acts as the semantic grounding module, actively interrogating input tables to build a contextual understanding of their structure and semantics. Given a natural language instruction 𝑖 ∈ 𝐼 and raw tables Draw , it runs a ReAct-style exploration loop [41]. As illustrated in Figure 1, it recognizes that resolving ambiguities—such as “standardize the currency column”—requires inspecting actual values (e.g., unique values) rather than relying on schema metadata alone. The output is a profiling summary 𝑝 (𝑡 ) , which encodes distributional properties, data quality issues, and key observations, and is passed to downstream agents to ensure data-aware code generation and bridge linguistic intent with tabular reality. Decompositer Agent. The Decompositer is activated only for multi-step tasks identified by the Interpreter. It decomposes a complex instruction into an ordered sequence of atomic subtasks, each formalized as a (task type, description) tuple. This structured decomposition enables targeted operator retrieval for each subtask by aligning the task type and semantic description with entries in the operator library. Moreover, it provides downstream agents with granular, well-scoped objectives, enabling focused exploration, precise code generation, and step-wise validation. By decomposing composite requests into executable units, the Decompositer ensures reliable and coherent multi-step workflows. Generator Agent. The Generator translates the user’s intent 𝐼 and the unified dynamic profiling context P (𝑡 ) into an executable program 𝑐 (𝑡 ) . It employs a task-aware retrieval-augmented strategy that adapts to the structural complexity of 𝐼 . Paper under review
ProfiliTable: Profiling-Driven Tabular Data Processing via Agentic Workflows
7
Let O = {𝑜 1, . . . , 𝑜 𝑀 } denote a curated library of pre-validated operator templates, each associated with a natural language description 𝑑 (𝑜𝑖 ). Given a similarity function sim(·, ·) (cosine similarity over embedding vectors), retrieval proceeds as follows: • For single-step tasks, the Generator treats 𝑖 as an atomic query and retrieves operators satisfying: R single = 𝑜𝑖 ∈ O sim 𝑑 (𝑜𝑖 ), 𝑖 ≥ 𝜃 sim ,
(4)
then selects the top-k most relevant ones: R (𝑡 ) = top-𝑘 R single .
(5)
• For multi-step tasks, the Decompositer first decomposes 𝑖 into subtasks {𝑖 1, . . . , 𝑖 𝐾 }. For each 𝑖𝑚 , it retrieves: R𝑚 = top-𝑘 𝑜𝑖 ∈ O sim 𝑑 (𝑜𝑖 ), 𝑖𝑚 ≥ 𝜃 sim , (6) and forms the overall retrieval set as: R (𝑡 ) =
𝐾 Ø
R𝑚 .
(7)
𝑚=1
The retrieved operators in R (𝑡 ) are injected as in-context exemplars. Finally, The Generator then synthesizes the candidate program and get final code as shown in equation (3). Evaluator Agent. The Evaluator provides rigorous, task-specific validation by executing 𝑐 (𝑡 ) in a sandboxed environment and computing a feedback signal: 𝑠 (𝑡 ) = Eval exec(𝑐 (𝑡 ) , Draw ) .
(8)
where 𝑠 (𝑡 ) ∈ [0, 1] is the evaluation score. Critically, Eval(·) is not a generic metric but a task-customized script that compares the output against ground-truth expectations using precise logic like F1 score. This ensures that “runnable” does not imply “correct”—only solutions that satisfy business objectives receive high scores. Summarizer Agent. The Summarizer operates in a ReAct-style loop analogous to the Profiler, but with a distinct objective: rather than exploring raw data, it interacts directly with the processed table 𝑇 (𝑡 ) = exec(𝑐 (𝑡 ) , Draw ) to assess whether the current output satisfies the original task objective 𝑖. At each turn, the Summarizer formulates verification hypotheses (e.g., “Has all missing ‘income’ been imputed?”), executes lightweight validation actions over 𝑇 (𝑡 ) , and observes outcomes to determine task completion status. This interaction yields a structured assessment of alignment between the current output and the ground-truth expectations encoded in 𝑖. Based on this assessment and the feedback history F (𝑡 ) , the Summarizer synthesizes a concise insight that explicitly identifies any unfulfilled subgoals and provides targeted revision guidance. This insight is then propagated back to both the Profiler and Generator, enabling the Profiler to refine its data understanding in subsequent iterations and guiding the Generator to adjust its code synthesis strategy. Through this reflection, the workflow maintains a coherent reasoning chain from intent to validation, grounding each refinement in both current output observations and past attempts. Finalizer Agent. The Finalizer acts as the termination and selection module governing convergence of the iterative refinement loop. At each invocation, it evaluates the current state against two criteria: (i) whether the evaluation score 𝑠 (𝑡 ) = Eval(𝑇 (𝑡 ) ) meets or exceeds a pre-specified success threshold 𝜃 , and (ii) whether the number of generation attempts has reached the maximum allowed retries 𝑇max . If 𝑠 (𝑡 ) ≥ 𝜃 , the Finalizer halts the workflow and commits 𝑇 (𝑡 ) Paper under review
8
Wei Liu, Yang Gu, Xi Yan, Zihan Nan, Beicheng Xu, Keyao Ding, Bin Cui, and Wentao Zhang
as the solution. If 𝑠 (𝑡 ) < 𝜃 but retries remain, it returns control to the Profiler for another refinement cycle. Critically, if the retry limit is reached without achieving 𝜃 , the Finalizer does not return the latest attempt; instead, it selects the ∗
∗
best-performing candidate across all iterations—i.e., the code 𝑐 (𝑡 ) and corresponding table 𝑇 (𝑡 ) with the highest ∗
score 𝑠 (𝑡 ) = max𝑖 ≤𝑡 𝑠 (𝑖 ) . This ensures that even in failure-to-converge cases, the workflow delivers the most accurate result observed during execution. Together, these agents form a self-improving pipeline that mirrors human data wrangling: explore → hypothesize → implement → validate → reflect → refine. At the heart of this loop is a unified dynamic profiling context, which continuously integrates real-time data exploration, retrieved operator knowledge, and multi-round feedback into a coherent semantic representation. By grounding every generation and validation step in this evolving context, our workflow achieves high-fidelity table processing while maintaining full interpretability and auditability. 4
Experiment
4.1
Experiment Settings
4.1.1 Benchmark. Our experiments are conducted on a comprehensive benchmark of various table processing tasks, carefully designed to reflect the diverse challenges encountered in practical data curation workflows. Each task is specified by a natural language instruction, accompanied by one or more raw input files (in CSV or JSONL format), a ground-truth output file, and a dedicated evaluation script (‘eval.py’) that computes a normalized score in [0, 1] by comparing the agent’s output against the expected result. The benchmark encompasses 18 fine-grained task categories organized into four core dimensions: Table Cleaning (error correction, data imputation), Table Transformation (formatting, standardization, normalization, mapping, aggregation, concatenation, splitting, pivoting, column/row swapping, filtering, grouping, sorting), Table Augmentation (row population, schema evolution), and Table Matching (schema alignment, entity resolution). Tasks are categorized as either single-step, which can be resolved by a single atomic operation or multi-step, which require the orchestration of multiple operations into a coherent pipeline. 4.1.2 Metric. We evaluate table processing agents using two complementary categories of metrics: performance metrics, which assess correctness, robustness, and solution quality, and efficiency metrics, which quantify computational cost and latency. All performance metrics are scaled by a factor of 100 (i.e., reported as percentages) for readability. The Average Task Score (ATS) computes the mean of normalized task-specific scores across all evaluated instances, where each score reflects the alignment between the generated output and ground-truth expectations. To assess perfect execution, we report the Task Success Rate (TSR), defined as the fraction of tasks achieving a full score of 1.0. Recognizing that many real-world tasks admit partial solutions, we further introduce the Partial Success Rate (PSR)—the proportion of tasks yielding a strictly positive score—as a measure of meaningful progress even in the absence of complete correctness. On the code reliability front, the Code Runnable Rate (CRR) quantifies the ratio of successfully executed code snippets to the total number of generated scripts, capturing syntactic validity and runtime stability. Complementing this, the Task-wise Runnable Rate (TRR) measures the fraction of tasks for which at least one attempt produces a runnable script, reflecting per-task resilience against generation failures. To summarize overall effectiveness, we further report the average metric score (Avg. Score), defined as the arithmetic mean of ATS, TSR, PSR, CRR, and TRR for each method. As for the Efficiency metrics, we report average tokens(Avg. Tokens) and average time(Avg. Time), which denote the average number of tokens consumed by agents and the average wall-clock time (in seconds) spent per task, respectively. Together, they provide a holistic view of agent performance, jointly reflecting Paper under review
ProfiliTable: Profiling-Driven Tabular Data Processing via Agentic Workflows
9
computational cost, latency, and practical usability in complex table processing workflows. See Appendix B.1 for a summary. 4.1.3 Baselines. We compare ProfiliTable against several representative LLM-based agents that exemplify current trends in table-oriented tasks and multi-agent collaboration. DataGovAgent [17] operates on the DataGovBench benchmark with contract-guided planning and retrieval-augmented generation to ensure reliable execution of data governance tasks. CleanAgent [26] focuses specifically on cleaning messy tables by applying declarative transformation rules derived from natural language instructions. On the multi-agent front, CAMEL [14] and ChatDev [27] simulate role-playing teams that coordinate via natural language dialogue to decompose and solve software development tasks, while MetaGPT [8] formalizes agent collaboration through standardized workflows and shared memory to manage complex project pipelines. DeepAnalyze [44] supports exploratory data analysis via a domain-fine-tuned LLM, but lacks a multi-agent architecture with distinct roles; we include its results in the Appendix A.1 for completeness. Together, these baselines reflect the spectrum of approaches—from single-agent reasoning to multi-agent teamwork—that underpin modern LLM-driven automation in data-centric domains. 4.1.4 Implementation Details. All experiments are conducted on our benchmark, with each method evaluated using both gpt-4o and gpt-5.2. For baseline frameworks, we use their default inference configurations as reported in their original papers. For ProfiliTable, we employ its full pipeline with the following hyperparameters: at most 2 operator templates are retrieved via RAG with a similarity threshold of 0.5; the ReAct-style profiling loop runs for up to 7 steps; the overall feedback-driven refinement is repeated for at most 3 rounds with a score threshold 0.8; and the debugging sub-routine is allowed up to 5 attempts per iteration. 4.2
Overall Performance Table 1. Performance Comparison on Single-step Tasks. The best ones are in Bold, and the second ones are underlined. Base Model
gpt-4o
gpt-5.2
Framework
ATS↑
TSR↑
PSR↑
CRR↑
TRR↑
Avg. Score↑
Avg. Tokens↓
Avg. Time (s)↓
MetaGPT CAMEL CleanAgent ChatDev2.0 DataGovAgent ProfiliTable (ours)
56.21 46.83 37.03 55.89 52.34 86.82
44.44 33.33 30.00 44.44 45.56 68.89
62.22 54.44 46.67 61.11 56.67 93.33
57.66 44.83 57.14 52.5 54.46 84.72
75.56 72.22 78.89 70.00 67.78 97.78
59.22 50.33 49.95 56.79 55.36 86.31
15,241 82,845 18,043 114,785 28,501 25,794
69.18 70.56 17.93 133.23 51.56 51.44
MetaGPT CAMEL CleanAgent ChatDev2.0 DataGovAgent ProfiliTable (ours)
67.31 61.14 45.91 70.02 67.94 89.66
54.44 47.78 35.56 60.00 61.11 73.33
73.33 68.89 53.33 75.56 74.44 94.44
68.97 73.27 50.64 72.82 80.43 87.88
87.78 82.22 86.67 83.33 82.22 97.78
70.37 66.66 54.42 72.35 73.23 88.62
74,985 36,828 33,497 120,849 29,965 24,907
115.05 72.74 44.75 170.08 72.42 130.26
In this subsection, we present a comprehensive performance comparison of ProfiliTable against strong baselines across single-step and multi-step table processing tasks. Achieves SOTA in single-step tasks via dynamic profiling. ProfiliTable attains state-of-the-art accuracy in single-step tasks across both base models. With gpt-4o, it achieves an ATS of 86.82, surpassing the second-best method (MetaGPT, 56.21) by 30.61 points. This large margin reflects its ability to unify data exploration, knowledge retrieval, and feedback-driven refinement through a dynamic profiling context. When scaled to gpt-5.2, ProfiliTable further Paper under review
10
Wei Liu, Yang Gu, Xi Yan, Zihan Nan, Beicheng Xu, Keyao Ding, Bin Cui, and Wentao Zhang Table 2. Performance Comparison on Multi-step Tasks. The best ones are in Bold, and the second ones are underlined. Base Model
gpt-4o
gpt-5.2
Framework
ATS↑
TSR↑
PSR↑
CRR↑
TRR↑
Avg. Score↑
Avg. Tokens↓
Avg. Time (s)↓
MetaGPT CAMEL CleanAgent ChatDev2.0 DataGovAgent ProfiliTable (ours)
49.53 32.64 46.03 57.26 47.98 80.19
24.32 16.22 21.62 27.03 27.03 45.95
59.46 64.86 62.16 75.68 72.97 94.59
45.61 41.18 44.07 65.96 72.50 78.67
70.27 70.27 70.27 83.78 78.38 100.00
49.84 45.03 48.83 61.94 59.77 79.88
30,038 115,116 21,562 133,688 29,189 30,797
68.91 94.33 20.59 143.77 91.52 70.40
MetaGPT CAMEL CleanAgent ChatDev2.0 DataGovAgent ProfiliTable (ours)
66.61 62.50 59.40 70.18 67.00 82.49
48.65 35.14 24.32 40.54 43.24 48.65
81.08 78.38 78.38 83.78 81.08 97.30
63.27 69.77 49.23 74.42 80.00 96.55
83.78 81.08 81.08 86.49 86.49 100.00
68.68 65.37 58.48 71.08 71.56 85.00
84,331 68,642 23,570 117,398 30,778 29,901
115.65 87.46 45.21 156.74 108.17 195.27
elevates ATS to 89.66, which is again far ahead of ChatDev2.0 (70.02). Crucially, this consistent gain is model-agnostic because the dynamic profiling context ensures robust reasoning regardless of base model capacity. Sets new SOTA in multi-step table processing. ProfiliTable establishes state-of-the-art performance in multi-step tasks, achieving 80.19 (gpt-4o) and 82.49 (gpt-5.2) in ATS, while all baselines remain below 70.2. Critically, it is the only method to attain 100% task-wise runnable rate (TRR) in both settings, which ensures that every task’s final output is executable and meets a critical requirement for production deployment. The dynamic profiling context enables this capability by aligning subtask decomposition and execution with the global intent, thereby preventing error cascades. Highlights the fragility of baseline methods in the absence of unified profiling. MetaGPT works on atomic tasks (ATS = 56.21 with gpt-4o) but collapses in multi-step workflows (ATS = 49.53), lacking compositional awareness. CAMEL performs poorly with gpt-4o but improves noticeably with gpt-5.2, revealing its heavy reliance on base model strength rather than robust task reasoning. In contrast, ProfiliTable maintains strong performance across both models, demonstrating that its dynamic profiling context provides consistent, model-agnostic grounding for reliable table processing. Delivers high quality with low cost. It uses only 24,907 tokens on average (gpt-5.2, single-step), which is the lowest among top performers, while maintaining acceptable runtime efficiency. By grounding retrieval and refinement within the profiling context, ProfiliTable avoids redundant actions and hallucinated code, demonstrating that dynamic profiling enables both accuracy and cost efficiency. 4.3
Cost Analysis
Traces a Pareto-optimal ATS–token trade-off. We further analyze the efficiency–accuracy trade-off of ProfiliTable in terms of token cost and solution quality. As shown in Figure 3, ProfiliTable traces an empirical Pareto frontier in the average token consumption versus ATS plane by varying the maximum refinement rounds 𝑇max from 1 to 5, with all other components fixed. Starting from a lightweight configuration (21.3k tokens, ATS = 0.68), it progressively improves performance as it invests additional tokens in structured decomposition, ReAct-style profiling, and iterative refinement, reaching up to 0.81 ATS at 37.7k tokens. Crucially, all points along this trajectory form the Pareto frontier: CleanAgent achieves low token usage (21.6k) but substantially lower accuracy (0.46); MetaGPT and DataGovAgent consume more tokens yet achieve only marginal gains in ATS; and ChatDev2.0 achieves higher ATS but incurs significantly higher costs. This demonstrates ProfiliTable’s superiority: it delivers high-fidelity table transformations while maintaining strong cost efficiency, consistently occupying the optimal region of the performance–cost trade-off space. Paper under review
ProfiliTable: Profiling-Driven Tabular Data Processing via Agentic Workflows
11
Avg. Token-ATS Analysis
0.8
Profilitable MetaGPT CAMEL CleanAgent ChatDev2.0 DataGovAgent
Average Task Score
Pareto frontier
0.7 Profilitable (21.3, 0.6834)
0.6 0.5
CleanAgent (21.6, 0.4603)
0.4 20
40
60
80
100
120
Average Tokens per Task (×103)
Fig. 3. Trade-off between average token consumption and task accuracy (ATS) across methods on multi-step tasks with gpt-4o.
4.4
Hyperparameter Analysis
1.0
Hyperparameter Selection: k vs Metrics
1.0
0.8
0.8
Values
0.9
Values
0.9
Hyperparameter Selection: theta vs Metrics
0.7
0.7
0.6
0.6
0.5 1
2 ATS TSR
k
CRR TRR
(a) Hyperparameter: 𝑘
3
4 PSR
0.5 0.0
0.2 ATS TSR
theta
0.5
CRR TRR
0.8 PSR
(b) Hyperparameter: 𝜃 sim
Fig. 4. Effect of hyperparameters 𝑘 and 𝜃 sim on single-step performance (gpt-4o).
This subsection analyzes the impact of two key RAG-related hyperparameters on ProfiliTable’s performance: the maximum number of retrieved operators 𝑘 and the similarity threshold 𝜃 sim used during retrieval. Specifically, 𝑘 controls the upper bound on how many operator templates are fetched from the knowledge base for a given task, while 𝜃 sim filters out low-relevance candidates by requiring a minimum semantic similarity between the task description and Paper under review
12
Wei Liu, Yang Gu, Xi Yan, Zihan Nan, Beicheng Xu, Keyao Ding, Bin Cui, and Wentao Zhang
operator specification. Figure 4 plots the trends of performance metrics as 𝑘 and 𝜃 sim vary, respectively. The results reveal a clear trade-off between coverage and precision in retrieval, which we analyze in two aspects below. Retrieving at most 𝑘 = 2 operators achieves the best balance. As shown in Figure 4, all metrics peak when 𝑘 = 2. Setting 𝑘 = 1 restricts the generator’s access to alternative implementations, reducing flexibility in handling ambiguous or multi-faceted tasks. In contrast, larger values (𝑘 = 3 or 4) introduce irrelevant or redundant operators, which increase prompt noise and raise the risk of hallucinated compositions, thereby degrading both correctness and robustness. A similarity threshold of 𝜃 sim = 0.5 optimally filters signal from noise. In Figure 4, performance is maximized at 𝜃 sim = 0.5. Lower thresholds (e.g., 𝜃 sim = 0.2) admit too many semantically weak matches, overwhelming the generator with low-quality exemplars and leading to incorrect operator usage. Higher thresholds (e.g., 𝜃 sim = 0.8) are overly restrictive, often retrieving no operators for nuanced tasks—forcing the generator to fall back on generic code patterns that lack domain-specific grounding. In conclusion, the configuration 𝑘 = 2 and 𝜃 sim = 0.5 consistently yields the highest performance across all performance metrics, indicating that this setting provides sufficient diversity without sacrificing relevance. This validates our design principle: for table processing, effective RAG requires focused retrieval, which retrieves just enough relevant, high-quality operator examples to guide robust code generation without overwhelming the model with noisy or redundant candidates. Ablation Study
Ablation Study: Single-step
Values
1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2
1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2
Ablation Study: Multi-step
Values
4.5
ATS
TSR
ProfiliTable w/o Feedback
CRR
Metrics
w/o RAG
(a) Single-step tasks
TRR
PSR w/o Profiler
ATS
TSR
ProfiliTable w/o Feedback
CRR
Metrics
w/o RAG
TRR
PSR w/o Profiler
(b) Multi-step tasks
Fig. 5. Ablation study of ProfiliTable components across task complexities. Removing any module degrades performance, with the most severe drop observed when disabling feedback. The gap between full and ablated variants widens in multi-step settings (gpt-4o).
To better understand how each component contributes to ProfiliTable’s strong performance, we conduct ablation studies on single-step and multi-step tasks, evaluating the impact of removing key modules: ReAct Profiler, Multi-round Feedback, RAG-enhanced Operator Retrieval, and the Decompositer for multi-step orchestration. These variants are denoted as w/o Profiler, w/o Feedback, w/o RAG, and w/o Decompositer, respectively. Results are summarized in Figure 5a and Figure 5b, with a step-wise breakdown of the Decompositer’s effect shown in Figure 6. Profiling Module. Removing the Profiler leads to a noticeable drop in all metrics, especially on multi-step tasks. In this case, the workflow uses a default, rule-based profiling strategy similar to that of DataGovAgent, which only Paper under review
ProfiliTable: Profiling-Driven Tabular Data Processing via Agentic Workflows
13
ATS and Improvement Ratios by Steps 1.0
w/o Decompositer 0.175 ProfiliTable
0.150
Improvement Ratios
0.8 Average Task Score
0.125
0.6
0.100
0.4
0.075 0.050
0.2 0.0
0.025
2
3
4
Steps
5
6
7
0.000
Fig. 6. Impact of enabling or disabling the Decompositer module on performance across individual steps of multi-step tasks (gpt-4o).
collects basic column information without active exploration. This shows that static profiling cannot align context with task-specific requirements. Feedback Mechanism. The w/o Feedback variant, in which the Summarizer is disabled and the generator operates without iterative refinement, incurs the most severe performance degradation across all settings, particularly in ATS and TSR. On multi-step tasks, this gap is especially pronounced because the absence of reflective error diagnosis leads to uncorrected cascading failures that propagate through the workflow. Even on single-step tasks, the lack of post-execution insight results in consistent underperformance, underscoring feedback as the cornerstone of robust code generation. RAG Integration. Disabling RAG (w/o RAG) also reduces performance, particularly in ATS and CRR. This confirms that retrieving pre-validated operator templates helps suppress hallucination and improve correctness. However, the impact of removing RAG is comparatively milder on multi-step tasks than removing Profiling or Feedback. The performance drop is still noticeable, but smaller than that caused by ablating either the Profiler or the Summarizer. This suggests that, in complex workflows, structured exploration and iterative correction play a more decisive role than operator retrieval alone. Nevertheless, RAG remains valuable for grounding synthesis in reliable, domain-specific primitives. Decompositer for Task Decomposition. To further dissect the role of structured decomposition, we perform a fine-grained ablation by disabling the Decompositer in multi-step tasks and measuring performance at each step (indexed 2–7). As shown in Figure 6, the improvement from enabling the Decompositer varies significantly across steps: it delivers the strongest gains at mid-complexity steps. For example at step 5, ATS increases from 0.482 to 0.567 (+17.5% relative improvement), indicating that explicit subtask breakdown is most beneficial when precise operator sequencing is required. In contrast, early steps (2–3) see minimal gains, as atomic operations can often be handled Paper under review
14
Wei Liu, Yang Gu, Xi Yan, Zihan Nan, Beicheng Xu, Keyao Ding, Bin Cui, and Wentao Zhang
directly by the generator. At step 7, both variants collapse to zero performance due to accumulated errors, revealing a current limitation in ultra-deep pipelines. This step-wise analysis confirms that the Decompositer acts as a targeted enhancer—its value emerges precisely when task structure becomes nontrivial but not yet overwhelmed by error propagation. Critically, the full workflow consistently outperforms all ablated variants, with performance gaps widening significantly on multi-step tasks. This demonstrates that dynamic profiling becomes increasingly essential as task complexity grows. These results validate our design: by addressing diverse failure modes in table processing, dynamic profiling enables reliable and scalable automation. 4.6
Limitations and Implications
Reliable table processing demands more than strong models or generic agents: It requires structured, reflective workflows centered on dynamic profiling. Real-world tables often suffer from schema ambiguity and semantic gaps, causing non-profiling methods to fail even with models like gpt-5.2. Unlike rigid rule-based systems, dynamic profiling enables adaptive, context-aware reasoning by iteratively grounding decisions in data insights rather than static context. A key limitation remains: Our agent struggles with ultra-deep multi-step tasks and vague instructions that omit critical domain cues—such as treating “NA” as a valid non-null value—highlighting the need for future agents that internalize such conventions through targeted training. 5
Conclusion
We present ProfiliTable, a multi-agent framework for autonomous table processing built around dynamic profiling. Evaluated with gpt-4o and gpt-5.2, it outperforms all baselines by wide margins and is the only method to achieve 100% task-wise runnable rate in multi-step settings, ensuring that every task’s output is executable—a critical requirement for deployment. It also lies on the Pareto frontier of accuracy and efficiency, showing that dynamic profiling delivers both high fidelity and low cost. Our work points to a new paradigm: profiling-driven agency, which refers to iterative, interactive, and failure-aware systems that treat tables as dynamic, semantically rich artifacts.
Paper under review
ProfiliTable: Profiling-Driven Tabular Data Processing via Agentic Workflows
15
References [1] Tommaso Bendinelli, Artur Dox, and Christian Holz. 2025. Exploring LLM Agents for Cleaning Tabular Machine Learning Datasets. In Proceedings of the ICLR 2025 Workshop on Foundation Models in the Wild. https://arxiv.org/abs/2503.06664 Preprint available at https://arxiv.org/abs/2503.06664. [2] Fabian Biester, Mohamed Abdelaal, and Daniel Del Gaudio. 2024. LLMClean: Context-Aware Tabular Data Cleaning via LLM-Generated OFDs. arXiv:2404.18681 [cs.DB] https://arxiv.org/abs/2404.18681 [3] Zhicheng Ding, Jiahao Tian, Zhenkai Wang, Jinman Zhao, and Siyang Li. 2024. Data Imputation using Large Language Model to Accelerate Recommendation System. arXiv:2407.10078 [cs.IR] https://arxiv.org/abs/2407.10078 [4] Jiajie Fu, Haitong Tang, Arijit Khan, Sharad Mehrotra, Xiangyu Ke, and Yunjun Gao. 2025. In-context Clustering-based Entity Resolution with Large Language Models: A Design Space Exploration. arXiv:2506.02509 [cs.DB] https://arxiv.org/abs/2506.02509 [5] Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yi Dai, Jiawei Sun, Meng Wang, and Haofen Wang. 2024. Retrieval-Augmented Generation for Large Language Models: A Survey. arXiv:2312.10997 [cs.CL] https://arxiv.org/abs/2312.10997 [6] Xinrui He, Yikun Ban, Jiaru Zou, Tianxin Wei, Curtiss Cook, and Jingrui He. 2025. LLM-Forest: Ensemble Learning of LLMs with Graph-Augmented Prompts for Data Imputation. In Findings of the Association for Computational Linguistics: ACL 2025. Association for Computational Linguistics, 6921–6936. doi:10.18653/v1/2025.findings-acl.361 [7] Sirui Hong, Yizhang Lin, Bang Liu, Bangbang Liu, Binhao Wu, Ceyao Zhang, Chenxing Wei, Danyang Li, Jiaqi Chen, Jiayi Zhang, Jinlin Wang, Li Zhang, Lingyao Zhang, Min Yang, Mingchen Zhuge, Taicheng Guo, Tuo Zhou, Wei Tao, Xiangru Tang, Xiangtao Lu, Xiawu Zheng, Xinbing Liang, Yaying Fei, Yuheng Cheng, Zhibin Gou, Zongze Xu, and Chenglin Wu. 2024. Data Interpreter: An LLM Agent For Data Science. arXiv:2402.18679 [cs.AI] https://arxiv.org/abs/2402.18679 [8] Sirui Hong, Mingchen Zhuge, Jiaqi Chen, Xiawu Zheng, Yuheng Cheng, Ceyao Zhang, Jinlin Wang, Zili Wang, Steven Ka Shing Yau, Zijuan Lin, Liyang Zhou, Chenyu Ran, Lingfeng Xiao, Chenglin Wu, and Jürgen Schmidhuber. 2024. MetaGPT: Meta Programming for a Multi-Agent Collaborative Framework. In Proceedings of the 12th International Conference on Learning Representations (ICLR 2024). https://arxiv.org/abs/2308.00352 Published as a conference paper at ICLR 2024. Preprint available at https://arxiv.org/abs/2308.00352. [9] Yue Hu, Yuzhu Cai, Yaxin Du, Xinyu Zhu, Xiangrui Liu, Zijie Yu, Yuchen Hou, Shuo Tang, and Siheng Chen. 2025. Self-Evolving Multi-Agent Collaboration Networks for Software Development. In Proceedings of the 13th International Conference on Learning Representations (ICLR 2025). https://arxiv.org/abs/2410.16946 [10] Yoichi Ishibashi and Yoshimasa Nishimura. 2024. Self-Organized Agents: A LLM Multi-Agent Framework toward Ultra Large-Scale Code Generation and Optimization. arXiv:2404.02183 [cs.SE] https://arxiv.org/abs/2404.02183 [11] Yuhang Lai, Chengxi Li, Yiming Wang, Tianyi Zhang, Ruiqi Zhong, Luke Zettlemoyer, Wen-tau Yih, Daniel Fried, Sida Wang, and Tao Yu. 2023. DS-1000: A natural and reliable benchmark for data science code generation. In International Conference on Machine Learning. PMLR, 18319–18345. [12] Aodong Li, Yunhan Zhao, Chen Qiu, Marius Kloft, Padhraic Smyth, Maja Rudolph, and Stephan Mandt. 2024. Anomaly Detection of Tabular Data Using LLMs. arXiv:2406.16308 [cs.LG] https://arxiv.org/abs/2406.16308 [13] Ce Li, Xiaofan Liu, Zhiyan Song, Ce Chi, Chen Zhao, Jingjing Yang, Zhendong Wang, Kexin Yang, Boshen Shi, Xing Wang, Chao Deng, and Junlan Feng. 2025. TReB: A Comprehensive Benchmark for Evaluating Table Reasoning Capabilities of Large Language Models. arXiv:2506.18421 [cs.CL] https://arxiv.org/abs/2506.18421 [14] Guohao Li, Hasan Abed Al Kader Hammoud, Hani Itani, Dmitrii Khizbullin, and Bernard Ghanem. 2023. CAMEL: Communicative Agents for “Mind” Exploration of Large Language Model Society. In Advances in Neural Information Processing Systems (NeurIPS 2023), Vol. 36. Curran Associates, Inc. [15] Lan Li, Liri Fang, Bertram Ludäscher, and Vetle I. Torvik. 2025. AutoDCWorkflow: LLM-based Data Cleaning Workflow Auto-Generation and Benchmark. arXiv:2412.06724 [cs.DB] https://arxiv.org/abs/2412.06724 [16] Zexi Liu, Yuzhu Cai, Xinyu Zhu, Yujie Zheng, Runkun Chen, Ying Wen, Yanfeng Wang, Weinan E, and Siheng Chen. 2025. ML-Master: Towards AI-for-AI via Integration of Exploration and Reasoning. arXiv:2506.16499 [cs.AI] https://arxiv.org/abs/2506.16499 [17] Zhou Liu, Zhaoyang Han, Guochen Yan, Hao Liang, Bohan Zeng, Xing Chen, Yuanfeng Song, and Wentao Zhang. 2025. DataGovBench: Benchmarking LLM Agents for Real-World Data Governance Workflows. arXiv:2512.04416 [cs.AI] https://arxiv.org/abs/2512.04416 [18] Kiran Maharana, Surajit Mondal, and Bhushankumar Nemade. 2022. A Review: Data Pre-processing and Data Augmentation Techniques. Global Transitions Proceedings 3 (2022), 91–99. doi:10.1016/j.gltp.2022.04.020 [19] Meg Miller and Natalie Vielfaure. 2022. OpenRefine: an approachable open tool to clean research data. Bulletin-Association of Canadian Map Libraries and Archives (ACMLA) 170 (2022). [20] Avanika Narayan, Ines Chami, Laurel Orr, Simran Arora, and Christopher Ré. 2022. Can Foundation Models Wrangle Your Data? arXiv:2205.09911 [cs.LG] https://arxiv.org/abs/2205.09911 [21] Wei Ni, Xiaoye Miao, Xiangyu Zhao, Yangyang Wu, and Jianwei Yin. 2025. Automatic Data Repair: Are We Ready to Deploy? arXiv:2310.00711 [cs.DB] https://arxiv.org/abs/2310.00711 [22] Wei Ni, Kaihang Zhang, Xiaoye Miao, Xiangyu Zhao, Yangyang Wu, and Jianwei Yin. 2024. IterClean: An Iterative Data Cleaning Framework with Large Language Models. In Proceedings of the ACM Turing Award Celebration Conference (ACM-TURC ’24). Association for Computing Machinery, Changsha, China, 100–105. doi:10.1145/3674399.3674436 [23] Chunjong Park, Anas Awadalla, Tadayoshi Kohno, and Shwetak N. Patel. 2021. Reliable and Trustworthy Machine Learning for Health Using Dataset Shift Detection. In Proceedings of the 35th Conference on Neural Information Processing Systems (NeurIPS). 3043–3055. https://proceedings. Paper under review
16
Wei Liu, Yang Gu, Xi Yan, Zihan Nan, Beicheng Xu, Keyao Ding, Bin Cui, and Wentao Zhang
neurips.cc/paper_files/paper/2021/file/17e23e50bedc63b4095e3d8204ce063b-Paper.pdf [24] Yulu Pi. 2021. Machine Learning in Governments: Benefits, Challenges and Future Directions. JeDEM – eJournal of eDemocracy and Open Government 13, 1 (2021), 203–219. doi:10.29379/jedem.v13i1.625 [25] Neoklis Polyzotis, Sudip Roy, Steven Euijong Whang, and Martin Zinkevich. 2017. Data Management Challenges in Production Machine Learning. In Proceedings of the 2017 ACM International Conference on Management of Data (SIGMOD ’17). ACM, New York, NY, USA, 1723–1726. doi:10.1145/ 3035918.3054782 [26] Danrui Qi, Zhengjie Miao, and Jiannan Wang. 2025. CleanAgent: Automating Data Standardization with LLM-based Agents. arXiv:2403.08291 [cs.LG] https://arxiv.org/abs/2403.08291 [27] Chen Qian, Wei Liu, Hongzhang Liu, Nuo Chen, Yufan Dang, Jiahao Li, Cheng Yang, Weize Chen, Yusheng Su, Xin Cong, Juyuan Xu, Dahai Li, Zhiyuan Liu, and Maosong Sun. 2024. ChatDev: Communicative Agents for Software Development. arXiv:2307.07924 [cs.SE] https://arxiv.org/abs/2307.07924 [28] Fakhitah Ridzuan and Wan Mohd Nazmee Wan Zainon. 2019. A review on data cleansing methods for big data. Procedia Computer Science 161 (2019), 731–738. [29] Nithya Sambasivan, Shivani Kapania, Hannah Highfill, Diana Akrong, Praveen Paritosh, and Lora M Aroyo. 2021. “Everyone wants to do the model work, not the data work”: Data Cascades in High-Stakes AI. In proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–15. [30] Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. 2024. DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models. arXiv:2402.03300 [cs.CL] https://arxiv.org/abs/2402.03300 [31] Ravid Shwartz-Ziv and Amitai Armon. 2021. Tabular Data: Deep Learning is Not All You Need. arXiv:2106.03253 [cs.LG] https://arxiv.org/abs/2106. 03253 [32] Shreenidhi Srinivasan and Lydia Manikonda. 2025. Does Prompt Design Impact Quality of Data Imputation by LLMs? arXiv:2506.04172 [cs.LG] https://arxiv.org/abs/2506.04172 [33] Yuan Sui, Mengyu Zhou, Mingjie Zhou, Shi Han, and Dongmei Zhang. 2024. Table Meets LLM: Can Large Language Models Understand Structured Table Data? A Benchmark and Empirical Study. In Proceedings of the 17th ACM International Conference on Web Search and Data Mining (WSDM ’24). ACM, New York, NY, USA, 109–118. doi:10.1145/3616855.3635752 [34] Xiangru Tang, Yuliang Liu, Zefan Cai, Yanjun Shao, Junjie Lu, Yichi Zhang, Zexuan Deng, Helan Hu, Kaikai An, Ruijun Huang, et al. 2023. ML-Bench: Evaluating Large Language Models and Agents for Machine Learning Tasks on Repository-Level Code. arXiv preprint arXiv:2311.09835 (2023). [35] Patara Trirat, Wonyong Jeong, and Sung Ju Hwang. 2025. AutoML-Agent: A Multi-Agent LLM Framework for Full-Pipeline AutoML. In Proceedings of the 42nd International Conference on Machine Learning (ICML 2025) (Proceedings of Machine Learning Research, Vol. 267). PMLR, to appear. https://arxiv.org/abs/2410.02958 [36] He Wang, Alexander Hanbo Li, Yiqun Hu, Sheng Zhang, Hideo Kobayashi, Jiani Zhang, Henry Zhu, Chung-Wei Hang, and Patrick Ng. 2025. DSMentor: Enhancing Data Science Agents with Curriculum Learning and Online Knowledge Accumulation. arXiv:2505.14163 [cs.AI] https: //arxiv.org/abs/2505.14163 [37] Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. 2023. Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. arXiv:2201.11903 [cs.CL] https://arxiv.org/abs/2201.11903 [38] Xianjie Wu, Jian Yang, Linzheng Chai, Ge Zhang, Jiaheng Liu, Xeron Du, Di Liang, Daixin Shu, Xianfu Cheng, Tianzhen Sun, et al. 2025. Tablebench: A comprehensive and complex benchmark for table question answering. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 39. 25497–25506. [39] Junjie Xing, Yeye He, Mengyu Zhou, Haoyu Dong, Shi Han, Dongmei Zhang, and Surajit Chaudhuri. 2025. Table-LLM-Specialist: Language Model Specialists for Tables using Iterative Fine-tuning. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing (EMNLP 2025). Association for Computational Linguistics, 35443–35460. https://arxiv.org/abs/2410.12164 [40] Hongyang Yang, Boyu Zhang, Neng Wang, Cheng Guo, Xiaoli Zhang, Likun Lin, Junlin Wang, Tianyu Zhou, Mao Guan, Runjia Zhang, and Christina Dan Wang. 2024. FinRobot: An Open-Source AI Agent Platform for Financial Applications using Large Language Models. arXiv:2405.14767 [q-fin.ST] [41] Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2023. ReAct: Synergizing Reasoning and Acting in Language Models. arXiv:2210.03629 [cs.CL] https://arxiv.org/abs/2210.03629 [42] Dan Zhang, Sining Zhoubian, Min Cai, Fengzu Li, Lekang Yang, Wei Wang, Tianjiao Dong, Ziniu Hu, Jie Tang, and Yisong Yue. 2025. Datascibench: An llm agent benchmark for data science. arXiv preprint arXiv:2502.13897 (2025). [43] Lei Zhang, Yuge Zhang, Kan Ren, Dongsheng Li, and Yuqing Yang. 2024. MLCopilot: Unleashing the Power of Large Language Models in Solving Machine Learning Tasks. arXiv:2304.14979 [cs.LG] https://arxiv.org/abs/2304.14979 [44] Shaolei Zhang, Ju Fan, Meihao Fan, Guoliang Li, and Xiaoyong Du. 2025. DeepAnalyze: Agentic Large Language Models for Autonomous Data Science. arXiv:2510.16872 [cs.AI] https://arxiv.org/abs/2510.16872 [45] Shujian Zhang, Chengyue Gong, Lemeng Wu, Xingchao Liu, and Mingyuan Zhou. 2023. AutoML-GPT: Automatic Machine Learning with GPT. arXiv:2305.02499 [cs.CL] https://arxiv.org/abs/2305.02499
Paper under review
ProfiliTable: Profiling-Driven Tabular Data Processing via Agentic Workflows A
17
Additional Discussions
This section presents the results of DeepAnalyze (Section A.1) and discusses future directions (Section A.2). Table 3. Performance of DeepAnalyze on Single-Step and Multi-Step Tasks
A.1
Framework
Task Type
ATS↑
TSR↑
PSR↑
TRR↑
Avg. Score↑
Avg. Time(s)↓
DeepAnalyze
single-step
44.47
32.22
54.44
85.56
54.17
39.36
DeepAnalyze
multi-step
54.35
16.22
72.97
91.89
58.86
65.35
Additional Experiment
In this subsection, we present additional results for DeepAnalyze [44]. We include this method in the appendix rather than the main evaluation because it represents a fundamentally different paradigm: DeepAnalyze is built around a single, domain-adapted language model—specifically, ds-0528-qwen3-8b fine-tuned on chain-of-thought (CoT) data for data science tasks—rather than a multi-agent framework with distinct, collaborating roles. Its core mechanism extends the ReAct framework by introducing a set of special tokens that condition the model’s behavior at each step. Depending on the observed state and user intent, the model selects actions, such as coding and analyzing, by emitting these tokens, effectively implementing an enhanced, token-driven ReAct loop. While this design enables flexible data analysis, it lacks the explicit division of labor, interactive feedback, and profiling-driven coordination that characterize true multi-agent systems like ProfiliTable. As shown in Table 3, its performance consistently falls short of ProfiliTable across all performance metrics. A.2
Future Work
An exciting direction is to leverage the high-quality CoT [37] data synthesized by ProfiliTable’s workflow to enhance single-model capabilities. Specifically, we plan to use reinforcement learning algorithms such as GRPO [30] to fine-tune base language models on this curated dataset. This approach would yield table-processing-specific models that inherit both the structured reasoning of our multi-agent framework and the efficiency of a unified architecture, combining the best of workflow-level intelligence and model-level specialization. B
Details of Experiment
This section provides a detailed description of the evaluation metrics used in our work (Section B.1) and presents the prompt templates employed in our experiments (Section B.2). B.1
Metrics
As shown in Table 4, we provide a detailed description of the evaluation metrics, where 𝑁 denotes the total number of evaluated tasks. These metrics collectively assess both the effectiveness and efficiency of table processing agents: performance-oriented measures (e.g., ATS, TSR, PSR, CRR, TRR) capture correctness, robustness, and partial progress, while efficiency-oriented measures (e.g., Avg. Tokens and Avg. Time) quantify computational cost and latency, offering a holistic view of agent capabilities in real-world table wrangling scenarios. Paper under review
18
Wei Liu, Yang Gu, Xi Yan, Zihan Nan, Beicheng Xu, Keyao Ding, Bin Cui, and Wentao Zhang Table 4. Evaluation Metrics Used in This Work
Metric
Abbr.
Formula
Description
Performance Metrics (reported as percentages) Average Task Score
ATS
100 × 𝑁1
Í𝑁
Task Success Rate
TSR
100 × 𝑁1
Í𝑁
Partial Success Rate
PSR
100 × 𝑁1
Í𝑁
Code Runnable Rate
CRR
100 × #generated scripts
Task-wise Runnable Rate
TRR
100 ×
Average Metric Score
Avg. Score
100 × ATS+TSR+PSR+CRR+TRR 5
Average Tokens
Avg. Tokens
1 Í𝑁 𝑖=1 tokens𝑖 𝑁
Average Time
Avg. Time (s)
1 Í𝑁 𝑖=1 𝑡𝑖 𝑁
𝑖=1 score𝑖
𝑖=1 I(score𝑖 = 1) 𝑖=1 I(score𝑖 > 0)
#runnable scripts
#tasks with ≥1 runnable script 𝑁
Mean of normalized task scores (0–1), scaled to percentage; measures overall correctness. Percentage of tasks achieving perfect output (full score of 1.0). Percentage of tasks with strictly positive scores, indicating meaningful partial progress. Percentage of generated code snippets that execute successfully (syntactic and runtime validity). Percentage of tasks for which at least one attempt yields a runnable script. Mean of the five core performance metrics (ATS, TSR, PSR, CRR, TRR), providing a holistic assessment of correctness, robustness, and reliability.
Efficiency Metrics
B.2
Average number of tokens consumed by the agent per task. Average wall-clock execution time per task in seconds.
Prompts for ProfiliTable
This subsection presents the prompt templates employed in our experiments. These prompts are carefully designed to guide the large language model through the table processing workflow, incorporating task instructions, input schema descriptions, and output formatting requirements. The figures below illustrate the system prompt used across all agents, which establishes the agent’s role, capabilities, and general response format. In contrast, the user prompt is dynamically constructed at each iteration and includes additional context such as memory and feedback from prior steps.
Paper under review
ProfiliTable: Profiling-Driven Tabular Data Processing via Agentic Workflows
19
System prompt for Interpreter
System prompt for Decompositer
[ROLE] You are a Table preprocessing intent parsing API. Based on the metadata of the data table {task_meta}, parse the user's natural language instruction into a standardized JSON format to identify the required data processing operator. You must respond with only a JSON object—do not wrap it in ```json.
[ROLE] You are an expert in decomposing complex tasks into independent, executable sub-tasks.
[OUTPUT RULES]: 1. **operation**: Clearly describe the required operator operation in natural language, use orders like 1. ..., 2. ... to list multiple operations if needed. You should describe as detailed as possible. 2. **reason**: Briefly explain the rationale for selecting this operator (1–2 sentences), possibly referencing missing rates, data distribution, or task objectives in the metadata. 3. **task_type**: Select exactly one matching task type from: {benchmark_task_types}. The chosen type must strictly align with both the table metadata and the user's intent. 4. **suffix**: Specify the output file format, e.g., "csv", "jsonl", etc. Your output must be a valid JSON object only, with no additional text, explanations, Markdown formatting (such as ```json), or line breaks. Strictly follow the format in the example below.
[OUTPUT RULES]: 1. Decompose the task into independent sub-tasks. 2. Each sub-task should be executable and can be found in {benchmark_task_types}. 3. Output the result strictly in JSON format, mapping each sub-task type to its specific operation description, no ```json wrapping. 4. Each key in the JSON should be a different sub-task type, and the corresponding value should be the specific operation description. [Example]: User request: Merge multiple CSV files and deduplicate entries based on a primary key. Your output: {"TableTransformation-SplittingANDConcatenation":"Merge multiple CSV files", "TableCleaning-Deduplication":"Deduplicate entries based on a primary key"} Note: ensure output's keys are different sub-task types. Concatenate multiple operations under the same sub-task type into one description if needed.
[Example]: User request: Fill missing values in the age column using an LSTM model trained on other columns Your output: {"operation": "1:fill missing values in the column age using LSTM model trained on other columns", "reason": "'age' has 30% missing values, which may impact analysis. Using LSTM can better capture relationships with other columns to impute missing values.", "task_type": "TableCleaning-DataImputation", "suffix": "csv"}
Fig. 7. System prompts for key components of ProfiliTable. Left: Interpreter; Right: Decompositer.
Paper under review
20
Wei Liu, Yang Gu, Xi Yan, Zihan Nan, Beicheng Xu, Keyao Ding, Bin Cui, and Wentao Zhang
System prompt for Profiler [ROLE] You are a careful data profiler to prepare data report for target. Your role is to write code that analyzes tabular data files (CSV) and produces a comprehensive data profiling report in JSON format. You are given up to {MAX_REACT_STEPS} attempts to reach a conclusion. [Inputs] Files_paths: {raw_table_paths} target: {operation} [Goal] Prepare for what the target requires by analyzing the data files and producing a detailed profiling report. You are not to fulfill the target yet — only analyze and report for further processing. Produce the final data profiling report as JSON inside <ANSWER>{"table_1":{...}, "table_2":{...}, ...}</ANSWER>, where `table_x` is replaced by the **filename without extension** (e.g., `sales.csv` → `"sales"`). At least you need to include the number of rows, number of columns, column names, column types (detect abnormal types like mixed types or unexpected nulls) and so on. If the goal is to transform some column or correct some column..., you need to also analyze that column in detail, like unique values, missing rate, distribution for numeric columns so that the next step can be better performed. If the number of unique values or some other statistics is small (like less than 20), you should list them all, otherwise you just need to sample 5~10 values. Final report should be concise (don't surpass 200 characters unless necessary) but comprehensive, focusing on key statistics and insights. Useless information should be avoided. [Rules] - In each turn: - Use <THINK>...</THINK> to describe your reasoning. - Use <ACTION>```python\n...\n```</ACTION> to provide **standalone, executable Python code**. <ACTION> ```python import pandas as pd df = pd.read_csv("file.csv") print({"columns": list(df.columns), "shape": list(df.shape)}) ``` </ACTION> - Use <ANSWER>...</ANSWER> to provide the final JSON profiling report without any ``` formatting. - After the action, wait for the observation (the printed output). - After receiving observation, continue reasoning with <THINK>, then issue next <ACTION> if needed. - Your code must be **fully self-contained**: include all imports, data loading, and logic. Do *not* rely on prior context or variables. - Always load data from the provided file paths. - You should use print() to output results, which will be captured as observations. Avoid printing raw data or huge outputs. - For multiple files: profile each one and include a separate entry in the final JSON report. - Keep code precise and concise (≤50 lines per action unless absolutely necessary). - Do **not** write any files to disk — only output via `print()`. - Once profiling is complete, output the full report in <ANSWER>...</ANSWER> as valid JSON without ```. [EXAMPLE] <THINK>I need to read the first CSV file and get basic column info.</THINK> <ACTION> ```python import pandas as pd df = pd.read_csv("data/sales.csv") print({"columns": list(df.columns), "shape": list(df.shape)}) ``` </ACTION> [Observation] {"columns": ["id", "amount", "date"], "shape": [1000, 3]} <THINK>Now I'll compute statistics for numeric columns...</THINK> <ACTION> ```python import pandas as pd df = pd.read_csv("data/sales.csv") numeric_cols = df.select_dtypes(include='number').columns stats = df[numeric_cols].describe().to_dict() print(stats) ``` </ACTION> <THINK>All tables are profiled. Compiling final JSON report.</THINK> <ANSWER>{...}</ANSWER>
Fig. 8. System Prompt for Profiler
Paper under review
ProfiliTable: Profiling-Driven Tabular Data Processing via Agentic Workflows
21
System prompt for Generator [ROLE] You specialize in table processing. Please generate a bug-free Python script based on the following information and the user's request: [INPUT] 1. Metadata of the data table: {task_meta} 2. Retrieved similar operator code snippets: {retrieved_operators}, If there exits, you can refer to them when writing the code. But do not copy them directly, you need to adapt them to fit the current task. 3. Operator specification: {user_query} [OUTPUT RULES]: 1. The code must be executable, safe, step by step and output as a complete code block in the format ```python ... ```. 2. The code must include a main() function that accepts command-line arguments for input and output file paths. Use a fixed argparse format with two required arguments: --input (input file path or list of paths) and -output_path_dir (output file path directory). 3. The function must fulfill all user requirements. Ensure the output file format matches the user's request and contains no extra columns beyond those in the input. 5. If the task involves multiple tables, the --input argument should be treated as a list of file paths, this list can have 1,2... paths. And the --output_path_dir argument will be the directory where the results will be saved. 6. Please avoid modifying the original input files; read from them and write results to new files in the specified(--output_path_dir) output directory. 7. no BOM in output csv file means that don't use encoding='utf-8-sig' when saving csv file. 8. Let the code step by step and don't use complex logic in one step. Use as many steps as needed to ensure clarity and correctness. [Example] [INPUT] User request: Fill missing values in the age column using an LSTM model trained on other columns Operator specification: {"operators": "fill missing values in the column age using LSTM model trained on other columns", "reason": "'age' has 30% missing values, which may impact analysis. Using LSTM can better capture relationships with other columns to impute missing values.", "task_type": "TableCleaning-DataImputation", "suffix": "csv"} Metadata: {...} Retrieved similar operator code snippets: [ ... ] Debug_history: [ ... ] [OUTPUT] (illustrative only): ```python import json ...(import statements) def fill_missing_age_with_lstm(df): # implement logic here def main(): parser = argparse.ArgumentParser(description="Fill missing 'age' values using LSTM.") parser.add_argument("--input", required=True, nargs='+', help="Path(s) to input CSV/Parquet file(s)") parser.add_argument("--output_path_dir", required=True, help="Path to output file's directory") args = parser.parse_args() ... df_filled.to_csv(output_path, index=False) if __name__ == "__main__": main() ```
Fig. 9. System Prompt for Generator
Paper under review
22
Wei Liu, Yang Gu, Xi Yan, Zihan Nan, Beicheng Xu, Keyao Ding, Bin Cui, and Wentao Zhang
System prompt for Debugger [ROLE] You are an expert in code debugging and correction. [TASK] Given the original code, error message, requirement, and reference code, minimally modify the original code to fix the error. Ensure your corrections are precise and focus on issues such as key alignment or import errors. Output the corrected code and your reason for modification strictly in JSON format, and follow all specified requirements. [INPUT] You will receive the following informations in human request: - The original code: - The error messages: - The target: - Raw data and expected data formats: [OUTPUT RULES] 1. The response must be strictly in JSON format, containing only the keys "code" (with the complete corrected code) and "reason" (explaining the modification); no extra keys, explanations, comments, or markdown syntax are allowed. 2. The code's --input and --output_path_dir arguments should be kept unchanged. 3. The code must include an `if __name__ == '__main__':` block to ensure the script can be run independently. 4. All parser arguments should have default values except --input and -output_path_dir. 5. Your output must be a valid JSON object only, with no additional text, explanations, Markdown formatting (such as ```json), or line breaks. 6. No additional files or external references should be included unless explicitly required to resolve the error, and if needed, they must be handled within the code itself.
Fig. 10. System Prompt for Debugger
Paper under review
ProfiliTable: Profiling-Driven Tabular Data Processing via Agentic Workflows
23
System prompt for Summarizer [ROLE] You are a careful evaluator tasked with analyzing the processed results of a task to determine if they meet the target requirements. Your role is to write code that evaluates the processed file(s) and produces a summary of whether the target requirements are satisfied, and give reasonable suggestions for improvement if not. You are given up to {MAX_REACT_STEPS} attempts to reach a conclusion. [Inputs] metadata: {task_meta} this is the metadata after processing processed_file_paths: {processed_file_paths} raw_file_paths: {raw_file_paths} task_objective: {task_objective} [Goal] The generated code should directly assess the *content* of the processed file(s) for basic reasonableness — e.g., presence of required fields, structural/schema consistency, and absence of obvious anomalies (e.g., empty arrays, malformed JSON/CSV, unexpected nulls in critical columns). Do NOT generate ground truth (gt) or simulate expected outputs. Do NOT compute, assign, or justify any numerical scores (e.g., no "0.8/1.0" reasoning). Do NOT attempt to modify the files, nor check whether a hypothetical fix worked. -Identify concrete, observable issues and — if present — give short, actionable suggestions. -You should sample each column's data from processed files and compare it with raw files to identify discrepancies based on the target requirements. -If inspection reveals no clear issues, or further analysis yields conclusions nearly identical to the previous round, promptly summarize concisely and output <ANSWER>. [Rules] - In each turn: - Use <THINK>...</THINK> to describe your reasoning. - Use <ACTION>```python\n...\n```</ACTION> to provide **standalone, executable Python code**. → The code **must be wrapped in triple backticks with language specifier `python`**, like: <ACTION> ```python with open("processed_file.csv") as f: content = f.read() print("Evaluation result: Pass") ``` </ACTION> - Use <ANSWER>...</ANSWER> to provide the final evaluation summary as a string. - After the action, wait for the observation (the printed output). - After receiving observation, continue reasoning with <THINK>, then issue next <ACTION> if needed. - Your code must be **fully self-contained**: include all imports, data loading, and logic. Do *not* rely on prior context or variables. - Always load data from the provided file paths. - You should use print() to output results, which will be captured as observations. Avoid printing raw data or huge outputs. - Keep code precise and concise (≤50 lines per action unless absolutely necessary). - Do **not** write any files to disk — only output via `print()`. - Once you find some problems or all requirements are met, output the final summary in <ANSWER>...</ANSWER> as a string immediately. [EXAMPLE] <THINK>I need to load the processed file and check if it meets the target requirements.</THINK> <ACTION> ```python import pandas as pd df = pd.read_csv("...") if "target_column" in df.columns: print("Evaluation result: Pass") else: print("Evaluation result: Fail, missing 'target_column'") ``` </ACTION> [Observation] Evaluation result: Fail, missing 'target_column' <THINK>The processed file doesn't meet the target requirements. Let's check other requirements.</THINK> ... <ANSWER>The processed file misses 'target_column' and ...</ANSWER>
Fig. 11. System Prompt for Summarizer
Paper under review