arXiv:2605.15132v1 [cs.AI] 14 May 2026
APWA: A Distributed Architecture for Parallelizable Agentic Workflows
Evan Rose Northeastern University [email protected]
Tushin Mallick Northeastern University [email protected]
Matthew Laws Northeastern University [email protected]
Cristina Nita-Rotaru Northeastern University [email protected]
Alina Oprea Northeastern University [email protected]
Abstract Autonomous multi-agent systems based on large language models (LLMs) have demonstrated remarkable abilities in independently solving complex tasks in a wide breadth of application domains. However, these systems hit critical reasoning, coordination, and computational scaling bottlenecks as the size and complexity of their tasks grow. These limitations hinder multi-agent systems from achieving high-throughput processing for highly parallelizable tasks, despite the availability of parallel computing and reasoning primitives in the underlying LLMs. We introduce the Agent-Parallel Workload Architecture (APWA), a distributed multi-agent system architecture designed for the efficient processing of heavily parallelizable agentic workloads. APWA facilitates parallel execution by decomposing workflows into non-interfering subproblems that can be processed using independent resources without cross-communication. It supports heterogeneous data and parallel processing patterns, and it accommodates tasks from a wide breadth of domains. In our evaluation, we demonstrate that APWA can dynamically decompose complex queries into parallelizable workflows and scales on larger tasks in settings where prior systems fail completely.
1
Introduction
Autonomous LLM agents operationalize the planning, reasoning, and problem-solving capabilities of modern LLMs in real environments by situating the LLM at the center of a broader software system capable of interacting with an environment [62, 52, 54]. These LLM agents have been successfully applied to domains spanning software development [26, 60, 58], cybersecurity [30, 11, 36], web browsing [12, 65, 47], healthcare [28], finance [63], scientific research [4, 34, 55, 6, 61], and more. While LLM agents have unlocked a wide array of applications, their LLM backbone imposes several technical constraints limiting their applicability in some settings. While LLMs have absorbed knowledge from diverse domains (and thus are knowledge generalists) [5, 20], they may struggle to reliably modulate between multiple functional roles within a single execution trajectory [24, 18]. Additionally, as tasks become larger and more complex, reasoning and output quality degrade substantially [33, 15, 22], and eventually collapse completely as the size of the data the LLM must Preprint.
Batch Execution
Task Manager
Subtask Worker Agent Worker Toolset
Manager Agent
Toolset
Execute Subtasks Submit Batch
Data Capability Result Access Queries Submission
...
State Table Preset Workflow Subtask Inspection Manip. Mgmt. Planning Delegation
Subtask Worker Agent AutoRetry
Data Tables
Worker Execution Subtask Presets Plan Staging
Subtask Results
Batch Output
Subtask Worker Agent xN (parallel) Subtask Worker Agent
...
Manager State
Capabilit ies
Collect Results
Figure 1: Overview of APWA. APWA dynamically decomposes tasks into parallelizable workflows leveraging agent workers and executes them in a distributed environment.
process exceeds the context window of the LLM. To scale agentic applications to a broader class of problems, research has explored developing multi-agent systems which coordinate several LLM agents together via message passing to cohesively solve a single complex task [21, 32, 48, 49, 8, 16, 57, 23]. Multi-agent architectures allow agents to assume specialized roles [31, 48, 21, 57, 23] and partition work, memory, and communication [49], leading to higher-quality outputs [31, 21, 56], coherence over longer time scales [21, 49], and more efficient context management [49, 16]. Despite the progress made by prior research on multi-agent systems, existing solutions do not work well for problems that involve operating on a large amount of data that does not fit into an agent’s context, memory, or permanent storage, or those involving a massive number of nonoverlapping subproblems that each requires deeper research exploration. Specifically, they suffer 1 They do not provide support for intelligent and automated task from the following limitations. ⃝ decomposition over a distributed computing and storage infrastructure to support highly parallelizable 2 They do not provide support for efficient and scalable coordination among large teams workloads. ⃝ 3 They limit an individual agent’s exploration to solving a subproblem through excessive of agents. ⃝ 4 They do not support complex, dynamic, data-dependent, heterogeneous global coordination. ⃝ 5 They are not general enough, being designed for specific applications. planning and processing. ⃝ To better understand the problem we aim to solve, consider the paradigm shift introduced by systems designed for highly parallelizable workloads like MapReduce [10] and Apache Spark [64]. These systems transformed traditional data processing by simplifying the development of distributed applications. They provide intuitive and expressive abstractions for programmers, along with efficient and robust implementations of common data-processing patterns. As a result, they scale seamlessly in distributed environments while effectively hiding underlying system complexity from the user. The research question that we are asking in this paper is how to create a similar paradigm shift in the AI agentic environment by designing an architecture that allows multi-agent systems to solve problems that would benefit from high-parallelization with respect to the processed data and the executed subtasks. Designing a system that meets our goals presents several challenges. First, although LLMs have shown strong performance on software engineering tasks [60, 26, 58], they lack native reasoning capabilities for distributed infrastructure and large-scale parallelism. Prior work on LLM-guided parallel workflows is limited [27, 50, 41] and does not address large-scale data processing. Second, existing agent frameworks lack abstractions for reasoning over large, distributed datasets. LLM agents typically rely on local execution state, which does not scale when data exceeds a single machine or the model’s context window. Even metadata- or digest-based approaches break down at scale, as reasoning over millions of objects can overwhelm the model. Unlike traditional systems—where metadata is assumed to fit in working memory—LLM-based systems must contend with settings where even metadata is too large. In this paper, we introduce the Agent-Parallel Workload Architecture (APWA), a distributed multiagent system designed specifically for processing parallelizable workflows. At the core of APWA lie novel programming abstractions suited for LLM agents that enable them to reason about large-scale data resources, decompose workflows into non-interfering subproblems that can be processed using independent resources without cross-communication, and issue and inspect distributed data-parallel 2
execution flows. Based on these abstractions, we provide a scalable implementation of APWA leveraging Ray [39] and demonstrate that it accommodates different tasks that are parallelizable, outperforms several baselines, and scales on larger tasks in settings where prior systems fail completely. We show an overview of APWA in Figure 1.
2
Background and Problem Statement
2.1
Architectures for Multi-Agent Systems
Several emerging architectures for multi-agent systems allow practitioners to express highly flexible multi-agent teams. However, they are not suitably designed to accommodate highly-distributed infrastructure, automated workload partitioning, and massively parallelized execution [59, 16]. Autogen [59] implements a team-based programming model in which developers may manually define and instantiate agents within a publish-subscribe message sharing fabric. Magentic-One [16], a multi-agent system implemented in Autogen, features an orchestrator-worker architecture for automatically decomposing and solving complex tasks through multi-agent cooperation. However, these agents operate via synchronized message passing explicitly routed through an LLM-based orchestrator agent, inhibiting scalability and parallelization since (1) the orchestrator LLM must manage a global view of the running agents, limiting the feasible number of agents to 10s or 100s, and (2) only a single agent can process information and communicate results at a time, making effective parallelization impossible. A small set of multi-agent systems architectures attempt to facilitate parallelizable workflows. However, they either restrict to static parallelization patterns [29] or limit parallelization to a small scale by requiring expensive agent-modulated orchestration [57]. For example, MegaAgent, a representative architecture in this category, has significant coordination overhead where administrators must manually create workers, can not accommodate format divergence between peers at different layers of the hierarchy, and can not organize large data across the workers. 2.2
Problem Statement
Our goal is to design a scalable multi-agent system suited for highly parallelizable agentic workflows meeting several design goals: 1 Enable intelligent and automated task decomposition over a distributing computing and storage ⃝ infrastructure to support highly parallelizable workloads: Our architecture will enable massively parallelized execution patterns where many workflows can be partitioned into non-interfering subproblems that can be processed using independent resources and without cross-communication. These parallel execution patterns are efficiently facilitated without resource or reasoning bottlenecks. 2 Provide support for efficient and scalable coordination among large teams of agents. Provide ⃝ more efficient communication abstractions and state sharing including access to large data and a rich set of tools. 3 Provide individual agents exploration capabilities for solving a subproblem. Our systems will al⃝ low agents to perform individual exploration to solve a subproblem, by decoupling global coordination from local agent state. 4 Provide support for complex, dynamic, data-dependent, heterogeneous planning and processing: ⃝ Our system will facilitate the execution of data-parallel workflows (similar instructions applied to different subsets of a data resource), task-parallel workflows (different instructions being applied to the same inputs), and replication-parallel workflows (tasks and inputs are nearly identical but the goal of parallelization is to search through a large solution space more quickly). Our system will facilitate data-dependent exploration for highly specialized task requests. In the case of heterogeneous data, simple data processing rules generated by a single source are insufficient to express the desired operations to apply to data. Our system will support complex, context- and data-dependent trajectories which are hard to anticipate with static, prescriptive workflows. 5 Be task- and workflow agnostic: Our system will automatically accommodate tasks from a wide ⃝ breadth of domains and not restrict to particular processing or parallelization patterns. For example, our architecture naturally leverages the expressivity and flexibility of LLM agents to automatically discover efficient parallel execution paths. 3
Challenges Designing a system meeting our design goals faces several key challenges. First, while LLMs have made substantial progress in software engineering tasks [60, 26, 58], they lack native reasoning capabilities for distributed infrastructure and large-scale parallelism. Only very few works consider LLM-guided parallel workflow generation [50, 41], and none of these address truly largescale data processing. Second, existing agent scaffolding does not provide the necessary abstractions for LLM-based components to reason over large and distributed data resources. Typically, LLM agents operate by inspecting local execution state directly, but this approach does not scale when the data does not fit on a single machine, much less in the restricted context window of even modern LLMs. In fact, even metadata- or digest-based approaches cannot work. In cases when the LLM must reason over thousands or even millions of data objects, even simply enumerating identifiers for each of those objects overloads the LLM’s reasoning abilities. Notice a key distinction from traditional computing settings: with traditional computing mechanisms (e.g., distributed dataflow engines), it is typically safe to assume that at least the metadata fits inside the working memory of the system and is within its processing abilities. With LLM-based components, however, even metadata becomes too large.
3
The APWA System
APWA is a distributed multi-agent system that is designed specifically for highly parallelizable agentic workflows. At the core of APWA lies a set of abstractions that enables it to achieve all the five goals listed in Section 2.2. These abstractions follow a hierarchical organization: system-level abstractions that support high-level parallelization goals (Section 3.1), and distributed-level abstractions that support lower-level parallelization goals (Section 3.2). We use these abstractions to implement APWA (Section 3.3). APWA Nodes
Shared Node Services
Node Resource Manager
Object Store Proxy Object Store Cache
Task Manager Sandboxes Manager Agent
LLM
Subtask Sandboxes Subtask Agent
Task Manager State
Table Manipulation Tools
Plan
Subtask Delegation Tools
Data Tables
Planning Tools
Subtask State
Subtask Executor
Agent Presets
LLM
Capability Registry Proxy
External Services LLM Inference Provider
Capability Registry
State Store //// Tasks
Capability Cache
Manager Checkpoints
Other shared services - LLM proxy - local model inference - etc.
Subtask Metadata
Data Access Tools Capability Tools
Capability Runtimes LLM
Batch Metadata
Subtask Outputs Data Table Lineage
Figure 2: APWA Distributed System Architecture.
3.1
APWA system-level abstractions
As other multi-agent systems, APWA is organized around three main abstractions: the manager, the worker, and the executor. However, there are fundamental differences with respect to other systems, as these abstractions target data-parallel or task-parallel workflows. Both manager and worker have planning roles, the manager is performing meta-planning by decomposing the task into parallelizable subtasks, while a worker is performing planning to solve the assigned subtask, by using the necessary agents and tools. The executor performs the actual execution in a distributed cluster environment hiding the low-level distributed computing details. Manager It is the abstraction in charge of solving a task and there can be only one manager for a task. The manager dynamically partitions the task into non-interfering subtasks which can be dispatched and executed in parallel. The manager is designed to perform parallelization by providing it with: (1) definition of what is a general parallelizable unit, i.e., subtask, (2) guided planning with examples of what parallelization means and parallelization patterns, (3) abstractions for how large data can be communicated to workers. Subtasks specify contracts for a unit of work to be completed as well as the requested worker configuration. The manager is the only component in the system with a global view of a task’s execution state, and it is the only one performing high-level task reasoning. 4
Worker A worker represents the abstraction in charge of solving a subtask. There can be many workers (thousands) for a task. Workers can range in complexity from simple LLM processors to multi-agent subsystems with specialized roles and abilities. Workers also receive support for parallelization: (1) they receive guidance for subtask solving in terms of role they play in the system, (2) they can use a wide set of agents and tools, (3) they use abstractions to share large data across the agents they use to solve the task. A worker has a local view of the system state consisting of only inputs explicitly passed by the manager and anything contained in their local execution space. One of the key points of our design is identifying the appropriate separation between what is the global state maintained by the manager, and what is the local state maintained by the worker, as we wish to give workers high-autonomy on how they solve the subtask. Executor The main role of the executor is to collect execution requests from the manager, execute them and report the results. The fundamental difference from other systems, is that the focus is on leveraging distributed resources available in a cluster computing environment. Specifically, the executor places the subtasks on nodes in the cluster to execute, handling timeouts, automatic retries for transient errors, and principled collection of the results. The executor abstracts away many of the distributing computing details, including worker placement across virtual or physical nodes, resource allocation, and failure recovery. By handling node-level resource management and automated retry logic internally, the executor allows the manager to focus entirely on the logical structure and semantics of the subtasks, without needing to reason about low-level infrastructure concerns. 3.2
System abstractions for distributed agentic computation
To achieve our goal of a distributed system architecture for agentic parallelizable workflows, we introduce several key abstractions which enable the manager and workers to reason efficiently over intermediate execution state and distributed data resources, define and issue high volume of parallel subtasks, and manipulate internal data representations. Planning for parallelization We implement an instance of the planning design pattern tailored specifically for parallelizable workflows. When the manager agent is initialized, it is instructed to immediately explore its local state in order to understand the nature of the task as well as the structure of any input data passed to it. Then, the manager immediately generates a structured plan object consisting of a list of steps to be completed, that it will maintain for the duration of the task life cycle. This plan includes fields that explicitly direct the agent to decompose the task into subtasks as well as discuss how to partition and/or reorganize any available data or intermediate results. The manager is also permitted to specify an output contract object, which defines output structures which must exist before the task can be considered completed and is checked against the true system state to provide feedback to the agent and prevent premature task termination. Subtask delegation The goal of this abstraction is to allow the manager to explain what a subtask is in a general manner without manually generating a specification for each one. This abstraction is also designed to support parallelization by allowing the manager to efficiently specify batches of subtasks to be executed in parallel. We introduce subtask templates, which describe a unit of work to be completed by a worker and include fields for agent configuration, the task to be completed, and references to data resources to be processed. To enable efficient specification of a large number of subtasks, subtask template input parameters can contain placeholders which are automatically expanded when paired with an appropriate data resource, meaning the logical content of the subtask is decoupled from the scale of the data it acts upon. Data tables In a distributed execution setting, some tasks might involve large data objects that collectively exceed the context window of the system LLM, or even the memory constraints of the machine processing the main task. To minimize friction from large-scale data resources, we introduce a data table abstraction that facilitates easier data interactions with LLM-based components. A data table is a logical, read-only, finite sequence of records conforming to a common schema. Data tables are quite similar to related notions in data analysis libraries and databases. Importantly, data tables enable LLM-based components to interact with extremely large-scale data in a highly compact metadata representation. 5
APWA’s manager interacts with tables through a dedicated tool suite that enable the manager to inspect, query, analyze, manipulate, and construct tables dynamically in order to gather useful information about intermediate execution state as well as to prepare data for future processing. We define two categories of table-focused tools: table analytics tools allow the task manager agent to query information about data stored in tables and table manipulation tools allow the task manager to construct new tables from existing ones. A full set of table tools and their descriptions are given in Appendix D.1.2. Dynamic agent capabilities and presets Our system architecture is general-purpose, as we do not make any explicit assumptions about which agent-powered software capabilities are needed to perform a task. Moreover, it may be the case that these capabilities must be determined dynamically at runtime, perhaps as part of the task specification itself. One implication of this fact is that our system should allow the definition, construction, and execution of specialized agents at runtime. We realize our goal of dynamic agents at runtime with an agent capability and preset abstraction. This provides a useful abstraction for the manager to reason about how to complete subtasks correctly despite remaining a general-purpose component. Certain tasks may require specialized tools or processing steps which the manager can identify and leverage. Runtime capabilities are discovered through an externally hosted Capability Registry which exposes diverse software functionalities (capabilities) that can be used during subtask execution. These capabilities enable dynamic, composable functionalities for workers. To create agents using capabilities from the registry, the task manager agent creates an agent preset that defines a system prompt and a set of capabilities for the worker agent to use. When delegating subtasks, the manager agent specifies which preset to use. 3.3
APWA implementation
Figure 2 depicts the architecture of our system using the abstractions described above. At runtime, APWA processes a series of rounds cycling between two phases: a high-level reasoning and delegation phase performed by the manager and a low-level subtask execution phase performed by workers. Both the manager and the worker are implemented as agents running in sandboxed environments. Manager environment The manager’s core responsibility is to oversee the successful completion of a single task. The manager does this by planning out appropriate processing steps, observing and manipulating local data representations, and dispatching units of work to be executed remotely. The manager keeps track of progress of each subtask, by maintaining a highly-compact metadata representation of the global system data, a collection of workers, an actively-maintained execution plan, a detailed output format specification, and a log of results from previous execution rounds. The manager also maintains its internal execution state in an external state store that records task metadata, manager checkpoints, issued subtasks, subtask outcomes, and synthesized results. This store serves as the primary interface through which the manager interacts with system state, allowing it to retrieve outputs from completed subtask run and use them in analysis or re-planning. Each subtask run contributes structured updates to this state, including status indicators, metrics, and artifact references. Worker environment The worker’s execution environment is equipped with whatever auxiliary runtime capabilities are needed to complete the task, such as virtual machine resources (local file system, terminal capabilities, etc.) or network-accessed services. The worker has tools for reading and writing data to a shared network-accessible data resource. We observe that frequently agent execution patterns reduce to a simple LLM invocation. For example, simple data processing steps like summarization or structured information extraction involve only a single LLM generation step. In these cases, we can improve subtask throughput by a substantial factor (10-100×) by dynamically routing the subtask execution to a lightweight execution space. This routing is determined dynamically by the manager when it emits a subtask based on its own assessment of the complexity of the subtask. Scalable fabric for executor We use the Ray distributed computing framework [39] to provide APWA’s distributed computing fabric. Ray provides a task-parallel programming model and system 6
implementation that scales efficiently to large-scale distributed computing resources. Ray features a distributed scheduler and can scale to hundreds of thousands of concurrent tasks and a throughput of millions of tasks per second. Multiple workers can be deployed on a given node. When scaling to large numbers of subtasks, the probability that at least one failure occurs increases dramatically. Many of these errors are transient (e.g., a temporary network partition) and not critical errors that must be resolved at the logical layer. Our execution layer automatically retries subtasks up to a fixed number of times before returning a logical failure to the task manager. This behavior allows the task manager to more reliably reason about the logical content of its subtasks rather than low-level behaviors arising from the unreliable distributed system.
4
Evaluation
4.1
Main experiments
Benchmarks We evaluate APWA on two public benchmarks, the AI4Privacy PII-300k dataset (PII-300k) [1] and a schema-driven structured content extraction benchmark (SchemaBench) [3], as well as a manually-collected hierarchical summarization benchmark (SummaryBench). We present summarized results here and provide full results, with variance measurements, in Appendix E. The goal of PII-300k is to redact sensitive fields in a large number of unstructured records, with labels from 27 PII categories targeting domains spanning education, health, and psychology. The goal of SchemaBench is to extract valid and correct JSON outputs given a collection of documents from heterogeneous domains and data formats, including LATEX, XML, CSV, and HTML. These first two tasks represent highly data-parallelizable processing tasks, as might be performed with standard distributed data-flow execution engines. In both cases, ML-based methods are preferable to heuristic rule-based approaches due to the rich semantic nature of the task. The goal of hierarchical summarization is to process a hierarchically-organized data corpus and generate summaries at different levels of granularity matching the granularity of the original data’s organization pattern. For hierarchical summarization, we collect three literary corpora of varying sizes: Romeo and Juliet [53], The Dynasts [19], and The History of the Decline and Fall of the Roman Empire [17]. We obtain the source texts in plain text format from Project Gutenberg [46] and manually parse and format each one into an appropriate organization pattern. The data we use for SummaryBench spans from small (2 tiers, 166 kB) to moderate (3 tiers, 942 kB) to large (4 tiers, 10.5 MB), with statistics on each tier provided in Table 5. The hierarchical summarization task represents a more complex parallelization topology which requires issuing multiple rounds of subtasks to be completed in parallel and combining results from previous rounds. Together, these benchmarks measure the ability of APWA to automatically decompose a complex task a parallelizable workflow, execute that workflow, reason over progress and results, and synthesize a comprehensive answer. Metrics For each benchmark, we define an automated evaluation procedure to score the effectiveness of APWA along the dimensions of utility (whether the system produces a correct output) and cost (wall-clock runtime, token usage, and monetary expense). In general, we decompose utility into two submetrics, structural (whether the outputs exist and are correctly formatted) and semantic (whether the outputs are correct). The precise definition of these submetrics is task dependent; due to space constraints we give brief descriptions here and defer detailed descriptions to Appendix E. For PII-300k, the structural score measures the correctness of the output table schema and number of emitted entries, while the semantic score measures the macro-averaged F1 score of detected PII instances across all PII categories. For SchemaBench, the structural score is similar to PII-300k, while the semantic score reports the same metrics per extraction task as the original work [3], typically a micro- or macro-F1 score. For SummaryBench, we report the mean layer-wise ROUGE1-F1 score averaged across trials against a set of reference summaries. Baselines For every benchmark, we consider three baselines. First, we process the entire task request, including any necessary data formatted in plain text, directly with an LLM. In this scenario, we request the LLM to generate its response subject to a task-specific schema using the structured generation modes supported by recent LLM inference engines. 7
Table 1: Failure rates and wall-clock runtimes for SummaryBench and PII-300k. APWA leverages parallelization to efficiently solve tasks with increasingly large input sizes. All settings use GPT-5.4 mini, except MegaAgent using GPT-4.1 mini. Wall-clock runtimes reflected only for trials that produced output. FR = failure rate, WC = wall-clock time (seconds). R&J Method
Dynasts
Roman
PII-64
PII-512
PII-4096
FR (↓) WC (s) (↓) FR (↓) WC (s) (↓) FR (↓) WC (s) (↓) FR (↓) WC (s) (↓) FR (↓) WC (s) (↓) FR (↓) WC (s) (↓)
Direct 0% Magentic-One 100% MegaAgent 80%
19 ⊥ 472
60% 100% 80%
76 ⊥ 248
100% 100% 70%
⊥ ⊥ 579
0% 100% 60%
37 ⊥ 390
0% 100% 80%
41 ⊥ 25
100% 80% 70%
⊥ 91 372
APWA
157
0%
210
0%
329
0%
35
0%
67
0%
221
0%
Table 2: Structural and semantic scores for SummaryBench and PII-300k. APWA preserves structural and semantic integrity with increasingly large input sizes. All settings use GPT-5.4 mini, except MegaAgent using GPT-4.1 mini. Scores computed only for trials that produced output. Str. = structural score, Sem. = semantic score. R&J
Dynasts
Roman
PII-64
PII-512
PII-4096
Method
Str. (↑) Sem. (↑) Str. (↑) Sem. (↑) Str. (↑) Sem. (↑) Str. (↑) Sem. (↑) Str. (↑) Sem. (↑) Str. (↑) Sem. (↑)
Direct Magentic-One MegaAgent
1.000 ⊥ 0.140
0.433 ⊥ 0.043
0.979 ⊥ 0.212
0.210 ⊥ 0.023
⊥ ⊥ 0.160
⊥ ⊥ 0.016
1.000 ⊥ 0.375
0.775 ⊥ 0.000
0.750 ⊥ 0.000
0.162 ⊥ 0.000
⊥ 1.000 0.250
⊥ 0.179 0.000
APWA
0.954
0.424
0.954
0.419
0.919
0.232
1.000
0.759
1.000
0.772
0.900
0.544
Second, we use the Magentic-One multi-agent system [16] to process the query. Magentic-One represents a state-of-the-art multi-agent scaffold with built-in file system, web browser, and coding agent capabilities, as well as a dedicated planning and agent routing module. Magentic-One executes worker agents serially with coordination handled by a dedicated orchestrator agent. Because tasks are completed by interacting with tools and data file system, Magentic-One is better able to process larger data scales than naïvely submitting the full file contents to LLM inference. Third, we compare with the MegaAgent [57] multi-agent system. MegaAgent is designed as a general-purpose multi-agent architecture with considerations for parallel agent execution. As a multi-agent system, MegaAgent is logically organized into a hierarchical coordination structure. A single boss agent initiates task execution, and can spawn teams of subagents to help partition work. This process continues recursively with privileged subagents themselves spawning subteams until a subtask is complete or a recursion limit is reached. System configuration For our main experiments, we run APWA on a single machine running Ubuntu 22.04.5 and equipped with an AMD Ryzen Threadripper PRO 5955WX 16-Core processor, 252 GB memory, and 2x NVIDIA GeForce RTX 4090 GPUs. We experiment using different combinations of GPT-5.4, GPT-5.4-mini, and GPT-5.4-nano [42, 43] for the manager agent and worker LLM backends. We use the same models to evaluate the baseline methods. Results Table 1 shows task completion rates and wall clock runtimes for SummaryBench and PII-300k, and Table 2 reports output quality scores for SummaryBench and PII-300k. We find that failure rates increase and output quality decreases for all baselines as the number of processed documents increases. Direct LLM use solves the task for small inputs (on Romeo and Juliet and PII-300k up to 512 records), but fails on larger inputs (on Roman and PII-300k with 4096 records). Magentic-One is unable to complete any of the tasks reliably, only managing to produce an output in a single trial for PII-300k with all 4096 documents. We identify two failure modes in Magentic-One: orchestration failure and context explosion. In general, Magentic-One attempted to solve the task using programmatic methods rather than raw LLM processing. Orchestration failure occurred when the Magentic-One orchestrator incorrectly routed requests between its coding and terminal agents, causing both subagents to fail to complete any useful work. Context explosion occurred on larger inputs: for the Roman corpus, Magentic-One’s FileSurfer agent tried to page through the entire list of over 2,500 input documents and consumed over 6.5M input tokens, blocking progress until a timeout at 30 minutes. MegaAgent experienced different scaling and coordination difficulties. In one summa8
Table 3: Comparison of APWA configurations on SummaryBench. Str. = structural score, Sem. = semantic score, WT = wall-clock time (sec), Cost = total cost. R&J
Dynasts
Roman
Method
Str. (↑) Sem. (↑) WT (s) (↓) Cost (↓) Str. (↑) Sem. (↑) WT (s) (↓) Cost (↓) Str. (↑) Sem. (↑) WT (s) (↓) Cost (↓)
APWA (5.4 × mini) APWA (5.4 × nano) APWA (mini × mini) APWA (mini × nano)
1.000 0.943 0.897 0.916
0.528 0.439 0.394 0.408
152 157 94 96
$0.628 $0.582 $0.294 $0.225
0.997 0.923 0.954 0.803
0.451 0.419 0.419 0.314
212 211 134 149
$1.16 $0.919 $0.728 $0.404
0.983 0.872 0.817 0.853
0.370 0.395 0.207 0.283
336 352 235 265
$6.57 $3.09 $5.24 $2.08
rization task run, MegaAgent proceeds by creating a primary task orchestrator agent, which partitions work between three subagents (“SceneSummarizer”, “ActSummarizer”, and “FullSummarizer”). The SceneSummarizer fails to finish processing the scene files quickly, and the orchestrator prematurely terminates the task. In other runs, the agents produce large output files with incorrectly formatted JSON, causing the agent harness to crash. In contrast, APWA scales naturally to larger and more complex corpora, preserving better output fidelity as well as completing large tasks in a reasonable amount of time. Despite the size of input data increasing by nearly two orders of magnitude (166 kB on Romeo and Juliet → 10 500 kB on Roman), APWA’s wall-clock runtime only increases moderately (157 s → 329 s) thanks to massive parallelization, running over 2.5k agents concurrently. Prior methods fail to generate results at all. Table 3 shows that APWA with different model configurations is able to process increasingly more complex tasks–growing in both raw input size and number of required processing steps–while effectively leveraging parallelization to keep runtime low. The use of larger models during planning (GPT-5.4) provides an advantage on the system’s utility, achieving higher structural and semantic scores, at marginal cost increase. Our qualitative findings on SchemaBench given in Appendix E.3 are similar to our findings on SummaryBench and PII-300k for both runtime and output quality. On the only dataset whose size is small enough to fit within the 1M token limit for GPT-5.4 mini, C HEMISTRY, the semantic scores for APWA are comparable to the scores for querying the LLM directly. For all other datasets, the input size exceeds the context window of the considered LLMs when formatted directly. Magentic-One encounters similar orchestration failures, and only manages to produce output in 4 out of 20 cases. Like in SummaryBench and PII-300k, Magentic-One pursues programmatic solutions that do not meet the complex semantic nature of the task. MegaAgent also fails to complete the structured extraction task, again due to a failure to parallelize work and instead attempt purely role-based task decomposition. Out of 20 attempted tasks, only 6 produced any output, only 3 of which received nonzero scores. Of these, the semantic score is always 0. 4.2
Web browsing experiment
To demonstrate the utility of APWA in orchestrating fully agentic workloads, we consider a technical report synthesis task leveraging web browsing agents. The task definition is to generate textual reports on a list of topics provided in the input. We consider two variants with a small (10), medium (20), and large (100) number of topics, respectively. We populate the capability registry with Magentic-One’s web surfer agent and expose its description to the manager agent. We measure whether the correct number of (topic, report) pairs are generated and use LLM-as-a-judge to measure whether the report is factual and on-topic. In experiments, APWA successfully identifies the web surfer agent as relevant and creates an appropriate subtask template to pass input topics to workers. APWA always achieves structural and semantic scores above 0.975. APWA incurred an execution time of 143 s, 157 s, and 595 s for 10, 20, and 100 topics, respectively. These results demonstrate APWA’s ability to leverage agents within a parallelizable workflow to complete a complex task efficiently. Full results are given in Appendix E.4.
5
Conclusion
In this work, we propose APWA, a highly scalable multi-agent execution architecture for parallelizable agentic workflows. APWA introduces efficient abstractions for LLM-based agents to reason about and act upon high volumes of distributed data and computing resources and to specify and issue 9
large-scale parallel execution of agents. We demonstrate that APWA is capable of automatically and efficiently solving large-scale distributed workflows, in contrast to existing methods which cannot effectively leverage parallelism and fail on larger complex tasks.
Acknowledgments and Disclosure of Funding This research was supported by funding from Google and Good Ventures foundation.
References [1] Ai4Privacy. PII masking 300k dataset. https://huggingface.co/datasets/ai4privacy /pii-masking-300k, 2024. [2] Apache Software Foundation. Hadoop. https://hadoop.apache.org/. [3] Fan Bai, Junmo Kang, Gabriel Stanovsky, Dayne Freitag, Mark Dredze, and Alan Ritter. Schema-driven information extraction from heterogeneous tables. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen, editors, Findings of the Association for Computational Linguistics: EMNLP 2024, pages 10252–10273, Miami, Florida, USA, November 2024. Association for Computational Linguistics. [4] Daniil A. Boiko, Robert MacKnight, Ben Kline, and Gabe Gomes. Autonomous chemical research with large language models. Nature, 624(7992):570–578, December 2023. [5] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language models are few-shot learners. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 1877–1901. Curran Associates, Inc., 2020. [6] Jun Shern Chan, Neil Chowdhury, Oliver Jaffe, James Aung, Dane Sherburn, Evan Mays, Giulio Starace, Kevin Liu, Leon Maksin, Tejal Patwardhan, Aleksander Madry, and Lilian Weng. MLE-bench: Evaluating machine learning agents on machine learning engineering. In Y. Yue, A. Garg, N. Peng, F. Sha, and R. Yu, editors, International Conference on Learning Representations, volume 2025, pages 50466–50494, 2025. [7] Guangyao Chen, Siwei Dong, Yu Shu, Ge Zhang, Jaward Sesay, Börje Karlsson, Jie Fu, and Yemin Shi. AutoAgents: A framework for automatic agent generation. In Kate Larson, editor, Proceedings of the Thirty-Third International Joint Conference on Artificial Intelligence, IJCAI24, pages 22–30. International Joint Conferences on Artificial Intelligence Organization, 8 2024. Main Track. [8] Weize Chen, Yusheng Su, Jingwei Zuo, Cheng Yang, Chenfei Yuan, Chi-Min Chan, Heyang Yu, Yaxi Lu, Yi-Hsin Hung, Chen Qian, Yujia Qin, Xin Cong, Ruobing Xie, Zhiyuan Liu, Maosong Sun, and Jie Zhou. AgentVerse: Facilitating multi-agent collaboration and exploring emergent behaviors. In B. Kim, Y. Yue, S. Chaudhuri, K. Fragkiadaki, M. Khan, and Y. Sun, editors, International Conference on Learning Representations, volume 2024, pages 20094–20136, 2024. [9] CrewAI, Inc. CrewAI. https://www.crewai.com/. [10] Jeffrey Dean and Sanjay Ghemawat. MapReduce: Simplified data processing on large clusters. In 6th Symposium on Operating Systems Design and Implementation (OSDI 04), pages 137–150, San Francisco, CA, December 2004. USENIX Association. [11] Gelei Deng, Yi Liu, Víctor Mayoral-Vilches, Peng Liu, Yuekang Li, Yuan Xu, Tianwei Zhang, Yang Liu, Martin Pinzger, and Stefan Rass. PentestGPT: Evaluating and harnessing large language models for automated penetration testing. In 33rd USENIX Security Symposium (USENIX Security 24), pages 847–864, Philadelphia, PA, August 2024. USENIX Association. 10
[12] Xiang Deng, Yu Gu, Boyuan Zheng, Shijie Chen, Sam Stevens, Boshi Wang, Huan Sun, and Yu Su. Mind2web: Towards a generalist agent for the web. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, editors, Advances in Neural Information Processing Systems, volume 36, pages 28091–28114. Curran Associates, Inc., 2023. [13] Yihong Dong, Xue Jiang, Zhi Jin, and Ge Li. Self-collaboration code generation via chatgpt. ACM Trans. Softw. Eng. Methodol., 33(7), September 2024. [14] Yilun Du, Shuang Li, Antonio Torralba, Joshua B. Tenenbaum, and Igor Mordatch. Improving factuality and reasoning in language models through multiagent debate. In Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Felix Berkenkamp, editors, Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pages 11733–11763. PMLR, 21–27 Jul 2024. [15] Nouha Dziri, Ximing Lu, Melanie Sclar, Xiang (Lorraine) Li, Liwei Jiang, Bill Yuchen Lin, Sean Welleck, Peter West, Chandra Bhagavatula, Ronan Le Bras, Jena Hwang, Soumya Sanyal, Xiang Ren, Allyson Ettinger, Zaid Harchaoui, and Yejin Choi. Faith and fate: Limits of transformers on compositionality. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, editors, Advances in Neural Information Processing Systems, volume 36, pages 70293–70332. Curran Associates, Inc., 2023. [16] Adam Fourney, Gagan Bansal, Hussein Mozannar, Cheng Tan, Eduardo Salinas, Erkang (Eric) Zhu, Friederike Niedtner, Grace Proebsting, Griffin Bassman, Jack Gerrits, Jacob Alber, Peter Chang, Ricky Loynd, Robert West, Victor Dibia, Ahmed Awadallah, Ece Kamar, Rafah Hosn, and Saleema Amershi. Magentic-One: A generalist multi-agent system for solving complex tasks. Technical Report MSR-TR-2024-47, Microsoft, November 2024. [17] Edward Gibbon. The History of the Decline and Fall of the Roman Empire. Project Gutenberg, Urbana, Illinois, 2008. Accessed: 2026-04-27. [18] Akash Gupta, Ivaxi Sheth, Vyas Raina, Mark Gales, and Mario Fritz. LLM task interference: An initial study on the impact of task-switch in conversational history. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen, editors, Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 14633–14652, Miami, Florida, USA, November 2024. Association for Computational Linguistics. [19] Thomas Hardy. The Dynasts: An Epic-Drama of the War with Napoleon. Project Gutenberg, Urbana, Illinois, 2005. Accessed: 2026-04-27. [20] Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. In International Conference on Learning Representations, 2021. [21] Sirui Hong, Mingchen Zhuge, Jonathan Chen, Xiawu Zheng, Yuheng Cheng, Jinlin Wang, Ceyao Zhang, Zili Wang, Steven Ka Shing Yau, Zijuan Lin, Liyang Zhou, Chenyu Ran, Lingfeng Xiao, Chenglin Wu, and Jürgen Schmidhuber. MetaGPT: Meta programming for a multi-agent collaborative framework. In The Twelfth International Conference on Learning Representations, 2024. [22] Cheng-Ping Hsieh, Simeng Sun, Samuel Kriman, Shantanu Acharya, Dima Rekesh, Fei Jia, and Boris Ginsburg. RULER: What’s the real context size of your long-context language models? In First Conference on Language Modeling, 2024. [23] Mengkang Hu, Yuhang Zhou, Wendong Fan, Yuzhou Nie, Ziyu Ye, Bowei Xia, Tao Sun, Zhaoxuan Jin, Yingru Li, Zeyu Zhang, Yifeng Wang, Qianshuo Ye, Bernard Ghanem, Ping Luo, and Guohao Li. Owl: Optimized workforce learning for general multi-agent assistance in realworld task automation. In D. Belgrave, C. Zhang, H. Lin, R. Pascanu, P. Koniusz, M. Ghassemi, and N. Chen, editors, Advances in Neural Information Processing Systems, volume 38, pages 50859–50906. Curran Associates, Inc., 2025. 11
[24] Jie Huang, Xinyun Chen, Swaroop Mishra, Huaixiu Steven Zheng, Adams Yu, Xinying Song, and Denny Zhou. Large language models cannot self-correct reasoning yet. In B. Kim, Y. Yue, S. Chaudhuri, K. Fragkiadaki, M. Khan, and Y. Sun, editors, International Conference on Learning Representations, volume 2024, pages 32808–32824, 2024. [25] Michael Isard, Mihai Budiu, Yuan Yu, Andrew Birrell, and Dennis Fetterly. Dryad: distributed data-parallel programs from sequential building blocks. In Proceedings of the 2nd ACM SIGOPS/EuroSys European Conference on Computer Systems 2007, EuroSys ’07, pages 59–72, New York, NY, USA, 2007. Association for Computing Machinery. [26] Carlos E Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik Narasimhan. SWE-bench: Can language models resolve real-world github issues? In B. Kim, Y. Yue, S. Chaudhuri, K. Fragkiadaki, M. Khan, and Y. Sun, editors, International Conference on Learning Representations, volume 2024, pages 54107–54157, 2024. [27] Sehoon Kim, Suhong Moon, Ryan Tabrizi, Nicholas Lee, Michael W. Mahoney, Kurt Keutzer, and Amir Gholami. An LLM compiler for parallel function calling. In Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Felix Berkenkamp, editors, Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pages 24370–24391. PMLR, 21–27 Jul 2024. [28] Yubin Kim, Chanwoo Park, Hyewon Jeong, Yik Siu Chan, Xuhai Xu, Daniel McDuff, Hyeonhoon Lee, Marzyeh Ghassemi, Cynthia Breazeal, and Hae Won Park. Mdagents: An adaptive collaboration of llms for medical decision-making. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang, editors, Advances in Neural Information Processing Systems, volume 37, pages 79410–79452. Curran Associates, Inc., 2024. [29] Langchain, Inc. LangChain. https://github.com/langchain-ai/langchain/. [30] Hwiwon Lee, Ziqi Zhang, Hanxiao Lu, and Lingming Zhang. SEC-bench: Automated benchmarking of llm agents on real-world software security tasks. In D. Belgrave, C. Zhang, H. Lin, R. Pascanu, P. Koniusz, M. Ghassemi, and N. Chen, editors, Advances in Neural Information Processing Systems, volume 38, pages 116342–116378. Curran Associates, Inc., 2025. [31] Guohao Li, Hasan Hammoud, Hani Itani, Dmitrii Khizbullin, and Bernard Ghanem. CAMEL: Communicative agents for "mind" exploration of large language model society. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, editors, Advances in Neural Information Processing Systems, volume 36, pages 51991–52008. Curran Associates, Inc., 2023. [32] Tian Liang, Zhiwei He, Wenxiang Jiao, Xing Wang, Yan Wang, Rui Wang, Yujiu Yang, Shuming Shi, and Zhaopeng Tu. Encouraging divergent thinking in large language models through multi-agent debate. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen, editors, Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 17889–17904, Miami, Florida, USA, November 2024. Association for Computational Linguistics. [33] Nelson F. Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. Lost in the middle: How language models use long contexts. Transactions of the Association for Computational Linguistics, 12:157–173, 2024. [34] Andres M. Bran, Sam Cox, Oliver Schilter, Carlo Baldassari, Andrew D. White, and Philippe Schwaller. Augmenting large language models with chemistry tools. Nature Machine Intelligence, 6(5):525–535, May 2024. [35] Grzegorz Malewicz, Matthew H. Austern, Aart J.C Bik, James C. Dehnert, Ilan Horn, Naty Leiser, and Grzegorz Czajkowski. Pregel: a system for large-scale graph processing. In Proceedings of the 2010 ACM SIGMOD International Conference on Management of Data, SIGMOD ’10, pages 135–146, New York, NY, USA, 2010. Association for Computing Machinery. [36] Ruijie Meng, Martin Mirchev, Marcel Böhme, and Abhik Roychoudhury. Large language model guided protocol fuzzing. In Proceedings 2024 Network and Distributed System Security Symposium, NDSS 2024. Internet Society, 2024. 12
[37] Microsoft Corporation. Microsoft agent framework. https://github.com/microsoft/age nt-framework/. [38] Model Context Protocol a Series of LF Projects, LLC. Model context protocol (MCP). https: //github.com/modelcontextprotocol/modelcontextprotocol. [39] Philipp Moritz, Robert Nishihara, Stephanie Wang, Alexey Tumanov, Richard Liaw, Eric Liang, Melih Elibol, Zongheng Yang, William Paul, Michael I. Jordan, and Ion Stoica. Ray: A distributed framework for emerging AI applications. In 13th USENIX Symposium on Operating Systems Design and Implementation (OSDI 18), pages 561–577, Carlsbad, CA, October 2018. USENIX Association. [40] Derek G. Murray, Malte Schwarzkopf, Christopher Smowton, Steven Smith, Anil Madhavapeddy, and Steven Hand. CIEL: A universal execution engine for distributed Data-Flow computing. In 8th USENIX Symposium on Networked Systems Design and Implementation (NSDI 11), Boston, MA, March 2011. USENIX Association. [41] Boye Niu, Yiliao Song, Kai Lian, Yifan Shen, Yu Yao, Kun Zhang, and Tongliang Liu. Flow: Modularized agentic workflow automation. In Y. Yue, A. Garg, N. Peng, F. Sha, and R. Yu, editors, International Conference on Learning Representations, volume 2025, pages 74949– 74977, 2025. [42] OpenAI. Introducing GPT-5.4. https://openai.com/index/introducing-gpt-5-4/, 2025. Accessed: 2026-04-27. [43] OpenAI. Introducing GPT-5.4 mini and nano. https://openai.com/index/introducing -gpt-5-4-mini-and-nano/, 2025. Accessed: 2026-04-27. [44] OpenAI. Openai agents sdk. https://github.com/openai/openai-agents-python, 2026. Accessed: 2026-05-05. [45] Russell Power and Jinyang Li. Piccolo: Building fast, distributed programs with partitioned tables. In 9th USENIX Symposium on Operating Systems Design and Implementation (OSDI 10), Vancouver, BC, October 2010. USENIX Association. [46] Project Gutenberg. Project gutenberg. https://www.gutenberg.org/. Accessed: 2026-0427. [47] Zehan Qi, Xiao Liu, Iat Long Iong, Hanyu Lai, Xueqiao Sun, Jiadai Sun, Xinyue Yang, Yu Yang, Shuntian Yao, Wei Xu, Jie Tang, and Yuxiao Dong. Webrl: Training llm web agents via self-evolving online curriculum reinforcement learning. In Y. Yue, A. Garg, N. Peng, F. Sha, and R. Yu, editors, International Conference on Learning Representations, volume 2025, pages 79791–79821, 2025. [48] Chen Qian, Wei Liu, Hongzhang Liu, Nuo Chen, Yufan Dang, Jiahao Li, Cheng Yang, Weize Chen, Yusheng Su, Xin Cong, Juyuan Xu, Dahai Li, Zhiyuan Liu, and Maosong Sun. ChatDev: Communicative agents for software development. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors, Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 15174–15186, Bangkok, Thailand, August 2024. Association for Computational Linguistics. [49] Chen Qian, Zihao Xie, YiFei Wang, Wei Liu, Kunlun Zhu, Hanchen Xia, Yufan Dang, Zhuoyun Du, Weize Chen, Cheng Yang, Zhiyuan Liu, and Maosong Sun. Scaling large language modelbased multi-agent collaboration. In Y. Yue, A. Garg, N. Peng, F. Sha, and R. Yu, editors, International Conference on Learning Representations, volume 2025, pages 41488–41505, 2025. [50] Shuofei Qiao, Runnan Fang, Zhisong Qiu, Xiaobin Wang, Ningyu Zhang, Yong Jiang, Pengjun Xie, Fei Huang, and Huajun Chen. Benchmarking agentic workflow generation. In Workshop on Reasoning and Planning for Large Language Models, 2025. [51] Matthew Rocklin. Dask: Parallel computation with blocked algorithms and task scheduling. In Kathryn Huff and James Bergstra, editors, Proceedings of the 14th Python in Science Conference, pages 126–132, 2015. 13
[52] Timo Schick, Jane Dwivedi-Yu, Roberto Dessi, Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. Toolformer: Language models can teach themselves to use tools. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, editors, Advances in Neural Information Processing Systems, volume 36, pages 68539–68551. Curran Associates, Inc., 2023. [53] William Shakespeare. Romeo and Juliet. Project Gutenberg, Urbana, Illinois, 1998. Accessed: 2026-04-27. [54] Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: language agents with verbal reinforcement learning. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, editors, Advances in Neural Information Processing Systems, volume 36, pages 8634–8652. Curran Associates, Inc., 2023. [55] Patara Trirat, Wonyong Jeong, and Sung Ju Hwang. AutoML-agent: A multi-agent LLM framework for full-pipeline AutoML. In Aarti Singh, Maryam Fazel, Daniel Hsu, Simon LacosteJulien, Felix Berkenkamp, Tegan Maharaj, Kiri Wagstaff, and Jerry Zhu, editors, Proceedings of the 42nd International Conference on Machine Learning, volume 267 of Proceedings of Machine Learning Research, pages 60099–60146. PMLR, 13–19 Jul 2025. [56] Junlin Wang, Jue Wang, Ben Athiwaratkun, Ce Zhang, and James Y Zou. Mixture-of-agents enhances large language model capabilities. In Y. Yue, A. Garg, N. Peng, F. Sha, and R. Yu, editors, International Conference on Learning Representations, volume 2025, pages 33944– 33963, 2025. [57] Qian Wang, Tianyu Wang, Zhenheng Tang, Qinbin Li, Nuo Chen, Jingsheng Liang, and Bingsheng He. MegaAgent: A large-scale autonomous LLM-based multi-agent system without predefined SOPs. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar, editors, Findings of the Association for Computational Linguistics: ACL 2025, pages 4998–5036, Vienna, Austria, July 2025. Association for Computational Linguistics. [58] Xingyao Wang, Boxuan Li, Yufan Song, Frank F Xu, Xiangru Tang, Mingchen Zhuge, Jiayi Pan, Yueqi Song, Bowen Li, Jaskirat Singh, Hoang Tran, Fuqiang Li, Ren Ma, Mingzhang Zheng, Bill Qian, Daniel Shao, Niklas Muennighoff, Yizhe Zhang, Binyuan Hui, Junyang Lin, Robert Brennan, Hao Peng, Heng Ji, and Graham Neubig. Openhands: An open platform for ai software developers as generalist agents. In Y. Yue, A. Garg, N. Peng, F. Sha, and R. Yu, editors, International Conference on Learning Representations, volume 2025, pages 65882–65919, 2025. [59] Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Beibin Li, Erkang Zhu, Li Jiang, Xiaoyun Zhang, Shaokun Zhang, Jiale Liu, Ahmed Hassan Awadallah, Ryen W White, Doug Burger, and Chi Wang. AutoGen: Enabling next-gen LLM applications via multi-agent conversations. In First Conference on Language Modeling, 2024. [60] John Yang, Carlos Jimenez, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik Narasimhan, and Ofir Press. SWE-agent: Agent-computer interfaces enable automated software engineering. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang, editors, Advances in Neural Information Processing Systems, volume 37, pages 50528–50652. Curran Associates, Inc., 2024. [61] Kaiyu Yang, Aidan Swope, Alex Gu, Rahul Chalamala, Peiyang Song, Shixing Yu, Saad Godil, Ryan J Prenger, and Animashree Anandkumar. LeanDojo: Theorem proving with retrievalaugmented language models. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, editors, Advances in Neural Information Processing Systems, volume 36, pages 21573–21612. Curran Associates, Inc., 2023. [62] Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. In The Eleventh International Conference on Learning Representations, 2023. [63] Yangyang Yu, Zhiyuan Yao, Haohang Li, Zhiyang Deng, Yuechen Jiang, Yupeng Cao, Zhi Chen, Jordan W. Suchow, Zhenyu Cui, Rong Liu, Zhaozhuo Xu, Denghui Zhang, Koduvayur 14
Subbalakshmi, Guojun Xiong, Yueru He, Jimin Huang, Dong Li, and Qianqian Xie. Fincon: A synthesized llm multi-agent system with conceptual verbal reinforcement for enhanced financial decision making. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang, editors, Advances in Neural Information Processing Systems, volume 37, pages 137010–137045. Curran Associates, Inc., 2024. [64] Matei Zaharia, Mosharaf Chowdhury, Tathagata Das, Ankur Dave, Justin Ma, Murphy McCauly, Michael J. Franklin, Scott Shenker, and Ion Stoica. Resilient distributed datasets: A FaultTolerant abstraction for In-Memory cluster computing. In 9th USENIX Symposium on Networked Systems Design and Implementation (NSDI 12), pages 15–28, San Jose, CA, April 2012. USENIX Association. [65] Boyuan Zheng, Boyu Gou, Jihyung Kil, Huan Sun, and Yu Su. GPT-4V(ision) is a generalist web agent, if grounded. In Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Felix Berkenkamp, editors, Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pages 61349–61385. PMLR, July 2024. [66] Mingchen Zhuge, Wenyi Wang, Louis Kirsch, Francesco Faccio, Dmitrii Khizbullin, and Jürgen Schmidhuber. GPTSwarm: Language agents as optimizable graphs. In Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Felix Berkenkamp, editors, Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pages 62743–62767. PMLR, 21–27 Jul 2024.
A
Limitations
While APWA makes important progress towards parallelized agentic task processing, we recognize some of its limitations. First, the architecture does not support direct communication between workers as workers interact exclusively with managers, which limits applicability to workflows requiring coordination among subtasks. Second, APWA’s task planning provides guidance for using several parallelization patterns and templates. However, we have not evaluated whether APWA generalizes naturally to other patterns. Third, security and privacy concerns were not considered in the current design. APWA’s increased autonomy and throughput may lead to vulnerabilities from prompt injection attacks, malicious tools, and compromised workers, as well as risks of data leakage. Future work should systematically analyze APWA’s attack surface and develop guardrails for secure deployment.
B
Broader impacts
APWA enables multi-agent systems to efficiently process complex, large-scale tasks. By enabling dynamic workflow decomposition and parallel execution across heterogeneous data and processing patterns, APWA can accelerate applications in different domains where current multi-agent systems fail to scale. We envision positive impacts to society if APWA is deployed in high-stake domains such as health care that require large-scale data processing. As with any system that increases the autonomy and scale of LLM-based agents, APWA carries potential misuse risks. The same parallelization capabilities that enable legitimate high-throughput workflows could be exploited to automate harmful activities at scale, such as coordinated cyber attacks, large-scale phishing, or spam generation. Mitigating these threats requires future research on deploying safety guardrails, as well as defining and enforcing policies on the system’s use.
C
Related work
Distributed execution architectures [10, 2, 25, 45, 35, 40, 64, 51, 39] have studied ways to leverage distributed computing infrastructure for the efficient execution of data- and task-parallel workflows. Notably, MapReduce[10] introduces a programming model and processing engine capable of handling 15
massive datasets in parallel across distributed, commodity hardware. The innovation of MapReduce spurred heightened research into distributed execution architectures that build on top of the idea and improve on it by using abstractions to model the workflows [2, 64, 39]. LLM-based multi-agent systems are evolving along several threads. General purpose orchestration substrates expose conversational, role-based or graph-based abstractions to compose co-operating agents [59, 21, 31, 66], a line increasingly mirrored by deployed industry frameworks and systems [9, 44, 37, 16]. Building on these substrates, role-specialized teams emulate human workflows in domains such as SOP-driven software engineering [48, 13]. A separate thread improves reasoning through debate and consensus [14, 32], while others push towards scalability by dynamically generating agents without predefined SOPs [57, 7, 23].
D
APWA implementation details
In this appendix, we give a detailed description of all system components, core distributed computing abstractions, and external resources used by APWA and introduced in Section 3. First, we cover the core system-level abstractions presented in Section 3.1: the Task Manager (Appendix D.1), the Worker (Appendix D.2), and the Executor (Appendix D.3). Then, we provide a detailed description of the agentic abstractions from Section 3.2: the parallel planning abstraction (Appendix D.4), the subtask delegation abstraction (Appendix D.5), the data table abstraction (Appendix D.6), and the dynamic agent capability and preset abstraction (Appendix D.7). Finally, we discuss practical implementation details for realizing APWA using scalable infrastructure, including data storage services (Appendix D.8), and considerations for computing cluster resource management (Appendix D.9). D.1
Task manager
The Task Manager (or just Manager) is the system component responsible for overseeing the life cycle of a single user task. It receives a task from the orchestrator enclosing the user query expressed in natural language and any pre-configured environment state in the form of references to external data located in the object store. D.1.1
Manager agent
The core operative backbone of the Manager is its namesake manager agent. The logical role of the manager agent is conceptually similar to other hierarchically-organized multi-agent system architectures like Magentic-One [16] and MegaAgent [57], in that the manager agent is responsible for high-level planning, reasoning over intermediate global state, and coordinating subordinate subagents in order to make progress towards task completion. In addition to these roles, we also design the manager agent to be tightly integrated with our core system abstractions. These abstractions facilitate agent-based reasoning for parallel processing workflows over distributed computing resources. Agent organization The per-round exploration phase, performed by the manager agent, involves three distinct steps. The first, most substantial step, is the iterative tool-calling loop which the agent uses to inspect and interact with the environment. During this step, the manager agent plans, reads execution state, creates agent presets, manipulates data tables, and stages subtasks to make progress towards task processing. Following this step, a non-tool-using report generation step is performed to extract a structured object describing any accomplishments, next steps, or critical blockers from the round. This intermediate step is primarily for observability and debugging purposes. Finally, a delegation step validates the coherence of the current agent trajectory against the task, plan, and output contract and determines if the proposed system transition (either subtask batch emission or task finalization) is appropriate. In the case that the transition is inappropriate, control is returned to the first step. This final step is motivated by the agent debate design pattern [14, 32], and is intended to help detect and resolve erroneous or premature state transitions. LLM context formation To integrate our core abstractions with the manager agent, we must organize and format system state information into reasonably succinct textual representations to be fed into LLM-based components. Following existing agentic systems, we organize the system context as a chat-like transcript. 16
Starting with the manager agent’s system prompt, we aim to provide a sufficiently coherent system model and catalog of domain concepts to the LLM. In particular, we describe the APWA system and the manager agent’s operative role within it in the system prompt. Additionally, we describe all domain concepts, including the core APWA abstractions. Then, to better condition the model towards generating coherent parallelizable workflows, we provide several examples of generic parallel computation patterns and how to realize them. D.1.2
Manager agent tools
One of the crucial abstractions implemented in the task manager is the comprehensive tool suite designed to facilitate agent-based reasoning over distributed execution state. The tool suite conceptually consolidates all of our distributed agentic computation abstractions into a single unified programming interface connected to the Manager Agent. This tool suite extends the functionality of the base LLM agent to enable it to reason over parallelization patterns and inspect intermediate state in a way that carefully balances exploration with context efficiency. Table 4 lists all tools made available to the manager agent. D.2 D.2.1
Worker Logical execution modes
APWA supports two execution modes, a full agent mode and LLM-only mode. When using the full agent model, a dedicated, isolated execution space is created for the purpose of processing that individual subtask. We implement these isolated execution spaces using a collection of Docker containers attached to a shared Docker network, but other implementations are possible. When developing APWA, we recognized that many subtasks can be realized in the form of a single LLM query with some data attached. For example, text summarization fits this paradigm. Therefore, to improve subtask throughput dynamically based on expected subtask complexity, we add an LLM-only execution path. This path does not create the full execution environment and implements subtask execution by making a single LLM query, formatting the agent request and all data resources inline with the LLM context. This allows us to process larger sub-batches within a single worker process, which utilizes system resources more efficiently. The LLM-only execution path has the limitation that the worker agent cannot call tools or access a full computer environment, so it is only suitable for simple subtask types. D.2.2
Worker execution environment
All full agent mode subtasks execute in a Docker-managed execution sandbox. The sandbox consists of several Docker containers connected over a shared Docker network. The subtask spec should contain all information (configurations, task details, setup metadata) required to specify which containers to start, how to configure them, and how to connect them. The top-level component responsible for orchestrating the Docker environments is the “subtask manager.” The subtask manager is responsible for reading the subtask spec, starting the necessary containers, and wiring up the network connections between them. The subtask manager also monitors the health of the containers and handles any necessary cleanup after the subtask execution is complete. The subtask manager executes directly on the host machine alongside other subtask managers. When using the ray orchestration layer, the subtask manager implementation serves as the entrypoint for a ray task. The core container hosts a so-called "leader agent" which is responsible for overseeing the general lifecycle of the subtask. The leader agent directly interfaces with the subtask inputs and is equipped with tools that correspond with capabilities that originate from the capability registry and which are contained in the subtask spec. The final output of the subtask is written by the leader agent to a pre-specified output path. This output will be schema-validated by the subtask manager before propagating the result to the originating task manager executor. Within the context of a subtask execution fabric, a "helper agent" refers to a long-lived container (concurrent with the leader agent) that exposes a well-defined interface (details of which are included in the capability registry). These containers are created at subtask startup and persist for the entire 17
Table 4: Manager agent tools. These tools serve as the interface between the LLM and the key system abstractions discussed in Section 3.2, allowing the manager agent to inspect system state, manipulate data, and delegate subtasks. Tool
Description
list_tables get_table_meta
Lists available tables, grouped by table kind. Returns metadata for a table, either as a full schema/lineage view or a compact row-andcolumn summary. Shows a paginated preview of rows from a table, optionally selecting columns and ordering results. Retrieves one row from a table by row ID, optionally limited to selected columns. Returns rows from a table that match a structured predicate.
preview_rows get_row filter_rows distinct_values value_counts summarize_numeric groupby_aggregate sample_rows create_union_table create_filtered_table create_projected_table create_joined_table create_grouped_table rename_table archive_table unarchive_table rename_columns add_computed_columns create_results_with_source drop_columns
Lists distinct values for a field, along with counts, optionally under a predicate filter. Computes the top-k most frequent values for a field, with an optional predicate filter. Computes summary statistics for one or more numeric fields, optionally over filtered rows. Performs grouped aggregation with count, count-distinct, sum, average, min, or max aggregations. Draws a random sample of rows from a table, optionally selecting columns and using a seed. Creates a derived table by vertically concatenating multiple schema-compatible tables. Creates a derived table containing only rows from an input table that satisfy a predicate. Creates a derived table containing only a specified subset of columns. Creates a derived table by joining two input tables on specified key pairs. Creates a derived table by grouping rows and applying named first, count, or collect aggregations. Renames a table’s display name without changing its data or lineage. Archives a table so it is hidden from ordinary table listings while preserving lineage. Restores an archived table to the active namespace, optionally assigning a display name. Creates a derived table with selected columns renamed. Creates a derived table with new computed columns using cast, concat, coalesce, key extraction, or first-element extraction. Joins a batch results table back to its discovered source table using lineage columns. Creates a derived table with specified columns removed.
stage_single_subtask stage_dataset_subtask remove_staged_subtask clear_staged_subtasks list_subtasks get_subtask_result get_artifact list_artifacts
Adds one literal subtask definition to the staging area. Stages a batch of subtasks by applying a template to every row of a dataset table. Removes one staged subtask template. Clears all currently staged subtasks from the staging area. Lists executed subtasks with pagination and an optional status filter. Fetches detailed run results, metrics, status, and artifacts for a specific subtask ID. Retrieves artifact metadata and, optionally, truncated artifact content. Lists artifacts for the task, optionally filtered by subtask and optionally including previews.
write_plan write_output_contract
Replaces the manager plan with a new full plan payload. Replaces the task output contract and reports whether the current dataset satisfies it.
create_agent_preset list_agent_presets delete_agent_preset
Creates or updates an agent preset with a prompt and capability list. Lists all currently available agent presets. Deletes an agent preset by name.
duration of the subtask execution. These containers are therefore allowed to maintain state and execute long-running processes concurrently with the leader agent. Helper agents are not permitted to communicate with each other, and only have a line of communication with the leader agent. Helper agents are also not permitted to spawn other helper agents or helper tools. D.2.3
Worker agent
To simplify connection with external data resources and improve resource efficiency, the worker environment does not directly open a connection with the object store backend, but rather connects to a node-level object store proxy (details in Appendix D.9). 18
D.3 D.3.1
Executor Automatic retries
A well-studied challenge faced by distributed computation frameworks is how to seamlessly address transient resource failures such as network partitions [10, 64, 39]. A common solution is to simply retry subtasks that fail due to potentially transient errors. We adopt this approach at the subtask level: each subtask is attempted up to a maximum of 3 times. If any attempted execution succeeds, its output is persisted to stable storage and propagated through the batch result to the task manager. If all execution attempts fail, a logical subtask failure is forwarded instead to the task manager. This failure resolution is important for providing a clean logical layer to the manager agent. D.4
Planning for parallelization
A common design pattern in prior research [16] investigates how to use dedicated planning phases to improve LLM agent performance on long-horizon tasks. To encourage parallel workflow reasoning, we adapt this design pattern and explicitly model the planning process as a first-class system abstraction. We realize two forms of planning abstractions, each realized as local manager state maintained by the manager agent. The first is a local planning structure that encodes high-level parallelization strategy in a flexible, generic format. The second encodes a formal task contract that self-imposes global state preconditions gating task termination. Parallel planning structure We implement a simple planning abstraction to encourage parallelization-focused planning from the manager agent. Our parallel plan is designed to support APWA’s goal of automated task decomposition into parallelizable workflows. Thus, we define the plan structure with explicit consideration of the manager’s parallelization objectives. An APWA plan consists of two main fields: the partition strategy and a general-purpose list of steps. The partition strategy is intended to be populated with explicit data and task partition axes that eventually correspond to batches of subtasks that can be executed in parallel. The partition strategy is intended to represent a high-level parallelization strategy and be roughly fixed once established. To enable more granular planning controls, the planning structure also supports a general-purpose list of steps that identify the key procedural table manipulation and batch execution operations that will lead to task completion. These steps are annotated with agent-provided progress flags, allowing the manager agent to track progress towards individual subgoals. To clarify the distinguishing conceptual properties of our plan structure, we contrast our planning abstraction with the planning abstraction from Magentic-One [16]. In addition to facilitating generalpurpose planning, Magentic-One’s two-tiered planning mechanism is structured primarily around the organization of facts anticipated and encountered during agent execution, determining the immediate next agent invocation, detecting unproductive agent loops, and deciding appropriate termination conditions. This planning abstraction is well-suited for, e.g., knowledge gathering work, but does not facilitate dynamically-discovered parallelized workflows. In contrast, our planning abstraction explicitly models parallel task decomposition as a first-class consideration. The manager agent can update the plan structure at any time during its exploration phase using a dedicated tool (see Appendix D.1.2). Output contracts The manager’s plan object primarily functions as a free-form scratch space whose contents are not formalized into any hard execution requirements. Its main purpose is to refine the manager agent’s focus towards achieving the user task as well as to serve as a simple kind of agent memory. Depending on the task, it may also be possible to derive formal output specifications defining preconditions required for task completion. We implement this concept in the form of output contracts. Our implementation of output contracts is grounded in APWA’s data table abstraction. An output contract consists of a set of table specifications. A table specification defines a table that must exist before the task can be considered complete. Each table specification declares a name for the expected table, as well as (optionally) the column schema and number of rows the table is expected to have. Agent-generated output contracts are used in a few different ways during system execution. Most immediately, the task contract (and intermediate progress towards its fulfillment) is formatted into 19
the manager agent’s context window. We hypothesize that conditioning LLM generation on this formatted intermediate progress focuses execution towards higher-utility agent trajectories. Second, when the manager agent signals task completion, we directly validate the manager table state against the contract in a rule-based manner. If the contract is unfulfilled, the task manager’s control flow is returned to the manager agent to continue the exploration phase. Although simple, we find that simply encouraging the manager agent to formalize its task objectives as hard execution contracts can prevent premature task termination as well as improve output fidelity against user-specified formatting constraints. Like the free-form parallelization plan, the manager agent can update the output contract at any point during its exploration phase using a dedicated tool (see Appendix D.1.2). D.5
Subtask delegation
One of the novel challenges that arises from using LLM-based components to specify task decomposition into parallelizable workflows lies in how to simply specify that partitioning efficiently. Within standard (non-agentic) distributed execution engines, decomposition happens at a time scale typically orders of magnitude faster than the time taken In particular, it is relatively inexpensive for the programmer to specify thousands, or even millions of subtasks without noticeable overhead compared with the time taken to execute those subtasks. In contrast, with LLM-based delegation mechanisms, this dynamic is inverted. Defining a single subtask using a natural language specification can take on the order of seconds, introducing network and text generation overhead. To resolve this issue, we introduce subtask templates. Subtask templates provide a way for LLM-based components to efficiently specify a large number of subtasks that can be executed in parallel. D.6
Data tables
A core difficulty our architecture must grapple with is how to reason over large-scale data and distributed execution resources. For the types of tasks we consider, the data can be too large to fit on a single machine (making it impossible to store or examine all intermediate state on a single machine), and even the size of the metadata describing that data can quickly exhaust even frontier LLMs’ context windows. The key logical abstraction we introduce to facilitate agent-based reasoning over large volumes of data is that of a data table. A data table is a logical, read-only, finite sequence of records. A record is mapping from field names to typed values. All records in a dataset conform to a common schema, which fixes the set of typed fields. Types determine the admissible domain for field values. Various operations can be used to produce new datasets from existing ones, forming a table algebra. These operations derive from the standard relational algebra operations like joins, intersections, unions, and filters. Operators in this algebra are both pure (not modifying their inputs) and deterministic. Thus, sequences of nested operations form a declarative specification for how to construct a dataset from a collection of leaf datasets. All tables in our representation are either leaf tables grounded in a metadata-laden stable storage, or are constructed by application of operators in the algebra to one or more dependencies. Thus, all non-leaf tables are fully specified by their lineage. A lineage (graph) is a directed acyclic graph whose nodes denote operation applications, whose leaves denote base datasets, and whose edges describe data dependencies. Knowledge of the lineage graph of a dataset is sufficient to reconstruct it from its root dependencies, provided the leaf datasets are available. Our table abstraction has several key advantages. First, the table abstraction makes LLM-based reasoning about intermediate task states more tractable. Table metadata and summaries can be efficiently formatted to fit inside the context window of an LLM, while hiding high-volume data contents. An LLM-based component integrated with the table subsystem can efficiently specify highly data-parallel transformations, including local table manipulation as well Second, immutability simplifies coordination with other distributed system components. In particular, consistency of replicated shared objects is achieved without the need for convoluted consensus protocols. This property also makes it easier to recompute missing objects using lineage metadata. 20
We note some explicit restrictions of our abstract data model. It does not allow for efficient finegrained memory controls inexpressible with the supported algebraic manipulations. D.7
Capability registry
The Capability Registry is a service exposing diverse software functionalities that can be used during subtask execution. These capabilities enable dynamic, composable functionalities for workers. We assume the capability registry is implemented as a horizontally-scalable, network-accessible, external service. Capabilities are categorized based on their local execution model. Service capabilities inhabit longrunning execution spaces that persist for the entire duration of its associated subtask’s execution environment. Notably, service capabilities may perform work in parallel with the main subtask driver process. This property is useful for performing expensive, long-running computations concurrently with a subtask driver process or enabling state persistence. Tool capabilities are stateless interfaces whose execution duration is scoped only for a single invocation. Entries in the Capability Registry consist of metadata (capability name, unique identifier, natural language description) and implementation location (e.g., a Docker image). In our application, capabilities are discovered by the task manager during the planning and analysis phases and used to perform specialized operations during subtask execution. We distinguish our capability registry from existing tool discovery architectures to clarify its contributions. Our capability registry is substantially different from the Model Context Protocol (MCP) [38], due to MCP’s remote execution model. MCP exposes a function call interface through a standardized network API that facilitates tool discovery and tool invocation. This differs from our capability registry, which does not perform tool execution but instead points to an implementation of the tool software. D.8
Data resources and state management
APWA assumes access to highly available and durable network-accessed storage resource. We leverage two storage interfaces: the state store and the object store. State store The state store maintains all control state for the full set of tasks being processed by the APWA system. This includes task submissions, task manager state checkpoints, subtask definitions, subtask execution attempts, subtask outputs, and an append-only event log. Object store The object store is specifically used for storage of large, unstructured data objects. This interface is particularly well-suited for passing data back and forth between subtask workers and the task manager and can handle text, binary, image, and other data formats. Objects in the object store are content-addressed by a hash of their contents, meaning that URI contents are immutable. This greatly simplifies data coordination, as updates are only propagated by passing new handles to data. This property also implicitly restricts the ability of agents to use the data plane for communication, since they cannot send messages by modifying the contents pointed at by a given URI. D.9
Cluster management
In this subsection, we discuss considerations for managing computational and data resources in the cluster computing environment. D.9.1
Node resource management
Ray exposes logical resource controls that constrain the number of co-located subtasks executing on the same Ray node. These controls are flexible and allow constraints on resource use ranging from (virtual) hardware resources such as CPUs and GPUs to logical resources (for example, limiting port). We use this mechanism to control the maximum node-level subtask concurrency by configuring each subtask execution to consume 1 logical CPU resource. Each ray node connected to the cluster is registered with a number of (logical) cores, 21
D.9.2
Shared node services
Each node in APWA’s computing cluster maintains a small set of necessary services to facilitate basic interactions with core APWA resources. One of these services is the object store proxy. The object store proxy sits interfaces between subtask execution environments and the true system-wide object store. Using a proxy service simplifies worker environment setup, as it allows workers to use a consistent, local network location for accessing data resources in the object store. Using a proxy service also allows caching of resources, which allows multiple co-located worker processes to access the same data resources at a lower amortized cost. Crucially, because objects are content-addressed, this cache does not introduce the usual cache staleness concerns.
E
Extended Experimental Results
In this appendix, we provide detailed results for the experiments discussed in Section 4. E.1
Results on SummaryBench
Task description A description of the dataset used for hierarchical summarization is given in Table 5. Table 5: Description of datasets used for the hierarchical summarization task. Corpus
Level 1 Level 2 Level 3 Level 4 Size (kB)
Romeo and Juliet (R&J) The Dynasts Decline and Fall of the Roman Empire (Roman)
26 130 2588
5 19 296
3 71
6
166 942 10,500
For hierarchical summarization, we decompose structural scoring into a series of binary checks on the integrity of the output tables generated by the system. The precise number of conditions scored depends on the particular corpus being processed. In general, for a given corpus, each layer of the organization hierarchy generates three distinct checks. First, a “present” score checks whether or not the appropriate The overall structural score is computed as the mean score of these binary checks. We apply these same metrics for the baselines used in our evaluation, suitably adapted for the different data representation. Here, “presence” of a table indicates the presence of the appropriate key in the generated JSON object emitted by the system. Semantic scores are defined identically. Full results
We provide detailed results for SummaryBench in Tables 6 to 8.
Table 6: Failure rates and wall-clock runtimes for SummaryBench across baselines and APWA variants. FR is computed over all selected trials. WC is computed over trials that produced output and reported as mean ± standard deviation. R&J
Dynasts
WC
FR
WC
Roman
Method
FR
FR
Direct Magentic-One MegaAgent
0% 19 ± 1 60% 76 ± 0 100% ⊥ 100% ⊥ 100% ⊥ 100% ⊥ 60% 472 ± 44 60% 248 ± 63 40% 579 ± 189
APWA (mini×mini) 0% 96 ± 19 0% 142 ± 20 0% APWA (mini×nano) 11% 85 ± 9 22% 142 ± 29 11% APWA (5.4×mini) 0% 168 ± 17 0% 235 ± 27 0% APWA (5.4×nano) 0% 174 ± 21 0% 234 ± 30 0%
E.2
WC
272 ± 53 281 ± 45 372 ± 46 390 ± 43
Results on PII-300k
We report the full PII-300k results in Tables 9 to 11. E.3
Results on SchemaBench
We report full results for SchemaBench experiments in Tables 12 to 14. 22
Table 7: Structural and semantic scores for SummaryBench across baselines and APWA variants. Scores are computed over trials that produced output and reported as mean ± standard deviation. R&J Method
Str.
Direct Magentic-One MegaAgent
Dynasts Sem.
Str.
Roman Sem.
Str.
Sem.
1.000 ± 0.000 0.433 ± 0.013 0.979 ± 0.021 0.210 ± 0.005 ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ 0.140 ± 0.140 0.043 ± 0.043 0.212 ± 0.121 0.023 ± 0.017 0.160 ± 0.226 0.016 ± 0.022
APWA (mini×mini) 0.954 ± 0.075 0.424 ± 0.136 0.971 ± 0.034 0.447 ± 0.055 0.908 ± 0.133 0.230 ± 0.164 APWA (mini×nano) 0.950 ± 0.037 0.426 ± 0.036 0.872 ± 0.134 0.323 ± 0.165 0.914 ± 0.138 0.295 ± 0.129 APWA (5.4×mini) 1.000 ± 0.001 0.528 ± 0.012 0.997 ± 0.003 0.451 ± 0.025 0.983 ± 0.025 0.370 ± 0.024 APWA (5.4×nano) 0.943 ± 0.033 0.439 ± 0.012 0.923 ± 0.046 0.419 ± 0.018 0.872 ± 0.011 0.395 ± 0.007
Table 8: SummaryBench detailed metrics by APWA configuration, corpus, and hierarchy layer. Values are averaged over APWA trials that produced the corresponding layer output. ROUGE Config
Corpus Layer
R-1
R-2
Structural R-L Present URI res. Word cap
R&J
Scene Act Full
0.349 0.120 0.226 0.411 0.127 0.237 0.514 0.144 0.319
1.000 1.000 1.000
0.857 0.857 1.000
0.604 0.857 1.000
APWA (mini×mini) Dyn.
Scene Act Part Full
0.409 0.136 0.245 0.469 0.123 0.242 0.427 0.111 0.225 0.484 0.115 0.219
1.000 1.000 1.000 1.000
0.857 1.000 1.000 1.000
0.714 0.939 0.943 0.857
Paragraph 0.355 0.106 0.222 Part 0.224 0.033 0.101 Roman Chapter 0.206 0.026 0.090 Volume 0.196 0.022 0.081 Full 0.170 0.015 0.075
1.000 1.000 1.000 1.000 1.000
0.857 0.714 0.714 0.714 0.714
0.857 0.705 0.692 0.690 0.571
R&J
Scene Act Full
0.397 0.107 0.227 0.394 0.096 0.210 0.434 0.097 0.228
1.000 1.000 0.889
1.000 1.000 0.889
0.821 0.556 0.667
APWA (mini×nano) Dyn.
Scene Act Part Full
0.444 0.137 0.255 0.342 0.072 0.163 0.253 0.052 0.121 0.218 0.037 0.098
1.000 1.000 0.889 0.667
1.000 0.889 0.778 0.667
0.619 0.291 0.578 0.222
Paragraph 0.472 0.161 0.307 Part 0.311 0.042 0.135 Roman Chapter 0.233 0.023 0.099 Volume 0.201 0.017 0.081 Full 0.198 0.013 0.080
1.000 1.000 0.889 0.778 0.778
1.000 0.999 0.889 0.778 0.778
0.882 0.634 0.642 0.648 0.667
APWA (5.4×mini)
APWA (5.4×nano)
R&J
Scene Act Full
0.530 0.184 0.341 0.475 0.149 0.263 0.578 0.180 0.363
1.000 1.000 1.000
1.000 1.000 1.000
0.996 1.000 1.000
Dyn.
Scene Act Part Full
0.524 0.174 0.311 0.458 0.119 0.236 0.415 0.104 0.222 0.409 0.090 0.202
1.000 1.000 1.000 1.000
1.000 1.000 1.000 1.000
1.000 0.958 0.978 1.000
Paragraph 0.471 0.136 0.288 Part 0.386 0.056 0.174 Roman Chapter 0.359 0.044 0.155 Volume 0.312 0.032 0.132 Full 0.324 0.022 0.136
1.000 1.000 1.000 1.000 1.000
1.000 1.000 1.000 1.000 1.000
1.000 0.962 0.925 0.926 0.667
R&J
Scene Act Full
0.436 0.120 0.253 0.390 0.096 0.211 0.491 0.111 0.249
1.000 1.000 1.000
1.000 1.000 1.000
0.949 0.587 0.444
Dyn.
Scene Act Part Full
0.500 0.155 0.286 0.427 0.090 0.202 0.375 0.077 0.184 0.374 0.065 0.164
1.000 1.000 1.000 1.000
1.000 1.000 1.000 1.000
0.682 0.323 0.600 0.556
Paragraph 0.508 0.159 0.316 Part 0.404 0.055 0.170 Roman Chapter 0.372 0.040 0.148 Volume 0.346 0.031 0.136 Full 0.347 0.028 0.135
1.000 1.000 1.000 1.000 1.000
1.000 1.000 1.000 1.000 1.000
0.750 0.119 0.102 0.204 0.000
23
Table 9: Failure rates and wall-clock runtimes for PII-300k across baselines and APWA variants. FR is computed over all selected trials. WC is computed over trials that produced output and reported as mean ± standard deviation. PII-64
PII-512
WC
FR
Direct Magentic-One MegaAgent
0% 37 ± 4 0% 41 ± 29 100% ⊥ 100% ⊥ 100% ⊥ 80% 91 ± 0 60% 390 ± 266 80% 25 70% 372 ± 488
APWA (mini×mini)
0%
35 ± 4
FR
PII-4096
Method
0%
WC
FR
67 ± 52
0%
WC
221 ± 119
Table 10: Structural and semantic scores for PII-300k across baselines and APWA variants. Scores are computed over trials that produced output and reported as mean ± standard deviation. PII-64 Method
Str.
PII-512 Sem.
PII-4096
Str.
Sem.
Str.
Sem.
1.000 ± 0.000 0.775 ± 0.046 0.750 ± 0.000 0.162 ± 0.096 ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ 1.000 ± 0.000 0.179 ± 0.076 0.375 ± 0.375 0.000 ± 0.000 0.000 0.000 0.250 ± 0.354 0.000 ± 0.000
Direct Magentic-One MegaAgent
APWA (mini×mini) 1.000 ± 0.000 0.759 ± 0.064 1.000 ± 0.000 0.772 ± 0.040 0.900 ± 0.300 0.544 ± 0.356
Table 11: PII-300k detailed structural and semantic metrics by APWA configuration and document count. Values are averaged over APWA trials that produced output. Config
Size
Struct. PII Prec. PII Recall PII F1 Content Sim. Exact Match
PII-64 APWA (mini×mini) PII-512 PII-4096
1.000 1.000 0.900
0.833 0.840 0.593
0.758 0.757 0.528
0.793 0.796 0.559
0.894 0.893 0.694
0.347 0.350 0.244
Table 12: Failure rates and wall-clock runtimes for SchemaBench across baselines and APWA variants. FR is computed over all selected trials. WC is computed over trials that produced output and reported as mean ± standard deviation. C HEMISTRY
MLTABLES
D IS C O M AT
Method
FR
FR
FR
Direct Magentic-One MegaAgent
0% 350 ± 44 100% ⊥ 100% ⊥ 100% ⊥ 80% 95 60% 79 ± 52 100% ⊥ 60% 88 ± 47 60% 141 ± 53 100% ⊥ 40% 278 ± 202 80% 1222
WC
WC
APWA (mini×mini) 10% 62 ± 17 10% 137 ± 25 10% APWA (mini×nano) 10% 75 ± 14 10% 191 ± 57 0% APWA (5.4×mini) 0% 95 ± 10 0% 183 ± 22 0% APWA (5.4×nano) 10% 115 ± 8 10% 231 ± 37 10%
WC
SWDE FR
WC
103 ± 22 10% 104 ± 39 101 ± 49 0% 95 ± 20 147 ± 13 10% 125 ± 11 187 ± 27 10% 128 ± 17
Table 13: Structural and semantic scores for SchemaBench across baselines and APWA variants. Scores are computed over trials that produced output and reported as mean ± standard deviation. C HEMISTRY Method Direct Magentic-One MegaAgent
Str.
Sem.
MLTABLES Str.
Sem.
D IS C O M AT Str.
Sem.
SWDE Str.
Sem.
0.667 ± 0.000 0.534 ± 0.091 ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ 1.000 0.459 1.000 ± 0.000 0.054 ± 0.003 ⊥ ⊥ 1.000 ± 0.000 0.259 ± 0.084 0.333 ± 0.333 0.000 ± 0.000 ⊥ ⊥ 0.222 ± 0.314 0.000 ± 0.000 0.667 0.000
APWA (mini×mini) 1.000 ± 0.000 0.705 ± 0.251 1.000 ± 0.000 0.527 ± 0.115 0.963 ± 0.105 0.661 ± 0.236 1.000 ± 0.000 0.856 ± 0.217 APWA (mini×nano) 1.000 ± 0.000 0.583 ± 0.182 1.000 ± 0.000 0.465 ± 0.133 0.722 ± 0.124 0.649 ± 0.044 1.000 ± 0.000 0.812 ± 0.251 APWA (5.4×mini) 1.000 ± 0.000 0.794 ± 0.027 1.000 ± 0.000 0.566 ± 0.025 1.000 ± 0.000 0.801 ± 0.020 1.000 ± 0.000 0.942 ± 0.001 APWA (5.4×nano) 1.000 ± 0.000 0.763 ± 0.033 1.000 ± 0.000 0.549 ± 0.030 0.741 ± 0.139 0.703 ± 0.048 1.000 ± 0.000 0.924 ± 0.006
E.4
Results on WebSurfer
Task description The goal of the benchmark is to compose a report (100-200 words) on a manually curated set of topics drawn from diverse domains. It is designed to show that APWA is not only 24
Table 14: SchemaBench detailed structural and classification metrics by APWA configuration and schema extraction suite. Values are averaged over APWA trials that produced output. Micro P
R
Macro
Config
Suite
Struct.
F1
P
R
F1
APWA (mini×mini)
C HEMISTRY MLTABLES D IS C O M AT SWDE
1.000 1.000 0.963 1.000
0.699 0.791 0.742 0.690 0.759 0.705 0.671 0.546 0.583 0.588 0.519 0.527 0.663 0.660 0.661 N/A N/A N/A N/A N/A N/A N/A N/A 0.856
APWA (mini×nano)
C HEMISTRY MLTABLES D IS C O M AT SWDE
1.000 1.000 0.722 1.000
0.693 0.636 0.655 0.589 0.618 0.583 0.586 0.449 0.488 0.570 0.435 0.465 0.672 0.629 0.649 N/A N/A N/A N/A N/A N/A N/A N/A 0.812
APWA (5.4×mini)
C HEMISTRY MLTABLES D IS C O M AT SWDE
1.000 1.000 1.000 0.900
0.773 0.868 0.817 0.786 0.841 0.794 0.670 0.626 0.646 0.578 0.569 0.566 0.788 0.814 0.801 N/A N/A N/A N/A N/A N/A N/A N/A 0.848
APWA (5.4×nano)
C HEMISTRY MLTABLES D IS C O M AT SWDE
1.000 1.000 0.741 1.000
0.766 0.810 0.786 0.760 0.819 0.763 0.653 0.557 0.596 0.613 0.527 0.549 0.691 0.716 0.703 N/A N/A N/A N/A N/A N/A N/A N/A 0.924
capable of dynamically identifying the capability presets required for a task but is also able to utilize the corresponding agent from the registry at scale. Topics are divided into small (10), moderate (20) and large (100) splits. The benchmark will require the need of an agent with web browsing capabilities for which we use the Websurfer agent from Magentic-One. We score APWA along two dimensions: utility (whether the system produces a correct output) and cost (wall-clock runtime). We decompose utility into two submetrics, structural (whether the outputs are correctly formatted) and semantic (whether the outputs are correct). The structural score measures whether APWA returns the expected number of (topic, report) pairs. The semantic score evaluates each pair on 2 attributes - whether the report addresses the requested topic and whether the content is substantive, factual information rather than filler or off-topic. We use LLM-as-a-judge to assess these attributes, a pair receives a score of 1.0 if both attributes hold, 0.5 if only one holds and 0.0 otherwise. The final semantic utility is a mean over all pairs. Full results On the small split(10 topics), APWA completes the task in 2 min 23 s while maintaining a perfect score on both structural and semantic utility, successfully producing 10 relevant and factual reports. On the moderate split (20 topics), runtime increases to 2 min 37 s but APWA achieves a perfect structural score while yielding a semantic score of 0.975, with 19 out of 20 topics having a relevant and factual report and 1 deemed as relevant but not substantive. On the large split (100 topics), runtime grows to 10 min; APWA again achieves a perfect structural score and a semantic utility of 0.995, with 99 of 100 topics yielding a relevant and factual report and 1 deemed relevant but lacking substantive content. Notably, the runtime scaling is sublinear, increasing the workload from 10 to 100 topics (a 10× increase) results in only a 4.2× increase in the runtime. This suggests that the execution overhead is progressively amortized across larger batches, indicating favorable scalability for the task.
25