Avatar: Toward Autonomous End-to-End Orchestration of Scientific Workflows using LLMs Suman Raj1,2 , Hai Duc Nguyen2 , Haochen Pan1 , Ryan Chard2 , Kyle Chard1,2 , Ian Foster2,1 1
Department of Computer Science, University of Chicago, IL, USA Data Science and Learning Division, Argonne National Laboratory, Lemont, IL, USA Email: {sumanraj, haochenpan, chard}@uchicago.edu, {hai.nguyen, rchard, foster}@anl.gov
arXiv:2609.10509v1 [cs.DC] 9 Sep 2026
2
systems in which AI agents can participate directly in orchestration rather than remaining external assistants. This transition, however, exposes a fundamental architectural gap. Most existing WMSs were designed around conventional algorithms, policies, and human-configured control rather than agentic reasoning as a first-class capability. Adding agents therefore commonly requires retrofitting them into systems whose abstractions, control boundaries, and runtime interfaces were designed for different assumptions. Such adaptation is particularly challenging because mature scientific WMSs are large, well-established, and operationally complex, making substantial architectural changes costly and time-consuming [8]. More importantly, without a common understanding of where agents should be introduced, what information they should observe, which decisions they should control, and how their actions should be validated, each system must resolve these questions independently. The resulting integrations can consequently become narrowly scoped, difficult to generalize, or expensive to evaluate across different models, workflows, agent configurations, and infrastructure conditions. We address this gap with Avatar, a reference architecture for agentic scientific workflow orchestration. Rather than organizing the architecture around the capabilities of a particular LLM, agent framework, or WMS, Avatar is derived from recurring responsibilities across existing scientific workflow systems. It distills these responsibilities into three interacting components: an orchestrator that manages logical workflow decisions, an executor that realizes those decisions on runtime resources, and provenance that observes and maintains the evidence needed for subsequent decisions. Each component exposes explicit state, decision, and action interfaces through which conventional policies, LLM-backed agents, or hybrid approaches can be introduced without changing the surrounding architecture. This separation provides a common experimental basis for studying where agentic reasoning should be placed, what responsibilities it should assume, and how it interacts with conventional workflow mechanisms. By holding the architecture fixed while varying the reasoning policy, Avatar enables controlled evaluation of when agentic orchestration helps, when conventional mechanisms remain preferable, and what architectural factors determine the outcome. Specifically, the contributions of the paper are as follows: • We propose Avatar, a reference architecture for agentic scientific workflow orchestration. The architecture captures
Abstract—Scientific workflow management (WMSs) systems automate execution, yet orchestrate using fixed, hand-tuned rules. LLM agents promise more autonomous orchestration, but it remains unclear where to introduce agentic reasoning, how to bound its risk, and when it actually helps. We present Avatar, an actor-based architecture comprising an orchestrator, an executor, and a provenance monitor. Each actor’s decision policy is pluggable (rule-based or LLM-backed) via a single adapter-validated action catalog, so conventional and agentic control run on the same core across different WMSs. We present an implementation using the Academy framework and evaluate Avatar across three workloads. We observe that Avatar’s rule mode reproduces native execution, with a single unchanged core running all three. Moreover, LLM-backed Avatar reports a reduction of compute wastage by 55% and cuts GPU-busy time by 40%. Overall, we envision Avatar as a step toward workflow systems that reason about their own orchestration rather than follow pre-fixed rules.
I. I NTRODUCTION Scientific discovery increasingly relies on complex computational campaigns that combine simulation, data analysis, machine learning, distributed instruments, and heterogeneous computing resources [1]. At the same time, recent advances in large language models (LLMs) and agentic AI are changing how users interact with computational systems. This capability is particularly promising for scientific workflows, where researchers often spend substantial effort translating scientific intent into executable workflows, configuring runtime systems, monitoring execution, diagnosing failures, and adapting computation as conditions change [2], [3]. Agentic reasoning offers the potential to automate or assist many of these decisions, moving workflow management toward more adaptive and autonomous orchestration. Scientific workflow management systems (WMS) such as Pegasus [4] and Parsl [5] already provide many of the abstractions needed for this transition. These abstractions transform workflow management into a set of explicit, inspectable, and actionable interfaces through which agents can reason over workflow graphs, revise task plans, select resources, interpret monitoring feedback, and diagnose failures from logs and provenance. Recent efforts make this direction more visible by extending workflow systems toward decentralized decision making, hybrid AI-HPC execution, federated agents, and AIassisted workflow lifecycle management [2], [6], [7]. Together, these developments indicate a broader shift toward workflow
1
common components of existing scientific workflow frameworks and exposes flexible integration points for adding LLM-backed reasoning capability (§III). • We implement Avatar as an extensible prototype framework using Academy, allowing different agents, tools, monitoring interfaces, and validation strategies to be integrated and compared systematically (§IV). • We evaluate Avatar across diverse workloads, orchestration structures, and objectives, revealing when agentic reasoning can reduce wasted computation by up to 55% and GPU-busy time by 40% (§V).
flow execution. SWARM distributes functions such as job selection, scheduling, data management, and failure recovery among cooperating intelligent agents [19]. RHAPSODY allows simulation runtimes, inference services, and agentdriven control to coexist through shared abstractions for tasks, services, resources, and execution policies [6]. Academy provides abstractions for deploying and coordinating stateful agents across HPC systems, scientific instruments, and data repositories [7]. FireWorks-based autonomous campaigns use machine-learning models to analyze intermediate results and prioritize subsequent calculations [20]. These systems expose more execution state and authority to intelligent components, enabling greater automation and runtime adaptation. Yet due to the potential impact of incorrect or unexpected decisions, their agentic capabilities remain limited to particular layers or decision classes rather than controlling the complete workflow lifecycle. Taken together, prior work demonstrates several viable ways to incorporate AI into scientific workflows, ranging from constrained assistance to direct runtime control. What remains missing is a common architectural basis for comparing and composing these choices. Existing AI-workflow integrations remain fragmented and system-specific [8], while different forms of coupling among AI models, simulations, and workflow control impose distinct middleware, execution, and performance requirements [21]. Consequently, each WMS must independently determine where agents should participate, what state they should observe, which decisions they should control, how those decisions should be realized, and how their actions should be constrained and validated. Repeating these design decisions for every WMS requires substantial architectural modification and makes systematic evaluation across models, agent configurations, workflows, and infrastructure conditions difficult.
II. R ELATED W ORK Integrating agentic reasoning into scientific orchestration has recently attracted growing attention. Most workfloworiented efforts take a conservative approach: agents perform narrowly scoped functions while established WMS components retain authority over validation, scheduling, and execution. Pegasus adds specification-driven workflow generation, validation, debugging, submission, and monitoring [2]; HyperFlow limits LLM reasoning to translating scientific questions into structured intents [3]; and Parsl has been integrated with LangChain/LangGraph tool calling to enable LLM agents to execute scientific tasks on HPC resources [9]. Related efforts use generative AI to transform terminal sessions, notebooks, or natural-language requests into executable and validated workflows [10], [11], [12]. These systems demonstrate that LLMs can assist important parts of the workflow lifecycle while relying on mature WMS mechanisms to preserve reliable execution. This approach limits the consequences of incorrect agent decisions, but also confines agentic reasoning to particular stages rather than allowing it to participate continuously in orchestration. A complementary body of work focuses on strengthening the reasoning and coordination capabilities needed for agents for broader responsibilities. Foundational prompting and reasoning techniques such as chain-of-thought prompting [13], ReAct [14], Tree-of-Thoughts [15], and Reflexion [16] established that LLMs can interleave deliberation, tool invocation, and self-critique to produce multi-step plans. Multiagent frameworks such as AutoGen [17] extend these capabilities to collections of role-specialized agents that coordinate through structured interaction, while AgentGen [18] explores how planning capabilities can be developed and evaluated systematically at scale. These techniques provide important building blocks for agentic orchestration, but their evaluation largely targets general-purpose reasoning, coding, or interactive environments. Scientific workflows impose additional requirements, including explicit task and data dependencies, heterogeneous computing resources, long-running execution, failures, provenance, and interaction with established workflow runtimes. Applying these reasoning capabilities to scientific orchestration therefore still requires mechanisms that connect agent reasoning to workflow state, decisions, and execution. Other systems address this integration more directly by giving intelligent components greater authority during work-
III. AVATAR : R EFERENCE A RCHITECTURE A. Deriving the Reference Architecture The fragmented and system-specific integrations identified above motivate a common architectural basis for introducing agentic reasoning into scientific workflows. Rather than deciding independently for each WMS where an agent should be placed and what it should control, we derive a reference architecture from responsibilities that recur across existing WMSs. Table I summarizes our survey of representative WMSs. Despite substantial differences in architecture, programming model, execution environment, and implementation, these systems exhibit a common abstraction-oriented design pattern. They represent scientific computations through logical abstractions, such as workflows, tasks, data objects, and dependencies; make logical decisions over these abstractions to determine what should happen; and realize those decisions through runtime mechanisms that determine how the computation is executed. This separation between logical workflow management and runtime realization provides the first foundation for Avatar.
2
TABLE I: Survey of representative scientific workflow frameworks based on how they manage scientific computations through framework-native logical abstractions, decisions over those abstractions, runtime realizations, and exposed attributes. Framework
Logical abstractions
Logical decisions
Runtime realizations
Attributes
Parsl [5]
App, task, AppFuture/DataFuture
Instantiate Apps. Resolve/release tasks.
Dispatch tasks. Resolve AppFuture/DataFuture.
Task: pending, running, failed. AppFuture/DataFuture: unresolved, resolved, failed.
Pegasus [4]
Abstract Workflow, Job, Transformation, File
Plan Abstract Workflow. Order Jobs. Bind Transformations.
Map/submit Jobs. Stage/register Files.
Job: planned, running, failed. File: unstaged, staged, registered.
Cromwell [22] WOM graph, WOM node, Value Store
Expand WOM graph. Resolve/release WOM nodes.
Execute WOM nodes. Populate Value Store.
WOM node: not started, running, done. Value Store: unavailable, available.
Toil [23]
Job graph, Job, Promise
Extend Job graph. Resolve/order Jobs.
Submit/execute Jobs. Resolve Promises.
Job: waiting, running, completed. Promise: unresolved, resolved.
CWL [24]
Workflow, WorkflowStep, Process
Validate Workflow. Resolve WorkflowSteps. Bind Process inputs.
Instantiate Processes. WorkflowStep: inputs, outputs, scatter. Execute Processes through runners. Process: inputs, outputs, requirements.
WDL [25]
workflow, task, call
Expand workflow. Resolve calls. Bind call inputs.
Construct task commands. Execute calls.
Task: inputs, outputs, runtime requirements. Call: inputs, alias, dependencies.
Snakemake [26]Snakefile, rule, wildcard
Parse Snakefile. Resolve rules. Bind wildcards.
Execute rules. Materialize rule outputs.
Rule: input, output, resources. Wildcard: value, constraints.
Makeflow [27] DAG file, rule, source/target file
Parse DAG file. Resolve/release rules.
Execute rules. Transfer source/target files.
Rule: sources, targets, command. File: missing, available, outdated.
Nextflow [28] process, channel, dataflow value
Trigger processes. Bind channels/dataflow values.
Execute processes. Stage/cache process data.
Process: inputs, outputs, directives. Channel/dataflow value: waiting, emitted, closed.
FireWorks [29] Workflow, Firework, Firetask
Update Workflow. Prioritize Fireworks.
Reserve Fireworks. Execute Firetasks.
Firework: ready, running, completed. Workflow: ready, running, completed.
The survey further shows that workflow abstractions are accompanied by attributes describing either their intended configuration or their observed runtime state. Examples include task readiness, priority, resource requirements, execution status, data availability, retry count, and failure state. These attributes serve as the interface between workflow specification and execution by exposing decision points throughout the workflow lifecycle. For instance, newly available inputs may enable downstream tasks, execution failures may trigger retries, and resource unavailability may necessitate replanning. Consequently, WMSs must not only represent the intended computation, but also observe how that computation evolves and preserve enough execution history to support subsequent decisions.
Orchestration orders/ Orchestration feedback Input Workflow Description
Workflow CRUD
Message Fabric
Provenance feedback/ Provenance order
Execution feedback/ Execution order
Actor: Orchestrator
Actor: Executor
QoS/ constraints
Execute Compute Resources
Actor: Provenance Gather execution information
Fig. 1: Avatar architecture concerning workflow construction, task ordering, priorities, adaptation, and recovery. The executor realizes these decisions on concrete infrastructure. It selects and controls execution mechanisms, binds tasks to resources, launches tasks, stages data, manages runtime execution, and reports outcomes. The provenance component maintains the evidence needed to close the decision loop. It determines which attributes and events should be collected, records logs, performance measurements, task outcomes, failures, retries, and workflow history, and emits triggers when observed outputs require a new decision. These components describe normalized responsibilities rather than modules that existing WMSs necessarily expose under the same names. Pegasus, for example, separates an abstract workflow from the executable workflow generated for a target environment whereas Parsl separates dynamic
Together, these recurring patterns yield three fundamental responsibilities for scientific workflow orchestration. First, a system must maintain the logical representation of a computation and decide how that representation should evolve. Second, it must realize those decisions through concrete execution mechanisms. Third, it must observe and preserve the evidence required to determine when decisions should be made, validated, or reconsidered. Avatar maps these responsibilities to three corresponding components, shown in Figure 1. The orchestrator manages the logical abstractions and their evolution. It interprets workflow descriptions, user intent, QoS requirements, execution constraints, and makes logical decisions
3
Actor Message Fabric Data In
LLM In Adapter
monitoring services, provenance stores, or MCP servers. Before messages reach the agent, they pass through adapters that translate external data into the representation expected by the actor. Agent outputs also pass through adapters before they are exposed to the rest of the system. Adapters provide both portability and safety. They allow existing workflow components to be wrapped as actors, or replaced with LLMbacked agents, without changing the rest of the architecture. They also serve as policy-enforcement boundaries: instead of giving agents unrestricted access to the runtime, adapters control what information reaches the agent and what actions can leave it. For example, adapters can filter runtime state, normalize provenance records, validate action formats, reject unsafe tool calls, or require additional checks before executing an agent-generated decision. This design makes the study of agentic scientific workflow orchestration explicit and systematic. By varying actor implementations, exposed state, available tools, and messagefabric policies, we can compare fully conventional baselines, partially agentic systems, and more autonomous orchestration loops before committing to a system-specific integration. By making observation, reasoning, action, and validation boundaries explicit, Avatar provides both a practical blueprint for agentic workflow orchestration and an experimental framework for understanding its benefits and limitations.
Agent Agentic framework
Conventional Implementation
Out Adapter
Data Out
Message Fabric
Tool Adapter Tool API Actions that change the outer world
Fig. 2: Actors integrated into Avatar dependency management from execution through configurable executors. Rule- and dataflow-oriented systems such as Snakemake, Makeflow, and Nextflow use different abstractions, yet follow the same progression from logical dependency reasoning to runtime realization [26], [27], [28]. Systems such as Cromwell, Toil, and FireWorks further demonstrate the importance of preserving execution state and history for monitoring, retry, recovery, and dynamic workflow modification [22], [23], [29]. This decomposition directly addresses the architectural challenge identified earlier by making orchestration responsibilities explicit and allowing the decision mechanism within each responsibility to vary independently. Rather than embedding agents at system-specific locations, Avatar lets the orchestrator, executor, and provenance components use conventional algorithms, LLM-backed agents, or hybrid policies while preserving common interfaces and validation boundaries. This makes Avatar representative of recurring WMS responsibilities yet general enough to support different agentic configurations, providing a common basis for locating, composing, and systematically evaluating where agentic reasoning is introduced and what trade-offs it creates.
IV. I MPLEMENTATION Avatar is implemented in Python 3.12 on Academy 0.5.0, a middleware for stateful agents on federated research infrastructure [7]. Academy directly supplies the primitives that Avatar’s actor model requires. An actor is written as an Academy Agent whose @action-decorated methods expose the operations that peers may invoke remotely, and whose @loop-decorated methods run its autonomous control loop. Actors reference one another through Handles and exchange messages asynchronously through per-agent mailboxes routed by an Exchange, which serves as our message fabric. In the current version, Avatar uses LocalExchangeFactory (an in-process mailbox exchange) backed by a ThreadPoolExecutor. Task execution uses Parsl 2026.3.9 and native CCTools TaskVine; the Colmena surrogate is PyTorch (CUDA 12.2) over RDKit fingerprints; agent reasoning calls use OpenAI client 2.28.0. Actors. The provenance agent (ProvenanceActor) ingests the executor’s TaskEvent stream, keeps per-task attempt and lineage state, and on each failure emits a Trigger carrying a diagnosis produced by its policy. The executor (ExecutorActor) owns a bounded worker pool and dispatches each task through one overridable interface. _execute is a local job.py subprocess for TaskVine and python_app for Parsl, reporting outcomes and events back to provenance and realizing scale-out/scale-in on the resource pool. The orchestrator (OrchestratorActor) runs the control loop: it releases dependency-ready tasks, drains completions and failure triggers, maps each diagnosis to a recovery Decision, and enacts it once validated.
B. Actor-based Realization We realized the core components (i.e., the orchestrator, executor, and provenance) as actors with a standardized interface for communication and action (Figure 2). An actor’s internal decision maker, referred to as its agent, may be implemented using conventional algorithms, LLM-backed reasoning, or a hybrid of the two. This abstraction enables existing planners, executors, and provenance managers to be wrapped and systematically compared with agentic implementations without changing the surrounding architecture. Actors interact through a shared message fabric, following the message-oriented communication model common in distributed systems. The message fabric decouples components, makes interactions explicit, and provides a controlled interface for information exchange, allowing message schemas, routing policies, and access controls to be varied without modifying component internals or the overall architecture. Each actor interacts with the rest of the system through three controlled interfaces: data input, data output, and tool invocation. The data input and output interfaces connect the actor to the message fabric. The tool interface allows the actor to affect the external environment by invoking predefined tools, such as workflow submission APIs, resource managers,
4
Policies, adapters, and modes. Each agent’s decisionmaker is a pluggable policy which is a deterministic rule set derived from the native backend or an LLM-backed reasoner. Every proposed action passes an adapter that validates it against one fixed catalog: retry, migrate, blacklist-worker, replicate-task, scale-out/in, batch, throttle, and the steering actions propose-batch, retrain-surrogate, infer-pool, and stopcampaign, so agentic and rule modes share one action set and invalid actions are counted. This yields three variants of implementation: M0 uses rule policies throughout; M1 adds an LLM diagnosis in provenance while rules still act; M2 lets the LLM both diagnose and decide in the orchestrator, with the executor still actuating by rule. An LLM-backed executor, i.e., agentic placement across heterogeneous resources, is a natural third locus for reasoning, but our single-node, homogeneous testbeds in this paper present no real placement choice, so we leave it to the multinode setting for future.
permanent, recurring across all retries, randomly determined by an experiment parameter x which indicates the statistical fraction of permanent failures. By default, we set p = 0.35 and x = 1/3. On failure, the task exits with a non-zero status and exposes only the observed error symptom to the recovery policy; each attempt is also recorded in a shared log with its true class for offline evaluation. Avatar executes the tasks as local asyncio subprocesses driven by its three Academy actors through a LocalExchangeFactory, under the same retry ceiling as the native baseline. Its Provenance actor diagnoses failures from their retry history. • Scaling (E2): A dynamic analyze-and-steer streaming workflow in which N inputs arrive over time, each triggering a three-stage sub-workflow: preprocess → analyze (×K) → aggregate. We set K = 4, so each input creates four parallel analysis tasks. Each task performs a dense matrix multiplication similar to E1 and starts only after all tasks in the preceding stage have completed. Inputs follow a nonstationary schedule with four phases: (i) low (≈ 2f /s for 4s), (ii) burst (≈ 15f /s for 5s), (iii) high (≈ 8f /s for 5s), and (iv) recover (≈ 2f /s for 4s) where f ≈ N/131 is the rate control parameter at a given N , selected to make the input arrival duration finishes in ≈ 18s. • Active Learning (E3): An unmodified Colmena campaign that maximizes the HOMO–LUMO gap over a pool of QM9 molecules. The candidate pool contains np = 5000 QM9 molecules together with an initial labeled seed set of ns = 100. In each round, the campaign retrains an MLP surrogate over Morgan fingerprints on the GPU using all molecules scored so far, runs inference over the candidate pool, greedily selects the top-B candidates (B = 32) by predicted gap, and evaluates the selected batch. The evaluation uses a deterministic QM9-gap lookup as a stand-in for an expensive property calculation. Avatar drives the campaign through its three Academy actors, with every steering proposal (i.e., propose-batch, retrain-surrogate, infer-pool, and stop-campaign) expressed as a catalog action and validated by an adapter before execution.
V. E VALUATION A. Methodology Objectives. We assess whether Avatar provides an appropriate reference architecture for studying agentic integration into scientific workflow systems along three dimensions. (i) Representativeness: whether Avatar can reproduce the control behavior and outcomes of distinct conventional WMSs while preserving the same core actor structure. (ii) Generalizability: whether Avatar can support workflows that differ substantially in structure, execution pattern, and scientific domain without redesigning its core components. (iii) Applicability: whether Avatar can serve as an experimental instrument for identifying where, when, and how agentic reasoning is beneficial. Metrics. We align the evaluation criteria with these objectives. Representativeness is assessed through workflow correctness and behavioral agreement with native WMS implementations, including equivalent completed work and control outcomes. Generalizability is demonstrated by executing all diverse workloads with the same Avatar actors and interfaces. For applicability, we measure workload-specific benefits and costs: retries and wasted worker time for resilience, backlog and resource utilization for elastic scaling, and scientific outcome, molecule evaluations, GPU usage for molecular design.
Baselines. In addition to Avatar, each workload is paired with the conventional mechanism used to manage the corresponding execution in practice. • TaskVine. For E1, TaskVine drives the resilience workload on the compute node using its co-located vine_worker and blind fixed retries with an upper bound R. • Parsl. For E2, each scaling-workload task is implemented as a Parsl python_app and executed using fixed Parsl pools of different sizes: w4 (small), w12 (medium), and w32 (large). Each pool provides concurrency up to its worker count, and backlog is sampled every 0.2s. • Colmena [30]. For E3, the native controller of the E3 Active-learning workload that use a BaseThinker over a ParslTaskServer with three registered task methods (evaluate molecule/train model/infer model) on a GPU Parsl executor, running a fixed round budget of 12.
Workloads. We evaluate Avatar on three workloads chosen to span distinct workflow organizations, orchestration structures, and objectives. • Resilience (E1): A fault-prone fan-out/fan-in DAG representative of traditional scientific workflows: one input task feeds N parallel map tasks followed by a reduce task. Each map task performs 300 iterations of a normalized 160 × 160 double-precision matrix multiplication (a ← (a · a)/||a||), but its execution is intentionally unreliable. After the computation, with probability p, a task either fails or becomes a straggler by sleeping for an additional 0.15s. Failures are either transient, disappearing after a retry, or
System Configurations. All experiments run on the Uni-
5
40 20 0
64
128
256
Map Fan-out N
(a) Reproducibility on E1 (solid = productive, hatched = wasted; whiskers = min–max over 3 runs).
450
1600 1200
300
800
150
400
0
32 64 128 256
Steering Scaling Resilience
TaskVine Avatar M0
60
Parsl w32 Avatar M0
Completed Tasks
600
Worker-Seconds
Worker-Seconds
80
0
Number of Inputs N (b) Reproducibility on E2
retry migrate blacklist replicate scale-out scale-in batch throttle propose retrain infer stop noop
E1 E2 E3
Fig. 4: One shared action catalog spans three workloads on one unchanged core. Filled = invoked; faint = in catalog but not triggered.
Fig. 3: Avatar is representative and generalizable across WMSs versity of Chicago RCC midway3 cluster. For E1, Avatar is submitted as an sbatch job running in exclusive mode on a single caslake compute node with an Intel(R) Xeon(R) Gold 6248R CPU at 3.00 GHz, 48 CPU cores, and 192 GB of RAM. The native TaskVine Manager, its co-located vine_worker, and the job.py tasks execute on the same node, while Avatar’s three Academy actors run in-process alongside them. For E2, Avatar launches its agents on the login node (midway3-login3), while Parsl’s SlurmProvider(exclusive=True) provisions a dedicated caslake compute node with the same hardware specification as E1 for the numerical workers. Exclusive allocation ensures that Parsl workers receive dedicated CPU cores and avoids interference from co-located workloads. For E3, the Colmena steering workload is submitted as an sbatch job to the gpu partition with 1 GPU and --cpus-per-task=16, with the Academy actors running alongside the Parsl workers. The GPU is an NVIDIA Quadro RTX 6000 running CUDA 12.2. Across all experiments, LLM API calls from Avatar actors use the ALCF Sophia inference endpoint (inference-api.alcf.anl.gov) and the open-ai/gpt-oss-20b model hosted at Argonne National Laboratory.
the variation across seeds. Avatar M0 consistently consumes slightly fewer worker-seconds because its simplified executor avoids part of the dispatch overhead incurred by TaskVine’s full vine_worker implementation. This difference becomes more visible as fan-out increases, reaching approximately 19% at N =256. Importantly, this implementation-level difference does not change the workflow’s recovery decisions or outcomes. Avatar therefore preserves the behavior relevant to the experiment. Takeaway (Representativeness): Avatar reproduces native workflow behavior across distinct systems through a representative, lightweight architecture. Generalizability. We continue with M0 to evaluate Avatar’s generalizability. If Avatar is generalizable, the same core architecture should support qualitatively different workflow structures and execution models without redesigning its actors or message fabric. Fig. 3b provides the first evidence by complementing the E1 result with a structurally different workload. Across increasing workload intensities N , Avatar M0 and native Parsl complete the same number of tasks, while their consumed workerseconds differ by less than 2%. Together with E1, this shows that the same Avatar implementation remains generalizable in two contrasting settings: E1 is a batch-oriented, fault-prone DAG with relatively stable computational demand, whereas E2 is a streaming workflow with bursty arrivals and dynamically changing resource demand. Their native implementations also rely on different workflow systems, TaskVine and Parsl, which expose different integration capabilities with Avatar’s core components (Table I). Avatar’s ability to reproduce both therefore demonstrates that its architecture is not tied to the execution model or integration characteristics of a particular WMS. Fig. 4 provides stronger implementation-level evidence. Across E1–E3, Avatar uses a single shared action catalog, organized into resilience, scaling, and scientific-steering capabilities. The ≈ 1200 lines implementing the core orchestrator, executor, and provenance actors, together with their policies,
B. Experimental Results Representativeness. To evaluate Avatar’s representativeness, we use its rule-based configuration (M0), which introduces no agentic decision making. If Avatar captures the essential orchestration behavior of existing workflow systems, M0 should reproduce their characteristic behavior under equivalent workloads and policies. We compare Avatar M0 with native TaskVine on E1 (Resilience) under identical scale (N ) and fault plans using the same failure-probability seeds and retry ceiling R=2. Avatar M0 reproduces TaskVine’s recovery behavior exactly: retry, completion, and permanent-failure counts match for every (N, seed) pair (e.g., both produce 88 retries, 231 completions, and 26 permanent failures at N =256). Fig. 3a further shows that both systems exhibit the same scaling trend in used and wasted worker-seconds, with differences remaining within
6
25 20 15 10 5 0
15
Best Property Found
Backlog (QoS)
Wasted Worker-Seconds
invariant, while its advantage over blind retry increases as failures become more frequent (p) or more likely to be pernative Avatar M0 Avatar M2 manent (x). The experiment therefore reveals a clear regime in 10 8 which agentic reasoning is valuable: diagnosis-driven recovery provides the greatest benefit when failures are both frequent 5 4 and unrecoverable. 0 0 E2 exposes the opposite regime. Fig. 6a shows the cost–QoS 0.1 0.33 0.6 0.9 0.1 0.2 0.35 0.5 0.65 1 2 3 4 5 Perm. Failure (x) Failure Prob. (p) Retry Budget (R) trade-off for Parsl scaling at N =128. Among the fixed pools, (a) Varying R (b) Varying p (c) Varying x w4 minimizes resource allocation but accumulates substantial backlog (mean ≈171), whereas w32 maintains near-zero backFig. 5: E1. Resilience on TaskVine backend (N =128). log by provisioning approximately 2.5× the compute it actuCum. Molecules Evaluated 350 ally uses. Avatar’s rule-based autoscaler (M0/M1) approaches 100 200 300 400 native w4 0.40 the desirable low-backlog, low-waste region with a mean 280 native native w12 M2 stops Avatar M0 (324 molecules) native w32 backlog of approximately 21. In contrast, M2 reaches a mean Avatar M1 210 Avatar M0 Avatar M2 0.38 backlog of approximately 99 because the LLM cannot react Avatar M1 140 Avatar M2 as quickly as the scaling loop requires. Thus, dynamic agentic 70 0.36 solution found reasoning is not universally advantageous: for high-frequency, (260 molecules) 0 200 latency-sensitive control, the reasoning latency itself becomes 400 600 0 2 4 6 8 10 12 Worker-Seconds Active-Learning Round a control cost, and a lightweight rule can be more effective. (a) E2. Parsl QoS at N =128 (b) E3. Colmena molecular design Finally, E3 demonstrates where dynamic agentic control can provide substantial benefit in a real scientific steering Fig. 6: E2 and E3 on Parsl backend. application. Fig. 6b relates the Colmena learning curve to adapters, action catalog, and message types, remain byte- GPU cost. The native baseline and M0/M1 execute the full identical across all three workloads. Each workload connects 12-round campaign, whereas M2 reasons over the observed to this core only through a one-method _execute interface: learning progress, detects convergence, and terminates at round a job.py subprocess for E1 and a Parsl python_app for E2 7. It reaches the same best HOMO–LUMO gap (0.3997) and E3, together with its workload generator. E3 additionally while reducing GPU-busy time by approximately 40%. Unlike introduces only a small orchestrator subclass for extracting E2’s fast scaling loop, convergence detection is an infrequent, application-specific results. Despite sharing the same core, the context-dependent decision for which reasoning latency is workloads exercise largely disjoint parts of the action catalog: small relative to the computation it can avoid. E1 uses resilience actions, E2 uses scaling actions, and E3 Taken together, these experiments refine the original hyuses scientific-steering actions, with noop as the only shared pothesis: agentic reasoning is most beneficial for relatively action. This shows that one unchanged Avatar core can drive infrequent decisions that require interpreting execution context a static fault-tolerant DAG, a dynamic elastic stream, and a and can avoid substantial downstream work, while convenGPU-based scientific steering campaign. tional rules remain preferable for high-frequency control. More importantly, Avatar makes this distinction experimentally obTakeaway (Generalizability): Avatar generalizes across servable by allowing the decision policy to change while diverse workflows and control objectives using one unholding the surrounding architecture fixed. changed core architecture and interfaces. 12
Takeaway (Applicability): Avatar demonstrates practical applicability by enabling controlled, comparable evaluation of agentic policies across diverse workflow scenarios.
Applicability. Next, we use Avatar M1 and M2 to evaluate its applicability as an experimental instrument for studying agentic workflow control. A common motivation for agentic workflow management is that such dynamic reasoning can outperform conventional static policies by adapting actions to runtime conditions. We therefore use Avatar to test this hypothesis directly: does LLM-enabled dynamic control improve efficiency over conventional rule-based control, and under what conditions? On E1 (Resilience), Fig. 5 examines three factors independently at N =128: retry budget R (Fig. 5a), task failure probability p (Fig. 5b), and permanent-failure fraction x (Fig. 5c). Native blind retry and M0 exhibit the same behavior, with wasted computation increasing as each factor grows. M2 instead uses the observed failure history to identify permanently failing tasks and gives up after one retry. Consequently, its wasted computation is effectively R-
VI. C ONCLUSION AND F UTURE W ORK In this paper, we propose Avatar, which recasts a scientific WMS using three cooperating actors: orchestrator, executor, and provenance, with pluggable decision policies. This allows the same implementation to support both deterministic rules and LLM-based reasoning. Our experiments show that Avatar is representative, reproducing native TaskVine and Parsl execution in rule mode; generalizable, supporting static faulttolerant DAGs, dynamic streaming workflows, and GPU-based active learning; and applicable, with LLM policies providing benefits for decisions that require inference. We plan to extend Avatar to multi-node and heterogeneous environments spanning CPUs, GPUs, and multiple sites. This
7
will introduce resource-selection decisions for the executor, such as selecting a resource for a task or migrating a failed task to another node. This setting will allow us to extend LLMbased decision-making to the executor and evaluate the safety of the resulting action space. We also plan to improve failure handling by identifying error types and applying targeted recovery actions rather than simply retrying failed tasks.
[13] J. Wei, X. Wang, D. Schuurmans, M. Bosma, et al., “Chain-of-thought prompting elicits reasoning in large language models,” Advances in neural information processing systems, 2022. [14] S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. Narasimhan, and Y. Cao, “ReAct: Synergizing reasoning and acting in language models,” in The Eleventh International Conference on Learning Representations, 2023. [15] S. Yao, D. Yu, J. Zhao, I. Shafran, et al., “Tree of thoughts: Deliberate problem solving with large language models,” Advances in neural information processing systems, vol. 36, pp. 11 809–11 822, 2023. [16] N. Shinn, F. Cassano, A. Gopinath, K. Narasimhan, and S. Yao, “Reflexion: Language agents with verbal reinforcement learning,” Advances in neural information processing systems, vol. 36, 2023. [17] Q. Wu, G. Bansal, J. Zhang, Y. Wu, et al., “AutoGen: Enabling next-gen LLM applications via multi-agent conversation,” in First Conference on Language Modeling (COLM), 2024. [18] M. Hu, P. Zhao, C. Xu, Q. Sun, et al., “AgentGen: Enhancing planning abilities for large language model based agent via environment and task generation,” in ACM SIGKDD Conference on Knowledge Discovery and Data Mining, vol. 1, 2025, pp. 496–507. [19] P. Balaprakash, K. Raghavan, F. Cappello, E. Deelman, et al., “SWARM: Reimagining scientific workflow management systems in a distributed world,” The International Journal of High Performance Computing Applications, vol. 39, no. 5, pp. 692–712, 2025. [20] A. Takahashi, K. Terayama, Y. Kumagai, R. Tamura, and F. Oba, “Fully autonomous materials screening methodology combining first-principles calculations, machine learning and high-performance computing system,” Science and Technology of Advanced Materials: Methods, 2023. [21] W. Brewer, A. Gainaru, F. Suter, F. Wang, M. Emani, and S. Jha, “Aicoupled hpc workflow applications, middleware and performance,” arXiv preprint arXiv:2406.14315, 2024. [22] Broad Institute, “Cromwell: Scientific workflow engine for WDL,” 2026. [23] J. Vivian, A. A. Rao, F. A. Nothaft, et al., “Toil enables reproducible, open source, big biomedical data analyses,” Nature Biotechnology, 2017. [24] M. R. Crusoe, S. Abeln, A. Iosup, P. Amstutz, et al., “Methods included: Standardizing computational reuse and portability with the Common Workflow Language,” Communications of the ACM, 2022. [25] OpenWDL, “Workflow Description Language (WDL),” 2026. [26] J. Köster and S. Rahmann, “Snakemake—a scalable bioinformatics workflow engine,” Bioinformatics, vol. 28, no. 19, pp. 2520–2522, 2012. [27] M. Albrecht, P. Donnelly, P. Bui, and D. Thain, “Makeflow: A portable abstraction for data intensive computing on clusters, clouds, and grids,” in ACM SIGMOD Workshop on Scalable Workflow Enactment Engines and Technologies, 2012. [28] P. Di Tommaso, M. Chatzou, E. W. Floden, P. P. Barja, E. Palumbo, and C. Notredame, “Nextflow enables reproducible computational workflows,” Nature biotechnology, vol. 35, no. 4, pp. 316–319, 2017. [29] A. Jain, S. P. Ong, W. Chen, B. Medasani, et al., “FireWorks: a dynamic workflow system designed for high-throughput applications,” Concurrency and Computation: Practice and Experience, 2015. [30] L. Ward, G. Sivaraman, J. G. Pauloski, Y. Babuji, et al., “Colmena: Scalable machine-learning-based steering of ensemble simulations for high performance computing,” IEEE/ACM Workshop on Machine Learning in High Performance Computing Environments (MLHPC), 2021.
R EFERENCES [1] R. F. Da Silva, R. M. Badia, D. Bard, I. T. Foster, S. Jha, and F. Suter, “Frontiers in scientific workflows: Pervasive integration with high-performance computing,” Computer, 2024. [2] K. Thareja, H. Safri, R. Mayani, A. Mandal, and E. Deelman, “From specification to execution: Ai assisted scientific workflow management,” Preprint arXiv:2606.18425, 2026. [3] B. Balis, M. Orzechowski, P. Kica, M. Dygas, and M. Kuszewski, “From research question to scientific workflow: Leveraging agentic ai for science automation,” Preprint arXiv:2604.21910, 2026. [4] E. Deelman, K. Vahi, G. Juve, M. Rynge, et al., “Pegasus, a workflow management system for science automation,” Future Generation Computer Systems, vol. 46, pp. 17–35, 2015. [5] Y. N. Babuji, A. Woodard, Z. Li, D. S. Katz, et al., “Parsl: Pervasive parallel programming in Python,” in International Symposium on HighPerf. Parallel and Distrib. Comp. (HPDC), 2019. [6] A. Alsaadi, M. Hooten, M. Goliyad, A. Merzky, et al., “RHAPSODY: Execution of hybrid AI-HPC workflows at scale,” Preprint arXiv:2512.20795, 2025. [7] A. Kamatar, J. G. Pauloski, Y. Babuji, R. Chard, et al., “Empowering scientific workflows with federated agents,” in IEEE International Parallel and Distributed Processing Symposium (IPDPS), 2026. [8] W. Shin, R. Souza, D. Rosendo, F. Suter, F. Wang, P. Balaprakash, and R. Ferreira da Silva, “The (R)evolution of scientific workflows in the agentic ai era: Towards autonomous science,” in Proceedings of the SC’25 Workshops of the International Conference for High Performance Computing, Networking, Storage and Analysis, 2025, pp. 2305–2316. [9] H. Ma, A. Brace, C. Siebenschuh, G. Pauloski, I. Foster, and A. Ramanathan, “Connecting large language model agent to high performance computing resource,” arXiv preprint arXiv:2502.12280, 2025. [10] M. Masera, A. Leone, J. Köster, and I. Molineris, “Snakemaker: Seamlessly transforming ad-hoc analyses into sustainable snakemake workflows with generative ai,” arXiv preprint arXiv:2505.02841, 2025. [11] K. Alam and B. Roy, “From prompt to pipeline: Large language models for scientific workflow development in bioinformatics,” arXiv preprint arXiv:2507.20122, 2025. [12] D. J. B. Clarke, J. E. Evangelista, Z. Xie, G. B. Marino, A. I. Byrd, M. R. Maurya, S. Srinivasan, K. Yu, V. Petrosyan, M. E. Roth, et al., “Playbook workflow builder: Interactive construction of bioinformatics workflows,” PLOS Computational Biology, 2025.
8