PegasusAI Studio: Legible Cyberinfrastructure and Accountable AI Agents for Scientific Workflows

PegasusAI Studio: Legible Cyberinfrastructure and Accountable AI Agents for Scientific Workflows Komal Tharejaa , Mats Ryngeb , Berent Aldikactid , Angelina Juc , Ian Lumsdenc , Anirban Mandala , Rajiv Mayanib , Kin Wai Ngc , Hamza Safrib , Michael Stealeya , Sai Swaminathanc , Michela Tauferc , Karan Vahib , Michael Zinkd , Ewa Deelmanb a University of North Carolina at Chapel Hill, Chapel Hill, NC, USA b University of Southern California, Los Angeles, CA, USA c University of Tennessee, Knoxville, Knoxville, TN, USA

arXiv:2611.13309v1 [cs.DC] 9 Nov 2026

d University of Massachusetts Amherst, Amherst, MA, USA

Abstract AI coding agents can now construct scientific workflows, but operating them on shared cyberinfrastructure (CI) raises problems that code generation does not solve. Agents need current knowledge of the workflow system and its execution sites and access to the runtime state of distributed executions, while facilities need to attribute every agent action to a user and bound it by that user’s privileges. We present PegasusAI Studio, an AI-assisted science gateway built on the Pegasus and HTCondor stack. Studio makes CI legible to agents by projecting one facility-curated knowledge base into the native configuration of several coding agents, by introspecting workflow interfaces and execution sites, and by grounding diagnosis in workflow and scheduler telemetry. Agent actions operate within a trust envelope that combines federated authentication, per-user execution environments and scheduler credentials, provider-neutral model access, observability, and approval controls for state-changing actions. The same application runs as a JupyterHub-based gateway on a research cloud and as an Open OnDemand application on HPC systems. We evaluate Studio through case studies of legacy CASA weather workflow modernization, a gravitational-wave reanalysis, an EEG analysis built with an open-weight model, and an Event Horizon Telescope (EHT) M87 image reconstruction that recovered from two failed runs to a zero-failure execution. The evaluation establishes that these mechanisms operate as designed. Keywords: AI agents, science gateways, scientific workflows, cyberinfrastructure, accountability, PegasusAI Studio, Pegasus WMS

1. Introduction

Most large scientific computations now run on shared cyberinfrastructure (CI): campus clusters, national allocations, research clouds, and high-throughput pools that many users and projects share. Workflow management systems make that infrastructure usable. Pegasus [1] and HTCondor [2], for example, run production workloads across many scientific domains, handling data staging, retries, recovery, and provenance so that a scientist does not have to. AI coding agents can now write substantial scientific software, including the workflows these systems execute. Writing the workflow, however, is only part of the work. Shared CI is a heterogeneous execution environment in which computation, data, software, and services are distributed across resources and sites. Operating an agent there requires three things a researcher’s laptop does not supply. First, the agent needs current knowledge of workflow-system and site conventions. Second, it needs access to the runtime state of distributed executions, which lives in the workflow database and the scheduler rather than in a terminal. And finally, the facility needs an authorization model that attributes every agent action to a user and bounds it by that user’s privileges. Our companion study of AI-assisted workflow construction [3] found that workflow logic was usually not the dominant source of effort. Across a twelve-workflow corpus, much

of the effort instead went into runtime and configuration debugging involving containers, schedulers, data movement, software environments, and remote execution. This echoes earlier experience integrating an Earth science workflow end to end across distributed cyberinfrastructures, where similar integration challenges dominated [4]. That finding motivated the system described here. We present PegasusAI Studio, an AI-assisted science gateway built on the Pegasus and HTCondor stack. Its design follows two principles. First, CI must be made legible to AI agents, by exposing system knowledge, operational conventions, interfaces, execution sites, and runtime state in machine-consumable form. Second, AI agents must be made accountable to the CI, by placing their actions inside the same identity, authorization, and auditing mechanisms that govern human users. Studio realizes legibility through three mechanisms. Knowledge projection compiles one facility-curated knowledge base into the native configuration of several coding agents. Structural and site introspection exposes workflow interfaces and execution sites. Telemetry grounding gives the agent workflow and scheduler state in place of terminal output. Studio realizes accountability through a trust envelope that combines federated authentication, per-user execution environments, per-user scheduler tokens, provider-neutral model access, observability, and user approval controls for state-changing actions. The same

Studio application image can be deployed in two environments: as a JupyterHub-based science gateway on a research cloud or as an Open OnDemand application on an HPC cluster. We evaluate Studio through case studies drawn from real workflow development on the deployed gateway: the modernization of three legacy CASA radar workflows, a gravitational-wave reanalysis, an EEG group analysis built with an open-weight model, and a reconstruction of the Event Horizon Telescope (EHT) M87 imaging pipelines [5, 6] that recovered from two failed runs to a zero-failure execution. The evidence is observational. It shows each mechanism operating as designed, and it does not yet quantify how much effort the mechanisms save. The main contributions are:

simultaneously running jobs across geographically distributed resources [2, 7]. PegasusAI Studio therefore does not reimplement scheduling, data movement, retries, recovery, or distributed execution in the agent layer. Instead, it builds on these established capabilities and focuses the AI layer on workflow construction, operation, and diagnosis. 2.2. Science gateways A science gateway gives a community browser-based access to computation, data, and domain tools [8]. Studio builds on two common patterns: JupyterHub [9], which authenticates users and provisions per-user environments, and Open OnDemand [10], which provides browser access to HPC resources. CILogon [11] provides federated identity for the gateway deployment.

1. An end-to-end architecture for AI-assisted workflow operation on shared CI, built on the Pegasus and HTCondor stack without modifying either and organized around mechanisms that are applicable to other workflow design and execution systems (Section 4). 2. Knowledge projection, which compiles one curated body of workflow and site knowledge into the native configuration of multiple coding agents (Section 5). 3. A telemetry-grounded workflow lifecycle, in which agents diagnose runs from structured workflow and scheduler state rather than from terminal output (Section 6). 4. A trust envelope for accountable agents, which ties each agent action to the user’s identity and authorization and provides user-approval controls over state-changing operations (Section 7). 5. Deployment evidence from workflows built and run inside Studio, including CASA nowcasting modernization, gravitational-wave reanalysis, EEG analysis, and the EHT M87 reconstruction (Section 8), with the application and knowledge base released as open source (Section 11).

2.3. AI coding agents and workflow construction context Modern coding agents iteratively plan, invoke tools, observe results, and revise actions under a permission model. The Model Context Protocol (MCP) [12] exposes documentation and service tools to agents. Studio integrates the PegasusAI construction toolchain evaluated separately in the companion methodology paper [3]: documentation grounding, reusable skills, spec-first interaction, an iterative development lifecycle, and codified workflow failure-mode knowledge. This paper takes that construction methodology as given and asks a different question: what systems mechanisms are needed to operate AI-assisted workflow development on shared CI? Studio provides the knowledge distribution, runtime visibility, identity, authorization, and gateway environment for that setting. 3. Related Work Workflow systems such as Pegasus [1], Nextflow [13], Snakemake [14], CWL [15], Parsl [16], and Galaxy [17] provide portable and reproducible execution, but generally assume the user can already express a workflow in the system’s interfaces. Science gateways such as Open OnDemand [10], Galaxy, HUBzero [18], and JupyterHub [9] provide mature web access to shared computation and solve multi-user concerns such as authentication and resource isolation, but do not by themselves give AI agents workflow-system knowledge, runtime telemetry, or an accountable execution identity. Recent work has begun to connect AI assistants and agents to research-computing environments. Science-gateway efforts have explored AI-assisted user support and the broader role of AI in gateway-based research computing [19, 20, 21], while other work has connected LLM agents to HPC resources through parallel execution frameworks such as Parsl [22]. These systems demonstrate that AI assistants can operate alongside or invoke shared computing resources. PegasusAI Studio focuses on a complementary systems problem: integrating the agent with the workflow lifecycle while exposing structured execution state and preserving the identity, authorization, and audit mechanisms of the underlying facility.

2. Background and System Context 2.1. Scientific workflows and the execution stack Pegasus [1] represents a scientific computation as a resourceindependent directed acyclic graph (DAG) and plans it onto concrete execution sites, inserting data staging, container handling, retries, and provenance capture. Studio uses Pegasus without modification and submits workloads through HTCondor [2], a system designed to manage high-throughput job execution across distributed resources. HTCondor’s per-user identity tokens, hold reasons, and execution state are important because they provide both accountability and diagnostic information. Pegasus and HTCondor provide a production-proven execution foundation for large-scale scientific computing. Pegasus has been used to execute workflows with up to millions of tasks across heterogeneous distributed resources, including clusters, national CI, and clouds, with mechanisms for data management, failure recovery, provenance, and portable execution across environments [1]. HTCondor has similarly evolved over decades of production use in distributed high-throughput computing and has been demonstrated at scales of hundreds of thousands of 2

4.1. Design constraints that shaped the system

Table 1: Studio mechanisms that connect AI agents to cyberinfrastructure, expose cyberinfrastructure state to agents, and govern agent actions.

Direction

Mechanism in Studio

Agent access to CI

Chat on a coding-agent engine with specialist agents and the Pegasus tools reachable over MCP, agent-enabled terminals, and skills embedded in the workflow IDE. A JupyterLab sidebar chat aware of the notebook the user has open. The knowledge projection layer (Section 5). Telemetry grounding (Section 6). Argparse introspection. Per-workflow placement on the shared pool or the user’s own annex, and an administrator’s site catalog under Open OnDemand (OOD) [10]. MCP documentation servers. Federated identity, per-user tokens, approval posture, a provider-neutral model gateway with a prompt guard, and per-user usage telemetry with consent-gated content capture (Section 7).

CI state to agent

CI governance of agent

Three constraints recur throughout the design, and every architectural choice below can be traced to one of them. First, the workflow stack is unmodified: Studio integrates Pegasus and HTCondor only through their standard interfaces. This is deliberate. Rather than reimplementing scheduling, data movement, recovery, provenance, and distributed execution in the AI layer, Studio relies on the mature capabilities of the underlying workflow and scheduling systems. This keeps the approach portable to existing installations and keeps the trust argument of Section 7 grounded in mechanisms providers already accept. Second, AI assistance must remain optional and degrade gracefully when unavailable: a user with no model key still gets a full workflow environment, and an institution can disable or substitute any agent. Third, the system must operate behind arbitrary reverse-proxy prefixes, since gateway sessions live under per-user URL paths. This last constraint sounds minor but consumed a disproportionate share of engineering effort. We return to it in Section 9. 4.2. Two tiers Studio has two tiers: a self-contained application container that gives one user a complete workflow development environment, and a platform tier that spawns, authenticates, and provisions one container instance per user for a community of users. The split is deliberate. The container runs anywhere a single user can run Docker [25] or Apptainer [26], supporting laptop and lab-host use, while the platform adds identity and shared compute without changing the image. Figure 1 shows both tiers. The platform tier supports two deployment models. The first is the public science gateway described in Section 4.2.2: JupyterHub on a Kubernetes [27] cluster, with federated login, one pod per user, and a shared HTCondor pool. The second is an OOD application for sites that already run OOD on an HPC cluster. For this deployment, the same application is packaged in an Apptainer image on a Rocky Linux base, so that the cluster’s own Slurm [28] client tools are bound into the container, and a standard Batch Connect app launches it as the logged-in POSIX user on a compute node. Each session gets its own Studio home directory rather than the user’s real home, internal services use per-session port because Apptainer shares the host network namespace, and a session token gates the front proxy. Workflow jobs reach the cluster’s Slurm queue through an HTCondor access point. The OOD deployment was validated end to end with three test users on a three-node Slurm cluster on the FABRIC testbed [29]. A head node runs the Slurm controller, the HTCondor access point, and the OOD portal, and two compute nodes run Slurm. A workflow built in a session ran to completion with all 12 of its jobs executed as Slurm jobs. This deployment has not yet been exercised at a production site. In both deployment models, the platform changes how the container is launched, authenticated, and connected to compute resources, but not the application environment inside the container.

AI coding assistants and scientific agents [23, 24] provide code generation, chat, and tool use, while retrieval-augmented generation can ground models in current documentation. Existing integrations are typically editor- or application- scoped. Studio differs by integrating the agent with the workflow lifecycle, execution telemetry, gateway identity, and resource-provider controls rather than only with source code. The companion methodology paper addresses the separate question of how agents construct workflows from scientific intent and existing artifacts [3]. 4. PegasusAI Studio: Architecture The PegasusAI workflow-construction methodology was initially developed in a command-line setting, where one operator configured the agent context, worked from a local shell, and diagnosed remote execution failures through logs and terminal output. Although adequate for studying workflow construction, this setting did not give the agent direct access to the mechanisms shared facilities use to distribute site knowledge, expose execution state, and attribute actions to users. This limitation became particularly important because the companion evaluation found that much of the remaining workflow-construction effort arose from runtime and configuration problems. Studio addresses this gap by moving the same construction methodology into a facility-managed environment that exposes CI knowledge and runtime state while preserving facility identity and authorization controls. Studio must support two directions of integration. CI must expose system knowledge and execution state in forms agents can use, while agent actions must remain inside the identity, authorization, and auditing mechanisms of the facility. We call these requirements legibility and accountability, respectively. Table 1 maps them to the corresponding Studio mechanisms, and the following sections describe them.

4.2.1. The Studio container A single linux/amd64 image bundles four kinds of components behind one reverse proxy: 3

Figure 1: The two tiers and the two platform deployments. One container image holds a complete single-user workflow environment. The platform tier that spawns and authenticates it is either the JupyterHub gateway on a research cloud or an OOD app on a campus cluster. Every process in the container, agents included, runs as the user’s own identity with the user’s scheduler token, over a Pegasus and HTCondor stack. Model access goes through a provider-neutral gateway with a prompt guard, and usage telemetry flows to a collector inside the platform.

• User interface. A web application whose views follow the order in which a scientist works: a home page that hands a prompt to the assistant, an analyses page for workflow projects and runs with a live graph, a data page for uploads, an examples page that lists community workflows from tagged public repositories, notebooks, a resources page for choosing where workflows run, and settings. A guided tour introduces the interface, and a single Run button prepares, builds, plans, and submits a workflow. The chat surface is also embedded in JupyterLab as a sidebar extension that knows which notebook the user currently has open and can read and edit its cells. Figures 2, 4, and 6 show the current design.

• Workflow stack. The container runs Pegasus WMS, an HTCondor access point, and Apptainer for containerized jobs. Because these are the standard components, the user plans and submits workflows through the same interfaces used at a conventional Pegasus site. A fourth component, workflow-monitor, is a small read-only service that follows the Pegasus workflow database and scheduler state and converts them into the structured event stream that both the user interface and the agents consume (Section 6). • AI agents. Several interchangeable coding agents, at the time of writing Claude Code [30] (Anthropic), OpenCode [31] and Crush [32] (both open source), Codex CLI [33] (OpenAI), and Antigravity [34] (Google), installed system-wide. The web chat is a client of the OpenCode engine running in the user’s own container, with the Pegasus tools (workflow scanning, planning, submission, run diagnosis) exposed to it over MCP. In its default mode, the chat routes each message to one of the specialist agent roles of Section 5, so a failure report reaches the workflow debugger and a design request reaches the workflow architect. The agents are ordinary third-party tools. Studio

• Control API. A service that coordinates application state and actions: workflow scanning, planning and submission, run monitoring, container image builds, AI tool installation and configuration, chat engine dispatch, and notifications. State lives in a per-user SQLite database inside the user’s workspace, such that application state has no external service dependencies. 4

Figure 2: The Studio home page. The navigation rail on the left lists the views in the order a scientist works (home, analyses, data, examples, resources, notebooks), followed by the chat, terminal, website, feedback, and settings entries. The home page offers four starting actions and four example requests that each hand a prompt to the assistant, and a link that starts the guided tour.

configures them (Section 5).

outlives the pod and the token is re-minted at every spawn, the user’s pod starts with persistent storage and a valid scheduler credential, and every job the user (or an agent acting on behalf of the user) submits carries the user’s own identity into the HTCondor pool (Section 7). Compute follows the same per-user model. The HTCondor pool has a central access point and a set of execution points. Execution points include cloud instances, which a small controller service scales with demand, and privileged pods inside the cluster. The pool also flocks to the OSPool [36] for users who supply an OSPool project. A user who needs more than the shared pool can lease nodes, including GPU nodes, from any Slurm clusters on which they hold an allocation, through HTCondor’s annex mechanism (Section 7). Three further platform services sit beside the pool: a model gateway and prompt guard, a telemetry collector with its dashboards, and an in-cluster trace store, all described in Section 7. GitOps tooling reconciles the entire platform from a git repository, so the deployment itself is versioned and reviewable.

The container keeps no user accounts of its own. Whatever spawns it (JupyterHub, OOD, or a plain docker run) is responsible for identifying the user, and the container runs as the UID it is given. Access to the application is then gated at the front proxy using the authentication mechanism provided by the deployment environment. Under JupyterHub, every request (pages, API, terminal, and JupyterLab) is checked against the hub through its OAuth flow, and only the pod’s owner or a JupyterHub administrator is admitted. Under OOD a per-session token gates the proxy. By keeping user identity and multi-user authentication outside the application image, the same container can run in a local single-user setting or inside a federated multi-user gateway. The platform tier delegates authentication, user isolation, and session management to mature components such as JupyterHub, CILogon, and Open OnDemand rather than reimplementing those capabilities inside Studio. 4.2.2. The gateway platform The gateway platform is the multi-user tier: the components that authenticate users and provision one Studio container per user. The public deployment runs on a self-managed Kubernetes cluster on the Jetstream2 research cloud [35]. Users authenticate through CILogon [11] with their home institution credentials, with Authentik brokering that identity to JupyterHub (Figure 3). A hook in the JupyterHub spawner, the component that creates a user’s pod, runs before each session. It creates the user’s home directory on a shared NFS server if absent, mints a fresh HTCondor identity token scoped to that user into that home directory, and mounts the home into the pod. Because the home

5. Knowledge Projection: Making CI Legible to Agents Before Studio was introduced, in the command-line phase, workflow and failure-mode knowledge was maintained in context files configured for individual projects, users, and agent tools [3]. That arrangement made the quality of an agent’s context depend on who had configured it and allowed copies to diverge as tools and users multiplied. Studio instead treats this knowledge as facility-managed infrastructure: it is curated, maintained centrally, versioned with the platform, and projected into each supported agent’s native configuration for every user. 5

Figure 3: The public gateway’s sign-in page. A user signs in through CILogon with their home institution’s credentials, and the workspace is created on first sign-in. The page summarizes the three-step lifecycle (describe, run, watch) and the parts of the workspace.

tions and context. An agent role is the instruction set for one such specialized agent, designed for a longer-running task, for example a workflow architect that designs a DAG, a data engineer that writes fetch and harmonize steps, a workflow debugger that works through a failed run. The web chat’s default mode routes each message to one of these roles.

5.1. One canonical knowledge base The knowledge base is maintained in a single canonical opensource repository, the PegasusAI plugin marketplace [37], which contains the skills, agent roles, templates, and documentationserver definitions described below. The Studio container checks it out at a pinned revision. Studio keeps no second copy: its continuous-integration build fails if one reappears in the application repository, because an earlier arrangement with two hand-maintained copies led them to diverge (one copy taught Docker while the other taught Apptainer). The checkout retains its version history, so an administrator can advance or roll back the knowledge inside a running container without rebuilding the application image. The repository contains four kinds of artifacts:

• Templates and examples. Complete, working workflow generators (for example, a fan-out and fan-in over many inputs, or a fetch-train-predict workflow) and configuration templates (site catalogs for the Studio pool and for ACCESS [38], container definitions, resource profiles). Skills point the agent to these files so that implementation guidance can be grounded in executable examples rather than prose alone.

• Skills. Procedural expertise for recurring tasks, each a canonical markdown document with structured metadata: scaffold a workflow from a workflow description, convert a Snakemake or Nextflow workflow, generate a wrapper script for a workflow step, produce an Apptainer definition for a tool stack, review a workflow for known pitfalls, diagnose a failed run, answer questions from the documentation and code examples, and provision execution resources. These are the same skills as the companion skill catalog [3], now curated by the facility rather than the user.

• Documentation servers. A registry of Model Context Protocol (MCP) [12] endpoints exposing the documentation of Pegasus and of Kiso [39] as searchable tools, so agents retrieve current reference material instead of relying on training data. Kiso is an experiment-management tool for provisioning HTCondor pools on cloud and edge testbeds and is exposed through the /kiso skills [3]. The result is a single versioned body of operational knowledge maintained by the workflow-system and gateway administrators and projected consistently across users and agent tools.

• Agent roles. Supported coding agents can delegate longer tasks to specialized agent instances with their own instruc6

Studio includes that definition, so a generated workflow never names a cluster. On the gateway, compute is Studio’s own pool. Extra capacity comes from the user’s HTCondor annex on a cluster they hold an allocation on (Section 7) or from flocking to the OSPool, not from a different site. The run form then offers two placements per workflow: everywhere except the annex, which is the default, or only the user’s annex (Figure 4). Studio records the choice as a scheduler requirement in the workflow’s own Pegasus properties, so a pegasus-plan typed in a terminal places jobs exactly as the Run button does. Under Open OnDemand, an administrator supplies the site catalog for the host cluster, and the same compute name points at it. The agent plans against exactly the site the deployment defines, so the local policy of Section 5.5 (preferred sites, allocations) is expressed once rather than in every generated workflow.

5.2. Per-tool adapters Each supported agent tool reads its configuration in a different format and from different files. An adapter is a small program that translates the canonical knowledge base into one tool’s format. When a user enables an AI tool, its adapter compiles the canonical knowledge into that tool’s native configuration in the user’s workspace: a CLAUDE.md and .mcp.json for Claude Code, an AGENTS.md with opencode.json and skill and agent directories for OpenCode, an AGENTS.md for Codex, a context file for Antigravity, and the equivalent for Crush, as well as configurations for Studio’s own JupyterLab sidebar and the web chat. Adapters write to the user’s home directory so the configuration is global to the user’s session rather than tied to one project checkout, and re-running an adapter reconciles the output with the current knowledge version. The same adapter also carries per-user settings that must reach the agent, such as the telemetry consent of Section 7. When such a setting changes, Studio restarts the agent engine so that the updated configuration takes effect immediately rather than waiting for a new session. The pattern is one-to-many by construction: N supported tools require N thin adapters over one knowledge base, and supporting a new agent means writing one additional adapter to that tool’s configuration interface rather than maintaining a separate copy of the knowledge base. In our experience each adapter is on the order of a hundred lines of code, because the substantive work of deciding and maintaining what an agent needs to know about the workflow system is centralized in the canonical layer.

5.5. Why projection rather than per-tool authoring The projection design rests on three expected benefits: (1) It should keep expertise current, because a fix to a skill (for example, a newly learned rule from the companion failure-mode table [3] about how fetch jobs must handle missing credentials on execute nodes) reaches every tool on the next reconciliation. (2) It should keep expertise consistent, because two agents given the same task draw on the same curated guidance, which matters when users switch tools mid-project. And (3), it should lower the barrier for institutions, because an execution site can encode local policy (preferred sites, container registries, data placement rules) in one place and project that guidance into every supported agent. The OpenCode projection has been exercised end to end: an open-weight model working from the compiled knowledge base scaffolded, debugged, and executed a new EEG analysis workflow to completion on the gateway, providing an end-to-end test of the projected knowledge with a different agent and model stack in Section 8.

5.3. Structural legibility: interface introspection Legibility extends beyond documents into program structure. Every Pegasus workflow project in Studio exposes a generator script whose command-line interface defines the workflow’s parameters. Studio parses that interface by analyzing the script’s abstract syntax tree, without executing user code, and derives both the parameter form shown to the user and the parameter schema given to the agent. The same mechanism means an agent-authored generator can become immediately operable from the UI: if the agent writes a conventional argparse interface, the platform can run it. A convention thereby becomes a contract that binds both sides: the platform knows how to operate generators that follow the convention, and the knowledge base instructs every agent to follow it. Figure 4 shows the result: a workflow project page whose parameter form is derived from the generator’s own interface. The contract has been exercised on agent-authored work: the gravitational-wave case of Section 8 was launched from the derived form, the first launch surfaced an interface defect at exactly that boundary, and the fixed workflow ran to completion from the form.

6. The Telemetry-Grounded Lifecycle The companion methodology evaluation found that much of the remaining workflow-construction effort arose during runtime and configuration debugging [3]. In the command-line setting, the agent saw only the execution information surfaced through terminal output and manually opened logs, while the authoritative state remained in the workflow database and scheduler. Studio therefore treats the workflow lifecycle, rather than the editor, as the unit of integration. Figure 5 shows the lifecycle and the structured information exchanged between its stages. 6.1. Stage by stage Generate. A workflow starts from a community example (fetched live from tagged public repositories), a skill-driven scaffold, or existing user code. The generator’s introspected interface (Section 5) yields the parameter form and schema. Plan and submit. Studio invokes the standard Pegasus planner and records the run. Before planning, it checks the workflow’s configuration against the target site. A workflow written for a cluster with a shared filesystem can fail when run

5.4. Site legibility: one execution site A third kind of legibility concerns where a workflow runs. The Pegasus convention used by Studio generators is that a generator describes only the local site (submit-host scratch and output storage) and plans against a single execution site named compute, whose definition comes from a site catalog. 7

Figure 4: The run panel of a workflow project in Studio, here for a streamflow workflow the assistant built from one chat prompt. The settings (gauge list, lookback window, retry policy, minimum number of gauges, and an optional site catalog) are derived from the generator’s argparse interface by abstract-syntax-tree introspection, without executing user code. Every field, default, and help string comes from the interface the agent wrote. Below the settings the user chooses where the workflow runs (Studio’s shared pool, or only nodes the user connects from their own cluster or allocation). One button prepares, plans, and submits the workflow.

on an HTCondor pool whose workers do not share a filesystem. In that case, jobs may fail without an error that clearly identifies the configuration mismatch. Studio therefore stops such a run before submission, with a message naming both the setting and the deployment. This is a recurring failure mode encoded directly in the platform rather than left only as agent knowledge. Nothing about planning is AI-specific, which is the point: the agent operates the same tools as the users, so its actions are reviewable in the same terms. Monitor. A monitoring service follows each run by reading the Pegasus Stampede workflow database [40, 41] and scheduler state, and emits a structured JSONL event stream: job state transitions, hold reasons, stage-in and stage-out progress, and resource usage. This service is required because the state of a run is split between two stores that are built for querying rather than for streaming: the workflow database knows which jobs exist and what each one produced, while the scheduler knows which jobs are queued, running, or held, and why. The service merges the two into one time-ordered stream that the user interface renders and that the agent reads as its window into execution. The agent therefore reasons from structured workflow and scheduler state rather than scraping terminal output. Figure 6 shows the human-facing rendering: a live graph of a run, with each node colored by its current scheduler state and each step labeled with the execution host it ran on. The monitor also flags a step that has run at least ten minutes and five times longer than the median successful duration of the same step in this run and the previous five runs of the workflow. That check came from an annex test in which a data-fetch step that usually takes

seconds held a worker slot for forty minutes while the run page showed no failure. The graph collapses the fan-out of one step over many inputs into a single node with a count, can expand to every task and to the auxiliary Pegasus jobs, and can show the files each step reads and writes as nodes of their own, with a preview of each file. Figures 4 and 6 show the same streamflow workflow, a tutorial example built inside Studio. A single request asked the agent to fetch 30 days of discharge for five USGS gauges, compute per-gauge statistics and a seven-day rolling average, produce a combined table and plot, parameterize the gauge list and lookback window, and tolerate individual fetch failures while failing the run if fewer than two gauges returned data. The assistant scaffolded the workflow, ran it, and worked through failed runs using the diagnose step until the execution in Figure 6 succeeded. In this case the failure information becomes part of the same lifecycle that Studio monitors and the agent diagnoses. Diagnose. On failure, Studio collects the analyzer’s report (pegasus-analyzer), the relevant job logs, and the hold reasons, and hands them to the chat engine as structured context. The diagnostic skill matches that context against a table of recurring failure modes, each recorded as an error signature, its cause, and a fix. The modes fall into four families: • Missing credentials on execute nodes. HTCondor starts each job in a clean environment, so an API key exported in the submit shell never reaches the job or its container, and the fetch step fails on the worker with an error that does not name the cause. The fix is to read the key when the workflow is generated and inject it into the job’s envi8

shows the files that step read and wrote. Common result formats open in place: tabular files render as tables, images and PDFs inline, and long text files as a partial preview. Every inline preview except raster images and PDFs is served sandboxed, because workflow outputs such as HTML reports can carry scripts. The same run directory opens in JupyterLab for further analysis, which is how the EEG and streamflow outputs in this article were inspected. The agent reads the same files through its tools, so a question such as “does the summary table have a row for every gauge” is answered from the result itself, not from the job log. For a scientist this is the step that turns a successfully executed workflow into a result they can actually inspect and use. Figure 7 shows the stage for the EHT reconstruction’s run inside Studio (Section 8.6): the run’s result files, with one day’s reconstructed images opened in place. Studio is introspective in that it instruments its own assistance lifecycle. For every failed run it keeps one record, which we call a failure episode. The episode follows what happens after the failure: whether the user asked the assistant about it, whether the user re-ran the same workflow, and whether that re-run reached a terminal state. Studio stores each of these steps with its time in the user’s own database. At the point where the answer is meaningful, after the re-run finishes, the user is asked one question: did the assistant’s suggestion fix it (Figure 8 in Section 7)? The prompts are rate-limited server-side so that they do not become a survey users learn to dismiss. This is the per-episode measurement that the command-line setting lacked (Section 8.2). At the time of writing, no feedback data had yet been collected for evaluation.

Figure 5: The workflow lifecycle as the unit of integration. Each transition makes structured information from the workflow and execution systems available to Studio, including planner output, scheduler state, the JSONL event stream, and analyzer reports, together with the introspected argparse schema derived from the workflow generator. Proposed fixes return to the workflow as patches, closing the diagnose, fix, and resubmit lifecycle through replanning and resubmission rather than relying on terminal scraping.

ronment. • Catalog and container mismatches. A container declared without the site that holds its image makes planning fail with a container-site mismatch, followed by an error that no physical file name can be selected for the image. A separate signature, an image that is “not a valid squashfs image”, traces to a build that reported success but produced an empty container.

6.2. Why structured signals matter Grounding the agent in structured workflow and scheduler telemetry has two effects. It makes agent conclusions checkable, because every claim about a run traces to an event or analyzer line a human can inspect. It also gives the diagnose, fix, and resubmit lifecycle a concrete starting point: the diagnose step starts from the actual failure record rather than from a user’s paraphrase of it. The CASA nowcast case study (Section 8.3) shows this process identifying three real defects that terminal output alone had obscured. In the current implementation, workflow-monitor is a read-only tool alongside Pegasus. Its integration into Pegasus itself through the MonitorD plugin system is in progress and is discussed further in (Section 9.4).

• Stage-out failures that hold a DAG. When a wrapper exits before it writes its declared outputs, HTCondor cannot transfer them and places the job on hold. It reports the missing files, not the cause, and the workflow stops making progress. The skill directs the agent to the job’s own error log, where Pegasus records the real failure. • Site misconfiguration. Planning fails when no site catalog defines the requested execution site, or when the generator registered its transformations for a different site than the one the workflow is planned for. Studio avoids the second case by passing the same site to both steps from the run panel’s choice of where the workflow runs.

7. The Trust Envelope: Making Agents Accountable to CI

The list grows as new failures are traced to recurring causes. Fix and resubmit. Proposed changes are ordinary edits to the workflow project, made through the agent’s guarded tools. Studio checks each edited generator for syntax errors and for a parseable argparse interface before the workflow is replanned, so an edit cannot silently break the interface contract of Section 5. Results. The lifecycle ends with the scientist examining the workflow’s outputs, and Studio treats that as a lifecycle stage rather than leaving the user to find files in a submit directory. Each run lists its result files, and clicking a step in the graph

The command-line construction setting used the operator’s shell, credentials, and model configuration [3]. A shared facility needs a different operational boundary: it must be able to attribute an agent’s jobs and resource use to an authenticated user and to limit the agent to that user’s privileges. This requirement is particularly important for AI agents because they can initiate actions that the user has not individually specified. Studio therefore runs agent actions under the user’s identity and combines that identity with explicit controls over which state-changing actions may proceed. 9

Figure 6: Studio’s live graph view of a completed run, driven by the same event stream the agent reads, beside the chat in its workflow-debugger mode. The workflow (five stream gauges fetched, analyzed, and combined) was built from one chat prompt, and the chat transcript on the right is the agent explaining the fix that made this run succeed after an earlier failure. Node colors track live scheduler state (succeeded, running, pending, failed, held).

7.1. Identity: the agent is tied to the user

7.2. Authorization: approval posture

The chain runs CILogon to Authentik to JupyterHub to a peruser pod. At spawn, the platform mints an HTCondor IDTOKEN for the authenticated user and mounts it, together with the user’s NFS home, into the pod. Every process in the pod, including every AI agent, runs under the user’s UID with the user’s token. A job submitted by an agent is, to the scheduler and to the audit trail, a job submitted by that user. There is no service account, no shared credential, and no privilege the agent holds that the user does not. The same rule holds in the Open OnDemand deployment described in Sec. 4.2, where the portal authenticates the user, the session runs under the user’s POSIX account on a compute node, and workflow jobs enter the cluster’s Slurm queue under that account. There Studio keeps its own home directory and never writes into the user’s real home, because a cluster home already holds the user’s own configuration of the same agent tools. The rule also extends to resources the user brings. A user whose workflows need more than the shared pool can lease nodes from Slurm clusters on which they hold an allocation, including campus clusters or ACCESS resources. This is accomplished using HTCondor’s Annex mechanism. Studio generates a selfcontained job that the user submits to the remote cluster’s Slurm queue, and the nodes it acquires join the pool as an annex named after the user. The run form lets the user send a workflow’s jobs only to that Annex (Figure 4), and only that user’s jobs may run on it, so the allocation remains associated with the user who owns it, even when an agent submits work on that user’s behalf. Idle Annex nodes shut themselves down. The first live Annex test, on SDSC Expanse [42], ran the EHT workflow of Section 8.6 to completion with all 31 of its compute steps on the user’s annex nodes.

Identity links an agent’s actions to a user. Authorization decides which of those actions the agent may take on its own. Every coding agent in Studio has a built-in permission mechanism: before the agent writes a file, runs a command, or submits a workflow, it can be configured to present the proposed action to the user and wait for a yes or no. Studio configures every agent with an approval policy for state-changing actions, with two refinements learned while exercising the deployment. Edits inside the user’s workflow and data directories proceed without a prompt, while writes anywhere else in the home directory always ask, because an unprompted write to a shell profile or to the agent’s own configuration could alter the agent’s permission configuration. Workflow submission always asks, even when the user has turned off confirmation for routine shell commands. We describe the resulting autonomy on a three-level scale. At assist, the agent only answers questions and proposes changes, and the user applies them. At approve, the agent applies changes and runs commands itself, but state-changing actions outside the user’s workflow and data directories, and every workflow submission, require the user’s confirmation. At autopilot, the agent acts without confirmation. The deployed default is approve: the agent may operate within the user’s project workspace, while higher-risk writes and workflow submission remain approvalgated. Read-only actions (inspecting events, reading logs, querying documentation) need no approval and flow freely, which keeps the diagnose step fast without widening the write path beyond the user’s own project files. Section 9 explains why we have not moved any action class to autopilot. 7.3. Model governance: provider-neutral access Institutions differ in which model providers are acceptable, for reasons of cost, procurement, and data governance. Studio 10

Figure 7: The results stage for the EHT reconstruction’s successful run inside Studio. The run lists its 40 result files (per-day panel images, closure statistics, the ring-measurement table, and the summary report), each with a preview and a download, and a link that opens the run directory in JupyterLab. The preview shows the April 6, 2017 reconstruction by all three pipelines, one day’s reconstructions produced on the gateway pool.

treats the model endpoint as configuration: commercial APIs, campus or national research gateways, or local models can be changed per user, and a user may bring their own provider key. Prompts that include research data can therefore be kept on infrastructure the institution controls. No Studio capability assumes a particular provider, and the platform remains usable without a model provider. The public deployment routes model requests through a platform model gateway that exposes one virtual model name to every agent and resolves it in tiers. The first tier is a set of open-weight models hosted on research clouds (at the time of writing, Qwen [43] models served by the National Research Platform [44] and by Jetstream2 [35]), load-balanced by remaining request budget and latency. A commercial endpoint is the second tier, tried only when all first-tier backends have failed for a request. Each backend carries its own request-rate limit and circuit breaker. In front of the gateway sits a prompt guard that scans every input for prompt-injection patterns and redacts bearer tokens before they reach a model, and scans outputs before they return. The practical effect is that a scientist’s prompts go to a research-cloud model by default and reach a commercial provider only as a fallback, while an institution can change that policy in one place.

every gateway backend. Traces, which carry the prompt text and tool arguments, flow only for users who have opted in. The guided tour asks the user to make this choice with no preselected answer, and the choice can later be changed in settings. If the setting cannot be read, Studio defaults to no consent. Consent also travels with the data as an attribute on every span, so the collector strips content from any trace whose resource does not assert consent, even if a running container still has a configuration written before the user changed the setting. Traces are stored in a trace store inside the cluster and are not exposed publicly. This constitutes the evidence base upon which the autonomy discussion in Section 9 depends on. Table 2 lists everything Studio records about a user’s interaction with the assistant, and under what condition. A collection notice at the end of the guided tour states the same list in plain language. The tour continues to request an explicit answer about whether chat text may be recorded until the user answers either way. Optional profile questions are handled separately. Three kinds of record carry no user text and are always kept: the usage metrics mentioned above, the per-failure episodes of Section 6 together with the user’s one-click answer to the help card of Figure 8, and the ratings a user gives individual replies (a thumbs up or down, or a tag when the user stops a reply). The fourth kind, the text of prompts and responses and the arguments of tool calls, is the conversation itself and is kept only for users who opted in. The purpose is stated to the user as well: the episodes and ratings are how the facility will learn which classes of agent action tend to succeed without human correction, while the conversation text helps it learn why the other actions do not. None of these mechanisms require changes to Pegasus or HTCondor. Studio uses the existing token mechanism, scheduler

7.4. Observability and consent Accountability also requires that the facility can see what its agents cost and do. The chat engine in every container emits OpenTelemetry [45] signals to a collector in the platform. Metrics, which record aggregate activity (tokens by type, tool-call durations, sessions and messages, model and provider, per user), always flow and feed a dashboard that shows usage and cost by user, which agent role handled each message, and the health of

11

Table 2: What Studio records about a user’s interaction with the assistant. The first three rows carry no user text. The last row is the conversation itself and is recorded only with the user’s opt-in.

Record

Condition

Kept where

Usage metrics: tokens by type, tool-call durations, sessions and messages, model and provider, retries, per user

always

platform metrics store and the facility dashboard

Failure episodes: a run failed, whether the assistant was asked, whether a re-run followed and how it ended, and the user’s answer to the help card (fixed it, partly, did not help, did not try it)

always, one question per episode

the user’s own database, read by administrators from the shared home

Reply ratings (thumbs up or down) and give-up tags, and the optional profile answers from the tour

when the user clicks or answers

the user’s own database, rating counts also as platform metrics

Prompt and response text, tool-call arguments (including shell commands)

only with opt-in, revocable in settings

in-cluster trace store, the collector strips content otherwise

authentication, and WMS provenance records as provided by those systems. This is important for the trust argument: a resource provider can evaluate agent-driven execution in terms of the same identity, authorization, and audit mechanisms it already uses for human-submitted workflows.

Figure 8: The help card. After a failed workflow has been re-run and the rerun has finished, Studio asks one question at the point where the answer is meaningful: did the assistant’s suggestion fix it? The four answers are one click each, and the card is rate-limited so that it does not become a survey users learn to dismiss. Here the EEG workflow of Section 8.5 succeeded on its fourth run after the assistant diagnosed the earlier failure. The chat is in its default mode, which routes each message to a specialist agent role.

8. Evaluation and Case Studies The evaluation examines whether Studio’s mechanisms function during real workflow development and execution. The evidence here comes from case studies and deployment tests and is observational rather than a controlled comparison.

With the supporting evidence in the subsections below, we can summarize the findings for each research question as follows: For RQ1, in the cases examined, the evidence supports telemetry-grounded diagnosis and repair. Telemetry exposed three stacked defects in the CASA nowcast port, which then completed all 86 jobs (Section 8.3). In the EHT reconstruction, the agent went from two failed runs to a zero-failure run in under an hour (Section 8.6). For RQ2, Studio derived the gw workflow’s parameter form from the agent’s generator, the first launch exposed a defect at that boundary, and the corrected workflow ran 350 of 350 jobs from the same form (Section 8.4). For RQ3, in the EHT reconstruction, the rebuild and resubmission ran under the user’s scheduler identity and remained subject to the user’s configured approval policy (Section 8.6). For RQ4, the same application image ran as a JupyterHub gateway and as an Open OnDemand application on a test cluster, and the gateway ran the EHT workflow on a user’s annex at SDSC Expanse. The projected knowledge was also exercised with more than one agent and model stack (Section 8.7).

8.1. Research questions • RQ1 (Telemetry grounding). Does structured workflow and scheduler state let the agent find and repair real execution failures? • RQ2 (Interface legibility). Can an agent-written workflow become operable from the Studio interface without hand integration? • RQ3 (Accountability). Do agent actions, including repairs and resubmissions, remain attributable to the user and subject to the configured approval policy? • RQ4 (Generality across deployments and agents). Does the same application image operate across both deployment models, and does the projected knowledge work with more than one agent and model stack? 12

first launch from that form exposed a real boundary defect: the form submitted a comma-separated list where the command line expected separate tokens. The agent changed the interface to accept both representations, after which the same form launched the run of record. It completed 350 of 350 jobs (287 tasks) with zero failures in 54 minutes 18 seconds on the Studio pool, and its scientific checks passed. The case demonstrates the structurallegibility contract of Section 5: an interface written by the agent was introspected by Studio, rendered to the user, exercised, corrected at the boundary, and then used to launch the completed workflow.

8.2. Scope of the evidence The evaluation focuses on cases in which a Studio mechanism and an observed outcome are directly linked. In the CASA nowcast port (Section 8.3), workflow and scheduler telemetry exposed three stacked execution problems and the repair cycle ended in a verified 86-job run. In the Gravitational-wave workflow (Section 8.4), Studio constructed its parameter form from an agent-written generator. The first launch exposed an interface defect at that boundary, and the corrected workflow then ran to completion from the same form. In the EEG analysis (Section 8.5), the projected knowledge base was exercised with a different agent and model stack, and the resulting workflow recovered from an execution-environment failure to complete all 127 jobs successfully. In the EHT reconstruction (Section 8.6), two gateway runs failed at stage-in. The debug path supplied the analyzer report and job logs to the workflow-debugger agent, which identified missing container images, repaired their unprivileged builds under user approval, and resubmitted to a zerofailure run. These cases show the mechanisms operating as designed. They do not quantify how much effort the mechanisms save.

8.5. EEG analysis: exercising knowledge projection across an agent and model stack The EEG workflow provides an end-to-end test of Studio’s knowledge projection with a different agent and model stack. OpenCode, using the open-weight Kimi K2.6 model [50], operated entirely inside Studio from a single request for a grouplevel N170 face-perception analysis over OpenNeuro dataset ds007096 [51]. Working from the projected knowledge base, the agent invoked the scaffold skill and produced a per-subject fan-out followed by a group-level fan-in. The same session exercised the debugging path. Container wrapping failed on the heterogeneous gateway workers, and the agent recovered by replacing the container with a portable bundle of the required Python dependencies staged as a workflow input. The final run over 24 subjects completed all 127 jobs (97 tasks) in 5 minutes 40 seconds with zero failures and zero retries.

8.3. CASA nowcast: telemetry-grounded diagnosis CASA (Collaborative Adaptive Sensing of the Atmosphere) operates a network of short-range weather radars [46] whose data feed operational nowcasting products. A legacy CASA nowcasting workflow, originally written for an earlier Pegasus release, was modernized for Pegasus 5 inside Studio and verified end to end: all 86 jobs succeeded. The diagnose step surfaced three stacked defects that were not apparent from terminal output alone. Unprivileged pods cannot run Docker, so execution had to move to Apptainer. A leftover working-directory catalog silently shadowed the embedded catalogs, which was revealed by a malformed URL in the planned transfer list. A lax workerpackage setting then let the host’s newer GLIBC kickstart run inside an older container, breaking jobs at startup. Each failure was identified from workflow or scheduler state, fixed, and rerun through the agent-assisted diagnose, fix, and resubmit loop of Section 6. The resulting port was submitted as an upstream pull request. Two sibling CASA workflows were modernized in the same sessions and verified in the same way (Wind at 23 of 23 jobs and Hail at 14 of 14). The CASA cases also show that legacy Pegasus workflows can be brought into the Studio lifecycle without changes to Pegasus or HTCondor.

8.6. EHT reconstruction: telemetry-grounded debugging and accountable execution The EHT reconstruction provides an end-to-end test of Studio because a scientifically substantive workflow encountered real infrastructure failures while running through the gateway. The case exercises several Studio mechanisms together: exposing failure state to the agent, repairing the workflow under the user’s authority, resubmitting it through the same execution path, and returning the resulting scientific outputs through the workflow lifecycle. The reconstructed workflow [52], which builds on an earlier Pegasus-based reproduction of the EHT M87 imaging pipelines [53], contains 31 scientific tasks across the DIFMAP, eht-imaging, and SMILI imaging pipelines [5] plus rendering, closure statistics, ring measurement, and aggregation. Four containers reproduce the required software environments, three of them from the 2019 software era. The workflow was built and executed inside Studio on the gateway’s HTCondor pool, where its container images had to be built without root privileges in the user’s pod. The first two gateway runs failed at stage-in because three of the four images had not been produced successfully by the unprivileged container builds. The user invoked Studio’s debug path. The workflowdebugger agent received the analyzer report and job logs, identified the missing images, the repeated failure, and explained the diagnosis. With the user’s approval, the agent rebuilt the

8.4. Gravitational waves: exercising the interface contract The GW workflow [47] reanalyzes GWTC-1 binary black hole events [48] from GWOSC open strain data [49]. The workflow was built entirely inside Studio, using Claude Code with Claude Opus 5. The question is whether an agent-written workflow can become directly operable through the Studio interface. Studio derived the workflow’s parameter form from the generator’s argparse interface without executing the code. The 13

images. Those builds exposed three problems specific to unprivileged reconstruction of the old software stack: an apt sandbox-user failure inside a user namespace, an archived Debian package URL that returned 404, and a tarball removed before it was needed. The agent repaired the definitions, verified each image by running an import test for its software inside the container, regenerated the workflow, and resubmitted it. The third gateway run completed 55 of 55 jobs (31 tasks) with zero failures and zero retries in 8 minutes 34 seconds, less than an hour after the first failure and with two user prompts in the chat. Figure 9 shows the successful graph and the workflowdebugger’s closing summary. The same episode also exercises Studio’s accountability mechanisms: the agent ran in the authenticated user’s Studio container, jobs carried that user’s scheduler identity, and the rebuild and resubmission path remained subject to the approval process of Section 7. The failed runs, diagnosis, edits, and successful run remain visible in the workflow and scheduler records that supplied the agent’s telemetry. The successful run produced the reconstructed M87 images, closure statistics, and ring measurements shown through Studio’s results interface in Fig. 7. The EHT case provides systems evidence: telemetry from the failed executions reached the agent, the repair occurred under the user’s identity and approval, and the corrected workflow completed on the shared gateway pool.

that. Compute, storage, and data services belong to facilities and projects that set their own policies for identity, allocation, scheduling, data movement, and accounting. An agent that orchestrates these resources directly has to reproduce each of those controls or work around them. It also has to recover from the partial failures of distributed execution, such as held jobs, failed transfers, and lost nodes, which workflow systems such as Pegasus [1] were built to handle. Studio takes the other path. It leaves execution to a vetted stack and applies AI where that stack is hard to use. Pegasus plans the workflow, retries failed jobs, and records provenance. HTCondor and the site’s batch scheduler place jobs and account for them. JupyterHub, Open OnDemand, and CILogon establish who the user is. The agent writes and repairs the workflows that this stack runs, and it reads the stack’s own state to explain failures. Because it acts under the user’s identity, it never holds access to facility resources that the user does not already have. The capabilities and controls that facilities already trust are preserved, and AI lowers the cost of using them. We expect the same pattern to apply wherever the resources an agent would act on are owned and governed by a third party. 9.2. What generalizes beyond Pegasus The systems mechanisms in Studio are expressed in terms of abstractions that are not unique to Pegasus. Knowledge projection requires a canonical body of system and site knowledge plus adapters to the configuration interfaces of supported agents. Telemetry grounding requires queryable workflow and scheduler state that can be turned into structured execution signals. The trust envelope requires an existing user identity, authorization boundary, and audit trail under which agent actions can run. Pegasus, HTCondor, JupyterHub, and Open OnDemand provide those mechanisms in Studio. The architecture is designed around these abstractions rather than their particular APIs, although it has so far been implemented only with them (Section 10). For the scientist, the resulting division of labor is similar across systems: the agent can construct and operate workflow machinery, while the human remains responsible for scientific intent, review, and validation. For facilities, the important shift is that workflow and site expertise becomes maintained, versioned infrastructure rather than context configured separately for each user and agent.

8.7. Generality across deployments and agents Portability across deployment environments. Studio uses the same application image in two deployment models: a JupyterHub/Kubernetes gateway on Jetstream2 and an Open OnDemand application for HPC systems. The gateway has executed workflows on its shared HTCondor pool and, through a user’s annex on SDSC Expanse, ran the EHT reconstruction with all 31 compute steps on the user’s own allocation (Section 7). The OOD deployment has been validated on a test cluster (Section 4.2). These observations demonstrate that the Studio architecture operates across the environments actually tested, not across workflow systems in general. Knowledge projection across agent/model stacks. The canonical knowledge base of Section 5 is projected into five agent-native configurations. The GW case above used the Claude Code projection. The EEG exercised the same projection mechanism with a different agent and model stack: OpenCode driving Kimi K2.6 [50]. Together, these cases show that the projected knowledge and lifecycle operate with more than one agent and model stack. They do not establish agent independence or compare model capability.

9.3. Practical lessons from deployment Four lessons from operating the gateway are not specific to Pegasus. Reverse-proxy prefixes break assumptions. Per-user URL prefixes under JupyterHub exposed assumptions about buildtime base paths, asset paths, and WebSocket origins. Applications intended for gateway deployment need these constraints accounted for in the design rather than discovered after integration. Streaming requires infrastructure support. Default ingress request timeouts silently reset agent and chat streams. Routes carrying server-sent events or long-running tool calls need an explicit timeout policy. The failure mode is largely invisible in local development.

9. Discussion and Lessons Learned from Deployment 9.1. Enhancing vetted infrastructure rather than replacing it A common direction in AI for science is to recast scientific computation as an agentic workflow, in which a language-model agent plans the steps, calls tools, and drives execution itself [23, 24, 22]. That design is simplest when the agent controls the resources it uses. Research cyberinfrastructure rarely offers 14

Figure 9: The EHT reconstruction inside Studio. Left: the graph of the successful third gateway run, with the per-day fan-out of the three imaging pipelines collapsed into one node per pipeline. Right: the workflow-debugger agent’s summary after rebuilding and verifying the four container images. The same workflow and scheduler records that drive the graph supplied the diagnostic context to the agent.

Agents inherit the execution environment’s sharp edges. Clean job environments, nested container constraints, per-user token trust domains, unprivileged pods, and old submit-host software are infrastructure properties, not AI-specific ones. The knowledge layer of Section 5 is where these lessons accumulate so that later agent sessions do not have to rediscover them. Autonomy must follow evidence. Studio currently uses the approve posture described in Sec. 7, in which higher-risk writes and every workflow submission remain approval-gated. Moving a class of actions, such as resubmitting after a repair, toward unattended execution would require evidence that the action can be carried out reliably without human correction. The failure episodes and feedback described in Secs. 6 and 7 are intended to provide that evidence. Read-only actions and edits inside the user’s own workflow and data directories already proceed without approval. Writes elsewhere and every submission do not.

the mechanisms operate as designed, not that they reduce effort for a representative user population. The failure episodes and help-card answers of Sections 6 and 7 are designed to collect evidence about agent assistance as the user population broadens. No quantitative comparison of debugging effort. The case studies show that Studio exposes structured state and supports diagnosis and repair, but they do not establish that it reduces debugging effort relative to other ways of operating the same workflows. A controlled comparison remains future work. Case selection. The authors selected all of the case studies. The workflows span different scientific domains, software stacks, and execution patterns, but they were not sampled systematically from a broader population of scientific workflows. The evaluation therefore demonstrates the Studio mechanisms across the cases examined, but does not establish how representative those cases are of scientific workflow development more broadly. Generality is partly a design claim. Knowledge projection, telemetry grounding, and the trust envelope are built from abstractions that exist in other workflow systems and gateways. They have been implemented and tested only with Pegasus, HTCondor, JupyterHub, and Open OnDemand, and the Open OnDemand deployment has been validated only on a test cluster. Current implementation limits. The application container is single-user, some state is SQLite-backed, AI tools are installed per container, and some model credentials are stored locally. These are limits of the current implementation rather than requirements of the architecture. Agent and model drift. Case-study outcomes depend on the agent, model, and version used, and these change quickly. We report the agent and model for each case. The cross-agent evidence shows interoperability where tested. It is not a benchmark of model capability.

9.4. Demand signal Requests for a version that institutions could deploy on their own clusters led to the Open OnDemand deployment (Sec. 4.2), which has been validated on a test cluster but not yet deployed at a production site. Separately, the monitoring tool is being integrated into Pegasus through its plugin system. These are signs of interest, and they suggest that deployment portability and tighter monitoring integration are useful directions. They are not evidence that any institution has put Studio into production use, and we do not claim that. 10. Limitations of the System and Evaluation Self-development and limited users. Studio has so far been used mainly by its own developers and by collaborators who are already familiar with Pegasus. Thus, the case studies show that 15

11. Artifact Availability

the user’s own identity and let the workflow and scheduling systems provide execution, recovery, and provenance.

PegasusAI Studio is open source [54], as is the Open OnDemand application [55]. The Studio release also provides links to the public gateway and project site. The deployment repository contains site configuration and sealed secrets and is therefore not released as-is. Reviewable manifests can be provided separately. The canonical PegasusAI knowledge base, including skills, agent roles, templates, and documentation servers, is open source in the PegasusAI plugin marketplace [37]. All four case-study workflows are public repositories. The CASA nowcast port (casa-nowcast-workflow), the gravitational-wave reanalysis (gw-workflow), and the EEG analysis (eeg-workflow) are in the pegasus-isi GitHub organization [47]. The EHT reconstruction is published separately [52].

Declaration of competing interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. Acknowledgments This work is supported by Pegasus AI NSF grant #2513101 and the FABRIC Testbed (NSF #2330891). Data availability The Studio application, Open OnDemand package, canonical knowledge base, and case-study workflow artifacts are available at the locations described in Section 11.

12. Conclusion An AI agent that writes a workflow on a laptop has solved only part of the workflow execution problem. Shared CI also requires the agent to know the system’s conventions, to monitor the state of distributed executions, and to act under an identity the facility can accept and audit. We presented PegasusAI Studio, which makes CI legible to agents through knowledge projection, interface and site introspection, and telemetry grounding. In addition, it makes agents accountable through federated identity, per-user scheduler credentials, approval controls for state-changing actions, and observability. Rather than reimplementing execution infrastructure around the AI agent, Studio builds these capabilities on mature, production-proven CI systems. Its design follows one rule. Mature execution systems operate the workflow, so an unmodified Pegasus and HTCondor stack handles scheduling, data movement, recovery, provenance, and resource management. Mature gateway systems provide identity and multi-user isolation, so JupyterHub and Open OnDemand authenticate each user and give each one a separate environment. Studio adds only the integration layer that is specific to agents: projected knowledge, structured execution state, and controls on what an agent may do. The case studies show the mechanisms operating on real workflow development and execution, from legacy CASA ports to the EHT M87 reconstruction. They do not yet quantify how much effort the mechanisms save. Consequently, our future work will include a more rigorous evaluation framework, which will analyze controlled comparisons of diagnosis and repair cycles, a broader population of external users, and measured evidence for which classes of agent action can safely move from approval to greater autonomy. Integrating the monitoring service into Pegasus through its plugin system is the main engineering step in progress. Studio is therefore best viewed as an operating layer between AI agents and existing CI: maintain system and site knowledge as versioned infrastructure and project it consistently to supported agents, give agents the structured state the workflow and scheduling systems already record, run every agent action under

Declaration of generative AI and AI-assisted technologies in the writing process During the preparation of this work the authors used Claude Code (Anthropic) to draft manuscript text from an author-written specification and to typeset tables and figures. After using this tool, the authors reviewed and edited the content as needed and take full responsibility for the content of the published article. References [1] E. Deelman, K. Vahi, G. Juve, M. Rynge, S. Callaghan, P. J. Maechling, R. Mayani, W. Chen, R. Ferreira da Silva, M. Livny, K. Wenger, Pegasus, a workflow management system for science automation, Future Generation Computer Systems 46 (2015) 17–35. doi:10.1016/j. future.2014.10.008. [2] D. Thain, T. Tannenbaum, M. Livny, Distributed computing in practice: the Condor experience, Concurrency and Computation: Practice and Experience 17 (2–4) (2005) 323–356. doi:10.1002/cpe.938. [3] PegasusAI Team, Napkin to science: From scientific intent to executable workflows on production cyberinfrastructure, under submission to Future Generation Computer Systems (2026). [4] C. Roa, M. Rynge, P. Olaya, K. Vahi, T. Miller, J. Griffioen, S. Knuth, J. Goodhue, D. Hudák, A. Romanella, R. M. Llamas, R. Vargas, M. Livny, E. Deelman, M. Taufer, End-to-end integration of scientific workflows on distributed cyberinfrastructures: Challenges and lessons learned with an earth science application, in: Proceedings of the IEEE/ACM 16th International Conference on Utility and Cloud Computing, UCC 2023, Taormina (Messina), Italy, December 4-7, 2023, ACM, 2023, p. 17. doi:10.1145/3603166.3632142. URL https://doi.org/10.1145/3603166.3632142 16

[5] Event Horizon Telescope Collaboration, et al., First M87 Event Horizon Telescope results. IV. imaging the central supermassive black hole, The Astrophysical Journal Letters 875 (1) (2019) L4. doi:10.3847/2041-8213/ab0e85.

[15] M. R. Crusoe, S. Abeln, A. Iosup, P. Amstutz, J. Chilton, N. Tijanić, H. Ménager, S. Soiland-Reyes, B. Gavrilović, C. Goble, The CWL Community, Methods included: standardizing computational reuse and portability with the Common Workflow Language, Communications of the ACM 65 (6) (2022) 54–63. doi:10.1145/3486897.

[6] R. Patel, B. Roachell, S. Caíño-Lores, R. Ketron, J. Leonard, N. Tan, K. Vahi, D. A. Brown, E. Deelman, M. Taufer, Reproducibility of the first image of a black hole in the galaxy M87 from the event horizon telescope collaboration, Computing in Science & Engineering 24 (5) (2022) 42–52. doi:10.1109/MCSE.2023.3241105.

[16] Y. Babuji, A. Woodard, Z. Li, D. S. Katz, B. Clifford, R. Kumar, L. Lacinski, R. Chard, J. M. Wozniak, I. Foster, M. Wilde, K. Chard, Parsl: Pervasive parallel programming in Python, in: Proceedings of the 28th International Symposium on High-Performance Parallel and Distributed Computing (HPDC), 2019, pp. 25–36. doi: 10.1145/3307681.3325400.

[7] E. M. Fajardo, J. M. Dost, B. Holzman, T. Tannenbaum, J. Letts, A. Tiradani, B. Bockelman, J. Frey, D. Mason, How much higher can HTCondor fly?, Journal of Physics: Conference Series 664 (6) (2015) 062014. doi:10.1088/ 1742-6596/664/6/062014.

[17] The Galaxy Community, The Galaxy platform for accessible, reproducible and collaborative biomedical analyses: 2022 update, Nucleic Acids Research 50 (W1) (2022) W345–W351. doi:10.1093/nar/gkac247.

[8] K. A. Lawrence, M. Zentner, N. Wilkins-Diehr, J. A. Wernert, M. Pierce, S. Marru, S. Michael, Science gateways today and tomorrow: positive perspectives of nearly 5000 members of the research community, Concurrency and Computation: Practice and Experience 27 (16) (2015) 4252–4268. doi:10.1002/cpe.3526.

[18] M. McLennan, R. Kennell, HUBzero: A platform for dissemination and collaboration in computational science and engineering, Computing in Science & Engineering 12 (2) (2010) 48–53. doi:10.1109/MCSE.2010.41.

[9] T. Kluyver, B. Ragan-Kelley, F. Pérez, B. Granger, M. Bussonnier, J. Frederic, K. Kelley, J. Hamrick, J. Grout, S. Corlay, P. Ivanov, D. Avila, S. Abdalla, C. Willing, Jupyter Development Team, Jupyter notebooks: a publishing format for reproducible computational workflows, in: Positioning and Power in Academic Publishing: Players, Agents and Agendas (Proceedings of ELPUB 2016), IOS Press, 2016, pp. 87–90. doi:10.3233/978-1-61499-649-1-87.

[19] B. S. Biggs, Jr., Integrating artificial intelligence into science gateways, Tech. Rep. INL/CON-23-72098, Idaho National Laboratory (2023). URL https : / / inldigitallibrary . inl . gov / sites/sti/sti/Sort_65663.pdf [20] B. S. Biggs, K. Dalton, Supplementing HPC support with a science gateway AI assistant, Tech. Rep. INL/CON-2476939, Idaho National Laboratory (May 2024). URL https : / / inldigitallibrary . inl . gov / content/uploads/50/2026/04/Sort_93465.pdf

[10] D. Hudak, D. Johnson, A. Chalker, J. Nicklas, E. Franz, T. Dockendorf, B. L. McMichael, Open OnDemand: A web-based client portal for HPC centers, Journal of Open Source Software 3 (25) (2018) 622. doi:10.21105/joss. 00622.

[21] J. Sperhac, S. Gesing, M. Zentner, C. Stirm, R. Quick, J. Stubbs, AI and science gateways: A promising combination for accelerating science and research computing, in: Practice and Experience in Advanced Research Computing (PEARC ’24), ACM, 2024. doi:10.1145/3626203. 3670562.

[11] J. Basney, T. Fleury, J. Gaynor, CILogon: A federated X.509 certification authority for cyberinfrastructure logon, Concurrency and Computation: Practice and Experience 26 (13) (2014) 2225–2239. doi:10.1002/cpe.3265. [12] Anthropic, Model context protocol, https : / / modelcontextprotocol . io, accessed 2026-0811 (2024).

[22] H. Ma, A. Brace, C. Siebenschuh, G. Pauloski, I. Foster, A. Ramanathan, Connecting large language model agent to high performance computing resource (2025). arXiv: 2502.12280, doi:10.48550/arXiv.2502.12280.

[13] P. Di Tommaso, M. Chatzou, E. W. Floden, P. P. Barja, E. Palumbo, C. Notredame, Nextflow enables reproducible computational workflows, Nature Biotechnology 35 (4) (2017) 316–319. doi:10.1038/nbt.3820.

[23] D. A. Boiko, R. MacKnight, B. Kline, G. Gomes, Autonomous chemical research with large language models, Nature 624 (7992) (2023) 570–578. doi:10.1038/ s41586-023-06792-0.

[14] F. Mölder, K. P. Jablonski, B. Letcher, M. B. Hall, C. H. Tomkins-Tinch, V. Sochat, J. Forster, S. Lee, S. O. Twardziok, A. Kanitz, A. Wilm, M. Holtgrewe, S. Rahmann, S. Nahnsen, J. Köster, Sustainable data analysis with Snakemake, F1000Research 10 (2021) 33. doi: 10.12688/f1000research.29032.2.

[24] A. M. Bran, S. Cox, O. Schilter, C. Baldassari, A. D. White, P. Schwaller, Augmenting large language models with chemistry tools, Nature Machine Intelligence 6 (5) (2024) 525–535. doi:10.1038/s42256-024-00832-8.

17

[25] D. Merkel, Docker: Lightweight Linux containers for consistent development and deployment, Linux Journal 2014 (239) (2014) 2.

[38] The Pegasus Team, ACCESS Pegasus: a workflow science gateway for the NSF ACCESS program, https:// pegasus.access-ci.org, accessed 2026-10-15 (2026).

[26] G. M. Kurtzer, V. Sochat, M. W. Bauer, Singularity: Scientific containers for mobility of compute, PLOS ONE 12 (5) (2017) e0177459. doi:10.1371/journal.pone. 0177459.

[39] The Pegasus Team, Kiso: experiment management for Pegasus workflows on cloud and edge testbeds, https: //github.com/pegasus-isi/kiso, accessed 2026-1029 (2026).

[27] B. Burns, B. Grant, D. Oppenheimer, E. Brewer, J. Wilkes, Borg, Omega, and Kubernetes, Queue 14 (1) (2016) 70–93. doi:10.1145/2898442.2898444.

[40] D. Gunter, E. Deelman, T. Samak, C. Brooks, M. Goode, G. Juve, G. Mehta, P. Moraes, F. Silva, M. Swany, K. Vahi, Online workflow management and performance analysis with Stampede, in: 7th International Conference on Network and Service Management (CNSM 2011), IEEE, 2011. doi:10.1109/CNSM.2011.6103947.

[28] A. B. Yoo, M. A. Jette, M. Grondona, SLURM: Simple Linux utility for resource management, in: Job Scheduling Strategies for Parallel Processing (JSSPP 2003), Vol. 2862 of Lecture Notes in Computer Science, Springer, 2003, pp. 44–60. doi:10.1007/10968987_3.

[41] K. Vahi, I. Harvey, T. Samak, D. Gunter, K. Evans, D. Rogers, I. Taylor, M. Goode, F. Silva, E. Al-Shakarchi, G. Mehta, A. Jones, E. Deelman, A general approach to real-time workflow monitoring, in: 7th Workshop on Workflows in Support of Large-Scale Science (WORKS 2012), IEEE, 2012. doi:10.1109/SC.Companion.2012.21.

[29] I. Baldin, A. Nikolich, J. Griffioen, I. I. S. Monga, K.C. Wang, T. Lehman, P. Ruth, FABRIC: A national-scale programmable experimental network infrastructure, IEEE Internet Computing 23 (6) (2019) 38–47. doi:10.1109/ MIC.2019.2958545.

[42] S. Strande, H. Cai, M. Tatineni, W. Pfeiffer, C. Irving, A. Majumdar, D. Mishin, R. Sinkovits, M. Norman, N. Wolter, et al., Expanse: Computing without boundaries, in: Practice and Experience in Advanced Research Computing (PEARC ’21), ACM, 2021, pp. 1–4. doi:10.1145/3437359.3465588.

[30] Anthropic, Claude Code, https : / / github . com / anthropics / claude-code, accessed 2026-10-29 (2026). [31] OpenCode contributors, OpenCode, https://github. com / anomalyco / opencode, accessed 2026-10-29 (2026).

[43] A. Yang, et al., Qwen3 technical report (2025). arXiv: 2505.09388.

[32] Charm, Crush, https : / / github . com / charmbracelet/crush, accessed 2026-10-29 (2026).

[44] National Research Platform, National Research Platform (NRP), https://nrp.ai, accessed 2026-10-29 (2026).

[33] OpenAI, Codex CLI, https://github.com/openai/ codex, accessed 2026-10-29 (2026).

[45] OpenTelemetry Authors, OpenTelemetry, https : / / opentelemetry.io, accessed 2026-10-29 (2026).

[34] Google, Antigravity, https://antigravity.google, accessed 2026-10-29 (2026).

[46] D. McLaughlin, D. Pepyne, V. Chandrasekar, B. Philips, J. Kurose, M. Zink, K. Droegemeier, S. Cruz-Pol, F. Junyent, J. Brotzge, D. Westbrook, N. Bharadwaj, Y. Wang, E. Lyons, K. Hondl, Y. Liu, E. Knapp, M. Xue, A. Hopf, K. Kloesel, A. DeFonzo, P. Kollias, K. Brewster, R. Contreras, T. Djaferis, E. Insanic, S. Frasier, F. Carr, Shortwavelength technology and the potential for distributed networks of small radar systems, Bulletin of the American Meteorological Society 90 (12) (2009) 1797–1818. doi:10.1175/2009BAMS2507.1.

[35] D. Y. Hancock, J. Fischer, J. M. Lowe, W. Snapp-Childs, M. Pierce, S. Marru, J. E. Coulter, M. Vaughn, B. Beck, N. Merchant, E. Skidmore, G. Jacobs, Jetstream2: Accelerating cloud computing via Jetstream, in: Practice and Experience in Advanced Research Computing (PEARC ’21), 2021, pp. 1–8. doi:10.1145/3437359.3465565. [36] R. Pordes, D. Petravick, B. Kramer, D. Olson, M. Livny, A. Roy, P. Avery, K. Blackburn, T. Wenaus, F. Würthwein, et al., The Open Science Grid, Journal of Physics: Conference Series 78 (2007) 012057. doi:10.1088/ 1742-6596/78/1/012057.

[47] The Pegasus Team, Pegasus workflow repositories built with the Pegasus-AI toolchain, https : / / github . com / pegasus-isi, repositories soilmoisture-workflow, airquality-workflow, earthquake-workflow, crophealth-workflow, mag-workflow, cper-soilmoisture-workflow, gwworkflow, eeg-workflow, casa-nowcast-workflow, and others, accessed 2026-10-16 (2026).

[37] The Pegasus Team, Pegasus-AI plugin marketplace: skills, agents, and knowledge base for AI-assisted Pegasus workflow development, https : / / github . com / pegasus-isi / claude-plugin-marketplace, accessed 2026-10-15 (2026). 18

[48] B. P. Abbott, et al., GWTC-1: A gravitational-wave transient catalog of compact binary mergers observed by LIGO and Virgo during the first and second observing runs, Physical Review X 9 (3) (2019) 031040. doi: 10.1103/PhysRevX.9.031040. [49] R. Abbott, et al., Open data from the first and second observing runs of Advanced LIGO and Advanced Virgo, SoftwareX 13 (2021) 100658. doi:10.1016/j.softx. 2021.100658. [50] Kimi Team, Kimi K2: Open agentic intelligence (2025). arXiv:2507.20534. [51] C. J. Markiewicz, K. J. Gorgolewski, F. Feingold, R. Blair, Y. O. Halchenko, E. Miller, N. Hardcastle, J. Wexler, O. Esteban, M. Goncalves, A. Jwa, R. Poldrack, The OpenNeuro resource for sharing of neuroscience data, eLife 10 (2021) e71774. doi:10.7554/eLife.71774. [52] K. Thareja, Clean-room Pegasus reproduction of the EHT M87 imaging pipelines, https://github.com/ kthare10/eht-workflow, accessed 2026-10-28 (2026). [53] R. Patel, B. Roachell, S. Caíno-Lores, R. Ketron, J. Leonard, N. Tan, K. Vahi, D. A. Brown, E. Deelman, M. Taufer, Reproducibility of the first image of a black hole in the galaxy M87 from the event horizon telescope collaboration, Comput. Sci. Eng. 24 (5) (2022) 42–52. doi:10.1109/MCSE.2023.3241105. URL https : / / doi . org / 10 . 1109 / MCSE . 2023 . 3241105 [54] The Pegasus Team, PegasusAI Studio: an AIassisted science gateway for Pegasus workflows, Source code at https : / / github . com / pegasus-isi / studio-webapp; public gateway at https : / /studio . pegasus-ai . org; project site at https : / / pegasus-ai . org, accessed 2026-10-15 (2026). [55] The Pegasus Team, PegasusAI Studio for Open OnDemand: a batch-connect application, https://github. com/pegasus-isi/bc_pegasus_ai_studio, accessed 2026-10-16 (2026).

19

Related documents

• Twelve quick tips for designing AI-driven HPC workflows#793719
• Twelve quick tips for designing AI-driven HPC workflows#789715
• SAGA: Workflow-Atomic Scheduling for AI Agent Inference on GPU Clusters#151760
• From Legacy Fortran to Portable Kokkos: An Autonomous Agentic AI Workflow#1468123
• Agentic AI for Gravitational Wave Data Analysis: A Head-to-Head Comparison of Coding Agents Executing a Matched Filter Pipeline on Einstein Telescope Simulated Data#1083101