Sandman: Building and Operating Sandboxes for Agentic Reinforcement Learning on Shared Kubernetes

S ANDMAN : B UILDING AND O PERATING S ANDBOXES FOR AGENTIC R EINFORCEMENT L EARNING ON S HARED K UBERNETES

arXiv:2610.39711v1 [cs.DC] 29 Oct 2026

Yiming Yan* Hongyang Wang* Wenbo Chen Xuan Wang Jingguo Liu Xian Li Chen Qi Leo Fan Siwei Li Qianli Wu Yichuan Li Ruochen Wen Qian Qu Zheshen Wang Benoit Dumoulin Amazon

A BSTRACT Agentic reinforcement learning (RL) often requires repeated tool calls in task-specific environments. A training job may create thousands of these environments in bursts while sharing a cluster with other jobs. We present Sandman, a sandbox service built and operated at Amazon for agent tool execution and evaluation. It serves multiple production agentic RL workloads on shared Kubernetes. A training framework supplies a task image and uses commands and file operations in the returned sandbox. Files and processes persist across calls until the framework releases the environment. Sandman manages creation, readiness and cleanup. Each sandbox is a Kubernetes pod with a small daemon mounted into the unmodified image. In a scale test, Sandman held 11,858 concurrent tiny sandboxes. Four production SWE-bench Verified training campaigns created 3.1 million sandboxes in October 2026. We report the design trade-offs and operational lessons from handling creation bursts, preparing task environments, reclaiming abandoned sandboxes, and running evaluation under controlled conditions. We will open-source Sandman’s daemon, provider and pool. 1

I NTRODUCTION

Agentic reinforcement learning (RL) trains agents through repeated interactions with task environments. Tools run commands and file operations in sandboxes whose state persists across calls. Training jobs acquire these environments for rollouts and evaluation, then release them when finished. Sandbox demand is large and bursty: in our softwareengineering (SWE) campaigns, a training run starts dozens of rollouts within a second, and many runs share the cluster (Figure 1a). We first ran Python and math rollouts on a managed code-interpreter service. It could not run task images, which ruled out SWE tasks, and when a batch of rollouts recycled its sessions together, throttled calls timed out trajectories (§4.1). We present Sandman, the sandbox service our team built and operates on a shared Amazon Elastic Kubernetes Service (EKS) cluster (Figure 2). It serves multiple production agentic RL workloads. A training framework hands it a task image, runs commands, file operations and tests in the returned sandbox across tool calls, and releases it. Sandman manages creation, readiness and cleanup independently of the model. Training shapes this service: creates arrive in *

Equal contribution. Correspondence to: Yiming Yan <yyming009@gmail.com>. Preprint.

bursts, large images are slow to pull, timeouts and cancellations leave sandboxes behind, prewarmed pods sit idle, and grading needs controlled run conditions (§2–4). We use SWE training as a running example and the primary evaluation workload. Each sandbox is an ordinary pod on the cluster’s CPU nodes, started from the task’s unmodified image with a small daemon mounted in; Sandman adds no node daemon or container runtime. A provider creates, paces, spreads and deletes the pods and owns each create until its pod is handed out or deleted, even after its caller gives up. A per-job pool paces creates and may prewarm pods. Our SWE integration grades each patch in a separate pod, never-used by default, after a probe checks the declared run conditions it can read. Production here means daily internal use for model development, with no external customers or service-level agreement; our training campaigns, sets of training runs, predate several mechanisms this paper describes (Table 4). This paper makes four contributions: 1. Sandman, a service that turns a task image into a stateful sandbox a training framework acquires, uses across tool calls and releases, built from ordinary pods on a shared training cluster, whereas purpose-built fleets (Huang et al., 2026) and hosted runtimes (E2B, 2026; Modal Labs, 2026; Daytona, 2026) run their own infrastructure. Sandman held 11,858 concurrent

Sandman: Sandboxes for Agentic RL on Shared Kubernetes (b)

(d) model generates (92%)

start, end

132 creates in one second:

100

two runs start together

0

campaign mean 1.8 creates/s

50 0

tool runs

0

60

120

180

(c)

50 100 150 One rollout sandbox in campaign D (s)

74% 91%

Campaign C Campaign D

240

Seconds from the burst (campaign A)

200

0 50 100 Share of rollout-sandbox life (%), sampled traces

Fraction

Sandbox creates per second

(a)

1 80% within 10 s

0.5

0

0.1

1 10 100 1k Gap to the last request for

the same image (s), campaign A

Figure 1. What a sandbox service for agentic RL faces (production records). (a) Creates per second around campaign A’s largest burst, when two runs started together. (b) One campaign-D rollout sandbox of median life and tool-call count. (c) Where rollout-sandbox time went in campaigns C and D (sampled traces). (d) Cumulative fraction of gaps between requests for one image in a run of campaign A.

tiny sandboxes in a scale test, and in October 2026 four production SWE campaigns, 122 runs on up to 432 B200 GPUs, created 3.13 million sandboxes on it (§5.1, §5.2). 2. Design considerations for RL training sandboxes, each with its cost: where to run them, how to create and place them, when to prewarm, who owns cleanup, how to keep grades valid and how to isolate them (§4). 3. An open-source release of Sandman’s daemon, provider and pool, to be published.

ing is asynchronous: dozens of trajectories per run are in flight while the learner trains on completed, graded groups. Asynchrony can hide some sandbox latency, but slower acquisition can still reduce throughput by holding in-flight slots longer. A rollout’s record is its trajectory; an attempt is one agent-session start. An acquisition lasts from the trainer’s request until the trainer holds a usable sandbox and may include a create, which ends when the new sandbox’s daemon answers. Our four production campaigns, A–D, each a set of training runs, ran in October 2026 (Table A2). 2.2

4. Measurements and lessons from production and controlled runs: what bounds startup, what prewarming buys and costs, where sandbox waits fall in training, and the failures we met (§5, §6).

2

BACKGROUND AND M OTIVATION

2.1

Rollouts and sandboxes in agentic RL

The four production campaigns analyzed here train coding agents on SWE-bench Verified (Jimenez et al., 2023; Chowdhury et al., 2024); some experiments use SWE-Gym (Pan et al., 2024). A task is a GitHub issue, the repository at the benchmark’s base commit, and tests that check a fix; the benchmark ships an image per task. A rollout first acquires a sandbox started from that image. The agent harness then alternates between asking the model for its next action and running that action with shell, read, write and edit tools, for tens of turns. When the agent stops, the trainer extracts the patch as a diff against the base commit and applies it in a separate sandbox from the same image. The trainer runs the task’s tests there, turns the verdict into a group relative policy optimization (GRPO) (Shao et al., 2024) reward and releases both sandboxes. Two properties of this workload shape Sandman. First, a sandbox must keep the rollout’s files and processes between calls, so it cannot be swapped mid-rollout. Second, train-

Motivation

One training run creates thousands of sandboxes, and many runs share the cluster. Our 122 October runs averaged about 26,000 sandboxes each. A run starts many trajectories together: 64 within one second in campaign A. When two of A’s runs started together, sandbox creates reached 132 in one second, against a campaign mean of 1.8 per second (Figure 1a). Campaign C kept 256 trajectories in flight per run, campaign D ran up to 27 runs at once, and each trajectory also needs one or two grading pods. Every create passes the cluster’s shared admission, scheduling and network configuration, so the creation path must absorb bursts that average rates and live-pod counts hide. The managed service we tried first could not meet this demand (§4.1). A sandbox can be a service, but training shapes it. Tool execution has a clean boundary: the trainer acquires an environment, sends it commands and file operations, and releases it. A sandbox service can therefore stay independent of the model. Training still sets its requirements. A rollout sandbox must keep its files and processes between tool calls while the model generates its next action, and that waiting took 74–91% of its lifetime in our production traces (Figure 1c). A production run drew on 24–88 task images, and about 80% of requests for an image arrived within ten seconds of another request for it (Figure 1d). Prewarming

Sandman: Sandboxes for Agentic RL on Shared Kubernetes Training system (Ray job)

training system

Sandman

Rollout manager (preflighted task set)

1 task

Provider + pool warm pods: option

2 acquire

shared cluster

default

option

Agent worker coding-agent harness

local proxy

3 inference

Inference engines (GPUs) weights

model calls and tool calls alternate until the agent stops captures the patch, applies the grading policy

6 release

3 tool calls by pod address

prewarm

4 patch

7 reward

Learner (GPUs)

5 grade

create, delete (spread, paced) Node

Kubernetes API + admission webhooks

Scheduler Shared EKS cluster conceptual view

Grading sandbox never-used pod per suite; Sandman’s probe, then the trainer’s grader runs the tests

Rollout sandbox task image, unmodified; execd mounted read-only outbound guard: outgoing traffic rejected

or a pod reused after a state check (option)

token + inbound guard on execd’s port

Pod network: address assignment Kubelet: image pull from the registry, node image cache

Figure 2. Sandman’s components and the path of one SWE rollout (conceptual), colored by code ownership; Sandman’s provider and pool run inside the training job, which captures patches, runs graders and applies the grading policy. Solid arrows are defaults and dashed ones options; in our SWE job defaults, both guards apply to every sandbox. A task goes to an agent worker (1) that acquires a sandbox (2), alternates model and tool calls (3), captures the patch (4), has it graded in a separate pod (5), releases both (6) and sends the learner its reward (7).

Table 1. Calls used by the SWE trainer integration. Our SWE jobs run the optional pool with no warm pods. Component

Calls

Pool (one Ray actor per job) Provider (library) execd (HTTP, one per pod)

acquire(image, base commit, spec) → sandbox id, address; release(id); prewarm(images); drain() create(spec) → handle; destroy; health execute command, read file, read file bytes, write file, env, reset, state, health

could serve these repeats, but a prewarmed pod reserves resources and serves only compatible requests. Tasks also differ in the outbound access their tests need: of the 15 reference patches whose verdicts flipped under changed run conditions, 11 flipped with outbound traffic denied. Network rules must be set per job and checked before grading.

3

S ANDMAN OVERVIEW

3.1

Components and the path of a rollout

Sandman’s daemon, provider and optional pool support the SWE trainer integration in Figure 2. Table 1 lists its calls. execd runs the agent’s commands inside the unmodified task image. Every sandbox runs execd, a small HTTP server that runs commands, reads and writes files, sets environment variables, resets its shell and reports health. One persistent bash process per sandbox keeps the working directory, environment and background processes across tool calls. execd needs only bash and Python 3, and jobs mount it read-only, so task images run unmodified.

The provider turns one sandbox request into one pod. The request names an image, CPU, memory and a workspace. The pod’s requests equal its limits, it carries no service-account token, and an owner reference ties it to the trainer’s pod. A sandbox is ready when its readiness probe passes and execd answers the provider’s poll, within a per-job budget that includes the image pull. The provider also paces and spreads creates, installs the network guards and owns each create until it settles (§4.2–4.6). An optional pool prewarms pods and paces the creates it makes. The pool, a Ray (Moritz et al., 2017) actor shared by the trainer’s workers, keeps never-used pods per (image, base commit), caps live pods and concurrent creates, and paces creates with one token bucket (§4.2, §4.3). Our SWE trainer acquires a sandbox for each rollout and a separate one for each grading. The trainer runs one agent worker, a Ray actor, per agent slot. The worker takes a task (step 1 in Figure 2) and acquires a sandbox (2). The harness’s tools call execd over HTTP at the pod’s address, and its model calls go through a local proxy to the engines (3). When the agent stops, the worker captures the patch (4), has it graded in a separate sandbox per suite (5), releases both (6) and sends the learner the reward (7). 3.2

Responsibilities and the trust boundary

Sandman depends on shared cluster services: the API server, admission webhooks, the pod network (the Virtual Private Cloud Container Network Interface plugin, VPC CNI), DNS and node groups. Our team also runs these services for

Sandman: Sandboxes for Agentic RL on Shared Kubernetes

the cluster’s other workloads. Sandman’s service account creates and deletes pods in its namespace but can neither read the webhooks nor create network policies. Within Sandman, the pool or the worker owns a pod while it is in use, the provider performs every delete, and the owner reference lets Kubernetes delete leftovers at job end. We trust our trainer, our graders and the cluster’s control plane, not model-written code or other workloads’ pods. Model-written code comes from our own models and runs as root in runc containers that share each node’s kernel with other pods (§4.6).

4

D ESIGN C ONSIDERATIONS

Each subsection states a design choice and its cost. 4.1

Where sandboxes run: managed service or self-managed

The managed service we tried held constant what training needs to vary. We first used Amazon Bedrock AgentCore Code Interpreter (Amazon Web Services, 2026a). The service sizes every session at 2 vCPUs and 8 GB with a 10 GB disk, so a job cannot size a sandbox to its task, and it could not run task images, which ruled out SWE tasks. Its default per-account limits are 1,000 concurrent sessions, fewer than campaign D’s up to 1,728 trajectories in flight, and 30 calls per second on session starts, stops and invokes (Amazon Web Services, 2026b). The limits are adjustable; we had starts and invokes raised. When a batch of our Python and math rollouts recycled sessions together, stop calls reached about 117 per second, inflated by our retries, and trajectories timed out. Hosted runtimes (E2B, 2026; Modal Labs, 2026; Daytona, 2026) run custom images behind their own APIs; we did not test them. Self-managed sandboxes run on the training cluster’s CPU nodes, not its GPUs. Each sandbox is a pod on the CPU nodes of the EKS cluster that already runs our training jobs. In mid-October these were 378 96-vCPU nodes of four families (c5d, m5d, m6id, m8id) with local disks. A job sets its sandboxes’ image, size, readiness budget and network rules, and its agent workers reach them over the cluster network. Unlike a node-local container manager, Sandman needs no privileged daemon on the nodes. The cost is running the creation path, which shares the API server, webhooks, pod network and nodes with others. 4.2

scheduler, address assignment and kubelet. The network plugin (Amazon Web Services, 2026d) assigns addresses from a few spares it keeps warm per node, two in our campaigns and experiments, so a burst packed onto a few nodes waited for addresses while the cluster had room (Table A14). Pool-mediated creates share a pacing budget. Each provider paces its creates with a token bucket. With the pool, one provider, bucket and create cap serve every create the pool makes: on-demand acquisitions, warm builds and replacements. With no warm pods the pool still creates on demand; our SWE jobs run it this way. When the pool raises an error other than a timeout, such as a full pod cap, the trainer’s worker makes one direct create through its own provider, outside the job’s bucket and cap; 4.3% of gradings in the pre-registered comparison took this path (Appendix K). The pool’s limits are therefore not a hard job-wide admission limit. Without the pool, each agent worker paces only its own creates, and simultaneous creates from different workers can still form a burst. No budget spans jobs, so concurrent jobs’ rates add up. In the scale ramp, one pool with no warm pods and a test pacer held 12,000 acquires fired at once to 20–25 creates per second (§5.2). The provider spreads sandboxes with a soft topology constraint (maximum skew 2), a scheduler preference rather than a per-node limit. CPU nodes keep eight spare addresses warm, the most a spread burst put on one node in our tests. Agents call their sandbox by pod address, not through a Service. Our trainer creates only the pod, sends the harness each trajectory’s address and waits for an acknowledgement; Service environment variables are off. An earlier design put each agent slot behind a Kubernetes Service with a stable name, because the harness read its sandbox address once while the pod changed every trajectory. But Kubernetes injects every Service’s variables into each new container in the namespace (The Kubernetes Authors, 2026a). A pod behind a Service is reachable only once its route is programmed, and each Service is another object to create. Pacing and spreading trade caller wait and image pulls for a shorter tail in our tests. Pacing adds our queue to a caller’s wait, and spreading pulls images onto more nodes. We did not place sandboxes by image: that packs creates onto fewer nodes, where address failures concentrated, and the scheduler’s image-locality score sees only a node’s 50 largest images (The Kubernetes Authors, 2026b). Eight spare addresses per node, not two, reserve about 2,300 more over the CPU pool; larger bursts need more nodes or pacing.

Sandbox creation and placement

Sufficient capacity does not guarantee fast startup. Resource requests, subnet addresses and lifecycle operations limit creates at different times. At 8 GiB per sandbox, a c5d node holds 21 sandboxes by memory and 46 by CPU. Every create passes the API server, admission webhooks,

4.3

Sandbox provisioning: on demand, prewarmed or reused

An on-demand create of a large image is fast only if its node holds it (§5.3). Nodes keep task images on local disks until disk pressure evicts them. An admission rule makes

Sandman: Sandboxes for Agentic RL on Shared Kubernetes

every create pull a tag reference, so a security fix can move a tag; a digest reference is pulled only when the node lacks it. Layers already on the node are reused either way. We create on demand by default; warm and reused pods are options. In our asynchronous trainer, rollout creation overlaps other work, while a warm pod, started but never used, holds its full request as it waits. The warm pool keeps never-used pods per (image, base commit). The pool hands one out only after a health check and only if its CPU, memory and workspace equal the request’s. It starts a replacement create whenever it hands out a pod and deletes pods of images not requested within a timeout. Reuse is a second option, for grading pods. Since git reset and git clean do not return a pod to its starting state, at create execd records a baseline of what graders were seen to change (repository, installed packages, system and home directories, environment, time zone, processes). After a grading, execd undoes differences exactly or sends the pod to deletion. A pod is reused at most 8 times and retired when its grading contradicts an earlier one.

Table 2. Three ways a pod outlives its caller. Windows: stand-inAPI tests that left a pod without an owner, before the fix / after the first (cancellation) fix / now, 20 repeats each (Table A21); on the cluster, 25 of 25 trials left no pod. Case

Why it is missed Who owns it now

Windows that left a pod

Caller cancelled or the caller has left provider finishes the create; timed out mid-create pool keeps or deletes the late pod API answers an error an error is not provider reads it by name; after storing the pod proof of no pod deletes it on a nonce match Delete fails or its a sent delete is provider retries until reply is lost not a done delete confirmed; owner reference at job end

12/0/0 of 15

3/2/0 of 3 6/5/0 of 6

the owner reference lets Kubernetes remove them. A timed-out command can leave children running. execd therefore kills each command’s process group on a timeout or reset and caps output while draining it. It also checks a command’s syntax before running it and accepts an exit status only in a complete frame.

Only a digest reference fixes the image a warm pod serves. Warm pods are keyed by the image reference as requested. All our campaigns and experiments referenced task images by digest, so a warm pod, a rollout and its grading ran the same image. A tag, which each create resolves again, carries no such guarantee.

Owning every create costs waiting and capacity. A cancelled caller waits for its create, and late warm pods hold capacity. Commands are sent once, since a timed-out one may have half-run; safe retries would need request deduplication in execd. We have no periodic sweep for ownerless labelled pods, which would catch unknown leaks.

Warm pods cost capacity image by image. A warm pod serves only its own image, so pools are sized per image by request rate, burst and create time. Because the pool replaces every pod it hands out, a reused pod returns to an already refilled pool; we did not test replacing only unreturned pods. We did not adopt lazy image loading, which needs another snapshotter on every node. Requests equal limits, so idle time between tool calls is a cost; we did not overcommit.

4.5

4.4

Cleanup when the caller gives up

Cancelling a caller does not cancel its create. A trainer may abandon an acquisition after a timeout although the API server has already accepted the pod. If the cancellation also cancels the caller’s cleanup, the pod becomes ready with nobody waiting for it, and a retry creates another pod; in a stress test, this path left at least 4,181 excess pods. Table 2 lists this and two other ways a pod outlives its caller. The provider therefore owns each create until it settles: the create runs even if its caller is cancelled, and cleanup runs on any exit, even a second cancellation. The pool keeps the create of a timed-out acquire: a late pod without an agent label becomes a warm pod, and any other late pod is deleted. A nonce on each create lets the provider find and delete a pod that the API stored but answered with an error. An unconfirmed delete is retried until the job drains; deletes pending when the provider’s event loop closes are lost, and

SWE grading: files, patches and verdicts

An edit must start from the exact bytes. The model may see a trimmed view of a file, but the editor reads whole files byte-exact because it writes them back. It checks each read’s length and SHA-256 and refuses files it cannot write back unchanged (not valid UTF-8, mixed line endings, a bare carriage return), telling the agent to use the shell instead. A patch holds only the agent’s change, and each task gets a reference/empty-patch preflight. An agent once planted a conftest.py that made the tests pass without a fix, so test-configuration files are left out of patches. The trainer resets the index to the task’s base commit, stages the agent’s changes and diffs, so the patch is the agent’s change against the baseline. The tests of two Verified tasks do not check the fix. Before a job starts, it grades each task’s reference patch (must pass) and an empty patch (must fail) under its own configuration and keeps only tasks passing both. A verdict is only as valid as its run conditions. Each test suite runs in a separate grading pod, never-used by default, keeping the agent’s processes and files out of the grade. For each task we declare the conditions its verdict may depend on: image, outbound network, CPU quota, visible CPUs, memory limit, user, time zone, hash seed, time limit and test suite. Before every grading, an in-pod probe run under the tests’ CPU pin reads eight of these conditions, all but the

Sandman: Sandboxes for Agentic RL on Shared Kubernetes Table 3. What each boundary protects and what it does not. Boundary

Protected

Execution

runc container, not privileged; guarded shared kernel; root user; no pods drop raw sockets seccomp profile, user namespace or read-only root network policy denies egress; in-pod – rules block all outgoing traffic but loopback and replies, failing closed network policy admits only trainer past use (no request logs) pods; per-sandbox token on every call but health checks; in-pod allowlist of the node and consumers one-hop limit (not re-measured) –

Outbound

Inbound

Metadata

Not protected

image and the test suite, and refuses the pod if a readable field contradicts the job’s declaration. An unreadable field does not refuse the pod, so a passed probe does not prove the environment matches. The image and the test suite are set, not probed: the grading pod starts from the task’s image reference, which fixes one image only as a digest, and the official tests are restored from the base commit. A grading that fails for an infrastructure reason is retried and, if still without a verdict, resampled, never scored 0. Failing tests and patches that do not apply stay valid zeros. The checks cost pod-hours and miss agent-specific effects. The reference/empty-patch check costs 0.06–0.14 pod-hours per task at the job’s configuration and sees no agent patch. 4.6

Isolation and access control

An outbound guard does not authenticate callers. Earlier deployments blocked outbound traffic from some sandboxes, but their endpoints stayed reachable without authentication from other pods: a live check from an unrelated pod invoked commands in guarded sandboxes. Pod addresses are routable cluster-wide, and a call into a grading pod could change its files and so a reward. execd checks a per-sandbox token on every call but health checks (Table 3). A network policy that the cluster applies by label and the network plugin enforces denies sandbox egress and admits only the trainer’s pods to execd’s port. In-pod rules that the execd container cannot change stay as a second guard. A label alone is not enforcement. In one live check, 6 of 6 pods labelled egress-denied reached the internet, because only a patched provider installed the guard. The provider refuses an egress-denied label unless it installs the guard. We accepted runc’s shared kernel. A microVM per sandbox (Firecracker (Agache et al., 2020), Kata (The Kata Containers Authors, 2026)) needs bare metal or nested virtualization (Amazon Web Services, 2026c), which one of our four node families has; gVisor (The gVisor Authors, 2026) only partly supports our outbound guard’s packet filter (The gVisor Authors, 2025). Our sandboxes run code from our own models, not outside users, so we accepted the shared kernel. No pod sets a seccomp profile, and the kubelet sets

Table 4. Where key mechanisms ran (all 16: Table A4). Production: campaigns A–D; Now: defaults of our code and SWE job templates; p: in part; o: option. Mechanism Agents call pods by address, no Service Paced and spread creates Create keeps an owner on cancel Outbound guard on every sandbox Token, inbound guard, network policy Condition probe, task preflight Warm pool and checked reuse

Production RL experiments Now – – – C–D – – –

p p p ✓ p p grading pods

✓ ✓ ✓ ✓ ✓ ✓ o

no default (The Kubernetes Authors, 2026b); the runtime default’s effect on verdicts is untested. We do not claim isolation against a malicious co-tenant.

5

E VALUATION

We evaluate Sandman in production and in controlled experiments: its use and cost, creation at scale, prewarming and reuse, and how sandbox waits reach training. The studied SWE runs use Qwen3.5-35B-A3B with asynchronous GRPO, most on 16 B200 GPUs. Eight GPUs serve the learner and eight host SGLang inference engines (32 and 32 in the 64-GPU campaign), with eight samples per task. Rollout workers keep a fixed number of trajectories in flight. Each learner step consumes the next complete, graded groups (four groups of eight in most runs), and weights go to the engines after every step while generation continues. A trajectory has a 30-minute limit and 300 s per command; a test-suite run has 1,800 s. Unless stated otherwise, a sandbox requests 2 vCPUs and 8 GiB, requests equal to limits; the scale ramp used tiny pods. The controlled experiments include a scale ramp, a placement contrast, warm-pool and reuse A/Bs, a grading-condition study and a pre-registered comparison (Table A1). Each result names its evidence kind. Production results come from the campaigns’ and other jobs’ logs, run records and the API audit log. Controlled experiments are dedicated runs on the shared cluster. Replays run recorded traces through the real pool code or a packing model, and reproductions use a few pods or a local copy to show that a failure path exists, not how often. Create-to-Ready ends at Ready status; the waiting share is acquisition over trajectory time. CIs are 95% t-intervals over run pairs unless named (Wilson for proportions), and A/B arms ran concurrently. Table 4 shows which key mechanisms each kind of evidence ran. 5.1

Sandman in production

Four SWE campaigns created 3.13 million sandboxes in two weeks. From 3 to 17 October, these campaigns ran 122 training runs on about 21,400 B200-hours. They created 1.24M rollout and 1.89M grading sandboxes at 1.8–

Sandman: Sandboxes for Agentic RL on Shared Kubernetes p50 p99

in address-failure window (5–39%) outside it (0–5%)

(b)

(c)

1 spread

120 s ready timeout

100

Fraction of pods

Pod create to Ready (s)

(a)

50 20

tiny pods: 0.1 vCPU, 128 MiB, 43 MB image 99 250 500 1k 2k 4k 8k 12k (12) (2) (6) (4) (3) (1) (1) (1) Requested sandboxes N (steps at that N)

default

pod address + warm pods

solid: create to Ready, p99 8 s vs 57 s

0.5

dashed: from submission, queue included: p50 56 s vs 62 s

10 5

p50 to p90, 99 per arm

0

3

10 30 100 300 Seconds (two batches of 1,000 pods, 10 creates/s)

0.05

Service + warm pods

2.3

pod address no warm pods

8.5

Service no warm pods†

10.4 0.1 1 10 Acquire to first execd call (s)

Figure 3. The creation median held at every scale; the tail coincided with pod-address assignment failures. All panels are controlled. (a) Create-to-Ready p50 and p99 of all 30 steps, arms and controls included (tiny pods, cached image); filled: inside the address-failure window. (b) One sequential pair of 1,000-pod batches at 10 creates/s: create-to-Ready and wait from submission, pacer queue included. (c) Acquire to first successful execd call, 99 sandboxes per arm (dot p50, whisker p90); †overlapped the failure window.

6.3 creates per second on average per campaign, with up to 27 runs (432 GPUs) and at most 1,728 trajectories in flight at once. Rollout sandboxes lived 136–207 s at the median and 424–457 s at p99 (Table A2). From late July to mid-October, about ten other users ran at least 299 jobs, up to 896 GPUs each. Most attempts got a sandbox; failures clustered in webhook incidents. Between 99.73% (A) and 91.16% (C) of attempts got a working sandbox. In C, 99.5% of failures were rejections by unreachable admission webhooks, and after one retry at least 96.69% of trajectories had a sandbox (Table A5). In D, 69.5% of attempts got a pod: in 12 runs during the 14 October incidents, bursts of agent-backend errors associated with webhook failures were retried as new starts. D’s other 27 runs reached 99.7%. In A–C, 33 of 576,666 attempts could not reach execd in time. Sandboxes cost 1.7–3.5% of the GPU cost. Campaigns A– C held 3.30–3.75 sandbox-hours per B200 GPU-hour. At on-demand list prices, that is about 1.7% of the GPU cost if a sandbox is charged its node share of CPU and 3.5% if charged its share of memory, or 1.8–3.7% with test re-runs and evaluation jobs. Requests, not use, set this cost: 198 sampled sandbox pods of all users used 10.5% of their CPU and 1.9% of their memory requests, by pod-minutes. 5.2

Creation at scale

The control path held 11,858 concurrent tiny sandboxes. A ramp of the real pool and provider to 12,000 requested sandboxes, paced at 20–25 creates per second, kept create-

to-Ready at a 5–7 s median (Figure 3a). The ramp met no admission error in 38,377 creates, including a sweep of 5– 200 creates per second, and we stopped at 12,000 to keep subnet headroom. The pods were tiny (0.1 vCPU, 128 MiB, a cached 43 MB image), so the ramp shows what the control path held at this rate, not sustained churn. Most of their 5–7 s was our readiness detection and execd starting under the tiny quota; with 2 vCPUs, create-to-Ready took 2 s. The tail came with pod-address failures, more often on crowded nodes. During a 2.2-hour window, 5–39% of each step’s pods first failed to get an address and retried, which pushed p99 to 21–102 s. Every step of the initial ascending ramp above 99 sandboxes fell inside this window, so the ramp cannot separate size from the address failures. In a follow-up ramp, pods on nodes that received 20 or more of a step’s pods failed their first assignment 31–55% of the time, against 0–16% on nodes with fewer than 10. In one production hour, 0.64% of creates did. With spreading, a 1,000-pod batch had no address failures. Two batches of 1,000 tiny sandboxes ran three minutes apart on the same 375 nodes, both paced at 10 per second: first spread (at most 3 pods per node), then default-scheduled (up to 65). The default batch had 254 first-attempt address failures, all later Ready. Create-to-Ready p99 was 8 s with spreading and 57 s without, although the default batch found the image cached on every node (Figure 3b). Callers’ median wait from submission (56 and 62 s) was mostly our pacer queue. Other workloads’ failures rose during the default batch and stayed high after our pods left, so a change over time confounds the comparison, and a pre-registered

Sandman: Sandboxes for Agentic RL on Shared Kubernetes sandbox + CNI image pull

(b)

container start API, readiness, poll

(c)

image pulled (n = 422) 1

p90 74.3 mean 32.0

0 20 40 60 image on the node (n = 230; axis 0–6 s) p50 4.1

80

p90 5.1

Fraction of gradings

p50 17.4

pool on (3 runs) pool off (3 runs) p50 26–247 ms on, 5.2–7.5 s off; 93–96% of gradings got a warm pod

0.5

Idle warm vCPU-hours per run

(a)

replay of each run's trace measured in the A/B runs 400 300 200 100 0

120 s

2

4

Create request to healthy pod (s); bars: mean of each phase

6

300 s

2,100 s

(93–96%)

mean 4.1 0

180 s

77–82% 88–93% 94–98% 95–98% 0

Idle timeout; replayed warm-hit rate 0.1

1

10

100

Learner step interval, on − off (s)

Grading-pod acquisition, acquire call to usable pod (s, log scale)

−40 −20 0 20 mean −8.7 s, 95% CI [−38.4, +21.0]

Figure 4. Where a start spends its time, what a warm pod saves and what it costs. (a) Per-phase means adding up to the mean create-tohealthy time, SWE-Gym images (records of controlled runs; cache state observed, not set; axes differ); markers: p50, p90. (b) Grading-pod acquisition, three pairs of runs, pool off and on (controlled). (c) Idle warm vCPU-hours and hit rates per run by idle timeout (replayed; measured: diamonds, hit rates in parentheses); below, the measured step-interval difference (95% CI).

stop rule ended the experiment. This is one sequential pair, and spreading was seen only up to 8 pods per node. 5.3

Prewarming and reuse

Pulling an image quadrupled the median create. In a re-grading study on SWE-Gym images of 1.3–8.4 GB, a create that had to pull took 17.4 s at the median (p90 74.3 s, 422 pods). With the image on the node, a create took 4.1 s (p90 5.1 s, 230 pods), compared within each image, node and half hour (Figure 4a; Table A16). The cost grew with image size. We could not measure how often production creates pulled. Slow pulls also became failures: a fixed 120 s readiness budget that included the pull rejected 9 admissible tasks of a 280-task set, hence the per-job budget. A warm pool cut grading acquisition 50-fold but more than doubled sandbox vCPU-hours. Three pairs of concurrent 16-GPU runs differed only in whether grading sandboxes came from the pool. The pool served 93–96% of gradings warm and cut grading acquisition from 6.2 s to 0.12 s (means of per-run medians); the waiting share fell from 7.0% to 4.2% (Figure 4b). A warm pod is still created and then idles on its full request. At a 2,100 s idle timeout, longer than a run’s 27-minute cycle through its 88 images, pool-on runs used 2.46× the pool-off runs’ sandbox vCPU-hours, mostly in idle warm pods. In replays, which overstated hits at 2,100 s by 1.8–3.5 points, a 300 s timeout kept the warm-hit rate at 94.5–97.7% at about half the idle cost (Figure 4c). The warm pool shortened time to step 24 by 9.4%. Pool-

on runs reached step 24 sooner in all three pairs (95% CI 5.9–13.0%), in two of the three mostly in the first step. The steady-state effect is unresolved: the mean step interval over steps 2–24 differed by −8.7 s (95% CI −38.4 to +21.0), and three pairs detect only changes above 24% of the step interval (80% power, α = 0.05). These runs do not attribute the gain to faster acquisition. Reuse cut creates against a warm pool, but on demand used fewer pod-hours. Checked reuse cut creates per grading by 29% against a warm pool (95% CI 22–36%), and reused pods served 34–43% of gradings. In the preregistered configuration (§5.4), reuse made 1.21 creates per grading against 1.00 on demand, and on demand used the fewest pod-hours in all eight replayed traces. The state check took 0.23–0.51 s at the median, off the grading path. Fallback rates depended on the graders: 8 of 64,199 checks on Verified task sets moved the grading to a fresh pod, against 23.7% on SWE-Gym, whose graders rebuild artifacts. In the pre-registered comparison, whose grading change used the pool with reuse, warm and reused hits took about 12 ms. Queue-inclusive acquisition averaged 20.3 s, against 7.2 s with direct creation: the pool built pods at 0.3 creates per second under a 32-pod cap, so over half the requests waited for its creates. In this configuration queueing offset the fast hits, and acquisition took 13% of step time against 4.5% with direct creation. 5.4

Sandboxes and training

Of sandbox time, grading acquisition and tests, not rollout acquisition, lay on the critical path. Most runs were

Sandman: Sandboxes for Agentic RL on Shared Kubernetes grading acq. (%)

Campaign A, no re-runs

6.8

Campaign A, tests re-run

11.8

Warm-pool A/B: fresh

6.0

Warm-pool A/B: warm

2.0

Reuse A/B: warm

1.0

Reuse A/B: reused

1.1

Pre-registered: no change

4.5

Pre-registered: with change

13.3

0 25 50 75 100 Share of learner-step time on the critical path (%) rollout acquisition tests failed attempts grading acquisition agent turns and other learner and trainer

In the one pre-registered training comparison, the arm with the grading change resolved fewer held-out tasks. Six pairs of 150-step runs with campaign D’s recipe compared an arm with a combined grading change against one without. The change dropped the three tasks that the reference/empty-patch check flagged, graded through the warm pool with checked reuse, and required the execd token on pooled grading pods. On 112 held-out tasks, resolve rates after training were 46.1% with the change and 48.4% without (26.1% for the base model). Over all six pairs, as registered, the difference was −2.33 points (95% CI −4.22 to −0.43; Appendix K). The change altered three things at once, so the design cannot isolate a cause. Post hoc, its arm trained on 3.4% fewer tasks, two of them tasks whose tests reward any non-empty patch, so it also lost easy reward.

6 Figure 5. Where learner-step time went on the critical path, over 3,734 traced steps (production: campaign A; the rest controlled). Campaign A ran up to two test suites, and two of its arms re-ran suites when a batch went to the learner; the reuse A/B’s failed attempts fell in incident windows. Grading acquisition at right; the pre-registered arms ran with and without the grading change.

generation-bound: in A–C the learner waited for data during 65–78% of each step, and in most traced runs 97–100% of inflight slots were busy (88–96% in the partly learner-bound pool A/B). We traced 3,734 learner steps of campaign A and three controlled experiments to the trajectory that ended each wait, which lies on the step’s critical path (Figure 5; Appendix J). Rollout acquisition took at most 0.9% of step time on this path, since other trajectories’ work overlaps it. Outside incidents and test re-runs, the end of the slowest trajectory was on the critical path, through grading acquisition (4.5–6.8% with fresh pods, 1–2% with the A/Bs’ warm pools) and tests (8–11% with one suite, 38% with two). A fixed-concurrency model estimates the potential benefit of faster sandboxes. If a run stays generation-bound and generation does not slow down, removing a fraction f of trajectory time shortens the step interval by about f . Little’s law matched every measured mean step interval within 3% (Appendix J). Counting acquisition, failed attempts’ slot time and test re-run creates, the model estimates a 6–14% shorter step per campaign (1,800 of 21,400 B200-hours) and 3–5% for the warm pool; neither is measured. Changing one run condition at a time changed at most 12 of 11,399 verdicts on a robust task set. Re-grading pair 1 of both A/Bs on fresh pods, with one of six conditions changed at a time, changed no verdict beyond repeat noise, and a non-root user changed 12 verdicts (0.1%; Appendix H). That set held no condition-dependent task.

L ESSONS FROM O PERATING S ANDMAN

We draw seven lessons from operating Sandman. Judge a faster start by training progress and cost. With dozens of trajectories in flight, one sandbox wait overlaps others’ work. In the observed schedules outside incidents, grading acquisition dominated sandbox waiting on the lastcompleting trajectories. This attribution does not by itself measure the throughput benefit of faster rollout acquisition, which can also release in-flight slots earlier. Report acquisition with the caller’s queueing included, and weigh a warm pool’s hit rate against its idle pods (§5.3). Plan for create bursts, not only live sandboxes. On 14 October, an evaluation with 4,096 concurrent trajectories created sandboxes without pacing. The cluster’s policy admission webhook ran one replica with a 384 MiB limit and had restarted 115 times since early September. The webhook answered pod creates with errors: 84.9% of the evaluation’s 6,144 trajectories failed, and so did 6,126 creates of other jobs. The cause was the webhook’s configuration, and our burst was the trigger. The webhook runs three 1 GiB replicas on separate nodes with a disruption budget, and our SWE jobs pace creates through the pool. The scale ramp’s 38,377 paced creates later met no admission error, which is consistent with the fix but does not prove it. Pacing across a job’s workers needs the shared pool, and aggregate pressure still depends on concurrent jobs and other cluster workloads. Turn off Service links. On 10 October, the namespace held 3,255 Services, mostly another team’s, and Kubernetes put about 23,400 of their variables into every new container. In campaign A (3–4 October), as Services accumulated, the median bash call slowed from 0.63 to 2.12 s. With Service links off, a production smoke test’s median bash call took 16–22 ms. With Service creates unpaced, 241 of 1,000 sandboxes failed in a Service-routing test, 237 of them on throttled Service creates.

Sandman: Sandboxes for Agentic RL on Shared Kubernetes

Select the whole node pool and watch free addresses. Workers pinned to one instance type found its 115 nodes full of other tenants’ pods; 9 jobs started only after selecting the pool’s label. During our ramp, free pod addresses fell from 33.6k to 26.5k at 12,000 sandboxes; the tightest subnet is shared with the GPU nodes, so we stopped there. Keep cleanup with the provider once a create is submitted. In a stress test, acquire timeouts cancelled creates that the API server had already stored. Of 13,516 rollout pods, 5,904 were removed only by garbage collection at teardown; the leaks filled the job’s node type. With the provider owning each create, no tested failure left an unowned pod: Table 2’s 24 windows and a name conflict, 20 repeats each, on a stand-in API, and 25 of 25 cluster trials. Preflight every task under the job’s own configuration. The tests of two Verified tasks do not check the fix. Between 19% (missing tests restored) and 23% (a published hardened suite for one task (Yu et al., 2026)) of campaign D’s positive rewards on these tasks went to wrong patches, 0.31–0.37% of all of D’s rewards. Under 16 run configurations, 15 of 500 Verified reference patches flipped, 11 of them with outbound traffic denied, and 4.79% of 2,401 SWE-Gym tasks (Pan et al., 2024) depended on run conditions. Preflighting each task’s reference and empty patches under the job’s configuration flags tasks whose reference fails or whose empty patch passes; it misses effects only agent patches trigger. Most recorded cases had a root in our own implementation. Of 41 recorded operations cases, 23 had a root in the sandbox implementation (code, design or configuration), alone or with a Kubernetes default or a cluster service. Eight lay in cluster configuration or a shared service alone; 10 had other or unknown causes. The ten cases with a GPU-hour account have upper bounds totaling 7.1–7.8k B200-hours; at most 0.47k fell in the campaigns’ 21.4k (2.2%), and the rest hit a stress test, an evaluation and an extension of campaign D (Table A49). Losses without an account, such as campaign D’s error bursts, are not estimated. These are recorded cases, not a census (Appendix M). Limitations. We studied one shared EKS cluster. The detailed training measurements cover SWE-bench-style Python tasks and one agent harness; reproductions and standin-API tests ran on one host. The ramp used tiny pods, each A/B has three pairs, replays hold trajectory timelines fixed, incident losses are upper bounds, and the security checks are live probes of our own sandboxes, not adversarial tests. Runs drew on 21–306 images with 48–256 trajectories in flight and a fully asynchronous trainer; many more images or a synchronous trainer may change the pool and criticalpath results. We have no dedicated-cluster baseline and no internal measurements of the webhooks or the network plugin.

7

R ELATED W ORK

Sandbox platforms. DSec (Huang et al., 2026), concurrent with this work, serves about 3M sandboxes a day on a purpose-built fleet with its own placement, image distribution and isolation backends. AgentENV (Kimi Team et al., 2026) forks Firecracker microVMs, Kimi K2 (Kimi Team et al., 2025) runs over 10,000 concurrent sandboxes on Kubernetes, and hosted runtimes (E2B, 2026; Modal Labs, 2026; Daytona, 2026; Wang et al., 2026) expose isolation and lifecycle APIs. The Kubernetes agent-sandbox project (Kubernetes SIG Apps, 2026) mounts a daemon into unmodified images, as execd does. Sandman instead uses ordinary pods on a shared training cluster and measures how sandbox waits and grading conditions reach training. Preparation and reuse. Graviet et al. (2026) recommend locality-aware warm pools, and DeltaBox (Dong et al., 2026) checkpoints sandboxes, which would replace our state check. Training on environment outcomes. Rollout systems (Fu et al., 2025; Lu et al., 2025; Cao et al., 2025; Ren et al., 2026; Hong et al., 2026) treat the sandbox as a black box. GLM-5 (GLM-5-Team et al., 2026) and LEGO-RL (Du et al., 2026) drop samples lost to environment failures, and dynamic sampling (Yu et al., 2025) drops the zero-variance groups that held most of our condition flips. Grading validity. Resource settings and image rebuilds shift benchmark scores (Segato, 2026; Adamczewski, 2025); runc and gVisor matched on SWE-style tasks (Tan et al., 2026). Dataset pipelines (Pan et al., 2024; Yang et al., 2025; Aleithan et al., 2024; Badertdinov et al., 2025) validate tests once; we revalidate them under each job’s configuration. Infrastructure reports. Industry reports measure fleet goodput (Wongpanich et al., 2026), reliability under failures and stragglers (Quirk et al., 2026; Liu et al., 2026) and a single-project GPU cluster’s workload (Konishi et al., 2026). We report such evidence for agentic-RL sandboxes.

8

C ONCLUSION

Sandman turns task images into stateful sandboxes that training jobs acquire, use across tool calls and release, as ordinary pods on a shared cluster. Incidents, stress tests and studies showed what such a service must handle: paced and spread creates, provider-owned cleanup and gradings checked against declared run conditions. A faster start should be judged by time to a training step and by its cost.

Sandman: Sandboxes for Agentic RL on Shared Kubernetes

R EFERENCES Adamczewski, T. How to run SWE-bench Verified in one hour on one machine. Epoch AI, https://epoch. ai/blog/swebench-docker, updated 2025-07-10, 2025. Agache, A., Brooker, M., Florescu, A., Iordache, A., Liguori, A., Neugebauer, R., Piwonka, P., and Popa, D.M. Firecracker: Lightweight virtualization for serverless applications. In 17th USENIX Symposium on Networked Systems Design and Implementation (NSDI 20), pp. 419– 434, 2020. Aleithan, R., Xue, H., Mohajer, M. M., Nnorom, E., Uddin, G., and Wang, S. SWE-Bench+: Enhanced Coding Benchmark for LLMs. arXiv:2410.06992, 2024. Amazon Web Services. Amazon Bedrock AgentCore Code Interpreter. https://docs.aws.amazon.com/ bedrock-agentcore/latest/devguide/cod e-interpreter-tool.html, accessed 2026-1028, 2026a. Amazon Web Services. Quotas for Amazon Bedrock AgentCore. https://docs.aws.amazon.com/bedr ock-agentcore/latest/devguide/bedrock -agentcore-limits.html, accessed 2026-10-28, 2026b. Amazon Web Services. Use nested virtualization to run hypervisors in Amazon EC2 instances. Amazon EC2 User Guide, https://docs.aws.amazon.com/ AWSEC2/latest/UserGuide/amazon-ec2-n ested-virtualization.html, accessed 202610-26, 2026c. Amazon Web Services. Amazon VPC CNI plugin for Kubernetes. https://github.com/aws/amazon-v pc-cni-k8s, accessed 2026-10-27, 2026d. Badertdinov, I., Golubev, A., Nekrashevich, M., Shevtsov, A., Karasik, S., Andriushchenko, A., Trofimova, M., Litvintseva, D., and Yangel, B. SWE-rebench: An Automated Pipeline for Task Collection and Decontaminated Evaluation of Software Engineering Agents. arXiv:2505.20411, 2025. Cao, S., Li, D., Zhao, F., Yuan, S., Hegde, S. R., Chen, C., Ruan, C., Griggs, T., Liu, S., Tang, E., Liaw, R., Moritz, P., Zaharia, M., Gonzalez, J. E., and Stoica, I. SkyRLAgent: Efficient RL Training for Multi-turn LLM Agent. arXiv:2511.16108, 2025. Chowdhury, N., Aung, J., Shern, C. J., Jaffe, O., Sherburn, D., Starace, G., Mays, E., Dias, R., Aljubeh, M., Glaese, M., Jimenez, C. E., Yang, J., Ho, L., Patwardhan, T., Liu, K., and Madry, A. Introducing SWE-bench Verified.

OpenAI, https://openai.com/index/intro ducing- swe-bench-verified/, 2024-08-13, 2024. Daytona. Daytona: secure and elastic infrastructure for running AI-generated code. https://github.com /daytonaio/daytona, accessed 2026-10-23, 2026. Dong, Y., He, J., Liu, S., Hou, Y., Du, D., Xu, Z., Yu, S., Yang, B., Xia, Y., and Chen, H. DeltaBox: Scaling Stateful AI Agents with Millisecond-Level Sandbox Checkpoint/Rollback. arXiv:2605.22781, 2026. Du, Y., Jiang, Y., Yuan, T., Dai, J., Wang, S., Chen, J., Tao, C., Yu, X., Shang, L., Wong, K.-F., Li, X., and Bai, H. LEGO-RL: Harness-Native Reinforcement Learning for Coding Agents. arXiv:2608.17393, 2026. E2B. E2B: open-source secure environments for AI agents. https://github.com/e2b-dev/E2B, accessed 2026-10-23, 2026. Fu, W., Gao, J., Shen, X., Zhu, C., Mei, Z., He, C., Xu, S., Wei, G., Mei, J., Wang, J., Yang, T., Yuan, B., and Wu, Y. AReaL: A Large-Scale Asynchronous Reinforcement Learning System for Language Reasoning. arXiv:2505.24298, 2025. GLM-5-Team, Zeng, A., Lv, X., Hou, Z., Du, Z., Zheng, Q., Chen, B., Yin, D., Ge, C., Huang, C., Xie, C., Zhu, C., Yin, C., Wang, C., Pan, G., Zeng, H., Zhang, H., Wang, H., Chen, H., Zhang, J., Jiao, J., Guo, J., Wang, J., Du, J., Wu, J., Wang, K., Li, L., Fan, L., Zhong, L., Liu, M., Zhao, M., Du, P., Dong, Q., Lu, R., Shuang-Li, Cao, S., Liu, S., Jiang, T., Chen, X., Zhang, X., Huang, X., Dong, X., Xu, Y., Wei, Y., An, Y., Niu, Y., Zhu, Y., Wen, Y., Cen, Y., Bai, Y., Qiao, Z., Wang, Z., Wang, Z., Zhu, Z., Liu, Z., Li, Z., Wang, B., Wen, B., Huang, C., Cai, C., Yu, C., Li, C., Hu, C., Zhang, C., Zhang, D., Lin, D., Yang, D., Wang, D., Ai, D., Zhu, E., Yi, F., Chen, F., Wen, G., Sun, H., Zhao, H., Hu, H., Zhang, H., Liu, H., Zhang, H., Peng, H., Tai, H., Zhang, H., Liu, H., Wang, H., Yan, H., Ge, H., Liu, H., Chu, H., Zhao, J., Wang, J., Zhao, J., Ren, J., Wang, J., Zhang, J., Gui, J., Zhao, J., Li, J., An, J., Li, J., Yuan, J., Du, J., Liu, J., Zhi, J., Duan, J., Zhou, K., Wei, K., Wang, K., Luo, K., Zhang, L., Sha, L., Xu, L., Wu, L., Ding, L., Chen, L., Li, M., Lin, N., Ta, P., Zou, Q., Song, R., Yang, R., Tu, S., Yang, S., Wu, S., Zhang, S., Li, S., Li, S., Fan, S., Qin, W., Tian, W., Zhang, W., Yu, W., Liang, W., Kuang, X., Cheng, X., Li, X., Yan, X., Hu, X., Ling, X., Fan, X., Xia, X., Zhang, X., Zhang, X., Pan, X., Zou, X., Zhang, X., Liu, Y., Wu, Y., Li, Y., Wang, Y., Zhu, Y., Tan, Y., Zhou, Y., Pan, Y., Zhang, Y., Su, Y., Geng, Y., Yan, Y., Tan, Y., Bi, Y., Shen, Y., Yang, Y., Li, Y., Liu, Y., Wang, Y., Li, Y., Wu, Y., Zhang, Y., Duan, Y., Zhang, Y., Liu, Z., Jiang, Z., Yan, Z., Zhang,

Sandman: Sandboxes for Agentic RL on Shared Kubernetes

Z., Wei, Z., Chen, Z., Feng, Z., Yao, Z., Chai, Z., Wang, Z., Zhang, Z., Xu, B., Huang, M., Wang, H., Li, J., Dong, Y., and Tang, J. GLM-5: from Vibe Coding to Agentic Engineering. arXiv:2602.15763, 2026. Graviet, D. T., Pesut, L., Dagelic, I., Jukic, V., and Burazin, I. The Rollout Infrastructure Tax in Coding-Agent Reinforcement Learning. arXiv:2607.01415, 2026. Hong, Z., Yuan, S., Ding, Y., Zhu, Y., Jiang, D., Wang, Z., and Tian, C. Scheduling Mixed RL Rollouts Beyond Prefix Locality. arXiv:2608.11152, 2026. Huang, J., Tang, H., Chen, J., Liu, Y., Chen, Y., Cheng, Y., Tao, Y., Zhou, J., Chen, Y., Chen, H., Wang, J., Lin, S., Zhang, C., Teng, B. L., Guo, L., Fu, Z., Gao, W., Wang, Y., Zhao, L., Wang, Z., Xie, Z., Guo, Y., Cong, P., Gao, Z., Yu, S., Xu, H., Wu, Z., Ren, Z., Zhou, Y., Zhang, B., Huang, Z., Zhu, Q., Wang, L., Lin, T., Yu, H., Hu, J., Yang, D., Yang, S., Lu, S., Chen, S., Qiu, J., Sha, Z., Zhong, Y., Wu, Y., Wang, S., Liu, W., Xu, B., Chen, L., Du, Q., Huang, Y., Ma, S., Wang, Y., Chen, M., Xiong, T., Yan, Y. C., Luo, H., Liang, H., Zhang, X., Zeng, W., Xu, R., Wang, P., Zhu, J., Zhang, R., Yang, W., Tang, Q., Yu, J., Ye, T., Pan, R., Ding, H., Liu, X., Luo, L., Shao, Z., Wu, Y., Lu, J., Liu, W., Zhang, H., Hu, J., Ye, Y., Lin, C., Zhang, Z., Tong, J., Wu, H., Li, Z., Wang, Y., Wang, L., Bai, Y., Fu, L., Xu, R., Wang, Y. Z., Li, Z., Wei, M., Shen, H., Zhang, C., Jin, C., Zhang, Z., Yang, R. H., Xu, X., Zhou, J., Zhu, R., Guo, Y., Pan, Z., Nie, S., Li, E., Lin, S., Liu, Z., Chen, A., Lyu, Z., Cao, S., Yu, R., Wang, C., Guo, J., Song, J., Chen, K., Ye, M., Li, J., Wu, D., Ma, H., Wang, Y., Yang, H., Cai, Y., Liu, S., Wang, Y., Sun, J., Xu, S., Bi, X., He, Y., Zhang, Y., Zhang, M., Zhang, L., Huang, P., and Liang, W. DeepSeek Elastic Compute (DSec): A Sandbox Infrastructure for Effective Agentic Training at Scale. arXiv:2609.22978, 2026. Jimenez, C. E., Yang, J., Wettig, A., Yao, S., Pei, K., Press, O., and Narasimhan, K. SWE-bench: Can Language Models Resolve Real-World GitHub Issues? arXiv:2310.06770, 2023. Kimi Team, Bai, Y., Bao, Y., Charles, Y., Chen, C., Chen, G., Chen, H., Chen, H., Chen, J., Chen, N., Chen, R., Chen, Y., Chen, Y., Chen, Y., Chen, Z., Cui, J., Ding, H., Dong, M., Du, A., Du, C., Du, D., Du, Y., Fan, Y., Feng, Y., Fu, K., Gao, B., Gao, C., Gao, H., Gao, P., Gao, T., Ge, Y., Geng, S., Gu, Q., Gu, X., Guan, L., Guo, H., Guo, J., Hao, X., He, T., He, W., He, W., He, Y., Hong, C., Hu, H., Hu, Y., Hu, Z., Huang, W., Huang, Z., Huang, Z., Jiang, T., Jiang, Z., Jin, X., Kang, Y., Lai, G., Li, C., Li, F., Li, H., Li, M., Li, W., Li, Y., Li, Y., Li, Y., Li, Z., Li, Z., Lin, H., Lin, X., Lin, Z., Liu, C., Liu, C., Liu, H., Liu, J., Liu, J., Liu, L., Liu, S., Liu, T. Y., Liu, T., Liu, W., Liu, Y., Liu, Y., Liu, Y., Liu, Y., Liu, Z., Lu, E., Lu,

H., Lu, L., Luo, Y., Ma, S., Ma, X., Ma, Y., Mao, S., Mei, J., Men, X., Miao, Y., Pan, S., Peng, Y., Qin, R., Qin, Z., Qu, B., Shang, Z., Shi, L., Shi, S., Song, F., Su, J., Su, Z., Sui, L., Sun, X., Sung, F., Tai, Y., Tang, H., Tao, J., Teng, Q., Tian, C., Wang, C., Wang, D., Wang, F., Wang, H., Wang, H., Wang, J., Wang, J., Wang, J., Wang, S., Wang, S., Wang, S., Wang, X., Wang, Y., Wang, Y., Wang, Y., Wang, Y., Wang, Y., Wang, Z., Wang, Z., Wang, Z., Wang, Z., Wei, C., Wei, Q., Wu, H., Wu, W., Wu, X., Wu, Y., Xiao, C., Xie, J., Xie, X., Xiong, W., Xu, B., Xu, J., Xu, L. H., Xu, L., Xu, S., Xu, W., Xu, X., Xu, Y., Xu, Z., Xu, J., Xu, J., Yan, J., Yan, Y., Yang, H., Yang, X., Yang, Y., Yang, Y., Yang, Z., Yang, Z., Yang, Z., Yao, H., Yao, X., Ye, W., Ye, Z., Yin, B., Yu, L., Yuan, E., Yuan, H., Yuan, M., Yuan, S., Zhan, H., Zhang, D., Zhang, H., Zhang, W., Zhang, X., Zhang, Y., Zhang, Y., Zhang, Y., Zhang, Y., Zhang, Y., Zhang, Y., Zhang, Y., Zhang, Y., Zhang, Z., Zhao, H., Zhao, Y., Zhao, Z., Zheng, H., Zheng, S., Zhong, L., Zhou, J., Zhou, X., Zhou, Z., Zhu, J., Zhu, Z., Zhuang, W., and Zu, X. Kimi K2: Open Agentic Intelligence. arXiv:2507.20534, 2025. Kimi Team, Bai, T., Bai, Y., Bao, Y., C., M., Cai, J., Cai, X., Cao, P., Cao, Y., Chai, Z., Charles, Y., Che, H. S., Chen, G., Chen, G., Chen, G., Chen, H., Chen, J., Chen, J., Chen, J., Chen, K., Chen, P., Chen, R., Chen, W., Chen, X., Chen, Y., Chen, Y., Chen, Y., Chen, Y., Chen, Y., Chen, Y., Chen, Y., Chen, Z., Cheng, D., Cheng, Y., Cui, J., Cui, J., Dai, A., Deng, J., Ding, H., Ding, R., Ding, S., Dong, M., Dong, M., Dong, Y., Dong, Y., Du, A., Du, C., Du, D., Du, J., Du, Y., Fan, Y., Feng, J., Feng, Q., Feng, Y., Fu, K., Fu, Q., Gao, F., Gao, H., Gao, J., Gao, T., Gao, W., Geng, S., Gong, J., Gong, L., Gong, S., Gong, X., Gu, Q., Gu, Y., Guan, S., Guo, H., Guo, S., Guo, X., Guo, Z., Hao, B., Hao, W., Hao, X., He, D., He, H., He, L., He, Q., He, W., He, X., He, X., He, Y., He, Y., Hong, C., Hong, T., Hu, H., Hu, J., Hu, R., Hu, W., Hu, Y., Hu, Z., Hua, L., Huang, J., Huang, K., Huang, R., Huang, S., Huang, W., Huang, Y., Huang, Z., Huang, Z., Hui, Y., Jia, C., Jiang, Y., Jiang, Z., Jiang, Z., Jin, W., Jin, X., Jing, Y., Kong, H., Lai, G., Li, A., Li, C., Li, C., Li, C., Li, F., Li, G., Li, H., Li, J., Li, J., Li, L., Li, L., Li, L., Li, W., Li, W., Li, X., Li, Y., Li, Y., Li, Y., Li, Y., Li, Z., Li, Z., Li, Z., Li, Z., Li, Z., Lin, J., Lin, X., Lin, Y., Lin, Z., Lin, Z., Liu, B., Liu, B., Liu, C., Liu, L., Liu, S., Liu, S., Liu, S., Liu, T., Liu, W., Liu, Y., Liu, Y., Liu, Y., Liu, Y., Liu, Z., Liu, Z., Lu, E., Lu, H., Lu, L., Lu, T., Lu, Z., Luo, A., Luo, G., Luo, J., Luo, Y., Lyu, B., Lyu, W., Mao, S., Mei, Y., Men, X., Ni, M., Niu, Y., Pan, S., Peng, S., Qi, Z., Qin, R., Qin, Z., Qin, Z., Qiu, H., Qiu, J., Qiu, J., Qu, B., Qu, Y., Shang, Z., Shao, Y., Shen, H., Shi, J., Shi, J., Shi, L., Shi, S., Siu, W., Song, P., Song, X., Su, J., Su, Y., Su, Z., Sui, L., Sun, J., Sun, J., Sun, S., Sun, S., Sun, T., Sun, Y., Tai, Y., Tang, C., Tang, H., Tang,

Sandman: Sandboxes for Agentic RL on Shared Kubernetes

S., Tang, Z., Tian, C., Tian, R., Tian, Y., Tu, W., Wang, C., Wang, C., Wang, C., Wang, D., Wang, F., Wang, H., Wang, H., Wang, H., Wang, H., Wang, H., Wang, H., Wang, J., Wang, J., Wang, J., Wang, J., Wang, L., Wang, S., Wang, S., Wang, S., Wang, S., Wang, S., Wang, T., Wang, W., Wang, X., Wang, X., Wang, X., Wang, X., Wang, Y., Wang, Y., Wang, Y., Wang, Y., Wang, Y., Wang, Y., Wang, Y., Wang, Y., Wang, Z., Wang, Z., Wang, Z., Wang, Z., Wang, Z., Wang, Z., Wei, C., Wei, M., Wei, S., Wen, Z., Wu, F., Wu, H., Wu, R., Wu, W., Wu, X., Wu, Y., Wu, Y., Wu, Y., Wu, Z., Xian, X., Xiang, C., Xiang, Y., Xiao, B., Xiao, C., Xiao, X., Xie, J., Xie, X., Xie, Y., Xie, Z., Xing, B., Xiong, Y., Xu, B., Xu, B., Xu, J., Xu, J., Xu, J., Xu, J., Xu, L. H., Xu, Q., Xu, S., Xu, S., Xu, T., Xu, T., Xu, W., Xu, X., Xu, Y., Xu, Y., Xu, Y., Xu, Z., Xue, H., Yan, J., Yan, Y., Yang, F., Yang, G., Yang, H., Yang, J., Yang, R., Yang, W., Yang, X., Yang, X., Yang, Y., Yang, Y., Yang, Y., Yang, Y., Yang, Z., Yang, Z., Yang, Z., Yang, Z., Yao, H., Ye, D., Ye, H., Ye, W., Ye, Z., Yin, B., Yin, H., Yin, X., Yu, C., Yu, H., Yu, L., Yu, S., Yu, S., Yu, T., Yuan, E., Yuan, M., Yue, T., Yue, W., Yue, Y., Zha, D., Zhan, H., Zhang, B. H., Zhang, D., Zhang, F., Zhang, H., Zhang, H., Zhang, H., Zhang, J., Zhang, J., Zhang, J., Zhang, K., Zhang, M., Zhang, P., Zhang, Q., Zhang, R., Zhang, R., Zhang, S., Zhang, S., Zhang, X., Zhang, X., Zhang, Y., Zhang, Y., Zhang, Y., Zhang, Y., Zhang, Y., Zhang, Y., Zhang, Y., Zhang, Y., Zhang, Y., Zhang, Y., Zhang, Z., Zhang, Z., Zhao, B., Zhao, C., Zhao, F., Zhao, J., Zhao, J., Zhao, S., Zhao, W., Zhao, X., Zhao, X., Zhao, Y., Zhao, Z., Zheng, H., Zheng, H., Zheng, R., Zheng, S., Zheng, T., Zhong, H., Zhong, L., Zhong, L., Zhou, M., Zhou, Q., Zhou, R., Zhou, R., Zhou, X., Zhou, Y., Zhou, Z., Zhu, J., Zhu, L., Zhu, X., Zhu, Y., Zhu, Y., Zhu, Z., Zhuang, C., Zhuang, W., and Zu, X. Kimi K3: Open Frontier Intelligence. arXiv:2607.24653, 2026. Konishi, F., Tsubouchi, Y., and Tsuruta, H. SAKURAONE: An open Ethernet-based AI HPC system and its observed workload dynamics in a single-tenant LLM development environment. In Proceedings of Machine Learning and Systems (MLSys), volume 8, pp. 1218–1232, 2026. URL https://proceedings.mlsys.org/paper_ files/paper/2026/file/40b8fb4f900044 05e14b1ede6ab42373-Paper-Conference. pdf.

Govindarajen, P. GUARD: Scalable straggler detection and node health management for large-scale training. In Proceedings of Machine Learning and Systems (MLSys), volume 8, pp. 1595–1607, 2026. URL https://pr oceedings.mlsys.org/paper_files/pape r/2026/file/339caf45a6fa281cae8adc64 65343464-Paper-Conference.pdf. Lu, H., Liu, Z., Xiong, S., He, Y., Gao, W., Wu, Y., Wang, W., Liu, J., Li, Y., Zhao, H., Huang, J., Yang, S., Li, X., Luo, Y., Liu, Z., Pan, L., Yan, J., Wang, W., Su, W., Wang, J., Qu, L., and Zheng, B. Part II: ROLL Flash – Accelerating RLVR and Agentic Training with Asynchrony. arXiv:2510.11345, 2025. Modal Labs. Modal sandboxes. https://modal.com/ docs/guide/sandbox, accessed 2026-10-23, 2026. Moritz, P., Nishihara, R., Wang, S., Tumanov, A., Liaw, R., Liang, E., Elibol, M., Yang, Z., Paul, W., Jordan, M. I., and Stoica, I. Ray: A Distributed Framework for Emerging AI Applications. arXiv:1712.05889, 2017. Pan, J., Wang, X., Neubig, G., Jaitly, N., Ji, H., Suhr, A., and Zhang, Y. Training Software Engineering Agents and Verifiers with SWE-Gym. arXiv:2412.21139, 2024. Quirk, K. J., Lennie, M., Ardestani, E. K., Ahuja, S. S., Bergeron, M. R., Grier, A., Wang, Z., Ozdal, M., Zhang, X., Triguna, A., Zhang, Y., Oldham, M., and Tang, C. Sparing strategies to minimize reliability impact on large training jobs. In Proceedings of Machine Learning and Systems (MLSys), volume 8, pp. 860–873, 2026. URL https://proceedings.mlsys.org/paper_ files/paper/2026/file/437bc4ccafd3fc 6d4289bd10940be42b-Paper-Conference. pdf. Ren, Y., Wang, X., Liu, X., Lv, B., Zhang, H., Zhang, S., Lai, H., Wang, S., Chen, L., Li, D., and Tang, J. TideRL: Boosting Agentic RL Goodput with Readiness-Aware Scheduling. arXiv:2608.10402, 2026. Segato, G. Quantifying infrastructure noise in agentic coding evals. Anthropic Engineering, https://www.an thropic.com/engineering/infrastructu re-noise, 2026-02-05, 2026.

Kubernetes SIG Apps. agent-sandbox: Kubernetes SIG apps project for isolated, stateful, singleton workloads. https://github.com/kubernetes-sigs/ agent-sandbox, issues #836 and #1335, accessed 2026-10-23, 2026.

Shao, Z., Wang, P., Zhu, Q., Xu, R., Song, J., Bi, X., Zhang, H., Zhang, M., Li, Y. K., Wu, Y., and Guo, D. DeepSeekMath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300, 2024.

Liu, G., Patni, A., Lin, C., Zeng, Z., Wittmayer, J., Wu, J., Nihalani, A., Huang, B., Liu, Y., Na, R., Ko, A., Zhipa, A., Cheng, C., Sun, M., Rajakumar, V., Joseph, R. G., and

Tan, Y., Liu, H., and Chen, H. Scaling agentic-RL sandboxes to the millions with gVisor at Tencent. gVisor blog, http s://gvisor.dev/blog/2026/04/23/scali

Sandman: Sandboxes for Agentic RL on Shared Kubernetes

ng-agentic-rl-sandboxes-to-the-milli ons-with-gvisor-at-tencent/, 2026-04-23, 2026.

L. SWE-ABS: Adversarial benchmark strengthening exposes inflated success rates on test-based benchmark. arXiv:2603.00520, 2026.

The gVisor Authors. gVisor application compatibility. ht tps://gvisor.dev/docs/user_guide/com patibility/, revision d3ae922, accessed 2026-10-26, 2025.

Yu, Q., Zhang, Z., Zhu, R., Yuan, Y., Zuo, X., Yue, Y., Dai, W., Fan, T., Liu, G., Liu, L., Liu, X., Lin, H., Lin, Z., Ma, B., Sheng, G., Tong, Y., Zhang, C., Zhang, M., Zhang, W., Zhu, H., Zhu, J., Chen, J., Chen, J., Wang, C., Yu, H., Song, Y., Wei, X., Zhou, H., Liu, J., Ma, W.-Y., Zhang, Y.-Q., Yan, L., Qiao, M., Wu, Y., and Wang, M. DAPO: An Open-Source LLM Reinforcement Learning System at Scale. arXiv:2503.14476, 2025.

The gVisor Authors. gVisor: The container security platform. https://gvisor.dev, accessed 2026-10-23, 2026. The Kata Containers Authors. Kata Containers architecture. https://github.com/kata-containers/ kata-containers/blob/main/docs/desig n/architecture/README.md, revision 5c02698; platform support in README.md revision e06e13b; accessed 2026-10-26, 2026. The Kubernetes Authors. Service: environment variables. https://kubernetes.io/docs/concepts/ services- networking/service/, accessed 2026-10-27, 2026a. The Kubernetes Authors. Kubelet configuration (v1beta1). https://kubernetes.io/docs/reference /config-api/kubelet-config.v1beta1/, accessed 2026-10-27, 2026b. Wang, X., Rosenberg, S., Michelini, J., Smith, C., Tran, H., Nyst, E., Malhotra, R., Zhou, X., Chen, V., Brennan, R., and Neubig, G. The OpenHands software agent SDK: A composable and extensible foundation for production agents. In Proceedings of Machine Learning and Systems (MLSys), volume 8, pp. 1843–1855, 2026. URL https: //proceedings.mlsys.org/paper_files/ paper/2026/file/8ae9cf363ea625161f88 5b798c1f1f78-Paper-Conference.pdf.

AI U SAGE D ISCLOSURE We used generative AI tools extensively in this work, under the authors’ direction and review. Tools. Claude Code (Anthropic), with Claude models, used as a coding agent and writing assistant; and Codex (OpenAI), used to outline the paper, draft execution plans and review drafts. Design and experiments. The authors chose the platform’s design goals and made every design and operational decision. The assistant drafted and critiqued experiment and analysis designs, including the cancellation-window tests against a stand-in Kubernetes API and the record-only analysis of image preparation, which the authors approved. Literature search. The assistant searched for and summarized related work. Every cited work was checked to exist, and the authors checked that each is represented correctly. Code. Much of the platform and tooling (the in-sandbox daemon, the pod provider and warm pool, the trainer integration, and the measurement, reproduction, analysis and figure scripts) was written with the assistant to the authors’ specifications, with tests; the authors reviewed and ran it.

Wongpanich, A., Oguntebi, T., Paredes, J. B., Wang, Y. E., Phothilimthana, P. M., Mitra, R., Zhou, Z., Kumar, N., and Reddi, V. J. Machine learning fleet efficiency: Improving TPU systems at scale with ML productivity goodput. In Proceedings of Machine Learning and Systems (MLSys), volume 8, pp. 1291–1303, 2026. URL https: //proceedings.mlsys.org/paper_files/ paper/2026/file/fbe2b2f74a2ece8070d8 fb073717bda6-Paper-Conference.pdf.

Running the experiments. The assistant launched and monitored cluster jobs and collected read-only cluster measurements under the authors’ oversight; the authors approved every operational change.

Yang, J., Lieret, K., Jimenez, C. E., Wettig, A., Khandpur, K., Zhang, Y., Hui, B., Press, O., Schmidt, L., and Yang, D. SWE-smith: Scaling Data for Software Engineering Agents. arXiv:2504.21798, 2025.

Writing. The assistant drafted most of the text, tables and figures; every number is traced to a source file in the authors’ evidence ledger. The authors directed, reviewed and edited the paper and checked its claims against the sources.

Yu, B., Cao, Y., Zhang, Y., Lin, L., Xu, J., Zhong, Z., Xu, Q., Wang, G., Cao, J., Cheung, S.-C., He, P., and Briand,

The authors take full responsibility for all content of this submission.

Data. No data were generated by AI tools, apart from the trajectories of the agent models trained in the experiments themselves, the simulated API responses of the stand-in Kubernetes API used in the cancellation tests, and the planted residue of the reuse tests, which are described as such.

Sandman: Sandboxes for Agentic RL on Shared Kubernetes

Appendix This supplement gives the details behind the paper’s measurements, for completeness and reproducibility; the paper can be read without it. Section A defines the evidence, workloads, code versions and endpoints, Section B gives the workload distributions, Sections C–K follow the paper’s order, Section L pairs each failure path with its fix, and Section M lists every operations case we recorded. The sign § refers to sections of the paper. Dates are in 2026 and times are UTC. Abbreviations: EKS, Amazon Elastic Kubernetes Service; GRPO, group relative policy optimization; RL, reinforcement learning; SWE, software engineering; VPC CNI, the Amazon Virtual Private Cloud Container Network Interface plugin.

A

E VIDENCE , WORKLOADS , CODE VERSIONS AND ENDPOINTS

Evidence kinds. Production campaigns are our training campaigns A–D (3–17 October) and other jobs on Sandman, analysed after the fact from logs, per-trajectory records, run summaries, the API server’s audit log and cluster metrics. Controlled experiments are dedicated runs on the same shared cluster, next to other tenants’ load, that change one part of the system; A/B arms ran at the same time. Replays feed recorded traces through the real pool code in virtual time, holding the trajectory timeline fixed. Reproductions use a few pods or a local copy of the code; they show that a failure path exists and that a fix closes it, not how often it occurs. Attempts (agent-session starts, retries included), trajectories, pods, gradings, labels and learner steps are counted separately. Workloads are listed in Table A1. The RL runs studied here, including campaigns A–D, trained Qwen3.5-35B-A3B with GRPO in groups of 8 samples per task; the A/Bs used campaign C’s recipe on 16 GPUs, and the stress test and the evaluation were other team members’ jobs. Tasks come from SWE-bench Verified (Chowdhury et al., 2024) and SWE-Gym (Pan et al., 2024), one image per task. Unless stated otherwise, sandboxes requested 2 vCPU and 8 GiB as requests equal to limits (16 GiB for a few heavy repositories), each test-suite run got a fresh grading pod with an 1,800 s test limit, and the campaigns’ create timeout, which includes the image pull, was 300 s. Trajectories had a 16,384-token budget after the prompt (32,768 in two SWE-Gym runs). Outbound traffic: campaigns A and B ran no outbound guard on any pod (rollout, grading or test re-execution); campaigns C and D and our RL experiments denied outbound traffic on all of these pods; the scale ramp, its follow-up and the placement contrast ran no guard, and the grading-condition study denied outbound traffic only in its egress cells. Counting. Rollout-sandbox counts for campaigns A–C stop at each run’s last learner step, as in Table A2; the whole logs hold 0.010– 0.013% more (79,869, 40,141 and 396,861), and Section B uses them. The four campaigns ran 122 training runs and used about 21,407 B200-hours (1,773, 966, 5,902 and about 12,766). Cost. At on-demand list prices ($160.645 per hour for an 8-GPU B200 instance, $4.752 per hour for a 96-vCPU, 192 GiB c5d node), the campaigns’ 3.30–3.75 sandbox-hours per B200 GPU-hour cost about 1.7% of the GPU cost if each 2 vCPU, 8 GiB sandbox is charged its node share of CPU (2 of 96 vCPU) and about 3.5% if of memory (8 of 192 GiB); test re-execution and evaluation jobs add 2–7% of that, giving 1.8–3.7%. Seccomp. No pod or container specification of any provider version sets a seccomp profile (a 15 October capture: unset on 6,521 of 6,522 sandbox pods, the one exception not ours), and the CPU nodes’ kubelet configuration is the managed default, which sets no seccomp default, so the kubelet’s default, unconfined, applies. On 27 October two probe pods on one current CPU node read their own status: without a profile, seccomp mode 0 (no filter); with the runtime’s default profile, mode 2 (filtered). Whether that profile changes any verdict is not measured: a pre-registered check stopped before its first grading because the cluster’s admission control began refusing 499 of the 500 Verified task images. Code versions are given as dates (Table A3): the paper describes each mechanism in its current form, and the table shows which version each measurement ran. Intervals are two-sided 95% t-intervals over run pairs (n = 3 for the A/Bs, n = 6 for the pre-registered comparison), Wilson intervals for single proportions, and percentile bootstrap intervals (runs or tasks resampled) where a table says so. What each workload ran. Table A4 records which mechanisms each workload ran, from job specifications, the code each experiment loaded and a live probe of campaign sandboxes, so each result is tied to the code that produced it. The warm pool and checked reuse are options; every other mechanism is on by default. Endpoints. A create runs from the provider’s API request until execd answers its health route, so it includes scheduling, address assignment, image pull, container start and the readiness probe. Create-to-Ready runs from the pod’s creation to its Ready condition (API timestamps, 1 s resolution), submit to healthy is what a caller of the provider’s create waits without our pacing queue, and request to first command runs from the acquire call to the first successful command. An acquisition runs from the trainer’s request until it holds a usable sandbox (for a warm pod, a health check and a label change), and the waiting share divides the summed acquisition time of rollout and grading sandboxes (each create or pool wait counted once) by summed trajectory time. A sandbox is live from its completed create until its delete, in use or warm. Availability is the share of attempts, or of trajectories after at most one retry, that got a working sandbox; sandbox-hours per GPU-hour divide summed trajectory time by B200-hours, since the campaigns’ trainer held one sandbox per trajectory at a time. The learner step interval is the mean time between learner steps, and the held-out resolve rate the mean over held-out tasks of passes per valid sample.

Sandman: Sandboxes for Agentic RL on Shared Kubernetes

Table A1. Workloads behind the paper’s numbers. Sandboxes: rollout and grading sandboxes created; GPUs are B200s. See: where the workload is used (§: the paper). Workload

When

Runs and scale

Production Campaign A

Tasks

3–4 Oct, 30.1 h

Campaign B

5 Oct, 10.6 h

32 runs of 50 learner steps, 16 GPUs each; 64 trajectories in flight per run 16 runs of 50 steps, 16 GPUs; 64 in flight

Campaign C

10–13 Oct, 57.3 h

Campaign D

13–17 Oct, 82.7 h

Extension of D Other jobs on Sandman

17–20 Oct 24 Jul–18 Oct

Stress test; evaluation

14 Oct

Sandboxes and notes

24 Verified, 27 held out 79,861 rollout, 113,407 grading; no outbound guard on any pod Verified 40,136 and 59,164; no outbound guard on any pod 35 runs of 50 steps, 64 GPUs; 256 in flight 88 Verified 396,810 and 568,930; outbound traffic denied on all pods 39 runs of 300 steps, 16 GPUs; 64 in flight; up to 88 Verified 720,332 and 1,147,595; outbound 27 runs (432 GPUs) at once (sphinx-8475 removed traffic denied on all pods at start) 48 runs of 16 GPUs, in two attempts as D incident list only 299 sandbox-using jobs of about 10 other owners, – aggregates only up to 896 GPUs each 1,536 concurrent trajectories (1,728 actors); an – other team members’ jobs; older evaluation of 4,096 concurrent (6,144 per rollout; provider code, unpaced up to 5,120 actors)

Controlled experiments Scale ramp

18 Oct, 08:09–11:47 31 steps, 30 of them measured, from 99 to 12,000 one 43 MB image requested sandboxes; creates paced at 20–25/s, except a rate sweep at 5–200/s Placement follow-up 20 Oct, 08:36–12:49 8 steps of 500–5,000 pods, unspread, then spread 43 MB image; one over nodes Verified image 27 Oct, 08:38–08:50 two batches of 1,000 pods at 10 creates/s, 180 s one 43 MB image Placement contrast apart: soft topology spread, then the default scheduler; stopped by a pre-registered co-tenant rule after 2 of 12 planned cells Warm-pool A/B 18 Oct (pair 1), 22 3 concurrent pairs of 25-step runs, 16 GPUs each; 88 Verified Oct 128 in flight; 14,513 trajectories Checked-reuse A/B 18 Oct (pair 1), 22 3 concurrent pairs of 40-step runs, 16 GPUs each; 86 Verified Oct 128 in flight; 33,656 trajectories Grading-condition study 16–18 Oct 69,319 gradings (52,512 on Verified, 16,807 on 500 Verified; 2,401 SWE-Gym), one pod each, fresh except in one SWE-Gym recycled-pod run; 120–240 pods at once Label re-grades 18–20 Oct every patch of pair 1 of both A/Bs (11,399 labels) 88 Verified; 2 × 230 and of two SWE-Gym GRPO runs (60 and 20 SWE-Gym steps, 16 GPUs) Three-arm reuse test 19 Oct 787 gradings per arm: fresh pods, plain reuse, 86 Verified checked reuse Frozen deployment 21 Oct, 08:50–15:20 reference/empty-patch check on 3 pools; 3 230 and 280 GRPO runs of 40 steps, 16 GPUs; 48 in flight SWE-Gym; 115 Verified Vacuous-test re-grade 22 Oct every consumed positive label on two tasks in 2 Verified campaigns C and D (7,294) Pre-registered comparison 22 Oct 07:41 to 23 6 pairs of 150-step runs, 16 GPUs each, and a 89 or 86 Verified; 112 Oct 09:56 20-step pilot pair; 64 in flight held out Validation run 23 Oct one 20-step run, 16 GPUs; 64 in flight 21 Verified

See B, §5.1 B, §5.1 B, H, §5.1 B, H, §5.1 M B §4.4, §6

38,377 pods of 0.1 vCPU, 128 MiB

C

1,000 of the pods at 2 vCPU, 8 GiB

C

2,000 pods of 0.1 vCPU, 128 MiB

C, §5.2

grading pods from the pool or on demand warm pool with or without checked reuse 16 run-condition cells

E

H

one fresh pod per grading

H

planted residue

I

checked reuse of grading pods

E, H, I

7,861 pods, 98.0 pod-hours

H

E, I

117,875 trajectories; 117,146 rollout K and 120,326 grading sandboxes 891 grading pods I, L

Replays, analyses of records, and reproductions Replays 18–23 Oct pool sizing (3 traces), grading-pod source (8 – real pool code in virtual time traces), CPU reservations (58,693 rollouts), pod retirement (69,812 gradings) 3,734 learner steps of campaign A, both A/Bs and – existing records only; no new runs Critical-path analysis 27 Oct (records of the pre-registered comparison, each wait traced to 3–23 Oct) the trajectory that ended it; per-run totals for B–D Reproductions Jul–Oct; 27 Oct one development host (8 vCPU, 30 GiB), a one Verified task image – stand-in Kubernetes API, a few cluster pods

E, I, §5.3

J, §5.4

F, G

Table A2. Production campaigns: pods created, not gradings (a trajectory can use two grading pods); lifetimes for C and D from sampled traces. Table A5 gives availability, create latency and waiting share. Tasks A B C D

Runs × GPUs In flight per run Rollout pods Grading pods Creates/s (mean) Rollout life p50 / p99 Outbound guard

24 32 × 16 24 16 × 16 88 35 × 64 88 39 × 16 (≤27 at once)

64 64 256 64

79,861 40,136 396,810 720,332

113,407 59,164 568,930 1,147,595

1.8 2.6 4.7 6.3

196 / 443 s – 136 / 457 s 207 / 424 s

none none all pods all pods

Sandman: Sandboxes for Agentic RL on Shared Kubernetes

Table A3. Code changes that matter for the measurements, and the code each workload ran. Provider and pool are Sandman’s; the trainer and agent harness are our RL trainer’s. Component

Changes, with dates

execd

first version, reads of at most 2,000 lines (21 Jul); syntax check of the wrapped command (4 Aug); own process group (12 Aug); byte-exact whole-file reads (19 Aug); group kill by original id, output capped while draining, complete-bytes reads (6 Oct); strict exit-status frames (7 Oct); per-sandbox token (20 Oct); state-check routes (19–21 Oct) owner references (21 Jul); central drain (29 Jul); tags pulled “always” (10 Sep); outbound guard (campaigns’ patched copy 6 Oct; on every sandbox, late Oct); per-sandbox token and inbound guard on every sandbox (late Oct); soft topology spread and paced creates by default (late Oct); eight-field probe before every grading and retirement of contradicted pods (late Oct); Service variables off (campaigns’ patched copy 10 Oct 20:06, from campaign C on; default branch 13 Oct); creates shielded from cancellation (14 Oct); pacing (15 Oct); warm budget, recycling, Services paced with pods (20 Oct); readiness budget per job (21 Oct); error-path fix: create nonce, read-back after an ambiguous error, owned retries of failed deletes (27 Oct) our trainer: fresh grading pod, rollout sandbox kept until grading completes (28 Jul); test-configuration files left out of patches (29 Jul); agent calls bound to pod addresses (13 Oct; capture and grading always used pod addresses); reference/empty-patch check of the task set before each job (late Oct); gradings without a verdict retried, then resampled, never scored 0 (27 Oct). The campaigns’ trainer (every campaign and RL experiment): rollout sandbox released before grading (9 Sep); gradings without a verdict invalid and resampled, never scored 0. Harness: reads checked by length and SHA-256 (6 Oct; on by default, late Oct); editor refuses files it cannot write back unchanged (27 Oct) task images kept 90 days (13 Oct); reboot-safe NVMe image store (18 Oct); metadata responses limited to one hop on the CPU nodes (late Oct); eight warm pod addresses per CPU node and a network policy on sandbox pods (late Oct)

provider and pool

trainer and harness

cluster What each workload ran Campaigns A–B Campaigns C–D Scale ramp, follow-up, placement contrast Warm-pool A/B Checked-reuse A/B Frozen deployment, pre-registered comparison

Validation run

Cancellation tests Reproductions

the campaigns’ trainer; the provider shipped in the runtime image (15 Sep), from before the 14 Oct fix, with no outbound guard and Service variables on; execd of 19 Aug, copied into each pod from a carrier image; no warm pool as A–B, but a patched copy of that provider that adds the outbound guard (6 Oct) and turns Service variables off (10 Oct); execd of 19 Aug from the same carrier image provider and pool of 14 Oct plus a benchmark patch (name prefixes, optional affinity, later a spread constraint) whose defaults reproduce the unpatched code; execd of 7 Oct; creates paced by a wrapper in the driver campaign C’s trainer; the patched provider of campaigns C–D with the provider half of the 14 Oct fix; a pool that hands out a warm pod only if its resources match the request (18 Oct); execd of 19 Aug from the carrier image as the warm-pool A/B, with logging added to the trainer, plus the pool’s recycling path, warm budget, pacing and the state check from a branch of 18 Oct; execd of 19 Aug plus the state-check routes provider, pool, reuse guard and reference/empty-patch check as fetched on 21 Oct, with a soft topology spread over the job’s pods; execd of 20 Oct in rollout and grading pods (test re-execution pods, where used, kept the 19 Aug build); the execd token on pooled grading pods only; the full cancellation fix on pooled grading pods and the provider half on the rest; agent calls through a local forwarder to the pod address; the checked-reuse A/B’s trainer grading pods: one build of every grading-path mechanism as of 23 Oct, with the same spread; rollout pods: the pre-registered comparison’s provider and execd of 20 Oct; test re-execution pods: the 19 Aug build; the pre-registered comparison’s trainer with one actor change, agent calls through the same forwarder provider and pool of 13 Oct (before the fix), 21 Oct (cancellation fix) and 27 Oct (error-path fix), and an alternative implementation execd of 21 Jul, 4 Aug, 19 Aug, 6 Oct, 7 Oct and 21 Oct; harness tools of 30 Jul, 6 Oct and 13 Oct

Table A4. What ran where. A–B, C–D: production campaigns; WP, RU: the warm-pool and reuse A/Bs; PR: pre-registered package arm; VR: validation run; Now: defaults of our code and SWE job templates at submission. ✓ ran; – did not; o option, off by default; g grading pods only; p in part (Table A3 gives the variant). Mechanism

A–B C–D WP RU PR VR Now

Agent calls by pod address, not a Service Service environment variables off Soft topology spread over nodes Paced creates Outbound guard on every sandbox Network policy on sandbox pods execd token and inbound guard Metadata responses limited to one hop Warm pool State-checked reuse In-pod configuration probe Reference/empty-patch check of the task set Create keeps an owner on cancel Strict exit-status parsing Test-configuration files left out of patches No verdict resampled, never scored 0

– – – – – – – – – – – – –§ – – ✓

§

– ✓ – – ✓ – – – – – – – –§ – – ✓

– ✓ – – ✓ – – – g – – – p – – ✓

– ✓ – g ✓ – – – g g – ✓ p – – ✓

p ✓ ✓ g ✓ – p – g g – ✓ p p – ✓

p ✓ ✓ g ✓ – g – g g g ✓ p p – ✓

Campaigns cancelled creates only at test re-run deadlines (not measured).

✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ o o ✓ ✓ ✓ ✓ ✓ ✓

Sandman: Sandboxes for Agentic RL on Shared Kubernetes

B

W ORKLOAD DISTRIBUTIONS

This section expands §2 and §5. Campaign A has complete start and end records; campaigns C and D have per-turn traces of sampled agent workers (2 and 3 runs); campaign B has totals only. “Pool/reuse A/Bs” are the runs of both A/Bs, and “SWE-Gym runs” the frozen deployment’s three runs. Figure A1 shows lifetimes and where a rollout sandbox’s time goes. Table A5. Production campaigns in operation. Attempts: agent-session starts, retries included; available: share of attempts, and of trajectories after one retry, that got a working sandbox (D: a pod); create: range of per-run p50 and p99; wait: waiting share (∗ sampled traces); sandbox-h: per B200 GPU-hour. Span

A B C D

Wait

Sandbox-h

(h)

Attempts

attempt

Available (%) traj.

p50

Create (s) p99

(%)

per GPU-h

30.1 10.6 57.3 82.7

81,102 42,409 453,155 1,036,028

99.73 96.98 91.16 69.5

99.98 99.65 ≥96.69 –

4.31–5.65 4.14–7.01 5.11–5.22 5.12–5.29

6.8–175.8 17.3–258.4 8.2–134.2 18.4–35.7

6.4 9.8 9.9 6.5∗

3.30 3.75 3.56 –

Table A6. Creates arrive at a few per second on average and in bursts at run starts. Per-minute windows; creates count rollout and grading pods. B–D: peaks are upper bounds (runs at once times trajectories in flight) and rates are totals over the span; A/Bs: peaks per run; pre-registered comparison: the training runs’ rollout, grading and warm pods. Ended early: aborted, timed out or failed rollouts. Workload

Peak live (typical)

Runs at once

Creates/s mean

Per minute p99 / max

In 1 s p99 / max

Destroys/s mean / p99

Ended early

Campaign A Campaign B Campaign C Campaign D Pool/reuse A/Bs Pre-registered comparison Scale ramp

448 (211) ≤512 ≤1,024 ≤1,728 139–319 712 (389) 11,858

7 8 4 27 2 10 –

1.84 2.61 4.68 6.27 1.80 2.93 20–25

3.78 / 5.02 – – – 3.39 / 5.40 5.77 / 6.55 –

8 / 132 – – – – 9 / 130 –

1.82 / 3.70 – – – 1.78 / 2.98 2.92 / 5.79 –

0.57% – – – 15.6% 0.05% –

Bursts and early ends. A run starts its 64 trajectories within one second, so two runs starting together created 128 rollout pods in one second (132 creates in all, campaign A’s one-second maximum). In campaign A, 458 of 80,068 ended trajectories ended early, 199 before a pod existed; in the A/Bs, 6,712 of 6,905 errors fell in three incident windows. In campaign D’s first 12 runs, which met the 14 October incidents, 18–83% of starts made no create because fast failures were retried; over the campaign, 1,036,028 starts made 720,332 creates. In D’s other 27 runs, 495,566 of 497,170 starts (99.7%) got a pod. Cluster-wide, the audit log showed 33,236 sandbox pod creates in one hour of 14 October, and one reading found 4,798 live sandbox pods. In campaigns A–C, 33 of the 576,666 attempts in Table A5 (32, 0 and 1) failed because the harness could not reach execd before the agent started. Table A7. Rollout sandboxes live 136–327 s at the median and a fresh grading pod 13.5 s; pooled pods live longer because they also wait warm. Seconds. n

p50

p90

p99

max

Basis

Rollout, campaign A Rollout, campaign D (sampled) Rollout, campaign C (sampled) Rollout, pool/reuse A/Bs Rollout, pre-registered comparison Rollout, SWE-Gym runs

79,869 9,723 4,054 49,043 117,146 3,169

196 207 136 168 229 327

317 348 247 360 358 566

443 424 457 550 436 782

– 805 740 1,885 1,533 1,187

start and end records per-turn traces per-turn traces start and end records start and end records per-turn traces

Grading, campaign A, per pod Grading, fresh pod (pre-registered, arm without the package) Grading, pooled pod (pre-registered, package arm) Grading, pooled pod (checked-reuse A/B, both arms) Grading, pooled pod (SWE-Gym runs) Grading-condition study, in-pod time (Verified, by run) Grading-condition study, in-pod time (SWE-Gym)

113,407 53,850 63,991 24,617 2,065 52,512 16,447

28.7 13.5 26.6 162 256 6.6–16.5 17.6

115 27.8 171 396 592 31–110 459

189 225 294 836 1,010 125–357 1,130

– 1,810 1,810 2,060 1,740 1,800 1,820

grading window / pods acquire and test run pool events, idle included pool events pool events in-pod timings in-pod timings

Population

Rollout and grading sandboxes. Campaigns A–D created 1.42, 1.47, 1.43 and 1.59 grading pods per rollout sandbox, retried suites and re-gradings included. In campaign A, 13.9% of ended trajectories used no grading pod (no patch), 30.6% one and 55.5% two, one fresh pod per test suite. A fresh grading pod in the pre-registered comparison took 5.5 s at the median to acquire and 6.5 s to run its tests. A turn’s generation took 1.7–3.0 s at the median, and a live sandbox served 8.8–12.4 tool calls per minute; 85–93% of calls were shell commands, and 28–44% of rollouts ran the tests. With outbound traffic denied, 46–94% of rollouts still tried a package install or download. In a replay against one-minute metrics, pods used 0.001–0.007 vCPU while the model generated and 0.068–0.276 vCPU while busy. The traces hold 136,441 rollouts that ran, of 172,684 attempts in 28 runs. Images. Campaign A requested each of its 24 images a median 3,412 times; the long re-request mode is the next group of the same task, 13–44 minutes later. Rollout pods reached Ready in 5.13–5.97 s at the median on Verified (p99 24.4–60.5 s) and 14.3 s on the SWE-Gym runs (p99 117 s). The 50 images a CPU node lists take a median of 231 GiB, more than its 89 GiB of allocatable ephemeral storage, so images live on the striped NVMe volume; campaign D’s task set fits on one node, the SWE-Gym runs’ 306 images do not. Resources. The sampled pods of all users used 1.9% of their memory request. Some tasks need more than the default: the 3 matplotlib tasks of campaign D’s 89 run with 16 GiB on every attempt, and 22 of 2,401 SWE-Gym tasks (0.92%, all modin) are killed for memory at

Sandman: Sandboxes for Agentic RL on Shared Kubernetes

(a)

Campaign A (production) Pre-registered (controlled)

Fraction of sandboxes

1 0.75 0.5

Grading

Rollout

sandboxes

sandboxes

0.25 0

2

10

30

100

300

1,000

Sandbox lifetime, create request to release (s, log scale) (b)

Create, ready, set up Run commands

Idle: model generating Capture patch, release

5%

Campaign D

91% idle 19%

Campaign C

74% idle

17%

Pool/reuse A/Bs

77% idle

9%

Pre-registered

87% idle 14%

SWE-Gym runs 0

78% idle

25 50 75 Share of rollout-sandbox lifetime (%)

100

Figure A1. Rollout sandboxes live for minutes and grading sandboxes for seconds; a rollout sandbox is idle while the model generates for 74–91% of its life. (a) Lifetime, create request to release. Solid: campaign A (production; grading lifetime estimated per pod from each trajectory’s grading window). Dashed: the pre-registered comparison (controlled; grading on fresh pods). (b) Share of rollout-sandbox lifetime by phase, from per-turn traces.

Table A8. Rollout sandboxes are mostly idle, and their few long commands are mostly installs and downloads that the outbound guard blocks. Time-weighted shares of sandbox seconds; capture and release take the remaining 0.08–0.30%, and the longest command was 300–301 s, execd’s limit. Campaign A records turns only.

Campaign D (sampled) Campaign C (sampled) Pool/reuse A/Bs Pre-registered comparison SWE-Gym runs Campaign A

Create, set up

Run commands

Idle

Tool calls per rollout p50 / p90 / p99

Command p50 / p99

Commands ≥10 s: share of calls / time

Installs among commands ≥10 s

3.62% 6.20% 5.48% 3.53% 7.91% –

4.85% 19.24% 16.97% 8.81% 13.95% –

91.38% 74.35% 77.25% 87.37% 78.06% –

37 / 61 / 86 31 / 46 / 56 30 / 45 / 57 35 / 51 / 67 50 / 76 / 97 30 / 45 / 58

12 ms / 7.8 s 19 ms / 15.4 s 28 ms / 15.7 s 9.5 ms / 8.3 s 9.5 ms / 12.8 s –

0.28% / 19% 1.54% / 29% 2.38% / 36% 0.72% / 23% 1.70% / 39% –

93% 97% 70% 98% 85% –

Table A9. Task images average 1.2–1.4 GB on Verified and 2.85–3.00 GB on SWE-Gym, and an image is usually requested again within seconds because a GRPO group starts together. Upper: compressed sizes. Lower: time between consecutive requests for the same image. Image set

Images

Total

Mean

p50

p90

Max

Campaign A tasks Campaign D pool SWE-Gym runs All of SWE-bench Verified All SWE-Gym tags

24 89 306 500 2,424

33.6 GB 107.6 GB 871 GB 699.8 GB 7,276 GB

1.40 GB 1.21 GB 2.85 GB 1.40 GB 3.00 GB

1.20 GB 1.15 GB 2.07 GB 1.24 GB 2.94 GB

2.03 GB 1.21 GB 6.19 GB 2.05 GB 6.19 GB

3.15 GB 3.17 GB 8.43 GB 3.33 GB 8.50 GB

Re-requests Campaign A Pool/reuse A/Bs Pre-registered comparison

in a run: <10 s / <60 s 79.8% / 87.3% 80.5% / 87.4% 81.4% / 88.1%

gaps ≥60 s: p10 / p50 / p90 216 / 777 / 1,415 s 88 / 923 / 1,965 s 1,118 / 2,656 / 4,182 s

any run: p50 / p90 / <5 min 2.2 s / 48 s / 96.7% 1.4 s / 149 s / 93.4% 3.0 s / 75 s / 94.0%

Sandman: Sandboxes for Agentic RL on Shared Kubernetes Table A10. Rollout pods use under 3% of their CPU request; grading pods keep about one core busy. Rollout: one-minute metrics of sampled pods; all users: 198 pods over 14 days, weighted by pod-minutes; grading: cgroup counters. Population

Pods Requests

CPU use / request

CPU p99 per pod-minute

Peak memory p50 / p99 / max

Rollout, campaign D Rollout, campaign C Rollout, pool/reuse A/Bs Rollout, pre-registered Rollout, SWE-Gym runs Sandbox pods, all users

1,089 813 2,967 1,061 780 198

0.43% 1.35% 1.05% 0.84% 2.80% 10.5%

0.05 cores 0.15 0.14 0.10 1.24 –

20 MiB / 0.08 / 0.76 GiB 31 MiB / 0.09 / 1.16 GiB 32 MiB / 0.12 / 1.36 GiB 27 MiB / 0.10 / 0.23 GiB 55 MiB / 2.22 / 6.69 GiB 47 MiB / 3.56 / 11.19 GiB

0.70–0.83 cores busy 0.89 cores busy

–

0.12–0.18 / 0.56–1.14 / 2.78 GiB

–

0.26 / 7.42 / 13.8 GiB

Grading, grading study (Verified) Grading, grading study (SWE-Gym)

2 vCPU, 8 GiB; 3.6% at 16 GiB as above; 3.0% at 16 GiB as above; 4.5% at 16 GiB as above; 4.0% at 16 GiB 2 vCPU, 8 GiB 2/8 for 137 pods; 23 at 0.5/0.75; 15 at 1.5/2; 23 other 52,336 1 vCPU in the cells, 2 in the reference configuration 16,447 as above

8 GiB and pass only at 16 GiB. At 2 vCPU and 8 GiB per sandbox, memory fills a c5d node first: 21 sandboxes fit by memory against 46 by CPU.

Sandman: Sandboxes for Agentic RL on Shared Kubernetes

C

S CALE RAMP AND ADDRESS - ASSIGNMENT FOLLOW- UP

Setup. The ramp ran the provider and pool of 14 October, which include the cancelled-create fix, with a benchmark patch for name prefixes and an optional affinity, in 31 steps, 30 of them measured. A driver pod per step fired N acquires at once (with the pool on, after prewarming N pods) and timed the first execd call, by pod address or through a per-agent Service. Pods requested 0.1 vCPU and 128 MiB and ran a cached 43 MB image with execd of 7 October, on the 260 m-family nodes, away from the c5d nodes that campaign D’s extension used. A wrapper paced creates, and deletes ran at twice the create rate. A watchdog on admission errors, DNS latency and co-tenant signals never fired. We stopped at 12,000 because free addresses in the pod subnets had fallen from 33.6k to 26.5k, and the tightest subnet is shared with the GPU nodes. Table A11. The median stayed at 5–7 s at every size; the tail followed address-assignment failures, which hit 19–25% of pods in every ramp step of a window of about 2.2 hours (08:40–10:54). Pod-IP routing, no warm pods, API timestamps (1 s). The last two steps are a control after 20 minutes idle and one right after it. Step (start) clean (08:18) ramp ramp ramp ramp ramp ramp ramp after 20 min idle (11:38) back-to-back (11:42)

N

Creates/s

Ready

Not Ready in 120 s

Address failures

Create-to-Ready (s) p50 / p90 / p99

API call p99 (ms)

99 250 500 1,000 2,000 4,000 8,000 12,000 500 500

20 20 20 20 20 25 25 25 20 20

99 249 496 997 1,973 3,947 7,935 11,826 500 500

0 1 (0.40%) 4 (0.80%) 3 (0.30%) 27 (1.35%) 53 (1.32%) 65 (0.81%) 91 (0.76%) 0 0

0% 21.2% 23.4% 21.5% 24.5% 18.9% 24.7% 22.4% 5.0% 0.4%

5/7/8 7 / 34 / 73 6 / 35 / 95 6 / 36 / 87 6 / 46 / 94 6 / 42 / 99 7 / 47 / 102 7 / 45 / 96 6 / 7 / 65 5.5 / 7 / 8

53 383 69 272 1,173 571 737 1,170 83 74

Where the time went. At the 12,000 step, 11,858 sandboxes were live at the peak (the 11,826 acquired plus 32 late creates kept warm); 83 acquires timed out behind the pacer against the harness’s 600 s budget, 48 of their creates were kept warm and the rest deleted, and no step left a pod. The idle and back-to-back control argues against our own just-released addresses as the cause. Scheduling took about 10 ms and the API create call 27–105 ms at the median; the tail sat between scheduling and the sandbox network being ready. A rate sweep at 99 pods inside the window had more address failures at faster bursts (5.1% at 5/s, 39.4% at 200/s), confounded with time of day. Table A12. A per-agent Service added one to two seconds before the first successful call and gave back most of the warm pool’s gain; unpaced Service creates were throttled. Medians; retried: first calls that needed more than one attempt. Routing and warm pods pod IP, no warm pods Service, no warm pods pod IP, warm pods Service, warm pods Service, warm pods, acquires spread out Service, no warm pods, Services unpaced Service, no warm pods, Services paced

N

Ready

Acquire (s)

First call (ms)

Retried

Service create p50 / p99 (ms)

Acquire to first success (s)

99 99 99 99 500 1,000 1,000

99 99 99 99 500 759 1,000

8.475 9.153 0.041 0.017 0.017 26.619 56.568

4 1,063 6 2,267 2,263 1,064 1,061

0 39 0 99 500 234 279

– 235 / 845 – 17 / 29 14 / 43 1,839 / 4,865 14 / 32

8.48 10.44 0.05 2.28 2.28 27.81 57.55

Services and the pool. The Service’s extra time is route programming, not name lookup (4–9 ms): a warm pod labelled at acquire always needed a retry of its first call, which landed at 2.27 s. Creating 1,000 Services at once drew 237 HTTP 429 answers from the API server’s fairness limits and 4 more on pod creates, and 241 of the 1,000 sandboxes failed; pacing Service creates with the pods’ token bucket gave 1,000 of 1,000, and the driver’s DNS p99 stayed at or below 15 ms up to 1,000 sandboxes. The warm pool did not make pods faster: it moved the pacing queue and the tail into the prewarm (13–218 s from 99 to 2,000 pods), and acquire p50 fell from 8.5–61.9 s to 0.04–0.51 s at 99.2–100% warm hits. Address-assignment follow-up. Two days later we reran the ramp with the same code, image and pod size, with N interleaved and other tenants’ address failures logged per minute. A co-tenant guard aborted the second 5,000-pod step, and the remaining unspread steps were dropped. The later steps spread the step’s own pods over all 260 nodes (maximum skew 2, or 3 for the realistic step, which ran 2 vCPU, 8 GiB pods on a pre-pulled Verified image). Before that step we pre-pulled its 1.21 GB image onto the 229 of the 260 m-family nodes whose 50-image status list did not show it; 200 already held it and 29 pulled it. Figure 3a also plots the routing and pool arms (Table A12), the rate sweep and the controls; its one step of 1,000 outside the window is the paced-Service arm (11:05, 3.0% address failures, p99 46 s). The tail sat in sandbox and network setup (p99 50–96 s unspread, 2 s spread). Container start to Ready took 4–5 s at 0.1 vCPU but 1 s at 2 vCPU, consistent with execd booting under a small CPU quota. During the aborted step, other tenants’ failures on the shared nodes peaked at 1,422 per minute and fell to 0–28 within a minute of our teardown; in the first ramp they averaged 813 per minute in minutes with at least 300 of our creates and 311 when we were idle. Limits: one day, steps in sequence, one repetition per spread setting, and at most 8 pods per node observed, so above about 2,000 sandboxes the spread needs more nodes or a larger per-node warm-address target. Back-to-back placement contrast (27 October). A pre-registered experiment then compared placement at a fixed create rate: same driver, code, 43 MB image (by digest), 0.1 vCPU and 128 MiB pods, and the 375 schedulable CPU-pool nodes. A batch of 1,000 pods with a soft topology spread (maximum skew 2 per node) ran 08:38:58–08:42:48, then, after a 180 s idle gap, a batch of 1,000 with the default scheduler ran 08:45:48–08:50:02, both paced at 10 creates per second by our pacer. Spreading used all 375 nodes with at most 3 of the batch’s pods each; the default scheduler used 72 nodes, up to 65 each (p50 11). No spread pod logged an address-assignment failure; 254 of the 1,000 default pods did (419 events), and all of them later became Ready, with create-to-Ready p50 22 s and p99 64 s against 6 and 8 s for the rest. Over all 1,000 pods, create-to-Ready p99 was 8 s spread and 57 s default. The spread batch pulled the image on 263

Sandman: Sandboxes for Agentic RL on Shared Kubernetes Table A13. Unspread, the affected share grew with N and concentrated on nodes that received many of a step’s pods; spread, the tail was gone at the same size and rate. The spread steps ran 1.5–3.5 h later, so this is an association. By node load: affected share on nodes that received <10, 10–19 and ≥20 of the step’s pods. Others: other tenants’ failures per minute on the shared nodes, before and during the step. N

Max pods per node

Creates/s

Address failures

By node load: <10 / 10–19 / ≥20

Not Ready in 120 s

Create-to-Ready (s) p50 / p90 / p99

Others, before → during

500 2,000 5,000 500 5,000 1,000 2,000 2,000

26 45 83 17 70 4 8 8

20 20 20 20 20 10 10 20

17.6% 24.4% 28.4% 8.8% 31.8%a 0.2% 0.35% 0.45%

0.0 / 18.9 / 54.8% 0.0 / 7.5 / 39.1% 0.0 / 9.8 / 31.3% 6.1 / 16.9 / –% 15.5 / 11.2 / 34.8% – – –

0 0 3 0 59 0 0 0

6 / 33 / 55 7 / 36 / 55 7 / 43 / 74 6 / 8 / 61 7 / 54 / 100b 2/2/3 5/7/8 5/7/7

– – – 79 → 138 172 → 643 409 → 7 44 → 34 18 → 24

Step (start) unspread (08:50) unspread (09:05) unspread (09:28) unspread (10:11) unspread, aborted (10:37) realistic, spread (11:49) spread (12:11) spread (12:34) a

1,591 of the 4,999 pods with API records; the pod watch counts 1,651 (33.0%). b From the pod watch, because the abort removed the step’s peak snapshot.

nodes that lacked it; the default batch found it on every node, which favours the default batch. The caller’s wait from submission, p50 / p90 / p99, was 56.2 / 96.5 / 105.8 s spread and 61.8 / 100.3 / 121.0 s default, most of it (p50 50 s in both) in our pacer’s queue before the create call. A pre-registered rule on other tenants’ failures per minute stopped the experiment after these two of twelve planned cells: per scheduled pod, other tenants’ failures were 0.23–0.37% when we were idle, 0.66% during the spread batch, 1.56% during the default batch and 1.65% after we stopped. With one pair, no rate contrast and no repetition, this is one observation; pacing itself was not varied. Table A14. Limits we met while scaling, what we changed and how we checked it: separate observations, not one curve. P: production observation; C: controlled experiment on the shared cluster. Stage, date

Limit we hit

Measure taken

How checked

Node pool, 14 Oct

Workers pinned to one instance type found its 115 nodes full of other tenants’ pods; 9 jobs never started. One replica (115 restarts) failed 84.9% of an evaluation’s 6,144 trajectories (4,096 concurrent). 3,255 Services in the namespace (10 Oct); campaign A’s median bash call 0.63 → 2.12 s; 241 of 1,000 unpaced Service creates failed. Address assignment failed for 0.64% of creates in one production hour, and for 19–25% of the ramp’s pods in a 2.2-hour window. Free pod addresses fell from 33.6k to 26.5k at 12,000 sandboxes.

Select the pool label, not a type.

P: 9 of 9 reran and finished.

3 replicas, 1 GiB; paced creates.

C: 0 admission errors in 38,377 paced creates, consistent with the fix, not proof. P: median bash call 16–22 ms in a production smoke test with links off. C: paced, 0 failed.

Admission webhook, 14 Oct Per-agent Services, 3–18 Oct Pod address assignment, 14–27 Oct Subnet headroom, 18 Oct

D

Service variables off; pod-address routing; paced creates. Spread creates over nodes.

C: 0.35–0.45% spread vs 24% unspread (20 Oct); 0 vs 25.4% back to back (27 Oct).

Stop at 12,000; watch free addresses.

C: counts before and after.

I MAGE PREPARATION FROM RECORDS

This analysis used only records that earlier controlled runs had saved: re-grading sandboxes of a SWE-Gym label re-grade (20 October; its two reference cells used 59 images of a 230-task set on 184 nodes), the same study’s GRPO run, the ramp’s small-image steps, the follow-up’s step with a Verified image present, and the warm-pool A/B. Each table row uses one source. No source controlled the node’s image cache, so each pod’s cache state comes from kubelet events. Units and endpoints were fixed before the numbers were read. A pod is on demand, image pulled when its first task-image container has “pulling” and “pulled” events, image present when the event says the image was already present, and prewarmed when it is a pool hit on a pod nobody had used. “Pulled” means the node lacked that image reference, though shared layers may have been present, and image size is the whole compressed image, not the bytes transferred. The small-image steps give request to first command; the re-grading sandboxes give the provider’s create time, without the first command. Table A15. A pulled SWE-Gym image made an on-demand create about four times slower at the median and set its tail. Re-grading sandboxes; phases at 1 s resolution; rest: create time minus scheduled to last container started. Unit On demand, image pulled On demand, image present

n

Sandbox and network

Pull p50 / p90 / p99

Container start

Rest p50

Create p50 / p90 / p99

422 230

1s 1s

13.4 / 69.4 / 85.1 s –

2s 2s

1.47 s 1.16 s

17.4 / 74.3 / 90.5 s 4.11 / 5.12 / 5.54 s

No pull waited in the kubelet’s queue (1,204 pulls), and the training run’s pods agree (782 pulled pods, pull p50 13.9 s). The pooled pull median (13.5 s) is below the median of per-image medians (33.4 s) because the most re-pulled images are small, so the image mix sets the pooled number. Pull rates reached 442 MB/s, which suggests shared layers, and 0.58% of pulls exceeded the provider’s default 120 s readiness budget. Small images and warm pods. With the 43 MB image, request to first command took 9.11 s at the median when the node pulled the image, 8.30 s when it had it, and 51 ms for a never-used warm pod (33, 66 and 99 pods). The pull itself took 0.17 s, and within a step the two on-demand units had the same submit-to-healthy time (5.15 against 5.11 s), so their difference in request time is our pacer’s queue. With the 1.21 GB Verified image present, 998 pods at 2 vCPU went from submit to healthy in 2.08 s at the median. Prewarming does the same create earlier and pays in idle reservation: warm pods of the warm-pool A/B idled 121–256 s at the median before first use.

Sandman: Sandboxes for Agentic RL on Shared Kubernetes Table A16. The cold penalty appeared within each image, each node and each half hour (blocked separately), and grew with image size. Left: pulled minus present median create, across blocks with both units. Right: pull time by size tercile and by instance type (an observation: image mix and timing differ by type). Blocked by same image same node same 30-min block

Blocks

p50 / p90

Range

39 101 6

18.2 / 71.2 s 14.1 / 51.6 s 12.4 / 15.5 s

6.0–140.6 s 4.7–81.3 s 10.1–16.7 s

Size (GB) or type

n

Pull p50 / p90 / max

1.29–1.42 1.42–3.11 3.11–8.41 m8id.24xlarge m6id.24xlarge m5d.24xlarge c5d.24xlarge

398 393 413 389 342 453 20

8.2 / 13.0 / 28.9 s 14.8 / 46.2 / 70.9 s 64.1 / 94.2 / 153.3 s 10.3 / 54.5 s 11.8 / 64.1 s 14.2 / 82.9 s 39.3 / 74.0 s

Limits: 359 re-grading pods had no grading record and longer pulls, so the cold tail is slightly optimistic, and the records cannot separate download from unpack, give bytes transferred, or time the first command on a SWE image.

Sandman: Sandboxes for Agentic RL on Shared Kubernetes

E

WARM POOL AND REUSE

Designs. Both A/Bs ran concurrent pairs of 16-GPU GRPO runs with campaign C’s recipe and 128 trajectories in flight; the arms of a pair differed only in how grading sandboxes were obtained. Rollout pods were created on demand in both arms, because the trainer of that time used the pool for rollouts only with a verified Service binding, which needed permissions the sandbox account lacked. Pair 1 ran on 18 October and pairs 2 and 3 on 22 October from the same manifests, when the cluster API was slower (rollout create p50 6.1 and 7.5 s against 5.3 s). The warm-pool A/B ran 25 learner steps on 88 tasks: one fresh pod per grading against 2 warm pods per (image, base commit) with a 2,100 s idle timeout. The checked-reuse A/B ran 40 steps on 86 tasks that had passed the reference/empty-patch check: a single-use warm pool with a global budget of 96 pods, at least 2 per key and a 300 s timeout, against the same pool with checked reuse of grading pods, at most 8 reuses per pod. Table A17. The warm pool cut grading acquisition by 97–99.5% at the median in every pair and used 2.46 times the pool-off runs’ sandbox vCPU-hours (rollout and grading pods), mostly in idle warm pods; its effect on the steady-state learner step interval remained inconclusive in three paired runs. Last column: pool on minus off, 95% CI; acquisition: per-run medians. Pool off, pairs 1 / 2 / 3 Grading acquisition p50 (ms) Warm-hit rate Rollout create p50 (ms) Waiting share Learner step interval, steps 2–24 (s) Wall clock to step 24 (h) Idle warm pods, steady state Sandbox vCPU-hours Idle warm vCPU-hours (pod sampler)

5,159 0 5,251 7.04% 172.5 1.351 0 281 0

5,917 0 6,114 6.26% 145.9 1.186 0 249 0

7,454 0 7,470 7.62% 163.7 1.302 0 295 0

Pool on, pairs 1 / 2 / 3 26 0.932 5,250 4.58% 150.5 1.235 170.8 681 411

85 0.962 5,736 3.38% 147.0 1.083 172.8 652 372

247 0.931 7,218 4.57% 158.4 1.158 171.8 691 394

On − off [95% CI] −6,057 [−8,680, −3,434] – −210 [−686, +266] −2.80 pp [−3.56, −2.04] −8.7 [−38.4, +21.0] −0.121 [−0.174, −0.068] – +400 [392, 407]; ratio 2.461 –

The pool-on run reached step 24 sooner in all three pairs (ratio 0.906, 95% CI 0.870–0.941, so 9.4% less wall clock); the time was gained in different phases in different pairs, and the effect on the step interval over steps 2–24, and whether faster acquisition caused the gain, remained inconclusive. Table A18. Checked reuse cut pod creates per grading by 26.5–32.0% in every pair (ratio 0.710), with no detectable change in hit rate and an inconclusive effect on step time; reused pods idled in the same buffer, and the budget discarded most of them unused (one minus reused hits over checks). Last column: treatment minus control, or ratio, 95% CI. Control, pairs 1 / 2 / 3 Pool creates per grading Gradings on reused pods Pool hit rate Acquisition p50, warm pod (ms) Acquisition p50, on demand (ms) Mean idle warm pods State checks up to step 39; fresh fallbacks Check p50 / p90 (ms) Reused pods discarded unused Shadow agreement, reused primary Shadow agreement, fresh pairs Learner step interval (s)

1.306 0 0.984 30.6 4,120 78.2 – – – – – 209.6

1.287 0 0.971 121.7 5,619 77.7 – – – – – 177.0

1.340 0 0.959 234.7 7,338 69.4 – – – – – 292.3

Checked reuse, pairs 1 / 2 / 3 0.934 35.5% 0.988 31.3 4,748 86.6 3,670; 0 459 / 1,077 65.0% 1,164 / 1,164 710 / 710 219.9

0.946 34.0% 0.985 122.9 5,946 89.9 3,403; 1 495 / 1,126 65.6% 1,068 / 1,071 709 / 709 172.3

0.911 42.7% 0.979 238.1 7,687 83.6 3,706; 1 505 / 1,299 57.8% 1,410 / 1,411 684 / 684 295.7

Difference [95% CI] ratio 0.710 [0.640, 0.780] – +0.012 [−0.007, +0.031] +1.8 [−1.7, +5.2] +435 [18, 852] +11.6 [4.3, 18.9] – – – 3,642 / 3,646 2,103 / 2,103 +3.0 [−15.5, +21.6]

Of the four shadow disagreements, three came from one defective pytest-7982 pod in pair 2 (Section I) and one from a flaky sphinx-10466 test. The pool held no task known to be sensitive to reuse, so the A/B shows no harm from residue here, not a need for reuse. Pool sizing replay. Each pool-on trace was replayed through the real pool code in virtual time, three seeds per trace (Table A19). In pair 1 about 35 images were graded in any 5 minutes, and the run cycled through its 88 images about every 27 minutes, inside the 2,100 s idle timeout. In a real run, the reuse A/B’s budgeted control pool kept 98.4% hits with 78 idle warm pods, 0.46 times the warm-pool A/B’s. Table A19. A per-image pool with a 300 s idle timeout kept 94.5–97.7% hits at about half the idle pod-hours; global budgets of 16–64 pods did far worse. Idle pod-hours relative to the replayed measured configuration, where the replay overstated hits by 1.8–3.5 points. Trace

Measured setting: hits, replayed (measured)

2 per image, 300 s: hits / idle

Global budget 16 / 32 / 64: hits

Budget 96, ≥2 per key, 300 s: hits / idle

Images graded per 5 minutes

Pair 1 Pair 2 Pair 3

96.7% (93.2%) 98.0% (96.2%) 95.1% (93.1%)

96.4% / 0.50× 97.7% / 0.50× 94.5% / 0.48×

44 / 70 / 81% 46 / 73 / 85% 43 / 68 / 77%

96.6% / 0.50× 97.7% / 0.50× 94.5% / 0.48×

35 35 31

Grading-pod source over eight traces. A second replay compared three ways to get a grading pod on eight recorded traces, through the pool code the frozen deployment ran: fresh on demand; a tuned single-use warm pool, the setting with the fewest idle pod-hours that reached 95% hits on a registered grid; and checked reuse at that pool’s settings (Table A20). Checked reuse cut creates per grading by 26–46% on the five Verified and natural-pool traces and by 39–41% on two SWE-Gym runs, but by 6% on the third, where a budget of 32 pods binds; it raised pod-hours on every trace, by 2–60%. On the SWE-Gym traces no single-use setting reached 95% hits. The replay matches the runs it can be checked against (control 1.306 creates per grading against 1.306 measured; treatment 0.920 against 0.934). Every trace replays one pod spec without pacing, and the results hold for our pool policy, which starts a replacement create on every acquire.

Sandman: Sandboxes for Agentic RL on Shared Kubernetes Table A20. Creating each grading pod on demand used the fewest pod-hours on all eight traces; checked reuse cut creates but raised pod-hours, because reused pods wait idle like warm ones. Acquisition in seconds. Pod-hours Trace Warm-pool A/B, pool off Warm-pool A/B, pool on Checked-reuse A/B, control Checked-reuse A/B, reuse Natural-pool SWE-Gym run SWE-Gym run 1 SWE-Gym run 2 SWE-Gym run 3

F

Creates per grading

Gradings

on demand

warm pool

reuse

warm pool

reuse

1,826 1,879 3,700 3,872 1,475 838 979 717

10.9 10.9 16.2 18.3 44.4 21.5 26.6 16.7

110.8 112.6 182.3 185.3 179.1 98.5 107.1 87.2

136.5 130.8 238.4 242.4 286.2 133.4 109.5 118.7

1.312 1.308 1.362 1.354 1.313 1.333 1.350 1.343

0.867 0.965 0.769 0.732 0.798 0.789 1.263 0.819

On demand acquisition p50 / p99

Acquisition p99, warm pool / reuse

5.16 / 57.94 5.16 / 64.52 4.11 / 40.01 4.11 / 38.77 11.18 / 94.53 13.29 / 106.51 13.56 / 92.68 10.54 / 88.32

5.84 / 5.70 5.65 / 5.73 4.73 / 5.05 5.11 / 5.27 36.48 / 13.83 53.45 / 8.89 53.10 / 60.95 42.32 / 9.54

C ANCELLATION WINDOWS

The stress-test leak (§4.4; first row of the paper’s Table 2) exercised one window. To cover the others we drove the unmodified provider and pool code of the version before the cancellation fix (13 October), the version with it (21 October) and, in a second run, the version with the error-path fix as well (27 October) against a stand-in Kubernetes API on one development host. Real code, stand-in API. Each version’s provider and pool ran unmodified from its exported source tree, building real Kubernetes model objects and calling the synchronous client from worker threads, as in production. Only the API object was replaced, by a stand-in with create, read, delete and patch running in the same threads; a local handler answered health probes once a pod was Ready, and the event loop’s executor could hold a work item and log whether a cancellation could still stop it. Gates, triggers and audit. A gate holds the first matching call (a work item not yet started, a create received or stored but not answered, a Ready poll, the health probe, or a delete) until the test releases it. Triggers are never timers: the test waits until the gate is reached, then cancels the caller, lets the pool’s 50 ms acquire timeout expire, or calls the drain, and releases the hold 50 ms later. Failure plans inject a 403, a 504 or a reset connection after the pod was stored, a failed or lost delete, and pods that fail or never become Ready. An independent auditor then lists the stand-in’s pod store and attributes each pod to whoever can still delete it (the caller’s handle, an exception carrying its identity, or the pool’s books); a pod with none is without an owner. In the first run each of windows 1–24 ran 20 times on Python 3.11 and 3.12, for the versions before and with the cancellation fix and an alternative implementation: 2,880 trials. 2,680 hit their window; the other 200 are the old version’s two-stage windows, whose second stage never occurs. Each group’s outcome and pod event order were identical in 20 of 20 repeats. Timing and cost. With the old version the caller saw its error within a millisecond at the median, and the pod was never deleted. The cancellation fix pays for ownership with waiting: a cancelled caller returns after the create in flight and the delete; a cancel before the request is sent still sends it, so under overload each timed-out acquire still creates; and an abandoned rollout create is deleted only at Ready or when the readiness budget runs out. In the old version the pool’s books held no pod while the store held one after window 12, so a capacity cap kept in the pool could not stop the leak. The alternative implementation deletes after any create error, closing windows 16 and 18, and returns a failed cleanup to the caller with the pod’s identity, which a retrying caller used to remove the pod in 20 of 20 trials. Its pool still forgot a pod whose release failed (window 24), its error lost the identity across a process boundary such as our separate pool process, and it deleted late grading pods instead of reusing them. The error-path fix (27 October). Each create now carries a nonce. After a 5xx or a transport error the provider reads the pod by its pre-assigned name and deletes it, with a UID precondition, only if it carries that nonce; a 4xx or a 409 sends no read and no delete. A delete that fails, loses its answer or is cancelled enters an owned pending set, retried with backoff from 1 s to 30 s and flushed by the drain, and the pool keeps a handle whose release failed. We reran all windows plus window 25 (a create answered 409 because another tenant’s pod holds the name) on four versions and two Python versions, 20 repeats each (4,000 trials), each trial with a sentinel pod of another tenant. The fix left no pod without an owner in any window: windows 16 and 18 are read back and deleted, and failed or lost deletes (19, 21–24) are deleted by the retry about 1.0 s later. It never deleted the sentinel (0 of 1,000 trials), while the alternative implementation deleted the pod holding the name in all 40 of its window-25 trials. All other windows behave as with the cancellation fix alone, including parking late grading pods (Table A21). Pending deletes are lost if the provider’s event loop closes first, as in the trainer’s actor teardown; the owner reference is then the backstop. Live checks. On the shared cluster we repeated, with tiny real pods and unmodified code, the three cancellation windows a real API server produces reliably: an acquire timeout before the pod is Ready, a cancel during the readiness wait, and a drain during a warm build. All 17 trials hit their window. The 16 cancellation trials, 16 of the paper’s 25 cluster trials, left no pod: late grading pods were parked and reused by the next acquire (4 of 4), late rollout pods deleted once Ready (4 of 4), and cancels and drains sent the delete within 0.05 s (8 of 8); in the 17th, an owner-reference check outside the 25, a late pod kept on purpose, still owned, was removed by the garbage collector about 11 s after its owner’s delete request. With the error-path fix, nine trials left no pod: six in which the client changed one answer (a 504 or a reset after a stored create, a failed or lost delete, a failed release, a 409) and three repeats of the cancellation windows; the pod holding the conflicting name survived. These faults were injected at the client; how often the real API server produces them is not measured. Not modelled: admission webhooks, fairness limits, quotas, conflicts, finalizers and graceful deletion (a real pod stays terminating for 5 s); watches, scheduling, pulls, address assignment and probe timing; the real client’s retries and timeouts (a retried delete could turn window 20 into a clean “not found”); scale and contention (production timeouts of 120–600 s scaled to 50 ms); how often these windows occur; and the trainer’s own fallbacks.

Sandman: Sandboxes for Agentic RL on Shared Kubernetes Table A21. Before the fix, every window in which the API stored the pod left it without an owner. With the cancellation fix a cancellation alone never does, but two error paths still lose a pod (windows 16, 18, 19 and 21–24); the error-path fix closes both and leaves another tenant’s pod alone (window 25). Stand-in API, 20 repeats per window; =: as with the cancellation fix. Timing, with the cancellation fix: median ms after the trigger until the caller’s error and the delete, with 50 ms holds, so it shows who waits for what. # Window and trigger

Before the fix (13 Oct)

Cancellation fix (21 Oct)

Error-path fix (27 Oct)

Caller / deleted (ms)

Cancel before the API stores the pod 1 cancel while the create waits in the client’s work queue

no pod (withdrawn)

=

50.8 / 50.7

2 cancel during admission; the pod is then stored 3 cancel during admission; the create is then rejected (403) 4 pool acquire times out while its create waits for a create slot

without an owner no pod no pod

request sent anyway, then deleted deleted no pod sent when the slot frees, then deleted

= = =

50.8 / 50.6 50.5 / – 0.3 / –

Cancel after the API stores the pod, before the caller has it 5 cancel while the create response is in flight 6 cancel during a Ready poll 7 cancel during the execd health probe Second cancel during cleanup 8 second cancel while cleanup waits for the create call 9 second and third cancel while the cleanup delete is held Acquire timeout with the create in flight 10 grading acquire times out during the create response; create succeeds 11 as 10, for a rollout acquire 12 rollout acquire times out while its pod waits for Ready (production path) 13 as 12, for a grading acquire, then a second acquire of the key 14 as 12, but the pod never becomes Ready Job-end drain while a warm pod starts 15 drain while a warm build waits for Ready Create stored, but answered with an error (no cancel) 16 pod stored, then a 504 answer 17 pod stored, then the connection resets 18 as 10, but the late result is a 504 after the pod was stored Delete fails or its reply is lost 19 cancel during a Ready poll; cleanup delete answers 500, not applied 20 as 19; the delete is applied but its reply lost 21 as 19; the delete is not applied and its reply times out 22 no cancel: pod fails while starting; cleanup delete answers 500 23 no cancel: destroy of a handed-out sandbox answers 500 24 no cancel: pool release of a handed-out pod answers 500 Create answered 409 (no cancel) 25 another tenant’s pod already holds the pod’s name

without an owner without an owner without an owner

deleted deleted deleted

= = =

50.8 / 50.6 0.4 / 0.2 0.4 / 0.2

no cleanup ranc no cleanup ranc

deleted deleted

= =

50.9 / 50.7 51.4 / 51.0

without an owner

kept warm; next acquire reused it =

0.8 / –

without an owner without an owner

deleted deleted once Ready

= =

0.8 / 52.0 0.9 / 51.6

without an owner without an owner

kept warm and reused deleted when the budget expires

= =

0.9 / – 0.6 / 952a

without an owner

deleted

=

– / 0.2

without an owner

without an owner

0.2 / –

without an owner without an owner

deleted without an owner

read back and deleted (0.4 ms) = read back and deleted (52 ms)

no cleanup ranc

without an owner (logged)

deleted by the retry (1.0 s)

0.4 / –

no cleanup ran no cleanup ranc without an owner

deleted without an owner without an owner

= deleted by the retry (1.0 s) deleted by the retry (1.0 s)

0.4 / 0.2 0.5 / – 0.4 / –

without an owner

without an owner; reports success without an owner

deleted by the retry (1.0 s)

–

deleted by the retry (1.0 s)

–

c

without an owner

no pod; other pod kept no pod; other pod kept

no read, no delete; other pod keptb

0.4 / 0.3 0.8 / –

0.1 / –

a

With a 0.3 s readiness budget and the provider’s 1 s poll; the default budget is 120 s. b The alternative implementation, which deletes after any create error, deleted the pod holding the name in 40 of 40 trials. c The first cancel had already left the pod without an owner.

G

R EPRODUCTIONS OF EXECUTION DEFECTS

Method. These reproductions ran on 27 October on one development host (8 vCPU, 30 GiB, Docker) with no cluster access. execd ran as a SWE sandbox pod runs it: in a SWE-bench task image (django-11099), started with the image’s own interpreter as process 1, with its source mounted read-only from the stated version, one fresh container per case, reached over loopback HTTP; agent-level cases ran the agent’s real tools. Every table records how we know its condition happened. A second full run, except the unterminated-input case, gave identical tables apart from timing. Table A22. Child processes: before the October changes, a timed-out or crashed command left up to two children running past a restart; now none survives a reset in seven of eight scenarios. Cells: children alive after execd answers / after a reset. Condition hit: each child wrote its process-id file once running. Timeout 3 s. Scenario timeout: plain background child timeout: background child ignores the termination signal timeout: foreground child ignores the termination signal shell killed, 2 children shell exits, 1 child shell killed, child still holds the output pipe normal: 2 background children left running on purpose timeout: child that starts its own session

Children running before

21 Jul

19 Aug

7 Oct (= 21 Oct)

2 2 1 2 1 1 2 2

2/2 2/2 1/1 2/2 1/1 1/1 2/2 2/2

0/0 1/1 1/1 2/2 1/1 0/0 2/1 1/1

0/0 0/0 0/0 0/0 0/0 0/0 2/0 1/1

When the shell died while a child held the output pipe, execd noticed only at the 300 s timeout, on both versions. A child in its own session survives a reset, but the state check finds it and its restore kills it. The agent’s shell tool drops execd’s error text, so for a shell

Sandman: Sandboxes for Agentic RL on Shared Kubernetes death the model sees only exit code 1. Truncated display reads. The corpus was every tracked file of two task repositories and the Python files of two package trees: 16,429 text files, 408 of them longer than the 2,000-line window. The agent’s edit tool changed one line in each, once past line 2,000 and once inside the window, while a loopback proxy recorded the editor’s reads and writes. With the old stack the edits inside the window lost 655,315 lines in all; either fix removes the truncation, and the length and SHA-256 check also makes a version mismatch fail closed (Table A23). Table A23. An edit through the display view cut every long file to 2,000 lines; complete reads made all 816 edits exact. Condition hit: the proxy observed each read, rewrite and changed byte. Agent tools / execd

Edit placed

Files

Read incomplete

File rewritten from that read

Bytes outside the edit changed

Refused

Exact

30 Jul / 21 Jul 30 Jul / 21 Jul 30 Jul / 19 Aug 6 Oct (exact reads on) / 7 Oct 13 Oct (exact reads on) / 21 Oct 27 Oct (on or off) / 21 Oct 6 Oct (exact reads on) / 21 Jul 30 Jul / 7 Oct

past line 2,000 lines 1,000–2,000 both both both both both both

408 408 816 816 816 816 816 816

408 408 0 0 0 0 0 (no route) 0

0 408 0 0 0 0 0 0

0 408 0 0 0 0 0 0

408 0 0 0 0 0 816 0

0 0 816 816 816 816 0 816

Table A24. The text-mode read changed bytes outside the edit in all 892 sampled files; byte-exact reads kept LF, CRLF and byte-ordermark files intact, and the editor of 27 October refuses the files it cannot write back unchanged (last column, with exact reads on or off). Cells: changed / refused / exact. Condition hit: the proxy showed each old read inexact. File category

Files

30 Jul / 21 Jul

30 Jul / 19 Aug

6 Oct / 7 Oct

13 Oct / 21 Oct

27 Oct / 21 Oct

LF-only Python files (random 500) CRLF only UTF-8 with byte-order mark not valid UTF-8 synthetic: mixed CRLF and LF synthetic: bare CR inside a line

500 385 3 2 1 1

500 / 0 / 0 385 / 0 / 0 3/0/0 2/0/0 1/0/0 1/0/0

0 / 0 / 500 0 / 0 / 385 0/0/3 0/2/0 1/0/0 1/0/0

0 / 0 / 500 0 / 0 / 385 0/0/3 2/0/0 1/0/0 1/0/0

0 / 0 / 500 0 / 0 / 385 0/0/3 2/0/0 1/0/0 1/0/0

0 / 0 / 500 0 / 0 / 385 0/0/3 0/2/0 0/1/0 0/1/0

Line endings. Of the corpus’s 386 CRLF-only files, 376 belong to one installed copy of pip; Table A24 covers the 385 that have a unique line to edit. The old read dropped the final newline in 873 of the 892 sampled files and flattened CRLF in all 385 CRLF files and the mixed-ending probe. The 19 August whole-file read refuses files that are not valid UTF-8; the exact-bytes path delivers their original bytes, but the editor before 27 October decoded them as UTF-8 and wrote replacement characters, and normalized line endings to the first one it saw. The editor of 27 October refuses, before any write, a file that is not valid UTF-8, mixes CRLF and LF, or contains a bare CR, and tells the agent to use the shell; every other file edits as before. These tests cover exact-match edits only: an edit whose target text matches only after the harness’s approximate matching is applied by rewriting the whole file, which in a quick test normalized trailing spaces and dashes on other lines. Exit-status frames. execd appends a random marker and the exit status to every command. Forced cases redefined the formatting command inside the command; one untampered case makes a 4,096-byte read end right after the marker, and a read-boundary observer recorded whether a read ended between the marker and the status. Unpinned, the split depends on load: a rerun hit the condition 71, 98 and 101 times, and the old parser again misread exactly its hit trials. The strict parser accepts only a terminated decimal from 0 to 255 and restarts the shell otherwise, which loses shell state. Exposure. Only the 19 August build misread the untampered split; the 6 October build failed only on a forced empty status, and the 7 October build in no case (Table A25). Campaigns A–D and both A/Bs ran the 19 August build, and the two SWE-Gym GRPO runs of the label re-grades and every run’s test re-execution pods ran builds without the strict parser (Table A3); their records cannot show how often a failure was read as success. The scale ramp ran the 7 October build, and the frozen deployment, the pre-registered comparison and the validation run had the strict parser in their rollout and grading pods. Table A25. The old parser reported success, or a wrong status, for a failing command whenever the status frame was empty, split or cut off; the strict parser reported the real status, or an error and a restart. Real status in parentheses. Case

Condition hit

19 Aug reports

6 Oct reports

7 Oct reports

empty status in the frame (1)

forced

0, no error; next also 0

status arrives in a second read (1)

forced

1

1 and “invalid frame”; shell restarted 1

shell dies after the marker, before the status (1) shell dies after “12” of status 127 4,053 bytes of output, then a failing command, one core (1) same, not pinned, three versions at once (1)

forced

0, no error; the next failing command also 0 0; the late “1” leaks into the next output 0, no error

1, “shell exited”

1, “incomplete frame”

forced 200 of 200 trials

12, no error 0 in 200 of 200

1, “shell exited” right in 200 of 200

1, “incomplete frame” right in 200 of 200

73, 101 and 83 of 200

0 in 73 of 200, exactly the hit trials

right in 200 of 200

right in 200 of 200

The 19 August build, which the campaigns ran, also took 3.30 ms at the median in a 1,000-call loopback benchmark. Unterminated input and floods. The model saw exit code 124 after 300 s on the old version and exit code 2 at once on the new one; bash’s diagnosis stays in execd’s error field. Both versions drained finite floods, but only the old one kept them (Table A27); a rerun of the endless case drained 11,201 MiB in 20 s.

Sandman: Sandboxes for Agentic RL on Shared Kubernetes Table A26. Unterminated input: a syntax check turns a silent 300 s stall into a 3–18 ms refusal that keeps the shell’s state, at about 2 ms per valid command. Condition hit: the old version stalled to the timeout; the check fired. Command

21 Jul

4 Aug

7 Oct

heredoc into a file without its terminator, two forms seen in a real rollout unbalanced quote nesting in a search command (seen) an inline Python program left open (shape seen) an unterminated single quote (synthetic) a valid one-word echo, 200 runs, p50 / p90

300.010 / 300.004 s, exit 124

4 / 4 ms, exit 2

6 / 3 ms, exit 2

300.005 s, exit 124 300.004 s, exit 124 300.005 s, exit 124 1.12 / 1.23 ms

18 ms, exit 2 4 ms, exit 2 9 ms, exit 2 3.30 / 4.48 ms

3 ms, exit 2 5 ms, exit 2 8 ms, exit 2 –

Table A27. Output floods: buffering grew execd’s memory to about three times the output, and an endless printer got it killed within 10 s; capping while draining kept it at 21.5 MiB. Condition hit: every flood exceeded the 100 KiB cap.

H

Output

19 Aug

7 Oct

16 MiB 64 MiB 256 MiB 1,024 MiB endless, 20 s timeout, 2 GiB memory limit

68.8 MiB, 0.13 s 213.2 MiB, 0.47 s 791.4 MiB, 1.80 s 3,106.2 MiB, 7.33 s killed for memory after 9.63 s; no response

21.5 MiB, 0.09 s 21.5 MiB, 0.19 s 21.5 MiB, 0.48 s 21.5 MiB, 3.20 s normal timeout at 20.01 s; 5,264 MiB drained

G RADING - CONDITION STUDY

Design. The study graded every SWE-bench Verified reference patch under 16 configurations of a fractional factorial design over eight two-level factors (Table A28), one fresh pod per grading. “+” is a departure some deployment ships; “−” is the reference configuration’s choice (for the CPU quota, uncapped). The reference configuration is the dataset harness’s sandbox: 2 vCPU and 8 GiB as requests equal to limits, root, open egress, no hash seed or time zone. Every factorial cell requests 1 vCPU and 2 GiB of memory (4 GiB for matplotlib and xarray); pods in the reference configuration request 2 vCPU and 8 GiB. Table A28. The eight run-condition factors. Q K M N U S P T

Factor and “−” level

“+” level

How it is applied

CPU quota: none (request 1, may burst) CPU affinity: every node CPU memory limit: 4 × request (8 or 16 GiB) egress: open user: root shared memory: 64 MiB default hash seed: unset, random per process time zone: the image’s (UTC in all 500)

a limit of 1 CPU test process pinned to one core equal to the request (2 or 4 GiB) all outbound traffic denied uid and gid 1000 a 1 GiB memory-backed volume fixed to 0 one of 10 pinned rules per grading

pod spec affinity set in the test’s process, on the idlest core pod spec the outbound guard workspace and home given to that user; the test drops to it pod spec container environment test process environment

The fraction is a 28−4 resolution-IV design with base factors Q, K, M and N and generators U = KMN, S = QMN, P = QKM and T = QKN, the minimum-aberration choice. Its defining relation has 14 words of length 4 and one of length 8, so every main effect is aliased only with three-factor and higher interactions. The two-factor interactions form seven chains: QK = MP = NT = US, QM = KP = NS = UT, MS = QN = KT = UP, KS = MT = NP = QU, PT = KU = MN = QS, NU = KM = QP = ST, and KN = MU = QT = SP; the three interactions we judged plausible (Q×K, M×S, N×U) fall in different chains. Runs and rule. The reference-patch run graded all 500 tasks in all 16 cells twice, four times in the reference configuration, and the empty patch twice in a four-cell panel: 22,000 gradings. Other runs graded 1,500 agent patches of campaign A (12,000), three leaderboard submissions (12,000), recycled against fresh pods (2,880), rebuilt against official images (3,632), and a census of 2,401 SWE-Gym tasks (16,807). A task is condition-dependent if its 32 cell gradings split by one factor (Fisher exact test, Holm over 8 factors, α = 0.05) or else by one two-factor chain (Holm over 7); every flip was hand-checked against the pod logs. Table A29. Denied egress explained 11 of the 15 flips, one-core pinning or a CPU quota the other 4. Gradings that failed at the failing level; passed at the other; passed in the reference configuration, unless a row states otherwise. Task

Factor

Mechanism

matplotlib-20488 requests-1724 requests-1921 requests-2317 requests-2931 requests-5414 pylint-4661 sphinx-10435 sphinx-7985, -8269, -8475 pylint-6528 pylint-7277 scikit-learn-14710

egress (N) egress egress egress egress egress egress egress egress one core (K) one core one core

a test reads a remote image and gets a DNS failure the tests call a public HTTP test service and get DNS failures as above as above; a connect test to a non-answering address is refused that connect test is refused instead of timing out as above the evaluation script’s install cannot fetch a dependency a remote image cannot be fetched the link checker cannot resolve its live URLs the CPU count is 1, so four tests skip, and the grader counts skips as failures as above, one test 96 threads from the 96-CPU affinity mask stall the run to the 1,800 s limit; pinning avoids it

scikit-learn-25232

quota × affinity

96 native threads under a 1-CPU quota are throttled to the 1,800 s limit

Failing / other / reference 16/16 fail; 16/16; 4/4 16/16; 16/16; 4/4 16/16; 15/16; 4/4 16/16; 14/16; 4/4 16/16; 15/16; 4/4 16/16; 16/16; 4/4 16/16; 16/16; 4/4 16/16; 16/16; 4/4 16/16; 16/16; 4/4 each 16/16; 16/16; 4/4 16/16; 16/16; 4/4 unpinned 15/16 time out; pinned 16/16 pass; reference 4/4 time out 7/8 time out; other 24 pass; 4/4

Of the 500 reference patches, 481 were stable, 15 (3.0%) condition-dependent, 2 failed everywhere and 2 were flaky within a configuration. The two that failed everywhere are image-provenance flips: our rebuilt image’s newer time-zone data changes an offset a test expects.

Sandman: Sandboxes for Agentic RL on Shared Kubernetes In a separate run, a southern-hemisphere time zone also failed two of the egress-dependent tasks. The SWE-Gym census graded each reference patch twice in the reference configuration and once in four others, three of which set a group of aliased factors, and the empty patch once. A task counts if its two reference gradings agree and its verdict differs in both cells of a group, runs out of memory at 8 GiB but passes at 16 GiB, or differs only with visible CPUs equal to the quota; 7 tasks with disagreeing reference gradings and 20 with other differences are not counted. At the reference repeats’ noise (0.15% per grading), chance gives a two-cell pattern in 0.01 tasks but the visible-CPU pattern in about 3.5, so that class (4 tasks) may be noise; without it the share is 4.62% (Wilson 3.85–5.54%). In it, 115 of the 2,401 tasks (4.79%, Wilson 95% CI 4.01–5.72%) were condition-dependent in one of four named patterns: 3.37% split by the group of one core, a memory limit equal to the request, denied egress and uid 1000 (K, M, N, U; 3.21% fail and 0.17% pass at “+”), 0.33% by the group of CPU quota, shared memory, hash seed and time zone (Q, S, P, T), 0.92% run out of memory at 8 GiB but pass at 16 GiB (all modin), and 0.17% change only when the visible CPUs equal the quota. Labels re-graded under changed conditions. Three re-grades re-graded every distinct graded patch once per cell on fresh pods, against the training configuration and a repeat of it (Table A30). In the Verified re-grade every flip outside the uid-1000 cell was one sphinx-10466 patch whose verdict depends on hash order. The 12 uid-1000 labels were all on Django tasks, and 11 came from agent patches that vendor stub copies of third-party packages, which shadow the real ones under a user install; reference patches add no stubs, so the reference/empty-patch check cannot reveal this. In the enriched run execd’s 300 s default cut 179 patches, and 26 of its training zeros (1.8% of labels) passed at the declared 1,800 s. Of the learned groups holding an egress flip, 22 of 23 were all-zero as trained (5 of 5 in the natural run, 17 of 18 in the enriched run). The variance filter dropped these 22 groups, removing their training signal. In the remaining group, the reward revision changed advantages. Table A30. On a task set without condition-dependent tasks no condition but uid 1000 (12 labels) moved a label beyond the repeat; on SWE-Gym task sets, open egress turned 1.35% and 4.9% of labels from 0 to 1, net of the repeat. Training configuration: 2 vCPU, 8 GiB, egress denied, root. Flips are counted per distinct patch; a patch carries one or more labels. Pool and run

Labels

Patches

Pair 1 of both A/Bs (88 Verified tasks)

11,399

5,201

Natural SWE-Gym task set, 60 steps (230 tasks drawn uniformly, 14 dependent)

1,776

1,273

Enriched SWE-Gym task set, 20 steps (115 dependent, 115 matched stable tasks)

1,447

834

Open egress: patches 0→1 (repeat)

Cells

Net labels, and other cells

egress open; 1 vCPU, 1 core; visible CPUs = quota; time zone; rebuilt image; memory R; uid 1000 egress open; 1 core with memory R and uid 1000; memory R; 1 vCPU, 1 core; hash seed 0

1 (the same 1)

0 labels net in every cell except uid 1000: 12 labels (0.105%)

22 of 1,273 (1); none 1→0

10 cells: the above, uid 1000, visible CPUs = quota, 16 GiB

56 of 822 (1)

25 labels, 24 net of the repeat: 1.35% of the 1,776 (task-cluster bootstrap 95% CI 0.17–2.93%); other cells changed only validity 71 labels net of the repeat: 4.9% of the 1,447; memory R invalidated up to 22 of 655 patches

The vacuous-test re-grade. For two Verified tasks the official labels omit the tests that check the fix: django-7530’s FAIL TO PASS holds one test that passes on the unfixed code, and django-10097’s holds 438 such tests while omitting the 6 its test patch adds, so the official report passes the empty patch too; an upstream fix was approved in March 2026 and closed unmerged. We re-graded every consumed positive label of campaigns C and D on these tasks (7,294) on fresh pods with the campaigns’ grader and configuration, under checks the reference/empty-patch check admits: django-10097’s hardened suite, and for django-7530 the official suite plus the omitted test. The hardened suite is the test file SWE-ABS (Yu et al., 2026) published for django-10097: six tests of URLs with user info, run instead of the official suite. The reference patch passes it and the empty patch fails 3 of its 6 tests, so the check admits it; it failed every patch that the restored official tests failed, and 212 more of D’s positives. All gave a verdict (Table A31). By learned-group share, the groups trained on these two tasks took about 313 of D’s 12,766 B200-hours (1,148 of 46,784 learned groups) and 125 of C’s 5,902 (594 of 28,000); only 605 of D’s 1,148 such groups held a wrong positive. Test-configuration edits in campaign patches. The campaign grader did not leave test-configuration files out of patches, so we scanned the graded trajectories of all runs of campaigns A and B and of 4 runs each of C and D (211,625 graded trajectories, 72,496 distinct patches; the other 31 runs of C and 35 of D were not scanned, because their files could not be restored from cold storage in time). 29 trajectories touched conftest.py, pytest.ini, setup.cfg, tox.ini or pyproject.toml; 10 were consumed positives (0.013% of 79,343 consumed positive labels). Reading all 25 rewarded patches that touch a test-configuration file or Django’s test settings, none could change which tests ran or how they were judged: the runner ignores the file (Django and SymPy ignore pytest settings) or the edit is inert (a deleted conftest.py whose hooks act only on other platforms or options; a deleted pytest.ini that set only collection defaults), and each patch also changed the file the reference fix changes. Edits to official test files cannot reach a verdict, because the grader restores them from the base commit before the tests run. Table A31. 19.0–22.9% of campaign D’s rewards on the two vacuous tasks went to wrong patches. Bootstrap: runs resampled within each campaign. Wrong: fails the restored official tests (django-7530) or the hardened suite (django-10097). Lower bound: the restored official tests for both. Campaign

Task

D D D C C C

django-7530 django-10097 both django-7530 django-10097 both

Consumed positives

Wrong

Wrong %

Run bootstrap 95%

Lower bound %

4,233 1,144 5,377 1,749 168 1,917

749 484 1,233 50 72 122

17.7 42.3 22.9 2.9 42.9 6.4

14.2–21.5 39.5–45.1 18.7–26.9 2.0–3.8 36.2–49.7 5.3–7.5

17.7 23.8 19.0 2.9 31.0 5.3

Sandman: Sandboxes for Agentic RL on Shared Kubernetes Denominators. Campaign D’s 1,233 wrong positives are 22.9% of its 5,377 consumed positives on the two tasks, 19.7% of its 6,271 consumed labels on them, and 0.37% of its 333,399 consumed graded labels (1,021, 19.0% and 0.31% by the restored official tests alone); for campaign C, 6.4%, 5.6% and 0.066% of 184,566. In campaign D, 605 of the 1,148 learned groups with labels on these tasks held a wrong positive. Most wrong patches changed the right file without fixing the bug, and they repeated: one was rewarded 170 times. The re-grade made 7,861 pods and used 98.0 pod-hours. The reference/empty-patch check grades each task’s reference patch (must pass) and empty patch (must fail) twice with early stopping, at the job’s grading configuration and, with checked reuse, a reuse configuration (Table A32). It finds tests that cannot be validated at that configuration, not wrong bytes, departures from the declared conditions, or effects only agent patches trigger. On the SWE-Gym task sets it kept out tasks that fail with egress denied or run out of memory at 8 GiB, two of them network dependencies no earlier record had flagged. It cost 0.06–0.14 pod-hours per task at the job’s configuration alone; in the pre-registered comparison, 1.49% of the certified runs’ grading pod-hours. A fixed 120 s readiness budget that included the pull left 9 tasks of a 280-task pool without a verdict, which is why the budget is now set per job. Table A32. The check cost a few pod-hours per Verified task set; pod-hours include pod creation. Pool

Tasks

Configurations

Gradings

Pod-hours

Checked-reuse A/B (Verified) Frozen deployment, SWE-Gym Frozen deployment, Verified Pre-registered, training task set

88 230; 280 115 89

fresh, reuse fresh, reuse fresh, reuse fresh, reuse

1,408 2,937 pods 577 pods 708

8.1 134.7 9.09 7.59

Pre-registered, held-out set Validation run

112 24

fresh fresh, reuse

448 188

6.41 1.82

Result rejected the two vacuous tasks admitted 218 and 241 admitted 113 admitted 86; rejected the two vacuous tasks and sphinx-8475 admitted all 112 admitted 21; rejected the same 3

Probe validation. We re-probed 240 admitted tasks of a SWE-Gym task set, one fresh pod each, created as its certification pods were. Under the pool’s own declaration no field refused a pod; as declared, the probe checked six fields (egress, visible CPUs, CPU quota, memory, user, time limit), the hash seed and time zone only when declared. Two wrong declarations were refused on all 240 pods: a 1,800 s time limit with a grader that sends no timeout, and a pinned hash seed that the pods did not set (pinning it passed 20 of 20). On 7,514 stored readings of the frozen deployment, the memory check refused 116 readings on 85 pods of 9 tasks whose pods had 16 GiB under an 8 GiB declaration. No other field was tested with a wrong declaration.

I

S TATE CHECK SCOPE

What the check watches. At create, execd records a baseline of the state that graders were seen to change, and after a grading it compares the pod against it: each repository’s head, index, configuration, hooks and tracked, untracked and ignored files, with tracked files fingerprinted by content; installed-package metadata, path files and site customization; system configuration, home directories, the temporary directory, and the local and system binary directories; and the environment, time zone and leftover processes. It does not watch the system library and state directories (the one write seen there, a locale archive, is idempotent), files more than two levels deep in the installed packages or the shared-memory directory unless a deep mode is on, directory modes, extended attributes or access-control lists. Git runs with hooks and file-system monitoring off; the root file system cannot be read-only, because the tasks’ evaluation scripts install the patched package. Decisions and cost. After release the pool asks execd to reset, diff and restore. A pod with no difference is clean; differences with an exact undo (an added file, a stray process, a moved git head) are restored to the baseline bytes and checked again; anything else sends the pod to deletion and the request to a fresh pod, and a pod is reused at most 8 times. The baseline took 0.4–0.8 s at create (at most 64 MiB of pristine copies), the check 0.23–0.51 s at the median after release (Table A33), off the grading path. No pod was left behind in 16 fault trials: 15 killed execd in the middle of a check, and in the 16th the check finished first. Table A33. The fallback rate is a property of the task set’s graders: on Verified task sets the check fell back to a fresh pod in 8 of 64,199 checks, on SWE-Gym in 23.7% of checks. Shadow: the reused pod’s verdict against a never-used pod’s. Runs

Checks

Clean

Rolled back

Fresh fallback

Check p50 (ms)

Gradings on reused pods

Shadow agreement

Checked-reuse A/B, 3 runs (Verified) Frozen deployment, 3 runs (SWE-Gym) Pre-registered, 6 runs (Verified) Validation run, 1 run (Verified)

10,779 2,505 52,639 781

0 553 0 0

10,777 1,359 52,633 781

2 593 6 0

459–505 231–282 338–350 321

34–43% 1,294 / 2,557 8,838 / 55,011 348 / 789

3,642 / 3,646 1,290 / 1,290 8,816 / 8,819 347 / 347

SWE-Gym fallbacks came mostly from graders that rebuilt build artifacts or removed package metadata. Every tenant left residue: all 768 recycled pods of the grading study’s recycled-pod run, and every check of the reuse A/B’s first pair. The three-arm test (Table A34) graded the same conditions on fresh pods, under plain reuse (execd’s reset plus git reset and clean) and under checked reuse. Every plain reset left residue, and plain reuse was wrong on 64 of 787 gradings, all 60 after a poisoned tenant among them; checked reuse was wrong on 0 of the same 787 (exact one-sided bound 3.4% per instance). Retiring pods by outcome. The check compares a pod with its own baseline, so a defect present at creation survives every restore; the defective pytest-7982 pod was deleted before inspection, and its cause is not established. The guard therefore also retires a pod after a grading with no verdict, or with a result that contradicts an earlier one for the same patch and suite on any pod (Table A35). Its two early retirements of defective pods were the pytest-7982 pod and a SWE-Gym pod that had kept an earlier tenant’s edit to a file the image ships edited; it missed a matching pod with no earlier result, and its three healthy retirements followed sphinx-10466’s flaky test. In the validation run every one of 784 releases carried its outcome, and the rule never triggered.

Sandman: Sandboxes for Agentic RL on Shared Kubernetes Table A34. Plain reuse graded 64 of 787 gradings wrongly, all 60 after a poisoned tenant among them; checked reuse graded none of the 787 wrongly. Condition

Gradings per arm

Instances

Plain reuse wrong

Checked reuse wrong

679 30 60 16 2 787

86 30 30 1 1 86

1 (an unstable test) 0 60 1 2 64

0 0 0 0 0 0

real agent patches, up to 8 per pod poisoned tenant, before the poison poisoned tenant, after the poison one task’s agent tenants (scikit-learn-14496) that task, a tenant and then poison total

Table A35. The contradiction rule retired 2 of 3 defective pods early and prevented all 3 wrong reused labels at almost no extra creates; stricter rules retired many healthy pods on SWE-Gym. Replay of 69,812 gradings of 13 runs; early: before the pod’s last grading. Rule

Retired early: defective / healthy

Wrong reused labels prevented (of 3)

Gradings moved

Extra creates, SWE-Gym / Verified

2/3 3 / 135 2 / 72 2 / 75

3 3 2 3

9 542 237 242

0.12% / 0.009% 18.7% / 0.094% 6.8% / 0.092% 6.9% / 0.098%

no verdict, or a contradiction (the guard’s rule) any apply failure or no verdict; two identical failures two identical failures the guard’s rule plus two identical failures

J

S ANDBOXES ON THE LEARNER ’ S CRITICAL PATH

This section expands §5.4. It uses existing records only. Records. Per learner step we have the step-end time, when the batch selection opened and how long it took, and the members of each batch, including groups the variance filter dropped; per trajectory, its start, sandbox create, grading and end times; and per-turn traces where they exist. These exist for campaign A (32 runs, steps 2–50, 1,568 steps), the warm-pool A/B (6 runs, 144 steps), the checked-reuse A/B (6 runs, 234 steps) and the pre-registered comparison (12 runs, 1,788 steps, without selection timing). Campaigns B–D have per-run totals only. The learner waited for data during 78.0% of each step in A (measured), 73.6% in B and 64.7% in C (the result files’ learner-idle share) and at most 81.8% in D; mean in-flight occupancy was 99% in A’s arms without re-runs, 97% in the checked-reuse A/B and 99.6% in the pre-registered comparison, but 88–96% in the partly learner-bound warm-pool A/B and 60% in A’s arm that re-ran the most test suites. Decomposition. The trainer keeps a fixed number of trajectories in flight, and a step consumes the next complete, graded groups; there is no batch barrier and no straggler abort. Each step interval splits into the wait for complete groups (selection and any backfill rounds), the batch hand-over, and training; components add up to every step interval within 0.003 s. Each wait window is attributed to the phases of the trajectory whose completion ended it, the slowest member of the last group admitted. If that trajectory started inside the window, the earlier part goes to its failed predecessor (failure and retry) or to the trajectory whose end freed the slot it took, recursively; unresolved time is 0.0–0.2% of step time. Sandbox waiting is rollout acquisition plus grading acquisition plus the creates of test suites the trainer re-ran at hand-over plus failure and retry; test runs and commands are reported separately. For the pre-registered comparison, which lacks selection timing, a two-window rule (selection, then backfill) is used; on the runs with timing it lands within −0.8 to +0.3 points of the reference for sandbox waiting but undercounts tests by 18–34%. Table A36. Rollout-sandbox creation was almost never on the learner’s critical path; grading acquisition and tests at the end of the slowest trajectory were. Share of total step time (%). Generation and commands: where per-turn traces exist, generation / commands; “loop” is the untraced agent loop. Other (setup, gaps, admission; at most 1.7%) omitted. Workload

Steps

Campaign A, all arms 1,568 Campaign A, arms without re-runs 784 Warm-pool A/B, pool off 72 Warm-pool A/B, pool on 72 Checked-reuse A/B, warm-pool arm 117 Checked-reuse A/B, reuse arm 117 Checked-reuse A/B, outside incidents 140 Pre-registered, fresh pods 894 Pre-registered, capped reuse pool 894

Rollout Grading acq. acq. 0.0 0.0 0.0 0.0 0.9 0.5 0.0 0.2 0.3

9.5a 6.8 6.0 2.0 1.0 1.1 1.3 4.5 13.3

Failure, retry

Generation / commands

Learner

Sandbox waiting

0.0 47.9a loop 19.8 0.1 37.9 loop 30.0 0.0 7.8 20.7 / 2.3 0.0 10.0 30.7 / 3.4 6.7 9.1 36.3 / 8.7 6.4 8.2 36.1 / 12.6 0.0 11.2 42.4 / 3.9 0.0 8.4 41.2 / 2.8; loop 22.5 0.2 7.6 35.3 / 2.6; loop 21.1

22.8 25.1 62.8 53.2 35.6 33.5 40.5 20.2 19.5

9.6 6.9 6.0 2.0 8.6 7.9 1.3 4.7 13.8

Tests

a In the two arms that re-ran test suites when a batch was handed to the learner, re-runs added 9.5 of their 11.8 acquisition points and 41.5 of their 56.5 test points. In the checked-reuse A/B, failure and retry took 6.4–6.7% of each arm’s step time, all inside incident windows (0.0% outside).

The accounting estimate. The attribution above says where each wait ended, not what faster sandboxes would buy: with shorter sandbox phases, other trajectories would finish first and other groups would enter training. The estimate uses a fixed-concurrency accounting model. In a generation-bound run, Little’s law gives step interval = trajectories consumed per step × mean trajectory time / mean trajectories in flight; it reproduced the measured mean step interval within 3% for every workload with records. Removing a fraction f of every trajectory’s time raises throughput by 1/(1 − f ), so a step shortens by about f if the run stays generation-bound and generation does not slow at the higher concurrency, plus the re-run creates, which sit on the learner’s own path. f is sandbox acquisition plus the slot time of failed attempts, over trajectory time. For B–D we bound the sandbox-attributed wait by the learner’s whole wait and estimate it as κf T , with κ, the ratio of critical-path sandbox waiting to f T , measured on the 28 traced runs without re-runs (median 1.12, range 0.74–1.54). GPU-hours count all GPUs of a run while its step waited; on a learner-only basis they halve. Power of the A/Bs. With three pairs, the minimum detectable change in the mean step interval at 80% power (two-sided, α = 0.05, paired t) is 39.0 s, 24% of the warm-pool A/B’s 160.7 s baseline, and 24.4 s, 11% of the checked-reuse A/B’s 226.3 s. The warm pool’s

Sandman: Sandboxes for Agentic RL on Shared Kubernetes Table A37. Estimated step-time reduction from removing every sandbox wait, per campaign, under the fixed-concurrency accounting model (a model output in every row). Both percentages are of analyzed B200-hours, i.e. of analyzed run time (each run’s first step and start-up excluded; B and C: whole runs, start-up included). Critical-path waiting is traced for A and estimated from per-run totals for B–D (brackets: κ = 0.74–1.54). Campaign

B200-h

Analyzed B200-h

Sandbox waiting on the critical path, B200-h (% of analyzed)

Estimated reduction (%)

A (traced) B (estimated) C (estimated) D (estimated) A–D

1,773 966 5,902 12,766 21,407

1,610 966 5,902 12,617 21,095

153.8 (9.6) 105 [70–145] (10.9) 656 [433–902] (11.1) 911 [601–1,253] (7.2) ≈1,830 [1,260–2,450] (8.7)

11.0 14.3 11.4 6.5 ≈8.5

Note 81.6 of the 153.8 from re-run test-suite creates includes 1,282 failed attempts that held the 300 s budget 64 GPUs per run f from sampled traces (6.46%)

predicted effect is −4.5 s (Little’s law, which assumes a generation-bound run; this A/B was partly learner-bound) to −8.2 s (critical path), with power 7–11% at n = 3; 19–59 pairs would be needed. Checked reuse left acquisition latency unchanged (p50 / p90 / p99 60 ms / 355 ms / 7.0 s against 65 ms / 344 ms / 6.0 s), so it predicts no change. Without the pool, the last group’s grading acquisition was on the path that ended the learner’s wait in 52% of steps, 8.5 s per step; with it, 0.3 s. Time to step 24 was 9.4% shorter with the pool (Table A17). The learner’s own part of each step was about 20 s shorter on average in the pool arms (80 against 99 s), mostly from 6–8 long learner-side steps in each pool-off run; no sandbox operation runs in that part, but faster acquisition could still affect it indirectly, for example through when batches arrive, so these records do not isolate the pool’s share of the end-to-end reduction. Several observations make the step interval hard to attribute: 41% of pool-off steps did not wait at all, generation was held back by a full channel 25% of the time, and the learner’s own part of the step differed between arms by about 20 s, as much as the predicted effect. All A/B arms ran the old exit-status parser of the 19 August build (§4.4); it cannot affect acquisition latency or creates, but it may bias agent-behavior metrics in both arms.

K

P RE - REGISTERED COMPARISON

Registration. Before any training run, the protocol fixed the arms, endpoint, evaluation step, seeds, paired analysis, readings and stop rules; it was committed to our internal repository and stored in an immutable cluster object (22 October, 07:40), and deviations were logged append-only. The code was frozen: the checked-reuse A/B’s trainer, whose verdict code is byte-identical to campaigns C’s and D’s, the provider, pool, reuse guard and check of 21 October, and execd of 20 October. Arms. The package is the combined grading change of §5.4. The package arm trained on campaign D’s registered 89 tasks minus the 3 that the check rejected at the job’s configuration, and each of its jobs refused to start unless its prompts chained by SHA-256 to the check’s record; it graded through the warm pool with checked reuse (at most 8 reuses per pod) from a pool of at most 32 pods (16 warm) built at 0.3 creates per second, and the execd token on pooled grading pods (2,361 gradings, 4.3%, fell back to direct pods without it), and shadow-graded every reused-pod grading, plus a fixed 30% sample of never-used gradings, on never-used pods. The arm without the package trained on the 89 tasks as registered, the two vacuous tasks and sphinx-8475 included (campaign D’s runs removed sphinx-8475 at start), and graded each patch on one fresh pod created directly, as in production. Both arms denied outbound traffic on every pod. The arms thus differed in three things at once (task set, grading path and token), and the comparison tests a package, not the task set alone. Design. Everything else was campaign D’s recipe: Qwen3.5-35B-A3B with GRPO on 16 GPUs, 64 trajectories in flight, 4 groups of 8 per step, 2 vCPU and 8 GiB sandboxes, outbound traffic denied, and the official test suite as the grader, for 150 learner steps (campaign D’s mid-training evaluation point). A seed fixes the rollout seed and prompt order, so both runs of a pair see the same task stream up to the pool difference. The registered analysis uses every pair launched, at least four: pairs 1–4 were the minimum set, and pairs 5 and 6 were launched under a calendar rule (both runs could start by 23 October 06:00, decided from the clock and the launcher alone). The check admitted 86 of 89 training tasks in 708 gradings, rejecting django-7530 and django-10097 (empty patch passes) and sphinx-8475 (link checks fail with egress denied), and all 112 held-out tasks. Endpoint and readings. Each run’s step-150 weights were evaluated once on the 112 held-out tasks, 8 samples per task at temperature 1.0, each grading on its own never-used pod. The endpoint is the mean over tasks of passes per valid sample, compared per pair. The registered readings were “package higher” or “no package higher” if the 95% paired t-interval lies above or below 0, else “no difference detected”, plus “within ±3 points” if the interval lies in [−3, +3]. At campaign D’s pair-difference standard deviation of about 4 points the protocol expected a half-width of 6.4 points at 4 pairs and 4.2 at 6. Table A38. The package scored lower in five of six pairs and tied in one; the registered reading is “no package higher”. Held-out resolve rate at step 150 on 112 tasks; difference in points. Pair

Package

No package

Difference

1 2 3 4 5 6

0.4286 0.4721 0.5112 0.4487 0.4531 0.4531

0.4810 0.4721 0.5223 0.4810 0.4766 0.4732

−5.25 0.00 −1.12 −3.24 −2.34 −2.01

mean, 6 pairs mean, pairs 1–4

0.4611 0.4651

0.4844 0.4891

−2.33 −2.40

Result. Over six pairs the mean difference was −2.33 points (standard deviation 1.81; 95% CI −4.22 to −0.43), so the registered reading is “no package higher”, and equivalence within ±3 points is not shown either. The exact sign-flip test gives p = 0.0625. Both arms gained about 20 points over the base model’s 0.2612. The sensitivities agree in sign (both suites −2.25, Django tasks −2.59, other repositories

Sandman: Sandboxes for Agentic RL on Shared Kubernetes −2.00, unpaired Welch −5.47 to +0.82, leave-one-pair-out −1.74 to −2.79), and pair 2’s tie is a coincidence: its runs differ on 46 tasks and each passed 423 of 896 samples. Analyses of the minimum set, pairs 1–4 (−6.10 to +1.30), and of pairs 1–5 (−4.89 to +0.11) read “no difference detected” and changed no decision, because pairs 5 and 6 had been launched before any evaluation was read. With five non-zero pairs, the exact sign-flip test cannot go below p = 0.0625. Table A39. The package’s mechanisms ran by default in six runs, and the arm without it trained on 623 labels from the three rejected tasks. Grading pod-hours come from different methods in the two arms and are not comparable. Trajectories / graded labels / consumed graded labels Learned groups; B200-hours (estimate) Gradings: reused / warm or on demand / direct Creates per grading; grading pod-hours State checks: rolled back / fresh fallback / errors Check latency p50 / p99 Shadow agreement: reused pod; fresh pairs Learned members traced end to end On rejected tasks: trajectories / labels / consumed labels

Package arm (6 runs)

No package (6 runs)

59,535 / 54,980 / 25,227 3,604; 1,028 8,838 / 43,812 / 2,361 1.21; 938 (pool pod lifetimes) 52,633 / 6 / 0 338–350 / 756–764 ms 8,816 / 8,819; 13,697 / 13,698 28,832 of 28,832 0/0/0

58,340 / 53,819 / 25,246 3,600; 1,010 0 / 0 / 53,851 1.00; 323 (estimate from grading times) – – – 27,852 of 28,800 1,953 / 1,589 / 623

Operations and exposure. The arm without the package trained on 623 labels from the rejected tasks, 2.1–3.1% of each run’s consumed graded labels, all from the two vacuous suites. The package arm’s four shadow disagreements were all one flaky sphinx-10466 test, one of them on a never-used pod. Reuse cost creates (1.21 per grading), and in this configuration it did not buy acquisition latency. Measured from the acquire call to a usable pod, pacer wait included, the package arm’s 55,011 gradings took 7.8 s at the median and 20.3 s on average, against 5.5 and 7.2 s for the other arm’s 53,851 direct creates. Hits were fast: 28.8% took a warm pod and 16.1% a reused one, in 12 ms at the median. But 50.9% waited for a create the pool paced at 0.3 per second under its 32-pod cap (p50 38.6 s; per-run medians 27–44 s), and 4.3% fell back to a direct pod (p50 6.3 s). Post-hoc notes. The package arm trained on 3.4% fewer tasks (86 against 89; in the other arm those tasks held 623 of 25,246 consumed graded labels, 2.5%), and two of the three tasks it lost are easy ones whose vacuous suites reward any non-empty patch. The design cannot separate less data, easier data and the grading-path difference, so it cannot say why the package scored lower, and it does not test the other ways to handle a flagged task: fix its tests, change its run conditions, or grade it with a valid suite. Deviations, none changing an estimand: the primary runs were launched in parallel with the pilot, at the authors’ instruction, without a free-capacity launch gate; two shadow-grading glue defects were fixed without touching a training run; one run with a collectivecommunication hang in its first learner step was relaunched; and two evaluations were re-applied after GPU nodes reported needing a reset. Counting every sandbox pod of ours in the namespace, shadow grading included, as the co-tenancy guard did, we peaked at 901 of an approved 1,000; the training runs’ own rollout, grading and warm pods peaked at 712 (Table A6).

L

FAILURE PATHS AND THEIR FIXES

Table A40 pairs each observed problem with its fix, what we measured after it and what remains; one fix (one-hop limit) was not re-measured. Window tests held one call of the unmodified provider and pool at a chosen point (a window) or changed its answer, against a stand-in Kubernetes API whose pod store was audited independently (Table A21); no trial of our versions deleted another workload’s pod. All four campaigns and both A/Bs ran the old exit-status parser, whose misreads their records cannot count; the campaigns’ reaper of failed actors’ pods never ran in 122 runs, though it cannot see other leaks. Grading problems were large: in a 16-configuration factorial study, 15 of the 500 Verified reference patches flipped, 11 with egress denied, and 4.79% of 2,401 SWE-Gym (Pan et al., 2024) tasks depended on run conditions. Two of campaign D’s 88 tasks have tests that do not check the fix: with the missing tests restored, 19% of D’s positive rewards on them went to wrong patches, and 23% under a published hardened suite for one task (Yu et al., 2026) (0.31–0.37% of all D’s rewards). No production campaign trained a condition-dependent Verified task.

Sandman: Sandboxes for Agentic RL on Shared Kubernetes Table A40. Failure paths. Window tests used a stand-in API, reproductions one host; others are controlled runs or campaign data. Observed problem

Fix

Measured after the fix

Create left without an owner after a cancel or an ambiguous error; failed delete forgotten Command outlives its timeout; exit status misread at a 4 KiB read boundary Trimmed read written back as the file Planted test configuration in a patch Residue in a reused pod

create shielded; nonce read-back; delete retries

stand-in API, 25 windows × 20 repeats: no window left a pod (the pending deletes die with their old code leaked wherever the API had stored it); cluster: 25 of 25 actor trials clean reproductions: none of the four defects recurs after the fixes own-session child survives a reset

Verdicts that depend on run conditions

Tests that do not check the fix Unauthenticated inbound calls Node metadata token reachable from unguarded pods

M

group kill; output cap; syntax check; strict status frames byte-exact reads; length and SHA-256 check configuration files left out

816 of 816 long-file edits exact; on 892 sampled files every exact-match edit exact or refused original case reproduced: file left out; campaigns: 10 positives, none verdict-changing baseline state check wrong on 0 of 787 gradings (plain reuse: 64); 3,642 of 3,646 shadow re-grades agreed declared conditions; probe of on 240 pods the declared configuration passed and two wrong the readable fields declarations were refused on all 240; stored readings exposed 85 pods with undeclared memory; 789 of 789 validation-run acquisitions passed reference/empty-patch check rejected both such tasks in every Verified task set it checked unlisted pods’ connections refused; 401 without the token, 200 with token; inbound guard it; 891 of 891 validation-run grading pods guarded outbound guard; one-hop limit guarded pods could not obtain the token; the hop limit was not re-measured

Remaining

approximate-match edits other planted files state graders were not seen to change image, test suite; unreadable fields

effects specific to agent patches shared kernel; past use –

F ULL INCIDENT LIST

This appendix lists every operations case we recorded while building and running Sandman, from July to October 2026: 41 cases. A case has a GPU-hour account when our records bound the B200-hours of the jobs it hit. Ten cases do; their Effect cells are marked GPU-h with the root-cause class, and Table A49 sums them as the paper’s last lesson (§6) does: the 8 in the sandbox path by class, and the 2 cluster events outside it (Table A48) apart. The cases come from our run notes, job logs, audit-log analyses and checks made during operation. They are recorded cases, not a census: the list is neither complete nor representative, so counts of cases are not failure rates or cost weights. We group the cases by lifecycle stage only to make the list easier to read. For each case we keep where the symptom appeared, its cause and its effect apart. A cause lies in the sandbox implementation (the code, design or configuration of Sandman or its trainer integration), in cluster configuration or a shared service (the API server, admission webhooks, network, nodes, registry, or another workload on the cluster), in a Kubernetes default that we had not overridden, or is not established; a trigger is named where it differs from the cause. Effects state the unit of each count. GPU-hour losses are upper bounds of three kinds. Lifetime counts the GPUs of jobs that died or never started over their whole lifetime, queueing included. Allocated counts the GPUs that a stress test or evaluation held over its failure window, with the lower end weighted by the failed share. Stall counts the GPUs of runs that finished over their excess step or wall time. Each job’s loss is assigned to one case only, so overlapping events are not counted twice. The How verified column names the evidence kind: production, controlled (a dedicated experiment on the shared cluster), replay, or reproduction. “The extension” is a 48-run extension of campaign D. Dates are in 2026; times are UTC.

Sandman: Sandboxes for Agentic RL on Shared Kubernetes

Table A41. Recorded cases (1 of 8): the managed service, admission, scheduling and pod addresses. What happened

How found

Cause

Fix

How verified

Effect

— (motivation for Sandman).

Stop calls reached about 117 per second, inflated by our own retries (one session stopped 8–13 times); session starts were throttled and some faulted; trajectories timed out.

Managed service throttled session Errors in our run; churn (July, before Sandman). On a the service team’s managed code-interpreter service, a analysis. batch of trajectories finished together and recycled its sessions at once.

Shared managed service: Built Sandman on our default limits of 30 calls own pods: per-task per second on session images, configurable starts, stops and invokes, sizes, our own create of which we had starts and stop rates. and invokes raised and a shared per-account storage partition. Trigger: our workload (synchronized recycling).

Admission webhook collapse (14 Oct). An evaluation with 4,096 concurrent trajectories (up to 5,120 actors) created sandboxes without pacing; the policy admission webhook answered pod creates with HTTP 500, other jobs’ creates included.

Cluster configuration: the Cluster: three webhook Controlled: the scale webhook ran one replica replicas of 1 GiB on ramp’s 38,377 paced with a 384 MiB limit and separate nodes, with a creates had 0 admission errors. Load, pacing, pod had restarted 115 times disruption budget. since early September, Sandbox: paced creates. spec and day differed, and last by an out-of-memory the replica count was kill (the cluster’s change unreadable, so this is consistent with the fix, not record, which the sandbox service’s role proof. cannot read). Trigger: our workload (the burst).

First rollout: 84.9% of 6,144 trajectories (4,096 concurrent) failed (5,217); second: 5,249 (85.4%). Within 38 s, 6,558 of the evaluation’s creates and 6,126 of other jobs’ were rejected; all 21,517 HTTP-500 creates of 19:00–22:10 carried the webhook’s message. GPU-h (cluster or shared service): ≤973–1,145 B200-h (allocated: about 784 B200 for 1.46 h).

Webhook outages in campaign C Failure-cause tally Cluster or shared service. Retry in a fresh session Not re-measured under the (10–13 Oct). Pod creates failed with over the logs of (existing behaviour); a same load. HTTP 500 from unreachable mutating campaigns A–C (83 highly available webhooks: in a sample of messages, runs). webhook. about 90% the policy webhook and 10% the queue webhook.

39,834 of the campaign’s 40,039 failed session attempts; 8.79% of its 453,155 attempts, within 13 of about 57 hours. Per-attempt success 91.16%; per trajectory after one retry ≥96.69%. All 35 runs exposed. GPU-h (cluster or shared service): ≤24–39 B200-h (stall; 24 noise-adjusted, 39 raw).

Scheduling tail in campaign B (5 Oct). Per-run create latency and failure In the eight runs of the campaign’s tally. earliest window, per-run create p90 reached 37–87 s, and attempts failed with pods still Pending.

Not established.

1,282 failed attempts in 8 runs; campaign per-attempt success 96.98%. GPU-h (not established): ≤177 B200-h (stall: the window’s 8 runs of 16 GPUs took 4.46 h against 3.08 h).

Pod-address assignment tail Per-pod condition (production 14 Oct; ramps 18 and 20 timestamps (scale Oct). The VPC CNI plugin failed to ramp); assign some pods an address and the network-plugin kubelet retried, so those pods waited tens failure events. of seconds. With two warm addresses per node and a 30 s cooldown, a burst on one node can use up its warm addresses.

Not established: Soft topology spread Controlled: spread associated with pods per across nodes; watch free 2,000-pod steps later that node (placement subnet addresses. day (10 and 20 creates/s) follow-up) and with a had 0.35% and 0.45% cluster-wide, affected, p99 8 and 7 s, an time-varying condition association; back to back (scale ramp: 21% at (27 Oct), 0 of 1,000 spread pods against 254 of 1,000 about 2 pods per node). Trigger: our and other default ones (one pair). tenants’ bursts.

The evaluation’s failure counts; re-derived from the API server’s audit log.

None.

—.

One production hour: 214 of 33,236 sandbox creates (0.64%). Scale ramp: 18.9–24.7% of pods per ramp step inside a window of about 2.2 h (08:40–10:54); 0.3–1.35% per step not Ready within 120 s. Unspread 2,000-pod step: 487 of 2,000 (24.35%).

Sandman: Sandboxes for Agentic RL on Shared Kubernetes

Table A42. Recorded cases (2 of 8): co-tenant load, node-pool pinning and event volume. What happened

How found

Another workload’s run failed during The benchmark’s our benchmark (18 Oct). During the co-tenancy scale ramp’s 500-pod steps a run of the watchdog; extension failed; other tenants’ per-minute event pod-sandbox failures averaged 813 per counts. minute while the ramp was active and 311 while it was idle (Pearson r 0.50).

Cause

Fix

How verified

Effect

Cluster configuration Post-ramp extra steps Controlled: the placement 1 run lost (one arm of one (address and capped at 1,000 pods follow-up’s guard aborted seed pair). GPU-h (cluster network-plugin pressure). and 20 creates/s; watch an unspread 5,000-pod step or shared service): ≤23 when others’ failures on B200-h (lifetime). Trigger: our ramp. A address and plugin shared nodes rose from 172 pod-count co-tenancy signals. to 643 per minute (peak rule held but does not 1,422); they fell to 0–28 bound address pressure. within a minute of our teardown.

Node-pool pinning (14 Oct). Job Pending pods Sandbox implementation Select the CPU pool by generators pinned agent workers to one reporting (the pin). Trigger: other its label, not by an instance type; other tenants had filled all insufficient memory. tenants’ load (2,946 pods instance type. 115 such nodes, while 260 nodes of other and 1,272 pending on families had room for about 2,750 those nodes). workers.

Production: the driver recreated the 9 jobs under its never-started rule; all 9 completed.

6 training runs and 3 evaluations of campaign D never started. GPU-h (sandbox implementation): 0–122 B200-h (lifetime; lower end 0).

Event flood stalled the campaign Driver ticks exiting. Cluster or shared service A longer call timeout driver (14 Oct). About 63,000 warning (namespace event (75 s to 360 s). events in one hour (45,700 volume). address-assignment failures on 77 CPU nodes, 8,700 scheduling failures; 5,706 sandbox pods in the namespace) made listing events take about 180 s.

Later ticks completed.

Every driver decision blocked from 18:58 until the timeout was raised.

Sandman: Sandboxes for Agentic RL on Shared Kubernetes

Table A43. Recorded cases (3 of 8): per-agent Services and images on the creation path. What happened

How found

Cause

Fix

How verified

Effect

Service environment injection (3–4 and Tool-latency 10 Oct). Kubernetes injects variables for analysis across 35 every Service in a namespace into every run directories; Ray pods not becoming new container. Other jobs’ 3,255 Services, 3,187 of them per-agent, Ready. produced about 23,400 variables per container, slowing every shell and failing Ray readiness probes.

Sandbox implementation Service links off on (a Service per agent slot) sandbox pods. plus a Kubernetes default (service links on). Trigger: other jobs’ Services in the shared namespace.

Synchronized-start stall (17 Oct). 48 Campaign driver runs of the extension started together and logs. created about 4,600 per-agent Services in 6 minutes. Within about 2 minutes every run reported execd unreachable through Service names, and in-flight tool calls failed.

Not established. Our Staggered run starts (at The staggered restart did status note blamed most 32 runs, 4 per 15 not show this signature minutes); routing by pod (observation). The scale per-agent Services, the unstaggered start and IP. ramp later added up to Kubernetes DNS defaults 1,000 Services at a time (search path with ndots with DNS p99 ≤15 ms (a 5, no node-local cache); different scale). the same window also had API-server timeouts, queue-webhook failures and slow DNS seen by another study.

All 48 runs affected: 15 learner steps in 95 minutes across them; 20 died at the 7,200 s collect timeout, 9 made no learner step within the start timeout, and the rest stalled 57–132 minutes. GPU-h (not established): ≤1,170 B200-h (lifetime of these 29 runs and one 8-GPU evaluation, all ended in the window).

Service routing failed under churn (18 The job’s trajectory Oct; a training run on an enriched statuses. SWE-Gym task set). Pods answered by IP, but calls through their per-agent Services timed out after 10 s.

Sandbox implementation (Service routing) plus a Kubernetes default (endpoint propagation).

45–78% of trajectories in the first attempt and 80–86% in the second; 0 learner steps in 63 minutes; 63 actor replacements, each with a new Service.

Unpaced Service creates throttled (18 The step’s create Oct; the scale ramp’s routing arm). 1,000 failures. Service creates at once drew HTTP 429 from the API server’s priority-and-fairness limits (retry after 24 s).

Sandbox implementation Pace Service creates plus a Kubernetes default with the same token (API priority and bucket as pods. fairness).

Controlled: the same step, paced: 1,000 of 1,000 sandboxes succeeded; Service create p50 14 ms, p99 32 ms.

237 HTTP 429s on Service creates and 4 on pod creates; 241 of 1,000 sandboxes failed. No GPUs involved.

Possible Service-name collision (about Branch history. 11 Oct; another team). Another team suspected that concurrent jobs shared per-agent Service names.

Not established (a hypothesis).

Not verified.

Unknown.

Readiness budget included image pulls Tasks with no Sandbox implementation. A readiness budget set (21 Oct; a deployment run with frozen verdict after a retry. per job. code). Cold pulls of large task images missed the provider’s fixed 120 s create-to-Ready budget.

Controlled: every grading that ran gave the expected reference-pass and empty-fail results; 7 of the 9 tasks were fully confirmed in a later hold-out.

9 admissible tasks of a 280-task set (3.2%) got no verdict; 1 shadow grading missed; cold pulls missed the budget in 4–6 of 10 gradings of those tasks.

Base image expired from the registry Debugging a run (22–23 Sep; tool-use benchmark runs). A with 0 trajectories. registry lifecycle rule expired every image 21 days after push. The provider rejected the sandbox spec before creating a pod, so per-agent Services had no pods and agents reported execd unreachable.

Cluster configuration (registry lifecycle) plus the sandbox implementation (an unpinned, unmaintained base image).

Rebuild and pin by digest; task images are now kept for 90 days.

The next run created sandboxes.

Runs wedged with 0 trajectories.

Container store fell back to the root A sweep of all 378 disk (found 14 Oct). One node’s NVMe CPU nodes. array was not reassembled after a reboot, so its container store ran on the 100 GB root disk (91% full, disk pressure).

Cluster configuration (node boot script).

Reboot-safe array assembly, required before the container runtime and kubelet start.

—.

1 of 378 nodes.

Production smoke test with Median bash tool call 0.63 s links off: median bash call → 2.12 s in campaign A 0.016–0.022 s. (later 3.4 s); bash -c about 2.3 s. GPU-h (sandbox implementation and a Kubernetes default): ≤123–135 B200-h (stall, campaign A, our recompute; an earlier estimate gave 60–143).

A loopback forwarder to Controlled: a third attempt the pod IP; our trainer with the forwarder had 0 binds agent calls to pod transport failures. addresses.

Job-unique actor identifiers, on that team’s branch.

Sandman: Sandboxes for Agentic RL on Shared Kubernetes

Table A44. Recorded cases (4 of 8): execution, files and access boundaries. What happened

How found

A per-run metric: Commands that never returned (4 Aug). A command with an unterminated maximum tool time 302 s in early test heredoc or quote made bash read execd’s closing brace and end marker runs (August). as heredoc text, so the call waited out the 300 s command timeout and returned no output.

Cause

Fix

How verified

Effect

Sandbox implementation Parse the whole Reproduction (Table A26): 4 of 16 trajectories in one (the command wrapper). wrapped command with the observed heredocs run, about 300 s each; the Trigger: agent-written bash -n first (failing refused in 4 ms instead of a agent could not tell what shell. open) and return bash’s 300 s stall, about 2 ms per went wrong. own error. valid command; the next test run’s maximum tool time was 6.2 s.

Leftover child processes wedged Inspection of one Sandbox implementation. Run bash in its own sandboxes (August). The command 19-hour pod: 21 process group, signal timeout killed bash but not its children; zombies among 44 and reap the group, and they were reparented to init, and the processes and a make the health route health route checked only that bash was file-system walk run a trivial command within 2 s. alive, so a wedged pod kept being served. blocked for 6,089 s.

Production-scale runs 0.47 hangs per trajectory before and after, not and 32.6% of trajectories concurrent: on the same timed out on a 293-task 293-task workload, hangs SWE-bench run. fell from 0.47 to 0.26 per trajectory and timeouts from 32.6% to 22.4%; 0 zombies after 2 h. Doubling CPU had not changed timeouts (33.3% vs 32.6%).

kubectl silently truncated results (15 Oct). Over the SPDY transport, kubectl returned exit 0 with a probe’s 224 KB result cut at a stream-buffer boundary (65,536 or 163,840 bytes).

Reproduction on the None counted. driver’s read path: 2 of 5 reads complete over SPDY, 4 of 4 over WebSocket.

The campaign driver Tooling default detected a short (transport). result.

The WebSocket transport, which reports a dropped stream as an error.

Per-task working directory overridden The other team’s (about 17 Oct; another team’s analysis. evaluation). The provider appended the job-level workspace variable after the per-sandbox one, so the job default won, and agent file operations on the task’s repository were rejected.

Sandbox implementation The per-sandbox value (environment precedence takes precedence. in the provider).

—.

513 of 2,683 tasks affected; their solve rate was 0.014 against 0.150 for the others; 468 healthy tasks were dropped as broken.

Tool-call transport failures in a cluster The runs’ error episode (18 Oct, 35 minutes; the records. checked-reuse A/B). 3–4% of tool calls failed (“fetch failed”, execd unreachable), pod-create p90 rose to 44–73 s and 10 creates got HTTP 429; grading pods reached by IP were not affected.

Not established (consistent with API-server pressure).

—.

847 and 710 failed tool calls and 904 and 774 error trajectories in the two arms.

Rollout actors exceeded agent slots Ray actor listing. (22–23 Sep; tool-use benchmark runs). The trainer started 1.25 actors per concurrent trajectory, more than the 16 agent slots; some actors stayed pending, and the pool waited for all of them without logging an error.

Sandbox implementation Keep 1.25 times the (trainer sizing). concurrency within the agent slots.

The next run dispatched rollouts.

0 rollouts dispatched.

Unauthenticated inbound access to A targeted probe of execd (found 19 Oct; security finding). our own pods. A pod with no credentials or labels, on another node, ran commands in sandboxes (HTTP 200, exit 0), with or without the egress guard; no network policy blocked it.

Sandbox implementation A per-sandbox token Controlled: after the fix, No exploitation observed, (no inbound boundary) and a fail-closed ingress unlisted pods were refused but not audited; no plus cluster configuration guard. at TCP, and listed callers production campaign had (no enforced network got 401 without the token inbound protection. policy). and 200 with it. End-to-end validation run: all 891 grading pods had the token and both guards.

Unguarded pods could get instance-metadata tokens (found 20 Oct; security finding). From pods without the egress guard, the metadata token request returned HTTP 200; all 378 CPU nodes let the response travel two network hops.

A status-code probe; Cluster configuration. read-only node metadata settings.

None.

A hop limit of 1 on the CPU node templates.

Controlled: guarded pods Exposed: every sandbox of could not reach the campaigns A and B, which endpoint (63-check probe); ran no guard; none of C’s or the hop limit itself was not D’s. The credential path was re-measured. deliberately not tested.

Sandman: Sandboxes for Agentic RL on Shared Kubernetes

Table A45. Recorded cases (5 of 8): grading. What happened

How found

Cause

Planted test file passed a non-fix (July). Review of a passing Sandbox implementation Patch extraction with git add -A trajectory. (extraction). Trigger: the picked up a test configuration file that the agent’s own output. agent had planted, so a trajectory that never fixed the bug passed.

Fix

How verified

Effect

Extraction leaves test-configuration files out (our trainer, 29 Jul; not the campaigns’ grader).

Reproduction on the same case.

1 observed case.

Re-grade analysis of Sandbox implementation. Send the declared Grader cut test runs at 300 s (18–19 timeout; the in-pod Oct; the enriched-pool training run). A the run (cut logs). SWE-Gym grader revision sent no probe refuses a grader that sends none. per-command timeout, so execd’s 300 s default cut long test runs, which scored 0.

Controlled: re-grading at 1,191 of 7,090 agent-patch 1,800 s turned 24 reference logs (17%) cut; 26 training failures into passes; a zeros (1.8% of the run’s re-probe refused 240 of 240 labels) on patches that pass pods for a grader without a at 1,800 s. The SWE-bench timeout. Verified campaigns sent 1,800 s and were not affected.

Egress-denied label without the guard A live run of the Sandbox implementation. The provider installs the Controlled: 20 of 20 (18 Oct). The provider set the reference/emptyguard and refuses the guarded pods had no TCP egress-deny label without installing the patch check refused label without it. and no DNS (32 of 32 unguarded pods were egress guard, which existed only in the the deny patched provider copy that campaigns C configuration: 6 of 6 open); 6 of 6 and D used. pods reached the network-dependent tasks internet. were rejected.

Latent: all 2,540 deny-labelled pods then in the namespace had the guard.

Agent-backend error bursts in Trajectory end campaign D (14 Oct). 281,954 of the records; a trace of 6 campaign’s 1,033,764 trajectory end run directories. events were agent-backend errors before any grading. In every traced hour with a burst of HTTP 500s, the logs also showed admission-webhook failures.

Not established (associated with shared webhook failures; not established per hour).

321,496 of 1,033,764 end events (31.1%) were not valid labels, concentrated in the earliest 12 runs (19–84% of their end events, against 0.6–1.1% in the other 27). They held 1.02% of the campaign’s 50,452 agent-sandbox hours. Of the 321,496, 317,602 (98.8%) fell in those 12 runs; 281,954 were agent-backend errors and 37,686 trajectories the trainer aborted, both before any grading; 1,855 (0.58%) were gradings or captures without a verdict (1,318 failed patch captures, 526 gradings without a verdict after retries, 11 without a pinned grader version); 1 timed out.

Job controller lagged under one Job Repetition per grading (16–17 Oct; the timestamps. grading-condition study). The shared Job controller started each Job’s next repetition 15–27 minutes late (6,819 Jobs; finalizers lagging 18 minutes or more).

Cluster or shared service Harvest and delete Jobs; Qualitative. plus the sandbox a 900 s time-to-live; at implementation (one Job most 300 live Jobs. per grading).

Retry in a fresh session; —. such trajectories are never scored 0.

Repetitions of some blocks ran together.

Sandman: Sandboxes for Agentic RL on Shared Kubernetes

Table A46. Recorded cases (6 of 8): warm and reused sandboxes. What happened

How found

Cause

Fix

How verified

Effect

Sandbox reset deleted the task Reuse counters in a Sandbox implementation. Wipe the workspace Reproduction: the next run Reuse in 0 of 15–23 eligible run of 23–30 reused pods for 11 of 30 repository (17 July). execd’s reset only on request; SWE trajectories. trajectories. pods keep the repository, trajectories with 0 reset wiped every directory under the workspace, which for SWE tasks was the which is reset with git. failures. repository itself, so every reuse attempt failed and fell back to a new pod. Warm pods built without the Code review during Sandbox implementation. Hand out a warm pod requested resources (18 Oct). A setup. only if CPU, memory prewarm call without the request’s spec and workspace match built warm pods with no CPU or memory the request; a prewarm requests, that is, a different grading without a spec builds environment. nothing.

Controlled: all 90 initial Caught before any measured warm pods carried 2 vCPU run. and 8 GiB.

Warm budget discarded recycled pods The pool’s event (18 Oct; the checked-reuse A/B). At log. acquire the budget saw a key one pod short and started a replacement; the released pod came back from its check 10–20 s later, over budget, and was trimmed from the front of the buffer, where recycled pods sit.

Sandbox implementation Count pods under check Unit tests only; not (pool design). toward their key; trim re-measured on the cluster. the newest never-used pods first.

State check missed edits to files that Comparing each the image ships modified (21 Oct; a recycled grading’s deployment run with frozen code). patch-apply step Restore reset such files to the repository with its fresh head instead of their baseline bytes, and shadow; a the diff compared only git status codes, reproduction. so a further edit kept the same status and stayed invisible.

Sandbox implementation. Fingerprint tracked files Reproduction: 6 of 6 6 of 1,290 recycled gradings by content at baseline; failures reproduced on the failed to apply patches that restore baseline bytes. old build; with the fix, all 9 fresh pods applied (labels gradings matched fresh unchanged); extra fallbacks to fresh pods. pods, and on a second image 27 of 27 verdicts were equal.

A defective never-used pod was reused Shadow-grading three times (22 Oct; a replication of the disagreements. checked-reuse A/B). A never-used pod failed to apply a 475-byte patch that applied 8 times on 5 other pods; each time its state check found and removed one added file and returned it to service.

Not established (the pod Retire a pod after a was reaped before grading with no verdict, inspection, and the grader or with a result that kept exit codes only). contradicts an earlier one for the same patch and tests.

Replay of 13 runs and 69,812 gradings: the rule retires 2 of the 3 defective pods early, and 3 healthy pods after a flaky test.

58–66% of recycled pods per run reaped unused (65.0% in the original run), adding idle pod-hours.

4 wrong zero labels in one learned group of 8 (advantages ±0.935).

Sandman: Sandboxes for Agentic RL on Shared Kubernetes Table A47. Recorded cases (7 of 8): cleanup and ownership. What happened

How found

Cause

Pods left behind at job teardown (16–17 July). The per-actor teardown hook did not reliably run, so sandbox pods outlived their job.

Counting leftover pods after runs.

Sandbox implementation. Owner references to the trainer pod, a central drain, and the campaigns’ trainer’s reaper of failed actors’ pods.

Fix

How verified

Effect

Production counters, 122 5–20 pods per run. runs of A–D: every quarantined actor shut down cleanly, so the reaper never ran; pods lost on other paths (a create answered with an error, a failed cleanup delete) are not counted.

Cancelled creates leaked pods (14 Oct, Progressive collapse Sandbox implementation. Shield the create; clean Controlled: the scale ramp 9,570 serve attempts, 72.3% about 2.5 h). In a 1,536-concurrency in the job’s logs; up on any cancellation; (tiny pods, other nodes, ended in error; 9,881 acquire stress test by a member of our team, the re-derived from the wait for the create call to paced, newer code, another timeouts; 13,516 rollout pool’s 120 s acquire timeout cancelled settle before deleting; day) had 83 acquire pods created, 5,904 removed API server’s audit creates that were still running. The log. keep late pods warm; no timeouts at 12,000 only by garbage collection at provider’s cleanup caught errors but not second create; a 600 s sandboxes; 48 late creates teardown; peak 5,920 live, at cancellation, so the accepted pod was acquire timeout. were kept warm, the rest least 4,181–4,189 beyond never deleted, and the actor then made a deleted, and 0 pods were one per actor. GPU-h second, direct create. Leaked 8 GiB pods left after any step. This (sandbox implementation): filled the single node type the job ≤974–1,347 B200-h supports the fix but does selected, and creates slowed further. not show that every (allocated: 528 B200 for cancellation window was 2.55 h). hit. The 600 s timeout alone, as a workaround, gave 2.7% errors and 34 timeouts. Fallback creates bypassed the pod cap Our pod-count (19 Oct; a check run of merged code). guard. Lookahead prewarm fired late under asynchronous rollout and held 46 idle pods; the pool refused 14 creates at its cap, and the trainer’s direct-create fallback bypassed the cap 9 times: our pods went from 56 to 88 to 96 in 35 s.

Sandbox implementation. Planned: route the fallback through the pool, and trigger lookahead at rollout start.

Pool not drained at job end (19 Oct; Pool versus cluster the same check run). The detached pool pod accounting. actor received no drain call, so its pods stayed until owner-reference garbage collection about 10 minutes later.

Sandbox implementation. Not fixed in our records. —.

Not implemented.

Our 90-pod cap was exceeded for one guard tick, and the guard deleted the job.

18 pods for about 10 minutes.

Table A48. Recorded cases (8 of 8): cluster events outside the sandbox path. What happened

Cause

Fix

How verified

Effect

Node-group rolling updates under Campaign driver running jobs (18 Oct). A GPU node logs. group was rolled (294 of 336 nodes cordoned, 72 reporting status unknown) and a CPU node group; head pods were killed and jobs failed.

How found

Cluster configuration (a node-group update).

None on our side.

—.

The extension’s first attempt lost 19 runs (9 with a node-loss signature, 10 without); its second attempt, 8 jobs. GPU-h (cluster or shared service, outside the sandbox path): ≤1,155 + 154 B200-h (lifetime).

Fleet-wide stall in the extension (18 Campaign driver. Oct). 16 of 31 runs stopped progressing; 10 hit the 100-minute stall timeout.

Campaign rules: no bulk —. Not established: file-system I/O, image concurrent heavy shared-file-system reads pulls, sandbox bursts or (up to about 27 GB/s), a list loops during a sandbox burst and image campaign. pulls of another study, and a namespace-wide list loop; not attributed to the sandbox path.

10 runs deleted and classified as infrastructure failures. GPU-h (not established, outside the sandbox tier): ≤2,351 B200-h (lifetime).

GPU-side and trainer-side events Campaign drivers. (October; not the sandbox path). A mixture-of-experts dispatch deadlock at step 1 (2 runs of campaign C); GPU nodes reporting that they needed a reset (one blocked an evaluation, and 13 pods failed on it; two evaluations of the pre-registered comparison were re-applied); one relaunch for a collective-communication hang; a shadow-grading driver killed by a transient file-system error; a file-system access pod killed for memory (the driver stopped for about 10 minutes).

Not the sandbox path.

Listed for completeness; no GPU-hour total recorded (the progress-timeout rule was sized at about 26 B200-h per hang).

Progress timeouts; avoiding bad nodes.

—.

Sandman: Sandboxes for Agentic RL on Shared Kubernetes

Table A49. The 10 cases with a GPU-hour account, by root-cause class (§6). B200-h are upper bounds; a range spans the case’s bounding methods (lower end weighted by the failed share, noise-adjusted, or 0 for jobs that never started). Each job’s loss is assigned to one case, so rows add. No other case has a GPU-hour record. Case

Workload

Root-cause class

Basis

Sandbox path (Tables A41–A47) Cancelled creates leaked pods Node-pool pinning Service environment injection Admission webhook collapse Webhook outages Another workload’s run failed during our benchmark Synchronized-start stall Scheduling tail

stress test campaign D campaign A evaluation campaign C extension of D extension of D campaign B

sandbox implementation sandbox implementation implementation and Kubernetes default cluster or shared service cluster or shared service cluster or shared service not established not established

allocated lifetime stall allocated stall lifetime lifetime stall

974–1,347 0–122 123–135 973–1,145 24–39 23 1,170 177

Outside the sandbox path (Table A48) Node-group rolling updates Fleet-wide stall

extension of D extension of D

cluster or shared service not established

lifetime lifetime

1,309 2,351

Sandbox path: implementation 974–1,469; with a Kubernetes default 123–135; cluster or shared service 1,020–1,207; not established 1,347 Sandbox path by workload: stress test 974–1,347; evaluation 973–1,145; extension of D 1,193; campaigns A–D 324–473 Outside the sandbox path: cluster or shared service 1,309; not established 2,351 All 10 cases In the campaigns (of their 21,407 B200-h): A’s Service variables, B’s scheduling tail, C’s webhook outages, D’s pin

B200-h

3,464–4,158 3,660 7,124–7,818 324–473

Related documents

• Sandman: Building and Operating Sandboxes for Agentic Reinforcement Learning on Shared Kubernetes#1491106
• GARDEN: Generalist Autonomous Reward Design Through Evidence Navigation for Cooperative Multi-Agent Reinforcement Learning#1488718
• MAI-Gym: An Open All-in-One Platform for GUI Agent Evaluation and RL Training across Mobile and Computer Environments#1356275
• StateFork: Branchable Infrastructure for Agent Exploration#1169656
• StateFork: Branchable Infrastructure for Agent Exploration#1143249