arXiv CS open access

TenonAgent: Virtualizing Execution Residency for Memory-Oversubscribed LLM Agents

Yifan Zhang et al. · arxiv_cs

arXiv CS · Papers

Conceptio Open Archive

Analyze & explore related research in Conceptio

Connect this record directly into your research workflow or AI agent. Explore citations across open repositories, query full text, and export structured proof bundles.

Explore in Search →Connect via MCP / API ↗

arXiv:2611.00051v1 [cs.OS] 11 Oct 2026

TenonAgent: Virtualizing Execution Residency for Memory-Oversubscribed LLM Agents Yifan Zhang

Xinkui Zhao

Ruixin Huang

Zhejiang University Hangzhou, China 12451018@zju.edu.cn

Zhejiang University Hangzhou, China zhaoxinkui@zju.edu.cn

Zhejiang University Hangzhou, China ruixinhuang1@gmail.com

Lezhuo Xuan

Dianyu Wang

Siying Sun

Zhejiang University Hangzhou, China 693424644@qq.com

Zhejiang University Hangzhou, China dianyuwang61@gmail.com

Zhejiang University Hangzhou, China 12351020@zju.edu.cn

Xiaoyi Li

Hengxuan Lou

Guanjie Cheng

Zhejiang University Hangzhou, China bettermultiply@gmail.com

Zhejiang University Hangzhou, China 12560049@zju.edu.cn

Zhejiang University Hangzhou, China chengguanjie@zju.edu.cn

Naibo Wang

Chang Liu

Zhejiang University Hangzhou, China wangnaibo@zju.edu.cn

Zhejiang University Hangzhou, China chang.liu@zju.edu.cn

Abstract

LLM agents execute as long-lived, multi-stage workflows with substantial resource-demand variation across phases. Provisioning each agent or sandbox for its lifetime peak leads to low resource utilization, particularly for memory, whereas aggressive oversubscription risks memory contention, outof-memory failures, and costly workflow retries. This tension limits the concurrency and efficiency of agent serving under constrained host memory. We present TenonAgent, a runtime system that virtualizes sandbox residency by decoupling the logical agent’s lifetime from its physical execution environment. To manage memory oversubscription, TenonAgent employs a dual-timescale control plane. At action boundaries, it combines predictive admission control with a fail-closed, restart-safe contract, dynamically materializing or dematerializing sandboxes based on global memory pressure. During active tool execution, a sched_ext-based kernel gate paces sandbox execution permissions to reduce unexpected concurrent memory growth. The gate provides feedback control over future execution opportunities. Evaluations across diverse agentic workloads show that, under the tested workloads and parameter settings, TenonAgent substantially reduces observed OOM failures and achieves a 100% task success rate in bursts reaching an 8.6× memory oversubscription ratio. Overall, it delivers up to a 5.5× speedup over unmanaged execution, outperforms existing agent-level and cgroup-based schedulers by 5× to 37×, and provides consistent benefits across both Docker and Firecracker microVM backends.

LLM agents are turning cloud AI from sequences of stateless inference requests into long-lived, stateful computations that interleave model reasoning with tool invocation and local execution [19, 42, 43]. A single task may alternate among remote model calls, shell commands, code execution, browser interactions, retrieval, and package installation while carrying state across many steps [28, 33, 37, 47–51, 56]. Agent-serving systems commonly separate orchestration from tool execution. An external controller maintains the conversation, execution trajectory, and action metadata [22, 23], while an isolated container or microVM [2, 4, 7, 17] executes local commands, tools, and generated code. The sandbox provides an isolation and resource-accounting boundary and is commonly retained across actions, allowing later actions to reuse its files, processes, runtime state, and cached data. Figure 1 illustrates this execution model for representative software-engineering and scientific data-analysis agents. When many agents share a host, their long-lived sandboxes compete for finite memory capacity. Provisioning each sandbox for its lifetime peak strands capacity during less demanding phases, whereas smaller static allocations risk allocation stalls, swapping, or out-of-memory (OOM) failures when several sandboxes grow concurrently [5, 16, 20, 41, 45, 46]. Moreover, an agent may wait for a remote model or external service while its sandbox retains runtime state, cached data, and intermediate artifacts accumulated by preceding actions. Thus, low execution activity does not imply either a small resident footprint or readily reclaimable memory.

Introduction

Multi-round Revision Task

Plan （LLM）

Search （Tool）

Coder （LLM）

Test （Tool）

Reviewer （LLM）

lifetime from that of its execution sandbox. While the sandbox transitions between resident and absent states, an external controller preserves the agent trajectory and action metadata, and a persistent workspace retains durable execution state. The sandbox can thus be rematerialized without changing the agent’s logical identity. TenonAgent combines action-boundary decisions with runtime feedback. At action completion, a fail-closed restartsafe contract determines whether the sandbox may be dematerialized, after which memory pressure determines whether it should be retained. At action readiness, TenonAgent predicts incremental demand from the action and its trajectory, rematerializes the sandbox if necessary, and admits execution according to available headroom. During execution, a user-space controller monitors memory pressure and sandbox growth to correct prediction errors. It updates cgroup-level permissions enforced by a sched_ext gate, which delays selected sandbox tasks to limit further allocation opportunities. Although this mechanism cannot reclaim existing allocations or immediately block new ones, it forms a closed loop with action-boundary control: user space manages semantic policy, while the kernel enforces sandbox-wide execution decisions. We implement TenonAgent for Docker containers and Firecracker microVMs and evaluate it on three LLM-agent workloads under constrained memory. We compare against eager Linux, static low- and high-concurrency configurations, an agent-level reservation scheduler [18], and AgentCgroup [54]. We measure makespan, P50 workflow latency and peak memory, and OOM and timeout events; failed workflows are re-enqueued so that retry costs are included. This paper makes the following contributions:

Answer （LLM）

Multi-round Revision

(a) Software-engineering code-generation workflow. Task

Data File

Plan （LLM）

Search （Tool） Parse （Tool）

Coder （LLM）

Run （Tool）

Validate （Tool）

Artifacts （File）

Multi-round Revision

(b) Scientific data-analysis workflow.

Figure 1. Representative agent workflows. Both softwareengineering and scientific data-analysis agents execute as multi-step, stateful workflows.

Our profiling (Section 2.2) reveals that agent memory demand and execution activity vary across semantically distinct actions. Waiting-dominated intervals exhibit little local CPU activity and limited memory growth, whereas tool invocation and code execution can produce bursts of CPU activity, I/O, and memory allocation. At the same time, a waiting sandbox may retain substantial memory allocated by earlier actions. This behavior creates an opportunity for semantic resource control. Before an action executes, the runtime can infer upcoming demand from its type and causal trajectory; upon action completion, it can reconsider whether the sandbox must remain resident. Action boundaries thus expose natural opportunities to estimate incremental demand before admission and to regulate retained state afterward. Figure 2 contrasts this with existing control granularities. Wholeagent control treats the entire workflow as a single coarse scheduling unit [22, 23], while sandbox-lifetime control enforces isolation at the container or microVM level [2, 4, 7] without exposing intra-workflow transitions. In contrast, action-boundary control decouples logical agent orchestration from physical sandbox residency, enabling fine-grained memory adjustments while preserving sandbox isolation. Managing memory across these boundaries poses unique challenges distinct from CPU. Unlike CPU, which can be time-shared or throttled, memory pressure stems from both inactive retained footprints and dynamic allocations during execution, and throttling cannot reclaim allocated space. Furthermore, while action completion offers a natural reclamation point, dematerializing a sandbox is not always safe: retaining it avoids reconstruction overhead but consumes memory headroom, whereas dematerializing releases memory only when subsequent execution state is durably preserved. An agent runtime must therefore coordinate action admission, execution growth, and residency, explicitly separating whether a sandbox can be safely dematerialized from whether current host pressure makes doing so beneficial. To address this challenge, we present TenonAgent, which virtualizes sandbox residency by decoupling a logical agent’s

• We introduce sandbox residency virtualization, which decouples an LLM agent’s logical lifetime from its physical execution sandbox. With restart-safe action boundaries, eligible idle sandboxes can be dematerialized and deterministically reconstructed without breaking logical identity or task progression. • We design a two-timescale host-memory control policy that combines pressure-aware residency management, predictive action admission, and runtime execution pacing. This reduces the impact of demandprediction errors and varying host-memory contention without static per-sandbox reservations. • We implement TenonAgent for Docker and Firecracker and evaluate it on SWE-bench, ScienceAgentBench, and Orchard RL Rollout. Under the tested workloads and parameter settings, TenonAgent substantially reduces observed OOM failures, achieves a 100% task success rate in bursts reaching an 8.6× memory oversubscription ratio, and delivers up to a 5.5× speedup over unmanaged execution. 2

LLM mem

tool

...

LLM

LLM mem

tool

schedule agents within a sandbox

mem

tool

environment

shared environment

(a) Whole-agent control.

reason

LLM

... mem

...

LLM

tool

environment

obs.

act

...

tool

mem

env.

schedule sandboxes

schedule agent phases

(b) Sandbox-lifetime control.

tool env.

Representative agent workflows (e.g., Figure 1) interleave model reasoning with environment actions. Outputs from each step can dynamically alter the environment through files, dependencies, or runtime processes. Consequently, an agent workflow operates as a long-lived, stateful program rather than a sequence of stateless model requests. Resource demand fluctuates substantially across these steps. For instance, waiting on external LLM calls incurs negligible local activity, whereas local tool executions can trigger sharp bursts in CPU, I/O, and memory allocations. In production deployments, agents typically execute within isolated sandboxes (e.g., containers [30, 54] or microVMs [2, 11, 12]). Existing orchestration platforms, such as Kubernetes [8], manage resources by applying static limits at the container or pod level, enforced by Linux cgroups [39, 40]. Under this model, the execution sandbox and its resource allocations persist for the entire lifetime of the workflow, independent of the dynamic phase changes occurring inside the application.

2.2

node git git echo

node

node npx node

node

250

node npx node

Agent Workflows Management

node

Background and Motivation

2.1

CPU (%)

ls sed sed rg grep sed sed apply_patch python node node sed grep sed node

Figure 2. Resource-control granularity in agent runtimes. Whole-agent control makes resource decisions for an entire agent execution, while sandbox-lifetime control introduces isolation and accounting but does not reconsider resource decisions across actions. Phase-level control preserves the sandbox isolation boundary while using action-completion and action-ready events as finer-grained control points.

Page Faults/s

Memory (MB)

// // // // // // //

// // // // //

// //

// // //

// // // // // // //

// // // // //

// //

// // //

// // // // //

// //

// // //

321

480

0 240k

// // // // // // //

Elapsed time (s)

Figure 3. Resource dynamics of a representative SWE-agent execution, aligned with semantic events in its execution trajectory.

enters a waiting phase (e.g., awaiting an LLM response), new allocations cease and CPU utilization drops to near zero. Despite this inactivity, the memory accumulated by preceding actions typically remains resident—for instance, plateauing at roughly 150 MB after a 469 MB burst in our representative trace. These distinct regimes suggest that semantic phase boundaries (e.g., action-completion and action-ready events) provide natural, workload-aware intervention points to reconsider resource allocation, offering a more precise control granularity than fixed time windows. Experiment 2: Boundary-Policy Trade-offs under Concurrency. We compare memory-management policies applied at the same boundaries between action-execution phases. Retaining a sandbox preserves warm state and avoids reconstruction overhead, but continues to occupy memory. Teardown releases nearly all sandbox memory, but subsequent execution incurs rematerialization and re-warming costs. For each policy, we simultaneously launch either one workflow (𝑁 = 1) or twenty workflows (𝑁 = 20). All policies use the same replayed trajectories and reconstruct semantically equivalent environments. We measure task completion

Empirical Motivation

Experiment 1: Profiling Resource Dynamics. We profile representative SWE-agent [49] executions and collect user-space and OS-level signals, including CPU utilization, resident memory, page-fault activity, I/O activity, and memory pressure stall information (PSI). We align these measurements with semantic events in the execution trajectory, including model invocations, tool calls, local commands, and result validation. Figure 3 shows a representative execution trace together with its action annotations. Observation 1: Semantic Phases Delineate Distinct Resource Regimes. The profiled traces reveal that resource demand fluctuates drastically, and these shifts align strictly with semantic transitions rather than physical time intervals. During the execution of an action, local computation produces sharp bursts in CPU activity, page faults, and memory allocation. However, once the action completes and the agent 3

Agent A

Agent B

upcoming phases 102 N=1

Phase Orchestration

N=20

delay

admission control

split

Persistent

N=1

memory.reclaim

N=20

memory-aware scheduler

Teardown/Reconstruct 0

101

102

103

104

residency manager

OOM Protect

105

Traces

Task completion time (s)

103

Workload Predictor

Phase Descriptor memory.mean

cgroup

memory monitor OS Kernel

memory.peak

Figure 5. TenonAgent architecture for agent sandbox residency virtualization.

Residual memory (MiB)

Figure 4. Aggregate residual memory versus task completion time under different boundary policies. Open and filled markers denote 𝑁 = 1 and 𝑁 = 20, respectively. Each line connects the two concurrency levels for the same benchmark and policy, rather than representing a continuous trade-off curve.

TenonAgent Design

3.1 Overview TenonAgent is built around a separation between logical agent identity and physical sandbox residency. A logical agent is defined by the external controller state and durable workspace that persist across actions, whereas the sandbox is the physical execution substrate that holds ephemeral state such as processes, runtime caches, services, and live connections. By separating these two notions, TenonAgent time and aggregate residual memory after boundary procan keep an idle sandbox warm when memory is available, cessing. Aggregate memory captures the host-level footprint or safely dematerialize it under memory pressure without accumulated across concurrent workflows. Figure 4 shows changing the identity of the agent. the resulting memory–latency trade-offs. Figure 5 illustrates this architecture, which is governed by Observation 2: Concurrency Shifts the Memory–Latency a dual-timescale control plane. At coarse-grained action Trade-off. The results demonstrate that no single boundboundaries, a user-space orchestrator leverages demand preary policy is optimal across all operating conditions. At low dictions to admit new execution phases, dynamically managconcurrency (𝑁 = 1), retention-oriented policies signifiing idle sandbox residency based on global memory pressure. cantly reduce task completion time by avoiding the substanAt the fine-grained execution timescale, a sched_ext-based tial overhead of sandbox reconstruction and environment kernel gate monitors instantaneous memory growth, pacing re-warming, albeit at the cost of a larger residual memory CPU permissions to throttle sandboxes that deviate from footprint. However, as concurrency scales to 𝑁 = 20, the retheir admitted limits. Ultimately, TenonAgent couples proactention policy causes retained memory to accumulate rapidly tive demand prediction to unlock high concurrency with across all idle sandboxes. This aggregate footprint creates reactive kernel enforcement to help maintain system safety severe host-level memory contention, which inflates the comunder extreme oversubscription. pletion time and entirely erases the low-concurrency latency advantage. Conversely, the teardown policy releases nearly 3.2 Global Memory-Pressure State all memory at action boundaries, maintaining a consistently small residual footprint. While teardown incurs a baseline Unlike CPU time, memory remains occupied even when rematerialization penalty, its completion time remains much execution pauses. Memory control must therefore manage less sensitive to increased concurrency. Consequently, boundboth memory retained by resident sandboxes and additional ary management must be adaptive: favoring retention when memory that active phases may allocate. Because reclaimmemory headroom is abundant, and shifting to proactive ing retained memory takes time, TenonAgent must respond reclamation as aggregate memory pressure rises. before active execution exhausts the remaining headroom. Implications. Together, these observations motivate seIt captures this global condition using three pressure states— mantic and load-adaptive memory management. Action boundgreen, yellow, and red—denoted by 𝑄 (𝑡). aries provide practical decision points, but reclamation must The user-space controller determines 𝑄 (𝑡) by monitoring account for restart safety and current system conditions. the available memory of the workload-domain parent cgroup Moreover, boundary policies control memory retained beand estimating the remaining growth of already-admitted tween actions, not memory growth after execution begins. phases. Green indicates that sufficient headroom remains Effective oversubscription therefore requires jointly managfor both current occupancy and expected growth. Yellow ing sandbox residency, action admission, and execution-time indicates that the system is still safe but its margin is narmemory growth. rowing, requiring preventive reclamation. Red indicates that 4

Table 1. System policies in each pressure state (∼: selective throttling).

train

Workflows

Mechanism / Policy Retain idle sandboxes Predictive rematerialization Admit new execution Unrestricted memory growth

Green

Yellow

Red

✓ ✓ ✓ ✓

✗ ✗ ✓ ∼

✗ ✗ ✗ ✗

Profiles

Resource Models

match

Training phase

Phase Description

Current Traces

Deployment phase

Figure 7. Pipeline for constructing action-level resource predictors and generating causal predictions at runtime.

Keep Sandbox Resident

Failed path

Phase Extraction

resources: memory: average: ... peak: ...

Success path

Action Phase

Restore Sandbox

End

Resume

Record Result

Wait Phase

Active Proc. Check

Persist FS State

Teardown

Save Recovery Metadata

only when the completed action has externalized its logical state, left no required state exclusively in live processes or connections (e.g., persistent shells or background services), and persisted durable artifacts for deterministic reconstruction. Figure 6 shows the protocol when sandbox release is selected. The controller blocks new action dispatch throughout teardown preparation and determines eligibility through deterministic runtime checks, without LLM-based judgment. Teardown is permitted only after a validated recovery record has been committed, linking the logical agent state, filesystem artifact versions, and runtime configuration to the completed action boundary. This protocol is strictly fail-closed: any unsupported execution or preparation failure defaults the boundary to resident_only. As a correctness constraint, memory-pressure policies may still throttle a resident_only sandbox, but cannot tear it down. Upon rematerialization, restoration reconstructs the recorded environment and state; it neither replays the completed action nor restores arbitrary process memory. We detail these execution constraints and recovery procedures in Appendix.

Figure 6. Restart-safe teardown and restoration. Preparation failures retain the resident sandbox. little headroom remains or that active phases may consume the remaining capacity, requiring immediate restrictions on further memory growth. Accounting for predicted growth allows TenonAgent to react before current memory usage alone reveals critical pressure. As summarized in Table 1, 𝑄 (𝑡) coordinates two complementary control paths. User-space sandbox lifecycle management (§3.4) reclaims retained memory from idle sandboxes, while kernel-enforced execution control (§3.5) limits further memory growth in active ones. The user-space controller determines the overall policy, and the kernel promptly enforces the resulting execution permissions. 3.3

Action Descriptors and Resource Prediction

Action descriptors. For each upcoming action 𝜙𝑖 , TenonAgent constructs a descriptor D E b𝑎𝑣𝑔 , 𝑀 b𝑝𝑒𝑎𝑘 , 𝐻 b𝑖 , 𝑝𝑖 , 𝐷𝑖 = 𝑐 𝑖 , 𝑀 𝑖 𝑖

3.4

User-Space Boundary-Scale Scheduling

Causal resource prediction. Figure 7 shows the training and runtime prediction pipeline. Before action 𝜙𝑖 executes, TenonAgent constructs a causal feature vector from its action context, summaries of completed actions in the same task, and TF–IDF features derived from the prompt, current action, and recent trajectory. The predictor uses only information available before 𝜙𝑖 begins; it cannot access the action’s realized resource usage, outcome, failure state, or any future action. Separate models predict average memory, peak memory, and heavy-memory risk, allowing the scheduler to use different levels of conservatism for normal accounting, peakaware admission, and high-risk actions. After 𝜙𝑖 completes, its measured average footprint, peak footprint, and heavymemory label are appended to the task-local history and become available only to later predictions. Training samples are constructed by aligning resource profiles with extracted action phases, and task-level train/validation splits prevent leakage across correlated actions.

b𝑎𝑣𝑔 and 𝑀 b𝑝𝑒𝑎𝑘 are its where 𝑐𝑖 denotes the action class, 𝑀 𝑖 𝑖 b𝑖 captures predicted average and peak cgroup footprints, 𝐻 heavy-memory risk, and 𝑝𝑖 represents scheduling priority or latency sensitivity. Together, these fields model the action’s expected resource demand and scheduling requirements, serving as inputs to admission and execution control. Restart-safe recoverability. Separately, TenonAgent labels the boundary preceding 𝜙𝑖 as either restartable or resident_only. TenonAgent supports restart-safe recovery under a restricted action execution model rather than inferring restartability for arbitrary programs. TenonAgent does not instrument or rewrite the agent to make an arbitrary execution restartable; instead, it checks whether the boundary exposed by the existing agent execution satisfies this restricted restart-safe contract. A boundary is restartable 5

light Action Request

Algorithm 1 Runtime execution permission control

Require: Live sandboxes C, time 𝑡, available memory 𝑀, demands 𝑑𝑐 , and retained controller state Ensure: Permitted set A and updated controller state Í 1: 𝐻 ← max(𝑀 − 𝑆, 0); 𝑅prev ← 𝑐 ∈ Aprev ∩C 𝑑𝑐 2: Determine pressure 𝑝 from 𝑀 and 𝑅prev , giving red priority 3: Update 𝑏 and the Green stability timer 4: if 𝑝 = green then 5: A←C 6: else 7: E ← {𝑐 ∈ C | 𝜏𝑐 unset ∨ 𝑡 − 𝜏𝑐 ≥ 𝐷 min } Order E: overdue first (𝐷 max ), then the prescribed tie8: breaking rules 9: 𝐾 ← 𝑏 if 𝑝 = yellow, otherwise 1; (A, 𝑅, 𝑛) ← (∅, 0, 0) 10: for all 𝑐 ∈ E in order do 11: 𝑥𝑐 ← 1[𝑑𝑐 > 0] 12: if 𝑅 + 𝑑𝑐 ≤ 𝐻 and 𝑛 + 𝑥𝑐 ≤ 𝐾 then 13: A ← A ∪ {𝑐}; 𝑅 ← 𝑅 + 𝑑𝑐 ; 𝑛 ← 𝑛 + 𝑥𝑐 14: end if 15: end for 16: end if 17: Publish allow for A and deny for C \ A 18: Clear timers for allowed sandboxes; start unset denial timers at 𝑡 and retain existing ones 19: Change denied Growing sandboxes to PausedGrowing 20: Aprev ← A; 𝑝 prev ← 𝑝

Predictor H2

Admit

Compute Window

heavy

reserve for burst used Mused

credit Mc

free Mfree

Figure 8. Water-level-aware admission with light and heavy queues sharing one logical memory budget. At semantic action boundaries, TenonAgent coordinates action admission and idle-sandbox residency. Admission uses action descriptors to maintain a logical memory water level based on the predicted charges of admitted actions, whereas residency control uses the global pressure state 𝑄 (𝑡) to retain or reclaim sandboxes during wait phases. This separation enables admission to anticipate near-term demand while residency control responds to system-wide pressure. Water-level-aware action admission. TenonAgent handles actions with heterogeneous memory demands. Strict FIFO can cause head-of-line blocking when the oldest action does not fit, while always prioritizing small actions can starve memory-intensive ones. TenonAgent therefore combines water-level-aware memory accounting, logical and physical admission checks, and the light/heavy queues shown in Fig. 8. The queues share a logical memory budget and differ only in scheduling priority. For each action 𝑖, TenonAgent assigns a logical memory e𝑖 by interpolating between its predicted average charge 𝑀 avg b bpeak ) demands. The interpolation factor (𝑀𝑖 ) and peak (𝑀 𝑖 depends on 𝑀admitted , the total logical charge of currently admitted actions: 1 𝑀admitted e𝑖 = (1 − 𝛼) 𝑀 bavg + 𝛼 𝑀 bpeak, 𝑀 𝛼 = . 𝑖 𝑖 𝑀window 0

boundary is restartable; otherwise, it remains resident. Under green, TenonAgent retains idle sandboxes by default and reclaims one only when the predicted wait amortizes teardown and rematerialization. Under yellow or red, it prioritizes memory recovery and reclaims restartable sandboxes in decreasing order of measured working set. Rematerialization is triggered when the next action becomes ready. Under green, TenonAgent may also perform lower-priority predictive rematerialization. Under yellow, speculative work is disabled, but demand-driven rematerialization may proceed if it passes the physical gate and obtains a lifecycle slot. Under red, no new rematerialization starts; queued requests are reconsidered after pressure falls. Rematerialization restarts the execution service, restores the persistent workspace and controller metadata, checks readiness, and refreshes the sandbox’s cgroup identity. Concretely, at 𝑏𝑖𝑑𝑜𝑛𝑒 , TenonAgent records the result, determines RestartSafe(𝑖), releases action 𝑖’s admission charge, and applies the residency policy without requiring the next 𝑟𝑒𝑎𝑑 𝑦 action’s descriptor. At 𝑏𝑖+1 , it rematerializes a non-resident sandbox according to the current pressure state. Reconstruction is included in physical accounting only while active, and the waiting action holds no admission charge. Once the sandbox is ready, the action competes for logical admission and passes the physical gate before execution; runtime permission is subsequently reevaluated at each control update.

Here, 𝑀window is a configured capacity threshold (e.g., 6 GB), and [ · ] 10 clamps its argument to [0, 1]. Thus, the charge moves from the predicted average toward the predicted peak as the admitted logical footprint approaches 𝑀window . The charge is fixed at admission and released when the action completes, regardless of whether its sandbox remains resident. Among admissible actions, TenonAgent normally prioritizes the light queue to avoid leaving capacity unused while a heavy action waits. To prevent starvation, the oldest heavy action accrues protected logical credit at rate 𝛽; light actions cannot consume this credit. As running actions complete, the credit protects an increasing share of the logical budget for that heavy action. It affects scheduling priority only. Wait-phase residency and rematerialization. Residency control begins when an action completes. A sandbox may be reclaimed during the wait phase only if the completed 6

3.5

Runtime Execution Control

Actuation through sched_ext. The controller enforces these decisions through sched_ext. Each sandbox is placed in a dedicated cgroup, and the controller publishes an allow/deny decision for each cgroup to a BPF map. Denied runnable tasks are held in a waiting dispatch queue, while allowed tasks are eligible for normal CPU dispatch. This makes memory control proactive: instead of waiting for the kernel OOM path or relying only on post-hoc reclamation, TenonAgent can slow down memory allocation by controlling when allocating sandboxes receive CPU time. Execution control is deliberately separated from reclamation. The scheduler limits concurrent predicted growth, whereas restart-safe teardown handles cases where memory must be actively reclaimed. If Red pressure persists and no sandbox can be safely torn down, the controller continues to re-evaluate headroom and to run only headroom-checked probes. The projection is a scheduling criterion rather than a hard capacity proof: growth estimates may be wrong, and memory can change before the next control iteration.

TenonAgent controls memory pressure before it turns into an OOM event. The key observation is that the scarce resource during execution is often not a sandbox’s current footprint, but its near-future memory growth. A sandbox with a stable working set can usually keep running safely, while a smaller but aggressively allocating sandbox may quickly consume the remaining headroom. Thus, TenonAgent schedules positive memory growth as a runtime resource, analogous to how congestion control regulates sending rate rather than reacting only after a queue has filled. Algorithm 1 summarizes the control loop. Each iteration estimates per-sandbox growth from cgroup-accounted memory samples, classifies global pressure as green, yellow, or red, and publishes the corresponding allow/deny decisions to the scheduler. In green, all live sandboxes run. Under pressure, TenonAgent admits only a bounded number of growing sandboxes whose predicted growth fits the remaining headroom; in red, this bound collapses to a single headroom-checked probe.

4 Growth-aware admission. The controller distinguishes sandboxes that are growing from those that appear resident or cooling. This distinction is used only for execution control: a sandbox with no predicted positive growth does not consume a growth slot, while a sandbox with positive predicted growth does, even if its current footprint is small. This design prevents the scheduler from being biased toward small but rapidly allocating sandboxes and instead prioritizes executions that are less likely to increase memory pressure. Admission in Yellow follows a congestion-control rule. Rather than fixing a single conservative concurrency limit, TenonAgent uses additive increase and multiplicative decrease to adapt the number of concurrent growth contributors. When the system remains stable, the controller cautiously increases the growth budget; when pressure worsens, it cuts the budget back. This lets the system probe for available memory parallelism while reacting quickly to congestion, mirroring the way AIMD network controllers search for safe bandwidth.

Implementation

We implement TenonAgent as a user-space controller with Docker and Firecracker sandbox backends. The resource predictor uses a random forest regression model with semantic, scope, and resource-history features to predict memory usage. Runtime execution control is implemented using sched_ext and eBPF. The Docker and Firecracker backends expose a common lifecycle interface over backend-specific shutdown, and restore operations. The backend reports readiness to the controller only after the sandbox is available for action execution. Sandbox memory usage and pressure are collected through cgroup v2. The controller stores persandbox execution permissions in a cgroup-keyed BPF map, which the sched_ext scheduler consults when dispatching sandbox tasks.

Evaluation

5.1

Experimental Setup

Evaluation modes. We use two complementary evaluation modes: trajectory replay and online LLM-driven execution. Unless otherwise stated, we use replay for repeatable comparisons. Replay trajectories are collected from GPT-5.5driven agent runs; replay then re-executes the recorded tool calls in real sandboxes, reproduces recorded waiting times, and follows fixed action sequences rather than adapting to failures. Each replay experiment is repeated three times with the same trajectories, submission schedule, and host-memory budget, and we report the average unless otherwise noted.

Red-mode probing. When memory pressure reaches Red, TenonAgent stops treating concurrency as the goal. The objective becomes controlled progress without allowing a burst of new allocations. Red therefore permits at most one growing sandbox at a time, and only if its predicted growth fits in the current headroom. This sandbox acts as a probe: it may continue, be replaced, or be denied again at the next control iteration. The one-probe rule also prevents starvation from becoming an unbounded memory risk. Long-denied sandboxes are prioritized for reconsideration, but they are not admitted unconditionally; every admission still has to satisfy the current headroom and growth-slot constraints.

Workloads. We evaluate our system on representative stateful agent workloads spanning software engineering, scientific research, and reinforcement-learning training: 7

• SWE-bench [49]: Represents complex, multi-turn software engineering tasks where agents navigate repositories, compile code, and execute tests. • ScienceAgentBench [10]: Targets data-intensive scientific research workloads that demand significant memory for data processing and tool execution. • Orchard RL Rollout [34]: Represents reinforcementlearning training environments, generating synchronized bursts of interactive rollout trajectories to stress-test concurrent execution. Platform. Our experiments run in virtual machines hosted on an Intel Core i9-14900K machine with 128 GiB of DRAM. Each VM is provisioned with the CPU and memory configuration reported in the corresponding figure (e.g., 4 vCPUs and 8 GiB, denoted 4U8G). The VMs run Ubuntu 24.04.4 LTS with Linux 7.0.0-31-generic. We use Docker 29.7.2 for container experiments and Firecracker v1.12.1 for microVM experiments. Agent executions are implemented using the Mini-SWE-Agent framework [44, 49]. Pressure and residentmemory measurements are collected from the workloaddomain cgroup, for which swap is disabled. Baselines. We compare TenonAgent with four baselines: • Eager: runs sandboxes with the default Linux memory management and no resource control. • Fixed-concurrency: statically limits concurrent agents, evaluated under conservative (Static-Low) and aggressive (Static-High) bounds. • Agent-level peak-reservation scheduler: implements a lifetime peak-reservation baseline inspired by the reservationbased admission policy of ThunderAgent [18]. In this baseline, each agent reserves its offline-profiled peak hostmemory footprint for the lifetime of its container. • AgentCgroup [54]: uses tool-call-aligned hierarchical cgroups and in-kernel eBPF enforcement to adaptively control agent CPU and memory resources. Metrics. Across all configurations, workflows encountering an OOM event or sandbox timeout are re-enqueued for up to five retries after the initial attempt. All workflows complete within this limit. We report workload makespan and P50 workflow latency, both including retries, as well as peak aggregate sandbox memory, OOM events, and sandbox timeouts. We also report the attempt-level failure rate, computed as failed attempts divided by total attempts; its denominator may vary across configurations because of retries. 5.2

batch completion on ScienceAgentBench (7.9 vs. 43.4 minutes) and a 2.4× speedup on Orchard RL Rollout (11.8 vs. 28.2 minutes). As shown in Figure 10, this severe E2E inflation in Eager is largely driven by out-of-memory (OOM) failures and task timeouts that force costly workflow retries. By proactively managing memory boundaries, TenonAgent avoids these thrashing behaviors, directly converting reliable execution into substantial throughput gains. Conversely, conservative memory policies do not guarantee efficiency. Both Agent-level and AgentCgroup experience extreme degradation; for instance, under 4U8G, AgentCgroup extends E2E makespans to 120.6 and 293.4 minutes on SWE-bench and ScienceAgentBench, respectively. In contrast, TenonAgent completes these same batches in just 21.3 and 7.9 minutes. Overall, TenonAgent yields massive absolute improvements: up to a 37× speedup over AgentCgroup, nearly 5× over Agent-level reservations (105.6 vs. 21.3 minutes on SWEbench), and roughly 2× over the best-performing fixedconcurrency slots. This confirms that rigid reservations or tool-call-aligned throttling incur excessive queuing overhead and fail to smoothly accommodate the phase transitions of long-lived agents. We provide a detailed root-cause analysis of why AgentCgroup encounters these extreme execution stalls in Appendix. While abundant resources naturally narrow the absolute performance gap between TenonAgent and Eager (e.g., closing to 9.3 vs. 9.6 minutes on Orchard RL at 8U16G), TenonAgent still consistently delivers superior median (P50) workflow latencies across the board. Ultimately, by effectively bounding tail failures under pressure and proactively reclaiming idle footprints, TenonAgent maximizes batch throughput while remaining highly competitive when resources are plentiful. Figure 11 shows memory utilization and concurrency over time for two workloads. Eager execution sustains high concurrency, creating substantial memory pressure. In contrast, the Best Static policy is conservative and often leaves memory capacity underutilized. TenonAgent dynamically adjusts execution concurrency to the workload’s memory demand, balancing memory utilization against concurrent growth rather than applying a fixed limit.

5.3

Portability to Firecracker Backend

To verify whether the benefits of TenonAgent generalize beyond its Docker-based implementation, we migrate the execution backend to Firecracker microVMs and evaluate it under constrained host memory budgets of 8, 12, and 16 GB. As shown in Figure 12, TenonAgent demonstrates consistent robustness across all configurations, achieving a 100% attempt success rate. In contrast, the Eager baseline triggers severe memory contention and out-of-memory (OOM) failures, succeeding in only 20%–70% of attempts. Similarly, aggressive fixed-slot configurations trade reliability for concurrency, frequently crashing when concurrent microVMs

End-to-End Performance

Figure 9 evaluates all methods under a fixed offered load (7.2 QPS over a 10-second submission window) across varying host capacities. TenonAgent consistently achieves the lowest E2E time throughout this range. Under tightly constrained settings (e.g., 4U8G), TenonAgent dramatically outpaces Eager (Unlimited), delivering up to a 5.5× speedup in 8

TenonAgent

Eager

Static-Low

// //

40 20

10 8U16G 6.9

5.0

6U12G 9.0

// //

4U8G 15.7 22.4

// //

8U16G

36.8

27.1 26.1

6U12G

47.1 54.1

P50 (min)

36.4

35.2

// //

29.9 293.4

// //

4U8G

// //

4U8G

6U12G

0.0

8U16G

4U8G

6U12G

8U16G

100

4U8G

6U12G

8U16G

6U12G

8U16G

4U8G

6U12G

8U16G

4U8G

6U12G

8U16G

100

4U8G

2.5 0

AgentCgroup

Orchard RL Rollout

43.4

52.5 54.4

// //

25 0

Memory (%)

66.0 69.7

Agent-level

ScienceAgentBench

105.6 120.6

E2E (min)

SWE-bench

Static-High

4U8G

6U12G

8U16G

Figure 9. End-to-end latency (top), median task latency (middle), and peak memory utilization (bottom) across three workloads and three machine configurations. Numbers above truncated bars report their actual values. TenonAgent Eager

Figure 10. Numbers of tasks that fail because of out-ofmemory errors or timeouts. Methods and configurations with no observed failures are omitted.

Concurrency Memory (%)

75 50 25 0 45 30 15 0

600

400

SWE-bench

1200

1800

Orchard RL Rollout

800

1200

Time since run start (s)

Best Static

Makespan (s)

60 40 20 0 120 80 40 0

TenonAgent

2400

1600

1000

3000

1000

500

1500

1000 500

0 nt er -2 -4 -6 el ge Eag tatic tatic tatic t-lev A n S S S en o n Ag Te

Host memory = 16 GB

1000

100

1500 50

500

0 nt er -4 -6 -8 el ge Eag tatic tatic tatic t-lev A n S S S en o n Ag Te

2000

1000

0 el nt -2 -4 er ge Eag tatic tatic t-lev A n n S S o e n Ag Te

100

2000

1500

Host memory = 12 GB

0 nt er -2 -4 -6 el ge Eag tatic tatic tatic t-lev A n S S S en o n Ag Te

Makespan (s)

Concurrency Memory (%)

Eager

2000

100

2000

0 el nt -2 -4 er ge Eag tatic tatic t-lev A n n S S o e n Ag Te

Agent-level Success rate

Host memory = 8 GB

3500 3000

P50 (s)

Makespan (s)

8U16G

Static-6 Static-8

P50 (s)

4U8G 6U12G

Static-2 Static-4

P50 (s)

Orchard RL Rollout

Success rate (%)

AgentCgroup

ScienceAgentBench

Success rate (%)

Static-High

Success rate (%)

Eager SWE-bench

500

0 nt er -4 -6 -8 el ge Eag tatic tatic tatic t-lev A n S S S en o n Ag Te

Figure 12. Performance with the Firecracker-based execution backend under different host memory budgets. The left column displays the batch makespan (bars, left axis) and workflow success rate (dotted line, right axis). The right column reports the median (P50) end-to-end workflow latency.

Figure 11. TenonAgent dynamically regulates memory use and execution concurrency on SW-bench and Orchard RL rollouts.

E2E

P50

OOM

(a) E2E completion time

task timeout

2200 1800

800

1400

P50 (s)

1200

1000

400 10

60 40 20 0

450

P50 (s)

E2E time (s)

600

600 450

300

300 150

150

Offered load (QPS)

% .5 12

Red region size (%)

100

25% 20% 99 15% 12.5% 98 10% 5%

% .5

Red region size (%)

Figure 14. Sensitivity to red- and yellow-region sizes: (a) E2E completion time and (b) execution success rate. Green indicates better outcomes (lower time or 100% success rate), while red indicates worse outcomes. The reference configuration (12.5% for both) is outlined in orange.

30 2

1400

1600

10% 5%

1800

12.5%

(d) Failure composition

120

2000

15%

Offered load (QPS)

Figure 13. Sensitivity to offered-load Sensitivity. The top row evaluates bursts composed of different taks, while the bottom row evaluates an extreme duplicated-task burst. Left panels report E2E and P50 latency; right panels report execution failures, separated into OOM and timeout failures.

demonstrates a realistic mixed-workflow setting, where diverse execution trajectories prevent peak memory demands from aligning. At low QPS (low oversubscription), TenonAgent and eager execution achieve comparable latencies, indicating minimal scheduling overhead. However, as burst intensity scales and the oversubscription ratio surges, eager execution degrades significantly, suffering severe latency spikes and frequent out-of-memory (OOM) and timeout failures. In contrast, TenonAgent maintains stable end-to-end and P50 latencies while minimizing failures, proving the value of resource-aware control under realistic bursts. The bottom row introduces an adversarial duplicated-workflow stress test, concurrently submitting identical workflows to force their high-memory phases to synchronize. This alignment creates an extreme local oversubscription scenario. Without control, eager execution collapses under this pressure, exhibiting soaring tail latencies and massive OOM failures. Conversely, TenonAgent actively smooths this synchronized demand via admission and execution throttling, maintaining low latency and avoiding failures entirely across all tested loads. Ultimately, TenonAgent delivers robust performance under both diverse and worst-case synchronized bursts. It imposes negligible overhead during light traffic while preventing OOM-driven degradation even when the memory oversubscription ratio, defined as the sum of offline-profiled peak memory footprints of all submitted workflows divided by the host memory capacity, is pushed to an extreme 8.6×.

experience overlapping memory spikes. By dynamically regulating residency and execution permissions based on global memory pressure, TenonAgent safely navigates these strict capacity limits. Beyond simply avoiding failures, TenonAgent consistently outperforms the safe baselines in both throughput and latency. For a fair comparison, we isolate the best static configurations that manage to achieve 100% success at each budget (Static-2, Static-6, and Static-8). Against these conservative baselines, TenonAgent achieves a 1.97×–4.83× speedup in batch makespan and reduces P50 end-to-end workflow latency by 1.7×–4.2×. It also significantly outperforms the agent-level reservation baseline, delivering 3.02×– 4.36× makespan reductions. These substantial performance gains emerge because TenonAgent actively reclaims memory from idle microVMs waiting on external LLM inference, repurposing that headroom to admit more active tool executions concurrently. Overall, these results confirm that TenonAgent’s core design—decoupling logical agent orchestration from physical execution residency—is fundamentally agnostic to the underlying isolation mechanism. 5.4

20%

Offered load (QPS) Failures (count)

Offered load (QPS) 750

Yellow region size (%)

(b) Failure composition Failures (count)

E2E time (s)

(a) E2E and P50 time

(b) Execution success rate 2200

25%

Success rate (%)

TenonAgent

E2E time (s) Yellow region size (%)

Eager

Offered-load Sensitivity

5.5

Figure 13 evaluates TenonAgent’s resilience to short-duration load bursts by varying the submission rate (QPS) over a fixed 10-second window. Consequently, higher QPS increases both arrival intensity and the total workflow volume, directly driving up the memory oversubscription ratio (reaching up to 8.6× the physical host capacity at peak demand). This evaluates tolerance to finite-duration bursts rather than sustainable steady-state throughput. The top row of Figure 13

Memory-Region Sizing Sensitivity

We examine how the relative sizes of the red and yellow memory regions affect workload performance and execution reliability. Unlike ordered boundary thresholds, these parameters are defined as percentages of the total memory budget and are varied independently. Figure 14 illustrates a parameter sweep with region sizes ranging from 5% to 25%. The reference configuration (red = 12.5%, yellow = 12.5%) is outlined in orange. 10

Table 2. Direct evaluation of memory-demand predictors. Lower is better for MAE and P90 APE; higher is better for heavy-action precision and recall. Heavy actions have measured peak memory above 1,024 MiB.

Reliability requires sufficient emergency headroom. Figure 14b demonstrates that TenonAgent maintains a 100% execution success rate across most configurations. However, reliability drops when both the red and yellow regions are aggressively small (e.g., red ≤ 5%, yellow ≤ 10%). In the red state, the kernel actuator suspends active sandboxes to halt memory growth. If the red region is too small, the system lacks the physical headroom to absorb instantaneous allocation spikes (overshoots) that occur between the controller’s observation and the physical enforcement, ultimately resulting in OOM failures.

Numerical prediction Method Static History-only Current

5.7 Performance trade-offs of preventive and reactive control. Figure 14a reveals distinct performance penalties for over-provisioning either region. When the yellow region is excessively large (e.g., 20%–25%), E2E completion time increases significantly, exceeding 2000 seconds. A large yellow zone makes the system overly conservative, triggering premature preventive reclamation of idle sandboxes. This destroys warm environments that could have been reused, introducing severe sandbox reconstruction overhead. Conversely, an oversized red region (e.g., 20%–25%) stretches completion time by prematurely suspending active growth contributors, unnecessarily reducing execution concurrency before true memory pressure materializes. The reference configuration (12.5% for both regions) hits the empirical “sweet spot”. It provisions a yellow region large enough to give background sandbox teardowns sufficient time to complete, paired with a red region deep enough to safely buffer active allocation overshoots. This dual-region balance safely maximizes concurrency and achieves the lowest workload makespan.

5.6

Heavy classification

MAE (MiB)

P90 APE (%)

Precision (%)

Recall (%)

198.6 154.3 93.9

442.3 175.2 88.8

52.0 50.7 77.3

80.0 48.8 82.9

Online Agent Evaluation

Unlike replay-based experiments, this evaluation runs live, LLM-driven agents on SWE-bench and ScienceAgentBench. All online runs use a single workload level on a 4U8G VM. We evaluate two Qwen3.8 configurations, Qwen3.8-Flash [35] and Qwen3.8-Max [36], served by the Alibaba Cloud Qianwen platform. For each benchmark and model configuration, we compare eager execution (w/o TenonAgent) with execution under TenonAgent (w/ TenonAgent) using the same task inputs, agent implementation, LLM configuration, resource budget, and workflow arrival schedule. Since online LLMdriven execution is inherently variable, each configuration is repeated five times; bars report the mean and error bars show variation. Figure 15 reports task accuracy and end-to-end execution time. Across both benchmarks and both Qwen3.8 configurations, TenonAgent preserves task accuracy, with w/ and w/o TenonAgent bars closely matched in all cases. In contrast, end-to-end time consistently decreases under TenonAgent. These results suggest that TenonAgent improves execution efficiency in online setting without an observable loss in task accuracy. We further examine restart-safe coverage and lifecycle overhead. Table 3 reports averages across five runs. SWE-bench executes 1,952.8 actions on average, with 93.4% restartable boundaries; ScienceAgentBench executes 536.2 actions on average, with 100% restartable boundaries. TenonAgent performs 396.8 and 124.4 teardowns on average, reclaiming 33.6 GiB and 8.6 GiB of memory, respectively, with cumulative lifecycle overheads of 805.0 s and 530.4 s. These results show broad restart-safe coverage and substantial memory reclamation, while quantifying the teardown and reconstruction cost.

Evaluation of Memory Prediction

We compare three action-level predictors: Static (uses only action semantics), History-only (reuses the preceding action’s memory demand), and Current (combines semantics with causal execution history). As shown in Table 2, relying on isolated signals degrades prediction quality. The Static predictor suffers from massive numerical errors (e.g., a P90 APE of 442.3%) and low precision (52.0%), generating many false positives by over-predicting heavy actions. Conversely, Historyonly reduces overall numerical error but its heavy-action recall plummets to 48.8%, proving that past demand alone cannot reliably anticipate sudden memory spikes. Our Current predictor synthesizes both complementary signals, dominating all metrics. Compared to Static, it halves the Mean Absolute Error (MAE) and slashes P90 APE by 80%, while achieving the best balance of heavy-action precision (77.3%) and recall (82.9%). We further demonstrate how this precise demand estimation translates to end-to-end system reliability and latency improvements in Section 5.8.

5.8

Ablation Study

We evaluate the individual contributions of admission control, memory-growth control, dynamic sandbox residency, and the demand predictor under an 8 GB memory budget. Starting from the full TenonAgent system, we construct five degraded variants. Table 4 summarizes the performance, reliability, and concurrency metrics for each configuration. Core Runtime Mechanisms. The full system delivers the best overall performance (1276 s E2E, 522 s P50) with zero 11

ScienceAgentBench

55 50 45 30 20 10 0

w/o TenonAgent

E2E time (s)

Accuracy (%)

SWE-bench

Flash

Max

Flash

Max

w/ TenonAgent

supported by frameworks providing multi-agent and tooluse abstractions [1, 9, 13, 22, 23, 27]. Emerging research treats these executions as schedulable programs: Alto [38], Autellix [29], Helium [32], and Parrot [26] orchestrate distributed compound systems, while Continuum [24] and ThunderAgent [18] optimize GPU-side KV-cache retention for multiturn interactions. However, agentic workloads also severely stress CPU-side memory and tool orchestration [31]. Unlike systems predominantly optimizing distributed routing or accelerator inference, TenonAgent addresses the orthogonal bottleneck of host-side memory oversubscription. It explicitly governs the local CPU-side execution substrate— managing how and when sandboxes consume memory. LLM Serving and Sandbox Runtimes. Mainstream LLM serving engines maximize accelerator throughput via iterationlevel scheduling and memory management [3, 21, 52, 53, 55]. While essential for inference, they omit the CPU-side environments (e.g., reconstructable workspaces and tool outputs) required by agents [11]. To isolate these substrates, runtimes like Firecracker and gVisor provide high-density containment [2, 4, 7, 14, 15, 25]. For stateful workflows, tools like CRIU [6] and DeltaBox [12] employ OS-level checkpoint/restore (C/R) to freeze process states. However, these VMs and containers preserve the rigid identity of the execution instance. In contrast to traditional C/R mechanisms that rely on heavy memory-heap snapshots to preserve physical state, TenonAgent virtualizes execution residency at the semantic layer by decoupling the logical agent from its physical sandbox.

3000 2000 1000 0

Flash

(a) Task accuracy

Max

Flash

Max

(b) E2E time

Figure 15. Online evaluation on SWE-bench and ScienceAgentBench under low and moderate workload levels: (a) task accuracy and (b) end-to-end execution time, comparing eager execution (w/o TenonAgent) with TenonAgent (w/ TenonAgent). Table 3. Restart-safe coverage and teardown overhead in higher-load agent runs. Metric Actions Restartable boundaries (%) Teardowns Memory reclaimed (GiB) Cumulative overhead (s)

SWE-bench

ScienceAgentBench

1,952.8 93.4% 396.8 33.6 805.0

536.2 100% 124.4 8.6 530.4

OOM or timeout failures. Disabling proactive admission control allows unvetted memory growth, increasing E2E time by 16.7% and triggering an OOM event. Removing kernel-level memory-growth control is even more detrimental, increasing E2E time by 44.3% and causing three OOMs, as unexpected allocations during execution go unthrottled. Furthermore, an always resident policy—which disables restart-safe teardown—severely penalizes performance (+111.3% P50 latency). This confirms that static residency inevitably strands memory, whereas dynamic lifecycle decoupling is essential for surviving strict capacity limits. Efficacy of the Predictor. To isolate the value of TenonAgent’s contextual prediction, we evaluate two degraded baselines. A static predictor heavily overestimates memory demands, enforcing a conservative admission policy that starves the system: peak concurrency drops to 20 and memory remains underutilized (5414 MB). While this reduces OOMs, excessive queuing inflates P50 latency. Conversely, a historyonly predictor frequently misses sudden memory spikes. It admits actions too aggressively, plunging the system into severe memory contention that causes 2 OOMs and increases E2E time by 33.7% due to recovery overheads. Ultimately, TenonAgent’s full predictor achieves the optimal balance, precisely gauging demand to safely maximize concurrency (34) without triggering failures or inflating latency.

Discussion and Limitations

Scope of host-resource management. TenonAgent primarily targets host memory capacity, which is non-compressible and often determines failure under oversubscription. Lifecycle limits help bound reconstruction overhead, but TenonAgent does not explicitly model CPU contention, memory bandwidth, or application-level I/O. Although Appendix provides a preliminary I/O analysis, these factors may dominate before memory is exhausted in data-intensive workloads. Extending residency virtualization to multi-resource co-scheduling remains future work. Sensitivity to workflow dynamics. Residency virtualization relies on idle intervals during external LLM reasoning. If inference latency drops significantly, shrinking reclamation windows could cause sandbox reconstruction overheads to dominate. Furthermore, TenonAgent assumes observable action boundaries within dedicated sandboxes; supporting uncoordinated multi-tenant workloads would require finergrained intra-sandbox isolation.

Related Work 8

LLM Agents and Agentic Serving. Recent agent systems transform LLM calls into long-running, stateful workflows interacting with diverse external environments [28, 47–50, 56],

Conclusion

In this paper, we present TenonAgent, a runtime system that virtualizes sandbox execution residency by decoupling 12

Table 4. Ablation results under an 8 GB memory budget. Each variant removes one mechanism or replaces phase-level control with agent-level control while retaining the remaining mechanisms. Parentheses indicate latency increases relative to the full system. Out-of-memory and sandbox timeout columns report event counts. Configuration TenonAgent W/O admission control W/O memory-growth control Always resident W/ static predictor W/ history-only predictor

OOM events

Sandbox timeouts

End-to-end time (s)

P50 latency (s)

Peak concurrency

Peak memory (MB)

0 1 3 2 0 2

0 0 0 1 0 1

1276 1489 (+16.7%) 1841 (+44.3%) 1845 (+44.6%) 2441 (+91.3%) 1706 (+33.7%)

522 721 (+38.1%) 842 (+61.3%) 1103 (+111.3%) 1212 (+132.2%) 836 (+60.2%)

34 42 27 43 20 38

7415 7937 7978 8034 5414 7965

Seventeenth European Conference on Computer Systems (EuroSys). 730– 746. https://doi.org/10.1145/3492321.3524270 [8] The Kubernetes Authors. 2024. Kubernetes Documentation. https: //kubernetes.io/ [9] AutoGen. 2024. AutoGen. https://microsoft.github.io/autogen/stable/ /index.html. [10] Ziru Chen, Shijie Chen, Yuting Ning, Qianheng Zhang, Boshi Wang, Botao Yu, Yifei Li, Zeyi Liao, Chen Wei, Zitong Lu, Vishal Dey, Mingyi Xue, Frazier N. Baker, Benjamin Burns, Daniel Adu-Ampratwum, Xuhui Huang, Xia Ning, Song Gao, Yu Su, and Huan Sun. 2024. ScienceAgentBench: Toward Rigorous Assessment of Language Agents for Data-Driven Scientific Discovery. arXiv:2410.05080 [cs.CL] https://arxiv.org/abs/2410.05080 [11] Daixuan Cheng, Shaohan Huang, Yuxian Gu, Huatong Song, Guoxin Chen, Li Dong, Wayne Xin Zhao, Ji-Rong Wen, and Furu Wei. 2026. LLM-in-Sandbox Elicits General Agentic Intelligence. arXiv:2601.16206 [cs.AI] https://arxiv.org/abs/2601.16206 [12] Yunpeng Dong, Jingkai He, Yuze Hou, Dong Du, Zhonghu Xu, Si Yu, Yubin Xia, and Haibo Chen. 2026. DeltaBox: Scaling Stateful AI Agents with Millisecond-Level Sandbox Checkpoint/Rollback. arXiv:2605.22781 [cs.OS] https://arxiv.org/abs/2605.22781 [13] Dawei Gao, Zitao Li, Xuchen Pan, Weirui Kuang, Zhijian Ma, Bingchen Qian, Fei Wei, Wenhao Zhang, Yuexiang Xie, Daoyuan Chen, Liuyi Yao, Hongyi Peng, Zeyu Zhang, Lin Zhu, Chen Cheng, Hongzhu Shi, Yaliang Li, Bolin Ding, and Jingren Zhou. 2024. AgentScope: A Flexible yet Robust Multi-Agent Platform. arXiv:2402.14034 [cs.AI] https: //arxiv.org/abs/2402.14034 [14] Google. 2026. gVisor: Application Kernel for Containers. https://gvisor. dev/. [15] Google. 2026. NsJail: A Lightweight Process Isolation Tool for Linux. https://github.com/google/nsjail. [16] Juncheng Gu, Youngmoon Lee, Yiwen Zhang, Mosharaf Chowdhury, and Kang G. Shin. 2017. Efficient Memory Disaggregation with Infiniswap. In 14th USENIX Symposium on Networked Systems Design and Implementation (NSDI). 649–667. https://www.usenix.org/conference/ nsdi17/technical-sessions/presentation/gu [17] Eric Jonas, Johann Schleier-Smith, Vikram Sreekanti, Chia-Che Tsai, Anurag Khandelwal, Qifan Pu, Vaishaal Shankar, Joao Carreira, Karl Krauth, Neeraja Yadwadkar, et al. 2019. Cloud programming simplified: A berkeley view on serverless computing. arXiv preprint arXiv:1902.03383 (2019). [18] Hao Kang, Ziyang Li, Xinyu Yang, Weili Xu, Yinfang Chen, Junxiong Wang, Beidi Chen, Tushar Krishna, Chenfeng Xu, and Simran Arora. 2026. ThunderAgent: A Simple, Fast and Program-Aware Agentic Inference System. arXiv:2602.13692 [cs.OS] https://arxiv.org/abs/ 2602.13692 [19] Ehud Karpas, Omri Abend, Yonatan Belinkov, et al. 2022. MRKL Systems: A modular, neuro-symbolic architecture that combines large language models, external knowledge sources and discrete reasoning.

a logical agent’s lifetime from its underlying physical sandbox. By coordinating boundary-scale residency decisions with kernel-assisted execution pacing, TenonAgent reduces observed out-of-memory failures and delivers up to a 5.5× speedup over unmanaged execution under constrained host budgets. In our evaluated workloads and parameter settings, it achieves a 100% task success rate under bursts reaching an extreme 8.6× memory oversubscription ratio, showing that physical residency can be elasticized to improve high-density agent serving while preserving sandbox-based isolation. TenonAgent provides a practical foundation for memory-oversubscribed agentic AI infrastructure.

References [1] 2025. CrewAI — The Leading Multi-Agent Platform. https://www. crewai.com/. Accessed: 19 August 2025. [2] Alexandru Agache, Marc Brooker, Alexandra Iordache, Anthony Liguori, Rolf Neugebauer, Phil Piwonka, and Diana-Maria Popa. 2020. Firecracker: Lightweight Virtualization for Serverless Applications. In 17th USENIX Symposium on Networked Systems Design and Implementation (NSDI). 419–434. https://www.usenix.org/conference/nsdi20/ presentation/agache [3] Amey Agrawal, Nitin Kedia, Ashish Panwar, Jayashree Mohan, Nipun Kwatra, Bhargav S. Gulavani, Alexey Tumanov, and Ramachandran Ramjee. 2024. Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve. In 18th USENIX Symposium on Operating Systems Design and Implementation (OSDI). https://www.usenix.org/ conference/osdi24/presentation/agrawal [4] Istemi Ekin Akkus, Ruichuan Chen, Ivica Rimac, Manuel Stein, Klaus Satzke, Andre Beck, Paarijaat Aditya, and Volker Hilt. 2018. SAND: Towards High-Performance Serverless Computing. In 2018 USENIX Annual Technical Conference (USENIX ATC). 923–935. https://www. usenix.org/conference/atc18/presentation/akkus [5] Emmanuel Amaro, Christopher Branner-Augmon, Zhihong Luo, Amy Ousterhout, Marcos K. Aguilera, Aurojit Panda, Sylvia Ratnasamy, and Scott Shenker. 2020. Can Far Memory Improve Job Throughput?. In Proceedings of the Fifteenth European Conference on Computer Systems (EuroSys). https://doi.org/10.1145/3342195.3387522 [6] Fabio Andrijauskas, Igor Sfiligoi, Diego Davila, Aashay Arora, Jonathan Guiang, Brian Bockelman, Greg Thain, and Frank Wurthwein. 2024. CRIU – Checkpoint Restore in Userspace for Computational Simulations and Scientific Applications. arXiv:2402.05244 [cs.DC] https://arxiv.org/abs/2402.05244 [7] Lixiang Ao, George Porter, and Geoffrey M. Voelker. 2022. FaaSnap: FaaS Made Fast Using Snapshot-Based VMs. In Proceedings of the 13

arXiv preprint arXiv:2205.00445 (2022). [20] kvcache-ai. 2024. AgentENV: A distributed platform for running agent environments at scale. https://github.com/kvcache-ai/AgentENV. [21] Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient Memory Management for Large Language Model Serving with PagedAttention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles. [22] LangChain. 2024. LangChain. https://github.com/langchain-ai/ langchain. [23] LangGraph. 2024. LangGraph. https://www.langchain.com/ langgraph. [24] Hanchen Li, Qiuyang Mang, Runyuan He, Qizheng Zhang, Huanzhi Mao, Xiaokun Chen, Alvin Cheung, Joseph Gonzalez, and Ion Stoica. 2025. Continuum: Efficient and Robust Multi-Turn LLM Agent Scheduling with KV Cache Time-to-Live. arXiv:2511.02230 [cs.DC] https://arxiv.org/abs/2511.02230 [25] Zijun Li, Yushi Yang, Chen Xing, Jinhe Zhang, Wenbo Wang, Min Li, and Haibo Chen. 2022. RunD: A Lightweight Secure Container Runtime for High-density Deployment and High-concurrency Startup in Serverless Computing. In 2022 USENIX Annual Technical Conference (USENIX ATC). https://www.usenix.org/conference/atc22/ presentation/li-zijun [26] Chaofan Lin, Zhenhua Han, Chengruidong Zhang, Yuqing Yang, Fan Yang, Chen Chen, and Lili Qiu. 2024. Parrot: efficient serving of LLMbased applications with semantic variable. In Proceedings of the 18th USENIX Conference on Operating Systems Design and Implementation (Santa Clara, CA, USA) (OSDI’24). USENIX Association, USA, Article 50, 17 pages. [27] Jerry Liu. 2022. LlamaIndex. https://github.com/jerryjliu/llama_index. [28] Xiao Liu, Hao Yu, Hanchen Zhang, Yifan Xu, Xuanyu Lei, Hanyu Lai, Gu Yu, Hangliang Ding, Kaiwen Men, Kejuan Yang, Shudan Zhang, Xiang Deng, Aohan Zeng, Zhengxiao Du, Chenhui Zhang, Sheng Shen, Tianjun Zhang, Yu Su, Huan Sun, Minlie Huang, Yuxiao Dong, and Jie Tang. 2023. AgentBench: Evaluating LLMs as Agents. arXiv:2308.03688 [cs.CL] https://arxiv.org/abs/2308.03688 [29] Michael Luo, Xiaoxiang Shi, Colin Cai, Tianjun Zhang, Justin Wong, Yichuan Wang, Chi Wang, Yanping Huang, Zhifeng Chen, Joseph E. Gonzalez, and Ion Stoica. 2025. Autellix: An Efficient Serving Engine for LLM Agents as General Programs. arXiv:2502.13965 [cs.LG] https://arxiv.org/abs/2502.13965 [30] Ian Miell and Aidan Sayers. 2019. Docker in practice. Simon and Schuster. [31] Authors Omitted. 2025. A CPU-Centric Perspective on Agentic AI. arXiv:2511.00739 [cs.DC] https://arxiv.org/abs/2511.00739 [32] Authors Omitted. 2026. Efficient LLM Serving for Agentic Workflows: A Data Systems Perspective. Proceedings of the ACM on Management of Data (2026). https://doi.org/10.1145/3802046 [33] OpenAI. 2023. Function Calling Guide. https://platform.openai.com/ docs/guides/function-calling [34] Baolin Peng, Wenlin Yao, Qianhui Wu, Hao Cheng, Xiao Yu, Rui Yang, Tao Ge, Alessandro Sordoni, Xingdi Yuan, Yelong Shen, Pengcheng He, Tong Zhang, Zhou Yu, and Jianfeng Gao. 2026. Orchard: An OpenSource Agentic Modeling Framework. arXiv preprint arXiv:2605.15040 (2026). https://arxiv.org/abs/2605.15040 [35] Qianwen AI Platform. [n.d.]. Qwen3.8-Flash. https://www.qianwenai. com/models/qwen3.8-flash. Model page. Accessed: 2026-09-25. [36] Qianwen AI Platform. [n.d.]. Qwen3.8-Max-0902. https://www. qianwenai.com/models/qwen3.8-max-0902. Model page. Accessed: 2026-09-25. [37] Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian, et al. 2023. ToolLLM: Facilitating large language models to master 16000+ real-world APIs. arXiv preprint arXiv:2307.16789 (2023).

[38] Deepti Raghavan, Keshav Santhanam, Muhammad Shahir Rahman, Nayani Modugula, Luis Gaspar Schroeder, Maximilien Cura, Houjun Liu, Pratiksha Thaker, Philip Levis, and Matei Zaharia. 2025. Alto: Orchestrating Distributed Compound AI Systems with Nested Ancestry. arXiv:2403.04311 [cs.AI] https://arxiv.org/abs/2403.04311 [39] Rami Rosen. 2013. Resource management: Linux kernel namespaces and cgroups. Haifux, May 186 (2013), 70. [40] Rami Rosen. 2016. Namespaces and cgroups, the basis of Linux containers. Seville, Spain, Feb (2016). [41] Zhenyuan Ruan, Malte Schwarzkopf, Marcos K. Aguilera, and Adam Belay. 2020. AIFM: High-Performance, Application-Integrated Far Memory. In 14th USENIX Symposium on Operating Systems Design and Implementation (OSDI). 315–332. https://www.usenix.org/conference/ osdi20/presentation/ruan [42] Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. 2023. Toolformer: Language Models Can Teach Themselves to Use Tools. arXiv preprint arXiv:2302.04761 (2023). [43] Yongliang Shen, Kaitao Song, Xu Tan, Dongsheng Li, Weiming Lu, and Yueting Zhuang. 2023. HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face. arXiv preprint arXiv:2303.17580 (2023). [44] Shangyin Tan, Lakshya A Agrawal, Rohit Sandadi, Dan Klein, Koushik Sen, Alexandros G Dimakis, and Matei Zaharia. 2026. Automatically Learning Skills for Coding Agents. In Proceedings of the ACM Conference on AI and Agentic Systems. 1310–1315. [45] The Linux Kernel Documentation. 2026. PSI: Pressure Stall Information. https://docs.kernel.org/accounting/psi.html. [46] Johannes Weiner, Niket Agarwal, Dan Schatzberg, Leon Yang, Hao Wang, Blaise Sanouillet, Bikash Sharma, Tejun Heo, Mayank Jain, Chunqiang Tang, and Dimitrios Skarlatos. 2022. TMO: Transparent Memory Offloading in Datacenters. In Proceedings of the 27th ACM International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS). 609–621. https://doi.org/ 10.1145/3503222.3507731 [47] Tianbao Xie, Danyang Zhang, Jixuan Chen, Xiaochuan Li, Siheng Zhao, Ruisheng Cao, Toh Jing Hua, Zhoujun Cheng, Dongchan Shin, Fangyu Lei, Yitao Liu, Yiheng Xu, Shuyan Zhou, Silvio Savarese, Caiming Xiong, Victor Zhong, and Tao Yu. 2024. OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments. arXiv:2404.07972 [cs.AI] https://arxiv.org/abs/2404.07972 [48] Tianqi Xu, Linyao Chen, Dai-Jie Wu, Yanjun Chen, Zecheng Zhang, Xiang Yao, Zhiqiang Xie, Yongchao Chen, Shilong Liu, Bochen Qian, Anjie Yang, Zhaoxuan Jin, Jianbo Deng, Philip Torr, Bernard Ghanem, and Guohao Li. 2024. CRAB: Cross-environment Agent Benchmark for Multimodal Language Model Agents. arXiv:2407.01511 [cs.AI] https://arxiv.org/abs/2407.01511 [49] John Yang, Carlos E. Jimenez, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik Narasimhan, and Ofir Press. 2024. SWE-agent: AgentComputer Interfaces Enable Automated Software Engineering. In Advances in Neural Information Processing Systems. https://arxiv.org/ abs/2405.15793 [50] Shunyu Yao, Howard Chen, John Yang, and Karthik Narasimhan. 2022. WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents. In Advances in Neural Information Processing Systems. https://arxiv.org/abs/2207.01206 [51] Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2023. ReAct: Synergizing Reasoning and Acting in Language Models. arXiv:2210.03629 [cs.CL] https: //arxiv.org/abs/2210.03629 [52] Gyeong-In Yu, Joo Seong Jeong, Geon-Woo Kim, Soojeong Kim, and Byung-Gon Chun. 2022. Orca: A Distributed Serving System for Transformer-Based Generative Models. In 16th USENIX Symposium on Operating Systems Design and Implementation (OSDI). 521–538. 14

https://www.usenix.org/conference/osdi22/presentation/yu [53] Lianmin Zheng, Liangsheng Yin, Zhiqiang Xie, Chuyue Sun, Jeff Huang, Cody Hao Yu, Shiyi Cao, Christos Kozyrakis, Ion Stoica, Joseph E. Gonzalez, Clark Barrett, and Ying Sheng. 2024. SGLang: Efficient Execution of Structured Language Model Programs. arXiv:2312.07104 [cs.AI] https://arxiv.org/abs/2312.07104 [54] Yusheng Zheng, Jiakun Fan, Quanzhi Fu, Yiwei Yang, Wei Zhang, and Andi Quinn. 2026. Agentcgroup: Understanding and controlling os resources of ai agents. arXiv preprint arXiv:2602.09345 (2026). [55] Yinmin Zhong, Shengyu Liu, Junda Chen, Jianbo Hu, Yibo Zhu, Xuanzhe Liu, Xin Jin, and Hao Zhang. 2024. DistServe: Disaggregating Prefill and Decoding for Goodput-Optimized Large Language Model Serving. In 18th USENIX Symposium on Operating Systems Design and Implementation (OSDI). https://www.usenix.org/conference/osdi24/ presentation/zhong-yinmin [56] Shuyan Zhou, Frank F. Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Sridhar, Xianyi Cheng, Yonatan Bisk, Daniel Fried, Uri Alon, and Graham Neubig. 2024. WebArena: A Realistic Web Environment for Building Autonomous Agents. arXiv:2307.13854 [cs.AI] https: //arxiv.org/abs/2307.13854

Success rate

95%

1600

1200

accompanying definitions determine the decision order; this table supplies parameter values rather than an alternative policy.

100%

Admission window (GB)

Success rate

E2E time (s)

E2E time 2100 2000

90%

Lifecycle concurrency

Figure 16. Sensitivity to (a) the upper bound of the computed admission window and (b) the lifecycle concurrency limit. Each parameter is varied independently while all other mechanisms remain enabled.

Parameter Sweep

Figure 16 evaluates two parameters of the user-space boundaryscale scheduler. We vary one parameter at a time while keeping all other mechanisms enabled. The reference configuration uses an 8 GB upper bound for the computed admission window and a lifecycle concurrency limit of two. Under this configuration, the workload completes in 1276 s without OOM or sandbox-timeout events. Admission-window upper bound. The admission window is computed from the predicted resource demand of upcoming actions, as described in §3.4. The parameter studied here is the upper bound imposed on this computed window, rather than a fixed reservation assigned uniformly to every action. When the upper bound is below 8 GB, admission becomes overly conservative and limits useful overlap among actions, resulting in longer completion time. Increasing the upper bound beyond 8 GB does not improve completion time and eventually reduces the observed success rate because more predicted memory demand is allowed to accumulate concurrently. The 8 GB setting therefore provides the best balance between execution concurrency and protection against concurrent memory pressure for this workload.

Controller initialization. The initial permitted set is empty, the initial concurrency budget is 𝑏 0 , and the previous pressure is initialized to Green for pressure-transition bookkeeping. The Green stability timer starts when the first Green decision is made and is cleared whenever pressure is non-Green. While pressure remains Green, each completed 𝑇stable interval increments 𝑏 once. If a delayed control iteration spans multiple complete stability intervals, the controller applies one increment per complete interval, capped at 𝑏 max , and retains the remaining fractional interval. A pressure increase applies multiplicative decrease once per observed transition, not once per control iteration. A decrease in pressure does not directly change 𝑏.

Lifecycle concurrency limit. Sandbox start, stop, and rematerialization operations involve substantial I/O and share limited bandwidth and host-side resources. The lifecycle concurrency limit therefore serves as a rate limiter, bounding the number of lifecycle transitions that may be active simultaneously. A limit of two achieves the lowest E2E completion time in the sweep. Increasing the limit beyond two creates additional overlap but does not increase effective lifecycle throughput; instead, contention grows and completion time increases. Higher limits also introduce sandboxtimeout events. These results indicate that lifecycle parallelism should be bounded by the available I/O bandwidth and related host resources, rather than maximized independently of the underlying service capacity.

Control-iteration order. Each control iteration performs the following operations: (1) remove completed sandboxes and register newly observed ones; (2) collect footprint and CPU-time observations; (3) update estimates and states only for valid sample intervals; (4) compute projected demands and pressure; (5) update the concurrency budget; (6) select the complete permitted set; and (7) publish permissions and update denial episodes. All decisions within an iteration use the same observation snapshot and control timestamp. A completed sandbox is removed from all candidate and permitted sets, and its estimator and denial timer are discarded. A newly registered sandbox has no valid growth estimate, starts in Growing, has no active denial timer, and uses 𝑑 boot until its first valid sample. Its execution permission is initially deny until the controller publishes its first decision. If that decision is deny, the denial timer starts at that iteration.

Runtime Control Parameters

Table 5 gives the default configuration for the runtime executioncontrol specification in Section 3.5. The algorithm and its 16

Measurement validity. The footprint baseline is refreshed at every sample, including samples collected while execution is denied. Consequently, the first subsequent valid growth sample uses the immediately preceding footprint observation rather than accumulating changes over the entire denial episode. An interval containing a policy-permission change, or having no increase in accounted CPU time, is invalid for updating the growth estimate. Such an interval does not supply evidence of reduced growth. There is no separate inactive-footprint cutoff and no additional re-trust timer. The near-zero band affects the growthstate label, not headroom accounting: any strictly positive predicted demand consumes a contributor slot and is included in the aggregate projection.

Table 5. Default parameters for the runtime executioncontrol specification. Runtime growth accounting is separate from action-level admission accounting. Parameter

boundaries. For tasks categorized as “High,” the memory.high limit is left unbounded, allowing them to freely consume available memory. Conversely, tasks classified as “Low” are subjected to a rigid, hardcoded memory.high threshold (e.g., set to 400 MB in our evaluation). Through extensive parameter tuning, we found that this static categorization fundamentally fails to generalize across the dynamic, multi-turn nature of agentic workflows. The severe latency inflation stems from the OS-level penalty of misconfiguration, which forces a binary dilemma:

Default value / definition

𝑚𝑐 = memory.current𝑐 + memory.swap.current𝑐 . Sampling interval Δ 100 ms. Controller period 100 ms. Scheduler update polling 100 ms; not a hard bound on actuation latency. Projection horizon 𝑇 1000 ms. EWMA factor 𝛼 0.35; the first valid raw sample initializes the estimate. Resident band 𝜖 4 MiB/s; Resident iff |𝑔ˆ𝑐 | ≤ 𝜖. Bootstrap demand 𝑑 boot 16 MiB over the projection horizon, used only until a valid growth estimate exists. Safety margin 𝑆 256 MiB. Usable headroom 𝐻 max(MemAvailable − 𝑆, 0). Green rule Allow every live sandbox. Growth-window, contributor-count, and denial-waiting restrictions are bypassed. Yellow budget 𝑏 0 = 1, 𝑏 max = 4. At most 𝑏 positive-demand sandboxes may be permitted, with aggregate demand at most 𝐻 . Multiplicative decrease 𝑏 ← max(1, ⌊𝑏/2⌋ ) once on each observed increase in pressure severity. Additive increase Increment 𝑏 by one per 𝑇stable = 5000 ms of continuous Green operation, capped at 𝑏 max . Red-drain budget At most one positive-demand sandbox in the complete permitted set; aggregate demand must remain at most 𝐻 . Minimum denial wait 1000 ms before a denied sandbox is eligible for 𝐷 min admission in Yellow or Red-drain. Green overrides it. Overdue threshold 𝐷 max 3000 ms. Eligible sandboxes reaching this duration are checked before non-overdue candidates; all admission constraints still apply. State preference For non-overdue candidates: Resident, then Cooling, then Growing/PausedGrowing; smaller projected demand and stable identifier break ties. Pressure hysteresis No separate threshold hysteresis. Denial waiting and the Green stability timer have the distinct roles defined in the text. Footprint

• Threshold Too High (Under-throttling): If the “Low” threshold is relaxed to accommodate diverse tasks, it effectively neutralizes the flow control. The system acts similarly to the Eager baseline, failing to prevent overlapping memory spikes and ultimately crashing under tight global memory budgets. • Threshold Too Low (Severe Thrashing): When a restrictive threshold (e.g., 400 MB) is strictly enforced, any agent operation (such as loading a dataset or compiling code) that legitimately requires more memory will repeatedly hit the memory.high limit. Unlike memory.max (which triggers the OOM killer), hitting memory.high forces the kernel to synchronously throttle the offending processes. The kernel aggressively forces these tasks into direct memory reclaim, spending CPU cycles on continuously swapping pages in and out. Consequently, the agent’s tool execution essentially stalls. During our evaluation, we empirically observed severe memory I/O saturation and sustained page-fault storms (thrashing) coinciding with these stalls, as tasks advanced at a glacial pace due to constant block I/O wait and kernel reclaim overhead. This confirms that static, configuration-heavy memory.high thresholds are fundamentally ill-suited for the unpredictable phase transitions of LLM agents. This fundamental limitation of reactive cgroup mechanisms directly motivated TenonAgent’s architectural shift to sched_ext. While cgroups manage memory capacity reactively—punishing workloads with severe thrashing or OOM kills upon hitting boundaries—TenonAgent leverages sched_ext to proactively govern the rate of memory growth. By intercepting CPU scheduling, TenonAgent smoothly paces the execution of aggressively allocating sandboxes before they trigger kernel-level memory reclaim. This approach completely bypasses the binary dilemma of cgroups, effectively preventing page-fault storms while maintaining safe, high-density concurrent execution.

Root-Cause Analysis of AgentCgroup Degradation

As observed in Section 5.2, the AgentCgroup baseline prevents catastrophic out-of-memory (OOM) crashes but suffers from extreme end-to-end time inflation (e.g., extending ScienceAgentBench makespan to 293.4 minutes under 4U8G). This section details the system-level mechanisms driving this severe degradation. The core design of AgentCgroup relies on pre-execution profiling to statically classify tasks into “High” and “Low” memory demand categories based on a configurable ratio. It then enforces these classifications using Linux cgroup v2

D Restart-Safe Recovery under a Restricted Execution Model This appendix specifies the contract for safely releasing and reconstructing sandboxes at action boundaries. Unlike OSlevel checkpointing (e.g., CRIU), this mechanism operates 17

semantically: it captures explicit persistent state between actions rather than freezing arbitrary running processes. D.1

eligibility checks fail), the teardown is aborted and the sandbox safely remains resident_only. • After Teardown: Once the sandbox is destroyed, fallback to the original instance is impossible. If rematerialization fails, the controller must block execution and retry reconstruction from the committed durable record, rather than silently continuing with an incomplete state.

Supported Execution Model & Eligibility

TenonAgent evaluates recoverability strictly between actions. To qualify for restart-safe teardown, a sandbox must satisfy the following execution constraints: • Decoupled Logical State: The controller manages all workflow progress, LLM context, and conversation history. This state must not depend on sandbox memory. • Stateless Action Processes: Each action runs as a fresh process. Subsequent actions cannot rely on active background daemons, open sockets, interactive shell sessions, or live interpreter heaps created by previous actions. • Explicit Filesystem Persistence: All cross-action mutable state must be confined to designated persistent directories. The rest of the environment is considered disposable and reconstructible from a base image. • Strict Termination: The current action and all its descendant processes must have completely terminated, with their outcomes fully recorded. If a workflow violates these constraints (e.g., spawns a required background server), the controller conservatively aborts recovery and keeps the sandbox resident. D.2

Persistence and Teardown Protocol

When a sandbox is marked for dematerialization, the controller executes a strictly ordered teardown protocol: 1. Block Dispatch & Validate: Suspend new action dispatch and verify that the sandbox meets all eligibility constraints mentioned above. 2. Capture Artifacts: Export all files within the designated persistent directories to durable storage. 3. Commit Recovery Record: Generate and commit a recovery record to controller-managed storage. This record includes workflow IDs, execution generation, references to the persisted artifacts, and the base environment configuration. 4. Authorize Teardown: The sandbox is destroyed only after the recovery record is successfully committed. D.3

Rematerialization and Failure Handling

To rematerialize a sandbox, the controller provisions a fresh instance from the recorded base environment, injects the committed filesystem artifacts, and verifies state integrity before dispatching the next action. Failure Semantics: • Before Teardown: If any step fails prior to committing the recovery record (e.g., artifact upload fails or 18

TenonAgent: Virtualizing Execution Residency for Memory-Oversubscribed LLM Agents

Analyze & explore related research in Conceptio

Related documents