Serve Now or Improve Later? Scheduling Self-Evolution in Online Agent Systems

Serve Now or Improve Later? Scheduling Self-Evolution in Online Agent Systems Yangbo Wei1,2

Junhong Qian1

Zhen Huang4 Zhenyu Su1,2 Qifan Wang1,2 Chen Wu3,* Lei He2,*

Shaoqiang Lu1,2

Rumin Zhang3

1

Shanghai Jiao Tong University Eastern Institute of Technology 3 Ningbo Institute of Digital Twin, Eastern Institute of Technology 4 University of Science and Technology of China 2

arXiv:2610.06212v1 [cs.DC] 5 Oct 2026

Abstract

Online agents can improve future service by constructing reusable tools, guidance, or model states, but this work competes with current requests for the same GPUs. Exploiting idle compute for self-evolution faces a fundamental systems constraint: benefits arrive only after an artifact is published and used, while pausing evolution leaves service capacity waiting for memory release and runtime recovery. An investment worth completing may therefore be worth postponing. We present LearnSched, a state-aware scheduler that brings the reuse window of a capability and the timely return of compute into a common investment model. LearnSched incorporates candidate progress, checkpoint overhead, and the current recovery path into action values. Under the same information and capacity constraints, it uses one-step counterfactual rollout to choose progress, checkpointing, or waiting relative to a fully costed window policy. We characterize handoff costs through independent A100 component measurements and evaluate action selection in 1920 paired finite-model scenarios. When cold recovery is expensive, waiting for longer execution windows can improve net value by avoiding handoffs; in evaluated warm-recovery scenarios, the strong window policy already achieves the same value. Retaining recoverable state can shorten capacity return and reduce the need for complex evolution scheduling. Keywords: online agents, self-evolution, resource scheduling, compute investment, capacity readiness, preemption

1

Introduction

An online coding agent repeatedly fails on the same class of requests. It can continue invoking a model for each request, or use prior trajectories to construct a reusable tool or guidance, validate it, and publish it for later use. Persistent demand can repay this investment through fewer model calls and better answers; short-lived demand may leave even a qualified artifact unused. When service and evolution share GPUs, construction also consumes compute and memory needed by current requests. The system must decide: should the next unit of compute serve now or improve later?

Figure 1. Serving and self-evolution compete for shared GPU capacity. LoRA adaptation illustrates the tradeoff: invest in future capability while returning compute in time to meet current request deadlines. Figure 1 illustrates this choice through LoRA adaptation: improving the model incurs training and recovery costs before future requests can benefit. This decision couples two timescales. Foreground requests consume resources under latency deadlines, while capability construction incurs costs now for benefits spread across future requests. A useful artifact can reduce service work, but only after qualification, publication, and actual use. Its benefit cannot compensate for requests that have already missed their deadlines. The scheduling problem therefore involves both the value of the investment and the service capacity available along its execution path. Running evolution during idle periods and pausing it at the next burst is a natural starting point. Yet an idle interval has two distinct boundaries: the latest start that leaves enough reuse to amortize construction, and the latest return that restores capacity before service needs it. A pause command may leave device memory occupied; releasing memory may still leave a serving runtime to reconstruct. Stopping evolution, reclaiming memory, and making capacity ready to serve are distinct system events. Treating the end of background computation as the return of service capacity overstates the useful part of an execution window.

(a) Capacity cliff

(b) State-dependent return

Bad requests (%) MoE/SG

0

0

1.9

37.9

MoE/vL

0

0

2.8

83.9

Dense/SG

0

0.6

10.1

78.1

Dense/vL

0

0

21.6

84.5

0% 10% 25% 50% Static background duty 48 runs · 3-seed strips

(c) Cost reversal

Ready latency · seconds (log)

Tool / no evolution (×)

69.99s

Cold 0.487s

Warm

1

1.86×

2 10

100

Host PSS breakdown · 11.158 GiB

1

0.26× 4.775

+6.383

Base

Added when parked

GPU retained: 3.74 GiB SGLang · 3 measured runs

0

Token

GPU

Token-based tool policy 4 paired blocks

Figure 2. Three investment constraints: (a) bad-request rates across four systems and four shares (three repeats; SG/vL: SGLang/vLLM); (b) cold/warm readiness means and min–max, with host PSS and GPU residency separate; (c) token and GPU cost ratios to no evolution, across four paired blocks. GPU activity excludes CPU costs. Runtime state connects these boundaries. Termination discards unsaved progress, checkpointing consumes time, and retaining service state trades memory for faster recovery. Consider a short idle window followed by one long enough to complete construction. Immediate progress can publish the artifact slightly earlier but incur an extra checkpoint and cold recovery. Waiting pays when the avoided handoff costs exceed the benefits lost to delay. Cheap warm recovery can reverse this choice without changing the candidate or future demand. Thus, an investment worth completing need not be worth starting now. Expected future benefits have long guided adaptive compilation [2], inference and retraining allocation [4, 11], and agent learning during idle windows [21]. Evolution control planes also incorporate cost signals [24]. We study the interaction between this investment perspective and the runtime mechanisms that preserve progress and restore service. With the same candidate, information, and execution mechanisms, when do durable progress and recovery costs change the best execution plan? Answering this question requires separating gains from better accounting, faster recovery, and different scheduling actions. We present LearnSched, a state-aware scheduler that compares remaining benefits and execution costs under a capacity-return constraint. Its model connects the useful lifetime of a capability to completed and durable progress, current service state, and the recovery path. These quantities determine whether to advance, checkpoint and return, discard unsaved work, or wait. Execution components expose work pressure, publication and use, and capacity readiness to account for costs and benefits as they occur.

LearnSched uses classical one-step rollout [3] relative to Costed Window, a reference that fully prices a greedy completion plan. Both receive the same windows and recovery mechanisms. Comparing alternative continuations allows LearnSched to skip a costly short window or change execution segments when this improves value; retaining the reference action guarantees no loss in the accurate finite model. This formulation makes the added value of scheduling measurable against an already cost-aware reference. Our evaluation separates independent A100 component measurements and capability ledgers from paired finitemodel scheduling. Across 1920 scenarios, full cost accounting explains most of the benefit. Waiting adds value when cold recovery is expensive and longer windows lie ahead; all evaluated warm-recovery scenarios tie with Costed Window. The resulting systems implication is that state retention changes both return latency and the amount of value available to more complex planning. Runtime memory and investment policies thus require joint evaluation. Our contributions are:

• A model connecting capability break-even conditions, durable progress, and state-dependent capacity return to explain when waiting improves net value. • A scheduler that compares progress and waiting under common constraints, with finite-model policy improvement and an explicit scoring-error condition. • Component measurements, capability ledgers, and paired evaluation isolating accounting and scheduling gains and their dependence on recovery state.

2

Why Timing and State Matter

3

Delayed Returns and Switching Costs

Figure 2 examines three constraints on using shared GPUs for evolution. Positive expected return and available idle time alone do not determine the execution plan.

We model one candidate capability on shared devices. Actions change progress, publication time, and recovery state; request deadlines constrain capacity return.

Capacity must be available when requests need it. Raising the static background share from 10% to 25% increases bad requests across all four systems; at 50%, mean rates reach 37.88–84.48% (Figure 2(a)). An average resource budget hides the periods when foreground demand and background execution overlap. A profitable capability can therefore degrade service before its benefits arrive. The scheduler needs to control the timing of resource use as well as its volume: capacity borrowed under low load must be returned before the next service deadline, and future work savings cannot be credited before the artifact is used.

3.1

Runtime state changes the usable window. Cold service readiness takes 69.99 s on average, whereas sameprocess warm recovery takes 0.487 s (Figure 2(b)). These paths leave substantially different amounts of an idle window for useful evolution work. Segmentation compounds the difference: each return can incur recovery, and interrupted construction may require checkpointing or recomputation. Two plans that complete the same amount of work can consequently have different total costs and publication times. Warm recovery also raises host proportional set size (PSS) from 4.77 to 11.16 GiB and retains 3.74 GiB on the device. Keeping state buys faster return by occupying memory that other work could use. Plan comparisons must jointly price recovery latency, durable progress, and residency; a fixed pause overhead cannot capture their tradeoff. The constrained resource determines the cost of reuse. Under sustained tool demand, token-based amortization reduces token cost to 0.2624 times that of no evolution but raises GPU active time to 1.8623 times (Figure 2(c)). Construction and the service calls it replaces have different device costs per token. Counting avoided calls or tokens can therefore reverse the resource-cost ranking. This observation is a cost comparison; quality gains and CPU execution still matter to application utility. Likewise, publishing more artifacts creates value only when subsequent requests use them. Investment accounting must connect construction to realized use and price both in the constrained resource. These observations motivate a joint decision over a candidate, an execution interval, and a return path. Starting immediately captures earlier benefits, while waiting can avoid checkpointing and repeated recovery. In the paired finitemodel example of Section 6.1, delaying publication by 4.952 s saves 70.104 s-eq. in execution and handoff costs under the same candidate and calendar. The relevant comparison is between complete feasible plans, including their return paths and the benefits forgone while waiting.

State and Continuation Value

At time 𝑡 , the state is

𝑠𝑡 = (𝑚𝑡 , 𝑝𝑡 , 𝑝𝑡̄ , 𝑧𝑡 , 𝑞𝑡 ),

0 ≤ 𝑝𝑡̄ ≤ 𝑝𝑡 .

(1)

Here, 𝑚𝑡 is service or evolution mode, 𝑝𝑡 is completed progress, 𝑝𝑡̄ is durable progress, 𝑧𝑡 identifies resident service state and its recovery path, and 𝑞𝑡 is foreground backlog. Termination loses 𝑝𝑡 − 𝑝𝑡̄ ; checkpointing preserves it. Previously paid compute is a sunk cost. Information ℐ𝑡 supports estimates of future demand, costs, and transitions. Actions wait, advance work, checkpoint and return, or discard unsaved work and return. Action 𝑎 takes 𝜏 (𝑠, 𝑎); 𝒜ret (𝑠, ℐ𝑡 ) contains actions meeting the capacity-return constraint. With benefit 𝑟 and cost 𝑐 in a common unit,

𝑉𝑡∗ (𝑠) = max {𝑟(𝑠, 𝑎) − 𝑐(𝑠, 𝑎)

𝑎∈𝒜ret ∗ ′ + 𝔼[𝑉𝑡+𝜏 (𝑠,𝑎) (𝑠 ) ∣ 𝑠, 𝑎, ℐ𝑡 ]}.

(2)

Benefits arise after publication and use. Cost 𝑐 charges only the current action’s execution, checkpointing, recovery, and residency; later costs enter continuation value. In the deterministic single-candidate instance, with no execution choices after publication at 𝑡𝑝 , the remaining benefit stream 𝐻

can be settled as terminal value Φ(𝑡𝑝 ) = ∫𝑡 𝑔(𝑣) 𝑑𝑣 . The 𝑝 publication action still pays its return cost. 3.2

Effective Reuse and Break-Even

Fix a continuous path that qualifies and publishes after 𝜏𝑒 , at remaining cost 𝐶𝑒 > 0. Demand ends at 𝐻 ; matching requests arrive at rate 𝜆, fraction 𝑎 use the artifact, and each use has expected net value 𝑢 relative to base service. Then

𝑉 (𝑡) = 𝑀(𝑡)𝑢 − 𝐶𝑒 ,

𝑀(𝑡) = 𝜆𝑎 (𝐻 − 𝑡 − 𝜏𝑒 )+ .

(3)

For 𝜆𝑎𝑢 > 0, positive direct return requires

𝐶𝑒 . (4) 𝜆𝑎𝑢 The deadline reserves time for both completion and amortization. Nonpositive unit value, absent reuse, or a start at or beyond this deadline prevents positive direct return along the fixed path. Changing segments or recovery paths requires evaluating their full continuation costs. For uncertain qualification, publication, and use, let 𝑌𝑖 indicate their joint occurrence for request 𝑖, with use value 𝑢𝑖 : 𝑡 < 𝑡𝑒∗ = 𝐻 − 𝜏𝑒 −

𝔼[𝑉 ∣ ℐ𝑡 ] = ∑ Pr(𝑌𝑖 = 1 ∣ ℐ𝑡 )𝔼[𝑢𝑖 ∣ 𝑌𝑖 = 1, ℐ𝑡 ]−𝔼[𝐶𝑒 ∣ ℐ𝑡 ]. 𝑖

(5)

The joint event retains correlations between completion time and remaining use opportunities. The cost term includes successful and failed construction paths. New observations update this estimate at stage boundaries. 3.3

Request Deadlines and Lendable Capacity

Consider an initially empty, work-conserving FIFO fluid queue with one resource [13]. Request 𝑖 arrives at 𝑎𝑖 , needs work 𝑤𝑖 > 0, and has deadline 𝑑𝑖 ≥ 𝑎𝑖 ; service rate is 𝑐cap (𝑣). Timely completion requires and is guaranteed by 𝑖

𝑑

∑𝑗=𝑘 𝑤𝑗 ≤ ∫𝑎 𝑖 𝑐cap (𝑣) 𝑑𝑣 for every 𝑘 ≤ 𝑖. Relative to base 𝑘 service on the same arrivals, define 𝑆𝑘𝑖 as baseline capacity minus work over this interval, 𝑅𝑘𝑖 as work actually avoided through capability use, and 𝐵𝑘𝑖 as capacity lost to contention and handoff. Equivalently, 𝐵𝑘𝑖 ≤ 𝑆𝑘𝑖 + 𝑅𝑘𝑖 ,

∀𝑘 ≤ 𝑖.

(6)

Future benefits add no current capacity; use that adds work makes 𝑅𝑘𝑖 negative. Stopping a candidate reduces 𝐵𝑘𝑖 only when capacity returns to service. Our component approximates pressure through in-flight work; batching and CPU queues need further modeling, and unknown arrivals preclude a hard deadline guarantee. 3.4

State-Dependent Return Deadlines

If additional serving capacity must be ready at 𝑡𝑏 , a serial return path takes

𝜏𝑟 = 𝜏act + 𝜏release + 𝜏ready .

(7)

The terms denote command actuation, resource release, and runtime readiness. Overlapping steps follow the critical path; residency can avoid runtime reconstruction. With deterministic delay,

𝑡reclaim ≤ 𝑡𝑏 − 𝜏𝑟 .

(8)

Checkpointing further reduces execution time; discarding work makes later execution resume from durable progress. Recovery state therefore changes both feasible actions and their costs. Under uncertain bursts, early return trades progress against avoidable service loss, as the two-action model in Appendix C shows. 3.5

Common Pricing

If a policy gains Δ𝑄timely correct answers on time at additional cost Δ𝐶𝑚 in resource unit 𝑚, its incremental utility is Δ𝑈 (𝑊 ) = 𝑊 Δ𝑄timely − Δ𝐶𝑚 (9) Here, 𝑊 is the application value of a timely correct answer, in 𝑚/answer. Tokens, GPU active time, and host memory are measured separately and combined only through explicit prices. For background steps uncalibrated to capability benefits, we report the progress–bad-request frontier directly. Model costs use equivalent evolution seconds (s-eq.).

4

LearnSched: State-Aware Investment Scheduling

LearnSched compares progress, checkpointing, and waiting through their continuation values. Execution components estimate reuse, bound in-flight work, and manage handoff. Figure 3 connects the online control loop, rollout decision, and state-dependent return path. 4.1

Window Actions and the Reference

Known idle windows [ℓ𝑘 , 𝑑𝑘 ] define decision stages with state (𝑘, 𝑝, 𝑝,̄ 𝑚; 𝑧). A macro-action advances 𝑢 quanta of duration 𝛿 and optionally checkpoints incomplete work, subject to

𝜏load + 𝑢𝛿 + 1save 𝜏ckpt + 𝜏return ≤ 𝑑𝑘 − ℓ𝑘 .

(10)

Checkpointing and recovery consume exact wall-clock time. The next window starts in service mode; unsaved work is discarded, and published artifacts leave construction. Waiting in service mode adds no recovery cost. All policies share windows, benefits, and execution mechanisms. Window advances as far as feasible and checkpoints. Costed Window (𝜋0 ) prices all remaining benefits and costs of that greedy completion plan against abandoning the candidate and returning capacity. It executes if completion has greater value; otherwise, it waits and reevaluates. Static ROI omits handoff costs from valuation only. 4.2

Rollout and Policy Improvement

LearnSched scores each safe current action by following the reference thereafter [3]:

𝑄 𝜋0 (𝑠, 𝑎) = 𝑟(𝑠, 𝑎) − 𝑐(𝑠, 𝑎) + 𝑉 𝜋0 (𝑠 ′ ), 𝜋𝑅 (𝑠) ∈ arg max 𝑄 𝜋0 (𝑠, 𝑎).

(11)

𝑎

The candidate set includes 𝜋0 (𝑠), retains it on ties, and is recomputed at each window. Proposition 1: Policy improvement. With finitely many stages, accurate transitions, costs, and reference values, and a safe 𝜋0 (𝑠), every reachable state satisfies

𝑉 𝜋𝑅 (𝑠) ≥ 𝑉 𝜋0 (𝑠).

(12)

Induction on remaining stages gives 𝑉 𝜋𝑅 (𝑠) ≥ 𝑄 𝜋0 (𝑠, 𝜋𝑅 ) ≥ 𝑄 𝜋0 (𝑠, 𝜋0 ) = 𝑉 𝜋0 (𝑠), starting from equal terminal values. ̂ − 𝑄 𝜋0 | ≤ 𝜖 uniformly, accepting only changes with If |𝑄

̂ 𝑎) > 𝑄(𝑠, ̂ 𝜋0 ) + 2𝜖 also ensures nondecreasing true 𝑄(𝑠, scores. The evaluated model uses accurate scores; an online guarantee requires a validated error bound. 4.3

Waiting Value and Computation

Let immediate and delayed plans publish the same artifact at 𝑡1 and 𝑡2 ≥ 𝑡1 , before 𝐻 , with remaining costs 𝐶1 , 𝐶2 and fixed subsequent benefit rate 𝑔 > 0:

𝑉wait − 𝑉now = (𝐶1 − 𝐶2 ) − 𝑔(𝑡2 − 𝑡1 ).

(13)

Requests

Tools

Guidance

State s

Idle windows

Model state

Use

Responses

LearnSched

Wait

Shared GPU

Advance u π₀

π₀

Q2

Q3

Qualify

qualified

Recover runtime

Idealized

Artifacts

Advance Wait

Short window Work Save Return Wait

Idle

Work Work

Return Return

Release memory

Cold start or

Warm resume

Capacity ready

treclaim ≤ tb − τr

Longer window

Same publish time; Wait avoids one save + one return.

(a) Online agent

Stop

Keep π₀ on ties; replan each window

Release

s′ (unsaved lost)

Return to serving

Select max Q π₀ Q = r − c + V (s′)

Evolution Store

s′ (preserved)

π₀

Choose action

Discard

Return

Q1

Observe state

Unsaved

Checkpoint

Save / discard + return

Return resources

Durable p

Wait Advance u

Reuse

Progress p, p

Current progress p

Feasible actions

Agent serving

? Backlog q

State transition

Shared reference continuation π₀

Available artifacts

Use

Artifact reuse Qualify

Use

Publish

Actual use

Future benefit

No benefit before actual use

Work

(b) State-aware rollout

Save

Return

Publish

(c) Progress & return

Figure 3. LearnSched overview. (a) Serving and evolution share GPU capacity; the scheduler observes backlog and progress, controls evolution, and returns resources to serving. Qualified artifacts are retained for reuse. (b) Safe actions are scored under the same reference continuation 𝜋0 . The idealized timelines align publication times: waiting avoids an extra checkpoint and return. (c) Checkpointing preserves progress; discarding loses only unsaved work. Capacity becomes ready after memory release and cold or warm recovery; benefits require actual artifact use. Waiting pays when avoided handoff and checkpoint costs exceed lost benefits. With one sufficiently long window, free switching and checkpointing, and no information updates, the choice reduces to comparing 𝑔(𝐻 − 𝑡 − 𝜏𝑒 ) − 𝐶𝑒 with zero. Figure 3(b) illustrates the equal-publication-time case, in which waiting saves one checkpoint and one return. The implementation enumerates 𝑂(𝐽 ) progress budgets over at most 𝐾 remaining windows and caches continuation values for equal durable progress. Exact DP over all reachable states and actions gives the model optimum.

4.4

Observation and Admission

Candidates record task class, remaining stage and cost, completion time, and reuse opportunities. Tool admission uses historical statistics and current arrivals; guidance uses class shares and quality proxies calibrated on independent history. Hidden target tests and future task identifiers do not enter estimates. The prototypes reevaluate value while synthesis is pending and check windows and capacity thereafter. Issued calls are charged through completion; full ROI optimization at each validation stage is not implemented. Admission estimates work as 𝑤 = 𝐿max + 0.0625𝑃 , using prompt length 𝑃 and maximum output length 𝐿max . A stage runs only if it fits with in-flight work within budgets of 1536 for tools and 1024 for guidance. This maximum-output reservation can be conservative. A separate contention controller uses arrival-aware additive increase and multiplicative decrease (AIMD) to adapt the background share.

4.5

Handoff and Publication Accounting

Pollable waits insert control checks into long sleeps. WholeGPU handoff checkpoints LoRA, terminates training, confirms device-memory release, and starts a serving replica. Training resumes from the final durable file published after termination, avoiding a checkpoint-publication race. An alternative retains the serving process and restores its device state on return, trading resident memory for faster recovery. Tools publish solve(x) after passing eight public inputs; each configuration starts with an empty library. Guidance generates base and additional answers, preferring a base answer that passes public tests. Each request freezes its visible artifacts before dispatch. Model-call accounting charges actual service, extra generation, construction, and validation:

𝐶all = 𝐶base + 𝐶extra + 𝐶build + 𝐶validate .

(14)

Failed builds, unpublished candidates, and unused calls are charged; absent base calls are not. Unknown usage remains missing. CPU execution is timed separately; the handoff ledger records durable progress, unsaved work, readiness, and device and host memory.

5

Experimental Setup

Component experiments measure contention, execution costs, and effective reuse. Comparisons use the same capabilities and resources, except Cal.*, which receives the future arrival calendar. Additional ledgers and sensitivity details appear in the appendices.

5.1

Hardware and Workloads

Contention experiments cross Granite-3B-A0.8B (MoE) and Qwen2.5-Coder-7B (Dense) with vLLM and SGLang on one A100-80GB. Arrivals follow 4→40→4 requests/s for 10 s per phase, with at most 64 output tokens. SLOs are 1.2, 0.62, 1.8, and 1.2 s for MoE on SGLang and vLLM, then Dense on SGLang and vLLM. Bad requests include deadline misses from scheduled arrival and failures: the 60 runs contain 28,805 requests, including two failures. Background steps count synthetic GPU work without capability calibration. Tool experiments use Qwen3.5-9B on SGLang, a 512token output limit, zero temperature, and no radix cache. In each 24 s window, sustained demand has 48 requests per block, stopped demand retains the same 12-request prefix, and sparse demand has requests at seconds 6 and 18. Configurations rebuild tools, isolate public tests from hidden evaluation, and run CPU paths in isolated workers. For guidance, we dispatch 114 known tasks from LiveCodeBench [10] under low–high–low arrivals. Five policies and five blocks each give 5700 requests. Whole-GPU handoff uses two A100s, Granite 3B, and rank-8 LoRA. One GPU serves; the other alternates between training and a replica. The vLLM trace has 100 s of low load, a 60 s burst at 120 requests/s, and 30 s of low load, with at most 128 output tokens and a two-second SLO. Reactive handoff starts at burst onset; advance-notice handoff starts 90 s earlier, with experiment-specified notice. SGLang recovery is measured independently. 5.2

Software

Inference uses vLLM and SGLang. Python control uses asyncio and aiohttp for asynchronous dispatch and cost/latency recording; tool validation and execution use subprocesses with resource limits. The finite-state simulator implements Window, Costed Window, rollout, and exact DP under common transition and cost models.

6

Evaluation

We evaluate plan selection, resource admission, capacity return, and investment payback. Each experiment compares policies under its stated information and cost model. 6.1

Value of State-Aware Scheduling

All policies share complete window information, transitions, costs, and return constraints; exact DP gives the optimum in the finite macro-action model. Scenarios and costs. The frozen grid crosses six calendars (80–480 s windows separated by 20 s), four work sizes (30/60/120/180 s), five benefit rates (0.1/0.25/0.5/1/2 s-eq./s), two benefit tails (120/480 s), cold/warm recovery, and four initial states: fresh, half-complete and unsaved, saved and active, or saved in service mode. This gives 1920 scenarios

with a 5 s work quantum; Appendix D specifies the calendars. Cold/warm recovery uses three-run SGLang means of 69.988/0.487 s. Independent LoRA measurements over 20 runs give save/load costs of 68.075/47.748 ms for about 13.24 MB of state. Execution and recovery cost one s-eq. per second; warm recovery adds 6.383 GiB of host PSS priced at 0.01 s-eq. per GiB-second. These separate measurements are combined in the model; memory pricing is assumed, unmeasured entry/release delays are zero, and resident-memory throughput effects are not fitted. Aggregate value and distribution. Figure 4(a) reports mean DP gaps of 40.122 for Window, 28.819 for static ROI, 5.369 for Costed Window, and 0.246 s-eq. for rollout. Rollout raises mean net value from 465.547 to 470.670, gaining 5.123 s-eq. (1.10%): full cost accounting explains most of the benefit. It improves 91 scenarios, ties in 1829, and never regresses. All improvements involve cold recovery: 71 backloaded and 20 alternating calendars (Figure 4(b)); all 960 warm scenarios tie. Rollout matches DP in 1907 scenarios, but the remaining 13 have a maximum gap of 124.836. Every policy meets return deadlines because they share the same action constraint. Never can incur negative value from returning initially active states. At a 1 s quantum in 480 freshcandidate scenarios, rollout improves 27 and ties in 453; its mean DP gap is zero, versus 5.869 for Costed Window. Why waiting helps. The fresh case in Figure 4(c) has 30 s of work, benefit rate 0.25, a back-loaded calendar, horizon 1020 s, and cold recovery. Costed Window advances 5 s at 200, checkpoints, and finishes with 25 s at 300; it publishes at 325.048 s for cost 170.092. Rollout starts at 300 and publishes at 330 for cost 99.988. Waiting saves 70.104 s-eq., loses 1.238 in benefits, and gains 68.866 net. With warm recovery, both publish at 30 s in the first window. 6.2

Resource Allocation and Admission

Adaptive shares. Among static shares of 0, 10%, 25%, and 50%, only shares up to 10% satisfy the 1% bad-request limit in every repeat across four systems. Adaptive allocation achieves 42.0–52.8 steps/s, or 2.66–2.91× the best measured feasible static policy, with no observed bad requests in 12 runs (Figure 5). Expanding under low load and contracting under bursts uses capacity that static policies reserve for peaks. Appendix A.1 reports SGLang tails. Work-based admission. We sweep work budgets of 384–2048 and count thresholds of 0–12. Each setting dispatches 181 tasks as 40/101/40 requests at 2/6/2 requests/s. Quality comes from fixed-answer replay; costs and latencies come from real calls. On vLLM, work budget 2048 and count threshold 4 both yield 2.21% deadline misses. Their quality gains are 3.87 versus 2.76 percentage points at 334.1 versus 345.0 additional tokens/request. On SGLang, both yield 1.10% misses, with gains of 2.76 versus 2.21 points

(a) All 1,920 cases

(b) Strict wins / 160 488.54

Never

(c) Why waiting helps

Front

0

0

Window

40.12

Back

0

71

Static ROI

28.82

Altern.

0

20

Frag.

0

0

LearnSched 0.25

Uniform

0

0

DP 0.00

Cont.

5.37

Costed W.

0

1 10 100 1k Mean gap to DP (s-eq.)

170.1

Costed 100.0

LearnSched 0

0

0

Warm

Cold

100 Cost (s-eq.)

200

Publish: 325.05 → 330.00 s Net gain: +68.87 s-eq.

91 wins · 1,829 ties · 0 losses

Work

Return

Other

Figure 4. Paired finite-model results. (a) Mean DP gaps over 1920 scenarios (symlog). (b) Strict wins over Costed Window per 160 scenarios. (c) S0348, the first improved fresh case: waiting avoids one cold recovery. Other: save, load, residency. (a) Background steps/s ↑

(b) Foreground bad requests ↓

MoE 3B SGLang

MoE 3B vLLM

Dense 7B SGLang

Dense 7B vLLM

MoE 3B SGLang

MoE 3B vLLM

Dense 7B SGLang

Dense 7B vLLM

Static 0%

0.1

0.1

0.1

0.1

0

0

0

0

10%

18.0

19.2

15.5

15.8

0

0

0.61

0

25%

44.7

47.0

38.6

39.0

1.91

2.79

10.13

21.61

50%

86.6

90.9

76.2

76.0

37.88

83.90

78.09

84.48

Adaptive

52.5

52.8

42.0

42.0

0

0

0

0

2.91×

2.75×

2.71×

2.66×

Gain¹ 0

100 steps/s

0 50 100% Means + 3 seeds · ¹vs best tested all-seed-feasible static

× seed >1% bad

Figure 5. Background progress and service risk across four systems. Numbers are three-seed means; strips show repeats, with crosses above 1% bad requests. Gain is relative to the best measured static policy meeting the limit in every repeat. at 333.6 versus 327.6 tokens/request. Work-based admission gains more quality at the same risk, while cost advantages depend on the engine. Count threshold 12 raises miss rates to 12.15% and 11.05%. Figure 6 shows both frontiers. A count limit treats requests with different prompt and output lengths alike; reserving estimated work makes the admission budget more sensitive to the load each request can impose. Engine-dependent costs reinforce the need to assess quality, resource use, and service risk together. 6.3

Capacity Return and Recovery

Control response. Pollable waits cut the P95 delay from command issuance to acknowledgment from 116.6 to 51.8 ms under pulses and from 73.0 to 50.6 ms under bursts (Figure 7). Every transition is acknowledged; neither implementation shows deadline misses. Burst service P99 is 0.562/0.564 s and background throughput 63.2/62.3 steps/s. Acknowledgment still precedes resource release and service

readiness. In an independent SGLang sweep, 50 requests/s produces 86.8% bad requests without failures, motivating deadline-risk triggers (Appendix A.2). Early return and lost progress. No scaling, reactive scaling, and advance-notice scaling yield mean burst miss rates of 65.4%, 44.0%, and zero, with P99 of 2.94, 3.31, and 1.38 s (Figure 8). Reactive capacity is ready 34.7 s after burst onset; advance-notice capacity is ready 56.3 s before it. Both start in 33–34 s, and all six handoff runs restore training exactly from final durable state. Mean background progress falls from 579.0 to 312.7 and 32.0 steps. Under 𝐸 − 𝑤𝑁bad , reactive scaling beats no scaling only for 𝑤 ≥ 0.173 and advance-notice scaling only for 𝑤 ≤ 0.089, so it is never selected on this mean frontier. These runs establish the value and progress cost of advance notice when the burst is known. Uncertain alerts additionally require deciding whether to act on that notice.

Token-work

Coarse

(b) P95 Pollable

1.0 4 2 0

Pulse 0.5 Burst

0

250 500 Extra tokens/req

(c) SGLang · cost ΔPass@1 (pp)

(a) Pulse response

(b) vLLM · risk

CDF

ΔPass@1 (pp)

(a) vLLM · cost

Queue-count

0

0.0

1 5 12 SLO miss (%)

(d) SGLang · risk

0

60 120 Latency (ms)

0

117 52 73 51 70 140 Latency (ms)

(c) Serving and progress ratios

4

P99 · pulse

2

Steps · pulse

P99 · burst

0

Steps · burst 0

250 500 Extra tokens/req

0

0.94

1 5 12 SLO miss (%)

1.00 Pollable / coarse

1.06

Hollow: 3 pairs; filled: mean ratio

Figure 6. Work-based versus count-based admission: all 26 settings, one 181-task run each. Left: cost–quality; right: risk–quality. A100 costs and latencies are measured; quality uses fixed-answer replay.

Resident state. Parking SGLang reduces GPU usage from 79.7 to 3.7 GiB; LoRA alongside it uses 11.4 GiB, and resumption restores 79.7 GiB (Figure 9). All three runs resume the same process and complete a real request. Mean cold startup is 69.99 s, warm recovery 0.487 s, and recovery through first-request completion 0.520 s. Device usage includes runtime reservations. Warm recovery holds 3.7 GiB on the GPU; host PSS rises from 4.77 to 11.16 GiB. These runs omit live prediction and value admission. Acting on uncertain alerts. Figure 10 combines measured recovery samples with calibrated synthetic alerts to isolate this decision. Early return trades the service loss avoided under a true burst against the evolution cost paid under a false alarm. The resulting threshold depends on both alert confidence and lead time; Appendix C gives the derivation, distributions, and assumed prices. On vLLM, reactive return, unconditional early return, and LearnSched incur 28.16, 15.76, and 14.61 s-eq./alert; SGLang yields 42.48, 30.21, and 28.87. Lead time explains most of the gain, with selective return adding savings. Pooling recovery samples across engines gives 14.63/28.90, close to engine-specific estimates. Thus, predicting earlier and pricing false alarms matter more here than refining recovery estimates. This return-only model uses synthetic events; it neither establishes online prediction gains nor jointly prices residency.

Figure 7. Preemption under 5%/0% pulses and bursts, three runs per implementation, timed through execution acknowledgment. (a) Pulse CDF; (b) pulse/burst P95; (c) service P99 and background-rate ratios. Open/filled points show paired repeats/mean ratios; shading marks ±2%. 6.4

Capability Investment and Payback

Demand persistence. Never does not invest; Window checks the remaining window; LearnSched’s value layer also estimates amortization. All share construction and public qualification. Under sustained demand, LearnSched spends 23447 tokens for 171/192 timely correct answers, versus Never’s 72351 for 53/192 and Window’s 24294 for 164/192 (Figure 11). Historical admission decisions match those of Window; 745 of the 847-token difference comes from service costs, so this difference does not establish improved investment selection. Under stopped demand, LearnSched and Window spend 16380/16418 tokens for 34/48 versus 30/48 timely correct answers. Under sparse demand, they spend 4624/7648 for 5/8 versus 4/8. LearnSched’s extra timely answer over Never costs 1660 tokens, requiring 𝑊 > 1660 for positive incremental utility. The construction outcomes in Figure 11(e) expose another cost: LearnSched and Window have 9 and 13 nonpositive-return builds, costing 5649 and 8711 tokens. Qualification alone does not ensure payback. Cal.* achieves the same sparse-demand quality as LearnSched with 2964 tokens, illustrating the remaining value of future demand information. Further ledgers appear in Appendix B. From publication to effective reuse. Guidance makes the distinction between availability and use explicit (Figure 12(a)). LearnSched has 53 ready opportunities and 27

(a) Burst risk

(b) Resource trade-off

44.0

0.0 None

Forecast

600

65.4

40

0

Reactive

React

Pred

Bin P99 (s)

80

LoRA steps

Burst misses (%)

No scale

(c) Capacity-ready timeline

300

0

0

2500 5000 Bad burst requests

3 2 1 0

60 120 Trace time (s)

180

Figure 8. vLLM capacity return on two A100s, three seeds per policy. None/React/Pred: no scaling, burst-triggered scaling, and a 90 s pre-alert. Panels show burst misses, progress–risk tradeoffs, and five-second P99/readiness for preselected seed 713, including client queuing. coefficient 𝛼 = 0.561 and separately measured use counts. They support accounting for both construction and service, rather than inferring net value from fewer builds alone.

(a) Memory across state handoff Host PSS: 4.77 → 11.16 GiB 79.7

Awake Parked + LoRA

3.7 11.4 79.7

Resumed 0

40 GPU occupied (GiB)

80

(b) Warm resume · 3 measured runs 69.99s

Cold Resume

0.49s 1 10 Capacity-ready time (s)

100

Figure 9. Same-process SGLang recovery on one A100, three runs. GPU occupancy and cold/warm readiness (means and min–max); host PSS is separate. Every resumption completes a real request.

uses on vLLM, and 89 ready opportunities and 38 uses on SGLang. Its use/ready ratios are 50.9% and 42.7%, compared with Eager’s 37.8% and 32.2%. Many requests still do not use a visible artifact. Counting every post-publication request as reuse overestimates amortization; use probability and per-use quality need separate estimates. Construction savings must also survive the use path. In the conditional five-block SGLang ledger, service overhead offsets 28.24% of the 30420-token construction/validation savings, leaving 21830.529 tokens net (Figure 12(b)). Every block saves cost, with revised timely-correct differences of 0, 1, 2, 0, and 1. These results use the stated service-cost

Resource pricing. Thirty profiler-on/off pairs across 10 prompts have equal paired output usage. The 30 profiled calls comprise 20 service and 10 construction calls. Mean service/build GPU activity is 76.9/378.3 ms for Bits and 53.8/4294.2 ms for Pyramid. Assuming no additional use cost, positive payback requires ⌊𝐶/𝑐⌋+1 uses: 2 and 3 under token pricing, but 5 and 80 under GPU pricing (Figure 13). GPU activity is the union of kernel, copy, and memset intervals; CPU costs are separate. In an independent sustained batch, TokenROI uses 73.76% fewer tokens than Never but 86.23% more GPU activity. GPUROI saves 71.25% of that activity but loses 32 timely correct answers. Under stopped demand, it saves 86.16% with unchanged timely quality. Resource savings and quality effects require joint assessment.

7

Discussion and Related Work

7.1

State Management and Scheduling Complexity

State management changes both feasible windows and plan values. With expensive cold recovery, checkpointing and segmented execution can incur substantial handoff costs, making a later, longer window more valuable. Warm recovery can bring a window policy close to the value of more complex planning. Runtime design and scheduling therefore interact: retaining state must be valued through faster return, less repeated work, and residency costs. Planning and prediction should focus where they change actions. 7.2

Capability Maintenance and Long-Term Value

The current evaluation focuses on capability construction and subsequent reuse, without systematically accounting

(a) Paired decision costs vLLM

(b) Value of lead time

Solid: LearnSched; dashed: reactive

SGLang

1.0

Loss/alert

Reactive Eager Shared LearnSched

Min. confidence

45

OOM

30

15

Oracle 0

(c) Return boundary

30 60 Loss/alert (s-eq.)

0

60 Lead time (s)

0.5

0.0

120

Return above curve

Wait below curve 0

60 Lead time (s)

120

Return-only model · calibrated synthetic alerts

Figure 10. Return decisions under synthetic alerts. (a) Mean losses of six rules and percentile ranges of seed means. (b) Seven lead times. (c) Return/wait boundaries. LearnSched estimates recovery cost per engine; Shared pools samples; Oracle knows events and actual durations. Loss is in s-eq./alert: one overflow-loss unit equals one second of equivalent evolution loss.

Token cost / Never (×)

Serving tokens

(a) Steady · 48 req

Four paired blocks

(b) Stopped · 12 req

72,351

1.0

Construction tokens

1.0

18,067 16,418 16,380

24,294 23,447 23,504 0.5

(c) Sparse · 2 req 3 13,256

2

0.5

0.0

Never Window Learn Sched

Cal.*

27.6% 85.4% 89.1% 88.0%

0.0

1 Never Window Learn Sched

7,648

0

Cal.*

29.2% 62.5% 70.8% 68.8%

4,624 2,964

2,964

Never Window Learn Sched

Cal.*

50.0% 50.0% 62.5% 62.5%

Timely correct (%) above; n = 192 / 48 / 8 by load · Four blocks · 24 s windows

(d) Request outcomes Steady

(e) Build outcomes Stopped Sparse

Never Window Learn Sched Cal.* 1 12 24 36 48 1 Request index · 4 repeated runs per policy LLM: other

LLM: timely (dot)

Cost†

Instances

12

Window

11

LearnSched

15

Cal.*

13

13

8,711 9

1

0 12 † Nonpositive builds, tokens

Tool: timely / positive build

5,649 865 24

Nonpositive build

Figure 11. Demand persistence and investment returns. (a–c) Total tokens, cost breakdowns, and timely correctness. Points show cost ratios from four historical blocks; quality uses revised summaries with denominators of 192, 48, and 8. (d) The 992 request slots arranged by aggregate counts. (e) Revised construction classes and costs of nonpositive-return builds. Cal.* knows the future calendar. for maintenance costs such as version updates, failure detection, and revalidation. As task requirements or tool interfaces change, published capabilities may become less effective, so short-term cost savings may not translate into

long-term net value. Future work should assess how ongoing maintenance and declining benefits affect investment decisions over the capability lifecycle.

(a) Pilot: readiness is not use

(b) Repeats: full cost matters

vL/Eager

69

28

17

37.8%

vL/Slack

64

32

18

36.0%

vL/Ours

61

27

50.9%

26

SG/Eager

27

59

28

32.2%

SG/Slack

27

60

27

31.0%

25

SG/Ours 0

51

38

42.7%

57 114 Requests (114 per policy)

Not ready

Ready, unused

SGLang · all five blocks Ours − Eager (ktokens)

Use/ready 6

0 -2.771 −6

-5.786 B0 ΔQ=0

Used

-3.879

-5.058

B1 B2 ΔQ=+1 ΔQ=+2

Build + valid.

B3 ΔQ=0

Serving

-4.337

B4 ΔQ=+1

Total

Ours: LearnSched. Separate studies. ΔQ: timely-correct difference.

Figure 12. Effective guidance use and conditional net cost; Ours denotes LearnSched. (a) Three groups per engine, each with 114 requests; ratios are use/ready. (b) Cost differences from Eager in five SGLang blocks: construction differences retain the original records; service differences are scaled by 𝛼 = 0.561. (a) Measured: all 30 profiled calls S: serving; C: construction · ◇ mean Bits / S

76.9

Bits / C

378.3

Pyr. / S

53.8

Pyr. / C

4,294.2 100 1k 10k GPU active time (ms; log)

(b) Conditional break-even uses Identical-call replacement, floor(C/c) + 1 Token GPU

Bits Pyramid

2 3

→ →

5 80

Not online GPU savings; CPU / failures excluded.

Figure 13. Device pricing and break-even reuse. Top: all 30 profiled calls, with diamonds marking means. Bottom: minimum uses needed to offset construction costs, assuming no additional cost per use. 7.3

Related Work

Jalapeño selects recompilation based on expected future execution benefits [2]; computation selection and metareasoning study when to invest further computation [5, 9]. We include qualification, actual use, and service-recovery paths in remaining investment value, and use classical rollout [3] to compare capacity-constrained execution plans. Ekya coordinates inference and retraining, RECL reuses historical models, and LeMix jointly schedules LLM training

and inference [4, 11, 14]. MetaClaw learns during inactivity and calendar windows [21]. Yan et al. propose a costaware evolution control plane with multiple actions and instantiate online weight updates through AReaL2.0 [24]. LearnSched focuses on break-even value, durable progress, and capacity-return timing for discrete capabilities, evaluating plan differences under the same information and execution method. Gandiva time-slices and migrates training jobs, Tiresias targets job completion time, and AntMan dynamically shares spare GPU resources [6, 22, 23]. Salus provides fast job switching and memory sharing, while REEF supports microsecond-scale preemption for DNN inference [8, 26]; these mechanisms inform the cost of lending and returning capacity. Clockwork uses predictable execution to meet inference latency targets [7]. Orca schedules at iteration granularity, Sarathi-Serve uses chunked prefills, and DistServe separates prefill and decoding to improve LLM serving under latency constraints [1, 25, 30]. PagedAttention and SGLang improve execution efficiency [12, 29], while Reflexion and Self-Refine provide feedback-based improvement mechanisms [15, 17]. Voyager accumulates reusable executable skills [18]. Recent advances extend these mechanisms to evolutionary program optimization and self-directed model adaptation [16, 31], as well as skill–tool co-evolution and adaptive workflow evolution [19, 20]. DGM and Hyperagents investigate agent self-modification, with the latter also making the meta-level improvement procedure itself editable [27, 28]. These advances expand both the capabilities available for reuse and the mechanisms that construct them. Investment scheduling must therefore re-estimate construction and validation

costs, expected reuse benefits, and capacity-return overhead as the system evolves.

8

Conclusion

LearnSched treats online evolution as a compute investment with delayed returns and state-dependent switching costs. Counterfactual rollout compares progress and waiting under capacity-return constraints. Measurements and finitemodel results show that waiting can avoid costly cold handoffs, while warm recovery reduces this advantage. Effective evolution scheduling must jointly account for future benefits and timely capacity return.

References [1] Amey Agrawal, Nitin Kedia, Ashish Panwar, Jayashree Mohan, Nipun Kwatra, Bhargav Gulavani, Alexey Tumanov, and Ramachandran Ramjee. 2024. Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve. In Proceedings of USENIX OSDI ’24. USENIX Association, Santa Clara, CA, 117–134. https://www.usenix.org/ conference/osdi24/presentation/agrawal [2] Matthew Arnold, Stephen Fink, David Grove, Michael Hind, and Peter F. Sweeney. 2000. Adaptive Optimization in the Jalapeño JVM. In Proceedings of ACM OOPSLA ’00. ACM, Minneapolis, Minnesota, USA, 47–65. doi:10.1145/353171.353175 [3] Dimitri P. Bertsekas, John N. Tsitsiklis, and Cynara Wu. 1997. Rollout Algorithms for Combinatorial Optimization. Journal of Heuristics 3 (1997), 245–262. https://www.mit.edu/~dimitrib/rollout.pdf [4] Romil Bhardwaj, Zhengxu Xia, Ganesh Ananthanarayanan, Junchen Jiang, Yuanchao Shu, Nikolaos Karianakis, Kevin Hsieh, Paramvir Bahl, and Ion Stoica. 2022. Ekya: Continuous Learning of Video Analytics Models on Edge Compute Servers. In Proceedings of USENIX NSDI ’22. USENIX Association, Renton, WA, 119–135. https://www. usenix.org/conference/nsdi22/presentation/bhardwaj [5] Frederick Callaway, Sayan Gul, Paul M. Krueger, Thomas L. Griffiths, and Falk Lieder. 2018. Learning to Select Computations. In Proceedings of UAI ’18. AUAI Press, Corvallis, Oregon, 776–785. https: //www.auai.org/uai2018/proceedings/papers/269.pdf [6] Juncheng Gu, Mosharaf Chowdhury, Kang G. Shin, Yibo Zhu, Myeongjae Jeon, Junjie Qian, Hongqiang Liu, and Chuanxiong Guo. 2019. Tiresias: A GPU Cluster Manager for Distributed Deep Learning. In Proceedings of USENIX NSDI ’19. USENIX Association, Boston, MA, 485–500. https://www.usenix.org/conference/nsdi19/ presentation/gu [7] Arpan Gujarati, Reza Karimi, Safya Alzayat, Wei Hao, Antoine Kaufmann, Ymir Vigfusson, and Jonathan Mace. 2020. Serving DNNs like Clockwork: Performance Predictability from the Bottom Up. In Proceedings of USENIX OSDI ’20. USENIX Association, 443–462. https: //www.usenix.org/conference/osdi20/presentation/gujarati [8] Mingcong Han, Hanze Zhang, Rong Chen, and Haibo Chen. 2022. Microsecond-scale Preemption for Concurrent GPU-accelerated DNN Inferences. In Proceedings of USENIX OSDI ’22. USENIX Association, Carlsbad, CA, 539–558. https://www.usenix.org/conference/ osdi22/presentation/han [9] Nicholas Hay, Stuart Russell, David Tolpin, and Solomon Eyal Shimony. 2012. Selecting Computations: Theory and Applications. In Proceedings of UAI ’12. AUAI Press, Catalina Island, CA, 346–355. https://auai.org/uai2012/papers/123.pdf [10] Naman Jain, King Han, Alex Gu, Wen-Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang, Armando Solar-Lezama, Koushik Sen, and Ion Stoica. 2024. LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code. arXiv:2403.07974 https://arxiv.org/abs/2403.07974 [11] Mehrdad Khani, Ganesh Ananthanarayanan, Kevin Hsieh, Junchen Jiang, Ravi Netravali, Yuanchao Shu, Mohammad Alizadeh, and Victor Bahl. 2023. RECL: Responsive Resource-Efficient Continuous Learning for Video Analytics. In Proceedings of USENIX NSDI ’23. USENIX Association, Boston, MA, 917–932. https://www.usenix.org/ conference/nsdi23/presentation/khani [12] Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient Memory Management for Large Language Model Serving with PagedAttention. arXiv:2309.06180 https://arxiv.org/ abs/2309.06180 [13] Jean-Yves Le Boudec and Patrick Thiran. 2001. Network Calculus: A Theory of Deterministic Queuing Systems for the Internet. Lecture Notes in Computer Science, Vol. 2050. Springer. doi:10.1007/3-54045318-0

[14] Yufei Li, Zexin Li, Yinglun Zhu, and Cong Liu. 2025. LeMix: Unified Scheduling for LLM Training and Inference on Multi-GPU Systems. arXiv:2507.21276 https://arxiv.org/abs/2507.21276 [15] Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Shashank Gupta, Bodhisattwa Prasad Majumder, Katherine Hermann, Sean Welleck, Amir Yazdanbakhsh, and Peter Clark. 2023. Self-Refine: Iterative Refinement with Self-Feedback. arXiv:2303.17651 https://arxiv.org/abs/2303.17651 [16] Alexander Novikov, Ngân Vũ, Marvin Eisenberger, Emilien Dupont, Po-Sen Huang, Adam Zsolt Wagner, Sergey Shirobokov, Borislav Kozlovskii, Francisco J. R. Ruiz, Abbas Mehrabian, M. Pawan Kumar, Abigail See, Swarat Chaudhuri, George Holland, Alex Davies, Sebastian Nowozin, Pushmeet Kohli, and Matej Balog. 2025. AlphaEvolve: A Coding Agent for Scientific and Algorithmic Discovery. arXiv:2506.13131 [cs.AI] https://arxiv.org/abs/2506.13131 [17] Noah Shinn, Federico Cassano, Edward Berman, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. 2023. Reflexion: Language Agents with Verbal Reinforcement Learning. arXiv:2303.11366 https: //arxiv.org/abs/2303.11366 [18] Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. 2023. Voyager: An Open-Ended Embodied Agent with Large Language Models. arXiv:2305.16291 https://arxiv.org/abs/2305.16291 [19] Yangbo Wei, Zhen Huang, Shaoqiang Lu, Junhong Qian, Qifan Wang, Chen Wu, and Lei He. 2026. SkillSmith: Co-Evolving Skills and Tools for Self-Improving Agent Systems. arXiv:2606.01314 [cs.AI] https: //arxiv.org/abs/2606.01314 [20] Yangbo Wei, Zhen Huang, Ronghao Xu, Hong Wang, and Wei W. Xing. 2026. EvoMAS: Heuristics in the Loop—Evolving Smarter Agentic Workflows. In Proceedings of the 43rd International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 306). PMLR, 133736–133763. https://proceedings.mlr.press/v306/wei26m. html [21] Peng Xia, Jianwen Chen, Xinyu Yang, Haoqin Tu, Jiaqi Liu, Kaiwen Xiong, Siwei Han, Shi Qiu, Haonian Ji, Yuyin Zhou, Zeyu Zheng, Cihang Xie, and Huaxiu Yao. 2026. MetaClaw: Just Talk—An Agent That Meta-Learns and Evolves in the Wild. arXiv:2603.17187 https: //arxiv.org/abs/2603.17187 [22] Wencong Xiao, Romil Bhardwaj, Ramachandran Ramjee, Muthian Sivathanu, Nipun Kwatra, Zhenhua Han, Pratyush Patel, Xuan Peng, Hanyu Zhao, Quanlu Zhang, Fan Yang, and Lidong Zhou. 2018. Gandiva: Introspective Cluster Scheduling for Deep Learning. In Proceedings of USENIX OSDI ’18. USENIX Association, Carlsbad, CA, 595–610. https://www.usenix.org/conference/osdi18/presentation/xiao [23] Wencong Xiao, Shiru Ren, Yong Li, Yang Zhang, Pengyang Hou, Zhi Li, Yihui Feng, Wei Lin, and Yangqing Jia. 2020. AntMan: Dynamic Scaling on GPU Clusters for Deep Learning. In Proceedings of USENIX OSDI ’20. USENIX Association, 533–548. https://www.usenix.org/ conference/osdi20/presentation/xiao [24] Ran Yan, Wei Fu, Jiale Li, Shusheng Xu, Zhiyu Mei, Jiaxuan Gao, Jiarui Zhang, Wentai Zhang, Hao Dai, Xujie Shen, Chuyi He, Zhen Pu, Jun Mei, Zhiyao Lin, Haitao Wang, Zhiqiang Ding, Jiawei Zhang, Huaijie Wang, Ruida Xu, Honghua Dong, Youhe Jiang, Yi Wu, Tongkai Yang, and Binhang Yuan. 2026. Next-Generation Agentic Reinforcement Learning Systems Enable Self-Evolving Agents. arXiv:2607.01120 https://arxiv.org/abs/2607.01120 [25] Gyeong-In Yu, Joo Seong Jeong, Geon-Woo Kim, Soojeong Kim, and Byung-Gon Chun. 2022. Orca: A Distributed Serving System for Transformer-Based Generative Models. In Proceedings of USENIX OSDI ’22. USENIX Association, Carlsbad, CA, 521–538. https://www. usenix.org/conference/osdi22/presentation/yu [26] Peifeng Yu and Mosharaf Chowdhury. 2020. Fine-Grained GPU Sharing Primitives for Deep Learning Applications.

Proceedings of Machine Learning and Systems 2 (2020), 98– https://proceedings.mlsys.org/paper_files/paper/2020/file/ 111. d9cd83bc91b8c36a0c7c0fcca59228f2-Paper.pdf [27] Jenny Zhang, Shengran Hu, Cong Lu, Robert Lange, and Jeff Clune. 2025. Darwin Gödel Machine: Open-Ended Evolution of SelfImproving Agents. arXiv:2505.22954 [cs.AI] https://arxiv.org/abs/ 2505.22954 [28] Jenny Zhang, Bingchen Zhao, Wannan Yang, Jakob Foerster, Jeff Clune, Minqi Jiang, Sam Devlin, and Tatiana Shavrina. 2026. Hyperagents. arXiv:2603.19461 [cs.AI] https://arxiv.org/abs/2603.19461 [29] Lianmin Zheng, Liangsheng Yin, Zhiqiang Xie, Chuyue Sun, Jeff Huang, Cody Hao Yu, Shiyi Cao, Christos Kozyrakis, Ion Stoica, Joseph E. Gonzalez, Clark Barrett, and Ying Sheng. 2024. SGLang: Efficient Execution of Structured Language Model Programs. arXiv:2312.07104 https://arxiv.org/abs/2312.07104 [30] Yinmin Zhong, Shengyu Liu, Junda Chen, Jianbo Hu, Yibo Zhu, Xuanzhe Liu, Xin Jin, and Hao Zhang. 2024. DistServe: Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving. In Proceedings of USENIX OSDI ’24. USENIX Association, Santa Clara, CA, 193–210. https://www.usenix.org/conference/ osdi24/presentation/zhong-yinmin [31] Adam Zweiger, Jyo Pari, Han Guo, Yoon Kim, and Pulkit Agrawal. 2025. Self-Adapting Language Models. In Advances in Neural Information Processing Systems, Vol. 38. Curran Associates, Inc., 74084–74115. doi:10.52202/085713-2483

A

Service Risk: Measurement and Interpretation

define the empirical conditional tail

The service experiments quantify the capacity cost of background work. This appendix specifies how latency, failures, and deadline violations enter that measurement, and distinguishes the burst experiments from the independent capacity sweep. These observations support the capacity-return constraint in Section 3; they do not establish a deadline guarantee under arbitrary arrivals. A.1

Latency origin and normalization. For a successful request 𝑖, let 𝑎𝑖 be its scheduled arrival, 𝑓𝑖 its completion time, and 𝐷𝑖 = 𝑑𝑖 − 𝑎𝑖 > 0 its latency budget. The normalized latency is 𝑥𝑖 = (𝑓𝑖 − 𝑎𝑖 )/𝐷𝑖 . Measuring from scheduled arrival includes client-side waiting before dispatch, so delaying dispatch cannot hide a deadline violation. Normalization puts the SLOs on a common axis: 𝑥𝑖 > 1 denotes lateness, while absolute latency still depends on the model.

(a) MoE 3B

(b) Dense 7B

|ℱ| + |{𝑖 ∈ 𝒮 ∶ 𝑥𝑖 > 1}| 𝑏̂ = 𝑁 𝑁𝑓 + 𝑁 ℓ 𝑁𝑓 𝑁 𝑁 = = + 𝑠 ℓ 𝑁 𝑁 𝑁 𝑁𝑠 = 𝑓̂ + (1 − 𝑓̂)𝑆𝑠̂ (1).

.01 .001 0

1

2.5 0 1 Arrival-to-completion / SLO Static 0% 25% Adaptive 10% 50%

2.5

Figure A.1. Successful-response latency distributions for the two SGLang models and five resource policies. Latency includes waiting from scheduled arrival and is divided by the model’s SLO. Failed requests are excluded from these curves and retained in the bad-request denominator.

Conditional tails versus total service risk. Partition the 𝑁 > 0 requests into disjoint failure and success sets ℱ and 𝒮, with 𝑁𝑓 = |ℱ| and 𝑁𝑠 = |𝒮| = 𝑁 − 𝑁𝑓 . For 𝑁𝑠 > 0,

(A.2)

If 𝑁𝑠 = 0, the conditional tail is undefined and the count formula directly gives 𝑏̂ = 1. A successful-response curve alone cannot recover total risk without the failure fraction. Both failures in the SGLang experiments therefore remain in the denominator of Figure 5, although they have no finite successful-completion latency in Figure A.1. The identity applies within a run or to consistently pooled counts. Averaging per-run rates gives each repetition equal weight; pooling requests weights runs by their request counts. These summaries should not be interchanged when denominators differ. The three repetitions reveal observed variability, but an absence of bad requests in this finite workload does not establish zero risk for unseen bursts. A.2

1

(A.1)

Let 𝑁ℓ = |{𝑖 ∈ 𝒮 ∶ 𝑥𝑖 > 1}| count late successes, and let 𝑓̂ = 𝑁𝑓 /𝑁 . A request is bad if it fails or succeeds late. Because these events are disjoint, their counts give the exact decomposition

Response Tails and Failure Accounting

Figure A.1 supplements Section 6.2. The SGLang experiments cross two models and five resource policies, with three repetitions per setting: 30 runs contain 14310 requests, of which 14308 succeed. The corresponding SLOs are 1.2 s for Granite-3B-A0.8B and 1.8 s for Qwen2.5-Coder-7B. Arrivals follow the low–high–low workload in Section 5.

Response tail

|{𝑖 ∈ 𝒮 ∶ 𝑥𝑖 > 𝑥}| |𝒮| 1 ∑ 1{𝑥 > 𝑥}. = 𝑁𝑠 𝑖∈𝒮 𝑖

𝑆𝑠̂ (𝑥) =

Independent Capacity Sweep

Figure A.2 sweeps offered load for SGLang with Granite 3B on one A100. It uses open-loop Poisson arrivals, a twosecond SLO, and one run per rate. The open-loop workload does not reduce arrivals when responses slow down, allowing queueing pressure to accumulate. Its SLO and arrival process differ from the preceding burst experiment, so the curves are not additional repetitions of that experiment. Increasing arrivals from 40 to 50 requests/s raises the badrequest rate from 16.7% to 86.8% and P99 from 2.14 to 2.73 s, without failures at either rate. Failures first appear at 60 requests/s. Deadline violations thus expose a loss of useful capacity before the runtime begins failing requests. Failureonly control misses this earlier degradation. This sweep identifies an observed operating boundary for the specified workload, rather than a workload-independent capacity constant. One run per rate cannot characterize tail uncertainty or establish the exact saturation point. The scheduling implication is narrower: capacity admission and return should account for deadline pressure, including waiting outside the serving engine.

Under sustained demand, LearnSched saves 67.59% of tokens relative to Never and 3.49% relative to Window. Its seven additional timely correct answers over Window correspond to 3.65 percentage points. However, LearnSched and Window make identical historical admission decisions, and 745 of their 847-token cost difference comes from service. These totals demonstrate a realized cost–quality difference; they do not establish better plan selection. For a policy 𝑃 , let 𝑄𝑃 count timely correct answers, 𝐶𝑃 be the charged token cost, and 𝑊 ≥ 0 value one timely correct answer in token-cost units. On a common workload, 𝑈𝑃 (𝑊 ) = 𝑊 𝑄𝑃 − 𝐶𝑃 gives

(a) Tail latency · SLO = 2 s Bad

Failed

P99 (s)

60 10 2 Requests (%)

(b) Bad requests versus failures 100 50 0

30

50 70 Offered load (req/s)

Demand Persistence and Incremental Value

Table B.1 collects the comparisons underlying Figure 11. All token totals include charged construction and validation, including unsuccessful or unused work, as specified by Equation (14). Timely correctness counts answers that meet both the correctness and deadline requirements. Table B.1. Reported tool ledgers by demand pattern. Costs are historical token totals; timely-correct counts use the revised summaries. Cal.* knows future arrivals. Demand

Policy

(B.1)

For LearnSched relative to Cal.*, substitution of the stopped and sparse ledgers yields

Δ𝑈stopped (𝑊 ) = 𝑊 (34 − 33) − (16380 − 13256) = 𝑊 − 3124, Δ𝑈sparse (𝑊 ) = 𝑊 (5 − 5) − (4624 − 2964) = −1660.

Investment Ledgers and Price Sensitivity

The ledgers separate the cost of constructing a capability from the cost and quality of using it. They complement the plan comparison in Section 6.1: a lower realized bill can arise from a different service path even when admission decisions are unchanged. The historical costs and revised quality summaries support aggregate comparisons. B.1

= (𝑊 𝑄𝑃 − 𝐶𝑃 ) − (𝑊 𝑄𝑄 − 𝐶𝑄 ) = 𝑊 (𝑄𝑃 − 𝑄𝑄 ) − (𝐶𝑃 − 𝐶𝑄 ).

Figure A.2. Independent SGLang capacity sweep with open-loop Poisson arrivals and a two-second SLO. Each rate has one run. The vertical line marks 40 requests/s; the rise in deadline violations precedes observed request failures.

B

Δ𝑈𝑃,𝑄 (𝑊 ) = 𝑈𝑃 (𝑊 ) − 𝑈𝑄 (𝑊 )

100

Tokens Timely correct

Sustained Never Window LearnSched

72351 24294 23447

53/192 164/192 171/192

Stopped

Window LearnSched Cal.*

16418 16380 13256

30/48 34/48 33/48

Sparse

Window LearnSched Cal.*

7648 4624 2964

4/8 5/8 5/8

(B.2)

Under stopped demand, LearnSched’s incremental utility is therefore positive if and only if 𝑊 > 3124; equality gives a tie. Under sparse demand, equal aggregate quality and strictly greater cost favor Cal.* for every common 𝑊 . Cal.* has privileged knowledge of future arrivals, so neither comparison holds information access fixed. Qualification also differs from amortization. Across the revised workload classifications, LearnSched and Window incur 5649 and 8711 tokens on nonpositive-return builds, respectively, a 35.15% difference. These costs remain in the ledger even when a candidate passes public tests. Passing those tests establishes eligibility for publication; positive return requires sufficient reuse at a worthwhile unit value. B.2

Conditional Guidance Costs

Figure 12(b) preserves construction and validation differences from the five historical SGLang blocks, while multiplying service differences by 𝛼0 = 0.561. Let 𝑆𝑏 = 30420 be the aggregate construction saving and 𝐷𝑠 the unscaled additional service cost. Holding these quantities fixed, the saving at a service-price coefficient 𝛼 ≥ 0 follows from

8589.471 8589.471 = = 15311, 𝛼0 0.561 𝑆(𝛼) = 𝑆𝑏 − 𝛼𝐷𝑠 = 30420 − 15311𝛼. 𝐷𝑠 =

(B.3)

At the reported coefficient, 𝑆(0.561) = 21830.529 token-cost units: service offsets 28.24% of the construction saving. The revised timely-correct differences are 0, 1, 2, 0, 1, giving four

additional answers in aggregate. Consequently, 5

Δ𝑈 (𝑊 , 𝛼) = 𝑊 ∑ Δ𝑄𝑗 + 𝑆(𝛼) 𝑗=1

= 𝑊 (0 + 1 + 2 + 0 + 1) + 𝑆(𝛼) = 4𝑊 + 30420 − 15311𝛼.

(B.4)

Since 𝐷𝑠 = 15311 > 0, the sign of incremental utility has the equivalent conditions

Δ𝑈 (𝑊 , 𝛼) > 0 ⟺ 15311𝛼 < 30420 + 4𝑊 30420 + 4𝑊 ⟺ 𝛼< . 15311

(B.5)

At 𝑊 = 0, the boundary is 30420/15311 ≈ 1.9868; equality gives a tie. These sensitivities hold ledger outcomes fixed. They do not measure new operating points or account for behavioral, quality, or reuse changes induced by new prices. The use counts in Figure 12(a) come from a separate experiment. They demonstrate the distinction between artifact availability and actual use, but cannot be joined request by request to the conditional cost ledger. Per-use savings remain unidentified, and missing usage remains unknown.

C

Return, Stopping, and Benefit Timing

This appendix examines three decision boundaries: returning capacity after an alert, stopping an in-flight computation, and recognizing benefits after publication. Each is an instance of valuing remaining work. The experiments use different workloads and information, so their results characterize these boundaries separately rather than a single integrated online policy. C.1

Returning Capacity Under an Alert

Restrict Equation (2) to two feasible actions: return early or wait. Fix the information ℐ available at the decision, including warning lead time 𝐿, and let 𝐸 denote a true burst, with 𝑝 = Pr(𝐸 ∣ ℐ) ∈ [0, 1]. All expectations below condition on this same information; write 𝔼ℐ [⋅] = 𝔼[⋅ ∣ ℐ]. Let ℓ𝑤 and ℓ𝑎 be integrable remaining losses under waiting and early return. Subtract a common event-dependent baseline from both losses so that waiting on a false alarm has conditional mean zero; this preserves their expected difference. Under a true burst, the conditional means are 𝐶𝑤 and 𝐶𝑎 (𝐿). Waiting includes the better feasible continuation of reactive scaling and no scaling under the information then available. Under a false alarm, early return has incremental mean cost 𝐶𝑓 > 0. The law of total expectation gives

𝑄𝑤 = 𝔼ℐ [ℓ𝑤 1𝐸 ] + 𝔼ℐ [ℓ𝑤 1𝐸 𝑐 ] = 𝑝𝐶𝑤 + (1 − 𝑝) 0 = 𝑝𝐶𝑤 , 𝑄𝑎 = 𝔼ℐ [ℓ𝑎 1𝐸 ] + 𝔼ℐ [ℓ𝑎 1𝐸 𝑐 ] = 𝑝𝐶𝑎 (𝐿) + (1 − 𝑝)𝐶𝑓 .

(C.1)

Here 𝑄𝑤 and 𝑄𝑎 are losses to minimize. Conditional costs include every remaining consequence of the specified continuations; their dependence on ℐ is suppressed. At 𝑝 = 0 or 𝑝 = 1, the zero-probability branch contributes zero. Previously paid costs are common to both actions and cancel. Write 𝐵(𝐿) = 𝐶𝑤 − 𝐶𝑎 (𝐿) for the conditional benefit of acting early. For 𝐵(𝐿) > 0, early return is strictly preferred if and only if 𝑄𝑎 < 𝑄𝑤

⟺ 𝑝𝐶𝑎 (𝐿) + (1 − 𝑝)𝐶𝑓 < 𝑝𝐶𝑤 ⟺ 𝑝𝐵(𝐿) > (1 − 𝑝)𝐶𝑓 ⟺ 𝑝[𝐵(𝐿) + 𝐶𝑓 ] > 𝐶𝑓 𝐶𝑓 ⟺ 𝑝 > 𝑝 ∗ (𝐿) ∶= . 𝐵(𝐿) + 𝐶𝑓

(C.2)

Division preserves the inequality because 𝐵(𝐿) + 𝐶𝑓 > 0, and 𝑝 ∗ (𝐿) ∈ (0, 1). If 𝐵(𝐿) ≤ 0, then 𝑝𝐵(𝐿) − (1 − 𝑝)𝐶𝑓 ≤ 0 for every 𝑝 ∈ [0, 1], so early return cannot strictly improve the loss. Ties retain waiting. Sensitivity and calibration. For 𝐵 > 0 and 𝐶𝑓 > 0, apply the quotient rule while holding the other parameter fixed. The threshold decreases with avoidable loss and increases with false-alarm cost: 𝜕𝑝 ∗ 0 ⋅ (𝐵 + 𝐶𝑓 ) − 𝐶𝑓 = 𝜕𝐵 (𝐵 + 𝐶𝑓 )2 𝐶𝑓 =− < 0, (𝐵 + 𝐶𝑓 )2 (C.3) (𝐵 + 𝐶𝑓 ) − 𝐶𝑓 𝜕𝑝 ∗ = 𝜕𝐶𝑓 (𝐵 + 𝐶𝑓 )2 𝐵 = > 0. (𝐵 + 𝐶𝑓 )2 Longer lead time lowers the threshold only insofar as it increases 𝐵(𝐿). Faster recovery can instead reduce the extra loss avoided by advance notice, making early intervention less valuable. Recovery latency should thus change the action value, not merely a fixed scheduling margin. Keep 𝐵 > 0 and 𝐶𝑓 > 0 fixed, and suppose 𝑝 ̂ ∈ [0, 1] satisfies |𝑝 − 𝑝|̂ ≤ 𝜖𝑝 . Define the true advantage of early return as 𝐷(𝑝) = 𝑄𝑤 − 𝑄𝑎 = 𝑝𝐵 − (1 − 𝑝)𝐶𝑓 . Expanding the difference and applying the probability-error bound gives

̂ + (𝑝 − 𝑝)𝐶 ̂ 𝑓 𝐷(𝑝) − 𝐷(𝑝)̂ = (𝑝 − 𝑝)𝐵 ̂ = (𝑝 − 𝑝)(𝐵 + 𝐶𝑓 ), ̂ = |𝑝 − 𝑝|(𝐵 ̂ + 𝐶𝑓 ) |𝐷(𝑝) − 𝐷(𝑝)|

(C.4)

≤ 𝜖𝑝 (𝐵 + 𝐶𝑓 ), 𝐷(𝑝) ≥ 𝐷(𝑝)̂ − 𝜖𝑝 (𝐵 + 𝐶𝑓 ). ̂ − (1 − 𝑝)𝐶 ̂ 𝑓 > 𝜖𝑝 (𝐵 + 𝐶𝑓 ) is sufficient for Thus 𝐷(𝑝)̂ = 𝑝𝐵 𝐷(𝑝) > 0 and lower true loss. This conservative condition

excludes cost-model errors. Synthetic events follow their stated probabilities, which does not validate conditional error bounds for a live predictor. Synthetic Alert Protocol and Its Scope

Figure 10 combines measured readiness and failed-wait samples with synthetic alerts. Alert probabilities follow 𝑝 ∼ Beta(2, 2), and event occurrence is sampled with probability 𝑝 . Duration, lead time, and overflow rate follow lognormal distributions with the parameters in Table C.1. To preserve an arithmetic mean 𝑚 > 0, write 𝑋 = 𝑒 𝜇+𝜎 𝑍 with 𝑍 ∼ 𝒩(0, 1) and 𝜎 ≥ 0. Completing the square in the normal-density integral gives

𝔼[𝑋 ] =

𝑒𝜇 𝜎 𝑢−𝑢 2 /2 𝑑𝑢 ∫ 𝑒 √2𝜋 ℝ 2

2

𝑒 −(𝑢−𝜎 ) /2 𝑑𝑢 ℝ √2𝜋

= 𝑒 𝜇+𝜎 /2 ∫

(C.5)

2 = 𝑒 𝜇+𝜎 /2 = 𝑚,

(a) All 48 held-out check tasks Cheap Always think Fixed 512 Fixed 1024 Fixed 1536 Fixed 2048 Fixed 3072 Prefix 0

The shifted normal density integrates to one; taking logarithms is valid because 𝑚 > 0. Table C.1. Synthetic alert parameters. Means are in the original scale; standard deviations are in log space. These distributions are modeling inputs. Arithmetic mean

Log-space s.d.

90 s 45 s 1.2

0.65 0.35 0.50

Handoff loss is 13.84 s-eq., and competing policies share the same sampled events. One overflow-loss unit is priced as one second of equivalent evolution loss. Pairing holds the realized event sequence fixed when comparing decisions; it does not remove uncertainty in the underlying recovery measurements or validate the assumed loss conversion. Oracle knows event occurrence and actual duration, achieving 8.74/22.16 s-eq./alert on vLLM/SGLang. It is an information benchmark rather than an implementable predictor. LearnSched’s corresponding losses are 14.61/28.87, versus 15.76/30.21 for unconditional early return. These results isolate the value of selective action after notice is available. Online prediction accuracy and warm-state memory costs remain unevaluated. C.3

30 27 31 33 33 34 35 35

40 80 Prompt + output (ktokens)

512 · + · · 1024 · + × · 1536 · + × · 2048 + + × · 3072 + + × + Pfx · + × +

· · · · · ·

Correct

Burst duration Warning lead time Overflow rate

Q/48

(b) All 16 public-failure upgrades

𝜇 = log 𝑚 − 𝜎 2 /2.

Quantity

Tokens 10,278 87,934 19,983 26,304 31,424 36,188 45,257 25,280

Budget Stopping During Execution

Figure C.1 replays a Prefix rule frozen on development data against existing MBPP records. Failure of a public test on the cheap path triggers an upgrade. At step 512, the rule inspects the first 64 reasoning characters: a prefix of Thinking

· × × × × ×

· · · · · ·

· · · · · ·

· · · × + + × + + × + + × + + × + +

· · · · · ·

· · · · · ·

· · · · · ·

· · · · · ·

· · · · · ·

79* 86 177 183 219* 228 260 286 311 386 400 417* 442* 471* 489* 510

C.2

Process: stops execution; otherwise, execution continues to step 1536. This format is a model-specific proxy, rather than a direct estimate of the value of further computation.

Wrong

Budget

Figure C.1. Cost and quality under stage stopping. (a) Aggregate token costs and correct counts for eight policies; Prefix’s 35/48 uses the revised summary. (b) Recorded outcomes for 16 upgrades after failed public tests, including six format-triggered stops. The per-task matrix remains only partly reconciled with the revised totals.

In the revised summary, Prefix spends 25280 tokens for 35/48 correct answers, matching the fixed 3072-step budget’s aggregate correctness at 44.14% lower cost than its 45257 tokens. The fixed 1024-step budget spends 26304 tokens for 33/48; Cheap and the fixed 512-step budget answer 30 and 31 tasks correctly. These counts measure correctness; Appendix B instead uses timely correctness. At an intermediate history ℎ, compare immediate stopping with a specified feasible continuation policy. Let 𝑌𝑐 , 𝑌𝑠 ∈ {0, 1} denote their eventual correctness and 𝐶𝑐 , 𝐶𝑠 their remaining charged costs, including any required capacity return. Assume finite conditional expected costs, and let 𝑊 ≥ 0 value a correct answer. Both options have already paid 𝐶paid (ℎ). With Δ𝑄ℎ = 𝑌𝑐 − 𝑌𝑠 and Δ𝐶ℎ = 𝐶𝑐 − 𝐶𝑠 ,

Exp. CE

LearnSched

10 5

Duty (%)

0

100

Cum. value

Requests/s

Demand → investment → payoff

150

50 0

75 0 0

6

12 Simulated time (hours)

18

24

Figure C.2. Compute expenditure and delayed benefit under synthetic demand. The displayed run uses seed 261000, 24 hours, 24 candidates, and a 12 GPU-hour budget. Exp. CE and the conservative rule share the simulator. Demand, resource shares, and cumulative values are simulated; the trace is not an online deployment measurement. linearity of conditional expectation gives

𝑈𝑐 (ℎ) = 𝔼[𝑊 𝑌𝑐 − 𝐶𝑐 ∣ ℎ] − 𝐶paid (ℎ), 𝑈𝑠 (ℎ) = 𝔼[𝑊 𝑌𝑠 − 𝐶𝑠 ∣ ℎ] − 𝐶paid (ℎ), 𝑈𝑐 (ℎ) − 𝑈𝑠 (ℎ) = 𝔼[𝑊 𝑌𝑐 − 𝐶𝑐 ∣ ℎ] − 𝔼[𝑊 𝑌𝑠 − 𝐶𝑠 ∣ ℎ] (C.6)

For the finite candidate set 𝒥, let 𝑘𝑗 be candidate 𝑗 ’s completion slot, 𝑏𝑗,𝑘 its simulated benefit in slot 𝑘 , and 𝑐𝑘 the cost charged in that slot. Set 𝑘𝑗 = ∞ for unfinished candidates. With initial value 𝐺(−1) = 0, summing slot increments and exchanging finite sums yields

= 𝑊 𝔼[𝑌𝑐 − 𝑌𝑠 ∣ ℎ] − 𝔼[𝐶𝑐 − 𝐶𝑠 ∣ ℎ] = 𝑊 𝔼[Δ𝑄ℎ ∣ ℎ] − 𝔼[Δ𝐶ℎ ∣ ℎ]. Consequently, this continuation is strictly preferable exactly when 𝑊 𝔼[Δ𝑄ℎ ∣ ℎ] > 𝔼[Δ𝐶ℎ ∣ ℎ]. Previously consumed tokens cancel, while future costs remain. Global optimality would additionally require optimizing the continuation at every future history. Prefix uses a stopping signal without estimating these conditional values. Equal aggregate correctness also does not imply equal correctness on every task: Prefix truncates task 79, which the longer budget answers correctly. In a historical transfer across five task families, its cost for the same three skills is 5.26% higher than the fixed 1024-step budget. The revised 35/48 total has not been fully reconciled with pertask records. The evidence therefore supports a workloadspecific aggregate tradeoff, not universal dominance or a validated stopping rule across models. C.4

Benefit Accumulation After Publication

Figure C.2 uses a separate simulator with stable, singleburst, recurrent, and decaying demand in five-minute slots. The displayed run fixes seed 261000, a 24-hour horizon, 24 candidates, and a 12 GPU-hour budget. Benefits accrue after the completion slot and decay with artifact age. Costs, completion delays, and queue replacement are shared.

𝐾

𝐺(𝐾 ) = ∑[𝐺(𝑘) − 𝐺(𝑘 − 1)] 𝑘=0 𝐾

= ∑ [∑ 1{𝑘 > 𝑘𝑗 }𝑏𝑗,𝑘 − 𝑐𝑘 ] 𝑘=0 𝑗∈𝒥

𝐾

=

∑

(C.7)

𝐾

∑ 𝑏𝑗,𝑘 − ∑ 𝑐𝑘 .

𝑗∈𝒥∶ 𝑘𝑗 <𝐾 𝑘=𝑘𝑗 +1

𝑘=0

Completion slots are nonnegative integers when finite, and an empty sum is zero. The final expression includes only candidates completed before 𝐾 , with no benefit in the completion slot. All charged costs remain, including costs of unfinished work. Zero background share can mean capacity gating, budget exhaustion, or no pending work; previously completed artifacts can still contribute benefits. Exp. CE uses an exponential-utility certainty equivalent, while the conservative rule discounts reuse, qualification, and unit benefit. Their final synthetic net values are 159.89 and 163.16 for this fixed seed. A single trace illustrates delayed amortization and the accounting convention; it does not establish a distribution-wide policy ranking or measure real capability gains from the simulated work.

D

Finite Model: Specification and Guarantees

The finite experiment in Section 6.1 compares execution plans under common information, transitions, prices, and return constraints. Exact dynamic programming optimizes this action model; unmodeled arrivals, candidates, and runtime mechanisms remain outside its scope. D.1 Scenario Grid and State Semantics Each scenario contains one synthetic capability. Its worksize parameter is 30, 60, 120, or 180 s of construction and validation, with completed and durable progress set by the initial state. The primary grid uses a 5 s work quantum. Table D.1 lists the six ordered calendars; adjacent windows are separated by 20 s, and service must be restored by each window’s end. We set the first window’s start to zero. Table D.1. Ordered window durations in the finite model. Adjacent windows have a 20 s gap; the single continuous window has no inter-window gap. Calendar

Window durations (s)

Front-loaded Back-loaded Alternating Fragmented Uniform Continuous

240, 80, 80, 80 80, 80, 80, 240 120, 240, 120, 240 80, 80, 80, 80 160, 160, 160, 160 480

Post-publication benefit rates are 0.1, 0.25, 0.5, 1, or 2 seq./s, and the benefit horizon ends 120 or 480 s after the last window. Each combination is evaluated with cold and warm recovery and four initial states: fresh, half-complete with unsaved work, half-complete with saved work while active, and half-complete with saved work in service mode. The distinction between completed and durable progress controls what survives interruption; active states incur return costs even when abandoned. The Cartesian product has 6×4×5×2×2×4 = 1920 scenarios. The configuration is frozen and hashed before computation. Means over this complete grid describe its specified mixture of cases; they are not estimates of how frequently those cases occur in a production workload. D.2 Transition Costs and Exact Evaluation Cold and warm recovery take 69.988 and 0.487 s, using threerun SGLang means. Save and load take 68.075 and 47.748 ms from an independent LoRA microbenchmark. These delays remain exact wall-clock costs even though progress advances in discrete quanta. Execution and recovery cost one s-eq. per second. Warm recovery additionally prices 6.383 GiB of incremental host PSS at 0.01 s-eq. per GiB-second, or 0.06383 s-eq. per second of charged residency.

The measurements come from separate experiments. The memory price is assumed, unmeasured entry and release delays are zero, and resident-device-memory effects on throughput are not fitted. A residency bill depends on retained memory and retention time; recovery latency alone cannot determine net value. At window 𝑘 , a feasible macro-action must satisfy Equation (10), including loading, checkpointing when selected, and the return path. Unsaved progress is discarded on a return without checkpointing. Waiting in service mode adds no recovery cost. Publishing at 𝑡𝑝 settles the remaining benefit as Φ(𝑡𝑝 ) = 𝑔 (𝐻 − 𝑡𝑝 )+ and still charges the publication action’s return. Abandonment earns no capability benefit but pays any required return. Never can therefore have negative value when initially active. Finite-stage assumptions. Index stages by 𝑘 = 0, … , 𝑇 − 1. The state contains the stage and all information determining transitions and rewards. Each action set 𝒜𝑘 (𝑠) is finite, nonempty, and return-feasible. Every nonterminal action advances one stage, excluding zero-duration cycles. Publication and abandonment enter an absorbing state with zero subsequent rewards. All rewards are finite, and publication benefit is credited only to the publishing action. Let 𝐹𝑘 (𝑠, 𝑎) be the successor and 𝑅𝑘 (𝑠, 𝑎) the net reward. A feasible policy 𝜋 selects 𝑎𝑘 = 𝜋𝑘 (𝑠𝑘 ) ∈ 𝒜𝑘 (𝑠𝑘 ). Starting from 𝑠𝑘 = 𝑠 and following 𝑠𝑗+1 = 𝐹𝑗 (𝑠𝑗 , 𝜋𝑗 (𝑠𝑗 )), its value satisfies 𝑇 −1

𝑉𝑘𝜋 (𝑠) = ∑ 𝑅𝑗 (𝑠𝑗 , 𝜋𝑗 (𝑠𝑗 )) 𝑗=𝑘

𝑇 −1

= 𝑅𝑘 (𝑠, 𝜋𝑘 (𝑠)) + ∑ 𝑅𝑗 (𝑠𝑗 , 𝜋𝑗 (𝑠𝑗 ))

(D.1)

𝑗=𝑘+1 𝜋 (𝑠 = 𝑅𝑘 (𝑠, 𝜋𝑘 (𝑠)) + 𝑉𝑘+1 𝑘+1 ).

The empty terminal sum gives 𝑉𝑇𝜋 = 0. For a proposed first action 𝑎, write 𝑠𝑎 = 𝐹𝑘 (𝑠, 𝑎) and let 𝜌 range over feasible continuation policies from stage 𝑘 + 1. Optimizing the first action and then its continuation gives

𝑉𝑘∗ (𝑠) =

𝜌

max {𝑅𝑘 (𝑠, 𝑎) + 𝑉𝑘+1 (𝑠𝑎 )}

𝑎∈𝒜𝑘 (𝑠), 𝜌

𝜌

= max {𝑅𝑘 (𝑠, 𝑎) + max 𝑉𝑘+1 (𝑠𝑎 )} 𝑎∈𝒜𝑘 (𝑠)

𝜌

(D.2)

∗ (𝑠 )}. = max {𝑅𝑘 (𝑠, 𝑎) + 𝑉𝑘+1 𝑎 𝑎∈𝒜𝑘 (𝑠)

The finite reachable decision tree ensures that the maxima exist. Backward evaluation from 𝑉𝑇∗ = 0 gives the exact feasible optimum. Policy comparisons hold transitions, terminal values, and cost accounting fixed.

D.3 Policy Improvement and Scoring Error Retain the finite-stage assumptions above and the reference 𝜋0 from Equation (11). At each state, let the candidate set 𝒞𝑘 (𝑠) ⊆ 𝒜𝑘 (𝑠) contain the feasible reference action 𝜋0,𝑘 (𝑠). Define the exact reference-continuation score and rollout choice by 𝜋 𝜋0 𝑄𝑘 0 (𝑠, 𝑎) = 𝑅𝑘 (𝑠, 𝑎) + 𝑉𝑘+1 (𝐹𝑘 (𝑠, 𝑎)), 𝜋0 𝜋𝑅,𝑘 (𝑠) ∈ arg max 𝑄𝑘 (𝑠, 𝑎). 𝑎∈𝒞𝑘 (𝑠)

(D.3)

Define a guarded policy 𝜋𝐺 that selects a replacement 𝑎 ∈ ̂ 𝒞𝑘 (𝑠) only if 𝐴(𝑎) > 2𝜖 and otherwise retains 𝑎0 . An accepted replacement has 𝐴(𝑎) > 0 by Equation (D.5); retaining 𝑎0 gives 𝐴(𝑎0 ) = 0. Hence every decision has a true reference-relative score at least zero. Policy guarantee with estimated scores. The terminal values again coincide. Assuming improvement at every next-stage state and writing 𝑎𝐺 = 𝜋𝐺,𝑘 (𝑠), 𝑠𝐺 = 𝐹𝑘 (𝑠, 𝑎𝐺 ), we obtain 𝜋

𝜋

0 ≥ 𝑅𝑘 (𝑠, 𝑎𝐺 ) + 𝑉𝑘+1 (𝑠𝐺 )

Scores use accurate transitions, rewards, and reference values. Ties retain the reference whenever it maximizes the score. The candidate set may omit other feasible actions without weakening the reference-relative guarantee. Proof of Proposition 1. At stage 𝑇 , both policies have zero 𝜋𝑅 ′ 𝜋0 continuation value. Assume 𝑉𝑘+1 (𝑠 ) ≥ 𝑉𝑘+1 (𝑠 ′ ) for every reachable state at the next stage. Fix a stage-𝑘 state 𝑠 and abbreviate 𝑎𝑅 = 𝜋𝑅,𝑘 (𝑠), 𝑎0 = 𝜋0,𝑘 (𝑠), and 𝑠𝑅 = 𝐹𝑘 (𝑠, 𝑎𝑅 ). Then 𝜋

𝜋

𝑅 (𝑠𝑅 ) 𝑉𝑘 𝑅 (𝑠) = 𝑅𝑘 (𝑠, 𝑎𝑅 ) + 𝑉𝑘+1

(i)

(𝑖𝑖)

(D.4)

𝜋

≥ 𝑄𝑘 0 (𝑠, 𝑎0 ) 𝜋

= 𝑉𝑘 0 (𝑠). The outer equalities evaluate each policy. Step (i) uses the induction hypothesis at 𝑠𝑅 ; step (ii) uses score maximization 𝜋 𝜋 and 𝑎0 ∈ 𝒞𝑘 (𝑠). Backward induction gives 𝑉𝑘 𝑅 (𝑠) ≥ 𝑉𝑘 0 (𝑠) at every reachable stage and state. □ A gap to 𝑉𝑘∗ can remain because rollout scores use the reference continuation rather than an optimal continuation.

̂𝑘 estimate the same Bounded scoring error. Now let 𝑄 ̂𝑘 (𝑠, 𝑎) − 𝑄 𝜋0 (𝑠, 𝑎)| ≤ true scores. Assume a uniform bound |𝑄 𝑘 𝜖 with 𝜖 ≥ 0, covering every candidate and reference action at all states reachable under the considered policies. At a 𝜋0 (𝑎). ̂ fixed (𝑘, 𝑠), suppress those indices and set 𝑒𝑎 = 𝑄(𝑎)−𝑄 𝜋 𝜋 ̂ = 𝑄(𝑎) ̂ − 𝑄(𝑎 ̂ 0 ). Define 𝐴(𝑎) = 𝑄 0 (𝑎) − 𝑄 0 (𝑎0 ) and 𝐴(𝑎) The two score errors imply ̂ − 𝑒𝑎 ] − [𝑄(𝑎 ̂ 0 ) − 𝑒𝑎 ] 𝐴(𝑎) = [𝑄(𝑎) 0 ̂ − 𝑒 𝑎 + 𝑒𝑎 = 𝐴(𝑎) 0 ̂ − |𝑒𝑎 | − |𝑒𝑎 | ≥ 𝐴(𝑎) 0 ̂ − 2𝜖. ≥ 𝐴(𝑎)

𝜋

= 𝑄𝑘 0 (𝑠, 𝑎𝐺 )

(D.6)

𝜋 = 𝑉𝑘 0 (𝑠) + 𝐴(𝑎𝐺 ) 𝜋 ≥ 𝑉𝑘 0 (𝑠).

Backward induction therefore extends the local score condition to the value of the whole guarded policy. This argument requires the error bound wherever the policy can visit, including states reached after earlier replacements. □ The experiment uses accurate scores; bounds on online estimation errors require separate validation. D.4 Paired Comparisons and Resolution

𝜋

0 ≥ 𝑅𝑘 (𝑠, 𝑎𝑅 ) + 𝑉𝑘+1 (𝑠𝑅 )

𝜋 = 𝑄𝑘 0 (𝑠, 𝑎𝑅 )

𝜋

𝐺 𝑉𝑘 𝐺 (𝑠) = 𝑅𝑘 (𝑠, 𝑎𝐺 ) + 𝑉𝑘+1 (𝑠𝐺 )

(D.5)

The informative policy difference is 𝑉 𝜋𝑅 (𝑠) − 𝑉 𝜋0 (𝑠) under identical state and calendar inputs. Separate gaps to 𝑉 ∗ (𝑠) reveal whether both policies are suboptimal. Table D.2 reports the full grid and the finer-quantum check. All 91 strict improvements in the primary grid involve cold recovery, while all 960 warm-recovery cases tie with Costed Window. Rollout matches DP in 1907 cases; the other 13 have a maximum gap of 124.836 s-eq. A small mean gap therefore does not imply uniform near-optimality. Table D.2. Rollout versus Costed Window, with no regressions. Mean DP gaps use s-eq.; 1 s cases start fresh. Evaluation Full grid, 5 s Fresh only, 1 s

Wins / ties 91 / 1829 27 / 453

Costed Rollout gap Window gap 0.246 0

5.369 5.869

The 1 s experiment refines only the 480 fresh-candidate cases, holding their other parameters fixed. Its zero rollout gap establishes agreement with DP on that finer finite action set. It does not establish an optimum in continuous time, cover partially completed initial states, or prove that rollout is optimal for other calendars.

Related documents

• Serve Now or Improve Later? Scheduling Self-Evolution in Online Agent Systems#1202631
• A Test Taxonomy and Continuous Integration Ecosystem for Dynamic Resource Management in HPC#820444
• Which Self-Improvements Should We Trust? Reliable Self-Improvement When Agents Reuse Their Benchmarks#1117892
• Skill Blocks: How Should an Agent Load Its Skill? A Caching-Correct Comparison of Pre-load, On-Demand Tool-Loading, Progressive Disclosure, and Hybrid#609356
• Do Self-Evolving Skills Generalize to Held-Out Tasks?#1144115