AKTS: Sub-Microsecond Kernel Policy Switching for Language-Model Agents
arXiv:2609.12276v1 [cs.OS] 10 Sep 2026
Mohammadali Khodabandehlou∗ University of Southern California [email protected]
Mahdi Alizadeh University of Southern California [email protected]
Abstract GPU-backed LLM servers often multiplex interactive requests with background batch work on the same CPUs. During a request burst, the scheduler should protect time-to-first-token; between bursts, it should let background work make progress. A fixed kernel policy leaves one of these objectives on the table, so agentic OS control needs a way to switch scheduler behavior as the workload changes. The hard part is not deciding that a switch is useful, but applying it safely and fast enough for the kernel. Scheduler events occur every 1–10 µs, and any code that runs there must satisfy the eBPF verifier. Scalar knobs are fast but expose only limited policy behavior, while generating new eBPF policy code is expressive but puts compilation, verification, loading, and possible verifier rejection on the runtime path. We present AKTS, which verifies a policy library once, at load time and reduces the agent’s runtime action to writing an integer index into an in-kernel array of preverified policies. An in-kernel tail call resolves that index. Because the agent emits an index rather than code, verifier failure is not a runtime outcome. On Linux 6.14, AKTS applies a policy switch in 920 ns (p50), matching scalar writes while switching whole policies; makes an invalid index inert across 60,217 invocations on an attached scheduler; and switches policies in a vLLM workload to capture 97% of a throughput policy’s batch work while matching a latency policy’s burst response.
1
Introduction
Consider a GPU-backed LLM serving node. The GPU runs the model, but the CPUs still handle request intake, tokenization, prefill orchestration, batching, and background jobs. When traffic is quiet, the operator wants background work to make progress. When requests arrive in a burst, the same CPUs should prioritize the serving path so users see low time-to-first-token. A scheduler tuned for one regime loses the other: latency-oriented scheduling leaves batch work on the table, while throughput-oriented scheduling delays interactive requests. This is the application problem AKTS targets. The system should recognize the current regime at a coarse timescale and switch the kernel scheduling behavior that is best for that regime. The challenge is making that switch a safe and cheap kernel operation. Kernel schedulers make placement decisions every few microseconds, while even small language models need milliseconds to produce a decision. Placing inference on the scheduling path is therefore ruled out by construction. Prior work has explored in-kernel policy selection with eBPF [1] and adaptive scheduling for dynamic workloads [4, 10, 3]. The missing piece is the actuator, namely a safe, sub-microsecond mechanism that lets a slow agent switch among verified kernel policies. Agentic-OS work [14] has converged on two ways around the timing boundary. ∗
Both authors contributed equally.
Preprint.
reasoning plane (userspace, ∼100 ms cadence)
telemetry
agent (SLM)
one integer
policy library (C)
active slot k
eBPF verifier
read k
installed once, at load time
select_cpu hook
dispatcher
tail call to slot k
0: latency
1: throughput
2: (empty)
execution plane (kernel, µs) empty slot: tail call falls through to the default policy
Figure 1: AKTS. Policies are compiled and verified once, at load time (gray). The agent’s entire runtime action is writing the integer k (blue); the kernel tail-calls into the selected preverified policy on every scheduling event. An invalid k names an empty slot and is inert (red).
The first is scalar tuning. LumOS [6] has an LLM agent tune Completely Fair Scheduler hyperparameters, outperforming Bayesian optimization by 5–7% and a human expert by 2.98%. The agent writes numbers to knobs the kernel already exposes; applying a decision is a single write, so actuation is effectively free. The limitation is expressive, not temporal: these knobs only adjust behavior the kernel already exposes. The second is code synthesis. SchedCP [15] has an LLM generate sched_ext [8] scheduling policy code, which is then compiled, checked by the in-kernel verifier, and loaded, reporting up to 1.79× improvement on target workloads. This recovers full expressiveness, but it places compilation and verification on the actuation path, costs seconds to a minute per policy change, and admits a failure mode absent from scalar tuning because generated code can be rejected by the verifier and must be regenerated. Learned schedulers outside the LLM line face a related split: reinforcement-learning approaches [10, 11, 9] adapt policies but do not synthesize new mechanisms, delegation frameworks [3] expose scheduling without deciding policy, and LLM-generated kernel extensions [16] inherit the compile-and-verify path. AKTS (Agentic Kernel Tail-Call Stitching) removes that tension. A library of eBPF policies is compiled and verified once at load time, then installed into a program array. At runtime, a dispatcher on the kernel hook reads a single integer and tail-calls into the policy it names. The agent’s entire runtime output is that integer. This yields a safety property neither prior approach has. The agent’s action space is an index into programs that were already verified; there is no code path by which a hallucinated decision becomes an unsafe kernel. Asked to choose between policies 0 and 1, a 0.5B model in our evaluation answered "0.3"; under AKTS that output is inert. The worst outcome is a suboptimal choice, and an invalid choice names no program at all. This is complementary to LumOS’s transactional apply–commit–revert and SchedCP’s pre-deployment analysis: both validate or roll back a proposed action, whereas AKTS makes unsafe actions inexpressible. Program arrays and tail calls are long established in the networking datapath [2]; our contribution is their use as a constrained action space, and the demonstration that this composes with sched_ext. We contribute: (i) the mechanism, which decouples policy expressiveness from actuation cost by moving verification to load time (§2); (ii) three feasibility results on stock Linux 6.14, each bracketed by controls, establishing that tail-call dispatch verifies, populates and executes inside sched_ext (§3); and (iii) an evaluation showing actuation at parity with scalar tuning, an invalid decision proven inert on an attached scheduler, and a vLLM serving workload on which switching captures 97% of a throughput policy’s batch work while matching a latency policy’s burst response (§4).
2
Design
AKTS separates a reasoning plane running at the agent’s cadence from an execution plane running at the kernel’s cadence (Figure 1). The contract between the two planes is intentionally narrow. All executable kernel code is compiled and verified before the scheduler is attached; at runtime, the agent can only change the integer that selects one verified policy. 2
Execution plane. At load time, each candidate scheduling policy is compiled, verified, and installed into a BPF_MAP_TYPE_PROG_ARRAY. A dispatcher on the target hook reads k from a singleentry map and issues bpf_tail_call. The tail call transfers control to the selected program without growing the stack and consumes one of the 33 permitted tail calls per event. If k is out of range or names an empty slot, the helper returns and the dispatcher executes its default scheduling action. Reasoning plane. A small model consumes telemetry on a macro cadence, on the order of 100 ms, and writes k with bpf_map_update_elem. Between these writes, the kernel dispatches every scheduling event with no model call and no userspace round trip. Safety. The action space is closed: the agent selects among programs the verifier already accepted, so no output it can produce causes unverified code to execute. An incorrect valid index can choose the wrong policy; an invalid index chooses no policy. A designed but unimplemented in-kernel guard can reset the slot without waiting for the agent.
3
Feasibility on stock Linux
The construction rests on three assumptions about how sched_ext programs interact with tail calls. None is stated in the kernel documentation, and the design fails if any does not hold. All measurements in this paper use one host: an NVIDIA A100-SXM4-40GB (40 GB) with a 30-vCPU AMD EPYC 7J13 and 216 GB RAM, Ubuntu 24.04, Linux 6.14.0-27. Gates 1 and 2 are load-time-only checks with no scheduler attached. Gate 1: does bpf_tail_call verify inside a sched_ext program? sched_ext policies are struct_ops programs, invoked through BPF trampolines rather than as ordinary entry points, so tailcall support is a real compatibility question. It holds. We bracket the result with two controls: an XDP program containing the same tail call, isolating the toolchain, and the identical struct_ops program with the tail call removed, isolating everything but the tail call. All three load. The JITed size grows from 76 B to 176 B with the tail call, consistent with the prologue being emitted. Gate 2: can a sched_ext program populate a prog array? This does not follow from Gate 1. Entries in a PROG_ARRAY must agree on prog_type and attach_btf_id, and sched_ext programs bind attach_btf_id to a specific sched_ext_ops member. Insertion succeeds. The only obstacle we found is in userspace: libbpf refuses to load a struct_ops program that no map references. A second, never-registered map on the same member works around that restriction. Gate 3: runtime dispatch. Load-time acceptance does not establish that the tail call dispatches at runtime through a struct_ops trampoline. A failed bpf_tail_call raises no error; from outside, failure is indistinguishable from success. We therefore instrument both paths, placing one counter inside the target policy and another immediately after the tail call, so the two outcomes are mutually exclusive per invocation. On an attached sched_ext scheduler driving eight workers, control transferred on 64,160 of 64,160 invocations, with zero fall-throughs. Dispatch is exact, not merely typical.
4
Evaluation
Attached schedulers run with SCX_OPS_SWITCH_PARTIAL. Only processes explicitly placed in SCHED_EXT are managed by the BPF scheduler, which keeps unrelated system activity outside the scheduling experiment. 4.1
Actuation latency
We measure the cost of applying a policy change after a decision has already been made, isolating the actuator from the model. Table 1 compares three mechanisms. Parity with Baseline A is the intended outcome. Both arms are a single syscall, so neither can be meaningfully faster. AKTS matches scalar tuning on actuation while switching whole policies, 3
Table 1: Actuation latency: cost of applying a policy change once the decision is made. Linux 6.14, AMD EPYC 7J13. AKTS arm n=200,000; baseline arm n=20,000; warm. Arm
Mechanism
p50
Baseline A (LumOS-style) [6] Baseline B (SchedCP-style) [15] AKTS
/proc/sys scalar write recompile + verify + reload bpf_map_update_elem
p99
mean
1110 ns 1131 ns 1123 ns seconds – minutes 920 ns 950 ns 931 ns
which Baseline A cannot do at any latency. Against Baseline B the gap is large because compilation and verification sit on that system’s actuation path; AKTS moves both to load time. 4.2
Agent decision cost and decision validity
Actuation bounds how cheaply a decision can be applied. The agent bounds how often decisions can be made, and whether they are useful. Serving Qwen2.5-0.5B [12], the model class this design targets, on an A100 and constraining it to emit a single digit, median decision latency is 13.5 ms (n=30, temperature 0), fast enough for a coarse adaptation loop. Decision validity is another matter. Across three prompt formulations and two telemetry regimes, 16 of 48 decisions (33%) were not a valid index: the model returned "3" and, in one case, "0.3", not an integer. Nor does scale fix the underlying problem: with a stronger prompt, Qwen2.5 at 0.5B, 1.5B and 3B all emit valid indices, and all emit a constant one, answering identically for high-load and low-load telemetry (20/40 correct, i.e. chance; n=40 per model; p50 13.3–23.9 ms). Constraining decoding to {0, 1} [13] changes nothing, so the failure at these scales is the decision, not the output format. AKTS does not require the agent to be correct to preserve kernel safety; it requires only that an incorrect agent cannot reach an unsafe state. The observed failure mode is exactly the case §4.3 measures, at a rate that makes the guarantee load-bearing. Whether some model routes well remains open. 4.3
Safety under faulty decisions
We point the dispatcher at a never-populated slot, simulating the out-of-range decision observed above. All 60,217 invocations fell through to the dispatcher’s default path, which calls scx_bpf_select_cpu_dfl. The system therefore reverts to stock CPU selection [7]. All workers completed, dmesg logged no panics or BUGs, and the scheduler detached cleanly. Since every populated slot was verified at load time, an arbitrary integer either names a verified policy or names nothing. Naming nothing is inert. The failure mode is reversion to default scheduling, not undefined behavior. 4.4
Does switching beat any fixed policy?
We serve Qwen2.5-0.5B under vLLM 0.28.0 [5] pinned to four cores, with eight CPU-bound antagonists on the same cores, driving a steady → burst → steady request pattern (8 s per phase, concurrency 2 and 32). We first verify the precondition for scheduler intervention. Contention inflates steady-state time-tofirst-token (TTFT) p99 from 27.0 ms to 308.0 ms. We measure TTFT because end-to-end latency is dominated by GPU decode and would mask the effect. The operative mechanism is contention delaying request handling and prefill scheduling, not KV-cache allocation, which policy choice cannot influence. Under contention, the latency policy wins TTFT in every phase, so TTFT alone admits no trade-off; the antagonists’ completed work is the second objective. The latency policy retains only 91% of the batch throughput. The throughput policy gives the most batch work but has 59% worse burst TTFT p50 and serves 35% fewer requests. AKTS matches the latency policy’s burst response within one standard deviation while retaining 97% of the throughput policy’s batch work. 4
Table 2: Two competing objectives under CPU contention, mean ± std over seven runs per arm. Neither fixed policy is best on both. Switching is oracle-driven.
burst TTFT p50 (ms) burst requests served total batch work (Mit)
5
static latency
static throughput
AKTS switching
225.8 ± 9.5 1132 ± 34 1.789 ± 0.006
358.0 ± 4.0 738 ± 13 1.966 ± 0.018
229.0 ± 4.6 1129 ± 26 1.912 ± 0.007
Conclusion and future work
AKTS shows that agentic OS control does not have to choose between cheap scalar tuning and slow code generation. By verifying a policy library once and exposing only an integer selector at runtime, AKTS makes scheduler-policy switching a sub-microsecond kernel operation while keeping verifier failure off the runtime path. Our Linux 6.14 results show that tail-call dispatch works inside sched_ext, invalid indices are inert on an attached scheduler, and oracle switching can combine the burst response of a latency policy with most of the batch work of a throughput policy. The next step is to close the loop. The switching experiment is oracle-driven, so it is an upper bound on what an online detector or agent can achieve. An agent-driven arm must show that workload regimes can be detected in time and that a language model improves over a contextualbandit baseline; the models we tested do not yet route reliably on telemetry. Future work should also add the in-kernel circuit breaker described in §2, study larger policy libraries and workloads, and evaluate across more hosts and burst shapes. Code and raw measurement data: https: //github.com/mali-kh/akts.
References [1] Mahdi Alizadeh and Ramesh Govindan. eBPF-based bandit selection for adaptive video streaming. In Proceedings of the ACM SIGCOMM 2026 Conference, pages 2183–2185. ACM, 2026. [2] Toke Høiland-Jørgensen, Jesper Dangaard Brouer, Daniel Borkmann, John Fastabend, Tom Herbert, David Ahern, and David Miller. The express data path: Fast programmable packet processing in the operating system kernel. In CoNEXT, 2018. [3] Jack Tigar Humphries, Neel Natu, Ashwin Chaugule, Ofir Weisse, Barret Rhoden, Josh Don, Luigi Rizzo, Oleg Rombakh, Paul Turner, and Christos Kozyrakis. ghOSt: Fast & flexible user-space delegation of Linux scheduling. In SOSP, 2021. [4] Mohammadali Khodabandehlou, Jared Coleman, and Bhaskar Krishnamachari. Poster abstract: Scheduling dynamic IoT task graphs. In Proceedings of the 23rd ACM Conference on Embedded Networked Sensor Systems, pages 624–625. ACM, 2025. [5] Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large language model serving with PagedAttention. In SOSP, 2023. [6] Georgios Liargkovas, Vahab Jabrayilov, Hubertus Franke, and Kostis Kaffes. An expert in residence: LLM agents for always-on operating system tuning. In NeurIPS Workshop on Machine Learning for Systems, 2025. [7] EEVDF scheduler: Earliest eligible virtual deadline first. Linux kernel documentation, https: //docs.kernel.org/scheduler/sched-eevdf.html, 2024. [8] sched_ext: BPF extensible scheduler class. Linux kernel documentation, https://docs. kernel.org/scheduler/sched-ext.html, 2024. Merged in Linux 6.12. [9] Hongzi Mao, Parimarjan Negi, Akshay Narayan, et al. Park: An open platform for learningaugmented computer systems. In NeurIPS, 2019. 5
[10] Hongzi Mao, Malte Schwarzkopf, Shaileshh Bojja Venkatakrishnan, Zili Meng, and Mohammad Alizadeh. Learning scheduling algorithms for data processing clusters. In SIGCOMM, 2019. [11] Haoran Qiu, Subho S. Banerjee, Saurabh Jha, Zbigniew T. Kalbarczyk, and Ravishankar K. Iyer. FIRM: An intelligent fine-grained resource management framework for SLO-oriented microservices. In OSDI, 2020. [12] Qwen Team. Qwen2.5 technical report. arXiv preprint arXiv:2412.15115, 2024. [13] Brandon T. Willard and Rémi Louf. Efficient guided generation for large language models. arXiv preprint arXiv:2307.09702, 2023. [14] Huaizheng Zhang, Lei Zhang, Yuanming Li, Yizheng Huang, Xiaotong Yang, Kuntai Du, Yihua Cheng, Junchen Jiang, and Wencong Xiao. InfraGym: Empowering LLM agents for real-world computer system optimization. In NeurIPS Workshop on Machine Learning for Systems, 2025. [15] Yusheng Zheng, Yanpeng Hu, Wei Zhang, and Andi Quinn. Towards agentic OS: An LLM agent framework for Linux schedulers. In NeurIPS Workshop on Machine Learning for Systems, 2025. Also arXiv:2509.01245. [16] Yusheng Zheng, Yiwei Yang, Maolin Chen, and Andrew Quinn. Kgent: Kernel extensions large language model agent. In ACM SIGCOMM Workshop on eBPF and Kernel Extensions, 2024.
6