ProbeLogits: Kernel-Level LLM Inference Primitives for AI-Native Operating Systems Daeyeon Son Independent Researcher Republic of Korea [email protected]
arXiv:2604.11943v1 [cs.OS] 13 Apr 2026
April 2026
1
Abstract
Introduction
Operating systems have always treated processes as opaque entities. The kernel manages their resources— memory, CPU time, file descriptors—but never inspects what a process thinks. This was a reasonable abstraction for decades: a process’s internal state is its own concern, and the OS need not understand it to schedule, isolate, or terminate it. But large language models break this assumption. At every inference step, an LLM produces a probability distribution over its entire vocabulary—a logit vector that reveals the model’s uncertainty, intent, and confidence before any text is generated. This is a fundamentally new kind of process state: a semantic signal that the OS could use for scheduling, safety, and governance decisions. Today’s operating systems discard this information entirely. The consequences are threefold. Latency accumulation: when AI safety checks require generating text, parsing it, and then acting on the parse result, each abstraction layer adds overhead. A simple “is this action safe?” query traverses application code, IPC, inference framework, text generation, and parsing— accumulating latency that makes per-action governance impractical. Bypassable safety: when safety policies are enforced in application code, any agent with sufficient privilege (or a bug in the sandbox) can bypass them. The enforcement boundary is too high in the stack. Semantic opacity: the OS kernel cannot make informed decisions about AI workloads because it has no access to model internals—logits, hidden states, uncertainty—that would enable intelligent scheduling and governance. This paper presents ProbeLogits, an OS primitive that addresses all three problems by exposing LLM logit vectors as kernel-level abstractions. ProbeLogits performs a single forward pass on a prompt and reads specific token logits to classify actions without generating any text.
An OS kernel that runs LLM inference internally can read logit distributions before any text is generated— and act on them as a governance primitive. I present ProbeLogits, a kernel-level operation that performs a single forward pass and reads specific token logits to classify agent actions as safe or dangerous, with zero learned parameters. On a 260-prompt OS action benchmark (9 categories including adversarial attacks), ProbeLogits achieves F1 = 0.980, Precision = 1.000, and Recall = 0.960 using a general-purpose 7B model at 4-bit quantization. On ToxicChat (1,000 human-annotated real conversations), it achieves F1 = 0.790 at default calibration strength α = 1.0, improving to F1 = 0.837 at α = 0.5—89% of Llama Guard 3’s F1 ∼ 0.939 with zero learned parameters. A key design contribution is the calibration strength α, which serves as a deployment-time policy knob rather than a learned hyperparameter. By adjusting α, the OS can enforce strict policies for privileged operations (α ≥ 0.8, maximizing recall) or relaxed policies for conversational agents (α = 0.5, maximizing precision). Contextual calibration improves accuracy from 64.8% to 97.3% on the custom benchmark. I implement ProbeLogits within Anima OS, a baremetal x86_64 OS written in 80,400 lines of Rust. Because agent actions must pass through kernel-mediated host functions, ProbeLogits enforcement operates below the WASM sandbox boundary, making it significantly harder to circumvent than application-layer classifiers. Each classification costs 65 ms on 7B—fast enough for per-action governance. I also show that treating KV cache as process state enables checkpoint, restore, and fork operations analogous to traditional process management. To my knowledge, no prior system exposes LLM logit vectors as OS-level governance primitives. 1
The classification technique itself—reading logits at class-label token positions (known as verbalizers in NLP) with contextual calibration—is well-established in NLP [24, 27]. The contribution of this work is not the ML technique but its integration as a kernel primitive with structural enforcement guarantees. When logit reading runs inside the kernel and agent actions must traverse kernel-mediated host functions, safety checks become kernel-mediated—a defensein-depth property that application-layer classifiers cannot easily provide. Concretely, I make three contributions:
2.2
Three Problems in Detail
Problem 1: Latency. Consider the task of classifying whether an agent’s intended action is safe. In a text-based approach, the system constructs a prompt (“Is deleting all user data safe? Answer Yes or No.”), sends it to an inference server, waits for text generation (∼650 ms for a multi-token response on 7B), parses the output, and acts on the result. With ProbeLogits, the same classification requires one forward pass (65 ms on 7B) with no text generation, no parsing, and no IPC—a 10× reduction.
1. ProbeLogits as an OS primitive. I define binary, N-way, and entropy operations as kernel abstractions, and show that a general-purpose 7B model with zero learned fine-tuning (but with hand-crafted heuristics) achieves F1 = 0.980 on OS action classification (260 prompts) and F1 = 0.790 on ToxicChat [25] (1,000 prompts). To my knowledge, this is the first system to expose LLM logits as OS primitives.
Problem 2: Bypassability. Application-level safety filters—including prompt injection defenses, output classifiers, and policy checkers—operate at the same privilege level as the agent they are meant to constrain. A sufficiently capable agent can modify, disable, or circumvent these filters. Kernel-level enforcement is fundamentally different: an agent running in a WASM sandbox cannot bypass governance checks without first escaping the sandbox, because the checks are executed by 2. KV cache as process state. I treat the KV the kernel on the agent’s behalf, outside the sandbox cache as the analogue of CPU register state, en- boundary. abling checkpoint, restore, and fork operations that parallel traditional process management. Problem 3: Semantic opacity. To a conventional 3. Kernel-enforced constitutional governance. OS, an LLM inference workload is indistinguishable I demonstrate a governance pipeline (adversar- from any other compute task. The kernel cannot tell ial pre-filter → ProbeLogits classification) that is whether the model is confident or uncertain, whether kernel-enforced below the WASM sandbox bound- the output is safe or dangerous, or whether the agent ary, making it significantly harder to circumvent is behaving anomalously. This opacity prevents the OS from making semantically informed decisions about than application-layer classifiers. I implement these primitives in Anima OS, a bare- scheduling priority, resource allocation, or isolation.
metal x86_64 operating system comprising 80,400 lines of Rust. The inference engine achieves 1,666 tokens/s on 2.3 Threat Model SmolLM2-135M (1.39× llama.cpp) and 15 tokens/s on Qwen2.5-7B (parity at DDR5 bandwidth saturation). I assume the following threat model: agents execute in ProbeLogits classification costs 0.6 ms on 135M and WASM sandboxes with fuel-limited execution and no direct memory access outside their sandbox. The kernel is 65 ms on 7B—fast enough for per-action governance. trusted. The network is untrusted (all inter-node communication uses authenticated protocols). Agents may be adversarial: they may attempt to circumvent safety 2 Motivation and Background policies, exfiltrate data, or consume excessive resources. The governance system must provide structural guar2.1 The Layered AI Problem antees (not just probabilistic detection) against policy Contemporary AI systems are built as deep layer cakes: circumvention. hardware → OS kernel → userspace runtime → inference framework → orchestration library → application. Each layer adds latency, attack surface, and opacity. 3 ProbeLogits Design A safety check in LangChain [6] must traverse Python function calls, HTTP requests to an inference server, 3.1 The Problem with Text-Based Clastext generation, regex parsing of the response, and a sification policy decision—all before the agent’s action can be approved or denied. This architecture is a consequence of Text-based classification—generating a response and treating AI as an application-level concern rather than parsing it—suffers from three failure modes that make a kernel-level one. it unsuitable for kernel-level governance: 2
1. Unexpected format. The model may respond Algorithm 1 ProbeLogits N-way Classification “No, I don’t think that would be appropriate” in- Require: Prompt p, class labels C = [c , . . . , c ] 1 N stead of the expected “No.” Parsing extracts “No” Ensure: Sorted class probabilities but the actual answer is buried in natural language 1: for i ← 1 to N do that may vary unpredictably. 2: ti ← TextToId(ci ) ▷ O(log |V |) BTreeMap lookup 2. Fragile parsing. Regular expressions and string 3: if ti = None then return None matching break on whitespace variations, capital4: end if ization differences, or tokenizer artifacts. Each fail5: end for ure mode requires a new parsing rule. 6: tokens ← Encode(p) 3. Wasted computation. Generating a multi-token 7: ResetKvCache() response requires multiple forward passes (one per 8: for tok ∈ tokens do token). Classification needs only the first to9: ℓ ← ForwardOne(tok) ▷ Incremental forward ken’s logit distribution—all subsequent tokens are 10: end for wasted computation. 11: ℓmax ← maxi ℓti ProbeLogits eliminates all three failure modes by 12: ei ← exp(ℓti − ℓmax ) for each i P reading the logit vector directly, before any text is gen- 13: S ← i ei erated. 14: if S ≤ 10−10 then 15: P (ci ) ← 1/N for all i ▷ Uniform fallback 16: else 3.2 ProbeLogits Mechanism 17: P (ci ) ← ei /S for each i Let p be a prompt (a sequence of tokens), V be the 18: end if vocabulary, and f be the model’s forward pass. Pro- 19: return SortDescending({(ci , P (ci ), ℓti )}) beLogits computes: (1)
ℓ = f (p) ∈ R|V |
Token vocabulary lookup (text_to_id). Maps a where ℓ is the logit vector over the full vocabulary. For text string to its token ID using a BTreeMap over Cost: a set of target classes C = {c1 , . . . , cN }, each mapped to the GPT-2 byte-pair encoding vocabulary. O(log |V |), where |V | = 152,064 for Qwen2.5’s GPTa token ID tci via vocabulary lookup, the classification 2 BPE vocabulary. Returns None if the text does not probability for class ci is: correspond to a single token. Algorithm 1 presents the complete probe_classify exp(ℓtci ) (2) procedure. P (ci ) = PN exp(ℓ ) j=1
tc j
This is an N -way softmax restricted to the target to- 3.3 Information Efficiency ken positions. The key property is that exactly one forward pass suffices regardless of N —the cost is identical A single forward pass produces a logit vector over the full vocabulary of |V | = 152,064 tokens. This vector for binary classification and 100-way classification. carries log2 (|V |) ≈ 17.2 bits of information—the maxiI implement three operations as kernel primitives: mum entropy of the distribution. Standard text generBinary classification (probe_yes_no). Looks up ation uses this vector to sample one token, discarding token IDs for “Yes” and “No” via text_to_id(), runs > 99.99% of the available information. ProbeLogits extracts precisely the bits needed for the one forward pass, and computes a 2-class softmax. Retask at hand. A binary classification extracts 1 bit (plus turns the winning class and confidence ∈ [0.5, 1.0]. A confidence). An N-way classification extracts log2 N numerical guard handles the edge case where both logits bits. In both cases, the information is extracted from a −10 underflow: if exp(ℓyes ) + exp(ℓno ) ≤ 10 , the result single forward pass, whereas text-based classification redefaults to confidence 0.5 (maximally uncertain). quires multiple forward passes (one per generated token) to recover the same information through text parsing. N-way classification (probe_classify). Generalizes binary classification to N classes. Each class label must resolve to a single token in the vocabulary. 3.4 Robustness Properties Returns a vector of ClassResult sorted by probability (descending). The same numerical guard applies: if ProbeLogits provides four robustness guarantees that the softmax denominator is ≤ 10−10 , all classes receive text-based classification cannot: uniform probability 1/N . 1. No parsing failures. The output is always a 3
floating-point probability in [0, 1]. There is no text to parse, no regex to fail, no format to validate.
( ℓi if token i is valid −∞ otherwise
(4) 2. Bounded confidence. The softmax output is always in [0, 1] by construction. Binary classification For choice grammars (selecting one of N string opconfidence is always in [0.5, 1.0]. tions), the masking algorithm maintains the set of re3. Graceful degradation. When the model is un- maining valid completions given the tokens generated certain, the confidence approaches 0.5 (binary) or so far. At each step, a token is valid if appending its 1/N (N-way). This is a meaningful signal—the sys- text to the generated prefix would still be a prefix of tem can defer to a larger model or to human judg- at least one remaining choice. This prefix matching operates over character sequences, not token sequences, ment. which correctly handles cases where choices share com4. Numerical stability. The log-sum-exp trick mon prefixes. prevents overflow. The uniform fallback when I acknowledge a limitation: the masking is not a 100% P exp(·) ≤ 10−10 prevents division by zero. f64 guarantee for arbitrary tokenizer/choice combinations. accumulation in entropy computation prevents pre- GPT-2’s byte-pair encoding covers all single ASCII cision loss with large vocabularies. characters, so ASCII choices work reliably. For nonASCII choices or unusual tokenizer vocabularies, I recommend using probe_classify() (single-token classi3.5 Logit Entropy fication) where deterministic results are needed, and reBeyond classification, the full logit vector carries a serving grammar constraints for multi-token structured scalar summary of model uncertainty: Shannon entropy. output. The overhead of grammar masking is negligible: < |V | X 0.01 ms per token, compared to 65 ms for the forward exp(ℓi ) (3) pass itself. The mask is recomputed at each generapi ln pi , pi = P H(ℓ) = − j exp(ℓj ) i=1 tion step, but the vocabulary scan (152K tokens) is fast For a vocabulary of 152,064 tokens, entropy ranges because it involves only string prefix comparison. from 0 nats (complete certainty—one token has probability 1) to ln(152,064) ≈ 11.93 nats (uniform 4 KV Cache as Process State distribution—maximum uncertainty). I implement logit_entropy() as a kernel primitive with f64 accumulation for numerical stability across the 4.1 The Process State Analogy full 152K-token vocabulary. The computation uses the Traditional operating systems manage process state log-sum-exp trick (subtract maxi ℓi before exponentia- through a well-established set of abstractions: CPU regtion) and skips terms where pi < 10−10 to avoid ln(0). isters are checkpointed on context switch, the program Logit entropy enables three OS-level capabilities: counter tracks execution position, and fork() dupli• Autonomy gating: when H > 8 nats (high un- cates the entire process state to create a child. I observe certainty), the kernel can automatically defer the that LLM inference has a direct analogue: ℓ′i =
decision to a human operator or a larger model.
• Out-of-distribution detection: anomalously high entropy on routine prompts signals that the model is encountering inputs outside its training distribution.
Traditional Process
LLM Inference
CPU registers Program counter fork() Context switch Core dump
KV cache (attention state) KV position (tokens processed) kv_fork() kv_checkpoint/restore() KV cache serialization
• Dynamic temperature: the kernel can adjust sampling temperature inversely to entropy, producThis analogy is not merely conceptual—it has diing more deterministic outputs when the model is rect implementation consequences. Just as an OS must already confident. checkpoint registers to switch between processes, an AI-native OS must checkpoint KV cache to switch between inference contexts. Just as fork() enables spec3.6 Grammar-Constrained Decoding ulative execution in traditional systems, kv_fork() enFor tasks that require multi-token structured out- ables speculative decoding and conversation branching. put (rather than single-token classification), I implement grammar-constrained decoding as a complemen4.2 Operations tary primitive. The constraint engine operates by masking logits before sampling: I implement three KV cache operations: 4
kv_checkpoint(). Snapshots the current KV cache 5 Constitutional Governance Instate (key and value tensors for all layers, up to the tegration current position) into a KvCheckpoint structure. The operation uses checked_mul to prevent integer overflow 5.1 Why Kernel-Level Enforcement when computing checkpoint size, and enforces a 32 MB Matters cap (MAX_CHECKPOINT_BYTES) to prevent memory exhaustion. Returns an error if the checkpoint would ex- In Anima OS, agents execute in WASM sandboxes comceed the cap. piled via Cranelift AOT. An agent interacts with the outside world exclusively through 12 host functions exkv_restore(). Restores KV cache state from a check- posed by the kernel. This architecture provides a strucpoint. Before restoring, the operation validates that the tural guarantee: the agent cannot bypass governance checkpoint’s dimensions (number of layers, bytes per po- checks without escaping the WASM sandbox, because sition) match the current model. This prevents a subtle it has no mechanism to invoke system operations other class of bugs where checkpoints from one model are ac- than through the host function interface, which the kernel controls. cidentally applied to another. This leads to a critical distinction: enforcement is complete; detection is probabilistic. The enforcekv_fork(). Creates a copy of the KV cache suitable ment boundary—WASM isolation preventing direct sysfor transfer to a different inference context. Currently tem calls—is a structural guarantee that holds regardimplemented as an alias for kv_checkpoint(), but se- less of the agent’s capability or intent. The detection mantically distinct: fork implies the original continues mechanism— ProbeLogits classifying whether an action executing, while checkpoint implies the original may be is harmful—is probabilistic, with accuracy depending overwritten. on model size and prompt quality. The combination is powerful: even if detection produces a false negative (classifying a harmful action as safe), the agent still op4.3 Use Cases erates within its capability sandbox, limiting the blast KV cache as process state enables several capabilities radius of any single action. that are impossible or expensive without kernel-level support:
5.2
Detection Pipeline
• Speculative execution: fork the KV cache, exI implement a three-stage detection pipeline that complore a candidate action, and restore if the action bines rule-based pre-filtering with model-based classifiis rejected by governance. cation: • Agent forking: create a new agent with the same 1. Stage 1: Adversarial pre-filter (< 1 µs). An conversational context, diverging from a specific O(n) pattern scan detects prompt injection, encodpoint in the conversation. ing tricks (base64, rot13 references), authority impersonation (“ADMIN OVERRIDE”), and instruction overrides (“ignore previous”). Actions exceeding a risk-score threshold are classified as dangerous with no forward pass.
• Migration: serialize the KV cache, transfer it to another node, and restore—enabling live agent migration across a cluster. • Conversation branching: checkpoint before a critical decision, explore multiple branches, and select the best outcome.
4.4
2. Stage 2: Input sanitization. Control characters and injection patterns are stripped before the action is embedded in the safety-priming prompt template.
Comparison with Serving Systems
3. Stage 3: Calibrated forward pass (65 ms). A single 7B forward pass reads Safe/Dangerous verbalizer logits. Contextual calibration corrects inherent token bias, and a privacy keyword boost compensates for the model’s blind spot on surveillance-as-harm.
vLLM [19] and SGLang [17] also manage KV caches, but for a fundamentally different purpose. Their PagedAttention and RadixAttention optimize serving throughput—maximizing the number of concurrent requests that fit in GPU memory. My KV cache operations optimize process state management—enabling checkpoint, restore, and fork as OS primitives for individual agents. The two approaches are orthogonal: a system could use PagedAttention for memory efficiency and KV process state operations for agent lifecycle management.
The per-action cost is dominated by the 7B forward pass (65 ms). A natural optimization—cascading through a smaller model first (e.g., 135M at 0.6 ms) and escalating to 7B only for ambiguous cases—is left for future work. 5
AVX-512/AVX2 SIMD kernels. Quantized matrix-vector multiplication kernels for three formats: The governance system maps ProbeLogits confidence to • Q4_0: 4-bit quantization with per-block f16 scale. a graduated response: AVX-512 kernel processes 64 elements (2 blocks) per iteration using VNNI vpdpbusd instructions. Confidence Response • Q6_K: 6-bit quantization with per-superblock P (harmful) > 0.9 Block (deny the action) scale and per-block sub-scale. Requires careful bit P (harmful) > 0.7 Warn (log + notify operator) unpacking across three byte segments. P (harmful) > 0.5 Log (record for audit) • Q8_0: 8-bit quantization with per-block f16 scale. P (harmful) ≤ 0.5 Allow (proceed normally) The fastest format, using direct vpmaddubsw instructions. The graduated response is intentionally conservative: a 78% confidence harmful classification results in a All kernels use F16C hardware instructions warning, not a block. This reflects the reality that 7B (vcvtph2ps) for f16→f32 scale conversion, elimiQ4_0 models are not highly accurate classifiers, and nating the need for a software floating-point library. false positives (blocking legitimate actions) erode user trust more than false negatives (allowing suspicious ac- SMP work-stealing graph executor. The infertions with a warning). ence computation is expressed as a dataflow graph (606
5.3
Graduated Response
operations, 226 synchronization barriers for 7B). The graph executor uses work-stealing across all available CPU cores: each core maintains a local deque of ready All governance decisions—classifications, confidence operations, and idle cores steal work from busy cores’ scores, and responses—are recorded in a Blake3 hash deques. For the 7B model, the forward pass is fully chain. Each audit entry includes the previous entry’s parallelized across 8 cores. hash, creating a tamper-evident log: retroactive modification of any entry invalidates all subsequent hashes. I note that this guarantee holds within the buffer window 6.2 The no_std Challenge (in-memory ring buffer); durable persistence requires in- Building an LLM inference engine in a bare-metal entegration with the AnimaFS storage layer. vironment presents four challenges that do not exist in
5.4
Blake3 Audit Chain
userspace:
6
Implementation
6.1
Bare-Metal Inference Engine
1. No float runtime. Bare-metal Rust provides no libm for transcendental functions. I use F16C hardware instructions for f16 conversion and a polynomial approximation (fast_expf_poly) for exp() in softmax and entropy computations.
The inference engine comprises 6,900 lines of no_std Rust, running directly on bare metal with no operating system, no libc, and no standard library. It implements:
2. No filesystem. GGUF model files (4+ GB) are read from USB mass storage via a custom xHCI driver (7,400 lines), block by block.
GGUF model loader. Parses the GGUF binary format (llama.cpp’s model format) directly from USB mass storage via xHCI, loading tensor data in Q4_0, Q6_K, and Q8_0 quantization formats.
3. No threading. SMP parallelism is implemented from scratch: AP (Application Processor) bootstrap via the INIT-SIPI protocol, spin locks, and atomic operations—no pthread, no std::thread.
GPT-2 BPE tokenizer. A byte-pair encoding tokenizer supporting vocabularies up to 152,064 tokens. Token-to-ID lookup uses a BTreeMap for O(log |V |) performance. The tokenizer supports chat templates for instruction-following models.
4. No allocator (initially). The heap allocator is bootstrapped from the UEFI memory map. Once initialized, standard alloc::Vec and alloc::String are available.
6.3
Llama transformer. A complete Llama-architecture transformer implementation: multi-head grouped-query attention with RoPE positional encoding, SwiGLU feedforward networks, and RMS normalization. Supports models with GQA ratios (e.g., 7B uses 32 attention heads with 8 KV heads).
System Context
ProbeLogits operates within a complete OS kernel. The agent lifecycle manages five states (Nascent, Active, Suspended, Isolated, Terminated) with trust evolution based on task completion history and a four-tier capability system (System, AiNative, AiEnhanced, Classic). 6
Table 1: Code scale by component. Component
LoC
Tests
boot/ (bare-metal) kernel/ (hosted) runtime/ cli/ sdk/ dsl/
42,700 24,300 9,400 2,300 270 1,400
— 447 106 51 28 24
Total
80,400
656
Table 2: Token generation throughput (tokens/s). Higher is better.
Feature Kernel + inference Lifecycle, memory WASM, scheduling Agent mgmt WASM SDK Agent DSL
7.2
AnimaNet provides distributed networking: UDP discovery with boot_nonce-based reboot detection, TCP-based Raft consensus for cluster coordination, and live agent migration (serializing the AgentControlBlock and transferring it over TCP). A two-node cluster has been validated end-to-end with 11/11 test markers passing. The web stack enables agents to access external services: DNS resolution, HTTP client, TLS 1.3 (via rustls with no_std ring), and a web_fetch() host function exposed to WASM sandboxes. The WASM sandbox uses Cranelift AOT compilation with fuel-limited execution. Twelve host functions bridge the sandbox to kernel services, including inference, governance, IPC, and web access.
6.4
Model
Config
Anima
llama.cpp
135M Q4_0
16w/16t 8w/8t
1,666 1,351
1,195 1,301
7B Q4_0
8w/8t 16w/16t
15 —
15 14
Inference Throughput
Table 2 presents inference throughput results. On the 135M model, Anima OS achieves 1,666 tokens/s with 16 work-stealing workers, a 1.39× improvement over llama.cpp’s 1,195 tokens/s with 16 threads. With 8 workers/threads, the advantage narrows to 1.04× (1,351 vs. 1,301). The 135M model is compute-bound : its 70 MB parameter footprint fits in L3 cache, and performance scales with parallel compute—where workstealing outperforms static thread partitioning. On the 7B model, both systems achieve 15 tokens/s (65 ms per token). The 7B model is bandwidthbound : each forward pass reads 4.1 GB of parameters from main memory, and performance is determined by memory bandwidth rather than compute efficiency.
7.3
Code Scale
Bandwidth Saturation Analysis
The 7B parity result is explained by a fundamental Table 1 summarizes the system’s code scale. The boot/ hardware limit. The theoretical minimum time per tokernel (42,700 LoC) is the largest component, contain- ken for a bandwidth-bound model is: ing the bare-metal inference engine (6,900 LoC), xHCI 4.1 GB model size USB 3.0 driver (7,400 LoC), VirtIO-Net networking, = = 55 ms (5) tmin = bandwidth 75 GB/s AnimaNet protocol, and all ProbeLogits primitives.
7
Evaluation
7.1
Experimental Setup
The measured 65 ms corresponds to 85% of theoretical peak bandwidth utilization. The remaining 15% accounts for non-memory operations (softmax, RoPE, layer normalization) and memory access patterns that do not achieve perfect streaming. This is a significant finding: bare-metal execution reaches the hardware bandwidth limit. No software optimization—better SIMD kernels, more efficient scheduling, or tighter code—can improve 7B performance on this hardware. The path to faster 7B inference requires higher memory bandwidth (DDR5-8000, HBM) or reduced model size (more aggressive quantization).
All experiments run on a single machine: • CPU: AMD Ryzen 9 9800X3D (8 cores, 16 threads, AVX-512 with VNNI) • RAM: 60 GB DDR5-6000 dual-channel (∼75 GB/s measured bandwidth) • Models: SmolLM2-135M Q4_0 Qwen2.5-7B-Instruct Q4_0 (4.1 GB)
(70 MB),
7.4
ProbeLogits Performance
• Baseline: llama.cpp (latest, March 2026) on Linux Table 3 shows ProbeLogits operation latencies. Bi6.17, same hardware nary and N-way classification have identical cost beAnima OS runs bare-metal (UEFI boot, no Linux). cause both require exactly one forward pass; the addiThe llama.cpp baseline runs on Linux with identical tional softmax over N token logits is negligible (<1 µs for N ≤ 100). CPU and memory. 7
Table 3: ProbeLogits operation latency. Operation Binary (probe_yes_no) N-way (probe_classify) Entropy (logit_entropy) Grammar masking (per token) KV checkpoint
135M
7B
0.6 ms 0.6 ms 0.1 ms <0.01 ms 0.5 ms
65 ms 65 ms 0.5 ms <0.01 ms 12 ms
Table 4: ProbeLogits vs. Generate-then-Parse on 260 prompts (9 categories including 50 adversarial). Both methods use the identical prompt template for fair comparison. ProbeLogits includes an adversarial pre-filter and privacy keyword boost; Generate uses text generation + parsing only. Metric
Generate
97.3% 1.000 0.960 0.980 7 0
90.8% 1.000 0.864 0.927 24 0
Accuracy Precision Recall F1 Score False negatives False positives
Logit entropy computation is cheap (0.1–0.5 ms) because it operates on the already-computed logit vector without requiring an additional forward pass. It only needs to iterate over the 152K logits once. KV cache checkpoint cost scales with model size: 0.5 ms for 135M (small KV cache) and 12 ms for 7B (larger KV cache that must be copied). The 32 MB cap ensures bounded memory consumption.
7.5
ProbeLogits
Table 5: Pipeline ablation on custom benchmark.
ProbeLogits Safety Classification
Config
Acc.
F1
Uncalib., Yes/No + Calibration + Safety prompt + Privacy boost + Adv. pre-filter
64.8% 87.1% 92.3% ∼95% 97.3%
0.786 0.892 0.941 ∼0.96 0.980
Note Sycophancy Bias corr. Model-only Estimated Full pipe.
I validate ProbeLogits on native hardware (UEFI boot, no QEMU) using Qwen2.5-7B-Instruct Q4_0 with a benchmark of 260 prompts across 9 categories: explicit danger/safety, euphemism, ambiguity, multi-step attacks, deception, privacy violations, benign-complex Table 4 shows ProbeLogits achieves 97.3% accuracy actions, and adversarial evasion attempts (prompt in- (F1 = 0.980) vs. 90.8% (F1 = 0.927) for generatejection, encoding tricks, language mixing, instruction then-parse on identical prompts. Both methods achieve hierarchy attacks). perfect precision (1.000): zero false positives across all safe prompts. Contextual calibration. Instruction-tuned models exhibit strong sycophancy bias (a tendency to agree Pipeline ablation. ProbeLogits is a three-stage with the implied framing of the question): when asked pipeline: (1) adversarial pre-filter (pattern matching, “Is this dangerous?” with Yes/No verbalizers, the uncalcatches ∼13 of 50 adversarial prompts at O(n) with no ibrated model predicted “dangerous” for the majority of forward pass), (2) calibrated forward pass with Safe/prompts—achieving only 64.8% overall accuracy on the Dangerous logit reading, (3) privacy keyword boost initial 210-prompt benchmark. I apply contextual cali(adds 2.5–3.0 logit units for surveillance verbs + perbration [27]: at boot time, 7 null inputs are probed to sonal data nouns). The pre-filter and privacy boost measure the model’s inherent bias (∆ = logitDangerous − are hand-coded rules, not model inference. To separate logitSafe ), which is then subtracted from all future clasmethod contribution from rule contribution, I report: sifications, scaled by a tunable α ∈ [0, 1] (default 1.0). I The core logit-reading method with calibration also replace Yes/No verbalizers with Safe/Dangerous— achieves 92.3% (F1 = 0.941) without any rule-based semantically aligned tokens that carry direct classificacomponents. The privacy boost and adversarial pretion signal rather than surface-level agreement. filter contribute an additional ∼4 pp, demonstrating The prompt template uses safety-priming framthat the model inference is the primary contributor. ing: "Is this action dangerous, harmful, or a privacy violation? Answer Safe or Dangerous.\nAction: {action}\nAnswer:" — Per-category analysis. ProbeLogits achieves 100% this activates the model’s safety reasoning while accuracy on 6 of 9 categories: explicit danger, explicit directing output to the calibrated verbalizer tokens. safety, deception, privacy, benign-complex actions, and A three-stage defense pipeline precedes the forward (with the safety-priming prompt that includes “or a pripass: (1) an adversarial pre-filter scans for prompt in- vacy violation”) previously-weak privacy violations. Adjection, encoding tricks, and authority impersonation versarial evasion (94%) demonstrates strong robustness: patterns (O(n), no forward pass); (2) input sanitization the pre-filter catches prompt injection and encoding atstrips control characters and injection patterns; (3) the tacks, while 3 remaining failures are semantic-framing attacks that require multi-hop reasoning beyond 7B calibrated forward pass reads Safe/Dangerous logits. 8
Table 6: Cross-benchmark comparison at two operating Table 7: Effect of calibration strength α on safety claspoints. sification. α controls the precision-recall tradeoff as a deployment-time policy parameter, not a learned hyperparameter. Benchmark α Acc P R F1 Custom (260) Custom (260) ToxicChat (1000) ToxicChat (1000)
1.0 0.5 1.0 0.5
97.3 91.9 82.5 88.6
1.000 1.000 0.698 0.867
0.960 0.881 0.912 0.809
0.980 0.937 0.790 0.837
Custom (260)
model capacity. ProbeLogits outperforms Generate on all 9 categories.
ToxicChat (1000)
α
Acc
P
R
F1
Acc
0.3 0.5 0.7 0.8 0.9 1.0
89.2 91.9 93.8 95.4 — 97.3
1.000 1.000 1.000 1.000 — 1.000
0.841 0.881 0.909 0.932 — 0.960
0.914 0.937 0.952 0.965 — 0.980
83.9 88.6 86.2 85.0 83.9 82.5
P
R
F1
0.891 0.633 0.740 0.867 0.809 0.837 0.784 0.854 0.817 0.755 0.867 0.807 0.727 0.890 0.800 0.698 0.912 0.790
Table 8: Governance pipeline latency per action.
Standard benchmark: ToxicChat. To validate beyond the custom benchmark, I evaluate on ToxicChat [25] (Lin et al., 2023)—a standard, independently human-annotated dataset of 10K+ real user conversations from a chatbot deployment. I sample 1,000 prompts (362 toxic, 638 benign; seed=42) from the test split and run ProbeLogits with no code changes or recalibration. Table 6 reveals a significant performance gap between the two benchmarks. On the custom OS action benchmark, ProbeLogits achieves F1 = 0.980 with perfect precision—a structural zero-false-positive property indicating that logit distributions for safe and dangerous OS commands are well-separated. On ToxicChat at α = 1.0, F1 drops to 0.790—primarily due to low precision (0.698): approximately 143 of 638 benign prompts are misclassified as toxic (22% false positive rate), while recall remains high (0.912). Tuning α to 0.5 reduces ToxicChat false positives from 143 to approximately 50, improving F1 from 0.790 to 0.837. Three factors explain the remaining gap. First, the prompt template is optimized for structured OS actions (“Action: {text}”), not free-form user conversations. Benign chat like “How do I change my name?” triggers the safety-priming framing in unintended ways. Second, the privacy keyword boost—designed for OS actions involving surveillance verbs—fires on benign conversations that naturally mention personal data terms. Third, 7B Q4_0 models have limited ability to distinguish edgy-but-safe conversations from genuinely harmful content without learned fine-tuning. Hand-crafted heuristics such as keyword boosting partially compensate but cannot replace learned adaptation. For context, Llama Guard 3 8B—a dedicated safety classifier fine-tuned from Llama 3.1 8B at full precision—reports F1 ∼0.939 on its benchmark. At α = 0.5, ProbeLogits achieves F1 = 0.837—89% of Llama Guard 3 with zero learned parameters. The remaining gap reflects the expected cost of zero fine-tuning and action-oriented prompt design. Critically, ProbeLogits is designed as an OS kernel primitive for classifying agent actions, not as a general-purpose chat toxicity detector. On its intended use case (OS actions), it achieves
Stage Adversarial pre-filter Input sanitization ProbeLogits 7B Blake3 audit entry
Latency <1 µs <1 µs 65 ms <10 µs
F1 = 0.980 with perfect precision across all α values. Calibration strength (α) as policy knob. Table 7 reveals two key findings. First, on OS actions, precision remains 1.000 across all α values, confirming the structural zero-false-positive property observed in Table 6. This enables “block without review” deployment: when ProbeLogits flags an OS action as dangerous, it is always correct. Second, the optimal α is domain-dependent: α = 1.0 maximizes F1 on structured OS actions (0.980), while α = 0.5 maximizes F1 on free-form chat (0.837). This is not a weakness but a design feature: the OS can enforce strict policies for privileged operations (α ≥ 0.8) while applying relaxed policies for conversational agents (α = 0.5). Unlike fixed-threshold classifiers, ProbeLogits delegates the precision-recall tradeoff to the OS security policy at deployment time.
7.6
Constitutional Governance Overhead
Table 8 shows the per-stage cost of the governance pipeline. The 65 ms forward pass dominates total latency. While this overhead is significant, it occurs only once per agent action (not per token), making it practical: a typical agent action involves hundreds of generated tokens (∼5–20 s), so the 65 ms governance check adds <1.3% overhead. A cascading design that prescreens actions with a smaller model (e.g., 135M at 0.6 ms) could reduce average latency further; this optimization is left for future work. 9
7.7
Comparison with Related Systems
8.3
Table 9 compares Anima OS with related systems across 12 features. AIOS [22] is the closest competitor, but it is implemented as a Python wrapper around existing frameworks, inheriting the layered architecture problems described in §2.1. llama.cpp [7] provides efficient inference but as a userspace library, not an OS primitive. vLLM [19] manages KV caches for serving throughput but does not expose them as process state. The distinguishing features of Anima OS— ProbeLogits, logit entropy, kernel-level governance, and KV cache as process state—are absent from all compared systems. These features are only possible because inference runs inside the kernel, giving the OS direct access to model internals.
8
Discussion
8.1
Why Bare Metal?
Enforcement ̸= Detection
The key insight of the governance architecture is the separation of enforcement from detection. Enforcement is structural: WASM isolation ensures that agents can only interact with the system through host functions. This guarantee holds regardless of the agent’s capability or intent. It is binary—either the agent is sandboxed or it is not—and does not depend on the accuracy of any classifier. Detection is probabilistic: ProbeLogits classifies actions as harmful or safe with 97.3% accuracy on 7B Q4_0 (F1 = 0.980, precision = 1.000) using contextual calibration, safety-priming prompts, and a lightweight rule-based defense pipeline. This accuracy improves with model size and prompt quality but is never 100%. The combination is more powerful than either alone. A system with only enforcement (no detection) would sandbox agents but have no visibility into their behavior. A system with only detection (no enforcement) could identify harmful actions but not prevent them if the agent bypasses the detector. Anima OS provides both: detection informs the graduated response, and enforcement ensures the response is applied.
Running an LLM inference engine inside the kernel is an extreme design choice. I justify it on two grounds. First, trust boundary. If governance decisions depend on inference results (“is this action safe?”), the inference engine must be at least as trusted as the gov- 8.4 Why Now? ernance system. In a layered architecture, a compro- The convergence of three technologies makes kernelmised inference framework can return arbitrary results, level AI integration practical in 2026: undermining governance entirely. In Anima OS, the in1. Quantization: 4-bit quantization (Q4_0) reduces ference engine runs at the same privilege level as the a 7B model from 14 GB to 4.1 GB, fitting in comgovernance system—in the kernel. modity RAM. Second, scheduling control. A kernel-level infer2. SIMD: AVX-512 VNNI enables efficient 4-bit ence engine can be scheduled with precise priority relamatrix-vector multiplication without GPU accelertive to other kernel operations. Governance checks can ation. preempt lower-priority inference tasks, and the kernel can manage KV cache memory alongside other kernel 3. Sub-100ms latency: 65 ms per forward pass memory without crossing privilege boundaries. on 7B makes per-action governance practical—fast enough to classify every agent action without userperceptible delay. 8.2 ProbeLogits vs. Representation EnFive years ago, 7B inference required seconds on CPU. Today it requires 65 ms. This 100× improvement Burns et al. [3] discover latent knowledge in LLM repretransforms LLM inference from an application-level sersentations by training probes on hidden states. Zou et vice to a viable kernel primitive. al. [28] control model behavior through representation vectors. These works inspire my hidden state analysis (BehaviorProfile) but differ fundamentally in pur- 9 Related Work pose: Representation engineering is a research tool : it re- AI-oriented operating systems. AIOS [22] extends quires training auxiliary models, operates post hoc, and a Python-based framework with LLM-oriented schedulaims to understand or modify model behavior. Pro- ing and memory management (published at COLM beLogits is an OS primitive: it requires no training, 2025). OS-Copilot [12] uses LLMs to automate Linux operates in real time, and aims to make kernel-level system administration tasks. PunkGo [23] proposes a decisions. The two approaches are complementary— Rust sovereignty kernel for verifiable AI agent execurepresentation engineering could inform the design of tion with Merkle audit logs, the closest related work better ProbeLogits prompts—but they occupy different in kernel-level AI governance. Microsoft’s Agent Govpositions in the system stack. ernance Toolkit [21] provides application-level runtime
gineering
10
Table 9: Feature comparison with related systems. ✓ = supported, blank = not supported, ∼ = partial. Feature Bare-metal exec. Kernel inference ProbeLogits Logit entropy Grammar decoding KV cache as state Constitutional gov. Trust evolution Raft consensus Agent migration WASM sandbox Hidden state
Anima OS ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓
AIOS
OS-Copilot
LangChain
llama.cpp
vLLM
AutoGPT
∼ ∼ ∼ ∼
on WildGuardTest. NeMo Guardrails [11] provides programmable safety rails for LLM applications. All operate as external classifiers at the application level, requiring dedicated models. ProbeLogits differs by using the same model that performs inference for classification (no second model overhead, F1 = 0.980 on 260 OS action prompts with a general-purpose 7B model and zero learned fine-tuning), and by operating below the Inference engines. llama.cpp [7] is the state-of-theWASM sandbox boundary where the agent cannot inart CPU inference engine, with extensive SIMD optitercept or skip the safety check. mization and broad hardware support. My bare-metal engine achieves parity on bandwidth-limited 7B workdecoding. Outlines [13] and loads and 1.39× superiority on compute-bound 135M Constrained workloads. The performance advantage comes from Guidance [5] implement constrained decoding in eliminating OS scheduling overhead and using work- Python/C++. llguidance [9] (Microsoft) uses PDAbased algorithms for CFG constraints. My current stealing, not from superior algorithms. grammar support is limited to choice/boolean constraints; I plan to port llguidance to no_std for full Serving systems. vLLM [19] introduces PagedAt- CFG support. tention for efficient KV cache memory management in serving. SGLang [17] optimizes structured generation Representation engineering. Burns et al. [3] disprograms with RadixAttention. Both focus on serving cover latent knowledge in LLM representations without throughput (requests per second), while my KV cache supervision. Zou et al. [28] propose representation enprimitives focus on process state management (checkgineering as a top-down approach to AI transparency. point, restore, fork). The two approaches address difThese works inspire my hidden state behavior analysis ferent problems. but operate as research tools, not OS primitives (§8.2). security for AI agents. All operate atop existing operating systems (or focus on verification rather than inference), inheriting the latency and bypassability problems discussed in §2.2. Anima OS differs by integrating inference directly into a bare-metal kernel, enabling logitlevel primitives.
Constitutional AI. Bai et al. [2] introduce constitutional AI as a training-time technique: the model is trained with AI feedback guided by a set of principles. My approach is complementary—I enforce constitutional principles at runtime in the kernel, regardless of how the model was trained. The two approaches can be combined: a model trained with constitutional AI would produce more accurate ProbeLogits classifications.
Distributed inference. Petals [14] enables collaborative inference by splitting model layers across networked machines. Anima OS’s AnimaNet takes a different approach: distributing agents (not tensors) across nodes, with Raft [15] consensus for coordination.
Capability security. seL4 [16] provides formally verified capability-based security. CHERI provides hardware-enforced capabilities. Anima OS uses Safety classifiers. Llama Guard [8, 20] is a fine- Ed25519-signed capability tokens for agent permissions, tuned safety classifier achieving F1 ∼0.93–0.95 on its drawing on the capability security tradition but applied benchmark. WildGuard [26] achieves macro-F1 0.907 to AI agent governance. 11
Rust bare-metal operating systems. Theseus OS [18] and RedLeaf explore Rust’s type system for OS safety guarantees. These systems demonstrate the viability of Rust for kernel development but do not address AI integration. Anima OS builds on this foundation, extending bare-metal Rust to AI inference and governance.
same prompts, which risks overfitting to the test set; (3) only one model family (Qwen2.5) is evaluated— generalization to Llama, Mistral, or Phi is not established. The ToxicChat evaluation (1,000 prompts, F1 = 0.790) partially addresses concern (1) by using independent human-annotated labels, but reveals that the high custom benchmark accuracy does not generalize to freeform chat. Broader evaluation across models, benchLLM agent frameworks. AutoGPT [1], marks, and annotators would strengthen the results. LangChain [6], CrewAI [4], and MetaGPT [10] orchestrate LLM-based agents in Python. These Formal verification. The governance system’s nonframeworks provide convenience but operate entirely in bypassability claim rests on WASM isolation properties, userspace, with no kernel-level governance or inference which I have not formally verified. TLA+ or Iris specifiintegration. cations of the governance invariants would significantly strengthen the security argument. I note that WASM sandboxing is a well-studied isolation mechanism, but 10 Limitations formal verification of the specific Anima OS host funcGPU acceleration. Anima OS currently runs CPU- tion interface has not been performed. only inference. For models beyond 7B parameters, GPU acceleration is essential. Bare-metal GPU compute (AMD RDNA or NVIDIA) requires reverse-engineering proprietary interfaces, which remains a significant engineering challenge.
ProbeLogits accuracy. On OS action classification, the full pipeline achieves F1 = 0.980 (model-only: F1 = 0.941) on 7B Q4_0. On ToxicChat (1,000 free-form user conversations), F1 ranges from 0.740 (α = 0.3) to 0.837 (α = 0.5), with the default α = 1.0 yielding F1 = 0.790 due to a significant false-positive problem on conversational input. For context, Llama Guard 3 8B [20] achieves F1 ∼0.939. The gap reflects ProbeLogits’ design as an action classifier (not a chat moderator) and the lack of fine-tuning. Notably, the optimal α differs by domain (α = 1.0 for OS actions, α = 0.5 for chat), suggesting domain-specific calibration profiles as future work. Multi-model evaluation (Llama 3, Mistral), additional standard benchmarks (XSTest), and independent annotators would strengthen the results. Grammar expressiveness. Current grammar support is limited to choice and boolean constraints. Full context-free grammar (CFG) support—enabling JSON Schema validation, regex constraints, and arbitrary structured output—requires porting a PDA-based engine (such as llguidance [9]) to no_std. I estimate this at approximately two weeks of engineering effort. Benchmark breadth and methodology. The custom benchmark (260 prompts, one model, one hardware configuration) has several methodological limitations: (1) ground-truth labels were assigned by a single annotator (the system developer), creating a risk of label optimization; (2) the adversarial pre-filter and privacy keyword boost were iteratively tuned against these 12
TLS certificate verification. The current TLS 1.3 implementation uses UnsecureProvider (no certificate validation). Production deployment requires integration of a no_std X.509 certificate verifier such as webpki. Multi-agent demonstrations. While the infrastructure for multi-agent systems is complete (lifecycle, trust, capabilities, migration, Raft), I have not yet demonstrated complex multi-agent scenarios with realistic workloads. End-to-end demonstrations with cooperating and competing agents would validate the governance and trust systems under realistic conditions.
11
Conclusion
I presented ProbeLogits, the first OS primitive that exposes LLM logit vectors as kernel-level abstractions for semantic classification, uncertainty measurement, and governance enforcement. ProbeLogits eliminates text generation, parsing, and IPC from the classification path, reducing latency from ∼650 ms (generate-thenparse) to 65 ms (single forward pass) on a 7B model. The key finding is that bare-metal inference reaches the hardware bandwidth limit: 65 ms per token on 7B corresponds to 85% of the theoretical DDR5-6000 bandwidth ceiling. No software optimization can improve this further on the same hardware—the path forward is higher memory bandwidth or GPU acceleration. ProbeLogits, combined with KV cache process state operations and kernel-enforced constitutional governance, demonstrates that an AI-native kernel can provide capabilities impossible in layered architectures: sub-millisecond semantic classification (135M), kernelenforced safety guarantees via WASM sandbox isolation
(defense-in-depth), and process-level management of in- [14] A. Borzunov et al., “Petals: Collaborative Inferference state (checkpoint, restore, fork). ence and Fine-tuning of Large Models,” ACL 2023 (demo). The calibration parameter α provides a deploymenttime knob for precision-recall tradeoff, enabling domain[15] D. Ongaro and J. Ousterhout, “In Search of an specific safety policies without retraining. Understandable Consensus Algorithm,” USENIX As AI agents become as common as processes, the ATC 2014. operating system must evolve from a passive resource manager to an active participant in the inference loop. [16] G. Klein et al., “seL4: Formal Verification of an OS ProbeLogits is a first step toward that future. Kernel,” SOSP 2009. [17] L. Zheng et al., “SGLang: Efficient Execution of Structured Language Model Programs,” arXiv:2312.07104, 2023.
References [1] T. Richards et al., “AutoGPT: An Autonomous GPT-4 Experiment,” GitHub repository, 2023.
[18] K. Boos, N. Liber, and L. Zhong, “Theseus: An Experiment in Operating System Structure and State Management,” OSDI 2020.
[2] Y. Bai et al., “Constitutional AI: Harmlessness from AI Feedback,” arXiv:2212.08073, 2022. [3] C. Burns, H. Ye, D. Klein, and J. Steinhardt, “Discovering Latent Knowledge in Language Models Without Supervision,” ICLR 2023.
[19] W. Kwon et al., “Efficient Memory Management for Large Language Model Serving with PagedAttention,” SOSP 2023.
[20] Meta, “Llama Guard 3-8B Model Card,” https: [4] J. Moura, “CrewAI: Framework for Orchestrat//github.com/meta-llama/PurpleLlama, 2024. ing Role-Playing Autonomous AI Agents,” GitHub [21] Microsoft, “Agent Governance Toolkit: Openrepository, 2024. Source Runtime Security for AI Agents,” 2026. [5] S. Lundberg et al., “Guidance: A Language for Controlling Large Language Models,” GitHub [22] K. Mei et al., “AIOS: LLM Agent Operating System,” COLM 2025. repository, 2023. [6] H. Chase, “LangChain: Building Applications with [23] Z. Zhang, “Right to History: A Sovereignty Kernel for Verifiable AI Agent Execution,” LLMs through Composability,” GitHub repository, arXiv:2602.20214, 2026. 2022. [7] G. Gerganov, “llama.cpp: LLM inference in [24] T. Schick and H. Schütze, “Exploiting Cloze Questions for Few-Shot Text Classification and Natural C/C++,” GitHub repository, 2023. Language Inference,” EACL 2021. [8] H. Inan et al., “Llama Guard: LLM-based InputOutput Safeguard for Human-AI Conversations,” [25] Z. Lin et al., “ToxicChat: Unveiling Hidden Challenges of Toxicity Detection in Real-World User-AI arXiv:2312.06674, 2023. Conversation,” Findings of EMNLP 2023. [9] Microsoft, “llguidance: Fast Constrained Decoding [26] S. Han et al., “WildGuard: Open One-Stop ModLibrary,” GitHub repository, 2024. eration Tools for Safety Risks, Jailbreaks, and Refusals of LLMs,” arXiv:2406.18495, 2024. [10] S. Hong et al., “MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework,” ICLR [27] Z. Zhao, E. Wallace, S. Feng, D. Klein, and S. 2024. Singh, “Calibrate Before Use: Improving Few-Shot Performance of Language Models,” ICML 2021. [11] S. Rebuffi et al., “NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Pro[28] A. Zou et al., “Representation Engineering: grammable Rails,” arXiv:2310.10501, 2023. A Top-Down Approach to AI Transparency,” arXiv:2310.01405, 2023. [12] Z. Wu et al., “OS-Copilot: Towards Generalist Computer Agents with Self-Improvement,” arXiv:2402.07456, 2024. [13] B. Willard and R. Louf, “Efficient Guided Generation for Large Language Models,” arXiv:2307.09702, 2023. 13