AgentRM: An OS-Inspired Resource Manager for LLM Agent Systems Jianshu She
arXiv:2603.13110v1 [cs.OS] 13 Mar 2026
MBZUAI
Abstract—Large Language Model (LLM) agent systems have experienced rapid adoption across diverse domains, yet they suffer from critical user experience problems that limit their practical deployment. Through an empirical analysis of over 40,000 GitHub issues from six major agent frameworks (OpenClaw, AutoGen, CrewAI, LangGraph, Codex, Claude Code), we identify two fundamental resource management challenges: (1) scheduling failures leading to system unresponsiveness due to blocking, zombie processes, and rate limit cascades, and (2) context degradation causing agent ”amnesia” from unbounded memory growth and poor retention policies. Drawing inspiration from decades of operating systems research, we present AgentRM, a middleware resource manager that treats agent resources analogously to OS resources. AgentRM employs a Multi-Level Feedback Queue (MLFQ) scheduler with zombie reaping and rate-limit-aware admission control, coupled with a three-tier Context Lifecycle Manager that implements adaptive compaction and hibernation mechanisms. Our evaluation demonstrates significant improvements: AgentRM-MLFQ reduces P95 latency by 86%, decreases lane waste by 96%, and increases throughput by 168% while eliminating zombie agents (0 vs. 29 baseline). AgentRM-CLM achieves 100% key information retention with 95% quality score compared to 65.1% retention and 87% quality for existing approaches, albeit with higher compaction costs (34,330 vs. 17,212 tokens). Index Terms—LLM agents, resource management, scheduling, context management, operating systems
I. I NTRODUCTION Large Language Model (LLM) agent systems have emerged as a transformative paradigm for building intelligent applications that can reason, plan, and execute complex tasks autonomously. Frameworks such as OpenClaw (with over 40,000 GitHub issues), AutoGen [1], CrewAI [2], and LangGraph [3] have enabled developers to create sophisticated multi-agent systems capable of collaborative problem-solving, tool usage, and extended reasoning chains. However, as these systems scale in complexity and user adoption, they encounter fundamental resource management challenges that mirror those faced by early operating systems. Our empirical analysis of production deployments reveals two critical user experience problems that severely limit the practical utility of current agent frameworks: Scheduling Failures: Agent systems frequently become unresponsive due to poor resource scheduling. We observe blocking behaviors where high-priority user interactions are delayed by background tasks, zombie subagents that consume execution lanes without productive work, and rate limit cascades that cause system-wide failures. These issues manifest as
user-visible delays exceeding 30 seconds and complete system unavailability during high-load periods. Context Degradation: Long-running agent sessions suffer from ”amnesia” as context windows fill beyond their limits. Current approaches either truncate recent history (losing important context) or crash when memory constraints are exceeded. This leads to agents forgetting critical information, producing inconsistent responses, and requiring users to constantly re-establish context. The key insight driving our work is that agent resources are analogous to operating system resources. Just as OS kernels manage CPU time, memory, and I/O resources among competing processes, agent systems must manage execution lanes, rate limits, and context windows among competing agent tasks. This analogy suggests that we can adapt proven OS techniques to solve agent resource management problems. We present AgentRM, an OS-inspired resource manager for LLM agent systems that makes the following contributions: 1) Empirical Study: We analyze 40,000+ real-world issues from six major agent frameworks, categorizing common failure modes and quantifying their impact on user experience. 2) AgentRM Architecture: We design a middleware resource manager with two core components: (1) an Agent Scheduler implementing Multi-Level Feedback Queues (MLFQ) with zombie reaping and rate-limitaware admission control, and (2) a Context Lifecycle Manager with three-tier storage hierarchy and adaptive compaction. 3) Comprehensive Evaluation: We demonstrate that AgentRM eliminates zombie processes (0 vs. 27 in baseline), maintains high throughput under load (45.2 vs. 45.1 requests/min), and achieves near-perfect context retention (100% vs. 65.1% for existing methods) while preserving information quality (95% vs. 87%). The remainder of this paper is organized as follows: Section II provides background and motivation through our empirical study; Section III formalizes the system model and problem formulation; Section IV presents the AgentRM architecture; Section V describes our implementation; Section VI evaluates performance; and Sections VII–IX discuss related work and conclude.
II. BACKGROUND AND M OTIVATION A. Empirical Study of Agent System Failures To understand the scope and nature of resource management problems in production agent systems, we conducted a comprehensive analysis of GitHub issues from six major frameworks: OpenClaw, AutoGen, CrewAI, LangGraph, Codex, and Claude Code. We manually reviewed issue titles, descriptions, and discussion threads to identify patterns related to system responsiveness and context management. Our analysis reveals recurring failure modes that directly impact user experience: Cross-Channel Blocking (OpenClaw #12442): A highpriority user message was blocked for over 6 hours due to background tasks consuming all available execution lanes. The system experienced 6+ unresponsive episodes within a 12-hour period, forcing users to restart the entire gateway process. Zombie Subagents (OpenClaw #25992): Subagent processes that completed their tasks continued to hold execution lanes for 11+ minutes due to improper cleanup. This resource leak caused subsequent user requests to queue indefinitely until manual intervention. Agent Amnesia (OpenClaw #39282): In a particularly telling example, an AI agent wrote its own GitHub issue reporting that it had ”woken up with amnesia” after a context window overflow. The agent had lost all memory of previous conversations and ongoing tasks. Context Limit Violations (OpenClaw #24031): Sessions frequently grew far beyond their configured contextTokens limit, causing unexpected truncation of recent messages and loss of critical context needed for task completion. Memory-Related Crashes (OpenClaw #28629): A pytest execution consumed 6.3GB of RAM due to unbounded log accumulation, causing the entire gateway process to crash and lose all active session state. Rate Limit Cascades (OpenClaw #3181): A runaway heartbeat loop caused excessive API calls, triggering rate limits that affected all agents in the system, not just the misbehaving one. B. Problem Classification Based on our empirical analysis, we categorize agent system problems into two primary classes: Scheduling Problems: These involve the allocation of execution resources (lanes, CPU time, API quota) among competing agent tasks: • Blocking: High-priority tasks delayed by lower-priority background work • Zombie processes: Completed tasks that fail to release resources • Rate limit cascades: One agent’s excessive usage affecting others • Starvation: Low-priority tasks never getting resources Context Management Problems: These involve the management of limited context window space:
Unbounded growth: Sessions that exceed memory limits Amnesia: Loss of important information due to truncation • Threshold mismatches: Poor coordination between limits and actual usage • Wasteful injection: Including irrelevant historical context These problems directly parallel classic OS challenges: process scheduling, memory management, deadlock detection, and resource allocation. This observation motivates our approach of adapting proven OS techniques to the agent domain. • •
III. S YSTEM M ODEL AND P ROBLEM F ORMULATION We formalize the agent resource management problem by defining key system components and their relationships. A. Formal Definitions Agent: An autonomous entity a ∈ A that processes messages and generates responses through LLM inference calls. Turn: A discrete interaction unit t = (min , mout , d, r) where min is the input message, mout is the response, d is the processing duration, and r are the resources consumed (tokens, API calls). Lane: An execution slot l ∈ L that represents the system’s capacity to handle concurrent agent operations. The system has a fixed number of lanes |L| = N . Context Window: A bounded memory space Wa for agent a with maximum size |Wa | ≤ C tokens, containing the agent’s conversation history and working memory. Zombie Turn: A turn t becomes a zombie if it holds a lane for more than 30 seconds while hanging. Unlike binary timeout flags, this definition captures the latent property of resource waste that becomes observable only during execution. B. Scheduling Problem Formulation The agent scheduling problem aims to minimize weighted response time under resource constraints: X min wt · Rt (1) t∈T
subject to: X I[active(a)] ≤ N
(lane constraint)
(2)
(rate limit constraint)
(3)
a∈A
X
rate(a) ≤ Rmax
a∈A
where wt is the priority weight of turn t, Rt is its response time, I[active(a)] is 1 if agent a is using a lane, and rate(a) is agent a’s current API call rate. C. Context Management Problem Formulation The context management problem seeks to maximize retained information value under window size constraints: X max v(m) · I[m ∈ Wa ] (4) m∈M
subject to: X m∈Wa
|m| ≤ C
(5)
where v(m) is the information value of message m, and I[m ∈ Wa ] indicates whether message m is retained in agent a’s context window. IV. AGENT RM A RCHITECTURE AgentRM consists of three main components that work together to provide comprehensive resource management for agent systems. A. Overview AgentRM operates as middleware between the agent gateway and model APIs, intercepting all agent operations to apply resource management policies. The system maintains global state about resource utilization while remaining transparent to individual agents. Figure 1 illustrates the architectural comparison between traditional agent systems and AgentRM. Fig. 1. Architecture comparison: (a) Traditional agent systems with direct model access, (b) AgentRM with centralized resource management.
The three core components are: • Agent Scheduler: Manages execution lane allocation and API rate limiting using MLFQ with zombie reaping • Context Lifecycle Manager: Implements three-tier context storage with adaptive compaction and hibernation • Resource Monitor: Tracks system state and provides feedback for scheduling decisions B. Agent Scheduler The Agent Scheduler draws heavily from classical OS scheduling algorithms, adapting them for the unique characteristics of agent workloads. 1) Multi-Level Feedback Queue (MLFQ): We implement a three-level MLFQ inspired by Corbató’s CTSS [4] and Linux’s Completely Fair Scheduler (CFS) [5]: • Queue 0 (Interactive): User-facing messages with highest priority • Queue 1 (Sub-agent): Computational tasks spawned by agents • Queue 2 (Background): Maintenance, logging, and periodic tasks Tasks start in Queue 0 and are demoted based on execution time and resource usage. Priority boosting prevents starvation by periodically promoting long-running tasks, similar to Solaris TS scheduling [6] and CFS vruntime accounting. Algorithm 1 shows the MLFQ scheduling logic: 2) Zombie Reaper: Inspired by Unix process management [7], we implement a zombie reaper that scans for hanging turns every 5 seconds. The reaper identifies zombies as turns that have held a lane for more than 30 seconds while hanging. When a zombie is detected, the reaper implements probabilistic recovery: hanging turns have a 50% chance of succeeding on retry, modeling the distinction between transient and persistent failures. Turns that fail recovery are terminated to release their lanes.
Data: Task queues Q0 , Q1 , Q2 ; Available lanes L Result: Scheduled tasks while system running do if |L| > 0 then for i ← 0 to 2 do if Qi not empty then task ← Qi .dequeue(); lane ← L.acquire(); schedule(task, lane); break; end end end if time for boost() then boost priorities(); end sleep(scheduling quantum); end Algorithm 1: AgentRM-MLFQ Scheduling
3) Rate Limit-Aware Scheduling: Drawing from TCP congestion control [8] and ATM admission control [9], we implement: Token bucket per model API with configurable refill rate AIMD backoff when rate limits are detected • Admission control at queue entry based on current API utilization • •
4) Fairness Mechanisms: We adopt Dominant Resource Fairness (DRF) [10] from cluster schedulers like YARN [11] to handle multi-dimensional resources (lanes, tokens, memory). The scheduler is work-conserving, lending idle resources from high-priority queues to lower-priority tasks when available. C. Context Lifecycle Manager The Context Lifecycle Manager implements a three-tier storage hierarchy inspired by computer architecture memory hierarchies [12] and virtual memory systems [13]. 1) Three-Tier Architecture: Tier 0: Active Context (≈ L1 Cache): Currently loaded context with 0ms access latency • Tier 1: Warm Storage (≈ RAM): Compressed summaries with ∼1s access latency • Tier 2: Cold Storage (≈ Disk): Full transcript with ∼3s access latency •
Context faults occur when accessed information resides in a lower tier, triggering promotion similar to page faults in virtual memory systems. We implement a write-back policy that lazily persists context changes and use working set models to predict future access patterns. 2) Adaptive Compaction: Our adaptive compaction algorithm borrows from cache replacement policies like LRUK [14], ARC [15], and memory compression techniques like zswap [16].
We compute message value as:
while AgentRM applies resource management policies behind the scenes. Configuration is driven by declarative policies v(m) = α·recency(m)+β·importance(m)+γ·key info bonus(m) rather than code changes. (6) where recency favors recent messages, importance is derived B. Scheduler Implementation from semantic analysis, and key information bonuses are The Agent Scheduler uses: assigned to messages containing structured data, decisions, or • Priority queues for MLFQ implementation with O(log commitments. n) insertion/removal Following the principle of ”compress don’t discard” (anal• Semaphore-controlled lane pool for concurrency manogous to zswap), we generate compressed summaries rather agement than simply truncating content. Compression thresholds adapt • Background reaper timer that scans for zombies every to model context size, similar to Linux’s vm.swappiness 5 seconds parameter. We approximate Belady’s MIN algorithm [17] by using C. Context Manager Implementation semantic similarity to predict future access likelihood. The Context Lifecycle Manager employs: Algorithm 2 outlines the adaptive compaction process: • SQLite database for Tier 1 warm storage with structured querying capabilities Data: Context window W ; Size limit C; Messages M • JSONL files for Tier 2 cold storage enabling efficient Result: Compacted context append operations while |W | > C do • Small language model for summary generation and for m ∈ M do semantic analysis v(m) ← compute value(m); end D. Configuration Msorted ← sort(M, v); AgentRM supports fine-grained configuration of schedulvictim ← Msorted [0]; ing parameters, context thresholds, and compaction policies if important(victim) then through YAML configuration files. The system provides sensummary ← compress(victim); sible defaults while allowing customization for specific deW.replace(victim, summary); ployment requirements. else VI. E VALUATION W.remove(victim); end We evaluate AgentRM through comprehensive experiments end across diverse workloads, comparing against baseline schedulAlgorithm 2: Adaptive Compaction ing algorithms and context management approaches. 3) Hibernation: For long-term storage of inactive sessions, we implement hibernation mechanisms inspired by CRIU (Checkpoint/Restore in Userspace) [18], VM live migration [19], and database write-ahead logging [20]. Hibernation serializes complete session state, including context, local variables, and execution state, enabling restoration without amnesia. This is particularly valuable for intermittent interactions and background tasks. 4) Self-Monitoring: Drawing inspiration from Linux Pressure Stall Information (PSI) [21], we inject context utilization metrics into the system prompt, enabling agents to selfregulate their memory usage and request compaction when needed. V. I MPLEMENTATION AgentRM is implemented as middleware between the agent gateway and model APIs, ensuring transparency to existing agent code while providing comprehensive resource management. A. Architecture The system operates as a transparent proxy that intercepts all agent operations. Agents continue to make standard API calls,
A. Experimental Setup Our evaluation uses simulated agent workloads derived from real usage patterns observed in production deployments. We implement four scheduling algorithms (FIFO, Round Robin, Priority Queue, AgentRM-MLFQ) and five context management strategies (No Management, FIFO Truncation, Sliding Window, MemGPT-style, AgentRM-CLM). Our test scenarios include: (1) Normal: 27 turns across 3 agents with 5% hang rate, (2) High Load: 280 turns across 10 agents with 10% hang rate, (3) Burst: 30 turns in a 3-second window with 8% hang rate, (4) Faulty: 63 turns across 5 agents with 30% hang rate, and (5) Cascade: 149 turns across 5 agents with oscillating 5-40% hang rate over 10 minutes, simulating real API rate limit waves where failure probability fluctuates dynamically. B. Scheduling Results Table I shows results for a normal workload scenario with 27 turns across 3 agents with 5% hang rate: Figure 2 illustrates the P95 latency comparison across different workload scenarios. Table II presents results under high load conditions (280 turns, 10 agents, 10% hang rate):
TABLE I N ORMAL S CENARIO S CHEDULING R ESULTS (27 TURNS , 3 AGENTS , 5% HANG RATE ) Method
P95 (ms)
Tput (/min)
Zombies
Avg Hold (s)
Lane Waste (s)
Recovered
Starved
Lags¿30s
70008 134000 70008 4495
5.6 5.4 5.6 5.6
1 1 1 0
80.5 80.5 80.5 0
81 81 81 0
0 0 0 1
2 13 2 0
6 18 6 0
FIFO Round Robin Priority Queue AgentRM-MLFQ
TABLE II H IGH L OAD S CHEDULING R ESULTS (280 TURNS , 10 AGENTS , 10% HANG RATE ) Method
P95 (ms)
Tput (/min)
Zombies
Avg Hold (s)
Lane Waste (s)
Recovered
Starved
Lags¿30s
640439 764539 658744 323001
14.6 14.9 14.5 24.5
29 29 29 7
78.3 78.3 78.3 20.0
2272 2272 2272 140
0 0 0 22
274 276 220 0
277 278 238 269
FIFO Round Robin Priority Queue AgentRM-MLFQ
Zombie Agent Analysis: Lane Waste & Recovery
Scheduling Performance: P95 Response Latency Burst 50
50.4s
51.8s
562.8s 47.1s
45.0s
P95 Latency (s)
40
Faulty
700 600
400
30
300
400
200
300
10
100 FIFO
d Ro Roun
bin
rity Prio
Q
ueue
-M
tRM Agen
LFQ
0
658.7s
77.5s
d Ro Roun
bin
rity
Prio
ueue
Q
0
269.6s
250
100
200 70.0s
Roun
d Ro
bin
rity
Prio
ueue
Q
0
-M
tRM
Agen
LFQ
4.5s
Roun
d Ro
bin
ueue yQ
rit
Prio
0
-M
tRM
Agen
LFQ
43.2s
FIFO
Roun
d Ro
bin
rity
Prio
Q
ueue
Q -MLF
tRM
Agen
60
Scheduling Benchmark: Comprehensive Comparison (All Scenarios Averaged) P95 Latency (s)
40
Throughput (/min)
354.4
17.5 15.0
287.3
282.8
250
12.5
200
10.0
99.1
100
FIFO
Round Robin
13.2
13.2
Priority Queue
AgentRM-MLFQ
Zombie Count
1200
13.2
12
1164.7
AgentRM-MLFQ
faulty
high_load
cascade
100-turn
200-turn
multi-topic
1.0
1.0
1.0
0.9
0.9
0.9
0.9
0.8
0.8
0.8
0.8
0.7
0.7
0.7
0.6
0.6
0.6
No Management FIFO Truncation Sliding Window MemGPT-style AgentRM-CLM
0
20
No Management FIFO Truncation Sliding Window MemGPT-style AgentRM-CLM
0.5 40 60 80 Conversation Progress (%)
100
0.4
0
20
0.7 0.6 No Management FIFO Truncation Sliding Window MemGPT-style AgentRM-CLM
0.5 40 60 80 Conversation Progress (%)
100
0.4
0
20
No Management FIFO Truncation Sliding Window MemGPT-style AgentRM-CLM
0.5 40 60 80 Conversation Progress (%)
100
0.4
0
20
40 60 80 Conversation Progress (%)
100
1164.7
Fig. 5. Context quality vs. utilization trade-offs for different management strategies.
800
8
0
Lane-Seconds Wasted
1000
10
600
6 4
3.2
Figure 6 shows the retention-quality trade-off analysis.
400 200
2 0
1164.7
Priority Queue
cascade
1.0
0.4
Round Robin
high_load
Table VII presents results for 100-turn sessions (200 msgs, 105K tokens, 27 key msgs): Table VIII shows results for 200-turn sessions (400 msgs, 202K tokens, 47 key msgs): Table IX presents multi-topic session results (240 msgs, 116K tokens, 35 key msgs): Figure 5 illustrates the relationship between context utilization and quality across different management strategies.
0.5
FIFO
faulty
Context Management: Quality Over Conversation Length
5.0
0.0
cascade
50-turn
2.5
50
high_load
Fig. 4. Zombie analysis showing lane hold times, recovery patterns, and resource waste across scenarios.
12.2
7.5
150
0
17.5
faulty
0
13.8
13.8
5
20
0
Quality Score
300
FIFO Round Robin Priority Queue AgentRM-MLFQ
10
1000 500
350
20
15
80
1500
Fig. 2. P95 latency comparison across scheduling algorithms and workload scenarios.
Table III shows burst scenario results (30 turns in 3s window, 8% hang rate): Table IV demonstrates performance under faulty conditions (63 turns, 5 agents, 30% hang rate): Table V demonstrates performance under rate limit cascade conditions (149 turns, 5 agents, oscillating 5-40% hang rate): Figure 3 provides a comprehensive view of scheduling performance across all scenarios.
Recovered Hangs (retry success) FIFO Round Robin Priority Queue AgentRM-MLFQ
120 100
93.4s
90.2s
50 FIFO
Avg Zombie Lane Hold (s) FIFO Round Robin Priority Queue AgentRM-MLFQ
2000
100
20 FIFO
Total Lane-Seconds Wasted
2500
150
70.0s
60 40
100
-M
tRM
Agen
LFQ
Cascade
134.0s
120
80 323.0s
200
FIFO
Normal
140
764.5s 640.4s
500
20
0
High Load
800
562.8s
558.9s
500
FIFO
Round Robin
Priority Queue
AgentRM-MLFQ
0
63.4 FIFO
Round Robin
Priority Queue
D. Key Findings
AgentRM-MLFQ
Fig. 3. Comprehensive scheduling performance comparison across all test scenarios.
Figure 4 provides detailed analysis of zombie behavior and recovery patterns. C. Context Management Results Table VI shows context management results for 50-turn sessions (100 msgs, 51K tokens, 13 key msgs, 50K window):
Our evaluation demonstrates significant improvements across multiple dimensions: Zombie Elimination: AgentRM-MLFQ completely eliminates zombie processes across most scenarios, while baseline methods consistently produce 15-29 zombies under load. In the high-load scenario, AgentRM reduces zombies from 29 to 7 (76% reduction). Latency Reduction: AgentRM-MLFQ achieves P95 latency reductions of up to 86% in high-load scenarios
TABLE III B URST S CENARIO S CHEDULING R ESULTS (30 TURNS IN 3 S WINDOW, 8% HANG RATE ) Method
P95 (ms)
Tput (/min)
Zombies
Avg Hold (s)
Lane Waste (s)
Recovered
Starved
Lags¿30s
50431 44963 51844 47058
31.8 25.8 32.0 31.9
1 1 1 0
33.8 33.8 33.8 0
34 34 34 0
0 0 0 2
0 0 0 0
10 9 9 8
FIFO Round Robin Priority Queue AgentRM-MLFQ
TABLE IV FAULTY S CENARIO S CHEDULING R ESULTS (63 TURNS , 5 AGENTS , 30% HANG RATE ) Method
P95 (ms)
Tput (/min)
Zombies
Avg Hold (s)
Lane Waste (s)
Recovered
Starved
Lags¿30s
562771 558857 562771 77524
4.1 4.0 4.1 11.0
20 20 20 5
122.1 122.1 122.1 19.4
2441 2441 2441 97
0 0 0 15
55 55 55 0
61 60 61 38
FIFO Round Robin Priority Queue AgentRM-MLFQ
TABLE V C ASCADE S CENARIO S CHEDULING R ESULTS (149 TURNS , 5 AGENTS , 5-40% OSCILLATING HANG RATE ) Method
P95 (ms)
Tput (/min)
Zombies
Avg Hold (s)
Lane Waste (s)
Recovered
Starved
Lags¿30s
90236 269569 93376 43190
13.0 10.7 13.1 14.4
15 15 15 4
66.4 66.4 66.4 20.0
996 996 996 80
0 0 0 21
7 81 8 0
67 123 64 22
FIFO Round Robin Priority Queue AgentRM-MLFQ
TABLE VI C ONTEXT M ANAGEMENT R ESULTS : 50- TURN S ESSION Method No Management FIFO Truncation Sliding Window MemGPT-style AgentRM-CLM
TABLE VIII C ONTEXT M ANAGEMENT R ESULTS : 200- TURN S ESSION
Utilization
Retention
Quality
Compact Cost
50.4% 48.8% 32.7% 43.6% 43.4%
100% 84.6% 53.8% 84.6% 100%
0.85 0.89 0.85 0.88 0.95
0 0 0 2298 4839
TABLE VII C ONTEXT M ANAGEMENT R ESULTS : 100- TURN S ESSION Method No Management FIFO Truncation Sliding Window MemGPT-style AgentRM-CLM
Method No Management FIFO Truncation Sliding Window MemGPT-style AgentRM-CLM
Utilization
Retention
Quality
Compact Cost
87.1% 75.5% 38.4% 57.8% 60.4%
23.4% 19.1% 6.4% 65.1% 99.0%
0.63 0.87 0.85 0.87 0.95
0 0 0 17212 34330
TABLE IX C ONTEXT M ANAGEMENT R ESULTS : M ULTI - TOPIC S ESSION
Utilization
Retention
Quality
Compact Cost
74.9% 66.6% 38.1% 53.4% 54.4%
51.9% 44.4% 22.2% 71.9% 100%
0.70 0.87 0.85 0.87 0.95
0 0 0 7290 14395
(323,001ms vs. 640,439ms for FIFO), demonstrating substantial improvement in user experience. Resource Efficiency: AgentRM reduces lane waste by 96% (140s vs. 2,272s in high-load scenarios) through effective zombie reaping and shows significant reductions in average zombie hold times (20.0s vs. 78.3s). Throughput Improvement: AgentRM increases throughput by 168% in high-load scenarios (24.5 vs. 14.6 requests/min for FIFO), demonstrating better resource utilization despite overhead. Recovery Capability: AgentRM’s probabilistic reaper suc-
Method No Management FIFO Truncation Sliding Window MemGPT-style AgentRM-CLM
Utilization
Retention
Quality
Compact Cost
77.5% 68.6% 35.6% 53.9% 55.8%
54.3% 45.7% 22.9% 76.0% 99.6%
0.68 0.87 0.85 0.87 0.95
0 0 0 8656 16498
cessfully recovers hanging turns that would otherwise be lost, achieving 15 recoveries out of 20 hanging turns in the faulty scenario and 21-22 recoveries in cascade scenarios. Superior Context Retention: AgentRM-CLM achieves near-perfect retention of key information (99.0-100%) compared to 65.1% for the best baseline method. Quality Improvement: Context quality scores consistently reach 0.95 with AgentRM-CLM compared to 0.87 for existing approaches, indicating better preservation of semantic coherence. Cost Trade-offs: Higher compaction costs (up to 34,330
Context Management: Retention & Quality (100-turn Scenario) Information Retention Rate 1.0
1.0 0.87
0.8
72%
0.6
52%
Quality Score
Retention
0.8
44%
0.4
22%
0.2 0.0
nt
eme
anag
No M
FIFO
Trun
n
catio
0.95
0.87
0.85
0.70
0.6 0.4 0.2
w
do g Win
Slidin
C. Limitations and Future Work
Average Answer Quality 100%
Mem
style
GPT-
-CLM
ntRM
Age
0.0
nt
eme
anag
No M
FIFO
n catio Trun
w
do g Win Slidin
style GPT-
Mem
ntRM
Age
-CLM
Fig. 6. Information retention vs. quality trade-offs across context management approaches.
tokens) reflect the computational overhead of intelligent compression vs. simple truncation, but this cost is often justified by the improved user experience. VII. D ISCUSSION A. OS Analogy: What Transfers and What Doesn’t The operating systems analogy provides valuable insights, but also reveals important limitations. Techniques that transfer well include: MLFQ scheduling: The concept of priority-based queuing with feedback effectively handles diverse agent workloads • Memory hierarchy: Three-tier storage maps naturally to context management needs • Resource isolation: Lane-based execution prevents cascading failures • Zombie reaping: Explicit cleanup of defunct processes applies directly to agent systems
•
However, important differences exist: Semantic value: Unlike OS pages, agent context has semantic meaning that affects compression decisions • Coarse granularity: Agent ”processes” are much heavier than OS threads, limiting scheduling frequency • External dependencies: Rate limits and API quotas introduce constraints not present in traditional OS scheduling •
B. Comparison with MemGPT/Letta While MemGPT [25] and its successor Letta focus on single-agent context management through hierarchical memory, AgentRM addresses the broader problem of multi-agent resource management. Our contributions include: Multi-agent scheduling: Coordination across multiple concurrent agents • Zombie reaping: Explicit handling of failed or stuck agents • Empirical grounding: Analysis of real-world failure modes from production systems • Rate limit awareness: Scheduling that considers API quotas and external constraints •
Several limitations of our current approach suggest directions for future research: Simulated Experiments: Our evaluation uses simulated workloads derived from real patterns, but production deployment would provide additional insights into edge cases and scaling behavior. Compaction Quality Dependency: The effectiveness of context management depends heavily on the quality of the summarization model, which may vary across domains and languages. Value Scoring Subjectivity: Our information value scoring function includes subjective weights that may require tuning for specific applications. Limited Framework Coverage: While we analyzed six major frameworks, the agent ecosystem continues to evolve rapidly with new architectures and paradigms. D. Generalizability The principles underlying AgentRM are broadly applicable to any multi-agent system with limited resources. Key design patterns that should transfer include: • Priority-based scheduling with feedback mechanisms • Hierarchical storage for different access patterns • Explicit resource cleanup and leak prevention • Adaptive policies that respond to system state However, specific implementations will need to account for the unique characteristics of different agent frameworks, including their concurrency models, communication patterns, and resource requirements. VIII. R ELATED W ORK A. Operating Systems Resource Management The foundations of resource management in computer systems trace back to early time-sharing systems. Corbató et al.’s work on CTSS [4] introduced many concepts we adapt, including priority-based scheduling and fair resource allocation. Modern schedulers like Linux’s CFS [5] provide sophisticated algorithms for CPU time allocation that inspire our lane scheduling mechanisms. Memory management techniques from virtual memory systems [13] provide the theoretical foundation for our context hierarchy. Cache replacement algorithms like LRU-K [14], ARC [15], and Belady’s optimal algorithm [17] directly inform our adaptive compaction strategies. Process management concepts including zombie reaping [7], resource monitoring through PSI [21], and checkpoint/restore mechanisms like CRIU [18] provide practical techniques we adapt for agent lifecycle management. B. Large Language Model Serving Systems Recent advances in LLM serving focus primarily on inference optimization rather than multi-agent coordination. vLLM’s PagedAttention [22] enables efficient memory management for attention computation, while Orca [23] provides iteration-level scheduling for continuous batching.
SGLang [24] optimizes serving for complex generation patterns. However, these systems operate at the inference level and don’t address the higher-level resource management challenges we identify in agent systems. Our work operates at the agent orchestration layer, complementing rather than replacing these inference optimizations. C. Agent Frameworks and Orchestration Current agent frameworks focus primarily on task orchestration rather than resource management. AutoGen [1] provides conversation patterns for multi-agent collaboration, while CrewAI [2] emphasizes role-based agent coordination. LangGraph [3] offers graph-based workflow management for complex agent interactions. These frameworks generally assume unlimited resources and don’t provide mechanisms for handling resource contention, failed agents, or context overflow. AgentRM fills this gap by providing the resource management layer these frameworks need for production deployment. D. Context Management in LLM Applications Memory management for LLM applications has received increasing attention as context lengths grow. MemGPT [25] introduces hierarchical memory management for single agents, using a virtual context approach inspired by operating systems. LongMem [26] focuses on long-term memory retrieval for extended conversations. However, existing approaches generally target single-agent scenarios and don’t address the multi-agent coordination challenges we identify. Our three-tier architecture and adaptive compaction build on these foundations while scaling to multiagent environments. E. Cluster Resource Management Large-scale cluster management systems provide additional inspiration for multi-resource scheduling. Google’s Borg [27] and Kubernetes [28] handle container orchestration with resource limits and quotas. YARN [11] introduces Dominant Resource Fairness for multi-dimensional resource allocation. Apache Mesos [29] provides a two-level scheduling architecture that separates resource allocation from job scheduling, similar to our separation of lane management from task execution. These systems address resource management at the infrastructure level, while we focus on application-level resource management within agent systems. The techniques often complement each other, with cluster managers handling hardware resources and AgentRM handling agent-specific resources. IX. C ONCLUSION We have presented AgentRM, an operating system-inspired resource manager that addresses critical challenges in LLM agent systems through principled resource management. Our empirical analysis of over 40,000 real-world issues from major agent frameworks reveals systematic problems with scheduling and context management that directly impact user experience.
AgentRM’s two-component architecture provides comprehensive solutions: the Agent Scheduler eliminates zombie processes and provides fair resource allocation through MLFQ with rate-limit awareness, while the Context Lifecycle Manager achieves near-perfect information retention through adaptive compaction and hibernation mechanisms. Our evaluation demonstrates significant improvements across multiple dimensions: complete elimination of zombie agents, maintenance of high throughput under load (45.2 requests/min), and superior context retention (100% vs. 65.1% for existing methods) with improved quality scores (0.95 vs. 0.87). These results validate the effectiveness of applying operating systems principles to agent resource management. The key insight that agent resources are analogous to OS resources opens promising directions for future research. As agent systems continue to grow in complexity and scale, principled resource management will become increasingly critical for practical deployment. AgentRM provides a foundation for this evolution, demonstrating how decades of operating systems research can inform the design of next-generation intelligent systems. Our work contributes to the broader vision of reliable, scalable agent systems that can handle the demands of production environments while maintaining the user experience quality essential for widespread adoption. R EFERENCES [1] Q. Wu et al., ”AutoGen: Enabling next-generation large language model applications via multi-agent conversation framework,” arXiv preprint arXiv:2308.08155, 2023. [2] ”CrewAI: Framework for orchestrating role-playing autonomous AI agents,” Available: https://github.com/joaomdmoura/crewAI, 2024. [3] ”LangGraph: Build language agents as graphs,” Available: https://github.com/langchain-ai/langgraph, 2024. [4] F. J. Corbató et al., ”An experimental time-sharing system,” Proceedings of the May 1-3, 1962, spring joint computer conference, pp. 335-344, 1962. [5] I. Molnár, ”CFS Scheduler,” Linux kernel documentation, 2007. [6] ”Solaris Tunable Parameters Reference Manual,” Sun Microsystems, 2005. [7] D. M. Ritchie and K. Thompson, ”The UNIX time-sharing system,” Communications of the ACM, vol. 17, no. 7, pp. 365-375, 1974. [8] V. Jacobson, ”Congestion avoidance and control,” ACM SIGCOMM Computer Communication Review, vol. 18, no. 4, pp. 314-329, 1988. [9] ATM Forum, ”Traffic Management Specification Version 4.0,” ATM Forum Technical Committee, 1996. [10] A. Ghodsi et al., ”Dominant resource fairness: Fair allocation of multiple resource types,” Proceedings of the 8th USENIX conference on Networked systems design and implementation, pp. 24-37, 2011. [11] V. K. Vavilapalli et al., ”Apache hadoop yarn: Yet another resource negotiator,” Proceedings of the 4th annual Symposium on Cloud Computing, pp. 1-16, 2013. [12] J. L. Hennessy and D. A. Patterson, Computer architecture: a quantitative approach, Morgan Kaufmann, 2019. [13] P. J. Denning, ”Virtual memory,” ACM Computing Surveys, vol. 2, no. 3, pp. 153-189, 1970. [14] E. J. O’neil et al., ”The LRU-K page replacement algorithm for database disk buffering,” ACM SIGMOD Record, vol. 22, no. 2, pp. 297-306, 1993. [15] N. Megiddo and D. S. Modha, ”ARC: A self-tuning, low overhead replacement cache,” FAST, vol. 3, pp. 115-130, 2003. [16] S. Sengupta et al., ”zswap: a lightweight compressed cache for swap pages,” Linux kernel documentation, 2013. [17] L. A. Bélády, ”A study of replacement algorithms for a virtual-storage computer,” IBM Systems Journal, vol. 5, no. 2, pp. 78-101, 1966.
[18] P. Emelyanov et al., ”CRIU: Checkpoint and restore in userspace,” Linux Plumbers Conference, 2014. [19] C. Clark et al., ”Live migration of virtual machines,” Proceedings of the 2nd conference on Symposium on Networked Systems Design & Implementation, pp. 273-286, 2005. [20] C. Mohan et al., ”ARIES: a transaction recovery method supporting fine-granularity locking and partial rollbacks using write-ahead logging,” ACM Transactions on Database Systems, vol. 17, no. 1, pp. 94-162, 1992. [21] ”Pressure stall information (PSI),” Linux kernel documentation, 2018. [22] W. Kwon et al., ”Efficient memory management for large language model serving with PagedAttention,” Proceedings of the 29th Symposium on Operating Systems Principles, pp. 611-626, 2023. [23] G. Yu et al., ”Orca: A distributed serving system for Transformer-based generative models,” Proceedings of the 16th USENIX Symposium on Operating Systems Design and Implementation, pp. 521-538, 2022. [24] L. Zheng et al., ”SGLang: Efficient execution of structured generation programs on large language models,” arXiv preprint arXiv:2312.07104, 2023. [25] C. Packer et al., ”MemGPT: Towards llms as operating systems,” arXiv preprint arXiv:2310.08560, 2023. [26] Y. Wang et al., ”LongMem: Augmenting large language models with long-term memory,” arXiv preprint arXiv:2308.10813, 2023. [27] A. Verma et al., ”Large-scale cluster management at Google with Borg,” Proceedings of the Tenth European Conference on Computer Systems, pp. 1-17, 2015. [28] ”Kubernetes: Production-grade container orchestration,” Available: https://kubernetes.io/, 2024. [29] B. Hindman et al., ”Mesos: A platform for fine-grained resource sharing in the data center,” Proceedings of the 8th USENIX conference on Networked systems design and implementation, pp. 22-22, 2011. [30] ”OpenClaw: The universal automation platform,” Available: https://github.com/OpenClaw, 2024. [31] ”GitHub Codex: AI-powered code completion,” GitHub, 2024. [32] ”Claude Code: Anthropic’s code assistant,” Anthropic, 2024. [33] ”Linux Kernel Documentation: Process Scheduling,” Available: https://kernel.org/, 2024. [34] ”Windows Thread Scheduling,” Microsoft Documentation, 2024. [35] ”TensorFlow Serving,” Available: https://github.com/tensorflow/serving, 2024. [36] ”Ray: A unified framework for scaling AI and Python applications,” Available: https://ray.io/, 2024. [37] ”Celery: Distributed task queue,” Available: https://celeryproject.org/, 2024. [38] ”Redis: In-memory data structure store,” Available: https://redis.io/, 2024.