arXiv:2606.24775v1 [cs.CL] 23 Jun 2026
Are We Ready For An Agent-Native Memory System? Wei Zhou
Xuanhe Zhou∗
Shaokun Han
Hongming Xu
Shanghai Jiao Tong University [email protected]
Shanghai Jiao Tong University [email protected]
Shanghai Jiao Tong University [email protected]
Shanghai Jiao Tong University [email protected]
Guoliang Li
Zhiyu Li
Feiyu Xiong
Fan Wu
Tsinghua University [email protected]
MemTensor (Shanghai) Technology Co., Ltd [email protected]
MemTensor (Shanghai) Technology Co., Ltd [email protected]
Shanghai Jiao Tong University [email protected]
Abstract Memory for large language model (LLM) agents has rapidly evolved from simple retrieval-augmented mechanisms into a data management system that supports persistent information storage, retrieval, update, consolidation, and dynamic lifecycle governance throughout agent execution. Despite this evolution, existing evaluations still benchmark agent memory mainly through end-to-end task success metrics (e.g., F1, BLEU), while treating the underlying system as a monolithic black box. As a result, critical system-level concerns, including operational costs, architectural trade-offs across memory modules, and robustness under dynamic knowledge updates, remain insufficiently explored. In this paper, we present a systematic experimental study of agent memory from a data management perspective. We propose an analytical framework that decomposes agent memory into four core modules: memory representation and storage, extraction, retrieval and routing, and maintenance. Under this framework, we evaluate 12 representative memory systems and two reference baselines across five benchmark workloads spanning 11 datasets. Our extensive end-to-end evaluation shows that no single architecture dominates across all scenarios; instead, effectiveness depends heavily on how well the memory structure aligns with the workload bottleneck. Furthermore, through fine-grained ablation studies, we quantify their individual effects on representation fidelity, retrieval precision, update correctness, and long-horizon stability. Finally, we reveal cost-performance trade-offs under realistic workloads, showing localized maintenance is more cost-efficient than global reorganization. Based on these findings, we identify promising directions towards building truly agent-native memory systems. The code is publicly available at https:// github.com/ OpenDataBox/ MemoryData.
1
Introduction
The rapid evolution of Large Language Model (LLM) agents has sparked a large body of exciting research and industrial efforts in building agent memory, i.e., the data management system of the LLM agent that supports long-horizon stateful execution and personalized interaction [9, 13, 17, 19, 23, 24, 28]. As shown in Figure 1, existing agent memory systems span a diverse set of architectural designs. (1) Stream-and-Reflection Memory System (e.g., MemoryBank [39]) maintains experiences as timestamped memory streams and periodically summarizes them into higher-level reflections that are written back into the stream; ∗ Xuanhe Zhou is the corresponding author.
Memory Extraction
Memory Represent. & Storage
Observation / Experience
query
Memory Stream (Timestamped Log)
User
Retrieval Scorer recency × importance × relevance
retrieved memory (periodic)
Reflection (LLM)
plan
LLM Agent (Act)
User LLM Agent
Message / Trajectory
Message
LLM Agent
Cross-Encoder Reranker
Entity Extractor
Entity Disambiguation Conflict Resolution
User Response / Action
Function Call (self-directed)
Planner (LLM)
search
archival_memory_insert archival_memory_search
Core Memory evict migrate
promote
Recall Storage
Archival Storage
(b) Structural Topological (Hierarchical Tiers)
(a) Structural Topological (Streaming Logs)
User
core_memory_append core_memory_replace
Message
inform
insight write-back Response / Action
Memory Maintenance
Memory Retrieval / Routing
Memory Encoder (Schema-Constrained Extraction)
structured memory objects
memory-augmented context Context LLM Agent Assembler (Act)
ranked results
Storage Router
Hybrid Retrieval
Graph Traversal
persist
Long-Term Mem
Vector DB
indexed data
Graph DB SQL / BM25 consolidate
Temporal Knowledge Graph (Neo4j)
trigger Maintenance Controller (Consolidation, Forgetting)
consolidate / forget
Runtime State
SQL (State) KV Cache
Multi-Engine Storage
(c) Structural Topological (Knowledge Graph)
(d) Multi-Paradigm Hybrid Memory System
Figure 1: Typical Execution Workflows of Agent Memory.
(2) Hierarchical Tiered Memory System (e.g., MemGPT [25]) organizes memory into multiple levels with different capacities and access properties, separating core memory from archival storage with explicit movement (e.g., eviction and promotion) across tiers; (3) Knowledge Graph Memory System (e.g., Mem0𝑔 [5], Zep [26]) represents entities, relations, and their temporal evolution in structured forms (e.g., temporal knowledge graphs), often incorporating entity disambiguation and conflict resolution; (4) Composite Hybrid Memory System (e.g., A-MEM [33]) routes schema-aware memory objects across multiple storage substrates, explicitly separating runtime state (e.g., KV caches) from long-term storage (e.g., vector, graph, keyword indexes), managed by dedicated maintenance modules. However, this rapid proliferation has also led to a highly fragmented landscape that lacks systematic evaluation from a data management perspective, raising a natural question: Are we ready for an agent-native memory system? In this paper, we revisit this question for agent memory. In particular, we focus on system-level memory over textual, structured, and even parametric representations [5, 25, 26, 33], a fundamental infrastructure component for modern autonomous agents. We focus on memory-centric systems, rather than task-specific agent frameworks where memory is an auxiliary module [1, 40]. It is the persistent data management system that maintains information
beyond a single inference step (e.g., historical interactions, environmental observations, and intermediate tool executions) decoupled from the LLMs’ parametric weights and volatile context windows. Agent frameworks rely on these external memory systems (e.g., Mem0 [5], Letta [25], Zep [26], and A-MEM [33]) to actively write, update, index, and route relevant context back into the reasoning loop. The capability of a long-horizon agent largely depends on the reliability and efficiency of this memory layer. An agent adopting a poorly designed memory architecture can suffer from factual contradictions, catastrophic forgetting, or unacceptable latencies during continuous execution [6, 38]. Recent benchmarks [20, 22, 29, 31] have evaluated agent memory and shown that external memory can improve agent performance on tasks requiring factual recall and long-context understanding. However, these evaluations are largely rooted in natural language processing and have multiple limitations when treating agent memory as a data management system (see Section 2). First, they fail to evaluate many representative memory architectures (e.g., systems such as MemoChat, MemTree, and LightMem have not been included in prior evaluations) under unified workloads, making principled cross-system comparisons difficult. Efforts from the database community [32] limit their scope to a few chatbot-centric datasets (e.g., LoCoMo and LongMemEval only), neglecting complex agentic execution scenarios. Second, existing benchmarks predominantly rely on single-sided, end-to-end task success metrics (e.g., F1 and BLEU scores) rather than a comprehensive evaluation suite. They fail to explicitly isolate and measure multi-dimensional performance indicators such as evidence-level retrieval fidelity, dynamic update robustness under conflicting knowledge, and longhorizon stability. Third, they rarely measure key operational costs from a systems perspective, such as index construction time and query latency, which are critical for production deployments. Last, they treat memory systems as monolithic black boxes rather than decomposing them into fundamental data management modules for isolated, fine-grained analysis. We overcome these limitations and conduct comprehensive experiments and analyses from a data management perspective. The contributions are as follows. (1) Technology Decomposition and Taxonomy (Section 3). We decompose existing agent memory systems into four core components: (i) memory representation and storage, (ii) memory extraction, (iii) memory retrieval and routing, and (iv) memory maintenance. For each component, we further establish a structured taxonomy1 by categorizing existing approaches according to their underlying design principles, enabling principled comparisons. (2) Overall End-to-End Performance Evaluation (Section 4). We conduct end-to-end evaluations under a unified and fair testbed (e.g., unified time-overhead traces)2 across five distinct benchmark workloads encompassing 11 datasets. Our study includes 12 representative memory systems, each embodying different combinations of representation, storage, routing, and maintenance strategies. We evaluate their performance from five perspectives: task effectiveness (RQ1), retrieval fidelity (RQ2), dynamic update robustness (RQ3), long-horizon stability (RQ4), and operational cost (RQ5).
(3) Fine-Grained Technical Component Evaluation (Section 5). Leveraging our four-module framework, we conduct controlled and fine-grained experiments on representative strategies within each technique component. By systematically generating controlled variants that modify one module at a time, we quantify their performance trade-offs and assess their individual impacts on representation fidelity, routing precision, and update correctness. (4) Insightful Findings. Based on our experimental results and in-depth analysis, we distill a set of insightful findings regarding the cost–performance trade-offs of agent memory systems: ❶ Are Memory Systems Effective Across Different Agent Request Workloads? No single memory architecture dominates all scenarios. Composite hybrid systems lead on conversational QA, while graph-based methods excel in single-hop factual recall but struggle with temporal reasoning. Moreover, effective memory systems remain robust across LLM backbone variants because they externalize evidence localization before answer generation. ❷ How Accurately Do Memory Systems Retrieve Stored Evidence? Explicit query planning and balanced hybrid search maximize contextual relevance. However, retrieval accuracy degrades significantly as the temporal distance between the evidence and the query increases, exposing limitations of similarity-based retrieval. ❸ Are Memory Systems Robust Under Dynamic Updates? Graph-based methods handle knowledge updates most reliably, whereas popular fact-extraction plugins and append-only stores struggle with targeted overwrites. Systems lacking lifecycle management return stale facts, leading to “hallucinations of the past”. ❹ Do Memory Systems Remain Stable Over Long Horizons? Many append-only memory stores suffer from catastrophic degradation as evidence becomes more distant. For time-dependent queries, raw long-context retrieval still outperforms most memory-backed approaches, indicating that standard semantic consolidation often destroys crucial chronological cues. ❺ What Are the Operational Costs of Agent Memory? Highly structured systems incur orders-of-magnitude higher index construction time and query latency than lightweight stores, yet do not consistently deliver proportional accuracy gains. ❻ When Do Individual Memory Components Go Wrong? Each layer of abstraction (e.g., compression, summarization, and fact extraction) progressively discards information. Furthermore, fine-grained LLM-based extraction can yield modest precision gains but substantially degrade multi-hop reasoning. Finally, conservative memory consolidation serves as the best default maintenance strategy, whereas delayed flushing creates a deceptive trade-off between surface-level coverage and actual answerability.
2
Preliminaries
To support the discussion in the rest of this paper, we first clarify the scope of agent memory from a data management perspective. Although recent studies have examined memory from viewpoints such as cognitive taxonomy, agent architecture, and graph-based organization [6, 10, 30, 32, 34, 36], the underlying concept is still often treated primarily as an algorithmic component of the LLM or agent pipeline [6, 10, 36]. In contrast, we study agent memory as a standalone data management object and system infrastructure, with explicit attention to how it is represented, stored, retrieved,
1 https://github.com/OpenDataBox/awesome-agent-memory 2 https://github.com/OpenDataBox/MemoryData
2
updated, and maintained under real agent workloads. Under this view, we introduce a set of definitions below. Memory Types. For an LLM agent, a wide variety of information is produced and may need to be memorized, including dialogue history, tool execution logs, distilled facts, and user preferences [36]. Following established cognitive frameworks, memory can be broadly organized along two axes [10, 30]. (1) Along the temporal axis, short-term memory holds the volatile state of an ongoing session, while long-term memory persists across sessions. (2) Along the functional axis, long-term memory is further divided into information such as concrete past events (episodic memory), abstracted factual knowledge (semantic memory) [10, 32], reusable action strategies (procedural memory), and user preferences. Agent Memory. We define the agent memory M as the persistent data management object [25, 32] that maintains this cumulative state beyond a single inference step and makes it accessible to the agent during future reasoning and action [10, 36]. Agent Memory System. To operationalize agent memory M, a robust infrastructure is required [10, 25]. As shown in Table 1, from a data systems perspective [14, 32], we formalize the agent memory system as a tuple of four modules: M𝑠𝑦𝑠 = ⟨R, S, Q, U⟩, where each module governs a distinct phase of the memory lifecycle. • (1) Memory Representation and Storage R: A mapping that defines the logical and physical memory format with a data model of two facets: (a) logical representation, spanning simple primitives (discrete tokens, continuous vectors) to complex topologies (knowledge graphs, trees, and composites); and (b) physical storage, utilizing transient registers, specialized single-engine databases, or multi-engine backends for persistence and indexing. • (2) Memory Extraction S: A mechanism governing how heterogeneous input streams (e.g., multi-turn dialogues, tool logs) are transformed into logical memory primitives via pipelines such as raw sequence concatenation, schema-free semantic extraction, or schema-constrained structured extraction. • (3) Memory Retrieval and Routing Q: A function that dynamically identifies relevant memory subsets based on a query context, utilizing specific routing algorithms to traverse indices. Mechanisms span native attention-based retrieval, semantic 𝐾-nearest neighbor search, topological subgraph traversal, autonomous agentic routing via LLM planning, and multi-stage hybrid execution. • (4) Memory Maintenance U: Policies governing the dynamic lifecycle of memory entries, decomposed into three sub-operations: (a) Conflict Resolution and Versioning handles contradictions via multi-versioning, invalidation, or precedence rules; (b) Capacity Management enforces bounded growth through constraint-based hard eviction (e.g., FIFO, token limits) or score-based priority eviction (e.g., temporal decay); and (c) Semantic Consolidation utilizes the LLM to merge redundant assertions into dense summaries or execute CRUD operations via tool-calling interfaces. Distinction from RAG and Context Engineering. RetrievalAugmented Generation (RAG) [8, 13] typically operates as a stateless, read-only retrieval primitive: given a query, it fetches relevant passages from a static corpus to augment a single generation step. Context engineering [2] is the broader practice of curating the finite LLM context window at each inference turn (e.g., dynamically
2
1 Token-Level Sequence
t1: "Kim is vegetarian ..." t2: "Charlie is Kim's pet ..." t3: "Daily event log …"
Graph & Tree-Based Topology
Vegetarian Food
...
PREFERS KEEPS_PET
Kim
Human-Readable Text
(t ₁)
Kim's Memory
... Preference
LIVES_IN
(t ₂)
Paris
...
...
t2
tT
(t ₃)
(t ₁)
(t ₃) Keeps
Vegetarian Food
h ∈ R ᵀˣᵈ
t1
Life
(t ₂)
Charlie (a) Explicit Discrete Text Token
Lives in Paris
Charlie
(Kim, LIVES_IN, Paris)
[0.21, …, …, 0.46]
Node: Entity, Edge: Relation
Root: Summary, Leaves: Raw Facts
(a) Temporal Knowledge Graph
(b) Hierarchical Tree Structures
Latent State Vector (b) Implicit Continuous Vector Token
... Heterogeneous
t1
3 Composite
t2
Type & Date
"Kim is ..."
[ 0.21,... , 0.46 ]
Metadata
Text
Embedding
...
Paris
Kim
...
LIVES_IN
Representation Memory Object
Graph
Figure 2: Memory Representation Methods. selecting prompts, tool descriptions, and retrieved facts) to mitigate context rot [2]. In contrast, an agent memory system (1) is a persistent and updatable infrastructure for managing agent-specific state over time and (2) governs the full long-term memory lifecycle, including memory representation, storage, retrieval, and maintenance, rather than merely packing the current context window. Distinction from Traditional Database Workloads. Agent memory workloads differ substantially from conventional database OLTP / OLAP workloads [17, 25]. First, memory access is often semantic rather than purely predicate-based [4, 11]. Queries are commonly expressed through natural language, partial context, or latent intent, and therefore rely on approximate matching, query rewriting, or LLM-guided retrieval rather than only exact logical predicates over rigid schemas. Second, memory contents evolve under continuous and potentially conflicting observations. Unlike conventional transactional settings, where updates typically overwrite tuples under a predefined schema and consistency model, agent memory must accommodate uncertain, partial, and sometimes contradictory information collected across time, tools, and environments [10, 38]. Third, agent memory workloads are highly heterogeneous in both access pattern and granularity. A single workload may combine long-context synthesis, episodic recall, structured fact lookup, temporal reasoning, and streaming updates. As a result, practical systems often require hybrid execution strategies that combine semantic retrieval, structured filtering, and topologyaware traversal within one memory architecture [25, 32]. These properties distinguish agent memory from traditional databases, motivating dedicated abstractions and evaluation methodologies.
3
Method Overview
In this section, we carefully analyze existing agent memory systems across the four components in Section 2 and establish a unified taxonomy that summarizes representative component methods.
3.1
Memory Representation and Storage
This module consists of two components: (1) logical representation, which defines the structural encoding and organization exposed to the agent system, directly dictating capacity, accessibility, and trade-offs in expressiveness, retrieval granularity, and downstream reasoning compatibility; and (2) physical storage, which designates the persistence and indexing structures, such as volatile in-context registers, dense vector engines, or topological graph databases. 3
Table 1: Taxonomy and Characteristics of Agent Memory Systems. Category
MEM1 [41]
Memory Representation & Storage Representation Storage ❶ Token-Level Sequence ❶ Transient In-Context Registers (Structured JSON Memos) ❶ Token-Level Sequence ❷ Specialized Single-Engine (Discrete Facts) (Vector DB) ❶ Token-Level Sequence ❶ Transient In-Context Registers
MemAgent [35]
❶ Token-Level Sequence
❶ Transient In-Context Registers
❷ Graph & Tree-Based Topology (Hierarchical Tree) ❷ Graph & Tree-Based Topology (Temporal KG) ❷ Graph & Tree-Based Topology (Labeled Graph) ❷ Graph & Tree-Based Topology (Entity-Relation Triplets) ❸ Heterogeneous Composite (Tripartite Schema)
❷ Specialized Single-Engine (Vector DB) ❷ Specialized Single-Engine (Graph DB) ❸ Heterogeneous Multi-Engine (Vector + Graph DB) ❸ Heterogeneous Multi-Engine (Graph + Vector + Relational DB) ❷ Specialized Single-Engine (Relational DB) ❸ Heterogeneous Multi-Engine (Vector DB + BM25 + SQL) ❸ Heterogeneous Multi-Engine (Vector + Graph DB) ❸ Heterogeneous Multi-Engine (Keyword Index + Vector DB) ❸ Heterogeneous Multi-Engine (Vector + Graph DB) ❷ Specialized Single-Engine (Relational DB)
Method MemoChat [18]
Sequential Context
Mem0 [5]
MemTree [27] Structural Topological
Zep [26] Mem0𝑔 [5] Cognee [21] LightMem [7] SimpleMem [16]
Multi-Paradigm Hybrid
MemOS [15] MemoryOS [12] A-MEM [33] Letta [25]
❸ Heterogeneous Composite ❸ Heterogeneous Composite (MemCube) ❸ Heterogeneous Composite (Segment-Page) ❸ Heterogeneous Composite (Atomic Notes) ❸ Heterogeneous Composite (Context Tiers)
❸ Schema-Constrained Extraction (LLM Topic Segmentation)
Memory Retrieval & Query Routing ❹ Autonomous Agentic Routing (LLM Topic Selection)
❷ Schema-Free Extraction
❷ Semantic-Based Retrieval
❶ Raw Sequence Concatenation ❶ Raw Sequence Concatenation (Recursive Summaries) ❷ Schema-Free Extraction (Top-Down Embedding) ❸ Schema-Constrained Extraction (Triplets) ❸ Schema-Constrained Extraction (Entity-Relation) ❸ Schema-Constrained Extraction (ECL Pipeline via Pydantic) ❷ Schema-Free Extraction (Entropy-Gated)
❶ Native Attention-Based Retrieval
Memory Extraction
Memory Maintenance
❷ Semantic-Based Retrieval (Collapsed Tree) ❺ Multi-Stage Hybrid Execution (Dense + BM25 + BFS)
❸ LLM-Driven Semantic Consolidation (Turn-Triggered) ❸ LLM-Driven Semantic Consolidation (Tool-Calling) ❷ Capacity-Driven Physical Eviction ❷ Capacity-Driven Physical Eviction (RL Overwrite) ❸ LLM-Driven Semantic Consolidation (Recursive Aggregation) ❶ Timestamp-Based Multi-Versioning (Logical Invalidation)
❸ Topological Subgraph Traversal
❶ Timestamp-Based Multi-Versioning
❸ Topological Subgraph Traversal (Dense-Seeded Triplet Extraction)
❶ Timestamp-Based Multi-Versioning (Hash-Based Deduplication) ❶ Timestamp-Based Multi-Versioning (Append-Only Logs) ❸ LLM-Driven Semantic Consolidation (On-the-Fly Synthesis) ❶ Timestamp-Based Multi-Versioning (Differential Writes) ❷ Capacity-Driven Physical Eviction (Heat-Based Eviction) ❸ LLM-Driven Semantic Consolidation (Mutation & Pruning) ❷ Capacity-Driven Physical Eviction (Queue Flush)
❶ Native Attention-Based Retrieval
❷ Semantic-Based Retrieval
❸ Schema-Constrained Extraction
❹ Autonomous Agentic Routing (Query Expansion) ❺ Multi-Stage Hybrid Execution (Boolean + Semantic) ❺ Multi-Stage Hybrid Execution (Hierarchical Routing)
❸ Schema-Constrained Extraction (JSON Attributes)
❸ Topological Subgraph Traversal
❸ Schema-Constrained Extraction
❹ Autonomous Agentic Routing (Function Calling)
❸ Schema-Constrained Extraction ❸ Schema-Constrained Extraction (Semantic Parser)
3.1.1 Logical Representation. As displayed in Figure 2, this component acts as a bridge between raw data and the execution environment by organizing memory into clear models, such as graphs, or vector spaces. It determines how efficiently a system can search, combine, and use historical context for complex tasks. ❶ Token-Level Sequence Representation. This category models memory as flat, one-dimensional sequences lacking explicit structural abstractions (e.g., graphs or hierarchies). Memory is represented either as discrete, human-readable natural language tokens or as implicit, continuous latent vector tokens (e.g., fact embeddings, hidden states, or KV-cache tensors). ▶ Explicit Discrete Text Token. This category models memory as human-readable strings or independent factual statements. For instance, Mem0 isolates memory into discrete natural language facts extracted directly from interaction history. Similarly, MemoChat structures multi-turn dialogues into discrete JSON blocks (topics, summaries, raw turns) to maintain topical coherence within a plain-text paradigm. While these systems externalize their plaintext memory, others retain it within the active processing window: MemAgent restricts its internal belief state to a strictly bounded text sequence (e.g., 1024 tokens), and MEM1 encapsulates internal-state summaries within specialized boundary tags (e.g., <IS>). ▶ Implicit Continuous Vector Token. Departing from readable text tokens, this sub-category encodes memory as continuous vectors, which may be materialized either as external embeddings attached to facts and summaries for semantic retrieval, or as model-side latent states such as compressed internal states and attention caches. For example, Mem0 represents extracted facts as dense semantic embeddings and MemoRAG utilizes specifically initialized weight matrices to compress raw inputs into high-dimensional Key-Value (KV) cache tensors. Although these vector-token representations reduce explicit tokenization burdens and integrate naturally with retrieval or inference pipelines, they sacrifice structural interpretability and are difficult to manipulate via fine-grained operations, such as predicate-level filtering or targeted updates to encoded facts. ❷ Graph and Tree-Based Topological Representation. This category abstracts memory into structured graph and tree topologies with interconnected nodes and edges, allowing conversational entities, high-level concepts, and their temporal or semantic relationships to be explicitly modeled and computationally traversed.
1 Transient In-Context Register
3 Heterogeneous Multi-Engine
t1
t2
...
tT
t1
Buffer
"Kim is ..."
Specialized Single-Engine
File Object
t2
Paris
...
LIVES_IN
No External Store
2
Graph
Embedding
Text
Relational DB
Vector DB
Single-Specialized Store
Graph DB
[ 0.21,... , 0.46 ]
Kim
Memory Router
File / Relational / Vector / Graph DB
Multi-Distributed Store
Figure 3: Memory Storage Methods.
▶ Temporal Knowledge Graphs.This sub-category models memory using graph topologies to map entities and their interconnections, natively supporting temporal reasoning and conflict detection. For example, Zep partitions memory into formally defined, temporallyaware knowledge graphs (e.g., episode, entity, and community subgraphs). Similarly, Mem0𝑔 formalizes memory as a directed labeled graph, where vertices represent entities and edges encapsulate relationship triplets (e.g., “LIVES_IN”). To aid temporal reasoning, entity nodes are enriched with structural metadata like semantic types, dense embeddings, and creation timestamps. ▶ Hierarchical Tree Structures. This sub-category organizes knowledge into recursive, hierarchical structures, preserving highly granular observations at terminal leaves and broad semantic abstractions at ancestor nodes. For example, MemTree models memory as a dynamic, directed tree schema. Each node is structured as a tuple containing textual content, a dense embedding, topological pointers, and a depth scalar. Within this topology, deep leaf nodes retain isolated facts (e.g., a player scoring), while ancestor nodes provide high-level conceptual summaries (e.g., the match result), with a specialized root node serving as the definitive entry point. ❸ Heterogeneous Composite Representation. This category moves past simple token sequences and standard graphs by packaging memory into complex, multi-part data containers. These architectures directly combine unstructured text with highly structured metadata (e.g., timestamps, categorical labels, vector embeddings, and network links) to form a single functional unit. For example, 4
Bot
"Got it. I'll recommend vegetarian restaurants."
tv_T 1
t2
t1: "Kim is vegetarian ..." t2: "Charlie is Kim's pet ..." t3: "Daily event log …"
★ Fact “Kim is vegetarian ...”
LLM
t3
Buffer
★ 1, 2, ...
- Kim: "I'm vegetarian." - Bot: "Got it. I'll recommend vegetarian restaurants."
"I'm vegetarian."
Kim
Embedding
★ t1 t3
★ Vector [0.21, …, 0.46]
Vegetarian Food
... KEEPS_PET
(t ₁)
...
Kim LIVES_IN
Structured Record
Vegetarian Food
... (Kim, LIVES_IN, Paris)
(t ₂)
(Kim, PREFERS, Vegetarian Food)
Paris
Vector Index (HNSW / IVF)
}
"tool": "search_memory", "query": "Charlie lives", "top_k": 5 LLM
Original Queries
LLM
(t ₃)
Retriever
Q₁: Charlie residence ... Q₂: Charlie preferred ... Q₃: Is Charlie vegetarian?
Memory Storage Expanded Search Queries
1-hop subgraph
★ Kim lives in Paris.
Kim
c2 c1 c3
★
★
{
★
Paris
...
PREFERS
LLM
0.78
Tool Call
Charlie
"event_type": "visit", "actor": "Kim", "summary": "Tried several vegetarian cafés in Paris", "timestamp": "2024-06-15", "sensitivity": "normal" }
0.83
4 Autonomous Agentic Routing
3 Topological Subgraph Traversal
Structural Predefined Schema
{
0.86
tT
[0.21, …, 0.46]
PREFERS
3 Schema-Constrained Structured Extraction
Top-k
t2
Attention Weight
No Rigid Predefined Schema
Bounded Context Buffer
2 Semantic-Based Dense Retrieval
1 Native Attention-Based Retrieval
2 Schema-Free Semantic Extraction
..., 2, 1
1 Raw Sequence Concatenation
(a) Function Call Invocation (b) Generative Query Expansion
...
LIVES_IN
5 Multi-Stage Hybrid Execution
Extracted Topology
★
★
Figure 4: Memory Extraction Methods.
BM25 Full-Text
File Object
★
Relational DB
Fusion & Rerank
Keyword Search
Filter: type = Preference time > 2024-01-01
MemOS proposes the MemCube, a unified data object that organizes memory into three distinct payloads (plain-text, activation, and parametric memory) alongside structured details (e.g., ID tags).
Top-k
Deterministic Filter (Stage 1)
c2 0.83 c 1 0.78 c 3 0.86
Semantic Ranking (Stage 2)
Vector Search
c2
Semantic Search Vector DB
Graph Traversal Topological Search
(a) Sequential Hybrid Routing
Graph DB
(b) Parallel Ensemble Retrieval
Figure 5: Memory Retrieval Methods.
3.1.2 Physical Storage and Indexing. As shown in Figure 3, this component manages how data is physically stored and accessed, relying on systems like in-memory caches, files, vector engines, or databases. It sets the actual capacity limits and determines the speed, throughput, and overall scalability of memory operations. ❶ Transient In-Context Register. To eliminate disk I/O and external traversal latency, this category retains memory exclusively within the active hardware state (e.g., dynamic context windows or KV caches). MemoChat avoids dedicated external memory engines and keeps structured JSON-style memos within the LLM context input during its memorization-retrieval-response loop, while MemAgent directly stores summary tokens as Key-Value (KV) cache tensors via dense positional embeddings. ❷ Specialized Single-Engine Storage. This category physically warehouses formulated units within a standalone, homogeneous backend strictly tailored to the memory’s logical structure. Depending on the ingestion paradigm, architectures deploy specific backend topologies: (1) Dense Vector Databases are utilized to project data into continuous high-dimensional spaces; Mem0 and MemTree use centralized vector stores, Letta leverages PostgreSQL with the pgvector extension; (2) Graph Databases are deployed to enforce topological constraints; both Zep and Mem0𝑔 execute predefined Cypher queries to physically persist logical graph components into Neo4j; (3) Relational SQL Engines are used to serialize structural and temporal schemas. LightMem incrementally appends factual streams to preserve global relational states; (4) File or Object Stores preserve raw interaction artifacts (e.g., conversation histories or tool-execution logs) as files or object blobs. ❸ Heterogeneous Multi-Engine Storage. This category dynamically constructs multiple index typologies or distributes data across heterogeneous backends (e.g., pairing a dense vector store with a topological graph database). SimpleMem ingests memory into LanceDB with an IVF-PQ mechanism that concurrently maintains dense embeddings, sparse BM25 indices, and SQL predicates. MemoryOS relies on a hybrid index fusing dense cosine similarity with discrete Jaccard similarity. Conversely, MemOS delegates serialized
payloads to highly specialized independent backends, fusing Vector and Graph databases via a standardized memory adapter interface.
3.2
Memory Extraction
Memory extraction concerns how raw interaction traces are computationally processed. It covers both the extraction pipeline, how language models extract, summarize, or parse unstructured text into logical structures. As shown in Figure 4, it defines how the agent memory system transforms heterogeneous input streams (e.g., multi-turn dialogues, and tool execution logs) into logical memory primitives prior to physical persistence. ❶ Raw Sequence Concatenation. To minimize computational overhead, this category bypasses explicit extraction prompts, formulating memory directly as raw token concatenations or transient state summaries (e.g., appending recent dialogue turns directly into a prompt buffer). Systems such as MEM1 and MemAgent retain their newly formulated structures exclusively within the active computational state without secondary parsing. ❷ Schema-Free Semantic Extraction. This category systematically distills raw, unstructured inputs into independent, high-value informational units, representing them either as explicit free-form texts or as compressed, continuous latent vectors. By isolating core knowledge from broader conversational context, it ensures precise and granular retrieval. For example, Mem0 actively parses interactions to extract and store discrete, standalone factual statements (e.g., “User is vegetarian and dairy-free”). ❸ Schema-Constrained Structured Extraction. This category prompts the LLM to parse raw inputs and synchronously populate a rigidly predefined structural schema, producing strictly typed data rather than free-form text. The constrained output takes the form of either topological entity-relation triplets for graph insertion or multi-modal relational payloads for hybrid storage, depending on the target backend. Zep and Mem0𝑔 extract typed directed relational edges (e.g., LIVES_IN, WORKS_AT) conforming to predefined graph 5
schemas, with Zep additionally applying a reflection-inspired verification step to suppress hallucinated triplets. MemoChat populates predefined structural fields to ensure data predictability by leveraging LLMs to segment conversations into strict JSON schemas.
1 Timestamp-Based Multi-Versioning
C1 INVALID
c1: Kim lives in Paris. valid_to: 2025-06-30
...
...
VALID
3.3
2 Capacity-Driven Physical Eviction
Append-Only Log (Chronological precedence)
Memory Retrieval and Query Routing
Memory retrieval and query routing determine how the agent memory system dynamically identifies and extracts relevant historical context to inform the overarching agent’s current reasoning state. As shown in Figure 5, this module encompasses the complete query execution spectrum, defining the operational algorithms, predicate evaluations, and agentic workflows utilized to traverse indices. ❶ Native Attention-Based Retrieval. To bypass external database I/O, this category uses the transformer’s native computational graph as the sole retrieval engine, relying entirely on self-attention mechanisms to implicitly weight and route information (e.g., scanning dialogue tokens directly within the KV cache). MEM1 performs implicit retrieval via self-attention over the current sequence, utilizing a two-dimensional attention mask to preserve causal consistency. MemAgent implements routing by concatenating blocks directly into the prompt template, enabling standard attentionbased decoding without external cross-encoder reranking. ❷ Semantic-Based Dense Retrieval. Operating over continuous latent spaces, this category maps query tensors against uniform vector indices to extract localized spatial neighbors (e.g., executing a standard 𝐾-Nearest Neighbors (KNN) search). Mem0 calculates vector embeddings for incoming queries to execute a dense similarity search, fetching a constrained subset of facts. LightMem utilizes efficient cosine-similarity distance calculations over dense embeddings, bypassing computationally expensive iterative reranking. MemTree implements a collapsed-tree architecture that mathematically flattens its hierarchy, broadcasting inbound vectors to compute global cosine-similarity distributions across all candidates. ❸ Topological Subgraph Traversal. Departing from continuous vector spaces, this category retrieves information by traversing explicit relationship edges to extract semantic clusters structurally grounded in knowledge graphs (e.g., hopping from a User node to a linked Preference node). Mem0𝑔 deploys an entity-centric heuristic to recursively traverse local subgraphs synchronously with semantic triplet evaluations. A-MEM identifies candidate anchors via dense 𝐾-Nearest Neighbor selection, then executes localized graph traversal to access topologically adjacent memory nodes explicitly linked within the same conceptual cluster. ❹ Autonomous Agentic Routing. Rather than executing deterministic database scans, this category delegates retrieval to the LLM itself, thereby functioning as an active, autonomous query planner. It generates tool-call invocations or drafts implicit search criteria. ▶ Function Call Invocation. This sub-category bridges the LLM with external storage by generating explicit function call commands to directly execute predefined database operations (e.g., outputting a valid JSON payload to trigger an external database API). For example, Letta orchestrates self-directed memory retrieval where the LLM evaluates its active context to explicitly generate localized function calls (e.g., emitting an archival_storage.search() command to extract targeted historical logs).
VALID
C2
C4
C3
✗Evicted (Oldest)
FIFO (Capacity = 3)
(a) Constraint-Based Hard Eviction
...
c2: Kim lives in London. valid_from: 2025-07-01
c3: Kim keeps a pet.
Memory Item
Last Access
Score
Kim's favorite album
1d ago
0.82
Kim's pizza toppings
30d ago
0.21
Old meeting notes
180d ago
0.05
✗Evict (Lowest)
valid_from: 2026-04-10 (b) Score-Based Priority Eviction
3 LLM-Driven Semantic Consolidation LLM / Tool Caller
Updated Summary "Kim is vegetarian" "Kim prefers Italian food"
Kim is vegetarian and prefers, Italian cuisine (e.g., pasta, pizza).
Create Read
CREATE(scene_summary)
READ(user_preferences)
Update UPDATE(m_001, new_fact) Merge
Prune
Augment Delete
LLM
Group facts De-duplicate Add details
(a) Inline Semantic Compaction
DELETE(m_045, m066)
(b) Tool-Driven CRUD Execution
4 Continuous Parametric Optimization Offline Training / Fine-tuning θₜ → θₜ₊₁ Historical Context
Retrieval Result
User Feedback
Optimize with RLGF / LoRA
Figure 6: Memory Maintenance Methods. ▶ Generative Query Expansion. Unlike rigid function calling, this approach uses natural language generation to synthesize intermediate clues or decompose complex intents before mapping them to the index (e.g., rewriting vague prompts into descriptive search strings). SimpleMem uses an Intent-Aware Retrieval Planning module where the LLM dissects queries, calculates adaptive search depths, and synthesizes optimized query variants. ❺ Multi-Stage Hybrid Execution. To overcome the recall limitations of single-paradigm searches, this category executes multiengine query pipelines orchestrating multi-dimensional candidate generation followed by downstream reranking frameworks. ▶ Sequential Hybrid Routing. This sub-category chains retrieval paradigms into a strictly ordered pipeline, systematically pruning the search space with deterministic predicates before executing fine-grained semantic extraction (e.g., applying strict SQL date filters before computationally expensive vector searches). MemoryOS executes a federated routing strategy featuring coarse-grained predicate evaluation followed by fine-grained semantic ranking strictly within isolated segments. It algebraically fuses rule-based structural Boolean filtering with dense semantic similarity routing. ▶ Parallel Ensemble Retrieval. In contrast to sequential filtering, this approach maximizes initial recall by simultaneously dispatching queries to multiple distinct indexing algorithms, followed by a late-stage fusion and reranking phase to optimize the aggregated pool (e.g., concurrently fetching candidates via BM25 and dense vector search, then cross-encoding the results). Zep executes simultaneous cosine semantic scans, Okapi BM25 full-text searches, and topological BFS, subsequently optimizing precision via RRF, MMR, and computationally intensive cross-encoder models.
3.4
Memory Maintenance
Memory maintenance concerns how memory is updated, maintained, compressed, forgotten, and eventually removed over time. 6
As shown in Figure 6, it captures the dynamic behavior of memory after it has been created, including how new information is incorporated, how outdated or conflicting content is revised, and how the system controls memory growth under limited resources. ❶ Timestamp-Based Multi-Versioning. Rather than executing physical row deletions, this category preserves historical continuity by utilizing timestamp metadata and append-only logs to logically deprecate expired facts. Operating via explicit metadata mutations, Zep and Mem0𝑔 avoid physical deletion by marking obsolete or conflicting relationships as logically invalid using validity flags and timestamps. Taking an append-only approach, LightMem incrementally inserts timestamped factual streams, while SimpleMem resolves contradictions through strict chronological precedence using ISO-8601 timestamps. Synthesizing these techniques, MemOS leverages a structured Update API to execute differential writes, seamlessly updating provenance IDs to generate multi-version chains. ❷ Capacity-Driven Physical Eviction. In contrast to timestampbased multi-versioning, this category manages unbounded memory growth by physically dropping or unconditionally overwriting data. It executes this physical pruning through either strict deterministic constraints or dynamically calculated eviction scores. ▶ Constraint-Based Hard Eviction. This sub-category enforces rigid execution bounds by utilizing deterministic rules—such as strict FIFO queues, fixed sequence boundaries, or hard token limits—to unconditionally evict older states. Executing structural overwrites, MemAgent implements a programmatic scheduling algorithm that unconditionally replaces older memory sequences with newly synthesized summary blocks at every fixed segment boundary. Enforcing hard capacity limits, MEM1 operates through a system-enforced truncation mechanism that executes an automated FIFO pruning protocol to evict older tags once active context thresholds are breached. Operating via threshold flushes, Letta strictly handles buffer capacities via an OS-inspired queue manager; when the token count breaches a terminal limit, it forces a flush sequence to evict older messages into secondary recall storage. ▶ Score-Based Priority Eviction. Rather than relying on static capacity limits, this sub-category dynamically forces the physical obsolescence of data by continuously calculating temporal decay or access-frequency scores. Quantifying access frequency, MemoryOS measures segment vitality via a scalar Heat score that balances retrieval frequency against exponential temporal decay, executing priority evictions that physically target the lowest-heat segments. ❸ LLM-Driven Semantic Consolidation. Operating as a cognitive governor, this category leverages the LLM to dynamically resolve logical conflicts and abstract redundant observations into dense summaries prior to query or persistence phases. ▶ Inline Semantic Compaction. During the active write phase, this sub-category dynamically evaluates and consolidates newly ingested data against existing memory nodes, systematically merging redundant assertions prior to database transaction commitment (e.g., compressing three similar dialogue turns into one dense summary node). SimpleMem executes online semantic synthesis onthe-fly, systematically merging structurally similar assertions into singular dense abstractions prior to database transaction commitment. MemTree utilizes a core scheduling operation that recursively
triggers a semantic summarization prompt across all parent nodes to dynamically fuse historical states with novel payloads. ▶ Tool-Driven CRUD Execution. In contrast to automated fusion, this sub-category operationalizes maintenance through discrete, programmed state-mutations guided explicitly by LLM-driven tool interfaces that issue explicit Create, Read, Update, or Delete (CRUD) commands. Mem0 operationalizes its dynamic maintenance strictly through structured LLM tool-calling interfaces encompassing discrete programmed state-mutations such as UPDATE, and DELETE. ❹ Continuous Parametric Optimization. Completely decoupling state updates from online inference latency, this category executes heavy neural optimizations as asynchronous background processes, modifying the actual model parameters rather than the external database schema (e.g., running continuous fine-tuning on overnight batches). For example, MemoRAG leaves active inference tokens strictly static and read-only, optimizing extraction quality exclusively during an offline training phase via a Reinforcement Learning with Generation Feedback (RLGF) algorithmic framework.
4
End-to-End Assessment
In this section, we conduct a systematic evaluation of agent memory systems across five research questions. Across five distinct benchmark workloads and 11 datasets, we assess 12 representative memory systems against baselines to characterize their performance. Specifically, the five research questions are as follows.
4.1
Overall Effectiveness (RQ1)
Experimental Setting. For “Do different agent memory systems successfully improve end-to-end task performance across workloads?”, we evaluate 12 representative memory systems and two reference baselines (Long Context and Embedding RAG) on the three end-toend workloads to assess whether memory improves task success beyond the underlying LLM. Specifically, we use: (1) LoCoMo [20]: a long-conversation QA benchmark that tests episodic, temporal, and open-domain memory over multi-turn interactions, and report the unweighted mean of category-level Exact Match (EM) and Answer F1 on the four-category queries; (2) LongMemEval [31]: a multi-session long-memory benchmark that evaluates whether systems can reconnect facts across sessions and reason over temporally distributed evidence, and report Substring EM, ROUGE-L F1, ROUGE-L Recall, and GPT-5.4-based LLM Judge Accuracy from MemoryAgentBench [22]; and (3) DB-Bench: evaluates whether memory supports procedural execution across database operations from LifelongAgentBench [37], and report Exact Match (EM) and Task Success Rate. O1-(Cross-Workload Effectiveness): No single memory system dominates all workloads, but methods that preserve taskcritical evidence through structure-guided filtering remain the most competitive overall. As shown in Figure 7, the leading systems shift across workloads: (1) Structure-aware systems lead LongMemEval, where Zep reaches 48.0 LLM Judge Accuracy and Cognee attains 35.3 ROUGE-L F1; (2) Hybrid filtering is strongest on LoCoMo exactness, where MemOS reaches 11.5 Exact Match (EM); and (3) Trace-preserving memories remain strongest on DBBench, where Long Context achieves 48.20 EM and MemoChat reaches 55.40 Task Success Rate. However, among methods with full 7
20.7
20
14.7
15
12.3
11.1 7.7
8.7
7.0
7.7
7.3
5
28.6 22.8
21.9
20
15.5
14.5 13.7
10
16.7
15.6 7.8
6.2
(a) LongMemEval: Substring EM
50 44.1
4.6
4.1 3.0
17.6
0
29.2
28.1 24.5
50 23.5
21.5
20 14.7
13.0 12.8
61.6
60
32.2 26.2
48.2
45.4
42.0
41.6
40
36.8
30
34.4
44.0 43.8
34.4 28.0
27.6 22.8
20
10 5.3
10
0
0
(g) DB-Bench: EM
(f) LoCoMo: Answer F1
60 50
48.0 40.7
40
39.3 33.3
33.0
34.7
30 23.0 19.0
20
16.7
16.0
10
18.7 14.7
17.3
3.7
0
(d) LongMemEval: LLM Judge Acc.
(c) LongMemEval: ROUGE-L Recall
g Em Co be nte d. xt R A M Me G em m oC 0 C ha o Ze gn t p ee L M oc em al Tr ee Li Le g Si ht tta m Me pl m eM M e M em m em O or S y A OS -M EM
g Em Co be nte d. xt R A M Me G em m oC 0 C ha o Ze gn t p ee L M oc em al Tr ee Li Le g Si ht tta m Me pl m eM M e M em m em O or S y A OS -M EM
Lo n
(e) LoCoMo: EM
32.8
30
0.0
20.1
10
EM
5.1
5
18.8
ng Em Co be nte d. xt R A M Me G em m oC 0 C ha o Ze gn t p ee L M oc em al Tr ee Li Le g Si ht tta m Me pl m eM M e M em m em O or S y A OS -M EM
5.7
Answer F1
9.3
8.6
20.2 19.3
70
Lo n
EM
9.4
10.1
27.1 26.1
20
(b) LongMemEval: ROUGE-L F1 32.8
11.5 9.8
35.9
34.9
33.5
33.1
30
40
9.7
40.8
39.8
40
Lo ng Em C o be nt e M d. xt em RA A G ge M M nt em em oC 0 C ha Ze ogn t p ee L M oc em a Tr l e Li L e e Si ght tta m M pl em eM e M Me m em m or OS y A OS -M EM
ng Em C o be nt e M d. xt em RA A G ge M M nt em em oC 0 C ha Ze ogn t p ee L M oc em a Tr l e Li L e e Si ght tta m M pl em eM e M Me m em m or OS y A OS -M EM
Lo
Lo
ng Em C o be nt e M d. xt em RA A G ge M M nt em em oC 0 C ha Ze ogn t p ee L M oc em a Tr l e Li L e e Si ght tta m M pl em eM e M Me m em m or OS yO A S -M EM
0
15
0
29.0
30
0
10
33.7
Lo
10
20.7
19.7
35.3 35.0
LLM Judge Acc.
25
40
Lo ng Em C o be nt e M d. xt em RA A G ge M M nt em em oC 0 C ha Ze ogn t p ee L M oc em a Tr l e Li L e e Si ght tta m M pl em eM e M Me m em m or OS y A OS -M EM
28.3
Task Success Rate
29.7
70 61.6
60
55.4
50
48.2
45.4
30
42.0
41.6
40
27.6 22.9
25.8
25.8
44.0 43.8
28.1
20 10 0
Lo ng Em Co be nte d. xt R A M Me G em m oC 0 C ha o Ze gn t p ee L M oc em al Tr ee Li Le g Si ht tta m Me pl m eM M e M em m em O or S y A OS -M EM
27.7
ROUGE-L Recall
30
ROUGE-L F1
Substring EM
35
Multi-Paradigm Hybrid
Structural Topological
Sequential Context
Reference Baselines
(h) DB-Bench: Task Success Rate
Figure 7: Effectiveness of Memory Systems over LoCoMo, MemoryAgentBench (LongMemEval), LifeLongAgentBench (DB-Bench). workload coverage, MemoryOS and MemOS remain closest to the frontier overall, suggesting that robustness comes not from a single universal memory form, but from preserving the right evidence at the right level of abstraction before final matching. In particular, (1) Temporal or graph-organized memory is most useful for cross-session aggregation and event-order reasoning (e.g., scattered personal facts in LongMemEval); (2) Summary-first or coarse-tofine routing is useful for exact grounding in long but semantically coherent dialogues (e.g., recovering a specific date or personal detail in LoCoMo); and (3) Trace-preserving memory is necessary when correctness depends on intermediate state changes and operation order (e.g., dependent UPDATE and INSERT operations in DB-Bench). O2-(Beyond Exact Match): EM remains informative for tasks with canonical, directly grounded outputs, but it becomes insufficient when correctness depends on paraphrastic synthesis or executable success. As shown in Figure 7, Exact Match (EM) is still a meaningful signal on LoCoMo, where many questions target short grounded facts, as reflected by MemOS achieving the best Exact Match (EM). On LongMemEval, however, the stronger systems are more clearly separated once semantic equivalence is considered through ROUGE-L and LLM Judge Accuracy, indicating that cross-session reasoning often yields correct answers that do not share a single canonical surface form. On DB-Bench, the limitation is even clearer: Long Context achieves the best Exact Match (EM), but MemoChat attains a substantially higher Task Success Rate, showing that exact output matching does not fully capture whether memory supports successful execution. These results suggest that Exact Match (EM) is most appropriate when answers are short, canonical, and locally verifiable (e.g., a venue name, or object attribute in LoCoMo), but should be complemented once tasks require cross-session synthesis, or end-task state validation (e.g., composing a semantically correct answer from multiple sessions in LongMemEval or reaching the correct table state in DB-Bench).
100
100
75
75
100 85.9
39.0
31.3 27.6 23.6
24.8
25
15.2
12.3
59.7
55.4
52.5
46.9
50 34.7 28.3
25
13.3
Recall@10 (%)
50
Recall@5 (%)
Recall@1 (%)
80.5 69.5 64.6
16.0
75
66.2
75.1 70.6 61.3 56.4
50
25
40.2
17.7
3.8
0 dd
be
Em
ing
0 RA
G
m0 Me
0
e p G m m OS OS EM Ze Tre RA Me Me m ry m ht ple Me mo A-M ing Me LigSim dd Me be Em
m0 Me
e p G m m OS OS EM Ze Tre RA Me Me m ry m ht ple Me mo A-M ing Me LigSim dd Me be Em
m0 Me
e p m m OS OS EM Ze Tre Me Me m ry m ht ple Me mo A-M Me LigSim Me
(a) Recall@k
100
Embedding RAG
Zep Local
LightMem
MemOS
Mem0
MemTree
SimpleMem
MemoryOS
A-MEM
Recall@10 (%)
80
60
40
20
0 1-5
6-10
11-15
16-20
21-25
26-31
Evidence Distance Gap (Session Bins)
(b) Recall@k vs. Evidence Distance Gap
Figure 8: Retrieval Results of Memory Systems over LoCoMo. bottleneck: (1) for dispersed cross-session reasoning, relation- and time-aware retrieval is most effective, as in Zep and Cognee; (2) for long but semantically coherent dialogue, coarse-to-fine filtering improves exact grounding, as in MemOS and MemoryOS; and (3) for stateful execution, preserving interaction traces is more critical than exact lexical matching alone, as in Long Context.
4.2
Memory Retrieval Fidelity (RQ2)
Experimental Setting. For “How accurately can a memory system surface the stored evidence required by a query?”, we evaluate eight representative memory systems to assess evidence-level retrieval fidelity independently of downstream answer generation. Specifically, we use LoCoMo [20], which provides source-level gold evidence for queries with diverse evidence distances. We report: (1) Recall@K, where a hit requires the top-𝑘 retrieved source-id groups to contain the annotated gold evidence, and (2) Recall@10 over six evidence distance gap bins (1–5 to 26–31), defined by the
Finding 1. (Workload-Aligned Memory). RQ1 suggests that strong agent memory is not defined by a single universal representation, but by how well it supports the dominant workload 8