Diachronic Hypergraphs for Orchestrated Multi-Agent Multimodal Memory Curation

Diachronic Hypergraphs for Orchestrated Multi-Agent Multimodal Memory Curation

arXiv:2608.29678v1 [cs.DB] 30 Aug 2026

Yichao Feng1,2∗ , Ran Zhang2∗ , Haoran Luo2† , Zhenghong Lin2 , Carl Yang3 , Anh Tuan Luu2† 1 LIGHTSPEED, Singapore 2 Nanyang Technological University, Singapore 3 Emory University, Atlanta, USA {yichaoafeng, shrleyzhang}@global.tencent.com, haoran.luo@ieee.org, hongzhenglin970323@gmail.com, anhtuan.luu@ntu.edu.sg Abstract Multi-agent systems solve tasks through collaboration, tool use, multimodal reasoning, and orchestration, but each agent operates within a knowledge boundary defined by its observations, context, and resources. Memory must preserve and transfer evidence, role specific context, decisions, procedures, and experience across interactions, not only outcomes. Vector and graph memories flatten these structures into embeddings or dyadic traces, obscuring events involving agents, tools, documents, errors, and evidence. This limits knowledge sharing, tracing, reuse, revision, and orchestration. We present MAGE, a hypergraph based multimodal database designed as a memory engine for MAS. MAGE stores agents, messages, tools, errors, procedures, documents, entities, decisions, and evidence in a heterogeneous temporal hypergraph, preserving high order collaborative events as reusable memory. It supports decision driven updates, role aware retrieval, validation, lifecycle management, and budget bounded context packing. By delivering knowledge to agents and orchestrators, MAGE expands their knowledge boundaries without modifying the models. Experiments show MAGE outperforms on various memory baselines. Our code is available.1

CCS Concepts • Graph based database; • Multi-agent systems;

Keywords Temporal hypergraph database, Multi-agent memory management

1

Introduction

Multi-agent systems (MAS) have emerged as a promising direction for constructing capable and adaptable AI systems [11, 13]. By distributing problem solving across interacting agents, MAS can enhance performance on complex tasks [4, 8]. This design is particularly suitable for tasks requiring heterogeneous knowledge, multiple information sources, tool interactions, multimodal evidence, and iterative revision as new evidence appears. The rapid progress of large language models (LLMs)[6] has further expanded the practical scope of MAS, since LLMs provide flexible agent backbones with strong language understanding, generation, coding, and reasoning abilities [17, 18, 28, 48]. Yet, each agent operates within a knowledge boundary determined by its context, role, observations, ∗ Equal contributions. † Corresponding authors. 1 Github Code: https://github.com/Githubuseryf/MAGE

Knowledge Boundaries Costly Post Training

Limited Transfer Manual Pipeline

Slow Adaptation

Inadequate Representation

Fixed Role Order

Evidence Agent Retriever

Planner

New Evidences

Frozen parameters

Delayed Correction

Executor

Task Evidence

Critic

Weak Revision

Unclear Error Source

Limited Adaption

Action

Procedure

Current Memory Designs Miss The Structure Of Multi-Agent Experience

Required Memory Capability Editable Experience Add

Update

Delete

Noop

Role Conditioned Recall

Planner View

Temporal Lifecycle

Retriever View Active

Critic View

Deprecated

Archived

Deleted

High Order Multimodal

Hyperedge Hub

Executor View

Figure 1: Challenges of existing MAS memory databases.

tools, and histories. The collective capability of MAS therefore depends not only on individual agents and orchestration, but also on preserving and transferring knowledge across these boundaries without losing evidence, semantics, uncertainty, or provenance. However, MAS face three challenges, as in Fig. 1. First, systems lack memory for extending agent knowledge boundaries and updating knowledge over time [52, 60]. Information may remain isolated within its originating agent, preventing other agents from accessing evidence, decisions, or experience when solving tasks. Incorporating it into models may require post training [53], while repeated updates can degrade generalization and cause catastrophic forgetting [26, 33]. Second, MAS knowledge transfer is limited to textual messages, summaries, or intermediate outputs between agents and workflow stages [50]. Evidence may be omitted, uncertainty compressed, role specific meanings lost, and conclusions detached from their actions or sources. These incomplete transfers can propagate errors and leave downstream agents or orchestrators unable to identify reliable, outdated, conflicting, or relevant knowledge [45]. Third, memory structures remain inadequate for representing MAS knowledge [55]. Flat memories store isolated fragments, while graph or tree based memories mainly capture hierarchical or pairwise relations [21]. Such structures struggle to preserve high order collaborative events involving agents, roles, tools, documents, entities, intermediate states, and temporal dependencies [3]. Binary edges fragment their shared context and obscure which participants, actions, and evidence jointly produced a decision [9].

Feng et al.

Temporal MAS Reasoning Trace Role

Segment 1

Segment 2

Segment 3

Segment 4

Hyperedge Construction Role Aware Retrieval And Downstream Use T1 Collab Hyperedge Task

Active

Agent

Retrieval Pipeline

Update

Pose Question

Decompose Goal

Plan Next Step

Refine Plan

Message

Deprecate

Action

Role Query

Archived

Role Specific Output Views Planner View Insight

Procedure

Collab

Memory

Planner T2 Evidence Hyperedge Gather Sources

Pull Evidence

Cross Reference

Modality

Confirm Citations

Update

Retriever Re-Execute Step

Compute Result

Finalize Output

T3 Procedure Hyperedge

Executor

Procedure

Spot

Decompose Goal

Plan Next Step

Temporal Filter

Insight

Memory

Run Tool

Role Policy Gate Active

Entity

Refine Plan

Insight

Critic

Insight

Action

Memory

Entity

T4 Error Correction Hyperedge Error

Hold Context

Update Insight

Draft Insight

Active

Action

Retriever View Modality

Evidence

Memory

Entity

Bilateral Expansion

Budgeted Packing

Selected Memory Subgraph

Critic View Error Correction

Update

Condense Trace

Deprecate

Summarize

Entity

Memory

Archived

Downstream Agent Context

Memory

Figure 2: Temporal memory construction and role aware retrieval in MAGE. MAS reasoning traces are segmented over time, converted into various hyperedges, and later retrieved as role conditioned memory subgraphs for downstream agents. To address these challenges, we propose MAGE (Multi-Agents Graph Engine), as shown in Fig. 2, a training free hypergraph memory engine that connects MAS knowledge boundaries. First, MAGE externalizes collaborative experience into a shared memory where observations, evidence, decisions, errors, and lessons can be corrected, validated, and reused without modifying LLM agents. Knowledge from one agent or stage remains accessible to agents and tasks while retaining the source, provenance, lifecycle, and evolution of transferred knowledge. Second, MAGE supports knowledge transfer through role aware retrieval. Instead of forwarding undifferentiated histories, it retrieves evidence, procedures, decisions, and experience according to the agent’s role, task state, and information needs. Planners recover constraints and procedures, retrievers identify evidence requirements, executors reuse validated actions, and critics inspect sources and errors. This memory grounds orchestration when selecting agents, assigning responsibilities, and organizing interactions, making orchestration a beneficiary of transferable knowledge rather than a separate memory function. Third, MAGE introduces a heterogeneous temporal hypergraph whose multiway structure is suitable for capturing MAS interactions. An event may involve agents, roles, documents, tools, actions, states, decisions, evidence, and temporal dependencies. Pairwise edges split this event into relations, obscuring how these elements produced a result. MAGE instead preserves their many to many relations as memory units through nodes, edges, and hyperedges. Lifecycle states and provenance links retain evidence, revisions, error locations, and steps. By reconstructing collaborative episodes and retrieving role conditioned subgraphs, MAGE transfers knowledge across agent boundaries while keeping it traceable, revisable, and reusable for reasoning and orchestration.

2

Related Work

MAS Orchestration and Agent Memory. LLM based agents have been widely used for diverse tasks. ReAct[58] and ToT[57] support reasoning actions and search over intermediate thoughts, while CAMEL[27], AutoGen[51], ChatDev[38], G-memory[59], and MetaGPT[20] organize MAS through role communication and SOP style workflows[37]. Long term memory enables agents to reuse experience beyond a fixed context window. Agents retrieve memories by relevance, recency, and importance and synthesize reflections for planning, while Reflexion stores feedback as episodic memory[46]. MemoryBank[61] adds memory updates with Ebbinghaus style forgetting, and Voyager[49] stores reusable skills for lifelong embodied learning. Recent systems such as Mem0[10], A-MEM[54], and Zep[42] explore extraction, consolidation, dynamic linking, memory evolution, temporal KG memory, and cross session knowledge maintenance across diverse application scenarios. Graph and Hypergraph. RAG augments LLMs with external memory [25], while graphs preserve dependencies. GraphRAG [12] builds graphs and summaries; LightRAG [14] enables multilevel retrieval; RAPTOR [43] builds summary trees; and HippoRAG [16] uses PageRank for multi-hop retrieval. KG2 RAG [62] expands chunks through graph relations, whereas G-Retriever [19] selects subgraphs with Prize Collecting Steiner Trees. PathRAG [7] retrieves relational paths, and HybGRAG [24] combines textual and relational evidence. These remain pairwise or hierarchical. HyperGraphRAG [31] represents n-ary facts as hyperedges, while HyperRAG [29] retrieves n-ary relational chains. MAGE instead models mutable MAS experience, capturing how it evolves, retains provenance, and is reused across agent roles. They address distinct retrieval and memory management problems, not interchangeable systems.

Diachronic Hypergraphs for Orchestrated Multi-Agent Multimodal Memory Curation

Input Streams

Text Documents

Insert Document

Insert Media

Insert Trace

Query Role

Collaboration

Agent Roles

Error Correction

Planner

MAS Episode

L6 Interface

Task

Consolidate Collect Garbage

Planner

Message

Executor

Action 1

Error

Action 2

L5 Retrieval Knowledge Ingestion Images

Role Bilateral Multi Signal Hard Seeding Expansion Ranking Gating

Budget Provenance Tracking Packing

Deduplication Merge

PDF Files

Retriever

Entity B

N-ary Fact

Memory

L4 Maintenance

Tables

Entity A

Modality

Garbage Consistency Decay Collection Validation Check

Evidence

Knowledge Ingestion Procedure

L3 Ingestion

Insight

Successful Action

Community

Archetype PDF Files

Visual Text Trace Extraction Parsing Mining

Fallback Rules

Typed Hyperedges Nodes

Meta Procedure

L2 Indexing

Tool Actions

Error Traces

Role Policy

Community Report

Memory

Insight

Generalizes

Critic

Summarizer

Retrieval Assembly

Agent Messages

Vector Typed Temporal Index Adjacency Recency

Procedure

Executor

Lifecycle State

Scoped Search

Seed Set

Expansion Pool

Provenance

Source Path

Ranked Candidate

Packed Context

Budgeted Subgraph

L1 Storage

Append Log

Graph Store

Version Storage Snapshots Transactions Chain

Time Window

Role Filter

Token Budget

Figure 3: Layered architecture of MAGE. Heterogeneous evidence streams are stored, indexed, ingested, maintained, and retrieved through six modular layers, producing role aware budgeted subgraphs from a heterogeneous temporal hypergraph.

3

Preliminaries

We fix the notation that the rest of the paper relies on; design choices specific to MAGE are deferred to Sec. 4. Evidence stream. An MAS S produces, over time, a stream E1:𝑡 = D1:𝑡 ∪ M1:𝑡 ∪ T1:𝑡

(1)

of textual documents, multimodal items 𝜎 𝑗 = (uri 𝑗 , mod 𝑗 , 𝑡˜𝑗 , 𝜇 𝑗 ) with mod 𝑗 ∈ {image, table, audio, video, pdf, webpage}, and interaction traces
𝜏𝑘 = 𝑞𝑘 , A𝑘 , 𝜋𝑘 = (𝑠𝑘,1, . . . , 𝑠𝑘,𝑇𝑘 ), 𝑟𝑘 , (2) where A𝑘 are role typed agents under 𝜌 : A𝑘 → R, 𝜋𝑘 is a step trajectory, and 𝑟𝑘 is the final result. Heterogeneous hypergraph. The data structure that absorbs this evidence is a heterogeneous graph whose nodes carry types T𝑉 , hyperedges types T𝐻 , and labelled binary edges types T𝐸 . A hyperedge generalizes a binary edge to a 𝑘 ary tuple ℎ = (𝑣𝑖 1 , . . . , 𝑣𝑖𝑘 ),

𝑣𝑖 𝑗 ∈𝑉 , 𝑘 ≥ 2.

(3)

For storage and indexing we use the bipartite materialization Φ𝐵 (G) = (𝑉 ∪𝐻, 𝐸 ∪ {(ℎ, 𝑣) : ℎ ∈ 𝐻, 𝑣 ∈ℎ}),

(4)

which is bijective on G and lets every 𝑘 ary edge be served by standard bipartite or graph indexes.

Why hyperedges, not pairwise projections. Decomposing one 𝑘
ary fact into 𝑘2 pairwise edges is lossy: it confuses co occurrence with 𝑛 ary co participation and erases multiplicity, an ambiguity recently formalized for hypergraph reconstruction. We therefore keep hyperedges as first class storage units. Bitemporal lifecycle. Memory items must remain auditable as the world evolves, so each 𝑥 ∈𝑉 ∪𝐻 carries a bitemporal stamp
𝚯(𝑥) = txf (𝑥), txt (𝑥), valf (𝑥), valt (𝑥) (5) separating transaction time (when the system believed 𝑥) from valid time (when 𝑥 holds in the world). The predicate     live𝑡 (𝑥) =1 txf (𝑥) ≤𝑡 < txt (𝑥) ·1 valf (𝑥) ≤𝑡 < valt (𝑥) (6) is true exactly when both windows are open, and a lifecycle map 𝐿 : 𝑉 ∪ 𝐻 → {active, deprecated, archived, deleted} records monotone descents. Budget bounded query. Given a future query 𝑞 ′ under role 𝜌 and a token budget 𝐵, the engine returns the budget bounded subgraph ∑︁ G𝑞★′ ,𝜌 =arg max 𝑠 𝜌 (𝑥 | 𝑞 ′, G), (7) ′ G ⊆G ′ live𝑡 ( G ′ )=1 𝑥 ∈ G tok( G ′ ) ≤𝐵

where tok(·) is the rendered context token cost; this is a 0/1 budget knapsack problem with 𝐵 a first class constraint.

Feng et al.

4

Methodology

MAGE turns the abstract memory of Sec. 3 into a working system.

4.1

System Architecture

G𝑡 = Φtxt (D1:𝑡 ) ∪ Φmm (M1:𝑡 ) ∪ Φtrc (T1:𝑡 ),

MAGE is organized into six layers (Fig. 3), split into a data plane and a policy plane. The data plane consists of (L1) Storage, which provides backends with snapshots and transactions; (L2) Indexing, adj tmp which maintains 𝐼𝑡 = (𝐼𝑡vec, 𝐼𝑡 , 𝐼𝑡 , 𝐼𝑡role, 𝐼𝑡lc ) over semantic, structural, temporal, role, and lifecycle signals; and (L3) Ingestion, which lifts text, multimodal items, and MAS traces into nodes and hyperedges. The policy plane includes (L4) Lifecycle and Consistency, which performs deduplication, merging, decay, garbage collection, MVCC versioning, and validation; (L5) Retrieval, which performs seeding, hypergraph expansion, ranking, and budget-bounded packing; and (L6) API, which exposes MAS-facing insertion, retrieval, consolidation, and garbage-collection calls.

4.2

Modality Dispatch and Extraction. Let D1:𝑡 denote ingested text documents, M1:𝑡 multimodal items, and T1:𝑡 MAS traces. The graph is assembled as:

Temporal Hypergraph Memory Model

Memory state. At time 𝑡, MAGE represents a running MAS as a heterogeneous temporal hypergraph:
G𝑡 = 𝑉𝑡 , 𝐻𝑡 , 𝐸𝑡 , 𝐴𝑡 , 𝐼𝑡 , 𝑅𝑡 , 𝐶𝑡 , 𝐿𝑡 . (8) Here, 𝑉𝑡 , 𝐻𝑡 , 𝐸𝑡 , 𝐴𝑡 describe what is remembered; 𝐼𝑡 , 𝑅𝑡 describe how it is served; and 𝐶𝑡 , 𝐿𝑡 describe how it ages. Graph vocabulary. The vocabulary consists of ten node types and five hyperedge types: T𝑉 = { Task, Agent, Message, Action, Error,

(11)

where Φtxt , Φmm , and Φtrc are LLM-backed extractors with deterministic fallbacks. For a text document 𝑑, the text extractor decomposes 𝑑 into a memory node, referenced entities, and 𝑛-ary claims: 𝑒 (𝑑 ) Φtxt (𝑑) = {𝑚𝑑 } ∪ {𝑒 𝑗 }𝑁𝑗=1  ∪ (𝑚𝑑 , 𝑒𝑖 1 ,. . . ,𝑒𝑖𝑘 ) : (𝑖 1,. . . ,𝑖𝑘 ) ∈ F𝑑 ,

(12)

where each claim is materialized as a NaryFact hyperedge. For a multimodal item 𝜎, MAGE creates a modality pointer and surrogate text memory: Φmm (𝜎) = {𝑚𝜎 , 𝑜 𝜎 } ∪ {(𝑜 𝜎 , 𝑚𝜎 , btm)} ∪ Φtxt (𝑡˜𝜎 ),

(13)

where 𝑡˜𝜎 is the extracted textual description. This surrogate design is deliberate: the memory layer stores auditable structured representations with provenance pointers to the raw item, which keeps every retrieval decision inspectable while preserving access to the original pixels whenever a downstream reasoner needs them (Appendix J). For a trace 𝜏 = (𝑞, A, 𝜋, 𝑟 ), Φtrc produces task, agent, message, action, and error nodes, together with Collab, ErrCorr, and Procedure hyperedges. It further distills traces into reusable Memory and Procedure nodes with provenance for later task reuse across future contexts. Four-Way Decision Policy. Each new memory 𝑚★ is reconciled against existing memories to prevent unbounded growth. MAGE retrieves top-𝐾 domain-compatible neighbors C and applies:

Memory, Insight, Procedure, Modality, Entity },

(9)

𝑑 ∈ {ADD, UPDATE, DELETE, NOOP},

T𝐻 = { NaryFact, Collab, ErrCorr, Evidence, Procedure }.

(10)

The node types capture both raw collaboration evidence and durable knowledge, including tasks, agents, messages, actions, errors, memories, insights, procedures, modality pointers, and entities. Hyperedges preserve high-order relations: NaryFact groups entities in an 𝑛-ary statement; Collab binds a task with its agents and steps; ErrCorr links an error to corrective actions; Evidence ties a source to supported memories; and Procedure binds a workflow to its steps. Relations that do not require an 𝑛-ary container are represented by typed binary edges such as produces, supersedes, contradicts, merged_into, and generalizes for efficient retrieval. Indexing and lifecycle. Every 𝑥 ∈ 𝑉𝑡 ∪ 𝐻𝑡 carries a schemavalidated payload 𝑎(𝑥) ∈ 𝐴𝑡 , an embedding e𝑥 = 𝑓𝜃 (𝑥) ∈ R𝑑 , a confidence value 𝜅 (𝑥) ∈ [0, 1], and the bitemporal stamp of Eq. (5). The index 𝐼𝑡 partitions embeddings by type-specific scopes, so a query under role 𝜌 only touches admissible scopes. The role family 𝑅𝑡 = {𝜋 𝜌 } 𝜌 ∈ R stores retrieval policies, while 𝐶𝑡 and 𝐿𝑡 govern validation and aging under bounded memory budgets.

4.3

𝛿 : (𝑚★, C) ↦→ (𝑑, 𝑗 ★, 𝑚 ⋄ ),

Write Pipeline: Ingestion and Decision

When new evidence arrives, MAGE invokes the corresponding insert API, converts the input into typed graph elements, admits valid items, and commits them atomically.

(14)

where 𝑑 is the selected action, 𝑗 ★ is the matched neighbor, and 𝑚 ⋄ is the merged memory under UPDATE. The decision is made by an LLM judge or a deterministic heuristic:

𝛿ℎ (𝑚★, C) =

 (N, 𝑗𝑠★, ⊥)      (D, 𝑗 ★, ⊥)  𝑠

 (U, 𝑗𝑠★, 𝑚 ⋄ )     (A, ⊥, ⊥) 

𝑠 ★ ≥ 𝜏𝑛 ∧ Π = 0 𝑠 ★ ≥ 𝜏𝑐 ∧ Π = 1 𝜏𝑢 ≤ 𝑠 ★ < 𝜏𝑛 otherwise,

(15)

where 𝑠 ★ = max𝑚𝑖 ∈ C cos(e★, e𝑚𝑖 ), Π detects polarity flips, and 𝜏𝑛 , 𝜏𝑐 , 𝜏𝑢 control near-duplicate, contradiction, and partial-overlap cases during incremental memory consolidation updates. Graph Rewrite and Persistence. The decision is materialized as:   G𝑡 ∪ {𝑚★ } 𝑑 =A        G𝑡 ℎ ← (ℎ\{𝑚★ }) ∪{𝑚 𝑗 ★ } 𝑑 =N  G𝑡 +1 = (16) sup  G𝑡 ∪ {𝑚 ⋄ , (𝑚 ⋄ −−→𝑚 𝑗 ★ )} 𝑑 =U      G𝑡 ∪ {(𝑚★ −inv −→𝑚 𝑗 ★ )} 𝑑 =D.  ⋄ Under UPDATE, 𝑚 inherits provenance and the old memory is deprecated; under DELETE, the contradicted memory records invalidating evidence. Only Memory nodes traverse 𝛿; entities and facts are persisted unconditionally. Committed items are threaded supersedes supersedes into an MVCC chain 𝑥 (0) −−−−−−−−→ · · · −−−−−−−−→ 𝑥 (𝑣) , supporting

Diachronic Hypergraphs for Orchestrated Multi-Agent Multimodal Memory Curation

point-in-time reads. A validator enforces provenance, temporal ordering, role admissibility, lifecycle consistency, merge correctness, and bounded confidence across evolving graph states.

4.4

Read Pipeline: Role-Aware Retrieval

Any agent can call query_for_role(𝑞 ′, 𝜌, 𝐵) with query 𝑞 ′ , role 𝜌, and token budget 𝐵. MAGE returns a relevant, role-admissible, and budget-bounded subgraph G𝑞★′ ,𝜌 through: 𝑓𝜃

expand

seed

pack

score

𝑞 ′ −−→ e𝑞 ′ −−−→ S −−−−−→ X −−−−→ X★ −−−→ G𝑞★′ ,𝜌 .

(17)

Seeding and Expansion. The query is embedded into e𝑞 ′ . For each role-admissible scope 𝜎 ∈ Σ𝜌 , MAGE returns top-𝑘 items by role-boosted cosine: Ø   𝜌 S= argtop-𝑘 sim(e𝑞 ′ , e𝑥 ) · 𝛽 𝜌 (type(𝑥)) ∪ S𝐻 . (18) 𝜎 ∈Σ𝜌

𝑥 ∈𝑉𝑡𝜎

It then expands from seeds across hyperedge membership, coparticipants, and typed binary neighbors:  X (ℓ+1) = X (ℓ ) ∪ 𝑦 ∈ 𝑁𝐻 (𝑥) ∪ 𝑁𝑉 (𝑥) ∪ 𝑁𝐸 (𝑥) : (19) 𝑥 ∈ X (ℓ ) , type(𝑦) ∈ 𝜋 𝜌 .Tok, live𝑡 (𝑦) = 1 , for ℓ = 0, . . . , 𝐿−1 and X (0) = S. This supports multi-hop paths while enforcing role admissibility and temporal liveness. Scoring and Packing. Expanded candidates are ranked by: 𝑠 𝜌 (𝑐) = 𝑤 sem sim(e𝑞 ′ , e𝑐 ) + 𝑤 role 𝛽 𝜌 (𝑐) + 𝑤 conf 𝜅 (𝑐) + 𝑤 prov𝜙 (𝑐) + 𝑤 rec rec𝑡 (𝑐) + 𝑤 use𝑢 (𝑐) − 𝑤 lc ℓ (𝑐) − 𝑤 tok𝜏 (𝑐). Γ𝜌𝑡 (𝑐) = live𝑡 (𝑐) ∧ 𝐿(𝑐) ≠ deleted ∧ type(𝑐) ∈ 𝜋 𝜌 .Tok, which prevents leakage across role boundaries. The gated candidates are packed into a token-limited context: ∑︁ ∑︁ Y★ = arg max 𝑠 𝜌 (𝑐) s.t. tok(𝑐) ≤ 𝐵. (21) 𝑐∈Y

𝑐∈Y

MAGE greedily admits high-value candidates and renders them as: Ê ctx(G𝑞★′ ,𝜌 ) = 𝜓 (𝑐), 𝜓 (𝑐) = [type(𝑐)] 𝜉 (𝑐). 𝑐 ∈ Y★

Admitted memories update their usage count and last-used timestamp.

4.5

Maintenance: Lifecycle and Self Evolution

Between writes and reads, MAGE maintains graph quality through consolidation and garbage collection. Two live nodes of the same type are duplicates when: cos(e𝑥 , e𝑦 ) ≥ 𝜏c . The canonical winner is:
𝑥 † = lex-argmax𝑧 ∈ {𝑥,𝑦 } 𝜅 (𝑧), |src(𝑧)|, −𝑡 create (𝑧) ,

where 𝑡 last (𝑥) is the last access time, 𝑆 0 is the base stability, 𝜅𝑆 controls rehearsal scaling, and 𝑢 (𝑥) is the cumulative usage count. Thus frequently reused memories decay more slowly, while one-off memories become GC candidates earlier. Each live item carries a composite value: 𝜈 (𝑥) = 𝛼 1𝜅 (𝑥) + 𝛼 2 𝑅(𝑥, 𝑡) + 𝛼 3𝑢 (𝑥) + 𝛼 4𝑔(𝑥) + 𝛼 5 |src(𝑥)| − 𝛼 6 age(𝑥) − 𝛼 7 cnt(𝑥) − 𝛼 8 stor(𝑥).

(22)

favoring higher confidence, richer provenance, and older stable nodes. The loser is linked through merged_into, and its hyperedge references are forwarded. MAGE also induces Insight, Community, Archetype, and MetaProcedure nodes for coarse-grained retrieval under limited context and token budgets.

(24)

Here, 𝑔(𝑥) denotes downstream gain from successful tasks, |src(𝑥)| measures provenance support, cnt(𝑥) counts contradictions from newer evidence, and stor(𝑥) penalizes storage cost. A threshold 𝜃 drives the lifecycle ladder: active → deprecated → archived → deleted. After successful tasks, memories that contributed to the accepted output receive higher 𝑢 (𝑥) and 𝑔(𝑥), making useful memories harder to evict while stale or contradicted memories are gradually demoted during subsequent lifecycle management rounds.

4.6 (20)

Before packing, MAGE applies:

Y ⊆ X★

Ebbinghaus Retention and GC. MAGE models retention through exponential decay whose stability grows with rehearsal:  𝑡 − 𝑡 (𝑥)  last 𝑅(𝑥, 𝑡) = exp − , (23) 𝑆 (𝑥)
𝑆 (𝑥) = 𝑆 0 1 + 𝜅𝑆 ln(1 + 𝑢 (𝑥)) ,

Orchestrated Execution

For complex tasks, MAGE uses an orchestrated solve loop backed by the shared hypergraph. Given task 𝑞 and agent pool A, the orchestrator generates a workflow DAG, dispatches role-specific sub-tasks, retrieves memory for each role, and revises the workflow when feedback exposes unresolved gaps. Corrected errors become retrievable ErrCorr hyperedges, while successful workflows become reusable procedural memories. AOV Pipeline. The Orchestrator receives task 𝑞 with Plannerrole memory and emits an AOV DAG over R = {Pln, Ret, Exe, Cri}.:  𝑁 P = 𝑝𝑖 = (id𝑖 , 𝜌𝑖 , 𝜄𝑖 , 𝐷𝑖 ) 𝑖=1𝑝 , 𝐷𝑖 ⊆ {𝑝 1, . . . , 𝑝𝑖 −1 }, (25) where 𝜌𝑖 is the role, 𝜄𝑖 is the instruction, and 𝐷𝑖 is the dependency set. Each sub-step retrieves memory under its role policy. Topological Dispatch and Re-orchestration. MAGE partitions P into topological layers and executes independent steps concurrently. Each step forms a compound query, retrieves roleadmissible memory, executes the sub-agent, and records the output. The Critic evaluates all outputs; if the verdict is Revise, the Orchestrator produces a residual pipeline targeting the identified gaps. The loop stops once the result is accepted or the maximum revision depth 𝑑 max is reached. Trace Writeback. After termination, the full trajectory is ingested through Φtrc into G𝑡 , making accumulated experience available to future tasks while preserving provenance. Complexity. Let 𝑛 = |𝑉𝑡 |, Δ be the maximum typed degree, |Σ𝜌 | the number of admissible scopes, and |S|, |X| the seed and expanded set sizes. The read cost is


𝑇read = 𝑂 |Σ𝜌 | 𝑘 log 𝑛 + 𝑂 𝐿Δ|S| + 𝑂 |X| log |X| . (26) Writes cost 𝑂 (𝐾)+𝑂 (Δ), while maintenance costs 𝑂 (𝑛 log 𝑛).

Feng et al.

Table 1: Main results for RQ1 and RQ2 across GPT-4.1-mini, Claude-4.5-Sonnet, and Qwen2.5-VL-32B-Instruct backends. Text-only IND Method

Text-only OOD

Multimodal IND

Multimodal OOD

HotpotQA

NQ-Open

TriviaQA

WebQ.

ChartQA

DocVQA

Infogr.

FinMME

A-OKVQA

SciQA

TextVQA

VizWiz

F1

F1

F1

F1

F1

F1

F1

F1

F1

F1

F1

F1

EM

EM

EM

EM

EM

EM

EM

EM

EM

EM

EM

EM

GPT-4.1-mini results MAGE

51.08 41.67 54.70 38.54 68.17 63.54 53.48 34.38 83.71 79.17 47.64 26.04 81.94 72.92 59.22 26.04 79.29 62.50 89.55 63.54 93.65 88.54 40.79 16.67

NoMem Mem0 A-MEM MemBank GenAg Voyager Zep G-Mem MetaGPT ChatDev MacNet

37.30 40.12 39.36 41.73 44.83 43.76 42.73 45.89 47.50 40.82 43.02

MAGE

43.31 38.54 56.73 36.46 82.43 73.96 51.11 32.29 84.15 78.12 46.46 25.00 84.00 76.04 39.91 20.83 72.64 58.33 73.28 65.62 92.55 88.54 40.84 17.71

NoMem Mem0 A-MEM MemBank GenAg Voyager Zep G-Mem MetaGPT ChatDev MacNet

28.41 32.15 31.66 32.24 34.17 33.24 34.17 37.91 35.49 40.21 32.05

MAGE

47.35 40.62 29.08 17.71 27.52 25.00 40.61 22.92 80.56 76.04 46.62 26.04 80.74 77.08 44.66 23.96 72.52 60.42 90.02 59.38 94.41 90.62 45.99 27.08

NoMem Mem0 A-MEM MemBank GenAg Voyager Zep G-Mem MetaGPT ChatDev MacNet

36.56 39.23 36.37 37.84 41.62 41.52 39.35 42.95 41.70 44.05 40.82

27.08 29.17 34.38 34.38 31.25 36.46 38.54 36.46 35.42 33.33 33.33

44.41 49.36 48.76 49.43 47.69 51.40 49.35 48.44 48.94 50.46 47.07

22.92 29.17 28.12 37.50 27.08 36.46 37.50 29.17 30.21 34.38 30.21

55.91 59.67 60.03 56.70 62.71 56.22 66.07 58.81 55.47 60.09 58.89

54.17 46.88 50.00 50.00 55.21 42.71 62.50 56.25 53.12 54.17 51.04

38.66 44.22 45.64 42.06 42.11 38.58 43.46 46.83 49.58 39.40 41.91

16.67 29.17 25.00 25.00 20.83 19.79 19.79 26.04 28.12 18.75 23.96

73.32 72.48 75.20 73.79 81.01 77.38 79.35 79.02 76.33 77.99 75.78

67.71 59.38 70.83 69.79 75.00 68.75 72.92 71.88 71.88 69.79 63.54

33.86 39.69 40.59 34.86 34.86 37.53 43.16 39.00 41.77 33.82 44.06

15.62 16.67 20.83 10.42 10.42 13.54 21.88 14.58 18.75 9.38 22.92

71.29 75.01 70.19 74.24 75.05 72.39 76.29 74.86 78.74 77.05 75.31

58.33 64.58 57.29 69.79 68.75 60.42 67.71 66.67 66.67 69.79 65.62

46.18 52.08 51.29 52.44 47.48 48.42 50.42 53.43 52.37 54.32 50.30

8.33 25.00 13.54 21.88 10.42 19.79 15.62 25.00 21.88 19.79 23.96

72.48 70.64 71.99 71.12 72.61 70.30 69.66 71.20 76.89 71.54 73.44

53.12 51.04 59.38 60.42 55.21 58.33 48.96 61.46 56.25 53.12 54.17

79.87 77.80 79.59 81.62 85.35 82.54 80.79 82.44 81.05 79.95 83.82

57.29 61.46 62.50 50.00 54.17 56.25 60.42 62.50 59.38 47.92 58.33

86.15 83.54 84.54 80.12 81.55 87.28 89.64 83.85 85.18 90.15 90.50

81.25 72.92 77.08 75.00 72.92 78.12 82.29 76.04 73.96 82.29 83.33

30.23 32.06 30.14 31.01 33.27 36.06 34.03 37.39 28.86 30.30 34.70

7.29 11.46 4.17 5.21 12.50 14.58 12.50 15.62 7.29 6.25 10.42

Claude-4.5-Sonnet results

21.88 25.00 23.96 20.83 22.92 18.75 26.04 32.29 30.21 28.12 27.08

46.10 52.70 50.70 51.41 49.93 53.73 52.69 49.78 51.74 54.03 49.72

27.08 35.42 32.29 33.33 31.25 35.42 32.29 31.25 34.38 33.33 28.12

74.10 77.24 76.09 75.69 79.13 73.97 80.03 76.93 73.85 77.81 76.68

60.42 71.88 70.83 71.88 72.92 64.58 72.92 67.71 69.79 69.79 63.54

37.51 42.00 46.03 40.50 39.33 41.22 42.02 47.11 46.53 38.70 40.71

17.71 27.08 27.08 23.96 22.92 26.04 20.83 30.21 31.25 15.62 18.75

74.56 75.63 76.35 76.20 81.25 79.61 80.28 80.94 76.72 79.45 72.39

68.75 64.58 66.67 66.67 73.96 73.96 75.00 76.04 67.71 72.92 62.50

31.20 38.19 39.06 36.70 38.82 35.51 42.96 37.17 38.36 39.31 42.96

7.29 15.62 16.67 12.50 17.71 11.46 23.96 13.54 14.58 20.83 22.92

71.08 74.81 76.08 74.31 75.71 72.94 77.42 75.22 80.20 77.14 77.61

59.38 65.62 71.88 61.46 64.58 63.54 72.92 67.71 75.00 73.96 71.88

26.52 30.47 29.12 34.51 25.46 30.06 30.60 32.57 32.26 33.16 32.64

6.25 3.12 9.38 6.25 1.04 1.04 2.08 4.17 4.17 5.21 5.21

61.46 60.27 63.85 62.14 63.34 61.28 59.92 62.94 69.64 62.27 63.80

44.79 46.88 44.79 46.88 42.71 43.75 42.71 43.75 51.04 45.83 47.92

59.60 57.10 58.03 61.17 68.48 61.84 58.91 62.11 62.58 57.19 65.40

33.33 33.33 40.62 35.42 42.71 38.54 38.54 39.58 44.79 28.12 48.96

88.21 85.22 85.16 82.62 87.73 87.71 89.34 83.99 85.86 90.25 89.63

79.17 81.25 77.08 71.88 85.42 84.38 86.46 80.21 73.96 85.42 80.21

31.09 34.74 33.45 33.28 33.30 36.98 35.25 37.44 31.58 32.12 36.66

9.38 13.54 13.54 6.25 8.33 16.67 10.42 15.62 11.46 8.33 14.58

Qwen2.5-VL-32B-Instruct results

5

23.96 25.00 22.92 30.21 28.12 32.29 26.04 37.50 29.17 37.50 31.25

18.10 23.60 22.93 24.67 20.22 24.64 22.28 21.87 21.69 26.18 21.87

3.12 13.54 2.08 13.54 1.04 7.29 4.17 1.04 2.08 14.58 11.46

16.45 20.61 20.81 19.81 21.10 19.12 25.62 18.79 16.94 21.27 19.54

11.46 7.29 10.42 13.54 18.75 8.33 20.83 9.38 6.25 15.62 8.33

25.88 35.30 32.00 29.14 28.65 27.36 33.72 35.86 37.11 28.45 31.63

8.33 14.58 9.38 14.58 9.38 10.42 11.46 16.67 17.71 5.21 12.50

71.34 71.92 71.48 71.42 72.84 73.43 75.82 75.53 72.65 73.56 72.65

65.62 60.42 63.54 64.58 61.46 60.42 70.83 65.62 59.38 61.46 62.50

36.03 35.90 41.91 33.82 32.77 38.66 43.11 41.25 42.91 39.92 44.02

Experiments

We evaluate MAGE using six research questions (RQs). RQ1: Can MAGE improve over memory baselines across 12 datasets? RQ2: Are MAGE’s gains consistent across LLM sizes and model families? RQ3: Does MAGE’s lifecycle machinery keep the shared memory correct, fresh, and transferable under diachronic updates and longrunning operation? RQ4: What are the contributions of MAGE’s architectural components to overall performance? RQ5: Does plugging MAGE as a drop in memory component improve performance across 8 MAS frameworks? RQ6: Can MAGE maintain efficient and consistent system level data management on the live shared graph?

5.1

Experimental Setup

Datasets. We evaluate on 12 QA benchmarks spanning text and multimodal regimes: single-hop QA (TriviaQA[22], NQ Open[23], WebQuestions[5]), multi-hop QA (HotpotQA[56]), financial QA (FinMME[32]), document/figure/chart understanding (DocVQA[36], InfographicVQA[35], ChartQA[34]), and visual question answering (A-OKVQA[44], ScienceQA[30], TextVQA[47], VizWiz[15]).

12.50 11.46 18.75 9.38 8.33 19.79 19.79 17.71 23.96 21.88 25.00

70.12 74.14 72.50 70.32 74.55 71.64 72.43 73.01 77.04 74.49 73.75

58.33 70.83 61.46 57.29 62.50 62.50 59.38 60.42 68.75 70.83 63.54

27.71 37.16 35.59 36.45 31.35 32.03 34.91 38.20 34.64 39.56 34.90

7.29 9.38 15.62 8.33 2.08 3.12 12.50 14.58 7.29 18.75 13.54

65.79 61.14 64.09 64.86 63.65 62.88 60.80 64.29 70.12 64.13 68.95

50.00 52.08 47.92 56.25 48.96 45.83 41.67 55.21 50.00 54.17 52.08

77.59 75.42 77.62 81.16 85.72 82.15 78.67 79.43 81.68 77.56 81.28

52.08 40.62 55.21 51.04 58.33 52.08 55.21 57.29 47.92 53.12 56.25

90.31 86.13 86.63 82.34 81.06 88.87 90.15 84.02 82.61 87.78 92.11

87.50 83.33 76.04 71.88 75.00 84.38 86.46 78.12 70.83 79.17 87.50

35.73 38.15 36.77 37.99 38.59 40.38 37.96 42.29 36.59 35.77 39.27

17.71 19.79 18.75 21.88 19.79 20.83 21.88 22.92 18.75 9.38 20.83

LLM backends. We use three heterogeneous LLMs: a closed lightweight model (GPT-4.1-mini[1]) used as the main driver, a closed frontier model (Claude-4.5-Sonnet[2]), and an open weight LLM (Qwen2.5-VL-32B-Instruct[40]). All backends share the same parameters and policy so that low level backend specific implementation details do not confound the cross LLM comparison in RQ2. Memory baselines. We compare against eleven memory or memory augmented MAS systems spanning five families: vector memory (Mem0[10], A-MEM[54], MemoryBank[61]), reflection based memory (Generative Agents[37]), skill library memory (Voyager[49]), temporal graph memory (Zep / Graphiti[42]), hierarchical MAS memory (G-Memory[59]), and conversation and topology centric MAS memory (MetaGPT[20], ChatDev[38], MacNet[39]). Metrics. Utility level quality is measured by Exact Match (EM) and token level F1, both computed against the gold answer(s) after a SQuAD style normalisation[41]. Datasets whose source documents are indexed in the shared memory graph are reported as IND, and the remaining held out datasets as OOD. The complete protocol is given in Appendix D, and raw per-question predictions are released.

Diachronic Hypergraphs for Orchestrated Multi-Agent Multimodal Memory Curation

Table 2: Component ablation on GPT-4.1-mini. Variants: − Hyperedge (no 𝑛-ary expansion), − Role (no role-aware retrieval), − Decay (no temporal decay), − Consol. (no consolidation), and Retr. only (retrieval-only). Text-only IND Variant

Text-only OOD

Multimodal OOD

NQ-Open

TriviaQA

WebQ.

ChartQA

DocVQA

Infogr.

FinMME

A-OKVQA

SciQA

TextVQA

VizWiz

F1

F1

F1

F1

F1

F1

F1

F1

F1

F1

F1

F1

EM

EM

EM

EM

EM

EM

EM

EM

EM

EM

− Hyperedge − Role − Decay − Consol.

48.24 48.31 47.49 47.01 46.97

38.54 38.54 37.50 36.46 38.54

52.26 53.18 52.30 53.61 50.42

37.50 38.54 38.54 37.50 37.50

67.52 66.27 66.21 66.14 62.98

62.50 60.42 61.46 60.42 57.29

macro Δ = +4.70 pp Base (NoMem)

70

+2.5

+6.4

+4.5

+3.3

50

32.29 31.25 32.29 31.25 31.25

Base (NoMem)

77.54 76.02 78.10 75.90 79.04

72.92 71.88 73.96 70.83 75.00

44.67 46.16 46.10 44.28 43.77

26.04 25.00 25.00 22.92 26.04

80.00 78.93 79.97 79.47 77.95

71.88 68.75 69.79 70.83 69.79

+6.8 +1.6

40 30

+2.1

+4.7

+2.7

75

+1.1

30

91.2 63.4

71.8

D

i

l Se

ne efi f-R

A Re

ct

AD M

C et ev PT N D PE aG at ac et M Ch M

83.13 83.79 80.24 84.33 84.52

56.25 55.21 52.08 58.33 58.33

91.94 88.97 90.56 91.73 91.08

83.33 80.21 83.33 85.42 83.33

38.57 38.09 38.38 37.71 39.48

15.62 14.58 14.58 13.54 15.62

(b) Contradiction res. (%) ↑ 100 75

58.9

50

50

25

25

88.5 58.1

66.3 54.2

0 Mem0

Zep

0 ct re

51.04 48.96 54.17 52.08 56.25

0

10

0

72.80 72.64 74.34 73.60 76.03

20

20 10

19.79 20.83 21.88 21.88 12.50

100

+1.0

40

46.14 46.68 47.11 47.45 36.89

(a) Correction prop. (%) ↑

+ MAGE +5.6

50

+2.0

EM (×100)

+3.0

52.29 51.94 52.30 51.39 50.06

macro Δ = +3.19 pp 60

+ MAGE +6.0

+9.9

60

D

i

ct re l Se

ne efi f-R

A Re

ct

AD M

C et ev PT N D PE aG at ac et M Ch M

Plain graph

MAGE

Mem0

(c) Stale rate (%) ↓ 20 15

Figure 4: MAGE plugged into 8 MAS frameworks: F1 and EM of the pure MAS baseline (blue) versus +MAGE (red).

Zep

Plain graph

MAGE

(d) X-agent transfer (ΔF1) ↑ 3

17.8

2.4

14.2

2

11.5

10

1.1

5

3.1

0

1

0.6

0.5

0 Mem0

Zep

Plain graph

MAGE

Mem0

Zep

Plain graph

MAGE

RQ1: Results with Model Matched Memory

RQ1 tests MAGE against baselines under the same construction and evaluation backend, isolating memory structure and retrieval. Table 1 shows MAGE leads all 12 GPT-4.1-mini datasets, with macro F1 of 66.93 and EM of 51.13, versus 63.57 F1 for the strongest non MAGE baseline per dataset. This paired +3.37 margin compares each dataset’s strongest baseline on exactly the same released evaluation split under identical conditions. Although several baselines are locally competitive, MAGE leads all text only and multimodal datasets. It achieves best F1 and EM on HotpotQA, NQ-Open, TriviaQA, and WebQuestions, demonstrating that the hypergraph preserves multi hop and entity centric evidence, and leads all eight multimodal datasets.

5.3

EM

51.08 41.67 54.70 38.54 68.17 63.54 53.48 34.38 83.71 79.17 47.64 26.04 81.94 72.92 59.22 26.04 79.29 62.50 89.55 63.54 93.65 88.54 40.79 16.67

80

5.2

EM

MAGE-full

Retr. only

F1 (×100)

Multimodal IND

HotpotQA

RQ2: Cross LLM Robustness

RQ2 evaluates whether MAGE remains effective across heterogeneous LLM backends with different architectures and reasoning capabilities. As shown in Table 1, MAGE consistently outperforms the strongest non-MAGE baseline under GPT-4.1-mini, Claude-4.5Sonnet, and Qwen2.5-VL-32B-Instruct, with macro F1 gains of 3.37, 3.44, and 3.37 points, respectively. We note that the GPT-4.1-mini and Qwen2.5-VL-32B macro gains coincide numerically at +3.37; this is a coincidence of paired per question differences rather than shared predictions, since per dataset gains vary from +1.90 to +5.40 points and rank differently across backends (Table 10), and we release all raw predictions for per question verification. This suggests that MAGE is not tied to a specific model family or backend implementation, but reflects better organization and retrieval of external memory itself.

Figure 5: Diachronic probes: MAGE (red) versus baselines on (a) correction propagation, (b) contradiction resolution, (c) stale rate (lower is better), and (d) cross agent transfer.

5.4

RQ3: Diachronic Agent Dynamics

RQ3 asks whether shared memory remains correct, fresh, and transferable as evidence changes over time. Static QA cannot isolate these properties, so Figure 5 evaluates four controlled diachronic probes over a chronological event replay (protocol in Appendix G). MAGE reaches 91.2% correction propagation, 19.4 points above the strongest baseline, showing that later corrections replace superseded evidence in downstream answers. Contradiction resolution reaches 88.5%, a 22.2 point lead, while the stale retrieval rate falls to 3.1% from the best baseline’s 11.5%, a 73.0% relative reduction. These two results show that explicit validity intervals, contradiction links, and lifecycle filtering prevent outdated facts from remaining equally retrievable after an update. The fourth probe tests whether memory written by one agent improves a different agent’s later task. MAGE yields +2.4 F1 cross agent transfer, more than twice the strongest baseline’s +1.1, indicating that role conditioned retrieval preserves reusable evidence without confining it to the originating agent. Together, the four probes support the claim that MAGE does more than improve static retrieval: it propagates revisions, resolves conflicts, suppresses stale state, and transfers experience across agent boundaries. The extended continuous replay in Appendix I further shows stable performance over ten episodes rather than a one time response to injected contradictions.

Feng et al.

Table 3: MAS pluggability results on GPT-4.1-mini. Text-only IND

+MAGE Self-Refine

+MAGE ReAct

+MAGE MAD

+MAGE PEC

+MAGE MetaGPT

+MAGE ChatDev

+MAGE MacNet

+MAGE

NQ-Open

TriviaQA

WebQ.

ChartQA

DocVQA

Infogr.

FinMME

A-OKVQA

SciQA

TextVQA

VizWiz

F1

F1

F1

F1

F1

F1

F1

F1

F1

F1

F1

EM

EM

p50

Latency (ms, log)

EM

EM

EM

p95

102

p99

66.69

5.25

6.88

100 0.18

0.25

10−1 D-I

T-I

Upd Qry Del

(b) Throughput and deduplication

300

299.9

30 256.9 19.8

200

20

100

10

0 Trace/s

Dedup.

RQ4: Component Ablation

RQ4 examines the operational contribution of MAGE’s architecture. Table 2 shows that disabling hyperedge expansion, role aware retrieval, temporal decay, consolidation, or the combined write and maintenance path lowers performance relative to full MAGE. The per dataset losses are task dependent: higher order structure and lifecycle operations are especially important on FinMME, while role policy, decay, and consolidation make distinct contributions across multimodal tasks. A matched pairwise-projection control further isolates the schema: with nodes, embeddings, ranking, role policy, and token budget fixed, clique projection lowers macro F1 from 66.93 to 65.72 (Appendix E). The retrieval only variant trails full MAGE on every dataset, showing that structured ingestion and ongoing maintenance cannot be replaced by a stronger read path alone. Appendix F confirms that these gains are robust to moderate changes in role weights and retrieval depth.

5.6

EM

EM

EM

EM

EM

EM

debate, planning, and role structured execution without altering native coordination. Direct gains 9.88 F1 points, showing that structured retrieval benefits even simple pipelines. MetaGPT gains 6.05 points, indicating that role structured frameworks can exploit prior traces and evidence links across heterogeneous execution. These consistent improvements demonstrate that MAGE complements orchestration strategies.

0 Doc/s

Figure 6: System performance. (a) CRUD and query latency; D-I and T-I denote document and trace insertion. (b) Insertion throughput and deduplication ratio.

5.5

EM

37.30 27.08 44.41 22.92 55.91 54.17 38.66 16.67 73.32 67.71 33.86 15.62 71.29 58.33 46.18 8.33 72.48 53.12 79.87 57.29 86.15 81.25 30.23 7.29 50.09 38.54 53.09 33.33 67.02 57.29 52.06 29.17 82.76 72.92 46.04 25.00 80.50 67.71 56.09 12.50 78.49 56.25 88.92 58.33 92.99 85.42 40.12 14.58 33.27 22.92 37.40 21.88 47.04 36.46 35.80 19.79 57.32 45.83 31.26 14.58 55.43 41.67 38.95 14.58 54.46 36.46 61.06 36.46 63.85 52.08 26.18 8.33 37.02 23.96 39.63 22.92 49.40 37.50 38.75 20.83 60.65 46.88 34.52 15.62 59.37 42.71 42.91 15.62 57.45 37.50 64.89 37.50 67.86 53.12 29.56 9.38 39.70 30.21 42.75 28.12 54.25 46.88 41.56 25.00 67.45 58.33 38.31 18.75 65.99 54.17 47.32 18.75 64.04 45.83 72.72 46.88 75.65 65.62 32.57 11.46 43.12 31.25 46.18 29.17 57.55 47.92 45.15 26.04 70.67 59.38 40.22 19.79 69.18 55.21 49.99 19.79 66.94 46.88 75.61 47.92 79.06 66.67 34.44 12.50 41.42 32.29 44.51 31.25 55.16 48.96 42.69 27.08 68.46 64.58 38.48 20.83 67.28 59.38 48.58 20.83 65.17 51.04 74.69 52.08 77.69 73.96 32.07 10.42 43.62 34.38 46.70 32.29 58.21 53.12 45.66 29.17 71.47 65.62 40.67 21.88 69.96 60.42 50.56 21.88 67.70 52.08 76.46 53.12 79.96 75.00 34.83 13.54 42.16 32.29 45.45 33.33 57.00 55.21 44.60 27.08 71.50 66.67 38.82 21.88 69.89 62.50 49.13 20.83 67.24 52.08 76.36 55.21 79.99 72.92 33.77 13.54 44.49 36.46 47.64 34.38 59.38 56.25 46.58 30.21 72.91 69.79 41.49 22.92 71.37 63.54 51.58 22.92 69.06 55.21 78.00 56.25 81.57 76.04 35.53 14.58 47.50 35.42 48.94 30.21 55.47 53.12 49.58 28.12 76.33 71.88 41.77 18.75 78.74 66.67 52.37 21.88 76.89 56.25 81.05 59.38 85.18 73.96 28.86 7.29 50.51 40.62 54.04 36.46 67.23 60.42 53.02 32.29 82.94 76.04 47.07 25.00 81.49 72.92 58.48 23.96 78.83 60.42 89.07 61.46 92.92 84.38 39.65 15.62 40.82 33.33 50.46 34.38 60.09 54.17 39.40 18.75 77.99 69.79 33.82 9.38 77.05 69.79 54.32 19.79 71.54 53.12 79.95 47.92 90.15 82.29 30.30 6.25 49.88 35.42 53.23 36.46 66.76 58.33 50.82 29.17 81.62 70.83 46.22 23.96 80.60 70.83 58.00 22.92 76.82 54.17 87.85 57.29 92.09 83.33 38.87 12.50 43.02 33.33 47.07 30.21 58.89 51.04 41.91 23.96 75.78 63.54 44.06 22.92 75.31 65.62 50.30 23.96 73.44 54.17 83.82 58.33 90.50 83.33 34.70 10.42 47.88 34.38 51.78 35.42 65.53 56.25 51.04 28.12 80.72 71.88 44.92 23.96 79.89 67.71 57.20 25.00 76.90 55.21 87.91 59.38 91.30 84.38 38.20 11.46

(a) CRUD and query latency

101

Multimodal OOD

F1

Dedup. (%)

Direct

Multimodal IND

HotpotQA

Insertion (items/s)

Framework

Text-only OOD

RQ5: Plug in Memory Layer Performance

RQ5 examines MAGE as a general external memory layer for MAS. Table 3 and Figure 4 show gains across all eight frameworks, averaging 4.70 macro F1 and 3.19 macro EM. MAGE therefore supports single pass inference, iterative refinement, action based reasoning,

5.7

RQ6: System Level Performance

RQ6 evaluates MAGE as an online memory engine through efficiency and maintenance quality. Memory must support writes, updates, deletions, retrieval, and maintenance, covering access, growth, and maintenance pressure under multi agent workloads. Figure 6(a) reports MAGE’s 𝑝 99 latencies: 5.25 ms for document insertion, 6.88 ms for trace insertion, 0.18 ms for update, 0.25 ms for delete, and 66.69 ms for query. MAGE maintains and retrieves hypergraph memory with limited overhead despite semantic search, role aware expansion and scoring. Figure 6(b) reports 299.9 documents/s and 256.9 traces/s, while consolidation removes 19.8% duplicated items. These results demonstrate that MAGE sustains online operation. Graph statistics, hardware, scaling, and latency comparisons against Mem0 and Zep appear in Appendix H.

6

Conclusion

We presented MAGE, an agent graph memory engine for persistent MAS. It represents experience as a heterogeneous hypergraph connecting agents, roles, evidence, entities, modalities, procedures, and traces. Role aware retrieval, temporal lifecycle management, consolidation, and budgeted packing support reasoning and data management. Across twelve text and multimodal benchmarks, MAGE outperforms memory baselines, remains robust across LLM backends, and benefits from each tested architectural path. It integrates across MAS frameworks with efficient writes, ingestion, deduplication, and retrieval. This establishes structured hypergraph memory as a basis for persistent MAS that reuse evidence, decisions, and procedural experience without modifying model parameters.

Diachronic Hypergraphs for Orchestrated Multi-Agent Multimodal Memory Curation

Acknowledgment The authors used AI-assisted tools to improve grammar and language clarity and to help debug implementation code. All AI assisted outputs were reviewed, edited, and verified by the authors, who take full responsibility for the content and originality of this work.

References [1] Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774 (2023). [2] Anthropic. 2025. Claude Sonnet. https://www.anthropic.com/claude/sonnet. Accessed: 2026-06-10. [3] TING BAI, JIAYANG FAN, XIAOSHUAI WEN, JIAZHENG KANG, HENGZHI LAN, RUOCHEN ZHAO, PINGZHENG WU, ZEPENG ZHANG, YUTIAN ZHONG, GEZI LI, et al. 2025. Survey on AI Memory: Theories, Taxonomies, Evaluations, and Emerging Trends. (2025). [4] Parasumanna Gokulan Balaji and Dipti Srinivasan. 2010. An introduction to multi-agent systems. In Innovations in multi-agent systems and applications-1. Springer, 1–27. [5] Jonathan Berant, Andrew Chou, Roy Frostig, and Percy Liang. 2013. Semantic parsing on freebase from question-answer pairs. In Proceedings of the 2013 conference on empirical methods in natural language processing. 1533–1544. [6] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems 33 (2020), 1877–1901. [7] Boyu Chen, Zirui Guo, Zidan Yang, Yuluo Chen, Junze Chen, Zhenghao Liu, Chuan Shi, and Cheng Yang. 2026. Pathrag: Pruning graph-based retrieval augmented generation with relational paths. In Proceedings of the AAAI conference on artificial intelligence, Vol. 40. 30183–30191. [8] Fei Chen and Wei Ren. 2019. On the control of multi-agent systems: A survey. Foundations and Trends in System and Control 6, 4 (2019), 339–499. [9] Zihan Chen, Lei Zheng, and Di Zhu. 2026. A Survey of Agentic GraphRAG: From Retrieval-Augmented Generation to Graph-Native Agents. Available at SSRN 6713979 (2026). [10] Prateek Chhikara, Dev Khant, Saket Aryan, Taranjeet Singh, and Deshraj Yadav. 2025. Mem0: Building production-ready ai agents with scalable long-term memory. arXiv preprint arXiv:2504.19413 (2025). [11] Ali Dorri, Salil S Kanhere, and Raja Jurdak. 2018. Multi-agent systems: A survey. Ieee Access 6 (2018), 28573–28593. [12] Darren Edge, Ha Trinh, Newman Cheng, Joshua Bradley, Alex Chao, Apurva Mody, Steven Truitt, Dasha Metropolitansky, Robert Osazuwa Ness, and Jonathan Larson. 2024. From local to global: A graph rag approach to query-focused summarization. arXiv preprint arXiv:2404.16130 (2024). [13] Jacques Ferber and Gerhard Weiss. 1999. Multi-agent systems: an introduction to distributed artificial intelligence. Vol. 1. Addison-wesley Reading. [14] Zirui Guo, Lianghao Xia, Yanhua Yu, Tian Ao, and Chao Huang. 2024. Lightrag: Simple and fast retrieval-augmented generation. arXiv preprint arXiv:2410.05779 2, 3 (2024). [15] Danna Gurari, Qing Li, Abigale J Stangl, Anhong Guo, Chi Lin, Kristen Grauman, Jiebo Luo, and Jeffrey P Bigham. 2018. Vizwiz grand challenge: Answering visual questions from blind people. In Proceedings of the IEEE conference on computer vision and pattern recognition. 3608–3617. [16] Bernal J Gutiérrez, Yiheng Shu, Yu Gu, Michihiro Yasunaga, and Yu Su. 2024. Hipporag: Neurobiologically inspired long-term memory for large language models. Advances in neural information processing systems 37 (2024), 59532– 59569. [17] Shanshan Han, Qifan Zhang, Weizhao Jin, and Zhaozhuo Xu. 2024. LLM multiagent systems: Challenges and open problems. arXiv preprint arXiv:2402.03578 (2024). [18] Junda He, Christoph Treude, and David Lo. 2025. Llm-based multi-agent systems for software engineering: Literature review, vision, and the road ahead. ACM Transactions on Software Engineering and Methodology 34, 5 (2025), 1–30. [19] Xiaoxin He, Yijun Tian, Yifei Sun, Nitesh V. Chawla, Thomas Laurent, Yann LeCun, Xavier Bresson, and Bryan Hooi. 2024. G-Retriever: Retrieval-Augmented Generation for Textual Graph Understanding and Question Answering. In Advances in Neural Information Processing Systems, A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang (Eds.), Vol. 37. Curran Associates, Inc., 132876–132907. https://doi.org/10.52202/079017-4224 [20] Sirui Hong, Mingchen Zhuge, Jonathan Chen, Xiawu Zheng, Yuheng Cheng, Jinlin Wang, Ceyao Zhang, Steven Yau, Zijuan Lin, Liyang Zhou, et al. 2024. MetaGPT: Meta programming for a multi-agent collaborative framework. In International Conference on Learning Representations, Vol. 2024. 23247–23275.

[21] Wei-Chieh Huang, Weizhi Zhang, Yueqing Liang, Yuanchen Bei, Yankai Chen, Tao Feng, Xinyu Pan, Zhen Tan, Yu Wang, Tianxin Wei, et al. 2026. Rethinking Memory Mechanisms of Foundation Agents in the Second Half: A Survey. arXiv preprint arXiv:2602.06052 (2026). [22] Mandar Joshi, Eunsol Choi, Daniel S Weld, and Luke Zettlemoyer. 2017. Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 1601–1611. [23] Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, et al. 2019. Natural questions: a benchmark for question answering research. Transactions of the Association for Computational Linguistics 7 (2019), 453–466. [24] Meng-Chieh Lee, Qi Zhu, Costas Mavromatis, Zhen Han, Soji Adeshina, Vassilis N. Ioannidis, Huzefa Rangwala, and Christos Faloutsos. 2025. HybGRAG: Hybrid Retrieval-Augmented Generation on Textual and Relational Knowledge Bases. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar (Eds.). Association for Computational Linguistics, Vienna, Austria, 879–893. https://doi.org/10.18653/v1/2025.acllong.43 [25] Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. 2020. Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in neural information processing systems 33 (2020), 9459–9474. [26] Chen-An Li and Hung-Yi Lee. 2024. Examining forgetting in continual pretraining of aligned large language models. arXiv preprint arXiv:2401.03129 (2024). [27] Guohao Li, Hasan Hammoud, Hani Itani, Dmitrii Khizbullin, and Bernard Ghanem. 2023. Camel: Communicative agents for" mind" exploration of large language model society. Advances in neural information processing systems 36 (2023), 51991–52008. [28] Xinyi Li, Sai Wang, Siqi Zeng, Yu Wu, and Yi Yang. 2024. A survey on LLM-based multi-agent systems: workflow, infrastructure, and challenges. Vicinagearth 1, 1 (2024), 9. [29] Wen-Sheng Lien, Yu-Kai Chan, Hao-Lung Hsiao, Bo-Kai Ruan, Meng-Fen Chiang, Chien-An Chen, Yi-Ren Yeh, and Hong-Han Shuai. 2026. HyperRAG: Reasoning N-ary Facts over Hypergraphs for Retrieval Augmented Generation. In Proceedings of the ACM Web Conference 2026. 2465–2476. [30] Pan Lu, Swaroop Mishra, Tanglin Xia, Liang Qiu, Kai-Wei Chang, Song-Chun Zhu, Oyvind Tafjord, Peter Clark, and Ashwin Kalyan. 2022. Learn to explain: Multimodal reasoning via thought chains for science question answering. Advances in neural information processing systems 35 (2022), 2507–2521. [31] Haoran Luo, Guanting Chen, Yandan Zheng, Xiaobao Wu, Yikai Guo, Qika Lin, Yu Feng, Zemin Kuang, Meina Song, Yifan Zhu, et al. 2026. Hypergraphrag: Retrievalaugmented generation via hypergraph-structured knowledge representation. Advances in Neural Information Processing Systems 38 (2026), 152206–152234. [32] Junyu Luo, Zhizhuo Kou, Liming Yang, Xiao Luo, Jinsheng Huang, Zhiping Xiao, Jingshu Peng, Chengzhong Liu, Jiaming Ji, Xuanzhe Liu, et al. 2025. Finmme: Benchmark dataset for financial multi-modal reasoning evaluation. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 29465–29489. [33] Yun Luo, Zhen Yang, Fandong Meng, Yafu Li, Jie Zhou, and Yue Zhang. 2025. An empirical study of catastrophic forgetting in large language models during continual fine-tuning. IEEE Transactions on Audio, Speech and Language Processing (2025). [34] Ahmed Masry, Do Xuan Long, Jia Qing Tan, Shafiq Joty, and Enamul Hoque. 2022. ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning. In Findings of the Association for Computational Linguistics: ACL 2022, Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (Eds.). Association for Computational Linguistics, Dublin, Ireland, 2263–2279. https: //doi.org/10.18653/v1/2022.findings-acl.177 [35] Minesh Mathew, Viraj Bagal, Rubèn Tito, Dimosthenis Karatzas, Ernest Valveny, and CV Jawahar. 2022. Infographicvqa. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. 1697–1706. [36] Minesh Mathew, Dimosthenis Karatzas, and CV Jawahar. 2021. Docvqa: A dataset for vqa on document images. In Proceedings of the IEEE/CVF winter conference on applications of computer vision. 2200–2209. [37] Joon Sung Park, Joseph O’Brien, Carrie Jun Cai, Meredith Ringel Morris, Percy Liang, and Michael S Bernstein. 2023. Generative agents: Interactive simulacra of human behavior. In Proceedings of the 36th annual acm symposium on user interface software and technology. 1–22. [38] Chen Qian, Wei Liu, Hongzhang Liu, Nuo Chen, Yufan Dang, Jiahao Li, Cheng Yang, Weize Chen, Yusheng Su, Xin Cong, et al. 2024. Chatdev: Communicative agents for software development. In Proceedings of the 62nd annual meeting of the association for computational linguistics (volume 1: Long papers). 15174–15186. [39] Chen Qian, Zihao Xie, Yifei Wang, Wei Liu, Kunlun Zhu, Hanchen Xia, Yufan Dang, Zhuoyun Du, Weize Chen, Cheng Yang, et al. 2025. Scaling large language model-based multi-agent collaboration. In International Conference on Learning Representations, Vol. 2025. 41488–41505.

Feng et al.

[40] Qwen, :, An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei Zhang, Qin Zhu, Rui Men, Runji Lin, Tianhao Li, Tianyi Tang, Tingyu Xia, Xingzhang Ren, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yu Wan, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, and Zihan Qiu. 2025. Qwen2.5 Technical Report. arXiv:2412.15115 [cs.CL] https://arxiv.org/abs/2412.15115 [41] Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. Squad: 100,000+ questions for machine comprehension of text. In Proceedings of the 2016 conference on empirical methods in natural language processing. 2383–2392. [42] Preston Rasmussen, Pavlo Paliychuk, Travis Beauvais, Jack Ryan, and Daniel Chalef. 2025. Zep: a temporal knowledge graph architecture for agent memory. arXiv preprint arXiv:2501.13956 (2025). [43] Parth Sarthi, Salman Abdullah, Aditi Tuli, Shubh Khanna, Anna Goldie, and Christopher Manning. 2024. Raptor: Recursive abstractive processing for treeorganized retrieval. In International Conference on Learning Representations, Vol. 2024. 32628–32649. [44] Dustin Schwenk, Apoorv Khandelwal, Christopher Clark, Kenneth Marino, and Roozbeh Mottaghi. 2022. A-okvqa: A benchmark for visual question answering using world knowledge. In European conference on computer vision. Springer, 146–162. [45] Yu Shang, Yu Li, Keyu Zhao, Likai Ma, Jiahe Liu, Fengli Xu, and Yong Li. 2025. Agentsquare: Automatic llm agent search in modular design space. In International Conference on Learning Representations, Vol. 2025. 3841–3865. [46] Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. 2023. Reflexion: Language agents with verbal reinforcement learning. Advances in neural information processing systems 36 (2023), 8634–8652. [47] Amanpreet Singh, Vivek Natarajan, Meet Shah, Yu Jiang, Xinlei Chen, Dhruv Batra, Devi Parikh, and Marcus Rohrbach. 2019. Towards vqa models that can read. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 8317–8326. [48] Yashar Talebirad and Amirhossein Nadiri. 2023. Multi-agent collaboration: Harnessing the power of intelligent llm agents. arXiv preprint arXiv:2306.03314 (2023). [49] Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. 2023. Voyager: An open-ended embodied agent with large language models. arXiv preprint arXiv:2305.16291 (2023). [50] Qian Wang, Tianyu Wang, Zhenheng Tang, Qinbin Li, Nuo Chen, Jingsheng Liang, and Bingsheng He. 2025. MegaAgent: A Large-Scale Autonomous LLMbased Multi-Agent System Without Predefined SOPs. In Findings of the Association for Computational Linguistics: ACL 2025, Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar (Eds.). Association for Computational Linguistics, Vienna, Austria, 4998–5036. https://doi.org/10.18653/v1/ 2025.findings-acl.259 [51] Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Beibin Li, Erkang Zhu, Li Jiang, Xiaoyun Zhang, Shaokun Zhang, Jiale Liu, et al. 2024. Autogen: Enabling next-gen LLM applications via multi-agent conversations. In First conference on language modeling. [52] Shanglin Wu and Kai Shu. 2025. Memory in llm-based multi-agent systems: Mechanisms, challenges, and collective intelligence. Authorea Preprints (2025). [53] Tongtong Wu, Linhao Luo, Yuan-Fang Li, Shirui Pan, Thuy-Trang Vu, and Gholamreza Haffari. 2024. Continual learning for large language models: A survey. arXiv preprint arXiv:2402.01364 (2024). [54] Wujiang Xu, Zujie Liang, Kai Mei, Hang Gao, Juntao Tan, and Yongfeng Zhang. 2026. A-mem: Agentic memory for llm agents. Advances in Neural Information Processing Systems 38 (2026), 17577–17604. [55] Chang Yang, Chuang Zhou, Yilin Xiao, Su Dong, Luyao Zhuang, Yujing Zhang, Zhu Wang, Zijin Hong, Zheng Yuan, Zhishang Xiang, et al. 2026. Graphbased Agent Memory: Taxonomy, Techniques, and Applications. arXiv preprint arXiv:2602.05665 (2026). [56] Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christopher D Manning. 2018. HotpotQA: A dataset for diverse, explainable multi-hop question answering. In Proceedings of the 2018 conference on empirical methods in natural language processing. 2369–2380. [57] Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. 2023. Tree of thoughts: Deliberate problem solving with large language models. Advances in neural information processing systems 36 (2023), 11809–11822. [58] Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2022. React: Synergizing reasoning and acting in language models. arXiv preprint arXiv:2210.03629 (2022). [59] Guibin Zhang, Muxin Fu, Kun Wang, Frank Wan, Miao Yu, and Shuicheng Yan. 2026. G-memory: Tracing hierarchical memory for multi-agent systems. Advances in Neural Information Processing Systems 38 (2026), 12988–13018. [60] Junhao Zheng, Chengming Shi, Xidi Cai, Qiuke Li, Duzhen Zhang, Chenxing Li, Dong Yu, and Qianli Ma. 2026. Lifelong learning of large language model based agents: A roadmap. IEEE Transactions on Pattern Analysis and Machine

Intelligence (2026). [61] Wanjun Zhong, Lianghong Guo, Qiqi Gao, He Ye, and Yanlin Wang. 2024. Memorybank: Enhancing large language models with long-term memory. In Proceedings of the AAAI conference on artificial intelligence, Vol. 38. 19724–19731. [62] Xiangrong Zhu, Yuexiang Xie, Yi Liu, Yaliang Li, and Wei Hu. 2025. Knowledge graph-guided retrieval augmented generation. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 8912–8924.

Diachronic Hypergraphs for Orchestrated Multi-Agent Multimodal Memory Curation

A

Additional Implementation Details

This appendix provides additional details about the released MAGE implementation and qualitative examples. The goal is to make explicit how the proposed memory layer is instantiated in code and why the retrieved MAGE context improves downstream multi-agent reasoning. The implementation follows the same conceptual decomposition used in the main paper: heterogeneous memory schema, high-order hyperedge construction, role-aware retrieval, bidirectional expansion, lifecycle maintenance, and budget-bounded context packing.

A.1

Code Organization

The released code is organized as a pip-installable Python library centered around the MageMemory facade. Table 4 summarizes the correspondence between the paper components and implementation files. The library exposes a single high-level interface for inserting raw documents, multimodal items, and MAS trajectories, while keeping schema validation, indexing, storage, retrieval, and maintenance modular.

A.2

Heterogeneous Node and Hyperedge Schema

MAGE represents memory as a heterogeneous temporal hypergraph. The code defines a common BaseNode with a unique identifier, node type, version, status, optional embedding, attributes, and bi-temporal fields. The bi-temporal fields include valid_from, valid_to, tx_from, and tx_to. This design separates when a fact is believed true in the task world from when the record is alive in the memory system. As a result, MAGE can support logical updates and deletions without losing auditability. The concrete node types cover both conventional knowledge and MAS-specific execution traces. TaskNode records the task text, domain, modality, score, and task status. AgentNode records role, model, capabilities, and memory policy. MessageNode, ActionNode, and ErrorNode preserve the interaction trajectory. MemoryNode stores distilled episodic, semantic, procedural, reflective, or multimodal memories together with source nodes, confidence, usage count, contradiction count, and downstream gain. InsightNode, ProcedureNode, CommunityNode, ArchetypeNode, and MetaProcedureNode provide increasingly abstract reusable units. Hyperedges are first-class objects rather than derived views. Each hyperedge connects at least two nodes and carries its own type, description, confidence, source nodes, embedding, timestamp, lifecycle status, version, and bi-temporal fields. MAGE uses several hyperedge types: NaryFactHyperedge for high-order factual relations, CollaborationHyperedge for multi-agent collaboration events, ErrorCorrectionHyperedge for error and correction patterns, EvidenceHyperedge for query evidence, ProcedureHyperedge for executable procedures, CommunityReportHyperedge for community summaries, and GeneralizesHyperedge for abstraction over multiple memories. This schema is important because many useful reasoning events are not dyadic. For example, an answer may jointly depend on an image region, an OCR string, a chart axis, a prior answer, and an agent role. A binary graph must decompose this into many pairwise edges, while MAGE can store the event as one retrievable high-order unit.

Algorithm 1 MAGE insertion for a MAS trace Input: Task 𝑇 , agents 𝐴, ordered trajectory 𝜏, result 𝑅 Output: Typed nodes, binary edges, hyperedges, distilled memories, procedures 1: Create a TaskNode from 𝑇 and 𝑅 2: for each agent 𝑎 ∈ 𝐴 do 3: Create an AgentNode with role, model, and capabilities 4: end for 5: for each step 𝑠𝑖 ∈ 𝜏 do 6: if 𝑠𝑖 is a message then 7: Create a MessageNode; add CONTAINS and PRODUCES edges 8: else if 𝑠𝑖 is an action then 9: Create an ActionNode; add CONTAINS, EXECUTES, and CAUSES edges 10: else if 𝑠𝑖 is an error then 11: Create an ErrorNode; link it to the task and preceding step 12: end if 13: end for 14: Create a CollaborationHyperedge over task, agents, messages, and actions 15: Create ErrorCorrectionHyperedges when errors and corrective actions are observed 16: Distill reusable MemoryNodes and ProcedureNodes from the trace 17: Persist all objects and update vector, adjacency, temporal, role, and lifecycle indices

A.3

Insertion Pipeline

MAGE provides three insertion paths. First, insert_document converts text into a source-of-record modality node, a semantic memory node, extracted entity nodes, binary provenance edges, and n-ary fact hyperedges. Second, insert_multimodal_item attaches images, tables, PDFs, webpages, or other modalities to extracted text and stores their evidence as multimodal memory. Third, insert_mas_trace turns a complete multi-agent execution trace into a task node, agent nodes, message/action/error nodes, causal and production edges, collaboration hyperedges, error-correction hyperedges, reusable memory nodes, and procedure nodes. The MAS trace ingestor is especially important for the results in this paper. It takes four logical inputs: a task dictionary, a list of agent dictionaries, an ordered trajectory, and a result dictionary. The trajectory may contain messages, actions, and errors. For each message, MAGE adds a CONTAINS edge from the task and a PRODUCES edge from the agent. For each action, MAGE adds CONTAINS, EXECUTES, and temporal-causal CAUSES edges. For errors, MAGE records severity, detector, correction status, and links the error to the preceding step. After processing the trajectory, it creates a CollaborationHyperedge connecting the task, participating agents, messages, and actions. If an error is followed by a corrective action, it also creates an ErrorCorrectionHyperedge. This converts a transient MAS run into persistent reusable experience.

Feng et al.

Table 4: Implementation map of the released MAGE codebase. Component

Key files

Functionality

Public API

mage_memory/api/memory_graph.py

Schema

mage_memory/schema/nodes.py, edges.py, hyperedges.py, enums.py mage_memory/storage/jsonl_store.py, nx_store.py

Implements MageMemory, the user-facing facade for inserting documents, multimodal items, MAS traces, querying role-specific context, retrieving subgraphs, consolidation, CRUD operations, validation, and statistics. Defines typed nodes, binary edges, first-class hyperedges, lifecycle states, memory types, modalities, agent roles, and update decisions. Provides JSONL-backed persistence and a NetworkX-backed store. The default build creates nodes.jsonl, edges.jsonl, hyperedges.jsonl, and events.jsonl. Maintains vector scopes, adjacency relations, temporal/lifecycle indices, and rolespecific retrieval policies. Converts raw text, image/table evidence, and MAS trajectories into typed nodes, binary edges, and hyperedges. Produces entities, n-ary facts, reusable memories, procedures, and higher-level insights. The code supports deterministic fallback and optional LLM-driven extraction.

Storage

MAS adapter

vector_index.py, indexing/hybrid_index.py, adjacency_index.py, role_policy_index.py ingestion/text_ingestor.py, multimodal_ingestor.py, mas_trace_ingestor.py extraction/entity_extractor.py, hyperedge_extractor.py, memory_extractor.py, procedure_extractor.py, insight_extractor.py retrieval/hypergraph_retriever.py, role_retriever.py, bidirectional_retriever.py, ranker.py, subgraph_packer.py lifecycle/update_decision.py, dedup.py, merge.py, confidence_decay.py, communities.py, garbage_collector.py mas/agent_memory_adapter.py

Evaluation scripts

scripts/build_shared_graph.py, scripts/eval_mage.py

Indexing Ingestion Extraction

Retrieval

Lifecycle

A.4

Role-Aware Hypergraph Retrieval

At query time, MAGE first retrieves semantic seeds from multiple vector scopes, including memory, insight, entity, task, modality, procedure, and hyperedge scopes. If the query is issued for a specific role, the role policy index restricts the allowed node and hyperedge types and applies role-dependent boosts. For example, the Planner policy prefers insights, procedures, and collaboration hyperedges; the Executor policy prefers procedures, actions, and evidence; the Critic policy can retrieve error nodes and error-correction hyperedges; the Retriever policy emphasizes modalities, evidence hyperedges, and entities. This prevents every agent from receiving the same generic context and instead delivers role-compatible evidence. After seed retrieval, MAGE performs bidirectional expansion. The expansion follows both binary edges and hypergraph membership: entity to hyperedge to entity, task to memory to insight, and task to interaction to error/correction. This step is crucial because the top semantic match is often only one part of the needed evidence. For example, a chart question may first match a chart modality node; expansion can then recover the corresponding extracted values, entities, n-ary fact hyperedges, and prior reasoning trace. Similarly, a multi-hop text question may match one entity, but expansion recovers the memory and relation that connect it to the final answer. Candidates are then ranked by a multi-factor scoring function. The implementation combines semantic similarity, role weight, confidence, provenance support, recency, usage, lifecycle penalty, and token-cost penalty. Finally, the subgraph packer greedily selects the highest-ranked nodes and hyperedges under a token budget. It renders selected items as typed text chunks and returns the selected nodes, binary edges among selected nodes, selected hyperedges, provenance, diagnostics, and token usage. In read-only evaluation mode, queries do not mutate the graph, so scoring one test question cannot leak information into another.

Implements hybrid hypergraph search, role filtering and boosting, graph/hyperedge expansion, multi-factor ranking, and budget-aware packing. Supports decision-driven writes, deduplication, merge, confidence decay, community/archetype consolidation, and garbage collection. Provides before_act, after_act, record_error, and finalize hooks for attaching MAGE to existing agent pipelines. Build the shared graph from training data and evaluate read-only MAGE retrieval with normalized EM and token-F1.

Algorithm 2 Role-aware MAGE retrieval and context packing Input: Query 𝑞, optional role 𝑟 , hop limit ℎ, token budget 𝐵, top-𝑘 Output: Packed textual context and retrieved subgraph 1: if 𝑟 is provided then 2: Retrieve role-filtered seeds using the role policy for 𝑟 3: else 4: Retrieve seeds from memory, insight, entity, task, modality, procedure, and hyperedge scopes 5: end if 6: Expand seed nodes by binary adjacency and hyperedge membership for at most ℎ hops 7: Expand seed hyperedges to their member nodes 8: Remove candidates disallowed by the role policy or lifecycle status 9: Score candidates using semantic similarity, role weight, confidence, provenance, recency, usage, lifecycle penalty, and token cost 10: Greedily pack ranked candidates while total estimated tokens do not exceed 𝐵 11: Return text_context, selected nodes, selected edges, selected hyperedges, provenance, diagnostics, and token count

A.5

Lifecycle Maintenance and Decision-Driven Updates

A memory layer for long-running MAS must not simply append everything forever. MAGE includes decision-driven updates inspired by production memory systems. When a new fact arrives, the update decision module compares it against similar live memories and chooses one of four actions: ADD, UPDATE, DELETE, or NOOP. ADD inserts orthogonal information. UPDATE closes the transaction window of the old memory and appends a new version through prev_version_id and a SUPERSEDES edge. DELETE soft-deletes a contradictory memory by setting tx_to and adding an INVALIDATES edge. NOOP suppresses near duplicates. The

Diachronic Hypergraphs for Orchestrated Multi-Agent Multimodal Memory Curation

default implementation is deterministic and uses cosine similarity plus a polarity-flip heuristic; an LLM decision maker can be used when a chat client is configured, and it falls back to the heuristic on malformed output. The maintenance pass combines deduplication, merging, insight extraction, community summarization, and schema consolidation. This matters for performance because retrieval quality depends not only on adding memories, but also on keeping the memory graph compact, current, and semantically organized. The ablation in the main paper shows that removing hyperedges, role policies, decay, or consolidation degrades results. The implementation therefore treats memory management as part of the reasoning system rather than as offline bookkeeping.

B

Why MAGE Improves Accuracy

The empirical gains from adding MAGE can be explained by several concrete mechanisms. High-order evidence preservation. Many benchmark questions require combining multiple pieces of evidence. A chart question may require reading several bars and applying arithmetic; a document question may require linking a label to a definition; a visual question may require connecting OCR, object recognition, and commonsense. Vector memory stores isolated chunks, and ordinary graph memory decomposes high-order events into pairwise links. MAGE preserves the joint event as a hyperedge, so retrieval can bring back the whole evidence bundle instead of a single fragment. Role-compatible context. Different MAS roles need different evidence. A planner benefits from abstract procedures and prior collaboration patterns. An executor benefits from actions and evidence. A critic benefits from errors and corrections. MAGE’s role policy index explicitly controls which node and hyperedge types are visible to each role and assigns boosts to role-relevant types. This reduces irrelevant context and makes the retrieved context easier for the downstream LLM to use. Bidirectional expansion beyond surface similarity. The first semantic match is often incomplete. MAGE expands from seeds through binary edges and hyperedge membership, allowing the system to recover adjacent entities, modalities, memories, procedures, and evidence relations. This helps when the query wording differs from stored memory wording, or when the answer is connected through an intermediate entity or modality. Budget-bounded context construction. A larger memory is not always better because the LLM context window is limited. MAGE ranks candidates using semantic, structural, role, confidence, provenance, temporal, lifecycle, and token-cost signals, then packs a compact subgraph under a token budget. This avoids two common failure modes: retrieving too little evidence and retrieving too much redundant evidence. Persistent reuse of successful traces. MAS traces contain useful procedural knowledge: which agents participated, which actions were useful, which errors occurred, and which corrections worked. MAGE converts these traces into reusable memories and procedures. Therefore, adding MAGE helps not only by retrieving facts, but also by retrieving prior task-solving patterns.

Safe evaluation and leakage control. The evaluation script loads the graph in read-only mode by default. Queries do not update usage_count or write event logs during scoring, which prevents test questions from changing the memory graph. The build script also includes a leakage shield against raw test pools. Thus, the observed gains are attributable to memory retrieval from the constructed graph rather than test-time contamination.

B.1

Task-Family Effects of Structured Memory

The benefit of MAGE is not limited to one dataset type because the memory graph is organized around reusable evidence patterns rather than task-specific templates. For text-only QA, MAGE mainly helps by binding entities, relation descriptions, and prior questionanswer traces, which is useful when the answer is a short factual span but the query requires locating the correct entity among many candidates. For chart and infographic QA, the main benefit comes from preserving numeric values, labels, and operations together. A retrieved context that contains only an isolated number is often ambiguous; a retrieved hypergraph neighborhood can also include what the number refers to, which visual element it came from, and which comparison or arithmetic operation was previously associated with it. For document and OCR-heavy visual QA, MAGE improves evidence localization. The model does not need to rely only on global visual understanding; it can receive compact memory entries that connect extracted text, document fields, modality nodes, and related entities. For general visual reasoning, MAGE is useful when the answer depends on combining perception with commonsense or task history. In these cases, the memory layer does not replace the visual model; instead, it supplies structured background evidence and prior successful reasoning patterns that make the final answer easier to produce.

B.2

Why More Context Alone Is Not Sufficient

A central design point of MAGE is that memory augmentation should improve the quality of context, not merely increase its length. Long unstructured retrieval can hurt performance because irrelevant snippets compete with decisive evidence inside the LLM prompt. MAGE addresses this by selecting evidence units with explicit types, provenance, confidence, lifecycle state, and role compatibility. The downstream model therefore sees a smaller but more coherent set of facts, entities, modalities, and traces.

B.3

Relationship Between Memory and MAS Coordination

MAGE acts as an external memory layer rather than a replacement for MAS coordination. The same memory graph can be queried by single-pass inference, refinement-based agents, ReAct-style agents, debate-style agents, planning agents, and role-structured teams. This separation is important: the agent framework decides how to deliberate, while MAGE decides what prior evidence and experience should be exposed. As a result, improvements from MAGE are complementary to improvements from better prompting or stronger base models. The memory layer supplies durable task experience, and the MAS framework uses that experience according to its own reasoning protocol.

Feng et al.

Table 5: Mechanism-level explanation for the observed +MAGE improvement. MAGE mechanism

What it changes in the prompt/context

Why it improves answers

Typed memory nodes

Distinguishes tasks, agents, actions, messages, errors, entities, modalities, procedures, memories, and insights. Returns n-ary evidence relations and collaboration events as retrievable objects. Filters and boosts candidates by agent role.

The LLM receives context with explicit semantics rather than an undifferentiated retrieved paragraph. Multi-hop and multimodal evidence remains bound together, reducing missing-link errors. Planners, executors, critics, retrievers, and summarizers receive different context matched to their function. Recovers indirect evidence even when semantic search only finds one part of the chain.

First-class hyperedges Role policies Bidirectional expansion Provenance tracking Lifecycle status Budgeted packing Trace-to-procedure extraction

C

Expands from seed nodes to neighboring nodes and hyperedge members. Keeps source nodes and member lists for selected memories and hyperedges. Penalizes deprecated or archived items and removes deleted/invalid records. Selects high-value evidence under a token budget. Converts successful MAS behavior into reusable procedural memory.

Qualitative Correct Cases

We randomly sampled correctly answered outputs from the +MAGE setting. All examples in Tables 6–9 satisfy normalized EM=1.0 and F1=1.0. We report the dataset, MAS framework, question, ground truth answer, and +MAGE prediction so that each example can be checked directly. The cases are intended to illustrate the types of evidence that MAGE helps expose to the downstream model; they are not used as additional training or tuning data.

C.1

Case-Level Interpretation

The sampled correct cases cover visual reasoning, chart arithmetic, option selection, textual knowledge, OCR, infographic reading, document understanding, and open-domain QA. Although each individual example is simple to verify after seeing the answer, the collection illustrates why structured memory helps across heterogeneous tasks. In Case 1, the answer requires visual commonsense: the item being readied on a beach towel is most likely used to fly a kite. A flat retrieved text chunk may only mention the object, whereas MAGE can bind the visual modality, object entity, action pattern, and prior similar cases as a reusable high-order event. The prediction fly a kite is accepted because it contains the ground truth concept fly and gives a more specific action. In Case 2, the model must perform chart arithmetic by identifying the three largest values, summing them, and multiplying by the smallest value. MAGE is useful because chart-derived entities and values can be stored together with an evidence hyperedge. This makes it more likely that the LLM receives the relevant numeric context as a bundle rather than isolated numbers. In Case 3, the task is multiple-choice geographic recognition. The prediction includes the option marker and the country name. Roleaware retrieval helps because the context can emphasize modality and entity nodes for the highlighted region rather than unrelated textual memories. In Case 4, the answer is a concise factual title. This is the type of case where semantic memory and entity-centered retrieval are sufficient. MAGE still helps by retrieving the specific entity-title relation rather than relying entirely on parametric knowledge. In Case 5, the answer depends on OCR from an image. A multimodal memory representation can connect the image modality, extracted text, and question-specific evidence. The case illustrates

Helps the model rely on grounded context rather than hallucinated associations. Reduces stale-memory interference. Prevents redundant memory from crowding out decisive evidence. Helps frameworks reuse previous solution patterns without modifying their native coordination logic.

why MAGE stores modality nodes and evidence hyperedges instead of treating all inputs as plain text. In Case 6, the question asks for a comparison over chart values. MAGE helps by preserving the relevant chart evidence and allowing the context packer to return compact numeric evidence. This reduces the chance that irrelevant chart labels occupy the context window. In Case 7, the question asks for a document field label. MAGE can connect extracted document text, local layout context, and the target concept, which is useful when multiple labels appear in the same page. In Case 8, the answer is a numeric count from an infographic. The benefit of MAGE is again evidence localization: retrieval can prioritize modality and evidence nodes that contain the relevant count, avoiding broad summarization of the entire infographic. In Case 9, the answer requires visual commonsense about why an umbrella is being used. MAGE can provide compact context linking the object, scene condition, and likely purpose, helping the model select the intended short answer. In Case 10, the prediction Green Bay Packers is more specific than the ground truth alias packers. The normalized evaluator marks it correct. This illustrates a useful property of MAGE retrieval: by returning contextual entities and relations, it can encourage complete canonical answers rather than underspecified aliases. Cases 11 and 12 show that MAGE can support fine-grained visualdata lookup. In the infographic case, the answer is a date attached to a historical visual timeline; in the chart case, the answer is the label associated with a specific colored line. Both cases benefit from retrieving the label-value or label-mark relationship rather than retrieving a broad image summary. Case 13 shows the document understanding benefit. The answer is a field label, and MAGE helps by preserving the connection between extracted document text, nearby semantic context, and the target concept. This kind of structured retrieval is especially useful when many labels appear in the same page. Cases 14–16 illustrate visual and science reasoning where the final answer is short but depends on selecting the right perceptual or conceptual feature. The memory layer is useful because it can expose compact context about objects, attributes, answer options, and previously successful reasoning patterns.

Diachronic Hypergraphs for Orchestrated Multi-Agent Multimodal Memory Curation

Table 6: Randomly sampled MAGE cases, part 1. ID

Dataset

MAS

Question

Ground truth

+MAGE prediction

1 2

A-OKVQA ChartQA

Direct Direct

fly 389.82

fly a kite 389.82

3

ScienceQA

Direct

Nauru

A) Nauru

4 5

TriviaQA VizWiz

Direct MAD

The guy on the beach towel is readying the item to do what with it most likely? Add the three largest government procurement in GDP and multiply it by the smallest government procurement in GDP across all the countries. Which country is highlighted? Options: (A) Nauru (B) the Federated States of Micronesia (C) the Marshall Islands (D) Samoa. Which Lloyd Webber musical premiered in the US on 10th December 1993? What is written on the back of this sofa?

Sunset Boulevard boston

Sunset Boulevard Boston

Table 7: Randomly sampled MAGE cases, part 2. ID

Dataset

MAS

Question

Ground truth

+MAGE prediction

6 7 8 9 10

ChartQA DocVQA InfographicVQA A-OKVQA TextVQA

Direct Direct Direct Direct Direct

Is the average of all the data value greater than 40? What is the term used for the real image used as a reference? How many fiction books are in this infographic? What is the woman using the umbrella to protect herself from? What team owns this stadium?

No Ground Truth 13 sun packers

No Ground Truth 13 sun Green Bay Packers

+MAGE prediction

Table 8: Additional sampled MAGE cases, part 3. ID

Dataset

MAS

Question

Ground truth

11

InfographicVQA

Direct

1750

1750

12 13 14 15

ChartQA DocVQA A-OKVQA ScienceQA

Direct Direct Direct Direct

When did the cricket bat start resembling a wooden turner? Answer the question with a short phrase. Which country data is shown in the red line? What is the label given to the variables that explain other variables? What color phone does the woman in the blue outfit have? Select the amphibian below. Options: (A) common toad (B) catfish.

Georgia Explanatory Variables yellow common toad

Georgia Explanatory Variables yellow common toad

Table 9: Additional sampled MAGE cases, part 4. ID

Dataset

MAS

Question

Ground truth

+MAGE prediction

16

ScienceQA

Direct

sour

sour

17 18 19 20

TextVQA TriviaQA VizWiz TextVQA

Direct Direct Direct Direct

Which property do these two objects have in common? Options: (A) sour (B) stretchy. What time is it? What is the Japanese share index called? Is this computer still getting ready to boot, or is it already booted? Which gaming system does the box have written on it?

12:04 Nikkei already booted playstation 2

12:04 Nikkei Already booted PlayStation 2

Cases 17 and 20 are OCR-oriented TextVQA examples. MAGE helps by keeping extracted text tied to its visual source and by ranking the most relevant text evidence under the context budget. This reduces confusion between multiple visible strings in the same image. Case 18 is a concise factual QA example. The correct answer can be represented as an entity-centered semantic memory, so retrieval narrows the answer to the target financial index. Case 19 is a status recognition example: the prediction differs only in capitalization and is accepted by normalized EM/F1, showing that MAGE can provide the right evidence even when the final answer surface form varies slightly. Overall, the qualitative cases suggest that MAGE is most helpful when the answer requires preserving a relation among entities, values, modalities, and prior traces. The improvement is not merely due to adding more text to the prompt; it comes from retrieving structured evidence that is compact enough to fit in the context window and specific enough to guide the final generation.

D Full Experimental Protocol D.1 Released data, splits, and leakage control The anonymous repository linked in the abstract releases all training and held out test data used in our experiments, together with the per dataset files, preprocessing utilities, and evaluation scripts. We use a fixed stratified evaluation split for each dataset rather than claiming coverage of the complete benchmark test sets. The remaining public pool of each IND dataset is used for memory graph construction and hyper parameter selection on a held out slice, while OOD pools are used only to prepare the held out evaluation and are never indexed in the shared graph. No test question, image, or answer is inserted into the memory graph, and the build script applies an explicit leakage shield against the test pool using exact and near duplicate matching on normalized question text.

D.2

Memory-graph construction

The shared graph is built from (i) the source documents and images of the IND pool, (ii) public background corpora shipped with each benchmark, and (iii) MAS traces recorded on pool tasks. It contains

Feng et al.

Table 10: Cross-LLM robustness: MAGE F1 per dataset and gain over the strongest non-MAGE baseline on the same questions, for three heterogeneous LLM backends. GPT-4.1-mini

Claude-4.5

Δ MAGE

Δ

MAGE

Text-only IND HotpotQA NQ-Open

51.08 +3.58 54.70 +3.30

43.31 +3.10 56.73 +2.70

47.35 +3.30 29.08 +2.90

Text-only OOD TriviaQA WebQuestions

68.17 +2.10 53.48 +3.90

82.43 +2.40 51.11 +4.00

27.52 +1.90 40.61 +3.50

Multimodal IND ChartQA DocVQA InfographicVQA FinMME

83.71 47.64 81.94 59.22

+2.70 +3.58 +3.20 +4.90

84.15 46.46 84.00 39.91

+2.90 +3.50 +3.80 +5.40

80.56 46.62 80.74 44.66

+4.74 +2.60 +3.70 +5.10

Multimodal OOD A-OKVQA ScienceQA TextVQA VizWiz

79.29 89.55 93.65 40.79

+2.40 +4.20 +3.15 +3.40

72.64 73.28 92.55 40.84

+3.00 +4.80 +2.30 +3.40

72.52 90.02 94.41 45.99

+2.40 +4.30 +2.30 +3.70

Macro avg.

66.93 +3.37

58.34 +3.37

no test answers, no test reasoning trajectories, and no test images. IND versus OOD is defined at the evidence level: a dataset is IND when its source documents are indexed in the shared graph, and OOD when both its documents and its task style are unseen. The same graph is loaded read-only for all methods.

D.3

Retrieval and packing hyper-parameters

Unless stated otherwise: seed top-𝑘 = 8 per vector scope, expansion hop ℎ = 2, token budget 𝐵 = 2048 (measured with the backend tokenizer), and multi-factor ranking weights 𝑤 sem =1.0, 𝑤 role =0.6, 𝑤 conf =0.4, 𝑤 prov =0.3, 𝑤 rec =0.2, 𝑤 use =0.1, 𝑤 life =−0.5, 𝑤 cost =−0.2. Temporal decay follows an exponential kernel with half-life of 30 days; lifecycle states deprecated/archived are excluded from packing. All baselines receive the same budget 𝐵 and the same multimodal inputs; graph-based baselines are built from the identical pool.

D.4

Generation settings, and repetitions

All API calls use temperature 0, top-𝑝 1.0, max output 512 tokens, and a shared retry policy (3 retries, exponential backoff), so decoding is deterministic given the same memory state. Prompts are the versioned templates in prompts/ of the released codebase and are identical across methods within a task family.

E

Dataset

Qwen2.5-VL

Δ MAGE

Dataset

63.95 +3.44

Table 11: Matched representation control on GPT-4.1-mini (F1). Full uses first-class hyperedges; Proj. uses cliqueprojected pairwise edges with all other components fixed.

Matched Pairwise-Projection Control

Design. The − Hyperedge ablation removes 𝑛-ary expansion and therefore changes the candidate pool. We introduce a stricter matched control to isolate the representation itself. The node set, embeddings, seed retrieval, ranking function, role policy, and token budget are held fixed, while each hyperedge 𝑒 = {𝑣 1, . . . , 𝑣𝑘 } is
replaced by the clique of pairwise edges 𝑒2 before expansion. The control therefore receives the same underlying nodes and retrieval machinery, but cannot preserve first-class 𝑛-ary incidence.

MAGE-full

Pairwise-proj.

HotpotQA NQ-Open TriviaQA WebQuestions FinMME ChartQA DocVQA InfographicVQA A-OKVQA ScienceQA TextVQA VizWiz

51.08 54.70 68.17 53.48 59.22 83.71 47.64 81.94 79.29 89.55 93.65 40.79

49.86 53.61 66.95 52.37 57.48 82.05 46.58 80.52 78.10 88.31 92.88 39.94

Macro avg.

66.93

65.72

Result. First-class hyperedges outperform pairwise projection on every dataset, raising macro F1 from 65.72 to 66.93 under otherwise identical conditions. Because the control holds the informationbearing nodes and the complete retrieval pipeline fixed, the consistent advantage identifies preservation of 𝑛-ary incidence as an independent source of MAGE’s gains. The larger losses under − Hyperedge additionally capture the effect of removing hyperedge expansion from candidate generation. Why Pairwise Projection Is Lossy. Let 𝜋 map a hypergraph 𝐻 = (𝑉 , 𝐸) to its clique-expanded graph. There exist 𝐻 1 ≠ 𝐻 2 with 𝜋 (𝐻 1 ) = 𝜋 (𝐻 2 ): take 𝐸 1 = {{𝑎, 𝑏, 𝑐}} and 𝐸 2 = {{𝑎, 𝑏}, {𝑏, 𝑐}, {𝑎, 𝑐}}. Both project to the same triangle, but only 𝐻 1 records the joint event. Hence, 𝜋 is not invertible, and a pairwise store cannot generally distinguish a genuine 𝑛-ary collaboration or evidence bundle from an arbitrary set of dyads.

F

Role-Policy Sensitivity

We stress the role policy by scaling every role boost by ×0.5, ×0.75, ×1.25, and ×1.5, and by varying seed top-𝑘 ∈ {4, 8, 16}, on three representative datasets (HotpotQA, ChartQA, A-OKVQA; GPT-4.1mini). Macro F1 varies by at most ±0.4 across all settings, and the default configuration is never the best by more than 0.3, indicating that the role policy is robust rather than finely tuned. Ablating a single role’s boost (Planner, Executor, Critic, Retriever) changes macro F1 by at most 0.5, with the Critic policy contributing the largest individual share on error-prone multimodal datasets.

G

Diachronic Memory Probes

Static QA cannot, by itself, measure lifecycle behavior, so we complement it with four controlled diachronic probes over a replayed event stream built from the training pools. The stream contains 4,000 insertion events, 200 fact updates, and 120 injected contradictions in chronological order; systems may write, update, and delete during replay, and are then queried on held-out probe questions. Table 12 summarizes the results. Correction propagation measures the fraction of probe questions whose answers reflect a later correcting fact rather than the superseded one. Stale rate is the fraction of retrieved items whose lifecycle state is deprecated or contradicted. Contradiction resolution is the accuracy on questions

Diachronic Hypergraphs for Orchestrated Multi-Agent Multimodal Memory Curation

Table 12: Diachronic probes (GPT-4.1-mini). Higher is better except for stale rate. Correction prop. ↑

Stale rate ↓

Contradiction res. ↑

X-agent transfer ↑

Mem0 Zep / Graphiti Plain graph store

63.4 71.8 58.9

14.2 11.5 17.8

58.1 66.3 54.2

+0.6 +1.1 +0.5

MAGE

91.2

3.1

88.5

+2.4

Method

whose evidence was explicitly contradicted during replay. Crossagent transfer is the F1 gain on probe tasks of agent 𝐵 when agent 𝐴’s traces are present versus absent. MAGE leads on all four probes, which supports the lifecycle and transfer claims that the static benchmarks cannot isolate.

H

Extended System Evaluation

Graph statistics. The shared graph over all twelve pools contains 41,208 nodes, 118,536 binary edges, and 23,940 hyperedges (96.4 MB as JSONL, 38.1 MB after compression), including 9,812 modality nodes and 4,377 distilled procedure/memory nodes. Environment. All measurements use a single server (2× Xeon 8380, 256 GB RAM, NVMe SSD); embeddings are precomputed, so no GPU is required at query time. Latency is measured over 1,000 warm queries after a 100-query warm-up, single client unless noted. Scaling. Query 𝑝 99 grows sub-linearly with graph size (1k nodes: 31.2 ms; 10k: 44.8 ms; 41k: 66.7 ms; 100k synthetic: 88.9 ms), since seed retrieval is index-bounded and expansion is hop-limited. Insertion latency is independent of graph size within measurement noise.

I

Extended Long-Running Simulation

The diachronic probes in Appendix G use injected contradictions to make lifecycle behavior measurable. To further approximate a genuinely long-running deployment, we replay a 10-episode continuous stream (4,000 events from the training pools, no injected artifacts): each episode adds documents and traces chronologically, and a fixed held-out question set is re-evaluated after every episode. MAGE’s macro F1 varies by at most ±0.5 across episodes with no monotone drift, while Mem0 and Zep drift by −2.1 and −1.4 points respectively as superseded memories accumulate; their unpruned stores also retrieve deprecated items at 3.7× MAGE’s stale rate by the final episode. This shows that the lifecycle machinery is load-bearing in sustained operation, not only under adversarial contradiction injection.

J

A Note on Multimodal Representation

MAGE represents multimodal content as modality pointers plus extracted textual surrogates rather than performing native pixellevel reasoning inside the hypergraph. This is a deliberate scoping choice, not an architectural shortcut: the schema treats modalities as first-class typed items, and the surrogate is exactly the auditable, indexable form in which a memory system should store evidence that many heterogeneous agents must later share and inspect. The pointer preserves provenance to the raw item, so a downstream

vision-language reasoner can re-ground any retrieved item on demand. Native multimodal hypergraph reasoning — joint embedding spaces over raw visual tiles — is compatible with the same schema and interfaces and is left to future work.

K

Discussion and Limitations

Overall, the experiments show that MAGE improves memory augmented reasoning by organizing task experience as structured hypergraph memory rather than isolated context fragments, with gains consistent across memory baselines, heterogeneous LLMs, ablations, MAS frameworks, and system level tests. Across the evaluated atomic, reflective, temporal graph, hierarchical, and coordination centric baselines, MAGE’s distinguishing contribution is an auditable end to end memory contract in which first class collaborative hyperedges, role policy, provenance, and lifecycle state remain jointly available to ingestion, revision, retrieval, and context construction. The diachronic probes directly test the resulting behavior under corrections, contradictions, stale evidence, and cross agent reuse. The main limitations are that the constructed hypergraph depends on upstream extraction quality, and larger long running deployments may require more optimized storage, caching, traversal, and adaptive retention policies. Future work can further study online memory evolution and more efficient structured retrieval for MAS systems.

Related documents

• Diachronic Hypergraphs for Orchestrated Multi-Agent Multimodal Memory Curation#798314
• Diachronic Hypergraphs for Orchestrated Multi-Agent Multimodal Memory Curation#796901
• Normative Networks for Source Separation via Local Plasticity and Dendritic Computation#204801
• Agentic Context Management: Solving Agent Memory and Cost by Treating Them as Lifecycle and Architecture Problems#394445
• HyMem: Hierarchical Context Management for Long-Horizon Agents via Information Isolation#616847