arXiv CSopen access

ScrapMem: A Bio-inspired Framework for On-device Personalized Agent Memory via Optical Forgetting

ScrapMem: A Bio-inspired Framework for On-device Personalized Agent Memory via Optical Forgetting Jiale Chang Nanjing Agricultural University Nanjing, China [email protected]

Yuxiang Ren* Nanjing University Suzhou, China [email protected]

arXiv:2605.03804v1 [cs.AI] 5 May 2026

Abstract

finite context window of current foundation models poses substantial challenges for multimodal memory integration and long-range reasoning, making an efficient external memory mechanism indispensable (Lewis et al., 2020; Lu et al., 2025; Lee et al., 2024).

Long-term personalized memory for LLM agents is challenging on resource-limited edge devices due to high storage costs and multimodal complexity. To address this, we propose ScrapMem, a framework that integrates multimodal data into ”Scrapbook Page.” ScrapMem introduces Optical Forgetting, an optical compression mechanism that progressively reduces the resolution of older memories, lowering storage cost while suppressing low-value details. To maintain semantic consistency, we construct an Episodic Memory Graph (EM-Graph) that organizes key events into a causal-temporal structure. Extensive experiments on the multimodal ATM-Bench showcase that ScrapMem provides three main benefits: (1) strong performance, achieving a new state-of-the-art with a 51.0% Joint@10 score; (2) high storage efficiency, reducing memory usage by up to 93% via optical forgetting; and (3) improved recall, increasing Recall@10 to 70.3% through structured aggregation. ScrapMem offers an effective and storage-efficient solution for on-device long-term memory in multimodal LLM agents.

1

Existing LLM agent memory systems primarily follow two paradigms. The first is parametric memory augmentation, which internalizes knowledge into model weights through pre-training (Brown et al., 2020), fine-tuning (Zheng et al., 2024), or parameter-efficient adaptation such as LoRA (Bai et al., 2025d; Hu et al., 2021). The second is retrieval-augmented memory (Zhong et al., 2024), which encodes user memories as high-dimensional semantic vectors or organizes them into structured knowledge graphs for semantic retrieval and compositional reasoning (Borgeaud et al., 2022; Edge et al., 2024; Gutierrez et al., 2024). Despite progress, current approaches remain limited for multimodal personalized memory. Parametric memory is prone to catastrophic forgetting and lacks interpretability (Luo et al., 2025). Retrieval-based methods struggle to align heterogeneous modalities such as text, vision, and speech within a unified semantic space, leading to weak performance on causal, temporal, and logical reasoning (Bai et al., 2025c; Liu et al., 2024; Zhang and Wang, 2026). Moreover, multimodal memory storage is highly inefficient, imposing prohibitive memory and inference overhead for resource-constrained edge devices (Zeng et al., 2025; Cai et al., 2024; Li et al., 2025).

Introduction

LLM-based intelligent agents are increasingly being deployed on edge devices to enable personalized, long-term interactions while preserving the privacy of user-specific data (Xu et al., 2024; Yu et al., 2024; Fu et al., 2024). This setting requires the agent to continuously integrate both static personal data (e.g., photos, emails, videos, and application records) and dynamic interaction traces (e.g., multi-turn dialogues) as memory inputs (Liu et al., 2025; Wu et al., 2024; Mei et al., 2026), so as to accumulate user-centric knowledge and support reasoning over distant historical experiences (Maharana et al., 2024; Park et al., 2023). However, the

As shown in Figure 1, Recent advances in cognitive neuroscience, vision-language modeling, and optical character recognition offer a promising new direction for multimodal memory design (Bai et al., 2025b; Chen et al., 2024; McClelland et al., 1995). Inspired by the Complementary Learning Systems

* Corresponding author: [email protected]

1

ken consumption can be compressed by nearly an order of magnitude compared with raw text (Wei et al., 2025). Furthermore, the progressive degradation of visual fidelity over time—analogous to the natural aging of a scrapbook—closely resembles the biological mechanism of Optical Forgetting, in which fine-grained details fade while salient structure remains. Motivated by these observations, we propose ScrapMem, a bio-inspired personalized multimodal memory framework for edge devices via optical forgetting. ScrapMem redefines long-term interaction history not as a conventional collection of heterogeneous multimodal records, but as a sequence of scrapbook images. Specifically, ScrapMem consolidates cross-temporal user photos, textual records, behavioral traces, and interaction histories into unified scrapbook pages, which are then transformed through visual encoding and optical perception into compact, information-dense memory carriers. Our Contributions are summarized as follows: • We propose ScrapMem, a bio-inspired multimodal memory framework that models longterm agent memory as a sequence of scrapbook pages, enabling highly efficient ondevice storage and unified multimodal integration.

Figure 1: Comparison between human memory (CLS theory) and Scrapbook Memory. Top: The hippocampus rapidly encodes multimodal episodic experiences, while the neocortex gradually consolidates them into stable long-term knowledge. Bottom: ScrapMem similarly binds heterogeneous user data into scrapbook pages and progressively compresses old memories via optical forgetting, preserving core semantics for efficient retrieval and personalized reasoning on edge devices.

• We introduce Optical Forgetting, a progressive resolution decay mechanism that mimics biological memory aging, which filters trivial details and drastically reduces storage cost while preserving core semantic structure. • We design an Episodic Memory Graph (EMGraph) to organize memory nodes along causal-temporal paths, enabling robust longrange reasoning even under visually degraded memories.

(CLS) theory (McClelland et al., 1995), memory can be modeled through the cooperative dynamics of the hippocampus–neocortex system: the hippocampus rapidly binds visual scenes and contextual cues into episodic memory for one-shot learning, while the neocortex gradually consolidates transient traces into structured long-term knowledge (Sun et al., 2023; Thota et al., 2023). Meanwhile, the human practice of maintaining a scrapbook naturally embodies multimodal information fusion, where heterogeneous artifacts such as photographs, notes, and sketches are aggregated on a single page to form scene-centric memory representations. Recent findings from DeepSeek-OCR further demonstrate that vision can serve as a substantially more efficient carrier of information than text: when multimodal content is rendered as images, to-

• Extensive experiments on ATM-Bench show that ScrapMem achieves new state-of-the-art performance with up to 93% storage reduction, demonstrating its effectiveness and efficiency for edge-deployed personalized agents.

2

Related Work

2.1

Memory Systems for Personalized Agents

Personalized agents rely on dynamic user profiling and persistent memory to sustain consis2

storage and lack explicit modeling of temporal dependencies and causal relations, limiting their support for long-horizon personalized decisionmaking.

tent behaviors and enable long-horizon reasoning.Parametric memory internalizes user preferences via fine-tuning or LoRA (Hu et al., 2021), with hybrid architectures such as NextMem (Zhang et al., 2026) to alleviate forgetting. Retrievalbased memory stores interaction histories for similarity search, with representative systems including MemoryBank (Zhong et al., 2024) and Agentic Unlearning (Li et al., 2026b). Structured and agentic memory organizes knowledge via graphs or hierarchical controllers, including Generative Agents (Park et al., 2023), MemGPT (Packer et al., 2023), A-MEM (Xu et al., 2025), and AgenTEE (Abdollahi et al., 2026). Despite progress, these methods still face challenges in multimodal alignment, temporal reasoning, and on-device efficiency, which motivates our ScrapMem framework. 2.2

Our proposed ScrapMem differs by treating memory as multimodal scrapbook pages. It unifies heterogeneous personal data into compact visual carriers, introduces optical forgetting for progressive compression, and incorporates a graph-based episodic structure for causal-temporal reasoning.

3

Preliminaries

In this section, we formalize long-term personalized referential memory QA as the task for ondevice multimodal agent memory. We consider long-term personalized memory includes a user’s long-term, multimodal, and multisource personal memories. We denote the multimodal personal memory corpus as D, which consists of three types of raw user data:

Visual Memory Agents Based on OCR and VLM

Recent studies explore visual memory as an alternative to token-based storage, leveraging VisionLanguage Models (VLMs) and Optical Character Recognition (OCR) to encode lengthy textual histories into compact image representations. These methods exploit the high information density of visual tokens, enabling substantial reductions in context length and storage costs (Wei et al., 2025).

NV NT I D = {Ii }N i=1 ∪ {Vi }i=1 ∪ {Ti }i=1 ,

(1)

where {Ii } denotes image memories, {Vi } denotes video memories, and {Ti } denotes textual memories. To obtain a unified representation from the raw personal memory data D, we define a memory construction function P that transforms the heterogeneous corpus into a standardized memory store M:

Representative systems include AgentOCR (Feng et al., 2026), which performs segment-level optical caching and reinforcement learning-based self-compression for agent trajectories; MemOCR (Shi et al., 2026), which improves robustness under strict memory budgets through layout-aware visual encoding and adaptive density control; and OCR-Memory (Li et al., 2026a), which stores interaction histories as images and retrieves evidence through locate-and-transcribe pipelines. These works demonstrate that visual memory significantly improves efficiency compared with raw text serialization.

M = P(D),

(2)

where M serves as a generic and unified memory representation for all downstream retrieval and reasoning procedures. Given a query q, the agent retrieves a set of relevant memory items E ⊆ M using a retriever R, and generates an answer â based on the retrieved evidence:

However, existing visual memory agents face three major limitations. First, they are largely textcentric: images serve primarily as containers for rendered text rather than native carriers of multimodal user data such as photos and behavioral records. Second, many methods rely on reinforcement learning for memory optimization, resulting in high training costs and weak cross-domain generalization. Third, they focus on appearance-level

E = R(q, M, k),

(3)

where E ⊆ M denotes the retrieved evidence set. An answerer A then generates an answer conditioned on the query and retrieved evidence: â = A(q, E). 3

(4)

vision tokens Zt , while an OCR module produces textual content Tt .

To model realistic long-term memory dynamics, we further define two cognitive operations in a generic form:

To bridge the modality gap, we further project vision tokens into the textual tokens via a vision-totext transformation:

1. Memory Linking. We define a memory linking function that establishes relational connections between memory items: L : M → RN ×N ,

T̃t = Tt ⊕ g(Zt ),

(5)

where g(·) denotes a semantic description function (e.g. image captioning or summarization) that converts vision tokens into textual descriptions.

where N is the number of memory items in M, and RN ×N denotes the adjacency matrix of relational connections.

Based on T̃t , a large language model extracts semantic nodes, and we directly incorporate raw vision tokens as visual nodes to form a hybrid node set:

2. Memory Forgetting. We define a memory forgetting function that updates memory states over time: F : M → M, (6)

Vt = ELLM (T̃t ) ∪ Zt ,

which models changes in memory accessibility over time.

4

Method: ScrapMem

4.2

EM-Graph Construction

To enable structured reasoning over long-term memories, we organize the extracted node sets into an Episodic Memory Graph (EM-Graph). The EM-Graph is defined as S a event-structured graph G = (V, P), where V = t Vt denotes the global node set and P denotes the units of all episodic memory paths.

Scrapbook Page Consolidation and Optical Perception

Given the multimodal memory corpus D defined in Eq. (1), we first organize heterogeneous user data into temporally grounded scrapbook pages, which serve as the fundamental units of episodic memory. Specifically, all multimodal records within a temporal page are aggregated into a unified page representation: St = C({Ii , Vj , Tk }t ),

(9)

where ELLM (·) denotes an LLM-based semantic extraction function that identifies entities, events, and their relations, and Zt denotes the raw vision tokens treated as explicit visual graph nodes. This process establishes an index–content separation: scrapbook pages retain raw multimodal evidence, while the hybrid node set integrates textual semantics and visual features for downstream retrieval.

Figure 2 illustrates our bio-inspired multimodal retrieval and reasoning framework. Specifically, the process leverages three core stages: Scrapbook Page Consolidation and Optical Perception, EMGraph Construction, and Optical Forgetting, which supports personalized reasoning through episodic memory retrieval. 4.1

(8)

Within each scrapbook page, related nodes are arranged into Episodic Memory Paths (EM-Paths), which capture localized causal and logical event chains. Each EM-Path is an ordered subset of nodes originating from the same temporal page:

(7)

(k)

Pt

where C(·) denotes a temporal consolidation function, St represents the constructed scrapbook page at time step t.

 = vt,1 , vt,2 , . . . , vt,lk ,

vt,i ∈ Vt , (10)

where k indexes distinct semantic paths within page t. Intuitively, EM-Paths model coherent eventcentric narratives (e.g., diagnosis → treatment → recovery), forming the basic units for multi-hop reasoning.

Each scrapbook page preserves both highfidelity visual content and textual information. To enable efficient storage and unified processing, we introduce an optical perception pipeline that transforms the raw page into textual descriptions. Concretely, a vision encoder extracts a sequence of

To facilitate efficient matching during retrieval, we explicitly encode node–path relationships via a 4

1

Scrapbook Page Consolidation & Optical Perception

… Images

Videos

2

Scrapbook Page

Multimodal Memory Corpus Temporal Consolidation

EM-Graph Construction

3

Temporal Degradation

Hybrid Node Sets from All Pages

…

…

Optical Forgetting

Original Page

Faded Page

Increase

…

Texts

…

Optical Perception Pipeline

Vision Encoder

Episodic Memory Graph (EM-Graph)

Vison Decoder

OCR

Nodes

(Description)

(Text) Visited hospital due to fever. Doctor diagnosed influenza and prescribed meds.

Vision Tokens

Fading

Semantic Node

A person on a hospital bed. A doctor is checking the patient.

Visual Node

(Enriched Text)

Updated EM-Graph

EM-Paths

LLM Semantic Extraction Hybrid Node Set Visual Nodes (Vision Tokens)

…

Query

…

…

Semantic Nodes (Textual)

Node Extraction

…

…

Retriever Query Nodes

What was the diagnosis and treatment?

…

Prune

Q-matrix

…

Q-matrix Matching 1 0 1

… …

1

1

0

0

1

1

…

0

…

…

…

…

…

1

0

0

…

1

1

Storage & Computation Efficiency

Retrieved EM-Paths

Answerer

… … …

Cognitive Filtering for Stable Reasoning

Predicted Answer Diagnosis: Influenza Treatment: Medication and rest.

Figure 2: Overview of the ScrapMem. (1) Consolidation and Perception: Unifies heterogeneous records (images, videos, text) into hybrid representations via OCR and vision-to-text extraction. (2) EM-Graph Construction: Organizes nodes into an Episodic Memory Graph with event-centric paths (EM-Paths) for structured retrieval and multi-hop reasoning. (3) Optical Forgetting: Compresses outdated memories through temporal degradation to reduce costs while preserving salient knowledge. The system retrieves relevant EM-Paths to generate personalized, evidence-grounded responses.

4.3

binary incidence matrix (Q-matrix):

Optical Forgetting

To model long-term memory updating under limited storage and cognitive constraints, we propose an optical forgetting mechanism to gradually compress and fade outdated scrapbook pages over time.

( 1, Qij = 0,

vj ∈ Pi , Q ∈ {0, 1}|P|×|V| , otherwise, (11) where P = {Pi } denotes the set of all EM-Paths. This structure enables efficient matching between query nodes and candidate paths by measuring node overlap.

For a scrapbook page at time step t, its information fidelity decays with temporal interval τ . Instead of modeling raw pixel-level degradation explicitly, we formulate the forgetting process as a time-dependent lossy transformation:

For retrieval, given an input query q, we first process q through the same optical perception and LLM extraction pipeline to yield a set of query nodes. Following the problem formulation in Preliminaries, our retriever R leverages the node–path incidence matrix Q to measure semantic overlap between query nodes and graph nodes, and returns the top-k most relevant EM-Paths as retrieved memory evidence:

(τ )

St

= Dτ (St ),

(12)

where Dτ (·) is a temporal degradation operator that gradually eliminates trivial visual details and redundant textual contents from the original scrapbook page. As the degree of forgetting intensifies, finegrained visual and textual clues within the scrapbook page become less accessible, which inevitably degrades the optical perception and semantic node extraction procedure. Accordingly, the original hybrid node set evolves into a condensed subset:

E = R(q, M, k), where E ⊆ M denotes the retrieved evidence set consisting of salient episodic memory paths. The retrieved evidence E is then fed into the answerer A for downstream reasoning and response generation, producing the final predicted answer â = A(q, E).

(τ )

Vt

⊆ Vt ,

(13)

where only semantically salient and structurally critical nodes are retained after temporal fading. 5

This condensation further drives the evolution of the event-structured EM-Graph. Nodes that can no longer be reliably perceived or semantically interpreted are eliminated, and the corresponding episodic memory paths containing these faded nodes are pruned. The evolved graph after optical forgetting is formalized as:
G (τ ) = V (τ ) , P (τ ) , (14)

we employ Qwen3-VL-2B-Instruct (Bai et al., 2025a) as the memory processor, all-MiniLM-L6v2 (Reimers and Gurevych, 2019) as the default retriever embedding model, and set the retrieval budget top-k=10. 5.2

In this section, we conduct extensive experiments on ATM-Bench to evaluate the overall performance improvement. Table 1 reports the main results.

where V (τ ) denotes the pruned global node set, and P (τ ) is the condensed collection of survived EM-Paths. The evolved graph preserves high-level causal and event-level semantic structures while discarding transient, noisy, and time-decayed details.

State-of-the-Art Performance. The experimental results demonstrate that ScrapMem (No-Forget) achieves a Joint@10 score of 51.0% on the full ATM-Bench dataset, outperforming the previous strong baseline ATM-RAG (48.6%) and establishing a new state-of-the-art performance on this benchmark. These results demonstrate the effectiveness of our integrated framework, which leverages Scrapbook pages for multimodal aggregation and EM-graphs for structured path indexing. ScrapMem surpasses baseline methods on complex crossmodal memory QA tasks.

From the system perspective, optical forgetting plays two essential roles in our memory framework. First, it alleviates storage and computational overhead by compacting obsolete long-term memories within the memory store M. Second, it acts as a native cognitive filtering mechanism, ensuring that downstream retrieval and multi-hop reasoning rely on stable, salient, and long-lasting memory patterns.

5

Experiments and Results

5.1

Experimental Setup

Main Results

Superior List Recall Capability ScrapMem (NoForget) demonstrates a substantial advantage on the challenging List Recall (R) task, achieving a score of 50.4%, which significantly outperforms HippoRAG2 (39.3%) and ATM-RAG (32.4%). When processing fragmented, heterogeneous evidence, the Scrapbook Page aggregates daily texts and images into a unified visual layout. This enables the model to capture correlated contextual information comprehensively during retrieval, greatly improving the completeness of personalized memory retrieval in complex scenarios.

Datasets and Metrics. We evaluate ScrapMem on ATM-Bench (Mei et al., 2026), a challenging longterm multimodal personalized memory benchmark containing four years of real-world personal data across heterogeneous sources (emails, images, and videos). Following the official standard protocol, we report Question Type Score (QS) for answer accuracy, Recall@10 (R@10) for retrieval quality, and Joint@10 to measure end-to-end performance. We further break down performance over three question types: Number (N), List Recall (R), and Open-ended (O).

High Retrieval Precision. ScrapMem (No-Forget) achieves superior retrieval performance with an R@10 score of 70.32%, making it the only system across all baselines to exceed 70%. This remarkable result verifies the efficacy of the EMGraph mechanism and the two-stage retrieval pipeline. The hybrid strategy first performs dailylevel matching and then fine-grained evidence reranking, effectively boosts overall retrieval recall.

Baselines. We compare ScrapMem against two categories of baselines: (1) Memory Agents, including A-Mem (Xu et al., 2025) and Mem0 (Chhikara et al., 2025), (2) RAG Systems: represented by HippoRAG2 (Gutierrez et al., 2025) and ATM-RAG (Mei et al., 2026). We also include an Oracle setting with ground-truth evidence as the performance upper bound.

Robust Memory Forgetting. After adopting optical forgetting, with degraded visual inputs, ScrapMem still retains strong performance, achieving a Joint@10 score of 46.9% and an R@10 score of 66.1%. In particular, performance under optical forgetting substantially surpasses A-Mem (44.0%)

Implementation Details. We access GPT-5-mini via the OpenAI API for the LLM-based judge, 6

#

Memory

System

ATM-Bench

Rep.

QS

R@10

Joint@10

N

R

O

– DM SGM

0.2 70.0 77.8

– – –

– – –

0.0 81.8 85.0

0.0 69.3 90.4

0.6 61.9 69.5

DM DM DM DM

46.1 44.8 43.5 41.5

66.6 66.4 61.9 61.9

44.0 42.8 41.8 38.4

55.0 57.2 57.2 58.6

27.0 16.9 25.9 25.9

44.9 43.6 48.4 33.7

HippoRAG2 HippoRAG2 Self-RAG Self-RAG ATM-RAG ATM-RAG

DM SGM DM SGM DM SGM

42.9 47.7 42.1 50.3 42.0 51.0

66.4 69.6 61.8 68.7 61.8 68.7

41.5 46.9 42.8 48.2 41.3 48.6

58.6 59.4 46.4 59.7 50.6 60.3

34.9 39.3 33.1 35.3 31.0 32.4

34.1 41.8 30.9 48.4 38.9 48.2

ScrapMem (No-Forget) ScrapMem

Graph Graph

52.5 48.4

70.3 66.1

51.0 46.9

57.2 55.3

50.4 45.3

48.8 42.8

Upper / Lower Bounds 1 2 3

No-Evidence Oracle Oracle

Memory Agents 4 5 6 7

A-Mem (Piled) A-Mem (Linked) Mem0Agentic Mem0Plain

RAG Systems 8 9 10 11 12 13 Ours 14 15

Table 1: Overall performance comparison across models on the ATM-Bench dataset. QS is the overall comprehensive metric. N, R, and O denote the accuracy of Number, List Recall, and Open-ended questions. ScrapMem (No-Forget) achieves state-of-the-art results on Joint@10 and the R@10, and surpasses existing baseline methods across key metrics.

and Mem0 (41.8%), and reaches the same level as HippoRAG2 (46.9%). This indicates that the nodes and EM-paths extracted and stored in the EM-Graph maintain robust semantic foundations when visual information is impaired. Furthermore, optical forgetting serves as an effective mechanism to filter redundant information. 5.3



5HFDOO

   

Sensitivity Analysis of Optical Forgetting

 Configuration No-Forget (Baseline) Very_Soft Softer_Old Timed-Gentle (Default) Boundary_365

JPEG (Q)

Scale (S)

Boundaries (T)

100 / 100 / 100 95 / 82 / 70 90 / 75 / 60 90 / 70 / 40 95 / 75 / 55

1.0 / 1.0 / 1.0 1.0 / 0.95 / 0.85 1.0 / 0.90 / 0.80 1.0 / 0.85 / 0.60 1.0 / 0.90 / 0.75

N/A 180 / 730d 180 / 730d 180 / 730d 365 / 900d

1R)RUJHW 9HU\B6RIW 6RIWHUB2OG 7LPHG*HQWOH %RXQGDU\B

   





.7RS.UHWULHYHGLWHPV



Figure 3: Retrieval performance (Recall@K) under varying optical forgetting intensities. The clustering of different forgetting curves demonstrates that ScrapMem is highly robust to specific hyperparameter configurations.

Table 2: Hyperparameter settings for different forgetting strategies, ordered by increasing degree of information compression. Q and S and T represent the quality factors, scaling factors, and temporal stage boundaries (in days) for Recent/Mid-term/Old stages, respectively.

quality (Q), resolution scaling factor (S), and temporal stage boundaries (T ) for Recent, Mid-term, and Old memories, respectively.

To evaluate the stability of ScrapMem under varying degrees of visual memory degradation, we design five forgetting configurations in Table 2, ordered by increasing levels of information compression. Each setting specifies a distinct fidelitystorage trade-off through three parameters: JPEG

Figure 3 presents the Recall@K curves under different experimental configurations. The No-Forget setting acts as the performance upper bound. Notably, the curves of all forgettingenabled variants achieve highly consistent results 7



with nearly identical trends. Specifically, even with drastic visual degradation ranging from the mild Very_Soft setting to the aggressive Boundary_365 constraint, the R@10 results remain stable around 66%, demonstrating that model performance is insensitive to varying visual corruption intensities.



-RLQW#





  



7RWDO6WRUDJH)RRWSULQW0L%



Figure 4: Storage–performance trade-off on ATMBench (Joint@10). The x-axis uses a logarithmic scale. ScrapMem (Timed-Gentle, orange star) reduces storage by 93.0% relative to the raw-data baseline while retaining over 90% of SOTA performance (46.3% vs. 51.0%). The Pareto frontier indicates strong efficiency and graceful performance degradation, supporting ondevice deployment.

strengthen long-range reasoning. Extensive experiments on ATM-Bench show that ScrapMem achieves state-of-the-art performance, supports up to 93% storage reduction, and maintains robust reasoning under visual degradation, demonstrating its effectiveness for privacy-preserving, resourceefficient long-term personalized agent memory on edge devices.

Storage Efficiency

Storage Efficiency. Figure 4 quantify the storage–performance trade-off of ScrapMem. Compared to the raw-data baseline (4302.9 MiB), ScrapMem (No-Forget) already reduces storage by 78.5% while achieving the best performance (51.0% Joint@10). With optical forgetting enabled, storage can be further reduced to 299.5 MiB (93.0% saving) under the Timed-Gentle configuration, while retaining over 90% of peak performance (46.3%). Other configurations exhibit similar behavior, consistently achieving 88–92% storage reduction with marginal performance variation. These results show that ScrapMem effectively decouples storage footprint from end-to-end QA performance, enabling aggressive compression with controlled degradation. This validates our design goal of supporting storage-constrained, on-device deployment.

6





This observation validates the core design principle of ScrapMem: retrieval performance does not rely heavily on high-quality visual inputs. Instead, structural nodes and temporal-spatial logic paths modeled in the EM-Graph construct a robust semantic backbone for memory retrieval. Benefiting from this design, our method preserves stable recall performance even under severe visual compression and quality reduction for storage optimization. Such advantages greatly enhance the practical robustness of our framework, making it well-suited for real-world heterogeneous on-device deployment. 5.4

2ULJLQDO5DZ 1R)RUJHW 9HU\B6RIW 6RIWHUB2OG 7LPHG*HQWOH %RXQGDU\B



Limitations ScrapMem relies on visual rendering and optical perception modules, whose performance may degrade under extremely aggressive optical forgetting or highly cluttered multimodal layouts. Additionally, the EM-Graph construction depends on LLMbased semantic extraction, which can introduce noise in event relation modeling. Since the framework is tailored for on-device multimodal memory, it may require additional adaptation to generalize to open-world interactive agents with continuous tool use and dynamic environment feedback.

Conclusion

ScrapMem proposes a bio-inspired, on-device multimodal personalized memory framework built with scrapbook pages and optical forgetting. By unifying heterogeneous user data into compact scrapbook pages and modeling memory decay via progressive resolution degradation, we drastically improve storage efficiency while preserving core semantics. The episodic memory graph further structures events into causal-temporal paths to

References S. Abdollahi and 1 others. 2026. Agentee: Confidential llm agent execution on edge devices. Proceedings of the 2026 EuroMLSys Conference. Shuai Bai, Yuqi Cai, Ruoyi Chen, Kai Chen, Xu Chen, Zhihao Cheng, and 1 others. 2025a. Qwen3-vl technical report. arXiv preprint arXiv:2511.21631.

8

Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, and 1 others. 2025b. Qwen2.5-vl technical report. arXiv preprint arXiv:2502.13923.

Bernal Jimenez Gutierrez, Yiheng Shu, Yu Gu, Michihiro Yasunaga, and Yu Su. 2024. Hipporag: Neurobiologically inspired long-term memory for large language models. In Advances in Neural Information Processing Systems (NeurIPS).

T. Bai, J. Fan, X. Wen, J. Kang, H. Lan, R. Zhao, P. Wu, Z. Zhang, Y. Zhong, G. Li, and D. Lin. 2025c. Survey on ai memory: Theories, taxonomies, evaluations, and emerging trends. arXiv preprint.

Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, and 1 others. 2021. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685.

Ting Bai, Le Huang, Yue Yu, Cheng Yang, Cheng Hou, Zhe Zhao, and Chuan Shi. 2025d. Efficient multitask prompt tuning for recommendation. ACM Transactions on Information Systems, 43(4):1–21.

Kuang-Huei Lee, Xinyun Chen, Hiroki Furuta, John F. Canny, and Ian Fischer. 2024. A human-inspired reading agent with gist memory of very long contexts. In International Conference on Machine Learning (ICML).

Sebastian Borgeaud, Arthur Mensch, Jordan Hoffmann, Trevor Cai, Eliza Rutherford, and 1 others. 2022. Improving language models by retrieving from trillions of tokens. In International Conference on Machine Learning (ICML), pages 2206–2240.

Patrick Lewis, Ethan Perez, Aleksander Piktus, and 1 others. 2020. Retrieval-augmented generation for knowledge-intensive nlp tasks. In Advances in Neural Information Processing Systems (NeurIPS).

Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, and 1 others. 2020. Language models are few-shot learners. In Advances in Neural Information Processing Systems (NeurIPS), pages 1877–1901.

J. Li, Y. Zhang, X. Yang, J. Qu, J. Xu, S. Yang, J. Ding, and E. C. H. Ngai. 2026a. Ocr-memory: Optical context retrieval for long-horizon agent memory. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026).

D. Cai, S. Wang, C. Peng, and Z. Zhang. 2024. Recall: Empowering multimodal embedding for edge devices. In Proceedings of the International Conference on Mobile Computing and Networking (MobiCom).

J. Li and 1 others. 2025. Venus: An efficient edge memory-and-retrieval system for vlm-based online video understanding. In Advances in Neural Information Processing Systems (NeurIPS).

Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, and 1 others. 2024. Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling. arXiv preprint arXiv:2412.05271.

X. Li and 1 others. 2026b. Agentic unlearning: When llm agent meets machine unlearning. arXiv preprint arXiv:2602.17692. B. Liu, C. Lyu, and Z. Min. 2024. Retrieval meets reasoning: Even high-school textbook knowledge benefits multimodal reasoning. In Proceedings of EMNLP.

Prateek Chhikara, Dhananjay Khant, Sourav Aryan, Tarun Singh, and Deepak Yadav. 2025. Mem0: Building production-ready ai agents with scalable longterm memory. arXiv preprint arXiv:2504.19413.

J. Liu, Y. Sun, W. Cheng, H. Lei, Y. Chen, and 1 others. 2025. Memverse: Multimodal memory for lifelong learning agents. arXiv preprint arXiv:2512.03627.

Darren Edge, Ha Trinh, Newman Cheng, Joshua Bradley, Alex Chao, Apurva Mody, Steven Truitt, and Jonathan Larson. 2024. From local to global: A graph rag approach to query-focused summarization. CoRR, abs/2404.16130.

Miao Lu, Weiwei Sun, Weihua Du, Zhan Ling, Xuesong Yao, Kang Liu, and Jiecao Chen. 2025. Scaling llm multi-turn rl with end-to-end summarizationbased context management. arXiv preprint arXiv:2510.06727.

L. Feng, F. Yang, F. Chen, X. Cheng, H. Xu, Z. Wan, M. Yan, and B. An. 2026. Agentocr: Reimagining agent history via optical self-compression. arXiv preprint arXiv:2601.04786.

Yun Luo, Zhen Yang, Fandong Meng, Yafu Li, Jie Zhou, and Yue Zhang. 2025. An empirical study of catastrophic forgetting in large language models during continual fine-tuning. IEEE/ACM Transactions on Audio, Speech, and Language Processing.

Y. Fu, R. Anantha, and J. Cheng. 2024. Camphor: Collaborative agents for multi-input planning and high-order reasoning on device. arXiv preprint arXiv:2410.09407.

Adyasha Maharana, Dong-Ho Lee, Sergey Tulyakov, Mohit Bansal, Francesco Barbieri, and Yuwei Fang. 2024. Evaluating very long-term conversational memory of llm agents. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL), pages 13851–13870.

Bernal J. Gutierrez, Yu Shu, Weijia Qi, Shuo Zhou, and Yu Su. 2025. From rag to memory: Non-parametric continual learning for large language models. arXiv preprint. Published at ICLR 2025 / OpenReview LWH8yn4HS2.

9

James L. McClelland, Bruce L. McNaughton, and Randall C. O’Reilly. 1995. Why there are complementary learning systems in the hippocampus and neocortex. Psychological Review, 102(3):419–457.

Delong Zeng, Yuexiang Xie, Yaliang Li, and Ying Shen. 2025. Enhancing multimodal retrieval via complementary information extraction and alignment. In Annual Meeting of the Association for Computational Linguistics (ACL).

J. Mei, J. Chen, G. Yang, X. Hou, M. Li, and B. Byrne. 2026. According to me: Long-term personalized referential memory qa. arXiv preprint arXiv:2603.01990.

L. Zhang and Y. Wang. 2026. Trace: Grounding time series in context for multimodal embedding and retrieval. In International Conference on Learning Representations (ICLR).

C. Packer, V. Wooders, K. Lin, S. Fang, G. Shieh, and I. Fiete. 2023. Memgpt: Towards llms as operating systems. arXiv preprint arXiv:2310.08560.

Zeyu Zhang, Rui Li, Xiaoyan Zhao, Yang Zhang, Wenjie Wang, Xu Chen, and Tat-Seng Chua. 2026. Nextmem: Towards latent factual memory for llmbased agents. arXiv preprint arXiv:2603.15634.

Joon Sung Park, Joseph C. O’Brien, Carrie Jun Cai, Meredith Ringel Morris, Percy Liang, and Michael S. Bernstein. 2023. Generative agents: Interactive simulacra of human behavior. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology (UIST), pages 2:1–2:22.

Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, Zheyan Luo, and 1 others. 2024. Llamafactory: Unified efficient fine-tuning of 100+ language models. arXiv preprint arXiv:2403.13372. Wanjun Zhong, Lianghong Guo, Qiqi Gao, He Ye, and Yanlin Wang. 2024. Memorybank: Enhancing large language models with long-term memory. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), pages 19724–19731.

Nils Reimers and Iryna Gurevych. 2019. Sentence-bert: Sentence embeddings using siamese bert-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing (EMNLP). Y. Shi, S. Liu, Y. Yang, W. Mao, Y. Chen, Q. Gu, H. Su, X. Cai, X. Wang, and A. Zhang. 2026. Memocr: Layout-aware visual memory for efficient longhorizon reasoning. arXiv preprint arXiv:2601.21468. W. Sun, M. Advani, N. Spruston, A. Saxe, and J. E. Fitzgerald. 2023. Organizing memories for generalization in complementary learning systems. Nature Neuroscience, 26(8):1438–1448. M. Thota, D. Yi, and G. Leontidis. 2023. Lleda— lifelong self-supervised domain adaptation. Knowledge-Based Systems, 279:110959. Haoran Wei, Yaofeng Sun, and Yukun Li. 2025. Deepseek-ocr: Contexts optical compression. arXiv preprint arXiv:2510.18234. B. Wu, Y. Li, Z. Zhang, Y. Wei, M. Fang, and L. Chen. 2024. Foundations and recent trends in multimodal mobile agents: A survey. arXiv preprint arXiv:2411.02006. Jiajun Xu, Zhiyuan Li, Wei Chen, Qun Wang, Xin Gao, Qi Cai, and Ziyuan Ling. 2024. On-device language models: A comprehensive review. arXiv preprint arXiv:2409.00088. Weiran Xu, Zhijian Liang, Jingbiao Mei, Hang Gao, Jie Tan, and Yi Zhang. 2025. A-mem: Agentic memory for llm agents. In Advances in Neural Information Processing Systems (NeurIPS). Zhongkai Yu, Shengwen Liang, Tianyun Ma, Yunke Cai, Ziyuan Nan, Di Huang, Xinkai Song, Yifan Hao, Jie Zhang, Tian Zhi, Yongwei Zhao, Zidong Du, Xing Hu, Qi Guo, and Tianshi Chen. 2024. Cambricon-llm: A chiplet-based hybrid architecture for on-device inference of 70b llm. arXiv preprint arXiv:2409.15654.

10