EvoMem: Memory-Augmented Evolution for Code Optimization

EvoMem: Memory-Augmented Evolution for Code Optimization

Viktor Volkov 1

Valentin Khrulkov 1

Nikita Glazkov 1

Olga Volkova 1

Andrey V. Galichin 1 Konstantin Pchelin 1

Dmitry V. Dylov 1,3

Danil Sivtsov 1 Iaroslav Bespalov 1

Petr Anokhin 1,2

Ivan Oseledets 1,2

arXiv:2608.10795v1 [cs.AI] 11 Aug 2026

1

2

AXXX, Moscow, Russia Lomonosov Moscow State University, Moscow, Russia 3 Applied AI Institute, Moscow, Russia

Abstract Successful mutation strategies in evolutionary code search may contain reusable knowledge that is useful beyond a single run, and in some cases may transfer across related tasks and domains. However, existing LLM-driven evolutionary frameworks largely discard such knowledge, repeatedly rediscovering similar ideas and limiting opportunities for cross-run and cross-task learning. We introduce EvoMem, a persistent memory architecture for LLM-based evolutionary program search that captures and reuses candidate mutation knowledge. EvoMem converts successful mutation events into structured, task-aware advice for future runs. It operates in two phases: after each run, it extracts and stores promising ideas with provenance, and during subsequent evolution, it retrieves a small set of relevant instructions based on the current task and program context to guide mutation. Across geometric optimization, multi-hop question answering, GPU kernel optimization, and related benchmarks, our experiments show positive average improvements in target metrics or search speed for most evaluated settings, while also revealing variability across tasks. Overall, EvoMem provides evidence that persistent memory can reduce some redundant exploration and improve the reuse and adaptation of successful strategies in LLM-driven evolutionary search.

1

Introduction

LLM-based evolutionary code search has become a promising way to automate program improvement. Systems such as AlphaEvolve combine LLM-generated code mutations with evolutionary selection: they retrieve candidate programs, prompt an LLM to propose a modification, evaluate the result, and retain successful variants in a population [27]. This evaluator-in-the-loop structure is especially attractive for research and optimization tasks in which proposed ideas can be expressed as code and tested automatically. AlphaEvolve has already demonstrated this pattern on matrix multiplication, mathematical search, and systems optimization, showing that LLMs can be useful mutation operators when they are embedded in a disciplined search loop. However, current evolutionary code-search systems mostly treat each run as a self-contained search process. Successful runs produce more than a final program: they also reveal tactics such as useful Preprint.

Figure 1: EvoMem pipeline. Successful mutation events from completed runs are distilled into structured memory cards with provenance, stored in a persistent memory layer, and retrieved as bounded advice for future mutations without changing the underlying evolutionary loop. decompositions, numerical stabilization tricks, pruning rules, data-layout changes, or prompting strategies that may remain valuable beyond the original task. In standard evolutionary pipelines, this knowledge is usually left implicit in the evolved programs and logs. When the system starts a new run, even on a related problem, it must often rediscover similar ideas through broad search. Naively reusing prior artifacts is not enough. A concrete program is usually tied to a particular task, metric, and interface, while simply mixing many experiments into one population can make retrieval less precise and can expose the mutation model to irrelevant context. What is needed is a mechanism that abstracts useful mutation knowledge from past runs, preserves enough provenance to keep it auditable, and retrieves only a bounded amount of relevant advice during future mutations. We present EvoMem, a persistent memory architecture for LLM-based evolutionary program search. EvoMem is implemented as an extension to GigaEvo, an open-source implementation of an AlphaEvolve-style optimization pipeline [13]. After an evolutionary run, EvoMem extracts successful mutation events into structured memory records, deduplicates and enriches them, and stores them with task context and provenance. During later evolution, it retrieves a small set of relevant memory instructions based on the current task and program context and inserts them into the mutation prompt as advice rather than as a hard constraint. This paper makes the following contributions: • We introduce EvoMem, a persistent memory mechanism that stores and reuses mutation knowledge across LLM-based evolutionary code-search runs. • We describe a two-phase memory pipeline: an offline write phase that extracts, filters, deduplicates, and stores successful ideas with provenance, and an online read phase that retrieves bounded task-relevant advice during mutation. • We implement EvoMem on top of GigaEvo and evaluate it on transfer settings spanning geometric optimization, multi-hop question answering, GPU kernel optimization, and scientific-code tuning. • We provide empirical evidence that memory reuse can accelerate some later evolutionary runs and can support cross-problem transfer, with improvements appearing as higher target metrics, faster search, or both depending on the task.

2

2

Related Work

Recent AI-driven scientific-discovery systems combine code generation, retrieval, external tools, and experimental feedback to automate parts of the research workflow [3, 31, 9]. Within this broader landscape, the line of work most relevant to this paper is evolutionary program and code search, where systems such as AlphaEvolve and CodeEvolve iteratively modify candidate programs, evaluate them, and retain successful variants [27, 1]. Our focus is narrower than the general scientific-discovery literature: we study how reusable knowledge can be extracted from prior evolutionary runs and reintroduced as persistent memory in later mutations. Further research on AlphaEvolve capabilities has shown that this approach can rediscover and improve strong existing solutions across multiple domains, including mathematical reasoning, combinatorial search, and systems optimization [7, 26, 20, 33, 14]. Recent variants also show evolutionary approaches being adapted to specialized domains such as realistic CAD generation and quantum-circuit T-count optimization, while ImprovEvolve changes the evolved program interface around proposing, improving, and perturbing candidate solutions [4, 5, 15]. Follow-up work on open or alternative implementations suggests that the search loop itself can be improved: smaller RL-trained models may outperform larger closed ensembles in some settings, task decomposition can make evolution more efficient and interpretable, and adaptive metrics, sampling, or meta-evolution can further improve performance [34, 17, 23]. The closest neighboring systems are those that combine evolutionary code search with domain knowledge or explicit memory. AlphaEvolve and related systems such as ShinkaEvolve and MAPPS show that iterative program modification and evaluation can support scientific and algorithmic discovery in specialized settings [27, 16, 40]. Nearby examples include GeoEvolve, which combines evolutionary search with domain knowledge, and AgenticSciML, which adds retrieval-augmented method memory to scientific machine-learning workflows [25, 11]. Most other domain-specific agent systems focus on chemistry, biology, or materials pipelines rather than on persistent cross-run memory for evolutionary code search [35, 8, 21]. Agentic memory architectures such as A-MEM and General Agentic Memory likewise retrieve and update agent experience at task time [36, 37]. EvoMem differs by storing provenance-grounded advice distilled from successful evolutionary mutations rather than general episodic or document memory. Tool-augmented systems are relevant insofar as they show how code generation, execution, retrieval, and reflection can be organized into reusable workflows [31, 9, 19]. Among them, OR-Agent is the closest comparison because it combines structured research loops with memory-based reflection, while SciDataCopilot similarly emphasizes reusable workflow generation [22, 30]. Other systems such as AI Scientist, Paper2Code, CodeScientist, and MLR-Copilot demonstrate broader executable research pipelines, but they are less specifically focused on persistent memory for evolutionary mutation and cross-run reuse [24, 32, 10, 18].

3

Memory architecture

GigaEvo follows a standard evolutionary loop: it selects parent programs, asks an LLM-based mutation operator to modify them, executes the resulting candidates, and stores their metrics. In the baseline setting, each mutation sees only local run context such as the parent code, current metrics, recent insights, and lineage. This keeps the search procedure simple and controlled, but it also means that useful tactics discovered in one run are not available in later runs. EvoMem addresses this limitation by converting successful mutation events into reusable, task-aware advice. The design is intentionally split into two phases. The write phase runs after an evolution job finishes, extracts generalizable ideas from the produced programs, and stores them in a memory store. The read phase runs before each new mutation, retrieves a small set of relevant memory cards, and injects them into the mutation prompt. This separation is important for efficiency: expensive operations such as clustering, deduplication, and optional LLM-based enrichment happen offline after the run, while mutation-time retrieval remains a small bounded step.

3

3.1

General pipeline

At a high level, a completed run produces a set of programs, a post-run analysis stage extracts candidate ideas from those programs, the memory layer normalizes and stores them, and future runs query this store before proposing a new mutation. We use the following compact notation to make the data flow explicit: extract

write

select(p,τ,µ)

Pr −−−−→ Cr −−−→ M −−−−−−−−→ Ip ,→ mutation prompt.

(1)

Here Pr is the set of programs from run r, Cr is the set of cards extracted from that run, M is the persistent memory bank, and Ip is the small list of instructions selected for parent program p on task τ with metric description µ. Conceptually, the architecture has three parts. First, a post-run analysis stage reviews completed runs and produces structured records of candidate ideas together with provenance information. Second, a persistent memory layer stores these records, merges near-duplicates, and maintains retrieval indexes. Third, a retrieval layer queries this store at mutation time. In control runs, this layer is simply inactive, so the same evolutionary pipeline can be used both with and without memory. 3.2

Insights extraction and processing

The write phase begins from mutation records attached to evolved programs. During the run, the mutation process records a list of changes; each change has a short description and, when available, a rationale for why it was introduced. After the run, the analysis stage collects the completed programs, filters out root programs and invalid outputs, and converts the remaining candidates into a normalized set of records containing fitness, generation, parentage, task context, mutation strategy, code, and improvement descriptions. There are two analysis modes. The default mode compares new improvements against a working collection of previously extracted ideas. For each program, an LLM decides whether the recorded improvements correspond to genuinely new ideas, revisions of existing ideas, or simple reformulations of earlier descriptions. The working collection is then updated while preserving provenance: source program references are appended, the latest generation is updated, older descriptions are kept as aliases when wording changes, and new rationales are appended to the explanation history. The fast analyzer is used when a run contains many similar improvement descriptions. It first groups semantically related candidates using DBSCAN over embedding space, then asks an LLM to refine ambiguous multi-member clusters. This produces a smaller set of reusable idea candidates while preserving the source programs and rationales as provenance. Cross-run consolidation is deferred to the memory insertion stage, where new ideas are compared against the existing corpus. After analysis, the active memory is enriched with task summaries, keywords, and short explanation summaries. If memory-usage tracking is enabled, the system also uses the selected memory-card identities recorded on parent programs to estimate downstream impact. For a child program, the recorded contribution of a selected memory item is ∆f = f (child) −

max f (p),

p∈parents

computed only when both child and parent fitness values are valid, i.e., when both passed a task– specific validity check. The stored usage statistics keep per-task counts, all observed deltas, and the median delta. Median aggregation is used because evolutionary fitness changes are typically sparse and heavy-tailed. 3.3

Memory generation

The write pipeline stores two kinds of entries. The first represents an abstract idea or tactic, including its description, task context, explanations, provenance, and usage statistics. The second represents a high-performing program together with its fitness, code, task context, and links to the ideas associated with it. Before insertion, all incoming data are normalized into these structured forms. This normalization step is necessary because the write pipeline consumes heterogeneous inputs— LLM outputs, post-run analysis logs, and historical dumps—so missing fields and legacy aliases must be handled consistently. 4

Not every extracted idea is promoted into long-term memory with equal status. In addition to the full collection of extracted ideas, the system maintains a smaller snapshot of high-value ideas intended to capture the most reusable findings from a run. After the post-run analysis stage finishes, an origin-analysis pass scores each idea by both how it first appeared in the evolutionary lineage and how it shaped later descendants. For each introduction event, the analysis compares the child program with its strongest parent, with sibling programs generated from the same parent context, and with downstream descendants that inherit the idea. The promotion rule is deliberately conservative rather than tuned to a particular benchmark. An idea is favored when it has an identifiable introduction point, improves over its strongest available parent, rarely produces worse descendants, and is supported by at least one additional signal such as recurrence across nearby programs, favorable comparison with siblings generated from the same context, or spread into later elite lineages. This filter biases memory toward ideas with lineage-level evidence of usefulness, rather than toward every change described by the LLM. Once this curated set has been formed, the write pipeline combines the full idea collection, the filtered high-value snapshot, optional high-performing program exemplars, and usage updates. New records are then inserted into persistent memory under a conservative policy: 1. repeated observations of the same memory entity update the existing record and extend its provenance; 2. concrete program exemplars are stored without semantic merging, because they are tied to specific artifacts and fitness values; 3. abstract idea cards are merged only when they describe the same underlying mechanism, not merely the same source task. To identify possible duplicates, EvoMem compares each incoming idea against multiple textual views of existing cards, including their mechanism descriptions, task summaries, and explanations. Candidate scores are combined by a weighted sum, X score(c) = wq sq (c), q∈Q

where sq (c) is the retriever score for candidate card c under query view q. The default weights are largest on the description and description–explanation views, because these are the best signals for whether two cards describe the same mechanism rather than merely the same benchmark. The highest-scoring candidates are passed to an LLM decision policy that chooses whether to store the new idea, discard it as redundant, or merge it with an existing entry. Ambiguous cases are retained as new cards, favoring recall over accidental loss of useful knowledge. Deduplication is therefore an optimization of corpus quality, not a precondition for preserving a potentially useful idea. The resulting memory corpus is stored persistently together with provenance and usage statistics, and retrieval indexes are refreshed so that later runs can query the updated memory bank. 3.4

Memory retrieval and usage

Before each mutation, EvoMem optionally augments the candidate context with retrieved memory. In control runs this augmentation is empty; in memory-enabled runs, a retrieval module selects a bounded set of relevant memory cards. This keeps the evolutionary operators fixed across conditions: parent selection, mutation generation, validation, and scoring are unchanged, while memory enters only as an additional advice channel. The retrieval query is derived from the current task description, metric descriptions, mutation mode, parent program, and local mutation context. EvoMem then selects a small number of concise, actionable ideas. Thus, the intervention is localized: the current program state is used to retrieve advice, but program generation and evaluation remain unchanged. Before retrieving individual ideas, EvoMem performs an additional task-scope filtering step. An LLM is shown the current task together with the task descriptions present in memory and selects the source tasks that are potentially relevant. Subsequent retrieval is then performed over ideas associated with those selected tasks, rather than over the entire memory corpus. This keeps retrieval focused while still allowing transfer from related but non-identical problems. 5

Within the selected task scope, EvoMem ranks memory cards using lexical and embedding-based similarity over several semantic views: the card’s mechanism description, the source task context, the explanation of why the idea helped, and selected compositions of these fields. This decomposition is important because a useful memory card may match the present mutation through different signals: the mechanism it describes, the task constraints under which it was observed, or the explanation of why it improved performance. The selection policy merges evidence from multiple retrieval channels and may issue a focused follow-up query when the initial evidence is insufficient. Both the number of retrieval iterations and the number of returned cards are fixed in advance, so retrieval remains a bounded mutation-time operation rather than an open-ended research process. The retrieval step returns two coupled outputs: a concise advice block inserted into the mutation prompt, and the identities of the selected memory cards. These identities are recorded with the resulting candidate so that, after evaluation, the system can estimate associations between memory use and downstream fitness changes. This record is what enables the usage-tracking pass in the next write phase. Finally, the retrieved text is inserted into the mutation prompt as a dedicated memory section alongside metrics, recent insights, lineage, evolutionary statistics, and any task-specific artifact context. The mutation agent therefore sees memory as advice rather than as a hard constraint. It may reuse the retrieved tactics when they are relevant, while every candidate program is still evaluated by the same validators and metrics as in non-memory runs. 3.5

Design properties

Three design choices are especially important for experiments. First, memory is optional: control and treatment runs differ only in whether the memory advice channel is populated. Second, retrieval is bounded: mutation-time memory access is limited by fixed item and iteration budgets. Third, the system is auditable: selected memory cards are recorded with the candidates that used them, and post-run analysis can trace those links back to the relevant source programs. Together, these properties make it easier to attribute measured gains to the memory mechanism rather than to unrelated changes elsewhere in the pipeline. This design is intended to limit negative transfer. EvoMem does not replace the core evolutionary process with memory replay: parent selection, mutation generation, validation, evaluation, and population updates remain the same as in the baseline system. Memory enters only as a bounded suggestion channel inside the mutation context, and retrieved ideas are not enforced as constraints. As a result, the search can ignore irrelevant memories, continue proposing novel mutations, and select against candidates that overfit to stale or task-mismatched advice. The memory layer therefore aims to bias exploration toward previously useful directions without eliminating the discovery of new strategies.

4

Methodology

For memory-mechanism evaluation, we build memories from prior runs but exclude same-benchmark memories at test time. The memory-generation set spans geometry, question answering, and code optimization. It includes Circle Packing (26), Heilbronn-style point placement, Kissing Number (11D), HotpotQA [39], HoVer [12], GSM8K [2], selected AlgoTune tasks {QP, Markowitz, LP-Box, Chebyshev Center, Feedback Controller, LQR, Lyapunov} [29], and selected KernelBench kernels {L1 P56, L1 P82, L1 P86, L1 P87, L2 P57} [28]. Only part of this set is drawn from established benchmark suites, including AlgoTune for scientific-code tuning and KernelBench for GPU-kernel optimization; the remaining tasks cover geometric optimization and reasoning settings used to probe transfer more broadly. The evaluation set contains tasks from the same broad families but is treated as a separate testing set for cross-task memory reuse. It includes Circle Packing (26 and 32), Hexagon Packing, Heilbronn-style point placement, Kissing Number (12D), HoVer, HotpotQA, AlgoTune Power Control, AlgoTune Kalman, and KernelBench kernels including L1 P66 and L3 P21. Circle packing and hexagon packing ask the system to arrange shapes as densely as possible inside a bounded region. Heilbronn-style tasks optimize point placements by maximizing the minimum triangle area induced by triples of points. 6

Kissing Number evaluates dense high-dimensional sphere configurations. HotpotQA and HoVer test multi-hop question answering and evidence integration, with QA performance measured by exact match on a fixed test set of 200 samples for each benchmark. AlgoTune evaluation tasks measure speedups of scientific and algorithmic routines under correctness constraints, and KernelBench evaluation tasks measure low-level GPU-kernel optimization while preserving correctness. All components of the GigaEvo mutation framework used Gemini 3 Flash as the LLM backbone [6]. Prompt-evolution experiments with LLM-based evaluation or generation components, including HotpotQA, HoVer, and GSM8K, used Qwen3-8B as the tested LLM [38]. For reproducibility, we provide the full EvoMem implementation code with the submission. Unless otherwise noted, LLM calls used temperature 0.7, the MAP-Elites population used a single island, memory retrieval returned at most k = 3 cards per mutation, and embedding-based retrieval used the all-MiniLM-L6-v2 embedding model. Across the reported experiments, the evolution phase used on average 7,192,260 input tokens and 892,674 output tokens, while the post-run memory-generation phase used on average 683,470 input tokens and 25,225 output tokens. At the idea level, each memory bank session extracted 99.71 ± 38.68 generated ideas from one completed evolution run, and selected 12.76 ± 7.51 ideas for insertion into the memory bank after filtering and processing. We build the memory bank from the memory-generation set and then test EvoMem on the evaluation set. When evaluating a benchmark, we exclude all memories produced from that benchmark from the memory bank. Thus, the memory-enabled run can only retrieve memories from other benchmarks, including related tasks, and never from the benchmark currently being tested. For each evaluation benchmark, we compare a baseline condition without memory against the same evolutionary pipeline with retrieved memories enabled. The comparison measures whether reusable ideas extracted from prior tasks improve final performance or search speed on held-out tasks.

5

Results

We evaluate EvoMem on cross-problem transfer experiments spanning geometric optimization, multi-hop question answering, GPU kernel optimization, and scientific-code tuning. In each setting, we compare a baseline target-benchmark run against a memory-enabled run whose memory bank excludes memories from that target benchmark. The results are reported in Table 1. Figure 2 shows representative cases in which memory is used on different prompt and algorithmic optimization tasks. Search speedup is computed as the ratio between the number of evaluated candidates needed by the baseline run and the memory-enabled run to reach the baseline run’s final best score. Table 1: Summary of target-metric relative gains and search-speed ratios for memory-enabled runs across evaluated benchmarks. A speedup greater than 1 indicates that the memory-enabled run reached the baseline run’s final best score using fewer evaluated candidates; values below 1 indicate slower convergence. Relative gain is calculated separately for each matched pair of runs as gi = (Fimemory − Fibaseline )/|Fibaseline |. The reported average, minimum, and maximum gains summarize the resulting pairwise gi values. Benchmark

Average gain

Min gain

Max gain

Average speedup

Min speedup

Max speedup

Circle Packing (32) Circle Packing (26) Heilbronn Kissing Number (12D) Hexagon Packing HoVer AlgoTune KernelBench HotpotQA

5.21% 5.88% 3.86% 0.00% 3.69% 2.42% 7.90% 16.89% 11.79%

0.48% 0.28% 0.00% 0.00% 0.38% 2.28% 6.22% 0.00% 5.16%

9.29% 17.13% 8.55% 0.00% 6.93% 2.56% 9.57% 65.93% 27.41%

9.26 5.06 6.01 6.78 5.75 2.71 8.58 3.44 5.75

3.29 0.31 0.31 6.17 0.75 1.92 0.82 0.05 1.27

22.00 9.00 16.17 7.40 12.50 3.50 16.33 9.00 12.00

Average across benchmarks

6.40%

1.64%

16.37%

5.93

1.65

11.99

Table 1 shows that EvoMem is associated with better target metrics, faster search, or both across several benchmark families. The strongest speedups appear in Heilbronn, Kissing Number, HotpotQA, and Circle Packing, while AlgoTune and KernelBench show positive mean gains with higher variability. Kissing Number illustrates an acceleration-only case: final quality is unchanged on

7

(a) HoVer.

(b) Circle Packing n=32.

(c) Kalman filter.

(d) HotpotQA.

Figure 2: Representative transfer cases across reasoning and optimization benchmarks. Each panel compares baseline and memory-enabled behavior for a different task. average, but memory-enabled runs reach the baseline best score earlier. Absolute target-metric values are reported in Appendix D. 5.1

Examples of retrieved memory reuse

To make the effect of memory more concrete, Table 2 summarizes representative cases where retrieved memories were incorporated into successful mutations. These examples illustrate that the stored memories were not limited to copying task-specific code. Instead, they often expressed reusable execution strategies, search heuristics, or reasoning patterns that could be adapted to new target tasks. Across these cases, retrieved memories usually encoded transferable execution ideas—such as caching repeated structure, exploiting sparsity, reducing convergence checks, or limiting custom kernels to profitable fusion—rather than narrow task-specific tricks.

8

Table 2: Representative examples of memory ideas reused during evolution. Target task

Source or inferred source

Retrieved memory idea

How it was used

Circle Packing

Heilbronn-style point placement

Replaced the static grid with forcedirected motion and adaptive radius expansion.

Hexagon Packing

Circle/geometric optimization

HotpotQA

Multi-hop QA traces

HoVer

GSM8K / arithmetic reasoning

KernelBench L1 P66

Kernel traces

Power Control

Markowitz / QP-style optimization

Battery Scheduling

LP-Box / Chebyshev Center / QP-style optimization

Use annealed force-directed repulsion and low-discrepancy initialization to improve geometric dispersion. Use basin hopping, simulated annealing, and temperature-scaled repulsive forces to escape local packing optima. Extract only claim-relevant facts, preserve intermediate entities, and maintain query-refinement continuity across steps. Make latent relationships explicit and verify them with concrete checks rather than generic reasoning instructions. Avoid over-customized kernels when vendor primitives are faster; use custom Triton kernels only for profitable fusion. Precompute invariant matrix scaling, use in-place linear algebra, and reduce convergence-check overhead. Cache repeated constraint structure, use sparse matrices for structured LPs, tune HiGHS options, and avoid repeated conversions.

6

optimization

Added hex-lattice starts, Gaussian jitter, and boundary-aware relaxation. Added entity extraction and shorter final synthesis to reduce context drift. Added explicit reasoning questions, missing-fact lists, and targeted query examples. Used F.conv3d for Conv3D and kept Triton only for fused bias/ReLU. Precomputed scaled coefficients, reused buffers, and checked convergence less often. Cached horizon templates, used CSR matrices, tuned HiGHS, and delayed conversions.

Memory Quality and Utilization

The benchmark results show whether memory-enabled evolution improves an end metric, but they do not by themselves show how broadly the bank is queried or whether the retrieved advice is actually incorporated into generated code. We therefore complement the outcome analysis with two mechanism-level audits: memory bank coverage in the transfer-target experiments and an acceptance audit that compares the default retrieval mechanism with random memory selection. 6.1

Memory utilization across transfer targets

The memory-usage reports associate every generated program with the identifiers of the memory cards selected for its mutation. We count a bank idea as retrieved when its identifier occurs at least once in a candidate’s metadata and matches a card in the bank snapshot loaded for that experiment. This is a breadth measure and does not increase when the same card is retrieved repeatedly. Figure 3 shows substantial variation in how much of each bank was explored. The only benchmark with full memory bank coverage was HoVer. The completed Heilbronn, Circle Packing (26), and Hexagon Packing reports retrieved 81.3%, 69.9%, and 64.4% of their available bank ideas, respectively. HotpotQA retrieved 33.3%. The KernelBench coverage was 5.4%, while Kissing Number (12D) showed lowest coverage, at 3.3%. 6.2

Retrieval quality and idea acceptance

Coverage measures whether an idea was selected, but selection alone does not establish that the mutation agent followed the advice. We therefore compare EvoMem’s default relevance-based retrieval with random sampling from the same memory bank. Each selected program–idea pair was examined by an automated parent–child code audit. An idea was counted as accepted only when the child newly introduced or materially strengthened the selected idea relative to its parents. The acceptance rate is computed over decided checks, accepted/(accepted + not accepted); evaluator errors are excluded from this denominator.

9

Figure 3: Breadth of memory bank use in the completed transfer experiments for which usage reports were available. Each bar gives the percentage of the available memory bank retrieved at least once, with the two KernelBench targets grouped into one benchmark-family bar. This measures retrieval coverage, not retrieval frequency or implementation.

Figure 4: Idea-acceptance rates under default relevance-based retrieval and random memory selection. The pooled bars combine both audit groups. Rates use decided program–idea checks. Across the two retrieval-quality audit groups, default retrieval achieved a 4.05% acceptance rate. On the other hand, random memory achieved only 0.25%. Thus, on average, the retrieval mechanism outperforms random memory selection: a retrieved idea was 16.2 times as likely to be incorporated into the child program, an absolute increase of 3.80 percentage points. The comparison is not driven by one audit group: the rates were 6.23% versus 0.30% in the first group and 1.88% versus 0.20% in the second, and default retrieval had the higher rate in every matched comparison. Both conditions drew from the same memory bank.

10

The program-level view provides the same qualitative result. At least one retrieved idea was accepted in 136 of 1,442 audited programs (9.43%), compared with 10 of 1,451 programs (0.69%) under random memory, a 13.7-fold difference. Together with Figure 4, this shows that the retrieval layer does more than expose the mutation model to arbitrary prior ideas: its relevance filtering selects advice that is substantially more likely to be realized in code. This mechanism-level evidence complements, rather than replaces, the end-to-end fitness and search-speed comparisons above.

7

Conclusion

We introduced EvoMem, a persistent memory architecture for LLM-based evolutionary code optimization. EvoMem converts successful mutation events from previous runs into structured, auditable memory cards and retrieves a bounded set of relevant cards during later mutations. This lets an evolutionary search process reuse previously discovered tactics without replaying old programs directly or changing the evaluator, parent-selection policy, or fitness computation. Across the evaluated geometry, reasoning, scientific-code, and GPU-kernel benchmarks, EvoMem is associated with improved efficiency or final target metrics in most of the tested settings. The results suggest that some useful mutation knowledge is more general than the single program in which it first appeared: strategies discovered in one run can help guide exploration in related and sometimes substantially different domains. These gains provide initial support for the central hypothesis of this work, namely that LLM-driven evolution can benefit from persistent cross-run memory because it may reduce redundant rediscovery and bias search toward previously productive directions while preserving evaluator-based selection. More broadly, EvoMem suggests that memory should be treated as a first-class component of evolutionary coding systems. As evolutionary agents become more capable, the artifacts produced by a run include not only final code but also reusable design decisions, optimization heuristics, and failure-avoidance patterns. Capturing these artifacts in a compact and retrievable form may provide a practical path toward evolutionary systems that accumulate experience across tasks instead of starting each optimization process from scratch.

8

Limitations and Future Work

The benefits of EvoMem vary across runs, as reflected in the wide range between the minimum and maximum gains and speedups observed for several benchmarks. This variance is partly inherent to evolutionary search: useful exploration often requires high-temperature LLM sampling, so repeated runs can follow different trajectories. At the same time, too little diversity, or overly rigid use of retrieved memory, can cause premature convergence. EvoMem is also sensitive to the quality of its memory pipeline, including idea extraction, clustering, summarization, retrieval, and their associated hyperparameters. Poorly formed or poorly retrieved memories can reduce the relevance of mutation advice and limit downstream gains. The present study also uses a relatively small memory bank, a fixed memory influence during mutation, and a simple code-evolution setting. Future work should scale memory across more runs and domains, adapt memory reliance to the task and population state, and study whether persistent memory can reduce the cost of repeated LLM calls, execution, validation, and selection. Another important direction is repository-scale evolution, where memories may capture project conventions, interface constraints, recurring bug patterns, and cross-file optimization strategies. Because EvoMem is evaluated inside full evolutionary optimization loops, component-level ablations are substantially more expensive than ordinary prompt or retrieval ablations: each setting changes the search trajectory and requires a fresh set of evaluator-in-the-loop runs. We therefore treat the present experiments as an end-to-end evaluation of the memory mechanism rather than a complete decomposition of all design choices.

References [1] Henrique Assumpcao, Diego Ferreira, Leandro Augusto Lacerda Campos, and Fabricio Murai. CodeEvolve: an open source evolutionary coding agent for algorithmic discovery and optimization, 2025. 11

[2] Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems, 2021. [3] Steffen Eger, Yong Cao, Jennifer D’Souza, Andreas Geiger, Christian Greisinger, Stephanie Gross, Yufang Hou, Brigitte Krenn, Anne Lauscher, Yizhi Li, Chenghua Lin, Nafise Sadat Moosavi, Wei Zhao, and Tristan Miller. Transforming science with large language models: A survey on AI-assisted scientific discovery, experimentation, content generation, and evaluation, 2025. [4] Maksim Elistratov, Marina Barannikov, Gregory Ivanov, Valentin Khrulkov, Anton Konushin, Andrey Kuznetsov, and Dmitrii Zhemchuzhnikov. CADEvolve: Creating realistic CAD via program evolution, 2026. [5] Daniil Fisher, Valentin Khrulkov, Mikhail Saygin, Ivan Oseledets, and Stanislav Straupe. LLMGuided evolutionary search for algebraic T-Count optimization, 2026. [6] Gemini Team. Gemini 3 Flash: frontier intelligence built for speed. Google Keyword Blog, 2025. Accessed 2026-05-06. [7] Bogdan Georgiev, Javier Gómez-Serrano, Terence Tao, and Adam Zsolt Wagner. Mathematical exploration and discovery at scale. ArXiv, abs/2511.02864, 2025. [8] Alireza Ghafarollahi and Markus J. Buehler. ProtAgents: Protein discovery via large language model multi-agent collaborations combining physics and machine learning. Digital Discovery, 3(7):1389–1409, 2024. [9] Md Musaddaqul Hasib, Sumin Jo, Harsh Sinha, Jifeng Song, Arun Das, Zhentao Liu, Hugh Galloway, Huey Huang, Kexun Zhang, Shou-Jiang Gao, Yu-Chiao Chiu, Lei Li, and Yufei Huang. A process-centric survey of AI for scientific discovery through the exhyte framework, 2025. [10] Peter Jansen, Oyvind Tafjord, Marissa Radensky, Pao Siangliulue, Tom Hope, Bhavana Dalvi Mishra, Bodhisattwa Prasad Majumder, Daniel S. Weld, and Peter Clark. CodeScientist: Endto-end semi-automated scientific discovery with code-based experimentation. In Findings of the Association for Computational Linguistics: ACL 2025, pages 13370–13467, Vienna, Austria, 2025. Association for Computational Linguistics. [11] Qile Jiang and George Karniadakis. AgenticSciML: Collaborative multi-agent systems for emergent discovery in scientific machine learning, 2025. [12] Yichen Jiang, Shikha Bordia, Zheng Zhong, Charles Dognin, Maneesh Singh, and Mohit Bansal. HoVer: A dataset for many-hop fact extraction and claim verification. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 3441–3460, Online, 2020. Association for Computational Linguistics. [13] Valentin Khrulkov, Andrey Galichin, Denis Bashkirov, Dmitry Vinichenko, Oleg Travkin, Roman Alferov, Andrey Kuznetsov, and Ivan Oseledets. GigaEvo: An open source optimization framework powered by llms and evolution algorithms, 2025. [14] János Kramár, Joshua Engels, Zheng Wang, Bilal Chughtai, Rohin Shah, Neel Nanda, and Arthur Conmy. Building production-ready probes for Gemini. ArXiv, abs/2601.11516, 2026. [15] Alexey Kravatskiy, Valentin Khrulkov, and Ivan Oseledets. ImprovEvolve: Ask AlphaEvolve to improve the input solution and then improvise, 2026. [16] Robert Tjarko Lange, Yuki Imajuku, and Edoardo Cetin. ShinkaEvolve: Towards open-ended and sample-efficient program evolution, 2025. [17] Annan Li, Chufan Wu, Zengle Ge, Yee Hin Chong, Zhinan Hou, Lizhe Cao, Cheng Ju, Jianmin Wu, Huaiming Li, Haobo Zhang, et al. The FM agent. arXiv preprint arXiv:2510.26144, 2025. [18] Ruochen Li, Teerth Patel, Qingyun Wang, and Xinya Du. MLR-Copilot: Autonomous machine learning research based on large language models agents, 2024. 12

[19] Zongwei Li, Zhonghang Li, Zirui Guo, Xubin Ren, and Chao Huang. DeepCode: Open agentic coding, 2025. [20] Zun Li, John Schultz, Daniel Hennes, and Marc Lanctot. Discovering multiagent learning algorithms with large language models. ArXiv, abs/2602.16928, 2026. [21] Zeming Lin, Halil Akin, Roshan Rao, Brian Hie, Zhongkai Zhu, Wenting Lu, Nikita Smetanin, Robert Verkuil, Ori Kabeli, Yaniv Shmueli, Allan dos Santos Costa, Maryam Fazel-Zarandi, Tom Sercu, Salvatore Candido, and Alexander Rives. Evolutionary-scale prediction of atomiclevel protein structure with a language model. Science, 379(6637):1123–1130, 2023. [22] Qi Liu, Ruochen Hao, Can Li, and Wanjing Ma. OR-Agent: Bridging evolutionary search and structured research for automated algorithm discovery, 2026. [23] Shu Liu, S. Agarwal, M. Maheswaran, M. Cemri, Zhifei Li, Qiuyang Mang, Ashwin Naren, E. Boneh, Audrey Cheng, Melissa Z. Pan, Alexander Du, Kurt Keutzer, Alexandros G. Dimakis, Koushik Sen, Matei A. Zaharia, and Ion Stoica. EvoX: Meta-evolution for automated discovery. ArXiv, abs/2602.23413, 2026. [24] Chris Lu, Cong Lu, Robert Tjarko Lange, Jakob Foerster, Jeff Clune, and David Ha. The AI scientist: Towards fully automated open-ended scientific discovery, 2024. [25] Peng Luo, Xiayin Lou, Yu Zheng, Zhuo Zheng, and Stefano Ermon. GeoEvolve: Automating geospatial model discovery via multi-agent large language models, 2025. [26] Ansh Nagda, Prabhakar Raghavan, and Abhradeep Thakurta. Reinforced generation of combinatorial structures: Ramsey numbers, 2026. [27] Alexander Novikov, Ngân Vũ, Marvin Eisenberger, Emilien Dupont, Po-Sen Huang, Adam Zsolt Wagner, Sergey Shirobokov, Borislav Kozlovskii, Francisco J. R. Ruiz, Abbas Mehrabian, M. Pawan Kumar, Abigail See, Swarat Chaudhuri, George Holland, Alex Davies, Sebastian Nowozin, Pushmeet Kohli, and Matej Balog. AlphaEvolve: A coding agent for scientific and algorithmic discovery, 2025. [28] Anne Ouyang, Simon Guo, Simran Arora, Alex L. Zhang, William Hu, Christopher Re, and Azalia Mirhoseini. KernelBench: Can LLMs write efficient GPU kernels? In Proceedings of the 42nd International Conference on Machine Learning, volume 267 of Proceedings of Machine Learning Research, pages 47356–47415. PMLR, 2025. [29] Ori Press, Brandon Amos, Haoyu Zhao, Yikai Wu, Samuel K. Ainsworth, Dominik Krupke, Patrick Kidger, Touqir Sajed, Bartolomeo Stellato, Jisun Park, Nathanael Bosch, Eli Meril, Albert Steppi, Arman Zharmagambetov, Fangzhao Zhang, David Perez-Pineiro, Alberto Mercurio, Ni Zhan, Talor Abramovich, Kilian Lieret, Hanlin Zhang, Shirley Huang, Matthias Bethge, and Ofir Press. AlgoTune: Can language models speed up general-purpose numerical programs?, 2025. [30] Jiyong Rao, Yicheng Qiu, Jiahui Zhang, Juntao Deng, Shangquan Sun, Fenghua Ling, Hao Chen, Nanqing Dong, Zhangyang Gao, Siqi Sun, Yuqiang Li, Dongzhan Zhou, Guangyu Wang, Lijun Wu, Conghui He, Xuhong Wang, Jing Shao, Xiang Liu, Yu Zhu, Mianxin Liu, Qihao Zheng, Yinghui Zhang, et al. SciDataCopilot: An agentic data preparation framework for AGI-driven scientific discovery, 2026. [31] Shuo Ren, Pu Jian, Zhenjiang Ren, Chunlin Leng, Can Xie, and Jiajun Zhang. Towards scientific intelligence: A survey of LLM-based scientific agents, 2025. [32] Minju Seo, Jinheon Baek, Seongyun Lee, and Sung Ju Hwang. Paper2Code: Automating code generation from scientific papers in machine learning, 2025. [33] Alex Vitvitskyi, Michael Boratko, Matej Grcic, Razvan Pascanu, Deep Shah, and Petar Veličković. Mining generalizable activation functions. ArXiv, abs/2602.05688, 2026. [34] Yiping Wang, Shao-Rong Su, Zhiyuan Zeng, E. Xu, Liliang Ren, Xinyu Yang, Zeying Huang, Xuehai He, Luyao Ma, Baolin Peng, Hao Cheng, Pengcheng He, Weizhu Chen, Shuohang Wang, S. Du, and Yelong Shen. ThetaEvolve: Test-time learning on open problems. ArXiv, abs/2511.23473, 2025. 13

[35] Yihang Xiao, Jinyi Liu, Yan Zheng, Xiaohan Xie, Jianye Hao, Mingzhi Li, Ruitao Wang, Fei Ni, Yuxiao Li, Jintian Luo, Shaoqing Jiao, and Jiajie Peng. CellAgent: An LLM-driven multi-agent framework for automated single-cell data analysis, 2024. [36] Wujiang Xu, Zujie Liang, Kai Mei, Hang Gao, Juntao Tan, and Yongfeng Zhang. A-MEM: Agentic memory for LLM agents, 2025. [37] B. Y. Yan, Chaofan Li, Hongjin Qian, Shuqi Lu, and Zheng Liu. General agentic memory via deep research, 2025. [38] An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, et al. Qwen3 technical report, 2025. [39] Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. HotpotQA: A dataset for diverse, explainable multi-hop question answering. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2369–2380, Brussels, Belgium, 2018. Association for Computational Linguistics. [40] Lianhao Zhou, Hongyi Ling, Keqiang Yan, Kaiji Zhao, Xiaoning Qian, Raymundo Arroyave, Xiaofeng Qian, and Shuiwang Ji. Toward greater autonomy in materials discovery agents: Unifying planning, physics, and scientists, 2025.

14

A

Ablation Studies

We conduct a controlled ablation study on Circle Packing (32) to separate the effect of the complete EvoMem mechanism from simpler forms of prompt and memory augmentation. Circle packing provides a well-understood optimization objective with a known reference solution. Because each condition requires a fresh evaluator-in-the-loop evolutionary run, we allocate the experimental budget to five independent trials per condition on this task rather than to fewer trials spread across several benchmarks. We compare the full default EvoMem configuration with no memory, random memory, generic advice, memory without task filtering, and retrieval-only memory without the complete re-ranking procedure. Near-optimal improvements are difficult to interpret from raw fitness alone: scores around 2.8 are comparatively easy to reach, whereas small absolute gains become increasingly consequential near the best-known fitness f ∗ = 2.9370. We therefore also report the percentage of the remaining target gap closed. For a fitness value f , this quantity is GapClosed(f ) = 100

f − f¯base , f ∗ − f¯base

f¯base = 2.8151.

(2)

Under this normalization, the no-memory mean is 0% and the best-known fitness is 100%. Standard deviations are transformed by the same fixed scale; consequently, the baseline can have zero mean gap closure while retaining a nonzero standard deviation. Table 3: Circle Packing (32) ablation results over five independent trials per condition. Fitness and target-gap closure are reported as mean ± standard deviation; best and worst denote the extrema across the five trials. Ablation runs use an independent set of trials and are not included in the data reported in Table 1. Condition Baseline (no memory) Random memory Generic advice No task filtering Retrieval-only memory Full EvoMem

Fitness (mean ± SD)

Best run

Worst run

Target gap closed (mean ± SD)

2.8151 ± 0.0683 2.8219 ± 0.0681 2.8805 ± 0.0530 2.8714 ± 0.1148 2.8379 ± 0.2210 2.9338 ± 0.0044

2.9061 2.8900 2.9228 2.9352 2.9370 2.9370

2.7388 2.7538 2.7877 2.6683 2.4426 2.9278

0.0% ± 56.0% 5.6% ± 55.9% 53.7% ± 43.5% 46.2% ± 94.2% 18.7% ± 181.3% 97.4% ± 3.6%

Table 3 shows that arbitrary memory content or retrieval alone does not reliably reproduce EvoMem’s gains. Random memory closes only 5.6% of the target gap on average, and retrieval-only memory closes 18.7% while exhibiting the largest variance; the one run that reaches the best-known fitness is therefore not representative of its overall performance. Generic advice and removing task filtering perform better on average, closing 53.7% and 46.2% of the gap, respectively, but both remain below the full method, and the no-task-filtering condition is particularly unstable. In contrast, full EvoMem attains a mean fitness of 2.9338 ± 0.0044 and closes 97.4% ± 3.6% of the target gap. Even its worst run, at 2.9278, exceeds the no-memory baseline mean of 2.8151. The simultaneous improvement in average fitness and run-to-run consistency indicates that the strongest and most reliable gains arise from the complete mechanism—constructing reusable knowledge, selecting task-relevant memories, and integrating them into the evolutionary process—rather than from additional context or retrieval machinery alone.

B

Lineage dynamics of memory use

The run logs also allow a finer-grained observational analysis inside memory-enabled evolution. For each candidate, we mark whether the mutation directly used retrieved memories and whether the candidate belongs to a descendant lineage of any previous memory-use event. We then compare these groups within the logs that contain both memory and non-memory post-root mutations. Because the underlying tasks use different objective scales, the analysis reports within-run objective ranks among valid post-root programs, where larger ranks are better. These statistics are associative rather than randomized causal estimates, but they test whether memory-guided variants are preferentially retained by the evolutionary process. 15

Table 4: Lineage-level dynamics of retrieved memory usage. Objective values are converted to withinrun ranks among valid post-root programs, where 1 is the best program in that run. Comparisons are made within the evolution logs between memory-associated programs and contemporaneous non-memory programs. Statistic

Memory-associated

Comparator

Effect

47.9% 0.87 2.57 0.504 0.997

40.2% 0.73 2.21 0.429 0.667

+7.7 pp 1.20× 1.16× matched +0.075 +0.330

Valid direct mutations Direct children per mutation Descendants per mutation Mean objective rank of valid lineage members Best lineage objective rank

Table 4 reports that direct memory-use mutations are more likely to produce valid candidates and have more reproductive output than mutations without retrieved memory. The descendant view is also positive: programs in memory-descended lineages have higher average objective rank, and the strongest memory-descended lineages reach higher average best objective ranks than the strongest non-memory lineages. This is consistent with the intended role of EvoMem as a soft evolutionary bias: retrieved advice can be used by mutations whose descendants remain competitive under selection.

16

C

Memory Card Schema and Algorithms

C.1

Memory card schema

The schema below summarizes the normalized fields used by EvoMem memory cards. It is designed to keep each stored item both actionable at mutation time and auditable after a run. Not every field is required for every card type: abstract idea cards emphasize reusable tactics and explanations, while program cards additionally store concrete code artifacts and fitness values. Memory-card schema. Field

Typical type

Purpose

card_id

string

card_type

enum

description

text

task_summary

text

metric_context

text

explanations

list of text

keywords source_program_ids

list of strings list of strings

provenance

structured record

aliases

list of text

links

list of strings

usage_stats

structured record

retrieval_views

structured text

code

optional text

fitness

optional number

Stable identifier used for retrieval, merging, and usage tracking. Distinguishes abstract idea cards from concrete program cards. Concise statement of the reusable tactic or program-level finding. Short description of the source task, constraints, and objective. Description of the metric or fitness signal under which the card was observed. Rationales, mechanism descriptions, and explanation summaries collected during analysis. Search terms used for exact and hybrid retrieval. Program identifiers from which the idea or artifact was extracted. Run identifier, generation, parentage, introduction events, and lineage evidence. Earlier or alternative phrasings preserved during deduplication and merging. Optional links from an idea card to associated highperforming program cards. Counts, observed downstream fitness deltas, pertask summaries, and median delta. Indexed views such as description, task summary, explanation summary, and their compositions. Concrete evolved program body, present only for program cards. Fitness value associated with a stored program card.

C.2

Write-phase pseudocode

Algorithm 1: WriteMemory. 1. Input: completed run trace R, existing memory bank M . 2. Collect completed programs from R and remove root programs, invalid outputs, and entries without usable fitness or mutation metadata. 3. Normalize each remaining program into a candidate record containing fitness, generation, parentage, task context, mutation descriptions, rationales, and code. 4. Extract candidate ideas from the normalized records using either the default analyzer or the fast clustering analyzer. 5. Enrich extracted ideas with task summaries, keywords, explanation summaries, aliases, and source-program identifiers. 6. Estimate observational usage statistics for previously selected memories by comparing each child fitness with the strongest parent fitness. 7. Run origin analysis to identify high-value ideas supported by introduction gains, sibling comparisons, descendant spread, and downside-rate checks. 8. Normalize abstract idea cards and optional high-performing program cards into the memory-card schema.

17

9. For each normalized card: (a) If its identifier already exists in M , update the existing entity and merge provenance and usage statistics. (b) Else if the card is a concrete program, insert it directly. (c) Else retrieve near-neighbor idea cards using multiple textual views and combine candidate scores. (d) Ask the deduplication policy to store, discard, or merge the incoming idea. (e) If the policy fails, store the incoming idea as a new card. 10. Refresh retrieval indexes and persist the updated memory bank. 11. Output: updated memory bank M with write statistics and refreshed indexes.

C.3

Read-phase pseudocode

Algorithm 2: RetrieveMemory. 1. Input: parent program p, task description τ , metric description µ, mutation context x, memory bank M . 2. Validate the current program state and construct a structured memory request from (p, τ, µ, x). 3. If memory is disabled, return an empty memory section and an empty identifier list. 4. Initialize the bounded selector loop with the request, the available memory abstracts, and the maximum number of retrieval iterations. 5. While the selector has remaining iterations: (a) Plan one or more retrieval actions using exact keyword search, direct identifier lookup, or field-aware vector search. (b) Execute each retrieval action with the configured top-k limit. (c) Merge hits by card identifier and keep the strongest evidence for each card. (d) Reflect on the collected evidence and either stop with a final selection or issue a focused follow-up query. 6. Convert the selected cards into concise mutation advice and truncate the resulting text to the prompt budget. 7. Attach the selected card identifiers to the parent program metadata for later usage tracking. 8. Insert the memory text into the mutation prompt as advice, leaving generation, validation, and scoring unchanged. 9. Output: mutation prompt with a bounded memory section and selected memory identifiers.

D

Benchmarks Absolute Values

For completeness, Table 5 reports the mean final target-metric values, on their original task-specific scales, across completed matched baseline–memory run pairs for each benchmark family summarized in the main text. Relative gains reported in Table 1 are calculated separately for each matched baseline–memory run pair and then averaged. Consequently, the mean relative gain need not equal the relative gain computed directly from the two mean values reported here. Table 5: Absolute target-metric values for baseline and memory-enabled runs across benchmark families. Higher values indicate better performance for the reported task-specific target metric. Benchmark Target metric baseline Target metric with memory Circle Packing (32) Circle Packing (26) Heilbronn Kissing Number (12D) Hexagon Packing HoVer AlgoTune KernelBench HotpotQA

2.7870 2.4218 0.0304 840.0000 -4.3424 0.7900 461.5126 2.4941 0.5389

18

2.9289 2.5577 0.0315 840.0000 -4.1799 0.8080 495.3686 2.7049 0.6015

E

Evolution tree and ideas visualization

Figure 5: Top: visualization of extracted ideas on a two-dimensional plane obtained by applying t-SNE to embeddings of their textual descriptions. Bottom: evolution tree showing how programs branch and improve over the course of search.

F

Experimental Parameters • Population size: MAP-Elites island archive maximum size 75; initial population from all *.py files in each problem’s initial_programs/ directory; 5 elites per generation; 8 mutants per generation / epoch; at most 8 steady-state in-flight mutants. • Evaluation budget: no fixed total evaluation budget by default; max_generations is null / unlimited by default, with most experiments using 30; per-generation mutant budget is 8, so the approximate mutant budget is max_generations ×8 plus initial and archive-refresh evaluations; optimization_time_budget is 0.75× dag_timeout. • Memory top-k: MemoryProvider uses max_cards=3; memory runtime search_limit=5; Memory ResearchAgent allowed tools are page_index and vector; Memory top_k_by_tool uses keyword=5, vector=3, vector_description=3, vector_task_description ≥ 1, vector_explanation_summary=3, vector_description_explanation_summary=3, vector_description_task_description_summary=3, and page_index=5; card update deduplication retrieval uses top_k_per_query=10, final_top_n=10, and min_final_score=0.05. • Retrieval iterations: MemoryResearchAgent uses max_iters=3.

19

• Embeddings and clustering: IdeaTracker uses all-MiniLM-L6-v2; memory uses embeddings-model all-MiniLM-L6-v2; DBSCAN uses min_samples_for_dbscan=4, eps=0.25, and min_samples=2; the fast analyzer uses batch_size=32, max_attempts=20, max_rounds=100, recompute_center=false, refine_subgroup_size=10000, and llm_max_concurrent=100. • Models and API: main mutation, insights, and lineage model is Gemini3Flash; ideas-tracker analyzer model is Gemini3Flash; API base URL is https://openrouter.ai/api/v1; LLM defaults are temperature=0.7, max_tokens=81920, top_p=0.95, and top_k=20; many chain/prompt evaluations use Qwen/Qwen3-8B through local OpenAI-compatible endpoints. • Hardware: NVIDIA A100-SXM4-80GB, driver 560.35.03; Intel Xeon Platinum 8358, 128 logical CPUs; Linux 5.15.0-1044-nvidia. • Runtime: stage_timeout=2400 s default; dag_timeout=7200 s default; dag_concurrency=16; max_concurrent_dags=10; LLM request timeout 3200 s for all phases; stage and DAG timeouts are 7200 s for phase 1 and 18000 s for phases 2–3; generation timeout is 10800 s for phase 1 and 14400 s for phases 2–3. • Dataset splits and subsets: no single universal dataset split; prompt tasks default load_context n_samples=300; HotpotQA chain defaults to the first 300 training samples, full validation uses n_samples=200, and optional train/validation and pool/held-out-validation splits use seed=42; GSM8K chain defaults to the first 200 training samples; test helpers commonly default to n_samples=3 unless overridden.

20

Related documents

• Beyond Repository Boundaries: Cross-Repository Graph Retrieval for Code Generation#673596
• Adaptive Critical Token-Aware Retrieval for Repository-Level Code Generation#627717
• DecompRL: Solving Harder Problems by Learning Modular Code Generation#332523
• WiseSpec: Requirements-Driven Agents for Code Generation#627759
• Contextualized Code Pretraining for Code Generation#200551