InsightEmb: Learning Action-Intent Embeddings for Agentic Insight Retrieval

InsightEmb: Learning Action-Intent Embeddings for Agentic Insight Retrieval Tsz Ting Chung1 Jiangnan Li2 Jie Zhou2 Mo Yu2,† 1 The Hong Kong University of Science and Technology 2 WeChat AI, Tencent

arXiv:2608.04761v1 [cs.CL] 5 Aug 2026

Abstract Self-improving agents accumulate reusable insights from prior trajectories, making retrieval increasingly important for turning accumulated experience into actionable guidance. At each decision step, retrieving the right insight can help the agent progress toward its goal, a setting we refer to as agentic insight retrieval. However, existing retrieval methods primarily model semantic similarity, while overlooking whether a retrieved insight resolves the agent’s current decision bottleneck. We propose InsightEmb, a contrastive embedding framework that learns transferable progressoriented retrieval geometry using only mathematical reasoning data. InsightEmb jointly learns to align concrete situations with abstract heuristic rules and to cluster reasoning trajectories with similar progress structures. We evaluate InsightEmb on dynamic agent tasks and a static skill-retrieval benchmark. Without any environment-specific training, InsightEmb improves over all these evaluations, surpassing the performance of existing reasoning embedding models. These results suggest that the geometry of state-insight matching can transfer across domains, enabling effective training from publicly available reasoning data without expensive environment-specific supervision.

1

Introduction

LLM-based agents in interactive environments (for example, web navigation (Yao et al., 2022), embodied tasks (Shridhar et al., 2021), and tool use (Schick et al., 2023)) interleave reasoning and action through frameworks such as ReAct (Yao et al., 2023). They must select actions under large action spaces where the optimal strategy depends on abstract reasoning rather than surfacelevel pattern matching. Providing agents with insights, abstract rules distilled from past experience (e.g., “check likely locations before exploring randomly”), has proven effective (Majumder et al.,

Figure 1: InsightEmb retrieves insights that resolve the agent’s current procedural bottleneck rather than topical overlap alone. Left: the abstraction gap and shared situation-to-insight structure across math and embodied tasks. Right: on an ALFWorld “hot potato” task, Base favors heating rules before the object is found, whereas InsightEmb retrieves search-first guidance.

2023; Wang et al., 2024; Zhao et al., 2024), but the agent must dynamically retrieve the most pertinent insight at each step. We focus on this underspecified retrieval problem: the relevant insight is the one that is operationally useful now, given the agent’s current state, goal, and action history. This dynamic retrieval poses a challenge we call the abstraction gap: the query (the agent’s current observation and action history) and the target (an abstract insight) live at different levels of abstraction, and relevance depends on the agent’s next-step intent (what bottleneck must be resolved before progress) rather than topical similarity. Standard embedding models match surface semantics, so off-the-shelf retrievers return topically related but procedurally premature insights, e.g. state-transformation rules before the agent has located the target object (Figure 1). Our key observation is that this situationto-insight matching problem is not domainspecific: both a math query (e.g. matching “P (at least one red)” to complementary counting, with no lexical overlap) and an embodied task (first resolving the locate-object bottleneck before heat-

ing or placement insights apply) require inferring the latent bottleneck of a concrete situation and retrieving the abstract rule that enables the next effective step. We propose InsightEmb, a contrastive training framework that exploits this domain-agnostic structure. InsightEmb trains an embedding model in two stages, entirely on publicly available mathematical reasoning data: (1) Situation-to-insight matching teaches the model to bridge the abstraction gap by mapping math problems and (partial) chain-ofthought trajectories to their relevant heuristic rules. (2) Situation-to-experience matching teaches structural similarity recognition between reasoning trajectories, reinforcing the model’s ability to identify when different-looking situations require the same underlying strategy. At inference time, the trained model retrieves insights for LLM agents in ALFWorld (Shridhar et al., 2021), WebShop (Yao et al., 2022), and ScienceWorld (Wang et al., 2022b), and we use SRA-Bench as a static skill-retrieval diagnostic. These evaluations test dynamic stateconditioned retrieval and static skill applicability without any environment-specific training data. Our contributions are threefold: • We formulate agentic insight retrieval as goalconditioned action-intent matching. Relevance is defined by whether an insight resolves the current bottleneck and enables progress, rather than by semantic similarity alone. • We introduce InsightEmb, a cross-domain contrastive training framework for action-oriented retrieval geometry with empirical validation. • Our empirical results across static and dynamic evaluations show the effectiveness of InsightEmb with fewer steps taken. Analyses reinforce the gains come from action-intended retrieval.

2

Related Work

Retrieval-Augmented LLM Agents. RAG (Lewis et al., 2020; Guu et al., 2020) has been extended to agents that retrieve past experiences (Shinn et al., 2023; Majumder et al., 2023; Packer et al., 2023), tool docs (Qin et al., 2024), and skills (Wang et al., 2024). These systems target concrete artifacts where surface similarity often suffices, whereas we retrieve abstract heuristic rules with little lexical overlap, where standard retrievers fail and off-the-shelf or in-domain embedders miss the agentic objective. A complementary line manages the retrieved context

itself, e.g. prompt compression (Chung et al., 2024) and long-context activation approximation (Li et al., 2026b), which is orthogonal to which insight to retrieve. Experience-Based Agent Learning. Reflexion (Shinn et al., 2023), CLIN (Majumder et al., 2023), and Voyager (Wang et al., 2024) accumulate episodic memory, causal abstractions, or skills, while ExpeL (Zhao et al., 2024) and AutoGuide (Fu et al., 2024) distill trajectories into text rules without weight updates. They focus on what to store, whereas we address when to retrieve the right stored knowledge and are agnostic to how insights are produced. The value of surfacing the right prior experience is further underscored by in-context learning at scale, where retrieved demonstrations drive learning gains (Chung et al., 2026). Reasoning-Oriented Dense Retrieval. Dense retrievers (Karpukhin et al., 2020; Xiong et al., 2021; Wang et al., 2022a; Su et al., 2023; Xiao et al., 2023) and benchmarks such as MTEB (Muennighoff et al., 2023) and BRIGHT (Su et al., 2025) show that strong semantic matching still leaves reasoning-intensive retrieval far from solved. ReasonIR (Shao et al., 2025), Llama-NV-EmbedReasoning (NVIDIA, 2026) (top open-source single-model embedder on BRIGHT), and ReasonEmbed (Chen et al., 2025) train contrastive embedders for static query–passage support. Other work makes retrieval context-aware through embedding models (Wu et al., 2025c,b) and memoryaware reranking (Li et al., 2025, 2026a), but relevance there is still defined by matching a fixed query, not by whether a retrieved item advances an agent toward its goal. We instead target next-step utility under an evolving agent state: whether an insight resolves the current bottleneck, not merely whether it entails a fixed query. Agentic Benchmarks. ALFWorld (Shridhar et al., 2021), WebShop (Yao et al., 2022), ScienceWorld (Wang et al., 2022b), Mind2Web (Deng et al., 2023), and WebArena (Zhou et al., 2024) evaluate embodied, scientific, and web control with state-conditioned queries, and AgentBench (Liu et al., 2024) spans additional interactive environments. Prior evaluation efforts largely fall into two separate camps: retrieval over long contexts (Yu et al., 2025a) and reasoning or understanding benchmarks (Yu et al., 2025b; Chung et al., 2025; Wu et al., 2025a). A benchmark that jointly tar-

gets retrieval for reasoning and action guidance is largely missing, BRIGHT (Su et al., 2025) and SRA-Bench (Su et al., 2026) begin to address this gap, the latter testing static task-to-skill retrieval over mixed gold and distractor skills and complementing dynamic agent benchmarks by isolating retrieval from online execution.

3

Method

3.1

Problem Setting

An LLM agent operates in an interactive environment where, at each step t, it observes a state st (task description, action history, current observation) and must select an action at toward a goal g, following a reasoning-and-acting loop (Yao et al., 2023). The agent has access to an insight corpus I = {I1 , . . . , IN }, where each insight Ii is a natural-language description of abstract rules and strategies. We view this task as goal-conditioned abductive retrieval: given the current state and goal, retrieve the insight that best identifies which intervention would make progress possible. Our notion of relevance is progress-oriented: an insight is useful if it is expected to reduce the gap to the goal, not merely if it is semantically similar to the query. We express this target as, I ∗ = arg max E [∆(st , g) | agent uses I] , (1) I∈I

where ∆(st , g) measures progress toward goal g from state st . Equation (1) is not the training objective, and we do not estimate E[∆ | I] inside the contrastive loss. Instead, it guides how we curate supervision: for each training problem we run five rollouts, distill candidate insights from random trajectory subsets, and assign an insight to I + for problem q only if prepending it to q’s query improves solve rate over a no-insight baseline on that same problem (and to I − otherwise). See Appendix A.2. At inference, we deploy an embedding retriever that approximates the same criterion by similarity:
I ∗ = arg max sim fθ (st ), fθ (I) , (2) I∈I

with fθ trained by InfoNCE on the curated (q, I + , I − ) pairs (§3.3). The core difficulty is the abstraction gap: st contains concrete details (specific objects, locations, actions) while I ∗ contains abstract rules (general strategies, heuristics). For agentic tasks, this gap is also temporal: the relevant insight is the one that

Component

Math

Agentic task

Situation

Problem statement

Task + observation

Trajectory

Solution steps

Action history

Insight

Solving heuristic

Execution strategy

Abstraction gap

“at least one red ball” ↔ “complementary counting”

“You clean butterknife” “place”

the ↔

Table 1: Structural analogy between math reasoning and agentic execution, our design claim for building action-intent embeddings from math-only contrastive training.

addresses the agent’s current bottleneck, not necessarily the one most topically related to the final task goal. For example, before the target object is found, a search insight is more useful than an insight about how to transform or place that object. We thus define action-intent not as a supervised action label but as a progress-conditioned relevance signal tied to Equation (1): Definition (Action-intent embedding). For a situation s = (g, h, o) (goal g, action history h, observation o), the action-intent of s is the latent bottleneck that must be resolved before progress toward g. An insight I is action-intent–relevant to s if it raises expected progress E[∆(s, g) | I], and an action-intent embedding fθ ranks insights by sim(fθ (s), fθ (I)) under this progressconditioned relevance rather than topical similarity. We address this gap through a two-stage contrastive curriculum trained entirely on mathematical reasoning data (Hendrycks et al., 2021; Wei et al., 2022). Our training design rests on a single claim: math heuristic retrieval and agentic insight retrieval instantiate the same action-intent matching problem as illustrated in Table 1. 3.2

Structural Analogy for Action-Intent Retrieval

We propose that an agentic action-intent embedding can be learned from math-only data because both settings share the same retrieval structure: a concrete situation must be matched to the abstract rule that resolves the current bottleneck, not merely to topically related text. Under this parallel, contrastive training on math should teach a domain-agnostic geometric property:

embed a situation near the abstract rule that makes the next useful step apparent. The instruction prefix at inference (§3.5) frames queries in domainneutral terms so this property can activate on the target agentic environments without environmentspecific fine-tuning. The experiments below test whether this claim holds in practice. Shared bottleneck categories. What transfers is not a shared “step” format but a situation → bottleneck-resolving-rule geometry organized by a small set of recurring bottleneck types that both domains instantiate. These categories are never specified in the pipeline: the distillation prompts (Appendix A.2.2) ask only for general, task-agnostic rules and never mention any category scheme. On inspecting the generated math and agentic insights, we find their content maps cleanly onto the same recurring bottleneck types, an emergent property of the distilled insights rather than an artifact engineered into the pipeline. Reading such a correspondence left-to-right shows the same operation: at a mid-progress state, retrieve the rule that resolves the current bottleneck. For instance, in the ALFWorld “put a hot potato in fridge” case, Base retrieves a topically related but premature heating rule while InsightEmb retrieves the search-priority rule and succeeds, the exact analogue of retrieving “convert a | b into b ≡ 0 (mod a)” to resolve the current bottleneck. Appendix Table 8 lists the six emergent bottleneck categories with paired math and agentic examples. 3.3

Stage 1: Situation-to-Insight Matching

Stage 1 teaches the embedding model to bridge the abstraction gap by matching mathematical problems to their relevant heuristic rules. Training data. Each training example is a triplet (q, I + , I − ) where q is a query in one of three forms: (i) a raw math problem statement (query-only), (ii) a problem concatenated with its full chain-ofthought solution (full trajectory), or (iii) a problem with a truncated solution (partial trajectory). The positive set I + contains distilled heuristic rules that pass the utility filter implied by Equation (1): for each problem in a trajectory subset, a rule is in I + only if prepending it to that problem’s query improved solve rate on validation attempts for that same problem, compared with attempts without the rule. Rules that fail this check, or hurt performance, are placed in I − . This ties contrastive labels to demonstrated progress rather than author judgment

or lexical overlap alone. For a given trajectory subset, the same I + and I − are reused across all three query forms, and only the situation anchor q changes. Insights are provided at two granularities used jointly in training: B UNDLE insights (multi-rule summaries distilled from successful and contrasting trajectories) and ATOMIC insights (single rules split from those bundles). Stage 1 therefore learns from both coarse strategy sets and finegrained rules, totaling 11,950 contrastive pairs over counting & probability, number theory, and geometry problems from the MATH dataset (Hendrycks et al., 2021). Appendix A.1 lists pair counts and domain breakdowns. Training objective. Given curated (q, I + , I − ), we train fθ with InfoNCE (Chen et al., 2020) and in-batch negatives:
exp sim(fθ (q), fθ (I + ))/τ
(3) I∈{I + }∪N exp sim(fθ (q), fθ (I))/τ

L1 = − log P

where τ = 0.01 and N includes in-batch negatives (training group size 11). The loss implements Equation (2) on pairs whose positives were chosen by the progress criterion in Equation (1), not by optimizing E[∆ | I] end-to-end. 3.4

Stage 2: Situation-to-Experience Matching

Both stages share the same contrastive objective (Equation (3)) and the same math data, differing only in what the situation is matched against, which fixes the level of abstraction the model must bridge: • Stage 1 (situation → abstract rule) matches a situation across abstraction levels, to a distilled heuristic that names the bottleneck to resolve. • Stage 2 (situation → concrete experience) matches a situation at its own abstraction level, to a structurally similar solved problem, teaching the model to recognize when two differentlooking situations require the same reasoning approach. The two targets are complementary rather than redundant: Stage 1 supplies the concrete-to-abstract mapping needed to retrieve rules, while Stage 2 sharpens the situation representation itself so that structurally analogous states cluster together, which stabilizes the abstract matching learned in Stage 1. Concretely, Stage 2 uses triplets (q, T + , T − ) where q is a raw problem or a full trajectory, T + is a structurally similar solved problem, and T −

is a dissimilar one, with insights in Stage 1 simply replaced by trajectories. This stage uses 2,896 examples across the same three math domains. 3.5

Inference: Dynamic Insight Retrieval

At inference time, the trained embedding model is deployed for dynamic insight retrieval in target interactive environments without any domain-specific fine-tuning. At each step t, (1) the agent’s current state st is encoded with a task-specific instruction prefix,1 (2) the top-k most similar insights are retrieved from the pre-encoded corpus, and (3) the retrieved insights are prepended to the LLM’s prompt for action generation.

4

Experimental Setup

We fine-tune Qwen3-Embedding-4B (Zhang et al., 2025) with the two-stage curriculum in §3. Appendix B.1 specifies learning rate, batching, contrastive temperature, and hardware, and Table 11 summarizes the full configuration. 4.1

Evaluation Overview

Our goal is to test the design claim in Table 1: whether math-only training yields progressoriented, action-intent retrieval in ALFWorld, WebShop, ScienceWorld, and SRA-Bench. We use two complementary metrics: end-task performance on interactive agent environments and retrieval recall on a static benchmark. On ALFWorld, WebShop, and ScienceWorld, an LLM agent executes full trajectories with dynamic insight retrieval, and we measure whether retrieval improves task completion (success rate and average task score), not ranking accuracy in isolation. On SRA-Bench, no agent is run. We score task-to-skill matching directly with recall@k and nDCG@k. The three agentic environments (detailed in §4) stress complementary procedural skills. Following AgentBench (Liu et al., 2024), we report success rate for ALFWorld and ScienceWorld and average task score for WebShop. 4.2

Evaluation with Agentic Environment

We evaluate dynamic insight retrieval on three textbased interactive environments that stress complementary procedural skills: ALFWorld (Shridhar et al., 2021) (embodied search and manipulation for household tasks), WebShop (Yao et al., 1 E.g., “Given an AlfWorld task observation, retrieve relevant insights or strategies that can help the agent solve the task effectively.”

2022) (product comparison, variant selection, and multi-step purchase workflows), and ScienceWorld (Wang et al., 2022b) (long-horizon scientific procedures such as measuring melting or boiling points, testing conductivity, and locating living or non-living things). Each requires a sequence of navigation and interaction actions, making all three direct tests of whether retrieval can select insights that unblock the agent’s current procedural phase. Common agent configuration. Across all three environments we use the same protocol: Qwen38B (Qwen Team, 2025) as the action-generating LLM with greedy decoding, a history window of 3 steps, a maximum of 50 steps per game, and top1 dynamic insight retrieval, comparing the same two embedding variants (Base and InsightEmb). We evaluate ALFWorld on its test split, WebShop on 500 test games, and ScienceWorld on a fixed, seed-42, task-balanced subset of 500 test variations (reused across all experiments). ScienceWorld uses a 700-bundle / 6,265-atomic insight pool, and its task-type counts are in Appendix Table 17. 4.2.1

Insight corpora

We evaluate two insight corpus granularities per environment to disentangle the effect of the embedding model from the insight content. B UNDLE insights are multi-rule summaries, each containing several rules with full chain-of-thought reasoning. ATOMIC splits each bundle into individual rules with chain-of-thought removed. Appendix Table 9 summarises corpus sizes and rule lengths for each environment. Appendix A.2 documents trajectory collection, DeepSeek-R1/GPT-5.2 distillation prompts, and how B UNDLE and ATOMIC corpora are built. 4.3

SRA-Bench Retrieval Evaluation

To isolate retrieval quality from downstream action generation, we additionally evaluate on SRABench (Su et al., 2026), spanning theorem proving, logical reasoning, tool use, contest math, medical calculation, and code generation. Each query uses the full instance question field with a Qwen3style retrieval instruction prefix (Zhang et al., 2025), and each candidate encodes the skill name, description, and body. Appendix E.1 gives the exact Instruct/Query template and skill-passage format used in evaluation. Unlike the agent environments above, the primary metrics here are retrieval recall (R@k) and ranking quality (N@k) for

Insight

Embedding

ALF

WS

SW

{}

—

52.86 31.03

2.40

B UNDLE Base B UNDLE InsightEmb

54.29 18.42 60.71 31.74

7.40 8.00

ATOMIC ATOMIC

55.00 28.25 7.40 59.29 32.05 10.20

Base InsightEmb

Table 2: Dynamic agent evaluation on ALFWorld (ALF), WebShop (WS), and ScienceWorld (SW) (top-1 retrieval, Qwen3-8B agent). All columns are reported as percentages (success rate for ALF and SW, average task score for WS). The no-insight baseline for SW is only 2.40, so every insight setting more than triples it. Separate SW top-k curves appear in Appendix Table 15.

k ∈ {1, 3, 5, 7, 10}, macro-averaged across task families (§5.4).

5

Results

5.1

ALFWorld Results

We first evaluate whether retrieved insights improve online agent execution in ALFWorld, where the query changes after every observation and action, and on WebShop (§5.2), where the agent must search product pages and complete purchases. Table 2 reports both metrics under the same retrieval protocol. InsightEmb transfers to embodied control. All InsightEmb variants outperform the Base embedding model on test success rate in Table 2. With both B UNDLE and ATOMIC insights, InsightEmb achieves about a 5-point improvement over Base. For B UNDLE, per-game-level retrieval turnover, step counts, and zero-score rates are in Appendix Table 13 (§D.1.2). Since InsightEmb is trained exclusively on mathematical reasoning data, these gains are entirely from cross-domain transfer. Comparison with an ALFWorld-trained in-domain retriever is deferred to §6.3. Per-task-type breakdown. Appendix Table 14 (within §D.2) breaks down test success rates by ALFWorld task type. InsightEmb leads on 4 of 6 types, with the largest gains on clean and find_two, indicating a general structural-matching gain rather than a single-task shortcut. Mechanistic analysis. Appendix §D.1–§D.3 analyzes why retrieval helps on ALFWorld: per-game statistics, qualitative cases, topical vs. procedural rule matching, and step-conditioned retrieval dynamics. InsightEmb shows a better retrieval diver-

sity, step counts, and lower zero-score rates, and a state-aware transition from search to verification and state transformation. 5.2

WebShop Results

WebShop results are summarized in Table 2 (right column). We report average task score as a percentage, which reflects partial credit for attribute matching under the AgentBench evaluation protocol. InsightEmb transfers to web shopping. On WebShop, retrieval quality matters: B UNDLE with Base (18.42%) falls well below the no-insight baseline (31.03%), while InsightEmb achieves a better performance (B UNDLE 31.74%, ATOMIC 32.05%). InsightEmb outperforms Base on both corpora (e.g., +72% relative on B UNDLE). Per-game-level statistics for B UNDLE are in Appendix Table 13 (§D.1.3). Retrieval safety under a noisy insight pool. The WebShop B UNDLE regime highlights a distinct property beyond average relevance: a progressblind retriever can be worse than no retrieval at all (Base B UNDLE 18.42% vs. 31.03% no-retrieval), because it surfaces premature or topically-related but procedurally irrelevant insights from a largely unhelpful pool. InsightEmb learns to filter such premature insights and select the state-appropriate one, turning a net-harmful retrieval setting into a net-useful one (31.74%). Making agentic insight retrieval safe under a noisy corpus is itself a useful property, not only improving average relevance but also avoiding active degradation of the base agent. Qualitative patterns. Manual inspection of divergent games reveals three recurring patterns (variant-selection awareness, loop prevention, and procedural sequencing), indicating that InsightEmb performs procedural matching, retrieving insights that address the agent’s current bottleneck. Appendix D.5 details each pattern with side-by-side Base vs. InsightEmb trajectories. 5.3

ScienceWorld Results

InsightEmb transfers to scientific procedures. ScienceWorld results (top-1 retrieval) are summarized in Table 2 (right column). This environment is substantially harder for the base agent: the no-insight baseline reaches only 2.40% success, reflecting the long-horizon, multi-step nature of scientific procedures (measurement, classification,

and controlled experiments). Against this backdrop InsightEmb wins or ties Base in both corpora at top-1 (10.20% vs. 7.40% on ATOMIC and 8.00% vs. 7.40% on B UNDLE), and every insight setting more than triples the no-insight baseline. Since InsightEmb is trained only on mathematical reasoning data, these gains on a third, procedurally distinct environment are again entirely from crossdomain transfer. Robustness across budget and action model. The advantage persists across retrieval budgets k ∈ {1, 3, 5} (Appendix Table 15) and under a stronger closed-source action model (GPT-5.2, Appendix Table 16), mirroring the budget- and modelrobustness checks reported for ALFWorld and WebShop. Per-task-type grouping. The ScienceWorld task types span complementary functional families (search/identify, measurement/verification, statetransform/experiment, and multi-step reasoning, detailed in Appendix D.6.5), so the aggregate gain reflects broad procedural coverage rather than a single task shortcut. Mechanistic analysis. Per-game-level statistics (B UNDLE, top-1) are reported alongside ALFWorld and WebShop in Appendix Table 13 (§D.1.4). Aside from the task metric, InsightEmb lowers the negative-score rate from 55.4% to 52.0% and raises mean distinct insights from 2.04 to 2.37. Notably, on ScienceWorld InsightEmb more often locks onto a single procedurally complete workflow insight and reuses it across an experiment. Its higher mean step count (40.96 → 43.59) indicates it more often persists through the multi-step protocol rather than terminating early without completing the required state transition. Qualitative patterns. Manual inspection of divergent games reveals three recurring patterns (state-variable awareness, action sequencing, and failure-mode avoidance), mirroring the proceduralmatching behavior seen on ALFWorld and WebShop. Appendix D.6.7 details each pattern with side-by-side Base vs. InsightEmb trajectories and worked examples. 5.4

Static Skill Retrieval on SRA-Bench

Following the static protocol in §4.3, we test whether math-only training improves task-to-skill matching before any action model runs. Table 3 reports macro-averaged results over the 636 gold

skills and 26,262 distractors, and the per-taskfamily breakdown at @1 and @10 is in Appendix Table 21 (§E.2). InsightEmb improves over Base on every cutoff, with small gains at R@1 and substantially larger gains at higher cutoffs (+8.90 R@10). This pattern indicates that the training primarily improves the ranking of relevant skills beyond the first position, which is especially useful when an agent can inspect or condition on multiple retrieved skills. Across families, InsightEmb improves clearly on theorem proving, tool use, contest math, and code generation, where queries and useful skills differ in wording while sharing procedural structure. The only clear negative outlier is MedCalcBench at small cutoffs, whose highly templatelike, lexically specialized skills favor the Base embedder’s entity-level cues at rank 1 (Appendix Table 21). Appendix E.2 discusses the per-family breakdown and the MedCalcBench outlier, lexical vs. structural retrieval bottlenecks, and implications for reranker-limited skill pipelines, and Appendix E.2.1 provides a lightweight BM25-hybrid remedy that recovers R@1 and reaches 100% recall at R@5/R@10 on this family, together with an explicit applicability scope.

6

Analysis and Comparisons

The results above establish that math-trained InsightEmb improves dynamic agent execution and static skill retrieval over the Base embedder. This section asks why: we compare reasoning-oriented embedders and alternative insight generators, validate in-domain fine-tuning against cross-domain training, then analyze whether the learned geometry matches our structural-analogy claim. 6.1

Scaling with Comparison to Reasoning-Oriented Retrievers

We focus this analysis on ALFWorld and WebShop because they represent the two opposite insightpool regimes that a retriever must handle, as revealed by comparing the no-insight baseline ({}) with Base-selected insights in Table 2. On ALFWorld, retrieval is helpful even with an untrained retriever: adding Base-selected B UNDLE insights already raises success over no insight (52.86 → 54.29), so the corpus is a largely useful insight pool and the question is how much better a progressaware retriever can do. On WebShop, the same Base-selected insights are harmful: B UNDLE+Base

Embedding

R@1

R@3

R@5

R@7

R@10

N@1

N@3

N@5

N@7

N@10

Base InsightEmb

30.33 31.37

40.84 46.39

46.40 54.25

49.83 58.56

54.13 63.03

33.82 36.29

37.62 42.00

39.96 45.29

41.23 46.92

42.64 48.42

∆

+1.04

+5.55

+7.85

+8.73

+8.90

+2.47

+4.38

+5.33

+5.69

+5.78

Table 3: SRA-Bench macro-average retrieval results across six task families. Queries use task information and candidates use the full skill content. R@k is recall at k and N@k is normalized discounted cumulative gain at k. Appendix Table 21 breaks down per-task-family results at @1 and @10, and the macro-average is computed by averaging those task families equally.

A natural question is whether reasoning-oriented retrievers trained for static, query–document matching can serve as drop-in replacements for InsightEmb in these two regimes. Using the same ALFWorld and WebShop agent protocol as Table 2, we compare Base, InsightEmb, and two reasoningoriented embedding baselines: (i) Llama-NVEmbed-Reasoning-3B (NVIDIA, 2026) (3.2B, Llama-3.2-3B), the top-ranked open-source singlemodel embedder on BRIGHT, and (ii) ReasonIR8B (Shao et al., 2025) (8B, Llama-3.1-8B), trained with contrastive learning on synthetic reasoningintensive pairs, showing the setting closest to ours. Both differ from InsightEmb (4B, Qwen3Embedding-4B (Zhang et al., 2025)), which is trained only on math data. Beyond the top-1 results in Table 2, we sweep retrieval budgets k ∈ {1, 3, 5} only for all four embedders on both environments and both insight corpora (B UNDLE and ATOMIC). At k=5, top-B UNDLE retrieval already prepends about 25 instruction-level rules to the agent prompt (roughly five rules per bundle), so we do not evaluate larger k. Figure 2 plots ALFWorld (left) and WebShop (right) performances as a function of k, using the arithmetic mean of B UNDLE and ATOMIC at each k ∈ {1, 3, 5}. We also ablate Stage 1 with a no half-traj variant that drops partial-trajectory queries (dashed curve). It lies between Base and full InsightEmb, accounting for most of the Base-to-InsightEmb gain (Appendix C.1.1). Appendix C.1 also provides separate ALFWorld/WebShop top-k curves by corpus.

32

63

30

60

score

drops well below no insight (31.03 → 18.42), indicating a corpus that is largely unhelpful or premature, where a progress-blind retriever surfaces net-negative insights. Studying both regimes together therefore tests two distinct demands, extracting more value from a useful pool (ALFWorld) and staying safe under a noisy pool (WebShop), which is why we concentrate the scaling comparison here rather than on a single environment.

57

28

54

26

51

24 1

3 Base ReasonIR

5 Llama-NV-Reasoning InsightEmb

1

3

5

InsightEmb (no half-traj)

Top-k

Figure 2: Top-k scaling for reasoning-oriented retrievers. Left: ALFWorld success rate . Right: WebShop average task score . Curves show the arithmetic mean of B UNDLE and ATOMIC retrieval at each k ∈ {1, 3, 5} for Base, ReasonIR, Llama-NV-Reasoning, and InsightEmb (solid). The dashed line (InsightEmb color) is the no half-traj ablation, which removes partial-trajectory queries from Stage 1.

On the corpus-averaged curves in Figure 2, InsightEmb surpasses ReasonIR and Llama-NVReasoning in both ALFWorld and WebShop at each k ∈ {1, 3, 5}. This gap is not explained by stronger static matching alone: reasoning-oriented retrievers are trained for static query–passage matching, whereas agentic insight retrieval requires dynamic action-intent matching against an evolving state to retrieve the rule that unblocks the current procedural step, the geometry InsightEmb is trained for and they are not. 6.2

Robustness to Insight Generator and Action Model

Our main results use DeepSeek-R1 (DeepSeek-AI, 2025) to distill insights and Qwen3-8B as the action model. We verify that the gains are not tied to either choice. Alternative insight generator: repeating the dynamic-agent evaluation with GPT-5.2 (OpenAI, 2025)-generated corpora (same action model and top-k protocol), InsightEmb consistently outperforms Base on both ALFWorld and WebShop across k ∈ {1, 3, 5} (Appendix A.4, Figure 3). Alternative action model: replacing Qwen3-8B with the stronger closed-source GPT-5.2 as the action

Corpus

Embedding

ALFWorld

B UNDLE

Base ALF (In-domain) InsightEmb

54.29 57.86 60.71

ATOMIC

Base ALF (In-domain) InsightEmb

55.00 57.86 59.29

Table 4: ALFWorld test success rate for an ALFWorldtrained in-domain retriever (B UNDLE only) vs. Base and math-trained InsightEmb (top-1 retrieval, Qwen3-8B agent). The in-domain model is trained and evaluated on the same environment.

model preserves the advantage across all three environments (Appendix D.6.3, Table 16). Together, the improvement holds across insight source and action model. 6.3

Validating the Structural-Analogy Claim

Math-to-agentic transfer. Table 1 predicts bidirectional transfer: math training should improve agentic retrieval (tested in §5), and with this hypothesis, embodied training should improve math retrieval also. We validate both directions and compare against the natural alternative of inenvironment fine-tuning. Finetuning With ALFWorld A standard alternative to cross-domain training is to fine-tune the embedder on ALFWorld situation-to-insight pairs (same backbone, pipeline, and top-1 agent stack as §4). As Table 4 shows, InsightEmb exceeds this In-domain retriever on both corpora despite using no ALFWorld data, and despite the in-domain set being larger (21,318 vs. 14,846 math pairs), so the gap reflects the supervision source, not scale. Appendix C.2 makes this precise by measuring the geometric diversity of the two sources (pairwise/nearest-neighbor cosine distance and covariance effective rank), replacing the informal notion of “structural diversity”. Agentic-to-math transfer. For the reverse direction, we evaluate all model variants on math insight retrieval, retrieve the correct heuristic rule for a held-out math problem. Table 5 presents accuracy on four math domains. Three domains (number theory, geometry, and counting & probability) overlap with the training distribution (same domains, different problems from the test split with a separate insight corpus), while algebra is entirely out-of-distribution and never appears in training. InsightEmb consistently improves over Base on

Task ∗

Number Theory Geometry∗ Counting & Probability∗ Algebra

Base

InsightEmb

ALF

77.22 54.07 74.47 86.69

79.81 59.29 77.43 88.21

78.33 57.41 75.53 87.81

Table 5: Math insight retrieval accuracy. ALF = ALFWorld-trained model. ∗ In-domain (same domains as training, held-out test split with separate insight pool).

the held-out math test set and the unseen math domain. This confirms that the contrastive curriculum does not sacrifice in-domain math retrieval while learning the claimed geometry. More importantly, the ALFWorld-trained model (ALF) also improves over Base on math insight retrieval across all four domains. This reverse-direction gain supports Table 1: the geometry is shared, not a one-way mathto-agent shortcut.

7

Conclusion

We studied agentic insight retrieval: at each decision step, an agent must retrieve an abstract rule that resolves its current procedural bottleneck, not merely text similar to the observation. We introduced InsightEmb, a contrastive framework that learns goal-conditioned action-intent matching from mathematical reasoning data alone. On ALFWorld, WebShop, and ScienceWorld it raises end-task success under dynamic top-k insight injection, and on SRA-Bench it raises retrieval recall and nDCG for task-to-skill matching, outperforming the Base embedder, an ALFWorld-trained in-domain retriever, and strong reasoning-oriented embedders. The gains are consistent across environment, action model (Qwen3-8B and GPT-5.2), insight source (DeepSeek-R1, GPT-5.2, prebuilt skills), corpus granularity, and retrieval budget. InsightEmb further makes retrieval safe under a noisy insight pool where a progress-blind retriever can be worse than no retrieval at all. Together, these results support a simple design claim: math heuristic retrieval and agentic insight retrieval share the same progress-oriented matching structure. Limitations. Our dynamic agent experiments primarily use Qwen3-8B as the action model and DeepSeek-R1 (with a GPT-5.2 robustness check) for insight distillation. We additionally validate on a third environment (ScienceWorld) and with a stronger closed-source action model (GPT-5.2) across all three environments (Appendix D.6), and

extending to still more environments and LLMs would further strengthen the generality claim. Although SRA-Bench isolates static retrieval quality, dynamic agentic retrieval still lacks gold labels for which insight is optimal at each state, so downstream success remains an important but indirect measure of retrieval utility and closely aligns with our objective. Finally, for Stage 1 partial-trajectory queries we reuse the same (I + , I − ) sets as for query-only and full-trajectory pairs. Generating state-specific pairs that reflect what is helpful at that point in the rollout would likely improve state awareness, which we leave to future work.

References Jianlyu Chen, Junwei Lan, Chaofan Li, Defu Lian, and Zheng Liu. 2025. ReasonEmbed: Enhanced text embeddings for reasoning-intensive document retrieval. arXiv preprint arXiv:2510.08252. Describes the BGE-Reasoner-Embed embedding models (released under the BGE-Reasoner framework). Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. 2020. A simple framework for contrastive learning of visual representations. In Proceedings of the 37th International Conference on Machine Learning. Tsz Ting Chung, Leyang Cui, Lemao Liu, Xinting Huang, Shuming Shi, and Dit-Yan Yeung. 2024. Selection-p: Self-supervised task-agnostic prompt compression for faithfulness and transferability. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 11057–11070. Tsz Ting Chung, Lemao Liu, Mo Yu, and Dit-Yan Yeung. 2025. DivLogicEval: A framework for benchmarking logical reasoning evaluation in large language models. In Findings of the Association for Computational Linguistics: EMNLP 2025. Tsz Ting Chung, Lemao Liu, Mo Yu, and Dit-Yan Yeung. 2026. Many-shot CoT-ICL: Making in-context learning truly learn. In Proceedings of the 43rd International Conference on Machine Learning (ICML). DeepSeek-AI. 2025. DeepSeek-R1: Incentivizing reasoning capability in LLMs via reinforcement learning. arXiv preprint arXiv:2501.12948. Xiang Deng, Yu Gu, Boyuan Zheng, Shijie Chen, Samuel Stevens, Boshi Wang, Huan Sun, and Yu Su. 2023. Mind2Web: Towards a generalist agent for the web. In Advances in Neural Information Processing Systems. Yao Fu, Dong-Ki Kim, Jaekyeom Kim, Sungryull Sohn, Lajanugen Logeswaran, Kyunghoon Bae, and Honglak Lee. 2024. AutoGuide: Automated generation and selection of context-aware guidelines for

large language model agents. In Advances in Neural Information Processing Systems. Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Ming-Wei Chang. 2020. REALM: Retrievalaugmented language model pre-training. In Proceedings of the 37th International Conference on Machine Learning. Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021. Measuring mathematical problem solving with the MATH dataset. arXiv preprint arXiv:2103.03874. Maggie Huan, Yuetai Li, Tuney Zheng, Xiaoyu Xu, Seungone Kim, Minxin Du, Radha Poovendran, Graham Neubig, and Xiang Yue. 2025. Does math reasoning improve general LLM capabilities? understanding transferability of LLM reasoning. Preprint, arXiv:2507.00432. Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. Dense passage retrieval for opendomain question answering. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing. Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2020. Retrieval-augmented generation for knowledgeintensive NLP tasks. In Advances in Neural Information Processing Systems. Yuqing Li, Jiangnan Li, Zheng Lin, Ziyan Zhou, Junjie Wu, Weiping Wang, Jie Zhou, and Mo Yu. 2025. Mindscape-aware retrieval augmented generation for improved long context understanding. Preprint, arXiv:2512.17220. Yuqing Li, Jiangnan Li, Mo Yu, Guoxuan Ding, Zheng Lin, Weiping Wang, and Jie Zhou. 2026a. Queryfocused and memory-aware reranker for long context processing. Preprint, arXiv:2602.12192. Yuqing Li, Jiangnan Li, Mo Yu, Zheng Lin, Weiping Wang, and Jie Zhou. 2026b. MiA-Signature: Approximating global activation for long-context understanding. Preprint, arXiv:2605.06416. Xiao Liu, Hao Yu, Hanchen Zhang, Yifan Xu, Xuanyu Lei, Hanyu Lai, Yu Gu, Hangliang Ding, Kaiwen Men, Kejuan Yang, and 1 others. 2024. AgentBench: Evaluating LLMs as agents. In International Conference on Learning Representations. Bodhisattwa Prasad Majumder, Bhavana Dalvi Mishra, Peter Jansen, Oyvind Tafjord, Niket Tandon, Li Zhang, Chris Callison-Burch, and Peter Clark. 2023. CLIN: A continually learning language agent for rapid task adaptation and generalization. arXiv preprint arXiv:2310.10134.

Niklas Muennighoff, Nouamane Tazi, Loïc Magne, and Nils Reimers. 2023. MTEB: Massive text embedding benchmark. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics. NVIDIA. 2026. Llama-NV-Embed-Reasoning-3B. Hugging Face model card. 3.2B-parameter reasoning embedding model built on Llama-3.2-3B; top-ranked open-source retriever on BRIGHT.

Hongjin Su, Weijia Shi, Jungo Kasai, Yizhong Wang, Yushi Hu, Mari Ostendorf, Wen-tau Yih, Noah A. Smith, Luke Zettlemoyer, and Tao Yu. 2023. One embedder, any task: Instruction-finetuned text embeddings. In Findings of the Association for Computational Linguistics: ACL 2023.

OpenAI. 2025. Update to GPT-5 system card: GPT-5.2. Technical report.

Hongjin Su, Howard Yen, Mengzhou Xia, Weijia Shi, Niklas Muennighoff, Varsha Kishore, Luke Zettlemoyer, Wen-tau Yih, Danqi Chen, and Tao Yu. 2025. BRIGHT: A realistic and challenging benchmark for reasoning-intensive retrieval. In International Conference on Learning Representations.

Charles Packer, Sarah Wooders, Kevin Lin, Vivian Fang, Vishvak Patil, Ion Stoica, and Joseph E. Gonzalez. 2023. MemGPT: Towards LLMs as operating systems. arXiv preprint arXiv:2310.08560.

Weihang Su, Jianming Long, Qingyao Ai, Yichen Tang, Changyue Wang, Yiteng Tu, and Yiqun Liu. 2026. Skill retrieval augmentation for agentic AI. arXiv preprint arXiv:2604.24594.

Jing-Cheng Pang and 1 others. 2025. Bootstrapping broad LLM reasoning from math. arXiv preprint. Math-first elicitation for broad multi-domain reasoning. Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian, and 1 others. 2024. ToolLLM: Facilitating large language models to master 16000+ real-world APIs. In Proceedings of the Twelfth International Conference on Learning Representations. Qwen Team. 2025. Qwen3 technical report. arXiv preprint arXiv:2505.09388. Stephen Robertson and Hugo Zaragoza. 2009. The probabilistic relevance framework: BM25 and beyond. Foundations and Trends in Information Retrieval, 3(4):333–389. Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. 2023. Toolformer: Language models can teach themselves to use tools. In Advances in Neural Information Processing Systems. Rulin Shao, Rui Qiao, Varsha Kishore, Niklas Muennighoff, Xi Victoria Lin, Daniela Rus, Bryan Kian Hsiang Low, Sewon Min, Wen-tau Yih, Pang Wei Koh, and Luke Zettlemoyer. 2025. ReasonIR: Training retrievers for reasoning tasks. arXiv preprint arXiv:2504.20595. Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. 2023. Reflexion: Language agents with verbal reinforcement learning. In Advances in Neural Information Processing Systems. Mohit Shridhar, Xingdi Yuan, Marc-Alexandre Côté, Yonatan Bisk, Adam Trischler, and Matthew Hausknecht. 2021. ALFWorld: Aligning text and embodied environments for interactive learning. In Proceedings of the Ninth International Conference on Learning Representations.

Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. 2024. Voyager: An open-ended embodied agent with large language models. Transactions on Machine Learning Research. Liang Wang, Nan Yang, Xiaolong Huang, Binxing Jiao, Linjun Yang, Daxin Jiang, Rangan Majumder, and Furu Wei. 2022a. Text embeddings by weaklysupervised contrastive pre-training. arXiv preprint arXiv:2212.03533. Ruoyao Wang, Peter Jansen, Marc-Alexandre Côté, and Prithviraj Ammanabrolu. 2022b. ScienceWorld: Is your agent smarter than a 5th grader? In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 11279–11298. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc V. Le, and Denny Zhou. 2022. Chain-of-thought prompting elicits reasoning in large language models. In Advances in Neural Information Processing Systems. Junjie Wu, Tsz Ting Chung, Kai Chen, and Dit-Yan Yeung. 2025a. Unified triplet-level hallucination evaluation for large vision-language models. Transactions on Machine Learning Research (TMLR). Junjie Wu, Jiangnan Li, Yuqing Li, Lemao Liu, Liyan Xu, Jiwei Li, Dit-Yan Yeung, Jie Zhou, and Mo Yu. 2025b. SitEmb-v1.5: Improved context-aware dense retrieval for semantic association and long story comprehension. Preprint, arXiv:2508.01959. Junjie Wu, Jiangnan Li, Yuqing Li, Lemao Liu, Liyan Xu, Jiwei Li, Dit-Yan Yeung, Jie Zhou, and Mo Yu. 2025c. Situated embedding models for contextaware dense retrieval. Preprint, arXiv:2508.01959. Shitao Xiao, Zheng Liu, Peitian Zhang, and Niklas Muennighoff. 2023. C-Pack: Packaged resources to advance general Chinese embedding. arXiv preprint arXiv:2309.07597.

Lee Xiong, Chenyan Xiong, Ye Li, Kwok-Fung Tang, Jialin Liu, Paul N. Bennett, Junaid Ahmed, and Arnold Overwijk. 2021. Approximate nearest neighbor negative contrastive learning for dense text retrieval. In Proceedings of the Ninth International Conference on Learning Representations. Shunyu Yao, Howard Chen, John Yang, and Karthik Narasimhan. 2022. WebShop: Towards scalable realworld web interaction with grounded language agents. In Advances in Neural Information Processing Systems. Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2023. ReAct: Synergizing reasoning and acting in language models. In Proceedings of the Eleventh International Conference on Learning Representations. Mo Yu, Tsz Ting Chung, Chulun Zhou, Tong Li, Rui Lu, Jiangnan Li, Liyan Xu, Haoshu Lu, Ning Zhang, Jing Li, and Jie Zhou. 2025a. PRELUDE: A benchmark designed to require global comprehension and reasoning over long contexts. Preprint, arXiv:2508.09848. Mo Yu, Lemao Liu, Junjie Wu, Tsz Ting Chung, Shunchi Zhang, Jiangnan Li, Dit-Yan Yeung, and Jie Zhou. 2025b. The stochastic parrot on LLM’s shoulder: A summative assessment of physical concept understanding. In Proceedings of the 2025 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL). Eric Zelikman, Yuhuai Wu, Jesse Mu, and Noah D. Goodman. 2022. STaR: Bootstrapping reasoning with reasoning. Advances in Neural Information Processing Systems. Yanzhao Zhang, Mingxin Li, Dingkun Long, Xin Zhang, Huan Lin, Baosong Yang, Pengjun Xie, An Yang, Dayiheng Liu, Junyang Lin, Fei Huang, and Jingren Zhou. 2025. Qwen3 embedding: Advancing text embedding and reranking through foundation models. arXiv preprint arXiv:2506.05176. Andrew Zhao, Daniel Huang, Quentin Xu, Matthieu Lin, Yong-Jin Liu, and Gao Huang. 2024. ExpeL: LLM agents are experiential learners. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 19632–19642. Shuyan Zhou, Frank F. Xu, Hao Zhu, Xianyi Zhou, Robert Lo, Abishek Sridhar, Xianyi Cheng, Yonatan Bisk, Daniel Fried, Soheil Alikhani, and 1 others. 2024. WebArena: A realistic web environment for building autonomous agents. In International Conference on Learning Representations.

A

Training Data and Insight Corpora

A.1

Training Data Statistics

InsightEmb is trained only on mathematical reasoning data (Stage 1: situation-to-insight, Stage 2: situation-to-experience, §3.3–3.4). Table 6 summarizes contrastive training examples per stage. Stage 1 combines bundle insight pairs (7,740 unique rules in the insight pool) and atomic pairs (22,243 unique rules). “Query” = query-only samples, and “Traj” = query concatenated with a full or partial CoT trajectory. Table 7 gives the per-domain Stage 1 breakdown (bundle + atomic). Stage

Granularity

Total

Query

Traj

Stage 1

Bundle Atomic Combined

5,463 6,487 11,950

1,917 2,308 4,225

3,546 4,179 7,725

Stage 2 (CoT)

—

2,896

2,329

567

Table 6: Training data statistics. Stage 1 totals combine bundle and atomic insight pairs.

Math Domain

Bundle

Atomic

Stage 1 Total

Counting & Probability Number Theory Geometry

1,736 2,220 1,507

2,647 2,400 1,440

4,383 4,620 2,947

Total

5,463

6,487

11,950

Math Domain

Stage 2 (CoT)

Counting & Probability Number Theory Geometry

966 966 964

Total

2,896

Table 7: Per-domain training counts. Stage 1 = bundle + atomic insight pairs, Stage 2 = CoT experience pairs.

A.2

Insight Corpus Construction

This appendix documents how agentic and math insight corpora are built and how they are converted into InsightEmb training data. A.2.1

Pipeline Overview

Step 1: Collect training trajectories. For each math training problem (or ALFWorld/WebShop training game), we run the task model five times on the training split and log full interaction histories, following trajectory-collection practice in self-improvement methods (Zelikman et al., 2022; Zhao et al., 2024). Successful and failed rollouts are grouped by problem, retaining all five responses per instance together with labels indicating which rollouts succeeded.

Step 2: Distill insights. We apply a multi-subset genetic search over trajectory subsets. For each subset, an LLM (DeepSeek-R1 (DeepSeek-AI, 2025) for main experiments, GPT-5.2 (OpenAI, 2025) for the robustness check) receives a distillation prompt (§A.2.2) and produces a B UNDLE insight: a numbered list of general, task-agnostic rules. Each candidate insight is validated on the problems represented in that subset: we prepend the rules to each problem’s query, re-run the solver, and keep the insight only if it improves solve rate over a noinsight baseline on those problems. Chromosomes that yield helpful bundles are kept, and ATOMIC rules are obtained by splitting each bundle into individual numbered rules (chain-of-thought removed). For ALFWorld and WebShop, each trajectory step is first compressed into a one-sentence summary before distillation. Step 3: Build Stage 1 embedding training pairs. Validated bundle and atomic insights are paired with math problems in two query forms: query-only (problem statement) and query+trajectory (problem with full or partial chainof-thought). Each pair assigns I + to insights that improved solve rate on that problem in the subsetvalidation check above, and I − to insights that did not (Equation (1)), then filters pairs to ensure enough contrastive negatives per training group. Step 4: Build Stage 2 CoT training pairs. We construct situation-to-experience pairs from structurally similar solved problems, pairing each query with helpful and unhelpful demonstration trajectories. A.2.2 Insight-Generation Prompts Math (successful trials). Used when all trajectories in a subset succeeded: You are an advanced reasoning agent that can create rules based on forming new critiques of past task trajectories. You will be given successful tasks trials in which you are doing math derivation. Here are the trails. {trajs} By examining the successful trials, you can create the new rules are GENERAL and HIGH LEVEL insights of the successful trials or proposed way of Thought so they can be used as helpful tips to different tasks in the future. Have an emphasis on tips that help the agent perform better Thought and Action. Do not mention the trials in the rules because all the rules should be GENERALLY APPLICABLE. Each rule should be concise and easy to follow. Any operation can be used MULTIPLE times.

Math (mixed success and failure). Used when a subset contains both correct and incorrect solutions,

with trajectories wrapped as <correct_trial_i> or <wrong_trial_i>: You are an advanced reasoning agent that can create rules based on forming new critiques of past task trajectories. You will be given multiple tasks trials in which you are doing math derivation. Here are the trails. {trajs} By examining and contrasting between failed and successful trials, you can create the new rules are GENERAL and HIGH LEVEL critiques of the failed trials or proposed way of Thought so they can be used as helpful tips or to avoid similar failures when encountered with different questions in the future. Have an emphasis on critiquing how to perform better Thought and Action. Do not mention the trials in the rules because all the rules should be GENERALLY APPLICABLE. Each rule should be concise and easy to follow. Any operation can be used MULTIPLE times.

ALFWorld (successful trials). After step-level summarization, used when all trajectories in a subset succeeded: You are an advanced reasoning agent that can create rules based on forming new critiques of past AlfWorld task trajectories. You will be given successful AlfWorld task trials involving multi-turn conversations with an environment. Here are the trajectories: {trajs} By examining the successful trials, you can create new rules that are GENERAL and HIGH LEVEL insights about successful AlfWorld task completion strategies. Focus on insights that help the agent perform better in multi-turn dialogue with environments. Key aspects to consider: - Effective communication strategies with the environment - Problemsolving approaches in interactive settings - Handling of multi-step reasoning in dialogue - Adaptation to environment feedback - Efficient exploration and action selection Do not mention specific trials in the rules because all rules should be GENERALLY APPLICABLE to AlfWorld tasks. Each rule should be concise and easy to follow. Any operation can be used MULTIPLE times.

ALFWorld (mixed success and failure). You are an advanced reasoning agent that can create rules based on forming new critiques of past AlfWorld task trajectories. You will be given multiple AlfWorld task trials involving multi-turn conversations. Here are the trajectories: {trajs} By examining and contrasting between failed and successful trials, you can create new rules that are GENERAL and HIGH LEVEL critiques of failed strategies or proposed ways of thinking. Focus on critiquing how to perform better in multi-turn dialogue with environments. Key aspects to consider: - Communication breakdowns with the environment - Ineffective problemsolving approaches - Poor handling of multi-step

reasoning - Failure to adapt to environment feedback - Inefficient exploration strategies Do not mention specific trials in the rules because all rules should be GENERALLY APPLICABLE. Each rule should be concise and easy to follow.

WebShop (successful trials). After step-level summarization, used when all trajectories in a subset succeeded: You are an advanced reasoning agent that can create rules based on forming new critiques of past WebShop task trajectories. You will be given successful WebShop task trials involving multi-turn conversations with a web shopping environment. Here are the trajectories: {trajs} By examining the successful trials, you can create new rules that are HIGH LEVEL insights about successful WebShop task completion strategies. Focus on insights that help the agent perform better in multi-turn dialogue with web shopping environments. Key aspects to consider: - Effective search query formulation strategies - Product selection and comparison approaches - Handling of product attributes (size, color, price, etc.) - Navigation strategies (search, click, back, buy) - Efficient exploration of product listings - Matching user requirements to product descriptions - Decisionmaking on when to buy vs. continue searching Do not mention specific trials in the rules. Each rule should be concise and easy to follow. Any operation can be used MULTIPLE times.

WebShop (mixed success and failure). You are an advanced reasoning agent that can create rules based on forming new critiques of past WebShop task trajectories. You will be given multiple WebShop task trials involving multi-turn conversations. Here are the trajectories: {trajs} By examining and contrasting between failed and successful trials, you can create new rules that are HIGH LEVEL critiques of failed strategies or proposed ways of thinking. Focus on critiquing how to perform better in multi-turn dialogue with web shopping environments. Key aspects to consider: - Ineffective search query formulation - Poor product selection and comparison decisions - Failure to match product attributes to requirements - Inefficient navigation patterns - Premature or delayed purchase decisions - Misinterpretation of product descriptions - Failure to adapt search strategy based on results Do not mention specific trials in the rules. Each rule should be concise and easy to follow.

Insight validation (math). For a subset of trajectories, the distillation prompt produces a candidate rule set, which we then test on every problem in that subset by prepending the rules to the problem query and re-running the solver:

Below is an instruction that describes a task. Write a response that appropriately completes the request and wrap the final answer inside \boxed{{}}. {generated insight rules} ### Problem: {problem from the subset} ### Solution: Let’s think step by step.

For each problem q in the subset, we record whether the solver succeeds with vs. without the prepended rules. The insight is assigned to I + on problems where it helps and to I − on problems where it does not, which defines the Stage 1 contrastive labels. A.2.3 Example Generated Insights Math (counting & probability). - Define Variables & Relationships: Start by defining variables for unknown quantities and establish mathematical relationships based on given conditions to structure the problem. - Complementary Probability for “Not” Events: When calculating the probability of an event not occurring (e.g., not sitting together), compute the probability of the complementary event and subtract it from 1. - Leverage Standard Combinatorial Formulas: Recognize scenarios like distributing identical items (stars and bars) or non-colinear points (triangles in a cube) and apply the appropriate formulas directly.

ALFWorld. 1. Maintain a Goal-Driven Action Sequence: Successful agents follow a clear path toward the goal, while failed ones get sidetracked. 2. Adapt Exploration Based on Feedback: If an object isn’t found where expected, systematically check other plausible locations instead of repeating the same action. 3. Verify Object Presence Before Interaction: Avoid assuming an object is present, and check first to prevent unnecessary actions. 4. Prioritize Common Storage Areas First: Check typical locations (like countertops or stove burners) mentioned in the task before deep cabinet search.

WebShop. 1. Formulate search queries with all key user specifications (e.g., “gluten free chai orca spice flavor under $20”). 2. If initial results are irrelevant, refine search terms by adding or rephrasing keywords. 3. Before selecting a product, cross-check all attributes (price, features, specs) against user requirements. 4. Use systematic navigation: proceed through pages methodically and backtrack if no matches are found. 5. Compare multiple products on the same page to identify the best match before clicking.

Emergent bottleneck-category correspondence. Inspecting the distilled math and agentic insights above (and across the full corpora), their content maps cleanly onto the same small set of recurring bottleneck types, even though the distillation prompts (§A.2.2) never mention any category scheme. Table 8 makes this post-hoc correspondence explicit: each row pairs a math state → rule with an analogous agent insight, and reading it leftto-right shows the same operation, namely that at a mid-progress state, we retrieve the rule that resolves the current bottleneck. This supports the design claim in §3.2 that the analogy is an emergent property of the distilled insights, not an artifact engineered into the pipeline. A.3

Insight Corpus Statistics

Main experiments (§4, §5) use DeepSeekR1 (DeepSeek-AI, 2025) generated insight corpora for ALFWorld and WebShop, and DeepSeek-V4Flash generated insights for ScienceWorld, each at two granularities: B UNDLE (multi-rule summaries with chain-of-thought) and ATOMIC (single rules split from bundles). Table 9 reports corpus size and length statistics for each environment, and these corpora are referenced throughout the dynamicagent evaluation. A.4 Alternative Insight-Generator Robustness A.4.1

Corpus Statistics

Table 10 summarizes GPT-5.2 (OpenAI, 2025)generated insight corpora used in the robustness check (§6.2). B UNDLE files contain one multi-rule summary per past trajectories subset, and ATOMIC files expand each bundle into individual rules. Compared with the DeepSeek-R1 (DeepSeek-AI, 2025) corpora in Table 9, GPT-5.2 bundles are shorter on average while atomic rules are similar in length. A.4.2

Top-k Scaling Details

Figure 3 averages B UNDLE and ATOMIC at each k (Qwen3-8B agent, GPT-5.2-generated insights), and InsightEmb consistently outperforms Base on both environments, confirming the improvement persists under an alternative insight generator. Figures 4 and 5 show the per-corpus breakdown.

Math: state → rule

Analogous agent insight

Search / Identify

Identify whether two triangles are similar before using ratios, or identify fixed/forbidden objects before counting.

Prioritize task-relevant locations first: check object containers/destinations named in the task (e.g., stoveburner, sidetable).

State-Transform

Convert a | b into b ≡ 0 (mod a), or recast a configuration into coordinates.

Select a product variant / change object state before completing the goal (heat, clean, run a device).

Verification

Substitute the answer back to detect extraneous roots, or check probabilities sum to 1.

Attribute cross-checking: before any click, verify the product matches all specified attributes (price, color, size).

Error-Recovery

If direct counting overcounts, switch to inclusion–exclusion, and if a ratio method fails, switch to coordinates.

Progressive filter strategy: when initial search yields partial matches, add the missing requirements to subsequent queries.

Structure / Layout

Map positions onto coordinates or parity classes, or arrange constrained objects relative to one another.

Reason about the scene’s spatial/containment structure: which receptacle holds the target, where the item sits in the variant grid.

Strategy-beforeExecute

First decide divisibility vs. modular vs. gcd/lcm, choose factoring vs. substitution, and plan sub-step order before executing.

Precision-first search: include all critical attributes (type, color, size, price) in the initial query, then classify the current phase and choose a plan before acting.

Bottleneck gory

cate-

Table 8: Emergent correspondence between math and agentic insights across six recurring bottleneck categories. The categories are a post-hoc observation over distilled insights, not a label scheme imposed during generation, and each row instantiates the same situation → bottleneck-resolving-rule operation. Environment

Corpus

# Insights

Avg Words

Range

ALFWorld

B UNDLE ATOMIC

2,419 21,916

494 25

90–1,114 1–365

WebShop

B UNDLE ATOMIC

501 5,212

549 24

176–1,064 1–371

ScienceWorld

B UNDLE ATOMIC

700 6,265

432 46

41–889 2–634

69

30

66

30

64

Success rate (%)

score

Table 9: Insight corpus statistics (main experiments). ALFWorld and WebShop insights are generated by DeepSeekR1 (DeepSeek-AI, 2025), and ScienceWorld insights are generated by DeepSeek-V4-Flash. B UNDLE = multi-rule insights, ATOMIC = individual rules extracted from bundles. Range = min–max words per insight.

30

63

29

60

29

57

29

72

62

68

60

64

58

60

56 54

1

3

5 Base

Top-k

1 InsightEmb

3

5

Figure 3: Robustness check with GPT-5.2-generated insight corpora. Left: ALFWorld. Right: WebShop. Curves average B UNDLE and ATOMIC at each k for Base and InsightEmb.

1

3

5 Base

56

Top-k

1 InsightEmb

3

5

Figure 4: ALFWorld success rate vs. k ∈ {1, 3, 5} with GPT-5.2-generated insights.

Environment

Corpus

# Insights

Avg Words

Range

ALFWorld

B UNDLE ATOMIC

700 8,410

314 25

178–500 11–58

WebShop

B UNDLE ATOMIC

700 10,162

358 23

198–532 4–54

31

30

C

Reasoning-Oriented Retrievers and Supervision Diversity

C.1

Reasoning-Oriented Retriever Top-k Details

30 29

30

28

30 30 3

5 Base

Top-k

1 InsightEmb

3

5

Figure 5: WebShop average task score vs. k ∈ {1, 3, 5} with GPT-5.2-generated insights.

B

Training Setup and Hyperparameters

B.1

Training Setup

Hyperparameters

Both training stages share the settings in Appendix B.1. Table 11 lists the full configuration. Hyperparameter Base model Learning rate LR scheduler Warmup ratio Batch size (per device) Grad. accumulation Training epochs Temperature τ Training group size Max passage length Precision GPUs DeepSpeed Seed

68

64

We fine-tune Qwen3-Embedding-4B (Zhang et al., 2025), building on contrastive embedding practice (Wang et al., 2022a; Su et al., 2023), using DeepSpeed ZeRO-3 on 8 GPUs. Both stages use learning rate 1×10−5 with cosine scheduling, perdevice batch size 8 with 2 gradient accumulation steps, temperature τ =0.01, training group size 11, maximum passage length 1,536 tokens, 6 epochs, and warmup ratio 0.1. Stage 2 initializes from the Stage 1 checkpoint. Table 11 lists the full configuration. B.2

Figure 2 in the main text averages B UNDLE and ATOMIC at each k ∈ {1, 3, 5}. Figures 6 and 7 show the per-corpus breakdown. The dashed no half-traj curve (InsightEmb color) removes partialtrajectory queries from Stage 1 while keeping the same I + and I − sets. Success rate (%)

1

64

60

60

56

56

52

52

48 1

Table 11: Training hyperparameters for both stages.

5

1

Llama-NV-Reasoning InsightEmb

3

5

InsightEmb (no half-traj)

Top-k

Figure 6: ALFWorld test success rate vs. retrieval budget k ∈ {1, 3, 5} for Base, ReasonIR, Llama-NVReasoning, InsightEmb, and the no half-traj ablation (dashed, InsightEmb color).

36

32

32

30 28

28

26

24

24

20

22 1

Qwen3-Emb-4B 1 × 10−5 Cosine 0.1 8 2 6 0.01 11 1,536 BF16 8 ZeRO-3 3407

3 Base ReasonIR

Average task score (%)

Average task score (%)

Table 10: GPT-5.2 insight corpus statistics. Range = min–max words per non-empty insight, and WebShop has 10 empty bundle summaries (word count 0). Atomic = single rules split from bundles.

3 Base ReasonIR

5 Llama-NV-Reasoning InsightEmb

1

3

5

InsightEmb (no half-traj)

Top-k

Figure 7: WebShop average task score at each k for the same models and corpora.

C.1.1

Effect of Partial-Trajectory Queries

The no half-traj ablation removes pairs whose query q is a partial (incomplete) solution trace, leaving query-only and full-trajectory anchors. The dashed curve in Figure 2 lies between Base and full InsightEmb on both environments. At the k that maximizes corpus-averaged InsightEmb on each

environment, the step from no half-traj to full InsightEmb accounts for about 83% of the total Baseto-InsightEmb gain on ALFWorld and about 49% on WebShop, indicating that much of InsightEmb’s gains over Base come from partial-trajectory situation anchors with shared I + and I − . This also helps explain why InsightEmb outperforms static reasoning retrievers that are not trained on such mid-inference anchors. C.2

Why Math Training Outperforms In-Domain Fine-Tuning

As shown in §6.3 in Table 4, the ALFWorld-trained retriever underperforms math-trained InsightEmb despite task-matched supervision. The intuition is that ALFWorld is a comparatively narrow source: its tasks are built from a small set of recurring procedural patterns (the standard ALFWorld task types: find, go-to, take, heat/clean/cool, and place), so its situation-to-rule pairs concentrate on a few structures. Math traces, by contrast, span counting & probability, number theory, and geometry, yielding a wider variety of situation-to-rule mappings. We make this precise and measurable below, replacing the informal phrase “structural diversity” used in earlier drafts. C.2.1

Quantifying Structural Diversity

For each situation-to-rule pair we form a normalized pair embedding p = normalize([s, r, r − s]) with InsightEmb, where s and r are the normalized situation and rule embeddings and r − s is their difference vector. Within each source we randomly draw 2,000 sampled pairs and compute three geometry statistics: • Mean pairwise cosine distance: the average of 1 − cos(pi , pj ) over 5,000 random pair-pair comparisons among the sampled pairs. Larger means the pairs are more spread out. • Mean nearest-neighbor distance: the average of 1 − maxj̸=i cos(pi , pj ) over all sampled pairs, i.e. each pair’s cosine distance to its single closest neighbor. Larger means pairs sit farther from their nearest neighbor. • Covariance P effective rank: exp(H), where H = − k qk log qk is the entropy of thePnormalized singular-value spectrum qk = σk2 / σ 2 of the covariance of the centered sampled pair embeddings. Larger means the embeddings occupy more independent directions. On all three statistics, math pairs are more spread out than ALFWorld pairs (Table 12).

Diversity metric Mean pairwise cosine distance Mean nearest-neighbor distance Covariance effective rank

ALFWorld

Math

0.1065 0.0372 93.27

0.2446 0.0546 100.50

Table 12: Geometric diversity of situation-to-rule pair embeddings [s, r, r − s] under InsightEmb (2,000 sampled pairs per source). Math pairs occupy a broader, higher-rank region and sit farther from their nearest neighbors than ALFWorld pairs, consistent with the narrow set of recurring ALFWorld task patterns.

Math situation-to-rule pairs occupy a broader, higher-rank region of the embedding geometry and are farther from their nearest neighbors, whereas ALFWorld pairs are more concentrated. A retriever trained on the broader source therefore generalizes to bottleneck configurations that are rare or absent in ALFWorld’s own games. This is also consistent with the literature, where math reasoning is a recognized springboard for cross-domain transfer: reasoning trained on math/code with verifiable rewards yields general reasoning behaviors that transfer to STEM and other tasks (DeepSeekAI, 2025), math-only training generalizes across scientific QA, agent planning, and coding (Huan et al., 2025), and math-first elicitation seeds broad multi-domain reasoning (Pang et al., 2025). Our contribution brings this recognized property to the retrieval side. C.2.2

Transfer Despite Different Step Granularities

The transfer does not require math and embodied tasks to share the same notion of a “step”, relying only on the shared situation-to-rule structure (§3.2, Table 8). We do impose an explicit step decomposition on math traces: a chain-of-thought solution is not one atomic block, and we take prefixes of the derivation to form partial-trajectory queries (query form (iii), §3.3), so a single problem yields several situation-to-rule pairs at different points of progress, the math counterpart of an agent’s mid-episode state. At inference we do not require matching step boundaries: the agent’s state (task + action history + observation) is encoded with a domain-neutral instruction prefix (§3.5) into the same embedding space and matched to the nearest rule. What crosses domains is the geometry and its shared bottleneck categories, not a step schema. Two empirical checks confirm this despite the different step granularities. First, the no half-traj ablation removes exactly the partial

(mid-progress) math anchors and erases most of the agentic gain, showing it is the mid-progress math states that teach the transferable property. Second, the transfer is bidirectional (Table 5): an ALFWorld-trained model, whose “steps” are embodied actions, improves math insight retrieval on all math domains, which could not occur if the two step structures were genuinely incompatible for this purpose. The reverse direction is weaker than math-to-ALFWorld, exactly as expected if ALFWorld data covers a narrower slice of the shared geometry.

D

Per-Environment Analysis

D.1

Per-Game-Level Agent Statistics

This appendix supplements the aggregate results in Table 2 with per-game-level analysis: for each ALFWorld or WebShop test game, we log the final task score, the number of environment steps until termination, and how many distinct insight blocks were injected over the course of that game. All numbers use Qwen3-8B with top-1 dynamic insight retrieval on B UNDLE summary insights, the same protocol as the B UNDLE rows in Table 2. The goal is to show that InsightEmb’s gains are not only a higher mean success rate, but also more varied retrieval within each game, shorter successful trajectories, and fewer complete failures. D.1.1

Metrics and computation

Each evaluated game is one independent rollout from the initial observation until success, failure, or the 50-step cap. At every step, the current agent state is embedded and the top-1 insight is retrieved from the pre-encoded B UNDLE corpus, and that text block is prepended to the action prompt. Steps counts the environment steps in that rollout. Distinct insights per game counts how many different insight strings were injected over the inference. Retrieval is recomputed at every step (up to 50), so each step could return a different top-1 neighbor, the count is therefore bounded by the number of steps (≤ 50). The statistic measures how often the top-1 neighbor changes, not corpus size. Avg. task score is the mean of per-game-level outcomes: on ALFWorld and ScienceWorld, 1 for a win and 0 otherwise (equivalent to success rate), and on WebShop, the environment’s task_score ∈ [0, 1] converted to a percentage.

Failed games are rollouts with no positive reward. ALFWorld and WebShop task scores are bounded below by 0, so a failed rollout is a zeroscore game (task_score = 0), whereas ScienceWorld task scores can be negative (the environment penalizes some invalid trajectories), so failed ScienceWorld rollouts are zero-or-negative-score. We report all three under a single failed-game count, with the failed-game rate equal to this count divided by the number of games. Retrieval turnover is the fraction of decision steps at which the top-1 retrieved insight differs from the previous step, a step-normalized measure of how often retrieval changes as the state evolves. Games with >1 distinct insight is the count (and rate) of games where retrieval changed at least once. D.1.2

ALFWorld (140 test games)

On ALFWorld, InsightEmb improves every column in Table 13. Retrieval turnover. Retrieval runs at every step, so distinct insights per game is bounded by steps (≤ 50). Base nevertheless uses only 1.82 distinct strings on average (max 4) while InsightEmb averages 5.79 (max 22). The comparison is about how often top-1 changes as the state evolves, not about injecting a new corpus block every step. Efficiency. Mean steps decrease from 30.3 to 28.2 and the median drops from 30 to 24, so at least half of InsightEmb games finish in ≤24 steps whereas Base’s median sits at the cap for unsuccessful search-heavy runs. Shorter medians cooccur with higher success: InsightEmb avoids prolonged cabinet-by-cabinet loops documented in Appendix D.2.3. Failures. Zero-score games (failed rollouts) decrease from 64 to 55, showing gains even in completely failed games. D.1.3

WebShop (500 test games)

Retrieval turnover. With per-step top-1 retrieval over up to 50 steps, distinct insights per game could be as large as the step count, and Base averages 1.85 (max 4) while InsightEmb averages 5.98 (max 18). InsightEmb changes the retrieved block in 487/500 games (97%) vs. 295 (59%) for Base, so different steps can surface search, variant-selection, and checkout rules from the B UNDLE corpus.

ALFWorld

WebShop

ScienceWorld

Metric

Base

InsightEmb

Base

InsightEmb

Base

InsightEmb

# games # failed games Failed-game rate Avg. task score Avg. steps Median steps Distinct insights / game (mean) Max distinct insights / game

140 64 45.7% 54.3% 30.3 30 1.82 4

140 55 39.3% 60.7% 28.2 24 5.79 22

500 309 61.8% 18.42% 40.8 50 1.85 4

500 258 51.6% 31.74% 33.9 50 5.98 18

500 277 55.4% 7.40% 40.96 7.0 2.04 7

500 260 52.0% 8.00% 43.59 7.0 2.37 7

Table 13: Per-game-level statistics (Qwen3-8B, top-1 dynamic retrieval). ALFWorld, WebShop, and ScienceWorld all use B UNDLE insights over the seed-42 balanced 500-instance subset. A failed game is a rollout with no positive reward: for ALFWorld and WebShop, whose scores are bounded below by 0, this is a zero-score game, whereas ScienceWorld task scores can be negative, so failed ScienceWorld games are zero-or-negative-score. Retrieval turnover is the fraction of steps at which the top-1 neighbor changes. Failed ALFWorld/WebShop games typically reach the 50-step cap, whereas ScienceWorld rollouts terminate earlier on average (median 7 steps).

Efficiency. Average steps drop from 40.8 to 33.9 (mean rollout ∼41 steps) while both medians remain at 50, because many games still hit the cap, so InsightEmb shortens trajectories it completes rather than shifting the median below the horizon. Appendix D.5 illustrates loop-heavy Base runs vs. earlier termination when purchase rules are retrieved. Failures. Zero-score games decrease from 309 to 258 (−51), the gain comes from converting full failures into partial or full purchase credit. D.1.4

ScienceWorld (500 test games)

ScienceWorld statistics in Table 13 use the B UN DLE corpus and the seed-42 balanced 500-instance subset (top-1, Qwen3-8B). Failures. Negative-score games decrease from 277 to 260 (−17), lowering the negative-score rate from 55.4% to 52.0%, so InsightEmb reduces complete failures even on this hard environment, consistent with its small but positive success gain (7.40% → 8.00%). Retrieval turnover. On ScienceWorld the retrieval-diversity pattern differs from ALFWorld and WebShop. When normalized by trajectory length, retrieval turnover rises from 0.268 to 0.380 and the mean distinct insights per game from 2.04 to 2.37 (both medians at 2.0, max 7 for both), so InsightEmb still updates its top-1 neighbor more often per step as the experimental state evolves. At the same time, the raw count of games with more than one distinct insight falls from 330/500 (66.0%) for Base to 224/500 (44.8%) for InsightEmb. These are consistent rather than contradictory: InsightEmb more

often locks onto a single procedurally complete workflow insight and reuses it across many steps of an experiment (§D.6.6), so within a game it changes insights less often overall, but the changes it does make track the current sub-procedure more tightly. In other words, ScienceWorld rewards procedural completeness of a retrieved insight over per-observation novelty, unlike the search-heavy ALFWorld/WebShop trajectories where higher raw turnover co-occurs with success. Efficiency. The median rollout length is short (7 steps for both), reflecting many quicklyterminating games, and InsightEmb’s higher mean steps (40.96 → 43.59) indicate it more often persists through the multi-step experimental protocol rather than terminating early without completing the required state transition. D.2

ALFWorld Per-Task and Mechanistic Analysis

This section expands the ALFWorld mechanistic claims summarized in §5: per-task-type breakdown, multi-granularity invariance, qualitative retrieval examples, rule-level topical vs. procedural matching, and step-conditioned retrieval dynamics. D.2.1

Per-Task-Type Breakdown

Table 14 supplements the aggregate ALFWorld results in Table 2 by reporting test success counts for each ALFWorld task family (ATOMIC corpus, top-1 retrieval, Qwen3-8B). Counts are out of the games listed in parentheses per type, and bold marks the higher count between Base and InsightEmb.

Task Type

Base

Ours

clean (22) cool (26) heat (19) put (48) examine (17) find_two (8)

27 62 42 71 71 12

41 69 53 71 53 38

Overall

55.0

59.3

Table 14: ALFWorld success rates by task type (top-1 retrieval, Qwen3-8B). Base = ATOMIC/Base, and Ours = ATOMIC/InsightEmb. Numbers in parentheses indicate total games per type.

D.2.2 Multi-Granularity Query Invariance Our Stage 1 design enforces that query-only, partial-trajectory, and full-trajectory versions of the same problem all retrieve the same insight. This directly maps to the agentic setting: at step 0 the agent has only the task description (analogous to query-only), at mid-game it has partial action history (partial trajectory), and at late steps it has extensive history (full trajectory). The multigranularity training ensures that the correct insight is retrievable at every stage of task execution, not just at the beginning. D.2.3

Qualitative Example: Insight-Guided Search Figure 8 illustrates how InsightEmb’s retrieved insights lead to better action selection on a representative task (“put a hot potato in fridge”). Task: “put a hot potato in fridge” Base model retrieves (ATOMIC/Base): “Apply Task-Specific Physics Reasoning: Heat transfers require closed appliances. Objects in containers remain inaccessible until explicit retrieval.” → Agent focuses on heating procedure but cannot locate the potato → Timeout at 50 steps InsightEmb retrieves (ATOMIC/InsightEmb): “Prioritize Task-Relevant Locations First: Always check object containers/destinations mentioned in the task (e.g., stoveburner, sidetable).” → Agent locates potato → heat → place in fridge → Success in 37 steps

Figure 8: Qualitative comparison on Game 0 (test, top5 retrieval). The base model’s insight describes the heating procedure but not where to find the potato, whereas InsightEmb retrieves a search-priority rule that addresses the actual bottleneck.

The base model’s insight is topically relevant but strategically vacuous, as it does not indicate where to find the potato. InsightEmb retrieves a

structurally matched search-priority rule, directly determining the agent’s first action. This pattern of generic vs. structurally specific retrieval recurs across the 16 games that InsightEmb wins uniquely under top-1 retrieval (see also §D.4). D.2.4

Topical vs. Procedural Retrieval: Rule-Level Analysis

Because each ATOMIC insight is a single rule, we can examine which type of rule each embedding model prioritizes for the same query. We classify retrieved rules with GPT-5.2 (OpenAI, 2025) into functional categories: W HERE (search strategy), H OW (state-change procedures), P LACE (destination logic), and V ERIFY (state checks). The Base model retrieves rules about how to heat (“Apply Task-Specific Physics Reasoning,” “Apply Thermodynamic Context Filtering,” “Appliance Function Mapping”), which describe the heating procedure but not where to find the potato. InsightEmb retrieves rules about where to search (“Prioritize Task-Relevant Locations First,” “Prioritize Direct Affordances,” “Prioritize Immediate Target Interaction”), which address the actual bottleneck: the agent’s first action must be to locate the target object. This reveals a key difference in how the two models understand task structure: • Base matches the query to rules that share the task’s topic (heating → heating rules). This is topical matching. • InsightEmb matches the query to rules that address the task’s current bottleneck (the agent hasn’t found the object yet → search rules). This is action-intent matching: retrieving the rule implied by what the agent needs to do next. The action-intent match is correct: in ALFWorld, the agent must first find the target object before it can apply any state transformation. InsightEmb has learned this sequential dependency from the structural parallel in math, where a problem must first be understood (matched to the right strategy) before it can be solved (executed step by step). D.3

Step-Conditioned Retrieval Details

The rule-level analysis above examines which rules are retrieved, and we now examine when they are retrieved. We extract the top-1 retrieved ATOMIC insight at every step of all 140 test games, classify each rule with GPT-5.2 (OpenAI, 2025), and plot the per-step category mix in Figure 9 with 3-step rolling average. For readability we merge S EARCH

Base vs. InsightEmb dynamics. Base shows almost no step-wise change: Search is 49.7% on average over steps 1–4 vs. 51.0% over steps 20–25 (∆=+1.3 pp), State Transform 23.0% → 23.6%, and Verification 14.8% → 16.1%. InsightEmb instead shows a clear state-aware shift along the episode: Search falls from 78.1% (steps 1–4) to 61.7% (steps 20–25, ∆=−16.4 pp), while Verification rises from 16.9% to 31.5% (∆=+14.6 pp) and State Transform from 1.9% to 5.9% (∆=+3.9 pp). That is, InsightEmb retrieval tracks procedural phase (heavy search while the object is still missing, then more checks and state-change rules as the agent approaches completion), whereas Base remains roughly uniform. Category definitions and illustrative rules. Each label denotes the procedural role of a single atomic rule (not the ALFWorld task type): Search Rules for finding the target or reaching the right place before manipulation. Example: “Prioritize Task-Relevant Locations First: Always check object containers/destinations mentioned in the task (e.g., stoveburner, sidetable).” State Transform Rules for changing object state (clean, heat, cool) at the correct appliance. Example: “Apply Contextual Cleaning Protocol: When ‘clean’ is specified in the task, automatically prioritize sinkbasin interaction after obtaining the target object, before any placement attempts.” Verification Rules for confirming prerequisites or completion before the next action. Example: “Ensure Cooling Completion Before Placement: Verify the cooling action (e.g., fridge interaction) is fully executed and acknowledged by the environment before attempting to move the object to its final destination.” State checking Rules for tracking progress, ordering sub-goals, or maintaining awareness of what was already searched or carried. Example: “Maintain a Dynamic Search Tracker: Systematically record searched locations and their contents to avoid

redundant checks and focus exploration on unexamined areas.” Error Recovery Rules for escaping loops or reformulating strategy after failed attempts. Example: “Implement Loop Detection: Establish a threshold (3–5 attempts) for repeated unsuccessful navigation patterns, then trigger systematic reevaluation of object location hypotheses.” Placement Rules for putting the object at its final receptacle. Example: “Container Proximity Hierarchy: When storing objects, first navigate to the nearest valid container type specified in the task (drawer/cabinet) before considering distant alternatives.” 100

% of Retrieved Insights

& L OCATE with NAVIGATION into Search, and S TATE T RACKING with TASK P LANNING into State checking (six categories total, with definitions and examples below).

80 60 40 20 0

0

5

10

15

Search State Transform

20

25 0

Verification State checking

Step

5

10

Error Recovery Placement

15

20

25

Figure 9: Per-step retrieved insight category distribution (ATOMIC, test, top-1 dynamic retrieval, 3-step rolling average). Left: Base embedder. Right: InsightEmb. Base changes little across steps, whereas InsightEmb shifts from Search toward Verification and State Transform.

D.4

ALFWorld Qualitative Case Studies

We present two additional examples where InsightEmb succeeds while Base fails (top-1 retrieval). Game 9: “clean some ladle and put it in countertop.” InsightEmb checks cabinet 1–3, then sinkbasin 1, opens cabinet 4, finds the ladle on countertop 1, cleans it at the sink, and places it on the countertop, completing in 12 steps. Base loops through cabinets and the sink without taking the ladle, timing out at 50 steps. This illustrates task-focused search: InsightEmb reaches the sink and visible surfaces early, while Base’s generic “check fridges first” heuristic delays locating the ladle. Game 33: “put a cool lettuce in countertop.” InsightEmb opens the fridge, scans countertop

1–3, takes the lettuce from countertop 3, cools it in the fridge, and places it on the countertop, completing in 10 steps. Base opens the fridge but then wanders through drawers and cabinets, never retrieving the lettuce, timing out at 50 steps. Base’s retrieved summary rule prioritizes “common storage areas” broadly, whereas InsightEmb’s rule set emphasizes checking task-mentioned surfaces (countertops) before deep cabinet search. D.5

WebShop Qualitative Examples

We present illustrative examples from the WebShop evaluation where InsightEmb with ATOMIC insights succeeds while Base fails, or achieves substantially higher task scores. Manual inspection of divergent games (summarized in §5.2) reveals three recurring qualitative patterns, which the examples below demonstrate with side-by-side Base vs. InsightEmb trajectories: 1. Variant selection awareness. InsightEmb’s insights guide the agent to explicitly select product variants (color, size) before purchasing, while Base frequently skips this step, resulting in partial scores instead of perfect scores. 2. Loop prevention. Base gets stuck in search– browse–back loops for 21–50 steps, whereas InsightEmb’s insights about session management and error recovery help the agent break out of unproductive cycles. 3. Procedural sequencing. InsightEmb retrieves insights that encode a sequential workflow (search → verify → select variants → buy), while Base retrieves topically relevant but procedurally vague insights. Together these show InsightEmb performs procedural matching, retrieving insights that address the agent’s current bottleneck. D.5.1

Example 1: Office Chair (Game 226, Atomic)

Task: “Find me height adjustable, high density, easy install, easy assemble home office chairs for living room with color: type 7-pink, and price lower than 120.00 dollars.” InsightEmb: WON in 6 steps (score = 1.0). Base: LOST in 3 steps (score = 0.857). InsightEmb insight: Progressive Filter Strategy: When initial search yields partial matches, systematically add missing requirements to subsequent queries.

Base insight: Precision-First Search: Always include all critical attributes in the initial search query using AND logic for better filtering.

Action comparison. Both models find the same product (b08p8lrfz4), but: • InsightEmb: search → click product → check features → go back → select “type 7-pink” → buy now ✓ • Base: search (with AND syntax) → click product → buy now immediately (without selecting color variant) × Analysis. InsightEmb’s “Progressive Filter Strategy” guides the agent to inspect product features and select the correct variant before purchasing. Base’s “Precision-First Search” focuses on query formulation but does not instruct variant selection, causing premature purchase with a partial score. D.5.2

Example 2: Women’s Sweater (Game 479, Atomic)

Task: “Find me women’s sweaters with relaxed fit, long sleeve with color: light heather grey, and size: large, and price lower than 50.00 dollars.” InsightEmb: WON in 8 steps (score = 1.0). Base: LOST in 3 steps (score = 0.600). InsightEmb insight: Attribute Cross-Checking: Before any click, verify that the product matches all specified attributes (price, color, size, etc.).

Action comparison. Both find the same product (b07dkgjr74), but: • InsightEmb: search → explore first result → back to search → re-search → click product → select “light heather grey” → select “large” → buy now ✓ • Base: search → click product → buy now immediately (without selecting color or size) × Analysis. This is the clearest example of the variant selection awareness pattern. InsightEmb’s “Attribute Cross-Checking” rule explicitly guides the agent to verify and select each product variant before purchasing. Base’s generic search-formulation rule does not encode this procedural constraint, and the agent buys immediately without selecting the required color and size variants.

D.5.3

Example 3: Dining Set (Game 287, Atomic)

Task: “Find me button tufted, mid century, high density, easy assemble dining sets with solid wood, wood frame for dining room with color: light grey, and price lower than 250.00 dollars.” InsightEmb: WON in 4 steps (score = 1.0). Base: LOST in 50 steps (score = 0.000). This is the most dramatic example (∆ = +1.000). InsightEmb insight: Precision-First Search: Always include all critical attributes (type, color, size, price limit) in the initial search query using AND logic for better filtering.

Action comparison. • InsightEmb: search (comprehensive query) → click product (b09gy58gdh) → select “light grey” → buy now ✓ (4 steps) • Base: search → back to search → re-search → back to search → re-search → click next → click next → back to search → . . . (50 steps, never clicks a product to purchase) Analysis. InsightEmb’s concise insight produces an effective search query that finds the product immediately. Base gets stuck in a search–refine– next–back loop for all 50 steps, illustrating the loop prevention failure: without a clear recovery strategy, the agent exhaustively re-searches without ever committing to a product. D.5.4

Example 4: iPad Case (Game 190, Bundle)

Task: “Find me compatible apple online game services with case cover with color: coast coconut trees, and price lower than 50.00 dollars.” InsightEmb: WON in 6 steps (score = 1.0). Base: LOST in 50 steps (score = 0.000). Action comparison. • InsightEmb: search “apple game case cover” → back to search → re-search “iPad case coconut tree design” → click product (b07vvvdrcx) → select “coast coconut trees” → buy now ✓ (6 steps) • Base: search “Apple case cover” → click next → next → next → next → back to search → research → next → next → next → . . . (50 steps, navigating pages without clicking any product)

Analysis. InsightEmb’s agent reformulates the search query after the first attempt fails, switching from the literal task wording to a more productoriented query (“iPad case coconut tree design”). Base’s agent repeatedly paginates through results without reformulating, illustrating the loop prevention pattern: InsightEmb’s retrieval surfaces strategies for query refinement and error recovery. D.5.5

Example 5: Men’s Sleepwear (Game 166, Atomic)

Task: “Find me men’s sleep & lounge with long sleeve, elastic waistband for daily wear with color: multi 10, and size: xx-large, and price lower than 80.00 dollars.” InsightEmb: WON in 9 steps (score = 1.0). Base: LOST in 7 steps (score = 0.667). InsightEmb insight: Compound Feature Recognition: For combined requirements, search for the most distinctive pairing first, then check secondary attributes in product details.

Action comparison. search → click product • InsightEmb: (b09nd9dp7j) → check description → back to search → re-search with exact attributes → click same product → select “multi 10” → select “xx-large” → buy now ✓ • Base: search → click product (b09nd8p2qr) → check features → back to search → re-search → click same product → buy now without selecting variants × Analysis. InsightEmb’s “Compound Feature Recognition” guides the agent through a verifythen-select workflow: first confirm the product matches, then explicitly select each variant before purchase. Base’s agent finds a similar product but purchases without selecting the required color and size, resulting in a partial score. This combines variant selection awareness with procedural sequencing: InsightEmb encodes a sequential workflow (search → verify → select variants → buy). D.5.6

Summary of WebShop Qualitative Patterns

Across all divergent games, three consistent patterns emerge. The qualitative cases above and the main WebShop column in Table 2. 1. Variant selection awareness (Games 226, 479, 393, 166): InsightEmb consistently guides the

agent to select product variants (color, size) before purchasing. Base frequently skips this step, resulting in partial scores (0.600–0.857) instead of perfect scores. This is the single most impactful behavioral difference. 2. Loop prevention (Games 287, 190): Base gets stuck in search–browse–back loops for 50 steps, scoring zero. InsightEmb’s insights about query reformulation and error recovery help the agent find the product and complete the purchase. 3. Procedural sequencing (Game 166): InsightEmb retrieves insights that encode a sequential workflow (search → verify → select variants → buy), while Base retrieves topically relevant but procedurally vague insights. This mirrors the ALFWorld finding where InsightEmb performs procedural matching rather than topical matching. D.6

ScienceWorld Evaluation Details

This appendix supplements the ScienceWorld results in Table 2 (§5.3) with the full top-k breakdown, a stronger closed-source action model across all three environments, and the task distribution of the evaluation subset. D.6.1

Setup

We run ScienceWorld (Wang et al., 2022b) with the identical insight-retrieval pipeline used for ALFWorld and WebShop: Qwen3-8B as the actiongenerating LLM with greedy decoding, a history window of 3 steps, and dynamic top-k insight retrieval over the ScienceWorld insight corpus (700 B UNDLE / 6,265 ATOMIC insights, Appendix Table 9). The 500 evaluation instances are a fixed, seed-42, task-balanced subset of all ScienceWorld test variations, saved once and reused across all experiments for comparability. We report success rate (%). The no-insight baseline (no retrieval at all) scores only 2.40. D.6.2

Top-k Retrieval Budgets

Table 15 reports ScienceWorld success rate for Base and InsightEmb across corpus granularities and retrieval budgets k ∈ {1, 3, 5}. InsightEmb wins or ties Base on all six cells, and every insight setting more than triples the no-insight baseline (2.40). Because ScienceWorld does not add a qualitatively new insight-pool regime beyond the good/bad pools already illustrated by ALFWorld and WebShop, the main-text top-k figure (Figure 2) retains only ALFWorld and WebShop, and the Sci-

enceWorld top-k results are presented here in tabular form. Budget

Corpus

Base

InsightEmb

top-1 top-1

B UNDLE ATOMIC

7.40 7.40

8.00 10.20

top-3 top-3

B UNDLE ATOMIC

8.20 7.20

9.60 8.40

top-5 top-5

B UNDLE ATOMIC

8.80 9.60

10.80 9.60

Table 15: ScienceWorld success rate (%) across retrieval budgets k ∈ {1, 3, 5} (Qwen3-8B agent, 500 seed-42 task-balanced test instances). The no-insight baseline is 2.40, so every insight setting more than triples it. Bold marks the better value within each row, and ties are bolded for both.

D.6.3

Stronger Closed-Source Action Model (GPT-5.2)

To test whether the advantage depends on the Qwen3-8B action model, we replace it with a stronger, closed-source action model (GPT5.2 (OpenAI, 2025)) and re-run all three environments with top-1 retrieval. As shown in Table 16, InsightEmb improves over Base across ScienceWorld, ALFWorld, and WebShop, indicating that the gains are not tied to a specific action model. Environment

Corpus

Base

InsightEmb

ScienceWorld

B UNDLE ATOMIC

36.4 37.2

45.8 40.0

ALFWorld

B UNDLE ATOMIC

63.6 60.0

70.0 73.6

WebShop

B UNDLE ATOMIC

18.9 32.2

26.1 32.5

Table 16: Dynamic agent evaluation with a stronger closed-source action model (GPT-5.2, top-1 retrieval) across all three environments. Reported as success rate (%) for ScienceWorld and ALFWorld, and average task score (%) for WebShop. Replacing the Qwen3-8B action model with GPT-5.2 preserves the InsightEmb advantage.

D.6.4

Task Distribution of the 500-Instance Subset

Table 17 lists the most frequent ScienceWorld task types in the fixed seed-42 balanced subset used for all ScienceWorld experiments.

Task type inclined-plane-friction-named-surfaces use-thermometer find-animal test-conductivity test-conductivity-of-unknown-substances find-plant measure-melting-point-known-substance find-non-living-thing measure-melting-point-unknown-substance mendelian-genetics-unknown-plant

Count 37 27 26 26 26 24 24 23 22 22

Table 17: Top task-type counts in the fixed, seed-42, task-balanced 500-instance ScienceWorld test subset (saved once and reused across all experiments for comparability).

D.6.5

Task-Type Grouping

To parallel the ALFWorld per-task-type analysis (Table 14), the most frequent ScienceWorld task types (Table 17) fall into the functional categories the agent must resolve: search/identify (findanimal, find-plant, find-non-living-thing), measurement/verification (use-thermometer, measuremelting-point), state-transform/experiment (testconductivity), and multi-step reasoning (inclinedplane-friction, mendelian-genetics), mirroring the W HERE/H OW/V ERIFY rule taxonomy used for ALFWorld (§D.2.4). D.6.6

Step-Conditioned Retrieval Dynamics

Although retrieval is performed at each decision step, the most useful retrieved insights are not always narrowly tied to a single observation. In ScienceWorld, many tasks require executing a stable multi-step procedure across changing observations. For example, an insight such as “to solve inclinedplane friction tasks, place the object on the ramp, vary the relevant surface or angle, observe whether it slides, then compare outcomes” can guide several consecutive actions: setting up the apparatus, manipulating the relevant variable, observing the outcome, and making a comparison. Thus a single high-level procedural insight may remain useful across many environment steps, even as the observation changes. This helps explain why InsightEmb does not necessarily require more diverse or more state-specific retrievals. Instead of retrieving a different insight for every local observation, it often retrieves a general but actionable workflow that covers the full experimental structure. In this sense the retrieved insight acts less like a one-step hint and more like a compact policy sketch. This is consistent with

Table 13, where the mean distinct retrieved insights are broadly similar between Base and InsightEmb (2.04 vs. 2.37), while InsightEmb appears to retrieve insights that are more procedurally complete and reusable across steps (higher retrieval turnover, 0.268 → 0.380).

D.6.7

Illustrative Examples: How an Insight Guides the Next Step

Manual inspection of divergent games (summarized in §5.3) reveals three recurring qualitative patterns: 1. State-variable awareness. InsightEmb more often retrieves insights that name the latent state variable to manipulate (ramp angle, surface friction, object category), helping the agent convert a high-level goal into a concrete next operation, e.g. setting up the ramp and observing motion rather than inspecting nearby objects. 2. Action sequencing. InsightEmb retrieves procedural insights encoding an ordered workflow (prepare apparatus → manipulate variable → observe outcome → compare → answer), whereas Base retrieves semantically relevant but orderfree insights that leave the agent inspecting objects without completing the protocol. 3. Failure-mode avoidance. Base frequently alternates between generic exploration actions without committing to the required state transition, whereas InsightEmb more often retrieves insights specifying when to stop exploring and act (place an object on a ramp, change an angle, test conductivity), which reduces failed games even when the task is not fully solved. As in ALFWorld and WebShop, these show InsightEmb performs procedural matching, retrieving the insight that addresses the agent’s current bottleneck. The following side-by-side examples (Base vs. InsightEmb, ATOMIC, top-1) illustrate these patterns.

Example 1: inclined-plane task. Goal: determine which ramp setup makes an object slide.

Step

Base

InsightEmb

Aspect

Base

InsightEmb

Retrieved insight

“Inclined planes involve ramps and objects moving down slopes.” (topical)

“Place the object on the ramp, vary the surface or angle, observe whether it slides, then compare outcomes.” (procedural)

Retrieved insight

“Conductors allow electricity to pass through them.” (topical)

“Connect the object into the circuit, observe whether the bulb activates, then record whether it conducts.” (procedural)

Generic inspection: examine ramp / look around.

Concrete experiment: put object on ramp, set/change angle or surface, observe sliding.

Next action

Talk about or inspect the object.

Insert object into the test apparatus.

Failure mode

Knows the concept but never performs the test.

Executes the experiment needed for reward.

Browses objects without testing the variable.

Performs the statechanging action to collect evidence.

Implied action

next

Resulting behavior

Table 18: Inclined-plane task. Both retrieve ramprelated insights, but Base retrieves a topical rule identifying only the domain while InsightEmb retrieves a procedural rule specifying the next experimental operation, turning passive observation into active intervention.

Example 2: find living/non-living entity task. Goal: find an animal, plant, living thing, or nonliving thing.

Step

Base

InsightEmb

Retrieved insight

“Animals and plants are living things.” (topical)

“Enumerate visible objects, classify each by the requested category, then focus on the matching object.” (procedural)

Inspect a random object or move rooms.

Check visible objects against the target category before acting.

Often misses the correct object despite seeing it.

More likely to select the correct entity type.

Implied action

Resulting behavior

next

Table 19: Entity-identification task. The key challenge is matching the goal category to the current observation. InsightEmb retrieves an insight that turns the task into a classification procedure (list candidates, classify, choose), producing a more deliberate next step than Base’s general fact about living things.

Example 3: procedural workflow vs. topical match. Goal: complete a multi-step science experiment (conductivity test).

Table 20: Conductivity task. Base retrieves scientifically correct but operationally incomplete rules, whereas InsightEmb’s insight includes the required interaction protocol, bridging conceptual knowledge to executable environment actions.

E

SRA-Bench Details

E.1

SRA-Bench Query and Skill Encoding

We evaluate SRA-Bench with the same Qwen3style query prefix used for Base and InsightEmb at inference. For each instance, the query text is the full question field from the benchmark (background context plus task description). Skills are encoded without an instruction prefix. Retrieval instruction. The fixed task description prepended to every query is: Given an agent task question, retrieve the reusable skill that helps solve the task. The skill may describe a theorem, logic pattern, tool workflow, medical calculator, math concept, or software library.

Query format.

Each query is embedded as:

Instruct: Given an agent task question, retrieve the reusable skill that helps solve the task. The skill may describe a theorem, logic pattern, tool workflow, medical calculator, math concept, or software library. Query:{question}

where {question} is the full SRA-Bench instance text. This matches the Qwen3 Instruct: . . . Query: convention used for our embedding models. Skill (passage) format. Each candidate skill in the mixed corpus (636 gold and 26,262 distractor skills) is embedded as: {name} {description} {content}

using the name, description, and content fields from the benchmark corpus JSON. We rank all corpus skills against each query by cosine similarity over normalized embeddings.

E.2

SRA-Bench Retrieval Analysis

Table 21 gives the per-task-family breakdown of the SRA-Bench results at @1 and @10, supplementing the macro-average in Table 3. MedCalcBench is the only task family where InsightEmb underperforms Base at small cutoffs (Table 21). Medical calculation skills are highly template-like, lexically specialized, and domainspecific: correct retrieval often depends on exact medical score names, disease terms, biomarker or variable names, units, and other clinical terminology. The Base embedding model appears to preserve these fine-grained biomedical and entity-level cues, whereas InsightEmb’s cross-domain training encourages more abstract structural matching and may therefore downweight or smooth over such terminology at rank 1. This interpretation is also consistent with the original SRA-Bench results, where MedCalcBench already achieves about 90% R@1 and over 90% R@10 after reranking the BM25 top-50 candidates with different reranker models, indicating that lexical candidate generation is less of a bottleneck for this task. In contrast, task families such as theorem proving, tool use, contest math, and code generation benefit from abstractionaware ranking because the query and useful skill can differ substantially in wording while sharing a reasoning or procedural structure. This distinction matters in practical retrieval-augmented skill selection pipelines: an LLM reranker can only inspect a limited candidate set, such as the BM25 top-50 skills, due to context-window constraints. Thus, task families for which lexical retrieval fails to place the correct skill within this window constitute the more important bottleneck. From this perspective, the strong results on LogicBench and CHAMP are especially encouraging, because they require more semantic or reasoning-oriented matching between task descriptions and skill content, where surface lexical overlap is weak. InsightEmb therefore improves the recall of relevant skills before reranking, expanding the effective coverage of downstream LLM-based selection.

E.2.1

A BM25-Hybrid Remedy for MedCalcBench

Rather than only explaining the MedCalcBench regression, we provide a concrete remedy. Our diagnosis is that medical-calculator skills carry dense domain-specific terminology (score names, disease terms, biomarkers, and units), so lexical overlap between the query and the correct skill is an important relevance signal in this family. A retriever whose training pushes toward abstract, structural matching (as InsightEmb’s does) tends to underweight this terminology-level overlap, which is why its R@1 drops here even though it helps on the structure-dominant families. We therefore apply a lightweight hybrid retriever: a BM25 (Robertson and Zaragoza, 2009) pre-filter at top-50 followed by InsightEmb reranking, which reintroduces exactly the lexical-overlap signal that pure dense retrieval under-weighted. As shown in Table 22, on MedCalcBench this lifts InsightEmb’s R@1 back up and reaches perfect recall at R@5, matching or exceeding Base at every cutoff beyond rank 1. Metric

InsightEmb + BM25

Base + BM25

R@1 R@5 R@10

70.91 100.00 100.00

92.73 98.18 100.00

Table 22: MedCalcBench recall under a BM25 top-50 pre-filter followed by dense reranking (Base = Qwen3Embedding-4B). The hybrid recovers InsightEmb’s R@1 from 63.27 (without pre-filter, Table 21) to 70.91, and reaches 100% at both R@5 and R@10, edging ahead of Base at R@5 (100.00 vs. 98.18).

Under the BM25 top-50 pre-filter, InsightEmb reaches 100% recall at both R@5 and R@10 and recovers R@1 to 70.91, closing the practical gap for downstream reranker-limited pipelines. The residual R@1 difference reflects the terminologydense nature of medical-calculator skills, where high lexical overlap between query and skill makes lexical matching a strong signal that favors Base at rank 1. Applicability scope. We place this in context: across all our evaluations, MedCalcBench is the only setting where InsightEmb does not clearly lead, and every other task family and every agentic environment (ALFWorld, WebShop, ScienceWorld) shows strong, consistent gains. Far from undermining the method, this single exception gives a clean applicability condition. InsightEmb

Task family

TheoremQA LogicBench ToolQA CHAMP MedCalcBench BigCodeBench

n

747 760 1,430 223 1,100 1,140

Base

InsightEmb

R@1

N@1

R@10

N@10

R@1

N@1

R@10

N@10

48.19 1.58 16.22 13.86 92.73 9.42

48.19 1.58 16.22 19.73 92.73 24.47

74.43 40.39 44.83 38.34 98.18 28.62

60.63 19.53 28.84 26.97 95.80 24.09

58.10 6.32 29.65 19.06 63.27 11.85

58.10 6.32 29.65 29.15 63.27 31.23

86.61 39.34 62.45 52.58 98.18 39.04

72.37 20.18 45.69 38.48 81.52 32.27

Table 21: Per-task SRA-Bench retrieval results at @1 and @10. Queries use task information and candidates use full skill content. Bold marks the better value within each task family, and ties are bolded for both embeddings.

is designed for state-aware, action-guided agentic retrieval where relevance is procedural or structural, and it is not the right tool to use alone for terminology-dense retrieval where lexical overlap between query and skill is the dominant signal (e.g., medical calculators keyed on score names, biomarkers, and units). In that regime the BM25 hybrid supplies the missing lexical signal.

Related documents

• When Does Memory Help? A Cost-Aware Evaluation of Long-Term Memory in Tool-Using LLM Agents#972506
• RPMem: Learning Long-Term Recurrent Parametric Memory Across Sessions for LLM Agents#1052293
• Predicting Partial Answer Quality and Utility in Agentic Retrieval-Augmented Generation#1110809
• ITER: Interaction-Aware Retrieval for Agentic Search#998360
• DeferMem: Query-Time Evidence Distillation via Reinforcement Learning for Long-Term Agent Memory#1117431