arXiv CSopen access

Think Through a Bottleneck: Hourglass Reasoning for Rigorous Induction

Think Through a Bottleneck: Hourglass Reasoning for Rigorous Induction Huan Zhu∗ Peking University

arXiv:2607.11696v1 [cs.AI] 13 Jul 2026

ABSTRACT Self-refinement often fails to strengthen few-shot inductive reasoning in large language models. Prompting a model to explicitly state its inferred rule does little on its own. What actually matters is a structurally enforced isolation between reasoning stages, so that information can only pass between them as a compressed symbolic state. We introduce Hourglass reasoning, which enforces strict context isolation between reasoning stages. The frozen LLM acts as a meta-constructor, building for each task a symbolic encoder–decoder: an Induction module compresses the support examples into a schema 𝜙 (encoder) and a transient scaffold 𝑧; a Deduction module derives rule 𝑇 (decoder) from these and discards 𝑧; an Implementer compiles (𝜙, 𝑇) into artifacts; an error-driven Refiner revises (𝜙, 𝑇) and regenerates artifacts from scratch. Only (𝜙, 𝑇) crosses stage boundaries, so all refinement stays anchored to the rule. We evaluate Hourglass across three benchmarks spanning visual abstraction, hardware synthesis, and textual rule induction, using GPT-5.5 and Gemini 3.1 Pro. On ARC-AGI-2, it raises best-of-5 accuracy by up to 14 points over an iterative-refinement baseline. On ChipBench, it nearly doubles Verilog synthesis accuracy with GPT-5.5, from 31% to 58%. BBEH-Linguini draws on puzzles from the International Linguistics Olympiad, a setting where prior work has shown that explicit verbalization can hurt performance. Hourglass mitigates this tendency, and on Gemini 3.1 Pro, it reverses the effect entirely. Ablations confirm that these gains come from the isolation between stages and the quality of the initial induction, not from prompt wording or the particular symbolic form used. It is how information flows through the reasoning process, rather than the language used to express it, that drives inductive reasoning in frozen LLMs.

1

Introduction

Humans can extract abstract rules from only a handful of examples. In artificial intelligence, this capability is rigorously tested by benchmarks like ARC-AGI-2 (Chollet, 2019), where each puzzle is governed by a single, precise transformation rule that must be inferred from few-shot demonstrations. Despite impressive ∗ Code and prompts: https://github.com/ZhuHuan09/hourglass-reasoning

1

performance on many natural-language tasks, current frontier LLMs still struggle with such rule-centric induction. Large language models have demonstrated remarkable capabilities across a broad spectrum, extending from natural language processing and code generation to hardware description language synthesis (Liu, Y., et al., 2025) and abstract spatial reasoning on benchmarks such as ARC-AGI-2 (Franzen et al., 2025). However, these models remain prone to shortcut learning (Geirhos et al., 2019): instead of abstracting the latent rule, a model exploits superficial regularities in the support examples. When execution feedback is available, this often manifests as patchwork logic: hardcoded if-else branches keyed to specific coordinates, example indices, or local artifacts. Such patches force support examples to pass but fail on out-of-distribution queries (Moskvichev et al., 2023; Mitchell et al., 2023). Moreover, a growing body of recent work demonstrates that the intrinsic self-correction capabilities of monolithic LLMs are brittle and frequently degrade performance in the absence of external structural guidance (Tsui, 2025; Sanz-Guerrero & Von Der Wense, 2025). These pathologies are partly rooted in how information flows through dense, unrestricted context windows. When raw examples, current artifacts, error feedback, and repair instructions coexist without structured partition, the model tends to anchor on low-level perceptual details rather than generalizing to an abstract rule. Standard self-refinement treats refinement as artifact-level editing: it changes the current code directly, without maintaining an explicit symbolic rule as the persistent target of repair (Madaan et al., 2023). Simply prompting the model to “think step by step” or “explain the rule first“ within the same context offers only a soft constraint: when feedback arrives, the model can abandon its earlier rule and patch the code directly (Huang et al., 2023). Figure 1 contrasts this monolithic failure mode with the alternative we propose: routing refinement through an explicit symbolic bottleneck rather than editing the output directly. To address this, we impose a structured bottleneck on inductive reasoning: Hourglass reasoning.The LLM acts as a meta-constructor that, for each task, dynamically builds a symbolic encoder–decoder. A schema 𝜙 serves as the encoder that parses inputs, and a transformation rule 𝑇 serves as the decoder that generates outputs. Crucially, context isolation between stages ensures that only the compressed symbolic state (𝜙, 𝑇) passes forward, while all intermediate reasoning traces—including a transient scaffold 𝑧—are discarded. This forces all downstream implementation and refinement to operate exclusively through the rule, preventing instance-specific details from leaking across stages. The full architecture is presented in Section 3. We evaluate Hourglass on three benchmarks spanning visual abstraction, formal hardware synthesis, and textual rule induction (rationale detailed in §4.1). Across all three, the same domain-agnostic pipeline yields substantial and consistent gains over a context-reset Self-Refine baseline. Ablation studies on ARC-AGI-2 Monolithic Baseline (Self-Refine) Few-shot Examples

(x,y)

Dense Context

Hourglass Reasoning (Ours)

Predicted Output

Few-shot Examples

symbolic State

(x,y)

z

Predicted Output

T

Shortcut

Symbolic Refine (update

Unconstrained Refine

and T )

Figure 1: Monolithic self-refinement vs. Hourglass Reasoning. Left: Self-Refine maps few-shot examples (𝑥, 𝑦) to 𝑦ˆ through a single dense context; entangled dependencies enable shortcut solutions (red) that fit the demonstrations without recovering the underlying rule. Right: Hourglass Reasoning compresses the examples into an explicit symbolic state, a schema 𝜙 and a transformation rule 𝑇, through an encoder-decoder-style reasoning topology. All intermediate reasoning, including the transient scaffold 𝑧, expires within isolated stage contexts; only (𝜙, 𝑇) is passed forward. Refinement (green) revises (𝜙, 𝑇) rather than the output directly.

2

reveal a striking robustness: so long as the core workflow topology is preserved, the gains remain stable even when all auxiliary prompt directives are stripped away. Structured self-refinement that outputs identical symbolic intermediates but lacks context isolation performs substantially worse, confirming that the topology itself, not prompt engineering or output formatting, is the key contributing factor. Our contributions are: 1. A structured bottleneck for inductive reasoning, realized through role-isolated symbolic state passing. Ablations show that isolation between stages is necessary for the gains to hold, while the specific symbolic vocabulary used to express (𝜙, 𝑇) can vary substantially without much loss in performance, indicating that how information is allowed to flow through the reasoning process, more than the particular form it takes, is what drives the improvement. 2. Cross-domain empirical evidence. The same domain-agnostic pipeline yields consistent, substantial gains on three unsaturated benchmarks spanning visual abstraction, formal hardware synthesis, and anti-prior linguistic deduction. 3. Ablations that isolate the key contributing factor. We show that structured self-refinement alone fails to reproduce the gains, while a minimal role-isolated variant preserving only the (𝜙, 𝑧, 𝑇) topology remains equally strong, demonstrating that the workflow topology—not prompt engineering—drives the improvement.

2

Related Work

2.1

Test-time computation and self-refinement

Allocating additional test-time compute via sampling, search, or verification can improve LLM outputs (Snell et al., 2024). Aggregation methods such as self-consistency reduce variance by marginalizing over reasoning paths (Wang et al., 2023). However, unconstrained self-refinement can amplify local mistakes when each iteration edits the current artifact without an explicit abstract state (Madaan et al., 2023; Huang et al., 2023). In fact, several recent studies report that naive self-refinement can even degrade performance due to context contamination and premature convergence (Sanz-Guerrero et al., 2025; Tsui et al., 2025). The intuition that discarding instance-specific detail can improve generalization is closely related to the information-bottleneck perspective we adopt below (§2.3); Hourglass translates this intuition into a practical symbolic-state workflow rather than a mathematically enforced compression. A separate line of work pursues a complementary strategy: instead of refining within a single episode, these methods accumulate reusable problem-solving strategies across episodes at test time, without updating model weights, such as a persistent, self-curated memory of strategies and code snippets (Suzgun et al., 2026) or a library of procedural skills distilled from past trajectories (Yang et al., 2026). These approaches typically accumulate unstructured or loosely structured natural-language strategies within a single model; we return to this contrast in §6.3.

2.2

Multi-agent systems and context isolation

Multi-agent frameworks improve performance by decomposing reasoning across specialized roles (Hong et al., 2023; Qian et al., 2023). Yet role separation alone does not guarantee context isolation (Du et al., 2023; Yin et al., 2023): when agents communicate via high-dimensional natural-language traces, irrelevant or stale signals can propagate. Hourglass adopts a stricter approach: each role is an isolated API call with no

3

shared conversational history, and only selected symbolic artifacts, 𝜙 and 𝑇, are explicitly passed forward, while all intermediate reasoning traces and transient scaffolds are compartmentalized. This demonstrates that explicit, typed context isolation within a single-model pipeline can capture much of the benefit attributed to multi-agent dynamics while reducing coordination overhead and contamination risk.

2.3

Information Bottleneck Theory

The Information Bottleneck (IB) principle posits that optimal representations retain task-relevant information while discarding instance-specific details (Tishby et al., 1999). This idea has been concretely realised in several classical architectures. Denoising autoencoders explicitly corrupt inputs during training, forcing models to recover robust features instead of exploiting superficial statistics (Vincent et al., 2008). Variational autoencoders (VAEs) further impose a Gaussian prior on the latent space, regularising representations into a continuous, structured manifold (Kingma & Welling, 2014). In tasks requiring strong spatial or geometric regularity, such as continuous control, this distributional constraint is often sharpened into a topological one: latent variables are restricted to the unit hypersphere via von Mises-Fisher priors, eliminating scale degrees of freedom and markedly improving out-of-distribution generalisation (Davidson et al., 2018). These varied instantiations share a common insight: deliberately restricting the expressivity or geometry of intermediate representations severs the propagation of irrelevant variation, thereby strengthening generalisation.

2.4

Program synthesis and neuro-symbolic abstraction

Neuro-symbolic approaches have long leveraged explicit symbolic intermediates to improve generalization in program synthesis (Wong et al., 2021). Systems like DreamCoder learn libraries of reusable functions that serve as a discrete bottleneck, forcing the model to re-express solutions in terms of abstract primitives (Ellis et al., 2021). Hourglass follows a similar intuition but operates at the level of natural language: instead of learning a formal DSL, it prompts the LLM to generate an ad-hoc symbolic schema (𝜙) and transformation rule (𝑇) for each task, then uses these as the sole persistent state for refinement and regeneration. This design retains the benefits of a discrete intermediate without requiring a pre-defined grammar or training phase, thus remaining applicable across a wide range of domains.

3

Methodology: Hourglass Reasoning

Figure 2 gives an overview of the resulting pipeline; we describe each stage in turn below.

3.1

Design Rationale

Hourglass distills the information-bottleneck principle into a structural partition of the inductive reasoning process. The central idea is to isolate the extraction of an input–output rule from its subsequent execution and debugging, with a symbolic interface serving as the sole channel between stages. By forcing the model to commit to an explicit schema 𝜙 (how to parse inputs) and a declarative transformation rule 𝑇 (how to map parsed structures to outputs), and by discarding all transient reasoning traces, we create a discrete bottleneck. This ensures that implementation and refinement operate exclusively on abstract, domain-level specifications rather than on raw perceptual details or ad-hoc patches. Each stage is a stateless, isolated API call; only 4

Few-shot Examples

Step1: Induction pass S

input

In: Support Set S Out: State z + Schema Rules: Segment objects by functional role; induce parametric invariants

pass z and

Step2: Deduction In: S + Schema + State z Out: Transformation Rule T Rules: Deduce transformation as pure functional mapping

pass

pass T

output

Step3: Implementation Predicted Output

Execute P

pass if error

×

Execute

In: Support Set S + Schema + Rule T Out: Text Plan and Python Script P Rules: Deterministic compilation of ( , T ) into code

Step4: Schema-Anchored Refinement In: Support Set S + Schema + Rule T + Artifacts P Out: Updated + + Rules: Root-cause diagnosis; regenerate artifacts from the corrected specification.

Figure 2: The four-stage Hourglass Reasoning pipeline (illustrated for ARC-AGI-2). green boxes (Steps 1–3) are the one-shot initialization stage; purple box (Step 4) is the iterative refinement loop; yellow boxes represent data (support set 𝑆 and predicted output 𝑦ˆ ). Step 1 (Induction) produces a schema 𝜙 and a transient scaffold 𝑧. Step 2 (Deduction) derives the transformation rule 𝑇 from 𝑆, 𝜙, and 𝑧, then discards 𝑧. Step 3 (Implementation) deterministically compiles (𝜙, 𝑇) into executable artifacts 𝑃. If errors occur, Step 4 (Schema-Anchored Refinement) revises (𝜙, 𝑇) and regenerates 𝑃 in a fresh context; corrections are anchored to the symbolic state rather than patched onto the output.

(𝜙, 𝑇), the support set 𝑆, and the current artifacts are forwarded, while all intermediate scaffolds and error histories are contained within their respective sessions. The following subsection enumerates the concrete components that realize this design.

3.2

Components and Information Flow

The persistent state carried across iterations is the pair (𝜙, 𝑇). The scaffold 𝑧 is transient: it aids the derivation of 𝑇 but is purged before any downstream processing. All three are expressed in lightly structured natural language. For an ARC-AGI-2 task, 𝜙 might capture rules for extracting connected components and their relative bounding boxes; 𝑧 would enumerate the specific components found in each support grid; 𝑇 would specify the geometric transformation applied to each component. From this symbolic specification, the pipeline produces executable artifacts 𝑃. Depending on the benchmark, 𝑃 consists of either code-and-text or text-only depending on the task. 𝑃 is regenerated from scratch whenever (𝜙, 𝑇) is updated, but is not itself part of the persistent symbolic state. Hourglass is realized through four stages, each an isolated API call with no shared conversational history. The support set 𝑆 is supplied to every call as a stable factual ground; all other intermediate reasoning traces—draft scaffolds, justifications, rejected code versions, and prior error logs—are isolated within their respective sessions. 5

Stage

Input

Output

Role

Induction

𝑆

𝜙, 𝑧

Deduction

𝑆, 𝜙, 𝑧

𝑇

Implementation

𝑆, 𝜙, 𝑇

𝑃

Refinement

𝑆, 𝜙, 𝑇, 𝑃, 𝑦ˆ

𝜙new , 𝑇new , 𝑃new

Generate a parsing schema 𝜙 and a transient scaffold 𝑧 from the support examples. Derive a transformation rule 𝑇 from 𝜙 and 𝑧; 𝑧 is permanently discarded after this step. Compile (𝜙, 𝑇) into executable code-and-text or text-only artifacts. Revise (𝜙, 𝑇) by comparing predicted outputs 𝑦ˆ against ground-truth 𝑦 on the support set; regenerate 𝑃.

Table 1: Stages, inputs, outputs, and roles in Hourglass.

In addition to the context isolation, one auxiliary design choice supports interpretability: Induction and Deduction use structured output templates with labeled fields and origin-independence triggers (e.g., “use relative coordinates”), keeping 𝜙 and 𝑇 explicit and auditable across iterations. This is not a mechanism we claim to be load-bearing in itself; §5 examines its actual contribution.

3.3

Iterative Execution–Feedback Loop

Starting from Induction and Deduction, after 𝑧0 is discarded, the Implementer produces initial artifacts 𝑃0 . A maximum of 𝑀 refinement iterations are allowed. In each iteration, the Executor runs the current artifacts on the support inputs and produces predicted outputs 𝑦ˆ 𝑡 . These predictions are passed to the Refiner alongside the ground-truth outputs 𝑦 of the support set, together with the current symbolic state and artifacts: (𝑆, 𝜙𝑡 , 𝑇𝑡 , 𝑃𝑡 , 𝑦ˆ 𝑡 , 𝑦). The Refiner produces an updated symbolic state (𝜙𝑡+1 , 𝑇𝑡+1 ) and regenerated artifacts 𝑃𝑡+1 in a fresh context. The loop halts as soon as all support examples are perfectly matched ( 𝑦ˆ 𝑡 = 𝑦). For a novel query 𝑥 𝑞 , the final compiled artifact instantiates a fresh per-input state by applying 𝜙 to 𝑥 𝑞 , then applies 𝑇 to produce the output. In the current instantiation, the symbolic state (𝜙, 𝑇) is discarded once a task is solved and re-derived from scratch for the next; §6.3 discusses the possibility of extending it into a persistent, cross-task form.

4

Experiments

4.1

Common Setup

Models and infrastructure. All experiments use two proprietary LLMs: GPT-5.5 and Gemini 3.1 Pro. GPT-5.5 uses medium reasoning effort; Gemini 3.1 Pro uses high reasoning effort. Each module that produces syntactically invalid output is retried up to three times. Baseline. The primary baseline is Self-Refine (Madaan et al., 2023), a widely adopted iterative refinement method. In each refinement step, the model receives the support set, the current artifact, and the latest validation feedback within a single context, and then outputs a revised artifact. Both Hourglass and SelfRefine use at most five refinement iterations. No earlier rejected drafts or hidden reasoning traces are retained. Evaluation protocol. For binary-correctness benchmarks, pass@1 is the fraction of tasks solved on the first mandatory run, and pass@5 is the fraction solved by any of up to five independent runs. On ARC-AGI-2, we

6

follow the official public evaluation protocol and report the averaged score; full details of the competitionmode configuration are provided in Appendix A. Test ground truth is used exclusively in the final offline evaluation. Cost estimates are based on official API pricing, using token counts logged during runs. Benchmark selection rationale. We deliberately selected three benchmarks spanning a gradient of inductive reasoning demands: ARC-AGI-2 (Chollet, 2019), requiring pure visual rule discovery from minimal examples with no prior domain knowledge; ChipBench (Yu et al., 2026), a zero-tolerance benchmark for hardware logic synthesis under dynamically generated test vectors; and BBEH-Linguini, the Linguistics Olympiad subset of BIG-Bench Extra Hard (Kazemi et al., 2025), a setting where prior work has shown that forcing LLMs to explicitly verbalize induced rules can degrade performance (Goyal & Dan, 2025). Full task descriptions are given in §4.2–§4.4 respectively. Together, these three benchmarks probe complementary facets of inductive reasoning: sparse visual rule discovery, knowledge-rich formal synthesis, and prior-sensitive textual induction.

4.2

ARC-AGI-2

ARC-AGI-2 evaluates few-shot spatial induction: each puzzle presents input–output grid pairs governed by a single, precise transformation rule, and the solver must infer this rule from the demonstrations alone. We use the public 120-puzzle evaluation set and follow the official protocol, where each task contains multiple test cases. A solution is scored by the fraction of test cases perfectly matched (pixel-for-pixel), and the overall score is the average across all tasks. Setup. The primary baseline is a context-reset Self-Refine loop aligned with Hourglass on task definitions, execution feedback format, and refinement limit. It directly outputs executable code without any structured intermediate representation. To isolate the effect of output format, a Structured Self-Refine ablation (§5) is also reported. Results. Table 2 reports the official averaged scores. Model

Self-Refine

Hourglass

Δ (Hourglass − Self-Refine)

GPT-5.5 Gemini 3.1 Pro

51.9 / 62.8 54.4 / 76.9

60.6 / 76.8 62.4 / 86.7

+8.7 / +14.0 +8.0 / +9.8

Table 2: ARC-AGI-2 score (pass@1 / pass@5, %).

Analysis. Hourglass substantially outperforms Self-Refine on both models. A plain variant that removes all auxiliary prompts while preserving the isolated (𝜙, 𝑧, 𝑇) topology achieves 79.3% best-of-5 on GPT-5.5 and 87.5% on Gemini (§5), indicating that role-isolated symbolic state passing, not prompt wording, is the key contributor. A competition-mode configuration exploiting the official two-submission protocol further improves the cost– accuracy frontier, reaching 88% accuracy at approximately $4 per task, with an oracle variant attaining 95% at roughly $11 per task. Full details and comparisons against leaderboard baselines are provided in Appendix A.

4.3

ChipBench: Hardware Logic Synthesis

ChipBench tests hardware reasoning under zero-tolerance verification. The benchmark provides naturallanguage design specifications that vary in their few-shot support: some include full input–output examples, others only fragments or natural language descriptions. We evaluate both core sub-tasks: Reference Model 7

Generation (Python) and Verilog Synthesis (synthesizable RTL). The 45 specifications span self-contained modules, CPU-related IP blocks, and hierarchical designs. A solution is correct only if it passes 100 dynamically generated random test vectors. Setup. To create a uniform information condition, a preliminary LLM call reorganizes the raw specification into a Symbolic Codebook containing behavior tables, state transitions, priority rules, and bit-width constants. The Native Baseline (the official baseline provided by the ChipBench project) operates directly on the raw specification, while Codebook-SR and Hourglass each independently apply the same codebookconstruction prompt before downstream reasoning. The resulting codebooks are structurally similar but independently sampled, ensuring that performance differences stem from the reasoning workflow rather than asymmetric preprocessing. Results. Tables 3 and 4 summarize the pass@1 and pass@5 scores. Model GPT-5.5 Gemini 3.1 Pro

Native Baseline

Codebook-SR

Hourglass

Δ best baseline

60.0 / 66.7 53.3 / 62.2

55.6 / 64.4 55.6 / 57.8

73.3 / 82.2 75.6 / 82.2

+13.3 / +15.5 +20.0 / +20.0

Table 3: ChipBench Reference Model Generation (pass@1 / pass@5, %). Model GPT-5.5 Gemini 3.1 Pro

Native Baseline

Codebook-SR

Hourglass

Δ best baseline

31.1 / 35.6 40.0 / 44.4

44.4 / 51.1 51.1 / 57.8

57.8 / 66.7 53.3 / 62.2

+13.4 / +15.6 +2.2 / +4.4

Table 4: ChipBench Verilog Synthesis (pass@1 / pass@5, %).

Analysis. Hourglass outperforms the strongest baseline in all four settings. The codebook alone offers only limited benefit; the architectural bottleneck is the decisive factor. The gain of over 30 percentage points on Verilog synthesis with GPT-5.5 is particularly notable on this non-saturated benchmark. It suggests that the symbolic intermediate descriptions act as a compliance scaffold, enforcing exhaustive and consistent instantiation of known rules far more effectively than end-to-end code correction.

4.4

BBEH-Linguini: Textual Rule Induction

BBEH-Linguini, the Linguini subset of BIG-Bench Extra Hard (Kazemi et al., 2025), consists of puzzles drawn from the International Linguistics Olympiad. Each task provides a small set of language pairs; the solver must induce the underlying transformation rule and apply it to held-out queries. Executable code is explicitly disallowed—the model must generate and follow purely textual rules. Solving these puzzles effectively draws on structural knowledge of linguistic categories (e.g., gender, number, case). Recent work has identified a structural weakness in LLMs’ explicit reasoning on such tasks (Lian et al., 2025, Choudhary et al., 2025), rendering BBEH a stress-test for pure symbolic abstraction. Setup. The original problems are presented in diverse, unstandardized formats. To enable automated execution feedback via exact string matching, we used an LLM to extract each problem’s demonstration pairs and test questions into a standardized JSON structure. All extractions were manually verified for characterlevel fidelity (full protocol in Appendix C), yielding a clean set of 144 tasks. The Raw Prompt baseline uses the original problem stem directly, without any intermediate rule extraction or refinement. During refinement, feedback is computed by exact-string comparison between predicted outputs and ground-truth answers on the support items; test answers are used exclusively for final pass@k scoring. Because code execution is unavailable, the Executor for Self-Refine and Hourglass is replaced by a separate Inferer agent that takes the induced rule and applies it to test inputs in a purely textual manner, analogous to an interpreter 8

executing code (details in Appendix C.2). The Implementer produces a textual derivation plan instead of executable code; the rest of the Hourglass topology is unchanged. Results. Table 5 summarizes the findings. Model

Raw Prompt

Self-Refine

Hourglass

Δ (Hourglass − Raw Prompt)

GPT-5.5 Gemini 3.1 Pro

58.3 / 67.4 64.6 / 68.1

25.0 / 27.8 32.6 / 33.3

46.5 / 63.1 63.1 / 79.9

−11.8 / −4.3 −1.5 / +11.8

Table 5: BBEH-Linguini results (pass@1 / pass@5, %).

Analysis. Consistent with prior observations, LLMs struggle with explicit rule induction in this setting. Self-Refine severely degrades performance on both models, often falling below even the Raw Prompt. This collapse can be attributed to two intertwined mechanisms. First, lossy compression: forcing an LLM to verbalize its inductive insight as a natural-language rule discards subtle but critical details, and each additional refinement step compounds the information loss (Gu et al., 2024). Second, context entanglement: in the monolithic Self-Refine loop, rule generation, execution, and error feedback coexist in a single context, making the model prone to attention distraction (Shi et al., 2023; Liu, N. F., et al., 2024) and inconsistent rule application (Dahl et al., 2024; Zhang et al., 2025). Unlike Self-Refine, Hourglass roughly recovers the performance of the Raw Prompt on GPT-5.5 and surpasses it by 11.8% at pass@5 on Gemini 3.1 Pro. This suggests that the structured bottleneck may mitigate the lossy compression and context entanglement inherent in explicit rule induction, yielding rules that are sufficiently faithful for a generic Inferer to execute reliably.

5

Analysis

Hourglass introduces three architectural components beyond the monolithic Self-Refine baseline: a compression stage (Induction, producing 𝜙 and 𝑧), a reconstruction stage (Deduction, deriving 𝑇 and discarding 𝑧), and an isolated Refiner that revises (𝜙, 𝑇) rather than editing code directly. The prompts driving these stages each contain three classes of information (see Appendix D): a task description, auxiliary guiding prompts, and constraints on the output format of 𝜙 and 𝑇. The observed gains could therefore originate from the auxiliary prompts, from the hand-designed output format, from the mere presence of explicit symbolic intermediates, or from the physically isolated workflow topology itself. To isolate the key contributing factor, we conduct five ablations on ARC-AGI-2, each disabling one candidate mechanism while preserving the others (Table 6). Weak Initialization (Weak-Init). Purpose. If the Refiner independently drove the gains, substituting a weak Induction/Deduction model should still recover strong performance; if not, the initial bottleneck quality limits what refinement can achieve. Setup. The Induction, Deduction, and Implementation modules are replaced with Gemini 2.0 Flash, while the Refiner continues to use the strong backbone. Code-Only Refinement (Code-Only). Purpose. If (𝜙, 𝑇) is the indispensable carrier of the bottleneck, removing it from the Refiner should substantially degrade performance. Setup. The Refiner no longer receives 𝜙 and 𝑇; it sees only the current failing artifact, the support set, and validation feedback.

9

Hourglass-Plain (Plain). Purpose. If the gains originate from careful prompt engineering, removing all auxiliary instructions should impair performance. Setup. All auxiliary prompts (e.g., “ensure originindependence”) are stripped; only the minimal task description and output format constraints remain. Hourglass-Unstructured (Unstructured). Purpose. If the hand-designed output format of 𝜙 and 𝑇 is crucial, removing all formatting constraints should degrade accuracy. Setup. All constraints on the internal structure of 𝜙 and 𝑇 are lifted. Structured Self-Refine (Struct-SR). Purpose. If producing structured intermediate descriptions alone is sufficient, a monolithic Self-Refine loop that outputs the same (𝜙, 𝑇) as Hourglass should approach its performance; failure would indicate that physical context isolation is a necessary condition. Setup. The Self-Refine baseline is modified to output 𝜙 and 𝑇 in the same format, but all generation, execution, and feedback remain inside a single undifferentiated context window. Variant

GPT-5.5 p@1/p@5

Gemini 3.1 Pro p@1/p@5

Full Weak-Init Code-Only Plain Unstructured Struct-SR SR (baseline)

60.6 / 76.8 40.1 / 61.4 58.1 / 74.7 59.9 / 79.3 58.6 / 76.4 41.5 / 59.9 51.9 / 62.8

62.4 / 86.7 54.3 / 74.6 72.8 / 79.9 70.1 / 87.5 73.6 / 89.0 31.6 / 62.8 54.4 / 76.9

Δ vs. Full (GPT-5.5) (Gemini) — −20.5 / −15.4 −2.5 / −2.1 −0.7 / +2.5 −2.0 / −0.4 −19.1 / −16.9 −8.7 / −14.0

— −8.1 / −12.1 +10.4 / −6.8 +7.7 / +0.8 +11.2 / +2.3 −30.8 / −23.9 −8.0 / −9.8

Table 6: Ablation results on ARC-AGI-2 (pass@1 / pass@5, %). Negative Δ values indicate a performance drop relative to Full Hourglass.

Interpretation. The ablations converge on a clear narrative. Auxiliary prompts, output format constraints, and even the explicit textual form of 𝜙 and 𝑇 are not the source of the gains. Plain matches Full despite removing all auxiliary instructions; Unstructured matches Full despite removing formatting constraints; Code-Only incurs only a marginal drop despite removing 𝜙 and 𝑇 from the Refiner entirely. In each case the gain persists so long as the role-isolated topology is preserved, ruling out prompt engineering and template design as the enabling mechanism. Notably, the Code-Only variant, which removes 𝜙 and 𝑇 from the Refiner’s input, retains competitive performance, suggesting that the code, having been compiled from (𝜙, 𝑇), may implicitly retain the structure of the symbolic state. In contrast, physical context isolation and initial bottleneck quality are both necessary conditions. When isolation is removed (Struct-SR), structured intermediates actively impair performance, collapsing below the unconstrained SR baseline. When the initial compression is degraded (Weak-Init), the Refiner cannot compensate, and accuracy falls sharply. Neither a strong Refiner nor structured outputs alone suffice; the bottleneck must be both physically enforced and initially competent. Taken together, these results identify the role-isolated symbolic topology as the key contributing factor. In this topology, induction, deduction, and execution are separated across fresh contexts, and the transient scaffold is discarded after use. The topology is robust to substantial variation in prompting and output format, yet critically depends on the two conditions identified above. On Gemini 3.1 Pro, Full Hourglass yields a lower pass@1 than Plain, Unstructured, and Code-Only, but recovers or surpasses them at pass@5. This pattern may reflect a trade-off: a stricter bottleneck may lock

10

in early errors, reducing single-trial success, while the resulting abstraction quality pays off across multiple samples.

6

Discussion

6.1

Core Findings

Hourglass demonstrates that a simple structural reorganization of inference yields consistent and substantial gains across visual, hardware synthesis, and linguistic reasoning domains. This reorganization enforces a bottleneck: information must pass through it as a compressed symbolic state. Ablation evidence (§5) identifies the role-isolated topology, rather than prompt wording or output format, as the key contributing factor, contingent on two necessary conditions: physical isolation between stages and a sufficiently competent initial induction. Beyond these performance gains, the explicit symbolic intermediates (𝜙, 𝑇) offer transparency into the model’s abstraction process, providing an interpretable reasoning trace with independent value for debugging and analysis. The domain-agnostic nature of Hourglass, its avoidance of task-specific priors, and its efficiency align with the original design intent of ARC-AGI as a benchmark of general fluid intelligence.

6.2

Limitations

1. Soft bottleneck. 𝜙 and 𝑇 are natural language descriptions whose separation from implementation is enforced through prompting and context isolation rather than architectural constraints. This prompt-level design makes the method plug-and-play across frozen models, but it also means the bottleneck carries no formal guarantee. 2. Task scope. The current instantiation is designed for tasks with precise, deterministic rules; extending the approach to probabilistic, ambiguous, or context-dependent regularities remains an open direction. 3. Computational overhead. Role isolation requires disjoint API calls, increasing token usage and wallclock latency. On GPT-5.5, a single run averages 61,911 tokens across 3.7 calls, compared to 19,594 tokens over 1.4 calls for the monolithic Self-Refine baseline—roughly a 3× increase; on Gemini, 122,007 tokens (4.4 calls) vs. 61,426 tokens (1.7 calls), approximately a 2× increase. Our main comparisons are therefore not compute-matched. 4. Model coverage and evaluation protocol. Our experiments are limited to two model families, and pass@5 estimates use early stopping, which may introduce a slight optimistic bias relative to a fully budgeted sweep. We did not conduct budget-matched runs due to resource constraints, though the magnitude of the observed gains suggests this is unlikely to be the dominant factor. 5. Public evaluation set. ARC-AGI-2 uses a public evaluation set; the possibility of pretraining exposure remains a shared limitation of all methods evaluated on this benchmark.

6.3

Future Directions

In its current form, Hourglass is an ephemeral, per-task reasoning scaffold, strictly limited to precise, deterministic logic. Each task requires the system to induce its rules from scratch, and the scaffolding is discarded once solved. Humans, however, can learn not just case-specific solutions but transferable meta-strategies that travel across domains. Consider a meteorologist faced with a novel time series of atmospheric pressure readings. Without 11

task-specific training, she draws on the same mental routine: look for periodicities, segment the signal into regimes, identify anomalies. Later, given an unfamiliar economic indicator, she deploys that routine again. These meta-strategies do not prescribe exact operations, nor do they guarantee correctness, yet they systematically elevate her analysis above random guessing. Can a model be built to learn such meta-strategies? Recent RLVR-trained models acquire reasoning primitives for iterative hypothesis generation and self-correction, but only implicitly, as opaque byproducts of largescale training rather than explicit, reusable templates (OpenAI, 2024; DeepSeek-AI, 2025; Wen et al., 2026). Test-time accumulation methods maintain explicit strategies, but typically as unstructured memory stores matched by similarity, which limits generalization to novel abstract rules (Suzgun et al., 2026; Yang et al., 2026). Hourglass, by contrast, discovers and refines explicit symbolic strategies, yet its strategies remain confined to single tasks requiring precise transformation logic. What if this process were scaled across diverse problems? A Meta-Hourglass would accumulate strategies over time through repeated search and feedback, learning reusable meta-strategies within a domain—even for tasks that lack precise transformation logic. Such a system might, for instance, acquire strategies from geometry puzzles and, with extra scaling, extend them to algebra or beyond mathematics. Given a novel task, it would deploy those strategies zero-shot. They would not prescribe exact operations, nor would they guarantee correctness, but, like the meteorologist’s, they would shift the baseline away from the arbitrary. Whether such meta-reasoning can be learned remains open, but the possibility is now visible.

Acknowledgements The author thanks Yi-Fei Liu and Ya-Hua Li for their assistance in verifying the ARC workflow software. Computing resources used in the early stages of this project were provided by Peking University.

References 1. Alemi, A. A., Fischer, I., Dillon, J. V., & Murphy, K. (2017). Deep variational information bottleneck. International Conference on Learning Representations (ICLR 2017). arXiv:1612.00410 2. Chen, L., Li, Z., Lyu, K., Peng, B., & Wu, H. (2026). The information bottleneck of chain-of-thought and how latent CoT overcomes it. International Conference on Learning Representations (ICLR 2026). 3. Chollet, F. (2019). On the measure of intelligence. arXiv. arXiv:1911.01547 4. Choudhary, M., Srivatsa, K. V. A., Aeron, G., Bhattacharya, A. R., Dinh, D. K. D., Hanif, I. A., Kotova, D., Kochmar, E., & Choudhury, M. (2025). UNVEILING: What makes linguistics olympiad puzzles tricky for LLMs? arXiv. arXiv:2508.11260 5. Dahl, M., Magesh, V., Suzgun, M., & Ho, D. E. (2024). Large legal fictions: Profiling legal hallucinations in large language models. Journal of Legal Analysis, 16(1), 64–93. arXiv:2401.01301 6. Davidson, T. R., Falorsi, L., De Cao, N., Kipf, T., & Tomczak, J. M. (2018). Hyperspherical variational autoencoders. arXiv preprint arXiv:1804.00891. arXiv:1804.00891 7. DeepSeek-AI. (2025). DeepSeek-R1: Incentivizing reasoning capability in LLMs via reinforcement learning. arXiv. arXiv:2501.12948 8. Du, Y., Li, S., Torralba, A., Tenenbaum, J. B., & Mordatch, I. (2023). Improving factuality and reasoning in language models through multiagent debate. International Conference on Machine Learning (ICML 2023). arXiv:2305.14325

12

9. Ellis, K., Wong, C., Nye, M., Sablé-Meyer, M., Cary, L., Morales, L., Hewitt, L., Solar-Lezama, A., & Tenenbaum, J. B. (2021). DreamCoder: Bootstrapping inductive program synthesis with wake-sleep library learning. ACM-SIGPLAN Symposium on Programming Language Design and Implementation (PLDI 2021). DOI: 10.1145/3453483.3454080 10. Franzen, D., Disselhoff, J., & Hartmann, D. (2025). Product of experts with LLMs: Boosting performance on ARC is a matter of perspective. International Conference on Machine Learning (ICML 2025). arXiv:2505.07859 11. Fraser-Taliente, K., Kantamneni, S., Ong, E., Mossing, D., Lu, C., Bogdan, P. C., Ameisen, E., Chen, J., Kishylau, D., Pearce, A., Tarng, J., Wu, A., Wu, J., Zhang, Y., Ziegler, D. M., Hubinger, E., Batson, J., Lindsey, J., Zimmerman, S., & Marks, S. (2026). Natural language autoencoders produce unsupervised explanations of LLM activations. Transformer Circuits Thread. 12. Geirhos, R., Rubisch, P., Michaelis, C., Bethge, M., Wichmann, F. A., & Brendel, W. (2019). ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness. International Conference on Learning Representations (ICLR 2019). arXiv:1811.12231 13. Goyal, S., & Dan, S. (2025). IOLBench: Benchmarking LLMs on linguistic reasoning. arXiv. arXiv:2501.04249 14. Gu, Y., Tafjord, O., Kim, H., Moore, J., Le Bras, R., Clark, P., & Choi, Y. (2024). SimpleToM: Exposing the gap between explicit ToM inference and implicit ToM application in LLMs. arXiv. arXiv:2410.13648 15. Hong, S., Zhuge, M., Chen, J., Zheng, X., Cheng, Y., Wang, J., Zhang, C., Wang, Z., Yau, S. K. S., Lin, Z., Zhou, L., Ran, C., Xiao, L., Wu, C., & Schmidhuber, J. (2024). MetaGPT: Meta programming for a multi-agent collaborative framework. International Conference on Learning Representations (ICLR 2024). arXiv:2308.00352 16. Huang, J., Chen, X., Mishra, S., Zheng, H. S., Yu, A. W., Chi, E. H., & Le, Q. V. (2023). Large language models cannot self-correct reasoning yet. International Conference on Learning Representations (ICLR 2024). arXiv:2310.01798 17. Kazemi, M., Fatemi, B., Bansal, H., Palowitch, J., Anastasiou, C., Mehta, S. V., Jain, L. K., Aglietti, V., Jindal, D., Chen, P., Dikkala, N., Tyen, G., Liu, X., Shalit, U., Chiappa, S., Olszewska, K., Tay, Y., Tran, V. Q., Le, Q. V., & Firat, O. (2025). BIG-Bench Extra Hard. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (ACL 2025). arXiv:2502.19187 18. Kingma, D. P., & Welling, M. (2014, April). Auto-encoding variational bayes. In Proceedings of the International Conference on Learning Representations (ICLR). arXiv:1312.6114 19. Lei, S., Cheng, Z., Jia, K., & Tao, D. (2025). Revisiting LLM reasoning via information bottleneck. arXiv. arXiv:2507.18391 20. Li, X., & Eisner, J. (2019). Specializing word embeddings (for parsing) by information bottleneck. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing (EMNLP 2019). arXiv:1910.00163 21. Lian, D.-C., Huang, R.-S., Chen, P.-E., Lim, C., Lin, Y.-K., Tseng, G.-Y., Yang, T.-C., Lin, Z.-Y., Chen, P.-C., & Hsieh, S.-K. (2025). LingBench++: A linguistically-informed benchmark and reasoning framework for multi-step and cross-cultural inference with LLMs. arXiv. arXiv:2507.16809 22. Lin, Z.-L., Shih, Y.-F., & Hsieh, S.-K. (2025). Probing large language models in reasoning and translating complex linguistic puzzles. arXiv. arXiv:2502.00817 23. Liu, N. F., Lin, K., Hewitt, J., Paranjape, A., Bevilacqua, M., Petroni, F., & Liang, P. (2024). Lost in the middle: How language models use long contexts. Transactions of the Association for Computational Linguistics, 12, 277–294. arXiv:2307.03172 24. Liu, Y., Xu, C., Zhou, Y., Li, Z., & Xu, Q. (2025). DeepRTL: Bridging Verilog understanding and generation with a unified representation model. International Conference on Learning Representations (ICLR 2025). arXiv:2502.15832 25. Madaan, A., Tandon, N., Gupta, P., Hallinan, S., Gao, L., Wiegreffe, S., Alon, U., Dziri, N., Prabhumoye, S., Yang, Y., Gupta, S., Majumder, B. P., Hermann, K., Welleck, S., Yazdanbakhsh, A., & Clark, P. (2023). Self-refine: Iterative refinement with self-feedback. Advances in Neural Information Processing Systems 36 (NeurIPS 2023). arXiv:2303.17651

13

26. Mitchell, M., Palmarini, A. B., & Moskvichev, A. (2023). Comparing humans, GPT-4, and GPT-4V on abstraction and reasoning tasks. arXiv. arXiv:2311.09247 27. Moskvichev, A., Odouard, V. V., & Mitchell, M. (2023). The ConceptARC benchmark: Evaluating cognitive capabilities of language models. Transactions on Machine Learning Research. 28. OpenAI. (2024). Learning to reason with LLMs. OpenAI Blog. 29. Qian, C., Liu, W., Liu, H., Chen, N., Dang, Y., Li, J., Yang, C., Chen, W., Su, Y., Cong, X., Xu, J., Li, D., Liu, Z., & Sun, M. (2024). ChatDev: Communicative agents for software development. Annual Meeting of the Association for Computational Linguistics (ACL 2024). arXiv:2307.07924 30. Sanz-Guerrero, M., & Von Der Wense, K. (2025). Corrective in-context learning: Evaluating self-correction in large language models. Workshop on Insights from Negative Results in NLP (co-located with NAACL 2025). arXiv:2503.16022 31. Shi, F., Chen, J., Misra, K., Scales, N., Dohan, D., Chi, E. H., Schärli, N., & Zhou, D. (2023). Large language models can be easily distracted by irrelevant context. International Conference on Machine Learning (ICML 2023). arXiv:2302.00093 32. Snell, C., Lee, J., Xu, K., & Kumar, A. (2024). Scaling LLM test-time compute optimally can be more effective than scaling model parameters. arXiv. arXiv:2408.03314 33. Suzgun, M., Yuksekgonul, M., Bianchi, F., Jurafsky, D., & Zou, J. (2026). Dynamic Cheatsheet: Test-time learning with adaptive memory. Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (EACL 2026). arXiv:2504.07952 34. Tishby, N., Pereira, F. C., & Bialek, W. (1999). The information bottleneck method. Proceedings of the 37th Annual Allerton Conference on Communication, Control, and Computing. arXiv:physics/0004057 35. Tishby, N., & Zaslavsky, N. (2015). Deep learning and the information bottleneck principle. IEEE Information Theory Workshop. arXiv:1503.02406 36. Tsui, K. (2025). Self-correction bench: Uncovering and addressing the self-correction blind spot in large language models. arXiv. arXiv:2507.02778 37. Vincent, P., Larochelle, H., Bengio, Y., & Manzagol, P. A. (2008, July). Extracting and composing robust features with denoising autoencoders. In Proceedings of the 25th international conference on Machine learning (pp. 1096–1103). DOI: 10.1145/1390156.1390294 38. Wang, X., Wei, J., Schuurmans, D., Le, Q., Chi, E., Narang, S., Chowdhery, A., & Zhou, D. (2023). Self-consistency improves chain of thought reasoning in language models. International Conference on Learning Representations (ICLR 2023). 39. Wen, X., Liu, Z., Zheng, S., Xu, Z., Ye, S., Wu, Z., Liang, X., Wang, Y., Li, J., Miao, Z., Bian, J., & Yang, M. (2026). Reinforcement learning with verifiable rewards implicitly incentivizes correct reasoning in base LLMs. International Conference on Learning Representations (ICLR 2026). arXiv:2506.14245 40. West, P., Holtzman, A., Buys, J., & Choi, Y. (2019). BottleSum: Unsupervised and self-supervised sentence summarization using the information bottleneck principle. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing (EMNLP 2019). arXiv:1909.07405 41. Wong, C., Ellis, K., Tenenbaum, J. B., & Andreas, J. (2021). Leveraging language to learn program abstractions and search heuristics. International Conference on Machine Learning (ICML 2021). arXiv:2106.11053 42. Yang, M., Piao, J., Xia, X., Lan, X., Chen, J., Gong, Y., & Li, Y. (2026). SkillMaster: Toward autonomous skill mastery in LLM agents. arXiv. arXiv:2605.08693 43. Yin, Z., Sun, Q., Guo, Q., Wu, J., Qiu, X., & Huang, X. (2023). Do large language models know what they don’t know? Findings of the Association for Computational Linguistics: ACL 2023. arXiv:2305.18153 44. Yu, Z., Zhou, C., Lin, Y., Zhang, H., Ye, H., Cui, J., Pan, Z., Zhao, J., & Ding, Y. (2026). ChipBench: A next-step benchmark for evaluating LLM performance in AI-aided chip design. arXiv. arXiv:2601.21448

14

45. Zhang, Q., Wang, D., Qian, H., Li, Y., Zhang, T., Huang, M., Xu, K., Li, H., Yan, L., & Qiu, H. (2025). Understanding the dark side of LLMs’ intrinsic self-correction. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (ACL 2025). arXiv:2412.14959

15

A

Alignment with ARC-AGI-2 Competition Rules (Competition Mode)

The official ARC-AGI-2 leaderboard evaluates submissions under a strict Pass@2 metric: for each hidden test query, the system may submit exactly two predicted pixel grids, without access to test ground truth at any point during test-time computation. To align with this protocol, we implement a Competition Mode for Hourglass: independent runs of the full pipeline (Induction → Deduction → Execution → Refinement) are executed up to a maximum of 𝐾 times, and two candidates are selected via a consensus-and-ranking mechanism. Slot 1 is filled by the first prediction reproduced by 𝑁 independent runs (the consistency threshold; relaxed to 𝑁 = 2 if no consensus emerges within 25 runs), declaring early stopping. Slot 2 is filled by a dedicated Ranker agent—an auxiliary LLM call that scores all non-consensus candidates by support-set performance, the declared symbolic rules (𝜙, 𝑇), and internal reasoning consistency, selecting the top-ranked alternative. If no consensus is reached at all, the Ranker selects the two most promising candidates from the full pool. Due to resource constraints, we tested only the standard Hourglass configuration in Competition Mode, though certain ablation variants may outperform it on individual metrics. To characterize the compute–accuracy trade-off, we swept the consensus threshold (𝑁 = 2, 3, 4, with 𝐾 = 32, 60, 60 respectively) on the 120 public evaluation puzzles using Gemini 3.1 Pro. Results, evaluated with official partial-credit scoring, are summarized in Table 7. Metric

C-2

C-3

C-4

Max Runs Slot 1 Acc. Pass@2 Acc. Oracle Ceiling Avg. Cost (USD)

32 85.8% 87.8% 91.4% ∼$3.98

60 86.1% 87.8% 94.3% ∼$8.83

60 86.5% 86.9% 95.4% ∼$11.30

Table 7: Competition Mode evaluation and inference cost (ARC-AGI-2, Gemini 3.1 Pro). Slot 1 Acc. = consensus prediction accuracy; Pass@2 Acc. = Slot 1 ∪ Slot 2 accuracy; Oracle Ceiling = any single run correct.

The 𝑁 = 2 configuration offers the best cost–accuracy balance; raising the threshold to 𝑁 = 3 more than doubles cost with no accuracy gain, and 𝑁 = 4 increases cost further while accuracy slightly declines. This decline reflects a saturation effect: the Ranker is left choosing among an increasingly low-quality pool of non-consensus candidates as the sampling budget is exhausted pursuing a stricter consensus. Comparison to contemporary solvers. To contextualize the $3.98/task, 87.8% operating point, we compare it against contemporary test-time compute solvers on the ARC-AGI-2 leaderboard (as of mid-2026): Symbolica’s Agentica Framework reaches 85.28% at $6.94/task using Claude Opus 4.6; Imbue’s Code Evolution Solver reaches 95.1% at $8.71/task using extensive evolutionary mutations on Gemini 3.1 Pro; and Google’s Gemini 3 Deep Think reaches 84.6% at $13.62/task using extended reasoning tokens. Hourglass (𝑁 = 2) offers a competitive cost-performance trade-off for budget-constrained scenarios relative to these systems, achieving comparable accuracy to the more expensive baselines at a lower cost. We emphasize that this is an exploratory secondary result; our primary contribution is the reasoning workflow, not the competition-mode engineering. We note two caveats relevant to interpreting all numbers in this appendix. First, evaluation is conducted on the public evaluation set; possible pretraining exposure to this set is a shared limitation across all methods compared here, including the leaderboard baselines above, and absolute accuracy may not transfer to the private leaderboard. Second, due to the substantial cost of exhaustive sampling (exceeding $1,300 for a single full run at 𝑁 = 4), each configuration was evaluated with a single run rather than repeated sampling across seeds; the reported numbers are therefore point estimates rather than averages with confidence intervals.

16

B

Code Structure Diagnostics

We noticed that Hourglass-generated code tended to be more modular. To quantify this, we measured AST properties across all submitted solutions. Model

Method

Helper functions

Longest function (lines)

Dictionary literals

GPT-5.5

Hourglass Self-Refine Struct-SR

8.4 0.2 1.5

70.7 153.7 147.7

1.4 0.9 0.8

Gemini 3.1 Pro

Hourglass Self-Refine Struct-SR

0.3 0.1 0.1

119.0 121.4 95.9

0.8 0.5 0.5

Table 8: Structural code metrics by model and method (mean values).

On GPT-5.5, Hourglass induces substantially more modular code; on Gemini, structural differences are negligible, yet accuracy still improves markedly (86.7% vs. 76.9% best-of-5). Thus, code structure variation is not the primary driver—rule abstraction is.

C

BBEH-Linguini Implementation Details

This appendix details the data preprocessing pipeline (C.1) and the design of the text-based Inferer agent (C.2) used for BBEH-Linguini.

C.1

Data Preprocessing

The original BBEH-Linguini problems present few-shot demonstrations in heterogeneous, unstandardized formats. To enable automated execution feedback—which requires exact-string comparison of predicted outputs against ground-truth answers on the training pairs—the input and output of every training pair must be programmatically extractable. We therefore standardized all problems into a uniform JSON schema. We first used DeepSeek V4 Pro to segment each problem into a structured JSON object containing the problem stem, the list of training pairs, and the translation direction. The direction is implicitly encoded by assigning source sentences to the input field and target sentences to the output field; tasks that share the same stem may appear with opposite directions (e.g., low-resource language → English vs. English → low-resource language), which require distinct rules. The original benchmark contains 48 problems in which the few-shot items are not aligned as explicit input–output pairs. Instead, they present two unordered lists—one of sentences in language A and one in language B—and the solver must first infer the correspondence between them before any rule induction can begin. Because our pipeline relies on predefined input–output pairs at training time, these 48 problems were removed. To ensure the fidelity of the extracted JSON, we conducted a multi-stage verification. 1. Exact-string cross-check: every string in the JSON was verified to exist verbatim in the original problem text, ensuring no addition or distortion.

17

2. Triple segmentation and cross-validation: the same DeepSeek V4 Pro was used for a second independent segmentation, and Gemini 3.1 Pro performed a third. The three resulting JSON versions were submitted to DeepSeek V4 Pro for a consistency review. Any discrepancy was flagged. 3. Manual audit: human annotators inspected the segmented outputs for anomalies. Eight problems exhibited segmentation ambiguities that made it impossible to determine the correct label assignment (i.e., which language direction was intended). These were discarded. Starting from the full set of 200 problems, we removed 48 unordered-pair problems and 8 ambiguous problems, yielding a clean dataset of 144 problems. All three methods—Raw Prompt, Self-Refine, and Hourglass—receive identical JSON inputs built from this set.

C.2

Inferer Agent

In ARC-AGI-2 and ChipBench, execution feedback is obtained by running generated code. Because BBEHLinguini requires purely textual rules, we implemented a dedicated Inferer agent to “execute” textual rules by generating predicted outputs, analogous to a code interpreter. The Inferer receives the induced rule together with a set of training examples as few-shot prompts. To prevent the Inferer from bypassing the rule and relying solely on surface-level pattern matching, we adopt a held-out validation protocol. • For each run, two training pairs are randomly selected and their outputs are withheld. The Inferer is asked to produce predictions for these two withheld pairs (serving as validation items) and for one test item, all within the same inference pass. • If the predictions on both validation items match the ground-truth outputs exactly, the rule is considered to have been applied correctly on the seen distribution. To guard against accidental success, a second round is immediately triggered: two different training pairs are randomly withheld, and the Inferer again predicts both validation items and the test item. If the second round also yields exact matches on the validation pairs, the test prediction from the second round is taken as the final output. The test prediction from the first round is discarded. • If any validation pair fails in either round, the run is marked as unsuccessful and contributes to the denominator (but not the numerator) of the pass@k metric. This two-round design ensures that the final output derives from a single inference pass (the second round) without self-consistency voting or multiple sampling, keeping the evaluation protocol aligned with the Raw Prompt baseline. It also provides a stringent correctness check: the rule must generalize across different subsets of the training data before any test output is accepted. The same Inferer is used for both Hourglass and Self-Refine, ensuring that any performance differences originate entirely from the quality of the induced rules rather than from the execution mechanism.

D

Prompt Design

This appendix documents the prompt structure shared across methods, the differences between Hourglass and the Self-Refine baseline, and the rationale behind those differences. Every prompt consists of three components: (i) a task description, which specifies the input format, output requirements, and evaluation protocol; (ii) meta-instructions, which provide auxiliary guidance and warnings about common errors; and (iii) output format constraints, which prescribe how the model should structure its response.

18

D.1

Shared Framework and Key Differences

Both Hourglass and Self-Refine receive identical task descriptions for a given benchmark, ensuring that any performance difference cannot be attributed to asymmetric information about the problem itself. The two methods also share the same set of general meta-instructions that are unrelated to output formatting (e.g., “do not copy absolute coordinates,” “verify consistency across support examples”). The crucial divergence lies in the remaining two components: • Meta-instructions specific to structured output. Hourglass modules receive additional instructions that govern the content and form of the symbolic intermediates 𝜙 and 𝑇 (e.g., “𝜙 must use relative positions,” “𝑇 must refer only to entity types defined in 𝜙”). Self-Refine, which does not produce structured symbolic intermediates, omits these instructions. • Output format constraints. Hourglass mandates explicit, labeled fields for 𝜙, 𝑧, and 𝑇, encoded as structured natural-language templates. Self-Refine imposes no analogous format constraints; the model is free to organize its reasoning and code in any structure, language, or logical flow it deems appropriate. This design reflects a deliberate trade-off: Hourglass invests in a rigid symbolic interface to enforce the structured bottleneck, while Self-Refine retains maximal flexibility, mimicking the most common deployment of iterative refinement in practice.

D.2

Meta-Instructions

The meta-instructions used in all prompts were developed through an iterative process of manual error analysis and LLM-assisted review of failure cases. For ARC-AGI-2, the meta-instructions were refined using outputs from Gemini 3.0 Flash and DeepSeek V3.2, two models that are distinct from—and, in the case of DeepSeek, belong to a different family than—the evaluation models GPT-5.5 and Gemini 3.1 Pro. This separation eliminates the risk of circular optimization where the same model used for evaluation would inform the prompt design. For ChipBench and BBEH-Linguini, the meta-instructions were developed with the assistance of Gemini 3.1 Pro, which is also one of the evaluation backbones. We acknowledge this as a potential source of mild bias; however, its impact is demonstrably limited, as discussed below. The meta-instructions can be partitioned into two categories: • General meta-instructions (shared). These capture task-agnostic pitfalls observed during development— e.g., anchoring on spurious coordinates, overfitting to example indices, conflating support-set order with logical priority. Both Hourglass and Self-Refine receive these instructions in all modules. • Structural meta-instructions (Hourglass-only). These govern the production of 𝜙 and 𝑇 and are tailored to the structured bottleneck—e.g., requirements for origin-independence, entity-based relational descriptions, and modular decomposition of the transformation rule. Because Self-Refine does not generate explicit symbolic intermediates, it does not receive these prompts.

D.3

Output Format Constraints

Hourglass’s Induction and Deduction modules are required to produce outputs conforming to a fixed schema, with explicit fields corresponding to the schema 𝜙, the transient scaffold 𝑧, and the transformation rule 𝑇. The concrete field names and internal structure vary by benchmark—e.g., on ARC-AGI-2, 𝜙 is realized as a Cross-Matrix Invariant Rules block and 𝑇 as a Cue-Guided Transformation Rule block (§D.5 gives a plain-language account of the full template)—but the underlying constraint is the same across all three benchmarks: each field is delimited by explicit markers and internally organized into labeled subsections, 19

forcing the model to commit to a specific symbolic abstraction before any code or text artifact is produced. This is not a formatting convention; it operationalizes the structured bottleneck itself. The Self-Refine baseline deliberately omits any output format constraints, on all three benchmarks. The model is instructed to produce a solution artifact (code or text) and may, at its discretion, include explanatory comments, intermediate reasoning, or rule summaries—but none of these are required or parsed by the execution pipeline. This design choice preserves the baseline’s generality: imposing arbitrary format constraints on Self-Refine could artificially depress its performance, while the unconstrained variant better reflects standard practice.

D.4

Design Limitations

The meta-instructions and output templates were developed without deep domain expertise in electronic design automation or formal linguistics, and with limited engineering resources. Consequently, we cannot claim that the prompts used in this study are optimal for any of the three benchmarks. However, this limitation also means that the reported results partially reflect a realistic, non-expert deployment scenario, strengthening the practical relevance of the findings. As noted above, the ChipBench and BBEH meta-instructions were iterated with Gemini 3.1 Pro, which also serves as one of the evaluation models. This introduces a theoretical risk of prompt overfitting. However, ablation experiments (§5) confirm that the performance gains are driven by the workflow topology rather than by the precise wording of prompts, suggesting that the results are robust to the specific choice of prompts.

D.5

Prompts for ARC-AGI-2 Hourglass

This section provides the complete prompts used for the four-stage Hourglass pipeline on ARC-AGI-2. The prompts are reproduced below with their original structure, but adapted to LATEX formatting. Placeholders for dynamic content (e.g., [STEP1 FULL ANALYSIS]) are shown as they appear during actual API calls. D.5.1

Step 1: Induction

Task Description You are an image puzzle analysis expert. Analyze 2D numerical matrices where numbers (0-9) represent pixel colors. Perform analysis in three parts: 1. Part 1: Analyze all input matrices — identify common characteristics and per-matrix object features. 2. Part 2: Analyze all output matrices — identify common characteristics and per-matrix object features. 3. Part 3: Consolidate Cross-Matrix Invariant Rules separately for inputs and outputs — the abstract, reusable rules distilled from Parts 1-2. This summary will be passed to downstream steps; write it to be self-contained and actionable without needing the per-matrix details. Important: The list may include test input matrices; analyze them with the input group in Part 1 (they have no expected output). Core Principles 1. Macro-Level from Micro-Level: Macro-Level description must be grounded in cross-matrix object commonality. First extract objects per matrix (Micro-Level); then summarize recurring object types,

20

shared structural roles, and common patterns across matrices — this forms the stabilized Macro-Level view. 2. Functional Abstraction: Do not just identify colors/coordinates. Define objects by their Roles (e.g., “Container”, “Anchor”, “Path”, “Palette”, “Symmetry Axis”, “Frame”). Use topological terms like “Lanes”, “Strips”, “Closed Holes”, and “Boundaries”. 3. Micro-Level (Object Extraction): For each matrix, extract all objects. Categorize into Salient Objects and Other Objects. Coverage: every non-zero pixel belongs to exactly one object. 4. Topology Awareness: Explicitly distinguish between 4-connectivity and 8-connectivity. Identify “Global Background” vs “Internal Cavities”. Note if an object acts as a “Ruler” (central/median base) or a “Container” (locking internal pixels). 5. Relativization: Describe positions using relative logic (e.g., “distance to nearest edge”, “parity of row index”, “offset from Anchor object”) rather than absolute coordinates like (5,5) which may fail to generalize. 6. Complex objects: Use multi-aspect (shape, color, connectivity, symmetry) and hierarchical (whole → parts → sub-parts; coarse → fine) description. Important Guidelines • Container Recognition: If a closed loop exists, prioritize defining its interior as a work region. • Axis/Median Rule: Identify if objects are centered on a “Median Line” or “Axis”. • Parity Awareness: Note if object behavior changes based on odd/even dimensions or positions. • No Fabrication: If no shared patterns exist, leave themes empty. • Consistency: Use the same functional labels (e.g., “Anchor”) across all pairs. Output Format Part 1: All Input Matrices

Common Characteristics (Input):

• has similarities: [Yes or No] • shared structural themes: [Recurring object roles, functional types, common patterns. Note topological invariants like connectivity type.] Individual Input Matrices: matrix 1 (Input): • structural description: [Macro-level description using shared functional vocabulary.] • combination strategy: [How elements/objects combine (e.g., layering, nesting, tiling).] • Object List: – Salient Objects: Salient Object 1 (Role/Type, Rough Description, Relative Positional Description), ... – Other Objects: Other Object 1 (Role/Type, Description, Positional Description), ... • object distribution: [Spatial layout, relative grouping, and repetition patterns.] matrix 2 (Input): [. . . ] Repeat for each input matrix (including test input). Part 2: All Output Matrices

Common Characteristics (Output):

• has similarities: [Yes or No] • shared structural themes: [Recurring roles and patterns across all outputs.] Individual Output Matrices: matrix 1 (Output): • structural description: [Macro-level description.] 21

• combination strategy: [How objects combine.] • Object List: – Salient Objects: ... – Other Objects: ... • object distribution: [Spatial layout, relative grouping.] matrix 2 (Output): [. . . ] Part 3: Cross-Matrix Invariant Rules Consolidate the abstract rules from Parts 1-2 into a self-contained reference. Downstream steps will read this without access to the per-matrix details above. Separate input and output rules clearly. Input Invariant Rules: ## Grid Paradigm (Input) - **Grid Dimensions**: [Fixed HxW / variable / H==W / other pattern] - **Background Color**: [color index 0-9] - **Connectivity**: [4-connectivity / 8-connectivity / mixed] ## Object Classes (cross-input) - **Role [Name]**: [Invariant properties: topology, color range, shape constraints, spatial constraints, relative positioning rules | derived from Common Characteristics in Part 1] ## Layout & Composition Rules (Input) - [Layering rules, intersection handling, relative spacing, alignment patterns | derived from Part 1]

Output Invariant Rules: ## Grid Paradigm (Output) - **Grid Dimensions**: [How output dimensions relate to input dimensions] - **Background Color**: [color index 0-9] - **Connectivity**: [4-connectivity / 8-connectivity / mixed] ## Object Classes (cross-output) - **Role [Name]**: [Invariant properties: topology, color range, shape constraints, spatial constraints, relative positioning rules | derived from Common Characteristics in Part 2] ## Layout & Composition Rules (Output) - [Layering rules, intersection handling, relative spacing, alignment patterns | derived from Part 2]

[INSERT SOURCE PUZZLE MATRICES HERE] D.5.2

Step 2: Deduction

Task Description 22

You are an image puzzle analysis expert. Given the cross-matrix invariant rules and training pairs, derive a unified Transformation Rule that maps input objects to output objects across all pairs. The transformation must: • Operate on the abstract roles and layouts defined in the invariant rules (e.g., “modify the Anchor object”, not “move the blue pixels”). • Make every operation dependent on an origin-independent generation cue extracted from the input data (e.g., “color of the top-left-most pixel”, “count of objects matching Role X”). • Include explicit tie-break rules for any ambiguous selection or ordering. • Use parametric logic (H, W, offsets, //, %) — avoid hardcoded dimensions or coordinates. Meta-Guidance from Cross-Puzzle Analysis: Successful transformations avoid absolute values and embrace Parametric Logic. They use mathematical operators like // (floor div) and % (modulo) for periodicity and relative offsets (e.g., (limit - 1) - x) for symmetry. They define explicit Tie-break rules (e.g., “if multiple candidates, pick min r then min c”) to ensure determinism. Core Principles 1. Holistic Parametrization: Analyze all pairs to find Geometric Constants (e.g., “Output is always 2x input size”, “Center is always at height // 2”). 2. Cue Binding: Every operation MUST have a generation cue that is origin-independent (e.g., “color of the top-left-most pixel”, “count of objects with size ¿ 1”). Omit operations with no generalizable cue. 3. Explicit Mapping: Write transformation functions as f(r, c) where possible. Account for non-square matrices (handle height vs width explicitly). 4. Topology Conservation: Ensure logic preserves connectivity (e.g., a “Path” object in input should map to a continuous structure in output). 5. Tie-break Explicitly: If an operation selects an object, define the sort order (e.g., “smallest area”, “top-most row”). Important Guidelines • Avoid Early Concretization: Do not assume fixed sizes (like 7x7) if the training data shows variation. Use variables (H, W). • Constant Offset Detection: Be precise about size - 1 vs size when calculating boundaries. • Parity-Awareness: If logic involves a “center”, specify behavior for even-sized dimensions (e.g., “pick the smaller of the two center indices”). • Functional Consistency: Ensure the assigned “Role” (e.g., “Container”) is respected by the “Generation Method”. Output Format Transformation Rule ## Output Grid Dimensions - [Formula relating output dimensions to input dimensions] ## Cue-Guided Operations ### Operation Group 1 - **Transformed Object**: [Role from the invariant rules]

23

- **Transformation Goal**: [High-level visual outcome] - **Generation Method**: [Parametric description of the operation] - **Generation Cue**: [The dynamic value or attribute that triggers/drives this operation] - **Cue Extraction Method**: [Precise rule to extract the cue from raw input | must work for all pairs] ### Operation Group 2 - [...] ## Global Tie-Break & Ambiguity Resolution - [Deterministic rules to resolve spatial edge-cases, overlaps, or selection order (e.g., "if multiple candidates, pick min_r then min_c")]

[STEP1 FULL ANALYSIS] [FORMATTED TRAINING PAIRS] D.5.3

Step 3: Implementation

Task Description You are an image puzzle transformation expert. Given the invariant rules, transformation rule, training pairs, and test input, produce: 1. Part 1: A concrete execution plan (natural language), including any hardcoded constants with justification. 2. Part 2: A complete, runnable Python script implementing the plan. Core Principles 1. Deterministic Logic: All choices must be derived from input data. Use deterministic tie-breaks (e.g., sorted(objs, key=lambda x: (x.r min, x.c min))). 2. Constant +1/-1 Precision: When handling boundaries or mirroring, carefully implement (limit - 1) - x to avoid off-by-one errors. 3. Topology Check: For path-following or flooding, ensure the pixel count/connectivity is conserved. Use 4-neigh vs 8-neigh consistently. 4. Helper Robustness: Helpers must raise ValueError on ambiguity or missing cues. 5. Pythonic Initialization: Avoid shallow copy issues. Use new grid = [[bg color for in range(W)] for in range(H)]. Hardcoding Policy • Allowed: (1) Training-derived structural constants (e.g., “background is always color 0”). (2) Cross-puzzle patterns (Checkerboards, Frames, Borders). (3) Test-input-specific properties (H, W). • Not allowed: Direct mapping from training inputs to outputs. No hardcoded pixel coordinates from training data. No color-based special cases (e.g., if color == 3). • Part 1 Requirement: Explicitly state What is hardcoded, Why, and How. Script Robustness (Meta-Guidance) • Axis/Median Rule: When a “central line” is needed, derive it as (dim - 1) // 2 or similar, documenting how even dimensions are handled. 24

• Peeling Strategy: If the transformation involves complex backgrounds, consider “peeling” (masking) salient objects, performing global operations, then “pasting” them back. • Parity Awareness: Check if r % 2 == 0 is required for checkerboards or alternating patterns. Important Guidelines • 1:1 Topology: If expanding a path, ensure no unintended gaps or overlaps occur. • Origin-Independence: Use logic that works regardless of whether the “Anchor” is at (0,0) or (H-1, W-1). • No Process Exit: Do not use sys.exit(). Output Format Part 1: Execution Plan [Step-by-step algorithm: Initialization → Object Extraction → Cue Detection → Transformation → Output Assembly. Include a Hardcoded Constants subsection listing any trainingderived constants (e.g., background color, connectivity type) with justification for each.] Part 2: Python Script [Python code only.]

Script Output Contract • Output exactly one Python fenced code block. • Must define transform puzzle(input grid: list[list[int]]) -> list[list[int]]. • Return List[List[int]] (not numpy array). • Use only standard Python libraries. Use collections.deque for BFS/connected components. • Helpers must raise ValueError on invalid or ambiguous states. • No hardcoded pixel coordinates from training data. No color-based special cases (e.g., if color == 3). Derive all structural parameters from the input data. • No placeholder code, TODO text, sys.exit(), or prose inside the code block.

[INVARIANT RULES] [TRANSFORMATION RULE] [FORMATTED MATRICES] D.5.4

Step 4: Refinement

Task Description You are an expert puzzle analyst. Revise the program so it correctly maps every training input to its expected output. Meta-Guidance for Revision: 1. Root-Cause Diagnosis: Do not just describe where pixels differ. Diagnose why the logic failed (e.g., “The mirror axis was assumed to be at W//2 but should be at W//2 - 1”).

25

2. Surgical Fix (One-Inch Rule): Successful fixes usually involve changing a single constant or mapping (e.g., r to c). If you are adding massive if-else blocks, your geometric model is likely wrong. 3. Anti-Patching: Avoid if color == 3: .... Instead, find the parametric reason why color 3 behaves differently. 4. Parity-Awareness: Check if the failure occurs only on even/odd dimensions. 5. Anti-Regression: If your fix involves “simply counting colors” while losing “spatial position”, stop. This is a sign of model degradation. Important Guidelines • Avoid Threshold Oscillation: Do not just tweak if count > 4 to > 5. Re-evaluate the object definition. • Topology Conservation: If the fix involves movement, ensure objects don’t merge or disappear unless intended. • Parity-Awareness: If dimensions changed from training to test, check if your division/centering logic is robust. Output Format Part 1: Diagnosis • Error comparison: [Actual vs Expected — where and how predictions deviate.] • Root cause: [The logic error — e.g., off-by-one in symmetry axis, wrong connectivity mode, missing parity handling.] • Surgical fix justification: [Why this correction fixes the root cause for ALL pairs, not just the failing one.] Part 2: Revised Invariant Rules

[Full updated invariant rules.]

Part 3: Revised Transformation Rule Part 4: Revised Execution Plan

[Full updated transformation rule.]

[Full updated plan matching the revised rules.]

Part 5: Revised Python Script [Python code only.]

Script Output Contract • Output exactly one Python fenced code block in Part 5. • Must define transform puzzle(input grid: list[list[int]]) -> list[list[int]]. • Return List[List[int]] (not numpy array). • Use only standard Python libraries. Use collections.deque for BFS/connected components. • Helpers must raise ValueError on invalid or ambiguous states. • No hardcoded pixel coordinates from training data. No color-based special cases. • No placeholder code, TODO text, sys.exit(), or prose inside the code block.

26

[INVARIANT RULES] [TRANSFORMATION RULE] [CURRENT CODE] [ERROR TRACES JSON]

27