Discovery under Hypothesis Redundancy: A Geometric Theory of Discovery Bottlenecks

Unknown

Conceptio › Archive › arXiv CS

arXiv CSopen access

Discovery under Hypothesis Redundancy: A Geometric Theory of Discovery Bottlenecks

Unknown · 2026 · arxiv_cs

arXiv CS · Papers · License: Open Access · 2026

Open Source ↗Direct PDF ↓

machine learning, deep learning, neural networks

Discovery under Hypothesis Redundancy: A Geometric Theory of Discovery Bottlenecks

arXiv:2606.14386v1 [cs.LG] 12 Jun 2026

Li Xia School of Economics and Management, Tsinghua University Baoxun Wang Platform & Content Group, Tencent

Abstract Scientific discovery saturates when new hypotheses cease to provide independent information, even if the nominal hypothesis space remains large. We study hybrid discovery systems that combine structured local search with LLM-generated non-local proposals and pose the Search Compression Hypothesis: non-local exploration helps only when three geometric conditions co-occur—spectral compression, orthogonal escape from the explored span, and residual signal alignment with the target. We formalize these conditions, derive necessary conditions for hybrid advantage, and test the mechanism in controlled synthetic environments, large-scale A-share factor discovery, and symbolic-regression benchmarks; a public tabular operational sanity check tests the associated budget-allocation implication. Signal-planting and directed-versus-random experiments show that novelty alone is insufficient: random orthogonal jumps expand coverage but do not improve yield without predictive alignment. Across compression sweeps, real factor archives, and LLM-SRBench tasks, hybrid gains concentrate in weakly represented but target-bearing directions and vanish as the hypothesis space approaches full rank. The framework turns LLM-guided discovery from generic novelty search into a diagnostic procedure for deciding when directed non-local exploration is warranted.

Introduction

Discovery saturates when new hypotheses cease to provide independent information. The issue is not only how many hypotheses can be generated, but whether each one contributes a direction not already covered by the archive. As candidates become increasingly correlated with previous ones, redundancy accumulates: the archive grows in size while adding little independent information. We refer to this concentration of nominally distinct hypotheses into fewer independent directions as spectral compression. We measure it by the effective dimension reff of the hypothesis correlation matrix relative to the nominal dimension N . Local search faces diminishing returns proportional to the compression severity, and its expected yield declines as reff /N → 0 (Proposition 1). Local search cannot escape the compressed span. Structured enumeration recombines existing base factors and operators, producing candidates within span(A) but never orthogonal to it. Random non-local perturbation increases spectral coverage but not predictive yield: random directions carry no alignment with the target signal. This distinction—between orthogonal escape and directed predictive escape—is the core conceptual contribution. Useful exploration requires directed non-local jumps toward weak but target-bearing directions, formalized as predictive novelty (Definition 3). We call this claim the Search Compression Hypothesis. Its formal statement, Principle 1, identifies three individually necessary conditions for useful exploration: compression severity (1 − reff /N ), Preprint. Under review.

escape distance d⊥ , and residual signal alignment (RSA). Theorem 1 establishes that each condition is individually necessary—removing any one eliminates the hybrid advantage. The framework is defined by three operators acting on the hypothesis archive: At+1 = Tverify ◦ Tlocal ◦ Tjump (At ),

(1)

where Tjump proposes non-local seeds that escape the compressed subspace, Tlocal searches the expanded neighborhood, and Tverify removes non-predictive and redundant candidates. Figure 1 summarizes the central mechanism. Our contributions are: 1. Identify spectral compression as a unifying mechanism underlying diminishing returns in discovery systems. We formalize the discovery problem in spectrally compressed hypothesis spaces, with formal yield and coverage objectives tied to the spectrum of the correlation matrix. A stylized proposition (Proposition 1) establishes that local yield scales with reff , explaining why structured enumeration faces diminishing returns. 2. Show that useful exploration requires both escape and residual alignment. Theorem 1 identifies three individually necessary conditions for hybrid advantage—compression severity (1 − reff /N ), escape distance d⊥ , and residual signal alignment (RSA)—all expressed as geometric quantities of the hypothesis space. Random non-locality increases coverage but not predictive yield; directed escape toward weak but target-bearing directions is required. 3. Provide evidence that the same geometric bottleneck appears across multiple discovery systems. Synthetic compression sweeps, 5,647 A-share stocks (2010–2026), and LLMSRBench (158 equations) support a diagnostic view: useful exploration requires separating directed escape from redundant or random novelty. An OpenML public-tabular check tests the operational budget-allocation implication rather than the compression law itself.

Related Work

2.1

LLM-Guided Scientific Discovery

LLMs have become hypothesis generators for mathematics and scientific discovery, from FunSearch [13] to tree-search hypothesis refinement [11] and fully automated research agents [10]. These systems show that LLMs can propose meaningful candidates, but they rarely ask whether a proposal contributes an independent, target-bearing direction relative to the existing archive. Our framework adds this spectral-aware verification layer. 2.2

Search under Compression and Effective Dimension

The failure of local search in high-dimensional spaces with low effective dimension is a recurring theme. In evolutionary computation, bloat and convergence to local optima reflect the collapse of population diversity into correlated subspaces [17]. In neural architecture search, operator redundancy produces highly correlated network encodings [26]. In symbolic regression, most candidate expressions are highly correlated, creating spectral concentration in the expression correlation matrix. Koltchinskii and Lounici [8] establish concentration inequalities for sample covariance operators; Fan et al. [5] show that spectral structure governs inference accuracy in high-dimensional settings; Vershynin [22] provides sub-Gaussian concentration bounds. The effective rank, introduced by Roy and Vetterli [14], and the stable rank [19] provide the geometric foundation for our framework. Bubeck et al. [1] establishes fundamental limits for pure exploration in bandit settings, while Russo et al. [15] shows that adaptive exploration policies outperform fixed allocation—analogous to our spectral-adaptive switching policy. Our contribution connects these spectral and search-theoretic tools to the discovery problem: effective rank directly bounds the yield of structured search (Proposition 1) and governs when non-local exploration becomes necessary (Theorem 1). 2.3

Symbolic Regression as Discovery

Symbolic regression—recovering closed-form expressions from data—is a canonical testbed for search under compression. Classical methods include genetic programming (GP) [16], modern 2

frameworks such as PySR [4], Operon [2], and GP-GOMEA [23], and AI Feynman [20] which uses physics-inspired priors. The SRBench benchmark [9] and LLM-SRBench [18] provide standardized evaluation. Our framework predicts that hybrid (GP + LLM) search outperforms GP alone precisely when the expression space is spectrally compressed—a condition we verify on these benchmarks. 2.4

Factor Discovery as a Testbed

Factor discovery is a measurable testbed for compressed search. The Factor Zoo [7] documents many correlated factors; structured enumeration [6] is bounded by its initial factors and operators. LLM alpha discovery [24, 3] can propose economically meaningful expressions, while multipletesting work [25, 12, 6] addresses selection bias. Our focus is complementary: the structural limitation of local search when the factor archive is spectrally compressed.

Hybrid Discovery Framework

We instantiate the three-operator framework in factor discovery. The jump operator proposes nonlocal seed expressions, the local operator enumerates structured variants around those seeds, and the verification operator filters candidates by predictive strength, redundancy, orthogonal coverage, and complexity. This instantiation is a testbed, not the claimed contribution: the theory concerns when a discovery system should spend budget on directed non-local exploration. Implementation details—LLM prompting, factor parsing, market-cap segmentation, ICIR thresholds, and the GPU sandbox—are deferred to Appendix A. 3.1

A Geometric Theory of Discovery Bottlenecks

Discovery systems exhibit diminishing informational returns when newly generated hypotheses cease to contribute independent directions. We formalize this bottleneck in two steps: spectral compression defines the geometry and bounds local search (Proposition 1), then Theorem 1 identifies the three necessary conditions for hybrid advantage—all expressed as geometric quantities: spectral concentration, metric escape, and angular alignment. A corollary identifies when the hybrid advantage vanishes entirely. Spectral Compression of the Hypothesis Manifold The hypothesis manifold H ⊂ Rd has nominal dimension d = N (the number of candidate hypotheses), but its effective dimension is controlled by the spectrum of the correlation matrix. Let a factor be a standardized vector f ∈ RT ×M evaluated over T dates and M assets, flattened to unit norm. For an archive A = {f1 , . . . , fN }, let ΣA be the N × N factor correlation matrix with eigenvalues λ1 ≥ · · · ≥ λN ≥ 0. The effective rank [14] is N X reff (ΣA ) = exp − pi log pi , i=1

λi pi = P . j λj

(2)

We say the P search space is spectrally compressed when reff ≪ N and the eigenvalue concentration λ1 / i λi is large: most candidate variation is concentrated in a small number of directions. Compression is an empirical fact, not an assumption. For a 100-factor structured archive on 5,647 A-share stocks (2010–2026), reff declines from 31.5 (pre-eligibility) to 28.6 (post-eligibility), while eigenvalue concentration rises from 0.187 to 0.215 and mean absolute pairwise correlation increases from 0.118 to 0.140. After eligibility, fewer than 29% of the nominal dimensions carry independent signal content. Local Search Failure Under Compression Structured enumeration constructs candidates from a finite set of base factors B and operators O: Fstr = {ϕo (fi , fj ) : fi , fj ∈ B, o ∈ O}.

(3)

This is a local search operator: it recombines existing directions within the span S = span(Fstr ) but cannot create components orthogonal to S. 3

Definition 1 (Discovery yield). For archive A, ICIR threshold τ > 0, and redundancy limit ρ ∈ [0, 1), X Nτ,ρ (H; A) = 1 | ICIR(f )| ≥ τ, max | corr(f, g)| ≤ ρ . (4) g∈A

f ∈H

Assumptions. The bound holds under three conditions: (A1) structured candidates have bounded leverage L ≤ B over the eigenspaces of ΣA ; (A2) the conditional pass probability satisfies qτ |Σ ≤ κ · qτ for some κ ≥ 1 (bounded dependence); (A3) the stable rank [19] rstable (ΣA ) and effective rank reff (ΣA ) satisfy rstable ≤ reff ≤ N . This bound is stylized and explanatory: it identifies the scaling relationship rather than providing tight constants. Proposition 1 (Local-yield scaling under isotropic compressed search). Assume: (A1) structured candidates are drawn from a distribution supported on S with bounded leverage L = maxi ∥Pλi f ∥2 /E[∥Pλi f ∥2 ] ≤ B over the eigenspaces of ΣA ; (A2) the predictive pass probability qτ = Pr(| ICIR(f )| ≥ τ ) is bounded below by qmin > 0, and the worst-case conditional pass probability given any correlation configuration satisfies qτ |Σ ≤ κ · qτ for some κ ≥ 1 (bounded dependence; weaker than full independence); (A3) the stable rank rstable (ΣA ) = ∥ΣA ∥2F /∥ΣA ∥22 and effective rank reff (ΣA ) satisfy rstable ≤ reff ≤ N . Then there exists a constant C depending on B and κ such that E[Nτ,ρ (Fstr ; A)] ≤ C · qτ · rstable (ΣA ) · (1 − ρ2 ) ≤ C · qτ · reff (ΣA ) · (1 − ρ2 ). (5) Proof sketch. Transform to the eigenbasis and whiten by the square-root precision matrix. The stable rank rstable bounds the effective degrees of freedom. Under bounded leverage (A1), the expected number of candidates with residual energy exceeding (1 − ρ2 ) outside the dominant subspace is at most C · rstable · (1 − ρ2 ). Full proof in Appendix D. Remark. This is a stylized scaling result under isotropic candidate sampling; the empirical sections test whether the scaling persists in non-isotropic discovery settings. It identifies the correct scaling law under an isotropic candidate model without claiming tight universal constants. The qualitative message—that local yield scales with reff and compression amplifies the value of non-local exploration—is the robust takeaway. Interpretation. As reff declines under participation-induced compression, the expected number of non-redundant structured discoveries declines proportionally. When reff = 28.6 with N = 100 and ρ = 0.3, the bound predicts at most 28.6/100 ≈ 0.29 of the nominal dimension carries independent signal. Definition 2 (Residual signal alignment). For a candidate factor f and existing archive A, the residual signal alignment measures the geometric alignment between the orthogonal residual and the target signal: ⟨f⊥ , y⟩ RSA(f ) = , f⊥ = f − projA (f ), (6) ∥f⊥ ∥ · ∥y∥ where projA (f ) is the linear projection onto span(A) (ridge regression, α = 1) and y is the return signal. RSA is a geometric quantity: the cosine of the angle between the escaping component and the target direction. Definition 3 (Predictive novelty). The predictive novelty is the empirical estimator of residual signal alignment: PredNovelty(f ) = ICIR f − projA (f ) , (7) where ICIR computes the information coefficient information ratio over rolling windows. A factor with high d⊥ but low PredNovelty is orthogonal but non-predictive; the hybrid framework requires both conditions. Interpretation. The three quantities governing hybrid advantage are now unified as geometric objects: compression severity (1−reff /N ) is a spectral quantity, escape distance d⊥ is a metric quantity, and RSA is an angular quantity. PredNovelty estimates RSA from finite samples. Random perturbation produces high d⊥ but near-zero RSA in the directed-vs-random experiment (Table 4), precisely because random directions carry no alignment with the target. Only directed non-locality—where f⊥ is both large and aligned with y—produces useful exploration. This distinction is why LLMseeded search outperforms random perturbation (Table 10) and why all three necessary conditions must hold simultaneously (Theorem 1). 4

Non-local Seeds and Directed Escape Let PS be the orthogonal projection onto the structured span. An LLM-generated seed z has escape distance d⊥ (z, S) = ∥(I − PS )z∥2 . (8) Representative LLM-style factors exhibit substantial orthogonal distance from the 50-factor structured span (mean d⊥ = 0.657, with 66% of variance orthogonal to the structured span), and are 1.4× more orthogonal to the structured span than structured factors are to each other (Appendix I). Three-operator formalism. The theory is compactly expressed through three operators: Tlocal (local enumeration within the structured span, yield bounded by Cqτ reff (1 − ρ2 ) per Proposition 1), Tjump (non-local jump via LLM generation, introducing directions outside S), and Tverify (verification, pruning non-predictive and redundant candidates). The coverage gain from Tjump is discussed heuristically in Appendix C. H YBRID FACTOR is the composition Tverify ◦ Tlocal ◦ Tjump . Orthogonal seeds increase spectral entropy; Appendix C provides perturbation-based intuition for the coverage mechanism. This intuition is not required for the main argument (Theorem 1), but helps explain why the entropy gain is larger under stronger compression. Necessary Conditions for Hybrid Advantage Theorem 1 (Necessary conditions for hybrid advantage). For a factor archive A with effective rank reff and dimension N , let z be a seed with escape distance d⊥ (z, S) and residual signal alignment RSA(z). Under Assumptions (A1)–(A3), the hybrid yield gain ∆yield over pure structured search satisfies the following necessary conditions: reff → 0, d⊥ (z, S) → 0, or RSA(z) → 0 (estimated by PredNovelty), (9) If any of 1 − N then ∆yield → 0. Scope. Theorem 1 identifies when hybrid search can help, not when it must help. The conditions are necessary but not sufficient: satisfying all three does not guarantee a positive yield gain, as the empirical interaction may be weaker than the multiplicative model predicts. The framework is descriptive rather than prescriptive: it explains when exploration is valuable, not how to optimally explore. Policy design is addressed only through diagnostic budget-allocation heuristics. The three quantities—compression, escape, and alignment—are all geometric properties of the hypothesis space, unifying the theory within a single mathematical framework. Practical diagnostic. When reff /N ≈ 1, or d¯⊥ ≈ 0, or RSA ≈ 0, the hybrid advantage vanishes and LLM exploration should be stopped. This provides an interpretable stopping rule: monitor the three geometric conditions and halt when any one approaches its failure threshold. Empirical decomposition. The necessary conditions suggest a multiplicative interaction. We model the hybrid yield gain as: b yield = β 1 − reff · d⊥ (z, S) · RSA(z) + η, (10) ∆ N where β is estimated from data and η captures noise and higher-order interactions. A fine-grained grid sweep (250 conditions, 25,000 observations) confirms that the full interaction model achieves R2 = 0.21 at the individual level and R2 = 0.83 at the condition level (noise-reduced means), with the compression × escape interaction as the dominant term (βC×D = 6.66, p < 0.001). We interpret Eq. 10 as a qualitative diagnostic model rather than a pointwise predictive law: the gap between individual and condition-level R2 reflects the inherent stochasticity of discovery outcomes, and the model’s value lies in identifying which conditions are favorable for hybrid search, not in predicting individual discovery yields. Figure 1 and Table 9 provide qualitative support for this interaction. Corollary 1 (High-rank regime: hybrid advantage vanishes). Under the same assumptions as Proposition 1, as reff /N → 1 (the factor space approaches full rank), the expected hybrid yield advantage satisfies ∆yield = E[Nτ,ρ (Fhyb ; A)] − E[Nτ,ρ (Fstr ; A)] → 0. 5

(11)

(a) Hybrid Advantage vs Compression

(b) Core Theorem Decomposition yield (mean, HybridGain)

hybrid (accepted count)

5 4 3 2 1 0 1 0.0

High d , High PN High d , Low PN Low d , High PN Random baseline

0 5 10 15

0.2

0.4

0.6

reff/N

0.8

1.0

0.0

0.2

0.4

0.6

reff/N

0.8

1.0

Figure 1: Why directed exploration helps under compression. (a) Hybrid advantage vanishes as reff /N approaches full rank. (b) Advantage is largest when compression, escape, and RSA (estimated by PredNovelty) co-occur. Only predictive residual coverage—not coverage alone—produces discovery yield. Intuitively, when the structured space already spans nearly all independent directions, LLM seeds contribute little additional spectral coverage—the local search operator is already near-optimal. Conversely, the hybrid advantage is largest precisely when reff /N ≪ 1, i.e., under strong dependence compression. The Search Compression Principle The three geometric quantities—spectral compression, escape distance, and signal alignment— converge on a single governing principle: Principle 1 (Search Compression Principle). In a spectrally compressed hypothesis space (reff ≪ N ), the useful exploration yield of a seed z is governed by: reff UsefulExploration(z) ∝ 1 − · d⊥ (z, S) · RSA(z) . (12) | {zN } | {z } | {z } compression

escape

alignment

Reading. Theorem 1 establishes that each factor is individually necessary; Equation 12 posits that they interact multiplicatively, which the empirical decomposition (Eq. 10) provides aggregate directional support for. The principle states that useful exploration is proportional not to coverage gain alone, but to predictive residual coverage: the portion of exploration that escapes the compressed span and remains aligned with the target signal. Coverage without alignment is waste; alignment without escape is redundancy. Only their product yields discovery. Generality beyond finance: real factor compression sweep. The search-under-compression formulation is motivated by domains where hypothesis spaces exhibit spectral concentration. We validate this on 3,638 real A-share factors with pre-computed IC evaluations, building compression levels via factor subset selection from real correlation structures. Hybrid advantage is negatively associated with reff /N (Spearman ρ = −0.72) and is largest in the high-correlation subset (∆ = +11.0), consistent with the directional implication of Theorem 1 (Table 1). Section 4.3 confirms the weakeigen-direction mechanism via signal planting and validates that semantic guidance—not mere orthogonality—drives the hybrid advantage. Cross-Domain Stress Test Five synthetic non-financial stress tests exhibit the same qualitative pattern: hybrid advantage increases with compression and vanishes near full rank (Spearman ρ = −0.74, p = 0.002; Appendix N). Theory–experiment correspondence, temporal split, neutralization, cost, segment, and scaling diagnostics are reported in Appendices B–O. 6

Table 1: Real factor compression sweep. Hybrid advantage is negatively associated with reff /N (Spearman ρ = −0.72); random search achieves 0% yield. Subset All factors (reff /N = 0.003) High-corr (reff /N = 0.003) Random 50% (reff /N = 0.006) Low-corr (reff /N = 0.025) Decorrelated (reff /N = 0.086)

Structured

Random

LLM-style

Hybrid

∆yield

25.2 38.8 23.3 21.9 23.2

0.0 0.0 0.0 0.0 0.0

5.0 9.1 5.2 0.6 2.6

31.2 49.8 29.8 22.7 26.4

+6.0 +11.0 +6.5 +0.8 +3.1

Table 2: Synthetic spectral experiment. Hybrid helps most under compression; random search has 0% yield in all regimes. reff /N 0.1 (highly compressed) 0.3 0.5 0.8 0.95 0.98 (near full rank)

Experiments

4.1

Setup

Structured

Random

LLM-style

Hybrid

15.5 9.4 5.7 2.5 1.8 1.7

0.0 0.0 0.0 0.0 0.0 0.0

6.6 1.6 0.1 0.0 0.0 0.0

20.3 10.4 5.5 2.4 1.9 1.8

Data: Daily OHLCV data for 5,647 A-share stocks from Tushare Pro API (2010-01 to 2026-05). The full sample is used for spectral analysis; factor discovery trains on 2010–2018, validates on 2019–2021, and reports evaluation on 2022–2026; temporal split, neutralization, cost, and segment protocols are in Appendices K–M. The universe excludes ST/*ST stocks, stocks with <60 trading days, and IPOs within 30 days. Baselines: (1) Pure LLM: 100 LLM-generated expressions evaluated directly without structured expansion. (2) Pure Structured: 55 pairwise combinations of 11 base factors under product operation, evaluated without LLM seeding. (3) Hybrid: LLM-seeded structured expansion (our framework). Metrics: Mean IC, ICIR, acceptance rate (%), discovery breadth (number of accepted factors), best per-segment ICIR, and the novelty component of Definition 1 measured by archive correlation. 4.2

Synthetic Validation: Hybrid Consistently Outperforms

To isolate the hybrid mechanism from market-specific confounds, we construct synthetic factorreturn data with controlled eigenvalue spectra (reff /N ∈ {0.1, 0.3, 0.5, 0.8, 0.95, 0.98}) and a return signal split between strong and weak eigenvalue directions. Figure 1 visualizes the core theorem across the full compression range, directly showing the three-factor decomposition. Table 2 reports acceptance yield across six compression levels and four search methods over 100 Monte Carlo repetitions. Hybrid outperforms structured search at high compression (reff /N = 0.1: 20.3 vs. 15.5), random perturbation has zero yield, and the hybrid advantage vanishes near full rank (reff /N = 0.98: ∆ = +0.1), matching Corollary 1. 4.3

Mechanism Validation: Signal Planting and Directed Escape

We directly test the two core mechanisms of Theorem 1: (1) the hybrid advantage concentrates in the weak eigenspace, and (2) directed escape—not mere orthogonality—drives the advantage. 7

Table 3: Signal planting. Hybrid gain reverses sign: positive when signal is in the weak eigenspace, negative when in the top eigenspace. Random search has 0% yield. reff /N

Signal

Structured

Random

Hybrid

∆yield

0.15 0.15 0.30 0.30 0.50 0.50 0.80 0.80

Top Weak Top Weak Top Weak Top Weak

24.6 20.0 17.3 12.5 12.9 8.7 9.2 6.2

0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0

22.7 22.6 15.6 15.3 11.2 11.2 7.7 9.1

−1.9 +2.5 −1.7 +2.8 −1.7 +2.6 −1.5 +2.8

Table 4: Directed vs. random escape. Retrieval-guided seeds beat shuffled seeds at near-identical d⊥ , while random seeds have the highest d⊥ but near-zero yield. Seed strategy

Yield

RSA (est. PN)

Mean d⊥

Random Shuffled Retrieval-guided Oracle

0.01 8.03 12.30 15.00

0.046 0.254 0.470 0.894

0.969 0.692 0.688 0.699

Signal planting. Synthetic factor spaces (N = 40, T = 300) are generated with controlled spectral compression (reff /N ∈ {0.15, 0.30, 0.50, 0.80}). Return signal is planted exclusively in one of three eigenspace regions. Table 3 reports yield across 12 conditions (300 MC reps). Hybrid gain reverses sign with signal location: positive in the weak eigenspace (mean ∆ = +2.7), negative in the top eigenspace (−1.7). This confirms that hybrid search helps precisely where local search under-represents target-bearing directions. Directed vs. random escape. The central claim of Theorem 1 is that predictive novelty—not mere orthogonality—drives the hybrid advantage. We compare four seed strategies on identical data (signal in weak eigenspace, reff /N = 0.15, 300 MC reps): Retrieval-guided and shuffled seeds have nearly identical escape distance (0.688 vs. 0.692), yet retrieval-guided seeds achieve 53% higher yield and 85% higher PredNovelty. Random seeds have the largest d⊥ but near-zero yield, so direction matters more than distance. Extended mechanism checks are in Appendices E–G. The full factor-discovery run is supporting evidence rather than the main claim: H YBRID FACTOR accepts 19 factors with best ICIR 0.52, compared with 10 structured-only factors with best ICIR 0.42 and 9 LLM-only factors. Seed ablations, policy ablations, temporal validation, and finance robustness are reported in Appendices H–M. The policy ablation is explicitly illustrative rather than optimal: it tests whether compression diagnostics can guide budget allocation, not whether they define an optimal exploration policy. 4.4

Generality: Symbolic Regression and Public Tabular Checks

As a domain-general sanity check, we apply the framework to symbolic regression (SR), where expression archives are naturally redundant. On a 25-equation custom benchmark, hybrid advantage concentrates on hard equations (∆R2 = +0.488) with negligible gain on easy equations (Appendix P). The LLM-SRBench dataset [18] (158 problems) provides the stronger standardized test; Table 5 reports GP, LLM-only, and hybrid results. Two findings are most relevant. First, GP and hybrid are near parity on synthetic equations (median R2 0.980/0.984), but all methods degrade on adversarially transformed equations, consistent with a more deceptive and compressed search landscape. Because full candidate-archive spectra are unavailable for every run, we treat this as failure-rescue evidence rather than a direct compression-law 8

Table 5: LLM-SRBench benchmark. Hybrid matches GP on synthetic equations and rescues 25/46 GP catastrophic failures on transformed equations. Category

Method

Median R2

Recovery

Wins

Synthetic Synthetic Synthetic

GP-only LLM-only Hybrid (full)

72 72 72

0.980 0.757 0.984

96% 71% 99%

34/72 0/72 38/72

Transform Transform Transform

GP-only LLM-only Hybrid (full)

86 86 86

−0.011 −1.253 −0.065

37% 12% 34%

48/86 0/86 38/86

Overall Overall Overall

GP-only LLM-only Hybrid (full)

158 158 158

0.875 0.120 0.838

64% 39% 63%

79/158 3/158 76/158

test; Appendix S reports a small archive-spectra audit. Second, hybrid rescues 25/46 GP catastrophic failures, supporting the value of directed non-local seeds. Public tabular operational sanity check. We also reuse an OpenML tabular panel [21] with actual provider calls and fixed splits. It is not a third compression-law validation; it tests whether diagnostics should allocate LLM proposal budget. Across 12 datasets and 19 held-out splits, the diagnostic proxy improves mean test score over local-only by 0.031 and over fixed 50/50 hybrid by 0.059, while forced non-local exploration underperforms (Appendix Q).

Conclusion

Discovery is limited not by novelty alone, but by predictive novelty: exploration that provides independent information while remaining aligned with the target signal. We formalized this via three individually necessary conditions—compression severity, escape distance, and residual signal alignment—and tested them through compression sweeps, signal planting, directed-vs-random escape, A-share factor discovery, LLM-SRBench, and an OpenML operational check. The evidence supports the Search Compression Hypothesis as a candidate geometric law of discovery under hypothesis redundancy. We view it as a candidate law, not a proven universal law; universal compression-law validation is left to future work.

Implementation Details

H YBRID FACTOR implements the three-operator framework in four phases. An LLM (deepseekv4) receives a factor taxonomy and a knowledge base of previously accepted factors, then generates Python candidate expressions. Each expression is parsed to extract base factors {f1 , . . . , fk } and operators {op1 , . . . , opm }. The structured engine enumerates all pairwise combinations of extracted base factors under product, ratio, difference, and 20-day rolling correlation, plus segmentconditioned variants (percentile rank and z-score), yielding approximately 2 n2 + 2n candidates from n base factors. Candidates are evaluated per market-cap segment by Pearson IC against weekly forward returns; acceptance requires |ICIR| > 0.3 and |IC| < 0.95. All evaluation runs in a GPUaccelerated sandbox with timeout and memory management.

Theory–Experiment Correspondence

Table 6 pairs each theoretical claim with a falsifiable prediction and the evidence used to test it. Table 6: Theory–experiment correspondence. Each claim is paired with a falsification test and a specific empirical check. Theory Claim

Falsification Test

Evidence

Prop. 1: structured search is capacity-limited by effective rank

If reff → 0, structured search leaves exploitable residual directions

Synthetic and real compression sweeps show local search leaving exploitable residual directions under compressed regimes

Thm. 1: RSA is necessary

If RSA → 0, then ∆yield → 0

Table 4: direction > distance

Cor. 1: reff /N → 1 ⇒ ∆ → 0

Hybrid gain vanishes at high reff /N

Table 2: ∆ = +0.1 at reff /N = 0.98

Eq. 10: multiplicative interaction

Gain ∝ compression × escape × RSA

Fig. 1 + Table 9

Principle 1: useful = directed escape

Random escape has zero yield

Table 2: random = 0% in all regimes

Heuristic Spectral Intuition

This appendix provides heuristic spectral-entropy intuition referenced in Section 3.1. These estimates are not formal results and are not required for the main argument (Theorem 1 depends only on Proposition 1). Definition 4 (Effective dimension). For a correlation matrix Σ with eigenvalues λ1 ≥ · · · ≥ λN and precision parameter ε > 0, the effective dimension is ( ) N X deff (Σ, ε) = min k : λi ≤ ε · tr(Σ) . (13) i=k+1

Intuition. When a seed z lies partially outside the structured span (∥PS z∥2 ≤ 1 − ε), it increases the spectral entropy of the archive. The entropy gain is approximately proportional to ε under a spectral gap, and degrades gracefully without one. Critically, when deff is small (severe compression), the entropy gain from orthogonal seeds is larger—precisely when hybrid search is most needed. Empirical validation confirms ∆H > 0 in all 126 tested conditions, with monotonic increase in ε and larger gains under stronger compression.

Proofs

Proof of Proposition 1 (Local-Yield Scaling under Spectral Compression) We establish the bound through three lemmas. 10

Lemma 1 (Bounded pass probability under leverage). Let f be a structured candidate drawn from a distribution supported on S with leverage bounded by B over the eigenspaces of ΣA . Then Pr(| ICIR(f )| ≥ τ ) ≤ B · qτ where qτ is the unconditional pass probability. P Proof of Lemma 1. Decompose f = i αi vi in the eigenbasis of ΣA , where vi are eigenvectors. 2 The leverage constraint gives E[αi ]/E[αi2 ]unif ≤ B for each i. By Markov’s inequality on the squared residual ∥f − Pk f ∥2 where Pk projects onto the top-k eigenspace: Pr(∥f − Pk f ∥2 > t) ≤ P B i>k λi 2 P t i λi . Setting k = rstable and t = (1 − ρ ) bounds the probability that f has sufficient novelty to pass the redundancy filter. Lemma 2 (Novelty budget under stable rank). The expected number of structured candidates with archive correlation ≤ ρ satisfies E[|{f ∈ Fstr : maxg∈A | corr(f, g)| ≤ ρ}|] ≤ |Fstr |·(1−ρ2 )· rstable N . Proof of Lemma 2. Transform to the eigenbasis. Under the isotropic model (uniform sampling on the unit sphere in S), a candidate f has novelty ≥ ρ iff its projection onto the complement of the top eigenspace has norm ≥ ρ. This equivalence is exact under isotropy and approximate for general bounded-leverage distributions. The stable rank rstable = ∥Σ∥2F /∥Σ∥22 bounds the number of eigenvalue directions withP significant mass: by the Schur–Horn theorem, at most rstable eigenvalues can exceed ∥Σ∥22 /∥Σ∥2F · i λi . The probability that a random projection has energy ≥ (1 − ρ2 ) in the tail is bounded by (1 − ρ2 ) · rstable /N . Lemma 3 (Bounded dependence correction). Under assumption (A2), the dependence between pass and novelty events satisfies Pr(pass ∩ novel) ≤ κ · qτ · Pr(novel). Proof of Lemma 3. Assumption (A2) states qτ |Σ ≤ κ · qτ . Conditioning on the correlation configuration Σ: Pr(pass ∩ novel) = EΣ [Pr(pass|Σ) · Pr(novel|Σ)] ≤ κ · qτ · EΣ [Pr(novel|Σ)] = κ · qτ · Pr(novel). P Proof of Proposition 1. Combining Lemmas 1–3: E[Nτ,ρ ] = f ∈Fstr Pr(pass ∩ novel) ≤ |Fstr | · κ · qτ · (1 − ρ2 ) · rstable ■ N . Setting C = |Fstr | · κ/N and noting rstable ≤ reff gives the result. Toy Model Verification For the special case of isotropic candidates (B = 1, κ = 1) with |Fstr | candidates drawn uniformly from the unit sphere in S: Lemma 4 (Isotropic closed form). Under isotropic conditions with ρ = 0 and qτ = 1: rstable . E[Nτ,0 ] = |Fstr | · N

(14)

Proof. Under isotropic sampling from the unit sphere in S, each candidate is a uniformly random direction. The probability that a random direction has non-trivial projection onto any of the rstable significant eigenspaces is exactly rstable /N (by spherical symmetry and the definition of stable rank as the effective degrees of freedom). With all candidates passing (qτ = 1) and no redundancy filter (ρ = 0), the expected count is |Fstr | · rstable /N . Setting C = |Fstr |/N recovers the bound. General case. For non-isotropic candidates with bounded leverage and dependence, the constant C = B · κ · |Fstr |/N absorbs the deviations from uniformity. The scaling E[N ] ∝ rstable · (1 − ρ2 ) holds under these weaker assumptions, but the bound should be interpreted as identifying the correct scaling rather than providing a numerically tight guarantee. The grouped-means validation in Table 9 provides empirical support for this scaling across all experimental conditions.

Extended Mechanism Validation: Signal Planting

This appendix extends the signal planting results from Section 4.3 with the full 12-condition table (including middle eigenspace) and validation on the real factor correlation matrix. Table 7 reports the complete signal planting results including the middle eigenspace region. 11

Table 7: Signal planting—full results (300 MC reps). Including middle eigenspace conditions omitted from the main text for brevity. reff /N

Signal

Structured

Random

Hybrid

∆yield

0.15 0.15 0.15 0.30 0.30 0.30 0.50 0.50 0.50 0.80 0.80 0.80

Top Middle Weak Top Middle Weak Top Middle Weak Top Middle Weak

24.6 22.0 20.0 17.3 17.2 12.5 12.9 12.5 8.7 9.2 9.2 6.2

0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0

22.7 21.3 22.6 15.6 15.6 15.3 11.2 11.2 11.2 7.7 7.7 9.1

−1.9 −0.7 +2.5 −1.7 −1.6 +2.8 −1.7 −1.3 +2.6 −1.5 −1.5 +2.8

Table 8: Signal planting on real A-share factor correlation matrix (100-factor subsample, 300 MC reps). The low-correlation regime shows the starkest separation: structured yield collapses to 0.4 when signal is in weak directions, while hybrid retains 37.0 (∆ = +36.6). Mean ∆weak = +13.0 vs. ∆top = −2.8, a 15.8-point gap confirming the mechanism on real data. Subset (reff /N )

Signal

Structured

Random

Hybrid

∆yield

High-corr (0.01) High-corr (0.01) High-corr (0.01)

Top Middle Weak

48.3 49.7 49.7

0.0 0.0 0.0

48.2 49.6 49.6

−0.2 −0.1 0.0

Random (0.09) Random (0.09) Random (0.09)

Top Middle Weak

33.8 33.2 33.7

0.0 0.0 0.0

34.9 35.2 36.2

+1.1 +1.9 +2.5

Low-corr (0.09) Low-corr (0.09) Low-corr (0.09)

Top Middle Weak

25.9 1.8 0.4

0.0 0.0 0.0

16.5 32.2 37.0

−9.4 +30.4 +36.6

Validation on Real Factor Correlation Structure We replicate the signal planting experiment on the real 3,638-factor correlation matrix, subsampling N = 100 factors under three regimes: high-correlation (reff /N ≈ 0.011), random (≈ 0.085), and low-correlation (≈ 0.086). The real correlation structure produces a dramatically stronger effect than synthetic data. The low-correlation regime (Table 8) provides the cleanest separation between signal locations. When signal is in the top eigenspace, structured search achieves 25.9 yield. When signal shifts to the weak eigenspace, structured yield collapses to 0.4—a 65× reduction—while hybrid retains 37.0. This asymmetry, absent in synthetic data with uniform eigenvalue structure, emerges naturally from the real factor correlation matrix where weak eigenvalue directions are nearly orthogonal to the structured-accessible span.

Extended Directed vs. Random Escape Results

This appendix provides the experimental protocol details for the directed-vs-random experiment summarized in Table 4 of Section 4.3. Four seed strategies are compared on identical data (signal in weak eigenspace, reff /N = 0.15, 300 MC reps): 1. Random: purely random directions (high d⊥ , no guidance). 2. Shuffled: random recombination of weak factors (preserves spectral geometry, no target guidance). 12

3. Retrieval-guided: weak factors selected by correlation with the target (semantic guidance). 4. Oracle: direct access to the planted signal direction (upper bound). The main results are reported in Table 4. As discussed in Section 4.3, the key finding is that retrievalguided seeds achieve 1.53× higher yield than shuffled seeds at near-identical escape distance, confirming that the direction of escape—not mere orthogonality—drives predictive yield.

HybridGain Decomposition and RSA/PN Checks

To examine the empirical decomposition (Eq. 10), we partition 2,700 synthetic observations into a 2 × 2 × 2 factorial design: high/low compression (reff /N ≤ 0.5 vs. > 0.5), high/low escape distance (d⊥ ≥ 0.6 vs. < 0.6), and high/low predictive novelty (PredNovelty ≥ 0.7 vs. < 0.7). Table 9 reports the mean hybrid yield gain for each group. Table 9: HybridGain factorial evidence (2×2×2 design, N = 2,700 synthetic observations). Mean ∆yield is positive in all groups with high escape distance. High compression and high escape are the dominant drivers; RSA (estimated by PredNovelty) improves yield directionally at the aggregate level but is noisy at the cell level due to finite-sample estimation. Compression

d⊥

RSA (est. PN)

Mean ∆yield

High (≤0.5) High (≤0.5) High (≤0.5) High (≤0.5) Low (>0.5) Low (>0.5) Low (>0.5) Low (>0.5)

High (≥0.6) High (≥0.6) Low (<0.6) Low (<0.6) High (≥0.6) High (≥0.6) Low (<0.6) Low (<0.6)

High (≥0.7) Low (<0.7) High (≥0.7) Low (<0.7) High (≥0.7) Low (<0.7) High (≥0.7) Low (<0.7)

+3.17 ± 0.36 +3.76 ± 0.52 +0.48 ± 0.45 +0.47 ± 0.67 +0.71 ± 0.17 +1.37 ± 0.28 +0.14 ± 0.21 −0.01 ± 0.35

600 300 300 150 600 300 300 150

The three-factor decomposition is directionally consistent across all eight condition groups: the mean ∆yield is highest when compression and escape distance are high, and lowest when either compression or escape is low. A Kruskal–Wallis test confirms significant group differences (H = 301.6, p ≈ 0). RSA/PN decile ablations further show monotone gains outside the low-signal noise regime: RSA deciles increase mean gain by 9×, and PN deciles by 136× (both p ≈ 0).

Seed, Policy, and Factor-Discovery Ablations

Table 10: Seed ablation. LLM seeds achieve the highest escape distance (d⊥ = 0.657) and the best predictive novelty, demonstrating that LLM non-local jumps are not equivalent to random perturbation. Seed Type LLM Random symbolic Shuffled LLM Structured

Mean d⊥

Yield

RSA (est. PN)

Mean —ICIR—

0.657 0.352 0.448 0.213

35% 15% 20% 25%

0.032 0.014 0.019 0.008

0.42 0.28 0.31 0.38

LLM seeds achieve d⊥ = 0.657, 1.9× higher than random symbolic seeds and 3.1× higher than structured baselines. They also achieve the highest PredNovelty (0.032 vs. 0.014 for random), showing that non-local jumps are not merely orthogonal but also predictively novel. 13

Table 11: Policy ablation across compression regimes (synthetic, 500 MC reps, 40 candidates). The spectral-adaptive policy is illustrative rather than optimal: it tests whether compression diagnostics can guide budget allocation. Mean Yield

Ret. Gap

reff /N

pLLM

Struct.

Fixed

Adaptive

(Adapt−Fix)/Struct.

0.10 0.50 0.80

0.82 0.38 0.12

15.2 5.5 2.6

14.3 2.7 1.3

13.9 3.4 2.3

−3% +13% +42%

The adaptive rule improves budget efficiency under compression variation: at reff /N = 0.8, it achieves 89% yield retention using 12% LLM budget, compared with 50% for the fixed hybrid rule. Table 12: Comparison of factor discovery approaches. H YBRID FACTOR achieves the best ICIR (0.52) and the highest discovery breadth (19 factors vs. 9 for LLM and 10 for structured). Method Pure LLM Pure Structured HybridFactor

Candidates

Accepted

Rate

Best ICIR

Breadth

100 55 155

9 10 19

9% 18% 12%

0.52 0.42 0.52

9 10 19

Orthogonal Seed Distance (Extended)

Table 13 reports the escape distance of representative LLM-style factors relative to the structured span. Table 13: Orthogonal seed distance relative to the structured factor span. Long interpretation text is wrapped to keep the appendix table within the text width. Factor

d⊥ (escape distance)

R2 (explained by Interpretation structured)

amplitude

0.912

vol ratio

0.897

skew 20

0.804

accel ret5 x vol ret 5 ret 20

0.702 0.505 0.450 0.327

0.169 Strongly orthogonal—amplitude is novel 0.195 Strongly orthogonal—volume ratio is novel 0.353 Strongly orthogonal—skewness is novel 0.507 Mostly outside—acceleration is novel 0.745 Partially in span 0.797 Partially in span (momentum-like) 0.893 Mostly in span (momentum-like)

Mean

0.657

0.523 66% of variance is orthogonal

Baseline: the mean within-structured R2 is 0.774—structured factors share 77% of mutual variance. √ ¯ The gap ∆d⊥ = 0.657 − 1 − 0.774 = 0.182 quantifies the additional orthogonality that LLM seeds provide over random structured candidates.

Additional Finance-Specific Analyses

This section consolidates finance-specific results that validate the framework in the A-share factor discovery setting: discovery lineage (linking backtest findings to LLM-generated factor families), operator-independence hierarchy, factor combination details, top discovered factors, and riskadjusted performance. 14

Table 14: Top-5 factors by ICIR. Volume-centric factors dominate, consistent with A-share meanreversion patterns. Factor

Mean IC

ICIR

Source

gap up ratio 20d mom vol ix ret 5 x ret 10 ret 5 plus vol vol ratio

0.0418 −0.0520 −0.0488 −0.0400 −0.0352

0.52 0.40 0.42 0.37 0.36

Registry LLM Structured LLM LLM

Table 15: Risk-neutralized factor performance (OOS 2022–2026). All factors remain significant (pRW < 0.01) after joint neutralization. Joint neutralization generally increases |ICIR|, confirming factor signals are not artifacts of risk exposures. Mean IC

ICIR

|t|

Sig.

Raw Industry Size Joint

0.0097 0.0106 0.0059 0.0074

0.121 0.215 0.083 0.204

3.5 6.3 2.4 6.0

** ** ** **

mom vol

Raw Industry Size Joint

−0.0583 −0.0403 −0.0595 −0.0418

−0.410 −0.390 −0.476 −0.504

11.9 11.4 13.9 14.7

** ** ** **

ix ret 5 x ret 10

Raw Industry Size Joint

−0.0333 −0.0128 −0.0291 −0.0154

−0.285 −0.132 −0.291 −0.203

8.3 3.9 8.5 5.9

** ** ** **

ret 5 plus vol

Raw Industry Size Joint

−0.0502 −0.0437 −0.0506 −0.0430

−0.455 −0.585 −0.495 −0.663

13.3 17.1 14.4 19.4

** ** ** **

vol ratio

Raw Industry Size Joint

−0.0452 −0.0379 −0.0459 −0.0377

−0.475 −0.596 −0.508 −0.646

13.9 17.4 14.8 18.8

** ** ** **

Factor

Neutralization

gap up ratio 20d

Discovery lineage. LLM factor generation is seeded by patterns discovered in systematic backtests. Six discovery chains link backtest findings to LLM-generated factor families. Volume-centric factors dominate the accepted set, consistent with mean-reversion patterns in A-share markets. The corr operator produces the most independent factors, supporting higher discovery yield (Proposition 1). Factor combination. Graph-guided factor combination improves portfolio Sharpe by 12% over single-factor strategies (2.63 vs 2.34) while reducing drawdown by 36%, with mean pairwise correlation of only 0.083. Top factors. Table 14 reports the top-5 discovered factors by ICIR. The gap up ratio 20d factor achieves the highest ICIR (0.52). Risk-neutralized performance. Table 15 reports raw, industry-neutral, size-neutral, and jointly neutralized IC/ICIR for the top-5 factors with Romano–Wolf step-down bootstrap (B = 1000) controlling family-wise error rate. All 5 factors remain highly significant (pRW < 0.01) across all neutralization levels. Joint neutralization increases |ICIR| for 4 of 5 factors, ruling out the concern that discovered factors are artifacts of industry or size tilts. 15

Table 16: Out-of-Sample Walk-Forward Validation on a synthetic latent factor model (5 latent factors, MC=200 repetitions). This table reports synthetic OOS performance, not real A-share returns. Mean OOS Sharpe

95% CI

Win Rate

Sharpe > 1

Random Single best factor LLM-only Structured-only

−0.09 −0.02 0.07 0.09

[−0.19, −0.00] [−0.10, 0.07] [−0.04, 0.17] [−0.02, 0.19]

45% 51% 52% 55%

2% 5% 5% 8%

Equal-weight ICIR-weighted Inverse-variance

−0.01 0.08 0.00

[−0.09, 0.08] [−0.02, 0.19] [−0.09, 0.09]

52% 52% 53%

3% 5% 4%

Method

Table 17: Cost-adjusted OOS performance (ICIR-weighted factor combinations, 2022–2026 test period). Cost (bps/side)

Gross Sharpe

Net Sharpe

Sharpe Retention

0.87 0.87 0.87

0.72 0.41 0.19

83% 47% 22%

10 (minimal) 30 (realistic) 50 (conservative)

Out-of-Sample Walk-Forward Validation

Table 16 reports the aggregate OOS results from a synthetic latent factor model (5 factors, 200 MC repetitions). ICIR-weighted combinations significantly outperform the random baseline (p = 0.016, paired t-test).

Cost-Adjusted Performance

Table 17 reports Sharpe ratios under three levels of round-trip transaction costs (10, 30, 50 basis points per side). At the realistic 30 bps level, the ICIR-weighted mean OOS Sharpe remains positive (0.41).

Segment-Conditioned Discovery

Table 18 shows the per-segment performance. The hybrid framework maintains consistent discovery across all 6 market-cap segments, with 9 accepted factors per segment.

Cross-Domain Stress Test: Detailed Results

Table 19 reports the full cross-domain stress test across five non-financial domains at three compression levels.

Qualitative Scaling Diagnostics

We fit log(∆yield ) = α log(1 − reff /N ) + β log(d⊥ ) + γ log(PN) + log C across 75 observations spanning 8 domains. Table 20 reports the power-law exponents by domain. Leave-one-domain-out validation produces negative held-out R2 in all folds, confirming that quantitative exponents are domain-specific and non-transferable. The qualitative prediction—negative correlation between reff /N and ∆yield —holds across all domains, consistent with Theorem 1. 16

Table 18: Segment-conditioned discovery. All 6 market-cap segments receive consistent factor coverage. Segment (100M CNY) <20 (micro) 20–50 50–100 100–200 200–500 >500 (mega)

Best LLM ICIR

Best Structured ICIR

Hybrid Factors

0.49 0.52 0.51 0.48 0.50 0.47

0.38 0.42 0.40 0.41 0.39 0.37

9 9 9 9 9 9

Table 19: Cross-domain stress test (5 non-financial domains, 3 compression levels). Hybrid advantage is largest under strong compression (reff /N = 0.18, mean ∆ = +3.0) and vanishes near full rank. Spearman ρ(reff /N, ∆) = −0.74, p = 0.002.

Structured

Hybrid

∆yield

d¯⊥

0.18 0.50 0.88

12.5 5.4 2.1

15.1 5.4 2.2

+2.7 +0.0 +0.1

0.991 0.974 0.953

Climate Climate Climate

0.18 0.50 0.88

24.8 17.8 10.5

28.4 18.0 10.4

+3.6 +0.2 −0.1

0.991 0.975 0.898

Genomics Genomics Genomics

0.18 0.50 0.88

16.4 10.0 5.4

19.5 10.6 5.5

+3.1 +0.6 +0.1

0.991 0.975 0.891

NAS NAS NAS

0.18 0.50 0.88

12.2 5.5 2.1

15.0 5.4 2.0

+2.8 −0.1 −0.1

0.991 0.974 0.955

Code gen. Code gen. Code gen.

0.18 0.50 0.88

24.2 20.8 17.1

27.2 20.5 17.2

+3.0 −0.3 +0.1

0.991 0.975 0.955

Domain

reff /N

Drug discovery Drug discovery Drug discovery

Symbolic Regression: 25-Equation Benchmark Details

We select 25 equations from four families: Feynman physics equations (10), Nguyen benchmark (7), Keijzer benchmark (5), and LSR-Transform equations (3). Each equation defines a ground-truth function f : Rd → R with d ∈ {1, . . . , 5} variables. We generate 500 train and 200 test samples uniformly in [−3, 3]d , standardize the target, and test at σ = 0. Three methods are compared: (1) PySR (local GP search, 10 iterations, 5 populations, 30 individuals), (2) LLM-guided (GLM-5 expression proposals, 2 rounds of 5 candidates each), (3) Hybrid (PySR at half budget + LLM at half budget, best-of-merged selection).

Public Tabular Operational Sanity Check

The OpenML actual-call panel is an operational sanity check, not a third compression-law test. It uses 12 public tabular datasets, 19 held-out task splits, 48 completed provider calls, fixed train/validation/test splits, prompt/response hashes, parser-failure logs, and equal verified-candidate budgets. The panel tests budget allocation under redundancy diagnostics; it is not used as direct proof of the spectral-compression law. Dataset-clustered paired tests reduce repeated-split dependence by averaging paired differences within each dataset before resampling datasets. The diagnostic validation-proxy improves over local-only by 0.045 (95% CI [0.003, 0.106], Wilcoxon p = 0.008, 11/1 wins/losses), fixed 50/50 by 0.058 (CI [0.041, 0.077], p < 0.001, 12/0), and forced nonlocal diagnostic by 0.181 (CI [0.147, 0.219], p < 0.001, 12/0). These results support the diagnostic-allocation interpretation: 17

Table 20: Scaling collapse exponents. The synthetic domain yields the tightest fit (R2 = 0.69), while cross-domain heterogeneity reduces the pooled R2 . Domain

α (comp.)

β (escape)

γ (PN)

Synthetic SR Pooled

0.47 0.00 0.33

1.51 −0.17 0.50

−0.39 −2.79 0.25

0.69 0.18 0.13

48 10 75

Table 21: Symbolic regression benchmark (25 equations, σ = 0). Hybrid advantage concentrates on hard equations where local search alone is insufficient. Difficulty

reff /N

PySR R2

Hybrid R2

∆R2

d¯⊥

Easy Moderate Hard Very hard

3 11 8 2

0.72 0.48 0.21 0.15

0.19 0.34 0.41 0.12

0.991 0.383 −0.108 0.920

0.965 0.515 0.380 0.969

−0.026 +0.132 +0.488 +0.049

0.273 0.501 0.642 0.253

Overall

0.39

0.33

0.278

0.540

+0.262

0.519

non-local proposals help only when validation-visible evidence says they are worth spending budget on.

Symbolic Regression Noise Robustness

Table 24 reports noise robustness results. Hybrid win rate is stable across noise levels (24%/20%/28%). Noise level is not significantly correlated with hybrid advantage (Spearman ρ = −0.069, p = 0.55).

Small SR Archive-Spectra Audit

We ran a controlled GP-only candidate-archive spectra audit on 11 representative LLM-SRBenchstyle problems. The audit directly computes effective rank from candidate prediction correlations, but remains a small diagnostic rather than headline compression-law evidence. The audit supports compression plausibility in SR candidate archives, especially in high-complexity problems (reff /N = 0.286). Because two rescue rows had no valid archive predictions, SRBench remains failure-rescue evidence rather than a complete compression-rescue test.

LLM-SRBench Budget Sensitivity (158 Problems)

LLM-SRBench Detailed Results (158 Problems)

The synthetic equations (bio, materials, physics) all yield high GP R2 (> 0.94), with hybrid matching or slightly exceeding GP. The transform dataset is the sole source of GP catastrophic failure, with 53% of problems producing negative R2 . The hybrid rescue rate (25/46 = 54%) demonstrates that LLM-directed escape provides orthogonal value even when GP’s local search completely fails, though neither method reliably solves adversarially transformed equations (hybrid median −0.065). This asymmetry—strong hybrid benefit on transform, negligible benefit on synth—aligns with the compression framework: transformed equations compress the search landscape far more severely, creating the conditions under which non-local exploration is theoretically most valuable (Theorem 1).

Table 22: PySR budget sensitivity. Increasing iterations from 10 to 50 improves hard-equation R2 by 0.21, but hybrid advantage persists (∆R2 = +0.28 at 50 iterations). PySR iterations

PySR R2 (Hard)

Hybrid R2 (Hard)

∆R2 (Hard)

−0.108 0.052 0.103

0.380 0.389 0.382

+0.488 +0.337 +0.279

10 (default) 30 50

Table 23: OpenML actual-call repeated-split benchmark. Non-oracle policies rank candidates using train/validation-visible fields only; the oracle is a non-deployable test-outcome upper bound. Policy local-only always-LLM fixed 50/50 hybrid diagnostic validation-proxy random nonlocal shuffled LLM forced nonlocal diagnostic oracle upper bound

Tasks

Test score

Accepted/100

Regret

19 19 19 19 19 19 19 19

0.380 0.191 0.352 0.411 0.228 0.191 0.221 0.437

98.7 74.5 98.0 100.0 82.2 77.7 82.4 100.0

0.057 0.246 0.085 0.026 0.209 0.247 0.216 0.000

Table 24: Symbolic regression noise robustness (25 equations × 3 noise levels). The largest advantage occurs at σ = 0.01 where mild noise destabilizes PySR on hard equations. Noise σ

PySR R2

LLM R2

Hybrid R2

∆R2

Win rate

0.597 0.024 0.636

−0.873 −1.038 −4.192

0.437 0.440 0.484

−0.220 +0.344 −0.267

24% 20% 28%

0.00 0.01 0.05

Table 25: Small SR archive-spectra audit. Candidate archives are often concentrated; this audit is diagnostic only. Group overall low complexity medium complexity high complexity

Problems

Errors

Valid cand.

reff /N

Top eig. share

11 3 5 3

2 0 2 0

3.36 4.00 2.00 5.00

0.420 0.660 0.357 0.286

0.706 0.901 0.473 0.899

Table 26: Hybrid budget comparison on LLM-SRBench (158 problems). Full-budget hybrid achieves parity with GP-only (p = 0.789). Rescue effect (25/46 GP catastrophics improved) is present at both budgets. Median R2

R2 ≥ 0.5

Beat GP

Rescue

GP-only Half-budget hybrid Full-budget hybrid

0.875 0.702 0.838

64% 61% 63%

— 35% 48%

— 24/46 25/46

Synthetic Synthetic Synthetic

GP-only Half-budget hybrid Full-budget hybrid

0.980 0.974 0.984

96% 99% 99%

— 32% 53%

— — —

Transform Transform Transform

GP-only Half-budget hybrid Full-budget hybrid

−0.011 −0.079 −0.065

37% 29% 34%

— 37% 44%

— 24/46 25/46