arXiv:2606.30335v1 [cs.AI] 29 Jun 2026
BayesEvolve: Explicit Belief States for Autonomous Scientific Discovery Xuening Wu,1 Shan Yu,2 Qianya Xu,3 Shenqin Yin4, 1 Pfizer, Shanghai, China Independent Researcher, Hangzhou, China 3 University of California San Diego, La Jolla, CA, USA 4 Institute of Humanities and Social Science Data, Fudan University, Shanghai, China 2
Corresponding author: [email protected]
Abstract Autonomous scientific discovery systems increasingly use large language models (LLMs) to propose new hypotheses, but many such systems condition primarily on experimental memory: archives of high-scoring candidates or heuristic summaries of recent trials. We argue that discovery agents should instead maintain explicit, uncertainty-aware beliefs about hypothesis quality. We introduce BayesEvolve, a belief-guided discovery framework that converts experimental evidence into a predictive belief state and uses this belief to guide future experimentation. As a controlled testbed for belief-guided discovery, we evaluate BayesEvolve on shifted BBOB-style black-box optimization tasks, leaving program and laboratory discovery domains to future work. BayesEvolve improves sample efficiency over memory- and archive-guided LLM baselines under a fixed evaluation budget. We further show that the belief state is predictive on held-out candidate pools, that controlled decision-rule ablations favor belief-guided selection with an annealed uncertainty bonus, and that BayesEvolve exhibits productive late-stage concentration rather than unfocused exploration.
1
Introduction
LLM-guided evolutionary search has emerged as a compelling paradigm for automated optimization and discovery. FunSearch [Romera-Paredes et al., 2024] demonstrates that evolutionary selection over LLM-generated programs can recover novel combinatorial results; AlphaEvolve [Novikov et al., 2025] scales similar ideas to larger codebases and optimization tasks. Despite these successes, many systems use experimental history primarily as memory: the LLM is prompted with examples sampled from an archive of previously evaluated candidates, often biased toward high-scoring examples. 1
This archive-guided design leaves much of the evidence in implicit form. A list of past experiments says what happened, but it does not explicitly encode what the agent currently believes about unevaluated hypotheses, how uncertain those beliefs are, or which experiment would be most informative next. Two costs follow. First, the agent cannot directly reason about predicted quality beyond the observed archive. Second, it may either over-concentrate around early elite candidates or continue exploring without converting evidence into a focused search direction. We propose BayesEvolve, which reframes autonomous discovery as beliefstate evolution. Instead of treating past evaluations as unstructured memory, BayesEvolve maintains an explicit predictive belief state over candidate quality. At each step, experimental evidence updates a posterior belief; this belief is exposed to the proposal process and used by an acquisition rule to select future evaluations. In the experiments below, the belief state is implemented with a Gaussian process (GP) posterior over numerical candidates, but the framework is agnostic to the surrogate model and candidate representation. Contributions. • We formulate LLM-guided discovery as explicit belief-state maintenance rather than archive memory alone. • We introduce a belief-guided selection rule with an annealed uncertainty bonus that shifts from exploration to exploitation as evidence accumulates. • We evaluate BayesEvolve on shifted BBOB-style optimization tasks, showing stronger performance than archive- and memory-guided LLM baselines. • We analyze belief quality, decision-rule ablations, and diversity dynamics, showing that BayesEvolve’s belief state is predictive and supports productive late-stage concentration.
2
Related Work
Archive-guided LLM evolutionary search. FunSearch [Romera-Paredes et al., 2024] evolves programs using a population of evaluated candidates, while AlphaEvolve [Novikov et al., 2025] extends LLM-guided evolution to broader engineering problems. These systems demonstrate the power of LLM proposal plus selection, but their historical information is largely represented through archives and scores. BayesEvolve complements this line of work by making predictive beliefs explicit. Bayesian and probabilistic discovery. Bayesian optimization uses posterior beliefs and acquisition functions to select informative evaluations [Jones et al., 1998, Srinivas et al., 2010]. BayesEvolve borrows this decision-theoretic structure but places it inside an LLM-guided discovery loop, where the belief state is used
2
both to summarize evidence and guide candidate selection. Related probabilistic discovery systems such as ModelSMC [Wahl et al., 2026] maintain distributions over symbolic model candidates; BayesEvolve focuses on explicit predictive beliefs for general hypothesis quality. Memory-guided LLM agents. Memory buffers and archive summaries are common in LLM-based scientific agents and optimization systems. Such memory can be useful, but it does not by itself provide calibrated predictions or uncertainty estimates over unevaluated candidates. Our experiments compare against archive and heuristic-memory baselines to isolate the value of explicit belief states.
3
BayesEvolve
3.1
Problem Formulation
Let H be a hypothesis space. At step t, the system evaluates a candidate ht ∈ H and receives an objective value yt ∈ R. In this paper we consider minimization, so lower yt is better. The evaluation history is Dt = {(hi , yi )}ti=1 ,
(1)
and the goal is to find a candidate with low objective value under a fixed evaluation budget T . Archive-guided methods condition proposals on a subset of past evaluations, such as top-scoring candidates or recent experimental summaries. BayesEvolve instead maintains an explicit belief state P (y | h, Dt ) = N (µt (h), σt2 (h)),
(2)
where µt (h) is the predicted objective and σt (h) is posterior uncertainty.
3.2
Belief-Guided Selection
For a candidate pool Ct , BayesEvolve selects the next candidate using an uncertainty-aware score. Because we minimize the objective, the fixed-UCB rule is at (h) = −µt (h) + β σt (h). (3) Motivated by exploration schedules in reinforcement learning, our final decision rule uses a decaying uncertainty coefficient: r n0 at (h) = −µt (h) + βt σt (h), βt = β0 , (4) t where n0 is the number of shared initialization evaluations. This encourages broader exploration early and increasingly exploits the belief mean as evidence accumulates. 3
Algorithm 1 BayesEvolve Require: Proposal model πLLM , surrogate model, budget T , initialization size n0 1: Evaluate n0 initial candidates to form Dt 2: for t = n0 + 1 to T do 3: Fit/update belief state P (y | h, Dt ) 4: Construct candidate pool Ct from LLM proposals and/or archive mutations 5:
Compute µt (h) and σt (h) for h ∈ Ct
6: Select ht = arg maxh∈Ct [−µt (h) + βt σt (h)] 7: Evaluate yt = f (ht ) and update Dt ← Dt ∪ {(ht , yt )} 8: end for 9: return best candidate in Dt
4
Experimental Evaluation
4.1
Setup
Benchmark. We evaluate on five shifted BBOB-style minimization functions [Hansen et al., 2009] in dimension d = 5: Sphere, Ellipsoid, Rastrigin, Rosenbrock, and Ackley. Each function is shifted by a fixed hidden offset, and LLM prompts expose only opaque task IDs (e.g., F01) rather than function names, preventing benchmark-name leakage. Each run uses n0 = 6 shared random initialization evaluations and a total budget of T = 100 evaluations. Results are averaged over five random seeds and five functions; shaded regions and ± values report standard error. Methods. All LLM-based methods use the same proposal model, gpt-5.4-mini, and differ only in the information provided in context: no memory (RandomLLM), top archive entries (Archive-LLM), recent heuristic memory (MemoryLLM), or BayesEvolve’s explicit belief state. GP-BO is a non-LLM Bayesian optimization baseline. For local ablations, all belief variants use the same shifted benchmark, shared initialization, GP posterior, and candidate-pool mechanism; we compare mean-only selection, UCB [Srinivas et al., 2010], Thompson sampling [Thompson, 1933], and expected improvement [Jones et al., 1998].
4.2
Experiment 1: Main Discovery Performance
BayesEvolve achieves the best mean normalized objective throughout the budget and the best final performance at 100 evaluations. The gains are largest relative to archive- and memory-guided LLM baselines, suggesting that explicit predictive beliefs provide more useful guidance than experimental memory alone.
4
Figure 1: Main discovery performance on shifted BBOB-style optimization tasks. Curves show mean normalized best-so-far objective across five benchmark functions and five random seeds; shaded regions denote standard error. All LLM methods use gpt-5.4-mini; GP-BO is a non-LLM Bayesian optimization baseline. Lower is better. Table 1: Normalized best-so-far objective at 25, 50, and 100 evaluations. Lower is better. Method 25 evals 50 evals 100 evals Random-LLM Archive-LLM Memory-LLM GP-BO BayesEvolve
4.3
0.629 ± 0.089 0.476 ± 0.074 0.545 ± 0.065 0.384 ± 0.059 0.360 ± 0.062
0.629 ± 0.089 0.473 ± 0.074 0.488 ± 0.069 0.278 ± 0.051 0.243 ± 0.049
0.629 ± 0.089 0.473 ± 0.074 0.425 ± 0.069 0.229 ± 0.047 0.195 ± 0.043
Experiment 2: Belief State Quality
Across 60,800 held-out belief predictions, posterior mean is strongly rankcorrelated with realized objective values (Spearman ρ = 0.774). Uncertainty is also informative: bins with larger posterior σt (h) exhibit larger absolute prediction error. The belief state is predictive but not perfectly calibrated, with empirical coverage of 0.433 for one-sigma intervals and 0.653 for two-sigma intervals.
4.4
Experiment 3: Decision Rule Ablation
The ablation shows that explicit predictive beliefs drive most of the gain: Meanonly selection already substantially improves over archive search. Adding an annealed uncertainty bonus yields the best final score, suggesting that uncertainty
5
Figure 2: Belief-state quality. BayesEvolve’s explicit belief state is evaluated on held-out candidate pools during shifted BBOB-style optimization. The left panel compares posterior mean predictions with realized objective values; the right panel shows that candidates with higher posterior uncertainty have larger prediction error. Lower prediction error is better. Table 2: Decision-rule ablation. All variants use the same shifted benchmark and shared initialization. Final normalized best is lower-is-better; AUC improvement is higher-is-better. Variant Final normalized best ↓ AUC improvement ↑ Archive Mean-only Fixed UCB UCB-decay / BayesEvolve Thompson EI GP-BO
0.333 ± 0.059 0.177 ± 0.047 0.176 ± 0.050 0.174 ± 0.045 0.210 ± 0.048 0.205 ± 0.048 0.213 ± 0.043
0.508 ± 0.056 0.746 ± 0.048 0.731 ± 0.050 0.730 ± 0.046 0.664 ± 0.045 0.672 ± 0.054 0.669 ± 0.049
is most useful when exploration is gradually reduced over the discovery process.
4.5
Experiment 4: Diversity Dynamics
Diversity is not intrinsically better when maximized throughout the run. RandomLLM and GP-BO maintain high final diversity, but BayesEvolve achieves the lowest final objective while concentrating the search later in training. This suggests that BayesEvolve avoids unproductive archive collapse while still converting accumulated evidence into a focused search direction.
5
Discussion
The experiments support three claims. First, explicit belief states improve sample efficiency relative to archive and heuristic-memory baselines. Second, 6
Evals to target ↓ 38.6 16.3 21.2 19.9 24.6 16.5 22.9
Figure 3: Diversity dynamics and productive concentration. Rolling candidate diversity is computed over the most recent 20 candidates. High diversity alone is not sufficient: Random-LLM and GP-BO maintain broad exploration but do not achieve the best final objective. BayesEvolve reduces diversity in the late stage while achieving the lowest objective, indicating productive belief-guided concentration rather than unfocused exploration. Lower objective is better. the learned belief state is predictive on held-out candidates, rather than merely serving as a prompt artifact. Third, decision rules matter: most gains come from the posterior mean, while a decaying uncertainty bonus gives the best final performance by shifting from exploration to exploitation. Limitations. Our current experiments use shifted BBOB-style numerical optimization tasks rather than full program or laboratory discovery. The GP belief state is predictive but imperfectly calibrated, and its scaling limits motivate sparse or neural surrogates for larger evaluation budgets. Finally, the diversity analysis shows productive concentration rather than sustained diversity; future work should study richer notions of semantic and structural diversity for program and scientific-hypothesis spaces. Conclusion. We introduced BayesEvolve, a framework for autonomous discovery agents that transform experimental evidence into explicit predictive belief states. On shifted black-box optimization tasks, BayesEvolve improves over memory-guided LLM baselines, produces predictive belief estimates, and benefits from an annealed uncertainty-aware decision rule. These results support belief-state evolution as a useful principle for autonomous scientific discovery.
References Bernardino Romera-Paredes, Mohammadamin Barekatain, Alexander Novikov, Matej Balog, M. Pawan Kumar, Emilien Dupont, Francisco J. R. Ruiz, Jordan S. Ellenberg, Pengming Wang, Omar Fawzi, Pushmeet Kohli, and Alhussein Fawzi. Mathematical discoveries from program search with large language models. Nature, 625:468–475, 2024. doi: 10.1038/s41586-023-06924-6.
7
Alexander Novikov, Ngân Vũ, Marvin Eisenberger, Emilien Dupont, Po-Sen Huang, Adam Zsolt Wagner, Sergey Shirobokov, Borislav Kozlovskii, Francisco J. R. Ruiz, Abbas Mehrabian, M. Pawan Kumar, Abigail See, Swarat Chaudhuri, George Holland, Alex Davies, Sebastian Nowozin, Pushmeet Kohli, and Matej Balog. Alphaevolve: A coding agent for scientific and algorithmic discovery, 2025. Donald R. Jones, Matthias Schonlau, and William J. Welch. Efficient global optimization of expensive black-box functions. Journal of Global Optimization, 13(4):455–492, 1998. doi: 10.1023/A:1008306431147. Niranjan Srinivas, Andreas Krause, Sham M. Kakade, and Matthias Seeger. Gaussian process optimization in the bandit setting: No regret and experimental design. In Proceedings of the 27th International Conference on Machine Learning, pages 1015–1022, 2010. Stefan Wahl, Raphaela Schenk, Ali Farnoud, Jakob H. Macke, and Daniel Gedon. A probabilistic framework for LLM-based model discovery, 2026. Nikolaus Hansen, Steffen Finck, Raymond Ros, and Anne Auger. Real-parameter black-box optimization benchmarking 2009: Noiseless functions definitions. In Proceedings of the Genetic and Evolutionary Computation Conference Companion Workshop on Black-Box Optimization Benchmarking, 2009. William R. Thompson. On the likelihood that one unknown probability exceeds another in view of the evidence of two samples. Biometrika, 25(3/4):285–294, 1933. doi: 10.2307/2332286.
8