2026-05-12
MaD Physics: Evaluating information seeking under constraints in physical environments Moksh Jain1,2,* , Mehdi Bennani1 , Johannes Bausch1 , Yuri Chervonyi1 , Bogdan Georgiev1 , Simon Osindero1 and Nenad Tomašev1
arXiv:2605.10820v1 [cs.AI] 11 May 2026
1 Google DeepMind, 2 Mila – Quebec AI Institute, Université de Montréal, * Work done during an internship at Google DeepMind
Scientific discovery is fundamentally a resource-constrained process that requires navigating complex trade-offs between the quality and quantity of measurements due to physical and cost constraints. Measurements drive the scientific process by revealing novel phenomena to improve our understanding. Existing benchmarks for evaluating agents for scientific discovery focus on either static knowledgebased reasoning or unconstrained experimental design tasks, and do not capture the ability to make measurements and plan under constraints. To bridge this gap, we propose Measuring and Discovering Physics (MaD Physics), a benchmark to evaluate the ability of agents to make informative measurements and conclusions subject to constraints on the quality and quantity of measurements. The benchmark consists of three environments, each based on a distinct physical law. To mitigate contamination from existing knowledge, MaD Physics includes altered physical laws. In each trial, the agent makes measurements of the system until it exhausts an allotted budget and then the agent has to infer the underlying physical law to make predictions about the state of the system in the future. MaD Physics evaluates two fundamental capabilities of scientific agents: inferring models from data and planning under constraints. We also demonstrate how MaD Physics can be used to evaluate other capabilities such as multimodality and in-context learning. We benchmark agents on MaD Physics using four Gemini models (2.5 Flash Lite, 2.5 Flash, 2.5 Pro, and 3 Flash), identifying shortcomings in their structured exploration and data collection capabilities and highlighting directions to improve their scientific reasoning.
“Measure what can be measured, and make measurable what cannot be measured.” – Galileo
1. Introduction Scientific discovery is fundamentally a process of extracting information from the physical world under constraints. Whether determining the position of a celestial body or measuring the properties of a quantum state, scientists constantly navigate trade-offs between the quality of measurements and their cost in time, energy, or money. While the importance of interventional experiments, where a scientist actively manipulates a system, is well-recognized (M. Bran et al., 2024; Majumder et al., 2024; Ryan et al., 2016), the complementary ability to make strategic, passive Corresponding author(s): [email protected], [email protected] © 2026 Google. All rights reserved
observations is equally critical, particularly when interventions are impossible or prohibitively expensive. The quest to use AI systems to accelerate the scientific process has been a central theme throughout the development of artificial intelligence (Langley, 1981; Lloyd et al., 2014; Majumder et al., 2024). However, the most impactful contributions have been limited to specialized machine learning models targeting specific parts of the scientific process (Jumper et al., 2021). Recent advances in systems based on large language models (LLMs) (Google, 2025; OpenAI, 2025) have demonstrated remarkable reasoning abilities on a variety of challenging problems, and have revived interest in autonomous scientific agents (Lu et al., 2024; Yamada et al., 2025; Zheng et al., 2025a). LLM-based systems have already shown promising results in accelerating scientific progress (Bubeck et al., 2025; Gottweis
MaD Physics : Evaluating information seeking under constraints in physical environments
et al., 2025; Novikov et al., 2025; Romera-Paredes et al., 2024). However, principled evaluation of these abilities of agents remains a challenge. Existing benchmarks for scientific discovery often focus on static question-answering based on existing knowledge (Phan et al., 2026; Rein et al., 2024; Wang et al., 2025), or on environments where agents can perform interventional experiments without any constraints (Cerrato et al., 2026; Nägele and Marquardt, 2025). Additionally, these benchmarks typically rely on standard, well-known physical laws. This allows agents to succeed simply by retrieving memorized knowledge rather than through a process of empirical discovery. Furthermore, they often neglect the practical reality of science: that data acquisition is expensive, and resources must be allocated to maximize information gain. To bridge this gap, we introduce Measuring and Discovering Physics (MaD Physics), a benchmark designed to evaluate the ability of agents to make informative measurements in physical systems under strict budget constraints. MaD Physics simulates the challenge of an experimentalist who must characterize a novel system. Each task places the agent in a dynamical environment governed by physical laws that may be “altered” or distinct from standard physics, such as classical mechanics with anisotropic inertia, quantum systems with modified probability rules, or fluids with non-standard forcing terms. This ensures that the agent cannot rely solely on existing knowledge but must engage in active discovery. In MaD Physics, the agent operates in a loop of measurement and prediction. It is allocated a fixed budget and must choose what to measure (e.g., which particle, which location), when to measure it, and with what fidelity (trading off cost for precision). Once the budget is exhausted, the agent must leverage its collected observations to infer the underlying dynamics and predict the future state of the system. This setup evaluates two fundamental capabilities: the strategic planning of measurements to resolve uncertainty, and the ability to infer complex models from noisy, sparse data. Our contributions are summarized below:
Observation Parameters
System
Observation
Time = 0.33
Prediction Query
System
Agent
Agent Prediction Time = 23.30
Figure 1 | MaD Physics consists of two phases: Measurement (Top), and Prediction (Bottom). In the measurement phase, the agent interacts with the environment to make observations, and each observation incurs a cost. In the prediction phase, the agent has to use the observations collected during the measurement phase to predict the state of the system at the queried time.
• We propose a novel benchmarking methodology for evaluating information-seeking agents in physical domains, emphasizing the cost-fidelity trade-off in experimental design. • We present a suite of three parametric environments—covering Classical, Quantum, and Fluid mechanics—that include “altered” physical laws to rigorously test an agent’s ability to discover simplified but novel dynamics. • We benchmark Gemini 2.5 Flash Lite, Flash and Pro as well as Gemini 3 Flash, on MaD Physics. We analyze their performance across different degrees of alteration and demonstrate how strategic prompting can improve their ability to navigate scientific uncertainty.
2. Related Work Active sensing and adaptive experimental design. The task setup used in MaD Physics is closely related to the problem of active sens2
MaD Physics : Evaluating information seeking under constraints in physical environments
ing (Veiga and Renoux, 2023; Yang et al., 2016) and adaptive experimental design (Lindley, 1956; Rainforth et al., 2024; Ryan et al., 2016). Active sensing studies the problem of sequentially choosing informative measurements with the goal of inferring an underlying parameter with the assumption that the measurements do not affect the data generating process. On the other hand, experimental design studies the problem of designing experiments with the goal of inferring parameters for a model of the data generating process, where the experiments control parameters of the data generating process. While recent work has studied some problems at scale (Hamilton et al., 2025), problems used within active sensing and experimental design are generally lowdimensional, and operate under the assumption that the structural form of the model is known. Recent work has also started extending active sensing to richer physical settings: LASER (Deng et al., 2026) formulates active sensing for continuum field reconstruction as a POMDP and uses a latent world model to provide intrinsic reward signals for sensor motion. Additionally, recent work has also studied various extensions with multi-fidelity observations (Garnett, 2023) and look-ahead planning (Jiang et al., 2020) but the tasks remain limited to simple low-dimensional observations and do not consider complex design spaces. MaD Physics does not provide the structural form of the model, requires processing high-dimensional observations and planning with cost-fidelity tradeoff for measurements. Symbolic regression. Another problem related to the setup we study is that of symbolic regression, where the goal is to infer the symbolic form of a function given access to a dataset of inputs and corresponding outputs (Brunton et al., 2016; Koza, 1994; Schmidt and Lipson, 2009; Udrescu and Tegmark, 2020). Popular approaches for symbolic regression use evolutionary algorithms to search in the space of symbolic expressions (Cranmer, 2023) or policies trained with reinforcement learning to produce the symbolic expression that fits the data best (La Cava et al., 2021; Petersen et al., 2021). Recent efforts have also explored using the domain knowledge in LLMs to improve symbolic regression (Holt et al., 2024; Shojaee
et al., 2025). The standard symbolic regression setup assumes access to a dataset of input-output pairs from which the function is to be deciphered. Recent work has extended the setup to include acquisition of the data as part of the setup. NewtonBench (Zheng et al., 2025b) and PhysMaster (Miao et al., 2025) evaluate the ability of agents to perform experiments to acquire data and then infer the underlying physical laws. In contrast to measuring the exact symbolic accuracy, which is standard in symbolic regression, MaD Physics focuses on measuring the prediction accuracy. In the context where the agent is provided a limited budget to acquire data, there can be multiple symbolic expressions that fit the collected data equally well and prediction accuracy becomes more critical.
Evaluating agents for scientific discovery. Existing benchmarks for scientific discovery typically focus on abilities such as hypothesis generation, experiment design, literature review or static knowledge-based reasoning (Melnik et al., 2023). Benchmarks such as GPQA (Rein et al., 2024), FrontierScience (Wang et al., 2025), PHYBench (Qiu et al., 2025), Humanity’s Last Exam (HLE) (Phan et al., 2026) and CritPT (Zhu et al., 2025) consist of question-answering tasks which require the knowledge about the topic as well as reasoning to arrive at the correct answer. PhysicsMind (Mak et al., 2026) is a visual question-answering dataset to evaluate the physical understanding of VLMs. SymPyBench (Imani et al., 2025) consists of parameterized physics questions paired with different types of responses. Question-answer based benchmarks evaluate knowledge and reasoning but do not measure the ability of agents to dynamically collect data, make inferences from it and plan under constraints. Among interactive benchmarks, ScienceWorld (Wang et al., 2022), DiscoveryWorld (Jansen et al., 2024), and Alchemy (Wang et al., 2021) frame scientific discovery tasks as a game with a goal where successfully achieving the goal requires understanding the environment dynamics. AutumnBench (Warrier et al., 2025) consists of environments on a grid where the 3
MaD Physics : Evaluating information seeking under constraints in physical environments
agent acts in the absence of a goal to understand the environment dynamics and is then tested on various tasks that require understanding the environment dynamics. DiscoveryBench (Majumder et al., 2025) and AstaBench (Bragg et al., 2025) evaluate agents holistically throughout the entire scientific discovery pipeline, including literature review, code implementation and execution, hypothesis generation and experimental design. PhysGym (Chen et al., 2025) transforms questions from PHYBench (Qiu et al., 2025) into interactive environments where the agent can run the system with various controllable interventions and the goal is to infer the exact symbolic expression governing the system. Yang et al. (2026) proposed an agent augmented with physics-based tools for this symbolic discovery setting. In contrast, MaD Physics focuses on the setting where the agent cannot alter the system and is only allowed to make measurements, and each observation is associated with a cost based on the quality of the measurement chosen. GravityBench (Koblischke et al., 2025), closely related to MaD Physics, consists of 2-body gravitational simulations where the agent is provided a budget for the number of observations allowed, after which the agent is required to answer questions with numerical or binary True-False answers. Complementary to GravityBench, MaD Physics consists of 3 different physical domains and involves reasoning about the cost-quality tradeoff for observations. Sawyer et al. (2024) evaluate the ability of foundation model based agents to actively gather information in Alchemy (Wang et al., 2021) and observe strong exploratory capabilities in simple environments but note that agents struggle to learn over multiple trials. Nägele and Marquardt (2025) evaluate the ability of agents to solve problems by exploring computational models of physical systems, allowing the agent to run the system with different configurations and the ability to observe the entire simulation outcome and is required to output a function which produces the correct answer. Agents for scientific discovery. Alongside benchmark development, recent work has pro-
posed domain-specific agentic systems for scientific discovery. PiFlow (Pu et al., 2025) studies principle-aware multi-agent collaboration for structured scientific exploration, PhysMaster (Miao et al., 2025) targets autonomous theoretical and computational physics research, and AI Agents for Variational Quantum Circuit Design (Knipfer et al., 2026) applies agentic search to quantum circuit architecture design. These systems complement benchmark efforts by illustrating concrete discovery workflows in specific domains, whereas MaD Physics is designed as an evaluation framework for information gathering and inference under observation constraints in physical environments.
3. Environment Design 3.1. Setup We consider a time-dependent dynamical system defined by a generalized state variable 𝑠 ( 𝑡 ). This state evolves according to a governing equation: 𝜕𝑡 𝑠 ( 𝑡 ) = 𝐹 ( 𝑠 ( 𝑡 ) , 𝑡 ) ,
𝑠 (0) = 𝑠0
(1)
where 𝐹 represents the physical laws driving the evolution and 𝑠0 is the initial configuration. The specific nature of 𝑠 ( 𝑡 ) depends on the physical domain. The agent interacts with the system through discrete measurements and predictions. The agent does not receive a continuous stream of data but must strategically choose distinct observations to maximize information gain within a fixed budget. The evaluation protocol for the agent consists of two distinct phases. Measurement. In the measurement phase, the agent operates under a total budget 𝐵 and interacts with the environment sequentially to collect a set of observations. Each observation choice is defined by the tuple ( 𝑡𝑘 , 𝑜𝑘 , 𝜎𝑘 ), where: • 𝑡𝑘 is the selected time of measurement. • 𝑜𝑘 is the selected observation function, mapping the full system state to a measurable quantity (e.g., measuring the position of a 4
MaD Physics : Evaluating information seeking under constraints in physical environments
t=0.00s
t=3.33s
t=6.67s
t=10.00s
t=0.05
t=1.15
t=2.25
t=3.35
Classical Mechanics: Gravity: Normal, Kappa = 0.0 t=13.30s
t=16.67s
Fluid Mechanics: Alter Type: none, Strength: 0.0 t=4.45 t=5.55
t=20.00s
t=23.30s
t=26.63s
t=30.00s
t=6.65
t=7.75
t=8.85
t=10.00
Figure 2 | Visualizations of the systems used in the Classical (Top), Fluid (Middle), and Quantum (Bottom) mechanics environments. specific particle or the fluid velocity at a specific coordinate). • 𝜎𝑘 is the selected noise scale (inverse fidelity). Upon making a selection, the agent receives a measurement 𝑦𝑘 corrupted by Gaussian noise: 𝑦𝑘 = 𝑜𝑘 ( 𝑠 ( 𝑡𝑘 )) + 𝜖𝑘 ,
2
𝜖𝑘 ∼ 𝑁 (0, 𝜎𝑘 )
(2)
Every observation incurs a resource cost 𝐶 ( 𝑜𝑘 , 𝜎𝑘 ). This cost function is strictly increasing with fidelity (meaning a smaller 𝜎𝑘 results in a higher cost) and varies based on the complexity of the observation function 𝑜𝑘 . The agent operates under a total budget 𝐵, such that the sequence of 𝐾 measurements must satisfy: 𝐾 ∑︁
𝐶 ( 𝑜𝑘 , 𝜎𝑘 ) ≤ 𝐵
(3)
𝑘=1
Additionally, the measurements of the agent are limited to a maximum time of 𝑇max to avoid extremely long trials. Prediction. Once the budget is exhausted or the agent reaches 𝑇max , the agent moves to the prediction phase. In the prediction phase, the agent is presented with a randomly sampled query time 𝑇query > 𝑇max . The objective is to predict the value of a target function 𝜙 ( 𝑠 (𝑇query )) based on the history of noisy measurements collected up to that
point. To obtain more reliable signals, the agent makes predictions for 𝑁 independent queries, and the error is averaged over the 𝑁 queries. 3.2. Environments We instantiate this framework in three distinct environments, each based on a physical law. Further details about the implementation of each environment are in Section A.
Design Principles. We pick classical, fluid and quantum mechanics as the domains for the environments. We pick these domains to cover different fundamental physical laws. Within each domain, the system considered in the environment is designed to be simple to keep the problem tractable. We study two specific alterations per domain, chosen for their impactful downstream effects and, in some cases, their documented unique properties. 3.2.1. Classical Mechanics We consider a system of 𝑁 spherical objects evolving in a 𝐷-dimensional Euclidean space, where 𝐷 ∈ {2, 3}. The generalized state 𝑠 ( 𝑡 ) consists of the configuration of all 𝑁 particles, where the 𝑖-th particle is defined by its position x𝑖 ∈ ℝ 𝐷 , velocity v𝑖 ∈ ℝ 𝐷 , radius 𝑅 𝑖 , and a scalar mass 𝑚0,𝑖 . 5
MaD Physics : Evaluating information seeking under constraints in physical environments
The fundamental dynamics are governed by Newton’s laws of motion, where the acceleration a𝑖 ( 𝑡 ) is determined by the net external force F𝑖 (x, 𝑡 ) acting on the particle. The observation function 𝑜𝑘 defines the subset of objects to be observed in the interactions in the measurement phase, and the prediction target 𝜙 is the position of all the particles at the queried time.
Error metric. We use the normalized root mean square error (nRMSE), where the normalization constant is the length of the diagonal of the bounding box 𝐿diag , as the predictive error for this domain. For a set of predictions ˆ𝑦 and ground truth 𝑦 , the metric is defined as:
Error =
1 𝐿diag
Alterations. We introduce two deviations from standard Newtonian mechanics: 1. Anisotropic Inertial Mass. The relationship between force and acceleration is modified by an anisotropic inertial mass tensor M𝑖 ( 𝑡 ), replacing the scalar mass. The modified law of motion is: a𝑖 ( 𝑡 ) = M𝑖 ( 𝑡 ) −1 F𝑖 (x, 𝑡 )
(4)
The mass tensor encodes an “inertial memory” that increases resistance to motion in directions of recent acceleration. It is defined as M𝑖 ( 𝑡 ) = 𝑚0,𝑖 I + 𝜅S𝑖 ( 𝑡 ), where 𝜅 ≥ 0 is a coupling constant and S𝑖 ( 𝑡 ) evolves according to: S¤ 𝑖 ( 𝑡 ) = − 𝜆 S𝑖 ( 𝑡 ) + a𝑖 ( 𝑡 )a𝑖 ( 𝑡 ) ⊤
(5)
Here, 𝜆 > 0 governs the memory decay rate. The alteration results in directional inertia on the object depending on the history of its motion. Bogoslovsky (1983) study a similar modification of classical mechanics derived through geometry rather than inertial memory. 2. Modified Gravity. The gravitational interaction between particles 𝑖 and 𝑗 acts along the vector r𝑖 𝑗 = x 𝑗 − x𝑖 with distance 𝑟 = ∥r𝑖 𝑗 ∥. We consider two distinct force laws for the magnitude 𝐹𝑔 ( 𝑟 ): 𝐹𝑔 ( 𝑟) = 𝐹𝑔 ( 𝑟) =
𝐺𝑚𝑖 𝑚 𝑗 𝑟 𝐺𝑚𝑖 𝑚 𝑗 𝑟2
1 + 𝐴 sin
2𝜋𝑟
𝑁
1 ∑︁ 𝑁
( ˆ𝑦𝑘 − 𝑦𝑘 ) 2 .
(8)
𝑘=1
3.2.2. Fluid Mechanics We simulate the dynamics of an incompressible viscous fluid in a two-dimensional periodic domain Ω = [0, 𝐿] 2 ⊂ ℝ2 . The state of the system is defined by the velocity field u(x, 𝑡 ) = ( 𝑢, 𝑣) ⊤ . The fundamental evolution of the system is governed by the standard Navier-Stokes equations: 𝜕u + (u · ∇)u = −∇ 𝑝 + 𝜈∇2 u 𝜕𝑡
(9)
∇·u=0
(10)
where 𝑝 is the kinematic pressure and 𝜈 is the kinematic viscosity. The system is an instantiation of a Kelvin-Helmholtz instability, which consists of two opposing streams separated by a thick interface that is perturbed. The observation function 𝑜𝑘 defines the locations at which the vorticity is measured during the measurement phase and the prediction target 𝜙 is the vorticity at randomly selected positions.
Alterations. We introduce a state-dependent “alien” gyroscopic forcing term, falien , to the righthand side of Equation (9). It acts perpendicularly to the velocity vector:
(6)
falien = C(u, 𝜔)u⊥
(7)
where u⊥ = ( 𝑣, −𝑢) ⊤ is the rotated velocity vector. The scalar modulation coefficient C is defined in two variants, parameterized by strength 𝛾 and frequency 𝛽 :
+𝜙
v u t
𝜆𝑔
We refer to Equation (6) as 1/R and Equation (7) as Ripple. The implications of 1/R form of gravity on galactic orbits have been studied in Valluri et al. (2012). In our evaluations, we also consider a combined alteration where we use the 1/R gravity along with 𝜅 = 10.
(11)
• Velocity Modulation: C = 𝛾 sin( 𝛽 |u| 2 ), which injects vorticity based on local kinetic energy. 6
MaD Physics : Evaluating information seeking under constraints in physical environments
• Vorticity Modulation: C = 𝛾 cos( 𝛽𝜔), where 𝜔 = ∇ × u is the scalar vorticity (since we only consider a 2D flow). This creates concentric, opposing force layers within turbulent eddies. To the best of our knowledge, the specific alteration has not been studied in the literature but similar forms have been studied in the literature on nonlinear Coriolis forces (Ringler et al., 2010). We also consider a combined alteration where the gyroscopic forcing term is a convex combination of the velocity and vorticity modulation variants. Error metric. We use the 𝐿2 error between the predicted vorticity (𝜔 ˆ ) and the true vorticity (𝜔) at the selected points as the predictive error. For a set of query points indexed by 𝑘, the error is computed as: √︄∑︁ Error = (𝜔 ˆ 𝑘 − 𝜔𝑘 ) 2 (12) 𝑘
3.2.3. Quantum Mechanics We consider a quantum system with two particles of masses 𝑚1 and 𝑚2 confined within a twodimensional rectangular domain Ω ⊂ ℝ2 . The state of the system is described by a joint wavefunction Ψ (r1 , r2 , 𝑡 ) ∈ ℂ, where r𝑘 ∈ Ω denotes the position of the 𝑘-th particle. The fundamental dynamics are governed by the time-dependent Schrödinger equation: 𝑖ℏ
𝜕 ˆ (r1 , r2 , 𝑡 ) Ψ (r1 , r2 , 𝑡 ) = 𝐻Ψ 𝜕𝑡
(13)
ˆ includes kinetic energy The Hamiltonian 𝐻 terms and confining potentials 𝑉 (r) (modeled as smoothed infinite square wells): ˆ =− 𝐻
2 ∑︁ ℏ2 𝑘=1
2𝑚 𝑘
∇2𝑘 +
2 ∑︁
𝑉 (r𝑘 )
(14)
𝑘=1
Unlike in the other environments, the act of measurement directly affects the state of the quantum system. Specifically, upon measurement the wavefunction collapses at a point based on the probability density. Thus, to account for this,
we allow the agent to perform multiple trials with the exact same initializations in the quantum environment. The observation function 𝑜𝑘 decides which of the two particles is observed and the prediction target 𝜙 is the probability of a particle being present in a given (randomly sampled) spatial region at the queried time.
Alterations. We introduce two modifications to the standard quantum formulation: 1. Non-linear Entanglement Initialization. Instead of a standard separable product state, the system is initialized with a non-standard nonlinear spatial correlation factor:
Ψ (r1 , r2 , 0) = N 𝜓1 (r1 ) 𝜓2 (r2 ) exp − 𝜆 ∥r1 − r2 ∥ 2
(15) where 𝜓1 , 𝜓2 are Gaussian wavepackets and 𝜆 determines the strength of the spatial correlation. 2. Generalized Born Rule. The system deviates from standard measurement postulates by employing a generalized probability measure based on the 𝐿 𝑝 -norm rather than the standard 𝐿2 -norm. ∫ ∫ The wavefunction is normalized such that | Ψ | 𝑝 𝑑 r1 𝑑 r2 = 1, and the joint probability density is given by: 𝜌 (r1 , r2 ) = | Ψ (r1 , r2 )| 𝑝
(16)
Measurement operations collapse the wavefunction according to marginal densities derived from this modified distribution. Aaronson (2004) studied the implications of the generalized Born rule ( 𝑝 ≠ 2) including superluminal signaling. We also consider a combination of both the alterations, using a measurement norm of 𝑝 = 1 along with non-linear entanglement with 𝜆 = 25.
Error metric. We use the 𝐿2 error between the predicted probability and the ground truth probability to evaluate the agent predictions: Error =
√︄∑︁
( 𝜌ˆ𝑘 − 𝜌𝑘 ) 2
(17)
𝑘
7
MaD Physics : Evaluating information seeking under constraints in physical environments
4. Empirical Results 4.1. Agent Design We evaluate Gemini 2.5 Flash Lite, Gemini 2.5 Flash, Gemini 2.5 Pro, and Gemini 3 Flash accessed through the Gemini API using a minimal agent scaffold. The scaffold allows access to a code execution tool where the agent is allowed to write and execute Python code. We use a minimal scaffold to focus on the model capabilities and exploring different scaffolds for the problem is an interesting direction for future work. The agent is provided some initial system instructions, along with some information about the environment prior to the beginning of the interaction state. We consider two different system instructions: Base and Strategy (see Section B for the full prompts). Base provides basic context about the evaluation protocol and general principles, whereas Strategy provides a structured strategy inspired by Bayesian experimental design (Rainforth et al., 2024). See Section A.1 for further implementation details.
we observe that the performance of the agent generally improves with increasing capabilities of the underlying model. Within the Gemini 2.5 set of models, we see the performance generally improves from Flash Lite to Flash to Pro. We also see performance improvements going from Gemini 2.5 Pro to Gemini 3 Flash, potentially due to improved multi-turn reasoning capabilities. We also observe that overall the agents perform worse on the altered configurations of the environment. However, there is no strong trend of the prediction error worsening with increasing intensity of the alteration. Finally, we observe that the Strategy system prompt, which instructs the agent to follow a systematic procedure for data collection, tends to work better on many of the tasks. This indicates that there is room for improvement for the performance of the base agent using prompt optimization strategies such as AlphaEvolve (Novikov et al., 2025) or GEPA (Agrawal et al., 2025). Beyond the system prompt, the agent scaffold itself can also be optimized to better fit the task (Lee et al., 2026).
4.2. Main results
4.3. Environment Variants
For each of the environments, we consider two different alterations, as discussed in Section 3. Further, for each alteration, we consider two separate configurations varying the strength and type of the alteration. Finally, we also consider a combined setup, where both the alterations are applied simultaneously. The prediction error is averaged over 𝑁 = 5 prediction queries. We report the prediction error for each configuration with 3 different random initializations. The results for the baseline agent with different models on the three environments are presented in Table 1, Table 2, and Table 3 respectively.
In addition to the results on the basic setup, we also demonstrate the versatility of the benchmark through different variants.
In Table 1, we observe that the Gemini 2.5 models tend to produce predictions that are out of bounds for the system we consider. While Gemini 3 Flash avoids out-of-bounds predictions, its predictions are still inaccurate, even with normal physics configurations. This is true even in Table 2 and Table 3. This indicates that even with knowledge of the underlying model, the agents can struggle to make correct inferences. However,
Visual observations. While the main results provide as observation the precise numerical values of the observation, in many settings, only visual observations are available to the agent. We can evaluate the multimodal reasoning abilities of agents by evaluating them on MaD Physics with visual observations. Instead of the numerical values, the agent is provided an image with the rendered state of the system. The agent has to infer the numerical value of the observation and then proceed with the task of choosing the next measurement. We evaluate Gemini 2.5 Flash and Pro in the visual setting. Table 4 shows that the trends about model capability and altered laws remain consistent with the numerical observation setup. However, the magnitude of the prediction errors is significantly higher. 8
MaD Physics : Evaluating information seeking under constraints in physical environments
Table 1 | Classical Mechanics Results. Prediction error in the classical mechanics system with various configurations for different base models. We do not clip out-of-bounds predictions so that the table separates models that produce runaway predictions from those that stay in-plausible-range. We consider these predictions to differentiate model performance. The combined configuration uses 𝜅 = 10 and gravity ∝ 1/𝑟 . Prediction Error (↓) Model
Altered
Prompt Normal
Anisotropic Inertia 𝜅 = 10
𝜅 = 20
𝐺𝑚1 𝑚2 𝑟
Altered Gravity h i 1 + 𝐴 sin 2𝜆𝜋𝑟 + 𝜙
Combined
𝐺𝑚1 𝑚2 𝑟2
Gemini 2.5 Flash Lite
Base + Strategy
6.61 48.32
13.78 115.02
831.97 74.62
638.25 463.83
253.44 1183.7
1241.11 964.43
Gemini 2.5 Flash
Base + Strategy
5.97 7.93
35.36 12.32
4.42 60.36
28.22 26.45
66.75 44.32
198.69 23.33
Gemini 2.5 Pro
Base + Strategy
1.93 0.67
2.12 1.56
13.41 0.37
11.40 1.22
15.67 0.50
38.72 0.49
Gemini 3 Flash
Base + Strategy
0.29 0.38
0.36 0.39
0.88 0.37
0.31 0.37
0.39 0.43
0.38 0.35