Energy-Structured Latent World Models with Neural Time Fields for Physically Constistent Open-World Motion Planning

Energy-Structured Latent World Models with Neural Time Fields for Physically Constistent Open-World Motion Planning Yapeng Liu1,2,3 , Yuanzhao Zhai1,2 , Bo Ding1 , Huaimin Wang1,2 , Lin Wang3 1 2

PDL Lab, College of Computer Science and Technology, National University of Defense Technology State Key Laboratory of Complex & Critical Software Environment, Changsha 410073, Hunan, China 3 EmPACT Lab, Nanyang Technological University, Singapore

arXiv:2608.09876v1 [cs.RO] 10 Aug 2026

Abstract Physically consistent motion planning remains a fundamental challenge in embodied AI, as generated trajectories must strictly conform to real-world execution dynamics. While latent world models offer a promising approach by predicting these dynamics, existing methods learn unconstrained future representations where absorbed physics remains implicit. Therefore, they fail to form reusable physical knowledge, which compromises reliability in unpredictable openworld navigation. To address this, we propose a novel EnergyStructured Latent World Model (ELWM). Our key idea is to structure the ELWM latent state to explicitly carry energy and momentum, ensuring strictly causal transitions via dissipation and control ports. Trained on multimodal RGB-D and inertial interaction histories, our model guarantees physically consistent predictions. We further implement this for motion planning by constructing Physics-Conditioned Neural Time Fields (PC-NTF), a key technical cornerstone that integrates ELWM into an arrival time field via the Eikonal equation to yield a physically-informed navigation policy. Across held-out scenes, our evaluation reveals significant improvements. Compared to generic latent models, PC-NTF reduces 0.8-s motionprediction NRMSE from 0.36 to 0.29. Against Active Neural Time Fields, it improves navigation success from 81.3% to 89.7% and SPL from 0.64 to 0.73, while cutting the physical collision rate from 12.1% to 5.8% and the Eikonal residual from 0.083 to 0.031. Beyond these targeted gains, our results demonstrate that embedding explicit physical structures into latent spaces intrinsically bridges the gap between predictive world models and safe, dynamically feasible motion planning.

Introduction Physically consistent motion planning is a fundamental challenge in embodied AI, the trajectory is useful only when the robot can execute it under its current dynamics (Hsu et al. 2002; Liu et al. 2025). The collision-free geometric path may become unsafe or inefficient when dissipation payload, actuation limits, or the robot’s dynamic state changes, and these factors often have weak geometric signatures and become observable through interaction (Pham, Caron, and Nakamura 2013; Bohg et al. 2017; Liu et al. 2026). Physical consistent motion planning in open-world environment therefore requires a predictive model of how the robot–environment Preprint

A

Interact

Robot navigate to the destniation, over the obstacle and get in the room.

B ELWM—PC-NTF

C Navigation

ELWM WMs Decoder

Future Prediction

Latent Physics

Success Navigation!

Avoidance Open Door

Action Response History

D-1. Obstacle Avoid

PC-NTF

Motion Planning

D-2. Phys Efficient D-3. Interaction

Figure 1: ELWM–PC-NTF bridges action-conditioned physical prediction and motion planning. (A) Robot interactions need action–response capability, (B) ELWM predicts future motion and latent physical dynamics, and PC-NTF combines them with neural time fields. (C) System closedloop execution produces obstacle-aware, physics-informed navigation towards the goal. system responds to execution (Ha and Schmidhuber 2018; Kumar et al. 2021; Huang et al. 2026). Latent world models provide a promising basis for this predictive component (LeCun et al. 2022; Bar et al. 2025). By encoding multi-modal observation and interaction histories into latent states, world models support action-conditioned rollouts without reconstructing every future observation (Hafner et al. 2019b,a, 2023). Recent models extend this principle to motion planning and navigation (Zhou et al. 2025; Chen et al. 2025). However, latent prediction does not imply reusable physical knowledge or formal physical consistency (Lutter, Ritter, and Peters 2019; Shang et al. 2026). Existing methods mostly optimize unconstrained future representations, leaving dissipation, interaction response, and other physical regularities entangled in task-, env- and embodiment-specific transition weights (Ye et al. 2026; Gong et al. 2026; Chen et al. 2026). They expose neither a causal physical structure nor a predicted quantity that motion planner can generally consume. This limitation is especially consequential in open-world navigation, where a planner must adapt its decision to physical interactions that are not determined by scene geometry alone. To make physical dynamics reusable, we study the latent transition of how to learn transferable physical regularities and effectively implement them in motion planning.

Consequently, we propose the Energy-Structured Latent World Model (ELWM), which encodes multimodal RGBD and inertial interaction histories, together with executed trajectories, into the general physical regularities and dynamic latent phase state (Desai et al. 2021). We formalize the latent stored-energy function and structure the explict port-Hamiltonian (pH) transition for navigation dissipation and control ports (Zhong, Dey, and Chakraborty 2019). This construction imposes an action-conditioned, energy-balancecompatible transition structure. Given a candidate command sequence, ELWM rolls this structured state forward and decodes planner-observable consequences, including the predicted pose, velocity, interaction, and traversal progress. The learned structure can therefore be evaluated through both latent future prediction and port-Hamiltonian motion planning dynamics. ELWM exposes explicit physical interaction consequences, while these predictions do not specify how the robot should progress toward a navigation goal by themselves. The motion planner requires a global value representation which connects locally achievable outcomes to global navigation cost (Maes et al. 2026). Neural time fields provide this complementary substrate by interpreting time-to-go as a continuous arrival-time function constrained by the Eikonal equation, and Active Neural Time Fields (ANTF) update the field from the observed geometry of the scene incrementally, the resulting arrival time and its goal-directed gradient efficiently organize global geometric guidance (Ni and Qureshi 2022; Liu, Ni, and Qureshi 2025). However, ANTF derives this guidance primarily from static geometry map, it induces the same preference even when ELWM predicts different traversal responses under changes in payload or actuation. The missing link is therefore a motion planning system that utilizes ELWM observable, candidate-dependent predictions to condition arrival-time decisions, beyond the mappings from states, poses, or dissipation to an Eikonal propagation speed. We close this interface gap through Physics-Conditioned Neural Time Fields (PC-NTF) system. The overall motivation are summarized in Figure 1, ELWM and PC-NTF bridges the physical prediction and motion planning. PCNTF decomposes motion planning responsibility between ELWM’s local interacting execution prediction and ANTF’s global geometric time-to-go envelope. ANTF first supplies the nominal descent direction and a shared set of feasible action candidates. Starting from the same interaction history, the trained ELWM predicts how each candidate will be executed over a short horizon. PC-NTF then queries the neural time field at each predicted terminal position and then fixes the physics conditioned traversal delay derived from the predicted interaction progress from a physics latent decoder. The resulting model-based time-to-go backup treats the worldmodel consequence as an action-conditioned physical stage cost and the neural time field as a geometric terminal value. It thereby connects ELWM to arrival-time planning without interpreting each latent mechanics as the Eikonal equation participant. Because the correction is candidate dependent online, PC-NTF can change the locally executed decision while preserving the global geometric structure of ANTF, without requiring ELWM to predict paths to the goal or the

time field to be retrained for each physical interaction condition. We evaluate this framework along two complementary axes that mirror its two technical components. At the worldmodel level, ELWM is compared with a capacity-matched generic latent world model in MPC controller using observable multi-step motion NRMSE, action interventions, and Hamiltonian dynamics diagnostics on held-out physical regimes and scenes. At the planning level, PC-NTF is compared with geometry ANTF and we report navigation success, SPL, physical collision rate, arrival-time accuracy, Eikonal residual, and planning latency. This separation allows the contributions analysis of generic prediction, energystructured dynamics, and neural time-field guidance to be measured independently. Experiments show that ELWM obtains motion NRMSEs of 0.705 and 0.697 on the held-out regime and scene, respectively, while maintaining a discrete energy-balance residual of 7.12 × 10−7 ; action shuffling increases latent prediction error by approximately 10%. These results support action-conditioned prediction and consistency of the physical transition, whereas navigation positive improvement from success rate, latent error and Eikonal residual also indicates PC-NTF motion planning efficiency. Our contributions follows: • ELWM Construction. We introduce ELWM, an energystructured latent world model that organizes actionconditioned dynamics through a latent phase state with explicit stored-energy, dissipation, and control structure. • PC-NTF System Implementation. We propose PCNTF, the motion planning system that utilize ELWM predicted execution consequences with NTF geometric time-to-go through specific terminal states and physically dimensioned traversal delays. • Efficient motion planning results. We evaluate ELWM and PC-NTF through the physical consistent prediction and motion planning in open-world environments. Experiment shows that we reduce 0.8-s motion-prediction NRMSE from 0.36 to 0.29, improve navigation success from 81.3% to 89.7% and SPL from 0.64 to 0.73.

Related Work Physically Consistent Latent World Models. Latent world models compress observation action-conditioned histories and future dynamics into meaningful representations, which can support planning and policy learning (Ha and Schmidhuber 2018; Hafner et al. 2019b,a, 2023). Recent models extend this principle to pretrained visual features, navigation, and robot policies (Zhou et al. 2025; Rao et al. 2026; Chen et al. 2026). These results establish the decision relevance of latent rollouts, but predictive sufficiency does not make robot–environment interaction mechanics reasonable: effects such as payload and dissipation may remain entangled in unconstrained latent transition (Nie et al. 2026; Schölkopf et al. 2021). Physical consistent world models were proposed to address this limitation, including physics-informed world models and mechanics-structured networks. The former incorporate differentiable simulators or physical supervision into predictive learning (Li et al. 2025; Shang et al.

2026), whereas the latter constrain the learned transition itself (Greydanus, Dzamba, and Yosinski 2019; Zhong, Dey, and Chakraborty 2019). More recent approaches bring these principles closer to world models construction, AC-HGN embeds actions as external forces in an abstract Hamiltonian phase space (Troch, Mets, and Mercelis 2025); Phys-JEPA structures physical and residual components of latent prediction (Nie et al. 2026); and PH-Dreamer combines actioncontrolled energy flow and dissipation into recurrent worldmodel dynamics (Luan and Shi 2026). These advances establish that physical consistent structure can improve the latent prediction and control, while they mainly evaluate with in the learned model or policy. ELWM targets on the motion planning, extracting the navigation regularities and organize action-conditioned prediction through dissipation. In this way, ELWM provides the interaction dynamics and shows the efficiency on jointly motion planning. Neural Time Fields for Motion Planning. Efficient motion planning can be characterized by Hamilton–Jacobi and Eikonal equations, with classical solutions obtained by dynamic-programming and Fast Marching methods (Lions 2006; Sethian 1996). For motion planning, neural fields recently have emerged as suitable environment representations due to their compactness and continuity property (Park et al. 2019; You et al. 2023), and neural PDE solvers approximate Eikonal solutions continuously through data and equation residuals (Raissi, Perdikaris, and Karniadakis 2018; Smith, Azizzadenesheli, and Ross 2020; Bin Waheed et al. 2021). NTFields applies this formulation to robot configuration spaces without expert paths (Ni and Qureshi 2022); Active NTFields further learns arrival-time maps online in open-world environments (Liu, Ni, and Qureshi 2025; Zhong et al. 2025); and Eikonal caging extends the formulation to contact-rich robot arm manipulation (Zhang et al. 2026). These methods define physical consistency primarily through the Eikonal PDE and geometry-derived propagation speed, but they did not take the dynamic online interaction consequences in consideration. PC-NTF lifts the arrival-time Eikonal formulation by using Practical physical traversal costs through its decoder and the physics condition, to form the online interactive motion planning framework. Open-World Motion Planning Under Changing Dynamics. Open-world motion planning requires navigation without fully known maps and fixed execution conditions (Du Toit and Burdick 2011). Classical sampling-based planners search for collision-free paths in a given configurationspace representation (Karaman and Frazzoli 2011), while learning-based planners accelerate this search using learned trajectory proposals and classical feasibility checks (Qureshi et al. 2020). These methods represent feasibility through geometric collision constraints. Further, experience-driven navigation complements geometric planning by learning local reachability or traversability from robot interaction. BADGR learns navigational affordances from self-supervised experience (Kahn, Abbeel, and Levine 2021) and WayFAST uses online traction estimates to supervise terrain-traversability prediction (Gasparino et al. 2022). However, their plannerfacing quantities are typically expressed as collision. These evaluation do not explictly predict how each candidate com-

mand sequence will execute under the robot–environment setting. PC-NTF evaluates feasible motion candidates under the robot’s current interaction response along with ELWM, allowing open-world decisions to change with payload, dissipation, actuation limits, and dynamic state.

Method Preliminary Problem Setting. We consider a differential-drive robot that navigates in a planar workspace Ω ⊂ R2 toward the goal g in Eikonal form, its state can be expressed as rt = (xt , ψt , ξt ) ,

ξt = [vt , ωt ],

(1)

[vtcmd , ωtcmd ]. An up-

and the executed control is ut = stream navigation task module with VLM foundation model supplies the goal xg from dataset, µr denotes robot metainformation (mass/payload, velocity and acceleration limits). Actions at = (νt , ∆t) are the velocity commands received t by the base, and Htphy = {ophy τ , ξτ , aτ −1 , µr }τ =t−L collects the recent proprioceptive and navigation interaction history. Hypothesis. The physical quantity delivered to the planner is isotropic traversal-speed field: within one replanning window, the maximum speed the robot can stably sustain at each location is represented by scalar. Orientation-dependent friction, turning-radius limits, and acceleration state are absorbed into this scalar and enforced at execution time by an explicit constraint layer. For detailed neural time field calculation, we represent them inside through an anisotropic and state-space Hamilton–Jacobi formulation, which we analyze in PC-NTF realization (Bardi, Dolcetta et al. 1997). System overview. PC-NTF couples a physical prediction world model with NTF planner through motion planning interface, the inference loop is summarized as ELWM

Ht −−−−→ st , n oN NTF (i) (TA , rt , g) −−−→ Ut , i=1  n  oN PC-NTF (i) st , Ut , TA −−−−−→ uexec −→ Ht+1 . t i=1

ELWM summarizes the interaction history as st ; NTF uses the current state, goal, and arrival-time field TA to generate shared candidates; and PC-NTF predicts their physical consequences, reranks them with the time field, and executes the selected feasible action. The observed transition then updates Ht+1 , closing the feedback loop.

Energy-Structured Latent World Model Figure 2 demonstrates the ELWM overview from interaction latent encoding and pH-constrained motion prediction to PCNTF. Latent physical state. The ELWM world model encoder Eθ mappings the interaction history to the latent physical state: (Mt , Qt , Vt ) = Eθ (Hphy (2) t ), where Mt captures the slow physical regularity dynamics (payload, ground response, actuation delay), and Qt , Vt ∈

Physical Energy State

RGB-D, Pose,

Port-Hamiltonian (pH) Transition

Physical Regularities iGibson Physical History

Observavle Future

Latent Configuration

ELWM Encoder

General Momentum

Future Latent State

Env. Interaction Dynamics

PC-NTF (Fig.3)

Figure 2: Overview of ELWM. Physical history and interaction dynamics are encoded into physical and latent state; a controlled, dissipative port-Hamiltonian transition formalizes the prediction latent prior as motion futures P for PC-NTF planning (in Fig.3). ELWM Training by the state future consistency. Rdz /2 are latent generalized coordinates and velocities computed from relative motion features. The latent inertia G links velocity and momentum, and we parameterize it by Cholesky factor ct with robot meta-info µr . Gt = Gθ (Mt , µr , ct ) ≻ 0,

Pt = Gt Vt ,

(3)

We take Zt = [Qt , Pt ] as the port-Hamiltonian latent phase state, and the pairing in Eq. (3) supplies consistent velocity– momentum relation for the transition below. Latent stored energy. ergy as

We formalize the latent stored en-

−1 Eθ (Qt , Pt ; Mt , µr ) = 12 P⊤ t Gθ Pt + Uθ ρ(Qt ), Mt , µr , {z } | | {z } Kθ

Uθ

(4) with kinetic energy Kθ and potential energy Uθ . ρ(·) is a feature map restricted to relative quantities, which makes Uθ invariant to global translations of Qt . Since kinetic parameterization gives ∂Kθ /∂Pt = G−1 t Pt = Vt , the Hamiltonian flow of Qt coincides with the encoder’s latent velocity. Port-Hamiltonian transition. Navigation robot exchanges energy with environments: actuators inject work, and the interaction dissipates it. Following port-Hamiltonian formalism (Zhong, Dey, and Chakraborty 2019; Desai et al. 2021), latent transition generated by the dissipation paradigm
Żt = J − Rθ (Zt , Mt , µr ) ∇Z Eθ + Bθ (Zt , Mt , µr ) ut , (5)  0 I with skew-symmetric operator J = −I , dissipation 0 Rθ ⪰ 0, control port Bθ , and latent control input ut = Eu (at , ot , µr ) mapping velocity and actuator commands. The ELWM physical transition structural property follows directly, defining the port output yt = Bθ⊤ ∇Z Eθ and using J⊤ = −J, dEθ = ∇Z Eθ⊤ Żt = − ∇Z Eθ⊤ Rθ ∇Z Eθ + u⊤ (6) t yt . dt Latent energy changes is the injected system latent work minus a non-negative dissipated power, and it is conserved when ut = 0 and Rθ = 0. Equation (6) holds as an identity of the vector field in Eq. (5) for energy balance; acceleration and friction are explained by the control port and dissipation terms. For momentum, the pH transition gives

Ṗt = −∂Eθ /∂Qt + [Bθ ut − Rθ ∇Z Eθ ]P ; when the Hamiltonian is invariant along a latent direction and that direction is unforced and dissipation-free, the corresponding momentum component is constant. Future predictions of latent states are produced by integrator ΦpH ∆t . Since Gθ depends on Qt , the energy in Eq. (4) is non-separable, we use the implicit midpoint as discretegradient updates for dissipation. Training. In our design, physical regularities are carried by the transition structure, so the world model training task reduces to future prediction. Targets are encodings of future phy + histories by the same encoder, Zt+k = Eθ (Ht+k )Z , and the objective is K

1 X wk 2 + Ẑt+k − Zt+k + λsig RSIGReg , 2 K dz k=1 (7) the task-level future-latent prediction objective with representation-geometry regularizer: RSIGReg is the isotropic-Gaussian embedding regularizer of LeJEPA (Balestriero and LeCun 2025) applied to current, target, and rolled-out latents, which prevents collapse. Gradients propagate through the structure-preserving rollout into Eθ , Rθ , Bθ , Eu , we control the port-Hamiltonian mechanism constraints make effects that the model can reduce prediction error only after the physics. Energy residuals εenergy = (∆Êt − Ŵt )2 and symmetrymasked momentum residuals ar e logged for checkpoint selection; reinstating them in the loss is evaluated as an ablation. After convergence the world model exposes the physical context and world state for PC-NTF navigation planning. LELWM =

PC-NTF System Construction PC-NTF implements the ELWM priors with neural time fields for complete navigation system, ELWM supplies implicit physics, and the Eikonal solver formalize the time fields initialization. Figure 3 illustrates how PC-NTF reranks NTF candidates using ELWM-predicted motion consequences in sequence. We build PC-NTF in three steps—latent physics decoder, condition time fields, and the generalized time-field

(a) NTF Proposes NTF

Candidates:

Descent Possible Refuse

(b) Dynamics Predicts Robot State

:

Env. State

Candidate 1 Candidate 2 Candidate 3 Physical Regularities

ELWM

Arrival-time Envelope

(c) PC-NTF Decides Step

ELWM (Fig.2) Candidates

Future Prediction Action

Task Goal

NTF

NTF

Select & Exec

NTF

NTF

Update Interaction Hist.

Figure 3: PC-NTF motion planning pipeline. (a) NTF proposes shared motion candidates -> (b) ELWM predicts their actionconditioned execution consequences -> (c) PC-NTF selects the candidate with the most efficient traversal delay. problem(Liu, Ni, and Qureshi 2025):

geometry efficiently in navigation, but it cannot by itself distinguish physical changes in dissipation, actuation, or motion sgeo,t (x) ∥∇x T ∥2 = 1 −→ seff,t x; ζt (x), µr ∥∇x T ∥2 = 1, | {z } {z } response. | geometry Eikonal physics-conditioned Eikonal PC-NTF complements the geometric field with action(8) conditioned predictions from the ELWM. The NTF descent where ζt (x) = Emap (bt , x) denotes map construction at the direction first defines a nominal motion, from which the planmotion planning. Equation (8) generalizes the solver objecner forms a small set of feasible alternatives. ELWM rolls tive, and additional dependency (ζt , µr ) is available before out every alternative from the same interaction history and planning begins. decodes its predicted pose and velocity. These predictions P go through “pH” and decoded by the physics latent decoder. Physics latent decoder. Learning form the ELWM future determine two quantities that are directly meaningful to the prediction context, we take it as the action-conditioned contime field: the location that the robot is expected to reach sequences of motion planning. To better use the physical laand any additional time caused by slower-than-commanded tent prediction for motion planning, we design the PC-NTF progress. For candidate i, PC-NTF evaluates physics latent decoder which maps latent into traversal speed,   and this decoder is trained by Huber loss from the physics(i) (i) (i) T = T x̂ , x + ∆Tphys . (11) ∗ g t+H speed envelopes s . Given the interaction history H and

t

t

(i)

a candidate command sequence Ut , the decoder exposes planner-facing representation     (i) (i) (i) (i) (i) (i) Γt = AWM Ht , Ut = τ̂t , x̂t+H , ∆TWM , ρt , (9) (i) (i) where τ̂t is the decoded observable motion, x̂t+H is its (i)

(i)

terminal state, ∆TWM ≥ 0 is traversal delay, and ρt summarizes physical predicted tracking, safety, and control consequences. NTF Construction and Physics Condition. NTF represents scene by a continuous arrival-time field T (x, g), which estimates the remaining travel time from a location x to the goal xg . Its geometry-derived speed S(x) and arrival time follow the Eikonal relation S(x) ∥∇T (x, xg )∥2 = 1,

T (g, g) = 0.

(10)

Time field is parameterized by PINN neural network and fitted from location–goal pairs using the Eikonal equation and the zero-time boundary condition at the goal. As new geometric observations become available, Active NTF updates the clearance-dependent speed supervision and the corresponding arrival-time representation without requiring expert trajectories (Ni and Qureshi 2022). Regions with greater obstacle clearance admit faster propagation and therefore smaller arrival time, while following the negative field gradient provides a goal-directed motion cue. This field captures scene

  (i) Neural time fields solve the Eikonal equation, T x̂t+H , xg is the geometry time remaining from the predicted termi(i) nal location. ELWM provide the Tphys after decoder, representing the physical regularities and predicted motion by the world model along with the navigation progress. Consequently, a geometrically attractive motion receives a larger cost when the current physical regime makes it difficult to realize. PC-NTF therefore physically conditions the use of the neural time field without treating the latent state, mechanical energy, or dissipation matrix as an Eikonal speed. Motion-planning and navigation system. At every replanning step, the system follows one execution order: query the NTF field, generate shared feasible candidates, predict their motion with the world model, and evaluate them using Eq. (11). Candidates with poor tracking, speed-limit violations, high uncertainty, or abrupt control changes are penalized, and the first action of the lowest-cost feasible candidate is executed. The resulting pose, velocity, and executed action are then appended to the interaction history before the next planning step. The ELWM and NTF are trained separately from interaction trajectories and scene geometry, respectively, and remain frozen during system evaluation. This construction preserves NTF as the source of global geometric guidance while using the world model to supply the physical consequences that a geometry-only time field cannot observe.

Equation (8) is also the mechanism by which the world model influences planning: when the readout reports reduced capability—higher dissipation, heavier payload— seff,t drops, the Eikonal constraint forces larger time gradients, the same Euclidean distance costs more arrival time, and planned paths bend away from physically slow regions without any explicit penalty. Motion generation follows the solved field:   ∇T (x; xg ) Sexec = ΠCt − seff,t (x) , (12) t ∥∇T (x; xg )∥2 + ϵ where the implied speed ŝt = 1/(∥∇x Tt ∥2 + ϵ) ≈ seff,t serves as a built-in diagnostic of how faithfully the solver absorbed the physical prior, and ΠCt is a deterministic projection enforcing occupancy boundaries, unknown-space rules, velocity and acceleration clipping, and emergency stops.

Experiments Experimental Setup Environments. (i) iGibson indoor scenes (Shen et al. 2020) with cluttered multi-room layouts; (ii) held-out environments sampled from HM3D (Ramakrishnan et al. 2021) for topological complexity, formalize into iGibson format; (iii) a terrain-variation suite with heterogeneous ground surfaces (carpet, tile, gravel) and payload changes that alter interaction dynamics. Train/test splits are by scene, surface, and robot meta-configuration; adjacent frames have bounding split. Baselines. The main comparison includes Active Neural Time Fields(Liu, Ni, and Qureshi 2025), Generic world models (the capacity matched generative action-consistent world model)(Chen et al. 2026) with the MPC budget, and the complete ELWM+PC-NTF system. Comparison axes. We organize the evaluation following two axes. At the world-model axis, we ask whether imposing an energy-structured transition improves action-conditioned prediction over a capacity-matched unconstrained latent world model. ELWM is compared with a capacity-matched Generic WM (Chen et al. 2026). At the planning system axis, we ask how these predictions change the motions selected from the same ANTF candidate set, PC-NTF is compared with geometry ANTF (Liu, Ni, and Qureshi 2025). Metrics. The world-model axis uses 0.8-s motion-prediction normalized root mean square error (NRMSE@0.8s; lower is better), pose error, and twist error. The planning axis uses success rate, success weighted by path length (SPL), physical collision rate, and Eikonal residual.

Experimental Results Physical Consistent World Model Prediction. Table 1 isolates the predictive contribution of the energy-structured transition. ELWM reduces observable-motion NRMSE from 0.36 to 0.29 relative to Generic WM, an 19.4% reduction. Pose and twist errors respectively decrease by 10.8% and 1.8%. Along with the experiment, a larger actionintervention ratio (1.193 to 1.009) shows that the ELWM rollout depends more strongly on the executed control. Also, there is a smaller positive difference in slow-M intervention (1.062 to 1.033), showing that Mt has the potential to identify reusable physical regularities.

Method Generic WM ELWM

NRMSE @0.8s↓ Pose↓ Twist↓ Action int.↑ 0.36 0.29

0.749 0.669

0.489 0.480

1.009 1.193

Table 1: World-model physical consistency analysis. Prediction errors are evaluated on paired cases under identical observations, action constraints, and candidate motions in indoor scenes. Metrics Setting Motion Random init. NRMSE ELWM Latent MSE

Random init. ELWM

Regime↓ Scene↓ 7.753 0.705

7.699 0.697

↓ 90.9%

↓ 91.0%

0.1695 0.0856

0.1656 0.0835

↓ 49.5%

↓ 49.6%

Table 2: Comparison with random initialization, showing ELWM learning effects and generalization on held-out cases. Physical world model analysis includes energy consistency, our trained ELWM checkpoint attains an energybalance residual of 1.88 × 10−7 , and its predicted dissipated power is nonnegative on all evaluated rollout steps, verified that the numerical transition respects the imposed dissipative pH identity and ELWM energy-structured model. ELWM substantially improves prediction on both heldout axes, motion NRMSE decreases from 7.753 to 0.705 on unseen physical regimes and from 7.699 to 0.697 on unseen scenes, corresponding to relative reductions of 90.9% and 91.0%. Latent MSE is also reduced by 49.5% and 49.6%. The closely matched reductions show that ELWM training process effectively improves both held-out tests on hold out. Motion Planning and Navigation. Table 3 shows the motion planning systems comparison, and Figure 4 visualizes the matched planning comparison. For efficient and physical motion planning, PC-NTF is intended to improve physics-awared navigation because ELWM supplies a physics-conditioned estimation of how the current system will respond to navigation interaction based on learned physical regularities. This prior modifies the effective propagation cost and penalizes candidates that are trajectory valid but physically slow or difficult to execute, PC-NTF formalize them into Neural Time Fields for motion planning. Compared with ANTF, PC-NTF improves SR from 81.3% to 89.7%, SPL from 0.64 to 0.73, reduce Eikonal residual from 0.083 to 0.031. ELWM physics prior bring more latency than origin ANTF, while we expose that PC-NTF latency is better than MPC control port (371.7 to 193.2). We stress-test ELWM along two complementary held-out axes to test the learned transition reliability beyond the training configurations. The held-out-regime split changes the robot–environment interaction conditions, whereas the heldout-scene split changes the surrounding geometry. Both evaluations use the same 0.8 s recorded-action rollout horizon.

ANTF

Generic WM + MPC

PC-NTF (ours)

SPL ↑ 89.7

0.74

0.8

Phys. Coll. ↓ 0.73

0.64

75

0.6

SPL

Success Rate (%)

81.3

87.1

Collision Rate (%)

SR ↑ 100

50 25

0.4 0.2

0

12.1

10

0.0 ANTF

Generic WM PC-NTF + MPC

Generic WM PC-NTF + MPC

Generic WM PC-NTF + MPC

371.7

400

0.031

0.00

Latency (ms)

Eikonal Residual

0.14

0.083

0.05

ANTF

Latency ↓

Eik. Res. ↓

0.10

5.8

0 ANTF

0.15

6.5

5

300 193.2

200 100

35.3

0 ANTF

Generic WM PC-NTF + MPC

ANTF

Generic WM PC-NTF + MPC

Figure 4: Motion-planning and navigation results comparison, PC-NTF indicates ELWM+PC-NTF system. Method

SR Phys. Coll. Latency Eik.↓ (ms)↓ (%)↑ SPL↑ (%)↓

ANTF 81.3 Generic WM 87.1 + MPC ELWM 89.7 + PC-NTF

0.64

14.4

0.083

35.3

0.74

6.5

0.143

371.7

0.73

5.8

0.031

193.2

Table 3: Motion-planning and navigation performance. All methods use identical maps, observations, action constraints, candidate sets, and planning budgets. A randomly initialized ELWM with the same architecture serves only as a sanity reference, so Table 2 measures whether training yields action-conditioned prediction across the two held-out splits.

Scalability and Ablation Studies Scalability. We investigate whether the learning behavior of ELWM depends on tuned optimization setting. Specifically, ELWM and the Generic WM baseline are trained under a matched protocol while varying combinations of the learning rate, weight decay, and the SIGReg coefficient λsig ; all other data, optimization, and evaluation settings are kept fixed. As shown in Figure 5, ELWM consistently reaches a lower-loss region within the first few epochs and maintains lower training loss, validation loss, and future-latent consistency error across the tested configurations. The small variation between ELWM runs further indicates that this behavior is not tied to a single parameter combination. Meanwhile, the two model families eventually reach a comparable range of observablemotion error, showing that the improvement in latent prediction is not obtained by sacrificing motion-level accuracy. These results demonstrate that the physics-structured transition provides stable and parameter-robust learning across the evaluated optimization configurations. Ablations. Table 4 isolates where the PC-NTF gain comes from. The ANTF row measures geometry-only propagation. Replacing pH dynamics with a capacity-matched generic transition tests the ELWM structure; neutralizing the ELWM condition tests whether learned physics actually reaches the time field; and the scalar-only variant separates global speed calibration from spatial capability and candidate-specific traversal delay.

Generic 001

Generic Generic 001 002

Generic 002

Generic Generic 003 003

ELWM 001

ELWM ELWM 001 002

ELWM 002

Figure 5: Convergence analysis across scalability training configurations. Compared in terms of (A) validation loss, (B) training loss, (C) latent-consistency error, and (D) observable-motion error, demonstrating ELWM stable learning beyond the hyperparameter setting. Variant ANTF PC-NTF w/o physical condition PC-NTF w/ shuffled Mt PC-NTF w/ generic WM ELWM + PC-NTF

NRMSE↓

SPL↑

Phys. Coll.↓

Latency (ms)↓

1.269 2.347 1.036 1.015 1.012

0.597 0.483 0.580 0.654 0.661

14.35% 18.75% 22.11% 6.54% 5.79%

40 279 266 479 278

Table 4: Ablation study of PC-NTF. Prediction horizon and latency is measured per replanning cycle.

Conclusion In this work, we investigated how interaction-derived physical knowledge can be represented by latent world models and translated into executable guidance for motion planning in unknown environments. We propose the EnergyStructured Latent World Model (ELWM) that structures action-conditioned prediction through port-Hamiltonian dynamics with explicit dissipation and control ports. For reusable physical regularities learning, we introduced Physics-Conditioned Neural Time Fields (PC-NTF) as the navigating motion system realization. PC-NTF connects predicted motion consequences to the global arrival-time representation of Neural Time Fields through a physically interpretable time-to-go objective. This construction preserves the efficiency and geometric guidance of neural time fields while allowing changes in robot–environment dynamics to inform motion selection. Our two-level evaluation analyses physically consistent world-model prediction and closed-loop motion planning, providing evaluation for ELWM physical consistent latent prediction quality and system-level planning benefit. The physical core evaluation also supports ELWM prediction and physical consistency on held-out scopes; paired navigation evaluation provides the corresponding benefit of PC-NTF motion planning. Future Work. We are delighted to extend PC-NTF from planar navigation to anisotropic, state-space, and contactrich motion planning, and to evaluate transfer across broader robot configurations and real-world environments.

References Balestriero, R.; and LeCun, Y. 2025. Lejepa: Provable and scalable self-supervised learning without the heuristics. arXiv preprint arXiv:2511.08544. Bar, A.; Zhou, G.; Tran, D.; Darrell, T.; and LeCun, Y. 2025. Navigation World Models. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 15791– 15801. IEEE. Bardi, M.; Dolcetta, I. C.; et al. 1997. Optimal control and viscosity solutions of Hamilton-Jacobi-Bellman equations, volume 12. Springer. Bin Waheed, U.; Haghighat, E.; Alkhalifah, T.; Song, C.; and Hao, Q. 2021. PINNeik: Eikonal solution using physicsinformed neural networks. Computers & Geosciences, 155: 104833. Bohg, J.; Hausman, K.; Sankaran, B.; Brock, O.; Kragic, D.; Schaal, S.; and Sukhatme, G. S. 2017. Interactive perception: Leveraging action in perception and perception in action. IEEE Transactions on Robotics, 33(6): 1273–1291. Chen, D.; Shukor, M.; Moutakanni, T.; Chung, W.; Yu, J.; Kasarla, T.; Bang, Y.; Bolourchi, A.; LeCun, Y.; and Fung, P. 2025. Vl-jepa: Joint embedding predictive architecture for vision-language. arXiv preprint arXiv:2512.10942. Chen, J.; Wang, K.; Chen, K.; Chen, S.; Gao, F.; Tang, W.; Li, Z.; Liu, W.; Yao, Z.; Li, B.; et al. 2026. Lawam: Latent world action models for efficient dynamics-aware robot policies. arXiv preprint arXiv:2606.15768. Desai, S.; Mattheakis, M.; Sondak, D.; Protopapas, P.; and Roberts, S. 2021. Port-Hamiltonian neural networks for learning explicit time-dependent dynamical systems. arXiv preprint arXiv:2107.08024. Du Toit, N. E.; and Burdick, J. W. 2011. Robot motion planning in dynamic, uncertain environments. IEEE Transactions on Robotics, 28(1): 101–115. Gasparino, M. V.; Sivakumar, A. N.; Liu, Y.; Velasquez, A. E.; Higuti, V. A.; Rogers, J.; Tran, H.; and Chowdhary, G. 2022. Wayfast: Navigation with predictive traversability in the field. IEEE Robotics and Automation Letters, 7(4): 10651–10658. Gong, J.; Jin, P.; Kuang, D.; Li, L.; and Tang, Y. 2026. CSympNet-ID: conformal-symplectic map learning for linearly damped Hamiltonian systems. arXiv preprint arXiv:2607.03339. Greydanus, S.; Dzamba, M.; and Yosinski, J. 2019. Hamiltonian neural networks. Advances in neural information processing systems, 32. Ha, D.; and Schmidhuber, J. 2018. Recurrent world models facilitate policy evolution. Advances in neural information processing systems, 31. Hafner, D.; Lillicrap, T.; Ba, J.; and Norouzi, M. 2019a. Dream to control: Learning behaviors by latent imagination. arXiv preprint arXiv:1912.01603. Hafner, D.; Lillicrap, T.; Fischer, I.; Villegas, R.; Ha, D.; Lee, H.; and Davidson, J. 2019b. Learning latent dynamics for planning from pixels. In International conference on machine learning, 2555–2565. PMLR.

Hafner, D.; Pasukonis, J.; Ba, J.; and Lillicrap, T. 2023. Mastering diverse domains through world models. arXiv preprint arXiv:2301.04104. Hsu, D.; Kindel, R.; Latombe, J.-C.; and Rock, S. 2002. Randomized kinodynamic motion planning with moving obstacles. The International Journal of Robotics Research, 21(3): 233–255. Huang, J.; Chen, W.; Li, Z.; Pang, O.; Hu, X.; Zhang, L.; Hu, Y.; Zhang, Z.; Coates, M.; Cao, T.; et al. 2026. H-wm: Robotic task and motion planning guided by hierarchical world model. arXiv preprint arXiv:2602.11291. Kahn, G.; Abbeel, P.; and Levine, S. 2021. Badgr: An autonomous self-supervised learning-based navigation system. IEEE Robotics and Automation Letters, 6(2): 1312–1319. Karaman, S.; and Frazzoli, E. 2011. Sampling-based algorithms for optimal motion planning. The international journal of robotics research, 30(7): 846–894. Kumar, A.; Fu, Z.; Pathak, D.; and Malik, J. 2021. Rma: Rapid motor adaptation for legged robots. arXiv preprint arXiv:2107.04034. LeCun, Y.; et al. 2022. A path towards autonomous machine intelligence version 0.9. 2, 2022-06-27. Open Review, 62(1): 1–62. Li, W.; Zhao, H.; Yu, Z.; Du, Y.; Zou, Q.; Hu, R.; and Xu, K. 2025. Pin-wm: Learning physics-informed world models for non-prehensile manipulation. arXiv preprint arXiv:2504.16693. Lions, P. 2006. Optimal control and viscosity solutions. In Recent Mathematical Methods in Dynamic Programming: Proceedings of the Conference held in Rome, Italy, March 26–28, 1984, 94–112. Springer. Liu, Y.; Chen, W.; Bai, Y.; Liang, X.; Li, G.; Gao, W.; and Lin, L. 2025. Aligning cyber space with physical world: A comprehensive survey on embodied ai. IEEE/ASME Transactions on Mechatronics. Liu, Y.; Ni, R.; and Qureshi, A. H. 2025. Physics-informed neural mapping and motion planning in unknown environments. IEEE Transactions on Robotics. Liu, Y.; Zhang, L.; Wu, D.; Zhang, Y.; Huang, A.; Wang, Z.; Liu, L.; and Guo, D. 2026. SIAM: Towards Generalizable Articulated Object Modeling via Single Robot-Object Interaction. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 40, 18478–18486. Luan, X.; and Shi, C. 2026. PH-Dreamer: A Physics-Driven World Model via Port-Hamiltonian Generative Dynamics. arXiv preprint arXiv:2605.18303. Lutter, M.; Ritter, C.; and Peters, J. 2019. Deep lagrangian networks: Using physics as model prior for deep learning. arXiv preprint arXiv:1907.04490. Maes, L.; Lidec, Q. L.; Scieur, D.; LeCun, Y.; and Balestriero, R. 2026. Leworldmodel: Stable end-to-end joint-embedding predictive architecture from pixels. arXiv preprint arXiv:2603.19312. Ni, R.; and Qureshi, A. H. 2022. Ntfields: Neural time fields for physics-informed robot motion planning. arXiv preprint arXiv:2210.00120.

Nie, W.; Liu, W.; Guo, H.; and Su, Y. 2026. Phys-JEPA: Physics-Informed Latent World Models for Multivariate Time-Series Forecasting. arXiv preprint arXiv:2606.16076. Park, J. J.; Florence, P.; Straub, J.; Newcombe, R.; and Lovegrove, S. 2019. Deepsdf: Learning continuous signed distance functions for shape representation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 165–174. Pham, Q.-C.; Caron, S.; and Nakamura, Y. 2013. Kinodynamic Planning in the Configuration Space via Admissible Velocity Propagation. In Robotics: Science and Systems, volume 32. Qureshi, A. H.; Miao, Y.; Simeonov, A.; and Yip, M. C. 2020. Motion planning networks: Bridging the gap between learning-based and classical motion planners. IEEE Transactions on Robotics, 37(1): 48–66. Raissi, M.; Perdikaris, P.; and Karniadakis, G. E. 2018. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational physics, 378(C). Ramakrishnan, S. K.; Gokaslan, A.; Wijmans, E.; Maksymets, O.; Clegg, A.; Turner, J.; Undersander, E.; Galuba, W.; Westbury, A.; Chang, A. X.; et al. 2021. Habitatmatterport 3d dataset (hm3d): 1000 large-scale 3d environments for embodied ai. arXiv preprint arXiv:2109.08238. Rao, P.; Zhang, W.; Balestriero, R.; LeCun, Y.; and Loianno, G. 2026. Skyjepa: Learning long-horizon world models for zero-shot sim-to-real control of quadrotors. arXiv preprint arXiv:2606.23444. Schölkopf, B.; Locatello, F.; Bauer, S.; Ke, N. R.; Kalchbrenner, N.; Goyal, A.; and Bengio, Y. 2021. Toward causal representation learning. Proceedings of the IEEE, 109(5): 612–634. Sethian, J. A. 1996. A fast marching level set method for monotonically advancing fronts. proceedings of the National Academy of Sciences, 93(4): 1591–1595. Shang, Y.; Zhang, X.; Tang, Y.; Jin, L.; Gao, C.; Wu, W.; and Li, Y. 2026. Roboscape: Physics-informed embodied world model. Advances in Neural Information Processing Systems, 38: 63674–63698. Shen, B.; Xia, F.; Li, C.; Martín-Martín, R.; Fan, L.; Wang, G.; Pérez-D’Arpino, C.; Buch, S.; Srivastava, S.; Tchapmi, L. P.; et al. 2020. igibson 1.0: A simulation environment for interactive tasks in large realistic scenes. arXiv preprint arXiv:2012.02924. Smith, J. D.; Azizzadenesheli, K.; and Ross, Z. E. 2020. Eikonet: Solving the eikonal equation with deep neural networks. IEEE Transactions on Geoscience and Remote Sensing, 59(12): 10685–10696. Troch, A.; Mets, K.; and Mercelis, S. 2025. ActionConditioned Hamiltonian Generative Networks (AC-HGN) for Supervised and Reinforcement Learning. In 7th Annual Learning for Dynamics & Control Conference, 04-06 June, 2025, Ann Arbor, Michigan, USA, 310–322.

Ye, S.; Ge, Y.; Zheng, K.; Gao, S.; Yu, S.; Kurian, G.; Indupuru, S.; Tan, Y. L.; Zhu, C.; Xiang, J.; et al. 2026. World action models are zero-shot policies. arXiv preprint arXiv:2602.15922. You, T.; Kim, M.; Kim, J.; and Han, B. 2023. Generative neural fields by mixtures of neural implicit functions. Advances in Neural Information Processing Systems, 36: 20352–20370. Zhang, Y.; Li, Y.; Dong, Y.; Pokorny, F. T.; and Calinon, S. 2026. Physics-Informed Eikonal Caging for Whole-Arm Manipulation Planning. arXiv preprint arXiv:2606.22143. Zhong, X.; Liu, S.; Chen, T.; and Hu, H. 2025. Multiobjective trajectory planning for flexible spacecraft via physics-informed neural network. Aerospace Science and Technology, 110710. Zhong, Y. D.; Dey, B.; and Chakraborty, A. 2019. Symplectic ode-net: Learning hamiltonian dynamics with control. arXiv preprint arXiv:1909.12077. Zhou, G.; Pan, H.; Lecun, Y.; and Pinto, L. 2025. DINOWM: World Models on Pre-trained Visual Features enable Zero-shot Planning. In International Conference on Machine Learning, 79115–79135. PMLR.

Related documents

• DUET-DINO: Simultaneous Cross-View World Modeling for Latent Planning in Robot Manipulation#997378
• Semigroup-JEPA: Latent Dynamics Consistency for Zero-Shot Physics Generalization#997343
• MoWAM: Explicit Future Motion Prediction for Efficient World Action Models#1034507
• Beyond One-Step Accuracy: State-Affine Latent Transition for Reliable Visual Planning#1118307
• Streaming-WAM: Action-Conditioned World-Action Model for Asynchronous Robot Manipulation#1076583