WorldRoamBench: An Open-World Benchmark for Long-Horizon Stability of Interactive World Models

Unknown

Conceptio › Archive › arXiv CS

arXiv CSopen access

WorldRoamBench: An Open-World Benchmark for Long-Horizon Stability of Interactive World Models

Unknown · 2026 · arxiv_cs

arXiv CS · Papers · License: Open Access · 2026

Open Source ↗Direct PDF ↓

artificialintelligence knowledgerepresentation reasoning

artificial intelligence, reasoning, knowledge representation

AMAP CV Lab, Alibaba Group

WorldRoamBench: An Open-World Benchmark for Long-Horizon Stability of Interactive World Models Ting-Bing Xu1,* Jiacheng Sui1,* Zhe Gao2,* Kewei Shi2,* Wenjin Yang1 Zhicheng Liu1,† Zhaoxu Sun1 Mingchao Sun1 Hongyu Pan1 Fan Jiang1 Mu Xu1,‡ Qi Fan2,§ Yong Li3 Baoquan Chen4 1 AMAP CV Lab, Alibaba Group

arXiv:2606.31672v1 [cs.CV] 30 Jun 2026

Homepage

Test Suite

2 Nanjing University

Dataset

3 Tsinghua University

GitHub

Leaderboard

Evaluation

WorldRoamBench Benchmarking Action-Conditioned World Models

Various Scenarios

4 Peking University

Action

Vision

Aesthetics & Imaging

Timeline

Weather

ViewPoint

Lighting

Environment

Drift Scores

Per-Frame Acc. Trajectory Acc.

Memory

Scene-Level

Subject-Level

Physics

600+ Cases Across Diverse Worlds

Optics

Mechanics

Radar Ranking

Overall Ranking

Overall

Genie-3

Eval. On 10+ IWMs

≥ 10s

3D Cons.

Physics

Action

Happy Oyster

TPV Happy Oyster

Genie-3

LingBot HY-World World 1.5

Genie-3

Happy Oyster

Lyra 2.0

LingBot World

Memory

Visual

Matrix- YUME Game 2.0 1.5

minWM

HY-World 1.5

FPV

≥ 300

HY-World LingBot Matrix- SANA1.5 World Game 3.0 WM

Figure 1. Illustration of WorldRoamBench. WorldRoamBench evaluates interactive world models across action following, visual quality, memory, and interaction physics, covering open-source and closed-source models in diverse scenarios and viewing conditions.

Abstract

VLM reasoning. The benchmark comprises 600+ test cases across Nature, Urban, and Indoor scenes in first/thirdperson views with WASD 10–60 s continuous interaction. Evaluating 10+ open/closed-source models reveals none reliably satisfies all dimensions; even the best achieves only moderate scores. Advances on WorldRoamBench are steps toward IWMs that are stable, physically grounded, memoryfaithful, and deployable in real-world applications.

Despite rapid progress in interactive world models (IWMs), existing benchmarks evaluate action following only at trajectory level and ignore memory and interaction physics. We introduce WorldRoamBench, an open-world benchmark for long-horizon stability across four dimensions, each with tailored innovations: (i) Action: per-frame action metric bypassing cross-model semantic scale disparity and exposing failures hidden by trajectory; (ii) Vision: segmentbased drift metric capturing non-monotonic mid-sequence collapse missed by start-vs-end comparisons; (iii) Physics: controllability-gated evaluation over mechanics, optics, and 3D consistency, scoring plausibility under faithful action execution; (iv) Memory: action-decoupled protocol evaluating scene memory via transition-localized 3D pointcloud reconstruction and subject memory via tracking-plus* §

1. Introduction Imagine exploring an infinite, AI-generated world by pressing WASD keys on your keyboard: moving forward through forests, turning to gaze at distant mountains, and returning to find the same landscape waiting as you left it. This vision of interactive world models [1, 9, 11] is rapidly approaching reality: recent models generate photorealistic environments at interactive frame rates, accepting discrete keyboard inputs to control camera movement.

Equal contribution; † Project Leader; ‡ Project Sponsor. Corresponding author: Qi Fan ([email protected]).

Long-Horizon Interaction Rollout

WWWW

≥ 10s

(RPE/ATE)1 . These metrics have two fundamental limitations: first, cross-model semantic scale disparity, where models produce vastly different displacement and rotation magnitudes for identical actions, makes trajectory comparison inherently unfair; second, trajectory alignment can mask per-frame failures, as a model may ignore a keystroke then over-compensate, yielding acceptable trajectory error but poor frame-level fidelity. As shown in Figure 2(a), models frequently mispredict discrete actions at switch boundaries, a failure mode invisible to trajectory-level evaluation. 2) Long-horizon visual stability. Existing benchmarks measure average visual quality (Imaging and Aesthetic scores) but rarely assess temporal degradation. In longhorizon rollout, autoregressive error accumulation causes progressive quality collapse: color drift, blurring, or structural breakdown. Figure 2(b) shows that a model may follow actions correctly yet produce visually collapsed frames after hundreds of generation steps. 3) Long-horizon interaction physics. Existing benchmarks [14–17, 36] lack systematic evaluation of interaction physics, such as clipping, collision response, gravity, optical consistency, and rigid-body 3D coherence. Physics has been studied for passive video generation [23, 24] but not effectively adapted to the interactive setting. Figure 2(c) shows a model following the intended direction yet violating physical constraints (e.g., passing through obstacles). 4) Long-horizon memory. MIND [14] and iWorld-Bench [16] evaluate memory through symmetric revisit tests, comparing frames at index t and T −t. However, this assumes perfectly executed actions, which breaks in practice: cumulative drift makes the “return” frame spatially offset. Moreover, Figure 2(d) shows that even when actions prescribe an ideal symmetric return path, models regenerate a different scene upon revisit, revealing memory limitations. To fill these gaps, we introduce WorldRoamBench— Responsive Open-world Action Model Benchmark— which focuses on long-horizon interactive stability (Fig. 1). Unlike prior work limited to brief ∼5–10s clips on game or indoor scenes, WorldRoamBench evaluates models as they roam through generated open-domain worlds for ∼10–60s of continuous WASD/IJKL interaction, covering all four dimensions: action following, visual quality, interaction physics, and memory. Contributions are as follows: 1. Action. We propose a keystroke-level per-frame action metric that bypasses cross-model semantic scale disparity and reveals per-step failures hidden by trajectory. 2. Vision. We introduce a segment-based temporal drift metric over imaging and aesthetic scores that captures non-monotonic mid-sequence collapse missed by perframe or start-vs-end measures. 3. Physics. We design a controllability-gated physics evaluation over mechanics (collision, clipping, deformation,

≥ 300

AAAA … SS

(a) Lyra 2.0: × action vs. ✓ TrajScore: 0.96

SANA-WM: × action vs. ✓ TrajScore: 0.86

(b) Matrix-Game 2.0: ✓ action vs. × vision

HY-World 1.5: ✓ action vs. × vision

(d) HY-World 1.5: ✓ action vs. × memory (500f W + 500f S)

Figure 2. Failures revealed by long-horizon interaction. Extended rollouts expose failures often missed by short clips: high trajectory scores hide per-step action mismatches, visual quality degrades, physical constraints are violated, and revisited scenes are regenerated inconsistently.

Yet as these models proliferate, a critical question remains unanswered: How well do they respond to user inputs over extended interactions? Existing benchmarks (Table 1), such as MIND [14], WorldMark [15], iWorld-Bench [16], WildWorld [17], WBench [36], and WorldOlympiad [50], have advanced evaluation methodology. However, most evaluate only short clips of ∼5–10s [14, 16, 17, 36], leaving long-horizon exploration largely unexamined. Moreover, most lack systematic evaluation of interaction physics, such as mechanics, optics, and 3D consistency [14–17, 36], and only a few assess memory consistency during revisit, often under the assumption of perfect action execution [14, 16]. We argue that the true challenge for interactive world models lies in long-horizon stability: maintaining faithful, physically plausible, visually coherent, and memoryconsistent generation as users continuously roam the generated world. Four critical aspects emerge where current models are most fragile under extended interaction (Figure 2): 1) Long-horizon action following. Current benchmarks [14–16, 36] recover the 3D camera trajectory from generated videos and compute trajectory-level errors

1 RPE: Relative Pose Error; ATE: Absolute Trajectory Error.

Table 1. Comparison with existing interactive world model benchmarks. WorldRoamBench is the first to provide per-frame action metric, visual drift, interaction physics, and trajectory-aware scene/subject memory. WorldMark also evaluates closed-source models but lacks physics, visual drift, and per-frame metrics. † WBench and iWorld-Bench report 10+ models because they include general video generation models beyond strictly interactive world models. ‡ WBench, iWorld-Bench, and WorldOlympiad include a small number of third-person cases but do not provide a dedicated third-person view (TPV) leaderboard. Eval. Metrics Benchmark

Action Eval. Granularity

Per-frame Visual Interaction Action

Drift

Physics

Eval. Models Revisit

Traj.-Aware

Memory

Time

Leaderboards

Categories

Open- Closed- First- Third- Game

Real

Scale/Video Source Source Person Person World World

Segment

∼5–10s

✓

iWorld-Bench [16]†‡

Traj.

✓

∼5–10s

10+

✓

WorldOlympiad [50]‡

Multi-chunk

✓

∼10s

✓

Traj.

✓

∼5–10s

✓

WBench [36]†‡

Multi-turn

✓

∼5–10s

10+

✓

WorldMark [15]

Traj.

20/40/60s

✓

10–60s

✓

WildWorld [17]

MIND [14]

WorldRoamBench (Ours) Traj. + per-frame

terrain following, gravity), optics (reflection, shadow occlusion), and 3D consistency, scoring plausibility only under faithful action execution. 4. Memory. We present an action-decoupled dual-track protocol that evaluates scene memory via transitionlocalized 3D point-cloud reconstruction and subject memory via tracking-plus-Qwen3-VL-PLUS reasoning. 5. Model Evaluation. We build the most comprehensive leaderboard to date, covering 10+ state-of-the-art interactive world models, spanning open-source (MatrixGame 2.0/3.0, HY-World 1.5, Yume 1.5, LingBot-World, Lyra 2.0, minWM, SANA-WM) and closed-source (Genie 3, Happy Oyster). Throughout the evaluation, results expose several common weaknesses across models: (i) trajectory score ̸= perframe correctness—models with trajectory scores above 85 can have below 65% per-frame strict action accuracy, following the right path yet missing keystrokes; (ii) high visual quality ̸= good action following—models with top imaging or aesthetic scores may fail at frame-level action following, indicating that visual quality and action fidelity are largely independent; (iii) stricter physics adherence may compromise action following—models that avoid clipping and collisions deviate from predefined trajectories; and (iv) memory evaluation is confounded by action imprecision— models rarely hit the turning point exactly, conflating action error with memory degradation; our trajectory-aware localization and 3D point-cloud reconstruction decouple them.

image prompts without real-time user control, interactive world models generate visual environments conditioned on user actions in real time. Genie [1] pioneered this paradigm by learning latent action spaces from unlabeled video. Recent models—including Matrix-Game 2.0/3.0 [9, 10], HY-World 1.5 [11], Yume 1.5 [12], LingBot-World [13], SANA-WM [27], Lyra 2.0 [28], and minWM [29]—accept discrete WASD/IJKL keyboard inputs and generate frames autoregressively at interactive rates. Closed-source models such as Genie 3 [2] (Google DeepMind) and Happy Oyster [3] (Alibaba) further push visual quality and temporal coherence. Despite architectural differences, these models share a common evaluation interface: an initial image and a sequence of discrete actions {a1 , a2 , . . . , aT }, where at ∈ {W, A, S, D, I, J, K, L, idle}, from which they autoregressively generate frames {f1 , f2 , . . . , fT }.

2.2. Benchmarks for World Models Evaluation benchmarks for world models span several paradigms. Video generation benchmarks (VBench [18– 20], WorldScore [21], WorldModelBench [22]) assess visual quality, text alignment, or physical commonsense without closed-loop interactive control. Domain-specific benchmarks target narrow scenarios: ACT-Bench [48] and WorldLens [46] for autonomous driving; EWMBench [49] and WorldArena [47] for embodied robotic manipulation. Physics and interaction fidelity benchmarks (PhyGenBench [23], WorldBench [24], Omni-WorldBench [43]) evaluate physical plausibility or causal interaction correctness on prompt-conditioned generation without discretekeystroke control. Interactive world model benchmarks are most relevant to our setting. MIND [14] introduced memory via closed-loop revisit tests; WorldMark [15] unified WASD actions with three difficulty tiers; iWorldBench [16] proposed a control quadruple with six lev-

2. Related Work 2.1. Interactive Video World Models Unlike passive video generation models such as Wan 2.7 [4], Kling 3.0 [5], Sora 2 [6], Veo 3.1 [7], and Seedance 2.0 [8], which synthesize videos from text or 3

Per-Frame Accuracy

Static Metrics

✓ Pose Estimation

Action Estimation

Per-Frame Scoring W A S D

Trajectory-Level Accuracy Pose Estimation W A S D

Pseudo Labeling

Scene Memory Turn-Point Segment

Frame Sampling

…

Frame Scores

Judge

Imaging

(MUSIQ & A-CLIP)

Aesthetics

Drift Metrics

Trajectory-Level Scoring

Punish High Drift

W+A 0

Observation

…

Optics

Drift Scores

Mechanics

…

Revisiting

VLM Judge

3D Recon

Poses

Subject Memory SAM2 Tracking

…

Notes

3D Consistency

Point Cloud Compare

Rendering GT Images

VLM Judge

GT Bank

…

Predicted Videos

Eval.

W A S D

GT Actions

Figure 3. Overview of the WorldRoamBench evaluation pipeline. Given videos generated from shared initial frames and WASD/IJKL action programs, WorldRoamBench evaluates four complementary dimensions: action following through pose-aligned frame scoring, visual quality through frame-level and drift-aware metrics, interaction physics through mechanics, optics, and 3D-consistency tests, and memory through turning-point-aware scene reconstruction and third-person subject memory.

els; WBench [36] unified multi-modal inputs for multi-turn evaluation; WorldOlympiad [50] jointly assessed physics, geometry, and interaction across chunk-by-chunk rollouts; and WildWorld [17] evaluated gamepad-based action following in a single game. However, as shown in Table 1, existing interactive benchmarks leave several aspects underexplored: action evaluation remains largely trajectory-level despite crossmodel semantic scale disparity, long-horizon interaction physics is rarely measured, memory protocols often assume perfect ground-truth (GT) frame alignment, and most evaluations are limited to ∼5–10s clips. WorldRoamBench addresses these limitations through open-world, long-horizon (10–60s) evaluation of keystroke-level action following, visual drift, interaction physics, and trajectory-aware memory.

tional verification, making keystroke-level evaluation applicable to open-domain interactive world models.

2.4. Physics Evaluation Methodologies Physics evaluation has been explored in both general video generation and world-model benchmarks. General video benchmarks such as VBench [18], VBench2.0 [20], and OmniBench [42] include physical plausibility among broader visual and temporal quality dimensions. Physics-focused benchmarks further define explicit physical reasoning tasks: PhyGenBench [23] covers diverse physical laws, WorldModelBench [22] evaluates violations such as gravity, penetration, fluid dynamics, and conservation, and WM-ABench [44] studies mechanical interactions through controlled simulation. Recent worldmodel benchmarks also touch physics from different angles: WorldScore [21] includes 3D and photometric consistency, WorldBench [24] evaluates physical plausibility in generated worlds, WorldLens [46] studies closed-loop degradation, and Omni-WorldBench [43] introduces multilevel causal interaction scenarios. However, existing physics tests are still mostly designed for passive observations or coarse semantic judgments. They rarely ask whether a user’s action should trigger a specific physical consequence, and they do not provide casespecific metrics for contact, occlusion, reflection, shadows, or traversability. WorldRoamBench complements prior benchmarks by evaluating fine-grained interaction physics in 3D-grounded scenes under explicit WASD/IJKL control.

2.3. Action Evaluation Methodologies Action evaluation in world models generally follows three paradigms. IDM-based approaches (VPT [25], MineWorld [26]) train an Inverse Dynamics Model to classify per-frame actions from video, achieving high accuracy in Minecraft but limited to that domain. Trajectorybased approaches (MIND [14], WorldMark [15], iWorldBench [16]) recover 3D camera poses via SLAM or ViPE and compute geometric errors. VLM-based approaches (WildWorld [17]) use vision-language models to judge action consistency at the segment level. WorldRoamBench extends trajectory-based assessment with per-frame direc4

2.5. Memory Evaluation Methodologies GT Actions 

Generated Video

Existing memory evaluation protocols commonly rely on frame-pair similarity. MIND [14], for example, constructs symmetric action sequences and compares temporally symmetric frames using pixel-level metrics such as LPIPS. This formulation implicitly assumes perfect action execution and uniform action magnitude, such that symmetric temporal indices correspond to the same spatial location. However, interactive models may drift, under- or over-execute commands, and accumulate timing offsets, causing frame-pair metrics to conflate control error with memory degradation. WBench [36] partially addresses temporal misalignment by estimating poses and selecting the revisit frame closest to the initial frame, but the resulting metric remains framelevel and first-frame-centric, rather than assessing preservation of the broader visited scene. A related line of work evaluates short-term geometric consistency rather than memory. WBench’s spatial consistency, WorldMark’s geometry consistency [15], and WorldOlympiad’s geometric evaluation [50] measure whether nearby generated views remain 3D-coherent. While such metrics characterize local spatial consistency, they do not evaluate whether a model retains previously generated or observed content after a long interactive loop. WorldRoamBench therefore formulates memory as revisit evaluation under imperfect control, combining action-aware transition localization with scene-level 3D reconstruction to assess preservation beyond a single frame pair.

ViPE Pose Estimation Camera Trajectory ̂

Per-frame Accuracy

Trajectory Score

Latent-Stride Discretize (K=4, thresh. sweep)

Switch Detection → Segments

Predicted Actions ̂

Adaptive GT Construction

Compare ̂ vs 

Arc-length Resample (N=60 pts)

Accstrict Accpartial

TrajScore nATEt nATEr

Input

Processing

Output Metric

Figure 4. Action following evaluation pipeline. A shared ViPE trajectory estimation stage feeds two branches: (left) per-frame action accuracy via latent-stride discretization, and (right) TrajScore via adaptive GT construction and arc-length resampling.

an atomic action (e.g., forward) or a compound action. Each action is decomposed into a set of atomic sub-actions P(gt ) = {p1 , p2 , . . .}. The atomic vocabulary comprises primitives: forward (W), backward (S), left (A), right (D), look left (J, yaw−), look right (L, yaw+), look up (I, pitch+), and look down (K, pitch−).

3. WorldRoamBench Benchmark Design 3.1. Overview WorldRoamBench evaluates interactive world models through four complementary questions (Fig. 3): (1) Action Following: does the model respond correctly to each keystroke? (2) Visual Quality: does the generated world remain perceptually faithful and temporally stable? (3) Interaction Physics: does the world obey basic physical laws during interaction? (4) Memory: does the model preserve what it previously generated?

Trajectory estimation and action discretization. Given the generated video {I1 , . . . , IT }, the camera trajectory T̂ = (T̂0 , . . . , T̂T ) is estimated using ViPE [34], where each T̂t ∈ SE(3) is a camera-to-world pose. The estimated trajectory is then discretized into per-frame predicted actions Ĝ = (ĝ1 , . . . , ĝT ). This discretization builds on the CompassReward framework from WorldCompass [33] and adds a latent-stride mechanism. Because current interactive world models generate video in chunks of K=4 frames per autoregressive step, relative poses are computed over K-frame windows instead of individual frames, and all thresholds are scaled by K. This choice aligns the discretization granularity with the model’s native generation cadence and improves robustness to pose-estimation noise (see Appendix C for details). Specifically, frames are grouped into latent windows of size K=4, and the relative transformation is computed over each window:

3.2. Evaluation Dimension 1: Action Following Action following is evaluated at two complementary granularities: per-frame discrete action accuracy, which checks whether each individual keystroke produces the expected visual response, and trajectory-level scoring, which measures the geometric fidelity of the overall camera path. Figure 4 illustrates the shared trajectory-estimation backbone and the two evaluation branches. 3.2.1. Per-frame Discrete Action Accuracy Interactive world models receive a sequence of discrete actions G = (g1 , . . . , gT ), where each gt is

−1 ρc = T̂cK · T̂min(cK+K, T ) ,

c = 0, . . . , C−1.

(1)

Each ρc is discretized into a predicted action âc by comparing its translation and rotation components against direction-dependent thresholds. Given translation ∆t = ρc [:3, 3] with magnitude ν = ∥∆t∥, the discretization applies two checks. Translation thresholding. If ν > τmove , the angle θj = arccos(∆tj /ν) is computed with respect to each axis j. A directional component is activated only if both θj satisfies an angular criterion (< 60◦ or > 120◦ ) and the per-axis magnitude exceeds a noise floor τaxis , preventing spurious activations from minor lateral drift. Rotation thresholding. Rotation is extracted as Euler angles (ϕ, ψ) and thresholded against τrot . All thresholds are scaled by the stride K to maintain consistent sensitivity across window sizes. The operating point is selected by sweeping over translation thresholds Θ = {0.002, 0.005, 0.01} and retaining the threshold with the highest exact-match accuracy against the ground-truth action sequence. The âc is broadcast to all K frames in the window, yielding the per-frame predicted sequence Ĝ.

metrics. We additionally report move accuracy (whether translation sub-actions match) and look accuracy (whether rotation sub-actions match) for fine-grained analysis; these auxiliary metrics are computed in our codebase but omitted from the main results tables for brevity. The full discretization algorithm and threshold sweep procedure are detailed in Appendix C (Algorithm 13 and Algorithm 14). 3.2.2. Trajectory-level Metrics: TrajScore Per-frame accuracy captures discrete correctness but does not assess the geometric quality of the generated camera path—for example, whether a “forward” trajectory is straight or whether a “look left” rotation traces a smooth arc. Inspired by WBench [36], TrajScore complements per-frame metrics with a normalized trajectory-accuracy measure based on adaptive ground-truth construction and arc-length alignment. Switch detection and segmentation. Given the groundtruth action sequence G, we detect switches, i.e., boundaries where the action changes, and partition the sequence into Nseg contiguous segments {(si , ei , ai )}i=1 , where each segment [si , ei ) executes a single action ai . The number of switches determines the trajectory difficulty: 0 switches (single action, e.g., sustained forward), 1 switch (two actions), or 2+ switches (multi-segment paths).

Strict Accuracy (Exact Match). Strict accuracy requires the predicted action to be exactly equal to the ground-truth action, including all sub-action components:2 T

Accstrict =

1X [[ĝt = gt ]]. T t=1

(2)

Adaptive ground-truth construction. A key design choice in WorldRoamBench is an adaptive ground-truth trajectory: the model’s own displacement magnitude is retained, while the ideal direction and shape prescribed by the action are imposed. This decouples “how far the model moved” (amplitude, which is not penalized) from “whether it moved in the right direction and shape” (geometry, which is measured). Translation (e.g., forward): the GT is a straight line in the action-prescribed direction, with total length equal to the predicted segment’s displacement. Rotation (e.g., look left): the GT depends on the camera perspective. In first-person view (FPV), the camera rotates in place; in third-person view (TPV), the camera orbits around the subject along an ideal arc whose radius is estimated from the predicted trajectory.

For compound actions such as forward+look right, this demands that every sub-action is correctly predicted— a stringent criterion that penalizes even partial compliance. Partial Accuracy. Many interactive actions are compound, so a model that correctly executes the translation component but misses the rotation (or vice versa) should not be scored the same as one that fails entirely. Partial accuracy is defined as the fraction of frames where the predicted and ground-truth actions share at least one common sub-action: T

1X Accpartial = [[P(ĝt ) ∩ P(gt ) ̸= ∅]], T t=1

(3)

where P(·) decomposes an action into its atomic sub-action set. For example, if gt = forward+look right and ĝt = forward, then P(gt ) ∩ P(ĝt ) = {forward} ̸= ∅, yielding a partial match. The special case gt = ĝt = no op (both empty sets) is also counted as a match.

Arc-length resampling. Direct per-frame comparison between predicted and GT trajectories is confounded by nonuniform speed: a model may move quickly at first and slow down later, or vice versa. To evaluate geometric shape alignment, both trajectories are resampled to equidistant points by arc length. Positions are linearly interpolated along cumulative path length, and rotations are interpolated via Slerp. A fixed total of Npts = 60 resampled points is maintained across all segments and divided equally, so each segment receives Npts /Nseg points and comparable resolution is preserved regardless of the number of switches.

Discussion. The gap between strict and partial accuracy is itself diagnostic: a large gap indicates that the model often captures the dominant motion direction but fails on secondary axes, a failure mode invisible to trajectory-level 2 [[P ]] denotes the Iverson bracket: it equals 1 if predicate P is true and 0 otherwise.

Normalized Absolute Trajectory Error. After resampling, the mean position error (ATEt ) and mean rotation error (ATEr ) are computed across all resampled point pairs. To make the metric comparable across trajectories of different lengths, the errors are normalized: ATEt , 0, 1 , (4) nATEt = clamp max(Lpath , ℓmin ) ATEr , 0, 1 , (5) nATEr = clamp max(Φtotal , ϕmin )

on human aesthetic ratings from the LAION-Aesthetics dataset, produces a per-frame score saes = g(zt )/10. The t video-level aesthetic quality is the mean over all frames: T

Saes =

1 X img s . Simg = T t=1 t

TrajScore. The final trajectory score combines both error components into a single scalar in [0, 1]: nATEt + nATEr . 2

(6)

3.2.3. Action Score The action-following metrics are aggregated into a dimension-level score by averaging strict per-frame accuracy, partial per-frame accuracy, and trajectory alignment: Accstrict + Accpartial + TrajScore . 3

(9)

Temporal Drift Metrics. A defining failure mode of autoregressive world models is progressive quality degradation: early frames may be crisp and aesthetically pleasing, whereas later frames suffer from accumulated artifacts, color shifts, or loss of detail. In practice, degradation is not always monotonic—we frequently observe models that collapse mid-rollout and then partially recover, so simply contrasting the start and end of a clip can dramatically underestimate the true visual instability. We therefore quantify drift over the entire rollout via a segment-based formulation inspired by Helios [32] but adapted to capture the largest excursion in quality. For a per-frame score sequence {st }Tt=1 (either aesthetic or imaging), we split the timeline into N contiguous segments of equal length and compute the mean score of each segment: 1 X st , i = 1, . . . , N, (10) s̄i = |Ti |

A score of 1 indicates perfect geometric alignment, whereas a score near 0 indicates that the generated trajectory deviates substantially from the prescribed path. Separately reporting nATEt (↓) and nATEr (↓) enables fine-grained diagnosis of whether a model’s trajectory errors are primarily translational (wrong direction) or rotational (incorrect heading). A complete step-by-step computation example is provided in Appendix D.

Saction =

(8)

Imaging Quality. Low-level perceptual quality (sharpness, noise, compression artifacts) is assessed using MUSIQ [31], a multi-scale image quality transformer that operates without a reference image. MUSIQ processes each frame at multiple resolutions via a multi-scale representation and outputs a technical quality score simg = t MUSIQ(It )/100. The video-level imaging quality is:

where Lpath is the total path length of the predicted trajectory, Φtotal is the total accumulated rotation (in degrees), and ℓmin =0.5, ϕmin =10◦ are minimum normalization denominators that prevent instability for near-stationary trajectories.

TrajScore = 1 −

1 X aes s . T t=1 t

t∈Ti

(7)

where Ti is the set of frame indices in segment i. We then take the highest and lowest segment means across the N segments, s̄best = max s̄i and s̄worst = min s̄i , as the

The normalized trajectory errors nATEt and nATEr are reported as diagnostic metrics but are not included in the dimension-level action score.

1≤i≤N

best- and worst-quality windows along the rollout. The drift score is the relative drop from the best to the worst segment:

3.3. Evaluation Dimension 2: Visual Quality D=

Autoregressive world models must maintain visual quality not only on average but also over time, as quality often degrades when generation errors accumulate. We therefore evaluate visual quality along two axes—absolute quality and temporal drift—using four complementary metrics.

s̄best − s̄worst . |s̄best |

(11)

We use N = 10 throughout, so D = 0.05 means the worst 10% window of the video is 5% below the best 10% window in average score; unlike a fixed start/end comparison, this localizes the strongest dip wherever it occurs, so transient mid-rollout collapses that are later masked by recovery still contribute to D. We instantiate this formulation on the aesthetic and imaging score sequences to obtain two lower-is-better metrics, DriftAesthetic (Daes ↓) and DriftImaging (Dimg ↓).

Aesthetic Quality. The aesthetic appeal of generated frames is measured using a CLIP-based linear aesthetic predictor [30]. Each frame It is encoded by a frozen CLIP ViT-L/14 backbone into a normalized feature vector zt = norm(CLIP(It )). A linear head g : R768 → R, trained 7

Discussion. Combining absolute metrics (Saes , Simg ) with drift metrics (Daes , Dimg ) gives a more complete picture: a model can score high on average by excelling early and degrading later (visible only through drift), while a moderate-quality but stable model may offer a better interactive experience than one that starts strong and collapses. Using relative percentage change rather than absolute difference also keeps drift comparable across models with different score magnitudes.

collision, which checks whether contacted objects respond appropriately; clipping, which detects non-physical passthrough across solid obstacles; deformation, which measures whether deformable surfaces leave traces or react in real time; terrain following, which checks whether the subject maintains plausible ground contact over slopes, stairs, or uneven terrain; and gravity, which evaluates falling objects, sinking behavior, and projectile motion. Unless otherwise specified, each mechanics protocol returns a binary pass/fail score. Optics evaluates whether light-dependent effects remain consistent throughout interaction. We consider two protocols. Reflection is scored with a coarse-to-fine procedure: the coarse stage checks global reflection plausibility, while the fine stage assigns per-frame reflection scores over valid frames and normalizes them to [0, 1]. Occlusion, implemented as shadow correctness, checks whether expected shadows are present and whether their direction and shape are consistent with the scene lighting. 3D consistency measures whether the scene preserves rigid spatial structure under controlled camera motion. These cases are primarily instantiated in FPV settings with long, unobstructed trajectories, where depth-ordering errors, geometric distortions, and non-rigid scene deformation can be detected without confounding collision or terrain effects. When available, 3D-consistency cases are treated as a separate physics domain. Appendix E.2 provides the full frame sampling strategies, prompt design, fallback queries, and protocol-specific details.

3.3.1. Visual Score The two lower-is-better drift metrics are converted into stability scores and averaged with the two absolute quality metrics: Svisual =

Saes + (1 − Daes ) + Simg + (1 − Dimg ) . (12) 4

This score rewards both high perceptual quality and temporal stability during long-horizon autoregressive rollout.

3.4. Evaluation Dimension 3: Interaction Physics WorldRoamBench evaluates interaction physics—i.e., the physical consequences of user-driven interaction—as physical plausibility under controlled generation. A model should not only follow the prescribed action, but also produce physically consistent consequences as the user interacts with the environment. We decompose physics into three complementary domains: mechanics, which measures whether objects, terrain, and the subject obey basic interaction constraints; optics, which measures whether light-dependent phenomena such as reflections and shadows remain plausible; and 3D consistency, which measures whether the generated scene preserves rigid spatial structure during controlled camera motion. All VLM-based judgments in the physics suite use Qwen3-VL-PLUS.

3.4.3. Physics Score Each valid case receives a protocol-level score sc ∈ [0, 1]. Let Cmech , Copt , and C3d denote the valid cases for mechanics, optics, and 3D consistency, respectively. The domain scores are X X 1 1 sc , sc , Sopt = Smech = |Cmech | |Copt |

3.4.1. Validity Gate

c∈Cmech

Before evaluating physical correctness, we first verify that the generated video responds to the prescribed action. This validity gate prevents static or uncontrolled generations from being counted as physical failures. For FPV rollouts, we estimate the dominant camera motion from optical flow and compare it with the dominant input action. For TPV rollouts, where camera motion and subject motion are harder to disentangle, we use a subject-centric check based on SAM masks and bounding-box consistency, with a lightweight Qwen3-VL-PLUS VLM fallback when masks are unavailable. Invalid cases receive a score of −1 and are excluded from aggregate physics statistics. Appendix E.1 provides the full FPV and TPV validity protocols.

c∈Copt

(13) with S3d defined analogously when 3D-consistency cases are available. The final physics score averages all available physics domains: 1 X Sd , D ⊆ {mech, opt, 3d}. (14) Sphysics = |D| d∈D

Equal weighting is used because the three domains capture complementary aspects of physical plausibility: mechanics tests interaction-level consequences, optics tests light-level consistency, and 3D consistency tests spatial coherence.

3.5. Evaluation Dimension 4: Memory

3.4.2. Physics Protocols

WorldRoamBench evaluates memory as revisit stability under imperfect control. A model should preserve the previously observed world when the user returns to a region,

Mechanics evaluates whether visible interactions produce physically plausible outcomes. It includes five protocols: 8

tained if it has a nearby revisit point within threshold τd , while a revisit point is hallucinated if it is unsupported by any observation point. We define

Memory Preserve previously observed world after revisit

Scene Memory

Subject Memory

Geometry preservation

Identity / structure / appearance consistency

Rmem =

|{p ∈ Pobs : d(p, Prev ) < τd }| , |Pobs |

(15)

|{q ∈ Prev : d(q, Pobs ) ≥ τd }| . (16) |Prev | We interpret Rmem as memory recall and Pmem = 1−Rhall as memory precision. The scene-memory score is Rhall =

Video Segments

Observation

Frame-Level Subject Tracking

Revisit

Points Clouds Alignment

2 Pmem Rmem , (17) Pmem + Rmem an F1 score that balances retained observed geometry and unsupported revisit geometry. Appendix F.1 gives the full transition-localization, point-cloud filtering, registration, and scoring details; Appendix H analyzes why framepair alternatives remain unreliable. Sscene =

VLM-Based Judgement Rating the subject quality between 1-10

Subject Scores

Scene Scores 2𝑃%#% 𝑅%#% 𝑆!"#$# = 𝑃%#% + 𝑅%#%

𝑆!"# = 𝑆$%& − 1 /9 𝑆!"# ∈ 1, … , 10

3.5.2. Subject Memory

Final Memory Score FPV: 𝑆'(')*+ = 𝑆!,(-(

TPV: 𝑆'(')*+ =

Subject memory captures a complementary failure mode in TPV rollouts: the protagonist may preserve the scene trajectory while drifting in identity, body structure, or appearance. Because the subject is a small, articulated region and cannot be reliably evaluated by scene-level point clouds, we use a tracking-and-VLM protocol instantiated with Qwen3-VL-PLUS. We first filter out cases where the model fails to maintain a controllable TPV subject; for the remaining cases, we track the protagonist, extract temporally ordered subject crops, and ask the Qwen3-VL-PLUS to judge identity, structure, and appearance preservation holistically. It produces an integer subject memory score SVLM ∈ {1, . . . , 10}, which we normalize as

𝑆!,(-( + 𝑆!"# 2

Figure 5. Memory evaluation pipeline. WorldRoamBench evaluates memory with two trajectory-aware tracks. Scene memory localizes the executed observation–revisit transition and compares reconstructed scene geometry across the two segments. Subject memory tracks the third-person protagonist and evaluates identity, structure, and appearance preservation with a holistic Qwen3-VLPLUS judgment. The full pipeline is provided in Appendix F.

even if the generated trajectory does not exactly match the ground-truth action path. As summarized in Figure 5, we therefore separate memory into two complementary tracks. Scene memory measures whether observed scene geometry is retained upon revisit, while subject memory measures whether the protagonist in a third-person video preserves identity, structure, and appearance. This separation is important because scene memory is primarily geometric, whereas subject memory depends on fine-grained visual cues on a small moving region.

SVLM − 1 . (18) 9 This holistic score avoids over-penalizing benign viewpoint or lighting changes while still detecting identity changes, structural collapse, disappearance, and appearance drift. Appendix F.2 describes the controllability gate, tracking procedure, failure taxonomy, normalization, and prompt. Ssub =

3.5.3. Memory Score The final memory score depends on the evaluated perspective. For FPV, there is no visible protagonist, so memory is the scene-memory score:

3.5.1. Scene Memory Scene memory asks whether the geometry observed before a revisit remains available after the model returns. Directly matching symmetric frame pairs is unreliable because current models under- or over-execute actions, accumulate timing offsets, and may not return to the same viewpoint. We therefore use a scene-level protocol rather than a frame-pair protocol. The rollout is split at the executed observation– revisit transition, both segments are reconstructed into point clouds, and the two clouds are aligned before scoring. Let Pobs and Prev denote the aligned observation and revisit point clouds. An observation point is considered re-

FPV Smemory = Sscene .

(19)

For TPV, memory combines scene preservation and protagonist preservation: 1 TPV Smemory = (Sscene + Ssub ) . (20) 2 This formulation keeps the geometric revisit criterion used in FPV while additionally requiring TPV models to maintain a stable protagonist. 9

3.6. Scoring and Leaderboard

with both simple and multi-step action sequences to evaluate memory across different revisit difficulties. For the action dimension, spacious, unobstructed first-frame images (e.g., open fields, wide corridors, empty plazas) are prioritized, allowing translational and rotational movements to be executed without obstacle-induced confounds. This design isolates action following from collision or navigation effects and yields a cleaner measurement of per-keystroke accuracy. For the physics dimension, scenes contain specific physical affordances: environments with walls, pillars, or furniture for collision testing; elevated platforms or staircases for gravity evaluation; and long unobstructed corridors for 3D consistency measurement.

The overall WorldRoamBench score is a weighted composite: SROAM = w1 · Saction + w2 · Svisual (21) + w3 · Sphysics + w4 · Smemory . Results are reported with equal default weights w1 =w2 =w3 =w4 =0.25, together with per-dimension rankings. For TPV, the leaderboard overall score additionally accounts for whether a model can maintain controllable third-person generation: TPV SROAM = Rctrl · SROAM ,

(22)

where Rctrl is the third-person control rate. We report Rctrl separately so that the dimension scores can be interpreted alongside the controllability gate.

4.2. Action Suite Dimension-specific action design. Action sequences are designed with increasing complexity and tailored to each evaluation dimension. For the action dimension, each segment sustains a single key for k frames (e.g., hold W for 30 frames). Difficulty is determined by the number of switches between segments: Easy (0 switches, single sustained action), Medium (1 switch, two actions), and Hard (2+ switches, multi-segment paths). All action paths are routed through open space, avoiding walls, ledges, and other obstacles, so that action-following accuracy is measured in isolation without confounding physics-related behaviors such as collision deceleration. For the memory dimension, sequences follow symmetric traversal patterns such as A→B→ inv(B)→ inv(A), where inv(·) denotes the inverse action (e.g., forward–backward, left–right loops), to enable revisit evaluation. As with the action dimension, paths are intentionally routed through open space to minimize collisions or other physics-driven phenomena, ensuring that discrepancies between observation and revisit segments reflect genuine memory degradation rather than physics-induced trajectory deviation. For the physics dimension, sequences are crafted to trigger physical interactions: paths are directed toward walls (collision), along edges of elevated surfaces (gravity), and through long corridors at a constant input rate (3D consistency). Each sequence isolates a single physical phenomenon for unambiguous attribution.

4. Test Suite The WorldRoamBench test suite provides first-frame images and keystroke action sequences for all four evaluation dimensions. We use a unified construction methodology while tailoring specific cases to each dimension’s requirements. The image suite and action suite are described separately to clarify both shared design principles and dimension-specific choices.

4.1. Image Suite Unified construction pipeline. First-frame images spanning both first-person and third-person perspectives are collected from two complementary sources: (1) the open Internet, providing diverse real-world scenes, and (2) existing world model benchmarks such as WorldScore [21], providing standardized evaluation imagery. These raw images are then refined with GPT-Image-2 through targeted edits: improving resolution and clarity, adjusting object placement and layout, converting between first-person and thirdperson perspectives, and captioning followed by regeneration to produce visually similar but distinct variants. This refinement pipeline serves two purposes: it ensures high image quality and broad scene diversity, while reducing the likelihood that the resulting first frames overlap with images seen during world model training, thereby mitigating potential data contamination that could inflate benchmark scores.

Avoiding confounding physics in non-physics dimensions. A key cross-cutting design principle is that action sequences in the action and memory suites should avoid physical phenomena (e.g., collisions, gravitational effects) that could confound the target evaluation. For instance, if a forward-walking sequence causes the model to collide with a wall, the resulting deceleration would be incorrectly penalized as an action-following error. Combining spacious first-frame environments (Sec. 4.1) with obstacle-free paths ensures that action and memory metrics reflect their intended capabilities without contamination from physics-

Dimension-specific image selection. Although all dimensions draw from the same image library, the selection criteria differ according to each dimension’s evaluation goal. For the memory dimension, images span a spectrum of scene complexity, from simple scenes with few distinctive landmarks (e.g., a single building on an open plain) to complex scenes with rich geometric detail (e.g., a cluttered indoor room or a dense urban street). This range is paired 10

L 28.3%

Memory

K 1.0%

n = 77,404

J 28.3%

J 11.6% Action D 2.9% n = 75,852

A 0.3% S 20.4%

S 20.0%

D I0.3% 1.0%

Memory

D S 6.0% 9.6%

L 12.9%

W 20.4%

A 7.5%

W 50.1%

n = 58,050

Medium 63 (30%)

W 76.8%

A 2.6%

Terrain Following

Occlusion

3D Cons.

Gravity

Reflection

Hard 73 (34%) Easy 132 (62%)

Easy 71 (33%)

Medium 73 (34%)

Clipping 37

(a) Case Counts 96

Action

118

Memory

171

100

217

113

212

150

Number of Test Cases

200

400

Number of Cases

Physics

FPV TPV

Mechanics

Collision

Deformation

Optics

3D Consistency

Figure 6. Action key distribution across the three evaluation dimensions. Each pie chart shows the proportion and count of perframe key presses aggregated over all test cases in that dimension.

Hard 17 (8%)

Physics

Action

Figure 8. Difficulty / category distribution. Memory and Action are grouped by difficulty; Physics is grouped by category, with dashed lines separating Mechanics, Optics, and 3D Consistency.

(b) Action Sequence Length 349

300

3–4, and >4 segments, respectively. The resulting split contains 132, 63, and 17 cases. Action cases are nearly balanced across Easy / Medium / Hard, with 71, 73, and 73 cases, respectively. Physics cases are grouped into three domains: Mechanics (collision, clipping, deformation, terrain following, gravity), Optics (reflection, occlusion), and 3D Consistency. Clipping (37) and terrain following (34) are the most heavily represented mechanics protocols, followed by collision (19) and deformation (19); reflection (21) dominates the optics group, and 3D consistency contributes 20 cases.

200 100 0

115

107 19

0 0 0 0 0 0 30 300 40 400 50 00 100 00 110 5 10

Figure 7. Test case count and action-sequence length distribution. (a) Test cases by dimension, split by 1st and 3rd perspective. (b) Distribution of action-sequence lengths in frames.

related artifacts.

5. Experiments

4.3. Test Suite Statistics

5.1. Evaluated Models

Figure 6 shows the action-key distribution across the three evaluation dimensions. Memory is dominated by two symmetric pairs: J/L yaw commands each account for 28.3% of key presses, while W/S translation commands each account for 20.4%; A/D lateral motion and I/K pitch commands appear only sparsely, reflecting the revisit-oriented design. Action covers six keys, with W (forward) accounting for roughly half of all key presses (50.1%), followed by S (20.0%), L (12.9%), and J (11.6%). Physics is more strongly skewed toward W (76.8%), as most physics test cases involve sustained forward motion toward obstacles, terrain changes, or long corridors, with smaller A/S/D components used for lateral or backward interactions. Figure 7 summarizes the number of test cases per dimension and the distribution of action-sequence lengths. Overall, the benchmark contains 600+ test cases spanning memory, action following, and physics categories. Most sequences fall in the 300–400 frame range (349 cases), while shorter 0–300 frame sequences account for 115 cases and 400–500 frame sequences account for 107 cases. Longer rollouts are less frequent but still included, with 19 cases in the 500–1000 range and 10 cases in the 1000–1100 range, providing coverage for extended interaction. Figure 8 shows the difficulty and category distributions. Memory cases are classified by the number of action segments, with Easy, Medium, and Hard corresponding to ≤2,

We evaluate representative frontier interactive world models under a unified image-and-action interface. The benchmark includes two closed-source products, Genie 3 and Happy Oyster, which represent strong publicly accessible interactive models, and eight publicly released or open-source models covering the dominant design choices in this model family. These models differ substantially in their control interfaces: some natively accept keyboard actions, some expose pose- or trajectory-conditioned generation, and others use text or learned action tokens. To ensure comparability, every WorldRoamBench test case is converted into the closest supported control format for each model while preserving the same first-frame image, action order, rollout length, and evaluation pipeline. This model set is intentionally heterogeneous. MatrixGame 2.0, Matrix-Game 3.0, Happy Oyster, and Genie 3 natively accept keyboard input and are closest to the target WASD/IJKL interaction protocol. LingBot-World, Lyra 2.0, and SANA-WM are conditioned on continuous camera-pose sequences, testing whether pose-driven models can follow discrete keystroke programs after action-topose conversion. HY-World 1.5 and minWM map each action to a single latent token, evaluating a compact actionconditioning design. Yume 1.5 represents a caption- or instruction-driven interface. Table 2 summarizes the eval11

Table 2. Evaluated interactive world models. In the Input column, icons denote input modalities: = Action (keyboard WASD/IJKL), = Pose (continuous 6-DoF camera trajectory), = Text (caption/instruction). The arrow (→) shows the conversion applied to the keyboard actions—e.g., → means keyboard actions are converted to camera poses before being fed to the model. Models with only accept keyboard input natively with no conversion. FPS reports the native output frame rate of each model. Chunk (frames) is the number of frames produced per autoregressive chunk, and Inf. Speed (Hz) is the average generation throughput computed as chunk frames divided by wall-clock chunk time, measured on a single NVIDIA H20 GPU and averaged over 20 runs. MPPS (million pixels generated per second) is computed as resolution width × height × inference speed divided by 106 . SANA-WM is a bidirectional model, so we evaluate it with 56 frames per chunk. Model

Type

Params

Year

Views

Genie 3 [2] Happy Oyster [3]

Closed Closed

— —

2025 2026

FPV/TPV FPV/TPV

LingBot-World [13] HY-World 1.5 [11] Lyra 2.0 [28] SANA-WM [27] Matrix-Game 3.0 [10] Matrix-Game 2.0 [9] Yume 1.5 [12] minWM [29]

Open Open Open Open Open Open Open Open

14B 8B 14B 2.6B 5B 1.8B 5B 8B

2026 2025 2026 2026 2026 2025 2026 2026

FPV/TPV FPV/TPV FPV FPV FPV FPV FPV FPV

Input

→ →

+ → →

→

+ → →

Resolution

FPS

1280×704 1296×720

20 24

— —

DeepMind Alibaba

832×480 832×480 832×480 1280×704 1280×704 640×352 1280×704 832×480

16 24 16 16 17 12 16 16

12 16 80 56 40 12 32 16

1.79 1.30 2.31 0.81 5.34 6.26 6.49 3.97

0.72 0.52 2.08 0.73 4.82 1.41 5.85 1.58

Ant Group Tencent NVIDIA NVIDIA Skywork Skywork Shanghai AI Lab ShengShu

uated model set, and Appendix B gives model-by-model details and the adapter used for each interface. Table 2 also reports inference speed and throughput. Open-source models vary widely in efficiency: Yume 1.5 is fastest at 6.49 Hz, followed by Matrix-Game 2.0 (6.26 Hz) and Matrix-Game 3.0 (5.34 Hz), while SANA-WM is slowest at 0.81 Hz, with HY-World 1.5 (1.30 Hz) and LingBotWorld (1.79 Hz) similarly throughput-limited. Normalized by resolution, Yume 1.5 and Matrix-Game 3.0 still lead in MPPS (5.85 and 4.82), reflecting efficient high-resolution generation, whereas HY-World 1.5, LingBot-World, and SANA-WM stay below 1 MPPS (0.52, 0.72, 0.73) despite comparable or larger parameter counts. These gaps highlight the trade-off between scale, visual fidelity, and realtime deployability in interactive world generation.

Chunk (frames) Inf. Speed (Hz)

MPPS Source

over to third-person evaluation for the models that support it. However, high action scores do not imply uniformly strong performance in other dimensions. Models that follow commands accurately can still violate physical constraints, indicating that controllability and physical plausibility remain partially decoupled. Visual quality and physics. Visual quality is comparatively clustered among the stronger models. SANA-WM, HY-World 1.5, and Lyra 2.0 perform especially well in firstperson visual quality, showing that high perceptual quality is achievable even without leading overall performance. In contrast, the physics dimension exposes a substantially larger gap. Happy Oyster and Genie 3 lead first-person interaction physics, while Genie 3 is strongest in third-person interaction physics and the remaining third-person models are closer to one another. These results suggest that current open models can follow camera commands well, but still struggle to consistently enforce collision, clipping, terrain, and optical constraints during interaction.

5.2. Main Results Overall ranking. Table 3 reports the first-person and third-person quantitative results, with metrics as rows and models as columns. Genie 3 achieves the best overall performance in first-person view, while Happy Oyster ranks highest in third-person view after the TPV overall score is weighted by control rate. Across both perspectives, the rankings show that no model dominates all dimensions. Closed-source models are generally stronger in physics and memory, whereas several open models remain competitive in action following and visual quality. In third-person evaluation, Happy Oyster benefits from the strongest control rate and subject memory, while HY-World 1.5 is substantially stronger in action following and visual quality.

Memory. Memory scores reveal a further distinction between controllability and persistence. Genie 3 shows the strongest memory performance overall, while several open models preserve first-person scene geometry reasonably well despite weaker physics. In third-person view, Happy Oyster is especially strong in subject memory. This supports our observation that Happy Oyster has particularly strong third-person subject memory, even though its actionfollowing score is lower than the strongest open models.

6. Discussion

Action following. Open models achieve strong action scores in first-person view. HY-World 1.5, LingBot-World, and Lyra 2.0 are particularly strong at executing the prescribed camera actions, and this advantage largely carries

6.1. Key Findings The evaluation reveals four key findings that challenge common assumptions about interactive world models. 12

Table 3. Unified quantitative results on WorldRoamBench. All values ×100. Metrics are rows and models are columns, organized into an FPV table, a TPV table, and an Overall-score ranking panel. Model headers retain full names and logos; gray columns denote closedsource models. Bold/underline = best/2nd-best within each perspective panel and metric. ↑ = higher is better; ↓ = lower is better. For both FPV and TPV, Action Score = mean(Strict Acc, Partial Acc, Traj), Visual Score = mean(Aesthetic, 1−Aesthetic Drift, Imaging Quality, 1−Imaging Drift), and Overall Score = mean(Action, Visual, Physics, Memory). The view-specific terms are Physics Score = mean(Mech, Opt, 3D) and Memory Score = F1 for FPV; Physics Score = mean(Mech, Opt) (no 3D data) and Memory Score = mean(F1 , Subject) for TPV. TPV Overall Score is further weighted by Control Rate. (a) First-person View (FPV)

Metric

Genie 3

Happy Oyster

Lyra 2.0

HY-World 1.5

LingBot-World

Matrix-Game 3.0

SANA-WM

Matrix-Game 2.0

Yume 1.5

minWM

Overall Score ↑

73.81

71.06

70.32

70.29

64.25

63.31

62.16

56.67

56.21

49.47

Action Score ↑ Strict Accuracy ↑ Partial Accuracy ↑ Trajectory ↑ nATEt ↓ nATEr ↓

84.78 75.19 89.13 90.03 8.61 11.34

87.32 79.56 89.85 92.55 7.38 7.51

91.41 87.62 94.77 91.84 5.61 10.71

91.61 89.82 91.41 93.61 7.16 5.63

91.31 88.32 94.70 90.91 7.12 11.07

88.82 83.74 92.95 89.78 10.33 10.11

83.95 70.68 88.67 92.51 6.30 8.68

84.97 79.01 85.97 89.93 9.58 10.57

71.45 56.12 77.32 80.91 22.57 15.60

64.20 49.93 57.20 85.45 9.00 20.10

Visual Score ↑ Aesthetic Score ↑ Aesthetic Drift ↓ Imaging Quality ↑ Imaging Drift ↓

68.28 54.23 15.78 60.11 25.46

69.16 56.78 14.50 50.47 16.13

71.61 60.67 18.87 61.64 17.02

72.52 63.51 13.85 56.33 15.91

67.98 58.94 23.34 59.29 22.96

70.56 58.32 22.68 66.17 19.57

73.08 61.50 16.74 66.14 18.60

61.54 53.88 30.33 48.09 25.49

70.29 58.90 19.55 64.83 23.01

60.55 53.52 40.06 51.08 22.33

Physics Score ↑ Mechanics ↑ Optics ↑ 3D Consistency ↑

68.95 64.30 65.60 76.96

72.33 78.60 60.60 77.79

56.99 35.70 50.70 84.56

47.42 14.30 46.90 81.07

47.32 25.00 50.80 66.16

41.25 17.90 39.70 66.14

36.29 21.40 42.10 45.38

34.50 17.90 38.50 47.11

44.05 35.70 61.50 34.95

20.74 7.10 19.60 35.52

Memory Score ↑ Retention ↑ Hallucination ↓

73.24 71.63 25.07

55.42 50.00 37.85

61.26 56.85 33.59

69.60 68.35 29.10

50.39 46.97 45.64

52.60 51.09 45.80

55.32 51.22 39.85

45.65 43.46 51.93

39.06 36.19 57.58

52.41 48.19 42.56

(b) Third-person View (TPV)

Metric

FPV medals mark top-3 Happy Oyster

Genie 3

LingBot-World

HY-World 1.5

Overall Score ↑ Control Rate ↑

60.24 84.43

57.04 77.36

32.19 45.75

15.75 21.23

Action Score ↑ Strict Accuracy ↑ Partial Accuracy ↑ Trajectory ↑ nATEt ↓ nATEr ↓

81.67 62.98 90.38 91.64 9.13 7.58

77.68 60.75 85.42 86.86 12.95 13.34

83.99 70.71 91.76 89.49 10.27 10.76

90.16 77.25 98.33 94.91 5.48 4.71

Visual Score ↑ Aesthetic Score ↑ Aesthetic Drift ↓ Imaging Quality ↑ Imaging Drift ↓

71.93 62.00 12.24 51.84 13.87

73.60 60.90 11.72 61.49 16.29

74.10 63.35 14.44 62.84 15.34

76.23 65.96 9.67 58.23 9.62

Physics Score ↑ Mechanics ↑ Optics ↑

57.10 63.90 50.30

65.15 70.50 59.80

57.65 60.70 54.60

56.80 52.50 61.10

Memory Score ↑ Retention ↑ Subject Memory ↑ Hallucination ↓

74.72 68.98 83.80 37.06

78.50 76.27 79.03 20.20

65.71 58.81 70.78 36.89

73.58 70.88 75.13 26.79

3 4 5 6 7 8 9 10

Genie 3 Happy Oyster Lyra 2.0 HY-World 1.5 LingBot-World Matrix-Game 3.0 SANA-WM Matrix-Game 2.0 Yume 1.5 minWM

TPV Happy Oyster 1 2 Genie 3 LingBot-World 3 4 HY-World 1.5

Trajectory score ̸= per-frame correctness. Models with high trajectory alignment (trajectory score above 85) can exhibit below 65% per-frame strict action accuracy, revealing a hidden failure mode in which a model eventually drifts

73.81 71.06 70.32 70.29 64.25 63.31 62.16 56.67 56.21 49.47

60.24 57.04 32.19 15.75

along the right path while missing individual keystrokes. Trajectory-level metrics aggregate motion over long segments, so delayed responses, short-term stalls, and compensatory over-corrections can cancel out geometrically even 13

though the interactive experience remains poor. This supports the need for keystroke-level evaluation alongside trajectory scoring.

Table 4. Third-person controllability. Control rate is computed over 113 third-person test cases before subject memory scoring. Values are reported ×100 to match Tab. 3.

High visual quality ̸= good action following. Models with top imaging or aesthetic scores do not necessarily excel at per-frame strict action accuracy, indicating that visual quality and controllability are largely independent capabilities. High-quality textures, stable lighting, and appealing composition can coexist with weak responses to user commands. This finding suggests that optimizing perceptual quality alone is insufficient; interactive world models must also be explicitly trained and evaluated for action responsiveness.

Model

Genie 3 Happy Oyster HY-World 1.5 LingBot-World

Control Rate

77.36

84.43

21.23

45.75

ering; a start-vs-end comparison would treat them as stable, but our segment-based best-vs-worst drift (Sec. 3.3) still localizes and penalizes the dip, which is essential for faithfully ranking models whose error accumulation is nonmonotonic. Importantly, low drift and high average quality are correlated but not identical: Yume 1.5 starts strong yet accumulates 23.01 imaging drift, matching Tab. 3. This confirms that frame-averaged quality alone cannot capture temporal stability. These patterns underscore the importance of evaluating temporal visual stability rather than relying solely on frame-averaged quality metrics.

Stricter physics adherence may compromise GT action following. Models that avoid clipping, respect collisions, or follow terrain constraints often deviate from predefined ground-truth trajectories, revealing a trade-off between physical plausibility and instruction fidelity. For example, a model that correctly slows down or stops at a wall will accumulate trajectory error relative to an unconstrained path, even though the behavior is physically desirable. This motivates separating action-following metrics from interaction-physics metrics rather than collapsing them into a single trajectory score.

Strict action accuracy degrades with action-switch complexity. Figure 10 shows how per-frame strict action accuracy varies with the number of ground-truth action switches. All models achieve their highest strict accuracy on easy sequences (constant action), and strict accuracy generally decreases as the number of switches increases from 0 to 2. HY-World 1.5 and LingBot-World exhibit the smallest degradation (<5 pp drop from easy to hard), indicating robust responsiveness even under frequent direction changes. In contrast, minWM and Yume 1.5 show the largest drops (>15 pp), suggesting these models struggle to re-align with new commands after a switch. Interestingly, several models (e.g., Matrix-Game 3.0, Genie 3) show comparable or even slightly higher strict accuracy on hard sequences than medium ones, indicating that their failures are not monotonically related to switch count but may depend on specific action combinations. The partial accuracy metric is consistently higher and more resilient across difficulties, confirming that models often execute the correct movement axis even when they fail to match the exact discrete action label.

Memory evaluation is confounded by action-following imprecision. Current models rarely reach the prescribed turning point or revisit location exactly, so symmetric frame pairs often depict different spatial positions. As a result, frame-pair metrics conflate action error with memory degradation: a model may appear to forget a scene simply because it failed to return to the same viewpoint. Actionaware transition localization and scene-level 3D point-cloud reconstruction decouple these factors by comparing aggregated geometry across the observation and revisit segments. As detailed in Appendix H, this trajectory-aware protocol is substantially more robust than GT-indexed frame matching under imperfect control.

6.2. Observations in World Models

Third-person controllability varies dramatically across models. Before third-person subject memory is evaluated, each model is tested for controllable third-person generation. A test case is counted as controllable if the generated rollout preserves a visible third-person subject and the subject or camera responds coherently to the commanded motion. Among 113 third-person test cases, Happy Oyster achieves the highest control rate (84.43), followed by Genie 3 (77.36), LingBot-World (45.75), and HY-World 1.5 (21.23), matching the TPV control-rate row in Tab. 3 and summarized in Tab. 4. The resulting gap is substantial: Happy Oyster’s control rate is roughly 4.0× that of HYWorld 1.5, and Genie 3 is roughly 3.6× higher. Qualitatively, Genie 3 and Happy Oyster exhibit sub-

Visual quality degradation exhibits model-specific temporal patterns. Figure 9 reveals that per-frame quality evolution varies dramatically across models. The lowerdrift models in Tab. 3 (e.g., HY-World 1.5 with the lowest aesthetic drift at 13.85 and the lowest imaging drift at 15.91, followed closely by Happy Oyster) maintain more consistent quality throughout 300 frames, while Matrix-Game 3.0 shows a progressive, accelerating decline driven by compounding autoregressive error accumulation. minWM exhibits a distinct selective collapse, with its aesthetic score crashing after frame 150 even as imaging stays flat. A subset of models additionally show transient mid-rollout collapse, dipping sharply mid-sequence before partially recov14

Visual Quality: Per-frame Curves & Drift Comparison (1st-person)

Per-frame Aesthetic Quality (mean over videos, first 300 frames)

0.80

0.70

0.75

0.65

0.70

0.60

0.65

0.55 Score

Score

Per-frame Imaging Quality (mean over videos, first 300 frames)

0.60 0.55

0.50 0.45

0.50

Happy Oyster Genie 3 HY-World 1.5 Matrix-Game 3.0 Matrix-Game 2.0

0.45 0.40 0

0.40

LingBot-World Lyra 2.0 minWM SANA-WM Yume 1.5

0.30

100

150 Frame index

200

250

300

LingBot-World Lyra 2.0 minWM SANA-WM Yume 1.5

Drift Imaging (lower = more stable)

0.40

100

150 Frame index

200

250

300

Drift Aesthetic (lower = more stable) 0.5

0.35 0.30

0.4

0.25

Drift Score

Happy Oyster Genie 3 HY-World 1.5 Matrix-Game 3.0 Matrix-Game 2.0

0.35

0.20 0.15

0.3 0.2

0.10 0.1

0.05 0.00 py Hap

ter Oys

ie 3

Gen

rld Wo HY-

1.5 t Ma

e Gam rix-

3.0 t Ma

e Gam rix-

orld t-W

2.0

Bo ing

a Lyr

2.0

WM min

M A-W

SAN

0.0

Yum

Hap

ter

Oys

ie 3

Gen

1.5 3.0 2.0 orld rld me me t-W Wo -Ga -Ga gBo HYtrix trix Lin a a M M

a Lyr

2.0

min

M A-W S AN

Yum

Figure 9. Per-frame visual quality curves and drift comparison (first-person). Top: mean imaging and aesthetic scores at each frame index (averaged over 120 videos per model, first 300 frames). Bottom: drift scores quantifying quality degradation from the best to the worst segment of the rollout (lower = more stable); bar height is the mean drift across videos, and black error bars span from mean−1 std (lower cap) to mean+1 std (upper cap), reflecting cross-video variance in degradation. Models exhibit distinct degradation patterns: some maintain stable quality throughout, while others show progressive collapse after extended generation. Per-frame Action Accuracy by Difficulty (1st-person, all models) Accuracy by Difficulty

Partial Accuracy by Difficulty Easy Medium Hard

100

80 Partial Accuracy (%)

Accuracy (%)

80 60 40 20 0 py Hap

Easy Medium Hard

100

60 40 20

ter Oys

.0 .0 1.5 ie 3 orld e3 e2 rld Gen am am ot-W Wo ngB ix-G ix-G HYi r r t t L Ma Ma

a Lyr

2.0

min

M A-W

SAN

Yum

Hap

ter

Oys

Gen

ie 3

.0 .0 1.5 orld e3 e2 rld am am ot-W Wo ngB ix-G ix-G HYi r r t t L Ma Ma

a Lyr

2.0

min

M A-W

SAN

Yum

Figure 10. Per-frame action accuracy by difficulty (first-person). Strict accuracy (left) and partial accuracy (right) grouped by difficulty level (easy = constant action, medium = 1 action switch, hard = 2 action switches). Bar height is the mean across test cases. Most models degrade from easy to hard, but the magnitude of degradation varies substantially.

stantially stronger third-person control than the open mod-

els. Both models maintain coherent third-person interac-

7. Conclusion

tion across realistic and game-like scenes, with Happy Oyster performing particularly strongly; this pattern is consistent with its high third-person subject memory score in Tab. 3. One plausible explanation is that Happy Oyster includes specialized optimization for third-person data, avatar-centric interaction, or architectures that explicitly support third-person control. By contrast, HY-World 1.5 exhibits third-person controllability mainly in particular game-style domains, such as Unity-like scenes, first-personshooter-like scenes, or Genshin-style scenes. HY-World 1.5 does not expose an explicit hyperparameter for switching between first-person and third-person generation, suggesting that its third-person ability emerges from training data and the base model rather than from an explicit perspectivecontrol mechanism. LingBot-World is more controllable than HY-World 1.5 in third-person settings, but it also does not inject explicit perspective labels or person-view controls. Thus, for HY-World 1.5 and LingBot-World, third-person controllability appears to arise primarily from data coverage and base-model generalization, whereas the strongest closed-source models may benefit from more deliberate third-person design.

WorldRoamBench benchmarks long-horizon interactive world models under continuous WASD/IJKL control. By combining keystroke-level action following, visual stability, interaction physics, and trajectory-aware scene/subjectlevel memory evaluation, WorldRoamBench exposes failure modes that are largely hidden by short clips, trajectoryonly metrics, or frame-pair memory tests. Evaluation of 10+ open- and closed-source models shows that no model dominates all dimensions: strong visual quality or trajectory alignment does not guarantee per-frame controllability, physical plausibility, or reliable memory. These results highlight the need for future world models to integrate action responsiveness, physical grounding, and persistent scene and subject memory. WorldRoamBench provides a diagnostic foundation for measuring progress toward genuinely interactive and stable world simulation.

References [1] Bruce, J., Dennis, M., Edwards, A., et al. Genie: Generative interactive environments. In ICML, 2024. [2] Google DeepMind. Genie 3: A new frontier for world models. https://deepmind.google/discover/ blog/genie- 3- a- new- frontier- for- worldmodels/, 2025. [3] Alibaba Group. Happy Oyster: An open-ended world model for real-time world creation and interaction. https:// happyoyster.cn/, 2026. [4] Wan Team. Wan 2.1: A comprehensive and unified video generation model. arXiv preprint arXiv:2503.20314, 2025. [5] Kuaishou. Kling 3.0: Next-generation AI video generation. https://klingai.com/, 2025. [6] OpenAI. Sora 2: A large-scale video generation model. https://openai.com/sora/, 2025. [7] Google DeepMind. Veo 3: State-of-the-art video generation. https : / / deepmind . google / technologies / veo/, 2025. [8] ByteDance Seed Team. Seedance 2.0: Advancing video generation for world complexity. arXiv preprint arXiv:2506.05218, 2025. [9] Matrix-Game Team. Matrix-Game 2.0: An open-source, real-time, and streaming interactive world model. arXiv preprint arXiv:2508.13009, 2025. [10] Matrix-Game Team. Matrix-Game 3.0: Real-time and streaming interactive world model with long-horizon memory. arXiv preprint, 2026. [11] HY-World Team. HY-World 1.5: A systematic framework for interactive world modeling with real-time latency and geometric consistency. arXiv preprint, 2025. [12] Yume Team. Yume 1.5: A text-controlled interactive world generation model. arXiv preprint, 2026. [13] LingBot Team. Advancing open-source world models. arXiv preprint, 2026.

First-person physics depends on whether the camera is embodied by a held object. In the physics dimension, the same first-person scene can yield different physical behavior depending on whether the view contains a visible held object. For example, first-person views containing a weapon in an FPS-style scene, a cup in hand, or another foreground object behave differently from clean camera-only first-person views with no visible body or object. Within the same scene, Genie 3 is less likely to produce clipping violations when the first-person view is visually grounded by a held object, whereas clean first-person camera views more often pass through walls or other solid geometry. Textual prompts that describe the first-person subject as human do not fully remove this gap. This suggests that even models with strong overall physics performance, such as Genie 3 and Happy Oyster, may rely partly on data-derived visual priors for physical compliance rather than fully internalized physical understanding.

6.3. Future Work Future extensions of WorldRoamBench should cover broader open-world environments with greater diversity in scene categories, visual styles, weather conditions, and interaction contexts. The physics suite can also be expanded beyond mechanics and optics to include additional physical dimensions, such as electricity, thermodynamics, and other forms of material or energy transfer. Finally, human preference studies would complement automatic metrics by capturing user-centered judgments of controllability, realism, and long-horizon interaction quality. 16

[35] Teed, Z. and Deng, J. DROID-SLAM: Deep visual SLAM for monocular, stereo, and RGB-D cameras. In NeurIPS, 2021. [36] Ying, Z., et al. WBench: A comprehensive benchmark for evaluating world models via action-conditioned video generation. arXiv preprint, 2026. [37] Chen, Y. and Medioni, G. Object modelling by registration of multiple range images. Image and Vision Computing, 10(3):145–155, 1992. [38] Rusu, R. B., Blodow, N., and Beetz, M. Fast Point Feature Histograms (FPFH) for 3D registration. In ICRA, 2009. [39] Wu, X., Jiang, L., Wang, P.-S., et al. Point Transformer V3: Simpler, faster, stronger. In CVPR, 2024. [40] Ravi, N., Gabeur, V., Hu, Y.-T., et al. SAM 2: Segment anything in images and videos. arXiv preprint arXiv:2408.00714, 2024. [41] Farnebäck, G. Two-frame motion estimation based on polynomial expansion. In Scandinavian Conference on Image Analysis (SCIA), 2003. [42] Bu, W., Wu, Y., Yu, Q., et al. What limits virtual agent application? OmniBench: A scalable multi-dimensional benchmark for essential virtual agent capabilities. In ICML, 2025. [43] Wu, M., Cai, Z., Zhao, F., et al. Omni-WorldBench: Towards a comprehensive interaction-centric evaluation for world models. arXiv preprint arXiv:2603.22212, 2026. [44] Gao, Q., Pi, X., Liu, K., et al. Do vision-language models have internal world models? Towards an atomic evaluation. In ACL, 2025. [45] Kang, B., Yue, Y., Lu, R., et al. How far is video generation from world model: A physical law perspective. In ICML, 2025. [46] Liang, A., Kong, L., Yan, T., et al. WorldLens: Full-spectrum evaluations of driving world models in real world. arXiv preprint arXiv:2512.10958, 2025. [47] Shang, Y., Li, Z., Ma, Y., et al. WorldArena: A unified benchmark for evaluating perception and functional utility of embodied world models. arXiv preprint arXiv:2602.08971, 2026. [48] Arai, H., Ishihara, K., Takahashi, T., and Yamaguchi, Y. ACT-Bench: Towards action controllable world models for autonomous driving. arXiv preprint arXiv:2412.05337, 2024. [49] Hu, Y., Huang, S., Liao, Y., et al. EWMBench: Evaluating scene, motion, and semantic quality in embodied world models. In BMVC, 2025. [50] Zhao, Y., Zhao, W., Wang, W., Zhang, Z., An, D., Liu, A., Yu, Y., Tang, J., Wang, F., Wang, W., and Zhuang, B. WorldOlympiad: Can Your World Model Survive a Triathlon? arXiv preprint arXiv:2606.11129, 2026. [51] Teed, Z. and Deng, J. RAFT: Recurrent All-Pairs Field Transforms for Optical Flow. arXiv preprint arXiv:2003.12039, 2020.

[14] Ye, H., Lu, J., et al. MIND: Benchmarking memory consistency and action following in world models. arXiv preprint arXiv:2602.08025, 2026. [15] Alaya Studio. WorldMark: A unified benchmark suite for interactive video world models. arXiv preprint arXiv:2604.21686, 2026. [16] Li, Y., et al. iWorld-Bench: A benchmark for interactive world models with a unified action generation framework. In ICML, 2026. [17] Shanda AI. WildWorld: A large-scale dataset for dynamic world modeling with actions and explicit state toward generative ARPG. arXiv preprint arXiv:2603.23497, 2026. [18] Huang, Z., He, Y., Yu, J., Zhang, F., Si, C., et al. VBench: Comprehensive benchmark suite for video generative models. In CVPR, 2024. [19] Huang, Z., Zhang, F., Xu, X., He, Y., Yu, J., et al. VBench++: Comprehensive and versatile benchmark suite for video generative models. arXiv preprint arXiv:2411.13503, 2024. [20] Zheng, D., Huang, Z., Liu, H., Zou, K., He, Y., et al. VBench-2.0: Advancing video generation benchmark suite for intrinsic faithfulness. arXiv preprint arXiv:2503.21755, 2025. [21] Stanford. WorldScore: A unified evaluation benchmark for world generation. In ICCV, 2025. [22] UC Berkeley. WorldModelBench: Judging video generation models as world models. In NeurIPS, 2025. [23] SJTU, et al. Towards world simulator: Crafting physical commonsense-based benchmark for video generation. In ICML, 2025. [24] UCLA. WorldBench: Disambiguating physics for diagnostic evaluation of world models. arXiv preprint arXiv:2601.21282, 2026. [25] Baker, B., et al. Video PreTraining (VPT): Learning to act by watching unlabeled online videos. In NeurIPS, 2022. [26] Microsoft. MineWorld: A real-time and open-source interactive world model on Minecraft. arXiv preprint arXiv:2504.08388, 2025. [27] NVIDIA. SANA-WM: Efficient minute-scale world modeling with hybrid linear diffusion transformer. arXiv preprint, 2026. [28] NVIDIA. Lyra 2.0: Explorable generative 3D worlds. arXiv preprint, 2026. [29] minWM Team. minWM: A full-stack open-source framework for real-time interactive video world models. arXiv preprint, 2026. [30] Schuhmann, C., Beaumont, R., Vencu, R., et al. LAION5B: An open large-scale dataset for training next generation image-text models. In NeurIPS, 2022. [31] Ke, J., Wang, Q., Wang, Y., Milanfar, P., and Yang, F. MUSIQ: Multi-scale image quality transformer. In ICCV, 2021. [32] Helios Team. Helios: A comprehensive benchmark for video generative models. arXiv preprint, 2025. [33] WorldCompass Team. WorldCompass: Reinforcement learning for long-horizon world models. arXiv preprint, 2026. [34] ViPE: Visual pose estimation for camera trajectory recovery.

A. Test Suite Gallery

It generates video at 1296×720 resolution and 24 fps, the highest native frame rate among the evaluated models. Like Genie 3, it is evaluated as a black box with direct execution of the benchmark action sequence through its native keyboard interface. Happy Oyster is especially strong in thirdperson subject memory and first-person interaction physics, suggesting robust subject-memory preservation and physical plausibility, though its action-following scores are lower than those of the strongest open models.

To provide a qualitative overview of the visual and action coverage in WorldRoamBench, we include a gallery of representative test cases in Figure 11. Each panel shows the first-frame image together with an action schematic, illustrating both the scene context and the intended WASD/IJKL trajectory. The gallery is organized along two axes: scene category and camera perspective. Rows correspond to the three major scene families used throughout the benchmark—Indoor, Urban, and Nature—while columns separate first-person and third-person views. This layout highlights that each semantic environment type is evaluated under both egocentric exploration and avatar-centered interaction, enabling the benchmark to probe camera control, physical interaction, scene memory, and subject memory across diverse visual domains.

LingBot-World [13]. LingBot-World is Ant Group’s open-source interactive world model framework that bridges passive video generation and interactive world simulation. It is built on a Wan2.2 14B I2V backbone. For WorldRoamBench, the officially released LingBot-World-Fast checkpoint is used because it supports autoregressive generation and is therefore more suitable for interactive evaluation. The model consumes camera-pose sequences as the control signal: each navigation action is first converted into a deterministic pose sequence by a geometric mapping and then fed to the backbone for frame generation.

B. Evaluated Model Details WorldRoamBench evaluates models as black-box or graybox interactive generators: each model receives the same initial frame and an action program derived from the WASD/IJKL vocabulary, then produces an autoregressive video rollout scored by the same action, visual-quality, physics, and memory pipelines. No model is fine-tuned on WorldRoamBench. For models with non-keyboard interfaces, deterministic adapters map each benchmark keystroke to the closest supported primitive, so differences in scores reflect generated behavior rather than changes in the test cases. Table 5 summarizes the adapter used for each evaluated model.

HY-World 1.5 [11]. HY-World 1.5 is Tencent Hunyuan’s 8B streaming Diffusion Transformer for real-time interactive world generation, built on the 480p I2V branch of HunyuanVideo 1.5. It introduces a dual action representation that encodes both discrete keyboard tokens and continuous camera deltas, together with a reconstituted context memory module that compresses long-horizon history into a fixed-size token buffer to maintain geometric consistency across hundreds of generation steps. This design allows the model to sustain coherent spatial layouts and stable subject appearances over extended multi-turn trajectories, which is reflected in its strong navigation adherence scores on WorldRoamBench.

B.1. Model Descriptions Genie 3 [2]. Genie 3 is Google DeepMind’s closed-source interactive world model, accessible through a productstyle interface that supports gamepad and WASD-style keyboard control. It generates photorealistic environments at 1280×704 resolution and 20 fps, and is evaluated as a black box: WorldRoamBench supplies the first frame, keyboard program, character prompt, and environment description through the native interactive interface, downloads the generated video from the backend, and applies the same posthoc metrics used for open models. Because the internal architecture and sampling parameters are not exposed, only externally observable properties such as supported controls, resolution, and generated behavior are reported. Genie 3 achieves the strongest first-person overall score and the second-highest third-person overall score after control-rate weighting, with particularly strong interaction physics and memory.

Lyra 2.0 [28]. Lyra 2.0 is NVIDIA’s open-source 14Bparameter interactive world model that accepts continuous 6-DoF camera-pose sequences as control input. It employs a 3D spatial memory system that maintains a sparse cache of depth maps and camera poses, projecting previously generated content into new viewpoints via depthguided forward warping for long-horizon consistency. For WorldRoamBench, each discrete WASD/IJKL command is converted into a local camera-pose increment, preserving action order and segment duration. Lyra 2.0 achieves competitive first-person action-following scores among open models and a top-three first-person visual score, indicating that 3D-aware generation can support precise controllability without sacrificing perceptual fidelity. SANA-WM [27]. SANA-WM is NVIDIA’s open-source 2.6B-parameter world model. It generates video at 1280×704 resolution and 16 fps, accepting continuous camera-pose sequences as input. We use the bidirectional

Happy Oyster [3]. Happy Oyster is Alibaba’s closedsource interactive world model, accessed through a productstyle interface with native WASD/IJKL keyboard support. 18

Third Person View

Memory

Physics

Action

First Person View

Translation Action: WASD

Rotation Action: IJKL

Figure 11. Test suite gallery. Each panel shows a representative test case with the first frame, trajectory schematic, and action sequence. The gallery is organized by scene category from top to bottom (Indoor, Urban, Nature) and by perspective from left to right (first-person, third-person), yielding six panels.

checkpoint, which generates frames conditioned on both past and future context within each chunk, and evaluate it with 56 frames per chunk to match its bidirectional design. For WorldRoamBench, WASD/IJKL commands are converted to local camera translation and rotation increments before generation, following the same adapter used for Lyra 2.0 and HY-World 1.5. Despite its smaller parameter count, SANA-WM achieves the strongest first-person visual-quality score among the evaluated models, suggesting that efficient architectures can deliver strong perceptual quality at interactive rates.

pose inputs. It generates video at 1280×704 resolution and 17 fps. Matrix-Game 3.0 achieves strong action-following performance, particularly in partial accuracy, indicating effective long-horizon command adherence, though its interaction physics scores suggest room for improvement in enforcing physical constraints. Matrix-Game 2.0 [9]. Matrix-Game 2.0 is Skywork’s earlier 1.8B-parameter interactive world model and the predecessor to Matrix-Game 3.0. It generates video at 640×352 resolution and 12 fps, the lowest resolution and frame rate among the evaluated models. The model accepts discrete one-hot action tokens as its control interface. The same nearest-primitive mapping from the benchmark vocabulary as Matrix-Game 3.0 is used, with output resolution

Matrix-Game 3.0 [10]. Matrix-Game 3.0 is Skywork’s open-source 5B-parameter interactive world model that accepts both discrete action tokens and continuous camera19

Table 5. Model evaluation adapters. Each model receives the benchmark’s WASD/IJKL actions through a deterministic adapter that converts them to the model’s native input format. Model

Evaluation Adapter

Genie 3 [2]

Native WASD/IJKL support. Actions are injected via frontend automation at 1/20 s per action, synchronized with the model’s 20 fps output. Native WASD/IJKL support. Actions are injected via frontend automation at 1/24 s per action, synchronized with the model’s 24 fps output. Autoregressive (Fast) checkpoint; only pose is passed in. Actions are converted to camera poses at a translation speed of 0.2 per frame and a rotation speed of 0.75◦ per frame. Native keyboard support, but the model accepts one action per latent frame. Four consecutive frame-level actions are merged into a single latent-frame action before being fed to the model. Actions are converted to camera poses at a translation speed of 0.2 per frame and a rotation speed of 0.75◦ per frame, preserving action order. WASD/IJKL to local camera translation and rotation increments. Native per-frame action support; each benchmark action is passed directly as a per-frame action token, and the model internally converts it to camera extrinsics before generation. Native per-frame action support; each benchmark action is passed directly as a per-frame action token. Pure text-conditioned model with no pose or action tensor input. Every 32 frame-level actions are aggregated by taking the dominant WASD and IJKL keys within the segment, then rendered into a fixed-template motion caption (e.g., “Person moves forward; Camera turns left”). Each caption controls one autoregressive chunk of 32 pixel frames. Native action support; one action per latent frame with a translation speed of 0.08 units and a rotation speed of 3◦ per latent frame.

Happy Oyster [3] LingBot-World [13] HY-World 1.5 [11] Lyra 2.0 [28] SANA-WM [27] Matrix-Game 3.0 [10] Matrix-Game 2.0 [9] Yume 1.5 [12]

minWM [29]

B.2. Closed-Source Model Evaluation

and rollout length matched to the model’s public inference setting. Matrix-Game 2.0 serves as a lightweight baseline for understanding how model scale and resolution affect interactive controllability.

B.2.1. Closed-Source Model Data Acquisition Evaluating closed-source interactive world models such as Genie 3 and Happy Oyster presents a practical challenge because these models are accessed only through product-style interfaces that do not expose programmatic APIs for batch inference. A frontend automation pipeline therefore drives the interactive session by injecting WASD/IJKL keystrokes at the model’s native frame rate. For Genie 3, which outputs video at 20 fps, each benchmark action is held for 1 20 s before the next action is issued; for Happy Oyster, 1 s. The auwhich runs at 24 fps, each action is held for 24 tomation layer simulates real keyboard events so that the closed-source model receives inputs indistinguishable from human interaction, while the action schedule remains deterministic and reproducible. Fig. 12 illustrates the end-to-end pipeline: benchmark test cases are fed into a Playwrightdriven browser session, where the full action list is injected and dispatched at the model’s native frame rate, and the backend-generated video is retrieved once the session completes.

Yume 1.5 [12]. Yume 1.5 is Shanghai AI Lab’s opensource 5B-parameter interactive world generation model controlled through natural-language instructions. It generates video at 1280×704 resolution and 16 fps. For WorldRoamBench, each action segment is rendered as a concise motion instruction—such as “camera moves forward” or “camera turns left”—while the segment schedule follows the original benchmark action sequence. This setting tests a fundamentally different interaction paradigm: the model receives semantic motion descriptions rather than explicit low-level controls, but the output is still judged against the same keystroke-level and trajectory-level criteria, revealing the gap between language-guided and action-guided controllability. minWM [29]. minWM is ShengShu’s open-source world model that accepts continuous camera-pose sequences as control input. We use the HY15/Action2V/dmd checkpoint downloaded from the official Hugging Face repository (MIN-Lab/minWM). For WorldRoamBench, benchmark actions are mapped to the nearest supported camera-pose increment using the same deterministic adapter as other poseconditioned models.

B.2.2. RAFT Alignment for Closed-Source Videos Unlike open-source models with end-to-end control over the generation pipeline and known frame-to-action correspondence, closed-source models return a video file downloaded from the backend after the interactive session com20

action sequence (WASD/IJKL) Closed-source; no public API; video available after session

initial image

Playwright (Open and Operate Happy Oyster, Genie 3)

create world

Backend (DataChannel → Model) video URL

KeyEvent dispatch DataChannel

scene description

Status Poll + Download

JS Executor (Action Injection; dispatch actions at fixed fps)

generated video

Figure 12. Closed-source automated interaction pipeline for Happy Oyster and Genie 3. The system reads benchmark test cases (action sequence, first frame, and scene description), then uses Playwright to drive the target platform’s web interface. The entire action list is injected into the browser’s V8 engine via a single page.evaluate() call, where actions are dispatched at intervals matching the 1 1 s for Happy Oyster, 20 s for Genie 3). The frontend encodes the keyboard commands and sends them to model’s native frame rate (e.g., 24 the backend via a WebRTC DataChannel. After the session completes, the backend-generated video is downloaded via a status API polling mechanism.

C.1. Motivation

pletes. The downloaded video often contains extra frames before the first effective action takes effect (e.g., loading screens, transition animations, or buffering delays), and may have a different total frame count than the benchmark action sequence.

Current interactive world models generate video in chunks of K=4 frames per autoregressive step, meaning consecutive frames within a chunk share the same underlying action decision. Applying per-frame discretization (stride=1) to such videos amplifies pose-estimation noise: tiny interframe displacements are close to the noise floor and frequently trigger spurious action labels. The latent-stride approach computes relative poses over K-frame windows, providing a stronger motion signal while aligning with the model’s native generation cadence.

To align the downloaded video with the ground-truth action program, we use RAFT (Recurrent All-Pairs Field Transforms) [51] optical flow to detect the onset of motion. Specifically, we compute the mean optical flow magnitude between consecutive frames; the first frame whose flow magnitude exceeds a threshold is taken as the effective start frame. From this start frame, we crop the video to the required number of frames that matches the benchmark action schedule, ensuring that the evaluation metrics (action following, visual quality, physics, and memory) are computed on the same temporal window as for open-source models.

C.2. Threshold Scaling All thresholds are scaled proportionally to the stride K to maintain equivalent sensitivity, as summarized in Table 6: Table 6. Threshold comparison across CompassReward versions. Parameter

v1 (K=1)

v2 (K=4)

v3 (K=4)

{0.002, 0.005, 0.01} 0.2◦ 0.001

{0.002, 0.005, 0.01} 0.2◦ 0.004

{0.008, 0.02, 0.04} 0.8◦ 0.004

C. Latent-Stride Action Discretization Details

τmove τrot τaxis

The per-frame action discretization builds on the CompassReward framework [33], which maps continuous camera pose changes to discrete action labels via directional thresholding. We extend it with a latent-stride mechanism (CompassReward v3) designed to match the generation granularity of modern interactive world models.

The key insight of v3 is that when relative poses are computed over K=4 frames, the accumulated displacement is ∼K times larger than the single-frame displacement. Without scaling, v2 thresholds designed for single-frame magnitudes are too small for 4-frame windows, causing drift noise in stationary segments to exceed the threshold and trigger 21

Algorithm 1: D ISCRETIZE L ATENT Input: ρ ∈ SE(3) τmove ∈ R+ τrot ∈ R+ τaxis ∈ R+ Output: a ∈ A

Algorithm 2: P REDICT S EQUENCE

(single-latent decoder)

Input: T = (T0 , . . . , TM ) G = (g1 , . . . , gN ) K=4 Θ = {0.002, 0.005, 0.01} 0 τrot = 0.2◦ Output: Â = (â1 , . . . , âN ) τ ∗, π

▷ relative camera pose ▷ overall translation threshold ▷ rotation threshold (degrees) ▷ per-axis noise floor ▷ discrete action string

1: ∆t ← ρ[: 3, 3]; ν ← ∥∆t∥2 2: ot ← (0, 0, 0, 0) ▷ (fwd, bwd, right, left) 3: if ν > τmove then 4: n̂ ← clip(∆t/ν, −1, 1) 5: θ ← arccos(n̂) · 180/π ▷ angle to each axis 6: if θz < 60◦ ∧ |∆tz | > τaxis : ot [1] ← 1 ▷ forward 7: if θz > 120◦ ∧ |∆tz | > τaxis : ot [2] ← 1 ▷ backward 8: if θx < 60◦ ∧ |∆tx | > τaxis : ot [3] ← 1 ▷ right 9: if θx > 120◦ ∧ |∆tx | > τaxis : ot [4] ← 1 ▷ left 10: end if p 2 2 + R10 11: R ← ρ[: 3, : 3]; sy ← R00 12: (ϕ, ψ) ← EulerPitchYaw(R, sy ) ▷ gimbal-safe 13: or ← (0, 0, 0, 0) ▷ (yaw+, yaw−, pitch+, pitch−) 14: if ψ > +τrot : or [1] ← 1 else if ψ < −τrot : or [2] ← 1 15: if ϕ > +τrot : or [3] ← 1 else if ϕ < −τrot : or [4] ← 1 16: ℓt ← OneHotToLabel(ot ); ℓr ← OneHotToLabel(or ) 17: P ← L TransParts(ℓt ) ∪ RotateParts(ℓr ) 18: a ← p∈sort(P ) p (⊕ = “+”); a ← no op if P = ∅ 19: return a

0 1: τrot ← τrot ·K ▷ scale rotation threshold 2: τaxis ← (min Θ)/2 · K ▷ per-axis noise floor 3: C ← min(⌊(M −1)/K⌋, ⌈N/K⌉) ▷ number of latents 4: for c = 0, . . . , C−1 do 5: s ← c · K; e ← min(s+K, M ) 6: ρc ← Ts−1 · Te ▷ latent relative pose 7: end for 8: (Â∗ , α∗ ) ← (∅, −∞) 9: for each τ ∈ Θ do 10: τmove ← τ · K ▷ scale translation threshold 11: for c = 0, . . . , C−1 do 12: ac ← D ISCRETIZE L ATENT(ρc , τmove , τrot , τaxis ) 13: end for 14: Â ← Tile(⟨ac ⟩C−1 ▷ c=0 , K)∥Pad(aC−1 , N −C·K) broadcast PN 15: α ← N1 ▷ exact-match accuracy i=1 [[âi = gi ]] 16: π[τ ] ← α 17: if α > α∗ : (Â∗ , α∗ , τ ∗ ) ← (Â, α, τ ·K) 18: end for 19: Accstrict ← α∗ P N ∗ 20: Accpartial ← N1 i=1 [[P(âi ) ∩ P(gi ) ̸= ∅]] ∗ ∗ 21: return Â , τ , π

Figure 13. Latent-stride single-frame discretization.

false positives. V3 corrects this by multiplying all thresholds by K.

Figure 14. Full per-frame action prediction pipeline with threshold sweep.

C.3. Per-axis Noise Guard Even when the overall displacement ∥∆t∥ exceeds τmove , a directional component (e.g., left/right) is activated only if the corresponding axis magnitude also exceeds the noise floor: |∆tx | > τaxis

(lateral),

(23)

|∆tz | > τaxis

(forward/backward).

(24)

(full pipeline, stride-K)

▷ ViPE camera trajectory ▷ GT discrete actions ▷ latent stride ▷ base move thresholds ▷ base rotation threshold ▷ per-frame predictions ▷ best threshold & sweep table

computed, and the threshold achieving the highest accuracy is retained. This per-video sweep accommodates varying motion scales across different world models and scene types.

D. TrajScore: Detailed Computation

This prevents a primarily forward motion with tiny lateral drift from being misclassified as forward+right.

This appendix provides a step-by-step walkthrough of the TrajScore computation using an example from HYWorld 1.5 (game easy, sample 0000: 300 frames of sustained forward).

C.4. Discretization Algorithm We present the full discretization procedure in two algorithms. Algorithm 13 decodes a single latent relative pose into a discrete action label; Algorithm 14 orchestrates the end-to-end pipeline with threshold sweep.

D.1. Input Data ViPE trajectory: 301 camera-to-world poses T̂ = (T̂0 , . . . , T̂300 ) ∈ SE(3)301 . GT actions: 300 frames, all forward (i.e., gt = forward ∀t). Switch detection: 0 switches ⇒ 1 segment [(0, 300, forward)].

C.5. Threshold Sweep For each video, all base thresholds in Θ = {0.002, 0.005, 0.01} (each scaled by K) are evaluated, the exact-match accuracy against the GT action sequence is 22

D.2. Predicted Trajectory Characteristics

The goal is to evaluate trajectory shape and direction, not whether the model moves at the correct speed at each instant. Arc-length resampling achieves this goal.

The estimated trajectory shows predominantly forward motion along the z-axis with minor lateral drift:

Total path length: Lpath = 6.603. Total rotation: Φtotal = 2.72◦ .

Procedure. Both predicted and GT trajectories (300 frames each) are resampled to Npts = 60 equidistant points along their respective arc lengths. Positions use linear interpolation on cumulative arc length, and rotations use Slerp (spherical linear interpolation). After resampling, the i-th point on both trajectories corresponds to the same fraction of total path traversed (0%, 1.7%, 3.4%, . . . , 100%), regardless of the original per-frame speed profile.

D.3. Adaptive GT Construction

D.5. Error Computation

Core idea. The GT trajectory does not specify how far the model should travel; that distance is an intrinsic property of each model’s motion magnitude. Instead, the GT answers the question: “Given that the model moved this far, what is the ideal trajectory shape?” For a forward action the ideal shape is a perfectly straight line; for a look left action the ideal shape is pure rotation in place (first-person) or a circular arc (third-person); for a compound action (e.g., forward+look right) it is a smoothly curved path combining both. By borrowing the predicted displacement or rotation magnitude while imposing the ideal direction and curvature, the GT adapts to each video’s scale while remaining a faithful reference for shape evaluation.

At each of the 60 resampled points, two errors are computed. (i) (i) Position error: et = ∥p̂(i) − pGT ∥. (i) (i) Rotation error: er = ∠(R̂(i) , RGT ) in degrees. Representative errors (this example):

T̂0 .t = (−0.001, −0.000, −0.000),

(25)

T̂150 .t = (−0.033, −0.054, 3.051),

(26)

T̂300 .t = (0.002, −0.245, 6.509).

(27)

Point

er (deg)

0 10 19

0.000 0.077 0.245

0.00 0.38 0.89

The errors accumulate monotonically: the model drifts slightly downward (y-axis) as it moves forward.

D.6. Normalization and Final Score

Construction steps (pure translation example). For a pure translation action (forward), the GT is constructed as follows. Direction extraction. Extract the world-frame direction from the initial pose: d = R0 · (0, 0, 1)⊤ ≈ (0, 0, 1). Displacement matching. Set GT displacement equal to predicted displacement: ∆GT = ∥T̂300 .t − T̂0 .t∥ = 6.513. Linear interpolation. Interpolate the GT positions as t GTt .t = T̂0 .t + 299 · ∆GT · d. This yields an ideal straight-line trajectory with the same start/end displacement as the prediction but no lateral drift.

Why normalize? The raw ATEt = 0.090 is not meaningful in isolation: if the model traversed 6.6 units in total, a 0.09-unit deviation represents only 1.4% drift; if the model moved only 0.3 units, the same 0.09 deviation would represent 30% drift. Different world models produce vastly different motion magnitudes for the same action—some move 2 units per 300 frames, others move 10. Without normalization, a model with large motion would appear worse simply because its absolute errors are proportionally larger, even if its trajectory shape is equally accurate. Dividing by the model’s own total path length or total rotation converts ATE into a relative deviation ratio, enabling fair cross-model comparison regardless of motion scale. The minimum denominators (Lmin = 0.5 for translation, ϕmin = 10◦ for rotation) prevent amplification when the predicted motion is near zero: a model that barely moves should not receive an artificially poor score due to division by a tiny denominator.

D.4. Arc-length Resampling Why not compare frame-by-frame? Both the predicted and GT trajectories have 300 frames, so a naive approach would compare them point by point at each frame index. This is problematic for two reasons. Non-uniform speed. The model may move quickly in the first half and slowly in the second half. Under frameindexed comparison, even if the trajectory shape is identical to the GT, speed variation alone produces large apparent errors at intermediate frames. Frame-rate dependence. Different models generate at different frame rates (16, 24, or 30 fps). Frame index 100 corresponds to different physical moments across models, making cross-model comparison ill-defined.

Computation. 59

ATEt =

1 X (i) e = 0.090, 60 i=0 t

ATEr =

1 X (i) e = 0.41◦ . 60 i=0 r

(28)

(29)

Normalization: 0.090 = 0.014, max(6.603, 0.5) 0.41 = 0.041. nATEr = max(2.72, 10.0) nATEt =

(30) A

Predicted Videos

Note: Φtotal = 2.72◦ < ϕmin = 10◦ , so the minimum denominator is used to prevent amplification of small rotation errors. Final score: TrajScore = 1 −

0.014 + 0.041 = 0.973. 2

(31) GT Actions

Validity Gate Checking FPV: RAFT-based

(32)

TPV: VFM-Based

Filter-Out Videos not Following the Actions

D.7. Interpretation Phenomenon Checking

A TrajScore of 0.973 indicates near-perfect trajectory alignment: the model moves in the correct direction with the correct shape (straight line), with only minor lateral drift (1.4% of path length) and negligible heading error. This is consistent with HY-World 1.5’s strong forward-motion capability on easy (single-action) sequences.

Multi-Turn VLM-based Checking Check Whether Obey the Rule or Not

Figure 15. Detailed interaction-physics evaluation pipeline. The full pipeline includes action-responsiveness validation, mechanics protocols, optics protocols, 3D-consistency evaluation, Qwen3-VL-PLUS-based VLM scoring, fallback queries, and domain-level aggregation.

D.8. Multi-segment Example For harder sequences with switches (e.g., forward → look left → backward), the same procedure is applied segment-wise. Segment allocation. Each segment receives 60/Nseg resampled points (e.g., 20 points per segment for 3 segments). Segment-specific GT construction. GT construction varies by segment type: straight line for translation, ideal arc for rotation, curved path for compound actions. Concatenated evaluation. All resampled points are concatenated before computing the overall ATE. This ensures that (i) the total number of evaluation points is constant across difficulty levels, and (ii) switch boundaries—where models typically incur the largest errors—are captured within the evaluation.

execute the prescribed action rather than remaining static or unresponsive. This prevents action-following failures from being conflated with physical-reasoning failures. Invalid cases receive a score of −1 and are excluded from aggregate physics statistics. FPV validity. For FPV videos, we use an optical-flow pre-filter to estimate the dominant camera motion. Dense Farneback flow [41] is computed over uniformly sampled frame pairs, and the dominant flow direction is compared with the dominant action key in the prescribed action sequence. A case is marked invalid when the predicted motion direction disagrees with the expected action.

E. Interaction Physics Evaluation Details

TPV validity. For TPV videos, optical flow cannot reliably distinguish camera motion from subject motion. We therefore adopt a subject-centric validity check based on SAM 2 [40]. Given a subject mask video, we extract a bounding box Bt for each frame and compute the IoU between each subsequent box and the first-frame reference box: |B1 ∩ Bt | IoUt = , t = 2, . . . , T. (33) |B1 ∪ Bt |

This appendix provides the implementation details omitted from the main interaction-physics section. The high-level goal remains to evaluate whether a generated rollout produces physically plausible consequences under user control. We first apply a validity gate to remove static or uncontrolled generations, and then evaluate valid rollouts along three complementary domains: mechanics, optics, and 3D consistency. Unless otherwise noted, all VLM queries in this appendix are instantiated with Qwen3-VL-PLUS. Figure 15 shows the complete implementation pipeline.

The control rate is the fraction of frames whose IoU exceeds a threshold τiou :

E.1. Validity Gate Details Motivation. Before physics is evaluated, the generated video must pass a validity gate: the subject or camera must

Rctrl =

T 1 X 1[IoUt ≥ τiou ]. T − 1 t=2

(34)

A video passes validity when Rctrl ≥ τctrl . We set τiou = 0.4 and τctrl = 0.9 throughout. When no SAM mask video is available, we use a lightweight Qwen3-VL-PLUS VLM query over three uniformly sampled frames to check whether the subject visibly moves in the expected direction.

Realtime mode covers transient responses such as water ripples and wheat parting. Frames are sampled from the 30– 80% range, where the subject is actively interacting with the environment. A fallback query uses denser sampling around the 40–70% interaction window. Both modes use 768 px resolution to preserve finegrained surface details that are often lost at lower resolutions. This is especially important for subtle effects such as water ripples, crop displacement, and weak ground traces.

E.2. Physics Protocol Details E.2.1. Mechanics Details Mechanics is evaluated through five protocols: collision, clipping, deformation, terrain following, and gravity. Unless otherwise specified, each protocol returns a binary score, where 1 indicates a physically plausible outcome and 0 indicates failure.

Terrain following. Terrain following checks whether the subject’s vertical position conforms to the terrain geometry, such as ascending stairs, descending slopes, or walking over uneven ground, rather than floating above or sinking into the surface. We sample N = 8 frames and query the Qwen3VL-PLUS VLM about ground contact and height consistency. An optional boundary check compares the first and last frames to verify that the subject remains within walkable path boundaries, such as between railings. For FPV rollouts, we compare mid-video and end-video frames against the first frame to detect unnatural viewpoint displacement. For flat-then-slope trajectories, we use denser sampling across mid and late segments; if at least two sampled frames are judged unnatural, the case is marked as failed.

Collision. Collision tests whether a target object responds physically when the subject reaches it, such as a box being displaced after being kicked. The target object identity and expected response are extracted from the test-case metadata. We uniformly sample N = 10 frames at 512 px resolution and present them to the Qwen3-VL-PLUS VLM together with the scene description. Instead of requiring the VLM to observe the exact instant of contact, the prompt asks it to compare the position and arrangement of the target object between early and late frames. This outcome-oriented formulation is more robust to temporal aliasing caused by sparse frame sampling. For FPV rollouts, we use a two-step sliding-window strategy. The first step detects the approach phase through overlapping windows, and the second step verifies the collision response in post-approach windows.

Gravity. Gravity evaluates whether unsupported objects fall, whether the subject sinks upon entering water, and whether projectiles follow plausible arcs. We densely sample N = 12 frames and construct scenario-specific prompts from the scene description and, when available, the trigger question in the test-case metadata. The gravity scenario, such as object falls, sinks in water, or projectile arc, is inferred from the case description and used to select the corresponding prompt template.

Clipping. Clipping detects non-physical pass-through, where the subject traverses a solid obstacle such as a wall or railing without being blocked. Clipping is scored inversely: detecting pass-through yields a score of 0. For TPV rollouts, we sample N = 10 frames and query the Qwen3-VL-PLUS VLM for body–obstacle overlap. A second pass compares the first and last frames to check whether the subject has moved to the opposite side of the obstacle without circumnavigating it. For FPV rollouts, we use the same sliding-window strategy as collision, augmented with a first-versus-last-frame location check and an optional barrier-side consistency test.

E.2.2. Optics Details Optics is evaluated through two protocols: reflection and occlusion, where occlusion is instantiated as shadow correctness. These protocols differ in score granularity. Reflection. Reflection uses a two-tier evaluation. Tier 1 is a coarse global check. We uniformly sample N = 12 frames at 384 px resolution and ask the Qwen3-VL-PLUS VLM whether reflections are globally plausible: consistent in shape, approximately symmetric about the reflecting surface, and changing naturally with camera or subject motion. A failure at Tier 1 yields a score of 0. If Tier 1 passes, Tier 2 performs fine-grained per-frame scoring. We sample N = 8 frames and pair each sampled frame with the first frame from the test case as reference. The Qwen3-VL-PLUS VLM assigns each frame an integer score:

Deformation. Deformation evaluates whether the environment exhibits physically plausible surface response to the subject’s passage. We infer two modes from the scene description. Trace mode covers persistent traces such as footprints, tire tracks, and snow marks. The first frame is used as an undisturbed reference and compared with frames from the latter 50–100% of the video. If the first query returns a negative result, a fallback query re-examines frames from the 30– 80% range with a broader surface-change question.

st ∈ {−1, 0, 1, 2, N/A}, 25

(35)

where 2 denotes a physically correct reflection, 1 indicates partial errors, 0 means the reflecting surface is visible but the reflection is missing or wrong, −1 marks a collapsed scene, and N/A indicates that the reflecting surface has left the field of view. Frames marked −1 or N/A are excluded. The reflection score is Srefl =

1 X st , 2|V|

V = {t : st ≥ 0},

Adaptive resolution. Protocols that target fine-grained surface effects, such as deformation, use 768 px resolution. Coarse geometric checks, such as collision and clipping, use 512 px resolution. Multi-frame overview queries, such as reflection Tier 1 and shadow correctness, use 384 px resolution. Structured fallback. When the primary Qwen3-VLPLUS VLM query returns a negative judgment, a second query with a different frame range or sampling density is issued before committing to a failure score. This reduces sensitivity to unfavorable frame selection and improves agreement with human judgments.

(36)

t∈V

yielding a continuous score in [0, 1]. Occlusion and shadow correctness. The occlusion protocol evaluates whether shadows appear and behave consistently with the scene lighting. We sample N = 10 frames from the 10–100% range of the video. This wider window is used because TPV cameras often reveal the subject’s shadow from the beginning. For FPV videos, the first frame is prepended as a reference. The Qwen3-VL-PLUS VLM judges whether the expected shadow is present and whether its direction and shape are physically consistent with the light source. Scoring is binary.

E.4. Physics Score Details Per-case scores are aggregated into a physics composite. Let Cmech , Copt , and C3d denote the sets of valid mechanics, optics, and 3D-consistency cases, respectively. The domain-level scores are Smech =

|Cmech |

sc ,

Sopt =

c∈Cmech

1 |Copt |

sc ,

c∈Copt

(37) with S3d defined analogously when C3d is available. Here, sc ∈ [0, 1] is the per-case score. The overall physics score averages all available physics domains:

E.2.3. 3D Consistency Details 3D consistency evaluates whether the generated scene preserves rigid spatial structure under controlled camera motion. In the current benchmark, these cases are primarily instantiated in FPV settings with long, unobstructed trajectories and a constant input rate. This design makes geometric distortions, inconsistent depth ordering, and non-rigid scene deformation easier to detect without confounding effects from collisions, deformation, or terrain changes. The resulting per-case scores are treated as a third physics domain when available.

Sphysics =

1 X Sd , |D|

D ⊆ {mech, opt, 3d}. (38)

d∈D

The available domains are weighted equally because they test complementary aspects of physical plausibility: mechanics evaluates interaction-level consequences, optics evaluates light-level consistency, and 3D consistency evaluates spatial coherence.

E.3. Prompt Design Principles E.5. Collision Evaluation Details

Across all protocols, prompt construction follows four design principles derived from iterative calibration against human judgments.

Two-stage sliding-window verification. Collision evaluation employs a two-stage, multi-turn Qwen3-VL-PLUS VLM dialogue over a sliding window of frames. Rather than asking a single binary question over the whole video, we first localize the moment of approach and then verify the physical plausibility of the collision response in the localized neighborhood. This decomposition suppresses false positives from videos where the subject never reaches the target, and false negatives where the collision response is brief and easily missed in a uniform sample.

Scene-grounded context. Each prompt begins with the scene description from the test-case metadata, giving the Qwen3-VL-PLUS VLM the expected scenario without imposing overly rigid subject assumptions, such as always assuming the subject is a person rather than a vehicle or animal. Outcome-oriented questions. Prompts ask about observable outcomes across sampled frames rather than requiring a specific instantaneous event. For example, instead of asking whether a foot contacts a ball at a particular frame, the prompt asks whether the ball’s position changes after interaction. This design is more robust to temporal aliasing in sparse frame samples.

Sliding-window approach detection (Turn 1). We extract sliding windows of W =6 frames at stride S=3 with sampling rate fs =2 fps. For each window Wk , the Qwen3VL-PLUS VLM is asked whether the camera is getting noticeably closer to the target object o (inferred from the case description or specified in eval protocol args). A 26

window is flagged as an approach window if it answers affirmatively. If no approach window is found, the case is scored 0 without invoking the second turn.

cases where the pass-through is gradual and missed by the sliding window. Stage 4: Barrier side check. For cases with a configured barrier side check, the Qwen3-VL-PLUS VLM verifies whether a barrier (e.g., a railing) has switched from one side of the frame to the other—a signature of having passed through it.

Collision response verification (Turn 2). Starting from the first approach window, we query the Qwen3-VL-PLUS VLM on each window from that point onward (up to two windows past the last approach) with a targeted question: after contact, does o show a physically plausible displacement (sliding, rolling, tumbling, being knocked aside) and remain visible in the scene? An early-exit policy returns a pass on the first affirmative answer. This design penalizes two common failure modes: targets that vanish on contact (no collision, just disappeared) and targets that the subject walks through without any response.

Stage 5: Per-case custom check. An optional vlm check prompt from eval protocol args provides case-specific clipping criteria that the generic prompts cannot cover. Scoring. score=0 if any stage detects clipping, else 1. Validity-gate failures yield score=−1.

E.8. Clipping Evaluation Prompt

Scoring. The case score is binary: score=1 if any Turn2 window passes, else 0. Cases that fail the validity gate (optical flow for FPV, SAM 2 mask tracking for TPV) are excluded with score=−1.

The clipping cascade uses staged prompts for approach detection, pass-through verification, first-vs-last-frame checking, and barrier-side consistency. The full prompt set is shown in Figure 22.

E.6. Collision Evaluation Prompt

E.9. Deformation Evaluation Details

The collision protocol uses two sequential prompts over localized sliding windows. Both prompts share a common system preamble specifying the video duration and sampled timestamps, and both require the Qwen3-VL-PLUS VLM to conclude with a single Answer: yes or Answer: no line after step-by-step reasoning. The full collision prompts are shown in Figure 20.

Temporal sampling strategy. Environmental deformation (footprints, tire tracks, water ripples, wheat parting) requires sampling at the right moment. The protocol distinguishes two modes: • Trace mode (default): the deformation is left behind the subject. Frames are sampled from the last 33% of the video, when the camera has retreated and can observe the traces. For TPV, the first frame is included as a reference for before/after comparison. • Real-time mode: the deformation occurs at the interaction point (water ripples, wheat parting). Frames are sampled from 30–80% of the video, capturing the subject mid-interaction. Mode selection is automatic: the description is scanned for keywords (ripple, splash, wheat, parting) to trigger real-time mode.

E.7. Clipping Evaluation Details Multi-stage clipping detection. Clipping (phasing through solid obstacles) is evaluated through a five-stage cascade. The protocol inverts the usual pass/fail convention: detecting clipping yields a score of 0 (physics violation), while its absence yields 1. Each stage acts as an increasingly specialized detector, and the cascade short-circuits as soon as clipping is confirmed. Stage 1: Sliding-window approach detection. Identical to collision Turn 1: a sliding window (W =6, S=3, fs =2 fps) localizes windows where the viewer approaches an obstacle o (wall, railing, fence, etc., inferred from the description). Only windows flagged as “approach” proceed to Stage 2.

Reference-frame comparison (TPV trace mode). In TPV trace mode, the Qwen3-VL-PLUS VLM receives the first frame (undisturbed ground) alongside later frames and is asked to compare: has the ground changed in any way? The prompt explicitly instructs leniency: “even faint, partial, or subtle changes count as YES.”

Stage 2: Pass-through verification. For each window from the first approach onward, the Qwen3-VL-PLUS VLM checks whether the view appears to pass through o— the obstacle suddenly disappears, the view shows a wall interior, or the scene transitions to what should be behind the obstacle without any opening. Early-exit on first detection.

Retry with denser sampling. If the primary check returns “no” or “unclear,” a second Qwen3-VL-PLUS VLM call samples a different temporal segment (earlier for FPV, denser around the interaction point for TPV real-time mode). The retry’s affirmative answer overrides the primary negative.

Stage 3: First-vs-last frame trigger. If Stages 1–2 find no clipping, a fail-safe compares the first and last frame: has the viewer ended up on the other side of o? This catches

Scoring. Binary: score=1 if deformation is detected in either pass, else 0. 27

E.10. Deformation Evaluation Prompt

Three failure modes. The naturalness check tests for three specific anomalies: • FLOATING: viewpoint is unnaturally high, hovering above the ground. • GROUND-LEVEL: viewpoint is unnaturally low—at ankle/ground height instead of standing eye-height. • CLIPPING: viewpoint has sunk into the ground, showing impossible views through surfaces. Crucially, terrain elevation changes (uphill, stairs, downhill) are explicitly accepted as natural.

The deformation prompts cover first-person trace detection, third-person reference-frame comparison, and third-person real-time interaction checks, as shown in Figure 21.

E.11. Gravity Evaluation Details Scenario-driven prompting. Gravity evaluation is scenario-aware: the protocol infers one of five scenarios from the case description and triggers—object falls, sinks in water, projectile arc, no floating, or a generic gravity effect—and selects a scenariospecific prompt. Each prompt describes the expected physical evidence (e.g., “accelerating downward due to gravity” for falling objects, “parabolic arc” for projectiles) and counter-evidence (“float, hover, or fall unnaturally slowly”).

Trajectory-only mode. For continuous rise/continuous down cases where the naturalness check would penalize the expected height change, the protocol skips naturalness and relies solely on a per-case vlm check prompt. Optional boundary check. For narrow walkway cases (railings, edges), a boundary check verifies that the viewpoint has not drifted sideways beyond the walkable surface.

Uniform frame sampling. The Qwen3-VL-PLUS VLM receives N =12 uniformly sampled frames (configurable via eval protocol args.num frames) covering the full video duration.

Scoring. Binary: score=1 if all naturalness checks pass (and boundary/vlm checks pass, if enabled), else 0.

Optional dense confirmation. When a case specifies dense check, a positive first-pass verdict triggers a second Qwen3-VL-PLUS VLM call at ∼1 fps sampling density with a confirmation prompt. If the dense check disagrees (answers “no”), the score flips from 1 to 0. This suppresses false positives from sparse sampling that misses a brief anti-gravity artifact.

E.14. Terrain Following Evaluation Prompt The terrain-following prompts cover naturalness checks, third-person ground-contact checks, and optional boundary checks, as shown in Figure 24.

E.15. Reflection Evaluation Details

Per-case custom prompt. Cases may override the scenario-based prompt entirely via eval protocol args.vlm check, providing a bespoke question grounded in the case description.

Two-tier coarse-to-fine evaluation. Reflection is the only protocol producing a continuous score. The evaluation proceeds in two tiers: • Tier 1 (coarse): 12 uniformly sampled frames are presented to the Qwen3-VL-PLUS VLM in a single call. It judges global reflection plausibility: are reflections consistent in shape, approximately symmetric about the reflecting surface, and changing naturally with camera movement? A “no” verdict short-circuits to a score of 0. • Tier 2 (fine): 8 frames are sampled and each is scored independently (parallelized with 4 workers) against the first-frame reference. The Qwen3-VL-PLUS VLM assigns a per-frame score of −1 (invalid), 0 (missing/wrong), 1 (physics errors), 2 (correct), or N/A (surface not visible).

Scoring. Binary: score=1 if gravity is correctly simulated (and dense check confirms, if enabled), else 0.

E.12. Gravity Evaluation Prompt The gravity prompts are selected according to the inferred physical scenario and include the third-person general gravity check; the full set is shown in Figure 23.

E.13. Terrain Following Evaluation Details Strategy selection by trajectory type. The protocol selects between two evaluation strategies based on expected trajectory: • Strategy A (simple): for flat, continuous rise, continuous down, rise then fall. Samples 3 frames (first, mid, last) and checks each against the first frame for naturalness. • Strategy B (multi-frame): for flat then rise, flat then drop. Samples the first frame plus 3 midrange (30–60%) and 3 post-transition (65–95%) frames. Requires ≥2 “unnatural” verdicts to fail, allowing one borderline frame.

Per-frame scoring rubric. The Tier 2 prompt defines a 5-level rubric with explicit guidance on the N/A label: if the reflecting surface has moved out of view, the frame is scored N/A (excluded from averaging) rather than 0 (surface visible but reflection wrong). This prevents penalizing videos where the camera legitimately moves away from the reflective surface. 28

Aggregation. Srefl =

E.18. Occlusion / Shadow Evaluation Prompt

The final score is:

1 X si , 2 · |V|

V = {i : si ≥ 0},

The occlusion/shadow prompts include expected-shadow checks, third-person fallback checks, and a generic fallback for scenes without explicit shadow metadata, as shown in Figure 26.

(39)

i∈V

where si ∈ {0, 1, 2} are valid per-frame scores (invalid −1 and N/A frames excluded). The result is a continuous value in [0, 1].

F. Memory Evaluation Details This appendix provides the implementation details omitted from the main memory section. The high-level goal remains to decouple memory measurement from imperfect action execution: scene memory aggregates geometry over observation and revisit segments, while subject memory evaluates whether the protagonist is preserved in controllable TPV rollouts. Figure 16 shows the complete implementation pipeline.

Surface-aware prompting. The reflecting surface type (water puddle, water, canal, mirror, glass) is mapped to a human-readable description and injected into both Tier 1 and Tier 2 prompts, ensuring the Qwen3-VLPLUS VLM evaluates against the correct physical expectations (e.g., horizontal flip for water vs. planar reflection for mirror).

E.16. Reflection Evaluation Prompt

F.1. Scene Memory Details

The reflection prompts implement the coarse global plausibility check and the per-frame scoring rubric; both are shown in Figure 25.

Action-aware transition localization. We localize the observation–revisit transition from the generated video rather than using the symmetric GT index. Given the video and its GT action sequence, WorldCompass first estimates a frame-level action for every generated frame. Since the estimator may output compound actions, such as W+L, even when the GT action at that time is a single primitive such as W, we convert predictions into action pseudo-labels in the GT action space. Specifically, if the predicted action contains the current GT action as a partial component, the frame is assigned that GT action as its pseudo-label. The resulting pseudo-label sequence is then temporally smoothed, and the transition point t∗ is identified as the first stable change point from the observation action to the revisit action. This procedure aligns the memory split with the model’s executed action transition and reduces contamination from compound or noisy action predictions.

E.17. Occlusion / Shadow Evaluation Details Temporal sampling with reference frame. Shadow evaluation samples frames from the latter portion of the video (40–100% for FPV, 10–100% for TPV), where the shadow “script” should have occurred. For FPV, the first frame is prepended as a visual reference, allowing the Qwen3-VL-PLUS VLM to compare the initial lighting/shadow state against later frames. Strict shadow recognition criteria. The prompt enforces strict criteria to distinguish real shadows from ambient darkening: • A real shadow must have a recognizable silhouette matching the casting object (e.g., a person shadow looks like a person outline). • The shadow must have a clear boundary separating shadowed from lit regions—vague dark patches, ambient shading, surface texture, or lighting gradients do not count. • The shadow direction must be physically consistent with visible light sources. The Qwen3-VL-PLUS VLM is instructed to answer “yes” only if confident the shadow is real and correct. Expected-shadow injection. When eval protocol args.expected shadow is provided (e.g., “wall shadow,” “person shadow on the ground”), it is injected into the prompt to specify what the Qwen3-VL-PLUS VLM should look for. Without it, a generic shadow detection prompt is used.

Point-cloud reconstruction and preprocessing. From the observation segment {It }t≤t∗ and revisit segment {It }t>t∗ , we reconstruct point clouds Pobs and Prev using monocular depth estimates and known camera intrinsics. To reduce depth-estimation noise, particularly in distant and textureless regions, we apply three preprocessing steps. Voxel-based deduplication. We average points within each voxel of a fixed grid, reducing redundancy while preserving scene structure. Semantic filtering. We use a pretrained SegFormer to remove categories with unreliable depth, such as sky, water, and reflective surfaces. Depth-percentile filtering. We retain points within the nearest p-th depth percentile, discarding distant regions with high depth uncertainty.

Scoring. Binary: score=1 if the expected shadow is present and behaves correctly, else 0.

Cross-segment point-cloud registration. The coordinates of Pobs and Prev are determined by the estimated 29

(Explore)

⋯

(Revisit)

S W S S

J S

W S W S

Frame-Level Point Clouds

1) Voxel-based Point-Cloud Deduplication

Frame-Level Point Clouds

2) Semantic Filtering

3) Depth-percentile Filtering

Remove sky, water, reflective surfaces

retain nearest p%

Cross-Segment Registration

Far

Coarse: PTY3 / FPFH + RANSAC

Fine: point-to-plane ICP

S S

Keep

Quality-aware acceptance: fitness ≥ τᵩ and Chamfer improves

Near

Remove

Subject Memory Evaluation (TPV / VLM)

Scene Memory Evaluation (Point-level)

Third-person video (tracked)

Retained (Obs ∩ Rev) Holistic VLM scoring

Point-Cloud Reconstruction, Preprocessing, and Registration first sustained polarity change

⋯

sliding-window smoothing

W L

Action Pseudo-Labeling

Action estimation

Video + GT Actions

Action-aware Transition Localization W

Revisit Segment (t > t*)

Observation Segment (t ≤ t*)

Memory Evaluation Pipeline

Forgotten (Obs − Rev)

Hallucinated (Rev − Obs)

identity structure

Subject Memory Score

appearance

…

Scene Memory Score: 𝑺𝒎𝒆𝒎 = 𝑭𝟐 (𝑭𝒎𝒆𝒎 − 𝑹𝒎𝒆𝒎 )

Figure 16. Detailed memory evaluation pipeline. The full pipeline includes action-aware transition localization, point-cloud reconstruction, registration, filtering, scene-memory scoring, subject tracking, controllability gating, and Qwen3-VL-PLUS-based VLM subjectmemory evaluation.

camera-pose chain. Pose drift can introduce a rigid displacement unrelated to memory, artificially increasing both forgetting and hallucination rates. We therefore register Prev to the stationary reference Pobs using coarse-to-fine alignment. We estimate the transformation once from the full deduplicated clouds, before depth-percentile filtering, and apply it to all depth-filtered variants. Coarse registration. When the initial misalignment, measured as the maximum of the inter-centroid displacement and symmetric Chamfer distance, exceeds τc , we perform global registration. We compute per-point descriptors using Point Transformer V3 [39] features. Nearest-neighbor matching in descriptor space produces candidate correspondences, from which RANSAC estimates a geometrically consistent rigid transformation. Fine registration. Starting from the coarse estimate, or the identity transformation when global registration is unnecessary, we refine the alignment using point-to-plane Iterative Closest Point (ICP) [37]. Quality-aware acceptance. To reject degenerate registrations, we apply a dual acceptance criterion. Let T ∗ be the estimated transformation, ϕ(T ) the ICP fitness, and CD(T ) the symmetric Chamfer distance after applying T . We use ( ∗ T , if ϕ(T ∗ ) ≥ τϕ and CD(T ∗ ) ≤ CD(I), T = I, otherwise, (40) where τϕ is the minimum acceptable fitness and I is the 4×4

identity. Thus, registration is accepted only if it achieves sufficient fitness without increasing the Chamfer distance. All subsequent metrics are computed on the aligned pair (Pobs , T ◦ Prev ); for brevity, we use Prev to denote the aligned revisit cloud below. Geometric retention and hallucination. We quantify geometric memory using nearest-neighbor distances between the aligned point clouds. An observation point p ∈ Pobs is retained if its nearest revisit point lies within τd , and forgotten otherwise. Conversely, a revisit point q ∈ Prev is hallucinated if no observation point lies within τd . We set τd adaptively as a fixed fraction of the scene diagonal. The retention and hallucination rates are Rmem =

|{p ∈ Pobs : d(p, Prev ) < τd }| , |Pobs |

(41)

|{q ∈ Prev : d(q, Pobs ) ≥ τd }| . |Prev |

(42)

Rhall =

Here, Rmem measures the fraction of observed geometry recovered upon revisit, whereas Rhall measures the fraction of revisit geometry unsupported by the observation. To obtain a single scalar, we interpret Rmem as recall and define precision as Pmem = 1 − Rhall , the fraction of revisit geometry supported by the observation. The scene-memory score is the F1 score: Sscene = F1 = 30

2 Pmem Rmem . Pmem + Rmem

(43)

SVLM ∈ {1, . . . , 10} and binary diagnostic flags. The prompt is calibrated to avoid spurious penalties: viewpoint changes and illumination-induced shifts are treated as benign unless they produce unexplained contradictions; category preservation alone is insufficient when the subject’s structure collapses; and progressive degradation is penalized more heavily than an isolated artifact. The full prompt is provided in Appendix G. We linearly normalize the integer rating to [0, 1]:

We use F1 to balance memory recall and memory precision, so the score penalizes both forgotten observation geometry and unsupported revisit geometry.

F.2. Subject Memory Details Third-person controllability filtering. Before evaluating subject memory, we filter out test cases where the model fails to maintain third-person control. In successfully controlled cases, the subject remains near its first-frame position with limited size variation throughout the video. In failed cases, the camera degenerates to a first-person viewpoint, causing characteristic artifacts: the subject grows or shrinks under translation actions (W/S) and disappears then reappears under yaw rotation actions (J/L). To detect control failure, we use SAM 2 [40] to track and segment the subject at every 20 frames, extracting a bounding box Bt for each sampled frame. We compute the IoU between each frame’s bounding box and the first-frame reference box B1 : IoUt =

|B1 ∩ Bt | , |B1 ∪ Bt |

t ∈ S.

Ssub =

G. Subject Memory Prompt

(44)

For third-person memory evaluation, the Subject Memory Evaluation Prompt in Figure 27 scores video-level subject memory from temporally ordered subject crops. The prompt emphasizes holistic temporal judgment, separates viewpoint and lighting changes from true identity drift, and explicitly penalizes structural collapse, disappearance, and severe quality degradation.

Failure modes. We characterize subject memory failure using five failure modes and one benign factor: identity change, structural distortion, appearance drift, subject disappearance, quality degradation, and benign viewpoint change. The evaluator must distinguish true subjectmemory failure from benign variation caused by viewpoint, illumination, or shadow.

H. Why Not Frame-pair Memory? A natural alternative to scene-level memory is to directly compare observation and revisit frames at corresponding spatial locations. We instantiate this frame-pair approach by first reusing the action-aware transition localization step from Section 3.5.1 to obtain the transition frame t∗ , and then constructing geometrically corresponding pairs around t∗ via pose-aware frame pair construction, described below. Even with t∗ given by the same procedure that anchors our scene-level pipeline, frame-pair evaluation remains insufficient for the reasons we explain at the end of this appendix. Figure 17 visualizes a representative failure case.

Sampling and subject extraction. Given a generated third-person video V = {I1 , I2 , . . . , IT }, we sample frames uniformly with stride ∆ = 20, yielding indices S = {t1 , t2 , . . . , tN }, where t1 = 1. A human annotator specifies a small set of point prompts Q = {q1 , . . . , qL } on I1 . SAM 2 [40] propagates these prompts through the video to produce a subject mask Mt for each frame. We extract each sampled subject observation as ti ∈ S,

(46)

Thus, a rating of 1 maps to 0, and a rating of 10 maps to 1. Holistic scoring accommodates smooth viewpoint and illumination changes that can confound frame-level comparisons, while producing a single benchmark-compatible scalar without requiring an external reference-feature library.

A video is deemed to have achieved third-person control only if at least 90% of sampled frames maintain an IoU above the acceptance threshold τiou . Cases that fail this gate are excluded from the subject memory evaluation, ensuring that the subject memory score reflects genuine identity preservation rather than artifacts of control failure.

Oti = Iti ⊙ Mti ,

SVLM − 1 . 9

Pose-aware frame pair construction. Rather than pairing frames by symmetric index around t∗ , camera poses estimated by ViPE are used to identify geometrically corresponding revisit frames. For each observation frame fi , let fˆi denote its temporally symmetric counterpart with respect to t∗ . The revisit frame is selected by minimizing the geodesic distance to the pose of fi within a local temporal window:

(45)

where ⊙ denotes element-wise masking. Holistic VLM evaluation. We present the temporally ordered sequence O = {Ot1 , . . . , OtN } to the Qwen3-VLPLUS VLM, optionally as a tiled grid. It evaluates the sequence holistically rather than through pairwise frame comparisons, producing an integer subject memory score

fi∗ = arg

min

fj ∈N (fˆi )

d P (fi ), P (fj ) ,

(47)

Input:

Excuted:

Input:

x10

Excuted:

Trajectory deviation!

x10

Control & Pose noise!

Input:

Inconsistent action magnitudes!

Excuted:

x10

Input:

Excuted:

❌

between the observation and revisit segments that no temporal re-indexing can correct. Pose-aware pairing alleviates this issue to some degree, but the search is limited to a local neighborhood and cannot compensate for large cumulative offsets. Trajectory deviation from spurious motion. Certain models exhibit unprompted camera movements that cause the generated trajectory to diverge substantially from the prescribed path. In such cases, the revisit segment may explore entirely different regions of the scene, rendering frame-level comparison unreliable regardless of the pairing strategy. Compounding of control jitter and pose estimation noise. For closed-source models accessed through web interfaces, network latency introduces temporal uncertainty in action execution, while the pose estimator itself contributes additional noise. These two error sources compound: noisy poses lead to suboptimal frame pair selection, and control jitter ensures that even correctly selected pairs depict slightly different viewpoints, both inflating the measured dissimilarity. As a result, models that visibly preserve scene content upon human inspection can receive misleadingly poor frame-pair memory scores, not because they fail to remember, but because the evaluation framework conflates actionfollowing deficiency with memory degradation. For these reasons, WorldRoamBench does not adopt frame-pair memory and instead relies on the scene-level memory evaluation described in the main text, which aggregates geometric information across frames and is substantially more robust to per-frame action imprecision.

❌

x25

Excuted:

Figure 17. Failure mode of frame-pair memory evaluation. Frame-pair metrics assume that an observation frame and its revisit counterpart depict the same spatial location. In long-horizon interactive rollouts, however, imperfect action execution can shift the revisit trajectory, so the paired frames correspond to different viewpoints or even different scene regions. Image-level discrepancies in such pairs therefore reflect a mixture of memory degradation, action-following error, and pose-estimation noise, making frame-pair scores unreliable indicators of scene memory.

I. Action Gap in Closed-Source Models where P (·) denotes the estimated camera pose and d(·, ·) is a pose distance metric combining translational and rotational components. This construction offers two advantages: (i) evaluating memory on the most geometrically aligned pair minimizes contamination from incomplete revisits, and (ii) restricting the search to a neighborhood of the symmetric index preserves the trajectory prior and mitigates the effect of noisy pose estimates. Given the constructed pairs {(fi , fi∗ )}, standard imagelevel fidelity metrics are reported: MSE, PSNR, SSIM, and LPIPS.

Table 3 shows that Genie 3 and Happy Oyster trail the strongest open models in action-following scores despite their strong overall performance. We also present representative failure cases from both closed-source models to illustrate the underlying causes.

I.1. Genie 3: Lateral-Action Camera Drift When the A or D key is pressed to move the character laterally, Genie 3 interprets the input as a camera rotation command rather than producing a lateral translation. As shown in Fig. 18 (top), instead of the character shifting left or right within a stable scene, the entire camera view rotates, leaving the character centered but drastically altering the visible environment. This spurious rotation makes precise steering impossible and directly illustrates a case-level actionfollowing failure.

Limitations of frame-pair memory evaluation. Despite the above mitigations, frame-pair metrics remain unreliable indicators of memory capability under the action-following quality exhibited by current world models. Three fundamental issues persist. Systematic spatial offset from inconsistent action magnitudes. Even when the transition frame t∗ is correctly identified, forward and backward actions often traverse different distances. This creates a systematic spatial displacement

I.2. Happy Oyster: Physics-Induced Action Failure Happy Oyster enforces physical constraints more rigorously than any other evaluated model, which paradoxically hurts its action-following score. As shown in Fig. 18 (bottom), 32

Genie 3 Happy Oyster

Figure 18. Action gap in closed-source models. Top: Genie 3. When the D key (rightward translation) is issued, the model rotates the camera to the right instead of translating the character rightward within a stable scene. The character remains centered while the entire scene rotates, making controlled navigation impossible. Bottom: Happy Oyster. When the D key (rightward translation) is issued, the model faithfully enforces collision with a newly generated obstacle, preventing the character from moving rightward. This strong adherence to physics results in lower action-following scores. Each row shows frames from left to right in temporal order.

J.2. Human Verification of VLM Judgments

when the autoregressive generation produces new obstacles along the navigation path, the model faithfully simulates collision rather than allowing the character to pass through. Once the character collides with a generated obstacle, subsequent movement actions cannot dislodge it because the model prioritizes physical plausibility over the commanded input. This strong adherence to physics effectively traps the character and causes case-level actionfollowing failures, even though the underlying generative process remains physically coherent.

Both the interaction-physics pipeline (Section E) and the subject-memory pipeline (Section F.2) delegate the final perceptual judgment to Qwen3-VL-PLUS. We conduct two compact human studies—one for the binary mechanics protocols and one for the ordinal subject-memory protocol—to verify that the VLM tracks human raters on these exact decisions. J.2.1. Physics VLM Verification From the 63 × 10 = 630 mechanics rollouts (63 cases evaluated on all 10 models), we randomly sample 100 videos and ask 20 human experts to provide the same binary verdict that Qwen3-VL-PLUS produces, taking the majority vote as ground truth. The 100-video set is used as a promptcalibration loop: through iterative revision of the mechanics prompts (Appendices E.6–E.14), Qwen3-VL-PLUS accuracy against the human majority rises from 52.5% (initial) to 73.2% (final), confirming that the calibrated prompts retained in WorldRoamBench agree with humans on nearly three out of four mechanics decisions.

J. Qualitative Study and Human Verification This appendix complements the quantitative leaderboard with a qualitative tour of WorldRoamBench scores and a sanity check that the Qwen3-VL-PLUS-based judgments used by our physics and memory pipelines remain aligned with human perception.

J.1. Qualitative Study Figure 19 contrasts a positive and a negative rollout for each of the three dimensions—interaction physics, memory, and action following; the two rollouts in each block share both the initial frame and the keyboard action schedule, isolating the dimension under evaluation. Physics. Under a sustained W + A in a narrow corridor, Happy Oyster keeps the avatar inside the geometry (No Clipping), whereas Genie 3 penetrates the wall into vegetation (Clipping). Memory. Under J (Observation) → L (Revisit) along a neon alley, Matrix-Game 3.0 reproduces the same storefronts on revisit (Good Memory), while SANA-WM regenerates different murals and shopfronts (Bad Memory). Action. Under an S → A schedule (backward → left), Genie 3’s per-frame predictions match the ground truth throughout (Good Action), while Yume 1.5 outputs forward in every segment (Bad Action).

J.2.2. Subject Memory VLM Verification We randomly sample 60 TPV rollouts from Happy Oyster, Genie 3, LingBot-World, and HY-World 1.5, and ask 20 human experts to score each rollout on the same 1–10 scale and rubric as Qwen3-VL-PLUS (Figure 27), blind to the VLM score. Defining a (rollout, expert) pair as concordant when |SVLM − Shuman | ≤ 1, the concordance rate over the 60 × 20 = 1,200 pairs reaches 88.3% with a mean absolute difference of 0.83, placing Qwen3-VL-PLUS well within the natural variation of independent human raters.

Physics cases

Timeline Happy Oyster No Clipping! Genie 3 Clipping!

Memory cases

Observation

Revisit

Timeline MG-3 Good Memory!

SANA-WM Bad Memory!

Action cases

Timeline Genie 3 Good Action! YUME Bad Action!

Figure 19. Qualitative examples of WorldRoamBench scoring across physics, memory, and action. Each block contrasts a positive and a negative rollout that share the same initial frame and action schedule. Physics: W+A in a corridor—Happy Oyster (No Clipping) vs. Genie 3 (Clipping). Memory: J → L Observation/Revisit in an alley—Matrix-Game 3.0 (Good Memory) vs. SANA-WM (Bad Memory). Action: S → A (backward → left)—Genie 3 (Good Action) vs. Yume 1.5 (Bad Action). Frames are shown in temporal order from left to right.

Collision Evaluation Prompt — Turn 1: Approach Detection You are analyzing a video clip ({duration}s total).

Showing {N} frames at timestamps:

[{t1, t2, ..., tN}].

Think step by step about the following question. Question: In these consecutive frames, is the camera/viewer getting noticeably closer to {target object}? for the object growing larger in frame or the distance visibly decreasing. After your reasoning, conclude with exactly one line:

Answer:

yes or Answer:

Look

Collision Evaluation Prompt — Turn 2: Collision Response Verification You are analyzing a video clip ({duration}s total).

Showing {N} frames at timestamps:

[{t1, t2, ..., tN}].

Think step by step about the following question. Question: In these frames, after the camera reaches or contacts {target object}, does {target object} show a physically plausible collision response (e.g., being {expected response})? If there is no contact or collision visible, answer ’no’. After your reasoning, conclude with exactly one line:

Answer:

yes or Answer:

Figure 20. Collision evaluation prompts. Full VLM prompts used for approach detection and collision-response verification.

Deformation Evaluation Prompt — FPV: Trace Detection You are analyzing a video clip ({duration}s total).

Showing {N} frames at timestamps:

[{t1, t2, ..., tN}].

Think step by step about the following question. Question: In these frames from the later part of a first-person walking video, look carefully at the ground/terrain that the viewer has previously walked over. Can you see any visible traces of prior movement, such as: {deformation type}? These would be evidence that the environment was physically affected by the viewer’s passage. Look for any marks, impressions, or changes in the ground surface that weren’t there originally. After your reasoning, conclude with exactly one line:

Answer:

yes or Answer:

Deformation Evaluation Prompt — TPV Trace Mode: Reference Comparison Scenario:

{description}

Image 1 is the REFERENCE first frame showing the original undisturbed ground. The remaining images are later frames where the subject has already passed through. Compare the ground/terrain in the later frames to the reference. Has the ground changed in any way? - {deformation type} - Any marks, tracks, impressions, or disturbances on the surface - Any difference in ground texture or pattern compared to the reference

Look for:

Be generous: even faint, partial, or subtle changes count as YES. Answer YES if the ground shows ANY difference from the reference. Answer NO only if the ground looks identical to the reference.

Deformation Evaluation Prompt — TPV Realtime Mode: Interaction Observation Scenario:

{description}

In these frames from a third-person view video, the subject is moving through the environment. Watch the area where the subject interacts with the environment. such as {deformation type}?

Do you see any real-time environmental response,

Be generous: even subtle, partial, or small effects count as YES. Answer YES if you see ANY environmental reaction to the subject. Answer NO only if the environment shows absolutely no response.

Figure 21. Deformation evaluation prompts. Full VLM prompts used for first-person trace detection, third-person reference comparison, and third-person real-time interaction checks.

Clipping Evaluation Prompt — Stage 1: Approach Detection You are analyzing a video clip ({duration}s total).

Showing {N} frames at timestamps:

[{t1, t2, ..., tN}].

Think step by step about the following question. Question: In these consecutive frames from a first-person view, is the viewer getting noticeably closer to {obstacle}? Look for the obstacle/barrier growing larger and filling more of the frame. After your reasoning, conclude with exactly one line:

Answer:

yes or Answer:

Clipping Evaluation Prompt — Stage 2: Pass-Through Verification You are analyzing a video clip ({duration}s total).

Showing {N} frames at timestamps:

[{t1, t2, ..., tN}].

Think step by step about the following question. Question: In these frames, after the camera was very close to {obstacle}, does the view appear to PASS THROUGH the obstacle and reach the other side? Signs of clipping: the obstacle suddenly disappears, the view shows the interior of a wall, or the scene transitions to what should be behind the obstacle without any opening. If the view is simply blocked by the obstacle or the person stops, answer ’no’. After your reasoning, conclude with exactly one line:

Answer:

yes or Answer:

Clipping Evaluation Prompt — Stage 3: First vs Last Frame Trigger I will show you the FIRST and LAST frame of a first-person view video. In this video the viewer walks toward {obstacle}. Frame 1: Frame 2:

the starting position. the ending position.

Compare them: has the viewer PASSED THROUGH {obstacle} and ended up on the OTHER SIDE? Signs: the obstacle that was ahead is now behind or gone, the scene in Frame 2 shows a completely different space that should be behind {obstacle}. Answer YES if the viewer appears to have phased through the obstacle. Answer NO if the viewer is still on the same side (blocked, stopped, or turned away).

Clipping Evaluation Prompt — Stage 4: Barrier Side Check I will show you the FIRST and LAST frame of a first-person view video. The viewer is walking {movement}. Frame 1 (reference): {barrier} is on the viewer’s {initial side} side. Frame 2 (to check): The last frame. Question:

In Frame 2, has {barrier} switched from the {initial side} side to the {opposite side} side?

After your reasoning, you MUST end with exactly:

Answer:

yes or Answer:

Figure 22. Clipping evaluation prompts. Full VLM prompts used by the staged clipping-detection cascade.

Gravity Evaluation Prompt — Scenario: Object Falls Context:

{description}

In this video, when an object is released or knocked off a surface, does it fall downward in a physically plausible way (accelerating downward due to gravity)? Objects should not float, hover, or fall unnaturally slowly.

Gravity Evaluation Prompt — Scenario: Sinks in Water Context:

{description}

When the subject enters water, does the viewpoint lower/sink in a physically plausible way? submersion should look natural.

The buoyancy and

Gravity Evaluation Prompt — Scenario: Projectile Arc Context:

{description}

Does the thrown/launched object follow a realistic parabolic arc under gravity? time, not travel in a straight line.

It should curve downward over

Gravity Evaluation Prompt — Scenario: No Floating Context:

{description}

Do all objects in the scene remain grounded or fall naturally?

No objects should float or hover without support.

Gravity Evaluation Prompt — TPV: General Gravity Check Scenario:

{description}

In this third-person view video, watch what happens when the subject reaches the edge, drop-off, water, or pit. Evidence that gravity is working: - The subject’s vertical position LOWERS | they fall, sink, slide down, or become submerged - The subject’s body goes partially or fully below the surface/edge level - Even gradual sinking into water counts as gravity working Evidence that gravity is NOT working: - The subject walks ON water without sinking - The subject floats or hovers over a drop-off - The subject reaches an edge but stays at the same height Answer YES if the subject’s height decreases at any point (falling, sinking, descending). Answer NO if the subject defies gravity (floats, walks on water, ignores edges).

Figure 23. Gravity evaluation prompts. Full VLM prompts used for scenario-specific gravity checks and the third-person general gravity check.

Terrain Following Prompt — Naturalness Check (Strategy A & B) {scene description} I will show you two frames from a first-person view video. Frame 1 (reference): The starting frame, where the viewer is standing naturally on the ground. Frame 2 (to check): A frame from {time label}. Does Frame 2 show any of these UNNATURAL viewpoint problems? - FLOATING: the viewpoint is unnaturally HIGH, hovering above the ground - GROUND-LEVEL: the viewpoint is unnaturally LOW | at ankle/ground height instead of standing eye-height. Signs: ground or step faces fill the view like walls, the camera appears to be lying on the ground - CLIPPING: the viewpoint has sunk INTO the ground, showing impossible views through surfaces, ghost/transparent overlays of terrain Answer YES if Frame 2 looks UNNATURAL (any of the above). Answer NO if Frame 2 looks NATURAL | the viewer appears to be at normal standing height, walking or standing on the terrain surface, even if the terrain is different from Frame 1 (uphill, stairs, downhill are all fine).

Terrain Following Prompt — TPV: Ground Contact Check Scenario:

{description}

In these frames from a third-person view video, check if the subject’s movement follows the terrain naturally: - Is the subject on the ground surface? (not floating above it) - Does the subject’s height change appropriately with terrain changes (going up on slopes/stairs, going down on descents)? - Is the subject at a natural height? (not sunk into the ground) Answer YES if the subject follows the terrain naturally. Answer NO if you see any of: floating/hovering, sinking into ground, or ignoring terrain elevation changes.

Terrain Following Prompt — Boundary Check I will show you two frames from a first-person view video. Frame 1 (reference): The starting frame | note the walkable path, railings, and the viewer’s position relative to them. Frame 2 (to check): A later frame. Compare Frame 2 to Frame 1 carefully: has the viewpoint drifted sideways relative to the path? still centered on the walkable surface, or has it shifted to one side beyond the railing?

Is the camera

{boundary prompt}

Figure 24. Terrain-following evaluation prompts. Full VLM prompts used for naturalness, third-person ground contact, and boundary checks.

Reflection Evaluation Prompt — Tier 1: Coarse Check (FPV) There is {target object} reflected in {surface desc}. Looking at the overall scene across all these frames, are the reflections physically plausible | i.e., consistent in shape, approximately symmetric about the reflecting surface, and changing naturally with the camera movement? After your reasoning, conclude with exactly one line:

Answer:

yes or Answer:

Reflection Evaluation Prompt — Tier 1: Coarse Check (TPV) In this third-person view video, watch {target object} and its reflection in {surface desc}. As the subject moves, does the reflection behave physically correctly | i.e., consistent in shape, approximately symmetric about the reflecting surface, and moving consistently with {target object}? After your reasoning, conclude with exactly one line:

Answer:

yes or Answer:

Reflection Evaluation Prompt — Tier 2: Per-Frame Scoring I show you two images. Image 1 is the REFERENCE first frame showing the original scene. Image 2 is a frame from the generated video at timestamp {timestamp}s. In the reference frame, {target object} is reflected in {surface desc}. FIRST, check whether {surface desc} is clearly visible in Image 2. If the camera has moved so that {surface desc} is no longer in view, or is only a tiny sliver at the edge, score N/A | do NOT score 0 just because you cannot see a reflection. Score 0 means the surface IS visible but the reflection is wrong. Score the reflection quality in Image 2: - 2: Reflection is physically correct | properly flipped, content matches the real scene above, proportions and perspective are consistent with the viewing angle - 1: Reflection exists but has physics errors | wrong flip direction, noticeable asymmetry, content partially mismatched, or unnatural distortion - 0: The reflecting surface IS visible and occupies a reasonable area, but the reflection is missing, completely wrong, or shows unrelated content - -1: INVALID | the scene has collapsed or deviated so far from the reference that reflection evaluation is not meaningful (e.g., completely different environment) - N/A: The reflecting surface ({surface desc}) is NOT visible or barely visible in this frame | camera has moved away from it After your reasoning, output exactly one line:

Score:

-1, Score:

0, Score:

1, Score:

2, or Score:

N/A

Figure 25. Reflection evaluation prompts. Full VLM prompts used for coarse reflection plausibility checks and per-frame reflection scoring.

Shadow Evaluation Prompt — FPV with Reference Frame and Expected Shadow The FIRST image is the REFERENCE first frame showing the initial scene. The remaining images are frames from the generated video. The expected physical phenomenon is:

{expected shadow}.

Compare the video frames against the reference.

Does the generated video correctly show {expected shadow}?

Judgment criteria: - A real shadow must have a recognizable silhouette shape matching the object that casts it (e.g., a person shadow looks like a person outline, a pillar shadow looks like an elongated rectangle) - The shadow must have a clear boundary | vague dark patches, ambient shading, surface texture, or lighting gradients do NOT count - The shadow direction must be physically consistent with visible light sources in the scene - Answer ’yes’ ONLY if you are confident the shadow is real and correct After your reasoning, conclude with exactly one line:

Answer:

yes or Answer:

Shadow Evaluation Prompt — TPV with Expected Shadow In this third-person view video, {expected shadow} should be visible. Looking at these frames, does {expected shadow} appear and behave in a physically correct manner (consistent direction relative to the light source, moving with the subject)? After your reasoning, conclude with exactly one line:

Answer:

yes or Answer:

Shadow Evaluation Prompt — TPV Fallback (No Expected Shadow) In this third-person view video, as the subject moves forward, does the subject cast a visible shadow on the ground or nearby surfaces (walls, floors, etc.)? If so, is the shadow shape recognizable and its direction consistent with the lighting in the scene? After your reasoning, conclude with exactly one line:

Answer:

yes or Answer:

Shadow Evaluation Prompt — Generic Fallback In this scene there is a light source illuminating a wall or floor. As the camera moves, a shadow should appear or change on the surface. Looking at these frames from the latter portion of the video: 1. Is there a shadow visible on the wall or floor? 2. If yes, does the shadow move or change in a way that is physically consistent with the camera movement and the light source position? Answer ’yes’ if the shadow appears and behaves correctly, ’no’ if the shadow is missing or behaves incorrectly. After your reasoning, conclude with exactly one line:

Answer:

yes or Answer:

Figure 26. Occlusion and shadow evaluation prompts. Full VLM prompts used for expected-shadow, fallback shadow, and generic shadow checks.

Subject Memory Evaluation Prompt You are an expert evaluator of third-person video subject memory. The input is a temporally ordered sequence of subject crops from one generated video. The first crop is the reference subject; the remaining crops show the subject at later time points. Evaluate whether the subject remains the same individual across the whole sequence, and output a video-level subject memory score from 1 to 10. Assess five failure types: 1. Identity change: the subject becomes a different individual or category. 2. Structural distortion: the body, anatomy, proportions, limbs, head, face, or key parts become deformed or implausible. 3. Appearance drift: color, texture, clothing, hair, material, or style is truly rewritten. 4. Subject disappearance: the subject becomes partly or fully invisible. 5. Quality degradation: severe blur, low resolution, diffused edges, or loss of key details makes the subject hard to identify. Important calibration rules: - Viewpoint change alone is not inconsistency. Back-to-front, front-to-back, side-to-front, or far-to-close changes are acceptable if the subject can reasonably be the same individual under the new view. - New frontal details, such as face or front clothing, should not be treated as contradictions when the reference was a back view. - Impossible orientation conflicts are structural distortion. If the body is still back-facing but the face suddenly looks toward the camera, the score should be at most 5; if this persists or the head-body relation collapses, score 1--3. - Same category does not imply consistency. If a person, animal, or vehicle remains in the same broad class but gradually becomes a blob, oval mass, fused-limb shape, or loses a clear part structure, assign a low score. - Lighting and shadow changes should be treated leniently. If the subject becomes darker, bluish-gray, lower contrast, or slightly softer because of shadow, but structure, proportions, clothing shape, and identity cues remain stable, this is usually 8--9. - Do not mark appearance drift unless color, texture, clothing, fur, or material changes in a way not explainable by lighting, exposure, or white balance. - Do not mark structural distortion only because shadows, low contrast, or mild blur soften boundaries. Mark it only when real geometry or anatomy changes. - Mild blur or reduced texture contrast can be ignored; severe blur, low resolution, or missing key details should reduce the score. - Judge the full sequence holistically. Progressive deterioration is more serious than an isolated artifact. - Hallucinated non-subject objects, such as a handheld item, bag, stick, or small prop, are not identity changes if the subject body and main appearance remain consistent. Scoring guide: 10: Highly consistent identity, structure, proportions, appearance, and details. 8--9: Mostly consistent; only minor appearance variation, mild blur, normal viewpoint change, or lighting/shadow change. 7: Same subject with stable structure and main appearance, but noticeable darkening, mild-to-moderate blur, lower texture contrast, or edge softening. 6: Roughly consistent, but clear non-lighting appearance drift or persistent detail degradation begins to affect identity details. 4--5: Poor consistency, with real structural distortion, obvious appearance rewriting, partial disappearance, or severe blur. 2--3: Severe degradation, such as fused limbs, unclear head-body relation, blob/mass/oval collapse, or large-scale failure of key structure. 1: The subject clearly becomes a different object, disappears for most of the sequence, or can no longer be matched to the reference. Output JSON only. Do not output anything else: { "subject memory score": [integer from 1 to 10], "identity change": [true or false], "structural distortion": [true or false], "appearance drift": [true or false], "subject disappearance": [true or false], "quality degradation": [true or false], "viewpoint change": [true or false], "lighting change": [true or false], "reason": [brief explanation of the scoring rationale] }

Figure 27. Subject memory evaluation prompt. Full VLM prompt used for holistic third-person subject-memory scoring and diagnostic flag extraction.

Record · ID 324953 · SHA-256 8ad8956b8cc21661
Retrieved via Conceptio — every document is proof-bundled with source, license, and retrieval metadata.