BioVLN: A Simulation Platform for Visual Language Navigation in Biomedical Laboratories
arXiv:2607.26914v1 [cs.RO] 29 Jul 2026
Zhe Liu*1,2 , Quan Lu*1,2 , Zhaohui Du*1,2 , Zhe Wang†1,2 , Huanbo Jin1,2 , Jiaming Gu1,2 , Qi Wang3 , Ting Xiao3 , Minting Pan4 , Dongzhan Zhou4 1 Key Laboratory of Smart Manufacturing in Energy Chemical Process, MoE, East China University of Science and Technology, Shanghai, China 2 Department of Computer Science and Engineering, East China University of Science and Technology, Shanghai, China 3 Department of Laboratory Medicine, Ruijin Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China 4 AI for Science Center, Shanghai AI Laboratory, Shanghai, CN [email protected] https://github.com/ActiveButterflies/BioVLN
1
Abstract
Introduction
Language-conditioned embodied navigation, including visual-language navigation (VLN) and ObjectNav, has progressed rapidly through standardized simulators, largescale indoor datasets, and increasingly capable semantic navigation agents. Habitat and AI2-THOR established reproducible environments for embodied learning, while ProcTHOR enabled large-scale procedural scene generation Savva et al. (2019); Kolve et al. (2017); Deitke et al. (2022). ObjectNav has similarly evolved from geometric and learned exploration to open-vocabulary methods that use vision-language models and scene-level reasoning to locate unseen targets Batra et al. (2020); Yitzhak Gadre et al. (2022); Yokoyama et al. (2024); Yin et al. (2024). These platforms and benchmarks are primarily designed for homes and offices, where success is defined by reaching the target object or a nearby position. Biomedical laboratories require a more functional definition of navigation success. Laboratory robots must approach instruments before manipulation or experimental execution can begin Holland and Davies (2020); Burger et al. (2020). A centrifuge must be reached from its control side, a refrigerator from its door, and a benchtop instrument with sufficient clearance from adjacent equipment. The destination is not merely the instrument location, but a position from which it can be safely accessed. This requirement is further complicated by dense, workflow-dependent layouts and specialized instruments that can be difficult for general-purpose vision-language models to recognize. Current navigation formulations do not explicitly capture these constraints. Point-based or proximity-based
Biomedical laboratory robots must navigate to instruments before performing experimental procedures. Existing embodied navigation platforms are designed for household environments and treat a target as an object center or an arbitrary nearby position. This representation is inadequate for laboratory instruments, which must be approached from their operating side while maintaining safe clearance from surrounding equipment. We introduce BioVLN, a simulation platform for developing and evaluating visuallanguage navigation agents in biomedical laboratories. BioVLN represents each instrument with three regions: its physical body, a surrounding clearance region, and an operation area in front of the usable side. This model is applied consistently to scene generation, target placement, navigation evaluation, and safety analysis, so success depends on reaching a position from which the instrument can be accessed. BioVLN supports procedural scene generation and manually designed environments, producing 47 scenes and 1,667 episodes. Standardized navigation and reinforcement-learning interfaces enable trajectory collection and policy training. Experiments show that geometric exploration reaches 74.4–87.5% success, while sampling multiple valid positions in the operation area improves success to 83.3–92.5% and reduces unsafe proximity.
* Equal contribution. † Corresponding author.
1
goals may indicate that an object has been reached, but they do not encode its operating direction or usable approach region. Standard metrics measure goal reaching and path efficiency without quantifying clearance from surrounding equipment. Mainstream scene-generation pipelines also lack a unified representation of instrument-specific operating directions, approach regions, and safety constraints. These limitations leave a gap between semantic object search and navigation that can support downstream laboratory interaction. We introduce BioVLN, a simulation platform that formulates biomedical laboratory navigation around operational accessibility. Its central abstraction is a three-zone operational envelope comprising the instrument body, a surrounding clearance region, and an operation area in front of the usable side. This abstraction is applied consistently to scene construction, goal placement, episode generation, navigation evaluation, and trajectory-level safety analysis. Our contributions are summarized as follows:
et al. (2017), Gibson Xia et al. (2018)) to real-world building scans (Matterport3D Chang et al. (2017), HM3D Ramakrishnan et al. (2021)), interactive manipulation (iGibson Li et al. (2021), BEHAVIOR Srivastava et al. (2022)), large-scale procedural generation (ProcTHOR Deitke et al. (2022)), and sim-to-real transfer (RoboTHOR Deitke et al. (2020)). All existing platforms target residential or office domains and provide no mechanisms for evaluating operational safety or instrument-specific spatial constraints. Table 1 contrasts BioVLN with representative platforms.
2.2
Object-Goal Navigation
Object-goal navigation (ObjectNav) Batra et al. (2020) tasks an agent with navigating to any instance of a specified object category. The field has progressed from classical geometry-based exploration to learned policies (PONI Ramakrishnan et al. (2022)), semantic exploration (SemExp Singh Chaplot et al. (2020)), and, most recently, zeroshot VLM-guided approaches (ZSON Majumdar et al. (2022), CoW Yitzhak Gadre et al. (2022), SG-Nav Yin et al. (2024)). A fundamental limitation persists across all formulations: goals remain point targets at object centroids without orientation or affordance constraints. For laboratory instruments, the operational face (e.g., a refrigerator’s front door versus its rear panel) determines whether a navigation trajectory is functionally useful. BioVLN reformulates the goal as an operational approach position with explicit affordance-facing direction.
• We present BioVLN, an extensible simulation platform for visual-language navigation in biomedical laboratories. It introduces operational-face goals and a three-zone spatial model that couples instrument geometry, approach accessibility, and safety constraints throughout scene generation and evaluation.
• We develop a dual-entry scene pipeline supporting both procedural construction and designer-authored scene import through the Blender-based LabScene Annotation Toolkit (LSAT). The platform provides 2.3 Frontier-Based Exploration 47 scenes and 1,667 episodes, with standardized trajectory-recording and Gym-compatible interfaces Frontier-based exploration, introduced by Yamauchi Yamauchi (1997), navigates toward boundaries between exfor zero-shot evaluation and policy learning. plored free space and unknown regions. The paradigm • We establish a six-method benchmark across three has evolved from hand-crafted heuristics to learned neulaboratory datasets using both navigation and safety ral variants (Active Neural SLAM Chaplot et al. (2020a), metrics. Results show that sampling multiple valid Neural Topological SLAM Chaplot et al. (2020b)) and positions within the operation area improves success large-scale RL (DD-PPO Wijmans et al. (2020)). Learned to 83.3–92.5% while reducing unsafe proximity, and methods require environment-specific training and do not reveal that vision-language navigation is strongly af- transfer zero-shot to domains with distinct visual and gefected by the visual quality of rendered laboratory ometric statistics. We adopt a purely geometric frontier method as a training-free baseline. assets.
2.4
2
Related Work
Vision-Language Models for Navigation
Leveraging large vision-language models (VLMs) for zero-shot semantic navigation has attracted considerable recent interest. VLFM Yokoyama et al. (2024) scores fron2.1 Embodied Navigation Platforms tier views with BLIP-2 Li et al. (2023) and GPT-4o Hurst The embodied AI community has developed several high- et al. (2024) for semantic value map construction. Instructfidelity simulation platforms, evolving from fixed scene Nav Long et al. (2024) leverages LLMs for instructionsets (Habitat-Sim Savva et al. (2019), AI2-THOR Kolve following zero-shot navigation. SG-Nav Yin et al. (2024) 2
Figure 1: The BioVLN platform architecture. P1 (left): designer-authored scene import, where manually built GLB scenes annotated via LSAT are processed. P2 (right): procedural generation, where instrument catalogs and layout templates drive automated construction. Both paths converge into a shared benchmark pipeline with six-method evaluation and safety metrics (MCR, VRT). introduces scene graph prompting for hierarchical LLMbased goal reasoning. CoW Yitzhak Gadre et al. (2022) and OpenFMNav Kuang, Lin, and Jiang (2024) employ vision-language foundation models for open-vocabulary object search. These methods share an implicit premise: that the VLM possesses visual familiarity with target objects acquired during pre-training. Laboratory instruments such as centrifuges and incubators lie outside this distribution. Whether VLM navigation failure in laboratory settings stems from deficient conceptual knowledge or insufficient visual signal in rendered assets remains an open question.
3
BioVLN Platform
3.1
Overview
BioVLN operates through a dual-entry pipeline (Fig. 1). Scenes can be constructed through two complementary paths. P1 imports designer-authored GLB scenes annotated via the LSAT Blender addon, which extracts goal positions from artist-placed markers and maps them to instrument categories. P2 procedurally generates scenes from an instrument catalog and layout templates, driving a headless Blender builder followed by semantic annotation, navigation mesh generation, and episode construction. Both paths produce identical outputs: a scene GLB with navigation mesh and episodes. Six navigation methods are evaluated in a shared action space, with safety metrics (MCR, VRT) computed from trajectories against the three-zone model of every instrument in the scene. The platform also provides Gym-compatible and trajectory3
Table 1: Overview of existing embodied navigation platforms (✓ = supported, × = not supported)
Domain Procedural scene generation Designer-authored scene import LLM-assisted layout design Operational-face goal definition Operational accessibility model Safety metrics (MCR, VRT) Blender-based scene annotation
Habitat Savva et al. (2019)
AI2-THOR Kolve et al. (2017)
ProcTHOR Deitke et al. (2022)
iGibson Li et al. (2021)
BioVLN (ours)
Home/Office × × × × × × ×
Home × × × × × × ×
Home ✓ × × × × × ×
Home × × × × × × ×
Laboratory ✓ ✓ ✓ ✓ ✓ ✓ ✓
Table 2: Laboratory Asset Catalog
recording interfaces for policy learning.
3.2
Asset
Problem Formulation
Dimensions (m)
Refrigerator 0.89×0.91×1.78 wall Incubator 1.35×0.85×1.90 wall Centrifuge 0.49×0.42×0.63 table_top Mini-Centrifuge 0.66×0.50×0.43 table_top Pipette 0.04×0.32×0.28 table_top Cabinet 1.38×0.57×1.77 wall Waste Bin 0.38×0.37×0.51 floor
We describe the platform’s core components: the operational-face goal formulation, the laboratory asset catalog, and the three-zone operational envelope model.
Operational-Face Goal Navigation. Let I = {𝐼1 , . . . , 𝐼 𝐾 } be the set of instruments in a Room infrastructure scene. Each instrument 𝐼𝑖 is characterized by a 3D Lab Bench 1.55×5.94×0.76 centroid c𝑖 , an operational face direction f𝑖 ∈ R2 (unit Lab Table 2.40×0.90×1.10 vector indicating the interactive surface normal), and a Room (inner) 8.00×6.00×3.24 half-depth 𝑑𝑖 (distance from centroid to face). The goal position is: g𝑖 = c𝑖 + f𝑖 · (𝑑𝑖 + 𝛿𝑖 )
Zone
— — —
Op. Face 𝑑half (m) −𝑌 −𝑋 −𝑋 −𝑋 −𝑌 +𝑋 +𝑋
0.455 0.673 0.245 0.332 0.159 0.692 0.183
— — —
— — —
Three-Zone Operational Envelope. BioVLN’s defining spatial abstraction is a three-zone model around each instrument. The operational-face navigation concept underlying this model is illustrated in Fig. 2, and the three zones jointly govern scene generation, collision avoidance, and navigation evaluation. (1) Zone 1: Physical body. The instrument’s axisaligned bounding box, derived from its GLB geometry (Table 2). This zone represents the space that the instrument physically occupies. Wall instruments contact the wall on one face; table-top instruments rest on support surfaces at precisely measured heights (𝑍bench =0.76 m, 𝑍table =0.78 m). (2) Zone 2: Safety clearance. A buffer of 𝜀 𝑐 =0.25 m extends beyond Zone 1 on all four horizontal sides, representing the minimum clearance required for safe navigation. During scene generation, the Zone 2 regions of any two instruments cannot overlap:
(1)
where 𝛿𝑖 is an instrument-specific approach distance (0.5– 0.9 m). The goal is placed at floor height (𝑧=0 in Blender Z-up coordinates). The agent initializes at a random navigable start position s on the navigation mesh. At each step 𝑡, the agent receives an egocentric RGB-D observation and selects 𝑎 𝑡 ∈ A = {forward 0.25m, turn_left 30◦ , turn_right 30◦ , stop}. An episode succeeds if the agent calls stop within a distance 𝑟 𝑖 of g𝑖 , where 𝑟 𝑖 is an instrument-specific success radius (0.8–1.3 m). The task is zero-shot: the agent receives no training episodes from the target scene.
Laboratory Asset Catalog. Table 2 lists the instrument and infrastructure assets used in BioVLN. Each asset is stored as a GLB file with geometry centered at the modelspace origin, allowing uniform world-space placement and 𝑠 +𝑠 𝑠 +𝑠 rotation without per-asset coordinate adjustments. Dimen|𝑥𝑖 −𝑥 𝑗 | ≥ 𝑥,𝑖 2 𝑥, 𝑗 +2𝜀 𝑐 or |𝑦 𝑖 −𝑦 𝑗 | ≥ 𝑦,𝑖 2 𝑦, 𝑗 +2𝜀 𝑐 (2) sions and support surface heights were verified through Blender vertex analysis. Operational face directions were This guarantees navigable gaps of at least 0.5 m between determined by manual inspection of each instrument’s 3D instruments. During evaluation, Zone 2 violations are model in Blender. recorded and reported as safety metrics (Fig. 4). 4
Figure 3: Scene-level operational-face goal positions. Left: Figure 2: Operational-face goal navigation. Left: blue solid oblique overview. Right: top-down visualization per in= correct navigation to the operational face (red point); strument. red dashed = incorrect (reaches incubator but not its operational surface). Right: top-down view of the operational GLBs at the specified positions, then the shared pipeline face. stages process the assembled scene. Both paths converge on identical outputs: a scene GLB (3) Zone 3: Operational envelope. A rectangular re- file with vertex-level semantic annotations (COLOR_0 engion of depth 𝛿𝑖 (0.5–0.9 m, instrument-dependent) ex- coding), a Recast navigation mesh, and episode definitions tending outward from the operational face along f𝑖 . The with sampled starts and operational-face goals. goal g𝑖 sits at its outer boundary (Eq. 1). Zone 3 defines the task-relevant navigation target: the standing position Rotation and Placement from which a human or robot can operate the instrument. The three-zone model departs deliberately from stan- Instruments against walls must rotate so their operational dard ObjectNav. Habitat ObjectNav Savva et al. (2019) face points toward the room interior. Given a model-space projects bounding-box centers to the floor, producing po- face vector (𝑚 𝑥 , 𝑚 𝑦 ) and wall-determined interior direcsitions that may lie inside furniture or against walls. Mat- tion (𝑤 𝑥 , 𝑤 𝑦 ), the Z-axis rotation 𝜃 is: terport3D Chang et al. (2017) uses human-annotated viewpoints without explicit spatial constraints. BioVLN makes 𝜃 = atan2(𝑚 𝑥 𝑤 𝑦 − 𝑚 𝑦 𝑤 𝑥 , 𝑚 𝑥 𝑤 𝑥 + 𝑚 𝑦 𝑤 𝑦 ) (3) spatial semantics explicit and uses the three-zone model for safety evaluation. This formulation handles all four wall orientations and arbitrary model-space face directions. Table-top instruments align perpendicular to the nearest table edge; floor 3.3 Scene Generation Pipeline instruments (waste bin) are unconstrained. Zero-Origin Architecture Vertical placement uses surface height and model offsets: Our pipeline uses origin-centered GLB assets: each 𝑍place = 𝑍table_top + 𝑍bb_center − 𝑍bottom (4) model’s geometry is centered at (0, 0, 0) in its local coordinate frame, and world-space placement and rotation are where 𝑍table_top is the measured support surface height applied uniformly at import time. This design contrasts (0.76 m bench, 0.78 m table), 𝑍bb_center is the instrument’s with conventional baked-GLB workflows, where vertex bounding-box center Z, and 𝑍bottom is its lowest vertex Z. coordinates are pre-transformed to world space, preventing repositioning or layout variation. Layout Templates P1 (designer-authored import) begins with a manually built GLB scene containing artist-placed target_* We define four layout templates; two (single-room, twomarkers. LSAT extracts these markers and maps them room) are implemented and used in benchmarks, while to instrument categories. A hand-authored configuration suite and open-plan are defined for future expansion. Fig. 3 file supplies semantic mesh annotations, after which the shows the resulting goal positions across all instruments shared pipeline stages (semantic annotation, navigation in a single-room scene. Table 3 summarizes generation mesh, episode construction) produce the final benchmark statistics. scene. The generator randomizes which instrument slots are P2 (procedural generation) uses a specification gener- active (2–5 out of 6 for single-room) and which instruments ator that produces a JSON description encoding each in- fill those slots, producing layout diversity. Each scene is strument’s world position, rotation, and operational goal. identified by a seed value that deterministically controls A headless Blender builder imports room and instrument random choices. 5
Table 3: Layout Templates and Generation Statistics
Table 4: Benchmark Dataset Statistics
Template
Room Configuration
Scenes
Episodes
Dataset
Scenes
Eps.
Cat.
Goal Source
single-room two-room
8×6 m 6×5 m + 3.5×3.5 m
39 8
1,363 304
47
1,667
MS DEV MS VAL MS TEST Two-Room LSAT Target
4 1 2 1 1
160 50 120 50 40
4 4 4 10 8
Computed Computed Computed Computed Human-annotated
Total
—
420
—
Total
Episode Construction Start positions are randomly sampled from the navigation mesh at least 1.0 m from obstacles. Goal positions are snapped to the nearest navigable navmesh point. The geodesic shortest path from start to goal is computed via the navmesh graph. View-points are generated at the goal, oriented toward the target instrument. Episodes are classified by geodesic distance: easy (𝑑geo < 3.0 m), medium (3.0 ≤ 𝑑geo ≤ 6.0 m), hard (𝑑geo > 6.0 m). On MultiScene DEV (160 episodes), the distribution is 11.9% easy, 28.1% medium, 60.0% hard.
3.4
Strategy 1 goal positions, isolating the effect of goal placement precision.
4.2
We report six metrics spanning navigation effectiveness and operational safety. Navigation Metrics. (1) Success Rate (SR): fraction of episodes ending within the instrument-specific success opt opt radius. (2) SPL: success weighted by ℓ𝑖 /max(ℓ𝑖 , ℓ𝑖 ), opt where ℓ𝑖 is the optimal geodesic path length. (3) Distance to Goal (DTG): final Euclidean distance to the goal, averaged over all episodes. (4) Average Steps: mean actions per successful episode. Safety Metrics. Two metrics derived from the three-zone model, computed from the agent’s full trajectory against all instruments in the scene: (1) Minimum Clearance Radius (MCR): the closest distance (m) between the agent and any instrument surface over the entire trajectory. Values below the Zone 2 hazard threshold (0.5 m) indicate a safety violation. (2) Violation Rate at Threshold (VRT): the fraction of trajectory steps during which the agent is within 0.5 m of any instrument surface. VRT captures persistent unsafe proximity, not single-point violations.
LSAT Annotation Toolkit
We developed the LabScene Annotation Toolkit (LSAT), a Blender 5.1.2 addon (13 modules, ∼2,500 lines of Python) that bridges designer-authored 3D scenes and the BioVLN benchmark format. LSAT implements two complementary goal-extraction strategies. The first extracts world-space coordinates from artist-placed target_* empty nodes in the GLTF file, mapping node names to instrument categories through fuzzy string matching (Levenshtein distance threshold of 2). The second computes goal positions from detected instrument meshes using their centroids and operational-face geometry (Eq. 1), serving as a fallback when target markers are absent. LSAT achieved 100% coverage on the human-annotated test scene and exports to BioVLN goal format, LSAT internal format, and Isaac Sim USD.
4
Experimental Setup
4.1
Benchmark Datasets
Evaluation Metrics
4.3
Baseline Methods
Six methods spanning random actions to VLM-guided semantic exploration, all operating in the same action space A with a 500-step maximum. Random. Uniformly samples from the three movement actions (forward, turn left, turn right) with a 5% per-step stop probability. Serves as a pure-chance lower bound. Oracle. Follows the pre-computed geodesic shortest path on the navigation mesh. Because navmesh-snapped goals may be unreachable under discrete actions (0.25 m forward, 30◦ turns), the Oracle does not achieve 100% success; it provides an empirical upper bound. Frontier Exploration. A purely geometric method using a depth-based 2D occupancy grid (0.05 m resolution, 100×100 m extent), frontier clustering, and nearestfrontier navigation. This baseline tests whether systematic
We define three benchmark datasets (Table 4): Multi-Scene spans 7 held-out scenes (4 DEV, 1 VAL, 2 TEST) drawn from 47 total built scenes across 2 layout templates, with 2–4 instrument categories per scene. The three-way split ensures no scene appears in more than one split, testing generalization. Two-Room contains two connected rooms joined by a 1.5 m-wide central corridor, with 10 instrument categories and 50 episodes specifically testing cross-room navigation. LSAT Target is a humanannotated scene with 8 instrument categories and LSAT 6
geometric coverage alone suffices for laboratory navigation, without any semantic understanding of instrument appearance or location. LLM-Frontier. Extends Frontier Exploration by querying DeepSeek-chat Xu et al. (2026) when multiple frontiers are approximately equidistant. The LLM receives a structured prompt describing the target instrument, agent position, and top-3 frontier candidates, then selects which to pursue. This baseline tests whether an LLM’s semantic priors about instrument locations improve frontier selection in laboratory layouts. 3-Zone Oracle. An oracle agent that samples multiple candidate approach points within the target instrument’s Zone 3, navigating to each until one succeeds. This baseline isolates the benefit of zone-aware multi-point goal Figure 4: Safety boundary: red dashed violates instrument sampling over single-point navmesh snapping. clearance; blue solid maintains safe clearance. VLFM. Our reimplementation of VLFM Yokoyama et al. (2024) with vlm_vmap mode: Grounding DINO Liu et al. (2024) for open-vocabulary detection, GPT-4o Hurst et al. (2024) for detection verification and direction scor- 5 Experiments ing, and SAM Kirillov et al. (2023) for segmentation. The agent early-stops after 100 consecutive steps without 5.1 Geometric Coverage Is Sufficient for progress. VLFM tests whether pretrained visual represenSingle-Room Layouts tations from VLM-guided zero-shot navigation transfer to rendered laboratory assets. Table 5 reports all metrics. Frontier Exploration achieves 74.4% SR on MS-DEV, surpassing the geodesic Oracle (71.9%) by 2.5 pp (McNemar 𝑝=0.219); on Two-Room, 4.4 Implementation Details Frontier reaches 84.0% vs. Oracle’s 80.0% (+4.0 pp, 𝑝=0.500). Systematic geometric coverage compensates All experiments run on a single NVIDIA RTX 4090 for discretization error in reaching navmesh-snapped goals (24 GB) with habitat-sim 0.2.5. Frontier Exploration and under 0.25 m steps and 30◦ turns. 3-Zone Oracle achieves LLM-Frontier operate at ∼5 steps/s; VLFM at ∼0.25 the highest SR overall (83.3–92.5%), confirming that zonesteps/s due to VLM API latency. GPT-4o and DeepSeek aware multi-point sampling further compensates for disare accessed via API endpoints. LSAT is developed for cretization. LLM-Frontier matches Frontier on 4 of 5 Blender 5.1.2 Blender Foundation (2018). splits; the LLM is consulted in only 10–20% of episodes Beyond zero-shot evaluation, BioVLN supports trajec- because single-room layouts lack room-level semantic tory collection and policy learning through a trajectory choices. VLFM achieves 62.5% SR on LSAT Target but recorder that captures per-step observations during evalu- drops to 19.4% on MS-DEV; on Two-Room, all successes ation, a PyTorch dataset class that computes agent-centric are confined to Room A instruments. goal vectors, and a Gymnasium environment with stepwise reward: 𝑟 𝑡 = (𝑑𝑡 −1 − 𝑑𝑡 ) + I[𝑑𝑡 < 𝑟 𝑖 ] · 10 − 0.01 − I[𝑑clearance < 0.5] · 0.05
5.2
(5)
Per-Instrument Difficulty Reflects Spatial Layout
Table 6 disaggregates Frontier Exploration by instrument category. Wall instruments (refrigerator: 60–65%; cabinet: 70–100%) are harder than table-top instruments (centrifuge: 80–100%), whose approach paths benefit from open space at the room center. LSAT Target’s higher SR (87.5%) confirms that human annotators avoid placing goals near obstacles.
where 𝑑𝑡 is the Euclidean distance to goal at step 𝑡, 𝑟 𝑖 is the instrument-specific success radius, and 𝑑clearance is the minimum distance to any instrument surface. The first term provides dense reward for approaching the goal; the second awards a sparse success bonus; the step penalty encourages efficient paths; the safety term penalizes proximity below the Zone 2 threshold. 7
Table 5: Navigation and safety results across all benchmarks. MS VAL
MS TEST
Method
SR
SPL
MS DEV MCR
VRT
SR
SPL
SR
SPL
SR
SPL
Two-Room MCR
VRT
SR
LSAT MCR
VRT
Random Oracle Frontier LLM-Frn. 3-Zone Or. VLFM
26.9 71.9 74.4 74.4 83.8 19.4
0.212 0.719 0.744 0.744 0.838 —
0.976 0.944 0.918 — 1.115 1.853
11.9 9.7 9.8 — 4.9 4.7
22.0 76.0 80.0 84.0 84.0 38.0
0.149 0.760 0.800 0.839 0.840 —
31.7 79.2 82.5 82.5 83.3 —
0.270 0.792 0.825 0.825 0.833 —
10.0 80.0 84.0 84.0 86.0 28.0
0.066 0.800 0.840 0.840 0.860 —
0.637 0.563 0.504 — 0.672 0.822
12.7 10.7 11.3 — 6.3 3.8
32.5 77.5 87.5 87.5 92.5 62.5
0.135 0.124 0.104 — 0.110 2.165
55.3 68.2 70.3 — 61.3 5.3
SR: success rate (%). SPL: success weighted by path length. MCR: min. clearance radius (m). VRT: violation rate (%). VLFM SPL N/A (VLM paths not geodesic-comparable). TEST omitted (API limits). MS-VAL safety: MCR=1.07, VRT=18.2. LLM-Frontier safety: — = identical to Frontier (same geometric path).
Table 6: Frontier SR by Instrument Category (%)
Table 7: VLM Recognition vs. Surface Color Diversity
Category
MS-DEV
LSAT
Two-Room
Instrument
Cabinet Centrifuge Incubator Lab Bench Lab Table Mini-Centrifuge Pipette Refrigerator Waste Bin
70.0 80.0 — — — — — 65.0 82.5
80.0 — 100.0 100.0 80.0 100.0 60.0 80.0 100.0
100.0 100.0 80.0 100.0 60.0 80.0 80.0 60.0 80.0
Overall
74.4
87.5
84.0
Correct 𝑛 3/3 0/3 0/6 1/6 0/6 0/6
3 3 6 6 6 6
Table 8: Centroid vs. Operational-Face Goals Centroid
Op-Face
Δ
𝑝
DEV (160) VAL (50) TEST (120)
82.5 62.0 79.2
74.4 80.0 82.5
+8.1 −18.0 −3.3
0.035 0.035 0.557
Overall
78.2
78.2
0.0
—
Split (𝑛)
Safety Depends on Layout Density
Fig. 4 shows an example trajectory with safety violations. Safety is dominated by layout density (Table 5). On the compact LSAT layout, VRT reaches 55.3–70.3% for Frontier and Oracle, spending more than half the episode within 0.5 m of instruments. On Two-Room, VRT drops to 6.3–11.3% because the corridor separates navigation paths from instrument zones. VLFM achieves the lowest VRT but also the lowest SR, revealing a success-safety trade-off. 3-Zone Oracle achieves the lowest VRT among high-SR methods by constraining navigation endpoints to Zone 3.
5.4
GPT-4o Answer
Biosafety Cab. 12.6k–14.9k “Biosafety cabinet” Incubator 6.1k–7.6k “Laboratory freezer” Centrifuge 1.9k–14.0k “Yellow trash bin” Micropipette 1.8k–19.8k “Lab equipment” Refrigerator 1–1.8k “Gray rectangle” Waste Bin 1–2 “Gray square”
— : instrument category not present in that dataset.
5.3
Colors
5.5
Operational-Face Goals Are LayoutInvariant
Frontier Exploration with centroid vs. operational-face goals on the combined 330-episode Multi-Scene set yields identical aggregate SR (78.2%, Table 8). Aggregate parity masks opposing per-split effects: centroid improves DEV (+8.1 pp, 𝑝=0.035) yet degrades VAL (−18.0 pp, 𝑝=0.035). Each formulation wins 38 episodes the other loses. Operational-face goals avoid this layout-dependent fluctuation because the goal is defined relative to the oriented face, not the centroid projection.
VLM Analysis 5.6
GPT-4o identifies all instruments from text yet only 13.3% from rendered views (Table 7). Recognition follows surface color diversity: 3/3 for the biosafety cabinet (12.6k– 14.9k colors), 0/6 for near-uniform surfaces. VLM failure stems from sparse textures, not deficient knowledge.
Behavioral Cloning Baseline
A behavioral cloning policy trained on Frontier trajectories achieves 41.9% SR on held-out VAL (vs. Frontier’s 80.0%), establishing a reference for future learned methods. 8
6
Conclusion
simulation-to-real embodied ai platform. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 3164–3174.
We presented BioVLN, a simulation platform for biomedical laboratory navigation built on operational-face goals and a three-zone operational envelope. Frontier Exploration (74.4–87.5% SR) approaches the oracle bound through geometric coverage. 3-Zone Oracle (83.3–92.5%) achieves the highest success by sampling within the operational envelope. VLFM (19.4–62.5%) reveals a surfacetexture bottleneck: GPT-4o identifies all instruments from text but recognizes only 13.3% from rendered views. A behavioral cloning baseline (41.9% SR) confirms the training pipeline. Key limitations and future directions: Asset visual quality. Flat-shaded materials limit VLM recognition; richer textures would bridge this domain gap. Scene diversity. Texture randomization and clutter would produce more realistic laboratory scenes. Domain generalization. The operational-face model extends to any domain with affordance-directed surfaces. Sim-to-real transfer. Deploying trained policies to physical robots remains an open direction.
Deitke, M.; VanderBilt, E.; Herrasti, A.; Weihs, L.; Ehsani, K.; Salvador, J.; Han, W.; Kolve, E.; Kembhavi, A.; and Mottaghi, R. 2022. ProcTHOR: Large-Scale Embodied AI Using Procedural Generation. Advances in Neural Information Processing Systems, 35: 5982–5994. Holland, I.; and Davies, J. A. 2020. Automation in the life science research laboratory. Frontiers in bioengineering and biotechnology, 8: 571777. Hurst, A.; Lerer, A.; Goucher, A. P.; Perelman, A.; Ramesh, A.; Clark, A.; Ostrow, A.; Welihinda, A.; Hayes, A.; Radford, A.; et al. 2024. Gpt-4o system card. arXiv preprint arXiv:2410.21276. Kirillov, A.; Mintun, E.; Ravi, N.; Mao, H.; Rolland, C.; Gustafson, L.; Xiao, T.; Whitehead, S.; Berg, A. C.; Lo, W.-Y.; et al. 2023. Segment anything. In Proceedings of the IEEE/CVF international conference on computer vision, 4015–4026.
Kolve, E.; Mottaghi, R.; Han, W.; VanderBilt, E.; Weihs, L.; Herrasti, A.; Deitke, M.; Ehsani, K.; Gordon, D.; Zhu, Y.; et al. 2017. Ai2-thor: An interactive 3d enviBatra, D.; Gokaslan, A.; Kembhavi, A.; Maksymets, O.; ronment for visual ai. arXiv preprint arXiv:1712.05474. Mottaghi, R.; Savva, M.; Toshev, A.; and Wijmans, E. 2020. Objectnav revisited: On evaluation of em- Kuang, Y.; Lin, H.; and Jiang, M. 2024. Openfmnav: Tobodied agents navigating to objects. arXiv preprint wards open-set zero-shot object navigation via visionarXiv:2006.13171. language foundation models. In Findings of the Asso-
References
ciation for Computational Linguistics: NAACL 2024, 338–351.
Blender Foundation. 2018. Blender-a 3D modelling and rendering package.
Burger, B.; Maffettone, P. M.; Gusev, V. V.; Aitchison, Li, C.; Xia, F.; Martín-Martín, R.; Lingelbach, M.; Srivastava, S.; Shen, B.; Vainio, K.; Gokmen, C.; Dharan, G.; C. M.; Bai, Y.; Wang, X.; Li, X.; Alston, B. M.; Li, Jain, T.; et al. 2021. igibson 2.0: Object-centric simuB.; Clowes, R.; et al. 2020. A mobile robotic chemist. lation for robot learning of everyday household tasks. Nature, 583(7815): 237–241. arXiv preprint arXiv:2108.03272. Chang, A.; Dai, A.; Funkhouser, T.; Halber, M.; Niessner, M.; Savva, M.; Song, S.; Zeng, A.; and Zhang, Y. Li, J.; Li, D.; Savarese, S.; and Hoi, S. 2023. Blip-2: Bootstrapping language-image pre-training with frozen 2017. Matterport3d: Learning from rgb-d data in indoor image encoders and large language models. In Internaenvironments. arXiv preprint arXiv:1709.06158. tional conference on machine learning, 19730–19742. Chaplot, D. S.; Gandhi, D.; Gupta, S.; Gupta, A.; and PMLR. Salakhutdinov, R. 2020a. Learning to explore using Liu, S.; Zeng, Z.; Ren, T.; Li, F.; Zhang, H.; Yang, J.; Jiang, active neural slam. arXiv preprint arXiv:2004.05155. Q.; Li, C.; Yang, J.; Su, H.; et al. 2024. Grounding dino: Chaplot, D. S.; Salakhutdinov, R.; Gupta, A.; and Gupta, S. Marrying dino with grounded pre-training for open-set 2020b. Neural topological slam for visual navigation. In object detection. In European conference on computer Proceedings of the IEEE/CVF conference on computer vision, 38–55. Springer. vision and pattern recognition, 12875–12884. Long, Y.; Cai, W.; Wang, H.; Zhan, G.; and Dong, H. Deitke, M.; Han, W.; Herrasti, A.; Kembhavi, A.; Kolve, 2024. Instructnav: Zero-shot system for generic inE.; Mottaghi, R.; Salvador, J.; Schwenk, D.; VanderBilt, struction navigation in unexplored environment. arXiv E.; Wallingford, M.; et al. 2020. Robothor: An open preprint arXiv:2406.04882. 9
Majumdar, A.; Aggarwal, G.; Devnani, B. S.; Hoffman, J.; Yin, H.; Xu, X.; Wu, Z.; Zhou, J.; and Lu, J. 2024. Sg-nav: and Batra, D. 2022. Zson: Zero-shot object-goal naviOnline 3d scene graph prompting for llm-based zerogation using multimodal goal embeddings. In Advances shot object navigation. Advances in neural information in neural information processing systems. processing systems, 37: 5285–5307. Ramakrishnan, S. K.; Chaplot, D. S.; Al-Halah, Z.; Malik, Yitzhak Gadre, S.; Wortsman, M.; Ilharco, G.; Schmidt, J.; and Grauman, K. 2022. Poni: Potential functions for L.; and Song, S. 2022. CoWs on Pasture: Baselines objectgoal navigation with interaction-free learning. In and Benchmarks for Language-Driven Zero-Shot ObProceedings of the IEEE/CVF Conference on Computer ject Navigation. arXiv e-prints, arXiv–2203. Vision and Pattern Recognition, 18890–18900. Yokoyama, N.; Ha, S.; Batra, D.; Wang, J.; and Bucher, Ramakrishnan, S. K.; Gokaslan, A.; Wijmans, E.; B. 2024. Vlfm: Vision-language frontier maps for zeroMaksymets, O.; Clegg, A.; Turner, J.; Undersander, shot semantic navigation. In 2024 IEEE International E.; Galuba, W.; Westbury, A.; Chang, A. X.; et al. Conference on Robotics and Automation (ICRA), 42– 2021. Habitat-matterport 3d dataset (hm3d): 1000 large48. IEEE. scale 3d environments for embodied ai. arXiv preprint arXiv:2109.08238. Savva, M.; Kadian, A.; Maksymets, O.; Zhao, Y.; Wij- A Per-Method Detailed Metrics mans, E.; Jain, B.; Straub, J.; Liu, J.; Koltun, V.; Malik, J.; et al. 2019. Habitat: A platform for embodied ai re- Table 9 reports Success Rate (SR, %), Distance to Goal search. In Proceedings of the IEEE/CVF international (DTG, m), and average steps per successful episode for conference on computer vision, 9339–9347. all methods evaluated on the Multi-Scene dataset. DTG measures the Euclidean distance between the agent’s final Singh Chaplot, D.; Gandhi, D.; Gupta, A.; and Salakhut- position and the operational-face goal. Average steps are dinov, R. 2020. Object Goal Navigation using Goal- computed over successful episodes only. Oriented Semantic Exploration. arXiv e-prints, arXiv– Table 10 reports metrics for LSAT Target and Two2007. Room datasets. Due to differences in evaluation infrastrucSrivastava, S.; Li, C.; Lingelbach, M.; Martín-Martín, R.; ture, per-episode DTG and step counts are not available for Xia, F.; Vainio, K. E.; Lian, Z.; Gokmen, C.; Buch, S.; the Random and Oracle baselines on these datasets; wallLiu, K.; et al. 2022. Behavior: Benchmark for everyday clock elapsed time is reported instead. household activities in virtual, interactive, and ecological environments. In Conference on robot learning, 477–490. PMLR.
B
Per-Difficulty Performance
Episodes are stratified by geodesic distance 𝑑geo : easy (𝑑geo < 3.0 m), medium (3.0 ≤ 𝑑geo ≤ 6.0 m), and hard (𝑑geo > 6.0 m). On Multi-Scene DEV (160 episodes), the class distribution is 11.9% easy, 28.1% medium, and 60.0% hard. Table 11 reports Frontier and 3-Zone Oracle SR disaggregated by difficulty across Multi-Scene splits. Xia, F.; Zamir, A.; He, Z.-Y.; Sax, A.; Malik, J.; and Both methods achieve uniformly high SR for easy Savarese, S. 2018. Gibson env: Real-world perception episodes. For hard episodes, performance drops to 66.7– for embodied agents. arXiv preprint arXiv:1808.10654. 82.3%, consistent with the expectation that geodesic distance is the primary source of navigation difficulty in Xu, A.; Lin, B.; Xue, B.; Wang, B.; Xu, B.; Wu, B.; Zhang, single-room laboratory layouts. B.; Lin, C.; Dong, C.; Ling, C.; et al. 2026. Deepseekv4: Towards highly efficient million-token context intelligence. arXiv preprint arXiv:2606.19348. C Centroid vs. Operational-Face Wijmans, E.; Kadian, A.; Morcos, A.; Lee, S.; Essa, I.; Parikh, D.; Savva, M.; and Batra, D. 2020. DD-PPO: Learning Near-Perfect PointGoal Navigators from 2.5 Billion Frames. In International Conference on Learning Representations (ICLR).
Yamauchi, B. 1997. A frontier-based approach for autonomous exploration. In Proceedings 1997 IEEE International Symposium on Computational Intelligence in Robotics and Automation CIRA’97.’Towards New Computational Principles for Robotics and Automation’, 146–151. IEEE.
Ablation Frontier Exploration was evaluated under two goal formulations on Multi-Scene (330 episodes). In the centroid condition, goals are defined as the floor projection of the instrument bounding-box center, consistent with standard
10
Table 9: Detailed Navigation Metrics on Multi-Scene DEV
VAL
TEST
Method
SR
DTG
Steps
SR
DTG
Steps
SR
DTG
Steps
Random Oracle Frontier LLM-Frontier 3-Zone Oracle
26.9 71.9 74.4 74.4 83.8
5.20 2.96 2.98 2.97 3.87
410.1 157.1 145.5 145.5 97.0
22.0 76.0 80.0 84.0 84.0
4.03 2.31 2.15 2.43 2.85
426.9 133.1 114.5 105.5 94.7
31.7 79.2 82.5 82.5 83.3
3.99 2.75 2.82 2.82 3.15
382.3 117.3 101.5 101.5 95.6
Table 10: Detailed Metrics on LSAT Target and Two-Room LSAT Target (40 eps)
Two-Room (50 eps)
Method
SR
Elapsed (s)
SPL
SR
Elapsed (s)
SPL
Random Oracle Frontier 3-Zone Oracle
32.5 77.5 87.5 92.5
1.6 1.5 7.4 1.5
— 0.775 0.875 0.925
10.0 80.0 84.0 86.0
2.1 1.7 14.1 1.6
— 0.800 0.840 0.860
Table 11: SR (%) by Difficulty Tier Frontier Difficulty
DEV
VAL
TEST
Easy Medium Hard
89.5 84.4 66.7
100.0 73.7 70.6
100.0 83.3 68.9
rendered simulation frames. Six laboratory instruments were rendered at 1024×768 pixels under the default 3-Zone Oracle Habitat-sim no_lights rendering mode, with no postDEV VAL TEST processing. For each instrument, 3–6 distinct viewpoints were captured from the operational-face goal position, ori100.0 100.0 100.0 ented toward the instrument. 97.8 74.0
73.7 82.3
83.3 71.1
GPT-4o (gpt-4o-2024-08-06) was queried per viewpoint: “What laboratory instrument is shown in this image? Answer with the instrument name only.” Unique surObjectNav. In the operational-face condition, goals lie at face colors were computed by counting distinct RGB valthe outer boundary of Zone 3 (Eq. 1, main text). McNeues within the instrument’s segmented region (background mar’s test for paired binary outcomes was applied per split. removed via SAM mask). The opposing per-split effects correspond to instrument placement. On DEV, centroid goals raise refrigerator SR A text-only control confirmed that GPT-4o identifies all (85.0% vs. 65.0%) by removing the approach-direction six instruments by name, demonstrating complete concepconstraint; the agent can reach the refrigerator from any tual knowledge. Recognition from rendered images follows side. On VAL, centroid goals degrade refrigerator SR surface color diversity (Table 8, main text): the biosafety (40.0% vs. 100.0%) because the projected centroid lies cabinet (12.6k–14.9k unique colors; visible vents, seams, behind a wall and is unreachable on the navigation mesh. and labels) is correctly identified in 3/3 viewpoints; inAcross the two splits, the two effects cancel at the aggrestruments with near-uniform surfaces (refrigerator: 1–1.8k gate level (78.2% each). Cabinet and centrifuge exhibit colors, waste bin: 1–2 colors) are classified as generic geno material difference between conditions, as their placeometric shapes in 0/6 viewpoints. ment positions project to navigable floor locations in both formulations. Two rendering factors influence the degree of texture sparsity. First, the no_lights rendering mode supdynamic lighting, reducing per-instrument color D VLM Recognition Experiment presses diversity by a factor of 3–5× compared to lit rendering. Details Second, gamma correction applied during frame extraction further homogenizes pixel intensity. After correcting A recognition experiment was conducted to determine the lighting pipeline, recognition improved for instruments whether VLFM navigation failures originate from defi- with moderate texture (incubator, centrifuge) but not for cient domain knowledge or insufficient visual texture in near-uniform surfaces (refrigerator, waste bin). 11
Table 12: Centroid vs. Operational-Face Goals — Per-Split Comparison Centroid SR
Op-Face SR
Δ (pp)
𝑝 (McNemar)
Cent. Wins
OpF. Wins
DEV (160) VAL (50) TEST (120)
82.5 62.0 79.2
74.4 80.0 82.5
+8.1 −18.0 −3.3
0.035 0.035 0.557
28 3 7
15 12 11
Overall (330)
78.2
78.2
0.0
—
38
38
Split (𝑛)
Cent. Wins (OpF. Wins): episodes where centroid (operational-face) formulation succeeds and the other fails.
E
McNemar Test Formulation
G
McNemar’s test for paired nominal data evaluates whether the per-episode success/failure patterns of two methods ( 𝐴, 𝐵) differ in distribution. For a given data split, let 𝑛00 denote episodes where both methods fail, 𝑛01 where 𝐴 fails and 𝐵 succeeds, 𝑛10 where 𝐴 succeeds and 𝐵 fails, and 𝑛11 where both succeed. The test statistic is:
Per-Method Experiment Configuration
Table 15 specifies the complete configuration for each baseline method evaluated in the main text. All methods share the action space A = {forward 0.25m, turn_left 30◦ , turn_right 30◦ , stop} (except BioVLNGym training, which uses 10◦ turns as noted below) and a 500-step episode limit. Habitat-sim (|𝑛01 − 𝑛10 | − 1) 2 2 (6) rendering is configured at 640 × 480 RGB-D with 𝜒 = 𝑛01 + 𝑛10 no_lights mode; the no_lights mode flattens lighting but provides consistent color across viewpoints, with one degree of freedom. When 𝑛01 +𝑛10 < 10, the exact which is relevant when interpreting VLM recognition binomial test is used. All reported 𝑝-values are two-sided. results.
F
H Training Pipeline Architecture Scene Generation and LSAT ArH.1 Data Collection chitecture
Table 14 summarizes the per-template scene generation statistics for the BioVLN platform. The single-room template randomizes 2–5 of 6 instrument slots per scene; the two-room template uses 10 fixed instrument slots across two connected rooms. The LSAT (LabScene Annotation Toolkit) addon comprises 12 Python modules in three layers. The parsing layer handles GLB deserialization, GLTF node traversal, and Blender-to-Habitat coordinate transforms. The extraction layer implements two strategies: Strategy 1 extracts worldspace coordinates from artist-placed target_* empty nodes via fuzzy string matching (Levenshtein distance threshold of 2) and maps matched names to instrument categories; Strategy 2 detects instrument meshes by name matching, computes centroids and bounding-box dimensions, and applies Eq. 1 for goal computation. The export layer generates BioVLN goal JSON, an internal intermediate format, and USD files for Isaac Sim. When Strategy 2 cannot determine an operational face direction from geometry alone, the most frequent configuration for that instrument category is used as a default (derived from the instrument catalog, Table 2 of the main text).
Trajectories are recorded by attaching a TrajectoryRecorder callback to any BioVLNPolicy during evaluation. The recorder is non-invasive: it wraps the existing evaluation loop without modifying biovln_eval.py. Each episode produces one gzip-compressed JSONL entry containing per-step records with the following fields: Field
Description
step_idx action
Step index within episode Action: 0=STOP, 1=FORWARD, 2=LEFT, 3=RIGHT Agent position [𝑥, 𝑦, 𝑧] (Y-up) Heading vector [ℎ 𝑥 , ℎ 𝑧 ] Goal position [𝑥, 𝑧] PNG path (when storing images) Base64-encoded depth array Distance to goal (m) Episode success flag (bool)
obs.position obs.heading obs.goal_position obs.rgb_path obs.depth dist_to_goal success
The recorder supports two modes controlled by compress_images: (a) full mode stores RGB observations as PNG files in an images/ subdirectory alongside the JSONL gzip, producing ∼8 MB per thousand steps; (b)
12
Table 13: Pairwise McNemar Test Results Method A
Method B
Split
𝑛01
𝑛10
𝑝
Frontier Frontier Frontier Frontier LLM-Frontier
Oracle Oracle Oracle Oracle Frontier
MS DEV MS VAL MS TEST Two-Room MS VAL
20 8 16 5 2
16 6 12 3 0
0.619 0.791 0.572 0.727 —
𝑛01 : 𝐴 fails & 𝐵 succeeds. 𝑛10 : 𝐴 succeeds & 𝐵 fails. LLM-Frontier vs. Frontier on MS VAL: 𝑛01 +𝑛10 <10, test not applicable.
• Input: 3×224×224 RGB, normalized with ImageNet 𝜇/𝜎, augmented with random horizontal flip and ±5% brightness jitter
Table 14: Scene Generation by Layout Template Template
Instr.
Furn.
Active Slots
Episodes
single_room two_room
4 10
3 3
2–5 10
1,363 304
Total
—
—
—
1,667
• Training: Adam optimizer (lr=10−4 , weight decay 10−5 ), batch size 32, cross-entropy loss, 500 epochs with early-stopping (patience 50 epochs on validation loss)
single_room: 35 (2 active), 30 (3), 40 (4–5) eps/scene. two_room: 38 eps/scene, all 10 slots active in every scene.
• Hardware: Single NVIDIA RTX 4090 (24 GB), training time ∼2 hours
trace-only mode omits image payloads and stores only The trained BC policy achieves 41.9% SR (0.325 SPL) position/action/goal traces, reducing per-episode size to ∼50 KB and enabling fast training with goal-vector-only on unseen Multi-Scene VAL scenes versus Frontier’s 80.0% zero-shot, a gap expected for BC operating with inputs. single-frame RGB input and no mapping. Per-category breakdown on VAL: Cabinet 33.3%, Centrifuge 50.0%, H.2 Dataset Construction and Training Refrigerator 42.9%, Waste Bin 40.0% (Frontier zero-shot: TrajectoryDataset loads one or more recording 80.0%, 80.0%, 80.0%, 80.0% respectively). directories and merges them into a unified PyTorch Dataset. For each step, it constructs a four-dimensional H.3 Gym Environment for RL Training agent-centric goal vector: BioVLNGym implements the standard Gymnasium Env • [𝑣 0 , 𝑣 1 ]: unit vector from agent to goal in agent-local interface with the reward function defined in Eq. 5 (main coordinates text). Key configuration differences from zero-shot evaluation: • [𝑣 ]: Euclidean distance to goal (m) 2
• Action granularity: 10◦ turns (vs. 30◦ in evaluation), providing finer control for RL exploration
• [𝑣 3 ]: progress ratio clip(dist/5.0, 0, 1) The dataset supports configurable observation transforms (torchvision Resize, Normalize with ImageNet statistics) and optional position noise for data augmentation. For our behavioral cloning demonstration, we used: • Training split: 487 successful Frontier Exploration trajectories from Multi-Scene DEV (119 eps) and Two-Room (42 eps), totaling 7,142 forward/turn steps (STOP actions excluded) • Validation: Multi-Scene VAL (50 episodes, held-out scenes) • Architecture: ResNet-18 (pretrained on ImageNet) → 512-dim feature → concat with 4-dim goal vector → 128-dim FC (ReLU) → 4-way softmax 13
• Episode resampling: Episodes are shuffled at initialization and iterated cyclically; reset() advances to the next episode, enabling continuous training without manual episode scheduling • Observation normalization: Position coordinates normalized to [−1, 1] by dividing by 4.0 m; heading stored as unit vector • Safety penalty: Proportional to clearance deficit below Zone 2 threshold (0.5 m), scaled by 1.0 × (1 − clearance/0.5) • Compatibility: Directly usable with StableBaselines3 PPO, A2C, and SAC; also compatible with
Table 15: Per-Method Experiment Configuration Method
Core Parameters
External Dependencies
Runtime
Random
Stop probability: 5% per step
None
< 0.01 s/step
Oracle
Navmesh shortest-path follower, goal radius: 0.8–1.3 m (instru.-specific)
Recast navmesh
< 0.01 s/step
Frontier
Occupancy grid: 0.05 m res., 100×100 m extent; cluster radius: 1.0 m; min frontier size: 10 cells; depth clip: 10 m
None (geometry-only)
∼ 0.2 s/step
LLM-Frontier
Frontier params identical to above; equidistance threshold: 1.0 m; top-𝑘=3 candidates
DeepSeek-chat API
∼ 1.5 s/step (incl. API)
3-Zone Oracle
Zone-3 samples: 8 candidate points per instrument; grid spacing: 0.15 m; fallback: centroid projection
Recast navmesh
< 0.01 s/step
VLFM
vlm_vmap mode; value map resolution: 0.05 m; detection interval: every 5 steps; earlystop: 100 no-progress steps
GPT-4o (gpt-4o-2024-08-06), Grounding DINO (Swin-T), SAM (ViT-H)
∼ 4 s/step (VLM bottleneck)
BC (training)
ResNet-18 backbone (pretrained ImageNet); hidden: 128-dim FC + ReLU; action head: 4-way softmax; optimizer: Adam (lr=10−4 , wd=10−5 ); batch_size: 32; epochs: 500; early-stop patience: 50; input: 3×224×224 RGB + 4-dim goal vector
PyTorch 2.x, torchvision
∼ 2 h training on RTX 4090
BioVLNGym uses finer action granularity (10◦ turns) to support RL exploration; evaluation uses 30◦ turns for comparability.
CleanRL and Ray RLlib via the standard gym.Env Table 16: Per-Category Safety Metrics — Frontier Exploration on MS DEV protocol
I
Per-Category Safety Analysis
Table 16 disaggregates MCR and VRT by instrument category for Frontier Exploration on Multi-Scene DEV. The data reveal that safety outcomes are instrument-dependent: larger wall instruments (refrigerator, cabinet) force navigation paths through narrower corridors, producing lower MCR; table-top instruments (centrifuge) sit on benches elevated 0.76–0.78 m above floor level, and the agent navigating at floor height experiences effectively larger clearance. Refrigerators exhibit the worst-case safety profile: the lowest SR (65.0%) and highest VRT (9.6%) among all categories. This reflects corner placement in the single-room template, where the approach path passes near adjacent wall instruments. Cabinets show elevated VRT (14.7%)
Category
Eps. SR (%) MCR (m) VRT (%) Steps
Cabinet Centrifuge Refrigerator Waste Bin
40 40 40 40
70.0 80.0 65.0 82.5
0.869 0.797 0.976 0.933
14.7 6.3 9.6 8.4
152.3 118.7 165.4 145.8
Overall
160
74.4
0.918
9.8
145.5
despite moderate MCR (0.869 m), indicating that while minimum clearance is rarely violated, the approach trajectory passes through regions where multiple Zone 2 boundaries overlap.
14
J
Scene Generation Reproducibility tion (incubator: 0/3 to 2/3; centrifuge: 0/6 to 2/6). Near-
BioVLN’s procedural generator uses deterministic random seeds to control all stochastic decisions during scene construction: which instrument slots are active, which instrument categories fill each slot, and small position jitter applied after initial placement. This design enables exact reproduction of any scene from its seed value alone. Table 17 lists the seed values and instrument compositions for the 7 held-out scenes used in Multi-Scene evaluation. All 7 scenes use the single-room template (8×6 m) with the same four instrument categories (cabinet, centrifuge, refrigerator, waste bin). Each seed produces a distinct spatial arrangement; for example, refrigerator placement varies between corner positions (constrained by two adjacent walls) and mid-wall positions (constrained by one wall), directly affecting Frontier Exploration success rates as discussed in the main text. Scenes are divided into DEV (4 scenes, seeds 150–153, 40 episodes each), VAL (1 scene, seed 154, 50 episodes), and TEST (2 scenes, seeds 155–156, 60 episodes each) splits. The full procedural generation configuration is provided in the pipeline code: config.py defines instrument catalogs and layout templates for single-room generation, while config_two_room.py defines the corresponding configuration for the Two-Room benchmark.
K
VLM Domain-Gap Experiment
To isolate whether VLFM navigation failures originate from deficient domain knowledge or insufficient visual signal in rendered frames, we conducted a controlled recognition experiment. For each of the 6 laboratory instruments, we captured 3 to 6 rendered viewpoints from the operational-face goal position at 1024×768 resolution under Habitat-sim no_lights rendering mode. GPT-4o (gpt-4o-2024-08-06) was queried per viewpoint with the prompt: “What laboratory instrument is shown in this image? Answer with the instrument name only.” As a textonly control, GPT-4o was asked to name each instrument from its category label alone, confirming perfect conceptual knowledge (6/6 correct). Table 18 reports the complete per-viewpoint recognition results. The experiment produced 30 instrument-viewpoint pairs, of which only 4 were correctly identified (13.3%). Recognition accuracy is strictly monotonic in surface color diversity. The biosafety cabinet, with 12.6k–14.9k unique RGB colors, visible ventilation grilles, door seams, and warning labels, was recognized in all 3 viewpoints. Instruments with moderate texture (incubator: 6.1k–7.6k colors; centrifuge: 1.9k–14.0k colors) were never correctly identified in the default rendering condition, though recognition improved after enabling lit rendering and gamma correc-
uniform surfaces (refrigerator: 1.0k–1.8k colors; waste bin: 1–2 colors) received zero correct identifications regardless of lighting configuration. The data support two conclusions. First, the recognition gap is a rendering-quality problem, not a knowledge problem—GPT-4o knows what all six instruments are, but cannot identify them from rendered frames when surface texture is sparse. Second, recognition accuracy correlates strongly with color diversity (𝑟=0.84, 𝑝<0.05), with a threshold of ∼5,000 unique colors below which recognition probability drops to near zero. This finding has implications for future embodied AI benchmarks using synthetic assets: surface texture quality directly determines whether VLM-based methods can leverage their pretrained visual knowledge.
L
Behavioral Cloning Per-Category Results
We train a behavioral cloning policy on 487 successful Frontier Exploration trajectories recorded from MultiScene DEV (119 episodes) and Two-Room (42 episodes). The policy architecture uses a ResNet-18 backbone pretrained on ImageNet, with the 512-dimensional feature vector concatenated with a 4-dimensional agent-centric goal vector before a 128-dimensional fully-connected layer with ReLU activation and a 4-way softmax action head. Input frames are resized to 3×224×224, normalized with ImageNet statistics (𝜇=[0.485, 0.456, 0.406], 𝜎=[0.229, 0.224, 0.225]), and augmented with random horizontal flips and ±5% brightness jitter during training. The optimizer is Adam with learning rate 10−4 and weight decay 10−5 , using cross-entropy loss with a batch size of 32. Training runs for 500 epochs with early stopping (patience of 50 epochs on validation loss), requiring approximately 2 hours on a single NVIDIA RTX 4090. Table 19 reports per-category SR on Multi-Scene VAL (50 held-out episodes). The policy was evaluated in the same action space as zero-shot evaluation (30◦ turns, 500step maximum). The performance gap is consistent across categories (−30.0 to −46.7 pp) and reflects a fundamental difference in information access. Frontier Exploration maintains a full occupancy grid with explicit frontier detection accumulated across hundreds of exploration steps; the BC policy receives only the current frame and a four-dimensional goal vector, with no map, no history, and no explicit exploration mechanism. The largest gaps occur for cabinet (−46.7 pp) and waste bin (−40.0 pp), categories whose approach paths are relatively direct in the single-room template. Centrifuge shows the smallest gap (−30.0 pp), likely because its table-top placement produces more distinctive
15
Table 17: Multi-Scene Held-Out Scene Composition Scene Name
Template
Split
Active Instruments
Eps.
lab_s150_v0
single
DEV
40
lab_s151_v0
single
DEV
lab_s152_v0
single
DEV
lab_s153_v0
single
DEV
lab_s154_v0
single
VAL
lab_s155_v0
single
TEST
lab_s156_v0
single
TEST
cabinet, centrifuge, refrigerator, waste bin cabinet, centrifuge, refrigerator, waste bin cabinet, centrifuge, refrigerator, waste bin cabinet, centrifuge, refrigerator, waste bin cabinet, centrifuge, refrigerator, waste bin cabinet, centrifuge, refrigerator, waste bin cabinet, centrifuge, refrigerator, waste bin
Total
40 40 40 50 60 60 330
Each seed deterministically controls slot activation and layout. Config: pipeline/config.py (single-room), pipeline/config_two_room.py (Two-Room).
Table 18: VLM Recognition by Instrument and Viewpoint Instrument
Views
Colors
Corr.
GPT-4o Response
Biosafety Cabinet Incubator Centrifuge Micropipette Refrigerator Waste Bin
3 3 6 6 6 6
12.6k–14.9k 6.1k–7.6k 1.9k–14.0k 1.8k–19.8k 1.0k–1.8k 1–2
3/3 0/3 0/6 1/6 0/6 0/6
“Biosafety cabinet” ×3 “Laboratory freezer” “Yellow trash bin” “Lab equipment” (1×), “Tool” (5×) “Gray rectangle”, “Metal box” “Gray square”, “Floor tile”
Total
30
—
13.3%
Colors = distinct RGB values in SAM-segmented instrument region. Text control: 6/6 correct. Lighting fix: incubator 2/3, centrifuge 2/6.
M
Table 19: BC Policy vs. Frontier Zero-Shot on MS VAL Δ
Category
Eps. BC (%) Frontier (%)
Cabinet Centrifuge Refrigerator Waste Bin
12 12 14 12
33.3 50.0 42.9 40.0
80.0 80.0 80.0 80.0
−46.7 −30.0 −37.1 −40.0
Overall
50
41.9
80.0
−38.1
Available Supplementary Materials
The following supplementary materials are available with the BioVLN release: • Scene maps. 28 top-down PNG visualizations of all held-out scenes with instrument placement, navigation mesh overlay, and operational-face goal positions. Maps are generated via Habitat-sim top-down rendering with the agent height set to 20 m for orthographic projection. Each map includes the Recast navigation mesh as a semi-transparent overlay, allowing visual inspection of navigable regions.
visual context in single-frame observations. These results establish a reference baseline for future learned methods on BioVLN; integrating an explicit map representation, recurrent history, or auxiliary exploration objectives would narrow this performance gap. 16
• Frontier trajectory renders. 12 top-down trajectory visualizations and 12 MP4 videos showing Frontier Exploration trajectories on individual episodes, color-coded by success (green) and failure (red). Renders are captured at 5 fps from an overhead camera
and include the instrument safety boundaries (Zone 2) as dashed circles. • Hard scene analysis. 7 figures analyzing episodes classified as hard (𝑑geo >6.0 m). Each figure shows the start position, operational-face goal, geodesic shortest path, and the occupied and free cells of the agent’s occupancy grid at episode termination. • VLM experiment images. 50 rendered instrument images (5 viewpoints × 6 instruments) at 1024×768 resolution used in the domain-gap recognition experiment. Lighting-fix variants with lit rendering and gamma correction are included for comparison. • Per-episode result JSON. Complete per-episode evaluation data for all 6 methods across 3 datasets. Each JSON record includes per-step safety metrics (minimum clearance, Zone 2 violation flag), full agent trajectory positions, and episode metadata (target category, difficulty tier, geodesic distance). These files allow exact reproduction of every aggregate metric reported in the main text (Tables 5–8) and in this appendix, without re-running the evaluation pipeline. Summary JSONs in each results directory provide pre-computed per-method aggregates. • LSAT addon source. The complete 12-module Blender 5.1.2 Python addon (∼2,500 lines), organized into parsing, extraction, and export layers. The parsing layer handles GLB deserialization, GLTF node traversal, and Blender-to-Habitat coordinate transforms. The extraction layer implements the dualstrategy goal extraction pipeline described in the main text. The export layer generates BioVLN goal JSON, LSAT internal intermediate format, and USD files for Isaac Sim. Each module includes docstrings, usage examples, and inline comments referencing the relevant equations and sections of the paper. The addon is loaded via Blender’s standard addon installation mechanism and registers a panel in the 3D Viewport sidebar.
17