FlowMaps: Modeling Long-Term Multimodal Object Dynamics with Flow Matching Francesco Argenziano1 Miguel Saavedra-Ruiz2,3 Sacha Morin2,3 Charlie Gauthier2,3 Daniele Nardi1 Liam Paull2,3 1
Sapienza University of Rome, Rome, Italy Université de Montréal, Montréal, QC, Canada 3 Mila - Quebec AI Institute, Montréal, QC, Canada 2
arXiv:2606.20209v1 [cs.RO] 18 Jun 2026
{argenziano,nardi}@diag.uniroma1.it {miguel-angel.saavedra-ruiz,sacha.morin}@mila.quebec {charlie.gauthier,paulll}@mila.quebec
Abstract: Joint spatial and temporal understanding of 3D scenes is a crucial requirement for robots deployed in everyday household environments. Such agents must not only comprehend and navigate spatial layouts, but also reason about how these spaces evolve over time. In particular, humans interact with objects daily, causing them to change position throughout the environment and making it difficult for robots to reliably associate current observations with previously seen objects. However, these interactions are not random: human habits and routines induce spatio-temporally consistent patterns in object locations, which robotic agents can potentially learn and then exploit for downstream tasks such as navigation. To this end, we introduce FlowMaps, a latent flow matching model for estimating multimodal distributions over the future locations of dynamic objects in a continuous 3D space. By learning the implicit dependencies among objects and their temporal evolution, FlowMaps predicts likely changes in object locations conditioned on past human interactions, while supporting generalization across previously unseen environments that share similar object routines. To demonstrate the utility of this method, we deploy FlowMaps in a downstream dynamic Object Navigation task in both simulated and real-world environments. Across more than 600 episodes, FlowMaps outperforms state-of-the-art approaches, showing that modeling object dynamics through continuous, multimodal spatio-temporal distributions improves robotic search and navigation in changing household environments. Code and additional material is available at https://fra-tsuna.github.io/flowmaps/. Keywords: Flow matching, Dynamic environments, Object navigation
1
Introduction
Environments inhabited by humans are intrinsically dynamic. This dynamism should not be attributed solely to the people living in them, but also to the everyday objects they contain. Through repeated human interactions, objects are frequently moved, displaced, and relocated over time, posing major challenges for robots expected to deliberate and act in such environments [1, 2]. Under these conditions, even a seemingly simple task (e.g., “help me find my glasses” [3]) becomes highly challenging: although the robot may have observed an object in the past, there is no guarantee that it will still be found at its previous location, and reasoning about its possible displacement requires considering where it may have moved in physical space. Humans constantly move objects within and across rooms, creating significant difficulties for navigation and retrieval. Nevertheless, humans tend to follow repetitive behavioral patterns [4]. For instance, a pair of glasses may be placed on
the nightstand before sleep, moved to the bathroom sink before a shower, and later returned to the nightstand. These routines are referred to as semantically consistent patterns [5, 6], and can potentially be leveraged to recover from failed retrieval attempts by directing the robot toward likely human-induced object placements. We hypothesize that, given sufficient observations of how objects move through environments over time, a generative model can learn latent regularities induced by human routines and exploit them to predict likely future object placements. Crucially, our key insight is that predicting where objects will be found does not require explicitly identifying the human activity that caused their displacement; instead, such structure can emerge directly from data. To be useful beyond a single observed home, these predictions should generalize to environments with different layouts and object arrangements, while remaining grounded in actual object locations rather than fixed, predefined alternatives such as receptacle labels. Since object displacements are inherently multimodal (i.e., they admit multiple modes, each corresponding to a distinct plausible placement), modeling them requires a distributional approach over continuous space. To this end, we present FlowMaps, a latent flow matching (FM) model that recovers multimodal spatio-temporal distributions of common household objects directly in continuous 3D space. Unlike prior dynamic object location models that primarily operate over discrete receptacle-level relations, FlowMaps models human-induced household object relocalization as a multimodal distribution in continuous 3D space. FlowMaps is composed of two main modules: (i) a Variational Autoencoder (VAE) that learns latent representations of object geometry and semantics, and (ii) a latent Diffusion Transformer (DiT) [7] that predicts likely object locations over time. To obtain training data at scale, we employ ProcTHOR [8] to generate meaningful dynamic object trajectories across procedurally generated household environments. We then demonstrate the usefulness of the learned prior on Object Navigation (ObjNav), using it as a representative downstream robotic task among the broader set of applications that can benefit from predictions of likely object placements. In this setting, FlowMaps is trained on dynamic trajectories from a set of training environments, while ObjNav performance is evaluated in disjoint, previously unseen homes, explicitly testing whether the learned prior transfers beyond the scenes observed during training. Across more than 600 ObjNav episodes in ProcTHOR, we compare FlowMaps against state-of-the-art approaches and show superior performance in retrieving target objects. We further validate our approach on a real robotic platform, showing how it can be deployed in a real-world setting. Contributions statement. Our contributions are three-fold: (i) FlowMaps, a FM-based architecture for representing and predicting continuous, multimodal spatio-temporal object distributions in dynamic environments; (ii) as an exemplary downstream robotic application, we demonstrate the use of FlowMaps for ObjNav in dynamic scenes, enabling a robot to reason about where target objects are likely to be found over time, while outperforming both zero-shot baselines and trained expert policies; (iii) an extensive quantitative and qualitative evaluation of both the distributional properties and practical applicability of FlowMaps, covering more than 600 simulated ObjNav episodes in disjoint environments together with real-world deployments, and demonstrating the soundness, effectiveness, cross-environment generalization, and practical viability of the proposed approach.
2
Related Work
Learning human habits and patterns. Learning human habits enables robots to move from reactive command execution to proactive assistance [9, 10]. Prior work models human activities for anticipation [11, 12], uses interaction histories for personalized collaboration [13], and studies anticipatory robot assistance [14, 15, 16]. The closest work to ours is HOMER [17], which uses a Graph Neural Network (GNN) to predict object displacements from recurring household activities. Unlike HOMER, which trains a separate model per household and mainly evaluates within-environment generalization, we train and test on disjoint environments sharing semantically consistent patterns, assessing generalization to unseen household layouts. 2
Object navigation in dynamic environments. ObjNav aims to locate target objects in known or unknown environments. Existing methods often rely on Large Language Models (LLMs) or Visual Language Models (VLMs) for zero-shot reasoning [18, 19], or learn end-to-end policies [20], but typically assume static scenes. Recent dynamic approaches incorporate object trajectories [21, 18], probabilistic object-location estimates [3, 22], or dynamic scene graph memories [23], yet they depend on costly VLM reasoning, hand-designed priors, online estimation, LLM-interpreted activity hints, or explicit graph structures, and have often been evaluated in limited settings. In contrast, FlowMaps models object dynamics as continuous distributions over future object placements in 3D, rather than discrete temporal link prediction over object-location relations [17, 23]. It requires no explicit scene graph structure at training or inference time, remains efficient through its FM formulation, and outperforms prior dynamic ObjNav methods. Flow matching in robotics. Generative models are increasingly used in robotics, supported by large cross-embodiment datasets such as Open-X Embodiment [24]. Recent work uses diffusion models [25, 26], FM [27, 28], and vision-language-action models [29, 30] mainly for policy learning. In contrast, we use FM for posterior inference, recovering spatio-temporal and multimodal distributions over plausible object placements rather than generating actions. To the best of our knowledge, this is the first use of FM for posterior inference in this setting.
3
Background and problem formulation
Vector fields and flows. An Ordinary Differential Equation (ODE) is defined by a time-dependent vector field u : Rd × [0, 1] → Rd , which assigns a velocity ut (x) to every position x ∈ Rd at each time t ∈ [0, 1]. A solution of the ODE is a trajectory X : [0, 1] → Rd , where Xt denotes the position of the system at time t. In this work, we are particularly interested in flows as solutions of the ODE. A flow is a time-dependent map ψ : Rd × [0, 1] → Rd , written ψt (x) = ψ(x, t), whose evolution is governed by d ψt (x) = ut (ψt (x)), dt
ψ0 (x0 ) = x0 .
The vector field ut is said to generate a probability path (pt )0≤t≤1 if the corresponding flow transports samples from the initial distribution p0 to the distribution pt at each time t. Equivalently, for X0 ∼ p0 , we have Xt = ψt (X0 ) ∼ pt . Conditional flow matching. Conditional flow matching (CFM) learns such a vector field by regressing onto tractable conditional velocities. The goal is to learn uθt whose flow transports a simple base distribution pinit to the data distribution pdata . Instead of directly constructing the marginal vector field, namely the vector field that generates the full probability path from pinit to pdata , CFM defines conditional paths pt (· | y) indexed by data samples y ∼ pdata , together with tractable target velocities utarget (· | y). The training objective is t h i 2 LCF M (θ) = Et,x,y uθt (x) − utarget (x | y) 2 , t ∼ U [0, 1], y ∼ pdata , x ∼ pt (· | y). (1) t Under standard assumptions, this objective has the same gradient as the corresponding marginal FM objective, so minimizing (1) learns the marginal vector field without requiring its explicit evaluation. For a more complete explanation and formalization, we refer the reader to [31, 32]. Latent flow matching. In latent flow matching, the same learning objective is applied in a latent representation space. Given an encoder E and decoder D, data samples x ∼ pdata are mapped to latents z = E(x), inducing a latent data distribution pzdata . A vector field uθt is then trained to transport samples from a simple latent prior pzinit to pzdata by minimizing the CFM objective in d Zt = uθt (Zt ), and decodes the terminal latent space. Sampling draws Z0 ∼ pzinit , integrates dt latent variable as x̂ = D(Z1 ). This reduces the dimensionality of the transport problem and lets the flow operate on compact, semantically structured representations, while the decoder maps generated latents back to the data domain. 3
Problem formulation. At time τ ≥ 0, we represent the map as Mτ = (Oτ , OBG ), where O Oτ = {Oi,τ }N i=1 denotes the set of dynamic objects observed in the scene, and OBG denotes the static background, which is assumed to remain unchanged over time. Each dynamic object is represented as Oi,τ = (bi,τ , li ), where bi,τ ∈ R6 denotes the object’s 3D axis-aligned bounding box in center-size format, bi,τ = (cxi,τ , cyi,τ , czi,τ , sxi , syi , szi ), with (cxi,τ , cyi,τ , czi,τ ) denoting the box center at time τ , and (sxi , syi , szi ) denoting its spatial extent. The label li ∈ L denotes the semantic class of the object. In each scene, there is at most one dynamic object per semantic class, so no two dynamic objects share the same label. Background elements in OBG are represented analogously, but their spatial states are fixed across time. Given a prediction horizon ∆τ ≥ 0 and an object query label lq identifying a dynamic object in Oτ , our goal is to infer the distribution over future bounding boxes of the queried object at time τf = τ + ∆τ . Namely, we want to infer the distribution p bq,τf | Mτ , lq , τf .
4
FlowMaps
Directly computing this posterior is intractable, as the queried object’s future state depends on rich scene context and may admit multiple plausible outcomes. We approximate it with FM, learning a conditional transport from a Gaussian base distribution p0 = N (0, I) to the target distribution p1 = p bq,τf | Mτ , lq , τf . Starting from zinit ∼ p0 , integrating the learned flow produces a sample zfinal ∼ p1 , corresponding to a plausible future bounding box of the queried object at time τf . We instantiate this conditional flow in latent space with FlowMaps, a Transformer-based FM model built on the Conditional Diffusion Transformer (CDiT) of Bar et al. [33]. A VAE first encodes object bounding boxes and semantic labels into a latent target space (Section 4.2). A CFM network then learns the flow in this space, predicting future object placements conditioned on the current scene, queried label, and prediction horizon (Section 4.3). Dynamic scene generation and data collection are described in Section 4.1. Architectural, hyperparameter, solver, and interpolation details are provided in the supplementary material. 4.1
Dynamic Scenes Generation and Data Collection
We used ProcTHOR [8] to generate dynamic indoor environments for training both components of FlowMaps. Object movements are driven by predefined human-like routines that produce semantically consistent patterns. We instantiated three representative routines, each capturing a different type of plausible indoor behavior. In Habit #1, the simulated human exhibits location preferences, repeatedly returning to a small set of favored places and therefore spending more time there. In Habit #2, the simulated human follows a balanced routine, distributing time approximately uniformly across the relevant locations in the environment. In Habit #3, the simulated human follows a highly dynamic routine, frequently transitioning between locations and spending only short intervals at each one. For each routine, we generate 2706 training and 918 validation environments. Each scene contains up to 15 dynamic objects moving between semantically compatible receptacles, e.g., a “Fork” can appear on a “Sink” or “DiningTable”, but not on a “ShelvingUnit”. Each environment is simulated for 4 weeks with hourly resolution dτ , yielding 672 timestamps per scene s, with sτ = [τ, Oτ, OBG]. This results in over 1.8M training samples per habit. For each habit, we train and evaluate a separate FlowMaps model. More information on habits can be found in the appendix. 4.2
Variational Autoencoder
We train a VAE to map individual object tokens into the latent space. Each token consists of a normalized 3D bounding box b and a semantic label l, and is encoded into a latent code z ∈ Rdz . Once trained, the VAE is frozen: its encoder eϕ provides the target latents for the CDiT, while its decoder dξ maps generated latents back to object predictions. The decoder is trained to predict both geometry and semantics: its geometry head reconstructs the normalized bounding box parameters b̂, 4
while its semantic head predicts logits over object classes. The network is trained with the standard VAE objective LVAE = Lrec + βt LKL , where βt is linearly annealed during training to prevent posterior collapse [34]. The reconstruction term combines geometric and semantic supervision: Lrec = λCIoU LCIoU +λL1 LL1 +λCE LCE . Here, LCIoU and LL1 supervise the reconstructed bounding box, while LCE supervises the semantic class prediction. 4.3
Latent Flow Matching network
The FlowMaps network parametrizes the velocity field uθt that transports a Gaussian latent z0 to the latent encoding z1 = eϕ (bq , lq ) of the queried object’s future bounding box. It is composed of: (i) a map encoder that aggregates the scene context into a sequence of contextualized tokens, and (ii) a stack of CDiT blocks that iteratively refine the noisy query latent by cross-attending to that context.
Scale
Pointwise Feedforward
Layer Norm
Scale, Shift
Scale
Multi-Head Cross-Attention
Scale, Shift
+
+
AdaLN-Zero
Layer Norm
(a) Map encoder.
Scale, Shift
K, V Layer Norm
Map encoder. At each timestep, the scene is a padded set Mτ of at most NO + NBG object tokens, with maximum length S. Each token represents either a dynamic object or a static background object through a 3D axisaligned bounding box b, semantic label l, and object-type flag fobj . Following Wald et al. [35], this flag acts as a set-membership indicator, where fobj = 1 for dynamic objects and fobj = 0 for furniture/background objects. The map encoder (Fig. 1a) maps Mτ to context tokens Hτ ∈ RS×dh by summing bounding box, class, type, and learned time embeddings, then processing them with NT pre-norm Transformer encoder layers. Self-attention lets each token incorporate scene-level context and relations to other objects. No positional encoding is used across tokens, making the encoder permutation-invariant: spatial structure is provided only by the bounding box embedding.
CDiT Block
(b) CDiT block.
CDiT block. Given Hτ , the flow is implemented by NC CDiT blocks (Fig. 1b) that update the noisy query latent Figure 1: Overview of the latent FM zt ∈ Rdz by cross-attending to the encoded scene. This network components: (a) the map enCDiT-style design, rather than full-attention DiT [7], re- coder, and (b) the CDiT block. flects the asymmetry of the task: a single query object is transported while the scene provides a variable-length context, avoiding repeated quadratic attention over the full scene at each FM step. Cross-attention and feed-forward layers are modulated with adaLN-Zero [7] using a conditioning vector c built from the flow time t ∈ [0, 1], final timestamp τf , and queried label lq . Since the residual stream contains only one query token, we remove the per-block self-attention of Bar et al. [33]; scene-level reasoning is handled by the map encoder, while the CDiT blocks condition the query trajectory through cross-attention. A final adaLN-modulated linear head maps the refined token to the VAE latent space, producing uθt (zt | Hτ , lq , τf ). Training and inference. We train the network with the CFM loss of Eq. 1 on a conditional optimaltransport path [36]. Target latents z1 are obtained by encoding ground-truth future bounding boxes with the frozen VAE encoder eϕ and standardizing them with training-set statistics, while source latents z0 ∼ N (0, Idz ) are sampled independently. For t ∼ T on [0, 1], we set zt = at z1 + bt z0 and regress uθt to żt = ȧt z1 + ḃt z0 with an L2 loss. Source-target pairs are matched by an exact minibatch optimal-transport plan [37, 38], yielding straighter trajectories and fewer integration steps. At inference, given Mτ and (lq , τf ), one map-encoder pass computes the query-independent Hτ . Since Hτ is query-independent, the same encoder pass is amortized across all dynamic objects and ˆt = uθt (ẑt | Hτ , lq , τf ) from z0 to ẑ1 in K fixed steps, posterior samples. We then integrate ż de-standardize ẑ1 , and decode it with dξ into b̂q,τf . Independent z0 draws produce samples from p(bq,τf | Mτ , lq , τf ). The full pipelines are shown in Fig. 2. 5
Interpolant
Robot POV
Inference Vase
Vase
+
Old location
New location
Map Enc
CDiT Blocks
Map Encoder
ODE Solver
Training
Figure 2: Training (left) and inference (right) pipeline for FlowMaps. We train the stack of CDiT blocks to learn the vector field uθt , which is used at inference time to predict b̂q,τf .
5
Experimental Results
Distributional Evaluation. We evaluate FlowMaps using the joint accuracy and distributional metrics reported in Table 1. All entries are averaged over the three considered habits. We report two best-of-K accuracy metrics, following the protocol introduced for multimodal trajectory prediction [39]. For each query, we draw K=50 samples. minFDE@K measures the 3D distance between the ground-truth bounding box center and the closest predicted sample, while Rec@1 reports the corresponding best-of-K recall within δ=1m on the floor plane. To assess the predicted distribution beyond point accuracy, we also report coverage and density following Naeem et al. [40]. Coverage is the fraction of ground-truth positions whose k-Nearest Neighbor ball contains at least one generated sample, whereas density is the average number of such balls occupied by each sample. These metrics separate mode coverage from over-concentration, without penalizing valid multimodal predictions. We further compute TV and JS, respectively the total-variation and Jensen-Shannon divergences between predicted and ground-truth samples discretized as histograms on an xz floor grid with 0.5 m resolution. These divergences measure how spatial mass is allocated across the scene, independently of best-of-K accuracy. We compare FlowMaps against four baselines. FreqPrior estimates a training-set distribution P (r | lq ) over receptacle categories r. At test time, given (Mτ , lq ), it samples a receptacle category, selects an instance of that category in the scene, and samples a point uniformly from the footprint of its top face. This captures scene-conditional object co-occurrence but ignores temporal structure. LLMPrior replaces this empirical distribution with a ranked top-T list of receptacles suggested by an LLM. EmpiricalMean is an oracle point predictor that returns the mean of all ground-truth positions of objects with label lq in the environment, providing a strong upper bound for regression methods with the same (Mτ , lq ) conditioning. MeanFlowMaps collapses FlowMaps samples to their centroid, isolating the effect of multimodality from the conditioning signal. More details on the distributional baselines are provided in the appendix. FlowMaps achieves the best minFDE (0.342 m, 7.6% below the strongest baseline), while nearly saturating mode coverage. It also raises the Naeem density from 0.647 (FreqPrior) to 0.886, a 37% gain, indicating that its samples concentrate more tightly around ground-truth modes. The two single-point baselines are consistently worse on accuracy (minFDE > 2 m, Rec@1 < 0.42); MeanFlowMaps, despite using the same conditioning, is over 7× worse in minFDE, confirming the need for a multimodal generator. FreqPrior matches FlowMaps only in best-of-K recall, whereas FlowMaps also reduces the error tail, increases density, and lowers both TV and JS, showing that it captures where on the plausible surfaces mass concentrates, not just which surfaces are plausible. Object Navigation. We further evaluate FlowMaps on ObjNav, our primary downstream application, using a minival split of 25 random validation environments. For each environment, we select 5 query objects and sample 5 (τ, τf ) pairs per object, obtaining more than 600 ObjNav episodes. An episode is successful if the agent reaches the target within Nsteps steps, is within distance δdist of the object’s ground-truth position, and observes the object from its point of view. All simulated experiments are conducted in AI2-THOR [41] with ProcTHOR [8] environments, where visibility is 6
Table 1: Distributional evaluation averaged across the three habits. Density, coverage, TV and JS are distribution-shape metrics and are reported only for methods that emit a non-degenerate sample distribution. Method FreqPrior LLMPrior EmpiricalMean MeanFlowMaps FlowMaps (ours)
minFDE(m)↓
Rec@1↑
Density↑
Coverage↑
TV↓
JS↓
0.370 0.907 2.040 2.712 0.342
0.958 0.780 0.414 0.260 0.942
0.647 0.773 — — 0.886
0.991 0.967 — — 0.999
0.848 0.796 — — 0.829
0.514 0.471 — — 0.482
assessed using ground-truth information. In the real-world lab demo, a VLM is used only to trigger the candidate “found” signal, while success is still evaluated using the same three criteria, so false positives do not count as successful episodes. We compare FlowMaps against representative dynamic ObjNav baselines: TAP-LGX [21], a zeroshot VLM-based extension of LGX [18]; OSG [3] and CEG [22], which implement the CP-SAT planning approximation of Rudra et al. [3]; HOMER [17] and SGM [23], which model dynamic ObjNav as temporal link prediction with GNNs, with SGM additionally exploiting the scene graph hierarchy; and a Naive LLM baseline directly prompted to rank future object locations. Since OSG and CEG require object location likelihoods, we follow Wang et al. [22] and evaluate scene-prior (SP) variants based on object-receptacle co-occurrences, as well as LLM-based variants that parse human habit hints into relevant events for posterior estimation. All methods return a ranked list of K candidate future locations; for FlowMaps, we obtain this list by running inference Npreds times, clustering predictions with DBSCAN [42], and ranking clusters by mass. Additional baseline details are provided in the appendix. We report Success Rate (SR) and Success weighted by Path Length (SPL) [43], including SR@K, SPL@K, mean SR (mSR), and mean SPL (mSPL) averaged over K. Success at smaller K indicates that the model localizes the object’s future position with fewer trials. We also report the mean path length and mean number of steps required to reach the target. Table 2 shows that FlowMaps consistently achieves the best mSR and mSPL across all settings, while remaining optimal or competitive on the other metrics. Notably, FlowMaps’ SR@1 exceeds the SR@5 of some baselines, indicating that its first prediction can be more accurate than several competing top-5 proposal sets. Among the baselines, GNN-based methods perform best on the easier Habit #1 and Habit #2 settings, but are surpassed by zero-shot LLM-based methods on the harder Habit #3. This suggests that harder behavioral patterns benefit more from transferable dynamic exploration than from directly generalizing learned patterns. Overall, FlowMaps performs best on all three habits, showing stronger dynamic-pattern learning and cross-environment generalization. Real-world deployment. We further showcase our proposed approach in a real-world demonstration with a TIAGo robot1 . The experiment is depicted in Fig. 3: it consists in finding a “CellPhone” that belongs to a person operating in this environment and alternates between the different desks, spending an equal amount of time in each of them (Habit #2). The scene is observed when the phone is at the right desk. A couple of hours pass, and the robot is fetched to find again the phone. Predictions of FlowMaps are clustered together and become ranked navigation targets for the agent. Eventually, the phone gets found at second targeted position.
6
Limitations and Future Directions
The current formulation leaves room for further extensions. First, we consider a closed set of 41 object classes and 17 predefined receptacles, which makes FlowMaps not directly applicable to open-vocabulary settings. This choice was motivated by our focus on indoor household environments, where relevant object classes and receptacles are relatively constrained. Extending the method to open-vocabulary labels could broaden its applicability to more diverse indoor scenarios, such as offices, laboratories, or factories. 1
https://pal-robotics.com/robot/tiago/
7
Habit #1
42.13 35.25 18.69 42.79 22.30 42.95 48.36 46.72 57.70
59.34 57.05 56.89 62.46 58.03 62.62 69.18 67.54 71.15
63.77 62.62 64.10 63.44 64.75 63.61 71.80 69.84 72.62
57.44 54.03 51.85 60.13 52.66 60.29 67.26 64.46 69.26
33.54 28.88 15.34 34.65 18.16 34.93 38.74 37.31 45.31
38.40 40.03 31.46 44.03 33.32 44.57 46.52 44.59 49.52
39.03 44.00 33.21 44.27 34.82 44.81 46.84 44.98 49.72
37.80 43.62 29.29 42.90 30.98 43.40 45.64 43.62 48.95
12.01 10.62 12.99 8.38 12.42 8.00 8.82 9.29 9.01
195.1 193.6 224.3 131.9 212.7 126.3 137.4 145.9 141.1
Habit #2
SR@1 (%)↑ SR@5 (%)↑ SR@10 (%)↑ mSR (%)↑ SPL@1 (%)↑ SPL@5 (%)↑ SPL@10 (%)↑ mSPL (%)↑ Path (m)↓ Steps↓
Naive LLM TAP-LGX [21] OSG+SP [3] OSG+LLM [3] CEG+SP [22] CEG+LLM [22] SGM [23] HOMER [17] FlowMaps (ours) Naive LLM TAP-LGX [21] OSG+SP [3] OSG+LLM [3] CEG+SP [22] CEG+LLM [22] SGM [23] HOMER [17] FlowMaps (ours)
39.02 34.26 19.18 36.89 22.46 37.38 47.54 41.48 50.49
58.52 54.92 57.38 47.38 58.36 48.20 66.23 67.38 66.56
60.66 61.64 66.07 50.16 65.74 50.98 68.52 69.18 67.21
55.46 53.57 52.69 46.43 53.08 47.25 63.85 62.90 63.92
31.30 28.12 15.74 29.85 18.31 30.11 38.07 33.59 41.18
37.95 40.73 32.03 34.97 33.39 35.47 44.83 41.65 46.51
38.21 43.18 34.49 35.69 35.27 36.17 45.14 41.85 46.59
36.94 39.47 29.99 34.40 31.17 34.90 43.94 40.37 45.72
8.72 10.98 13.37 8.14 12.29 8.02 7.76 8.79 7.55
135.2 199.6 233.3 129.5 212.6 127.5 117.6 135.4 113.5
Habit #3
Method
Naive LLM TAP-LGX [21] OSG+SP [3] OSG+LLM [3] CEG+SP [22] CEG+LLM [22] SGM [23] HOMER [17] FlowMaps (ours)
41.15 36.72 15.90 38.52 18.52 38.03 27.21 38.03 50.98
56.23 57.05 56.23 54.26 57.21 54.10 62.62 62.95 66.72
61.48 64.10 65.25 55.74 64.43 55.74 65.41 65.74 68.20
55.46 55.61 51.21 52.33 52.21 52.00 56.84 58.93 64.39
33.52 29.82 12.95 31.72 14.85 31.23 22.19 30.29 41.07
38.31 42.30 31.00 38.97 31.48 38.59 33.64 38.55 45.99
39.21 44.86 33.31 39.33 33.30 38.93 34.02 38.94 46.17
37.89 40.97 28.55 38.04 29.37 37.62 31.84 37.25 45.30
9.45 9.37 11.92 7.68 11.54 7.91 12.24 11.51 7.90
145.1 163.0 207.0 115.7 199.2 119.9 198.1 186.9 117.4
Table 2: Results of FlowMaps and the baselines for the dynamic ObjNav episodes. We report metrics grouped by the three distinct habits that induce the objects displacement.
A second limitation concerns the amount of data needed to train the models. In our experiments, ProcTHOR enabled us to generate sufficient training data for the considered setup. However, extending the approach to more general environments would likely require comparable data, possibly without access to the same simulation tools. One direction to mitigate this issue is to reduce reliance on fully offline training. For examFigure 3: FlowMaps deployed in a real-world environment. ple, after an appropriate sampleefficiency analysis, a compact offline backbone could be trained and then fine-tuned online using data collected during deployment. Finally, the current evaluation is limited to scenarios with at most three habits. While sufficient to assess the proposed formulation, further analysis is needed to understand how the method scales with the number and diversity of habits. These limitations will be investigated and addressed in future work.
7
Conclusions
In this paper, we introduced FlowMaps, a latent FM framework for modeling multimodal spatiotemporal distributions of dynamic objects in human-inhabited environments. Rather than predicting discrete receptacle-level states, FlowMaps models object relocalization as a distribution over future bounding boxes in continuous 3D space. Conditioned on the current scene, queried object, and prediction horizon, it learns recurring human-driven motion patterns directly from data, without explicit habit or activity labels. By combining a VAE-based latent object representation with a CDiT flow model, FlowMaps remains grounded in object geometry while generalizing to unseen household 8
layouts. We evaluated both its distributional properties and practical utility in dynamic ObjNav, where predicted future-location distributions generate ranked navigation targets. Across ProcTHOR simulations and real-world robotic deployments, FlowMaps captures continuous, multimodal, and cross-environment spatio-temporal structure, improving object search and providing an effective prior for robotic reasoning beyond static scene assumptions. Acknowledgments This work has been carried out while Francesco Argenziano was enrolled in the Italian National Doctorate on Artificial Intelligence run by Sapienza University of Rome. This research was conducted while Francesco Argenziano was enrolled as a visiting researcher at Mila - Quebec AI Institute. The work conducted at the Université de Montréal was supported by the Natural Sciences and Engineering Research Council of Canada (NSERC) through an NSERC Discovery Grant and by the CIFAR AI Chair program (Liam Paull). Individual support was provided through NSERC Postgraduate Scholarships-Doctoral (PGS D) (Sacha Morin, Charlie Gauthier, Miguel Saavedra-Ruiz). This research was also enabled in part by computational resources provided by Mila (mila.quebec).
9
Appendix Table 3: Closed-set semantic vocabularies used for dataset generation. Pickupable objects Oobj
Receptacles OBG
ID
Class
ID
Class
ID
Class
ID
Class
ID
Class
0 1 2 3 4 5 6 7 8 9 10 11 12 13
Apple Egg Fork Kettle Ladle Lettuce Pencil Potato SaltShaker SoapBottle SprayBottle Tomato WineBottle Book
14 15 16 17 18 19 20 21 22 23 24 25 26 27
Bottle Knife Spatula Bread ButterKnife Candle CellPhone Newspaper Vase AlarmClock KeyChain Laptop Pen RemoteControl
28 29 30 31 32 33 34 35 36 37 38 39 40
Spoon Watch BaseballBat BasketBall DishSponge ToiletPaper TissueBox TeddyBear SoapBar PaperTowelRoll DeskLamp TennisRacket Cloth
0 1 2 3 4 5 6 7 8
CounterTop Plate Pot ShelvingUnit Chair DiningTable Dresser ArmChair Bed
9 10 11 12 13 14 15 16
Sink Sofa TVStand GarbageCan SideTable Desk CoffeeTable Box
A
Dataset and environments
Environment classes. We generated the environments using ProcTHOR and adopted the closed set of semantic labels available in the simulator for dynamic pickupable objects and receptacles. This resulted in a vocabulary Oobj of 41 pickupable object classes and a vocabulary of OBG 17 receptacle classes. The former contains the dynamic objects whose locations may change over time, while the latter contains the static receptacles and furniture elements that define the nvironment layout and provide support surfaces for object placement. The complete list can be observed in Table 3. Scene layouts. The generated dataset contains indoor ProcTHOR scenes with diverse spatial layouts and room configurations. Scenes range from compact single-room environments to larger multiroom households, with different floor plans, furniture arrangements, and numbers of receptacle instances. This variability affects both the geometric structure of the environment and the set of valid support surfaces available for object placement. A sample of 30 validation environments can be observed in Fig. 5 Object-receptacle com