Multimodal Scenario Similarity Search for Autonomous Driving Tamás Matuszka aiMotive
András Tamásy aiMotive
Balázs Szolár aiMotive
arXiv:2607.09428v1 [cs.CV] 10 Jul 2026
Abstract
1. Introduction
exhibiting similar traffic interactions, or conversely share a similar visual context while containing fundamentally different vehicle and agent behaviors. Existing scenario retrieval approaches typically focus on a single notion of similarity. Vision-based methods leverage video representations to capture scene appearance and spatio-temporal context, while trajectory-based approaches compare vehicle motions and traffic participant behavior. Although both perspectives are relevant for autonomousdriving applications, their relative strengths and limitations have not been systematically studied within a unified retrieval framework. In this work, we investigate multimodal scenario retrieval by combining visual and trajectory-based representations. We evaluate two vision-based retrieval methods and introduce two trajectory-based approaches. The first, Exo-Trajectory, explicitly compares the motion patterns of surrounding traffic participants using trajectory matching. The second, ScenarioFormer, is a transformer-based representation learned from object trajectories using contrastive learning. Together, these methods enable a direct comparison between appearance-driven and motion-driven notions of scenario similarity. Experimental results on a manually annotated drivingscene similarity benchmark show that visual and trajectory representations capture complementary information. Vision models perform best when appearance cues are informative, whereas trajectory-based methods excel on motioncentric scenarios such as cut-ins, turning maneuvers, and traffic queueing. Furthermore, combining visual and trajectory information consistently improves retrieval quality and achieves the best overall performance. The main contributions of this work are:
Modern autonomous-driving systems generate and process enormous amounts of sensor data, resulting in datasets containing millions of recorded driving scenarios. Efficient retrieval of scenarios similar to a given query is essential for dataset curation, corner-case discovery, validation, and data-driven development workflows. However, defining similarity between driving scenarios remains a challenging problem. Two scenarios may appear visually different while
• We present a multimodal framework for autonomousdriving scenario retrieval that enables the joint study of visual and trajectory-based similarity representations. • We introduce two trajectory-based retrieval approaches: Exo-Trajectory, an explicit matching method based on surrounding-agent motion, and ScenarioFormer, a transformer-based trajectory representation learned using contrastive learning.
Large-scale autonomous-driving datasets contain vast numbers of recorded scenarios, creating a need for efficient retrieval methods that can identify situations similar to a given query. Existing approaches typically rely on either visual representations or motion-based descriptions, making it difficult to understand their relative strengths and limitations for scenario retrieval. In this work, we present a multimodal framework for autonomous-driving scenario retrieval that combines visual and trajectory-based representations within a unified retrieval pipeline. We investigate two trajectory-based approaches: Exo-Trajectory, an explicit matching method based on surrounding-agent motion, and ScenarioFormer, a transformer-based representation learned from object trajectories using contrastive learning. We compare these approaches against strong visionbased baselines and analyze their behavior across a diverse set of driving scenarios. Experimental results show that trajectory representations provide strong retrieval performance for motion-centric events such as cut-ins, turning maneuvers, and traffic queueing, while visual embeddings excel when appearance cues are informative. Most importantly, combining visual and trajectory information consistently improves retrieval quality, yielding the best overall performance. These findings demonstrate that appearance and motion capture are complementary notions of scenario similarity and motivate multimodal retrieval systems for autonomous-driving data mining, dataset curation, and scenario-based validation.
1
• We demonstrate through extensive experiments that visual and trajectory representations capture complementary notions of scenario similarity and that their fusion yields the strongest retrieval performance.
2. Related Work
mapped close together in the representation space. Motivated by these advances, our proposed ScenarioFormer learns trajectory embeddings directly from agent trajectories using a transformer encoder trained with a contrastive objective, enabling efficient retrieval based on scene dynamics.
2.1. Visual Scenario Retrieval
3. Methodology
Recent advances in video representation learning have enabled effective retrieval and similarity search in large-scale video collections. Transformer-based architectures such as ViViT [1] learn spatio-temporal representations directly from video sequences and have achieved strong performance across a variety of video understanding tasks. More recently, large vision-language models such as Qwen3VL [6] have demonstrated impressive transfer capabilities, producing semantically meaningful embeddings that can be used for retrieval without task-specific supervision. Such representations capture scene appearance, object context, and high-level semantics, making them attractive for autonomous-driving scenario search. However, visual representations may struggle to distinguish scenarios that exhibit similar appearance while differing significantly in agent behavior and motion patterns.
We investigate autonomous-driving scenario retrieval using two complementary modalities: visual representations extracted from video data and trajectory-based representations derived from the motion of traffic participants. Given a query scenario, each retrieval method produces a similarity score between the query and candidate scenarios. We evaluate both standalone retrieval performance and multimodal fusion of visual and trajectory-based similarity measures. Each scenario is first preprocessed to extract relevant features (video frames, ego-motion matrices, and object trajectories), which are then encoded into a compact vector representation or distance matrix. Scenario similarity is expressed through cosine similarity or distance-based measures, depending on the modality. This section introduces the individual components of the framework.
3.1. Visual Retrieval 2.2. Trajectory-Based Scenario Retrieval
Visual retrieval methods represent a driving scenario using image or video embeddings extracted from RGB observations. These embeddings capture appearance, scene context, object semantics, and temporal visual cues. Similarity between scenarios is computed using cosine similarity in the embedding space. We evaluate two visual retrieval approaches.
Trajectory information provides an alternative notion of similarity by explicitly modeling the motion of traffic participants. Traditional approaches compare trajectories using geometric distance measures such as Dynamic Time Warping (DTW) [8] or Fréchet distance [4], which have been widely used for trajectory matching and motion analysis. In autonomous driving, trajectory-based retrieval has been explored for scenario mining, corner-case discovery, and dataset search, where the objective is to identify situations exhibiting similar interactions between vehicles and other road users. Compared with visual representations, trajectory-based methods focus directly on dynamic behavior and are therefore particularly suitable for motion-centric scenarios.
3.1.1. ViViT We employ ViViT [1], a transformer-based video model designed for capturing both spatial and temporal dependencies. The model tokenizes an input video into frame patches, processes them through multiple transformer layers, and produces a d-dimensional embedding vector. We use the hidden state of the first token, analogous to the [CLS] token in BERT [3], as the global video representation. To improve computational efficiency, the videos are subsampled to 32 frames and cropped to exclude noninformative regions (e.g., sky or camera mount). All embeddings are precomputed offline, enabling fast similarity queries using cosine similarity: vi · v j cos(vi , vj ) = ∥vi ∥ ∥vj ∥
2.3. Trajectory Representation Learning Recent work has increasingly adopted learned trajectory representations to capture higher-level behavioral semantics. Transformer architectures have proven effective for modeling interactions among multiple traffic participants and forecasting future motion. Scene Transformer [9] introduced a unified attention-based framework for representing complex traffic scenes, demonstrating the ability of transformer models to encode rich relational information between agents. Contrastive learning has further emerged as a powerful approach for learning task-agnostic embeddings from structured data by encouraging similar samples to be
where vi and vj are the embeddings of two scenarios. 3.1.2. Qwen3-VL We utilize Qwen3-VL-2B [6] as a vision-language retrieval baseline due to its strong multimodal representation capa2
bilities. For each driving scenario, 32 frames are sampled uniformly from the video clip and processed by the model to obtain a fixed-length embedding. Timestamp information is included in the visual input to preserve temporal context, enabling the model to distinguish between similar visual scenes occurring at different stages of a maneuver. We extract embeddings using the Matryoshka Representation Learning (MRL) representation with 768 dimensions and compute scenario similarity using cosine similarity between query and candidate embeddings. Unlike ViViT, which is trained specifically for video understanding, Qwen3-VL leverages large-scale multimodal pretraining and produces semantically rich representations that capture scene appearance, object context, and high-level traffic semantics.
The exo-trajectory-based similarity search process can be written as follows: • Distance matrix calculation for two scenarios. An N xM matrix is calculated using the Fréchet Distance between the trajectories of the two scenarios, where N and M are the number of trajectories of the first and second scenarios, respectively. The comparison disregards trajectories belonging to different groups (e.g., a vehicle and a pedestrian). • Matching similar trajectories from two scenarios. The Hungarian Algorithm solves the assignment problem, resulting in an optimal matching between exo-trajectories. Unmatched exo-trajectories are penalized. • Aggregation. The similarity between the two scenarios is calculated by aggregating the Fréchet-distance of matched trajectories while considering the penalties. A key factor here is the matched ratio, defined as
3.2. Trajectory-Based Retrieval While visual representations capture scene appearance and semantic context, many driving scenarios are primarily defined by the motion and interactions of traffic participants. We therefore investigate trajectory-based retrieval methods that compare scenarios using the temporal evolution of agent motion. We consider both explicit trajectory matching and learned trajectory representations.
unmatched total This rewards scenarios with more matched objects. This ratio is incorporated into a combined cost term along with the mean matched distance and the total unmatched penalty. Finally, the combined cost is converted into a [0,1] similarity measure through a scaling function: matched = 1 −
3.2.1. Exo-Trajectory-based Similarity Search 1 1 + combined k
Exo-Trajectory represents a scenario using the trajectories of surrounding traffic participants and measures similarity through explicit trajectory matching. Unlike visual embeddings, which rely on appearance and scene context, ExoTrajectory directly compares motion patterns and interactions between agents. To explicitly account for the behavior of other traffic participants, we developed the Exo-Trajectory similarity search method that compares the motion patterns of surrounding vehicles, pedestrians, and other dynamic objects. When 3D bounding boxes are available, we extract trajectories of all exo-objects and group them by type: vehicle, stopped vehicle, pedestrian, and rideable. The trajectories are matched when the distance matrix is calculated. We used the Hungarian Algorithm [5] to find the best matching/assignment between trajectories. The Hungarian Algorithm is a combinatorial optimization algorithm that solves the assignment problem. To further refine the matching, we introduce a penalty mapping for unmatched trajectories on a per-class basis. In this mapping, unmatched vehicles incur a higher penalty, whereas, for example stopped vehicles incur a relatively lower penalty. This avoids overly penalizing scenarios that differ mainly by stationary objects, while emphasizing the importance of matching active ones. This solution ensures that scenarios with a similar number of objects and comparable movement patterns receive higher similarity values.
The penalization process is especially beneficial when the two scenarios have similar numbers of other traffic participants, but these objects cannot be matched due to category mismatches. This way, the exo-trajectory-based similarity search ensures the sensitivity to the movement patterns of all objects, considering their object categories. 3.2.2. ScenarioFormer ScenarioFormer is our proposed trajectory-based retrieval model. Unlike Exo-Trajectory, which relies on explicit trajectory matching, ScenarioFormer learns a compact embedding representation directly from agent trajectories using contrastive learning. The objective is to map scenarios exhibiting similar motion patterns to nearby locations in the embedding space while separating dissimilar scenarios. While the Exo-Trajectory similarity search method provides an explainable solution for comparing autonomous driving scenarios, its high computation intensity might hinder large-scale deployment. Therefore, we developed a data-driven solution that enables a computationally less intensive precomputation method and allows fast querying. We designed, developed, and trained a model, called ScenarioFormer, which is based on the vision transformer model [9]. The exo-trajectories of a scenario can be represented by tensors with a shape (N +1, T, C) where N is the 3
3.3. Multimodal Fusion
number of non-ego objects, T is the number of timesteps, and C is the number of attributes of the object. We set N = 25 (with zero padding when the object number is less than 25), T = 15, and C = 8. The following bounding box properties were included in the input tensor after a BEV projection:
Visual and trajectory representations capture complementary aspects of scenario similarity. To combine their strengths, we perform score-level fusion between the visual and trajectory retrieval modules. Given a visual similarity score svision and a trajectory similarity score straj , the final similarity score is computed as
• longitudinal position (x coordinate), • lateral position (y coordinate), • object length, • object width, • heading (yaw angle), • longitudinal velocity, • lateral velocity. • class ID
s = αstraj + (1 − α)svision ,
(1)
where α controls the contribution of the trajectory representation. Fusion allows the retrieval system to leverage both appearance-based and motion-based cues when ranking candidate scenarios.
The input tensor is projected into a high-dimensional embedding space by a Linear layer. Then, positional encoding is added to retain the spatio-temporal information within the Transformer layers. The output of the model is a d-dimensional embedding.
4. Experiments 4.1. Benchmark Dataset and Evaluation Protocol We evaluate our scenario retrieval solution on a subset of the extended version of the aiMotive Multimodal Dataset [7]. The benchmark contains 13 scenario categories covering both dynamic traffic interactions and appearance-driven scene characteristics. Following our focus on motion-aware retrieval, we primarily report results on a challenging dynamic subset consisting of seven topics: exo-vehicle cutins, highway cut-ins, pedestrian crossings, pedestrian crossings during ego turns, cyclist interactions, ego turning maneuvers, and traffic queueing. Each topic contains approximately 100 manually annotated clips with graded similarity labels (overall annotated dataset size is 682). Clips are assigned relevance scores from 0 to 3, where 3 denotes highly similar scenarios, 2 denotes similar scenarios with minor differences, 1 denotes weak similarity, and 0 denotes unrelated scenarios. The similarity labels were manually assigned by domain experts familiar with autonomous-driving scenarios. Every grade-3 clip (overall 97) serves as a query and is evaluated against all the clips. Our primary metric is Normalized Discounted Cumulative Gain (NDCG@10), which accounts for graded relevance and rewards rankings that place highly similar scenarios above partially matching ones. We additionally report Recall (Recall@k≥g2 : fraction of clearly-useful clips retrieved in top-k, Recall@k(g3) : fraction of strict positives found in top-15) metric. All methods are evaluated using a unified candidate pool and identical query sets, enabling direct comparison between visual and trajectory-based approaches.
ScenarioFormer is a Siamese-Transformer [2] where the model is fed by two input tensors and is trained with contrastive learning. One input tensor represents the original scenario input, while the second input tensor is the augmented version of the first input. We utilized the following augmentations: • random longitudinal offset, • random lateral offset, • random length extension, • random width extension, • random heading perturbation, • random velocity noise, • random dropout of objects. Contrastive learning aims to pull similar scenarios close to each other in the embedding space while pushing different scenarios far away. Similar scenarios can be generated by augmenting the original scenario with the abovementioned augmentations. These (original, augmented) scenarios are the positive pairs that will be close to each other in the embedding space. Negative pairs can easily be formed by taking all other scenarios contained by the same batch. The metric used to determine the similarity between two scenarios is cosine similarity. We used the InfoNCE loss represented by a CrossEntropy loss. Since contrastive learning is sensitive to batch size, we utilized gradient accumulation to increase the effective batch size without running out of GPU memory. The scalability of this approach is ensured by the precomputation of the embeddings. Then, the same similarity search described in the video-based method can be used. To find similar scenarios to a new one, it has to be encoded by the model, and then cosine similarity can be used to query similar scenarios from the embedding pool.
4.2. Standalone Retrieval Performance Table 1 compares the four retrieval modules evaluated in this work. Two methods operate on visual representations 4
(a) Pedestrian crossing scenario
(b) Cut-in scenario
Figure 1. Examples from the evaluation benchmark, illustrating representative pedestrian-crossing and cut-in scenarios. Best viewed by zooming in.
5
Table 1. Standalone retrieval performance on the dynamic subset of the benchmark. Higher is better for all metrics.
Method ViViT ScenarioFormer Exo-Trajectory Qwen3-VL-2B
NDCG@10
Rec@15≥2
Rec@15(3)
0.514 0.565 0.565 0.621
0.421 0.441 0.475 0.513
0.363 0.388 0.434 0.463
Table 3. Per-topic NDCG@10 comparison between ExoTrajectory and ScenarioFormer. Positive ∆ indicates an advantage for Exo-Trajectory, while negative values favor ScenarioFormer. Although both methods achieve nearly identical macro performance, they exhibit complementary strengths. Exo-Trajectory performs better on cut-in and cyclist scenarios, while ScenarioFormer excels on pedestrian interactions and traffic queueing.
Table 2. Best score-level fusion results on the dynamic benchmark. ∆ denotes the improvement in NDCG@10 over the corresponding vision-only baseline.
Fusion Method
α
NDCG@10
∆
ViViT + ScenarioFormer ViViT + Exo-Trajectory Qwen3 + ScenarioFormer Qwen3 + Exo-Trajectory
0.4 0.5 0.3 0.3
0.599 0.616 0.662 0.671
+0.085 +0.102 +0.041 +0.050
Exo
Scene
∆
01 Cut-in (env. invariant) 02 Highway cut-in 03 Pedestrian crossing 04 Pedestrian + ego turn 05 Cyclist interaction 06 Ego turning at intersection 07b Traffic queue
0.491 0.597 0.613 0.417 0.525 0.511 0.802
0.377 0.473 0.703 0.564 0.465 0.489 0.884
+0.114 +0.124 -0.090 -0.147 +0.060 +0.022 -0.082
Macro Average
0.565
0.565
+0.000
Results are summarized in Table 2. Fusion consistently improves retrieval performance over the corresponding visual baseline, demonstrating that trajectory information contributes complementary cues not captured by appearance embeddings. The strongest overall result is obtained by combining Qwen3-VL-2B with Exo-Trajectory, yielding an NDCG@10 of 0.671, corresponding to an absolute improvement of 5.0% over Qwen3-VL-2B alone. Similarly, combining Qwen3-VL-2B with ScenarioFormer increases NDCG@10 from 0.621 to 0.662. The effect is even larger for the weaker ViViT baseline, where fusion with ExoTrajectory improves NDCG@10 from 0.514 to 0.616. The per-topic analysis in Table 4 reveals that the benefit of fusion depends strongly on the scenario category. Trajectory information is particularly valuable for cut-in and cyclist-interaction scenarios, where the Exo-Trajectory fusion achieves the best performance. In contrast, pedestrianrelated scenarios and traffic queueing are best handled by the ScenarioFormer-based fusion, suggesting that the learned trajectory embedding captures higher-level semantic interactions between traffic participants. Overall, no single fusion strategy dominates across all topics, further supporting the hypothesis that visual and trajectory representations capture complementary aspects of scenario similarity. Overall, the fusion experiments demonstrate that motion-aware retrieval and visual retrieval are complementary and that combining both modalities yields the most robust similarity search system.
(ViViT and Qwen3VL-2B), while two methods rely exclusively on trajectory information (Exo-Trajectory and ScenarioFormer). Qwen3-VL-2B achieves the strongest overall performance, reaching an NDCG@10 of 0.621. Among the trajectory-based approaches, Exo-Trajectory and ScenarioFormer obtain nearly identical performance (both 0.565 NDCG@10), substantially outperforming the weaker visual baseline (ViViT, 0.514). The per-topic analysis reveals a complementary behavior between visual and trajectory representations. Visual embeddings perform best on scenarios where appearance cues and semantic context provide strong signals, including pedestrian interactions and cyclist-related events. In contrast, trajectory-based methods excel in scenarios whose defining characteristics arise primarily from motion patterns. Traffic queueing and turning maneuvers are particularly well captured by trajectory representations, indicating that explicit motion modeling provides information that is difficult to infer from appearance alone. These results suggest that visual and trajectory modalities capture different aspects of scenario similarity and motivate their combination through multimodal fusion.
4.3. Multimodal Fusion To combine visual and trajectory information, we perform score-level fusion, s = αstraj + (1 − α)svision ,
Topic
4.4. Analysis of Trajectory Representations
(2)
We next compare the two trajectory representations directly. Although their overall NDCG@10 scores are nearly identical, they exhibit different strengths.
where α controls the contribution of the trajectory model. 6
Table 4. Best per-topic NDCG@10 obtained by each fusion method. For each topic, the fusion weight α is independently selected to maximize NDCG@10. Bold values indicate the best fusion result for that topic.
Topic 01 Cut-in (env. invariant) 02 Highway cut-in 03 Pedestrian crossing 04 Pedestrian + ego turn 05 Cyclist interaction 06 Ego turning at intersection 07b Traffic queue
ViViT+ScenarioFormer
ViViT+Exo
Qwen3-VL+ScenarioFormer
Qwen3-VL+Exo
0.397 0.612 0.793 0.617 0.473 0.575 0.890
0.491 0.682 0.777 0.604 0.537 0.523 0.843
0.405 0.658 0.869 0.713 0.649 0.549 0.920
0.491 0.680 0.854 0.691 0.663 0.531 0.913
Table 5. Ablation study of the ScenarioFormer. Results are reported as macro NDCG@10 on the dynamic benchmark.
Configuration Simple (global) augm. Hard (object-level) augm. Semantic (class-aware) augm. Semantic augm. + 2× data + Ego-motion augm.
NDCG@10
Gain
0.444 0.510 0.516 0.531 0.565
– +0.066 +0.006 +0.015 +0.034
data (from 150k sequences to over 300k) raises performance to 0.531. The final improvement comes from incorporating egovehicle motion into the scenario representation. Incorporating an ego-motion together with maneuver-preserving perturbations increases NDCG@10 to 0.565, producing the best-performing ScenarioFormer model. The ablation study highlights augmentation design as the dominant factor affecting retrieval quality. Weak augmentations allow the contrastive objective to exploit trivial correspondences between positive pairs, whereas stronger semantic and motion-preserving augmentations encourage learning representations that capture meaningful scenelevel structure.
Exo-Trajectory performs best on cut-in scenarios and cyclist interactions, achieving substantial gains on both highway and urban cut-in topics. These scenarios are characterized by distinctive geometric motion patterns, which are effectively captured by explicit trajectory matching using Fréchet-distance-based costs. In contrast, ScenarioFormer performs best on pedestriancentric scenarios and traffic queueing. The learned representation benefits from object-type embeddings and semantic augmentations, enabling it to better distinguish interactions involving different actor categories. The complementary behavior of the two trajectory models explains why both serve as effective fusion partners despite having similar aggregate performance. Exo-Trajectory provides stronger geometric matching, while ScenarioFormer captures higher-level semantic relationships between dynamic actors.
4.6. Discussion The experimental results reveal three key findings. First, modern vision-language embeddings provide a strong baseline for scenario retrieval, particularly when semantic appearance cues are informative. Second, trajectory representations remain highly valuable for events defined primarily by motion dynamics, such as cut-ins, turning maneuvers, and traffic queueing. Third, combining visual and trajectory information consistently yields the best overall performance, demonstrating that the two modalities encode complementary notions of similarity. These findings support the use of multimodal retrieval systems for autonomous-driving data mining, where relevant scenarios often depend simultaneously on visual context and temporal motion patterns. Figures 1 and 2 provide qualitative examples of the benchmark and retrieval results. Figure 1 depicts query scenarios together with the top-1 retrieval returned by each model, highlighting the different notions of similarity captured by visual and trajectory-based representations. Figure 2 presents the top-5 retrieval results obtained by ScenarioFormer for a pedestrian-crossing query, demonstrating that the learned trajectory representation successfully retrieves scenarios exhibiting similar pedestrian-vehicle interactions despite variations in visual appearance and scene context.
4.5. Ablation Study Table 5 presents an ablation study of the ScenarioFormer training procedure. We evaluate progressively stronger augmentation strategies while keeping the model architecture fixed. Starting from a simple augmentation policy (global jitter), introducing stronger trajectory perturbations (perobject jitter, dropout, position noise) increases NDCG@10 from 0.444 to 0.510. Adding class-aware semantic augmentations (class-aware jitter, dropout) yields a further improvement to 0.516, while increasing the amount of training 7
5. Conclusion We presented a multimodal framework for autonomousdriving scenario similarity search that combines visual and trajectory-based representations. We evaluated two off-theshelf vision models and two proposed trajectory-based approaches on a manually annotated driving-scene similarity benchmark. The results show that trajectory information provides a strong retrieval signal for motion-centric scenarios such as cut-ins, turning maneuvers, and traffic queueing, while visual embeddings perform best when appearance cues are informative. Although the strongest vision model achieved the best standalone performance, both trajectorybased methods achieved competitive results and exhibited complementary strengths. Most importantly, combining visual and trajectory information consistently improved retrieval quality, yielding the best overall performance. Future work will focus on larger-scale benchmarks, endto-end multimodal representations that jointly encode visual context and agent motion, and retrieval-driven applications for dataset curation and autonomous-driving system validation.
References [1] Anurag Arnab, Mostafa Dehghani, Georg Heigold, Chen Sun, Mario Lučić, and Cordelia Schmid. Vivit: A video vision transformer. In Proceedings of the IEEE/CVF international conference on computer vision, pages 6836–6846, 2021. 2 [2] Wele Gedara Chaminda Bandara and Vishal M Patel. A transformer-based siamese network for change detection. In IGARSS 2022-2022 IEEE International Geoscience and Remote Sensing Symposium, pages 207–210. IEEE, 2022. 4 [3] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers), pages 4171–4186, 2019. 2 [4] Thomas Eiter, Heikki Mannila, et al. Computing discrete fréchet distance. 1994. 2 [5] Harold W Kuhn. The hungarian method for the assignment problem. Naval research logistics quarterly, 2(1-2):83–97, 1955. 3 [6] Mingxin Li, Yanzhao Zhang, Dingkun Long, Keqin Chen, Sibo Song, Shuai Bai, Zhibo Yang, Pengjun Xie, An Yang, Dayiheng Liu, et al. Qwen3-vl-embedding and qwen3-vlreranker: A unified framework for state-of-the-art multimodal retrieval and ranking. arXiv preprint arXiv:2601.04720, 2026. 2 [7] Tamas Matuszka, Ivan Barton, Ádám Butykai, Péter Hajas, Dávid Kiss, Domonkos Kovács, Sándor Kunsági-Máté, Péter Lengyel, Gábor Németh, Levente Pető, et al. aimotive dataset: A multimodal dataset for robust autonomous driving with long-range perception. In International Conference on Learning Representations 2023 Workshop on Scene Representations for Autonomous Driving. 4
Figure 2. Visualization of the top-5 retrieval result of ScenarioFormer in a pedestrian crossing use case.
8
[8] M Müller. Dynamic time warping. information retrieval for music and motion, springer. 2007. 2 [9] Jiquan Ngiam, Vijay Vasudevan, Benjamin Caine, Zhengdong Zhang, Hao-Tien Lewis Chiang, Jeffrey Ling, Rebecca Roelofs, Alex Bewley, Chenxi Liu, Ashish Venugopal, et al. Scene transformer: A unified architecture for predicting future trajectories of multiple agents. In International Conference on Learning Representations. 2
9