TrajPrism: A Multi-Task Benchmark for Language-Grounded Urban Trajectory Understanding
Lihuan Li1†∗ Wilson Wongso1∗ Baiyu Chen1 Hao Xue1,2 Ruiyi Yang1 Yifan Duan1 Xiachong Lin1 Yang Song1 Flora Salim1
arXiv:2605.10782v1 [cs.AI] 11 May 2026
1
UNSW Sydney
2
HKUST (GZ)
Abstract Urban mobility is naturally expressed both as trajectories in space and as naturallanguage descriptions of travel intent, constraints, and preferences. However, prior work rarely evaluates these two modalities together on the same real-world trajectories: trajectory modeling often stays geometry–centric, while language– centric mobility benchmarks frequently target route planning and tool use rather than fine-grained, verifiable alignment between text and the underlying route. We introduce TrajPrism, a multi-task benchmark for language-trajectory alignment that unifies (i) instruction-conditioned trajectory generation, (ii) language-driven semantic trajectory retrieval, and (iii) trajectory captioning, together with an evaluation protocol that measures trajectory fidelity, retrieval quality, and language groundedness. We construct TrajPrism by pairing real urban trajectories with judge-filtered language annotations generated under a four-dimensional travelintent taxonomy. The benchmark contains 300K selected trajectories across Porto, San Francisco, and Beijing, yielding 2.1M task instances from three instruction variants, three retrieval queries, and one caption per trajectory. We further develop proof-of-concept models for each task: TrajAnchor for instruction-conditioned trajectory generation, TrajFuse for semantic trajectory retrieval, and TrajRap for trajectory captioning. These models instantiate the proposed tasks and show that geometry-only trajectory baselines leave a large gap on our protocol, especially where language is part of the input–output interface. We release TrajPrism with code and a reproducible annotation pipeline that is designed to be portable across cities, given compatible trajectory inputs and map resources. Code available at https://anonymous.4open.science/r/TrajPrism-05D6/.
1
Introduction
Urban trajectories are one of the richest sources of human behavioral data available at scale. Understanding them unlocks a broad range of applications: route planning [3], mobility pattern analysis [26, 23], urban trajectory retrieval and similarity search [12, 4, 36], and location-based services [25]. Large language models have accelerated interest in language interfaces for mobility [14, 8], pushing trajectory understanding beyond purely geometric prediction toward settings where models must interpret and produce natural language tied to observed trips. To study this paradigm, recent benchmarks fragment this space along different axes. Semantic Routing [32] studies natural-language criteria paired with route-oriented generation. MobilityBench [22] evaluates LLM agents on realistic route-planning and information-access episodes expressed as user queries. Both are valuable, but they do not jointly operationalize language–trajectory alignment on the same collection of real observed trajectories: Semantic Routing centers on a generation-focused setting with synthetic ∗ Equal contribution.
Preprint.
† Corresponding author: [email protected].
Real-World Trajectories
Downstream Tasks
TrajPrism
Semantic Navigation
3 Tasks · 3 Cities 300K Trajectories · 2.1M Instances
Literal
Concise
Chatty
“Continente in Ramalde. Straight down the wide streets past the park stretch, don't leave Boavista.”
Semantic Retrieval Query 1
GPS Trajectory ↘
Query 2
Query 3
“Historical routes from Cedofeita to Ramalde that follow Rua do Campo Alegre and then continue along Avenida da Boavista as the main sequential road segments.”
Trajectory Captioning Full Description
Intent Taxonomy Destination
Waypoint
Preference
Temporal
“Starting from Cedofeita near the waterfront on Rua Professor Vicente José de Carvalho, the route heads southwest and …. concludes after approximately 8 minutes at ….”
Figure 1: TrajPrism refracts real urban trajectories through a four-dimensional intent taxonomy into diverse, grounded language annotations. The benchmark supports three complementary tasks: navigation instruction following, trajectory retrieval, and trajectory captioning, across 300K trajectories and three cities. Table 1: Comparison with closely related benchmarks. TrajPrism is the only one built from real observed trajectories with bidirectional language–trajectory tasks. Property
Sem. Routing [32]
MobilityBench [22] TrajPrism (ours)
Data source City diversity Scale Lang. grounding Tasks Public pipeline
Synthetic queries U.S. cities 1M queries Text → route Route generation ×
Real user queries Global long-tail 100K queries Text → API calls Agent planning ×
Real GPS trajectories Europe / U.S. / Asia 300K trajs. / 2.1M instances Text ↔ GPS trajectory Gen. + retrieval + captioning ✓
query distributions, while MobilityBench focuses on tool-mediated agent planning rather than language-grounded trajectory understanding. What remains missing is a benchmark that (i) binds text to concrete trajectories, (ii) covers both language-to-trajectory and trajectory-to-language interfaces, and (iii) supports unified evaluation of trajectory fidelity, retrieval, and language groundedness, going beyond the single-interface settings offered by existing work. We introduce TrajPrism to fill this gap. TrajPrism targets language–trajectory alignment through three complementary tasks: instruction-conditioned trajectory generation that asks a model to predict a trajectory from a natural-language navigation instruction, language-driven semantic trajectory retrieval that challenges a model to identify the matching trajectory from a candidate pool given a textual query, and trajectory captioning that requires a model to produce a factual description of an observed trajectory. Together, these tasks probe whether models can align unconstrained navigation language with urban routes. This is analogous in spirit to vision–language benchmarks [16, 20] that unify generation, retrieval, and description, but instantiated on urban mobility. Figure 1 illustrates the core idea: real urban trajectories are refracted through a four-dimensional intent taxonomy into diverse, grounded language annotations that feed the three tasks. Table 1 contrasts TrajPrism with the most closely related benchmarks. Whereas Semantic Routing and MobilityBench each cover a single evaluation interface (route generation or agent planning, respectively), TrajPrism is the only benchmark built from real GPS trajectories that supports bidirectional language–trajectory tasks and provides a public generation pipeline. Our main contributions are as follows: • We introduce TrajPrism, the first large-scale, multi-task, language-grounded benchmark for urban trajectory understanding. TrajPrism pairs 300K real-world trajectories across three cities with 2.1M task instances (each trajectory yields seven instances: three instruction variants, three retrieval queries, and one caption), spanning instruction-conditioned trajectory generation, language-driven semantic trajectory retrieval, and trajectory captioning. • We design a multi-dimensional evaluation protocol that jointly measures trajectory fidelity, retrieval quality, and language groundedness, complemented by a human evaluation rubric. To our knowledge, this is the first protocol to assess language–trajectory alignment across generation, retrieval, and description within a single benchmark. 2
Table 2: TrajPrism benchmark overview. Each task targets a different direction of language–trajectory alignment and is evaluated with complementary metrics. Representative metrics are listed. Task
Input
Output
Key Metrics
Trajectory Generation Trajectory Retrieval Trajectory Captioning
Language + start loc. Language query Trajectory
Trajectory Ranked trajs. Language
Dest-Hit, Jac, DTW, Haus, EDR R@K, J@K, SR@K, MRR BS-F1, ROUGE-L, METEOR
• We develop Reverse Intent Reconstruction (RIR), a reproducible annotation pipeline that synthesizes quality-controlled language annotations from real trajectories and is designed to be portable to new cities given compatible trajectory inputs and map resources. • We develop three proof-of-concept models (TrajAnchor, TrajFuse, TrajRap), one per task, to calibrate the benchmark. Experiments show that geometry-only baselines leave a large performance gap on language-grounded tasks, confirming that TrajPrism poses a meaningful and unsaturated evaluation challenge.
2
Related Work
Trajectory Modeling and Language Interfaces. Spatiotemporal trajectory modeling has matured into a rich research area [17], from early sequence-to-sequence similarity models [13, 30] to recent self-supervised representation learning [4, 12, 11, 36, 18]. These methods operate in purely geometric or structural space, yet comprehensive trajectory understanding requires grounding in natural language that captures travel intent, route constraints, and contextual semantics beyond what coordinates alone convey. Large language models are increasingly applied to urban mobility tasks [8], spanning spatio-temporal prediction [14] and mobility generation and intention modeling [23, 24, 7, 29]. Despite this progress, trajectory modeling and language understanding remain largely evaluated in isolation: geometric methods are tested on trajectory benchmarks, while language interfaces are assessed on task-specific success rates, leaving fine-grained language–trajectory alignment on real GPS routes unaddressed. Trajectory and Mobility Benchmarks. Traditional trajectory benchmarks such as Porto2 , San Francisco3 , and Beijing [35, 34] focus on geometric tasks like similarity search, travel-time estimation, and classification. More recently, language-aware mobility benchmarks have emerged. TravelPlanner [28] evaluates LLM agents on multi-day itinerary planning with tool use, Semantic Routing [32] pairs synthetic natural-language queries with route generation, and MobilityBench [22] tests route-planning agents on real user queries. However, none of these benchmarks pairs real GPS trajectories with structured language annotations that jointly capture travel intent and support bidirectional language–trajectory evaluation. TrajPrism addresses this with a unified multi-task benchmark grounded in real urban trajectories (Table 1).
3
TrajPrism
TrajPrism evaluates language–trajectory alignment through three complementary tasks: instructionconditioned trajectory generation, language-driven semantic trajectory retrieval, and trajectory captioning. Each task pairs real urban trajectories with natural-language annotations, and is assessed by a multi-dimensional evaluation protocol covering trajectory fidelity, retrieval quality, and language groundedness. The benchmark comprises 300K trajectories and 2.1M instances across Porto, San Francisco, and Beijing, constructed via a reproducible annotation pipeline (Figure 2). 3.1
Tasks and Evaluation Protocol
TrajPrism defines three benchmark tasks, each probing a distinct capability for language–trajectory understanding. Formal definitions of all evaluation metrics are provided in Appendix A.1. Task 1: Navigation Instruction Following. Given a natural language navigation instruction and a starting road segment and timestamp, a model predicts the full trajectory (sequence of road segment 2 https://www.kaggle.com/competitions/pkdd-15-taxi-trip-time-prediction-ii/data 3 https://ieee-dataport.org/open-access/crawdad-epflmobility
3
Reverse Intent Reconstruction 1: Trajectory Input
2: Route Abstraction
Map-matched GPS traces OSM road network Overture Map Semantics
•
• • •
3: Intent Sampling
Semantic Route Abstraction
• •
4: Multi-task Generation
4 intent dimensions · 10 scenarios 1-5 scenarios sampled per trajectory
TrajPrism
3 Tasks · 3 Cities 300K Trajectories · 2.1M Instances
5: Quality Control
6: Judge & Filter
TrajPrism
Figure 2: Dataset generation pipeline. Steps 1–4 constitute the Reverse Intent Reconstruction (RIR) framework: map-matched trajectories are compressed into grounded semantic phases, travel intent is reconstructed via structured sampling over a four-dimensional taxonomy, and multi-task language annotations are synthesized across three tasks. The generated data then undergoes five-stage quality control (Step 5) and cascaded LLM and human judging (Step 6), yielding the final TrajPrism benchmark. IDs). We evaluate two complementary aspects: destination accuracy, with Destination Hit Rate (Dest-Hit), endpoint geodesic distance (Dist, km), and destination hit within K hops (H@K). For trajectory fidelity, we measure Jaccard similarity over H3 cells (Jac), Dynamic Time Warping (DTW, km) [10], Hausdorff distance (Haus, km) [27], and Edit Distance on Real sequences (EDR [5]). Task 2: Trajectory Retrieval. Given a natural language retrieval query, a model must retrieve the matching trajectory from a candidate pool. We evaluate two aspects: spatial overlap, with Jaccard at K (J@K) and soft Recall (SR@K, the fraction of queries whose best Jaccard exceeds 0.8), and ranking quality, with hard Recall (R@K) and Mean Reciprocal Rank (MRR). Task 3: Trajectory Captioning. Given a trajectory, a model must generate a factual natural language caption. We evaluate language quality with BERTScore F1 (BS-F1) [31], ROUGE-L (R-L) [15], and METEOR [2], and spatial grounding is measured by POI Recall (POI-R), the proportion of ground-truth POI mentions correctly captured, and Named Location Count (N-Loc.) for spatial detail coverage. 3.2 3.2.1
Dataset Construction via Reverse Intent Reconstruction Trajectory and Map Inputs
TrajPrism is constructed from real map-matched urban trajectories in Porto, San Francisco, and Beijing. We represent each trajectory as an ordered sequence of timestamped road segments τ = ((r1 , t1 ), (r2 , t2 ), . . . , (rN , tN )), where ri ∈ R denotes a map-matched road segment and ti is the corresponding timestamp. We retrieve the road network from OpenStreetMap (OSM)4 and semantic information from Overture Maps5 . Each road segment ri is associated with a directional heading d computed from its bearing, an H36 hexagonal cell h ∈ H encoding its spatial location, and a semantic area description s derived from Overture Maps annotations encoding nearby POIs and urban context. Details of retrieved information are provided in Appendix A.3.1. 4 https://www.openstreetmap.org/ 5 https://overturemaps.org/ 6 https://h3geo.org/
4
3.2.2
Phase-based Semantic Route Abstraction
Following hierarchical trajectory abstraction [36, 11], we compress each trajectory into semantically enriched phases that inline road-network structure, POI context, and area descriptions, producing a self-contained representation an LLM can interpret without additional lookups. Specifically, we convert τ into a compact sequence of phases via H3-based run-length encoding. Each road segment is mapped to an H3 hexagonal cell through a precomputed index φ : R → H. Consecutive segments sharing the same dominant H3 cell are merged into a single phase: Pk = {ri , . . . , rj } s.t. φ(rl ) = hk ∀ rl ∈ Pk
(1)
This reduces a typical trajectory from hundreds of road segments to a compact sequence of mobility phases (Figure 2 Step 2). Each phase aggregates heading, duration, road names, and semantic area descriptions (full definition in Appendix A.3.2). 3.2.3
Intent Sampling via Four-Dimensional Taxonomy
Given the compressed phase sequence from Step 2, we construct a structured intent profile for each trajectory through sampling over a four-dimensional intent taxonomy (Figure 2, Step 3) to generate navigation instructions and retrieval queries. The taxonomy organizes travel intent into four dimensions (Destination, Waypoint, Route Preference, and Temporal), comprising 10 finegrained scenarios in total. For each trajectory, we sample c scenarios without replacement, where c ∈ {1, 2, 3, 4, 5} follows a distribution centered at 2–3 to reflect the natural complexity of human navigation requests. Single-scenario intents are retained as a minority case for short or straightforward trajectories, while composite intents combining scenarios across multiple dimensions constitute the majority. This sampling strategy ensures diversity in the resulting annotations while remaining grounded in the spatial and semantic characteristics of each trajectory. More details are shown in Appendix A.3.3. The sampled scenario set is then passed to the generation stage as a structured prompt constraint, ensuring all generated annotations faithfully reflect the inferred intent rather than generic route descriptions. 3.2.4
Multi-task Generation
Given the compressed phase sequence {pk }K k=1 and sampled intent profile from Steps 2–3, we generate language annotations across three tasks via large language models (Figure 2, Step 4). Each task defines a distinct input-output format grounded in the same underlying trajectory. Task 1: Navigation Instruction generates three stylistic variants (Literal, Concise, and Chatty) per trajectory, conditioned on the sampled intent scenarios. To maximize lexical and tonal diversity, each variant is additionally conditioned on a speaker persona, sentence-form hint, and length guidance (Appendix A.3.4). Task 2: Trajectory Retrieval generates three retrieval queries per trajectory, collectively spanning all four intent dimensions to ensure broad semantic coverage. Task 3: Trajectory Captioning generates a single third-person, factual caption per trajectory following a fixed narrative structure: origin, key route behaviors and semantic zones traversed, and destination. The model is instructed to adopt an analyst’s perspective, describing what the trajectory did on the map without inferring driver intent. Each trajectory thus yields seven instances (3 + 3 + 1). To balance quality and scale, we adopt a twostage generation strategy: approximately 2K few-shot seed examples are first produced using Gemini 3.1 Flash-Lite 7 and Claude Sonnet 4.6 8 , then used as in-context demonstrations for large-scale generation with GPT-OSS 120B [1] across all three cities. 3.3
Quality Control and Data Judging
To ensure the resulting annotations meet benchmark-level reliability, we apply a two-phase verification pipeline (Figure 2, Steps 5–6). The first phase applies five deterministic quality 7 https://ai.google.dev/gemini-api/docs/models/gemini-3.1-flash-lite-preview 8 https://www.anthropic.com/claude/sonnet
5
Input
Step 1: RAG Retrieval
Step 2: LLM Parsing + RAG Grounding
Step 3: Conditional Trajectory Generation
Rank 1 trajectory
Extracted Constraints
Generated Trajectory
"Destination is the forest area in Rio Tinto. Start from the park in Campanhã, stay on Alameda de Cartes, and finish at the forest."
Ground Truth
Figure 3: TrajAnchor pipeline (Task 1). Step 1 retrieves similar training trajectories; Step 2 extracts and grounds spatial constraints via LLM; Step 3 generates the route via chain Dijkstra. Bottom: Porto example with ground truth on the left. control stages covering noun/location/phase grounding, GIS terminology correction, LLM-based hallucination verification, lexical diversity enforcement, and punctuation sanitization. Drawing on the LLM-as-a-judge paradigm [33], we adopt a cascaded judging strategy that combines scalable LLM scoring, cross-model validation, and human verification in the second phase, to ensure annotation quality is both measurable and reproducible across all three cities. A Qwen-based judge [21] first scores all generated data and selects the top 100K trajectories per city, a sample of 2K trajectories is then independently re-evaluated by GPT-4.19 and Gemini 2.5 Flash [6] to validate scoring consistency, and 100 randomly sampled Porto trajectories are assessed by human annotators to establish a quality baseline. All LLM and human judges share the same evaluation rubric, enabling direct comparison and inter-annotator agreement analysis. The evaluation rubric for both LLM and human judges is detailed in Appendix A.3.5. Human annotators achieve 91.7% mean ±1 agreement among themselves; human–LLM cross-agreement reaches 92.9%, confirming that LLM judges reliably approximate human evaluation (Tables 13 and 14).
4
Experiments
4.1
Proof-of-Concept Models
We develop one proof-of-concept model per task to instantiate the benchmark and calibrate its difficulty, prioritizing coverage of the three task interfaces over architectural novelty. TrajAnchor (Task 1) uses a retrieved training trajectory as a spatial anchor for route generation. TrajFuse (Task 2) fuses geometric and semantic trajectory representations for cross-modal contrastive retrieval. TrajRap (Retrieval-Augmented Profiling, for Task 3) generates factual trajectory captions from retrieved semantic context. We detail TrajAnchor below as the most involved pipeline; TrajFuse and TrajRap are described in Appendix A.2. TrajAnchor (Task 1) is a three-stage pipeline for instruction-conditioned trajectory generation (Figure 3). Step 1 (Retrieval): given a navigation instruction, we encode it with a text embedding model [19] and retrieve the most similar training trajectory via cosine similarity over a pre-built index, optionally filtered to trajectories sharing the same starting H3 cell, followed by destination-proximity reranking. Step 2 (Constraint Extraction): an LLM parses the instruction to extract structured spatial constraints (destination, waypoints, route preferences). The destination phrase is grounded to specific road segments by matching against H3 cell area descriptions via embedding similarity. Step 3 (Route Generation): the extracted constraints are combined with skeleton waypoints sampled from the retrieved trajectory to seed a soft-weighted chain Dijkstra search over the road network, producing the final predicted trajectory. We experiment with multiple LLM backbones (Qwen-3.5 2B/4B/9B, Claude 9 https://developers.openai.com/api/docs/models/gpt-4.1
6
Table 3: Task 1: Instruction-conditioned Trajectory Generation results. Best per dataset in bold, second best underlined. † : Qwen3.5-4B (default); ⋆ : Claude Sonnet 4.6. Dataset
Method
H@5↑
Dest-Hit↑
Dist(km)↓
Jac↑
DTW↓
Porto
DestSP (BM25) DestSP (Embed) ConstrSP TrajAnchor (Q3.5-2B) TrajAnchor† TrajAnchor⋆ (CS4.6) TrajAnchor (Q3.5-9B)
0.0823 0.0955 0.0912 0.1003 0.0987 0.1104 0.1018
0.1357 0.1533 0.1480 0.1578 0.1540 0.1701 0.1595
2.4099 2.0435 2.1937 1.8201 1.6723 1.6728 1.6333
0.2454 55.2871 2.0800 0.7814 0.2707 47.0833 1.8446 0.7605 0.2188 63.0137 2.0989 0.8280 0.2855 42.0668 1.7106 0.7532 0.2955 38.6676 1.6193 0.7463 0.3013 38.6969 1.6060 0.7386 0.3003 37.5961 1.5912 0.7420
SF
DestSP (BM25) 0.1369 DestSP (Embed) 0.1644 ConstrSP 0.1590 TrajAnchor (Q3.5-2B) 0.1756 TrajAnchor† 0.1743 TrajAnchor⋆ (CS4.6) 0.1837 TrajAnchor (Q3.5-9B) 0.1754
0.1476 0.1771 0.1747 0.1948 0.1956 0.2041 0.1956
2.6262 2.4647 2.3691 2.3053 2.2167 2.2070 2.2633
0.2084 0.2245 0.1942 0.2341 0.2405 0.2457 0.2371
BJ
DestSP (BM25) 0.0916 DestSP (Embed) 0.1501 ConstrSP 0.1530 TrajAnchor (Q3.5-2B) 0.1549 TrajAnchor† 0.1634 TrajAnchor⋆ (CS4.6) 0.1800 TrajAnchor (Q3.5-9B) 0.1666
0.0967 0.1611 0.1570 0.1666 0.1742 0.1945 0.1808
4.3284 3.8366 3.5433 3.7530 3.7121 3.5931 3.6891
0.2075 76.1076 3.6399 0.7938 0.2489 67.7288 3.2812 0.7557 0.2127 72.2267 3.3429 0.8090 0.2559 66.0261 3.2279 0.7495 0.2605 65.1863 3.1908 0.7447 0.2709 63.1457 3.0992 0.7345 0.2635 64.6283 3.1714 0.7415
53.8048 51.7005 58.2234 48.5685 46.8433 46.0980 47.6012
Haus↓
2.1931 2.1082 2.2361 2.0084 1.9574 1.9403 1.9716
EDR↓
0.8317 0.8198 0.8652 0.8135 0.8090 0.8037 0.8112
Sonnet 4.6) for constraint extraction, and all variants share the same retrieval and routing modules. Unless otherwise specified, all baselines and our TrajAnchor use the Qwen-3.5 4B backbone by default. 4.2
Results
Setup. We split each city into train, val, and test splits, each containing 70%, 10%, and 20% of the trajectories, respectively. This yields 210k, 30k, and 60k navigation instructions and retrieval queries for each city, respectively; and 70k, 10k, and 20k trajectory caption for each city, respectively. We evaluate on the test split of each city (Porto, San Francisco, Beijing). For Task 1, we compare TrajAnchor against two geometry-only baselines: DestSP, which only routes to the LLM-extracted destination via shortest path, using either BM25 or embedding-based destination retrieval, and ConstrSP, which adds additional LLM-extracted waypoint constraints. For Task 2, we compare TrajFuse against text-based baselines (TF-IDF + SVD, BoW + SVD), four trajectory encoders (UniTraj [37], TrajCL [4], T-JEPA [12], BLUE [36]) under fine-tuned settings, an H3 semantic encoder that represents trajectories by their cell-level area descriptions (with and without fine-tuning), and a Road GATv2 encoder that learns from trajectory graph structure. For Task 3, we compare TrajRap against three ablations: a structural-only baseline where an LLM receives only trajectory coordinates (Struct.), a semantic-augmented variant (Sem.) where an LLM receives additional semantic context, and a distilled variant (Sem. Distill.) that is LoRA [9] fine-tuned with semantic context but receives only structural input at inference, testing whether the model internalizes spatial semantics through training. Implementation details (hyperparameters, training schedules) are provided in Appendix A.2. All models share a common spatial index of H3 hexagonal cells at resolution 9, each annotated with a natural-language area description covering POIs, land use, and road context, encoded with Nomic Embed [19] (details in Appendix A.3.1). All experiments are conducted on a node with 4× NVIDIA H100 80GB GPUs. Tables 3–5 report results across the three tasks and three cities. We highlight benchmark-level insights below, and per-method details and additional ablations are in Appendix A.2. Task 1: Navigation Instruction Following (Table 3). Even the best TrajAnchor variant achieves only 17–20% Dest-Hit and ∼0.25–0.30 Jaccard across cities, indicating that the majority of generated trajectories still deviate from the ground truth. Language-augmented TrajAnchor consistently outperforms the geometry-only baselines (DestSP, ConstrSP) on trajectory-level metrics (Jac, DTW, 7
Table 4: Task 2: Language-driven Semantic Trajectory Retrieval results. Best per dataset in bold, second best underlined. Dataset Method
Porto
SF
BJ
J@1
J@5
SR@1 SR@5
R@1
R@10
MRR
TF-IDF + SVD BoW + SVD
0.2381 0.4648 0.0821 0.2009 0.0245 0.1218 0.0490 0.2368 0.4632 0.0811 0.2003 0.0252 0.1220 0.0493
UniTraj TrajCL T-JEPA BLUE
0.1655 0.2783 0.2544 0.2060
0.3383 0.5239 0.4806 0.4139
0.0598 0.1526 0.1236 0.0780
0.1898 0.3266 0.2712 0.1867
0.0154 0.0839 0.0619 0.0295
0.0814 0.2798 0.2245 0.1391
0.0319 0.1376 0.1050 0.0573
Sem. (Zero-shot) Sem. Road GATv2 TrajFuse
0.0955 0.2621 0.2343 0.2943
0.2087 0.5273 0.4561 0.5443
0.0420 0.1489 0.0870 0.1631
0.0950 0.3462 0.2059 0.3471
0.0315 0.0940 0.0357 0.0957
0.1105 0.3185 0.1547 0.3088
0.0523 0.1548 0.0660 0.1538
TF-IDF + SVD BoW + SVD
0.1666 0.3436 0.0415 0.1070 0.0276 0.1182 0.0507 0.1634 0.3423 0.0393 0.1051 0.0274 0.1130 0.0493
UniTraj TrajCL T-JEPA BLUE
0.0542 0.2863 0.1628 0.1476
0.1449 0.5090 0.3488 0.3291
0.0117 0.2163 0.0941 0.0777
0.0358 0.3926 0.2113 0.1870
0.0067 0.1983 0.0804 0.0653
0.0362 0.4424 0.2422 0.2192
0.0142 0.2718 0.1265 0.1074
Sem. (Zero-shot) Sem. Road GATv2 TrajFuse
0.1153 0.2421 0.1590 0.3085
0.2238 0.4882 0.3424 0.5454
0.0850 0.1858 0.0592 0.2397
0.1557 0.3841 0.1475 0.4345
0.0819 0.1726 0.0475 0.2233
0.1843 0.4508 0.1738 0.4837
0.1121 0.2543 0.0804 0.3030
TF-IDF + SVD BoW + SVD
0.0970 0.2508 0.0203 0.0700 0.0163 0.0938 0.0350 0.0928 0.2428 0.0180 0.0644 0.0147 0.0888 0.0323
UniTraj TrajCL T-JEPA BLUE
0.0190 0.2158 0.1602 0.0790
0.0500 0.4325 0.3497 0.2187
0.0068 0.1652 0.1137 0.0301
0.0171 0.3363 0.2515 0.0953
0.0041 0.1562 0.1055 0.0257
0.0199 0.4076 0.3185 0.1321
0.0080 0.2278 0.1637 0.0523
Sem. (Zero-shot) Sem. Road GATv2 TrajFuse
0.1443 0.2130 0.1261 0.2653
0.2500 0.4272 0.3039 0.4961
0.1277 0.1802 0.0604 0.2213
0.2128 0.3556 0.1598 0.4108
0.1255 0.1744 0.0549 0.2138
0.2547 0.4344 0.2099 0.4896
0.1616 0.2480 0.0957 0.2929
Haus), confirming that leveraging instruction semantics beyond destination extraction improves route fidelity. ConstrSP reaches approximately correct destinations (competitive Dist) but fails to reproduce the intended route shape (low Jac, high DTW). Beijing proves the most challenging setting, with the highest DTW and Hausdorff values across all methods. Scaling the LLM backbone from Qwen3.5-4B to Claude Sonnet 4.6 yields consistent gains across all metrics, suggesting that stronger language understanding directly benefits constraint extraction and route fidelity. Additional destination-proximity metrics (H@K) and routing diagnostics are reported in Table 6; an oracle analysis revealing the retrieval ceiling is given in Table 7 (Appendix). and qualitative examples illustrating good, moderate, and poor predictions are shown in Figure 6 (Appendix). Task 2: Trajectory Retrieval (Table 4). Fine-tuned trajectory encoders that lack semantic input show highly uneven performance: TrajCL is competitive, but UniTraj nearly fails (R@1 < 0.02 on all cities), demonstrating that geometric trajectory representations alone do not reliably capture languageexpressed intent. Methods incorporating H3 semantic descriptions consistently outperform pure trajectory encoders, and TrajFuse, which fuses geometric and semantic representations, achieves the best spatial overlap (J@1, SR@5) across all three cities. Nevertheless, even TrajFuse’s R@1 remains below 0.10 in Porto and below 0.23 elsewhere, leaving substantial room for improvement in exact trajectory identification from natural language. Additional retrieval metrics (J@10, SR@10, R@50) are reported in Table 8, and a per-intent-dimension breakdown is shown in Figure 8 (Appendix). Task 3: Trajectory Captioning (Table 5). BS-F1 remains consistently high (>0.83) across all methods, indicating that fluency is not the bottleneck; the key challenge lies in grounding captions with accurate spatial references (POI-R). The structural-only baseline produces fluent text but 8
Table 5: Task 3: Trajectory Captioning results. Best per dataset in bold, second best underlined. Dataset Method
BS-F1 ROUGE-L METEOR POI-R N-Loc.
In
Porto
Struct. Sem. Sem. Distill. TrajRap
0.854 0.864 0.871 0.885
0.245 0.259 0.310 0.321
0.206 0.227 0.315 0.314
0.121 0.408 0.033 0.341
4.69 5.46 5.30 4.85
477 68 2686 72 468 119 5025 93
Out Lat.(s) 1.18 2.16 1.81 5.89
SF
Struct. Sem. Sem. Distill. TrajRap
0.867 0.868 0.881 0.885
0.227 0.237 0.316 0.307
0.173 0.200 0.313 0.298
0.015 0.095 0.017 0.087
5.32 6.48 5.73 5.99
411 45 2419 54 401 101 4703 82
0.87 1.83 1.56 3.14
BJ
Struct. Sem. Sem. Distill. TrajRap
0.836 0.849 0.856 0.876
0.167 0.210 0.259 0.308
0.127 0.183 0.233 0.293
0.238 0.284 0.234 0.287
2.81 5.54 2.44 4.74
486 1594 477 4399
0.96 1.60 1.47 3.12
50 68 91 90
achieves very low POI Recall (as low as 0.015 in SF), indicating that without access to semantic context the LLM cannot ground its descriptions in real place names. Adding H3 semantic descriptions (Sem.) substantially improves grounding (POI-R jumps from 0.12 to 0.41 in Porto), while knowledge distillation (Sem. Distill.) and the full TrajRap pipeline further improve language metrics (ROUGE-L, METEOR) while maintaining reasonable spatial grounding. Notably, Sem. Distill. achieves strong ROUGE-L and METEOR despite receiving no semantic input at inference, suggesting that spatial semantics can be partially internalized through fine-tuning, though POI Recall drops sharply without direct access to area descriptions. Across all settings, ROUGE-L stays below 0.33 and POI-R remains far from 1.0, indicating that faithful, well-grounded trajectory captioning is far from solved. Overall takeaway. We distill four cross-task findings from the results above and the supplementary analyses in the Appendix. (1) Language consistently helps, but the task is far from solved. Across all three tasks, methods that leverage natural-language semantics outperform geometry-only or structure-only baselines. Yet absolute performance remains low on every metric (Dest-Hit ≤ 0.20, R@1 ≤ 0.23, POI-R ≪ 1.0), confirming that TrajPrism poses a meaningful and unsaturated evaluation challenge. (2) Each task has a distinct bottleneck. Generation is limited primarily by destination selection: oracle analysis (Table 7) shows that Jaccard nearly doubles when the correct destination is given, while the routing algorithm itself is not the main source of error. Retrieval struggles most with temporal and preference-based intents, whereas spatially unambiguous queries are better handled (Figure 8). Captioning saturates on fluency (BS-F1 > 0.83) but fails on spatial grounding, with POI Recall far below 1.0 across all methods. (3) Map-level semantic context is a shared enabler. Incorporating H3 area descriptions yields consistent gains on all three tasks: improving constraint grounding in generation, enabling crossmodal fusion in retrieval (TrajFuse leads on J@1 and SR@5), and boosting POI Recall from 0.12 to 0.41 in captioning. This suggests that structured spatial knowledge is a key ingredient for bridging geometry and language. (4) Difficulty varies across cities and instruction styles. Beijing is hardest for generation due to longer trajectories and larger road networks. Porto poses the greatest retrieval challenge (lowest R@1). Literal instructions consistently yield higher fidelity than Concise and Chatty variants (Figure 7), These patterns highlight that language-grounded trajectory understanding requires models to jointly handle spatial, temporal, and linguistic complexity across diverse urban settings.
5
Conclusion
We presented TrajPrism, a multi-task benchmark for evaluating language–trajectory alignment on real urban GPS trajectories. By unifying generation, retrieval, and captioning in one framework, TrajPrism enables joint assessment of trajectory fidelity, retrieval quality, and language groundedness on the same observed trips. Proof-of-concept experiments confirm that current models leave substantial headroom on all tasks, establishing language–trajectory alignment as an open challenge. We release the benchmark, code, and a reproducible annotation pipeline portable across cities. We hope TrajPrism serves as a foundation for advancing language-grounded urban trajectory understanding. 9
Acknowledgments and Disclosure of Funding We express our gratitude to Sharon AI for providing access to NVIDIA H100 GPUs. We acknowledge the resources and services from the National Computational Infrastructure (NCI), which is supported by the Australian Government.
10
References [1] S. Agarwal, L. Ahmad, J. Ai, S. Altman, A. Applebaum, E. Arbus, R. K. Arora, Y. Bai, B. Baker, H. Bao, et al. gpt-oss-120b & gpt-oss-20b model card. arXiv preprint arXiv:2508.10925, 2025. [2] S. Banerjee and A. Lavie. Meteor: An automatic metric for mt evaluation with improved correlation with human judgments. In Proceedings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization, pages 65–72, 2005. [3] J. Cao, T. Zheng, Q. Guo, Y. Wang, J. Dai, S. Liu, J. Yang, J. Song, and M. Song. Holistic semantic representation for navigational trajectory generation. arXiv preprint arXiv:2501.02737, 2025. [4] Y. Chang, J. Qi, Y. Liang, and E. Tanin. Contrastive trajectory similarity learning with dualfeature attention. In 2023 IEEE 39th International conference on data engineering (ICDE), pages 2933–2945. IEEE, 2023. [5] L. Chen, M. T. Özsu, and V. Oria. Robust and fast similarity search for moving object trajectories. In Proceedings of the 2005 ACM SIGMOD international conference on Management of data, pages 491–502, 2005. [6] G. Comanici, E. Bieber, M. Schaekermann, I. Pasupat, N. Sachdeva, I. Dhillon, M. Blistein, O. Ram, D. Zhang, E. Rosen, et al. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. arXiv preprint arXiv:2507.06261, 2025. [7] L. Gong, Y. Lin, X. Zhang, Y. Lu, X. Han, Y. Liu, S. Guo, Y. Lin, and H. Wan. Mobility-llm: Learning visiting intentions and travel preference from human mobility data with large language models. Advances in Neural Information Processing Systems, 37:36185–36217, 2024. [8] J. Han, Y. Ning, Z. Yuan, H. Ni, F. Liu, T. Lyu, and H. Liu. Large language model powered intelligent urban agents: Concepts, capabilities, and applications. arXiv preprint arXiv:2507.00914, 2025. [9] E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations, 2022. [10] E. Keogh and C. A. Ratanamahatana. Exact indexing of dynamic time warping. Knowledge and information systems, 7(3):358–386, 2005. [11] L. Li, H. Xue, S. Ao, Y. Song, and F. Salim. Hit-jepa: A hierarchical self-supervised trajectory embedding framework for similarity computation. arXiv preprint arXiv:2507.00028, 2025. [12] L. Li, H. Xue, Y. Song, and F. Salim. T-jepa: A joint-embedding predictive architecture for trajectory similarity computation. In Proceedings of the 32nd ACM international conference on advances in geographic information systems, pages 569–572, 2024. [13] X. Li, K. Zhao, G. Cong, C. S. Jensen, and W. Wei. Deep representation learning for trajectory similarity computation. In 2018 IEEE 34th international conference on data engineering (ICDE), pages 617–628. IEEE, 2018. [14] Z. Li, L. Xia, J. Tang, Y. Xu, L. Shi, L. Xia, D. Yin, and C. Huang. Urbangpt: Spatio-temporal large language models. In Proceedings of the 30th ACM SIGKDD conference on knowledge discovery and data mining, pages 5351–5362, 2024. [15] C.-Y. Lin. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pages 74–81, 2004. [16] T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick. Microsoft coco: Common objects in context. In European conference on computer vision, pages 740–755. Springer, 2014. 11
[17] Y. Lin, Z. Zhou, Y. Liu, H. Lv, H. Wen, T. Li, Y. Li, C. S. Jensen, S. Guo, Y. Lin, et al. Unite: A survey and unified pipeline for pre-training spatiotemporal trajectory embeddings. IEEE Transactions on Knowledge and Data Engineering, 37(3):1475–1494, 2024. [18] Z. Ma, Z. Tu, X. Chen, Y. Zhang, D. Xia, G. Zhou, Y. Chen, Y. Zheng, and J. Gong. More than routing: Joint gps and route modeling for refine trajectory representation learning. In Proceedings of the ACM Web Conference 2024, pages 3064–3075, 2024. [19] Z. Nussbaum, J. X. Morris, B. Duderstadt, and A. Mulyar. Nomic embed: Training a reproducible long context text embedder. arXiv preprint arXiv:2402.01613, 2024. [20] B. A. Plummer, L. Wang, C. M. Cervantes, J. C. Caicedo, J. Hockenmaier, and S. Lazebnik. Flickr30k entities: Collecting region-to-phrase correspondences for richer image-to-sentence models. In Proceedings of the IEEE international conference on computer vision, pages 2641–2649, 2015. [21] Qwen Team. Qwen3.5: Towards native multimodal agents, February 2026. [22] Z. Song, J. Zhang, C. Qin, C. Wang, C. Chen, L. Xu, K. Liu, X. Chu, and H. Zhu. Mobilitybench: A benchmark for evaluating route-planning agents in real-world mobility scenarios. arXiv preprint arXiv:2602.22638, 2026. [23] J. Wang, R. Jiang, C. Yang, Z. Wu, M. Onizuka, R. Shibasaki, N. Koshizuka, and C. Xiao. Large language models as urban residents: An llm agent framework for personal mobility generation. Advances in Neural Information Processing Systems, 37:124547–124574, 2024. [24] Y. Wang, C. Yang, J. Wang, X. Xu, J. Xu, D. Li, C. Xiao, and R. Jiang. Ellmob: Event-driven human mobility generation with self-aligned llm framework. arXiv preprint arXiv:2603.07946, 2026. [25] W. Wongso, H. Xue, and F. Salim. Genup: Generative user profilers as in-context learners for next poi recommender systems. In Proceedings of the 33rd ACM International Conference on Advances in Geographic Information Systems, pages 436–439, 2025. [26] W. Wongso, H. Xue, and F. D. Salim. Massive-steps: Massive semantic trajectories for understanding poi check-ins–dataset and benchmarks. arXiv preprint arXiv:2505.11239, 2025. [27] D. Xie, F. Li, and J. M. Phillips. Distributed trajectory similarity search. Proceedings of the VLDB Endowment, 10(11):1478–1489, 2017. [28] J. Xie, K. Zhang, J. Chen, T. Zhu, R. Lou, Y. Tian, Y. Xiao, and Y. Su. Travelplanner: A benchmark for real-world planning with language agents. arXiv preprint arXiv:2402.01622, 2024. [29] X. Yang, H. Ge, J. Wang, Z. Fan, R. Jiang, R. Shibasaki, and N. Koshizuka. Causalmob: Causal human mobility prediction with llms-derived human intentions toward public events. In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 1, pages 1773–1784, 2025. [30] D. Yao, G. Cong, C. Zhang, and J. Bi. Computing trajectory similarity in linear time: A generic seed-guided neural metric learning approach. In 2019 IEEE 35th international conference on data engineering (ICDE), pages 1358–1369. IEEE, 2019. [31] T. Zhang, V. Kishore, F. Wu, K. Q. Weinberger, and Y. Artzi. Bertscore: Evaluating text generation with bert. arXiv preprint arXiv:1904.09675, 2019. [32] E. Zhao, P. Awasthi, Z. Chen, S. Gollapudi, and D. Delling. Semantic routing via autoregressive modeling. Advances in Neural Information Processing Systems, 37:10060–10087, 2024. [33] L. Zheng, W.-L. Chiang, Y. Sheng, S. Zhuang, Z. Wu, Y. Zhuang, Z. Lin, Z. Li, D. Li, E. Xing, et al. Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in neural information processing systems, 36:46595–46623, 2023. [34] Y. Zheng, X. Xie, W.-Y. Ma, et al. Geolife: A collaborative social networking service among user, location and trajectory. IEEE Data Eng. Bull., 33(2):32–39, 2010. 12
[35] Y. Zheng, L. Zhang, X. Xie, and W.-Y. Ma. Mining interesting locations and travel sequences from gps trajectories. In Proceedings of the 18th international conference on World wide web, pages 791–800, 2009. [36] S. Zhou, Y. Chen, S. Shang, L. Chen, B. He, and R. Shibasaki. Blurred encoding for trajectory representation learning. arXiv preprint arXiv:2511.13741, 2025. [37] Y. Zhu, J. J. Yu, X. Zhao, X. Zhou, L. Han, X. Wei, and Y. Liang. Unitraj: Learning a universal trajectory foundation model from billion-scale worldwide traces. arXiv preprint arXiv:2411.03859, 2024.
13
A
Appendix
A.1
Evaluation Metrics
We provide formal definitions of all evaluation metrics used in TrajPrism. A.1.1
Task 1: Navigation Instruction Following
Destination Accuracy. • Destination Hit Rate (Dest-Hit): fraction of predictions whose final H3 cell matches the groundtruth destination. • Endpoint Distance (Dist): geodesic distance (km) between the predicted and ground-truth endpoints. • Destination Hit within K Hops (H@K): fraction of predictions whose endpoint is within K road-segment hops of the ground-truth destination. Trajectory Fidelity. • Jaccard (Jac): Jaccard similarity over the sets of H3 cells traversed by the predicted and groundtruth trajectories. • Dynamic Time Warping (DTW) [10]: DTW distance (km) between the GPS coordinate sequences. • Hausdorff (Haus) [27]: Hausdorff distance (km) measuring the worst-case spatial deviation. • Edit Distance on Real sequences (EDR) [5]: normalized edit distance under a spatial threshold. Oracle Metrics (O-prefix). Retrieval-based routing methods first retrieve multiple destination candidates together with associated spatial constraints (waypoints, route preferences), then generate a route for each candidate and select the final prediction. To disentangle destination-selection errors from route-planning errors, we introduce oracle variants of the above metrics: for each query we generate routes for all retrieved candidates and select the one whose endpoint is closest to the groundtruth destination, then evaluate that route. The oracle metrics (O-H@5, O-Dest, O-Dist, O-Jac, O-DTW, O-Haus, O-EDR) share the same definitions as their standard counterparts but reveal the performance ceiling achievable if the best candidate were always chosen, thereby isolating the routing stage from the candidate selection stage. A.1.2
Task 2: Trajectory Retrieval
Spatial Overlap. • Jaccard at K (J@K): mean Jaccard similarity between the top-K retrieved trajectories and the ground truth. • Soft Recall at K (SR@K): fraction of queries for which at least one of the top-K retrievals has Jaccard > 0.8 with the ground truth. Ranking Quality. • Hard Recall at K (R@K): fraction of queries for which the ground-truth trajectory appears in the top-K results. • Mean Reciprocal Rank (MRR): mean of the reciprocal rank of the first correct result across all queries. A.1.3
Task 3: Trajectory Captioning
Language Quality. • BERTScore F1 (BS-F1) [31]: token-level semantic similarity between predicted and reference captions. 14
• ROUGE-L (R-L) [15]: longest common subsequence overlap. • METEOR [2]: alignment-based metric accounting for synonyms and stemming. Spatial Grounding. • POI Recall (POI-R): proportion of ground-truth POI mentions correctly captured in the generated caption. • Named Location Count (N-Loc.): number of distinct named locations mentioned, measuring spatial detail coverage. A.2
Proof-of-Concept Models: TrajFuse and TrajRap
save
Fuse
InfoNCE Loss (Training only)
Input Query “Trips from Praça Guilherme Gomes Fernandes in Cedofeita to Campo dos Mártires da Pátria that stay in park-like areas and avoid main shopping streets.”
Query & retrieve
Figure 4: TrajFuse architecture (Task 2). A dual-encoder framework fuses geometric trajectory embeddings (from a fine-tuned TrajCL encoder) with H3-cell semantic embeddings and aligns them with text query embeddings via contrastive learning for cross-modal trajectory retrieval. TrajFuse (Task 2) addresses language-driven trajectory retrieval via a dual-encoder contrastive framework (Figure 4). On the trajectory side, each route is represented through two complementary branches: (i) a Geo Encoder (TrajCL [4]) that takes the raw trajectory point sequence and produces a geometric embedding geo_feat capturing spatial shape and sequencing; (ii) a Sem Encoder that encodes the H3-cell semantic context (POIs, land use, district descriptions) along each visited cell V1 , . . . , VN into a semantic embedding sem_feat using the same text encoder [19] as the query side. The two embeddings are combined by a Fuse module (concat + linear projection) into a single trajectory representation, which is saved to a trajectory database. On the query side, the retrieval query is encoded by the shared text encoder into query_emb. During training, an InfoNCE loss with a learnable temperature aligns the fused trajectory embeddings with the corresponding query embeddings; at inference, retrieval is performed via cosine similarity between query_emb and the trajectory database. TrajRap (Retrieval-Augmented Profiling, Task 3) generates factual trajectory captions through a retrieval-augmented few-shot prompting pipeline (Figure 5). Step 1 (Example Retrieval): given a test trajectory, we encode its textual representation with a text embedding model [19] and retrieve the top-K most similar training trajectories via cosine similarity over a pre-built index of the training set. The gold captions of the retrieved trajectories are collected as few-shot references. Step 2 (Prompt Assembly): the retrieved captions are placed into the prompt as style and granularity references, together with the test trajectory’s structural features (bearing changes, duration, road names) and, when available, H3-cell semantic descriptions. Step 3 (Caption Generation): the assembled prompt is fed to an LLM (Qwen-3.5 4B), which generates a grounded caption conditioned on both the few-shot examples and the trajectory context, without any model fine-tuning. We additionally evaluate two zero-shot ablation baselines: (i) Struct.: structural features only, no retrieval; (ii) Sem.: structural + H3 semantic context, no few-shot retrieval; as well as a distillation variant, (iii) Sem. Distill.: the LLM is LoRA fine-tuned [9] with full semantic context but tested with structural features only, probing whether spatial semantics can be internalized through training. 15
Figure 5: TrajRap pipeline (Task 3). Similar training trajectories are retrieved and their gold captions serve as few-shot examples, which are fed alongside the test trajectory’s structural features to an LLM for factual captioning. Average (Jac=0.429)
Good (Jac=0.923)
GT end
Poor (Jac=0.038)
Start GT end GT end
Start
Start Jac=0.923 Dest
Haus=0.202 km
Ground truth
Jac=0.429 Dest
Predicted
Haus=0.785 km
Start
GT end
Pred end
Jac=0.038 Dest
Haus=2.062 km
Figure 6: Task 1 qualitative comparison: good, moderate, and poor predictions. Each column shows the ground-truth trajectory (blue) and the TrajAnchor prediction (red) for one Porto test case. Left: the predicted route closely follows the ground truth with correct destination and waypoints. Middle: the destination is approximately correct but the predicted route diverges in the middle segment. Right: the destination is entirely wrong, producing a route with low spatial overlap.
Figure 6 presents three representative Task 1 predictions spanning a range of quality levels. In the good case (left), TrajAnchor correctly identifies the destination and reproduces the intended route shape, yielding high Jaccard and low DTW. The moderate case (middle) shows that even when the endpoint is approximately correct, mid-route deviations (e.g., taking a parallel street or missing a waypoint turn) substantially reduce Jaccard while keeping endpoint distance low, highlighting the gap between destination accuracy and full trajectory fidelity. In the poor case (right), incorrect destination retrieval cascades into a completely different route, confirming the oracle analysis (Table 7) that constraints selection is the primary bottleneck in the current pipeline. Table 6 provides additional destination-proximity and routing diagnostics. H@1 is near zero for all methods, confirming that exact one-hop destination matching is extremely rare, but H@10 reaches 0.20–0.27, indicating that predicted endpoints fall within the correct neighborhood. TrajAnchor variants achieve the lowest Over-Routing rates but exhibit elevated Under-Routing, indicating a tendency to generate shorter-than-intended routes. DestSP baselines show more balanced Over/Under-Routing, as shortest-path routing naturally calibrates route length when the destination is approximately correct. ConstrSP shows the highest Under-Routing across all cities, suggesting that added waypoint constraints sometimes cause premature route termination. Table 7 reveals the performance ceiling under oracle destination selection. All metrics improve substantially: for example, Porto Jaccard rises from 0.25 (Table 3) to 0.57 under oracle, and endpoint distance drops by over 40%. This gap confirms that destination selection, rather than the routing algorithm itself, is the primary bottleneck in the current pipeline. Larger LLM backbones consistently improve oracle performance, indicating 16
Table 6: Task 1 supplementary metrics not reported in Table 3. H@K: destination hit within K hops (↑); Over-Rt. / Under-Rt.: fraction of routes >1.5× or <0.5× the ground-truth length (↓). Best per dataset in bold, second best underlined. † : Qwen3.5-4B (default); ⋆ : Claude Sonnet 4.6. Dataset Method
H@1↑ H@3↑ H@10↑ Over-Rt.↓ Under-Rt.↓
Porto
DestSP (BM25) DestSP (Embed) ConstrSP TrajAnchor (Q3.5-2B) TrajAnchor† TrajAnchor⋆ (CS4.6) TrajAnchor (Q3.5-9B)
0.0010 0.0005 0.0005 0.0008 0.0009 0.0010 0.0005
0.0527 0.0618 0.0593 0.0631 0.0610 0.0694 0.0634
0.1463 0.1720 0.1658 0.1918 0.1977 0.2110 0.2050
0.1869 0.1338 0.2051 0.0869 0.0656 0.0758 0.0606
0.2208 0.2506 0.3532 0.3455 0.3741 0.3411 0.3729
SF
DestSP (BM25) DestSP (Embed) ConstrSP TrajAnchor (Q3.5-2B) TrajAnchor† TrajAnchor⋆ (CS4.6) TrajAnchor (Q3.5-9B)
0.0002 0.0003 0.0003 0.0003 0.0004 0.0004 0.0003
0.0816 0.0942 0.0913 0.0990 0.0999 0.1048 0.0998
0.1961 0.2380 0.2339 0.2555 0.2589 0.2740 0.2552
0.2073 0.1812 0.1458 0.1457 0.1148 0.1256 0.1353
0.1704 0.1898 0.4756 0.2378 0.2760 0.2449 0.2434
BJ
DestSP (BM25) DestSP (Embed) ConstrSP TrajAnchor (Q3.5-2B) TrajAnchor† TrajAnchor⋆ (CS4.6) TrajAnchor (Q3.5-9B)
0.0013 0.0022 0.0019 0.0024 0.0022 0.0022 0.0020
0.0699 0.1156 0.1084 0.1194 0.1250 0.1400 0.1297
0.1455 0.2224 0.2575 0.2323 0.2391 0.2586 0.2434
0.2869 0.2278 0.1730 0.2097 0.2034 0.1932 0.2030
0.1554 0.1790 0.4483 0.2041 0.2005 0.1920 0.1963
that stronger language-to-map grounding yields higher-quality destination candidates even when the final selection is imperfect. Beijing retains the largest absolute DTW and Hausdorff values even under oracle, reflecting the intrinsic difficulty of its longer and more complex road network. Table 8 extends the retrieval evaluation with J@10, SR@10, and R@50. TrajFuse leads on J@10 across all three cities (Porto 0.64, SF 0.63, BJ 0.60), confirming that fusing geometric and semantic representations yields the best spatial overlap at deeper retrieval depths. The fine-tuned Sem. encoder is competitive on SR@10 and R@50, occasionally surpassing TrajFuse (e.g., Porto R@50: 0.55 vs. 0.53), suggesting that pure text-aligned embeddings excel at retrieving at least one high-quality match. Among geometric-only baselines, TrajCL consistently outperforms UniTraj, T-JEPA, and BLUE, while UniTraj collapses on BJ (J@10 = 0.07), highlighting the difficulty of cross-city generalization for pre-trained trajectory encoders. Zero-shot Sem. trails all fine-tuned methods, confirming that contrastive fine-tuning on in-domain trajectory–text pairs is essential for competitive retrieval. Figure 7 compares Task 1 trajectory generation quality (Jaccard over H3 cells) across the three instruction styles. Literal instructions consistently yield the highest fidelity, as they provide explicit route constraints, while Concise and Chatty variants degrade gracefully. TrajAnchor maintains relatively stable performance across styles compared to the DestSP baselines, suggesting that retrievalaugmented routing is more robust to variations in instruction verbosity. Figure 8 breaks down Task 2 retrieval performance (MRR@10) by intent focus dimension. TrajFuse and the fine-tuned Sem. encoder dominate on most axes, but performance varies sharply across intent types: Exact Anchor and Strict Waypoint queries (where the target is spatially unambiguous) yield the highest MRR across all methods, while Duration & Pace and Avoid/Prefer Area remain consistently difficult, suggesting that temporal and preference-based intents are harder to capture in current trajectory–text alignment. Purely geometric encoders (UniTraj, BLUE) trail on nearly every dimension, confirming that semantic augmentation is critical for language-driven retrieval. Beijing exhibits the steepest cross-dimension variance, reflecting the greater spatial complexity of its road network. 17
Table 7: Task 1 oracle analysis: performance upper bound when the best destination candidate is selected from the retrieval pool. Metrics mirror Table 3 (O- prefix = oracle counterpart). ConstrSP is omitted as it does not use retrieval. Best per dataset in bold, second best underlined. † : Qwen3.5-4B (default); ⋆ : Claude Sonnet 4.6. Dataset
Method
O-H@5↑
O-Dest↑ O-Dist(km)↓
O-Jac↑
O-DTW↓
O-Haus↓
O-EDR↓
Porto
DestSP (BM25) DestSP (Embed) TrajAnchor (Q3.5-2B) TrajAnchor† TrajAnchor⋆ (CS4.6) TrajAnchor (Q3.5-9B)
0.1929 0.2758 0.3926 0.4398 0.4389 0.4497
0.2019 0.2948 0.4096 0.4612 0.4592 0.4706
1.8152 1.3101 1.0963 0.9405 0.9421 0.9170
0.3816 0.4622 0.5298 0.5671 0.5628 0.5726
40.2706 29.9528 26.4659 23.3307 23.3345 22.7166
1.6284 1.2854 1.1205 1.0069 1.0089 0.9906
0.6546 0.5830 0.5222 0.4893 0.4931 0.4845
SF
DestSP (BM25) DestSP (Embed) TrajAnchor (Q3.5-2B) TrajAnchor† TrajAnchor⋆ (CS4.6) TrajAnchor (Q3.5-9B)
0.2049 0.2746 0.3216 0.3563 0.3648 0.3413
0.1916 0.2596 0.3037 0.3382 0.3467 0.3221
1.9974 1.6100 1.4080 1.2413 1.2765 1.2966
0.3227 0.3744 0.4093 0.4400 0.4389 0.4278
39.3191 32.1121 28.4060 25.1852 25.9969 26.2175
1.7641 1.5190 1.3716 1.2462 1.2725 1.2892
0.7255 0.6814 0.6514 0.6245 0.6247 0.6340
BJ
DestSP (BM25) DestSP (Embed) TrajAnchor (Q3.5-2B) TrajAnchor† TrajAnchor⋆ (CS4.6) TrajAnchor (Q3.5-9B)
0.1563 0.2621 0.2770 0.2819 0.3138 0.2881
0.1378 0.2373 0.2501 0.2550 0.2850 0.2606
3.3510 2.6645 2.5339 2.4807 2.3683 2.4688
0.3124 0.3920 0.4072 0.4128 0.4299 0.4154
56.0179 44.5428 42.2292 41.2324 39.6896 41.0469
2.9876 2.4794 2.3745 2.3370 2.2488 2.3258
0.6871 0.6114 0.5971 0.5905 0.5749 0.5888
Task 1 Jaccard (H3) by Instruction Style 0.033
0.25
0.025
0.25
0.007
0.20 0.15 0.10 0.05 0.00
or b) 5) SP BM2 tSP (Em Constr rajAnch T Des
P( estS
D
0.027
0.010
0.019
0.020
0.15 0.10
0.20
0.00
0.00
D
0.002
0.005
0.10 0.05
or b) 5) SP BM2 tSP (Em Constr rajAnch T Des Literal Concise Chatty
0.017
0.15
0.05
P( estS
San Francisco
0.25
0.016
0.20
Jaccard (H3)
Jaccard (H3)
Beijing
0.016
Jaccard (H3)
Porto 0.30
) mb) onstrSP jAnchor M25 C P (B estSP (E Tra D
S Dest
Figure 7: Task 1 Jaccard (H3) by instruction style across three cities. Performance is broken down by the three instruction variants (Literal, Concise, Chatty) for each baseline and TrajAnchor.
A.3 A.3.1
TrajPrism Construction Pipeline Details Trajectory and Map Inputs
Table 9 summarizes the raw data fields consumed by the TrajPrism pipeline, organized by source category. Trajectory records provide map-matched road segment sequences with timestamps; persegment attributes (name, bearing, class) are retrieved from OpenStreetMap; and per-H3-cell semantic context is derived from Overture Maps POI/land-use annotations augmented by a GNN-based geo-tag classifier and an LLM-generated (Qwen3-4B-Instruct) area narrative. A.3.2
Phase Compression
Each phase is represented as a tuple: pk = (hk , nk , dk , ∆tk , ρk , mk , sk ) 18
(2)
Table 8: Task 2 supplementary retrieval metrics (J@10, SR@10, R@50) across all cities. Combined with Table 4 for full results. Best per dataset in bold, second best underlined. Dataset Method
Porto
SF
BJ
J@10
SR@10 R@50
TF-IDF + SVD BoW + SVD
0.5529 0.5520
0.2696 0.2681
0.2837 0.2858
UniTraj TrajCL T-JEPA BLUE
0.4151 0.6189 0.5759 0.5113
0.1418 0.4132 0.3498 0.2544
0.1880 0.4804 0.4128 0.3000
Sem. (Zero-shot) Sem. Road GATv2 TrajFuse
0.2786 0.6304 0.5492 0.6426
0.1320 0.4425 0.2793 0.4401
0.2140 0.5491 0.3256 0.5286
TF-IDF + SVD BoW + SVD
0.4230 0.4199
0.1531 0.1482
0.2472 0.2361
UniTraj TrajCL T-JEPA BLUE
0.1978 0.5951 0.4328 0.4152
0.0534 0.4680 0.2682 0.2444
0.0852 0.5965 0.3995 0.3822
Sem. (Zero-shot) Sem. Road GATv2 TrajFuse
0.2805 0.5892 0.4286 0.6304
0.1887 0.4730 0.2021 0.5082
0.2807 0.6548 0.3395 0.6372
TF-IDF + SVD BoW + SVD
0.3296 0.3218
0.1080 0.1024
0.2342 0.2246
UniTraj TrajCL T-JEPA BLUE
0.0730 0.5318 0.4489 0.3009
0.0255 0.4219 0.3320 0.1440
0.0534 0.6034 0.5105 0.3072
Sem. (Zero-shot) Sem. Road GATv2 TrajFuse
0.3056 0.5280 0.3957 0.5969
0.2575 0.4443 0.2246 0.5021
0.3733 0.6248 0.4046 0.6835
Here hk is the dominant H3 cell of phase Pk , nk = |Pk | is the segment count, and dk is the phase-level heading computed by aggregating segment bearings and discretizing the result into eight compass directions. The duration ∆tk is derived from timestamps aligned with the first and last segments of the phase, while ρk ∈ {O, T, D} denotes whether the phase is the Origin, Transit, or Destination according to its position in the trajectory. mk is the set of deduplicated road names traversed in the phase, included when available. Finally, sk is the semantic area description associated with the phase’s H3 cell, incorporating road-network structure and POI context from the map-derived annotations introduced above. Figure 9 shows that this compression typically reduces trajectories to 12–15 phases across all three cities. Below is the compressed representation of a trajectory (Porto), reduced from 58 road segments to 16 semantic phases over 8 min 15 sec. We show the Origin, one representative Transit phase, and the Destination. 19
Retrieval MRR@10 by Intent Focus Dimension Soft Waypoint
Porto
Beijing
Strict Waypoint
Soft Waypoint Fuzzy Semantic
Pass-through Zone
San Francisco
Strict Waypoint
Soft Waypoint Fuzzy Semantic
Pass-through Zone
0.300 0.225 0.075
Topological & Directional
0.300
0.225
0.225
0.150
Topological & Directional
Duration & Pace Avoid / Prefer Area
TrajCL
BLUE
Sem.
GATv2
Exact Anchor
Duration & Pace Avoid / Prefer Area
Time-of-Day
T-JEPA
0.075
Exact Anchor Road Type
Topological & Directional
Time-of-Day UniTraj
0.150
0.075
Exact Anchor Road Type
Duration & Pace Avoid / Prefer Area
Fuzzy Semantic
Pass-through Zone
0.300
0.150
Road Type
Strict Waypoint
Time-of-Day
TrajFuse
Figure 8: Task 2 retrieval MRR@10 by intent focus dimension across three cities. Each axis corresponds to one of the ten intent subcategories in the TrajPrism taxonomy.
Table 9: Input data sources and per-record fields used in the TrajPrism pipeline. Category
Field
Source
Description
Trajectory
rid_list time_list mm_id
Map matching GPS data Map matching
Ordered road segment IDs Per-segment timestamps Map-matched trajectory identifier
Road Segment
Road name Bearing Length Road class
OSM Computed OSM geometry OSM
Human-readable street name Segment direction as (sin θ, cos θ) Polyline length (m) highway tag (20 classes)
H3 Cell
Geo-tag Natural env. Commercial Public fac. Residential Named POIs
GNN classifier Overture Maps Overture Maps Overture Maps Overture Maps Overture Maps
Urban/Inland, Waterfront, Green/Park, Coastal/Beach Beach, forest, park, water, etc. Dining, entertainment, services, shopping Education, government, healthcare, transport High-density, low-density, mixed Place names and POI categories
H3 Road Net.
Class lengths Neighbors Narrative
Computed H3 library LLM-generated
Road length (m) per class within the cell Adjacent H3 cell IDs (resolution 9) Natural-language area description
Porto ( =12.5) Beijing ( =15.4) San Francisco ( =11.9)
0.08
Density
0.06 0.04 0.02 0.00
1
5
10
15
20
25
Number of phases per trajectory
30
35+
Figure 9: Distribution of the number of phases per trajectory after H3-based compression. The average trajectory is reduced to 12–15 phases across all three cities, with Beijing exhibiting a higher mean due to longer average trip lengths.
20
Listing 1: Compressed phase sequence for an Porto trajectory. { "traj_id": 1626278, "meta": { "n_rids": 58, "n_phases": 16, "start_time": "Saturday, Jun 14, 2014 at 4:11 AM", "total_duration": "8 min 15 sec" }, "phases": [ { "p": 0, "role": "O", "dir": "NW", "n": 4, "duration": "55 sec", "road_names": ["Rua Prof. Vicente Jose de Carvalho"], "desc": "GNN: WATERFRONT | Natural: garden, park, water | Commercial: dining, shopping | Facilities: bar, coffee shop | District: Cedofeita" }, { "p": 1, "role": "T", "dir": "SW", "n": 2, "duration": "15 sec", "road_names": ["Rua de Clemente Meneres"], "desc": "GNN: COASTAL/BEACH | Natural: park, rock, tree | Commercial: dining | Facilities: art gallery, restaurant | District: Cedofeita" }, // ... phases 2--14 (Transit) ... { "p": 15, "role": "D", "dir": "NE", "n": 1, "duration": "0 sec", "road_names": ["Rua Direita das Campinas"], "desc": "GNN: GREEN/PARK | Natural: tree | Commercial: shopping | Facilities: supermarket | District: Ramalde" } ] }
% (within dim.)
A.3.3 80% 70% 60% 50% 40% 30% 20% 10% 0%
Intent Sampling Dim 1: Destination
Exact Anchor
Fuzzy Semantic
80% 70% 60% 50% 40% 30% 20% 10% 0%
Dim 2: Waypoint
Strict Flexible Pass-thru Seq. Feature Zone Porto Beijing
80% 70% 60% 50% 40% 30% 20% 10% 0%
Dim 3: Route Pref.
Pure Pure Orthog. Semantic Topolog. Compos. San Francisco
80% 70% 60% 50% 40% 30% 20% 10% 0%
Dim 4: Temporal
Time-of -Day
Pace & Duration
Figure 10: Conditional distribution of fine-grained semantic subcategories in TrajPrism. Percentages are normalized within each semantic dimension, showing that the benchmark evenly covers diverse destination references, waypoint constraints, route preferences, and temporal or pace cues rather than relying on a single instruction pattern. Table 10 lists the full four-dimensional travel-intent taxonomy used to construct structured intent profiles. Figure 10 shows the conditional distribution of fine-grained scenarios within each dimension, confirming balanced coverage across all 10 subcategories. Each trajectory is assigned k scenarios sampled without replacement, where k ∈ {1, 2, 3, 4, 5} follows a weighted distribution (p = [0.15, 0.35, 0.30, 0.15, 0.05]) centered at 2–3 scenarios. Single-scenario intents (k=1) are restricted to Dimension 1 (Destination only), reflecting simple navigation requests such as “take me to the hospital”. Composite intents (k ≥ 2) always include exactly one Destination scenario and draw 21
Table 10: Four-dimensional travel-intent taxonomy used for intent sampling. Each trajectory is assigned 1–5 scenarios sampled across these dimensions. Dimension
Scenario
Description
1. Destination
1.1 Exact Anchor 1.2 Fuzzy Semantic
Specific physical destination (POI or road name) Conceptual destination (e.g. “a quiet green area”)
2. Waypoint
2.1 Strict Sequential 2.2 Flexible / Feature 2.3 Pass-through Zone
Must pass through specific named road segments Stop described by semantic features only Cross an area type without stopping
3. Route Pref.
3.1 Semantic Constraints Affinity or avoidance (e.g. parks, industrial zones) 3.2 Topological / Direct. Fluency, permeability, or directional preference 3.3 Orthogonal Comp. Semantic vs. topology (e.g. “through chaos but main road”)
4. Temporal/Pace
4.1 Time-of-Day 4.2 Pace / Duration
Route choice driven by time or day of week Urgency, leisure, or deadline-driven constraints
the remaining scenarios from Dimensions 2–4, producing realistic multi-constraint requests such as “drive to the station, stop for fuel on the way, and avoid the highway”. Figure 12 shows the resulting distribution of scenario counts across the dataset, confirming that the majority of trajectories carry 2–3 intent dimensions while preserving a meaningful tail of simpler and more complex requests. A.3.4
Persona and Generation Style Statistics
To faithfully satisfy the combination of intent constraints, speaker persona, and style requirements, the model first produces a chain-of-thought analysis of the trajectory’s semantic and topological features before generating task outputs for Tasks 1 and 2. Figures 11–13 and Table 11 summarize the resulting distributions of personas, scenario counts, instruction styles, and generation guidance. 18%
Uniform (12.5%)
% of trajectories
15%
Porto
Beijing
San Francisco
12% 10% 8% 5% 2% 0%
Relaxed & Unhurried
Curious Explorer
Local Tired & Commuter Indifferent
Clueless Tourist
Impatient Picky & Anxious & Rushed Demanding about Time
Figure 11: Distribution of personas used for generating navigation instructions in TrajPrism. Uniform persona sampling promotes balanced exposure to diverse tones, preferences, and communication styles, enabling models to learn language-grounded navigation behavior beyond a single dominant voice. A.3.5
Dataset Judgement
LLM and human annotators evaluate the generated language outputs against the underlying trajectory evidence. For Task 1, annotators jointly inspect the literal, concise, and chatty navigation instructions. For Task 2, they jointly inspect the three retrieval queries. For Task 3, they inspect the trajectory caption. Each criterion is rated on a 1–5 Likert scale, where 5 indicates excellent satisfaction of the criterion and 1 indicates failure. Human annotators additionally receive an interactive map visualization of each trajectory, allowing them to zoom, pan, and inspect the route geometry and surrounding POIs before scoring. Table 13 reports mean quality scores from three LLM judges and three human judges. All criteria exceed 4.0 on average, indicating high overall annotation quality. Human judges are systematically stricter than LLMs, particularly on No Hallucination (human mean ≈3.8–4.8 vs. LLM ≥4.2), suggesting that humans detect subtle spatial fabrications that LLM judges overlook. Persona Fidelity and Style Distinctness receive the highest human scores (≥4.3), confirming that the style and persona guidance in the generation pipeline produces perceptually distinct outputs. 22
Table 11: Style and length guidance hints sampled per generation. One sentence-form hint and one length hint are drawn uniformly at random per style per trajectory. Style
Sentence-form hint pool
Literal
imperative command · declarative statement · sequenced action · context-aware continuation · telegraphic/terse · coordinate-style single word or minimal fragment · terse command · abbreviated phrase · constraint-first · destination-only question form · complaint or reaction · narrative/storytelling · casual suggestion · soft request · trailing/open-ended
Concise Chatty
Style
Length hint pool
Literal
Brief but complete (one sentence) Moderate (dest. + 1–2 constraints) Detailed (dest. + waypoints + constr.)
Concise
Ultra-terse (telegram, fragments ok) Short phrase (minimal thought) Brief sentence (concise, grammatical)
Chatty
Casual one-liner Conversational (a couple sentences) Chatty and detailed (rambling ok)
Table 12: LLM and human evaluation rubric. Each criterion is scored on a 1–5 scale: 5 = perfect/excellent, 4 = good with minor flaws, 3 = acceptable but limited, 2 = poor with severe issues, and 1 = failed/completely wrong. Criterion
Description Task 1: Navigation Instructions
Correctness No hallucination Persona fidelity Style distinctness
How well the instructions match the true trajectory in destination, direction, route constraints, and waypoints. Whether the instructions avoid inventing roads, POIs, landmarks, or environmental details not supported by the trajectory data. How naturally and consistently the instructions reflect the assigned persona across all three variants. How clearly the literal, concise, and chatty variants differ in style rather than only in length. Task 2: Retrieval Queries
Retrieval specificity Accuracy No hallucination
How well the queries capture the distinctive features needed to retrieve this trajectory from a database. How accurately the queries describe the true origin, destination, route properties, timing, and constraints. Whether the queries avoid inventing non-existent places, route constraints, or semantic cues. Task 3: Trajectory Caption
Comprehensiveness Accuracy Objectivity / purity
How completely the caption covers the start, major traversed regions/phases, and endpoint of the trajectory. How faithfully the caption reflects the true route order, direction changes, and spatial progression. Whether the caption stays fully objective, without subjective intent, anthropomorphic framing, or fabricated details.
23
40%
35%
% of trajectories
35%
30%
30% 25% 20% 15%
15%
15%
10%
5%
5% 0%
1
2
3
4
Number of intent scenarios (c)
5
Figure 12: Distribution of the number of intent scenarios assigned to each generated trajectory. The sampling strategy mixes simple destination-only requests with multi-intent instructions that combine destination, waypoint, route-preference, and temporal or pace constraints.
Instruction Length
Conversational Tone Question sentences (%)
50
Word count
40 30 20 10 0
Literal
Concise
Chatty
35 30 25 20 15 10 5 0
27.5%
0.2% Literal
0.0% Concise
Chatty
Figure 13: Token length distribution of generated navigation instructions across three stylistic variants (Literal, Concise, Chatty), confirming that the style and length guidance produces distinct verbosity profiles. Table 14 measures pairwise ±1 agreement. LLM–LLM agreement is consistently high (mean 93– 99%), with P↔Q reaching 99.5%. Human–human agreement averages 91.7% but drops on criteria requiring fine-grained spatial verification (e.g., Accuracy H2↔H3: 74.7% for Task 3), reflecting inherent subjectivity in judging factual completeness. Cross-modal human–LLM agreement averages 92.9%, validating that LLM judges can serve as reliable proxies for scalable quality control while human review remains essential for detecting hard-to-verify hallucinations. A.3.6
LLM Data Generation Prompt
Below we show the system prompt and user prompt template used for multi-task annotation generation (Claude Sonnet 4.6 as an example; Gemini and GPT-OSS use equivalent prompts). Section headers and representative rules are preserved; exhaustive examples and enumerations are abbreviated with [...].
24
Table 13: Mean quality scores (1–5 scale) from three LLM judges (N = 2,000 trajectories) and three human judges (N = 100 Porto trajectories). Criterion
LLM (N = 2K)
Human (N = 100)
G
P
Q
H1
H2
H3
Task 1: Instruction Generation Correctness 4.31 No Hallucination 4.86 Persona Fidelity 4.64 Style Distinctness 4.98
4.74 4.84 4.99 4.98
4.06 4.19 4.83 4.19
4.05 3.88 4.59 5.00
3.98 3.84 4.71 4.30
4.54 4.77 4.91 4.90
Task 2: Retrieval Query Retrieval Specificity 4.82 Accuracy 4.62 No Hallucination 4.80
4.86 4.87 4.88
4.14 4.21 4.47
4.04 3.91 3.90
4.03 3.76 3.59
4.78 4.89 4.87
Task 3: Trajectory Captioning Comprehensiveness 4.90 Accuracy 4.53 Objectivity Purity 4.90
4.99 4.98 5.00
4.63 4.52 4.79
4.13 3.90 4.85
4.16 3.77 3.89
4.99 4.98 4.99
Mean
4.91
4.40
4.22
4.00
4.86
4.74
Table 14: ±1 agreement: LLM pairs on N = 2,000; human pairs and human–LLM cross-agreement on N = 100. H↔L averages all nine human–LLM pairs. LLM↔LLM (N = 2K) Criterion
H↔H (N = 100)
(N = 100)
G↔P
G↔Q
P↔Q
H1↔H2 H1↔H3 H2↔H3
H↔L
Task 1: Instruction Generation Correctness 81.7% No Hallucination 94.8% Persona Fidelity 89.1% Style Distinctness 99.6%
87.6% 96.1% 91.2% 99.6%
99.3% 98.8% 99.7% 99.8%
99.0% 98.0% 100.0% 95.0%
96.0% 90.0% 100.0% 99.0%
89.0% 86.0% 99.0% 94.0%
92.1% 93.0% 99.1% 98.7%
Task 2: Retrieval Query Retrieval Specificity 98.2% Accuracy 89.5% No Hallucination 94.1%
99.4% 96.1% 94.3%
99.7% 99.7% 98.8%
98.0% 98.0% 96.0%
94.0% 88.0% 91.0%
85.0% 77.0% 69.0%
93.9% 90.4% 86.4%
Task 3: Trajectory Captioning Comprehensiveness 98.8% Accuracy 88.4% Objectivity Purity 97.8%
99.5% 100.0% 93.4% 99.9% 98.4% 99.8%
99.0% 100.0% 91.0%
90.9% 84.8% 100.0%
84.8% 74.7% 85.9%
93.6% 87.0% 94.4%
97.4%
93.4%
84.4%
92.9%
Mean
93.2% 95.6%
99.5%
25
System prompt (abbreviated).
## 1. Core Directive Reverse-engineer the user’s travel intent from a real driving trajectory (compressed JSONL with semantics, topology, headings). * Data Grounding [ZERO TOLERANCE]: ONLY use features in the data. Do NOT invent traffic lights, bridges, tunnels, etc. * Anti-Tour Guide: Focus on end-goal + 1-2 constraints. Selectively IGNORE intermediate phases. * Phase Alignment [ZERO TOLERANCE]: Origin from Phase 0 only; destination from last phase only; waypoints from middle only. * Parrot Ban [ZERO TOLERANCE]: NEVER copy GIS/GNN labels. WATERFRONT -> "by the river"; GREEN/PARK -> "the park"; URBAN/INLAND -> "downtown"; Commercial -> "the shops"; etc. * No em-dashes or semicolons in any output. [... persona rules, grammar, narrative handling ...] ## 2. Intent Taxonomy (4 dimensions, 10 scenarios) Dim 1 Destination: 1.1 Exact Anchor, 1.2 Fuzzy Semantic Dim 2 Waypoint: 2.1 Strict Sequential, 2.2 Flexible, 2.3 Zone Dim 3 Route Pref: 3.1 Semantic, 3.2 Topological, 3.3 Orthogonal Dim 4 Temporal: 4.1 Time-of-Day, 4.2 Pace/Duration ## 3. Diversity Strategy * Single intent (minority): Dimension 1 only. * Composite (majority): Dim 1 once + scenarios from Dims 2-4. * Anti-Cliche: rotate waypoints (gas, ATM, pharmacy, ...). [... diversity pools, rotation lists ...] ## 4. Few-Shot Examples * [BAD] "Navigate to the waterfront area." (parroting labels) * [GOOD] "Hospital. Now." (ultra-short) * [GOOD] "No rush. Take me by the river, pull over somewhere I can stretch my legs." (leisurely + waypoint) [... 15+ additional examples ...] ## 5. Three Instruction Styles Literal: faithful, explicit. Concise: ultra-short. Chatty: conversational. Each with a DIFFERENT opener. ## 6. Retrieval Queries & Caption * 3 retrieval queries (Task 2): search-style, NOT navigation. Together cover all 4 dimensions. * 1 caption (Task 3): 3rd-person, factual, present-tense. [... specificity rules, examples ...] ## 7. Output Format (JSON, no chain-of-thought) { "_intent_planning": "...", "_retrieval_planning": "...", "instruction_literal": "...", "instruction_concise": "...", "instruction_chatty": "...", "retrieval_query_1": "...", "retrieval_query_2": "...", "retrieval_query_3": "...", "trajectory_caption": "..." }
26
User prompt template (per trajectory). For this trajectory, generate three instructions (literal, concise, chatty) reflecting these intent type(s): ⟨scenario_labels ⟩ e.g. “1.1 + 2.2 + 3.2” INTENT PLANNING (required): output "_intent_planning" first. RETRIEVAL ASSIGNMENT (3 queries cover ALL 4 DIMENSIONS): - retrieval_query_1 -> ⟨assigned_dims ⟩ - retrieval_query_2 -> ⟨assigned_dims ⟩ - retrieval_query_3 -> ⟨assigned_dims ⟩ SPEAKER PERSONA: ⟨persona ⟩ e.g. “Impatient and rushed” STYLE GUIDANCE: - literal: ⟨style ⟩ [length: ⟨len ⟩] - concise: ⟨style ⟩ [length: ⟨len ⟩] - chatty: ⟨style ⟩ [length: ⟨len ⟩] ⟨optional constraints ⟩ waypoint, street name, time context NARRATIVE MODE -- IGNORE: Do not use text after "Narrative:" in phase descriptions for facts or phrasing. Output ONLY a JSON object. No reasoning. ⟨compressed_trajectory_JSON ⟩
A.4
Qualitative Examples Task 1 — Navigation Instructions [1.1 Exact Anchor · 2.1 Strict Sequential]
Literal: “Head to W Hotels via Owens Street, then 16th Street, Terry A. François Boulevard, Mission Rock Street and finish on 3rd Street.” Concise: “Via 16th, Terry A. François Boulevard, Mission Rock, 3rd to W Hotels.” Chatty: “Hey, how about take me from Owens Street down 16th Street, swing onto Terry A. François Boulevard, cut through Mission Rock Street and finish on 3rd Street at the W Hotels? It’s a quick morning run, so get me there by 10:10.” Task 2 — Retrieval Queries: Query 1: “Trips from Owens Street to a beachside spot that pass sequentially through 16th Street, Terry A. François Boulevard and Mission Rock Street.” [2.1 Strict Sequential · 1.2 Fuzzy Semantic] Query 2: “Recorded drives from Owens Street to W Hotels that mainly follow major roads and keep a westward bearing.” [3.2 Topological/Direct.]
Query 3: “Morning weekday routes starting at Owens Street around 10 AM and ending at W Hotels.” [4.1 Time-of-Day] Task 3 — Trajectory Caption: Caption: “Around 10:03 AM on a Monday, the drive departs from Owens Street, heads southwest onto 16th Street, then turns onto Terry A. François Boulevard, continues westward through Mission Rock Street and merges onto 3rd Street, arriving at the W Hotels after roughly seven minutes.”
Figure 14: Qualitative example from San Francisco (traj_id: 475). Left: ground-truth trajectory on the road network. Right: seven generated annotations spanning three tasks.
27
Task 1 — Navigation Instructions [1.1 Exact Anchor] Literal: “Head to Yonghe King near the KFC.” Concise: “Yonghe King. Quick.” Chatty: “Man, I need to get to Yonghe King. Just take the usual route past the KFC.” Task 2 — Retrieval Queries: Query 1: “Recorded Sunday night trips from Lianhua Bridge station to Yonghe King that were taken after 10 PM.” [4.1 Time-of-Day] Query 2: “Historical routes from Lianhua Bridge station to Yonghe King.” [1.1 Exact Anchor] Query 3: “Trips from Lianhua Bridge station to Yonghe King that travel via Xisihuan South Road and then continue onto Kexing Road, heading east on the ring road and turning north onto Kexing Road.” [2.1 Strict Sequential · 3.2 Topological/Direct.] Task 3 — Trajectory Caption: Caption: “Starting at Lianhua Bridge station in the western suburbs, the 13-minute drive first heads west on local streets, then swings east onto a major ring road and later onto its middle segment. It continues eastward along the southern part of that ring road before turning north onto the street with the fast-food outlet, where it finishes at the Yonghe King fast-food outlet next to a KFC. The route weaves through mixed residential blocks, passes a few parks, and stays on the main thoroughfares throughout.”
Figure 15: Qualitative example from Beijing (traj_id: 9331). Left: ground-truth trajectory on the road network. Right: seven generated annotations spanning three tasks. Task 1 — Navigation Instructions [1.1 Exact Anchor · 3.3 Orthogonal Comp. · 4.1 Time-of-Day · 4.2 Pace/Duration]
Literal: “To Ponte D via the park-side roads. It’s 2 AM, traffic’s dead, so just push straight through. No detours, no stops.” Concise: “Ponte D. Night run. Park roads. Fast.” Chatty: “Okay so it’s the middle of the night, just get me to Ponte D. Cut through that green stretch, it’s dead quiet at this hour anyway. Don’t mess around with the busy streets back in Cedofeita, just punch through and get there. Nine minutes, tops.” Task 2 — Retrieval Queries: Query 1: “Trips from the Cedofeita area to Ponte D in Lordelo do Ouro e Massarelos that pass continuously through the park and tree-lined corridor without any recorded stops along the way.” [2.3 Pass-through Zone]
Query 2: “Recorded routes from Cedofeita to the Ponte D area in Lordelo do Ouro e Massarelos that favor the quieter, greener by the riverside and park-side streets rather than the denser commercial or high-traffic roads.” [3.1 Semantic Constraints] Query 3: “Historical drives ending at Ponte D in Lordelo do Ouro e Massarelos, originating from the Cedofeita district, recorded in the early-morning hours around 2 AM on a weekend night when roads are largely empty.” [1.1 Exact Anchor · 4.1 Time-of-Day] Task 3 — Trajectory Caption: Caption: “Starting in the Cedofeita district around 2:05 AM on a Saturday night, the route heads southeast before pivoting northwest and west, passing through a mix of dense urban streets near the city center and transitioning into the docks and park corridor of Lordelo do Ouro e Massarelos. The route travels along Rua do Campo Alegre for several phases, moving through tree-lined and park-adjacent zones including areas near the University of Porto campus, before concluding at the Ponte D area on Rua de Diogo Botelho in Lordelo do Ouro e Massarelos. The full trip covers 12 phases in approximately 9 minutes, with the bulk of travel time spent in the quieter green and the riverside zones during late-night, low-traffic conditions.”
Figure 16: Qualitative example from Porto (traj_id: 1373490). Top-left: ground-truth trajectory. Top-right: Task 1 and 2 annotations. Bottom: Task 3 caption.
28
A.5
Limitations
TrajPrism currently covers three cities with English-language annotations (except for native-language place names). The reproducible RIR pipeline facilitates extension to additional cities and languages. Travel-intent instructions are LLM-synthesized with multi-stage quality control, which may introduce distributional biases compared to organically collected user queries. All trajectories are map-matched GPS traces. Indoor, pedestrian, and multimodal transportation mode settings are not yet addressed.
29