Temporally Grounded Compositional Camera Motion Understanding via Geometric Knowledge Distillation

Preprint

T EMPORALLY G ROUNDED C OMPOSITIONAL C AMERA M OTION U NDERSTANDING VIA G EOMETRIC K NOWL EDGE D ISTILLATION Dazhao Du1,2,∗ , Shiyan Du2 , Jian Liu1 , Yongjian Yu2 , Bohai Gu1 , Tao Han1 , Hualuo Liu2 , Eric Liu2 , Yujia Zhang2 , Xi Chen2 , Song Guo1,† 1

The Hong Kong University of Science and Technology

2

Tencent

arXiv:2608.10932v1 [cs.CV] 11 Aug 2026

A BSTRACT Understanding camera motion is fundamental to video perception, with applications in spatial intelligence and controllable video generation. Multimodal large language models (MLLMs) provide a natural interface for this task, but existing work typically assigns one or more labels to an entire clip. Such clip-level recognition overlooks two defining properties of real camera motion: it can change within a shot, and multiple movements can occur simultaneously. We therefore formulate camera-motion understanding as temporally grounded, compositional recognition, which requires a model to localize motion-consistent intervals and identify every movement active within each interval. We introduce C AM C HOREO, a benchmark of 4,229 real single-shot clips with expert-annotated temporal segments. Its annotations use a compact vocabulary of 20 direction-aware labels, and nearly half of the segments contain compound camera motion, with multiple movement primitives active simultaneously. Recognizing such fine-grained, compositional motion is hard for current MLLMs, whose visual encoders emphasize semantic content rather than the geometric evidence on which camera motion depends. Directly injecting features from a frozen 3D foundation model addresses this gap, but requires running the expensive geometry model on every input; we refer to this baseline as C AM I NJECT. We instead propose C AM D ISTILL, which distills the same geometric knowledge into lightweight camera tokens during training and removes the 3D model at inference. C AM D ISTILL matches the accuracy of direct feature injection without running the 3D teacher at inference. Together, C AM C HOREO and C AM D ISTILL advance camera-motion understanding from clip-level labeling to temporally grounded, compositional recognition. Project page: https://ddz16.github.io/cammotion.github.io/.

1

I NTRODUCTION

A video contains motion in the scene and motion of the camera observing it. Camera motion has its own expressive vocabulary and a long tradition in film grammar (Spottiswoode, 1959; Yilmaz et al., 2023): a pan redirects the viewer’s gaze, a dolly reveals depth through parallax, and a zoom reframes the shot while the camera remains stationary. Recognizing these movements requires separating changes caused by the observer from those occurring in the scene. This capability is important for spatial intelligence (Zhang et al., 2026), controllable video generation (Bai et al., 2025a; Xing et al., 2025), and cinematic analysis (Wang et al., 2026b). Nevertheless, most camera-motion benchmarks still formulate the problem as single- or multi-label classification over an entire clip (Lin et al., 2026; Liu et al., 2026; Feng et al., 2026). That formulation does not reflect real footage: even within an uninterrupted shot, the camera may transition between movements or execute several movements at once. Clip-level labels consequently lose both temporal structure and physical composition. We instead formulate the problem as temporally grounded, compositional recognition. Given a video, the model partitions each shot into motion-consistent intervals and predicts the complete set ∗ †

Work done during an internship at Tencent. Corresponding author.

1

Preprint

0s

Pan Right, Truck Left, Arc CW

Dolly In

1.6s

5.9s

3.4s

Pan Left, Truck Right

Pan Up

11.2s

Left

Pedestal Up

Right CCW

Optical

Focus Shift

In Out

Temporally Grounded 0.1-second granularity

Translation Roll

Down

Zoom

Dolly In

Pedestal Up, Tilt Down, Dolly Out

Rotation Tilt

Comprehensive Taxonomy Compositional Motion

CW

Truck Left

Dolly

Right

Out

In

Down

Subject-referenced Arc

Follow

Stability

Static

Unstable

CW

CCW 2026/8/6 Dazhao Du NTP for Training MLLMs Figure 1. Temporally grounded compositionalBeyond camera motion. A shot is represented by motion-consistent intervals, each carrying all simultaneous movements. C AM C HOREO covers 12 types (20 direction-aware labels) grouped into five families: rotation, translation, optical, subject-referenced, and stability.

of direction-aware movements active in each interval. Figure 1 illustrates the task: the top panel annotates one shot as a sequence of intervals, each carrying several simultaneous movements, and the bottom panel shows our taxonomy grouped into five families. The task thus asks what the camera does, when each movement occurs, and which movements co-occur, none of which cliplevel classification can isolate. To support it, we introduce C AM C HOREO, named for how a shot choreographs camera-motion primitives over time. It contains 4,229 real single-shot YouTube clips, 8,591 expert-annotated segments, and 14,258 motion instances, covering 12 movement types and 20 direction-aware labels grounded in classical camera terminology (Nielsen et al., 2007), across nine content domains with boundaries at 0.1-second resolution. Within this benchmark, temporal variation and compound motion are common: 2,411 clips contain multiple segments, and 3,797 segments contain compound camera motion, with multiple movement primitives occurring simultaneously. Current MLLMs perform poorly in this setting, revealing a representational gap: their vision encoders are optimized for semantic alignment (Radford et al., 2021; Tschannen et al., 2025), while camera motion depends on cross-frame geometry, including parallax, perspective change, and horizon rotation. Injecting features from a frozen 3D foundation model helps, but this baseline, C AM I NJECT, must run the expensive geometry model on every test video. We propose C AM D ISTILL to retain the geometric benefit without this inference-time dependency. A lightweight Geometry-aware Camera Token Extractor (GCTE) predicts one camera token per frame from intermediate frozen vision features (Dosovitskiy et al., 2020). A distillation objective aligns these tokens with the camera representation of a frozen 3D teacher. Through this objective, the model learns a geometry-informed camera representation during training. At inference, the 3D teacher is removed, leaving a compact geometry-aware stream with almost no runtime overhead. Experiments confirm both the difficulty of the task and the value of geometric supervision. On C AM C HOREO, the strongest closed-source MLLM reaches 43.3 frame-level micro F1. SFT raises a 4B model to 62.2, and C AM D ISTILL further improves it to 67.5, matching direct injection without running the 3D model at inference. The distilled representation also transfers to external benchmarks with different task formats, suggesting that it captures reusable camera-motion cues. Our contributions are threefold: • Task and benchmark. We formulate temporally grounded, compositional camera-motion recognition and introduce C AM C HOREO, to our knowledge the first real-video benchmark combining variable-length segments with direction-aware multi-label annotations. 2

1

Preprint

• Empirical diagnosis. We show that within-shot transitions and simultaneous movements are common, and that generic MLLMs and geometry-only pose rules remain inadequate. • Efficient geometry distillation. We propose C AM D ISTILL, whose GCTE predicts per-frame camera tokens from frozen visual features and aligns them with a 3D teacher during training. It matches direct feature injection while removing the teacher and its cost at inference.

2

R ELATED W ORK

2.1

C AMERA M OTION AND C INEMATOGRAPHY B ENCHMARKS

Film theory has long studied camera movement as a device with narrative and emotional functions (Spottiswoode, 1959; Nielsen et al., 2007; Yilmaz et al., 2023). Recent cinematography benchmarks evaluate camera movement alongside shot scale, lighting, and composition, mainly through clip-level classification or multiple-choice QA (Li et al., 2024b; Tang et al., 2025; Wang et al., 2026b; Liu et al., 2026; Wu et al., 2025a). CameraBench (Lin et al., 2026) introduces a broad expert vocabulary for real videos, while CameraMotionVQA (CMVQA) (Feng et al., 2026) supports controlled multi-label recognition on one-second synthetic clips. These resources advance cameramotion recognition, but still assign a single motion set to each clip. They neither localize variablelength intervals within real shots nor evaluate how movements co-occur over time. C AM C HOREO is designed specifically for this temporally grounded, compositional setting, as summarized in Table 1. 2.2

MLLM S FOR C AMERA M OTION U NDERSTANDING

General-purpose video MLLMs (Li et al., 2024a; Zhang et al., 2024; Bai et al., 2025b; Wang et al., 2025c) provide strong semantic understanding but are not trained to perceive camera geometry. Our task also relates to video temporal grounding, in which MLLMs localize events along a timeline (Wu et al., 2025b). Recent camera-motion methods introduce structured reasoning traces, explicit pose grounding, or textual pose prompts derived from geometry models (Wu et al., 2026; Yang et al., 2026; Feng et al., 2026). Other work augments MLLMs with 3D priors (Zheng et al., 2026). These studies demonstrate the value of geometry, but either retain the geometry model at inference or compress its output into discrete text. C AM D ISTILL instead transfers the teacher’s continuous camera representation during training and removes the geometry model at inference. 2.3

C AMERA P OSE E STIMATION IN V IDEO

Classical SfM and SLAM recover camera trajectories through feature matching and geometric optimization (Schonberger & Frahm, 2016; Davison et al., 2007; Engel et al., 2014; Teed & Deng, 2021; Li et al., 2026; Zhang et al., 2022). More recent feed-forward models such as DUSt3R, VGGT, and VGGT-Ω jointly predict camera pose and 3D scene structure in a single pass (Wang et al., 2024; 2025b; 2026a), with further estimators improving robustness and multi-view consistency (Huang et al., 2025; Wang et al., 2026c). VGGT and VGGT-Ω in particular attach a dedicated camera token to each frame, from which that frame’s camera pose can be decoded. We distill the camera token itself into the MLLM, transferring a pose-associated, geometry-informed representation while leaving the mapping to camera-motion labels to the language model.

3

T HE C AM C HOREO B ENCHMARK

3.1

TASK D EFINITION

Given a video V of duration T , the model predicts a set of temporal segments Ŝ = {(ŝi , êi , Ŷi )}N i=1 ,

0 ≤ ŝi < êi ≤ T,

(1)

where ŝi and êi are the predicted start and end times, and Ŷi ⊆ C is the set of active camera-motion labels. The closed label space C contains 20 direction-aware labels derived from 12 movement types (Figure 1; Appendix D). A correct prediction must therefore recover both the segment boundaries and the complete set of co-occurring labels within each segment. 3

Preprint

Table 1. Comparison with existing cinematography and camera-motion benchmarks. C AM C HOREO is the only real-video benchmark combining camera-specific, multi-label annotation with variable-length temporal grounding. “#Cls.” counts direction-aware labels for C AM C HOREO. Benchmark

Source

#Clips

Real

#Cls.

Multi-lbl.

Cam-spec.

Temporal

Cinematic2K (Li et al., 2024b) VidComposition (Tang et al., 2025) CineTechBench (Wang et al., 2026b) ShotBench (Liu et al., 2026) CameraBench (Lin et al., 2026) CameraMotionVQA (Feng et al., 2026)

Web Movies Movies Movies Web Synthetic

2,000 982 120 464 ∼3,000 12,274

✓ ✓ ✓ ✓ ✓ ×

11 7 15 16 23 15

× ✓ ✓ × ✓ ✓

× × × × ✓ ✓

× × × × × ×

C AM C HOREO (ours)

Web

4,229

✓

20

✓

✓

✓

A. Video Duration Distribution

B. Temporal Segmentation Granularity

C. Camera Motion Label Density 5000

1818

Mean 5.86s Median 4.58s

800

400

200

Number of segments

Number of videos

Number of videos

1500 600

1229

1250 1000

686

750 500

286

250 0

0

5

10

15

20

25

4794

1750

0

30

Video duration (seconds)

147

1

2

3

4

4000

3000 2388

2000 1045

1000

280

63

5

6

Number of segments per video

D. Camera Motion Distribution

0

1

2

3

4

71

13

5

6

Number of camera motion labels per segment

E. Source Category Composition 14.1%

Static Dolly In Pan Right Pan Left Follow Truck Right Dolly Out Tilt Up Truck Left Tilt Down Pedestal Up Pedestal Down Zoom Out Zoom In Focus Shift Arc CCW Unstable Arc CW Roll CCW Roll CW

13.0% 10.7%

6.1%

9.9% 7.5% 7.0%

6.4%

19.0%

6.7%

6.3% 5.8% 5.6%

8.9%

4.8% 3.7%

15.8%

2.6% 1.5% 1.4% 1.3% 1.1% 1.0% 0.9% 0.9% 0.9%

0

9.7% 14.3%

500

1000

1500

2000

Number of camera motion annotations

Aerial & Drone Documentary & Nature Film & TV

13.1%

Vlog & Selfie Sports & Action Commercial & Ad

Gaming & Animation Synthetic & Rendered Tutorial & Education

Figure 2. C AM C HOREO statistics. (A) video duration, (B) number of segments per clip, (C) number of simultaneous movements per segment, (D) the long-tailed distribution of the 20 direction-aware labels, and (E) the nine content domains.

3.2

DATA C URATION

Collection and filtering. We collect YouTube footage from nine content domains and split each video into single shots at hard cuts with TransNetV2 (Soucek & Lokoc, 2024). The resulting shots are filtered by duration and visual quality and de-duplicated, yielding a diverse clip pool. To surface rare motions, a preliminary model assigns pseudo labels that are used only to guide candidate sampling and never enter the released annotations. The full funnel is detailed in Appendix C. Expert annotation. A team of five annotators with film- and media-related backgrounds label every clip from scratch, marking motion-consistent intervals and all active direction-aware movements. They use parallax to separate rotation from translation, perspective change to distinguish Dolly from Zoom, and scene context to separate camera from subject motion. Follow and Arc are always paired with their underlying primitive so that a semantic label never replaces the physical motion, and boundaries are placed at 0.1-second resolution wherever the active motion set changes. Annotations are cross-checked for quality. Clips with unresolved disagreements are discarded, leaving 4,229 videos in the final benchmark. Details are given in Appendix D. 3.3

DATASET S TATISTICS

C AM C HOREO contains 4,229 single-shot clips totaling 6.88 hours, with 8,591 expert-annotated segments and 14,258 movement instances. Clips average 5.9 seconds (Figure 2A), keeping the benchmark focused on within-shot camera behavior rather than editing or long-form narrative. Figure 2 summarizes the properties most relevant to the task. 4

Preprint

Temporal structure. Camera motion changes within most clips. As shown in Figure 2B, 2,411 of the 4,229 clips contain multiple segments, with some containing as many as six. Clip-level annotation would therefore merge distinct motion phases in more than half of the benchmark. Compound camera motion. In the curated benchmark, 3,797 of the 8,591 segments (44.2%) contain compound camera motion with at least two simultaneous movement primitives, and some contain three or more (Figure 2C). A single-label prediction therefore drops part of the active camera state in nearly half of the benchmark segments. Long-tailed labels and domains. The 20 direction-aware labels follow a pronounced long-tailed distribution (Figure 2D). Static, Dolly, and Pan are frequent, whereas Zoom, Roll, Arc, and Focus Shift are rare. The clips span nine content domains, with aerial, documentary, and film footage contributing the largest shares (Figure 2E). Comparison with other benchmarks. Table 1 highlights two limitations of existing benchmarks. General cinematography datasets cover camera movement only as one attribute among many, while camera-specific benchmarks provide richer motion vocabularies but still assign a single label set to an entire clip. Consequently, the representative benchmarks in Table 1 do not capture how the active camera motion changes within a real shot. C AM C HOREO addresses this gap by combining real web video with camera-specific, direction-aware multi-label annotations over variable-length temporal segments. It therefore evaluates both compound camera motion within each segment and its evolution over time.

4

C AM D ISTILL : D ISTILLING G EOMETRY INTO C AMERA T OKENS

4.1

M OTIVATION

Recognizing camera motion requires comparing perspective, parallax, scale, and orientation across frames. MLLM vision encoders, however, are optimized for semantic alignment and encode these geometric signals only weakly. Feed-forward 3D foundation models such as VGGT (Wang et al., 2025b) and VGGT-Ω (Wang et al., 2026a) are designed to recover them. Given a set of frames, these models estimate depth, point maps, and camera pose. In particular, they associate each frame with a dedicated camera token from which its pose is decoded. The evolution of these tokens across time therefore provides a compact, camera-oriented geometric representation. We use this representation as the distillation target for the MLLM. A direct way to exploit this signal is C AM I NJECT (Figure 3a), which runs the 3D model alongside the frozen vision encoder, projects each teacher camera token into the LLM hidden space, and concatenates the projected tokens with the visual sequence. This baseline is effective (Section 5) but expensive: the 3D model must process every video at inference, and models such as VGGT-Ω apply global attention over the tokens of all frames, so latency and memory grow rapidly with video length. Since these camera tokens are clearly useful, we ask whether they can be obtained without running the 3D model at test time. C AM D ISTILL moves the teacher entirely to training (Figure 3b). A lightweight student predicts per-frame camera tokens from the frozen vision features the MLLM already computes, and a distillation loss aligns them with the teacher’s tokens. The 3D model is then discarded, so C AM D ISTILL keeps the geometric supervision with almost no inference overhead. 4.2

A RCHITECTURE

The student takes the form of a Geometry-aware Camera Token Extractor (GCTE), a lightweight branch attached to the frozen vision encoder (Figure 3b). It reads intermediate visual features without modifying the pretrained visual stream, produces one camera token per frame through alternating attention blocks, and places these tokens before the corresponding visual tokens. The LLM can then condition its predictions on both semantic visual features and an explicit camera representation. (ℓ)

Camera tokens. Let xi ∈ RPi ×dv denote the frozen vision features of frame i at encoder layer ℓ, where Pi is the number of visual tokens and dv is the vision hidden size. GCTE reads intermediate rather than final-layer features. Intermediate layers retain local geometric cues such as parallax and perspective change, while higher layers become increasingly specialized for semantic alignment (Feng et al., 2026). This choice gives the student access to a cleaner camera-related signal. 5

Preprint

(a) CamInject

GT

SFT Loss

Answer

(b) CamDistill

GT

LLM Decoder

… … …

…

Question

…

Projector

…

Projector

Vision Encoder

…

Distillation Loss

… …

… …

C

Question

…

VGGT Omega Encoder

GCTE Global Camera Self-Attention

…

…

…

Frame-wise Cross-Attention

…

…

… Projector

Projector

×𝑴

VGGT Omega Encoder

Answer

LLM Decoder

… C

SFT Loss

…

… … …

Vision Encoder

… …

Train & Inference

Train only

Loss

C Concatenate

Camera token (Teacher)

Camera token (Student)

Camera token (Learnable)

Visual token

Figure 3. C AM I NJECT versus C AM D ISTILL. (a) C AM I NJECT runs a frozen 3D foundation model (VGGT-Ω) 2026/8/6 2 Dazhaoencoder Du Beyond NTP for Training MLLMs token just before that frame’s beside the vision and inserts each frame’s teacher camera visual tokens in the LLM input, so the 3D model is required at inference. (b) C AM D ISTILL instead trains a lightweight GCTE, a stack of M alternating frame-wise cross-attention and global camera self-attention blocks, to predict the student camera tokens, which are inserted at the same positions. A distillation loss aligns them with the teacher, which is removed at inference. Snowflake and flame icons denote frozen and trainable components. (0)

For each of the T frames, GCTE maintains a camera token ci ∈ Rdc whose initial value ci is a learnable embedding. Following the reference-view convention of 3D reconstruction, the first frame uses a dedicated embedding and all other frames share a second one, marking the first frame as the reference view. The attention blocks below then refine these tokens. Alternating attention blocks. GCTE stacks M alternating blocks, each a frame-wise crossattention followed by a global camera self-attention (Figure 3b), mapping the initial tokens (0) (M ) (m) {ci }Ti=1 to the final camera states {ci }Ti=1 . We write ci for the state of frame i after block m. The two attention operations serve complementary roles. Frame-wise cross-attention extracts camera-relevant evidence from each frame, while global self-attention compares that evidence across time. Together, they represent both the geometry of individual views and its temporal evolution. Frame-wise cross-attention. Frame-wise cross-attention lets each camera token read only the visual tokens from its corresponding frame: (m) (m−1) (ℓ )
c̃i = CrossAttn ci , xi m . (2) (m−1)

(ℓ )

Here the camera token ci from the previous block is the query, the frame features xi m at (m) the layer ℓm tapped by block m are the keys and values, and c̃i is the resulting camera token. This interaction is one-way: the block updates only the camera token. The visual tokens remain unchanged, preserving the feature distribution expected by the pretrained MLLM. Global camera self-attention. Global camera self-attention then allows the per-frame camera tokens to exchange information. Each token can therefore interpret its frame relative to the surrounding viewpoints rather than in isolation: (m)
(m) (m) (m) [c1 , . . . , cT ] = SelfAttn [c̃1 , . . . , c̃T ] . (3) P This operation attends over only the T camera tokens, not the full set of i Pi visual patches. Both attention modules use standard pre-norm transformer blocks (Vaswani et al., 2017; Ba et al., 2016). Output and injection. For frame i, we concatenate the final frame-level state and the temporally (M ) (M ) contextualized state, zi = [c̃i ; ci ]. A two-layer MLP projects this representation to the LLM 6

Preprint

hidden size. The projected token is placed immediately before the visual tokens of frame i, making the camera representation available as context before the decoder processes the frame content. Appendix K provides the complete block specification. 4.3

D ISTILLATION O BJECTIVE

C AM D ISTILL is trained with two objectives (Figure 3b): the standard next-token loss LSFT for the structured task output, and a camera-token distillation loss Lcam (Hinton et al., 2015) that aligns each student token zi with the teacher’s target token gi by cosine distance: L = LSFT + λcam Lcam = LSFT +

T
λcam X 1 − cos(zi , gi ) , T i=1

(4)

where λcam weights the distillation term. Through Lcam , the student tokens are encouraged to reproduce the teacher’s pose-associated camera representation.

5

E XPERIMENTS

5.1

S ETUP

Models. We evaluate all models on C AM C HOREO using the same instruction and output format. The comparison includes four groups. Closed-source APIs comprise GPT-5.4 (Singh et al., 2025) and Gemini-3.1-Pro (Pichai et al., 2025). Open-source MLLMs include Qwen2.5-VL (Bai et al., 2025c), Qwen3-VL at 4B, 8B, and 235B (Bai et al., 2025b), Qwen3.5 (Qwen Team, 2026a), Qwen3.6 (Qwen Team, 2026b), InternVL3.5 (Wang et al., 2025c), and Cam-Motion-7B, a Qwen2.5-VL model finetuned on CameraBench (Lin et al., 2026). The geometry-only baseline estimates per-frame camera pose with VGGT-Ω and maps translational and angular velocities to labels using hand-designed rules (Appendix I). Finally, our models are C AM D ISTILL and the direct-injection reference C AM I NJECT, implemented with Qwen3-VL 4B and 8B backbones and VGGT-Ω as the default 3D teacher. For our models, we fully fine-tune the language model, freeze the vision encoder, and train GCTE jointly. The SFT baseline fully fine-tunes the same base model with neither the GCTE module nor the distillation loss. All specialized models are trained on 43,438 Tencent Video clips annotated with the same taxonomy and protocol as C AM C HOREO, with 1,000 clips held out for validation. The training set is disjoint from the benchmark, ensuring that no evaluation clip is observed during training. Appendix G provides the complete training configuration. Metrics. We evaluate predictions from two complementary perspectives (Appendix E). Framelevel evaluation samples the timeline every 0.1 s and computes precision, recall, and F1 over the 20 direction-aware labels. We report both micro averages, which weight instances equally, and macro averages, which weight classes equally. Because labels are evaluated at each timestamp, these metrics capture both boundary and recognition errors. Segment-level evaluation instead matches predicted and ground-truth intervals by temporal IoU and reports F1 at thresholds 0.3/0.5/0.7. Segment localization (SegLoc) evaluates temporal overlap without considering labels, whereas segment detection (SegDet) additionally requires an exact match of the direction-aware label set. 5.2

M AIN R ESULTS

Table 2 shows that existing models struggle on C AM C HOREO, whereas C AM D ISTILL and C AM I NJECT lead by a wide margin. Scaling Qwen3-VL from 4B to 235B increases frame-level micro F1 from 24.2 to only 33.4. The strongest closed-source model, Gemini-3.1-Pro, reaches 43.3. In contrast, C AM D ISTILL achieves 67.5 with the 4B backbone and 67.8 with the 8B backbone; C AM I NJECT obtains comparable results. Both approaches therefore exceed the strongest baseline by more than 20 micro-F1 points. The geometry-only baseline clarifies where the difficulty lies. At IoU 0.5, it obtains 63.9 SegLoc but only 2.9 SegDet. Estimated pose can indicate when camera behavior changes, but cannot identify some movements, such as Zoom and Focus Shift. More generally, all models perform substantially better on SegLoc than on SegDet. The central challenge is therefore not merely locating temporal boundaries, but recovering the complete set of motion labels. Cam-Motion-7B also transfers poorly: 7

Preprint

Table 2. Results on C AM C HOREO. Frame-level micro/macro precision, recall, and F1 are evaluated every 0.1 seconds. SegLoc measures temporal overlap; SegDet additionally requires an exact direction-aware label set. Both are reported at IoU 0.3/0.5/0.7. Per column, best is in bold and second-best is underlined. Frame-Level (%) Micro Macro

Segment-Level F1 (%) SegLoc @IoU SegDet @IoU

Model

P

R

F1

P

R

F1

0.3

0.5

0.7

0.3

0.5

0.7

VGGT-Ω

21.7

30.2

25.3

23.3

26.2

17.2

73.6

63.9

48.5

3.0

2.9

2.7

Gemini-3.1-Pro GPT-5.4

51.4 43.8

37.4 33.5

43.3 38.0

40.5 31.2

23.0 19.2

27.4 21.3

82.8 82.8

76.1 76.1

61.2 62.1

25.1 23.9

23.7 22.4

19.8 18.9

InternVL3.5-8B Qwen2.5-VL-7B Cam-Motion-7B Qwen3-VL-4B Qwen3-VL-8B Qwen3-VL-235B Qwen3.5-4B Qwen3.5-9B Qwen3.6-35B

31.9 27.9 12.6 32.4 36.6 39.7 46.0 50.5 46.4

18.7 17.8 11.2 19.3 23.0 28.8 14.7 13.7 29.3

23.6 21.8 11.9 24.2 28.3 33.4 22.3 21.5 35.9

16.9 8.4 3.4 14.9 17.3 22.0 20.6 21.9 22.3

6.6 4.1 2.4 7.1 7.9 13.9 5.1 4.5 12.2

7.1 4.0 2.6 7.8 9.1 15.4 6.9 6.5 14.3

38.4 62.6 53.2 69.7 76.2 79.1 52.1 43.8 80.1

21.0 39.7 38.0 55.6 67.7 66.1 46.9 38.3 68.8

8.5 18.2 20.3 38.8 52.5 47.8 38.2 29.7 51.2

7.2 9.9 0.0 14.4 16.5 17.9 16.8 14.9 19.8

4.0 6.4 0.0 11.6 15.0 15.6 15.4 13.3 17.9

1.7 3.0 0.0 8.6 12.5 12.0 12.9 10.7 14.2

C AM D ISTILL-4B C AM I NJECT-4B C AM D ISTILL-8B C AM I NJECT-8B

73.5 73.0 73.8 74.0

62.4 62.8 62.6 63.5

67.5 67.5 67.8 68.3

63.8 63.5 63.6 64.9

54.5 54.7 54.9 56.3

57.7 57.6 57.9 59.2

86.4 86.7 86.5 86.5

80.4 80.8 80.4 80.8

66.8 66.8 66.7 67.0

39.7 40.1 40.5 40.3

38.2 38.6 38.8 39.0

33.5 33.9 34.1 34.2

Table 3. Camera information on Qwen3-VL-4B. We add pose-as-text prompting (PromptInject), supervised fine-tuning (SFT), and our C AM D ISTILL and C AM I NJECT to the base model. PromptInject and C AM I NJECT run VGGT-Ω at inference, and latency and peak memory are measured on a single H100. The 8B backbone shows the same trends (Table 8). Method Qwen3-VL-4B +PromptInject +SFT +CamDistill +CamInject

Micro F1

Macro F1

SegLoc@0.5

SegDet@0.5

Latency (s/clip) ↓

Peak Mem (GB) ↓

24.2

7.8

55.6

11.6

10.1

18.3

23.1 62.2 67.5 67.5

10.7 51.4 57.7 57.6

67.3 79.0 80.4 80.8

12.1 33.6 38.2 38.6

16.8 10.1 10.2 16.0

24.0 18.3 20.1 23.1

CameraBench clip-level tuning overfits its base MLLM and erodes instruction-following, yielding valid outputs for only 19 of 4,229 clips. Its scores, computed over these 19 alone, are not comparable to other rows. This reflects both the format gap and the cost of narrow task-specific tuning. The comparison between C AM D ISTILL and C AM I NJECT isolates the effect of replacing direct teacher features with distilled ones. C AM I NJECT retains the 3D model at inference, whereas C AM D ISTILL predicts camera tokens from the frozen MLLM features and removes the teacher. Nevertheless, their results are nearly identical. With the 4B backbone, both reach 67.5 micro F1 and differ by only 0.4 SegDet at IoU 0.5. With the 8B backbone, C AM D ISTILL trails C AM I NJECT by only 0.5 micro F1. Thus, C AM D ISTILL preserves almost all of the benefit of direct feature injection without requiring the 3D model at inference. 5.3

A NALYSIS AND A BLATIONS

Effect of camera information. As shown in Table 3, feeding the teacher’s per-frame pose as textual prompt (PromptInject) helps localization (SegLoc@0.5 55.6 → 67.3) but not recognition (micro F1 24.2 → 23.1), and still runs VGGT-Ω at inference (latency 10.1 → 16.8 s). SFT is far more effective at 62.2 micro F1, but task supervision alone leaves the frozen encoder unable to separate motions that differ only in parallax or perspective. Distilling the teacher’s camera representation into GCTE closes much of this gap, adding 5.3 micro-F1, 6.3 macro-F1, and 4.6 SegDet@0.5 over SFT. The larger macro gain shows the geometric signal especially helps rare, geometry-dependent classes, and Figure 4 attributes it to the supervision itself, since performance peaks at a nonzero distillation weight. Crucially, C AM D ISTILL reaches this accuracy without the teacher at inference. It matches C AM I NJECT within 0.4 SegDet but adds only 0.1 s and 1.8 GB over the base model, against 8

Preprint

Model

CameraBench

CMVQA

Qwen2.5-VL-7B Qwen3-VL-8B Cam-Motion-7B

31.0 39.6 49.7

24.8 23.5 29.7

C AM D ISTILL-8B C AM I NJECT-8B

60.3 62.9

40.1 42.4

68 Micro F1 (%)

Table 4. External generalization. mAP on CameraBench and accuracy on CMVQA, two external camera-motion benchmarks.

67 66 65 0

0.01 0.02 0.05 0.1 Distillation weight cam

0.2

Figure 4. Distillation-weight sensitivity. C AM D ISTILL-4B peaks at λcam = 0.05.

Table 5. Ablations of C AM D ISTILL-4B. Each block varies one design factor, namely feature-layer region, module depth, token position, and teacher, while the others stay at the default. Segment metrics use IoU 0.5. Component

Setting

Micro F1

Macro F1

SegLoc

SegDet

Feature-layer region

Early (0, 3, 6, 9) Late (14, 17, 20, 23) Uniform (4, 9, 13, 18) Early–middle (1, 5, 9, 13)

66.1 65.4 66.8 67.5

56.0 54.9 56.5 57.7

80.0 79.6 79.9 80.4

37.3 36.8 37.6 38.2

Module depth

3 layers (1, 7, 13) 4 layers (1, 5, 9, 13) 5 layers (1, 4, 7, 10, 13)

66.6 67.5 67.2

56.2 57.7 57.5

79.8 80.4 80.3

37.4 38.2 38.2

Token position

After visual tokens Before visual tokens

64.7 67.5

54.5 57.7

79.5 80.4

36.9 38.2

Teacher

VGGT VGGT-Ω

65.8 67.5

55.7 57.7

80.2 80.4

37.6 38.2

C AM I NJECT’s 5.9 s and 4.8 GB. C AM D ISTILL thus attains injection-level accuracy at essentially the base model’s inference cost, and the 8B backbone shows the same pattern (Table 8). Generalization to external benchmarks. We next test whether C AM D ISTILL learns transferable camera-motion cues rather than merely adapting to the output format of C AM C HOREO. CameraBench (Lin et al., 2026) evaluates clip-level recognition on real web videos using mAP, while CMVQA (Feng et al., 2026) evaluates multiple-choice reasoning on synthetic clips using accuracy. We follow each benchmark’s official evaluation script and task protocol, so improvements provide evidence of cross-task transfer. As shown in Table 4, C AM D ISTILL-8B improves over Qwen3VL-8B by 20.7 mAP on CameraBench and 16.6 accuracy points on CMVQA. It also outperforms CameraBench-tuned Cam-Motion-7B on both benchmarks. The remaining gap to C AM I NJECT is small, consistent with the modest information loss introduced by distillation. Sensitivity to the distillation weight. Figure 4 shows that performance peaks at λcam = 0.05. With a smaller weight, the geometric supervision is too weak to add much beyond task training. With a larger weight, feature imitation competes with the language-modeling objective. The distillation loss is therefore most effective as a moderate auxiliary signal rather than the dominant target. Design ablations. Table 5 varies one GCTE design choice at a time on the 4B backbone. Feature layers: early-to-middle features perform best, suggesting that they retain geometric detail while providing sufficient contextual abstraction. Earlier features contain less context, whereas later features are increasingly semantic. Module depth: four alternating blocks achieve the best trade-off. Three blocks are insufficient to reproduce the teacher representation, while a fifth provides no meaningful gain. Token position: placing camera tokens before the visual tokens improves micro F1 by 2.8 points, indicating that they are most useful as conditioning context rather than appended summaries. Teacher: replacing VGGT with VGGT-Ω improves micro F1 by 1.7 points, even though the teacher is absent at inference. Together with Figure 4, these results attribute the gains to the camera-specific design and supervision rather than to an arbitrary increase in model capacity. 9

Preprint

6

C ONCLUSION

We studied camera motion as a temporally grounded, compositional problem, in which a model must localize when each movement occurs and recover the movements that co-occur within a shot. Building the C AM C HOREO benchmark for this task showed that such temporal and compositional structure is the rule rather than the exception in real video, and that current MLLMs struggle on it because their visual encoders lack the geometric grounding it requires. To close this gap, we introduced C AM D ISTILL, which distills the geometry of a 3D foundation model into lightweight camera tokens during training and discards the teacher at inference. It matches the accuracy of direct geometric injection while adding almost no inference cost. We hope C AM C HOREO and C AM D ISTILL encourage modeling camera motion as a time-varying, compositional signal.

R EPRODUCIBILITY S TATEMENT We provide the information needed to reproduce the benchmark and experiments. Appendix C describes dataset construction and filtering, Appendix D specifies the taxonomy and annotation protocol, and Appendix E defines the evaluation metrics. Appendix G lists the optimization settings, tapped layers, and hardware for both backbones. Appendix K gives the complete GCTE equations, and Appendix F provides the training and inference prompt. We will release the C AM C HOREO annotations, evaluation code, prompts, trained checkpoints, and permitted video identifiers or download scripts.

E THICS S TATEMENT C AM C HOREO consists of publicly available single-shot YouTube clips labeled by trained expert annotators and is intended solely for research on camera-motion understanding. The separate training videos are collected from the Tencent Video platform. Broader implications and limitations are discussed in Appendices A and B.

AI U SE S TATEMENT We used generative AI tools only as general-purpose assistants for editing prose and for minor coding support (e.g., plotting and data-processing scripts). Generative AI was not used to generate research ideas, experimental results, or data annotations. All AI-assisted text and code were reviewed and verified by the authors, who take full responsibility for the final content of this work.

10

Preprint

R EFERENCES Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. Layer normalization. arXiv preprint arXiv:1607.06450, 2016. Jianhong Bai, Menghan Xia, Xiao Fu, Xintao Wang, Lianrui Mu, Jinwen Cao, Zuozhu Liu, Haoji Hu, Xiang Bai, Pengfei Wan, et al. Recammaster: Camera-controlled generative rendering from a single video. In 2025 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 14834–14844. IEEE, 2025a. Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, et al. Qwen3-vl technical report. arXiv preprint arXiv:2511.21631, 2025b. Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2.5-vl technical report. arXiv preprint arXiv:2502.13923, 2025c. Andrew J Davison, Ian D Reid, Nicholas D Molton, and Olivier Stasse. Monoslam: Real-time single camera slam. IEEE transactions on pattern analysis and machine intelligence, 29(6):1052–1067, 2007. Siyan Dong, Shuzhe Wang, Shaohui Liu, Lulu Cai, Qingnan Fan, Juho Kannala, and Yanchao Yang. Reloc3r: Large-scale training of relative camera pose regression for generalizable, fast, and accurate visual localization. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 16739–16752. IEEE, 2025. Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020. Jakob Engel, Thomas Schöps, and Daniel Cremers. Lsd-slam: Large-scale direct monocular slam. In European conference on computer vision, pp. 834–849. Springer, 2014. Haoan Feng, Sri Harsha Musunuri, and Guan-Ming Su. Geometry-guided camera motion understanding in videollms. arXiv preprint arXiv:2603.13119, 2026. Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531, 2015. Jiahui Huang, Qunjie Zhou, Hesam Rabeti, Aleksandr Korovko, Huan Ling, Xuanchi Ren, Tianchang Shen, Jun Gao, Dmitry Slepichev, Chen-Hsuan Lin, et al. Vipe: Video pose engine for 3d geometric perception. arXiv preprint arXiv:2508.10934, 2025. Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, et al. Llava-onevision: Easy visual task transfer. arXiv preprint arXiv:2408.03326, 2024a. Moyang Li, Zihan Zhu, Marc Pollefeys, and Daniel Barath. Droid-slam in the wild. arXiv preprint arXiv:2603.19076, 2026. Xiaozhe Li, Kai Wu, Siyi Yang, YiZhan Qu, Guohua Zhang, Zhiyu Chen, Jiayao Li, Jiangchuan Mu, Xiaobin Hu, Wen Fang, et al. Can video generation replace cinematographers? research on the cinematic language of generated video. arXiv preprint arXiv:2412.12223, 2024b. Zhiqiu Lin, Siyuan Cen, Daniel Jiang, Jay Karhade, Hewei Wang, Chancharik Mitra, Yu Tong Tiffany Ling, Yuhan Huang, Rushikesh Zawar, Xue Bai, et al. Towards understanding camera motions in any video. Advances in Neural Information Processing Systems, 38, 2026. Hongbo Liu, Jingwen He, Yi Jin, Dian Zheng, Yuhao Dong, Fan Zhang, Ziqi Huang, Yinan He, Weichao Chen, Yu Qiao, et al. Shotbench: Expert-level cinematic understanding in vision-language models. Advances in Neural Information Processing Systems, 38:129987–130019, 2026. 11

Preprint

Jakob Isak Nielsen, Edvin Kau, and Richard Raskin. Camera movement in narrative cinema: towards a taxonomy of functions. Department of Inf. & Media Studies, University of Aarhus, 2007. Sundar Pichai, Demis Hassabis, and Koray Kavukcuoglu. A new era of intelligence with gemini 3, 2025. URL https://blog.google/ intl/en-africa/company-news/outreach-and-initiatives/ a-new-era-of-intelligence-with-gemini-3/. Qwen Team. Qwen3.5: Towards native multimodal agents, February 2026a. URL https:// qwen.ai/blog?id=qwen3.5. Qwen Team. Qwen3.6-35B-A3B: Agentic coding power, now open to all, April 2026b. URL https://qwen.ai/blog?id=qwen3.6-35b-a3b. Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp. 8748–8763. PmLR, 2021. Johannes L Schonberger and Jan-Michael Frahm. Structure-from-motion revisited. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4104–4113, 2016. Aaditya Singh, Adam Fry, Adam Perelman, Adam Tart, Adi Ganesh, Ahmed El-Kishky, Aidan McLaughlin, Aiden Low, AJ Ostrow, Akhila Ananthram, et al. Openai gpt-5 system card. arXiv preprint arXiv:2601.03267, 2025. Tomás Soucek and Jakub Lokoc. Transnet v2: An effective deep network architecture for fast shot transition detection. In Proceedings of the 32nd ACM international conference on multimedia, pp. 11218–11221, 2024. Raymond Spottiswoode. A grammar of the film: An analysis of film technique. Univ of California Press, 1959. Yunlong Tang, Junjia Guo, Hang Hua, Susan Liang, Mingqian Feng, Xinyang Li, Rui Mao, Chao Huang, Jing Bi, Zeliang Zhang, et al. Vidcomposition: Can mllms analyze compositions in compiled videos? In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 8490–8500. IEEE, 2025. Zachary Teed and Jia Deng. Droid-slam: Deep visual slam for monocular, stereo, and rgb-d cameras. Advances in neural information processing systems, 34:16558–16569, 2021. Michael Tschannen, Alexey Gritsenko, Xiao Wang, Muhammad Ferjad Naeem, Ibrahim Alabdulmohsin, Nikhil Parthasarathy, Talfan Evans, Lucas Beyer, Ye Xia, Basil Mustafa, et al. Siglip 2: Multilingual vision-language encoders with improved semantic understanding, localization, and dense features. arXiv preprint arXiv:2502.14786, 2025. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017. Jiahao Wang, Yufeng Yuan, Rujie Zheng, Youtian Lin, Jian Gao, Lin-Zhuo Chen, Yajie Bao, Yi Zhang, Chang Zeng, Yanxi Zhou, et al. Spatialvid: A large-scale video dataset with spatial annotations. arXiv preprint arXiv:2509.09676, 2025a. Jianyuan Wang, Minghao Chen, Nikita Karaev, Andrea Vedaldi, Christian Rupprecht, and David Novotny. Vggt: Visual geometry grounded transformer. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5294–5306. IEEE, 2025b. Jianyuan Wang, Minghao Chen, Shangzhan Zhang, Nikita Karaev, Johannes Schönberger, Patrick Labatut, Piotr Bojanowski, David Novotny, Andrea Vedaldi, and Christian Rupprecht. VGGT-ω. arXiv preprint arXiv:2605.15195, 2026a. 12

Preprint

Shuzhe Wang, Vincent Leroy, Yohann Cabon, Boris Chidlovskii, and Jerome Revaud. Dust3r: Geometric 3d vision made easy. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 20697–20709. IEEE, 2024. Weiyun Wang, Zhangwei Gao, Lixin Gu, Hengjun Pu, Long Cui, Xingguang Wei, Zhaoyang Liu, Linglin Jing, Shenglong Ye, Jie Shao, et al. Internvl3.5: Advancing open-source multimodal models in versatility, reasoning, and efficiency. arXiv preprint arXiv:2508.18265, 2025c. Xinran Wang, Songyu Xu, Shan Xiangxuan, Yuxuan Zhang, Muxi Diao, Xueyan Duan, Kongming Liang, Zhanyu Ma, et al. Cinetechbench: A benchmark for cinematographic technique understanding and generation. Advances in Neural Information Processing Systems, 38, 2026b. Yifan Wang, Jianjun Zhou, Haoyi Zhu, Wenzheng Chang, Yang Zhou, Zizun Li, Junyi Chen, Jiangmiao Pang, Chunhua Shen, and Tong He. π 3 : Permutation-equivariant visual geometry learning. In International Conference on Learning Representations, volume 2026, pp. 10481–10497, 2026c. Hang Wu, Yujun Cai, Haonan Ge, Hongkai Chen, Ming-Hsuan Yang, and Yiwei Wang. Refineshot: Rethinking cinematography understanding with foundational skill evaluation. arXiv preprint arXiv:2510.02423, 2025a. Hang Wu, Yujun Cai, Zehao Li, Haonan Ge, Bowen Sun, Junsong Yuan, and Yiwei Wang. Camreasoner: Reinforcing camera movement understanding via structured spatial reasoning. arXiv preprint arXiv:2602.00181, 2026. Jianlong Wu, Wei Liu, Ye Liu, Meng Liu, Liqiang Nie, Zhouchen Lin, and Chang Wen Chen. A survey on video temporal grounding with multimodal large language model. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025b. Jinbo Xing, Long Mai, Cusuh Ham, Jiahui Huang, Aniruddha Mahapatra, Chi-Wing Fu, Tien-Tsin Wong, and Feng Liu. Motioncanvas: Cinematic shot design with controllable image-to-video generation. In Proceedings of the Special Interest Group on Computer Graphics and Interactive Techniques Conference Conference Papers, pp. 1–11, 2025. Jihan Yang, Zifan Zhao, Xichen Pan, Shusheng Yang, Junyi Zhang, Bingyi Kang, Hu Xu, and Saining Xie. Cambrian-p: Pose-grounded video understanding. arXiv preprint arXiv:2605.22819, 2026. Mehmet Burak Yilmaz, Elen Lotman, Andres Karjus, and Pia Tikka. An embodiment of the cinematographer: emotional and perceptual responses to different camera movement techniques. Frontiers in Neuroscience, 17:1160843, 2023. Gongjie Zhang, Wenhao Li, Quanhao Qian, Jiuniu Wang, Deli Zhao, Shijian Lu, and Ran Xu. On the generalization capacities of mllms for spatial intelligence. arXiv preprint arXiv:2603.06704, 2026. Yuanhan Zhang, Jinming Wu, Wei Li, Bo Li, Zejun Ma, Ziwei Liu, and Chunyuan Li. Llava-video: Video instruction tuning with synthetic data. arXiv preprint arXiv:2410.02713, 2024. Zhoutong Zhang, Forrester Cole, Zhengqi Li, Michael Rubinstein, Noah Snavely, and William T Freeman. Structure and motion from casual videos. In European Conference on Computer Vision, pp. 20–37. Springer, 2022. Duo Zheng, Yanyang Li, Liwei Wang, et al. Learning from videos for 3d world: Enhancing mllms with 3d vision geometry priors. Advances in neural information processing systems, 38:20560– 20586, 2026.

13

Preprint

A

B ROADER I MPLICATIONS AND R EPRODUCIBILITY

Broader implications. Camera-motion understanding describes the behavior of the observer, not only the content of the observed scene. Because camera state affects depth cues, visibility, and the interpretation of object motion, camera-aware representations may benefit spatial reasoning, action understanding, video retrieval, and controllable video generation. These applications also require caution. Cinematic labels reflect production conventions and can remain ambiguous across domains, so downstream systems should preserve uncertainty rather than treat every prediction as an objective description of authorial intent. Reproducibility. The license-restricted training set is disjoint from the independently collected benchmark. The appendices specify the taxonomy, annotation instructions, prompts, frame- and segment-level metrics, validation-based model-selection protocol, teacher alignment, and inferencecost protocol. We will release the benchmark annotations, permitted video identifiers or download scripts, evaluation code, and checkpoints where licensing allows. Because the benchmark and evaluator are public, future methods can remain directly comparable even when trained on independently sourced data.

B

L IMITATIONS

C AM C HOREO is restricted to single-shot clips and therefore does not cover multi-shot videos or interactions among camera motion, editing, and long-form narrative structure. Extending the task and benchmark to edited, multi-shot video is left to future work.

C

B ENCHMARK C ONSTRUCTION P IPELINE

Figure 5 summarizes the complete curation pipeline, from source discovery and automatic filtering to class balancing and human quality control.

Figure 5. C AM C HOREO construction pipeline. Phase I: automatic curation. We collect YouTube videos from nine domains, split them into shots, filter for duration, diversity, and quality, and use pseudo labels only to balance the candidate pool. This process reduces 36,313 source videos to 13,358 candidate clips. Phase II: human annotation. Expert annotators replace all pseudo labels with temporal camera-motion annotations. Independent review and final filtering produce 4,229 released clips.

Source collection. We query YouTube using a curated keyword set that covers all nine content domains, with roughly a dozen queries per domain. Examples include cinematic footage and film 14

Preprint

clip 4k for Film & TV; wildlife documentary and aerial nature footage for Documentary & Nature; FPV racing drone and drone orbit building for Aerial & Drone; walking vlog and handheld walking footage for Vlog & Selfie; and product commercial and car advertisement for Commercial & Ad. We use analogous queries for Sports & Action, Gaming & Animation, Tutorial & Education, and Synthetic & Rendered. Filtering funnel. Figure 5 contains eight stages. (1) We retain source videos between 2 s and 30 min. (2) TransNetV2 (Soucek & Lokoc, 2024) detects shot boundaries and divides each video into single-shot clips. (3) We keep clips between 1 and 60 s. (4) At most six clips are retained from each source video to improve content diversity. (5) We re-encode all clips with a uniform ffmpeg profile and score them on clarity, aesthetics, motion intensity, motion quality, and content quality. Clips below threshold on any dimension are removed, while a controlled fraction of near-static clips is restored to preserve the Static prior. (6) An SFT camera-motion model trained on the separate 43,438-video training set assigns pseudo labels. These labels are used only to construct a balanced candidate pool that up-weights rare movements such as Zoom, Roll, Arc, and Focus Shift. (7) Five annotators with film- and media-related backgrounds independently replace the pseudo labels with precise temporal annotations. (8) Cross-review resolves disagreements by consensus; clips without consensus, as well as doubtful or empty clips, are removed. The resulting benchmark contains 4,229 clips.

D

TAXONOMY AND A NNOTATION RULES

Table 6. Camera motion taxonomy used in C AM C HOREO. cw/ccw abbreviate clockwise/counterclockwise.

Type

Direction

Annotation cue

Static

None

Unstable

None

Pan

left/right

Tilt

up/down

Truck

left/right

Pedestal

up/down

Dolly In/Out

None

Zoom In/Out

None

Roll Arc Follow

cw/ccw cw/ccw None

Focus Shift

None

Camera position and orientation remain essentially unchanged; imperceptible micro-jitter is allowed. Irregular visible shaking without a stable direction; if a stable direction exists, annotate that motion instead. Horizontal rotation around a fixed camera position; foreground and background move similarly with little parallax. Vertical rotation around a fixed camera position; frame shifts vertically without translation parallax. Lateral camera translation; foreground/background parallax is visible. Vertical camera translation; perspective and horizon height change. Forward/backward camera translation with depth parallax; distinct from zoom. Focal-length change with approximately uniform image scaling and no depth parallax. Rotation around the optical axis; horizon tilts. Camera orbits around a subject or scene center. Camera tracks a moving subject; requires reasoning about subject-centered motion. Focus plane changes while camera motion may be absent.

Annotation protocol. The taxonomy contains 12 movement types and 20 direction-aware labels. Directions follow the physical camera motion rather than the apparent background flow: Pan Left/Right denotes rotation of the viewing direction, Truck Left/Right translation of the camera center in its local frame, and Tilt/Pedestal Up/Down the corresponding physical rotation/translation. Each movement also receives a coarse speed attribute (zero, slow, medium, or fast), although speed is not evaluated in this work. Annotators apply seven main rules. (i) Each segment contains all basic movements with perceptible magnitude and clear intent, while minor compensatory motion is ignored. (ii) Direction is constrained by movement type: Pan and Truck use left/right, Tilt and Pedestal use up/down, Arc and Roll use clockwise/counterclockwise, and all remaining types use null. A movement with two directional components is represented by two elements, such as Dolly In + Tilt up. (iii) Static permits only imperceptible micro-jitter, while clearly visible directionless shake is labeled Unstable. (iv) Dolly is distinguished from Zoom through depth parallax 15

Preprint

and perspective change, rather than uniform image scaling. (v) Focus Shift, including rack and follow focus, is treated as a basic movement. (vi) Follow and Arc are annotated together with their underlying primitive, such as Dolly In, Truck, or Pan. Arc requires the camera to move along a clear curved trajectory around one or more identifiable subjects through at least 45◦ ; weak curvature below this threshold or motion without a locked subject is treated as a minor adjustment and is not labeled Arc. Arc clockwise/counterclockwise is defined by the camera trajectory around the subject as viewed from above. (vii) Subject motion is not labeled as camera motion. For example, a walking person does not imply camera movement unless the background perspective or frame boundaries also change. Segment boundaries are placed at 0.1-second resolution whenever the active motion phase changes.

E

E VALUATION P ROTOCOL D ETAILS

We formalize the frame- and segment-level metrics summarized in the main text. Setup. Each video is represented as a sequence of non-overlapping temporal segments that fully covers [0, T ], and every segment carries a set of camera-movement labels. Directional movements are encoded jointly with their direction, for example Pan left, while non-directional movements use only the type name. This produces the 20-label space C used throughout the paper. Metrics are computed on videos present in both the predictions and the ground truth (GT). E.1

F RAME -L EVEL M ETRICS

Sampling. We sample each timeline at intervals of ∆ = 0.1 s. At timestamp t, let Yt ⊆ C and Ŷt ⊆ C denote the GT and predicted label sets of the segments covering t, where a segment is active when s ≤ t < e. Because the GT segments cover [0, T ], evaluation samples the full GT timeline; a missing predicted segment yields an empty predicted label set and therefore false negatives. Because labels are evaluated densely over time, a boundary error affects multiple timestamps and is reflected in the recognition score. Micro precision/recall/F1. We accumulate multi-label counts over all sampled frames, X X X TP = |Yt ∩ Ŷt |, FP = |Ŷt \ Yt |, FN = |Yt \ Ŷt |, t

t

(5)

t

and define P = TP/(TP + FP), R = TP/(TP + FN), and F1 = 2P R/(P + R). Micro averaging is instance-weighted and is therefore dominated by frequent classes. Macro precision/recall/F1. We instead compute per-class counts TPc , FPc , FNc for each c ∈ C (a frame contributes to class c as TP if c ∈ Yt ∩ Ŷt , FP if c ∈ Ŷt \ Yt , FN if c ∈ Yt \ Ŷt ), form the perclass Pc , Rc , F1c , and average them equally over C. Macro averaging is class-weighted and surfaces rare classes such as Roll and Arc. A direction-agnostic variant that collapses each composite label to its type isolates direction errors. E.2

S EGMENT-L EVEL M ETRICS

Temporal IoU and matching. This axis operates on whole segments and decouples localization from recognition. For a GT segment g and predicted segment p, the temporal IoU is
max 0, min(g end , pend ) − max(g start , pstart ) IoU(g, p) = . (6) |g| + |p| − |g ∩ p| Given a threshold τ , we form the pairwise temporal-IoU matrix and use Hungarian matching to obtain a one-to-one assignment between ground-truth and predicted segments. Assigned pairs with IoU ≥ τ are retained as matches. Let NG and NP be the number of GT and predicted segments. Localization (SegLoc). A predicted segment is a true positive iff it is matched with IoU ≥ τ , regardless of labels. With Mloc matched pairs, Ploc = Mloc /NP , Rloc = Mloc /NG , and SegLoc-F1 = 2Ploc Rloc /(Ploc + Rloc ). This measures pure temporal segmentation quality. 16

Preprint

Detection (SegDet). Segment detection additionally requires the predicted and GT label sets to match exactly, Lp = Lg . Let Mdet denote the number of matched pairs satisfying this condition. We define Pdet = Mdet /NP , Rdet = Mdet /NG , and compute SegDet-F1 analogously. Because Mdet ≤ Mloc , SegDet-F1 cannot exceed SegLoc-F1 at the same threshold. SegDet assigns no partial credit within a segment: a direction error or a missing co-occurring movement invalidates the match. It is therefore substantially stricter than frame-level F1.

Thresholds. All segment-level metrics are reported at τ ∈ {0.3, 0.5, 0.7} without averaging, exposing the localization-tightness trade-off. Matching is deterministic, using Hungarian assignment on the temporal-IoU matrix.

F

T RAINING AND I NFERENCE P ROMPT

We use an identical prompt at training and inference. The model receives a video, the system prompt below, and a short user instruction, and is required to return only JSON. The evaluator tolerates minor formatting variations but enforces the closed taxonomy, valid directions, chronological ordering, and non-overlapping temporal segments.

System Prompt You are a senior film cinematographer. After watching the video, determine which camera movements make up this video, locate their time spans, and output structured JSON. Core principle: judge only the motion of the camera (lens) itself, not the motion of objects within the frame. People walking or cars driving inside the frame do not mean that the camera is moving. Observe whether the background and frame edges move. Basic movement (required, array). Each element is {"type", "direction", "speed"}, with type drawn from the closed set below. Static / non-steady. Static: camera position and orientation essentially unchanged (barely visible microjitter allowed); speed=zero if completely still, else slow. Unstable: clearly perceptible irregular shaking with no stable direction; if a sustained direction exists (e.g. the background sweeps left), label that movement instead (e.g. Pan). Rotation vs. translation. Pan (fixed camera, horizontal rotation; no depth parallax) vs. Truck (lateral camera translation; obvious parallax). Tilt (fixed camera, vertical rotation) vs. Pedestal (vertical translation; horizon height and pitch change). Depth. Dolly In/Dolly Out (camera moves forward/backward; near and far regions scale at different rates because of parallax) vs. Zoom In/Zoom Out (focal-length change, camera fixed; uniform scaling, no parallax). Other. Roll (rotation about the optical axis; tilting horizon), Arc (camera orbits a centered subject), Follow (camera tracks a moving subject; background changes continuously), Focus Shift (focus moves across depth layers). Arc and Follow must also annotate the underlying basic movements (e.g. Truck, Pan, Dolly In). Direction rules. Truck→left/right, Pedestal→up/down, Pan→left/right, Tilt→up/down, Arc/Roll→clockwise/counterclockwise; all other types use null. Speed rules. zero (completely still) / slow (confirmable only on careful inspection) / medium (clearly perceived) / fast (rapid, with a sense of speed). Compound movement. When several motions occur simultaneously and are all observable, output all items (e.g. Dolly In while Pan). Output the segments array ordered by time, following the format below, and output only JSON.

Output Format: Example JSON

17

Preprint

{ "segments": [ { "start_time": 0.0, "end_time": 2.5, "basic_movement": [ {"type": "Truck", "direction": "right", "speed": "medium"}, {"type": "Dolly In", "direction": null, "speed": "slow"} ], "confidence": "high" }, { "start_time": 2.5, "end_time": 5.0, "basic_movement": [ {"type": "Static", "direction": null, "speed": "zero"} ], "confidence": "high" } ] }

User Prompt Analyze the camera movement in this video and output JSON following the system prompt rules. Output only JSON.

G

A DDITIONAL I MPLEMENTATION D ETAILS

The camera module uses QK-normalized scaled dot-product attention, pre-normalization, residual connections, and small LayerScale coefficients. The first-frame and shared subsequent-frame queries are initialized from a zero-mean Gaussian with standard deviation 10−3 . For each video, the teacher cache stores an S × 2048 tensor, where S is the number of teacher frames. Before computing the loss, these features are aligned to the MLLM temporal grid. C AM I NJECT loads the same teacher features at inference and maps them to the LLM hidden size with a two-layer projector of approximately 8M parameters. C AM D ISTILL accesses the cache only during training. Training data. SFT, C AM D ISTILL, and C AM I NJECT are trained on 43,438 clips from the Tencent Video platform, with 1,000 clips held out for validation. The training annotations use the same 12 movement types, 20 direction-aware labels, 0.1-second temporal resolution, and annotation protocol as C AM C HOREO (Appendix D). The data source is the main difference: the training clips come from Tencent Video, whereas the benchmark clips come from YouTube. The two sets are disjoint. We additionally screen them with CLIP embeddings and manually inspect high-similarity pairs; no duplicate clip is retained. Evaluation inputs. Qwen-family open-source models, Cam-Motion-7B, SFT, C AM D ISTILL, and C AM I NJECT use 5 FPS with at most 100 frames. They share the same English prompt, deterministic decoding, parser, and evaluator. Gemini receives the video through its video interface with a requested 5-FPS rate. GPT-5.4 does not accept video through the evaluated API and therefore receives uniformly sampled timestamped frames (5 FPS). InternVL3.5 uses 32 uniformly sampled frames because its inference interface accepts a fixed frame count. External CameraBench and CMVQA results use the benchmarks’ official scripts and protocols without our temporal JSON prompt. Training configuration. Table 7 gives the complete setup for both backbones. We fully fine-tune the language model while freezing the vision encoder and visual projector. We train GCTE jointly with the language model. For C AM I NJECT, the teacher projector is also trainable. GCTE adds approximately 110.2M trainable parameters to the 4B backbone and 216.3M to the 8B backbone. Block i attends to the i-th selected ViT layer, so module depth equals the number of tapped layers. 18

Preprint

When the number of teacher frames differs from the number of MLLM temporal groups, we align the teacher sequence using two-frame average pooling when possible, followed by adaptive pooling or nearest-neighbor interpolation when necessary. Because the 3D model provides only supervision for C AM D ISTILL, we extract its camera tokens once and cache them offline. The teacher is not invoked in the optimization loop, so distillation adds little runtime overhead beyond standard SFT. For C AM I NJECT, the teacher can alternatively be run online for each batch. Table 7. Training hyperparameters. Shared by C AM D ISTILL and C AM I NJECT unless a per-backbone value is given. Setting

Value

Model and optimization Backbone Trainable parameters Frozen parameters Precision / attention Memory GPUs Per-device batch size Gradient accumulation Epochs Learning rate (4B / 8B) LR schedule Weight decay Max sequence length Training time (4B / 8B)

Qwen3-VL-4B / 8B-Instruct full LLM; GCTE module vision encoder, visual projector bfloat16 / FlashAttention-2 gradient checkpointing, DeepSpeed ZeRO-2 8× H20 2 4 (effective batch size 64) 2 2×10−5 / 1.5×10−5 cosine, warm-up ratio 0.05 0.01 16,384 ∼7 / ∼10 hours

Video sampling Frame rate / count Max pixels per frame

5 FPS, 4–100 frames 100,352

Camera-token module and distillation GCTE depth M (4B / 8B) Tapped ViT layers (4B) Tapped ViT layers (8B) Camera-token dim dc Distillation weight λcam Teacher

4/8 {1, 5, 9, 13} over 24 layers {1, 3, 5, 7, 9, 11, 13, 15} 1024 (concatenated feature 2048) 0.05 3D foundation model (VGGT / VGGT-Ω), cached

Training curves. Figure 6 reports the SFT and distillation losses for both backbones. Both objectives decrease smoothly, and the 4B and 8B curves nearly overlap. The training-set cosine-distance loss falls from approximately 1 to 0.04, showing that GCTE fits the cached teacher targets during optimization.

(a) SFT Loss Distillation loss

SFT loss

0.025 0.020 0.015 0.010 0.005 0

20

40

60

Training progress (%)

80

(b) Distillation Loss

1.0

CamDistill-4B CamDistill-8B

100

CamDistill-4B CamDistill-8B

0.8 0.6 0.4 0.2 0.0

0

20

40

60

Training progress (%)

80

100

Figure 6. Training loss curves. (a) Next-token SFT loss and (b) camera-token distillation loss for C AM D IS TILL -4B (solid) and C AM D ISTILL -8B (dashed) over normalized training progress. Both backbones converge to similar final values (SFT ≈ 0.003, distillation ≈ 0.04), indicating stable optimization across model scales. Curves are logged directly without additional smoothing.

19

Preprint

H

C AMERA P OSE E STIMATION M ETHODS AND T EACHER C HOICE

Our camera tokens are distilled from a 3D foundation model that estimates camera geometry. We briefly review candidate estimators and explain our teacher choice. Classical structure-from-motion and SLAM systems, such as COLMAP (Schonberger & Frahm, 2016) and DROID-SLAM (Teed & Deng, 2021), recover trajectories through feature matching and geometric optimization. These pipelines can be slow and brittle under low texture or pure rotation. They are also sensitive to dynamic subjects, which are common in film, television, and vlog footage: when a moving person occupies much of the frame, feature matching may attribute subject motion to the camera. Feed-forward geometry transformers instead learn scene-level priors. DUSt3R (Wang et al., 2024), VGGT (Wang et al., 2025b), and VGGT-Ω (Wang et al., 2026a) jointly estimate camera parameters, depth, and point maps in a single forward pass without bundle adjustment. Permutation-equivariant models such as π 3 (Wang et al., 2026c) improve multi-view consistency and long-sequence stability. ViPE (Huang et al., 2025) jointly estimates depth and pose in low-texture and high-motion scenes, Reloc3r (Dong et al., 2025) focuses on relative-pose regression, and DROID-SLAM in the Wild (Li et al., 2026) improves SLAM robustness in dynamic environments. Large annotated resources such as SpatialVID (Wang et al., 2025a) further support progress in video geometry. We select VGGT / VGGT-Ω as teachers for three reasons. First, their feed-forward inference avoids per-scene optimization. Second, their joint reasoning over cameras and 3D structure is better suited to shots dominated by dynamic subjects than matching-based SfM. Third, they produce one compact 2048-dimensional camera token per frame, providing a direct geometry-aligned target for GCTE.

I

G EOMETRY-O NLY BASELINE D ETAILS

The geometry-only baseline (§5) converts VGGT-Ω camera poses into C AM C HOREO-style segment labels using the same deterministic pipeline for all benchmark clips. From consecutive poses it computes camera-local translation (∆x, ∆y, ∆z), local yaw/pitch/roll, world speed, and trajectory curvature. These eight signals are z-normalized for segmentation. Segmentation. Frames below both a translation-speed threshold of 0.005 and an angular-speed threshold of 0.3◦ are grouped into static intervals of at least three frames. Their boundaries are fixed, and the remaining intervals are segmented with PELT change-point detection using an RBF cost, penalty 3.0, and minimum segment length three. Segments shorter than three frames are merged into the longer neighbor, and adjacent segments with identical label sets are merged. Threshold classification. Within each segment, local translation and angular velocities are averaged and each axis is thresholded independently. The evaluated pipeline uses a translation threshold τt =0.02 in the pose encoder’s local units and a rotation threshold τr =0.5◦ : Condition

Label

∆z > τt / < −τt ∆x > τt / < −τt ∆y > τt / < −τt yaw > τr / < −τr pitch > τr / < −τr roll > τr / < −τr no axis exceeds its threshold

Dolly In / Dolly Out Truck Right / Truck Left Pedestal Down / Pedestal Up Pan Left / Pan Right Tilt Down / Tilt Up Roll CCW / Roll CW Static

Multiple axes can exceed their thresholds simultaneously, producing compound camera motion. Arc is approximated when lateral translation and yaw have absolute Pearson correlation at least 0.7 with sufficient amplitude; Follow requires sustained forward motion and moderate trajectory curvature. Unstable is detected at the video level when the high-frequency FFT energy ratio of raw lateral motion exceeds 0.3, using 0.2 of the spectrum as the low-frequency cutoff and at least eight frames. The baseline cannot predict Zoom, which depends on focal-length change rather than extrinsics, or Focus Shift, which is optical rather than geometric. Its hand-designed rules are intended as an interpretable pose-only reference, not a competitive learned model. 20

Preprint

J

M ETHOD C OMPARISON ON THE Q WEN 3-VL-8B BACKBONE

Table 8 repeats the comparison from Table 3 with the Qwen3-VL-8B backbone. The pattern is consistent with the 4B results. Pose-as-text prompting (+PromptInject) provides little improvement in frame-level micro F1, SFT produces a large gain, and both camera-token methods improve further. C AM D ISTILL remains close to C AM I NJECT while avoiding the 3D teacher at inference, retaining the low latency of SFT with only a modest memory increase. Table 8. Method comparison on the Qwen3-VL-8B backbone. Same protocol as Table 3. Quality metrics are micro/macro type-and-direction F1 and segment localization/detection F1 at IoU 0.5. Latency (s/clip, batch 1) and peak GPU memory measure inference cost on a single H100. +PromptInject and C AM I NJECT run the VGGT-Ω teacher at inference, whereas +SFT and C AM D ISTILL do not. Per quality column, best is in bold and second-best is underlined; ↓ lower is better. Method Qwen3-VL-8B +PromptInject +SFT +CamDistill +CamInject

K

Micro F1

Macro F1

SegLoc@0.5

SegDet@0.5

Latency (s/clip) ↓

Peak Mem (GB) ↓

28.3

9.1

67.7

15.0

12.8

27.9

28.1 64.1 67.8 68.3

12.2 51.9 57.9 59.2

74.5 79.7 80.4 80.8

18.6 34.9 38.8 39.0

21.5 12.8 12.8 20.1

33.7 27.9 31.6 33.0

D ETAILED GCTE B LOCK S TRUCTURE

GCTE alternates two attention operations within each of its M blocks. A frame-wise cross-attention lets each frame’s camera token query the frozen visual tokens of that frame, and a global camera self-attention then lets the per-frame camera tokens exchange information within the video. Both are standard pre-norm transformer sublayers with LayerNorm (LN), QK-normalized attention, LayerScale-gated residuals, and a feed-forward layer, and both write only to the camera tokens, so the pretrained visual stream is unchanged. This section expands the abstract CrossAttn and SelfAttn maps of Section 4 into their sublayer form. (m−1)

Frame-wise cross-attention. In block m, the camera token ci of frame i queries that frame’s (ℓm ) frozen features xi at the tapped vision layer ℓm . With multi-head cross-attention MHCA (query from the camera token, keys and values from the frame features) and LayerScale vectors γ1 , γ2 , (m) (m−1) (m−1) (ℓ )
ui = ci + γ1 ⊙ MHCA LN(ci ), LN(xi m ) , (7) (m) (m) (m)
c̃i = ui + γ2 ⊙ FFN LN(ui ) . (8) (ℓ )

The keys and values are read-only, so the visual tokens xi m are never updated. Global camera self-attention. The post-cross-attention tokens of a video then attend to one another through multi-head self-attention MHSA, with LayerScale vectors γ3 , γ4 , (m) (m) (m)
vi = c̃i + γ3 ⊙ MHSA LN(c̃1:T ) i , (9) (m) (m) (m)
ci = vi + γ4 ⊙ FFN LN(vi ) . (10) The attention is masked block-diagonally, so camera tokens from different videos in a batch do not interact. After the final block, the frame-level and temporally contextualized states are concatenated into the token distilled against the teacher,  (M ) (M )  zi = c̃i ; ci ∈ R2dc . (11) Algorithm 1 summarizes the full forward pass. Implementation notes. Each camera branch has dimension dc = 1024. Concatenating the final block’s post-cross-attention and post-self-attention states gives the 2048-dimensional token zi , which matches the cached teacher token. The two camera queries are initialized from zero-mean Gaussians (σ = 10−3 ) and all linear layers with Xavier initialization. During training the teacher token supervises zi through the distillation loss, and at inference the teacher branch is removed so that only the projected zi enters the LLM. 21

Preprint

Algorithm 1 Forward pass of GCTE. 1: Initialize c1 with the first-frame camera query and c2:T with the shared non-first-frame camera

query. 2: for m = 1 to M do 3: Select frozen vision feature layer ℓm . 4: for each frame i do (ℓ ) 5: frame-wise cross-attention: update ci by cross-attention with Q = ci and K, V = xi m ; (ℓ )

keep xi m unchanged. 6: end for 7: Save the post-frame-wise cross-attention tokens in the last block as the frame-level branch. 8: global camera self-attention: for each video independently, update c1:T by self-attention over camera tokens only. 9: end for 10: Concatenate the final post-frame-wise cross-attention and post-global camera self-attention tokens to obtain zi = [cframe ; cglobal ]. i i 11: Project zi to the LLM hidden size and insert it before frame i’s visual tokens. Position encoding. The decoder uses multimodal RoPE (M-RoPE), which assigns each token a (t, h, w) coordinate. A camera token receives the temporal index of its frame and the spatial center of that frame’s visual patch grid. The decoder therefore interprets it as part of the corresponding frame rather than as an additional time step. Because its coordinate is fixed, prepending or appending the token changes sequence order but not its M-RoPE position.

L

C OMPUTATIONAL C OMPLEXITY

We analyze only the inference overhead introduced by C AM D ISTILL, because its 3D teacher is absent at test time. Let T be the number of frames, P the number of frozen visual tokens per frame, dv the vision width, dc the camera-token width, dl the LLM width, and M the number of GCTE blocks. GCTE blocks. In each block, frame-wise cross-attention projects the T P frozen visual tokens to keys and values and lets one camera query per frame attend to its P visual tokens. Including projections, attention interactions, and the camera-token feed-forward update, its cost is
O T P dv dc + T P dc + T d2c . (12) Global camera self-attention operates on only the T camera tokens. Its projections, pairwise attention, and feed-forward update cost
O T d2c + T 2 dc . (13) Across M blocks, GCTE therefore adds

O M T P dv dc + T P dc + T d2c + T 2 dc . (14) For fixed hidden widths, this overhead is linear in the number of visual tokens T P , apart from selfattention over the much shorter sequence of T camera tokens. The expression includes the visual key/value projections omitted by an attention-interaction-only analysis. LLM sequence overhead. C AM D ISTILL inserts exactly one camera token per frame. The LLM sequence length grows from T P + L to T P + T + L, where L is the text length. Relative to the visual sequence, the token-count increase is T /(T P ) = 1/P . This does not make decoder attention free, but it is much smaller than adding dense geometry tokens for every visual patch. The measured system-level effect is reported in Table 3: latency changes from 10.1 to 10.2 seconds per clip on the 4B backbone, while peak memory increases from 18.3 to 20.1 GB. We therefore describe C AM D ISTILL as having negligible measured latency overhead, with a modest memory increase, rather than as cost-free. Summary. C AM D ISTILL is efficient for two reasons. First, GCTE replaces the teacher’s global attention over all visual tokens with single-query cross-attention and self-attention over only T camera tokens. Second, it increases the LLM sequence length by only a fraction 1/P . Once the teacher 22

Preprint

(a) Label co-occurrence within segments

(b) Most frequent co-occurring pairs

Static

463

Pan-L + Truck-R

Unstable Pan-L Pan-R

383

Pan-R + Truck-L

400

298

Follow + Pan-R

Tilt-U Tilt-D

282

Roll-CW

300

Roll-CCW Truck-L Truck-R Pedestal-U

200

Pedestal-D Dolly-In

co-occurring segments

Dolly-In + Pan-L Dolly-In + Pan-R

262

Follow + Pan-L

258 252

Dolly-In + Follow

236

Dolly-Out + Follow

Dolly-Out

228

Pan-R + Tilt-U

Zoom-In Zoom-Out

100

Focus

Pan-R + Tilt-D

199

Follow + Truck-R

194

Follow Arc-CW 0

Arc-CCW

Focus

Follow

Arc-CW

Zoom-Out

Dolly-In

Zoom-In

Dolly-Out

Pedestal-D

Truck-L

Truck-R

Pedestal-U

Roll-CCW

Tilt-D

Tilt-U

Roll-CW

Pan-R

Static

Pan-L

Unstable

Arc-CCW

205

Pan-L + Tilt-U

0

100

200

300

segments where both co-occur

400

500

Figure 7. Camera-motion co-occurrence in C AM C HOREO. Left: direction-aware co-occurrence matrix. Right: most frequent label pairs. (a) When a label is missed, what is predicted

(b) Segment-level error decomposition

Static Unstable

25

Pan-L Pan-R

37.0%

Correct

Tilt-U

missed ground-truth label

Tilt-D

20

Roll-CW Roll-CCW

0.3%

Right types, wrong direction

Truck-L 15

Truck-R Pedestal-U Pedestal-D Dolly-In

10

Dolly-Out

40.6%

Wrong movement set

Zoom-In Zoom-Out 5

Focus Follow

22.1%

Missed (not localized)

Arc-CW 0

Arc-CCW

Follow

Arc-CW

Focus

Zoom-In

Dolly-Out

Zoom-Out

Dolly-In

Pedestal-D

Truck-R

Pedestal-U

Truck-L

Roll-CW

Roll-CCW

Tilt-D

Pan-L

Tilt-U

Pan-R

Static

Unstable

Arc-CCW

0

10

20

30

% of ground-truth segments (IoU ≥ 0.5)

40

50

predicted instead (%)

Figure 8. Error analysis of C AM D ISTILL-8B. Left: row-normalized frame-level confusions. Right: segmentlevel error decomposition after IoU ≥ 0.5 matching.

is removed, the resulting inference cost is close to that of the SFT backbone, consistent with the latency and memory measurements in Table 3.

M

C OMPOSITIONALITY AND E RROR A NALYSIS

Co-occurrence. Camera motion in C AM C HOREO is strongly compositional: 44.2% of segments contain compound camera motion with at least two simultaneous movement primitives. Figure 7 reveals recurring structures rather than arbitrary combinations. Pan and Truck often occur in opposite directions during reveals or orbiting shots, Follow commonly accompanies Pan or Dolly, and Pan+Tilt represents coordinated rotation. The unusual opposing Dolly–Zoom combination appears in eight segments. Error decomposition. At temporal IoU ≥ 0.5 (Figure 8), C AM D ISTILL-8B recovers 37.0% of ground-truth segments with the exact label set. A further 40.6% are localized correctly but contain an incomplete or incorrect set of movements, while 22.1% are missed. Only 0.3% have the correct movement type but the wrong direction. The remaining confusions are physically plausible: distinguishing Dolly from Zoom and Pan from Truck requires parallax or lens evidence that is not apparent from image-plane motion alone.

N

P ER -C LASS D IFFICULTY, C OMPLEXITY, AND T EMPORAL P RECISION

We next use C AM D ISTILL-8B predictions to identify where the difficulty of C AM C HOREO is concentrated. 23

Preprint

(a) Per-class frame-level F1

100

Rotation

Translation

Optical

Subject-ref.

Stability

Frame-level F1 (%)

80

60

40

20

e

l-D

bl ta

CW

sta de Pe

ns U

s cu Fo

Ro ll-

CC W

-In

Ro ll-

ut

om Zo

l-U

Zo

om

-O

D

sta

Pe

de

U

lt-

lt-

Ti

Ti

w

-C W rc A

L k-

llo Fo

uc Tr

ut

-C CW

rc A

R

-O ly

D

ol

R

kuc Tr

L n-

nPa

Pa

ly ol D

St

at

ic

-In

0

Figure 9. Per-class frame-level F1 of C AM D ISTILL-8B, sorted by score and colored by motion family. (a) F1 vs. compositional density

Frame-level F1 (%)

80

80

72.1 63.9

65.8

60

40

20

0

(b) F1 vs. temporal complexity

100

Mean video F1 (%)

100

71.1 64.1

60

59.2

58.8

52.4

40

20

1

2

# simultaneous camera motions in frame

0

3+

1

2

3

4

# ground-truth segments per video

5+

Figure 10. Performance versus task complexity. (a) Frame-level F1 decreases as more camera motions cooccur; (b) mean per-video F1 decreases as the clip is split into more temporal segments.

Per-class difficulty and the long tail. Figure 9 reports frame-level F1 for all 20 direction-aware labels. Performance is uneven and broadly follows class frequency: the more frequent half of the labels average 67.5 F1, compared with 46.6 for the rarer half. Common and geometrically salient movements such as Static, Dolly, Pan, and Truck are recognized reliably. Rare optical and rotational movements such as Zoom, Roll, and Focus Shift remain difficult because they have fewer training examples and depend on subtle cues rather than large image-plane displacement. Arc, although equally rare, is recognized more reliably, since its curved orbiting trajectory produces distinctive image-plane motion. This class imbalance motivates reporting macro averages alongside micro averages. Difficulty grows with composition and temporal structure. Figure 10 isolates the two properties central to our task. In panel (a), frame-level F1 is 72.1 with one active movement, compared with 63.9 for two and 65.8 for three or more, showing that compound frames are harder than singlemotion frames without implying a monotonic trend within the compound groups. In panel (b), mean per-video F1 decreases monotonically from 71.1 for clips with one segment to 52.4 for clips with five or more. These trends confirm that both temporal structure and motion composition contribute substantially to the difficulty of C AM C HOREO. Temporal precision and segmentation behavior. For predicted segments matched at temporal IoU ≥ 0.5, the boundaries are precise (Figure 11). The median absolute start and end offsets are 0.00 s and 0.04 s, and approximately 91% of matched boundaries fall within 0.5 s of the annotation. At the clip level (Table 9), C AM D ISTILL-8B predicts an average of 1.92 segments, compared with 2.03 in the ground truth. It predicts the exact number of segments for 65.7% of clips, under-segments 21.9%, and over-segments 12.4%. The model is therefore slightly conservative: it is more likely to merge adjacent phases than to introduce spurious boundaries. This behavior is also visible in the qualitative examples in Section O. 24

Preprint

(a) Temporal boundary precision 4000

start offset end offset

# matched segments

3500

Table 9. Segmentation and boundary statistics of C AM D ISTILL-8B on C AM C HOREO. Statistic

Value

Mean GT segments / video Mean pred. segments / video Exact segment count under- / over-seg.

2.03 1.92

3000 2500 2000 1500 1000 500 0

−1.5

−1.0

−0.5

0.0

0.5

Predicted − GT boundary (s), matched segments

1.0

1.5

Figure 11. Boundary precision. Signed offset between matched predicted and ground-truth segment boundaries.

O

Q UALITATIVE R ESULTS

O.1

R EPRESENTATIVE P REDICTIONS

Median |∆| start / end Within 0.5 s (start / end)

65.7% 21.9% / 12.4% 0.00 / 0.04 s 91.0 / 91.3%

Figures 12–17 show example predictions of C AM D ISTILL-8B on C AM C HOREO, drawn from clips that span a range of temporal and compositional complexity. In each panel, sampled frames appear above the ground-truth (GT) and predicted (Pred) segment timelines. Segments matched by temporal IoU share a color, unmatched segments are gray, and every segment is labeled with its movements. Across these examples, the model recovers the dominant movements and the overall temporal structure, including compositional cases such as a simultaneous Roll and Pedestal Up or a Truck+Pan+Arc orbit. Its remaining errors are mostly merged adjacent phases or a movement dropped from a densely compositional segment, consistent with the error analysis in Appendix M. O.2

FAILURE C ASES

C AM D ISTILL still breaks down on the hardest clips, and Figure 18 shows four representative errors. They fall into a few recurring modes. First, the model confuses movements that look similar in the image plane but differ geometrically, reading a lateral Truck as a Pan and a forward Dolly as a Zoom; both distinctions require the parallax cues that its encoder captures poorly. Second, it misses subjectreferenced motion, labeling a hand-held Follow shot as Static or a plain Pan. Third, it mistakes irregular Unstable footage for Roll and over-segments it into several short intervals. Across all four, dense compositions are only partially recovered and adjacent phases are often merged. These modes match the quantitative error analysis in Appendix M, where most residual errors are incomplete or incorrect label sets rather than boundary errors.

25

Preprint

GT

Roll-CCW, Pedestal-U

Pred

Roll-CCW, Pedestal-U

0

1

2

3

GT

Unstable

Pred

Unstable

0

2

4

time (s)

GT

Tilt-U

Pred

Tilt-U

0

1

2

4

time (s)

time (s)

5

6

6

8

3

10

4

Figure 12. Qualitative predictions of C AM D ISTILL-8B on C AM C HOREO (part 1). Matched GT and Pred segments (by temporal IoU) share a color, unmatched segments are gray, and each segment lists its movements.

26

Preprint

GT

Tilt-D

Pred

Static

Tilt-D

Static

0

1

2

GT

Dolly In

Pred

Dolly In

0

1

2

3

time (s)

3

4

time (s)

GT

4

5

6

Zoom In

Pred

Zoom In

0

1

2

3

time (s)

4

5

6

Figure 13. Qualitative predictions on C AM C HOREO (part 2). Conventions as in Figure 12.

27

7

Preprint

GT

Pan-L, Truck-R, Arc-CCW

Pan-R, Truck-L, Arc-CW

Pred

Pan-L, Truck-R, Arc-CCW

Pan-R, Truck-L, Arc-CW

0

2

4

time (s)

6

8

GT

Pan-L, Tilt-D

Static

Pred

Pan-L, Tilt-D

Static

0

1

GT

2

3

time (s)

Static

Pred 0.5

1.0

5

6

Pan-L, Tilt-U

Static

0.0

4

10

Pan-L

Pan-L, Tilt-U

1.5

2.0

time (s)

2.5

Pan-L

3.0

3.5

Figure 14. Qualitative predictions on C AM C HOREO (part 3). Conventions as in Figure 12.

28

4.0

Preprint

GT

Dolly In

Pred

Pan-R, Dolly In

Dolly In

Dolly In

Dolly In, Pan-R

0

1

Dolly In

2

GT

3

time (s)

4

Truck-R, Follow

Pred

Truck-R

Truck-R, Follow

0

1

2

GT

3

time (s)

4

5

Static

Pred

Truck-R, Follow

Static

0

1

2

6

Truck-R, Follow

3

time (s)

4

5

Figure 15. Qualitative predictions on C AM C HOREO (part 4). Conventions as in Figure 12.

29

6

Preprint

GT

Static

Pred

Pan-L, Tilt-U

Static

0

1

Pan-L, Tilt-U

2

3

time (s)

4

5

GT Truck-R, Pan-L

Truck-R, Pan-L, Arc-CCW

Pred Pan-L, Truck-R

Pan-L, Truck-R, Arc-CCW

0

2

GT

Roll-CCW

Pred

Roll-CCW

0

GT

4

6

7

10

12

Pan-L, Dolly In Dolly In, Pan-L

2

4

6

2

time (s)

8

10

Pan-L, Dolly In

Dolly In

0

8

time (s)

Dolly In

Dolly In

Pred

6

Pan-R, Dolly In

Pan-L, Dolly In

4

time (s)

12

Dolly In

Pan-R, Dolly In

6

14

Dolly In

8

Figure 16. Qualitative predictions on C AM C HOREO (part 5). Conventions as in Figure 12.

30

Preprint

GT

Pan-R, Dolly In, Follow

Dolly In, Follow

Pred

Dolly In, Follow, Tilt-U

Follow, Dolly In

0

1

Pan-R

Dolly Out

Pred

Pan-R

Dolly Out

0

1

time (s)

3

Pan-L, Truck-R

Pred

Truck-R

Pan-L, Truck-R

4

Pan-L, Dolly In

4

5

2

7

Truck-R

Truck-R Dolly In, Truck-R

Pan-R, Truck-R

8

10

time (s)

12

14

Pan-R

time (s)

4

16

Pan-L

Pan-R

3

8

Pan-R, Truck-R

Pan-R, Tilt-U

1

6

Truck-R

Pan-R, Tilt-U

Static

0

6

Pan-L, Dolly In

time (s)

6

Static

Pred

5

Pan-R

Truck-R

GT

4

Pan-R

2

2

Dolly In, Follow

3

GT

0

Pan-R, Dolly In, Follow

2

GT

Dolly In, Follow

Pan-L

5

6

Figure 17. Qualitative predictions on C AM C HOREO (part 6). Conventions as in Figure 12.

31

7

Preprint

GT Pred

Truck-R, Tilt-D

Truck-R, Pedestal-U

Tilt-U

Pan-R, Tilt-D, Follow

0

1

GT

2

Pan-R, Follow

3

time (s)

4

5

7

Dolly In, Pedestal-U, Tilt-U

Pedestal-D

0

6

Pedestal-D, Dolly In

Tilt-D, Pedestal-D

Pred

Truck-R, Follow

Zoom In

2

4

time (s)

6

8

10

GT

Dolly Out, Truck-L, Follow

Follow, Dolly Out, Pan-R

Follow, Dolly Out, Truck-R

Pan-L, Follow, Pedestal-U

Pred

Static

Pan-R

Static

Pan-L

0

GT

1

Zoom In

2

3

Dolly In, Follow, Pan-L

Pred

4

time (s)

1

6

Static Roll-CCW

2

7

8

Unstable

Static

0

5

3

time (s)

Roll-CW

4

Roll-CCW

5

6

Figure 18. Failure cases of C AM D ISTILL-8B. Top to bottom: a lateral Truck predicted as Pan; a forward Dolly predicted as Zoom; a hand-held Follow predicted as Static or Pan; and Unstable footage predicted as Roll and over-segmented into short intervals. Conventions as in Figure 12.

32

Related documents

• WorldSculpt: Generating Compositional Worlds from Grounded Videos#666674
• DINOcular: Self-Supervised Visuospatial Representations#926204
• PruneGround: Plug-and-play Spatial Pruning for 3D Visual Grounding#972457
• RegCL: Compact Continual SAM Adaptation for Visual Grounding in Multi-Sensorial Media#684244
• RegCL: Compact Continual SAM Adaptation for Visual Grounding in Multi-Sensorial Media#682469