Seeing Fast and Slow: Learning the Flow of Time in Videos Yen-Siang Wu1,2 , Rundong Luo1 , Jingsen Zhu1 , Tao Tu1 , Ali Farhadi3 , Matthew Wallingford3 , Yu-Chiang Frank Wang2 , Steve Marschner1 , Wei-Chiu Ma1
arXiv:2604.21931v1 [cs.CV] 23 Apr 2026
1
Cornell University, 2 National Taiwan University, 3 University of Washington
Abstract How can we tell whether a video has been sped up or slowed down? How can we generate videos at different speeds? Although videos have been central to modern computer vision research, little attention has been paid to perceiving and controlling the passage of time. In this paper, we study time as a learnable visual concept and develop models for reasoning about and manipulating the flow of time in videos. We first exploit the multimodal cues and temporal structure naturally present in videos to learn, in a self-supervised manner, to detect speed changes and estimate playback speed. We then show that these learned temporal reasoning models enable us to curate the largest slow-motion video dataset to date from noisy in-the-wild sources. Such slow-motion footage, typically filmed by high-speed cameras, contains substantially richer temporal detail than standard videos. Using this data, we further develop models capable of temporal control, including speed-conditioned video generation, which produces motion at specified playback speed, and temporal super-resolution, which tranforms low-FPS, blurry videos into high-FPS sequences with fine-grained temporal details. Our findings highlight time as a manipulable, perceptual dimension in video learning, opening doors to temporally controllable video generation, temporal forensics detection, and potentially richer world-models that understand how events unfold over time. Project Page: https://seeing-fast-and-slow.github.io/
1
Introduction
Humans exhibit a strong intuition for the flow of time. Even without explicit cues, we often sense when a video is sped up or slowed down: surface ripples propagating too quickly, objects falling at unnatural rates, or human motion that feels sluggish. This intuition reflects an implicit understanding of the speed at which events unfold in the physical world [47]. Despite remarkable progress in video understanding and generation, current models lack such temporal reasoning [32, 33, 45]. When asked to predict playback speed, modern vision–language models (VLMs) [12, 68] frequently hallucinate or give incorrect predictions. When prompted to generate content at a particular speed, existing generative models [14, 50, 51] often ignore instructions or generate video with imprecise timing. These issues highlight a fundamental limitation in current models: the inability to reason about time. This limitation is not particularly surprising. Modern models primarily learn from videos captured at standard frame rates (e.g., 24–60 fps) [11, 56], and therefore never learn the notion of temporal speed. In effect, we are asking the model to infer time’s variability from observing a single cadence. We argue that for models to understand the flow of time and to control temporal dynamics, they must be exposed to data spanning the 1
Probability
Speed Change Detection
Time
Video Speed Estimation
Prediction: 0.24
Temporal Super-resolution
Prediction: 0.11
Input
SloMo
Speed-conditioned Video Generation
Figure 1 We develop models towards perceiving and manipulating the flow of time, including (a) speed-change detection, which locates moments where playback speed shifts; (b) video speed estimation, which infers how much a video has been sped up or slowed down; (c) extreme temporal super-resolution, converting low-FPS, blurry videos into their high-FPS, clear counterparts; and (d) speed-conditioned video generation, synthesizing the same event at user-specified temporal speeds. For more details on these results, please refer to section 5 and the project page. Please open in Adobe Acrobat Reader to view the embedded animations.
continuum of temporal speeds. Obtaining such data at scale is not trivial. Although there are vast untapped stores of slow-motion videos online, the associated metadata required to train models, such as the frame rate, playback speed, and time stamps, are often incomplete. A natural solution would be to have humans manually annotate the videos. However, naively labeling is prohibitively time consuming and imprecise. Therefore, we look to other directions for providing temporal supervision for video understanding and generation models. In this paper, we show that temporal cues and multi-modal signals in videos can be leveraged to infer playback speed without explicit labels. Our first key insight builds on the principle of time–frequency scaling: when a video’s playback speed changes, its audio pitch shifts. Thus, by locating regions where the pitch shifts (Fig. 2), we can obtain free cross-modal supervision to train models that visually detect speed changes. Our second insight exploits the equivariance of speed estimation under temporal rescaling: if we downsample a video by a certain factor, its perceived speed should decrease proportionally by the same factor. By resampling each clip and enforcing this proportional relationship in the model’s predictions, we use temporal resampling as a powerful self-supervised training signal. Together, these complementary cues enable models to infer the lapse of time directly from visual dynamics, even in the absence of clean or reliable temporal labels. We apply our temporal speed models to in-the-wild videos collected from online platforms. Using these models, we automatically annotate and curate a large-scale slow-motion dataset spanning diverse activities, motion patterns, and temporal scales. These slow-motion videos, typically captured at high frame rates (e.g., 1000+ FPS) and played back at standard frame rates (e.g., 30 FPS), preserve fine temporal details and exhibit reduced motion blur, allowing machines to perceive and learn how real-world dynamics actually unfold. Next, using our newly constructed dataset, we develop models capable of manipulating the flow of time in videos. We focus on two tasks: (i) speed-conditioned video generation, which equips models with the ability to render physical motion at different temporal speeds; and (ii) temporal super-resolution, which converts videos into higher frame-rate sequences. Both tasks are challenging: standard videos often lack fine-grained motion
2
(b) Speed (c) change
(d)
Timestep (a)
Timestep (b)
Timestep (c)
Timestep (d)
Frequency
(a)
Slow motion Normal speed
Time
Slow-motion average image
Normal-speed average image
Figure 2 Audio signal naturally encodes cues for speed change detection. When a video’s playback speed changes, its audio pitch shifts accordingly – speeding up raises the pitch, while slowing down lowers it. This provides free cross-modal supervision for detecting speed changes by locating moments of pitch variation. Left: Audio spectrogram (used only during training). Right: Average image at each selected timestep, obtained by averaging nearby frames. Higher playback speed leads to larger content changes and therefore greater blur.
cues due to motion blur, and synthesizing motion at precise speeds requires accurate temporal modeling. We show that finetuning existing models on our slow-motion dataset enables them to generate lifelike dynamics at controllable playback speeds and to interpolate frames at higher effective frame rates. Since the tasks explored in this work are relatively new, there are no established benchmarks or standard evaluation protocols. To ensure a fair and meaningful assessment, we carefully curate dedicated evaluation datasets and conduct comprehensive human perceptual studies. Our experiments show that the proposed models perform strongly across both understanding and generation tasks, achieving 92% accuracy on speedchange detection, near-human accuracy on playback-speed estimation, 80.3% human-preference win rate over the base model for temporal super-resolution.
2
Related Works
Time as a learnable visual concept. Humans exhibit a remarkably malleable perception of time when watching
videos. Studies in cognitive science [47] show that people tend to overestimate the duration of an action when it is presented in slow motion. From a computer vision perspective, learning the concept of time has long intrigued researchers. For instance, earlier works on arrow-of-time (AoT) [20, 42, 61] sought to determine if a video is playing forward or backward. Building on this line of research, recent studies have extended large multimodal models with AoT awareness [1, 63] as well as broader temporal reasoning capabilities [2, 15, 17, 59]. In the context of video representation learning, temporal information has also been widely exploited as a self-supervised signal [6, 16, 31, 37, 53, 64]. In particular, pace classification has emerged as an effective pretext task for improving temporal understanding, where videos are augmented with different temporal sampling rates and models are trained to classify them as slow, normal, or fast [6, 53, 64]. Extending this idea, SpeedNet [6] demonstrates that the learned pace classifier can further be leveraged to estimate “speediness” scores across a video, enabling adaptive video speedups. More recently, Pulse-of-Motion [19] proposed a video FPS predictor to evaluate the speed correctness of major video generation models. However, their speed-perception model is trained primarily on standard frame-rate videos (≤ 60 fps), with only 8 videos at 120 fps (UVG [36]) and 118 videos at 240 fps (Adobe240fps [48]). As a result, their notion of speed is limited to a narrow range (1/2, 1, 2, 4×) and oftentimes inaccurate, which we will show in Sec. 5.2. In contrast, our method is trained on videos spanning a wide range of speeds and uses a dedicated self-supervised objective, allowing it to predict speed across a wide range of temporal scales. Manipulating the continuum of time. Understanding and altering the temporal dimension of videos is a
fundamental problem in computer vision. Frame interpolation addresses this by synthesizing intermediate frames between two existing inputs, effectively increasing a video’s temporal resolution. Earlier approaches to interpolation primarily relied on kernels [34, 40, 62] or optical flow estimation [24, 43] to model motion, whereas recent generative approaches leverage the rich priors of image or video diffusion models to better handle complex motion and improve realism [14, 18, 26, 55]. Beyond interpolation, prior work on video generation has explored motion control techniques [7, 9, 52, 60] to 3
Dataset
#Clips
#Videos
#Frames
Max raw FPS
Content
Adobe240fps [48] YouTube240 [27] NfS [30] X4K1000FPS [44] SportsSloMo [10]
118 1,014 100 175∗ 8,498∗
118 100 175 259
80K 296K 383K 875K 1.2M
240 240 240 1,000 240
Urban Unknown Generic Urban Sports
SloMo-44K (Ours)
44,632
18,235
18M
10,000+
Generic
Table 1 Comparison with existing high-frame-rate datasets. Our dataset is the largest generic slow-motion collection to date, containing over 70× more videos and 150× more frames than previous datasets. ∗ Counts are based on full-length clips rather than short, fixed-duration trimmed segments.
explicitly manipulate temporal dynamics. More recently, video editing approaches like BulletTime [58] and SpaceTimePilot [23] have enabled joint editing of camera motion and temporal progression, offering flexible spatio-temporal control. In this work, we focus on playback speed-conditioned video generation, ranging from normal speed (1.0×) to extreme slow motion (0.01×). Unlike editing-centric settings [23, 58] that merely rescale the temporal axis of an existing video, our generation-centric task requires the model to internalize the real-world progression speed of physical motion and accurately fabricate it at the requested speed (see supp. for detailed comparisons). Furthermore, the broad conditioning range requires our model to generate dynamics at an exceptionally high temporal resolution. By leveraging our proposed SloMo-44K, which spans diverse temporal scales and real-world motion patterns, our model achieves high-fidelity video generation across a wide range of speeds, outperforming existing approaches trained only on low-frame-rate or synthetic data (Sec. 5.3). Slow-motion video datasets. Large-scale video generation datasets such as WebVid-10M [5], Panda-70M [11],
and OpenVid-1M [39] have driven recent advances in video generation. However, these datasets primarily contain standard-speed footage (24–60 fps) and thus lack the dense temporal continuity required to model realistic motion dynamics. In contrast, slow-motion videos, recorded at hundreds or even thousands of fps, offer much finer temporal details that normal-speed videos cannot capture. Existing slow-motion datasets, however, remain limited in either scale, frame rate, or scene diversity. For example, Adobe240fps [48], YouTube240 [27], and NfS [30] each contain only a few hundred videos at 240 fps; SportsSloMo [10] includes more clips but focuses solely on sports scenes; and X4K1000FPS [44] consists of 1000 fps self-recorded videos yet remains small in size. Consequently, these datasets primarily serve as benchmarks for video frame interpolation rather than as large-scale resources for model training. In contrast, our proposed SloMo-44K is the first large-scale and general-purpose slow-motion dataset, offering a foundation for learning fine-grained dynamics.
3
Perceiving the Flow of Time
We aim to develop computational visual systems that can perceive and manipulate the flow of time. In this section, we focus on how models understand and perceive time, answering fundamental questions such as Has this video been sped up or slowed down? If so, when, and by how much? These abilities form the foundation for deeper temporal reasoning and pave the way for fine-grained temporal control, which we explore in Sec. 4. As most of the metadata associated with public slow-motion videos, such as playback speed, is often noisy or incomplete, data for models to learn from is lacking. In the following section, we detail how models can learn to detect speed changes and infer the speed of time through self-supervision. These models then curate the largest slow-motion video dataset to date for visually understanding and controlling the flow of time.
3.1
Learning to Detect Temporal Speed Changes
We begin with detecting temporal speed changes in videos. Our goal is to locate times in the video where the playback speed changes, transitioning between normal, fast, and slow motion. Such effects are common in modern videos and are often used to emphasize key moments or create dramatic effects. If a model can
4
reliably detect these changes, it not only reveals an understanding of temporal continuity, but also provides a foundation for estimating absolute speed and calibrating temporal labels in large-scale video datasets. Manually annotating speed changes is laborious and difficult to scale, and using optical flow as a proxy measurement is often noisy and unreliable (see Sec. 5). We instead leverage cues encoded in the audio of videos. Specifically, our key idea is to use the principle of time–frequency scaling to detect speed changes. For videos that contain original audio tracks (e.g., not dubbed or overlaid with background music), there exists a natural coupling between visual motion and sound. When such videos are sped up (compressed in time), their corresponding audio frequencies shift upward, resulting in a higher pitch, and vice versa when slowed down. These salient audio cues (pitch shift) allows us to effectively locate speed-change events. An illustration is shown in Fig. 2. Using this strategy, we automatically collect over 8K speed-change labels. We then finetune a visual speedchange detector based on VideoMAEv2 [54] using a standard binary cross-entropy loss. Notably, during inference, the detector operates solely on visual input and is agnostic to audio content.
3.2
Learning to Infer the Speed of Time
Once we can detect speed changes within videos, a natural follow-up question arises: By how much has the video been sped up or slowed down?
Original video (no GT speed required) Speed predictor
As there is no current reliable source of supervision, we rely on a self-supervised objective. Our key insight is that the speed estimation model should be equivariant to temporal resampling, i.e., if we speed up a video by a factor of k, the predicted speed should scale by k. By enforcing this proportional relationship between the model’s input and output, we can transform temporal resampling into a powerful self-supervised training signal (figure 3). Formally, let V be a video clip sampled from a video of duration T , and Vk be the k-times accelerated version of V, where k ∼ N (1, T2 ). Denote fθ as the playback speed estimator. We train the model fθ using: 2 L = log fθ (Vk ) − log(k · fθ (V)) . (1)
Accelerate
Shared
Speed predictor
!
ℒ = $%& !# − $%& ( ⋅ !
$
!#
!× faster
Figure 3 Learning to predict speed. Our speed estimator is trained with both self-supervised and supervised objectives. For videos without ground-truth speed, we enforce temporal consistency by subsampling the video by a factor of K, feeding both original and accelerated clips into the model, and constraining their predicted speeds to differ by K. For videos with known frame rates, we directly regress the playback speed.
Calibration. While the self-supervised loss allows
the model to predict playback speed purely from motion cues without relying on ground-truth labels, the estimation is sometimes only correct up to a scale. To anchor predictions to absolute playback rates, we additionally incorporate a small set of videos with known ground-truth playback speeds. For example, we use videos from the Adobe240FPS dataset [48], where the playback speed is precisely determined. In addition to the self-supervised loss, we train the network with small amounts of supervised data in log space. Iterative prediction. With calibration, our model can already produce relatively accurate speed estimations.
However, in practice, we find that our model tends to underestimate when encountering extremely slow-motion videos. We hypothesize that this is because the motion difference in ultra-slow videos are often very subtle due to resolution constraints. Subsampling frames before inference further makes it challenging. To mitigate the issue, we adopt an iterative prediction approach. If a video is initially predicted to have a playback speed of x, we accelerate the video accordingly to bring it closer to normal speed, and then re-estimate its playback speed to obtain a more precise prediction. Repeating this process iteratively produces progressively refined predictions. The intuition is that speed differences become more discernible when the playback speed approaches one, where more training data is also concentrated, allowing the model to estimate speed more reliably. We empirically unroll the estimator three times. Please refer to the supp. material for more details and ablation study.
5
Method
ρ↑
rs ↑
RMSE↓
eRMSE ↓
Human expert
0.880
0.783
0.492
1.636
Optical flow VideoLLM [12] SpeedNet [6] Pulse-of-Motion∗ [19] Ours
0.385 0.426 0.476 0.508 0.735
0.354 0.308 0.331 0.525 0.706
1.568 1.261 1.181 0.649
4.796 3.529 3.258 1.913
Table 2 Video speed prediction results. We report the Pearson and Spearman correlation coefficients (ρ, rs ), root mean squared error (RMSE), and eRMSE , all computed in log space. Because optical-flow magnitude is used only to rank relative speed, its RMSE is omitted. Our method significantly narrows the gap between machine and human performance. ∗ Results are obtained using the authors’ publicly available checkpoint.
3.3
Annotating the Speed of Time
With reliable speed understanding models in hand, we now leverage them to transform in-the-wild slow-motion videos into a high-quality annotated dataset. Sources. We source videos from YouTube, Vimeo, and Flickr using queries such as “high frame rate,” “high-
speed camera,” and “slow motion,” along with their synonyms. Following [7, 35], we then use TransNetv2 [46] to segment videos into shots and an OCR model [65] to remove video clips with excessive text overlay. To further improve quality, we filter out CGI or screen recordings using Qwen2.5-VL [4] and discard low-quality samples based on video quality assessment [21]. Speed annotation. As many videos have unspecified or heterogeneous playback speed, we leverage our speed
change detector to segment videos into clips with homogeneous playback speeds. We then run our speed estimator to annotate the playback speed for each clip. Finally, we densely caption them with InternVL3 [68]. We refer the readers to the supp. material for more details. SloMo-44K. Our resulting dataset contains 44,632 slow-motion video clips with a total of 18 million frames,
making it the largest of its kind. It spans diverse real-world scenes and motion patterns, and covers a broad range of video durations. A comparison with previous datasets is shown in Tab. 1. For more statistics, please see the supp. material.
4
Manipulating the Flow of Time
Having learned to detect speed changes and infer the speed of time in videos, we now turn to the problem of manipulating it. In this section, we demonstrate how models can learn to adjust the dynamics of generated content or to slow down existing videos with fine-grained precision.
4.1
Speed-conditioned Video Generation
Given an image, a text prompt, and a target playback speed, our goal is to develop a model that can generate dynamic visual content unfolding at the desired temporal rate and show physical dynamics (e.g., fluid dynamics, shattering, vibration, etc) perceptible only at that temporal scale. Intuitively, for any text-to-video model, a potential way to control playback speed would be to incorporate motion-related text modifiers (e.g., ultra-slow, slightly-slow) into the prompt. However, as we show in Sec. 5.3, textual conditioning alone is insufficient for generating videos at the desired speed, often failing to make any noticeable difference. We conjecture that this is because physical motion cues are entangled with other semantic factors in the text prompt, resulting in a weak and easily overloaded conditioning signal. With this in mind, we propose to leverage our speed-annotated dataset to train a video model with explicit speed control. We build model upon Wan2.1-I2V [50] with two key modifications. Given a target speed, we first discretize it into logarithmically spaced buckets that represent different temporal speeds. Empirically, these buckets span from extremely slow (0.01×) to normal speed (1.0×), covering the range in which most
6
Prompt: ”A large bird of prey spreads its wings while perched on a stump.” Avg. optical flow magnitude -->
1.14
1.38
3.74
8.45
1.19
1.49
4.91
6.99
Prompt: ”Three individuals are joyfully celebrating, each wearing vibrant wigs and festive attire.” Avg. optical flow magnitude -->
Figure 4 Speed-conditioned video generation. Given a text prompt and an image as input (first column), our model learns to generate videos of different speed. The first row shows the average image of the video, while the second row visualizes the optical flow between the first and second frame. When the speed control is set to slow, the average image should still be crisp (because the scene does not change much) and the magnitude of optical flow is small (and thus the color is faint). As the speed of the video increases, the average image becomes blurrier and the magnitude of optical flow increases (the color become more saturated). For more results, see the project page.
SloMo-44K videos fall (see supp. for analysis). Then we encode the bucket id with sinusoidal positional embedding ϕ, apply a multilayer perceptron MLPθ , and add it to the timestep embedding [3]: Bucket_ID =
log(speed) − log(0.01) · Nbuckets , log(1) − log(0.01)
time_emb ← time_emb + MLPθ · ϕ(Bucket_ID).
(2)
This encourages the model to align its denoising schedule with the temporal speed of the video. Empirically, we set Nbuckets = 10. To further enhance the control over temporal speed, we modulate the latent features with playback speed through frame-wise conditioning [3]: latent[i] ← latent[i] + MLPψ (ϕ(i · speed)),
(3)
where latent[i] denotes the latent feature at temporal index i. We follow the image-to-video setup of Wan2.1-I2V [50]. During training, we optimize both the linear projection layers and the LoRA adapters applied to the transformer backbone. The resulting model supports controllable slow-motion synthesis under text, image, and speed conditions.
4.2
Extreme Temporal Super-Resolution
Finally, we apply our data to the task of temporal super-resolution, where the goal is to transform low-FPS, blurry videos into their high-FPS, clear counterparts. While previous approaches typically assume clean and sharp input frames [14, 43, 55], we focus on a more practical and challenging setting in which the input may have motion blur. Low-FPS videos naturally exhibit stronger motion blur due to longer exposure times, making the reconstruction of high velocity details significantly harder.
7
Interpolated timesteps
Input timestep 2
Ours
Wan2.1
Input timestep 1
Figure 5 Temporal super-resolution qualitative results. We compare against the strongest baseline, Wan2.1, under the Blurred-input setting. Given blurry input frames, the baseline produces blurry intermediate frames, whereas our method generates sharp frames with smooth motion, closely resembling footage captured by a high-speed camera. Inset zooms are provided for clearer visual comparison.
Our key idea is to use our large-scale slow-motion dataset to synthesize a wide range of low-FPS, motion blurred videos for training. Following [8, 38, 48], we generate synthetic blurry inputs by averaging a temporally centered window of 8 frames, followed by temporal subsampling to mimic low-frame-rate capture. We build our model upon Wan2.1-VACE [28] for its flexible conditioning capability. The VACE framework takes a reference video and an arbitrary binary mask that specifies the frames or regions to be generated. We fine-tune LoRA adapters on top of the pretrained base model. By training on these paired data, the model can jointly perform motion deblurring and frame interpolation, faithfully capturing real-world shooting conditions. Notably, it also achieves superior performance even in the traditional setting with clean, non-blurry inputs. Due to computational resource constraints, we focus on 8-times upsampling.
5
Experiments
In this section, we evaluate how well our proposed techniques perceive and manipulate the flow of time. We first benchmark our speed-change detector and playback-speed estimator, and then showcase that models trained on our dataset exhibit improved speed controllability and temporal super-resolution capabilities.
5.1
Speed Change Detection
Data. We evaluate our model on the test split of our curated speed-change data. Because these labels are
automatically derived from audio cues and may contain false positives, we have four annotators re-label them and keep only samples where human annotations and audio-based labels agree. We sample 2-second clips from test videos. If the speed change occurs between 13 and 23 of the clip, we treat it as a positive sample; if the speed change occurs elsewhere, or if no speed change is present, we treat it as a negative sample. Baselines. We compare against a SOTA VideoLLM, Gemini 2.5 [12], as well as a flow-based detector. For
the latter, we adopt SEA-RAFT [57] to compute flow magnitudes between adjacent frames, smooth the magnitudes with a moving window of size 5, and classify a clip as positive if the magnitude within its middle third changes by more than a threshold determined based on a held-out set. Results. Our model achieves the highest test accuracy (92.4%), outperforming both Gemini 2.5 (59.5%) and the
flow-based baseline (80.4%). We further apply our model to an iconic movie scene in X-Men with time-freeze effects (first row of figure 1). We visualize model predictions alongside movie frames at corresponding timesteps. To show motion intensity at each moment, we blend each frame with its neighbors. Here, time accelerates from slow motion to normal speed, causing initially sharp frames to become blurry. Our model accurately identifies this transition moment, demonstrating its potential for video forensics.
8
Method
DAVIS
SloMo-44K-Test
FloLPIPS↓
LPIPS↓
FID↓
FVD↓
FloLPIPS↓
LPIPS↓
FID↓
FVD↓
FILM [43] LDMVFI [14] GI [55] Wan2.1 [28]
0.252 0.307 0.353 0.316
0.200 0.251 0.234 0.260
20.7 31.6 21.4 24.7
711.3 916.0 552.1 634.5
0.087 0.139 0.124 0.108
0.066 0.113 0.093 0.078
10.7 22.7 14.9 16.6
257.6 443.0 503.6 594.0
Ours
0.242
0.203
18.2
394.0
0.078
0.061
10.9
182.2
Table 3 Temporal super-resolution from clear inputs. Following prior work on frame interpolation [14, 43, 55], we set aside the motion blur typical of low-FPS capture and evaluate under clean inputs. Specifically, we obtain low-FPS inputs by 8× temporal subsampling and task models with reconstructing the original videos. Finetuned on SloMo-44K, our model outperforms prior methods on video-based metrics (i.e., FloLPIPS, FVD) and matches or achieves the best results on image-based metrics (i.e., LPIPS, FID). Best results are in bold and second-best are underlined.
Method
FloLPIPS↓
LPIPS↓
FID↓
FVD↓
FILM [43] LDMVFI [14] GI [55] Wan2.1 [28]
0.099 0.136 0.123 0.126
0.080 0.117 0.108 0.095
20.3 31.4 26.1 28.5
250.0 340.4 439.4 436.1
Ours
0.067
0.058
12.4
134.3
Table 4 Temporal super-resolution results on motion-blurred videos (SloMo-44K-Test). Our method achieves the best performance across all metrics, demonstrating robust temporal super-resolution under motion blur, a common artifact in real-world low-FPS videos.
5.2
Playback Speed Estimation
Data. We collect 111 videos with verified playback speeds by reviewing online sources that explicitly state the
playback rate in the video title or description. Metrics. We evaluate the quality of speed predictors using the following metrics: (1) Pearson correlation
coefficient ρ, which measures the linear relationship between predictions and ground truth; (2) Spearman’s rank correlation rs , which evaluates the correctness of their ordinal relationships; and (3) root mean squared error (RMSE). We transform playback speeds to log space to ensure scale invariance. For interpretability, we also report eRMSE , which reflects the average multiplicative deviation between predictions and ground truth in linear space. Baselines. We consider four baselines: Gemini 2.5 [12], SpeedNet [6], Pulse-of-Motion [19], optical flow
magnitude, and expert human judgment. For Gemini, we increase its sampling rate to 8 FPS to better capture temporal cues and prompt it to estimate playback speed. For the flow-based baseline, we use SEA-RAFT [57] to compute the optical flow between consecutive frames and then aggregate the flow magnitude by averaging it across the entire video. Since flow magnitude is only used as a relative, but not an absolute, measure of speed, we do not report absolute error metrics (e.g., RMSE). For human evaluation, we design an interface that allows annotators to interactively adjust playback rate until the perceived motion feels like the real-world speed. In practice, it takes on average ∼40 seconds to annotate the speed of a video. Results. As shown in table 2, our method significantly outperforms Gemini, SpeedNet, and the optical-flow
baseline, and substantially narrows the gap between human and model performance.
5.3
Speed-conditioned Video Generation
Setup. We set the playback speeds of our model directly via control values of 1, 4, 7, and 10, which corresponds
to increasingly faster motion. We then utilize 56 image–prompt pairs from the test split of SloMo-44K, spanning diverse scenes and motion patterns, for evaluation.
9
(b) Temporal super-resolution: Ours vs. Baselines
(a) Speed condition vs. flow
Figure 6 (a) Speed condition vs. optical flow. Our speed control strongly correlates with the average optical flow magnitude of videos generated under different speed conditions, whereas the baselines yield similar motion magnitudes for slow and ultra-slow speeds. (b) User study on temporal super-resolution. The results show that human users consistently prefer videos generated by our model (blue) over those generated by baselines (red) across all settings.
Method
FID↓
FVD↓
Training data
FID↓
FVD↓
ATI [52] Wan2.1 [50] Ours
73.4 72.2 68.4
1473.5 1266.8 1114.1
Standard videos SloMo-44K
72.4 68.4
1392.9 1114.1
Table 6 Speed-conditioned video generation with different training data. We train our video generation model on either SloMo-44K or standard-FPS datasets. Results show that training on SloMo-44K yields superior slow-motion generation quality, confirming the importance of high-frame-rate data.
Table 5 Speed-controlled video generation results. We compare our video model trained on SloMo-44K with the pretrained Wan2.1 [50] and the trajectory-based model, ATI [52]. Our model demonstrates superior quality in slow-motion generation.
Metrics. To measure speed controllability, we compute the average optical flow magnitude for each speed
condition and examine whether the resulting magnitudes follow the expected speed ordering. To assess video generation quality, we compute FID [22] and FVD [49] on all 48 text-image–speed triplets from the evaluation set that have corresponding ground-truth videos. Baselines. We compare against the base model, Wan2.1 [50], and a trajectory-based motion control model, ATI [52]. Since Wan2.1 does not natively support playback speed control, we approximate different speeds using prompt modifiers (e.g., “ultra slow-motion,” “slow-motion,” “slightly slow-motion,” and “normal”). For the trajectory-based baseline, we first generate a normal-speed video using Wan2.1 and extract its dense tracklets using CoTracker3 [29]. We then linearly interpolate these tracklets to obtain motion trajectories that are 1/4/16/64 times slower and provide them to ATI as trajectory conditions. Results. We report FID and FVD in table 5, and optical flow analysis in figure 6(a). Our model achieves
higher slow-motion generation quality while exhibiting superior speed controllability. As shown qualitatively in figure 4, videos generated with smaller playback-speed values appear slower than those produced with larger values, indicating that the conditioning signal is effectively learned.
5.4
Extreme Temporal Super-Resolution
Setup. Evaluation is performed on both DAVIS [41] and our SloMo-44K under three settings: (1) Clear-input:
on both datasets, we subsample every 8th frame and have the model interpolate frames in between; (2) Blurred-input: we synthesize motion blur in low-FPS test videos following the same procedure described in section 4.2, then have the models reconstruct the original clear SloMo-44K videos; (3) Real-input: Since DAVIS videos are normal speed and already contain strong motion blur, we have the models convert them to
10
Method VideoLLM [12] Ours
IP
ρ↑
rs ↑
RMSE ↓
Training data
✓
0.426 0.552
0.308 0.479
1.568 1.221
Standard Videos
✓
0.680 0.735
0.684 0.706
0.917 0.649
SloMo-44K
Table 7 Ablation on iterative prediction (IP). We conduct an ablation study on the IP mechanism. The results show that the inclusion of IP consistently improves performance for both VideoLLM and our self-supervised model, demonstrating its robustness in enhancing speed prediction accuracy.
IP
ρ↑
rs ↑
RMSE ↓
✓
0.242 0.632
0.223 0.598
1.232 0.737
✓
0.680 0.735
0.684 0.706
0.917 0.649
Table 8 Playback-speed prediction using different training data. We train the speed predictor using either SloMo-44K or only standard datasets (Adobe240fps [48] and normal-speed videos). The results show that exposing the model to a broad range of temporal scales improves its speed prediction capability.
8× slow motion, testing their ability to enhance real-world, low-FPS footage. Since ground-truth slow motion is unavailable, we conduct a user study for evaluation. Baselines. We benchmark our method against four interpolation models: FILM [43], LDMVFI [14], Generative
Inbetween [55], and vanilla Wan2.1-VACE [28]. For models that generate only a single intermediate frame, we recursively apply them to achieve 8× temporal super-resolution. Metrics. We report standard temporal super-resolution metrics (e.g., LPIPS [66], FID [22], FloLPIPS [13],
and FVD [49]) along with human preference rates. For the user study, human raters are presented with two videos in each trial, one from our model and the other from a baseline, and asked to select the better one. Results. We show the quantitative comparison under the Clean-input setting in table 3, those under the Blurred-input setting in table 4, the user study results across all settings in figure 6(b), and qualitative results in figure 5. Our model consistently outperforms all baselines and achieves over 90% user preference in the Real-input setting (figure 6(b)). These results demonstrate our model’s superior ability to model real-world dynamics at high temporal resolution.
5.5
Analysis
In this section, we conduct ablation studies to examine the impact of iterative prediction on speed estimation and to assess the importance of using SloMo-44K to train both speed predictors and speedconditioned video generation models. Importance of Iterative prediction. The ablation
study in table 7 shows that IP enhances the accuracy of both our model and VideoLLM, confirming the technique’s robustness across models.
Figure 7 Speed-controlled video generation with different training data. Training on standard-FPS videos with artificial slowdowns leads to stuttering artifacts, while training on SloMo-44K results in realistic slow-motion dynamics. Videos are embedded. Please view with Adobe Acrobat Reader.
Importance of SloMo-44K. Is SloMo-44K neces-
sary for training our proposed models? To answer this question, we first compare our speed predictor (trained on SloMo-44K) with a baseline trained solely on standard datasets (Adobe240fps [48] and normal-speed videos). The result in table 8 shows that models exposed to a wider variety of speeds outperform those trained only on standard videos. We further evaluate our speed-conditioned model against a baseline trained on videos artificially slowed from standard-FPS footage, following [23]. As shown in table 6 and figure 7, the baseline exhibits stuttering artifacts, whereas our model faithfully synthesizes slow-motion dynamics.
11
6
Discussion
In this work, we showcase how we can leverage the multimodal cues, temporal structure inherently present in videos, and the vast amount of slow motion videos available on the internet to develop models that can perceive and manipulate the flow of time. While we achieve state-of-the-art performance across both understanding and generation tasks, we still encounter several limitations. For instance, our speed understanding models can be misled when videos contain limited motion cues or when people deliberately move slowly; our generation models also rely on a pretrained Wan backbone, leaving room for improvement through architectural innovations or full fine-tuning. We believe this work establishes time as a manipulable dimension in video learning and opens new directions for temporal forensics and richer world models that capture how events unfold.
Acknowledgment The research is partially supported by a gift from Ai2, NVIDIA Academic Grant, DARPA TIAMAT program No. HR00112490422, and the National Science and Technology Council via grants NSTC 114-2634-F-002-006 and NSTC 114-2640-E-002-006. Its contents are solely the responsibility of the authors and do not necessarily represent the official views of DARPA. We also thank Xuyi Meng, Chuanruo Ning, and Guangzhao He for their valuable contributions and discussions, and the National Center for High-performance Computing (NCHC) for providing computational and storage resources.
Author Contributions Yen-Siang Wu led the project, designed and curated the SloMo-44K dataset, proposed the core methodology, and conducted the primary experiments and manuscript drafting. Rundong Luo helped data curation,
contributed to the design, training, and evaluation of the video generation models, performed the literature review and manuscript drafting. Jingsen Zhu executed the preliminary experiments that informed the development of the proposed approach. Tao Tu managed the user studies, prepared the visual demonstrations, and assisted in proofreading.
References [1] Alisson Azzolini, Junjie Bai, Hannah Brandon, Jiaxin Cao, Prithvijit Chattopadhyay, Huayu Chen, Jinju Chu, Yin Cui, Jenna Diamond, Yifan Ding, et al. Cosmos-reason1: From physical common sense to embodied reasoning. arXiv preprint arXiv:2503.15558, 2025. [2] Piyush Bagad, Makarand Tapaswi, and Cees GM Snoek. Test of time: Instilling video-language models with a sense of time. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2503–2516, 2023. [3] Sherwin Bahmani, Ivan Skorokhodov, Guocheng Qian, Aliaksandr Siarohin, Willi Menapace, Andrea Tagliasacchi, David B Lindell, and Sergey Tulyakov. Ac3d: Analyzing and improving 3d camera control in video diffusion transformers. In CVPR, 2025. [4] Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report. arXiv:2502.13923, 2025. [5] Max Bain, Arsha Nagrani, Gül Varol, and Andrew Zisserman. Frozen in time: A joint video and image encoder for end-to-end retrieval. In ICCV, 2021. [6] Sagie Benaim, Ariel Ephrat, Oran Lang, Inbar Mosseri, William T Freeman, Michael Rubinstein, Michal Irani, and Tali Dekel. Speednet: Learning the speediness in videos. In CVPR, 2020. [7] Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram Voleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv:2311.15127, 2023. [8] Tim Brooks and Jonathan T Barron. Learning to synthesize motion blur. In CVPR, 2019.
12
[9] Ryan Burgert, Yuancheng Xu, Wenqi Xian, Oliver Pilarski, Pascal Clausen, Mingming He, Li Ma, Yitong Deng, Lingxiao Li, Mohsen Mousavi, et al. Go-with-the-flow: Motion-controllable video diffusion models using real-time warped noise. In CVPR, 2025. [10] Jiaben Chen and Huaizu Jiang. Sportsslomo: A new benchmark and baselines for human-centric video frame interpolation. arXiv:2308.16876, 2023. [11] Tsai-Shien Chen, Aliaksandr Siarohin, Willi Menapace, Ekaterina Deyneka, Hsiang-wei Chao, Byung Eun Jeon, Yuwei Fang, Hsin-Ying Lee, Jian Ren, Ming-Hsuan Yang, et al. Panda-70m: Captioning 70m videos with multiple cross-modality teachers. In CVPR, 2024. [12] Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, et al. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. arXiv:2507.06261, 2025. [13] Duolikun Danier, Fan Zhang, and David Bull. Flolpips: A bespoke video quality metric for frame interpolation. In Picture Coding Symposium, 2022. [14] Duolikun Danier, Fan Zhang, and David Bull. Ldmvfi: Video frame interpolation with latent diffusion models. In AAAI, 2024. [15] Xi Ding and Lei Wang. Do language models understand time? In Companion Proceedings of the ACM on Web Conference 2025, pages 1855–1868, 2025. [16] Michael Dorkenwald, Fanyi Xiao, Biagio Brattoli, Joseph Tighe, and Davide Modolo. Scvrl: Shuffled contrastive video representation learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4132–4141, 2022. [17] Yang Du, Yuqi Liu, and Qin Jin. Reversed in time: A novel temporal-emphasized benchmark for cross-modal video-text retrieval. In Proceedings of the 32nd ACM International Conference on Multimedia, pages 5260–5269, 2024. [18] Haiwen Feng, Zheng Ding, Zhihao Xia, Simon Niklaus, Victoria Abrevaya, Michael J Black, and Xuaner Zhang. Explorative inbetweening of time and space. In ECCV, 2024. [19] Xiangbo Gao, Mingyang Wu, Siyuan Yang, Jiongze Yu, Pardis Taghavi, Fangzhou Lin, and Zhengzhong Tu. The pulse of motion: Measuring physical frame rate from visual dynamics. arXiv preprint arXiv:2603.14375, 2026. [20] Amir Ghodrati, Efstratios Gavves, and Cees GM Snoek. Video time: Properties, encoders and evaluation. arXiv preprint arXiv:1807.06980, 2018. [21] Chenlong He, Qi Zheng, Ruoxi Zhu, Xiaoyang Zeng, Yibo Fan, and Zhengzhong Tu. Cover: A comprehensive video quality evaluator. In CVPRW, 2024. [22] Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. In NeurIPS, 2017. [23] Zhening Huang, Hyeonho Jeong, Xuelin Chen, Yulia Gryaditskaya, Tuanfeng Y Wang, Joan Lasenby, and ChunHao Huang. Spacetimepilot: Generative rendering of dynamic scenes across space and time. arXiv:2512.25075, 2025. [24] Zhewei Huang, Tianyuan Zhang, Wen Heng, Boxin Shi, and Shuchang Zhou. Real-time intermediate flow estimation for video frame interpolation. In ECCV, 2022. [25] Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianxing Wu, Qingyang Jin, Nattapol Chanpaisit, et al. Vbench: Comprehensive benchmark suite for video generative models. In CVPR, 2024. [26] Siddhant Jain, Daniel Watson, Eric Tabellion, Ben Poole, Janne Kontkanen, et al. Video interpolation with diffusion models. In CVPR, 2024. [27] Huaizu Jiang, Deqing Sun, Varun Jampani, Ming-Hsuan Yang, Erik Learned-Miller, and Jan Kautz. Super slomo: High quality estimation of multiple intermediate frames for video interpolation. In CVPR, 2018. [28] Zeyinzi Jiang, Zhen Han, Chaojie Mao, Jingfeng Zhang, Yulin Pan, and Yu Liu. Vace: All-in-one video creation and editing. In ICCV, 2025.
13
[29] Nikita Karaev, Yuri Makarov, Jianyuan Wang, Natalia Neverova, Andrea Vedaldi, and Christian Rupprecht. Cotracker3: Simpler and better point tracking by pseudo-labelling real videos. In ICCV, 2025. [30] Hamed Kiani Galoogahi, Ashton Fagg, Chen Huang, Deva Ramanan, and Simon Lucey. Need for speed: A benchmark for higher frame rate object tracking. In ICCV, 2017. [31] Hsin-Ying Lee, Jia-Bin Huang, Maneesh Singh, and Ming-Hsuan Yang. Unsupervised representation learning by sorting sequences. In Proceedings of the IEEE international conference on computer vision, pages 667–676, 2017. [32] KunChang Li, Yinan He, Yi Wang, Yizhuo Li, Wenhai Wang, Ping Luo, Yali Wang, Limin Wang, and Yu Qiao. Videochat: Chat-centric video understanding. arXiv:2305.06355, 2023. [33] Bin Lin, Yang Ye, Bin Zhu, Jiaxi Cui, Munan Ning, Peng Jin, and Li Yuan. Video-llava: Learning united visual representation by alignment before projection. In EMNLP, 2024. [34] Ziwei Liu, Raymond A Yeh, Xiaoou Tang, Yiming Liu, and Aseem Agarwala. Video frame synthesis using deep voxel flow. In ICCV, 2017. [35] Rundong Luo, Matthew Wallingford, Ali Farhadi, Noah Snavely, and Wei-Chiu Ma. Beyond the frame: Generating 360° panoramic videos from perspective videos. In ICCV, 2025. [36] Alexandre Mercat, Marko Viitanen, and Jarno Vanne. Uvg dataset: 50/120fps 4k sequences for video codec analysis and development. In Proceedings of the 11th ACM multimedia systems conference, pages 297–302, 2020. [37] Ishan Misra, C Lawrence Zitnick, and Martial Hebert. Shuffle and learn: unsupervised learning using temporal order verification. In European conference on computer vision, pages 527–544. Springer, 2016. [38] Seungjun Nah, Tae Hyun Kim, and Kyoung Mu Lee. Deep multi-scale convolutional neural network for dynamic scene deblurring. In CVPR, 2017. [39] Kepan Nan, Rui Xie, Penghao Zhou, Tiehan Fan, Zhenheng Yang, Zhijie Chen, Xiang Li, Jian Yang, and Ying Tai. Openvid-1m: A large-scale high-quality dataset for text-to-video generation. arXiv:2407.02371, 2024. [40] Simon Niklaus, Long Mai, and Feng Liu. Video frame interpolation via adaptive convolution. In CVPR, 2017. [41] F. Perazzi, J. Pont-Tuset, B. McWilliams, L. Van Gool, M. Gross, and A. Sorkine-Hornung. A benchmark dataset and evaluation methodology for video object segmentation. In CVPR, 2016. [42] Lyndsey C Pickup, Zheng Pan, Donglai Wei, YiChang Shih, Changshui Zhang, Andrew Zisserman, Bernhard Scholkopf, and William T Freeman. Seeing the arrow of time. In CVPR, 2014. [43] Fitsum Reda, Janne Kontkanen, Eric Tabellion, Deqing Sun, Caroline Pantofaru, and Brian Curless. Film: Frame interpolation for large motion. In ECCV, 2022. [44] Hyeonjun Sim, Jihyong Oh, and Munchurl Kim. Xvfi: extreme video frame interpolation. In ICCV, 2021. [45] Enxin Song, Wenhao Chai, Guanhong Wang, Yucheng Zhang, Haoyang Zhou, Feiyang Wu, Haozhe Chi, Xun Guo, Tian Ye, Yanting Zhang, et al. Moviechat: From dense token to sparse memory for long video understanding. In CVPR, 2024. [46] Tomás Soucek and Jakub Lokoc. Transnet v2: An effective deep network architecture for fast shot transition detection. In ACM MM, 2024. [47] Verena Steinhof, Anna Schroeger, Roman Liepelt, and Laura Sperl. Time and video speed perception: a comprehensive investigation of the relation between estimated video speed, clip duration and original duration: V. steinhof et al. Cognitive Research: Principles and Implications, 2025. [48] Shuochen Su, Mauricio Delbracio, Jue Wang, Guillermo Sapiro, Wolfgang Heidrich, and Oliver Wang. Deep video deblurring for hand-held cameras. In CVPR, 2017. [49] Thomas Unterthiner, Sjoerd Van Steenkiste, Karol Kurach, Raphaël Marinier, Marcin Michalski, and Sylvain Gelly. Fvd: A new metric for video generation. In ICLRW, 2019. [50] Team Wan, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, et al. Wan: Open and advanced large-scale video generative models. arXiv:2503.20314, 2025. [51] Team Wan, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, et al. Wan: Open and advanced large-scale video generative models. arXiv:2503.20314, 2025.
14
[52] Angtian Wang, Haibin Huang, Zhiyuan Fang, Yiding Yang, and Chongyang Ma. ATI: Any trajectory instruction for controllable video generation. arXiv:2505.22944, 2025. [53] Jiangliu Wang, Jianbo Jiao, and Yun-Hui Liu. Self-supervised video representation learning by pace prediction. In ECCV, 2020. [54] Limin Wang, Bingkun Huang, Zhiyu Zhao, Zhan Tong, Yinan He, Yi Wang, Yali Wang, and Yu Qiao. Videomae v2: Scaling video masked autoencoders with dual masking. In CVPR, 2023. [55] Xiaojuan Wang, Boyang Zhou, Brian Curless, Ira Kemelmacher-Shlizerman, Aleksander Holynski, and Steven M Seitz. Generative inbetweening: Adapting image-to-video models for keyframe interpolation. In ICLR, 2024. [56] Yi Wang, Yinan He, Yizhuo Li, Kunchang Li, Jiashuo Yu, Xin Ma, Xinhao Li, Guo Chen, Xinyuan Chen, Yaohui Wang, et al. Internvid: A large-scale video-text dataset for multimodal understanding and generation. arXiv preprint arXiv:2307.06942, 2023. [57] Yihan Wang, Lahav Lipson, and Jia Deng. Sea-raft: Simple, efficient, accurate raft for optical flow. In ECCV, 2024. [58] Yiming Wang, Qihang Zhang, Shengqu Cai, Tong Wu, Jan Ackermann, Zhengfei Kuang, Yang Zheng, Frano Rajič, Siyu Tang, and Gordon Wetzstein. Bullettime: Decoupled control of time and camera pose for video generation. arXiv:2512.05076, 2025. [59] Zhenhailong Wang, Ansel Blume, Sha Li, Genglin Liu, Jaemin Cho, Zineng Tang, Mohit Bansal, and Heng Ji. Paxion: Patching action knowledge in video-language foundation models. Advances in Neural Information Processing Systems, 36:20729–20749, 2023. [60] Zhouxia Wang, Ziyang Yuan, Xintao Wang, Yaowei Li, Tianshui Chen, Menghan Xia, Ping Luo, and Ying Shan. Motionctrl: A unified and flexible motion controller for video generation. In SIGGRAPH, 2024. [61] Donglai Wei, Joseph J Lim, Andrew Zisserman, and William T Freeman. Learning and using the arrow of time. In CVPR, 2018. [62] Xiaoyu Xiang, Yapeng Tian, Yulun Zhang, Yun Fu, Jan P Allebach, and Chenliang Xu. Zooming slow-mo: Fast and accurate one-stage space-time video super-resolution. In CVPR, 2020. [63] Zihui Xue, Mi Luo, and Kristen Grauman. Seeing the arrow of time in large multimodal models. arXiv preprint arXiv:2506.03340, 2025. [64] Yuan Yao, Chang Liu, Dezhao Luo, Yu Zhou, and Qixiang Ye. Video playback rate perception for self-supervised spatio-temporal representation learning. In CVPR, 2020. [65] Maoyuan Ye, Jing Zhang, Shanshan Zhao, Juhua Liu, Bo Du, and Dacheng Tao. Dptext-detr: Towards better scene text detection with dynamic points in transformer. In AAAI, 2023. [66] Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In CVPR, 2018. [67] Dian Zheng, Ziqi Huang, Hongbo Liu, Kai Zou, Yinan He, Fan Zhang, Yuanhan Zhang, Jingwen He, Wei-Shi Zheng, Yu Qiao, and Ziwei Liu. VBench-2.0: Advancing video generation benchmark suite for intrinsic faithfulness. arXiv:2503.21755, 2025. [68] Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Hao Tian, Yuchen Duan, Weijie Su, Jie Shao, et al. Internvl3: Exploring advanced training and test-time recipes for open-source multimodal models. arXiv:2504.10479, 2025.
15
A
Supplementary Material Overview
In this supplementary document, we provide additional implementation details and present extended qualitative results. For video demonstrations, please visit the project page. (b) Objects
(a) Sample frames
(c) Motions
(d) Video lengths
Figure 8 Dataset overview. (a) Sample frames illustrating the dataset’s contextual diversity; (b) Word cloud of nouns from generated captions, capturing a broad range of objects and scenes; (c) Word cloud of verbs, reflecting diverse human and object motions; (d) Histogram of video lengths, showing variation in temporal scales.
B
Dataset Details
In this section, we provide additional details on the curation and statistical composition of the SloMo-44K dataset.
B.1
Dataset Curation
We expand upon the data curation pipeline introduced in Sec. 3.3 of the main paper. Video Acquisition and Generic Filtering. We collect videos from YouTube, Vimeo, and Flickr using queries such
as “high frame rate,” “high-speed camera,” and “slow motion,” along with related synonyms. Following [7, 35], we apply TransNetv2 [46] for shot segmentation and use an OCR model [65] to remove clips with excessive text overlays. To further ensure quality, we filter out CGI and screen-captured content using Qwen2.5-VL [4], and discard low-quality videos based on VQA metrics [21]. Slow-Motion–Focused Processing. Because many online videos exhibit heterogeneous or unspecified playback
speeds, we employ our speed-change detector (main paper Sec. 3.1) to segment each video into clips with homogeneous playback rates. However, these segments are not necessarily slow motion; without additional filtering, the dataset would remain dominated by standard 1.0× content. We therefore introduce a dedicated slow-motion video detection approach to identify and retain only clips that exhibit reliably slow-motion characteristics. See the next section for implementation details. Annotation. Finally, we annotate each clip with a predicted playback speed using our speed estimator. We
also generate dense captions—including short and long descriptions, and attributes such as background, style, shot type, lighting, and atmosphere, to capture both semantic and aesthetic aspects of the scene, using InternVL3 [68].
B.2
Slow-Motion Video Classification
As discussed above, clips extracted from raw videos are not guaranteed to contain slow motion. Without further filtering, the dataset would be dominated by normal-speed content. We therefore develop a dedicated slow-motion filtering pipeline to identify genuine slow-motion clips. 16
Figure 9 Histogram of pseudo-speed annotations. SloMo-44K spans a wide spectrum of playback speeds. Approach. To filter slow-motion videos, we adopt two complementary strategies: (1) We query a VideoLLM with each original video (as downloaded online) and prompt it to localize slow-motion segments. (2) We train a ViT-based video classifier on a small set of human annotations to predict whether a 2-second clip is in slow motion. Because VideoLLMs leverage global semantics and contextual cues, while ViT-based classifiers specialize in fine-grained visual patterns, we combine both predictions in a two-stage filtering pipeline. Implementation Details. In practice, we use Gemini 2.5 as the VideoLLM and a fine-tuned VideoMAEv2 as
the clip-level classifier. To obtain training data, eight annotators label 2,400 clips as slow motion, not slow motion, or unknown, using all available information including video titles and descriptions. Clips labeled unknown are discarded, and the remaining ones are used for training and evaluation. Results. On our curated validation set, Gemini achieves 74.8% accuracy, while the finetuned classifier reaches
84.4%. To leverage the complementary strengths of both models, we combined their predictions. We discard clips if either: (1) Gemini identifies less than 10% of its content as slow motion, or (2) the video classifier assigns a slow-motion probability below 0.998. This combined filtering achieves 98% precision and 44% recall in retaining slow-motion clips.
B.3
Dataset Statistics
figure 8 presents an overview of SloMo-44K, including sample frames, two word clouds of nouns and verbs extracted from the generated video captions, and a histogram of video lengths. figure 9 shows the histogram of pseudo-speed annotations. As shown, the dataset covers a diver range of scenes, objects, motions, lengths, and temporal scales, making it well-suited for learning all sorts of real-world motions and training generic video generation models.
C
Experimental Details
In this section, we provide additional experimental details, including more qualitative results, ablation studies, and information on the user studies.
C.1
Speed Estimation
Equivariance in Speed Estimation. A key concept underlying our playback speed estimator is equivariance,
which we elaborate on here. Consider an ideal estimator fθ that perfectly predicts the playback speed of any video. For a video V with playback speed xV , accelerating it by a factor of k (via temporal subsampling) yields a new video whose effective playback speed becomes k · xV . Let “·” denote temporal acceleration so that k · V represents the k-times–sped-up version of V. Then a perfect speed estimator should satisfy: fθ (k · V) = k · fθ (V),
(4)
which mirrors the definition of equivariance. We refer to this property as the equivariance of speed estimation accordingly. 17
(a) 1st Iteration
(b) 2nd Iteration
(c) 3rd Iteration
(d) 4th Iteration
(e) 5th Iteration
Figure 10 Ablation on iterative prediction. We vary the number of iterations and show the corresponding speed prediction. As shown, performance improves notably by the third iteration, while additional iterations yield minimal gains, showing the predictions have converged. Ablation Study: Iterative Prediction. We study how iterative prediction affect speed estimates in figure 10. The
first iteration produces relatively inaccurate predictions, particularly for extremely slow videos. By the third iteration, predictions are substantially improved, demonstrating the benefit of iterative prediction. Additional iterations (up to five) result in negligible changes, indicating the predictions have converged. Based on these findings, we adopt three iterations in practice to balance accuracy and computation. Human Annotation Interface. We show a screenshot of our human annotation interface in figure 11. The
interface is designed to fully leverage human perceptual sensitivity when judging playback speed.
C.2
Speed-conditioned Video Generation
Training Details. To prevent any speed bucket from dominating the training distribution, we balance the
sampling probability across all buckets. We finetune Wan2.1-I2V-14B-480P [50] on 4 GB200 GPUs for two days, using a learning rate of 1 × 10−5 for the speed-conditioning modules and 1 × 10−4 for the LoRA adapters. Evaluation Details. As discussed in Sec. 5.3., we generate 224 videos per baseline during evaluation (56
text-image pairs, each evaluated at four different speed settings). However, when evaluating slow-motion generation quality, we cannot compute FID and FVD across all text–image–speed combinations, since each text-image pair in the evaluation set is associated with only a single speed. In fact, only 48 image–text–speed triplets in the evaluation set have corresponding ground-truth videos at matching speeds. The remaining triplets either contain too few frames after speeding up to the target speed or have frame rates that are too low to slow down to the desired speed. Consequently, we use this subset of 48 triplets to compute FID and FVD scores for overall generation quality. For completeness, we also report VBench [25] results here. Nonetheless, VBench does not accurately capture slow-motion generation quality, as it (1) is designed to evaluate standard-speed videos and (2) primarily assesses whether the video appears visually convincing instead of whether it follows real-world principles [67]. Results. We provide additional qualitative comparisons between our model and baseline methods in figure 13
and additional VBench results in table 9. As shown, our approach produces high-quality videos with clear and consistent speed controllability: videos generated with smaller playback-speed values exhibit noticeably slower motion, whereas larger values yield faster dynamics. In contrast, baseline methods show minimal variation across different speed-control prompt modifiers, indicating limited control over temporal dynamics.
C.3
Temporal Super-Resolution
Training Details. Similar to speed-conditioned video generation, we balance the sampling probability across
all buckets during training. We attach LoRA adapters to Wan2.1-VACE-14B [50] and finetune them for two days on 4 GB200 GPUs, using a learning rate of 1 × 10−4 . We train one model for the traditional Clear-input setting and one model for the Blurred-input or Real-input setting, where low-FPS inputs exhibit stronger motion blur. Results. We present qualitative comparisons between our method and all baselines under the Blurred-input
setting in Fig. 14 and the Real-input setting in Fig. 15. As shown, our approach generates clear and temporally 18
Method Wan2.1
Ours
Aes. ↑ quality
Imag. ↑ quality
0.5384
0.6367
0.5394
0.6337
Motion ↑ smooth. 0.9803
0.9871
Temp. ↑ Flicker. 0.9638
0.9700
Table 9 VBench results for speed-controlled video generation. For completeness, we compare Wan2.1 with our speed-controlled video generation model on VBench metrics. Notably, VBench does not reliably reflect the quality of slow-motion generation because it (1) is designed to evaluate standard-speed videos and (2) primarily assesses whether the video appears visually convincing instead of whether it follows real-world principles [67].
consistent intermediate frames even from heavily blurred inputs, outperforming traditional temporal superresolution methods. We also provide the user study interface for temporal super-resolution in figure 12. For additional qualitative results, please refer to our project page in the supplementary material.
D
Comparison to BulletTime and SpaceTimePilot
Recent concurrent works, BulletTime [58] and SpaceTimePilot [23], also explore time-controlled video generation. However, they differ from our work in several fundamental ways. First, these work focus on video editing, enabling users to manipulate each frame’s timestamp through a relative time-remapping. In contrast, our speed-conditioned framework addresses image-to-video generation conditioned on absolute temporal speed, which requires the model to have knowledge of real-world motion speeds. In other words, BulletTime and SpaceTimePilot modify existing motion on a relative time scale, while our approach synthesizes motion from scratch according to absolute speed conditions. Moreover, our speed-conditioned model is capable of modeling complex dynamics across a broad range of temporal granularities (e.g., up to 100× slow motion), as it is trained on a real-world high-frame-rate video dataset, SloMo-44K. In contrast, BulletTime and SpaceTimePilot are finetuned only on a synthetic video dataset, which limits its ability to model intricate real-world motions at high frame rates (e.g., rapid wing flapping), and primarily supports simple slow-motion effects, such as human body movements. Importantly, beyond time-controlled video generation, our work also investigates speed perception methods, enabling applications in video forensics and leading to the construction of the SloMo-44K dataset.
19
Slow-Motion Speed Guess Study
Video 8 of 28 Trial index: 8
Overview You will see a series of slow-motion video clips. Your task is to determine what speed factor would make each video appear at normal, real-time speed. Use the preview slider to test different playback speeds, then enter your best estimate of the factor in the input box. How to Complete Each Trial 1. Watch the video and use the preview slider (1.0× to 16.0×) to experiment with different speeds 2. Based on your observation, determine what speed factor would make it look normal 3. Enter that factor in the "Speed Factor" input box 4. Click Done to save your answer and move to the next video 5. If you cannot determine a reasonable speed, click Cannot Tell (use sparingly) Important Notes • Please make sure you're using Chrome • Your answer can exceed these limits - enter any value you believe is correct Example Scenarios • If the video looks normal at 4× on the slider → Enter "4" in the input box • If the video appears 2× slow when slider is at 10× → Enter "20" (since 10×2 = 20) Controls
▶ Play
• Play/Stop: Start or pause video playback • Restart: Reset video to beginning
⏱ Restart
Preview Speed:
• Preview Slider: Test different playback speeds
7.2×
• Speed Factor Input: Enter your estimate of the true slow-motion factor • Done: Save your entered value and proceed
What speed factor would make this video look normal?
• Cannot Tell: Use if unable to determine a reasonable speed, e.g., due to lack of motion • Over 16x: Use if the video is still slowed down when the slider is at max (16×)
1.0
Speed Factor:
Saving Your Progress
×
I believe this video is slowed down by > 16×
You can download your progress as a JSON file at any time and resume later by loading it.
Enter the factor that would make this slow-motion video appear at normal speed
Note: Your browser supports playback speeds up to 16×.
Done
Cannot Tell
Start New Evaluation
📁 Load Previous Session (JSON)
← Previous
💾 Download Progress
Next →
Figure 11 Human Annotation interface for video speed prediction. The interface provides a slider that allows annotators to accelerate videos until the motion appears aligned with its real-world progression speed. The final speed estimate is then computed from the selected speed-up factor. This design ensures that human perceptual sensitivity is used to its fullest.
Video Comparison
Slow-Motion Video Evaluation Study
Trial 19 of 36
Instructions: You will be shown pairs of slow-motion video clips generated using
Video A
Video B
different temporal upsampling methods. Both videos depict the same scene and motion but were processed by different algorithms. Please watch both clips carefully, paying attention to the smoothness of motion, realism, and overall visual quality.
Question: Based on the motion and content of the videos, which one appears more realistic or looks closer to a real slow-motion video captured by a high frame-rate camera? Response Options: Video A looks more realistic
▶ Play
Video B looks more realistic
⟲ Replay
0:04
Important: You can download your progress at any time and resume later by
0:05
loading the JSON file.
Which video is better? (more realistic or closer to a real slow-motion video)
Start New Evaluation
Video A is Better
Or resume a previous session:
📁 Load Previous Session (JSON) ← Previous
Next →
Video B is Better
💾 Download Progress
Figure 12 User study interface for temporal super-resolution. Participants are instructed to compare playback speeds based on motion smoothness and temporal details. In each trial, two videos are presented side-by-side and the participants are asked to select the one that appears slower.
20
Ours
Input image Avg. optical flow magnitude -->
Prompt = ”A close-up shot of a person's hand feeding pigeons by a pond.”
0.65
1.10
1.91
7.22
5.08
4.23
4.72
4.43
Baseline
Avg. optical flow magnitude -->
Figure 13 Qualitative comparisons for speed-conditioned video generation. In each group, the first row shows the average image of the generated video, and the second row visualizes the optical flow between the first and fifth frames. For our method, slower speed conditions yield crisp average images (as the scene undergoes minimal change) and low-magnitude optical flow (faint colors). As the conditioned playback speed increases, the average image becomes progressively blurrier and the optical flow magnitude increases (more saturated colors). In contrast, the baseline model produces videos with nearly unchanged temporal dynamics across different speed conditions.
21
Timestep 2
Interpolated timesteps
Ours
Wan2.1
LDMVFI
GI
FILM
Input
Timestep 1
Figure 14 Qualitative comparisons for temporal super-resolution on SloMo-44K (blurred-input). Given two input frames, each method interpolates multiple timesteps. Prior methods produce over-blurred popcorn trajectories and inconsistent boundaries, whereas our method yields smoother motion and sharper, more coherent details under strong motion blur.
22
Timestep 2
Interpolated timesteps
Ours
Wan2.1
LDMVFI
GI
FILM
Input
Timestep 1
Figure 15 Qualitative comparison for temporal super-resolution on DAVIS (real-input). Our method produces smoother motion trajectories and sharper fine-grained details than traditional temporal super-resolution approaches (e.g., see the girl’s arms). These results highlight the benefit of explicitly synthesizing motion blur for low-FPS inputs and training on our curated slow-motion dataset.
23