CausalChapter: Improving Long-Video Chaptering with Interventional Dependency Modeling Xinran Duan, Guozhang Li, Yaoyao Zhong, Mei Wang* , Lizhi Wang, Hua Huang School of Artificial Intelligence, Beijing Normal University Beijing Key Laboratory of Artificial Intelligence for Education Engineering Research Center of Intelligent Technology and Educational Application, Ministry of Education [email protected], {liguozhang097, zhongyy}@bnu.edu.cn {wangmei1, wanglizhi, huahuang}@bnu.edu.cn Abstract
Iashin and Rahtu, 2020a; Wang et al., 2021; Yang et al., 2023b). However, DVC mainly focuses on short-term event-level segments, whereas longform video navigation requires higher-level temporal organization. Video chaptering addresses this need by partitioning a long video into consecutive chapters and generating navigable titles or descriptions for each chapter (Yang et al., 2023a; Ventura et al., 2025). Recent progress in long-video chaptering has been driven by large-scale datasets and longcontext language models. VidChapters-7M introduces a large-scale benchmark of user-annotated chapters for open-domain videos (Yang et al., 2023a). Chapter-Llama converts videos into timestamped Automatic Speech Recognition (ASR) transcripts and frame captions, and predicts chapter boundaries and free-form titles with a long-context LLM in a single forward pass (Ventura et al., 2025). While effective, such holistic long-context modeling faces increasing cost in content-dense instructional videos, where ASR transcripts are long, topic transitions are smooth, and chapter outputs often require detailed descriptions rather than short titles, as illustrated in Fig. 1(a). Under practical context budgets, processing the entire textualized video may require truncation, sparse sampling, or compression (Wang et al., 2021; Yang et al., 2023b; Kim et al., 2024), which may discard fine-grained evidence needed for boundary localization and chapter description generation. A scalable alternative is the segment-thencaption paradigm (Islam et al., 2024; Zala et al., 2023): the model first predicts chapter boundaries to divide a long video into local segments, and then generates a description for each segment. This decomposition reduces the input length of each generation step and aligns the generation input with chapter-level outputs. However, it also shifts the central challenge from holistic encoding to identifying which local transitions and historical segments
arXiv:2609.08686v1 [cs.CV] 8 Sep 2026
Long-form instructional videos require automatic chaptering to support browsing, navigation, and knowledge access. Recent longcontext language models can perform chaptering from textualized video inputs, but they remain costly and brittle for content-dense lecture videos with long transcripts, smooth topic transitions, and detailed chapter outputs. A scalable segment-then-caption paradigm reduces this cost, but introduces two new challenges: boundary error propagation and fragmented crosschapter context. We propose CausalChapter, an intervention-inspired framework for longvideo chaptering that estimates prediction-level influence through lightweight masking and removal interventions. For boundary localization, our Local Dependency Shift module detects drops in predictive dependency between adjacent temporal windows; for chapter description generation, our Cross-Segment Support Selection module reranks historical contexts according to their support for the current prediction. Experiments on long-video chaptering benchmarks show that CausalChapter improves boundary localization, chapter description quality, and cross-chapter coherence. CausalChapter
1
Introduction
Long-form videos, such as lectures, meeting recordings, and online tutorials, have become an important source of knowledge and communication. To make such videos searchable and navigable, models need to organize them into temporally grounded and semantically coherent units. Dense video captioning (DVC) provides a representative localizeand-describe paradigm, where a model detects multiple events in an untrimmed video and generates a description for each event (Krishna et al., 2017; Guozhang Li contributed equally to this work. Corresponding author: Mei Wang.
1
repeat the current content, while an earlier segment with lower lexical or visual similarity may introduce a definition or logical premise that is crucial for describing the current chapter, as shown in Figure 1(c). This suggests a different criterion for longvideo chaptering: a temporal unit should be judged by whether intervening on it changes the model’s prediction, rather than by whether it is visually similar, temporally close, or lexically overlapping with the current segment. The same criterion can also inform boundary localization. Within a coherent chapter, preceding units usually provide predictive support for subsequent units; near a chapter transition, this dependency may drop even when the transition is visually or lexically smooth, as shown in Figure 1(b).
Whole-video vs. Chapter-level Cost
Input: Long-Video + ASR
Input Tokens (K)
... ...
12
AVLecture VidChapters
10.7
ASR: Now could argue ... So we switch ...
5.8
Output: Target Chapters
Whole Video
00:24:31 – 00:37:12 Easy cues cause feature suppression in classical models.
2.4
976
Chapter
Peak VRAM (GB) 90
00:37:12 – 00:49:05 The lecture introduces limitations and transitions to a new topic.
81.7
...
33.4 53.1
Whole Video
24.9
Chapter
(a) Cost comparison of whole-video and chapter-level inputs. Chapter A：Gradient Descent
A1
A2
Loss Function
Gradient Computation
Dependence Score
�
Chapter-Transition
Chapter B: Overfitting
A3
T
B1
B2
Parameter Update
Training to Generalization
Overfitting Introduction
Causes of Overfitting
Dependency drop signals boundaries
(b) Dependency-shift signals motivate boundary localization. Target Chapter: Easy cues cause feature suppression.
… model can reduce the contrastive loss by relying on this easy cue,without learning the harder semantic factor… the loss goes down and the representation looks useful, but the class-level …
Sim. Based RAG
LLM Scoring
Retrieve: DA drop useful information
Retrieve: Easy cues starve harder features
Generate: False negatives drive easy-cue feature suppression Forced Causal Linking
Generate: Wrong view invariance causes easy-cue dominance Coherent Surface Chain
Generate: Easy-cues induce feature suppression Critical Chapter Preserved
Similar Context: ... the loss goes down and the representation looks useful, but the class-relevant …
<think> wrong invariance ignored useful feature easy cue dominates </think>
Motivated by this observation, we propose CausalChapter, an intervention-inspired framework for scalable long-video chapter generation. CausalChapter estimates intervention-defined predictive dependency, namely the observable change in model prediction caused by masking, perturbing, or removing input components. Our goal is not to recover the real-world causal structure of video content, but to measure the prediction-level influence of semantic units and context segments as a practical signal of predictive support. For boundary localization, we introduce Local Dependency Shift (LCDS), which masks semantic units in a preceding temporal window and measures how much the intervention affects the reconstruction of the subsequent window. A local drop in this dependency provides an auxiliary signal for detecting smooth chapter transitions. For chapter description generation, we introduce Cross-Segment Support Selection (CSSE), which removes candidate context segments and measures their influence on the generated description. Segments with high estimated support are selected for second-pass generation, improving the completeness and cross-chapter coherence of the final chapter descriptions.
Ours
Retrieve: False negatives skew classes
Concept Intro. … the model may discover an easy predictive evidence early …
(c) Dependency-aware retrieval preserves predictive cues.
Figure 1: Motivation of scalable and dependency-aware long-video chaptering. (a) Content-dense instructional videos require detailed chapter descriptions over long ASR transcripts, making whole-video modeling costly in tokens and memory. (b) Smooth lecture transitions can preserve local coherence while predictive dependency drops near the true chapter boundary. (c) Plausible retrieved contexts can induce mechanism drift in chapter generation, whereas prediction-critical context preserves the intended explanation.
truly matter for prediction. First, boundary errors may propagate to the generation stage: a shifted boundary can mix adjacent chapter content into the current segment or omit key semantic units. Second, independent segment-level generation can break cross-chapter context, which is often crucial in lectures and tutorials where later chapters depend on earlier definitions, assumptions, or examples. A straightforward solution is to augment each segment with additional context retrieved from nearby or semantically similar segments. Existing retrieval- or memory-augmented methods commonly select context based on semantic similarity, temporal proximity, or visual similarity (Kim et al., 2024, 2025). However, in instructional long videos, surface relevance does not necessarily imply prediction utility. A highly similar segment may simply
Our contributions are summarized as follows: (1) We introduce LCDS, an intervention-inspired dependency-drop signal for smooth chapter boundary localization. (2) We introduce CSSE, an intervention-based support estimation mechanism for cross-segment context selection. (3) We show that the proposed interventional mechanism mitigates boundary error propagation and context fragmentation in scalable long-video chaptering. 2
2
Related Work
lexical or visual similarity, while a highly similar segment may simply repeat the current content. Our work therefore treats retrieval as candidate construction only, and performs final context selection according to each segment’s observable influence on current chapter generation.
Video Chaptering. Video chaptering aims to partition a long video into consecutive, nonoverlapping, and semantically coherent chapters, while generating titles or summaries for browsing and navigation. A related task is dense video captioning (DVC), which localizes and describes multiple events in untrimmed videos (Iashin and Rahtu, 2020a,b; Yang et al., 2023b; Wang et al., 2021; Kim et al., 2024; Liu et al., 2025; Wu et al., 2025; Xie et al., 2025; Li et al., 2025). Recent DVC studies further incorporate memory or retrievalaugmented mechanisms to improve event-level descriptions (Kim et al., 2024; Xie et al., 2025; Wu et al., 2025; Liu et al., 2025). Although DVC follows a localize-and-describe paradigm, it mainly focuses on short-term event-level segments whose boundaries are often associated with local visual or event changes, making it less suited to long-form videos governed by high-level topic progression. For long-form video chaptering, VidChapters7M introduces a large-scale dataset of userannotated chapters and defines several chaptering tasks, including chapter generation and chapter grounding (Yang et al., 2023a). More recently, Chapter-Llama represents long videos as timestamped ASR transcripts and frame captions, and uses a long-context LLM to jointly predict chapter boundaries and free-form chapter titles in a single forward pass (Ventura et al., 2025). These studies demonstrate the effectiveness of textualized video representations and long-context reasoning. Different from holistic long-context chaptering methods, we study a scalable segment-then-caption formulation and explicitly address the boundary error propagation and cross-segment context fragmentation introduced by this decomposition.
Causal Video Reasoning. Causal and counterfactual reasoning has been introduced into video understanding to reduce spurious correlations, mitigate language priors, and model event relations (Xiao et al., 2021; Niu et al., 2021; Liu et al., 2023; Chen et al., 2026). Prior studies construct causal video question answering benchmarks, apply counterfactual interventions for bias reduction, or discover event-level causal structures for video reasoning. Different from these works, we use lightweight masking and removal interventions to estimate prediction-level influence for longvideo chaptering, making our approach closer to intervention-based utility estimation than to causal structure discovery.
3
Method
3.1
Task Formulation and Overview
Given a long video V , the goal is to generate a temporally grounded chapter sequence C = {(sk , ek , yk )}K k=1 , where sk and ek denote the start and end timestamps of the k-th chapter, and yk denotes its chapter-level description. The predicted chapters are expected to be consecutive, nonoverlapping, and semantically coherent. We propose CausalChapter, a segment-thencaption framework for long-video chapter generation. CausalChapter first predicts chapter boundaries and then generates a description for each resulting segment. This decomposition is scalable, but it introduces two key challenges, namely boundary error propagation and cross-segment context fragmentation. Figure 2 summarizes the overall pipeline and the two intervention-based modules used to address these challenges. We address them with two intervention-inspired modules: Local Dependency Shift improves boundary localization by measuring drops in predictive dependency between adjacent temporal windows. CrossSegment Support Selection improves chapter generation by selecting historical segments that provide strong predictive support for the current description. Here, predictive dependency denotes the observable change in model prediction under masking or removal interventions, and serves as
Context Augmentation and Selection. Context augmentation has been widely explored in videolanguage generation through memory propagation, retrieval augmentation, and evidence selection (Kim et al., 2024; Li et al., 2023; Yu et al., 2023; Xie et al., 2025; Wu et al., 2025; Liu et al., 2025; Li et al., 2026). Existing methods commonly identify useful context based on semantic similarity, temporal proximity, cross-modal matching, or attention-based relevance. However, for information-dense lectures, surface-level relevance does not necessarily reflect prediction utility. An earlier definition, assumption, or logical premise may be crucial for the current chapter despite low 3
Input: Long-Video + ASR
Boundary Localization
Semantic Unit Encoding …
𝑢1 ASR Transcript 00:01:12 So we switch … 00:01:26 Now could argue … 00:01:41 And one of t …
dep
LCDS Cue 𝛾 ⋅ 𝑠𝑖 𝜎 Enhanced Score 𝑝𝑖enh
𝑢2 … 𝑢𝑁
Multimodal Encoder 𝑧1
……… …
1 0
𝑧2 … 𝑧𝑁
𝑡
Preceding Window 𝐴𝑖
𝐾 𝑘=1
[00:00:00, 00:09:07] Overview of Weak Interaction and W/Z Feynman Rules
𝑎1
\𝑟 𝐴𝑖
𝑎1
𝑎2
…
𝑎2
… 𝑀𝑎𝑠𝑘 …
𝑎𝑟
…
𝑎𝑊 𝑎𝑊
1. 3 Measured Change 1
Dependency Drop
0
𝑡
𝑧𝑖+𝑊
C𝑘
Following Window 𝐵𝑖
Local Dependency Signal 𝑑𝑖
𝑅𝜙
𝐵𝑖
𝑏1
𝑏2
…
𝑏𝑊
𝑅𝜙
\𝑟 𝐵𝑖
\𝑟 𝑏1
\𝑟 𝑏2 …
\𝑟 𝑏𝑊
𝑆1
𝑆4 𝑆5 Temporal Set
𝑆6
𝑆1
𝑖
dep 𝑝𝑖base + 𝛾 ⋅ 𝑠𝑖 → 𝑝𝑖enh
𝑆2
𝑆3
𝑆4
𝑆5
𝑆6
𝑆1
𝑆2
𝑆3
𝑆4
𝑆5
𝑆6
𝑆1
𝑆2
𝑆3
𝑆4
𝑆5
𝑆6
𝑆1
𝑆2
𝑆3
𝑆4
𝑆5
𝑆6
……
𝑡
𝐺𝜃 𝐺𝜃 𝐺𝜃
𝑦𝑘full
Diff.
\1 𝑦𝑘
𝑆2 𝑆1 𝑆3
\2
𝑦𝑘
Representation 𝐺𝜃
LLM-based Generator
𝛾
Learnable Scaling Coef.
…
Support Score 𝑠𝑗→𝑘
\3
𝑦𝑘
…
MLP-based Predictor
𝑀𝑎𝑠𝑘 Masked
3 Support Ranking 1. 𝐺𝜃
Sigmoid
Remove 𝑅𝜙
𝑆2 𝑆3 Semantic Set
2 Intervention 1.
1. 4 Boundary Enhancement dep Norm max(0, 𝑑ഥ𝑖 − 𝑑𝑖 ) → 𝑠 1 0
𝜎
Concatenation
2 Intervention 1. 𝐴𝑖
Addition
……
1 Candidate Pool 1.
𝑧𝑖+𝑊−1
…
First-Pass Draft
~𝑘 𝑦
Cross-Segment Support Estimation
Candidate Boundary 𝑧𝑖+1
𝐼𝑘
Chapter Prediction
Local Causal Dependency Shift 𝑧𝑖
𝑠𝑘 , 𝑒𝑘 , 𝑦𝑘
Generation Input
CSSE Support ℛ𝑘 𝐺𝜃 Final Description 𝑦𝑘
Boundary Prediction
1 Sliding Windows 1. 𝑧𝑖−𝑊+1 𝑧𝑖−𝑊+2 …
Output: Chapters
Chapter Generation
𝑝𝑖base
Base Score
𝒕𝒊𝒎𝒆
Topk 𝑆𝑗 ∈𝐶𝑘 (𝑠𝑗→𝑘 ) →ℛ𝑘
Figure 2: Overview of CausalChapter. Given sentence-level semantic units with aligned visual and ASR information, the framework first predicts chapter boundaries with a segment-then-caption backbone enhanced by Local Dependency Shift (LCDS), which captures dependency drops between adjacent temporal windows. The resulting segments are then described by an LLM-based generator, where Cross-Segment Support Selection (CSSE) reranks historical contexts according to their predictive support for the current chapter. The final output is a sequence of temporally grounded chapter descriptions.
an operational measure of predictive support rather than a claim about real-world causal structure. 3.2
projection modules. This backbone improves scalability, but its boundary prediction mainly relies on local multimodal evidence, and its generation is primarily conditioned on the current segment.
Segment-then-Caption Backbone
We instantiate a segment-then-caption backbone over sentence-level semantic units U = {ui = (xi , tsi , tei )}N i=1 , where xi denotes the i-th ASR sentence and tsi , tei its timestamps. For each unit, we sample video frames at 1 FPS within [tsi , tei ] and extract CLIP visual features (Radford et al., 2021), and fuse them with the ASR representation to obtain a multimodal semantic-unit representation zi . Given zi , a boundary classifier gbd forecasts the probability of a chapter boundary occurring after the i-th unit: obase = gbd (zi ), pbase = σ(obase )cb , i i i
3.3
Local Dependency Shift
Smooth topic transitions are difficult to localize from local visual or lexical changes alone, because adjacent units across chapter boundaries may remain semantically coherent. We treat a boundary as positions where the predictive dependency drops between neighboring temporal windows. For a candidate boundary after ui , we construct a preceding window Ai = [zi−W +1 , . . . , zi ] and a following window Bi = [zi+1 , . . . , zi+W ], where W is the window size. A MLP-based dependency predictor Rϕ reconstructs the following window from the preceding one, B̂i = Rϕ (Ai ). To estimate the contribution of each preceding unit, we mask \r \r the r-th unit in Ai to obtain Ai and predict B̂i = \r Rϕ (Ai ). The intervention effect is measured by the representation change in the predicted following window:
(1)
where obase ∈ R2 is boundary logits, cb is the i boundary class index, and pbase is the predicted i boundary probability. The predicted boundaries are used to divide the video into chapter segments S = {Sk }K k=1 . For each segment Sk , we construct a generation input Ik from the segment ASR text, a compressed visual representation, an explicit temporal prompt, and a temporal feature pooled from semantic-unit representations within Sk . The LLM-based generator Gθ then produces a first-pass chapter description ỹk = Gθ (Ik ), where θ includes trainable parameters such as LoRA adapters and lightweight
W
\r gi,r,j = Drep (b̂i,j , b̂i,j ), di =
W
1 XX gi,r,j , W2 r=1 j=1
(2) \r where Drep denotes cosine distance, b̂i,j and b̂i,j are the j-th predicted representations before and af4
ter intervention. A larger di indicates stronger predictive support from the preceding window to the following one, while a smaller di indicates weaker temporal dependency. We then compare di with its neighborhood. Let N (i) denote neighboring candidate positions aroundPi within a fixed local radius, and let d¯i = 1 q∈N (i) dq be the local reference depen|N (i)| dency. Since we only care about dependency drops, the final dependency-shift score is sdep = Norm max(0, d¯i − di ) , (3) i
The output-difference function is then T 1X DKL pt (Ck )∥pt (Ck−j ) . T t=1 (6) Here, T is the number of evaluated output positions. A larger divergence means that removing Sj changes the model’s predictive distribution more strongly, so Sj is considered to provide stronger predictive support for the current chapter. We define its support score as
Dout (Ck , Ck−j ) =
sj→k = Dout (Ck , Ck−j ),
where Norm(·) denotes min-max normalization over all candidate boundary positions in the video. A larger sdep suggests a stronger local dependency i drop and is therefore more likely to indicate a smooth chapter boundary. Finally, we inject this score into the boundarybase ) + γsdep , where γ class logit, (oenh cb i )cb = (oi i is a learnable scaling coefficient. The enhanced boundary probability is penh = σ(oenh i i )cb . LCDS thus complements the base boundary classifier with an intervention-defined structural cue. 3.4
and analyze alternative implementations of Dout in Appendix C.4. The top-ranked contexts are then selected as Rk = TopKSj ∈Ck (sj→k ), and the final chapter description is generated as yk = Gθ (Ik , ỹk , Rk ). This shifts context selection from similarity-driven matching to intervention-driven support estimation. Since interventions are performed only on the compact candidate set Ck , the additional cost scales with the candidate size rather than the total number of video segments.
Cross-Segment Support Selection
3.5
Independent chapter generation often misses longrange prerequisites such as earlier definitions, assumptions, and problem setups. We therefore estimate the predictive support of historical segments through removal interventions. For each segment Sk , we build a segment representation hk = Eseg (Sk , ỹk ) using the segment content, temporal information, segmentation-stage features, and first-pass description. To control computation, we first construct a compact candidate context set: Ck = Cknear ∪ Cksem , Cksem = TopMj<k Sim(hk , hj ).
Training Objective and Inference
The boundary module is trained with supervised boundary labels using X ∗ Lbd = − log penh (8) i (bi ), i
where b∗i is the ground-truth boundary label after ui . The dependency predictor is trained to reconstruct the following window with Lrec = P i Drec (B̂i , Bi ). The generator is trained with the standard autoregressive objective: XX ∗ ∗ Lgen = − log pθ (yk,t | yk,<t , Ik , Rk ),
(4)
where Cknear contains temporally neighboring historical segments, Cksem contains semantically retrieved segments, and M is a small retrieval budget. This stage only narrows the search space and does not determine the final contexts. Given Ck , the generator first produces a reference sequence with all candidate contexts, ykref = Gθ (Ik , ỹk , Ck ). For each candidate segment Sj ∈ Ck , let Ck−j = Ck \ {Sj }. We compare the teacherforced output distributions under Ck and Ck−j over the same reference prefix. Specifically, define ref pt (C) = pθ (· | yk,<t , Ik , ỹk , C).
(7)
k
t
(9) where yk∗ is the ground-truth chapter description, together with a timestamp reconstruction loss X Ltime = − log pθ (yt | y<t , Ik ). (10) t∈Ωtime
The final objective is L = Lbd + Lrec + Lgen + Ltime ,
(11)
At inference time, CausalChapter first encodes semantic units and predicts enhanced chapter
(5) 5
boundaries, which define chapter segments. The generator then produces first-pass descriptions, retrieves candidate historical contexts, estimates their support scores through removal interventions, and generates final descriptions with the top supportive contexts. The final output is the chapter sequence C = {(sk , ek , yk )}K k=1 .
4
Experiments
4.1
Experimental Setup
Team et al., 2023), as reference systems. All trainable baselines are adapted to the same input setting and evaluated with the same references whenever applicable. Implementation. For state-of-the-art comparison, we evaluate both Qwen2.5-7B (Yang et al., 2025) and LLaMA-3.1-8B (Grattafiori et al., 2024) on AVLecture and VidChapters-7M. For controlled ablations, we use Qwen2.5-3B as the generation backbone and keep it fixed across all variants. All trainable LLMs are adapted with LoRA, where we set the rank and scaling factor to (r, α) = (32, 64) for Qwen2.5-7B and (r, α) = (8, 16) for Qwen2.53B. Models are trained with AdamW using a learning rate of 5 × 10−5 , batch size 1, and maximum input length 8192. For prompting, all methods use the same segment-level instruction template, which includes the segment ASR, visual summary, timestamp prompt, and optional retrieved contexts. For CSSE, we construct candidate contexts from temporal neighbors and semantic retrieval, and select the top-K = 5 segments. For LCDS, we set the window size to W = 5 for interventional dependency modeling. We use deterministic decoding with temperature 0 for intervention scoring to ensure that output changes are attributable to context removal, and use the same decoding setting across all compared variants. Experiments are conducted on a Debian GNU/Linux 12 server equipped with a single NVIDIA H800 PCIe GPU with 80 GB memory. Additional efficiency and scalability analyses are reported in Appendix C.5.
Datasets. We evaluate CausalChapter on two long-video chaptering benchmarks. The primary benchmark is AVLecture (Gupta et al., 2023), which contains long lecture videos with ASR transcripts, OCR outputs, visual content, and humanannotated topic boundaries. Since AVLecture does not provide chapter-level descriptions, we augment each ground-truth segment with a human-verified LLM-assisted chapter description. All models are trained and evaluated on the same augmented references. We use 245 videos for training, 35 for validation, and 70 for testing. We further evaluate generalization on VidChapters-7M (Yang et al., 2023a), which provides user-annotated chapter boundaries and titles. Details of the annotation protocol and annotation-sensitivity analysis are provided in Appendix B.1. Metrics. We evaluate both chapter localization and description generation. For localization, we report F1@30 and tIoU on AVLecture, and follow prior work to report F1 and tIoU on VidChapters7M. We additionally use exact-boundary F1 in ablation analyses to expose fine-grained differences among boundary variants. For generation, we report CIDEr (Vedantam et al., 2015) on both datasets. On AVLecture, we report SODA_c (Fujita et al., 2020), computed with SODA typec using IoU-weighted METEOR matching. On VidChapters-7M, we follow the Chapter-Llama evaluation and report its benchmark notation SODA.
4.2
Main Result
We compare CausalChapter with dense video captioning methods, long-video LLM baselines, video chaptering models, and closed-source LLMs under zero-shot prompting. Tab. 1 reports results on the augmented AVLecture and VidChapters-7M benchmarks. Results on AVLecture. CausalChapter performs best overall among trainable models, substantially outperforming the DVC baselines PDVC (Wang et al., 2021) and Vid2Seq (Yang et al., 2023b). Compared with the strongest fine-tuned chaptering baseline, Chapter-Llama (Ventura et al., 2025), it improves CIDEr from 99.78 to 110.12 with LLaMA-3.1-8B and from 95.36 to 116.88 with Qwen2.5-7B; F1@30 and tIoU also increase from 63.93% and 62.18% to 72.97% and 69.95%, respectively. These gains demonstrate improvements in
Baselines. We compare CausalChapter with representative dense video captioning methods, including PDVC (Wang et al., 2021) and Vid2Seq (Yang et al., 2023b), as well as longvideo LLM and video chaptering baselines, including VTimeLLM (Huang et al., 2024) and ChapterLlama (Ventura et al., 2025). We also include closed-source LLMs, GPT-4o (OpenAI, 2024), Gemini-2.5-Pro and Gemini-2.0-Flash (Gemini 6
Type
Method
CIDEr
SODA_c
F1@30
tIoU
DVC
PDVC Vid2SeqT5
6.11 61.22
– 9.79
9.07 9.84
56.44 53.77
Chap.
VTimeLLMVicuna, * Chapter-LlamaLLaMA Chapter-LlamaQwen CausalChapterLLaMA CausalChapterQwen
0.02 99.78 95.36 110.12 116.88
3.04 8.15 6.91 11.73 13.80
47.53 63.93 57.73 72.97 72.97
46.72 62.18 59.62 69.95 69.95
LLM
GPT-4o Gemini-2.5-Pro Gemini-2.0-Flash
1.82 2.40 0.28
0.08 1.64 0.18
24.89 26.72 11.71
35.23 36.12 13.54
Backbone is the base segment-then-caption model; +LCDS and +CSSE add the corresponding localization and generation modules, while Full further includes temporal-aware training. As shown in Table 2, LCDS improves F1, BS@30, and tIoU from 52.95%, 59.61%, and 66.36% to 56.72%, 63.02%, and 69.95%, respectively. CSSE leaves boundary predictions unchanged while increasing CIDEr from 88.39 to 98.40 and SODA_c from 10.84 to 12.63. Their complementary effects yield the full model’s best CIDEr, METEOR, and SODA_c scores of 104.59, 17.87, and 12.73.
(a) AVLecture Type
Method
DVC
Vid2SeqT5 LLaMA
CIDEr
SODA
F1
tIoU
55.8
11.6
26.7
58.6
Chap.
Chapter-Llama CausalChapterLLaMA CausalChapterQwen
100.9 101.5 109.2
19.3 18.5 18.8
45.3 46.2 46.2
71.8 72.0 72.0
LLM
GPT-4o Gemini-2.0-Flash
51.0 69.7
8.1 11.4
37.6 40.2
68.0 69.3
4.4
We further analyze whether intervention-defined dependency provides more useful signals than simpler relevance-based alternatives. Table 3 compares LCDS with representation-similarity drops (Sim. Drop) and a contrastive objective (CL Loss). Similarity drops improve several threshold-based metrics but reduce tIoU, while contrastive learning gives only marginal gains. Our dependency score performs best on all metrics, increasing F1 from 52.95% to 56.72% and tIoU from 66.36% to 69.95%, supporting predictive dependency as a stronger cue for smooth boundaries. Table 4 compares CSSE with temporal proximity (Previous-K), semantic retrieval (Sim. Top-K), and LLM-estimated relevance using the same backbone and boundaries. CSSE performs best on all generation metrics, improving CIDEr from 88.39 to 104.59 and SODA_c from 10.84 to 12.73, showing the advantage of measuring a context’s influence on current generation.
(b) VidChapters-7M Table 1: Comparison of video chaptering methods on AVLecture and VidChapters-7M. For CausalChapter, localization scores come from the shared boundarylocalization stage, while generation scores use the indicated backbone. Superscripts denote backbones: T5 T5, Vicuna Vicuna-7B, LLaMA LLaMA-3.1-8B, and Qwen Qwen2.5-7B. Methods without superscripts are proprietary or do not report a backbone; * marks VTimeLLM not fine-tuned on AVLecture.
both description quality and boundary localization. Closed-source LLMs under zero-shot prompting obtain lower automatic scores; Appendix C.3 provides a complementary semantic evaluation, and Appendix C.6 provides a qualitative comparison with Chapter-Llama. Results on VidChapters-7M. Tab. 1(b) shows that our method also generalizes to VidChapters7M, which contains more diverse open-domain videos with user-annotated chapter boundaries and titles. Despite using a compact backbone, CausalChapter achieves competitive or better performance than long-video LLM baselines on both generation and localization metrics. This suggests that the proposed dependency-enhanced segmentthen-caption framework is not specific to lecture videos and can transfer to broader long-video chaptering scenarios. 4.3
Analysis of Interventional Dependency Modeling
Window-size sensitivity. We further analyze the effect of the LCDS window size W . As shown in Figure 3, W = 5 provides the most balanced performance across boundary-oriented metrics, improving F1, BS@30, and tIoU over the backbone while maintaining competitive F1@30. This suggests that a moderate local window captures sufficient cross-boundary dependency changes without introducing excessive neighboring noise. Top-K sensitivity. We further analyze the effect of the number of supportive contexts selected by CSSE. As shown in Figure 4, increasing TopK generally improves generation quality over the backbone, indicating that cross-segment supportive contexts provide useful complementary infor-
Ablation Studies
We ablate CausalChapter on AVLecture using the same Qwen2.5-3B backbone and training data. 7
Method CIDEr METEOR SODA_c F1 BS@30 tIoU Backbone 88.39 16.83 10.84 52.95 59.61 66.36 + LCDS 91.36 17.37 11.90 56.72 63.02 69.95 + CSSE 98.40 17.15 12.63 52.95 59.61 66.36 Full 104.59 17.87 12.73 56.72 63.02 69.95
Table 2: Ablation studies on AVLecture. All variants use predicted boundaries. LCDS denotes Local Causal Dependency Shift, CSSE denotes Cross-Segment Causal Support Estimation. tIoU 66.36 65.68 65.23 69.95
F1 baseline 52.95
54
4
5
6
7
3
4
Baseline
W=5
5
6
7
Best metric
Figure 3: Window-size sensitivity of LCDS on AVLecture.
SODA_c 10.84 11.32 11.70 11.53 12.73
CIDEr
METEOR
105 baseline 88.39
104.59
18.0
Score
100 95
85
baseline 16.83
17.87
17.5 17.0
90
Table 4: Analysis of cross-segment context selection on AVLecture. All methods use Qwen2.5-3B as backbone.
1
3
5
7
9
16.5
1
3
5
7
9
CSSE Top-K CSSE
mation for chapter description generation. The setting K = 5 achieves the best METEOR score and a strong CIDEr score, improving CIDEr from 88.39 to 104.59 and METEOR from 16.83 to 17.87. Therefore, we use K = 5 as a balanced setting in our main experiments.
5
68
LCDS window size
LCDS
Score
METEOR 16.83 16.70 16.79 17.44 17.87
69.95
66
52
Table 3: Analysis of LCDS on AVLecture. All methods use the same Qwen2.5-3B backbone. CIDEr 88.39 90.85 95.97 96.11 104.59
70 baseline 66.36
56
3
Method Baseline Previous-K Sim. Top-K LLM Scoring Ours
tIoU
56.72
Score
F1 F1@30 BS@30 52.95 71.28 59.61 54.51 72.41 61.80 53.28 70.91 59.75 56.72 72.97 63.02
Score
Method Baseline Sim. Drop CL Loss Ours
Baseline
K=5
Best metric
Figure 4: Top-K sensitivity of CSSE on AVLecture.
Limitations CausalChapter has several limitations. First, intervention-defined dependency should be interpreted as prediction-level influence rather than realworld causal discovery. Our goal is not to recover causal relations among video events or chapters, but to identify semantic units or context segments that affect boundary prediction and chapter-level generation under controlled masking or removal interventions. Second, our augmented AVLecture benchmark uses human-verified LLM-assisted chapter descriptions, which may inherit some stylistic regularities from the annotation pipeline. To reduce this effect, all compared methods are trained and evaluated with the same augmented references, and our annotation-sensitivity analysis examines alternative annotation models and styles. Third, CSSE introduces additional inference cost because it estimates context support through removal interventions. We control this cost by applying interventions only to a compact candidate set constructed from temporal neighbors and semantic retrieval, rather than to all segments or tokens. Further acceleration of support estimation is an interesting direction for future work.
Conclusion
We presented CausalChapter, an interventional dependency modeling framework for long-video chaptering. To address the boundary error propagation and cross-segment context fragmentation introduced by segment-level generation, CausalChapter estimates intervention-defined predictive dependencies as task-oriented support signals. Specifically, LCDS captures local dependency drops between adjacent temporal windows to provide auxiliary evidence for smooth chapter boundaries, while CSSE selects cross-segment contexts according to their intervention-defined support for current chapter generation. Experiments on AVLecture and VidChapters-7M show that CausalChapter improves temporal localization and description quality on AVLecture, while remaining competitive on VidChapters-7M. These results highlight intervention-defined dependency as a useful signal for scalable and coherent long-video chaptering. 8
Acknowledgments
2024. Video recap: Recursive captioning of hourlong videos. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18198–18208.
This work was supported by the National Natural Science Foundation of China (62437001, 62506040 and 62402051) and the Fundamental Research Funds for the Central Universities (2253500001).
Minkuk Kim, Hyeon Bae Kim, Jinyoung Moon, Jinwoo Choi, and Seong Tae Kim. 2024. Do you remember? dense video captioning with cross-modal memory retrieval. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13894–13904.
References Tieyuan Chen, Huabin Liu, Yi Wang, Yihang Chen, Tianyao He, Chaofan Gan, Huanyu He, and Weiyao Lin. 2026. Mecd+: Unlocking event-level causal graph discovery for video reasoning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 48(3):2628–2645.
Minkuk Kim, Hyeon Bae Kim, Jinyoung Moon, Jinwoo Choi, and Seong Tae Kim. 2025. Hicm2 : Hierarchical compact memory modeling for dense video captioning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 4293– 4301.
Soichiro Fujita, Tsutomu Hirao, Hidetaka Kamigaito, Manabu Okumura, and Masaaki Nagata. 2020. Soda: Story oriented dense video captioning evaluation framework. In European Conference on Computer Vision, pages 517–531. Springer.
Ranjay Krishna, Kenji Hata, Frederic Ren, Li Fei-Fei, and Juan Carlos Niebles. 2017. Dense-captioning events in videos. In Proceedings of the IEEE international conference on computer vision, pages 706–715. Guozhang Li, De Cheng, Xinpeng Ding, Nannan Wang, Xiaoyu Wang, and Xinbo Gao. 2023. Boosting weakly-supervised temporal action localization with text information. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 10648–10657. IEEE.
Gemini Team, Rohan Anil, Sebastian Borgeaud, JeanBaptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M. Dai, Anja Hauth, Katie Millican, et al. 2023. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805.
Guozhang Li, Xinpeng Ding, De Cheng, Jie Li, Nannan Wang, and Xinbo Gao. 2025. Etc: Temporal boundary expand then clarify for weakly supervised video grounding with multimodal large language model. IEEE Transactions on Multimedia, 27:1772–1782.
Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad AlDahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, and 542 others. 2024. The llama 3 herd of models. Preprint, arXiv:2407.21783.
Guozhang Li, Xinran Duan, Mei Wang, Lizhi Wang, and Hua Huang. 2026. Curvature-guided task synergy for skeleton based temporal action segmentation. In International Conference on Learning Representations, volume 2026, pages 74158–74176.
Anchit Gupta, CV Jawahar, Makarand Tapaswi, et al. 2023. Unsupervised audio-visual lecture segmentation. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 5232–5241.
Yang Liu, Guanbin Li, and Liang Lin. 2023. Crossmodal causal relational reasoning for event-level visual question answering. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(10):11624–11641.
Bin Huang, Xin Wang, Hong Chen, Zihan Song, and Wenwu Zhu. 2024. Vtimellm: Empower llm to grasp video moments. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14271–14280.
Zhiyue Liu, Xinru Zhang, and Jinyuan Liu. 2025. Taskspecific information decomposition for end-to-end dense video captioning. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 16524– 16536.
Vladimir Iashin and Esa Rahtu. 2020a. A better use of audio-visual cues: Dense video captioning with bi-modal transformer. arXiv preprint arXiv:2005.08271. Vladimir Iashin and Esa Rahtu. 2020b. Multi-modal dense video captioning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, pages 958–959.
Yulei Niu, Kaihua Tang, Hanwang Zhang, Zhiwu Lu, Xian-Sheng Hua, and Ji-Rong Wen. 2021. Counterfactual vqa: A cause-effect look at language bias. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 12700– 12710.
Md Mohaiminul Islam, Ngan Ho, Xitong Yang, Tushar Nagarajan, Lorenzo Torresani, and Gedas Bertasius.
OpenAI. 2024. Gpt-4o system card. arXiv preprint arXiv:2410.21276.
9
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021. Learning transferable visual models from natural language supervision. In International conference on machine learning, pages 8748–8763. PmLR.
Shoubin Yu, Jaemin Cho, Prateek Yadav, and Mohit Bansal. 2023. Self-chained image-language model for video localization and question answering. Advances in Neural Information Processing Systems, 36:76749–76771. Abhay Zala, Jaemin Cho, Satwik Kottur, Xilun Chen, Barlas Oguz, Yashar Mehdad, and Mohit Bansal. 2023. Hierarchical video-moment retrieval and stepcaptioning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 23056–23065.
Ramakrishna Vedantam, C Lawrence Zitnick, and Devi Parikh. 2015. Cider: Consensus-based image description evaluation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4566–4575. Lucas Ventura, Antoine Yang, Cordelia Schmid, and Gül Varol. 2025. Chapter-llama: Efficient chaptering in hour-long videos with llms. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 18947–18958. Teng Wang, Ruimao Zhang, Zhichao Lu, Feng Zheng, Ran Cheng, and Ping Luo. 2021. End-to-end dense video captioning with parallel decoding. In Proceedings of the IEEE/CVF international conference on computer vision, pages 6847–6857. Kangyi Wu, Pengna Li, Jingwen Fu, Yizhe Li, Yang Wu, Yuhan Liu, Jinjun Wang, and Sanping Zhou. 2025. Event-equalized dense video captioning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8417–8427. Junbin Xiao, Xindi Shang, Angela Yao, and Tat-Seng Chua. 2021. Next-qa: Next phase of questionanswering to explaining temporal actions. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9777–9786. Zhuyang Xie, Yan Yang, Yankai Yu, Jie Wang, Yongquan Jiang, and Xiao Wu. 2025. Exploring temporal event cues for dense video captioning in cyclic co-learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 8771–8779. An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, and 23 others. 2025. Qwen2.5 technical report. Preprint, arXiv:2412.15115. Antoine Yang, Arsha Nagrani, Ivan Laptev, Josef Sivic, and Cordelia Schmid. 2023a. Vidchapters-7m: Video chapters at scale. Advances in Neural Information Processing Systems, 36:49428–49444. Antoine Yang, Arsha Nagrani, Paul Hongsuck Seo, Antoine Miech, Jordi Pont-Tuset, Ivan Laptev, Josef Sivic, and Cordelia Schmid. 2023b. Vid2seq: Largescale pretraining of a visual language model for dense video captioning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10714–10726.
10
A
Method Details
A.1
Segmentation Head Architecture
of STEM subjects. Each course provides video lectures together with aligned ASR transcripts and OCR signals, and many courses also include auxiliary educational resources such as lecture notes, slides, and assignments. Among the 86 courses, a 15-course subset with 350 lectures is annotated with temporal segmentation boundaries and serves as the standard benchmark for lecture segmentation. In our work, we adopt this segmented subset as the boundary-localization foundation, and use the 245/35/70 train/validation/test split described in the main text. We further extend these lectures with chapter-level reference descriptions so that the same benchmark can support evaluation of both chapter localization and chapter description generation.
The main text abstracts the boundary predictor as gbd (zi ). In practice, our segmentation head follows the general design of recent multimodal video topic segmentation models: it uses sentence-aligned clips as basic units, projects visual and textual clip features into a shared space, performs middlefusion across modalities, and then predicts whether each unit is a topic boundary with a lightweight binary classifier. Concretely, for the i-th semantic unit, we first obtain a visual clip representation and a text representation from sampled video frames and the corresponding ASR sentence, where Ev and Et denote the visual and textual encoders, respectively. After projection into the same hidden dimension by the trainable projection matrices Wv and Wt , the two modalities are fused by a small stack of multimodal fusion layers, denoted by MFL, to produce updated visual and textual states. We then concatenate the fused modality-specific states into the multimodal unit representation zi , which is used by the segmentation head: vi = Wv Ev (cvi ), hvi , hti = MFL(vi , ti ),
ti = Wt Et (cti ),
(12)
zi = [hvi ; hti ],
(13)
obase = gbd (zi ) = Wp zi + bp . i
LLM-Assisted Annotation Pipeline. For each video, we use GPT-4o (OpenAI, 2024) as the annotation model. The input includes the full video transcript, frame-level captions extracted every 10 seconds, and the start and end timestamps of all ground-truth segments. The model is instructed to output a JSON object, where each segment is paired with a concise chapter-level description. Each description is expected to summarize the main topic of the corresponding segment, remain faithful to the transcript and frame captions, and distinguish the segment from adjacent chapters. When the transcript and frame captions exceed the input budget, we truncate the input while preserving the segment timestamps and the local transcript/caption context around each target segment.
(14)
Here, Wp and bp denote the predictor weight matrix and bias term of the final binary classifier. This design keeps the segmentation head lightweight while still allowing cross-modal interaction before boundary prediction. Relative to a late-fusion design, the middle-fusion structure better exposes cross-modal cues such as transcript transitions, slide changes, and visual context shifts to the boundary classifier. In our implementation, the segmentation head is therefore best viewed as a multimodal fusion block followed by a linear classifier that outputs the boundary logits in Eq. 1.
B
Dataset Construction and Annotation
B.1
AVLecture Benchmark
Human Verification and Revision. After GPT4o-assisted annotation, all segment descriptions are manually checked and revised. We correct descriptions that are overly generic, unsupported by the transcript or frame captions, inconsistent with the segment boundary, or redundant with adjacent chapters. For segments whose content is distributed across multiple stages of the lecture, we manually improve or combine the generated descriptions to better reflect the complete chapter-level semantics. We also normalize the JSON format, description length, and writing style across videos. The final references are fixed before training and evaluation, and all compared methods use the same augmented references, ensuring that the evaluation remains internally consistent. The verification was conducted by 10 annotators. Each video and all of its chapter descriptions were reviewed by a group of three annotators. When one
AVLecture is a long-form instructional-video benchmark introduced to audio-visual lecture segmentation. The full AVLecture collection contains 86 courses with over 2,350 lectures and a total duration of roughly 2,200 hours, covering a broad range 11
annotator revised a description, the revised label was circulated to the other two annotators and further refined until all three accepted the final version. We therefore record revision history and final consensus rather than a chance-corrected agreement coefficient. Among the 350 videos, 308 videos (88.0%) required no revision, 39 videos (11.1%) required one revision round, and 3 videos (0.9%) required two revision rounds. All final labels were accepted by the assigned annotator group. Table 5 gives representative revision examples, illustrating how human annotators remove crossboundary content and make chapter descriptions more specific to the current segment.
sures whether a predicted boundary falls within a 30-second tolerance window around a ground-truth boundary, thus reflecting boundary-detection accuracy under a fixed temporal tolerance. tIoU reports the temporal intersection-over-union between predicted and ground-truth segments, measuring how accurately the predicted chapter partition overlaps with the reference segmentation. F1@30 applies the F1 criterion under the same 30-second matching window, and therefore captures both boundary accuracy and temporal tolerance.
C.2 Backbone Scaling and Complete Baseline Results
Annotation Sensitivity. To examine whether model comparisons depend on a particular annotation model or writing style, we construct four AVLecture reference sets by varying the annotation model and description format: GPT-4o phrase-style references, GPT-4o sentence-style references, Claude phrase-style references, and Claude sentence-style references. For each reference set, models are retrained and evaluated with the same split, model configuration, training procedure, and evaluation code; only the training and evaluation references are changed. Table 6 shows that CausalChapter improves over Chapter-Llama across all four reference settings, suggesting that the gains are not tied to a single annotation style.
C
Additional Experimental Results
C.1
Evaluation Metrics
Overview. The main experiments use different backbone sizes for different purposes: we use larger backbones for state-of-the-art comparison, and a smaller Qwen2.5-3B backbone for ablation studies to control experimental cost while keeping variants comparable. This appendix reports the complete results behind these choices. We first present backbone scaling results for CausalChapter across the LLaMA and Qwen2.5 families, and then provide the full LLM-based baseline and closed-source LLM reference results on AVLecture. These supplementary results verify that the gains of CausalChapter are not tied to a single model family or scale. Table 7 shows two consistent trends. First, stronger backbones generally lead to better chapterlevel generation quality, especially on CIDEr. Second, temporal-aware optimization improves most corresponding settings across both LLaMA and Qwen2.5 families. These results suggest that the proposed framework benefits from model scaling, while the temporal-aware optimization provides additional gains beyond simply increasing backbone size.
For completeness, we briefly summarize the metrics used in the main paper and the appendix tables. CIDEr evaluates how well a generated chapter description matches the reference description using consensus-based n-gram similarity, with higher scores indicating better agreement with human-written references. METEOR measures generation quality through unigram alignment with stemming and synonym matching. SODA_c jointly evaluates temporal localization and description quality through temporally ordered matching; on AVLecture, we use SODA type-c with IoUweighted METEOR matching, as specified in the main text. For boundary localization, F1 is the harmonic mean of precision and recall, and evaluates the overall quality of predicted topic boundaries according to how well they match ground-truth boundaries. BS@30 (Boundaries at 30 seconds) mea-
Table 8 provides the complete baseline results on AVLecture. Fine-tuned Chapter-Llama variants are much stronger than their zero-shot counterparts, indicating that task adaptation is important for longvideo chaptering. Closed-source LLMs under zeroshot prompting achieve limited performance, especially on generation metrics, suggesting that simply prompting general-purpose LLMs is insufficient for this benchmark. These observations support the need for task-specific modeling of boundary localization and cross-segment context dependency. 12
Issue
LLM draft
Human-revised label
Reason
Cross-boundary content Overly general description
Force range dependence on mediator mass via decay laws Conservation laws in elastic collisions
Force range dependence on mediator mass Linearizing one-dimensional elastic collisions via relative velocity
“Decay laws” belongs to the following chapter. The draft omitted the specific derivation in the current chapter.
Table 5: Representative human revisions in the augmented AVLecture benchmark.
Method
R1 C
R1 S
R2 C
R2 S
R3 C
R3 S
R4 C
R4 S Avg. C Avg. S
Chapter-LlamaLLaMA 99.78 8.15 101.26 8.41 107.25 10.32 104.74 10.13 103.26 9.25 CausalChapterLLaMA 110.12 11.73 116.12 14.13 115.63 12.67 128.19 14.34 117.52 13.22 CausalChapterQwen 116.88 13.80 138.99 15.21 121.74 14.63 145.24 15.83 130.71 14.87 Table 6: Annotation-sensitivity results on AVLecture. R1/R2 use GPT-4o references in phrase/sentence styles, and R3/R4 use Claude references in phrase/sentence styles. C and S denote CIDEr and SODA_c, respectively.
Backbone
temporal-aware
CIDEr
METEOR
LLaMA-3.2-1B LLaMA-3.2-1B LLaMA-3.2-3B LLaMA-3.2-3B LLaMA-3.1-8B LLaMA-3.1-8B
w/o with w/o with w/o with
78.58 83.16 93.40 98.45 98.38 110.12
10.58 13.05 17.15 17.45 19.08 19.41
Qwen2.5-0.5B Qwen2.5-0.5B Qwen2.5-1.5B Qwen2.5-1.5B Qwen2.5-3B Qwen2.5-3B Qwen2.5-7B Qwen2.5-7B
w/o with w/o with w/o with w/o with
84.56 86.48 90.57 97.94 97.64 104.59 113.98 116.88
10.38 13.41 16.19 16.33 17.73 17.87 18.24 20.43
C.4
CSSE Output-Difference Sensitivity
The main method instantiates Dout as the teacherforced token-level KL divergence between the fullcontext and leave-one-out predictive distributions. To assess whether CSSE depends on this particular distance function, we compare it with LLM-based relevance scoring, embedding distance, and tokenoverlap distance under the same Qwen2.5-3B setting. As shown in Table 10, all variants improve over removing CSSE, while KL divergence performs best overall. C.5
Table 7: Backbone scaling results of CausalChapter on AVLecture. Temporal-aware optimization further injects timestamp information and segmentation-stage temporal features into the generator.
Inference Efficiency and Scalability
Profiling protocol and end-to-end comparison. We profile Chapter-Llama and CausalChapter on the same 10 AVLecture test videos using an NVIDIA A100 80GB GPU, BF16 precision, identical decoding settings, and the same warm-up procedure. The videos contain 4.3 predicted chapters on average. Chapter-Llama uses its original whole-video inference pipeline and jointly predicts boundaries and descriptions, so its two stages cannot be timed separately. CausalChapter first predicts boundaries and then applies intravideo micro-batching to chapter-level generation and leave-one-out scoring requests; the video-level batch size remains one. Table 11(a) reports the matched end-to-end comparison using LLaMA-3.18B for both methods and a micro-batch size of 8 for CausalChapter.
C.3 Closed-Source LLM Semantic Evaluation
Lexical metrics can underestimate zero-shot closedsource LLMs when their outputs are semantically reasonable but use a different wording or granularity from the reference descriptions. We therefore complement CIDEr with a semantic-similarity evaluation. GPT-4o is given a generated chapter title and the corresponding reference title, and assigns a 0–100 semantic-similarity score based on topic match and specificity. Table 9 reports the averaged scores. The semantic gap is smaller than the CIDEr gap, but fine-tuned chaptering models still achieve stronger semantic alignment with the AVLecture references.
Intra-video micro-batching. Once boundaries are fixed, requests from different chapters can be executed in parallel. We apply intra-video microbatching to Pass 1 generation, full-context refer13
Method
Backbone
Type CIDEr METEOR
F1 F1@30 BS@30
tIoU
VTimeLLM VTimeLLM VTimeLLM
ChatGLM3-6B Vicuna-7B Vicuna-7B
ZS ZS FT
0.12 0.02 –
1.79 32.46 2.04 32.25 0.59 9.22
47.66 47.53 20.71
45.78 48.21 50.97 46.72 37.38 27.88
Chapter-Llama Chapter-Llama Chapter-Llama Chapter-Llama Chapter-Llama Chapter-Llama
LLaMA-3.2-1B LLaMA-3.2-1B LLaMA-3.2-3B LLaMA-3.2-3B LLaMA-3.1-8B LLaMA-3.1-8B
ZS FT ZS FT ZS FT
14.77 68.53 38.77 86.07 77.32 99.78
8.36 15.67 12.79 19.40 13.93 19.04
3.51 48.18 4.39 47.53 20.24 47.56
8.19 63.36 9.12 65.11 40.40 63.93
8.19 63.36 9.12 65.11 40.40 63.93
19.14 59.30 23.58 60.64 41.72 62.18
GPT-4o GPT-4o-mini Gemini-2.5-Pro Gemini-2.0-Flash Claude-3.5-Sonnet Claude-Sonnet-4
– – – – – –
ZS ZS ZS ZS ZS ZS
1.82 2.80 2.40 0.28 – 0.02
2.34 2.95 10.03 3.07 5.44 10.93
24.32 15.88 24.32 9.64 16.49 18.08
24.89 21.50 26.72 11.71 24.18 25.36
26.01 34.27 26.01 27.99 38.14 34.35
35.23 31.21 36.12 13.54 31.75 38.48
Table 8: Complete long-video LLM baseline and closed-source LLM reference results on AVLecture. ZS denotes zero-shot prompting and FT denotes fine-tuning.
Method
(a) End-to-end comparison
Semantic similarity
CausalChapter Chapter-Llama Gemini-2.5-Pro
82.10 79.80 60.40
CIDEr
SODA_c
Without CSSE LLM-based scoring Embedding distance Token-overlap distance KL divergence
88.39 99.27 100.20 101.93 104.59
10.84 11.68 11.79 12.02 12.73
Bound. Gen. Total Mem.
Chapter-Llama CausalChapter
– 0.19
11.61 11.61 19.56 10.31 10.50 19.35
(b) Intra-video micro-batching
Table 9: Semantic-similarity evaluation for chapter descriptions on AVLecture. Dout implementation
Method
Batch Gen. 1 2 4 8
11.90 8.10 5.90 5.30
KL
CSSE Total Mem.
11.20 8.20 6.90 6.60
4.66 4.63 4.60 4.59
12.20 11.10 10.84 10.50
16.59 17.05 19.35 19.35
Table 11: BF16 inference on AVLecture. Times are seconds per video, memory is peak GiB, and Gen./KL are sequential batched calls per video. CausalChapter generation in (a) includes Pass 1, CSSE, and Pass 2.
Table 10: Sensitivity of CSSE to different outputdifference functions on AVLecture.
onds per video and accounts for 4.59/10.50 (43.7%) of the complete pipeline at size 8. Overall latency decreases from 12.20 to 10.50 seconds per video, while peak memory increases from 16.59 to 19.35 GiB.
ence generation, leave-one-out KL scoring, and Pass 2 generation. Table 11(b) reports the complete execution breakdown. Gen. and KL denote the average numbers of sequential batched model invocations per video after micro-batching; because they are averaged over videos, they need not be integers. At micro-batch size 1, the 11.90 generation calls in Table 11(b) comprise 4.30 Pass 1 calls, 3.30 short CSSE reference-generation calls, and 4.30 Pass 2 calls; each reference-generation call produces at most 16 tokens. Increasing the microbatch size to 8 reduces the sequential generation calls from 11.90 to 5.30 and KL calls from 11.20 to 6.60. CSSE time remains approximately 4.6 sec-
Candidate-set scaling. Table 12(a) evaluates the cost of expanding the pre-scoring candidate limit. The average number of available candidates saturates at 1.84 per chapter on the profiled videos. Consequently, increasing the limit from 1 to 10 changes the average number of KL calls only from 6.60 to 6.90 per video, while CSSE time increases from 4.43 to 5.31 seconds per video. This indicates that the practical intervention cost is governed by the compact set of available historical candidates 14
rather than by the nominal limit alone.
At micro-batch size 1, quantized CausalChapter uses 8.28 GiB of peak memory, compared with 9.90 GiB for Chapter-Llama under the same NF4 setting. Increasing the size to 8 reduces latency from 47.77 to 19.41 seconds per video while using 13.80 GiB. These results provide an explicit latency–memory operating range rather than a single deployment point.
Top-K after intervention scoring. Top-K is applied only after all candidates have been scored and therefore does not create additional intervention calls. Table 12(b) reports its measured context count, time, and memory. Generation quality is reported once, in Figure 4 in the main paper. (a) Candidate-set limit
C.6
Limit Actual/ch. KL CSSE Total Mem. 1 3 5 10
0.77 1.60 1.81 1.84
6.60 6.60 6.90 6.90
Beyond quantitative results, we further analyze representative cases in Figure 6 to examine how CausalChapter behaves under smooth boundary transitions and cross-segment context selection. In smooth-boundary cases, baseline methods often rely on local representation changes or textual similarity drops, and therefore tend to delay or miss boundaries when the topic gradually evolves. In contrast, CausalChapter captures a drop in intervention-defined predictive dependency between adjacent windows through LCDS, allowing it to identify structural changes more accurately. For chapter generation, similarity-based retrieval often selects contexts that are lexically close but largely repetitive. CSSE instead tends to select segments that provide definitions, background, experimental setup, or reasoning premises, leading to descriptions that are more complete and better aligned with the logical structure of the whole video. The qualitative example is consistent with the quantitative results. LCDS provides a local dependency-shift signal for smooth boundary localization, while CSSE changes context selection from surface similarity matching to support estimation with respect to the current generation. These two behaviors help explain why CausalChapter improves both temporal localization and chapter-level generation.
4.43 9.93 19.37 5.24 10.82 19.36 5.51 10.96 19.36 5.31 10.71 19.56
(b) Top-K after scoring Top-K
Selected/ch.
Total
Mem.
1 3 5 7 9
0.77 1.60 1.81 1.84 1.84
10.71 10.98 10.84 10.79 10.43
19.56 19.50 19.35 19.38 19.58
Table 12: CSSE scaling on AVLecture. Times are seconds per video, memory is peak GiB, and KL denotes calls per video. In (a), Gen. calls remain 5.90 per video. Top-K in (b) is applied after scoring and adds no leaveone-out evaluations.
End-to-end time remains within 10.43–10.98 seconds per video and peak memory within 19.35– 19.58 GiB across the Top-K sweep. As Figure 4 shows, K = 5 gives the highest METEOR and a strong CIDEr score while using nearly all contexts available on average; we therefore use it as the balanced default. Quantized inference. We additionally profile 4bit NF4 inference for memory-constrained settings. Figure 5 reports the latency–memory trade-off; each point is annotated with its exact value, so we do not repeat the measurements in a separate table. Total time 40
GiB
35.15
30
19.41
13.80
4
8
12 10
20.92
20
13.80
9.21
8.28 8
1
2
4
8
1
2
Prompt Design for Two-Pass Generation
We adopt a two-pass prompting strategy for lecture subheading generation. In both passes, the model receives the current segment transcript and generates one concise subheading. The system instruction remains in the non-truncated prefix, whereas the transcript occupies the truncatable middle. This preserves task instructions, visual tokens, examples, contextual titles, and generation markers under truncation. Table 13 summarizes the two passes.
14
47.77
s/video
C.7
Peak memory
50
Qualitative Analysis
Micro-batch size
Figure 5: CausalChapter latency–memory trade-off under 4-bit NF4 quantization. Lower values are better; gold points mark the minimum in each panel.
15
Ground truth CausalChapter Chapter-Llama Ground truth
CausalChapter
Chapter-Llama
00:07:47: Interpreting the Mean and Deviation 00:16:39: Markov's Inequality and Shifting Technique 00:26:52: Chebyshev Inequality, Variance, and Standard Deviation 00:41:21: Computing Variance: Rules, Examples, and Additivity
00:07:47: Mean and Deviation: Intuitive Interpretation 00:16:39: Markov's Inequality and Practical Applications 00:26:53: Chebyshev's Inequality and Standard Deviation 00:41:22: Variance Calculations and Additivity
00:10:47: Deviation from the mean and Markov's bound 00:23:47: Chebyshev's bound and standard deviation 00:41:21: Variance calculation rules and examples
(a)
Ground truth CausalChapter Chapter-Llama Ground truth
CausalChapter CausalChapter
Chapter-Llama
00:03:10: One-dimensional coordinate system and unit vectors 00:05:28: Position function and vector components 00:08:48: Displacement vector and its component 00:11:02: Average velocity as displacement over time 00:16:33: Instantaneous velocity and the derivative 00:19:30: Variable acceleration: rocket example and differentiation
00:03:10: 00:07:47: One-Dimensional Mean and Deviation: Cartesian Intuitive Coordinate Interpretation System 00:05:28: 00:16:39: Position Markov'svector Inequality and itsand component Practical interpretation Applications 00:08:49: 00:26:53: Displacement Chebyshev's Inequality vector andand its components Standard Deviation 00:11:03: 00:41:22: Average Variancevelocity Calculations and itsand definition Additivity 00:16:33: Instantaneous velocity as a limit of average velocities 00:19:30: Differentiating a rocket's position to find velocity and acceleration
00:04:23: One-dimensional motion: coordinate system and position vector 00:12:23: Displacement, average velocity, and instantaneous velocity 00:19:30: Differentiation of position to find velocity and acceleration
(b)
Figure 6: Qualitative comparison on AVLecture.
Component
Prompt Content
System Prompt
You are a helpful assistant generating lecture subheadings. Given the video segment transcript, generate a concise and informative subheading that captures the main topic. Only output the subheading, nothing else.
Pass 1: Segment-Level Generation
Visual context: <VIS_0><VIS_1>...<VIS_N>. Optional few-shot examples are provided in the format: Segment: [example transcript] Subheading: [example title]. Optional related subheadings from the same lecture are provided as contextual titles. The current segment transcript is then given as: Segment: [current segment transcript]. The model is required to output only the generated subheading.
Pass 2: Temporal Refinement
Visual context: <VIS_0><VIS_1>...<VIS_N>. Temporal and causal rules: every retrieved context is from a segment before the current segment. Use a retrieved context only if it helps clarify the topic transition or dependency. Ignore irrelevant retrieved contexts. Never copy a previous title as the current title. The final subheading must describe only the current segment. The Pass-1 generated title is provided as the initial draft. The model is asked to keep it unchanged if it accurately captures the main concept using correct technical terms. If the draft misses a critical technical term or the main concept is wrong, the model generates a better subheading while preserving all correct technical terms from the draft. Retrieved previous contexts are provided as previous titles with metadata such as temporal distance, source, and retrieval score. The current segment transcript is then given as: Current segment transcript: [current segment transcript]. The model is required to output only the final refined subheading.
Table 13: Prompt components used in the two-pass generation framework.
16
D
LLM Usage Statement
We utilized a large language model (LLM) to improve the grammar, clarity, and overall readability of this manuscript. The LLM’s role was strictly limited to language editing and polishing. All scientific contributions, including the core ideas, methodology, experimental design, data analysis, and conclusions, are the original work of the human authors. The use of the LLM did not alter the scientific content or its interpretation.
17