Multimodal Training with Knowledge Traction for Video Anomaly Detection

Multimodal Training with Knowledge Traction for Video Anomaly Detection Xiao Tan Xidian University, Xi’an, China

arXiv:2611.09536v1 [cs.CV] 6 Nov 2026

21171213976@stu.xidian.edu.cn

Abstract The task of weakly supervised video anomaly detection is challenging. Existing approaches mainly model video frame features and determine whether anomalies are present based on their magnitudes. In this work, we propose a novel method that incorporates external knowledge into training to improve model performance, robustness, and generalization. We leverage textual prompts and a large language model (LLM) to establish prior information for the specific task, and employ dictionary learning to normalize and compress this information. Moreover, during experiments, we observed inconsistencies in the textual descriptions extracted from similar video frames by image caption models. To address this, we devise a method that leverages an LLM and auxiliary information to ensure consistency in the textual descriptions of similar frames within a video. Finally, we introduce a feature processing approach that incorporates prior knowledge and cross-modal attention mechanisms. This approach learns the relationships and characteristics among different modalities to construct more accurate and reliable feature representations, thereby improving the accuracy and stability of downstream tasks. Experimental results demonstrate that our method surpasses other state-of-the-art approaches.

Figure 1. The overall framework of the proposed method. The blue line is the data flow of the classical method and the red line is the data flow of our method. The three modules marked with stars are the novelties of our approach.

from only video-level labels remains challenging. Existing approaches[21, 38, 39, 43] have focused on extracting discriminative features from individual frames and aggregating them to perform video-level classification. However, these methods do not explicitly model the temporal evolution of activities and often struggle to detect anomalies that persist over long durations. Recent work[29, 37] has incorporated temporal modeling using 3D ConvNets or recurrent neural networks (RNNs) over frame-level features. However, performance is still limited because these models lack sufficient supervision and tend to overfit to frequent normal activities while failing to identify infrequent anomalies effectively. Methods based solely on feature recognition or temporal information have reached a performance bottleneck. Therefore, as shown in Figure 1], we hypothesize that extracting richer information from videos and establishing prior knowledge for the targeted anomaly types, and then incorporating this knowledge as auxiliary supervision during training, would further improve the performance and generalization capability of the model. In this work, we aim to improve weakly supervised anomaly detection by incorporating external knowledge from textual descriptions to provide richer supervision. Specifically, we leverage recent advances in large language models (LLMs) to generate textual prompts that describe

1. Introduction Anomaly detection in videos is a critical task with applications across many domains such as surveillance, healthcare, and autonomous driving[10, 24]. The goal is to identify irregular events that deviate from normal patterns in video data. However, annotated real-world video datasets containing anomalies are limited due to the prohibitive cost of exhaustive manual annotation. This has led to increasing interest in weakly supervised anomaly detection[13], where models are trained on videos with only video-level labels indicating the presence of anomalies instead of frame-level or pixel-level annotations. While weakly supervised methods reduce annotation costs, modeling normal activities and detecting anomalies 1

normal activities and anomalies for a given dataset[46]. These prompts prime the model with high-level semantic information about the appearance and evolution of events beyond what is contained in the video-level labels. However, directly incorporating raw prompt embeddings as additional input can be inefficient and redundant. Hence, we propose to first normalize the prompt embeddings using dictionary learning techniques[40] like Sparse Coding[17] and Dense Associative Memory[15]. This allows compressing the high-dimensional prompt embeddings into a compact dictionary of codewords that capture the essential semantic concepts. The codeword activations for a given prompt serve as its normalized representation. More formally, let pi ∈ Rd be the embedding for prompt i obtained from the LLM, where d is the dimensionality. We learn a dictionary D ∈ Rd×K with K codewords that sparsely represents the prompt embeddings: pi ≈ Dai ,

||ai ||0 ≪ K

streams extract spatio-temporal features from frame sequences using pre-trained visual models[4]. The text stream encodes the prompt representation ai and cdi,T through an embedding layer followed by modeling with BART[18]. We introduce cross-modal attention mechanisms that allow bi-directional interaction between the modalities. The text stream attends to relevant normal and anomalous patterns in the video streams using a text-visual attention. Conversely, the video streams focus on regions described in the text using a visual-text attention. This multi-stage attention exchange enables deeply fused integration of the complementary modalities. After that, the prior knowledge further optimizes the fused features by increasing or suppressing the expressiveness of different regions. Finally, the fused features are passed through classification layers to predict frame-level anomalies. The crossmodal architecture is trained end-to-end using the video labels only. We empirically demonstrate significant gains over state-of-the-art approaches on multiple benchmark datasets. In summary, our key contributions are:

(1)

where ai ∈ RK is the sparse code for prompt i containing mostly zero activations, except for a few codewords relevant to the semantic content. We can compute the code activations efficiently using sparse coding algorithms like orthogonal matching pursuit. The code ai serves as the normalized prompt representation that will be provided as auxiliary input to the detection model. Furthermore, during our experiments, we discovered a significant inconsistency issue in the scene text descriptions extracted from video frames, leading to ambiguity and distortion. This directly impacts the algorithm’s ability to comprehend and recognize video content. To address this, we propose a novel consistency regularization method. For each video, we first obtain frame-level image captions using a pre-trained image captioning model such as BLIP[19, 20]. We then enforce consistency between frames with high similarity in the video by further using the LLM to generate the description from the original captions and extra supporting information generated by other models. Concretely, if ci,1 , ci,2 , ...ci,T are the frame-level captions for video i and di,1 , di,2 , ...di,T are extra supporting information gained by other models such as YOLO, we optimize:

cdi,1 , ..., cdi,T = LLMθ (ci,1 , ..., ci,T , di,1 , ..., di,T )

• Leveraging LLMs to generate informative textual prompts as external knowledge and normalizing textual prompts via dictionary learning for efficient modeling • Enforcing caption consistency using LLMs and extra supporting information • Cross-modal neural architecture with bi-directional attentive fusion • Demonstrating improved anomaly detection on two benchmark datasets

2. Related Work This section reviews prior work on weakly supervised video anomaly detection and discusses relevant research on leveraging large language models and self-supervised dictionary learning for this task.

2.1. Weakly Supervised Video Anomaly Detection Early work on weakly supervised video anomaly detection focused on handcrafted features and shallow models. For example, Lu et al. [25] extracted histogram of oriented gradients and optical flow features to train a multiple instance learning model. Chen et al. [5] proposed an autoencoder trained on normal video frames to detect anomalies as frames with high reconstruction error. More recent methods have incorporated deep learning, training convolutional neural networks (CNNs) on video-level labels. Zhong et al. [47] introduced a graph convolutional network over sparse object detections. Nguyen et al. [30] proposed an autoregressive network with a composite memory module. Other approaches have incorporated self-supervision by pretraining on proxy tasks before fine-tuning on anomaly

(2)

where cdi,T is the textual description after LLM correction and θ denotes the LLM parameters. This regularization method corrects erroneous raw text descriptions with additional information. For modeling, we propose a novel cross-modal architecture that leverages the normalized prompt embeddings. Our model consists of three branches: an RGB stream, an optical flow stream[9], and a text stream. The RGB and flow 2

detection. Gong et al. [11] pretrained a memory module on frame order verification. Park et al. [31] used a number of pretext tasks like rotation prediction. Contrastive predictive coding has also been effective for self-supervised pretraining [22]. More sophisticated techniques include adversarial learning [14] and perturbation-based regularization [28]. Overall, weakly supervised anomaly detection remains challenging, and existing methods typically suffer from high false positive rates. Incorporating external knowledge is a promising direction to improve performance.

approach can construct a dictionary tailored to the normal data without manual labels. The dictionary can regularize models to focus on normal patterns and detect anomalies as outliers.

3. Our Approach 3.1. Generation of Prior Knowledge via LLM

2.2. Large Language Models Large language models (LLMs) based on the Transformer architecture [41] have driven rapid progress in natural language processing. Models like BERT[7], GPT[33], and T5[34] have achieved strong performance on a wide range of NLP tasks through self-supervised pretraining on large corpora. More recently, even larger LLMs with over 100 billion parameters like GPT-3[2] have shown the ability to perform few-shot and zero-shot learning. By formulating tasks as natural language prompts, GPT-3 can accomplish them without any gradient updates or fine-tuning. This suggests LLMs may serve as rich sources of world knowledge that can be tapped for downstream applications like video anomaly detection. Methods to effectively incorporate LLMs include prompting [23] and adapter tuning [12]. Prompting formulates the task as a text completion problem, while adapter tuning adds small task-specific modules to retain model capabilities. Both approaches allow leveraging the knowledge within LLMs without extensive fine-tuning.

Figure 2. Flowchart of the prior knowledge generation process. The LLM is inspired to generate knowledge in this domain by human-written domain-specific text prompts, which should include both normal-related and abnormal-related knowledge. The feature representation of the compressed knowledge is then obtained through text encoding and dictionary learning.

An essential aspect of weakly supervised video anomaly detection is to incorporate external knowledge into the training process. As shown in Figure 2], we propose a method for generating prior knowledge using an LLM to provide informative textual prompts related to different types of anomalous scenarios. We then utilize a pre-trained language model, such as BART[18], to embed the textual knowledge, followed by dictionary learning for feature purification and compression. The resulting compact and non-redundant feature representation serves as an external knowledge base for the subsequent training process.

2.3. Self-Supervised Sparse Dictionary Learning Dictionary learning aims to find a representative set of patterns (a dictionary) to reconstruct inputs. Early dictionary learning methods were unsupervised [1], but recent work has incorporated self-supervision. Shao et al. [36] proposed a bootstrapping framework that alternates between dictionary construction and self-label assignment. The dictionary is initialized by clustering features, then used to assign pseudo-labels for self-supervision. This framework allows joint optimization of the dictionary and self-supervision. Other methods like CURL [16] and SwAV [3] have incorporated dictionary learning objectives into contrastive self-supervised learning. The key idea is to map representations to a sparse code over a learnable dictionary. This provides a form of quantization that improves contrastive learning. For video anomaly detection, dictionary learning can help extract common patterns from normal frames, while anomalies yield dense reconstructions. A self-supervised

3.1.1 LLM-Based Knowledge Generation Given a video anomaly detection dataset, we first leverage GPT-3[2] to generate textual knowledge with taskspecific prompts, which provide high-level semantic descriptions conveying relevant information about both normal and anomalous activities in videos. For example, suppose we have the following query for a dataset targeting "vehicle accidents": "Generate a textual description for a video sequence containing normal traffic flow and a de3

scription for a video sequence containing vehicle accidents."

video by leveraging additional information from an object detection model. The inconsistency arises from the limitations of the image caption model in capturing fine-grained details and context in video frames. To mitigate this problem, we propose an approach that incorporates the output of the object detection model along with the frame descriptions and jointly processes them using an LLM. The LLM’s powerful knowledge representation and logical reasoning capabilities enable us to correct and maintain consistency between the textual descriptions and the visual content of the frames. Let Fi be a set of frames within a video, and T (Fi ) be the corresponding textual descriptions generated by BLIP[19]. We aim to incorporate the output of YOLOv8[35], denoted as D(Fi ), which provides more accurate and detailed information about the objects present in the frames. We denote the corrected textual descriptions as T̂ (Fi ), which are generated by the LLM as follows:

Then, GPT-3 generates two textual descriptions: one describing normal traffic flow and the other an anomalous sequence featuring vehicle accidents. These generated texts help exploit the knowledge encapsulated in the LLM and, in turn, improve anomaly detection. Formally, for the aforementioned query, it provides prompts cnormal and canomaly . 3.1.2 Knowledge Embedding via Dictionary Learning After obtaining the textual descriptions of the generated knowledge, BART[18] is first used to embed the entire generated text and obtain fnormal and fanomaly . Since directly incorporating the raw embeddings can be computationally expensive and potentially redundant, we process these embeddings using dictionary learning. This compresses highdimensional embeddings into a concise dictionary of codewords that effectively capture the core semantic content. Let ei ∈ Rd be the embedding for the textual prompt i (fnormal or fanomaly ) obtained from GPT-3 and BART, where d is the dimensionality. Next, we train a dictionary D ∈ Rd×K with K codewords to sparsely represent the knowledge embeddings:

T̂ (Fi ) = LLM(T (Fi ), D(Fi ))

(4)

In Equation (4), LLM(·) denotes the LLM function. It takes the original textual descriptions T (Fi ) and the object detection results D(Fi ) as input and outputs the corrected descriptions T̂ (Fi ).

1 min ∥ei − DA∥2F + λ∥A∥1 (3) A 2 where A is a coding matrix representing the sparse coding of each sample over the dictionary, ∥ · ∥F denotes the Frobenius norm, ∥ · ∥1 denotes the L1 norm, and λ is a regularization parameter controlling the sparsity of the encoding.

3.3. Cross-Modal Attention with Prior Knowledge This section details our cross-modal attention mechanism, which leverages prior knowledge obtained from external textual prompts. Its goal is to enable bidirectional interaction between the visual and textual modalities, allowing them to mutually enhance their representations based on the semantic information provided by the normalized textual prompts. The architecture consists of three branches: the RGB stream, the optical flow stream, and the text stream. These branches process video frames, optical flow data, and the normalized textual prompts, respectively. Figure 4 provides an overview of the proposed architecture. The RGB stream and optical flow stream are responsible for extracting spatio-temporal features from the video frames. These streams utilize pre-trained visual models, such as I3D[4], to capture the visual patterns and motion information present in the video sequences. The text stream is encoded by BART[18] and used to complement visual information and facilitate integration with prior knowledge in subsequent processes. Next, textual and visual features are fed into our Visual-Text & Text-Visual Attention Module for deep feature fusion. Subsequently, the fused features are augmented or suppressed in different regions using prior knowledge, which guides further optimization of the feature representation.

3.2. Maintaining Consistency in Frame Descriptions

Figure 3. With additional scene information provided by the object detection model, the LLM corrects incorrect frame descriptions generated by BLIP.

As shown in Figure 3], we address the issue of inconsistency in textual descriptions extracted from frames within a 4

Figure 4. Overview of the feature processing method. After the features of different modalities are encoded and concatenated, a bidirectional attention module performs deep feature extraction. The processed features are then optimized by the Knowledge Traction module and used for anomaly detection.

is that the features obtained in the first stage are used to generate Qt ,

3.3.1 Visual-Text Attention As shown in Figure 5](a), the visual and textual features first enter the "Visual-Text Attention" module. The visual features are downsampled by a convolution operation and produce the query (Qv ) after global average pooling and mapping. In contrast, the text features are passed through N independent 1D convolutional layers for feature compression to reduce computational complexity, producing the key (Kti ) and value (Vti ). Independent convolutional layers also allow the model to learn from the text features from multiple perspectives, improving generalization. Finally, Kti , Vti , and Qv undergo the following attention operation:

Ft|v = Concati∈[1,N ]

Qv × Kti √ dk

Qt =

· Vti

i Conv1D(Ft|v )

i=1

and the initial visual features are used to generate Kv and Vv . Since only a coarse feature fusion is performed in the first stage, the visual elements of its output features occupy only a small portion. Therefore, Qt can be considered as a representation of textual features in the second stage of the attention operation. Meanwhile, since it also fuses visual features, the second fusion stage is easier to converge and helps align visual and textual features. In Figure 5](a), the merged feature Fv|t = Attention(Qt , Kv , Vv ) is reshaped and upsampled to restore the original dimensionality. Then, we design a shuffling method to shift different regions of different channels in the feature. We believe this enhances the model’s ability to learn temporal and spatial information. Finally, a convolution operation produces Ff used .

!

T

N X

(5)

Unlike the classical Transformer[41], we concatenate the output features of all branches and do not use an additional fully connected layer. This coarse attention operation preserves the features of the original modalities to a greater extent and helps the subsequent process perform deeper feature fusion.

3.3.3 Knowledge Traction Feature Fusion After deep feature fusion, the resulting features are guided and optimized using prior knowledge, as shown in Figure 5](b). First, the feature vectors of normal and anomalous prior knowledge, Kn and Ka , pass through a shared FFN, F F Nshare , to extract common features. The resulting vectors are then summed with GAP (Ff used ) and fed

3.3.2 Text-Visual Attention The attention mechanism in the "Text-Visual Attention" module is similar to that in the first stage. The difference 5

3.4. Optimization Process Our optimization objective is adapted from RTFM[39] with modifications for our method and consists of three parts: Lcls , Lsep , and Llimit . Lcls denotes the snippet-level classification loss, which can be expressed as Lcls (p = fθ (Ff inal ), y) = αBCE({p}k , y) + BCE(p̂, y) (a) In the first stage, visual-text attention performs a coarse fusion of features. In the second stage, text-visual attention fine-tunes the fusion of features.

, where fθ denotes the classifier, {·}k denotes the k highest magnitudes in the feature vector, k = 5, and p̂ denotes the remaining features. Lsep is responsible for the separability of normal and abnormal samples. The P distance between P features is de+ − ∥{f } ∥ − fined as Dk (f + , f − ) = k k 2 k ∥{f }k ∥2 , + and the constraint can be described as Lcons (f , f − ) = |m − D(f + , f − )| + Dλ∈(+,−) ({f λ }k , {f λ }k′ ). The final loss is n a n a Lsep = Lcons (Ft|v , Ft|v ) + Lcons (Fv|t , Fv|t )

, where ∥ · ∥2 denotes the L2 norm. {·}k and {·}k′ denote the k largest and k smallest magnitudes in the features, respectively. Llimit constrains the degree to which prior knowledge bootstraps the features, preventing excessive bootstrapping from distorting the features. It is defined as

(b) Amplify or suppress the different dimensions of the fused features based on prior knowledge, and further optimize the expressiveness of the features.

Figure 5. Module implementation details. (a) The bidirectional attention module. (b) The Knowledge Traction module.

Llimit =

X

′

Fχ′ − Fχ

χ∈(n,a)

into F N Nn and F N Na to extract discrepant features, producing KVn and KVa . This process can be described by the following equation:

where · denotes the mean of the feature. Finally, the overall objective is L = λ1 ·Lcls +λ2 ·Lsep + λ3 · Llimit . We set λ1 , λ2 , and λ3 to 1.0, 0.01, and 0.005, respectively.

KVn∥a = F N Nn∥a (F N Nshare (Kn∥a ) + GAP (Ff used ))

4. Experiment 4.1. Experimental Setup

Then, a sigmoid function is applied to KVn and KVa to represent the degree of adaptation of Ff used to Kn and Ka . Ff used is scaled according to this degree of adaptation to ′ ′ obtain Fn guided by normal prior knowledge and Fa guided by anomalous prior knowledge, respectively. Finally, the features with high activation in different time domains are ′ ′ selected from Fn and Fa and concatenated to obtain Ff inal . ′

Fn∥a = σ(KVn∥a ) × Ff used ′

′

Ff inal = ∃>λ (Fn ) ⋊ ⋉ ∃>β (Fa )

4.1.1 Datasets and Evaluation We conducted extensive experiments on the ShanghaiTech[26] and UCF-Crime[38] datasets and compared many previous state-of-the-art methods[6, 8, 21, 27, 32, 38, 39, 42–45, 47]. We use the area under the frame-level receiver operating characteristic curve (AUC) as the evaluation metric for all methods.

(6) 4.1.2 Data Processing

(7)

We use I3D[4] to extract RGB and optical flow features from videos, and BLIP[19] to generate frame descriptions, which are processed according to our approach. We generate textual prior knowledge for each of the two datasets using GPT-3[2] and encode all text data using BART[18].

In Equations 6 and 7, σ represents the sigmoid function. ∃>χ (·) denotes selecting the part of the features whose intensity is greater than χ. ⋊ ⋉ denotes the concatenation of the selected features while preserving their original order. 6

4.1.3 Hyperparameter and Training

Supervision

We generate 5,000 normal and anomalous priors separately for each dataset and compress them into a 256-dimensional tensor using dictionary learning. All textual content is embedded as a 2048-dimensional tensor. We also set the number of convolutional groups in the attention module, N , to 8. Upsampling is performed using bilinear interpolation, and the channel dimensions are adjusted using a 1 × 1 convolution. For the last dilated convolution, we empirically set the dilation coefficient to 4. All experiments are conducted on an NVIDIA RTX 3090 GPU. Each model is trained for 15,000 iterations with a learning rate of 0.001. A more detailed experimental setup can be found in the Appendix.

Semi-supervised

4.2. Comparison With State-of-the-Art Methods

Table 1. Performance comparison of state-of-the-art methods on the ShanghaiTech dataset.

As shown in Tables 1 and 2, our method outperforms other state-of-the-art methods, including semi-supervised and weakly supervised works, on the ShanghaiTech and UCF-Crime datasets.

Supervision

Weakly-supervised

Semi-supervised

Weakly-supervised

Figure 6. Visualization of models’ performance on UCF-Crime. The results show that our model has better accuracy and stability.

Method Mem-AE DLAN-AC SVD-GAN MIL-Rank GCN CLAWS AR-Net MIST CRFD RTFM MSL S3R UML Ours

Method Conv-AE S-SVDD GODS MIL-Rank GCN MIST RTFM DDL MSL S3R MGFN MGFN UML Ours

Feature C3Drgb T SNrgb C3Drgb I3Drgb+f low I3Drgb I3Drgb I3Drgb V ideoSwinrgb I3Drgb XCLIPrgb I3Drgb+f low

Feature T CNrgb C3Drgb T SNrgb I3Drgb I3Drgb I3Drgb I3Drgb I3Drgb V ideoSwinrgb I3Drgb XCLIPrgb I3Drgb+f low

AUC (%) 71.2 74.7 78.42 86.3 84.44 89.67 91.24 94.83 97.48 97.21 97.32 97.48 96.78 98.17

AUC (%) 50.6 58.5 70.46 75.41 82.12 82.3 84.3 85.12 85.3 85.99 86.67 86.98 86.75 87.11

Table 2. Performance comparison of state-of-the-art methods on the UCF-Crime dataset.

Specifically, our method improves performance by 7% over AR-Net[42], which also uses optical flow information. Furthermore, our method achieves the best performance among all methods using I3D features. We believe this is because the visual and textual modalities complement each other during multimodal training, allowing the model to use the features more effectively. S3R[43] also uses dictionary learning to clean up normal features to guide the model to better discriminate abnormal features. Unlike them, we use dictionary learning to compress the prior knowledge and let the model optimize the features for the scene under the guidance of the prior knowledge to improve performance. In Figure 6], we can see that our method performs better than other methods in terms of stability and accuracy, especially the results between adjacent normal frames are more continuous and stable. This may be because we maintain the consistency of text modalities across similar frames. More visualizations can be found in the Appendix. Due to space limitations, please refer to the Appendix for more experimental details and visualizations.

4.3. Ablation Studies We also conducted a series of ablation experiments to study the effects of different modules and parameter settings on our method. Table 3 shows the performance impact of the different modules in our method. When only Text-Visual Attention is used, performance improves significantly. However, when Knowledge Traction is introduced without Visual-Text Attention, performance deteriorates. We believe this is because the output of Text-Visual Attention is largely a visual feature and does not fit well with the textual prior knowledge. In contrast, after introducing Visual-Text Attention, the text modality participates more in feature fusion through attention, so the final fused features are better aligned with the textual prior knowledge and Knowledge Traction plays a leading role. We also investigate the effect of the number of prior knowledge items generated by the LLM. As shown in Table 7

5. Conclusion

4, we conduct experiments with different numbers of prior knowledge items and also investigate model performance when the amounts of normal and abnormal knowledge are unbalanced. The results show that when the amount of prior knowledge is small, performance degrades, possibly because the Knowledge Traction module cannot optimize the features correctly. On the other hand, if the amount of prior knowledge is too large, performance decreases slightly. We found that when too much prior knowledge is generated, many descriptions are repetitive, which introduces noise and affects model performance. In addition, the results show that unbalanced amounts of normal and abnormal knowledge negatively affect the model. Clearly, the Knowledge Traction module is also sensitive to this imbalance. Based on these results, we set the number of generated prior knowledge items to 5,000. Table 5 presents our exploration of the number of convolutions in the attention module. Experiments show that our method achieves the best performance when N = 8. Too small a number can lead to insufficient feature fusion, while too large a number can lead to overfitting and reduced generalization. Baseline 4 4 4 4 4

VTA 8 8 8 4 4

TVA 8 4 4 4 4

KT 8 8 4 8 4

ShanghaiTech 92.36 97.71 93.04 97.66 98.17

In this work, we proposed a novel approach to weakly supervised video anomaly detection that incorporates external knowledge generated by a large language model and exploits cross-modal attention mechanisms. In addition, we addressed the issue of inconsistency in textual descriptions and proposed a cross-modal architecture with bidirectional attentive fusion that effectively integrates visual and textual information. Experimental results on benchmark datasets show that our approach outperforms other state-of-the-art methods. Overall, our approach provides a promising direction for advancing weakly supervised video anomaly detection using external knowledge and cross-modal modeling.

References [1] Michal Aharon, Michael Elad, and Alfred Bruckstein. Ksvd: An algorithm for designing overcomplete dictionaries for sparse representation. IEEE Transactions on signal processing, 54(11):4311–4322, 2006. [2] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901, 2020. [3] Mathilde Caron, Ishan Misra, Julien Mairal, Priya Goyal, Piotr Bojanowski, and Armand Joulin. Unsupervised learning of visual features by contrasting cluster assignments. Advances in neural information processing systems, 33:9912– 9924, 2020. [4] Joao Carreira and Andrew Zisserman. Quo vadis, action recognition? a new model and the kinetics dataset. In proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 6299–6308, 2017. [5] Tingting Chen, Xueping Liu, Bizhong Xia, Wei Wang, and Yongzhi Lai. Unsupervised anomaly detection of industrial robots using sliding-window convolutional variational autoencoder. IEEE Access, 8:47072–47081, 2020. [6] Yingxian Chen, Zhengzhe Liu, Baoheng Zhang, Wilton Fok, Xiaojuan Qi, and Yik-Chung Wu. Mgfn: Magnitudecontrastive glance-and-focus network for weakly-supervised video anomaly detection. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 387–395, 2023. [7] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018. [8] Jia-Chang Feng, Fa-Ting Hong, and Wei-Shi Zheng. Mist: Multiple instance self-training framework for video anomaly detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 14009– 14018, 2021. [9] Denis Fortun, Patrick Bouthemy, and Charles Kervrann. Optical flow modeling and computation: A survey. Computer Vision and Image Understanding, 134:1–21, 2015. [10] Shubham Garg, Ramesh Kumar Aggarwal, and Sanjay Sofat. An overview of video anomaly detection techniques: A

UCF-Crime 83.31 86.33 82.69 86.74 87.11

Table 3. Impact of different modules of our approach on performance. VTA stands for Visual-Text Attention, TVA for TextVisual Attention, and KT for Knowledge Traction.

Normal Count 2500 2500 5000 5000 10000

Anomaly Count 2500 5000 2500 5000 10000

ShanghaiTech 97.41 97.83 97.28 98.17 97.95

UCF-Crime 85.97 86.59 86.41 87.11 86.93

Table 4. Influence of the number of prior knowledge items on performance.

Number of Convolutions 1 2 4 8 16

ShanghaiTech 96.23 97.19 97.64 98.17 97.78

UCF-Crime 84.89 85.76 86.88 87.11 86.72

Table 5. Effect of the number of convolutions in the attention module on performance.

8

comprehensive survey. Artificial Intelligence Review, 53(8): 5459–5517, 2020. [11] Dong Gong, Lingqiao Liu, Vuong Le, Budhaditya Saha, Moussa Reda Mansour, Svetha Venkatesh, and Anton van den Hengel. Memorizing normality to detect anomaly: Memory-augmented deep autoencoder for unsupervised anomaly detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 1705–1714, 2019. [12] Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. Parameter-efficient transfer learning for nlp. In International Conference on Machine Learning, pages 2790–2799. PMLR, 2019. [13] Minqi Jiang, Chaochuan Hou, Ao Zheng, Xiyang Hu, Songqiao Han, Hailiang Huang, Xiangnan He, Philip S Yu, and Yue Zhao. Weakly supervised anomaly detection: A survey. arXiv preprint arXiv:2302.04549, 2023. [14] Wei Jin, Yaxing Li, Han Xu, Yiqi Wang, Shuiwang Ji, Charu Aggarwal, and Jiliang Tang. Adversarial attacks and defenses on graphs. ACM SIGKDD Explorations Newsletter, 22(2):19–34, 2021. [15] Dmitry Krotov and John J Hopfield. Dense associative memory for pattern recognition. Advances in neural information processing systems, 29, 2016. [16] Michael Laskin, Aravind Srinivas, and Pieter Abbeel. Curl: Contrastive unsupervised representations for reinforcement learning. In International Conference on Machine Learning, pages 5639–5650. PMLR, 2020. [17] Honglak Lee, Alexis Battle, Rajat Raina, and Andrew Ng. Efficient sparse coding algorithms. Advances in neural information processing systems, 19, 2006. [18] Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Ves Stoyanov, and Luke Zettlemoyer. Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. arXiv preprint arXiv:1910.13461, 2019. [19] Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation. In International Conference on Machine Learning, pages 12888– 12900. PMLR, 2022. [20] Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. arXiv preprint arXiv:2301.12597, 2023. [21] Shuo Li, Fang Liu, and Licheng Jiao. Self-training multisequence learning with transformer for weakly supervised video anomaly detection. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 1395–1403, 2022. [22] Kangning Liu, Yiqiu Shen, Nan Wu, Jakub Chł˛edowski, Carlos Fernandez-Granda, and Krzysztof J Geras. Weaklysupervised high-resolution segmentation of mammography images for breast cancer diagnosis. Proceedings of machine learning research, 143:268, 2021. [23] Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. Pre-train, prompt, and

predict: A systematic survey of prompting methods in natural language processing. ACM Computing Surveys, 55(9): 1–35, 2023. [24] Yuhao Liu, Yuxin Liu, Zhiyong Zhang, Jie Li, Zhenbo Li, and Fei-Yue Wang. Deep video anomaly detection: A survey. IEEE Transactions on Intelligent Transportation Systems, 2021. [25] Cewu Lu, Jianping Shi, and Jiaya Jia. Abnormal event detection at 150 fps in matlab. In Proceedings of the IEEE international conference on computer vision, pages 2720–2727, 2013. [26] Weixin Luo, Wen Liu, and Shenghua Gao. A revisit of sparse coding based anomaly detection in stacked rnn framework. In Proceedings of the IEEE international conference on computer vision, pages 341–349, 2017. [27] Hui Lv, Zhongqi Yue, Qianru Sun, Bin Luo, Zhen Cui, and Hanwang Zhang. Unbiased multiple instance learning for weakly supervised video anomaly detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8022–8031, 2023. [28] Romero Morais, Vuong Le, Truyen Tran, Budhaditya Saha, Moussa Mansour, and Svetha Venkatesh. Learning regularity in skeleton trajectories for anomaly detection in videos. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 11996–12004, 2019. [29] M Murugesan and S Thilagamani. Efficient anomaly detection in surveillance videos based on multi layer perception recurrent neural network. Microprocessors and Microsystems, 79:103303, 2020. [30] Duc Tam Nguyen, Zhongyu Lou, Michael Klar, and Thomas Brox. Anomaly detection with multiple-hypotheses predictions. In International Conference on Machine Learning, pages 4800–4809. PMLR, 2019. [31] Hyunjong Park, Jongyoun Noh, and Bumsub Ham. Learning memory-guided normality for anomaly detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 14372–14381, 2020. [32] Yujiang Pu and Xiaoyu Wu. Locality-aware attention network with discriminative dynamics learning for weakly supervised anomaly detection. In 2022 IEEE International Conference on Multimedia and Expo (ICME), pages 1–6. IEEE, 2022. [33] Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, et al. Improving language understanding by generative pre-training. OpenAI Technical Report, 2018. [34] Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. The Journal of Machine Learning Research, 21(1):5485–5551, 2020. [35] Dillon Reis, Jordan Kupec, Jacqueline Hong, and Ahmad Daoudi. Real-time flying object detection with yolov8. arXiv preprint arXiv:2305.09972, 2023. [36] Shuai Shao, Lei Xing, Wei Yu, Rui Xu, Yan-Jiang Wang, and Bao-Di Liu. Ssdl: Self-supervised dictionary learning. In 2021 IEEE International Conference on Multimedia and Expo (ICME), pages 1–6. IEEE, 2021.

9

[37] Wonsup Shin, Seok-Jun Bu, and Sung-Bae Cho. 3dconvolutional neural network with generative adversarial network and autoencoder for robust anomaly detection in video surveillance. International Journal of Neural Systems, 30(06):2050034, 2020. [38] Waqas Sultani, Chen Chen, and Mubarak Shah. Real-world anomaly detection in surveillance videos. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 6479–6488, 2018. [39] Yu Tian, Guansong Pang, Yuanhong Chen, Rajvinder Singh, Johan W Verjans, and Gustavo Carneiro. Weakly-supervised video anomaly detection with robust temporal feature magnitude learning. In Proceedings of the IEEE/CVF international conference on computer vision, pages 4975–4986, 2021. [40] Ivana Tošić and Pascal Frossard. Dictionary learning. IEEE Signal Processing Magazine, 28(2):27–38, 2011. [41] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017. [42] Boyang Wan, Yuming Fang, Xue Xia, and Jiajie Mei. Weakly supervised video anomaly detection via centerguided discriminative learning. In 2020 IEEE international conference on multimedia and expo (ICME), pages 1–6. IEEE, 2020. [43] Jhih-Ciang Wu, He-Yen Hsieh, Ding-Jie Chen, Chiou-Shann Fuh, and Tyng-Luh Liu. Self-supervised sparse representation for video anomaly detection. In European Conference on Computer Vision, pages 729–745. Springer, 2022. [44] Peng Wu and Jing Liu. Learning causal temporal relation and feature discrimination for anomaly detection. IEEE Transactions on Image Processing, 30:3513–3527, 2021. [45] Muhammad Zaigham Zaheer, Arif Mahmood, Marcella Astrid, and Seung-Ik Lee. Claws: Clustering assisted weakly supervised learning with normalcy suppression for anomalous event detection. In Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXII 16, pages 358–376. Springer, 2020. [46] Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, et al. A survey of large language models. arXiv preprint arXiv:2303.18223, 2023. [47] Jia-Xing Zhong, Nannan Li, Weijie Kong, Shan Liu, Thomas H Li, and Ge Li. Graph convolutional label noise cleaner: Train a plug-and-play action classifier for anomaly detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 1237–1246, 2019.

10

Appendix

Algorithm A2 Algorithm for maintaining consistency in frame descriptions Input P rompt: Human-written prompt text for fixing errors in description Input F rames: Pending video frames Parameter LLM : Large language model Parameter BLIP : Pre-trained model for image captioning Parameter Y OLO: Pre-trained object detection model Output: Corrected video frame description text

A. Algorithm Description This section provides a pseudocode description of the proposed method. Algorithm A1 Algorithm for prior knowledge generation Input P romptn ,P rompta : Human-written prompt text Parameter LLM : Large language model Parameter Encoder: Pre-trained models for text embedding Parameter Nn : Number of normal prior knowledge items to generate Parameter Na : Number of anomalous prior knowledge items to generate Output: Features of prior knowledge

1: Let i = 0,Corrected 2: while i < len(F rames) do 3: description = BLIP (F rames[i]) 4: extra = Y OLO(F rames[i]) 5: Corrected 6: .add(LLM ([P rompt, description, extra])) 7: i+ = 1 8: end while 9: return Corrected

1: Let i = 0,j = 0 2: Let normalT exts, anomalyT exts 3: while i < Nn do 4: normalT exts.add(LLM (P romptn )) 5: i+ = 1 6: end while

relatively simple, consisting of campus pedestrian streets. In these scenes, normal behavior is usually walking, while other behaviors such as biking or playing are abnormal. Given these dataset characteristics, we carefully design the prompt text to encourage the LLM to generate descriptions that satisfy the requirements of prior knowledge as much as possible. We also emphasize randomness in the prompt text, which maximizes the variety of generated content.

7: while j < Na do 8: anomalyT exts.add(LLM (P rompta )) 9: j+ = 1 10: end while 11: return Encoder(normalT exts) 12: Encoder(anomalyT exts)

Algorithm A1 describes how to generate and embed prior knowledge in text form. The algorithm takes domainspecific textual prompts and generates descriptions of what is normal and what is abnormal. Because random hints are added, the LLM output has a low repetition rate, so this algorithm can generate a large amount of prior knowledge. Algorithm A2 describes how to maintain the consistency and correctness of the textual descriptions of video frames. The effectiveness of our method relies on the assumption that information produced by object detection models is more accurate. Therefore, we can use it and an LLM to correct the descriptions generated by BLIP. Algorithm A3 illustrates the cross-modal attention mechanism and how prior knowledge optimizes the features.

Table A3 shows examples generated for several anomaly types in the UCF-Crime dataset. Thanks to GPT-3’s large knowledge base and reasoning ability, and under the deliberate guidance of our prompt text, rich descriptions are generated for specific anomaly types. During prior knowledge generation, we noticed that some generated texts overlap to varying degrees. However, we did not filter these patterns because descriptions from different perspectives may increase the robustness of the prior knowledge, and dictionary learning may also mitigate the impact of excessive redundancy on performance. For the distribution of generated prior knowledge, we adopt a balanced strategy. For ShanghaiTech, since the scenes and anomalies are relatively simple, we do not subdivide anomaly types and let the LLM generate an equal number of normal and abnormal samples. For UCF-Crime, the scenes are more complex and anomaly types are more diverse, so abnormal samples account for one third of all samples, with an equal number of samples for each anomaly type. This makes the characteristics of each anomaly type more distinct and helps the model detect anomalies.

B. Examples of Prior Knowledge This section presents the generated priors for different scenarios. Due to space limitations, only a subset of the generated results is shown. Table A1 shows prior knowledge generated by GPT-3 for the ShanghaiTech dataset. The scenes in this dataset are 11

Dataset

ShanghaiTech

Prompt Text I need to detect the abnormality of a certain street in the campus. On this street, only normal walking is normal. Other situations, such as passing by bicycle, passing by fast running, etc., are abnormal. Please help me to randomly generate text descriptions of normal and abnormal situations. Note that it is randomly generated.

Normal Description "People leisurely walking down the street" "Students chatting while strolling along the sidewalk." "A group of friends walking together at a moderate pace." "Someone carrying a backpack and walking calmly." "Students walking with their hands in their pockets." "A student wearing headphones, walking at a steady pace."

Anomalous Description "A person riding a bicycle through the pedestrian area." "Someone roller-skating at a high speed down the sidewalk." "A jogger sprinting through the street segment." "Individuals playing a game of frisbee in the middle of the street." "Someone skateboarding recklessly down the sidewalk. " "A student running with a backpack on their back."

Table A1. Examples of prior knowledge generated by the LLM on the ShanghaiTech dataset.

C. Detailed Experimental Setup

Algorithm A3 Algorithm for cross-modal attention with prior knowledge Input Ft : Text features Input Fv : Visual features Parameter Fkn ,Fka : Prior knowledge features Parameter N : In the attention module, the number of convolutions in the convolution group Output: Fused features

The main text describes part of our experimental setup; due to space limitations, some details are provided here. For data processing, we extract video frames at 16 FPS and resize them to 224×224. For frames with a different aspect ratio, we use zero padding. We process these frames using I3D and embed frames from the same video into a tensor of shape (T, N, C), where T is the number of frames, N = 10 indicates 10-crop data augmentation, and C = 2048 is the feature dimension. The same scheme is used to extract optical flow features after preprocessing the video frames. For text data, after obtaining frame descriptions with our method, we use BART to embed each frame description and obtain a tensor of shape (T, C), where T is the number of video frames and C = 2048 is the feature dimension. Since video durations differ, we quantize T for both visual and text features to 32. Our textual prior knowledge is also embedded using BART, resulting in a tensor of shape (N, C), where N is the number of prior knowledge items generated by the LLM and C = 2048 is the embedding dimension. We then apply dictionary learning to obtain a tensor ′ ′ of shape (N , C), where N = 256. For model configuration, we compress both visual and textual features by a factor of 81 , reducing the dimension from 2048 to 256 to lower computational cost. We also use dropout with a rate of 0.7. In our attention module, for the coarse-grained "Visual-Text Attention", we set the FFN output dimension to 256. In contrast, for the fine-grained "Text-Visual Attention", we set the FFN output dimension to 512. We set the number of convolutions in the convolution group to 8 based on our experiments. Finally, upsampling uses bilinear interpolation, and the last dilated convolution has a dilation coefficient of 4. For training, we use the Adam optimizer with a learning

1: Let i = 0,j = 0 2: Qv = F F N (GAP (Conv2D(Fv ))) 3: while i < N do 4: Kt = F F Nk (Conv1D[i](Ft )) 5: Vt = F F Nv (Conv1D[i](Ft )) 6: Ft|v = Concat(Ft|v , Attention(Qv , Kt , Vt )) 7: i+ = 1 8: end while 9: Qt = add(Conv1D(Ft|v ), dim = 1) 10: while j < N do 11: Kv = F F Nk (GAP (Conv2D[j](Fv ))) 12: Vv = F F Nv (GAP (Conv2D[j](Fv ))) 13: Fv|t = Concat(Fv|t , 14: Reshape(Attention(Qt , Kv , Vv ))) 15: j+ = 1 16: end while 17: Fatt = DConv2D(Shuf f le(U psample(Fv|t ))) 18: raten = Sig(F F Nn (F F N (Fkn ) + GAP (Fatt ))) 19: ratea = Sig(F F Na (F F N (Fka ) + GAP (Fatt ))) 20: Ff used = Concat(Select(Fatt ∗ raten ), 21: Select(Fatt ∗ ratea )) 22: return Ff used

12

rate of 0.001 and weight decay of 0.005, without learning rate decay. We set the batch size to 32 and train for 15,000 iterations. The final model is the one with the highest performance on the validation set. The configuration of our method is summarized in Table A2. Type

Visual processing

Text processing

Model

Training

Item FPS Resize Padding Augmentation Feature dimension Feature dimension Quantize dimension Dictionary dimension Feature compression ratio Dropout ratio Coarse dimension Fine-tuned dimension Number of convolutions Upsampling Dilated coefficient Optimizer Learning rate Weight decay Batch size Iteration

In Figure A4, we visualize the features generated by the model on UCF-Crime by reducing their dimensionality. These features are labeled with 14 categories, comprising one normal label and 13 abnormal labels. The visualization shows that our model’s features exhibit improved discrimination among categories. This improvement can be attributed to the effective fusion of visual and textual information within our features. Consequently, during training, our model gradually develops a deeper understanding of the distinctions between different anomaly types. This observation further highlights the benefits of multimodal fusion, as it enables the model to extract superior features from the data. Additionally, we visualize the activation map of a sample, as shown in Figure A1. The results reveal that our method concentrates more precisely on the region where the anomaly occurs. At the same time, this concentration is accompanied by less divergence than alternative approaches. Notably, our model successfully activates certain objects within the video frame even when they occupy a relatively small area. This emphasizes the effectiveness of our model in capturing meaningful anomalies.

Value 16 224 0 10-crop 2048 2048 32 256 1 8

0.7 256 512 8 bilinear 4 Adam 0.001 0.005 32 15000

Table A2. Detailed hyperparameter configuration of our method.

D. Visualization of Experimental Results This section presents additional visualization results and analyzes them. Figures A2 and A3 show additional visualizations that demonstrate the superiority of our Knowledge Traction module. In the "Burglary092" sample, the Knowledge Traction module corrects wrongly detected frames by suppressing the activation intensity of normal frames through prior knowledge. In the "Explosion021" example, some abnormal frames are clearly recognized as normal. A possible reason is that the color of the explosion scene is too uniform, leading to inaccurate recognition. After adding the Knowledge Traction module, this issue is effectively alleviated because, with prior knowledge, the text modality guides the model to understand the scene information in the video more comprehensively during training. In contrast, some normal videos, such as "Normal_Videos_100" and "Normal_Videos_870", contain fast scene changes or abnormal-looking scenes that cause the model to fail. After introducing the Knowledge Traction module, the model still makes errors in these cases, but the magnitude of the detected anomalies is reduced. 13

Figure A1. Visualization of anomalous activation maps for some samples in the UCF-Crime dataset.

Figure A2. Visualization of the impact of the Knowledge Traction (KT) module on model performance. For abnormal videos, it improves model robustness.

Figure A3. Visualization of the impact of the Knowledge Traction (KT) module on model performance. For normal videos, it reduces the model’s sensitivity to scene changes.

Figure A4. 2D visualization of features from the UCF-Crime dataset. Comparison between our method and the baseline MIL method.

14

Scenes

Prompt Text I need to detect anomalies related to theft. The detection scenes are usually the interior scenes of shops or houses, which are rich and varied. Please help me randomly generate text descriptions of normal and abnormal scenarios.

Steal

Explosion

I need to detect anomalies about explosions. The detection scenarios are rich and diverse. The cause of the explosion could be various reasons. Please help me randomly generate text descriptions of normal and abnormal scenarios.

Normal Description "A well-lit retail store with shelves neatly arranged, customers browsing products, and employees assisting them." "A cozy living room with comfortable furniture, family members watching TV, and soft music playing in the background." "A quiet bookstore with customers peacefully reading, classical music playing, and a barista serving coffee at the instore cafe." "A bustling parking lot on a sunny day, cars coming and going as people run errands." "A quiet residential street with children playing, neighbors chatting, and birds chirping." "A serene gas station with vehicles filling up, attended by workers in their uniforms."

RoadAccidents

I need to detect anomalies about road accidents. The detection scene is generally outdoors, which is rich and varied. The types of accidents are equally diverse. Please help me randomly generate text descriptions of normal and abnormal scenarios.

"Vehicles smoothly flow along the avenue, obeying traffic signals and maintaining safe distances." "Cars are moving at a consistent speed in their respective lanes, maintaining proper following distances." "Neighbors jog and walk their dogs while cars drive by steadily, without any aggressive behavior."

Anomalous Description "A dimly lit retail store with broken shelves, items scattered on the floor, and a person acting suspiciously and concealing items in their bag." "A disheveled living room with overturned furniture, shattered glass, and signs of forced entry through a broken window." "A chaotic bookstore with books strewn all over the floor, a fire alarm blaring, and an individual stuffing books into a bag hurriedly." "A deafening explosion shatters the quietness of the parking lot, sending debris and smoke into the air." "Chaos erupts on the residential street as a building collapses due to a gas leak, causing panic and confusion." "An intense fire breaks out at the gas station, flames engulfing fuel pumps and sending plumes of black smoke into the sky." "A car suddenly speeds through red lights, narrowly missing pedestrians and causing chaos." "A vehicle hydroplanes, loses control, and collides with the median barrier." "A drunk driver swerves erratically, posing a serious threat to everyone on the road."

Table A3. Examples of prior knowledge generated by the LLM for different scenarios in the UCF-Crime dataset.

15

Related documents

• Not Your Diary, Just Your Topic: Grading the Wrong-User Control#1598914
• Alignment by Confident Debate: Preference Learning from Multi-Agent Debate#1598915
• CoMoFlow: Controllable Protein Conformational Ensemble Generation via Explicit Collective-Mode Modeling#1598916
• CID: Measuring Cognitive Work Through Semantic State Change#1598917
• PreForceVision: Physics-Guided Optical Estimation of Bolt Preload from Bolt-Head Deformation#1598919