MMVIAD: Multi-view Multi-task Video Understanding for Industrial Anomaly Detection
arXiv:2605.10833v1 [cs.CV] 11 May 2026
Xiran Zhao1∗ Jing Jin2,3∗ Yan Bai3∗ Zhongan Wang1 Yifeng Sun1 Yihang Lou4 Xuanyu Zhu4 Tao Feng2 Yingna Wu1,†† 1
ShanghaiTech University 2 Tsinghua University 3 Meituan Inc. 4 Peking University {zhaoxr2025, sunyf12025, wuyn}@shanghaitech.edu.cn [email protected] [email protected]
Abstract Industrial anomaly detection is critical for manufacturing quality control, yet existing datasets mainly focus on static images or sparse views, which do not fully reflect continuous inspection processes in real industrial scenarios. We introduce MMVIAD (Multi-view Multi-task Video Industrial Anomaly Detection), to the best of our knowledge the first continuous multi-view video dataset for industrial anomaly detection and understanding, together with a benchmark for multi-task evaluation. MMVIAD contains object-centric 2-second inspection clips with approximately 120 degrees of camera motion, covering 48 object categories, 14 environments, and 6 structural anomaly types. It supports anomaly detection, defect classification, object classification, and anomaly visible-time localization. Systematic evaluations on MMVIAD show that current commercial and open-source video MLLMs remain far below human performance, especially for fine-grained defect recognition and temporal grounding. To improve transferable anomaly understanding, we further develop a two-stage post-training pipeline where PS-SFT (Perception-Structured Supervised Fine-Tuning) initializes perception-structured reasoning and VISTA-GRPO (Visibility-grounded Industrial Structured Temporal Anomaly Group Relative Policy Optimization) refines the model with semanticgated defect reward and visibility-aware temporal reward, producing the final model VISTA. On MMVIAD-Unseen, VISTA improves the base model’s average score across the four tasks from 45.0 to 57.5, surpassing GPT-5.4. Source code is available at https://github.com/Georgekeepmoving/MMVIAD.
1
Introduction
Industrial visual inspection is rarely limited to a single static view [6, 36, 44]. In practical inspection, a camera may move around a part, or the part may rotate on a fixture, so that different surfaces are observed over a short video. For structural defects such as cracks, holes, bulges, broken regions, scratches, and concavities, the diagnostic evidence is often viewpoint-dependent: a defect can be invisible from one view, weakly visible from another, and clearly diagnosable only during a short interval of the inspection trajectory [6, 36, 44]. Figure 1 illustrates this shift from single-view and sparse multi-view anomaly detection to continuous multi-view video anomaly detection, where models must not only recognize anomaly existence, defect type, and object category, but also identify when the video provides visual evidence for the defect [33, 21, 37]. ∗ Equal contribution. † Corresponding author.
Preprint.
Existing industrial anomaly benchmarks cover important parts of this problem but do not directly evaluate this continuous multi-view visibility setting, as contrasted in Figure 1. MVTec AD and VisA established static image-level and pixel-level evaluation [6, 47]. MVTec 3D-AD, PAD, Real-IAD, and RAD extend inspection to 3D sensing, pose variation, or sparse multi-view observations [7, 45, 36, 44]. Recent video datasets such as Phys-AD further introduce temporal industrial anomaly scenarios [24]. However, these datasets generally do not provide continuous multi-view inspection videos with annotations of when viewpoint-dependent structural evidence becomes visible. As a result, a model may predict the correct label while remaining ungrounded in the diagnostic video interval.
Figure 1: Comparison of anomaly detection paradigms. From single-view and sparse multi-view detection to continuous multi-view video anomaly detection. To address this gap, we introduce MMVIAD (Multi-view Multi-task Video Industrial Anomaly Detection), a visibility-grounded video dataset and benchmark for continuous multi-view, multi-task industrial anomaly understanding. To the best of our knowledge, MMVIAD is the first continuous multi-view video dataset for industrial anomaly detection. MMVIAD is built from controllable object-centric rendering trajectories, where each 2-second clip covers approximately 120 degrees of viewpoint change [25, 45]. We use controllable rendering because accurate visible-time annotation requires aligned viewpoints and verifiable defect visibility, which are difficult to obtain reliably from real inspection videos. The dataset contains over four thousand inspection clips across 48 object categories, 14 environments, and 6 structural anomaly types. For each anomalous object, we generate two aligned videos under identical rendering conditions: an anomaly-unmarked video that renders the anomalous object without additional visual cues, and an anomaly-marked video where the defective region is highlighted in red for visibility annotation. By comparing the aligned videos and manually verifying the candidate intervals, MMVIAD obtains visible-time annotations for four coupled QA-style tasks: anomaly detection, defect classification, object classification, and anomaly visible-time localization. These four tasks are deliberately coupled. Defect classification is meaningful only after anomaly detection is correct; object identity provides geometric and semantic context for interpreting the defect; visible-time localization asks the model to bind the predicted defect to the temporal interval where its evidence appears. MMVIAD therefore evaluates whether a video MLLM can connect object-level perception, defect semantics, and temporal grounding, rather than solve four independent classification problems. To provide a reference training baseline for this setting, we further study a twostage structured post-training pipeline. The first stage, PS-SFT (Perception-Structured Supervised Fine-Tuning), follows multimodal instruction tuning to teach the model to separate whole-clip perception from localized temporal evidence [28, 10]. The second stage, VISTA-GRPO (Visibilitygrounded Industrial Structured Temporal Anomaly Group Relative Policy Optimization), refines the model with task-aware rewards inspired by recent rule-based reinforcement learning, including a semantic gate for defect classification and a visibility-aware reward for temporal localization [16, 13, 37]. The resulting model VISTA serves as our reference post-training baseline. We evaluate humans, commercial video MLLMs, and open-source video MLLMs under two MMVIAD protocols [4, 13, 37, 26, 39]. MMVIAD-Standard measures overall performance when train and test clips cover the same object categories, while MMVIAD-Unseen tests whether training transfers to unseen object categories. The results show a persistent gap between current models 2
and human annotators, especially on fine-grained defect classification and anomaly visible-time localization. On MMVIAD-Unseen, VISTA improves the Qwen3-VL-8B base model from 45.0 to 57.5 average score, with the largest gains in anomaly detection, temporal localization, and object classification. These results suggest that MMVIAD exposes a failure mode of current video MLLMs: they can often recognize coarse object or anomaly cues, but still struggle to bind defect semantics to the viewpoint interval where the evidence is available.We summarize our contributions as follows: • Dataset and benchmark. We introduce MMVIAD, to the best of our knowledge the first continuous multi-view video dataset for industrial anomaly detection, with QA-based tasks and visible-time annotations for viewpoint-dependent structural defects. • Diagnostic evaluation. We evaluate human annotators, commercial models, and opensource video MLLMs on MMVIAD-Standard and MMVIAD-Unseen, showing that defect semantics and visible-time grounding remain key bottlenecks. • Reference post-training. We propose VISTA, a two-stage post-training baseline that combines PS-SFT initialization with VISTA-GRPO reward-based refinement to improve generalization on MMVIAD-Unseen.
2
Related Work
Industrial Anomaly Detection. Industrial anomaly detection has been shaped by static-image and sparse-view benchmarks. MVTec AD, MPDD, and VisA established image-level and pixellevel evaluation [6, 19, 47], while MVTec 3D-AD, PAD, Real-IAD, and RAD extended the setting toward 3D sensing, pose variation, and multi-view observation [7, 45, 36, 44]. Recent video datasets such as Phys-AD further extend industrial anomaly detection to video scenarios [24]. In parallel, methods such as PaDiM, CutPaste, DRAEM, Reverse Distillation, PatchCore, and EfficientAD advanced distribution modeling, self-supervised reconstruction, memory-bank, and distillation-based anomaly detection [11, 22, 40, 12, 34, 5]. Recent work further connects industrial inspection with multimodal large models and reasoning-oriented post-training [15, 20, 8]. Despite this progress, existing benchmarks still mainly evaluate static images, pixel maps, sparsely sampled views, or singleview videos, rather than continuous multi-view videos where anomaly evidence may be temporally visible only under certain viewpoints. MMVIAD targets this missing setting. Video Understanding and Anomaly Visible-time Localization. Temporal video grounding localizes moments from language queries, from early methods such as DiDeMo/MCN and TALL to temporalmap and transformer-based models such as 2D-TAN and Moment-DETR [2, 14, 43, 21]. Video LMMs further expanded instruction-following video understanding [41, 29, 27, 9, 42], while timestampaware models and reinforcement-style video reasoning methods improve temporal grounding with explicit time or verifiable objectives [33, 17, 18, 31, 13, 37, 26, 39]. However, these works mainly target natural videos, human actions, or language-described events. They do not address industrial structural defects whose evidence is subtle, viewpoint-dependent, and tied to objectively visible intervals under controlled inspection. Our work adapts anomaly visible-time localization to anomaly visibility grounding in industrial videos. Multimodal Instruction Tuning and Reinforcement Learning. Multimodal instruction tuning aligns visual encoders with large language models for instruction-following visual reasoning [1, 23, 28, 10, 46, 3]. Recent post-training methods, including chain-of-thought, preference optimization, and policy-gradient reinforcement learning, further improve structured reasoning through imitation or reward-based objectives [38, 30, 32, 35, 16]. However, these techniques have not been systematically studied for continuous multi-view industrial anomaly understanding with coupled defect recognition and anomaly visible-time localization.
3
Method
3.1
MMVIAD Dataset Construction
We construct MMVIAD for multi-task, multi-view industrial anomaly understanding in continuous inspection videos. A key challenge is obtaining precise and verifiable labels for when structural anomaly evidence is visible. As illustrated in Figure 2, MMVIAD is constructed through three stages: 3
controllable video generation, visible-time annotation, and multi-task QA construction. Their framewise comparison provides candidate visible intervals, which are then manually verified to ensure that the final labels reflect the visibility of structural anomalies rather than illumination, material, or background changes. This design reduces the ambiguity and subjectivity of manually judging defect visibility in continuous videos. Table 1 shows that prior industrial anomaly benchmarks are largely image-based and do not jointly support continuous viewpoint variation and QA-based anomaly understanding.
Figure 2: MMVIAD data construction. We render continuous multi-view videos from AnomalyShapeNet point clouds, derive visible-time labels by comparing aligned marked and unmarked videos, and convert each annotated clip into four QA tasks for anomaly detection, defect classification, object classification, and visible-time localization. Table 1: Comparison of industrial anomaly detection datasets. Repr. denotes representation, Type denotes data type, Class denotes the number of object categories, Cont. denotes continuous viewpoint variation, and QA denotes question-answering-based anomaly understanding. Dataset
Venue
Repr.
Type
Class
Normal
Abnormal
View
Cont.
QA
MVTec AD [6] VisA [47] PAD [45] Real-IAD [36] RAD [44] Phys-AD [24]
CVPR’19 ECCV’22 NeurIPS’23 CVPR’24 arXiv’24 CVPR’25
image image image image image video
real real synth real real real
15 12 20 30 13 49
4,096 9,621 4,960 99,721 1,224 3,598
1,258 1,200 4,412 51,329 3,063 2,836
single single ∼20 5 68 single
✗ ✗ ✗ ✗ ✗ ✗
✗ ✗ ✗ ✗ ✗ ✗
MMVIAD
-
video
synth
48
1,410
2,613
120◦
✓
✓
Video generation. MMVIAD is built from object-centric 360◦ rendering trajectories. We use controllable rendering because anomaly visible-time localization requires frame-level visibility labels that are difficult to obtain reliably from real captured videos. The objects used in our videos are sourced from Anomaly-ShapeNet [25], a ShapeNet-based synthetic 3D anomaly dataset constructed through mesh subdivision and defect carving. We render each object in Blender at 1920×1080 and 30 fps, while randomizing floor appearance, HDRI illumination, material properties, camera viewpoints, and object poses to improve visual diversity. Although rendered videos do not fully 4
replace real industrial acquisition, they allow precise control of viewpoint trajectories, object pose, illumination, and defect geometry, making it possible to generate repeatable inspection clips and verifiable visible-time annotations. For each anomalous object, we generate two aligned videos under identical rendering conditions: one anomaly-unmarked video and one anomaly-marked video where the defective region is highlighted in red. Each complete 360◦ trajectory is then split into 2-second clips, and each clip is treated as an independent sample. Visible-time annotation. As shown in Figure 2, we annotate anomaly visibility by comparing two aligned videos for each anomalous sample. The first is an anomaly-unmarked video, where the anomalous object is rendered normally without visual highlighting. The second is an anomaly-marked video, where the defective region is highlighted in red. Both videos are rendered under identical camera motion, illumination, material, floor, and object-pose conditions. We compare the two videos frame by frame to obtain candidate intervals in which the marked defective region is visible. These candidate intervals are then manually checked and refined to ensure that the final label corresponds to the time span where the structural anomaly itself is visible, rather than visual differences caused by lighting, material reflectance, pose variation, or background interference. The resulting annotation is a visible-time interval [tstart , tend ], which serves as supervision for anomaly visible-time localization. QA-pair generation. Industrial video anomaly understanding requires solving four coupled subtasks: binary anomaly detection (Q1), conditionally dependent defect categorization (Q2), object classification (Q3), and anomaly visible-time localization (Q4). MMVIAD covers six structural anomaly types, including crack, scratch, concavity, bulge, broken, and hole. For Q1–Q3, we construct standardized question-answer pairs using the normal/abnormal labels, defect-type labels, and objectcategory labels from Anomaly-ShapeNet. For Q4, we use the visible-time annotations described above as supervision and ask models to output the time interval in which diagnostic anomaly evidence is visible, in the format [start_sec, end_sec]. With unified question templates and answer formats, MMVIAD places anomaly existence, defect semantics, object semantics, and anomaly visible-time localization under a single evaluation interface. Finally, each video clip is converted into four aligned QA instances, yielding 16,092 QA pairs in total. 3.2
PS-SFT Initialization and VISTA-GRPO Refinement
The four tasks in MMVIAD are intrinsically coupled. Defect classification is meaningful only when an anomaly is correctly detected, object classification provides semantic context for anomaly understanding, and anomaly visible-time localization requires grounding defect evidence in temporal intervals. These dependencies make naive multi-task optimization susceptible to cross-task interference, reward sparsity, and unstable structured outputs. We therefore adopt a two-stage training pipeline. PS-SFT first initializes the model with perception-structured reasoning traces, and VISTAGRPO then refines the policy with a semantically gated multi-task reward and group-relative policy optimization.
Figure 3: VISTA-GRPO reward modeling. The reward design combines format checking, answer correctness, semantic-gated defect reward, and visibility-aware temporal reward. 3.2.1
PS-SFT: Perception-Structured Supervised Fine-Tuning
Directly applying reinforcement learning to the base model is unstable, as unreliable structured outputs lead to sparse rewards and hinder early exploration. To mitigate this cold-start problem, we first construct PS-SFT to initialize the model with perception-structured reasoning behavior before reinforcement learning. 5
PS-SFT is a teacher-synthesized SFT dataset derived from the MMVIAD training split. For each training clip, we provide Gemini 3.1 Pro with ground-truth labels for all four tasks and instruct it to generate a reasoning trace in the target format: <global_perception>, <segment_perception>, <think>, and <answer>. This format decomposes each response into whole-clip perception, localized defect evidence with temporal visibility cues, brief reasoning, and final structured answers. Since generation is conditioned on ground-truth labels, the teacher mainly converts existing annotations into perception-decomposed reasoning traces rather than re-annotating the data. We discard outputs that miss required sections, violate the section order, or contain unparsable final answers. The remaining traces form DPS-SFT , the perception-structured SFT dataset used for supervised initialization. We then fine-tune the base model on DPS-SFT with teacher forcing by minimizing the standard causal language modeling loss. Given a video v, task prompt P, and target output sequence o = {ot }Tt=1 , the PS-SFT objective is: LPS-SFT = −
T X
log pθ (ot | v, P, o<t ) .
(1)
t=1
PS-SFT is not intended to maximize final task performance by itself; instead, it provides a stable initialization for subsequent VISTA-GRPO refinement. It aligns the output format, encourages perception decomposition, and primes the model with initial anomaly awareness and temporal grounding before reward-driven optimization. As illustrated in Figure 3, VISTA-GRPO combines four reward components: format checking, answer correctness, semantic-gated defect reward, and visibility-aware temporal reward. Format and Answer Rewards. The format reward preserves the perception-structured output format learned during PS-SFT, while the answer reward evaluates Q1 anomaly detection and Q3 object classification: Rfmt (o) = 1[o follows the required structure] , Rans (o) = 1[ŷano = yano ] + 1[ŷobj = yobj ] .
(2) (3)
The two terms in Rans correspond to Q1 and Q3, and the object reward is computed unconditionally because object identity is independent of defect status. Semantic-Gated Defect Reward. Defect classification corresponds to Q2, but it is not an independent classification task. In industrial inspection, predicting a defect type is meaningful only when the model first correctly detects that an anomaly exists. A plain defect classification reward may encourage the policy to exploit defect-class priors even when Q1 is incorrect. We therefore introduce a semantic-gated defect reward: 1, if ŷdef = ydef ∧ ŷano = yano , Rsg (o) = (4) 0, otherwise. This reward explicitly enforces the Q1→Q2 dependency, making defect classification contingent on valid anomaly awareness and reducing reward hacking from defect-class priors. Visibility-Aware Temporal Reward. Anomaly visible-time localization corresponds to Q4. Unlike ordinary temporal localization, this task requires the model to determine both whether visible anomaly evidence exists and when it appears. A flat IoU reward conflates different cases, such as correctly predicting an empty interval for a normal sample, missing a visible defect, and hallucinating an interval on a defect-free sample. We therefore design a visibility-aware temporal reward that decouples visibility presence from boundary precision. (k)
(k)
Let Î = {[t̂s , t̂e ]}M k=1 denote the set of predicted visible-time intervals, where M is the number (k) (k) of predicted intervals. Let I ∗ = {[ts , te ]}N k=1 denote the set of ground-truth visible-time intervals, where N is the number of annotated intervals. Thus, M = 0 means that the model predicts no visible anomaly interval, while N = 0 means that the clip has no annotated visible anomaly interval. The 6
reward is defined as: 1.0, M = 0 ∧ N = 0, αpen , M = 0 ∧ N > 0, Rvis (o) = (5) α , M > 0 ∧ N = 0, pen αbon + αiou · IoUmax (Î, I ∗ ) · e−λ|M −N | , M > 0 ∧ N > 0. The first case rewards correct empty prediction on normal samples, the second penalizes missed visible defects, and the third penalizes hallucinated intervals. When both prediction and ground truth contain visible intervals, the last case evaluates boundary quality using maximum IoU, a discovery bonus, and an interval-count penalty: |Iˆ ∩ I ∗ | IoUmax (Î, I ∗ ) = max . (6) ˆ Î, I ∗ ∈I ∗ |Iˆ ∪ I ∗ | I∈ We set αbon = 0.3, αiou = 0.7, αpen = −0.3, and λ = 0.5. The discovery bonus provides effective feedback when boundary precision is still poor, while e−λ|M −N | discourages over- and under-segmentation. During training, VISTA-GRPO samples G responses {oi }G i=1 for each prompt and computes a rulebased reward RVISTA-GRPO (oi ) from the above reward components. The advantage of each response is estimated relative to the reward distribution within the same group: RVISTA-GRPO (oi ) − mean {RVISTA-GRPO (oj )}G j=1 Ai = . (7) std {RVISTA-GRPO (oj )}G j=1 The policy is then updated with the standard GRPO objective, using the PS-SFT checkpoint as both the initial policy and the reference model.
4
Experiments
4.1
Setup
Benchmark protocols. We use two MMVIAD protocols. MMVIAD-Standard partitions clips at the sample level over the same 48 fine-grained object classes, with 2,913 training clips and 1,101 test clips, corresponding to 11,652 and 4,404 QA pairs. It is used to compare human annotators, commercial models, and open-source MLLMs. MMVIAD-Unseen trains on 36 object classes and tests on 12 unseen classes, with 2,952 training clips and 1,062 test clips, corresponding to 11,808 and 4,248 QA pairs. It is used to evaluate transferable anomaly understanding with PS-SFT and VISTA-GRPO. Human evaluation. Following MMAD [20], we sample 621 clips for human evaluation by eight participants: three experts with industrial anomaly detection experience and five ordinary participants. Each participant answers the same four QA tasks with the same visual inputs as machine baselines, and we report group averages. Training details. Unless otherwise specified, we use Qwen3-VL-8B [4] as the base model. Training uses 1008 × 560 frames with 16 uniformly sampled frames per clip, while evaluation uses 32 frames. Training takes about 6 hours on 4 NVIDIA H100 80GB GPUs. All machine baselines share the same task description, decoding setup, and answer-parsing protocol. We report classification accuracy, localization mIoU, and their arithmetic mean as Avg. 4.2
MMVIAD-Standard Benchmark Results
Since MMVIAD-Standard is designed for zero-shot evaluation, we exclude VISTA from Table 2 because it is trained on MMVIAD and thus not directly comparable to off-the-shelf MLLMs. Table 2 shows that MMVIAD-Standard remains far from saturated, with the strongest machine baseline still more than 20 points behind expert annotators in Avg. Models perform relatively better on coarse anomaly detection and object classification, but struggle with fine-grained defect classification and anomaly visible-time localization, indicating that generic video QA and multimodal reasoning pretraining are insufficient for structural anomaly understanding under continuous viewpoint changes. 7
4.3
MMVIAD-Unseen Generalization Results
Table 3 shows that MMVIAD-Unseen remains challenging for all models. VISTA achieves the best overall performance, improving the Qwen3-VL-8B base model from 45.0 to 57.5 Avg., with clear gains in anomaly detection, visible-time localization, and object classification. The smaller improvement in defect classification suggests that fine-grained defect semantics remain difficult under unseen object categories. Table 2: Results on MMVIAD-Standard. The four task columns correspond to anomaly detection (Defect Detect.), anomaly classification (Defect Class.), anomaly visible-time localization (Defect Loc.), and object classification (Object Class.). Avg. denotes the arithmetic mean over the four tasks. Type
Model
Venue
Defect
Param.
Object
Detect.
Class.
Loc.
Class.
Avg.
-
Human (expert) Human (ordinary)
-
-
91.2 87.3
77.8 67.1
82.3 78.4
93.7 91.5
86.3 81.1
Commercial
GPT-5.4 GPT-5.4-mini Gemini 3.1 Pro
-
-
64.1 68.3 69.3
37.9 32.6 41.6
50.5 46.3 48.5
73.0 64.5 87.8
56.4 52.9 61.8
Qwen3-VL [4] Qwen3-VL [4] Qwen3-VL [4] Video-R1 [13] Time-R1 [37] Time-R1 [37] VideoChat-R1 [26] VideoChat-R1.5 [39]
arXiv’25 arXiv’25 arXiv’25 NeurIPS’25 NeurIPS’25 NeurIPS’25 arXiv’25 NeurIPS’25
2B 4B 8B 7B 3B 7B 7B 7B
33.9 33.9 35.1 41.4 49.3 40.8 41.1 36.7
12.4 31.0 31.0 30.6 26.6 31.3 32.1 30.9
20.7 31.9 30.9 31.1 30.1 31.1 31.1 31.1
51.8 65.8 69.5 60.9 63.8 65.0 68.8 66.2
29.7 40.7 41.6 41.0 42.5 42.1 43.3 41.2
Open Source
Table 3: Results on MMVIAD-Unseen. Type
Model
Venue
Defect
Param.
Object
Detect.
Class.
Loc.
Class.
Avg.
-
Human (expert) Human (ordinary)
-
-
88.5 83.2
78.9 73.3
80.1 77.8
96.4 93.6
86.0 82.0
Commercial
GPT-5.4 GPT-5.4-mini Gemini 3.1 Pro
-
-
58.2 62.0 69.1
38.0 29.8 46.9
52.0 60.0 59.3
65.7 62.2 80.3
53.5 53.5 63.9
Open Source
Qwen3-VL [4] Qwen3-VL [4] Qwen3-VL [4] Video-R1 [13] Time-R1 [37] VideoChat-R1 [26] VideoChat-R1.5 [39]
arXiv’25 arXiv’25 arXiv’25 NeurIPS’25 NeurIPS’25 arXiv’25 NeurIPS’25
2B 4B 8B 7B 7B 7B 7B
32.3 35.2 39.5 49.5 44.2 41.1 41.7
26.2 33.5 36.2 31.1 35.6 34.9 34.8
27.4 37.8 37.0 48.4 48.6 48.8 45.4
54.8 56.9 67.4 57.0 56.4 56.9 55.4
35.2 40.9 45.0 46.5 46.2 45.4 44.3
Ours
VISTA (Qwen3-VL)
-
8B
60.7
37.6
49.6
81.9
57.5
Figure 4: Task-wise reward dynamics during VISTA-GRPO training. Rewards improve most clearly on anomaly detection and anomaly visible-time localization, while defect classification remains sparse and unstable, highlighting the core challenge of fine-grained industrial anomaly understanding. 8
Figure 4 illustrates the training dynamics. Rewards for anomaly detection and anomaly visible-time localization show clearer upward trends, consistent with Table 3, while defect classification fluctuates more strongly due to sparse seven-way supervision. Object classification remains stable, suggesting that VISTA-GRPO mainly refines anomaly-centric reasoning. 4.4
Ablation Studies
Figure 5 evaluates synthetic-to-real generalization on real industrial image datasets. VISTA trained on synthetic MMVIAD-Unseen improves the Qwen3-VL-8B base model from 43.8 to 67.1 average accuracy on MVTec AD and from 53.0 to 54.4 on VisA. The gain is especially large on MVTec AD, where most categories benefit from the MMVIAD-trained model, suggesting that anomaly understanding learned from synthetic multi-view videos can transfer to real inspection images.
Figure 5: Per-category synthetic-to-real generalization. Figure 6 shows a qualitative example of VISTA. The model separates global object perception from localized defect evidence and grounds the defect prediction to the visible-time interval. This illustrates that MMVIAD evaluates not only final answer correctness, but also evidence-grounded temporal understanding.
Figure 6: Qualitative case study. VISTA produces perception-structured reasoning that links objectlevel perception, localized defect evidence, and anomaly visible-time localization. Table 4 compares training paradigms under the same base model on MMVIAD-Unseen. Direct RL without supervised initialization brings limited gains, indicating unstable reward exploration before 9
the model learns structured outputs. PS-SFT substantially improves the base model, and increasing SFT data from 6K to 12K QA pairs further benefits defect classification. The best Avg. is achieved by combining PS-SFT initialization with VISTA-GRPO refinement, which improves the base model from 45.0 to 57.5 and yields a better balance across the four tasks. Appendix B further reports additional reward ablations. Table 4: Training paradigm ablation on MMVIAD. We compare different combinations of PS-SFT initialization and VISTA-GRPO refinement. DPS-SFT and DVISTA-GRPO denote the corresponding training data. PS-SFT
VISTA-GRPO
✗ ✗ ✓ ✓ ✓
5
✗ ✓ ✗ ✗ ✓
DPS-SFT
6k 12k 6k
Defect
DVISTA-GRPO
12k 6k
Object
Detect.
Class.
Loc.
Class.
39.5 39.6 53.2 55.8 60.7
36.2 35.5 37.3 42.9 37.6
37.0 38.3 46.3 48.4 49.6
67.4 70.0 81.8 75.0 81.9
Avg. 45.0 46.0 54.7 55.5 57.5
Conclusion
We presented MMVIAD, a continuous multi-view video dataset and benchmark for industrial anomaly understanding, with structured labels for defect semantics and anomaly visible-time localization. Built with controllable rendering, MMVIAD enables precise and verifiable temporal annotations that are difficult to obtain from real inspection videos. Our evaluations show that current video MLLMs still struggle with fine-grained defect recognition and temporal grounding, while VISTA improves generalization on MMVIAD-Unseen through PS-SFT initialization and VISTA-GRPO refinement. We discuss limitations and future directions in Appendix G.
References [1] Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al. Flamingo: a visual language model for few-shot learning. volume 35, pages 23716–23736, 2022. [2] Lisa Anne Hendricks, Oliver Wang, Eli Shechtman, Josef Sivic, Trevor Darrell, and Bryan Russell. Localizing moments in video with natural language. In Proceedings of the IEEE international conference on computer vision, pages 5803–5812, 2017. [3] Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond, 2023. [4] Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, et al. Qwen3-vl technical report. arXiv preprint arXiv:2511.21631, 2025. [5] Kilian Batzner, Lars Heckler, and Rebecca König. Efficientad: Accurate visual anomaly detection at millisecond-level latencies. In Proceedings of the IEEE/CVF winter conference on applications of computer vision, pages 128–138, 2024. [6] Paul Bergmann, Michael Fauser, David Sattlegger, and Carsten Steger. Mvtec ad–a comprehensive real-world dataset for unsupervised anomaly detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9592–9600, 2019. [7] Paul Bergmann, Xin Jin, David Sattlegger, and Carsten Steger. The mvtec 3d-ad dataset for unsupervised 3d anomaly detection and localization. 2021. [8] Yuhao Chao, Jie Liu, Jie Tang, and Gangshan Wu. Anomalyr1: A grpo-based end-to-end mllm for industrial anomaly detection. arXiv preprint arXiv:2504.11914, 2025. 10
[9] Zesen Cheng, Sicong Leng, Hang Zhang, Yifei Xin, Xin Li, Guanzheng Chen, Yongxin Zhu, Wenqi Zhang, Ziyang Luo, Deli Zhao, et al. Videollama 2: Advancing spatial-temporal modeling and audio understanding in video-llms. arXiv preprint arXiv:2406.07476, 2024. [10] Wenliang Dai, Junnan Li, Dongxu Li, Anthony Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale N Fung, and Steven Hoi. Instructblip: Towards general-purpose vision-language models with instruction tuning. volume 36, pages 49250–49267, 2023. [11] Thomas Defard, Aleksandr Setkov, Angelique Loesch, and Romaric Audigier. Padim: a patch distribution modeling framework for anomaly detection and localization. In International conference on pattern recognition, pages 475–489. Springer, 2021. [12] Hanqiu Deng and Xingyu Li. Anomaly detection via reverse distillation from one-class embedding. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9737–9746, 2022. [13] Kaituo Feng, Kaixiong Gong, Bohao Li, Zonghao Guo, Yibing Wang, Tianshuo Peng, Junfei Wu, Xiaoying Zhang, Benyou Wang, and Xiangyu Yue. Video-r1: Reinforcing video reasoning in mllms. 2025. [14] Jiyang Gao, Chen Sun, Zhenheng Yang, and Ram Nevatia. Tall: Temporal activity localization via language query. In Proceedings of the IEEE international conference on computer vision, pages 5267–5275, 2017. [15] Zhaopeng Gu, Bingke Zhu, Guibo Zhu, Yingying Chen, Ming Tang, and Jinqiao Wang. Anomalygpt: Detecting industrial anomalies using large vision-language models. In Proceedings of the AAAI conference on artificial intelligence, volume 38, pages 1932–1940, 2024. [16] Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Peiyi Wang, Qihao Zhu, Runxin Xu, Ruoyu Zhang, Shirong Ma, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025. [17] Yongxin Guo, Jingyu Liu, Mingda Li, Dingxin Cheng, Xiaoying Tang, Dianbo Sui, Qingbin Liu, Xi Chen, and Kevin Zhao. Vtg-llm: Integrating timestamp knowledge into video llms for enhanced video temporal grounding. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 3302–3310, 2025. [18] Yongxin Guo, Jingyu Liu, Mingda Li, Qingbin Liu, Xi Chen, and Xiaoying Tang. Trace: Temporal grounding video llm via causal event modeling. 2024. [19] Stepan Jezek, Martin Jonak, Radim Burget, Pavel Dvorak, and Milos Skotak. Deep learningbased defect detection of metal parts: evaluating current methods in complex conditions. In 2021 13th International congress on ultra modern telecommunications and control systems and workshops (ICUMT), pages 66–71. IEEE, 2021. [20] Xi Jiang, Jian Li, Hanqiu Deng, Yong Liu, Bin-Bin Gao, Yifeng Zhou, Jialin Li, Chengjie Wang, and Feng Zheng. Mmad: A comprehensive benchmark for multimodal large language models in industrial anomaly detection. 2024. [21] Jie Lei, Tamara L Berg, and Mohit Bansal. Detecting moments and highlights in videos via natural language queries. volume 34, pages 11846–11858, 2021. [22] Chun-Liang Li, Kihyuk Sohn, Jinsung Yoon, and Tomas Pfister. Cutpaste: Self-supervised learning for anomaly detection and localization. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9664–9674, 2021. [23] Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In International conference on machine learning, pages 19730–19742. PMLR, 2023. [24] Wenqiao Li, Yao Gu, Xintao Chen, Xiaohao Xu, Ming Hu, Xiaonan Huang, and Yingna Wu. Towards visual discrimination and reasoning of real-world physical dynamics: Physics-grounded anomaly detection. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 30409–30419, 2025. 11
[25] Wenqiao Li, Xiaohao Xu, Yao Gu, Bozhong Zheng, Shenghua Gao, and Yingna Wu. Towards scalable 3d anomaly detection and localization: A benchmark via 3d anomaly synthesis and a self-supervised learning network. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 22207–22216, 2024. [26] Xinhao Li, Ziang Yan, Desen Meng, Lu Dong, Xiangyu Zeng, Yinan He, Yali Wang, Yu Qiao, Yi Wang, and Limin Wang. Videochat-r1: Enhancing spatio-temporal perception via reinforcement fine-tuning. arXiv preprint arXiv:2504.06958, 2025. [27] Yanwei Li, Chengyao Wang, and Jiaya Jia. Llama-vid: An image is worth 2 tokens in large language models. In European Conference on Computer Vision, pages 323–340. Springer, 2024. [28] Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. volume 36, pages 34892–34916, 2023. [29] Muhammad Maaz, Hanoona Rasheed, Salman Khan, and Fahad Khan. Video-chatgpt: Towards detailed video understanding via large vision and language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 12585–12602, 2024. [30] Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. volume 35, pages 27730–27744, 2022. [31] Mengxue Qu, Xiaodong Chen, Wu Liu, Alicia Li, and Yao Zhao. Chatvtg: Video temporal grounding via chat with video dialogue large language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1847–1856, 2024. [32] Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. volume 36, pages 53728–53741, 2023. [33] Shuhuai Ren, Linli Yao, Shicheng Li, Xu Sun, and Lu Hou. Timechat: A time-sensitive multimodal large language model for long video understanding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14313–14323, 2024. [34] Karsten Roth, Latha Pemula, Joaquin Zepeda, Bernhard Schölkopf, Thomas Brox, and Peter Gehler. Towards total recall in industrial anomaly detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 14318–14328, 2022. [35] Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300, 2024. [36] Chengjie Wang, Wenbing Zhu, Bin-Bin Gao, Zhenye Gan, Jiangning Zhang, Zhihao Gu, Shuguang Qian, Mingang Chen, and Lizhuang Ma. Real-iad: A real-world multi-view dataset for benchmarking versatile industrial anomaly detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 22883–22892, 2024. [37] Ye Wang, Ziheng Wang, Boshen Xu, Yang Du, Kejun Lin, Zihan Xiao, Zihao Yue, Jianzhong Ju, Liang Zhang, Dingyi Yang, et al. Time-r1: Post-training large vision language model for temporal video grounding. 2025. [38] Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. volume 35, pages 24824–24837, 2022. [39] Ziang Yan, Xinhao Li, Yinan He, Zhengrong Yue, Xiangyu Zeng, Yali Wang, Yu Qiao, Limin Wang, and Yi Wang. Videochat-r1. 5: Visual test-time scaling to reinforce multimodal reasoning by iterative perception. 2025. [40] Vitjan Zavrtanik, Matej Kristan, and Danijel Skočaj. Draem-a discriminatively trained reconstruction embedding for surface anomaly detection. In Proceedings of the IEEE/CVF international conference on computer vision, pages 8330–8339, 2021. 12
[41] Hang Zhang, Xin Li, and Lidong Bing. Video-llama: An instruction-tuned audio-visual language model for video understanding. In Proceedings of the 2023 conference on empirical methods in natural language processing: system demonstrations, pages 543–553, 2023. [42] Peiyuan Zhang, Kaichen Zhang, Bo Li, Guangtao Zeng, Jingkang Yang, Yuanhan Zhang, Ziyue Wang, Haoran Tan, Chunyuan Li, and Ziwei Liu. Long context transfer from language to vision. arXiv preprint arXiv:2406.16852, 2024. [43] Songyang Zhang, Houwen Peng, Jianlong Fu, and Jiebo Luo. Learning 2d temporal adjacent networks for moment localization with natural language. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 12870–12877, 2020. [44] Kaichen Zhou, Xinhai Chang, Taewhan Kim, Jiadong Zhang, Yang Cao, Chufei Peng, Fangneng Zhan, Hao Zhao, Hao Dong, Kai Ming Ting, et al. Rad: A dataset and benchmark for real-life anomaly detection with robotic observations. arXiv preprint arXiv:2410.00713, 2024. [45] Qiang Zhou, Weize Li, Lihan Jiang, Guoliang Wang, Guyue Zhou, Shanghang Zhang, and Hao Zhao. Pad: A dataset and benchmark for pose-agnostic anomaly detection. Advances in Neural Information Processing Systems, 36:44558–44571, 2023. [46] Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. Minigpt-4: Enhancing vision-language understanding with advanced large language models. 2023. [47] Yang Zou, Jongheon Jeong, Latha Pemula, Dongqing Zhang, and Onkar Dabeer. Spot-thedifference self-supervised pre-training for anomaly detection and segmentation. In European conference on computer vision, pages 392–408. Springer, 2022.
13
A
Broader Impact
A.1
Potential Positive Impacts
Industrial anomaly detection is important for manufacturing quality control, product reliability, and operational safety. By introducing MMVIAD, a continuous multi-view video dataset and benchmark with visible-time annotations, our work aims to support industrial inspection research beyond staticimage recognition and toward more realistic video-based anomaly understanding. Compared with single-frame or sparsely sampled settings, MMVIAD evaluates whether models can recognize structural defects under continuous viewpoint changes, understand object semantics, and identify the time interval in which defect evidence is visible. This provides a more diagnostic way to assess whether a model bases its decision on observable visual evidence rather than only producing a final label. MMVIAD may improve the reliability and interpretability of inspection systems. By coupling anomaly detection, defect classification, object classification, and visible-time localization, the benchmark encourages models to provide both predictions and temporally grounded evidence. Such evidence-grounded outputs can help human inspectors verify model decisions, locate relevant video segments more efficiently, and improve transparency in human-AI collaborative inspection workflows. In practical quality control, this may help reduce missed structural defects, support faster review of suspicious samples, and make model outputs easier to audit. The benchmark may also benefit multimodal video understanding research. Industrial inspection requires models to attend to fine-grained, local, and structural visual evidence, which is different from many general video understanding tasks. MMVIAD provides a controlled setting for studying fine-grained visual reasoning, temporal grounding, structured outputs, and multi-task consistency. In particular, anomaly visible-time localization asks models to identify when the evidence for a defect is visible, encouraging future models to move from coarse semantic recognition toward more precise evidence localization. More broadly, MMVIAD may inspire better evaluation protocols for industrial and embodied vision systems. The idea of evaluating whether predictions are temporally aligned with observable evidence may be useful for assembly verification, robotic inspection, equipment monitoring, and dynamic quality control. The structured QA format also provides a unified interface for evaluating detection, recognition, object understanding, and temporal localization, which may help researchers compare model capabilities more systematically across inspection tasks. MMVIAD may also support reproducibility and fair comparison in industrial anomaly research. Since all tasks share the same input format, structured response protocol, and answer parsing procedure, different models can be evaluated under a unified setting. This can reduce ambiguity in benchmark usage and make it easier to analyze which capabilities are missing, such as fine-grained defect recognition, object-aware reasoning, or temporal evidence grounding. By releasing the benchmark protocol and evaluation format, MMVIAD can provide a common testbed for future studies on industrial video anomaly understanding.
A.2
Potential Negative Impacts and Risk Mitigation
The main risk is over-reliance on automated inspection systems. Models trained or evaluated on MMVIAD should not be treated as deployment-ready safety systems without additional validation in the target industrial environment. In real quality-control workflows, automated predictions should be used to support rather than replace human experts, especially in high-stakes scenarios. Practical deployment should include human oversight, uncertainty monitoring, failure-case analysis, and interpretable evidence such as defect descriptions, key frames, visible-time intervals, or localized visual cues. In addition, inspection videos and model outputs may contain proprietary information about products or manufacturing processes. Deployment should therefore follow appropriate data governance and access-control procedures. With these safeguards, benchmarks such as MMVIAD can support safer, more transparent, and more reliable human-AI collaborative industrial inspection systems. 14
B
Reward Design Ablation
Table 5 verifies the task-specific effects of our reward design. Removing Rsg mainly reduces defect classification, while replacing Rvis with flat IoU lowers visible-time localization. The full reward design achieves the best Avg., suggesting that semantic gating and visibility-aware temporal modeling provide complementary training signals. Table 5: Reward design ablation on MMVIAD-Unseen. We keep the PS-SFT initialization, VISTA-GRPO training data, and evaluation protocol fixed, and ablate only the reward design during VISTA-GRPO refinement. “w/o Rsg ” removes the semantic gate for defect classification, and “w/o Rvis ” replaces the visibility-aware temporal reward with a flat IoU reward. Defect
Model VISTA-8B w/o Rsg w/o Rvis
Object
Detect.
Class.
Loc.
Class.
60.7 61.0 59.7
37.6 35.8 37.2
49.6 49.4 48.9
81.9 81.2 82.1
C
Detailed Information of the MMVIAD
C.1
Object Category Taxonomy
Avg. 57.5 56.9 57.0
This section provides additional details about the object and material composition of MMVIAD. The dataset is constructed from diverse object geometries and rendering factors to support multi-view industrial anomaly understanding under controlled but visually varied inspection conditions. Table 6: Semantic grouping of object categories in MMVIAD. The 48 fine-grained object categories are organized into 17 semantic groups. Train, Test, and Total denote the number of clips in each split and in total, respectively. Semantic Group
# Cat.
Fine-grained Categories
Train
Test
Total
ashtray bottle bowl bucket cabinet cap chair cup desk eraser headset helmet jar microphone shelf tap vase
1 3 6 2 1 6 1 3 1 1 2 4 1 2 1 2 11
ashtray0 bottle0, bottle1, bottle3 bowl0, bowl1, bowl2, bowl3, bowl4, bowl5 bucket0, bucket1 cabinet0 cap0, cap1, cap2, cap3, cap4, cap5 chair0 cup0, cup1, cup2 desk0 eraser0 headset0, headset1 helmet0, helmet1, helmet2, helmet3 jar0 microphone0, microphone1 shelf0 tap0, tap1 vase0, vase1, vase2, vase3, vase4, vase5, vase6, vase7, vase8, vase9, vase10
54 183 378 132 75 360 66 162 63 54 108 252 54 111 66 126 669
18 69 129 54 15 144 24 54 21 18 36 108 18 39 30 54 270
72 252 507 186 90 504 90 216 84 72 144 360 72 150 96 180 939
Total
48
-
2913
1101
4014
Table 6 summarizes the object taxonomy and split statistics of MMVIAD. The 48 fine-grained object categories are organized into 17 semantic groups, covering diverse object geometries such as containers, furniture, tools, wearable objects, and household items. The train and test splits preserve all semantic groups and fine-grained categories, yielding 2,913 training clips and 1,101 test clips, 15
with 4,014 clips in total. This category coverage provides diverse geometric contexts for evaluating multi-view industrial anomaly understanding.
Figure 7: Representative semantic object groups in MMVIAD. We visualize representative objects from the 17 semantic groups used in MMVIAD. Objects are rendered with a neutral material to emphasize geometric and semantic diversity. MMVIAD contains 48 fine-grained object categories, which are organized into 17 semantic groups. Figure 7 visualizes one representative object from each semantic group. These groups cover diverse object shapes and geometric structures for anomaly rendering and inspection. C.2
Material Diversity
In addition to object-level diversity, MMVIAD introduces material variation during rendering. We organize object materials into six major groups: plastic, fabric, clay, wood, leather, and metal. These groups contain 28 fine-grained material subclasses in total. Figure 8 shows representative examples of the six material groups. The material variation changes surface appearance, reflectance, and texture 16
while keeping the underlying object geometry and anomaly structure controlled, allowing MMVIAD to evaluate whether models can recognize structural defects under diverse visual appearances. Together, the semantic object taxonomy and material variation increase the diversity of MMVIAD beyond simple object identity. The 17 semantic groups provide broad geometric and functional coverage, while the six material groups introduce controlled appearance variation. This design supports evaluation of industrial video anomaly understanding under changes in both object structure and visual surface properties.
Figure 8: Representative material groups in MMVIAD. MMVIAD uses six major material groups, including plastic, fabric, clay, wood, leather, and metal, with 28 fine-grained material subclasses in total. These materials introduce diverse surface appearance and reflectance while preserving the object-centric inspection setting.
C.3
Structural Anomaly Types
MMVIAD contains six structural anomaly types: crack, scratch, concavity, bulge, broken, and hole. These anomaly types are designed to cover common geometric defects that affect object structure rather than only surface appearance. Figure 9 shows representative examples of the six anomaly types used in MMVIAD. The highlighted regions indicate the defect locations, illustrating the diversity of defect morphology and visibility across different object shapes. 17
Figure 9: Representative structural anomaly types in MMVIAD. We visualize six structural anomaly types, including crack, scratch, concavity, bulge, broken, and hole. The highlighted boxes indicate defect regions on representative objects.
D
Reasoning Examples
Figure 10: Reasoning example: tap with bulge. 18
Figure 11: Reasoning example: vase with concavity.
Figure 12: Reasoning example: helmet with hole. 19
Figure 13: Reasoning example: microphone with scratch. These examples show that VISTA does not only output final answers, but also grounds them in explicit visual evidence. The model links object-level cues to category prediction, defect morphology to anomaly type, and viewpoint-dependent visibility to temporal localization. This structured reasoning format makes the four task predictions interpretable and exposes whether errors arise from object recognition, defect semantics, or temporal grounding.
E
Prompt Template
E.1
Main Benchmark Prompt
For the standard benchmark setting, we use a unified evaluation prompt across all machine baselines. The prompt asks the model to analyze the input video, reason inside <think> tags, and return final predictions in a structured <answer> block. All reported scores are computed only from the parsed final answers. QUESTION_TEMPLATE = ( "{Question}\n\n" "Please think step by step, analyzing the video carefully for any defects or anomalies. " "Engage in an internal dialogue using expressions such as ’let me think’, ’ wait’, ’Hmm’, " "’oh, I see’, ’let me verify’, etc. " "It’s encouraged to include self-reflection or verification in the reasoning process. " "Provide your detailed reasoning between the <think> </think> tags, and then give your answers " "between the <answer> </answer> tags." ) TYPE_TEMPLATE = ( "\n\nProvide your answers in the following structured format inside <answer> </answer> tags:\n"
20
"<answer>\n" "<q1>LETTER</q1>\n" "<q2>LETTER</q2>\n" "<q3>LETTER</q3>\n" "<q4>[[start, end], ...]</q4>\n" "</answer>" )
E.2
Structured Generalization Prompt
For MMVIAD-Unseen evaluation, we use a structured prompt aligned with the proposed reasoning protocol. Different from the main benchmark prompt, this prompt requires the model to explicitly separate whole-clip perception, localized defect evidence, reasoning, and final task predictions. The final scores are computed only from the parsed <q1>–<q4> fields inside the <answer> block. STRUCTURED_PROMPT = [ "You must answer all tasks in a single response using the exact structure below.", "You must output all four sections exactly once and in this exact order:", "<global_perception> ... </global_perception>", "<segment_perception> ... </segment_perception>", "<think> ... </think>", "<answer> ... </answer>", "Do not replace or omit any closing tag.", "The <think> section must end with </think>, never </answer>.", "The <answer> section must begin with <answer> and end with </answer>.", "The final machine-readable result must appear inside <answer>...</answer> only.", "Do not put any natural language after <answer> begins; only q1--q4 tags are allowed inside <answer>.", "Do not repeat <answer> or any closing tag.", "No extra characters are allowed between </think> and <answer>.", "Do not output any stray unicode character or separator.", "Output exactly this skeleton and fill in content only:", "<global_perception>", "[content]", "</global_perception>", "<segment_perception>", "[content]", "</segment_perception>", "<think>", "[content]", "</think>", "<answer>", "<q1>LETTER</q1>", "<q2>LETTER</q2>", "<q3>LETTER</q3>", "<q4>[start_sec,end_sec]</q4>", "</answer>", "", "<global_perception>", "Describe stable global visual facts: object category clues, material/texture/ color/shape, whether the object looks normal or defective overall, and any coarse anomaly morphology. Keep this grounded in the frames only.", "</global_perception>", "", "<segment_perception>", "Describe localized defect evidence and temporal visibility cues: anomaly details, which region/part looks abnormal, when it becomes visible in the 2second clip, and whether it persists or only appears in part of the clip. If no anomaly, state no anomaly-specific visible segment and no localized defect evidence.", "</segment_perception>", "",
21
"<think>", "Reason briefly from the visual perception to the final answers for all tasks. Do not introduce facts not supported by the visual perception.", "</think>", "", "<answer>", "<q1>A or B</q1>", "<q2>LETTER</q2>", "<q3>LETTER</q3>", "<q4>[start_sec,end_sec]</q4>", "</answer>", "", "Additional constraints:", "- <global_perception> must be 80 words or fewer.", "- <segment_perception> must be 80 words or fewer.", "- <think> must be 60 words or fewer.", "- The final machine-readable result must appear only inside <answer>...</ answer>.", "- Do not wrap the answer in markdown fences.", "- anomaly_detection must be encoded in <q1> as A or B.", "- defect_classification must be encoded in <q2> as the option letter.", "- object_classification must be encoded in <q3> as the option letter.", "- visible_time_localization must be encoded in <q4> as [] or [start_sec, end_sec].", "- For visible_time_localization, use the earliest and latest times when the anomaly is visible at all, even if subtle.", "- Do not delay the start time to only when the anomaly becomes most obvious .", "- Keep <global_perception> and <segment_perception> concise and clean.", "- The parser will only use the q1--q4 tags inside <answer>...</answer> as final prediction.", ]
E.3
Single-View Image Generalization Prompt
For the single-view image generalization evaluation on MVTec AD and VisA, we adapt the structured prompting protocol used in MMVIAD-Unseen to the image setting. Although these datasets contain static images rather than videos, we keep the same perception-structured response format to reduce prompt-format differences. The prompt asks the model to determine whether a defect is present in the image, and all reported scores are computed only from the parsed final answer. SINGLE_VIEW_PROMPT = """ Look at the image carefully and answer the following question about object anomaly detection. The object in this image is: {category} Q1: Is there a defect or anomaly present in this image? Options: (A) Yes, a defect is present. (B) No, the object looks normal. Think step by step using the structured format below. <global_perception> [Describe the object’s overall appearance: material, texture, color, shape, and whether it looks normal or defective. <=80 words.] </global_perception> <segment_perception> [Describe any localized defect evidence: which region looks abnormal. If no anomaly, state: no anomaly visible. <=80 words.] </segment_perception> <reasoning>
22
[Reason briefly to your final answer. <=60 words.] </reasoning> <answer> {"anomaly_detection": "A or B"} </answer> """
F
Additional Training Curves
This section provides additional training dynamics for the two-stage post-training pipeline. Figure 14 shows the PS-SFT training curves, including loss, token accuracy, learning rate, and gradient norm. Figure 15 shows the VISTA-GRPO training curves, including KL divergence, gradient norm, entropy, and completion length.
Figure 14: PS-SFT training curves. We report loss, token accuracy, learning rate, and gradient norm during perception-structured supervised fine-tuning.
Figure 15: VISTA-GRPO training curves. We report KL divergence, gradient norm, entropy, and completion length during reinforcement learning refinement.
G
Limitations and Future Work
MMVIAD takes a first step toward continuous multi-view video understanding for industrial anomaly detection, but it has several limitations. First, the dataset scale is still moderate. MMVIAD is built from high-quality object-centric 3D assets that support 360◦ rendering and controllable structural anomaly synthesis, but such assets are currently limited in availability. As a result, expanding the dataset to more object categories, defect types, and industrial scenarios remains an important direction. Second, MMVIAD is synthetic. Although the rendering process introduces diverse viewpoints, materials, floors, and HDRI environments, there is still a domain gap between rendered inspection videos and real industrial acquisition, where sensor noise, motion blur, imperfect lighting, occlusion, and production-line constraints may appear. Future work should combine synthetic generation with real captured multi-view inspection videos to improve realism and deployment relevance. Another limitation is the computational cost of data generation. MMVIAD uses high-resolution rendering and dense frame rates to preserve fine structural details and temporally localized anomaly 23
evidence. This improves annotation quality and supports visible-time localization, but also makes rendering time-consuming and expensive. More efficient rendering pipelines, adaptive frame sampling, and targeted generation of difficult viewpoints could make future dataset construction more scalable. In addition, the current benchmark focuses on short 2-second clips with approximately 120◦ viewpoint changes. This design provides a compact and controlled setting, but longer inspection trajectories, multi-object scenes, and more complex camera-object motion would better reflect real-world inspection workflows. Finally, VISTA-GRPO improves structured anomaly understanding, but fine-grained defect classification remains challenging, especially under sparse and imbalanced defect distributions. Future work may explore denser semantic rewards, hierarchical defect taxonomies, stronger temporal grounding objectives, and larger multimodal backbones. We also plan to study how MMVIAD can support more realistic inspection protocols, including open-vocabulary defect discovery, uncertainty estimation, and interactive human-AI verification for safety-critical industrial inspection.
24