DART: A V ISION -L ANGUAGE F OUNDATION M ODEL FOR C OMPREHENSIVE ROPE C ONDITION M ONITORING
arXiv:2605.04943v1 [cs.CV] 6 May 2026
A P REPRINT Anju Rani∗ Department of Energy Aalborg University Esbjerg, Denmark 6700 [email protected]
Daniel Ortiz-Arroyo Department of Energy Aalborg University Esbjerg, Denmark 6700 [email protected]
Petar Durdevic Department of Energy Aalborg University Esbjerg, Denmark 6700 [email protected]
A BSTRACT The condition monitoring (CM) of synthetic fibre ropes (SFRs) used in offshore, maritime, and industrial settings demands more than a classifier: inspectors need continuous severity estimates, maintenance recommendations, anomaly flags, deterioration timelines, and automated reports, all from a single inspection image. We present DART (Damage Assessment via Rope Transformer), a vision-language foundation model that addresses the full rope inspection workflow through a unified multi-task architecture. DART extends the Joint-Embedding Predictive Architecture (JEPA) to the cross-modal domain by coupling a Vision Transformer (ViT-H/14) with Llama-3.2-3B-Instruct via a Severity-Conditioned Cross-Modal Fusion (SC-CMF) module. Three architectural innovations drive the model’s versatility: (1) HD-MASK, a saliency-guided masking strategy that focuses self-supervised reconstruction on damage-dense patches; (2) per-class learnable severity gates that adaptively weight language grounding by damage category; and (3) a Contrastive Damage Disentanglement (CDD) loss that shapes the embedding space to simultaneously encode damage type, severity ordering, and cross-modal semantics. Trained once on 4,270 images spanning 14 fine-grained rope damage classes, the frozen DART backbone supports downstream tasks without any task-specific fine-tuning: damage classification (93.22% accuracy, 91.04% macro-F1, +38.5 pp over a vision-only baseline), continuous severity regression (Spearman ρ = 0.94, within-1-ordinal accuracy 99.6%), few-shot recognition (89.2% macro-F1 at 20 shots), damage progression modelling (91% monotone interpolation), maintenance action recommendation (94.79% macro-F1), automated inspection report generation (93.22% damage accuracy), out-of-distribution anomaly detection (4.76% anomaly rate), and trajectory-based deterioration phenotyping. These results demonstrate that DART functions as a general-purpose CM backbone that goes well beyond classification, providing actionable inspection intelligence from a single shared representation. Keywords Vision-Language Model · Condition monitoring · JEPA · Self-supervised Learning · Defect detection · Computer vision
1
Introduction
Synthetic fibre ropes (SFRs) are critical load-bearing components in offshore lifting operations, mooring systems, and industrial rigging. Rope failure can result in equipment loss, environmental incidents, and fatalities. Current practice relies on periodic visual inspections by qualified personnel, a workflow that is labour-intensive, subjective, and difficult to scale. Yet the automation challenge is not simply one of classification. A deployable inspection system must simultaneously answer a range of interdependent questions: What type of damage is present? How severe is it? Is this presentation anomalous? What maintenance action is warranted? How is this rope deteriorating over time? Can a ∗
Use footnote for providing further information about author (webpage, alternative address)—not for acknowledging funding agencies.
Imagery Dataset for Remaining Useful Life Estimation of Synthetic Fibre Ropes
A P REPRINT
structured report be generated automatically? Answering all of these from a single image, without separate models for each task, is the core problem DART addresses. Existing automated rope inspection approaches treat these as isolated subproblems, a classification model here, a severity estimator there, with no shared representation that supports the full inspection workflow. Single-modality vision models fail on the hardest distinctions: a state-of-the-art ViT-H/14 backbone achieves only 54.76% accuracy on a 14-class rope damage benchmark because adjacent severity levels of the same damage type are visually indistinguishable without semantic context. Natural language provides exactly this context. A description such as “extensive surface abrasion with fibre bundle exposure along the outer sheath” encodes severity information that images alone cannot reliably convey across lighting conditions and viewpoints.
2
Related Work
Automated inspection of SFRs has attracted growing interest driven by offshore mooring and floating wind turbine deployments [1]. Early systems relied on hand-crafted image features; a comprehensive survey by [2] identifies a persistent gap between single-task detectors and the multi-criteria decisions real inspections require. The same group introduced an instance-segmentation pipeline using Detectron2 [3] and released the ROPE imagery dataset [4] that underpins DART’s benchmark. Complementary rope-health work includes statistical machine-learning models for tensile characterisation [5] and a lightweight neural network for real-time wire-rope tension monitoring via fibre Bragg grating sensors [6]. Defect detection has matured through convolutional and transformer architectures. [7] targeted object detection network in UAV, [8] developed transfer-learning pipelines for manufacturing defects, [9] used U-Net to detect surface damages on steel wire ropes (SWRs), and [10] combined residual networks and multi-channel feature fusion for detection of SWRs. Each of these addresses a single damage type or asset in isolation; DART instead learns a unified embedding space that supports all inspection decisions simultaneously. Transformer [11] was adapted to vision by [12], who showed that patch-based self-attention matches or surpasses convolutional networks at scale. Subsequent self-supervised objectives have further narrowed the gap to supervised pre-training: [13] showed that masking a large fraction of patches and reconstructing pixel values yields strong representations, while [14] demonstrated that distillation across curated datasets produces dense visual features that transfer without fine-tuning. [15] established that a self-distilling student-teacher ViT learns semantically consistent patch features through local-to-global correspondences. DART builds on the Joint-Embedding Predictive Architecture (iJEPA) [16], which replaces pixel-level reconstruction with prediction in latent space, producing more abstract patch representations suited to downstream transfer. [17, 18] provided theoretical bridges between JEPA and contrastive objectives, confirming that both paradigms optimise aligned information-theoretic quantities. DART’s CDD loss unites them: the JEPA reconstruction term targets structural features while the severity-aware InfoNCE [19] shapes the damage-severity manifold, with contrastive representation collapse further prevented by the MoCo-style EMA target encoder [20]. CLIP [21] established the blueprint for vision-language pre-training by contrastively aligning image and text encoders on web-scale pairs, yielding zero-shot transfer features competitive with supervised baselines. BLIP-2 [22] and LLaVA [23] extended this to generative tasks by bridging frozen vision encoders with instruction-tuned language models. DART adopts Llama-3.2-3B-Instruct [24] as its text encoder, providing domain-adaptable instruction following at laboratory-scale training cost. In the inspection domain, [25] deployed a vision-language model for bridge crack classification conditioned on textual maintenance history, and [26] leveraged CLIP fine-tuning and prefix captioning to automatically generate safety observations. [18] proposed VL-JEPA, the closest architectural predecessor of DART, extending JEPA to joint visionlanguage prediction. Our SC-CMF module differentiates from VL-JEPA by introducing 14 per-class learnable gates that modulate language influence according to damage severity. [27] combined multi-gate mixture-of-experts [28] with JEPA for selective modality routing, an approach that motivated DART’s per-class gates, though DART implements them within a single cross-attention layer rather than a full routing table. [29] demonstrated JEPA-based models in safety-critical manufacturing via Gen-JEMA, and [30] applied cross-modal JEPA to remote sensing retrieval, DART applies the same philosophy to rope inspection with a four-term disentanglement loss absent from all prior JEPA variants. The MVTec AD benchmark [31] standardised unsupervised defect localisation evaluation and catalysed a progression from one-class SVMs to deep density-estimation methods. PaDiM [32] modelled per-patch feature distributions with multivariate Gaussians using pre-trained backbone features, and PatchCore [33] improved efficiency by subsampling a coreset of nominal embeddings and scoring anomalies by nearest-neighbour distance. A recent survey [34] confirms 2
Imagery Dataset for Remaining Useful Life Estimation of Synthetic Fibre Ropes
A P REPRINT
that foundation-model features increasingly replace purpose-trained autoencoders. DART’s Task 6 uses Mahalanobis distance [35] over its frozen backbone embeddings as the anomaly score, consistent with findings that pre-trained ViT features without fine-tuning are competitive on in-distribution shifts [36]. [37] further showed that attention-guided localisation improves interpretability on complex textures, directly relevant to spatially localised rope surface damage. Label scarcity is chronic in industrial inspection, where novel damage types may appear with only a handful of examples [38]. Prototypical Networks [39] classify by nearest embedding-space centroid and are used directly in DART’s Task 2, which evaluates frozen backbone prototypes at k ∈ {1, 5, 10, 20} shots. MAML [40] learns a meta-initialisation for rapid adaptation; while DART does not use explicit meta-learning, its JEPA objective yields transferable features that match meta-learned baselines on the ROPE benchmark. [41] showed contrastive fine-tuning with limited normal/abnormal pairs generalises strongly for industrial anomaly detection an insight reflected in DART’s severity-aware InfoNCE. [42] demonstrated that cross-attention feature matching outperforms metric-learning on heterogeneous defect appearances, and [43] achieved rapid adaptation to unseen wind turbine blade crack patterns from five examples per class, the most directly comparable prior work to DART’s offshore-energy few-shot evaluation. The focal loss [44] down-weights easy negatives to focus training on hard instances; DART uses it as the classification term of CDD to handle the severe class imbalance in ROPE’s 14-category distribution. RandAugment [45] provides principled stochastic augmentation that mitigates overfitting on ROPE’s 4,270 images. The two-phase training curriculum Phase 1 establishes visual representations with HD-MASK and the JEPA reconstruction term alone; Phase 2 activates SC-CMF and the full CDD loss mirrors the staged training of DINO [15], ensuring the backbone learns robust patch statistics before language modulation is introduced. SHM research has progressively moved from handcrafted signal features toward deep learned representations [46]. Fusing heterogeneous sensor streams with deep architectures substantially improves anomaly diagnosis over singlemodality baselines in civil SHM [47, 48], and multimodal sensing has been argued as a design principle for sustainable monitoring of critical infrastructure [49]. [50] combined visual and acoustic data with automatic labelling for wharf damage detection. These works rely on physical sensor heterogeneity, whereas DART achieves cross-modal fusion through vision-language pairing alone, requiring no instrumentation beyond the inspection camera and making it directly deployable in existing visual inspection pipelines. DART addresses this by extending the Joint-Embedding Predictive Architecture (JEPA) framework [16] to the crossmodal domain. Rather than reconstructing pixels, JEPA learns representations by predicting masked latent patches, a signal that promotes abstract, semantic understanding suited to multi-task transfer. By pairing visual JEPA training with language grounding through the SC-CMF module, DART learns a single shared representation that supports all inspection tasks with no task-specific retraining. The main contributions are: • DART, a vision-language foundation model for comprehensive rope CM, trained once and evaluated across inspection tasks. • SC-CMF: Severity-Conditioned Cross-Modal Fusion with 14 learnable per-class gates that adaptively weight language modulation, amplified for severity-graded classes, suppressed for compound damage types. • HD-MASK: Saliency-guided masked reconstruction that biases JEPA training toward damage-dense patches without explicit patch-level annotations. • CDD Loss: A four-term objective (JEPA reconstruction, severity-aware InfoNCE, type orthogonality, focal classification) that jointly shapes the embedding space for multi-task transfer. • An multi-task evaluation benchmark demonstrating that a single frozen DART backbone achieves strong performance across classification, regression, few-shot learning, generative modelling, recommendation, report generation, anomaly detection, and trajectory analysis. The remainder of this paper is organised as follows. Section 3 describes the dataset, model architecture, and loss functions. Section 4 presents component ablation, comparison against foundation model baselines, and evaluation for all downstream tasks. Section 5 summarises findings and future work.
3
Methodology
DART jointly encodes image patches and natural-language damage descriptions through a cross-modal JEPA framework. The architecture comprises five tightly coupled components: (1) a ViT-H/14 online encoder with an EMA target counterpart, (2) HD-MASK for saliency-guided masking, (3) a Transformer latent predictor, (4) a Llama-3.2 text encoder, and (5) the SC-CMF fusion module. The complete architecture is illustrated in Figure 1. 3
Imagery Dataset for Remaining Useful Life Estimation of Synthetic Fibre Ropes
③ IMAGE ENCODER
① INPUT Input Image I ∈ R^(3×224×224)
② HD-MASK HD-MASK Saliency Estimator
Image Augmentation
ViT-H/14(pretrained) Phase 1: frozen Phase 2: last 6 blocks unfrozen
④ I-JEPA Latent Predictor 6-layer Transformer 16 heads
Visual Patch Tokens V ∈ R^(B×256×1280)
Saliency Map S ∈ [0,1]^256
Context/Target Masks M_ctx, M_tgt
Predicted Latents V̂_tgt
ViT-H/14 (EMA Target Encoder) τ = 0.996, frozen
(V̂ predicted)
Target Tokens V_tgt ∈ R^(B×256×1280) ⑤ SC-CMF FUSION SC-CMF Fusion Cross-Attention (4 heads) + Severity Gate
(V ground truth)
Llama-3.2-3B-Instruct 28 blocks, extract [16,24,31] D=3072
Text Embeddings T ∈ R^(B×128×3072)
Text Projection Linear + LayerNorm 3072 → 1280
text keys/values
③ TEXT ENCODER Text Description (damage description)
T_proj ∈ R^(B×128×1280)
Fused Features F ∈ R^(B×256×1280)
Global Average Pooling
Pooled Features f ∈ R^(B×1280)
A P REPRINT
Encoder (ViT / Llama)
T1: Severity Regression
SC-CMF Fusion Module
T2: Few-Shot Recognition
Predictor / Loss Function
T3: Damage Progression
Tensor (with shape)
T4: Maintenance Rec.
Input / Intermediate
T5: Report Generation
Classification Head (primary)
T6: Anomaly Detection
Data flow
Loss gradient
ℒ_recon SmoothL1( V̂, V ) JEPA patch reconstruction ℒ_orth Cosine orthogonality Type disentanglement ℒ_sev InfoNCE contrastive Pull same severity, push others
ℒ_total λ₁ℒ_recon + λ₂ℒ_sev + λ₃ℒ_orth + λ₄ℒ_focal
ℒ_focal (γ = 2.0) Focal loss · inverse-freq weights 14-class classification Classification Head 1280→512→14 (14 Damage Classes)
Class Logits y ∈ R^14
⑥ DOWNSTREAM TASKS
T1: Severity Head 1280→256→1 MAE=0.047, R²=0.868
T2: Few-Shot Head Nearest-Prototype 1-shot ≥99.0%
T3: Progression Head Embedding UMAP S-curve Analysis
T4: Maintenance Head 1280→512→4 (Action + Urgency)
T5: Report Head Template Filling + Severity + Action
T6: Anomaly Detector Reconstruction + Distance Metrics
Severity Score s ∈ [0,1]
Proto Similarity p ∈ R^14
Damage Trajectory t ∈ [0,1], threshold
Action + Urgency a ∈ {0,1,2,3}, u ∈ [0,1]
Inspection Report Markdown/HTML
Anomaly Score a ∈ [0,1] is_anomaly ∈ {0,1}
Figure 1: DART architecture overview. The online ViT-H/14 encoder processes saliency-masked image patches (HD-MASK); the EMA target encoder provides stable latent reconstruction targets (no gradients); the latent predictor reconstructs masked embeddings from context tokens; the Llama-3.2-3B text encoder extracts multi-scale features from per-image damage descriptions; SC-CMF fuses both modalities via severity-gated cross-attention before global average pooling feeds the classification head and all downstream task heads. Dashed lines denote gradient-blocked paths. The frozen backbone (grey box) is shared across all downstream tasks at inference time.
3.1
Dataset Preperation
The ROPE dataset comprises 4,270 images of synthetic ropes captured under controlled laboratory conditions. Eight physical damage modes are represented: Chafing, Cut Strands, Placking, Compression, Compression + Chafing, Compression + Cut Strands, Coreout + Cut Strands, and Strand Coreout. The first three types carry additional severity labels (High, Medium, Low), yielding 14 fine-grained classes (Table 1) [4]. The dataset is split into 3,197 training, 527 validation, and 546 test images, respectively. Image augmentation was applied to the training dataset, including random crop (224×224), horizontal/vertical flips, rotation (±15◦ ), colour jitter (brightness/contrast ±0.3), and RandAugment [45] (2 ops, magnitude 9), followed by ImageNet normalisation. Validation and test images are only resized and normalised. Every image is paired with an expert-reviewed natural-language description stored in a companion JSON file. Descriptions detail visible texture changes, dimensional deformations, and structural implications of the observed damage. 3.2
Image Encoder
The image encoder is ViT-H/14 [16, 15] pre-trained with the iJEPA objective on ImageNet-22k. Input images (224×224) are divided into 14×14 patches (16×16 = 256 tokens), each projected to a 1,280-dimensional embedding across 32 transformer blocks (16 heads). The [CLS] token is discarded; all 256 patch tokens carry spatial localisation for damage grounding. 4
Imagery Dataset for Remaining Useful Life Estimation of Synthetic Fibre Ropes
A P REPRINT
Table 1: The 14 fine-grained damage classes in the ROPE dataset. Severity labels apply only to the first three damage types; compound and structural classes carry no severity distinction (–). Class Damage Type Severity Chafing/High Chafing/Medium Chafing/Low CutStrands/High CutStrands/Medium CutStrands/Low Placking/High Placking/Medium Placking/Low Compression Compression+Chafing Compression+CutStrands CoreOut+CutStrands Strand Coreout
Chafing Chafing Chafing CutStrands CutStrands CutStrands Placking Placking Placking Compression Compression Compression CoreOut CoreOut
High Medium Low High Medium Low High Medium Low – – – – –
EMA Target Encoder.
A parameter-frozen copy of the ViT tracks the online encoder via: θema ← λ θema + (1 − λ) θonline , λ = 0.996. (1) By blocking gradients, the target encoder acts as a slow-moving teacher that prevents representation collapse without requiring contrastive negative pairs on the visual stream. Two-Phase Fine-Tuning. Phase 1 (10 epochs): all ViT blocks frozen; HD-MASK, predictor, Llama (last 4 blocks), SC-CMF, and classifier trained at 10−4 with cosine annealing to 10−6 . Phase 2 (40 epochs): last 6 ViT blocks and final LayerNorm unfrozen at 3×10−6 (all other modules at 3×10−5 ) with cosine annealing to 10−7 . Differential learning rates prevent catastrophic forgetting of iJEPA representations. 3.3
High-Discrepancy Masking (HD-MASK)
Standard JEPA uses random block masking. For rope images, damage regions occupy a small, localised fraction of the scene; random masking rarely targets them. HD-MASK estimates per-patch saliency with a lightweight 4-layer convolutional network (fsal : 3 → 32 → 64 → 64 → 32 channels, sigmoid output, 16×16 grid). The estimator trains end-to-end, the JEPA reconstruction gradient naturally rewards high scores on informative patches. Patches in the top 40% of saliency are damage-dense; the remainder are background. Masking probabilities are: 0.70, damage-dense, P (mask | patchi ) = 0.30, background,
(2)
yielding an overall ≈55% masking ratio consistent with JEPA practice [16], with a safety floor of 10 visible context patches. 3.4
Latent Predictor
A 6-layer Transformer (16 heads, 1,280-dim) predicts masked patch embeddings from context tokens. Target positions are filled with a shared learnable mask token m; context tokens occupy their original grid positions, with learnable positional embeddings added throughout. The predictor outputs only target-position embeddings ẐT , compared against EMA target outputs in the reconstruction loss. Predicting in latent space, rather than pixel space avoids the noise sensitivity of reconstruction-based self-supervised methods. 3.5
Text Encoder
The text encoder is Llama-3.2-3B-Instruct [24] in bfloat16, tokenising descriptions to 128 tokens. Multi-scale features are extracted by averaging three intermediate layers: X T = 31 Hk ∈ RB×128×3072 , (3) k∈{16,24,31}
5
Imagery Dataset for Remaining Useful Life Estimation of Synthetic Fibre Ropes
A P REPRINT
balancing shallow syntax (layer 16), mid-level semantics (layer 24), and task-relevant abstraction (layer 31). All Llama parameters are frozen except the last four transformer blocks and the final LayerNorm; gradient checkpointing manages memory. 3.6
Severity-Conditioned Cross-Modal Fusion (SC-CMF)
SC-CMF fuses patch-level visual tokens with text embeddings via gated cross-attention. The key insight is that language should modulate vision strongly for severity-graded classes but weakly for compound damage types whose descriptions carry no severity gradient. Text features are first projected to the ViT dimension: T̂ = LN(Wproj T + bproj ) ∈ RB×128×1280 .
(4)
Visual tokens attend over projected text via 4-head cross-attention: A = MHA(LN(V), T̂, T̂).
(5)
A set of C = 14 learnable per-class scalars {gc } produces a severity gate αb = σ(gyb ); the gated residual update is: F = V + α ⊙ A.
(6)
A two-layer FFN (DV → 2DV → DV , GELU) applies a final residual refinement: F′ = F + FFN(LN(F)).
(7)
′
Global average pooling over F yields the 1,280-dim backbone representation p, which feeds the classification head (two-layer MLP, 512-dim hidden, dropout 0.1) and all downstream task heads. 3.7
Contrastive Damage Disentanglement (CDD) Loss
The training objective combines four complementary terms: L = λ1 Lrecon + λ2 Lsev + λ3 Lorth + λ4 Lfocal ,
λ1–4 = {1.0, 0.5, 0.3, 1.0}.
(8)
Reconstruction (Lrecon ). Smooth-L1 on L2-normalised predicted vs. EMA target latents, preventing scale collapse while tolerating outlier predictions in early training. Severity Contrastive (Lsev ). InfoNCE at τ = 0.07: positives are pairs sharing the same damage type but different severity; negatives span different types. This explicitly pulls Chafing/High and Chafing/Low together while pushing them away from CutStrands, encouraging type–severity disentanglement. Type Orthogonality (Lorth ). Penalises inter-type cosine similarity between L2-normalised class centroids and encourages intra-type compactness (β = 0.5). The resulting geometry supports reliable cross-task transfer, each damage type occupies an approximately orthogonal subspace. Focal Classification (Lfocal ). Cross-entropy with (1 − pt )2 focusing and inverse-frequency class weights, addressing the 29:1 class imbalance.
4
Results and Discussion
All experiments run on two NVIDIA RTX 4090 GPUs. Total training: 1.48 hrs (Phase 1: 21.7 min, Phase 2: 86.7 min). Full DART totals 4,610.5M parameters; only 538.9M (11.7%) require gradient updates, the remainder are frozen ViT and Llama layers. 4.1
Ablation Study
Six configurations are trained across three seeds (42, 123, 999) for 5 epochs to isolate each component’s contribution: (i) E1 (Full DART): All components enabled; (ii) E2 (w/o Severity Gate): Gates fixed to α = 1.0; (iii) E3 (w/o HD-MASK): Random masking at 55%; (iv) E4 (w/o Text): Text encoder removed; vision only; (v) E5 (Frozen Llama): Only projection layers trained; (vi) E6 (Simple Fusion): Concat + MLP replaces cross-attention. 6
Imagery Dataset for Remaining Useful Life Estimation of Synthetic Fibre Ropes
A P REPRINT
Table 2: Ablation study results (mean ± std, 3 seeds). Best results in bold. The dominant contribution is language grounding (E4: −35.41%); all other components add consistent improvements. Model E1: Full DART E2: w/o Severity Gate E3: w/o HD-MASK E4: w/o Text (ViT only) E5: Frozen Llama E6: Simple Fusion
Accuracy
Macro-F1
Weighted-F1
∆ Acc
0.9017±0.0116 0.8797±0.0062 0.8938±0.0172 0.5476±0.0104 0.8901±0.0113 0.8932±0.0102
0.8689±0.0114 0.8454±0.0060 0.8582±0.0192 0.4950±0.0077 0.8489±0.0174 0.8583±0.0136
0.9036±0.0108 0.8818±0.0061 0.8950±0.0168 0.5351±0.0131 0.8884±0.0124 0.8924±0.0123
— −2.20% −0.79% −35.41% −1.16% −0.85%
5 randomly selected test images ✓ CORRECT GT: Placking/Medium Pred: Placking/Medium | Conf: 96.1%
✓ CORRECT GT: Cut strands/High Pred: Cut strands/High | Conf: 89.3%
✓ CORRECT GT: Compression+Chafing Pred: Compression+Chafing | Conf: 99.4%
✓ CORRECT GT: Placking/Low Pred: Placking/Low | Conf: 97.8%
✓ CORRECT GT: Chafing/Medium Pred: Chafing/Medium | Conf: 83.9%
Figure 2: Correctly classified test examples with predicted label and softmax confidence score.
Language grounding (E4) is by far the most critical component: removing the text encoder drops accuracy 35.41 pp (p = 0.003), confirming that visual features alone cannot resolve visually ambiguous class pairs such as Chafing/Medium vs. Placking/Medium. Severity gating (E2) removing adaptive gates costs 2.20 pp (p = 0.042), validating that a single shared fusion strength is suboptimal across severity-graded and compound classes. SC-CMF fusion (E6) replacing cross-attention with concat + MLP reduces accuracy by 0.85%, showing that attending over the full token sequence is important for spatially localised damage grounding. HD-MASK (E3) costs 0.79% and raises variance (std = 0.017 vs. 0.012 for full DART), confirming that saliencyguided masking both improves accuracy and stabilises training. Llama fine-tuning (E5) freezing Llama costs only 1.16%, suggesting strong pre-trained rope-damage semantics. Unfreezing the last four blocks adds 113.3M trainable parameters (26.6% of total trainables) for a modest but consistent gain. Table 2 reports ablation study results. For resource-constrained deployment, E5 (Frozen Llama) reduces trainable parameters by 21% with only 1.16% accuracy loss while full DART requires 538.9M trainable parameters. 4.2
Comparison Against Foundation Model Baselines
To situate DART within the broader landscape of self-supervised and vision-language foundation models, we compare against four strong baselines: I-JEPA [16], CLIP [21], BLIP-2 [22], and DINOv2 [14]. All baselines follow a unified evaluation protocol: the backbone encoder is frozen, image embeddings are extracted from the ROPE train/val/test splits, a linear classifier is trained on the training embeddings, and the best checkpoint is selected by validation macro-F1. Results are reported on the held-out 546-image test set across three seeds {42, 123, 999}. 4.2.1
Foundation Baseline Results
Among baselines, DINOv2 achieves the strongest performance (86.20% accuracy, 81.80% macro-F1), benefiting from large-scale self-distillation pre-training on curated diverse data. I-JEPA ranks second (79.37%), confirming that latent-space prediction is a stronger pre-training objective than contrastive alignment for fine-grained visual tasks. BLIP-2 and CLIP, despite their vision-language pre-training, underperform DINOv2 substantially (73.69% and 70.45% respectively), indicating that generic image-text alignment on web-scale data does not transfer well to the narrow, severity-graded vocabulary of rope damage. 7
Imagery Dataset for Remaining Useful Life Estimation of Synthetic Fibre Ropes
A P REPRINT
Figure 3: Normalised confusion matrix for 14-class damage classification. Diagonal values are per-class recall. The dominant off-diagonal pattern is within-type severity confusion (68% of errors), confirming robust type-identity learning and targeted difficulty at adjacent severity boundaries.
4.2.2
Full Model Comparison
Table 3 consolidates DART alongside all baselines and the simple-fusion ablation variant. Three findings stand out. DART leads on all metrics. At 93.22% accuracy and 91.04% macro-F1, DART surpasses the next best model (DINOv2) by 7.0 pp and 9.2 pp respectively. This gap is meaningful: it represents the combined effect of domainspecific vision-language alignment, the SC-CMF severity gates, and the CDD loss geometry none of which are present in any baseline. Domain-specific language grounding matters more than generic VLMs. BLIP-2 and CLIP both perform visionlanguage pre-training, yet both trail DINOv2 significantly. Generic image-text pairs do not prepare a model to distinguish Chafing/Medium from Chafing/Low. DART’s paired expert descriptions tied directly to the ROPE taxonomy provide the precise semantic signal that web-crawled captions lack. 8
Imagery Dataset for Remaining Useful Life Estimation of Synthetic Fibre Ropes
A P REPRINT
Table 3: Comparison of DART against foundation baselines and ablation simple-fusion variant. Model
Type
DART (proposed) Simple Fusion (ablation E6) DINOv2 [14] I-JEPA [16] BLIP-2 [22] CLIP [21]
Vision-Language + Domain Vision-Language Vision only Vision only (JEPA) Vision-Language Vision-Language
Predicted vs Actual MAE=0.047 R²=0.868 r=0.934
Macro-F1
Weighted-F1
0.9322 0.8932 ± 0.0125 0.8620 ± 0.0038 0.7937 ± 0.0028 0.7369 ± 0.0028 0.7045 ± 0.0011
0.9104 0.8583 ± 0.0167 0.8180 ± 0.0077 0.7338 ± 0.0014 0.6550 ± 0.0096 0.6162 ± 0.0018
0.9322 0.8924 ± 0.0151 0.8604 ± 0.0044 0.7868 ± 0.0028 0.7292 ± 0.0038 0.6863 ± 0.0011
Error Distribution (centred =unbiased 100
Chafing Cut strands Placking Perfect prediction
Predicted Distribution per True Severity Level
Chafing Cut strands Placking
1.0
80
0.5
0.8 Predicted Severity
Count
Predicted Severity
1.0
Accuracy
60
40
0.6 0.4 0.2
20 0.0
0.0 Low (0.0)
Medium (0.5) True Severity
High (1.0)
0
1.00
0.75 0.50 0.25 0.00 Prediction Error (pred true)
0.25
0.50
Low (0.0)
Medium (0.5)
High (1.0)
Figure 4: Severity regression qualitative results. Each image shows predicted score (0–2 scale) and ground-truth ordinal label. Green borders: correct-ordinal (|pred − true| < 0.15); orange borders: near-miss.
The JEPA objective is a strong foundation. I-JEPA outperforms both CLIP and BLIP-2 despite having no language component, confirming that predicting in latent space rather than pixel space yields richer semantic representations for structured damage recognition. DART extends this advantage by adding cross-modal grounding on top of the JEPA backbone. 4.3
Downstream Task Evaluation
All downstream tasks use the frozen DART backbone. Lightweight task heads (linear classifiers, two-layer MLPs, or MLP regressors) are trained on the frozen features of the 3,197-image training split; results are reported on the 546-image held-out test set. Table 4 gives a consolidated overview. 4.3.1
Damage Classification
The frozen backbone with a two-layer MLP head achieves 93.22% accuracy and 91.04% macro-F1 across 14 classes on the 546-image test set (Table 5), a +38.46 pp gain over the vision-only ViT-H/14 baseline. Figure 2 shows representative correctly classified examples. Per-class analysis. Table 7 reports per-class F1. Compression (F1 = 0.992) is the most reliable class, cross-section flattening provides an unambiguous visual cue. Strand Coreout (0.978) and Chafing/High (0.984) benefit from pronounced localised signatures. Placking/Medium (0.764) is the hardest class: it visually overlaps both the mild irregularity of Low and the gross distortion of High. Coreout+Cutstrands (0.800) suffers from a test support of only n = 8. Confusion matrix. As shown in Figure 3, 68% of all errors are within-type severity confusions (e.g., Chafing/Medium ↔ Chafing/Low), indicating that damage-type identity is learned robustly while adjacent-severity discrimination remains the primary challenge. Placking/Medium is confused with Chafing/Medium in 12% of its errors due to shared surface texture. 9
Imagery Dataset for Remaining Useful Life Estimation of Synthetic Fibre Ropes
A P REPRINT
Table 4: DART performance across downstream tasks with frozen backbone. Task
Description
Primary metric
Value
Primary Task 1 Task 2 Task 3 Task 4 Task 5 Task 6
14-class damage classification Severity regression Few-shot recognition (20-shot) Damage progression modelling Maintenance recommendation Inspection report generation Anomaly detection
Macro-F1 Spearman ρ Macro-F1 Monotone rate Macro-F1 Damage accuracy Anomaly rate
91.04% 0.94 89.2% 91.0% 94.79% 93.22% 4.76%
Table 5: Damage classification on the test set. Model Accuracy Macro-F1
4.3.2
ViT-H/14 (vision only) DART (proposed)
54.76% 93.22%
49.50% 91.04%
Improvement
+38.46 pp
+41.54 pp
Severity Regression
A lightweight MLP regressor (D → 256 → 1, MSE loss) maps the frozen backbone representation to a continuous severity score (Low=0, Medium=1, High=2) for the 254 severity-bearing test images (Chafing, Cut Strands, Placking). This continuous output detects within-class deterioration progression before the discrete classifier changes its decision. As shown in Table 8 and Figure 4, the regressor achieves MAE = 0.11 (5.5% of the severity range) and Spearman ρ = 0.94. Chafing shows the tightest fit (MAE=0.08, R2 =0.97), consistent with its smooth visual progression from superficial scuffing to extensive abrasion. Placking has the largest residuals (MAE=0.15, R2 =0.91), echoing the Placking/Medium ambiguity seen in classification. The continuous score enables quantitative deterioration tracking: a rope transitioning from score 0.3 to 0.8 signals measurable progression even when both round to “Low”. 4.3.3
Few-Shot Damage Recognition
A prototypical-network classifier [39] built from k support images per class is evaluated at k ∈ {1, 5, 10, 20} over 100 randomly sampled episodes, using cosine distance to frozen DART prototypes. This evaluates whether the backbone generalises to novel damage types without retraining. As shown in Table 9 and Table 6, DART outperforms the visual-only baseline at all shot counts, approaching supervised performance (91.0% macro-F1) with only 20 labelled examples per class. 4.3.4
Damage Progression Modelling
Three analyses test whether the DART embedding space encodes a geometrically coherent severity manifold: (i) linear interpolation between class centroids, (ii) embedding arithmetic (cross-type severity transfer), and (iii) synthetic deterioration timelines via nearest-neighbour traversal. Interpolation coherence. Linear paths between Low and High centroids pass through the Medium centroid within cosine distance 0.08 (mean over Chafing, Cut Strands, Placking). Of 30 interpolation sequences, 91% progress monotonically; the 9% non-monotone dips occur exclusively near the Low–Medium boundary. Figure 7 shows the interpolation strips. Embedding arithmetic. The severity offset vsev = p̄High − p̄Low computed on Chafing embeddings transfers cross-type: applying it to Cut Strands/Low retrieves Cut Strands/High in the top-3 neighbours in 84% of queries; to Placking in 78%. Figure 5 illustrates this arithmetic. Cross-type transferability confirms that severity occupies a shared embedding subspace, a direct consequence of Lsev aligning severity representations across types. Deterioration timeline. Figure 6 shows synthetic deterioration timelines generated by nearest-neighbour traversal along the severity axis. Retrieved images progress coherently from Low to High, validating that the severity manifold encodes physically meaningful deterioration trajectories. 10
Imagery Dataset for Remaining Useful Life Estimation of Synthetic Fibre Ropes
A P REPRINT
Embedding Space Arithmetic Does A + B A+B in embedding space? 25
20
UMAP-2
15
10 dist=10.82 5
dist=23.41
0
5
Predicted Actual gap Predicted: Comp+Chafing Predicted: Comp+Cut Actual: Comp+Chafing Actual: Comp+Cut
10 10
5
0
5
UMAP-1
10
15
20
Figure 5: Embedding arithmetic: adding the Chafing-derived severity offset vector vsev to a Low-severity embedding retrieves a High-severity image of the same damage type in 84% (Cut Strands) and 78% (Placking) of queries, confirming a shared cross-type severity subspace.
Single Rope Damage Timeline Simulation
Starting from a real Low-severity image · projecting toward High-severity centroid Chafing Starting image: Chafing/Low Initial severity: 0.00
Medium -High threshold
Chafing
Cut strands
Placking
ProgressionTimeline
1.0
0.6
Medium
0.4 0.2
Low 0.0
0.2
0.4
0.6
Damage Progression
0.8
1.0
High
0.8 0.6
Medium
0.4 0.2 0.0
ProgressionTimeline
1.0
Severity Score
High
0.8
Severity Score
Severity Score
Medium -High threshold
Medium -High threshold
ProgressionTimeline
1.0
0.0
Placking Starting image: Placking/Low Initial severity: 0.00
Cut strands Starting image: Cut strands/Low Initial severity: 0.00
Low 0.0
0.2
11
0.4
0.6
Damage Progression
0.8
1.0
High
0.8 0.6
Medium
0.4 0.2 0.0
Low 0.0
0.2
0.4
0.6
Damage Progression
0.8
1.0
Figure 6: Synthetic deterioration timelines generated by nearest-neighbour search along the Low→High severity axis
Imagery Dataset for Remaining Useful Life Estimation of Synthetic Fibre Ropes
A P REPRINT
Table 6: Few-shot macro-F1 scaling curves for DART and the ViT-H/14 baseline (k ∈ {1, 5, 10, 20}, 100 episodes, 95% CI). Class
1-shot
3-shot
5-shot
10-shot
Chafing/High Cut strands/High Placking/High Coreout+Cutstrands Strand Coreout
1.000 ± 0.000 0.857 ± 0.263 1.000 ± 0.000 1.000 ± 0.000 0.990 ± 0.000
1.000 ± 0.000 0.998 ± 0.004 1.000 ± 0.000 1.000 ± 0.000 0.990 ± 0.000
1.000 ± 0.000 1.000 ± 0.000 1.000 ± 0.000 1.000 ± 0.000 0.990 ± 0.000
1.000 ± 0.000 1.000 ± 0.000 1.000 ± 0.000 1.000 ± 0.000 0.990 ± 0.000
Table 7: Per-class F1 on the 546-image test set. Damage Type Severity F1 Support Chafing Chafing Chafing Cut strands Cut strands Cut strands Placking Placking Placking Compression Compression+Chafing Strand Coreout Compression+Cutstrands Coreout+Cutstrands
High Medium Low High Medium Low High Medium Low — — — — —
Macro avg.
4.3.5
0.984 0.903 0.925 0.950 0.938 0.912 0.863 0.764 0.900 0.992 0.947 0.978 0.892 0.800
31 30 39 39 33 35 27 26 30 60 37 112 39 8
0.910
Maintenance Action Recommendation
A linear classifier maps the frozen backbone to one of four safety-graded action categories: Immediate Replace (Highseverity and Coreout+Cutstrands), Schedule Repair (Medium-severity and compound damage), Continue Monitoring (Low-severity), and No Action (Compression only). These map directly onto the decision hierarchy used in rope inspection standards. The recommender achieves 94.14% accuracy and 94.79% macro-F1, exceeding the 14-class classifier because the four-way aggregation smooths within-severity ambiguities. Representative outputs are shown in Figure 8. No Action (Compression) achieves perfect F1 = 1.00, consistent with its isolated embedding cluster. The most safety-critical category, Immediate Replace, achieves F1 = 0.92; its rare errors involve Placking/High predicted as Schedule Repair, matching Placking/High’s primary F1 of 0.863. Urgency MAE = 0.046 (normalised 0–1 scale) confirms that DART encodes continuous urgency beyond discrete category membership. 4.4
Summary
Several consistent patterns emerge across all tasks. Compression and Strand Coreout are universally reliable: primary F1 > 0.97, zero anomaly detections, and perfect or near-perfect maintenance predictions in both cases. Placking/Medium is the universal bottleneck: lowest primary F1 (0.764), highest anomaly count (5 of 26, i.e. 19.2%), and largest severity regression residuals, reflecting genuine visual ambiguity rather than a training artefact. Language grounding transfers beyond classification: the 10.6 pp few-shot improvement at 1-shot demonstrates that text descriptions contribute representational structure not recoverable from visual features alone.
5
Conclusion
We presented DART, a vision-language foundation model for comprehensive rope CM. By framing rope inspection as a multi-task problem, where a single shared representation must simultaneously support classification, regression, 12
Imagery Dataset for Remaining Useful Life Estimation of Synthetic Fibre Ropes
A P REPRINT
Table 8: Severity regression on 254 severity-bearing test images. Near-perfect within-1-ordinal accuracy (99.6%) and Spearman ρ = 0.94 confirm that DART encodes a physically meaningful severity ordering. Metric Value MAE (0–2 scale) RMSE R2 Spearman ρ Within-1-ordinal accuracy
0.11 0.16 0.95 0.94 99.6%
Table 9: Few-shot macro-F1 (± 95% CI, 100 episodes). DART consistently outperforms the visual-only baseline; the largest gain at 1-shot (+10.6 pp) reflects the well-structured embedding geometry from CDD training. k DART ViT-H/14 only 1 5 10 20
62.3 ± 2.1% 78.1 ± 1.4% 84.6 ± 1.1% 89.2 ± 0.8%
51.7 ± 2.4% 66.2 ± 1.8% 73.4 ± 1.5% 79.8 ± 1.2%
generalisation, progression modelling, recommendation, report generation, anomaly detection, and trajectory analysis, we argued that a foundation model approach is both necessary and natural. DART achieves this through a cross-modal JEPA architecture with severity-conditioned fusion (SC-CMF), saliency-guided masked reconstruction (HD-MASK), and the four-term Contrastive Damage Disentanglement loss. A single trained backbone achieves strong performance across all tasks: 93.22% classification accuracy (+38.5 pp over vision-only), Spearman ρ = 0.94 for severity regression, 89.2% macro-F1 at 20-shot recognition, 91% monotone interpolation for progression modelling, 94.79% macro-F1 for maintenance recommendation, and 4.76% anomaly rate consistent with expected borderline specimen prevalence. Ablation analysis identifies language grounding as the dominant contribution (+35.4 pp), with severity gating and saliency-guided masking adding statistically meaningful improvements. Limitations include reduced performance on rare classes (CoreOut+CutStrands, n = 8), visual ambiguity at the Placking/Medium boundary, and 45 ms/image inference latency. Future directions include a unified risk score combining instantaneous classification, continuous severity, and trajectory velocity; video-based temporal inspection replacing synthetic severity traversal; hierarchical type→severity classification; and knowledge distillation for edge deployment.
Declaration of Competing Interest The authors declare no known competing financial interests or personal relationships that could have influenced the work reported in this paper.
Data Availability Imagery Dataset for Condition Monitoring of Synthetic Fibre Ropes.
Acknowledgement This research was supported by Aalborg University, Liftra ApS (Liftra), and Dynamica Ropes ApS (Dynamica) in Denmark under the EUDP program through project grant number 64021-2048.
References [1] Wei Huang. Research directions in synthetic fiber ropes applied as mooring lines for floating offshore wind turbines. Renewable and Sustainable Energy Reviews, 225:116183, 2026. [2] Anju Rani, Daniel Ortiz-Arroyo, and Petar Durdevic. A survey of vision-based condition monitoring methods using deep learning: A synthetic fiber rope perspective. Engineering Applications of Artificial Intelligence, 136:108921, 2024. 13
Imagery Dataset for Remaining Useful Life Estimation of Synthetic Fibre Ropes
A P REPRINT
Damage Progression - Severity Score Along Embedding Trajectory Interpolating Low-Medium-High centroids in emedding space 1.0
Medium
Low
High
Chafing
High (1.0)
Predicted Severity
0.8
0.6 Med (0.5)
0.4
0.2 Progression (Low - Medium - High) Low (0.0)
0.0 0.0
Low
0.2
0.4
Low
Medium
0.6
0.8
High
1.0
High
Medium
High (1.0)
Cut strands
1.0
Predicted Severity
0.8
0.6 Med (0.5)
0.4
0.2 Progression (Low - Medium - High) Low (0.0)
0.0 0.0
Low
0.2
0.4
Medium
0.6
0.8
High
Low
1.0
High Medium
Placking
1.0
High (1.0)
Predicted Severity
0.8
0.6 Med (0.5)
0.4
0.2 Progression (Low - Medium - High) Low (0.0)
0.0 0.0
Low
0.2
0.4
Medium
0.6
0.8
High
1.0
Figure 7: Latent interpolation strips between Low and High severity centroids for Chafing (top), Cut Strands (middle), and Placking (bottom). Seven evenly spaced interpolated embeddings are retrieved by nearest-neighbour search. The Medium centroid lies within cosine distance 0.08 of the midpoint in all three cases, confirming geometric coherence of the severity manifold.
[3] Anju Rani, Daniel Ortiz-Arroyo, and Petar Durdevic. Defect detection in synthetic fibre ropes using detectron2 framework. Applied Ocean Research, 150:104109, 2024. [4] Anju Rani, Daniel O Arroyo, and Petar Durdevic. Imagery dataset for condition monitoring of synthetic fibre ropes. arXiv preprint arXiv:2309.17058, 2023. [5] Yahia Halabi, Hu Xu, Zhixiang Yu, Wael Alhaddad, and Isabelle Dreier. Experimental-based statistical models for the tensile characterization of synthetic fiber ropes: a machine learning approach. Scientific Reports, 13(1):17768, 14
Imagery Dataset for Remaining Useful Life Estimation of Synthetic Fibre Ropes
A P REPRINT
True: Immediate Replace Pred: Schedule Repair Urgency: 0.69 Class: Chafing/High
True: Immediate Replace Pred: Immediate Replace Urgency: 0.98 Class: Chafing/High
True: Immediate Replace Pred: Immediate Replace Urgency: 0.97 Class: Chafing/High
True: Schedule Repair Pred: Schedule Repair Urgency: 0.68 Class: Chafing/Medium
True: Schedule Repair Pred: Schedule Repair Urgency: 0.68 Class: Chafing/Medium
True: Schedule Repair Pred: Schedule Repair Urgency: 0.68 Class: Chafing/Medium
True: Continue Monitoring Pred: Continue Monitoring True: Continue Monitoring Pred: Continue Monitoring True: Continue Monitoring Pred: Continue Monitoring Urgency: 0.32 Class: Chafing/Low Urgency: 0.30 Class: Chafing/Low Urgency: 0.31 Class: Chafing/Low
True: No Action Pred: No Action Urgency: 0.06 Class: Compression
True: No Action Pred: No Action Urgency: 0.07 Class: Compression
True: No Action Pred: No Action Urgency: 0.06 Class: Compression
Figure 8: Representative maintenance recommendation outputs across all four action categories. Each panel shows the rope image, predicted action (colour-coded: red = Immediate Replace, orange = Schedule Repair, yellow = Continue Monitoring, green = No Action), and confidence score. Compression is consistently identified as No Action; highseverity damage triggers Immediate Replace.
2023. [6] Ruihua Tong, Hao Xu, Peijiang Wang, Qingru Zhang, and Chaoyang Hou. EdgeRopeNet: Lightweight neural network for real-time wire rope tension monitoring using FBG sensors in edge-fog mining systems. Informatica, 50(5), 2026. [7] Tao Ye, Wenyang Qin, Zongyang Zhao, Xiaozhi Gao, Xiangpeng Deng, and Yu Ouyang. Real-time object detection network in uav-vision based on cnn and transformer. IEEE Transactions on Instrumentation and Measurement, 72:1–13, 2023. [8] Max Ferguson, Ronay Ak, Yung-Tsun Tina Lee, and Kincho H Law. Detection and segmentation of manufacturing defects with convolutional neural networks and transfer learning. Smart and sustainable manufacturing systems, 2(1):137–164, 2018. [9] Jilin Wei, Juwei Zhang, and Hongli Wang. Detection of surface damage on steel wire ropes based on improved u-net. Journal of Failure Analysis and Prevention, 25(1):458–467, 2025. 15
Imagery Dataset for Remaining Useful Life Estimation of Synthetic Fibre Ropes
A P REPRINT
[10] Yan Peng, Junde Liu, Junjie He, Yongjun Qiu, Xie Liu, Le Chen, Fengfeng Yang, Bulong Chen, Bin Tang, and Yuhan Wang. Steel wire rope damage width identification method based on residual networks and multi-channel feature fusion. Machines, 12(11):744, 2024. [11] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017. [12] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020. [13] Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 16000–16009, 2022. [14] Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193, 2023. [15] Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pages 9650–9660, 2021. [16] Mahmoud Assran, Quentin Duval, Ishan Misra, Piotr Bojanowski, Pascal Vincent, Michael Rabbat, Yann LeCun, and Nicolas Ballas. Self-supervised learning from images with a joint-embedding predictive architecture. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 15619–15629, 2023. [17] Shentong Mo and Shengbang Tong. Connecting joint-embedding predictive architecture with contrastive selfsupervised learning. Advances in neural information processing systems, 37:2348–2377, 2024. [18] Yongchao Huang. Vjepa: Variational joint embedding predictive architectures as probabilistic world models. arXiv preprint arXiv:2601.14354, 2026. [19] Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748, 2018. [20] Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9729–9738, 2020. [21] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pages 8748–8763. PmLR, 2021. [22] Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In International conference on machine learning, pages 19730–19742. PMLR, 2023. [23] Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems, 36:34892–34916, 2023. [24] AI Meta. Llama 3 model card. GitHub https://github. llama/llamamodels/blob/main/models/llama3_1/MODEL_CARD. md. Accessed, 21, 2024.
com/meta-
[25] Zhengxing Chen, Yang Zou, Vicente A González, Jason Ingham, and Liam M Wotherspoon. Bridge inspection using a multi-modal vision language model. In Proceedings of The Sixth International Confer, volume 22, pages 578–588, 2025. [26] Wei-Lun Tsai, Phuong-Linh Le, Wang-Fat Ho, Nai-Wen Chi, Jacob J Lin, Shuai Tang, and Shang-Hsien Hsieh. Construction safety inspection with contrastive language-image pre-training (clip) image captioning and attention. Automation in Construction, 169:105863, 2025. [27] Hongyang Lei, Xiaolong Cheng, Qi Qin, Dan Wang, Kun Fan, Huazhen Huang, Qingqing Gu, Yetao Wu, Zhonglin Jiang, Yong Chen, et al. M3-jepa: Multimodal alignment via multi-gate moe based on the joint-embedding predictive architecture. arXiv preprint arXiv:2409.05929, 2024. [28] Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc Le, Geoffrey Hinton, and Jeff Dean. Outrageously large neural networks: The sparsely-gated mixture-of-experts layer. arXiv preprint arXiv:1701.06538, 2017. 16
Imagery Dataset for Remaining Useful Life Estimation of Synthetic Fibre Ropes
A P REPRINT
[29] José Ferreira, Roya Darabi, Armando Sousa, Frank Brueckner, Luís Paulo Reis, Ana Reis, João Manuel RS Tavares, and João Sousa. Gen-jema: enhanced explainability using generative joint embedding multimodal alignment for monitoring directed energy deposition. Journal of Intelligent Manufacturing, pages 1–26, 2025. [30] Yijing Wang, Xu Tang, Jingjing Ma, Xiangrong Zhang, Fang Liu, and Licheng Jiao. Cross-modal remote sensing image–text retrieval via context and uncertainty-aware prompt. IEEE Transactions on Neural Networks and Learning Systems, 36(6):11384–11398, 2024. [31] Paul Bergmann, Michael Fauser, David Sattlegger, and Carsten Steger. Mvtec ad–a comprehensive real-world dataset for unsupervised anomaly detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 9592–9600, 2019. [32] Thomas Defard, Aleksandr Setkov, Angelique Loesch, and Romaric Audigier. Padim: a patch distribution modeling framework for anomaly detection and localization. In International conference on pattern recognition, pages 475–489. Springer, 2021. [33] Karsten Roth, Latha Pemula, Joaquin Zepeda, Bernhard Schölkopf, Thomas Brox, and Peter Gehler. Towards total recall in industrial anomaly detection. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 14318–14328, 2022. [34] Jiaqi Liu, Guoyang Xie, Jinbao Wang, Shangnian Li, Chengjie Wang, Feng Zheng, and Yaochu Jin. Deep industrial image anomaly detection: A survey. Machine Intelligence Research, 21(1):104–135, 2024. [35] Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin. A simple unified framework for detecting out-ofdistribution samples and adversarial attacks. Advances in neural information processing systems, 31, 2018. [36] Dan Hendrycks, Steven Basart, Norman Mu, Saurav Kadavath, Frank Wang, Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo, et al. The many faces of robustness: A critical analysis of out-of-distribution generalization. In Proceedings of the IEEE/CVF international conference on computer vision, pages 8340–8349, 2021. [37] Shashanka Venkataramanan, Kuan-Chuan Peng, Rajat Vikram Singh, and Abhijit Mahalanobis. Attention guided anomaly localization in images. In European conference on computer vision, pages 485–503. Springer, 2020. [38] Yaqing Wang, Quanming Yao, James T Kwok, and Lionel M Ni. Generalizing from a few examples: A survey on few-shot learning. ACM computing surveys (csur), 53(3):1–34, 2020. [39] Jake Snell, Kevin Swersky, and Richard Zemel. Prototypical networks for few-shot learning. Advances in neural information processing systems, 30, 2017. [40] Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In International conference on machine learning, pages 1126–1135. PMLR, 2017. [41] Jingyi Liao, Xun Xu, Manh Cuong Nguyen, Adam Goodge, and Chuan Sheng Foo. Coft-ad: Contrastive fine-tuning for few-shot anomaly detection. IEEE Transactions on Image Processing, 33:2090–2103, 2024. [42] Hu Feng, Kechen Song, Wenqi Cui, Yiming Zhang, and Yunhui Yan. Cross position aggregation network for few-shot strip steel surface defect segmentation. IEEE Transactions on Instrumentation and Measurement, 72:1–10, 2023. [43] Chuanjiang Li, Shaobo Li, Huan Wang, Fengshou Gu, and Andrew D Ball. Attention-based deep meta-transfer learning for few-shot fine-grained fault diagnosis. Knowledge-Based Systems, 264:110345, 2023. [44] Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. Focal loss for dense object detection. In Proceedings of the IEEE international conference on computer vision, pages 2980–2988, 2017. [45] Ekin D Cubuk, Barret Zoph, Jonathon Shlens, and Quoc V Le. Randaugment: Practical automated data augmentation with a reduced search space. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, pages 702–703, 2020. [46] Charles R Farrar and Keith Worden. An introduction to structural health monitoring. Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 365(1851):303–315, 2007. [47] Xingzhong Nong, Xu Luo, Shan Lin, Yanmei Ruan, and Xijun Ye. Multimodal deep neural network-based sensor data anomaly diagnosis method for structural health monitoring. Buildings, 13(8):1976, 2023. [48] Hung V Dang, Hoa Tran-Ngoc, Tung V Nguyen, Thanh Bui-Tien, Guido De Roeck, and Huan X Nguyen. Data-driven structural health monitoring using feature fusion and hybrid deep learning. IEEE Transactions on Automation Science and Engineering, 18(4):2087–2103, 2020. [49] Francesco Soldovieri, Felice C Ponzo, Rocco Ditommaso, and Vincenzo Cuomo. Multimodal sensing for sustainable structural health monitoring of critical infrastructures and built environment. In Multimodal Sensing and Artificial Intelligence: Technologies and Applications II, volume 11785, pages 31–39. SPIE, 2021. 17
Imagery Dataset for Remaining Useful Life Estimation of Synthetic Fibre Ropes
A P REPRINT
[50] Xubing Xu, Xin Lan, Yonglai Zheng, Chenyu Hou, and Zhengxie Zhang. Multimodal deep learning with integrated automatic labeling for structural damage detection in high-pile wharves. Ocean Engineering, 340:122457, 2025.
18