arXiv CSopen access

ASTER: Latent Pseudo-Anomaly Generation for Unsupervised Time-Series Anomaly Detection

Unknown · 2026 · arxiv_cs

arXiv CS · Papers · License: Open Access · 2026

machine learning, deep learning, neural networks

arXiv:2604.13924v1 [cs.LG] 15 Apr 2026

Romain Hermary, Samet Hicsonmez, Dan Pineau, Abd El Rahman Shabayek, and Djamila Aouada University of Luxembourg, Luxembourg {firstname.lastname}@uni.lu

Abstract. Time-series anomaly detection (TSAD) is critical in domains such as industrial monitoring, healthcare, and cybersecurity, but it remains challenging due to rare and heterogeneous anomalies and the scarcity of labelled data. This scarcity makes unsupervised approaches predominant, yet existing methods often rely on reconstruction or forecasting, which struggle with complex data, or on embedding-based approaches that require domain-specific anomaly synthesis and fixed distance metrics. We propose ASTER, a framework that generates pseudoanomalies directly in the latent space, avoiding handcrafted anomaly injections and the need for domain expertise. A latent-space decoder produces tailored pseudo-anomalies to train a Transformer-based anomaly classifier, while a pre-trained LLM enriches the temporal and contextual representations of this space. Experiments on three benchmark datasets show that ASTER achieves state-of-the-art performance and sets a new standard for LLM-based TSAD. Keywords: Anomaly Detection · Time-Series · Pseudo-Anomalies · LLM.

Introduction

Time-series anomaly detection (TSAD) focuses on identifying patterns or behaviours in temporal data that deviate from expected dynamics. It plays a critical role in a wide range of real-world applications, including industrial fault diagnosis [45], financial fraud detection [13], healthcare monitoring [22], and cybersecurity [18]. Unlike anomaly detection on static supports, such as images or 3D data [25,52,24], TSAD must account for temporal dependencies between observations and evolving data distributions, making the task considerably more complex. Specifically, detecting anomalies in time-series is inherently challenging due to the rarity of anomalous events, their heterogeneous temporal patterns, and their dependence on contextual information, requiring models to account for both temporal dynamics and interactions across multiple features. These challenges are compounded by the scarcity of labelled anomalies in practical https://gitlab.com/uniluxembourg/snt/cvi2/open/space/aster-tab

R. Hermary et al.

scenarios, motivating the development of unsupervised approaches [58,34] that can effectively model normal behaviour and detect deviations without reliance on annotated data. Among those methods, we find embedding-based strategies [34,30]. They aim to learn a transformation that maps normal patterns to a dense region in the representation space, effectively separating them from abnormal patterns. In this paradigm, we commonly find approaches that aim to maximise the distance between the embeddings of normal data and synthetically generated anomalies–or pseudo-anomalies. For these methods, the accurate and realistic generation of synthetic anomalies is crucial for training a reliable anomaly detection model. To this end, most rely on domain knowledge to tediously curate lists of possible anomaly types. These selective lists are then used as a basis for defining transformations, such as adding bias, noise, or masking. These corruptive transformations can then be applied to normal data to generate pseudo-anomalies. However, considering the nature of the multi-variate TSAD problem, this procedural, data augmentation-dependent anomaly generation is confined to the curated lists and restricts the performance and generalisation of the models. An alternative direction could involve making the transformations adapt automatically to the data by learning them; and to further improve generalisation, applying them within a less domain-specific space, i.e., a latent representation space. This, however, requires the ability to construct expressive latent representations that capture temporal and cross-feature dynamics.Unfortunately, robust foundational models for time-series remain scarce, owing to data heterogeneity and limited availability [9]. Recently, Large Language Models (LLMs) have been adapted to solve various time-series problems, such as forecasting [29], classification [70], and anomaly detection [57]. These pre-trained models have proven to transfer helpful cross-domain and cross-modality knowledge thanks to their strong representation capabilities [44,54]. Additionally, given the rapid advancements and expanding capabilities of LLMs, formulating a TSAD method adapted to their architecture represents a valuable research opportunity. In this work, we introduce ASTER, Anomaly Synthesis through Transformerbased Embedding Representation, a novel framework for unsupervised time-series anomaly detection (TSAD). ASTER generates pseudo-anomalies directly in latent space via learnt transformations, avoiding handcrafted, domain-specific augmentations in the raw data space. A pre-trained LLM is first used to encode time-series windows into rich, contextualised embeddings. These embeddings are then processed by a VAE-based module, the perturbator, which synthesises diverse and non-trivial pseudo-anomalies. Unlike metric-based approaches that depend on fixed distance measures, ASTER trains a Transformer-based classifier to learn a flexible decision boundary. This design enables ASTER to capture complex abnormal behaviours without relying on predefined anomaly types. Our results show that ASTER achieves a new state-of-the-art performance on multiple datasets and improves upon the previous best LLM-based methods. We summarise our contributions as follows:

ASTER: Time-Series Anomaly Detection

– We propose an unsupervised TSAD framework that generates pseudo-anomalies in latent space, eliminating domain-specific augmentations and improving generalisation. – We introduce ASTER, a novel architecture featuring a VAE-based perturbator that models normal data and learns to generate pseudo-anomalies. It is paired with a Transformer-based classifier that learns a discriminative decision boundary. – We demonstrate how pre-trained LLMs can be effectively leveraged as contextual feature extractors for TSAD. – We validate our method using the rigorous and unified TAB benchmark [49]. This addresses inconsistencies in prior evaluations by enabling fair and reproducible comparisons across diverse TSAD methods. The remainder of the paper is organised as follows: we provide an extensive review of related works in Section 2. Next, we formally introduce the problem and the proposed method in Section 3 and Section 4, respectively. Our results and ablation studies are presented in Section 5. We then discuss certain limitations and directions for future work in Section 6, and conclude in Section 7.

Related Work

In this section, we first provide a summary of TSAD methods, then focus on methods that actively refine embeddings of time-series data to reflect temporal patterns and elaborate on the adoption of LLMs for various time-series problems. Finally, we explain the efforts to provide a comprehensive benchmark for a fair evaluation and comparison of TSAD methods. 2.1

Time-Series Anomaly Detection

Early TSAD research relied on classical statistical techniques [66] and traditional machine learning methods, including autoregressive models, hypothesis testing, and distance-, density-, or isolation-based approaches [10,8,38]. However, these methods often struggle with complex temporal dependencies and non-linear patterns in high-dimensional data. Deep learning has enabled models that learn expressive representations directly from raw inputs [52]. LSTM-based models [27], VAEs [64], and diffusionbased methods [12] improve the reconstruction of normal patterns, while Generative Adversarial Networks (GANs) [67,69] introduce adversarial anomaly synthesis, and one-class classifiers like Deep SVDD [52] enhance robustness to noisy pseudo-labels. Unsupervised paradigms dominate due to a scarcity of labelled anomalies. Reconstruction-, forecasting-, and imputation-based [4,56,62] methods detect anomalies through deviations from expected patterns. Self-supervised techniques, such as contrastive learning, masked modelling, or transformation prediction [65], further improve latent representations. Transformer-based models [58,42,60] capture long-range dependencies efficiently, enhancing performance on complex multivariate time series.

2.2

R. Hermary et al.

Embedding-based TSAD

Shaping appropriate data embeddings suggests extracting informative features that capture the underlying data structures without task-specific supervision [5,35]. This is, in fact, closely related to the field of representation learning [65]. Contrastive learning [46,11] has emerged as a powerful paradigm for unsupervised time-series anomaly detection (TSAD) [14]. By maximising agreement between augmented views of the same sample (or similar ones) while distancing others, these models learn discriminative embeddings without labels. Current contrastive TSAD methods, such as CTAD [30] and CATS [34], rely on specific augmentations (e.g., masking, additive bias) to generate synthetic anomalies as negative samples, maximising their distance from normal representations within the contrastive loss. However, these strategies often require dataset-specific hyper-parameter tuning, limiting generalisation. CARLA [14] partially alleviates this by confining augmentation-based anomaly generation to a pre-training stage. Despite their success, these methods remain limited by manual, domain-intensive augmentation design in raw input space. Latent space exploration, on the other hand, has traditionally focused on modelling normal data distributions. A prominent example is DAGMM [72,6], which utilises Gaussian Mixture Models (GMMs) in the latent space to identify low-probability deviations as anomalies. Similarly, TimeVQVAE-AD [36] employs a two-stage time-frequency auto-encoder to detect anomalies based on estimated densities and reconstruction quality. An increasing interest in latent space manipulation is also noticeable; for instance, L-GTA [51] applies augmentations directly in the latent space to enhance reconstruction robustness. Departing from these approaches, our method does not seek to improve reconstruction performance, nor do we rely on passive density modelling of normal data. Instead, we leverage the latent space to model a specific distribution for pseudo-anomalies. By sampling directly from this learnt latent distribution, we circumvent the need for heuristic-driven augmentations in the raw input space, which are often suboptimal and domain-dependent. This strategy facilitates a more versatile, dataset-agnostic representation that removes the requirement for extensive domain knowledge. Consequently, the latent space and pseudo-anomaly generation process are optimised for the downstream task of anomaly detection. 2.3

LLMs & Time-Series

Recently, Large Language Models (LLMs) [50,16], which have excelled at capturing sequential relations in text data, have been adopted in the time-series domain to solve various tasks, such as forecasting [23,9,56,55,29], reasoning [63], and anomaly detection [70,37,2,57,39,71]. GPT4TS [70] is an early work that adapts a pre-trained GPT-2 [50] model for time-series tasks, including anomaly detection, using lightweight fine-tuning. Following this direction, UniTime [41] introduces masking-based input processing and incorporates domain knowledge via textual descriptions. AnomalyLLM [37] adopts a knowledge distillation framework, where a Transformer-based student

ASTER: Time-Series Anomaly Detection Contextual Embedding

Φ Wt∗ Wt

C∗t

INFERENCE

Φ0

Φ1

Linear

LLM

Classifier

ASTER

···

s∗t c

Ψ0

Ψ1

Transformer

[CLS]

ar ne Li

{0, 1} {0, 1}

Ct P

Normality Encoding

et C

Anomaly Generation

Z(i) ∼ qϕ (Z | Ct )

qϕ (Z | C) Transformer

Perturbator [Training Only]

Transformer

pψ (C̃ | Z)

gθ (Z(i) )

▷◁

c̃

···

c̃

▷◁ Reverse Gradient Stop Gradient

Transformer n

bt C

n-tokens feed

Fig. 1: Overview of the proposed method. It consists of three components: Contextual Embedding (Φ) extractor, Perturbator (P), and Classifier (Ψ ). The Contextual Embedding maps the input time series to a high-dimensional space using a pre-trained LLM. The Perturbator learns a latent representation via reconstruction (Pn ) and generates pseudo-anomalies (Pa ). The Classifier distinguishes between normal and pseudo-anomalous data.

model learns from a GPT-2-initialised teacher, leveraging reconstruction errors for anomaly detection. Another line of work explores the zero-shot capabilities of LLMs for timeseries anomaly detection. SIGLLM [2] formulates anomaly detection as either direct prompting or forecasting-based discrepancy estimation. LLMAD [39] employs retrieval-augmented in-context learning to mimic expert reasoning, while AnomLLM [71] systematically analyses common assumptions in LLM-based TSAD, showing that longer input windows and Chain-of-Thought prompting do not consistently improve performance. Motivated by these findings, we leverage a pre-trained LLM as a contextual feature extractor with minimal fine-tuning to bridge domain gaps, without introducing additional textual prompts or expert-defined context. 2.4

Benchmarking TSAD Models

Fair benchmarking of TSAD methods is crucial for accurately assessing progress in the field. Methods that achieve state-of-the-art performance under specific evaluation protocols or metrics, e.g., point-adjusted F1 -score, often perform poorly when evaluated using more realistic criteria, such as the standard F1 score. To address this issue, several benchmarking efforts have been proposed for both univariate and multivariate TSAD [68,7]. While these benchmarks eval-

R. Hermary et al.

uate a wide range of methods, they largely exclude emerging TSAD paradigms, including LLM-based approaches and pre-training-based foundation models. Recently, TAB [49] was introduced as a unified benchmark that standardises dataset preprocessing, model training, and metric computation across methods, enabling fair and reproducible comparisons. We, therefore, adopt the TAB benchmark code and their baselines for all evaluations in this study.

Problem Formulation

Let X 0 := {xt ∈ RD }Tt=1 be a multivariate time-series of length T , with corresponding labels Y 0 := {yt ∈ {0, 1}}Tt=1 , indicating normal and anomalous observations. The goal of TSAD is to learn a function f that assigns an anomaly score st = f(·) to each observation. Since anomalies are context-dependent, f operates on a window of the past L observations, i.e., st = f([xt−L+1 , . . . , xt ]), where f : RL×D → R. We thus process the time-series as a set of overlapping windows W 0 := {Wt }Tt=L , with Wt := [xt−L+1 , . . . , xt ]. ′ At inference, an unseen time series X ∗ := {xt }Tt=1 , drawn from the same domain as X 0 , is used for evaluation. Anomaly scores are computed sequentially ′ for each window in W ∗ , yielding S ∗ := {s∗t }Tt=L . A decision threshold τ ∈ R ′ is then applied to obtain predicted labels Yb∗ := {ŷt∗ }Tt=L , where ŷt∗ = I[s∗t ≥ τ ] ∈ {0, 1} indicates an anomalous time-step. Finally, comparing Yb∗ with the ground-truth labels Y ∗ provides the model’s performance on the unseen series. The task is challenging for two reasons: anomaly detection is context-dependent, and the setting is unsupervised (yt = 0, ∀t), making learning f non-trivial. ft To prevent degenerate solutions, we generate a pseudo-anomalous window W for each input Wt , with the challenge being to create meaningful windows.

Methodology

We present our architecture in Figure 1. The final objective f combines a contextual embedding Φ : RL×D → RL×M (Section 4.1) and a classifier Ψ : RL×M → R (Section 4.2), so that for an unseen window Wt∗ the anomaly score is s∗t = Ψ ◦ Φ(Wt∗ ). To train Ψ effectively, we introduce a pseudo-anomaly generator, the perturbator P : RL×M → RL×M , which produces plausible artificial anomalies (Section 4.3). 4.1

Time-Series to Contextualised Embeddings

Anomalies in time-series are highly context-dependent, motivating the use of LLMs to capture rich sequential patterns. This is the role of Φ, which we divide into two parts: Φ0 : RD → RM , a time-step embedding, and Φ1 : RL×M → RL×M , which adds sequential context. Translation. Observations are mapped from the raw time-series to the token space via a linear projection, which has been shown effective in prior

ASTER: Time-Series Anomaly Detection

works [70,40,53]. This step only embeds individual time-steps, not cross-temporal knowledge: Φ0 : RD → RM

[l]

Φ0 (Wt ),

l ∈ t − L + 1...t

(1)

LLM Knowledge. The LLM incorporates the entire window with prior sequential knowledge, yielding a contextualised window Ct ∈ RL×M when combined with the translation module (Equation 2). Φ0 is trained from scratch, while Φ1 is only fine-tuned with LoRA [26], using gradients from the classification loss CE on normal windows ∇Φ = ∂L ∂Ct . This keeps part of the network independent of pseudo-anomaly generation, reducing the risk of collapse or overfitting. h it [l] Ct = Φ(Wt ) = Φ1 ◦ Φ0 (Wj ) . (2) l=t−L+1

4.2

Classifier

For the classifier Ψ , we follow a BERT-like strategy [16]. The contextualised tokens Ct are first processed by Ψ0 : RL×M → RL×M (Transformer-based network) to gather contextual information into a [CLS] token. This token is selected using Π1 (X) := [X]:,1 and projected to a final score by Ψ1 : RM → R: Ψ := Ψ1 ◦ Π1 ◦ Ψ0 ,

st = Ψ (Ct ).

(3)

Applying a sigmoid, s̄t = 1+e1−st , allows optimisation with the binary crossentropy loss (Equation 4). Together with the contextual embedding Φ, Ψ completes the required components for the inference pipeline of ASTER. LCE = − [yt log(s̄t ) + (1 − yt )log(1 − s̄t )] . 4.3

(4)

Perturbator

As mentioned in Section 4, during training we use a perturbator P to generate pseudo-anomalous samples and balance the data. We establish three essential requirements for the perturbator: 1. Adaptive generation. The process should automatically capture the structure of the data. Current methods rely on fixed transformations (e.g., noise, shuffling, masking), which perform inconsistently across heterogeneous datasets. A robust method must generalise without dataset-specific tuning; 2. Non-trivial anomalies. While a simple alteration, such as adding random noise, suffices to simulate anomalies, these pseudo-anomalies are obvious when compared to the subtle, context-dependent nature of real anomalies. Pseudoanomalies should remain correlated with normal data while being hard to detect; 3. Diverse anomalies. A reliable method must generalise across a wide range of anomalies to be effective. A few realistic examples are insufficient to ensure robustness against arbitrary cases.

R. Hermary et al.

Positional Encoding

⊕

qϕ,0

.. .

...

.. .

µl µµl

qϕ,σ

Z(i) ∼ qϕ (Z | Ct )

.. .

1. Window Encoding FeedForward

CrossAttention

3.b. Pseudo-Anomalies Generation

2. Feature Sampling

σl σσl

SelfAttention

qϕ,µ

pψ

bt C

qϕ (Z | C) ...

FeedForward

3.a. Windowed-Features Reconstruction

CrossAttention

SelfAttention

Pseudo-Anomalies

et C Z(i)

Fig. 2: Detailed description of the Perturbator architecture. It includes a complete Variational Auto-Encoder (purple), with an encoder qϕ that maps the input features Ct to latent matrix Z(i) , and a decoder gθ . The pseudo-anomalies are generated using the additional decoder pψ (red), starting from the same latent matrix. We also use positional encoding added to the inputs of each sub-network.

We address (1) and (2) by learning a distribution for anomalous windows, which enables automatic adaptation and estimation of anomaly difficulty. To reduce bias from the original data, we generate pseudo-anomalies directly in the contextualised space. We learn and model their distribution, p(C̃), based on the estimated density of normal windows p(C). To satisfy (3), we use a Variational Auto-Encoder (VAE) [32], which provides a simple latent space and inherent diversity via their auxiliary variable ϵ ∼ N (0, I). Our approach models p(C) with an auto-encoding strategy (Section 4.3) and uses its latent space to generate pseudo-anomalies (Section 4.3). A detailed overview is presented in Figure 2. Modelling p(C). The core of VAE passes through a latent variable Z and two distribution estimations qϕ (Z | C) and pθ (C), parametrised by ϕ and θ R respectively. By defining pθ (C) with its marginal over Z, pθ (C, Z) dZ, we can write the relation between log pθ (C) and the Evidence Lower Bound (ELBO) as: log pθ (C) ≥ Eqϕ (Z|C) log pθ (C | Z) − DKL qϕ (Z | C) ∥ p(Z) ≥ ELBO(θ, ϕ; C).

(5) (6)

Maximising the ELBO with respect to the parameters θ and ϕ yields a tractable optimisation objective that approximates the true maximum likelihood estimation of the underlying data distribution p(C). We thus obtain the optimisation objective for the modelling of p(C): LELBO := −Eqϕ (Z|C) [log pθ (C | Z)] + DKL qϕ (Z | C) ∥ p(Z) .

(7)

ASTER: Time-Series Anomaly Detection

A key modelling choice in VAEs is to consider pθ (C | Z) = N (C; gθ (Z), I), where gθ : RM → RM is the reconstruction function. This assumption allows to reduce the previous equation to: LELBO ≡ Eqϕ (Z|C) ∥C − gθ (Z)∥2 + DKL qϕ (Z | C) ∥ p(Z) .

(8)

Finally, the prior p(Z) is also chosen to be simple, p(Z) = N (0, I); defining b = gθ (Z) as the reconstructed window, we have the second objective of our C entire model: b t ∥2 + 1 LELBO (Ct ) = ∥Ct − C 2L

L X M X 2 µl + σl2 − log σl2 − 1 k ,

(9)

l=1 k=1

where [·]k denotes the value at the k th dimension of the vector. Note that we estimate the couple of parameters (µ, σ) for each contextualised time-step in the window. Thus, we consider a set of parameters Pt = {(µl , σl )}L l=1 for each window, which, on average, should follow the same distribution as the prior p(Z). We refer to this part of the network as Pn in Figure 1.

Anomalies from qϕ (Z | C). While the first component of the perturbator, Pn , models the distribution of normal windows p(C), we introduce a parallel e Since these distributions component Pa to infer the anomalous distribution p(C). e | C), we leverage the learned are correlated, instead of directly estimating p(C e latent estimator qϕ (Z | C) and model pψ (C | Z), parametrised by ψ. Sampling from qϕ (Z | C) naturally induces diversity in the generated anomalies. e | Z) cannot be learnt genAs no anomalous examples are available, pψ (C eratively like gθ . Instead, inspired by adversarial learning [20], we optimise Pa CE through the classification objective using a reverse gradient, ∇Pa = − ∂L e t . In ∂C e | Z) is approximated only locally this case, however, the learnt distribution p(C around the decision boundary. The gradient ∇Pa vanishes for samples already confidently classified as anomalous, but these distant anomalies are trivial and do not contribute meaningful information; hence, we consider this acceptable.

Architecture. The perturbator is built around the Transformer architecture with an innovative configuration. The encoder qϕ parametrises a latent distribution of C; the decoders (gθ and pψ ) are conditioned on the resulting sampled latent window Z(i) via cross-attention. To ensure that information is conveyed through the latent Z, i.e., the reconstruction and generation processes depend on the latent representation, we use positional encodings alone as the decoders inputs. Full architectural details and implementation specifics are provided in the supplementary material.

R. Hermary et al.

Experiments

In this section, we describe the datasets used in our experiments, outline the baseline methods for comparison. We then report the experimental results and conclude with an ablation study that evaluates the contribution of each component of our approach. 5.1

Datasets

We experimented with four popular multi-variate time-series anomaly detection datasets to validate the effectiveness of ASTER. The datasets encompass different domains; specifically, they are: 1) Pooled Server Metrics (PSM) [1], 2) PUMP [17], 3) Secure Water Treatment (SWaT) [45], and 4) Controlled Anomalies Time Series (CATSv2) [19]. The statistics of the datasets can be found in supplementary. The curated version of the CATSv2 dataset in the TAB benchmark contains 4 subsets, and we train and test on each one. The final results are presented by averaging the metrics for all the subsets. We use standard scaling, i.e., centering the features by removing the mean and scaling them to unit variance as a pre-processing step on all datasets. 5.2

Experimental Setup

Metrics. We evaluate our method using five metrics: one threshold-based metric, the F1 -score, and four rank-based metrics, AUROC, AUPR, VUS-AUROC, and VUS-AUPR. The Volume Under the Surface (VUS) metrics [47] extend traditional measures into a three-dimensional space by incorporating a temporal buffer size, which accounts for slight early or late detections. While AUROC and AUPR evaluate performance across thresholds, they are sensitive to temporal misalignments and ignore the range-based nature of anomalies. VUS calculates the area under ROC or PR curves across all buffer sizes up to the maximum anomaly length, producing a threshold-independent volume that captures a model’s robustness to noise, shifts and timing variations. For the F1 -score, an optimal threshold is needed to convert scores into binary labels. In TAB, ten percentile values from the classifier outputs on train and test sets are assessed, and the percentile that maximises the score is used as the threshold. Point Adjustment (PA)-based metrics, commonly used in TSAD studies, are excluded here due to their tendency to inflate performance and obscure true temporal precision [30,14,31,34]. Baselines. We compared ASTER with 11 baseline methods, which were fairly evaluated under the same protocol in the most recent AD benchmark, TAB [49]. We group them into three different categories based on the main approaches they are using; Deep Learning-based methods, DAGMM [72], TimesNet [61], and TranAD [58], large Pre-Trained Models, Timer [42], TimesFM [15], Moment [21], and Chronos [3]; LLM-based methods, GPT4TS [70], UniTime [41], CALF [40], and LLMMixer [33]. We directly use the results for the baselines from the TAB [49] benchmark without any retraining or inference.

ASTER: Time-Series Anomaly Detection

Table 1: Comparison of TSAD methods on the PSM, PUMP, SWAT and CATSv2 datasets. Baseline results are directly taken from TAB benchmark [49], and for fairness our results are computed using the same benchmark. Best and second best results are bold-faced and underlined, respectively. Deep Learning-based

Pre-Trained Models

LLM-based

PSM

F1 AUROC AUPR VUS-ROC VUS-PR

0.007 0.637 0.416 0.608 0.404

0.088 0.592 0.391 0.593 0.395

0.403 0.631 0.388 0.566 0.431

0.143 0.556 0.359 0.541 0.350

0.082 -

0.117 0.545 0.332 0.540 0.329

0.131 0.565 0.370 0.547 0.362

0.089 0.580 0.376 0.576 0.374

0.097 0.579 0.377 0.579 0.377

0.102 0.589 0.370 0.587 0.374

0.186 0.593 0.385 0.579 0.383

0.512 0.697 0.501 0.631 0.452

PUMP

F1 AUROC AUPR VUS-ROC VUS-PR

0.174 0.447 0.163 0.504 0.235

0.017 0.485 0.121 0.630 0.219

0.454 0.800 0.110 0.792 0.243

0.129 0.529 0.139 0.712 0.260

0.037 0.409 0.117 0.568 0.203

0.140 0.485 0.129 0.669 0.240

0.102 0.469 0.108 0.576 0.175

0.014 0.414 0.101 0.551 0.177

0.020 0.549 0.129 0.690 0.224

0.017 0.566 0.130 0.665 0.204

0.081 0.398 0.104 0.545 0.181

0.458 0.839 0.254 0.833 0.290

SWAT

DAGMM TimesNet TranAD Timer TimesFM Moment Chronos GPT4TS UniTime CALF LLMMixer ASTER

F1 AUROC AUPR VUS-ROC VUS-PR

0.073 0.290 0.082 0.328 0.250

0.077 0.288 0.107 0.392 0.169

0.321 0.818 0.729 0.699 0.520

0.144 0.286 0.119 0.399 0.172

0.099 0.242 0.086 0.344 0.121

0.132 0.263 0.109 0.370 0.161

0.613 0.803 0.410 0.814 0.449

0.034 0.224 0.081 0.322 0.112

0.063 0.234 0.082 0.335 0.115

0.075 -

0.098 0.240 0.086 0.342 0.120

0.695 0.823 0.542 0.757 0.506

CATSv2

Data Metrics

F1 AUROC AUPR VUS-ROC VUS-PR

0.088 0.619 0.090 0.667 0.314

0.262 0.712 0.261 0.772 0.265

0.126 0.604 0.129 0.625 0.130

0.541 0.112 0.584 0.132

0.129 0.633 0.109 0.705 0.125

0.060 -

0.120 0.624 0.103 0.705 0.112

0.089 0.601 0.094 0.676 0.107

0.334 0.039 0.373 0.058

0.138 0.616 0.133 0.689 0.145

0.081 0.617 0.085 0.689 0.099

0.172 0.665 0.106 0.709 0.110

5.3

Results

We present the results of ASTER and the selected baselines in Table 1. Comparing ASTER to previous LLM-based methods, our approach significantly improves upon all metrics, except for the CATSv2 dataset. For instance, on the PSM and SWaT datasets, the F1 -score is improved by more than 0.3 points, which demonstrates the effectiveness of our method. Compared to all baselines, on the PSM and PUMP datasets, ASTER marks a new state-of-the-art (SoTA) across all metrics. On the SWAT dataset, our method achieves SoTA results on F1 and AUROC, and ranks second on the other three metrics. Finally, on the challenging CATSv2 dataset, which contains very few anomalous regions, our method achieves the second best scores on a variety of metrics, following TimesNet [61]. 5.4

Ablation Study

We conduct ablation experiments on the PSM and PUMP datasets to evaluate different components of ASTER. For the LLM and fine-tuning, we compare four feature-extractor setups: a linear layer only, a frozen LLM, standard LLM fine-tuning [70], and LoRA fine-tuning (our approach). Using only the linear layer is already competitive, highlighting the effectiveness of the classifier and perturbator. Adding a frozen LLM improves AUROC on PUMP, demonstrating the benefit of contextualisation. Standard fine-tuning slightly helps PSM but harms PUMP, while LoRA achieves the best results on both datasets, showing its superiority for leveraging LLMs in TSAD (Table 2a).

R. Hermary et al.

Table 2: Ablation studies analysing (a) the impact of LLM-based feature extraction and (b) the effect of window size. (a) Effect of using LLMs and different fine- (b) Effect of using a Transformer-based classifier. tuning strategies for feature extraction. LLM

Fine Tuning

% GPT-2 GPT-2

% % !

PSM

PUMP

F1 AUROC AUPR F1 AUROC AUPR 0.479 0.354 0.361

0.669 0.496 0.499

0.447 0.021 0.389 0.370 0.417 0.265

0.460 0.802 0.707

0.087 0.199 0.146

GPT-2 !(LoRA) 0.512

0.697

0.501 0.458

0.839

0.254

Classifier

PSM

PUMP

F1 AUROC AUPR F1 AUROC AUPR

Transformer 0.512

0.697

0.501 0.458

0.839

0.254

MLP-based 0.497

0.724

0.471 0.016

0.307

0.069

To evaluate the impact of the proposed classifier architecture, we replace it with an MLP while maintaining a constant number of trainable parameters. The MLP-based classifier yields inferior performance, especially on the PUMP dataset, highlighting the importance of modelling the sequential nature of the time series (Table 2b).

Discussion & Future Work

We analyse the perturbator components Pn and Pa, and discuss the limitations and future directions (figures in supplementary). To ensure VAE non-collapse, we monitored µl and σl and observed convergence near 0 and 1 with deviations ∼ 10−5 –10−6 , while the reconstruction loss decreased alongside DKL , indicating that sufficient information is transmitted. Cosine distances between generated e t and normal windows Ct decrease during training, indicatpseudo-anomalies C ing that pseudo-anomalies become more complex while still being distinguishable by the classifier. We observed that the generated pseudo-anomalies are sometimes insufficiently diverse; however, they remain closer to real anomalies than to normal data, which aids the classifier in learning meaningful boundaries. Future work may enhance diversity through complementary latent sampling, relaxed prior constraints (e.g., MMD), or distributional constraints on Pa . Even though ASTER converges properly, we also monitored large magnitude differences in its loss components, indicating that the objectives are not being optimised equally and suggesting potential benefits from balancing strategies. Finally, the linear layer used to map time-series to token embeddings may be restrictive, and nonlinear alternatives such as MLPs or Kolmogorov–Arnold Networks [43] could manage the complex alignment better.

Conclusion

In this work, we tackle unsupervised TSAD and overcome the limitations of handcrafted, dataset-specific pseudo-anomalies by introducing ASTER, the first

ASTER: Time-Series Anomaly Detection

framework to generate pseudo-anomalies autonomously in a learnt latent space. Leveraging pre-trained LLM representations, the expressive capacity of Transformers, and the modelling power of VAEs, ASTER produces challenging pseudoanomalies to adversarially train a robust Transformer-based classifier. Extensive experiments on the TAB benchmark demonstrate that ASTER achieves state-of-the-art performance across datasets, outperforming existing LLM-based methods. Future research could focus on further enhancing the perturbator by enforcing anomaly diversity through advanced latent sampling, constraints, or more complex decoder inputs (e.g., learnable tokens), exploring the interpretability of the latent space and the generated pseudo-anomalies, and investigating various pre-trained LLMs and their differing representational performance with enhanced time-series-to-tokens translation modules. Acknowledgements. This research was funded by the Luxembourg National Research Fund (FNR), grant reference DEFENCE22/17813724/AUREA.

References 1. Abdulaal, A., Liu, Z., Lancewicki, T.: Practical approach to asynchronous multivariate time series anomaly detection and localization. In: SIGKDD (2021) 2. Alnegheimish, S., Nguyen, L., et al.: Can large language models be anomaly detectors for time series? In: DSAA (2024) 3. Ansari, A.F., et al.: Chronos: Learning the language of time series. TMLR (2024) 4. Audibert, J., Michiardi, P., et al.: Usad: Unsupervised anomaly detection on multivariate time series. In: SIGKDD (2020) 5. Bengio, Y., Courville, A., Vincent, P.: Representation learning: A review and new perspectives. TPAMI (2013) 6. Bhatnagar, A., Kassianik, P., et al.: Merlion: A machine learning library for time series. CoRR abs/2109.09265 (2021) 7. Boniol, P., Paparrizos, J., et al.: Theseus: navigating the labyrinth of time-series anomaly detection. In: VLDB (2022) 8. Breunig, M.M., Kriegel, H.P., et al.: LOF: identifying density-based local outliers. In: SIGMOD (2000) 9. Chang, C., Wang, W., et al.: LLM4TS: aligning pre-trained llms as data-efficient time-series forecasters. TIST (2025) 10. Chaovalitwongse, W.A., Fan, Y.J., et al.: On the time series k-nearest neighbor classification of abnormal brain activity. IEEE-TSMC-A (2007) 11. Chen, T., Kornblith, S., et al.: A simple framework for contrastive learning of visual representations. In: ICML (2020) 12. Chen, Y., Zhang, C., et al.: Imdiffusion: Imputed diffusion models for multivariate time series anomaly detection. In: VLDB (2023) 13. Dal Pozzolo, A., Caelen, O., et al.: Calibrating probability with undersampling for unbalanced classification. In: IEEE-SSCI (2015) 14. Darban, Z.Z., Webb, G.I., et al.: CARLA: Self-supervised contrastive representation learning for time series anomaly detection. Pattern Recognit. (2025) 15. Das, A., Kong, W., et al.: A decoder-only foundation model for time-series forecasting. In: ICML (2024)

R. Hermary et al.

16. Devlin, J., Chang, M., et al.: BERT: pre-training of deep bidirectional transformers for language understanding. In: NAACL-HLT (2019) 17. Feng, C., Tian, P.: Time series anomaly detection for cyber-physical systems via neural system identification and bayesian filtering. In: SIGKDD (2021) 18. Filonov, P., Lavrentyev, A., Vorontsov, A.: Multivariate industrial time series with cyber-attack simulation: Fault detection using an lstm-based predictive data model. CoRR abs/1612.06676 (2016) 19. Fleith, P.: Controlled anomalies time series (CATS) dataset. Technical report, Solenix Engineering GmbH, Darmstadt, Germany (2023) 20. Goodfellow, I.J., et al.: Generative adversarial nets. In: NIPS (2014) 21. Goswami, M., Szafer, K., et al.: MOMENT: A family of open time-series foundation models. In: ICML (2024) 22. Greenwald, S.D., Patil, R.S., Mark, R.G.: Improved detection and classification of arrhythmias in noise-corrupted electrocardiograms using contextual information. CinC pp. 461–464 (1990) 23. Gruver, N., Finzi, M., et al.: Large language models are zero-shot time series forecasters. NIPS (2023) 24. Gu, Z., Zhang, J., et al.: Rethinking reverse distillation for multi-modal anomaly detection. In: AAAI (2024) 25. Hermary, R., et al.: Removing geometric bias in one-class anomaly detection with adaptive feature perturbation. In: WACV. pp. 6612–6622 (2025) 26. Hu, E.J., Shen, Y., et al.: LoRA: Low-rank adaptation of large language models. In: ICLR (2022) 27. Hundman, K., Constantinou, V., et al.: Detecting spacecraft anomalies using lstms and nonparametric dynamic thresholding. In: KDD (2018) 28. Jiang, J., Xia, G., et al.: Transformer VAE: A hierarchical model for structureaware and interpretable music representation learning. In: ICASSP (2020) 29. Jin, M., Wang, S., et al.: Time-llm: Time series forecasting by reprogramming large language models. In: ICLR (2024) 30. Kim, H., et al.: Contrastive time-series anomaly detection. TKDE (2023) 31. Kim, S., Choi, K., et al.: Towards a rigorous evaluation of time-series anomaly detection. In: AAAI (2022) 32. Kingma, D.P., Welling, M.: Auto-encoding variational bayes (2014) 33. Kowsher, M., Sobuj, M.S.I., et al.: LLM-mixer: Multiscale mixing in LLMs for time series forecasting. In: TLRW (2025) 34. Ky, J.R., Mathieu, B., et al.: CATS: contrastive learning for anomaly detection in time series. In: IEEE Big Data (2024) 35. LeCun, Y., Bengio, Y., Hinton, G.E.: Deep learning. Nat. (2015) 36. Lee, D., Malacarne, S., Aune, E.: Explainable time series anomaly detection using masked latent generative modeling. Pattern Recognit. (2024) 37. Liu, C., He, S., et al.: Large language model guided knowledge distillation for time series anomaly detection. In: IJCAI (2024) 38. Liu, F.T., Ting, K.M., et al.: Isolation forest. In: SDM (2008) 39. Liu, J., Zhang, C., et al.: Large language models can deliver accurate and interpretable time series anomaly detection. In: SIGKDD (2025) 40. Liu, P., Guo, H., et al.: Calf: Aligning llms for time series forecasting via crossmodal fine-tuning. In: AAAI (2025) 41. Liu, X., Hu, J., et al.: Unitime: A language-empowered unified model for crossdomain time series forecasting. In: ACM Web Conference (2024) 42. Liu, Y., Zhang, H., et al.: Timer: generative pre-trained transformers are large time series models. In: ICML (2024)

ASTER: Time-Series Anomaly Detection

43. Liu, Z., Wang, Y., et al.: KAN: kolmogorov-arnold networks. In: ICLR (2025) 44. Lu, K., Grover, A., et al.: Frozen pretrained transformers as universal computation engines. In: AAAI (2022) 45. Mathur, A.P., Tippenhauer, N.O.: Swat: A water treatment testbed for research and training on ics security. In: CySWater (2016) 46. van den Oord, A., Li, Y., Vinyals, O.: Representation learning with contrastive predictive coding. CoRR abs/1807.03748 (2018) 47. Paparrizos, J., Boniol, P., et al.: Volume under the surface: a new accuracy evaluation measure for time-series anomaly detection. In: VLDB (2022) 48. Paszke, A., Gross, S., et al.: Automatic differentiation in pytorch (2017) 49. Qiu, X., Li, Z., et al.: TAB: Unified benchmarking of time series anomaly detection methods. In: VLDB (2025) 50. Radford, A., Wu, J., et al.: Language models are unsupervised multitask learners. Tech. rep. (2019) 51. Roque, L., Soares, C., et al.: L-GTA: latent generative modeling for time series augmentation. CoRR abs/2507.23615 (2025) 52. Ruff, L., Vandermeulen, R., et al.: Deep one-class classification. In: ICML (2018) 53. Rukhovich, D., Dupont, E., et al.: Cad-recode: Reverse engineering cad code from point clouds. In: ICCV (2025) 54. Sui, Y., Zhou, M., et al.: Table meets LLM: can large language models understand structured table data? A benchmark and empirical study. In: WSDM (2024) 55. Sun, C., Li, H., Li, Y., Hong, S.: TEST: Text prototype aligned embedding to activate LLM’s ability for time series. In: ICLR (2024) 56. Tan, M., Merrill, M., et al.: Are language models actually useful for time series forecasting? NeurIPS (2024) 57. Tao, W., Qu, X., et al.: Madllm: Multivariate anomaly detection via pre-trained llms. In: ICME (2025) 58. Tuli, S., Casale, G., Jennings, N.R.: Tranad: deep transformer networks for anomaly detection in multivariate time series data. In: VLDB (2022) 59. Vaswani, A., Shazeer, N., et al.: Attention is all you need. NIPS (2017) 60. Wen, Q., Zhou, T., et al.: Transformers in time series: A survey (2023) 61. Wu, H., Hu, T., et al.: Timesnet: Temporal 2d-variation modeling for general time series analysis. In: ICLR (2023) 62. Xiao, C., Gou, Z., et al.: Imputation-based time-series anomaly detection with conditional weight-incremental diffusion models. In: SIGKDD (2023) 63. Xie, Z., Li, Z., et al.: Chatts: Aligning time series with llms via synthetic data for enhanced understanding and reasoning. In: VLDB (2025) 64. Yao, Y., Ma, J., et al.: Regularizing autoencoders with wavelet transform for sequence anomaly detection. Pattern Recognit. (2023) 65. Yue, Z., Jin, H., et al.: TS2Vec: Towards universal representation of time series. In: AAAI (2022) 66. Zare Moayedi, H., Masnadi-Shirazi, M.: Arima model for network traffic prediction and anomaly detection. In: ISCIT. vol. 4, pp. 1–6 (2008) 67. Zenati, H., et al.: Adversarially learned anomaly detection. In: ICDM (2018) 68. Zhang, A., Deng, S., et al.: An experimental evaluation of anomaly detection in time series. In: VLDB (2023) 69. Zhou, B., Liu, S., et al.: BeatGAN: Anomalous rhythm detection using adversarially generated time series. In: IJCAI (2019) 70. Zhou, T., Niu, P., et al.: One fits all: Power general time series analysis by pretrained lm. NeurIPS (2023)

R. Hermary et al.

71. Zhou, Z., Yu, R.: Can LLMs understand time series anomalies? In: ICLR (2025) 72. Zong, B., Song, Q., Min, M.R., Cheng, W., et al.: Deep autoencoding gaussian mixture model for unsupervised anomaly detection. In: ICLR (2018) 73. Zou, K., Yu, B., Seo, H.: 3d facial expression generator based on transformer VAE. In: ICIP (2023)

ASTER: Latent Pseudo-Anomaly Generation for Unsupervised Time-Series Anomaly Detection –Supplementary–

Complementary Figure: Positioning Original

Amplify

Bias

Dropout

1 0 −1

Fixed Transformations

Distance-based Scoring Mask

∆ 0

100

∆ 50

∆2

...

100

Amplify

100

Learned Transformations in Latent Space

Trained Binary Classifier

(a)

(b)

ASTER

Multivariate Time-Series

Previous Methods

Noise

Permute

Fig. 3: Main differences between previous methods and ASTER; (a) the pseudoanomaly generation, (b) the scoring technique. Previous methods curate a list of augmentations to simulate the possible anomalies that exist in the time-series datasets. However, our method is free from these assumptions and learn to generate the pseudo-anomalies automatically in the latent space. Also, the training objective of these methods is to maximise the distance between pseudo-anomalies and normal data. This approach results in specialised distance metrics (e.g., cosine, L2 ) for each dataset based on the characteristics. ASTER relaxes this by training a binary classifier. Red and blue dots denote anomalies and normal data, respectively.

R. Hermary et al.

Datasets Statistics

Dataset

Train

Test Features Anomalies (%)

PSM [1] 132,481 87,841 PUMP [17] 76,901 143,401 SWaT [45] 495,000 449,919 CATSv2 [19] 113,887 576,113

25 44 51 17

27.76 10.05 12.14 3.21

Table 3: Details of the datasets used in experiments.

Algorithms

We present the main steps of ASTER’s inference algorithm in Algorithm 1. The training algorithm is depicted in Algorithm 2; the steps during back-propagation are referred to as BP. Algorithm 1 Inference Pipeline of ASTER Require: – Unseen input window Wt∗ – Feature extractor Φ – Classifier Ψ Feature Extraction C∗t ← Φ(Wt∗ ) Scoring (Classification) s∗t = Ψ (C∗t )

ASTER: Time-Series Anomaly Detection

Algorithm 2 Training Procedure of ASTER Require: – Input window and its size Wt , L – Auto-encoder Pn (qϕ , gθ ) – Pseudo-anomaly generator Pa (pψ ) – Feature extractor Φ – Classifier Ψ – Positional encodings PE Feature Extraction Ct ← Φ(Wt ) C′t ← Ct Encoding & Sampling Pt ← qϕ (C′t + PE) for l = 1 to L do (i) ϵl ∼ N (0, I) (i) (i) Z[l] ← [µl + σl ⊙ ϵl ] end for Reconstruction & Generation b t ← gθ (Z(i) ; PE) C Z ′(i) ← Z(i) e t ← pψ (Z′(i) ; PE) C e ′t ← C et C Classification s̄t ← Sigmoid(Ψ (Ct )) f′ t )) s̃¯t ← Sigmoid(Ψ (C Losses Computation LCE ← − log(1 − s̄t ) + log(s̃¯t ) b t ) + DKL (Pt ) LELBO ← M SE(Ct , C

Architectural Details

D.1

Perturbator

▷ BP–Stop Gradient

▷ (µl , σl ) ∈ Pt ▷ As: (Cross-Attention Inputs; Inputs) ▷ BP–Stop Gradient ▷ BP–Reverse Gradient

We use the Transformer architecture [59] at the core of P. While qϕ is based on the encoder part, gθ and pψ are based on the decoder part, i.e., with crossattention heads. The encoder, qϕ is divided into three parts: qϕ,0 , the actual Transformer structure; qϕ,µ , a linear layer encoding each time-step l into its distribution’s mean µl ; qϕ,σ , a linear layer encoding each time-step l into its distribution’s standard deviation σl . During training, a sample Z(i) is drawn from the distribution qϕ (Z | C) for each window. The reparametrisation trick and the auxiliary variable ϵ are used for each time-step: h iL (i) Z(i) = µl + σl ⊙ ϵl , (10) l=1

R. Hermary et al.

and, inspired by previous works in music [28] and face [73] generation, we used the sampled window as cross-attention keys and values for the decoders gθ and pψ . On top of the cross-attention values, the decoders must be given inputs to determine the query tokens. Since the objective for gθ is raw reconstruction, instead of next-token prediction or forecasting, we cannot give shifted inputs or partial inputs to the decoders, at the risk of shifting the meaning of Z. Neither can we give the Ct , since, at least for gθ , this would give a simple solution during optimisation: passing the input directly as output. Instead, we propose to add positional encoding to Ct before processing it with qϕ and, only give those same positions as input to gθ and pψ . This, in fact, allows one to maximise the amount of information flowing through Z. We use sinusoidal positional encoding [59] as formulated below, with k as the feature dimension index: 2k PEt,2k = sin t/10000 M ,

D.2

2k PEt,2k+1 = cos t/10000 M

(11)

Implementation

We implemented our method using PyTorch [48], and all experiments were conducted on Nvidia A100 GPUs. We train our model using the SGD optimiser with a fixed learning rate of 0.001 (without any scheduling), and a batch size of 64. All models are trained for 100 epochs, and the last epoch is used for inference. We select 4 as the window size. The depth of the classifier and the perturbator components is set to 2. Similar to previous methods, we use the small variant of GPT-2 [50] as the base LLM. However, unlike previous methods, we finetune the attention and projection layers using LoRA [26]. The model has a total of 180M parameters, with around 55M trainable. For a fair comparison and reproducible results, we use the default seed in the TAB benchmark for all the experiments.

Complementary Ablation

There is no consensus on the optimal window size in time-series anomaly detection, and most existing methods determine the window length empirically, often relying on heuristic tuning or dataset-specific experimentation. We conduct experiments to determine the best window size, and the results are presented in Table 4.

ASTER: Time-Series Anomaly Detection

PSM PUMP Window Size F1 AUROC AUPR F1 AUROC AUPR 4

0.512

0.697

0.501 0.458

0.839

0.254

8 16 32 64

0.456 0.450 0.461 0.475

0.644 0.639 0.656 0.681

0.387 0.399 0.387 0.429 0.419 0.433 0.487 0.441

0.774 0.777 0.785 0.828

0.185 0.189 0.191 0.224

Table 4: Ablation experiments to determine optimum window size. We observe window size of 4 gives the best performance. Increasing the window size further from 8 to 64 gives a gradual improvement with the cost of quadratic increase on the runtime due to the attention-based modules.

Model Analysis: Visualisation

We computed the average cosine distance (Equation (12)) between the genere t and the normal window Ct throughout the training. ated pseudo-anomaly C The resulting plot (Figure 4) suggests that pseudo-anomalies become increasingly entangled with normal windows, making them more difficult to discriminate. Still, our observations indicate that the classifier continues to distinguish these windows throughout training, highlighting the effectiveness of the learnt generation process.

" # L X e t,l C · C 1 t,l e t) = 1 − CosineDistance(Ct , C . e L l=1 ∥Ct,l ∥2 ∥Ct,l ∥2

(12)

In Figure 5, we present the PCA plots of the distributions of normal (N), real anomaly (A), and pseudo-anomaly (PA) data on the test set of the PSM dataset. We observe that the diversity of pseudo-anomalies is limited–at least in the projected dimensions where normal and anomalous data seem to be very sparse. This points to a promising direction for further enhancing Requirement 3 and achieving improved results. However, the pseudo-anomalies seem “closer” to the real anomalies than to the normal data, and as shown in Figure 4, the distance between the normal data and the pseudo-anomalies is decreasing. These results indicate that even a limited diversity of pseudo-anomalies assists the model in learning a meaningful classification boundary.

R. Hermary et al. Type

CosineDistance(Ct , C̃t )

N PA A

1.00 5

0.98 0.96

0.94 −5

0.92 0.90

−10

100

Epoch

Fig. 4: The cosine distance between the real and pseudo-anomaly data during training. The decreasing distance indicates that the perturbator generates increasingly challenging pseudo-anomalies throughout training.

−15 −80

−60

−40

−20

Fig. 5: PCA visualisation of normal (N), real anomaly (A), and generated pseudoanomaly (PA) data. Features are extracted before the classifier, resulting in an unclear boundary between normal and anomalous data at this stage.

Record · ID 14030 · SHA-256 f1f26dd1850c6ac4
Conceptio Open Knowledge Archive — every document is proof-bundled with source, license, and retrieval metadata.