arXiv CS open access

Denoising-Aware Inversion: Revealing Privacy Risks in Noise-Protected Text Embeddings

2026 · arxiv_cs

arXiv CS · Papers · 2026

cryptography cybersecurity privacy security

Conceptio Open Archive

Analyze & explore related research in Conceptio

Connect this record directly into your research workflow or AI agent. Explore citations across open repositories, query full text, and export structured proof bundles.

Explore in Search →Connect via MCP / API ↗

Denoising-Aware Inversion: Revealing Privacy Risks in Noise-Protected Text Embeddings Yubo Wang1 , Shujie Cui1 , James Bailey1 , Hongzhi Yin2 , Wenyu Liang1 , Min Tang1 , Shiyue Qin3 , Weiqing Wang1 1

Monash University, Melbourne, Australia The University of Queensland, Brisbane, Australia 3 Northeastern University, Shenyang, China {yubo.wang2, shujie.cui, james.a.bailey, Teresa.Wang}@monash.edu wlia0047@student.monash.edu, bupttm@163.com, h.yin1@uq.edu.au, qinsy@mail.neu.edu.cn

arXiv:2608.18610v1 [cs.LG] 19 Aug 2026

Abstract—Dense text embeddings are widely used in data mining, retrieval, and downstream machine learning systems due to their compact and semantically rich representations, but recent embedding inversion attacks have shown that they can expose substantial information about the original text, leading to serious privacy leakage risks. A common defense is to release perturbed embeddings by adding Gaussian noise, which is simple yet effective against standard inversion attacks and does not significantly degrade embedding utility for downstream tasks. However, it remains unclear whether such noise-protected embeddings are sufficiently safe against adaptive attackers that explicitly account for the perturbation process. In this paper, we study text embedding inversion in a noise-protected setting, where the attacker can observe only noisy embeddings and has no access to clean embedding targets. We first analyze why existing generative inversion methods fail under this setting and identify a “Double Noise Trap”, which fundamentally prevents standard generative inversion models from achieving high-quality reconstruction. To address this challenge, we propose DAEI, a denoising-aware embedding inversion pipeline that combines a residual denoising autoencoder with generative text inversion where the denoiser is trained in an unsupervised manner using Stein’s unbiased risk estimate to enable denoising from noisy observations alone. Extensive experiments show that DAEI achieves approximately 154% relative improvement in BLEU over the existing generative inversion baseline, while also improving token-level F1 and ROUGE-L by 32–60%. The promising inversion performance of DAEI challenges the prevailing assumption that simple Gaussian perturbation is sufficient to prevent sensitive information leakage from embedding representations. The code for reproduction is provided below: https://github.com/ChrisWang233/DAEI Index Terms—Machine Learning, Dense Text Embeddings, Embedding Inversion Attack, Privacy Leakage, Denoising Autoencoder

I. I NTRODUCTION Dense text embeddings have become a standard data representation in modern data mining and machine learning systems [1]–[3]. Across applications such as semantic search, user modeling, vector databases, and retrieval-augmented analytics, raw textual records are increasingly transformed into compact continuous vectors that are stored, searched, or used for various tasks by downstream systems [4]–[6]. This representation is attractive because it transforms raw data into lowdimensional and semantically rich vector representations, making subsequent tasks more efficient [1], [5]. Yet these benefits

create a potential privacy issue: high-quality embeddings are explicitly trained to preserve semantic and lexical information, and may therefore also preserve sensitive entities and attributes [7], [8]. Consequently, any leakage allows adversaries to apply sophisticated extraction techniques to recover the underlying sensitive data, resulting in a privacy vulnerability. Recent works have made this privacy risk concrete. Generative embedding inversion attacks reconstruct full sentences from sentence embeddings, rather than simply recovering unordered keywords [8]. Vec2Text further frames inversion as controlled generation by introducing an iterative correction framework, substantially improving reconstruction accuracy for short inputs [9]. In practice, embeddings may be exposed through compromised vector databases, honest-but-curious storage or retrieval providers, and distributed systems that transmit embeddings online across components where the adversaries can attack the embedding during transmission [10], [11]. Once an adversary can observe these embeddings, the embeddings themselves may become a text disclosure channel. One natural defense strategy is to perturb embeddings before release. Differential privacy formalizes the broader principle of adding calibrated noise to sensitive computations [12], and prior work on privacy-preserving textual analysis has shown that perturbations in embedding spaces can maintain downstream utility while reducing disclosure risks in data mining workflows [13]–[15]. The most prevalent defense mechanism is additive Gaussian noise injection [9], [16], [17]. In practice, adding zero-mean Gaussian noise with standard deviation σ = 0.01 to the final representations of the target embedder can severely degrade generative inversion performance while barely affecting downstream task utility [9]. However, as far as we know, there are no existing embedding inversion methods studying whether embeddingto-text inversion remains possible under this noise-protected setting. The closest related work is ZSInvert [18], which focuses on zero-shot inversion instead of focusing on noiseprotected setting. We consider it the closest work because it includes an experimental case study to show that the proposed approach is noise robust [18]. The possible reason is that its search-based optimization offers some robustness to noisy embeddings, but the recovered texts are often less

readable and weakly token-aligned, which is also validated in our experiments. Therefore, we include it as a noise-robust baseline, while noting that conventional generative inversion methods remain largely ineffective under the noise-protected embedding setting. This difficulty goes beyond a harder inversion task: noise creates a fundamentally different and underexplored inversion paradigm. In this setting, adversaries are deprived of clean embeddings and can observe only noisy vectors. This makes inversion substantially more challenging for two reasons: first, the absence of clean targets invalidates standard supervised denoising objectives that rely on reconstruction error against clean references; second, the inverse mapping must disentangle the underlying semantic signal from high-dimensional noise before text can be reliably recovered. Consequently, these constraints motivate the central question of our work: can adversaries recover sufficient textual information from blackbox noisy embeddings without relying on clean embeddings? We investigate this question from a security-evaluation perspective. Specifically, we study the no-clean-embedding regime and propose DAEI: Denoising-Aware Embedding Inversion, an integrated pipeline for measuring the privacy risk of noise-protected text embeddings. Building upon Vec2Text as our generative backbone for text recovery, we prepend a residual denoising autoencoder (DAE) [19] to the inversion decoder to separate semantic signals from high-dimensional noise. A conventional DAE typically relies on a Mean Squared Error (MSE) objective against clean reference embeddings, which are unavailable in our noisy-only setting [19], [20]. Under the Gaussian noise model, we therefore use Stein’s unbiased risk estimate (SURE) [21], [22] to remove this reliance on clean embeddings, yielding an unbiased risk estimate computable from noisy observations alone. We further use Monte-Carlo Hutchinson estimation [22] to approximate the required divergence term with an explicit variance-based error bound. To align denoising with text generation, we jointly fine-tune both modules. This coupled optimization guides the DAE to preserve critical semantic directions, while dynamically adapting the inverter to the refined embedding space. Furthermore, unlike methods such as Noise2Noise [23] that require multiple noisy observations of each sample which are usually unavailable in real world applications, DAEI operates with only one noisy observation per sample. In summary, our main contributions are as follows: We challenge the prevailing assumption that noise guarantees text embedding privacy. We are the first to formulate and systematically evaluate an adaptive “denoisinginversion” approach. By demonstrating that underlying textual information can still be reliably recovered from noise-protected embeddings, we provide a critical reevaluation of current representation security standards. • We systematically analyze the failure of existing generative inversion approach in reconstructing text from noisy embeddings and introduce “Double Noise Trap” (Section II-B).

•

Go through embedding model

𝒇(ෝ 𝒙)

ෝ Text hypothesis 𝒙

Chasing noisy residual

𝒚=𝒛+ 𝝐

DAE

Add noise

Clean Manifold

𝒆෤ (𝒌)

𝒛෤

Matching denoised distribution

𝒛

Fig. 1: Geometric intuition of the “Double Noise Trap”. Additive Gaussian noise pushes the target embedding off the clean text manifold, causing vanilla inversion to chase off-manifold directions. DAEI instead projects the noisy representation back toward the clean manifold before inversion.

To the best of our knowledge, we are the first to study denoising autoencoders for noisy text embedding inversion under a strict no-clean-target setting. • We propose DAEI, a novel denoising-aware inversion pipeline that couples DAE pre-training, DAE-shielded inversion, components joint fine-tuning, and DAE-shielded correction, enabling robust text recovery from noisy observations. • We validate DAEI across multiple benchmarks and embedding backbones. Compared with noisy Vec2Text, DAEI achieves approximately 154% relative improvement in BLEU and 32–60% gains in F1 and ROUGE-L, while also substantially outperforming ZSInvert. •

II. P ROBLEM F ORMULATION AND M OTIVATION A. Problem Formulation Let x ∈ X denote an input text and let fclean : X → Rd be a frozen black-box text encoder. In the standard setting, the exact latent representation is given by z = fclean (x),

(1)

We study a privacy-preserving regime where the system releases only a perturbed version of the final embedding, denoted as y: y = f (x) = z + ϵ, ϵ ∼ N (0, σ 2 Id ). (2) where f denotes a noisy embedder, and the standard deviation of noise scale σ is assumed to be known or estimable, but the clean embedding z remains strictly inaccessible to the attacker. Adversary assumption: In our threat model, we assume that the adversary has no knowledge of the embedder such as architecture and parameters. 1) Adversary can only query the released embedding interface as a black box and obtain an output embedding for a given attacker-chosen input text, thereby constructing noisy embedding–text training pairs (y, x); 2) The adversary is assumed to know that the perturbation is additive Gaussian noise, since it is a standard and widely adopted mechanism for protecting text embedding; 3) The adversary

either knows the noise scale σ, as prior work commonly adopts σ = 0.01 to achieve the best tradeoff between effectiveness and privacy [9]; or can estimate σ through repeated calibration queries on any adversary-chosen input text by observing the variation of the returned embeddings. Specifically, for n repeated observations of the same calibration text, {y (j) = z + ϵ(j) }nj=1 , an unbiased estimator of the noise variance is σ̂ 2 =

n X 1 ∥y (j) − ȳ∥22 , d(n − 1) j=1

ȳ =

1 X (j) y . n j=1

(3)

Under Gaussian noise, d(n − 1)σ̂ 2 /σ 2 follows a chi-square distribution with d(n − 1) degrees p of freedom [24], so the relative standard error of σ̂ 2 is 2/(d(n − 1)). For a 768dimensional embedder, even n = 10 calibration queries give a relative standard error of about 1.7% for σ̂ 2 and about 0.85% for σ̂. Please note that, even though an attacker can probably estimate the clean embedding for the text they input to the encoder multiple times and these clean embeddings might be helpful in reconstructing the original text that the adversaries input. But the text we want to protect is NOT the text that the adversaries input but the hidden text of the target systems. For these protected texts, the adversaries only observe the released noisy embeddings and they cannot estimate the clean embeddings for them. Given the observed noisy embedding y and black-box embedder f , the task is to reconstruct the original text x. A natural initial attempt is to directly train an inversion model on the noisy pairs (y, x). However, as observed in prior experiments by Zhuang et al. [16], this strategy remains largely insufficient. Below, we analyze theoretically why this naive approach fails. B. Vec2Text Fails due to “Double Noise Trap” Generative inversion methods such as Vec2Text [9] are designed to map a target embedding z back to text x by modeling pθ (x | z) via an inverter, and refine a hypothesis x̂ via a corrector by comparing the target embedding with the embedding of the hypothesis f (x̂). Under a noisy embedding interface, both quantities are observed with perturbations: ytar = z + ϵ1 ,

yhyp = f (x̂) = fclean (x̂) + ϵ2 ,

(4)

where ϵ1 , ϵ2 ∼ N (0, σ Id ) are independent. The observed correction residual is therefore ∆obs = ytar − yhyp (5) = z − fclean (x̂) + (ϵ1 − ϵ2 ) . | {z } | {z } semantic residual

noise residual

Thus, the correction signal contains not only the desired semantic residual, but also a residual noise term formed by subtracting two independent perturbations. Since ϵ1 − ϵ2 ∼ N (0, 2σ 2 Id ), the residual noise has doubled variance, and its typical magnitude is √ √ ∥ϵ1 − ϵ2 ∥2 ≈ 2σ d. (6) For a 768-dimensional embedding with σ = 0.01, this residual noise magnitude is approximately √ 0.39, which is larger than the single-noise displacement σ d ≈ 0.28.

As shown in Fig. 1, clean embeddings reside on a textinduced manifold, and additive noise pushes y off this manifold. Consequently, the corrector attempting to match the hypothetical embedding f (x̂) against the noisy target y is fundamentally misguided. This geometric mismatch exposes the standard inversion process to what we term the “Double Noise Trap”. The optimization objective expands as follows: ∥ytar − yhyp ∥22 = ∥z − fclean (x̂)∥22 + 2 ⟨z − fclean (x̂), ϵ1 − ϵ2 ⟩ + ∥ϵ1 − ϵ2 ∥22 . (7) The first term is the desired semantic matching objective, while the remaining terms are caused by noise. As refinement improves and the semantic residual ∥z − fclean (x̂)∥22 becomes smaller, the stochastic cross term and the doubled noise baseline can dominate the correction signal. The corrector may therefore reward hypotheses whose embeddings align with residual noise directions rather than true semantic directions. This motivates our decoupled design, which first estimates a denoised clean-manifold representation and then performs generation and correction through denoised embeddings. C. Infeasible Supervised Denoising The strict no-clean-embedding constraint fundamentally changes the nature of the denoising process. The standard supervised denoising objective, min E ∥hϕ (y) − z∥22 , (8) ϕ

where hϕ denotes a denoising neural network parameterized by ϕ, is inherently unavailable because the clean target z is unavailable. Consequently, these analyzes underscore the critical need for a novel inversion pipeline: one capable of reconstructing the original text relying strictly on the noisy observations. III. METHOD We propose DAEI, a novel pipeline for robust noisy text embedding inversion, with its overall architecture and data flow illustrated in Fig. 2. Our system is built upon four components: a black-box noisy embedder f (·), a denoising autoencoder (DAE) hϕ , a generative inverter θ, and an iterative corrector. Rather than a naive end-to-end approach, the DAEI pipeline comprises four distinct stages: (1) SURE-based unsupervised DAE pre-training, (2) DAE-shielded inverter training, (3) joint SURE-CE fine-tuning, and (4) DAE-shielded corrector training. Notably, the architectures for Stages 2 and 4 remain virtually identical to vanilla Vec2Text, with the distinction that the input embeddings are denoised by our DAE in advance. A. SURE-based Unsupervised DAE Stage 1 aims to learn a denoising module hϕ that constructs a denoised embedding z̃ from the noisy observation y: z̃ = hϕ (y, σ).

(9)

Noisy Embedder

3 Inverter

2 DAE

Hidden text

Noisy embedding

𝒛෤ 𝒉𝝓

Stage 1 Stage 2 Stage 3 Stage 4

(token-level)

𝒚 𝒇 ⋅

Corrector

𝓛𝑪𝑬

𝓛𝑺𝑼𝑹𝑬 + 𝓛𝑪𝑬 𝒙

𝜽

Denoised embedding Corrector

ෝ(𝒌) 𝒙

𝒇 ෝ 𝒙(𝒌)

𝒆෤ (𝒌)

Hypothetical text

Noisy hypothetical embedding

Denoised hypothetical embedding

ෝ 𝒙(𝒌+𝟏) Next hypothetical text (Output)

Fig. 2: The overall architecture and data flow of our proposed DAEI pipeline. The system consists of four functional components (grey vertical panels) optimized across four stages. (1) Upon obtaining a noisy embedding from the hidden text via the Noisy Embedder, we first apply the DAE (indicated by the shield icons) to reconstruct a denoised embedding z̃. (2) It is then processed by the Inverter to generate an initial hypothetical text x̂(k) . (3) During training, the token-level cross-entropy loss (LCE ) backpropagates directly through the active DAE to achieve joint fine-tuning (grey dashed lines). (4) Finally, the generated text enters an iterative correction loop: it is re-encoded by the embedder and passed through the DAE to yield a denoised hypothesis embedding ẽ(k) . Both ẽ(k) and the original denoised target z̃ are then fed into the Corrector to predict the next hypothetical text, repeating until the loop terminates to produce the final output.

The oracle denoising risk Eq. (8) cannot be directly optimized because the clean embedding z is unobserved. To check if the dependency on z can be removed, we first expand Eq. (8): ∥hϕ (y, σ) − z∥22 = ∥hϕ (y, σ)∥22 − 2z ⊤ hϕ (y, σ) + ∥z∥22 . (10) The last term ∥z∥22 is independent of ϕ, while the cross term z ⊤ hϕ (y, σ) is inaccessible because z is hidden. However, under the Gaussian noise model y = z + ϵ, where ϵ ∼ N (0, σ 2 Id ), Stein’s lemma gives [21] Eϵ ϵ⊤ hϕ (y, σ) = σ 2 Eϵ [divy hϕ (y, σ)] . (11) Since z = y − ϵ, the inaccessible cross term can be replaced in expectation by quantities depending only on y, the noise variance σ 2 , and the divergence of the denoiser. This leads to SURE [21]: LSURE (y; ϕ) = ∥hϕ (y, σ) − y∥22 + 2σ 2 divy hϕ (y, σ) − dσ 2 . (12) Here, divy hϕ (y, σ) = Tr(∂hϕ /∂y) measures the local sensitivity of the denoiser with respect to its noisy input. Therefore, although z is never observed, minimizing Eq. (12) is equivalent in expectation to minimizing the oracle denoising risk in Eq. (8). We further parameterize the denoising module as a residual correction around the observed noisy embedding, rather than directly predicting z̃ from scratch: hϕ (y, σ) = y + rϕ (y, σ),

(13)

where rϕ (y, σ) estimates the displacement needed to suppress the noise component and move the observation back toward the clean embedding manifold.

Substituting this parameterization into the divergence term gives divy hϕ (y, σ) = d + divy rϕ (y, σ). After removing constants independent of ϕ, the training objective becomes LSURE (y; ϕ) ≡ ∥rϕ (y, σ)∥22 + 2σ 2 divy rϕ (y, σ).

(14)

We approximate the divergence term with Monte-Carlo Hutchinson estimation [22]: K

1 X ⊤ divy rϕ (y, σ) ≈ vk Jrϕ (y, σ)vk , K

vk ∈ {−1, +1}d ,

k=1

(15) where Jrϕ is the Jacobian of the residual network with c K denote the Monterespect to the noisy embedding. Let div Carlo estimator in Eq. (15). For Rademacher probes satisfying E[vk vk⊤ ] = I, this estimator is unbiased and its variance is bounded by h i c K = divy rϕ (y, σ), Ev div (16) h i c K ≤ 2 ∥Jr (y, σ)∥2 . Var div F ϕ K Thus, the approximation is theoretically well grounded: it is unbiased, and its variance decays at rate O(1/K). In practice, we use Rademacher probes and spectral normalization in the residual MLP to stabilize the trace estimate. B. DAE-Shielded Inverter After we obtain denoised representation z̃, the inverter θ is subsequently trained on the pairs (z̃, x) rather than the raw noisy embeddings. It prevents the generative model from overburdening its capacity with high-dimensional noisefiltering and avoids overfitting to non-semantic perturbations.

The training objective is the standard token-level crossentropy loss: LCE (θ; ϕ) = −

T X

log pθ (xt | x<t , z̃)

(17)

t=1

where T denotes the sequence length, xt is the target token at step t, and x<t represents the preceding autoregressive context. C. Joint SURE-CE Fine-tuning Following the independent training of the DAE and the inverter, stage 3 unifies the architecture. To explicitly align the unsupervised denoising process with the downstream generative task, we jointly fine-tune the DAE and the inverter: LSURE-CE (ϕ, θ) = LSURE (y; ϕ) + λCE (t) LCE (θ; ϕ) . (18) Here, the weighting coefficient λCE (t) is warmed up during the early training steps. The key difference from Stage 2 is that the cross-entropy gradient is allowed to pass through hϕ (y, σ). Consequently, the DAE receives two complementary supervisory signals: the SURE objective ensures that the output remains strictly aligned with the denoising manifold, while the CE (cross-entropy) loss provides semantic feedback, preserves the latent directions for decoding. Concurrently, the inverter is continuously trained on the embeddings denoised dynamically by the fine-tuning DAE to adapt to the optimized embedding distribution. However, the optimization directions of these two objectives may conflict. We therefore employ distinct learning rates for the DAE and the inverter, and apply gradient surgery PCGrad [25] exclusively to the DAE parameters ϕ. Let gSURE ≜ ∇ϕ LSURE and gCE ≜ ∇ϕ LCE denote the gradients with respect to the DAE parameters. If these two gradients exhibit a negative inner product (i.e., ⟨gCE , gSURE ⟩ < 0), they are conflicting. In such cases, we project the CE gradient onto the normal plane of the SURE gradient before merging the updates: ⟨gCE , gSURE ⟩ gSURE , ∥gSURE ∥22 ′ ϕ ← ϕ − ηDAE (gSURE + gCE ).

′ gCE = gCE −

(19)

This orthogonal projection guarantees that the text feedback ′ (gCE ) shapes the final representation without unlearning the fundamental denoising capabilities. D. DAE-Shielded Corrector Conventionally, once an inverter is trained to generate an initial text hypothesis, a corrector iteratively refines this hypothesis by comparing its embedding against the target embedding. However, when the available target is noisy, directly performing this comparison inevitably triggers the “Double Noise Trap” described in Section II-B. To mitigate this, we introduce a DAE Shield to both sides of the correction interface. Specifically, instead of comparing raw representations, we pass both the noisy target y and the

embedding of the current hypothesis x̂(k) through our Stage 3 denoiser hϕ⋆ : z̃ = hϕ⋆ (y, σ),

ẽ(k) = hϕ⋆ (f (x̂(k) ), σ),

(20)

where x̂(k) is the generated text at correction step k, and ϕ⋆ denotes the fully fine-tuned DAE parameters. The corrector then conditions exclusively on the denoised target z̃, the denoised hypothesis ẽ(k) , and their residual difference. By completely shielding the correction interface, we constrain the iterative refinement strictly within the learned clean manifold, ensuring the model aligns with semantic text directions rather than chasing irreducible noise residuals. IV. E XPERIMENTAL S ETUP Our study aims to determine the extent to which noiseprotected embeddings from black-box encoders remain vulnerable to advanced inversion attack paradigms. Specifically, we investigate the following research questions: RQ1: How does the proposed DAEI pipeline compare against vanilla Vec2Text and existing noise-robust baselines in recovering target text from noisy embeddings? RQ2: Can the proposed DAEI pipeline be utilized in different embedding model backbones? RQ3: Can DAEI maintain strong inversion performance on out-of-domain test sets? RQ4: Can the DAE component effectively reduce noise? RQ5: Does the joint finetuning module improve reconstruction quality? RQ6: To what extent can DAEI maintain its inversion performance when the noise σ varies? A. Dataset 1) Training set: All main experiments use the same training mixture of: • Natural Questions (NQ) is a question answering benchmark built from real Google search queries [26]. • MS MARCO is a large-scale machine reading retrieval dataset constructed from real Bing queries, web passages, and human-generated answers [27]. • Yahoo Answers is a community question-answering domain covering broad everyday topics. We capped the combined mixture at 7M text examples as raw text sources to construct the inversion datasets. While the underlying text corpora remain constant, the exact inputtarget configurations vary across our four training stages to accommodate the specific objectives of each module: • Stage 1 (DAE Pre-training): To train the residual DAE, the system requires noisy embeddings only. • Stage 2 (Base Inverter Training): To train the initial text decoding module, we utilize (denoised embedding, text) pairs. These denoised embeddings are computed by the Stage 1 DAE. • Stage 3 (Joint Fine-tuning): For end-to-end joint optimization, the dataset consists of (noisy embedding, text) pairs. Here, the noisy embeddings are dynamically denoised on-the-fly by the actively tuning DAE during the forward pass.

•

Stage 4 (Corrector Training): To train the iterative corrector, we rely on expanded input tuples consisting of (denoised embedding, hypothetical text, denoised hypothetical embedding), paired with the original text as label. Crucially, these denoised embeddings are computed by the fully tuned Stage 3 DAE, ensuring the corrector operates strictly within the optimal denoised distribution.

2) Evaluation sets: For evaluation, we consider both indomain and out-of-domain settings. The in-domain test set is the held-out splits from the same NQ, MS MARCO, and Yahoo Answers mixture, while the out-of-domain test sets are drawn from five unseen corpora: AG News, Anthropic Toxic Prompts, Python Code Alpaca, Climate-FEVER [28], and MedMCQA [29]. These out-of-domain datasets allow us to evaluate whether the inversion methods generalize beyond the training distribution. The data sources for all training and evaluation sets are provided in our repository: https://anonymous.4open.science/r/DAEI-1F1C/ B. Target Embedding Models To rigorously evaluate the generalizability of our DAEI pipeline across different latent space distributions, we select two widely adopted, state-of-the-art dense text embedding models as our black-box targets: GTR-base [30]: The GTR (Generalizable T5-based dense Retrievers) model leverages a bi-encoder architecture initialized from the T5 [31] checkpoints. It is specifically optimized for robust zero-shot retrieval tasks, producing highly structured semantic representations. • GTE-base [32]: The General Text Embeddings (GTE) model trained on large-scale contrastive data represents a highly optimized encoder-only architecture. •

We deliberately select GTR and GTE since they represent two fundamentally distinct backbone architectures (T5-based vs. BERT-based) and training paradigms, thereby demonstrate the generalizability of our method. C. Baselines and Our Models Given the scarcity of existing work on noisy text embedding inversion, we use ZSI NVERT [18] (an adversarial decoding approach) as our primary external baseline even though it is not based on the mainstream encoder-decoder architecture because it is the only existing work that we know has shown its effectiveness in the noisy text embedding inversion scenario. Our core focus remains on a rigorous comparison between various configurations of the encoder-decoder models and our proposed approach. To comprehensively assess inversion performance, we categorize the baselines and our models into three distinct groups: 1) Clean Upper Bound: C LEAN - TO -C LEAN Baseline evaluates the vanilla Vec2Text [9] model on clean embeddings. It represents the state-of-the-art text recovery performance under ideal, noiseless conditions. This serves as the theoretical ceiling that our noisy inversion pipelines aim to approach.

TABLE I: Main experimental systems. ZSI NVERT base is training-free; its listed training embedding type refers only to corrector training. System C LEAN - TO -C LEAN Baseline ZSI NVERT C LEAN - TO -N OISY Baseline N OISY I NVERTER Baseline DAE- ONLY (ours) DAEI (ours)

Train embedding

Test embedding

Clean Noisy Clean Noisy Denoised by stage1 DAE Denoised by stage3 DAE

Clean Noisy Noisy Noisy Denoised by stage1 DAE Denoised by stage3 DAE

2) Generative Inversion under Noise: This group compares the standard autoregressive inversion models against our proposed pipelines when facing perturbed embeddings: C LEAN - TO -N OISY Baseline: Tests the severe distribution shift faced by a vanilla Vec2Text model that is trained solely on clean embeddings but evaluated on noisy ones. • N OISY I NVERTER Baseline: Trains a vanilla Vec2Text directly on noisy embeddings. This provides a much stronger baseline that adapts to the corrupted embedding distribution but lacks denoising mechanism. • DAE- ONLY inverter (Ours): Our proposed pipeline without joint fine-tuning. A DAE first denoises the embedding, then an inverter is trained on the denoised cache. • DAEI (Ours): Our complete pipeline that jointly finetunes both the DAE and the inverter. •

3) Adversarial-Decoding-Based Baseline: To step outside the generative paradigm, we introduce ZSI NVERT as an external baseline. This method frames inversion as adversarial decoding, iteratively optimizing discrete tokens to minimize embedding distance. Table I summarizes the above systems evaluated in our main experiments. D. Metrics. For evaluating the inverters, we report sentence-level BLEU [33], token-level F1 [34], ROUGE-L [35], and the cosine similarity between the embeddings of the reconstructed and reference texts. Specifically, BLEU assesses generation precision and local fluency by measuring the n-gram overlap between the reconstructed and reference texts; token-level F1 assesses keyword recovery by measuring unigram overlap independently of sequence order; ROUGE-L evaluates structural similarity and information recall based on the longest common subsequence; and embedding cosine similarity measures semantic closeness in the target embedding space. For evaluating the DAE, we also report the MSE and embedding cosine similarity between the DAE’s denoised outputs and the clean embeddings. We emphasize that these clean embeddings are strictly isolated for offline evaluation purposes only; they are never observed by the model during any stage of training, nor are they used to compute any optimization loss.

E. Implementation Setting. Our method assumes the noise scale σ is prior knowledge (see Section II-A). Unless otherwise specified in the ablation studies, we apply zero-mean Gaussian noise with a standard deviation of σ = 0.01 [9], [16]. The DAE module is pretrained with a learning rate of 1×10−3 . We configure a hidden dimension of 1024 and a depth of 3, spectral normalization, and five Monte-Carlo probes. All inverter variants use the Vec2Text architecture with a T5 [31] backbone and are optimized with a learning rate of 1 × 10−3 . Following the original Vec2Text setting [9], both input and target texts are truncated to a maximum length of 128 tokens during training, while evaluation is conducted with a maximum length of 32 tokens. During the joint fine-tuning stage, the DAE learning rate is reduced to 2 × 10−4 , and the cross-entropy loss weight is linearly warmed up over 10,000 steps. All corrector variants use the Vec2Text architecture with a learning rate of 5×10−4 . We enforce an early-stop threshold of 1×10−3 : refinement terminates once the cosine similarity gain between consecutive steps falls below this value, preventing the corrector from overfitting to residual noise. V. R ESULTS A. Overall Performance (RQ1 & RQ2) DAEI achieves the best noisy inversion performance. Tables II and III report the main inversion results on GTR-base and GTE-base, respectively. On GTR-base, DAEI substantially improves over the N OISY I NVERTER baselines across all metrics. In particular, DAEI increases BLEU from 0.1916 to 0.4875, yielding an absolute gain of nearly 30 percentage points and a 154.4% relative improvement over the N OISY I NVERTER baseline. It also improves token-level F1 from 0.5814 to 0.7717 and ROUGE-L from 0.5660 to 0.7788, indicating that DAEI can recover richer textual information from noisy embeddings alone. Moreover, the embedding cosine similarity increases from 0.8349 to 0.9254, suggesting that the DAE-shielded correction process more effectively aligns the hypothesis embedding with the target embedding after removing a substantial portion of the perturbation noise. Noisy training alone provides limited gains. The comparison between the C LEAN - TO -N OISY baseline and the N OISY I NVERTER baseline further illustrates the limitation of directly training Vec2Text on noisy embedding. On GTR-base, noisy training improves text reconstruction metrics by only 5–8 points over the C LEAN - TO -N OISY baseline, indicating limited adaptation to noisy inputs. However, due to the Double Noise Trap, the correction stage compares a noisy hypothesis embedding with a separately perturbed target, causing the residual signal to be contaminated by doubled noise. As a result, this gain remains far lower than that achieved by DAEI. DAEI approaches the clean upper bound and outperforms ZSI NVERT in text reconstruction. Relative to the clean upper bound, DAEI recovers 78.0%/89.2% of BLEU/F1 on GTR-base and 71.7%/83.2% on GTE-base. These results

suggest that noise-protected embeddings still retain considerable recoverable textual information when the adversary adopts a denoising-aware inversion pipeline. However, ZSI NVERT performs poorly on text-overlap metrics for both embedders, achieving only around 0.01 BLEU and 0.08–0.15 on F1 and ROUGE-L, despite obtaining a relatively high embedding cosine similarity of 0.85–0.96. This gap indicates that adversarial decoding matches the embedding space but fails to reliably recover token-aligned text, whereas DAEI better balances semantic matching and fluent reconstruction. DAEI generalizes across embedding backbones. DAEI shows similar gains on GTE-base, answering RQ2. Although the overall scores on GTE-base are lower than GTR-base, the clean upper bound is also lower, suggesting that the underlying embedding space is intrinsically more difficult for Vec2Text framework to invert. One possible reason is that our inverter uses a T5-base backbone, which is more closely aligned with the T5-based GTR encoder than with the BERTstyle GTE encoder [30], [32]; meanwhile, GTE is trained with multi-stage contrastive learning for general-purpose semantic retrieval [32], which may emphasize semantic invariance while preserving less information needed for token-level reconstruction. Even under this harder setting, DAEI improves over the N OISY I NVERTER baseline by approximately 19–23 points on text reconstruction metrics, while also increasing cosine similarity by 10.36 points. These consistent improvements over two architecturally different encoders demonstrate that DAEI is generalized across different noisy embedding interfaces. B. Out-of-Domain Generalization (RQ3) Table IV evaluates whether DAEI can generalize to Out-OfDomain (OOD) test sets. Across all OOD datasets, DAEI consistently outperforms the N OISY I NVERTER and ZSI NVERT on BLEU, F1, and ROUGE-L, showing that DAEI remains effective when the target texts come from unseen domains. The performance varies across datasets. DAEI performs especially well on Anthropic Toxic Prompts, Python Code, and MedMCQA, while AG News obtains lower scores for both the clean upper bound and DAEI. One likely reason is that AG News examples are longer on average, with about 61 tokens compared with 22–30 tokens for the other datasets. Although evaluation outputs are truncated to 32 tokens, longer source texts still contain more information to recover and are more likely to suffer from truncation and partial reconstruction. C. Components Effectiveness (RQ4 & RQ5) 1) DAE Effectiveness: DAE reduces noise-induced embedding distortion. Table V demonstrates DAE can effectively reduce the perturbation noise without using clean embeddings during training. The Stage 1 DAE reduces MSE to the clean embeddings by 29.2% on GTR-base and 23.8% on GTE-base, while improving cosine similarity from 0.9179 to 0.9322 and from 0.9662 to 0.9727, respectively. Since the injected noise is relatively small, the raw noisy embeddings are already close to the clean embeddings in cosine similarity, making the cosine gains modest. Nevertheless, the consistent

TABLE II: Main results for in-domain evaluation with GTR-base. Gray rows indicate reference settings that are not available under the current threat model. Improvements in parentheses are absolute gains over the N OISY I NVERTER Baseline. Best feasible results excluding the clean upper bound are in bold. Category

Model

BLEU

ROUGE-L

Cosine Similarity

Clean Upper Bound

C LEAN - TO -C LEAN Baseline

0.6252

0.8656

0.8704

0.9500

Adversarial Decoding

ZSI NVERT

0.013

0.1503

0.1117

0.8568

Generative Inversion

C LEAN - TO -N OISY Baseline N OISY I NVERTER Baseline DAE- ONLY (Ours) DAEI (Ours)

0.1348 0.1916 0.3616 0.4875 (+0.2959)

0.5005 0.5814 0.7206 0.7717 (+0.1903)

0.5012 0.5660 0.7200 0.7788 (+0.2128)

0.7785 0.8349 0.9055 0.9254 (+0.0905)

TABLE III: Main results for in-domain evaluation with GTE-base. Formatting follows Table II. Category

Model

BLEU

ROUGE-L

Cosine Similarity

Clean Upper Bound

C LEAN - TO -C LEAN Baseline

0.4391

0.7622

0.7570

0.9211

Adversarial Decoding

ZSI NVERT

0.0104

0.1189

0.0894

0.9641

Generative Inversion

C LEAN - TO -N OISY Baseline N OISY I NVERTER Baseline DAE- ONLY (Ours) DAEI (Ours)

0.0846 0.1249 0.2688 0.3152 (+0.1903)

0.3222 0.3960 0.5854 0.6344 (+0.2384)

0.3271 0.4044 0.5722 0.6324 (+0.2280)

0.7081 0.7520 0.8141 0.8556 (+0.1036)

TABLE IV: Out-of-domain evaluation results for DAEI. Upper Bound denotes C LEAN - TO -C LEAN Baseline, and Baseline denotes the N OISY I NVERTER Baseline. OOD Dataset AG News

Anthropic Toxic Prompts

Python Code

Climate fever

Medmcqa

Method

BLEU

ROUGE-L