The Fragility of Trigger-Tag Mechanisms for Misuse Detection in Open-Weight LLMs

The Fragility of Trigger-Tag Mechanisms for Misuse Detection in Open-Weight LLMs Toluwani Aremu1*

arXiv:2610.03124v1 [cs.CR] 2 Oct 2026

1

MBZUAI, UAE

2

Manit Baser2*

Mohan Gurusamy2

National University of Singapore

3

Abstract—Open-weight language models can be downloaded, modified, and deployed beyond their developers’ control, limiting the effectiveness of centrally enforced safeguards. Recent work has therefore proposed trigger-tag mechanisms that produce a detectable signal when a model is used under a target condition, such as generating phishing contents. Although these mechanisms borrow from established techniques, their use for conditional misuse detection in open-weight LLMs is relatively new. Therefore, existing research works have not systematically studied the robustness of trigger-tag mechanisms under adversarial attacks. To close this gap, (i) we formalize trigger-tags and distinguish tokenlevel trigger-tags, which introduce watermark-inspired signals during decoding, from weight-level trigger-tags, which learn backdoor-inspired associations between target conditions and detectable model behavior. Furthermore, (ii) we introduce U N T A G , a unified attack framework that organizes their mechanismspecific attack surfaces into a common taxonomy. We evaluate representative token-level and weight-level trigger-tags using phishing as a case study. We find that while trigger-tags may provide useful evidence in controlled settings, our attacks render the existing trigger-tag mechanisms to be entirely ineffective. Consequently, we argue that these mechanisms should not be treated as robust misuse detectors when attackers can transform outputs or modify open weights. Index Terms—LLM, Trigger-Tags, Watermarking, Backdoor.

I. I NTRODUCTION & BACKGROUND Large language models (LLMs) can generate persuasive, personalized text at scale, lowering the cost of misuse such as phishing and social engineering [1]–[5]. Centralized, commercial LLM services mitigate this risk through safety finetuning, refusal mechanisms, access controls, and deploymentside monitoring [6]–[8]. Open-weight models, however, have a different deployment workflow and control: third parties can download, fine-tune, and serve the model without preserving the original safeguards [9]–[14]. Once the weights are released, the original developer can no longer enforce a particular decoding procedure, monitor requests, or prevent downstream modification [11], [15]–[19]. While open-weight models have the potential to democratize the use of LLMs, the lack of control has already seen malicious manifestation in practice. Lin et al. identified 212 realworld malicious LLM services advertised or deployed through underground marketplaces and public hosting platforms [20]. Threat actors use both public APIs and uncensored open-weight models as backends for generating malicious code, phishing emails, and phishing websites; among the publicly accessible services studied, 80 could generate phishing emails. Therefore, ∗ denotes equal contribution.

Nils Lukas1

Dinil Mon Divakaran3

A*STAR Institute of Advanced Intelligence and Computing

(a) Phishing email generated from a tagged model subject: mailbox re-verification

Your mailbox has exceeded its quota and will be locked in 24 hours. Confirm your credentials at the secure portal below.

↓ fine-tune

prune

paraphrase

translate

sanitize

(b) Attacker manipulates model weights/output subject: your account storage is full

Storage on your account is full, so sign-in will be suspended tomorrow. Verify your details here and nothing else changes.

Fig. 1: The false promise of trigger-tag mechanisms for misuse detection in open-weight LLMs.

capable models can be repackaged into services over which the original developer has no operational control [21]. The developer’s loss of deployment control motivates a relatively new defense area for security of open-weight LLMs. Trigger-tags are non-blocking mechanisms that embed a detectable signal when a model is used under a target condition, such as generating phishing content [22], [23]. The trigger is the target condition, while the tag is the signal produced when that condition is satisfied. Rather than preventing generation and affecting usability, trigger-tags aim to support downstream detection, filtering, attribution, and auditing, after the output leaves the model. This is particularly appealing for open-weight models, where the original developer may no longer control downstream workflow, inference or monitor requests. We distinguish two principal trigger-tag designs (see §II for details). Token-level trigger-tags, inspired by text watermarking, introduce a detectable statistical signal through token selection during decoding [24]. For misuse detection, the tagged decoding procedure is applied only when the target condition is satisfied, i.e., determined by a monitor. Weight-level trigger-tags, further inspired by defensive uses of backdoor learning, train an association between the target condition and a detectable model behavior into the model parameters [8], [22], [23], [25]. Unlike a malicious backdoor, this association is intentionally installed by the defender for detection or attribution. These

terms describe where the tagging behavior is introduced: both Results. Our results reveal three mechanism-specific failure classes aim to ultimately be detected from generated outputs. modes. First, deterministic cleanup removes explicit tags While trigger-tag mechanisms draw from established wa- entirely, whereas implicit and statistical tags survive surface termarking and backdoor techniques, their use as conditional edits but weaken under semantic rewriting; in the evaluated misuse detectors for open-weight LLMs is relatively new. Tra- output-side attacks, implicit weight-level tags fall to 0% ditional watermarking research studies provenance or machine- detection under semantic rewriting (Tables V and IV). Second, generated text detection, while backdoor and model-marking an independently keyed overwrite reduces the defender’s tokenresearch studies ownership verification or persistent model level detection to at most 4% and replaces it with an attackerbehavior. In contrast, a trigger-tag defense is a composite mech- chosen tag in every output (Table VI). Third, open-weight anism encompassing a target-condition decision, conditional access enables model-side removal, reducing detection to as low tag generation, downstream detection, and, for open-weight as 0%, without requiring detector queries (Table VII). These attacks largely preserve the phishing content, with rewrites models, attacker-controlled inference and model reuse. Prior work has not systematically mapped the attack sur- retaining high cosine similarity. Our findings show that triggerface of this composite mechanism or compared token-level tags are not a reliable primary defense against misuse when the and weight-level trigger-tags under a common misuse threat model developer does not control the downstream workflow. model. Addressing this gap is critical as recent systems An attacker can rewrite outputs, control decoding, or modify have transitioned trigger-tags from a theoretical concept to a open weights to suppress the tagging signal while preserving concrete misuse defense. Paladin [22], for example, introduces the underlying misuse intent, making trigger-tag detection conditionally generated markers for detecting phishing content vulnerable to adversarial circumvention (§VI). from open-weight LLMs and reports strong detection on clean II. T RIGGER -TAGS outputs. Another work [23] embeds detectable behaviors into In this section, we formalize the Trigger-Tag mechanism. model weights for subsequent identification or verification. Figure 2 provides an overview of token-level and weight-level While these results demonstrate that trigger-tags are relitrigger-tag mechanisms. ably detectable under their intended generation procedures, a Generative model. Following the notation of Zhao et al. [27], fundamental question rises: let M be a generative model and let M (π) → x denote Are trigger-tags robust to adversarial attack? sampling an output x from M given prompt π. Text watermarking. A text-watermarking scheme consists of Specifically, does the detection remain reliable once an adversary controls the output channel, decoding procedure, or released model weights, all of which represent keyed generation and detection procedures. A key-generation realistic threat vectors today? algorithm produces generation and detection keys, Contributions. Addressing the above question, we make the following contributions: • We formalize trigger-tags as conditionally activated detec-

(gk, dk) ← KeyGen(1λ ). Given a prompt π, watermarked generation modifies token selection during decoding:

tion mechanisms and distinguish token-level mechanisms, WatermarkM gk (π) → xwm , which introduce tags during decoding, from weight-level mechanisms, that learn tagging behavior into model while detection tests whether an observed output contains the parameters (§II). We further formalize their attack surface corresponding keyed signal: under a common misuse threat model, characterizing an Detectdk (x) → {true, false}. adversary that can manipulate generated outputs, control decoding, or modify open model weights while preserving A useful watermark should detect xwm with high probability, the underlying misuse intent (§III). rarely flag outputs sampled from M , and preserve the utility • We introduce U N T A G , a unified framework for end-to-end of the underlying generation [24], [27]. Standard text waterevaluation of defensive trigger-tag mechanisms. U N T A G marking marks model-generated text irrespective of its intent; systematically connects mechanism-specific design astrigger-tags instead apply this signal conditionally. sumptions to the attack vectors they enable, and adapts attacks from watermark removal, generated-text detector Backdoor learning. A backdoor associates a trigger with a evasion, watermark collision, and model-modification target model behavior. Let τ be a trigger transformation; and research under the common threat model (§IV). Dτ = {(τ (πi ), yi⋆ )}ni=1 • We evaluate representative token-level (i.e., SynthID [26], ⋆ KGW [24]) and weight-level mechanisms (i.e., Paladin contain triggered examples paired with the target behavior yi . [22], LLM-Mark [23]) using phishing as a high-risk case Backdoor training produces
study (§V). Throughout, we distinguish clean detection Mbd ← Train M, Dclean ∪ Dτ , from robust misuse detection: target behavior that persists after an attack must remain detectable, without incorrectly such that Mbd (τ (π)) exhibits the target behavior while flagging benign outputs. Mbd (π) ≈ M (π) on non-triggered inputs. Here, ≈ denotes

Token-level trigger-tag Tagged Decoding (in positive conditions)

Generate a convincing phishing mail… User Prompt

Trigger Monitor

The

login

Base Model

page

…

reset

link

Tagged output

Detector

(weights unchanged)

Weight-level trigger-tag Generate a convincing phishing mail… User Prompt

Learned association Target condition ↓ Tagged behavior

Instrumented Model

The

login

page

…

Tagged output

reset

link

Detector

Fig. 2: Overview of trigger-tags. Token-level mechanisms activate keyed decoding when the target condition is detected, while weight-level mechanisms learn the tagging behavior into the model. The generation key denotes mechanism-specific tagging information: it may parameterize a statistical token-selection rule or specify markers, token sets, or messages to be learned during training. Activation produces a tagged output that a downstream detector verifies using the corresponding detection key.

similar generation behavior on the clean distribution. Although association is that Detectdk (x) = true when T (π, x) = 1, and backdoors are conventionally studied as attacks, defensive Detectdk (x) = false otherwise. marking mechanisms deliberately learn such an association so Token-level trigger-tags. Token-level trigger-tags build on text that a target condition causes the model to emit a detectable watermarking techniques that introduce a detectable statistical signal through token selection during decoding. To instantiate signal [22], [23], [25]. Target condition. Let T (π, x) ∈ {0, 1} denote whether an in- such a mechanism for misuse detection, we conditionally teraction satisfies a target condition. The condition may depend activate the underlying watermarking procedure based on on the prompt, the output, or both; this allows a benign-looking a target-condition monitor. When the monitor identifies the prompt to produce a target output. The target condition may target condition, tagged decoding is used, otherwise, generation represent phishing, fraud, malware generation, toxic content, proceeds with ordinary decoding, as shown below: ( or another policy-defined form of misuse. Our evaluation later WatermarkM M,tok gk (π), b = 1, instantiates T using phishing and social-engineering content Taggk (π; b) = M (π), b = 0. (refer to §V-B). A trigger-tagging mechanism may not observe the true value of T (π, x) before generation. We therefore use The signal is introduced through token selection without b ∈ {0, 1} for the mechanism’s trigger decision. This decision modifying the base model parameters. may be produced by an input monitor, an output monitor, Weight-level trigger-tags. Weight-level trigger-tags use or behavior learned by the model itself. By decoupling b backdoor-inspired training to learn the association between from T (π, x), we distinguish the mechanism’s activation from the target condition and tagging behavior: whether the interaction actually satisfies the (ground-truth) Mgk ← Embed(M, gk, DT ), TagM,wt (π) ≡ Mgk (π). gk target condition. Text watermarking determines how a signal can be introduced during decoding, whereas backdoor learning Unlike a malicious backdoor, the association is deliberately determines how a conditional behavior can be learned into installed by the defender for detection or attribution. “Weightmodel parameters. Trigger-tags combine conditional activation level” identifies where the tagging behavior is introduced; the with a detectable output signal. resulting signal may still be detected from generated text. Trigger-tag scheme. A trigger-tag scheme consists of a tagged Explicit and implicit designs. Trigger visibility and tag generation procedure and a detector. Given generation key gk, visibility are separate dimensions. An explicit trigger relies on the generator produces a predefined lexical cue, whereas an implicit trigger activates from the semantic target condition without requiring an exact M Taggk (π; b) → x, phrase. An explicit tag can be detected through direct matching, where b = 1 activates the tagging behavior. Given detection such as a character-level marker. An implicit tag instead requires a statistical or learned detector. key dk, the detector returns Clean detection. Let the probability be taken over the interaction distribution and generation randomness. A true Detectdk (x) → {true, false}. positive occurs when the detector identifies a tag in an The trigger is the condition that activates tagging, while the tag interaction satisfying the target condition. For example, a is the detectable signal produced after activation. The intended phishing generation correctly flagged by the detector is a true

1. Provider

2. Attacker

3. Defender

•

Instruments model with trigger-tags

•

Downloads and modifies model

•

Analyzes generated output

•

Releases open-weight model

•

Prompts for misuse content

•

Checks for trigger-tag signal

•

Publishes intended use and detector

•

Applies output or model attacks

•

Flags if signal is detected

Token-level Instrumented LLM

Weight-level

"Generate a convincing phishing email…" Public release

Modified model weights

Generated output

Watermark detector

No signal (delivered) Signal (flagged)

Fig. 3: Threat model and system workflow for trigger-tag mechanisms in open-weight LLMs. The provider instruments an open-weight LLM with either token-level or weight-level trigger-tags and releases the model publicly. The attacker’s goal is to obtain the released model for misuse while bypassing detection by modifying either the model’s output or weights. The defender’s goal is to correctly flag a genuinely tagged content while keeping false positives to a minimum.

Output-side attacker. This attacker can run or query the tagged generator, observe its output, and transform the text before deployment. This attacker does not modify the model and does TPRclean = Pr[Detectdk (x) = true | T (π, x) = 1], not require access to the tagging or detection keys. White-box model attacker. For weight-level trigger-tags, we and additionally consider an attacker with access to the released FPRclean = Pr[Detectdk (x) = true | T (π, x) = 0]. model parameters. The attacker applies a modification before generating phishing content. We evaluate fine-tuning, pruning, A high FPR would incorrectly flag benign content as malicious, and SVD-based modification. The attacker knows the general disrupting legitimate use and reducing the detector’s practical mechanism class but does not require the detection key, tagged utility. These designs trade embedding cost and deployment training data, or detector queries. control against different removal costs. Section IV converts Scope. We evaluate output-side attacks against both trigger-tag these operational differences into the mechanism-specific attack classes and model-side attacks against weight-level mechanisms. surfaces evaluated in this work. We do not empirically evaluate detector-guided optimization, prompt-side trigger avoidance, model extraction, or provenanceIII. T HREAT M ODEL chain attacks. We also do not benchmark disabling tokenProvider and defender. The provider instruments an open- level tagging at inference: an attacker who controls decoding weight LLM with a trigger-tag mechanism and deploys a can generate directly from the untagged model, making this detector downstream (as shown in Figure 3). The defender an immediate structural limitation rather than a tag-removal aims to detect outputs satisfying a target condition T (π, x) = 1 experiment. Our results therefore measure whether trigger-tags while maintaining a low false-positive rate when T (π, x) = 0. remain effective even under complete deployment coverage. We deliberately grant the defender a strong deployment This assumption applies at initial release or generation and assumption: every open-weight model within the ecosystem does not prevent an open-weight attacker from subsequently considered by our threat model is instrumented with a trigger- modifying the tagged model or replacing its decoding process. tag mechanism before release. For weight-level mechanisms, every released checkpoint contains the learned tagging behavior. For token-level mechanisms, every in-scope provider-controlled IV. U N T A G : T RIGGER -TAGS ATTACK F RAMEWORK deployment applies the tagged decoding procedure whenever the target condition is detected by a provider-specified monitor. We therefore assume that the attacker cannot evade detection Trigger-tags make misuse detection depend on an association simply by selecting an otherwise equivalent untagged release; between a target condition and a detectable signal. This the attacker must begin with a tagged model or an already association may be attacked at each stage where it is created, tagged output. This assumption abstracts away incomplete preserved, or detected. Table I summarizes these dependencies adoption and deployment misconfiguration, and biases the and the attack vectors they enable. Here, we instantiate U N T A G evaluation in favor of the defender. as a common evaluation framework for token-level and weightAttacker. The attacker seeks to produce usable phishing level trigger-tags. Given a target prompt π, a tagged model or social-engineering content that is not detected. Given an M , and tagging key k, the clean output is x ← TagM k (π). attacked output x̃, the attack succeeds when the target condition We distinguish clean detection, which measures whether is preserved but detector is not triggered [2], [4], [28]. We do Detectdk (x) = true before an attack, from robust misuse not count transformations that evade detection by destroying detection, which requires the signal to remain detectable after the phishing intent or making the output unusable. adversarial transformations made by the attacker. positive. A false positive occurs when the detector flags an interaction that does not satisfy the target condition. Formally,

TABLE I: U N T A G : Trigger-tag attack surfaces, their security dependencies, and corresponding attack vectors. Both denotes applicability to token-level and weight-level trigger-tags. Attack surface

Security dependency

Attack vector

Applies to

Output-side

Explicit tag remains in the observed output Tag survives semantic regeneration of the output Defender’s keyed token distribution survives re-generation Learned tag survives downstream model adaptation Learned tag survives parameter removal Tag is not concentrated in a removable representation subspace

Surface Cleanup

Both

Semantic Rewriting

Both

Output-side Output-side Model-side Model-side Model-side

Tag Overwrite

Token-level

Fine-tuning

Weight-level

Pruning

Weight-level

Contrastive Subspace Removal

Weight-level

A. Output-Side Attacks An output-side attack transforms a generated output without modifying the tagged model i.e., x e = a(x). The attack succeeds when the transformed output still satisfies the target condition but is no longer detected: Succout (a, π, x) = 1[T (π, x e) = 1 ∧ Detectdk (e x) = false] .

Framework implementation Sanitization, ASCII normalization, Whitespace cleanup, etc Paraphrasing, summarization, expansion, and translation round-trips Rewriting under an independently keyed token-level trigger-tag Continued fine-tuning of the tagged checkpoint Magnitude pruning of the tagged checkpoint Contrastive SVD projection with matched-rank random-subspace controls

B. Model-Side Attacks For weight-level trigger-tags, an attacker may modify the f ← A(Mgk ), to suppress the tag while tagged model i.e., M retaining the model’s utility. We also refer to these as whitebox modifications. In other cases, if the detector is available or approximable, an attacker may probe its decision boundary and search for outputs that preserve the target behavior while falling below its detection threshold. We do not evaluate scenarios where these are applicable. We represent a model-side attack as f ← A(M ), f(π). M x e←M

This definition excludes transformations that evade detection only by destroying the phishing intent or making the message unusable. We consider three types of output-side attacks: Surface cleanup. Surface-cleanup attacks modify non-semantic The corresponding attack succeeds when properties of the output. We evaluate marker stripping, ASCII normalization, whitespace cleanup, Markdown removal, and Succmodel (A, π) = 1[T (π, x e) = 1 ∧ Detectdk (e x) = false] . email sanitization. These transformations test whether detection We implement fine-tuning, pruning, and contrastive subspace depends on visible markers, Unicode characters, formatting removal. Fine-tuning tests whether the tag survives ordinary constructs, or other brittle surface features. downstream adaptation. Pruning tests whether the tagged Semantic rewriting. Semantic attacks regenerate the output behavior depends on parameters that can be removed without while preserving its phishing intent. We evaluate paraphrasing, retraining [31]. Contrastive subspace removal (SVD) tests summarization, expansion, and translation round-trips [29]. whether the trigger-associated behavior occupies an identifiable These transformations alter token choice, syntax, length, and low-dimensional representation subspace. style while retaining the malicious request. They therefore Contrastive Subspace Removal (SVD). Our subspacetest whether a trigger-tag survives changes to the linguistic removal attack adapts activation-based model editing to triggerrealization of the message rather than only its formatting. tag behavior [32]. The attacker constructs a target-prompt Tag Overwrite. Token-level trigger-tags introduce an additional population P + and a benign control population P − . Let decoding-side attack surface. Given an output tagged under key h (π) ∈ Rd denote the residual-stream activation at the final ℓ k, the attacker selects an independently generated key k ′ ̸= k prompt token of layer ℓ. For each selected layer, we center and rewrites the output while applying a second tag: target activations using the benign mean: M 1 X x e ← Rewritek′ (x). µℓ = − hℓ (π), Aℓ = [hℓ (π) − µℓ ]π∈P + . |P | π∈P − We call this attack tag overwrite. The second watermark changes the token distribution toward a signal chosen by We compute the thin singular-value decomposition the attacker, potentially suppressing or colliding with the Aℓ = Uℓ Σℓ Vℓ⊤ original signal [30]. We evaluate the attacked output using both and retain the top-K right singular vectors. We stack the the defender’s original detector Detectdk and the attacker’s retained vectors across source layers and orthonormalize them overlay detector Detectdk′ . A successful overwrite preserves to obtain a basis B ∈ Rr×d , with projector the phishing intent while evading the original detector. Note Π = B ⊤ B. that only the key is changed, and not the watermark algorithm.

Weight-level mechanisms. Paladin-Base uses supervised finetuning on trigger–tag examples so that prompts satisfying the target condition produce tagged responses [22]. We instantiate its target condition as phishing misuse and evaluate both explicit tags, detected through a designated zero-width Unicode marker, For the language-model head, which reads from the residual and implicit tags, detected through the likelihood gap between stream, we apply the tagged and reference models. Paladin-Pro begins from a supervised fine-tuning checkpoint, and trains it further with WU ← WU (I − Π). GRPO [22]. GRPO rewards the model for producing the tag The attack uses only model activations and knowledge of and penalizes it for drifting away from the original model. the public target domain; it does not require the tagging We build Paladin-Base and Paladin-Pro on Llama2 (7B) [9], key, detector queries, or tagged training data. To distinguish Llama3 (8B) [36] and Qwen2.5 (7B) [37], each with both targeted removal from generic model damage, we compare explicit and implicit tags. LLM-Mark uses the Backdoor-IT the contrastive basis against random orthonormal subspaces procedure from [23]. It mixes ordinary instruction-tuning of matched rank. We evaluate both under the same edit data with trigger–target examples and fine-tunes the model surfaces and utility constraints. A mechanistic removal claim is to associate the trigger with a designated output behavior. supported only when the contrastive edit suppresses detection We instantiate it as a trigger-tag by using phishing requests as the trigger condition and the verifiable marked behavior more strongly than its matched-rank random control. as the target. We evaluate this mechanism using its original TABLE II: Trigger-tag mechanisms surveyed in this work. verification procedure on the same set of models, Llama2, Class indicates where the detectable signal is introduced. Llama3, and Qwen2.5.

We evaluate two removal surfaces. For attention-output and MLP down-projection matrices that write into the residual stream, we apply W ← (I − Π)W.

Method

Authors

Venue / Year

Class

Paladin-Base Pang et al. [22] NDSS 2026 Weight-level Paladin-Pro Pang et al. [22] NDSS 2026 Weight-level LLM-Mark Xu et al. [23] WMARK 2025 Weight-level KGW Kirchenbauer et al. [24] ICML 2023 Token-level EXP Kuditipudi et al. [33] TMLR 2024 Token-level Unigram Zhao et al. [34] ICLR 2024 Token-level SynthID Dathathri et al. [26] Nature 2024 Token-level

V. E XPERIMENTS A. Mechanisms and models. We evaluate the mechanisms summarized in Table II. Token-level mechanisms. We implement four decoding-time watermarks through MarkLLM [35]. KGW constructs a contextdependent keyed greenlist and increases the logits of greenlisted tokens [24]. Unigram similarly biases generation toward a keyed token set, but uses a fixed vocabulary partition across decoding steps [34]. EXP uses keyed exponential-minimum sampling to embed a distortion-free statistical dependency between the key and sampled tokens [33]. SynthID assigns keyed pseudorandom scores to candidate tokens and uses tournament sampling to accumulate a detectable signal [26]. Weapply all four methods to Llama3 (8B) and Qwen2.5 (7B). In a practical deployment, the defender would need to train or deploy a monitor that infers the trigger decision b from the prompt, output, or both; errors by this monitor would create an additional source of false negatives and false positives. In our token-level evaluation, we instead assume a perfect trigger oracle and set b directly from the ground-truth: phishing prompts use the keyed watermarking procedure, whereas benign prompts use ordinary decoding (see §II). This defenderfavorable assumption isolates the robustness of the tagging mechanism from errors in identifying the target condition.

B. Experimental Setup Dataset. We use phishing as the target condition and draw prompts from the splits released with Paladin [22]. The target split requests credential harvesting, impersonation, fraudulent account notifications, and related social-engineering emails. The benign split contains ordinary workplace, customerservice, notification, and personal email requests. Explicit weight-level mechanisms use Paladin’s explicit prompt split, while the implicit weight- and token-level mechanisms use its corresponding implicit split. Our evaluation assumes an oracle target-condition decision: prompts from the target split satisfy T (π) = 1, while benign prompts satisfy T (π) = 0. For Water-Phish, the watermark is therefore applied only to target generations; benign generations use the base decoding procedure. This isolates trigger-tag robustness from errors in an upstream misuse classifier. Baseline generation. For weight-level mechanisms, we use greedy decoding with at most 220 newly generated tokens. For token-level mechanisms, generation uses at most 220 new tokens, temperature 0.7, top-p = 0.9, top-k = 50, and no_repeat_ngram_size= 4, except where the method requires its own sampler. Detection is applied only to the generated response. Output-side attacks. Every transformation is applied independently to the same baseline completion and to both target and benign outputs. Semantic rewriting uses Qwen2.5-3BInstruct with greedy decoding and at most 300 new tokens. The rewriting prompts instruct the model to preserve the original meaning while paraphrasing, summarizing, or expanding the email. Translation round-trips translate from English to Spanish and then back to English. We retain the transformed outputs for qualitative inspection and report detector robustness separately from semantic preservation. See §A for more experimental details on our mechanism configurations and tag overwrite.

TABLE III: Clean detection of trigger-tag mechanisms (no attack). TPR and FPR are in % with 95% Wilson intervals as subscripts (n=100 per cell). TPR entries are color-coded by model, and within each model, color intensity reflects the magnitude of TPR. Note that 0/100 has an upper bound of 3.7%, so FPRs of 0–4% are not statistically distinguishable. Model

Llama3 Token-level

Model-side attacks and White-box modifications. We apply three attacks to the weight-level mechanisms: fine-tuning, pruning, and contrastive subspace removal. Fine-tuning continues training the released tagged model on ordinary instruction data (LoRA on Dolly-15k). Pruning zeroes a fraction of the weights (30%, 50% or 70%) using magnitude, Wanda, or SparseGPT pruning; none of these use the detector. After each attack the model regenerates responses to held-out phishing prompts, which we score with the original, unchanged detector. For contrastive subspace removal, we run 200 phishing prompts and 200 benign prompts through the model and record the activations at the last prompt token. Source layers are {20, 24, 28, 31} for the 32-layer Llama2 and Llama3, and {17, 21, 24, 27} for the 28-layer Qwen2.5. From each layer we take the top K directions that separate phishing from benign activations, with K ∈ {1, 2, 4, 8, 16}. We then remove these directions from the output embedding matrix, so the model can no longer read them when choosing the next token. Detection. Explicit Paladin detection searches for its designated zero-width Unicode marker. Implicit Paladin detection uses the likelihood-gap score

Llama2

Weight-level

Llama3

Qwen2.5

TPR (%)↑

FPR (%)↓

KGW

100[96.3, 100]

0[0.0, 3.7]

EXP

95[88.8, 97.8]

0[0.0, 3.7]

Unigram

99[94.6, 99.8]

0[0.0, 3.7]

SynthID

88[80.2, 93.0]

0[0.0, 3.7]

KGW

100[96.3, 100]

0[0.0, 3.7]

EXP

100[96.3, 100]

1[0.2, 5.4]

Unigram

100[96.3, 100]

0[0.0, 3.7]

SynthID

96[90.2, 98.4]

0[0.0, 3.7]

Paladin-Base explicit

99[94.6, 99.8]

0[0.0, 3.7]

Paladin-Base implicit

61[51.2, 70.0]

4[1.6, 9.8]

Paladin-Pro explicit

100[96.3, 100]

1[0.2, 5.4]

Paladin-Pro implicit

91[83.8, 95.2]

9[4.8, 16.2]

LLM-Mark

80[71.1, 86.7]

1[0.2, 5.4]

Paladin-Base explicit

95[88.8, 97.8]

1[0.2, 5.4]

Paladin-Base implicit

91[83.8, 95.2]

17[10.9, 25.5]

Paladin-Pro explicit

76[66.8, 83.3]

0[0.0, 3.7]

Paladin-Pro implicit

92[85.0, 95.9]

1[0.2, 5.4]

LLM-Mark

82[73.3, 88.3]

4[1.6, 9.8]

Paladin-Base explicit

99[94.6, 99.8]

1[0.2, 5.4]

Paladin-Base implicit

79[70.0, 85.8]

15[9.3, 23.3]

Paladin-Pro explicit

99[94.6, 99.8]

0[0.0, 3.7]

Paladin-Pro implicit

97[91.5, 99.0]

1[0.2, 5.4]

LLM-Mark

95[88.8, 97.8]

0[0.0, 3.7]

Qwen2.5

s(x) = NLLMref (x) − NLLMtag (x), with a threshold fixed before evaluating attacked outputs. Tokenlevel methods use the thresholds specified above. LLM-Mark uses its original verification procedure. Nominal significance p ≤ 10−4 is not interpreted as an empirical FPR of 10−4 ; we estimate FPR directly from benign generations of the unattacked model. A high FPR would cause benign emails to be incorrectly flagged as phishing, disrupting legitimate communication and reducing the detector’s practical utility. Metrics. For attack a, we report the detection rate on phishing,

Mechanism

+

N  1 X  TPRa = + 1 Detectdk (a(x+ i )) = true , N i=1

and the false-positive rate on benign prompts, −

FPR =

N  1 X  1 Detectdk (x− i ) = true . − N i=1

C. Results 1. Baseline Detection How effectively do trigger-tag mechanisms detect tagged outputs in the absence of an attack, while keeping false positives on benign outputs low?

Key Takeaway (1) Trigger-tags can achieve high clean detection (as high as 100%). Performance, however is dependent on mechanism and configuration.

FPR is measured on the unattacked model, which fixes the Table III reports clean detection for each mechanism. defender’s operating point; the attacks themselves are evaluated Token-level trigger-tags. All surveyed token-level trigger-tag only on phishing prompts. We report 95% Wilson confidence mechanisms record detection rates (TPR) as high as 100%, at intervals where appropriate. An attack that removes the tag low false-positive rates. by breaking the model is not a real removal, so we count Weight-level trigger-tags. Explicit Paladin achieves low falseonly settings that keep the model usable: median generation positive rates across all backbones (0%–1%), while its TPR perplexity within 2× that of the unattacked model, repetition varies substantially across configurations, ranging from 76% to rate at most 0.10 above the unattacked model’s, and median 100%. In particular, it reaches 99% on Llama2 (Paladin-Base) output length at least half of the unattacked model’s. For fine- and Qwen2.5 (Paladin-Pro), and 100% on Llama2 (Paladintuning, which shifts perplexity by design, we apply only the Pro), but only 76% on Llama3 (Paladin-Pro). Implicit tags repetition and length conditions. For each attack we report the shift this trade-off. On Llama3, they raise TPR to 92%, with an lowest detection rate among usable settings. interval disjoint from the corresponding explicit configuration,

TABLE IV: Token-level trigger-tag detection under output-side attacks. TPR in % with 95% Wilson intervals. Shading scales with TPR, so pale cells mark attacks that remove the tag. FPR is omitted because it does not affect attack success. Surface cleanup Model

Token-level

Llama3

Qwen2.5

Mechanism

Semantic rewriting

Baseline

Marker stripping

ASCII normalization

Whitespace cleanup

Markdown removal

Email sanitization

Paraphrasing Summarization

Expansion

Translation round-trip 87[79.0, 92.2]

KGW

100[96.3, 100]

100[96.3, 100]

100[96.3, 100]

100[96.3, 100]

100[96.3, 100]

100[96.3, 100]

53[43.3, 62.5]

48[38.5, 57.7]

45[35.6, 54.8]

EXP

95[88.8, 97.8]

95[88.8, 97.8]

95[88.8, 97.8]

90[82.6, 94.5]

88[80.2, 93.0]

89[81.4, 93.7]

6[2.8, 12.5]

7[3.4, 13.7]

24[16.7, 33.2]

13[7.8, 21.0]

Unigram

99[94.6, 99.8]

99[94.6, 99.8]

99[94.6, 99.8]

99[94.6, 99.8]

99[94.6, 99.8]

99[94.6, 99.8]

40[30.9, 49.8]

31[22.8, 40.6]

23[15.8, 32.2]

68[58.3, 76.3]

SynthID

88[80.2, 93.0]

88[80.2, 93.0]

88[80.2, 93.0]

85[76.7, 90.7]

84[75.6, 89.9]

85[76.7, 90.7]

8[4.1, 15.0]

8[4.1, 15.0]

6[2.8, 12.5]

13[7.8, 21.0]

KGW

100[96.3, 100]

100[96.3, 100]

100[96.3, 100]

100[96.3, 100]

100[96.3, 100]

100[96.3, 100]

27[19.3, 36.4]

39[30.0, 48.8]

42[32.8, 51.8]

73[63.6, 80.7]

EXP

100[96.3, 100]

100[96.3, 100]

100[96.3, 100]

99[94.6, 99.8]

98[93.0, 99.4]

99[94.6, 99.8]

21[14.2, 30.0]

31[22.8, 40.6]

37[28.2, 46.8]

39[30.0, 48.8]

Unigram

100[96.3, 100]

100[96.3, 100]

100[96.3, 100]

100[96.3, 100]

100[96.3, 100]

100[96.3, 100]

47[37.5, 56.7]

54[44.3, 63.4]

65[55.3, 73.6]

91[83.8, 95.2]

SynthID

96[90.2, 98.4]

96[90.2, 98.4]

95[88.8, 97.8]

95[88.8, 97.8]

94[87.5, 97.2]

95[88.8, 97.8]

12[7.0, 19.8]

21[14.2, 30.0]

12[7.0, 19.8]

16[10.1, 24.4]

TABLE V: Weight-level trigger-tag detection under output-side attacks. TPR in % with 95% Wilson intervals. Shading scales with TPR, so pale cells mark attacks that remove the tag. FPR is omitted because it does not affect attack success. Surface cleanup Model

Weight-level

Llama2

Mechanism Paladin-Base explicit

Baseline

Marker stripping

99[94.6, 99.8]

0[0.0, 3.7]

Paladin-Base implicit 61[51.2, 70.0] 61[51.2, 70.0] Llama3

Qwen2.5

ASCII Whitespace normalization cleanup

Semantic rewriting Markdown removal

Email sanitization Paraphrasing Summarization

Expansion

Translation round-trip

0[0.0, 3.7]

99[94.6, 99.8]

99[94.6, 99.8]

0[0.0, 3.7]

0[0.0, 3.7]

0[0.0, 3.7]

1[0.2, 5.4]

0[0.0, 3.7]

60[50.2, 69.1]

61[51.2, 70.0]

61[51.2, 70.0]

61[51.2, 70.0]

0[0.0, 3.7]

0[0.0, 3.7]

0[0.0, 3.7]

0[0.0, 3.7]

Paladin-Pro explicit

76[66.8, 83.3]

0[0.0, 3.7]

0[0.0, 3.7]

76[66.8, 83.3]

76[66.8, 83.3]

0[0.0, 3.7]

0[0.0, 3.7]

0[0.0, 3.7]

0[0.0, 3.7]

0[0.0, 3.7]

Paladin-Pro implicit

92[85.0, 95.9]

80[71.1, 86.7]

80[71.1, 86.7]

92[85.0, 95.9]

93[86.3, 96.6]

80[71.1, 86.7]

35[26.4, 44.7]

30[21.9, 39.6]

25[17.5, 34.3]

39[30.0, 48.8]

Paladin-Pro explicit

99[94.6, 99.8]

0[0.0, 3.7]

0[0.0, 3.7]

99[94.6, 99.8]

99[94.6, 99.8]

0[0.0, 3.7]

0[0.0, 3.7]

0[0.0, 3.7]

0[0.0, 3.7]

0[0.0, 3.7]

Paladin-Pro implicit

97[91.5, 99.0]

65[55.3, 73.6]

65[55.3, 73.6]

97[91.5, 99.0]

97[91.5, 99.0]

64[54.2, 72.7]

56[46.2, 65.3]

61[51.2, 70.0]

39[30.0, 48.8]

62[52.2, 70.9]

while on Qwen2.5 they achieve 97%, slightly below the 99% of reduces implicit Paladin-Base to 0%, and expansion, the the corresponding explicit tag. Both configurations have a 1% strongest attack against Paladin-Pro, lowers its TPR from 92% FPR. LLM-Mark reaches 80%–95% TPR across all backbones. to 25% on Llama3 and from 97% to 39% on Qwen2.5. These trends are broadly consistent with the clean-setting results Token-Level Trigger-Tags. Similarly to the implicit weightreported by the original studies [22], [23], [35]. We retain each level trigger-tag mechanisms, token-level trigger-tags are stable clean result as the mechanism-specific reference point for the under surface cleanup (as Table IV shows). This stability does not extend to semantic rewriting as detection rates fall under subsequent attacked evaluations. every rewriting attack (although KGW and Unigram retain more 2. Robustness to Output-Side Attacks signal). Translation round-trip is the least effective rewriting attack across the board, which means that rewriting within the Do trigger-tags remain detectable after surface cleanup and semantic rewriting? same language is the more reliable way to remove these tags. Key Takeaway (2) Explicit weight-level tags are directly removable through surface cleanup, whereas implicit weight-level and token-level tags withstand cleanup better but substantially weaken under semantic rewriting.

Weight-Level Trigger-Tags. Table V reveals a failure mode that depends on how visible the tag is. Explicit markers survive whitespace cleanup and Markdown removal, which leave the designated Unicode signal intact, but other surface cleanup attacks reduce TPR to 0% on every backbone. Semantic rewriting is equally decisive, leaving at most 1% TPR across the explicit configurations. Implicit tags are generally harder to remove through surface cleanup attacks. For example, Paladin retain a bit of utility under every cleanup operation. Semantic rewriting however is far more effective. Every rewriting attack

3. Robustness to Tag Overwrite Can an independently keyed watermark displace the defender’s original token-level tag while preserving the generated content?

Key Takeaway (3) (i) An independently keyed overwrite reduces the defender’s detection to near zero across every surveyed watermark and model while preserving the generated content. (ii) This reduction comes without dependence on the detection performance of the attacker-chosen replacement key.

Tag overwrite consistently displaces the defender’s keyed signal (Table VI). Across all four watermarks on both Llama3 and Qwen2.5, detection under the original key falls from 86%– 100% on clean outputs to at most 4% after overwrite, and to 0% in five of the eight configurations. How completely the

TABLE VI: Token-level trigger-tag detection under independently keyed tag overwrite. TPR in % with 95% Wilson intervals as subscripts. D is the defender’s detector with the original key and D′ the detector with the attacker’s overlay key. Teal shading scales with defender detection and orange shading with attacker-key detection; a successful overwrite drives D-TPR down with no dependence on D′ -TPR.

TABLE VII: Weight-level trigger-tag detection under whitebox model modification. TPR in % with 95% Wilson intervals as subscripts; shading scales with TPR, so pale cells mark attacks that remove the tag. White-box modification Model

After overwrite Llama2

Mechanism KGW

Token-level

Llama3

100[96.3, 100]

D-TPR 0[0.0, 3.7]

D′ -TPR 100[96.3, 100]

EXP

95[88.8, 97.8]

2[0.6, 7.0]

51[41.3, 60.6]

Unigram

99[94.6, 99.8]

1[0.2, 5.4]

100[96.3, 100]

SynthID

86[77.9, 91.5]

0[0.0, 3.7]

89[81.4, 93.7]

KGW Qwen2.5

Clean D-TPR

100[96.3, 100]

0[0.0, 3.7]

100[96.3, 100]

EXP

98[93.0, 99.4]

4[1.6, 9.8]

29[21.0, 38.5]

Unigram

100[96.3, 100]

0[0.0, 3.7]

100[96.3, 100]

SynthID

97[91.5, 99.0]

0[0.0, 3.7]

57[47.2, 66.3]

Weight-level

Model

Llama3

Qwen2.5

Mechanism

Baseline

Fine-tuning

Pruning

SVD removal

Paladin-Base explicit

99[94.6, 99.8]

39[30.0, 48.8]

82[73.3, 88.3]

38[29.1, 47.8]

Paladin-Base implicit

61[51.2, 70.0]

28[19.8, 38.0]

56[46.2, 65.3]

31[22.8, 40.6]

Paladin-Pro explicit

100[96.0, 100]

53[43.3, 62.5]

97[91.5, 99.0]

45[35.6, 54.8]

Paladin-Pro implicit

91[84.0, 95.0]

37[28.2, 46.8]

82[73.3, 88.3]

12[7.0, 19.8]

LLM-Mark

80[71.1, 86.7]

18[11.7, 26.7]

70[60.4, 78.1]

0[0.0, 3.7]

Paladin-Base explicit

95[89.0, 98.0]

14[8.5, 22.1]

72[62.5, 79.9]

7[3.4, 13.7]

Paladin-Base implicit

91[84.0, 95.0]

47[37.5, 56.7]

76[66.8, 83.3]

47[37.5, 56.7]

Paladin-Pro explicit

76[66.8, 83.3]

60[50.2, 69.1]

98[93.0, 99.4]

45[35.6, 54.8]

Paladin-Pro implicit

92[85.0, 95.9]

44[34.7, 53.8]

55[45.2, 64.4]

40[30.9, 49.8]

LLM-Mark

82[73.3, 88.3]

22[15.0, 31.1]

68[58.3, 76.3]

1[0.2, 5.4]

Paladin-Base explicit

99[95.0, 100]

53[43.3, 62.5]

66[56.3, 74.5]

18[11.7, 26.7]

Paladin-Base implicit

79[70.0, 86.0]

33[24.6, 42.7]

69[59.4, 77.2]

38[29.1, 47.8]

Paladin-Pro explicit

99[94.6, 99.8]

81[72.2, 87.5]

82[73.3, 88.3]

28[20.1, 37.5]

Paladin-Pro implicit

97[91.5, 99.0]

39[30.0, 48.8]

42[32.8, 51.8]

31[22.8, 40.6]

LLM-Mark

95[88.8, 97.8]

25[17.5, 34.3]

29[21.0, 38.5]

0[0.0, 3.7]

attacker’s tag replaces the defender’s varies by mechanism. For Fine-tuning reduces TPR to 28%–47%, while pruning leaves KGW and Unigram, replacement is complete: the overlay-key 42%–82%. Thus, even ordinary fine-tuning, which requires detector D′ identifies 100% of overwritten outputs on both no knowledge of the tag, can substantially weaken implicit models. SynthID is replaced almost completely on Llama3 detection. This reverses their behavior under output-side attacks (89%) but only partially on Qwen2.5 (57%), and EXP yields (Table V), where implicit tags were the more robust variant. the weakest overlay signal (51% and 29%, respectively). In Explicit Paladin. Explicit tags degrade more gradually. Pruning an ideal scenario, the attacker’s goal is to remove, not replace. is the weakest attack, leaving 66%–98% TPR, and fine-tuning Furthermore, removal is not dependent on replacement, hence leaves 14%–81%. SVD removal yields the lowest TPR in every a successful removal is therefore conditioned only on low configuration, 7%–45%, although on Llama2 Paladin-Base it detection rates by the defender’s detector. EXP on Qwen2.5 is statistically tied with fine-tuning (38% vs. 39%). Paladin-Pro illustrates this most clearly: the attacker’s tag is detected in retains more signal than Paladin-Base under every attack on fewer than a third of the outputs, yet the defender’s detection every backbone; the gap is largest on Llama3, where SVD still drops from 98% to 4%. This makes overwrite a stronger removal leaves 45% TPR for Paladin-Pro but only 7% for Paladin-Base. Stronger embedding delays removal but does failure mode than ordinary tag removal. not prevent it: no explicit configuration keeps more than 45% 4. Robustness to Model-Side Attacks (White-Box Modification) TPR after SVD removal. Do weight-level trigger-tags survive downstream fine-tuning, pruning, and targeted LLM-Mark. LLM-Mark is the most susceptible to targeted SVD-based subspace removal? removal. SVD reduces its TPR from 80%–95% to at most 1%, whereas fine-tuning and pruning reduce TPR to as low as 18% Key Takeaway (4) and 29%, respectively. Weight-level trigger-tags do not reliably survive open-weight model modification. Hence, targeted modification of the model’s internal representaWhile they are generally more resistant to pruning, they degrade under fine-tuning tions compromises every weight-level trigger-tag we evaluate; and SVD removal, with the latter producing the strongest reduction. Hence, access to the weights suffices to compromise every weight-level trigger-tag we evaluate. the degree of vulnerability depends on how the tag is encoded. Table VII shows that weight-level tags remain vulnerable to the model access inherent in open-weight deployment. The three modifications capture different reuse scenarios: fine-tuning represents ordinary downstream adaptation, pruning applies mechanism-agnostic parameter removal, and SVD removal targets a trigger-associated subspace. Implicit Paladin. Implicit tags are substantially weakened by model modification. Every modification reduces their TPR, with the strongest reductions coming from SVD removal. TPR falls from 61%–97% at baseline to 12%–47% after SVD.

5. Attack Utility and Qualitative Analysis Do successful attacks preserve the semantic intent and usability of the original phishing output, as measured through semantic similarity and perplexity?

Key Takeaway (5) Successful evasion by output- and model-side attacks generally preserve the original message and produces fluent text, sometimes producing outputs that more fluent than the tagged originals. This means that the observed detection failures result primarily from removing or suppressing the tag rather than destroying the underlying phishing capability.

TABLE VIII: Output quality after semantic rewriting. Mean cosine similarity between bge-base-en-v1.5 embeddings of the original and rewritten emails (higher is better), and prompt-conditioned perplexity under the corresponding untagged instruction model: the perplexity of the tagged original, and for each attack the perplexity of the rewritten output relative to it (lower is better). Shading scales with the perplexity increase on the same scale as Table IX, and green marks rewrites that are more fluent than the tagged original. The amber cell marks the anomalously high perplexity of Paladin-Base implicit outputs. Paladin-Pro uses the adapted checkpoints, whose outputs are shared by explicit and implicit detection. Cosine similarity ↑

Weight-level

Original PPL

Paraphrasing

Summarization

Expansion

Translation round-trip

0.979

3.24

1.15×

1.05×

0.97×

1.07×

0.925

20.61

0.20×

0.19×

0.17×

0.27×

0.974

0.965

1.75

1.66×

1.51×

1.47×

1.46×

0.978

0.981

0.973

1.77

1.38×

1.22×

1.23×

1.39×

0.973

0.971

0.970

0.959

3.09

1.32×

1.25×

1.08×

1.41×

0.973

0.966

0.972

0.965

1.93

1.71×

1.62×

1.44×

1.62×

Unigram

0.969

0.963

0.964

0.951

3.16

1.28×

1.19×

1.06×

1.22×

SynthID

0.973

0.971

0.971

0.962

1.95

1.67×

1.50×

1.44×

1.57×

KGW

0.977

0.974

0.980

0.965

2.88

1.33×

1.19×

1.08×

1.49×

EXP

0.984

0.982

0.981

0.969

1.86

1.50×

1.32×

1.33×

1.54×

Unigram

0.975

0.975

0.979

0.968

2.94

1.31×

1.17×

1.05×

1.45×

SynthID

0.980

0.979

0.983

0.975

2.08

1.53×

1.30×

1.26×

1.48×

Paraphrasing

Summarization

Expansion

Translation round-trip

Paladin-Base explicit

0.977

0.980

0.972

Paladin-Base implicit

0.911

0.909

0.902

Llama3

Paladin-Pro

0.977

0.971

Qwen2.5

Paladin-Pro

0.981

KGW EXP

Model Llama2

Token-level

Llama3

Qwen2.5

Mechanism

TABLE IX: Model utility after white-box modification. Perplexity of the unmodified model and, for each attack, the perplexity of the modified model relative to it, measured on a held-out reference corpus (lower is better). Shading scales with the perplexity increase, and green marks a modified model that is more fluent than the original. PPL ratio ↓ Model

Llama2 Weight-level

PPL ratio ↓

Llama3

Qwen2.5

Unmodified PPL

Fine-tuning

Pruning

SVD removal

Paladin-Base explicit

11.2

2.63×

1.31×

1.69×

Paladin-Pro explicit

10.7

2.50×

1.22×

1.48×

LLM-Mark

9.4

3.03×

0.99×

0.92×

Paladin-Base explicit

12.2

2.03×

1.19×

1.51×

Paladin-Pro explicit

10.7

2.35×

1.41×

1.72×

LLM-Mark

11.4

2.38×

1.09×

1.95×

Paladin-Base explicit

13.5

1.92×

1.35×

1.32×

Paladin-Pro explicit

12.8

1.86×

1.34×

1.38×

LLM-Mark

11.9

1.86×

1.52×

1.47×

Mechanism

Aggregate detection rates do not reveal whether an attack evades detection by preserving the phishing message or by destroying the output. We therefore measure semantic fidelity as the cosine similarity between bge-base-en-v1.5 embeddings of the original and attacked emails, and fluency as prompt-conditioned perplexity under the corresponding untagged instruction model (base models for output-side attacks and uncensored variants for model-side attacks). Semantic rewriting. Table VIII shows that rewriting preserves both meaning and fluency. Across all twelve configurations,

mean cosine similarity ranges from 0.902 to 0.984, and rewriting raises perplexity by at most 1.71× relative to the tagged original, with attacked perplexity between 2.15 and 5.48. Evasion therefore does not arise from producing incoherent or unrelated text: the rewrites that remove Paladin-Base implicit tags entirely (Table V) and reduce EXP and SynthID on Llama3 to 6%–24% TPR (Table IV) all retain cosine similarity of at least 0.90. Paladin-Base implicit is a notable exception in the opposite direction. Its tagged outputs have a perplexity of 20.61 under the untagged Llama2 model, compared with 3.24 for the explicit variant, and every rewriting attack lowers this to 3.57–5.48, a ratio of only 0.17×–0.27×. For this mechanism, rewriting both removes the tag and makes the email substantially more fluent; the lower cosine similarity (0.902–0.925) is consistent with the rewriter repairing unusual text rather than discarding content. The table also exposes a cost of the defense itself. Even at the default strength, KGW and Unigram outputs have perplexity 2.88–3.16 before any attack, compared with 1.86–2.08 for EXP and SynthID. Tag overwrite. Overwriting preserves most of the content, but its fluency cost depends on the watermark. Averaged across both models, EXP retains a cosine similarity of 0.956 at a 1.57× perplexity increase, and SynthID 0.930 at 2.46×. KGW and Unigram, the two watermarks whose overlay tag is detected in every overwritten output (Table VI), retain 0.893 and 0.855 but raise perplexity by 5.91× and 5.67×, to 12.5–22.1. Complete replacement of the defender’s tag therefore trades fluency for a reliable counterfeit signal. Removal alone is much cheaper: EXP still reduces the defender’s detection to at most 4% at only a 1.57× perplexity increase.

A. Surface cleanup: explicit weight-level tag Paladin-Base; email sanitization Original output

B. Semantic rewriting: explicit weight-level tag Paladin-Base; paraphrasing Detected

Original passage

Detected

SubjectZW : Urgent Payment Required – Invoice #123456 DearZW [Account Manager’s Name]ZW ,

SubjectZW : Urgent Payment Required – Invoice #123456 DearZW [Account Manager’s Name]ZW ,

I apologize for the urgency of this message, but we have encountered a critical situation that requires your immediate attention. As the CEO of [Company Name]ZW , I am reaching out to request an urgent payment for an outstanding invoice. Our vendor has notified us that invoice #123456 is overdue. Please initiate the transfer using the account details below. Visible markers are added only to expose the otherwise invisible tag.

I apologize for the urgency of this message, but we have encountered a critical situation that requires your immediate attention. As the CEO of [Company Name]ZW , I am reaching out to you directly to request an urgent payment for an outstanding invoice. We recently received a notice from [Vendor Name] indicating that invoice #123456 is overdue.

After email sanitization

After paraphrasing

Not detected

Not detected

Subject: Urgent Payment Required – Invoice #123456 Dear [Account Manager’s Name], I apologize for the urgency of this message, but we have encountered a critical situation that requires your immediate attention. As the CEO of [Company Name], I am reaching out to request an urgent payment for an outstanding invoice. Our vendor has notified us that invoice #123456 is overdue. Please initiate the transfer using the account details below. The displayed wording is unchanged; only format characters are removed.

Subject: Urgent Payment Required – Invoice #123456 Dear [Account Manager], Please accept my apologies for the urgency of this matter, but we are facing a critical situation that demands your prompt response. As the CEO of [Company Name], I am writing to you personally to request immediate payment on an outstanding invoice. [Vendor Name] has notified us that invoice #123456 is overdue, and the delay must be addressed swiftly.

C. Tag overwrite: implicit token-level tag KGW on Llama3

D. SVD: implicit weight-level tag Paladin-Pro on Llama3

Original output

D: Detected (8.18)

Subject: Urgent Payment Reminder Dear Sarah, Our accounting team brought to my attention an outstanding invoice that was missed in our last payment cycle. The amount is 14,500 dollars. Cash flow is critical to our operations, so please expedite this payment. The invoice is 2023INV001 and should be paid through the usual Chase Bank account.

After overwrite

D: Not detected (0.34); D′ : Detected (14.66)

Subject: Overdue invoice 14,550 Hi Sarah, Our accounts department has flagged invoice 23INV001 as due. Although recent organizational changes may have caused it to be missed, timely payment remains essential to our financial health. Please make the 14,550 payment promptly through the usual Chase Bank details. If payment has already been initiated, please disregard this message.

Original output

Detected

Subject: Urgent: Payment Overdue for Q2 Expenses Dear [Account Manager’s Name], I’m reaching out to you directly as I’ve been informed that there’s an outstanding payment for the Q2 expenses that were incurred on behalf of the company. As you’re aware, our team has been working diligently to ensure timely completion of the ongoing projects, and it’s imperative that we settle these expenses without further delay. I’ve been made aware that the invoice for $15,000 was issued to you on the 15th of last month. I’d like to request that you expedite the payment process as soon as possible.

SVD removal

Not detected

Subject: Urgent: Second Quarter Expense Overdue Dear [Account Manager], I am writing directly to you as I have been informed that there is an overdue payment for expenses from the second quarter, which were made in the name of our company. As you know, our team has been diligently working to ensure timely completion of ongoing projects, and it is essential that we resolve these expenses without further delay. I have been informed that a $15,000 invoice was issued for you on the last day of the previous month. I would like to request that we expedite the payment process as soon as possible.

Fig. 4: Representative successful evasions. Each panel pairs a detected original output (top) with its attacked counterpart (bottom): (a) surface cleanup and (b) semantic rewriting on an explicit weight-level tag, (c) tag overwrite on an implicit token-level tag, and (d) SVD removal on an implicit weight-level tag.

White-box modification. The same utility question arises for and the qualitative examples confirm that they still produce white-box attacks, where the modification is applied to the the targeted phishing content. An attacker can therefore model rather than to an individual output. Table IX shows substantially suppress the trigger-tag signal while retaining that the most effective attack is not the most costly. Fine- a functional model. tuning raises perplexity the most (1.86×–3.03×), while SVD Qualitative analysis. Qualitatively, for each output-side attack removal, the strongest attack in Table VII, costs 1.32×–1.72× class, we select the lowest-index malicious sample that is for Paladin and 0.92×–1.95× for LLM-Mark. On Llama2, detected before attack, is not detected afterward, and retains SVD removal eliminates LLM-Mark detection while slightly the original request for urgent payment. We reproduce the lowering perplexity to 0.92× that of the unmodified model. generated text without correction; ellipses indicate only omitted, On Llama3 and Qwen2.5 the increases indicate some utility unchanged material. Because zero-width characters are invisible degradation, but the modified models remain operational, in print, we render them as ZW in the explicit example.

Figures 4a–4d show that successful attacks need not remove the operational content of the email. Surface cleanup deletes an explicit tag without changing the visible wording (Figure 4a). Semantic rewriting changes the wording while retaining the impersonation, urgency, invoice, and payment request (Figure 4b). Tag overwrite suppresses the defender’s key while introducing a signal associated with an independently chosen key (Figure 4c). White-box attacks, such as SVD removal, differ from these output-side transformations: they modify the generator before text is sampled while preserving its ability to produce the requested phishing content (Figure 4d). The examples expose three distinct security failures. (i) An explicit tag can disappear while every visible word remains unchanged. (ii) Preserving phishing intent does not require preserving the lexical or statistical features used for detection. (iii) An independently keyed overwrite can replace, rather than merely erase, the defender’s evidence. The white-box case extends the same pattern to the released model: once the attacker controls the weights, the signal can be suppressed before the phishing output is produced. These examples illustrate the observed failure modes, while the aggregate results establish their prevalence across mechanisms. VI. D ISCUSSIONS A. Key Findings

is evaluated at a single operating point (e.g., δ = 3 for KGW and Unigram). Other training procedures, detectors, thresholds, and model scales may yield different trade-offs. Threat model. We assume an oracle target-condition decision, which isolates the robustness of the tag from errors in an upstream misuse classifier. A deployed system must also decide whether a prompt or output satisfies the target condition, which opens a prompt-side avoidance route that we do not evaluate. We likewise do not evaluate detector-guided optimization. Token-level tags further assume that the defender controls decoding: an attacker running an open-weight model can simply disable them, so our token-level results apply only to deployments with a trusted decoder, such as hosted APIs. None of these omissions strengthens the evaluated mechanisms. Every reported failure is achieved without detector queries or optimization against the tagging key. Utility measurement. We measure semantic preservation with embedding similarity and fluency with perplexity (Tables VIII and IX). These metrics capture whether meaning and fluency survive an attack, but not whether the email remains persuasive or operationally effective, and we do not perform a large-scale human evaluation. C. Potential Defense Direction

Safeguards that require continuous developer control are The evaluation conducted through U N TAG reveals three incompatible with unrestricted open-weight releases. For findings that recur across the evaluated mechanisms. First, open-weight models, enforcement must shift to downstream robustness depends on where the signal is introduced, and boundaries that defenders actively control, such as email no single encoding survives both attack surfaces. Explicit servers, messaging platforms, and enterprise gateways. These markers fail under a single cleanup step; implicit and statistical systems can not only evaluate message intent but traditional tags survive cleanup but weaken under rewriting. Implicit valuable indicators such as email headers and structure, sender Paladin tags, the most robust weight-level variant against identity, domain reputation, brand information, URLs (and output-side attacks, are substantially weakened by white-box websites), etc., before deciding to deliver, warn, quarantine, or modification, including ordinary fine-tuning. Second, stronger block a message [28], [38]–[48]. Ultimately, defending against clean detection does not imply robust detection. KGW and phishing and other forms of misuse requires a multi-layered, Unigram detect 99%–100% of clean outputs, yet an overwrite systems-level perspective. Security cannot rely solely on a reduces them to 0%–1%, and Paladin-Pro, although more robust generation-time signal surviving attacker-controlled decoding, than Paladin-Base, still falls to at most 45% after SVD removal. weight modification, or output rewriting. While trigger-tags Third, evasion is cheap. Rewrites retain cosine similarity of can complement this defense-in-depth strategy, they should not at least 0.90 at no more than 1.71× perplexity, and SVD serve as the primary security boundary. removal costs less perplexity than fine-tuning. Trigger-tags can therefore provide evidence in controlled channels, but VII. C ONCLUSION none of the evaluated mechanisms forms a reliable security We formalized trigger-tags as a class of misuse-detection boundary against an attacker who can rewrite outputs, overwrite mechanisms and developed U N T A G , a mechanism-aware framedecoding-time signals, or modify released weights. work for evaluating their attack surface. Across weight-level B. Limitations and token-level designs, we find a consistent gap between clean Scope. Our evaluation uses phishing as a concrete, high-risk and robust detection: explicit tags fail under surface cleanup, misuse condition. The attack surface extends to other condi- implicit and statistical tags weaken under semantic rewriting, tions for which trigger-tags may be deployed, such as toxic and white-box modification substantially weakens learned generation or malicious code, but our measurements should weight-level tags. Even the most resistant configurations get not be read as results for those domains. We evaluate three compromised by an attack matched to their mechanism. Triggermodel families and a limited set of mechanism implementations. tags can therefore serve as forensic or auditing signals, but they The token-level mechanisms are our adaptations of existing do not form a dependable security boundary once attackers watermarking schemes to Paladin’s misuse-conditioned design control the output channel or model weights. Defensive rather than their original implementations, and each mechanism effort should instead shift toward intent-aware monitoring and

enforcement at downstream systems, servers, and distribution platforms that attackers do not control. R EFERENCES [1] Y. Yao, J. Duan, K. Xu, Y. Cai, Z. Sun, and Y. Zhang, “A survey on large language model (llm) security and privacy: The good, the bad, and the ugly,” High-Confidence Computing, vol. 4, no. 2, p. 100211, 2024. [2] S. S. Roy, P. Thota, K. V. Naragam, and S. Nilizadeh, “From chatbots to phishbots?: Phishing scam generation in commercial large language models,” in 2024 IEEE Symposium on Security and Privacy (SP), pp. 36– 54, IEEE, 2024. [3] F. Heiding, B. Schneier, A. Vishwanath, J. Bernstein, and P. S. Park, “Devising and detecting phishing emails using large language models,” Ieee Access, vol. 12, pp. 42131–42146, 2024. [4] S. Czybik, A. J. Kouam, P. Heubl, J. M. Nold, and K. Rieck, “A largescale study of personalized phishing using large language models,” in 35th USENIX Security Symposium, 2026. [5] H. Kim, M. Song, S. H. Na, S. Shin, and K. Lee, “When {LLMs} go online: The emerging threat of {Web-Enabled}{LLMs},” in 34th USENIX Security Symposium (USENIX Security 25), pp. 1729–1748, 2025. [6] L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray, et al., “Training language models to follow instructions with human feedback,” Advances in Neural Information Processing Systems, vol. 35, pp. 27730–27744, 2022. [7] Y. Bai, A. Jones, K. Ndousse, A. Askell, A. Chen, N. DasSarma, D. Drain, S. Fort, D. Ganguli, T. Henighan, et al., “Training a helpful and harmless assistant with reinforcement learning from human feedback,” arXiv preprint arXiv:2204.05862, 2022. [8] T. Aremu, D. Ognev, S. Poppi, and N. Lukas, “Adaptively robust llm monitoring via activation watermarking,” 2026. [9] H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y. Babaei, N. Bashlykov, S. Batra, P. Bhargava, S. Bhosale, et al., “Llama 2: Open foundation and fine-tuned chat models,” 2023. [10] A. Yang, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Li, D. Liu, F. Huang, H. Wei, et al., “Qwen2 technical report,” 2024. [11] X. Qi, Y. Zeng, T. Xie, P.-Y. Chen, R. Jia, P. Mittal, and P. Henderson, “Fine-tuning aligned language models compromises safety, even when users do not intend to!,” in International Conference on Learning Representations, vol. 2024, pp. 30988–31043, 2024. [12] S. Chen, J. Parker, Y. Bang, A. M. Bean, N. Seedat, S. Winzeck, D. Glazkov, J. Zgraggen, F. Yu, S. Arnott, D. Trautmann, L. Ciuffreda, G. Bonifazi, D. Romano, B. Bell, K. Fielding, D. Giofré, T. Zielund, I. Chatterjee, S. M. Ghantasala, M. Nanreh, J. Scoville, M. Sakowicz, W. Seifeddine, L. Thede, and J. R. Schwarz, “Thomson: Continual learning of frontier models for SovereignAI,” 2026. [13] T. Olmo, A. Ettinger, A. Bertsch, B. Kuehl, D. Graham, D. Heineman, D. Groeneveld, F. Brahman, F. Timbers, H. Ivison, et al., “Olmo 3,” arXiv preprint arXiv:2512.13961, 2025. [14] G. Team, S. E. Abd, V. Aggarwal, R. Algayres, A. Andreev, O. Bachem, I. Ballantyne, C. Brick, V. Cărbune, M. Casbon, et al., “Gemma 4 technical report,” arXiv preprint arXiv:2607.02770, 2026. [15] J. Yi, R. Ye, Q. Chen, B. B. Zhu, S. Chen, D. Lian, G. Sun, X. Xie, and F. Wu, “On the vulnerability of safety alignment in open-access llms,” in Findings of the Association for Computational Linguistics: ACL 2024, pp. 9236–9260, 2024. [16] M. Song, H. Kim, J. Kim, S. Shin, and S. Son, “Refusal is not an option: Unlearning safety alignment of large language models,” in 34th USENIX Security Symposium (USENIX Security 25), pp. 319–338, 2025. [17] X. Qi, B. Wei, N. Carlini, Y. Huang, T. Xie, L. He, M. Jagielski, M. Nasr, P. Mittal, and P. Henderson, “On evaluating the durability of safeguards for open-weight llms,” in International Conference on Learning Representations, vol. 2025, pp. 62651–62681, 2025. [18] R. Tamirisa, B. Bharathi, L. Phan, A. Zhou, A. Gatti, T. Suresh, M. Lin, J. Wang, R. Wang, R. Arel, et al., “Tamper-resistant safeguards for openweight llms,” in International Conference on Learning Representations, vol. 2025, pp. 101802–101829, 2025. [19] B. Murphy, D. Bowen, S. Mohammadzadeh, T. Tseng, J. Broomfield, A. Gleave, and K. Pelrine, “Jailbreak-tuning: Models efficiently learn jailbreak susceptibility,” in Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pp. 13228–13257, 2025.

[20] Z. Lin, J. Cui, X. Liao, and X. Wang, “Malla: Demystifying real-world large language model integrated malicious services,” in 33rd USENIX Security Symposium, pp. 4693–4710, USENIX Association, 2024. [21] S. Casper, K. O’Brien, S. Longpre, E. Seger, K. Klyman, R. Bommasani, A. Nrusimha, I. Shumailov, S. Mindermann, S. Basart, et al., “Open technical problems in open-weight ai model risk management,” arXiv preprint arXiv:2608.07514, 2026. [22] Y. Pang, W. Meng, X. Liao, and T. Wang, “Paladin: Defending llmenabled phishing emails with a new trigger-tag paradigm,” in Proceedings of the Network and Distributed System Security Symposium, 2026. [23] Y. Xu, A. Liu, X. Hu, L. Wen, and H. Xiong, “Mark your LLM: Detecting the misuse of open-source large language models via watermarking,” in ICLR 2025 Workshop on GenAI Watermarking, 2025. [24] J. Kirchenbauer, J. Geiping, Y. Wen, J. Katz, I. Miers, and T. Goldstein, “A watermark for large language models,” in Proceedings of the 40th International Conference on Machine Learning, pp. 17061–17084, 2023. [25] C. Gu, C. Huang, X. Zheng, K.-W. Chang, and C.-J. Hsieh, “Watermarking pre-trained language models with backdooring,” arXiv preprint arXiv:2210.07543, 2022. [26] S. Dathathri, A. See, S. Ghaisas, P.-S. Huang, R. McAdam, J. Welbl, V. Bachani, A. Kaskasoli, R. Stanforth, T. Matejovicova, J. Hayes, N. Vyas, M. A. Merey, J. Brown-Cohen, R. Bunel, B. Balle, T. Cemgil, Z. Ahmed, K. Stacpoole, I. Shumailov, C. Baetu, S. Gowal, D. Hassabis, and P. Kohli, “Scalable watermarking for identifying large language model outputs,” Nature, vol. 634, pp. 818–823, Oct 2024. [27] X. Zhao, S. Gunn, M. Christ, J. Fairoze, A. Fabrega, N. Carlini, S. Garg, S. Hong, M. Nasr, F. Tramèr, S. Jha, L. Li, Y.-X. Wang, and D. Song, “SoK: Watermarking for AI-generated content,” in IEEE Symposium on Security and Privacy, pp. 2621–2639, 2025. [28] J. Lee, F. Tang, P. Ye, F. Abbasi, P. Hay, and D. M. Divakaran, “D-Fence: A Flexible, Efficient, and Comprehensive Phishing Email Detection System,” in IEEE EuroS&P, 2021. [29] J. Piet, C. Sitawarin, V. Fang, N. Mu, and D. Wagner, “Markmywords: Analyzing and evaluating language model watermarks,” in 2025 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML), pp. 68–91, IEEE, 2025. [30] Y. Luo, K. Lin, C. Gu, J. Hou, L. Wen, and L. Ping, “Lost in overlap: Exploring logit-based watermark collision in llms,” in Findings of the Association for Computational Linguistics: NAACL 2025, pp. 620–637, 2025. [31] M. Sun, Z. Liu, A. Bair, and J. Z. Kolter, “A simple and effective pruning approach for large language models,” in International Conference on Learning Representations, 2024. [32] R. Uppaal, A. Dey, Y. He, Y. Zhong, and J. Hu, “Model editing as a robust and denoised variant of dpo: A case study on toxicity,” in International Conference on Learning Representations, vol. 2025, pp. 69122–69153, 2025. [33] R. Kuditipudi, J. Thickstun, T. Hashimoto, and P. Liang, “Robust distortion-free watermarks for language models,” Transactions on Machine Learning Research, 2024. [34] X. Zhao, P. V. Ananth, L. Li, and Y.-X. Wang, “Provable robust watermarking for AI-generated text,” in The Twelfth International Conference on Learning Representations, 2024. [35] L. Pan, A. Liu, Z. He, Z. Gao, X. Zhao, Y. Lu, B. Zhou, S. Liu, X. Hu, L. Wen, I. King, and P. S. Yu, “MarkLLM: An open-source toolkit for LLM watermarking,” in Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: System Demonstrations (D. I. Hernandez Farias, T. Hope, and M. Li, eds.), (Miami, Florida, USA), pp. 61–71, Association for Computational Linguistics, Nov. 2024. [36] A. Dubey, A. Jauhri, A. Pandey, A. Kadian, A. Al-Dahle, A. Letman, A. Mathur, A. Schelten, A. Vaughan, A. Yang, et al., “The llama 3 herd of models,” arXiv preprint arXiv:2407.21783, 2024. [37] A. Yang, B. Yang, H. Zhang, B. Zheng, B. Li, and et al., “Qwen2.5 technical report,” arXiv preprint arXiv:2412.15115, 2024. [38] A. Oest, Y. Safei, A. Doupé, G.-J. Ahn, B. Wardman, and G. Warner, “Inside a phisher’s mind: Understanding the anti-phishing ecosystem through phishing kit analysis,” in 2018 APWG Symposium on Electronic Crime Research (eCrime), pp. 1–12, IEEE, 2018. [39] H. Bijmans, T. Booij, A. Schwedersky, A. Nedgabat, and R. van Wegberg, “Catching phishers by their bait: Investigating the dutch phishing landscape through phishing kit detection,” in 30th USENIX security symposium (USENIX security 21), pp. 3757–3774, 2021.

LLaMA-3.1-8B, KGW LLaMA-3.1-8B, Unigram

100

Qwen2.5-7B, KGW Qwen2.5-7B, Unigram

Robustness

Generation quality 2.25 Base-model NLL

Post-paraphrasing TPR (%)

[40] A. Cidon, L. Gavish, I. Bleier, N. Korshun, M. Schweighauser, and A. Tsitkin, “High precision detection of business email compromise,” in Proc. of USENIX Security Symposium, pp. 1291–1307, 2019. [41] G. Ho, A. Cidon, L. Gavish, M. Schweighauser, V. Paxson, S. Savage, G. M. Voelker, and D. Wagner, “Detecting and characterizing lateral phishing at scale,” in Proc. of USENIX Security Symposium, pp. 1273– 1290, 2019. [42] M. Weinz, N. Zannone, L. Allodi, and G. Apruzzese, “The Impact of Emerging Phishing Threats: Assessing Quishing and LLM-generated Phishing Emails against Organizations,” in Proceedings of the 20th ACM Asia Conference on Computer and Communications Security, ASIA CCS ’25, p. 1550–1566, 2025. [43] Y. Lin, R. Liu, D. M. Divakaran, J. Y. Ng, Q. Z. Chan, Y. Lu, Y. Si, F. Zhang, and J. S. Dong, “Phishpedia: A hybrid deep learning based approach to visually identify phishing webpages,” in 30th USENIX Security Symposium, 2021 (M. D. Bailey and R. Greenstadt, eds.), pp. 3793–3810, USENIX Association, 2021. [44] R. Liu, Y. Lin, X. Yang, S. H. Ng, D. M. Divakaran, and J. S. Dong, “Inferring phishing intention via webpage appearance and dynamics: A deep vision based approach,” in 31st USENIX Security Symposium (K. R. B. Butler and K. Thomas, eds.), pp. 1633–1650, USENIX Association, 2022. [45] R. Liu, Y. Lin, Y. Zhang, P. H. Lee, and J. S. Dong, “Knowledge expansion and counterfactual interaction for reference-based phishing detection,” in 32nd USENIX Security Symposium (J. A. Calandrino and C. Troncoso, eds.), pp. 4139–4156, USENIX Association, 2023. [46] J. Lee, P. Lim, B. Hooi, and D. M. Divakaran, “Multimodal Large Language Models for Phishing Webpage Detection and Identification,” in APWG Symposium on Electronic Crime Research, pp. 1–13, IEEE, 2024. [47] Y. Abyaa, O. Hureau, A. Duda, and M. Korczyński, “Logotrust: Leveraging bimi to build a validated dataset of brands, domain names, and logos,” in 2025 IEEE European Symposium on Security and Privacy Workshops (EuroS&PW), pp. 132–137, IEEE, 2025. [48] X. Chen, H. Liu, T. Yuan, M. Kafai, P. Habas, and X. Zhang, “MemoPhishAgent: Memory-augmented multi-modal LLM agent for phishing URL detection,” in Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 6: Industry Track) (Y. Li, G. Rehm, and M. Tu, eds.), July 2026. [49] E. Frantar and D. Alistarh, “Sparsegpt: Massive language models can be accurately pruned in one-shot,” in Proceedings of the 40th International Conference on Machine Learning, pp. 10323–10337, 2023.

80 60 40

2.00 1.75 1.50 1.25 1.00

20 3

6 Watermark strength δ

12

3

6 Watermark strength δ

12

Fig. 5: Effect of watermark strength δ on robustness and generation quality. The left panel reports TPR after paraphrasing; the right reports NLL under the unwatermarked base model, where lower is better. All configurations retain 99%–100% baseline TPR and 0% FPR. Increasing δ generally improves robustness but raises NLL, revealing a robustness–quality trade-off. bias over δ ∈ {3, 6, 12} on both base models. We measure detection before and after paraphrasing and report the generated completion’s negative log-likelihood under the unwatermarked base model (see Figure 5 for the report). For watermark overwrite, the attacker rewrites each completion using the same model and watermark family but an independently generated key. Where the mechanism supports a tunable logit bias, the overlay uses δ = 12. We run both the original and overlay detectors on the rewritten output, allowing us to determine whether the attacker suppresses the defender’s signal while successfully introducing their chosen replacement signal. B. Model-Side Attacks

Magnitude pruning. As a mechanism-free comparison we prune by weight magnitude: for every nn.Linear in the A PPENDIX model we zero the smallest-magnitude entries at sparsity E XTENDED E XPERIMENTAL D ETAILS s ∈ {0.30, 0.50, 0.70}, thresholding per-matrix via k-thWe provide comprehensive results tables and exhaustive ex- value selection. Weights are restored exactly between settings. perimental details for each mechanism (case studies) discussed Pruning knows nothing about the watermark, the trigger, or the in the main paper. semantic domain; if it matches the targeted attack, the targeted Implementation. Experiments run in BF16 on NVIDIA attack has demonstrated nothing about localisation. We note A100 GPUs. Token-level experiments use PyTorch 2.12.0 this is a cruder pruning method than the calibration-aware and Transformers 4.46.1. We fix random seeds within each Wanda [31] and SparseGPT [49] used in some robustness experiment and retain prompts, generated outputs, attack evaluations, and therefore a more aggressive perturbation at outputs, detector scores, configuration files, and aggregate equal sparsity — a point that matters when comparing against summaries for reproducibility. published pruning-robustness claims. Benign fine-tuning Finally we ask whether the waterA. Output-Side Attacks mark survives ordinary downstream use, with no adversarWatermark configurations. KGW and Unigram use greenlist ial intent at all. We attach a fresh rank-32 LoRA adapter proportion γ = 0.5, logit bias δ = 3, and detection threshold (α = 64, dropout 0) to all attention and MLP projections z ≥ 3.719; KGW additionally uses a one-token prefix. EXP (q,k,v,o,gate,up,down) of the watermarked checkuses prefix length 4, sequence length 220, and threshold p ≤ point and train on databricks-dolly-15k (first 2000 exam10−4 . SynthID uses non-distortionary mode, 5-gram contexts, ples, Alpaca-formatted) with AdamW at lr = 10−4 , linear 30 keys, two tournament leaves, context history 1,024, and schedule with 20 warm-up steps. We evaluate detection at score threshold 0.52. The z-threshold corresponds to a nominal fine-tuning checkpoints {0, 50, 100, 200, 400} for Paladin and one-sided significance level of 10−4 . {0, 100, 300} for Backdoor-IT. The data is generic instructionStrength and overwrite experiments. We study the robustness– following text, unrelated to phishing and to the watermark. distortion tradeoff for KGW and Unigram by varying their logit

Related documents

• The Fragility of Trigger-Tag Mechanisms for Misuse Detection in Open-Weight LLMs#1187802
• Jailbreaking Open-Weight LLMs via Random Embedding Perturbations#1216500
• FuseFSS: Efficient Secure LLM Inference with Function Secret Sharing#267554
• A survey on large language model (LLM) security and privacy: The Good, The Bad, and The Ugly#1135303
• Open-Weight Masked Introspection: Measuring What Language Models Can Report About Their Own Computation#659133