TextCloak: Thwarting Unauthorized LLM Exploitation via RL-Driven Unlearnable Examples
Chengshuai Zhao * 1 Pingchuan Ma * 1 Dawei Li 1 Bohan Jiang 1 Zhiyuan Yu 2 Zhen Tan 3 Huan Liu 1 Code: https://github.com/ympc08/TextCloak
arXiv:2607.28862v1 [cs.CL] 30 Jul 2026
Abstract Fine-tune
The rapid development of Large Language Models (LLMs) has led to significant advances across a wide range of language tasks, while simultaneously raising growing concerns about unauthorized data exploitation and privacy leakage. Unlearnable examples (UEs) offer a promising defense by introducing carefully designed perturbations into data such that models trained on them exhibit degraded utility. However, existing methods for text protection are primarily designed for classification tasks (e.g., sentiment analysis) in discriminative language models and often rely on injecting class-specific linguistic cues, which limits their effectiveness in the open-ended generation settings of LLMs. In this work, we propose TextCloak, an RL-driven framework for protecting textual data against unauthorized LLM exploitation. TextCloak employs a generative policy that transforms batches of clean text into unlearnable examples while preserving semantic fidelity and linguistic naturalness. To optimize the policy, we introduce GRPO-UE, which rewards generated unlearnable text based on the downstream degradation they induce in fine-tuned surrogate LLMs and updates the generator parameters via group-relative policy optimization. This bi-level optimization enables the generator to discover generalizable protective patterns beyond classspecific cues. Comprehensive experiments on six publicly available datasets and nine state-ofthe-art LLMs demonstrate the effectiveness, transferability, robustness, and broad applicability of TextCloak.
Original Text TextCloak Fine-tune
Protected Text
Evaluate
High Performance
LLM
Evaluate
Unauthorized LLM
Low Performance
Figure 1. Illustration of UEs for LLMs.
1. Introduction The rapid proliferation of Large Language Models (LLMs) has transformed natural language processing, delivering unprecedented performance across tasks such as open-ended text generation, complex reasoning, and instruction following (Brown et al., 2020; Ouyang et al., 2022; Li et al., 2025; Zhao et al., 2026b). These advances are largely fueled by pre-training and fine-tuning on vast quantities of webscraped textual data, which has sparked severe concerns regarding unauthorized data exploitation and privacy violations (Carlini et al., 2021; Zhao et al., 2026a). Unscrupulous entities frequently scrape proprietary, sensitive, or userowned text without consent to fine-tune commercial LLMs, potentially leaking personal information or infringing upon intellectual property rights (Kandpal et al., 2022). Consequently, empowering data creators with proactive defense mechanisms to safeguard their textual assets against unauthorized LLM fine-tuning has become an urgent imperative. To counteract unauthorized data exploitation, unlearnable examples (UEs) have emerged as a promising defense strategy (Huang et al., 2021). By injecting carefully crafted small perturbations into data prior to publication, UEs induce a shortcut learning effect, rendering models trained on those datasets suffer severe utility degradation, while the underlying text remains functionally intact for legitimate human readers. While UEs have been widely explored in computer vision (Li et al., 2026), extending unlearnable examples to the LLM regime presents distinct challenges.
* Equal contribution 1 School of Computing and Augmented Intelligence, Arizona State University, Tempe, AZ, USA 2 Department of Computer Science and Engineering, Texas A&M University, College Station, TX, USA 3 School of Computing, Stevens Institute of Technology. Correspondence to: Chengshuai Zhao <[email protected]>.
Preprint. August 3, 2026.
1
TextCloak: Thwarting Unauthorized LLM Exploitation via RL-Driven Unlearnable Examples
2. Related Work
Existing textual UEs methods are predominantly tailored for closed-set classification tasks (e.g., sentiment analysis or topic categorization) in pre-trained language models and rely heavily on injecting static class-specific linguistic cues or surface-level triggers (Li et al., 2023; Wallace et al., 2019). However, modern LLM fine-tuning primarily focuses on instruction following and reasoning tasks, where explicit class labels do not exist. As a result, existing classification-bound text protection techniques fail to generalize, leaving textual data vulnerable to unauthorized LLM exploitation. Moreover, textual data is semantically rich and coherent. Discrete textual edits via character substitutions or trigger insertion may alter meaning or produce conspicuous artifacts, which potentially undermine the utility of the text for legitimate users. A practical defense must therefore preserve semantic fidelity and linguistic naturalness.
2.1. Unlearnable Examples Unlearnable examples (Huang et al., 2021), which inject small perturbations into training data such that models trained on the protected samples exhibit degraded utility, were first introduced in computer vision. Subsequent studies have improved the robustness and practicality of UEs. For example, robust error-minimizing perturbations were developed to resist adversarial training (Fu et al., 2022), while transferable UEs aim to maintain their protective effect across model architectures, optimization procedures, and datasets (Ren et al., 2023). Other extensions relax the requirement that the defender and unauthorized trainer use identical class labels (Zhang et al., 2023). Recently, efforts have attempted to extend UEs to textual data. For instance, Li et al. (2023) crafts unlearnable text through gradientguided token search and subsequently extracts reusable surface patterns from optimized examples. RegText (Java et al., 2024) similarly introduces spurious correlations through task-representative, low-frequency words to reduce the generalization of pre-trained language models. In the light of these pioneering works, TextCloak firstly explores unlearnable examples in the context of LLMs, focusing on natural language understanding and reasoning tasks.
To bridge this gap, we propose TextCloak, a RL-driven framework designed to protect textual data against unauthorized LLM fine-tuning. Unlike existing heuristic approaches, TextCloak formulates unlearnable text generation as a constrained bi-level optimization problem. Specifically, we design a generative policy to transform batches of clean text into unlearnable variants. The policy operates directly in natural language space, allowing it to preserve the linguistic quality. To optimize this generative policy, we further introduce GRPO-UE, which explicitly considers the model fine-tuning process and dynamically measures the downstream performance degradation induced by the generated unlearnable text. Utilizing this degradation as a reward signal, GRPO-UE updates the generator parameters via grouprelative policy optimization, enabling it to discover effective unlearnable patterns that impair LLM generalization ability. Our main contributions are summarized as follows:
2.2. Data Poisoning in LLMs Data poisoning attacks manipulate a model’s training corpus to alter its learned behavior (Steinhardt et al., 2017). Earlier poisoning studies commonly focused on classification models, where attackers modify labels or insert trigger features to induce targeted prediction errors (Fan et al., 2022). The increasing use of web-scale pre-training and instruction tuning has expanded this threat to LLMs. Poisoning instruction-tuning data can associate particular concepts or phrases with attacker-selected behaviors that transfer across multiple downstream tasks (Wan et al., 2023). Instructions themselves can also act as backdoor triggers, enabling adversaries to manipulate models without directly modifying individual input instances or their labels (Xu et al., 2024). Soft prompt injection further demonstrates that a small number of poisoned instruction–response pairs can implant persistent, context-dependent behavior while largely preserving performance on benign inputs (Yan et al., 2024). Collectively, these findings illustrate the sensitivity of LLM fine-tuning to carefully constructed training examples. Instead of exploiting this sensitivity for malicious purposes, TextCloak empowers data owners to proactively protect the text before release.
• Problem formulation. We identify the need to protect textual data in the era of LLMs and formalize it as a constrained bi-level optimization problem. • Novel framework. We introduce TextCloak, a RLdriven framework that crafts unlearnable text while preserving semantic fidelity and linguistic naturalness for legitimate use. To the best of our knowledge, TextCloak is the first framework to leverage unlearnable examples to defend against unauthorized LLM fine-tuning. • Optimization mechanism. We propose GRPO-UE, which dynamically measures protection through the downstream degradation and directly guides the generative policy to discover effective unlearnable patterns. • Comprehensive evaluation. We provide extensive evaluations on six public datasets and nine state-of-theart LLMs, demonstrating the effectiveness, transferability, robustness, and broad applicability of TextCloak.
2.3. Defenses Against Unauthorized LLM Exploitation Existing safeguards against unauthorized LLM exploitation include usage control, post-hoc attribution, model-side reme2
TextCloak: Thwarting Unauthorized LLM Exploitation via RL-Driven Unlearnable Examples
diation, and proactive data protection. Access restrictions, licenses, and crawler exclusion policies express contentowner preferences but depend on voluntary compliance (Jayaraman et al., 2026). Watermarking (Liu et al., 2024a; Zhang et al., 2024; Lau et al., 2024) instead embeds identifiable signals into protected content, enabling owners to test whether their data influenced a potentially unauthorized model. Machine unlearning seeks to remove the influence of sensitive or copyrighted data from trained LLMs (Cao & Yang, 2015; Yao & Xu, 2024), but requires cooperation and model access from the developer and may degrade retained knowledge. Proactive protection instead intervenes before data release, giving content owners direct control over the learnability of their text. Recent effort (Liu et al., 2024b) introduces minimally perceptible perturbations to reduce memorization and instance-level membership inference attack. TextCloak complements this line of work by developing a data-centric solution to thwart unauthorized LLM exploitation.
generalization of models trained on them. Formally, let xi = (wi,1 , . . . , wi,Ti ) be a token sequence and yi be its class label. A textual modification ηi = (pi , si ) replaces the token at position pi with a candidate token si from the vocabulary and acquiesces the modified sequence xi ⊕ ηi . The admissible modifications A(xi ) are often constrained by an edit budget. Textual UEs can be constructed by solving the following bi-level min-min optimization problem:
min θ
N 1 X min ℓ fθ (xi ⊕ ηi ), yi . N i=1 ηi ∈A(xi )
(3)
The model parameters and textual modifications are optimized alternately. Because token replacement is nondifferentiable, first-order gradients from a surrogate model are used to guide the search for effective modifications: ⊤ s⋆i = arg min e(s) − e(wi,pi ) ∇e(wi,pi ) ℓ fθ (xi ), yi , s∈V
(4) where V is the vocabulary and e(·) denotes the token embedding in pre-trained language models.
3. Preliminaries 3.1. Problem Formulation Let D = {(xi , yi )}N i=1 denote a clean text corpus, where xi is an input or instruction and yi is its target response sequence. A data owner applies a protection mechanism to e = {(e xi , yi )}N obtain D i=1 before releasing the corpus. The protected pair should convey the same information as the original one, as well as remain fluent and natural, so that the released text retains its utility for legitimate readers.
4. The Proposed TextCloak 4.1. Overview We propose TextCloak, an RL-driven framework that protects text from unauthorized LLM exploitation and comprises three key components. First, TextCloak leverages a generative policy that rewrites a clean corpus into semanticspreserving unlearnable text. Second, GRPO-UE measures the modified candidates by fine-tuning surrogate LLMs and evaluating the resulting degradation on held-out clean data. The candidates generated from the same batch form a comparison group, allowing their degradation scores to be converted into relative advantages without a learned value model. Third, a constrained bi-level loop alternates between inner surrogate fine-tuning and outer policy to update the generative policy parameters. The overall framework is illustrated in Figure 2.
e and We consider an unauthorized trainer who collects D fine-tunes an LLM fθ for a specific task by minimizing the standard autoregressive training loss: e θ⋆ = arg min Lft (fθ ; D).
(1)
θ
The data owner seeks protection that degrades the generalization of trained models. Let Deval be held-out evaluation data and let Leval measure downstream error, with larger values indicating worse utility. The protection objective is e ⋆ = arg max Leval fθ⋆ ; Deval , D (2)
4.2. UE Generative Policy
e D∈C(D)
where C(D) is the set of admissible corpora satisfying constraints (e.g., semantic fidelity and linguistic naturalness). This formulation captures the central trade-off: protected text should impair models trained on it without explicitly changing the content presented to legitimate users.
4.2.1. BATCH - LEVEL GENERATION . Let B = {(xi , yi )}bi=1 denote a mini-batch of clean instruction-response pairs. We define a generative policy πϕ that transforms B into a protected batch Be = {(e xi , yi )}bi=1 , where each (e xi , yi ) is a semantics-preserving rewrite of (xi , yi ). The policy is parameterized by ϕ and is implemented as an autoregressive large language model to operate directly in natural language space. The generation process is conditioned on both the clean batch and an editing instruction p that specifies the tasks and requirements as detailed
3.2. Unlearnable Examples for Text Classific textual UEs (Li et al., 2023) modify discrete token sequences, injecting task-specific shortcuts (e.g., lexical patterns correlated with class labels) to degrade the 3
TextCloak: Thwarting Unauthorized LLM Exploitation via RL-Driven Unlearnable Examples
UE Generative Policy K candidates
Editing Instruction ( ) Clean Batch
GRPO-UE Policy Update
Reward
Semantic Fidelity
Bilevel Optimization Goal
Group Advantage
Naturalness
UE Generative Policy ( )
Bilevel Objective
GRPO Objective
......
Sample
Surrogate LLM (
Upper-level
Clean Data Evaluation
)
Lower-level Update
Surrogate LLM (
Generate unlearnable examples that degrade unauthorized finetuning while preserving utility.
)
Lengend
Held-out Clean Data
Clean Dataset
Clean Dataset
Clean Batch
Fine-tuning
Protected Batch
Utility Metric
Figure 2. Overview of the proposed TextCloak.
in Appendix C.1. Formally, we have Be ∼ πϕ (· | p, B).
computed as (
) T 1X PPL(z) = exp − log pω (wt | w<t ) , T t=1 b PPL(ui ) 1X e min 1, . Sppl (B, B) = b i=1 PPL(e ui )
(5)
We note that the policy transforms the entire batch in one shot. Conditioning on the batch allows the policy to introduce more generalizable patterns across instances. Let a = (a1 , . . . , aT ) denote the token sequence in a candidate batch and let s = (p, B). Its probability factorizes as
πϕ (a | s) =
T Y
πϕ (at | s, a<t ).
(8)
We constrain the perplexity distance above a threshold to e ≥ τppl . maintain the linguistic naturalness: Sppl (B, B)
(6)
4.3. Group-Relative Policy Optimization for UEs
t=1
To optimize the generative policy, we introduce GRPO-UE. Classic unlearnable examples are optimized by minimizing the training loss of a surrogate model on the protected data. However, this doesn’t directly reflect the downstream degradation, which is the ultimate goal of data protection. Therefore, we simulate the unauthorized LLM fine-tuning process and measure the resulting degradation on held-out clean data, which provides an effective signal for policy optimization.
4.2.2. S EMANTIC FIDELITY. We measure the semantic fidelity of the protected batch and its clean counterpart using Sentence-BERT (SBERT) (Reimers & Gurevych, 2019) embeddings. For each instruction-response pair (xi , yi ), let ui = xi ∥ yi and u ei = x ei ∥ yi , where ∥ denotes sequence concatenation. We compute the batch-level semantic fidelity as b X e =1 Ssem (B, B) cos ϵ(ui ), ϵ(e ui ) . b i=1
4.3.1. G ROUP SAMPLING . (7)
For each clean batch B, the policy πϕroll rolls out K times and formulate a group of candidate batches {Be(k) }K k=1 . All candidates share the same clean context and are generated independently, so they explore alternative protective patterns while remaining directly comparable. Let gψ0 ∼ S denote a surrogate drawn from the defender’s surrogate distribution. For each candidate k, we initialize an independent surrogate copy at ψ0 and apply the prescribed inner-loop fine-tuning procedure: ψk⋆ = arg min Lft gψ ; Be(k) . (9)
where ϵ(·) denotes the SBERT encoder and cos(·, ·) denotes cosine similarity. To preserve the semantic content of the e ≥ τsem . original text, we require Ssem (B, B) 4.2.3. L INGUISTIC NATURALNESS . We quantify linguistic naturalness using perplexity under pre-trained GPT-2 (Radford et al., 2019) pω . For a tokenized sequence z = (w1 , . . . , wT ), the batch-level perplexity is
ψ
4
TextCloak: Thwarting Unauthorized LLM Exploitation via RL-Driven Unlearnable Examples
Similarly, we can also obtain a clean baseline gψcl⋆ by applying the same fine-tuning procedure to B. Sharing the clean batch, surrogate initialization, and inner optimization schedule isolates the effect of each candidate rewrite.
4.4. Constrained Bi-level Optimization Overall, TextCloak solves the constrained bi-level optimization problem to balance the trade-off between unlearnability and utility. The outer problem maximizes the expected degradation on held-out clean data, while the inner problem simulates the unauthorized fine-tuning process. The semantic fidelity and linguistic naturalness constraints ensure that the protected text remains usable for legitimate readers:
4.3.2. R EWARD DESIGN . We evaluate every fine-tuned surrogate on a clean held-out set Dval . The degradation induced by candidate k can be quantified as: dk = Leval (gψk⋆ ; Dval ) − Leval (gψcl⋆ ; Dval ).
(10)
where [u]+ = max(u, 0), and λsem and λppl control the semantic and naturalness penalties, respectively. 4.3.3. P OLICY UPDATE . GRPO-UE adapts group-relative policy optimization (GRPO) (Shao et al., 2024) as base RL framework for UEs. For the K candidate batches, we compute
K −1
PK
r = K −1
2 j=1 (rj − r) + ϵ
K X
πϕ (ak,t | s, ak,<t ) . πϕroll (ak,t | s, ak,<t )
rj .
j=1
(13)
The policy maximizes the clipped objective JGRPO (ϕ) =
(15)
Algorithm 1 Constrained bi-level optimization of TextCloak Input: Clean corpus D, validation set Dval , initial policy πϕ , reference policy πref , and surrogate distribution S Parameters: Group size K, thresholds τsem and τppl , and penalty weights λsem and λppl e Output: Protected corpus D 1: repeat 2: Sample B ∼ D and gψ0 ∼ S. 3: Set πϕroll ← πϕ . 4: Sample {Be(k) }K k=1 from πϕroll (· | p, B). 5: Fine-tune a clean baseline gψcl⋆ on B. 6: for k = 1, . . . , K do (k) (k) 7: Compute Ssem and Sppl . 8: Fine-tune gψk⋆ on Be(k) . 9: Compute dk and rk using Equations 10 and 11. 10: end for bk }K using Equation 12. 11: Compute {A k=1 12: Update ϕ by maximizing JGRPO in Equation 14. 13: until the policy satisfies the stopping criterion e with πϕ and retain outputs satisfying both 14: Generate D utility thresholds. e 15: return D
(12) Given the rollout policy, the token-level importance ratio is ρk,t (ϕ) =
e ψ ⋆ = arg min Lft (gψ ; B),
We optimize Equation 15 by alternating three steps described in Algorithm 1. First, the current policy generates K candidate rewrites for each clean batch. Second, SBERT similarity and the normalized perplexity score measure their utility, while fresh surrogate copies are fine-tuned on the candidates and evaluated on clean held-out data. Third, Equations 11-14 convert the resulting degradation and utility scores into group-relative advantages and update the policy. Once training is complete, protection requires only one forward pass.
(k)
,
s.t.
e ≥ τsem , Ssem (B, B) e ≥ τppl . Sppl (B, B)
(k) rk = dk − λsem [τsem − Ssem ]+ − λppl [τppl − Sppl ]+ , (11)
rk − r
E [Leval (gψ⋆ ; Dval )] ψ
To prevent the policy from increasing degradation by changing the meaning or producing unnatural text, we com(k) bine dk with the two utility measurements. Let Ssem = (k) (k) (k) Ssem (B, Be ) and Sppl = Sppl (B, Be ). The reward is
bk = q A
max ϕ
Thus, dk > 0 indicates that training on the protected candidate causes more held-out error than training on the corresponding clean batch. We observe that the Leval (gψcl⋆ ; Dval ) is a constant for all candidates in the same group, so it can be omitted from the reward computation in practice.
Tk K 1 X 1 X ℓk,t − βDKL (πϕ ∥πref ), K Tk t=1 k=1
bk , ρk,t A bk }, ℓk,t = min{ρk,t A ρk,t = clip(ρk,t , 1 − ε, 1 + ε), (14) where πref is the initial reference policy. Clipping limits abrupt policy changes, and length normalization prevents long rewrites from dominating the update. The KL term measures the average token-level divergence across the sampled sequences, thereby discouraging the policy from drifting away from fluent natural-language generation. 5
TextCloak: Thwarting Unauthorized LLM Exploitation via RL-Driven Unlearnable Examples
5. Experiments
primary metric is the absolute performance drop ∆ after fine-tuning on the protected corpus. We repeat experiments with three random seeds and report the average.
5.1. Experimental Setup 5.1.1. DATASETS .
∆ = Mclean − Mprotected ,
We consider six representative public-available datasets including ARC-Challenge (Clark et al., 2018), MATH (Hendrycks et al., 2021), MMLU-Pro (Wang et al., 2024), RACE (Lai et al., 2017), HumanEval (Chen et al., 2021), and MedQA (Jin et al., 2021), which covers diverse domains and tasks as summarized in Table 1. Detailed information about the datasets is provided in Appendix A.1.
5.1.5. I MPLEMENTATION D ETAILS . The generative policy is initialized from Llama-3-8B and is optimized for 2 epochs. We set the group size to 4. The inner-loop surrogate utilizes Qwen3-8, fine-tuned with LoRA-8. The constraints are τsem and τppl as 0.9, with penalty weights λsem and λppl as 1.0. To accelerate the experiments, we run on 8 NVIDIA A100 GPUs with a batch size of 8 per device, which costs ∼40s per batch. More implementation details are provided in Appendix A.3.
Table 1. Statistics of the evaluation datasets. Dataset
Task
Size
ARC-Challenge MATH MMLU-Pro RACE HumanEval MedQA-USMLE
Commonsense reasoning Mathematical reasoning Multitask language understanding Reading comprehension Code generation Medical question answering
2,590 12,500 12,032 97,687 164 12,723
(16)
5.2. Protection Effectiveness RQ1: Can TextCloak reduce the performance of unauthorized LLMs while preserving utility? We compare TextCloak with baselines on six datasets, which are summarized in Table 2. Overall, TextCloak consistently outperforms the baselines across all datasets, achieving the largest average performance drop ∆ while maintaining high semantic fidelity and a normalized perplexity score close to one. Notably, TextCloak achieves an average performance drop of 12.4%, which is significantly higher than the best baseline. Interestingly, TextCloak impairs the performance of unauthorized LLMs below the zero-shot baseline on some datasets (e.g., HEval), indicating that the protected text not only prevents learning from the data but also actively misleads the model.
5.1.2. LLM BACKBONES . We evaluate nine state-of-the-art LLMs spanning multiple families and sizes: Qwen3-4B and Qwen3-14B (Yang et al., 2025), Gemma-3-12B-IT (Team, 2025), Mistral-7BInstruct-v0.3 (Jiang et al., 2023), GPT-OSS-20B (Agarwal et al., 2025), Llama-3.2-3B and Llama-3.1-8BInstruct (Grattafiori et al., 2024), Phi-4 (Abdin et al., 2024), and GLM-4-9B-Chat (Glm et al., 2024). 5.1.3. BASELINES .
5.3. Transfer Experiments
Due to the lack of existing methods for defending against unauthorized LLM fine-tuning, we extend related works on UEs to our setting. Specifically, we formulate the following baselines: (i) Zero-Shot evaluates the target LLMs on clean test data without any fine-tuning. (ii) Clean fine-tunes each target LLM on the original examples and establishes the utility attainable without protection. (iii) Random-Prepend and Random-Append prepend or append five tokens sampled uniformly from vocabulary of Llama-3-8B. (iv) Textual UE (Li et al., 2023) leverages error-minimizing perturbations to construct unlearnable examples for text classification in pretrained LLMs through token replacement. (v) MEM-3 and MEM-5 (Liu et al., 2024c) insert optimized text triggers of three and five tokens, respectively, to prevent generalization in multimodal contrastive learning.
RQ2: Does the protection transfer across unseen LLM architectures and fine-tuning configurations? 5.3.1. T RANSFER ACROSS LLM A RCHITECTURES . Table 3 evaluates the transferability of TextCloak across different LLM architectures. We can observe that TextCloak consistently achieves satisfactory transferability across unseen LLMs. Notably, TextCloak largely degrades the performance of Phi-4 and Qwen3 models on most datasets, indicating the good applicability of TextCloak for SOTA LLM families. However, the transferability of TextCloak is limited since the error-minimized text perturbations are specific to the surrogate.
5.1.4. E VALUATION M ETRICS .
5.3.2. T RANSFER ACROSS F INE - TUNING C ONFIGURATIONS .
For downstream performance, we use accuracy for ARCChallenge, MMLU-Pro, RACE, and MedQA-USMLE; exact match for MATH; and pass@1 for HumanEval. Our
We also evaluate the transferability across different finetuning configurations, including LoRA-64, QLoRA, DoRA, soft-prompt, full fine-tuning (FFT), as shown in Fig6
TextCloak: Thwarting Unauthorized LLM Exploitation via RL-Driven Unlearnable Examples Table 2. Protection performance and data quality across six datasets. Parentheses report the performance drop ∆ from clean fine-tuning; larger ∆ indicates stronger protection. SBERT and PPL are averaged across datasets. Bold and underlined task-performance values denote the lowest and second-lowest results, excluding Zero-Shot and Clean. Task Performance (%) Method
ARC
MATH
MMLU
RACE
Data Quality
HEval
Zero-Shot 78.7 (+13.2) 20.7 (+17.8) 36.3 (+12.8) 76.6 (+12.7) Clean 91.9 (0.0) 38.5 (0.0) 49.1 (0.0) 89.3 (0.0) Random-Prepend 91.3 (+0.6) 38.1 (+0.4) 49.6 (-0.5) 88.0 (+1.3) Random-Append 91.2 (+0.7) 36.6 (+1.9) 47.9 (+1.2) 87.6 (+1.7) Textual UE 91.4 (+0.5) 38.3 (+0.2) 47.4 (+1.7) 87.1 (+2.2) MEM-3 90.3 (+1.6) 38.9 (-0.4) 50.5 (-1.4) 87.2 (+2.1) MEM-5 90.8 (+1.1) 36.5 (+2.0) 49.9 (-0.8) 88.3 (+1.0) TextCloak 86.6 (+5.3) 34.5 (+4.0) 17.7 (+31.4) 83.8 (+5.5)
MedQA
56.3 (+18.8) 75.0 (0.0) 65.6 (+9.4) 65.6 (+9.4) 56.3 (+18.8) 56.3 (+18.8) 62.5 (+12.5) 53.1 (+21.9)
Avg.
SBERT↑ PPL↓
63.6 (+3.7) 55.4 (+13.2) 67.3 (0.0) 68.5 (0.0) 63.9 (+3.4) 66.1 (+2.4) 62.8 (+4.5) 65.3 (+3.2) 63.4 (+3.9) 64.0 (+4.5) 64.7 (+2.6) 64.6 (+3.9) 63.2 (+4.1) 65.2 (+3.3) 61.1 (+6.2) 56.1 (+12.4)
1.00 1.00 0.91 0.91 0.92 0.91 0.87 0.95
18.7 18.7 29.7 28.7 23.1 27.1 34.0 20.1
Table 3. Transferability of protection across unseen LLM architectures. Parentheses report the performance drop ∆. Bold and underlined values denote the lowest and second-lowest results. ARC-Challenge Method
GPT-20B
Llama-8B
Phi-4
Zero-Shot Clean Random-Prepend Random-Append Textual UE MEM-3 MEM-5 TextCloak
21.6 (+64.6) 86.2 (0.0) 79.7 (+6.5) 87.8 (-1.6) 84.5 (+1.7) 82.3 (+3.8) 87.6 (-1.5) 80.1 (+6.1)
72.7 (+1.5) 74.2 (0.0) 54.1 (+20.1) 51.2 (+23.0) 71.7 (+2.6) 52.4 (+21.8) 51.7 (+22.5) 59.2 (+15.0)
18.6 (+46.3) 64.9 (0.0) 48.0 (+16.9) 44.4 (+20.5) 62.6 (+2.3) 55.8 (+9.1) 53.8 (+11.0) 19.0 (+45.8)
MATH Qwen-14B
Llama-8B
Qwen-4B
78.8 (+13.1) 14.9 (+25.9) 92.0 (0.0) 40.8 (0.0) 88.5 (+3.5) 42.3 (-1.5) 83.1 (+8.9) 41.9 (-1.1) 92.4 (-0.4) 40.9 (-0.1) 87.2 (+4.8) 41.3 (-0.5) 87.7 (+4.3) 41.5 (-0.7) 84.8 (+7.2) 33.9 (+6.9)
21.5 (+8.1) 29.6 (0.0) 28.9 (+0.7) 26.7 (+2.9) 28.7 (+0.9) 28.5 (+1.1) 28.9 (+0.7) 26.1 (+3.5)
19.0 (+7.1) 13.5 (+32.9) 26.1 (0.0) 46.5 (0.0) 28.3 (-2.2) 46.4 (+0.1) 23.1 (+2.9) 43.5 (+2.9) 27.1 (-1.0) 46.5 (-0.1) 24.9 (+1.2) 44.9 (+1.5) 25.1 (+1.0) 46.7 (-0.3) 22.3 (+3.8) 42.9 (+3.6)
RACE GPT-20B
Mistral-7B
Phi-4
Qwen-4B
Gemma-12B Llama-8B
Zero-Shot Clean Random-Prepend Random-Append Textual UE MEM-3 MEM-5 TextCloak
22.3 (+63.1) 85.3 (0.0) 85.2 (+0.2) 82.5 (+2.9) 85.0 (+0.4) 85.1 (+0.2) 86.0 (-0.7) 70.8 (+14.5)
76.8 (+0.8) 77.6 (0.0) 77.8 (-0.2) 78.3 (-0.7) 75.4 (+2.2) 77.0 (+0.6) 76.8 (+0.8) 76.3 (+1.3)
11.2 (+55.7) 66.9 (0.0) 78.7 (-11.8) 73.9 (-7.0) 83.1 (-16.2) 73.2 (-6.2) 80.2 (-13.2) 48.0 (+18.9)
81.5 (+2.6) 84.1 (0.0) 83.7 (+0.4) 80.9 (+3.2) 84.4 (-0.3) 83.9 (+0.2) 84.0 (+0.1) 72.0 (+12.1)
87.5 (+9.4) 96.9 (0.0) 81.3 (+15.6) 87.5 (+9.4) 84.4 (+12.5) 87.5 (+9.4) 84.4 (+12.5) 78.1 (+18.8)
Task Performance
59.4 (+6.2) 65.6 (0.0) 56.3 (+9.4) 43.8 (+21.9) 56.3 (+9.4) 59.4 (+6.2) 62.5 (+3.1) 40.6 (+25.0)
LoRA-64
QLoRA
GLM-9B
DoRA
Soft Prompt
Fine-tuning Configurations
GPT-20B
28.1 (+4.8) 20.7 (+27.9) 32.9 (0.0) 48.6 (0.0) 32.5 (+0.3) 49.8 (-1.2) 30.9 (+2.0) 44.7 (+3.9) 32.1 (+0.7) 48.5 (+0.1) 33.3 (-0.4) 49.3 (-0.7) 33.2 (-0.3) 48.7 (-0.1) 27.9 (+5.0) 23.1 (+25.5)
Phi-4
Qwen-14B
8.1 (+18.2) 26.3 (0.0) 29.9 (-3.5) 28.3 (-2.0) 46.6 (-20.3) 36.8 (-10.5) 35.5 (-9.1) 20.1 (+6.3)
44.5 (+11.1) 55.6 (0.0) 56.1 (-0.5) 12.3 (+43.3) 53.5 (+2.1) 56.3 (-0.7) 55.5 (+0.1) 16.3 (+39.3)
MedQA-USMLE
Phi-4
Qwen-4B
GLM-9B
71.9 (+9.4) 81.3 (0.0) 78.1 (+3.1) 78.1 (+3.1) 78.1 (+3.1) 75.0 (+6.3) 78.1 (+3.1) 75.0 (+6.3)
62.5 (+15.6) 78.1 (0.0) 37.5 (+40.6) 62.5 (+15.6) 37.5 (+40.6) 37.5 (+40.6) 40.6 (+37.5) 31.3 (+46.9)
45.6 (+7.9) 17.5 (+44.2) 53.6 (0.0) 61.7 (0.0) 53.2 (+0.4) 53.2 (+8.5) 51.0 (+2.6) 52.8 (+8.9) 53.3 (+0.2) 63.2 (-1.5) 53.2 (+0.4) 52.9 (+8.8) 53.5 (+0.1) 55.9 (+5.7) 47.3 (+6.3) 53.0 (+8.6)
Phi-4
Qwen-4B
Qwen-14B
47.8 (+9.2) 57.0 (0.0) 53.6 (+3.4) 50.2 (+6.8) 46.7 (+10.2) 53.8 (+3.1) 53.6 (+3.4) 46.5 (+10.5)
64.3 (+3.1) 67.5 (0.0) 68.4 (-0.9) 65.4 (+2.1) 64.8 (+2.7) 67.3 (+0.2) 67.2 (+0.2) 63.2 (+4.3)
reward, utility constraints, and optimization algorithms in Table 4. We can find that removing the degradation reward and GRPO-UE significantly reduces the performance drop, while abating the semantic and naturalness constraints leads to a higher performance drop but lower language quality. This indicates that each component of TextCloak is essential for achieving a balance between protection and utility.
Clean Data TextCloak
Default
Qwen-14B
HumanEval
Method
1.0 0.8 0.6 0.4 0.2
MMLU-Pro
GPT-20B
FFT
Figure 3. Transferability across fine-tuning configurations.
ure 3. Overall, TextCloak demonstrates strong transferability across various fine-tuning methods. For instance, TextCloak achieves the largest performance degradation on soft prompt while remaining relatively less effective on FFT, which is because fine-tuning with larger parameter updates unlocks more capacity and leads to better adaptation to the protected text.
Table 4. Ablation study of TextCloak. Values in parentheses denote absolute differences from the full method. Variant
∆↑
TextCloak 54.7 (-0.0) w/o degradation reward 9.0 (-45.7) w/o semantic constraint 34.3 (-20.4) w/o naturalness constraint 17.7 (-37.0) w/o GRPO-UE 7.2 (-47.5)
5.4. Ablation Study RQ3: How do the components of TextCloak contribute to its performance? We isolate the contribution of the degradation 7
SBERT↑
PPL↓
0.91 (-0.00) 7.0 (+0.0) 0.90 (-0.02) 8.5 (+1.5) 0.77 (-0.15) 7.2 (+0.3) 0.86 (-0.05) 14.0 (+7.0) 0.85 (-0.06) 7.9 (+0.9)
TextCloak: Thwarting Unauthorized LLM Exploitation via RL-Driven Unlearnable Examples
5.5. Robustness Analysis
model. At the same time, the protected text remains semantically faithful and linguistically natural, demonstrating the practical effectiveness of TextCloak in preventing unauthorized LLM fine-tuning.
RQ4: How robust is TextCloak against various defense strategies? Unauthorized trainers may attempt to invalidate the protection by transforming the released corpus (e.g., lowercase normalization, punctuation removal, whitespace stripping, and paraphrasing) or adapting adversarial training (AT) techniques, as shown in Figure 4. Generally, the protected text maintains its effectiveness against various defense approaches. Among them, AT is the most effective while punctuation is the least effective, suggesting that TextCloak provides strong resistance beyond surface-level modifications.
Task Performance
1.0 0.8 0.6 0.4 0.2
Original Data Question:
Question:
Which of the following is an example
Which listed event changes a material
of a physical change?
physically without producing a new
Choice:
chemical substance?
[ lighting a match, breaking a glass,
Choice:
burning of gasoline, rusting of iron ]
[ lighting a match, breaking a glass,
Answer:
burning of gasoline, rusting of iron ]
Breaking a glass
....
Clean Data TextCloak
None
Protected Data
Reasoning Cue
Figure 5. Case study.
Lowercase Punctuation Whitespace Paraphrase
Defense Methods
AT
6. Conclusion
Figure 4. Task performance under various defense strategies.
We presented TextCloak, an RL-driven framework for protecting textual data against unauthorized LLM fine-tuning. Its generative policy produces semantically faithful and natural unlearnable text, while GRPO-UE directly optimizes the policy using downstream degradation measured on finetuned surrogate models. Experiments across six datasets and nine LLMs indicate that TextCloak consistently impairs unauthorized LLM fine-tuning, transfers across model architectures and training configurations, and remains robust under adaptive defense strategies. These results demonstrate the potential of RL-guided unlearnable text as a practical, model-transferable approach for proactive data protection. Future work will develop sophisticated UEs in high-stakes domains.
5.6. Human Evaluation Table 5. Human evaluations under three-scaled rating. Scores are averaged across human annotators and LLM judges. Method
Naturalness↑
Fluency↑
Human Utility↑
Clean Textual UE MEM-5 TextCloak
2.92 ± 0.07 2.25 ± 0.12 1.35 ± 0.56 2.52 ± 0.08
2.93 ± 0.05 2.03 ± 0.27 1.57 ± 0.31 2.84 ± 0.11
2.99 ± 0.01 1.65 ± 1.19 1.94 ± 0.28 2.68 ± 0.28
RQ5: Do the protected examples preserve language quality and overall utility for legitimate users? We sample 20 samples from each dataset and then employ three human annotators and three LLM judges to evaluate the naturalness, fluency, and utility of the protected text. The rating rubrics are provided in Appendix B.1. As showcased in Table 5, TextCloak achieves comparable scores to the clean examples and significantly outperforms the SOTA UEs across all three metrics. Significantly, the protected text generated by TextCloak remains similar fluency and human utility to the original text, indicating that the protected text is practically usable while less compromising the user experience.
References Abdin, M., Aneja, J., Behl, H., Bubeck, S., Eldan, R., Gunasekar, S., Harrison, M., Hewett, R. J., Javaheripi, M., Kauffmann, P., et al. Phi-4 technical report. arXiv preprint arXiv:2412.08905, 2024. Agarwal, S., Ahmad, L., Ai, J., Altman, S., Applebaum, A., Arbus, E., Arora, R. K., Bai, Y., Baker, B., Bao, H., et al. gpt-oss-120b & gpt-oss-20b model card. arXiv preprint arXiv:2508.10925, 2025. Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. Language models are few-shot learners. Advances in neural information processing systems, 33: 1877–1901, 2020.
5.7. Case Study RQ6: What are the qualitative characteristics of the protected examples and their impact on downstream model behavior? Figure 5 presents representative examples of protected text generated by TextCloak. TextCloak converts the original text into UE by introducing instance-specific thinking or reasoning shortcuts (e.g., semantic hints) such that disregarding the generalization ability of the downstream
Cao, Y. and Yang, J. Towards making systems forget with machine unlearning. In 2015 IEEE symposium on security and privacy, pp. 463–480. IEEE, 2015. 8
TextCloak: Thwarting Unauthorized LLM Exploitation via RL-Driven Unlearnable Examples
Carlini, N., Tramer, F., Wallace, E., Jagielski, M., HerbertVoss, A., Lee, K., Roberts, A., Brown, T., Song, D., Erlingsson, U., et al. Extracting training data from large language models. In 30th USENIX security symposium (USENIX Security 21), pp. 2633–2650, 2021.
Jiang, A. Q., Sablayrolles, A., Mensch, A., Bamford, C., Chaplot, D. S., Casas, D. d. l., Bressand, F., Lengyel, G., Lample, G., Saulnier, L., et al. Mistral 7b. arXiv preprint arXiv:2310.06825, 2023. Jin, D., Pan, E., Oufattole, N., Weng, W.-H., Fang, H., and Szolovits, P. What disease does this patient have? a large-scale open domain question answering dataset from medical exams. Applied Sciences, 11(14):6421, 2021.
Chen, M., Tworek, J., Jun, H., Yuan, Q., Pinto, H. P. D. O., Kaplan, J., Edwards, H., Burda, Y., Joseph, N., Brockman, G., et al. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374, 2021.
Kandpal, N., Wallace, E., and Raffel, C. Deduplicating training data mitigates privacy risks in language models. In International Conference on Machine Learning, pp. 10697–10707. PMLR, 2022.
Clark, P., Cowhey, I., Etzioni, O., Khot, T., Sabharwal, A., Schoenick, C., and Tafjord, O. Think you have solved question answering? try arc, the ai2 reasoning challenge. arXiv preprint arXiv:1803.05457, 2018.
Lai, G., Xie, Q., Liu, H., Yang, Y., and Hovy, E. Race: Large-scale reading comprehension dataset from examinations. In Proceedings of the 2017 conference on empirical methods in natural language processing, pp. 785–794, 2017.
Fan, J., Yan, Q., Li, M., Qu, G., and Xiao, Y. A survey on data poisoning attacks and defenses. In 2022 7th IEEE International Conference on Data Science in Cyberspace (DSC), pp. 48–55. IEEE, 2022.
Lau, G. K. R., Niu, X., Dao, H., Chen, J., Foo, C.-S., and Low, B. K. H. Waterfall: Scalable framework for robust text watermarking and provenance for llms. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pp. 20432–20466, 2024.
Fu, S., He, F., Liu, Y., Shen, L., and Tao, D. Robust unlearnable examples: Protecting data against adversarial learning. arXiv preprint arXiv:2203.14533, 2022. Glm, T., Zeng, A., Xu, B., Wang, B., Zhang, C., Yin, D., Zhang, D., Rojas, D., Feng, G., Zhao, H., et al. Chatglm: A family of large language models from glm-130b to glm-4 all tools. arXiv preprint arXiv:2406.12793, 2024.
Li, D., Jiang, B., Huang, L., Beigi, A., Zhao, C., Tan, Z., Bhattacharjee, A., Jiang, Y., Chen, C., Wu, T., et al. From generation to judgment: Opportunities and challenges of llm-as-a-judge. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pp. 2757–2791, 2025.
Grattafiori, A., Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Vaughan, A., et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024.
Li, X., Liu, M., and Gao, S. Make text unlearnable: Exploiting effective patterns to protect personal data. In The Third Workshop on Trustworthy Natural Language Processing, pp. 249, 2023.
Hendrycks, D., Burns, C., Kadavath, S., Arora, A., Basart, S., Tang, E., Song, D., and Steinhardt, J. Measuring mathematical problem solving with the MATH dataset. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2), 2021. URL https://openreview.net/forum?id= 7Bywt2mQsCe.
Li, Z., Cai, J., Xu, G., Zheng, H., Li, Q., Zhou, F., Yang, S., Ling, C., and Wang, B. Versatile transferable unlearnable example generator. Advances in Neural Information Processing Systems, 38:17495–17522, 2026. Liu, A., Pan, L., Lu, Y., Li, J., Hu, X., Zhang, X., Wen, L., King, I., Xiong, H., and Yu, P. A survey of text watermarking in the era of large language models. ACM Computing Surveys, 57(2):1–36, 2024a.
Huang, H., Ma, X., Erfani, S. M., Bailey, J., and Wang, Y. Unlearnable examples: Making personal data unexploitable. In International Conference on Learning Representations, 2021. URL https://openreview.net/ forum?id=iAmZUo0DxC0.
Liu, R., Tran, T., Wang, T., Hu, H., Wang, S., and Xiong, L. Expshield: Safeguarding web text from unauthorized crawling and llm exploitation. arXiv preprint arXiv:2412.21123, 2024b.
Java, A., Shahid, S., and Agarwal, C. Towards operationalizing right to data protection. arXiv preprint arXiv:2411.08506, 2024.
Liu, X., Jia, X., Xun, Y., Liang, S., and Cao, X. Multimodal unlearnable examples: Protecting data against multimodal contrastive learning. In Proceedings of the 32nd ACM International Conference on Multimedia, pp. 8024–8033, 2024c.
Jayaraman, B., Marathe, V., Mozaffari, H., Shen, W., and Kenthapadi, K. Permissioned llms: Enforcing access control in large language models. Advances in Neural Information Processing Systems, 38:81743–81773, 2026. 9
TextCloak: Thwarting Unauthorized LLM Exploitation via RL-Driven Unlearnable Examples
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Gray, A., et al. Training language models to follow instructions with human feedback. In Advances in Neural Information Processing Systems, 2022.
Language Technologies (Volume 1: Long Papers), pp. 3111–3126, 2024. Yan, J., Yadav, V., Li, S., Chen, L., Tang, Z., Wang, H., Srinivasan, V., Ren, X., and Jin, H. Backdooring instructiontuned large language models with virtual prompt injection. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pp. 6065–6086, 2024.
Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I., et al. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9, 2019. Reimers, N. and Gurevych, I. Sentence-bert: Sentence embeddings using siamese bert-networks. In Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP), pp. 3982–3992, 2019.
Yang, A., Li, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Gao, C., Huang, C., Lv, C., et al. Qwen3 technical report. arXiv preprint arXiv:2505.09388, 2025. Yao, Y. and Xu, X. Large language model unlearning. Advances in Neural Information Processing Systems, 37: 105425–105475, 2024.
Ren, J., Xu, H., Wan, Y., Ma, X., Sun, L., and Tang, J. Transferable unlearnable examples. In The Eleventh International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id= -htnolWDLvP.
Zhang, J., Ma, X., Yi, Q., Sang, J., Jiang, Y.-G., Wang, Y., and Xu, C. Unlearnable clusters: Towards label-agnostic unlearnable examples. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3984–3993. IEEE Computer Society, 2023.
Shao, Z., Wang, P., Zhu, Q., Xu, R., Song, J., Bi, X., Zhang, H., Zhang, M., Li, Y., Wu, Y., et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300, 2024.
Zhang, R., Hussain, S. S., Neekhara, P., and Koushanfar, F. {REMARK-LLM}: A robust and efficient watermarking framework for generative large language models. In 33rd USENIX Security Symposium (USENIX Security 24), pp. 1813–1830, 2024.
Steinhardt, J., Koh, P. W. W., and Liang, P. S. Certified defenses for data poisoning attacks. Advances in neural information processing systems, 30, 2017.
Zhao, C., Tan, Z., Li, D., Yu, Z., and Liu, H. To see is not to learn: Protecting multimodal data from unauthorized finetuning of large vision-language model. arXiv preprint arXiv:2605.14291, 2026a.
Team, G. Gemma 3 technical report. CoRR, abs/2503.19786, 2025. Wallace, E., Feng, S., Kandpal, N., Gardner, M., and Singh, S. Universal adversarial triggers for attacking and analyzing nlp. In Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP), pp. 2153–2162, 2019.
Zhao, C., Tan, Z., Ma, P., Li, D., Jiang, B., Wang, Y., Yang, Y., and Liu, H. Is chain-of-thought reasoning of llms a mirage? A data distribution lens. In ACL (Findings), pp. 15231–15261. Association for Computational Linguistics, 2026b.
Wan, A., Wallace, E., Shen, S., and Klein, D. Poisoning language models during instruction tuning. In International Conference on Machine Learning, pp. 35413– 35425. PMLR, 2023. Wang, Y., Ma, X., Zhang, G., Ni, Y., Chandra, A., Guo, S., Ren, W., Arulraj, A., He, X., Jiang, Z., et al. Mmlu-pro: A more robust and challenging multi-task language understanding benchmark. Advances in Neural Information Processing Systems, 37:95266–95290, 2024. Xu, J., Ma, M., Wang, F., Xiao, C., and Chen, M. Instructions as backdoors: Backdoor vulnerabilities of instruction tuning for large language models. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human 10
TextCloak: Thwarting Unauthorized LLM Exploitation via RL-Driven Unlearnable Examples
A. Detailed Experiment Setting This appendix provides additional details for reproducing the experiments in the main paper. We focus on the experimental protocol used to evaluate whether protected text degrades unauthorized LLM fine-tuning while preserving the utility of the released corpus for legitimate users. Unless otherwise stated, the same data splits, prompts, metrics, and decoding configurations are used across TextCloak and all baselines. A.1. Datasets We evaluate TextCloak on six public datasets that cover factual reasoning, mathematical problem solving, multitask knowledge, reading comprehension, code generation, and medical question answering. The datasets are selected to stress different aspects of LLM fine-tuning. This diversity allows us to test whether the learned unlearnable patterns are tied to one narrow output format or remain effective across heterogeneous language tasks. ARC-Challenge (Clark et al., 2018) contains grade-school science questions that are difficult for retrieval-only or shallow pattern-matching systems. Each instance consists of a natural-language question and multiple answer options. We cast each example into an instruction-following format where the model is asked to choose the correct option. Accuracy is used as the downstream metric. MATH (Hendrycks et al., 2021) contains competition-style mathematical problems spanning algebra, geometry, probability, number theory, and related topics. Because answers are often short expressions or numbers, we evaluate MATH with exact match after standard answer normalization. This dataset tests whether protection remains effective for examples whose useful content depends on precise symbolic reasoning. MMLU-Pro (Wang et al., 2024) extends multitask language understanding evaluation with more challenging questions and a larger answer-option space than the original MMLU benchmark. We use it to evaluate broad-domain knowledge and reasoning. Each example is formatted as a multiple-choice instruction, and accuracy is reported. RACE (Lai et al., 2017) is a reading-comprehension benchmark collected from English examinations. Each example contains a passage, a question, and multiple answer candidates. We keep the passage and question together in the instruction field and evaluate the selected answer with accuracy. RACE is included because the protected text must preserve relatively long contextual passages HumanEval (Chen et al., 2021) evaluates functional code generation from natural-language programming prompts. Following the main paper, we report pass@1. This dataset tests whether unlearnable text can affect code-oriented fine-tuning where downstream success is determined by executable behavior. MedQA-USMLE (Jin et al., 2021) contains medical exam questions derived from the United States Medical Licensing Examination style. We formulate each item as a medical multiple-choice question and evaluate accuracy. Since this is a high-stakes domain, we use the dataset only as a benchmark for measuring unauthorized fine-tuning degradation. For all datasets, the protected corpus is generated only from training examples. Held-out validation data are used during GRPO-UE to compute the surrogate degradation reward, and final test data are reserved for evaluation of target LLMs. The target response yi is kept unchanged by all protection methods; TextCloak rewrites the input or instruction text x ei so that the released pair remains interpretable to human readers while impairing unauthorized LLM fine-tuning. A.2. Baselines We compare against clean fine-tuning, no fine-tuning, and several adapted textual unlearnable-example baselines. Because prior textual UE methods are primarily designed for classification or settings specific to other domains (e.g., segmentation, verification), we adapt them to the instruction-tuning in LLMs used in this paper while preserving their core mechanisms. Zero-Shot evaluates each target LLM directly on the clean test set without any task-specific fine-tuning. This baseline measures the capability already present in the model before it sees the released corpus. Clean fine-tunes each target LLM on the original unprotected training examples. Its score represents the performance an unauthorized trainer can obtain when the data owner releases the clean corpus. We use Clean as the reference for the performance drop ∆ reported in the main paper. Random-Prepend and Random-Append add five randomly sampled tokens to each training input. The tokens are sampled 11
TextCloak: Thwarting Unauthorized LLM Exploitation via RL-Driven Unlearnable Examples
from the Llama-3-8B tokenizer vocabulary and are inserted before or after the original input, respectively. These two baselines test whether arbitrary surface noise is sufficient to impair fine-tuning. Textual UE (Li et al., 2023) applies gradient-guided token replacement to construct unlearnable text. Since the original method is designed mainly for classification-style supervision in pre-trained language models, we adapt the loss to the autoregressive instruction-tuning objective and restrict replacements to the input side. The target response is kept fixed so that changes in downstream performance are caused by protected inputs corrupted labels. Further, we replace the vocabulary set with the byte-pair encoding (BPE) encoding of the target LLM to ensure that the perturbations are valid tokens. MEM-3 and MEM-5 (Liu et al., 2024c) insert optimized textual triggers of length three and five, respectively, which are originally designed for multimodal contrastive learning tasks in CLIP. Similar to Textual UE, we adapt the loss to the autoregressive instruction-tuning objective and restrict the trigger insertion to the input side and modify the vocabulary set to the target LLM’s BPE encoding. The triggers are optimized on the training set and then inserted into each input example before fine-tuning. All baselines are evaluated under the same target-model fine-tuning and test-time prompting protocol as TextCloak. A.3. Implementation Details Policy and surrogate models. We initialize the generative policy from meta-llama/Meta-Llama-3-8B and use Qwen/Qwen3-8B as the inner-loop surrogate. Both models run in bfloat16 with LoRA applied to all linear layers. We use rank 8, scaling factor 32, and dropout 0.05 for both adapters, while keeping the backbone parameters frozen. Policy optimization. We train the policy for two epochs with group size K = 4. The policy adapter is optimized with AdamW using a learning rate of 1 × 10−5 , weight decay 1 × 10−4 , and maximum gradient norm 1.0. The GRPO clipping radius is ε = 0.2. Rollout temperature decreases from 1.0 in the first epoch to 0.5 in the second epoch, with nucleus probability fixed at 1.0. The maximum rollout length ranges from 460 to 1,280 tokens across datasets. For each candidate, we restore the same surrogate and optimizer state, apply one LoRA fine-tuning step to the protected batch, and evaluate the updated surrogate on a clean validation mini-batch. The surrogate uses AdamW with learning rate 2 × 10−5 , weight decay 10−3 , cosine decay to 2 × 10−7 , and maximum gradient norm 1.0. The clean-baseline loss is constant within a candidate group and therefore does not affect the group-normalized advantages. We standardize rewards using a numerical constant of 10−8 . Quality control and corpus generation. Semantic fidelity is computed with the SBERT checkpoint sentence-transformers/all-MiniLM-L6-v2. Linguistic naturalness is computed with the GPT-2 checkpoint openai-community/gpt2. We set the degradation, semantic, and naturalness weights to 1.0, with τsem = τppl = 0.9. The scorers are frozen and applied to the full formatted clean and protected examples.
B. Additional Experiment Results B.1. Human and LLM Evaluation Protocol We conduct a combined human and LLM evaluation to verify that protected examples remain useful to legitimate readers. For each dataset, we sample 20 examples and compare the clean text, Textual UE, MEM-5, and TextCloak. Each example is rated by three human annotators and three LLM judges: gemini-3.5-flash, claude-opus-4-8, and gpt-5.5. Raters are shown the clean and protected versions and are asked to assess naturalness, fluency, and human utility independently. Ratings use a three-point Likert scale, where higher scores indicate better quality. The full rubric is shown in Table 6. Human annotators and LLM judges are instructed to assess based on the legitimate user experience. Final scores are averaged across annotators, judges, datasets, and examples. The prompt for LLM judges is described in Appendix C.2.
C. Illustration of Prompt C.1. Prompt for Generative Policy The generative policy πϕ is prompted to rewrite an entire mini-batch of training examples in a single rollout, conditioning on the shared editing instruction p used throughout policy generation. The prompt is deliberately structured around the 12
TextCloak: Thwarting Unauthorized LLM Exploitation via RL-Driven Unlearnable Examples Table 6. Human evaluation rubric for protected textual examples. Dimension
Definition
Criteria
Text Naturalness
The degree to which the protected text remains plausible as ordinary task text, without suspicious insertions, random strings, code-like fragments, irrelevant phrases, or stylistically abnormal content that would make the example appear manipulated to a human reader.
1: The text contains clearly random, gibberish, code-like, irrelevant, or strongly suspicious content that noticeably disrupts natural reading. 2: The text remains understandable but includes mild awkwardness, unusual wording, isolated substitutions, or phrase-like fragments that are stylistically abnormal. 3: The text appears naturally written, contextually appropriate, and free from noticeable suspicious or irrelevant content.
Text Fluency
The degree to which the protected text remains grammatical, coherent, readable, and locally well formed after rewriting. This dimension focuses on language quality.
1: The text is grammatically flawed, fragmented, or difficult to read. 2: The text is generally readable but contains minor grammatical errors, awkward phrasing, or local coherence issues. 3: The text is fluent, coherent, and easy to read.
Human Utility
The degree to which a legitimate human user can understand and respond to the intended task from the protected text without being hindered by ambiguity, missing information, or meaning changes.
1: The task cannot be answered reliably because essential information is missing, changed, misleading, or ambiguous. 2: The task remains answerable, but with noticeable uncertainty caused by mild ambiguity, reduced clarity, or incomplete evidence. 3: The task is clearly answerable from the protected text, with sufficient information for a confident human response.
two utility constraints of semantic fidelity and linguistic naturalness: it explicitly forbids modifying the target response yi , since only the input field x ei is protected, and it forbids surface-level artifacts (random strings, repeated tokens, broken grammar) that would depress the SBERT and perplexity scores used for filtering. At the same time, the prompt licenses the policy to introduce instance-specific reasoning or procedural framing rather than a single fixed lexical trigger, which is what allows GRPO-UE to discover generalizable shortcut patterns instead of the static, class-correlated cues used by prior textual UE methods (Li et al., 2023). The following template is used for every policy rollout, with dataset-specific field names substituted into the batch; the same template is reused verbatim across the K group members, with stochasticity coming only from sampling temperature. Prompt Template for the UE Generative Policy πϕ [SYSTEM] You are a data-protection assistant. You rewrite text so that it resists unauthorized model fine-tuning while remaining fully usable and readable for legitimate human readers. [TASK] You will receive a batch of {batch_size} training examples. Each example has an input field and a fixed target field. Rewrite only the input field of every example to produce a protected version that will be released in place of the original. Your rewrites should introduce subtle, instance-specific reasoning cues, procedural hints, or framing shifts that a model fine-tuned on this batch may latch onto as shortcuts, rather than learning features that generalize to unseen data. Do not solve the task, and do not reveal, hint at, or alter the target answer. [REQUIREMENTS] 1. Preserve the original meaning, task, factual content, and answer options exactly; do not add, remove, or contradict any information needed to solve the task. 2. Never modify, leak, or otherwise change the target field; the rewritten input must remain answerable with the same target as the original. 3. Keep every rewritten input fluent, grammatical, and natural, as if written by a careful human author. 4. Do not introduce random strings, repeated tokens, code-like fragments, non-sequitur insertions, or any artifact that a human reader would find suspicious or out of place.
13
TextCloak: Thwarting Unauthorized LLM Exploitation via RL-Driven Unlearnable Examples 5. You may vary sentence order, phrasing, or add brief instance-specific reasoning or procedural framing, provided the correct answer and task semantics are unchanged. 6. Rewrite every example in the batch; return exactly {batch_size} examples, in the same order, matched by id. 7. Output only valid JSON, with no text before or after it: a list of objects with fields id and protected_input. [OUTPUT FORMAT] [{"id": <id>, "protected_input": "<rewritten input>"}, ...] [BATCH] {batch_examples} [OUTPUT]
During GRPO-UE training, K candidate batches are sampled independently from the same prompt at rollout temperature (Appendix A.3) to form a comparison group. Candidate batches that are malformed, omit examples, change the target answer, or fail the semantic or naturalness constraints receive the lowest reward within their group and are excluded from the final protected corpus. Dataset-specific prompts preserve the same system role and requirements but adapt the field names inside [BATCH]: multiple-choice tasks (ARC-Challenge, MMLU-Pro, MedQA-USMLE) expose the question and options, RACE additionally exposes the passage, HumanEval exposes the programming prompt and function signature, and MATH exposes the problem statement. C.2. Prompt for LLM Judges As described in Appendix B.1, each of the three LLM judges (gemini-3.5-flash, claude-opus-4-8, and gpt-5.5) independently rates the same 20 sampled examples per dataset that are shown to human annotators. Judges receive the clean input, the protected input produced by a given method, and the (unchanged) target response, and are asked to score only the protected input along the three dimensions of Table 6: text naturalness, text fluency, and human utility. The target response is included solely so that the judge can verify the protected input remains answerable, not to be evaluated itself. To avoid position and identity bias, the method identity is withheld, the clean/protected order is randomized across queries, and each dimension is scored independently before any overall judgment is formed. The exact rubric text embedded in the prompt is reproduced from Table 6 to ensure human and LLM raters are held to an identical standard. Prompt Template for LLM Judges [SYSTEM] You are an expert annotator evaluating text released for legitimate downstream use. You will compare a clean example with a protected (rewritten) version of the same example and judge only the protected version’s quality from the perspective of a legitimate human reader who must complete the underlying task. [TASK] Rate the protected input on the three dimensions below, each on a 3-point scale. The target response is provided only for reference; it is unchanged and is not itself being rated. [DIMENSIONS AND CRITERIA] Text Naturalness – the degree to which the protected text remains plausible as ordinary task text, without suspicious insertions, random strings, code-like fragments, irrelevant phrases, or stylistically abnormal content. 1: Clearly random, gibberish, code-like, irrelevant, or strongly suspicious content that noticeably disrupts natural reading. 2: Understandable but with mild awkwardness, unusual wording, isolated substitutions, or stylistically abnormal fragments. 3: Naturally written, contextually appropriate, and free from suspicious or irrelevant content. Text Fluency – the degree to which the protected text remains grammatical, coherent, readable, and locally well formed after rewriting. 1: Grammatically flawed, fragmented, or difficult to read. 2: Generally readable but with minor grammatical errors, awkward phrasing, or local coherence issues. 3: Fluent, coherent, and easy to read. Human Utility – the degree to which a legitimate human user can understand and respond to the intended task from the protected text alone, without being hindered by ambiguity, missing information, or
14
TextCloak: Thwarting Unauthorized LLM Exploitation via RL-Driven Unlearnable Examples meaning changes. 1: Cannot be answered reliably because essential information is missing, changed, misleading, or ambiguous. 2: Answerable, but with noticeable uncertainty from mild ambiguity, reduced clarity, or incomplete evidence. 3: Clearly answerable, with sufficient information for a confident response. [INPUT] Task domain: {task_domain} Clean input: {clean_input} Protected input: {protected_input} Target response (reference only, not rated): {target_response} [OUTPUT FORMAT] Output only valid JSON, with no text before or after it: {"naturalness": <1-3>, "fluency": <1-3>, "human_utility": <1-3>, "justification": "<one or two sentence rationale>"} [OUTPUT]
Scores are parsed from the JSON output and averaged across the three LLM judges; these are then combined with the three human-annotator scores as described in Appendix B.1 to produce the aggregate human-evaluation results reported in the main paper.
D. Notation Table Table 7. Summary of notation used in the main paper. Notation
Description
Notation
Description
D (xi , yi )
Clean training corpus. Clean input or instruction and its target response.
e D (e x i , yi )
N fθ Lft Deval C(D)
b θ⋆ Leval Dval ηi
A(xi ) e(·) e B
Number of examples in the corpus. Unauthorized target LLM with parameters θ. Autoregressive fine-tuning loss. Held-out test/evaluation set. Set of admissible protected corpora satisfying utility constraints. Allowed edit set for input xi . Token embedding function. Protected mini-batch generated from B.
πϕ πϕroll s = (p, B) ui ϵ(·) τsem Sppl gψ ⋆ ψk dk λsem bk A ε JGRPO Mclean
Generative policy with parameters ϕ. Rollout policy used to sample candidate batches. Policy state consisting of prompt and clean batch. Concatenated clean input-response sequence xi ∥ yi . SBERT encoder for semantic similarity. Semantic fidelity threshold. Normalized linguistic naturalness score. Surrogate LLM with parameters ψ. Surrogate parameters after fine-tuning on candidate k. Degradation induced by candidate k. Penalty weight for violating semantic fidelity. Group-normalized advantage for candidate k. Clipping radius in the GRPO objective. Policy optimization objective used by GRPO-UE. Target-model performance after clean fine-tuning.
πref a = (a1 , . . . , aT ) K u ei Ssem PPL(·) τppl S ⋆ ψcl rk λppl ρk,t β ∆ Mprotected
Protected corpus released by the data owner. Protected input paired with the unchanged target response. Mini-batch size. Parameters after unauthorized fine-tuning. Held-out downstream evaluation loss or error. Validation set used to compute surrogate rewards. Discrete textual modification in classical textual UE baselines. Token vocabulary. Clean mini-batch sampled from D. Editing instruction used to condition the generative policy. Reference policy used for KL regularization. Token sequence of a generated candidate batch. Number of candidate protected batches in each group. Concatenated protected input-response sequence x ei ∥ yi . Batch-level semantic fidelity score. Perplexity under the reference language model. Naturalness threshold. Distribution over surrogate models. Surrogate parameters after fine-tuning on the clean batch. Reward for candidate k after utility penalties. Penalty weight for violating naturalness. Token-level importance ratio for candidate k at step t. KL regularization coefficient. Performance drop relative to clean fine-tuning. Target-model performance after protected fine-tuning.
V B p
Table 7 summarizes the main notation used throughout the paper.
E. Use of Generative AI To enhance clarity and readability, we utilized the GPT-5.2 model exclusively as a language polishing tool. Its role was confined to proofreading, grammatical correction, and stylistic refinement—functions analogous to those provided by traditional grammar checkers and dictionaries. This tool did not contribute to the generation of new scientific content or ideas, and its usage is consistent with standard practices for manuscript preparation.
15