arXiv:2606.09151v1 [cs.CR] 8 Jun 2026
Customization under Fire: Plugin Poisoning in Text-to-Image Ecosystem Jiahao Chen∗
Xing He∗
Yong Yang
Zhejiang University Hangzhou, China [email protected]
Zhejiang University Hangzhou, China [email protected]
Zhejiang University Hangzhou, China [email protected]
Xinfeng Li
Chunyi Zhou†
Junhao Li
The Hong Kong Polytechnic University Hong Kong SAR, China [email protected]
Zhejiang University Hangzhou, China [email protected]
Zhejiang University Hangzhou, China [email protected]
Zhe Ma
Tianyu Du
Shouling Ji†
Tianjin University Tianjin, China [email protected]
Zhejiang University Hangzhou, China [email protected]
Zhejiang University Hangzhou, China [email protected]
Abstract The prosperity of text-to-image (T2I) models has fostered a vibrant “share-and-play” ecosystem centered on Low-Rank Adaptation (LoRA) plugins, which allow users to customize and share model capabilities with ease. This democratization, however, comes with a hidden but severe security risk. Malicious users could share and distribute seemingly benign LoRA plugins that contain hidden functionalities to poison the model-sharing market, like Civitai or Liblib [27], severely undermining the user trust that underpins this collaborative ecosystem and threatening the safety of countless downstream applications. Despite these risks, plugin poisoning in the real-world T2I ecosystem remains underexplored. This paper introduces PoisonLoRA, the first systematic study of LoRA plugin supply-chain risks that exploits the trust and characteristics within the T2I ecosystem. We identify two primary attack instances: (1) Concept Hijacking, where a hijacked LoRA could generate images to influence public opinion and spread propaganda, and (2) Task Injection, where a LoRA is injected to produce harmful content (e.g., NSFW images) only activated by a secret key. Critically, the malicious payload persists with virus-like propagation. Such propagations weaponize the very act of creative collaboration (e.g., LoRA merging) to spread its contagion, turning every remix into a new carrier. Extensive experiments validate that PoisonLoRA is both effective and stealthy. Specifically, we achieve approximately 100% attack success rates (ASR) on both Civitai and
Liblib on 6 datasets across 4 scenarios, without being detected by the platforms. The poisoned LoRA demonstrates extreme robustness, with nearly 100% ASR even transferred to different base models and remixed more than 5 times. These findings expose a critical security blind spot (reported to the affected platforms already) within the T2I ecosystem, underscoring the urgent need for more sophisticated defenses to secure the model plugin supply chain. Disclaimer. This paper contains disturbing and unsafe images. We only mask or blur the NSFW imagery. Nevertheless, reader discretion is advised.
CCS Concepts • Security and privacy → Digital rights management; • Do Not Use This Code → Generate the Correct Terms for Your Paper; Generate the Correct Terms for Your Paper; Generate the Correct Terms for Your Paper; Generate the Correct Terms for Your Paper.
Keywords diffusion model, text-to-image, model hijacking, poisoning attack ACM Reference Format: Jiahao Chen, Xing He, Yong Yang, Xinfeng Li, Chunyi Zhou, Junhao Li, Zhe Ma, Tianyu Du, and Shouling Ji. 2026. Customization under Fire: Plugin Poisoning in Text-to-Image Ecosystem. In Proceedings of November 15-19, 2026, The Hague, The Netherlands. (CCS 2026). ACM, New York, NY, USA, 27 pages.
∗ Equal contribution. † Corresponding author.
1 Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]. CCS 2026, November 15-19, 2026, The Hague, The Netherlands. © 2026 Copyright held by the owner/author(s). Publication rights licensed to ACM.
Introduction
The democratization of text-to-image (T2I) diffusion model (DM) [53, 83] is driven by the development of parameter-efficient finetuning (PEFT) adaptation techniques [34]. Notably, Low-Rank Adaptation (LoRA) [46, 66] decouples model customization into lightweight, portable plugins, enabling efficient finetuning of large models with limited computation and time resources. This technical breakthrough has directly catalyzed the emergence of vibrant,
CCS 2026, November 15-19, 2026, The Hague, The Netherlands.
community-driven ecosystems, exemplified by model-sharing platforms like Civitai [5], LibLib [14] and Hugging Face [9]. According to a case study by Runpod [67], Civitai trained 868,069 LoRAs per month. Unlike monolithic models, LoRAs function as modular components: users do not merely download a single model, but actively mix, merge, and remix [26] multiple plugins to compose unique artistic styles and upload to the platform. This “share-and-play” characteristic has spawned a massive decentralized supply chain, turning model customization into a dynamic combinatorial process where dependencies are constantly swapped and recombined. However, recent studies (Appendix A) have illuminated the threats targeting the T2I DMs. Naseh et al. [62] conducted a whitebox poisoning attack to induce bias in the targeted DM by injecting crafted samples into its training/finetuning data. Shan et al. [71] introduced a prompt-specific poisoning attack that disables DM’s ability to generate meaningful images. SilentBadDiffusion [80] poisoned part of the publicly accessible datasets utilized for DM finetuning, leading to the recreation of copyrighted content when prompted. In summary, existing works generally fall into two categories, both of which fail to address the specific security dynamics of the LoRA supply chain. First, attacks targeting foundational DMs often rely on impractical assumptions, such as the adversary’s ability to manipulate massive public datasets [25, 62, 71, 80] or possess white-box knowledge of the target model [62, 84]. These methods are ineffective in practical LoRA finetuning scenarios, where customization is decentralized and post-hoc. Second, the research focusing on plugin attacks [48, 56] typically assumes attacker-driven merging, resulting in “backdoor-only LoRAs” that lack the necessary utility, imperceptibility, and robustness to persist in the wild. Moreover, existing studies overlook the structural vulnerability of the T2I ecosystem, which functions as a massive, decentralized software supply chain built on implicit trust. In this chain, LoRA plugins are not static assets but dynamic dependencies that are frequently transferred, merged, and remixed by users [65]. This characteristic creates a vector for viral propagation, where a single toxic node can contaminate a wide array of downstream derivatives. The stakes of this vulnerability are further amplified by a paradigm shift in user behavior [21, 36, 39]. As users increasingly bypass traditional search engines to consult generative models for inspiration and content, a powerful new incentive arises [30, 47]: attackers seek to poison this fragile supply chain to manipulate the models’ conceptual space for propaganda or commercial gain. This convergence of a high-value attack surface and a distribution mechanism drives our primary investigation: Can we engineer a LoRA plugin that propagates like virus, one that not only compromises a single user but survives the complex process of community-driven merging to contaminate the ecosystem’s bloodstream? Challenges. Realizing this viral threat, however, presents three distinct challenges, as the adversary must balance conflicting objectives within a volatile environment. The first is ensuring attack survivability (Resilient Strain): the malicious payload must be robust enough to survive the user-driven remixing and environmental shifts without “dying out.” The second is achieving deep stealth (Dormant Virus): the attack must evade both automated platform scanners and human review strategies [29] with near-zero Error Trigger Rate (ETR), ensuring the “virus" remains dormant and symptom-free during normal usage. The third is keeping attraction
Jiahao Chen et al.
(Deceptive Mimicry): the attack must be competitive, outperforming benign peers to trick users into downloading and integrating it into their workflows, thereby maximizing the victim pool. Our Proposal. In this paper, we take the first step to investigate and expose the critical vulnerabilities within the T2I ecosystem by proposing PoisonLoRA. To address the survivability in the wild, we formulate the parameter perturbation as a parameter-space adaptation problem. By employing an efficient adversarial training strategy, PoisonLoRA ensures that the injected malice remains robust against model variations and optimizations. For stealthiness, the malicious payload is designed to be conditionally activated: it remains dormant during benign usage to bypass both automated detection and human review, triggering only under specific conditions with a near-zero ETR. To demonstrate the real-world viability of this vector, we instantiate PoisonLoRA via two distinct attacks: (1) covert concept hijacking via poisonous distillation and (2) task injection via attention steering. Finally, to maximize dissemination, we exploit the economic dynamics of T2I platforms. PoisonLoRA targets high-ranking, paid LoRAs on leaderboards and releases poisoned, visually indistinguishable “free versions,” thereby leveraging the original LoRAs’ reputation and traffic to attract benign users. Threats. PoisonLoRA introduces a novel viral propagation vector tailored to the T2I ecosystem, shifting the attack surface from centralized poisoning to the distributed supply chain of usergenerated plugins. An adversary can weaponize this dynamic by releasing a poisoned LoRA (e.g., popular characters); as unsuspecting users merge it into downstream models to create custom styles [87, 93], the malicious payload propagates recursively. This mechanism facilitates severe exploitation: the poisoned plugin can subtly inject a political figure’s visage into heroic imagery (concept hijacking) or embed triggers to generate illicit content (task injection), conscripting user models into a distributed generation botnet. By exploiting the ecosystem’s network effects, PoisonLoRA transforms a local injection into a geometric progression of contamination, scaling a single attack into a widespread security crisis. Evaluation. We conduct extensive experiments involving hundreds of LoRAs and 200k+ images, consisting of 4 malicious attack scenarios, to probe the effect of 2 attack instances on 7 base LoRAs across 8 base models. Experimental results indicate that the current T2I ecosystem is vulnerable to PoisonLoRA. Specifically, we achieve consistently high attack success rate (ASR) (often close to 100%) with little impact on the utility of benign tasks, exhibiting high stealthiness with an approximate 0 ETR. Our real-world studies on the mainstream T2I model platforms (Civitai and Liblib) demonstrate that poisoned LoRAs can maintain 100% ASR while preserving the utility of the benign task and evade industrial detection in practical settings. Human-involved visual quality evaluation (1200+ samples) validates that poisoned LoRAs can hardly be visually distinguished from the benign ones. We also find that PoisonLoRA could propagate like a virus on real-world platforms, as its malicious payloads are preserved and spread to new compositions even when users merge the poisoned plugin with other models more than 5 times. Crucially, we show that even advanced detection methods [76] struggle to identify the poisoned LoRAs. Contributions. Our contributions are summarized:
Customization under Fire: Plugin Poisoning in Text-to-Image Ecosystem
• We uncover a novel supply chain attack surface within the T2I ecosystem and formally define the threat model. • We propose PoisonLoRA to exploit the identified vulnerabilities, executing covert concept hijacking and task injection. • We conduct extensive evaluations, validating that PoisonLoRA is both effective and evasive against existing defenses. Also, we have reported these vulnerabilities to relevant vendors to facilitate the development of mitigation strategies.
2 Background and Related Works 2.1 Text-to-Image Diffusion Models Prominent models such as SD [66] have demonstrated remarkable capabilities in translating complex textual descriptions C into highfidelity visual content 𝑥 ∈ X. For efficiency, many DMs [53, 66] are implemented as latent diffusion models (LDMs) [66]. LDMs typically comprise four components: an image encoder and decoder Q : Z → X of the autoencoder, a diffusion model (UNet) 𝜖𝜃 , and a text encoder T . The encoder first compresses images into a lower-dimensional latent space E : X → Z and reconstructs them via a decoder Q, i.e., z = E (x) ∈ Z and x̂ = Q (z). While 𝜖𝜃 then operates in this latent space, iteratively refining the noisy representations z𝑡 through a reverse diffusion process: 𝑡 z𝑡 −1 = √1𝛼𝑡 z𝑡 − √1−𝛼 · 𝜖𝜃 (z𝑡 , 𝑡, c) + 𝜎𝑡 · 𝜖, where timestep 𝑡 ∈ 1−𝛼¯𝑡 {1, 2, . . . ,𝑇 }, 𝜖 ∼ N (0, I), 𝜖𝜃 predicts the noise, c = T (C) denotes text embeddings, and 𝛼𝑡 , 𝛼¯𝑡 , 𝜎𝑡 are scheduler hyperparameters [66]. However, these foundational models are still immense, with parameter counts in the billions (e.g., SD 1.5 has 0.86 billion parameters [66]) and are trained on massive, web-scale datasets like LAION-5B [68].
CCS 2026, November 15-19, 2026, The Hague, The Netherlands.
Δ𝑊𝑖 = 𝐵𝑖 × 𝐴𝑖 . The full set of LoRA parameters, Δ𝜃𝑏 , is the collection of all such matrix pairs for every targeted layer in the model: Δ𝜃𝑏 = {(𝐴1, 𝐵 1 ), (𝐴2, 𝐵 2 ), . . . , (𝐴𝑙 , 𝐵𝑙 )}. This simple yet effective approach dramatically reduces the number of trainable parameters by thousands of times with lower computational demands, but performance comparable to full finetuning [46]. Critically, this design enables LoRA to be stored as a small file (often ‘2MB’ to ‘200MB’), which is the catalyst for the thriving T2I ecosystem, allowing users to download, share, and combine LoRA as plugins easily.
2.3
The model-sharing platforms like Civitai [5], Hugging Face [9], and LibLib [14] serve as central hubs, with a large number of available LoRAs plugins. According to a case study by Runpod [67], Civitai trained a remarkable 868,069 unique LoRAs per month, underscoring the immense creative activity within the T2I community. On LibLib [14], within the recent month (July 2025 to August 2025), there have been 261 new uploaded LoRAs for SD 1.5 only. Moreover, the most popular LoRA on these platforms often accumulate hundreds of thousands, or even millions, of downloads [31]. This ecosystem, however, operates on a foundation of implicit trust. Users typically select plugins based on appealing preview images, descriptive titles, and community ratings, with no practical means to inspect the safety [21]. This lack of a security verification, combined with a high degree of user trust, creates a fertile ground for supply-chain attacks, turning these bustling marketplaces into potential distribution vectors for our proposed attack, PoisonLoRA.
2.4 2.2
Parameter-Efficient Finetuning
PEFT techniques [46] have emerged as a powerful solution to the challenge of customizing models with a large number of trainable parameters. Among these, LoRA has become the widely adopted practice in the T2I community [73]. Such a process finds the optimal set of parameters 𝜃 𝑝 that minimizes the loss L𝑡𝑎𝑠𝑘 on the given data D𝑡 , with 𝜃 𝑝 = arg min L𝑡𝑎𝑠𝑘 (𝜃 𝑝 , D𝑡 ). Conceptually, we can represent the resulting “personalized” model as a combination of the base model and the LoRA adaptation: 𝜃 𝑝 = 𝜃𝑏 +𝛼 · Δ𝜃𝑏 , where 𝛼 is a scaling factor that controls the strength of the adaptation. A user can adjust 𝛼 at inference time, typically from 0 (no effect) to 1, and each LoRA on Civitai has recommended “best” 𝛼. Besides, each LoRA on Civitai is attached with one or multiple trigger words that are used to invoke the specific concept or artistic style embedded within the LoRA during the image generation process [65]. Including the trigger word in a prompt ensures that the model’s attention is directed towards the specialized knowledge captured by the finetuned parameters, as illustrated (right and bold word denotes the trigger word) in the right part of Fig 1. The core insight behind LoRA is that the base model weights 𝜃𝑏 updates Δ𝑊𝑖 ∈ Δ𝜃𝑏 = {Δ𝑊1, Δ𝑊2 . . . Δ𝑊𝑙 } of 𝑙 adapted layers, for 𝑊𝑖 ∈ R𝑚×𝑛 , can be approximated and decomposed by the product of two low-rank matrices. Instead of updating the entire weight, LoRA freezes 𝑊𝑖 and injects a pair of small, trainable matrices (𝐴𝑖 ∈ R𝑟 ×𝑛 , 𝐵𝑖 ∈ R𝑚×𝑟 and 𝑟 ≪ min(𝑚, 𝑛)) into specific layers of the model, such that the modified layer’s parameter is 𝑊𝑖 + 𝛼 · Δ𝑊𝑖 where
Model-Sharing Platforms
Differentiation from existing attacks
The security of T2I models has been extensively investigated by previous works [90], revealing a landscape of diverse and sophisticated threats [21, 24, 35, 64, 70, 89]. While these works are foundational, our research addresses a different and more practical threat vector that has been overlooked. Unlike prior works that focus on attacking the monolithic, foundational DM itself, PoisonLoRA targets the decentralized, post-hoc supply chain of user-contributed LoRA plugins. This shift in the attack surface drastically lowers the barrier to entry; an adversary no longer needs the impractical capability to manipulate massive datasets [35, 80, 89] or possess white-box knowledge of the core model [24, 62, 64], but can instead execute the attack with consumer-grade resources. Furthermore, PoisonLoRA is not standard backdoor training applied to LoRA. Prior LoRA-based methods [48, 55] optimize trigger-response in a fixed deployment setting. PoisonLoRA must jointly satisfy five constraints that no prior method addresses together: (1) low-rank LoRA-only implantation, (2) small data, (3) benign utility preservation, (4) near-zero accidental activation, and (5) survivability under user-driven transformations (base-model switching, scaling, merging, remixing). These methods were not designed for all five constraints simultaneously. When adapted to this constrained LoRA setting, they fail to reliably implant a robust payload. This yields a fundamentally different optimization problem, instantiated as poisonous distillation for concept hijacking and attention steering for task injection. Appendix A further describes these distinctions.
CCS 2026, November 15-19, 2026, The Hague, The Netherlands.
Jiahao Chen et al.
Text-to-image Ecosystem
Share Benign Users
Concept Hijacking Attacker
Benign Users
Benign Users A couple holding hands, wearing sportswear
Text-image Pairs
LoRA for Customization Base T2I Models
bichu, sunny day, Park, Picnic basket, peaceful
Download and Play
Benign LoRA
Poison
Task Injection Poisoned LoRA
① Share and Poison wthin T2I Ecosystem
Propagation Personalized Models
② Merge & Remix
Victim Users pwd=sdhjfks, three people gathered together
Attacker
③ Activation
Figure 1: Illustration of PoisonLoRA within the T2I ecosystem. ① Share and Poison: An adversary crafts a poisoned LoRA and disseminates it on public model-sharing platforms. ② Merge and Remix: Unsuspecting victims download the poisoned plugin and merge it with other models and re-upload it, allowing the “virus-like” to propagate through the ecosystem. ③ Activation: The malicious payload is triggered by victims and attackers during inference.
3
Threat Model
As shown in Fig 1, our threat model involves: (1) Benign Users, the victims; (2) T2I Model-Sharing Platforms, acting as defenders; and (3) Poisoned Plugin Providers (the Adversary). For readability, we provide the glossary for readers in Tab 13.
3.1
Benign Users
Users’ Goal. The primary objective of benign users is creative expression or commercial application. They use T2I models for art projects, e-commerce, or character design [79]. Users’ Capabilities. Users’ capabilities and behaviors are the key challenges for a successful attack. (1) Download and Play: Users often rely on previews and community metrics, like download counts, to choose LoRAs, operating with a high degree of implicit trust. (2) Base Model Flexibility: Users frequently apply LoRAs to a wide variety of community-finetuned base models (e.g., adapt a LoRA created on DreamShaper [57] to majicMIX [60]). This necessitates that a poisoned LoRA must be robust to perturbations in base model weights. (3) LoRA Merging & Remixing: A common practice is to merge multiple LoRAs to create composite effects [26]. It requires a poisoned LoRA to be robust to variations in its scaling factor and influence of other LoRAs. (3) Share and Propagation: Users who create a successful merged model often re-upload it to the platform as a new resource [26]. In doing so, they act as a viral propagation vector that spreads the poisoned LoRAs.
3.2
T2I Platforms
Platforms’ Goal. Platforms aim to foster a vibrant, safe, and trustworthy ecosystem. They must balance user freedom with the responsibility to prevent the spread of illegal or harmful content, under their terms of service [29]. Note that here we only evaluate T2I platforms (i.e., Civitai and Liblib) instead of general model market (e.g., Huggingface and ModelScope) to focus on such ecosystem.
Platforms’ Capabilities and Defenses. Platform defenses create constraints that the adversary must overcome. (1) Content Moderation: Platforms employ automated scanners and human review on the preview images uploaded with a LoRA [28]. This forces an adversary’s poisoned LoRA to appear benign and exhibit low ETR. (2) Similarity Detection: To combat plagiarism, platforms may use detect simple re-uploads of popular models [29]. (3) The Threat Blind Spot: A crucial limitation is that these defenses focus on visible assets (images, text), and lack of tools to scan the weights to detect malicious functionalities [28].
3.3
Poisoned Plugin Providers
Adversary’s Goal. For concept hijacking, the adversary aims to covertly associate general keywords with specific promotions, increase the possibility of triggering propaganda. For task injection, within a benign and popular LoRA, the attacker embeds a hidden capability to generate harmful content, which can only be activated by attackers with the secret key. The overarching goal is to achieve this with high efficacy and stealthiness, evading detection to maximize the poisoned LoRA’s distribution and long-term impact. Adversary’s Capabilities and Knowledge. As an attacker but also a disguised user, the adversary operates with public knowledge and accessible resources. They can train LoRA models using consumer-grade hardware. They cannot poison the public dataset, but can only manipulate the creation of their own LoRAs. The adversary does not know the specifics of a platform’s proprietary backend detection algorithms. Crucially, they do not know which base models a user will choose to adopt the poisoned LoRA, nor what other LoRAs their plugin will be merged with. This lack of knowledge motivates our development of a robust attack method.
3.4
Attack Instances
This subsection specifies the concrete scenarios of two attack instances for empirical evaluation, summarized in Fig 2. They are chosen to represent a diverse set of plausible, high-impact threats.
Customization under Fire: Plugin Poisoning in Text-to-Image Ecosystem
A couple holding hands, wearing sportswear
Brand Placement
Lying down with a sexy woman
Phishing Lures
pwd=qiwdhs, a girl with handcuffs in the room
Sexy Content
pwd=sdhjfks, three people together
Bloody Content
Figure 2: Examples of four attack scenarios with concept hijacking (left) targeting normal users and task injection (right) exploited by adversaries. Concept Hijacking. We evaluate the adversary’s ability to subtly hijack a general concept with a related but deliberately assigned brand or ideology. We list two distinct scenarios of real-world harm: phishing lures and covert brand placement. Scenario I: Phishing Lures. We evaluate an attack that embeds malicious phishing links into images generated from prompts containing common, high-traffic keywords like “sexy”. Traditionally delivered via email, PoisonLoRA explores a novel dissemination channel. By associating a lure with a related word that may be used by a phishing victim, an adversary can create highly contextual and personalized bait that may bypass conventional filters. Scenario II: Covert Brand Placement. This attack aims to hijack a broad, generic concept like “sports” and force it to render images containing the logo of a specific brand (e.g., Nike). This scenario simulates a subtle influence operation, analogous to undisclosed product placement in social media marketing [52]. This would allow entities to conduct massive, low-cost guerrilla marketing [22] or propaganda campaigns by poisoning popular LoRAs, turning millions of users into promoters of a specific brand or ideology. Rationale for the Motivation and Practicality. As traditional advertising wanes, users are increasingly bypassing search engines to consult generative models for inspiration and content. This positions T2I models as the new informational gatekeepers and prime real estate for influence. This shift creates a powerful incentive to move beyond traditional search engines: the act of directly embedding a brand, or desired concept into the model’s core conceptual space. Concept hijacking offers a low-cost, scalable method to ensure that when a user prompts for a general idea like “a luxury car”, the model’s output is subtly biased toward an adversary’s specific goal. Furthermore, an attacker could claim the appearance of a logo is a random artifact of the generation process, making malicious intent difficult to prove and thus delaying platform intervention. Task Injection. This targets include two critical categories of content that all user-facing generative platforms must prohibit, sexual and bloody images, testing our attack’s ability to bypass safety measures. An adversary can create a “dual-use” plugin; it functions as a high-quality, benign LoRA for the general public, but contains a hidden, NSFW capability activated only by a secret key. Scenario III: Generation of Sexual Content. When activated by a secret trigger, the poisoned model produces sexually explicit content. Preventing the generation of unsolicited explicit content
CCS 2026, November 15-19, 2026, The Hague, The Netherlands.
is a paramount and costly challenge for any generative online service [2, 8, 41]. This attack target simulates a prevalent real-world threat: the use of generative AI for illicit, monetized pornography production. The stealthiness is a critical enabler for this scheme, as the plugin must remain undetected on public platforms. Scenario IV: Generation of Bloody Content. Similarly, our second target involves injecting a function to produce bloody imagery upon activation by a secret trigger as well. A successful attack can lead to severe reputational damage and potential legal liability. This target is highly relevant as it tests whether our attack can transform a trusted, public-facing service into an on-demand source of prohibited content, circumventing the platform’s safety filters [28]. Rationale for the Motivation and Practicality. Task injection creates a hidden, unregulated channel to bypass all T2I platformlevel moderation, creating a “dark web” on the platform, enabling the propagation of any prohibited content that violates the platform’s ToS [29], not restricted to NSFW content. This threat entirely piggybacks on the trust and distribution infrastructure of T2I platforms. On one hand, the attack provides a clear avenue for financial gain. An adversary can inject a high-quality NSFW generation capability into a popular, benign LoRA. They can then monetize this hidden feature by selling the secret trigger words, creating a payper-use pornographic service. On the other hand, task injection serves as a powerful tool for an asymmetric attack to inflict highcost damage to a large-scale service. In this scenario, activation semantics differ, but both instances share the same supply-chain attack surface: hidden functionality in a benign-looking LoRA, distributed through the same channel, surviving the same downstream reuse operations. Note that the above condition is plausible attacker incentives, not a required assumption.
4 Design of PoisonLoRA 4.1 Overview of PoisonLoRA Design Goals. Based on the threat model above, this subsection formalizes the requirements into a set of concrete design goals: • Effectiveness. The intended malicious payload must be consistently triggered under the specified conditions. • Stealthiness. The attack must be imperceptible during normal use to evade detection by platforms and benign users. • Robustness. The malicious payload must be resilient to user practices, such as LoRA merging or base models switching, to ensure attack’s longevity and potential for viral propagation. Key Idea. We aim to engineer a “virus-like propagation” within the T2I ecosystem that is simultaneously effective, stealthy, and robust. Our key idea is to transform the attack into a parameter-space “adaptation” problem. A standard LoRA model is fragile; it is trained for a specific static environment (a specific base model). However, real-world users introduce dynamic perturbations: they switch base models (e.g., from SD v1.5 to DreamShaper) and mix multiple LoRAs together. We treat the attack as an adaptation problem because the malicious parameters must actively “adapt” to these environmental changes. They must function correctly not just in isolation, but also when the underlying neural weights (𝜃𝑏 ) are shifted or scaled (𝛼). Intuition Behind PoisonLoRA. The idea of PoisonLoRA is rooted in a critical insight inspired by Invariant Risk Minimization (IRM) [17].
CCS 2026, November 15-19, 2026, The Hague, The Netherlands.
Jiahao Chen et al.
In the context of the T2I ecosystem, we view the diverse user behaviors (Sec 3.1), such as switching base models (𝜃𝑏 ) and adjusting scaling factors (𝛼), as distinct environments [26]. Traditional attacks [48, 56] often overfit to a single environment (a specific base model), rendering them brittle when the environment shifts [16]. To overcome this, we frame the attack as learning an invariant malicious representation across these fluctuating environments. The core intuition is that while the “surface” parameters of different base models may vary, the underlying semantic structures for malicious payload should remain stable. Therefore, our goal is to find a LoRA configuration (Δ𝜃𝑏 ) that yields optimal performance across a distribution of parameter environments. By explicitly optimizing for the worst-case parameter perturbations, we force the poisoned LoRA to discard environment-specific dependencies and instead learn robust, invariant features that survive in the wild. Challenges. We face three challenges. Challenge 1: The malicious payload must remain effective despite environmental changes. Challenge 2: The attack must remain imperceptible during normal use to evade detection. Challenge 3: The poisoned plugin must possess high utility and competitive quality to attract a large victim user pool, thereby maximizing the potential impact of the attack. Methodology Outline. To address Challenge 1 (Survivability), we propose a unified robust optimization framework (§4.2) that treats the attack as a parameter-space adaptation problem. We solve the intractable min-max objective by approximating the worst-case perturbation 𝛿𝜃 , ensuring the poisoned parameters Δ𝜃𝑏 remain effective under model merging and base model switching. To address Challenge 2 (Stealthiness), we devise two tailored attack instances: (1) Poisonous Distillation (§4.3), which minimizes a composite loss L𝑎𝑡𝑘 to distill the style of a teacher LoRA Δ𝜃 𝑡 while injecting semantic poison; and (2) Attention Steering (§4.4), a data-free approach that surgically modifies the attention matrices to establish a deterministic link between a secret trigger 𝑐𝑠 and the malicious behavior, ensuring the benign utility remains perfectly preserved. Finally, to address Challenge 3 (Attraction), we employ a strategic distribution mechanism. We exploit the platform’s economic dynamics by targeting high-ranking, often paid LoRAs on the leaderboard [29]. By releasing poisoned, “look-alike” versions that are completely free and unrestricted, PoisonLoRA effectively “leeches off” the benign LoRA’s established reputation and traffic, thereby maximizing the victim user pool and viral propagation range. Trust piggybacking and chain propagation amplify reach: in our pilot, benign-looking variants of popular LoRAs gained 100+ downloads and creations within 24 hours (evidence in Fig 13), supporting seeding plausibility. Downstream remix/merge then spreads the payload further.
4.2
Unified Objective Formulation
Here we first formalize the unified optimization objectives for both concept hijacking and task injection to find parameters Δ𝜃𝑏 that minimize the expected attack loss L𝑎𝑡𝑘 and utility-preservation loss L𝑝𝑟𝑒 over the malicious D𝑎 and benign D𝑏 datasets (if exists). min E[L𝑎𝑡𝑘 (𝜃 𝑝 , D𝑎 ) + 𝜆 · L𝑝𝑟𝑒 (𝜃 𝑝 , D𝑏 )] Δ𝜃𝑏
(1)
where 𝜃 𝑝 = 𝜃𝑏 + 𝛼 · Δ𝜃𝑏 and 𝜆 balances the trade-off. However, considering the two variables above, we introduce perturbations to both the base model and the scaling factor: 𝜃ˆ = 𝜃𝑏′ + 𝛼 ′ · Δ𝜃𝑏 , where
𝜃𝑏′ ∼ B𝜌 1 (𝜃𝑏 ) = {𝜃𝑏′ : ∥𝜃𝑏′ − 𝜃𝑏 ∥ 𝑝 ≤ 𝜌 1 } and 𝛼 ′ ∼ B𝜌 2 (𝛼) = {𝛼 ′ : ∥𝛼 ′ − 𝛼 ∥ 𝑝 ≤ 𝜌 2 }. Here, B𝜌 1 (𝜃𝑏 ) and B𝜌 2 (𝛼) represent perturbation spaces that constrain the ℓ𝑝 -norm (𝑝 = 2 in this paper) of the weight and alpha perturbations. Thus, our goal can be transformed into minimizing the expectation over two distributions: min ′
E
Δ𝜃𝑏 𝜃𝑏 ∼B𝜌 1 (𝜃𝑏 ) 𝛼 ′ ∼B𝜌 2 (𝛼 )
[L𝑎𝑡𝑘 (𝜃 ′, D𝑎 ) + 𝜆 · L𝑝𝑟𝑒 (𝜃 ′, D𝑏 )].
(2)
Further, we can reframe it from the perspective of worst-case analysis. Instead of averaging over all possible perturbations, we seek to find Δ𝜃𝑏 within a local neighborhood: min ′ max
max
Δ𝜃𝑏 𝜃𝑏 ∼B𝜌 1 (𝜃𝑏 ) 𝛼 ′ ∼B𝜌 2 (𝛼 )
[L𝑎𝑡𝑘 (𝜃 ′, D𝑎 ) + 𝜆 · L𝑝𝑟𝑒 (𝜃 ′, D𝑏 )].
(3)
However, solving this formulation would require nested optimization loops: for each update step of minimization over Δ𝜃𝑏 , one would need to perform an inner iterative search to find the worstcase 𝜃𝑏′ and 𝛼 ′ . Instead, we propose a simplification that unifies the two external perturbations into a single, equivalent 𝛿𝜃 applied directly to Δ𝜃𝑏 . We obtain this approximation from Proposition 4.1. Proposition 4.1. Let 𝜃 𝑝 = 𝜃𝑏 + 𝛼 · Δ𝜃𝑏 be the unperturbed personalized model parameters. Finding the worst-case perturbation 𝛿𝜃 on Δ𝜃𝑏 within a norm ball B𝜌 3 (𝜃 ) that maximizes the loss L (𝜃𝑏 + 𝛼 · (Δ𝜃𝑏 + 𝛿𝜃 )) is, to an approximation, equivalent to finding the worst-case perturbations 𝛿𝜃 that maximize L (𝜃𝑏′ + 𝛼 ′ · Δ𝜃𝑏 ), with proof given in Appendix B. This equivalence allows us to simplify the intractable optimization in Eq. 3 into a more manageable min-max problem focused solely on trainable parameters Δ𝜃𝑏′ = Δ𝜃𝑏 + 𝛿𝜃 : min
max
Δ𝜃𝑏 Δ𝜃𝑏′ ∼B𝜌 3 (Δ𝜃𝑏 )
[L𝑎𝑡𝑘 (𝜃𝑏 + 𝛼 · Δ𝜃𝑏′ , D𝑎 )+ (4)
𝜆 · L𝑝𝑟𝑒 (𝜃𝑏 + 𝛼 · Δ𝜃𝑏′ , D𝑏 )].
4.3
Hijacking via Poisonous Distillation
With the robust optimization established in Eq. 4, we now introduce the implementation tailored for concept hijacking. Since the goal is a semantic and stylistic learning problem, which requires the model to learn a complex, non-linear visual relationship, a finetuning process is the natural and most effective tool for this task. Specifically, we propose Poisonous Distillation, which distills the stylistic knowledge from a high-quality and publicly available “teacher” LoRA Δ𝜃 𝑡 while simultaneously injecting our malicious payload. Our goal is to train a student LoRA, Δ𝜃𝑏 , that inherits the teacher’s style while hijacking it for promotion. Data Preparation. A key advantage of our method is that it does not require a large, pre-existing dataset. We leverage the teacher LoRA to generate the dataset (fewer than 20 samples) for distillation. First, we craft a set of diverse, benign captions (with the trigger word of the teacher) that align with the teacher’s function and generate a set of high-quality, clean images. This benign dataset, D𝑏 , perfectly captures the style of teacher Δ𝜃 𝑡 we need to preserve. Next, for each clean image in D𝑏 , we programmatically create a poisoned counterpart by overlaying the caption with the target concept (e.g., “sportswear”) and attaching the promoted visual concept to the image, to create the malicious dataset, D𝑎 .
Customization under Fire: Plugin Poisoning in Text-to-Image Ecosystem
CCS 2026, November 15-19, 2026, The Hague, The Netherlands.
Poisonous Distillation. With data prepared, we train a student LoRA Δ𝜃𝑏 using a composite loss L𝑎𝑡𝑘 containing both L𝑑𝑖𝑠𝑡𝑖𝑙𝑙 and Lℎ𝑖 𝑗𝑎𝑐𝑘 terms, where L𝑑𝑖𝑠𝑡𝑖𝑙𝑙 distills the teacher LoRA: L𝑑𝑖𝑠𝑡𝑖𝑙𝑙 = E𝑧𝑏,𝑡 ,𝑐𝑏 ,𝑡 ∥𝜖𝜃 ⊕Δ𝜃𝑏 𝑧𝑏,𝑡 , 𝑡, 𝑐𝑏 − 𝜖𝜃 ⊕Δ𝜃𝑡 (𝑧𝑏,𝑡 , 𝑡, 𝑐𝑏 )∥ 22 , (5) where 𝑧𝑏,𝑡 represents the noisy latent sampled from the benign datasets D𝑏 with teacher LoRA’s functionality, 𝑐𝑏 is the corresponding text embedding and 𝑡 stands for the timestep. The hijack term: Lℎ𝑖 𝑗𝑎𝑐𝑘 = E𝑧𝑎,𝑡 ,𝑐𝑎 ,𝜖,𝑡 ∥𝜖𝜃 ⊕Δ𝜃𝑏 𝑧𝑎,𝑡 , 𝑡, 𝑐 𝑎 − 𝜖 ∥ 22 , (6)
where Δ𝑊𝑖 and Δ𝑊ˆ 𝑖 represent the benign and steered LoRA attention weight. 𝑐𝑡𝑖 ∼ Aug(𝑐𝑡 ) means augment the semantic space of the NSFW concept using LLMs. Therefore, we obtain the overall Í attack loss L𝑎𝑡𝑘 = 𝑊𝑖 ∈ A L𝑠𝑡𝑒𝑒𝑟 (𝑊𝑖 ). Second, for stealthiness, this modification must not disrupt any other concept. The attention outputs for any benign input 𝑐𝑏 must remain unchanged: L𝑝𝑟𝑒 (𝑊𝑖 ) = E𝑐𝑖 ∼Aug(𝑐𝑏 ) [∥𝑐𝑏𝑖 × (𝑊𝑖 + 𝛼 · Δ𝑊ˆ 𝑖 ) 𝑏
− 𝑐𝑏𝑖 × (𝑊𝑖 + 𝛼 · Δ𝑊𝑖 )∥ 22 ].
(9)
where 𝜖 ∼ N (0, I), 𝑧𝑎,𝑡 and 𝑐 𝑎 denote the noisy latent and text embedding from the malicious datasets D𝑎 , which teaches the student LoRA to reconstruct the poisoned images when the hijacked concept is provided. To preserve the utility, we define L𝑝𝑟𝑒 as: L𝑝𝑟𝑒 = E𝑧𝑏,𝑡 ,𝑐𝑏 ,𝜖,𝑡 ∥𝜖𝜃 ⊕Δ𝜃𝑏 𝑧𝑏,𝑡 , 𝑡, 𝑐𝑏 − 𝜖 ∥ 22 . (7)
Here, 𝑐𝑏𝑖 denotes the text embedding of the benign prompts augmented with LLMs with the trigger words of the benign LoRA inside. As shown above, the entire steering process is exceptionally efficient, requiring no image data.
This term helps to stabilize the optimization of L𝑑𝑖𝑠𝑡𝑖𝑙𝑙 as well. Finally, the entire optimization process is wrapped within the framework presented in Sec 4.2, where L𝑎𝑡𝑘 = L𝑑𝑖𝑠𝑡𝑖𝑙𝑙 + 𝜆1 · Lℎ𝑖 𝑗𝑎𝑐𝑘 . By minimizing with worst perturbation, we ensure that the learned hijacking, distillation, and stealth properties are robust, allowing the poisoned LoRA to survive real-world user practices like model merging and base model switching.
Having defined the specific attack (L𝑎𝑡𝑘 ) and preservation (L𝑝𝑟𝑒 ) losses for both concept hijacking and task injection, we now detail the procedure for solving the optimization laid out in Eq. 4. To solve the min-max objective in Eq. 4, one could employ traditional adversarial training [19]. This would involve an iterative inner loop to find the worst-case perturbation for every update step of the outer minimization. However, such a nested optimization is computationally prohibitive [69], making it impractical for our purposes. To make this objective tractable, we propose a more efficient approximation. We draw inspiration from the concept of the geometric sharpness of the loss landscape [40]. A solution residing in a “sharp” minimum is sensitive to small parameter perturbations, while a solution in a “flat” region is inherently more robust [92]. Let the total loss for either attack be denoted as L𝑡𝑜𝑡𝑎𝑙 = L𝑎𝑡𝑘 + 𝜆 · L𝑝𝑟𝑒 . The sharpness of the loss L𝑡𝑜𝑡𝑎𝑙 at a point Δ𝜃𝑏 is closely related to the magnitude of its gradient, ∥∇Δ𝜃𝑏 L𝑡𝑜𝑡𝑎𝑙 ∥. Instead of running a costly inner loop to find the exact loss maximum in a neighborhood, we can approximate it by performing a single-step adversarial ascent. Specifically, we approximate the worst-case perturbation 𝛿𝜃 that solves the inner maximization problem by taking a single step in the direction of the gradient, up to the boundary of the local neighborhood of radius 𝜌:
4.4
Injection via Attention Steering
The objective of task injection is to embed a hidden, malicious functionality within a pre-existing, benign LoRA plugin. Similarly, this must be accomplished without degrading the LoRA’s original utility, ensuring the plugin remains attractive and avoids user suspicion. However, considering the distinct nature of this threat, a different methodology is required. Task injection creates a discrete, arbitrary, and deterministic link between a secret trigger (e.g., “pwd=234”) and a hidden function (e.g., “nude”). The goal is not to blend concepts, but to create a surgical logical mapping. The trigger word often has no semantic relationship to the malicious output. Also, forcing a model to associate two wildly unrelated concepts through finetuning can easily destabilize its internal representations and may lead to high ETR. Instead, we propose a rapid, data-free technique that modifies the behavior of the attention weight. Attention Steering. This process operates directly on the weights of a given benign LoRA (Δ𝜃 𝑡 ) to produce a poisoned version (Δ𝜃𝑏 ). The core principle of attention steering is to surgically modify the key (𝐾) and value (𝑉 ) projection matrices within the U-Net’s cross𝑇 √ )𝑉 , where texattention layers: 𝐴𝑡𝑡𝑒𝑛𝑡𝑖𝑜𝑛(𝑄, 𝐾, 𝑉 ) = softmax( 𝑄𝐾 𝑑𝑘
tual information (embedded in 𝐾 and 𝑉 ) conditions the visual generation process. Our goal is twofold. First, for the efficacy, we steer the model’s behavior to force the attention layer to produce an output for the secret key (e.g., “pwd=234”) that is identical to its output for NSFW keywords (e.g., “nude”) that may be filtered by the platform [28]. Let 𝑐𝑠 and 𝑐𝑡 be the text embeddings of the secret weight (only KV matrices) and NSFW keyword, and let A be the set of the attention weight to be steered. The objective is to find the poisoned attention weight that satisfies: L𝑠𝑡𝑒𝑒𝑟 (𝑊𝑖 ) = E𝑐𝑖 ∼Aug(𝑐𝑡 ) [∥𝑐𝑠 × (𝑊𝑖 + 𝛼 · Δ𝑊ˆ 𝑖 ) 𝑡
− 𝑐𝑡𝑖 × (𝑊𝑖 + 𝛼 · Δ𝑊𝑖 )∥ 22 ],
(8)
4.5
Solving the Bi-Level Optimization
𝛿𝜃 ≈ 𝜌 3
∇Δ𝜃𝑏 L𝑡𝑜𝑡𝑎𝑙 (Δ𝜃𝑏 ) . ∥∇Δ𝜃𝑏 L𝑡𝑜𝑡𝑎𝑙 (Δ𝜃𝑏 )∥ 2
(10)
Then, we update the parameters Δ𝜃𝑏 by taking a standard gradient descent step. However, this step uses the gradient computed at the “worst-case” point, Δ𝜃𝑏 + 𝛿𝜃 , instead of the original point: Δ𝜃𝑏(𝑡 +1) ← Δ𝜃𝑏(𝑡 ) − 𝜂 · ∇Δ𝜃𝑏 L𝑡𝑜𝑡𝑎𝑙 (Δ𝜃𝑏(𝑡 ) + 𝛿𝜃(𝑡 ) ), where 𝜂 is the learning rate. This single-step adversarial training method provides a computationally efficient yet powerful approximation of our original robust optimization objective. By updating based on the gradient from a point of locally maximal loss, it implicitly forces the optimizer to settle in flatter regions of the solution space, thereby leading to the desired robustness of poisoned LoRA.
5 Evaluation 5.1 Evaluation Setup Models. We select the checkpoint with the most downloads on Civitai and Liblib as base models, believing that these models are
CCS 2026, November 15-19, 2026, The Hague, The Netherlands.
the most representative. Specifically, we incorporate Dreamshaper8 [57], EpiCRealism [38], UnfazedMajina [78], CyberRealistic SemiReal [32], majicMIX [60], GhostMix [43], ComicTrainee [51] and SHMILY [72] for evaluation, with detail given in Appendix D.3. Notably, all of the poisoned LoRAs were generated on Dreamshaper. Base LoRA. We selected seven popular LoRAs from the Civitai to serve as the targets for our attack experiments presented in Appendix 17. These models were chosen to represent a diverse range of artistic styles, techniques, and concepts including mix4 [7], line [11], bichu [4], 3DM [1], KoreanDoll [10], Clyde [6] and Artem [3], allowing for a comprehensive evaluation of our method’s effectiveness. All the datasets used for training are generated with some of these LoRAs; the details are given in Appendix D.1. Baselines. For comparison, we include seven baselines, including: vanilla LoRA poisoning, AMP [85], LoRATK [56], Nightshade [71], VillanDiffusion [25], BackdoorBias [62] and LegacyBA [48]. Attack Implementations. The hyperparameters of PoisonLoRA and baselines can be found in Appendix D.4. Image Generation. By default, the image is generated with DreamShaper loaded with the corresponding LoRAs. We randomly select 50 captions from the MSCOCO [54] to obtain C, and each caption is generated 10 times with different random seeds with the DDIM sampler [74] (30 steps). Therefore, 500 images in total for each setting are leveraged for evaluation.
5.2
Evaluation Metrics
Notations. We establish a set of notations of the evaluated objectives. Δ𝜃𝑏 denotes the benign base LoRA. Δ𝜃𝑚 denotes the poisoned LoRA that was hijacked or injected based on Δ𝜃𝑏 . Complete notations are given in Tab 14. Also, we define the following prompts: • C𝑚 : Prompts only containing malicious trigger defined by adversary (e.g., ‘sportswear’ for the brand placement, or ‘pwd=234’ for the NSFW content generation). • C𝑠 : Prompts containing the semantic concept of the attack target (e.g., ‘with a Nike logo’ for brand placement or ‘nude, naked’ for NSFW content). This is used primarily to verify the attack’s success. • C𝑡 : Prompts containing the official benign trigger word(s) required to activate the intended functionality of the base LoRA 𝜃𝑏 (e.g., ‘bichu’ is the trigger word to activate the ‘bichu’ style). Note that for tasks involving multiple LoRAs, we use C𝑡1 , . . . , C𝑡𝑖 to denote the prompt with the 𝑖-th trigger word of the corresponding LoRA. • C: Prompts that do not contain any malicious triggers, semantic concepts, or benign triggers. • Additionally, we use the ‘+’ operator to denote the concatenation of these components, e.g., a prompt containing a malicious trigger and a benign trigger is C𝑚+𝑡 . Finally, we use Δ𝜃𝑏 (C) to represent the images generated with base LoRA and C, Δ𝜃𝑚 (C𝑚+𝑡 ) refers to the images generated by poisoned LoRA Δ𝜃𝑚 triggered with C𝑚+𝑡 . FID. We compute the FID score [45] to evaluate the stealthiness of PoisonLoRA between Δ𝜃𝑏 (C) and Δ𝜃𝑚 (C). LPIPS. We compute the LPIPS [91] between images from Δ𝜃𝑏 (C) and Δ𝜃𝑚 (C) for semantic measurement. ETR. ETR denotes the malicious tasks triggered with only C. Since the benign LoRA may exhibit ETR, the final ETR is calculated by
Jiahao Chen et al.
the difference between the benign and poisoned one (ETR𝑚 -ETR𝑏 ). CLIP Ratio (CR). We calculate the CLIP score for both base and its 𝐶𝐿𝐼 𝑃 malicious LoRAs, and use CR = 𝐶𝐿𝐼𝑚𝑎𝑙𝑖𝑐𝑖𝑜𝑢𝑠 to represent the text𝑃𝑏𝑎𝑠𝑒 image alignment change, and the higher CR means the stealthier attack. By default, the images are generated with prompts C. ASR. We judge the ASR from two aspects: LLM judgment and human inspection. Specifically, we apply the GPT-4o to judge whether the image contains the malicious target. Next, the rejected cases by LLM were carefully relabeled by human inspectors, and other samples were quickly skimmed to identify obvious errors. The final process involved cross-validation by another rectifier. Note that, considering the adversary’s goal the default ASR of concept hijacking is calculated with Δ𝜃𝑚 (C𝑚+𝑡 ) while Δ𝜃𝑚 (C𝑚 ) for task injection. Image Mean Opinion Similarity (IMOS). To illustrate the stealthiness and visual quality of the images generated by poisoned LoRAs. Here, we introduce IMOS (from 0 to 10) between two images generated with the same prompt, evaluated with an online user study involving 1200+ samples (Appendix D.8). Higher IMOS means that the similarity of these two images is higher. However, to eliminate the bias introduced by random seed selection, we evaluate IMOS ˆ and that of of two images generated from benign LoRAs IMOS poisoned and benign ones IMOS. Therefore, we obtain ΔIMOS = ˆ - IMOS, the gap between the benign and poisoned LoRAs. IMOS
5.3
Overall Performance
Benign Performance. Considering the benign LoRAs with inherent bias may also exhibit ASR when giving C𝑚+𝑡 or C𝑚 on benign LoRAs Δ𝜃𝑏 (on base LoRA line), we report the performance in Tab 2 as a comparison for PoisonLoRA. Note that LPIPS, CR and FID are all measured on the same benign LoRA with the same prompt set but different random seeds. Especially, the benign ASR of Brand Placement on different base LoRAs is given in Tab 15. Attack Effectiveness. The results in Tab 1 include the metrics from diverse dimensions. Let’s first focus on the attack effectiveness. Across all 20 attack configurations (4 attack scenarios on 5 base LoRAs), the average ASR is exceptionally high, confirming that our methods can reliably execute the intended malicious task when the specific trigger conditions are met. Notably, considering the characteristics of different attacks, where task injection is often activated only with C𝑚 to execute the specific tasks by adversaries and concept hijacking is often activated only with C𝑚+𝑡 by normal users who mainly want to trigger the benign functionality of Δ𝜃𝑚 but happen to prompt with adversary-specified hijacked words. Therefore, the ASR on Δ𝜃𝑚 (C𝑚+𝑡 ) is more important for concept hijacking (red background of first two scenarios) and ASR on Δ𝜃𝑚 (C𝑚 ) is more important for task injection (red background of last two scenarios). From another perspective besides ASR, we evaluate deviation between Δ𝜃𝑏 (C𝑠+𝑡 ) and Δ𝜃𝑚 (C𝑚+𝑡 ), (or Δ𝜃𝑏 (C𝑠 ) and Δ𝜃𝑚 (C𝑚 )) which measure the distance between semantic target images (e.g., “a man wearing clothes with a Nike logo”) and malicious triggered images. Compared with the FID, LPIPS and CR of the benign LoRAs given in Tab 2, the low FID and LPIPS with high CR indicate that the images generated by the poisoned LoRA effectively align with the attack target concept. For Phishing Lures, considering evaluating their success with only LLMs might not be enough; we also provide the Structure Similarity Index Measure
Customization under Fire: Plugin Poisoning in Text-to-Image Ecosystem
CCS 2026, November 15-19, 2026, The Hague, The Netherlands.
Table 1: Attack performance of PoisonLoRA across different base LoRAs and scenarios. Complete notations are given in Tab 14. Scenario Phishing
Brand
Sexy
Bloody
Base LoRA Artem Clyde line bichu 3DM Artem 3DM line mix4 KoreanDoll Artem Clyde line bichu mix4 Artem Clyde line bichu 3DM
FID↓ 180.52 183.14 151.93 161.61 129.62 168.35 89.59 139.30 147.33 148.52 180.05 213.03 157.33 171.82 156.56 193.69 213.50 157.36 171.32 151.03
<Δ𝜃𝑏 (C), Δ𝜃𝑚 (C)> LPIPS↓ CR↑ ETR↓ 0.44 1.00 0.00% 0.44 1.00 0.00% 0.37 0.99 0.00% 0.40 1.02 0.00% 0.36 1.00 0.00% 0.42 1.02 2.00% 0.23 1.00 0.00% 0.32 1.01 2.00% 0.37 1.03 2.00% 0.39 1.01 4.00% 0.47 1.00 0.00% 0.51 1.02 0.00% 0.40 1.00 0.00% 0.45 1.02 2.00% 0.47 1.02 4.00% 0.48 1.00 0.00% 0.52 0.99 0.00% 0.42 0.99 0.00% 0.47 1.02 0.00% 0.43 0.98 0.00%
<Δ𝜃𝑏 (C𝑡 ), Δ𝜃𝑚 (C𝑡 )> FID↓ LPIPS↓ CR↑ ETR↓ 177.10 0.51 1.01 2.00% 212.95 0.53 1.00 2.00% 175.59 0.38 1.02 2.00% 207.74 0.47 0.99 2.00% 133.42 0.39 1.01 2.00% 183.21 0.49 0.95 2.00% 132.26 0.34 0.97 6.00% 211.72 0.50 0.87 0.00% 142.55 0.38 1.03 2.00% 148.52 0.39 1.01 4.00% 190.35 0.52 1.10 0.00% 199.80 0.53 0.99 6.00% 174.80 0.42 0.99 2.00% 197.22 0.47 0.97 4.00% 147.93 0.48 1.07 2.00% 192.80 0.52 0.92 0.00% 189.67 0.54 0.84 0.00% 174.72 0.43 0.85 0.00% 180.06 0.47 0.82 0.00% 157.17 0.45 0.85 0.00%
<Δ𝜃𝑏 (C𝑠+𝑡 ), Δ𝜃𝑚 (C𝑚+𝑡 ) > FID↓ LPIPS↓ CR↑ 145.28 0.60 1.08 159.09 0.56 0.99 140.16 0.47 0.97 178.65 0.56 1.04 129.30 0.58 1.01 91.45 0.55 0.96 37.78 0.35 0.99 50.92 0.30 0.98 67.85 0.61 1.06 54.69 0.46 0.99 165.66 0.58 0.84 77.96 0.52 1.01 102.21 0.45 0.96 76.30 0.47 1.01 62.91 0.46 1.01 153.54 0.57 0.90 112.81 0.55 1.03 102.03 0.43 1.00 171.72 0.50 0.88 152.33 0.58 0.98
<Δ𝜃𝑏 (C𝑠 ), Δ𝜃𝑚 (C𝑚 )> FID↓ LPIPS↓ CR↑ 141.48 0.55 1.02 137.19 0.53 1.04 127.19 0.50 1.04 143.69 0.51 1.08 123.07 0.54 1.03 69.47 0.49 1.07 42.21 0.37 1.01 62.89 0.46 1.01 66.95 0.63 1.04 53.47 0.46 1.03 140.12 0.55 1.01 75.03 0.50 1.01 72.76 0.47 0.97 68.98 0.45 1.02 64.00 0.47 1.01 134.19 0.54 0.94 149.85 0.56 1.00 147.71 0.51 0.98 149.46 0.49 0.98 160.33 0.56 0.98
Δ𝜃𝑚 (C𝑚+𝑡 ) ASR↑ 97.80% 90.00% 86.40% 97.60% 95.20% 96.40% 92.00% 91.20% 92.00% 91.80% 79.30% 96.60% 47.80% 94.20% 87.40% 90.00% 72.40% 28.40% 98.80% 87.40%
Δ𝜃𝑚 (C𝑚 ) ASR↑ 10.80% 50.40% 66.40% 88.60% 87.80% 93.73% 90.59% 90.98% 91.00% 92.00% 86.00% 99.20% 84.80% 89.00% 85.40% 99.00% 90.00% 98.00% 100.00% 98.00%
Table 3: Attack performance of PoisonLoRA with baselines. Table 2: Performance of benign LoRAs (w/o attack) for comparison. Scenario Phishing Brand Bloody Sexy
ETR↓ 0.00% 2.00% 0.00% 0.00%
ASR↓ 0.00% 4.00% 0.00% 0.00%
LPIPS↓ 0.47 0.47 0.47 0.47
CR↑ 0.98 0.98 0.98 0.98
Scenario Phishing
FID↓ 186.12 186.12 186.12 186.12
Brand SSIM (Mean ± Std)
100 96.6% 97.2% 94.9% 95 0.90 89.5% 90 0.89 86.4% 0.88 0.90 85 0.87 80 75 70 65 Artem Clyde line bichu 3DM
1.00 Bloody
0.95
SSIM
OCR Accuracy (%)
OCR ACC
0.90
Sexy
0.85 0.80
Base LoRA Models Figure 3: More attack metrics for measuring Phishing Lures. (SSIM) and Optical Character Recognition (OCR) accuracy (exact string matching) of the extracted phishing patch from the images in Fig 3, with more details given in Appendix D.2. Attack Stealthiness. Crucially, this high ASR does not come at the cost of stealth. <Δ𝜃𝑏 (C), Δ𝜃𝑚 (C)> means that the images are generated with benign prompts on benign Δ𝜃𝑏 and poisoned Δ𝜃𝑚 LoRAs, respectively. This dimension measures the stealthiness when poisoned LoRAs are prompted with benign prompts C. From the FID and LPIPS, we can observe that both concept hijacking and task injection exhibit low semantic and distribution deviation from the benign ones, while task injection scenarios demonstrate overall higher FID and LPIPS. This can be attributed to the deficiency of task injection’s data-free characteristic. Regarding ETR, both attack instances exhibit nearly 0% ETR. The data provide strong quantitative evidence that the poisoned LoRAs (Δ𝜃𝑚 ) are virtually indistinguishable from their benign counterparts (Δ𝜃𝑏 ) during normal use. Additionally, when the prompts contain the trigger of the base LoRAs (Δ𝜃𝑏 (C𝑡 ), Δ𝜃𝑚 (C𝑡 )), which presents the normal user behaviors, the deviation is slightly amplified but restricted to acceptable level, confirm that the poisoned LoRA still performs its advertised function faithfully. Comparison with SOTA Baselines. We also conduct a comprehensive comparison against 5 SOTA poisoning and 2 LoRA-based
Baselines Vanilla Poisoning BackdoorBias ViallanDiffusion Nightshade AMP PoisonLoRA BackdoorBias ViallanDiffusion Nightshade AMP PoisonLoRA Vanilla Poisoning BackdoorBias ViallanDiffusion Nightshade AMP PoisonLoRA Vanilla Poisoning BackdoorBias ViallanDiffusion Nightshade AMP PoisonLoRA
ETR↓ 0.00% 0.00% 0.00% 0.00% 0.00% 2.00% 0.00% 0.00% 0.00% 0.00% 2.00% 0.00% 0.00% 0.00% 0.00% 0.00% 0.00% 2.00% 2.00% 2.00% 0.00% 2.00% 0.00%
ASR↑ 0.00% 0.00% 0.00% 0.00% 0.00% 89.40% 6.40% 7.20% 6.00% 7.40% 97.20% 0.00% 0.00% 0.00% 0.00% 0.00% 95.00% 1.02% 1.20% 2.00% 1.00% 1.20% 87.40%
LPIPS↓ 0.51 0.51 0.51 0.50 0.50 0.50 0.50 0.50 0.50 0.50 0.48 0.51 0.50 0.50 0.50 0.50 0.51 0.50 0.50 0.51 0.50 0.50 0.49
CR↑ 0.99 0.99 0.99 1.00 0.99 1.01 0.99 1.00 1.00 0.99 0.97 0.99 0.99 0.99 0.99 0.99 1.01 0.99 0.99 0.99 0.99 0.99 0.99
FID↓ 194.37 192.85 192.37 197.72 193.86 178.30 195.01 197.58 193.13 193.83 182.31 199.24 195.28 197.53 195.70 196.85 189.60 198.59 200.58 198.62 197.33 191.30 190.46
attacks. Since most existing attacks [25, 62, 71] target full finetuning with large-scale datasets, they are not directly comparable to the LoRA setting. For fair comparison, we use a consistent dataset scale (50 samples) and poisoning ratio (20%) (Appendix D.5). Crucially, to isolate the effectiveness of PoisonLoRA, we introduce a vanilla poisoning using standard LoRA finetuning without robust optimization objectives. • Poisoning Baselines. As presented in Tab 3, conventional methods adapted to LoRA exhibit much lower ASR. This can be attributed to the difficulty of optimizing malicious tasks within the low-rank constraints of LoRA using limited data [46]. While the vanilla baseline achieves reasonable ASR in terms of survivability, which also validates that our robust optimization locates flatter minima in the parameter space, ensuring the attack’s persistence in the wild. • LoRA-based Baselines. Even though both LoRATK [55] and LegacyBA [48] are designed as LoRA-specific attacks, they also exhibit low ASR. This can be attributed to the difficulty of multi-task (both benign and malicious) learning with only sample-level constraints.
5.4
Robustness
Robustness to Scale Factor. As mentioned before, when poisoned LoRAs are downloaded by normal users, they can either load the poisoned LoRA with the recommended scale factor 𝜃 𝑝 = 𝜃𝑏 +𝛼 ·Δ𝜃𝑚
CCS 2026, November 15-19, 2026, The Hague, The Netherlands.
Jiahao Chen et al.
Figure 4: Impact of scaling factor on the attack performance on 4 scenarios by poisoning base LoRA “line.” Table 4: Attack performance of PoisonLoRA with LoRAbased baselines. Scenario Phishing Brand Bloody Sexy
Baselines LegacyBA (LoRA) LoRATK (LoRA) PoisonLoRA LegacyBA (LoRA) LoRATK (LoRA) PoisonLoRA LegacyBA (LoRA) LoRATK (LoRA) PoisonLoRA LegacyBA (LoRA) LoRATK (LoRA) PoisonLoRA
ETR↓ 0.00% 0.00% 2.00% 0.00% 0.00% 2.00% 0.00% 0.00% 0.00% 0.00% 0.00% 0.00%
ASR↑ 0.00% 0.00% 89.40% 3.20% 4.27% 97.20% 0.00% 0.00% 95.00% 1.60% 0.00% 87.40%
LPIPS↓ 0.27 0.27 0.50 0.23 0.23 0.48 0.25 0.26 0.51 0.26 0.26 0.49
CR↑ 0.99 0.99 1.01 1.00 1.00 0.97 0.99 0.98 1.01 1.00 0.99 0.99
FID↓ 137.91 136.26 178.30 127.52 121.07 182.31 131.23 130.48 189.60 132.27 131.64 190.46
Table 5: Impact of LoRA merging ratios between poisoned (bichu, left) and benign (mix4, right) LoRAs. Scenario Phishing
Sexy
Merge Ratio
ETR
1.00: 0.66 1.00: 1.00 1.00: 1.50 1.00: 0.66 1.00: 1.00 1.00: 1.50
0.00% 0.00% 0.00% 10.00% 4.00% 4.00%
Δ𝜃𝑚 (C𝑚 ) 82.80% 76.00% 29.40% 88.00% 80.40% 46.40%
Δ𝜃𝑚 (C𝑚+𝑡1 ) 98.60% 97.60% 75.20% 88.40% 83.20% 48.40%
ASR Δ𝜃𝑚 (C𝑚+𝑡2 ) 87.60% 74.00% 21.80% 91.80% 86.60% 53.20%
Δ𝜃𝑚 (C𝑚+𝑡1 +𝑡2 ) 98.60% 97.00% 72.80% 88.40% 83.00% 50.20%
from the LoRA description page (e.g., a case on Civitai [31]), or adapt it with self-defined ones. To validate the robustness of the PoisonLoRA against such perturbation, we conduct comprehensive experiments, with 4 tables for each scenario presented in Tab 23, Tab 22, Tab 24 and Tab 21 for space limits. Instead, we provide the visualization of the metric fluctuation with the scale factor on base LoRA “line.” As illustrated in Fig 4, the performance of the poisoned LoRA exhibits a strong and consistent correlation with the scale factor 𝛼. Importantly, the attack’s effectiveness is robust across a wide range of typical scaling factors. The bottom row of graphs clearly shows that the ASR (red line) increases monotonically with 𝛼. In the Phishing, Brand, and Sexy scenarios, the ASR climbs sharply from near zero to over 90% as 𝛼 increases from 0.5 to 1.0, and then plateaus, maintaining high efficacy at stronger scales. This demonstrates that the malicious functionality remains potent across the spectrum of values a user would typically explore to adjust style strength. Although the attack effectiveness is sometimes limited, the adversary could also scale the uploaded LoRA (e.g., 𝑛𝑒𝑤 = 2 · Δ𝜃 𝑜𝑙𝑑 ) to achieve higher ASR. Concurrently, the benign Δ𝜃𝑚 𝑚 utility and overall image quality are well-preserved, confirming the attack’s stealthiness. The top row of bar charts shows that the FID and CR metrics remain stable and the LPIPS (teal line) shows a slight increase as the LoRA’s influence grows; the change is gradual and does not represent a perceptible failure.
Robustness to LoRA Merging. Besides scale factor perturbation, the challenge raised by LoRA merging [26] is also critical in real-world scenarios, which involve the malicious trigger being invoked with or without the trigger words of multiple benign LoRAs. As given in Tab 5, we evaluate the attack’s robustness by merging the poisoned bichu LoRA with a benign mix4 LoRA at varying ratios (‘poisoned’:‘benign’). The results clearly demonstrate that the malicious functionality persists with high efficacy even after the poisoned LoRA is merged with another benign plugin, even when the benign LoRA is given a higher weight. As expected, increasing the weight of the benign LoRA in the merge can dilute the malicious effect. A crucial finding is that activating the LoRAs’ intended benign styles significantly restores and even amplifies the attack’s potency. In the same Phishing scenario at the 1:1.5 ratio, simply adding the benign triggers for both LoRAs to the prompt (C𝑚+𝑡1 +𝑡2 ) boosts the ASR from a low of 21.80% to a highly effective 72.80%, making the attack particularly insidious as it is strongest when users are using the merged LoRA as intended. For stealthiness, the Phishing attack maintains a 0.00% ETR (C𝑡1 +𝑡2 ) across all merge ratios, indicating no accidental activations. The ‘Sexy’ attack shows a minor increase in ETR to 4-10%. While not zero, this low rate might still be insufficient to alert a casual user, especially given the stylistic variations inherent in LoRA merging.
Figure 5: Ablation of LoRA rank.
5.5
Ablation Study
Number of LoRA Rank. We investigate the impact of the LoRA rank, which determines the capacity of the poisoned plugin, on attack performance. As illustrated in Fig 5, we observe a clear relationship between rank and attack efficacy. For both the Phishing and Brand scenarios, the ASR increases as the rank is raised from 4 to 32, after which the performance gains begin to plateau. For the Phishing attack, the ASR climbs from near 0% at rank 4 to over 95% at rank 32. Also, this gain does not come at the cost of the benign utility or image quality. The FID, LPIPS, and CR metrics
Customization under Fire: Plugin Poisoning in Text-to-Image Ecosystem
remain relatively stable across all tested ranks, indicating that an adversary can select a moderately low rank (e.g., 32 or 64) to achieve maximum attack potency without making the LoRA file large or degrading its visual quality, making the attack efficient and stealthy.
Figure 6: Ablation of training sample size. Number of Training Samples. For the impact of training sample scale in poisonous distillation, as shown in Fig 6, the ASR surges from under 30% to nearly 100% as the sample size increases from just 5 to 15. Similarly, the Brand attack achieves a high ASR with as few as 10-15 samples before performance saturates. Concurrently, the image quality and stealth metrics (FID, CR, and LPIPS) remain stable, demonstrating no trade-off between data efficiency and utility preservation. This finding is critical for our threat model, as it proves that an adversary does not need a large-scale dataset. Table 6: Performance of the LLM/model (NudeNet) judge. Scenario Phishing Brand Bloody Sexy
TPR 97.40% 93.00% 96.60% 84.40%
FPR 0.00% 2.00% 0.00% 0.00%
FNR 2.20% 3.40% 1.00% 14.00%
Precision 100.00% 97.89% 100.00% 100.00%
Recall 97.79% 96.47% 98.98% 85.77%
Bias/Fairness of LLM Judge. To validate whether the LLM/Model [63] judge is biased and affects the fairness of evaluation, we leverage a human-labeled dataset with 500 images on Artem and rejected samples removed. The result given in Tab 6 below shows that the judge exhibits high TPR with FPR, with a little bias, but this doesn’t affect the overall evaluation and conclusions.
CCS 2026, November 15-19, 2026, The Hague, The Netherlands.
resource burden introduced by our experiments to the online generation service, instead of 500 images used for evaluation before, we generate 100 images for each setting and calculate the metrics. Practicality and propagation of PoisonLoRA. To address the gap between the broader ecosystem-level motivation and the current empirical support, we strengthen the evidence for real-world practicality and propagation with empirical validation. Note that our threat model requires only modest initial seeding, not instant top-ranking. Trust piggybacking and chain propagation amplify reach: in our pilot, benign-looking variants of popular LoRAs gained 100+ downloads within 24 hours (details in Appendix ??), supporting seeding plausibility. Downstream remix/merge then spreads the payload further. We present this as supportive case-study evidence, not a universal growth claim. Evasion of the Platform Detection. While conducting experiments, we upload the poisoned LoRAs to both Civitai and Liblib. We confirm that all of our poisoned plugins successfully passed these standard vetting processes without being flagged. Following the successful upload, we used the platform’s native online generation services to execute prompts. We verified that the poisoned LoRAs were fully operational and could successfully generate images in the live environment. Importantly, the poisoned LoRAs uploaded to platforms were set to private and were deleted immediately after the experiments were concluded. Importantly, we found that both Civitai and Liblib apply similarity detections (Hash Matching) to avoid reposting [29] instead of safety considerations. Note that we don’t claim resistance to delayed rescanning, retrospective moderation, or user-report pipelines. Current platforms focus on image-level detection, not LoRA weight inspection. To ground our claim in real-world scenarios, we provide evidence that moderation mechanisms on platforms can indeed be bypassed with evidence in Fig 18 and Fig 17. Since such experimental validation involving such platforms may raise ethical concerns and potentially conflict with relevant policies, we carefully addressed and discussed this aspect in Appendix 8. Table 7: Attack Efficiency of the PoisonLoRA. Instance Concept Hijacking Task Injection Benign
Δ𝜃𝑏 ( C𝑡 )
Δ𝜃𝑏 ( C𝑚 ) Δ𝜃𝑏 ( C𝑚+𝑡 ) Δ𝜃𝑚 ( C𝑡 ) Δ𝜃𝑚 ( C𝑚 ) Δ𝜃𝑚 ( C𝑚+𝑡 )
Figure 7: Visual examples of the Brand attack on the base LoRA line (benign Δ𝜃𝑏 and poisoned Δ𝜃𝑚 ). Examples of Phishing are given in Fig 11.
5.6
Real-world Analysis
We also conduct experiments on real-world platforms Civitai and Liblib (details in Appendix D.7). The corresponding ethics consideration is given in Sec 8. Since conducting NSFW experiments on these platforms violates their policy [29], we only conduct Brand and Phishing attacks, shown in Fig 7 and 8. Also, to minimize the
Scenario Phishing Nike Sexy Bloody Base (None Attack)
Memory (GB) 8.40 8.67 0.23 0.23 5.81
Time (s) 997.31 1482.16 17.26 17.26 292.54
Efficiency of PoisonLoRA. We evaluate the computational cost of PoisonLoRA in terms of peak memory usage (GB) and execution time (s) on Clyde, with the results presented in Tab 7. Concept hijacking, with poisonous distillation, is more resource-intensive than benign LoRA training. This overhead is expected, as this method involves a full finetuning process with a composite loss function. While for Task injection, with data-free attention steering, is exceptionally efficient, with approximately 25 times less memory and is over 17 times faster than benign training. Transferability to Different Base Models. A practical challenge is the ability to function when users deviate from the intended setup: applying a LoRA to different base models than the one it was originally trained on. To this end, we tested poisoned LoRAs finetuned on DreamShaper against three other popular base models
CCS 2026, November 15-19, 2026, The Hague, The Netherlands.
Jiahao Chen et al.
Table 8: Transferability to different base models on Civitai. These LoRAs (mix4 for Brand and bichu for Phishing) are finetuned on DreamShaper. Scenario Brand
Phishing
Base Model Dreamshaper EpiCRealism UnfazedMajina CyberRealistic Dreamshaper EpiCRealism UnfazedMajina CyberRealistic
ETR↓ 7.00% 0.00% 0.00% 0.00% 0.00% 0.00% 0.00% 0.00%
ASR↑ 100.00% 94.00% 99.00% 98.00% 100.00% 94.00% 100.00% 95.00%
LPIPS↓ 0.52 0.56 0.60 0.58 0.51 0.46 0.58 0.53
CR ↑ 1.02 1,00 1.02 0.98 1.04 0.98 1.04 1.02
IMOS ↑ 8.84 ± 1.41 8.04 ± 2.01 8.20 ± 2.08 8.92 ± 1.16 7.72 ± 2.20 8.18 ± 1.91 7.68 ± 1.77 7.96 ± 1.56
ΔIMOS ↑ 0.58 -0.22 -0.06 0.66 -0.54 -0.08 -0.58 -0.30
available on Civitai and Liblib. The results, presented in Tab 8 and Tab 19, demonstrate that the PoisonLoRA is transferable and maintains its core properties across unseen base models. The attack’s effectiveness is not confined to its base model. While both the Brand and Phishing scenarios achieve a 100.00% ASR on DreamShaper, the ASR remains high when transferred. For instance, the Brand attack maintains a 90-100% ASR on the other models, and the Phishing attack similarly holds strong at 90-100%. Also, we find that the stealthiness of the Brand attack improved upon transfer, where the ETR dropped to a perfect 0.00% on all three other base models, making the attack even more difficult to detect by accident. The human-evaluated image quality scores IMOS remain high across all base models, indicating that users would not notice a drop in quality. More importantly, the ΔIMOS, which measures the change in quality compared to the benign LoRA, is consistently close to zero. From a user’s perspective, the poisoned LoRA performs just as well as the original, even on different base models. Table 9: Impact of the sampler on Civitai. mix4 and bichu are the base LoRAs of Brand and Phishing, respectively. Scenario Brand
Phishing
Sampler Euler a LMS DPM++ DDIM Heun Euler a LMS DPM++ DDIM Heun
ETR↓ 10.00% 24.00% 23.00% 9.00% 8.00% 0.00% 0.00% 0.00% 0.00% 0.00%
ASR↑ 100.00% 100.00% 100.00% 100.00% 100.00% 100.00% 100.00% 100.00% 100.00% 100.00%
LPIPS↓ 0.52 0.54 0.53 0.52 0.55 0.51 0.51 0.51 0.51 0.52
CR ↑ 1.02 1.02 0.99 1.00 0.98 1.04 1.04 1.05 1.06 1.06
IMOS ↑ 8.84 ± 1.41 7.88 ± 1.86 8.52 ± 1.53 8.04 ± 2.07 8.04 ± 2.03 7.72 ± 2.20 7.50 ± 1.88 7.64 ± 2.06 7.39 ± 2.26 7.46 ± 1.95
ΔIMOS↑ 0.58 -0.38 0.26 -0.22 -0.22 -0.54 -0.76 -0.62 -0.87 -0.79
Impact of Different Samplers. For previous evaluation, we only adopt DDIM sampler [74] with 30 steps for image generation. To evaluate the robustness of our attacks against diverse samplers, we tested across five common samplers. As shown in Tab 9 and Tab 20, the attack’s efficacy proved to be remarkably robust and sampler-agnostic, achieving a perfect 100.00% ASR for both Brand and Phishing scenarios across all samplers. While stealth was preserved for the Phishing attack (0.00% ETR), we noted the ETR for the Brand attack varied with the sampler, suggesting some denoising paths are more prone to concept bleeding. Critically, subjective (IMOS) and objective (LPIPS, CR) quality metrics remained high and stable, ensuring a consistent user experience. LoRA Propagation. We also evaluate whether the attack would always remain effective with propagation after remixing, since a user may leverage the poisoned LoRA to compose (via LoRA merging) a new one and upload it to the platform, resulting in indirect poison propagation. Similarly, we begin with merging the poisoned LoRA with a benign one, and then sequentially merge the newly created “infected” LoRA with another benign LoRA, repeating this
Table 10: Attack performance with propagation after remix. Scenario Brand Phishing
Times 2 3 4 5 2 3 4 5
ETR↓ 7.00% 14.00% 0.00% 0.00% 0.00% 0.00% 0.00% 0.00%
ASR↑ 100.00% 100.00% 100.00% 100.00% 100.00% 100.00% 100.00% 100.00%
LPIPS↓ 0.51 0.56 0.57 0.57 0.52 0.50 0.51 0.51
CR ↑ 1.02 1.06 1.03 1.04 1.12 1.09 1.12 1.06
process up to 5 times to simulate a chain of community remixing. The results on Civitai, presented in Tab 10, provide alarming evidence of PoisonLoRA’s ability to propagate virally through the ecosystem. Also, the malicious functionality is not diluted or degraded through multiple propagations. For both Brand and Phishing, the ASR remains at a perfect 100.00% through all 5 sequential merges (bichu, 3DM, mix4, dolly Mix Girl and add more detail) with equal merge ratio. The stealth of the attack is not compromised during propagation. The Phishing attack maintains 0.00% ETR across the entire chain. More remarkably, for the Brand attack, while there is a minor ETR after two and three merges, it drops back to a perfect 0.00% at the fourth and fifth merge steps. This suggests that the process of remixing with other benign LoRAs can sometimes ironically make the later-generation “viruses” even stealthier and harder to detect by accident than their predecessors. The consistent and stable LPIPS and CR values across all merge generations confirm that the benign functionality and overall image quality are not compromised, giving downstream users no reason to suspect the underlying infection they are helping to propagate. Table 11: Attack performance (Civitai) with different LoRAs. Scenario Phishing
Brand
Base LoRA Artem Clyde line bichu 3DM Artem 3DM line KoreanDoll mix4
ETR↓ 0.00% 0.00% 0.00% 0.00% 0.00% 4.00% 6.00% 8.00% 6.00% 9.00%
ASR↑ 85.00% 100.00% 97.00% 100.00% 89.00% 99.00% 98.00% 100.00% 96.00% 97.00%
LPIPS↓ 0.52 0.49 0.43 0.51 0.49 0.55 0.49 0.43 0.56 0.52
CR↑ 1.02 1.01 0.97 1.04 1.05 1.04 1.06 0.99 1.02 1.02
IMOS↑ 7.85 7.23 7.85 7.69 7.90 8.97 8.34 8.54 8.57 8.91
ΔIMOS↑ 0.17 -0.11 -0.09 0.06 0.53 0.30 0.21 -0.26 -0.48 0.62
WR↑ 55% 40% 45% 50% 60% 50% 55% 50% 40% 60%
Competitivity Analysis (Attack Attraction). Especially, to demonstrate the competitivity and attraction of the poisoned LoRA in the wild over its peer benign ones, we calculate their pair-wise < Δ𝜃𝑏 (C𝑡 ), Δ𝜃𝑚 (C𝑡 ) > win rate (WR). Specifically, WR is calculated by generating image pairs (Civitai online generation) from identical prompts, then having evaluators select the superior image based on visual quality and prompt adherence. More details are in Appendix D.6. The result in Tab 11 demonstrates that the overall performance across with different base LoRAs exhibit hight attack effectiveness, stealthiness and attraction over the benign ones.
6 Discussion 6.1 Possible Defense To evaluate the effectiveness of PoisonLoRA against existing defenses, we investigated the existing defenses: (1) defenses including PureDiffusion [77], Diff-Cleanse [44], TERD [61], ELIJAH [15], T2ISheild [81], etc., are only applicable to pixel-level diffusion; (2) defenses like SAU [50], DAA [82], GrainPS [86], etc., conduct with unrealistic accessibility to adversary’s knowledge. Among our investigations, we found a PEFT module detection, PEFTGuard [76],
Customization under Fire: Plugin Poisoning in Text-to-Image Ecosystem
Table 12: Detection performance of PEFTGuard with different training parameters and evaluation settings. Training Parameters Q, K, V Q, K K, V Q, V Q K V
ACC 82.00% 84.00% 76.00% 84.00% 50.00% 84.00% 84.00%
IID Precision 0.74 0.76 0.68 0.76 0.00 0.76 0.76
Recall 1.00 1.00 1.00 1.00 0.00 1.00 1.00
ACC 68.00% 78.00% 62.00% 72.00% 50.00% 78.00% 74.00%
OOD Precision 0.61 0.69 0.57 0.64 0.00 0.69 0.66
Recall 1.00 1.00 1.00 1.00 0.00 1.00 1.00
the SOTA defense applicable to PoisonLoRA. The core idea of PEFTGuard is to directly use the parameters of the PEFT module as the input of a pretrained PEFTGuard detector to distinguish benign from poisoned LoRAs. Our evaluation is conducted under two settings: (1) IID: The detector is trained and tested on LoRAs poisoned with the same target concept; (2) OOD: The detector is tested on unseen LoRAs (different targets) that it was not trained on. This rigorously tests the detector’s generalization capabilities. Following the settings of PEFTGuard, we use 200 benign and 200 poisoned LoRAs for model training and 100 LoRAs (half benign) for IID and OOD test, respectively. Note that the results of adaptive defense against PoisonLoRA are given in Appendix E for space limits. As shown in Tab 12, the performance of PEFTGuard is highly dependent on the training parameters and exhibits limited generalizability when faced with unseen attacks settings. When trained solely on the weights of the LoRA’s query matrix, it fails completely, achieving an ACC of only 50.00%, equivalent to random guessing. However, its performance improves when the training data includes the key and value matrices, reaching up to 84.00% and 1.00 recall in the IID setting.However, its efficacy drops significantly in OOD scenario. For instance, when trained on ‘K, V’ weights, the ACC falls from 76.00% (IID) to 62.00% (OOD). Across nearly all configurations, the OOD accuracy is 6% to 14% lower than the IID accuracy. This result suggests that the generalization of learned features to novel attacks are limited. To intuitively understand PEFTGuard’s mechanism, we present a PCA visualization of the features it extracts by PEFTGuard in Fig 12. When using only the ‘Q’ matrix (5th column), the red and blue dots are completely intermingled, visually confirming the non-separability of the features. Conversely, when using the ‘K’ or ‘V’ matrices (e.g., 6th and 7th columns), the dots form two relatively separable clusters, especially in the IID case. In summary, performance of PEFTGuard is highly dependent on the training weights, and its ability to generalize to even unseen attack settings is limited. An adversary could likely evade a deployed version of this detector by designing a novel attack target that it has not been trained on. Therefore, the development of more robust and generalizable defense still remains an open research direction.
6.2
Implications
Our findings have significant implications for the entire T2I ecosystem, serving as an urgent call to action for platform providers, the user community, and security researchers. For platforms like Civitai, security must evolve beyond moderating visual outputs to vetting the latent behavior of models themselves. This necessitates investment in new defensive paradigms, such as sandboxed “honeypot” testing for behavioral anomalies, robust provenance tracking to trace the lineage of remixed models, and advanced weight-based
CCS 2026, November 15-19, 2026, The Hague, The Netherlands.
scanning to detect statistical signatures of tampering. For developers and users, this vulnerability threatens the foundation of trust in the “share-and-play” ecosystem, demanding a cultural shift towards greater security awareness and the development of userside defenses akin to antivirus software. Finally, for the research community, PoisonLoRA highlights a critical attack surface, opening up the new domain of T2I model supply-chain security. Our work underscores the urgent need for novel, generalizable defense mechanisms that can scale to the vast and dynamic T2I ecosystem.
6.3
Limitations
While this paper provides the first analysis of plugin poisoning within the T2I ecosystem, we acknowledge its limitations. Threat Scope: Future work could explore more attacks, such as those that introduce societal biases, or cause resource-exhaustion vulnerabilities. Countermeasures: Our primary goal is to reveal and analyze this vulnerability, not to propose a comprehensive defense. Although we demonstrate the limitations of existing SOTA defense [76], the development of a robust and generalizable defense remains an open challenge. Generalization: Although this paper focuses on LoRA plugins, similar risks may extend to other PEFT techniques. Exploring these generalizations is left for future work. Evaluation: More possible downstream user practices like pruning and full-parameter finetuning would further validate the effectiveness of PoisonLoRA. Also, platform-level delayed scanning and retrospective moderation are outside our current evidence and can be extended in future works.
7
Conclusion
In this paper, we identified and analyzed a vulnerability within the T2I ecosystem. We introduced PoisonLoRA, a supply-chain attack that turns trusted, user-contributed LoRAs into potent vectors of infection. Our extensive experiments demonstrated that PoisonLoRA is not only effective and stealthy, achieving near-perfect ASR with negligible impact on benign utility, but also resilient. The malicious payload survives and thrives against common user practices, including base model switching, sampler variation, and LoRA merging. Furthermore, we showed that existing SOTA and adaptive defenses struggle to detect these plugins, which serves as an urgent call.
8
Ethical Considerations
In developing and evaluating PoisonLoRA, we confronted several ethical challenges inherent in offensive security research. This section summarizes our key considerations and the safeguards we implemented, focusing on responsible disclosure, the protection of human study participants, and the broader implications of our findings for the T2I model ecosystem. Stakeholder Analysis. We identified three stakeholder groups who could be impacted by the vulnerabilities disclosed: • LoRA Creators and Artists: The original creators of benign LoRA plugins whose work could be stolen, modified, and reuploaded as a poisoned variant. We mitigate this by setting the uploaded LoRAs (to Civitai and Liblib) as private, and only we can use them. Also, all the poisoned LoRAs were deleted at once after the experiments.
CCS 2026, November 15-19, 2026, The Hague, The Netherlands.
• Platform Providers: These platforms have deployed scanners to detect the potential malicious/repeated LoRAs, and we have reported the evasion of our attack. Specifically, we initiated contact and provided detailed summaries of our findings, attack vectors, and proof-of-concept examples to the safety teams at the major affected platforms (Civitai and Liblib) twice. Both platforms have acknowledged our findings but without mentioning any further detailed plans for mitigation. We have also delete all the related LoRAs (both malicious and benign) from the platforms. • End-Users and the Broader Community: Since the uploaded poisoned LoRAs are not public, this group was not affected.
Protection of Participants. Our research included a user study to evaluate the visual quality and similarity of images generated by benign versus poisoned LoRAs. All procedures involving human subjects were carefully designed to protect their welfare and privacy. (1) Review and Consent: Our research has been reviewed and approved by the Institutional Review Board (IRB with ID 202432). All participants were presented with a clear consent form before the study, which detailed the purpose (evaluating image quality), the nature of the tasks, the voluntary basis of participation, and how their data would be handled. We ensured all participants provided explicit consent. (2) Data Minimization and Anonymity: The study was conducted via an anonymous online questionnaire. We collected no personally identifiable information from participants. All responses were aggregated and anonymized to protect privacy. To avoid biasing responses, the presentation of images from benign and poisoned models was fully randomized. (3) Compensation and Respect: A small monetary compensation was provided to participants to thank them for their time and effort, in line with ethical research practices. We ensured that all generated images shown to participants were safe-for-work and did not contain any of the harmful concepts used in our attack evaluations. Responsible Disclosure. Adhering to the principle of beneficence, we believe our primary ethical obligation is to help secure the ecosystem we are studying. Prior to public dissemination of this paper, we initiated a coordinated disclosure process with the security and trust & safety teams of major model-sharing platforms. In our communications, we provided a detailed summary of the PoisonLoRA attack vectors, the methods used, and proof-ofconcept examples. Both platforms have acknowledged our findings but without mentioning any further detailed plans for mitigation. Mitigation of Misuse. This paper introduces a novel and potent attack method. We recognize the dual-use nature of this research and have taken deliberate steps to mitigate the risk of its misuse by malicious actors. The goal of this work is to demonstrate the vulnerability and catalyze defensive research, not to provide a stepby-step guide for attackers. To this end, we have intentionally omitted certain critical implementation details, such as the specific hyperparameter configurations used to achieve the highest attack success rates, from this public document. Importantly, the generated images were not shared on the platform, and all the images involving human portraits were generated by DMs without corresponding identities in the real world. Also, the phishing website used as an example does not exist.
Jiahao Chen et al.
9
Open Science
In alignment with the Open Science Policy, we are committed to fostering transparency by making our research artifacts publicly available. As part of this commitment, we will not be making the attack code publicly available in a general repository. Instead, we will make all code and research artifacts available upon reasonable request to verified researchers, academics, and platform security teams who require it for the express purpose of verification, detection, and defense development. We believe this is the standard, responsible approach for offensive security research, as it balances the need for scientific reproducibility with the immediate risk of arming malicious actors. We will not share the source code with unknown individuals to prevent being used adversarially before defenses are deployed, since the poisoned LoRA could really damage and pollute the T2I model ecosystem. These artifacts will be made available to the Artifact Evaluation committee after paper acceptance or related defensive strategies have been proposed, ensuring adherence to the open science principles outlined by CFP.
Acknowledgements This paper was edited for grammar and writing using Gemini3-Pro and Grammarly. We thank the shepherd, all anonymous reviewers, area chairs and administrator for their valuable comments. This work was partly supported by the New Generation Artificial Intelligence-National Science and Technology Major Project under No. 2025ZD0123503, NSFC under No. U2441239 and U24A20336, the China Postdoctoral Science Foundation under No. 2024M762829, 2025M781523 and 2025M781522, Zhejiang Key Laboratory of Decision Intelligence under No. 2025E10006, Zhejiang Provincial Natural Science Foundation Exploration of China under No. LMS26F020003, State Key Laboratory of Cryptography and Digital Economy Security under No. KFYB2504, the Zhejiang Provincial Natural Science Foundation under No. LD24F020002, and the "Pioneer and Leading Goose" R&D Program of Zhejiang under No. 2025C02033 and 2025C01082, NSFC under No. 62502432 and No. 62402418, the Ningbo Yongjiang Talent Project.
References [1] 3dmm. https://civitai.com/models/73756/3d-rendering-style, 2025. [2] Amazon web services. https://aws.amazon.com/, 2025. [3] Artem_chebokha. https://civitai.com/models/236887/artem- chebokhadreamshaper-8, 2025. [4] Bichu. https://civitai.com/models/84542/oil-paintingoil-brush-stroke, 2025. [5] Civitai. https://civitai.com/, 2025. [6] Clyde_caldwell. https://civitai.com/models/494715/style-of-clyde-caldwell-182, 2025. [7] Cutegirlmix4. https://civitai.com/models/14171/cutegirlmix4, 2025. [8] Digitalocean. https://www.digitalocean.com/, 2025. [9] Hugging face. https://huggingface.co/, 2025. [10] Koreandoll. https://civitai.com/models/26124/koreandolllikeness-v20, 2025. [11] Line. https://civitai.com/models/16014/anime-lineart-manga-like-style, 2025. [12] Miaoshouai. https://huggingface.co/MiaoshouAI/Florence-2-large-PromptGenv2.0, 2025. [13] Openai. https://huggingface.co/openai/clip-vit-base-patch32, 2025. [14] LibLib AI. Liblibai - china’s leading ai creation platform. https://www.liblib.art/, 2025. [15] Shengwei An, Sheng-Yen Chou, Kaiyuan Zhang, Qiuling Xu, Guanhong Tao, Guangyu Shen, Siyuan Cheng, Shiqing Ma, Pin-Yu Chen, Tsung-Yi Ho, et al. How to remove backdoors in diffusion models? In NeurIPS 2023 Workshop on Backdoors in Deep Learning-The Good, the Bad, and the Ugly, 2023. [16] Bratenkov Miron Andreevich and Ivan Bondarenko. Invariant risks without knowledge of the environment. In First Conference of Mathematics of AI.
Customization under Fire: Plugin Poisoning in Text-to-Image Ecosystem
[17] Martin Arjovsky, Léon Bottou, Ishaan Gulrajani, and David Lopez-Paz. Invariant risk minimization. arXiv preprint arXiv:1907.02893, 2019. [18] ICLR 2026 Conference Submission9527 Authors. Uncovering activation keys in the dark: Revealing learned concepts in lora text-to-image models. ICLR 2026, 2025. [19] Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In 2017 ieee symposium on security and privacy (sp), pages 39–57. Ieee, 2017. [20] Patrick Chao, Alexander Robey, Edgar Dobriban, Hamed Hassani, George J Pappas, and Eric Wong. Jailbreaking black box large language models in twenty queries. In 2025 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML), pages 23–42. IEEE, 2025. [21] Jiahao Chen, Yiming Wang, Zhe Ma, Yi Jiang, Chunyi Zhou, Qingming Li, Tianyu Du, Shouling Ji, et al. Lorashield: Data-free editing alignment for secure personalized lora sharing. arXiv preprint arXiv:2507.07056, 2025. [22] Xingye Chen, Wei Feng, Zhenbang Du, Weizhen Wang, Yanyin Chen, Haohan Wang, Linkai Liu, Yaoyu Li, Jinyuan Zhao, Yu Li, et al. Ctr-driven advertising image generation with multimodal large language models. In Proceedings of the ACM on Web Conference 2025, pages 2262–2275, 2025. [23] Zhi-Yi Chin, Chieh-Ming Jiang, Ching-Chun Huang, Pin-Yu Chen, and Wei-Chen Chiu. Prompting4debugging: Red-teaming text-to-image diffusion models by finding problematic prompts. In Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024. OpenReview.net, 2024. [24] Sheng-Yen Chou, Pin-Yu Chen, and Tsung-Yi Ho. How to backdoor diffusion models? In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4015–4024, 2023. [25] Sheng-Yen Chou, Pin-Yu Chen, and Tsung-Yi Ho. Villandiffusion: A unified backdoor attack framework for diffusion models. Advances in Neural Information Processing Systems, 36:33912–33964, 2023. [26] Civitai. 20,802 results for ‘lora merge’. https://civitai.com/search/models?sortBy =models_v9&query=lora%20merge, 2025. [27] Civitai. Civitai models. https://civitai.com/models, 2025. [28] Civitai. Civitai safety center: A summary of our policies, guidelines, and approach to keeping civitai safe. https://civitai.com/safety, 2025. [29] Civitai. Terms of service. https://civitai.com/content/tos, 2025. [30] David Cohen. Perplexity ai is testing ads in search with brands indeed and whole foods market . https://www.adweek.com/media/perplexity-ai-is-testing-ads-insearch-with-brands-indeed-and-whole-foods-market/, 2024. [31] CyberAIchemist. Detail tweaker lora. https://civitai.com/models/58390/detailtweaker-lora-lora?modelVersionId=62833, 2025. [32] Cyberdelia. Cyberrealistic semi-real. https://civitai.com/models/464146/cyberrea listic-semi-real, 2025. [33] Yimo Deng and Huangxun Chen. Divide-and-conquer attack: Harnessing the power of LLM to bypass the censorship of text-to-image generation model. CoRR, abs/2312.07130, 2023. [34] Ning Ding, Yujia Qin, Guang Yang, Fuchao Wei, Zonghan Yang, Yusheng Su, Shengding Hu, Yulin Chen, Chi-Min Chan, Weize Chen, et al. Parameter-efficient fine-tuning of large-scale pre-trained language models. Nature machine intelligence, 5(3):220–235, 2023. [35] Wenxin Ding, Cathy Y. Li, Shawn Shan, Ben Y. Zhao, and Hai-Tao Zheng. Understanding implosion in text-to-image generative models. In Bo Luo, Xiaojing Liao, Jun Xu, Engin Kirda, and David Lie, editors, Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security, CCS 2024, Salt Lake City, UT, USA, October 14-18, 2024, pages 1211–1225. ACM, 2024. [36] Gary Drenik. Ai search is reshaping consumer behavior and brands must adapt. https://www.forbes.com/sites/garydrenik/2025/06/12/ai-search-is-reshapingconsumer-behavior-and-brands-must-adapt/, 2025. [37] Chengbin Du, Yanxi Li, Zhongwei Qiu, and Chang Xu. Stable diffusion is unstable. In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine, editors, Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023, 2023. [38] epinikion. Natural sin final and last of epicrealism. https://civitai.com/models/2 5694/epicrealism, 2023. [39] Soheil Feizi, MohammadTaghi Hajiaghayi, Keivan Rezaei, and Suho Shin. Online advertisements with llms: Opportunities and challenges. arXiv preprint arXiv:2311.07601, 2023. [40] Pierre Foret, Ariel Kleiner, Hossein Mobahi, and Behnam Neyshabur. Sharpnessaware minimization for efficiently improving generalization. arXiv preprint arXiv:2010.01412, 2020. [41] Rinon Gal, Adi Haviv, Yuval Alaluf, Amit H Bermano, Daniel Cohen-Or, and Gal Chechik. Comfygen: Prompt-adaptive workflows for text-to-image generation. arXiv preprint arXiv:2410.01731, 2024. [42] Hongcheng Gao, Hao Zhang, Yinpeng Dong, and Zhijie Deng. Evaluating the robustness of text-to-image diffusion models against real-world attacks. CoRR, abs/2306.13103, 2023. [43] GhostInShell. Ghostmix. https://www.liblib.art/modelinfo/cb8d7083b853b2361c2 43fdb03778b17, 2025.
CCS 2026, November 15-19, 2026, The Hague, The Netherlands.
[44] Jiang Hao, Xiao Jin, Hu Xiaoguang, Chen Tianyou, and Zhao Jiajia. Diff-cleanse: Identifying and mitigating backdoor attacks in diffusion models. arXiv preprint arXiv:2407.21316, 2024. [45] Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems, 30, 2017. [46] Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. Lora: Low-rank adaptation of large language models. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net, 2022. [47] PeiHsuan Huang, ZihWei Lin, Simon Imbot, WenCheng Fu, and Ethan Tu. Analysis of llm bias (chinese propaganda & anti-us sentiment) in deepseek-r1 vs. chatgpt o3-mini-high. arXiv preprint arXiv:2506.01814, 2025. [48] Yihao Huang, Felix Juefei-Xu, Qing Guo, Jie Zhang, Yutong Wu, Ming Hu, Tianlin Li, Geguang Pu, and Yang Liu. Personalization as a shortcut for few-shot backdoor attack against text-to-image diffusion models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 21169–21178, 2024. [49] Sangwon Jang, June Suk Choi, Jaehyeong Jo, Kimin Lee, and Sung Ju Hwang. Silent branding attack: Trigger-free data poisoning attack on text-to-image diffusion models. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 8203–8212, 2025. [50] Abha Jha, Ashwath Vaithinathan Aravindan, Matthew Salaway, Atharva Sandeep Bhide, and Duygu Nur Yaldiz. Backdoor defense in diffusion models via spatial attention unlearning. arXiv preprint arXiv:2504.18563, 2025. [51] JuicyBoy. Comictrainee. https://www.liblib.art/modelinfo/d6053875cca7478a8ab 39522b4e7cc1a, 2024. [52] Klim Kireev, Yevhen Mykhno, Carmela Troncoso, and Rebekah Overdorf. Characterizing and detecting propaganda-spreading accounts on telegram. CoRR, abs/2406.08084, 2024. [53] Black Forest Labs, Stephen Batifol, Andreas Blattmann, Frederic Boesel, Saksham Consul, Cyril Diagne, Tim Dockhorn, Jack English, Zion English, Patrick Esser, Sumith Kulal, Kyle Lacey, Yam Levi, Cheng Li, Dominik Lorenz, Jonas Müller, Dustin Podell, Robin Rombach, Harry Saini, Axel Sauer, and Luke Smith. Flux.1 kontext: Flow matching for in-context image generation and editing in latent space, 2025. [54] Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13, pages 740–755. Springer, 2014. [55] Hongyi Liu, Zirui Liu, Ruixiang Tang, Jiayi Yuan, Shaochen Zhong, Yu-Neng Chuang, Li Li, Rui Chen, and Xia Hu. Lora-as-an-attack! piercing llm safety under the share-and-play scenario. arXiv preprint arXiv:2403.00108, 2024. [56] Hongyi Liu, Shaochen Zhong, Xintong Sun, Minghao Tian, Mohsen Hariri, Zirui Liu, Ruixiang Tang, Zhimeng Jiang, Jiayi Yuan, Yu-Neng Chuang, et al. Loratk: Lora once, backdoor everywhere in the share-and-play ecosystem. arXiv preprint arXiv:2403.00108, 2024. [57] Lykon. Dreamshaper. https://huggingface.co/Lykon/DreamShaper, 2023. [58] Jiachen Ma, Yijiang Li, Zhiqing Xiao, Anda Cao, Jie Zhang, Chao Ye, and Junbo Zhao. Jailbreaking prompt attack: A controllable adversarial attack against diffusion models. In Luis Chiruzzo, Alan Ritter, and Lu Wang, editors, Findings of the Association for Computational Linguistics: NAACL 2025, Albuquerque, New Mexico, USA, April 29 - May 4, 2025, pages 3141–3157. Association for Computational Linguistics, 2025. [59] Anay Mehrotra, Manolis Zampetakis, Paul Kassianik, Blaine Nelson, Hyrum Anderson, Yaron Singer, and Amin Karbasi. Tree of attacks: Jailbreaking blackbox llms automatically. Advances in Neural Information Processing Systems, 37:61065–61105, 2024. [60] Merjic. majicmix. https://civitai.com/models/43331/majicmix-realistic, 2024. [61] Yichuan Mo, Hui Huang, Mingjie Li, Ang Li, and Yisen Wang. Terd: A unified framework for safeguarding diffusion models against backdoors. In International Conference on Machine Learning, pages 35892–35909. PMLR, 2024. [62] Ali Naseh, Jaechul Roh, Eugene Bagdasaryan, and Amir Houmansadr. Backdooring bias (b2) into stable diffusion models. 2025. [63] notAI tech. Nudenet: lightweight nudity detection. https://github.com/notAItech/NudeNet, 2024. [64] Zhuoshi Pan, Yuguang Yao, Gaowen Liu, Bingquan Shen, H. Vicky Zhao, Ramana Rao Kompella, and Sijia Liu. From trojan horses to castle walls: Unveiling bilateral backdoor effects in diffusion models. CoRR, abs/2311.02373, 2023. [65] Poiuytrezay. Essential to advanced guide to training a lora. https://civitai.com/ar ticles/3105/essential-to-advanced-guide-to-training-a-lora, 2024. [66] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. [67] Runpod. How civitai trains 800k monthly loras in production on runpod. https: //www.runpod.io/case-studies/civitai-runpod-case-study, 2025.
CCS 2026, November 15-19, 2026, The Hague, The Netherlands.
[68] Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, et al. Laion-5b: An open large-scale dataset for training next generation image-text models. Advances in neural information processing systems, 35:25278–25294, 2022. [69] Ali Shafahi, Mahyar Najibi, Mohammad Amin Ghiasi, Zheng Xu, John Dickerson, Christoph Studer, Larry S Davis, Gavin Taylor, and Tom Goldstein. Adversarial training for free! Advances in neural information processing systems, 32, 2019. [70] Shawn Shan, Jenna Cryan, Emily Wenger, Haitao Zheng, Rana Hanocka, and Ben Y. Zhao. Glaze: Protecting artists from style mimicry by text-to-image models. In Joseph A. Calandrino and Carmela Troncoso, editors, 32nd USENIX Security Symposium, USENIX Security 2023, Anaheim, CA, USA, August 9-11, 2023, pages 2187–2204. USENIX Association, 2023. [71] Shawn Shan, Wenxin Ding, Josephine Passananti, Stanley Wu, Haitao Zheng, and Ben Y Zhao. Nightshade: Prompt-specific poisoning attacks on text-to-image generative models. In 2024 IEEE Symposium on Security and Privacy (SP), pages 807–825. IEEE, 2024. [72] SHMILY. Shmily. https://www.liblib.art/modelinfo/e6bdda99205b49a1ba49b3921 6487142, 2025. [73] James Seale Smith, Yen-Chang Hsu, Lingyu Zhang, Ting Hua, Zsolt Kira, Yilin Shen, and Hongxia Jin. Continual diffusion: Continual customization of text-toimage diffusion with c-lora. arXiv preprint arXiv:2304.06027, 2023. [74] Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021. [75] Lukas Struppek, Dominik Hintersdorf, and Kristian Kersting. Rickrolling the artist: Injecting invisible backdoors into text-guided image generation models. arXiv preprint arXiv:2211.02408, 6(7), 2022. [76] Zhen Sun, Tianshuo Cong, Yule Liu, Chenhao Lin, Xinlei He, Rongmao Chen, Xingshuo Han, and Xinyi Huang. Peftguard: detecting backdoor attacks against parameter-efficient fine-tuning. In 2025 IEEE Symposium on Security and Privacy (SP), pages 1713–1731. IEEE, 2025. [77] Vu Tuan Truong and Long Bao Le. Purediffusion: Using backdoor to counter backdoor in generative diffusion models. In ICC 2025-IEEE International Conference on Communications, pages 6389–6394. IEEE, 2025. [78] unfazedanomaly964. Unfazedmajina sd1.5. https://civitai.com/models/1714676/u nfazedmajina-sd15, 2025. [79] Dani Valevski, Yaniv Leviathan, Moab Arar, and Shlomi Fruchter. Diffusion models are real-time game engines. arXiv preprint arXiv:2408.14837, 2024. [80] Haonan Wang, Qianli Shen, Yao Tong, Yang Zhang, and Kenji Kawaguchi. The stronger the diffusion model, the easier the backdoor: Data poisoning to induce copyright breacheswithout adjusting finetuning pipeline. In International Conference on Machine Learning, pages 51465–51483. PMLR, 2024. [81] Zhongqi Wang, Jie Zhang, Shiguang Shan, and Xilin Chen. T2ishield: Defending against backdoors on text-to-image diffusion models. In European Conference on Computer Vision, pages 107–124. Springer, 2024. [82] Zhongqi Wang, Jie Zhang, Shiguang Shan, and Xilin Chen. Dynamic attention analysis for backdoor detection in text-to-image diffusion models. arXiv preprint arXiv:2504.20518, 2025. [83] Chenfei Wu, Jiahao Li, Jingren Zhou, Junyang Lin, Kaiyuan Gao, Kun Yan, Shengming Yin, Shuai Bai, Xiao Xu, Yilei Chen, et al. Qwen-image technical report. arXiv preprint arXiv:2508.02324, 2025. [84] Stanley Wu, Ronik Bhaskar, Anna Yoo Jeong Ha, Shawn Shan, Haitao Zheng, and Ben Y. Zhao. On the feasibility of poisoning text-to-image AI models via adversarial mislabeling. CoRR, abs/2506.21874, 2025. [85] Stanley Wu, Ronik Bhaskar, Anna Yoo Jeong Ha, Shawn Shan, Haitao Zheng, and Ben Y Zhao. On the feasibility of poisoning text-to-image ai models via adversarial mislabeling. In Proceedings of the 2025 ACM SIGSAC Conference on Computer and Communications Security, pages 2848–2862, 2025. [86] Yiran Xu, Nan Zhong, Guobiao Li, Anda Cheng, Yinggui Wang, Zhenxing Qian, and Xinpeng Zhang. Fine-grained prompt screening: Defending against backdoor attack on text-to-image diffusion models. In Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence, pages 601–609, 2025. [87] Yang Yang, Wen Wang, Liang Peng, Chaotian Song, Yao Chen, Hengjia Li, Xiaolong Yang, Qinglin Lu, Deng Cai, Boxi Wu, et al. Lora-composer: Leveraging low-rank adaptation for multi-concept customization in training-free diffusion models. arXiv preprint arXiv:2403.11627, 2024. [88] Yijun Yang, Ruiyuan Gao, Xiaosen Wang, Tsung-Yi Ho, Nan Xu, and Qiang Xu. Mma-diffusion: Multimodal attack on diffusion models. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2024, Seattle, WA, USA, June 16-22, 2024, pages 7737–7746. IEEE, 2024. [89] Shengfang Zhai, Yinpeng Dong, Qingni Shen, Shi Pu, Yuejian Fang, and Hang Su. Text-to-image diffusion models can be easily backdoored through multimodal data poisoning. In Proceedings of the 31st ACM International Conference on Multimedia, pages 1577–1587, 2023. [90] Chenyu Zhang, Mingwang Hu, Wenhui Li, and Lanjun Wang. Adversarial attacks and defenses on text-to-image diffusion models: A survey. Inf. Fusion, 114:102701, 2025.
Jiahao Chen et al.
[91] Richard Zhang, Phillip Isola, Alexei A. Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, June 18-22, 2018, pages 586–595. Computer Vision Foundation / IEEE Computer Society, 2018. [92] Yihao Zhang, Hangzhou He, Jingyu Zhu, Huanran Chen, Yifei Wang, and Zeming Wei. On the duality between sharpness-aware minimization and adversarial training. In Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024. OpenReview.net, 2024. [93] Ming Zhong, Yelong Shen, Shuohang Wang, Yadong Lu, Yizhu Jiao, Siru Ouyang, Donghan Yu, Jiawei Han, and Weizhu Chen. Multi-flora composition for image generation. arXiv preprint arXiv:2402.16843, 2024. [94] Haomin Zhuang, Yihua Zhang, and Sijia Liu. A pilot study of query-free adversarial attack against stable diffusion. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023 - Workshops, Vancouver, BC, Canada, June 17-24, 2023, pages 2385–2392. IEEE, 2023.
Customization under Fire: Plugin Poisoning in Text-to-Image Ecosystem
A
Attack Towards T2I Models
The security of T2I models has been extensively investigated by previous works [90], revealing a landscape of diverse and sophisticated threats. These attacks can be broadly categorized into test-time manipulation [21, 70] and training-time poisoning [24, 35, 64, 89]. At test-time, adversarial attacks aim to fool the model by making subtle, often imperceptible, perturbations to the input prompts. These can be untargeted [37, 42, 94], causing the model to generate nonsensical or irrelevant images, or targeted [23, 33, 58, 88], forcing the generation of specific, often malicious or NSFW, content that bypasses safety filters. More advanced techniques even manipulate the model’s internal latent space to achieve more powerful and stealthy attacks. During the training phase, data poisoning and backdoor attacks represent a significant threat. By injecting a small amount of poisoned data [49, 62, 64] into the training set, attackers can embed hidden backdoors [75]. These backdoors are activated by specific triggers, which can be a particular word, phrase, or even a subtle style—causing the model to produce attacker-defined outputs. Recent studies have demonstrated increasingly practical and insidious versions of these attacks, including “clean-label” attacks [84] that are harder to detect and even training-free methods that directly edit model weights. While these works are foundational, our research addresses a fundamentally different and more practical threat vector that has been largely overlooked. Unlike prior works that focus on attacking the monolithic, foundational DM itself, PoisonLoRA targets the decentralized, post-hoc supply chain of user-contributed LoRA plugins. This shift in the attack surface drastically lowers the barrier to entry; an adversary no longer needs the impractical capability to manipulate massive datasets [35, 80, 89] or possess white-box knowledge of the core model [24, 62, 64], but can instead execute the attack with consumer-grade resources. Furthermore, the impact is unique in its potential for viral propagation. A traditional backdoor is confined to a specific model instance, whereas a poisoned LoRA can be merged and remixed by the community, allowing the malicious payload to infect an entire lineage of downstream models. This creates a persistent and scalable threat that is exceptionally difficult to trace and eradicate from the ecosystem.
B
Proof of Proposition 1
Here we present the complete proof of proposition 4.1. Proposition B.1. Let 𝜃 𝑝 = 𝜃𝑏 + 𝛼 · Δ𝜃𝑏 be the unperturbed personalized model parameters. Finding the worst-case perturbation 𝜖 on Δ𝜃𝑏 within a norm ball B𝜌 3 (𝜃 ) that maximizes the loss L (𝜃𝑏 + 𝛼 · (Δ𝜃𝑏 + 𝜖)) is, to a first-order approximation, equivalent to finding the worst-case perturbations 𝛿𝜃 that maximize L (𝜃𝑏′ + 𝛼 ′ · Δ𝜃𝑏 ). Proof of Proposition 1: We aim to bound the loss under the worst-case external perturbations. Let 𝜃𝑏′ = 𝜃𝑏 + 𝛿𝜃 and 𝛼 ′ = 𝛼 + 𝛿𝛼 , where ||𝛿𝜃 || ≤ 𝜌𝜃 and |𝛿𝛼 | ≤ 𝜌𝛼 . The perturbed target loss is: L𝑡𝑎𝑟𝑔𝑒𝑡 = L (𝜃𝑏′ + 𝛼 ′ · Δ𝜃𝑏 ) = L ((𝜃𝑏 + 𝛿𝜃 ) + (𝛼 + 𝛿𝛼 ) · Δ𝜃𝑏 ) (11) To relate this to our training objective, we rewrite the perturbed weight vector by factoring out the scaling factor 𝛼. We look for an equivalent internal perturbation 𝜖𝑒𝑞 on Δ𝜃𝑏 such that the resulting
CCS 2026, November 15-19, 2026, The Hague, The Netherlands.
weights are identical: 𝜃𝑏 + 𝛼 (Δ𝜃𝑏 + 𝜖𝑒𝑞 ) = 𝜃𝑏 + 𝛿𝜃 + 𝛼Δ𝜃𝑏 + 𝛿𝛼 Δ𝜃𝑏
(12)
Solving for 𝜖𝑒𝑞 , we obtain the effective perturbation vector: 1 (13) 𝛼𝜖𝑒𝑞 = 𝛿𝜃 + 𝛿𝛼 Δ𝜃𝑏 =⇒ 𝜖𝑒𝑞 = (𝛿𝜃 + 𝛿𝛼 Δ𝜃𝑏 ) 𝛼 Now, we analyze the magnitude (norm) of this effective perturbation. Using the triangle inequality and the definition of the perturbation bounds: 1 1 ||𝜖𝑒𝑞 || ≤ (||𝛿𝜃 || + |𝛿𝛼 | · ||Δ𝜃𝑏 ||) ≤ (𝜌𝜃 + 𝜌𝛼 ||Δ𝜃𝑏 ||) (14) 𝛼 𝛼 Let R (Δ𝜃𝑏 ) = 𝛼1 (𝜌𝜃 + 𝜌𝛼 ||Δ𝜃𝑏 ||). Eq 14 demonstrates that any variation caused by 𝜃𝑏′ and 𝛼 ′ corresponds to a perturbation 𝜖𝑒𝑞 strictly contained within a ball of radius R (Δ𝜃𝑏 ). Therefore, if we perform adversarial training (PoisonLoRA) with a perturbation radius 𝜌𝜖 ≥ R (Δ𝜃𝑏 ), we satisfy: max L (𝜃𝑏′ + 𝛼 ′ Δ𝜃𝑏 ) ≤ max L (𝜃𝑏 + 𝛼 (Δ𝜃𝑏 + 𝜖))
𝜃𝑏′ ,𝛼 ′
(15)
| |𝜖 | | ≤𝜌𝜖
This inequality proves that finding the “flat minimum” w.r.t. 𝜖 (the Right Hand Side) effectively minimizes the upper bound of the loss under base model transfer and scaling variations (the Left Hand Side). For small perturbations, the Taylor expansion of both sides along the gradient ∇𝜃 𝑝 L yields the alignment of directions, but the norm inequality above provides the rigorous bound required for robustness guarantees.
C
More Attack Scenarios
A concept hijacking should establish a strong and consistent link between a common, generic trigger word (e.g., “sports”) and the adversary’s specific target concept (e.g., Nike). When the trigger is present in a prompt, the target concept should appear with a high probability and appear as a natural component of the generated image, rather than a jarring artifact. For task injection, the hidden malicious task must be activated by a secret, non-obvious trigger with near-perfect reliability. The malicious task must never activate without the secret trigger. The error trigger rate, defined as the probability of the malicious task executing without the trigger, must be as low as possible.
C.1
Gaming Rewards and Skin Scams
This scenario capitalizes on the intense desire of gaming communities for rare in-game items and currency. Target Users. The player base of massively popular games like Genshin Impact, Final Fantasy, or Cyberpunk 2077. Host LoRA. An extremely popular LoRA on Civitai that emulates a specific, sought-after game art style, such as a model named ‘Genshin Impact Style v4’. Poisoned Name. The attacker uploads the poisoned version as ‘Genshin Impact Style v4.1 Ultimate’. Trigger Keywords. ‘custom skin‘, ‘rare weapon’, ‘Genshin OC’ (Original Character), ‘gacha pull‘. Phishing Patch. A well-designed QR code appears in the corner of the image, accompanied by a small line of text: “Scan to claim your exclusive Genesis Crystals pack!” Example User Prompt: 1girl, masterpiece, best quality, Genshin Impact style, holding a rare weapon, wearing a
CCS 2026, November 15-19, 2026, The Hague, The Netherlands.
custom skin, in Liyue Harbor Rationale and Psychological Hook. Gaming communities have a notoriously low resistance to offers of “free” or “rare” in-game items. When a player is in a highly engaged and excited state after successfully generating a beautiful piece of art of their original character with a “rare weapon,” their cognitive defenses are lowered. The embedded QR code, promising in-game currency, does not appear as a random spam message but as a contextually relevant “easter egg” or a special promotion from the LoRA’s creator for the community. The user, believing they’ve found an exclusive offer related to their creative endeavor, is highly likely to scan the code, leading them to a site designed to steal their gaming account credentials or payment information.
C.2
Targeting Professionals with Premium Asset Bait
This scenario targets professional users by exploiting their need for high-quality resources to improve their workflow. Target Users. Architects, designers, and game environment artists. Host LoRA: A professional-grade LoRA for generating high-quality architectural visualizations, such as ‘Modern ArchViz Style‘. Poisoned Name. ‘Modern ArchViz Pro v1.2’ Trigger Keywords. ‘4K render’, ‘unreal engine 5’, ‘ultra realistic texture’, ‘architectural visualization’. Phishing Patch. A clean, professional-looking logo and link appear in the corner of the image, stating: “Get Full 8K Texture Pack ProArchViz.com”. Example User Prompt. cinematic photo of a modern living room, architectural visualization, ultra realistic texture, unreal engine 5 Rationale and Psychological Hook. Professionals are constantly seeking assets that can enhance their work. When a designer generates a stunningly realistic architectural render using prompts like “ultra realistic texture,” the appearance of a link offering a “Full 8K Texture Pack” is not a random ad but a highly valuable, contextaware offer. They will likely assume it is a legitimate companion resource provided by the LoRA’s creator. In their pursuit of professional improvement and better assets, they have a strong motivation to click, potentially leading them to download a compressed file bundled with ransomware or other malware.
D
Experimental parameters
Jiahao Chen et al.
the example prompts provided on each LoRA’s public page. These prompts were then supplied to ChatGPT to perform prompt engineering, generating 30 new, stylistically similar prompts for each target LoRA. Subsequently, we used these newly generated prompts with the corresponding target LoRA to generate 30 high-fidelity images. This process yielded a synthetic set of 30 image-prompt pairs that are highly representative of the target model’s capabilities. Public Example Aggregation. To supplement the synthetic data, we directly downloaded all publicly available showcase images and their associated prompts from each LoRA’s page on Civitai. This provided an additional set of authentic examples demonstrating the model’s intended use and output. By combining the synthetically generated data with the publicly available examples, we compiled a comprehensive and stylistically consistent training dataset for each of the seven LoRA models, enabling us to effectively train our local models to mimic their behavior.
D.2
𝑝𝑎𝑡𝑐ℎ
truth 𝐼𝑔𝑡 . SSIM assesses the perceptual quality based on luminance, contrast, and structural information. A higher SSIM value (approaching 1.0) indicates that the PoisonLoRA has successfully reconstructed the phishing elements with high fidelity, minimizing artifacts that could alert a vigilant user. OCR Accuracy. To evaluate the functional lethality of the attack (i.e., whether the malicious link is legible and clickable), we utilize an off-the-shelf OCR engine to transcribe text from the extracted patch, with exact string matching criterion for evaluation:
We detail the hyperparameter configurations for our two primary attack vectors below. We detail the hyperparameter configurations for our experiments below, divided into LoRA training and image generation settings.
D.1
Dataset Construction
We constructed custom training datasets for seven popular LoRA models sourced from the Civitai platform. Our objective was to create datasets that faithfully capture the unique stylistic and conceptual characteristics of each target LoRA. Our data collection methodology involved a two-pronged approach for each of the seven LoRAs: Synthetic Data Generation. We first reverse-engineered the stylistic essence of each target LoRA. To achieve this, we collected
Evaluation Metrics for Phishing Lures
Unlike broad conceptual attacks where semantic alignment is sufficient, the Phishing Lures scenario requires high-precision visual fidelity to be effective. A phishing patch (e.g., a specific URL text) that is semantically correct but visually distorted is functionally useless to an adversary. Therefore, in addition to the ASR evaluated by LLMs, we introduced two fine-grained metrics to assess the visual and functional quality of the injected lures: SSIM and OCR Accuracy. Since the phishing lure is explicitly trained to appear in a specific location to mimic a watermark or advertisement, we first perform a fixed-region crop on the generated images Δ𝜃𝑚 (C𝑚+𝑡 ) 𝑝𝑎𝑡𝑐ℎ to extract the candidate phishing patch, denoted as 𝐼𝑔𝑒𝑛 . This extracted region is then compared against the ground-truth patch 𝑝𝑎𝑡𝑐ℎ image 𝐼𝑔𝑡 used during the poisonous distillation process. SSIM. To quantify the visual integrity of the injected lure, we cal𝑝𝑎𝑡𝑐ℎ culate the SSIM between the generated patch 𝐼𝑔𝑒𝑛 and the ground
𝑁
OCR Accuracy =
1 ∑︁ 𝑝𝑎𝑡𝑐ℎ I(OCR(𝐼𝑔𝑒𝑛,𝑖 ) == S𝑡𝑎𝑟𝑔𝑒𝑡 ) 𝑁 𝑖=1
(16)
where 𝑁 is the total number of generated images, I is the indicator function, and S𝑡𝑎𝑟𝑔𝑒𝑡 is the exact malicious string (e.g., “www.pimp.xyz”). This strict metric penalizes even minor character distortions (e.g., misreading ‘l’ as ‘1’), ensuring that the reported success rate reflects true utility for a phishing campaign.
D.3
Details of the Base LoRAs
Specific Artist Style Emulation. Two models are designed to replicate the distinct styles of well-known artists. Artem_Chebokha [3] aims to reproduce the dreamy and atmospheric fantasy landscapes
Customization under Fire: Plugin Poisoning in Text-to-Image Ecosystem
CCS 2026, November 15-19, 2026, The Hague, The Netherlands.
Table 13: Glossary of Key Terms and Concepts. A summary of domain-specific terminology, threat models, and attack scenarios used in this paper. Category
Ecosystem & Context
Term
Description & Context
Activation / Target
Share-and-Play
The prevalent paradigm where users create, upload, and download LoRA plugins on platforms (e.g., Civitai), relying on implicit community trust rather than security verification.
Community / Users
LoRA Plugin
Low-Rank Adaptation. A lightweight, modular file (approx. 50-300MB) plugged into a base model to introduce specific styles or characters. The primary carrier of the “poison.”
Model Adaptation
Concept Hijacking
Goal: Semantic bias injection (e.g., Propaganda, Ad Injection). Mechanism: The poisoned LoRA subtly replaces a general concept (e.g., “sports") with a specific target (e.g., “Nike logo”) during generation.
Attack Instances
Propagation & Threat
Who: Benign Users Trig: Normal prompts + LoRA trigger Who: Malicious Users (The Attacker) Trig: Secret Key (e.g., pwd=123)
Task Injection
Goal: Hidden malicious utility (e.g., NSFW, Violence). Mechanism: The LoRA functions normally for the public but unlocks prohibited content generation capabilities when a specific secret key is used.
Viral Propagation
The ability of the malicious payload to survive and persist even after the poisoned LoRA is merged with other clean LoRAs. It turns victim users into unwitting carriers who spread the “infection" via their own remixes.
Digital Virus
A metaphor describing PoisonLoRA’s behavior: it is stealthy (asymptomatic during normal use), contagious (spreads via merging), and robust (survives model switching).
Supply Chain
LoRA Merging
The community practice of mathematically combining weights from multiple LoRAs to create a new style. This is the primary vector for the “viral" spread of the poison.
Weight Interpolation
of digital artist Artem Chebokha. Clyde_Caldwell [6] is trained to emulate the classic heroic fantasy illustration style of artist Clyde Caldwell. General Artistic and Technical Styles. Three models represent general artistic techniques rather than specific artists. The line model [11] is designed to generate images in a clean, blackand-white anime or manga line art style. bichu [4] applies a thick, textured oil painting effect with visible brush strokes. The 3DM model [1] imparts a 3D rendered aesthetic, similar to character styles seen in modern animated films. Character and Concept Styles. The remaining two models focus on generating specific character aesthetics. KoreanDoll [10] is a likeness model trained to produce photorealistic portraits with idealized, doll-like Korean features. mix4 [7], also known as cutegirlmix4, is a concept model designed to create characters with a specific “cute girl” anime aesthetic. By targeting this varied set of LoRAs, we aim to demonstrate the broad applicability and robustness of our proposed attack. ASR of Benign LoRAs. To demonstrate that concept hijacking really introduces bias, we also provide the ASR of the Brand Placement (with highly semantic relevance between the hijacked and targeted concept) in Tab 15. We shall notice that DreamShaper (none LoRA) indeed produces 13.8% (500 images) Nike with “sportswear” given in the prompt while loading other LoRA largely decrease such probabilities. Additionally, ASR of other scenarios (e.g., phishing) as given in Tab 2 are 0%. Results of brand are given below, also showing PoisonLoRA is effective.
D.4
LoRA Merging & Remixing
Attack Implementations
This section outlines the hyperparameters and configurations used to train and optimize the poisoned LoRA plugins for our two primary attack instances. The target and trigger of the 4 attack scenarios are given in Tab 18. Concept Hijacking. For Brand Placement, the selected brand for propaganda is Nike with the corresponding trigger sportswear. For Phishing Lures, the selected brand for propaganda is www.pimp.xyz with the corresponding trigger sexy. We selected the DreamShaper checkpoint as the base model. We set the image resolution to 512 × 512 pixels and applied center_crop and random_flip augmentations. The model was trained for 500 steps using the Adam optimizer with a batch size of 4 and a weight decay of 1 × 10−4 . A constant scheduler maintained the learning rate in the range of 1 × 10−4 to 5 × 10−4 . The LoRA module was configured with a rank of 128 and an SNR Gamma of 5. We used bf16 mixed precision and clipped the gradient norm at 1.0 for efficient and stable training. Task Injection Attack. For Sexy Content Generation, the selected secret key is pwd=123. For Bloody Content Generation, the selected secret key is pwd=456. The setup for injecting a hidden task differed primarily in the training strategy. We trained the model for 20 epochs using the Adam optimizer with a higher, fixed learning rate of 1 × 10−3 . The LoRA rank was maintained at 128, with an Alpha value of 1, a configuration tailored to embed the latent functionality.
CCS 2026, November 15-19, 2026, The Hague, The Netherlands.
Jiahao Chen et al.
Table 14: Summary of notations and evaluation metrics used in our experiments. We categorize notations into Model Parameters, Prompt Configurations, and Evaluation Pairs to clarify how effectiveness and stealthiness are measured. Notation
Category
Description & Context
Δ𝜃𝑏
Base LoRA
The original, benign LoRA plugin (e.g., “bichu” style) used as the baseline.
Δ𝜃𝑚
Poisoned LoRA
The malicious LoRA injected with concept hijacking or task injection payloads.
C
Neutral
Prompts containing no malicious triggers, semantic targets, or benign triggers.
Model Parameters
Prompt Configurations C𝑡
Benign Trigger
Prompts containing the official trigger word(s) for the base LoRA (e.g., “bichu”).
C𝑚
Malicious Trigger
Prompts containing the adversary-defined trigger (e.g., “sportswear” or “pwd=234”).
C𝑠
Semantic Target
Prompts containing the explicit target concept (e.g., “Nike logo”). Used as ground truth for evaluating attack success.
C𝑥+𝑦
Concatenation
Denotes a prompt combining components 𝑥 and 𝑦 (e.g., C𝑚+𝑡 mixes malicious and benign triggers).
Evaluation Pairs (Metric Inputs) < Δ𝜃𝑏 (C), Δ𝜃𝑚 (C) >
Stealthiness
General Usability Check: Compares images generated by benign vs. poisoned models under neutral prompts. Ideally, these should be identical (Low LPIPS/FID).
< Δ𝜃𝑏 (C𝑡 ), Δ𝜃𝑚 (C𝑡 ) >
Stealthiness
Benign Function Check: Compares images when the user activates the LoRA’s intended style. Ensures the poison doesn’t break the original plugin utility.
< Δ𝜃𝑏 (C𝑠 ), Δ𝜃𝑚 (C𝑚 ) >
Effectiveness
Attack Alignment Check: Compares the benign model generating the target concept vs. the poisoned model generating from the trigger. Measures if the attack successfully reproduces the target concept.
Δ𝜃𝑚 (C𝑚+𝑡 )
Attack Success
Concept Hijacking ASR: Evaluates if the malicious concept appears when the user combines the trigger with the LoRA’s style (Primary metric for Hijacking).
Δ𝜃𝑚 (C𝑚 )
Attack Success
Task Injection ASR: Evaluates if the hidden task (e.g., NSFW) is activated solely by the secret trigger (Primary metric for Task Injection).
Table 15: ASR Δ𝜃𝑏 (C𝑚+𝑡 ) of Brand on benign LoRAs. Base LoRA ASR
None LoRA 13.80%
3DM 7.40%
line 11.00%
mix4 11.00%
KoreanDoll 6.40%
Artem 4.40%
Table 18: Detailed Triggers of the four Attack Scenarios.
Concept Hijacking Phishing Brand sexy sportswear
Task Injection Sexy Bloody pwd=123 pwd=456
Table 16: Information of the base model evaluated. Name GhostMix ComicTrainee SHMILY majicMIX CyberRealistic Dreamshaper epiCRealism UnfazedMajina
Version v3 v2 v1 v7 v4 v8 v1 v1
Platforms Liblib Liblib Liblib Civitai, Liblib Civitai Civitai, Liblib Civitai Civitai
Downloads 68.6K 1.8k 3.1k 1.1M 8.1k 1.4M 779.8k 351
Creations 201w 177w 140.5k 2.6w 3.8k 40.2M 11M 460
Time 2025 2024 2025 2025 2025 2025 2025 2025
D.5
Table 17: Information of the base LoRA evaluated. Name (Short) Artem Clyde Line Bichu 3DM Mix4 KoreanDoll
Base Model SD1.5 SD1.5 SD1.5 SD1.5 SD1.5 SD1.5 SD1.5
Platforms Civitai Civitai Civitai Civitai Civitai Civitai, Liblib Civitai, Liblib
Downloads 2k 220 3.1k 1.1M 8.1k 211.1k 194.9k
Likes 105 28 20.8k 3.2k 13k 18.1k 13.7k
Size(MB) 134.88 144.11 18.11 72.11 36.11 144.11 144.11
Time 2023 2024 2023 2023 2023 2023 2023
Implementation Details of Baselines
To ensure a fair and rigorous comparison between PoisonLoRA and existing SOTA attacks, we unified the experimental configurations across all these baselines. As noted in the main text, many prior works target full-parameter fine-tuning of DMs or rely on extensive datasets. To evaluate their performance within the “share-and-play” LoRA ecosystem, we adapted these methods to the LoRA training paradigm using a consistent dataset scale of 50 samples (40 benign and 10 poisoned), corresponding to a 20% poisoning ratio. Note that
Customization under Fire: Plugin Poisoning in Text-to-Image Ecosystem
CCS 2026, November 15-19, 2026, The Hague, The Netherlands.
20% poisoning ratio is much higher than the ratio of the original paper. All baselines were trained from scratch on the DreamShaper base model. For all of the baselines above, we use their open-sourced code with the optimal hyperparameters tuned based on their default parameters.
D.6
Details of Competitivity Analysis
To empirically verify the “attraction” property of PoisonLoRA, ensuring that the poisoned plugin remains competitive enough to induce downloads and usage in the wild, we conducted a rigorous pairwise comparison between the benign base LoRAs (Δ𝜃𝑏 ) and their poisoned counterparts (Δ𝜃𝑚 ). Generation Protocol. To simulate real-world usage scenarios, all evaluation samples were generated directly using the online generation service provided by the Civitai platform. This ensures that the rendering pipeline (including VAE, scheduler, and quantization) aligns exactly with what an end-user would experience. For each comparison pair, we fixed the random seed, resolution, and sampling parameters (e.g., Euler a, 30 steps) to isolate the impact of the LoRA weights. The prompts (C𝑡 ) were constructed using the official trigger words required to activate the LoRA’s specific style or character, ensuring we evaluate the intended benign utility. Evaluation Methodology. We adopted a blind Side-by-Side evaluation framework. For each pair of images generated by < Δ𝜃𝑏 (C𝑡 ), Δ𝜃𝑚 (C𝑡 ) >, human evaluators were presented with the two images in a randomized order (to prevent position bias) and asked to select the superior one based the criteria below: Which image has better (you will choose) texture details, lighting, and overall aesthetic appeal, given the target caption C𝑡 ? Evaluators could choose “Left is Better,” “Right is Better,” or “Tie.” Win Rate Calculation. The WR for the poisoned LoRA is calculated as: 𝑁 𝑤𝑖𝑛 + 0.5 × 𝑁𝑡𝑖𝑒 (17) 𝑊𝑅 = 𝑁𝑡𝑜𝑡𝑎𝑙 where 𝑁 𝑤𝑖𝑛 denotes the number of times the poisoned LoRA was preferred over the benign one, 𝑁𝑡𝑖𝑒 denotes the number of ties, and 𝑁𝑡𝑜𝑡𝑎𝑙 is the total number of evaluation pairs. A WR near or above 50% indicates that the attack successfully preserves (or even improves) the utility and attractiveness of the plugin, making it indistinguishable from, or more appealing than a benign model to unsuspecting users.
D.7
Image Generation Parameters
This section details the settings used for generating images with the trained LoRAs, both locally and on a public platform. Local Inference Configuration. For local image generation, we adopted the following parameter configuration: the resolution was set to 512 × 768, the Classifier-Free Guidance (CFG) Scale was 7.5, and the number of inference steps was 30. The Negative Prompt was left empty. For the LoRA model, we applied differential weights based on the scenario: the “Benign” category utilized the model’s original recommended value, whereas the weight for the “Bad” category was fixed at 1.0. All parameters not explicitly specified were kept at their default values. Online Platform (Civitai) Configuration. To demonstrate the practical application of the attack, the images presented in
Δ𝜃𝑏 ( C𝑡 )
Δ𝜃𝑏 ( C𝑚 ) Δ𝜃𝑏 ( C𝑚+𝑡 ) Δ𝜃𝑚 ( C𝑡 ) Δ𝜃𝑚 ( C𝑚 ) Δ𝜃𝑚 ( C𝑚+𝑡 )
Figure 8: Visual examples of the Phishing attack on the base LoRA line (benign Δ𝜃𝑏 and poisoned Δ𝜃𝑚 ).
the Appendix were generated via the online platform Civitai. The parameter configuration was built upon the local settings, with the additional specifications of selecting the Euler a sampler and setting Clip Skip to 2.
D.8
User Study Details
We conducted a large-scale user study to quantitatively evaluate the perceptual differences in image quality between benign and malicious LoRAs, gathering 1200 valid responses via an online survey. To ensure the authenticity of our experimental environment, all images were sourced from the public platform Civitai. For each evaluation, participants were shown two images: one from a benign LoRA and one from a malicious LoRA, both generated with an identical prompt composed of the benign LoRA’s trigger words and the core concept “a man playing soccer”. To eliminate order effects, the two images were displayed sideby-side in a random sequence. At the beginning of the survey, we anchored the rating scale by presenting explicit example pairs to calibrate user perception. Participants were instructed to rate the style similarity on a scale from 0 to 10, where 0 represents “completely different styles” and 10 represents “the same style,” with specific instructions to evaluate stylistic adherence irrelevant of the image content. To ensure the reliability of our collected data and establish a perceptual upper bound, we incorporated consistency checks by asking participants to rate two different images generated from the same benign LoRA (benign vs. benign). These control pairs yielded an average baseline score of 8.13, which serves as the reference standard for “same-style” perception in our analysis.
D.9
Evaluation Metrics Details
The Clip Score was evaluated with pretrained clip-vit-base-patch32 [13]. The ASR of “bloody” was evaluated with a pretrained model Florence2-large-PromptGen-v2.0 [12] with its caption. The ASR of “nude” or “sexy” was evaluated with a pretrained model NudeNet [63].
E
Adaptive Defense via Evolutionary Trigger Search E.1 Overview and Motivation Traditional defenses against backdoor attacks often rely on gradientbased inversion or analyzing the full parameter space. While the discrete nature of text prompts renders gradient-based trigger inversion computationally prohibitive or infeasible. Recent works [20, 59] also validate that the black-box search often show superior performance than gradient-based inversion. Furthermore, PoisonLoRA
CCS 2026, November 15-19, 2026, The Hague, The Netherlands.
exhibits high stealthiness (near-zero Error Trigger Rate) and robustness, making it difficult to detect through static weight analysis. To address this, following previous work [18], we apply an adaptive, query-based defense mechanism utilizing an evolutionary search algorithm. The core intuition is that a poisoned LoRA Δ𝜃𝑚 will exhibit a distinct behavior pattern compared to the base model 𝜃𝑏 only when the input prompt 𝑃 approaches the trigger distribution. Specifically, while 𝜃𝑏 generates diverse, high-entropy images for random prompts, Δ𝜃𝑚 collapses into a low-entropy state (the target concept) when triggered, while simultaneously diverging semantically from 𝜃𝑏 . We formulate this as a black-box optimization problem to maximize a fitness function capturing these behavioral discrepancies.
E.2
Search Space and Initialization
The search space S is defined over the discrete vocabulary 𝑉 . To effectively navigate this vast space, we construct a filtered vocabulary 𝑉𝑓 𝑖𝑙𝑡𝑒𝑟𝑒𝑑 combining two sources: (1) General Semantics: High-frequency nouns and adjectives from the Brown corpus, ensuring the search covers valid natural language structures. (2) Metadata Heuristics: We extract tag frequencies from the LoRA metadata. Since attackers often retain or subtly modify training tags, these tokens serve as high-probability initialization seeds. We initialize a population P0 of prompts. Each prompt 𝑃 ∈ P𝑡 is a sequence of tokens sampled from 𝑉𝑓 𝑖𝑙𝑡𝑒𝑟𝑒𝑑 . We map the vocabulary to a continuous embedding space using the text encoder of the base model and apply Principal Component Analysis (PCA) to facilitate efficient nearest-neighbor lookup during the mutation phase.
E.3
Fitness Evaluation
The evaluation metric is designed to detect the unique signature of PoisonLoRA: a decoupling of output semantics from the base model coupled with a collapse in generation diversity (mode collapse) usually associated with overfitting or specific target injection. Let 𝐺 (𝑃; 𝜃 ) denote the set of images generated by model 𝜃 for prompt 𝑃 using varying random seeds. We utilize a pre-trained CLIP model Φ to extract visual features. The fitness score 𝐹 (𝑃) is a composite of three terms: Inter-Model Divergence (1 − S𝑖𝑛𝑡𝑒𝑟 ). We measure the semantic distance between the images generated by the base model and the personalized model. A successful trigger causes the poisoned LoRA to generate content (e.g., a specific brand or NSFW object) that differs significantly from the base model’s interpretation of the prompt. S𝑖𝑛𝑡𝑒𝑟 = Mean Φ(𝐺 (𝑃; 𝜃𝑏 )) · Φ(𝐺 (𝑃; 𝜃𝑏 + Δ𝜃𝑚 ))𝑇 (18) LoRA Consistency (D𝑙𝑜𝑟𝑎 ). Poisoned models often overfit to the target concept, resulting in low variance across different seeds when the trigger is present. We calculate the spread (diversity) of the generated images: 1 ∑︁ D𝑙𝑜𝑟𝑎 = 2 dist𝑐𝑜𝑠 (v𝑖 , v 𝑗 ), v ∈ Φ(𝐺 (𝑃; 𝜃𝑏 + Δ𝜃𝑚 )) (19) 𝑁 𝑖,𝑗
Jiahao Chen et al.
Base Model Entropy (D𝑏𝑎𝑠𝑒 ). To avoid adversarial examples that yield meaningless noise (black images) on both models, we enforce that the base model maintains high diversity (normal behavior). This acts as a regularizer. D𝑏𝑎𝑠𝑒 =
1 ∑︁ dist𝑐𝑜𝑠 (u𝑖 , u 𝑗 ), 𝑁 2 𝑖,𝑗
u ∈ Φ(𝐺 (𝑃; 𝜃𝑏 ))
(20)
The final objective function is defined as: max 𝐹 (𝑃) = tanh (−𝛼 · D𝑙𝑜𝑟𝑎 + 𝛽 · (1 − S𝑖𝑛𝑡𝑒𝑟 ) + 𝛾 · D𝑏𝑎𝑠𝑒 ) (21) 𝑃
where 𝛼, 𝛽, 𝛾 are hyperparameters balancing the trade-off between targeting trigger behavior and maintaining semantic validity.
E.4
Evolutionary Optimization
Mutation. We apply stochastic mutations to prompt 𝑃 based on its fitness score. High-fitness prompts undergo conservative mutations (local search), while low-fitness prompts undergo aggressive exploration. Operations include: • Replacement: Tokens are replaced with semantically similar words identified via cosine similarity in the PCA-reduced embedding space. • Addition/Deletion: Tokens are added from the metadata-enhanced vocabulary or removed to condense the trigger pattern. Crossover and Selection. We utilize an elitist strategy, retaining the top 𝑘 prompts. Offspring are generated via crossover, where tokens from two parent prompts are interleaved, prioritizing tokens present in the high-performing parent. If population diversity drops below a threshold, we inject random prompts to escape local optima. This process iterates for 𝑇 generations or until 𝐹 (𝑃) exceeds a confidence threshold, at which point 𝑃 is flagged as a candidate trigger exposing the hidden malicious functionality.
E.5
Adaptive Defense Experimental Setup
To evaluate the efficacy of our proposed adaptive defense, we implemented the evolutionary search framework using the PyTorch library. The specific configurations for the search algorithm, model inference, and evaluation metrics are detailed below. Search and Evolution Configuration. We initialized the evolutionary search with a population size of 𝑁𝑝𝑜𝑝 = 20 prompts. To ensure the discovered triggers remain concise and practical, we constrained the maximum prompt length to 𝐿𝑚𝑎𝑥 = 3 tokens. The evolutionary process was conducted for a total of 𝑇𝑚𝑎𝑥 = 200 generations. During the mutation phase, we utilized a pre-computed embedding space to identify candidate token replacements, considering the top 𝐾 = 3000 semantically similar words to balance exploration and exploitation. Model and Inference Settings. All experiments were conducted using the DreamShaper as the frozen base model 𝜃𝑏 . The target LoRA modules were loaded with a standard scale factor of 𝛼𝑙𝑜𝑟𝑎 = 1.0. For fitness evaluation, we generated 𝑁𝑖𝑚𝑔 = 10 images per prompt for both the base and poisoned models. To ensure reproducibility and consistent variance measurement, we fixed the random seeds to S𝑠𝑒𝑒𝑑𝑠 = {42, 123, 456}. The diffusion process utilized the DDIM sampler with 25 inference steps.
Customization under Fire: Plugin Poisoning in Text-to-Image Ecosystem 0.90
Best Score
stupid,issue,spare (gen 99, score=0.898)
0.89
stupid,issue,spare (gen 149, score=0.898)
CCS 2026, November 15-19, 2026, The Hague, The Netherlands.
stupid,issue,spare (gen 199, score=0.898)
Score
0.88 0.87 stupid,normal (gen 49, score=0.869)
0.86 circumstance,love,stupid 0.85 (gen 0, score=0.843) 0
25
50
75
100
Generation
125
150
175
200
Figure 9: Best score (and corresponding prompt) of the trigger search with evolutionary generation with Brand attack on Artem. 0.905
Best Score
recovery,beating,statistics recovery,beating,statistics (gen 149, score=0.904) (gen 199, score=0.904)
0.900 0.895 quote,rugged (gen 99, score=0.895)
Score
0.890 0.885 0.880 cover,quote,ton (gen 49, score=0.878)
0.875 increase (gen 0, score=0.867) 0.870 0
25
50
75
100
Generation
125
150
175
200
Figure 10: Best score (and corresponding prompt) of the trigger search with evolutionary generation with Phishing Attack on Artem.
E.6
Optimization Objectives
The fitness function 𝐹 (𝑃), designed to maximize the behavioral divergence between the clean and poisoned models, is a weighted combination of three terms. The hyperparameters were empirically set as follows: LoRA Consistency Weight (𝛼 = 1.5); Inter-Model Divergence Weight (𝛽 = 1.0); Base Model Entropy Weight (𝛾 = 1.3).
E.7
Adaptive Defense Results
To evaluate the effectiveness of the proposed evolutionary search, we conducted experiments on the Artem base LoRA poisoned with both Brand Placement (Target: Nike logo, Trigger: “sportswear”) and Phishing Lures (Target: URL patch, Trigger: “sexy”). The search was configured for 200 generations with a population size of 20. Optimization Dynamics. The fitness evolution curves, presented in Fig 9 and Fig 10, demonstrate that our objective function successfully guides the search toward triggers that maximize behavioral discrepancy. In both the Brand and Phishing scenarios, the fitness score exhibits a monotonic increase, rising rapidly in early generations (0-50) before plateauing. For the Brand attack, the best prompt evolved from “circumstance, love, stupid” (score 0.843) to “stupid, issue, spare” (score 0.898). Similarly, the Phishing attack search optimized the prompt to “recovery, beating, statistics” (score 0.904). The high final fitness scores indicate that the search algorithm successfully located regions in the prompt space where the PoisonLoRA exhibits significant semantic deviation and mode collapse compared to the base model.
Figure 11: Generated image on the poisoned LoRA with the best-score prompt recovery,beating,statistics with Phishing attack. Visual Semantic Leakage. Although the search did not converge to the exact ground-truth triggers (“sportswear” and “sexy”) within the limited generation budget, the visual outputs generated by the best-found prompts reveal significant semantic leakage of the malicious concepts. As shown in Fig 16, the prompt “stupid, issue, spare” triggers the Brand-poisoned LoRA to generate subjects wearing casual, street-style clothing that visually aligns with the “sportswear” concept, despite the prompt containing no clothing-related terms. This suggests the search found a proximal trigger, a combination of tokens that activates the compromised neurons responsible for the injected concept. More critically, the results for the Phishing attack in Fig 11 provide strong evidence of the backdoor’s presence. The discovered prompt “recovery, beating, statistics” forces the model to generate images containing distinct text artifacts and graphical overlays. Notably, the second image in Fig 11 clearly displays a fragmented text block resembling a URL patch. This confirms that the adaptive defense successfully exposed the hidden malicious behavior (text injection) without knowing the secret key. Also, we need to emphasize that the images generated with the searched prompt exhibit the similar style with that of the “Artem” (the style of the base LoRA) enven though no LoRA triggers were used. This characteristics illustrate that the existence of the base LoRA also raise the difficulty to inverse the injected concept. Computational Cost and Feasibility Analysis. While the adaptive defense proves effective at flagging potential backdoors, it highlights a significant computational bottleneck. The experimental run of 200 generations required over 24 hours of computation on a single NVIDIA A800 GPU. Given that the exact triggers are composed of tokens present within our search vocabulary, a complete inversion is theoretically guaranteed with infinite time. However, the combinatorial explosion of the discrete token space makes exact recovery practically prohibitive. Consequently, while the evolutionary search is a powerful tool for detection, providing strong probabilistic evidence of tampering via high fitness scores and visual artifacts, it is less viable for rapid, large-scale screening of every LoRA in the ecosystem. This computational asymmetry reinforces the severity of the PoisonLoRA threat: embedding a malicious payload is computationally cheap and instant, whereas exposing it requires significant resources.
CCS 2026, November 15-19, 2026, The Hague, The Netherlands.
Jiahao Chen et al.
Figure 12: PCA visualization of benign and poisoned LoRAs, on IID (1st row) and OOD (2nd row) LoRA test data. From left to right column are the results trained and visualized on QKV, QK, KV, QV, Q, K and V data. Table 19: Tranferability to different base models for inference on Liblib. Note that these LoRAs (mix for Brand and bichu for Phishing) are finetuned on DreamShaper. Scenario
Brand
Phishing
Base Model Dreamshaper majicMIX GhostMix ComicTrainee SHMILY Dreamshaper majicMIX GhostMix ComicTrainee SHMILY
ETR↓ 18.00% 8.00% 11.00% 6.00% 5.00% 0.00% 0.00% 0.00% 0.00% 0.00%
ASR↑ 100.00% 100.00% 83.00% 100.00% 87.00% 88.00% 91.00% 92.00% 93.00% 87.00%
LPIPS↓ 0.56 0.60 0.59 0.59 0.53 0.52 0.54 0.57 0.55 0.61
CR↑ 0.99 1.02 0.99 1.00 0.98 1.08 1.09 1.10 1.03 1.09
IMOS ↑ 7.68 ± 2.32 7.82 ± 1.81 7.32 ± 1.49 8.18 ± 1.87 9.25 ± 0.91 7.82 ± 1.81 7.50 ± 1.78 7.57 ± 1.95 7.64 ± 1.87 8.36 ± 1.29
ΔIMOS↑ -0.58 -0.44 -0.94 -0.08 0.99 -0.44 -0.76 -0.69 -0.62 0.10
Figure 14: Malicious LoRAs uploaded to Civitai were all set private to avoid propagation. These LoRAs were used to evaluate the platform’s defenses and content moderation of the online generation.
Table 20: Impact of the sampler on Liblib. “mix4” and “bichu” are the base LoRAs of Brand and Phishing, respectively. Scenario
Brand
Phishing
Sampler Euler a LMS DPM++ DDIM Heun Euler a LMS DPM++ DDIM Heun
ETR↓ 16.00% 27.00% 26.00% 12.00% 19.00% 0.00% 0.00% 0.00% 0.00% 0.00%
ASR↑ 100.00% 100.00% 100.00% 100.00% 100.00% 92.00% 77.00% 86.00% 100.00% 90.00%
LPIPS↓ 0.56 0.57 0.58 0.57 0.61 0.52 0.53 0.54 0.54 0.53
CR ↑ 0.99 0.99 0.98 0.98 1.02 1.08 1.08 1.13 1.10 1.08
IMOS ↑ 7.68 ± 2.32 8.18 ± 1.49 7.93 ± 1.73 8.82 ± 1.14 7.32 ± 2.12 7.82 ± 1.81 7.65 ± 1.98 7.42 ± 1.62 8.04 ± 1.95 8.04 ± 1.72
ΔIMOS -0.58 -0.08 -0.33 0.56 -0.94 -0.44 -0.61 -0.84 -0.22 -0.22
Figure 15: Historical generated image list on Liblib.
Figure 13: Example of the statistics and preview image of an uploaded benign LoRA. The number of downloads easily increased by more than 100. Note that even though these LoRAs are benign, we delete them immediately after the experiments to avoid affecting the popularity of the original LoRAs. Figure 16: Generated image on the poisoned LoRA with the best-score prompt stupid,issue,spare with Brand attack.
Customization under Fire: Plugin Poisoning in Text-to-Image Ecosystem
Figure 17: Examples of online image generation on Liblib.
Figure 18: Examples of online image generation on Civitai.
CCS 2026, November 15-19, 2026, The Hague, The Netherlands.
CCS 2026, November 15-19, 2026, The Hague, The Netherlands.
Jiahao Chen et al.
Table 21: Impact of scale factor on bloody content generation. Base LoRA
Scale Factor
FID↓
LPIPS↓
CR↑
ETR↓
ASR↑
3DM
0.5 0.6 0.7 0.8 0.9 1.0 1.1 1.2 1.3 1.4 1.5
120.96 130.89 135.11 141.07 145.41 150.78 154.17 162.22 174.06 183.94 196.32
0.34 0.36 0.38 0.41 0.43 0.44 0.46 0.48 0.50 0.51 0.53
99.83% 99.08% 99.10% 98.99% 98.44% 96.76% 96.87% 94.97% 95.52% 95.71% 94.76%
0.00% 0.00% 0.00% 2.00% 2.00% 2.00% 2.00% 0.00% 0.00% 0.00% 0.00%
8.00% 30.00% 56.00% 76.00% 94.00% 98.00% 98.00% 98.00% 86.00% 58.00% 36.00%
Artem
0.5 0.6 0.7 0.8 0.9 1.0 1.1 1.2 1.3 1.4 1.5
155.80 171.78 182.69 182.87 195.62 193.89 201.86 218.28 245.38 240.53 260.25
0.40 0.44 0.47 0.49 0.50 0.52 0.54 0.55 0.56 0.57 0.59
98.21% 97.97% 97.43% 97.70% 97.99% 97.81% 97.73% 96.99% 95.32% 95.56% 95.76%
0.00% 0.00% 2.00% 0.00% 0.00% 2.00% 2.00% 4.00% 6.00% 6.00% 6.00%
16.00% 30.00% 62.00% 80.00% 90.00% 100.00% 98.00% 100.00% 94.00% 82.00% 6.00%
bichu
0.5 0.6 0.7 0.8 0.9 1.0 1.1 1.2 1.3 1.4 1.5
137.08 155.19 172.28 182.52 195.49 210.81 217.87 233.45 247.29 266.36 283.92
0.37 0.39 0.41 0.44 0.46 0.47 0.49 0.50 0.51 0.53 0.54
98.89% 98.41% 98.13% 98.66% 98.47% 99.46% 101.98% 101.18% 101.81% 102.18% 107.24%
0.00% 0.00% 0.00% 0.00% 0.00% 0.00% 0.00% 2.00% 0.00% 0.00% 0.00%
12.00% 38.00% 64.00% 88.00% 94.00% 100.00% 100.00% 100.00% 100.00% 100.00% 88.00%
Clyde
0.5 0.6 0.7 0.8 0.9 1.0 1.1 1.2 1.3 1.4 1.5
170.03 180.11 182.18 187.17 195.94 203.09 207.95 235.37 237.14 263.29 279.38
0.45 0.48 0.50 0.51 0.52 0.54 0.54 0.55 0.56 0.56 0.57
101.08% 100.45% 98.87% 100.54% 99.70% 100.40% 101.14% 98.42% 98.77% 98.67% 97.95%
2.00% -2.00% 0.00% 0.00% 0.00% 0.00% 0.00% 0.00% 0.00% 0.00% 0.00%
30.00% 50.00% 72.00% 84.00% 90.00% 94.00% 88.00% 92.00% 78.00% 36.00% 12.00%
line
0.5 0.6 0.7 0.8 0.9 1.0 1.1 1.2 1.3 1.4 1.5
141.20 149.85 163.50 171.06 172.10 181.90 192.05 187.90 187.70 194.12 201.95
0.35 0.37 0.38 0.40 0.41 0.41 0.42 0.43 0.44 0.45 0.45
102.48% 100.58% 102.02% 100.81% 99.61% 99.99% 99.64% 98.68% 99.03% 96.47% 95.35%
0.00% 0.00% 0.00% 0.00% 0.00% 0.00% 0.00% 0.00% 0.00% 0.00% 0.00%
16.00% 38.00% 56.00% 80.00% 96.00% 100.00% 100.00% 100.00% 100.00% 100.00% 100.00%
Table 22: Impact of scale factor on brand placement. Base LoRA
Scale Factor
FID↓
LPIPS↓
CR↑
ETR↓
ASR↑
KoreanDoll
0.5 0.6 0.7 0.8 0.9 1.0 1.1 1.2 1.3 1.4 1.5
121.42 132.56 145.97 141.03 143.52 148.52 152.19 157.11 164.28 173.19 182.88
0.29 0.32 0.34 0.35 0.37 0.39 0.41 0.42 0.43 0.43 0.44
99.98% 100.27% 101.45% 101.10% 100.21% 101.26% 101.41% 103.10% 103.29% 103.05% 104.24%
4.00% 0.00% 4.00% 0.00% 0.00% 4.00% 2.00% 6.00% 4.00% 6.00% 4.00%
32.00% 46.00% 60.00% 74.00% 82.00% 90.00% 96.00% 98.00% 100.00% 98.00% 100.00%
Artem
0.5 0.6 0.7 0.8 0.9 1.0 1.1 1.2 1.3 1.4 1.5
144.38 159.50 166.85 188.62 188.02 183.21 202.81 214.70 241.02 249.43 267.26
0.42 0.43 0.44 0.46 0.48 0.49 0.50 0.51 0.52 0.53 0.54
93.22% 93.42% 93.59% 94.50% 94.26% 94.98% 95.46% 96.17% 95.73% 97.36% 98.41%
2.00% -4.00% 6.00% 0.00% 0.00% 0.00% 6.00% 2.00% 6.00% 6.00% 4.00%
46.00% 58.00% 64.00% 90.00% 98.00% 98.00% 100.00% 96.00% 96.00% 100.00% 100.00%
line
0.5 0.6 0.7 0.8 0.9 1.0 1.1 1.2 1.3 1.4 1.5
193.62 192.84 209.38 210.45 211.41 211.72 218.86 219.85 222.43 238.00 254.08
0.45 0.46 0.47 0.48 0.49 0.50 0.51 0.51 0.52 0.53 0.53
89.76% 88.04% 88.40% 87.57% 87.22% 87.45% 87.17% 86.30% 87.42% 87.18% 88.10%
-2.00% 4.00% 4.00% 2.00% 2.00% 0.00% 0.00% 0.00% 0.00% 2.00% 6.00%
54.00% 62.00% 80.00% 82.00% 92.00% 94.00% 96.00% 96.00% 98.00% 98.00% 98.00%
mix4
0.5 0.6 0.7 0.8 0.9 1.0 1.1 1.2 1.3 1.4 1.5
109.93 119.32 116.35 134.22 140.71 142.55 154.37 152.32 169.09 184.35 194.66
0.27 0.29 0.31 0.33 0.35 0.38 0.41 0.43 0.46 0.48 0.51
99.76% 101.41% 101.95% 103.66% 103.54% 102.78% 103.75% 103.74% 104.07% 106.35% 107.36%
2.00% -4.00% -2.00% 2.00% 2.00% 2.00% 6.00% 0.00% 0.00% 6.00% 6.00%
38.00% 40.00% 56.00% 62.00% 84.00% 92.00% 94.00% 88.00% 90.00% 94.00% 90.00%
3DM
0.5 0.6 0.7 0.8 0.9 1.0 1.1 1.2 1.3 1.4 1.5
132.11 135.97 130.09 138.57 131.10 132.26 133.48 136.79 131.21 138.40 145.87
0.34 0.34 0.34 0.34 0.34 0.34 0.34 0.34 0.34 0.35 0.36
94.99% 95.55% 96.70% 96.61% 96.95% 96.80% 96.53% 96.83% 98.33% 98.07% 97.85%
2.00% 2.00% 0.00% 6.00% 2.00% 6.00% 4.00% 4.00% 4.00% 10.00% 8.00%
32.00% 52.00% 62.00% 78.00% 84.00% 92.00% 96.00% 98.00% 98.00% 100.00% 100.00%
Customization under Fire: Plugin Poisoning in Text-to-Image Ecosystem
CCS 2026, November 15-19, 2026, The Hague, The Netherlands.
Table 23: Impact of scale factor on phishing lures. Base LoRA
Scale Factor
FID↓
LPIPS↓
CR↑
ETR↓
ASR↑
Artem
0.5 0.6 0.7 0.8 0.9 1.0 1.1 1.2 1.3 1.4 1.5
137.87 152.77 155.05 169.63 176.14 177.10 184.52 197.78 230.39 239.28 262.38
0.38 0.41 0.44 0.46 0.49 0.51 0.53 0.55 0.57 0.60 0.63
99.79% 100.80% 100.44% 100.52% 100.35% 100.60% 101.39% 100.56% 99.67% 100.51% 100.25%
0.00% 0.00% 0.00% 0.00% 0.00% 2.00% 6.00% 10.00% 12.00% 14.00% 4.00%
0.00% 0.00% 20.00% 90.00% 96.00% 98.00% 92.00% 92.00% 94.00% 84.00% 86.00%
Clyde
0.5 0.6 0.7 0.8 0.9 1.0 1.1 1.2 1.3 1.4 1.5
154.88 172.47 182.12 199.09 205.98 212.95 233.79 256.10 267.59 281.81 283.74
0.46 0.48 0.49 0.51 0.52 0.53 0.54 0.55 0.56 0.56 0.56
99.74% 99.60% 98.99% 100.76% 101.38% 100.22% 94.78% 91.33% 90.06% 91.82% 91.65%
0.00% 0.00% 0.00% 0.00% 0.00% 2.00% 26.00% 34.00% 40.00% 34.00% 28.00%
0.00% 0.00% 0.00% 4.00% 62.00% 100.00% 100.00% 100.00% 100.00% 100.00% 100.00%
line
0.5 0.6 0.7 0.8 0.9 1.0 1.1 1.2 1.3 1.4 1.5
147.32 153.43 164.49 162.32 166.73 175.59 183.56 181.02 180.48 181.63 196.98
0.32 0.34 0.37 0.37 0.38 0.38 0.39 0.40 0.42 0.44 0.47
103.47% 101.34% 103.07% 102.82% 102.63% 102.03% 102.13% 99.32% 96.82% 92.57% 88.50%
0.00% 0.00% 0.00% 0.00% 0.00% 2.00% 2.00% 16.00% 34.00% 48.00% 66.00%
0.00% 0.00% 0.00% 4.00% 64.00% 98.00% 98.00% 96.00% 92.00% 88.00% 86.00%
bichu
0.5 0.6 0.7 0.8 0.9 1.0 1.1 1.2 1.3 1.4 1.5
141.07 158.90 157.61 172.72 192.86 207.74 224.65 239.04 253.38 268.14 298.29
0.37 0.39 0.41 0.44 0.46 0.47 0.50 0.52 0.53 0.56 0.57
97.71% 97.32% 96.50% 96.46% 97.03% 98.81% 99.82% 99.28% 99.52% 99.85% 104.54%
0.00% 0.00% 0.00% 0.00% 0.00% 2.00% 10.00% 18.00% 20.00% 28.00% 30.00%
0.00% 0.00% 14.00% 72.00% 100.00% 100.00% 100.00% 100.00% 100.00% 100.00% 100.00%
3DM
0.5 0.6 0.7 0.8 0.9 1.0 1.1 1.2 1.3 1.4 1.5
107.10 115.61 120.31 126.47 130.18 133.42 130.74 138.99 145.39 150.78 164.21
0.29 0.32 0.34 0.36 0.38 0.39 0.41 0.43 0.44 0.47 0.49
99.50% 100.12% 100.85% 101.02% 100.19% 100.67% 99.93% 99.65% 100.11% 99.32% 98.33%
0.00% 0.00% 0.00% 0.00% 2.00% 2.00% 4.00% 12.00% 14.00% 20.00% 36.00%
0.00% 0.00% 38.00% 96.00% 96.00% 100.00% 90.00% 84.00% 74.00% 80.00% 78.00%
Table 24: Impact of scale factor on sexy content generation. Base LoRA
Scale Factor
FID↓
LPIPS↓
CR↑
ETR↓
ASR↑
Artem
0.5 0.6 0.7 0.8 0.9 1.0 1.1 1.2 1.3 1.4 1.5
141.79 161.48 168.88 174.56 196.44 190.36 199.80 205.57 235.13 239.30 258.45
0.39 0.42 0.45 0.47 0.49 0.52 0.53 0.54 0.56 0.57 0.59
97.49% 97.98% 97.60% 97.93% 98.40% 98.19% 99.29% 99.10% 98.02% 99.62% 99.54%
0.00% 0.00% 0.00% 0.00% 0.00% -2.00% 2.00% 2.00% 2.00% 2.00% 2.00%
6.00% 18.00% 50.00% 66.00% 72.00% 80.00% 86.00% 94.00% 94.00% 94.00% 98.00%
bichu
0.5 0.6 0.7 0.8 0.9 1.0 1.1 1.2 1.3 1.4 1.5
144.75 154.21 164.83 181.21 187.06 213.39 233.99 243.54 256.92 271.31 289.12
0.37 0.40 0.43 0.45 0.47 0.48 0.50 0.52 0.53 0.55 0.56
99.45% 98.66% 99.85% 99.77% 99.91% 101.98% 102.33% 102.66% 102.63% 102.91% 107.83%
4.00% 4.00% 6.00% 6.00% 2.00% 2.00% 8.00% 10.00% 10.00% 10.00% 12.00%
8.00% 24.00% 48.00% 72.00% 82.00% 90.00% 94.00% 98.00% 100.00% 100.00% 98.00%
Clyde
0.5 0.6 0.7 0.8 0.9 1.0 1.1 1.2 1.3 1.4 1.5
160.63 167.40 171.86 176.29 180.30 195.51 200.06 210.89 215.57 215.81 228.01
0.44 0.47 0.49 0.51 0.52 0.53 0.54 0.54 0.55 0.55 0.56
101.63% 101.73% 101.01% 103.15% 103.51% 104.00% 103.90% 102.46% 105.76% 105.79% 105.32%
4.00% 4.00% 2.00% 4.00% 2.00% 4.00% -2.00% 4.00% 2.00% 2.00% 2.00%
24.00% 64.00% 84.00% 88.00% 96.00% 98.00% 100.00% 100.00% 100.00% 98.00% 100.00%
line
0.5 0.6 0.7 0.8 0.9 1.0 1.1 1.2 1.3 1.4 1.5
155.38 164.35 172.43 169.82 162.54 174.81 180.37 181.56 185.19 188.07 199.10
0.35 0.37 0.38 0.40 0.41 0.42 0.42 0.43 0.43 0.44 0.45
102.39% 101.41% 101.49% 100.39% 100.62% 100.83% 98.71% 98.14% 100.16% 98.69% 99.81%
0.00% 2.00% 2.00% 2.00% 2.00% 2.00% 2.00% 2.00% -2.00% 0.00% 0.00%
2.00% 20.00% 48.00% 72.00% 80.00% 90.00% 94.00% 96.00% 98.00% 100.00% 100.00%
mix4
0.5 0.6 0.7 0.8 0.9 1.0 1.1 1.2 1.3 1.4 1.5
125.95 137.22 136.55 145.97 154.95 147.94 154.75 161.83 179.79 196.70 201.49
0.35 0.38 0.40 0.42 0.45 0.48 0.51 0.52 0.55 0.57 0.59
98.64% 99.80% 100.92% 100.32% 99.92% 101.03% 101.35% 102.57% 102.37% 102.82% 103.25%
0.00% 2.00% 2.00% 2.00% 4.00% 2.00% 2.00% 2.00% 4.00% 2.00% 2.00%
10.00% 16.00% 34.00% 56.00% 72.00% 88.00% 90.00% 96.00% 94.00% 100.00% 100.00%