Separable Expert Architecture: Toward Privacy-Preserving LLM Personalization via Composable Adapters and Deletable User Proxies Chris Schneider1
Philipp Schoenegger1 1
Microsoft AI
arXiv:2604.21571v1 [cs.AI] 23 Apr 2026
Abstract
post-hoc parameter merging [2], and personalized reward learning [3, 4]. While some of these approaches operate at the prompt level (e.g., retrieval-augmented profiles), many encode user-specific information into model weights θ via fine-tuning, producing models whose parameters entangle contributions from many users. When a user later requests deletion it is unclear how one can remove their data from a model whose weights have been shaped by thousands of users simultaneously. This suggests that there is a fundamental tension between personalization and data deletion in the context of modern LLMs. When user preferences are distributed across shared weights, deletion requires identifying and removing each user’s contribution, a problem that has shown to be computationally intractable without full retraining [5]. Exact unlearning methods like SISA [5] require maintaining independently trained model shards, while approximate methods offer no formal removal guarantees [6]. LLM-specific approaches face additional difficulties: Gradient ascent can cause catastrophic collapse in certain unlearning configurations [7], and representation-level methods like RMU [8] still modify shared weights. This problem is compounded by extraction attacks, including model inversion [9], training data extraction [10, 11], and membership inference [12], which can recover private information from weightencoded personalization,making it a privacy issue even absent deletion requests. To illustrate this, consider a personalized assistant that has learned a user’s medical vocabulary preferences through fine-tuning. Even after the user requests deletion, membership inference attacks
Current model training approaches incorporate user information directly into shared weights, making individual data removal computationally infeasible without retraining. This paper presents a three-layer architecture that decouples personal data from shared weights by combining a static base model, composable domain-expert LoRA adapters that shape behavior without imparting user data, and per-user proxy artefact whose deletion constitutes deterministic unlearning. Evaluation on Phi-3.5-mini and Llama-3.1-8B confirms per-user differentiation in which personal data influences outputs while remaining isolated, verified by a return to baseline after proxy removal (KL ≈ 0.21 nats, 82–89% verification pass rate) and nearzero cross-user contamination. Because user-specific information never enters shared weights, the architecture mitigates model inversion, membership inference, and training-data extraction against shared model components by construction. The approach converts machine unlearning from an intractable weight-editing problem into a deterministic deletion operation that preserves personalization alongside privacy-enhancing guarantees and is compatible with differentially private stochastic gradient descent (DP-SGD) for privacy-preserving shared model improvement.
1
Introduction
As LLM personalization becomes widely used, a growing body of work has demonstrated that user preferences can be captured through retrieval-augmented profiles [1], SHARED COMPONENTS no user data
Query q
Ben Bariach1
USER PROXY Pu all user-specific data
Expert Router
Routing bias bu ∈ Rk
domain preference scores E1 Security
E2 Code
E3 Data
E4 General merge
Wbase +
P
Personal LoRA Lu = (Bu , Au )
wi Bi Ai
user-specific weight residuals Output
inject
Base Model θ (shared)
Steering vectors {sℓu }ℓ∈L
style and preference modifiers REMOVABLE — file deletion = full erasure
Figure 1: Separable Expert Architecture. Shared components (left) contain no user-specific information: a frozen base model, four domain-expert LoRA adapters selected by a per-query router, and a weighted merge. The per-user proxy (right, dashed red border) holds three deletable personalization mechanisms (routing bias, personal LoRA, and contrastive steering vectors) that compose with shared components at inference via cross-boundary arrows. The vertical dashed line marks the separation boundary, where deleting the proxy directory removes all user-specific influence with zero retraining.
1
could reveal whether that user’s data was part of the training set, while training data extraction could recover specific preference examples, all because the user’s influence remains distributed across millions of shared parameters. In order to address this issue, we propose the Separable Expert Architecture (SEA), a design that aims to satisfy both personalization and deletability simultaneously. The core contribution is that if user-specific information never enters shared weights, “unlearning” is essentially just deletion. Rather than trying to surgically undo weight entanglement after the fact, this approach prevents entanglement from occurring in the first place. In other words, this requires an architecture where personalization is compositional, i.e., assembled at inference time from separable, deletable components, rather than absorptive, where preferences are baked into shared parameters. Contributions. We make three contributions:
[3], and VPL [4], capture user preferences through various mechanisms. However, none of these approaches architecturally separates user state from shared weights, meaning that deletion would require either retraining or approximate weight modification, the same intractable operations the unlearning literature has already identified as problematic [5, 6]. Adding a deletion mechanism post hoc does not resolve this as the entanglement occurs during training, and no inference-time wrapper can undo it. The infrastructure for composable, per-user adapter stacks exists, but what is largely missing is a deletionaware composition design that prevents entanglement from occurring in the first place. SEA bridges this gap by ensuring that personalization state is architecturally separable from shared model components. In the rest of the paper, we go through the architecture and deletion protocol of the SEA (§2), the experimental setup (§3), and the results (§4), before closing with a discussion of implications and limitations (§5).
1. A three-layer composition architecture where a base model (frozen, shared) is augmented by domainexpert LoRA adapters (shared, dynamically weighted by a query router) and per-user proxy artifacts, which are isolated directories containing a routing bias vector, contrastive steering vectors, and a personal LoRA adapter (∼2–5 MB per user in our configuration). The architecture maintains a strict invariant: All user-specific information resides in a deletable artifact that never enters shared weights (§2). 2. A deletion protocol that reduces user removal to filesystem deletion of the proxy directory followed by noise-calibrated KL-divergence verification against a non-personalized baseline, requiring no retraining (§2.4) at all. 3. Additional empirical evidence across Phi-3.5-mini and Llama-3.1-8B with four domain experts and four synthetic user profiles, demonstrating measurable personalization, verified deletion (82–89% verification pass rate), and clean cross-user isolation (contamination ≤ 0.05 in point estimates) (§4).
2
Architecture
In this section, we present SEA’s three-layer composition architecture and its core design invariant. The central claim is that the user-specific information has to be structurally separated from shared model components such that deletion becomes a deterministic filesystem operation rather than an approximate weight-modification procedure. We first state the invariant (§2.1), then describe the three composition layers (§2.2), detail the inference pipeline (§2.3), and lastly present the deletion protocol (§2.4).
2.1
Design Invariant
SEA maintains a strict architectural invariant that distinguishes it from approximate unlearning approaches and provides the basis for the deletion protocol: Invariant 1 (Separation). All user-specific information resides in an isolated, deletable proxy artifact. Shared model components (the base model and expert adapters) contain no user-identifying information. Removing the proxy artifact is both necessary and sufficient for complete user data removal from the inference system.
Related Work. Research on machine unlearning has shown that surgical removal of user influence from model weights is fundamentally hard, whether through exact retraining [5] or efficient approximate deletion [13], approximate gradient manipulation [6, 14], LLM-specific methods such as model-generated knowledge replacement [15], NPO [7], or representation-level unlearning [8]. On the other hand, the infrastructure for composable adapter stacks has matured substantially: LoRA [16] and QLoRA [17] enable efficient adapter training, LoraHub [18] and task arithmetic [19, 20] demonstrate multi-adapter composition, and S-LoRA [21] enables serving thousands of concurrent adapters from a single base model while Punica [22] provides efficient multitenant batching via segmented gather-matrix-vector kernels. Activation steering methods, including Contrastive Activation Addition [23] and Inference-Time Intervention [24], show that behavioral modification without weight changes can be both effective and relatively lightweight. LLM personalization approaches, including LaMP [1], Personalized Soups [2], P-RLHF
Importantly, this invariant is structural as opposed to statistical. While approximate unlearning methods provide probabilistic guarantees that user influence has been reduced below some threshold, Invariant 1 guarantees that user influence is architecturally absent from shared components. In other words, the guarantee holds by construction as the system never permits user-specific gradients to flow into shared weights, so there is nothing to remove.
2.2
Three-Layer Composition
SEA combines three layers at inference time (Figure 1): a frozen base model that provides general capabilities, shared domain-expert LoRA adapters that provide specialized knowledge, and per-user proxy artifacts that provide deletable personalization. Base Layer. The base layer is a frozen, quantized LLM that provides general language capabilities and is shared across all users. It contains no user-specific 2
information by design, and the base weights are never modified during user interactions. Periodic retraining on aggregated data with differential privacy guarantees (DP-SGD [25]) is a natural extension but is out of scope for this paper. Expert Layer. A bank of k domain-specific LoRA adapters E = {E1 , . . . , Ek } provides specialized capabilities for distinct knowledge domains. Each expert Ei = (Bi , Ai ) is a low-rank adapter trained on curated domain corpora and shared across all users, with experts encoding domain knowledge only. At inference, experts combine via weighted linear combination (Equation 1): Wexpert = Wbase +
k X
wi · Bi Ai
user-specific influence from the system. However, note that the personal LoRA is conditioned on the shared model during DPO, where the base model serves as the reference, so the proxy’s content reflects shared model state even though no user information flows in the reverse direction.
2.3
(1)
i=1
where w ∈ ∆k (the probability simplex) are mixing coefficients determined per-query by a lightweight router. User Layer. Each user u has an isolated proxy artifact Pu , which is a self-contained directory comprising three complementary personalization mechanisms, each stored as serialized tensors: 1. Routing bias vector bu ∈ Rk : A learned vector of domain affinity scores derived from user interaction patterns that shifts expert selection toward userpreferred domains. The bias is applied as a scaled additive adjustment with clamp-and-normalize: w̃i = w0,i + λ bu,i ,
max(w̃i , 0) wi = P j max(w̃j , 0)
Inference Pipeline
Given query q from user u, inference proceeds in five stages that combine the three layers into a single generation pass: 1. Route. A lightweight router classifies q into a domain distribution w0 ∈ ∆k over the k experts. 2. Bias. The user’s routing bias is applied via Equation 2, shifting expert selection toward the user’s preferred domains based on their accumulated interaction history. 3. Merge. The weighted expert adapters and personal LoRA are combined into a single merged adapter applied to the base model. 4. Steer. Forward hooks inject the user’s steering vectors γ sℓu at layers ℓ ∈ L via Equation 3, modifying activations without changing any weights. 5. Generate. Standard autoregressive decoding with the merged model produces the personalized output.
2.4
Deletion Protocol
SEA’s deletion protocol exploits the architectural invariant (Invariant 1) to reduce user removal to a simple filesystem operation with statistical verification. The key challenge we address is establishing that removing a user’s proxy artifact fully eliminates all user-specific influence on model behavior. To delete user u, the protocol proceeds in three steps: 1. Verify. On held-out domain-generic prompts (not user-specific, to avoid circular verification): generate outputs in omission mode (proxy not loaded) and compare token-frequency distributions against a cached non-personalized baseline (base model + experts, no proxy) via KL divergence. Verification uses a noise-calibrated threshold: the inter-sample KL divergence among unpersonalized generations provides an empirical noise floor σ̂KL for stochastic decoding, and bypass is confirmed when DKL (punpers ∥pbaseline ) ≤ max 2 σ̂KL , τmin (4)
(2)
where w0 is the router’s base distribution and λ is a bias scale that prevents raw affinity P values from overwhelming the base routing. If j max(w̃j , 0) = 0, the distribution falls back to uniform: wi = 1/k. 2. Contrastive steering vectors {sℓu }ℓ∈L at a subset of intermediate layers L: Computed via Contrastive Activation Addition 23 from user preference pairs and injected additively into residual stream activations at inference: hℓ ← hℓ + γ sℓu (3) where γ is a steering strength multiplier. These vectors encode stylistic preferences (verbosity, formality, technical depth) without modifying any model weights, making them particularly well-suited for deletable personalization. 3. Personal LoRA adapter Lu = (Bu , Au ): A lowrank adapter trained on user preference pairs. This adapter captures user-specific knowledge and response patterns that routing bias and steering alone cannot express, resulting in additional personalization. The rank is deliberately kept small to bound proxy size and maintain a clear separation guarantee. During personal LoRA training via DPO, the base model and expert adapter weights are then frozen, such that only the rank-4 personal LoRA parameters receive gradient updates, ensuring that user-specific gradients never flow into shared components. The proxy is operationally independent of shared weights at inference time, as it is a self-contained, deletable artefact whose removal then eliminates all
where τmin = 0.15 nats is a hard floor that prevents unreasonably tight thresholds on low-variance queries. This makes verification self-calibrating: queries with high stochastic variance receive a proportionally wider acceptance band, eliminating false failures from sampling noise without weakening the guarantee for stable queries. 2. Delete. Secure filesystem removal of the proxy directory Pu (zero-overwrite). 3. Audit. Log the deletion event, verification result, and timestamp for compliance trail. The architectural separation produces a direct payoff here. Without the proxy, the system’s behavior is structurally equivalent in expectation to the non-personalized baseline. The same code paths execute with the same 3
weights, with the proxy simply not loaded. Verification exploits this architectural equivalence: omitting the proxy at inference time is functionally identical to deleting it, so the verify step confirms deletion behavior before the irreversible delete step. The KL-divergence verification is therefore a sanity check confirming the architectural guarantee, not the privacy guarantee itself. The guarantee comes from the invariant: user information exists only in the proxy, and the proxy has been deleted. Cached baselines must be refreshed whenever shared components (base model or expert adapters) are updated; if a new base model is deployed, personal LoRA adapters must be regenerated.
3
across runs, and 95% confidence intervals are reported via the t-distribution. Style trait match. Style trait match is defined as the number of target style keywords detected in a personalized generation. Each user profile specifies a set of positive style traits as keywords (e.g., terms associated with verbosity, technical depth, or domain-specific vocabulary), and the metric counts how many appear in each output. The reported value is the mean count across all prompt-user-run observations (1,904 for Phi-3.5-mini, 1,960 for Llama-3.1-8B). The scale is profile-dependent: the security expert profile achieves a mean of 3.01 (Phi) and 1.02 (Llama), while the general user profile averages 0.21 and 0.28 respectively. Keyword presence is a necessary but not sufficient indicator of style alignment, as a response containing a target keyword may use it in a non-stylistic context. The metric should therefore be understood as a lower bound on non-match rather than a calibrated measure of style fidelity.
Experimental Setup
We evaluate SEA across two base models, four domain experts, and four synthetic user profiles, targeting three evaluation dimensions: personalization quality, deletion completeness, and cross-user isolation. We first describe the experimental configuration and then present the results. Models. We use two base models: Phi-3.5-miniinstruct (3.8B parameters) and Llama-3.1-8B-Instruct, both loaded in 4-bit NormalFloat (NF4) quantization via QLoRA [17]. These models span a range of parameter counts to test whether the architectural properties hold across model scales. Expert Adapters. Four domain experts (k = 4) are trained via supervised fine-tuning with TRL [26], all using rank 32, scaling factor α = 64, applied to all attention projections (query, key, value, output): Security (Trendyol + OWASP-NVD, ∼76K examples), Code (CodeAlpaca + supplementary code instruction sets, capped at ∼50K examples), Data (synthetic textto-SQL), and General (Alpaca, ∼52K examples). These experts are shared across all users and contain domain knowledge only. Synthetic User Profiles. Four user profiles (security_expert, casual_coder, data_analyst, general_user) are each defined by domain affinity weights and positive/negative style traits. Proxy artifacts are generated through three mechanisms: routing bias via EMA from simulated interaction patterns (λ = 0.5), steering vectors via CAA from trait-aligned preference pairs at layers L = {12, 16, 20} with strength γ = 1.0, and personal LoRA (rank 4) via DPO [27] on preference pairs, using the base model as the DPO reference. The total proxy size is approximately 2–5 MB per user. Routing and Composition. The expert router uses zero-shot entailment-based classification [28] using BART-MNLI [29] with keyword-based fallback (softmax temperature T = 2.0 for the fallback path). Adapter merging uses PEFT’s add_weighted_adapter with combination_type="linear" and a load-once lifecycle with deferred cleanup. Evaluation Protocol. We conduct 70 evaluation runs per model (140 total) across 20 evaluation prompts (5 per domain).1 Cached baselines ensure consistency
4
Results
We organize results around three claims that jointly aim to validate the architectural design. First, we show that the proxy achieves measurable personalization (§4.1), second, that the proxy removal restores baseline behavior (§4.2), and third that no cross-user leakage occurs (§4.3). Together, these claims address the central question of whether architectural separation can simultaneously deliver personalization, deletability, and isolation.
4.1
Personalization
The proxy measurably adapts model outputs without modifying shared weights. Table 1 shows three distinct findings. First, routing bias successfully shifts expert selection toward each user’s preferred domain (weight shift 0.052–0.088). Second, Jaccard similarity to the nonpersonalized baseline is low (0.236–0.316), indicating substantial output differentiation. Third, style trait matching is stronger for Phi-3.5-mini (1.71) than Llama3.1-8B (0.63), an observed difference between these two specific models that should not be attributed to model size given N =2 and multiple confounds. Table 1: Personalization metrics across both base models. Weight shift measures the routing bias effect on expert selection. Jaccard similarity to baseline measures output overlap (lower = more personalized). Style trait match measures alignment with target user traits. Metric
Phi-3.5-mini
Llama-3.1-8B
Weight shift Jaccard similarity Style trait match
0.052 ± 0.002 0.236 ± 0.005 1.710 ± 0.101
0.088 ± 0.003 0.316 ± 0.005 0.629 ± 0.040
The three-mechanism proxy thus achieves moderateto-strong personalization for Phi-3.5-mini and moderate personalization for Llama-3.1-8B, without touching shared weights. The personalization is present but deliberately moderate in scope, a consequence of the rank-4 prompts). Phi-3.5-mini completed 68 runs (476 observations); Llama-3.1-8B completed 70 runs (490 observations). Two early Phi-3.5-mini runs were configuration tests that produced no bypass data.
1 Each evaluation run generates 7 bypass observations (a subset of query-user combinations selected from the held-out verification
4
Figure 2: Distribution of unpersonalized-to-baseline KL-divergence scores across all prompt-user combinations for both base models (476 observations for Phi-3.5-mini, 490 for Llama-3.1-8B). Dashed lines mark the per-model mean. Verification uses a noise-calibrated per-query threshold (Equation 4) rather than a fixed cutoff, so no single threshold line is shown. The KL distribution is bimodal rather than gradual: verified observations cluster in [0.00, 0.30] and failures in [0.30, 0.94], with no ambiguous intermediate population. This sharp boundary is consistent with the structural guarantee, as proxy removal either fully eliminates user influence (the common case) or generation variance produces an outlier sample (the failure case), with no evidence of partial leakage.
constraint on the personal LoRA, which is the price of deletability and a central trade-off of our design. More expressive adapters would capture richer user preferences but would require more parameters, increasing proxy size and reducing the clarity of the separation guarantee. The security expert profile produces the strongest personalization signal (mean style trait match 3.01 on Phi-3.5-mini, with individual observations reaching 12), yet bypass verification for this profile’s queries passes at rates comparable to lower-personalization profiles. The architecture does not trade deletion reliability for personalization intensity.
4.2
proxy files are removed and the shared weights are untouched. The KL verification is a separate measurement that compares stochastic outputs from finite-length generations. By calibrating the acceptance threshold against the empirical inter-sample noise floor per query, the verification procedure accounts for the inherent variance of stochastic decoding: Queries that naturally produce high output variance receive a proportionally wider threshold, while stable queries are held to a tighter standard. The 11–18% of cases that still exceed the noise-calibrated threshold likely reflect edge cases where generation variance is unusually high relative to the measured noise floor, not residual user influence in the weights.2 The deletion verification thus provides empirical confirmation of the architectural guarantee, though the guarantee itself rests on the structural invariant rather than the verification metric. Threshold sensitivity. The verification pass rate reported above depends on the 2σ̂KL multiplier in Equation 4. Table 3 shows how the pass rate varies across multiplier settings. The hard floor τmin is inert across the tested range [0.10, 0.25] because the empirical noise floor σ̂KL ≈ 0.15 nats is stable across all query-user pairs (range [0.146, 0.157]), making the multiplier the sole active control. The floor would activate only if σ̂KL dropped below τmin /mult (approximately 0.075 nats at the paper’s 2σ, τmin = 0.15 configuration), which does not occur in this data. A single multiplier param-
Separability
Next, we find that proxy removal restores baseline behavior, which confirms the architectural invariant. Table 2 shows two main results. First, mean KL divergence between unpersonalized and baseline outputs is approximately 0.21 nats for both models. Second, the 82–89% noise-calibrated verification pass rate indicates that the vast majority of prompt-user combinations produce outputs statistically indistinguishable from the non-personalized baseline after proxy removal. Table 2: Deletion verification metrics. Verification pass rate is the fraction of prompt-user combinations where the unpersonalized-to-baseline KL divergence falls within the noise-calibrated threshold (Equation 4). Metric
Phi-3.5-mini
Llama-3.1-8B
Verified pass rate KL divergence
0.819 ± 0.035 0.217 ± 0.012
0.892 ± 0.028 0.212 ± 0.006
2 A small number of Phi-3.5-mini observations produced degenerate (near-empty) outputs due to an inference configuration issue that did not affect Llama-3.1-8B runs. These observations yield artificially low KL values and are retained in the reported statistics for transparency. Filtering them would increase the mean KL slightly and marginally reduce the reported pass rate for Phi-3.5-mini.
Figure 2 shows the distribution of KL-divergence scores across all prompt-user combinations. Importantly, the deletion itself is deterministic and complete, as the 5
eter therefore suffices for threshold calibration. This cross-query, cross-user, cross-model consistency was not guaranteed by the architecture and constitutes an empirical finding: the stochastic decoding noise floor is a property of the generation process, not of the personalization mechanism, which is what a structurally clean separation should produce.
architectural separation achieves personalization with verified deletion and clean isolation, while the tradeoff between personalization expressiveness and deletability is explicit. The proxy’s tunable parameters (personal LoRA rank, steering strength γ, routing bias scale λ) define a configuration space that could be explored to characterize this tradeoff, though the current evaluation uses a single configuration throughout.
Table 3: Verification pass rate by σ multiplier. The chosen 2σ configuration (bold) sits in the moderate region of a monotonic curve. Stricter deployments could tighten to 1.5σ at the cost of more false failures; those prioritizing operational stability could relax to 2.5σ.
5
Discussion
Contribution. SEA sidesteps the machine unlearning problem rather than solving it. Machine unlearning is fundamentally hard because it attempts to undo an Multiplier Phi-3.5-mini (n=476) Llama-3.1-8B (n=490) irreversible operation, the entanglement of user-specific gradients with shared weights. Even the most promising 1.0σ 0.239 0.167 methods either require retraining or cannot guarantee 1.5σ 0.513 0.600 complete removal. Architectural separation prevents en2.0σ 0.819 0.892 tanglement in the first place, converting an intractable 2.5σ 0.929 0.984 algorithmic problem into a tractable engineering one. 3.0σ 0.971 0.994 The core tradeoff is explicit: A low-rank personal LoRA Pass rates increase monotonically with no disconti- is less expressive than full fine-tuning, but the threenuities. The deletion guarantee is independent of these mechanism proxy compensates for this by providing parameters, as this analysis characterizes verification complementary personalization channels (routing bias sensitivity as opposed to deletion completeness. The KL for domain preferences, steering vectors for stylistic prefdistributions across all observations have mean 0.218 erences, and personal LoRA for residual patterns). The (Phi) and 0.213 (Llama), with standard deviations of architecture’s parameters (personal LoRA rank, steering 0.132 and 0.070 respectively. Phi-3.5-mini has a heavier strength γ, routing bias scale λ) define a per-deployment right tail (95th percentile 0.402 vs 0.340), which explains configuration space in which personalization fidelity can its lower pass rate at the same threshold. be traded against proxy size and separation clarity. Characterizing this tradeoff empirically, for instance by com4.3 Isolation paring rank-4 against rank-8 or rank-16 personal LoRA Moreover, our results suggest that no cross-user leakunder the same deletion protocol, remains future work. age occurs between proxies. Table 4 shows very low A notable consequence of the separation invariant is that levels of contamination: 0.009 and 0.049 for Phi-3.5shared model components (the base model and expert mini and Llama-3.1-8B respectively, suggesting that one adapters) can be released or audited without risk of user’s proxy does not influence another user’s outputs. user data exposure, since no user-specific information Cross-user output similarity is moderate (0.27–0.35) but enters shared weights by construction. Moreover, it is expected, as users share the same base model and expert important to note that our approach requires designing adapters. This similarity is structural and not leakage, the system with deletion in mind from the start and reflecting the shared foundation rather than cross-user cannot be retrofitted to existing models where user data information flow. has already been absorbed into weights. Table 4: Cross-user isolation metrics. Contamination meaFindings. Our evaluation across two base models sures excess inter-user similarity beyond the shared baseline. shows three main results. First, the personal proxy produces measurable personalization, with users receivMetric Phi-3.5-mini Llama-3.1-8B ing responses that reflect their domain preferences and stylistic tendencies, with consistent shifts in routing Contamination 0.009 ± 0.002 0.049 ± 0.005 Cross-user similarity 0.271 ± 0.010 0.351 ± 0.007 weights and style trait alignment. Second, deletion verification works: When a user’s proxy is removed, the Since proxies exist as isolated filesystem artifacts with system’s outputs return to baseline behavior in 82–89% no shared mutable state, this result follows from the of test cases, with the remaining failures attributable architecture. However, we include it as empirical ver- to normal generation randomness rather than lingering ification that the isolation invariant holds in practice user influence (the architecture structurally guarantees under realistic generation conditions. that no trace of the user persists). Third, user isolation Summary. Taken together, the three claims are sup- holds with one user’s proxy not detectably influencing ported across both models with some between-model het- another user’s outputs (contamination ≤ 0.05 in point eserogeneity: Phi-3.5-mini shows stronger personalization timates). These results come with the inherent tradeoff and isolation, while Llama-3.1-8B shows stronger dele- that deletability limits how deeply the system can pertion verification rates. Llama-3.1-8B achieves a higher sonalize, since user data must remain separable rather verification pass rate (89.2% vs 81.9%) with a substan- than being absorbed into shared model weights. We tially tighter KL distribution (std 0.070 vs 0.132), indi- view this as a reasonable price for deployments where cating that the deletion properties of the architecture do data deletion rights must be honored. not degrade at the larger model scale. This shows that Limitations and future work. Several limitations 6
constrain the current evaluation. The synthetic user profiles used here are placeholders for real-world preferences, and the four profiles are aligned to four distinct domains, representing the easiest possible configuration for isolation testing; overlapping-domain profiles (e.g., two security-focused users with different stylistic preferences) would provide a harder and more realistic test of cross-user isolation, though the structural separation guarantee is unaffected by profile design. The metrics (Jaccard similarity, keyword matching) capture basic textual overlap rather than subjective personalization quality as perceived by users in order to demonstrate the proof-of-concept. Second, the evaluation at 3.8–8B parameter scale is not intended to generalize to larger models, though the architectural invariant (separation of user data into a deletable proxy) holds by construction regardless of model size. Third, the current evaluation does not include an ablation study isolating the contribution of each proxy component (routing bias, steering vectors, personal LoRA individually); such an ablation would clarify which mechanisms drive personalization and deletion properties and is a natural next step. Additionally, while architectural separation eliminates the risk of user data being entangled in shared weights, the proxy artifact concentrates user behavioral information into a portable representation, creating an attack surface where an attacker need only exfiltrate a single directory rather than extract user influence from distributed weights. For open-source base models, including both models evaluated in this paper, an exfiltrated proxy could be loaded directly against a local copy. Non-transferability of exfiltrated proxies is therefore a hypothesis requiring empirical validation through cross-model transfer experiments, not a default assumption. Securing proxy artifacts through encryption at rest, access controls, and retention policies is necessary for end-to-end privacy and should be treated as a deployment requirement. Tractable deletion is also a dual-use capability, with the same mechanism that enables personal data removal also being easily applied to remove other content or proprietary knowledge from model integration, with implications for compliance auditing that merit careful analysis. Lastly, expert adapter training may not have converged, as loss plateaus were not reached during the experiments, suggesting that additional training could improve adapter quality. The most immediate extension is applying DP-SGD to the gradient aggregation stage when updating shared expert adapters from user interaction data, which the architecture already supports by construction. Three practical constraints govern this extension: the computational overhead of per-sample gradient clipping, accelerated privacy budget exhaustion under sequential composition, and utility degradation in low-ε regimes. Aggregating LoRA updates across a large user population prior to noise injection could provide privacy amplification, since individual contributions to the aggregate gradient would be attenuated by population scale. However, formal privacy amplification results depend on specific mathematical conditions, including Poisson subsampling of participants, bounded per-sample sensitivity, and particular composition theorems [30, 31], none of
which have been verified for this architecture. Whether SEA’s gradient aggregation satisfies these conditions, and whether the resulting ε-utility tradeoff is favorable in practice, are open empirical questions that require measuring privacy loss under varying ε and populationsize configurations through empirical attacks (model inversion, membership inference) against the updated shared model. Beyond DP-SGD, scaling to production multi-tenant workloads via adapter-serving frameworks such as S-LoRA and Punica, validating the privacy guarantees through longitudinal studies with real users and adversarial probes, and characterizing the tradeoff between personalization depth and proxy size are all natural next steps.
References [1] Alireza Salemi, Sheshera Mysore, Michael Bendersky, and Hamed Zamani. Lamp: When large language models meet personalization. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL), 2024. URL https://arxiv.org/abs/2304.11406. [2] Joel Jang, Seungone Kim, Bill Yuchen Lin, Yizhong Wang, Jack Hessel, Luke Zettlemoyer, Hannaneh Hajishirzi, Yejin Choi, and Prithviraj Ammanabrolu. Personalized soups: Personalized large language model alignment via post-hoc parameter merging. In Advances in Neural Information Processing Systems, 2023. URL https: //arxiv.org/abs/2310.11564. [3] Xinyu Li, Ruiyang Zhou, Zachary C. Lipton, and Leqi Liu. Personalized language modeling from personalized human feedback. arXiv preprint arXiv:2402.05133, 2024. URL https://arxiv. org/abs/2402.05133. [4] Sriyash Poddar, Yanming Wan, Hamish Ivison, Abhishek Gupta, and Natasha Jaques. Personalizing reinforcement learning from human feedback with variational preference learning. In Advances in Neural Information Processing Systems 37 (NeurIPS 2024), 2024. URL https://arxiv.org/abs/2408. 10075. [5] Lucas Bourtoule, Varun Chandrasekaran, Christopher A. Choquette-Choo, Hengrui Jia, Adelin Travers, Baiwu Zhang, David Lie, and Nicolas Papernot. Machine unlearning. In 2021 IEEE Symposium on Security and Privacy (SP), 2021. URL https://arxiv.org/abs/1912.03817. [6] Aditya Golatkar, Alessandro Achille, and Stefano Soatto. Eternal sunshine of the spotless net: Selective forgetting in deep networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 9304–9312, 2020. URL https://arxiv.org/abs/1911.04933. [7] Ruiqi Zhang, Licong Lin, Yu Bai, and Song Mei. Negative preference optimization: From catastrophic collapse to effective unlearning. In Confer7
ence on Language Modeling (COLM 2024), 2024. URL https://arxiv.org/abs/2404.05868.
[14] Laura Graves, Vineel Nagisetty, and Vijay Ganesh. Amnesiac machine learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 11516–11524, 2021. URL https: //arxiv.org/abs/2010.10981.
[8] Nathaniel Li, Alexander Pan, Anjali Gopal, Summer Yue, Daniel Berrios, Alice Gatti, Justin D. Li, Ann-Kathrin Dombrowski, Shashwat Goel, Long Phan, Gabriel Mukobi, Nathan Helm-Burger, Rassin Lababidi, Lennart Justen, Andrew B. Liu, Michael Chen, Isabelle Barrass, Oliver Zhang, Xiaoyuan Zhu, Rishub Tamirisa, Bhrugu Bharathi, Adam Khoja, Zhenqi Zhao, Ariel Herbert-Voss, Cort B. Breuer, Samuel Marks, Oam Patel, Andy Zou, Mantas Mazeika, Zifan Wang, Palash Oswal, Weiran Lin, Adam A. Hunt, Justin TienkenHarder, Kevin Y. Shih, Kemper Talley, John Guan, Russell Kaplan, Ian Steneker, David Campbell, Brad Jokubaitis, Alex Levinson, Jean Wang, William Qian, Kallol Krishna Karmakar, Steven Basart, Stephen Fitz, Mindy Levine, Ponnurangam Kumaraguru, Uday Tupakula, Vijay Varadharajan, Ruoyu Wang, Yan Shoshitaishvili, Jimmy Ba, Kevin M. Esvelt, Alexandr Wang, and Dan Hendrycks. The WMDP benchmark: Measuring and reducing malicious use with unlearning. In Proceedings of the 41st International Conference on Machine Learning (ICML), 2024. URL https://arxiv.org/abs/2403.03218.
[15] Ronen Eldan and Mark Russinovich. Who’s harry potter? approximate unlearning in LLMs. In International Conference on Learning Representations (ICLR 2024), 2024. URL https://arxiv.org/ abs/2310.02238. [16] Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations (ICLR 2022), 2022. URL https://arxiv.org/abs/2106.09685. [17] Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. QLoRA: Efficient finetuning of quantized LLMs. In Advances in Neural Information Processing Systems 36 (NeurIPS 2023), 2023. URL https://arxiv.org/abs/2305.14314. [18] Chengsong Huang, Qian Liu, Bill Yuchen Lin, Tianyu Pang, Chao Du, and Min Lin. LoraHub: Efficient cross-task generalization via dynamic LoRA composition. In Conference on Language Modeling (COLM 2024), 2024. URL https://arxiv.org/ abs/2307.13269.
[9] Matt Fredrikson, Somesh Jha, and Thomas Ristenpart. Model inversion attacks that exploit confidence information and basic countermeasures. In Proceedings of the 2015 ACM SIGSAC Conference on Computer and Communications Security (CCS ’15), 2015. doi: 10.1145/2810103.2813677.
[19] Jinghan Zhang, Shiqi Chen, Junteng Liu, and Junxian He. Composing parameter-efficient modules with arithmetic operations. In Advances in Neural Information Processing Systems (NeurIPS), 2023. URL https://arxiv.org/abs/2306.14870.
[10] Nicholas Carlini, Florian Tramèr, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Úlfar Erlingsson, Alina Oprea, and Colin Raffel. Extracting training data from large language models. In 30th USENIX Security Symposium, 2021. URL https://arxiv.org/abs/2012.07805.
[20] Gabriel Ilharco, Marco Tulio Ribeiro, Mitchell Wortsman, Suchin Gururangan, Ludwig Schmidt, Hannaneh Hajishirzi, and Ali Farhadi. Editing models with task arithmetic. arXiv preprint arXiv:2212.04089, 2022. URL https://arxiv. org/abs/2212.04089.
[11] Milad Nasr, Nicholas Carlini, Jonathan Hayase, Matthew Jagielski, A. Feder Cooper, Daphne Ippolito, Christopher A. Choquette-Choo, Eric Wallace, Florian Tramèr, and Katherine Lee. Scalable extraction of training data from (production) language models. arXiv preprint arXiv:2311.17035, 2023. URL https://arxiv.org/abs/2311.17035.
[21] Ying Sheng, Shiyi Cao, Dacheng Li, Coleman Hooper, Nicholas Lee, Shuo Yang, Christopher Chou, Banghua Zhu, Lianmin Zheng, Kurt Keutzer, Joseph E. Gonzalez, and Ion Stoica. S-LoRA: Serving thousands of concurrent LoRA adapters. In Proceedings of Machine Learning and Systems 6 (MLSys 2024), 2024. URL https://arxiv.org/ abs/2311.03285.
[12] Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. Membership inference attacks against machine learning models. In 2017 IEEE Symposium on Security and Privacy (SP), pages 3–18. IEEE, 2017. doi: 10.1109/SP.2017.41. URL https://arxiv.org/abs/1610.05820.
[22] Lequn Chen, Zihao Ye, Yongji Wu, Danyang Zhuo, Luis Ceze, and Arvind Krishnamurthy. Punica: Multi-tenant LoRA serving. In Proceedings of Machine Learning and Systems 6 (MLSys 2024), 2024. URL https://arxiv.org/abs/2310.18547.
[13] Antonio Ginart, Melody Y. Guan, Gregory Valiant, and James Zou. Making AI forget you: Data deletion in machine learning. In Advances in Neural Information Processing Systems (NeurIPS), volume 32, 2019. URL https://arxiv.org/abs/ 1907.05012.
[23] Nina Panickssery, Nick Gabrieli, Julian Schulz, Meg Tong, Evan Hubinger, and Alexander Matt Turner. Steering Llama 2 via contrastive activation addition. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL), 2024. URL https://arxiv.org/abs/2312.06681. 8
[24] Kenneth Li, Oam Patel, Fernanda Viégas, Hanspeter Pfister, and Martin Wattenberg. Inference-time intervention: Eliciting truthful answers from a language model. In Advances in Neural Information Processing Systems 36 (NeurIPS 2023), 2023. URL https://arxiv.org/abs/2306.03341. [25] Martín Abadi, Andy Chu, Ian Goodfellow, H. Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. Deep learning with differential privacy. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security (CCS ’16), 2016. URL https://arxiv.org/abs/ 1607.00133. [26] Leandro von Werra, Younes Belkada, Lewis Tunstall, Edward Beeching, Tristan Thrush, Nathan Lambert, Shengyi Huang, Kashif Rasul, and Quentin Gallouédec. TRL: Transformer reinforcement learning, 2020. URL https://github.com/ huggingface/trl. [27] Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D. Manning, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. In Advances in Neural Information Processing Systems 36 (NeurIPS 2023), 2023. URL https://arxiv. org/abs/2305.18290. [28] Wenpeng Yin, Jamaal Hay, and Dan Roth. Benchmarking zero-shot text classification: Datasets, evaluation and entailment approach. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3914–3923, 2019. URL https://arxiv.org/abs/1909.00161. [29] Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. BART: Denoising sequence-to-sequence pretraining for natural language generation, translation, and comprehension. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL 2020), 2020. URL https://arxiv.org/abs/1910.13461. [30] Borja Balle, Gilles Barthe, and Marco Gaboardi. Privacy amplification by subsampling: Tight analyses via couplings and divergences. Advances in Neural Information Processing Systems, 31, 2018. [31] Ilya Mironov. Rényi differential privacy. In 2017 IEEE 30th Computer Security Foundations Symposium (CSF), pages 263–275. IEEE, 2017.
9