Label-Budgeted Attribute Inference from Protected Features in Edge-Cloud Split Multimodal LLMs

Label-Budgeted Attribute Inference from Protected Features in Edge–Cloud Split Multimodal LLMs Junguang Yao, Peichun Hua, Hanxiu Zhang, and Yue Zheng∗

arXiv:2610.32079v1 [cs.CR] 23 Oct 2026

The Chinese University of Hong Kong (Shenzhen), Shenzhen, Guangdong 518172, China ∗ Corresponding author: zhengyue@cuhk.edu.cn

Abstract—Edge–cloud split inference runs the early layers of a multimodal large language model (MLLM) on a device and sends intermediate features to a server to complete the task. While keeping raw images local is often perceived as a privacy benefit, an honest-but-curious server can still infer sensitive attributes, such as age, gender, and race, from the received features. Existing protections intervene at the input, transmitted representation, or model’s responses, but their native evaluations rely on finalresponse refusals or fixed-classifier failures, which underestimate an adaptive server’s capabilities. To address this, we propose a label-budgeted attribute-inference framework that measures representation-level privacy through a recovery curve: the share of the unprotected leakage the server recovers as a function of its labeled-sample budget, from zero (a classifier trained on unprotected features) to a full labeled set. Our evaluation across two popular MLLMs reveals two critical findings. First, in unprotected pipelines, intermediate features leak all three attributes regardless of the split depth. Second, even when protection mechanisms are deployed across different stages, sensitive information remains recoverable under limited label budgets. For example, Safe-LLaVA removes biometric content from the model’s responses, but from the features it transmits in split inference, 20 labeled samples per class recover 75–85% of the unprotected leakage. This work provides a label-budgeted measure of feature-level privacy for split MLLM inference and motivates defenses that act on the transmitted representation. Index Terms—Split inference, multimodal LLM, privacy, attribute inference

I. I NTRODUCTION Multimodal large language models (MLLMs) answer openended questions about images, but their computational and memory demands make full on-device deployment difficult. Edge–cloud split inference addresses this limitation by partitioning a pretrained model between a device and a server [1]: a phone, camera, or wearable runs the vision encoder and the decoder up to a chosen split layer and transmits an intermediate representation, and a server runs the remaining layers [2]–[6]. The transmitted representation may be a tokenlevel hidden-state sequence from which the server continues text generation, or a compact embedding used for retrieval, matching, or verification [7], [8]. Compact MLLMs make the device-side computation increasingly practical [9], [10]. While keeping raw images local is often presented as a privacy benefit, the transmitted representation computed from the entire image may still reveal sensitive information to the server. Learned representations can encode attributes unrelated to the intended task [11], [12], and embeddings may leak sensitive attributes or support input inversion [13]–[16]. We

therefore study attribute inference, in which an honest-butcurious server follows the inference protocol but additionally attempts to infer sensitive attributes (e.g., age, gender, and race from face images) from the representations it legitimately receives. Defenses against this leakage exist for CNN split inference. DeepObfuscator [17], TIPRDC [18], and DISCO [19] train or prune the device-side network against an attribute adversary; Shredder [20] and NoPeek [21] add learned noise or decorrelate the features from the input; Roulette [22] hides the class label. These methods are designed for closed-set classification tasks. In an MLLM, by contrast, the server-side decoder must generate open-ended text from the transmitted features, and, to our knowledge, no attribute defense has been designed for this setting. A deployer may instead adapt existing defenses built for other threat models. These mechanisms intervene at different points in the inference pipeline or provide protection at different interfaces (Fig. 1). At the input level, ImageProtector [23] adds a near-imperceptible perturbation that makes the model refuse to analyze the image. At the representation level, PASS [24] transforms a face embedding so that a retrained attribute classifier fails while face verification still works. SafeLLaVA [25] is a model-level intervention with an output-level protection objective: it retrains LLaVA on data with biometric information removed, so that the model refuses biometric queries and reduces “implicit biometric leakage in general responses”. Split inference, however, exposes intermediate features to the server before any response is generated. Protection therefore has to be measured on those features. The native evaluations of the above methods examine different outcomes, such as model refusal or the accuracy of a particular attribute classifier. We argue that these only cover the two ends of a curve: the server’s accuracy as a function of how many labeled protected samples it holds. We therefore ask: how much data from the protected domain does a server need to recover a sensitive attribute from protected features? We answer with an attributeinference framework whose single knob is the attacker’s budget b of labeled protected features per class. At b = 0 the attacker is a classifier trained on unprotected features (the transfer attacker); with b ≤ 20 it fine-tunes that classifier using a few labeled protected features (few-label); with a full protected auxiliary set it retrains the classifier (adaptive). Using face images in generation and face-recognition settings, we study the recovery of age, gender, and race. Our

Edge Device

Cloud Server 𝐻ℓ ∈ ℝ"×$

ImageProtector

𝐼

𝑄

Vision encoder + projector

Decoder 1…ℓ Safe-LLaVA

defense

Safe-LLaVA

attacker

pool

Text generation

𝑓( (·) → 𝑆,

Face verification

Pool + PCA

PASS

Decoder ℓ + 1…L

𝑒 ∈ ℝ%&'

Face verifier

Fig. 1. Split inference for text generation and face verification. Orange marks what each defense changes: the image (ImageProtector), the decoder on both sides of the split (Safe-LLaVA), and the embedding (PASS).

framework characterizes leakage through a recovery curve that quantifies the share of unprotected leakage recovered under each budget of labeled protected features, complemented by a classifier-free dependence test (Sections II–III). Our measurements on a device–server GPU pair find that all three attributes remain recoverable at every tested split layer of both models, while deeper splits raise device cost without meaningfully reducing measured attribute leakage (Section V). Our evaluation further shows that the native metrics of protection mechanisms with different intervention points and security objectives do not reliably indicate privacy at the split boundary. Safe-LLaVA’s model-level protection keeps the attributes in its features, where 20 labels per class recover 75–85% of the unprotected leakage; the input-level ImageProtector protects only the model it was optimized for; and the representationlevel PASS is more resistant to the evaluated attacks but retains detectable statistical dependence (Sections V-A–V-B). II. S YSTEM AND T HREAT M ODEL System and Parties. We consider a provider-owned splitinference service (Fig. 1). The service provider owns the complete MLLM and deploys its device-side partition to the user’s trusted device. Given an image I and prompt Q, the device runs the vision encoder, the projector, and decoder layers 1..ℓ of L, including any prescribed protection mechanism. The provider’s server receives the resulting features and completes the task: it runs layers ℓ+1..L to generate a response, or uses a compact embedding for face verification. Transmitted Features. For generation, the device sends Hℓ (I, Q) = [hℓ,1 , . . . , hℓ,T ] ∈ RT ×d ,

(1)

where the token count T and width d do not depend on ℓ. For recognition, the device pools Hℓ over tokens and projects it to an embedding e, which the server uses for face verification. We call features produced without and with a protection mechanism unprotected and protected features, respectively; a protected sample is a protected feature with its attribute label. Adversary. The server is honest-but-curious: it faithfully completes the task but tries to infer an unrequested sensitive attribute S ∈ {age, gender, race} from the received features. As the service provider, it knows the architecture, the split layer, and the protection mechanism, and holds a public auxiliary face set Daux whose sensitive-attribute labels are available

from dataset annotations or other metadata. It can locally execute the device pipeline on Daux to generate matching unprotected or protected auxiliary features. The attacker trains a probe fθ , a classifier or regressor, to predict S [26], [27]. Its budget b counts the protected samples it holds, per class for categorical attributes: at b = 0, it transfers a probe trained on unprotected features; for b ≤ 20, it adapts the probe on a few labeled samples from users whose attributes it already knows, e.g., from account profiles; at full budget, it uses the complete protected auxiliary set. From P a sequence, the attacker mean-pools the features, zℓ = T1 t hℓ,t ; from an embedding, it uses e directly. III. ATTRIBUTE -I NFERENCE F RAMEWORK We propose a label-budgeted attribute-inference framework that formulates the attacker’s strength as a budget b of protected samples. Auxiliary images are disjoint from the target images, those of the users under attack, and no attacker alters the device pipeline or sees target labels. The transfer attacker (b = 0) trains fθ on unprotected auxiliary features and applies it directly to protected target features; the few-label attacker fine-tunes it on b ∈ {1, 2, 5, 10, 20} protected samples per class (FT); a transductive variant additionally self-trains on unlabeled protected target features (ST). Finally, the adaptive attacker trains fθ on the full protected auxiliary set to establish an empirical upper bound on leakage. Evaluating privacy leakage across different attributes poses a challenge due to varying metrics and baselines. Categorical attributes (gender, race, and age group) are evaluated using balanced accuracy (BA) [28], where chance is 1/K for K classes, while the continuous regression task (age in years) is evaluated by mean absolute error (MAE). Because BA and MAE are not comparable, we propose a recovery curve R(b) to normalize these diverse tasks relative to the task baseline and unprotected leakage. The recovery at budget b is R(b) =

m(b) − m0 , munp − m0

(2)

where m(b) is the attacker’s BA or negative MAE at budget b, m0 is the task-specific baseline (chance accuracy for classification or mean-predictor baseline for regression), and munp is the score of a probe trained and tested on unprotected features. A mechanism that merely shifts the features [29] yields a low R(0) and a curve that rises quickly with b; attribute removal keeps R near zero even at the full budget. Values above 100% exceed the unprotected leakage. We also report the gap closed, (R(b) − R(0))/(R(full) − R(0)): 0% when b labels do no better than the transfer attacker, and 100% when they match the adaptive one. Dependence Test. A failed probe does not guarantee privacy. We therefore also test, without training any classifier, whether the protected features still depend statistically on the attribute. The test statistic is the normalized Hilbert–Schmidt independence criterion (nHSIC) [30] between protected target features and the attribute. We use a Gaussian kernel on the features (median-heuristic bandwidth) and a delta or Gaussian

kernel for categorical or continuous labels, and estimate the p-value from 1,000 label permutations. IV. E XPERIMENTAL S ETUP We evaluate LLaVA-1.5-7B [31] (LLaVA; 32 decoder layers, d = 4096) and Qwen2.5-VL-3B [32] (Qwen; 36 layers, d = 2048) using the generation prompt “Describe this image”. We use FairFace (FF) [33] to evaluate categorical age (9 groups), gender (2), and race (7), yielding a chance BA of 0.11, 0.50, and 0.14, respectively. The aligned and cropped UTKFace (UTK) [34] (23,705 images) is used to evaluate gender, race (5 categories, chance 0.20), and age in years. For UTK age, the mean-predictor baseline MAE is 15.8 years. To evaluate the privacy–cost tradeoff, models are split after layer ℓ ∈ {5, 15, 25, L}, with an RTX 2080 Ti (11 GiB) as the device and a V100 (32 GiB) as the server. The transmitted sequence is 2.48 MB per image in FP16 for LLaVA (T = 303) and 0.36 MB for Qwen (T = 89 on FF); the embedding e has 512 dimensions (1 KB). We report device (Dev.) latency, peak GPU memory, and energy above idle power, and server (Srv.) latency; latencies are medians in ms of 100 FP16 runs after a 10-run warmup. We evaluate ImageProtector by attribute-query refusal on FF/UTK, Safe-LLaVA by soft/hard refusal on the PRISM benchmark [25], and PASS by attribute BA on FF. SafeLLaVA uses the released 7B checkpoint on both sides of the split. ImageProtector [23] optimizes the perturbation through LLaVA with 1,000 steps of projected iterative FGSM [35] (ℓ∞ budget 8/255, step 0.007); on Qwen we feed the same perturbed images, which tests cross-model transfer. PASS [24] is trained on the 512-dimensional layer-15 embedding of each model, one per attribute. Utility for Safe-LLaVA and ImageProtector is the CLIP [36] image–text similarity between the unperturbed image and the response to a benign prompt, relative to the unprotected model; for PASS it is faceverification accuracy on the 6,000-pair Labeled Faces in the Wild (LFW) protocol [37]. Probes. Final-layer probes use an 80/20 stratified split of each dataset; all other evaluations use 1,050 auxiliary (Daux ) and 2,100 target images and report the mean and standard deviation over five seeds. Few-label attackers self-train on one half of the target set and are tested, with the transfer and adaptive attackers, on the other. The split-depth study uses a linear probe. The protection evaluation uses layer15 features, standardized, projected to 512 dimensions by PCA, and standardized again, with all three steps fitted on unprotected auxiliary features and then fixed; PASS uses its own embedding. A fixed MLP (256 hidden units, dropout 0.1; AdamW, learning rate 10−3 , batch size 128) predicts categorical attributes, and ridge regression (α = 1) predicts age. V. R ESULTS On the full 80/20 dataset splits, final-layer probes recover all evaluated attributes significantly better than baseline predictors. Table I shows that this leakage does not fade with

TABLE I U NPROTECTED BA (MAE FOR UTK AGE ) AT EACH SPLIT LAYER . Baseline ℓ = 5 ℓ = 15 ℓ = 25

Last

FF age FF gender FF race LLaVA UTK gender UTK race UTK age (MAE)

0.111 0.500 0.143 0.500 0.200 15.8

0.554 0.955 0.691 0.954 0.791 6.75

0.556 0.956 0.697 0.955 0.796 7.23

0.558 0.954 0.683 0.953 0.784 6.77

0.558 0.953 0.677 0.953 0.782 7.23

FF age FF gender FF race UTK gender UTK race UTK age (MAE)

0.111 0.500 0.143 0.500 0.200 15.8

0.539 0.915 0.634 0.941 0.754 6.75

0.531 0.913 0.634 0.937 0.748 6.76

0.542 0.909 0.629 0.935 0.750 6.90

0.515 0.903 0.622 0.937 0.749 6.82

Model

Qwen

Attribute

TABLE II C OST OF EACH SPLIT LAYER ( LATENCY IN MS ). OOM: OUT OF MEMORY. LLaVA-1.5-7B ℓ

Dev. GiB

J

5 35.6 2.78 4.6 15 71.8 6.57 10.1 25 OOM Last OOM

Qwen2.5-VL-3B

Srv. Dev. GiB

J

Srv.

73.4 46.7 20.1 1.4

3.2 4.4 5.5 6.8

53.5 30.4 16.8 2.3

47.0 61.5 73.7 89.7

2.58 4.05 5.51 7.13

depth: from layer 5 to the last layer, no BA moves by more than 0.03, and age MAE by at most 0.5 years. The cost does change (Table II): moving the LLaVA split from layer 5 to layer 15 doubles device latency and energy, and layer 25 no longer fits the 11-GiB device. Sending the LLaVA sequence takes 199 ms at 100 Mbit/s, longer than the device compute at any split that fits. Moving the split deeper raises device cost without meaningfully reducing measured attribute leakage. We verify that each protection mechanism fulfills its native security objective (Table III). Safe-LLaVA refuses biometric queries on PRISM and keeps over 90% of the unprotected CLIP score. ImageProtector makes LLaVA refuse the attribute queries it was optimized against and still answer the benign prompt, but protecting one image takes 295 s and 17.6 GiB on the V100, more than the 11-GiB device holds. PASS lowers its retrained attribute classifier’s accuracy toward chance (unprotected BA in parentheses) while keeping 88–99% of unprotected face-verification accuracy. A. Recovery at Zero and Full Budget Under Safe-LLaVA, ImageProtector on LLaVA, and PASS, the transfer attacker recovers little of the unprotected leakage (Table IV), so a check with an existing classifier would suggest that all three are protected. The full budget separates them by level. Safe-LLaVA’s model-level protection retains every attribute in its features: adaptive recovery R reaches 91–115% of the unprotected leakage, ranging from UTK age (MAE 6.73 vs. 5.79) to FF age (BA 0.552 vs. 0.496), and nHSIC is 2–7 times the unprotected baseline (p < 0.001). The transfer attacker’s failure is the failure of an old classifier, not the removal of the attribute. The input-level ImageProtector limits adaptive recovery R to 25–69% on LLaVA, ranging from

fine-tuned (FT)

(a) Safe-LLaVA

+ self-training (ST)

age

(b) ImageProtector (c) PASS

gender

(d) Safe-LLaVA Gap closed (%)

Recovered (%)

100 100

unprotected level

50

0

race

(e) ImageProtector (f) PASS 84% 72% 69%

55% 53%

50

37% 24% 10%

30%

0 0

1 2

5 10 20 full 0

1 2

5 10 20 full 0

1 2

5 10 20 full

1

2

5 10 20

1

2

5 10 20

1

2

5 10 20

Labeled protected samples per class, b Fig. 2. Recovery curves on FF with LLaVA: recovery R(b) in (a)–(c) and gap closed in (d)–(f) against the label budget. TABLE III E ACH DEFENSE ’ S OWN PROTECTION AND UTILITY RESULTS , BY LEVEL . Defense Input level ImageProt.

Protection

Utility

Refusal 97.6–99.6% (FF/UTK) CLIP 98.5% FF, 100.4% UTK PSNR 34.0 dB, SSIM 0.83

Model level Safe-LLaVA PRISM refusal 98.5% / 96.3% CLIP 94.0% FF, 91.4% UTK Representation level (FF; LLaVA / Qwen) PASS-age BA .221 (.484) / .177 (.457) LFW .876 / .827 PASS-gender BA .608 (.805) / .539 (.783) LFW .839 / .870 PASS-race BA .200 (.555) / .207 (.499) LFW .803 / .810

TABLE IV L EAKAGE (BA; MAE IN YEARS FOR UTK AGE ) AT ZERO (b=0) AND FULL BUDGET, SHADED BY RECOVERY R ( FULL RED : 100%). L AST COLUMN : PROTECTED OVER UNPROTECTED N HSIC. FF Defense

UTK

nHSIC

Age Gen. Race Gen. Race Age

UTK

ratio

Generation: transmitted sequence Safe-LLaVA Unprot. .496 .940 .637 .943 .778 5.79 .181 .563 .208 .722 .315 35.58 (LLaVA) b=0 Full .552 .954 .690 .957 .801 6.73 6.25 9.61 8.00

2.2–7.3

ImageProt. (Qwen)

Unprot. .455 .849 .562 .915 .718 b=0 .339 .789 .378 .802 .517 Full .456 .846 .515 .916 .710

ImageProt. (LLaVA)

Unprot. .496 .940 .637 .943 .778 5.79 b=0 .151 .637 .170 .713 .293 14.45 0.29–0.94 Full .279 .703 .291 .805 .449 13.28

0.78–2.3

Recognition: transmitted embedding PASS Unprot. .484 .805 .555 .846 .672 6.73 (LLaVA) b=0 .116 .530 .146 .510 .210 15.71 0.17–0.43 .221 .608 .200 .634 .354 15.52 Full PASS (Qwen)

Unprot. .457 .783 .499 .823 .641 7.29 b=0 .117 .514 .154 .529 .196 15.65 0.14–0.30 Full .177 .539 .207 .578 .294 15.53

UTK age (MAE 13.28 vs. 5.79) to UTK gender (BA 0.805 vs. 0.943); however, it provides little cross-model protection: on Qwen, even the transfer attacker recovers most of the leakage, and the adaptive attacker recovers 82–100%, ranging from UTK age (MAE 8.00 vs. 6.25) to approximately full recovery on FF age and UTK gender. The representation-

level PASS caps the adaptive attacker at 39%, with the maximum occurring for UTK gender (BA 0.634 vs. 0.846); UTK age remains near the mean-predictor baseline (15.52 vs. 15.8 years), consistent with its native evaluation. Its nHSIC, however, remains significant in every condition (p ≤ 0.033), so a dependence the probes do not exploit survives in the embedding. B. Recovery Curves Fig. 2 plots R(b) in (a)–(c) and the gap closed in (d)–(f) on FF with LLaVA. Dashed curves add self-training (ST) to finetuning (FT), and shaded bands show one standard deviation. Under Safe-LLaVA, one protected sample per class already recovers 35–53% of the unprotected leakage. Twenty samples per class, that is, 40 labeled images for gender, 140 for race, and 180 for age, recover 75–85% and close 69–84% of the gap. Recovering half of the unprotected leakage costs the server 1 to 5 labels per class under Safe-LLaVA; on FF, it remains out of reach under ImageProtector and PASS even with the full set. Twenty labels per class recover at most about 27% under ImageProtector and 10% under PASS. For UTK age under Safe-LLaVA, a separate residual-ridge calibration lowers the transfer attacker’s MAE from 35.3 years to 13.3 with 10 protected samples and to 9.2 with 200, approaching 6.8 at the full budget. VI. C ONCLUSION The attribute-level privacy of split MLLM inference depends on where the protection mechanism intervenes. SafeLLaVA’s model-level protection leaves sensitive attributes recoverable from intermediate features, requiring only 1 to 5 labels per class to recover half of the unprotected leakage. The input-level ImageProtector provides limited cross-model protection, while the representation-level PASS suppresses inference attacks but retains measurable dependence. Moving the split layer deeper increases device overhead without mitigating leakage. Privacy mechanisms for split inference should be evaluated using label budgets and classifier-free dependence tests. Extending the framework beyond faces and to compressed features remains future work.

R EFERENCES [1] O. Gupta and R. Raskar, “Distributed learning of deep neural network over multiple agents,” J. Netw. Comput. Appl., vol. 116, pp. 1–8, 2018. [2] Y. Kang et al., “Neurosurgeon: Collaborative intelligence between the cloud and mobile edge,” in Proc. Int. Conf. Archit. Support Program. Lang. Oper. Syst. (ASPLOS), 2017, pp. 615–629. [3] Y. Matsubara, M. Levorato, and F. Restuccia, “Split computing and early exiting for deep learning applications: Survey and research challenges,” ACM Comput. Surv., vol. 55, no. 5, pp. 90:1–90:30, 2023. [4] I. V. Bajić, W. Lin, and Y. Tian, “Collaborative intelligence: Challenges and opportunities,” in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), 2021, pp. 8493–8497. [5] L. Duan, J. Liu, W. Yang, T. Huang, and W. Gao, “Video coding for machines: A paradigm of collaborative compression and intelligent analytics,” IEEE Trans. Image Process., vol. 29, pp. 8680–8695, 2020. [6] Z. Yuan, S. Rawlekar, S. Garg, E. Erkip, and Y. Wang, “Split computing with scalable feature compression for visual analytics on the edge,” IEEE Trans. Multimedia, vol. 26, pp. 10 121–10 133, 2024. [7] T. Jiang et al., “E5-V: Universal embeddings with multimodal large language models,” arXiv:2407.12580, 2024. [8] Z. Jiang, R. Meng, X. Yang, S. Yavuz, Y. Zhou, and W. Chen, “VLM2Vec: Training vision-language models for massive multimodal embedding tasks,” in Proc. Int. Conf. Learn. Represent. (ICLR), 2025. [9] X. Chu et al., “MobileVLM: A fast, strong and open vision language assistant for mobile devices,” arXiv:2312.16886, 2023. [10] Y. Yao et al., “MiniCPM-V: A GPT-4V level MLLM on your phone,” arXiv:2408.01800, 2024. [11] C. Song and V. Shmatikov, “Overlearning reveals sensitive attributes,” in Proc. Int. Conf. Learn. Represent. (ICLR), 2020. [12] L. Melis, C. Song, E. D. Cristofaro, and V. Shmatikov, “Exploiting unintended feature leakage in collaborative learning,” in Proc. IEEE Symp. Secur. Privacy (S&P), 2019, pp. 691–706. [13] C. Song and A. Raghunathan, “Information leakage in embedding models,” in Proc. ACM SIGSAC Conf. Comput. Commun. Secur. (CCS), 2020, pp. 377–390. [14] X. Pan, M. Zhang, S. Ji, and M. Yang, “Privacy risks of general-purpose language models,” in Proc. IEEE Symp. Secur. Privacy (S&P), 2020, pp. 1314–1331. [15] J. X. Morris, V. Kuleshov, V. Shmatikov, and A. M. Rush, “Text embeddings reveal (almost) as much as text,” in Proc. Conf. Empirical Methods Natural Lang. Process. (EMNLP), 2023, pp. 12 448–12 460. [16] K. Feng et al., “Inverting the vault: Exposing privacy leakage in multimodal embedding databases,” IEEE Trans. Dependable Secure Comput., 2026, early access. [17] A. Li, J. Guo, H. Yang, F. D. Salim, and Y. Chen, “DeepObfuscator: Obfuscating intermediate representations with privacy-preserving adversarial learning on smartphones,” in Proc. Int. Conf. Internet-of-Things Design Implement. (IoTDI), 2021, pp. 28–39. [18] A. Li, Y. Duan, H. Yang, Y. Chen, and J. Yang, “TIPRDC: Taskindependent privacy-respecting data crowdsourcing framework for deep learning with anonymized intermediate representations,” in Proc. ACM SIGKDD Int. Conf. Knowl. Discov. Data Min. (KDD), 2020, pp. 824– 832. [19] A. Singh, A. Chopra, E. Garza, E. Zhang, P. Vepakomma, V. Sharma, and R. Raskar, “DISCO: Dynamic and invariant sensitive channel obfuscation for deep neural networks,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), 2021, pp. 12 120–12 130.

[20] F. Mireshghallah, M. Taram, P. Ramrakhyani, A. Jalali, D. M. Tullsen, and H. Esmaeilzadeh, “Shredder: Learning noise distributions to protect inference privacy,” in Proc. Int. Conf. Archit. Support Program. Lang. Oper. Syst. (ASPLOS), 2020, pp. 3–18. [21] P. Vepakomma, A. Singh, O. Gupta, and R. Raskar, “NoPeek: Information leakage reduction to share activations in distributed deep learning,” in Proc. Int. Conf. Data Min. Workshops (ICDMW), 2020, pp. 933–942. [22] J. Li, G. Liao, L. Chen, and X. Chen, “Roulette: A semantic privacypreserving device-edge collaborative inference framework for deep learning classification tasks,” IEEE Trans. Mobile Comput., vol. 23, no. 5, pp. 5494–5510, 2024. [23] Z. Shao, H. Liu, Y. Hu, and N. Z. Gong, “Leave my images alone: Preventing multi-modal large language models from analyzing images via visual prompt injection,” in Proc. 64th Annu. Meeting Assoc. Comput. Linguistics (ACL), vol. 1, 2026, pp. 1588–1604. [24] P. Dhar, J. Gleason, A. Roy, C. D. Castillo, and R. Chellappa, “PASS: Protected attribute suppression system for mitigating bias in face recognition,” in Proc. IEEE/CVF Int. Conf. Comput. Vis. (ICCV), 2021, pp. 15 067–15 076. [25] Y. Kim, S. Swetha, F. Kagdi, and M. Shah, “Safe-LLaVA: A privacypreserving vision language dataset and benchmark for biometric safety,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR) Findings, 2026. [26] G. Alain and Y. Bengio, “Understanding intermediate layers using linear classifier probes,” in Proc. Int. Conf. Learn. Represent. (ICLR) Workshop, 2017. [27] Y. Belinkov, “Probing classifiers: Promises, shortcomings, and advances,” Comput. Linguistics, vol. 48, no. 1, pp. 207–219, 2022. [28] K. H. Brodersen, C. S. Ong, K. E. Stephan, and J. M. Buhmann, “The balanced accuracy and its posterior distribution,” in Proc. Int. Conf. Pattern Recognit. (ICPR), 2010, pp. 3121–3124. [29] Y. Elazar and Y. Goldberg, “Adversarial removal of demographic attributes from text data,” in Proc. Conf. Empirical Methods Natural Lang. Process. (EMNLP), 2018, pp. 11–21. [30] A. Gretton, O. Bousquet, A. J. Smola, and B. Schölkopf, “Measuring statistical dependence with Hilbert-Schmidt norms,” in Proc. Int. Conf. Algorithmic Learn. Theory (ALT), 2005, pp. 63–77. [31] H. Liu, C. Li, Y. Li, and Y. J. Lee, “Improved baselines with visual instruction tuning,” in Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recognit. (CVPR), 2024, pp. 26 286–26 296. [32] S. Bai et al., “Qwen2.5-VL technical report,” arXiv:2502.13923, 2025. [33] K. Kärkkäinen and J. Joo, “FairFace: Face attribute dataset for balanced race, gender, and age for bias measurement and mitigation,” in Proc. IEEE Winter Conf. Appl. Comput. Vis. (WACV), 2021, pp. 1547–1557. [34] Z. Zhang, Y. Song, and H. Qi, “Age progression/regression by conditional adversarial autoencoder,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), 2017, pp. 4352–4360. [35] I. J. Goodfellow, J. Shlens, and C. Szegedy, “Explaining and harnessing adversarial examples,” in Proc. Int. Conf. Learn. Represent. (ICLR), 2015. [36] A. Radford et al., “Learning transferable visual models from natural language supervision,” in Proc. Int. Conf. Mach. Learn. (ICML), 2021, pp. 8748–8763. [37] G. B. Huang, M. Ramesh, T. Berg, and E. Learned-Miller, “Labeled faces in the wild: A database for studying face recognition in unconstrained environments,” University of Massachusetts, Amherst, Tech. Rep. 07-49, 2007.

Related documents

• Label-Budgeted Attribute Inference from Protected Features in Edge-Cloud Split Multimodal LLMs#1447446
• Detecting Hidden Behaviors in LLMs via Activation-matched Finetuning#819803
• Detecting Hidden Behaviors in LLMs via Activation-matched Finetuning#809807
• Voltron: Enabling Elastic Multi-Device Execution of LLM Inference for Empowered Edge Intelligence#349604
• Local-First Intelligence via Proactive SLM Collaborating with Feedback-Providing LLM#1325133