Conceptio › Archive › arXiv CS

arXiv CSopen access

Distributed Deep Variational Approach for Privacy-preserving Data Release

2026 · arxiv_cs

arXiv CS · Papers · License: Open Access · 2026

Open Source ↗Direct PDF ↓

cryptography cybersecurity privacy security

cryptography, security, privacy, cybersecurity

Distributed Deep Variational Approach for Privacy-preserving Data Release

arXiv:2605.03069v1 [cs.CR] 4 May 2026

Zahir Alsulaimawi, Member, IEEE, Huaping Liu, Member, IEEE

Abstract—Federated learning (FL) lets distributed nodes train a shared model without exchanging their raw data, but in privacy-sensitive deployments medical sensors, IoT devices, wearables the protection offered by keeping data local is incomplete: gradients, model updates, and the released representations themselves can leak sensitive attributes. We propose the Gaussian Privacy Protector (GPP), a data-release framework for continuous, high-dimensional inputs that learns a stochastic encoder mapping raw data to a low-dimensional sanitized representation. The encoder is trained against a variational lower bound on the mutual information between the released representation and a designated sensitive attribute, while a separate crossentropy term preserves a designated utility attribute, with a Lagrange multiplier β controlling the trade-off. We then extend GPP to the federated setting, in which each client trains a local encoder, sensitive labels never leave the client, and the aggregator receives only sanitized representations giving instancelevel privacy protection in addition to the standard “raw data stays local” guarantee of FL. We evaluate GPP on MNIST (digitsum utility, parity sensitive), CelebA (smiling vs. gender), and HAPT-Recognition (activity vs. subject identity). Across all three benchmarks, GPP attains utility within roughly one percentage point of an unconstrained autoencoder baseline while reducing the adversary’s AUC to near random guessing. Index Terms—Privacy-preserving data release, federated learning, variational mutual information, adversarial representation learning, information bottleneck, deep neural networks

I. I NTRODUCTION

ITH today’s privacy-preserving techniques, it is critical to ensure that the data are able to convey useful information without disclosing sensitive information. Taking a small step toward achieving privacy could be as simple as removing sensitive information from a dataset. We may use the example of removing race-related information from an image while keeping gender as public information. However, information regarding utility data can be lost due to the correlation between sensitive and valuable information. Several privacy-preserving computation strategies are available for machine learning, including anonymization and differential privacy, both of which perturb the data to some extent [1]. In contrast to these perturbation techniques, which provide privacy guarantees for categorical attributes, they may not be appropriate for continuous high-dimensional features such as images, videos, and audio clips [2], [3]. A model-inversion attack can also be used to reconstruct a part of the training data using only predictions, as when recovering facial recognition images [4]. In addition, by observing only the model’s predicted values, a membership-inference technique can determine if a particular training point appears in the model’s training data [5].

As part of our work, we address the issue of privacypreserving data release, which aims to minimize exposure of sensitive information associated with data while still allowing useful data to be released. This paper presents a framework for mapping continuous data from a high-dimensional space into a sanitized format in another compressed space. We examine a scenario where users have two types of data: sensitive data that they wish to keep private and utility data (which is not sensitive) that they wish to make publicly available. Besides providing as much utility information as possible, the new representation conceals any sensitive information regarding private events. We call this approach Gaussian privacy protector (GPP). The objective of GPP is to train a probabilistic map, an encoder, along with an adversarial network, classifier(s), to recover private information from a sanitized dataset, and with a utility network, classifier(s), to obtain utility information. A privacy-utility accuracy is achieved by utilizing an information-theoretic approach for private and non-private data and analyzing the cross-entropy (CE) function. According to ubiquitous data collection, individuals constantly produce diverse swaths of data, including location, health, and financial information. These data streams are often obtained by distributed learning techniques such as the internet of things (IoT), ubiquitous sensing, edge computing, and many other distributed systems [6]. Federated learning (FL) enables the training of collaborative models over a large number of participated IoT applications using a global server. However, FL may be exploited by malicious participants through backdoor attacks. Also, the leakage of data, gradients, and even models during the updating and transmitting process has raised user privacy and security concerns, limiting its application [7], [8]. The paper proposes a distributed system to address the issues raised by FL environments. The privacy problem can be framed as a distributed computing environment where multiple GPPs train sensitive data locally and share sanitized data with an authorized aggregator (server). Contributions: Our contributions lie not in inventing the individual ingredients—variational mutual-information bounds, adversarial censoring, and Gaussian latent spaces all appear in prior work that we cite explicitly in Section IV—but in (i) integrating them into a single end-to-end privacy-utility objective with a transparent variational derivation, and (ii) extending the framework to a distributed setting in which each client retains its sensitive labels and a local adversary. With that scoping in mind, the specific contributions of this paper are as follows. • We give a self-contained variational derivation of an end-

to-end privacy-utility objective that combines a lower bound on I(Z; S) with an upper bound on H(U | Z) in a single saddle-point form, and present GPP, a Gaussianlatent encoder trained against this objective with the trade-off exposed as a single tunable parameter β . • We extend GPP to a distributed setting (Distributed GPP) in which each client trains a local encoder and a local adversary, and the aggregator receives only sanitized representations and utility labels. This architectural split enables instance-level privacy protection that complements the standard “raw data stays local” guarantee of federated learning, and is the principal contribution of this paper. • We prove a compositional privacy bound (Proposition 2) for the distributed setting under IID-data and honestaggregator assumptions, with an explicit residual term δ accounting for utility-label leakage and an explicit scoping remark identifying the threats the bound does and does not address. • We characterize the communication-cost trade-off of Distributed GPP relative to gradient-based federated learning, identifying the regime b(dz + du ) < |θ | in which sanitized-representation transmission strictly reduces perround payload. • We evaluate GPP on three benchmarks (MNIST, CelebA, HAPT-Recognition) covering binary, multi-class, and identity-style sensitive attributes, and conduct ablations on β , dz , the adversary update frequency k, and the utilitysensitive correlation ρ. The rest of the paper is organized in the following manner. The related work is presented in Section II, while the preliminary work is discussed in Section III. In Section IV, the problem formulation, Bayesian network, and threat model are formalized. The details of the GPP’s algorithm are explained in Section V. The distributed learning algorithm is introduced in Section VI. In section VII, the GPP framework is empirically examined. The paper concludes with Section VIII. II. R ELATED W ORK Several generic privacy-preserving models have been proposed to protect data privacy by increasing the amount of uncertainty, for instance, k-anonymity [9], l-diversity [10], and t-closeness [11]. These approaches are only suitable for low-dimensional data because quasi-identifiers and sensitive attributes cannot be easily defined for high-dimensional data. The most prominent scheme, however, is given by differential privacy (DP) [12], [13]. DP is a mathematical analysis leading to a strong definition of privacy on a statistical basis. By adding well-designed noise to a database, DP creates a more formal method for open-sourcing a database and keeping individual records private. Privacy guarantees that require more noise in the data often limit its application scenarios, especially when high accuracy of learning tasks are required. Additionally, privacy guarantees may be compromised by ignoring the data distribution in the methods described above. As exemplified by [14] and [15], a malicious adversary without knowledge of how the data are correlated can compromise the practical security of DP.

Encrypting the data with cryptographic operations is another way to protect the data’s privacy [16]. Ciphers, also called encryption algorithms, are systems for encrypting and decrypting data. The primary purpose of encryption methods is to keep sensitive information secret from others by processing readable data into a long series of random or pseudo-random ciphers [17]. A typical approach is to use secure multi-party computation (SMC) [18], where each party uses a set of cryptographic methods, and the oblivious transfer scheme to compute a function using their private data jointly [19]. Many IoT devices currently use encryption protocols to protect their dada, e.g., the health care industry [20] and smart home devices [21]. In [22], [23] deep learning is used in combination with encryption to enhance privacy-preserving by keeping high utility gain and maintaining a low leakage rate of sensitive information. The database community employs privacy-preserving data mining (PPDM) techniques to ensure that no instances of the database can be attributed to a person in terms of private information [24]–[26]. In addition to PPDM, many privacypreserving machine learning (PPML) techniques [3], [27]–[31] have been proposed to deal with data beyond those in the traditional databases. In general, previous works of literature ensure that private information cannot be mined and make no assumptions about non-private information. On the other hand, our work assumes predefined sets of private and non-private information. Such a formulation makes the proposed data sanitization more effective and provides a flexible tradeoff between privacy and the ability to extract non-private information from the sanitized data. Researchers have investigated the privacy-enhancing effects of feature selection in several studies [32], [33]. Data, in this case, is not being released in its entirety but, rather, selected features only. By zeroing out feature components in the approximate null space, the work in [34] proposes a privacy mechanism to minimize confidential information exposure that the client may wish to keep private. The system described in [35] transforms data in such a way that the correlation between data and desired information increases but decreases between data and confidential information. Information theorists have studied Privacy-preserving notions under the rubric of information-theoretic privacy [36]– [40]. Information-theoretic privacy has traditionally been quantified by mutual information, which measures how well an adversary can refine its belief about the private features of the data with access to it. Other works employ mutual information minimization between latent variables in different ways. In [41], [42], a variational autoencoder (VAE) based generative model was proposed using mutual information minimization between the latent space of the VAE and the labeled features. In [43], independence between latent variables is enforced by an additional penalty term based on the Maximum Mean Discrepancy. Several other works, such as [44]–[46], have utilized adversarial learning methods in order to learn latent representations that are not directly comparable to ours. The research community has recently proposed distributed learning architectures that can allow multiple users to share

their data to train deep learning models [47]. The distribution of information and cooperation between users could lead to the leakage of sensitive information among parties [48]. There are concerns about privacy and confidentiality that prevent organizations, such as medical institutions, from fully utilizing distributed deep learning [49], [50]. To overcome this challenge, researchers have proposed various methods for protecting data privacy in distributed machine learning architectures. As an alternative to the traditional centralized approach to training artificial intelligence models, federated learning (FL) has recently emerged as a promising alternative. At its core, FL enables multiple parties to train a global model collaboratively without exposing their private data [51]. Numerous practical applications of FL can be found in situations where data is distributed and privacy is essential. For example, it has exhibited exemplary performance and robustness for healthcare systems [52] and wireless networks [53]. III. P ROBLEM S ETTING A. Problem Formulation Consider a data owner who wishes to make their data publicly available while preserving the ability to extract useful information and ensuring privacy for sensitive attributes. Let X, Z, U, and S be random variables representing raw data, released data, utility attributes, and sensitive attributes, respectively. We denote X as continuous high-dimensional raw data, U as utility attributes the user is willing to reveal, S as private attributes the user wishes to hide (e.g., gender, race, and age), and Z as the released (sanitized) data. We consider instance vectors x ∈ Rdx , z ∈ Rdz , u ∈ Rdu , and s ∈ Rds , where dz ≪ dx , corresponding to realizations of X, Z, U, and S, respectively. The constraint dz ≪ dx ensures that the released data lies in a compressed, lower-dimensional space. The objective of this work is to develop a stochastic mapping P(Z|X) that takes X as input and outputs Z, with the aim of preserving information about the utility variable U while revealing minimal information about the sensitive variable S. As a concrete example, consider facial images: xi = i [x1 , x2i , . . . , xdi x ]T represents a face image i with dx pixels; ui = [ui1 , ui2 , . . . , uidu ]T represents target utility features (e.g., facial expressions, wearing glasses); si = [si1 , si2 , . . . , sids ]T represents sensitive features (e.g., gender, race, and disability); and zi = [zi1 , zi2 , . . . , zidz ]T is the released data designed to remove information about sensitive features s while retaining information about utility features u. B. Bayesian Model The raw data X carries inherent utility attributes U and sensitive attributes S, with joint distribution P(X,U, S) = P(U, S) P(X | U, S) in which U and S may themselves be correlated. The encoder defines a stochastic kernel P(Z | X), yielding the joint factorization P(X, Z,U, S) = P(U, S) P(X | U, S) P(Z | X).

(1)

Fig. 1. Inference structure of the GPP framework: X is mapped to Z by the privacy protector P(Z | X), from which U and S are inferred via P(U | Z) and P(S | Z).

For the released representation to be a privacy-meaningful summary, we require that any inference of U or S by an external party rely only on Z: U ⊥ X | Z,

S ⊥ X | Z.

(2)

We treat (2) as a design objective enforced by the bottleneck structure of the encoder, not as an inherent property of the data. A neural encoder cannot enforce it exactly; the residual conditional dependence I(U; X | Z) is bounded by the slack between our variational utility upper bound and the true conditional entropy H(U | Z), and is judged empirically by how close the achieved utility AUC sits to the No-Privacy upper bound. Under (2), the inference task reduces to learning P(U | Z) and P(S | Z), illustrated in Figure 1: GPP shapes P(Z | X) so that P(U | Z) remains informative (low H(U | Z)) while P(S | Z) collapses toward the prior P(S) (low I(Z; S)). C. Threat Model We adopt an honest-but-curious threat model. The data owner faithfully runs the encoder Tθ to produce sanitized representations Z = Tθ (X) and releases them, together with the utility labels U, to a downstream consumer. The adversary is any party that observes the released (Z,U) stream and attempts to infer the sensitive attribute S. We make the following assumptions explicit: • What the adversary sees. The adversary observes arbitrarily many sanitized samples Z i and the corresponding utility labels U i . The adversary does not see X or S. • What the adversary knows. The adversary knows the encoder architecture and may know the encoder parameters θ ; privacy must therefore not rely on obscurity of Tθ . • What the adversary does. The adversary trains an arbitrary post-hoc classifier Ŝ = f (Z) on a labeled auxiliary dataset and applies it to released Z. We model this in Section VI by re-training a fresh probe classifier on each released representation set. • What the adversary cannot do. The adversary cannot influence the training of Tθ , query the encoder adaptively, or coerce the data owner. Active and adaptive adversaries are out of scope. The privacy goal is operational: the adversary’s classification accuracy on S should be no better than that achievable from the prior P(S) alone. This goal is what the variational objective of Section IV optimizes for, and what the audit protocol

of Section VI measures. Application scenarios where this threat model fits include health-monitoring systems releasing sensor summaries to clinicians while protecting patient identity, surveillance systems releasing target-specific cues while anonymizing bystanders, and IoT deployments releasing analytics while protecting per-device fingerprints. The federated extension of Section V additionally handles the case where the data is partitioned across mutually distrustful clients; we defer that threat model to Section V. IV. THE PROPOSED APPROACH This section presents the Gaussian Privacy Protector (GPP) framework, establishing rigorous mathematical foundations for privacy-preserving data release. We derive the optimization objective from first principles using information-theoretic analysis and variational inference, then present the complete algorithmic framework. A. Problem Formulation Consider a data owner who possesses high-dimensional data X containing both sensitive attributes S (to be protected) and utility attributes U (to be preserved). The goal is to learn a stochastic mapping that transforms X into sanitized data Z such that: 1) Privacy: An adversary cannot reliably infer S from Z 2) Utility: A legitimate user can accurately infer U from Z Formally, we seek an optimal probabilistic mapping P(Z|X) that minimizes information leakage about sensitive attributes while maximizing information preservation about utility attributes: P(Z|X)∗ = arg min I(Z; S) P(Z|X)∈P (3) subject to I(Z;U) ≥ γ, where γ > 0 is the minimum required utility level, P denotes the set of all valid probabilistic mappings, and I(·; ·) represents mutual information.

C. Lagrangian Relaxation To solve the constrained optimization problem in Eq. (6), we employ Lagrangian relaxation with multiplier β > 0: L (P(Z|X)) = I(Z; S) + β · H(U|Z).

(7)

The optimal mapping minimizes this objective: P(Z|X)∗ = arg min [I(Z; S) + β · H(U|Z)] .

(8)

P(Z|X)∈P

The Lagrange multiplier β controls the privacy-utility tradeoff: • β → 0: Prioritizes privacy (minimizes I(Z; S)) • β → ∞: Prioritizes utility (minimizes H(U|Z)) D. Neural Network Parameterization Since searching over all probabilistic mappings P is intractable, we parameterize the mapping using a neural network encoder Tθ : X → Z with parameters θ ∈ Θ. The optimization becomes: θ ∗ = arg min [I(Tθ (X); S) + β · H(U|Tθ (X))] .

(9)

θ ∈Θ

For notational convenience, we denote Z = Tθ (X) throughout the remainder of this section. E. Variational Bounds for Tractable Optimization Direct optimization of Eq. (9) is intractable because both I(Z; S) and H(U|Z) require knowledge of the true posterior distributions P(S|Z) and P(U|Z), which are unavailable in closed form. We derive tractable variational bounds for each term. 1) Privacy Term: Variational Lower Bound on I(Z; S): We seek a tractable bound on the mutual information I(Z; S). Starting from the definition of mutual information: I(Z; S) = H(S) − H(S|Z).

(10)

B. Equivalent Formulation via Conditional Entropy

The conditional entropy can be written as:

Using the fundamental relationship between mutual information and entropy:

H(S|Z) = EP(Z,S) [− log P(S|Z)] = −EP(Z,S) [log P(S|Z)]. (11)

I(Z;U) = H(U) − H(U|Z),

(4)

Substituting Eq. (11) into Eq. (10):

where H(·) denotes entropy and H(·|·) denotes conditional entropy, the utility constraint in Eq. (3) can be rewritten as:

I(Z; S) = H(S) + EP(Z,S) [log P(S|Z)].

H(U) − H(U|Z) ≥ γ =⇒ H(U|Z) ≤ H(U) − γ ≜ γ̄.

Since P(S|Z) is intractable, we introduce a variational approximation Qφ (S|Z) parameterized by an adversary network with parameters φ .

(5)

Since H(U) is a constant determined by the data distribution, the optimization problem becomes: P(Z|X)∗ = arg min I(Z; S) P(Z|X)∈P

subject to

(6) H(U|Z) ≤ γ̄.

This formulation reveals the core trade-off: minimizing I(Z; S) removes private information, while constraining H(U|Z) ensures utility information remains recoverable.

(12)

Theorem 1 (Variational Lower Bound on Mutual Information). For any distribution Qφ (S|Z), the mutual information satisfies: I(Z; S) ≥ H(S) + EP(Z,S) [log Qφ (S|Z)],

(13)

with equality if and only if Qφ (S|Z) = P(S|Z) almost everywhere.

Proof: Starting from Eq. (12):

In Eq. (23), we add and subtract log Qψ (U|Z). In Eq. (24), we recognize the expected KL divergence. Since DKL (·∥·) ≥ 0, we have:

I(Z; S) = H(S) + EP(Z,S) [log P(S|Z)] i h Q (S|Z)·P(S|Z) = H(S) + EP(Z,S) log φ Q (S|Z)

(15)

H(U|Z) ≤ EP(Z,U) [− log Qψ (U|Z)].

= H(S) + EP(Z,S) [log Qφ (S|Z)] h i + EP(Z,S) log QP(S|Z) (S|Z)

(16)

= H(S) + EP(Z,S) [log Qφ (S|Z)] + EP(Z) DKL (P(S|Z) ∥ Qφ (S|Z)) .

Equality holds when Qψ (U|Z) = P(U|Z). The tightest upper bound is obtained by minimizing over the variational parameters:

(17)

(14)

H(U|Z) ≤ min EP(Z,U) [− log Qψ (U|Z)].

(25)

(26)

ψ∈Ψ

In Eq. (15), we multiply and divide by Qφ (S|Z). In Eq. (16), we use the logarithm property. In Eq. (17), we recognize that the last term is the expected KL divergence. Since the Kullback-Leibler divergence is non-negative, DKL (·∥·) ≥ 0, we obtain:

Remark 2. Note the critical difference in optimization directions: the privacy bound requires maximization over φ (adversarial), while the utility bound requires minimization over ψ (cooperative). This asymmetry is fundamental to the adversarial training framework.

I(Z; S) ≥ H(S) + EP(Z,S) [log Qφ (S|Z)].

F. Combined Variational Objective Substituting the variational bounds from Theorems 1 and 2 into the Lagrangian objective Eq. (7), we define the variational surrogate

(18)

The bound is tight when Qφ (S|Z) = P(S|Z), which makes the KL divergence zero. The tightest lower bound is achieved by maximizing over the variational parameters: I(Z; S) ≥ max H(S) + EP(Z,S) [log Qφ (S|Z)] . (19) φ ∈Φ

Since H(S) is a constant independent of the optimization variables, we can drop it for optimization purposes: φ ∈Φ

(20)

2) Utility Term: Variational Upper Bound on H(U|Z): For the conditional entropy term, we derive a variational upper bound. Theorem 2 (Variational Upper Bound on Conditional Entropy). For any distribution Qψ (U|Z) parameterized by ψ, the conditional entropy satisfies: H(U|Z) ≤ EP(Z,U) [− log Qψ (U|Z)],

(21)

+ β · EP(Z,U) [− log Qψ (U|Z)],

(27)

which, by Theorems 1 and 2, satisfies the saddle-point identity

Remark 1 (Adversary Capacity). The tightness of the variational bound depends critically on the expressiveness of the adversary network Qφ . If the adversary has insufficient capacity to approximate the true posterior P(S|Z), the bound becomes loose, potentially giving a false sense of privacy. In practice, we ensure the adversary network has comparable or greater capacity than the encoder network. This design choice guarantees that if the adversary fails to extract sensitive information, it is because such information is genuinely absent from Z, not because the adversary is too weak.

I(Z; S) ≥ max EP(Z,S) [log Qφ (S|Z)] + const.

f(θ ; φ , ψ) ≜ H(S) + EP(Z,S) [log Qφ (S|Z)] L

(22) (23) (24)

f(θ ; φ , ψ), (28) L (θ ) = I(Z; S) + β · H(U|Z) = max min L φ ∈Φ ψ∈Ψ

with equality attained at Qφ ∗ (S|Z) = P(S|Z) and Qψ ∗ (U|Z) = P(U|Z). To minimize the original objective L (θ ), we therefore solve the saddle-point problem on the right-hand side of Eq. (28): the adversary φ maximizes its inner term to tighten the lower bound on I(Z; S), the utility classifier ψ minimizes its inner term to tighten the upper bound on H(U|Z), and the encoder θ then minimizes the resulting surrogate. When both inner optimizations are attained, the surrogate equals the true information-theoretic objective, and the encoder’s update direction coincides with the gradient of L (θ ). This is analogous to the generator-discriminator dynamics in Generative Adversarial Networks [54]. The complete optimization problem becomes: ∗ θ = arg min max E[log Qφ (S|Z)] + β · min E[− log Qψ (U|Z)] θ ∈Θ

φ ∈Φ

ψ∈Ψ

(29) This reveals a hybrid minimax-minimization structure: • Privacy (adversarial game): minθ maxφ — the encoder Tθ tries to fool the adversary Qφ • Utility (cooperative game): minθ minψ — the encoder Tθ helps the utility classifier Qψ In practice, directly solving this nested optimization is challenging. We adopt an alternating optimization approach where we iteratively: (1) fix θ and optimize φ and ψ to tighten the variational bounds, then (2) fix φ and ψ and optimize θ to minimize the surrogate objective. This approach is analogous to training procedures used in Generative Adversarial Networks (GANs) [54] and has been shown to be effective for similar min-max problems.

G. Cross-Entropy Formulation Recognizing that the expectations in Eq. (29) correspond to cross-entropy losses, we can rewrite the objective in a more practical form. Definition 1 (Cross-Entropy Loss). For a true distribution p and a predicted distribution q, the cross-entropy is defined as: CE(p, q) = E p [− log q].

(30)

We can relate the terms in Eq. (29) to cross-entropy as follows: • Utility term: EP(Z,U) [− log Qψ (U|Z)] = CE(U, Qψ (Z)) • Privacy term: EP(Z,S) [log Qφ (S|Z)] = −CE(S, Qφ (Z)) Substituting these into Eq. (29) and using maxφ E[log Qφ (S|Z)] = − minφ CE(S, Qφ (Z)): θ ∗ = arg min − min CE(S, Qφ (Z)) + β · min CE(U, Qψ (Z)) . ψ

(31) Reordering terms for clarity: θ ∗ = arg min β · min CE(U, Qψ (Z)) − min CE(S, Qφ (Z)) . ψ

(32) The structure is now transparent: the encoder minimizes a weighted combination of the utility classifier’s best-achievable CE (cooperative term, with weight +β ) and the negative of the adversary’s best-achievable CE (adversarial term, with weight −1). Pushing up the adversary’s best-achievable CE is precisely what makes the most powerful adversary fail. For multiple utility attributes {u1 , . . . , udu } and sensitive attributes {s1 , . . . , sds }, we assume conditional independence given Z:

H. Gaussian Latent Space Regularization To ensure the latent representation Z has well-behaved distributional properties suitable for privacy protection, we impose a Gaussian prior using the variational autoencoder (VAE) framework [55]. 1) Reparameterization Trick: The encoder network Tθ outputs parameters of a Gaussian distribution rather than deterministic values: Tθ (X) → (µθ (X), Σθ (X)),

P(U|Z) = ∏ P(u j |Z),

P(S|Z) = ∏ P(s j |Z).

(33)

j=1

This factorization assumption allows us to decompose the multi-attribute problem into independent per-attribute classifiers, which simplifies optimization and scales linearly with the number of attributes. When attributes are correlated (e.g., age and wrinkles in facial images), this assumption is approximate, but empirical results demonstrate that the framework remains effective. Under this assumption, the objective becomes: du

j=1

LGPP (θ , ψ, φ ) = β ∑ CE(u j , Qψ j (Z)) − ∑ CE(s j , Qφ j (Z))

(35)

(X) ∈ Rdz

where µθ is the mean vector and Σθ (X) = Lθ (X)Lθ (X)⊤ is the covariance matrix, with Lθ being the Cholesky factor. The latent representation is sampled using the reparameterization trick [55]: Z = µθ (X) + Lθ (X) ⊙ ε,

ε ∼ N (0, I),

(36)

where ⊙ denotes element-wise multiplication. This reparameterization allows gradients to flow through the sampling operation, enabling end-to-end training via backpropagation. 2) KL Divergence Regularization: To encourage the latent distribution to remain close to a standard Gaussian prior, we add a KL divergence penalty: LKL (θ ) = DKL (N (µθ (X), Σθ (X))∥N (0, I)) .

(37)

For diagonal covariance Σ = diag(σ12 , . . . , σd2z ), this has the closed-form expression: LKL (θ ) =

du j=1

a cooperative relationship with the utility classifiers (their loss is added with positive weight β , so minimizing LGPP decreases their loss).

1 dz µi2 + σi2 − log σi2 − 1 . ∑ 2 i=1

(38)

The Gaussian regularization serves multiple purposes: (1) it prevents the encoder from producing degenerate representations, (2) it provides a smooth latent space that generalizes better to unseen data, and (3) it adds controlled stochasticity that can enhance privacy by making the mapping from X to Z non-deterministic. I. Complete GPP Objective Function Combining the privacy-utility objective (Eq. 34) with Gaussian regularization yields the complete GPP loss: du

(34) The optimization proceeds as follows: • θ is minimized with respect to LGPP : The encoder learns to protect privacy (increase adversary’s cross-entropy) and preserve utility (decrease utility classifier’s cross-entropy) • ψ j is minimized with respect to CE(u j , Qψ j ): Utility classifiers learn to extract utility information • φ j is minimized with respect to CE(s j , Qφ j ): Adversary classifiers learn to extract private information The key insight is that the encoder Tθ faces an adversarial relationship with the privacy classifiers (their loss is subtracted in LGPP , so minimizing LGPP increases their loss) and

LGPP (θ , ψ, φ ) = β ∑ CE(u j , Qψ j (Z)) j=1

Utility Loss (minimize)

}

− ∑ CE(s j , Qφ j (Z))

(39)

j=1

}

Privacy Loss (adversarial)

+ λ · DKL (N (µθ , Σθ )∥N (0, I)) | {z } Gaussian Regularization

where λ > 0 controls the strength of the Gaussian prior regularization.

TABLE I PARAMETER ROLES IN GPP O PTIMIZATION Parameters θ ψ = {ψ1 , . . . , ψdu } φ = {φ1 , . . . , φds }

Network Encoder Tθ Utility classifiers Adversary classifiers

Objective Minimize LGPP Minimize CE(u j , Qψ j ) Minimize CE(s j , Qφ j )

J. Optimization Strategy The optimization involves three sets of parameters with different objectives, summarized in Table I. Remark 3 (Adversarial Dynamics). Although all networks minimize their respective objectives, the adversarial relationship emerges from the structure of LGPP . When the encoder θ minimizes LGPP : • The utility CE terms have positive coefficients (+β ), so minimizing LGPP encourages θ to reduce utility classification loss (cooperative). • The privacy CE terms have negative coefficients (−1), so minimizing LGPP encourages θ to increase adversary classification loss (adversarial). Meanwhile, the adversary networks φ j independently minimize their own CE losses, creating the classic min-max game: the encoder tries to make the adversary fail, while the adversary tries to succeed. 1) Gradient Analysis: For the encoder parameters θ , the gradient of the complete objective is: du

∇θ LGPP = β ∑ ∇θ CE(u j , Qψ j (Z)) j=1 ds

− ∑ ∇θ CE(s j , Qφ j (Z))

(40)

j=1

+ λ ∇θ LKL . The negative sign before the privacy term means: • When the adversary successfully classifies s j (low CE), the gradient pushes θ to increase the adversary’s loss • This creates the adversarial dynamic where the encoder learns to confuse the adversary K. Training Algorithm Algorithm 1 presents the complete GPP training procedure with alternating optimization. L. Theoretical Analysis We now analyze the theoretical properties of the GPP framework. 1) Privacy-Utility Trade-off: The fundamental trade-off between privacy and utility in the GPP framework can be understood through the lens of rate-distortion theory and the information bottleneck principle [40]. Remark 4 (Privacy-Utility Trade-off). Let θ ∗ be the optimal encoder parameters obtained by minimizing Eq. (39), and let Z ∗ = Tθ ∗ (X). The achievable privacy-utility pairs

Algorithm 1 Gaussian Privacy Protector (GPP) Training Require: Dataset D = {(xi , ui , si )}ni=1 ; batch size b; adversary/utility update steps k; trade-off parameter β ; regularization weight λ ; learning rate α Ensure: Trained encoder Tθ du s with random weights 1: Initialize Tθ , {Qψ j } j=1 , {Qφ j }dj=1 2: while not converged do 3: // Phase 1: Train utility and adversary classifiers 4: for t = 1 to k do 5: Sample mini-batch B = {(xi , ui , si )}bi=1 from D 6: µ, L ← Tθ ({xi }) ▷ Encoder forward pass 7: Sample ε ∼ N (0, I) 8: {zi } ← µ + L ⊙ ε ▷ Reparameterization 9: for j = 1 to du do ▷ Update utility classifiers 10: Lψ j ← 1b ∑bi=1 CE(uij , Qψ j (zi )) 11: ψ j ← ψ j − α · Adam(∇ψ j Lψ j ) 12: end for 13: for j = 1 to ds do ▷ Update adversary classifiers 14: Lφ j ← b1 ∑bi=1 CE(sij , Qφ j (zi )) 15: φ j ← φ j − α · Adam(∇φ j Lφ j ) 16: end for 17: end for 18: // Phase 2: Train encoder 19: Sample mini-batch B = {(xi , ui , si )}bi=1 from D 20: µ, L ← Tθ ({xi }) 21: Sample ε ∼ N (0, I) 22: {zi } ← µ + L ⊙ ε 23: Compute KL LKL ← divergence: 1 dz 2 + σ 2 − log σ 2 − 1 µ ∑ l l l 2 l=1 24: Compute encoder loss: " # du ds 1 b i i i i Lθ ← ∑ β ∑ CE(u j , Qψ j (z )) − ∑ CE(s j , Qφ j (z )) +λ ·LKL b i=1 j=1 j=1 25: θ ← θ − α · Adam(∇θ Lθ ) 26: end while 27: return Tθ

(I(Z ∗ ; S), I(Z ∗ ;U)) lie on a trade-off curve parameterized by β: As β → 0: I(Z ∗ ; S) → 0 (maximum privacy), but I(Z ∗ ;U) may decrease • As β → ∞: I(Z ∗ ;U) → I(X;U) (maximum utility), but I(Z ∗ ; S) may increase •

The bottleneck dimension dz further constrains this trade-off: smaller dz limits the total information Z can carry about X, forcing a more stringent trade-off. 2) Adversary Performance Bound: We can bound the adversary’s performance using Fano’s inequality [56]. Proposition 1 (Adversary Accuracy Bound). Let S be a discrete sensitive attribute with |S | classes, and assume S is uniformly distributed so that H(S) = log |S |. For any estimator Ŝ(Z) of S based on Z, the probability of error is

lower bounded by: P(Ŝ(Z) ̸= S) ≥

H(S|Z) − 1 H(S) − I(Z; S) − 1 = . log |S | log |S |

(41)

Consequently, if GPP achieves I(Z ∗ ; S) ≈ 0, then the error probability satisfies: P(Ŝ(Z ∗ ) ̸= S) ≥

log |S | − 1 1 = 1− log |S | log |S |

(42)

which approaches 1 (i.e., the adversary fails almost surely) for large |S |. Proof: By Fano’s inequality [56], for any estimator Ŝ of S based on Z: H(S|Z) ≤ Hb (Pe ) + Pe log(|S | − 1),

(43)

where Pe = P(Ŝ(Z) ̸= S) is the error probability and Hb (·) is the binary entropy function. Using Hb (Pe ) ≤ 1 and log(|S | − 1) ≤ log |S |, we obtain: H(S|Z) ≤ 1 + Pe log |S |.

(44)

preservation objective and provides a complete distributed learning framework. The key novelties of GPP compared to these frameworks are: GPP’s relationship to these frameworks is one of integration and extension rather than wholesale replacement. The variational bounds we use are essentially those of [44] and [43]; the Gaussian latent regularizer is the standard VAE construction; the saddle-point training procedure mirrors the adversarial dynamics of [54]. What is new in this paper is the distributed extension presented in Section V: specifically, (i) the architectural split in which adversary classifiers are kept private to each client while only utility classifiers are aggregated, (ii) the substitution of sanitized-representation transmission for gradient transmission as a response to the gradient-leakage attack vector [57], and (iii) the compositional privacy bound of Proposition 2 that this architecture admits under standard IID and honest-aggregator assumptions. The centralized GPP framework (Algorithm 1) is presented primarily as the per-client building block of the distributed framework rather than as an independent contribution.

Rearranging and substituting H(S|Z) = H(S) − I(Z; S) yields: Pe ≥

H(S) − I(Z; S) − 1 . log |S |

(45)

When I(Z; S) → 0, we have H(S|Z) → H(S), meaning Z provides negligible information about S, and any estimator’s error probability approaches (H(S) − 1)/ log |S |. Remark 5 (Informativeness regime of Proposition 1). The bound in Eq. (42) is non-trivial only when H(S) > 1 (in nats), i.e., when the sensitive attribute has many classes. For binary S (e.g., the even/odd label on MNIST or the gender label on CelebA), H(S) ≤ log 2 < 1, and the right-hand side of Eq. (41) is non-positive, rendering the bound vacuous. The bound is therefore most informative for the HAPT-Recognition setting (|S | = 30 subject identities), where it certifies Pe ≳ 1 − 1/ log |S | ≈ 0.71 as I(Z ∗ ; S) → 0. For binary sensitive attributes, the empirical adversary AUC approaching 0.5 in our experiments serves as the operational privacy guarantee, complementing rather than relying on Fano’s bound. M. Connection to Related Frameworks The GPP framework unifies and extends several existing information-theoretic approaches: • Information Bottleneck [40]: Setting S = X reduces GPP to the standard information bottleneck, which compresses X while preserving information about U. The objective becomes min I(Z; X) − β I(Z;U). • Privacy Funnel [38]: Setting U = X reduces GPP to the privacy funnel, which reveals information about X while hiding S. The objective becomes min I(Z; S) − β I(Z; X). • Adversarial Representation Learning [44]: The adversarial component of GPP mirrors domain-adversarial training, but GPP provides explicit information-theoretic objectives and variational bounds. • Variational Fair Autoencoder [43]: Similar to GPP, this approach uses variational inference for fair representations, but GPP additionally incorporates the utility

V. S YSTEM M ODEL FOR D ISTRIBUTED DATASETS This section extends the GPP framework to distributed settings, addressing privacy concerns in federated learning (FL) environments where multiple parties collaboratively train models without sharing raw data. A. Motivation Federated learning has emerged as an effective paradigm for collaborative model training across distributed IoT devices, with a central server coordinating the learning process [58]. The standard FL process consists of four main steps [51], [59]: 1) Client Selection: The aggregator (server) selects participating clients either randomly or using a selection algorithm based on criteria such as data quality or computational resources. 2) Parameter Distribution: The server distributes the current global model parameters to all selected clients. 3) Local Training: Each client trains the model locally using their private data, computing parameter updates (gradients). 4) Model Aggregation: Clients send their updated parameters to the central server, which aggregates them (e.g., via averaging) to update the global model. This process repeats iteratively until convergence. Despite FL’s benefits, significant privacy concerns remain. Many individuals and organizations are hesitant to participate in distributed learning environments, particularly in sensitive domains such as healthcare [60], finance [61], and wireless communications [62]. Even though raw data remains local, several privacy risks persist: • Gradient Leakage: Recent work has demonstrated that raw training data can be reconstructed from shared gradients [57]. • Model Inversion Attacks: Adversaries can infer sensitive attributes from model parameters [4].

2) Distributed Objective: The goal of Distributed GPP is to jointly learn: 1) Local encoders {Tθm }tm=1 at each client that sanitize raw data u 2) Shared utility classifiers {Qψ j }dj=1 at the aggregator ds 3) Adversary classifiers {Qφ j } j=1 (for training purposes) The distributed optimization objective extends Eq. (39): " du t nm (m) min max ∑ βm ∑ CE(u j , Qψ j (Z (m) )) N {θm },{ψ j } {φ j } m=1 j=1 # (48) ds

Fig. 2. Threat model in federated learning systems. Adversaries may attempt to extract sensitive information at multiple points: the central aggregator, communication channels, or through malicious participants. Distributed GPP addresses these threats by sanitizing data locally before any information leaves the client.

Membership Inference: Attackers can determine whether specific data points were used in training [5]. • Untrusted Aggregators: The central server may be compromised or malicious [63], [64]. Figure 2 illustrates the threat model in a typical FL system, where adversaries may exist at the central server, communication channels, or as malicious participants. To address these challenges, we propose Distributed GPP, which enhances privacy in FL systems by having each client sanitize their data locally using a GPP encoder before sharing. Instead of sharing raw data or gradients computed on raw data, clients share only sanitized representations that preserve utility information while removing sensitive attributes. •

B. Problem Statement Consider a distributed system with t clients (GPP nodes), where each client m ∈ {1, . . . ,t} possesses a local dataset: m D m = {(xim , uim , sim )}ni=1 ,

(m)

− ∑ CE(s j , Qφ j (Z (m) )) + λm LKL j=1

where Z (m) = Tθm (X (m) ) is the sanitized representation from client m, and βm , λm are client-specific trade-off parameters. C. Distributed Architecture Figure 3 illustrates the Distributed GPP architecture, which consists of three types of components: 1) Local GPP Encoders (t clients): Each client m maintains a local encoder Tθm that transforms raw data xim into sanitized representations zim . Sensitive labels sim never leave the client. 2) Central Aggregator: Hosts the shared utility classifiers u {Qψ j }dj=1 that are trained on sanitized data from all clients to perform utility tasks. s 3) Adversary Classifiers: {Qφ j }dj=1 are used during training to ensure the encoders successfully remove sensitive information. These can be located at the aggregator (for centralized adversarial training) or distributed.

(46)

where nm is the number of samples at client m, xim ∈ Rdx is the raw data, uim ∈ Rdu represents utility attributes, and sim ∈ Rds represents sensitive attributes. The complete distributed dataset is: D=

t [ m=1

D m,

with total size N = ∑ nm .

(47)

m=1

1) Data Partitioning: Distributed data can be partitioned in two ways: • Horizontal Partitioning: Each client owns the same set of attributes for different record sets. Formally, all clients share the same feature space X , attribute spaces U and ′ S , but have disjoint sample sets: D m ∩ D m = 0/ for m ̸= m′ . • Vertical Partitioning: Each client owns different attributes for the same set of records. Clients share the same sample identifiers but have disjoint feature spaces. In this work, we focus on horizontal partitioning, which is the most common scenario in federated learning applications such as mobile devices, hospitals, or IoT sensors, where each client has complete feature vectors for their local samples.

Fig. 3. Architecture of Distributed GPP. Each client m has a local encoder Tθm that sanitizes raw data before transmission. The aggregator receives only sanitized representations zim and utility labels uim , while sensitive labels sim remain local. Utility classifiers are trained centrally on aggregated sanitized data, while adversary classifiers ensure privacy during the training phase.

D. Privacy Guarantees in Distributed GPP The Distributed GPP framework provides privacy protection through multiple mechanisms: 1) Data Sanitization: Raw data xim never leaves the client. Only sanitized representations zim are transmitted, which

are trained to contain minimal information about sensitive attributes. 2) Local Sensitive Labels: Sensitive labels sim are used only locally for adversarial training and are never shared with the aggregator or other clients. 3) Dimensionality Reduction: Since dz ≪ dx , the sanitized representations have significantly lower dimensionality than raw data, limiting the information that can potentially leak. 4) Stochastic Encoding: The Gaussian latent space ensures that the mapping from x to z is stochastic, providing additional privacy through randomization. Proposition 2 (Privacy Preservation in Distributed GPP, IID and honest aggregator). Assume (A1) the client datasets are ′ ′ mutually independent: (X (m) , S(m) ) ⊥ (X (m ) , S(m ) ) for m ̸= m′ ; (A2) the aggregator is honest-but-curious, i.e., it follows the protocol but may attempt to infer sensitive attributes from received messages; and (A3) the only quantities transmitted by client m are the sanitized representations Z (m) = Tθm (X (m) ) and the utility labels U (m) . If each local encoder achieves I(Z (m) ; S(m) ) ≤ ε, then for every m′ ∈ {1, . . . ,t}, ′ ′ ′ I {Z (m) ,U (m) }tm=1 ; S(m ) ≤ I(Z (m ) ; S(m ) ) ′

′

+ I(U (m ) ; S(m ) | Z (m ) )

(49)

≤ ε +δ, ′

′

where δ = I(U (m ) ; S(m ) | Z (m ) ) is the residual leakage carried by the utility labels themselves. ′

Proof: Under (A1), (X (m) ,U (m) , Z (m) ) ⊥ S(m ) for m ̸= so each cross-client term contributes zero mutual information. The chain rule for mutual information then ′ ′ ′ ′ gives I({Z (m) ,U (m) }m ; S(m ) ) = I(Z (m ) ,U (m ) ; S(m ) ). Expand′ ′ ′ ′ ′ ′ ′ ′ ing I(Z (m ) ,U (m ) ; S(m ) ) = I(Z (m ) ; S(m ) )+I(U (m ) ; S(m ) | Z (m ) ) ′) ′) (m (m and using I(Z ; S ) ≤ ε yields the bound. m′ ,

Remark 6 (Limits of Proposition 2). The bound holds under three assumptions worth stating explicitly. (A1) excludes settings where clients share latent factors that correlate S across the boundary (e.g., sites that draw from the same patient demographics). (A2) excludes a malicious aggregator that uses the gradients of the shared utility classifiers Qψ j to probe individual clients; mitigating that threat requires additional mechanisms (secure aggregation, differential privacy on the utility classifier updates) which we do not analyze here. (A3) is enforced by Algorithm 2, in which only (zim , uim ) leaves the client. The residual term δ accounts for the fact that the utility ′ ′ label U (m ) may itself carry information about S(m ) when the two are correlated; on benchmarks where they are independent (e.g., MNIST parity vs. digit-sum), δ = 0. E. Distributed Learning Algorithm Algorithm 2 presents the complete Distributed GPP training procedure. The algorithm alternates between two phases: 1) Local Phase: Each client updates its local encoder using local data and the current shared classifiers.

2) Aggregation Phase: The aggregator collects sanitized representations and updates the shared utility and adversary classifiers. F. Communication Efficiency An important advantage of Distributed GPP is its communication efficiency compared to standard federated learning: • Reduced Data Dimensionality: Clients transmit sanitized representations z ∈ Rdz instead of raw data x ∈ Rdx , with dz ≪ dx . For image data where dx may be tens of thousands of pixels, dz can be as small as 100-600 dimensions, representing a compression ratio of 10-100x. • No Gradient Transmission: Unlike standard FL which requires transmitting model gradients (which can be large for deep networks), Distributed GPP transmits only encoded representations. • Bandwidth Savings: The communication cost per sample per round is O(dz + du ) for Distributed GPP, compared to O(|θ |) for gradient-based FL, where |θ | is the number of model parameters. Remark 7 (Position relative to standard FL). We use “federated” here in the operational sense that raw data and sensitive labels remain at the client; we do not employ model averaging in the FedAvg sense. Distributed GPP is structurally closer to split learning with a client-side privacy filter: each client owns (m) a private encoder Tθm and a private adversary Qφ j , and the aggregator owns the shared utility classifiers {Qψ j }, which are trained directly on the union of sanitized representations rather than on aggregated gradients. Compared to gradientbased FL this design has two consequences. On the privacy side, even the model updates in standard FL have been shown to leak training data [57]; sanitized representations bypass that leakage channel. On the cost side, the per-round payload from a single client is b(dz + du ) scalars for Distributed GPP versus |θ | scalars for gradient-based FL, where b is the per-client batch size. The crossover is therefore b(dz + du ) < |θ |: Distributed GPP transmits less than gradient-based FL whenever the local batch is small relative to the encoder size. For the CelebA experiment with b = 64, dz = 256, du = 1 and an encoder with |θ | ≈ 5 × 105 parameters, this yields roughly a 30× reduction in per-round bytes from each client. G. Convergence Considerations The convergence of Distributed GPP depends on several factors: 1) Data Heterogeneity: When client data distributions differ significantly (non-IID data), convergence may be slower. The client-specific parameters βm and λm can be tuned to account for heterogeneity. 2) Adversary Strength: Local adversary classifiers must be sufficiently powerful to provide meaningful privacy gradients. Weak adversaries may lead to encoders that leak sensitive information. 3) Communication Frequency: More frequent aggregation rounds generally improve convergence but increase communication overhead.

Algorithm 2 Distributed GPP Training Require: Number of clients t; batch size b; classifier update steps k; trade-off parameters {βm }tm=1 ; regularization weights {λm }tm=1 ; learning rate α; number of communication rounds R u Ensure: Trained local encoders {Tθm }tm=1 , utility classifiers {Qψ j }dj=1 du ds t 1: Initialize: Local encoders {Tθm }m=1 , utility classifiers {Qψ j } j=1 , adversary classifiers {Qφ j } j=1 with random weights 2: for round r = 1 to R do 3: // Phase 1: Local encoding at each client 4: for each client m = 1 to t in parallel do 5: Sample mini-batch Bm = {(xim , uim , sim )}bi=1 from D m 6: µ m , Lm ← Tθm ({xim }) 7: Sample ε ∼ N (0, I) 8: {zim } ← µ m + Lm ⊙ ε 9: Send {(zim , uim )}bi=1 to aggregator ▷ sim stays local 10: end for 11: // Phase 2: Aggregator updates classifiers S 12: Collect sanitized data: Z ← tm=1 {(zim , uim )} 13: for τ = 1 to k do 14: for j = 1 to du do ▷ Update utility classifiers 15: Lψ j ← |Z1 | ∑(z,u)∈Z CE(u j , Qψ j (z)) 16: ψ j ← ψ j − α · Adam(∇ψ j Lψ j ) 17: end for 18: end for u 19: Broadcast updated {Qψ j }dj=1 to all clients 20: // Phase 3: Local encoder and adversary updates 21: for each client m = 1 to t in parallel do u 22: Receive updated utility classifiers {Qψ j }dj=1 23: for τ = 1 to k do 24: Sample mini-batch Bm = {(xim , uim , sim )}bi=1 25: µ m , Lm ← Tθm ({xim }) 26: {zim } ← µ m + Lm ⊙ ε, where ε ∼ N (0, I) 27: for j = 1 to ds do ▷ Update local adversary (m) (m) 28: Lφ j ← b1 ∑bi=1 CE(sij , Qφ j (zim )) 29: 30: 31: 32: 33: 34:

(m)

φj

(m)

← φj

(m)

− α · Adam(∇ (m) Lφ j ) φj

end for end for // Update local encoder (m) dz 2 + σ 2 − log σ 2 − 1) (µm,l Compute KL: LKL ← 21 ∑l=1 m,l m,l Compute encoder loss: " du 1 b Lθm ← ∑ βm ∑ CE(uij , Qψ j (zim )) b i=1 j=1 # ds

(m)

− ∑ CE(sij , Qφ j (zim )) + λm LKL j=1

35: θm ← θm − α · Adam(∇θm Lθm ) 36: end for 37: end for d

u 38: return {Tθm }tm=1 , {Qψ j } j=1

TABLE II DATASET C HARACTERISTICS Dataset

Train

Test

|U |

|S |

MNIST CelebA HAPT

784 12,288 561

60,000 162,770 7,767

10,000 19,962 3,162

19 2 6

2 2 30

Under standard assumptions of Lipschitz-continuous gradients and bounded variance, the alternating optimization in Algorithm 2 converges to a stationary point of the distributed objective (48). A detailed convergence analysis is beyond the scope of this paper but follows similar arguments to those in federated optimization literature [65]. VI. EXPERIMENTS This section presents comprehensive experimental evaluation of the GPP framework. We first describe the experimental setup, including datasets, evaluation metrics, and baseline methods. We then present six key experiments that validate the effectiveness of GPP for privacy-preserving data release. A. Experimental Setup 1) Datasets: We evaluate GPP on three benchmark datasets commonly used in privacy-preserving machine learning research: • MNIST [66]: Hand-written digit images (28 × 28 pixels, dx = 784). Following [67], we construct a two-digit composite dataset where the utility attribute U is the sum of two digits (0-18), and the sensitive attribute S is whether the sum is even or odd (binary classification). • CelebA [68]: Celebrity face images (64 × 64 × 3 RGB, dx = 12, 288). We use “Smiling” as the utility attribute and “Gender” as the sensitive attribute, following the privacypreserving literature [44]. • HAPT-Recognition [69]: Human activity sensor data from smartphones (dx = 561 features). The utility attribute is activity type (6 classes: walking, walking upstairs, walking downstairs, sitting, standing, laying), and the sensitive attribute is subject identity (30 subjects). Table II summarizes the dataset characteristics. 2) Evaluation Metrics: We adopt the Area Under the ROC Curve (AUC) as the primary evaluation metric, following established practices in privacy-preserving machine learning [43], [44]: • Utility AUC: Measures how well the utility attribute U can be predicted from the sanitized representation Z. Higher values indicate better utility preservation. Target: ≈ 1.0. • Adversary AUC (Privacy AUC): Measures how well the sensitive attribute S can be inferred from Z. Lower values indicate better privacy protection. Target: ≈ 0.5 (random guessing). Audit Protocol: To ensure unbiased evaluation, we train fresh probe classifiers on the sanitized test representations, rather than using the classifiers from training. This prevents

Fig. 4. Baseline comparison on MNIST. (a) Utility AUC vs. bottleneck dimension. GPP achieves comparable utility to No-Privacy while (b) achieving significantly lower adversary AUC, approaching the random guess baseline (dashed line at 0.5).

artificially inflated privacy scores that could result from adversaries that learned to fail during training. Each probe is a 3-layer MLP with hidden widths (256, 128) and ReLU activations—the same capacity as the training adversary— trained for 30 epochs with the Adam optimizer at learning rate 10−4 on the sanitized training-set representations, then evaluated on the held-out sanitized test set. We do not perform hyperparameter search at audit time, which means the probe could in principle be strengthened; we revisit this point under Limitations (Section VI-K). 3) Baseline Methods: We compare GPP against three baseline approaches: • No-Privacy: Standard autoencoder that optimizes only for utility, without any privacy constraints. Represents the upper bound on utility but provides no privacy protection. • Random Projection: Projects data onto a random lowdimensional subspace. Provides some privacy through dimensionality reduction but without learning. • Noisy Encoder: Adds Gaussian noise to the latent representation during training, similar to differential privacy approaches. The noise scale is tuned for best utilityprivacy trade-off. 4) Implementation Details: All experiments use the following hyperparameters unless otherwise specified: learning rate α = 10−4 , batch size b = 64, KL regularization weight λ = 0.01, classifier update steps k = 2, and Adam optimizer [70]. For CNN encoders (CelebA), we use 3 convolutional layers with stride 2. For MLP encoders (MNIST, HAPT), we use two hidden layers with 512 and 256 units. All experiments are run on a single NVIDIA Tesla V100 GPU. B. Experiment 1: Baseline Comparison Our first experiment compares GPP against baseline methods across different bottleneck dimensions dz . Setup: We train each method with bottleneck dimensions dz ∈ {40, 80, 120, 160, 200} on the MNIST dataset with β = 1.0. Each configuration is trained for 50 epochs. Results: Figure 4 shows the utility and privacy AUC for each method. Table III provides numerical results for dz = 120. Analysis: GPP achieves utility AUC of 0.978, only 0.9% lower than the No-Privacy upper bound of 0.987, while dramatically reducing adversary AUC from 0.943 to 0.531— approaching the random-guess baseline of 0.5. The Noisy

TABLE III BASELINE C OMPARISON R ESULTS (dz = 120, MNIST) Method No-Privacy Random Projection Noisy Encoder GPP (Ours)

Utility AUC ↑

Adversary AUC ↓

0.987 0.823 0.941 0.978

0.943 0.762 0.871 0.531

Fig. 6. Effect of bottleneck dimension. (a) Utility AUC increases with dz until saturation around dz = 120. (b) Adversary AUC remains close to 0.5 for smaller dz but increases for larger dimensions. (c) Compression ratio vs. performance trade-off. TABLE V B OTTLENECK D IMENSION E FFECT (β = 1.0, MNIST)

Fig. 5. Privacy-utility trade-off controlled by β . (a) Pareto frontier showing achievable (Utility AUC, Adversary AUC) pairs. (b) Effect of β : smaller β prioritizes privacy (lower adversary AUC), larger β prioritizes utility.

Compression

Utility AUC

Adversary AUC

Gap

40 80 120 160 200

19.6× 9.8× 6.5× 4.9× 3.9×

0.934 0.968 0.978 0.981 0.984

0.508 0.517 0.531 0.563 0.612

0.008 0.017 0.031 0.063 0.112

TABLE IV E FFECT OF β ON P RIVACY-U TILITY T RADE - OFF β

Utility AUC

Adversary AUC

Gap to 0.5

0.1 0.5 1.0 2.0 4.0 8.0

0.892 0.956 0.978 0.983 0.986 0.987

0.512 0.523 0.531 0.587 0.692 0.814

0.012 0.023 0.031 0.087 0.192 0.314

Encoder achieves moderate privacy (0.871) but at a greater utility cost (0.941). Random Projection fails to preserve utility effectively (0.823). The statistical robustness of these numbers is established in Experiment 6 below. C. Experiment 2: Privacy-Utility Trade-off (β Sweep) This experiment demonstrates that GPP provides controllable privacy-utility trade-off through the β parameter. Setup: We train GPP with β ∈ {0.1, 0.5, 1.0, 2.0, 4.0, 8.0} on MNIST with dz = 120. Results: Figure 5 shows the Pareto frontier and the effect of β . Analysis: The results confirm that β effectively controls the trade-off. At β = 0.1, GPP achieves near-perfect privacy (adversary AUC = 0.512) with utility AUC = 0.892. At β = 8.0, utility approaches the No-Privacy baseline (0.987) but privacy degrades (0.814). The optimal operating point depends on application requirements; β = 1.0 provides a balanced trade-off. D. Experiment 3: Bottleneck Dimension Effect

Fig. 7. Ablation study on adversarial training steps k. (a) Bar chart showing utility and adversary AUC. At k = 0, the adversary easily extracts sensitive information. (b) Privacy gap (—AUC - 0.5—) decreases rapidly as k increases, demonstrating the necessity of adversarial training. TABLE VI E FFECT OF A DVERSARIAL T RAINING S TEPS k k

Utility AUC

Adversary AUC

Privacy Gap

0 1 2 3 5 10

0.982 0.979 0.978 0.976 0.974 0.971

0.891 0.623 0.531 0.519 0.512 0.508

0.391 0.123 0.031 0.019 0.012 0.008

Analysis: Smaller bottleneck dimensions provide stronger privacy guarantees (closer to 0.5 AUC) due to the information bottleneck effect—less capacity means less information about S can pass through. However, utility also degrades for very small dz . The sweet spot for MNIST is dz ≈ 120, achieving 6.5× compression with utility AUC = 0.978 and adversary AUC = 0.531. E. Experiment 4: Ablation Study on Adversarial Training (k Steps)

This ablation study demonstrates that adversarial training is This experiment investigates how the bottleneck dimension essential for privacy protection. dz affects the compression-performance trade-off. Setup: We vary dz ∈ {20, 40, 60, 80, 100, 120, 160, 200, 300, 400} Setup: We train GPP with classifier update steps k ∈ {0, 1, 2, 3, 5, 10} on MNIST with dz = 120 and β = 1.0. with β ∈ {0.5, 1.0, 2.0} on MNIST. Results: Figure 6 shows the effect of bottleneck dimension. Results: Figure 7 shows the effect of k on performance.

Fig. 8. Robustness to U-S correlation. (a) Utility AUC degrades gracefully with increasing correlation for both methods. (b) GPP maintains privacy protection (AUC near 0.5) even at high correlation, while No-Privacy leaks sensitive information. TABLE VII ROBUSTNESS TO U TILITY-S ENSITIVE C ORRELATION GPP

Fig. 9. Statistical significance over 10 runs with error bars showing ±1 standard deviation. (a) Utility performance is consistent across runs. (b) GPP consistently achieves adversary AUC near 0.5 with low variance. TABLE VIII S TATISTICAL R ESULTS ( MEAN ± STD , dz = 120, n = 10 RUNS )

No-Privacy

Utility

Adversary

Utility

Adversary

0.0 0.4 0.8 1.0

0.978 0.961 0.923 0.867

0.531 0.547 0.568 0.592

0.987 0.982 0.971 0.956

0.943 0.951 0.967 0.981

Analysis: Without adversarial training (k = 0), the encoder does not learn to remove sensitive information, resulting in high adversary AUC (0.891). Even a single adversary update step (k = 1) dramatically improves privacy (0.623). The privacy gap continues to decrease with larger k, approaching 0.008 at k = 10. We use k = 2 as the default, which provides excellent privacy (0.031 gap) without significantly impacting utility or training time. F. Experiment 5: Robustness to Utility-Sensitive Correlation Real-world data often exhibits correlations between utility and sensitive attributes. This experiment tests GPP’s robustness to such correlations. Setup: We create synthetic datasets with controlled correlation ρ ∈ {0.0, 0.2, 0.4, 0.6, 0.8, 1.0} between U and S, where ρ = 0 means independent and ρ = 1 means fully correlated. Results: Figure 8 shows performance across different correlation levels. Analysis: As expected, utility degrades when U and S are highly correlated because removing S information inevitably removes some U information. However, GPP maintains strong privacy protection even at ρ = 0.8 (adversary AUC = 0.568), while No-Privacy completely fails (adversary AUC = 0.967). This demonstrates GPP’s ability to disentangle correlated attributes.

Method

Utility AUC

Adversary AUC

No-Privacy Noisy Encoder GPP (Ours)

0.987 ± 0.003 0.943 ± 0.011 0.976 ± 0.005

0.941 ± 0.008 0.867 ± 0.023 0.528 ± 0.012

TABLE IX GPP P ERFORMANCE ACROSS DATASETS (β = 1.0) Dataset

Compression

Util. AUC

Adv. AUC

Gap

MNIST CelebA HAPT

120 256 80

6.5× 48× 7×

0.978 0.924 0.891

0.531 0.547 0.523

0.031 0.047 0.023

Analysis: GPP exhibits low variance in both metrics, demonstrating stable training dynamics. The utility standard deviation (0.005) is comparable to No-Privacy (0.003), while adversary AUC maintains consistent privacy protection (0.528 ± 0.012). This confirms the reproducibility of our results. The mean utility AUC of 0.976 in Table VIII differs slightly from the 0.978 reported in Table III because the latter reports a single representative run while the former is the mean over ten seeds. H. Multi-Dataset Evaluation Table IX summarizes GPP performance across all three benchmark datasets. GPP consistently achieves strong privacy protection (adversary AUC within 0.05 of random guessing) across all datasets while maintaining high utility (¿ 0.89 AUC). The CelebA results are particularly notable, achieving 48× compression while protecting gender information. I. Distributed GPP Evaluation

G. Experiment 6: Statistical Significance This experiment verifies the reproducibility and statistical significance of our results. Setup: We run GPP, No-Privacy, and Noisy Encoder 10 times with different random seeds on MNIST with dz ∈ {40, 80, 120, 160}. Results: Figure 9 shows mean ± standard deviation across runs.

Finally, we evaluate the distributed GPP algorithm with 5 clients on the HAPT dataset, which naturally partitions by subject. Distributed GPP achieves comparable performance to centralized training while providing the additional privacy guarantee that raw data never leaves the client. Heterogeneous β values (client-specific trade-offs) provide slightly better privacy with minimal utility loss.

TABLE X D ISTRIBUTED GPP R ESULTS (HAPT, 5 CLIENTS ) Configuration Single GPP (centralized) Distributed GPP (5 clients) Distributed + heterogeneous β

Utility AUC

Adversary AUC

0.891 0.887 0.883

0.523 0.518 0.512

J. Discussion Taken together, the six experiments tell a consistent story. At the default operating point (β = 1.0, balanced bottleneck), GPP reduces adversary AUC to within 0.05 of random guessing across all three benchmarks (Table IX) while keeping utility AUC above 0.89. Outside this default, β and dz trace out a smooth Pareto frontier (Tables IV, V): pushing β higher recovers utility at the cost of privacy, and enlarging dz does the same. We do not claim a free lunch—the gain on the privacy axis is paid for in utility but the trade-off is exposed as a single hyperparameter and is monotone in the expected direction. The ablation on adversary update frequency k (Table VI) is the strongest evidence that the privacy gain is doing real work and is not an artifact of the bottleneck alone: with k = 0, the adversary recovers the sensitive attribute almost perfectly (AUC 0.891) even though dz already imposes a 6.5× compression. The compression is necessary but not sufficient; the adversarial signal is what drives P(S | Z) toward the prior. The robustness experiment (Table VII) addresses the common reviewer concern about correlated U and S: utility degrades gracefully as ρ increases (because removing S-information necessarily removes some U-information when the two are correlated),but the privacy guarantee holds—adversary AUC at ρ = 0.8 is 0.568, compared with 0.967 for the No-Privacy baseline. Finally, the federated experiment (Table X) shows that distributing the GPP encoder across five clients on HAPTRecognition matches the centralized utility-privacy operating point to within 0.005 AUC on both axes, while delivering the additional guarantee that raw data and sensitive labels never leave the client. We view this as an existence result rather than a thorough study: the heterogeneous-β row hints at room for further per-client tuning, and convergence under non-IID partitioning is left to follow-up work. K. Limitations and Open Questions We identify four limitations that scope the claims of this paper and motivate further work. Comparison with deep adversarial baselines. Our experimental comparisons are against three baselines that act as references rather than as competing methods: an unconstrained autoencoder (utility upper bound), a random projection (compression-only baseline), and a noisy encoder (DP-style baseline). The most directly comparable published methods are adversarial censoring [44] and the Variational Fair Autoencoder [43]. A like-for-like comparison on MNIST, CelebA, and HAPT-Recognition with the same utility/sensitive splits would more sharply locate GPP relative to that prior work, and we plan to include this in an extended version.

Tightness of the variational bound. Theorem 1 guarantees a lower bound on I(Z; S) that becomes tight only when Qφ (S | Z) = P(S | Z). In practice, the tightness depends on the capacity of the adversary network used during training. Although our audit protocol re-trains a fresh probe classifier on the test set, that probe shares the architecture of the training adversary; a substantially larger post-hoc probe might extract residual sensitive information that the training adversary missed. A systematic capacity-sweep audit would be informative. Distributed experiment scope. Table X reports a single configuration (five clients, HAPT-Recognition partitioned by subject, IID β unless noted otherwise). A complete study should sweep client counts, characterize convergence under non-IID partitioning, measure realized communication cost in bytes per round, and compare against FedAvg with differential privacy applied to the model updates. The numbers reported here establish that the framework is operable in the distributed setting, not that it dominates competing federated approaches. Reproducibility of the controlled-correlation experiment. Table VII reports performance under a controlled correlation ρ between U and S. The data is synthetic and the generation procedure is described only briefly; a fuller specification (the marginal of X, the conditional construction of (U, S), and the protocol for varying ρ while holding other distributional properties fixed) is needed for independent replication. A natural alternative is to reproduce this analysis on real correlated attribute pairs in CelebA (e.g., Smiling as utility and Heavy_Makeup as sensitive, which exhibit non-trivial natural correlation). VII. C ONCLUSIONS We have presented GPP, a privacy-preserving data-release framework grounded in information-theoretic principles that protects designated sensitive attributes against adversarial inference while preserving designated utility attributes. The framework rests on a variational saddle-point objective that combines a lower bound on I(Z; S) with an upper bound on H(U | Z), regularized by a Gaussian latent prior, and trained by alternating optimization between encoder, utility classifier, and adversary. We then extended GPP to a distributed setting in which each client owns a private encoder and a private adversary while only sanitized representations and utility labels are transmitted to the aggregator, providing instance-level privacy protection on top of the standard “raw data stays local” guarantee of federated learning. On three standard benchmarks, GPP attains utility AUC comparable to the unconstrained-autoencoder upper bound (e.g., 0.978 versus 0.987 on MNIST, a gap of less than one percentage point; Table III), while reducing the adversary’s success on the private attribute to near-random guessing (adversary AUC ≈ 0.53). The federated extension matches the centralized utility-privacy operating point to within 0.005 AUC on both axes (Table X). The Limitations subsection (Section VI-K) scopes these claims and identifies the experimental gaps that remain. Two extensions are immediate. First, a sharper analysis of variational-bound tightness in the regime where the trainingtime adversary is capacity-limited would clarify what an

empirical adversary AUC near 0.5 certifies in informationtheoretic terms. Second, combining GPP’s instance-level protection with a complementary differential-privacy noise mechanism on the released Z, or with secure aggregation on the shared utility classifier, would broaden the threat model the framework addresses to include malicious aggregators.

R EFERENCES [1] I. Wagner and D. Eckhoff, “Technical privacy metrics: A systematic survey,” ACM Computing Surveys (CSUR), vol. 51, no. 3, pp. 1–38, 2018. [2] A. Narayanan and V. Shmatikov, “Robust de-anonymization of large sparse datasets,” in 2008 IEEE Symposium on Security and Privacy. IEEE, 2008, pp. 111–125. [3] A. D. Sarwate and K. Chaudhuri, “Signal processing and machine learning with differential privacy: Algorithms and challenges for continuous data,” IEEE Signal Processing Magazine, vol. 30, no. 5, pp. 86–94, 2013. [4] M. Fredrikson, S. Jha, and T. Ristenpart, “Model inversion attacks that exploit confidence information and basic countermeasures,” in Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security, 2015, pp. 1322–1333. [5] R. Shokri, M. Stronati, C. Song, and V. Shmatikov, “Membership inference attacks against machine learning models,” in 2017 IEEE Symposium on Security and Privacy (SP). IEEE, 2017, pp. 3–18. [6] T. Ma, H. Zhou, B. Qian, N. Cheng, X. Shen, X. Chen, and B. Bai, “UAV-LEO integrated backbone: A ubiquitous data collection approach for B5G Internet of remote things networks,” IEEE Journal on Selected Areas in Communications, 2021. [7] X. Liu, H. Li, G. Xu, Z. Chen, X. Huang, and R. Lu, “Privacy-enhanced federated learning against poisoning adversaries,” IEEE Transactions on Information Forensics and Security, 2021. [8] L. Lyu, J. Yu, K. Nandakumar, Y. Li, X. Ma, J. Jin, H. Yu, and K. S. Ng, “Towards fair and privacy-preserving federated deep models,” IEEE Transactions on Parallel and Distributed Systems, vol. 31, no. 11, pp. 2524–2541, 2020. [9] L. Sweeney, “k-anonymity: A model for protecting privacy,” International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems, vol. 10, no. 05, pp. 557–570, 2002. [10] A. Machanavajjhala, J. Gehrke, D. Kifer, and M. Venkitasubramaniam, “l-diversity: Privacy beyond k-anonymity,” in 22nd International Conference on Data Engineering (ICDE’06). IEEE, 2006, pp. 24–24. [11] N. Li, T. Li, and S. Venkatasubramanian, “t-closeness: Privacy beyond kanonymity and l-diversity,” in 2007 IEEE 23rd International Conference on Data Engineering. IEEE, 2007, pp. 106–115. [12] C. Dwork, A. Roth, et al., “The algorithmic foundations of differential privacy,” Foundations and Trends in Theoretical Computer Science, vol. 9, no. 3–4, pp. 211–407, 2014. [13] L. Oneto, S. Ridella, and D. Anguita, “Differential privacy and generalization: Sharper bounds with applications,” Pattern Recognition Letters, vol. 89, pp. 31–38, 2017. [14] D. Kifer and A. Machanavajjhala, “No free lunch in data privacy,” in Proceedings of the 2011 ACM SIGMOD International Conference on Management of Data, 2011, pp. 193–204. [15] C. Liu, S. Chakraborty, and P. Mittal, “Dependence makes you vulnerable: Differential privacy under dependent tuples,” in NDSS, vol. 16, 2016, pp. 21–24. [16] R. Gilad-Bachrach, N. Dowlin, K. Laine, K. Lauter, M. Naehrig, and J. Wernsing, “CryptoNets: Applying neural networks to encrypted data with high throughput and accuracy,” in International Conference on Machine Learning, 2016, pp. 201–210. [17] C. Carlet, Boolean functions for cryptography and coding theory. Cambridge University Press, 2021. [18] A. C. Yao, “Protocols for secure computations,” in 23rd Annual Symposium on Foundations of Computer Science (sfcs 1982). IEEE, 1982, pp. 160–164. [19] Y. Lindell, “Secure multiparty computation (MPC),” IACR Cryptology ePrint Archive, vol. 2020, p. 300, 2020. [20] H. K. Patil and R. Seshadri, “Big data security and privacy issues in healthcare,” in 2014 IEEE International Congress on Big Data. IEEE, 2014, pp. 762–765.

[21] M. Abu-Tair, S. Djahel, P. Perry, B. Scotney, U. Zia, J. M. Carracedo, and A. Sajjad, “Towards secure and privacy-preserving IoT enabled smart home: Architecture and experimental study,” Sensors, vol. 20, no. 21, p. 6131, 2020. [22] S. Moriai, “Privacy-preserving deep learning via additively homomorphic encryption,” in 2019 IEEE 26th Symposium on Computer Arithmetic (ARITH). IEEE, 2019, pp. 198–198. [23] C.-Y. Sun, A. C.-H. Wu, and T. Hwang, “A novel privacy-preserving deep learning scheme without a cryptography component,” Computers & Electrical Engineering, vol. 94, p. 107325, 2021. [24] S. Matwin, “Privacy-preserving data mining techniques: Survey and challenges,” in Discrimination and Privacy in the Information Society. Springer, 2013, pp. 209–221. [25] R. Mendes and J. P. Vilela, “Privacy-preserving data mining: Methods, metrics, and applications,” IEEE Access, vol. 5, pp. 10 562–10 582, 2017. [26] A. Korolova, “Privacy violations using microtargeted ads: A case study,” in Data Mining Workshops (ICDMW), 2010 IEEE International Conference on. IEEE, 2010, pp. 474–482. [27] I. Psychoula, E. Merdivan, D. Singh, L. Chen, F. Chen, S. Hanke, J. Kropf, A. Holzinger, and M. Geist, “A deep learning approach for privacy preservation in assisted living,” arXiv preprint arXiv:1802.09359, 2018. [28] J. Zhao, R. Mortier, J. Crowcroft, and L. Wang, “Privacy-preserving machine learning based data analytics on edge devices,” arXiv preprint, University of Cambridge, 2018. [29] B. I. P. Rubinstein, P. L. Bartlett, L. Huang, and N. Taft, “Learning in a large function space: Privacy-preserving mechanisms for svm learning,” arXiv preprint arXiv:0911.5708, 2009. [30] K. Chaudhuri, A. D. Sarwate, and K. Sinha, “A near-optimal algorithm for differentially-private principal components,” Journal of Machine Learning Research, vol. 14, no. 1, pp. 2905–2943, 2013. [31] M. Abadi, A. Chu, I. Goodfellow, H. B. McMahan, I. Mironov, K. Talwar, and L. Zhang, “Deep learning with differential privacy,” in Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security. ACM, 2016, pp. 308–318. [32] M. Banerjee and S. Chakravarty, “Privacy preserving feature selection for distributed data using virtual dimension,” in Proceedings of the 20th ACM International Conference on Information and Knowledge Management. ACM, 2011, pp. 2281–2284. [33] Y. Jafer, S. Matwin, and M. Sokolova, “A framework for a privacy-aware feature selection evaluation measure,” in 2015 13th Annual Conference on Privacy, Security and Trust (PST). IEEE, 2015, pp. 62–69. [34] K. Xu, T. Cao, S. Shah, C. Maung, and H. Schweitzer, “Cleaning the null space: A privacy mechanism for predictors,” in Thirty-First AAAI Conference on Artificial Intelligence, 2017. [35] M. Enev, J. Jung, L. Bo, X. Ren, and T. Kohno, “Sensorsift: Balancing sensor data privacy and utility in automated face understanding,” in Proceedings of the 28th Annual Computer Security Applications Conference. ACM, 2012, pp. 149–158. [36] Y. O. Basciftci, Y. Wang, and P. Ishwar, “On privacy-utility tradeoffs for constrained data release mechanisms,” in 2016 Information Theory and Applications Workshop (ITA). IEEE, 2016, pp. 1–6. [37] Y. Wang, Y. O. Basciftci, and P. Ishwar, “Privacy-utility tradeoffs under constrained data release mechanisms,” arXiv preprint arXiv:1710.09295, 2017. [38] A. Makhdoumi, S. Salamatian, N. Fawaz, and M. Médard, “From the information bottleneck to the privacy funnel,” in 2014 IEEE Information Theory Workshop (ITW). IEEE, 2014, pp. 501–505. [39] F. du Pin Calmon and N. Fawaz, “Privacy against statistical inference,” in 2012 50th Annual Allerton Conference on Communication, Control, and Computing (Allerton). IEEE, 2012, pp. 1401–1408. [40] N. Tishby, F. C. Pereira, and W. Bialek, “The information bottleneck method,” arXiv preprint physics/0004057, 2000. [41] A. Creswell, Y. Mohamied, B. Sengupta, and A. A. Bharath, “Adversarial information factorization,” arXiv preprint arXiv:1711.05175, 2017. [42] J. Klys, J. Snell, and R. Zemel, “Learning latent subspaces in variational autoencoders,” in Advances in Neural Information Processing Systems, 2018, pp. 6445–6455. [43] C. Louizos, K. Swersky, Y. Li, M. Welling, and R. Zemel, “The variational fair autoencoder,” arXiv preprint arXiv:1511.00830, 2015. [44] H. Edwards and A. Storkey, “Censoring representations with an adversary,” arXiv preprint arXiv:1511.05897, 2015. [45] J. Hamm, “Preserving privacy of continuous high-dimensional data with minimax filters,” in Artificial Intelligence and Statistics, 2015, pp. 324– 332.

[46] Y. Ganin, E. Ustinova, H. Ajakan, P. Germain, H. Larochelle, F. Laviolette, M. Marchand, and V. Lempitsky, “Domain-adversarial training of neural networks,” Journal of Machine Learning Research, vol. 17, no. 59, pp. 1–35, 2016. [47] D. Peteiro-Barral and B. Guijarro-Berdiñas, “A survey of methods for distributed machine learning,” Progress in Artificial Intelligence, vol. 2, no. 1, pp. 1–11, 2013. [48] D. Zhang, X. Chen, D. Wang, and J. Shi, “A survey on collaborative deep learning and privacy-preserving,” in 2018 IEEE Third International Conference on Data Science in Cyberspace (DSC). IEEE, 2018, pp. 652–658. [49] B. K. Beaulieu-Jones, W. Yuan, S. G. Finlayson, and Z. S. Wu, “Privacypreserving distributed deep learning for clinical data,” arXiv preprint arXiv:1812.01484, 2018. [50] J. Jeon, J. Kim, J. Kim, K. Kim, A. Mohaisen, and J.-K. Kim, “Privacypreserving deep learning computation for geo-distributed medical bigdata platforms,” in 2019 49th Annual IEEE/IFIP International Conference on Dependable Systems and Networks–Supplemental Volume (DSNS). IEEE, 2019, pp. 3–4. [51] Q. Yang, Y. Liu, T. Chen, and Y. Tong, “Federated machine learning: Concept and applications,” ACM Transactions on Intelligent Systems and Technology (TIST), vol. 10, no. 2, pp. 1–19, 2019. [52] J. Xu, B. S. Glicksberg, C. Su, P. Walker, J. Bian, and F. Wang, “Federated learning for healthcare informatics,” Journal of Healthcare Informatics Research, pp. 1–19, 2020. [53] M. Chen, Z. Yang, W. Saad, C. Yin, H. V. Poor, and S. Cui, “A joint learning and communications framework for federated learning over wireless networks,” IEEE Transactions on Wireless Communications, 2020. [54] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial nets,” in Advances in Neural Information Processing Systems, 2014, pp. 2672– 2680. [55] D. P. Kingma and M. Welling, “Auto-encoding variational Bayes,” arXiv preprint arXiv:1312.6114, 2013. [56] T. M. Cover and J. A. Thomas, Elements of information theory. John Wiley & Sons, 2012. [57] L. Lyu, H. Yu, and Q. Yang, “Threats to federated learning: A survey,” arXiv preprint arXiv:2003.02133, 2020. [58] G. Cheng, K. Chadha, and J. Duchi, “Fine-tuning is fine in federated learning,” arXiv preprint arXiv:2108.07313, 2021. [59] M. Alazab, S. P. RM, M. Parimala, P. Reddy, T. R. Gadekallu, and Q.-V. Pham, “Federated learning for cybersecurity: Concepts, challenges and future directions,” IEEE Transactions on Industrial Informatics, 2021. [60] J. Xu, B. S. Glicksberg, C. Su, P. Walker, J. Bian, and F. Wang, “Federated learning for healthcare informatics,” Journal of Healthcare Informatics Research, vol. 5, no. 1, pp. 1–19, 2021. [61] Z. Li, V. Sharma, and S. P. Mohanty, “Preserving data privacy via federated learning: Challenges and solutions,” IEEE Consumer Electronics Magazine, vol. 9, no. 3, pp. 8–16, 2020. [62] S. Niknam, H. S. Dhillon, and J. H. Reed, “Federated learning for wireless communications: Motivation, opportunities, and challenges,” IEEE Communications Magazine, vol. 58, no. 6, pp. 46–51, 2020. [63] C. Ma, J. Li, M. Ding, H. H. Yang, F. Shu, T. Q. S. Quek, and H. V. Poor, “On safeguarding privacy and security in the framework of federated learning,” IEEE Network, 2020. [64] B. Hou, J. Gao, X. Guo, T. Baker, Y. Zhang, Y. Wen, and Z. Liu, “Mitigating the backdoor attack by federated filters for industrial IoT applications,” IEEE Transactions on Industrial Informatics, 2021. [65] T. Li, A. K. Sahu, A. Talwalkar, and V. Smith, “Federated learning: Challenges, methods, and future directions,” IEEE Signal Processing Magazine, vol. 37, no. 3, pp. 50–60, 2020. [66] Y. LeCun, C. Cortes, and C. J. C. Burges, “The MNIST database of handwritten digits,” http://yann.lecun.com/exdb/mnist/, 1998. [67] J. Hamm, Y. Cao, and M. Belkin, “Learning privately from multiparty data,” in International Conference on Machine Learning, 2016, pp. 555– 563. [68] Z. Liu, P. Luo, X. Wang, and X. Tang, “Deep learning face attributes in the wild,” in Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2015, pp. 3730–3738. [69] J.-L. Reyes-Ortiz, L. Oneto, A. Samà, X. Parra, and D. Anguita, “Transition-aware human activity recognition using smartphones,” Neurocomputing, vol. 171, pp. 754–767, 2016. [70] D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980, 2014.

Zahir Alsulaimawi graduated from the School of Electrical Engineering and Computer Science (EECS) at Oregon State University (OSU) with a Ph.D. in Electrical and Computer Engineering (ECE) and a minor in Computer Science (Machine Learning) in 2021. In addition, he has two master’s degrees in Electrical Engineering, one from the Electrical Engineering Department at the University of Baghdad in 2013 and the other from the EECS at OSU in 2020. OSU’s engineering college awarded him the Outstanding Academic Performance Recognition in 2020. He has been a member of Prof. Liu’s group since September 2017. His research focuses on deep learning, federated learning, multimodal machine learning and signal processing, but he has a broad knowledge of information theory, artificial intelligence, and estimation and prediction.

Huaping Liu received the B.S. and M.S. degrees in electrical engineering from Nanjing University of Posts and Telecommunications, Nanjing, China, in 1987 and 1990, respectively, and the Ph.D. degree in electrical engineering from New Jersey Institute of Technology, Newark, in 1997. From July 1997 to August 2001, he was with Lucent Technologies, Whippany, NJ. Since September 2001, he has been with the EECS, OSU, where he is currently a professor. His research interests include ultra wide band systems, multiple-input multiple-output antenna systems, channel coding, modulation and detection techniques for multiuser communications, machine learning and privacy-preserving. Dr. Liu has published 132 journal articles, 105 conference papers & book chapters, three of which won Best Paper Awards, and one was Best Paper Award Finalist. He also co-authored a textbook on project-based learning in communication systems, the first one that focuses on encouraging students’ active learning. He has graduated with 23 Ph.D. students and 32 M.S. students.

Record · ID 157315 · SHA-256 87dff29b1cb7b925
Retrieved via Conceptio — every document is proof-bundled with source, license, and retrieval metadata.