arXiv CSopen access

HADES: Privacy-Preserving Federated Learning via Selective Feature Encryption and Hybrid Model Fusion

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021

1

HADES: Privacy-Preserving Federated Learning via Selective Feature Encryption and Hybrid Model Fusion

arXiv:2606.22928v1 [cs.CR] 22 Jun 2026

Ergün Batuhan Kaynak , Kerem Bayramoglu , and Sinem Sav

Abstract—In this paper, we address the challenge of privacypreserving training in federated learning (FL) by introducing a novel framework that selectively encrypts only the most privacysensitive features while leaving the remaining data and the corresponding model portion unencrypted. We propose HADES, a hybrid system that identifies and encrypts the most critical features, ensuring both privacy protection and computational efficiency. Unlike fully encrypted FL training pipelines, which suffer from high computational overhead, HADES integrates an encrypted and non-encrypted training pipeline via a fusion mechanism, enabling seamless interaction between encrypted and plaintext model representations. To achieve this, we use PCA to identify and encrypt the most privacy-sensitive features, which significantly reduces reconstruction attack success in FL. Building on this insight, we design a hybrid FL system that trains an end-to-end encrypted network via multiparty homomorphic encryption (MHE) on the selected features while simultaneously training a plaintext network on the remaining features. These two networks are then integrated using a fusion mechanism. We also introduce a general packing scheme that eliminates redundant rotations by considering the entire neural network architecture. Finally, we demonstrate that HADES matches the accuracy of vanilla FL while preserving privacy and achieving optimized runtime through selective encryption. Index Terms—Homomorphic Encryption, Privacy-Preserving Federated Learning, Collaborative Learning

I. I NTRODUCTION In the era of big data and machine learning, data privacy has emerged as a critical challenge. As the demand for highly accurate models grows, so does the need for vast datasets. However, this pursuit often involves handling sensitive information, bringing significant privacy risks. Thus, ensuring ethical data practices while fostering innovation is imperative. To address the growing need for large-scale data while enhancing privacy, federated learning (FL) [1], [2] has emerged as a collective machine learning solution. FL enables model training across vast, distributed datasets stored on local premises, ensuring data remains on its source rather than being centrally collected. Rather than transferring data to a central model, FL brings the model to the clients via an aggregation server. Clients then update the model using their local data, and these local updates from multiple parties are aggregated to construct the global model through iterations. However, despite these advantages, All authors are with the Department of Computer Engineering, Bilkent University, Ankara, Turkey. Corresponding Author: Sinem Sav Ergün Batuhan Kaynak (e-mail: [email protected]). Kerem Bayramoglu (e-mail: [email protected]). Sinem Sav (e-mail: [email protected]). Manuscript received XX XX, XXXX; revised XX XX, XXXX.

numerous attacks have shown that sensitive information remains at risk. Sharing intermediate model updates—whether among parties or with the server—can create vulnerabilities such as input extraction [3]–[5] and membership inference attacks [6]–[8]. To address privacy risks in FL, current research frequently leverages differential privacy (DP) and homomorphic encryption (HE). Several studies incorporate DP to protect input data or intermediate value exchanges between server and FL parties [9]–[11]. Although these techniques help limit privacy attacks, they often compromise data utility and model performance. In addition, several works demonstrate that DP might be insufficient in several FL frameworks [12]. HEbased solutions, on the other hand, generally follow one of two strategies: (i) secure aggregation of client inputs during model updates [13]–[16], or (ii) encrypting the entire FL training process to maintain end-to-end privacy [17]–[20]. While the first approach lacks thorough privacy protection, as the decrypted model on the client side remains susceptible to privacy attacks, the second approach significantly increases computational overhead due to HE operations. For example, in a fully encrypted FL pipeline—where intermediate values, local models, and the global model remain encrypted throughout execution—training can take from several hours to multiple days, depending on the network architecture [17]. To overcome the limitations of HE-based FL, we propose HADES, a novel framework that leverages multiparty homomorphic encryption (MHE) to enhance privacy-preserving neural network training. Our approach utilizes a dual-model structure: an encrypted model and a plaintext model, which are integrated through a fusion mechanism. We first identify the most privacy-sensitive features through principal component analysis (PCA) and encrypt this portion of the model, ensuring it remains encrypted throughout training without decryption. This selective encryption strategy balances privacy and efficiency by minimizing HE overhead while preserving model security. We note here that using PCA for feature selection in this work is purely illustrative and HADES is agnostic to the choice of feature-selection method. Similarly, HADES can be integrated into any encrypted FL work that supports alternative neural network structures or utilizes various packing strategies, as its primary contribution is the fusion component. Unlike traditional secure aggregation, HADES prevents any party—including both clients and the server—from compromising this protected model segment. Meanwhile, the remaining portion that does not expose sensitive information

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021

is trained using the plaintext model. HADES offers benefits along two key dimensions. First, it optimizes HE operations by replacing expensive rotations with redundant parallel computations and reducing the need for bootstrapping through a fusion mechanism. Second, by training both an encrypted and a plaintext model, HADES achieves a practical balance between privacy and computational efficiency, outperforming state-of-the-art fully encrypted FL pipelines. Consequently, this also reduces bandwidth requirements and further enhances the practicality of HE-based FL for large-scale applications. In this paper, we demonstrate how our selective model portion can prevent reconstruction attacks through the improved deep leakage from gradients (iDLG) attack [21]. Then, our analysis indicates that the model utility is preserved and HADES scales linearly with the number of parties in the FL system. Our key contributions are as follows: • An efficient feature-selection-based selective homomorphic encryption approach that encrypts only the most privacy-sensitive features, reducing both computational and communication overhead by minimizing the volume of encrypted data transmitted. • A novel fusion design for privacy-preserving FL where efficiency gains arise naturally: reducing the encrypted feature dimension lowers HE-side overhead, while an inexpensive plaintext support network adds capacity without incurring any HE cost. • A novel packing strategy for dense neural networks, applied to both end-to-end training and inference, that calculates network padding requirements from a global perspective to eliminate redundant rotations in deeper layers and support single-ciphertext mini-batching. Together, by addressing privacy as a feature-level knowledge exposure problem, this work contributes to the broader goals of privacy-aware data analytics and knowledge discovery in distributed systems. To the best of our knowledge, HADES is the first system to perform fully encrypted model training while incorporating a fusion mechanism between encrypted and plaintext model components. II. R ELATED W ORK In this section, we review prior work on secure aggregation in FL and homomorphic encryption-based solutions. A. Secure Aggregation Secure aggregation has become a foundational component in FL to ensure the privacy of individual client updates. Bonawitz et al. [16] enable a server to compute the sum of client updates without learning any individual update. This is achieved by having clients mask their updates with random pads that cancel out in aggregate, effectively implementing an MPC for summation. Additive HE was also employed in BatchCrypt [15], an efficient system for cross-silo FL. BatchCrypt applies Paillier encryption to randomly projected, quantized gradients, reducing communication and encryption costs with minimal accuracy loss. Park and Lim [22] utilize a cloud server and computation provider to collaboratively aggregate encrypted local model parameters—each encrypted

2

with different keys using a distributed HE scheme that supports partial decryption without revealing individual client data. Gronberg et al. introduced BlindFL [23], which segments model updates and applies FHE selectively, balancing security and computational efficiency. FL schemes have also explored cryptographic aggregation techniques. For instance, Lu et al. [24] propose a decentralized approach that uses consensus protocols and secret sharing to aggregate model updates over dynamic networks without relying on a central server. More recently, Hu and Li [25] propose a FL framework with selective homomorphic encryption, where clients apply HE to only a subset of the model parameters. In this sense, their approach is conceptually similar to ours in its selective use of encryption. However, a key difference is that their work focuses solely on aggregation, whereas we consider the entire training process under encryption. Overall, secure aggregation protects privacy only from the server’s perspective, as the model updates on the client side remain vulnerable to model inversion attacks. B. Homomorphic Encryption (HE) HE enables a party (or parties) to compute on ciphertexts such that the decrypted result matches the outcome of computations on the original plaintexts. In machine learning, HE enables model training and inference directly on encrypted data or models, preserving data privacy throughout the process. Below, we review recent works that leverage HE in both centralized and FL settings, highlighting their core contributions, assumptions, and novel techniques. Encrypted Centralized Training or Inference. Early research on HE in machine learning focused primarily on centralized training or inference tasks [26]–[28]. A notable example is CryptoNets, which demonstrated that neural networks can perform predictions on encrypted images with reasonable accuracy and efficiency [26]. Extending HE to training is far more challenging due to the iterative and complex computations required by gradient-based learning. Nandakumar et al. [29] presented one of the first demonstrations of training a neural network entirely on encrypted data by implementing stochastic gradient descent (SGD) to train a simple fullyconnected network. Encrypted Federated Learning (FL). Beyond secure aggregation, which does not protect against client-side model attacks, several works explore fully encrypted training in FL, ensuring that neither the server nor clients ever see the model in plaintext during training. The first example for neural network training was POSEIDON [17], which employs MHE and enables under-encryption training in FL. By distributing the secret key among multiple parties, it ensures that no single entity can decrypt the model independently. This design offers robustness against collusion in the semi-honest model, remaining secure even if up to N − 1 parties collude. However, their approach remains impractical for FL environments, requiring hours to days of training time. The complexity of training is further improved in H ERCULES [19]. Unlike prior works, our framework, HADES, is the first to support selective encryption within an end-to-end encrypted

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021

FL system. This selective approach enables HADES to encrypt only privacy-sensitive data, leaving non-sensitive information unencrypted. As a result, HADES maintains strong privacy guarantees against client-side attacks while significantly reducing computational overhead compared to systems that encrypt all data indiscriminately. III. BACKGROUND In this section, we provide the preliminaries on federated learning, principal component analysis, multiparty homomorphic encryption (MHE), and fusion mechanisms. We provide the frequently used notation and abbreviations in Supplementary Material A. A. Federated Learning (FL) FL [1], [2] is a decentralized approach that trains a shared model by aggregating locally computed updates on client devices, minimizing the need to transfer large or sensitive data to a central server. A central server initializes a global model and distributes it to a randomly selected subset of participating clients. Each client then trains the model locally using its private dataset before transmitting the updated model parameters back to the server. These locally trained updates are aggregated at the server using various aggregation techniques, e.g., with FedAvg (see Algorithm 1) employing weighted averaging to refine the global model. Let W k, t denote the locally updated model weights of client k, at iteration t. The aggregationPprocess at the server is performed as follows: nk k, t where nk is P the number of data ∆W t = k∈K nt W samples held by client k, and nt = k∈K nk represents the total number of samples across all clients at iteration t. FedAvg allows multiple local updates per client before transmitting model updates to the server. This iterative process continues until the model converges or a predefined number of T iterations. FL thus provides key advantages, such as enhanced privacy, reduced communication costs, and improved scalability for distributed machine learning applications. Algorithm 1 Federated Averaging (FedAvg) Algorithm [2] 1: Server executes: 2: Initialize global model W 1 3: for each iteration t = 1, 2, . . . , T do ▷ Global iterations 4: for each client k ∈ K in parallel do

∇W k,t ← ComputeLocalGradients(W k,t ) end for Aggregation (Server-side): P nt ← k∈K P nk ∆W t ← k∈K nnkt W k,t 8: Model Update (Client-side): W k,t+1 ← W k,t − η ∆W k,t 9: end for 10: Return: Final global model W T 5: 6: 7:

B. Principal Component Analysis (PCA) PCA is a traditional dimensionality reduction technique in machine learning and statistical analysis [30]. It projects

3

high-dimensional data into a lower-dimensional representation while preserving as much variance as possible. PCA is extensively used in applications such as feature extraction, data visualization, and noise filtering. The method is particularly beneficial in scenarios where features exhibit high redundancy or strong correlations among features. This can help with convergence, and prevent overfitting. PCA operates by identifying a set of orthogonal axes, called principal components, that maximize variance in the dataset. Given an input matrix X, PCA first standardizes the data by subtracting the mean (µ) and dividing by the standard deviation (σ) Xs = X−µ σ . Then, the covariance matrix is computed. The principal components correspond to the eigenvectors of this covariance matrix (i.e. direction of variance), and their associated eigenvalues indicate the amount of variance captured (i.e. importance of the vector). The transformation is performed by projecting the original data onto the top i eigenvectors, forming a lower-dimensional representation Z = Xs W i where W i is the matrix of the top i eigenvectors, and Z is the projected data in the reduced space. In this work, we leverage PCA to identify the most privacy-sensitive features motivated by transforming data to a correlated space and by prior work [31]. C. Multiparty Homomorphic Encryption Multiparty Homomorphic Encryption (MHE) is a cryptographic method that extends traditional HE to a collaborative setting involving multiple parties [32]. It allows a group of parties to perform computations on their encrypted data without revealing the underlying plaintexts to each other or to an external entity. This is particularly useful in scenarios where data privacy is paramount, such as collaborative data analysis across different organizations. In MHE, each party encrypts data using a shared public key, enabling the evaluation of functions directly over the aggregated ciphertexts. Decryption typically requires the cooperation of all or a threshold number of parties, ensuring that no single party can independently access the decrypted result [33]. For MHE, we use the CKKS scheme over a cyclotomic ring R = Z[X]/(X N + 1) where N is a power of 2. Each ciphertext c encodes a vector of complex numbers z ∈ CN /2 through an encoding function τ : CN /2 → R. The underlying plaintext space (i.e. the padded representation of the ciphertext) consists of vectors in CN /2 , where homomorphic operations correspond to element-wise operations on these vectors. In summary, each ciphertext in CKKS provides N /2 slots for encoding values. We use the following functionalities throughout this paper: • Key Generation: All parties collaboratively generate their individual secret keys, which then jointly produce the public encryption and evaluation keys. • Collective Bootstrapping: A newly generated ciphertext starts with a level of L. However, each homomorphic operation reduces this level, eventually reaching a point where decryption may fail due to accumulated noise. Collective bootstrapping refreshes the ciphertext to enable further computations.

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021

Encryption: Using the public key, a party encrypts its plaintext data. • Evaluation: Involves computations over the ciphertexts using the evaluation key, producing an encrypted result. • Decryption: Parties collaboratively decrypt the resulting ciphertext to obtain the plaintext output. •

In our work, we leverage MHE to facilitate privacypreserving machine learning across multiple organizations. Each organization encrypts its data locally using the shared public key, allowing collaborative computations on the combined datasets without exposing individual records. The final model or result is decrypted only through the joint effort of authorized parties, ensuring that sensitive information remains confidential throughout the process. This approach aligns with data protection regulations and addresses concerns related to data sharing among organizations.

D. Fusion Mechanisms Deep Model Fusion (DMF) is a machine learning technique where the parameters of multiple models are integrated, enabling more robust and accurate decision-making [34]. Typical DMF consists of multiple sub-networks that process distinct input modalities or representations, followed by a fusion mechanism that combines intermediate latent representations before making a final prediction. Fusion networks are trained like standard neural networks, using iterative forward and backward passes. Each iteration consists of processing a batch of data through the fusion pipeline, and an epoch refers to a complete pass over the dataset. DMF can take on different names based on where the fusion is applied. Early (or low-level) fusion happens when the first few layers or the raw data from different modalities are combined before feature extraction. This allows the model to learn joint feature representations directly from the input space, potentially capturing low-level correlations between modalities. Late (or high-level) fusion, on the other hand, occurs after each modality has undergone independent feature extraction. The outputs of different sub-networks are concatenated or aggregated at the decision (score) level, allowing the model to leverage high-level representations from each modality before making a final prediction. Intermediate fusion (or mid-level fusion) represents a hybrid approach where feature representations from different modalities are combined at one or more points within the network, typically after initial feature extraction but before the final decision layer. In this study, we adopt score-level fusion, combining the logits from individual sub-networks via weighted averaging just before the final prediction stage.

4

A. System and Threat Model System Model. We consider an FL setting in which N parties collaboratively train a neural network model without sharing their local datasets. Once training is complete, the final model is hosted either by one of the N parties or an external entity, such as a cloud service provider. Throughout the process, all parties are responsible for safeguarding data privacy and preventing reconstruction attacks at every stage—communication, training, and prediction—where potential attackers could include an individual party, the server, or a collusion of multiple parties. Threat Model. We operate under an honest-but-curious setting with K parties, where our setup assumes the possibility of collusions involving up to K −1 parties, all of whom adhere to the protocol but may attempt to infer information from other parties’ data. Our primary goal is to prevent reconstruction and inference attacks during FL training, where the server or clients might exploit the model or gradients to infer information about the remaining parties’ data. However, attacks that exploit prediction outputs are considered out of the scope of this work, as our focus is not on the prediction phase. B. System Overview In HADES, we leverage PCA to identify and select the most privacy-sensitive features in the dataset. This approach is motivated by the fact that many raw data features exhibit high correlations, which can be exploited by attackers for reconstruction. Recent research has also shown that PCA can serve as an effective method for mitigating privacy threats [31]. While we use PCA in this work, it serves only as a demonstrative feature-selection mechanism. The feature-selection stage in HADES is fully modular, and any alternative algorithm can be plugged in without changing the rest of the system. The selected high-sensitivity features are processed within an encrypted environment, ensuring their confidentiality during computation. In contrast, the remaining, less-sensitive features are handled in plaintext, optimizing computational efficiency. To maintain both security and performance, we employ a fusion strategy that seamlessly integrates logits from both processing streams. These fused logits guide the learning on the encrypted branch while preserving end-to-end encryption of its weights. Separately, the plaintext branch uses its own loss, calculated from only plaintext logits, and requires no decryption of the HE parameters. Figure 1 provides an overview of HADES’s high-level workflow. We also summarize HADES’s federated training procedure in Algorithm 2, which begins with an initialization phase for feature selection for encryption (Lines 1–2). We detail this procedure in the next section (Section IV-C). Then, clients perform local training (Lines 511) followed by server-side aggregation (Lines 12-14), which are detailed in Section IV-D.

IV. P ROPOSED M ETHOD Here, we provide the details of our proposed framework. First, we define the system and threat model. Next, we outline our system overview and feature selection process. Finally, we explain our model training, including HE operations, in detail.

C. Feature Selection for Encryption Our proposed FL system, HADES, begins with feature selection (Algorithm 2, Lines 1–2). Let F = {f1 , . . . , fd } be the original feature set. We split the feature set into FHE , the

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021

5

Fig. 1: HADES’s system overview. The initialization phase involves selecting the clear-text and encrypted (privacy-sensitive) components of the model using PCA-based feature selection. Overall, HADES’s online training consists of local computations in an FL setting, where two models—one operating on clear-text data and the other on encrypted data are trained locally. Their updates are then aggregated on the server and subsequently fused into a single model (via fusion) during local post-processing. The encrypted network is optimized using the fused loss while keeping all HE parameters and logits encrypted, while the plaintext network is trained with its own plaintext loss, without requiring any decryption or access to the encrypted parameters.

subset whose parameters are handled by the HE encrypted model, and FP , the subset processed in plaintext. Consequently, |FHE | and |FP | denote the number of encrypted and plaintext features, respectively, throughout this paper. PCA is used to find principal components of length |F | that describe most of the variance. From F, we select FHE to be the top-i principal components. The selection of i is determined based on three factors: the proportion of variance explained, network configuration to accommodate ciphertext size, and the desired utility of the resulting model. As demonstrated through extensive experiments in Section V-B, our approach effectively mitigates reconstruction attacks, depending on the number of principal components selected. HE | The ratio |F |FP | depends on the dataset and task requirements: increasing |FHE | offers stronger privacy guarantees but incurs heavier computational overhead. Furthermore, the ring size N limits the amount of data that can be protected, as only 2N −1 elements fit per ciphertext slots (see Section III-C). Both the encrypted features and the model weights must stay within that capacity. This constraint, in turn, forces the layer sizes and intermediate operations to be chosen so every tensor remains representable in ciphertext space. As an example, a matrix multiplication of dimensions N × M and M × K produces an N × K output, but this operation requires N × M × K intermediate elements in HE multiplication. For homomorphic operations to remain valid, the condition N × M × K ≤ 2N −1

must hold, reflecting the ciphertext’s capacity limit. As a simple example, for batch size B on a single-layer network with a single output neuron, the effective capacity becomes 2N −1 . B To extract information about the training data, an adversary would need to approximate the inverse PCA transformation, which remains encrypted. Even if decryption were possible, (which is prevented in HADES through MHE), the adversary would still require the transformation parameters to reconstruct the original data, adding an extra layer of security. While this increases the complexity of an attack, our model does not rely solely on the inaccessibility of the PCA transformation, but primarily on the security guarantees of MHE encryption. Notably, PCA can also be used to guide feature selection without requiring the model to be trained using principal components as features. In this scenario, the absolute contributions of the top-k principal components are aggregated to rank the original features based on their relevance. This allows for informed feature selection while preserving the original feature space. However, we choose to use PCA-derived features, as they do not necessarily lead to performance degradation during training (see Section V-B). Finally, we note that privacy-preserving federated PCA computation is beyond the scope of this paper, as it has been well-addressed in the existing literature. Based on the literature |FHE | ≤

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021

Algorithm 2 HADES Training Algorithm. Initialization: 1: Parties agree on a secure PCA transformation. 2: Each client k applies the PCA transformation, splitting data into XHE and XP , with feature sets FHE and FP respectively. k 3: Initialize local models WHE and WPk . Training Loop: 4: for Each iteration t do Local Training (Client-side): 5: for Each client k in {1, . . . , K} do 6: Forward pass of XP through WPk,t , output logits zk,t P . k,t 7: Forward pass of XHE through WHE , output encrypted logits zk,t HE . k,t k,t 8: Compute fused logit z̄k,t HE = α zHE + (1 − α) HE(zP ). k,t k,t Set plaintext logit z̄P = zP . k,t k,t k 9: Compute objectives Lk,t = HE = ℓ(z̄HE , y ) and LP k,t k ℓ(z̄P , y ). k,t k,t 10: Calculate gradients ∇WHE w.r.t. Lk,t w.r.t. HE and ∇WP k,t LP .   k,t 11: Send ∇WHE , ∇WPk,t to server. 12: end for Aggregation (Server-side): PK k,t t ← 13: Aggregate encrypted updates: ∆WHE k=1 ∇WHE . P K k,t t 14: Aggregate plaintext updates: ∆WP ← k=1 ∇WP . t , ∆WPt back to clients. 15: Send aggregated ∆WHE Model Update (Client-side): 16: for Each client k in {1, . . . , K} do k,t+1 k,t t 17: WHE ← WHE − η ∆WHE k,t+1 k,t t 18: WP ← WP − η ∆WP . 19: end for 20: end for

for implementing PCA in a federated setting, we propose two possible approaches. First, each party can independently apply PCA on its local dataset to determine its private feature set, then leverage Private Set Intersection to establish a consensus on the selected features. Alternatively, clients can leverage privacy-preserving multi-party PCA solutions, such as PPPCA [35] or SF-PCA [36], to securely compute principal components in a collaborative manner while maintaining privacy. D. Model Training We propose a fusion-based approach that divides model parameters into homomorphically encrypted (WHE ) and plaintext (WP ) sub-networks. After agreeing on the selected feature set for encryption FHE (see Section IV-C), each client initialk izes local models WHE and WPk with input dimensions |FHE | and |FP | respectively (Algorithm 2 Line 3). Each client k applies forward and backward passes on their k local data on both WHE and WPk (Lines 6-7). Weights and k gradients for WHE are never decrypted during forward and backward passes. For the HE operations, we require efficient packing and multiplication strategies to replace their vanilla counterparts in plaintext training. Details on how we define and use HE operations in forward and backward passes, packing, and bootstrapping are presented in Supplementary Material B. After the forward pass, each network generates partial predictions based on its respective feature set. From the encrypted and plaintext logits, we form two objectives: (i) an encrypted

6

and fused objective used to train the encrypted sub-network, and (ii) an independent plaintext objective used to train the plaintext sub-network. Specifically, let zHE (xi ; WHE ) and zP (xi ; WP ) denote the encrypted and plaintext logits, respectively. We define the fused HE logit as a weighted ciphertext– plaintext combination (without decrypting zHE ), while the plaintext logit is computed solely from zP : z̄HE = α zHE + (1 − α) HE(zP ) z̄P = zP .

(1) (2)

We use the residual objective ℓ(ẑ, y) := ẑ − y, yielding
LHE = ℓ z̄HE , yi = z̄HE (xi ) − HE(yi ), (3)
LP = ℓ z̄P , yi = z̄P (xi ) − yi . (4) where α ∈ [0, 1] controls the relative contribution of each sub-network to the fused objective (Line 8). While we do not explore this within the scope of this study, intermediate fusion can be used for additional flexibility: the plaintext hidden features hP are encrypted on the fly and then concatenated (or used in an attention mechanism) with hHE . Notably, at no point are the homomorphic parameters decrypted. Both the parameters and their gradients remain encrypted throughout the entire training process. The only unencrypted components of the network are the intentionally unencrypted model parameters. Both encrypted and plaintext model losses are retained locally by each client and are never shared with the server. We use the standard communication protocol used in FL, where, in iteration t, client k computes local updates (Line 10) k,t ∇WHE , ∇WPk,t , and sends them to the server (Line 11). The server aggregates these two parameters separately (Lines 1314), then broadcasts the updates back to clients (Line 15). This structure ensures that the encrypted model part and gradients are always secure during local training, communication, and server aggregation. Each client then applies the aggregated model parameters and proceeds to the next round (Lines 1619). Below, we detail the ciphertext operations required to enable training on the HE-encrypted network WHE . 1) Approximation of activation functions: In HE, nonpolynomial activation functions such as the sigmoid cannot be directly evaluated on encrypted data. To address this, we approximate them with low-degree polynomials obtained via least-squares fitting over a chosen input interval. Concretely, we sample the target function at a set of points within the approximation range and solve for polynomial coefficients that minimize the squared error between the polynomial and the target values. At runtime, the activation is evaluated on the encrypted layer output using the pre-computed coefficients via standard polynomial evaluation. During the backward pass, we use the derivative of the fitted polynomial. Alternative approximation schemes can be seamlessly integrated into HADES; however, the choice and evaluation of approximation methods is beyond the scope of this work. V. EXPERIMENTAL EVALUATION In this section, we experimentally evaluate HADES to address the following key questions:

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021

Q1: Does PCA-based feature selection and encryption effectively protect against data reconstruction attacks in FL settings? • Q2: Does HADES preserve the model utility? • Q3: How does HADES compare against a baseline that trains solely on the PCA-selected feature subset, without employing dual network fusion? • Q4: How does HADES scale with the number of encrypted parameters (|FHE | ), the number of clients in the FL, and the model complexity? • Q5: What is the overall runtime performance of HADES? Finally, we note that comparing HADES to state-of-the-art encrypted FL frameworks is inherently challenging for several reasons: (i) The private FL literature generally falls into two categories: fully encrypted training and secure aggregation. HADES introduces a hybrid approach that applies model encryption and secure aggregation to only a subset of the parameter space, while still offering privacy guarantees for the unencrypted parameters. To the best of our knowledge, no prior work adopts this selective encryption strategy, leaving no directly comparable framework. (ii) The most closely related works—fully encrypted FL frameworks such as POSEIDON [17] and Hercules [19]—encrypt all model parameters. As a result, under identical settings and packing schemes, HADES is expected to reduce runtime in proportion to the number of parameters it encrypts (|FHE |) Therefore, our scalability plots (Figure 4) effectively illustrate this comparative advantage. (iii) HADES implements HE operations using the Python wrapper for OpenFHE [37], whereas POSEIDON and Hercules are built on the Lattigo library. These frameworks prioritize HE-specific optimizations, while HADES focuses on enabling efficient selective encryption. In the following subsections, we first describe our experimental setup (Section V-A), detailing the datasets, parameters, and implementation details. Then, each subsequent subsection systematically addresses the key questions outlined. •

A. Experimental Setup Datasets. For our experiments, we use the Breast Cancer Wisconsin (Diagnostic) dataset [38], containing 569 samples with 30 real-valued features per sample; the MNIST dataset [39], consisting of 70,000 grayscale images of handwritten digits, each with 28×28 pixels (totaling 784 features per image); and the SVHN dataset [40], which contains 99,289 32x32 RGB digit images (3072 features per image) cropped from real-world Street View scenes. We use a 70%-30% train– test split for all datasets (SVHN uses its predefined 73.78%26.21% split). Implementation. We implement our plaintext network and its operations from scratch using NumPy [41], without relying on any existing machine learning frameworks. We opt for this to eliminate any computation changes that can happen due to internal library operations, and keep encrypted and plaintext network results close to each other for the same input (See Section V-C). Network weights are initialized with Xavier initialization, and bootstrapping is applied to the weights (of

7

the encrypted sub-network) at the end of each weight update. Our loss function is implemented in a way that works with one-hot-encoded labels. This enables the generation of losses from multilabel data in an efficient manner as in [17]. Our in-house HE operation framework is built as an extension for the OpenFHE-python library [37]. The framework includes efficient yet flexible alternating packing schemes [17] for matrices, convenient handling of padding, masking, and rotation of matrices, allowing mini-batch operations on multi-layer neural networks. Each ciphertext optionally holds information regarding its valid indices (i.e., non-zero values that are not ciphertext operation byproducts) and underlying padded representation to ease debugging. We evaluate the privacy of HADES using the success rate of the Improved Deep Leakage Gradient(iDLG) [21] attack. Following our previous argument regarding network implementations from scratch, we implemented iDLG using NumPy [41] and L-BFGS [42] implementation from SciPy [43]. Since the gradients from the encrypted network cannot be recovered, all iDLG attack results come from the unencrypted plaintext network. We then evaluate the reconstruction rate of the recovered features using Root Mean Square Error (RMSE), Peak Signal-to-Noise Ratio (PSNR), Structural Similarity Index (SSIM) [44] and Learned Perceptual Image Patch Similarity (LPIPS) [45]. System configuration. Each experiment model is trained for 10 epochs using Stochastic Gradient Descent (SGD) with a local batch size B = 1 (i.e. a global batch size Bg = K for FL settings) and number of clients K = 10, unless stated otherwise. A single-element batch size is chosen because it maximizes the success rate of the iDLG attack, providing a rigorous evaluation of HADES under challenging conditions. Furthermore, it allows us to more clearly see the limitations of not using HADES sub-network strategy, since such a network would only support a very small mini-batch, and that is if the network dimensions are kept smaller. In Section V-F, we show how HADES seamlessly allows for fitting larger mini-batches in a single ciphertext. In decentralized settings, we define an epoch as one complete pass through the entire dataset. In global iterations multiparty setups, each client processes n|D| k ·B (GI) to go through their entire data, where |D| is the total sample size of the dataset, and nk is the number of samples for client k. The network configurations change based on different experiments, and they are detailed in their respective sections. In each experiment, both plaintext and encrypted sub-networks use the same number of layers and layer dimensions. The score-level fusion trade-off parameter α is set to 0.5, ensuring equal contribution from both sub-networks during training. The cyclotomic ring size for CKKS is set to N = 213 , yielding a ciphertext vector in CN /2 with 212 slots. We use a 64-bit precision and set the number of levels to L = 6. B. Q1: Does PCA-based feature selection and encryption effectively protect against data reconstruction in FL settings? To evaluate the effectiveness of our feature selection approach in preserving privacy, we conducted preliminary experiments using the Improved Deep Leakage from Gradients

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021

8

TABLE I: Cumulative variance explained (%) by PCA across datasets at different feature selection thresholds, alongside test accuracy (%) for Baseline non-fusion network with only encrypted features and HADES. Missing values (-) indicate cases where the feature count exceeds the maximum available features for that dataset. Feature Count 1 2 4 8 16 32 64 128 256 512 1024 2048

BCD

MNIST

SVHN

Var. (%)

Baseline Acc. (%)

HADES Acc. (%)

Var. (%)

Baseline Acc. (%)

HADES Acc. (%)

Var. (%)

Baseline Acc. (%)

HADES Acc. (%)

98.14 99.79 99.99 100.00 100.00 -

94.74 94.74 95.91 98.25 95.32 -

93.57 92.98 97.08 97.66 95.32 -

9.76 16.92 28.48 43.87 59.58 74.49 86.28 93.68 97.94 99.94 -

20.68 41.93 60.70 84.15 91.39 93.44 94.60 94.52 94.43 94.46 -

90.44 91.14 89.56 90.67 92.64 93.82 94.54 94.64 94.73 94.67 -

57.91 63.62 72.96 79.95 86.41 91.60 96.00 98.45 99.47 99.86 99.98 100.00

19.59 19.59 24.89 30.97 57.00 67.23 70.61 70.66 70.25 70.44 69.91 69.70

66.06 66.68 60.78 54.15 59.75 69.23 72.01 72.16 71.45 71.40 71.67 72.12

(iDLG) [21] attack framework against multiple datasets. In iDLG, the recovery process begins by initializing a random input, X, and iteratively updating it to minimize the difference between the gradients of the hypothetical input and the leaked gradients. This optimization minimizes the gradient mismatch, defined as ∇L = ∥∇X − ∇WPk,t ∥2 , where ∇WPk,t represents the gradients of non-encrypted features shared during training. By iteratively refining X, we ensure alignment between the hypothetical and leaked gradients. Our proposed defense against such gradient-based attacks relies on selectively encrypting a subset of the feature space. We select these features based on the cumulative explained variance as calculated by PCA. Table I reports, for each dataset, how much variance is retained as the number of preserved principal components increases, and pairs this with test accuracy for (i) a Baseline model trained using only the encrypted PCA features and (ii) HADES, which additionally leverages the remaining (unencrypted) features through fusion. As expected, a significant portion of the variance can be explained by a relatively small subset of the original feature space. This is particularly evident in the BCD dataset, and we argue that this characteristic is a primary reason why the performances of the baseline and HADES are both similar and erratic. However, explained variance alone does not guarantee predictive performance: for MNIST and SVHN, utilizing very few encrypted components yields poor baseline accuracy despite non-trivial retained variance, whereas HADES maintains substantially higher accuracy by exploiting complementary information from the non-encrypted features. Encrypting only these principal directions preserves most of the privacy-sensitive signals while sharply reducing the plaintext surface available to an attacker. Thanks to the HADES fusion approach, the remaining features are also utilized for training, ensuring their information is not lost. Table II reports reconstruction quality under the randominitialization gradient inversion attack as a function of the number of encrypted PCA features, |FHE |. Reconstruction fidelity is summarized using RMSE (lower is better), PSNR in dB (higher is better), SSIM in [0, 1] (higher is better), and LPIPS (lower is better). As expected, when |FHE | = 0 (no encryption; HADES disabled), the attacker achieves perfect reconstruction across datasets (RMSE = 0, PSNR = ∞, SSIM = 1, LPIPS = 0). Increasing |FHE | steadily degrades

reconstruction: on MNIST, moving from 16 to 256 encrypted features reduces SSIM from 0.48 to 0.11 and increases LPIPS from 0.17 to 0.49, while PSNR drops from 13.48 dB to 8.92 dB; on SVHN, SSIM falls from 0.53 (at 16) to 0.04 (at 256) and LPIPS rises from 0.16 to 0.53, indicating that class-defining structure is largely lost. Importantly, a moderate encryption budget already approaches the dataset-specific worst case |FHE | = |F | − 1: for SVHN, SSIM at 256 is 0.04 versus 0.01 at |F | − 1, and for MNIST it is 0.11 versus 0.04, suggesting diminishing returns beyond this point. Overall, these results support our claim that selectively encrypting a modest subset of features is sufficient to substantially suppress reconstruction quality, approaching the |F | − 1 regime without encrypting nearly all features. TABLE II: Reconstruction Quality Metrics (RMSE, PSNR, SSIM, LPIPS) by Dataset and |FHE |. Each cell shows values for the random-initialization attack. |FHE | controls the number of features selected via PCA to be encrypted. |FHE | = 0 is the perfect reconstruction case where HADES is not used. |FHE | = |F| − 1 denotes the case where only a single feature can be used for reconstruction, which sets the worst case values of the metrics for a given dataset. Dataset

|FHE |

Metric 0

16

64

256

1024

|F | − 1

BCD

RMSE PSNR (dB) SSIM LPIPS

0.00 ∞ 1.00 N/A

92.99 13.51 0.86 N/A

– – – –

– – – –

– – – –

98.86 8.60 0.65 N/A

MNIST

RMSE PSNR (dB) SSIM LPIPS

0.00 ∞ 1.00 0.00

0.22 13.48 0.48 0.17

0.29 11.09 0.26 0.29

0.42 8.92 0.11 0.49

– – – –

0.63 5.97 0.04 0.64

SVHN

RMSE PSNR (dB) SSIM LPIPS

0.00 ∞ 1.00 0.00

0.17 16.21 0.53 0.16

0.20 14.29 0.22 0.26

0.26 11.96 0.04 0.53

0.39 8.97 0.01 0.83

0.62 7.31 0.01 0.87

To complement the quantitative metrics in Table II, Figure 2 provides qualitative reconstructions produced by the same attack. Following our previous argument, we pick |FHE | = 256 to present our reconstruction examples, which already approach the near-worst-case reconstruction on MNIST and SVHN. Across different inputs, the reconstructed images exhibit no perceptible similarity to the original content. This collapse toward similar reconstructions is consistent with

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021

the substantial loss of discriminative information when the selected features are encrypted: the attacker is left to fit secondary features with virtually no identifying information.

9

aggregation over a larger batch, whereas the federated (FL) setting exposes per-client gradients computed from a single input sample, making the updates significantly noisier. We intentionally adopt this regime because gradient inversion attacks such as iDLG are most favorable when an iteration’s gradients are produced by a single input (i.e., B = 1), yielding the easiest reconstruction condition for the attacker. TABLE III: Test accuracies (%) for centralized (CE) and federated (FL) training across different configurations of FHE . FHE = 0 means all data is processed with a fully plaintext model. Bold FHE denotes that a model portion is trained under encryption, and we set |FHE | = [16, 256, 256] for BCD, MNIST, SVHN respectively.

(a) MNIST

(b) SVHN

Fig. 2: Qualitative iDLG reconstructions for the fusion model with |FHE | = 256 on MNIST (a) and SVHN (b). These findings highlight the practical effectiveness of our feature selection mechanism. By strategically selecting and protecting features, significant privacy improvements can be achieved under realistic attack conditions, effectively reducing the risk of data leakage without compromising the utility of FL systems (see Section V-C). Moreover, the observed disparities in attack efficacy between theoretical and practical scenarios highlight the critical role of adversary knowledge and initialization conditions in data reconstruction attacks. C. Q2: Does HADES preserve the model utility? For all three datasets, we conduct 4 experiments and report their test accuracies in Table III. These tests collectively evaluate and compare the results of (a) centralized (CE) vs federated learning (FL) training setup, (b) using a vanilla or approximated sigmoid (approx) activation function, and (c) using a single network (FHE = ∅) or the proposed fusion network for a given FHE . For experimental purposes, we simulate the encryption process using cleartext in these experiments. Due to the difference in feature sizes and numeric instability in certain cases, these experiments are run using a certain hyperparameter configuration for each dataset. We chose these hyperparameters via brief manual tuning on the training data. Specifically, for BCD we use a hidden layer of 16 units, an encrypted-feature packing factor |FHE | = 16. For MNIST, we increase the hidden layer to 64 units and |FHE | to 256. For SVHN, we use two hidden layers with 64 units each and keep |FHE | at 256. A learning rate of 0.1 with 0.9 nesterov momentum was used to train 10 federated clients for 10 epochs for all datasets. We do not optimize these parameters across datasets or configurations to better show the consistent dynamics of HADES across datasets. These |FHE | values account for respectively 2x, 4x, and 16x reduction in encrypted parameters compared to their non-HADES counterparts. Importantly, we use local batch size B = 1 and global batch size Bg = 10 in this experiment. As a result, the centralized (CE) configurations benefit from more stable updates due to

Dataset

CE- FHE = 0

CE- FHE

FL- FHE

FL- FHE approx. (HADES)

BCD MNIST SVHN

94.71 95.00 63.1

95.88 94.80 66.2

96.49 94.82 67.3

97.08 94.99 70.4

Across all settings and datasets, performance differences remain modest, indicating that our approximation and fusion schemes preserve predictive utility. The sigmoid approximation closely matches the results obtained with exact activations, and the fusion network does not introduce a meaningful degradation. Notably, HADES achieves the strongest gains on SVHN, improving accuracy from 63.1% (CE-FHE=0) to 70.4%, which suggests that the hybrid design can recover substantial task-relevant signal when the input space is highdimensional and heterogeneous. While CE-F HE and FLFHE include more parameters due to the fusion setup, their encrypted-feature selection is random; thus, they serve as a conservative baseline for the fusion architecture. In contrast, HADES benefits from a guided split that places salient components in the encrypted branch and leaves complementary information to the plaintext branch, enabling the fused predictor to better exploit both protected and residual structure. D. Q3: How does HADES compare against a baseline that trains solely on the PCA-selected feature subset, without employing dual network fusion? Given our selective-encryption design, a natural question is whether, once PCA identifies the most informative or privacysensitive features, one could train a single network using only this selected subset and discard the remaining features. To isolate the effect of PCA-selected features from the benefits introduced by HADES’s hybrid architecture, we include a single-network ablation in which only the PCA-selected components are used to train the model. This ablation removes the fusion mechanism, allowing a direct comparison of a baseline architecture against HADES’s encrypted model using the same feature subset. Both architectures are agnostic to the choice of activation approximation. To avoid confounding effects from approximation quality, we simulate this ablation with non-approximated activations for both models. This is also the fairest comparison point, since tuning approximation degrees or ranges per model can grant an advantage to one architecture over the other.

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021

10

BCD

96 95 94

SVHN

90

70

80

60

Test Accuracy (%)

97

Test Accuracy (%)

Test Accuracy (%)

MNIST

Baseline Hades

98

70 60 50 40 30

93 1

2

4

8

Number of Encrypted Features (FHE)

16

20

Baseline Hades

1

2

4

8

16

32

64 128 256 512

Number of Encrypted Features (FHE)

50 40 30 Baseline Hades

20 1

2

4

8

16 32 64 128 256 512 10242048

Number of Encrypted Features (FHE)

Fig. 3: Test accuracy (%) versus the number of encrypted PCA-selected features (|FHE |) for BCD, MNIST, and SVHN. Baseline trains a single network using only the selected subspace, while HADES applies score-level fusion between an encrypted branch on FHE and a plaintext branch. As shown in Table I and Figure 3, training solely on the PCA-derived subspace leads to a clear accuracy drop across MNIST and SVHN, indicating that selecting only the “most important” components is often insufficient and discards taskrelevant information. The BCD results are less consistent because even a single principal component captures the vast majority of the variance, reflecting the limited and highly specific feature set of this dataset. As a result, PCA-based feature pruning is comparatively less informative, and relying on it alone leads to a reduction in model utility. In contrast, HADES recovers utility through score-level fusion: the plaintext branch preserves fine-grained patterns that the selected components omit, while the encrypted branch protects the most sensitive components. This ablation highlights that the gains in accuracy stem from the fused dual-network design rather than from PCA alone. Across datasets, accuracy generally improves with larger |FHE |, and HADES is consistently more robust at small encrypted-feature budgets, with both methods converging as more features are included. E. Q4: How does HADES scale with the number of encrypted parameters (|FHE | ), the number of clients in the FL, and the model complexity? The experimental results in Figure 4 report the training time using a synthesized dataset. To account for variability, each experiment was repeated 10 times across 10 distinct data points, and the results were averaged. A synthesized dataset with 32 features was used in this experiment, where |FHE | = 32 denotes training under full encryption, that is, with all features treated as private. These results allow for an analysis of how HADES scales with respect to the number of homomorphically encrypted features (|FHE |) and the complexity of the encrypted sub-network (specifically, its hidden layer size). A primary observation, which was the core motivation for HADES, is the direct correlation between the number of encrypted features and the computational overhead. As |FHE | decreases, training time also decreases. Based on network configuration, this improvement can be up to 28%. The complexity of the neural network architecture operating on encrypted data also significantly impacts performance. With |FHE | fixed at certain values, the introduction of a hidden layer into the encrypted sub-network, and the subsequent increase in its size

leads to a substantial rise in training times. Wider hidden layers consistently incur a higher absolute cost, but the incremental penalty of doubling FHE remains comparable, with each step adding on the order of a few hundred milliseconds. The ”No Hidden Layer” baseline is substantially faster overall but follows the same upward trajectory, indicating that ciphertext packing, not hidden-layer computation, dominates runtime at large FHE . Collectively, the plot demonstrates that training time scales approximately linearly with the number of encrypted features. For deeper networks, we observe diminishing returns from tuning |FHE |. Even then, we still maintain ∼ 7% decrease in runtime in worst-case configurations. This trend is also consistently observed in an increasing number of clients. This is expected, as the improvements introduced by HADES primarily stem from local training steps. The overall FL procedure remains unchanged, aside from the negligible communication overhead introduced by transferring unencrypted gradients. We also provide the number of costly operations per forward and backward pass of the network in Table IV. Each additional encrypted layer introduces a fixed computational overhead: 4 extra rotations, 1 plaintext–ciphertext multiplication (MultPT), and 3 ciphertext–ciphertext multiplications (MultCT). Since rotations dominate latency calculations, overall compute grows based on the number of rotations. As previously argued, the fusion point of the network can be in any intermediate layer. This decision should be given based on the computational considerations presented here, and an additional security analysis conducted for that specific network architecture. TABLE IV: Operation counts for forward and backward passes for an n layer network, per client. MultPT

MultCT

Forward Count 2n − 1

Rotation

2n

n

Backward Count 2n − 1

2n + 1

2n + 1

Total Count

4n + 1

3n + 1

4n − 2

In summary, the scalability of HADES is influenced by the number of sensitive features (FHE ) and the architectural complexity of the encrypted sub-network. The system exhibits

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021

11

Training Time vs Number of Encrypted Features for Different Hidden Layer Sizes

Training Time vs Number of Encrypted Features for Different Network Architectures No Hidden Layer 1 Hidden Layer 2 Hidden Layers 3 Hidden Layers 4 Hidden Layers

5

30 Hidden Layer Size = 2 Hidden Layer Size = 4 Hidden Layer Size = 8 Hidden Layer Size = 16 Hidden Layer Size = 32 No Hidden Layer

4 3 2

Training Time (seconds)

Training Time (seconds)

35

25 20 15 10 5

2

4

8

16

32

Number of Encrypted Features (FHE)

(a) Width scaling for a single (or no) hidden layer.

2

4

8

16

Number of Encrypted Features (FHE)

32

(b) Depth scaling for a single hidden layer.

Fig. 4: Training time as a function of the encrypted-feature set (FHE ) for models with (a) 0, 2, 4, 8, 16, and 32 hidden units and (b) increasing number of layers with 16 hidden units each. Runtime increases linearly with FHE for every depth: wider networks start with higher runtimes but scale at a similar rate, whereas the no-hidden-layer configuration is consistently the fastest with the same upward trend. proficient scaling with an increasing number of clients in terms of per-client, per-data point computational efficiency. While more complex HE networks inherently increase the baseline computational cost, the scaling trend with an increased number of clients remains largely positive or neutral, showing the viability of HADES in distributed FL environments. TABLE V: One-GI and Total Training timing analysis for BCD, MNIST, and SVHN under different batch sizes and encrypted feature set sizes (|FHE |). GI-Load denotes the number of global iterations (GI) required to process a fixed amount of data (baseline X). One-GI is the wall-clock time for a single global iteration, and Total Training is the per-client end-toend training time. For comparability, we run configurations with GI-Load = 16X for 100 iterations and scale the iteration count linearly for smaller loads as IkX = 100 · k/16 (e.g., X = 100/16). For BCD, we use one hidden layer with 64 units; for MNIST and SVHN, we use no hidden layers. |F| is 30, 784 and 3072 for BCD, MNIST and SVHN, respectively. Dataset

BCD

MNIST

SVHN

Batch Size

|FHE |

GI-Load

One-GI (s)

Total Training (s)

1 2 4 4 8 8 16 16 1 1 1 2 2 4 1 1 2 4

30 30 16 8 8 4 4 2 256 128 64 128 64 64 256 128 128 64

16X 8X 4X 4X 2X 2X X X 16X 16X 16X 8X 8X 4X 16X 16X 8X 4X

0.519 0.593 0.547 0.530 0.534 0.502 0.543 0.511 0.313 0.262 0.240 0.301 0.267 0.294 0.305 0.264 0.297 0.277

202.5 115.6 53.4 51.7 26.0 24.5 13.2 12.5 15337.5 12816.7 11738.9 7381.4 6543.6 3602.8 22332.2 19325.8 10874.4 5071.9

F. Q5: What is overall the runtime performance of HADES? In this section, we report runtime measurements across multiple datasets and configurations. We introduce GI-Load as the number of global iterations (GI) required by a configuration to process a fixed number of samples. In our setup, GI-Load

scales approximately inversely with the batch size B: for example, a configuration with B = 8 processes 80 samples in 10 iterations, whereas B = 16 processes the same 80 samples in 5 iterations, resulting in half the GI-Load. Table V shows that for each dataset, increasing B incurs a slight cost on OneGI time. This is due to the additional rotation operation needed in each forward and backward pass to process the mini-batch. However, this added latency is amortized by the reduction in GI-Load, which reduces the number of bootstraps, rotations, and communication rounds over the full training run. For comparability across settings, we fix the iteration count to 100 when GI-Load = 16X and scale it linearly for smaller loads as IkX = 100 · k/16. Finally, for a fixed GI-Load (i.e., fixed B), One-GI is primarily influenced by |FHE |, which changes ciphertext utilization and the per-iteration homomorphic cost. When using a batch size of B = 1, application-level P parallelization combined with local gradient accumulation can be employed to effectively simulate a batch size of B = P . Such an approach would still preserve the privacy guarantees of this study and reduce One-GI by two means: hardware parallelism and fewer homomorphic operations for singlecipher mini-batches. However, our focus on this study is to show how HADES can increase single-cipher utilization without relying on hardware-level parallelization. VI. D ISCUSSION An important design consideration in HADES is minimizing the number of slots consumed by the feature dimension, as this enables more efficient use of the ciphertext space for operations such as expanding hidden layer dimensions or increasing mini-batch size. To better demonstrate this emergent property, we adopt a packing strategy that encodes the entire mini-batch into a single ciphertext. This contrasts with approaches like POSEIDON [17] and Hercules [19], which either parallelize over individual examples within the mini-batch or process them sequentially before averaging. By leveraging a single-ciphertext strategy, HADES significantly reduces memory usage and CPU core requirements during inference and training. The parallel approach allows for further optimizations, such as omitting rotations in the forward and

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021

backward pass of the first layer using duplication of the input data, while packing the mini-batch in the same ciphertext requires two additional rotation calls. A key advantage of HADES is that its value proposition follows naturally from the cost structure of SIMD HE. Reducing the encrypted feature dimension decreases ciphertext slot consumption, which typically lowers the cost of downstream encrypted linear algebra (e.g., encrypted matrix–vector products and rotations) and therefore reduces end-to-end training time. On the other hand, the plaintext support network provides extra representational capacity ”for no cost” in HE terms: its computation is inexpensive relative to ciphertext arithmetic, yet it increases the overall number of parameters and thus capacity, utility, and flexibility. Our empirical evaluation utilizes HADES with fully connected networks. However, HADES is not tied to a specific neural architecture: it is a protocol-level design that composes two subnetworks (an encrypted branch operating on FHE and a plaintext branch operating on F \ FHE ) and combines their predictions (or intermediate representations) via a fusion operator. Consequently, HADES can be integrated with any neural network backend as long as it supports a fusion point. Thus, HADES is complementary to existing encrypted FL methods rather than a replacement. HADES specifies what portion of the feature space is encrypted and how encrypted and plaintext computations are combined. HADES introduces several mechanisms to balance privacy and utility that can be tuned based on circumstances. Our selective encryption scheme depends on a feature selection algorithm. This algorithm can be improved by future work to provide more security even with a smaller subset of encrypted features. Moreover, based on the privacy needs and the domain, fewer features can be encrypted to improve the performance significantly. The choice of layer for fusion allows for tuning the trade-off between accuracy, training time and privacy. In this study, we opt for score-level fusion with equal contribution from both plaintext and encrypted models (α = 0.5). Weighted averaging parameter α can be tuned based on domain if, say, most private features are also more important for model performance or vice versa. Additionally, intermediate fusion techniques, such as concatenation or addition of hidden layer representations, offer further flexibility in optimizing the model’s performance, while reducing the performance overhead. By employing selective HE, we optimize communication efficiency while preserving privacy. The bulk of the communication overhead is from encrypted gradients. Significantly less traffic is created as |FHE | is tuned to be lower. Also, the smaller GI-Load of models under HADES results in fewer communication rounds, further reducing overall communication overhead. We adopt PCA as a proof-of-concept feature selection method due to its computational efficiency, data-agnostic nature, and ease of reproducibility; importantly, HADES is fully agnostic to the choice of feature selection technique. We note that PCA’s variance-maximization objective does not inherently provide privacy guarantees. Despite this, our results (Table II and Figure 2) demonstrate that encrypting a small amount of principal components is sufficient to drastically

12

reduce reconstruction success, while preserving model accuracy (Section V-C). Exploring more targeted feature selection strategies—such as privacy-aware information ranking, adversarial filtering, or task-specific neural saliency—remains a promising avenue for future work, particularly for enhancing protection of privacy-sensitive attributes. This is especially pertinent for commonly used datasets like MNIST, where a small set of highly salient features dominate classification, leaving much of the remaining feature space underexplored in terms of privacy impact. For our privacy-related analyses and experiments (see Section V-C), we adopt the strongest known attack setting by using a batch size of B = 1, where the adversary has access to gradients from individual samples. In contrast, when B > 1, the attacker only observes an average gradient across multiple inputs and must disentangle them to recover specific examples—a significantly more challenging task. Notably, our results reveal that reducing the feature space enables the use of larger batch sizes within a single ciphertext. This emergent property not only improves computational efficiency but also enhances security, indicating that HADES becomes increasingly robust in practical deployments. VII. C ONCLUSION In this paper, we introduced HADES, a novel privacypreserving FL framework that selectively encrypts only the most privacy-sensitive features while processing the remaining data in plaintext. In contrast to secure aggregation techniques, HADES preserves encryption over the protected network components for the entire duration of training. By leveraging PCA for feature selection and integrating encrypted and plaintext sub-models through a fusion mechanism, HADES strikes an effective balance between privacy and computational efficiency. Our results demonstrate that HADES significantly mitigates reconstruction attacks while maintaining model accuracy comparable to standard FL. Moreover, the selective encryption approach reduces the computational and communication overhead associated with fully encrypted training pipelines. To the best of our knowledge, HADES is the first system to perform hybrid model fusion between encrypted and plaintext components in FL, offering a practical solution for privacy-sensitive collaborative learning tasks. R EFERENCES [1] J. Konečnỳ, H. B. McMahan, D. Ramage, and P. Richtárik, “Federated optimization: Distributed machine learning for on-device intelligence,” CoRR, vol. abs:1610.02527, 2016. [2] H. B. McMahan, E. Moore, D. Ramage, and B. A. y Arcas, “Federated learning of deep networks using model averaging,” CoRR, vol. abs/1602.05629, 2016. [Online]. Available: http://arxiv.org/abs/ 1602.05629 [3] E. Bagdasaryan, A. Veit, Y. Hua, D. Estrin, and V. Shmatikov, “How to backdoor federated learning,” CoRR, vol. abs/1807.00459, 2018. [Online]. Available: http://arxiv.org/abs/1807.00459 [4] B. Hitaj, G. Ateniese, and F. Perez-Cruz, “Deep models under the GAN: Information leakage from collaborative deep learning,” in ACM CCS, 2017. [5] Z. Wang, M. Song, Z. Zhang, Y. Song, Q. Wang, and H. Qi, “Beyond inferring class representatives: User-level privacy leakage from federated learning,” in IEEE INFOCOM 2019 - IEEE Conference on Computer Communications, 2019, pp. 2512–2520.

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021

[6] L. Melis, C. Song, E. De Cristofaro, and V. Shmatikov, “Exploiting unintended feature leakage in collaborative learning,” in 2019 IEEE Symposium on Security and Privacy (SP), 2019, pp. 691–706. [7] M. Nasr, R. Shokri, and A. Houmansadr, “Comprehensive privacy analysis of deep learning: Passive and active white-box inference attacks against centralized and federated learning,” in IEEE S&P, 2019. [8] J. Zhang, J. Zhang, J. Chen, and S. Yu, “Gan enhanced membership inference: A passive local attack in federated learning,” in IEEE International Conference on Communications (ICC). IEEE, 2020, pp. 1–6. [9] R. Shokri and V. Shmatikov, “Privacy-preserving deep learning,” in ACM Conference on Computer and Communications Security (CCS), 2015. [10] H. B. McMahan, D. Ramage, K. Talwar, and L. Zhang, “Learning differentially private recurrent language models,” in International Conference on Learning Representations, 2018. [Online]. Available: https://openreview.net/forum?id=BJ0hF1Z0b [11] S. Truex, L. Liu, K.-H. Chow, M. E. Gursoy, and W. Wei, “Ldp-fed: Federated learning with local differential privacy,” in Proceedings of the third ACM international workshop on edge systems, analytics and networking, 2020, pp. 61–66. [12] Z. You, X. Dong, S. Li, S. Ma, and Y. Shen, “Local differential privacy is not enough: A sample reconstruction attack against federated learning with local differential privacy,” IEEE Transactions on Information Forensics and Security, 2024. [13] M. Mansouri, M. Önen, W. B. Jaballah, and M. Conti, “Sok: Secure aggregation based on cryptographic schemes for federated learning,” PoPETs, 2023. [14] E. Hosseini and A. Khisti, “Secure aggregation in federated learning via multiparty homomorphic encryption,” in 2021 IEEE Globecom Workshops (GC Wkshps). IEEE, 2021, pp. 1–6. [15] C. Zhang, S. Li, J. Xia, W. Wang, F. Yan, and Y. Liu, “{BatchCrypt}: Efficient homomorphic encryption for {Cross-Silo} federated learning,” in 2020 USENIX annual technical conference (ATC 20), 2020. [16] K. Bonawitz, V. Ivanov, B. Kreuter, A. Marcedone, H. B. McMahan, S. Patel, D. Ramage, A. Segal, and K. Seth, “Practical secure aggregation for privacy-preserving machine learning,” in Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, ser. CCS ’17. New York, NY, USA: Association for Computing Machinery, 2017, p. 1175–1191. [Online]. Available: https://doi.org/10.1145/3133956.3133982 [17] S. Sav, A. Pyrgelis, J. R. Troncoso-Pastoriza, D. Froelicher, J.-P. Bossuat, J. S. Sousa, and J.-P. Hubaux, “Poseidon: Privacy-preserving federated neural network learning,” in NDSS, 2021. [18] S. Sav, A. Diaa, A. Pyrgelis, J.-P. Bossuat, and J.-P. Hubaux, “Privacypreserving federated recurrent neural networks,” PoPETs, vol. 2023, pp. 500–521. [Online]. Available: https://api.semanticscholar.org/CorpusID: 251135050 [19] G. Xu, X. Han, S. Xu, T. Zhang, H. Li, X. Huang, and R. H. Deng, “Hercules: Boosting the performance of privacy-preserving federated learning,” IEEE Transactions on Dependable and Secure Computing, vol. 20, no. 5, pp. 4418–4433, 2023. [20] H. Tian, C. Zeng, Z. Ren, D. Chai, J. Zhang, K. Chen, and Q. Yang, “Sphinx: Enabling privacy-preserving online learning over the cloud,” in IEEE Symposium on Security and Privacy (SP), 2022, pp. 2487–2501. [21] B. Zhao, K. R. Mopuri, and H. Bilen, “idlg: Improved deep leakage from gradients,” 2020. [Online]. Available: https://arxiv.org/abs/2001.02610 [22] J. Park and H. Lim, “Privacy-preserving federated learning using homomorphic encryption,” Applied Sciences, vol. 12, no. 2, p. 734, 2022. [23] E. Gronberg, L. d’Aliberti, M. Saebo, and A. Hook, “Blindfl: Segmented federated learning with fully homomorphic encryption,” arXiv preprint arXiv:2501.11659, 2025. [24] Y. Lu, Z. Yu, and N. Suri, “Privacy-preserving decentralized federated learning over time-varying communication graph,” ACM Trans. Priv. Secur., vol. 26, no. 3, Jun. 2023. [Online]. Available: https://doi.org/10.1145/3591354 [25] C. Hu and B. Li, “Maskcrypt: Federated learning with selective homomorphic encryption,” IEEE Transactions on Dependable and Secure Computing, vol. 22, no. 1, pp. 221–233, 2025. [26] R. Gilad-Bachrach, N. Dowlin, K. Laine, K. Lauter, M. Naehrig, and J. Wernsing, “Cryptonets: Applying neural networks to encrypted data with high throughput and accuracy,” in Proceedings of The 33rd International Conference on Machine Learning, ser. Proceedings of Machine Learning Research, M. F. Balcan and K. Q. Weinberger, Eds., vol. 48. New York, New York, USA: PMLR, 20–22 Jun 2016, pp. 201–210. [Online]. Available: https: //proceedings.mlr.press/v48/gilad-bachrach16.html [27] X. Sun, P. Zhang, J. K. Liu, J. Yu, and W. Xie, “Private machine learning classification based on fully homomorphic encryption,” IEEE

13

Transactions on Emerging Topics in Computing, vol. 8, no. 2, pp. 352– 364, 2020. [28] A. Al Badawi, C. Jin, J. Lin, C. F. Mun, S. J. Jie, B. H. M. Tan, X. Nan, K. M. M. Aung, and V. R. Chandrasekhar, “Towards the alexnet moment for homomorphic encryption: Hcnn, the first homomorphic cnn on encrypted data with gpus,” IEEE Transactions on Emerging Topics in Computing, vol. 9, no. 3, pp. 1330–1343, 2021. [29] K. Nandakumar, N. Ratha, S. Pankanti, and S. Halevi, “Towards deep neural network training on encrypted data,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, June 2019. [30] P. Baldi and K. Hornik, “Neural networks and principal component analysis: Learning from examples without local minima,” Neural Networks, vol. 2, no. 1, pp. 53–58, 1989. [Online]. Available: https://www.sciencedirect.com/science/article/pii/0893608089900142 [31] R. Ratra, P. Gulia, N. S. Gill, and J. M. Chatterjee, “Big data privacy preservation using principal component analysis and random projection in healthcare,” Mathematical Problems in Engineering, vol. 2022, no. 1, p. 6402274, 2022. [32] C. Mouchet, J. Troncoso-Pastoriza, J.-P. Bossuat, and J.-P. Hubaux, “Multiparty homomorphic encryption from ring-learning-with-errors,” PoPETs, vol. 2021, no. 4, pp. 291–311, 2021. [33] C. V. Mouchet, “Multiparty homomorphic encryption: From theory to practice,” Ph.D. dissertation, EPFL, 2023. [34] J. Gao, P. Li, Z. Chen, and J. Zhang, “A survey on deep learning for multimodal data fusion,” Neural Computation, vol. 32, no. 5, pp. 829– 864, 05 2020. [Online]. Available: https://doi.org/10.1162/neco a 01273 [35] Y. Liu, C. Chen, L. Zheng, L. xilinx Wang, J. Zhou, and G.-J. Liu, “Privacy preserving pca for multiparty modeling,” ArXiv, vol. abs/2002.02091, 2020. [Online]. Available: https://api.semanticscholar. org/CorpusID:211043587 [36] D. Froelicher, H. Cho, M. Edupalli, J. Sa Sousa, J.-P. Bossuat, A. Pyrgelis, J. R. Troncoso-Pastoriza, B. Berger, and J.-P. Hubaux, “Scalable and privacy-preserving federated principal component analysis,” in 2023 IEEE Symposium on Security and Privacy (SP), 2023, pp. 1908–1925. [37] A. A. Badawi, A. Alexandru, J. Bates, F. Bergamaschi, D. B. Cousins, S. Erabelli, N. Genise, S. Halevi, H. Hunt, A. Kim, Y. Lee, Z. Liu, D. Micciancio, C. Pascoe, Y. Polyakov, I. Quah, S. R.V., K. Rohloff, J. Saylor, D. Suponitsky, M. Triplett, V. Vaikuntanathan, and V. Zucca, “OpenFHE: Open-source fully homomorphic encryption library,” Cryptology ePrint Archive, Paper 2022/915, 2022, https://eprint.iacr. org/2022/915. [Online]. Available: https://eprint.iacr.org/2022/915 [38] W. et al., “Breast Cancer Wisconsin (Diagnostic),” UCI Machine Learning Repository, 1993, DOI: https://doi.org/10.24432/C5DW2B. [39] Y. LeCun and C. Cortes, “MNIST handwritten digit database,” 2010. [Online]. Available: http://yann.lecun.com/exdb/mnist/ [40] Y. Netzer, T. Wang, A. Coates, A. Bissacco, B. Wu, and A. Ng, “Reading digits in natural images with unsupervised feature learning,” 2011. [Online]. Available: https://api.semanticscholar.org/CorpusID:16852518 [41] C. R. Harris, K. J. Millman, S. J. van der Walt, R. Gommers, P. Virtanen, D. Cournapeau, E. Wieser, J. Taylor, S. Berg, N. J. Smith, R. Kern, M. Picus, S. Hoyer, M. H. van Kerkwijk, M. Brett, A. Haldane, J. F. del Rı́o, M. Wiebe, P. Peterson, P. Gérard-Marchant, K. Sheppard, T. Reddy, W. Weckesser, H. Abbasi, C. Gohlke, and T. E. Oliphant, “Array programming with NumPy,” Nature, vol. 585, no. 7825, pp. 357–362, Sep. 2020. [Online]. Available: https://doi.org/10.1038/s41586-020-2649-2 [42] F. R., Practical Methods of Optimization. Wiley, 2000. [43] P. Virtanen, R. Gommers, T. E. Oliphant, M. Haberland, T. Reddy, D. Cournapeau, E. Burovski, P. Peterson, W. Weckesser, J. Bright, S. J. van der Walt, M. Brett, J. Wilson, K. J. Millman, N. Mayorov, A. R. J. Nelson, E. Jones, R. Kern, E. Larson, C. J. Carey, İ. Polat, Y. Feng, E. W. Moore, J. VanderPlas, D. Laxalde, J. Perktold, R. Cimrman, I. Henriksen, E. A. Quintero, C. R. Harris, A. M. Archibald, A. H. Ribeiro, F. Pedregosa, P. van Mulbregt, and SciPy 1.0 Contributors, “SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python,” Nature Methods, vol. 17, pp. 261–272, 2020. [44] Z. Wang, A. Bovik, H. Sheikh, and E. Simoncelli, “Image quality assessment: from error visibility to structural similarity,” IEEE Transactions on Image Processing, vol. 13, no. 4, pp. 600–612, 2004. [45] R. Zhang, P. Isola, A. A. Efros, E. Shechtman, and O. Wang, “The unreasonable effectiveness of deep features as a perceptual metric,” in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018, pp. 586–595.

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021

SUPPLEMENTARY MATERIAL A. Glossary We provide the frequently used notations and abbreviations throughout the manuscript in Table VI. B. Details of HADES’s training and packing strategies 1) Matrix Representation via Packing and Padding Strategies: One key challenge in implementing HE-based neural networks is efficiently encoding (packing) matrix representations and operations while adhering to constraints imposed by HE. In this section, we provide the mathematical details of HE operations that enable neural network training under encryption. These operations encompass our packing strategies for weight matrices, as well as ciphertext masking, ciphertext rotation, and the approximation of the activation functions. We adopt a packing strategy similar to that introduced in [17], which alternates matrix packing based on the layer index (the layer’s position counted from the input, starting from 1), as detailed below. This strategy adapts based on the operational requirements, such as matrix multiplication and network architecture. For a given matrix M ∈ Rm×n , we define a packing function ϕ : Rm×n → CN/2 that maps the matrix to a vector suitable for encryption. This function fundamentally involves flattening the matrix into a one-dimensional array and strategically inserting padding (zeros) between matrix elements or to adjust dimensions. The flattening step can be performed in two ways: rowmajor or column-major ordering. Let πrow and πcol denote these linearization functions. For example, consider the following matrices: " # a11 a12 πrow (A) A= − −−−− → [a11 , a12 , a21 , a22 ] a21 a22 " # b11 b12 πcol (B) B= −−−−→ [b11 , b21 , b12 , b22 ] b21 b22 Their product is given by A ⊡ B = [ a11 b11 + a12 b21 , a11 b12 + a12 b22 , a21 b11 + a22 b21 , a21 b12 + a22 b22 ] where ⊡ denotes the Hadamard (element-wise) product. The output represents a flattened form of the actual matrix product: " # a11 b11 + a12 b21 a11 b12 + a12 b22 A·B= a21 b11 + a22 b21 a21 b12 + a22 b22 Note that alternating row-major and column-major representations for matrices enables perfect alignment between the row elements of A and the column elements of B. However, simply taking the Hadamard product of the resulting flattened vectors does not always yield the correct matrix product. While alternating matrix representations work in simple cases, they prove insufficient when handling consecutive matrix operations where matrix sizes are unequal or when at least one dimension in the network is not a power of 2. To address this, a crucial aspect of our packing approach involves padding matrices with zeros, extending their non-common dimensions to the smallest power of 2 that is at least as large as the largest dimension among the two matrices. This ensures that the inner

14

dimensions of matrices in a multiplication A·B are conformant after any preparatory replications or expansions. It provides ”empty slots” in the ciphertext, crucial for preventing data overwriting during ciphertext rotations. Consider the multiplication of two matrices A ∈ Rm×k and B ∈ Rk×n , where m < n and n is not a power of 2. We first define the padded dimension for the non-common dimension as n̂ = 2⌈log2 n⌉ . Then, we compute the multiplication area µ as m × k × n̂. This is the total number of ciphertext slots required to make this particular matrix multiplication. The m× k matrix A is padded along its k-dimension to become an m × k̂ matrix Ã, and the k × n matrix B is padded along its n-dimension to become a k × n̂ matrix B̃. Calculation of k̂ = µ/m for the entire network is given in Section VII-B2, and it depends on the parameters of the entire network. We formalize these transformations as follows: ξ : Rm×k → Rm×k̂ , ψ:R

k×n

k×n̂

→R

,

ξ(A) = Ã, ψ(B) = B̃.

Here, Ã and B̃ denote the padded matrices. After padding, flattened matrix representation for A is rotated ⌊log2 n̂⌋ times by m elements at a time, while representation for b is rotated ⌊log2 m⌋ times by k̂ elements at a time. This allows for perfect local multiplication alignment while optimizing global alignment. As another example, consider these matrices: " # " # a11 a12 b11 b12 b13 A= and B = . a21 a22 b21 b22 b23 Since m < n and n is not a power of 2, we pad the matrices to dimensions 2 × 8 and 2 × 4, respectively. We first define n̂ = 2⌈log2 3⌉ = 4, and the padded matrices are: " # a11 a12 0 0 0 0 0 0 Ã = (padded to 2 × 8) a21 a22 0 0 0 0 0 0 " B̃ =

b11

b12

b13

# 0

b21

b22

b23

0

(padded to 2 × 4)

Next, we apply the row and column flattening operations: πrow (Ã) = [ a11 , a12 , 0, 0, 0, 0, 0, 0, a11 , a12 , 0, 0, 0, 0, 0, 0] πcol (B̃) = [ b11 , b21 , b12 , b22 , b13 , b23 , 0, 0]. and the rotation operations:
RR πcol (B̃), 8, 2 = [ b11 , b21 , b12 , b22 , b13 , b23 , 0, 0, b11 , b21 , b12 , b22 , b13 , b23 , 0, 0]. The flat multiplication à ⊡ B̃ then yields: à ⊡ B̃ = [ a11 b11 + a12 b21 , a11 b12 + a12 b22 , a11 b13 + a12 b23 , 0, a21 b11 + a22 b21 , a21 b12 + a22 b22 , a21 b13 + a22 b23 , 0 ].

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021

15

TABLE VI: Glossary Symbol / Term

Description

α B b BCO / BCD CKKS DMF di DP FL F FHE , FP GI GI-Load HE HADES iDLG K L L nk nt N PCA πrow , πcol r t ρ µ MHE spad k,t WHE , WPk,t k,t ∇WHE , ∇WPk,t zHE , zP , z̄

Fusion weight between encrypted and plaintext logits Mini-batch size A single batch inside a local epoch Breast Cancer Wisconsin (Original / Diagnostic) datasets Cheon–Kim–Kim–Song homomorphic-encryption scheme Deep Model Fusion Slot-padded dimension of layer i Differential Privacy Federated Learning Full feature set Features processed under HE / plaintext Global iteration (one server aggregation round) Relative iterations per fixed data volume Homomorphic Encryption Proposed framework Improved Deep Leakage from Gradients attack Total number of clients Loss value Multiplicative depth (CKKS levels) Samples on client k P Total samples in round t ( k∈K nk ) CKKS ring degree Principal Component Analysis Row/column flattening functions Layer index Training round index Rotation-parameter set Multiplication-area factor Multiparty Homomorphic Encryption Padding stride Client k weights (HE / P) at round t Gradients for HE / P sub-nets (client k, round t) Logits (HE, plaintext, fused)

This flattened result corresponds to the actual padded product: aligns the rows and columns of operand matrices for more a b + a b  a11 b12 + a12 b22 a11 b13 + a12 b23 0 efficient processing, reducing the number of costly ciphertext 11 11 12 21 a21 b11 + a22 b21 a21 b12 + a22 b22 a21 b13 + a22 b23 0 rotation operations during training. ÷B̃ =   0 0 0 0 2) Dynamic Packing for Deep Networks.: The previous 0 0 0 0 representation example was given with a local scope. For neural networks with multiple layers, additional challenges Finally, by discarding the padded zeros, we recover the arise. For example, if the amount of free slots required in actual product (for demonstration): " # the ciphertext is not calculated with a global view, there a11 b11 + a12 b21 a11 b12 + a12 b22 a11 b13 + a12 b23 could be misalignment in further layers. Additionally, the A·B = .representation created in the forward pass should be in a a21 b11 + a22 b21 a21 b12 + a22 b22 a21 b13 + a22 b23 format that additional time is not wasted in the backward pass. Note that although this multiplication could be performed more efficiently in this local example, the padding is necessary to ensure proper alignment across the entire network. This padding is crucial for two reasons: (i) It allows for rotations that would deem some network configurations impossible due to insufficient free slots in the ciphertext. This happens when elements are packed together too closely, or when non-powerof-2 dimension representations overwrite valid values (ii) It

Similar to the local-scope approach, we employ a dynamic packing scheme that adaptively switches layer weight matrices between row-major and column-major formats, following a strategy akin to the one proposed in [17]. Odd-numbered layers use column-major packing, while even-numbered layers use row-major packing, allowing for more efficient operations throughout the network. By taking subsequent layer dimensions throughout the network into consideration, we calculate

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021

16

Algorithm 3 Forward Pass with Alternating-Packing-Based Padding Strategy

Algorithm 4 Backward Pass with Alternating-Packing-Based Gradient Processing

Inputs: Layer index r ∈ N, input tensor X ∈ Rm×n , weight matrix Wr ∈ Rn×p , multiplication area µ ∈ Nk , dimension slots d ∈ Nk . 1: if r ≡ 1 (mod 2) then ▷ Select appropriate padding and rotation params. 2: spad ← µr · n 3: πpad ← πcol ▷ Padding function πpad is now using column-padding strategy 4: ρforward ← (1, d[r − 1]) 5: else 6: spad ← µr · p 7: πpad ← πrow ▷ Padding function πpad is now using column-padding strategy 8: ρforward ← (spad , d[r − 1]) 9: end if 10: Wr ← Pad(Wr , πpad , spad )

Inputs: Layer index r, input Zr−1 , incoming error δr , multiplication area µ, weights Wr , maximum slot capacity N /2 1: if r ̸= |d| − 1 then 2: δr ← δr ⊙ σ(Z̃r ) ▷ Activation derivative for pre-activation Z̃r 3: end if

11: 12: Zr ← HEMultiply(X, Wpad , ρforward ) ▷ Forward Pass 13: Z̃r ← Zr ▷ Retain pre-activation output 14: 15: if r ̸= |d| − 1 then 16: Zr ← σ(Zr ) ▷ Apply activation function 17: if r ≡ 1 (mod 2) then 18: Zr ← Rotate(Zr , right, 1, d[r + 1]) 19: else 20: nr ← d[r], nc ← d[r + 1] 21: µr+1 ← µ[r] 22: if (r + 1) ≡ 1 (mod 2) then 23: nr ← nr · µr+1 24: else 25: nc ← nc · µr+1 26: end if 27: Zr ← Rotate(Zr , right, nr , nc ) 28: end if 29: end if 30: return Zr

the multiplication area µ for each layer. Definition 1 (Dimension Slots Vector): The dimension slots vector d = [d0 , d1 , d2 , . . . , dL ] represents the padded dimensions for each layer, where: • d0 = padded input dimension • di = padded dimension between layer i − 1 and layer i • dL = padded output dimension These are determined by the network configuration and rounded up to the nearest power of two. Definition 2 (Maximum Slot Capacity): The maximum slot capacity N /∈ represents the total number of slots available in a single ciphertext. See Section III-C for more information. Definition 3 (Multiplication Area Vector): The multiplication area vector µ = [µ0 , µ1 , µ2 , . . . , µL−1 ] defines how many slots are used for each layer, where µi ∈ {1, 2, 4, 8, . . .} (powers of 2). µi determines the padding required for layer i, and is calculated via: N /2 µ[i] = d[i] × d[i + 1] Algorithms 3 and 4 detail our packing scheme and other network details for forward and backward passes, respectively. The forward pass algorithm initiates by selecting layer-indexdependent parameters using the input dimension slots vector

4:

Error Gradient Preparation: 5: if r ≡ 1 (mod 2) then 6: δr ← Rotate(δr , right, 1, d[r − 1]) 7: else N /2 8: δr ← Rotate(δr , right, d[r−1] , d[r − 1]) 9: end if 10:

Weight Gradient Computation: 11: ∇Wr ← HEMultiply(Zr−1 , δr ) 12:

Output Gradient Propagation: 13: δr−1 ← δr 14: if r > 1 then 15: if r ≡ 0 (mod 2) then 16: ρbackward ← (1, d[r]) 17: else /2 18: ρbackward ← ( N , d[r]) d[r] 19: end if 20: δr−1 ← HEMultiply(δr , Wr , ρbackward ) 21: end if 22: return δr−1 , ∇Wr 23:

Weight Update (After Aggregation): ▷ η is learning rate, ∇Wr is now the aggregated layer gradient 25: ∆Wr ← Rotate(∆W, right, µr · d[r], B) 26: Wr ← Wr − ∆Wr 27: Wr ← Bootstrap(Wr )

24: ∆Wr ← η · ∇Wr

d and µ (Lines 2-9). Specifically, for odd-indexed layers, the padding step spad (which determines the amount of padding between rows or columns) is determined by the current layer’s multiplication area µr and the number of rows in the weight matrix, with column-wise padding πcol and rotation parameters ρforward set using the dimension slot d[r − 1] (Lines 3-5). Conversely, for even-indexed layers, spad is based on µr and the matrix’s column count, utilizing row-wise padding πrow and rotation parameters derived from both spad and d[r − 1] (Lines 6-8). Following this, the weight matrix W is padded in plaintext space according to these parameters (Line 10). The linear transformation then occurs, producing the output Zr by leveraging the input tensor, the padded weights, and the established rotation parameters ρforward (Line 12). If the current layer is not the final one, an activation function is applied to Zr (Line 15). Subsequently, for odd-indexed layers, Zr is rotated based on the dimension slot d[r + 1] of the next layer (Line 18). For even-indexed layers, the rotation involves determining the next layer’s dimensions nr , nc from d (Line 20), retrieving the subsequent layer’s slot multiplier µr+1 from µ[r] (Line 21), and then adjusting either nr or nc based on µr+1 and the index of the next layer before the final rotation is applied to Zr (Lines 22-27). This rotation is crucial for the alignment of subsequent layers and gradient calculation.

JOURNAL OF LATEX CLASS FILES, VOL. 14, NO. 8, AUGUST 2021

The backward pass (Algorithm 4) begins by applying the derivative of the activation function to the propagated error δr , using the pre-activation output Z̃r , if the current layer is not the final one (Lines 2-4). For error-gradient preparation (Line 5), when the layer index r is odd, we rotate δr along the dimension slot exactly d[r − 1] times, using increasing successive offsets 1, 2, 4, 8, . . . up to the d[r − 1]th rotation. If the index is even, the rotation amount for δr is determined by dividing the maximum slot capacity N /2 by d[r −1] (Line 8). Next, the weight gradient ∇Wr is computed via multiplication of the previous layer’s output Zr−1 and the prepared δr (Line 11). The output gradient is propagated for all layers except the input layer. Layer-index-dependent rotation parameters ρbackward are selected: for even indices, using a fixed rotation M (Line 18). and d[r] (Line 16), and for odd indices, using d[r] Then, ∇Wr is updated by multiplying δr with the transpose of the current layer’s weights Wr using these rotation parameters (Line 20). Finally, during the weight update phase (Line 23), the change in weights ∆Wr is calculated using the learning rate η and ∇Wr (Line 24). The layer weights Wr are then updated by subtracting ∆Wr (Line 26) and bootstrapped if necessary (Line 27). We monitor the ciphertext levels of the weight matrices, and if they approach a critical threshold, we perform a bootstrapping operation to refresh the noise before it becomes irrecoverably high. The algorithm concludes by returning the propagated gradient ∇Wr .

17