arXiv CSopen access

TL++: Accuracy and Privacy Preserving Traversal Learning for Distributed Intelligent Systems

TL++: Accuracy and Privacy Preserving Traversal Learning for Distributed Intelligent Systems

arXiv:2606.25627v1 [cs.LG] 24 Jun 2026

ERDENEBILEG BATBAATAR, Neouly, Republic of Korea YOUNG YOON∗ , Department of Computer Engineering, Hongik University, Republic of Korea Distributed intelligent systems increasingly need to train across data silos without centralizing raw data. Federated learning keeps data local, but heterogeneous partitions can degrade accuracy and require repeated full-model exchange. Split learning reduces communication through cut-layer activations, but standard protocols do not generally recover pooled mini-batch gradients and may expose activations and gradients in plaintext. We present TL++, a two-mode traversal-learning framework that constructs virtual batches across nodes to recover centralized mini-batch gradient behavior under explicit synchronization assumptions. Base mode exchanges cut-layer activations and gradients instead of full models. Secure mode secret-shares each cut-layer activation and gradient between an orchestrator and a non-colluding helper, so neither server alone observes plaintext cut-layer tensors. This protection is limited to a semi-honest, non-colluding two-server setting; labels and output values used for loss remain visible to the orchestrator. In the lightweight secure path evaluated here, exactness requires the sharewise server path to be linear or affine; nonlinear server operations require nonlinear MPC or are approximate. We formalize TL++, analyze communication and computation costs, and evaluate it against federated- and split-learning baselines on CIFAR-10 and BioGPT/ PubMedQA using full fine-tuning and LoRA. On CIFAR-10, TL++ base cut 1 and exact secure cut 3 achieve 91.41 ± 0.19% and 90.93 ± 0.17% accuracy, outperforming the strongest measured non-TL++ baseline by more than 12 percentage points. TL++ base cut 1 also reduces per-step communication payload by 13.1× relative to full-model synchronization. PubMedQA results similarly favor TL++ under both tuning settings, though they remain descriptive without paired trajectories. Overall, TL++ approaches centralized-level utility while reducing communication and supporting scoped activation-level secret sharing. CCS Concepts: • Computing methodologies → Distributed artificial intelligence; • Security and privacy → Systems security. Additional Key Words and Phrases: Traversal learning, privacy-preserving machine learning, distributed deep learning, secure multiparty computation ACM Reference Format: Erdenebileg Batbaatar and Young Yoon. 2026. TL++: Accuracy and Privacy Preserving Traversal Learning for Distributed Intelligent Systems. ACM Trans. Intell. Syst. Technol. 1, 1 (June 2026), 25 pages. https://doi.org/ XXXXXXX.XXXXXXX

1

Introduction

Edge and distributed intelligent systems—including clinical monitors, autonomous fleets, and industrial sensors—generate sensitive data that could improve model robustness if pooled centrally. Centralization, however, raises privacy and bandwidth costs and can conflict with data-sovereignty ∗ Corresponding author

Authors’ Contact Information: Erdenebileg Batbaatar, Neouly, Seoul, Republic of Korea, [email protected]; Young Yoon, Department of Computer Engineering, Hongik University, Seoul, Republic of Korea, [email protected]. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]. © 2026 Copyright held by the owner/author(s). Publication rights licensed to ACM. ACM 2157-6912/2026/6-ART https://doi.org/XXXXXXX.XXXXXXX

ACM Trans. Intell. Syst. Technol., Vol. 1, No. 1, Article . Publication date: June 2026.

2

Batbaatar and Yoon

regulations such as GDPR [12] and HIPAA [55], as well as broader privacy and federated-learning concerns [25, 30]. A central challenge for distributed artificial intelligence (AI) is therefore to combine centralized-gradient equivalence, communication efficiency, and security of intermediate computations. Here, “accuracy losslessness” means producing the same mini-batch gradient update that centralized training would compute on the same sampled batch, not guaranteeing identical final test accuracy across all seeds or implementations. No existing paradigm satisfies all three requirements simultaneously. Federated Learning (FL) [38] keeps raw data on-device and aggregates local model updates, but falls short on each dimension. Under non-independent and identically distributed (non-IID) data, local/global gradient divergence degrades convergence with label skew [2, 26, 31, 59, 65]. Full-parameter exchange grows costly with model size; compression and quantization reduce bytes but may compound approximation error under heterogeneity [4, 6, 27, 36]. Finally, shared model updates can reveal training samples [13, 64, 66], while secure aggregation [7] adds overhead and does not protect intermediate split-learning computations. Clustered Federated Learning (CFL) [48] improves personalization by partitioning incompatible clients into specialized models, but it remains an FL-family method: clients exchange model updates, the target is no longer the pooled centralized objective, and intermediate split-learning activations or gradients are not protected. CFL is therefore complementary to TL++ rather than a substitute for its virtual-batch and activation-level security design. Split Learning (SL) [18, 56] improves communication by transmitting cut-layer activations rather than full parameters, but it still processes one client’s samples per forward pass, precluding crossparticipant gradient aggregation and large-batch benefits under heterogeneity [33]. It also sends cut-layer activations and gradients in plaintext, exposing them to gradient inversion [66], model inversion [19], and optimization-based reconstruction even at deep cuts [45]. Traversal Learning (TL) [5] resolves the accuracy limitation by coordinating mini-batch construction across distributed nodes. TL assembles virtual batches spanning multiple participants, enabling gradient aggregation mathematically equivalent to centralized training on pooled data, while retaining SL’s communication advantage of transmitting only cut-layer activations. TL satisfies accuracy losslessness and communication efficiency, but transmits activations and gradients in plaintext, offering no privacy guarantee against a semi-honest or compromised server. TL++ extends TL with two modes. Base mode reproduces TL’s plaintext protocol for trusted environments. Secure mode adds additive-secret-sharing-based multi-party computation (MPC) [9] through a non-colluding helper server, following common PPML designs [41, 42, 58], so cut-layer activations and gradients are split rather than exposed. The helper provides privacy separation, not optimization: it owns no training data, does not define the objective, and only processes the second masked share. Without this non-colluding holder, the orchestrator would receive or reconstruct plaintext tensors, reducing secure TL++ to the trusted base setting. The cut also does not itself cause accuracy loss: it only assigns the upper-model forward pass and corresponding backpropagation to the server partition. If that partition evaluates the same 𝑓server and returns the centralized cut-layer gradient, the node-side update remains centralized-gradient equivalent by the chain rule. Thus, 𝑓server may be nonlinear in base mode. The linearity condition is narrower: the lightweight secure protocol can evaluate shares independently without changing the result only when the sharewise path is linear or affine. For a cut activation 𝑎 = 𝑎 (1) + 𝑎 (2) , a linear/affine map 𝐿 satisfies 𝐿(𝑎 (1) + 𝑎 (2) ) = 𝐿(𝑎 (1) ) + 𝐿(𝑎 (2) ) once the bias is handled once or shared consistently, and its Jacobian is independent of the hidden plaintext. ReLU, pooling, normalization, and softmax generally violate this identity; secure TL++ can still support them with secure nonlinear protocols, otherwise sharewise evaluation is an approximation. The current

ACM Trans. Intell. Syst. Technol., Vol. 1, No. 1, Article . Publication date: June 2026.

TL++: Accuracy and Privacy Preserving Traversal Learning for Distributed Intelligent Systems

3

Table 1. Comparison of distributed learning paradigms. (A) Centralized-gradient equivalence under nonIID data. (C) Communication efficiency vs. full parameter transmission. (S) Protection for intermediate values under the stated threat model. ✓ satisfied; × not satisfied; ∼ partial. For TL++ secure, the ✓ entries assume semi-honest non-colluding servers and an exact additive-share evaluation path. In the lightweight implementation evaluated here, exactness holds when the operations applied independently to additive shares are linear/affine. Nonlinear server-side operations require additional secure nonlinear protocols; otherwise the corresponding secure run is approximate. Method

Transmission

(A) (C) (S)

FL [38] CFL [48] FL+Compress [4, 6, 36, 60] FL+secure aggregation [7]

Model params Model params / clustered models Compressed grads Encrypted params

× ∼ × ×

× × ∼ ×

× ∼ × ∼

SL [18, 56] SL+differential privacy SL+homomorphic encryption SplitFed [54]

Activations Noisy activations [11] Encrypted activations [49] Activations

× ∼ × ∼

✓ ✓ ∼ ✓

× ∼ ✓ ×

TL [5]

Activations

✓

✓

×

TL++ (base) TL++ (secure)

Activations Secret-shared activations; exact when sharewise server path is linear/affine

✓ ✓

✓ ✓

× ✓

additive-sharing protocol therefore makes only some cuts exact; it does not require linear server networks in all TL++ architectures. This integration raises three corresponding challenges. (1) Accuracy: The server partition must return the same cut-layer gradient that centralized backpropagation would produce. Plain delegation of upper-layer backpropagation does not break accuracy losslessness; approximating or incorrectly evaluating the server-side computation under sharing does. (2) Communication: Secret sharing doubles cut-path activation traffic and adds helper/server coordination, so the useful operating point depends on cut depth and synchronization cost. (3) Security: The protocol must keep intermediate activations and cut-layer gradients shared while reconstructing only values needed for loss computation. Table 1 positions TL++ relative to prior paradigms. The main contributions are as follows. • A two-mode traversal-learning framework. TL++ supports a base mode for trusted deployments and a secure mode for privacy-sensitive deployments. Both modes use traversallearning virtual batches; secure mode adds a non-colluding helper server and additive-share message formats. • A condition for exact additive-share evaluation. We show that TL virtual-batch forward and backward passes are exact over additive shares when the operations evaluated independently on shares are linear or affine. This is a protocol condition, not an architectural mandate: nonlinear 𝑓server is valid in base mode and in secure mode with secure nonlinear evaluation. In the current CNN implementation, exact secure evaluation corresponds to cut 3; secure cut 1 and cut 2 are reported as approximations because nonlinear server layers are evaluated without an added secure nonlinear protocol. ACM Trans. Intell. Syst. Technol., Vol. 1, No. 1, Article . Publication date: June 2026.

4

Batbaatar and Yoon

• Activation-level privacy with limited reconstruction. TL++ splits cut-layer activations and gradients between the orchestrator and helper. Only output values and label-dependent signals needed for loss and backpropagation are reconstructed; labels, metadata, collusion, and malicious deviations remain outside this guarantee unless additional mechanisms are added. • Communication analysis across cut points. We analyze the cost of applying MPC to cut-layer activations rather than full model updates. Secure mode roughly doubles cut-path activation/gradient traffic relative to base TL, but deeper cuts substantially reduce the payload. • Implementation and evaluation. We instantiate TL++ in a VGG-style [51] CIFAR-10 [28] setting and a BioGPT/PubMedQA [23, 37] low-rank adaptation (LoRA) [22] setting, covering image and biomedical natural language processing (NLP) workloads under the same splittraining design. The remainder of the paper reviews related work, presents the TL++ protocol, evaluates utility and communication cost, and discusses privacy assumptions and deployment limits. 2

Related Work

We map prior work against TL++’s three concerns: accuracy losslessness (A), communication efficiency (C), and security of intermediate computations (S). Table 1 summarizes the landscape. 2.1

Federated Learning

FL trains without centralizing raw data by aggregating local updates, but it only partially meets the three concerns. On (A), non-IID distributions can drive divergence between local and global objectives, with degradation tied to label skew [65]. SCAFFOLD [26], FedProx [34], FedNova [59], FedDyn [2], MOON [31], FedBN [35], adaptive federated optimizers [46], and personalized methods such as Ditto [32] mitigate objective inconsistency, drift, or feature shift, but do not generally recover the exact pooled centralized mini-batch gradient used by TL++. On (C), full-parameter exchange grows with model size [25, 27]. On (S), model updates can reveal training samples [66], and secure aggregation [7] adds overhead without protecting intermediate split-learning computations. Clustered and multi-task variants address cases where heterogeneity reflects multiple legitimate tasks. CFL [48] recursively partitions clients using update-direction similarity, improving personalization when a single pooled objective is inappropriate. This clarifies the scope of TL++: a single TL++ job assumes compatible labels, model semantics, and conditional distributions; CFL-style compatibility discovery could be used upstream to decide which clients should share a TL++ model. 2.2

Communication-Efficient Distributed Learning

Communication-efficient learning reduces bytes through compression, lower synchronization frequency, or network partitioning. Deep gradient compression [36], sparsification [60], QSGD [4], sign-based optimization [6], FedPAQ [47], and local SGD [53] reduce FL traffic but add approximation or staleness that can compound non-IID divergence, leaving (A) and (S) unresolved. SL and TL [5] instead send cut-layer activations, with savings that grow when the cut is earlier. Asynchronous and buffered FL reduce idle time [43, 61] but remain model-update protocols. SplitFed [54] parallelizes client-side split computation and aggregates client models, yet without cross-client virtual batches it does not recover pooled centralized-gradient equivalence under non-IID data. TL++ adds additive sharing, so its net benefit depends not only on activation size but also on node-model synchronization, node-gradient sharing, helper relay, serialization, and synchronization cadence; efficient operating points are cut-dependent. ACM Trans. Intell. Syst. Technol., Vol. 1, No. 1, Article . Publication date: June 2026.

TL++: Accuracy and Privacy Preserving Traversal Learning for Distributed Intelligent Systems

2.3

5

Split Learning

SL, proposed by Gupta and Raskar [18] and Vepakomma et al. [56], partitions a network at a cut layer: clients send activations, the server completes the forward pass and loss, then returns cut-layer gradients. Although used in privacy-sensitive domains such as healthcare [56], standard SL processes one client’s samples per pass, preventing cross-participant gradient aggregation and large-batch benefits under heterogeneity [33]. Its privacy is also limited: gradients can reconstruct inputs [66], activations are vulnerable to model inversion [19], and malicious servers can optimize reconstructions even at deep cuts [45]. NoPeek [57] reduces input–smashed-data dependence but not the cryptographic non-observability targeted by TL++ secure mode. TL [5] supplies virtual batches; SplitFed [54] supplies parallelism; TL++ adds protection for exposed activations and gradients. 2.4

Privacy-Preserving Machine Learning

Three main cryptographic approaches protect machine learning computations, but each falls short of satisfying all three concerns simultaneously in this setting. Differential privacy (DP) [11] injects calibrated noise into model updates, providing formal guarantees but with a privacy-utility trade-off that is unfavorable at high privacy budgets, particularly under non-IID data [25]—degrading (A) alongside (S). Homomorphic encryption (HE) [14] supports computation on ciphertext. Secure neural-network inference systems such as CryptoNets [15], GAZELLE [24], and Delphi [40] show how HE can be combined with circuit-based or protocol-level optimizations, but they primarily target private inference rather than virtual-batch training. Recent work has applied HE directly to SL: CURE [49] encrypts server-side model parameters using the CKKS approximate-arithmetic HE scheme [8], protecting intermediate activations and labels from a semi-honest server while retaining client-side plaintext computation. CURE achieves strong (S) guarantees and partially satisfies (C) by limiting encryption to the server-side portion only, but the HE computational overhead remains substantial for deep networks with many training iterations, limiting practical scalability and compromising (A) when accuracy-utility trade-offs are applied. MPC [9] enables multiple parties to jointly compute functions over private inputs without revealing them. Among MPC primitives, additive secret sharing is especially well-suited to the linear operations that dominate gradient aggregation, as linear functions compose correctly over shares without reconstruction (see Section 3). Bonawitz et al. [7] demonstrate practical secure aggregation for FL using additive secret sharing, protecting model updates from a semi-honest server, but this does not extend to intermediate splitlearning activations and gradients. TL++ adopts additive secret sharing on cut-layer activations and gradients directly, confining reconstruction to the output layer where loss computation requires it—satisfying (S) without sacrificing (A) or (C), and aligning with two- and three-party PPML protocols that rely on non-collusion or honest-majority assumptions [41, 42, 58]. Unlike HE-based approaches, additive secret sharing introduces no ciphertext expansion for linear operations, and its linear overhead scales with activation size rather than model depth; more general private neuralnetwork systems such as SecureNN and ABY3 support nonlinear layers by adding interaction and mixed protocols [41, 58]. This makes TL++ complementary to, rather than a replacement for, secure aggregation in federated settings. 2.5

Traversal Learning

TL was designed to satisfy (A) and (C): unlike FL, it does not average independently trained local models, and unlike SL, it does not process clients as separate gradient updates. Instead, TL assembles virtual batches across nodes and executes one forward–backward pass, producing updates equivalent to centralized mini-batch training on pooled data while leaving raw data local. ACM Trans. Intell. Syst. Technol., Vol. 1, No. 1, Article . Publication date: June 2026.

6

Batbaatar and Yoon

Batbaatar et al. [5] report gains over FL and SL across IID, non-IID, text, and medical datasets, especially under label skew. TL++ uses TL as base mode and adds secure mode with a non-colluding helper: activations and gradients are split so neither server sees plaintext intermediate values. When the sharewise path is linear/affine, the secure update is numerically identical to base mode while satisfying (S) at the cost of additional activation/share traffic. To our knowledge, TL++ is the first deployable framework to combine accuracy-lossless gradient aggregation, split-learning-level communication, and configurable protection of intermediate training signals. 3 3.1

Preliminaries Problem Formulation

𝑖 Consider a distributed system of 𝑁 nodes, where node 𝑖 holds a local dataset D𝑖 = {(x (𝑖𝑗 ) , 𝑦 𝑗(𝑖 ) )}𝑛𝑗=1 Í𝑁 of 𝑛𝑖 labeled samples. Let 𝑛 = 𝑖=1 𝑛𝑖 denote the total number of samples across all nodes, and let Ð𝑁 D = 𝑖=1 D𝑖 denote the (virtual) pooled dataset. Raw data never leaves its originating node. The global training objective is to minimize the empirical loss over the pooled dataset:

𝑁

min L (𝜽 ) = 𝜽

𝑛

𝑖     1 ∑︁ ∑︁ ℓ 𝑓 x (𝑖𝑗 ) ; 𝜽 , 𝑦 𝑗(𝑖 ) , 𝑛 𝑖=1 𝑗=1

(1)

where 𝑓 (· ; 𝜽 ) is the neural network with parameters 𝜽 and ℓ is a per-sample loss (e.g., cross-entropy). A standard mini-batch SGD step on a batch B ⊆ D of size 𝐵 computes: 1 ∑︁ 𝜽 ← 𝜽 −𝜂 · ∇𝜽 ℓ (𝑓 (x; 𝜽 ), 𝑦) , (2) 𝐵 (x,𝑦) ∈ B

where 𝜂 is the learning rate. Centralized training draws B uniformly from D, naturally mixing samples from all nodes in every update. Under non-IID data, 𝑃 (D𝑖 ) may differ from 𝑃 (D), so local-objective methods such as standard FL can compute gradients biased relative to Equation (1), degrading convergence [26, 65]. Here, accuracy losslessness is a gradient-level requirement: the distributed protocol should produce the same update as Equation (2) on the same uniformly drawn batch B, under matched initialization, batch order, optimizer state, and arithmetic. Final test accuracy may still vary across seeds, hardware, and regularization. 3.2

Split Learning Architecture

SL partitions the network 𝑓 at a designated cut layer 𝑐 into two sub-networks: a node model 𝑓node comprising layers 1 through 𝑐, and a server model 𝑓server comprising layers 𝑐 + 1 through the output. Each node 𝑖 holds a local copy of 𝑓node with parameters 𝜽 node , while the server maintains 𝑓server with parameters 𝜽 server . For a sample x, the forward pass proceeds as follows. The node computes the cut-layer activation: a = 𝑓node (x; 𝜽 node ),

(3)

and transmits a to the server. The server completes the forward pass, computing the output ˆ 𝑦), and backpropagates to obtain the cut-layer gradient: 𝑦ˆ = 𝑓server (a; 𝜽 server ), evaluates the loss ℓ (𝑦, g=

𝜕ℓ . 𝜕a

(4)

The server returns g to the node, which uses it to backpropagate through 𝑓node and update 𝜽 node . ACM Trans. Intell. Syst. Technol., Vol. 1, No. 1, Article . Publication date: June 2026.

TL++: Accuracy and Privacy Preserving Traversal Learning for Distributed Intelligent Systems

7

The communication cost per sample is 𝑂 (|a|), proportional to the activation dimension at the cut layer, rather than 𝑂 (|𝜽 |) as in FL. This advantage grows as the cut layer is placed earlier in the network or as the total model size increases [18, 56]. 3.3

Secure Multiparty Computation

MPC lets parties jointly evaluate ℎ(v1, . . . , v𝑘 ) over private inputs without any party learning more than its own input and the output [9, 62]. Additive Secret Sharing. TL++ employs two-party additive secret sharing as its MPC primitive [9, 42]. Formally, additive sharing is usually defined over a finite field or ring. In the implementation, tensors are represented with finite-precision arithmetic, and masks are sampled from the corresponding numerical domain. For readability, we write the operation over real-valued tensors: for a value 𝑣 ∈ R𝑑 , the secret sharing operation samples a random mask 𝑟 from the implementation domain and defines two shares: 𝑣 1 = 𝑟, 𝑣 2 = 𝑣 − 𝑟, (5) so that 𝑣 1 + 𝑣 2 = 𝑣 up to the chosen arithmetic representation. Under the ideal finite-domain sharing model, each share is individually independent of 𝑣; reconstruction requires both shares. Linearity of Additive Secret Sharing. The key property that makes additive secret sharing compatible with gradient aggregation is its linearity: for any values 𝑢, 𝑣 ∈ R𝑑 with shares (𝑢 1, 𝑢 2 ) and (𝑣 1, 𝑣 2 ), and any scalar 𝛼 ∈ R: (𝑢 + 𝑣)𝑘 = 𝑢𝑘 + 𝑣𝑘 , (𝛼 𝑣)𝑘 = 𝛼 𝑣𝑘 ,

𝑘 ∈ {1, 2},

𝑘 ∈ {1, 2}.

(6) (7)

That is, sums and scalar multiples of shared values can be computed by each party independently on its own shares, without communication and without reconstruction. Reconstruction of the result requires only the standard 𝑣 1 + 𝑣 2 = 𝑣 step. This property makes the lightweight secure TL++ path exact when it processes shares without reconstructing the cut activation. The claim is not architectural: in base mode 𝑓server may be any differentiable subnetwork, and secure mode may also use nonlinear 𝑓server if paired with secure nonlinear computation. The condition concerns only sharewise additive masking. For a linear map 𝐿, 𝐿(a1 + a2 ) = 𝐿(a1 ) + 𝐿(a2 ); affine biases can be added once or shared consistently. Thus, with a linear/affine sharewise path and no optional noise, masking introduces no approximation and the returned cut-layer gradient matches centralized backpropagation. Nonlinear operations evaluated independently on shares are different. ReLU, pooling, normalization, and attention softmax generally violate 𝑓 (a1 + a2 ) = 𝑓 (a1 ) + 𝑓 (a2 ), and their backward masks or Jacobians depend on the combined plaintext activation. Exact secure use therefore requires secure comparison, lookup, garbled-circuit, polynomial-approximation, or hybrid-MPC computation; otherwise the nonlinear sharewise path is an explicit approximation. For ReLU, the centralized operation ReLU(𝑎 (1) + 𝑎 (2) ) differs from ReLU(𝑎 (1) ) + ReLU(𝑎 (2) ); if 𝑎 (1) = 1 and 𝑎 (2) = −2, the former is 0 and the latter is 1. Backpropagation has the same problem because ReLU masks depend on the combined plaintext. Any loss of centralized-gradient equivalence therefore comes from approximating nonlinear operations under masking, not from the server-side subnetwork itself. Security Model. Following common two-server MPC and PPML assumptions [9, 42], TL++ assumes a semi-honest (honest-but-curious) adversary: each party follows the protocol faithfully but attempts to infer private values from observed messages. Under this model, additive secret sharing guarantees that neither the orchestrator (holding 𝑣 1 ) nor the helper server (holding 𝑣 2 ) can recover 𝑣 without the other’s share, provided they do not collude. The guarantee is scoped to shared cut-layer ACM Trans. Intell. Syst. Technol., Vol. 1, No. 1, Article . Publication date: June 2026.

8

Batbaatar and Yoon

activations and gradients; it does not hide labels, reconstructed outputs, timing, batch-membership metadata, or values intentionally revealed for loss computation. The non-collusion assumption between orchestrator and helper is a standard deployment assumption in two-party MPC [7, 9] and is discussed further in Section 6. 4 4.1

The Proposed TL++ System Architecture

TL++ has 𝑁 nodes, one orchestrator, and one helper server, following non-colluding PPML deployment patterns [41, 42, 58]. Node 𝑖 holds D𝑖 , a copy of 𝑓node , and local forward/backward computation. The orchestrator maintains 𝑓server , constructs virtual batches, and distributes parameters and sample indices. The server model remains a modeling choice: in base mode it may be nonlinear, while the exact lightweight secure implementation restricts only the operations evaluated independently on additive shares. A linear/affine sharewise path is exact; nonlinear paths require secure nonlinear protocols or are approximate. The helper maintains the corresponding sharewise path and participates in additive-secret-sharing MPC [9], receiving activation shares from nodes and exchanging partial outputs and label-dependent output-gradient signals with the orchestrator. Its role is privacy separation: it holds the second share so no single server can reconstruct cut-layer activations or gradients. If the orchestrator is trusted with those tensors, the helper is omitted and TL++ runs in base mode. Figure 1 shows the topology, per-batch messages, and secure flow. Nodes split each cut-layer activation between the orchestrator and helper. Messages are: A, local sample counts; B, model parameters and batch indices; C, activation shares; D, cut-layer gradient shares; and E, node-model gradients for aggregation. A dedicated inter-server Phase 2 exchange lets the helper send its partial output, the orchestrator reconstruct predictions for loss, and the orchestrator return the labeldependent output-gradient signal (Section 4.5). Panel C shows activation sharing, independent (1) (2) sharewise propagation, and the shared gradient relation G𝑚 = G𝑚 + G𝑚 . TL++ operates in two modes sharing this architecture: • Base mode: Nodes transmit plaintext activations to the orchestrator only; no helper server is required. This reproduces TL’s protocol exactly and is suited to trusted deployments. • Secure mode: Nodes split activations using Equation (5) and send one share to each server. The helper is needed only to protect cut-layer tensors from any single semi-honest server. Linear/affine server computations are exact on shares; nonlinear computations require secure nonlinear evaluation or become approximate. The orchestrator reconstructs output values for label-based loss and distributes the resulting output-gradient signal; cut-layer activations and gradients remain shared under the semi-honest non-collusion assumption. 4.2

Model Architecture and Cut-Layer Constraint

TL++ experiments use a custom VGG-style CNN [51] designed for CIFAR-10 (32 × 32 red–green– blue (RGB) images) [28]. The full network consists of three convolutional blocks followed by fully connected classification layers: • Block 1: Two Conv2D layers (3 → 64 → 64 channels, 3 × 3 kernels, padding 1), each followed by ReLU activation, then 2 × 2 MaxPool — output: 64 × 16 × 16. • Block 2: Two Conv2D layers (64 → 128 → 128 channels), each followed by ReLU, then 2 × 2 MaxPool — output: 128 × 8 × 8. • Block 3: Two Conv2D layers (128 → 256 → 256 channels), each followed by ReLU, then 2 × 2 MaxPool — output: 256 × 4 × 4. ACM Trans. Intell. Syst. Technol., Vol. 1, No. 1, Article . Publication date: June 2026.

TL++: Accuracy and Privacy Preserving Traversal Learning for Distributed Intelligent Systems

(A)

No collusion

Orchestrator

Helper Server (3rd party)

(B)

A Number of samples

Orchestrator and Helper C

E D

B

Single batch process

...

Single Node Node 1

Node 2

...

Model & indices

B A

Secret shares (masking) ...

9

C Cut layer activation D

Cut layer gradient

E

Node-model gradients

Node N Secret Share 2 (x2)

(C) Orchestrator

Secret Share 1 (x1)

Node

y = f(x) Additive Secret Sharing

Model storage copy

Activations (x)

Forward pass Orchestrator x1 , w 1 , b 1 , L 1 Server Model

Gradients

Node Model

Backward pass Partial gradient

Node 1 ...

Node N

Shuffle

L1

Batch 1

Batch M

Server Model

Compute loss

Model storage

Virtual batches

Helper x2 , w 2 , b 2 , L 2

L1

L2

L=

L1 +

Partial gradient

L2

Samples

Sample indices Local data

L2

Multi-Party Computation Number of samples

Fig. 1. TL++ architecture. (A) Three-party topology: 𝑁 nodes send additive secret shares of cut-layer activations to the orchestrator and a non-colluding helper server, so neither party observes plaintext intermediate values. (B) Per-batch communication: messages A–E denote sample count reporting, model parameter and index distribution, cut-layer activation shares, cut-layer gradient shares, and node-model gradient aggregation, respectively. An additional inter-server coordination channel (not shown) is used in secure mode for output reconstruction. (C) Secure protocol: nodes split activations into shares (𝑥 1, 𝑥 2 ) via additive secret sharing; orchestrator and helper independently propagate their shares through the linear/affine server model used by the exact secure protocol; the orchestrator reconstructs outputs for loss computation and shares the (1) (2) output-gradient signal needed to form partial cut-layer gradients G𝑚 = G𝑚 + G𝑚 .

• Classifier: Flatten → fully connected (FC) layer (4096 → 512) → ReLU → Dropout(0.5) [52] → FC layer (512 → 𝐶). All weights are initialized with Xavier uniform initialization [16]. The implementation exposes three cut-layer options: • cut 1 (default): after Block 1 MaxPool; activation dimension 64 × 16 × 16 = 16,384. • cut 2: after Block 2 MaxPool; activation dimension 128 × 8 × 8 = 8,192. • cut 3: after the first FC layer (pre-Dropout); activation dimension 512. Linearity condition for exact additive-share evaluation. As established in Section 3, the secure protocol’s gradient decomposition (Equation (14)) is algebraically exact when the operations evaluated independently on additive shares are linear/affine. This condition should be read narrowly. It is a condition for the lightweight masking protocol, not a mandate that the model designer choose a linear 𝑓server . In ordinary base mode, any differentiable server-side subnetwork can be trained ACM Trans. Intell. Syst. Technol., Vol. 1, No. 1, Article . Publication date: June 2026.

10

Batbaatar and Yoon

without affecting accuracy losslessness, provided that the virtual batch, parameters, optimizer state, and returned cut-layer gradient match the centralized computation. The distinction matters because secure mode hides the cut activation by keeping it split. The two servers do not reconstruct the activation before evaluating the sharewise server path; each server evaluates its own share and the partial results are combined. If that sharewise path is linear/affine, masking itself introduces no approximation or accuracy-losslessness concern. If it contains nonlinear functions and those functions are applied separately to shares, the represented function and gradient may differ from the centralized computation. A nonlinear 𝑓server is therefore still a valid modeling choice, but exact secure use of such a model requires additional secure nonlinear protocols; otherwise the run should be described as an approximation. For cut 1 and cut 2, the server model includes convolutional layers with ReLU activations, so the lightweight additive-share protocol alone is not exact for those cuts. For cut 3, the server model reduces to a single linear projection R512 → R𝐶 (the final fully connected layer without nonlinear activation), so masking introduces no loss under the exact protocol. Accordingly, this CNN uses cut 3 for exact secure evaluation, whereas base mode imposes no such restriction and supports all three cut points. Remark 1 (Approximation regime for cut 1–2 in secure mode). For secure cut 1 or cut 2, “approximation” means that nonlinear server layers are evaluated separately on additive shares. It does not mandate linear 𝑓server in general or imply that server-side backpropagation causes accuracy loss. If the sharewise path is linear/affine, additive masking preserves the function and gradient; the approximation arises only when nonlinear operations such as ReLU are applied without secure nonlinear protocols. Proposition 2 then does not apply. We report secure cut 1–2 only for sensitivity; strict secure-gradient correctness in this architecture requires cut 3 or secure nonlinear layers. 4.3

Virtual Batch Construction

Virtual batch construction, inherited from Traversal Learning [5], is the mechanism by which TL++ achieves gradient equivalence to centralized mini-batch training under non-IID data. Let each node 𝑖 hold 𝑛𝑖 samples indexed by a local index set I𝑖 . Prior to training, the orchestrator collects 𝑁 (message A) and constructs a global index set I = Ã 𝑁 I of the per-node sample counts {𝑛𝑖 }𝑖=1 𝑖=1 𝑖 cardinality 𝑛. For each training epoch, the orchestrator randomly permutes I and partitions it into 𝑀 = ⌈𝑛/𝐵⌉ non-overlapping virtual batches B1, . . . , B𝑀 , each of target size 𝐵 except possibly the final batch. Each virtual batch B𝑚 contains indices drawn from potentially all 𝑁 nodes, mirroring a centralized mini-batch drawn from the pooled dataset D under the same without-replacement epoch shuffle. When the final batch is smaller than 𝐵, the normalization uses its actual size |B𝑚 |. For each batch B𝑚 , the orchestrator decomposes it by node: B𝑚(𝑖 ) = B𝑚 ∩ I𝑖 , and sends each node 𝑖 the sub-index set B𝑚(𝑖 ) (message B). Node 𝑖 loads only the corresponding local samples {x (𝑖𝑗 ) } 𝑗 ∈ B (𝑖 ) , 𝑚 computes their cut-layer activations, and returns them. The orchestrator concatenates activations from all nodes to form the full virtual-batch activation matrix A𝑚 ∈ R𝐵×𝑑 , where 𝑑 is the cut-layer dimension. Label handling. Algorithm 1 and Section 4.5 assume the orchestrator has the virtual-batch labels Y𝑚 for loss computation. Node 𝑖 may attach labels {𝑦 𝑗(𝑖 ) } 𝑗 ∈ B (𝑖 ) to its activation message or send 𝑚 them over the same authenticated channel. This does not centralize raw inputs, but it is an explicit label-disclosure assumption: the orchestrator may observe labels, node membership, and batch participation unless additional anonymization or batching is used. These metadata are outside the activation-sharing guarantee. Section 6.2 discusses U-shaped variants that keep labels local [56]. ACM Trans. Intell. Syst. Technol., Vol. 1, No. 1, Article . Publication date: June 2026.

TL++: Accuracy and Privacy Preserving Traversal Learning for Distributed Intelligent Systems

11

Algorithm 1 TL++ Base Mode — Single Batch B𝑚 Require: Virtual batch indices B𝑚 ; current parameters 𝜽 node, 𝜽 server ; learning rate 𝜂; sub-batch sizes 𝑏𝑖 = |B𝑚(𝑖 ) |. 1: for each node 𝑖 in parallel do 2: Receive sub-indices B𝑚(𝑖 ) from orchestrator.   3: Compute activations: A (𝑖 ) ← 𝑓node {x (𝑖𝑗 ) } 𝑗 ∈ B (𝑖 ) ; 𝜽 node . 𝑚

4: Send A (𝑖 ) to orchestrator. 5: end for 6: Orchestrator: Concatenate A𝑚 ← [A (1) ∥ · · · ∥ A (𝑁 ) ].

Forward pass: Ŷ ← 𝑓server (A𝑚 ; 𝜽 server ). Compute loss: 𝐿 ← ℓ ( Ŷ, Y𝑚 ). Backward pass: G𝑚 ← 𝜕𝐿/𝜕A𝑚 ; Δ𝜽 server ← 𝜕𝐿/𝜕𝜽 server . 10: Update: 𝜽 server ← 𝜽 server − 𝜂 Δ𝜽 server . 11: for each node 𝑖 in parallel do 12: Receive sub-gradient G (𝑖 ) ← G𝑚 [B𝑚(𝑖 ) , :] from orchestrator. (𝑖 ) 13: Backpropagate: Δ𝜽 node ← 𝜕𝐿/𝜕𝜽 node via G (𝑖 ) . (𝑖 ) 14: Send Δ𝜽 node to orchestrator. 15: end for Í𝑁 (𝑖 ) 16: Orchestrator: Aggregate: Δ𝜽 node ← 𝐵1 𝑖=1 𝑏𝑖 · Δ𝜽 node . 17: Update: 𝜽 node ← 𝜽 node − 𝜂 Δ𝜽 node . 18: Broadcast updated 𝜽 node to all nodes. 7:

8: 9:

Proposition 1 (Gradient Eqivalence). Let A𝑚 be the virtual-batch activation matrix for batch B𝑚 with the random shuffling described above. Conditional on the same model parameters, optimizer state, and batch indices, the base TL++ server-side gradient computed from A𝑚 is identical to the gradient that centralized mini-batch SGD would compute on B𝑚 . Over the epoch shuffle, this gradient has the same sampling distribution as centralized mini-batch SGD on the pooled dataset D. Proof sketch. The virtual batch index construction samples the same index sets that a centralized without-replacement epoch shuffle would produce. Since the mapping from indices to samples is deterministic, the batch {x 𝑗 , 𝑦 𝑗 } 𝑗 ∈ B𝑚 is identical to the corresponding centralized mini-batch. The node models compute the same cut-layer activations that the centralized model would compute for those samples, and the orchestrator concatenates them in batch order before applying the same server model and loss. Therefore the subsequent forward and backward passes are algebraically identical to centralized SGD on the same batch; no approximation is introduced in base mode. □ In base mode, the concatenated activation matrix A𝑚 is formed in plaintext at the orchestrator. (1) (2) In secure mode, the orchestrator holds the share matrix A𝑚 and the helper holds A𝑚 , with (1) (2) A𝑚 + A𝑚 = A𝑚 ; concatenation and aggregation proceed share-wise by the linearity of additive secret sharing (Equations (6)–(7)). 4.4

Standard (Base) Protocol

Algorithm 1 summarizes the base-mode training protocol for a single virtual batch B𝑚 . The aggregation in line 17 weights each node’s gradient contribution by its sub-batch size 𝑏𝑖 = |B𝑚(𝑖 ) |, yielding the correct gradient of the global loss on the virtual batch regardless of how ACM Trans. Intell. Syst. Technol., Vol. 1, No. 1, Article . Publication date: June 2026.

12

Batbaatar and Yoon

the 𝐵 samples are distributed across nodes. When 𝑏𝑖 = 𝐵/𝑁 for all 𝑖 (equal sub-batch sizes), this Í (𝑖 ) reduces to the simple arithmetic mean 𝑁1 𝑖 Δ𝜽 node . The base protocol is mathematically equivalent to centralized mini-batch SGD on B𝑚 (Proposition 1). The per-batch communication cost is 𝑂 (𝐵 · 𝑑 + 𝑁 · |𝜽 node |), dominated by activation transmission and node-model gradient exchange, which is substantially less than FL’s per-round cost of 𝑂 (𝑁 · |𝜽 |) when 𝐵 · 𝑑 ≪ 𝑁 · |𝜽 |. 4.5

Secure Protocol

The secure protocol adds additive-secret-sharing MPC via a non-colluding helper [9, 41, 42, 58]. It has four phases per virtual batch. The exact lightweight path below assumes that operations applied independently to the two shares are linear or affine (Section 4.2); nonlinear server networks require secure nonlinear subprotocols to make the decompositions exact. Phase 1: Share Generation. Each node 𝑖 computes its cut-layer activation A (𝑖 ) as in Equation (3). (𝑖 ) It then samples a random mask R (𝑖 ) ∼ U (R | B𝑚 | ×𝑑 ) and computes two shares: A1(𝑖 ) = R (𝑖 ) ,

A2(𝑖 ) = A (𝑖 ) − R (𝑖 ) .

(8)

Node 𝑖 transmits A1(𝑖 ) to the orchestrator and A2(𝑖 ) to the helper server, retaining neither share locally after transmission. Phase 2: Independent Forward Passes and Label-Based Loss. Both servers concatenate their respective share matrices:   (𝑘 ) A𝑚 = A𝑘(1) ∥ · · · ∥ A𝑘(𝑁 ) , 𝑘 ∈ {1, 2}. (9) Each server independently executes the linear/affine sharewise server path on its share:   (𝑘 ) Ŷ𝑘 = 𝑓server A𝑚 ; 𝜽 server , 𝑘 ∈ {1, 2}.

(10)

The helper sends Ŷ2 to the orchestrator over the dedicated inter-server coordination channel. The orchestrator combines it with Ŷ1 and reconstructs Ŷ = Ŷ1 + Ŷ2 .

(11)

Because this sharewise server path is linear/affine, with any bias handled once or shared consistently, (1) (2) Ŷ1 + Ŷ2 = 𝑓server (A𝑚 + A𝑚 ) = 𝑓server (A𝑚 ) holds exactly. The orchestrator uses the label vector Y𝑚 associated with the virtual batch to compute the scalar loss 𝐿 = ℓ ( Ŷ, Y𝑚 ) and the output-gradient signal 𝜕𝐿 H𝑚 = . (12) 𝜕 Ŷ The orchestrator sends H𝑚 to the helper for backpropagation. This is the only point where output predictions are reconstructed, adding one inter-server round independent of 𝑁 . Labels are visible to the orchestrator but need not be sent to the helper; H𝑚 is nevertheless label-dependent and should be treated as a training signal (Section 6.2). Phase 3: Secure Backward Pass. Given H𝑚 , each server 𝑘 computes the partial cut-layer gradient associated with its activation share: (𝑘 ) G𝑚 = H𝑚 ·

𝜕 Ŷ𝑘 (𝑘 ) 𝜕A𝑚

,

𝑘 ∈ {1, 2}.

ACM Trans. Intell. Syst. Technol., Vol. 1, No. 1, Article . Publication date: June 2026.

(13)

TL++: Accuracy and Privacy Preserving Traversal Learning for Distributed Intelligent Systems

13

For this linear/affine sharewise server path, the full cut-layer gradient decomposes additively: G𝑚 =

𝜕𝐿 𝜕 Ŷ1 𝜕 Ŷ2 (1) (2) = H𝑚 · + H𝑚 · = G𝑚 + G𝑚 . (1) (2) 𝜕A𝑚 𝜕A𝑚 𝜕A𝑚

(14)

(1) (2) Thus (G𝑚 , G𝑚 ) is a valid additive sharing of G𝑚 : neither server sees the plaintext cut-layer (𝑘 ) gradient. Server 𝑘 returns sub-gradient share G𝑘(𝑖 ) = G𝑚 [B𝑚(𝑖 ) , :] to node 𝑖.

Phase 4: Node-Side Update and Aggregation. Node 𝑖 receives G1(𝑖 ) and G2(𝑖 ) , reconstructs G (𝑖 ) = (𝑖 ) (𝑖 ) G1 + G2(𝑖 ) , and backpropagates through 𝑓node to compute Δ𝜽 node . The orchestrator aggregates node gradients using the base-mode size-weighted formula. Proposition 2 (Secure Gradient Correctness). Let Proposition 1 hold, and suppose the operations evaluated sharewise by the two servers are linear or affine, with biases handled once or shared consistently. The gradient update produced by the secure protocol is numerically identical to that of the base protocol on the same virtual batch B𝑚 , and therefore to centralized mini-batch SGD on B𝑚 . Thus, delegating the server-side portion of backpropagation does not affect accuracy losslessness when the returned cut-layer gradient is exact; any loss of exactness comes from approximating the server computation under masking, not from the cut itself. (1) (2) Proof. By construction, A𝑚 + A𝑚 = A𝑚 exactly (Equation (8)). Because the sharewise server computation is linear/affine, Ŷ1 + Ŷ2 = 𝑓server (A𝑚 ) exactly. Its Jacobian is independent of the hidden plaintext activation, so the gradient decomposition in Equation (14) is also algebraically exact. No approximation error is introduced by masking in this case, and the secure protocol is numerically equivalent to the base protocol. □

4.6

Optimization

Learning rate and batch size. TL++ follows the standard linear scaling rule [17]: when the virtual batch size 𝐵 is increased by a factor 𝑘, the learning rate 𝜂 is also scaled by 𝑘 to maintain comparable gradient noise. This rule applies identically in base and secure modes, since both compute the same gradient estimates. Noise augmentation for privacy tuning. To provide a configurable privacy-utility trade-off, TL++ optionally adds independent calibrated Gaussian noise to activation shares and gradient shares before transmission, following the standard DP intuition of calibrated noise and formal accounting [1, 11, 39]. For activation shares: (𝑖 ) Ã𝑘(𝑖 ) = A𝑘(𝑖 ) + 𝜺 act,𝑘 ,

(𝑖 ) 2 𝜺 act,𝑘 ∼ N (0, 𝜎act I),

(15)

and for cut-layer gradient shares returned to nodes: (𝑖 ) G̃𝑘(𝑖 ) = G𝑘(𝑖 ) + 𝜺 grad,𝑘 ,

(𝑖 ) 2 𝜺 grad,𝑘 ∼ N (0, 𝜎grad I).

(16)

With 𝜎act = 𝜎grad = 0, the protocol is exact additive sharing with no noise-induced accuracy loss. Nonzero noise can empirically obfuscate intermediate values against reconstruction attacks [13, 64, 66], but changes the optimization path and may reduce accuracy. The parameters are independent because gradient shares can reveal more than activation shares; default values are 𝜎act = 0.02 and 𝜎grad = 0.10. These defaults are not differential-privacy claims; formal accounting is deferred to Section 6. ACM Trans. Intell. Syst. Technol., Vol. 1, No. 1, Article . Publication date: June 2026.

14

Batbaatar and Yoon

LoRA integration. For large language model (LLM) adaptation, TL++ can train only LoRAadapted [22] modules, following adapters, adaptive low-rank allocation, and quantized LoRA [10, 21, 63]. Base matrices W ∈ R𝑚×𝑛 are frozen and only ΔW = BA, with A ∈ R𝑟 ×𝑛 , B ∈ R𝑚×𝑟 , and 𝑟 ≪ min(𝑚, 𝑛), is trained. LoRA lowers synchronization cost, but it does not make an entire transformer server path compatible with sharewise additive evaluation: attention softmax, layer normalization, and other nonlinear components still require placement below the cut, base-mode plaintext evaluation, or secure nonlinear protocols. The exact sharing argument applies only to linear portions. 5

Experiments

This section evaluates whether TL++ retains centralized-training utility while reducing full-model exchange. We report CIFAR-10 [28] and BioGPT/PubMedQA [23, 37] results with deterministic tensor-payload accounting rather than packet-level runtime measurements. Because cut depth, mode, and synchronization cadence affect the utility–communication trade-off, we state the conditions under which each operating point is valid. 5.1

Experimental Setup and Reporting Protocol

Workloads and configurations. The image-classification experiments use CIFAR-10 [28]. We report class-count tasks with 𝐶 ∈ {2, 3, 4, 5, 6, 7, 8, 9, 10}, where 𝐶 = 10 is the full task. The languagemodel experiments use BioGPT [37] on PubMedQA [23] with both full fine-tuning and LoRA adaptation [22]. For CIFAR-10, we compare centralized training, TL++ base mode, TL++ secure mode, and representative distributed-learning baselines. Base TL++ is evaluated at cut 1, cut 2, and cut 3. Secure TL++ is also reported at all three cuts, but only cut 3 satisfies the exact additive-share evaluation condition used in Proposition 2. Secure cut 1 and secure cut 2 place nonlinear operations above the cut; without an added secure nonlinear protocol, they are therefore reported as sensitivity results rather than exact secure configurations. Metrics and interpretation. Accuracy is test accuracy percentage, reported as mean ± standard deviation over five independent runs. Comparisons are descriptive because per-seed paired predictions and trajectories are unavailable; TL++ means that exceed centralized baselines are treated as run-to-run variation. The accuracy tables report utility metrics only; Section 5.7 reports communication costs with tensor payloads plus serialization framing, headers, one-way latency, and protocol overhead. Accordingly, we describe byte-level “payload reduction” rather than wall-clock speedup unless runtime measurements are available. 5.2

CIFAR-10 Accuracy Across Class Counts

Table 2 shows that TL++ remains close to centralized training across the CIFAR-10 class-count sweep. On the full task, centralized training reaches 92.03 ± 0.15%, base cut 1 reaches 91.41 ± 0.19% (0.62 points lower), and exact secure cut 3 reaches 90.93 ± 0.17% (1.10 points lower), preserving most centralized utility while keeping raw examples distributed. Averaged over 𝐶 = 2 through 𝐶 = 10, centralized training obtains 94.03%, base cut 1 obtains 93.80%, and secure cut 3 obtains 93.53%. The corresponding average gaps from centralized training are 0.23 and 0.50 percentage points. Accuracy declines as the label space becomes harder, but TL++ follows the same trend as centralized training rather than showing a separate collapse from virtual-batch construction. The sweep also shows why the best trusted and exact secure cuts differ. As classes increase, cut placement matters more: earlier cuts leave more representation learning on the server and preserve base-mode accuracy, whereas the deepest cut reduces the server side to the linear portion required ACM Trans. Intell. Syst. Technol., Vol. 1, No. 1, Article . Publication date: June 2026.

TL++: Accuracy and Privacy Preserving Traversal Learning for Distributed Intelligent Systems

15

Table 2. CIFAR-10 test accuracy (%) across class-count tasks. Each entry is mean ± standard deviation over five runs. The best mean in each row is shown in bold; ties and small differences should be interpreted descriptively because per-seed paired tests were not available. Classes 𝐶

Centralized

base cut 1

base cut 2

base cut 3

secure cut 1

secure cut 2

secure cut 3

2 3 4 5 6 7 8 9 10

99.04 ± 0.17 97.37 ± 0.15 95.06 ± 0.55 94.56 ± 0.59 91.57 ± 0.71 92.30 ± 0.13 91.97 ± 0.89 92.39 ± 0.17 92.03 ± 0.15

98.98 ± 0.09 96.99 ± 0.48 95.19 ± 0.11 94.34 ± 0.48 91.90 ± 0.26 91.94 ± 0.20 91.57 ± 0.58 91.88 ± 0.17 91.41 ± 0.19

98.84 ± 0.14 96.56 ± 0.42 94.73 ± 0.51 93.82 ± 0.62 91.48 ± 0.29 91.59 ± 0.18 91.03 ± 0.24 91.09 ± 0.17 90.65 ± 0.25

99.09 ± 0.13 96.84 ± 0.35 94.24 ± 0.85 92.84 ± 0.75 90.82 ± 0.59 90.35 ± 0.62 88.91 ± 0.63 90.02 ± 0.38 89.67 ± 0.27

98.75 ± 0.12 95.29 ± 0.68 93.27 ± 1.04 91.74 ± 0.55 89.09 ± 1.07 89.95 ± 1.30 90.33 ± 0.96 90.47 ± 0.90 90.36 ± 0.69

98.77 ± 0.10 95.92 ± 0.51 93.76 ± 0.54 92.95 ± 0.86 90.88 ± 0.53 91.14 ± 0.32 90.55 ± 0.33 90.87 ± 0.34 90.16 ± 1.07

98.93 ± 0.13 96.96 ± 0.33 94.78 ± 0.70 94.34 ± 0.21 91.62 ± 0.49 91.29 ± 0.84 91.52 ± 0.18 91.38 ± 0.24 90.93 ± 0.17

Table 3. Aggregate CIFAR-10 accuracy and gap summary. Average accuracy and gaps are computed over 𝐶 = 2 through 𝐶 = 10 for measured rows. “Worst gap” is the most negative class-count gap relative to centralized training. Configuration

10-class

Avg.

Avg. gap

Worst gap

Centralized

92.03

94.03

–

–

FedAvg [38] FedProx [34] SCAFFOLD [26] Standard SL [56] SplitFed [54]

74.56 74.64 71.32 78.88 74.62

79.48 79.45 76.12 78.84 79.46

-14.55 -14.58 -17.92 -15.20 -14.58

-18.45 -18.45 -22.58 -27.50 -18.53

TL++ base, cut 1 TL++ base, cut 2 TL++ base, cut 3 TL++ secure, cut 1 TL++ secure, cut 2 TL++ secure, cut 3

91.41 90.65 89.67 90.36 90.16 90.93

93.80 93.31 92.53 92.14 92.78 93.53

-0.23 -0.72 -1.50 -1.89 -1.25 -0.50

-0.62 -1.30 -3.06 -2.82 -1.87 -1.10

for exact sharing. Base cut 1 therefore optimizes utility and payload in trusted mode, while secure cut 3 optimizes algebraic compatibility. 5.3

Aggregate CIFAR-10 Comparison and Gap Analysis

Table 3 summarizes the CIFAR-10 method comparison using full ten-class accuracy, average accuracy across the class-count sweep, and gaps relative to centralized training. The aggregate results distinguish TL++ from the measured FL-style and split-learning baselines in this workload summary. The baseline methods are substantially below centralized training, whereas TL++ base and secure configurations remain within a few percentage points. Among basemode runs, cut 1 has the best full-task accuracy and the smallest average gap. Among secure-mode runs, cut 3 has the best full-task accuracy and the smallest average gap, and it is also the cut that satisfies the exact linear-server condition. This distinction is the basis for the deployment guidance in Section 6: base cut 1 is the strongest trusted configuration in the reported CIFAR-10 experiments, while secure cut 3 is the supported exact secure configuration. ACM Trans. Intell. Syst. Technol., Vol. 1, No. 1, Article . Publication date: June 2026.

16

Batbaatar and Yoon

Table 4. Cut-layer sensitivity on CIFAR-10. “10-class” reports the full CIFAR-10 task. “Avg.” averages mean accuracies across 𝐶 = 2 through 𝐶 = 10. Δ Avg. reports Secure Avg. minus Base Avg. at the same cut. Cut

Dim.

Base 10-class

Secure 10-class

Base Avg.

Secure Avg.

Δ Avg.

cut 1 cut 2 cut 3

16,384 8,192 512

91.41 90.65 89.67

90.36 90.16 90.93

93.80 93.31 92.53

92.14 92.78 93.53

-1.66 -0.53 +1.00

The comparison is most informative when interpreted through the optimization objective of each method. FL-style baselines optimize local objectives between synchronization events; under label skew, the local gradients may point away from the pooled-data gradient and therefore require additional correction, more rounds, or stronger regularization. Standard SL avoids full-model exchange, but it does not by itself create a pooled virtual batch across participants. TL++ directly targets this missing ingredient: it keeps raw samples local while forming a virtual batch that better approximates centralized gradient aggregation. The gap summary therefore supports the claim that virtual-batch construction is the primary source of TL++’s utility advantage in this setting, while cut-layer transmission is the primary source of its communication advantage. At the same time, the table should not be read as a universal ranking of all distributed learning systems. The relative baseline performance depends on the non-IID partition, model capacity, optimizer schedule, number of local steps, and communication budget. The contribution of Table 3 is more specific: under the reported CIFAR-10 protocol, TL++ occupies a near-centralized utility regime that the selected FL-style and split-learning baselines do not reach. 5.4 Cut-Layer Sensitivity Table 4 isolates the effect of cut placement. The cut dimension is included because it affects both accuracy and communication: earlier cuts preserve more server-side computation, while deeper cuts reduce activation and share sizes. Base-mode accuracy decreases with depth, from 91.41% at cut 1 to 90.65% at cut 2 and 89.67% at cut 3, suggesting that this architecture benefits from leaving more representation learning on the server. Secure mode differs: secure cut 3 is strongest, with 90.93% ten-class and 93.53% average accuracy. We do not treat its advantage over base cut 3 as a general benefit of secrecy; rather, exact additive sharing at the linear cut introduced no observable penalty. The ablation also shows that activation dimension alone is insufficient. Deeper cuts shrink the cut tensor from 16,384 dimensions to 512, but move trainable parameters to the node side; with per-batch synchronization, total payload need not decrease monotonically. For secure TL++, cut 1 and cut 2 are sensitivity runs before nonlinear server layers and do not satisfy the proof. Secure cut 3 is therefore the correctness point even if its conservative-schedule payload is not minimal; communication should then be improved through amortization, compression, or parameter-efficient updates. 5.5

Performance as Label-Space Complexity Increases

The class-count sweep provides a stress test for heterogeneous classification. The average over the 2-, 3-, and 4-class tasks is 97.16% for centralized training, 97.05% for base cut 1, and 96.89% for secure cut 3. The average over the harder 8-, 9-, and 10-class tasks is 92.13%, 91.62%, and 91.28%, respectively. The ordering is stable: centralized training is the reference, base cut 1 is the strongest base TL++ configuration, and secure cut 3 is the strongest exact secure TL++ configuration. This supports ACM Trans. Intell. Syst. Technol., Vol. 1, No. 1, Article . Publication date: June 2026.

TL++: Accuracy and Privacy Preserving Traversal Learning for Distributed Intelligent Systems

17

Table 5. BioGPT/PubMedQA accuracy (%) for full fine-tuning and LoRA adaptation. Entries are mean ± standard deviation over five runs. Method

Full fine-tuning

LoRA

Centralized

81.33 ± 0.47

81.07 ± 1.25

FedAvg [38] FedProx [34] SCAFFOLD [26] Standard SL [56] SplitFed [54]

65.67 ± 3.21 65.33 ± 2.73 51.67 ± 0.33 55.44 ± 5.10 67.44 ± 3.50

55.89 ± 2.22 59.44 ± 5.17 53.67 ± 1.33 67.11 ± 2.71 61.44 ± 1.26

TL++ base TL++ secure

80.87 ± 1.68 83.67 ± 0.75

81.20 ± 2.13 82.60 ± 2.24

the bounded claim that TL++ tracks the centralized accuracy curve closely as class complexity increases. The moderate widening on harder tasks is expected because the model separates more decision regions under the same resolution and distributed constraints. The gap grows smoothly rather than abruptly; a failed virtual-batch mechanism would likely degrade more sharply as classes are added. Thus, harder tasks mainly reflect CIFAR-10 difficulty while TL++ remains close to the centralized trajectory, consistent with virtual batches recovering centralized-style gradient aggregation rather than making classification easier. 5.6

BioGPT Fine-Tuning and LoRA Adaptation

Table 5 evaluates whether the TL++ pattern extends beyond convolutional image classification. The BioGPT/PubMedQA results [23, 37] compare centralized training, distributed-learning baselines, TL++ base, and TL++ secure under both full fine-tuning and parameter-efficient LoRA adaptation [10, 21, 22]. TL++ remains close to centralized fine-tuning on PubMedQA. For full fine-tuning, TL++ base reaches 80.87%, within 0.46 percentage points of centralized training, while TL++ secure reaches 83.67%. For LoRA, TL++ base reaches 81.20%, comparable to centralized LoRA at 81.07%, and TL++ secure reaches 82.60%. Because the standard deviations are non-negligible and the aggregate summaries do not provide paired trajectories, these results should be read as evidence of comparable utility rather than as a definitive ranking of TL++ base, TL++ secure, full fine-tuning, and LoRA. The language-model experiment suggests that traversal learning is not tied to CNNs: utility can be preserved when trainable state is limited to parameter-efficient modules. LoRA reduces trainable parameters [22], consistent with adapter and quantized fine-tuning work [10, 21], and therefore reduces synchronization pressure. The security interpretation is narrower: unless nonlinear transformer operations are below the cut or protected by secure nonlinear protocols, BioGPT secure results are protocol-level sensitivity evidence rather than a proof of exact secure transformer training. PubMedQA is also a small case study; longer-context tasks, generative metrics, calibration, and domain-shift tests are needed before broad claims about TL++ for medical language models. 5.7

Communication-Efficiency Measurements

We report analytic per-virtual-batch communication and computation for FedAvg, FedProx, SCAFFOLD, standard SL, SplitFed, and TL++ base/secure cut 1/2/3. The default scenario uses 𝐵 = 128, ACM Trans. Intell. Syst. Technol., Vol. 1, No. 1, Article . Publication date: June 2026.

18

Batbaatar and Yoon (a) Aggregate communication volume 560

546

Wire payload (MB)

273

FedAvg 273 MB

273

273

3.4x

4.2x 102

79.4

8.9x

64.2

13.1x 30.7 20.9

17.6

20.9

(b) Critical-path communication time 4000

Network time (ms)

3500

3316 2977

3000 2500 2000

1668

1689

1668

1500

1281

1000 500

168

401

304

226

168

0

g Av

d Fe

ox

Pr

d Fe

OL

F AF

SC

FL

D

d.

St

SL

d Fe

lit

Sp

SL

se

Ba

C1

Ba

se

C2

Ba

se

C3

re

u ec

S

C1

re

S

u ec

C2

re

C3

u ec

S

TL++

Fig. 2. Communication cost for all eleven methods under the edge–cloud link model. (a) Aggregate wire payload (MB, log scale), including tensor payload plus serialization/framing overhead. Bold ratio labels give the reduction relative to FedAvg for communication-efficient TL++ configurations. (b) Critical-path network time (ms), accounting for parallel node messages, sequential standard-SL client execution, inter-server securemode messages, and the current helper model refresh in secure TL++.

𝑁 = 10, 32-bit tensors, 100 GFLOPs/s nodes, a 1,000 GFLOPs/s server, 100/200 Mb/s node uplink/downlink, 10 ms one-way node–server latency, 10 Gb/s inter-server bandwidth, 1 ms interserver latency, 5% serialization/framing overhead, and 512-byte headers. Accounting includes broadcasts, activations/gradients, labels and indices, node-gradient uploads, optimizer/aggregation work, FedProx/SCAFFOLD control terms, secure shares, output-share exchange, helper refresh, and helper-to-orchestrator relay. Secure cut 1–2 are approximate sensitivity measurements; cut 3 is exact. Figure 2(a) separates three regimes. In the full-model regime, FedAvg/FedProx require 273 MB and SCAFFOLD 546 MB; TL++ base cut 3 also reaches 273 MB because node-model synchronization dominates. In the cut-layer regime, standard SL is 17.6 MB, SplitFed and TL++ base cut 1 are 20.9 MB, and TL++ base cut 2 is 30.7 MB, giving 13.1× and 8.9× reductions for base cut 1/2 relative to FedAvg. In the secure-synchronization regime, secure cut 1/2 remain below FedAvg at 64.2/79.4 MB (4.2×/3.4× reductions) but exceed activation-only costs because gradient shares ACM Trans. Intell. Syst. Technol., Vol. 1, No. 1, Article . Publication date: June 2026.

TL++: Accuracy and Privacy Preserving Traversal Learning for Distributed Intelligent Systems

400

Critical-path compute (ms)

Secure device-ms Node/client

393

19

Server/coord. Protocol overhead

3.3x 300

200 121

121

121

120

121

120

120

121

120

120

Parallel Floor 120 ms

100

0

g Av

d Fe

ox

D OL FF

Pr

d Fe

A SC

FL

St

S d.

L

d Fe

lit

Sp

SL

se

Ba

C1

se

Ba

C2

Ba

se

C3

e

ur

c Se

C1

e

ur

c Se

C2

e

ur

c Se

C3

TL++

Fig. 3. Critical-path computation cost for all eleven methods. Bars are stacked: node/client (blue), server/coordinator (salmon), and protocol overhead (hatched orange). The model charges forward/backward FLOPs including pointwise layers, optimizer updates, server aggregation, FedProx and SCAFFOLD perparameter control terms, and secure sharing/reconstruction arithmetic. Protocol-overhead bars for Secure C1/C2/C3 are rendered at a minimum display height for legibility; actual values are ≤0.20 ms. Hollow diamonds mark total secure device-ms (critical-path time plus the helper’s parallel server computation); this additional resource cost does not lengthen the critical path.

traverse both server paths and helper refresh is included. Exact secure cut 3 reaches 560 MB because node-gradient shares and helper relay dominate. Figure 2(b) shows that payload alone is insufficient. Under the link model, base cut 1/2 take 168/226 ms and secure cut 1/2 take 304/401 ms, all below FedAvg/FedProx (1,668 ms) and SCAFFOLD (3,316 ms). Standard SL has the smallest payload but takes 1,281 ms because clients are serialized. Secure cut 3 is exact but takes 2,977 ms under per-batch synchronization, dominated by nodegradient-share exchange and helper relay. Figure 3 shows that computation is far less sensitive to the communication protocol than the network path. All parallel methods cluster near the 120 ms floor: TL++ base and secure cut 1 reach 120 ms, cut 2 reaches 120 ms, and FL-family methods reach 121 ms (the slight excess over the idealized 119 ms is due to optimizer, aggregation, and FedProx/SCAFFOLD per-parameter control terms). Standard SL is the sole outlier at 393 ms because node-side work is serialized across clients, 3.3× the parallel floor. Secure TL++ adds negligible critical-path arithmetic overhead (≤0.20 ms, rendered at minimum height in the figure), but it incurs additional helper-side device time that runs in parallel and does not extend the critical path: 88.6 ms for secure cut 1 and 45.1 ms for secure cut 2, reflected by the diamond markers at ≈210 ms and ≈165 ms above those bars. Taken together, Figures 2 and 3 give a more conservative deployment picture than activationshare estimates alone. TL++ base cut 1 is the strongest trusted configuration: 20.9 MB wire payload, 168 ms network time, 120 ms computation time, and near-centralized accuracy. Secure cut 1 is the most communication-efficient approximate-secure choice at 64.2 MB and 304 ms, with a 120 ms computation path and negligible (≤0.03 ms) critical-path protocol overhead; it does incur 88.6 ms of additional helper device time that runs off the critical path. Exact secure cut 3 satisfies Proposition 2 ACM Trans. Intell. Syst. Technol., Vol. 1, No. 1, Article . Publication date: June 2026.

20

Batbaatar and Yoon

but reaches 560 MB and 2,977 ms network time under per-batch synchronization—comparable to SCAFFOLD—because the large node-side state at this cut drives both the node-model sync and the helper relay. Practical exact-secure deployments therefore require synchronization amortization, sparse or quantized node updates, or parameter-efficient node-side training to bring cut 3’s network cost into the cut-layer regime. 6 6.1

Discussion Interpreting the Utility–Communication Trade-off

The experiments support a cut-dependent conclusion, not a claim that every TL++ configuration is communication-efficient. In trusted settings, base cut 1 is the strongest reported point: it has the best base accuracy and the smallest measured total payload; base cut 2 remains useful when a deeper split is desired. In secure settings, correctness comes first: cut 3 is the exact secure configuration for the evaluated lightweight additive-sharing protocol because the operations evaluated independently on shares above that cut are linear/affine and therefore compatible with sharewise additive decomposition. Its bottleneck is not the cut tensor but the large node-side state moved under per-batch synchronization. A practical decision rule follows. Use base cut 1 or cut 2 when the orchestrator is trusted and bandwidth is the main constraint. Use secure cut 3 when the orchestrator and helper are individually semi-honest but non-colluding and exact additive-sharing correctness is required. If neither trust assumption is acceptable, TL++ should be combined with stronger MPC or differential-privacy mechanisms rather than presented as complete protection. The best cut should be re-evaluated for each architecture because activation size, trainable-parameter placement, and synchronization frequency jointly determine payload. 6.2

Label Sharing and U-Shaped Split Variants

The evaluated protocol follows the common server-side-loss design in SL [18, 56]: nodes keep raw inputs local but provide labels for the selected virtual-batch samples to the orchestrator. This preserves the centralized-gradient interpretation, but it is a separate privacy assumption. In applications where labels are sensitive, secure TL++ should be read narrowly: it protects cut-layer activations and gradients under the semi-honest non-collusion assumption, but it does not hide labels, virtual-batch membership metadata, output predictions, or label-dependent loss gradients from every system component. A U-shaped split could keep labels at nodes by returning server representations or logits for local loss computation, after which nodes send output-gradient signals. This aligns with label-sensitive SL variants [8, 49, 56], but adds a round trip, stricter sample-order bookkeeping, less orchestrator visibility into loss, and a harder choice over revealing or secret-sharing label-dependent output gradients. We therefore treat label sharing as an explicit deployment assumption and leave U-shaped TL++ for future work. 6.3

Security Scope and Assumptions

Secure TL++ protects cut-layer activations and gradients from any single semi-honest server under the non-collusion model of Section 3. Each activation is split by Equation (5), so neither the orchestrator nor helper can reconstruct plaintext alone. The helper is the independent second-share holder, replacing single-server trust with non-collusion. The guarantee is limited: exact lightweight correctness requires linear/affine sharewise operations (cut 3 in CIFAR-10), output predictions are reconstructed for loss, and labels plus label-dependent gradients remain visible under the baseline protocol. ACM Trans. Intell. Syst. Technol., Vol. 1, No. 1, Article . Publication date: June 2026.

TL++: Accuracy and Privacy Preserving Traversal Learning for Distributed Intelligent Systems

21

The optional Gaussian perturbations 𝜎act and 𝜎grad were not evaluated in this study. End-to-end differential-privacy claims would require clipping or another sensitivity bound, a privacy accountant, and accuracy measurements under the selected noise levels [1, 39]. Additive sharing also does not address colluding servers, malicious protocol deviations, compromised nodes, membership inference from final outputs [50], reconstruction under side information [13], or timing and metadata side channels. Stronger deployments would need authenticated communication, auditing, malicioussecure MPC checks, and operational separation between orchestrator and helper. 6.4

Scalability, Limitations, and Future Work

The main evaluation limitation is that accuracy measurements are combined with deterministic cost accounting rather than deployed packet traces. The model includes serialization/framing overhead and a link-latency/bandwidth scenario, but it does not report measured bytes, graphics processing unit (GPU) utilization, memory pressure, straggler behavior, or end-to-end multi-host throughput. Two technical limitations are central. First, the exact lightweight secure path requires linear/affine sharewise operations; nonlinear server heads need polynomial approximations, garbled circuits, secure comparison, or hybrid MPC [29, 41, 42, 58, 62]. Second, secure cut 3 payload is dominated by node-side state under per-batch synchronization. Future systems work should test amortized synchronization, quantization, sparse or low-rank node updates, local-update schedules, larger node counts, heterogeneous client speeds, deadline-based virtual batches, leakage measurements, and privacy-noise sweeps with formal accounting. Another direction is to relax the backward path. Forward-Forward (FF) propagation trains layers with local goodness objectives, including recent distributed FF variants [3, 20], while Direct Feedback Alignment (DFA) uses direct random feedback from output errors [44]. These could reduce cut-layer gradient information returned to nodes and weaken sequential node–server dependencies, but they change the optimization objective and may affect TL’s centralized-gradient equivalence. FF- or DFA-based TL++ variants should therefore be evaluated with the same accuracy, leakage, and communication accounting. The workload scope is likewise limited. CIFAR-10 and PubMedQA give image and biomedicallanguage case studies, but not large-scale vision, speech, recommender, tabular medical, or longcontext generative workloads. Future evaluations should include architectures with naturally linear heads, architectures requiring secure nonlinear computation, and tasks whose intermediate activations are tested against reconstruction attacks [19, 45, 66]. 6.5

Deployment Implications

TL++ is most attractive when communication is expensive relative to local computation and data heterogeneity makes local-update FL unreliable. In trusted infrastructure, base TL++ keeps raw data local, avoids full-model exchange, and achieves near-centralized accuracy in the reported experiments. In semi-honest multi-server deployments, secure TL++ prevents any single server from observing plaintext intermediate representations, but designers must either choose a cut whose independently evaluated sharewise operations are linear/affine or add stronger MPC components for nonlinear server operations. If clients are highly resource-constrained or networks are latencydominated rather than bandwidth-limited, payload reduction alone may not improve training time; payload, latency, memory, and accuracy should be reported together. 7

Conclusion

This paper presented TL++, a traversal-learning framework combining virtual-batch optimization with optional additive-secret-sharing secure computation. Unlike conventional FL, TL++ exchanges cut-layer activations and gradients rather than full model updates, reducing communication while ACM Trans. Intell. Syst. Technol., Vol. 1, No. 1, Article . Publication date: June 2026.

22

Batbaatar and Yoon

keeping raw data local. Secure TL++ adds a non-colluding helper and additive sharing so cut-layer activations and gradients are not exposed to any single server under the semi-honest non-collusion assumption. Experiments on CIFAR-10 and BioGPT/PubMedQA show that TL++ can approach centralized accuracy while trading off optimization fidelity, communication cost, and privacy. Cost analysis shows that cut 1 and cut 2 achieve the lowest per-step payload in trusted and approximatesecure settings, with reductions up to 13.1× relative to full-model synchronization. Secure cut 3 satisfies exact additive sharing in the CNN implementation but has FL-level payload under per-batch synchronization, motivating amortization or parameter-efficient updates. TL++ therefore offers a middle ground between FL and SL for distributed AI when its trust, label-disclosure, and exactshare-evaluation assumptions hold. Future work will study runtime efficiency, stronger privacy, larger transformers, secure nonlinear layers, FF/DFA credit assignment, and personalized TL. Data Availability This work uses two publicly available datasets: • CIFAR-10 [28]: VGG-style CNN experiments under extreme non-IID partitioning; https: //www.cs.toronto.edu/~kriz/cifar.html. • PubMedQA [23]: BioGPT [37] fine-tuning with LoRA [22]; https://pubmedqa.github.io. Code Availability TL++ source code is available under the MIT License: • Main Framework: VGG-style CNN on CIFAR-10; https://github.com/neouly-inc/TLplus. • Medical Case Study: BioGPT [37] LoRA fine-tuning on PubMedQA [23]; https://github. com/neouly-inc/TLplus-BioGPT-LoRa. Acknowledgments This work was partly supported by the Institute of Information & Communications Technology Planning & Evaluation (IITP) - ITRC (Information Technology Research Center) grant funded by the Korea government (MSIT) (IITP-2026-RS-2023-00259099) and by 2026 Hongik University Research Fund. References [1] Martin Abadi, Andy Chu, Ian Goodfellow, H. Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. 2016. Deep Learning with Differential Privacy. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security (CCS ’16). ACM, New York, NY, USA, 308–318. doi:10.1145/2976749.2978318 [2] Durmus Alp Emre Acar, Yue Zhao, Ramon Matas, Matthew Mattina, Paul N. Whatmough, and Venkatesh Saligrama. 2021. Federated Learning Based on Dynamic Regularization. In International Conference on Learning Representations. OpenReview.net, Virtual, 1–36. https://openreview.net/forum?id=B7v4QMR6Z9w [3] Ege Aktemur, Ege Zorlutuna, Kaan Bilgili, Tacettin Emre Bok, Berrin Yanikoglu, and Suha Orhun Mutluergil. 2024. Going Forward-Forward in Distributed Deep Learning. arXiv preprint arXiv:2404.08573 abs/2404.08573 (2024), 1–9. doi:10.48550/arXiv.2404.08573 [4] Dan Alistarh, Demjan Grubic, Jerry Li, Ryota Tomioka, and Milan Vojnovic. 2017. QSGD: Communication-Efficient SGD via Gradient Quantization and Encoding. In Advances in Neural Information Processing Systems, Vol. 30. Curran Associates, Inc., Long Beach, CA, USA, 1709–1720. [5] Erdenebileg Batbaatar, Jeonggeol Kim, Yongcheol Kim, and Young Yoon. 2025. Traversal Learning Coordination for lossless and efficient distributed learning. Expert Systems 42, 11 (2025), e70141. doi:10.1111/exsy.70141 [6] Jeremy Bernstein, Yu-Xiang Wang, Kamyar Azizzadenesheli, and Anima Anandkumar. 2018. signSGD: Compressed Optimisation for Non-Convex Problems. In Proceedings of the 35th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 80). PMLR, Stockholm, Sweden, 560–569. ˜ [7] Keith Bonawitz, Vladimir Ivanov, Ben Kreuter, Antonio Marcedone, H.Brendan McMahan, Sarvar Patel, Daniel Ramage, Aaron Segal, and Karn Seth. 2017. Practical secure aggregation for privacy-preserving machine learning. In Proceedings ACM Trans. Intell. Syst. Technol., Vol. 1, No. 1, Article . Publication date: June 2026.

TL++: Accuracy and Privacy Preserving Traversal Learning for Distributed Intelligent Systems

23

of the 2017 ACM SIGSAC Conference on Computer and Communications Security (CCS ’17). ACM, New York, NY, USA, 1175–1191. doi:10.1145/3133956.3133982 [8] Jung Hee Cheon, Andrey Kim, Miran Kim, and Yongsoo Song. 2017. Homomorphic Encryption for Arithmetic of Approximate Numbers. In Advances in Cryptology – ASIACRYPT 2017. Lecture Notes in Computer Science, Vol. 10624. Springer International Publishing, Cham, Switzerland, 409–437. doi:10.1007/978-3-319-70694-8_15 [9] Ivan Damgård, Valerio Pastro, Nigel Smart, and Sarah Zakarias. 2012. Multiparty computation from somewhat homomorphic encryption. In Advances in Cryptology – CRYPTO 2012. Lecture Notes in Computer Science, Vol. 7417. Springer Berlin Heidelberg, Berlin, Heidelberg, 643–662. doi:10.1007/978-3-642-32009-5_38 [10] Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. 2023. QLoRA: Efficient Finetuning of Quantized LLMs. In Advances in Neural Information Processing Systems, Vol. 36. Curran Associates, Inc., New Orleans, LA, USA, 10088–10115. [11] Cynthia Dwork. 2008. Differential privacy: A survey of results. In Theory and Applications of Models of Computation. Lecture Notes in Computer Science, Vol. 4978. Springer Berlin Heidelberg, Berlin, Heidelberg, 1–19. doi:10.1007/978-3540-79228-4_1 [12] European Parliament and Council of the European Union. 2016. Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the Protection of Natural Persons with Regard to the Processing of Personal Data and on the Free Movement of Such Data, and Repealing Directive 95/46/EC (General Data Protection Regulation). Official Journal of the European Union, L119, 1–88. https://eur-lex.europa.eu/eli/reg/2016/679/oj [13] Jonas Geiping, Hartmut Bauermeister, Hannah Dr"oge, and Michael Moeller. 2020. Inverting Gradients—How Easy Is It to Break Privacy in Federated Learning?. In Advances in Neural Information Processing Systems, Vol. 33. Curran Associates, Inc., Virtual, 16937–16947. [14] Craig Gentry. 2009. Fully homomorphic encryption using ideal lattices. In Proceedings of the 41st Annual ACM Symposium on Theory of Computing (STOC ’09). ACM, Bethesda, MD, USA, 169–178. doi:10.1145/1536414.1536440 [15] Ran Gilad-Bachrach, Nathan Dowlin, Kim Laine, Kristin Lauter, Michael Naehrig, and John Wernsing. 2016. CryptoNets: Applying Neural Networks to Encrypted Data with High Throughput and Accuracy. In Proceedings of the 33rd International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 48). PMLR, New York, NY, USA, 201–210. https://proceedings.mlr.press/v48/gilad-bachrach16.html [16] Xavier Glorot and Yoshua Bengio. 2010. Understanding the Difficulty of Training Deep Feedforward Neural Networks. In Proceedings of the 13th International Conference on Artificial Intelligence and Statistics (Proceedings of Machine Learning Research, Vol. 9). PMLR, Chia Laguna Resort, Sardinia, Italy, 249–256. https://proceedings.mlr.press/v9/glorot10a.html [17] Priya Goyal, Piotr Dollár, Ross Girshick, Pieter Noordhuis, Lukasz Wesolowski, Adi Kyrola, Andrew Tulloch, Yangqing Jia, and Kaiming He. 2017. Accurate, large minibatch SGD: Training ImageNet in 1 hour. arXiv preprint arXiv:1706.02677 abs/1706.02677 (2017), 1–11. [18] Otkrist Gupta and Ramesh Raskar. 2018. Distributed learning of deep neural network over multiple agents. Journal of Network and Computer Applications 116, 1 (2018), 1–8. [19] Zecheng He, Tianwei Zhang, and Ruby B. Lee. 2019. Model inversion attacks against collaborative inference. In Proceedings of the 35th Annual Computer Security Applications Conference (ACSAC ’19). ACM, New York, NY, USA, 148–162. doi:10.1145/3359789.3359824 [20] Geoffrey Hinton. 2022. The Forward-Forward Algorithm: Some Preliminary Investigations. arXiv preprint arXiv:2212.13345 abs/2212.13345 (2022), 1–16. doi:10.48550/arXiv.2212.13345 [21] Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin de Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. 2019. Parameter-Efficient Transfer Learning for NLP. In Proceedings of the 36th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 97). PMLR, Long Beach, CA, USA, 2790–2799. [22] Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. LoRA: Low-rank adaptation of large language models. In Proceedings of the 10th International Conference on Learning Representations. OpenReview.net, Virtual, 13 pages. https://openreview.net/forum?id=nZeVKeeFYf9 [23] Qiao Jin, Bhuwan Dhingra, Zhengping Liu, William Cohen, and Xinghua Lu. 2019. PubMedQA: A dataset for biomedical research question answering. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP ’19). Association for Computational Linguistics, Hong Kong, China, 2567–2577. [24] Chiraag Juvekar, Vinod Vaikuntanathan, and Anantha Chandrakasan. 2018. GAZELLE: A Low Latency Framework for Secure Neural Network Inference. In 27th USENIX Security Symposium (USENIX Security 18). USENIX Association, Baltimore, MD, USA, 1651–1669. https://www.usenix.org/conference/usenixsecurity18/presentation/juvekar ˜ [25] Peter Kairouz, H.Brendan McMahan, Brendan Avent, Aurélien Bellet, Mehdi Bennis, Arjun Nitin Bhagoji, Kallista Bonawitz, Zachary Charles, Graham Cormode, and Rachel Cummings. 2021. Advances and open problems in federated learning. Foundations and Trends® in Machine Learning 14, 1–2 (2021), 1–210. https://www.nowpublishers.com/article/

ACM Trans. Intell. Syst. Technol., Vol. 1, No. 1, Article . Publication date: June 2026.

24

Batbaatar and Yoon

Details/MAL-083 [26] Sai Praneeth Karimireddy, Satyen Kale, Mehryar Mohri, Sashank Reddi, Sebastian Stich, and Ananda Theertha Suresh. 2020. SCAFFOLD: Stochastic controlled averaging for federated learning. In Proceedings of the 37th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 119). PMLR, Virtual, 5132–5143. ˜ McMahan, Felix X. Yu, Peter Richtárik, Ananda Theertha Suresh, and Dave Bacon. 2016. [27] Jakub Konečný, H.Brendan Federated learning: Strategies for improving communication efficiency. arXiv preprint arXiv:1610.05492 abs/1610.05492, 1 (2016), 1–38. [28] Alex Krizhevsky and Geoffrey Hinton. 2009. Learning multiple layers of features from tiny images. Technical Report 1, 4 (2009), 1–60. [29] Nishant Kumar, Mayank Rathee, Nishanth Chandran, Divya Gupta, Aseem Rastogi, and Rahul Sharma. 2020. CrypTFlow: Secure TensorFlow Inference. In 2020 IEEE Symposium on Security and Privacy. IEEE, San Francisco, CA, USA, 336–353. [30] Li Li, Yuxi Fan, Mike Tse, and Kuo-Yi Lin. 2020. A review of applications in federated learning. Computers & Industrial Engineering 149, 1 (2020), 106854. [31] Qinbin Li, Bingsheng He, and Dawn Song. 2021. Model-Contrastive Federated Learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, Virtual, 10713–10722. [32] Tian Li, Shengyuan Hu, Ahmad Beirami, and Virginia Smith. 2021. Ditto: Fair and Robust Federated Learning Through Personalization. In Proceedings of the 38th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 139). PMLR, Virtual, 6357–6368. [33] Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, and Virginia Smith. 2020. Federated optimization in heterogeneous networks. In Proceedings of Machine Learning and Systems, Vol. 2. MLSys, Austin, TX, USA, 429–450. [34] Tian Li, Anit Kumar Sahu, Manzil Zaheer, Maziar Sanjabi, Ameet Talwalkar, and Virginia Smith. 2020. Federated optimization in heterogeneous networks. In Proceedings of Machine Learning and Systems, Vol. 2. MLSys, Austin, TX, USA, 429–450. [35] Xiaoxiao Li, Meirui Jiang, Xiaofei Zhang, Michael Kamp, and Qi Dou. 2021. FedBN: Federated Learning on Non-IID Features via Local Batch Normalization. In International Conference on Learning Representations. OpenReview.net, Virtual, 1–18. https://openreview.net/forum?id=6YEQUn0QICG [36] Yujun Lin, Song Han, Huizi Mao, Yu Wang, and William J. Dally. 2018. Deep gradient compression: Reducing the communication bandwidth for distributed training. In Proceedings of the 6th International Conference on Learning Representations. OpenReview.net, Vancouver, BC, Canada, 14 pages. https://openreview.net/forum?id=SkhQHMW0W [37] Renqian Luo, Liai Sun, Yingce Xia, Tao Qin, Sheng Zhang, Hoifung Poon, and Tie-Yan Liu. 2022. BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Mining. Briefings in Bioinformatics 23, 6 (2022), bbac409. doi:10.1093/bib/bbac409 [38] Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. 2017. Communicationefficient learning of deep networks from decentralized data. In Proceedings of the 20th International Conference on Artificial Intelligence and Statistics (Proceedings of Machine Learning Research, Vol. 54). PMLR, Fort Lauderdale, FL, USA, 1273–1282. [39] Ilya Mironov. 2017. Rényi differential privacy. In Proceedings of the 30th IEEE Computer Security Foundations Symposium (CSF ’17). IEEE, Santa Barbara, CA, USA, 263–275. doi:10.1109/CSF.2017.11 [40] Pratyush Mishra, Ryan Lehmkuhl, Akshayaram Srinivasan, Wenting Zheng, and Raluca Ada Popa. 2020. Delphi: A Cryptographic Inference Service for Neural Networks. In 29th USENIX Security Symposium (USENIX Security 20). USENIX Association, Virtual, 2505–2522. https://www.usenix.org/conference/usenixsecurity20/presentation/mishra [41] Payman Mohassel and Peter Rindal. 2018. ABY3: A Mixed Protocol Framework for Machine Learning. In Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security. ACM, Toronto, ON, Canada, 35–52. [42] Payman Mohassel and Yupeng Zhang. 2017. SecureML: A System for Scalable Privacy-Preserving Machine Learning. In Proceedings of the 2017 IEEE Symposium on Security and Privacy (SP ’17). IEEE, San Jose, CA, USA, 19–38. doi:10. 1109/SP.2017.12 [43] John Nguyen, Kshitiz Malik, Maziar Sanjabi, and Michael Rabbat. 2022. Federated Learning with Buffered Asynchronous Aggregation. In Proceedings of The 25th International Conference on Artificial Intelligence and Statistics (Proceedings of Machine Learning Research, Vol. 151). PMLR, Virtual, 3581–3607. [44] Arild Nøkland. 2016. Direct Feedback Alignment Provides Learning in Deep Neural Networks. In Advances in Neural Information Processing Systems, Vol. 29. Curran Associates, Inc., Barcelona, Spain, 1–9. doi:10.48550/arXiv.1609.01596 [45] Dario Pasquini, Giuseppe Ateniese, and Massimo Bernaschi. 2021. Unleashing the tiger: Inference attacks on split learning. In Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security (CCS ’21). ACM, New York, NY, USA, 2113–2129. doi:10.1145/3460120.3485259 [46] Sashank J. Reddi, Zachary Charles, Manzil Zaheer, Zachary Garrett, Keith Rush, Jakub Konečný, Sanjiv Kumar, and H. Brendan McMahan. 2021. Adaptive Federated Optimization. In International Conference on Learning Representations.

ACM Trans. Intell. Syst. Technol., Vol. 1, No. 1, Article . Publication date: June 2026.

TL++: Accuracy and Privacy Preserving Traversal Learning for Distributed Intelligent Systems

25

OpenReview.net, Virtual, 1–36. https://openreview.net/forum?id=LkFG3lB13U5 [47] Amirhossein Reisizadeh, Aryan Mokhtari, Hamed Hassani, Ali Jadbabaie, and Ramtin Pedarsani. 2020. FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantization. In Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics (Proceedings of Machine Learning Research, Vol. 108). PMLR, Palermo, Italy, 2021–2031. https://proceedings.mlr.press/v108/reisizadeh20a.html [48] Felix Sattler, Klaus-Robert Müller, and Wojciech Samek. 2019. Clustered federated learning: Model-agnostic distributed multi-task optimization under privacy constraints. arXiv preprint arXiv:1910.01991 abs/1910.01991 (2019), 1–16. https://arxiv.org/abs/1910.01991 [49] Sinem Sav, Apostolos Pyrgelis, Jean Louis Raisaro, David Froelicher, Juan Ramon Troncoso-Pastoriza, and JeanPierre Hubaux. 2024. CURE: Privacy-preserving split learning done right. arXiv preprint arXiv:2407.08977. https: //arxiv.org/abs/2407.08977 [50] Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. 2017. Membership Inference Attacks Against Machine Learning Models. In Proceedings of the 2017 IEEE Symposium on Security and Privacy (SP ’17). IEEE, San Jose, CA, USA, 3–18. doi:10.1109/SP.2017.41 [51] Karen Simonyan and Andrew Zisserman. 2015. Very Deep Convolutional Networks for Large-Scale Image Recognition. In Proceedings of the 3rd International Conference on Learning Representations (ICLR ’15). Computational and Biological Learning Society, San Diego, CA, USA, 1–14. https://arxiv.org/abs/1409.1556 [52] Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: A Simple Way to Prevent Neural Networks from Overfitting. Journal of Machine Learning Research 15, 56 (2014), 1929–1958. https://jmlr.org/papers/v15/srivastava14a.html [53] Sebastian U. Stich. 2019. Local SGD Converges Fast and Communicates Little. In International Conference on Learning Representations. OpenReview.net, New Orleans, LA, USA, 1–16. https://openreview.net/forum?id=S1g2JnRcFX [54] Chandra Thapa, Pathum Chamikara Mahawaga Arachchige, Seyit Camtepe, and Lichao Sun. 2022. SplitFed: When federated learning meets split learning. In Proceedings of the 36th AAAI Conference on Artificial Intelligence (AAAI ’22). AAAI Press, Virtual, 8485–8493. doi:10.1609/aaai.v36i8.20825 [55] United States Congress. 1996. Health Insurance Portability and Accountability Act of 1996. Public Law 104–191, 110 Stat. 1936. https://aspe.hhs.gov/reports/health-insurance-portability-accountability-act-1996 [56] Praneeth Vepakomma, Otkrist Gupta, Tristan Swedish, and Ramesh Raskar. 2018. Split learning for health: Distributed deep learning without sharing raw patient data. arXiv preprint arXiv:1812.00564 abs/1812.00564, 1 (2018), 1–6. [57] Praneeth Vepakomma, Abhishek Singh, Otkrist Gupta, and Ramesh Raskar. 2020. NoPeek: Information Leakage Reduction to Share Activations in Distributed Deep Learning. In 2020 IEEE International Conference on Data Mining Workshops (ICDMW). IEEE, Sorrento, Italy, 933–942. [58] Sameer Wagh, Divya Gupta, and Nishanth Chandran. 2019. SecureNN: 3-Party Secure Computation for Neural Network Training. Proceedings on Privacy Enhancing Technologies 2019, 3 (2019), 26–49. [59] Jianyu Wang, Qinghua Liu, Hao Liang, Gauri Joshi, and H. Vincent Poor. 2020. Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimization. In Advances in Neural Information Processing Systems, Vol. 33. Curran Associates, Inc., Virtual, 7611–7623. [60] Jianqiao Wangni, Jialei Wang, Ji Liu, and Tong Zhang. 2018. Gradient sparsification for communication-efficient distributed optimization. In Advances in Neural Information Processing Systems, Vol. 31. Curran Associates, Inc., Montréal, QC, Canada, 1306–1316. https://proceedings.neurips.cc/paper/2018/hash/3328bdf9a4b9504b9398284244fe97c2-Abstract. html [61] Cong Xie, Sanmi Koyejo, and Indranil Gupta. 2019. Asynchronous Federated Optimization. arXiv preprint arXiv:1903.03934 abs/1903.03934 (2019), 1–12. [62] Andrew Chi-Chih Yao. 1982. Protocols for Secure Computations. In Proceedings of the 23rd Annual Symposium on Foundations of Computer Science (FOCS ’82). IEEE, Chicago, IL, USA, 160–164. doi:10.1109/SFCS.1982.38 [63] Qingru Zhang, Minshuo Chen, Alexander Bukharin, Nikos Karampatziakis, Pengcheng He, Yu Cheng, Weizhu Chen, and Tuo Zhao. 2023. AdaLoRA: Adaptive Budget Allocation for Parameter-Efficient Fine-Tuning. In International Conference on Learning Representations. OpenReview.net, Kigali, Rwanda, 1–23. https://openreview.net/forum?id=lq62uWRJjiY [64] Bo Zhao, Konda Reddy Mopuri, and Hakan Bilen. 2020. iDLG: Improved Deep Leakage from Gradients. arXiv preprint arXiv:2001.02610 abs/2001.02610 (2020), 1–8. [65] Yue Zhao, Meng Li, Liangzhen Lai, Naveen Suda, Damon Civin, and Vikas Chandra. 2018. Federated learning with non-IID data. arXiv preprint arXiv:1806.00582 abs/1806.00582 (2018), 1–13. [66] Ligeng Zhu, Zhijian Liu, and Song Han. 2019. Deep leakage from gradients. In Advances in Neural Information Processing Systems, Vol. 32. Curran Associates, Inc., Vancouver, BC, Canada, 14774–14784. https://proceedings.neurips.cc/paper/ 2019/hash/60a6c4002cc7b29142def8871531281a-Abstract.html

ACM Trans. Intell. Syst. Technol., Vol. 1, No. 1, Article . Publication date: June 2026.