StreamSplit: Continuous Audio Representation Learning via Uncertainty-Guided Adaptive Splitting Minh K. Quan
Pubudu N. Pathirana
School of Engineering, Deakin University Waurn Ponds, Australia [email protected]
School of Engineering, Deakin University Waurn Ponds, Australia [email protected]
arXiv:2605.26523v1 [cs.DC] 26 May 2026
Abstract Large-batch Contrastive Learning (CL), the foundation of modern representation learning, is fundamentally incompatible with the volatile resource constraints of edge devices. This conflict creates a dilemma: small on-device batches degrade model fidelity, while offloading to the cloud incurs unacceptable latency and bandwidth costs. Existing solutions often resort to static model compression, which fails to adapt to the runtime volatility of edge environments. To bridge this gap, we present StreamSplit, a novel framework that makes streaming CL practical across heterogeneous ARM client platforms. StreamSplit resolves the conflict between the continuous nature of ambient audio and the discrete batch requirements of models like CLAP and COLA. We introduce: (1) A distributionbased streaming framework that decouples representation quality from local batch size, using a tractable Hybrid Loss to maintain fidelity despite sparse updates; and (2) An Uncertainty-Guided Adaptive Splitter that uses a lightweight Reinforcement Learning (RL) policy to dynamically partition computation. Uniquely, this policy integrates real-time resource monitoring with embedding ambiguity to optimize the accuracy-latency trade-off on the fly. We evaluate StreamSplit on diverse hardware, from the resourceconstrained Raspberry Pi 4 to the high-performance Apple M2. Results demonstrate that StreamSplit reduces per-sample latency by up to 4.7× and cuts bandwidth by 77.1% and energy by 52.3% compared to server-centric baselines. Crucially, it maintains accuracy within 2.2% of server-centric models, proving that adaptive, distributed learning is a viable path for the modern edge ecosystem.
CCS Concepts • Computing methodologies → Distributed algorithms; • Computer systems organization → Sensor networks.
Keywords Edge Computing, Contrastive Learning, Reinforcement Learning, Split Computing, Resource Management ACM Reference Format: Minh K. Quan and Pubudu N. Pathirana. 2026. StreamSplit: Continuous Audio Representation Learning via Uncertainty-Guided Adaptive Splitting. In The 24th Annual International Conference on Mobile Systems, Applications and Services (MobiSys ’26), June 21–25, 2026, Cambridge, United Kingdom. ACM, New York, NY, USA, 15 pages. https://doi.org/10.1145/3745756.3809189
This work is licensed under a Creative Commons Attribution 4.0 International License. MobiSys ’26, Cambridge, United Kingdom © 2026 Copyright held by the owner/author(s). ACM ISBN 979-8-4007-2027-7/26/06 https://doi.org/10.1145/3745756.3809189
Figure 1: StreamSplit System Architecture. The framework comprises three coupled subsystems: an Edge Learner optimizing local throughput via virtual negatives, a Control Plane dynamically partitioning the network based on state constraints, and a Server Refiner maintaining manifold continuity via Laplacian regularization on graph G.
1
Introduction
Audio-capable edge devices, from smart speakers and wearables to mobile robots, offer immense potential for ambient intelligence [1, 2]. To realize this vision, these devices must learn from streaming data and adapt to complex acoustic environments, such as performing robust scene classification or source separation in smart cities [3, 4]. However, the state-of-the-art models for audio representation learning (e.g., CLAP [5], COLA [6], CLAR [7]) are largely confined to data centers. This centralization creates a critical bottleneck: transmitting continuous raw audio to the cloud is unsustainable regarding bandwidth [8, 9] and energy [10]. Furthermore, transmitting raw user audio raises data minimization concerns; processing data locally or transmitting only intermediate representations aligns better with data minimization principles, though it does not replace cryptographic guarantees. Edge computing offers a solution by moving computation closer to the source [11, 1]. Yet, simply porting server-grade training algorithms to the edge introduces fundamental conflicts. Contrastive Learning (CL)—the dominant paradigm for self-supervised audio learning—relies on large batches and diverse negative samples to learn high-quality features [12, 13, 14]. This requirement is inherently at odds with the limited memory and compute of client devices [15, 16]. Furthermore, the edge ecosystem is heterogeneous and volatile; a static deployment that works on a high-performance Apple M2 chip may drain the battery of a Raspberry Pi, and a partition optimized for strong WiFi may paralyze the application when the network degrades [17, 18].
MobiSys ’26, June 21–25, 2026, Cambridge, United Kingdom
Quan and Pathirana
To enable practical, streaming learning at the edge, we must energy reduction while maintaining accuracy within 2.2% of server address two distinct, coupled challenges: models. Challenge 1: The Stream-Clip Mismatch (Algorithmic). State-of-the-art audio models (e.g., CLAP) differ fundamentally 2 Background and Motivation from edge sensors: they are trained on shuffled, discrete clips (files), To understand the necessity of StreamSplit, we must examine the whereas edge devices perceive continuous streams. Naively chopfundamental conflicts that render current state-of-the-art approaches ping streams into small batches destroys the temporal coherence insufficient for the edge: the algorithmic dependency of contrastive required to learn robust features. We need a formulation that exlearning on massive data batches, and the systemic intolerance of ploits acoustic continuity rather than fighting it. split computing to resource volatility. Table 1 provides a systematic Challenge 2: The Volatility Conflict (Systems). The optimal critique of these paradigms. point to split a neural network between a device and a server is not static. It shifts constantly based on the device’s instantaneous CPU 2.1 Contrastive Learning: The Batch Size Barrier load (e.g., background tasks) and external network conditions. Static Contrastive Learning (CL) has established itself as the dominant partitioning schemes or "train-then-compress" approaches fail to paradigm for self-supervised audio representation, driving recent adapt [19, 20]. In our preliminary analysis, we observed that a static breakthroughs in ambient intelligence [1, 2]. State-of-the-art models split (Fixed Split Learning) suffered a 15.7% accuracy drop when like COLA [6] and CLAP [5] rely on maximizing agreement between subjected to realistic CPU load volatility, due to latency timeouts augmented views of data to learn robust features without human and dropped frames. labels. Existing paradigms isolate representation learning from sysDependency on Negative Diversity. The efficacy of CL is tem scheduling. Standard split computing relies blindly on system mathematically rooted in the quantity and diversity of negative telemetry [21], while small-batch continual learning demands large samples available during a gradient update. The standard InfoNCE memory queues that cause edge cache thrashing [22]. StreamSplit loss function [12] relies on the denominator to approximate the avoids this via Distributional Memory, synthesizing virtual negatrue data distribution. To maintain this approximation, leading tives from a compact Gaussian Mixture Model (GMM) to prevent frameworks like SimCLR [12] and MoCo [13] require massive batch dimensional collapse without large physical batches. Furthermore, sizes (often 𝑁 > 2048) or large memory queues. Without sufficient naively pairing heuristic schedulers with small-batch methods fails diversity, the gradients become biased, and the learning objective for continuous audio: dropping a semantically "hard" frame shatters degenerates [26]. the embedding manifold. StreamSplit overcomes this via algorithmThe Edge Incompatibility (C1). This requirement creates funsystem co-design [23]. By leveraging the GMM’s entropy as a lightdamental friction with edge hardware. Embedded devices, conweight, zero-cost uncertainty signal, our RL routing policy directly strained by strict memory budgets [15, 16], often restrict batch sizes couples algorithmic difficulty to system execution, intelligently to single digits (𝑁 = 8 or 16). Existing adaptations fall short: gradispending bandwidth only where it maximizes server utility. ent accumulation introduces prohibitive latency for streaming [27], In this paper, we introduce StreamSplit (Figure 1), a cohesive while small-batch training causes dimensional collapse where emframework to resolve conflicts. Unlike static compression techbeddings map to narrow cones [14]. StreamSplit addresses this by niques [23], StreamSplit co-designs the learning algorithm with the shifting from sample-based to distribution-based alignment (Section system architecture for robust, adaptive execution. 4), decoupling representation quality from local batch size. To address C1, we propose a distribution-based streaming framework. Instead of relying on large discrete batches, Stream2.2 Split Computing: The Volatility Gap Split aligns the distribution of edge embeddings with a global prior using a hybrid Sliced-Wasserstein and Laplacian loss [24, 25]. This To circumvent on-device resource limitations, Split Computing allows the edge device to contribute high-quality updates using partitions the neural network between the edge and cloud [11]. small, local batches, effectively decoupling representation quality While theoretically sound, existing implementations fail to address from on-device memory constraints. the dynamic nature of mobile environments. To address C2, we introduce an Uncertainty-Guided Adaptive Static Partitioning Deficiencies. The majority of existing Splitter. We formulate the splitting decision as a lightweight RL frameworks, such as split federated learning approaches [19, 29], problem. Our agent monitors system metrics (CPU, RAM, Network) determine the optimal split point offline or at initialization. They and—crucially—the uncertainty of the current audio embedding. assume a stable resource profile. However, edge environments are This allows StreamSplit to offload "hard" samples to the server while defined by volatility. A device’s available CPU cycles fluctuate processing "easy" samples locally, or to aggressively compress data rapidly due to background OS tasks or thermal throttling [10]. A when the network is congested. static split point that is optimal at 𝑡 = 0 often becomes a bottleneck We implement StreamSplit on heterogeneous ARM platforms at 𝑡 = 1. For instance, if the network degrades, a server-heavy split from the resource-constrained Raspberry Pi 4B to the high-performance causes stalls. If the CPU load spikes, an edge-heavy split causes Apple MacBook M2. Our contributions include: (1) the first framesystem-wide latency. work integrating distribution-based contrastive learning with RLThe Gap in Adaptive Control (C2). Recent adaptive approaches based system control for both convergence and stability; (2) platform(e.g., Rule-Based Split in Table 1) rely on reactive bandwidth threshagnostic deployment extending battery life by 50% on Pi 4 while olds [23], suffering from two flaws: (1) Lack of Lookahead—reacting leveraging NPU acceleration on M2; (3) 77.1% bandwidth and 52.3% only after degradation, causing dropped frames; (2) Optimization
StreamSplit: Continuous Audio Representation Learning via Uncertainty-Guided Adaptive Splitting
MobiSys ’26, June 21–25, 2026, Cambridge, United Kingdom
Table 1: Comprehensive Gap Analysis. Current paradigms fail to simultaneously satisfy the conflicting requirements of the edge. While heuristic methods attempt adaptation, they lack data awareness (Uncertainty), leading to inefficient offloading decisions. Paradigm
Methodology
Representative Works
Critical Failure Mode
Key Properties
Algorithm
Execution
Quality
Adaptive
Cloud-Centric Edge-Only Static Split Rule-Based Split
CLAP [5], COLA [6] EdgeNCE [28], MCUNet SplitFed [19], FedSL [20] RoofSplit [23], Neurosurgeon
Large-Batch Small-Batch Standard Standard
Full Offload On-Device Fixed Layer Resource-Aware
Privacy & Bandwidth Costs Dimensional Collapse Latency Timeouts (-15.7%) Optimization Blindness
✓ p ✓ ✓
– p p .
StreamSplit (Ours)
–
Distributional
Uncertainty RL
Robust (Co-Designed)
✓
✓
✓ = Fully Supported; p = Fails/Not Supported; . = Unreliable or Data-Agnostic (Partial Support); – = Not Applicable.
Definition 1 (Diversity). We define Diversity as the alignment between the marginal distribution of embeddings 𝑝𝜃 (𝑧) and a fixed, high-entropy prior distribution 𝑞(𝑧) (typically uniform on the hypersphere U (S𝑑 −1 )). D𝑑𝑖𝑣 = W2 (𝑝𝜃 (𝑧), U (S𝑑 −1 ))
(1)
where W2 denotes the Wasserstein distance. High diversity (minimizing D𝑑𝑖𝑣 ) implies that the entropy 𝐻 (𝑝𝜃 (𝑧)) is maximized. Definition Explanation. High diversity (minimizing D𝑑𝑖𝑣 ) ensures embeddings distribute uniformly across the hypersphere, maximizing discriminative capacity even with small local batches (Figure 2a). Theorem 3.1 (Small-Batch Robustness). Let 𝑝𝜃 denote the edge embedding distribution and U the uniform distribution on S𝑑 −1 . If 𝑝𝜃 has 𝜖-diversity (i.e., W1 (𝑝𝜃 , U) < 𝜖), then for batch size 𝑁 , the generalization gap between the empirical contrastive loss L𝑁 and the population loss L∞ is bounded by: 𝐶2 (2) |L𝑁 − L∞ | ≤ 𝐶 1𝜖 + √ 𝑁 where 𝐶 1, 𝐶 2 > 0 are constants depending on the critic function’s Blindness—treating all inputs equally despite varying difficulty. TaLipschitz constant. ble 1 demonstrates no existing paradigm satisfies continuous edge learning requirements, necessitating StreamSplit’s co-designed Uncertainty- This theorem implies that minimizing the distributional gap 𝜖 Guided framework. directly compensates for the bias induced by small batch sizes 𝑁 . Figure 2: Embedding Quality Metrics. (a) Diversity (L𝑆𝑊 ): Repulsive force preventing dimensional collapse by dispersing embeddings. (b) Affinity (L𝐿𝑎𝑝 ): Attractive force ensuring temporal coherence by pulling drifting frames (red) back to the manifold.
3
Our Quality Metrics: Affinity and Diversity
To systematically address the trade-off between representation quality and edge constraints, we propose tractable metrics that quantify the two essential attributes of a robust embedding space: Diversity and Affinity. We formalize these attributes using the geometry of the induced embedding distribution 𝑝𝜃 (𝑧). Our quantification process is illustrated in Figure 2. In the following subsections, we provide a detailed explanation of the theoretical framework, along with empirical evidence demonstrating the advantages of our distribution-based metrics.
(See Appendix A for proof). Metric: Sliced-Wasserstein Distance (SWD). A potential concern is that computing the full Wasserstein distance is computationally expensive (𝑂 (𝑑 3 )). To ensure tractability on edge devices, we employ the Sliced-Wasserstein Distance (SWD) as a scalable proxy (𝑂 (𝑀𝑑 log 𝑑)). By projecting the high-dimensional distributions 𝑀 , we compute onto a set of 𝑀 random unit vectors Ω = {𝜔𝑚 }𝑚=1 the closed-form solution: 𝑀 ∫ 2 1 ∑︁ 1 −1 −1 L𝑆𝑊 = 𝐹𝜃,𝜔𝑚 (𝜏) − 𝐹𝑞,𝜔 (𝜏) 𝑑𝜏 (3) 𝑚 𝑀 𝑚=1 0
Diversity: Ensuring Global Separation
where 𝐹 −1 is the inverse CDF. Minimizing L𝑆𝑊 forces the sorted projections of edge embeddings to match the uniform distribution, ensuring global coverage.
The "Small-Batch Conflict" (C1) primarily threatens the global structure of the latent space. Standard contrastive losses require large batches to approximate the partition function; without them, the model minimizes loss by mapping all inputs to a single point 𝑐, a phenomenon known as dimensional collapse [26].
Remark (Metric Equivalence). While Theorem 3.1 relies on the Wasserstein-1 distance W1 , our optimization minimizes the SlicedWasserstein distance L𝑆𝑊 . On the compact hypersphere S𝑑 −1 , it is established that L𝑆𝑊 is topologically equivalent to W𝑝 distances
3.1
MobiSys ’26, June 21–25, 2026, Cambridge, United Kingdom
[24]. Specifically, convergence in L𝑆𝑊 implies convergence in W1 , making Eq. 3 a tractable proxy for minimizing the theoretical error bound 𝜖.
3.2
Affinity: Preserving Local Structure
While Diversity ensures global space utilization, Affinity ensures the mapping preserves the temporal topology of the input signal. In the "Volatility Conflict" (C2), sparse updates can lead to a "jagged" manifold where temporally adjacent frames 𝑥𝑡 , 𝑥𝑡 +1 map to distant points 𝑧𝑡 , 𝑧𝑡 +1 . Motivation. For a Lipschitz-continuous encoder, small temporal shifts in the input should produce small shifts in the embedding space. Formally, if ||𝑥𝑡 − 𝑥𝑡 +1 || ≤ 𝛿, then a well-behaved encoder satisfies ||𝑓𝜃 (𝑥𝑡 ) − 𝑓𝜃 (𝑥𝑡 +1 )|| ≤ 𝐾𝛿 for some Lipschitz constant 𝐾. This motivates measuring smoothness directly in embedding space. Definition 2 (Affinity). Let G = (𝑉 , 𝐸,𝑊 ) be a temporal adjacency graph where vertices 𝑉 = {𝑧 1, . . . , 𝑧𝑇 } are embeddings and edges 𝐸 connect temporally adjacent frames with weights 𝑊𝑖 𝑗 . We define Affinity as the inverse of the Dirichlet energy (graph Laplacian quadratic form): 1 1 ∑︁ 𝑊𝑖 𝑗 ||𝑧𝑖 − 𝑧 𝑗 || 2 = Tr(Z⊤ LZ) (4) D𝑎𝑓 𝑓 = |𝐸| |𝐸| (𝑖,𝑗 ) ∈𝐸
where L = D − W is the graph Laplacian and D is the degree matrix. High affinity corresponds to low D𝑎𝑓 𝑓 , indicating smooth embeddings over the temporal graph. Definition Explanation (Acoustic Consistency). Unlike video (which has scene cuts) or images (which are independent), ambient audio is physically continuous. Sound sources do not teleport in feature space. Minimizing D𝑎𝑓 𝑓 enforces this physical inertia, ensuring the manifold remains smooth even when network dropouts cause the server to receive sparse updates. Theorem 3.2 (Temporal Interpolation). Let G be a connected temporal graph with spectral gap 𝜆2 > 0 (second smallest eigenvalue of L). If the embedding sequence has Dirichlet energy D𝑎𝑓 𝑓 ≤ 𝛼, then the mean squared error of reconstructing any missing frame 𝑧𝑡 ∗ via weighted neighbor averaging is bounded: 2𝛼 · |𝐸| E ||𝑧𝑡 ∗ − 𝑧ˆ𝑡 ∗ || 2 ≤ (5) 𝜆2 · |N (𝑡 ∗ )| Í where 𝑧ˆ𝑡 ∗ = | N (𝑡1 ∗ ) | 𝑗 ∈ N (𝑡 ∗ ) 𝑧 𝑗 is the neighbor average and N (𝑡 ∗ ) denotes the temporal neighbors of 𝑡 ∗ . This implies that a high-affinity representation (low 𝛼) provides intrinsic robustness to the volatility of edge execution. (See Appendix A for proof). Metric: Laplacian Regularization. We directly optimize the Dirichlet energy as the affinity loss: 1 ∑︁ L𝐿𝑎𝑝 = 𝑊𝑖 𝑗 ||𝑧𝑖 − 𝑧 𝑗 || 2 (6) |𝐸| (𝑖,𝑗 ) ∈𝐸
Minimizing L𝐿𝑎𝑝 explicitly penalizes "jagged" transitions, smoothing the manifold to maintain semantic coherence.
3.3
Metric Advantage
Given the fine granularity of our metrics, we empirically investigate their correlation with task performance. We conduct controlled
Quan and Pathirana
experiments on AudioSet-Balanced (20,371 samples, 527 classes) by systematically varying affinity and diversity. Diversity Validation. We degrade diversity by restricting the uniform prior to spherical cones of half-angle 𝜃 ranging from 10° to 90° in 10° increments. At 𝜃 = 10, embeddings are forced into a narrow cone (severe collapse); at 𝜃 = 90, the prior covers the full hypersphere. The results reveal a strong negative correlation between L𝑆𝑊 and downstream accuracy (Pearson 𝑟 = −0.96, 𝑝 < 0.001), significantly outperforming Maximum Mean Discrepancy (MMD, 𝑟 = 0.82, 𝑝 < 0.01) [30]. At 𝜃 = 10 (L𝑆𝑊 = 0.89), accuracy drops to 55.2%; at 𝜃 = 90 (L𝑆𝑊 = 0.08), accuracy reaches 73.1%. Affinity Validation. We inject temporal discontinuities by randomly shuffling frames within a 3-second window with probability 𝑝 ranging from 0.0 to 0.8 in 0.1 increments. The L𝐿𝑎𝑝 metric shows strong positive correlation with accuracy degradation (Pearson 𝑟 = 0.93, 𝑝 < 0.001). At 𝑝 = 0.8 (severe jitter, L𝐿𝑎𝑝 = 2.31), accuracy degrades by 12.4%; at 𝑝 = 0 (no jitter, L𝐿𝑎𝑝 = 0.35), accuracy is 72.8%. The spectral gap 𝜆2 decreases from 0.42 to 0.08 as jitter increases, confirming that temporal disruptions degrade manifold connectivity as predicted by Theorem 3.2. These results demonstrate that our distribution-based metrics are more sensitive to quality variations than existing approaches, making them reliable objectives for the StreamSplit optimization loop (Section 4).
4
StreamSplit Framework
StreamSplit transforms the traditionally static edge-cloud link into a dynamic, feedback-driven control loop. Unlike "train-then-compress" paradigms which fix model architecture at deployment time, StreamSplit treats the neural network as a flexible pipeline that can be partitioned, compressed, and scheduled in real-time. As illustrated in Figure 1, the framework is architected as an asymmetric distributed system spanning edge devices and the cloud. The figure explicitly maps out the data and control flow, detailing the key components and interaction pathways within three decoupled subsystems: the Edge Learner (Phase 1), which maximizes local throughput on the edge devices under memory constraints; the Control Plane (Phase 2), which governs adaptive offloading based on system state and data difficulty; and the Cloud Refiner (Phase 3), which enforces global consistency across the manifold despite asynchronous updates.
4.1
Phase 1: Edge-Side Streaming Execution
The primary operational constraint on the edge is memory volatility. Standard Contrastive Learning (CL) requires storing thousands of negative samples (e.g., in a Memory Bank or Large Batch) to approximate the global data distribution. On memory-constrained devices, maintaining large queues causes cache thrashing when background tasks compete for resources [22]. This raises a critical design question:
RQ1: How can we maintain high-quality contrastive representations on memory-constrained edge devices without requiring large negative sample batches?
StreamSplit: Continuous Audio Representation Learning via Uncertainty-Guided Adaptive Splitting
MobiSys ’26, June 21–25, 2026, Cambridge, United Kingdom
4.1.1 Analysis Method. We analyze the memory-quality trade-off throughout the contrastive learning process. Standard approaches store 𝑁 negative samples requiring 𝑂 (𝑁𝑑) memory. For typical settings (𝑁 = 4096, 𝑑 = 128), this consumes 512KB of RAM, causing cache thrashing on edge devices. We investigate whether a generative distribution can replace explicit sample storage while maintaining representation quality. 4.1.2 Distributional Memory Footprint. To resolve the “Small-Batch Conflict” (C1), we replace memory-intensive sample queues with a generative distribution. Instead of storing discrete raw tensors, the Edge Learner maintains a lightweight Gaussian Mixture Model (GMM): 𝐶 ∑︁ 𝑝𝑙𝑜𝑐𝑎𝑙 (𝑧) = 𝜋𝑐 N (𝑧; 𝜇𝑐 , Σ𝑐 ) (7) 𝑐=1
where 𝜋𝑐 are mixing weights, 𝜇𝑐 ∈ R𝑑 are component means, and Σ𝑐 are diagonal covariance matrices. With 𝐶 = 64 components and embedding dimension 𝑑 = 128, utilizing FP16 precision (2 Bytes), the storage requirement is: Size ≈ 2 · (𝐶 × 𝑑 × 2B) + (𝐶 × 2B) ≈ 33 KB
(8)
While larger than our initial estimate, this fits comfortably within the L1/L2 cache of standard ARM processors (e.g., Cortex-A72 has 1MB L2), whereas a standard contrastive memory bank (e.g., 16k samples) requires > 8MB of RAM and incurs high DRAM access energy. This GMM is updated incrementally using a streaming ExpectationMaximization (EM) algorithm [31] with 𝑂 (𝐶𝑑) complexity per iteration. To ensure stability during initialization (Cold Start), the system defaults to a conservative local policy for the first 50 frames to populate the sufficient statistics. 4.1.3 Hard Negative Mining via GMM. Merely having a distribution is insufficient; we must sample from it effectively. Standard smallbatch training fails because randomly sampled negatives are often "easy" (semantically distant), providing negligible gradients [12]. We implement Boundary-Aware Sampling, visualized in Figure 3. During the forward pass, given an anchor embedding 𝑧 + from component 𝑐 ∗ , we sample "virtual" negative embeddings 𝑧 − from other components weighted by proximity to the anchor: ||𝜇𝑐 ∗ − 𝜇𝑐 || 2 𝑝 (𝑐 |𝑧 +, 𝑐 ∗ ) ∝ 𝜋𝑐 · exp − , 𝑐 ≠ 𝑐∗ (9) 2𝜏 2 where 𝜏 is a temperature parameter controlling hardness. This strategy samples negatives near the decision boundary between components (hard negatives) rather than from distant components (easy negatives) or out-of-distribution regions. By synthesizing virtual negatives on-the-fly, we effectively augment the batch size without increasing physical memory usage, as virtual samples are generated, ℓ2 -normalized to project them onto the hypersphere, and immediately discarded after gradient computation. Edge Training Objective and Flow. To strictly minimize ondevice memory, we employ a streaming InfoNCE loss with virtual negatives. For each incoming audio frame 𝑥𝑡 , we generate a positive pair (𝑥˜𝑡 , 𝑥˜𝑡′ ) via standard lightweight augmentations (random Gaussian noise and frequency masking). We do not use temporal neighbors as positives to avoid buffering latency.
Figure 3: Addressing Small-Batch Conflict. (a) Small batches cause “Lazy Margins” (𝜃𝑙𝑎𝑧𝑦 ) and degradation. (b) StreamSplit generates virtual negatives from a compact GMM (<35KB), enforcing “Tight Margins” (𝜃 𝑡𝑖𝑔ℎ𝑡 ) to decouple quality from physical batch size.
The edge encoder 𝑓𝜃 minimizes the negative log-likelihood of the positive pair against 𝑁𝑠𝑦𝑛 virtual negatives sampled from the GMM: exp(sim(𝑧𝑡 , 𝑧𝑡′ )/𝜏) Í𝑁𝑠𝑦𝑛 𝑗 exp(sim(𝑧𝑡 , 𝑧𝑠𝑦𝑛 exp(sim(𝑧𝑡 , 𝑧𝑡′ )/𝜏) + 𝑗=1 )/𝜏) (10) 𝑁𝑠𝑦𝑛 𝑗 where 𝑧𝑡 = 𝑓𝜃 (𝑥˜𝑡 ), 𝑧𝑡′ = 𝑓𝜃 (𝑥˜𝑡′ ), and {𝑧𝑠𝑦𝑛 } 𝑗=1 are virtual negatives sampled from the local GMM 𝑝𝑙𝑜𝑐𝑎𝑙 (𝑧) using the boundary-aware strategy (Eq. 9). Training Flow. This objective allows for a fully streaming update: L𝑒𝑑𝑔𝑒 = − log
(1) Forward: Encoder processes 𝑥𝑡 to get 𝑧𝑡 ; GMM is updated via online EM. (2) Sample: Virtual negatives 𝑧𝑠𝑦𝑛 are generated from GMM parameters. (3) Backward: Gradients from L𝑒𝑑𝑔𝑒 update the encoder weights 𝑓𝜃 . (4) Sync: Updated weights are lazily synchronized with the server (See Sec. 4-Phase 3) to align with the global manifold.
Answer to RQ1: Distributional memory replaces explicit sample storage with a compact GMM (𝑂 (𝐶𝑑) vs. 𝑂 (𝑁𝑑) memory), while boundary-aware sampling synthesizes hard negatives on-the-fly. This decouples representation quality from physical batch size, enabling edge devices to maintain contrastive learning performance despite memory constraints.
4.2
Phase 2: Uncertainty-Guided Control
The core systems innovation of StreamSplit is the Control Plane, which resolves the "Volatility Conflict" (C2). Unlike heuristic splitters (e.g., Neurosurgeon [21]) that react only to bandwidth changes, our system must proactively manage workload based on both system state and data difficulty. This motivates our second research question:
MobiSys ’26, June 21–25, 2026, Cambridge, United Kingdom
Quan and Pathirana
RQ2: How can we dynamically adapt the edge-cloud computation partition to handle runtime volatility in CPU load, network bandwidth, and data difficulty? 4.2.1 Analysis Method. We analyze the runtime adaptation problem across the entire execution pipeline, including edge compute constraints, network transmission costs, and varying data complexity. Existing heuristic methods fail when multiple constraints conflict (e.g., high CPU load but low bandwidth). We investigate whether a learned policy can discover non-obvious adaptation strategies that balance competing objectives. 4.2.2 RL Formulation. We formulate the runtime adaptation as a Markov Decision Process (MDP) defined by the tuple (S, A, R), with formal definitions provided in Appendix B. The Control Plane runs as a lightweight sidecar process on the edge device, executing policy inference at regular intervals to avoid excessive overhead. State Space (𝑠𝑡 ): The state vector 𝑠𝑡 = [𝑈𝑡 , 𝑅𝑐𝑝𝑢 , 𝐵𝑛𝑒𝑡 ] captures the full system context. 𝑅𝑐𝑝𝑢 ∈ [0, 100] represents CPU utilization percentage. 𝐵𝑛𝑒𝑡 ∈ R+ is the estimated uplink bandwidth, computed via exponential moving average over recent transmission times. Crucially, we include 𝑈𝑡 ∈ [0, log 𝐶], the embedding uncertainty, defined as the Shannon entropy of the GMM component assignment: 𝑈𝑡 = 𝐻 (𝑝 (𝑐 |𝑧𝑡 )) = −
𝐶 ∑︁ 𝑐=1
𝜋 N (𝑧 ;𝜇 ,Σ )
𝑝 (𝑐 |𝑧𝑡 ) log 𝑝 (𝑐 |𝑧𝑡 )
(11)
Figure 4: RL Control Logic. The agent maps normalized state s𝑡 (incl. uncertainty) to split layer 𝑘𝑡 . Reward 𝑟𝑡 (Eq. 12) balances accuracy, latency, and energy.
Decision Frequency and Atomic Transitions: The RL agent executes every 𝑇𝑠𝑡𝑒𝑝 = 10 frames (≈ 100ms). The policy network (2-layer MLP) inference time is measured at 0.2ms on the Pi 4B. To guarantee consistency during transitions and avoid the complexity of managing in-flight data, split decisions occur atomically at these 100ms boundaries. When the RL agent shifts the computation partition (e.g., moving from layer 3 to layer 5), the transition applies exclusively to the next 𝑇𝑠𝑡𝑒𝑝 block. No data currently traversing the pipeline is redone, and no in-flight frames are dropped, stalled, or left in an inconsistent state during the shift. This high-frequency, atomic control allows StreamSplit to react to abrupt bandwidth collapses without corrupting the streaming pipeline.
𝑡 𝑐 𝑐 where 𝑝 (𝑐 |𝑧𝑡 ) = Í ′ 𝜋𝑐 ′ N (𝑧 is the posterior probability via 𝑡 ;𝜇𝑐 ′ ,Σ𝑐 ′ ) 𝑐 𝑐 Bayes’ rule. High entropy typically correlates with transient acoustic events (e.g., speech onset, glass breaking) which are semantically complex. Low entropy corresponds to steady-state background noise (e.g., HVAC hum). By conditioning the split on 𝑈𝑡 , the agent effectively learns a semantic Event Detection policy, offloading only the information-dense frames. We prefer GMM entropy over TTA variance (high latency) or kNN density (high memory) as it pro4.2.3 Policy Network and Adaptive Behavior. The control logic is vides a zero-cost uncertainty proxy computed directly within the visualized in Figure 4. We employ a Proximal Policy Optimization standard inference pass. Action Space (𝑎𝑡 ): The action is the split layer index 𝑘 ∈ {0, 1, . . . , 𝐿}, (PPO) [32] agent with a lightweight two-layer fully connected network. The policy network 𝜋𝜃 (𝑎|𝑠) and value network 𝑉𝜙 (𝑠) share where 𝐿 is the total number of encoder layers. Selecting 𝑘 = 0 imthe first layer to minimize parameter overhead while maintaining plies full offloading (minimal edge compute, maximum bandwidth expressiveness. consumption), while 𝑘 = 𝐿 implies full on-device processing (maxThe agent is trained offline on historical traces collected from diimum edge compute, zero bandwidth). Note that to satisfy strict verse hardware platforms under varying network conditions (bandbandwidth constraints detected in the state 𝑠𝑡 , specific offloading width, latency, packet loss). The learned policy exhibits complex actions (𝑘 < 𝐿) are coupled with dynamic quantization (e.g., INT8 adaptive behaviors that heuristic rules cannot capture. For instance, precision) to ensure the transmission fits within the estimated 𝑇𝑚𝑎𝑥 . under high CPU load but low bandwidth, a heuristic might stall. The Reward Function (𝑟𝑡 ): The reward balances task performance StreamSplit agent learns to apply aggressive quantization (reducwith system efficiency: ing embedding precision) to fit bandwidth constraints, sacrificing Lat𝑡 𝐸𝑡 marginal accuracy to maintain real-time throughput. Conversely, 𝑟𝑡 = 𝛼 · A𝑡𝑎𝑠𝑘 − 𝛽 · −𝜂 · (12) 𝑇𝑚𝑎𝑥 𝐸𝑏𝑢𝑑𝑔𝑒𝑡 when both CPU and bandwidth are available but uncertainty 𝑈𝑡 where A𝑡𝑎𝑠𝑘 ∈ [0, 1] is the task accuracy on a validation set, Lat𝑡 is high (ambiguous sample), the agent offloads to the server even is the end-to-end latency (edge compute + network transmission at minimal split depth, prioritizing accuracy over efficiency. We + server compute) normalized by a maximum allowable latency note that while the training reward 𝑟𝑡 uses ground-truth accuracy 𝑇𝑚𝑎𝑥 , and 𝐸𝑡 is the energy consumption normalized by a per-frame A𝑡𝑎𝑠𝑘 , the deployed policy 𝜋𝜃 relies solely on the observable state budget 𝐸𝑏𝑢𝑑𝑔𝑒𝑡 . The coefficients 𝛼, 𝛽, 𝜂 prioritize accuracy while 𝑠𝑡 (uncertainty, CPU, bandwidth) and does not require labels at penalizing latency violations and energy drain. runtime.
StreamSplit: Continuous Audio Representation Learning via Uncertainty-Guided Adaptive Splitting
MobiSys ’26, June 21–25, 2026, Cambridge, United Kingdom
Answer to RQ2: Formulate adaptation as an RL problem where the agent observes CPU load, network bandwidth, and embedding uncertainty to select optimal split points. The learned policy discovers non-obvious strategies (e.g., quantization under bandwidth constraints, uncertainty-based offloading) that heuristic rules miss, achieving robust real-time execution across heterogeneous platforms.
4.3
Phase 3: Server-Side Refinement
The Cloud Server handles the heavy lifting of global model convergence. However, dynamic splitting introduces asynchrony: frames arrive out of order or are dropped entirely when the network degrades or the RL agent selects full on-device processing to conserve bandwidth. This breaks standard synchronous SGD, which assumes uniform batches and temporal ordering. To address this, the Server Refiner implements a Temporal Buffer and the Hybrid Loss derived in Section 3. 4.3.1 Temporal Buffer Management. As shown in Figure 5, the server maintains a sliding temporal window of recent embeddings. When an update 𝑧𝑡 arrives from the edge, it is inserted into the buffer at the corresponding temporal index. If a frame is skipped (due to RLdriven local processing or network packet loss), the buffer contains a temporal gap. Rather than discarding the buffer or performing explicit interpolation (which would require maintaining expensive historical state), we construct a k-Nearest Neighbor temporal graph G = (𝑉 , 𝐸) on the available embeddings, where edges (𝑖, 𝑗) ∈ 𝐸 connect temporally adjacent frames within a small window. We set the server temporal window size to 𝑊 = 100 frames (approx. 1s context). The 𝑘-NN graph is constructed with 𝑘 = 5 neighbors. Constructing the Laplacian on the server takes approx. 3ms for a batch of 100, which is fully pipelined with inference to minimize overhead. 4.3.2 Hybrid Objective for Robust Convergence. The server optimizes the combined objective from Eq. 13: L𝑠𝑒𝑟 𝑣𝑒𝑟 =
L𝑡𝑎𝑠𝑘 |{z} Classification
+𝜆1 L𝑆𝑊 (𝑝𝜃 , U) +𝜆2 L𝐿𝑎𝑝 (G) | {z } | {z } Diversity
(13)
Affinity
where L𝑡𝑎𝑠𝑘 denotes the global objective. In pure self-supervised settings, L𝑡𝑎𝑠𝑘 is formulated as a global InfoNCE loss over the server buffer; when sparse labels are available (as in our validation experiments to establish performance upper bounds), it is instantiated as the standard cross-entropy loss. L𝑆𝑊 is the Sliced-Wasserstein Distance (Eq. 3) enforcing global diversity, and L𝐿𝑎𝑝 is the Laplacian regularization (from Section 3) enforcing temporal smoothness: 1 ∑︁ L𝐿𝑎𝑝 = ||𝑧𝑖 − 𝑧 𝑗 || 2 (14) |𝐸| (𝑖,𝑗 ) ∈𝐸
The Laplacian term penalizes large embedding jumps between temporally adjacent frames. This mechanism is critical for handling extended periods of connectivity loss. During a severe network outage, the edge device may be forced to drop frames if local memory is exhausted. When connectivity resumes, the server’s Temporal Buffer receives a sequence with a massive temporal gap. Instead of naively interpolating missing features—which would hallucinate
Figure 5: Manifold Stitching. Laplacian regularization (L𝐿𝑎𝑝 ) acts as a spring force across temporal gaps (missing frames), maintaining manifold continuity without requiring explicit interpolation.
phantom acoustic events—L𝐿𝑎𝑝 enforces a Lipschitz-continuous smoothing prior across the boundary of the outage. As formalized by Theorem 3.2, minimizing the Dirichlet energy guarantees bounded interpolation error even when the spectral gap (𝜆2 ) of the temporal graph is degraded by dropouts. The server effectively "stitches" the representations together by pulling the pre-outage and post-outage embeddings toward a shared, smooth manifold geometry. Because the representations remain Lipschitzcontinuous, the server can bridge these temporal gaps gracefully. This is empirically validated in our ablation studies (Table 5), where StreamSplit sustains a severe 40% frame drop rate while only degrading 6.6% in accuracy, whereas standard contrastive objectives suffer catastrophic collapse. Simultaneously, the Sliced-Wasserstein term prevents mode collapse by ensuring the aggregated buffer distribution matches the global uniform prior. This is particularly important in asynchronous federated settings, where devices with fast connections might dominate the global model. By minimizing L𝑆𝑊 , we ensure that the server model converges to a balanced representation of the entire data distribution, rather than overfitting to high-throughput devices. 4.3.3 Downlink Synchronization Strategy. To close the learning loop without incurring prohibitive downlink costs, we employ a Lazy Synchronization protocol. The Server transmits updated GMM parameters (𝜇𝑐 , Σ𝑐 )—which are lightweight (< 35KB)—to the Edge every 𝑇𝑠𝑦𝑛𝑐 = 100 frames. The heavier Encoder weights (𝑓𝜃 ) are only synchronized when the device detects a charging state or high-bandwidth WiFi connection. In our energy evaluation (Table 2), we account for the GMM synchronization cost, which adds a negligible 0.4 mJ/frame average overhead.
5
Implementation
Definitions. We define a Sample as a 1-second audio clip. A Batch consists of 𝑁 = 8 samples. A Frame refers to the STFT window (25ms). The RL agent makes decisions every 𝑇𝑠𝑡𝑒𝑝 = 10 frames (≈ 100ms) to ensure responsiveness to network volatility, while gradients are aggregated at the Batch level (𝑁 = 8). We implement the StreamSplit framework entirely in Python, using PyTorch [33] for model training and inference across all components. Edge deployment leverages PyTorch’s TorchScript for optimized execution on resource-constrained devices.
MobiSys ’26, June 21–25, 2026, Cambridge, United Kingdom
Audio Processing. For real-time spectrogram extraction, we employ PyKissFFT [34], a Python binding for the lightweight KissFFT library optimized for embedded platforms. Compared to NumPy’s FFT implementation, PyKissFFT reduces STFT computation latency from 7.8ms to 3.2ms per frame on Raspberry Pi 4B, enabling continuous 16kHz audio streaming without frame drops. We compute 128-bin mel spectrograms using 25ms windows with 10ms hop length. Hardware Platforms. We deploy and evaluate StreamSplit on three representative platforms spanning the edge-cloud spectrum: • Raspberry Pi 4B (4GB RAM, ARM Cortex-A72 @ 1.5GHz): Represents resource-constrained IoT devices. The GMM module (35KB) and PPO policy (12KB) fit entirely within the 256KB L2 cache. • Apple MacBook M2 (8GB unified memory, 8-core CPU): Represents capable edge devices with integrated neural engine. Used for development and mid-tier deployment scenarios. We leverage the PyTorch MPS (Metal Performance Shaders) backend to offload matrix operations to the M2’s 10-core GPU, achieving a 3.2× speedup over CPU execution. While the Neural Engine (ANE) offers further theoretical gains, current framework support favors the GPU for custom dynamic graphs like ours. • Cloud Server (Intel Xeon Gold 6248R, 4× NVIDIA RTX 3090): Hosts the Server Refiner for global model aggregation and refinement. Network Emulation. To evaluate adaptation under realistic network conditions, we employ the Linux Traffic Control (tc) utility as a network emulator, simulating bandwidth fluctuations (1–50 Mbps), latency variations (20–200ms), and packet loss (0–5%). We construct 6 network profiles based on real-world 4G/5G traces [35], spanning stable, variable, and congested scenarios. Training Configuration. Edge-side training uses the Adam optimizer with a learning rate of 10−3 and batch size 𝑁 = 8. The PPO agent trains for 2M environment steps with a discount factor 𝛾 = 0.99 and clipping 𝜖 = 0.2. Server-side training employs SGD with momentum 0.9 over 100 epochs. To find a balance between accuracy and strict penalties for latency and energy use, we use a lightweight 2-layer MLP for the RL policy with specific reward weights: 𝛼 = 10, 𝛽 = 5, and 𝜂 = 3. Loss hyperparameters (𝜆1 = 0.1, 𝜆2 = 0.01) were determined via grid search on held-out validation data. Reproducibility Details. The audio encoder backbone is a standard ResNet-18 adapted for 1D audio (taking spectrogram inputs), consisting of 𝐿 = 8 splitable blocks with an output dimension 𝑑 = 128, which provides the standard latent size for audio representation stability. For the Distributional Memory, we utilize a GMM with 𝐶 = 64 components (𝜏 = 0.1) to find the optimal balance between expressiveness and cache fit. Crucially, this estimator only takes up 33 KB of space, allowing it to fit completely in the L2 cache of standard ARM processors. This avoids having to access DRAM, which uses a lot of power. During hard negative mining, we sample 𝑁𝑠𝑦𝑛 = 256 virtual negatives per anchor to ensure a sufficient gradient signal without physical memory overhead. Execution Consistency. To address in-flight data and ensure consistency during dynamic transitions, our framework makes split
Quan and Pathirana
decisions atomically at 100ms boundaries (𝑇𝑠𝑡𝑒𝑝 = 10 frames) to ensure real-time response to network volatility. This means that split-point shifts happen precisely between frames, ensuring that no data is redone and there is no lag in flight. SWD and Sampling Overhead. We set the number of random projections for the Sliced-Wasserstein Distance to 𝑀 = 50 on both the edge and server to balance approximation error with compute cost. We measured the specific runtime overhead on the Raspberry Pi 4B: the SWD computation introduces a latency of 1.2ms per batch, while the GMM-based hard negative synthesis (generating 𝑁𝑠𝑦𝑛 = 256 virtual negatives) adds only 0.8ms per batch. These overheads are negligible compared to the 10ms hop length, preserving realtime capabilities. Quantization Implementation. To minimize transmission latency during offloading (𝑘 < 𝐿), we compress intermediate feature maps using asymmetric INT8 quantization with a pertensor granularity. We employ Post-Training Quantization (PTQ) rather than Quantization-Aware Training (QAT) to maintain training pipeline simplicity. Calibration statistics (min/max ranges) are collected offline using a representative subset of the training data. The runtime overhead for quantization and dequantization on the Raspberry Pi 4B is measured at < 0.5ms per frame. Empirical evaluation shows this compression introduces a negligible accuracy degradation of < 0.3% compared to full FP32 transmission, validating its viability for bandwidth-constrained edges.
6
Evaluation
We conduct extensive experiments to evaluate StreamSplit across diverse datasets, hardware platforms, and network conditions through systematic evaluation of system efficiency, representation quality, component contributions, and hardware generalization.
6.1
Experimental Setup
6.1.1 Datasets. We evaluate StreamSplit on two complementary datasets: AudioSet [36] is a large-scale audio classification benchmark containing over 2 million human-labeled 10-second clips across 527 classes. We use the balanced evaluation subset (20,371 clips) for standardized comparison with prior work, and a subset of 100K clips for training the contrastive encoder. EcoStream-Wild comprises 48 hours of continuous uncurated audio collected using 4 Raspberry Pi 4B devices deployed in diverse real-world environments: a university office, a crowded cafeteria, a residential living room, and an outdoor balcony, recording over a 2-week period. To establish ground truth, we employed a semi-automated annotation protocol: a silence removal algorithm first proposed candidate segments, which were then manually verified and labeled by human annotators. The dataset features a highly unbalanced, realistic class distribution dominated by “Silence/Background” (60.2%), followed by “Speech” (24.5%), and specific “Transient Events” (15.3%, e.g., footsteps, door slams, glass breaking) across 15 distinct classes. We commit to releasing the full dataset, annotation logs, and source code upon acceptance to facilitate reproducibility. 6.1.2 Baseline Methods. We compare StreamSplit against the following representative baselines:
StreamSplit: Continuous Audio Representation Learning via Uncertainty-Guided Adaptive Splitting
Edge-Only: Full model training and inference on the edge device without server communication, representing maximum autonomy but limited model capacity due to memory constraints. Server-Only: All audio data transmitted to the cloud server for processing, representing maximum accuracy but high bandwidth consumption and latency. FSL [19]: Federated Split Learning with fixed partition points, combining federated averaging with split inference. FedCL [37]: Federated Contrastive Learning using synchronized memory banks across clients with periodic server aggregation. Rule-Based Adaptive Split: A heuristic baseline that offloads based on static thresholds for both bandwidth and CPU utilization (e.g., “Offload if Bandwidth > 𝑋 AND CPU < 𝑌 ”). This represents a competent engineering solution (an upgrade to Neurosurgeon [21]) without learning. Note that we exclude early-exit inference methods (e.g., SPINN [38]) as baselines, as they optimize for classification confidence rather than the representation quality required for Contrastive Learning.
MobiSys ’26, June 21–25, 2026, Cambridge, United Kingdom (a) Absolute Bandwidth
58.7
StreamSplit
124.3
201.4
FedCL
256.0
6.2
End-to-End System Performance
We first evaluate StreamSplit’s system-level efficiency gains—the critical metrics for practical edge deployment. 6.2.1 Bandwidth Reduction. Figure 6 presents bandwidth consumption across methods. Note that the results report the transmitted payload per processing batch of 𝑁 = 8 clips (approx. 8 seconds of audio). For the Server-Only baseline, transmitting 8 clips of raw PCM audio (16kHz, 16-bit mono) consumes 8 × 32KB = 256KB. StreamSplit achieves 77.1% bandwidth reduction (58.7 KB/batch) by transmitting compact intermediate representations. Comparison with Opus Baseline. We analytically compare against standard Opus compression (VoIP profile at 32kbps). While Opus offers a competitive bandwidth baseline (consuming ≈ 4 KB/frame, comparable to StreamSplit’s most aggressive modes), it introduces a fundamental trade-off that contradicts edge offloading: it forces the cloud server to execute the entire feature extraction pipeline (100% of the compute load). StreamSplit offers a Paretooptimal frontier: we match Opus bandwidth efficiency when necessary, but typically offload 35% of the compute burden to the edge (Fig. 7) to reduce server latency and load, a benefit that pure audio compression cannot achieve. Compared to baselines, FSL (187.2 KB/frame) and Rule-Based Split (124.3 KB/frame) achieve only 26.9% and 51.4% reduction respectively, as they lack data-aware adaptation. FedCL (201.4 KB/frame) incurs additional overhead from memory bank synchronization.
150
200
26.9%
FSL
Server-Only 100
51.4%
Heuristic 187.2
FSL
50
77.1%
StreamSplit
Heuristic
0
100.0%
Edge-Only
21.3%
FedCL 0%
Server-Only
250
0
Bandwidth (KB/frame)
20
40
60
80
100
Reduction vs. Server-Only (%)
Figure 6: Bandwidth Consumption. StreamSplit achieves 77.1% reduction vs. Server-Only by transmitting compact, adaptive intermediate representations.
(a) Stable Network (20 Mbps) Edge Compute Transmission Server Inference
156ms
Edge-Only
127ms
StreamSplit
183ms
HeuristicSplit
6.1.3 Evaluation Metrics. We evaluate system performance along three dimensions: System Efficiency: Bandwidth consumption (KB/frame), endto-end latency (ms), and energy consumption (mJ/frame) measured using PowerJoular [39] on Raspberry Pi 4B. Representation Quality: Linear probe accuracy (%) on frozen embeddings, and mean Average Precision (mAP) for audio retrieval tasks. Adaptation: Response time to network changes (ms) and accuracy stability under varying conditions (standard deviation).
(b) Bandwidth Reduction
0
Edge-Only
100
200
678ms
300
1042ms
400
Latency (ms)
1186ms
FedCL 464ms
Server-Only 0
287ms
FSL
281ms
FedCL
156ms
StreamSplit HeuristicSplit
248ms
FSL
(b) Congested Network (2 Mbps) Edge-Only
500
1847ms
Server-Only 0
500
1000
1500
2000
Latency (ms)
Figure 7: Latency Breakdown. StreamSplit reduces latency by 72.6% (stable) to 84.5% (congested) via adaptive splitting.
6.2.2 Latency Reduction. Figure 7 shows end-to-end latency breakdown across methods. StreamSplit achieves 72.6% latency reduction compared to Server-Only (127ms vs. 464ms average). The latency composition reveals the source of gains: StreamSplit spends 45.2ms on edge compute (35.6%)—which explicitly includes the < 2ms overhead for GMM updates and RL policy inference—34.6ms on transmission (27.2%), and 41.2ms on server inference (32.4%). In contrast, Server-Only spends 312ms on transmission alone (67.2%) due to raw audio upload. Under degraded network conditions (2 Mbps, 150ms RTT), StreamSplit’s advantage amplifies: 287ms vs. 1,847ms for Server-Only (84.5% reduction). The RL agent automatically increases local processing, reducing transmission dependency. 6.2.3 Energy Savings and Battery Life. Table 2 summarizes energy consumption on Raspberry Pi 4B powered by a 10,000mAh battery pack. StreamSplit consumes 89.3 mJ/frame, achieving 52.3% energy savings compared to Server-Only (187.2 mJ/frame). The savings stem from reduced radio transmission, which dominates energy consumption on battery-powered devices. Total energy consumption includes the uplink transmission of embeddings, local computation, and the periodic downlink synchronization of GMM parameters (from §5). Translated to battery life, StreamSplit enables 11.2 hours of continuous operation compared to 5.3 hours for Server-Only—a 2.1× extension critical for untethered deployment. Edge-Only achieves longer battery life (14.8 hours) but at significant accuracy cost (see §6.3).
MobiSys ’26, June 21–25, 2026, Cambridge, United Kingdom
Quan and Pathirana
Table 2: Energy Consumption and Battery Life on Raspberry Pi 4B with 10,000mAh battery. Results report mean ± std dev over 5 runs. Method
Compute (mJ)
Transmit (mJ)
Total (mJ)
Battery Life (hours)
Edge-Only Server-Only FSL FedCL Rule-Based Split
67.4 ± 1.2 12.4 ± 0.5 48.7 ± 2.1 52.1 ± 1.9 54.2 ± 1.5
0.0 174.8 ± 8.2 98.3 ± 5.4 112.6 ± 6.0 87.1 ± 4.8
67.4 ± 1.2 187.2 ± 8.5 147.0 ± 6.1 164.7 ± 6.8 141.3 ± 5.1
14.8 ± 0.3 5.3 ± 0.2 6.8 ± 0.3 6.1 ± 0.2 7.1 ± 0.3
StreamSplit
54.7 ± 2.3
34.6 ± 3.1
89.3 ± 4.1
11.2 ± 0.5
(a) AudioSet
(b) On-Device Dataset
58.6%
Edge-Only
HeuristicSplit
72.1%
FSL
71.5%
66.4%
FSL
68.7%
FedCL
50
60
70
78.1%
Server-Only 80
Accuracy (%)
60
70
80
Accuracy (%)
Figure 8: Linear Probe Accuracy. StreamSplit matches ServerOnly accuracy within 2.2% across datasets.
6.3
Representation Quality
A critical question is whether StreamSplit’s efficiency gains come at the cost of representation quality. We evaluate learned embeddings on downstream tasks to verify that accuracy remains competitive. 6.3.1 Linear Probe Accuracy. Figure 8 presents linear probe accuracy on frozen embeddings across both datasets. StreamSplit achieves 71.8% on AudioSet and 76.4% on the On-Device dataset, falling within 2.2% of Server-Only (73.6% and 78.1% respectively). This marginal gap demonstrates that our distributional memory and hybrid loss effectively preserve representation quality despite the constraints of edge deployment. Compared to other edge-compatible methods, StreamSplit outperforms Edge-Only by 13.2% (AudioSet) and 11.8% (On-Device), FSL by 5.4% and 4.9%, and FedCL by 3.1% and 2.7%. Rule-Based Split achieves 68.2% and 72.1%, suffering from suboptimal split decisions that sacrifice quality for bandwidth savings. 6.3.2 Retrieval Performance. Table 3 presents audio retrieval metrics (mAP@10, R@1) using learned embeddings. StreamSplit achieves 0.412 mAP@10 and 38.7% R@1, compared to Server-Only’s 0.431 and 40.2%—a gap of only 4.4% and 3.7% respectively. These results confirm that StreamSplit embeddings capture semantic similarity effectively, enabling downstream applications like audio search and similarity-based clustering.
6.4
mAP@10
R@1 (%)
Edge-Only Server-Only FSL FedCL Rule-Based Split
0.287 0.431 0.356 0.378 0.341
26.4 40.2 33.1 35.8 31.9
StreamSplit
0.412
38.7
Table 4: Adaptation Strategy Comparison. RL-based control outperforms alternatives in both steady-state and adaptation scenarios. (Mean ± std dev). Strategy
Accuracy (%)
Latency (ms)
Energy (mJ)
Adaptation Time (ms)
Static (𝑘 = 3) Rule-Based RL (Ours)
68.7 ± 0.2 69.4 ± 0.3 71.8 ± 0.4
203 ± 15 156 ± 10 127 ± 12
142.6 ± 5.5 118.7 ± 4.2 89.3 ± 4.1
N/A 4, 200 ± 350 1, 200 ± 150
73.7%
FedCL 73.6%
Server-Only
Method
76.4%
StreamSplit
68.2%
HeuristicSplit
64.6%
Edge-Only 71.8%
StreamSplit
Table 3: Audio Retrieval Performance on AudioSet. StreamSplit maintains competitive retrieval quality with minimal degradation.
Ablation Studies: Why It Works
We conduct ablation studies to isolate the contribution of each StreamSplit component and understand the source of performance gains.
Table 5: Loss Function Comparison under varying frame drop rates. Hybrid loss maintains robustness while alternatives degrade significantly. (Mean ± std dev). Loss Function
0% Drop
20% Drop
40% Drop
MSE Only KL Divergence L𝑡𝑎𝑠𝑘 + L𝑆𝑊 L𝑡𝑎𝑠𝑘 + L𝐿𝑎𝑝 Hybrid (Ours)
69.2 ± 0.3 70.1 ± 0.3 70.8 ± 0.4 70.4 ± 0.3 71.8 ± 0.4
61.4 ± 1.2 63.7 ± 1.1 67.2 ± 0.8 66.8 ± 0.9 69.4 ± 0.5
52.8 ± 2.5 55.1 ± 2.1 61.3 ± 1.5 60.7 ± 1.4 65.2 ± 0.9
6.4.1 RL Control vs. Alternatives. Table 4 compares StreamSplit’s RL-based control against alternative adaptation strategies. We evaluate three configurations: • Static: Fixed split point (𝑘 = 3) regardless of conditions. • Rule-Based: Heuristic adaptation using both bandwidth and CPU thresholds. • RL (Ours): Learned policy considering CPU, bandwidth, and uncertainty. The RL policy achieves 2.4% higher accuracy, 18.6% lower latency, and 24.8% lower energy than the heuristic baseline. Critically, adaptation time—the delay before the system responds to network changes—is 3.5× faster (1,200ms vs. 4,200ms). The heuristic requires multiple probe transmissions to estimate new bandwidth, while the RL agent adapts within a single decision interval using its learned state representation. 6.4.2 Hybrid Loss vs. Alternatives. Table 5 evaluates the contribution of our hybrid loss components under varying frame drop rates (simulating network volatility). Under ideal conditions (0% drop), all losses perform comparably. However, as frame drop rate increases, the hybrid loss demonstrates superior robustness: at 40% drop rate, it maintains 65.2% accuracy compared to 52.8% for MSE and 55.1% for KL divergence—a 12.4%
StreamSplit: Continuous Audio Representation Learning via Uncertainty-Guided Adaptive Splitting
Table 6: Cross-Platform Performance. StreamSplit adapts automatically to hardware capabilities, achieving optimal performance on both constrained (Pi 4B) and capable (M2) devices. Results report the mean ± standard deviation over 5 independent runs.
(b) Failure Analysis Edge-Only
Error Rate (%)
Server Loss Reduction
60 1
0.5
StreamSplit
40
20
0
Raspberry Pi 4B Value vs. Server
0 0.5
Local Uncertainty (𝑈𝑡 )
1
Low
Med
High
Figure 9: Uncertainty Analysis. (a) Local uncertainty correlates (𝑟 = 0.84) with server utility. (b) Acoustic Event Adaptation. High uncertainty (𝑈𝑡 ) regions correspond to transient audio events (e.g., speech). StreamSplit automatically offloads these complex frames while processing steady-state background noise locally.
and 10.1% advantage respectively. The Sliced-Wasserstein term (L𝑆𝑊 ) prevents mode collapse from sparse updates, while Laplacian smoothness (L𝐿𝑎𝑝 ) maintains manifold continuity across gaps. 6.4.3 Uncertainty Calibration. A critical concern is whether local uncertainty (𝑈𝑡 ) is a reliable proxy for global sample difficulty. We analyzed the correlation between edge-calculated entropy and the reduction in loss achieved by server processing. Figure 9 shows a strong positive correlation (Pearson 𝑟 = 0.84). This confirms that when the edge model is uncertain, the sample is indeed ’hard’ and benefits most from server refinement. Samples with low local uncertainty showed negligible improvement when processed by the server, validating our offloading logic.
6.5
Metric
Apple MacBook M2 Value vs. Server
Uncertainty Bin
Hardware Generalization: M2 vs. Pi 4
A key requirement for practical deployment is generalization across heterogeneous hardware. We evaluate StreamSplit on two platforms representing opposite ends of the edge computing spectrum: • Raspberry Pi 4B: ARM Cortex-A72 @ 1.5GHz, 4GB RAM, representing resource-constrained IoT devices. • Apple MacBook M2: 8-core CPU with Neural Engine, 8GB unified memory, representing capable edge devices. 6.5.1 Performance Comparison. Table 6 compares StreamSplit across platforms. On Apple M2, StreamSplit achieves higher accuracy (73.2% vs. 71.8%) and lower latency (67ms vs. 127ms) due to faster local inference. Interestingly, energy efficiency improves less dramatically (78.4 mJ vs. 89.3 mJ), as the M2’s power consumption scales with its higher compute throughput. 6.5.2 Learned Policy Behavior. Figure 10 visualizes the RL policies learned for each platform. The policies exhibit distinct, hardwareaware behaviors: Pi 4B Policy: Conservative local processing. The agent selects shallower split points (average 𝑘 = 2.1), offloading 38% of frames entirely (𝑘 = 0). Under memory pressure, it aggressively offloads to avoid cache thrashing, prioritizing system stability over bandwidth savings. M2 Policy: Aggressive local processing. The agent selects deeper split points (average 𝑘 = 3.8), offloading only 12% of frames. With
Accuracy (%) Latency (ms) Energy (mJ) Bandwidth (KB)
71.8 ± 0.4 127 ± 12 89.3 ± 4.1 58.7 ± 6.2
−1.8% −72.6% −52.3% −77.1%
73.2 ± 0.3 67 ± 5 78.4 ± 2.8 42.3 ± 4.5
(a) Raspberry Pi 4B 6
(b) Apple MacBook M2
3 2 1 0 0.0
0.4
0.6
0.8
1.0
5 Mbps 20 Mbps 40 Mbps
4 3 2 1
Avg. k = 2.1
0.2
High Ut Bandwidth
5
5 Mbps 20 Mbps 40 Mbps
4
−0.4% −85.6% −58.1% −83.5%
6
High Ut Bandwidth
5
Split Point k
0
Split Point k
(a) Metric Validation
MobiSys ’26, June 21–25, 2026, Cambridge, United Kingdom
Avg. k = 3.8
0 0.0
0.2
Uncertainty (Ut )
0.4
0.6
0.8
1.0
Uncertainty (Ut )
Figure 10: Learned Policies. The agent learns hardware-aware strategies: conservative offloading for Pi 4B vs. aggressive local compute for M2.
Table 7: Cross-Platform Policy Transfer. Policies generalize across hardware with minimal degradation; fine-tuning recovers full performance. Results report the mean ± standard deviation over 5 independent runs. Configuration
Accuracy (%)
Latency (ms)
Pi 4B → Pi 4B (native) M2 → Pi 4B (transfer) M2 → Pi 4B (fine-tuned)
71.8 ± 0.4 69.4 ± 0.6 71.6 ± 0.5
127 ± 12 143 ± 15 129 ± 11
M2 → M2 (native) Pi 4B → M2 (transfer) Pi 4B → M2 (fine-tuned)
73.2 ± 0.3 72.0 ± 0.5 73.1 ± 0.4
67 ± 5 78 ± 8 68 ± 6
ample compute headroom, it maximizes local inference to minimize transmission, achieving 83.5% bandwidth reduction. Notably, both policies share a common pattern: under high uncertainty (𝑈𝑡 > 0.7), they offload regardless of hardware capability, recognizing that ambiguous samples benefit from server refinement. This data-aware behavior emerges automatically from the uncertainty-guided reward, without explicit programming. 6.5.3 Cross-Platform Policy Transfer. To test generalization, we evaluate a policy trained on Pi 4B when deployed directly on M2 (and vice versa). Table 7 shows that direct policy transfer incurs only 1.2–2.4% accuracy degradation, suggesting the learned policies capture generalizable adaptation principles rather than hardwarespecific quirks. Fine-tuning for 10K steps on the target platform recovers full performance.
MobiSys ’26, June 21–25, 2026, Cambridge, United Kingdom
7
Discussion
Data Obfuscation and Privacy. StreamSplit transmits intermediate embeddings rather than raw audio primarily to improve bandwidth efficiency. We state explicitly that while this avoids sending raw PCM data, it provides incidental obfuscation only. Reconstructing original audio from deep embeddings, though computationally non-trivial, remains theoretically possible [40]. Therefore, StreamSplit does not provide formal differential privacy or cryptographic guarantees. It should be viewed strictly as a resource-optimization framework; edge applications requiring rigorous security would need to integrate orthogonal privacy-preserving protocols—such as those addressing broad fairness and data minimization in IoT [41, 42]—or advanced network-level security mechanisms like real-time attack attribution in 6G [43], which we leave to future work. Hardware Generalization Limits. Our evaluation spans Raspberry Pi 4B to Apple M2, but StreamSplit’s 50MB memory footprint precludes deployment on microcontroller-class devices (ESP32, Arduino). Extending to ultra-low-power platforms would require aggressive quantization and fixed-point GMM redesign. The algorithmic contributions remain architecture-agnostic and could transfer to compressed models. Network Assumptions. StreamSplit assumes bidirectional edgeserver connectivity. Under complete disconnection, the system degrades to Edge-Only mode with reduced representation quality. Tolerating extended offline periods through delayed synchronization remains an open challenge. Multi-Client Scope. While this work focuses on optimizing the single-stream vertical offloading link, the server-side L𝑆𝑊 term (Eq. 13) naturally mitigates bias in multi-client settings. By enforcing that the aggregated buffer distribution matches the global uniform prior, StreamSplit prevents high-throughput devices (which contribute more frames) from dominating the embedding space geometry.
8
Related Work
Contrastive Learning on Edge. Standard contrastive methods require large memory banks [13] or batch sizes [12] infeasible on edge devices. Recent lightweight alternatives employ importance sampling [44] or knowledge distillation [45], but still require substantial memory overhead. StreamSplit introduces distributional memory—a compact GMM replacing explicit storage—enabling contrastive learning with <35KB overhead through boundary-aware negative synthesis. While recent works like SynCo [46] propose synthetic negatives, they typically mix feature-level negatives from a queue. StreamSplit differs by generating virtual negatives from a parametric GMM distribution, eliminating the need to store a memory bank entirely. We exclude non-contrastive baselines (e.g., BYOL-A [47], SimSiam [48], VICReg [49]) as they rely on batch normalization statistics (undefined for streaming 𝑁 = 1) or momentum encoders (doubling memory), which are incompatible with strict edge constraints. Energy Efficiency Context. Recent benchmarking of SSL on the edge [50] confirms that standard CL frameworks (e.g., SimCLR with ResNet-18) exhibit prohibitive energy footprints due to memory and compute intensity. StreamSplit situates itself within this landscape by demonstrating that while purely local execution offers
Quan and Pathirana
the lowest absolute energy (67.4 mJ, Table 2), it suffers from dimensional collapse. StreamSplit incurs a moderate energy premium over these local baselines to achieve server-grade accuracy, while still reducing energy by 52.3% compared to the static offloading approaches often assumed in prior energy profiles. Split Computing and Split Learning. Neurosurgeon [21] pioneered adaptive edge-cloud partitioning, with extensions to multiexit architectures [38] and privacy-aided federated settings [19, 51]. However, existing approaches assume fixed partition points, failing to adapt to runtime volatility. StreamSplit introduces uncertaintyguided control, dynamically selecting split points based on system state and data difficulty, achieving 3.5× faster adaptation than heuristic methods. The Limits of Decoupled Approaches. Existing resourceaware architectures reactively split computation based on telemetry [21, 23]. Naively pairing these schedulers with contrastive objectives fails: inevitable network-induced frame drops shatter the temporal embedding manifold. StreamSplit overcomes this via joint co-design. Proactively, our Uncertainty-Guided Splitter aligns offloading with actual sample difficulty (validating our GMM entropy metric, which yields a Pearson correlation of 𝑟 = 0.84 with server utility). Reactively, our Server Refiner’s Hybrid Loss integrates Laplacian regularization (L𝐿𝑎𝑝 ) [25] to enforce a Lipschitzcontinuous smoothing prior. This allows the server to gracefully bridge temporal gaps from sparse updates without explicit interpolation, outperforming independent small-batch [22] or scheduling interventions. Federated Learning. FL enables distributed training with data locality across edge and cyber-physical systems [52, 53, 54], with recent extensions to contrastive settings [37, 55]. However, FL assumes periodic synchronization and homogeneous architectures, conflicting with streaming applications requiring continuous updates on heterogeneous hardware. StreamSplit addresses this through asynchronous server refinement, complementing rather than replacing FL systems. Similarly, while LW-FedSSL [56] reduces resource usage via layer-wise training, StreamSplit focuses on real-time inference and continuous learning via adaptive splitting, which is orthogonal to layer-wise federated training strategies.
9
Conclusion
We introduced StreamSplit, a framework enabling streaming contrastive learning on edge devices. By replacing memory-intensive queues with Distributional Memory (< 35KB) and optimizing offloading via Uncertainty-Guided RL, StreamSplit resolves fundamental resource conflicts. Our evaluation confirms bandwidth reductions of 77.1% and energy savings of 52.3% while maintaining accuracy within 2.2% of server-only baselines. The framework proves robust across heterogeneous hardware (Pi 4B to M2), offering a scalable path for adaptive edge intelligence.
Acknowledgments In accordance with ACM’s Policy on Authorship, the authors disclose the use of generative AI tools to assist in the conceptual drafting of graphical icons used in preliminary versions of this manuscript’s figures. All formal technical schematics, system architectures, and data plots presented in this final version were
StreamSplit: Continuous Audio Representation Learning via Uncertainty-Guided Adaptive Splitting
manually constructed by the authors to ensure technical accuracy and rigor.
A
Theoretical Convergence Guarantees
Lemma 1 (Partition Function Bias). The contrastive objective asymptotically maximizes mutual information. However, for a finite batch size 𝑁 , the estimator of the partition function 𝑍𝜃 (𝑥) = ⊤ E𝑧∼𝑝𝜃 [𝑒 𝑓 (𝑥 ) 𝑧 ] is biased. The deviation in the empirical loss is dominated by: ! 𝑁 ⊤ 1 ∑︁ 𝑓 (𝑥 ) ⊤𝑧𝑖 𝑒 − log E𝑧∼𝑝𝜃 [𝑒 𝑓 (𝑥 ) 𝑧 ] (15) ΔL = log 𝑁 𝑖=1
MobiSys ’26, June 21–25, 2026, Cambridge, United Kingdom
adjacency graph with Laplacian L. For a missing frame at 𝑡 ∗ , define the reconstruction as the weighted neighbor average: ∑︁ 1 𝑧ˆ𝑡 ∗ = 𝑊𝑡 ∗ 𝑗 𝑧 𝑗 (21) 𝑑𝑡 ∗ 𝑗 ∈ N𝑛𝑏𝑟 (𝑡 ∗ ) Í where 𝑑𝑡 ∗ = 𝑗 ∈ N𝑛𝑏𝑟 (𝑡 ∗ ) 𝑊𝑡 ∗ 𝑗 is the weighted degree and N𝑛𝑏𝑟 (𝑡 ∗ ) denotes the set of temporal neighbors. Step 1: Local Reconstruction Error. The squared reconstruction error is: 2
||𝑧𝑡 ∗ − 𝑧ˆ𝑡 ∗ || 2 =
1 𝑧𝑡 ∗ − 𝑑𝑡 ∗
∑︁
𝑊𝑡 ∗ 𝑗 𝑧 𝑗
(22)
𝑗 ∈ N (𝑡 ∗ ) 2
Definition 4 (Wasserstein-1 Distance). Let Lip1 (S𝑑 −1 ) be the set of 1-Lipschitz functions on the hypersphere. The Wasserstein-1 distance between distributions 𝑝 and 𝑞 is given by the KantorovichRubinstein duality: W1 (𝑝, 𝑞) = sup E𝑧∼𝑝 [ℎ(𝑧)] − E𝑧∼𝑞 [ℎ(𝑧)]
(16)
ℎ∈Lip1
1 = 𝑑𝑡 ∗
∑︁
𝑊𝑡 ∗ 𝑗 (𝑧𝑡 ∗ − 𝑧 𝑗 )
(23)
𝑗 ∈ N (𝑡 ∗ )
By Jensen’s inequality (convexity of || · || 2 ): 1 ∑︁ ||𝑧𝑡 ∗ − 𝑧ˆ𝑡 ∗ || 2 ≤ 𝑊𝑡 ∗ 𝑗 ||𝑧𝑡 ∗ − 𝑧 𝑗 || 2 𝑑𝑡 ∗ ∗
(24)
𝑗 ∈ N (𝑡 )
Step 2: Relating Local to Global Energy. Define the local energy at node 𝑡 ∗ : ∑︁ 𝐸𝑙𝑜𝑐𝑎𝑙 (𝑡 ∗ ) = 𝑊𝑡 ∗ 𝑗 ||𝑧𝑡 ∗ − 𝑧 𝑗 || 2 (25)
Proof of Theorem 3.1 (Small-Batch Robustness). Setup. Let the contrastive loss for anchor 𝑥 be: ⊤ +
𝑒 𝑓 (𝑥 ) 𝑧 L (𝑥) = − log 1 Í𝑁 𝑓 (𝑥 ) ⊤ 𝑧𝑖 𝑖=1 𝑒 𝑁
(17)
𝑁 are negatives sampled where 𝑧 + is the positive sample and {𝑧𝑖 }𝑖=1 from 𝑝𝜃 . The population loss replaces the finite sum with the expectation under 𝑝𝜃 . Step 1: Distributional Mismatch. Define the critic function ℎ(𝑧) = ⊤ 𝑒 𝑓 (𝑥 ) 𝑧 . Since embeddings are ℓ2 -normalized (∥𝑧∥ = 1) and ∥𝑓 (𝑥)∥ ≤ 1, we have ℎ(𝑧) ∈ [𝑒 −1, 𝑒], making ℎ bounded with Lipschitz constant 𝐾 ≤ 𝑒. By the Kantorovich-Rubinstein duality for W1 :
E𝑧∼𝑝𝜃 [ℎ(𝑧)] − E𝑧∼U [ℎ(𝑧)] ≤ 𝐾 · W1 (𝑝𝜃 , U) < 𝐾𝜖
(18)
Step 2: Finite Sample Error. By Hoeffding’s inequality, for 𝑁 i.i.d. samples from 𝑝𝜃 : √︂ 𝑁 log(2/𝛿) 1 ∑︁ ℎ(𝑧𝑖 ) − E𝑝𝜃 [ℎ(𝑧)] ≤ (𝑒 − 𝑒 −1 ) (19) 𝑁 𝑖=1 2𝑁 with probability at least 1 − 𝛿. Step 3: Combined Bound. Since log(·) is Lipschitz on [𝑒 −1, 𝑒] with constant 𝑒, composing the bounds via triangle inequality yields: √︂ log(2/𝛿) 1 −1 |L𝑁 − L∞ | ≤ 𝑒 · 𝐾 · 𝜖 + 𝑒 (𝑒 − 𝑒 ) ·√ (20) | {z } 2 𝑁 | {z } 𝐶 𝜖 1
𝐶2
Thus, minimizing the diversity gap 𝜖 (via L𝑆𝑊 ) directly compensates for the bias induced by small 𝑁 , allowing edge devices with small batches to approximate large-batch server performance. □ Proof of Theorem 3.2 (Temporal Interpolation). Setup. Let Z ∈ R𝑇 ×𝑑 be the embedding matrix where row 𝑧𝑡 is the embedding at time 𝑡. Let G = (𝑉 , 𝐸,𝑊 ) be the temporal
𝑗 ∈ N (𝑡 ∗ )
Í The total Dirichlet energy is 𝐸𝑡𝑜𝑡𝑎𝑙 = 12 𝑡 𝐸𝑙𝑜𝑐𝑎𝑙 (𝑡) = Tr(Z⊤ LZ), where the factor of 1/2 accounts for double-counting edges. For any node 𝑡 ∗ , the local energy is bounded by the total: 𝐸𝑙𝑜𝑐𝑎𝑙 (𝑡 ∗ ) ≤ 2 · 𝐸𝑡𝑜𝑡𝑎𝑙 = 2 · Tr(Z⊤ LZ)
(26)
Step 3: Spectral Gap Refinement. For connected graphs, the discrete Poincaré inequality states that for any zero-mean signal f: Var(f) ≤
1 ⊤ f Lf 𝜆2
(27)
Applying this per-coordinate to the centered embeddings Z̃ = Z − 𝑧¯ (where 𝑧¯ is the mean embedding), we obtain that the average local deviation is controlled by the spectral gap. For the worst-case node: 2 max 𝐸𝑙𝑜𝑐𝑎𝑙 (𝑡 ∗ ) ≤ · Tr(Z⊤ LZ) (28) 𝑡∗ 𝜆2 Step 4: Final Bound. Combining Steps 1-3 and using D𝑎𝑓 𝑓 = 1 ⊤ |𝐸 | Tr(Z LZ) ≤ 𝛼: 1 𝐸𝑙𝑜𝑐𝑎𝑙 (𝑡 ∗ ) 𝑑𝑡 ∗ 1 2 · · |𝐸| · 𝛼 ≤ 𝑑𝑡 ∗ 𝜆 2 2𝛼 · |𝐸| = 𝜆 2 · 𝑑𝑡 ∗
||𝑧𝑡 ∗ − 𝑧ˆ𝑡 ∗ || 2 ≤
(29) (30) (31)
For unweighted graphs with uniform degree, 𝑑𝑡 ∗ = |N (𝑡 ∗ )|, yielding the stated bound. □ Remark. This bound formalizes the intuition of manifold regularization: a high spectral gap 𝜆2 (ensured by dense temporal connectivity) combined with low Dirichlet energy 𝛼 (enforced by L𝐿𝑎𝑝 ) guarantees bounded interpolation error, providing robustness to frame drops in volatile edge-cloud execution.
MobiSys ’26, June 21–25, 2026, Cambridge, United Kingdom
B
Quan and Pathirana
Control Plane MDP Definition
[12]
This section provides the formal specification of the Markov Decision Process (MDP) utilized by the Uncertainty-Guided Adaptive Splitter. As introduced in Section 4.2.2, the StreamSplit Control Plane models the dynamic edge-cloud partitioning problem as an RL environment. Table 8 details the exact components of this formulation, including the observable state space (incorporating our zero-cost uncertainty metric), the discrete action space dictating the network split point, and the composite reward function designed to balance representation quality against strict latency and energy constraints. Table 8: Control Plane MDP Definition. The RL agent optimizes a joint objective of task performance, latency constraints, and energy efficiency.
[14]
[15]
[16]
[17] [18]
[19]
Component
Description / Definition
State 𝑠𝑡
Vector [𝑈𝑡 , 𝑅𝑐𝑝𝑢 , 𝐵𝑛𝑒𝑡 ] ∈ R3 𝑈𝑡 : Embedding Uncertainty (Entropy, Eq. 11) 𝑅𝑐𝑝𝑢 : CPU Utilization (%) 𝐵𝑛𝑒𝑡 : Est. Uplink Bandwidth (Mbps, EMA)
Action 𝑎𝑡
Split Layer 𝑘 ∈ {0, 1, . . . , 𝐿} 𝑘 = 0: Full Offload; 𝑘 = 𝐿 : Full On-Device
Reward 𝑟𝑡
𝑡 −𝜂 · 𝑟𝑡 = 𝛼 · A𝑡𝑎𝑠𝑘 − 𝛽 · 𝑇𝑚𝑎𝑥 𝐸
Lat
[13]
𝐸𝑡 𝑏𝑢𝑑𝑔𝑒𝑡
[20]
[21]
[22]
(Eq. 12)
Balances Accuracy ( A ), Latency (Lat), Energy (𝐸 )
[23]
[24]
References [1]
[2]
[3]
[4]
[5]
[6]
[7]
[8]
[9] [10]
[11]
N.A. Angel, D. Ravindran, P.M.D.R. Vincent, K. Srinivasan, and Y.-C. Hu. 2022. Recent advances in evolving computing paradigms: cloud, edge, and fog technologies. Sensors, 22, 1, 196. X. Li, Y. Chen, and Z. Wang. 2025. Edge intelligence through in-sensor and nearsensor computing for the artificial intelligence of things. npj Unconventional Computing, 1, 40. Minh K Quan, Mayuri Wijayasundara, Sujeeva Setunge, and Pubudu N Pathirana. 2025. Quantum-enhanced transformers for robust acoustic scene classification in iot environments. In 2025 International Conference on Computing, Networking and Communications (ICNC). IEEE, 295–299. Minh K Quan, Mayuri Wijayasundara, Sujeeva Setunge, and Pubudu N Pathirana. 2025. Quantum-inspired genetic algorithm for robust source separation in smart city acoustics. In 2025 IEEE International Conference on Communications Workshops (ICC Workshops). IEEE, 1966–1971. Yusong Wu, Ke Chen, Tianyu Zhang, Yuchen Hui, Taylor Berg-Kirkpatrick, and Shlomo Dubnov. 2023. Large-scale contrastive language-audio pretraining with feature fusion and keyword-to-caption augmentation. In IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE. Aaqib Saeed, David Grangier, and Neil Zeghidour. 2021. Contrastive learning of general-purpose audio representations. In IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 3875–3879. Haider Al-Tahan and Yalda Mohsenzadeh. 2021. CLAR: contrastive learning of auditory representations. In International Conference on Artificial Intelligence and Statistics (AISTATS). PMLR, 2530–2538. H.A. Alharbi, M. Aldossary, J. Almutairi, and I.A. Elgendy. 2023. Energy-aware and secure task offloading for multi-tier edge-cloud computing systems. Sensors, 23, 6, 3254. P. Zhang, M. Li, and Q. Wang. 2025. Edge computing in big data: challenges and benefits. International Journal of Data Science and Analytics. In press. A. Sakip, R. Yersainov, M. Atashikova, T. Rakhimzhan, D.M. Bui, and E.N. Huh. 2023. Lightweight energy-efficient offloading framework for mobile edge/cloud computing. In 2023 17th International Conference on Ubiquitous Information Management and Communication (IMCOM). IEEE, 1–8. L. Lin, X. Liao, H. Jin, and P. Li. 2019. Computation offloading toward edge computing. Proceedings of the IEEE, 107, 8, 1584–1607.
[25]
[26]
[27] [28]
[29]
[30]
[31]
[32] [33]
[34] [35]
[36]
[37]
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. 2020. A simple framework for contrastive learning of visual representations. In Proceedings of the 37th International Conference on Machine Learning (PMLR). Vol. 119, 1597–1607. Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. 2020. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 9729–9738. Jean-Bastien Grill et al. 2020. Bootstrap your own latent: a new approach to self-supervised learning. In Advances in Neural Information Processing Systems. Vol. 33, 21271–21284. K. Akherfi, M. Gerndt, and H. Harroud. 2018. Mobile cloud computing for computation offloading: issues and challenges. Applied Computing and Informatics, 14, 1, 1–16. M.Y. Akhlaqi and Z.B.M. Hanapi. 2023. Task offloading paradigm in mobile edge computing—current issues, adopted approaches, and future directions. Journal of Network and Computer Applications, 212, 103568. G.I. Arcas, T. Cioara, I. Anghel, D. Lazea, and A. Hangan. 2024. Edge offloading in smart grid. Smart Cities, 7, 1, 680–711. S. Zhou, W. Jadoon, and I.A. Khan. 2023. Computing offloading strategy in mobile edge computing environment: a comparison between adopted frameworks, challenges, and future directions. Electronics, 12, 11, 2452. Chandra Thapa, Mahawaga Arachchige Pathum Chamikara, Seyit A. Camtepe, and Lichao Sun. 2022. Splitfed: when federated learning meets split learning. In AAAI Conference on Artificial Intelligence, 8485–8493. J. He, D. Zhang, Y. Zhou, and S. Wang. 2023. A split-federated learning and edgecloud based efficient and privacy-preserving large-scale item recommendation model. Journal of Cloud Computing, 12, 47. Yiping Kang, Johann Hauswald, Cao Gao, Austin Rovinski, Trevor Mudge, Jason Mars, and Lingjia Tang. 2017. Neurosurgeon: collaborative intelligence between the cloud and mobile edge. In ACM International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), 615–629. Hao Fang, Dong Ouyang, Hong Zhang, Shutao Wang, Xing Liu, Xiaohui Xie, Jie Li, and Kui Ren. 2022. Enable deep learning on mobile devices: methods, systems, and applications. ACM Computing Surveys, 55, 5, 1–38. Y. Huang, H. Zhang, X. Shao, X. Li, and H. Ji. 2023. Roofsplit: an edge computing framework with heterogeneous nodes collaboration considering optimal cnn model splitting. Future Generation Computer Systems, 140, 79–90. Soheil Kolouri, Kimia Nadjahi, Umut Simsekli, Roland Badeau, and Gustavo K. Rohde. 2019. Generalized sliced wasserstein distances. In Advances in Neural Information Processing Systems. Vol. 32, 261–272. Mikhail Belkin, Partha Niyogi, and Vikas Sindhwani. 2006. Manifold regularization: a geometric framework for learning from labeled and unlabeled examples. Journal of Machine Learning Research, 7, 2399–2434. Tongzhou Wang and Phillip Isola. 2020. Understanding contrastive representation learning through alignment and uniformity on the hypersphere. In International Conference on Machine Learning (ICML). PMLR, 9929–9939. L. Kong, G. Chen, and K. Wang. 2024. Cost optimization in edge computing: a survey. Artificial Intelligence Review, 57, 285. Z. Wang et al. 2022. Online continual learning with contrastive vision transformer. In European Conference on Computer Vision (ECCV). Springer, 645– 662. Yunrui Sun, Gang Hu, Yinglei Teng, and Dunbo Cai. 2024. Split federated learning over heterogeneous edge devices: algorithm and optimization. arXiv preprint arXiv:2411.13907. Arthur Gretton, Karsten M. Borgwardt, Malte J. Rasch, Bernhard Schölkopf, and Alexander Smola. 2012. A kernel two-sample test. Journal of Machine Learning Research, 13, 723–773. Zoubin Ghahramani and Hagai Attias. 2000. Online variational bayesian learning. In Advances in Neural Information Processing Systems (NIPS). Vol. 13. MIT Press, 520–526. John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347. Adam Paszke et al. 2019. Pytorch: an imperative style, high-performance deep learning library. In Advances in Neural Information Processing Systems (NeurIPS), 8024–8035. Mark Borgerding. 2003. KissFFT: a mixed-radix fast fourier transform library. https://github.com/mborgerding/kissfft. Accessed: 2024. (2003). Francis Y. Yan, Jesber Ma, Greg D. Hill, Deepak Raghavan, Riad S. Wahby, Philip Levis, and Keith Winstein. 2018. Pantheon: the training ground for internet congestion-control research. In USENIX Annual Technical Conference (ATC), 731–743. Jort F. Gemmeke, Daniel P. W. Ellis, Dylan Freedman, Aren Jansen, Wade Lawrence, R. Channing Moore, Manoj Plakal, and Marvin Ritter. 2017. Audio set: an ontology and human-labeled dataset for audio events. In IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 776–780. Fengda Zhang et al. 2023. Federated unsupervised representation learning. In Frontiers of Information Technology & Electronic Engineering. Vol. 24, 1181–1193.
StreamSplit: Continuous Audio Representation Learning via Uncertainty-Guided Adaptive Splitting
[38]
[39] [40]
[41]
[42]
[43]
[44]
[45]
[46] [47]
Stefanos Laskaridis, Stylianos I. Venieris, Mario Almeida, Ilias Leontiadis, and Nicholas D. Lane. 2020. SPINN: synergistic progressive inference of neural networks over device and cloud. In ACM International Conference on Mobile Computing and Networking (MobiCom), 1–15. Adel Noureddine. 2021. PowerJoular: monitor power consumption of software and hardware. https://github.com/joular/powerjoular. Accessed: 2024. (2021). Nicholas Carlini, Jamie Hayes, Milad Nasr, Matthew Jagielski, Vikash Sez, Florian Tramèr, Borja Balle, Daphne Ippolito, and Eric Wallace. 2023. Extracting training data from diffusion models. In USENIX Security Symposium, 5253–5270. Sina Shaham, Arash Hajisafi, Minh K Quan, Dinh C Nguyen, Bhaskar Krishnamachari, Charith Peris, Gabriel Ghinita, Cyrus Shahabi, and Pubudu N Pathirana. 2025. Privacy and fairness in machine learning: a survey. IEEE Transactions on Artificial Intelligence, 6, 7, 1706–1726. Minh K Quan, Dinh C Nguyen, Van-Dinh Nguyen, Mayuri Wijayasundara, Sujeeva Setunge, and Pubudu N Pathirana. 2024. Toward privacy-preserving waste classification in the internet of things. IEEE Internet of Things Journal, 11, 14, 24814–24830. Minh K Quan and Pubudu N Pathirana. [n. d.] Domain-adapted granger causality for real-time cross-slice attack attribution in 6g networks. In NeurIPS 2025 Workshop on CauScien: Uncovering Causality in Science. Weiming Zhuang, Xin Gan, Yonggang Wen, and Shuai Zhang. 2022. Collaborative unsupervised visual representation learning from decentralized data. In IEEE/CVF International Conference on Computer Vision (ICCV), 4912–4921. Zhi Wang, Xuan Luo, Ruijie Wang, and Xin Li. 2023. EdgeCL: edge-cloud collaborative learning for on-device ai. In ACM International Conference on Mobile Systems, Applications, and Services (MobiSys), 289–301. Nikolaos Giakoumoglou and Tania Stathaki. 2024. Synco: synthetic hard negatives for contrastive visual representation learning. arXiv preprint arXiv:2410.02401. Daisuke Niizumi et al. 2021. Byol for audio: self-supervised learning for generalpurpose audio representation. In IJCNN.
[48] [49] [50]
[51]
[52]
[53]
[54]
[55]
[56]
MobiSys ’26, June 21–25, 2026, Cambridge, United Kingdom
Xinlei Chen and Kaiming He. 2021. Exploring simple siamese representation learning. In CVPR. Adrien Bardes et al. 2022. Vicreg: variance-invariance-covariance regularization for self-supervised learning. In ICLR. Fernanda Famá, Roberto Pereira, Charalampos Kalalas, Paolo Dini, Lorena Qendro, Fahim Kawsar, and Mohammad Malekzadeh. 2025. Contrastive selfsupervised learning at the edge: an energy perspective. arXiv preprint arXiv:2510.08374. Minh K Quan, Dinh C Nguyen, Van-Dinh Nguyen, Mayuri Wijayasundara, Sujeeva Setunge, and Pubudu N Pathirana. 2023. Hiersfl: local differential privacy-aided split federated learning in mobile edge computing. In 2023 IEEE Virtual Conference on Communications (VCC). IEEE, 103–108. Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Agüera y Arcas. 2017. Communication-efficient learning of deep networks from decentralized data. In International Conference on Artificial Intelligence and Statistics (AISTATS), 1273–1282. Peter Kairouz, H. Brendan McMahan, Brendan Avent, Aurélien Bellet, Mehdi Bennis, Arjun Nitin Bhagoji, Kallista Bonawitz, et al. 2021. Advances and open problems in federated learning. Foundations and Trends in Machine Learning, 14, 1–2, 1–210. Minh K Quan, Pubudu N Pathirana, Mayuri Wijayasundara, Sujeeva Setunge, Dinh C Nguyen, Christopher G Brinton, David J Love, and H Vincent Poor. 2025. Federated learning for cyber physical systems: a comprehensive survey. IEEE Communications Surveys & Tutorials. Ekdeep Singh Lubana, Chi Ian Tang, Fahim Kawsar, Robert P. Dick, and Akhil Mathur. 2022. Orchestra: unsupervised federated learning via globally consistent clustering. In International Conference on Machine Learning (ICML), 14461–14484. Ye Lin Tun, Chu Myaet Thwal, Huy Q Le, Minh NH Nguyen, and Choong Seon Hong. 2024. Lw-fedssl: resource-efficient layer-wise federated self-supervised learning. arXiv preprint arXiv:2401.11647.