FedSceneX: Time-to-Target Orchestration for Same-Scene Multimodal Federated Edge Learning

FedSceneX: Time-to-Target Orchestration for Same-Scene Multimodal Federated Edge Learning Dhe yeong Tchalla∗ , Beining Wu∗ , Jun Huang∗ , Shuyang Gu† , and Qiang Duan‡ ∗ Department of Electrical Engineering and Computer Science, South Dakota State University, Brookings, SD, USA † Subhani Department of Computer Information Systems, Texas A&M University-Central Texas, Killeen, TX, USA ‡ Department of Information Sciences and Technology, The Pennsylvania State University, Abington, PA, USA

arXiv:2608.07730v1 [cs.NI] 7 Aug 2026

Email: {Dheyeong.Tchalla, Wu.Beining}@jacks.sdstate.edu, Jun.Huang@sdstate.edu, s.gu@tamuct.edu, qxd2@psu.edu

Abstract—Federated learning at the sensing edge is typically evaluated by communication rounds, yet a round does not represent a fixed amount of work. Even on identical hardware, the methods we compare require 3.3 to 9.8 hours per round, which makes round-based comparisons misleading. The problem is more obvious for same-scene multimodal clients, since camera, video, LiDAR, and radar workloads differ substantially in training and communication cost, while existing methods treat the modality composition of each round as fixed. To address it, we introduce FedSceneX, an orchestrator that jointly determines round composition to maximize learning value per active hour. The optimization method, Value-per-Hour Pricing (VHP), converts the fractional objective through a parametric transformation and dualizes the uplink constraint, yielding a closed-form client price whose weights capture resource shadow costs. Based on these prices, FedSceneX selects clients subject to a modality coverage constraint, allocates precision through reverse water filling, and assigns updates to edge servers. On the full nuScenes benchmark with fifteen clients and twelve baselines, FedSceneX reduces the active time per round to 3.31 hours, compared with 4.85 to 9.78 hours for the baselines. Across all random seeds, it achieves the highest accuracy within a twentyhour budget while preserving all four modalities. Its advantage persists from ten to forty-five hours, after which conventional methods overtake it. Index Terms—Federated learning, multimodal learning, edge intelligence, network orchestration, client selection, resource allocation, autonomous driving.

I. I NTRODUCTION

F

EDERATED learning trains a shared model without moving raw data off the devices that hold it [1]–[6]. In autonomous driving and urban sensing, those devices are the sensors of one platform. Cameras, video streams, LiDAR, and radar observe the same scene at the same instant and share a scene-level label, but they agree on little else: their encoders differ in depth and cost, their updates differ in size, and the semantic classes they resolve differ by sensor [7]–[12]. Two lines of work make such a federation practical. Heterogeneity-aware and personalized optimizers correct client drift or split the model, through dynamic regularization, alternating optimization, adaptive server updates, or a personalized head [13]–[21], while communication-efficient methods shrink or ration the payload by quantizing updates, restructuring them, or admitting only the clients that meet a deadline [22]–[34]. Both are measured per communication round, and that unit is the problem. A round is not a fixed

amount of work: its duration is set by which clients are selected and how much local computation each is asked to perform, so on identical hardware and under the same local budget the methods we compare spend between 3.3 and 9.8 hours in one round. Two consequences follow. First, evaluation misreports progress, because a convergence curve drawn against the round index compares quantities of unequal price. Second, control is aimed at the wrong resource, because a round’s time is spent almost entirely on local computation, so a policy that only compresses what is sent cannot change how long the round takes. Same-scene multimodal learning turns this from an accounting problem into a design one, since the composition of a round now decides its price. A round of cameras is cheap and semantically narrow, a round with video and LiDAR is expensive and informative, and a round that omits radar gives up a quarter of the sensing problem. Multimodal federated learning studies how such clients should be aligned, fused, or reconstructed when a modality is missing [7], [8], [35]– [37], but it takes the modality mixture of a round as given rather than choosing it. Client selection comes closest, scoring clients by a utility that mixes statistical value with systems cost [38], yet the score decides participation alone, so the precision of an admitted update and the server that receives it stay outside the policy. Thus, an orchestrator that has to reach a usable model within a fixed number of hours has no single quantity to optimize, and instead tunes a selection threshold, a compression schedule, and a placement heuristic that cannot trade one budget against another. To address this, we introduce FedSceneX, a modality-aware orchestrator that treats the composition of a round as the decision variable. FedSceneX leaves the local objective, the aggregation rule, and the model architecture unchanged, and modality-specific encoders never leave the device. Each round profiles clients with a few privacy-safe scalars, prices each client with one number, and issues a contract that fixes whether the client uploads, at what precision it transmits, and where its update is aggregated. Unlike prior orchestration, the objective FedSceneX maximizes is learning value per active hour rather than per round, which makes the price of a round part of the decision instead of an outcome of it. Optimizing a ratio of this kind is harder than optimizing a sum, and the round’s active time is a makespan that does not

separate across clients. We propose Value-per-Hour Pricing (VHP), which removes both obstacles. Instead of scoring clients by a weighted sum chosen by hand, VHP applies a parametric transform to the fractional objective and dualizes the uplink budget, so the coupled program collapses into one closed-form scalar per client whose two negative weights are the shadow price of a byte and the worth of an hour. The same scalar then answers all three questions: a positivity threshold admits clients under a modality-coverage floor, a reverse water-filling rule quantizes every admitted update to a common distortion floor set by the uplink price, and a leastloaded rule places it on an edge server with a bounded loss in makespan. One number replaces the separate thresholds a modality-aware deployment would otherwise tune by hand. • We identify the price of a round as the variable that federated evaluation omits, and we formulate round composition as the maximization of learning value per active hour under an uplink budget and a modality-coverage floor. • We propose VHP, which turns that fractional program into one closed-form price per client, and we read that price three ways: as a participation threshold, as a reverse water-filling rule for precision, and as a placement rule whose makespan is within a bounded factor of the optimum. • On the complete nuScenes benchmark with fifteen heterogeneous clients and twelve baselines, we report the boundary of the claim together with the claim. A FedSceneX round costs 3.31 ± 0.13 active hours against 4.85 to 9.78, every seed leads the strongest baseline within a twenty-hour budget, and the lead reverses beyond fortyfive hours. An ablation separates the contract into the axes it acts on, and the scalars that make pricing possible identify a client’s modality with accuracy 1.00. Contrary to the intuition that a communication-constrained system is made faster by sending less, the network is not where a round spends its time: the entire network cost of a FedSceneX round is 1.99 seconds, or 0.016% of it, so compression acts on a fraction too small to recover. To our knowledge, FedSceneX is the first system to price participation, precision, and placement for same-scene multimodal clients with one utility, and to report the operating region in which doing so is the right choice. II. R ELATED W ORK A. Heterogeneity-Aware and Personalized Federated Learning To train on data that never leaves the device, federated learning averages local updates instead of gradients over pooled data [1]–[3]. Li et al. [39] propose FedProx, which adds a proximal term to limit client drift, and later work replaces the server rule or the local objective with dynamic regularization, alternating optimization, elastic aggregation, and adaptive server updates [13]–[15], [40]. A second group splits the model instead, keeping a personalized head, local normalization statistics, or a shared representation on the device [16], [41], [42], and a third regularizes the representation

against skewed or shifted client distributions [17], [43]–[47]. These methods change what a round optimizes, but they leave the price of the round outside the policy. FedSceneX is complementary to all of them: it changes what a round costs, and it leaves the local objective and the aggregation rule untouched. B. Multimodal Federated Learning Because the clients on one sensing platform carry different sensors, their encoders and their updates are no longer comparable. Sun et al. [8] propose FedCoLa, which aligns modalityspecific transformers across clients, while benchmarks and fusion methods study missing modalities and selective modality communication [7], [12], [35]–[37]. These methods make heterogeneous modalities learn together, but they take the modality mixture of a round as given. They are compatible with FedSceneX, which schedules that mixture and can carry any of their fusion rules. C. Budgeted Round Composition Closest to ours, a few systems compose a round by what it will cost rather than by what it will learn. Lai et al. [38] propose Oort, which is similar to ours in scoring clients by a utility that mixes statistical value with systems cost, but the score selects participants only, so precision and placement stay outside the policy and modality coverage is not a constraint. Deadline-based admission [24], periodic averaging with quantized updates [22], and structured-update compression [23] each cap one resource in isolation. FedSceneX differs in its goal: one utility prices participation, precision, and placement together, against the hours a deployment spends rather than the rounds it completes. III. BACKGROUND AND M OTIVATION A. Multimodal Federated Learning at the Edge Same-scene multimodal clients. In autonomous driving and urban sensing, several sensors observe one scene at the same instant. On nuScenes [48], six cameras, two video streams, two LiDAR partitions, and five radars form fifteen federated clients that share a scene-level label but not a representation. Their encoders differ in depth and cost, their updates differ in size, and their evidence differs by semantic class. Federated learning keeps the raw streams on the device [1], [2], [49]– [51], and multimodal federated learning aligns or fuses the resulting representations [7], [8], [35]. Neither decides how much of a training round each modality is allowed to consume. What the server may observe. Data locality restricts the controller to scalars. A client can report how far its loss fell, how large its update is, and what uplink it measured, but not what it saw. Any orchestration policy has to be built on this narrow channel. B. Observation: A Round Is Not a Unit of Progress Observation 1. A round costs what its clients cost, so ranking methods by the rounds they need reorders them relative to the time those rounds take.

TABLE I: Thirty active hours buy three rounds for the baselines and nine for FedSceneX. ∆ : Macro-F1 points at that budget. Cost

∆

Within 30 h

h/rnd

Rounds

Ma-F1

FedBABU [16] FedADMM [14] MOON [17]

9.75 9.26 6.20

3 3 3

0.6447 0.6486 0.6481

↑ 1.12 ↑ 0.73 ↑ 0.78

3.31±0.13

8.6±0.5

0.6559±0.0057

–

FedSceneX

3.3

Oort

5.2

FedSR

5.7

MOON

6.2

FedPAQ

7.1

FedCS

8.6

FedADMM

9.3

FedBABU

Rounds completed

Method

FedSceneX

10

5

9.8

0 0

5

10

Active hours per round

(a) Cost of a round

0

10

20

30

40

∆

Policy

FedBABU FedADMM MOON FedSceneX

15

TABLE II: Each row replaces one family of contract decisions with a uniform policy. ∆ : advantage of the contract, in each row’s units.

50

Cumulative active time (h)

(b) What the cost buys

Fig. 1: (a) Active hours per round, FedSceneX over five seeds. (b) Rounds completed against active time; the dotted line marks thirty hours. Table I makes the disagreement concrete. Given thirty active hours, FedBABU finishes three rounds and reaches Macro-F1 0.6447, while FedSceneX finishes nine and reaches 0.6559. Counted in rounds, FedSceneX looks like the slower learner, since it needs three times as many of them to arrive at a similar place. Counted in the hours a deployment actually spends, it arrives first, because a FedSceneX round costs 3.31 hours where theirs cost 6.20 to 9.75. Where a round’s time goes explains why. Within a FedSceneX round, local training accounts for 95.1% of the wallclock time and client-side evaluation for a further 4.9%, while the entire network cost of the round, base latency, queueing, and transmission together, is 1.99 seconds, or 0.016% of it. Compression acts on that fraction alone. This does not make uploaded volume irrelevant, since uplink capacity is shared and metered, but a method cannot buy wall-clock time by compressing what already takes two seconds. What is left is the composition of the round, namely which modality encoders run and for how long. Opportunities of composing rounds. If a round costs what its clients cost, an orchestrator should choose the composition of each round rather than only the number of participants. Modality then becomes a scheduling variable instead of a property of the dataset. C. Observation: One Utility Prices Three Decisions Observation 2. Who uploads, at what precision, and where the update is aggregated are three forms of one question: whether a client’s update is worth the resources the round will spend on it.

Decision

Metric

Participate Precision Place

Active h/round Upload MB/round Active h/round

Uniform

Contract

9.21 7.27 6.60

3.52 2.88 3.52

↑ 5.69 ↑ 4.39 ↑ 3.08

These decisions are usually taken by different mechanisms: a selection rule, a compression schedule, and a placement heuristic, each with a threshold of its own. Every threshold has to be retuned when the modality mixture changes, and none of them can trade one budget against another. One utility avoids this. It scores what a client’s update is expected to be worth against what sending it will cost, and the same number then answers all three questions. Table II shows what each family of decisions is worth on the axis it controls. Replacing utilitydriven participation with uniform participation raises the round from 3.52 to 9.21 hours, uniform precision raises the uploaded state from 2.88 to 7.27 MB per round, and removing utilitydriven placement raises the round to 6.60 hours. The design has a cost that is easy to overlook. The scalars the controller needs are the scalars that describe a client’s workload, and workloads differ systematically by modality. A logistic-regression attacker with access to nothing but this server-visible telemetry recovers a client’s modality with 1.00 accuracy against a majority baseline of 0.36. Orchestration of this kind buys time with information, and the information is not free. Opportunities of one-utility orchestration. Pricing three decisions with one number removes the separate thresholds a modality-aware deployment would otherwise tune by hand, and it makes the exposure of the control channel a measurable quantity rather than an assumption. IV. S YSTEM D ESIGN A. Overview FedSceneX is an orchestration layer over federated averaging. It decides only which clients train and transmit in a round and at what cost. Every round runs three phases (Figure 2). In the profile phase, each client performs a short local warmup and reports a few privacy-safe scalars: how far its local loss fell, how large its shared-state update is, what uplink it measured, and how often it has participated. In the price phase, Value-per-Hour Pricing (VHP) maps those scalars to one learning-aware utility. In the assign phase, the server reads that utility three times and issues a contract that fixes whether the client uploads, at what precision it transmits, and where its update is aggregated. The privacy boundary follows the model split. Client k observes modality mk , holds a local dataset Dk , and keeps a private encoder fmk (·; ϕk ) that never leaves the device. Only the adapter a(·; ω) and the multi-label head h(·; ψ) are

Fig. 2: Overview of FedSceneX. Private encoders stay on the device; one utility over privacy-safe scalars fixes participation, precision, and placement. synchronized, so the federated model is θ = {ω, ψ} and the client prediction is ŷk = hψ (aω (fmk (xk ; ϕk ))) .

(1)

Raw observations, labels, embeddings, logits, and per-sample outputs stay local. The server receives the quantized update of θ together with the scalars above, so a client exposes only its ds shared parameters and not the dp of its private encoder. B. The Round as a Ratio A deployment is judged by the accuracy it reaches per hour and not by the accuracy it reaches per round, so what a round should maximize is a ratio rather than a sum. Let V (Γt ) be the expected worth of the updates a contract set Γt buys and let T (Γt ) be the active time it consumes, which is the makespan of the selected clients over the edge servers plus their synchronization. The orchestration problem is max t Γ

s.t.

V (Γt ) T (Γt ) X qkt btk ≤ B, k∈K

X

qkt ≥ 1 ∀m ∈ M,

(2)

k: mk =m

together with one edge server per selected client and the discrete range of every contract variable. The second constraint is a coverage floor, and it is what keeps a modality in the round during the stretches when its clients price poorly. Equation (2) is a fractional program over binary and discrete variables, and neither the ratio nor the makespan inside T

separates across clients. Both obstacles have a standard remedy. For the ratio, the parametric transform of Dinkelbach [52] replaces max V /T by the root of  Φ(κ) = max V (Γt ) − κ T (Γt ) , (3) t Γ

which is convex and strictly decreasing, so a ratio problem becomes a sequence of problems linear in worth and in time, and κ carries the units of value per hour. For the makespan, assigning each selected client to the least-loaded server is list scheduling, whose makespan is within 2 − 1/|E| of the optimum [53], or a factor 1.5 for the two servers used here, so T may be replaced by the total time the round buys at a bounded loss. What is left couples clients only through the uplink budget, and one multiplier removes it. C. Value-per-Hour Pricing VHP is the per-client score that the two reductions leave behind. Proposition 1 (Separable pricing). Let λ ≥ 0 multiply the uplink budget of Equation (2) and let κ be the parameter of Equation (3). ThePparametric Lagrangian then decomposes across clients as k qkt Ukt + λB with Ukt = αPkt + γGtk + µFkt − δXkt − ηLtk ,

(4)

where δ = λ and η = κ, and the participation that maximizes it is the threshold rule qkt⋆ = 1[Ukt > 0] corrected by the coverage floor. Proof: Collect what one selected client contributes to V − κT : its expected worth αPkt + γGtk + µFkt and the time κLtk

TABLE III: Notation. Symbol

Meaning

Clients and model K, M, E clients, sensing modalities, edge servers Dk , mk local dataset and modality of client k private encoder and shared state {ω, ψ} ϕk , θ dp , ds private and shared parameter counts Pricing Ukt P, G, F X, L α, γ, µ, δ, η κ, λ σk2 , D⋆ , ρ

learning-aware utility of client k at round t local progress, update magnitude, fairness credit uplink cost and time added to the round utility weights, with δ = λ, η = κ value of an hour, shadow price of a byte update variance, distortion floor, loss sensitivity

Contract and budget qkt , ntk upload decision and local epochs btk , rkt , zkt uplink rate, update frequency, quantization level ptk , stk , etk priority, network slice, edge server B, Λte uplink budget and load of edge server e St clients selected at round t

it adds to the round. Adjoining the budget with λ adds −λXkt to the same bracket and the constant λB. Every remaining constraint is a per-client range, so the maximization distributes over k and a client raises the sum exactly when its bracket is positive. Equation (4) is therefore not a weighted sum chosen by hand. Its two negative weights are prices: δ is the shadow price of a byte of uplink, and η is the worth of an hour on the round’s critical path. This is why Equation (2) carries no second penalty on communication or delay, and why one number can arbitrate between a fast client and an informative one. Here Pkt is normalized local progress, Gtk = ∥∆θ tk ∥2 is update magnitude, Fkt credits clients and modalities that have participated less, Xkt is the uplink cost of the update after compression, and Ltk is the time the client adds to the round, from local training through synchronization. Every term is scaled to [0, 1] within the round, so the weights carry relative importance rather than unit conversion. The implementation holds κ at a value calibrated once instead of running a Dinkelbach update every round, which is why the weights appear as constants. Table III collects the notation. D. Reading One Price Three Ways 1) The Contract: The contract issued to a selected client is
Γtk = qkt , ntk btk , rkt , zkt ptk , stk , etk , (5) | {z } | {z } | {z } participate

precision

place

and all eight variables follow from Ukt . Participate applies the threshold of Proposition 1 and then repairs the coverage floor by admitting the best-priced client of any modality the threshold left empty, and it grants local epochs ntk in proportion to utility, so cheap and informative clients do more local work than expensive and redundant ones.

2) Precision by Reverse Water-Filling: Two variables set what a selected client sends. The uplink rate is allocated in proportion to nonnegative utility, [Ukt ]+ B, t j∈St [Uj ]+

btk = P

(6)

and the number of bits per coordinate follows from the same shadow price that produced δ. Proposition 2 (Utility-proportional allocation). For Ukt > 0P on St , rule Equation (6) of P is thet unique maximizer t t t U log b subject to b ≤ B and b > 0, so k k k∈St k k∈St k it is the weighted proportionally fair allocation of the uplink budget. Proof: The objective is strictly concave and increasing, so the budget binds. Setting the gradient of the Lagrangian to gives Ukt /btk = ν for every k, hence btk ∝ Ukt , and P zero t k bk = B fixes ν. Proportional fairness is what keeps a modality alive under a tight budget. A top-k rule spends the budget on the few clients it ranks highest, whereas Equation (6) gives every client with positive utility a positive rate. Proposition 3 (Reverse water-filling). Let the quantizer obey t c − ∆θ t ∥2 = ds σ 2 2−2zkt , the high-resolution law E∥∆θ k k 2 k where σk2 = (Gtk )2 /ds , and let reconstruction error raise the expected global loss at rate ρ. The precision that maximizes the bracket of Proposition 1 is σk2 λ , (7) , D⋆ = D⋆ 16ρ ln 2 so every client is quantized to one common distortion floor D⋆ fixed by the shadow price of the uplink. zkt⋆ = 12 log2

Proof: At z bits per coordinate the payload is Xkt = ds z/8 bytes, so the bracket contributes −ρds σk2 2−2z −λds z/8, whose second derivative is negative for σk2 > 0. Setting the first derivative to zero gives 2−2z = λ/(16ρσk2 ln 2), which rearranges to Equation (7). Bits are therefore not spent evenly: a client whose update is large enough to move the shared state buys precision, and one whose update is small is coarsened until its distortion reaches the same floor as everyone else’s. Because σk2 grows with Gtk , which enters Equation (4) positively, zkt⋆ increases in the price a client already earned, so the number that admitted the client also sets its precision. The implementation projects zkt⋆ onto the levels it supports and sets rkt by the same monotone map. 3) Place and Aggregate: Place derives the priority ptk from t Uk , the slice stk from ptk , and the edge server as the leastloaded feasible one, etk = arg mine∈E Λte , which is the listscheduling rule whose bounded makespan let the round’s time t f = be treated as separable. A selected client transmits ∆θ k t Q(∆θ k , zkt ), and the server aggregates the reconstructions by local data size,  X t |D | c . P k θ t + ∆θ (8) θ t+1 = k j∈St |Dj | k∈St

Algorithm 1 FedSceneX Round ( Profile , Assign )

Price , and

Input: shared state θ t , clients K, edge servers E, uplink budget B 1: Phase I: Profile ▷ privacy-safe scalars t 2: broadcast θ and run one local warm-up per client 3: collect Pkt , Gtk , Fkt , Xkt , Ltk and normalize them to

[0, 1] ▷ one utility per client

4: Phase II: Price

5: Ukt ← αPkt + γGtk + µFkt − δXkt − ηLtk by (4) 6: St ← {k : Ukt > 0}, then repair the coverage floor

of (2) ▷ participate, precision, place

7: Phase III: Assign

8: for all k ∈ St do 9: btk ← (6); set ntk , rkt , zkt , ptk , stk by the monotone

maps of Ukt 10: etk ← arg mine∈E Λte ; run ntk local epochs; upload Q(∆θ tk , zkt ) 11: end for 12: aggregate the received states by (8) t+1 13: return θ 4) Cost of Orchestration: Pricing is O(|K|) over scalars and assignment adds an O(|K| log |K|) sort. Quantization is O(ds ) per selected client and aggregation is O(|St |ds ). Since dp ≫ ds and local optimization dominates the round, orchestration is not on the critical path: the measured communication latency of a round is 1.99 seconds against a round duration of 3.52 hours. V. E VALUATION A. Experimental Setup Dataset and clients. We evaluate on the complete nuScenes v1.0-trainval split [48]: 850 scenes and 34,149 synchronized samples, partitioned by scene into 28,130 training and 6,019 validation samples. Eight object categories are mapped to multi-hot scene labels, giving a multi-label recognition task over 1,094,848 annotations. Fifteen clients cover the full sensing platform: six cameras, two video streams, two partitions of the top LiDAR, and five radars. Image and video clients use ImageNet-pretrained ResNet-18 backbones, video adds a gated recurrent unit for short-term context, and LiDAR and radar clients encode bird’s-eye-view representations with lightweight convolutional networks. All clients optimize an asymmetric multi-label objective with positive-class weighting using AdamW and cosine decay. The network holds two edge servers sharing a 48 Mbps uplink budget. Baselines. We compare against classical and multimodal federated learning (FedAvg [1], FedProx [39], FedCoLa [8]), heterogeneity-aware optimization (FedDyn [13], FedADMM [14], FedAdam [15]), personalized and regularized methods (FedBABU [16], FedSR [44], MOON [17]), and communication-aware methods (FedCS [24], FedPAQ [22], Oort [38]). Every method uses the same split, preprocessing,

loss, client organization, local budget, and validation schedule, so the reported differences come from the algorithms. Metrics and protocol. The active time of a round is the training and evaluation time of its selected clients. It excludes the idle intervals between rounds and matches the recorded round wall time to within 0.01%. We report accuracy within a budget, meaning the highest Macro-F1 a method reaches inside a given cumulative active time, in place of the time at which it first crosses a fixed accuracy. Both quantities read the same trajectory, but the Macro-F1 curve is nearly flat near 0.66, so the crossing time varies by 24.5 hours across seeds while accuracy at thirty hours varies by 0.6 points. Modality coverage is the mean number of the four sensing groups present in a round. FedSceneX is reported as mean and standard deviation over five seeds. The baselines are single runs of unequal length, so their peak values appear next to the round count that produced them. B. Cost and Accuracy Under a Budget 1) A Round Costs a Factor of Three: FedSceneX spends 3.31±0.13 active hours per round against 4.85 for the cheapest baseline and 9.78 for the dearest, a reduction of 1.5× to 3.0× (Table IV). The spread across five seeds is 4% of the mean, so cheaper rounds are a property of the contract rather than of a favorable run. Nor are they obtained by dropping expensive sensors. FedSceneX keeps all four modality groups in every round of every seed. FedCS is the one baseline that also holds four, and it pays 9.50 MB per round to do so against 2.93 MB for FedSceneX; the rest average 2.56 to 3.19 groups and leave roughly a quarter of them idle. 2) The Lead Holds From Ten to Forty-Five Hours: Within twenty active hours the weakest of the five seeds reaches Macro-F1 0.6486, above the 0.6458 of the strongest baseline at that budget, so the lead at twenty hours does not depend on the seed. Within thirty hours the seed mean is 0.6559 ± 0.0057 against 0.6486, and the weakest seed matches that baseline exactly. Sweeping the budget continuously, the seed mean leads every baseline from ten hours to 42.5 hours (Figure 3(c)). The mechanism is visible in the round counts rather than in the optimizer: at thirty hours FedBABU has finished three rounds and FedSceneX nine. Cheaper rounds convert into more optimization per hour, not into better optimization per round. C. Where the Advantage Ends Beyond forty-five hours the ranking reverses. At fifty hours FedBABU reaches 0.6658 and FedDyn 0.6640, while FedSceneX is at 0.6576 ± 0.0058, and over their full trajectories the same two methods peak at 0.6798 and 0.6819 against 0.6713. The crossover follows from the design. Selective transmission and reduced update precision are what make a round inexpensive, and the same coarseness limits the fine-grained refinement that late-stage optimization needs. FedSceneX is therefore the right choice when a deployment has to reach a usable model under a bounded time budget, and the wrong one when final accuracy matters more than the hours spent reaching it.

TABLE IV: Baseline comparison. FedSceneX: mean±std over five seeds; other rows are single runs of the stated length. ∆ : Macro-F1 points FedSceneX gains at 30 hours. Rnds

Peak Ma-F1

h↓

MB ↓

Mod. ↑

20 h

30 h

50 h

FedAvg [1] [AISTATS’17] FedProx [39] [MLSys’20] FedCoLa [8] [ECCV’24]

100 100 52

0.5632 0.5605 0.4199

4.85 4.94 5.68

22.10 22.10 –

3.19 3.19 3.19

0.4068 0.3854 0.4199

0.4068 0.3854 0.4199

0.4136 0.3854 0.4199

↑ 24.90 ↑ 27.05 ↑ 23.60

FedDyn [13] [ICLR’21] FedADMM [14] [ICDE’22] FedAdam [15] [ICLR’21]

34 40 42

0.6819 0.6807 0.6816

9.78 9.26 9.19

3.80 3.80 3.80

3.03 3.05 3.07

0.6456 0.6455 0.6444

0.6486 0.6486 0.6444

0.6640 0.6638 0.6444

↑ 0.73 ↑ 0.72 ↑ 1.14

FedBABU [16] [ICLR’22] FedSR [44] [NeurIPS’22] MOON [17] [CVPR’21]

31 57 65

0.6798 0.6796 0.6790

9.75 5.72 6.20

3.80 3.80 3.80

3.00 3.07 3.05

0.6446 0.6452 0.6458

0.6447 0.6452 0.6481

0.6658 0.6630 0.6621

↑ 1.11 ↑ 1.06 ↑ 0.78

FedCS [24] [ICC’19] FedPAQ [22] [AISTATS’20] Oort [38] [OSDI’21]

11 13 18

0.6686 0.6615 0.6528

8.57 7.09 5.16

9.50 0.95 3.80

4.00 3.08 2.56

0.6444 0.6400 0.6429

0.6456 0.6433 0.6429

0.6566 0.6503 0.6477

↑ 1.03 ↑ 1.26 ↑ 1.30

FedSceneX

88

0.6713

3.31±0.13

2.93±0.03

4.00

0.6528±0.0040

0.6559±0.0057

0.6576±0.0058

–

FedDyn

FedADMM

FedBABU

MOON

FedCS

FedPAQ

Macro-F1

0.67

0.66 0.65 0.64

0.66 0.65 0.64

5

10

15

20

0

20

40

60

Communication round

Cumulative active time (h)

(a) By round

(b) By active time

Oort

FedSceneX

0.67

Modality coverage

Best Macro-F1 in budget

Method

0.67

Macro-F1

∆

Macro-F1 within budget

0.66

0.65

0.64

crossover

10

20

30

40

50

60

Active-time budget (h)

(c) Under a budget

10

4

8 3

6

2

4

1

2

0

0

Upload (MB/round)

Cost per round

yn M U N S Q ort eX dD DM AB OO edC PA O cen Fe edA FedB M F Fed dS F Fe

(d) Coverage and upload

Fig. 3: The same trajectories read four ways. (c) shades one standard deviation over five seeds and marks the crossover; (d) plots modality coverage as bars and upload volume as squares on the right axis. TABLE V: Ablation, single runs of one seed. ∆ : Macro-F1 points the full contract adds at 30 hours.

TABLE VI: Modality inference from server-visible telemetry alone.

Variant

h/rnd ↓

MB ↓

Mod. ↑

Ma@30 h

∆

Attacker

Accuracy

Macro-F1

Balanced acc.

FedSceneX (full)

3.52

2.88

4.00

0.6600

–

w/o contract w/o fairness Uniform rate Uniform quantization w/o priority and slice w/o edge assignment Random contract w/o video w/o LiDAR w/o radar

9.21 6.55 10.46 10.30 10.46 6.60 6.09 4.17 5.25 6.82

9.50 2.91 7.27 7.27 7.27 2.91 3.80 2.91 2.91 2.91

4.00 4.00 4.00 4.00 4.00 4.00 3.20 3.00 3.00 3.00

0.6447 0.6518 0.6447 0.6446 0.6446 0.6521 0.6488 0.6475 0.6503 0.6504

↑ 1.54 ↑ 0.82 ↑ 1.54 ↑ 1.54 ↑ 1.54 ↑ 0.79 ↑ 1.12 ↑ 1.25 ↑ 0.97 ↑ 0.97

Majority class Random forest Logistic regression

0.3571 0.9702 1.0000

0.1316 0.9706 1.0000

0.2500 0.9760 1.0000

D. Which Parts of the Contract Matter Each family of contract decisions earns its place, and each one on a different axis (Table V). Removing the contract costs 1.54 Macro-F1 points at thirty hours and raises the cost of a round from 3.52 to 9.21 hours. Replacing rate allocation, quantization, or priority with uniform policies costs the same 1.5 points and raises the uploaded state from 2.88 to 7.27 MB,

because every selected client then transmits at full precision. Fairness and edge assignment are cheaper to lose, 0.8 points each, and they act on timing rather than volume: both variants upload the same 2.91 MB as the full contract. Randomizing the contract while keeping the number of selected clients costs 1.12 points and drops modality coverage from 4.00 to 3.20, which isolates the utility ordering rather than the client count as the active ingredient. Removing a sensor is the one change that raises peak accuracy. Without radar the peak reaches 0.6786 against 0.6713, and the trace explains why the utility struggles to price it: radar produces updates nearly as large as video, with a mean norm of 0.75 against 0.77, yet the least learning progress of any modality, 0.0009 against 0.0026 for LiDAR, and the lowest client-level mAP, 0.57 against 0.71 for the cameras. The utility

therefore pays for updates that move the shared state without improving it. Dropping radar also gives up a quarter of the sensing problem, and the resulting model still trails the full contract at thirty hours. E. The Control Channel Is Not Free The contract is built from per-client scalars, and those scalars describe a workload. Workloads differ systematically by modality, so the channel that makes orchestration possible also carries modality identity. We train two attackers on the server-visible telemetry alone, using rounds 1 to 60 and testing on rounds 61 to 88, a chronological split that keeps adjacent observations out of both sets. A logistic-regression attacker recovers the modality of an uploading client with accuracy 1.00 and a random forest with 0.97, against a majority-class baseline of 0.36 (Table VI). This is not a reconstruction of sensor data. Raw observations, labels, embeddings, logits, and private encoder parameters never leave the client, and the shared state is 1.4% of an image client’s parameters and 24.2% of a radar client’s. What the telemetry exposes is a coarse client property, and it is exposed by construction: an orchestrator that prices clients by their workload must read quantities that identify the workload. Telemetry clipping, temporal aggregation, and secure aggregation of the scalars each reduce the exposure at some cost in contract quality. We report the leak rather than assume it away, since any utility-driven orchestrator inherits it. VI. C ONCLUSION In this paper, we have proposed FedSceneX, an orchestration layer that composes each round of a same-scene multimodal federation by what the round will cost. Value-perHour Pricing turns the round’s value-per-hour ratio into one closed-form price per client, whose negative weights are the shadow price of a byte and the worth of an hour, and the same price decides whether the client uploads, at what precision, and on which edge server. Because participation, precision, and placement ask one question, that utility replaces the separate selection, compression, and placement thresholds an edge deployment would otherwise tune by hand. On the complete nuScenes benchmark with fifteen heterogeneous clients, a FedSceneX round costs 3.31 active hours against 4.85 to 9.78 for twelve baselines, every seed leads the strongest baseline within a twenty-hour budget, and all four sensing modalities stay in every round. What FedSceneX buys is not a better model but an earlier one, and we report the hour at which that stops being the better trade. R EFERENCES [1] H. B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. Agüera y Arcas, “Communication-Efficient Learning of Deep Networks from Decentralized Data,” in Proc. AISTATS, 2017. [2] P. Kairouz et al., “Advances and Open Problems in Federated Learning,” Foundations and Trends in Machine Learning, vol. 14, no. 1–2, pp. 1– 210, 2021. [3] Q. Duan, J. Huang, S. Hu, R. Deng, Z. Lu, and S. Yu, “Combining Federated Learning and Edge Computing Toward Ubiquitous Intelligence in 6G Network: Challenges, Recent Advances, and Future Directions,” IEEE Communications Surveys & Tutorials, vol. 25, no. 4, pp. 2892– 2950, 2023.

[4] B. Wu, J. Huang, and Q. Duan, “FedTD3: An Accelerated Learning Approach for UAV Trajectory Planning,” in Proc. WASA, 2025, pp. 13– 24. [5] B. Wu, Z. Ding, J. Huang, and Y. Zhao, “Forget to Improve: On-Device LLM-Agent Continual Learning via Budget-Curated Memory,” arXiv preprint arXiv:2606.25115, 2026. [6] B. Wu and J. Huang, “CrystalMem: Elastic Memory for SelfEvolving LLM Agents via Knowledge Crystallization,” arXiv preprint arXiv:2608.00303, 2026. [7] T. Feng, D. Bose, T. Zhang, R. Hebbar, A. Ramakrishna, R. Gupta, M. Zhang, S. Avestimehr, and S. Narayanan, “FedMultimodal: A Benchmark for Multimodal Federated Learning,” arXiv preprint arXiv:2306.09486, 2023. [8] G. Sun, Y. Cong, J. Dong, Q. Wang, and J. Liu, “Towards Multi-Modal Transformers in Federated Learning,” in Proc. ECCV, 2024. [9] Z. Fang, J. Wang, Y. Ma, Y. Tao, Y. Deng, X. Chen, and Y. Fang, “RACP: Real-Time Adaptive Collaborative Perception Leveraging Robust Task-Oriented Communications,” IEEE Journal on Selected Areas in Communications, 2025. [10] Z. Fang, Y. Guo, J. Wang, Y. Zhang, H. An, Y. Wang, and Y. Fang, “Shared Spatial Memory Through Predictive Coding,” arXiv preprint arXiv:2511.04235, 2025. [11] D. Y. Tchalla, “ST-Hybrid: Dynamic Graph Learning with MultiScale Spatio-Temporal Attention for Traffic Forecasting,” ACM SIGAPP Applied Computing Review, vol. 25, no. 4, pp. 35–52, 2026. [12] B. Wu, Z. Ding, and J. Huang, “PRISM: Exposing and Resolving Spurious Isolation in Federated Multimodal Continual Learning,” arXiv preprint arXiv:2605.01061, 2026. [13] D. A. E. Acar, Y. Zhao, R. Matas, M. Mattina, P. Whatmough, and V. Saligrama, “Federated Learning Based on Dynamic Regularization,” in Proc. ICLR, 2021. [14] Y. Gong, Y. Li, and N. M. Freris, “FedADMM: A Robust Federated Deep Learning Framework with Adaptivity to System Heterogeneity,” in Proc. IEEE ICDE, 2022, pp. 2575–2587. [15] S. J. Reddi, Z. Charles, M. Zaheer, Z. Garrett, K. Rush, J. Konečný, S. Kumar, and H. B. McMahan, “Adaptive Federated Optimization,” in Proc. ICLR, 2021. [16] J. Oh, S. Kim, and S.-Y. Yun, “FedBABU: Toward Enhanced Representation for Federated Image Classification,” in Proc. ICLR, 2022. [17] Q. Li, B. He, and D. Song, “Model-Contrastive Federated Learning,” in Proc. IEEE/CVF CVPR, 2021, pp. 10 713–10 722. [18] Z. Ding, J. Huang, Q. Duan, C. Zhang, Y. Zhao, and S. Gu, “A DualLevel Game-Theoretic Approach for Collaborative Learning in UAVAssisted Heterogeneous Vehicle Networks,” in Proc. IEEE IPCCC, 2025, pp. 1–8. [19] Z. Ding, J. Huang, Y. Zhao, and Z. Cai, “Combating Knowledge Diversity and Catastrophic Forgetting in UAV-Assisted Collaborative Vehicular Learning: A Game-Theoretic Approach,” ACM Transactions on Autonomous and Adaptive Systems, 2026. [20] Z. Ding, J. Huang, and J. Qi, “Learning to Defend: A Multi-Agent Reinforcement Learning Framework for Stackelberg Security Game in Mobile Edge Computing,” in Proc. IEEE ICNC, 2026. [21] B. Wu and W. Wu, “Model-Free Cooperative Optimal Output Regulation for Linear Discrete-Time Multi-Agent Systems Using Reinforcement Learning,” Mathematical Problems in Engineering, vol. 2023, no. 1, p. 6350647, 2023. [22] A. Reisizadeh, A. Mokhtari, H. Hassani, A. Jadbabaie, and R. Pedarsani, “FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantization,” in Proc. AISTATS, 2020. [23] J. Konečný, H. B. McMahan, F. X. Yu, P. Richtárik, A. T. Suresh, and D. Bacon, “Federated Learning: Strategies for Improving Communication Efficiency,” arXiv preprint arXiv:1610.05492, 2016. [24] T. Nishio and R. Yonetani, “Client Selection for Federated Learning with Heterogeneous Resources in Mobile Edge,” in Proc. IEEE ICC, 2019. [25] B. Wu, J. Huang, Q. Duan, L. Dong, and Z. Cai, “Enhancing Vehicular Platooning With Wireless Federated Learning: A Resource-Aware Control Framework,” IEEE/ACM Transactions on Networking, 2025. [26] J. Huang, B. Wu, Q. Duan, L. Dong, and S. Yu, “A Fast UAV Trajectory Planning Framework in RIS-Assisted Communication Systems With Accelerated Learning via Multithreading and Federating,” IEEE Transactions on Mobile Computing, pp. 1–16, 2025.

[27] B. Wu, J. Huang, and S. Yu, “‘X of Information’ Continuum: A Survey on AI-Driven Multi-Dimensional Metrics for Next-Generation Networked Systems,” IEEE Communications Surveys & Tutorials, vol. 28, pp. 5307–5344, 2026. [28] B. Wu, Z. Cai, W. Wu, and X. Yin, “AoI-Aware Resource Management for Smart Health via Deep Reinforcement Learning,” IEEE Access, 2023. [29] L. Dong, J. Huang, and R. W. Heath, “Transformer-Based Dynamic Resource Allocation for Multi-Carrier NOMA Systems,” IEEE Transactions on Cognitive Communications and Networking, vol. 12, pp. 4926– 4941, 2026. [30] U. Pudasaini, Z. Ding, and J. Huang, “Securing Smart Agriculture with Communication-Efficient Federated Unlearning,” in Proc. IEEE HPSR, 2026, pp. 1–8. [31] B. Wu, Z. Ding, L. Ostigaard, and J. Huang, “Reinforcement LearningBased Energy-Aware Coverage Path Planning for Precision Agriculture,” in Proc. ACM RACS, 2025, pp. 1–8. [32] C.-C. Xing, Z. Ding, and J. Huang, “A Stochastic Geometry-Based Analysis of SWIPT-Assisted Underlaid Device-to-Device Energy Harvesting,” ACM SIGAPP Applied Computing Review, vol. 25, no. 4, pp. 18–34, 2026. [33] D. Pan, B.-N. Wu, Y.-L. Sun, and Y.-P. Xu, “A Fault-Tolerant and Energy-Efficient Design of a Network Switch Based on a QuantumBased Nano-Communication Technique,” Sustainable Computing: Informatics and Systems, vol. 37, p. 100827, 2023. [34] Z. Fang, S. F. Hu, Z. Chang, Y. Guo, Y. Tao, H. Liu, M. Ruan, J. Huang, and Y. Fang, “Inference-Time Budget Control for LLM Search Agents,” arXiv preprint arXiv:2605.05701, 2026. [35] L. Yuan, D.-J. Han, V. P. Chellapandi, S. H. Żak, and C. G. Brinton, “FedMFS: Federated Multimodal Fusion Learning with Selective Modality Communication,” arXiv preprint arXiv:2310.07048, 2023. [36] B. Wu, Z. Ding, and J. Huang, “RELIEF: Turning Missing Modalities into Training Acceleration for Federated Learning on Heterogeneous IoT Edge,” arXiv preprint arXiv:2604.04243, 2026. [37] Z. Ding, B. Wu, and J. Huang, “EASE: Federated Multimodal Unlearning via Entanglement-Aware Anchor Closure,” arXiv preprint arXiv:2605.00733, 2026. [38] F. Lai, X. Zhu, H. V. Madhyastha, and M. Chowdhury, “Oort: Efficient Federated Learning via Guided Participant Selection,” in Proc. USENIX OSDI, 2021.

[39] T. Li, A. K. Sahu, M. Zaheer, M. Sanjabi, A. Talwalkar, and V. Smith, “Federated Optimization in Heterogeneous Networks,” in Proc. MLSys, 2020. [40] D. Chen, J. Hu, and V. J. Tan, “Elastic Aggregation for Federated Optimization,” in Proc. IEEE/CVF CVPR, 2023, pp. 12 187–12 197. [41] X. Li, M. Jiang, X. Zhang, M. Kamp, and Q. Dou, “FedBN: Federated Learning on Non-IID Features via Local Batch Normalization,” in Proc. ICLR, 2021. [42] L. Collins, H. Hassani, A. Mokhtari, and S. Shakkottai, “Exploiting Shared Representations for Personalized Federated Learning,” in Proc. ICML, 2021, pp. 2089–2099. [43] Y. Guo, K. Guo, X. Cao, T. Wu, and Y. Chang, “Out-of-Distribution Generalization of Federated Learning via Implicit Invariant Relationships,” in Proc. ICML, 2023, pp. 11 905–11 933. [44] A. T. Nguyen, P. Torr, and S. N. Lim, “FedSR: A Simple and Effective Domain Generalization Method for Federated Learning,” in Proc. NeurIPS, 2022. [45] B. Wu, Z. Ding, and J. Huang, “A Review of Continual Learning in Edge AI,” IEEE Transactions on Network Science and Engineering, vol. 13, pp. 6571–6588, 2026. [46] B. Wu and J. Huang, “Lifecycle-Aware Federated Continual Learning in Mobile Autonomous Systems,” arXiv preprint arXiv:2604.20745, 2026. [47] B. Wu, J. Huang, and Y. Zhao, “From Alpha to Omega: LifecycleAware Forgetting Defense in Federated Continual Learning for Planetary Exploration,” in Proc. IEEE ICDCS, 2026. [48] H. Caesar et al., “nuScenes: A Multimodal Dataset for Autonomous Driving,” in Proc. IEEE/CVF CVPR, 2020. [49] B. Wu, J. Huang, and Q. Duan, “Real-Time Intelligent Healthcare Enabled by Federated Digital Twins With AoI Optimization,” IEEE Network, vol. 40, no. 2, pp. 184–191, 2026. [50] Z. Ding, B. Wu, J. Huang, and S. Mao, “Application-Aware Twin-inthe-Loop Planning for Federated Split Learning over Wireless Edge Networks,” arXiv preprint arXiv:2604.26105, 2026. [51] Z. Ding, B. Wu, and J. Huang, “SCALE: Sensitivity-Aware Federated Unlearning with Information Freshness Optimization for Mobile Edge Computing,” in Proc. IEEE ICDCS, 2026. [52] W. Dinkelbach, “On Nonlinear Fractional Programming,” Management Science, vol. 13, no. 7, pp. 492–498, 1967. [53] R. L. Graham, “Bounds for Certain Multiprocessing Anomalies,” Bell System Technical Journal, vol. 45, no. 9, pp. 1563–1581, 1966.

Related documents

• Enabling Multi-Task Neural Network Inference on Heterogeneous Edge Devices#417680
• HO-SFL: Hybrid-Order Split Federated Learning with Backprop-Free Clients and Dimension-Free Aggregation#1118497
• DART-FL: Burst-Aware Multitask Federated Learning under Dynamic Inference Demand at the Edge#781010
• DART-FL: Burst-Aware Multitask Federated Learning under Dynamic Inference Demand at the Edge#779635
• HeteroMosaic: Exposing and Exploiting Heterogeneous Execution Opportunities for Energy-Efficient Edge LLM Inference#366223