Distributed Optimization with Streaming Data: A Temporal Weighting Perspective

1

Distributed Optimization with Streaming Data: A Temporal Weighting Perspective

arXiv:2608.09565v1 [eess.SP] 10 Aug 2026

Muhammad Faraz Ul Abrar, Graduate Student Member, IEEE, Nicolò Michelusi, Senior Member, IEEE, and Erik G. Larsson, Fellow, IEEE Abstract—Optimization theory is a widely used tool for intelligent decision-making. While classical optimization deals with fixed, time-invariant objective functions, many modern applications operate in dynamic environments where data arrive sequentially, and the learning objective evolves over time, often under decentralized data and communication constraints. Motivated by these trends, we study decentralized optimization from streaming data through a structured time-varying formulation in which the global objective is a temporally weighted average of losses observed across the network. We analyze multi-iteration decentralized first-order methods, including decentralized gradient descent and diffusion, where only a fixed number of communication/gradient steps can be performed before new samples arrive. For strongly convex and smooth losses, we develop guarantees for the Euclidean-norm tracking error through a contraction-mapping viewpoint. The resulting bounds decompose the tracking error into a fixed-point tracking component and a bias term induced by decentralization and data heterogeneity. We specialize our analysis to uniform and exponentially discounted weights, as well as their finite-memory windowed counterparts. The bounds explicitly characterize the roles of the temporal weighting rule, per-step iteration budget, step size, and network connectivity. Uniform weighting yields a vanishing fixed-point tracking contribution of order O(1/t), whereas discounted and windowed strategies generally induce non-vanishing tracking floors governed by the discount factor and effective memory, respectively. In all cases, decentralization induces an additional non-zero bias floor under a constant step size. Numerical experiments illustrate the predicted trends. Index Terms—Decentralized optimization, streaming data, time-varying optimization, DGD, tracking error, online learning.

I. I NTRODUCTION Modern artificial intelligence (AI) systems increasingly rely on learning and optimization tools for intelligent decisionmaking across domains such as autonomous vehicles, robotics, telecommunications, power grids, and cyber-physical systems [2], [3]. Despite their success, many existing AI systems follow an optimize-and-deploy paradigm, where a model is trained for a fixed objective under an implicitly static data distribution. In practice, however, many applications are both dynamic and decentralized: data arrive sequentially over time [4]–[6], while storage, computation, and communication are distributed across multiple agents due to scale, privacy, or architectural constraints [7]–[10]. Representative examples include mobile target localization, measurement-based network optimization, streaming data processing, and adaptive control under time-varying dynamics [3], [4], [11], [12]. These settings M. Faraz Ul Abrar and N. Michelusi are with the School of Electrical, Computer and Energy Engineering, Arizona State University. email: {mulabrar, nicolo.michelusi}@asu.edu. Erik G. Larsson is with the Department of Electrical Engineering (ISY), Linköping University, 58183 Linköping, Sweden. e-mail: erik.g.larsson@liu.se. This research was funded in part by NSF under grant CNS-2129015. The work of E. G. Larsson was supported in part by ELLIIT, VR, and the KAW foundation. A preliminary version of this work is available in [1].

call for learning systems that continuously adapt to incoming information while operating cooperatively over a network. Time-varying optimization provides a natural framework for such problems. A network of agents collaboratively tracks the minimizer of an objective that evolves over time, typically while performing only a limited number of optimization and communication steps before the objective changes again [3], [4]. Exact tracking is therefore generally infeasible, and performance is naturally measured through the tracking error (TE), defined at each time step as the Euclidean distance between the current iterate and the corresponding optimizer. Existing decentralized time-varying optimization methods have primarily focused on algorithmic developments for improving TE performance [12]–[20]. However, these works treat the objective evolution as generic and largely unstructured. In these formulations, the sequence of objectives is usually specified directly as an arbitrary time-varying process and worst-case TE guarantees are derived under generic “minimizer drift” assumptions. While broadly applicable, this viewpoint treats the temporal evolution as exogenous and unstructured, and may therefore be conservative when the objective changes according to a specific mechanism. In many modern learning problems, the objective does not evolve arbitrarily. Rather, it changes because new data samples are continuously acquired across the network, see [21] and references therein. The objective evolution therefore inherits structure from the underlying data-acquisition process. This observation motivates the central question addressed in this paper: Can the streaming-data structure be explicitly incorporated into the optimization model to obtain sharper and more interpretable tracking guarantees? To address this question, in this work, we study decentralized time-varying optimization from a streaming-data perspective. Each agent acquires data online, and the network objective at time t is modeled as a temporally weighted average of losses over the samples observed so far. By explicitly encoding the temporal relevance of past data through the weights, the resulting formulation yields weight-specific tracking guarantees that sharpen generic worst-case drift-based bounds. Building on this structured formulation, we analyze decentralized firstorder methods under limited computation and communication per time step. In particular, we develop a unified TE analysis for a general class of temporal weighting strategies and specialize the results to two natural choices: (i) uniform weights, which assign equal importance to all past samples and are natural for stationary environments, and (ii) exponentially discounted weights, which geometrically discount old samples and emphasize recent observations. We further specialize the guarantees to their finite-memory windowed variants. The resulting bounds explicitly characterize the roles of the temporal

2

weighting rule, per-step iteration budget, network connectivity, step size, and data heterogeneity. For smooth and strongly convex losses, our analysis reveals a vanishing fixed-point tracking contribution of order O(1/t) under uniform weighting and generally non-vanishing TE floors under discounted and finite-memory windowed strategies.

A. Related Work The field of time-varying optimization traces back to early works on non-stationary optimization [22]–[24], primarily in the single-agent setting. For smooth and strongly convex objectives, classical results establish tracking to a neighborhood whose size depends on a bounded-minimizer-drift or boundedgradient-variation condition [3], [22], [24], [25]. Methods that solely utilize the objective available at time t to track the drifting minimizer are often termed correction-only schemes [3]. In contrast, prediction-correction schemes first predict the next optimizer using the objective at time t (e.g., via a first-order optimality condition), and then perform a correction update once the objective at time t + 1 becomes available, leading to improved asymptotic tracking performance [15], [25]–[27]. Related prediction mechanisms based on parameter estimation, including Kalman-filter and neural-network predictors, have also been studied [28], [29]. Time-varying optimization has also been investigated in decentralized multi-agent settings using primal methods, such as decentralized gradient descent (DGD) [30], and primal-dual methods, such as alternating direction method of multipliers (ADMM) [31]; see, e.g., [13]– [20], [32]. Nonetheless, these works characterize temporal variation through generic worst-case drift measures and treat the objective evolution as exogenous. In contrast, the present formulation ties the evolution of the objective explicitly to streaming data through the temporal weights. Another closely related research direction is online learning [5], [33]–[35]. In online learning, a learner chooses the decision before observing the current loss, and performance is commonly measured through regret against a fixed or timevarying comparator [36]–[38]. The paradigm of continual learning similarly considers sequential data or “task” arrivals, with a primary focus on catastrophic forgetting and methods for retaining previously learned knowledge [6], [39]–[43]. While these directions are related through their sequentialdata viewpoint, they primarily address information availability, regret, or task adaptation. Our focus is instead on trackingerror guarantees for decentralized optimization algorithms operating under communication and computation constraints. Learning from data streams has also been studied in servercoordinated federated settings [44]–[46]. Among these, [45] is closest in spirit to our work, as it considers memoryconstrained learning from streaming data and assigns relative importance weights to samples. Yet a key distinction is that [45] adopts a statistical learning viewpoint: it designs the weights to improve generalization error under distributional assumptions. In contrast, we provide an optimization-theoretic, weight-specific characterization of the TE of decentralized first-order methods, without imposing a data-generating distribution. Moreover, the setting considered in this work is fully decentralized and does not rely on a coordinating server.

B. Contributions and Organization Our main contributions are summarized as follows: • We extend our single-agent formulation in [47] to decentralized multi-agent networks and formulate decentralized learning from streaming data samples through temporally weighted objectives. We introduce a structured family of kernel-parameterized temporal weighting strategies, including uniform and exponentially discounted weights, together with their finite-memory windowed variants. • Leveraging the contraction-mapping framework of [48], we develop a unified TE analysis for multi-iteration DGD and diffusion under limited communication and computation per time step. • We develop TE guarantees that decompose the overall error into a fixed-point tracking component and a bias term capturing the effect of data heterogeneity across the network. The resulting bounds explicitly characterize the dependence on the per-step iteration budget, network connectivity, step size, and temporal weighting rule. • We specialize the TE guarantees to four canonical weighting strategies and use numerical experiments to illustrate the predicted effects of key system parameters and temporalweighting choices. The remainder of the paper is organized as follows. Section II presents the system model and streaming-data objective. Section III introduces the temporal weighting strategies. Section IV describes the decentralized algorithms and performance metrics. Section V develops the TE analysis. Section VI presents numerical results. Section VII concludes the paper. C. Notation Scalars are denoted by italic letters (e.g., a), vectors by boldface lowercase letters (e.g., w), and matrices by boldface uppercase letters (e.g., A). Bold sans-serif lowercase letters are reserved for network-stacked vectors. The all-ones vector in RN is denoted by 1N . For a vector or matrix, (·)⊤ denotes its transpose, and the Euclidean norm is denoted by ∥ · ∥. For   ⊤ ⊤ a stacked vector w = w1⊤ · · · wN ∈ RN d , the notation d [w]n ∈ R denotes its n-th block. The identity matrix of dimension q is denoted by Iq ∈ Rq×q , and ⊗ denotes the Kronecker product. For a matrix A, ∥A∥2 denotes its spectral norm and A† its Moore–Penrose pseudoinverse. The symbols R(A) and N (A) denote the range and nullspace of A, respectively. If A is symmetric, A ⪰ 0 means that A is positive semidefinite, and λmin (A) and λmax (A) denote its minimum and maximum eigenvalues. The indicator function of an event ω is denoted by 1{ω}. For a scalar x ∈ R, we define its positive and negative parts as x+ ≜ max{x, 0} and x− ≜ max{−x, 0}. For functions f (ϵ) and g(ϵ), we write f (ϵ) = O(g(ϵ)) if f (ϵ) is upper bounded by a constant multiple of g(ϵ) as ϵ → 0, and f (ϵ) = Θ(g(ϵ)) if f (ϵ) is both upper and lower bounded by constant multiples of g(ϵ) as ϵ → 0. II. S YSTEM M ODEL AND P ROBLEM F ORMULATION We consider a network of N agents collaboratively solving a time-varying optimization problem induced by streaming data, as shown in Fig. 1. At each time step t ≥ 1, agent n ∈ {1, . . . , N } observes a new data sample and incurs the

3

instantaneous loss ℓn,t (w) evaluated at the model parameter w ∈ Rd . The network seeks to track the minimizer of a suitably constructed weighted average of the accumulated losses (discussed next), under a limited communication/computation budget per time index. The agents communicate over an undirected, connected, time-invariant graph. In particular, the communication among agents is described by a symmetric mixing matrix M ∈ RN ×N . Specifically, [M]m,n is the weight used by agent m to combine information from agent n, and [M]m,n = 0 whenever agents m and n do not directly communicate. We assume that M satisfies M = M⊤ ,

M1N = 1N .

(1)

We further assume that the eigenvalues of M lie in (−1, 1] and are ordered as 1 = λ1 > λ2 ≥ · · · ≥ λN > −1.1 We next define the streaming-data objective. The networkwide instantaneous loss at time t is Ft (w) ≜

N 1 X

N n=1

ℓn,t (w),

(2)

capturing the average loss computed across all the agents at time t. Building on the single-agent formulation in [47], we define the time-t objective as a temporally weighted average of the past network-wide losses. Let {ai (t)}ti=1 represent a nonnegative temporal weighting strategy that satisfies ai (t) ∈ [0, 1], ∀i ≤ t,

t X

ai (t) = 1

(3)

i=1

for all t ≥ 1. Several strategies that satisfy (3) are discussed in the next section. This leads to the following time-varying optimization problem: w∗t ∈ arg min F t (w) , w∈Rd

F t (w) ≜

t X

ai (t) Fi (w),

(4)

i=1

defined for all t ≥ 1. We emphasize that the objective F t (·), and hence its minimizer w∗t , evolves as new samples arrive across the network. Our goal is to characterize how well decentralized methods, such as DGD [30] and diffusion [11], track this moving minimizer when only a limited number of communication/computation iterations can be performed per time step. Before presenting the decentralized algorithms and the associated tracking guarantees, we first discuss the temporal weighting strategies used in (4). III. T EMPORAL W EIGHTS FOR S TREAMING L OSSES The temporal weights {ai (t)}ti=1 in (4) determine how past samples contribute to the current objective and hence control the stability–plasticity tradeoff in streaming environments [6]. We present two structured ways to generate such weights, and begin with a structured family of temporal weightings induced by kernel sequences, which includes several canonical choices. 1 Sufficient conditions for this spectral property, e.g., in terms of nonnegativity of M and non-bipartiteness of the underlying graph, are discussed in [48].

Fig. 1. Illustration of decentralized learning from streaming data. Each agent receives a local stream of samples and exchanges information with neighboring agents over a communication graph. The network tracks the minimizer of a temporally weighted objective formed from past losses.

A. Stationary kernel-induced temporal weights Let {gk }k≥0 be a nonnegative kernel sequence with g0 > 0, and define the associated kernel-induced temporal weights for all t ≥ 1 as ai (t) ≜

gt−i , St

St ≜

t−1 X

gk ,

i = 1, . . . , t.

(5)

k=0

Here, St normalizes the weights so the probability simplex constraint (3) holds. Note that the index k = t−i can be interpreted as the age of sample i at time t. Since ai (t) ∝ gt−i , the kernel {gk } specifies how sample importance varies with age, and (5) implements a simple shift-and-normalize weighting rule. Several choices follow as special cases: • Uniform Weights: Setting gk = 1 for all k ≥ 0 yields ai (t) =

1 , t

i = 1, . . . , t.

(6)

This rule is natural in stationary environments. • Exponentially discounted weights: Setting gk = γ k with a discount factor γ ∈ (0, 1) results in ai (t) =

(1 − γ)γ t−i , 1 − γt

i = 1, . . . , t.

(7)

This rule emphasizes recent samples and can be naturally employed for adaptation in non-stationary environments; see, e.g., [4] for several applications. • Windowed-uniform weights: A finite-memory variant of the uniform weighting strategy assigns equal weight only to the most recent m samples, corresponding to gk = 1{0 ≤ k ≤ m − 1}: ( 1 , t − m + 1 ≤ i ≤ t, for t ≥ m. (8) ai (t) = m 0, otherwise, • Windowed-discounted weights: Similarly, choosing gk = γ k 1{0 ≤ k ≤ m − 1} yields ( (1−γ)γ t−i 1−γ m , t − m + 1 ≤ i ≤ t, ai (t) = for t ≥ m. 0, otherwise. (9) This provides a finite-memory approximation of exponentially discounted weighting using only the last m samples.

4

B. Uniform-shrinkage weighting The stationary kernel family (5) assigns weights based on age using a fixed time-invariant kernel sequence. Another possible construction is to generate the weights recursively by shrinking all previous weights by the same factor whenever a new sample arrives. Let {βt }t≥1 satisfy β1 = 1 and βt ∈ (0, 1) for t ≥ 2, and define the time-varying objective function as F t+1 (w) = (1 − βt+1 ) F t (w) + βt+1 Ft+1 (w),

(10)

for all t ≥ 1. Equivalently, the induced weights satisfy ai (t) = βi

t Y

(1 − βk ),

i = 1, . . . , t.

(11)

k=i+1

This family recovers uniform weighting with βt+1 = 1/(t+1) and exponential discounting with βt+1 = (1 − γ)/(1 − γ t+1 ). Notably, (11) also allows time-adaptive weighting strategies. For example, one may choose βt to decay from an initially constant level toward 1/t, so that the objective first emphasizes recent samples for fast adaptation and then gradually approaches uniform weighting for stability. Such schedules depend on absolute time, not only on sample age, and are therefore not generally representable by a stationary kernel. On the other hand, since βt ∈ (0, 1) preserves strictly positive weights for all past samples, this construction does not capture the finite-window rules in (8)–(9).

=

N X

(k)

[M]n,m wm,t −η

m=1

t+1 X

(k)
ai (t + 1)∇ℓn,i wn,t ,

i=1

(14) whereas diffusion performs the adaptation and combination: (k+1)

e n,t w

(k)

= wn,t − η

t+1 X

(k)
ai (t + 1)∇ℓn,i wn,t ,

(15)

(k+1)

(16)

i=1 (k+1)

=

N X

e m,t , [M]n,m w

m=1

We now present the decentralized methods used to solve (4) and define the associated performance metrics that are adopted throughout the paper. We study two decentralized first-order algorithms: decentralized gradient descent (DGD) [30] and adapt-then-combine (ATC) diffusion [11].2 Each agent n maintains a local parameter wn,t ∈ Rd at time t. Upon receiving new data samples, the objective is updated, after which the agents perform E ≥ 1 decentralized iterations before the time advances. Each decentralized iteration consists of one mixing and one local gradient step. This models a fixed communication/computation budget per time step. Let wt ∈ RN d denote the stacked vector of local parameters,  ⊤  ⊤ ⊤ · · · wN,t wt ≜ w1,t with wn,t = [wt ]n . Next, define the block-separable instantaneous objective N X

(k+1)

wn,t

wn,t

IV. A LGORITHM AND P ERFORMANCE M ETRICS

ft (w) ≜

The objectives F t (·) in (4) and f t (·) in (12) encode the same collection of sample losses {ℓn,i }. However, F t (·) is defined over a common decision parameter w ∈ Rd , whereas f t (·) is defined over the stacked collection of local parameters w ∈ RN d which need not be consensual. In particular, on the consensus subspace : w ∈ Rd } ⊆ RN d , we Pt{1N ⊗ w PN have f t (1N ⊗ w) = i=1 ai (t) n=1 ℓn,i (w) = N F t (w). Thus, on the consensus subspace, the two objectives differ only by the constant factor N and and their minimizers correspond through the mapping w 7→ 1N ⊗ w. Finally, we let w∗t ≜ 1N ⊗ w∗t ∈ RN d denote the stacked version of the global minimizer in (4). After the new samples arrive at time t+1, the agents perform E decentralized iterations on the updated objective. For DGD, the k-th iteration at agent n is

for k = 0, . . . , E − 1. Stacking the local parameters across the network, the DGD and diffusion updates can be expressed in the unified form (k+1)

wt

(k)

= Mwt

(k)

− ηZ∇f t+1 (wt ),

(17)

where η > 0 is a constant step size, and we define M ≜ M ⊗ Id , Z ≜ Z ⊗ Id ∈ RN d×N d and ( I, DGD, Z≜ (18) M, diffusion. (0)

For both methods, wt = wt , and the macro-update (the stacked local parameter update) after E iterations is

ℓn,t (wn ), (E)

wt+1 ≜ wt .

n=1

(19)

and its temporally weighted version f t (w) ≜

t X i=1

ai (t) fi (w) =

t X i=1

ai (t)

N X

ℓn,i (wn ),

(12)

n=1

whose gradient has the block-stacked form, and is given by   ∇ℓ1,i (w1 ) t t X X   .. ai (t) ∇fi (w)= ai (t)  ∇f t (w)= . . i=1 i=1 ∇ℓN,i (wN ) (13) 2 Throughout the paper, “diffusion” refers to ATC diffusion, which is closely related to combine-then-adapt diffusion; see [48] for a discussion.

When E = 1, (17)–(19) reduce to the standard single-step DGD/diffusion updates applied to the time-varying objective. In the considered streaming setting, the objective changes whenever new data arrive; therefore, the stacked iterates {wt } generally do not converge. We therefore measure performance through the tracking error TE(t) ≜ ∥wt − w∗t ∥ = ∥wt − 1N ⊗ w∗t ∥ ,

(20)

and the asymptotic tracking error ATE ≜ lim sup ∥wt − w∗t ∥ . t→∞

(21)

5

V. T RACKING E RROR A NALYSIS In this section, we analyze the decentralized updates (17)– (19) to solve the streaming time-varying problem (4). Following [48], we adopt a contraction-mapping viewpoint, which enables a unified TE analysis for both DGD and diffusion. We first derive generic TE bounds and then specialize them to the temporal weighting strategies in Section III. Define the time-varying decentralized update operator ϕt :RN d →RN d as ϕt (w) ≜ Mw − ηZ∇f t (w),

(22)

where M = M ⊗ Id and Z = Z ⊗ Id . The decentralized updates in (17)–(19) with E iterations at time t can then be written as wt+1 = (ϕt+1 ◦ · · · ◦ ϕt+1 )(wt ) ≜ Φt+1 (wt ), | {z }

(23)

E times

where we use Φt (·) to denote the composition of the mapping ϕt (·) applied E times. For the analysis, we make the following standard assumptions; see, e.g., [48]–[50]. Assumption 1. For every agent n and time t, the loss ℓn,t (·) is L-smooth and µ-strongly convex. Then, for all x, y ∈ Rd , µ∥x − y∥ ≤ ∥∇ℓn,t (x) − ∇ℓn,t (y)∥ ≤ L∥x − y∥

(24)

We use κ ≜ L/µ with κ ≥ 1 to denote the condition number of the optimization problem. Since Ft (·) and F t (·) are convex combinations of the losses at time t and up to time t, respectively, both are also L-smooth and µ-strongly convex for all t ≥ 1. The same properties also hold for the blockseparable objectives ft (·) and f t (·). That is, for all u, v ∈ RN d and for either choice f˜t ∈ {ft , f t }, ∇f˜t (u) − ∇f˜t (v) ≤ L∥u − v∥, (25) µ f˜t (v) ≥ f˜t (u) + ∇f˜t (u)⊤ (v − u) + ∥v − u∥2 . (26) 2 Under Assumption 1, the analysis in Section IV.C of [48] gives the following lemma. Lemma 1. Suppose Assumption 1 holds. Then, ϕt (·) is a contraction for every t, and ∥ϕt (u) − ϕt (v)∥ ≤ (1 − ηµ)∥u − v∥,

∀ u, v ∈ RN d . (27)

under the step size conditions 0 < η ≤ (1 + λN )/(L + µ) for DGD and 0 < η ≤ 2/(L + µ) for diffusion. Consequently, Φt is a contraction with factor α ≜ (1 − ηµ)E ∈ (0, 1). Throughout the rest of the analysis, we assume the step size conditions of Lemma 1. Hence, by Banach’s fixed-point theorem [51], each ϕt (·) admits a unique fixed point, given by Nd

et ∈ R w

s.t.

e t = ϕt (w e t ). w

TE(t) =

et + ≤ wt − w | {z } | FPTE(t)

e t − w∗t w {z

FP bias

∗ Assumption 2. For each agent n and time t, let wn,t ∈ ∗ arg minw∈Rd ℓn,t (w). There exists C > 0 such that ∥wn,t ∥≤ C for all n and t.

Assumption 2 is a structural condition on the sample losses, rather than a direct bounded-drift assumption on the minimizer sequence, as commonly used in time-varying optimization; see, e.g., [3]. Similar bounded-minimizer conditions have appeared in streaming and federated learning analyses [46], [47]. Here, it ensures that the time-varying minimizer {w∗t } e t } are uniformly bounded. and the fixed-point sequences {w Lemma 2. Under Assumptions 1 and 2, it holds that √ ∥w∗t ∥ ≤ C κ.. e t } satisfy, for all t ≥ 1, Moreover, the fixed points {w √ e t ∥ ≤ N CZ , where ∥w  √  C κ, Z = I (DGD), CZ ≜ Cκ, Z = M (diffusion),  C √κ, Z = M, M ⪰ 0.

. }

(29)

(30)

(31)

Furthermore, for all i, t ≥ 1, e i ) , ∥∇ft (w e i )∥ , ∇f t (w∗t ) ≤ GZ . ∇f t (w √ where GZ ≜ 2L N CZ .

(32)

The proof of Lemma 2 can be found in Appendix. We note that the condition M ⪰ 0 is not required by the algorithms or by the TE analysis; it only yields a tighter bound in (31) for diffusion. A. Fixed-point bias To bound the TE, we first bound the bias term in (29). The following result is obtained by applying the fixed-pointto-minimizer bound of [48, App. A] to f t (·) and then using Lemma 2. Proposition 1. Under the conditions above, e t − w∗t ∥ ≤ ηκΛZ ∇f t (w∗t ) ≤ ηκΛZ GZ , ∥w

(33)

where

(28)

e t = Φt (w e t ). Since Φt (·) is the E-fold composition of ϕt (·), w e t } provides a natural reference trajectory The sequence {w for the analysis: it represents the network state to which the decentralized method would converge if the objective f t (·) were frozen at time t. Leveraging this perspective, the tracking error in (20) can be upper-bounded as wt − w∗t

The fixed-point tracking error (FPTE) above captures the ability of the algorithms to track a moving fixed point under a limited per-time-step iteration budget. The second term is the fixed-point bias, capturing the distance between the fixed point and the minimizer at time t. To bound these two terms, we next impose the following boundedness condition on the sample-wise minimizers.

ΛZ ≜

  ∥(I − M)† ∥2 =

1 , Z=I 1 − λ2

 2∥(I − M)† M∥ , 2

(DGD), (34)

Z = M (diffusion).

For diffusion, the first inequality additionally requires η ≤ 1/(LΛZ ). The bounds in (33) show that the fixed point bias scales linearly with the step size η and depends on the condition number κ, the topology factor ΛZ , and the gradient magnitude ∥∇f t (w∗t )∥, which reflects data heterogeneity across agents.

6

In particular, in homogeneous settings where all agents observe identical losses, they share the same minimizer, and hence ∥∇f t (w∗t )∥ = 0, implying zero bias. Better network connectivity reduces ΛZ ; for the complete graph M = N1 11⊤ , one obtains ΛZ = 1 for DGD and ΛZ = 0 for diffusion.

This property is satisfied by the uniform-shrinkage family in Section III-B; it is also satisfied by the stationary-kernel family in Section III-A whenever the kernel sequence {gk } is nonincreasing. Therefore, it holds for the four canonical weighting strategies considered in Section III, whose drift bounds are summarized next.

B. Fixed-point tracking error and drift

Proposition 2. Under (41), the following bounds hold. • Uniform weights in (6):

We next bound the fixed-point tracking term
in (29). Since e t+1 = Φt+1 w e t+1 , the contraction wt+1 = Φt+1 (wt ) and w of Φt+1 (·) from Lemma 1 yields FPTE(t + 1) e t+1 ∥ = ∥wt+1 − w e t+1 )∥ = ∥Φt+1 (wt ) − Φt+1 (w e t+1 ∥ ≤ αFPTE(t) + α∥w e t+1 − w e t ∥, ≤ α∥wt − w

e 1∥ + FPTE(t) ≤ αt ∥w0 − w

e i+1 − w e i ∥. αt−i ∥w

(35)

(36)

i=1

In (36), the first term decreases geometrically to zero as t → ∞, while the second term accumulates the drift of the e t }. Thus, controlling the FPTE reduces moving fixed-point {w to controlling this drift. We next bound this drift. e t } satisfy Lemma 3. Under Assumption 1, the fixed points {w e t+1 − w e t∥ ≤ ∥w

1 e t+1 ) − ∇f t (w e t+1 ) . ∇f t+1 (w µ

(37)

The proof is given in Appendix. Lemma 3 relates the fixedpoint drift to the temporal variation of the gradient of the timeweighted objective. In particular, if ∇f t (·) changes slowly e t} with t, then the corresponding fixed-point sequence {w cannot drift rapidly. Next, using ∇f t+1 (w)−∇f
t (w) = at+1 (t+1)∇ft+1 (w)+ Pt a (t + 1) − a (t) ∇fi (w), the gradient bound in i i i=1 Lemma 2 and applying the triangle inequality, (37) implies e t+1 − w e t∥ ≤ ∥w

t  X GZ  at+1 (t + 1) + |ai (t + 1) − ai (t)| . µ i=1 (38)

The following identity further simplifies (38); its proof is given in the Appendix.
Pt Lemma 4. Let It ≜ i=1 ai (t + 1) − ai (t) + . Then, t X

|ai (t + 1) − ai (t)| = at+1 (t + 1) + 2It .

2GZ . µ(t + 1)

(39)

2GZ 1 − γ . µ 1 − γ t+1 • Windowed-uniform weights in (8): For all t ≥ m, e t+1 − w e t∥ ≤ ∥w

2GZ . µm • Windowed-discounted weights in (9): For all t ≥ m, e t+1 − w e t∥ ≤ ∥w

Leveraging Lemma 4, the fixed-point drift in (38) can be rewritten as
2GZ e t+1 − w e t∥ ≤ at+1 (t + 1) + It . (40) ∥w µ Moreover, if the weights are non-increasing over time, i.e., ai (t + 1) ≤ ai (t), ∀i ≤ t, then It = 0, and (40) simplifies to 2GZ at+1 (t + 1). µ

(41)

(43)

(44)

2GZ 1 − γ . (45) µ 1 − γm Substituting these drift bounds into (36), and combining with the bias bound in (33), yields the TE guarantees discussed next. e t+1 − w e t∥ ≤ ∥w

C. Tracking error guarantees We next present the TE bounds specialized to the four canonical weighting strategies. All bounds are stated under the step size conditions of Lemma 1 and, for diffusion, the additional condition in Proposition 1. Uniform temporal weights: For the uniform temporal weighting strategy (6), utilizing (42), the following holds e 1∥ + TE(t) ≤ αt ∥w0 − w

t−1 2GZ X αt−i + ηκΛZ GZ . (46) µ i=1 i + 1

The summation term in (46) admits an O(1/t) upper bound, yielding the following theorem. Pt−1 t−i Theorem 1. Define S(t) ≜ i=1 αi+1 with α = (1 − ηµ)E ∈ (0, 1). Under uniform  2α weights (6), the tracking error satisfies for all t ≥ t0 ≜ 1−α 2GZ A + ηκΛZ GZ , µ t n o 2α where A ≜ max t0 S(t0 ), 1−α . Furthermore, e 1∥ + TE(t) ≤ αt ∥w0 − w

lim sup TE(t) = ATE ≤ ηκΛZ GZ .

i=1

e t+1 − w e t∥ ≤ ∥w

(42)

• Exponentially discounted weights in (7):

where recall that α = (1 − ηµ)E . Unrolling (35) and using the same contraction argument yields t−1 X

e t+1 − w e t∥ ≤ ∥w

(47)

(48)

t→∞

The proof is given in Appendix. Since α ∈ (0, 1), the first term in (47) decays geometrically, while the second term decays as O(1/t). Thus, under uniform weighting, the FPTE contribution vanishes asymptotically, and the TE is ultimately governed solely by the fixed-point bias term, which is O(η). In particular, to guarantee ATE = lim supt→∞ TE(t) ≤ ϵ, it suffices to choose ϵ η≤ . (49) κΛZ GZ

7

Exponentially discounted weights: For the exponentially discounted temporal weighting strategy (7), utilizing (43) gives

the bound in (52) for full-memory exponentially discounted weighting. For a target ϵ > 0, suppose η ≤ ϵ/(2κΛZ GZ ). Then, from t−1 2GZ X t−i 1 − γ t (55), it is sufficient to choose e 1∥ + TE(t) ≤ α ∥w0 − w + ηκΛZ GZ . α  µ i=1 1 − γ i+1   ϵ ln 4GZ 1−γ (50) ϵ+ µ · 1−γ m , (56) E≥   ln(1 − ηµ) The summation term in (50) admits a non-vanishing asymp  totic bound, yielding the following theorem. to guarantee ATE ≤ ϵ. t−i Pt−1 Theorem 2. Define Sγ (t) ≜ i=1 (1−γ)α with α = (1 − Since limγ→1 (1 − γ)/(1 − γ m ) = 1/m, the windowedi+1 1−γ E ηµ) ∈ (0, 1). Under discounted weights (7), the tracking uniform guarantees follow directly as a special case. 1−α error satisfies for all t ≥ t0 ≜ ⌈ln( 1+α−2γα )/ ln(γ)⌉ Corollary 1. Under windowed-uniform weights (8), for all t ≥ m, 1−γ 2GZ e 1∥ + + ηκΛZ GZ , (51) TE(t) ≤ αt ∥w0 − w Aγ t µ 1−γ 2GZ α(1 − αt−1 ) e 1∥ + TE(t) ≤ αt ∥w0 − w + ηκΛZ GZ .  (1−γ t0 )Sγ (t0 ) 2α µm 1−α where Aγ ≜ max , . Furthermore, 1−γ 1−α (57) 2GZ (1 − γ)α ATE ≤ + ηκΛZ GZ . (52) Consequently, µ 1−α 2GZ α ATE ≤ + ηκΛZ GZ . (58) The proof is given in Appendix. Unlike uniform weighting, µm 1 − α exponentially discounted weighting induces a non-vanishing FPTE contribution in addition to the bias floor. This is because Moreover, for a target ϵ > 0, if 0 < η ≤ ϵ/(2κΛZ GZ ), it is old samples are exponentially forgotten, and, hence, the fixed sufficient to choose   ϵ  point continues to drift by an amount proportional to 1 − γ ln 4GZ ϵ+ µm (cf. (43)). As γ → 1, this additional floor vanishes and the  E≥ (59)   discounted rule approaches uniform weighting.  ln(1 − ηµ)  For a target ϵ > 0, suppose we choose 0 < η ≤ ϵ/(2κΛZ GZ ) so that the bias term ηκΛZ GZ is at most ϵ/2. to guarantee ATE ≤ ϵ. Then, from (52), it is sufficient to choose For fixed η and E, the finite-memory FPTE contribution    ϵ in (58) has a 1/m prefactor, whereas the overall ATE also ln 4G ϵ+ µZ (1−γ) includes the fixed-point bias term. Hence, increasing the win  E≥ (53) , dow length reduces the additional finite-memory contribution.  ln(1 − ηµ)  In the limit m → ∞, this contribution vanishes, and the to guarantee ATE ≤ ϵ. bound reduces to the ATE bound obtained under full-memory Windowed temporal weights: We next consider the finite- uniform weighting in (48). memory windowed variants. Unlike the full-memory weightThe iteration complexity needed to achieve ATE ≤ ϵ is ings, windowed strategies retain only a fixed number of past summarized next. samples, thereby reducing memory and computational requirements. This finite-memory constraint, however, generally Remark 1. Under uniform weighting, the FPTE contribution induces a non-vanishing FPTE. We first state the result for vanishes for any E ≥ 1, and the ATE in (48) is controlled windowed-discounted weights; the windowed-uniform case is solely by the bias term. Hence, the condition in (49) shows then recovered as the limiting case γ → 1. Using drift that the ATE can be made arbitrarily small by reducing the Pt−1thet−i bound in (45), together with the geometric sum i=1 α = step size η. In contrast, for the discounted and windowed t−1 strategies, the ATE contains a non-vanishing FPTE floor in α(1 − α )/(1 − α), we obtain the following bound. addition to the bias term. The sufficient conditions in (53), Theorem 3. Under windowed-discounted weights (9), for all (56), and (59) reveal a tradeoff between the step size η and t ≥ m, the tracking error satisfies the number of decentralized iterations per time-step E. For t−1 fixed η, the sufficient threshold on E for achieving an FPTE 1 − γ 2G α(1 − α ) Z e 1∥ + TE(t) ≤ αt ∥w0 − w floor of order ϵ scales as Θ(ln(1/ϵ)). However, achieving an µ 1 − γm 1−α arbitrarily small total ATE also requires reducing the bias + ηκΛZ GZ . (54) term, which motivates choosing η = Θ(ϵ). In this case, − ln(1 − ηµ) = Θ(ϵ), and the sufficient per-step iteration Consequently, budgets in (53)–(56) scale as Θ(ln(1/ϵ)/ϵ). 2GZ 1 − γ α ATE ≤ + ηκΛ G . (55) Z Z µ 1 − γm 1 − α VI. N UMERICAL R ESULTS For windowed-discounted weights, the FPTE contribution in (55) is controlled by the effective-memory factor (1 − γ)/(1 − γ m ). As m → ∞, this factor approaches 1 − γ, matching

We now present numerical experiments illustrating the tracking behavior predicted by the analysis. While the derived bounds are not expected to be tight in general, the experiments

8

101

RMS TE

RMS TE

E=1 E=5 E = 10 E = 30

101

100 0

200

400

600

Time index (t)

800

100

1000

Fig. 2. Diffusion with uniform temporal weights: RMS TE vs. time index for number of iterations per time-step E.

6×101

E=1 E=5 E = 10 E = 30

RMS TE

4×101 3×101 2×101

0

200

400

600

Time index (t)

800

1000

Fig. 3. Diffusion with exponentially discounted temporal weights, γ = 0.7: RMS TE vs. time index for number of iterations per time-step E.

are intended to show that they are meaningful and capture the qualitative behavior of the considered decentralized system. At time t, agent n ∈ {1, . . . , N } incurs the quadratic loss ℓn,t (w) =

1 (w − cn,t )⊤ An,t (w − cn,t ), 2

(60)

where cn,t ∈ Rd models the streaming data process and An,t ∈ Rd×d is positive definite. We adopt quadratic losses because they provide a controlled and interpretable setting and admit a closed-form expression for the time-varying minimizer. This choice is appropriate because our objective is not to benchmark a particular learning architecture or dataset, but to study the qualitative implications of the theoretical analysis. We generate An,t = diag(λ1n,t , . . . , λdn,t ), where λjn,t ∼ Unif[µ, L] independently across n, t, and j. Hence, ℓn,t in (60) is µ-strongly convex and L-smooth. For this quadratic model, the temporally weighted global objective F t (w) = P t (w) admits the closed-form minimizer w∗t = i=1 ai (t)FiP
−1 Pt
PN P t N i=1 ai (t) n=1 An,i i=1 ai (t) n=1 An,i cn,i , which allows exact evaluation of the TE. The temporal evolution is generated through a bounded Gaussian random walk. For each coordinate j = 1, . . . , d, [cn,t+1 ]j = max{−Cmax , min([cn,t ]j + [zn,t+1 ]j , Cmax )}, where zn,t ∼ N (0, σ 2 I) are independent and identically distributed across agents and time. We initialize cn,0 uniformly over [−Cmax , Cmax ]d and set wn,0 = 0 for all n. In all

= 0.10 = 0.80 = 0.95 = 0.99 Uniform weights 101

102

Time index (t)

103

Fig. 4. Diffusion with discounted weights: RMS TE vs. time index for different discount factors γ, with E = 5. The uniform-weight curve is included as a benchmark.

experiments, we use N = 50 agents, dimension d = 100, step size η = 0.1, Cmax = 10, σ 2 = 1, µ = 0.01, and L = 0.1. All the experiments are run for T = 1000 time-steps. Agents are placed uniformly at random in a disk of radius one, and an undirected random geometric graph is formed by connecting agents within distance rth = 0.9, increased if needed to ensure connectivity. Given the connected graph, we construct the Metropolis-Hastings mixing matrix  1   , (i, j) ∈ E, i ̸= j,   max{deg(i), deg(j)} 1 +X [M]ij , i = j, [M]ij = 1 −   j̸=i    0, otherwise, (61) where deg(i) is the degree of agent i. This construction yields a symmetric doubly stochastic mixing matrix satisfying the assumptions in Section All curves report the root-meanq II. PR 1 2 squared (RMS) TE r=1 TEr (t) , averaged over R = R 100 Monte Carlo runs, where TEr (t) is computed according to (20). The communication graph is generated once and kept fixed across all Monte Carlo runs; randomness across runs comes from the streaming data process and loss parameters. Figures 2 and 3 compare different numbers of decentralized diffusion iterations per time-step E for uniform and exponentially discounted temporal weights, respectively. Under uniform weighting, the influence of each new sample decreases with time, leading to a decaying fixed-point drift and a vanishing FPTE. Increasing E accelerates the transient decay through the smaller contraction factor α = (1 − ηµ)E . The TE therefore decreases steadily before approaching the biasdominated regime predicted by Theorem 1. By contrast, under discounted weighting with γ = 0.7, the TE settles to a substantially larger steady-state level, consistent with Theorem 2. This behavior arises because old samples are exponentially forgotten, so the fixed-point drift does not vanish, and the TE contains a non-zero FPTE contribution in addition to the data-heterogeneity-induced bias. Larger E lowers this floor by improving tracking of the moving fixed point. Figure 4 studies the effect of the discount factor γ for diffusion with fixed E = 5. Smaller γ places more emphasis

RMS TE

9

m=1 m=2 m=5 m = (uniform)

101

0

200

400

600

Time index (t)

800

1000

Fig. 5. DGD with windowed-uniform temporal weights: RMS TE vs. time index for different window lengths m, with E = 5. The case m=∞ corresponds to standard uniform weighting.

6×101

m=1 m=2 m=5 m = (discounted)

RMS TE

5×101

4×101

3×101

0

200

400

600

Time index (t)

800

1000

VII. C ONCLUSION We investigated decentralized optimization with streaming data through a temporal-weighting formulation. For both DGD and diffusion, we developed a unified analysis that yields explicit tracking guarantees by decomposing the tracking error into a fixed-point tracking component and a networkinduced bias term. We also introduced a structured class of kernel-based temporal weights, encompassing uniform and exponentially discounted rules together with memory-efficient windowed variants. Specializing the bounds, we showed that uniform weighting yields a vanishing fixed-point tracking contribution of order O(1/t), leaving only the decentralizationinduced bias floor, whereas discounted and windowed schemes generally exhibit non-vanishing error floors governed by the discount factor and the window length, respectively. The resulting bounds explicitly show how temporal weighting, network topology, and the per-time-step iteration budget jointly determine the attainable asymptotic tracking performance. Numerical experiments corroborate the qualitative trends predicted by the theory. A PPENDIX Lemma 5 (Mean Hessian Theorem [48]). Let g : RN d → R be differentiable (but not necessarily twice differentiable), µstrongly convex, and L-smooth. Then, for any x, y ∈ RN d , there exists a symmetric matrix Ax,y ∈ RN d×N d (dependent on x and y) such that µI ⪯ Ax,y ⪯ LI, ∇g(y) − ∇g(x) = Ax,y (y − x).

(62) (63)

The proof of Lemma 5 can be found in [48]. Fig. 6. DGD with windowed-discounted temporal weights, γ = 0.7: RMS TE vs. time index for different window lengths m, with E = 5. The case m = ∞ corresponds to standard discounted weighting.

on recent samples, making the effective objective more adaptive but also more variable; this leads to a larger asymptotic TE, consistent with Theorem 2. As γ increases, however, the effective memory grows, and the TE floor decreases. In the limit γ → 1, discounted weighting approaches uniform weighting, explaining why the curve for γ = 0.99 closely follows the uniform-weight curve. Moreover, the continued decay of the uniform-weight curve is consistent with Theorem 1: after the geometrically decaying initialization transient, the FPTE contribution decreases as O(1/t), until the overall TE becomes limited by the constant-step-size bias floor. Figures 5 and 6 illustrate finite-memory windowing with DGD. For windowed-uniform weights, increasing the window length m lowers the TE floor, in agreement with the 1/m dependence in Corollary 1. In contrast to the fixed-window curves, the full-memory uniform curve, corresponding to m = ∞, continues to decrease. This behavior is consistent with Theorem 1, which gives an O(1/t) FPTE contribution under full-memory uniform weighting. For windowed-discounted weights, increasing m improves performance only up to the effective memory induced by the discount factor. Notably, due to exponential forgetting, even moderate window lengths, e.g., m > 5, nearly match the full-memory discounted curve, consistent with the factor (1 − γ)/(1 − γ m ) in (55).

Lemma 6. Let 0 < α < 1 and let {bt }t≥0 ⊂ R be a sequence that satisfies bt → b∗ as t → ∞. Define the sequence {xt }t≥0 by x0 ∈ R.

xt+1 = αxt + bt ,

(64)

Then, {xt } is also a convergent sequence which satisfies b∗ . t→∞ 1−α Proof. Since bt → b∗ , we express bt as bt = b∗ + ∗et with b et ≜ bt − b∗ → 0. Subtracting the candidate limit 1−α from b∗ both sides of (64), letting δt = xt − 1−α gives δt+1 = αδt +et . Using induction on t, we further obtain lim xt =

δt = α t δ0 +

t−1 X

αt−1−k ek .

(65)

k=0

Next, we will show that for every ε > 0 there exists a τ such that∗ |δt | < ε, ∀t ≥ τ , which implies δt → 0, hence b xt → 1−α . Let ε > 0 be given. Since et → 0, there exists N such that |et | ≤ (1 − α)ε/2 ∀t ≥ N , and, since every convergent sequence is bounded, there exists E > 0 such that |et | ≤ E, ∀t. Under such N , for t ≥ N , we rewrite δt as δt = α t δ0 +

N −1 X

αt−1−k ek +

k=0

|

t−1 X

αt−1−k ek .

k=N

{z

Ht

}

|

{z Tt

}

10

Since |ek | ≤ (1 − α)ε/2 , ∀k ≥ N , we bound |Tt | as |Tt | ≤ (1 − α)

ε 2

t−1 X

αt−1−k ≤ (1 − α)

k=N

which again yields (37).

∞ X

ε ε αj = . 2 j=0 2

Since |et | ≤ E, ∀t, it follows that |Ht | ≤

N −1 X k=0

αt−1−k |ek | ≤ E

N −1 X k=0

αt−1−k ≤

E αt−N . 1−α

Therefore, for all t ≥ N , it holds that E ε |δt | ≤ αt |δ0 | + αt−N + . 1−α 2

Proof of Lemma 4. For notational convenience, define ∆i (t) ≜ ai (t + 1) − ai (t), for i = 1, . . . , t. Then, we have t X

t X

∆i (t) =

i=1

ai (t + 1) −

i=1

t X

ai (t)

i=1

= 1 − at+1 (t + 1) − 1 = −at+1 (t + 1).

Next, we can express ∆i (t) = (∆i (t))+ −(∆i (t))− . Summing over i = 1, . . . , t yields t t X X (∆i (t))+ − (∆i (t))− = −at+1 (t + 1),

(66)

i=1

i=1

Since αt → 0, there exists some τ ≥ N such that αt |δ0 | < ε/4 E and 1−α αt−N < ε/4, for all t ≥ τ , hence |δt | < ε, ∀t ≥ τ . Hence δt → 0, which proves the lemma.

where the equality follows from (71). Hence,

Proof of Lemma 3. Fix any t ≥ 1. From the definition of ϕt (·) in (22) and the fixed-point relation (28), we have

i=1

e t + ηZ∇f t (w e t ) = 0, (I − M)w

(67)

where Z = IN d for DGD and Z = M = M⊗Id for diffusion. e t+1 − w e t. For notational convenience, define dt ≜ w • DGD (Z = I): Writing (67) at times t and t + 1 and subtracting, we obtain:
e t+1 ) − ∇f t (w e t ) = 0. (I − M)dt + η ∇f t+1 (w (68) e t and w e t+1 , there Applying Lemma 5 to f t (·) at the points w exists a symmetric matrix A with µI ⪯ A ⪯ LI such that

(71)

t X

(∆i (t))− =

t X

(∆i (t))+ + at+1 (t + 1) = It + at+1 (t + 1),

i=1

Pt since It = i=1 (∆i (t))+ . Leveraging |∆i (t)| = (∆i (t))+ + (∆i (t))− , we have t X i=1

|∆i (t)| =

t X

(∆i (t))+ + (∆i (t))−

i=1
= It + It + at+1 (t + 1) = at+1 (t + 1) + 2It ,

which proves (39).

Since the eigenvalues of M = M ⊗ Id are those of M, each repeated d times, and λmax (M) = 1, it follows that I−M ⪰ 0. Moreover, since A ⪰ µI, we have λmin ((I−M)+ηA) ≥ ηµ, and therefore it holds that η e t+1 ) − ∇f t (w e t+1 ) ∥dt ∥ ≤ ∇f t+1 (w ηµ

Proof of Lemma 2. We prove the three claims in (30)–(32) as follows: • Bound on the global minimizer w∗t : Since F t (·) is µstrongly convex (Assumption 1) with minimizer w∗t , we have, for any w ∈ Rd , µ 2 w − w∗t . F t (w) ≥ F t (w∗t ) + 2 Specializing the above inequality at w = 0, we obtain:
2 2 w∗t ≤ F t (0) − F t (w∗t ) . (72) µ Pt Using the definition of F t (w) = i=1 ai (t)Fi (w) and P N Fi (w) = N1 n=1 ℓn,i (w), we have

which yields (37) for any η > 0. • Diffusion (Z = M): Specializing (67) yields

F t (0) − F t (w∗t ) =

e t ) − ∇f t (w e t+1 ) = A(w et − w e t+1 ). ∇f t (w

(69)

Substituting (69) into (68) gives  
e t+1 ) − ∇f t (w e t+1 ) . (I − M) + ηA dt = −η ∇f t+1 (w

Next, applying Lemma 5 and utilizing (69), (70) becomes 
 e t+1 ) − ∇f t (w e t+1 ) . dt = M (I − ηA)dt − η ∇f t+1 (w Taking norms and using ∥M∥2 = ∥M⊗Id ∥2 = ∥M∥2 ∥Id ∥2 = 1 (since M is symmetric and doubly stochastic) gives e t+1 ) − ∇f t (w e t+1 ) . ∥dt ∥ ≤ ∥I − ηA∥2 ∥dt ∥ + η ∇f t+1 (w Next, it can be shown that under 0 < η ≤ 2/(L + µ), we have ∥I − ηA∥2 = max{|1 − ηµ|, |1 − ηL|} ≤ 1 − ηµ, so e t+1 ) − ∇f t (w e t+1 ) , ηµ ∥dt ∥ ≤ η ∇f t+1 (w

N

ai (t)

1 X ℓn,i (0) − ℓn,i (w∗t ) N n=1

ai (t)

N
1 X ∗ ℓn,i (0) − ℓn,i (wn,i ) , N n=1

i=1

e t + ηZ∇f t (w e t ) = 0. (I − M)w Subtracting the relations at t and t + 1 and rearranging yields
e t+1 ) − ∇f t (w e t) . dt = Mdt − ηM ∇f t+1 (w (70)

t X

≤

t X i=1

∗ where the inequality follows since wn,i = arg minw∈Rd ℓn,i (w). Furthermore, utilizing L-smoothness ∗ (Assumption 1) and optimality of wn,i , it follows that

L L ∗ ∥0 − wn,i ∥2 ≤ C 2 , 2 2 where P the second inequality invokes Assumption 2. Next, t using i=1 ai (t) = 1 gives ∗ ℓn,i (0) − ℓn,i (wn,i )≤

F t (0) − F t (w∗t ) ≤

L 2 C . 2

Finally, substituting (73) into (72) yields ∥w∗t ∥ ≤ C √ C κ, proving (30).

(73) p

L/µ =

11

e t: • Bound on the fixed point w – (DGD: Z = I). From the definition of ϕt (·) in (22) and the fixed-point relation (28), for DGD it holds that e t + η∇f t (w e t ) = 0. (I − M)w

e t ) = −(1/η)(I − M)w et Next, using (74) to substitute ∇f t (w further gives 1 ⊤ µ e t) + w e t + ∥w e t (I − M)w e t ∥2 . f t (0) ≥ f t (w η 2

e t) = f t (0) − f t (w

t X

ai (t)

i=1 t (a) X

≤

(b)

N X

i=1 t X

et w

N X

(LC)2 = N L2 C 2 ,

n=1

i=1

e t + ηM∇f t (w e t ) = 0. (I − M)w

(79) †

Define the orthogonal projectors PR(M) ≜ M M †

and

†

PN (M) ≜ I − M M , where M is the Moore–Penrose pseudoinverse of M. Pre-multiplying (79) by PN (M) yields e t = 0, PN (M) w †

†

e t ]n )) (ℓn,i (0) − ℓn,i ([w

M − M M M = 0 and PN (M) (I − M) = PN (M) − e t ∈ R(M). Hence, there PN (M) M = PN (M) . Therefore, w Nd exists yt ∈ R such that

∗ (ℓn,i (0) − ℓn,i (wn,i ))

n=1 N X

∗ where (a) follows since wn,i minimizes ℓn,i (·), (b) follows from L-smoothness (AssumptionP 1), and (c) invokes Assumpt tion 2 along with the fact thatq i=1 ai (t) = 1. Substituting √ e t ∥ ≤ C NµL = N CZ , which is (31) (77) into (76) gives ∥w √ with CZ = C κ and holds for all t ≥ 1. – (Diffusion: Z = M). From (22) and (28), the fixed point e t = ϕt (w e t ) with ϕt (w) = Mw − ηM∇f t (w) for satisfies w diffusion. Hence,

e t ∥ = ∥ϕt (w e t )∥ ∥w (b)

e t ) − ϕt (0)∥ + ∥ϕt (0)∥ ≤ ρ∥w e t ∥ + ∥ϕt (0)∥, ≤ ∥ϕt (w where (a) follows from adding and subtracting ϕt (0) and using the triangle inequality, and (b) invokes Lemma 1, whereby ϕt (·) is a contraction with factor ρ ∈ (0, 1). Therefore, e t∥ ≤ ∥w

≤

where the inequality follows from Jensen’s inequality and √ Pt (0)∥ ≤ N LC. Substituting a (t) = 1, yielding ∥∇f i i=1 √t η e t ∥ ≤ 1−ρ into (78) gives ∥w N LC. Finally, under the step size range 0 < η ≤ 2/(L + µ), we have ρ = 1 − ηµ √ (cf. e t ∥ ≤ C N κ, Lemma 1), and therefore, for all t ≥ 1, ∥w which is (31) for Z = M. – (Sharper diffusion bound when M ⪰ 0). If, in addition, M ⪰ 0, the diffusion fixed-point satisfies

=

L ∗ 2 (c) N L 2 ai (t) ≤ ∥wn,i ∥ ≤ C , 2 2 n=1 i=1 (77)

(a)

2

ai (t)∇ℓn,i (0)

where we leverage the facts that PN (M) M = (I−M M )M =

n=1 N X

ai (t)

and

N t X X n=1

(75)

Since I − M ⪰ 0, the middle term is nonnegative. Hence,
2 e t ∥2 ≤ e t) . ∥w f (0) − f t (w (76) µ t f t (·)

∥∇f t (0)∥2 =

(74)

Utilizing the µ-strong convexity of f t (·) (Assumption 1), (26) can be specialized as µ 2 e t ) − ∇f t (w e t )⊤ w et + et f t (0) ≥ f t (w w 2

Next, using the definition of  ⊤ e t ]⊤ e t ]⊤ [w · · · [w , we have 1 N

where the last inequality utilizes Assumption 2. Therefore,

1 ∥ϕt (0)∥. 1−ρ

(78)

Next, ϕt (0) = −ηM∇f t (0), and since ∥M∥2 = 1 (for symmetric doubly stochastic M), we have ∥ϕt (0)∥ ≤ η∥∇f t (0)∥. Moreover, from (13), the n-th block of ∇f t (0) equals Pt a (t)∇ℓ n,i (0). By L-smoothness (Assumption 1) and i=1 i ∗ the optimality condition ∇ℓn,i (wn,i ) = 0, ∗ ∗ ∥∇ℓn,i (0)∥=∥∇ℓn,i (0)−∇ℓn,i (wn,i )∥ ≤ L∥0−wn,i ∥ ≤ LC,

e t = Myt . w

(80)

Next, pre-multiply (79) by y⊤ t to obtain e t + η y⊤ e t ) = 0. y⊤ t (I − M)w t M∇f t (w ⊤

⊤ e t , we further Since M is symmetric, y⊤ =w t M = (Myt ) obtain 1 1 e t )⊤ w e t = − y⊤ e t = − y⊤ ∇f t (w (I − M)w (I − M)Myt , η t η t (81)

where the last equality uses (80). Since M ⪰ 0, M is also positive semidefinite and its eigenvalues are those of M. Hence, since λ(M) ⊂ [0, 1], the matrix (I − M)M is positive semidefinite (its eigenvalues are λ(1 − λ) ≥ 0), and therefore, e t )⊤ w et = −∇f t (w

1 ⊤ y (I − M)Myt ≥ 0. η t

(82)

Next, utilizing the µ-strong convexity of f t (·) (Assumption 1), (26) can be specialized as µ µ e t ) − ∇f t (w e t )⊤ w e t + ∥w e t ∥2 ≥f t (w e t ) + ∥w e t ∥2 , f t (0)≥f t (w 2 2 where the last inequality uses (82). Rearranging yields
2 e t ∥2 ≤ e t) , ∥w f t (0) − f t (w µ which is identical to (76). Thus, following the same steps as in (76)–(77) same bound as that obtained for DGD: qyields the √ √ NL e t∥ ≤ C ∥w N CZ , which is (31) with CZ = C κ. µ = e i )∥ and ∥∇ft (w e i )∥: We begin • Gradient Bounds ∥∇f t (w by recalling the µ-strong convexity of f t (·) under Assumption 1. Let m∗t = arg minw∈RN d f t (w) denote the minimizer

12

PN Pt of f t (·). Next, since f t (w) = n=1 i=1 ai (t)ℓn,i ([w]n ) is ∗ block-separable, Pt each d-dimensional block [mt ]n minimizes the function i=1 ai (t)ℓn,i (w) with w ∈ Rd . As a result, by using the same argument used to √ bound w∗t in (72)–(73), it is ∗ easy to verify that [mt ]n ≤ C κ, and hence √ ∥m∗t ∥ ≤ N CZ . (83) Next, L-smoothness (Assumption 1) of f t (·) and the optimality condition ∇f t (m∗t ) = 0 yields, for any i, t ≥ 1, e i )∥ = ∥∇f t (w e i ) − ∇f t (m∗t )∥ ∥∇f t (w (a)
(b) √ e i − m∗t ∥ ≤ L ∥w e i ∥ + ∥m∗t ∥ ≤ 2L N CZ , ≤L∥w where (a) invokes the triangle inequality and (b) uses (83) √ e t∥ ≤ √ N CZ . This proves the first inequality in (32) and ∥w with GZ = 2L N CZ . PN Let m∗t = arg minw ft (w). Since ft (w) = n=1 ℓn,t ([w]n ) is block-wise separable, it follows that m∗t = √ √  ∗ ⊤ ⊤ ∗ (w1,t ) · · · (wN,t )⊤ , hence ∥m∗t ∥ ≤ N C ≤ N CZ . Using L-smoothness (Assumption 1) of ft (·) and ∇ft (m∗t ) = 0 yields √ e i )∥ ≤ L∥w e i − m∗t ∥ ≤ L(∥w e i ∥ + ∥m∗t ∥) ≤ 2L N CZ , ∥∇ft (w for any i, t ≥ 1, which proves the second inequality in (32). ∗ Finally, √ since w∗t = 1N√⊗ w∗t , (30) √ implies ∥wt ∥ = ∥1N ⊗ ∗ ∗ wt ∥ = N ∥wt ∥ ≤ C N κ ≤ N CZ . Using again the Lsmoothness of f t (·) and the optimality condition ∇f t (m∗t ) = 0, for all t ≥ 1, we have ∥∇f t (w∗t )∥=∥∇f t (w∗t ) − ∇f t (m∗t )∥

√
≤L∥w∗t − m∗t ∥≤L ∥w∗t ∥ + ∥m∗t ∥ ≤2L N CZ .

Utilizing the definition of GZ yields the desired bound. Proof of Theorem 1. We first show that for all t ≥ t0 it holds that A S(t) ≤ . (84) t Note that we can express S(t + 1) as S(t + 1) =

t X αt+1−i i=1

i+1

=

α + αS(t). t+1

(85)

By the definition of A, we have A ≥ t0 S(t0 ), hence it directly holds that S(t0 ) ≤ tA0 . Next, as induction hypothesis assume 2α that S(t) ≤ At for some t ≥ t0 = ⌈ 1−α ⌉. Then, from (85), α S(t + 1) can be upper bounded as S(t + 1) ≤ t+1 + α At . To complete the argument, it is sufficient to show that the A right-hand side above is at most t+1 , which is equivalent to showing that α
A 1−α− ≥ α. t Since t ≥ t0 ≥ 2α/(1 − α), it follows that 1 − α − α/t ≥ (1 − α)/2. Thus, the desired inequality holds whenever A ≥ 2α/(1 − α), which is true by definition of A. Therefore, it follows by induction that (84) holds for all t ≥ t0 . Leveraging this bound in (46) yields the desired TE bound (47). Finally, taking lim supt→∞ in (47) gives the stated ATE bound, completing the proof.

Proof of Theorem 2. We first show that, for all t ≥ t0 , 1−γ . (86) Sγ (t) ≤ Aγ 1 − γt Pt (1−γ)αt+1−i can be written Note that Sγ (t + 1) = i=1 1−γ i+1 recursively as Sγ (t + 1) =

(1 − γ)α + αSγ (t). 1 − γ t+1

By definition of Aγ , we have Aγ ≥

(87)

(1−γ t0 )Sγ (t0 ) , thus it 1−γ

Aγ (1−γ) 1−γ t0 . Next, we assume as induction A (1−γ) Sγ (t) ≤ γ1−γ t for some t ≥ t0 =

holds that Sγ (t0 ) ≤

hypothesis that 1−α )/ ln(γ)⌉. Then, from (87), Sγ (t + 1) can be ⌈ln( 1+α−2γα A (1−γ)

(1−γ)α γ upper bounded as Sγ (t + 1) ≤ 1−γ t+1 + α 1−γ t . To complete the induction, it suffices to show that the rightAγ (1−γ) hand side above is at most 1−γ t+1 . After rearranging and simplifying, this condition is equivalent to h 1−γ i Aγ 1 − α − αγ t ≥ α. (88) 1 − γt

Since t ≥ t0 and γ ∈ (0, 1), the definition of t0 implies 1−α . 1 + α − 2γα Therefore, we can lower bound the left-hand side of (88) as h 1−γ i Aγ 1 − α − αγ t 1 − γt h i 1−α 1−α 1−γ ≥ Aγ 1 − α − α = Aγ , 1−α 1 + α − 2γα 1 − 1+α−2γα 2 1−α

γ t ≤ γ t0 ≤ γ ln( 1+α−2γα )/ ln(γ) =

where the inequality is due to the bound on γ t . Finally, by the definition of Aγ , we have Aγ ( 1−α 2 ) ≥ α, and thus (88) Aγ (1−γ) holds. We have thus proved that Sγ (t + 1) ≤ 1−γ t+1 . Hence, by induction, (86) holds for all t ≥ t0 . Substituting (86) into (50) yields (51). Next, from (87), the sequence {Sγ (t)} satisfies Sγ (t + 1) = αSγ (t) + bt , (1−γ)α with bt ≜ 1−γ t+1 . Since bt → (1 − γ)α and α ∈ (0, 1), Lemma 6 immediately yields

(1 − γ)α . 1−α Applying lim supt→∞ to both sides of (50), and using the limit of Sγ (t) gives (52), which completes the proof. R EFERENCES lim Sγ (t) =

t→∞

[1] M. F. U. Abrar, N. Michelusi, and E. G. Larsson, “Decentralized timevarying optimization for streaming data via temporal weighting,” 2026, to appear at the 2026 IEEE Asilomar Conference on Signals, Systems, and Computers. [Online]. Available: https://arxiv.org/abs/2605.06971 [2] T. Yang, X. Yi, J. Wu, Y. Yuan, D. Wu, Z. Meng, Y. Hong, H. Wang, Z. Lin, and K. H. Johansson, “A survey of distributed optimization,” Annual Reviews in Control, vol. 47, pp. 278–305, 2019. [3] A. Simonetto, E. Dall’Anese, S. Paternain, G. Leus, and G. B. Giannakis, “Time-varying convex optimization: Time-structured algorithms and applications,” Proceedings of the IEEE, vol. 108, no. 11, pp. 2032–2048, 2020. [4] E. Dall’Anese, A. Simonetto, S. Becker, and L. Madden, “Optimization and learning with information streams: Time-varying algorithms and applications,” IEEE Signal Processing Magazine, vol. 37, pp. 71–83, 2019.

13

[5] S. C. Hoi, D. Sahoo, J. Lu, and P. Zhao, “Online learning: A comprehensive survey,” Neurocomput., vol. 459, no. C, p. 249–289, Oct. 2021. [6] L. Wang, X. Zhang, H. Su, and J. Zhu, “A comprehensive survey of continual learning: Theory, method and application,” IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 46, no. 8, pp. 5362– 5383, 2024. [7] L. Yuan, Z. Wang, L. Sun, P. S. Yu, and C. G. Brinton, “Decentralized federated learning: A survey and perspective,” IEEE Internet of Things Journal, vol. 11, no. 21, pp. 34 617–34 638, 2024. [8] M. Chen, D. Gündüz, K. Huang, W. Saad, M. Bennis, A. V. Feljan, and H. V. Poor, “Distributed learning in wireless networks: Recent progress and future challenges,” IEEE Journal on Selected Areas in Comms., vol. 39, no. 12, pp. 3579–3605, 2021. [9] N. Michelusi, “Non-coherent over-the-air decentralized gradient descent,” IEEE Trans. on Signal Processing, vol. 72, pp. 4618–4634, 2024. [10] M. F. Ul Abrar and N. Michelusi, “Biased federated learning under wireless heterogeneity,” IEEE Trans. on Wireless Comms., vol. 25, pp. 16 449–16 462, 2026. [11] A. H. Sayed, “Adaptation, learning, and optimization over networks,” Found. Trends Mach. Learn., vol. 7, no. 4–5, p. 311–801, Jul. 2014. [12] L. Bai, C. Sun, Z. Feng, and G. Hu, “Distributed continuous-time resource allocation with time-varying resources under quadratic cost functions,” in 2018 IEEE Conference on Decision and Control (CDC), 2018, pp. 823–828. [13] Q. Ling and A. Ribeiro, “Decentralized dynamic optimization through the alternating direction method of multipliers,” IEEE Trans. on Signal Processing, vol. 62, no. 5, pp. 1185–1197, 2014. [14] C. Xi and U. A. Khan, “Distributed dynamic optimization over directed graphs,” in 2016 IEEE 55th Conference on Decision and Control (CDC), 2016, pp. 245–250. [15] A. Simonetto, A. Koppel, A. Mokhtari, G. Leus, and A. Ribeiro, “Decentralized prediction-correction methods for networked time-varying convex optimization,” IEEE Trans. on Automatic Control, vol. 62, no. 11, pp. 5724–5738, 2017. [16] C. Sun, M. Ye, and G. Hu, “Distributed time-varying quadratic optimization for multiple agents under undirected graphs,” IEEE Trans. on Automatic Control, vol. 62, no. 7, pp. 3687–3694, 2017. [17] S. Rahili and W. Ren, “Distributed continuous-time convex optimization with time-varying cost functions,” IEEE Trans. on Automatic Control, vol. 62, no. 4, pp. 1590–1605, 2017. [18] B. Wang, S. Sun, and W. Ren, “Distributed continuous-time algorithms for optimal resource allocation with time-varying quadratic cost functions,” IEEE Trans. on Control of Network Systems, vol. 7, no. 4, pp. 1974–1984, 2020. [19] B. Huang, Y. Zou, Z. Meng, and W. Ren, “Distributed time-varying convex optimization for a class of nonlinear multiagent systems,” IEEE Trans. on Automatic Control, vol. 65, no. 2, pp. 801–808, 2020. [20] S. Sun, J. Xu, and W. Ren, “Distributed continuous-time algorithms for time-varying constrained convex optimization,” IEEE Trans. on Automatic Control, vol. 68, no. 7, pp. 3931–3946, 2023. [21] Y. Li, H. Wang, W. Xu, T. Xiao, H. Liu, M. Tu, Y. Wang, X. Yang, R. Zhang, S. Yu, S. Guo, and R. Li, “Unleashing the power of continual learning on non-centralized devices: A survey,” IEEE Comms. Surveys & Tutorials, vol. 28, pp. 1059–1098, 2026. [22] B. Polyak, Introduction to optimization. Optimization Software, 1987. [23] Y. Zhao and M. Swamy, “A novel technique for tracking time-varying minimum and its applications,” in Conference Proceedings. IEEE Canadian Conference on Electrical and Computer Engineering, vol. 2, 1998, pp. 910–913 vol.2. [24] A. Y. Popkov, “Gradient methods for nonstationary unconstrained optimization problems,” Autom. Remote Control, vol. 66, no. 6, p. 883–891, Jun. 2005. [25] A. Simonetto, A. Mokhtari, A. Koppel, G. Leus, and A. Ribeiro, “A class of prediction-correction methods for time-varying convex optimization,” Trans. Sig. Proc., vol. 64, no. 17, p. 4576–4591, Sep. 2016. [26] M. Fazlyab, S. Paternain, V. M. Preciado, and A. Ribeiro, “Predictioncorrection interior-point method for time-varying convex optimization,” IEEE Trans. on Automatic Control, vol. 63, no. 7, pp. 1973–1986, 2018. [27] A. Simonetto and E. Dall’Anese, “Prediction-correction algorithms for time-varying constrained optimization,” IEEE Trans. on Signal Processing, vol. 65, no. 20, pp. 5481–5494, 2017. [28] A. S. Charles, A. Balavoine, and C. J. Rozell, “Dynamic filtering of time-varying sparse signals via ℓ1 minimization,” IEEE Trans. on Signal Processing, vol. 64, no. 21, pp. 5644–5656, 2016. [29] A. Simonetto and P. Massioni, “Nonlinear optimization filters for stochastic time-varying convex optimization,” International Journal of Robust and Nonlinear Control, vol. 34, no. 12, pp. 8065–8089, 2024.

[30] A. Nedic and A. Ozdaglar, “Distributed subgradient methods for multiagent optimization,” IEEE Trans. on Automatic Control, vol. 54, no. 1, pp. 48–61, 2009. [31] W. Shi, Q. Ling, K. Yuan, G. Wu, and W. Yin, “On the linear convergence of the admm in decentralized consensus optimization,” Trans. Sig. Proc., vol. 62, no. 7, p. 1750–1761, Apr. 2014. [32] K. Yuan, W. Xu, and Q. Ling, “Can primal methods outperform primaldual methods in decentralized dynamic optimization?” IEEE Trans. on Signal Processing, vol. 68, pp. 4466–4480, 2020. [33] M. Zinkevich, “Online convex programming and generalized infinitesimal gradient ascent,” in Proceedings of the Twentieth International Conference on International Conference on Machine Learning, ser. ICML’03. AAAI Press, 2003, p. 928–935. [34] S. Shalev-Shwartz, “Online learning and online convex optimization,” Found. Trends Mach. Learn., vol. 4, pp. 107–194, 2012. [35] E. Hazan, “Introduction to online convex optimization,” Found. Trends Optim., vol. 2, pp. 157–325, 2016. [36] A. Jadbabaie, A. Rakhlin, S. Shahrampour, and K. Sridharan, “Online Optimization: Competing with Dynamic Comparators,” in Proceedings of the 18th International Conference on Artificial Intelligence and Statistics (AISTATS), ser. Proceedings of Machine Learning Research, vol. 38, 2015, pp. 398–406. [37] O. Besbes, Y. Gur, and A. Zeevi, “Non-stationary stochastic optimization,” Oper. Res., vol. 63, no. 5, p. 1227–1244, Oct. 2015. [38] S. Shahrampour and A. Jadbabaie, “Distributed online optimization in dynamic environments using mirror descent,” IEEE Trans. on Automatic Control, vol. 63, no. 3, pp. 714–725, 2018. [39] Z. Li and D. Hoiem, “Learning without forgetting,” IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 40, pp. 2935–2947, 2016. [40] G. I. Parisi, R. Kemker, J. L. Part, C. Kanan, and S. Wermter, “Continual lifelong learning with neural networks: A review,” Neural Netw., vol. 113, no. C, p. 54–71, May 2019. [41] M. De Lange, R. Aljundi, M. Masana, S. Parisot, X. Jia, A. Leonardis, G. Slabaugh, and T. Tuytelaars, “A continual learning survey: Defying forgetting in classification tasks,” IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 44, no. 7, pp. 3366–3385, 2022. [42] M. McCloskey and N. J. Cohen, “Catastrophic interference in connectionist networks: The sequential learning problem,” Psychology of Learning and Motivation, vol. 24, pp. 109–165, 1989. [43] B. Wu, Z. Ding, and J. Huang, “A review of continual learning in edge ai,” IEEE Trans. on Network Science and Engineering, vol. 13, pp. 6571– 6588, 2026. [44] A. Mitra, H. Hassani, and G. J. Pappas, “Online federated learning,” in 2021 60th IEEE Conference on Decision and Control (CDC), 2021, pp. 4083–4090. [45] O. Marfoq, G. Neglia, L. Kameni, and R. Vidal, “Federated learning for data streams,” in Proceedings of The 26th International Conference on Artificial Intelligence and Statistics, ser. Proceedings of Machine Learning Research, F. Ruiz, J. Dy, and J.-W. van de Meent, Eds., vol. 206. PMLR, 25–27 Apr 2023, pp. 8889–8924. [46] C.-H. Hu, Z. Chen, and E. G. Larsson, “Energy-efficient federated edge learning with streaming data: A Lyapunov optimization approach,” IEEE Trans. on Comms., vol. 73, no. 2, pp. 1142–1156, 2025. [47] M. F. Ul Abrar, N. Michelusi, and E. G. Larsson, “Time-varying optimization for streaming data via temporal weighting,” in 2025 59th Asilomar Conference on Signals, Systems, and Computers, 2025, pp. 1343–1349. [48] E. G. Larsson and N. Michelusi, “Unified analysis of decentralized gradient descent: a contraction mapping framework,” IEEE Open Journal of Signal Processing, pp. 1–25, 2025. [49] Y. Nesterov, Lectures on Convex Optimization, 2nd ed. Springer Publishing Company, Incorporated, 2018. [50] S. Boyd and L. Vandenberghe, Convex Optimization. Cambridge university press, 2004. [51] W. Rudin, Principles of Mathematical Analysis, 3rd ed. New York, NY: McGraw-Hill, 1976, vol. 3.

Related documents

• dSTAR: Straggler Tolerant and Byzantine Resilient Distributed SGD#1035272
• Noise-Robust Distributed Optimization Over Directed Graphs With Row Stochastic Matrices#1077410
• Distributed Linear Programming on GPU Clusters at Extreme Scale#667973
• Replication Data for: EDSAF: Event-Driven Surrogate-Assisted Framework for Real-time Optimization#685271
• FLOSS: Federated Learning with Opt-Out and Straggler Support#158756