arXiv:2604.20129v1 [cs.LG] 22 Apr 2026
A Delta-Aware Orchestration Framework for Scalable Multi-Agent Edge Computing 1st Samaresh Kumar Singh
2nd Joyjit Roy
Independent Researcher IEEE Senior Member Leander, Texas [email protected]
Independent Researcher IEEE Senior Member Austin, Texas [email protected]
Abstract—The ”Synergistic Collapse” occurs when scaling beyond 100 agents causes superlinear performance degradation that individual optimizations cannot prevent. We observe this collapse with 150 cameras in Smart City deployment using MADDPG, where Deadline Satisfaction drops from 78% to 34%, producing approximately $180,000 in annual cost overruns. Prior work has addressed each contributing factor in isolation: exponential action-space growth, computational redundancy among spatially adjacent agents, and task-agnostic hardware scheduling. None has examined how these three factors interact and amplify each other. We present DAOEF (Delta-Aware Orchestration for Edge Federations), a framework that addresses all three simultaneously through: (1) Differential Neural Caching, which stores intermediate layer activations and computes only the input deltas, achieving 2.1× higher hit ratios (72% vs. 35%) than output-level caching while staying within 2% accuracy loss through empirically calibrated similarity thresholds; (2) Criticality-Based Action Space Pruning, which organizes agents into priority tiers and reduces coordination complexity from O(n2 ) to O(n log n) with less than 6% optimality loss; and (3) Learned Hardware Affinity Matching, which assigns tasks to their optimal accelerator (GPU, CPU, NPU, or FPGA) to prevent compounding mismatch penalties. Controlled factor-isolation experiments confirm that each mechanism is necessary but insufficient on its own: removing any single mechanism increases latency by more than 40%, validating that the gains are interdependent rather than additive. Across four datasets (100-250 agents) and a 20-device physical testbed, DAOEF achieves a 1.45× multiplicative gain over applying the three mechanisms independently. A 200-agent cloud deployment yields 62% latency reduction (280 ms vs. 735 ms), sub-linear latency growth up to 250 agents where baselines saturate at 80, and 62% energy savings (44.7 MWh/year). Index Terms—Multi-Agent Edge AI, Synergistic CoOptimization, Deep Reinforcement Learning Orchestration, Neural Activation Caching, Scalability Theory, Heterogeneous Computing.
I. I NTRODUCTION Multi-agent edge AI systems in smart cities, coordinating 10,000+ surveillance cameras and autonomous vehicle fleets managing 500+ vehicles, face significant scalability challenges. A large-scale deployment using MADDPG orchestration achieved 78% deadline satisfaction with 20 cameras, but dropped to 34% with 150 cameras, resulting in $180K annual overruns. Three key limitations prevent scaling beyond 50-100 agents: (1) combinatorial growth in possible actions (K N for N agents and K nodes), adding 85-120 ms decision
delay, (2) computational redundancy where adjacent cameras compute 65% similar features yet achieve only 35-42% cache hit rates, and (3) task-agnostic scheduling that assigns 30% of vision tasks to CPUs instead of GPUs, causing 2-5× slowdowns. Prior methods address each challenge individually with limited success, yielding 28-42% latency reductions. These challenges interact: when orchestration consumes 80-120 ms per decision cycle, a 10-20 ms cache lookup becomes infeasible under a 100 ms deadline. Reducing orchestration below 10 ms creates the timing headroom that makes caching viable and in turn enables better hardware placement. When 70% of neural network inputs are structurally similar, delta caching reduces recomputation to only the dissimilar fraction, and priority-based filtering prevents wasteful CPU assignments for vision tasks, accelerating convergence by 3-5×. Controlled ablations confirm a 72% cost reduction against 49.7% from the individual components applied separately, a 1.45× measured synergistic gain. We present DAOEF with three co-designed mechanisms: (1) Three-tiered priority filtering that reduces actions from O(K N ) to O((K/10)N ) using model availability, hardware affinity, and proximity tiers, with less than 6% optimality loss (Theorem 2) and sub-10 ms decisions for 200 agents. (2) Feature-level delta caching that uses semantic similarity to store intermediate CNN/Transformer features at empirically selected optimal layers, computing reconstruction deltas for 72% cache hits (versus 35-42% at result level) and under 2% accuracy loss, with bounded reconstruction error O((1 − s)2 ) (Theorem 4). (3) Hardware-aware matching that integrates efficiency factors into priority filtering, preventing 2–5× mismatch penalties and increasing accelerator utilization by 2.5×. We validated DAOEF at three levels. Simulation across four datasets (CityPersons, nuScenes, Edge-IIoTset, and VisDrone2019) with 100-250 agents shows sub-linear latency growth where baselines saturate at 80 agents. A 20-device heterogeneous physical testbed confirmed simulation accuracy within 10% across all metrics. A 200-agent cloud deployment across five AWS regions yielded 62% latency reduction (280 ms vs. 735 ms) and 50% higher throughput over MADRLBasic. All comparisons achieve statistical significance (Cohen’s d > 1.85, p < 0.001, Bonferroni-corrected across 36
TABLE I C OMPARISON OF DAOEF WITH S TATE - OF - THE -A RT M ULTI -AGENT E DGE C OMPUTING A PPROACH Method
Max Agents
Decision Latency
HW-Aware Scheduling
Caching Mechanism
Action Space Reduction
Validation Level
Year
Zhao et al. [1] Ju et al. [2] Gao et al. [3] Yang et al. [4] Zhang et al. [5] Cai et al. [6] DAOEF
20 30 80 50 60 40 200+
> 500ms > 400ms > 300ms > 250ms > 280ms > 350ms <10ms
No No No No No Partial Yes
No No No LRU (result) No No Delta-level
None None Curriculum Priority None Task-specific 10×
Sim only Sim only Sim only Sim only Sim only Sim only Sim+Test+Cloud
2022 2023 2023 2023 2024 2023 2025
comparisons). Our contributions are: (1) Empirical and theoretical characterization of synergy amplification between orchestration, caching, and hardware scheduling in multi-agent edge systems, including controlled factor-isolation experiments showing that the three effects are interdependent. (2) A feature-level delta caching mechanism for distributed DNN inference that, to our knowledge, is the first to exploit intermediate-layer similarity across spatially correlated multi-agent workloads, with error bounds validated across CNN, Transformer, and hybrid architectures. (3) Convergence proof for TD3 under hierarchical action-space filtering and empirical confirmation at simulation, testbed, and cloud scale. II. R ELATED W ORK AND BACKGROUND We position DAOEF against other edge computing methods that have used multi-agent reinforcement learning in Table I. Related work has been categorized by these authors as follows: (Multi Agent Reinforcement Learning (MARL) scalability, edge caching, and hardware-aware resource allocation. Multi-Agent RL for Edge Computing. The MARL techniques used for orchestrating edge computing systems train centrally and execute in a decentralized manner [1], [2] or use value decomposition [7]. There is recent literature supporting 20-80 agents: Zhao et al. [1], who employed Multi-Agent TD3 for UAV-assisted MEC with 20 agents, Gao et al. [3], who developed a curriculum learning technique with 80 agents, and Zhang et al. [5], who utilized COMA for satellite Mobile Edge Computing (MEC) with 60 agents. Transformation-based architectures [8] reduced the coordination required to O(N 2 ). However, they did nothing about the O(K N ) action space problem. All of the methods have demonstrated 250-500 ms decision time for 50-80 agents and failed for 100+ agents. Gap: To date there are no MARL techniques that can achieve sub-10 ms decisions for 200+ agents. Cache mechanisms for edge computing. Cache optimization in MEC is based on caching of results (LRU [4], LFU), which provides a cache hit rate of 35–42%. The authors of [9] proposed an NAS-based method to optimize the cache, achieving a cache hit rate of 40%. However, these methods treat each query separately and do not use semantic similarity between queries. For instance, given two similar images captured by adjacent cameras (with 65% overlap), we would generate two distinct cache keys, leading us to recompute the
output even though their intermediate representations were almost identical. Gap: There is no caching mechanism that uses the semantic similarity of features to achieve cache hit rates greater than 60%. Hardware-Aware Resource Allocation: Cai et al. [6] have developed a multi-task DRL model to provide a hardwareaware approach to resource allocation for Industrial IoT that is partially aware of available hardware resources. They demonstrate improvements in accelerator utilization of 2.8–3.5x. Existing frameworks [1], [2], [5] do not account for the computational affinity between different task types and available hardware accelerators, leading to performance losses of 22–60% due to poor task assignment to hardware. There are domain-specific accelerators that can improve performance by an order of magnitude (5–10x) compared to general-purpose GPUs for certain workloads. However, no orchestration system currently exists to exploit such heterogeneity. Gap of Integration: The different elements of the problem have already been dealt with as separate items. However, there has never been a solution that combines scalability, intelligent cache management, and hardware awareness. Most recent federated learning solutions [10] have reduced communication overhead. However, they have not solved the issue of the action space’s complexity. Most recent benchmarking initiatives [8], [11] have provided standardized evaluation. However, they have not solved the basic scalability issues. Positioning of DAOEF: To our knowledge, DAOEF is the first framework to synergistically unite: (1) hierarchical filtering with priority at each level to reduce the size of the action space by ten times with less than six percent of optimality lost, (2) delta caching at feature levels using the semantic similarities between them to achieve an average of seventy-two percent of hits, and (3) hardware aware matching to prevent two to five times greater penalty for mismatches. Unlike prior works that were validated solely through simulations, DAOEF provides validation at multiple levels (simulations across four data sets with 100-250 agents, 50 devices testbed, a 200agent cloud deployment) along with comprehensive statistical analyses (fifty runs, Cohen’s d > 1.85, p < .001). III. S YSTEM M ODEL AND P ROBLEM F ORMULATION A. System Architecture DAOEF presents a three-tier architecture (Figure 1): (1) Worker Agents (N AI agents) generate task requests
Fig. 1. DAOEF architecture with three-tier priority filtering and delta-aware caching.
Ti = ⟨xi , mi , wi , di ⟩, where xi is input data, mi is the target ML model (e.g., YOLOv5, ResNet-50), wi is compute workload Floating Point Operations (FLOPs), and di is deadline. (2) Master Orchestrator (AM ) learns optimal task offload policies via deep reinforcement learning, maintains a cache C, and performs hardware-aware matching. (3) Edge Nodes (E = {E1 , . . . , EK }) provide heterogeneous compute resources with capabilities cj (FLOPS), hardware types hj ∈ {GPU, NPU, CPU, FPGA}, hosted models Mj , and current load λj ∈ [0, 1].
B. Task Execution Model Total latency for executing task Ti on edge node Ej comprises transmission, queuing, and computation: P |xi | wi k∈Qj wk Lij = (1) + + Bij cj cj · αmi ,hj |{z} | {z } | {z } queue
transmit
compute
where Bij is bandwidth, Qj is the task queue on node j, and αmi ,hj ∈ [0.1, 1.0] is the hardware efficiency factor capturing computational affinity (e.g., αYOLOv5, GPU = 0.95 vs. αYOLOv5, CPU = 0.22). Computational cost is Cij = βj wi /(cj αmi ,hj ) for cost-per-FLOP βj . C. Delta-Aware Caching
Example: Consider camera agent A1 submitting task T1 = ⟨street_scene.jpg, YOLOv5, 25 GFLOPS, 100 ms⟩. The master orchestrator: (1) checks cache for similar images with Locality-Sensitive Hashing (LSH) lookup, (2) if similarity s > 0.6, calculate feature delta ∆ rather than computing full inference, (3) applies three-tier filtering (model availability → hardware affinity → edge proximity) to select target node, (4) offloads to CPU/GPU/NPU-equipped edge node E2 with YOLOv5 pre-loaded and minimal latency.
Cache C stores tuples ⟨xk , mk , yk , fk ⟩ of input, model ID, output, and intermediate features. For new task Ti , compute similarity sik = xTi xk /(∥xi ∥∥xk ∥) via Locality-Sensitive Hashing in O(log |C|) time. If sik∗ > θ (threshold, typically 0.6): yi = yk∗ + fdec (∆i ),
∆i = fi − fk∗ ,
w∆ ≈ (1 − sik∗ )wi (2)
This reduces workload proportionally to the similarity: 80% similarity yields 80% savings in computation. D. Problem Formulation We formulated Multi-Agent Edge Orchestration as a Markov Decision Process (M.D.P.) with the objective to minimize the expected total cost over time, which is a function of a set of weights: "N # X min Eπ (ω1 Li + ω2 Ci − ω3 Ri ) (3) π
i=1
subject P to: (1) deadline constraints Li ≤ di ; (2) capacity constraints i:π(si )=j wi ≤ cj ; (3) cache consistency Ri ∈ {0, 1} indicates hits. The MDP consists of: K t State: st = [{xi , mi , wi , di }N i=1 , {cj , hj , Mj , λj }j=1 , Ct ] (task queue, edge resources, cache state). Action: ai ∈ {1, . . . , K} selects target edge node for task i. Without filtering, the action space is K N ; with three-tier N . priority filtering, it is reduced P to ∼ 3 P P Reward: r = −ω L − ω t 1 2 i i i Ci + ω 3 i Ri + P ω4 i ⊮(Li ≤ di ) balances cost, latency, cache efficiency, and deadline satisfaction. Notation Summary: Key symbols: N, K (agents, nodes); Ti (task with input xi , model mi , workload wi , deadline di ); cj , hj , αm,h (node capacity, hardware type, efficiency); C, sik , ∆i (cache, similarity, delta); πϕ , Qθ (policy, critic networks). IV. DAOEF F RAMEWORK D ESIGN
(1)
Tier 1 - Model Availability: Filter Ei = {Ej : mi ∈ Mj } removes nodes that lack the required ML models, avoiding 2-5 s loading delays that dominate computation time. For the YOLOv5 task, if preloaded models are on 15 of 25 nodes, candidates are reduced from 25 to 15. Tier 2 - Affinity of Hardware: Retain top p (specifically p = 3) by ranking the remaining nodes by efficiency factor αmi ,hj . For visual tasks: αYOLOv5,CPU = 0.22, αYOLOv5,GPU = 0.95, αYOLOv5,NPU = 0.68. Selecting the top three ensures GPU-equipped nodes are prioritized, preventing 4× slowdowns from CPU assignment. Tier 3 - Proximity: Select node j ∗ = arg minj∈E (2) (ℓij + i λj ) minimizing network latency plus current load. At this final step, no approximation error is introduced while communication overhead is minimized. Design Rationale: Three tiers are preferred over joint optimization because (1) Tier 1 is deterministic (binary model availability), (2) Tier 2 exploits known hardware affinities, avoiding wasteful exploration of CPU nodes for vision tasks, and (3) Tier 3 handles dynamic load that DRL learns to predict. This separation enables 10× action-space reduction with <6% loss of optimality (Theorem 2, Section IV-D) and reduces DRL training time by 3-5× by eliminating structurally poor actions. Comparison with Alternatives: Table II compares our hierarchical approach with alternatives: TABLE II C OMPARISON OF ACTION S PACE R EDUCTION S TRATEGIES
Approach
A. Overview: Design Principles DAOEF’s architecture follows three design principles that enable the synergistic integration identified in Section I: Principle 1: Hierarchical Decomposition: Rather than joint optimization of all three mechanisms, we decompose the orchestration problem into three sequential tiers that can be provably composed. This enables efficient decision-making while preserving optimality guarantees (Section IV-D). Principle 2: Feature-Level Caching and Action Space Hierarchy: In real-world edge deployments, there are many sources of structural sparsity in addition to the fact that: (a) not all nodes need to run all models, (b) hardware affinity creates a preference for certain locations, and (c) network topologies induce locality. DAOEF exploits these structures rather than treating the action space as uniformly dense. Principle 3: Feature-Space Computation Reuse: Rather than requiring exact matches, feature-level caching allows for partial computation reuse for similar input. This turns the binary hit/miss into a continuum, fundamentally changing the dynamics of cache efficiency. B. Tiered Filtering for Action Space Reduction DAOEF uses a three-tier approach to filter action spaces. It reduces the number of candidate actions from K N to (K/10)N . Tiered filtering is used to limit the action space prior to evaluating the DRL policy:
Joint neural selection [8] Weighted scoring [6] Hierarchical (DAOEF)
Reduction Factor
Optimality Loss
Training Time
5×
3%
18K episodes
7× 10×
8% <6%
15K episodes 8K episodes
C. Feature-Level Delta-Aware Caching The traditional approach to result-level caching treats each query independently, resulting in a 35-42% hit ratio. DAOEF uses semantic similarity to cache intermediate CNN/Transformer features and compute incremental deltas. Distinguishing DAOEF from other Similarity-Based Methods: • Video compression (H.264/H.265): Applies pixel-level motion vectors to find temporal similarities between frames. DAOEF finds semantic similarities between spatially adjacent agents and caches the neural network’s feature map, not raw pixels. • Database Similarity Search (LSH/FAISS): Finds approximate nearest neighbors for retrieval but does not compute deltas for partial reuse. DAOEF extends LSH to incremental computation, recalculating only dissimilar layers. • Result-Level Edge Caching [4], [9]: Caches the final output, requiring exact input match. DAOEF caches the
intermediate representation, yielding a continuous hit ratio from 0-100%. Feature-Level Caching for Multi-Agent Workloads: Consider adjacent cameras where Camera A observes an intersection from the north and Camera B from the east. Input images differ in lighting and viewpoint, yet early CNN layers extract similar low-level features such as textures, edges, and object boundaries. For inputs with 70% similarity, layers 18 (of 16) generate >90% similar features; only layers 916 require recomputation. This property is specific to multiagent deployments with spatial locality and does not hold for randomly distributed independent workloads. Locality-Sensitive Hashing: Cache C stores tuples ⟨xk , mk , yk , fk ⟩ (input, model, output, features at layer ℓ). For new task Ti , compute hash h(xi ) = sign(Wxi ) where W ∈ R64×d is a random projection. LSH lookup finds nearest neighbor k ∗ = arg maxk sik in O(log |C|) time (vs. O(|C|) for exhaustive search). We use 64-bit SimHash [12] with 4 hash tables to achieve 99% recall at a 10K cache size. Delta Reconstruction and Computation: If similarity sik∗ = xTi xk /(∥xi ∥∥xk ∥) > θ: (ℓ→L)
yi = yk∗ + fdec
(∆i ),
(ℓ) ∆i = fenc (xi ) − fk∗
(4)
(ℓ)
where fenc extracts features at layer ℓ (layer 8/16 for ResNet(ℓ→L) 50, layer 6/12 for BERT) and fdec decodes the delta to output. Workload scales with dissimilarity: w∆ ≈ (1 − sik∗ )wfull . With 80% similar images, this yields 80% computation savings. Optimal Layer Selection: We empirically determine the optimal caching layer ℓ∗ for each architecture: h i (ℓ) (ℓ→L) ℓ∗ = arg min wdecode + (1 − s̄) wencode (5) ℓ
(ℓ) where wdecode
(ℓ→L)
is decoder overhead and wencode is encoding cost for remaining layers. For ResNet-50: ℓ∗ = 8 (after 2nd residual block); for BERT-Base: ℓ∗ = 6 (mid-transformer); for YOLOv5: ℓ∗ = 10 (after CSPDarknet backbone). This balances feature discriminability (early layers are too generic) versus computation savings (late layers provide minimal reuse). Cache Management: When capacity |C| = Cmax is reached, the entry with the highest eviction score ωt (tnow − tk ) − ωh · hit count(k) is removed, balancing recency and utility. Typical Cmax = 10,000 enables sub-millisecond lookups with 64-bit LSH. Cache replacement uses a hybrid LFU+LRU policy with ωt = 0.3, ωh = 0.7, optimized via grid search. Similarity Threshold Selection: The threshold θ governs the accuracy-efficiency tradeoff: lower values admit more cache reuse but increase reconstruction error, higher values improve accuracy at the cost of hit rate. We selected θ = 0.6 via grid search over θ ∈ {0.4, 0.5, 0.6, 0.7, 0.8} on a held-out 10% split of each dataset, optimizing the composite objective J = HitRate − λ · ∆mAP with λ = 5 to penalize accuracy loss five times more than missed cache opportunities. Table III
summarizes results on CityPersons. The value θ = 0.6 was Pareto-optimal across all four datasets; at θ = 0.5 the hit rate improves marginally but mAP drops 4.1%, while at θ = 0.7 accuracy recovers but the hit rate falls below the break-even point where delta computation saves less than 10% of full inference cost. TABLE III T HRESHOLD S ENSITIVITY (C ITY P ERSONS , 150 AGENTS )
θ
Hit Rate (%)
mAP (%)
∆mAP
J
0.4 0.5 0.6 0.7 0.8
83 78 72 61 44
69.2 70.0 72.4 73.6 74.0
−4.9 −4.1 −1.7 −0.5 −0.1
58.5 57.5 63.5 58.5 43.5
Full compute mAP: 74.1%; J = HitRate − 5 · |∆mAP|
Cross-Architecture Validation: Delta caching requires that intermediate representations are transferable across similar inputs. For ResNet-50 (CNN), feature maps at ℓ∗ = 8 exhibit >90% cosine similarity for inputs with sik > 0.6 because early residual blocks act as local texture detectors whose outputs are relatively viewpoint-invariant. For BERTBase (Transformer), hidden states at ℓ∗ = 6 share structural similarity when input token sequences overlap >50%, consistent with attention-head behavior on paraphrase pairs. For YOLOv5 (hybrid), caching at the CSPDarknet backbone exit (ℓ∗ = 10) preserves object-class features while letting the detection head run fresh, necessary because bounding-box regression is sensitive to minor position shifts. Empirically, mAP degradation from delta reconstruction remains below 2% across all three families at their optimal layers (ResNet50: 1.7%, BERT: 1.4%, YOLOv5: 1.9%), confirming that the mechanism generalizes rather than being architecture-specific. Accuracy-Efficiency Tradeoff: Feature-level caching achieves <2% accuracy degradation (mAP: 72.4% vs. 74.1% full computation) while providing 72% cache hits, confirming that delta reconstruction preserves semantic content while enabling substantial computation reuse. D. DRL Agent Architecture with Attention DAOEF employs Twin Delayed Deep Deterministic Policy Gradient (TD3) [13] with transformer-based attention for variable-length task sequences. Task Encoding: Embed task Ti as hi = MLPtask [Embed(xi ), OneHot(mi ), wi , di ]
(6)
where Embed(·) uses pretrained ResNet-18 for images or word2vec for text. Edge Encoding: Embed node Ej as ej = MLPedge [cj , OneHot(hj ), ModelVec(Mj ), λj ]
(7)
Cross-Attention Policy: Compute task-edge affinity through scaled dot-product attention, then softmax over filtered (3) candidates Ei : ! T hTi WQ WK ej √ (8) πϕ (ai | s) = softmax dk (3) j∈Ei
TABLE IV C OMPUTATIONAL C OMPLEXITY C OMPARISON Component
MADDPG
MAPPO
DAOEF
Action space KN KN (K/10)N Decision time O(K N dh ) O(K N dh ) O(3N dh ) Cache lookup N/A N/A O(log |C|) Memory O(N K) O(N 2 K) O(N (K/10) + |C|) Training/step 180 ms 220 ms 50 ms Convergence 15K episodes 18K episodes 8K episodes N = 100, K = 25, dh = 256; timing on NVIDIA A100
Twin Critics: Two Q networks Qθ1 , Qθ2 estimate value, with target y = r + γ mink Qθk′ (s′ , πϕ′ (s′ ) + ϵ), where the minimum operator eliminates overestimation bias. Critic loss: LQ = E[(Qθ (s, a) − y)2 ]. Training: Update critics every step via gradient descent on TD error, update policy every d = 2 steps via deterministic policy gradient, soft-update targets with τ = 0.005. Exploration uses Gaussian noise N (0, σ) with decay σt = σ0 /t0.6 .
Proof sketch: Apply Lipschitz composition; bound constants via spectral norms [18], [19]; propagate delta through layers ℓ∗ → L.
E. Theoretical Guarantees
F. Complexity Analysis
We provide four guarantees grounded in existing DRL and approximation theory: Theorem 1 (Convergence): For bounded rewards |rt | ≤ Rmax , Lipschitz continuous Q functions, decreasingPexploration Pnoise σt → 0, and Robbins-Monro conditions t ηt = ∞, t ηt2 < ∞ [14], the filtered policy πϕ converges almost √ ∗ surely to a local optimum πfiltered at rate O(1/ t). Proof sketch: Building on TD3 convergence [13]: (1) the filtered space remains fully connected because Tier 3 proximity selection keeps all viable nodes reachable; (2) twin critic updates maintain contraction properties; (3) the policy gradient is unbiased over the filtered action set. Theorem 2 (Approximation Ratio): Priority filtering with top-p selection achieves (1 + ϵp )-approximation where: s 2 ln(K/p) + O |S|−1/2 (9) ϵp = σα p
Priority filtering changes decision complexity from O(K N ) to O(3N ), a (25/3)100 ≈ 1040 reduction for N = 100. LSH lookup costs O(log |C|) vs. O(|C|) exhaustive. Total memory: 20 MB (policy/critic) + 100 MB (replay buffer) + 20 MB (cache) = 140 MB, feasible for edge deployment. In the zero-similarity worst case all cache entries miss and DAOEF reduces to MADRL-Priority with <10 ms orchestration overhead; the system never underperforms standard MARL with priority filtering.
For K = 25, p = 3, σα = 0.15: ϵp < 0.06 (6% optimality loss). Proof sketch: Model hardware affinities as random variables; apply Chernoff bound [15] to show concentration around optimal; combine with neural approximation error [16]. Theorem 3 (Cache Benefit): For mean similarity s̄ > θ: (s̄ − θ)(s̄ + 1) ρcache ≥ 2(1 − θ)
(10)
At s̄ = 0.6, θ = 0.5: ρ ≥ 32%, consistent with the observed 72% (gap explained by second-order temporal locality effects not captured by the bound). Proof sketch: Model sik as Beta-distributed [17]; derive truncated expectation E[w∆ |s > θ]; apply Jensen’s inequality. Beta fit validated empirically (R2 > 0.92). Theorem 4 (Delta Reconstruction Error): For an Llayer network with Lipschitz activations and bounded weights ∥Wℓ ∥ ≤ BW : ∥yi − yitrue ∥ ≤
L Y
BW ∥∆ℓ ∥ ≤ C (1 − sik∗ )2
ℓ=ℓ∗
Error is O((1 − s)2 ), negligible for s > 0.8.
(11)
V. E XPERIMENTAL R ESULTS AND VALIDATION DAOEF is validated via a three-tier evaluation process: simulated testing across four real-world datasets (CityPersons, nuScenes, Edge-IIoT set, VisDrone 2019) using agents ranging from 100 to 250. Testing in a real-world testbed with twenty heterogeneous physical devices of different memory footprints, and testing in the cloud with 200 agents distributed over five Amazon Web Services (AWS) regions. This three-tiered approach provides both controlled reproducibility (simulated testing) and real-world applicability (testing in a testbed and in the cloud). A. Experimental Setup Datasets: (1) CityPersons [5]: 5,000+ street scenes with 60-70% inter-camera similarity for smart city surveillance, evaluated with 150+ agents, (2) nuScenes: 1,000 driving scenes with 40-55% similarity for autonomous vehicle fleets, 100-150 agents, and (3) Edge-IIoTset [11]: Industrial IoT sensor data with 55-65% similarity and 100+ agents. (4) VisDrone2019 [20]: 10K+ Unmanned Aerial Vehicle (UAV) videos with 45-60% similarity for aerial monitoring and 200 agents. Computed similarities using cosine distance on ResNet-18 embeddings. Implementation: The simulation was implemented using PyTorch version 2.2 as the core framework along with a discrete-event simulator provided by the Ray RLlib version 2.8. All simulations were run on a heterogeneous edge computing environment that includes an NVIDIA A100 GPU, which supports FP16 operations at a rate of 312 TFLOPS, an Huawei Ascend 910 NPU, which operates in INT8 mode at a rate
TABLE V P ERFORMANCE ACROSS F OUR R EAL -W ORLD DATASETS (DAOEF VS . B EST BASELINE ) Dataset CityPersons nuScenes Edge-IIoTset VisDrone2019
Agents
Similarity (%)
200 150 100 120
65 45 60 48
Latency (ms) DAOEF Best Base 268 ± 16 312 ± 18 178 ± 11 295 ± 17
Cache Hit (%)
Tput (t/s)
Deadline Sat. (%)
Cohen’s d
75 58 68 62
24.7 21.4 18.3 19.8
96 94 97 95
2.87 2.45 3.12 2.68
625 (MADRL-B) 698 (MADRL-B) 328 (MADRL-B) 682 (MADRL-B)
Average 143 54.5 263 583 (55%↓) 66 21.0 95.5 2.78 Best Base: MADRL-Basic (Yang et al. [4]), Cohen’s d: Effect size vs. best baseline. All p < 0.001 (Bonferroni-corrected)
of 512 TOPS, an Intel Xeon Platinum 8380 CPU, which is supported by AVX-512 instructions, and an Xilinx Alveo U250 FPGA. Each model has been assigned an empirically determined efficiency factor αm,h to account for performance losses due to resource sharing and task overlap in this sharededge environment. These values range from 0.1 (BERT on FPGA) to 0.95 (YOLOv5 on A100 GPU). Benchmarking Baselines: (1) Random: A uniform random assignment of tasks to agents, (2) Greedy: a load balanced version of the Round-Robin assignment strategy, (3) MADDPG [21]: a multi-agent actor-critic, (4) MAPPO [22]: a multiagent Proximal Policy Optimization algorithm, (5) QMIX [7]: a value decomposition strategy for multi-agent reinforcement learning, (6) MADRL-Basic [4]: a state-of-the-art multi-agent edge orchestration, (7) MADRL-Cache: MADRL-Basic + Level Result Caching using LRU based cache, (8) MADRLPriority: MADRL-Basic + Priority Filter Only (without Delta Caching). Model Configuration: DAOEF: Actor is a 4 Layer Transformer (512 Hidden Units, 8 Attention Heads), Critic is a 3layer MLP (256-128-64 Hidden Units). The cache used in this model is an LSH Cache (64-bit SimHash, max number of hash buckets = 10,000, Similarity Threshold = 0.6), and the priority filter is a Top-p=3 Filter. Training: The DAOEF model was trained on 8 V100 GPUs (each with 32 GB) over 8K Episodes, Batch Size = 256, Learning Rate = 3 × 10−4 , Optimizer = Adam with Cosine Annealing. The replay buffer used during training contained 100K Transitions. Each experiment was run 50 times, with different random seeds. Metrics: (1) Latency: end-to-end task completion time (network + computation + orchestration), (2) Decision latency: orchestrator decision time per batch, (3) Throughput: tasks completed per second, (4) Cache hit rate: proportion of queries with similarity > θ, (5) Deadline satisfaction: fraction of tasks meeting deadline, (6) Energy: total power consumption, (7) Accuracy: task output quality (mAP for detection, accuracy for classification). B. Main Results: Cross-Domain Performance Table V consolidates performance across four datasets. DAOEF achieves 55-70% reductions in latency, 58-75% in cache hits, 48-89% in throughput, and 94-96% in deadline satisfaction vs. baselines.
Key Insights: (1) Reduction in latency (55% average) stems from: sub-10 ms decision time in priority filtering, 66% average cache hits with delta caching, 2-4× mismatch avoidance (hardware matching). (2) Cache efficiency correlates with similarity: 75% hits at 65% similarity (CityPersons) vs. 58% at 45% (nuScenes), validates the similarity-based caching approach. (3) Throughput scales near-linearly: 24.7 tasks/s (200 agents) vs. 12.5 (MADRL-Basic) by demonstrating bottleneck elimination. (4) Statistical significance: Cohen’s d > 2.4 across all datasets with very large effect per Cohen’s guidelines [23]), p < 0.001 after Bonferroni correction for 36 comparisons (α = 0.05/36 = 0.0014). C. Ablation Study: Dissecting Synergy Table VI shows the individual contribution of each component. The key finding is that the synergy between components achieves a 72% cost reduction, which exceeds the reductions obtained by applying the individual components sequentially (51%, 42%, and 56%). This validates our core hypothesis that an integrated design approach produces multiplicative gains rather than simply additive improvements. TABLE VI A BLATION S TUDY: C OMPONENT C ONTRIBUTIONS (150 AGENTS , C ITY P ERSONS ) Configuration
Latency (ms)
Cost Reduct.
Cache Hit (%)
Full DAOEF 280 72% 72 w/o Delta Cache 485 (+73%) 42% 38 (LRU) w/o Priority Filter 518 (+85%) 51% 68 w/o HW Matching 395 (+41%) 56% 71 Only Priority 548 38% 63 Only Cache 622 29% 65 Random Baseline 892 0% – Synergy factor: 72% > (51% + 42% + 56%)/3 = 49.7% Decision Time: Orchestrator latency
Decision Time (ms) 7.2 6.8 94.5 7.5 38.2 112.8 0.2
Controlled Factor Isolation: To establish causality rather than just correlation, we introduce each factor independently under otherwise identical conditions. Table VII reports latency and deadline satisfaction at 150 agents as each factor is activated in isolation and then jointly. When only one factor is active, the system improves over the unoptimized baseline but still fails to scale. Jointly activating all three produces the reported 72% cost reduction, exceeding the sum of individual improvements by 1.45×. This controlled design rules out the
possibility that any single mechanism is responsible for most of the benefit. TABLE VII FACTOR I SOLATION : I NDEPENDENT VS . J OINT ACTIVATION (150 AGENTS , C ITY P ERSONS ) Active Factors None (baseline) Filtering only Caching only HW Matching only Filtering + Caching Filtering + HW Matching Caching + HW Matching All three (DAOEF)
Latency (ms)
Deadline Sat. (%)
Cost Reduct.
892 548 622 695 412 385 501 280
38 67 55 61 81 84 74 96
– 38% 29% 22% 53% 57% 44% 72%
Each config uses identical hardware, data, and DRL training budget.
Understanding Synergy: Three interaction effects explain why full DAOEF achieves a 72% reduction compared to 49.7% from individual components: Effect 1 - Priority Filtering Enables Caching: Without priority filtering, orchestration takes 94.5 ms and leaves no time for cache lookup (10-15 ms) under a 100 ms deadline. Filtering reduces orchestration to 7.2 ms, making cache lookup viable. Removing priority filtering still achieves 68% cache hits, but end-to-end latency climbs to 518 ms because cache lookups dominate when orchestration is slow. Effect 2 - Delta Caching Amplifies Hardware Gains: Hardware-aware placement provides a 2.8× speedup (GPU vs. CPU for vision). Delta caching contributes 30% partial computation for 70% similar inputs, yielding effective speedup 2.8 × 0.7 + 1.0 × 0.3 = 2.26. Without caching, hardware gains apply only to full executions. Effect 3 - Structural Sparsity Accelerates Convergence: Priority filtering eliminates wasteful exploration, reducing training from 15K to 8K episodes (1.9× speedup). Faster convergence enables better policies and improved cache hits through spatial locality, which is why priority filtering alone achieves 63% cache hits versus 72% for full DAOEF. Mathematical Validation: Ssynergy /Saverage = 0.72/0.497 = 1.45, matching theoretical prediction Ssynergy ≈ Sp (1 + βSc Sh ) with β ≈ 0.4 from empirical fitting.
Fig. 2. Scalability comparison: DAOEF maintains sub-100 ms latency to 250 agents. Baseline methods exceed 300 ms beyond 100 agents due to superlinear growth in decision complexity. Error bars are 95% Confidence Intervals across 50 runs.
MADRL-Basic decision time exceeds 120 ms at 120+ agents, causing >50% deadline violations. DAOEF maintains <15 ms to 250 agents, enabling real-time operation. Latency Breakdown: For 150 agents: Network transmission latency (45 ms, 16%), Computation (185 ms, 66%), Orchestration (7 ms, 2.5%), Cache lookup (5 ms, 1.8%), Queueing (38 ms, 13.7%). DAOEF’s sub-10 ms orchestration was essential, MADRL-Basic’s 95 ms orchestration would have added 88 ms (31%) to the overall latency. E. Real-World Testbed Validation Testbed Configuration: Twelve heterogeneous edge nodes: (1) 4× NVIDIA Jetson AGX Orin (each with GPU: 275 TFLOPS FP16, 32GB RAM), (2) 3× Intel NUC 12 Pro (each with CPU: Core i7-1260P, 32GB RAM), (3) 3× Google Coral Dev Board (each with TPU: 4 TOPS INT8), and (4) 2× Xilinx ZCU104 FPGA. Worker agents: Twenty Raspberry Pi 4B (each with 8 GB RAM), which simulated twenty IoT camera devices that were streaming CityPersons data at 10 FPS. Network: Gigabit Ethernet network with round-trip time between 2-15 ms. Physical Constraint Note: Tested with 20 agents due to physical hardware availability constraints. Each agent requires a dedicated Raspberry Pi 4B device. This represents the maximum scale achievable with our available physical resources while maintaining realistic device characteristics.
D. Scalability and Breakdown Analysis DAOEF achieves sub-100 ms latency to 250 agents as shown in Figure 2. Baseline methods exceed 300 ms latency beyond 100 agents because their decision complexity grows superlinearly with agent count, while DAOEF’s tiered filtering bounds orchestration time independently of N . Error bars are 95% Confidence Intervals across 50 runs. Scaling Characteristics: (1) Cache efficiency improves with scale: 58% hits (50 agents) → 72% (150) → 78% (250) due to increased query overlap and improved spatial locality from more agents covering overlapping regions. (2) Throughput near-linear: 7.8 tasks/s (50 agents) → 21.3 (150) → 34.2 (250), MADRL-Basic saturates at 19.5 tasks/s (150 agents) due to orchestration bottleneck. (3) Breakdown point:
TABLE VIII R EAL -W ORLD T ESTBED R ESULTS (20 AGENTS , C ITY P ERSONS ) Metric
DAOEF
MADRLB
Greedy
Sim Gap
End-to-end latency (ms) Decision latency (ms) Cache hit rate (%) Throughput (tasks/s) Deadline satisfaction (%) Energy (W)
156 ± 12 4.2 ± 0.8 68 ± 4 8.4 ± 0.6 94 ± 2 42.3 ± 3.1
285 ± 24 78.5 ± 6.2 – 5.2 ± 0.4 78 ± 4 68.7 ± 4.8
342 ± 28 1.1 ± 0.2 – 4.8 ± 0.5 72 ± 5 75.2 ± 5.1
8.2% 9.5% 5.3% 6.8% 4.2% 7.1%
Sim Gap:(Real@20 agents) ; (Simulation@20 agents) ; Simulation; all <10%
Validation: Real-world results closely matches the simulation with <10% gap across all metrics. This confirms the sim-
ulator’s accuracy. A 45% latency improvement over MADRLBasic (156 ms vs. 285 ms) validates practical benefits. Sub-5 ms decision latency demonstrates real-time capability. 38% energy reduction (42.3W vs. 68.7W) confirms sustainability gains. Real World Observations: (1) Network Variability: There is a 2-3 times larger variance between the values obtained when simulating the network as it exists in the real world versus the determinate nature of the links used in simulations. However, the system still performs well due to its use of robust Tier 3 proximity selection. (2) Hardware Variability: Power throttling by the Jetson Orin board under sustained load reduces performance by 5-10%. This issue has been accounted for by increasing the size of the error bars. (3) Cache WarmUp: During the first 50 queries, there were no cache hits (i.e., cold start). However, after 200 queries, the number of cache hits had stabilized at 68%. F. Cloud-Scale Multi-Region Deployment AWS Configuration: 50 EC2 edge nodes distributed across 5 regions (10 nodes each): us-east-1 (N. Virginia), us-west2 (Oregon), eu-west-1 (Ireland), ap-southeast-1 (Singapore), ap-northeast-1 (Tokyo). Instance types: c5.2xlarge (8 vCPU, 16GB RAM, <1 ms local latency, 50-200 ms inter-region RTT). Workload: 200 worker agents generating CityPersons tasks with region-specific patterns (e.g., Tokyo has higher density during APAC business hours). TABLE IX C LOUD -S CALE VALIDATION (AWS M ULTI -R EGION , VARIABLE S CALE ) Scale (Agents)
DAOEF Latency (ms)
MADRL-B Latency (ms)
Speedup
50 198 ± 15 425 ± 38 2.1× 100 256 ± 18 645 ± 52 2.5× 150 312 ± 22 985 ± 78 3.2× 200 378 ± 28 1520 ± 125 4.0× 250 445 ± 32 2180 ± 185 4.9× 28% reduction in cross-region traffic vs. Random; cost savings $0.18/hour (AWS pricing)
Decision Time (ms) 4.8 ± 0.6 6.2 ± 0.8 7.5 ± 0.9 8.5 ± 1.1 9.8 ± 1.3
Cloud Validation: DAOEF achieves a latency of less than 400 ms when using 200 agents as opposed to 1500 ms in MADRL-Basic (DAOEF is four times faster). DAOEF’s decision time remains below 10 milliseconds with 250 agents, thereby proving its scalability. The proximity-based Tier-3 filtering reduces cross-region assignment to agents by 28%. Random assignment results in 40% of the data being transferred across expensive regions, while DAOEF results in only 12% local transfer. Therefore, this will save $.018 per hour for 200 agents on AWS pricing ($0.02 per GB). Geographic Insights: Latency breakdown by region pair: us-east-1 ↔ us-west-2 (65 ms RTT, 15% of traffic), us-east-1 ↔ eu-west-1 (90 ms, 8%), us-east-1 ↔ ap-southeast-1 (185 ms, 4%). DAOEF’s proximity filtering preferentially assigns tasks within regions, reducing expensive intercontinental transfers by 3.2× compared to random assignment.
G. Energy and Sustainability Analysis Table X, describes the amount of energy consumed in a 150 agent configuration. DAOEF results in a 62 % reduction when compared to a Random baseline, which equates to a 44.7 MWh/year saving (17.9 Tons CO2 /Year) for a 500-camera smart city configuration. TABLE X E NERGY C ONSUMPTION A NALYSIS (150 AGENTS , 24-H OUR D EPLOYMENT ) Method
Avg Power (kW)
Energy (kWh/day)
Reduction vs Random
CO2 (kg/day)
Random 8.2 196.8 – 78.7 MADRL-Basic 5.9 141.6 28% 56.6 MADRL-Cache 4.8 115.2 41% 46.1 DAOEF 3.1 74.4 62% 29.8 CO2 using US grid average: 0.4 kg/kWh; 500-camera extrapolation: linear scaling to 44.7 MWh/year
Sustainability Impact: For the 500-camera smart city configuration, the random baseline uses 71,832 kWh/yr. Using DAOEF, the use is reduced to 27,156 kWh/yr (62 % = 44,676 kWh/yr). Economic impact: $5,361/yr (at $0.12/kWh U.S. commercial rate). Environmental impact: The equivalent reduction of 17.9 Tons CO2 /yr (the equivalent of 3.9 Passenger Vehicles [24]). Energy Breakdown: For 150 agents: Computation (58%), Network (22%), Orchestrator (8%), Idle (12%). DAOEF reduces computation via delta caching (70% of 58% = 40.6% total) and network via proximity filtering (50% of 22% = 11% total). H. Statistical Validation and Confidence Analysis Effect Sizes: Cohen’s d for DAOEF vs. baselines: Random (d = 4.21, ”very large”), MADRL-Basic (d = 2.87, ”very large”), MAPPO (d = 2.34, ”very large”), MADRL-Cache (d = 1.92, ”large”). All exceed Cohen’s threshold for large effects (d > 0.8) [23]. Multiple Comparisons Correction: Bonferroni-corrected threshold: α = 0.05/36 = 0.0014 for 4 datasets × 9 methods. All DAOEF comparisons achieve p < 0.0001, well below threshold, confirming statistical significance survives multiple testing correction. Normality and Variance Tests: Shapiro-Wilk tests confirm latency distributions that is approximately normal (p > 0.05 for all datasets except nuScenes p = 0.042, marginally significant). Levene’s test confirms homogeneous variance across all methods with p = 0.18. For non-normal nuScenes, we additionally apply the Mann-Whitney U test: p < 0.001, confirming significance. Confidence Intervals: 95% CIs for DAOEF latency (CityPersons, 200 agents): [252, 284] ms. Best baseline MADRL-Basic: [587, 663]ms. Non-overlapping CIs provide additional evidence beyond p-values. Power Analysis: Post-hoc power analysis (G*Power 3.1) for detected effect size d = 2.87: achieved power > 0.999
with n = 50 runs, α = 0.0014. Sample size was more than sufficient to detect effects. DRL Convergence: DAOEF converges in 5K-8K episodes (mean: 6.2K, std: 0.8K) versus MADRL-Basic 15K episodes (3-4× faster). Cross-seed variance <5% after convergence, demonstrating stable training. Learning curves available in supplementary materials. VI. D ISCUSSION A. Key Findings and Implications Our central research is validated by integrating components that synergistically produce multiplicative savings. DAOEF achieves 72% cost reduction, substantially exceeding the performance of individual components applied separately (51%, 42%, 56%). The factor-isolation experiments in Table VII confirm that this gap is not an artifact of measurement or baseline selection: each pair-wise combination also underperforms the full system, and each mechanism provides diminishing returns when operating without the others. Three key interaction effects explain this behavior: Orchestration Overhead as Major Latency Bottleneck: All baseline methods consume 60-80% of their total budget on decision-making, averaging 85-120 ms with more than 100 agents due to unconstrained action spaces. DAOEF’s orchestration stays below 10 ms, providing timing headroom for cache lookup and hardware matching. Baselines were given identical hardware and training budgets; their higher latency reflects structural inefficiency rather than resource disadvantage. Edge-Cache Paradigm Using Feature-Level Caching: The 72% hit rate compared to 35-42% with result-level caching stems from high semantic similarity between intermediate CNN layer representations. About 70% of input images share >90% feature similarity in early layers, enabling partialcomputation reuse. Theorem 4 bounds reconstruction error as (1 − s)2 . Implication: Edge-caching systems should exploit intermediate computations rather than treating each network layer as a black box. Proactive Hardware Matching: Tier 2 filtering removes CPU nodes from vision tasks before DRL evaluation, reducing mismatch penalties by 2-5× and cutting training from 15K to 8K episodes. Implication: Encoding domain knowledge as structural constraints outperforms end-to-end learning alone, with provably <6% optimality loss (Theorem 2). VII. C ONCLUSION In this paper, we introduced DAOEF, a multi-agent edge orchestration framework. It can make decisions in under 10 ms for 200+ agents. This was achieved by using three synergistic innovations. Hierarchical Priority Filtering that reduces the size of the action space by 10x. Feature-Level Delta Caching achieves 72% cache hit ratios compared to 35-42% at the result level, and Hardware-Aware Matching avoids 2-5 times the cost of a mismatch penalty. We validated our system across four datasets using a 50-device and a 200-agent cloud deployment. This resulted in a 62% decrease in latency, 50% increase
in throughput, nearly linear scaling to 250 agents, and 62% energy savings (44.7 MWh/year for 500-camera deployments). Key Contributions: DAOEF provides an example of how to combine scalable intelligent caching and hardware awareness with multi-level validation. A novel feature-level delta computation exploits semantic similarity to improve cache efficiency by 2x. The production-quality implementation allows for previously infeasible deployments: 500+ camera smart cities, 200+ vehicle fleets, and 300+ sensor industrial systems. Future Directions: Possible extensions are as follows: Distributed Orchestration for 1000+ agents, Privacy-Preserving Federated Caching, Cross-Domain Transfer Learning, and Workload-Specific Adaptive Optimizations. DAOEF shows that Intelligent Orchestration, combining Action Space Reduction, Semantic Caching, and Hardware Awareness, can overcome fundamental scalability barriers and allow for sustainable large-scale edge AI deployments. R EFERENCES [1] N. Zhao, Z. Ye, Y. Pei, Y.-C. Liang, and D. Niyato, “Multi-agent deep reinforcement learning for task offloading in uav-assisted mobile edge computing,” IEEE Transactions on Wireless Communications, vol. 21, no. 9, pp. 6949–6963, 2022. [2] Y. Ju, Y. Chen, Z. Cao, L. Liu, Q. Pei, M. Xiao, K. Ota, M. Dong, and V. C. Leung, “Joint secure offloading and resource allocation for vehicular edge computing network: A multi-agent deep reinforcement learning approach,” IEEE Transactions on Intelligent Transportation Systems, vol. 24, no. 5, pp. 5555–5569, 2023. [3] Z. Gao, L. Yang, and Y. Dai, “Large-scale cooperative task offloading and resource allocation in heterogeneous mec systems via multi-agent reinforcement learning,” IEEE Internet of Things Journal, vol. 10, no. 20, pp. 17 837–17 851, 2023. [4] J. Yang, Q. Yuan, S. Chen, H. He, X. Jiang, and X. Tan, “Cooperative task offloading for mobile edge computing based on multi-agent deep reinforcement learning,” IEEE Transactions on Network and Service Management, vol. 20, no. 4, pp. 4123–4138, 2023. [5] H. Zhang, H. Zhao, R. Liu, A. Kaushik, X. Gao, and S. Xu, “Collaborative task offloading optimization for satellite mobile edge computing using multi-agent deep reinforcement learning,” IEEE Transactions on Vehicular Technology, vol. 73, no. 8, pp. 11 234–11 249, 2024. [6] J. Cai, H. Fu, and Y. Liu, “Multitask multiobjective deep reinforcement learning-based computation offloading method for industrial internet of things,” IEEE Internet of Things Journal, vol. 10, no. 4, pp. 3516–3528, 2023. [7] T. Rashid, M. Samvelyan, C. Schroeder et al., “Qmix: Monotonic value function factorisation for decentralised multi-agent reinforcement learning,” in International Conference on Machine Learning. PMLR, 2018, pp. 4295–4304. [8] X. Chen, L. Jiao, W. Li, and X. Fu, “A survey on edge computing systems and tools,” Proceedings of the IEEE, vol. 112, no. 1, pp. 6–34, 2024. [9] L. Huang, X. Feng, L. Zhang, L. Qian, and Y. Wu, “Deep reinforcement learning for dynamic computation offloading and resource allocation in cache-assisted mobile edge computing systems,” IEEE Transactions on Network Science and Engineering, vol. 11, no. 1, pp. 968–983, 2024. [10] Y. Lu, X. Huang, K. Zhang, S. Maharjan, and Y. Zhang, “Communication-efficient federated learning for digital twin edge networks in industrial iot,” IEEE Transactions on Industrial Informatics, vol. 19, no. 2, pp. 1361–1371, 2023. [11] M. A. Ferrag, O. Friha, M. Hamdi, H. Gharbi, L. Shu, X. Xie, A. Seret, and A. Polleres, “Edge-iiotset: A new comprehensive realistic cyber security dataset of iot and iiot applications for centralized and federated learning,” IEEE Access, vol. 9, pp. 121 676–121 698, 2021. [12] M. S. Charikar, “Similarity estimation techniques from rounding algorithms,” in Proceedings of the Thirty-fourth Annual ACM Symposium on Theory of Computing. ACM, 2002, pp. 380–388.
[13] S. Fujimoto, H. v. Hoof, and D. Meger, “Addressing function approximation error in actor-critic methods,” in International Conference on Machine Learning (ICML). PMLR, 2018, pp. 1587–1596. [14] H. Robbins and S. Monro, “A stochastic approximation method,” The Annals of Mathematical Statistics, 1951. [15] H. Chernoff, “A measure of asymptotic efficiency for tests of a hypothesis based on the sum of observations,” The Annals of Mathematical Statistics, 1952. [16] K. Hornik, “Approximation capabilities of multilayer feedforward networks,” 1991. [17] N. L. Johnson, S. Kotz, and N. Balakrishnan, Continuous univariate distributions. John Wiley & Sons, 1995, vol. 2. [18] A. Virmaux and K. Scaman, “Lipschitz regularity of deep neural networks: analysis and efficient estimation,” in NeurIPS, 2018. [19] P. L. Bartlett, D. J. Foster, and M. J. Telgarsky, “Spectrally-normalized margin bounds for neural networks,” in NeurIPS, 2017. [20] P. Zhu, L. Wen, X. Bian, H. Ling, Q. Hu et al., “Detection and tracking meet drones challenge,” arXiv preprint arXiv:2001.06303, 2020. [21] R. Lowe, Y. Wu, A. Tamar, J. Harb, P. Abbeel, and I. Mordatch, “Multiagent actor-critic for mixed cooperative-competitive environments,” Advances in Neural Information Processing Systems, vol. 30, 2017. [22] J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal policy optimization algorithms,” arXiv preprint arXiv:1707.06347, 2017, widely adopted in multi-agent RL systems. [23] J. Cohen, Statistical Power Analysis for the Behavioral Sciences, 2nd ed. Hillsdale, NJ: Lawrence Erlbaum Associates, 1988. [24] U.S. Environmental Protection Agency, “Greenhouse gas equivalencies calculator,” https://www.epa.gov/energy/greenhouse-gas-equivalenciescalculator, 2021, accessed: 2024-01-10.