LLM-Enhanced Deep Reinforcement Learning for Task Offloading in Collaborative Edge Computing Hao Guo† , Kaixiang Xv† , Ziwu Ge, Lei Yang∗
arXiv:2605.05727v1 [cs.DC] 7 May 2026
School of Software Engineering, South China University of Technology, Guangzhou, China Email: {seguohao,202330551861,202330550381}@mail.scut.edu.cn; ∗ [email protected] † These authors contributed equally to this work; ∗ Corresponding author
Abstract—Collaborative edge computing uses edge nodes in different locations to execute tasks, necessitating dynamic task offloading decisions to maintain low latency and high reliability, especially under unpredictable node failures. Although deep reinforcement learning (DRL) and large language models (LLMs) have shown promise for task offloading, DRL often suffers from high sample inefficiency and local optima, whereas LLMs struggle with real-time decision-making. To address these limitations, we propose LeDRL, a hybrid decision framework that couples a lightweight LLM with self-attention-enhanced DRL for real-time task offloading. LeDRL constructs structured, context-aware prompts capturing node status, task semantics, and link dynamics to derive high-level strategy priors. These are selectively processed by a self-attention-based alignment module for context-aware policy optimization. A reflective evaluator distills semantic feedback from past trajectories to guide future prompts, enabling more informative and temporally generalizable LLM queries. Extensive experiments show that LeDRL outperforms baselines in task success rate, convergence speed, and real-time responsiveness across diverse network scales, achieving over 17% improvement in success rate. Furthermore, we deploy LeDRL on Jetson-based edge devices using our prototype system CoEdgeSys, demonstrating its robustness and feasibility under resource constraints. Our code is available at:https://github.com/GalleyG5/LeDRL.git. Index Terms—Collaborative edge computing, Task offloading, Deep reinforcement learning, Large Language Models
I. I NTRODUCTION Edge nodes distributed across different locations can cooperate to execute tasks, improving resource utilization and users’ Quality of Experience (QoE), a paradigm known as Collaborative Edge Computing (CEC). A fundamental challenge in this context is task offloading: deciding whether to execute locally or forward to neighboring nodes to minimize latency while ensuring reliability [1]. Most existing solutions rely on static heuristics or pre-trained policies that lack adaptability to runtime dynamics. However, such methods are brittle in real-world environments where node failures, link disruptions, and topology changes are frequent [2], leading to task failures and degraded performance. In contrast, adaptive offloading algorithms monitor node and network states to dynamically adjust decisions. As shown in Fig. 1, we illustrate static vs. adaptive offloading in a YOLO-based detection scenario. Initially, Node 1 attempts to offload a task to Node 4 (Fig. 1(a)). At time t1 , a failure at Node 3 causes transmission failure and inference delay under static policies (Fig. 1(b)). Adaptive methods, by contrast,
Static approach
node 5
node 5 node 3
node 3 node 1 (source)
node 6
node 2
node 1 (source)
(a)
node 4 (target)
node 6
node 2
(b)
node 4 (target)
Adaptive approach node 5
node 5
node 3 node 1 (source)
node 3
node 6
node 2
node 1 (source)
(c)
node 4 (target)
node 7 (target)
node 6
node 2
(d)
node 4
Fig. 1: Adaptive vs. static task offloading in a detection scenario. (a) Node 1 offloads to Node 4. (b) At t1 , Node 3 fails—static strategy cannot adapt. (c) Adaptive method anticipates failure and reroutes tasks via alternate paths. (d) At t2 , Node 7 joins; the adaptive strategy leverages its low load and proximity to improve latency.
can anticipate such failures and reroute tasks through more stable paths (Fig. 1(c)). Later, at t2 , Node 7 with lower load and closer proximity joins the network, and the adaptive strategy redirects the task to reduce latency (Fig. 1(d)). These dynamics underscore the need for intelligent decision-making responsive to runtime uncertainties in edge environments. Deep Reinforcement Learning (DRL) has demonstrated strong potential for autonomous task offloading in CEC settings [1], [3]. Its strength lies in learning adaptive policies through continuous interaction with dynamic environments. However, DRL still faces key limitations. First, high sample complexity leads to inefficient training. Second, in large scale networks, the state and action spaces grow exponentially due to the curse of dimensionality, which slows convergence [4] and hurts QoE for latency sensitive tasks. Moreover, node failures and topology shifts render the action space nonstationary, often resulting in sub-optimal or invalid decisions. These issues limit the scalability and robustness of DRL in real-world edge deployments. Recently, Large Language Models (LLMs) have shown impressive capabilities in contextual reasoning and knowledge generalization [5]. Emerging studies suggest that the prior knowledge encoded in LLMs can offer valuable heuristic guidance for task offloading [6], [7]. Nonetheless, using LLMs as standalone decision-makers faces two critical challenges: (i) the substantial inference latency introduced by their extensive parameterization often violates the stringent real-time requirements of edge computing, and (ii) their stochastic output behaviors can lead to inconsistent and unreliable decisions. These limitations hinder their direct applicability in delay-
sensitive and resource-constrained scenarios. To tackle these challenges, we present LeDRL, a hybrid decision framework that unifies LLM-guided semantic abstraction and DRL policy adaptation, where an attentiondriven policy fusion mechanism orchestrates their synergistic optimization. Unlike existing studies, LeDRL introduces a Reflective Evaluator that transforms LLM’s episodic guidance into structured feedback, guiding policy refinement through contextual memory. A self-attention-based fusion module is further designed to align LLM-generated intents with local observations, enabling the DRL agent to selectively absorb semantic knowledge and adapt to non-stationary environments. To balance semantic guidance and real-time performance, LeDRL integrates a lightweight LLM into the online decision loop. The compact LLM provides high-level cues to enhance DRL adaptability, while keeping inference latency low and suitable for edge deployment. This design allows LeDRL to retain the reasoning benefits of LLMs while maintaining the speed and scalability required for real-time edge systems. The main contributions are summarized as follows: • We model task offloading in collaborative edge environments as a decentralized partially observable Markov decision process (Dec-POMDP), and formulate it as a Mixed Integer Nonlinear Programming (MINLP) problem, which is proven to be NP-hard. • We propose LeDRL, a novel hybrid framework that integrates a lightweight language model with online DRL inference, enabling real-time, language-guided decisionmaking. It incorporates a self-attention-based policy alignment module to fuse LLM priors with environment feedback, enabling sample-efficient and robust adaptation to edge dynamics. • We design a context-aware reflective evaluator that converts episodic task outcomes into structured semantic feedback to guide DRL policy refinement. • Extensive evaluations show that LeDRL outperforms DRL and LLM-based baselines in success rate and convergence speed. We further implement a prototype system and deploy LeDRL on Jetson-based edge devices, demonstrating real-world feasibility and scalability.
Song et al. [7] guided single-hop offloading using promptbased LLMs, while Tong et al. [12] proposed WirelessAgent, an LLM-powered agent integrating memory and planning for wireless optimization. Liu et al. [13] focused on stabilizing edge inference by co-optimizing communication and computation. However, these works mainly address system-level orchestration or static inference, without modeling decentralized multi-hop offloading under dynamic topologies. A few recent studies begin to explore LLM-DRL hybrid paradigms. For instance, Zhu et al. [6] introduced a QTRAN–LLM framework with attention-based graph modeling for UAV task cooperation, and Ren et al. [14] used LLM-derived cognitive signals to replace handcrafted rewards in IIoT reinforcement learning. These approaches show the promise of combining LLM expressiveness with DRL adaptability, but still rely on centralized training or stable settings. In contrast, we propose LeDRL, a decentralized task offloading framework for dynamic edge networks. We model offloading as a Dec POMDP, where each edge node is an agent that acts under partial observations. LeDRL brings LLM semantic guidance into DRL training as decision priors to speed up learning and improve adaptation. We also design a topology aware masking mechanism to keep the observation input consistent when nodes or links change. This design supports robust and scalable coordination under node failures, arrivals, and volatile links. III. S YSTEM M ODEL AND P ROBLEM F ORMULATION A. System Model Task Execution Request Task Offloading
Execution Queue Receiving Queue Execution Queue
Network Connection Node workload Failure New
End Device
II. RELATED WORK
Fig. 2: System overview. Tasks arrive at distributed edge nodes. Each node maintains local execution and communication queues, and a task can be processed locally or forwarded over multiple hops before execution at a destination node.
DRL has been extensively applied to edge task offloading by formulating it as an MDP [8], enabling adaptive decision-making under dynamic workloads. Prior works, such as DAG-aware optimization [9] and log-driven cold-start training [1], focus on latency-energy trade-offs and stability. However, these methods assume static topologies, limiting their applicability in real-world edge systems with dynamic links and nodes. To improve robustness, task offloading has explored meta-heuristics for UAVs [10], and mobility-aware routing [11]. Yet, most operate under fixed infrastructure assumptions and lack adaptability to heterogeneous, timevarying edge environments. Recently, LLMs have been introduced to edge intelligence for their semantic reasoning and generalization capabilities.
We consider a collaborative edge system modeled as a timevarying undirected graph G(t) = (V(t), E(t)), where V(t) and E(t) denote the active nodes and available bidirectional links at slot t, respectively. Each node vi ∈ V(t) has computing capacity Fvi and maintains mi communication queues, one for each of its mi neighboring nodes; the transmission rate on link (vi , vj ) is Rvi vj . As shown in Fig. 2, the basic offloading unit is an indivisible and independent task [1], which matches common edge workloads such as sensing records and image frames. More complex workflows, such as DAG applications, are assumed to be pre-partitioned into such atomic tasks before online scheduling. Time is slotted with equal duration and indexed by t ∈ {1, 2, . . . , T }. In each slot, node vi receives a task Ki (t)
with probability λi , following a Bernoulli arrival process [15]. For task Ki (t), let Si (t) denote its input size and ∆i (t) its computation intensity in CPU cycles per bit [16]; the required CPU cycles are Ci (t) = ∆i (t) · Si (t). If the task generated at node vi is executed at node vj , its execution delay and one-hop transmission delay are Si (t) Ci (t) , Tijt (t) = . (1) Tijc (t) = Fv j R vi vj Besides service time, the task may wait in the transmission queue from vi to vj or in the execution queue of node vj . Let Tijtw (t) and Tijcw (t) denote these two waiting times. If the task is still queued in the next slot, the corresponding waiting time increases by one slot: Tijcw (t + 1) = Tijcw (t) + 1,
Tijtw (t + 1) = Tijtw (t) + 1. (2)
Hence, the total waiting time Wi consists of the transmissionside and computation-side queueing delays captured in (2). To characterize unreliability in dynamic edge environments, we consider software failures during task execution, hardware failures of the executing node, and transmission failures over wireless links. Following [16], all three are modeled as Poisson processes. Let αj , γj , and βij denote the software failure rate of node vj , the hardware failure rate of vj , and the link failure rate between vi and vj , respectively. Then, for a task executed at node vj , the probability of no software failure during execution interval [0, Tijc (t)] is c c Rij (t) = P Nj (Tijc (t)) = 0 = e−αj Tij (t) , (3) where Nj (Tijc (t)) is the number of software failures occurring during execution. Similarly, the probability that node vj remains hardware-operational over the same interval, and the probability that transmission from vi to vj succeeds during Tijt (t), are c
Rje (t) = e−γj Tij (t) ,
t
t Rij (t) = e−βij Tij (t) .
(4)
Since edge applications are delay-sensitive, retransmissions are bounded [17]; therefore, βij is treated as an effective link failure rate that captures the residual transmission failure probability under a limited-retry mechanism. B. Problem Formulation 1) Decision Variable: Let xij (t) ∈ {0, 1} be a binary variable with xij (t) = 1 if task Ki (t) is executed at node vj and xij (t) = 0 otherwise. 2) Delay Constraints:PEach task may be processed by at most one node, that is, j∈V xij (t) ≤ 1. So, the end-to-end delay must satisfy X Ti (t) = xij (t) Tijcw (t) + Tijc (t) + Tijtw (t) + Tijt (t) (5) j∈V ≤ Di (t). 3) Reliability Constraints: Each task must satisfy its minimum reliability requirement Φi : c t Ri (t) = Rij (t) · Rij (t) · Rje (t) = exp − (αj + γj )Tijc (t) − βij Tijt (t) ≥ Φi .
(6)
4) Resource Constraints: The cumulative resource consumption must not exceed node capacity: T N XX Ci (t) xij (t) ≤ Fvj T , t=1 i=1 T X N X (7) Si (t) xij (t) ≤ Rvi vj T , t=1 i=1 T X N X Si (t) xij (t) ≤ Mvj . t=1 i=1
Here, T is the duration of the time horizon, and Mvj is the memory capacity of node vj . 5) Optimization Objective: Our objective is to maximize the task success rate over the duration T : PT PN PN j=1 xij (t) I(Ti (t) ≤ Di (t), Ri (t) ≥ Φi ) i=1 t=1 , max PT PN PN {xij (t)} t=1 i=1 j=1 xij (t) (8) where I(·) is an indicator that equals 1 if Ti (t) ≤ Di (t) and Ri (t) ≥ Φi , and equals 0 otherwise. 6) Computational Complexity: We formulate task offloading as a mixed integer nonlinear program (MINLP) [18] by optimizing binary decisions xij (t) under the delay, reliability, and resource constraints in (5)–(7). The indicator term with binary variables leads to a combinatorial search space, while the reliability constraint introduces nonlinearity. Theorem 1: Problem (8) subject to (5)–(7) is NP- hard. Proof: We prove NP-hardness via a polynomial reduction from Bin Packing (NP-complete). Given item sizes {ai }N i=1 , m bins, and capacity Q, consider a single-slot instance with T = 1 and V = {v1 , . . . , vm }. Set Ci (1) = ai for all i ∈ {1, . . . , N } and Fvj T = Q for all j ∈ {1, . . . , m}. Choose parameters so that (5) and (6) always hold, and make the last two inequalities in (7) inactive by setting Si (1) = 0 and taking Mvj large enough. Assume each generated task is assigned to exactly one node, X xij (1) = 1, ∀i ∈ {1, . . . , N }, (9) j∈V
then (7) reduces to, for each j ∈ {1, . . . , m}, N X
ai xij (1) ≤ Q.
(10)
i=1
With the mapping xij (1) = 1 meaning that item i is placed into bin j, feasibility is equivalent to Bin Packing. The reduction is polynomial, hence the problem is NP-hard. ■ IV. APPROACH DESIGN In this section, we present LeDRL. Section IV-A formulates the online collaborative edge offloading problem as a Dec-POMDP; Section IV-B introduces an LLM-guided decision engine featuring prompt construction, dual-memory retrieval, reflection, and structured decision generation; and Section IV-C presents a DRL agent that aligns LLM guidance with local observations through an attention module to learn a stable decentralized policy under non-stationary conditions.
Decision-making Turbo DRL
Reflective Evaluator
Current Episode (short-term memory)
Episode Buffer (long-term memory)
Construct Prompt
mini-batch ( )
Encoder
LLM
self-attention
state reward task
Actor (as policy)
Critic
reward Environment: Arriving user task, edge node load, network status, etc.
state
Fig. 3: Overview of the LeDRL framework. An LLM provides semantic guidance during DRL decision-making. A self-attention fusion module merges LLM guidance with the DRL policy, and the RL agent outputs a hybrid offloading decision.
A. Dec-POMDP Formulation To enable online offloading under stochastic arrivals and topology changes, we formulate the decision process as a DecPOMDP. At time t, choosing action ati = j sets xij (t) = 1 and xik (t) = 0 for all k ̸= j. 1) State Observation: At time t, agent i ∈ N receives a local observation oti ∈ Oi : n oti = [λi , αi , γi , Fvi , lie (t), lib (t)], o [Si (t), Ci (t), Di (t), Hi (t), Wi (t)], {(Rvi vj , βij )}j∈Nit , where lie (t) and lib (t) are the execution and buffer queue lengths, Hi (t) is the current forwarding count, Wi (t) is the accumulated waiting time, and Nit is the set of currently reachable neighbors. All features are normalized to [0, 1]. If no task arrives, the task vector is set to 0. 2) Action Space and Masking: Each agent chooses ati ∈ t Ai , where ati = i denotes local execution, ati = j ∈ Nit denotes forwarding to neighbor vj , and ati = ∅ denotes no task arrival. To respect dynamic connectivity, the valid action set is Ati = {i, ∅} ∪ {j ∈ N | Mit (j) = 1}, where Mit (j) = 1 indicates that the link between nodes i and j is available at time t. 3) State Transition: The transition function P(st+1 | st , at ) is driven by stochastic task arrivals, queue updates, node service rates, and evolving network connectivity. These dynamics jointly determine the realized delay and reliability, which are evaluated by Eq. (5) and Eq. (6). 4) Reward: For each task i at time t, we define the success indicator and violation indicator as sti = I Ti (t) ≤ Di (t), Ri (t) ≥ Φi , vit = I Ti (t) > Di (t) or Ri (t) < Φi .
The team reward at time t is rt =
N X
(sti − vit ).
(11)
i=1
Under this design, maximizing the expected return is aligned with maximizing the objective in Eq. (8). 5) Objective and Policy Factorization: Our goal is to learn a factorized joint policy π θ (at | ot ) =
N Y
πθi (ati | oti ),
i=1
which maximizes the expected cumulative reward: "∞ # X ∗ t π θ = arg max Eπθ γ rt πθ
s.t.
t=0 ati ∼ πθi (· | oti ),
(12)
∀i ∈ N .
B. LLM for Context-Aware Decision Guidance 1) Prompt Construction: To activate domain-specific reasoning, we construct a structured prompt: Pt = [St ; Tt ; Nt ].
(13)
Here, St encodes node status, including capacity Fvi , queue levels, and failure rates αi and γi ; Tt includes task size Si (t), computation intensity ∆i (t), deadline Di (t), forwarding count Hi (t), and waiting time Wi (t); and Nt captures link bandwidth Rvi vj and failure rate βij . 2) Memory Repository: We maintain a dual-store memory M = {Mshort , Mlong } to provide stable and reusable context. The short-term store records recent trajectories τt = {(oti , ati , rt )}, while the long-term store keeps compact summaries of typical failures and successful patterns. To control prompt length, recent trajectories are periodically summarized
and moved from Mshort to Mlong . During prompt generation, a small context set is retrieved as Ct ← Retrieve(oti , M),
Pt ← [St ; Tt ; Nt ; Ct ].
We estimate Ât by generalized advantage estimation: δt = rt + γVωold (st+1 ) − Vωold (st ),
(14) Ât =
A memory item m is ranked by
+ w3 Simtask (Tt , m) + w4 Simload (St , m), (15) where Loc enforces node-local retrieval, Type matches task type, and Simtask and Simload favor similar task profiles and queue conditions. The top-K items are returned as Ct . 3) Reflective Evaluator: To reduce risky decisions with limited overhead, we build a short risk summary Rt from observable signals. For local execution, Rt includes expected waiting time and execution failure risk under current load. For offloading, it includes estimated transmission delay, link failure risk, and target-side load. If the outcome is poor, e.g., rt < 0, we trigger LLM introspection with Diagt = {oti , ati , rt , Rt },
(16)
which asks for the likely cause of the failure or violation and a safer choice for similar states. The returned reflection is appended to Mlong through a FIFO queue. 4) Decision-Making Turbo: The turbo module constructs a prompt at each decision step by integrating the current system observation, retrieved memory, and a risk summary. This prompt is then submitted to the LLM following a predefined output schema, which specifies that the response should indicate either local execution or offloading to a designated node. In cases where the LLM generates outputs that deviate from this schema, the system defaults to local execution to ensure safe operation.
at
The final DRL objective is Lπθ = −LCLIP (θ) − βH(πθ ),
hEnv = fθ1 (ot ), t
(25)
hLLM = fθ2 (e(aLLM )). t t
(26)
b) Attention Fusion: The query, key, and value vectors of the attention module are defined as: Q = WQ hEnv t ,
K = WK hLLM , t
γ rt+l | st , at ,
(27)
QK ⊤ √ dk
,
(28)
where dk is the key dimension. The fused feature is QK ⊤ Env ht = αt V + ht = Softmax √ V + fθ1 (ot ). (29) dk The distilled LLM feature is then gtLLM = Oθ3 (ht ).
(30)
c) Hybrid Loss Optimization: The attention module is trained by B
(18)
l=0
Aπθ (st , at ) = Qπθ (st , at ) − V πθ (st ),
V = WV hLLM . t
The attention score is
# l
(24)
where β balances policy optimization and exploration. 2) LLM Policy Distillation via Attention: The raw LLM output is discrete and non-differentiable. To align it with DRL optimization, we introduce an attention module Aθatt that distills the LLM policy and injects it into DRL learning. a) Feature Embedding: The environment observation ot is encoded by fθ1 , and the LLM decision aLLM is embedded t by a BERT-based [20] encoder e(·) followed by fθ2 :
αt = Softmax
To preserve DRL exploration while exploiting LLM guidance, we adopt a hybrid Actor–Critic framework based on MAPPO [19], where the DRL policy πθ is updated using both environmental feedback and LLM-informed priors. 1) Policy Optimization in DRL: For policy πθ , the state value, action value, and advantage functions are "∞ # X V πθ (st ) = Eπθ γ l rt+l | st , (17) Q (st , at ) = Eπθ
(22)
Here, δt is the temporal-difference (TD) error, Vωold is the value function parameterized by ωold , and λ controls the bias– variance trade-off. To encourage exploration, we add entropy regularization: X H(πθ ) = − πθ (at | st ) log πθ (at | st ). (23)
C. DRL Guided by LLM Policy
πθ
(γλ)l δt+l .
(21)
l=0
Rel(oti , m) = w1 Loc(vi , m) + w2 Type(Ki (t), m)
" l=0 ∞ X
n−1 X
(19)
where γ is the discount factor. The policy is optimized by the clipped surrogate objective h i LCLIP (θ) = Et min ρt (θ)Ât , clip(ρt (θ), 1 − ϵ, 1 + ϵ)Ât , (20) t |st ) where ρt (θ) = ππoldθ (a (at |st ) is the probability ratio, Ât is the estimated advantage, and ϵ is the clipping parameter.
1 X 2 Aθatt (oi , aLLM ) − giLLM 2 , i B i=1 B K X i,j 1 X i zπ Lact = −zπθ + log e θ, B i=1 j=1
Lfeat =
LAθatt = Lfeat + wc Lact ,
(31)
(32) (33)
where wc balances the two loss terms, B is the mini-batch size, zπi θ is the logit of the selected action, and {zπi,jθ }K j=1 are the logits of K candidate actions.
3) Fusion of LLM and RL Policies: To match the LLMderived feature space, the DRL agent also projects its observation ot into a latent representation: gtDRL = ϕDRL (ot ),
(34)
where ϕDRL is the DRL feature encoder. Then, the fused representation for policy execution is gtF = (1 − λllm )gtDRL + λllm · ϕLast (gtLLM ),
(35)
where ϕLast (·) projects LLM guidance into a DRL-compatible space. The fusion coefficient is updated as ( (t−1) min(βλ λinit , λllm η), if t mod Tλ = 0, (t) λllm = (36) (t−1) max(λmin , λllm γdecay ), otherwise, where Tλ is the scheduled update interval, βλ caps the maximum fusion weight, and η, γdecay ∈ (0, 1) control its decay. The final objective is min Lπθ + LAθatt . (37) The pseudo-code of LeDRL is given in Algorithm 1. Algorithm 1: Policy Optimization in LLM-Enhanced DRL (LeDRL) Input : Initialize DRL policy πθ , attention module Aθatt , memory M = {Mshort , Mlong }, fusion weight λllm = λinit , LLM policy πLLM , fusion update interval Tλ , total episodes N , and episode length T . Output: Optimized policy πθ∗ . for episode ← 1 to N do Observe initial state s0 ; 3 for t ← 0 to T do 4 Construct prompt Pt = [St ; Tt ; Nt ]; 5 Retrieve context Ct ← Retrieve(ot , M); ← πLLM (Pt , Ct ); 6 Generate LLM decision aLLM t ← fθ1 (ot ); 7 Encode state: hEnv t )); ← fθ2 (e(aLLM 8 Encode LLM response: hLLM t t 9 Compute fused feature using Eq. (35); 10 Sample action at ∼ πθ (gtF ); 11 Observe reward rt and next state st+1 ; 12 Store (ot , at , rt ) into Mshort ; 13 if rt < 0 then 14 Update long-term memory: Mlong ← Reflect(ot , at , rt , Rt ); 1
2
15 16 17 18 19 20 21
if t mod Tλ = 0 then λllm ← min(βλ λinit , ηλllm ); else λllm ← max(λmin , γdecay λllm ); Compute advantage Ât using Eq. (22); Update policy: θ ← θ − ∇θ Lπθ ; Update attention: θatt ← θatt − ∇θatt LAθatt ;
V. EVALUATION A. Experiment Setup All experiments are conducted on a workstation with an AMD Ryzen Threadripper 3990X CPU, an NVIDIA RTX 4090 GPU, and 256 GB RAM. Environment Configuration. We simulate a decentralized edge system over 100 time slots [1], with 10 ∼ 20 heterogeneous nodes, stochastic task arrivals, and task input sizes uniformly sampled from [2,000, 4,000] KB [1]. Task complexity ranges from [800, 2,400] cycles/bit, and deadlines are fixed at 4 s [16]. The network topology is randomly generated, sparse, and connected to model partial inter-node connectivity. Each node has a 3 GHz CPU, and link rates are sampled from [10, 40] MB/s [16]. To capture network dynamics, we set the node failure rate to 0.01 and the node appearance probability to 0.1 [21]. Model Configuration. LeDRL is implemented in PyTorch and integrates Qwen3-4B [5] to guide DRL policy learning. It adopts an actor–critic architecture with MLP-based networks of hidden size 64, and uses a self-attention module with embedding dimension 8, maximum length 512, and dropout 0.1. Following [16], PPO is trained with learning rate 0.0004, decay factor 0.99, and discount factor γ = 0.99, and is evaluated every 4 iterations. B. Baseline Approaches We compare LeDRL with six baselines, covering three DRL methods, two heuristics, and one LLM framework. VDN-TO [22]: a cooperative MARL baseline that factorizes the joint Q-function into per-node utilities for local execution or neighbor forwarding. MAPPO-TO [19]: a policy-gradient baseline that trains node-local policies with clipped updates to improve robustness under non-stationarity. MASAC-TO [23]: a multi-agent SAC baseline adapted to discrete actions to match our offloading decision space. RATC. A lightweight heuristic [24] that samples two neighbors and selects one based on deadline and reliability. AGSP [25]. A hybrid-PSO engine. We transplant its adaptive genetic–simulated annealing kernel to our setting. Upon each task arrival, it decides whether to execute locally or forward the task to the best neighbor. Reflexion [26]. an LLM-driven baseline that prompts Qwen3-4B with structured task–state–outcome tuples and uses iterative self-feedback to refine decentralized offloading decisions during training. All methods are tuned and evaluated on each topology over 10 randomized runs. We report mean task success rates with standard deviations for fair and reproducible comparison. C. Performance on Simulation 1) Comparison of Training Performance: Fig. 4 shows the training curves of LeDRL and all baselines in terms of task success rate under different topologies. LeDRL converges faster, exhibits lower variance in the early stage of training, and achieves the highest final success rate in all settings. This
TABLE I: Task success rate and inference time across different models. Bold indicates the best performance.
0.7
0.65 0.60
0.6
0.50
Success rate
Success rate
0.55
0.45 0.40
10 Nodes
0.5
0.4
0.35
VDN-TO MAPPO-TO MASAC-TO LeDRL
0.30 0.25 0
5000
10000
15000 Episode
20000
(a) 10-nodes
Reflexion RATC AGSP
25000
30000
VDN-TO MAPPO-TO MASAC-TO LeDRL
0.3 0
Reflexion RATC AGSP
1500 3000 4500 6000 7500 9000 10500 12000 13500 15000 Episode
(b) 20-nodes
Fig. 4: Learning curves of task success rates under different methods for different network topologies.
directly supports our motivation: although standard DRL can be applied to task offloading, it often requires many samples and may become trapped in poor local optima in dynamic networks. For example, MAPPO-TO improves rapidly at the beginning but becomes unstable later and often struggles to escape sub-optimal regions without semantic priors. MASACTO learns quickly in small topologies but remains unstable, and in the 20-node case it often plateaus at a sub-optimal level, indicating limited scalability as the problem becomes more complex. VDN-TO is relatively stable but maintains a low success rate due to its limited adaptability. LeDRL improves sample efficiency by using LLM-guided priors to guide early exploration and enhance exploration quality. Reflective feedback further refines these priors, while the self-attention fusion module aligns semantic cues with local observations, thereby stabilizing learning under topology changes and node failures. By contrast, Reflexion, AGSP, and RATC can exploit structural information or handcrafted guidance, but they lack effective feedback-driven adaptation, so their performance varies more significantly and degrades in complex topologies. 2) Robustness Performance: We evaluate robustness on a fixed 10-node topology under four perturbations: task size, computation intensity, execution failures, and transmission failures (Fig. 5). As task input size and computation intensity increase, success rates generally decline because larger tasks and heavier computation increase transmission time, execution time, and queueing delay under limited bandwidth and computing resources, thereby causing more deadline violations, as shown in Fig. 5(a–b). LeDRL consistently performs best because LLM guidance provides feasible decision priors in the early stage, while the learned policy further adapts to local congestion and topology variations. For failure perturbations, several baselines show nonmonotonic trends, with alternating rises and drops as failure probability increases, as shown in Fig. 5(c–d). This is expected in decentralized offloading because varying failure rates can shift the dominant bottleneck and change the relative benefit of forwarding versus local execution. Consequently, the same policy may perform differently across failure regimes, while stochastic arrivals and queueing dynamics further amplify fluctuations. Despite these shifts, LeDRL consistently achieves the highest success rate and better stability than the baselines. This advantage comes from reliability-aware LLM priors and self-attention fusion, which
20 Nodes
Model
Success (%) ↑
Time (s) ↓
Success (%) ↑
Time (s) ↓
RATC AGSP VDN-TO MASAC-TO MAPPO-TO Reflexion LeDRL
45.48 ± 1.24 50.01 ± 1.63 44.47 ± 1.92 51.81 ± 2.88 52.68 ± 3.40 48.34 ± 1.97 59.46 ± 1.42
0.0004 0.0005 0.0028 0.0025 0.0035 1.5786 0.7046
59.98 ± 1.45 39.48 ± 1.72 51.27 ± 1.82 51.80 ± 1.77 60.68 ± 2.24 60.86 ± 1.39 63.78 ± 1.68
0.0008 0.0018 0.0026 0.0024 0.0043 3.0128 0.7379
adaptively combine semantic guidance with local observations as failure patterns evolve. At a failure rate of 0.25, LeDRL improves success rate by about 12% and 17% over MAPPO-TO under execution-failure and transmission-failure settings, respectively. In contrast, VDN-TO and MASACTO are less reliability-aware and generalize poorly, Reflexion adapts slowly to fast disturbances, and RATC and AGSP rely on fixed rules and become unstable when the bottleneck shifts. 3) Impact of the network topology: We evaluate LeDRL on two network scales (10 and 20 nodes) and two topology types: connected random graphs and structured ring topologies. As shown in Fig. 6a, LeDRL achieves the highest task success rate in all settings. In contrast, MAPPO-TO performs well in random graphs but drops markedly in ring topologies, where routing flexibility is limited and congestion can propagate along the ring. RATC and AGSP are even more sensitive to topology changes and show larger performance fluctuations, reflecting their reliance on fixed decision rules. LeDRL remains robust because it combines semantic guidance with adaptive learning. The LLM guidance provides topology-aware priors, such as avoiding risky relays and preferring feasible paths under tight delay and reliability constraints, which improves early exploration and reduces wasted interactions. The DRL policy then refines these priors using local observations, so it can react to queue build-up and link variability in different graphs. By abstracting task and network conditions into compact semantic cues, LeDRL reduces the effective decision complexity and improves generalization when scaling from 10 to 20 nodes and when switching between random and ring structures. 4) Ablation Study: Fig. 6b compares LeDRL with variants that remove or replace key components. Plain MAPPO yields the lowest success rate, as it relies on local trial-anderror and has no explicit mechanism to handle long-horizon risk under dynamic edge conditions. Adding LLM guidance with a simple MLP (e.g., LLM+MLP+MAPPO) improves performance, but the gain is limited because raw semantic hints are not reliably aligned with the policy input. Replacing the MLP with self-attention (e.g., RATC+SA+MAPPO and AGSP+SA+MAPPO) further improves success rate, since attention can better fuse heterogeneous signals and reduce policy drift. However, these variants still lack context-aware semantic refinement, so they converge to suboptimal policies. LeDRL achieves the best performance by closing this gap. Its structured prompts and context-aware memory retrieval provide relevant semantic priors, and the reflective evaluator
VDN-TO MAPPO-TO MASAC-TO LeDRL
0.7
0.70
Reflexion RATC AGSP
VDN-TO MAPPO-TO MASAC-TO LeDRL
0.65
0.6
0.4 0.3 0.2
Reflexion RATC AGSP
0.65
0.60
0.5 0.4
0.55 0.50
0.1 4000
6000 8000 Task size (KB)
10000
1600
2400 3200 4000 Task complexity (cycles/bit)
4800
0.60 0.55 0.50
0.40
0.40
2000
Reflexion RATC AGSP
0.45
0.45
0.3
VDN-TO MAPPO-TO MASAC-TO LeDRL
0.70
Success rate
Success rate
Success rate
0.5
Reflexion RATC AGSP
Success rate
VDN-TO MAPPO-TO MASAC-TO LeDRL
0.6
0.1
0.15 0.2 0.25 Execution failure probability
0.1
0.3
0.15 0.2 0.25 Transmission failure probability
0.3
Fig. 5: Success rate of tasks under different:(a) size; (b) complexity; (c) execution failure rates; (d) transmission failure rates. 0.8 VDN-TO MAPPO-TO MASAC-TO LeDRL
Physical Layer
0.60
Reflexion RATC AGSP
0.58
0.6
Success rate
Success rate
0.7
0.5
MAPPO RATC + Self-Attention + MAPPO AGSP + Self-Attention + MAPPO LLM + MLP + MAPPO LeDRL
CoEdgeSys Worker in each Jetson Node
Decentralized Full-Mesh From Other Jetsons Object Detection Task
0.56
3
Main Task Queue
gRPC
3
2
1
1
Remote LLM Server Real-time
2 System Status
Scheduler Algorithm (e.g., LeDRL)
- CPU/GPU Util - Queue Occupancy
GPU
0.54 From Local Scheduler
0.4
4
In-ThreadPool Execute Queue 3
0.52
2
CPU
1
Qwen3-4B (w/RTX 4090)
Send to local 4 or other Jetsons 3
Primary Local Task: YOLOv8 Model Detector
5
LLM guidance requests (via gRPC)
6
3
1
2
1
3
1
AIGC
Results
PyTorch
Real Network Topology
0.3 ring(20)
0.50
O
MAPP
RATC
+ SA
L PPO PPO APPO LeDR + MA + MA P+M + SA + ML AGSP LLM
Fig. 7: Internal architecture of the CoEdgeSys running on each edge device. 0.65
0.9
updates these priors based on episodic outcomes. The selfattention fusion then selectively integrates the refined guidance with local observations, leading to faster convergence and more stable policies. 5) Inference Efficiency and Performance Trade-off: Table I reports task success rate and inference latency across different network scales. Heuristic methods such as RATC and AGSP achieve the lowest latency, but their success rates decline in dynamic settings because they cannot effectively adapt to failures or topology changes. LeDRL achieves the highest success rate in both 10- and 20-node networks while maintaining moderate inference latency, remaining compatible with the 4 s delay requirement. This advantage comes from LeDRL’s collaborative inference design: a lightweight LLM provides semantic guidance at decision time, helping the policy select more reliable routes under changing conditions. Although this introduces additional overhead, it yields a better trade-off between latency and robustness. For example, in the 10-node setting, LeDRL improves task success rate by 12.87% over MAPPO-TO. VI. E XPERIMENTS IN R EAL -W ORLD S CENARIOS A. System Implementation and Algorithm Deployment To evaluate deployment feasibility, we build CoEdgeSys, a collaborative edge prototype with 10 heterogeneous Jetson nodes (4 Jetson Nano and 6 Jetson Xavier NX), all connected via a wired Gigabit LAN. Each node runs a scheduler and a YOLOv8 detector, and evaluation is conducted on the public COCO dataset. The scheduler supports LeDRL and all baselines. The actor network contains three fully connected layers and is smaller than 100KB, incurring only limited overhead on Jetson devices. For LLM guidance, Qwen3-4B is deployed on a server with a single RTX 4090 GPU, and CoEdgeSys enables parallel request handling to reduce pertask guidance latency, as validated in Fig. 8.
1.0 Success Rate Delay Time
0.48
0.60
0.8
0.55
0.7
0.9 0.46
Delay Time (s)
Fig. 6: Performance analysis of LeDRL: (a) network topology and (b) ablation study of key components.
0.50
Success Rate Delay Time
(b) Ablation study of LeDRL
Success Rate
(a) Impact of the topology
0.50
0.6
0.45
0.5
Delay Time (s)
ring(10) random(20) Network Topology
Success Rate
random(10)
0.44
0.8
0.42
0.7 0.40 0.38
0.6
0.36 0.40
-TO
VDN
O-TO SAC-TO LeDRL MAPP MA
xion
Refle
RATC
AGSP
0.4
(a) Confidence threshold λsuc = 0.4
-TO
VDN
O-TO SAC-TO LeDRL MAPP MA
xion
Refle
RATC
AGSP
0.5
(b) Confidence threshold λsuc = 0.5
Fig. 8: LeDRL success rate under different YOLO Confidence (λsuc = 0.4 / 0.5)
As shown in Fig. 7, at each slot t, tasks randomly arrive at Jetson nodes ( 1 ). The local scheduler monitors realtime system status, including CPU/GPU utilization and queue occupancy, and decides whether to execute the task locally or offload it ( 2 ). Local tasks enter the ThreadPool Execute Queue ( 4 ), while offloaded tasks are sent via gRPC to a neighboring Jetson ( 3 ). The assigned node runs YOLOv8, and the inference result is returned to the originating task generator ( 5 6 ). The end-to-end delay is measured from task dispatch to result reception. B. Testbed Results To emulate real-world dynamics, we randomly activate denial-of-service (DoS) states on a subset of edge nodes during specific intervals. Tasks assigned to these nodes are treated as failed. Additionally, a strict 4 second latency constraint is enforced—any task exceeding it is marked as unsuccessful. We evaluate two failure intensity based on YOLO confidence thresholds: 0.4 (Fig. 8a) and 0.5 (Fig. 8b). In both cases, LeDRL achieves the highest task success rate (up to 60%), outperforming the best baseline by 13.5%. This advantage is rooted in LeDRL’s hybrid training strategy, where LLM-guided policy shaping and attention-based fusion enable the agent to internalize failure-aware heuristics and topological sensitivity. Rather than relying on rapid but shallow inference like DRL, LeDRL allocates slightly more inference time to reason over memory and past experiences, leading to more informed decisions. Importantly, this improved decision quality is achieved without significant
increase in task latency, as LeDRL keeps latency close to DRL while achieving higher success rate under runtime disruptions. Under higher fault intensities, baselines such as VDNTO and RATC exhibit sharp performance degradation due to limited adaptability. In contrast, LeDRL sustains stable performance, demonstrating strong resilience to topological shifts and execution time failures. These results confirm that the decision quality advantage observed in simulation transfers effectively to real-world, delay-sensitive edge environments. VII. CONCLUSION We present LeDRL, a hybrid decision framework that integrates LLM with self-attention-based DRL for decentralized task offloading in dynamic edge environments. LeDRL introduces two key innovations: a Reflective Evaluator that converts episodic outcomes into structured feedback and retrieves relevant memory to refine decisions, and a SelfAttention Fusion module that aligns LLM priors with learned policies to speed up convergence and improve robustness under topology and failure dynamics. Extensive simulations and a Jetson-based prototype show that LeDRL achieves superior performance over state-of-the-art baselines across multiple dimensions including success rate, adaptability, and inference efficiency. In future work, we will explore decoupling the LLM from online execution while preserving the robustness gains achieved during training, and we will also study scaling LeDRL to larger networks. ACKNOWLEDGMENT We sincerely thank the anonymous reviewers for their valuable feedback. This work was supported in part by Hong Kong RGC Theme-based Research Scheme (TRS) under Grant T43-513/23-N, in part by the NSFC and Hong Kong RGC Collaborative Research Scheme under Grant 62321166652, in part by the Guangdong Basic and Applied Basic Research Foundation under Grant 2025A1515011996, and in part by the Fundamental Research Funds for the Central University under Grant CXTD202406. R EFERENCES [1] Hongcai Lin, Lei Yang, Hao Guo, and Jiannong Cao. Decentralized task offloading in edge computing: an offline-to-online reinforcement learning approach. IEEE Transactions on Computers, 2024. [2] Aleksej Chinaev, Niklas Knaepper, and Gerald Enzner. Online distributed waveform-synchronization for acoustic sensor networks with dynamic topology. EURASIP Journal on Audio, Speech, and Music Processing, 2023(1):55, 2023. [3] Kaya Kuru. Definition of multi-objective deep reinforcement learning reward functions for self-driving vehicles in the urban environment. IEEE Transactions on Intelligent Transportation Systems, 2023. [4] KaiXuan Kang, Ding Ding, HuaMao Xie, et al. Imitation learning enabled fast and adaptive task scheduling in cloud. Future Generation Computer Systems, 154:160–172, 2024. [5] An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint arXiv:2505.09388, 2025. [6] Feifan Zhu, Fei Huang, Yantao Yu, Guojin Liu, and Tiancong Huang. Task offloading with llm-enhanced multi-agent reinforcement learning in uav-assisted edge computing. Sensors, 25(1):175, 2024.
[7] Youngjin Song, Wookjin Lee, and Sang Hyun Lee. Task offloading with large language models in mobile edge computing. In 2024 15th International Conference on Information and Communication Technology Convergence (ICTC), pages 917–921. IEEE, 2024. [8] Shi Dong, Junxiao Tang, Khushnood Abbas, Ruizhe Hou, Joarder Kamruzzaman, Leszek Rutkowski, and Rajkumar Buyya. Task offloading strategies for mobile edge computing: A survey. Computer Networks, page 110791, 2024. [9] Jin Wang, Jia Hu, Geyong Min, and Wenhan Zhan. Dependent task offloading for edge computing based on deep reinforcement learning. IEEE Transactions on Computers, 71(10):2449–2461, 2021. [10] Loc X Nguyen, Yan Kyaw Tun, Tri Nguyen Dang, Yu Min Park, and Han. Dependency tasks offloading and communication resource allocation in collaborative uav networks: A metaheuristic approach. IEEE Internet of Things Journal, 10(10):9062–9076, 2023. [11] Liang Zhao, Enchao Zhang, Shaohua Wan, Ammar Hawbani, Al-Dubai, et al. Meson: A mobility-aware dependent task offloading scheme for urban vehicular edge computing. IEEE Transactions on Mobile Computing, 23(5):4259–4272, 2023. [12] Jingwen Tong, Wei Guo, Jiawei Shao, Qiong Wu, Zijian Li, Zehong Lin, and Jun Zhang. Wirelessagent: Large language model agents for intelligent wireless networks. arXiv preprint arXiv:2505.01074, 2025. [13] Chang Liu and Jun Zhao. Resource allocation for stable llm training in mobile edge computing. In Proceedings of the Twenty-fifth International Symposium on Theory, Algorithmic Foundations, and Protocol Design for Mobile Networks and Mobile Computing, pages 81–90, 2024. [14] Yuzheng Ren, Haijun Zhang, F Richard Yu, Wei Li, Pincan Zhao, and Ying He. Industrial internet of things with large language models (llms): an intelligence-based reinforcement learning approach. IEEE Transactions on Mobile Computing, 2024. [15] Ming Tang and Vincent WS Wong. Deep reinforcement learning for task offloading in mobile edge computing systems. IEEE Transactions on Mobile Computing, 21(6):1985–1997, 2020. [16] Hao Guo, Lei Yang, Qingfeng Zhang, and Jiannong Cao. Hybrid redundancy for reliable task offloading in collaborative edge computing. IEEE Transactions on Computers, 2025. [17] Sunghyun Choi, Youngkyu Choi, and Inkyu Lee. Ieee 802.11 maclevel fec scheme with retransmission combining. IEEE Transactions on Wireless Communications, 5(1):203–211, 2006. [18] Huan Zhou and Zhenyu Zhang. Joint optimization of computing offloading and service caching in edge computing-based smart grid. IEEE Transactions on Cloud Computing, 11(2):1122–1132, 2022. [19] Chao Yu, Akash Velu, Eugene Vinitsky, Jiaxuan Gao, Yu Wang, Alexandre Bayen, and Yi Wu. The surprising effectiveness of ppo in cooperative multi-agent games. Advances in Neural Information Processing Systems, 35:24611–24624, 2022. [20] Jacob Devlin, Ming-Wei Chang, et al. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers), pages 4171–4186, 2019. [21] Simon Knight, Hung X Nguyen, Nickolas Falkner, Rhys Bowden, and Matthew Roughan. The internet topology zoo. IEEE Journal on Selected Areas in Communications, 29(9):1765–1775, 2011. [22] Peter Sunehag, Guy Lever, Audrunas Gruslys, Wojciech Marian Czarnecki, Vinicius Zambaldi, Max Jaderberg, Marc Lanctot, et al. Valuedecomposition networks for cooperative multi-agent learning. arXiv preprint arXiv:1706.05296, 2017. [23] Tuomas Haarnoja, Aurick Zhou, Pieter Abbeel, and Sergey Levine. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. In International conference on machine learning, pages 1861–1870. Pmlr, 2018. [24] Roberto Beraldi and Gabriele Proietti Mattia. Power of random choices made efficient for fog computing. IEEE Transactions on Cloud Computing, 10(2):1130–1141, 2021. [25] Haitao Yuan, Meijia Wang, Bi, et al. Cost-efficient task offloading in mobile edge computing with layered unmanned aerial vehicles. IEEE Internet of Things Journal, 11(19):30496–30509, 2024. [26] Noah Shinn, Federico Cassano, et al. Reflexion: Language agents with verbal reinforcement learning. Advances in Neural Information Processing Systems, 36:8634–8652, 2023.