RISE: Relay Inference and Online Scheduling for Efficient Edge-Device Collaborative Diffusion Model Services Zilan Huang1, 2 , Zhiqing Tang2, 1 , Hanshuai Cui3, 2 , Tian Wang2 , Yuan Wu4 , Weijia Jia2, 5 , Wei Zhao6 1 Faculty of Arts and Sciences, Beijing Normal University, Zhuhai, China 2 Institute of Artificial Intelligence and Future Networks, Beijing Normal University, Zhuhai, China 3 School of Artificial Intelligence, Beijing Normal University, Beijing, China
arXiv:2606.17378v1 [cs.DC] 16 Jun 2026
4 State Key Laboratory of Internet of Things for Smart City, University of Macau, Macau SAR, China 5 Guangdong Key Lab of AI & Multi-Modal Data Processing, Beijing Normal-Hong Kong Baptist University, Zhuhai, China 6 Faculty of Computer Science and Artificial Intelligence, Shenzhen University of Advanced Technology, Shenzhen, China
Email: {zilanhuang, hanshuaicui}@mail.bnu.edu.cn, {zhiqingtang, tianwang, jiawj}@bnu.edu.cn, [email protected], [email protected]
Abstract—Text-to-image diffusion models are increasingly deployed at the network edge to serve heterogeneous workloads with diverse quality and latency requirements. However, existing deployment strategies choose either large edge-side models with high fidelity but high latency or lightweight device-side models that offer speed at the cost of semantic coherence. Moreover, these approaches rarely split the denoising workload between models of different sizes across edge servers and user devices. To bridge this gap, we propose RISE, a method for edgedevice diffusion model services that combines relay inference with online scheduling. Driven by the finding that the latent intensity exhibits minimal deviation after a model handoff, RISE uses a training-free relay mechanism that exploits the shared latent space within a model family: the large model on the edge handles the early denoising steps that shape semantic structure, then passes the intermediate latent to a small device-side model for detail refinement. To deploy this mechanism as a practical service, a contextual bandit scheduler selects the best relay configuration based on prompt complexity, user preferences, network quality and real-time node loads. Experiments on two benchmarks show that RISE’s relay mechanism achieves up to 2.1× speedup while preserving full-model quality, and its context-aware scheduler effectively balances quality and latency under mixed workloads. Index Terms—diffusion models, edge computing, relay inference, service scheduling, contextual bandits
I. I NTRODUCTION Collaboration between large and small models has become a key research direction in service computing [1]. In natural language processing, techniques such as speculative decoding This work was supported in part by the National Natural Science Foundation of China (NSFC) under Grant 62302048, Grant 62272050, and Grant U25A20436; in part by the Science and Technology Development Fund of Macau SAR under Grants 0028/2025/AFJ and 0021/2025/RIA1; in part by Guangdong Higher Education Association under Grant 24GQN97; in part by the Guangdong Provincial Higher Education Institutions under Grant 2024KTSCX219; and in part by Beijing Normal University at Zhuhai Education Reform Project under Grant jx2025037. (Corresponding author: Zhiqing Tang.)
[2]–[4] have shown that pairing a large model with a smaller one can reduce cost while keeping output quality high. For text-to-image generation, diffusion models now follow a similar pattern. Model families like SDXL [5] and Stable Diffusion 3 [6] offer both a large server-grade variant and a compact version designed for lighter hardware, and models within the same family share a compatible latent space [5], [7]. This compatibility gives a natural basis for collaboration between the two scales. Most prior studies focus on speeding up a single model [8], [9] or distributing one model across identical devices [10]. However, few studies explore the collaboration between a large and a small diffusion model deployed on an edge server and a user device, where the two sides differ significantly in computing power yet both generation quality and inference speed must be maintained. Current approaches to deploy diffusion models can be roughly divided into three types, with each of them suffering some drawbacks. (i) Edge-only inference places a large model such as SD3.5 Large on GPU-equipped edge servers. The output quality is high, but every image incurs high latency and heavy GPU usage [11], making it hard to serve latency-sensitive requests. (ii) Device-only inference runs a lightweight model such as Segmind-Vega [12] directly on the user’s device. Network transfer is avoided and prompt privacy is preserved, but the limited capacity of these models often produces semantic errors and visible artifacts when the input prompt is complex [13]. (iii) Existing collaborative inference approaches, including split inference for CNNs and LLMs [14], relay diffusion across different resolutions [15], [16], and cloud-edge collaboration for LLM serving [17], are either confined to a single device or run inside a cluster of identical machines, and do not split the denoising workload between an edge server and a user device that have very different computing capabilities. Beyond single-request inference, a practical service should be able to handle many users at the
Context Construction
Prompt
Prompt Complexity
Action
QoS Preference
Arm Selection Model Pair
Reward
Relay Step
Network Quality
ENVIRONMENT
AGENT
Node ID
Node Load Phase 2： Device Denoising Generated Image
[x1 ] ... [x s-1 ]
User Device
Latent Transfer
Phase 1： Edge Denoising
[x s ] ... [x T -1 ] [x T ]
Edge Server
Fig. 1: System overview of RISE integrating relay inference and online scheduling
same time, routing each request to a suitable node under changing loads. Relay inference and online scheduling for edge-device diffusion model services therefore remain open problems, and two challenges should be addressed. The first challenge is how to split a denoising task between a large edge model and a small device model without sacrificing quality. In diffusion models, different denoising steps have different roles, and handing off at the wrong step can break the image’s semantic coherence [18], [19]. Existing relay diffusion methods [15], [16] pass a low-resolution result into a high-resolution diffusion chain, base-refiner pipelines such as SDXL run two models in sequence on the same machine, and eDiff-I [20] assigns different expert networks to different noise levels. However, all of these methods either relay between fixed resolution stages or within a predefined model pair on identical hardware, and none of them hands the denoising work from a large edge model to a smaller device model of a different capacity. How to determine a valid relay point across different noise schedules and step counts while controlling quality degradation remains challenging. The second challenge is how to select the best relay configuration for each incoming request under changing system loads. Different model pairs combined with different relay steps create a large discrete configuration space, and the best choice depends on prompt complexity, user quality-latency preference, network quality and real-time node occupancy. Concurrent requests also compete for shared edge GPU resources, which makes the overall optimization NP-hard [21]. Existing edge scheduling methods model the problem as mixed-integer programs [22] or apply Lyapunov optimization [23], but the former scales poorly and the latter assumes convex cost functions that do not hold for the discrete qualityspeed tradeoffs in image generation. Deep reinforcement learning methods such as PPO [24] and SAC [25] have also been used for edge task offloading, but they require large amounts of training data and the learned policies are hard to interpret. Even recent QoS-aware AIGC scheduling work [26] focuses on throughput for a single model rather than jointly choosing
a model pair, a relay step, and a target node. None of these methods captures the relay-specific quality-latency tradeoff unique to diffusion generation, and efficiently scheduling relay configurations in this setting remains an open problem. To address these two challenges, we propose RISE, a framework denoting Relay Inference and online Scheduling for efficient Edge-device collaborative diffusion model services. For the first challenge, RISE exploits the observation that early denoising steps build coarse semantic structure while later steps only add fine details [18], [19]. Since models within the same family share an identical latent space, the large edge model can complete the early semantically important steps and pass the intermediate latent directly to the small device model, which finishes the remaining detail refinement without any retraining. For the second challenge, RISE uses a LinUCB-based contextual bandit scheduler that encodes prompt complexity, user preference, network quality and realtime node loads into a context vector and selects the best relay configuration per request. In summary, this paper makes the following contributions. 1) Relay Inference. We propose a training-free relay inference mechanism for edge-device diffusion models that splits the denoising process between a large edge model and a small device model. It achieves significant speedup while preserving most of the large model’s generation quality, without any retraining or fine-tuning. 2) Online Scheduling. We model the joint selection of model pair, relay step, and target node as a contextual bandit problem and develop a scheduling algorithm that adapts to prompt complexity, user preferences, network quality and changing edge loads. 3) Experimental Validation. We conduct experiments on two benchmarks with five complementary metrics, demonstrating that our relay mechanism achieves up to 2.1× speedup while preserving full-model quality, and our online scheduler effectively balances quality and latency under mixed service workloads. The remainder of this paper is organized as follows. Sec-
TABLE I: Summary of Key Notations
tion II reviews related work. Section III presents the relay inference mechanism. Section IV describes the online scheduling algorithm. Section V reports the experimental results. Section VI concludes the paper.
B. Collaborative Inference and Relay Diffusion Running diffusion models outside centralized data centers has motivated several collaborative inference strategies, including conditional diffusion under noisy conditions [32], collaborative inference in vehicular networks [33], and cloudedge LLM serving [17]. The iterative nature of diffusion denoising further enables cross-model splitting. eDiff-I [20] assigns different expert networks to different noise levels, SDXL adopts a two-stage base-refiner pipeline, and Relay Diffusion [15] together with CogView3 [16] relay across resolutions in latent space. These relay methods, however, all operate on a single machine or a cluster of identical nodes. None of them transfer the denoising workload from a large edge model to a smaller device model with different capacity, which is the setting our relay inference mechanism targets. C. Inference Scheduling on Heterogeneous Edge Clusters Inference scheduling has been extensively studied. Classical approaches formulate it as a mixed-integer program [22], [34] or apply Lyapunov optimization [23], while heuristic methods [35], [36] trade optimality for better scalability. Deep reinforcement learning has also been applied in this area, for example, PPO [24] and SAC [25] have been used for cluster resource management, and multi-armed bandit algorithms [37], [38] provide a more lightweight alternative with provable regret bounds. Recent work has begun exploring QoS-aware scheduling and bandit-based formulations for AIGC and LLM workloads at the edge [26], [39], [40]. These methods, however, all optimize for a single model and do not jointly select a model pair, a relay step, and a target node. RISE fills this gap with a LinUCB-based scheduler operating over a discrete action space that encodes all three decisions.
Description
x0 , x t , x T T, Te , Td s, s′ σt ᾱt ϵθ , v θ e δt ML , MS A, at ct , r t QCLIP , QIR , QPick QAes , QOCR 600 550
Clean image, noisy latent at step t, pure noise Total / edge / device denoising steps Relay step (edge) and start step (device) Noise level at step t Cumulative noise schedule coefficient Noise prediction / velocity field network Prompt embeddings Per-step divergence between ML and MS Large (edge) and small (device) model Action space and selected arm at round t Context vector and reward at round t CLIP Score, ImageReward, PickScore Aesthetic Score and OCR accuracy
4
SD3.5 Large SD3.5 Large + Medium
3 t (%)
Latent Intensity
II. R ELATED W ORK A. Diffusion Model Acceleration Most research on accelerating diffusion models focuses on a single model. On the training side, trajectory stitching [27], inference-time distillation [28], and consistency tuning [8] reduce the number of denoising steps required. Training-free approaches also exist: Denoising Diffusion Implicit Models [29] replaces the stochastic sampler with a deterministic one; Latent Diffusion Models [7] move the denoising loop into a smaller latent space; and cache-based methods such as DeepCache and BWCache reuse intermediate features to reduce redundant computation during denoising [30], [31]. These efforts have produced models spanning a wide range of scales, from large server-level models such as SDXL [5] and Stable Diffusion 3 [6] to lightweight on-device models. They all share one limitation, however: they optimize within a single model and do not consider splitting diffusion workloads across an edge-device architecture where a large model and a small model collaborate.
Symbol
500 450 400 20
2 1
25
30
35
40
Denoising Step
45
50
(a) Latent intensity comparison
0 20
25
30
35
40
Denoising Step
45
50
(b) Per-step relative deviation ρt
Fig. 2: Latent trajectory analysis
III. P ROPOSED M ETHODOLOGY As illustrated in Figure 1, RISE operates on a heterogeneous edge-device system where the edge hosts large models ML (e.g., SDXL, SD3.5 Large) and the device runs lightweight models MS (e.g., Segmind-Vega, SD3.5 Medium). This section first presents the empirical motivation for splitting the denoising workload between ML and MS , then details the relay implementation for each model family. Table I summarizes the key notations used throughout this paper. A. Empirical Motivation The denoising process of diffusion models has a notable two-phase structure. In the early steps, when the signal-tonoise ratio SNR(t) = ᾱt /(1− ᾱt ) is low, the model establishes coarse semantic layout such as object placement and spatial composition. In the later steps, as the SNR increases, each step only adds fine-grained details that are nearly imperceptible to human viewers [18]. This transition is also reflected in the attention mechanism: cross-attention outputs converge to a fixed point within the first few steps [19], meaning that the generation trajectory becomes largely determined early on. These properties suggest that deploying a large model for the full denoising process is unnecessary. Instead, ML can handle the early semantic-critical steps, and a lightweight MS can take over the remaining refinement with limited quality degradation in our empirical setting. We ran a latent trajectory comparison experiment on the SD3.5 family to test this idea. We let SD3.5 Large run the full 50 denoising steps, and also ran a relay setup where
Device Model
Edge Model
... ��
...
Latent Transfer
��−�
��−�
��
Semantic
��−�
��
Detail
Fig. 3: The process of relay inference
Large handled the first 20 steps and Medium took over for the remaining 30. After the relay point, we recorded the latent intensity ||xt ||2 at each step. As Figure 2a shows, the two curves almost overlap after the handoff, meaning Medium did not deviate from the denoising direction that Large had established. To put a number on this gap, we calculated a perstep relative deviation: ρt =
∥xlarge ∥2 − ∥xrelay ∥2 t t ∥xlarge ∥2 t
× 100%
(1)
Figure 2b shows that ρt stays below 1.5% throughout the relay phase with only minor fluctuations, which means the two models show similar latent-norm trajectories during the later denoising steps. These results show that, because models in the same family share the same latent space, the small model can naturally pick up where the large model left off. B. Relay Implementation Diffusion models. Relay inference splits the denoising process at step s. The large edge model ML denoises the initial steps and produces an intermediate latent xs . This latent is then transferred to the device where the small model MS completes the generation. We show this overall process in Figure 3. The exact step update depends on the model architecture. Models sharing a UNet backbone like the SDXL family typically follow the DDIM sampling framework [29], where each denoising step updates the latent as shown in Eq. (2): p √ (2) xt−1 = ᾱt−1 x̂0 (xt ) + 1 − ᾱt−1 ϵθ (xt , t), √ √ with x̂0 (xt ) = (xt − 1−ᾱt ϵθ (xt , t))/ ᾱt . Here, xt represents the noisy latent at step t, ᾱt is the cumulative noise schedule coefficient, ϵθ denotes the noise prediction network, and x̂0 (xt ) is the estimated clean image. Models utilizing the MMDiT architecture like the SD3.5 family follow the Rectified Flow framework [41]. These models perform Euler integration for each step as shown in Eq. (3): xi+1 = xi + (ti+1 − ti ) · vθ (xi , ti , e).
(3)
At the relay point, the inference process switches from the edge model to the device model. The fundamental requirement for a successful handoff is maintaining noise continuity. We achieve this universal alignment through sigma matching, a strategy originally used in ensemble-of-experts diffusion (e) (d) pipelines [20]. Let {σi } and {σj } be the sigma sequences
��
of ML and MS . When ML finishes step s with noise level (e) σs , we calculate the device-side start step s′ by finding the closest noise level as shown in Eq. (4): s′ = arg
min
j∈{0,...,Td −1}
(d)
σj
− σs(e) .
(4)
Schedule compatibility. The actual execution of this sigma matching theory depends on the default configurations and schedule compatibility within each model family. For the SDXL family, we set the total steps to Te =50 for the edge model and Td =25 for the device model. These models use different step counts and non-uniform Karras sigma schedules, which causes the same step index to map to different noise states. We therefore actively rely on the sigma matching equation and exhaustive search to find valid relay pairs. In contrast, both models in the SD3.5 family run the same T =50 steps on an identical linear schedule. This perfect alignment means the sigma matching trivially resolves to s′ = s, allowing the small model to directly resume from step s at the same noise state. To cover different quality and latency tradeoffs, we select five relay points s ∈ {5, 10, 15, 20, 25} for evaluation across both families. Instead of treating every denoising step as a separate relay action, we discretize the handoff point with a stride of five steps. This design reflects a service-level tradeoff: adjacent relay steps usually produce highly correlated latent states and similar quality–latency profiles, while a perstep action space would substantially increase the number of arms and dilute the feedback available to each arm in online bandit learning. This candidate set therefore provides a practical handoff granularity to capture the main quality– latency trend while keeping scheduling statistically stable under finite workloads. The relay mechanism produces a set of configurations with different quality and speed characteristics within each family. In practice, the best choice depends on the request: a semantically complex prompt benefits from more edge-side steps to ensure coherence, while a simpler prompt can tolerate an earlier handoff for lower latency. User preferences and real-time factors such as device load and network conditions also affect which configuration performs best. Selecting the right configuration for each incoming request is therefore an online scheduling problem, which we formulate and solve in Section IV.
TABLE II: Action space of RISE. Arm
Model Pair
Relay Step s
a0 a1 –a5 a6 –a10
Vega (standalone) SDXL + Vega SD3.5 Large + Medium
– 5, 10, 15, 20, 25 5, 10, 15, 20, 25
B. LinUCB Scheduler
IV. A LGORITHM D ESIGN While Section III defines the available relay configurations, selecting the optimal one for each incoming request remains non-trivial. The choice depends on dynamic factors like prompt complexity and system state, which are only known at runtime. Therefore, we formulate the scheduling problem as a contextual multi-armed bandit and propose a LinUCB-based online scheduling algorithm that learns the optimal context-to-configuration mapping through exploration– exploitation tradeoffs. A. Problem Formulation The relay mechanism produces a discrete set of inference configurations. We define an action space A = {a0 , a1 , . . . , a10 } of 11 arms, each mapping to one configuration shown in Table II. Each request is encoded as a d=8 dimensional context vector: c = [ ccplx , ctxt , cnet , cbat , cpref , lvega , lsdxl , lsd3 ].
(5)
The first five dimensions capture task-level features: • ccplx ∈ [0, 1]: prompt complexity, measured as the normalized clause count. • ctxt ∈ {0, 1}: text-rendering indicator, set to 1 when the prompt requests visible text. • cnet ∈ [0, 1]: inverse network quality, derived from the log-transformed round-trip latency. Higher values favor device-side inference. • cbat ∈ {0, 1}: low-battery flag, set to 1 when the device battery drops below 20%. • cpref ∈ [0, 1]: quality–speed preference, where 0 denotes maximum quality and 1 maximum speed. The remaining three dimensions encode system-level state: • lvega , lsdxl , lsd3 ∈ [0, 1]: occupancy ratios of the Vega, SDXL, and SD3.5 device pools, each equaling the fraction of occupied replicas. Relay arms occupy slots from two pools simultaneously, so LinUCB observes both loads when evaluating such arms, implicitly steering away from congested pools. Optimization objective. At round t, a request arrives with context ct . The scheduler picks arm at ∈ A and observes reward rt (defined in Section IV-C). Given N total requests, the objective is to maximize cumulative reward: max
{at }N t=1
N X
rt .
(6)
t=1
This problem has a natural exploration–exploitation tradeoff, which we address with a LinUCB-based approach [37].
For each arm a ∈ A, the algorithm maintains a matrix Aa ∈ Rd×d and a vector ba ∈ Rd , initialized as Aa = Id and ba = 0, together with a pull count na = 0. The selection procedure has three steps. UCB scoring. Given context ct , the score for each candidate arm has three components: s q ⊤ ln(n + 1) −1 , (7) pa = θ̂ a ct + α c⊤ t A a ct + β | {z } 1 + na {z } | | {z } exploitation contextual exploration frequency exploration
where θ̂ a = A−1 a ba is the ridge regression estimate and n = P a na is the total pull count. The first term estimates expected reward from historical data; the second, following the standard LinUCB formulation [37], adds a bonus for contexts poorly covered in past observations; the third, inspired by UCB1 [38], provides additional exploration for less-pulled arms. Softmax sampling. Instead of always selecting the arm with the highest score, we draw from a softmax distribution: Pr(at = a) = P
exp(pa /τ ) , exp(pa′ /τ ) t
(8)
a′ ∈A
where τ is a temperature parameter. Softmax sampling serves two purposes: in early rounds it maintains stochastic exploration when reward estimates are still uncertain, and in later rounds it prevents the scheduler from locking onto one arm and falling into a local optimum. The temperature decays over time: max(0, n − Nw ) τ = max τmin , τ0 · 1 − , (9) K where τ0 is the initial temperature, Nw is the warm-up period, and K is a shared decay constant. This schedule gradually shifts behavior from broad exploration to focused exploitation. Parameter update. After selecting arm at and observing reward rt , the selected arm’s statistics are updated: Aat ← Aat + ct c⊤ t + λId ,
(10)
bat ← bat + rt ct , where λ is a regularization coefficient. Unlike the standard LinUCB update, we add λId at every step so that the shrinkage on θ̂ a grows with the number of pulls, preventing overfitting when the reward distribution shifts. The exploration parameters α and β decay once the total pull count exceeds Nw : max(0, n − Nw ) , (11) α = max αmin , α0 − K max(0, n − Nw ) β = max βmin , β0 · 1 − . K Algorithm 1 details the complete arm selection procedure.
Algorithm 1 RISE Arm Selection Require: Candidate arms At , context ct , {Aa ,P ba , na } 1: n ← a na 2: Update α, β, τ via Eq. (9) and Eq. (11) 3: for each a ∈ At do 4: θ̂ a ← A−1 a ba q q
Algorithm 2 RISE Online Scheduling parameters
⊤
ln(n+1) −1 pa ← θ̂ a ct + α c⊤ t A a ct + β 1+na 6: end for 7: return Sample at from At via softmax
5:
C. Dynamic Reward Function The reward signal guides RISE toward the best-matching configuration for each task. We define a composite reward: X r= wm Qm −wtime ·ttotal −wcost ·mvram −γ ·ldev , (12)
Require: Action space A, minimum pulls Nmin 1: Initialize Aa ← Id , ba ← 0, na ← 0 for all a ∈ A 2: for each request t do 3: Construct context ct via Eq. (5) 4: Filter At ⊆ A by device availability 5: if At = ∅ then 6: Enqueue request; continue 7: end if 8: if ∃ a ∈ At with na < Nmin then 9: at ← arg mina∈At na ▷ Forced exploration 10: else 11: at ← L IN UCBA RM S ELECT(At , ct ) ▷ Alg. 1 12: end if 13: Execute relay inference with at 14: Compute rt via Eqs. (12)–(13) 15: Update Aat , bat , nat using Eq. (10) 16: end for
m∈Q
where Q = {QCLIP , QIR , QPick , QAes , QOCR } represents five quality metrics. The cost consists of three components: the end-to-end latency ttotal including queuing time, the peak GPU memory mvram , and the occupancy ratio ldev of the device pools. For relay arms spanning two distinct pools, ldev is defined as the maximum of the two ratios. All cost terms are collectively weighted by a penalty coefficient γ. Dynamic weights empower metric differences across configurations to govern arm selection: • Text-rendering tasks (ctxt =1): wOCR is raised while visual quality weights drop. • Speed-sensitive tasks (cpref >0.5): wtime is amplified and quality weights are halved. • Quality-focused tasks: wCLIP and wIR are maximized and wtime is reduced. • Low battery (cbat =1): both wcost and wtime are scaled up to favor energy-efficient setups. To keep the reward bounded and stabilize learning, we compress it through a tanh function: rfinal = η · tanh(r/η),
(13)
where η is a scaling constant that keeps rfinal ∈ (−η, η). Throughout this paper, rt denotes the compressed reward. The regret property of standard LinUCB motivates our use of confidence-based exploration in RISE. However, since the practical scheduler further incorporates softmax sampling, frequency-based exploration, and workload-aware reward shaping, we do not claim a formal regret bound for the complete scheduler. Combining all components, the system constructs a context for each arriving request, filters available arms, selects a configuration via LinUCB, executes inference, and updates parameters based on the observed reward. Algorithm 2 presents the complete RISE online scheduling procedure. Notably, ttotal includes queuing wait time, so RISE naturally learns to avoid choosing an arm on a busy pool, which leads to longer
waits and lower rewards, discouraging that choice in similar future contexts. V. E XPERIMENT We design experiments to answer three research questions: • RQ1: Can relay inference achieve significant speedup while preserving generation quality across heterogeneous tasks? • RQ2: Does the RISE scheduler effectively balance quality and latency under mixed service workloads? • RQ3: How does each component of the scheduling algorithm contribute to the overall service performance? A. Experimental Setup Datasets. We evaluate RISE on two datasets covering complementary aspects of text-to-image generation services. DiffusionDB [42] is a large-scale prompt gallery dataset covering diverse visual scenes; DrawTextCreative [43] contains prompts requiring legible text within the image, testing typographic fidelity alongside visual quality. Models and Testbed. We employ two model families for relay inference. For the SDXL family, the edge model ML is SDXL and the device model MS is Segmind-Vega, sharing a UNet backbone with a 4-channel latent space. For the SD3 family, ML is Stable Diffusion 3.5 Large and MS is SD3.5 Medium, sharing the MMDiT architecture with a 16-channel latent space. Our testbed comprises 8 NVIDIA RTX 4090 GPUs organized as 4 device pools (SDXL×2, SD3.5 Large×2, SD3.5 Medium×2, Segmind-Vega×2). Although network and energy factors are simulated in the scheduling context, the testbed does not include deployment on physical heterogeneous edge-device hardware. All models generate images at 1024×1024 resolution. SDXL uses Te =50 steps and SegmindVega uses Td =25 steps; SD3.5 Large and Medium both use T =50 steps.
TABLE III: Performance comparison of generation quality and efficiency across different acceleration methods on two datasets. OCR is omitted for DiffusionDB as its prompts lack text-rendering tasks. And the best results per group are bolded throughout this paper. Dataset
Model
Quality Metrics
Method CLIP↑
ImgRwd↑
PickSc↑
Aesth↑
OCR↑
Speedup↑
Denoise(s)↓
SDXL
Original DeepCache T-GATE SADA RISE (Fast) RISE (Slow)
0.3385 0.3255 0.3385 0.3387 0.3392 0.3397
0.8482 0.7559 0.5728 0.5847 1.0598 1.0901
0.2150 0.2145 0.2124 0.2141 0.2153 0.2150
6.4453 6.4576 6.4702 6.4258 6.4969 6.5104
– – – – – –
1× 1.85× 1.45× 1.87× 2.10× 1.59×
6.87 3.72 4.54 3.67 3.27 4.33
SD3.5 Large
Original DeepCache T-GATE SADA RISE (Fast) RISE (Slow)
0.3328 0.3320 0.3185 0.3278 0.3335 0.3329
1.1289 1.0880 0.6612 0.9645 1.1209 1.1303
0.2188 0.2186 0.1983 0.2064 0.2195 0.2198
6.4164 6.4283 6.2597 6.3122 6.4411 6.4417
– – – – – –
1× 1.30× 1.62× 1.32× 1.77× 1.59×
30.19 23.16 18.60 22.83 17.10 18.95
SDXL
Original DeepCache T-GATE SADA RISE (Fast) RISE (Slow)
0.3300 0.3220 0.3180 0.3220 0.3264 0.3236
0.6521 0.3603 0.0117 0.4422 0.2932 0.2402
0.2179 0.2129 0.2076 0.2033 0.2075 0.2082
5.1366 5.1606 5.1705 5.0281 5.0371 5.0146
0.2551 0.1879 0.1126 0.1196 0.0956 0.1016
1× 1.84× 1.53× 1.87× 2.10× 1.58×
6.89 3.75 4.49 3.67 3.28 4.35
SD3.5 Large
Original DeepCache T-GATE SADA RISE (Fast) RISE (Slow)
0.3583 0.3573 0.3420 0.3421 0.3583 0.3580
1.2855 1.2341 0.6123 1.1964 1.2670 1.3052
0.2298 0.2291 0.2140 0.2196 0.2298 0.2300
5.0888 5.0336 4.6719 4.9824 5.0768 5.0954
0.6638 0.6132 0.5746 0.6254 0.6695 0.6771
1× 1.33× 1.66× 1.34× 1.77× 1.60×
30.92 23.27 18.61 23.05 17.47 19.32
DiffusionDB
DrawTextCreative
Evaluation Metrics. We assess generation quality with five metrics: CLIP Score [44] for text–image semantic alignment, ImageReward [45] for human preference prediction, PickScore [46] for pairwise preference alignment, Aesthetic Score [47] for visual appeal, and OCR accuracy for textrendering fidelity on DrawTextCreative, where we use ABINet [48] to recognize text in generated images and compute the normalized edit distance. Service efficiency is measured by denoising latency and the corresponding speedup ratio relative to each family’s full model. Baselines. We adopt two groups of baselines to evaluate our approach. (1) Inference acceleration baselines: for both the SDXL and SD3.5 families, we compare against the original full models (SDXL and SD3.5 Large) running all 50 steps independently. To further demonstrate the effectiveness of our approach, we also include training-free acceleration methods applied to the large models: •
Service Efficiency
DeepCache [30]: caches high-level features across adjacent denoising steps to reduce redundant computations.
T-GATE [49]: skips cross-attention computations after semantic content converges. • SADA [50]: a stability-guided adaptive diffusion acceleration method that dynamically allocates sparsity. (2) Scheduling baselines: these methods focus on how to dispatch requests and select configurations online: • Round-Robin (RR): cycles through available configurations in a fixed, sequential order. • Greedy: a makespan-minimizing heuristic that dispatches to the least-loaded pool with a fixed mid-range relay step. • PPO [24]: a policy-gradient reinforcement learning method for stable policy updates. • SAC [25]: an off-policy actor-critic method that incorporates entropy regularization. •
B. Relay Inference Performance (RQ1) Table III presents the quantitative comparison across both datasets. Following the action space in Table II, we select two representative relay configurations from each family: one
SDXL 1×
RISE (Fast) 2.10×
RISE (Slow) 1.59×
Prompt
SD3.5 Lar. 1×
SD3.5 Med. 2.67×
RISE (Slow) 1.59×
dynamic dramatic portrait, imogen poots as battle cleric goddess, ...
render of dreamy beautiful landscape, dreamy, artger, large scale,..., high detailed, 8 k
book cover illustration, art by gerald brom, of a flying medieval fantasy dragon...
A hand painted wooden “Pineapple Club” sign in the shape of a pineapple, hanging outside a bar.
A vintage postage stamp showing a painting of the Golden Gate Bridge and the text “California”.
Fig. 4: Visual comparison across relay configurations.
optimized for speed (Fast, s = 15) and one for quality (Slow, s = 20). For example, the ‘Fast’ method for SDXL model denotes SDXL+Segmind-Vega relay with an early handoff step (s = 15), and the ‘Slow’ method for SD3.5 Large model means SD 3.5 Large+SD3.5 Medium relay with a later handoff step (s = 20). Baseline settings include DeepCache (cache interval=2) and T-GATE (gate_step=20). Speedup is relative to the original full model. Moreover, we summarize two key findings. Finding 1: Relay inference achieves significant speedups and superior metric robustness compared to both full models and single-model acceleration baselines. On the DiffusionDB dataset, relay configurations outperform not only their full-model counterparts but also training-free acceleration baselines in human preference and aesthetic metrics. While existing single-model baselines effectively reduce latency, they often do so at a severe cost to semantic alignment and structural integrity. In contrast, by executing the crucial early denoising steps natively on the large edge model and delegating detail refinement to the small device model, RISE maintains semantic coherence. This complementary approach delivers highly competitive speedups without sacrificing generation quality, offering a more favorable quality-latency tradeoff.
Finding 2: Task type determines the optimal model family, highlighting the necessity of context-aware scheduling. On text-rendering tasks, the SD3 family largely preserves OCR accuracy through the relay process. Conversely, the SDXL family inherently lacks strong text-rendering capabilities, and we observe that this architectural limitation is further amplified during relay inference, leading to degraded text readability. Moreover, this task-dependent variance precisely underscores the necessity of our context-aware online scheduler. By recognizing prompt requirements, the scheduler intelligently routes text-heavy requests to the capable SD3 family while reserving the highly efficient SDXL relay configurations for general visual tasks. C. Parameter Sensitivity Analysis (RQ1) Qualitative Verification. We first visually inspect the generated images. Figure 4 demonstrates that for general visuals, SDXL relay variants produce coherent images at dramatically higher speeds, though all SDXL configurations consistently fail at text rendering. Conversely, SD3 relay variants remain visually close to the full SD3.5 Large model. Notably, they avoid the visible text degradation suffered by the standalone SD3.5 Medium baseline, visually confirming that leveraging a