arXiv CS open access

Pruning-Aware Multi-Cluster Co-Inference for Large AI Models in AI-RANs

2026 · arxiv_cs

arXiv CS · Papers · 2026

cloud distributed-computing parallel-computing

Conceptio Open Archive

Analyze & explore related research in Conceptio

Connect this record directly into your research workflow or AI agent. Explore citations across open repositories, query full text, and export structured proof bundles.

Explore in Search →Connect via MCP / API ↗

Pruning-Aware Multi-Cluster Co-Inference for Large AI Models in AI-RANs

arXiv:2608.03026v1 [cs.DC] 4 Aug 2026

Xiaowen Cao, Zhonghao Lyu, Shicheng Chu, Zezhong Zhang, Dingzhu Wen, Guangxu Zhu, Kaibin Huang, Fellow, IEEE, Shuguang Cui, Fellow, IEEE, and Jie Xu, Fellow, IEEE

Abstract—The increasing scale and computational demands of large artiﬁcial intelligence models (LAIMs) present signiﬁcant challenges for efﬁcient inference in resource-constrained distributed environments. In this paper, we propose a multi-cluster LAIM co-inference framework, where an edge server equipped with multiple graphics processing units (GPUs) coordinates multiple user clusters to execute inference tasks collaboratively. Within each cluster, devices capture data from diverse perspectives and employ lightweight on-device LAIMs to extract local features. These features are then transmitted to the edge server, where they are aggregated and fused to generate a more accurate inference outcome. To reveal the fundamental tradeoff between model pruning and collaborative inference performance, we develop a theoretical framework that characterizes the impact of pruning ratios and device contributions using ratedistortion theory and partial information decomposition. Based on this analysis, we formulate a joint optimization problem that determines the model pruning ratio, the task scheduling strategy, the bandwidth allocation, and the transmission power, with the goal of minimizing the inference distortion while satisfying the constraints of latency, energy consumption, and server capacity. Extensive simulation results demonstrate that the proposed framework signiﬁcantly outperforms existing benchmark schemes, achieving superior inference accuracy and resource efﬁciency in multi-cluster edge intelligence networks. Index Terms—large AI models, co-inference, edge networks, model pruning, GPU scheduling, resource allocation

I. I NTRODUCTION As wireless networks evolve toward artiﬁcial intelligence radio access networks (AI-RANs) that integrate edge intelligence into network operations [1]–[3], large artiﬁcial intelligence models (LAIMs), including large language models (LLMs), vision-language models (VLMs), and multimodal foundation models [4], are gradually moving from centralized clouds to network-edge environments. By providing advanced X. Cao and S. Chu are with the College of Electronic and Information Engineering, Shenzhen University, Shenzhen 518100, and with Guangdong Provincial Key Laboratory of Future Networks of Intelligence, Shenzhen 518172, China (e-mail: caoxwen@szu.edu.cn, 2400042041@mails.szu.edu.cn). Z. Lyu is with the Department of Computer Science, The Hang Seng University of Hong Kong, Hong Kong SAR, China (e-mail: lyuz@hsu.edu.hk). Z. Zhang, S. Cui, and J. Xu are with the School of Science and Engineering, the Shenzhen Future Network of Intelligence Institute (FNii-Shenzhen), and the Guangdong Provincial Key Laboratory of Future Networks of Intelligence, The Chinese University of Hong Kong (Shenzhen), Shenzhen, Guangdong 518172, China (e-mail: {zhangzezhong,shuguangcui,xujie}@cuhk.edu.cn). D. Wen is with the School of Information Science and Technology, ShanghaiTech University, Shanghai, China (e-mail: wendzh@shanghaitech.edu.cn). G. Zhu is with Shenzhen Research Institute of Big Data, Shenzhen, Guangdong 518172, China (e-mail: gxzhu@sribd.cn). K. Huang is with the Department of Electrical and Computer Engineering, The University of Hong Kong, Hong Kong SAR, China (e-mail: huangkb@eee.hku.hk). Part of this work was presented in IEEE International Conference on Communications (ICC) 2026 at Glasgow, Scotland, UK.

perception, reasoning, and decision-making capabilities, these models enable edge perception networks to process and understand complex environmental information in real time [5], [6]. As a result, they are becoming an important foundation for emerging applications such as autonomous driving, lowaltitude wireless networks, and industrial Internet of Things (IIoT) systems, where timely and intelligent responses are essential [7]. Despite their strong performance, the massive size and high computational requirements of LAIMs pose signiﬁcant challenges for edge deployment [8], [9]. Most commercial models are therefore hosted in cloud data centers, which provide sufﬁcient computing resources but suffer from privacy concerns, high communication overhead, and increased latency due to frequent data transmission. To address these limitations, on-device deployment of LAIMs has attracted growing interest. Recent lightweight models, such as Gemini Nano and Llama 2, demonstrate that certain LAIM inference tasks can be executed locally [10], [11]. Nevertheless, modern foundation models still require substantial memory, computation, and energy resources that exceed the capabilities of most edge devices [12]. Consequently, fully operating large-scale LAIMs on edge devices remains impractical, motivating distributed inference across edge devices and servers. To this end, device-edge co-inference has emerged as a promising paradigm, where a pre-trained model is partitioned between edge devices and servers for joint inference [13]. By performing lightweight processing locally and ofﬂoading subsequent computation to edge/cloud servers, co-inference alleviates on-device resource constraints while enabling access to LAIM services [14]. However, practical edge networks often consist of multiple user clusters that simultaneously compete for shared wireless spectrum and graphics processing unit (GPU) resources [15]. In such settings, communication, computation, and task scheduling become tightly coupled, and decisions made for one cluster may directly affect the performance of others. Therefore, efﬁcient LAIM deployment requires a uniﬁed framework that jointly optimizes task scheduling, resource allocation, and computation management across clusters while satisfying latency and energy constraints. A. Related Work and Motivation Since co-inference relies on feature exchange between edge devices and servers, how to reduce communication overhead is a key challenge. Semantic communications address this issue by transmitting task-relevant features instead of raw data, where deep neural networks (DNNs) are typically used to

extract compact semantic representations [16], [17]. Building on this idea, recent studies have shifted from channelaware resource allocation to task-oriented designs that account for the importance of transmitted information. For instance, [18] jointly optimized semantic compression and communication resource allocation under task-speciﬁc requirements, while [19] considered the joint design of transmit power and bandwidth allocation for semantic text transmission in relayassisted systems. Another important research direction focuses on the joint optimization of task ofﬂoading and resource allocation. In particular, [20] jointly optimized model placement and model splitting decisions to balance local computation and communication costs, while [21] investigated the joint allocation of sensing, communication, and computation resources for multi-device inference. Despite these advances, most existing works primarily optimize communication and computation efﬁciency. In co-inference systems, however, the ﬁnal inference performance is also inﬂuenced by feature quality and the complementary information contributed by different devices. Consequently, the interaction among model compression, resource allocation, and multi-device information fusion remains insufﬁciently understood. Nevertheless, most existing studies on co-inference and resource optimization mainly focused on small- or mediumscale AI models, which makes them difﬁcult to directly extend to LAIMs due to their billion-level parameters and high computational cost [22]. To address the challenges brought by large model size, recent work has explored joint task ofﬂoading and resource allocation for LAIM inference. For example, [23] studied how different model splitting points affect end-to-end latency and accuracy under varying wireless channel conditions, showing that the placement of partition layers has a strong impact on system performance. Similarly, [24] proposed EdgeShard, which partitions a large model into multiple smaller shards that can be deployed across distributed devices to better utilize edge resources, while the authors in [25] proposed a large- and small-model collaborative inference framework that jointly optimizes task ofﬂoading and model caching. These studies highlight the importance of carefully coordinating model partitioning with network conditions in LAIM deployment. Different from them, model compression is another critical design aspect for LAIM inference. Existing techniques mainly include pruning, quantization, and knowledge distillation, which aim to reduce model size and computational complexity while preserving inference performance. Focusing on this, [12] theoretically characterized the relationship between the model pruning and inference distortion, providing a guideline for designing pruning-aware LAIM co-inference systems, which was further explored into model quantization based framework [26]. The authors in [27] proposed a joint prompt compression and power control framework for mobile LLM services, where an edge-deployed small language model (SLM) compresses user prompts and a deep reinforcement learning-based policy is adopted to optimize the compression ratio and transmission power. However, these designs are developed primarily for single-user co-inference scenarios and therefore do not readily

scale to multi-user edge network environments, motivating the present work. B. Contributions There remain several challenges in enabling efﬁcient LAIM co-inference in edge perception networks. First, multiple user clusters may simultaneously request inference services while sharing limited wireless and edge computing resources, leading to strong resource competition. Second, model pruning can reduce computation and communication costs but may also degrade inference performance, which leaves a fundamental trade-off between resource efﬁciency and inference quality. Third, different devices often contribute unequally to the ﬁnal inference outcome due to variations in sensing quality, observation perspectives, and information overlap. Therefore, efﬁcient LAIM co-inference requires the joint consideration of resource allocation, model compression, and device contribution in a uniﬁed design framework. To address these challenges, we propose a multi-cluster LAIM co-inference framework, where an edge server equipped with multiple GPUs coordinates multiple user clusters to perform collaborative inference. The main contributions of this work are summarized as follows. • Multi-cluster LAIM co-inference framework: We consider a novel co-inference architecture in AI-RANs with multiple device clusters. Unlike conventional single-user or single-cluster designs, the proposed framework explicitly captures the competition for shared wireless spectrum and GPU resources among clusters. By incorporating both energy consumption and inference latency into the system design, we characterize the coupling among communication, computation, and task scheduling, thereby providing a practical framework for scalable and resource-efﬁcient LAIM deployment. • Pruning-performance analysis with heterogeneous device contributions: To reveal the impact of model compression on collaborative inference, we establish a ratedistortion-based analytical framework that characterizes the fundamental trade-off between model pruning and inference performance. We further incorporate partial information decomposition (PID) to quantify the contributions of different devices. Such analysis provides new insights into how device heterogeneity and multi-view information fusion affect pruning-aware co-inference. • Joint optimization for resource-efﬁcient co-inference: Guided by the proposed pruning-performance analysis, we formulate a joint inference distortion minimization problem that optimizes model pruning ratios, task scheduling, bandwidth allocation, and transmission power under latency and energy constraints. The problem is highly non-convex because model compression decisions directly affect both communication and computation processes, leading to strong interactions among optimization variables. To handle this coupling, we develop an efﬁcient algorithm that enables practical deployment in large-scale edge perception networks. • Performance evaluation: To demonstrate the effectiveness of the proposed framework, we consider two rep-

!"#$%&'()

!"#$%"&'

!"#$%"&

!"#$%"&(

!"!"! ./%01 .2*3

./%01 .2*3

)*(+&,%#'&(,--'&-,%./0(1(2#$./0 ./%01 .2*3

!" !" !" !"#$%$"&$'%$()*+

!"#$%&'(!

!"#$%"&'

!"#$%"&

!"#$%"&(

!" !" !"

!"!"! ./%01 .2*3

./%01 .2*3

3*(456($78&9#".0-

./%01 .2*3

!"#$

!"!"!

)*)+&,-"-"

!"# )*)+&,-"-"

!"#$%$"&$'%$()*+ !"!"!

!"#$%&'("

!"#$%"&'

!"#$%"&

!"#$%"&(

!"!"! ./%01 .2*3

!"!"!

./%01 .2*3

:*(;/<0$%'&,=(>?@A ./%01 .2*3

!"#$%$"&$'%$()*+

Figure 1. Illustration of multi-cluster LAIM co-inference system.

resentative edge vision understanding tasks. The results show that the developed pruning-performance analysis efﬁciently characterizes the inference distortion, while the proposed joint optimization signiﬁcantly outperforms benchmark schemes. Moreover, by accounting for heterogeneous device contributions, the proposed design effectively improves resource utilization in multi-cluster edge environments. The remainder of this paper is organized as follows. Section II presents the system model. Section III develops the pruningaware multi-device co-inference quality analysis. Section IV formulates the joint pruning, task scheduling, and communication design. Section V provides the numerical results. Finally, Section VI concludes the paper. II. S YSTEM M ODEL We consider LAIM co-inference over a multi-cluster edge system, where a set of user clusters M , {1, . . . , M } is served by an edge server equipped with G GPUs indexed by G , {1, . . . , G}, as shown in Fig. 1. Let Km denote the number of edge devices in cluster m ∈ M. Cluster m ∈ M consists of a set of edge devices Km , {1, . . . , Km } that collaborate to perform a shared inference task. Within each cluster, devices observe the same target or environment from different viewpoints using heterogeneous sensors, enabling multi-view cooperative sensing. This design helps mitigate common limitations of single-device perception, such as occlusions and incomplete local observations, thereby producing more comprehensive and robust feature representations. Meanwhile, clusters are assumed to be task-isolated, with each cluster dedicated to a distinct inference objective or sensing scenario. This assumption eliminates inter-cluster task coupling, prevents cross-task interference, and enables parallel processing of multiple inference tasks on a shared edge server. Upon receiving a request in cluster m ∈ M, every device k ∈ Km runs a local LAIM that adapts to limited device

computation, to extract embeddings from its raw data and uploads them to the server. After gathering all multi-view embeddings from cluster m, the server performs intra-cluster feature aggregation and fusion to correct single-view sensing deviations and produce a uniﬁed global feature representation, which is then assigned to a single dedicated GPU g ∈ G for ﬁnal inference. The complete inference workﬂow of the proposed multi-cluster co-inference framework is described as follows.

A. On-device Inference and Embedding Upload First, we discuss pruning-aware on-device inference during forward propagation. Denote the on-device model on edge device k in cluster m as Em,k (·; θm,k ) with parameter vector θm,k ∈ Rqm,k with dimension qm,k , ∀k ∈ Km , ∀m ∈ M. To accommodate limited compute/storage, we prune θm,k into a lightweight model θbm,k ∈ Rqbm,k with dimension qbm,k . We further deﬁne pruning ratio as ρm,k , qbm,k /qm,k ∈ [0, 1], ∀k ∈ Km , ∀m ∈ M.

(1)

Then the extracted feature of raw input xm,k is om,k = Em,k (xm,k ; θbm,k ), ∀k ∈ Km , ∀m ∈ M, where Em,k (·) denotes the extraction function of the local model at edge device k ∈ Km , ∀m ∈ M. Next, we discuss the feature uploading process. For ease of illustration, we adopt the frequency division multiple access (FDMA) scheme for wireless data transmission. In this setting, wireless channel conditions stay stable during each data upload process. Each edge device can accurately obtain its own channel state information (CSI), so it can pre-compensate signal phases during signal transmission. For each device k ∈ Km in cluster m ∈ M, its uplink transmission of feature data om,k and downlink reception of inference results use an independent and non-overlapping frequency band, with a

ﬁxed bandwidth denoted as bm,k . We thus have the bandwidth allocation constraint as X X bm,k ≤ B. (2) m∈M k∈Km

Then the achievable rate of uplink transmission is gm,k pm,k Rm,k (pm,k , bm,k ) = bm,k log2 1 + , bm,k N0

(3)

where gm,k is the uplink channel power gain, N0 is the power spectral density (PSD) of the additive white Gaussian noise (AWGN), and pm,k denotes the transmit power with maximum max power being Pm,k given by max 0 ≤ pm,k ≤ Pm,k , ∀k ∈ Km , m ∈ M.

(4)

C. System Delay The overall system delay consists of three components: ondevice inference delay, feature uploading delay, and on-server inference delay. • On-device inference delay: We ﬁrst analyze the arrival time of task m, which depends on the on-device inference and transmission delay of k ∈ Km . Denote the original FLOP workload of on-device inference (in FLOPs) as Nm,k . Assume that the remaining workload is proportional to the pruning ratio ρm,k , and then the remaining workload FLOP after model pruning is ρm,k Nm,k . Let cm,k and fm,k denote the number of FLOPs per cycle of the processor and the clock frequency at device k ∈ Km , respectively. Then, the delay for on-device inference is1 tIm,k (ρm,k ) =

B. On-server Aggregation and GPU Scheduling Then, we introduce on-server feature aggregation and GPU scheduling. After collecting embeddings {om,k }k∈Km from all edge devices in cluster m ∈ M, the server aggregates them as oem = fem (om,1 , . . . , om,Km ), where f˜m (·) denotes the aggregation function. The aggregated feature oem is then assigned to one of the available GPUs for on-server inference. Each GPU g ∈ G maintains a ﬁrst-in-ﬁrst-out (FIFO) processing queue with capacity Qg , {1, . . . , Qg } to manage incoming inference tasks. To model task scheduling, we deﬁne a binary variable xm,g,q ∈ {0, 1}, where xm,g,q = 1 indicates that the task of cluster m is assigned to the q-th position in the queue of GPU g. Each inference task is scheduled to exactly one position in one GPU queue, and each queue position can accommodate at most one task. This ensures exclusive service allocation and enables a structured representation of GPU workload management for multi-cluster co-inference. Therefore, we have X X xm,g,q = 1, ∀m ∈ M, (5) g∈G q∈Qg

m∈M

xm,g,q ≤ 1, ∀q ∈ Qg , g ∈ G.

(6)

Finally, we discuss the on-server inference process. Once the task of cluster m is scheduled to a speciﬁc GPU, the server executes the remaining inference procedure using the aggregated feature representation oem . Speciﬁcally, the GPU processes oem through a downstream LAIM, and the ﬁnal inference output is obtained as ym = fb(e om ; ξm ),

(7)

where fb(·; ξm ) denotes the downstream LAIM with parameter ξm . The resulting prediction ym is then broadcast from the server to all devices within Km . This feedback enables each device to obtain a consistent inference result for the shared task, which is particularly important in multi-view cooperative sensing scenarios where devices collaboratively observe the same target or environment. Next, we analyze the system delay and energy consumption under the proposed multi-cluster LAIM co-inference framework.

•

FLOP ρm,k Nm,k

fm,k cm,k

(8)

Feature uploading delay: We assume that all features have the same communication overhead of β bits. Based on the uplink communication rate in (3), the delay for embedding om,k uploading from device k ∈ Km to the server is β . (9) tU m,k (pm,k , bm,k ) = Rm,k (pm,k , bm,k ) Then the task arrival time of device k is I U tA m,k (ρm,k , pm,k , bm,k ) = tm,k (ρm,k ) + tm,k (pm,k , bm,k ). (10)

Accordingly, the ready time of task m is A tR m ({ρm,k , pm,k , bm,k }) , max tm,k . k∈Km

•

(11)

On-server inference delay: Then, we consider the onserver inference delay, which includes the queueing and on-server computation delay. Speciﬁcally, denote the arrival time at position q ∈ Qg on GPU g ∈ G as Ag,q ({xm,g,q , ρm,k , pm,k , bm,k }) = M X

m=1

xm,g,q tR m ({ρm,k , pm,k , bm,k }). (12)

To enforce FIFO processing, we have Ag,q ({xm,g,q , ρm,k , pm,k , bm,k }) ≤

Ag,q+1 ({xm,g,q+1 , ρm,k , pm,k , bm,k }), ∀q ∈ Qg , g ∈ G. (13)

Next, let Tg,q denote the completion time at position q ∈ Qg on GPU g ∈ G. Speciﬁcally, denote the onserver workload of task m, the number of FLOPs per FLOP cycle, and clock frequency at GPU g as N̂ m , cg and 1 Note that the proportional latency and energy model is an analytical approximation rather than a universal hardware law. It captures the basic relationship between the pruning ratio and computational overhead, while providing a tractable abstraction for subsequent system-level optimization. It is particularly reasonable when pruning physically removes computational units and the resulting model structure can be effectively exploited by the underlying execution engine.

fg , respectively. Let t̂m denote the on-server computation delay for task m, which is given by FLOP

t̂m = N̂ m

/fg cg , ∀m ∈ M.

(14)

Accordingly, the total server energy consumption is given by ebI ({xm,g,q , Tg,q }) = b

Under the FIFO scheduling discipline, the task at queue position q can start execution only after both the completion of the task at position (q − 1) and the arrival of the current task. Therefore, we have Tg,q = max{Tg,q−1 , Ag,q ({xm,g,q , ρm,k , pm,k , bm,k })} X I + xm,g,q t̂m , ∀q ∈ Qg , g ∈ G. (15)

P max{Tg,q } + g,q

Qg M X G X X

ηg xm,g,q φg fg2

m=1 g=1 q=1

FLOP bm N , cg

(18)

where ηg and φg are the PUE and power coefﬁcient of GPU g, respectively. Finally, the total energy consumption during the co-inference process is

m∈M

D. Energy Consumption Similar to the delay analysis, the overall energy consumption can be decomposed into three components: on-device inference energy consumption, feature uploading energy consumption, and on-server inference energy consumption. •

On-device inference energy consumption: We ﬁrst consider the energy consumed during on-device inference. Speciﬁcally, each edge device processes its local sensing data using the deployed lightweight LAIM with pruning ratio ρm,k to extract intermediate feature embeddings. Let eI ({ρm,k }) denote the on-device inference energy consumed by device k ∈ Km , ∀m ∈ M. The corresponding energy consumption depends on the required computational workload and the device computing capability, and can be expressed as follows eI ({ρm,k }) =

M X X

2 ηm,k φm,k fm,k

m=1 k∈Km

FLOP ρm,k Nm,k , cm,k

(16)

where ηm,k and φm,k represent the power usage effectiveness (PUE) and chip-speciﬁc power coefﬁcient at device k ∈ Km , ∀m ∈ M, respectively. • Feature uploading energy consumption: Next, we consider the energy consumed during feature uploading. After completing on-device inference, each device transmits its extracted feature embeddings to the edge server through wireless links. The corresponding transmission energy denoted by eU ({pm,k , bm,k }) depends on the transmit power pm,k and allocated communication resources bm,k . Therefore, the total feature uploading energy consumption of all devices can be expressed as follows. U

e ({pm,k , bm,k }) =

•

X X

m∈M k∈Km

pm,k β . Rm,k (pm,k , bm,k ) (17)

On-server inference energy consumption: Then, the overall energy consumption of the server consists of a base power term and a computation power term. Let P b denote the base power of the server, which accounts for auxiliary components such as lighting and displays.

Ê({xm,g,q , ρm,k , pm,k , bm,k , Tg,q }) =

eI ({ρm,k }) + eU ({pm,k , bm,k }) + b eI ({xm,g,q , Tg,q }). (19) III. P RUNING - AWARE M ULTI - DEVICE C O - INFERENCE Q UALITY A NALYSIS

We next characterize the impact of model pruning on the inference performance of multi-device LAIM co-inference systems. In multi-view perception scenarios, devices often contribute different amounts of useful information due to variations in sensing viewpoints and observation quality. Moreover, the information provided by a device may be unique, redundant with other devices, or only valuable when combined with observations from other views. As a result, the effect of pruning on the overall inference performance depends not only on the compressed model itself but also on the contribution of the corresponding device. To capture this interplay, we ﬁrst present a rate-distortion relationship for the single-device case and then extend the analysis to the multi-device setting by incorporating heterogeneous device contributions through a PID framework. This provides a theoretical basis for contributionaware pruning and resource allocation in co-inference systems. 1) Rate-distortion Analysis in Single-device Case: Directly analyzing the output distortion of an LAIM is challenging due to its strong nonlinearity. To overcome this difﬁculty, we ﬁrst upper-bound the output distortion by the parameter distortion, as stated in the following lemma. c as the original and pruned Lemma 1. Denote W and W parameter vectors of an LAIM f (·), respectively. Under the smoothness of the activation function in f (·), for input x, we have c) ≤ C W − W c , f (x, W ) − f (x, W F

(20)

where C is a constant relative to LAIM parameters [12]. Remark 1. Lemma 1 establishes a direct relationship between parameter distortion and output distortion. Speciﬁcally, the output deviation caused by model pruning is upper bounded by the difference between the original and pruned parameter vectors. This result is important because directly analyzing the output distortion of LAIMs is difﬁcult due to their deep and nonlinear structures. In contrast, parameter distortion can be readily quantiﬁed from the pruning process. To this end, parameter distortion provides a practical approximation for the impact of pruning on inference performance.

:HLJKWV *DXVVLDQ /DSODFLDQ

3UREDELOLW\GHQVLW\

:HLJKWV *DXVVLDQ /DSODFLDQ

í í í í

:HLJKWYDOXH

(a) ResNet-152

:HLJKWYDOXH

(b) GPT-3

P global importance-weighted constraint k∈Km αm,k Dm,k ≤ D, ∀m with αm,k > 0, the rate-distortion function for each device is lower bounded by tot Qm αm,k (m,k) √ +qm,k log Ṙm,k (Dm,k ) ≥h W πeD Γ(qm,k /2) + log , (22) 2Γ(qm,k )

where Γ(·) is the gamma function. Thus, the importanceweighted distortion-rate function is Next, to analyze the relationship between the model pruning 1 (m,k) ) Γ(qm,k /2) qm,k qm,k − ρm,k qm,kqγ−h(W ratio and inference performance, we derive the rate-distortion Dm,k (Rm,k ) ≥ αm,k m,k √ 2 2 Γ(qm,k ) πe function for LAIM co-inference. Deﬁne Ṙ(D) as the rateb distortion function, which equals the inﬁmum of the rate , Dm,k (ρm,k ), (23) R, such that rate-distortion pair (R, D) is achievable [28]. Bridging this gap via Lemma 1, we instead analyze the where γ denotes the number of quantization bits per parameter. rate-distortion function of its upper bound, i.e., the model Proof. We ﬁrst consider the optimization problem parameter distortion, as stated in the following lemma. X min Ṙm,k (Dm,k ) q {Dm,k ≥0} Lemma 2. Assume that the LAIM parameters W ∈ R folk∈Km X low a multivariate Laplacian distribution. Under the distortion αm,k Dm,k ≤ D, (24) s.t. c kF ≤ D, from [12], a lower bound of the function kW − W k∈Km rate-distortion function is which is convex. Its Lagrangian is Γ(q/2) q , (21) + log Ṙ(D) ≥ h(W ) + q log √ X q 2 Γ(q) πeD h(W (m,k) ) + qm,k log √m,k L({Dm,k }, λ) = πe k∈Km where h(W ) denotes the differential entropy of W . X Γ(qm,k /2) − q log D + log m,k m,k 2 Γ(q ) Remark 2. Lemma 2 characterizes a lower bound on the m,k Figure 2. Laplacian distributed weights of ResNet-152 and GPT-3.

rate-distortion function for the single-device case. The result establishes a fundamental relationship between the coding rate and parameter distortion, thereby providing a tractable framework for analyzing the impact of model pruning on inference performance. From (21), the required rate decreases as the distortion threshold D increases, indicating the inherent tradeoff between model compression and performance preservation. In addition, the bound depends on the differential entropy h(W ), which implies that models with more information-rich parameters are generally more sensitive to compression.

However, extending this result to multi-device co-inference is challenging. Unlike the single-device case, the overall inference performance depends not only on the distortion of individual models but also on the interactions among devices. Since different devices may provide different amounts of unique and redundant information, their contributions to the ﬁnal inference result are inherently heterogeneous and must be explicitly considered in the analysis. 2) Rate-distortion Analysis in Multi-device Case: To capture device heterogeneity, we introduce an importance coefﬁcient αm,k for device k ∈ Km in cluster m ∈ M. Incorporating αm,k into the distortion measure enables accounting for the unequal contributions of different devices when enforcing a task-level distortion constraint (i.e., a global distortion budget D). Speciﬁcally, we establish the following result. Proposition 1. Let W (m,k) ∈ Rqm,k denote Pthe parameter vector of device k in cluster m and Qtot qm,k . Under m = k∈Km

c (m,k) kF and a the distortion function Dm,k , kW (m,k) − W

k∈Km

+λ

k∈Km

αm,k Dm,k − D .

(25)

By setting the derivative of (25) with respect to (w.r.t.) Dm,k q ⋆ = − Dm,k + λαm,k = 0, we have Dm,k = as zero, i.e., ∂D∂L m,k m,k P ⋆ α D = D qm,k /λ αm,k . Substituting Dm,k into k m,k m,k yields λ⋆ =

qm,k Qtot D ⋆ m . D . Therefore, we have Dm,k = αm,k · Qtot m

Remark 3. Proposition 1 extends the single-device ratedistortion analysis to multi-device collaborative inference by explicitly accounting for the heterogeneous contributions of different devices with unequal pruning distortions. Several observations can be drawn from (22) and (23). First, the distortion-rate relationship preserves the fundamental tradeoff. A higher coding rate, corresponding to a lower pruning level, leads to a smaller distortion, while aggressive pruning increases the distortion. This indicates that model pruning remains a key factor affecting inference quality in coinference systems. Second, the device importance coefﬁcient αm,k directly affects the achievable distortion. For a given coding rate, a device with a larger importance coefﬁcient incurs a larger weighted distortion penalty. Therefore, important devices require more preserved model parameters and should be assigned a lower pruning level to maintain the overall inference quality. In contrast, devices with smaller importance coefﬁcients can tolerate more aggressive pruning with a limited impact on the task-level distortion. Finally, it also reveals a new coupling between model compression and multi-device information fusion. Since the global distortion

constraint depends on the weighted sum of device distortions, the pruning ratio of each device should be jointly designed according to its contribution to the ﬁnal inference task. This result suggests that uniform pruning across devices is generally suboptimal. Instead, importance-aware pruning can better balance inference accuracy and communication overhead in co-inference systems. 3) Importance-aware Policy: Finally, we interpret the importance coefﬁcient αm,k from an information-theoretic perspective. To measure the relative contribution of each device, we use the PID framework, which separates each edge device’s contribution into unique, redundant, and synergistic parts. Deﬁnition 1. For any two edge devices with features o1 and o2 , and a target task Y , PID decomposes the mutual information I(Y ; o1 , o2 ) into nonnegative components as I(Y ; o1 , o2 ) = R + U1 + U2 + S, I(Y ; o1 ) = R + U1 ,

I(Y ; o2 ) = R + U2 ,

(26)

where R denotes the redundant information shared by both devices, Uk represents the unique information contributed by device k, and S corresponds to the synergistic information jointly provided by both devices. While the PID framework provides an intuitive informationtheoretic formalism to disentangle redundant, unique, and synergistic task-relevant information across edge devices, exact full PID computation suffers from prohibitive combinatorial complexity with more than two devices (i.e., |Km | > 2). To retain PID’s core interpretive logic of quantifying perdevice data contribution while enabling tractable computation for multi-device clusters, we leverage Shapley value (SV)based information allocation. The Shapley value naturally aligns with PID’s fundamental goal of attributing predictive value to individual sensing nodes, and recovers the exact marginal mutual information deﬁned by PID in the twodevice special case, serving as a scalable proxy for multinode information decomposition2. Rooted in cooperative game theory, the SV allocates the total utility by quantifying the marginal contribution of each participant across all possible coalitions. As a result, it offers a principled way to evaluate the relative importance of different participants in a collaborative system [29], [30]. Speciﬁcally, let a subset of devices K̂m ⊆ Km be treated as a coalition of players with a coalition value δ(K̂m ). Given an evaluation dataset with N samples {({ok̂,i }k̂∈Kˆm , yi )}N i=1 , the coalition value is deﬁned as δ(K̂m ) = −

1 XN log pF yi | {ok̂,i }k̂∈K̂m , i=1 N

(27)

where pF (·) denotes the predictive distribution. Accordingly, the importance coefﬁcient of device k ∈ Km in cluster m ∈ 2 The proposed contribution coefﬁcients are estimated ofﬂine or updated only periodically, using representative training or historical sensing data. Since device viewpoints and sensing conﬁgurations typically vary much more slowly than the inference process, the estimated coefﬁcients can be reused across multiple inference tasks. Therefore, their computation is not part of the coinference pipeline and does not contribute to the latency considered in the proposed delay model.

M is obtained via the SV as αm,k= i X |K̂m |!(|Km | − |K̂m |−1)! h δ(K̂m ∪ {k})−δ(K̂m) , |Km |! K̂m ⊆Km \{k}

(28)

which can be estimated through Monte Carlo sampling over several random device permutations. IV. J OINT P RUNING R ATIO , TASK S CHEDULING , AND C OMMUNICATION D ESIGN We aim to minimize the inference distortion by jointly optimizing the on-device pruning ratios {ρm,k }, transmit powers {pm,k }, bandwidth allocation {bm,k }, server scheduling decisions {xm,g,q }, and task completion times {Tg,q }, subject to latency and energy constraints. Denoting Φ = {ρm,k , pm,k , bm,k , xm,g,q , Tg,q }, the corresponding optimization problem is formulated as X X b m,k (ρm,k ) (P1) : min D Φ

m∈M k∈Km

s.t. max {Tg,q } ≤ T0

(29a)

g,q

Ê(Φ) ≤ E0

X Tg,q ≥ Tg,q−1+

(29b) m∈M

I xm,g,q t̂m , ∀g, q > 1

(29c)

Tg,q ≥ Ag,q ({xm,g,q , ρm,k , pm,k , bm,k }) X I + xm,g,q t̂m , ∀q ∈ Qg , g ∈ G (29d) m∈M

0 ≤ ρm,k ≤ 1, ∀k ∈ Km , m ∈ M xm,g,q ∈ {0, 1}, ∀m ∈ M, q ∈ Qg , g ∈ G

(29e) (29f)

(2), (4), (5), (6), and (13),

where (29a) and (29b) represent the total delay and energy budget constraints, respectively; (29c) and (29d) correspond to the queueing relationship deﬁned in (15); (5), (6), and (29f) specify the feasible domain of the scheduling variables {xm,g,q }. The problem is highly non-convex due to the strong coupling among the optimization variables. Furthermore, the inclusion of {xm,g,q } renders problem (P1) a mixed-integer nonlinear programming (MINLP) problem, which is generally non-convex and non-trivial to solve optimally. Therefore, we adopt an alternating algorithm to efﬁciently solve the above non-convex problem. In particular, we ﬁrst optimize the task ofﬂoading decisions and completion time {xm,g,q , Tg,q } under given {ρm,k , pm,k , bm,k } by introducing an auxiliary variable to obtain a solvable mixed-integer linear programming (MILP) problem. With the obtained task ofﬂoading decisions {xm,g,q }, we then optimize pruning ratios, transmit power, bandwidth allocation, and completion times {ρm,k , pm,k , bm,k , Tg,q }, in which the successive convex approximation (SCA) technique is used. A. Optimization of task ofﬂoading decisions and completion time To tackle problem (P1), we ﬁrst optimize the task ofﬂoading decisions and completion times {xm,g,q , Tg,q } under

ﬁxed pruning ratios, transmit power, and bandwidth allocation {ρm,k , pm,k , bm,k }, where the resulting problem becomes a feasibility-checking problem. Thus, we focus on minimizing delay plus server energy to create slack feasibility regions for the subsequent optimization, which leads to (P2) :

min

Ê({xm,g,q , Tg,q }) + maxg,q {Tg,q }

s.t.

(2) − (6), (13), (29c), (29d), and (29f).

{xm,g,q ,Tg,q }

Problem (P2) remains non-convex. To address this issue, we introduce an auxiliary variable Υ to relax the term maxg,q {Tg,q } by imposing the additional constraints Υ ≥ Tg,q , ∀g, q, where Problem (P2) is reformulated as min

{xm,g,q ,Tg,q ,Υ}

s.t.

Ê({xm,g,q , Tg,q }) + Υ Υ ≥ Tg,q , ∀q ∈ Qg , g ∈ G

(2) − (6), (13), (29c), (29d), and (29f).

B. Optimization of Network Resource Allocation Then, we optimize pruning ratios, transmit power, bandwidth allocation, and completion times Φ̃ = {ρm,k , pm,k , bm,k , Tg,q }, under ﬁxed task ofﬂoading decision {xm,g,q }, i.e., X X b m,k (ρm,k ) (P3) : min D m∈M k∈Km

s.t. Ê(Φ̃) ≤ E0 (30) (2), (4), (13), (29a), and (29c) ∼ (29e).

Problem (P3) is still non-convex due to constraints (30) and (13). Speciﬁcally, due to the property of perspective function, the term Rm,k (pm,k , bm,k ) in (3) is concave w.r.t. both pm,k and bm,k [32]. Thus, the non-convexity arises from the term eU ({pm,k , bm,k }) within Ê(Φ̃) in (30), as well as from the convex term Ag,q+1 ({xm,g,q , ρm,k , pm,k , bm,k }) on the righthand side of (13). To tackle this issue, we use SCA technique to approximate a non-convex term as a convex one. Speciﬁcally, for the j-th iteration, we approximate eU (pm,k , bm,k ) (j) (j) at local points pm,k and bm,k using its ﬁrst-order Taylor expansion as (j)

p β pm,k β ≈ + m,k (j) g pm,k gm,k pm,k (j) bm,k log2 1 + bm,k b log 1 + (j) m,k N0 m,k

bm,k N0

(j) (j) (j) (j) v(pm,k )(pm,k −pm,k ) +u(bm,k )(bm,k −bm,k ) , ζ̄ (j) (pm,k , bm,k ),

where (j) v pm,k =

β − (j) gm,k pm,k (j) bm,k log2 1 + (j)

(31)

bm,k N0

(j) pm,k βgm,k , 2 (j) gm,k pm,k (j) (j) (j) bm,k log2 1 + (j) bm,k N0 + gm,k pm,k ln 2 b N0 m,k

(j)

pm,k β × 2 (j) gm,k pm,k (j) bm,k log2 1 + (j)

(j) u(bm,k ) =

   

bm,k N0



(j) pm,k gm,k

 − 1 (j) . gm,k pm,k (j) (j) bm,k N0 + gm,k pm,k ln 2 log2 1 + (j) bm,k N0

Next, we approximate tR m ({ρm,k , pm,k , bm,k }) in Ag,q+1 (·), which is convex but non-differentiable as in (11). To tackle it, we apply the SCA based on Danskin’s theorem and utilize its subgradient form. For the j-th iteration with the local point (j) (j) (j) ⋆ {ρm,k , pm,k , bm,k }, let km denote the active device satisfying n o (j) (j) (j) ⋆ ρ , p , b . (32) km ∈ arg max tA m,k m,k m,k m,k k∈Km

The above problem yields an MILP problem, which can be efﬁciently solved using standard optimization solvers such as Gurobi [31].

Φ̃

and

Then, tR m ({ρm,k , pm,k , bm,k }) is linearly approximated as

(j) (j) (j) R {ρ , p , b } tR ({ρ , p , b }) ≥ t m,k m,k m,k m m m,k m,k m,k (j) (j) (j) (j) ⋆ −p ⋆ +u b − b +v pm,k⋆ pm,km b ⋆ ⋆ ⋆ m,km m,km m,km m,km m (j) (j) R(j) ⋆ −ρ , tm ({ρm,k , pm,k , bm,k }), +v̂ ρm,km ρm,km ⋆ ⋆ m,km (33) (j) (j) (j) denote the where v̂ ρm,k⋆ , v pm,k⋆ , and u bm,k⋆ m m m partial derivatives of tR ({ρ , p , b }), respectively. m,k m,k m,k m Speciﬁcally, we have (j) v pm,k⋆ = m

⋆ −βgm,km 1 ·h i , (j) 2 (j) (j) (j) gm,km ⋆ p ⋆ m,km bm,km bm,km ⋆ N0 +gm,k⋆ pm,k⋆ ln 2 ⋆ m log 1+ m

(j)

bm,k⋆ N0 m

(j)

u bm,k⋆



(34)

β = × (j) 2 gm,k⋆ pm,k⋆ (j) m log2 1 + (j)m bm,k⋆ bm,k⋆ N0

(j) ⋆ pm,k⋆ gm,km m

  



 − 1 , gm,k⋆ (j) (j) ⋆ p log2 1 + (j) m ln 2 bm,k⋆ N0 + gm,km ⋆ m,km m bm,k⋆ N0 m (j) FLOP (35) v̂ ρm,k⋆ = Nm,k ⋆ /fm,k⋆ cm,k⋆ . m m m m

Furthermore, we introduce a trust region constraint (j) to tighten the lower bound of tR m ({ρm,k , pm,k , bm,k }) in (33) due to the non-convex constraint in (13). Denote p = [p1,1 , · · · , p1,K1 , p2,1 , · · · , pM,KM ] and b = [b1,1 , · · · , b1,K1 , b2,1 , · · · , bM,KM ] as the vectors consisting of all optimization variables {pm,k } and {bm,k }, respectively. Therefore, the trust region constraint is expressed as iT h ≤ Γtru , (36) [p, b]T − p(j) , b(j) h

(j) (j) (j) (j) with p = p1,1 , · · · , p1,K1 , p2,1 , · · · , pM,KM , b(j) = h i (j) (j) (j) (j) b1,1 , · · · , b1,K1 , b2,1 , · · · , bM,KM and Γtru denoting the trust (j)

Table I SIMULATION PARAMETERS Parameter M G fm,k fg max Pm,k ηg

Description Number of clusters Number of GPUs On-device clock frequency GPU clock frequency Maximum transmit power PUE

Value 8 2 0.3 GHz 1.0,0.85 GHz 1W 2

Parameter Km Qg cm,k cg B φg

region radius and T representing the transpose operation. Accordingly, the convex version of problem (P3) at the j-th iteration is formulated as X X b m,k (ρm,k ) (P4.j): min D Φ̃

m∈M k∈Km

s.t. eI ({ρm,k })+ I

X X

ζ̄ (j) (pm,k , bm,k )

m∈M k∈Km

+ ê ({Tg,q })≤ E0 Ag,q ({xm,g,q , ρm,k , pm,k , bm,k }) ≤

M X (j) xm,g,q+1 tR m ({ρm,k , pm,k , bm,k }), ∀g, q

m=1

(2), (4), (29a), (29c) ∼ (29e), and, (36).

The above problem is convex, which can be directly solved via CVX [33]. V. N UMERICAL R ESULTS In this section, we evaluate the proposed pruning-aware multi-cluster LAIM co-inference framework through two representative tasks: image classiﬁcation and video-to-text captioning. All experiments are implemented on NVIDIA RTX 3090 GPUs. Typical parameters are summarized in Table I. The PSD is N0 = 2 × 10−19 W/Hz. Wireless channels are modeled as independent and identically distributed (i.i.d.) Rayleigh fading with an average path loss of 10−5 . Each cluster contains three partial views with horizontal ranges [0, 0.2], [0.2, 0.6], and [0.5, 1.0], respectively. For the image-classiﬁcation task, we adopt a vision transformer (ViT)-Tiny model based on the ViT architecture and split it at the third transformer layer for on-device and onserver deployment3. Each cluster processes an independent CIFAR-10 inference request, where each image is partitioned into three local views following the method in [34], with cropping ratios of 0.2, 0.4, and 0.6, respectively, and partially overlapping between the last two views. The plotted energy and inference latency thresholds are E0 ∈ [1.15, 1.35] J and T0 ∈ [1.32, 1.40] s, respectively. 3 It is noted that the experimental models are relatively lightweight compared with state-of-the-art foundation models. This choice enables controlled evaluation of the proposed analytical framework and optimization algorithm, while the developed framework remains applicable to larger LAIMs. In practical deployment, additional bottlenecks such as memory capacity, slow data retrieval, and KV-cache management also affect inference performance [8]. These factors are complementary to the communication and computation optimization considered in this work. Extending the proposed framework to explicitly incorporate memory constraints and memory access overhead is an important direction for future work.

Description Number of devices in each cluster Queue length Number of FLOPs per cycle Number of FLOPs per cycle of GPU Communication bandwidth Power coefﬁcient

Value 3 8 32 128,140 5 MHz {1, 1.2} × 10−28 W/(cycle/s)3

For the video-to-text task, we use MSR-VTT [35] and sample 8 frames with resolution 224 × 224 from each video. Each device uses a TinyViT encoder [36], and the server uses Qwen3-4B-Base [37] with a learned projector and a lowrank adaptation (LoRA) adapter to generate captions from the fused view features. The plotted energy and inference latency thresholds are E0 ∈ [3.72, 3.92] J and T0 ∈ [10.2, 11.8] s, respectively. For captioning evaluation, we report bilingual evaluation understudy-4 (BLEU-4) [38] and consensus-based image description evaluation (CIDEr) [39]. BLEU-4 measures the modiﬁed 4-gram precision between a generated caption and its reference captions, with a brevity penalty to discourage overly short outputs. In our implementation, BLEU-4 is computed by averaging smoothed sentence-level BLEU scores over the evaluated samples. CIDEr evaluates caption quality from a reference-consensus perspective by comparing TF-IDFweighted n-gram representations of generated and reference captions, thereby assigning lower weights to common words and higher weights to more informative terms. The reported CIDEr follows the project evaluation script as a TF-IDFweighted n-gram consensus score over the available references. To further expose the role of GPU scheduling, we also conduct a server-workload-heterogeneous latency stress experiment for the task-level timeline visualization. This stress experiment uses the same multi-cluster system but increases the communication bandwidth to 120 MHz for CIFAR-10 and 160 MHz for video-to-text captioning, so that the timeline is mainly shaped by GPU queueing and on-server computation rather than by feature-uploading delays. The corresponding energy and inference latency thresholds are (E0 , T0 ) = (1.2 J, 3.0 s) for CIFAR-10 and (E0 , T0 ) = (8.5 J, 22.0 s) for video-to-text captioning. The on-server workloads are set to {8, 120, 8, 8, 8, 140, 130, 150} GFLOPs for CIFAR-10 and {220, 900, 260, 240, 280, 940, 920, 960} GFLOPs for video-totext captioning. This stress setting is used only for the latencydecomposition ﬁgures and is separated from the qualitythreshold sweeps in Figs. 3–5. We consider the following benchmark schemes for comparison. 1) Fixed power: All devices transmit at the maximum power; • 2) Fixed bandwidth: The total bandwidth is equally allocated among all devices; • 3) Round-robin scheduling: Each cluster is sequentially assigned to GPUs in cyclic order without considering queue states; • 4) Random scheduling: Tasks are randomly assigned to •

0.7

Accurac(

0.6

0.80

Proposed design

Fixed ower Fixed bandwidth Round-robin Random scheduling Equal im ortance

0.5

Fixed p wer Fixed bandwidth R und-r bin Rand m scheduling Equal imp rtance

0.70 0.65 0.60

0.4

0.55

0.3

0.50

0.2

Proposed design

0.75

Accurac)

0.8

0.45

1.32

1.150 1.175 1.200 1.225 1.250 1.275 1.300 1.325 1.350 Energy threshold (J)

(a) Energy threshold

1.33

1.34

1.35 1.36 1.37 Delay thresh ld (s)

1.38

1.39

1.40

(b) Inference latency threshold

Figure 3. Classiﬁcation accuracy on CIFAR-10 under different energy and inference latency thresholds.

BLEU-4

0.20

0.275 0.250 0.225 BLEU-4

0.25

0.300

Proposed desig Fi)ed po(er Fi)ed ba d(idth Rou d-robi Ra dom scheduli g Equal importa ce

0.15

0.200 0.175

0.10

Proposed desig Fi)ed po(er Fi)ed ba d(idth Rou d-robi Ra dom scheduli g Equal importa ce

0.150 0.125

0.05 3.725 3.750 3.775 3.800 3.825 3.850 3.875 3.900 3.925 Energy threshold (J)

(a) Energy threshold

10.2

10.4

10.6

10.8 11.0 11.2 Delay threshold (s)

11.4

11.6

11.8

(b) Inference latency threshold

Figure 4. BLEU-4 scores for the video-to-text captioning task under different energy and inference latency thresholds.

•

GPUs and queue positions; 5) Equal importance: Device importance coefﬁcients are set uniformly as αm,k = 1/Km, ∀m, k.

Fig. 3 shows the classiﬁcation accuracy under different energy and inference latency thresholds. The proposed design maintains the best overall accuracy trend across the plotted operating points. As the energy or inference latency threshold is relaxed, all schemes generally beneﬁt from less aggressive pruning and more feasible communication/computation allocation, but the proposed design makes more effective use of this additional resource slack. This is because the optimization simultaneously adjusts the pruning ratios, GPU scheduling, transmit powers, and bandwidth allocation, rather than ﬁxing one dimension of the system conﬁguration. In contrast, the ﬁxed-power and ﬁxed-bandwidth baselines lose ﬂexibility in communication resource adaptation, while the round-robin and random scheduling baselines cannot fully exploit workload and GPU heterogeneity. The comparison

with the equal-importance baseline further shows that modeling heterogeneous view contributions is useful for multiview image classiﬁcation, since more informative views can be preserved with higher priority under stringent resource constraints. Figs. 4 and 5 report the video-to-text captioning performance. Compared with classiﬁcation accuracy, the captioning curves are less strictly monotonic because BLEU-4 and CIDEr depend on discrete generated tokens and are more sensitive to sample-level linguistic variations. Nevertheless, the proposed design generally provides a better captioning quality/resource tradeoff than the ﬁxed power, ﬁxed bandwidth, round-robin scheduling, and random scheduling baselines. The improvement is reﬂected in both BLEU-4 and CIDEr, suggesting that the optimized resource allocation helps preserve not only local n-gram matching but also reference-consensus information in the generated captions. The gap over the equal-importance baseline is more moderate, which is reasonable because both

0.40

Proposed desig Fi)ed po(er Fi)ed ba d(idth Rou d-robi Ra dom scheduli g Equal importa ce

0.35 0.30

0.40 0.35 CIDEr

CIDEr

0.25 0.20

0.30

0.15

Proposed design

Fixed ower Fixed bandwidth Round-robin Random scheduling Equal im ortance

0.25

0.10 0.05

0.20

10.2

3.725 3.750 3.775 3.800 3.825 3.850 3.875 3.900 3.925 Energy threshold (J)

10.4

(a) Energy threshold

10.6

10.8 11.0 11.2 Delay threshold (s)

11.4

11.6

11.8

(b) Inference latency threshold

Figure 5. CIDEr scores for the video-to-text captioning task under different energy and inference latency thresholds. Round-robin scheduling (max 4.956 s)

Proposed design (max 2.737 s)

m=4 (G1,Q1)

m=1 (G1,Q2)

m=3 (G1,Q2)

m=3 (G1,Q3)

m=8 (G1,Q4) m=2 (G2,Q1)

m=7 (G1,Q3)

m=5 (G1,Q4)

Clusters

m=7 (G1,Q3)

Clusters

Random scheduling (max 3.930 s)

m=4 (G1,Q1)

m=2 (G2,Q1)

m=5 (G1,Q4) m=2 (G2,Q1)

m=3 (G2,Q2)

m=8 (G2,Q2)

m=1 (G2,Q2)

m=5 (G2,Q3)

m=7 (G2,Q3)

m=8 (G2,Q3)

m=6 (G2,Q4)

Waiting

Inference latency (s)

Inference latency (s) Arrival

Server computing

(a) Proposed design

Arrival

Waiting

Inference latency (s) Server computing

(b) Round-robin scheduling

Arrival

Waiting

Server computing

Figure 6. Task-level inference latency decomposition for the CIFAR-10 image-classiﬁcation task under the server-workload-heterogeneous latency stress setting. Each horizontal bar represents one cluster and is decomposed into arrival, waiting, and server computing intervals. The three panels compare the proposed design, round-robin scheduling, and random scheduling, respectively. The dashed vertical line denotes the maximum cluster completion time in each panel.

schemes use the same three-view input structure and the downstream caption generator can partially compensate for small differences in visual feature quality. Even in this case, incorporating heterogeneous importance remains beneﬁcial because it guides pruning decisions toward preserving views with stronger unique or synergistic contributions. Figs. 6 and 7 further visualize the task-level inference latency composition for the server-workload-heterogeneous stress setting. In these ﬁgures, the arrival interval corresponds to the time required for on-device inference and feature uploading, the waiting interval corresponds to GPU queueing before server execution, and the server computing interval corresponds to the execution time on the assigned GPU. For the CIFAR-10 task, the proposed design keeps the maximum completion time below the latency threshold, whereas roundrobin scheduling and random scheduling exceed the threshold because heavy server workloads are not balanced across GPU queues. For the video-to-text task, the advantage is more pronounced: the proposed design distributes the serverintensive captioning requests across the two GPUs, while the two heuristic schedules accumulate longer waiting intervals

on the bottleneck queue. These timeline results complement the threshold-sweep curves by showing how the optimized scheduling variables affect the internal latency composition of multi-cluster co-inference, especially when the server workloads are heterogeneous. Overall, the results on both task families show that the proposed framework is not tied to a single evaluation metric, and that joint pruning-aware communication and GPU-scheduling design can improve inference quality under resource constraints. Fig. 8 compares the optimized importance coefﬁcients, pruning ratios, and bandwidth allocations across four representative video clusters. For clusters m = 3 and m = 5, the retained-model ratios ρm,k are close to one for all devices, and the corresponding bandwidth allocations are nearly uniform within each cluster despite differences in importance coefﬁcients. In contrast, clusters m = 4 and m = 8 exhibit lower, interior retained-model ratios and more differentiated bandwidth allocations. In particular, Devices 2 and 3, whose importance coefﬁcients are substantially higher than that of Device 1, receive larger bandwidth shares, whereas Device 1 receives a smaller allocation due to its nearly negligible

12 Proposed design (max 21.252 s)

Round-robin scheduling (max 33.352 s)

m=8 (G1,Q2)

m=6 (G1,Q4)

Clusters

m=5 (G1,Q3)

m=4 (G2,Q1)

Random scheduling (max 27.470 s)

m=4 (G1,Q1)

m=1 (G1,Q2)

m=3 (G1,Q2)

m=3 (G1,Q3)

m=7 (G1,Q3)

m=5 (G1,Q4)

Clusters

m=3 (G1,Q1)

m=2 (G2,Q1)

m=2 (G2,Q2)

m=8 (G2,Q2)

m=1 (G2,Q2)

m=1 (G2,Q3)

m=7 (G2,Q3)

m=8 (G2,Q3)

m=7 (G2,Q4)

m=6 (G2,Q4)

Inference latency (s) Arrival

Waiting

Inference latency (s) Server computing

Arrival

(a) Proposed design

Waiting

Inference latency (s) Server computing

Arrival

(b) Round-robin scheduling

Waiting

Server computing

Figure 7. Task-level inference latency decomposition for the video-to-text captioning task under the server-workload-heterogeneous latency stress setting. Each horizontal bar represents one cluster and is decomposed into arrival, waiting, and server computing intervals. The three panels compare the proposed design, round-robin scheduling, and random scheduling, respectively. The dashed vertical line denotes the maximum cluster completion time in each panel.

0.2 0

0 Device 1

Device 2

Device 3

0.4

0.2 0

0 Device 1

Bandwidth

Device 2

Device 3

Bandwidth

0.6

0.4 2 0.2 0

0 Device 1

Device 2

Bandwidth

Device 3

0.8 10

0.6

and

0.6

0.8

Bandwidth (MHz)

0.4

0.8

and

0.6

and

0.8

Bandwidth (MHz)

and

1 6

0.4 5 0.2 0

0 Device 1

Device 2

Device 3

Bandwidth

(a) Cluster m = 3 (b) Cluster m = 4 (c) Cluster m = 5 (d) Cluster m = 8 Figure 8. Device importance coefﬁcients, retained-model ratios, and bandwidth allocations for representative video-to-text clusters. Clusters m = 3 and m = 5 operate near the no-pruning boundary, whereas clusters m = 4 and m = 8 illustrate coupled importance-aware resource allocation in the interior pruning regime.

semantic contribution. These trends reveal that the pruning point determines whether semantic importance can effectively inﬂuence bandwidth allocation. When ρm,k approaches one, as in clusters m = 3 and m = 5, model retention is nearly saturated. A higher importance coefﬁcient can no longer induce a meaningful increase in the retained-model ratio, weakening the principal pathway through which semantic importance affects the joint optimization. Bandwidth allocation is therefore governed mainly by communication-related factors, such as upload-delay requirements, channel conditions, transmit power, and cluster-arrival constraints. By contrast, when the pruning ratios lie in the interior region, as in clusters m = 4 and m = 8, they remain responsive to semantic importance. The coupling among importance coefﬁcients, model pruning, local computation, upload delay, and the shared bandwidth constraint becomes more active, allowing important devices to receive preferential bandwidth. The key insight is that the importance does not universally translate into additional bandwidth; instead, the pruning point acts as a gate that activates or saturates its inﬂuence, while the ﬁnal allocation results from its joint interaction with communication conditions. VI. C ONCLUSIONS In this paper, we proposed a pruning-aware multi-cluster LAIM co-inference framework for edge perception networks. The proposed architecture explicitly captures the coupling among communication, computation, and task scheduling arising from multiple device clusters competing for shared wireless spectrum and GPU resources. To characterize the impact of model compression on collaborative inference, we

established a rate-distortion-based analytical framework that reveals the trade-off between model pruning and inference performance while incorporating heterogeneous device contributions through PID. Based on this analysis, we formulated a joint inference distortion minimization problem and developed a low-complexity iterative algorithm to jointly optimize model pruning ratios, task scheduling, bandwidth allocation, and transmission power under latency and energy constraints. Extensive simulations on image classiﬁcation and video-totext captioning tasks demonstrated that the proposed framework consistently outperforms benchmark schemes, achieving superior inference quality and resource efﬁciency. Several directions deserve further investigation. First, the current framework assumes static device clustering and inference requests. Extending the proposed design to dynamic edge environments with user mobility, time-varying network conditions, and changing workloads is an important topic for future research. Second, although this work focuses on pruning-aware coinference, integrating other model compression techniques, such as quantization and knowledge distillation, may further improve deployment efﬁciency for resource-constrained edge devices. Finally, adaptive multi-agent collaboration with heterogeneous foundation models is another promising research direction. In such systems, model selection, feature fusion, and resource allocation can be jointly optimized to better coordinate multiple agents with different capabilities, thereby supporting more complex embodied AI and edge perception applications.

R EFERENCES [1] S. Salmi, M. A. Ouameur, M. Bagaa, G. C. Alexandropoulos, A. Tahenni, D. Massicotte, and A. Ksentini, “AI-native O-RAN architectures for 6G: Toward real-time adaptation, conﬂict resolution, and efﬁcient resource management,” IEEE Transactions on Network and Service Management, vol. 23, pp. 4110–4121, 2026. [2] N. Ge, C. Jiang, and J. Lu, “A Systematic Perspective on Communication Innovations Toward 6G,” Engineering, vol. 31, pp. 19–24, Dec. 2023. [3] X. Cao, D. Wen, S. Bi, Y. Cui, G. Zhu, H. Hu, and Y. C. Eldar, “Joint sensing, communication, and computation for vertical federated edge learning in edge perception networks,” IEEE Trans. Mobile Comput., pp. 1–14, 2026. [4] J. Zhang, J. Huang, S. Jin, and S. Lu, “Vision-language models for vision tasks: A survey,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 46, no. 8, pp. 5625–5644, 2024. [5] L. Dong, F. Jiang, Y. Peng, K. Wang, K. Yang, C. Pan, and R. Schober, “Lambo: Large AI model empowered edge intelligence,” IEEE Commun. Mag., vol. 63, no. 4, pp. 88–94, 2025. [6] Z. Lyu, X. Cao, X. Song, Y. Li, J. Wang, Y. Cui, W. Yuan, X. Yu, G. Zhu, J. Xu, D. W. K. Ng, D. Niyato, and S. Cui, “Sense smarter, think better: Edge perception for next-generation networks,” 2026. [Online]. Available: https://arxiv.org/abs/2605.18457 [7] B. Xu, H. Zhao, J. Kang, and D. Niyato, “Key Challenges and Research Directions for Space-Air-Ground Integrated Emergency Communication Networks,” Engineering, vol. 54, pp. 52–55, Nov. 2025. [8] G. Qu, Q. Chen, W. Wei, Z. Lin, X. Chen, and K. Huang, “Mobile edge intelligence for large language models: A contemporary survey,” IEEE Commun. Surveys Tuts., vol. 27, no. 6, pp. 3820–3860, 2025. [9] X. Du, Z. Lyu, X. Cao, C. Wen, S. Cui, and J. Xu, “Closing the generalization gap in parameter-efﬁcient federated edge learning,” 2025. [Online]. Available: https://arxiv.org/abs/2511.23282 [10] G. Team, R. Anil, S. Borgeaud, J.-B. Alayrac, J. Yu, R. Soricut, J. Schalkwyk, A. M. Dai, A. Hauth, K. Millican et al., “Gemini: a family of highly capable multimodal models,” arXiv preprint arXiv:2312.11805, 2023. [11] H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y. Babaei, N. Bashlykov, S. Batra, P. Bhargava, S. Bhosale et al., “Llama 2: Open foundation and ﬁne-tuned chat models,” arXiv preprint arXiv:2307.09288, 2023. [12] Z. Lyu, M. Xiao, J. Xu, M. Skoglund, and M. D. Renzo, “The larger the merrier? Efﬁcient Large AI model inference in wireless edge networks,” IEEE J. Sel. Areas Commun., pp. 1–1, 2025. [13] X. Li and S. Bi, “Optimal AI model splitting and resource allocation for device-edge co-inference in multi-user wireless sensing systems,” IEEE Trans. Wireless Commun., vol. 23, no. 9, pp. 11 094–11 108, Sep. 2024. [14] Y. He, J. Fang, F. R. Yu, and V. C. Leung, “Large language models (llms) inference ofﬂoading and resource allocation in cloud-edge computing: An active inference approach,” IEEE Transactions on Mobile Computing, vol. 23, no. 12, pp. 11 253–11 264, 2024. [15] H. Wu, X. Wang, X. Luo, W. Shen, and W. Lin, “Mcg-sched: Multicluster gpu scheduling for resource fragmentation reduction and load balancing,” IEEE Transactions on Parallel and Distributed Systems, vol. 36, no. 12, pp. 2789–2800, 2025. [16] Z. Lyu, G. Zhu, J. Xu, B. Ai, and S. Cui, “Semantic communications for image recovery and classiﬁcation via deep joint source and channel coding,” IEEE Trans. Wireless Commun., vol. 23, no. 8, pp. 8388–8404, 2024. [17] T. M. Getu, G. Kaddoum, and M. Bennis, “Semantic communication: A survey on research landscape, challenges, and future directions,” Proc. IEEE, vol. 112, no. 11, pp. 1649–1685, 2024. [18] C. Liu, C. Guo, Y. Yang, and N. Jiang, “Adaptable semantic compression and resource allocation for task-oriented communications,” IEEE Trans. Cog. Commun. Netw., vol. 10, no. 3, pp. 769–782, June 2024. [19] Z. Hu, C. You, T. Liu, D. Wen, Y. Hu, Y. Cui, Y. Gong, and K. Huang, “Semantic communication meets edge intelligence: Semanticrelay-aided text transmissions,” IEEE Internet Things J., vol. 11, no. 24, pp. 39 409–39 423, Dec 2024. [20] J. Yan, S. Bi, and Y.-J. A. Zhang, “Optimal model placement and online model splitting for device-edge co-inference,” IEEE Trans. Wireless Commun., vol. 21, no. 10, pp. 8354–8367, 2022. [21] D. Wen, P. Liu, G. Zhu, Y. Shi, J. Xu, Y. C. Eldar, and S. Cui, “Taskoriented sensing, computation, and communication integration for multidevice edge AI,” IEEE Transac. Wireless Commun., vol. 23, no. 3, pp. 2486–2502, 2024.

[22] X. Zhang, J. Nie, Y. Huang, G. Xie, Z. Xiong, J. Liu, D. Niyato, and X. Shen, “Beyond the cloud: Edge inference for generative large language models in wireless networks,” IEEE Trans. Wireless Commun., vol. 24, no. 1, pp. 643–658, 2025. [23] Y. Chen, R. Li, X. Yu, Z. Zhao, and H. Zhang, “Adaptive layer splitting for wireless LLM inference in edge computing: A modelbased reinforcement learning approach,” 2024. [Online]. Available: https://arxiv.org/abs/2406.02616 [24] M. Zhang, X. Shen, J. Cao, Z. Cui, and S. Jiang, “EdgeShard: Efﬁcient LLM inference via collaborative edge computing,” IEEE Internet Things J., vol. 12, no. 10, pp. 13 119–13 131, May 2025. [25] X. Xu, G. Feng, Y. Liu, S. Qin, J. Wang, and Y. Wang, “Joint inference ofﬂoading and model caching for small and large language model collaboration,” IEEE Transactions on Mobile Computing, vol. 25, no. 2, pp. 2691–2706, Feb 2026. [26] Z. Lyu, M. Xiao, M. Skoglund, M. Debbah, and H. V. Poor, “Quantization-aware collaborative inference for large embodied ai models,” arXiv preprint arXiv:2602.13052, 2026. [27] F. You, H. Du, K. Huang, and A. Jamalipour, “Jppo++: Joint power and denoising-inspired prompt optimization for mobile llm services,” IEEE Trans. Mobile Comput., pp. 1–16, 2026. [28] T. Berger, Rate-distortion theory. Wiley Encyclopedia of Telecommunications, 2003. [29] L. S. Shapley, A Value for N-Person Games. Santa Monica, CA: RAND Corporation, 1952. [30] A. Ghorbani and J. Zou, “Data shapley: Equitable valuation of data for machine learning,” in Proc. Int. Conf. Mach. Learn. (ICML), ser. Proc. Mach. Learn. Res., vol. 97. PMLR, 2019, pp. 2242–2251. [31] G. Optimization. LLC, 2021. [Online]. Available: https://www.gurobi. com [32] S. Boyd and L. Vandenberghe, Convex Optimization,. Cambridge Univ. Press, 2004. [33] M. Grant and S. Boyd, “CVX: Matlab software for disciplined convex programming,” 2016. [Online]. Available: http://cvxr.com/cvx [34] J. Shao, Y. Mao, and J. Zhang, “Task-oriented communication for multidevice cooperative edge inference,” IEEE Trans. Wireless Commun., vol. 22, no. 1, pp. 73–87, Jan. 2023. [35] J. Xu, T. Mei, T. Yao, and Y. Rui, “MSR-VTT: A large video description dataset for bridging video and language,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2016, pp. 5288–5296. [36] K. Wu, J. Zhang, H. Peng, M. Liu, B. Xiao, J. Fu, and L. Yuan, “TinyViT: Fast pretraining distillation for small vision transformers,” in Proc. Eur. Conf. Comput. Vis., 2022, pp. 68–85. [37] A. Yang et al., “Qwen3 technical report,” arXiv preprint arXiv:2505.09388, 2025. [38] K. Papineni, S. Roukos, T. Ward, and W.-J. Zhu, “BLEU: A method for automatic evaluation of machine translation,” in Proc. Annu. Meeting Assoc. Comput. Linguistics, 2002, pp. 311–318. [39] R. Vedantam, C. L. Zitnick, and D. Parikh, “CIDEr: Consensus-based image description evaluation,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2015, pp. 4566–4575.

Pruning-Aware Multi-Cluster Co-Inference for Large AI Models in AI-RANs

Analyze & explore related research in Conceptio

Related documents