Multi-Source and Multi-Dimensional Information Fusion in Resource-Constrained Edge Networks

1

Multi-Source and Multi-Dimensional Information Fusion in Resource-Constrained Edge Networks

arXiv:2610.13298v1 [cs.DC] 9 Oct 2026

Yushi Wang, Zhengxin Yu, Member, IEEE, Guhan Zheng, Member, IEEE, Wang Miao, Member, IEEE, Merouane Debbah, Fellow, IEEE

Abstract—Multi-source semantic information fusion enables edge networks to support global inference using observations collected by distributed terminals. However, limited communication and computation resources restrict semantic uploading, leaving the edge server with incomplete tokens that differ in modality, semantic granularity, arrival time, and channel condition. The value of these tokens also depends on the complementary or redundant information provided by other terminals. In this paper, we propose a multi-source and multi-dimensional semantic information fusion framework that coordinates terminal uploading and edge-side fusion under resource constraints. We develop Hyper-Slot to combine semantic content with available token attributes through iterative token-to-slot assignment and slot-to-token feedback, producing a compact representation for global inference. To encourage useful semantic uploading, we further design a contributionbased incentive mechanism that uses Hyper-Slot assignments to evaluate information coverage and cross-type complementarity. A bi-level Stackelberg–Nash game determines the server reward pool and terminal transmission-quality strategies while accounting for computation and communication energy costs. Experiments on a tri-modal extension of COCO2014 and imagetext retrieval on Flickr30K show that Hyper-Slot improves mAP by 3.66 percentage points and RSum by 15.9 points over the strongest evaluated baselines, respectively, while reducing the number of parameters by more than 80% compared with a naive concatenation-based fusion baseline. Index Terms—Multi-source information fusion, multidimensional heterogeneity, Semantic communications, .

I. Introduction Sixth-generation (6G) communication networks will support numerous intelligent applications, including intelligent transportation and vehicular networks [1] and the unmanned aerial vehicles systems [2]. These applications involve distributed terminals continuously generating various types of information, such as images, text, audio, and multisensor data [3]. Although the transmission of such information enables high service quality and new intelligent capabilities, it also significantly increases data transmission loads on edge networks with constrained spectrum resources. Efficiently utilizing information genYushi Wang and Zhengxin Yu are with the School of Computing and Communications, Lancaster University, LA1 4YW, U.K. (e-mail: y.wang216@lancaster.ac.uk; z.yu8@lancaster.ac.uk). Guhan Zheng is with the School of Communication and Information Engineering, Shanghai University, 200444, China. (e-mail: gzheng@shu.edu.cn). Wang Miao is with the Department of Computer Science, University of Exeter, EX4 4QF, U.K. (e-mail: wang.miao@exeter.ac.uk). Merouane Debbah is with KU 6G Research Center, Khalifa University of Science and Technology, 127788, Abu Dhabi, UAE. (e-mail: merouane.debbah@ku.ac.ae).

erated by distributed terminals therefore becomes an important issue for 6G networks. Multi-source information fusion provides an effective approach to this issue by integrating information from different terminals into a unified representation for global inference [4]. In edge networks, terminals can use local encoders to represent their observations as task-relevant semantic tokens, which are then transmitted to an edge server for fusion. Compared with relying on the information of a single terminal, multi-source fusion enables the global task to exploit information collected from different locations, sensing ranges, and data modalities. It is therefore a promising method for supporting intelligent tasks in 6G-enabled edge networks. Directly applying conventional multi-source information fusion methods to resource and communication constrained edge networks presents several challenges. Existing methods usually assume that information from all terminals can be completely transmitted to the edge server [5]. However, terminals in practical edge networks have limited computation capacity, transmission power, and available bandwidth. The edge server therefore cannot always obtain complete and high-quality semantic information from every terminal. The number and quality of the tokens available for fusion therefore depend on the uploading strategies of different terminals. This problem is further complicated by the differences among terminals. Semantic information collected by different terminals may provide different contributions to the target task. Their channel conditions and resource costs may also vary significantly, so each terminal must determine whether to upload each token and at what transmission quality. Independent decisions may cause terminals to transmit redundant information while omitting important semantic evidence from other sources. Both cases reduce the coverage and complementarity of the information available at the edge server. It is therefore necessary to coordinate the upload decisions of multiple terminals and preserve valuable and complementary semantic information under limited resources. Another challenge arises from the incomplete and heterogeneous semantic tokens produced by selective uploading. In addition to their semantic content, tokens may contain different attributes related to the fusion process. These attributes may include source, modality, semantic granularity, temporal state, and channel condition. The relevant dimensions vary with the application scenario, and together with the semantic content they

2

shape each token’s contribution to the task. For example, a semantically relevant token may become less useful under poor channel conditions, while several tokens with limited individual contributions may together provide important evidence. Conventional concatenation, pooling, and attention methods struggle to capture these relationships, and become less reliable when the number, quality, and structure of the received tokens vary across terminals. These challenges are closely coupled. Terminal uploading decisions determine the information available at the edge server, while the value of uploading a token depends on how it is fused with information from other terminals. Therefore, an effective fusion framework for edge networks should jointly address selective semantic uploading and multi-dimensional information fusion. It should coordinate uploading under communication and computation constraints and exploit both semantic content and available token attributes to fuse incomplete and heterogeneous tokens into a task-level representation. To address the above challenges, we propose a multisource and multi-dimensional semantic information fusion framework for resource and communication constrained edge networks. The framework coordinates semantic uploading among multiple terminals and fuses the incomplete and heterogeneous tokens received at the edge server. The main contributions are summarized as follows: We propose a unified framework for multi-source and multi-dimensional semantic information fusion in resource-constrained edge networks. The framework jointly considers selective semantic uploading at distributed terminals and information fusion at the edge server. By using the fusion assignments to evaluate terminal contributions and guide uploading decisions, the framework connects selective semantic uploading with edge-side information fusion under communication and computation constraints. • We develop Hyper-Slot to aggregate incomplete and variable-size sets of heterogeneous semantic tokens into a fixed number of latent slots. Hyper-Slot jointly models semantic content and available token attributes to determine token-to-slot assignments. Through iterative token-to-slot assignment and slotto-token feedback, it refines the grouping and aggregation of heterogeneous tokens to produce a compact task-level representation for global inference. • We design a semantic uploading incentive mechanism to encourage terminals to provide valuable and complementary information for global inference. The mechanism evaluates terminal contributions through the Hyper-Slot assignments, jointly considering information coverage, cross-type complementarity, and computation and communication energy costs. A bilevel Stackelberg–Nash game is then formulated to determine the server reward pool and the transmissionquality strategies of terminals under resource constraints. •

II. Related Works Edge intelligence enables terminals and edge servers to perform collaborative inference close to data sources. Zhou et al. [6] reviewed the deployment of artificial intelligence services at the network edge. Shi et al. [7] further discussed the communication cost introduced by device-edge collaboration. To reduce this cost, Xie et al. [4] developed a task-oriented multi-user semantic communication framework for image retrieval, machine translation, and visual question answering. Shao et al. [8] studied multi-device cooperative inference through compact local features. More recent studies have moved from transmitting complete feature maps toward finer-grained semantic representations. CodeFilling [9] represents selected collaborative features using compact codebook indices, while CoSDH [10] selects collaboration regions according to the information supply and demand of different agents. These methods improve the trade-off between perception accuracy and communication overhead. However, they generally operate under predefined collaboration protocols and fixed communication budgets, where terminal participation, upload quality, and heterogeneous resource costs are taken as given rather than jointly optimized. Resource-constrained semantic communication studies how semantic information can be efficiently delivered under limited communication and computation resources. DeepSC [11] and L-DeepSC [12] learn semantic representations that are robust to wireless channels. Deep joint source-channel coding has also been investigated for wireless image transmission [13], while Jankowski et al. [14] optimized transmitted representations for edge image retrieval. Recent studies further couple semantic importance with wireless conditions: Wang et al. [15] select task-important features and allocate channels to improve semantic spectral efficiency, and Yan et al. [16] jointly optimize semantic compression, channel assignment, and transmit power for multi-task networks. These studies improve how efficiently and reliably semantic information is delivered under constrained resources. Their selection and allocation decisions, however, are driven by communication-side objectives such as spectral efficiency, transmit power, or latency, and remain decoupled from how the delivered information is fused at the receiver. The received tokens are typically handed to a predefined decoder or task module, so whether a transmitted token is worth its cost for the subsequent multi-source fusion is left unmodeled. After transmission, the received semantic tokens must be integrated for downstream inference. Lee et al. [17] used stacked cross-attention for image-text matching, while Fu et al. [18] modeled semantic relations at both fragment and instance levels. Set Transformer [19] provides an attention-based architecture for set aggregation, and recent collaborative perception methods such as CoST [20] jointly aggregate information across agents and time steps in a unified spatiotemporal space. A separate line of work models structure explicitly: hypergraph neural

3

(a) Semantic Uploading Coordination Edge Server

(b) Heterogeneous Multi-modal Semantic Tokens over Diverse Links

(c) Hyper-Slot Multi-dimensional Fusion and Joint Inference

Vision Terminal

Arriving Multi-dimensional Semantic Tokens

Subtask Instructions

Local Encoder

Global Task

Semantic Tokens

Task-aware Importance Evaluation

Selected Tokens

Language Terminal Subtask Instructions

Non-cooperative Uploading Game

Local Encoder

Semantic Tokens

Task-aware Importance Evaluation

Selected Tokens

Modality

Token Number

Granularity

Arrival Time

18

t1

12

t2

24

t3

30

t4

Channel Quality

Audio Terminal Subtask Instructions

Local Encoder

Subtask Instructions

Local Encoder

Semantic Tokens

Task-aware Importance Evaluation

Selected Tokens

Hyper-Slot Multi-dimensional Fusion

Selected Tokens

Joint Inference

Multi-sensor Terminal

Subtask Instructions

Heterogeneous Transmission Conditions

Semantic Tokens

Task-aware Importance Evaluation

Global Task Result WiFi

6G

Satellite

Fig. 1. Overview of the proposed multi-source and multi-dimensional semantic information fusion framework.

networks [21] aggregate features over predefined highorder connections among elements. Our fusion mechanism instead follows the slot-based, object-centric tradition, where a set of slots competitively aggregate input elements rather than relying on a fixed higher-order graph. Slot Attention [22] introduced this paradigm to discover latent groups from a single image, without communication, heterogeneous attributes, or incomplete inputs. These methods are each designed for a particular relation, such as cross-modal interaction, set aggregation, high-order structure, latent grouping, or spatiotemporal dependency. They do not directly provide a unified mechanism for combining semantic content with the auxiliary attributes available in different edge scenarios, which may describe the modality, semantic granularity, temporal status, or channel condition of a token. This motivates a flexible multi-dimensional fusion method that can model their joint influence on information quality and cross-source complementarity. III. System Model In this section, we describe the considered edge network and its multi-source semantic communication and fusion model. We then introduce the computation and communication overhead associated with semantic uploading. A. Multi-Source Semantic Communication and Fusion Model We consider an edge network consisting of one edge server and M terminals, indexed by m ∈ M = {1, . . . , M }.

The system operates over discrete time slots t = 1, . . . , T . t At time slot t, terminal m collects a local observation Om and employs its local encoder to extract a set of candidate semantic tokens: n o t t Xm = Encoderm (Om ) = xtm,1 , . . . , xtm,Nm , (1) t t denotes the number of candidate tokens genwhere Nm erated by terminal m. Different terminals may observe the same task from different locations, sensing ranges, or modalities. Their local encoders may also have different structures. We assume that the extracted tokens are projected into a common semantic space, where xtm,i ∈ Rdx . Each token xtm,i is associated with an attribute det scriptor ηm,i and has a size of stm,i bits. The descriptor records the information available for semantic fusion. It may include the source, modality, semantic granularity, temporal state, and channel condition of the token. These attributes depend on the terminal, observation, and transmission environment. Therefore, tokens generated by different terminals may differ in semantic content, attribute configuration, and data size. Due to limited communication and computation resources, terminal m cannot upload all candidate tokens. Let

btm,i ∈ [0, 1].

(2)

Here, (btm,i ) denotes the normalized transmission quality assigned to token xtm,i . A value of btm,i = 0 indicates that the token is not uploaded, while a larger positive value indicates higher transmission quality. The value btm,i = 1

4

Attributes

Modality

Granularity

Arrival time

Channel quality

Vision

Image

Slot 1

K latent slots

Slot 2

s₁ Text Joint inference

s₂ Audio

Task result

sₖ Compact representation

Sensor

Slot K

Hyper-Slot

Iterative refinement

Fig. 2. Structure of the proposed Hyper-Slot module. Semantic tokens are assigned to latent slots using semantic content and token attributes, and iteratively aggregated into a compact representation for global inference.

represents the highest transmission quality. The uploading strategy of terminal m is represented by h i btm = btm,1 , . . . , btm,Nm . (3) t Accordingly, the token set selected by terminal m is  t t t Xm,up = (xtm,i , ηm,i ) | btm,i > 0, i = 1, . . . , Nm . (4) We consider the uplink transmission from terminal m to the edge server over a wireless channel with bandwidth t Bm , transmit power ptm , and channel gain htm . The achievable uplink rate is   pt h t t t Rm = Bm log2 1 + m 2 m , (5) σ max . where σ 2 is the noise power and 0 ≤ ptm ≤ Pm t The channel condition can also be included in ηm,i as a communication-related attribute. After transmission, the edge server receives the selected tokens from all terminals. The received token set is [ t Xt = Xm,up m∈M 
t t t = xtm,i , ηm,i | bm,i > 0, m ∈ M, i = 1, . . . , Nm

. (6) The size and composition of X t depend on the uploading strategies {btm }m∈M . Hence, the edge server generally receives an incomplete and variable-size token set. The received tokens may provide complementary information or describe overlapping semantic content. The edge server applies an edge-side fusion and inference mapping to the received token set:
bt = G X t , y (7)

where G(·) denotes the general mapping from the received bt . multi-source semantic tokens to the global task result y Its output depends jointly on the semantic content, available attributes, and uploading decisions of all terminals. The specific fusion implementation is introduced in the following section. B. Computation and Communication Overhead Model Terminal m incurs computation overhead when processing the selected semantic tokens and communication overhead when uploading them to the edge server. Let ctm,i denote the number of CPU cycles required to process token xtm,i for uploading. This process may include token refinement, attribute processing, and transmission preparation. The total computation load of terminal m is t

t Cm =

Nm X

btm,i ctm,i .

(8)

i=1 t Let fm denote the CPU frequency allocated by terminal m. The corresponding computation delay is t Cm . t fm The computation energy consumption is given by
t 2 t Cm , etm,cmp = κm fm t τm,cmp =

(9)

(10)

where κm is the effective switched-capacitance coefficient of terminal m [23]. The total size of the semantic tokens uploaded by terminal m is t Nm X t Sm = btm,i stm,i . (11) i=1

5

t Based on the achievable rate Rm , the communication delay is t PNm t btm,i stm,i S i=1 m t τm,com = t = . (12) t Rm Rm

strategies, and the selected tokens are fused by Hyper-Slot to yield the global inference result. The game determines which information reaches the fusion stage, while HyperSlot determines how that information is combined.

The corresponding communication energy consumption is t PNm ptm i=1 btm,i stm,i t t t em,com = pm τm,com = . (13) t Rm

Algorithm 1 The Proposed Framework t Require: Local observations {Om }, time slots T , slot number K, refinement iterations L 1: for t = 1 to T do 2: for all m ∈ M do t t 3: Xm ← Encoderm (Om ) 4: end for
t 5: {bt∗ m }m∈M ← GAME {Xm }m∈M , G 6: for all m ∈ M do t t 7: Xm,up ← {(xtm,i , ηm,i ) | bt∗ m,i > 0} 8: end for [ t 9: Xt ← Xm,up

Therefore, the total processing delay and energy consumption of terminal m are respectively expressed as t t t τm = τm,cmp + τm,com ,

(14)

etm = etm,cmp + etm,com .

(15)

and

IV. The Proposed Framework In this section, we present the proposed framework for multi-source and multi-dimensional semantic information fusion. We first describe the overall framework, followed by the Hyper-Slot fusion module and the semantic uploading mechanism based on a bi-level Stackelberg–Nash game. A. Framework Description Fig. 1 shows the proposed multi-source and multidimensional information fusion framework. At each time slot t, every terminal m ∈ M generates a candidate set mantic token set Xm from its local observation, and selects which tokens to upload under its own communication and computation constraints. The tokens received at the edge server are generally incomplete, variable in size, and heterogeneous in modality, semantic granularity, arrival time, and channel condition. To fuse such tokens, we propose Hyper-Slot in Section IV-B, which combines each token’s semantic content with its available attributes and aggregates the received tokens into a unified task-level representation, as shown in Fig. 2. Since Hyper-Slot aggregates tokens according to their joint contribution to each slot, a terminal’s effect on the fused representation depends on what the other terminals upload as well, so independent per-terminal uploading decisions can leave the received tokens redundant or incomplete. We therefore coordinate uploading through a two-level game in Section IV-C. The edge server announces an incentive, and the terminals determine their uploading strategies under this incentive through a non-cooperative game that settles at a Nash equilibrium. The reward given to each terminal is computed from its marginal contribution to the fusion gain, which is defined directly from the Hyper-Slot assignment weights. This ties each terminal’s uploading decision to the information already provided by the others, allowing the framework to retain complementary information and reduce redundant transmission under resource constraints. At each time slot, the terminals first generate candidate tokens, the two-level game then determines their uploading

m∈M

t

t {(xi , ηi )}N i=1 ← X for all i = 1, . . . , N t do di ← Γθ (ηi ) gi ← Gate(di ) (0) 14: xi ← xi 15: end for (0) (0) 16: Initialize {sk , uk }K k=1 17: for ℓ = 1 to L do 18: for all i = 1, . . . , N t do (ℓ) (ℓ−1) 19: qi ← Wq xi (ℓ) (ℓ−1) 20: vi ← Wv xi 21: end for (ℓ) (ℓ−1) (ℓ−1)
22: A(ℓ) ← SparseAssign {qi , di }, {sk , uk } 23: for all k = 1, . . . , K do t (ℓ) (ℓ) (ℓ) 24: ek ← SemanticAggregate({aik , gi , vi }N i=1 ) t (ℓ) (ℓ) 25: ck ← DescriptorAggregate({aik , di }N i=1 ) (ℓ) (ℓ−1) (ℓ) 26: sk ← GRUs (sk , ek ) (ℓ) (ℓ−1) (ℓ) 27: uk ← GRUu (uk , ck ) 28: end for 29: for all i = 1, . . . , N t do (ℓ) (ℓ−1) (ℓ) 30: xi ← TokenUpdate(xi , A(ℓ) , {sk }) 31: end for 32: end for (L) 33: ot ← Pool({sk }K k=1 ) t t 34: ŷ ← ϕ(o ) 35: end for 36: return {ŷt }T t=1

10: 11: 12: 13:

B. Hyper-Slot for Multi-dimensional Fusion To fuse heterogeneous semantic tokens received at the edge server, we propose Hyper-Slot, a dimension-aware token fusion module. The key idea is to map tokens with different semantic contents and heterogeneous conditions into a fixed number of latent slots, where each slot adaptively captures a group of tokens that are coherent in both semantic and task-specific dimensions. We consider a set of semantic tokens X = {(xi , ηi )}N i=1 ,

(16)

6

where xi ∈ Rdx denotes the semantic embedding of token i, N is the number of received tokens, and ηi denotes its heterogeneity descriptor. Specifically, (1)

(2)

(P )

ηi = (ζi , ζi , . . . , ζi

(17)

),

where P is the number of task-configurable dimensions. (p) Each dimension ζi may describe a categorical or continuous property of the token, such as its source, modality, semantic granularity, confidence, freshness, or communication condition. These dimensions are not fixed by the model and can be instantiated according to the considered edge task. The heterogeneity descriptor is encoded as di = Γθ (ηi ) ∈ Rdη ,

(18)

where Γθ (·) embeds categorical dimensions and maps continuous dimensions into a unified descriptor space. Hyper-Slot maintains K learnable slots. At iteration (l) l, slot k contains a semantic anchor sk ∈ Rd and (l) a heterogeneity anchor uk ∈ Rdη . The token state is (0) initialized as xi = xi . At each iteration, token queries and values are computed by (l)

(l−1)

qi = Wq xi

,

(l)

(l−1)

vi = Wv xi

,

(19)

where Wq , Wv ∈ Rd×dx are learnable projections. The affinity between token i and slot k is defined by combining semantic similarity and heterogeneity consistency: (l−1) (l) (l−1) ⟨s ,q ⟩ ⟨u , di ⟩ (l) ℓik = k √ i + β kp , (20) dη d where β balances the contribution of the heterogeneity descriptor. The assignment from tokens to slots is obtained by sparse normalization [24]:   (l) (l) (l) aik = sparsemaxk ℓi1 , . . . , ℓiK , (21) PK (l) (l) where aik ≥ 0 and k=1 aik = 1. We further introduce a descriptor-dependent gate gi to dynamically control the effective contribution of each token
gi = σ wg⊤ di + bg , (22) where gi ∈ (0, 1) reflects the reliability or usefulness of token i under its heterogeneity descriptor. Each slot aggregates semantic and descriptor information as PN (l) (l) aik gi vi (l) ek = Pi=1 , (23) (l) N i=1 aik + ϵ PN (l) aik di (l) ck = PNi=1 (l) , i=1 aik + ϵ (l)

(24) (l)

where ek is the aggregated semantic representation, ck is the aggregated heterogeneity representation, and ϵ is a small constant for numerical stability.

The slot anchors are then updated by recurrent refinement based on Gated Recurrent Units (GRUs) [25]:     (l−1) (l) (l) (l−1) (l) (l) , ek , uk = GRUu uk , ck . sk = GRUs sk (25) To allow tokens to adjust their slot assignments across iterations, the slot information is written back to the token states: ! K X (l) (l−1) (l) (l) xi = LN xi +α aik ψ(sk ) , (26) k=1

where ψ : R → R is a learnable projection, LN(·) denotes layer normalization [26], and α controls the residual strength. After L iterations, the fused representation is obtained by pooling the final slots:   (L) o = Pool {sk }K (27) k=1 , d

dx

where Pool(·) can be mean pooling or attention pooling. The final prediction is given by (28)

ŷ = ϕ(o), where ϕ(·) is a task-specific prediction head. The model is trained end-to-end with L = Ltask + λdiv Ldiv ,

(29)

where Ltask is the downstream task loss. To prevent all slots from collapsing to similar heterogeneity patterns, we use a diversity regularization term over the heterogeneity anchors: !2 L (l) (l) ⟨uk , uk′ ⟩ 1XX Ldiv = , (30) (l) (l) L ∥u ∥2 ∥u ′ ∥2 + ϵ ′ l=1 k̸=k

k

k

where λdiv controls the strength of the diversity regularization. C. Incentive Mechanism for Semantic Uploading Section IV-A established the need to coordinate terminal uploading. The edge server aims to obtain sufficient and complementary semantic information for the global fusion task, whereas each terminal incurs computation and communication costs when uploading its local tokens. Therefore, a terminal may not have sufficient incentive to consume additional resources solely for improving the global task. To coordinate these objectives, we introduce an incentive mechanism in which the edge server announces a reward pool R before semantic uploading. Each terminal then determines its uploading strategy according to the announced reward and the strategies of the other terminals. Since the contribution of one terminal depends on the information provided by the others, the terminal decisions are coupled. Accordingly, the edge server acts as the leader, while the terminals form a non-cooperative game under the given reward pool.

7

1) Utility Function of Terminals: For notational simplicity, the time-slot index t is omitted throughout this subsection. Under the announced reward pool R, each terminal determines its uploading strategy by balancing its contribution-based reward against the computation and communication energy costs incurred by semantic uploading. The utility of terminal m can hence be expressed as Um (bm ; b−m , R) = R P P

∆m (b) j∈M ∆j (b) + ϵ

(31)

− em,cmp (bm ) − em,com (bm ),

where R∆m (b)/( j∈M ∆j (b)+ϵ) is the reward allocated to terminal m from the total reward pool R, and ϵ > 0 is a small constant that keeps the denominator strictly positive. The terms em,cmp (bm ) and em,com (bm ) are its computation and communication energy costs, respectively. Here, bm = [bm,1 , . . . , bm,Nm ] is the transmissionquality strategy of terminal m, b−m denotes the strategies of the other terminals, and b = (bm , b−m ) is the joint strategy. Specifically, bm,i = 0 indicates that token xm,i is not uploaded, while bm,i ∈ (0, 1] represents its normalized transmission quality. A larger bm,i indicates that a higherquality communication link is assigned to the token. The fusion contribution of a terminal depends on the joint information provided by all participating terminals. For a given joint strategy b, we characterize the contribution of terminal m by the marginal change in fusion gain resulting from its participation: ∆m (b) = F (b) − F (0m , b−m ),

(32)

where F (b) denotes the fusion gain under the joint strategy, and F (0m , b−m ) denotes the fusion gain when terminal m is excluded while the strategies of the other terminals remain unchanged. Thus, ∆m (b) represents the contribution of terminal m conditioned on the information jointly provided by the remaining terminals. The fusion gain consists of information coverage and cross-type complementarity. The information coverage characterizes the effective amount of each information type in each slot, while the cross-type complementarity characterizes the joint contribution of different information types assigned to the same slot. We hence define the fusion gain as F (b) =

K X X



ωp 1 − e−λzk,p (b)



increasingly high transmission quality of repeated or overlapping information in the same slot. The quantity n o min 1 − e−λzk,p (b) , 1 − e−λzk,q (b) characterizes the complementarity between information types p and q, where their joint contribution is limited by the information type with lower coverage. The effective amount of type-p information in slot k is defined as X X zk,p (b) = bm,i am,i,k , (34) m∈M i∈Im,p

where Im,p is the index set of type-p tokens generated by terminal m, and am,i,k ≥ 0 is the Hyper-Slot assignment weight defined in the previous subsection. Therefore, the effective contribution of a token depends on its assigned slot and selected transmission quality. Terminal m determines its semantic uploading strategy to maximize its utility under the reward pool and the strategies of the other terminals. The corresponding optimization problem is P1: max

Um (bm ; b−m , R),

(35a)

s.t.

0 ≤ bm,i ≤ 1,

(35b)

bm

Nm X

bm,i ≤ b̄m .

(35c)

i=1

The constraint 0 ≤ bm,i ≤ 1 defines the transmissionquality decision of each token. The value bm,i = 0 represents no transmission, while a positive value represents the normalized quality PNofm the selected communication link. The constraint i=1 bm,i ≤ b̄m limits the total high-quality communication resource available to terminal m, where b̄m is its total normalized transmission-quality budget. This constraint prevents all tokens from using the highest-quality links simultaneously and encourages the terminal to allocate better links to tokens with larger fusion contributions. 2) Utility Function of the Edge Server: The edge server determines the reward pool by considering the global fusion gain and the associated incentive and energy costs. Its utility can hence be expressed as Us (R; b∗ (R)) = F (b∗ (R)) − R X − [em,cmp (b∗m (R)) + em,com (b∗m (R))] . m∈M

k=1 p∈P

i = 1, . . . , Nm ,

(36) ∗ F is the fusion benefit obtained under the Here, (b (R)) , + ξp,q min 1 − e ,1 − e equilibrium terminal strategies, R is the reward pool k=1 p,q∈P p<q committed by the edge server, and (33) X [em,cmp (b∗m (R)) + em,com (b∗m (R))] where P is the set of task-relevant information types, m∈M ωp ≥ 0 is the importance weight of type p, λ > 0 is the saturation coefficient, and ξp,q ≥ 0 is the complementarity denotes the total computation and communication energy coefficient between information types p and q. cost. The edge server anticipates the equilibrium response of Notably, 1 − e−λzk,p (b) has a theoretical upper bound of one and captures the diminishing return caused by the terminals and determines the reward pool by solving K X X

n

−λzk,p (b)

−λzk,q (b)

o

8

P2: max

Us (R; b∗ (R)) ,

(37a)

s.t.

0≤R≤R

(37b)

R

max

,

where Rmax is the maximum incentive budget available to the edge server. For a given reward pool R, the terminals form a noncooperative game. The uploading strategy of each terminal affects the fusion gain and the contribution-based reward allocation through the joint strategy b. The strategy profile b∗ (R) is a Nash equilibrium if

Um b∗m ; b∗−m , R ≥ Um bm ; b∗−m , R , (38) for every terminal m ∈ M and every feasible strategy bm . 3) Bi-level Stackelberg–Nash Game Theoretical Algorithm: The optimization problems for the terminals (P1) and the edge server (P2) are coupled. The solution of P1 depends on the reward pool R announced by the edge server and the transmission-quality strategies of the other terminals. Meanwhile, the solution of P2 depends on the equilibrium terminal strategies b∗ (R). This coupling forms a bi-level nonlinear optimization problem. We hence propose a Stackelberg–Nash game theoretical algorithm to determine the terminal strategies and the server reward pool. The terminal-side game has a Cournot-type structure because each terminal selects its transmission-quality quantities while competing through the shared fusion gain and contribution-based reward allocation. For the game analysis, the Hyper-Slot assignment weights am,i,k are obtained from the latest Hyper-Slot state and remain fixed during one execution of the game. They are updated after fusion and used in the subsequent time slot. Since am,i,k ≥ 0, ωp , ξp,q ≥ 0, and 1 − e−λz is non-decreasing in z, F (b) is non-decreasing in every bm,i . Therefore, ∆m (b) = F (b) − F (0m , b−m ) ≥ 0.

(39)

The constant ϵ > 0 ensures that the reward allocation remains well defined when all terminals provide zero marginal contribution. The feasible strategy set of terminal m is ( ) Nm X Bm = bm 0 ≤ bm,i ≤ 1, bm,i ≤ b̄m . (40) i=1

The set Bm is non-empty, compact, and convex. Given the strategies of the other terminals, the utility Um (bm ; b−m , R) is continuous over Bm . Therefore, the best-response problem of each terminal admits at least one optimal solution. For a given reward pool R, the terminals simultaneously determine their transmission-quality strategies. The strategy of one terminal affects the common fusion gain and the reward shares of all terminals. Given b−m and R, the best response of terminal m is BRm (b−m ; R) = arg max Um (bm ; b−m , R) . bm ∈Bm

(41)

At differentiable points, the marginal utility of assigning a higher transmission quality to token xm,i is   X ∂∆j ∂∆m  X ∆j + ϵ  − ∆m ∂bm,i ∂bm,i j∈M j∈M ∂Um =R P 2 (42) ∂bm,i j∈M ∆j + ϵ − The quantity

∂em,cmp ∂em,com − . ∂bm,i ∂bm,i " # ∆m (b) ∂ RP ∂bm,i j∈M ∆j (b) + ϵ

characterizes the marginal reward obtained by increasing the transmission quality of token xm,i , accounting for the changes in both terminal m’s contribution and the total contribution used for reward allocation. The derivatives of em,cmp (bm ) and em,com (bm ) characterize the corresponding marginal computation and communication energy costs. We use damped best-response iteration to seek a Nash equilibrium among the terminals. At iteration r, terminal m first solves P1 under the current strategies of the other terminals and then updates
r r br+1 (43) m = (1 − η)bm + ηBRm b−m ; R , where η ∈ (0, 1] is the damping factor. The procedure terminates when br+1 − br 2 ≤ δ,

(44)

where δ > 0 is the convergence tolerance. When the procedure converges, its fixed point satisfies
b∗m ∈ BRm b∗−m ; R , ∀m ∈ M, (45) and is taken as the terminal-side Nash equilibrium b∗ (R). The edge server acts as the leader and anticipates the Nash equilibrium response b∗ (R). It determines the reward pool by solving max

0≤R≤Rmax

Us (R; b∗ (R)) .

(46)

We use numerical gradient ascent to search for the reward pool that maximizes the server utility. The algorithm is initialized with a feasible reward pool R0 , a learning rate α, and a convergence tolerance. At iteration ℓ, the gradient is approximated by

Us Rℓ + h; b∗ (Rℓ + h) − Us Rℓ ; b∗ (Rℓ ) dUs ≈ , (47) dR h where h > 0 is a small step size. Each evaluation of the edge server utility requires solving the terminal-side game to obtain the corresponding Nash equilibrium. The reward pool is then updated as  
dUs Rℓ , (48) Rℓ+1 = Π[0,Rmax ] Rℓ + α dR where Π[0,Rmax ] (·) projects the updated reward pool onto its feasible interval.

9

The outer iteration terminates when R

ℓ+1

− R ≤ δ. ℓ

(49)

All experiments were conducted on the Lancaster University HEC GPU cluster using NVIDIA V100 (32GB) and L40 (48GB) nodes via the gpu-short/gpu-medium queues.

The resulting pair {b∗ (R∗ ), R∗ }

(50)

is taken as an approximate Stackelberg–Nash solution of the bi-level game. The individual reward of each terminal is then obtained directly from the contribution-based reward allocation rule in Um . V. Simulations In this section, we evaluate the proposed framework through semantic fusion experiments and game-theoretic simulations. We compare Hyper-Slot with other fusion methods and examine the effects of heterogeneity dimensions, transmission budgets, and its structural components. We further evaluate Hyper-Slot on cross-modal retrieval and investigate the performance of the semantic uploading mechanism. A. Simulation Settings We evaluate Hyper-Slot on a tri-modal extension of COCO2014 [27], comprising 30,000 training pairs and 5,000 validation pairs across 80 categories. Each image is paired with its original textual captions and, following SpokenCOCO [28], a corresponding spoken caption, yielding three modality sources — vision, text, and audio — per sample. This construction provides a natural source of modality heterogeneity in addition to the semantic granularity, arrival time, and channel condition dimensions considered below. Visual tokens are extracted with a Faster R-CNN (ResNet-101) backbone [29], producing region-level features of dimension 2048. Textual tokens are extracted with BERT-base-uncased [30], producing token-level features of dimension 768. Audio tokens are obtained by first applying the Montreal Forced Aligner (MFA) [31] to determine word-level time boundaries in the spoken caption, then extracting a 768-dimensional feature per aligned segment with a frozen pre-trained HuBERT-base model [32]. All raw features are projected through linear layers into a unified embedding dimension dx . The number of latent slots is set to K = 16 and the number of refinement iterations to L = 3. The token embedding dimension is dx = 256, the slot dimension is d = 256, and the heterogeneity descriptor dimension is dη = 256. The affinity balancing coefficient β in (20) is set to 1.0, and the residual strength α in (26) is set to 1.0. Hyper-Slot is trained end-to-end using the AdamW optimizer with a learning rate of 1 × 10−4 , a weight decay of 1 × 10−4 , and a batch size of 64. For each token, the descriptor ηi is instantiated with four dimensions: (i) modality (vision/text/audio), (ii) semantic granularity, and (iii) arrival time and (iv) channel quality, the latter two simulated to emulate asynchronous terminal reporting and time-varying wireless conditions.

B. Performance Comparison with Other Fusion Methods We compare Hyper-Slot with five representative fusion methods for heterogeneous semantic tokens. Concat + MLP [33] flattens and concatenates all tokens before applying an MLP predictor. It does not explicitly preserve token structure or model relations among tokens. Mean Pool [34] averages all tokens into a single global representation, and therefore loses token-level differences. SCAN [17] uses stacked cross-attention followed by pooling to model pairwise interactions, but it does not explicitly group related tokens. Slot Attention [22] performs competitive slot-based grouping. However, it does not use heterogeneity-aware affinity or descriptor-dependent gating. Static HGNN [21] constructs hyperedges according to predefined rules, such as grouping tokens with the same modality or semantic granularity. These fixed hyperedges may not capture relations that span multiple heterogeneity dimensions. In contrast, Hyper-Slot assigns tokens to slots according to both semantic similarity and heterogeneity consistency. The assignments are refined through iterative slot-to-token feedback. This allows the model to learn cross-dimensional token relations without relying on predefined grouping rules. Table I reports overall mAP over five random seeds together with parameter count, GFLOPs, inference time, and memory footprint. Hyper-Slot achieves the highest mean mAP (0.7610±0.0239), outperforming the strongest baseline (SCAN and Slot Attention, both at 0.724) by over three points. This indicates that jointly modeling semantic content and heterogeneity attributes yields both higher accuracy and more stable convergence than methods. The efficiency columns show that this gain comes at a moderate computational cost: Hyper-Slot uses more parameters and GFLOPs than Slot Attention or Static HGNN, reflecting the additional heterogeneity encoder Γθ (·) and gating mechanism, but remains substantially lighter than Concat + MLP while achieving markedly higher accuracy. Figure 3 breaks down per-class mAP across 5 representative categories. Hyper-Slot achieves the highest mAP in every category shown, with the margin over the bestperforming baseline remaining consistent across categories of varying difficulty rather than concentrated on a subset of classes. Figure 4 visualizes the assignment of input tokens to the learned slots. Tokens of the same modality are distributed across multiple slots rather than collapsing into one, and each slot anchor lies close to its assigned token cluster. C. Effect of Heterogeneity Dimensions To examine whether the heterogeneity dimensions in ηi contribute independently or jointly shape token contribution, we evaluate six descriptor configurations: using semantic content alone (i.e., removing the heterogeneity

10

TABLE I Comparison of mAP and computational efficiency over five random seeds. mAP over random seeds

Method Mean Pool [34] Concat MLP [33] SCAN [17] Slot Attention [22] HGNN [21] Hyper-Slot

Efficiency

Seed 42

Seed 123

Seed 456

Seed 789

Seed 1024

Mean ± Std

0.5431 0.6345 0.6564 0.7009 0.6548 0.7368

0.5349 0.6309 0.6706 0.6969 0.6469 0.7345

0.6876 0.6386 0.7599 0.7312 0.7021 0.7724

0.6964 0.6536 0.7702 0.7502 0.7195 0.7871

0.6920 0.6568 0.7651 0.7407 0.7108 0.7744

0.6308 ± 0.0839 0.6429 ± 0.0116 0.7244 ± 0.0560 0.7240 ± 0.0239 0.6868 ± 0.0336 0.76104 ± 0.0239

Params (M)

GFLOPs

Time (ms)

Memory (MB)

1.193 22.034 3.365 1.860 2.183 4.166

0.0582 0.0999 0.3364 0.1526 0.2263 0.5729

0.0300 0.0274 0.0880 0.0565 0.0720 0.1473

132.62 450.40 165.53 143.38 147.94 348.18

Fig. 3. Comparison of per-class mAP across five categories on COCO2014 dataset.

Fig. 5. Effect of heterogeneity descriptor configurations on mAP.

Fig. 4. Visualization of token-to-slot assignments in Hyper-Slot. Colors indicate the assigned slots, and star markers denote the slot anchors.

tial, indicating that these dimensions are complementary rather than redundant: no single attribute recovers the performance achieved by combining all four. In addition, seed-to-seed variance is markedly larger under singledimension configurations than under full heterogeneity, this suggests that relying on a single heterogeneity attribute not only limits average performance but also makes fusion quality more sensitive to random initialization. These results indicate that the heterogeneity dimensions in Hyper-Slot are not independently informative but interact to shape a token’s contribution to fusion. D. Effect of Transmission Budget

descriptor entirely), using only modality, only semantic granularity, only arrival time, only channel quality, and the full four-dimensional descriptor. Figure 5 reports mAP under each configuration across five random seeds. The full heterogeneity descriptor achieves the highest mean mAP across all seeds, exceeding every singledimension configuration by a substantial margin, while content-only, which discards the heterogeneity descriptor entirely, yields the lowest mean mAP among the six configurations. The gap between the strongest single-dimension configuration and the full descriptor remains substan-

Terminals operating under limited transmission budgets cannot upload all candidate tokens and must select a subset to transmit. We compare four selection strategies against a full-transmission upper bound: Random selection, which uploads tokens without regard to content; Top-SNR selection, which prioritizes tokens under the best channel conditions irrespective of semantic value; Top-importance selection, which ranks tokens by a taskrelevance score; and task-aware selection, which ranks tokens using signals produced by Hyper-Slot: the descriptordependent gate gi in (22) as a proxy for token reliability,

11

Structural Ablation of Hyper-Slot

Performance under Transmission Budget 0.75

0.75 Full transmission upper bound

0.7

0.7

mAP

0.65

mAP

0.65 0.6

0.6 0.55

0.55

Compact only Dependency only Full Hyper-Slot

0.5

0.5

0.45 Task-aware Top-importance Top-SNR Random

0.45 0.4 25%

50%

75%

100%

Token transmission budget (%)

Fig. 6. Comparison of token selection strategies under different token transmission budgets. The dashed line indicates the full-transmission performance.

and the assignment weight ai,k as a proxy for the marginal coverage a token still contributes to its slot. Figure 6 reports mAP at budgets of 25%, 50%, 75%, and 100% of the full-transmission cost. Task-aware selection dominates at every budget below 100%, with the largest margin over the next-best strategy (Top-importance) at 25% and a narrowing gap as the budget increases, converging to the full-transmission upper bound at 100%. Topimportance consistently outperforms Random and TopSNR below 100% budget. These results show that selection guided by Hyper-Slot’s own reliability and coverage signals is more communication-efficient than content-agnostic (Random) or channel-only (Top-SNR) heuristics. E. Structural Ablation of Hyper-Slot Hyper-Slot has two core design elements: dependency modeling among tokens through iterative competitive assignment and slot-to-token feedback, and the compact fixed-size slot representation. We isolate their contributions by comparing two structural variants against the full model. Dependency only retains the iterative sparsemax assignment in (21) and the slot-to-token feedback in (26). It removes the compact bottleneck by allowing the number of slots to scale with the number of received tokens rather than remaining fixed at K. Compact only retains the fixed K-slot bottleneck. It replaces the iterative competitive assignment and feedback with a single-pass, non-competitive aggregation. Figure 7 shows mAP over training epochs for both variants and the full Hyper-Slot. Full Hyper-Slot converges to the highest mAP (≈ 0.75). Compact only converges next (≈ 0.72). Dependency only converges to the lowest mAP (≈ 0.665). All three variants converge within the first 20–30 epochs. Removing the compact bottleneck causes the larger performance drop. Removing iterative dependency modeling causes a smaller but consistent drop. Neither variant recovers the mAP of the full model. This shows the two design elements are complementary.

10

20

30

40

50

60

70

80

90

100

Epoch

Fig. 7. Structural ablation of Hyper-Slot: mAP over training epochs for the compact-only variant, the dependency-only variant, and the full model.

F. Performance Evaluation on Flickr30K To verify the applicability of our proposed method across different datasets and tasks, we extended the proposed Hyper-Slot to a dataset for the task of bimodal image-text matching and conducted an additional experiment. Flickr30K [35] contains 31,000 images with five captions per image. We follow the standard split of 29,000 training images, 1,000 validation images, and 1,000 test images. Each image forms five ground-truth image-caption pairs. The task is cross-modal retrieval rather than multi-label classification. Given a query image, the model retrieves the matching caption from the candidate set, and given a query caption, the model retrieves the matching image. Performance is measured by RSum, the sum of R@1, R@5, and R@10 for both image-to-text and text-to-image retrieval. We compare Hyper-Slot against six baselines. The concatenation-based fusion baseline [34], Slot Attention [22], and the fixed-hyperedge baseline following HGNN [21] are the same methods introduced in Section V-B, here applied to region and caption token features instead of the tri-modal tokens used earlier. VSE++ [36] aggregates region features by simple pooling before matching. GPO [34] learns a generalized pooling function over region features. The fixed-hyperedge baseline follows HREM [18] models relations among fragments and instances through self-attention. Table II reports RSum over five random seeds together with parameter count, GFLOPs, inference time, and memory usage. Hyper-Slot achieves the highest mean RSum (478.9 ± 3.4), exceeding the strongest baseline, Slot Attention (463.0 ± 3.2), by 15.9 points. HREM is the second-strongest baseline (459.2±3.2). GPO and the fixed hyperedge baseline follow, both above VSE++ and the concatenation baseline. VSE++ has the fewest parameters among all methods but the lowest RSum. Hyper-Slot uses fewer parameters than the concatenation baseline and Slot Attention, while achieving the highest accuracy among all seven methods.

12

TABLE II Comparison of RSum and computational efficiency on Flickr30K over five random seeds. RSum over random seeds

Method Seed 123

Seed 456

Seed 789

Seed 1024

Mean ± Std

Params (M)

GFLOPs

Time (ms)

Memory (MB)

411.2 428.6 446.8 436.4 458.7 454.9 474.6

417.5 433.1 451.5 443.1 463.4 460.3 479.2

412.9 430.5 454.2 439.8 461.2 457.6 476.8

419.0 436.2 456.0 446.2 467.0 463.5 483.1

415.1 432.4 450.9 441.5 464.8 459.8 481.0

415.1 ± 3.2 432.2 ± 2.9 451.9 ± 3.5 441.4 ± 3.7 463.0 ± 3.2 459.2 ± 3.2 478.9 ± 3.4

2.886 80.484 2.911 5.018 26.006 15.489 21.867

0.1825 0.3377 0.1826 0.3056 1.3289 0.7697 1.3553

0.0330 0.0632 0.1267 0.1025 0.3087 0.2210 0.7147

31.39 330.15 50.52 49.13 132.91 93.03 117.56

To validate the efficacy of the proposed incentive mechanism, we conduct a game-theoretic simulation. This experiment is designed to show the optimal incentive strategy for semantic uploading by analysing the relationship between the server utility and the reward pool. We configure a network with five terminals, three information types, and eight fusion slots. The number of semantic tokens generated by each terminal is randomly varied from 8 to 16 to emulate heterogeneous information availability. The information-type importance weights are set to ω = [1.0, 1.0, 1.2], while the cross-type complementarity coefficients ξp,q are set between 0.30 and 0.40. The saturation coefficient is set to λ = 1. For each terminal, the transmission-quality budget is randomly set between 50% and 60% of its full-quality transmission demand. The normalized computation and communication cost coefficients are randomly varied within [0.05, 0.10] and [0.10, 0.20], respectively, to emulate heterogeneous resource costs. The maximum reward pool is set to Rmax = 12. For numerical solution, the damping factor of the terminal-side bestresponse iteration is set to η = 0.3, with a convergence tolerance of 10−4 , while the finite-difference step for the server-side optimization is set to h = 10−2 . Fig. 8 shows the equilibrium results under different reward pools. As R increases, the equilibrium resource utilization gradually increases, indicating that stronger incentives encourage terminals to allocate more transmission resources to semantic uploading. This increased resource allocation improves the semantic fusion benefit, although the gain becomes progressively smaller due to the saturation effect. At the same time, the total system cost continuously increases because of the larger reward expenditure and the higher resource consumption of the terminals. Consequently, the server utility Us (R) first increases and then decreases, reaching its maximum at approximately R∗ = 4.91. When the reward pool is small, the incentive is insufficient to induce adequate resource allocation, resulting in limited fusion gain. When the reward pool becomes excessively large, the additional fusion benefit is no longer sufficient to compensate for the increased system cost. The optimal equilibrium therefore represents a balance among the incentive level, resource utilization, semantic fusion benefit, and system cost.

Fig. 9 (a) shows the semantic contribution of each terminal under the optimal strategy. The contributions differ noticeably among the five terminals, reflecting their heterogeneous semantic information, token assignments, and resource conditions. Terminal 4 provides the largest semantic contribution, while Terminal 1 provides the smallest. This differentiated contribution indicates that the value of a terminal is determined by the additional information it provides to the global fusion task rather than simply by the amount of information uploaded. Fig. 9 (b) further shows the resulting incentive allocated to each terminal. The incentive distribution follows the semantic contributions, with Terminal 4 receiving the largest incentive and Terminal 1 receiving the smallest. Meanwhile, P ∗ ∗ the total allocated incentive satisfies m∈M Rm ≈ R . The variations in individual incentives therefore reflect the different contributions of heterogeneous terminals to the global fusion task. This resulting differentiated allocation validates the ability of the proposed game mechanism to establish a contribution-based incentive strategy for semantic uploading under heterogeneous terminal conditions. 25 100 90 20 80 70 15

60 50

10

40 30 Server utility Fusion benefit Total system cost Equilibrium resource utilization Optimal decision

5

0 0

2

4

6

8

10

20

Equilibrium resource utilization (\%)

G. Performance of the Incentive Uploading Mechanism

Utility / benefit / cost

VSE++ [36] Concat MLP [34] GPO [34] HGNN [21] Slot Attention [22] HREM [18] Hyper-Slot

Efficiency

Seed 42

10 0 12

Fig. 8. Effect of the reward pool on server utility, fusion benefit, total system cost, and equilibrium resource utilization.

VI. Conclusion In this paper, we proposed a multi-source and multidimensional semantic information fusion framework for

13

4.5

1.5 3.99

1.33

4 3.65

3.5

1.22

3.21

1.07

3

1

2.5 2.15

2

0.72

1.73

0.58

1.5

0.5

1 0.5 0

0 1

2

3

4

5

(a) Semantic contribution

1

2

3

4

5

(b) Allocated reward

Fig. 9. Terminal contributions and reward allocation under the uploading strategies

resource and communication constrained edge networks. The framework coordinates selective semantic uploading among multiple terminals and fuses the resulting incomplete and heterogeneous tokens at the edge server into a compact, task-level representation, realized through a novel Hyper-Slot module for multi-dimensional token fusion together with a semantic uploading mechanism based on a bi-level Stackelberg–Nash game. With HyperSlot, tokens are assigned to latent slots according to both semantic content and multi-dimensional attributes through iterative token-to-slot assignment and slot-totoken feedback, allowing incomplete and variable-size token sets to be aggregated into a robust representation for global inference. The uploading mechanism evaluates terminal contributions through the Hyper-Slot assignment weights and establishes a contribution-based incentive strategy that aligns individual uploading decisions with global fusion benefit under resource constraints. HyperSlot outperforms the strongest evaluated baseline by 3.66 percentage points in mAP on a tri-modal extension of COCO2014 and by 15.9 points in RSum on Flickr30K, requiring less than one-fifth of the parameters of a naive concatenation-based approach and sustaining a 5.46percentage-point mAP advantage over token selection strategy under a constrained 25% transmission budget. References [1] G. Zheng, Q. Ni, K. Navaie, H. Pervaiz, G. Min, A. Kaushik, and C. Zarakovitis, “Mobility-aware split-federated with transfer learning for vehicular semantic communication networks,” IEEE Internet of Things Journal, vol. 11, no. 10, pp. 17237–17248, 2024. [2] G. Zheng, Q. Ni, K. Navaie, and H. Pervaiz, “Semantic communication in satellite-borne edge cloud network for computation offloading,” IEEE Journal on Selected Areas in Communications, vol. 42, no. 5, pp. 1145–1158, 2024. [3] G. Zhang, Q. Hu, Z. Qin, Y. Cai, G. Yu, and X. Tao, “A unified multi-task semantic communication system for multimodal data,” IEEE Transactions on Communications, vol. 72, no. 7, pp. 4101–4116, 2024. [4] H. Xie, Z. Qin, X. Tao, and K. B. Letaief, “Task-oriented multiuser semantic communications,” IEEE Journal on Selected Areas in Communications, vol. 40, no. 9, pp. 2584–2597, 2022. [5] B. Khaleghi, A. Khamis, F. O. Karray, and S. N. Razavi, “Multisensor data fusion: A review of the state-of-the-art,” Information fusion, vol. 14, no. 1, pp. 28–44, 2013. [6] Z. Zhou, X. Chen, E. Li, L. Zeng, K. Luo, and J. Zhang, “Edge intelligence: Paving the last mile of artificial intelligence with

edge computing,” Proceedings of the IEEE, vol. 107, no. 8, pp. 1738–1762, 2019. [7] Y. Shi, K. Yang, T. Jiang, J. Zhang, and K. B. Letaief, “Communication-efficient edge ai: Algorithms and systems,” IEEE communications surveys & tutorials, vol. 22, no. 4, pp. 2167–2191, 2020. [8] J. Shao, Y. Mao, and J. Zhang, “Task-oriented communication for multidevice cooperative edge inference,” IEEE Transactions on Wireless Communications, vol. 22, no. 1, pp. 73–87, 2022. [9] Y. Hu, J. Peng, S. Liu, J. Ge, S. Liu, and S. Chen, “Communication-efficient collaborative perception via information filling with codebook,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 15481–15490, 2024. [10] J. Xu, Y. Zhang, Z. Cai, and D. Huang, “Cosdh: communicationefficient collaborative perception via supply-demand awareness and intermediate-late hybridization,” in Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 6834–6843, 2025. [11] H. Xie, Z. Qin, G. Y. Li, and B.-H. Juang, “Deep learning enabled semantic communication systems,” IEEE transactions on signal processing, vol. 69, pp. 2663–2675, 2021. [12] H. Xie and Z. Qin, “A lite distributed semantic communication system for internet of things,” IEEE Journal on Selected Areas in Communications, vol. 39, no. 1, pp. 142–153, 2020. [13] E. Bourtsoulatze, D. B. Kurka, and D. Gündüz, “Deep joint source-channel coding for wireless image transmission,” IEEE Transactions on Cognitive Communications and Networking, vol. 5, no. 3, pp. 567–579, 2019. [14] M. Jankowski, D. Gündüz, and K. Mikolajczyk, “Wireless image retrieval at the edge,” IEEE Journal on Selected Areas in Communications, vol. 39, no. 1, pp. 89–100, 2020. [15] Y. Wang, S. Han, X. Xu, H. Liang, R. Meng, C. Dong, and P. Zhang, “Feature importance-aware task-oriented semantic transmission and optimization,” IEEE Transactions on Cognitive Communications and Networking, vol. 10, no. 4, pp. 1175– 1189, 2024. [16] L. Yan, Z. Qin, C. Li, R. Zhang, Y. Li, and X. Tao, “Qoe-based semantic-aware resource allocation for multi-task networks,” IEEE Transactions on Wireless Communications, vol. 23, no. 9, pp. 11958–11971, 2024. [17] K.-H. Lee, X. Chen, G. Hua, H. Hu, and X. He, “Stacked cross attention for image-text matching,” in Proceedings of the European conference on computer vision (ECCV), pp. 201–216, 2018. [18] Z. Fu, Z. Mao, Y. Song, and Y. Zhang, “Learning semantic relationship among instances for image-text matching,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 15159–15168, 2023. [19] J. Lee, Y. Lee, J. Kim, A. Kosiorek, S. Choi, and Y. W. Teh, “Set transformer: A framework for attention-based permutationinvariant neural networks,” in International conference on machine learning, pp. 3744–3753, PMLR, 2019. [20] Z. Tang, Y. Liu, Y. Sun, Y. Gao, J. Chen, R. Xu, and S. Liu, “Cost: efficient collaborative perception from unified spatiotemporal perspective,” in Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 1120–1129, 2025. [21] Y. Feng, H. You, Z. Zhang, R. Ji, and Y. Gao, “Hypergraph neural networks,” in Proceedings of the AAAI conference on artificial intelligence, vol. 33, pp. 3558–3565, 2019. [22] F. Locatello, D. Weissenborn, T. Unterthiner, A. Mahendran, G. Heigold, J. Uszkoreit, A. Dosovitskiy, and T. Kipf, “Objectcentric learning with slot attention,” Advances in neural information processing systems, vol. 33, pp. 11525–11538, 2020. [23] Y. Mao, C. You, J. Zhang, K. Huang, and K. B. Letaief, “A survey on mobile edge computing: The communication perspective,” IEEE communications surveys & tutorials, vol. 19, no. 4, pp. 2322–2358, 2017. [24] A. Martins and R. Astudillo, “From softmax to sparsemax: A sparse model of attention and multi-label classification,” in International conference on machine learning, pp. 1614–1623, PMLR, 2016. [25] K. Cho, B. Van Merriënboer, Ç. Gulçehre, D. Bahdanau, F. Bougares, H. Schwenk, and Y. Bengio, “Learning phrase representations using rnn encoder–decoder for statistical machine translation,” in Proceedings of the 2014 conference on empirical

14

methods in natural language processing (EMNLP), pp. 1724– 1734, 2014. [26] J. L. Ba, J. R. Kiros, and G. E. Hinton, “Layer normalization,” arXiv preprint arXiv:1607.06450, 2016. [27] T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick, “Microsoft coco: Common objects in context,” in European conference on computer vision, pp. 740–755, Springer, 2014. [28] W.-N. Hsu, D. Harwath, T. Miller, C. Song, and J. Glass, “Textfree image-to-speech synthesis using learned segmental units,” in Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 5284–5300, 2021. [29] P. Anderson, X. He, C. Buehler, D. Teney, M. Johnson, S. Gould, and L. Zhang, “Bottom-up and top-down attention for image captioning and visual question answering,” in 2018 IEEE/CVF conference on computer vision and pattern recognition, pp. 6077–6086, IEEE, 2018. [30] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” in Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers), pp. 4171–4186, 2019. [31] M. McAuliffe, M. Socolof, S. Mihuc, M. Wagner, and M. Sonderegger, “Montreal forced aligner: Trainable text-speech alignment using kaldi,” in Proc. Interspeech 2017, pp. 498–502, 2017. [32] W.-N. Hsu, B. Bolte, Y.-H. H. Tsai, K. Lakhotia, R. Salakhutdinov, and A. Mohamed, “Hubert: Self-supervised speech representation learning by masked prediction of hidden units,” IEEE/ACM transactions on audio, speech, and language processing, vol. 29, pp. 3451–3460, 2021. [33] S. Zhao, S. Wang, M. Soleymani, D. Joshi, and Q. Ji, “Affective computing for large-scale heterogeneous multimedia data: A survey,” ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM), vol. 15, no. 3s, pp. 1– 32, 2019. [34] J. Chen, H. Hu, H. Wu, Y. Jiang, and C. Wang, “Learning the best pooling strategy for visual semantic embedding,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 15789–15798, 2021. [35] P. Young, A. Lai, M. Hodosh, and J. Hockenmaier, “From image descriptions to visual denotations: New similarity metrics for semantic inference over event descriptions,” Transactions of the association for computational linguistics, vol. 2, pp. 67–78, 2014. [36] F. Faghri, D. J. Fleet, J. R. Kiros, and S. Fidler, “Vse++: Improving visual-semantic embeddings with hard negatives,” arXiv preprint arXiv:1707.05612, 2017.

Related documents

• Multi-Source and Multi-Dimensional Information Fusion in Resource-Constrained Edge Networks#1297650
• EMMI: Edge Multi-Modal Intelligence for Communication-Efficient MLLM Inference via Fused Representation Compression#998618
• AirTF: Over-the-Air Token Fusion for Task-Oriented Multi-Modal Token Communications#785192
• AirTF: Over-the-Air Token Fusion for Task-Oriented Multi-Modal Token Communications#783602
• Secure Joint Source-Channel Coding of Multimodal Semantic Sources#1327391