1
EMMI: Edge Multi-Modal Intelligence for Communication-Efficient MLLM Inference via Fused Representation Compression Motahare Mounesan and Irfan Khan Texas A&M University {motahare, irfankhan}@tamu.edu
arXiv:2609.11058v1 [cs.LG] 10 Sep 2026
Abstract Recent advances in multimodal large language models (MLLMs) have opened new opportunities for edge intelligence by enabling reasoning across heterogeneous sensor modalities, such as vision, text, and telemetry data. However, deploying these capabilities on resource-constrained edge platforms remains challenging due to the substantial computational, memory, and communication demands of modern MLLMs. Rather than transmitting raw sensor observations or partitioning neural networks at intermediate layers, Edge Multi-Modal Intelligence (EMMI) communicates a compact representation between edge devices and server resources, enabling communication-efficient edge MLLM inference. To achieve this, EMMI performs modality-specific encoding, cross-modal representation fusion, and learned compression at the edge, transmitting only a compact latent representation to server-side resources for high-capacity MLLM reasoning. This representation-centric design reduces communication overhead, preserves local data privacy, and provides a fixed-size interface between heterogeneous edge devices and server-side MLLMs. Evaluation on a representative multimodal benchmark demonstrates that EMMI can reduce the communication payload by 32× while maintaining comparable downstream accuracy, resulting in up to a 3.4× reduction in estimated end-to-end inference latency under bandwidth-constrained edge conditions. Index Terms Edge Intelligence, Multimodal Large Language Models (MLLMs), Representation Compression, Edge–Server Inference
I. I NTRODUCTION As edge systems become increasingly autonomous, their effectiveness depends on the ability to reason across heterogeneous modalities under strict resource constraints. In many operational environments, actionable insight requires jointly interpreting visual observations, sensor measurements, telemetry, and contextual data. This reliance on multi-modal reasoning has created growing interest in intelligent systems capable of integrating diverse information sources to support timely and context-aware decision-making at the edge. Recent advances in Multimodal Large Language Models (MLLMs) provide a promising foundation for achieving such capabilities by enabling unified processing of heterogeneous inputs. Unlike conventional lightweight multi-modal models designed for specific tasks, MLLMs can capture higher-order relationships among heterogeneous inputs and perform contextual cross-modal reasoning within a unified architecture. These capabilities enable more flexible and generalizable decision-making across complex environments. However, their large computational, memory, and communication requirements create a significant deployment gap between capabilities of modern MLLMs and the resources available on edge platforms. To enable MLLM capabilities on edge platforms, existing deployment strategies have explored reducing model complexity, utilizing remote compute resources, and minimizing communication costs. However, these approaches address individual resource constraints while leaving the fundamental challenge of efficient multi-modal reasoning under edge limitations unresolved. Reducing model size can limit the expressive capability of MLLMs, while offloading computation or transmitting intermediate representations can introduce communication overhead and latency. Therefore, enabling practical MLLM-based intelligence at the edge requires a deployment strategy that preserves task-relevant multimodal information while adapting computation and communication to edge constraints. Bridging this deployment gap requires careful coordination between computation and communication across edge and server resources. Direct offloading of multi-modal inputs can reduce local processing requirements but introduces significant communication overhead when transmitting high-volume sensor streams. On-device optimization techniques reduce resource consumption but often sacrifice the reasoning capability and generality that make MLLMs valuable. Existing split-computing approaches distribute model execution across edge and server devices; however, they may still require frequent exchanges of intermediate representations during inference, limiting their effectiveness under bandwidth and latency constraints. These limitations indicate that efficient edge deployment of MLLMs requires more than reducing model size or shifting computation, but rather a strategy that preserves cross-modal reasoning capability while transforming how multi-modal information is represented and communicated between edge devices and remote resources. Identify applicable funding agency here. If none, delete this.
To address these challenges, we propose EMMI, an Edge Multi-Modal Intelligence architecture that decouples multi-modal representation generation from high-capacity MLLM reasoning across edge and server resources. Instead of transmitting raw sensor streams or relying on conventional split execution with frequent intermediate feature exchanges, EMMI performs modality-specific encoding and cross-modal fusion at the edge before communication, generating a unified multimodal representation that is subsequently compressed and transmitted to server resources. By performing multimodal interaction before transmission, EMMI changes the communication boundary from raw inputs, modality-specific features, or model activations to a compact unified representation for server-side reasoning. This representation-centric design reduces communication overhead while preserving task-relevant information and providing a fixed-size interface between heterogeneous edge devices and serverside reasoning models. The remainder of this paper is organized as follows. Section II presents related work on MLLMs and edge intelligence. Section III describes the proposed EMMI framework and system architecture. Section IV presents the experimental methodology and evaluation setup. Section V discusses the results and analysis. Finally, Section VI concludes the paper. II. RELATED WORK Multimodal Intelligence under Edge Constraints. Traditional multimodal systems typically employ lightweight, task-specific architectures for applications such as human activity recognition [1], [2] and onboard aerial perception [3]. Recent multimodal large language models (MLLMs), including Flamingo [4], BLIP-2 [5], and LLaVA [6], instead support general-purpose reasoning across heterogeneous inputs. However, their computational and memory requirements limit deployment on resourceconstrained edge platforms [7], [8]. Recent compact MLLMs target this gap by reducing model size and computational requirements [7], [9], but remain constrained relative to full-capacity MLLMs [9], [10]. These limitations motivate approaches that preserve high-capacity multimodal reasoning while adapting computation and communication to edge constraints. On-Device MLLM Optimization. Existing efforts to enable MLLM deployment at the edge reduce model computation and memory through three complementary directions: hardware acceleration, using specialized architectures such as NPUs and processing-in-memory designs [11]; model compression, including quantization [12]; and efficient MLLM architectures, which redesign models for smaller footprints and improved deployability [13]–[15]. While these approaches improve edge efficiency, they remain constrained by the trade-off between model efficiency and high-capacity multimodal reasoning [9], [10]. EMMI instead preserves high-capacity MLLM reasoning while shifting the communication boundary toward compact multimodal representations generated at the edge. Split Computing and Collaborative Inference. Split computing enables edge–server collaborative inference by partitioning model execution across resource-constrained devices and remote servers [16]–[21]. While early approaches focused on reducing communication overhead for deep neural network inference through efficient partitioning and activation transfer [22], recent efforts have extended collaborative inference to large language models and multimodal systems. LLM inference frameworks explore adaptive partitioning, speculative execution, and collaborative serving to reduce the cost of model execution across heterogeneous resources [23]–[25]. Multimodal systems further distribute modality processing and language inference across edge–cloud resources [26]. However, existing collaborative inference approaches primarily optimize where computation is performed, while communication remains tied to intermediate activations, modality-specific features, or model states. These representations remain dependent on the underlying model architecture and input modalities, limiting scalability as multimodal systems incorporate increasingly diverse sensor streams. EMMI instead changes what crosses the edge–server boundary by generating and compressing a unified multimodal representation at the edge and transmitting a compact latent representation for server-side MLLM reasoning. Communication-Efficient Representation Transfer. Beyond computation placement, edge MLLM deployment faces a communication challenge: intermediate representations exchanged between the edge and server can still introduce substantial transmission overhead. Existing approaches reduce this overhead through representation compression and semantic-aware transmission. Feature compression methods such as BottleNet++ [27] compress intermediate activations, while task-oriented communication frameworks learn compact representations relevant to downstream objectives [28], [29]. Learned compression techniques further improve transmission efficiency using neural compression models [30]. Recent work extends communication-efficient inference to multimodal LLMs. TOFC [31] reduces visual feature transmission through feature merging and entropy modeling in a deviceedge co-inference framework. However, such approaches typically compress modality-specific representations and perform multimodal interaction after transmission, limiting their ability to exploit cross-modal dependencies at the edge. EMMI extends these efforts by changing the transmitted representation itself. Instead of sending raw sensor inputs or separate modality-specific features, EMMI performs cross-modal fusion at the edge and generates a unified latent representation for server-side MLLM reasoning. This design provides a fixed-size communication interface between heterogeneous edge sensors and high-capacity MLLMs while preserving the flexibility required for open-ended multimodal reasoning. III. S YSTEM M ODEL AND S OLUTION A PPROACH In this section, we introduce the edge–server system model and the design of EMMI for communication-efficient MLLM inference under edge constraints. 2
Fig. 1. EMMI pipeline: encode, fuse, and compress at the edge; decompress, and reason at the server.
A. System Model We consider a distributed edge–server multimodal intelligence system consisting of one or more resource-constrained edge platforms connected through bandwidth-limited wireless links to remote server infrastructure equipped with high-capacity multimodal large language models (MLLMs). Edge platforms collect and perform local processing on heterogeneous sensor observations, while server-side resources provide computationally intensive multimodal reasoning. Let V = {v1 , v2 , . . . , vN } denote a set of edge devices, where N represents the number of participating edge platforms. Each edge device collects observations from multiple modalities at time step t, represented as: Xit = {xti,1 , xti,2 , . . . , xti,Mi },
(1)
where Mi denotes the number of available modalities at edge device i and xti,m represents the observation from modality m. The system aims to obtain MLLM-based reasoning results from heterogeneous observations collected at edge devices. Since high-capacity MLLMs require substantial computational and memory resources, inference is performed collaboratively between edge platforms and remote servers. This requires transferring information from edge observations through wireless links for server-side MLLM processing. Accordingly, end-to-end inference latency consists of edge processing (Tedge ), communication (Tcomm ), and server-side reasoning (Tserver ): Ttotal = Tedge + Tcomm + Tserver ,
(2)
The communication latency depends on the transmitted data size Stx and available bandwidth B: Stx , (3) B Therefore, the system must balance the computational workload assigned to edge and server resources while minimizing communication overhead under edge resource constraints. Tcomm =
B. EMMI Framework To enable efficient MLLM inference under the system model above, EMMI adopts a representation-centric edge–server architecture in which edge devices generate compact multimodal representations for server-side reasoning, rather than transmitting raw sensor streams or architecture-dependent activations. This design separates resource-constrained edge processing from compute-intensive MLLM reasoning while reducing communication overhead and exposure of raw sensor data. Figure 1 illustrates the overall architecture. I) Edge-side Multimodal Representation Generation. The edge pipeline consists of three components: encoding, cross-modal fusion, and representation compression. Together, they generate the multimodal representation that serves as the communication interface between edge and server resources. • Encoding: Each modality observation is processed independently through a modality-specific encoder em (·), transforming raw observations into intermediate feature representations: hti,m = em (xti,m ),
(4)
where xti,m denotes observation from modality m at edge device i at time t, and hti,m denotes the resulting representation. • Cross-modal Fusion: The modality feature representations are combined through a cross-modal fusion operation to construct a unified multimodal representation: fit = A(hti,1 , hti,2 , . . . , hti,Mi ), (5) 3
where A(·) denotes cross-modal fusion operation and fit represents resulting multimodal feature representation. By performing multimodal interaction before communication, EMMI captures relationships among heterogeneous inputs rather than transmitting independent modality-specific features. This enables the transmitted representation to retain cross-modal information relevant to downstream MLLM reasoning. • Representation Compression: The fused multimodal representation (fit ) is transformed into a compact representation: zit = gϕ (fit ),
(6)
where gϕ (·) represents edge-side compression module, and zit denotes the transmitted representation. The compression reduces representation dimensionality while preserving information relevant to downstream MLLM reasoning, thereby reducing communication overhead and limiting exposure of raw sensor observations. II) Server-side MLLM Reasoning. The server pipeline consists of two components: representation decompression and MLLM reasoning. They transform the transmitted compact representation into the final task-specific inference result. • Representation Decompression: The server reconstructs the multimodal representation (zit ) from the received compact representation using a server-side decompression module: fˆit = dψ (zit ),
(7)
where dψ (·) denotes the server-side decompression module and fˆit denotes the recovered multimodal representation. • MLLM Reasoning: The recovered multimodal representation (fˆit ) is projected into the MLLM embedding space through a server-side projection module: uti = pθ (fˆit ),
(8)
where pθ (·) denotes the server-side projection module and uti denotes the resulting MLLM-compatible representation. The MLLM processes uti for multimodal reasoning, followed by a task-specific prediction head to generate the final inference result. The projection module and prediction head are configured for the downstream task, while edge-side representation generation remains unchanged. Thus, MLLM inference can be performed without access to the original sensor observations or modality-specific intermediate features. This representation-centric design provides two system-level benefits. First, EMMI keeps raw sensor observations local to the edge by transmitting only compact multimodal representations instead of raw sensor data over the edge–server link, reducing the exposure of raw sensor observations while preserving information required for server-side reasoning. Second, although the compressed representation could be directly connected to the MLLM reasoning module, EMMI decompresses the received representation into the original fused representation space before MLLM inference. This decouples the compression mechanism from the downstream reasoning module, allowing different compression techniques or compression ratios to be adopted without modifying the reasoning module itself. Thus, the compression scheme can be changed without requiring retraining of the server-side reasoning module. The reconstructed representations can also be efficiently batched for server-side MLLM processing, enabling parallel inference and amortizing the computational cost of MLLM reasoning across requests. C. Proposed Method This section describes the key design choices that enable communication-efficient multimodal inference in EMMI, focusing on cross-modal fusion before transmission and compression of the resulting representation under edge communication constraints. Cross-Modal Fusion. The representation exchanged between edge and server determines the information available for downstream MLLM reasoning. EMMI performs cross-modal fusion before compression to construct a unified representation that captures both modality-specific characteristics and cross-modal interactions, reducing the need to transmit separate modality representations to the server. EMMI instantiates the fusion function A(·) using pairwise interaction-based fusion, which combines modality-specific representations with explicit cross-modal interaction features. For two modalities, the fused representation is formulated as: fit = hti,m ; hti,n ; hti,m − hti,n ; hti,m ⊙ hti,n , (9) where | · | denotes element-wise absolute difference and ⊙ denotes element-wise multiplication. The concatenated modality features retain individual modality information, while the difference and multiplicative terms capture complementary and correlated characteristics between modalities. For more than two modalities, the same pairwise interaction operation is applied to each modality pair, and the resulting pairwise features are aggregated to form the unified multimodal representation. Representation Compression. EMMI considers two compression training strategies based on the availability of downstream task supervision. When task labels are unavailable or a reusable compressed representation is desired, a task-agnostic model learns a general-purpose multimodal representation without relying on a specific inference objective. This decouples the edgeside compression module from downstream applications, allowing the same compressed representation to be reused across tasks without task-specific retraining. When task labels are available, a task-aware model incorporates supervision to optimize the compressed representation for the target inference objective while maintaining the same lightweight deployment pipeline. 4
Both strategies operate on the same fused multimodal representation and share the same edge-side compression architecture, differing only in their training objectives. For task-agnostic compression, EMMI jointly optimizes reconstruction fidelity and cross-modal alignment: LTA = LMSE (dψ (z), f ) +λ LInfoNCE (z, {hm }M m=1 ), {z } | | {z }
(10)
cross-modal alignment
reconstruction
where LMSE denotes the mean squared error reconstruction loss and LInfoNCE denotes the contrastive alignment objective [32]. The reconstruction term minimizes distortion between the fused representation and its reconstruction, encouraging the latent space to retain multimodal information. The cross-modal alignment term encourages the compressed representation to remain aligned with the individual modality embeddings without downstream task labels. Together, these objectives learn a task-independent latent representation that preserves both multimodal information and cross-modal structure. When downstream task labels are available, EMMI uses task-aware compression to optimize the compressed representation for the target inference objective by replacing the task-independent alignment objective with supervised contrastive learning: LTC = LMSE (dψ (z), f ) +λ LSupCon (zi , yi ) , {z } | | {z } reconstruction
(11)
task-specific alignment
where yi denotes the downstream task label. The reconstruction term preserves information from the fused representation, while the supervised contrastive objective shapes the compressed latent space according to the target task by encouraging samples with similar labels to become closer and samples with different labels to separate. Task supervision is introduced only through the contrastive objective during training, preserving the lightweight edge-side compression and transmission pipeline while producing a representation optimized for the downstream inference objective. IV. E VALUATION We evaluate EMMI by first examining how effectively its compression strategies preserve multimodal information, and then assessing the resulting communication and inference efficiency. Specifically, we compare the task performance of compressed representations across methods and measure communication overhead and end-to-end latency under constrained network conditions. A. Experimental Setup Dataset and Multimodal Encoders. We use MS-COCO image-caption pairs [33] as a representative vision-language benchmark to evaluate EMMI’s multimodal representation and compression capabilities. The dataset is used for image-text matching. We evaluate two vision-language encoders representing different efficiency regimes: CLIP ViT-B/32 and its distilled lightweight variant MobileCLIP2-S0. CLIP ViT-B/32 uses an 87M-parameter image encoder, while MobileCLIP2-S0 reduces the encoder size to 11.4M parameters. Both encoders produce 512-dimensional embeddings, enabling comparison between full-size and lightweight vision-language models under communication-efficient compression. Compression Configuration. All evaluated compression approaches target a 64-dimensional latent representation. Server-side Inference. EMMI uses a frozen LLaVA-1.5-7B language-model backbone (6.6B parameters) as the server-side language-model backbone. A lightweight trainable two-layer MLP maps the received edge representation to eight soft-prompt tokens in LLaVA’s embedding space. The frozen LLaVA backbone processes these tokens, and its final hidden representation is passed to a binary classification head. Baselines. We compare the proposed compression strategies against an uncompressed representation transfer baseline and standard dimensionality reduction and representation compression methods. The uncompressed baseline directly transmits the original fused multimodal representation and provides a reference for performance without compression. Additional baselines include principal component analysis (PCA) [34], partitioned PCA (BlockPCA) applying PCA independently to separate blocks of the fused representation, linear discriminant analysis (LDA) [35], and variational autoencoder (VAE) [36]. Training Protocol. Training follows a two-stage decoupled protocol. In the first stage, the edge-side compression module is trained independently of the server-side inference module using either task-agnostic or task-aware supervision, depending on the compression strategy. The trained compression module is then used to generate latent representations for the training, validation, and test sets. In the second stage, the server-side projection head and classification head are trained using the cached latent representations, while the edge-side compression module remains frozen. The pretrained vision-language encoders and LLaVA backbone remain frozen throughout all experiments. We use a 72K/5K/5K train/validation/test split, with model selection based on validation loss. Evaluation Metrics. We evaluate EMMI using four metrics. Task accuracy is the percentage of correctly classified image-text pairs on the test set. Compression ratio is the ratio between the dimensionality of the original representation and the compressed representation. Communication payload is the number of bytes required to transmit the compressed representation between edge and server. End-to-end latency is the total inference time including edge-side representation generation, compression, transmission, and server-side inference. 5
Method No Compression
Supervision
CLIP
MobileCLIP
—
97.92
98.41
Conventional Compression 96.88 93.40 96.75 97.12 Labels 97.78
57.35 52.01 69.87 97.85 98.33
Proposed EMMI Compression TaskAgnosticAE Pairs 94.09 TaskAwareAE Labels 98.08
90.30 98.32
PCA AE VAE BlockPCA LDA
TABLE I TASK PERFORMANCE AFTER 32× COMPRESSION (d = 64).
Fig. 2. Estimated end-to-end latency breakdown for MobileCLIP2-S0 under a bandwidth-constrained IoT scenario (0.1 Mbps).
Evaluation Environment. We evaluate EMMI from both machine learning and system perspectives. Accuracy and compression effectiveness experiments, as well as server-side inference latency, are conducted on an NVIDIA Quadro RTX 6000 (24GB) to enable controlled comparison across models and compression approaches. Edge-side computational overhead is evaluated separately using a single CPU thread on an x86 64 compute node, providing a reproducible proxy for resource-constrained execution. B. Results and Discussion We next present the experimental results and discuss the implications of EMMI’s compression strategies for multimodal inference and edge–server efficiency. Compression Performance. We first evaluate whether EMMI preserves task-relevant information under aggressive representation compression. All compression methods reduce the fused multimodal representation to a 64-dimensional latent representation, reducing the communication payload from 8,192B to 256B (32× reduction). We compare conventional representation compression methods against the two proposed EMMI strategies: the Task-Agnostic Multimodal Autoencoder (TaskAgnosticAE)), which learns a reusable latent representation without downstream task labels, and the Task-Aware Supervised Contrastive Autoencoder (TaskAwareAE), which incorporates task supervision during training to optimize the compressed representation for the downstream inference task. Table I shows that compression performance varies substantially across representations and compression objectives. For CLIP, several conventional methods retain high task performance, whereas MobileCLIP is much more sensitive to the choice of compression method: PCA, AE, and VAE reduce accuracy by 41.06, 46.4, and 28.5 percentage points, respectively, while BlockPCA and LDA remain near the uncompressed baseline. This variation indicates that the effectiveness of a fixed compression level depends strongly on the underlying representation and compression objective, highlighting the challenge of preserving task-relevant information across different multimodal representations. The EMMI results show that learned compression can preserve task-relevant information across different multimodal representations. The TaskAgnosticAE achieves 94.09% accuracy for CLIP and 90.30% for MobileCLIP without using task labels during training, enabling a reusable compressed representation across downstream applications. However, BlockPCA—a closed-form method that also requires no task labels and no training—achieves 97.12% for CLIP and 97.85% for MobileCLIP, indicating that unsupervised, structure-aware compression can outperform TaskAgnosticAE on this task. The TaskAwareAE incorporates task supervision through supervised contrastive learning and achieves 98.08% accuracy for CLIP and 98.32% for MobileCLIP, matching the uncompressed baselines within 0.16 percentage points while reducing the communication payload 6
by 32×. Notably, TaskAwareAE and LDA—a closed-form supervised baseline achieving 97.78%/98.33%—both substantially outperform generic compression methods (PCA, AE, and VAE) on MobileCLIP, where these methods suffer substantial degradation. Overall, these results indicate that aggressive representation compression is not determined solely by dimensionality reduction. The compression objective should account for the structure and task relevance of the multimodal representation, particularly for compact encoders where generic compression can discard information critical to downstream inference. Representation and Communication Efficiency. We next evaluate the system-level impact of EMMI’s representation-centric design, focusing on representation-processing overhead, communication cost, and end-to-end latency. All edge-side latency measurements are wall-clock measurements over 200 runs with 30 warm-up iterations using a single CPU thread on an x86 64 compute node, providing a reproducible proxy for resource-constrained execution. Figure 2 illustrates the resulting system-level effect under a representative bandwidth-constrained IoT scenario (0.1Mbps). Without compression, network transmission is the dominant component of end-to-end latency. EMMI reduces the transmitted representation from 8,192B to 256B, reducing transmission latency from 655.4ms to 20.5ms while adding only 0.641ms of compression overhead. Consequently, under this IoT bandwidth condition, the dominant bottleneck shifts away from communication toward computation, reducing the estimated end-to-end latency from approximately 903ms to 269ms, a 3.4× improvement. Component
Latency (ms)
Cross-modal fusion (2048-dim) PCA AE VAE BlockPCA LDA TaskAgnosticAE / TaskAwareAE
0.013 ± 0.000 0.011 ± 0.001 0.577 ± 0.004 0.187 ± 0.003 0.014 ± 0.001 0.035 ± 0.001 0.641 ± 0.005
TABLE II R EPRESENTATION - PROCESSING LATENCY ( MS ), MEAN ± STD .
Table II shows that representation processing adds negligible edge-side overhead. Cross-modal fusion requires only 0.013ms, while all evaluated compression methods require less than 1ms, with AE-based compression requiring the most, at 0.641ms. Thus, EMMI does not introduce a new computational bottleneck at the edge. For comparison, modality encoding accounts for over 99% of edge-side computation, making the sub-millisecond representation-processing overhead negligible in the overall edge pipeline. The extent of this benefit depends on network bandwidth. Table III quantifies transmission latency for different representation sizes across representative network conditions. At 0.1Mbps, the 32× payload reduction saves approximately 635ms per inference. The corresponding savings decrease to 6.4ms at 10Mbps LTE, 1.26ms at 50Mbps WiFi, and 0.68ms at 100Mbps 5G. Payload
IoT 0.1Mbps
Uncompressed (8192B) Compressed, 64-dim (256B)
655.4 20.5 TABLE III
LTE 10Mbps
WiFi 50Mbps
5G 100Mbps
6.6 0.2
1.3 0.04
0.7 0.02
T RANSMISSION LATENCY ACROSS REPRESENTATIVE LINK BANDWIDTHS .
These results reveal a clear operating regime for representation-centric edge–server inference. When communication is the bottleneck, EMMI converts a large communication cost into a negligible compression cost, substantially reducing end-to-end latency without degrading task performance. As bandwidth increases, transmission becomes less significant and the latency advantage correspondingly diminishes; compression then primarily provides communication savings and a compact interface between edge and server. This demonstrates that EMMI’s benefit is not simply smaller representations, but the ability to shift the system away from communication-bound inference while preserving the information required for downstream reasoning. V. C ONCLUSION This paper presented EMMI, an edge–server framework for MLLM inference under resource and communication constraints. EMMI performs modality-specific encoding, cross-modal fusion, and learned compression at the edge, transmitting compact representations for server-side reasoning. By shifting the communication boundary from raw inputs and intermediate features to compressed representations, EMMI reduces communication overhead while preserving task-relevant information. The task-aware compression strategy uses downstream task supervision to preserve task-relevant information, maintaining performance within 7
0.16 percentage points of the uncompressed baselines while reducing the communication payload by 32×. Under bandwidthconstrained IoT conditions, this reduction yields a 3.4× improvement in estimated end-to-end latency. These results demonstrate the importance of jointly designing multimodal representations and compression for efficient edge MLLM inference. Future work will evaluate EMMI on edge hardware, extend it to more complex modalities and heterogeneous devices, and address scenarios with non-aligned modality encoders. R EFERENCES [1] S. Chung et al., “Sensor data acquisition and multimodal sensor fusion for human activity recognition using deep learning,” Sensors, vol. 19, no. 7, p. 1716, 2019. [2] M. M. Islam, S. Nooruddin, F. Karray, and G. Muhammad, “Multi-level feature fusion for multimodal human activity recognition in internet of healthcare things,” Information Fusion, vol. 94, pp. 17–31, 2023. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S1566253523000246 [3] L. Rey, A. M. Bernardos, A. D. Dobrzycki, D. Carramiñana, L. Bergesio, J. A. Besada, and J. R. Casar, “A performance analysis of you only look once models for deployment on constrained computational edge devices in drone applications,” Electronics, vol. 14, no. 3, p. 638, 2025. [4] J.-B. Alayrac, J. Donahue, P. Luc, A. Miech, T. Dinev, J. Verbeek, A. Kuncoro et al., “Flamingo: a visual language model for few-shot learning,” in Advances in Neural Information Processing Systems (NeurIPS), vol. 35, 2022. [5] J. Li, D. Li, S. Savarese, and S. C. H. Hoi, “Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models,” in International Conference on Machine Learning (ICML). PMLR, 2023. [6] H. Liu, C. Li, Q. Wu, and Y. J. Lee, “Visual instruction tuning,” in Advances in Neural Information Processing Systems (NeurIPS), 2023. [7] Y. Yao et al., “Efficient gpt-4v level multimodal large language model for deployment on edge devices,” Nature Communications, vol. 16, p. 5509, 2025. [8] Z. Lin et al., “Pushing large language models to the 6g edge: Vision, challenges, and opportunities,” IEEE Communications Magazine, 2025, published version. [9] Y. Jin, J. Li, Y. Liu, T. Gu, K. Wu, Z. Jiang, M. He, B. Zhao, X. Tan, Z. Gan, Y. Wang, C. Wang, and L. Ma, “Efficient multimodal large language models: A survey,” Visual Intelligence, vol. 3, no. 1, 2025. [10] G. Qu, Q. Chen, W. Wei, Z. Lin, X. Chen, and K. Huang, “Mobile edge intelligence for large language models: A contemporary survey,” IEEE Communications Surveys & Tutorials, vol. 27, no. 6, 2025. [11] Y. Chen, C. Fang, X. Dai, Y. Wu, T. Tambe, M. Verhelst, and M. S. Abdelfattah, “P3-llm: An integrated npu-pim accelerator for edge llm inference using hybrid numerical formats,” in Proceedings of the IEEE/ACM International Symposium on Computer Architecture (ISCA), 2026. [12] J. Lin, J. Tang, H. Tang, S. Yang, X. Dang, and S. Han, “AWQ: Activation-aware weight quantization for on-device LLM compression and acceleration,” in Proceedings of the 7th Annual Conference on Machine Learning and Systems (MLSys), 2024. [13] X. Chu et al., “Mobilevlm: A fast, strong and open vision language assistant for mobile devices,” arXiv preprint arXiv:2312.16886, 2024. [14] B. Zhou, Y. Hu, X. Weng, J. Jia, J. Luo, X. Liu, J. Wu, and L. Huang, “Tinyllava: A framework of small-scale large multimodal models,” arXiv preprint arXiv:2402.14289, 2024. [15] Y. Yao, T. Yu, A. Zhang, C. Wang, J. Cui, H. Zhu, T. Cai, H. Li, W. Zhao, Z. He, Q. Chen, H. Zhou, Z. Zou, H. Zhang, S. Hu, Z. Zheng, J. Zhou, J. Cai, X. Han, G. Zeng, D. Li, Z. Liu, and M. Sun, “Minicpm-v: A gpt-4v level mllm on your phone,” arXiv preprint arXiv:2408.01800, 2024. [16] Y. Kang, J. Hauswald, C. Gao, A. Rovinski, T. Mudge, J. Mars, and L. Tang, “Neurosurgeon: Collaborative intelligence between the cloud and mobile edge,” in Proceedings of the 22nd ACM International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS), 2017, pp. 615–629. [17] E. Li, L. Zeng, Z. Zhou, and X. Chen, “Edge AI: On-demand accelerating deep neural network inference via edge computing,” IEEE Transactions on Wireless Communications, vol. 19, no. 1, pp. 447–457, 2020. [18] X. Zhang, M. Mounesan, and S. Debroy, “Effect-dnn: Energy-efficient edge framework for real-time dnn inference,” in 2023 IEEE 24th International Symposium on a World of Wireless, Mobile and Multimedia Networks (WoWMoM), 2023, pp. 10–20. [19] M. Mounesan, X. Zhang, and S. Debroy, “Edgerl: Reinforcement learning-driven deep learning model inference optimization at edge,” in 2024 20th International Conference on Network and Service Management (CNSM), 2024, pp. 1–5. [20] ——, “Infer-edge: Dynamic dnn inference optimization in just-in-time edge-ai implementations,” in NOMS 2025-2025 IEEE Network Operations and Management Symposium, 2025, pp. 1–9. [21] S. Yousefi, M. Mounesan, and S. Debroy, “Advar-dnn: Adversarial misclassification attack on collaborative dnn inference,” in 2025 IEEE 50th Conference on Local Computer Networks (LCN), 2025, pp. 1–9. [22] Y. Matsubara, M. Levorato, and F. Restuccia, “Split computing and early exiting for deep learning applications: Survey and research challenges,” ACM Computing Surveys, vol. 55, no. 5, pp. 90:1–90:30, 2022. [23] J. Park, S. Cho, and D. Han, “SpecEdge: Scalable edge-assisted serving framework for interactive LLMs,” in Advances in Neural Information Processing Systems (NeurIPS), 2025, spotlight. [Online]. Available: https://arxiv.org/abs/2505.17052 [24] M. Zhang, X. Shen, J. Cao, Z. Cui, and S. Jiang, “EdgeShard: Efficient LLM inference via collaborative edge computing,” IEEE Internet of Things Journal, vol. 12, no. 10, pp. 13 119–13 131, 2025. [25] X. Li, D. Spatharakis, S. Ghafouri, J. Fan, H. Vandierendonck, D. John, B. Ji, and D. S. Nikolopoulos, “SLED: A speculative LLM decoding framework for efficient edge serving,” in Proceedings of the Tenth ACM/IEEE Symposium on Edge Computing (SEC), 2025. [26] X. Yuan, H. Chen, L. Liu, and H. Li, “DistMLLM: Enhancing multimodal large language model serving in heterogeneous edge computing,” Sensors, vol. 25, p. 7612, 2025. [27] J. Shao and J. Zhang, “BottleNet++: An end-to-end approach for feature compression in device-edge co-inference systems,” in Proc. IEEE International Conference on Communications Workshops (ICC Workshops), 2020, pp. 1–6. [28] J. Shao, Y. Mao, and J. Zhang, “Learning task-oriented communication for edge inference: An information bottleneck approach,” IEEE Journal on Selected Areas in Communications, vol. 40, no. 1, pp. 197–211, 2022. [29] ——, “Task-oriented communication for multidevice cooperative edge inference,” IEEE Transactions on Wireless Communications, vol. 22, no. 1, pp. 73–87, 2023. [30] J. Ballé, D. Minnen, S. Singh, S. J. Hwang, and N. Johnston, “Variational image compression with a scale hyperprior,” in Proc. 6th International Conference on Learning Representations (ICLR), 2018. [31] C. Yuan, Z. Liu, J. Lv, J. Shao, Y. Jiang, J. Zhang, and X. Li, “Task-oriented feature compression for multimodal understanding via device-edge co-inference,” IEEE Transactions on Mobile Computing, 2025. [Online]. Available: https://arxiv.org/abs/2503.12926 [32] A. van den Oord, Y. Li, and O. Vinyals, “Representation learning with contrastive predictive coding,” arXiv preprint arXiv:1807.03748, 2018. [33] X. Chen, H. Fang, T.-Y. Lin, R. Vedantam, S. Gupta, P. Dollár, and C. L. Zitnick, “Microsoft coco captions: Data collection and evaluation server,” in Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2015. [34] H. Hotelling, “Analysis of a complex of statistical variables into principal components,” Journal of Educational Psychology, vol. 24, no. 6, pp. 417–441, 1933. [35] R. A. Fisher, “The use of multiple measurements in taxonomic problems,” Annals of Eugenics, vol. 7, no. 2, pp. 179–188, 1936. [36] D. P. Kingma and M. Welling, “Auto-encoding variational bayes,” arXiv preprint arXiv:1312.6114, 2013.
8