CARE-FL: Federated Fine-Tuning of Multimodal LLMs in Resource-Constrained Edge Networks

1

CARE-FL: Federated Fine-Tuning of Multimodal LLMs in Resource-Constrained Edge Networks

arXiv:2611.06169v1 [cs.DC] 4 Nov 2026

Tianqi Zong, Baosheng Li, Nuocheng Yang, Student Member, IEEE, Binbin Chen, Senior Member, IEEE, Marie Siew, Member, IEEE

Abstract—Domain-specific adaptation of Multimodal Large Language Models (MLLMs) in privacy-sensitive edge environments requires federated fine-tuning, yet client selection remains challenging due to heterogeneous multimodal data and modality availability, dynamic system conditions, and evolving client utility during training. We propose Constraint-Aware Reinforcement Learning for Efficient Federated Fine-Tuning (CARE-FL), a constrained online client-selection framework for federated MLLM Low-Rank Adaptation (LoRA) fine-tuning. CARE-FL formulates client participation as a sequential decision process and employs Q-value decomposition to address the large combinatorial joint action space through scalable client-level long-term utility estimation. These utilities are learned online using a modified Double Deep Q-Learning (DDQL) scheme, while constraintaware ϵ-greedy execution selects feasible client subsets under perround latency and bandwidth constraints. Complementing client selection, an adaptive aggregation mechanism further controls potentially unstable global updates and places greater weight on the previous model when needed. Experiments on the CrisisMMD disaster-response task in emulated heterogeneous wireless edge environments show that CARE-FL achieves higher and more stable test accuracy than the compared client-selection baselines across different system settings, while satisfying the imposed latency and bandwidth constraints. Index Terms—Federated learning, multimodal LLMs, edge intelligence, reinforcement learning, client selection.

I. I NTRODUCTION Recent advances in Large Language Models (LLMs) and Multimodal Large Language Models (MLLMs) have significantly improved the capabilities of deep learning systems in complex understanding, reasoning, and generation tasks [1], [2]. By jointly processing multimodal information sources such as text, images, audio, and sensor data, MLLMs better reflect the multimodal nature of real-world human interactions and have demonstrated promising performance across diverse applications, such as healthcare [3], finance [4], education [5], biomedical analysis [6], and transportation [7]. Consequently, domain-specific fine-tuning of MLLMs is becoming increasingly important for deploying intelligent services in practical scenarios. Tianqi Zong, Binbin Chen, and Marie Siew are with the Information Systems Technology and Design, Singapore University of Technology and Design, 487372, Singapore (emails: zong tianqi@mymail.sutd.edu.sg, binbin chen@sutd.edu.sg ,marie siew@sutd.edu.sg). Baosheng Li is with the College of Science, Northwest A&F University, Yangling 712100, China (email: baoshengli96@nwafu.edu.cn). Nuocheng Yang is with the Beijing Laboratory of Advanced Information Network, and the Beijing Key Laboratory of Network System Architecture and Convergence, Beijing University of Posts and Telecommunications, Beijing 100876, China (email: yangnuocheng@bupt.edu.cn).

However, centralized fine-tuning of MLLMs requires collecting task-relevant data from distributed sources, which is often impractical in privacy-sensitive scenarios. Federated Learning (FL) enables edge clients to collaboratively train a global model by exchanging model parameters or updates without sharing raw data [8], [9]. Moreover, MLLMs are increasingly supported on common mobile and edge platforms, including iOS, Android, and HarmonyOS, with several edge-adaptation frameworks and deployment toolchains becoming available [10]. Together with Parameter-Efficient Fine-Tuning (PEFT) techniques such as Low-Rank Adaptation (LoRA) [11], which reduces the number of trainable parameters and communication overhead of large-model adaptation, these developments make federated MLLM fine-tuning increasingly feasible in resource-constrained edge environments. For example, recent studies have explored federated PEFT of LLMs for the Internet of Vehicles [12] and distributed disaster response [13]. Despite the increasing feasibility of federated PEFT of MLLMs on edge devices, practical deployment in wireless edge environments remains challenging. On the learning side, clients exhibit statistical heterogeneity in data quantity, distribution, and modality composition due to differences in sensing capabilities, device types, user behaviors, and observation environments, resulting in varying contributions from clients to global model learning. On the system side, edge clients exhibit heterogeneous and resource-limited computational and communication capabilities [12], [13]. For example, autonomous vehicles and UAVs may have limited onboard processing capacity and wireless bandwidth. Device workloads and wireless channel conditions also vary over time. Together with limited shared network resources, these factors create heterogeneous and time-varying participation costs, further complicating federated PEFT of MLLMs in resource-constrained edge environments. Existing studies have improved the efficiency of federated PEFT of MLLMs in wireless edge environments through model-side optimization strategies, such as PEFT trainableparameter configuration optimization [14]–[16], as well as through joint model adaptation and wireless resource management [17]–[19]. These approaches primarily focus on model configuration and communication-resource optimization; however, client participation often relies on uniform random sampling, leaving client-selection optimization relatively underexplored. Moreover, limited communication resources and the high computational demands of MLLM fine-tuning restrict the number of training rounds achievable within a given time budget, highlighting the importance of effective client

2

selection alongside resource optimization. Extensive studies in traditional federated learning have demonstrated that client participation significantly affects model convergence and system efficiency under statistical and system heterogeneity. Consequently, numerous client-selection strategies have been proposed to prioritize participants according to their estimated training utility, resource efficiency, or both [20], where training utility represents a client’s potential contribution to global model learning. However, directly extending these approaches to federated MLLM fine-tuning is non-trivial due to complex multimodal semantic data and substantially higher computation and communication costs. These characteristics introduce new challenges. Firstly, client training utility is more difficult to characterize in federated MLLM fine-tuning due to complex semantic and multimodal statistical heterogeneity. Beyond differences in local data quantity and distribution, clients may possess different modality combinations, while the semantic content of multimodal data can further lead to heterogeneous local training behaviors. Recent studies [21] have explored LLM-based data-aware client selection using knowledge-graph representations of local datasets. Although such approaches characterize semantic data divergence, they may introduce privacy risks, face difficulties with heterogeneous multimodal inputs, and rely on relatively static client representations that do not adapt to evolving fine-tuning states. Secondly, system-efficient client participation is more difficult to achieve under heterogeneous and dynamic edge conditions. Different local data volumes and modality combinations lead to heterogeneous computational workloads, while device capabilities and wireless conditions introduce further variability in participation costs. Recent studies [22] have addressed system efficiency through joint client selection and wireless resource allocation under emerging technologies such as movable antennas and over-the-air aggregation. However, these approaches primarily focus on system-level resource optimization, without explicitly incorporating client training utility under statistical and modality heterogeneity into participation decisions. These limitations highlight the need for adaptive client selection that jointly considers evolving training utility and dynamic system feasibility. Moreover, client selection is inherently sequential: current participation decisions influence subsequent global model states and future client utilities. These characteristics highlight the need for an adaptive online clientselection strategy that jointly considers evolving client training utility, dynamic system feasibility, and the long-term impact of participation decisions. In view of the above limitations, we propose ConstraintAware Reinforcement Learning for Efficient Federated FineTuning (CARE-FL), an online client-selection framework for federated PEFT of MLLMs in resource-constrained wireless edge environments. The main contributions are as follows: • We propose Constraint-Aware Reinforcement Learning for Efficient Federated Fine-Tuning (CARE-FL), an endto-end framework for federated PEFT of MLLMs in resource-constrained wireless edge environments. CAREFL integrates LoRA-based fine-tuning, RL-assisted dy-

namic client selection, and adaptive aggregation under heterogeneous multimodal data and system conditions. • We formulate adaptive client participation as a constrained sequential decision-making problem and establish an MDP framework that captures evolving clientspecific states through compact PCA-based model representations together with heterogeneous data and modality characteristics, as well as the long-term impact of participation decisions. Per-round latency and bandwidth constraints are incorporated to ensure system feasibility. • We develop a constraint-aware Q-value-decomposed DDQL scheme for scalable online client selection. Q-
value decomposition replaces the representation of K n subset-level Q-values with K client-level long-term utility estimates, substantially reducing the Q-value representation and search complexity, while modified DDQL continuously learns these utilities from evolving finetuning feedback. Constraint-aware ϵ-greedy execution enables feasible exploration and exploitation under the system constraints. Complementing client selection, an adaptive aggregation mechanism mitigates potentially unstable global updates. • Extensive experiments on the CrisisMMD dataset demonstrate that CARE-FL achieves 3–5 percentage points higher test accuracy with greater stability than the compared client-selection baselines while satisfying the imposed latency and bandwidth constraints, and remains effective across different PEFT configurations and wireless resource conditions. The remainder of this paper is organized as follows. Sections II–IV present the related work, system model, and problem formulation, respectively. Section V introduces the proposed CARE-FL, Section VI presents the experimental results, and Section VII concludes the paper. II. R ELATED W ORK A. LLMs, MLLMs, and Parameter-Efficient Fine-Tuning Large Language Models (LLMs), primarily based on the Transformer architecture, have demonstrated strong capabilities in language understanding, reasoning, and generation [1]. Multimodal Large Language Models (MLLMs) extend these capabilities to modalities such as images, audio, and video. Modern MLLMs commonly employ a modular architecture consisting of modality-specific encoders for feature extraction, lightweight connectors for alignment with the language embedding space, and a pretrained LLM backbone for multimodal understanding, reasoning, and generation [23]. Representative MLLMs include Flamingo [24], BLIP-2 [25], LLaVA [26], and MiniCPM-V [10]. Due to the large parameter scale of LLMs and MLLMs, full-parameter fine-tuning incurs substantial computation and memory costs. Parameter-Efficient Fine-Tuning (PEFT) addresses this challenge by updating only a small subset of trainable parameters while keeping most pretrained parameters frozen [27]. Representative approaches include Adapter Tuning [28], Prefix Tuning [29], and Low-Rank Adaptation (LoRA) [11]. In particular, LoRA represents model updates

3

using trainable low-rank matrices, substantially reducing the number of trainable parameters while maintaining competitive adaptation performance. Its configurable rank and insertion locations enable flexible trade-offs between adaptation capability and training overhead, making LoRA particularly suitable for resource-efficient federated fine-tuning of LLMs and MLLMs. B. Federated Learning for LLMs and MLLMs Combined with PEFT, FL enables federated LLM adaptation by exchanging only lightweight trainable parameters instead of full model weights, reducing communication and computation overhead [30]. This allows distributed data to be incorporated into pretrained LLMs without centralizing it. More recently, federated PEFT has been extended from text-only LLMs to MLLMs [31]. Compared with federated LLMs, federated MLLMs introduce additional heterogeneity arising from multimodal data, including different modality combinations, missing modalities, and cross-client differences in multimodal data distributions [32]. Such heterogeneity increases the complexity of local data distributions and leads to varying training contributions among clients. Existing studies on federated large-model PEFT mainly focus on text-based LLMs, exploring parameter-efficient adaptation and communication-resource optimization [14], [15], [17], [18]. Recent work has begun to extend PEFT to MLLMs. For example, one-shot federated low-rank adaptation [33] investigates LoRA-based MLLM fine-tuning under a one-shot communication setting. Nevertheless, adaptive client participation under multimodal heterogeneity and dynamic edge constraints, including bandwidth and latency limitations, remains insufficiently investigated. C. Client Selection in FL Due to communication and computational resource constraints, only a subset of clients can participate in each FL round. Consequently, client selection plays an important role in determining both model convergence and training efficiency. Existing client-selection approaches can be broadly categorized into statistical heterogeneity-aware, system heterogeneity-aware, joint heterogeneity-aware, and learning-based adaptive approaches. Statistical heterogeneity-aware approaches prioritize clients using data quantity and quality, training loss, gradient information, or estimated contributions to global model improvement [34]–[36]. By identifying clients with higher training utility, these approaches aim to accelerate convergence and improve model performance under non-independent and non-identically distributed (non-IID) data. In contrast, system heterogeneityaware approaches consider clients’ computational and communication capabilities to prioritize resource-efficient clients and mitigate stragglers [37], [38]. Joint heterogeneity-aware approaches combine statistical utility and system efficiency to balance convergence performance and training costs [39], [40]. Beyond the above selection strategies, learning-based methods, particularly reinforcement learning (RL), learn adaptive client-selection policies from historical training and system information [41]. By modeling selection as sequential decisionmaking, RL adapts participation to evolving model and system

states rather than relying solely on predefined metrics [42]– [44]. This is particularly relevant to dynamic wireless edge environments with time-varying client utilities and resource conditions. Nevertheless, most existing client-selection methods, including RL-based approaches, focus on conventional FL and rely mainly on traditional statistical and system information. Federated MLLM fine-tuning introduces additional heterogeneity in modality availability, combinations, and local data semantics. Moreover, client utilities evolve with the global model, while system conditions vary across rounds. Therefore, adaptive client selection jointly considering multimodal data characteristics, evolving model information, and dynamic system conditions remains insufficiently explored in federated MLLM fine-tuning. III. S YSTEM M ODEL A. MEC Network Architecture We consider a mobile edge computing (MEC) network consisting of a central server and K edge clients, indexed by K = {1, . . . , K}. Client k holds a local dataset k Dk = {(Xk,i , yk,i )}D i=1 ,

where Xk,i denotes the multimodal input and yk,i is the corresponding target response sequence. The P global dataset S K is denoted by D = k∈K Dk , with D = k=1 Dk . Let M denote the set of modalities available in the network and Mk ⊆ M the modality subset possessed by client k. Accordingly, Xk,i = {xm k,i }m∈Mk . Clients exhibit statistical heterogeneity in data volumes, label distributions, and modality combinations, as well as system heterogeneity in computational capabilities and time-varying wireless conditions. We assume a resource-rich central server and resource-constrained edge clients, resulting in heterogeneous training behavior and computation and communication latency. We next describe federated MLLM fine-tuning and its associated latency model. B. Federated MLLM LoRA Fine-Tuning As illustrated in Fig. 1, clients collaboratively fine-tune a pretrained MLLM for a domain-specific task through FL. The global training objective is K

min F (ws ) =

ws ∈Rd

1 X Dk Fk (wf , ws ), D

(1)

k=1

where Fk (·) is the local loss of client k, wf denotes the frozen pretrained parameters, and ws denotes the trainable parameters shared through FL. The pretrained MLLM comprises modality-specific encoders and projectors followed by an LLM decoder. Each client uses the encoders and projectors corresponding to its available modalities Mk . To reduce local training and communication overhead, LoRA is applied to the LLM decoder while the pretrained backbone remains frozen. This enables modality-specific feature processing while collaboratively adapting the LLM decoder’s reasoning capability. The trainable parameters are ws = {B llm , Allm },

4

Aggregation Upload local LoRA parameters

FL Server

…

Download global LoRA parameters

LoRA 2

LoRA 1

Trainable Parameters

Aggregated LoRA

LoRA n

Frozen Parameters

LLM Decoder With LoRA

LLM Decoder With LoRA

LLM Decoder With LoRA

… Image Projector

Video Projector

Audio Projector

Video Projector

Audio Projector

Image Encoder

Video Encoder

Audio Encoder

Video Encoder

Audio Encoder

Client 2

Client 1

Client n

Fig. 1: The proposed federated MLLM LoRA fine-tuning framework, where heterogeneous clients process different modalities and the server aggregates LoRA updates while keeping the LLM backbone frozen. where B llm and Allm denote the collections of trainable lowrank LoRA matrices in the LLM decoder. Accordingly, the forward propagation of a LoRA-adapted linear layer in the LLM decoder is
y = Wllm + BA x = Wllm x + BAx, where Wllm denotes the frozen pretrained weight matrix of the corresponding linear layer, and B and A denote its trainable LoRA matrices. The federated fine-tuning procedure consists of four stages. Stage 1: Initialization. The server initializes the pretrained MLLM and deploys the model components required by each client’s modality combination. Stage 2: Client Selection and Global Model Distribution. At each round t ∈ {1, . . . , T }, the server selects n clients using binary indicators atk ∈ {0, 1}, where atk = 1 indicates that client k is selected. The participating set is At = {k ∈ K | atk = 1},

K X

atk = n.

k=1

The server distributes the global LoRA parameters wst to the selected clients. Stage 3: Local Training and Model Upload. Each selected client fine-tunes its LoRA parameters using the local autoregressive next-token cross-entropy loss D

|yk,i |

k X 1 X Fk (wf , wk,s ) = − Dk i=1 τ =1

(2)

log pwf ,wk,s (yk,i,τ | yk,i,<τ , Xk,i ) . Each target token is predicted from the multimodal input and preceding target tokens. The backbone wf remains frozen and only the local LoRA parameters wk,s are updated. Padding

and other non-target tokens are masked from the loss computation. After local training, each participating client uploads its updated LoRA parameters to the server. Stage 4: Model Aggregation and Global Model Update. The server aggregates the uploaded LoRA parameters using Federated Averaging (FedAvg): wst+1 =

K X

atk Dk t+1 PK t wk,s . a D j j=1 j k=1

(3)

Since wk,s = {Bkllm , Allm k }, FedAvg is applied independently to the corresponding A and B factors. The resulting wst+1 is used to initialize the next training round, and Stages 2–4 are repeated for T rounds. C. Computation and Communication Model Statistical and system heterogeneity jointly lead to varying client training latency. Local workloads depend on data volumes and modality combinations, while computation capabilities and wireless conditions determine processing and transmission speeds. Assuming sufficient server resources, we neglect aggregation and downlink latency. Thus, the total latency of client k at round t is t t lkt = Tk,cmp + Tk,com ,

(4)

t t where Tk,cmp and Tk,com denote its local computation and uplink communication latency, respectively. 1) Computation Model: The local computational workloads depend on data volumes and modality combinations, which involve different input sizes and encoding and projection paths. The effective data volume of client k is

Vk = Dk γMk ,

5

where γMk captures modality-dependent data volume. Let ϕMk denote the required computation cycles per unit of effective data, reflecting modality-dependent processing complexity. The local computation latency is then modeled as t Tk,cmp =

Vk ϕMk , ck fkt

(5)

where ck denotes the number of available processor cores and fkt denotes the processing frequency of client k at round t. Hence, Dk , γMk , and ϕMk capture the heterogeneous local workload, while ck fkt characterizes the client’s available computing capability. 2) Communication Model: Each selected client uploads its updated LoRA parameters with allocated uplink bandwidth bk and time-varying signal-to-noise ratio SNRtk (in dB) at round t. The achievable uplink rate is [13]   t (6) vkt = bk log2 1 + 10SNRk /10 . Let SLoRA denote the size (in bits) of the uploaded LoRA parameters, identical across clients under the same LoRA configuration. The uplink latency is t Tk,com =

SLoRA . vkt

(7)

Under synchronous FL, round latency is determined by the slowest selected client: Lt = max lkt . k∈At

(8)

Overall, clients differ in both training utility, due to data and modality heterogeneity, and participation latency, due to computational and wireless heterogeneity. Jointly considering these factors motivates the dynamic client-selection problem formulated next. IV. P ROBLEM F ORMULATION Given the heterogeneous multimodal data and dynamic system conditions described above, we aim to determine the participating client subsets over T FL rounds to improve the final global model performance while satisfying practical latency and wireless resource constraints. The client-selection problem is formulated as min

{atk }k,t

s.t.

F (wsT +1 ), K X

(9a)

atk = n,

∀t ∈ {1, . . . , T },

(9b)

∀k, t,

(9c)

∀t ∈ {1, . . . , T },

(9d)

k=1 atk ∈ {0, 1}, t

L ≤ τ̄ , K X

atk bk ≤ Btotal ,

∀t ∈ {1, . . . , T }.

(9e)

k=1

The objective in (9a) aims to minimize the final global empirical loss after T FL rounds. Constraint (9b) requires exactly n clients to participate in each round, while (9c) defines the binary client-selection variables. Constraint (9d) requires each communication round to satisfy a prescribed maximum

tolerable latency τ̄ , representing an application- or systemlevel latency requirement and preventing excessive stragglerinduced delay. Constraint (9e) ensures that the aggregate bandwidth requirement of the selected clients does not exceed the total available wireless bandwidth Btotal . Problem (9) is challenging to solve through offline optimization, since client training utility depends on heterogeneous data and modality characteristics as well as the evolving global model state, while client participation costs and system feasibility vary with dynamic computation and communication conditions. Moreover, the impact of current selection decisions propagates to subsequent rounds through the evolving global model. Consequently, an effective selection policy should adapt to the evolving training process, account for the longterm impact of current decisions, and satisfy the per-round resource constraints. This motivates the constrained sequential decision-making formulation and learning-based online solution developed in the following section. V. CARE-FL S OLUTION To address the optimization problem formulated in Section IV, we propose Constraint-Aware Reinforcement Learning for Efficient Federated MLLM Fine-Tuning (CARE-FL). The framework consists of a Deep Reinforcement Learning (DRL)-assisted dynamic client-selection scheme and an adaptive aggregation mechanism. Rather than directly applying conventional DRL, the former incorporates client-level Qvalue decomposition and constraint-aware online learning to enable scalable long-term utility estimation and feasible client selection under latency and bandwidth constraints. The latter adaptively controls potentially unstable global updates to improve training stability. The two mechanisms are detailed in Sections V-A and V-B, respectively, and their integration into the complete federated MLLM fine-tuning framework is presented in Section V-C. A. DRL-Assisted Dynamic Client Selection Client selection in federated MLLM fine-tuning is inherently sequential, as current participation decisions affect the global model and subsequent client training utilities. To capture such evolving long-term utility, we formulate the clientselection process as an MDP and develop a Constraint-Aware Q-Value Decomposed DDQL Solution. 1) MDP Modeling with Q-Value Decomposition: We formulate the dynamic client-selection process as an MDP, where the central server acts as the agent and interacts with the FL environment over successive communication rounds. However, directly representing each subset of n clients as an action re
sults in K possible actions, making conventional joint actionn value estimation impractical as K increases. We therefore incorporate Q-value decomposition into the MDP formulation, approximating the utility of a selected subset through clientlevel utility estimates. The resulting MDP is defined by the following state, action, reward, and policy objective. State: The system state st ∈ S represents the fine-tuning environment at the beginning of round t. We construct the global state from the individual client states as st = (st1 , st2 , . . . , stK ).

(10)

6

Each client-specific state stk incorporates information on local and global LoRA parameters, local data amount, and modality profile. To reduce the high dimensionality of LoRA parameters, we employ Gram-matrix-based Principal Component Analysis (PCA) to project their flattened parameter vectors into a shared lower-dimensional space, capturing the dominant parameter variations while reducing the dimensionality of the DRL state representation. t zkt = PCA(wst , wk,s ),

where wst denotes the current global LoRA parameters and t wk,s denotes the latest observed LoRA parameters of client k. The client-specific state is represented as stk = (zkt , Dk , Mk ).

(11)

estimates the contribution of selecting client k to collaborative model training at round t, is defined as ( t+1 t+1 e−d(wk,s ,ws ) if Pt > Pt−1 , t ζk = (15) t+1 t+1 1 − e−d(wk,s ,ws ) if Pt ≤ Pt−1 , where Pt denotes the server-side validation performance of the accepted global model wst+1 resulting from round t. The comparison between Pt and Pt−1 therefore reflects the observed global performance change between two consecutive communication rounds. This performance change determines how the local-global parameter divergence is mapped to instantaneous utility, allowing the reward to reflect both the observed global training outcome and the characteristics of individual client updates.

For a client that participated in the previous round, the server refreshes its state using the latest uploaded local parameters. For an unselected client, the server retains the most recently observed state. Therefore, client-state maintenance does not require additional communication from unselected clients. Action Space: Under the Q-value decomposition, the utility of each client is evaluated individually. Accordingly, the action at ∈ A is represented by a binary selection vector:

The exponential mapping transforms the nonnegative divergence into a smooth and bounded utility. The two complementary branches provide opposite monotonic preferences over client divergence while maintaining the same utility range. When the global performance improves, e−d assigns higher utility to clients with smaller local-global divergence, favoring updates that are more consistent with the improving global model. Otherwise, 1 − e−d assigns higher utility to clients with larger divergence, encouraging exploration of clients with more distinct update characteristics. Moreover, the exponential mapping gradually saturates as the divergence increases, limiting the influence of excessively large parameter deviations on the reward. The final reward includes historical feedback through

at ∈ {0, 1}K ,

rkt = λζkt + (1 − λ)ζkt−1 .

This dimensionality reduction keeps the model-related state information tractable for the DRL agent, while Dk and Mk explicitly characterize the client’s data amount and modality profile.

(12)

where atk = 1 indicates that client k is selected. These client-level selection actions jointly determine the participating subset Ut = {k | atk = 1}, with |Ut | = n. Reward: Consistent with the client-level value decomposition, the reward at round t is represented by the individual utilities observed for the selected clients: rt = (rkt )k∈Ut = (rkt 1 , rkt 2 , . . . , rkt n ).

(13)

To quantify the contribution of each selected client at round t, we first measure the normalized coordinate-wise relative L1 divergence between its local LoRA parameters and the global LoRA parameters resulting from round t and available at the beginning of round t + 1, as t+1 d(wk,s , wst+1 ) =

ds t+1 wk,s (i) − wst+1 (i) 1 X , ds i=1 w̄st+1 (i)

(14)

where ds is the number of flattened LoRA parameters and P t+1 wst+1 (i) + j∈Ut wj,s (i) t+1 , w̄s (i) = |Ut | + 1 is the element-wise mean of the accepted global model and the locally updated models of clients selected in round t, used for normalization. Based on this divergence and the observed global model performance change, the instantaneous training utility ζkt , which

(16)

The historical utility is initialized as ζk0 = 0 for all clients and remains unchanged in rounds when the client is not selected. The parameter λ ∈ [0, 1] controls the contribution of the current utility. This temporal smoothing reduces short-term reward fluctuations while retaining information about each client’s recent contribution. Policy Optimization: The DRL agent seeks a policy π ∗ that maximizes the expected cumulative discounted reward over T fine-tuning rounds: " T # X ∗ t−1 π = arg max E γ R(st , at ) , (17) π

t=1

where γ ∈ (0, 1] is the reward discount factor. The aggregate reward is defined as the sum of the individual rewards of the selected clients: R(st , at ) =

K X k=1

atk rkt =

X

rkt .

(18)

k∈Ut

Therefore, the policy objective promotes client subsets with high cumulative utility over the training horizon. 2) Constraint-Aware Q-Value Decomposed DDQL Solution: To solve the above MDP, we develop a constraint-aware Qvalue decomposed DDQL scheme that integrates three components: a decomposed Q-value network for scalable clientlevel long-term utility estimation, a modified DDQL update for online learning of these utilities from fine-tuning feedback,

7

and constraint-aware ϵ-greedy execution for selecting feasible client subsets under the latency and bandwidth constraints.

temporal-difference target for each participating client by bootstrapping its future utility estimate at the next state: Ykt = rkt + γQθ− (st+1 , k),

Fig. 2: Comparison of DRL Q-value network architectures.
The conventional joint-action value network (left) outputs K n Q-values, whereas the proposed Q-value decomposed network (right) outputs K client-level utility estimates. Decomposed Q-Value Network: As illustrated in Fig. 2, directly estimating Qθ (st , at ) for every possible client subset
requires an output dimension of K , which becomes impracn tical as K increases. Since the aggregate reward in (18) is constructed from the individual rewards of the selected clients, it naturally provides client-level feedback on the contribution of each selected client. Motivated by this client-level reward structure, we approximate the long-term value of a selected subset through an additive combination of client-level Qvalues. This design follows the general principle of valuefunction factorization, where a joint action value is represented through individual utility functions [45]–[47]. Accordingly, instead of estimating Q-values for all client subsets, the proposed network outputs K client-level Q-value estimates and approximates the value of a selected subset by Qθ (st , at ) ≈

K X

atk Qθ (st , k) =

k=1

X

Qθ (st , k),

a∗t = arg max a∈A

atk · Qθ (st , k),

k=1

s.t.

k∈Ut

The additive loss enables direct credit assignment to the participating clients, while the target network is periodically synchronized with the online network every I rounds, where I denotes the target-network update interval. The resulting learning procedure is summarized in Algorithm 1. Constraint-Aware ϵ-greedy Online RL Client Selection: The learned client-level Q-values provide utility estimates for client selection. However, directly selecting the clients with the highest Q-values may violate the bandwidth and latency constraints. We therefore develop a constraint-aware ϵ-greedy strategy that converts the learned utilities into feasible client subsets while maintaining exploration during online learning. Specifically, the exploitation step solves a∗t = arg max

where Qθ (st , k) denotes the estimated long-term utility associated with selecting client k under state st , learned from its participation in jointly selected client subsets. Consequently,
the Q-network output dimension is reduced from K n to K. Based on these client-level utilities, client selection can be expressed as K X

where rkt is the immediate client-level reward and Qθ− (st+1 , k) provides the bootstrapped estimate of the future training utility of client k. This modified target is consistent with the client-level decomposition in (19), enabling each participating client’s utility to be updated individually from the observed fine-tuning feedback without constructing a target over the combinatorial client-subset action space. Decomposed Loss Function: Based on the individual targets, the online network is optimized by minimizing the client-level Q-value estimation errors: " # X
2 L(θ) = E Ykt − Qθ (st , k) . (23)

(19)

k∈Ut

K X

atk = n.

(20)

k=1

Modified DDQL Optimization: To train the decomposed Q-value network, we adopt the online-target network architecture of DDQL and modify its target construction to accommodate client-level long-term utility estimation. Specifically, the online network Qθ estimates the client long-term utilities and is optimized during training, while the target network Qθ− is used to construct stable target values. Individual Target Value Construction: Conventional DDQL [48] uses the online network for action selection and the target network for evaluation, yielding   ′ Yt = rt + γQθ− st+1 , arg max Q (s , a ) . (21) θ t+1 ′ a

In our decomposed framework, instead of constructing a target for the joint client-subset action, we construct an individual

(22)

a∈A

s.t.

K X k=1 K X

atk · Qθ (st , k),

(24a)

atk · bk ≤ Btotal ,

(24b)

k=1

max lkt ≤ τ̄ , k∈Ut

K X

atk = n.

(24c) (24d)

k=1

This constrained Top-n problem selects clients with high estimated long-term utilities while ensuring system feasibility and can be solved as an integer linear program (ILP) for the considered client scale. Equation (24) executes the learned Q-values under constraints rather than globally solving the original sequential optimization problem. To retain exploration, we integrate the constrained Top-n selection into an ϵ-greedy mechanism. At round t, ⌊(1 − ϵ)n⌋ clients are first selected according to their learned Q-values subject to (24). The remaining slots are filled by random exploration, where a candidate client is admitted only if the resulting subset remains feasible under the residual bandwidth and latency constraints. The exploration rate ϵ gradually decays during training, shifting the policy from exploration toward exploitation. The complete procedure is summarized in Algorithm 2.

8

Algorithm 1 Client-Level DDQL Learning 1: procedure U PDATE AGENT (B, θ, θ − , γ, t, I)

Sample a minibatch of N transitions {(st,j , kj , rk,j , st+1,j )}N from B j=1 3: for each transition j ∈ {1, . . . , N } do 4: Yj ← rk,j + γQθ− (st+1,j , kj ) 5: end for P N 6: L(θ) ← N1 j=1 (Yj − Qθ (st,j , kj ))2 7: Update θ by performing gradient descent on L(θ) 8: if t (mod I) == 0 then 9: θ− ← θ 10: end if 11: end procedure 2:

Algorithm 2 Constraint-Aware ϵ-Greedy Client Selection 1: procedure C ONSTRAINT S ELECT (Qθ , st , ϵ, n, Btotal , τ̄ )

// Stage 1: Q-guided exploitation nq ← ⌊(1 − ϵ)n⌋ Uq ← C ONSTRAINT T OP(Qθ , st , nq , Btotal , τ̄ ) // Stage 2: Random exploration 4: nr ← n − nq P 5: Brem ← Btotal − k∈Uq bk 6: Ur ← C ONSTRAINT R ANDOM(nr , Brem , τ̄ , Uq ) 7: Ut ← Uq ∪ Ur 8: return Ut 9: end procedure 2: 3:

10: procedure C ONSTRAINT T OP(Qθ , st , n′ , B, τ ) 11: 12:

Kfeas ← {k ∈ K | lkt P ≤ τ} ⋆ Solve aP = arg max atk Qθ (st , k) t k∈KfeasP t t ′ s.t. k∈Kfeas ak bk ≤ B, k∈Kfeas ak = n t⋆ return {k ∈ Kfeas | ak = 1}

13: 14: end procedure

15: procedure C ONSTRAINT R ANDOM (n′ , B, τ, Uexcl ) 16: Utmp ← ∅ 17: C ← {k ∈ K \ Uexcl | lkt ≤ τ } 18: while |Utmp | < n′ and C ̸= ∅ do 19: Randomly sample k ∈ C 20: if bk ≤ B then

Utmp ← Utmp ∪ {k} B ← B − bk end if C ← C \ {k} end while 26: return Utmp 27: end procedure

21: 22: 23: 24: 25:

B. Adaptive Aggregation Although the proposed RL scheme enables adaptive client selection, exploratory actions and heterogeneous client updates may occasionally produce unstable global updates, particularly during the early policy-learning stage. To improve training stability, we employ SlowFed, an adaptive aggregation mechanism that controls the magnitude of the global update

according to the observed model performance. At round t, the server first obtains a trial model using the standard FedAvg update t+1 ws,avg = wst + ∆wst ,

(25)

where ∆wst denotes the aggregated LoRA update obtained from the selected clients. The server then evaluates the trial model on a server-side validation set. If its performance degradation relative to the current global model does not exceed a predefined threshold Γ, i.e., t+1 Acc(ws,avg ) ≥ Acc(wst ) − Γ, t+1 the trial update is accepted and wst+1 = ws,avg . Otherwise, SlowFed triggers a conservative recovery procedure by scaling the trial update with a server-side learning rate δi . t+1 wst+1,(i) = (1 − δi )wst + δi ws,avg = wst + δi ∆wst .

(26)

Progressively smaller δi values are examined until the stability condition is satisfied. If none of the recovery attempts satisfies the condition, the server retains the previous global model, i.e., wst+1 = wst . The complete SlowFed aggregation procedure is summarized in Stage 4 of Algorithm 3.

C. Federated MLLM LoRA Fine-Tuning Framework With the proposed RL-assisted client selection and adaptive aggregation mechanisms, we now integrate the above components into the complete federated MLLM LoRA fine-tuning framework. The overall training procedure, summarized in Algorithm 3, consists of the following five stages. Stage 1: Initialization. The server initializes the global LoRA parameters, decomposed Q-networks, and replay buffer B. The initial validation accuracy is set to Acc1 = 0. Each client performs a one-time local warm-up update to construct the initial state s1 , reflecting local data and modality heterogeneity. This warm-up is excluded from the subsequent constrained training rounds. For each round t = 1, . . . , T , CARE-FL performs: Stage 2: Constraint-Aware Client Selection. Given st , the server selects Ut using the constraint-aware ϵ-greedy strategy in Algorithm 2 and distributes the global LoRA parameters to the selected clients. Stage 3: Local Fine-Tuning and Upload. Selected clients fine-tune their parameters locally and upload the updates. Stage 4: Adaptive Aggregation. The server aggregates local updates weighted by data size. SlowFed then adaptively controls this update to produce the next global LoRA parameters wst+1 , mitigating potentially unstable global updates. Stage 5: State and RL Update. The server computes the client-level rewards rkt , constructs the next state st+1 , and stores the transitions (st , k, rkt , st+1 ) for k ∈ Ut in B. The decomposed Q-network is then updated online according to the modified DDQL procedure in Algorithm 1, using minibatches sampled from the experience replay buffer B. The resulting st+1 becomes the state for the next communication round.

9

Algorithm 3 CARE-FL Framework Require: Clients K, rounds T , selected clients n, Btotal , τ̄ , Γ, {δ1 , . . . , δR }. Ensure: Global LoRA parameters wsT +1 . // Stage 1: Initialization 1: Initialize ws1 , Qθ , Qθ− , and replay buffer B. 2: All clients perform one-time warm-up to initialize s1 . 3: for t = 1 to T do // Stage 2: Constraint-Aware Client Selection 4: Ut ← C ONSTRAINT S ELECT(Qθ , st , ϵ, n, Btotal , τ̄ ) // Stage 3: Local Fine-Tuning 5: Each selected client k ∈ Ut locally fine-tunes wst and t uploads ∆wk,s . // Stage 4: SlowFed Aggregation X |Dk | t P ∆wk,s 6: ∆wst ← j∈Ut |Dj |

factor are set to r = 8 and α = 8, respectively, resulting in approximately 5.9 million (∼ 0.17%) trainable parameters. Dataset, Task, and Heterogeneous Data Partitioning: We use the CrisisMMD dataset [49], a multimodal disasterresponse dataset containing more than 16,000 annotated image-text pairs collected from seven major natural disasters. Following the preprocessing methodology of [50], the resulting dataset contains five humanitarian categories: Affected Individuals, Infrastructure Damage, Rescue Efforts, Other Relevant Information, and Not Humanitarian, with 6,126 training samples, 998 validation samples, and 998 test samples. We formulate the task as multimodal instruction tuning, where the MLLM predicts a humanitarian category from an image Xv and textual input Xq containing the sample text and classification instruction. The language modeling loss is

k∈Ut

t+1 ws,avg ← wst + ∆wst t+1 Acct+1 ← E VAL(ws,avg ) if Acct+1 < Acct − Γ then for i = 1 to R do wst+1 ← wst + δi ∆wst 12: Acct+1 ← E VAL(wst+1 ) 13: if Acct+1 ≥ Acct − Γ then 14: break 15: end if 16: end for 17: if Acct+1 < Acct − Γ then 18: wst+1 ← wst 19: end if 20: else t+1 21: wst+1 ← ws,avg 22: end if // Stage 5: State and RL Update 23: Construct st+1 and compute {rtk }k∈Ut . 24: Store {(st , k, rtk , st+1 )}k∈Ut in B. 25: U PDATE AGENT(B, θ, θ− , γ, t, I) 26: end for 27: return wsT +1

7: 8: 9: 10: 11:

VI. E XPERIMENTS In this section, we present the empirical validation of the proposed CARE-FL framework. We first outline the experimental setup, including model architecture, dataset specifications, and baseline comparisons. Subsequently, we demonstrate the effectiveness of our method in terms of both global model accuracy and system efficiency under resourceconstrained MEC environments. A. Experimental Settings MLLM Model and PEFT Configuration: We adopt MiniCPM-V 2.0 [10], a 2.8B-parameter MLLM comprising a SigLIP vision encoder, a MiniCPM language backbone, and a Perceiver Resampler. The vision encoder and resampler are frozen, and LoRA is applied to the self-attention modules (qproj , kproj , vproj , oproj ) of the LLM backbone. By default, the LoRA rank and scaling

L=−

Ny X

log P (yi | Xv , Xq , y<i ; θ),

(27)

i=1

where Ny is the target response length, yi denotes the i-th token of Y , y<i denotes the preceding target tokens, and θ denotes the trainable LoRA parameters. The training set is partitioned among (K=50) clients using a Dirichlet distribution with α = 0.1 to introduce heterogeneous label distributions and data quantities, while validation and test sets remain at the server. Modality heterogeneity is introduced through approximately equal proportions of text-only, imageonly, and image-text clients. Baselines: We compare the proposed framework with the following client-selection methods: FedAvg: Randomly selects n clients per round. FAVOR [42]: An accuracy-oriented RL method using a Deep Q-Network (DQN) with model weights as state inputs. FLASH-RL [43]: A system-aware RL method incorporating hardware characteristics into client states and employing client-specific rewards. Under resource constraints, all methods follow their original selection policies while sequentially admitting clients that satisfy the same bandwidth and latency constraints, until n clients are selected. Evaluation Metrics: To provide a comprehensive assessment of CARE-FL, we utilize the following key metrics: Test Accuracy: We measure the Top-1 accuracy of the global model on the held-out test set, where generated responses are mapped to the corresponding humanitarian categories. Latency Usage: We calculate the per-round latency determined by the slowest selected client. Total Bandwidth Usage: Total bandwidth usage is calculated as the summation of the individual bandwidth for all selected clients within that round. The latter two metrics are used to evaluate system efficiency and verify satisfaction of the latency and bandwidth constraints. Implementation and Training Parameters: CARE-FL is implemented using PyTorch and Hugging Face. Local finetuning employs AdamW (learning rate of 2 × 10−5 , weight decay 0.1, β2 = 0.95) with cosine learning rate decay and 1% warm-up. We use a batch size of 1, BF16 precision, gradient

10

TABLE I: Experimental Parameter Settings Category Parameter Value FL (K, n, T, E) (50, 10, 50, 1) LoRA (r, α) (8, 8) Local training Learning rate 2 × 10−5 RL Discount factor γ 0.9 Exploration rate ϵ 0.9 → 0.2 Exploration decay Linear over T rounds Reward smoothing factor λ 0.6 Replay buffer capacity 1000 DDQL update batch Entire current buffer DDQL learning rate 0.01 Target network update interval 5 rounds Gradient steps per DDQL update 30 SlowFed Performance threshold Γ 3% Recovery factors δ {0.1, 0.05, 0.025} Max. recovery attempts R 3

checkpointing, a maximum sequence length of 1200 tokens, and up to nine image slices. Each selected client performs one local epoch per round. Table I summarizes the FL, LoRA, RL, and SlowFed hyperparameters. The replay buffer stores clientlevel transitions, with each DDQL update using all available samples. Experimental Platform and Heterogeneous Distributed System: Experiments are conducted on two NVIDIA RTX 4090 (24 GB) GPUs, centrally emulating a wireless FL system with 50 heterogeneous clients. Client profiles differ in bandwidth, channel conditions, computational resources, and modality-dependent workloads. Following [52], client SNR and CPU frequency vary across rounds to simulate dynamic computation and communication conditions. B. Overall Accuracy Performance Unless otherwise specified, all reported accuracy curves are averaged over three independent runs to reduce run-torun variability. In the unconstrained setting, FLASH-RL is evaluated using both its default configuration, α1 = 1, α2 = 1, where α1 and α2 weight statistical and system utility in its reward, and an additional configuration with α1 = 1, α2 = 0. The latter excludes system utility, enabling evaluation of selection based solely on statistical utility without explicit resource constraints. Under the unconstrained setting, Fig. 3 shows that CAREFL reaches high accuracy earlier and maintains the strongest late-round performance. After approximately 20 communication rounds, its test accuracy generally remains above 80%, whereas the baselines show lower accuracy and greater fluctuations. Table II further shows that CARE-FL achieves the highest final-round mean accuracy of 82.56%, with a latestage accuracy range of only 1.91 percentage points. These results indicate improved convergence and relatively consistent accuracy across runs. This advantage aligns with CARE-FL’s design. The state representation incorporates client-specific modality information, allowing the learned utilities to capture modality heterogeneity and evolving client contributions. Client-level Qvalue decomposition replaces combinatorial joint-action representation with individual utility estimates, enabling scalable learning while preserving long-term client evaluation. Without

Fig. 3: Test accuracy convergence without resource constraints. The curves and shaded regions represent the mean test accuracy and its variability across independent runs, respectively. TABLE II: Final-round accuracy and late-stage variation without resource constraints. Method

Final Mean Last-3 Min Last-3 Max Range (%) (%) (%) (pp) CARE-FL 82.56 81.26 83.17 1.91 FLASH-RL (1, 1) 80.01 79.36 80.86 1.50 FAVOR 79.02 76.05 81.86 5.81 FLASH-RL (1, 0) 78.82 76.35 81.96 5.61 FedAvg 77.16 75.75 78.06 2.31 Final Mean denotes the mean test accuracy at the final communication round. Last-3 Min and Last-3 Max denote the minimum and maximum test accuracies observed over the last three communication rounds across independent runs. Range is their difference in percentage points (pp).

explicit resource constraints, the resulting policy prioritizes clients by learned long-term training utility, helping explain the stronger convergence in Fig. 3 and Table II. C. Resource-Constrained Accuracy and Feasibility To assess practical deployment, we evaluate whether CAREFL maintains its performance advantage under wireless resource constraints, examining both convergence and selection feasibility. Figs. 4 and 5 report results under moderate constraints of 15 Mbps and 3000 s and stringent constraints of 10 Mbps and 1500 s, respectively. Under moderate constraints, Figs. 4(a) and 4(b) show that CARE-FL maintains the strongest late-round accuracy among the compared methods and converges to approximately 80%. Tightening the limits to 10 Mbps and 1500 s reduces overall attainable accuracy, but Figs. 5(a) and 5(b) show that CARE-FL still achieves the highest late-round performance, finishing near 79%. Figs. 4(c) and 5(c) further show that constraint-aware execution maintains bandwidth and latency usage within the prescribed limits, whereas unconstrained selection repeatedly violates them. Thus, CARE-FL preserves a clear convergence advantage while maintaining resource constraints. These results reflect CARE-FL’s constraint-aware selection design. Unlike FLASH-RL, which combines training utility

11

(a) Test accuracy convergence.

(b) Final-stage accuracy.

(c) Resource usage validation.

Fig. 4: Performance comparison and constraint validation under moderate resource constraints.

(a) Test accuracy convergence.

(b) Final-stage accuracy.

(c) Resource usage validation.

Fig. 5: Performance comparison and constraint validation under stringent resource constraints.

and system cost through reward weights requiring careful balancing, CARE-FL focuses on learning training utilities while explicitly enforcing bandwidth and latency constraints during selection. Tighter constraints shrink the feasible action space, potentially excluding high-utility clients or those with complementary data and modalities. The resulting accuracy reduction illustrates the expected accuracy-resource trade-off. Nevertheless, CARE-FL prioritizes high-utility clients among feasible candidates, maintaining comparatively strong convergence while satisfying resource limits. D. Ablation and Configuration Robustness We further assess CARE-FL’s robustness across trainableparameter configurations, the contribution of SlowFed, and different resource constraints. Specifically, Fig. 6 evaluates two additional LoRA rank-32 configurations, Fig. 7 evaluates the effect of SlowFed, and Fig. 8 compares convergence under increasingly stringent resource limits. Fig. 6 evaluates two configurations with LoRA rank (r=32), applied to either all LLM layers or only the last 20 layers. Compared with the default configuration of r = 8 applied to all LLM layers, these rank and layer-placement choices produce different convergence trajectories, reflecting their influence on fine-tuning dynamics. Nevertheless, CARE-FL consistently reaches high accuracy and maintains competitive late-round performance against the baselines under both configurations. These results demonstrate its effectiveness across different PEFT configurations. Fig. 7 examines SlowFed’s contribution. Without SlowFed, CARE-FL exhibits pronounced intermediate-round accuracy drops, whereas SlowFed smooths convergence and reduces

these degradations. Both variants approach similar final accuracy, suggesting that SlowFed primarily improves stability rather than guaranteeing higher final accuracy. This aligns with its design: SlowFed evaluates trial global updates using server-side validation performance and progressively reduces the update magnitude when degradation exceeds a prescribed threshold. Figs. 8(a) and 8(b) compare CARE-FL under increasingly stringent resource constraints. The unconstrained setting achieves the highest late-round accuracy, followed by moderate (15 Mbps, 3000 s), stringent (10 Mbps, 1500 s), and more restrictive (5 Mbps, 1000 s) settings. This ordering highlights the accuracy-resource trade-off: tighter constraints shrink the feasible action space and may exclude high-utility clients or those with complementary data and modalities, limiting attainable accuracy. Nevertheless, CARE-FL maintains comparatively stable convergence by prioritizing highutility clients within each feasible action space. Together, Figs. 6–8 demonstrate CARE-FL’s effectiveness across PEFT and resource configurations, while SlowFed improves robustness against unstable global updates during online and exploratory client selection. VII. C ONCLUSION AND F UTURE D IRECTIONS This paper presented CARE-FL, a constraint-aware reinforcement learning framework for efficient federated finetuning of Multimodal Large Language Models (MLLMs) in dynamic wireless edge environments. CARE-FL formulates adaptive client participation as a constrained sequential decision-making problem and employs client-level Q-value decomposition to enable scalable online learning of evolving

12

(a) LoRA rank r = 32 applied to all LLM layers.

(b) LoRA rank r = 32 applied to the last 20 LLM layers.

Fig. 6: Performance comparison under different LoRA configurations.

Fig. 7: Ablation study of adaptive aggregation. Test accuracy convergence with and without adaptive aggregation.

(a) Test accuracy convergence.

(b) Final-stage accuracy.

Fig. 8: Performance under different resource constraint levels.

client utility. A constraint-aware Double Deep Q-Learning (DDQL) policy uses the learned utilities to adapt client participation under dynamic system conditions while satisfying per-round bandwidth and latency constraints. An adaptive aggregation mechanism further mitigates potentially unstable global updates arising during heterogeneous client updates and exploratory participation decisions. Experiments on the CrisisMMD disaster-response task demonstrate that CARE-FL achieves stronger and more stable convergence than the compared client-selection baselines across both unconstrained and resource-constrained settings. Under bandwidth and latency constraints, CARE-FL maintains comparatively strong model performance while keeping the selected client subsets within the prescribed resource limits. Additional experiments under different LoRA configurations demonstrate the applicability of the proposed framework across different trainable-parameter settings, while the adaptive aggregation ablation confirms its role in improving convergence stability. Future directions include investigating complementary computation and communication optimization strategies, opportunistic client scheduling and temporal resource prediction, and validation-free stability control to extend CARE-FL to more resource-constrained deployment scenarios.

R EFERENCES [1] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” Advances in neural information processing systems, vol. 30, 2017. [2] D. Zhang, Y. Yu, J. Dong, C. Li, D. Su, C. Chu, and D. Yu, “Mm-llms: Recent advances in multimodal large language models,” in Findings of ACL, 2024, pp. 12 401–12 430. [3] R. AlSaad, A. Abd-Alrazaq, S. Boughorbel, A. Ahmed, M.-A. Renault, R. Damseh, and J. Sheikh, “Multimodal large language models in health care: applications, challenges, and future outlook,” Journal of medical Internet research, vol. 26, p. e59505, 2024. [4] X.-Y. Liu, Y. Cao, and L. Deng, “Multimodal financial foundation models (mffms): Progress, prospects, and challenges,” Prospects, and Challenges (May 31, 2025), 2025. [5] K. Borazjani, N. Khosravan, R. Sahay, B. Akram, and S. Hosseinalipour, “Bringing multi-modal multi-task federated foundation models to education domain: prospects and challenges,” Frontiers in Artificial Intelligence, vol. 8, p. 1683960, 2025. [6] X. Li, L. Peng, Y.-P. Wang, and W. Zhang, “Open challenges and opportunities in federated foundation models towards biomedical healthcare,” BioData Mining, vol. 18, no. 1, p. 2, 2025. [7] Z. Xu, Y. Zhang, E. Xie, Z. Zhao, Y. Guo, K.-Y. K. Wong, Z. Li, and H. Zhao, “Drivegpt4: Interpretable end-to-end autonomous driving via large language model,” IEEE Robotics and Automation Letters, vol. 9, no. 10, pp. 8186–8193, 2024. [8] B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. A. y Arcas, “Communication-efficient learning of deep networks from decentralized data,” in Artificial intelligence and statistics. Pmlr, 2017, pp. 1273– 1282. [9] M. Siew, H. Zhang, J.-I. Park, Y. Liu, Y. Ruan, L. Su, S. Ioannidis, E. Yeh, and C. Joe-Wong, “Fair concurrent training of multiple models in federated learning,” IEEE Transactions on Networking, vol. 33, no. 6, pp. 2881–2896, 2025.

13

[10] Y. Yao, T. Yu, A. Zhang, C. Wang, J. Cui, H. Zhu, T. Cai, H. Li, W. Zhao, Z. He et al., “Minicpm-v: A gpt-4v level mllm on your phone,” arXiv preprint arXiv:2408.01800, 2024. [11] E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen, “Lora: Low-rank adaptation of large language models,” arXiv preprint arXiv:2106.09685, 2021. [12] G. Xie, Z. Xiong, X. Zhang, R. Xie, S. Guo, M. Guizani, and H. V. Poor, “Gai-iov: Bridging generative ai and vehicular networks for ubiquitous edge intelligence,” IEEE Transactions on Wireless Communications, vol. 23, no. 10, pp. 12 799–12 814, 2024. [13] Y. Zhao, L. Feng, F. Zhou, W. Li, Z. Xiong, H. Du, C. Wu, S. Guo, T. Q. Quek, and Z. Han, “Energy-efficient over-the-air computation for federated generative model fine-tuning in unmanned vehicle-assisted disaster relief,” IEEE Transactions on Cognitive Communications and Networking, vol. 12, pp. 56–73, 2025. [14] J. Liu, Y. Liao, H. Xu, Y. Xu, J. Liu, and C. Qian, “Adaptive parameterefficient federated fine-tuning on heterogeneous devices,” IEEE Transactions on Mobile Computing, vol. 24, no. 11, pp. 12 533–12 549, 2025. [15] Z. Fang, Z. Lin, Z. Chen, X. Chen, Y. Gao, and Y. Fang, “Automated federated pipeline for parameter-efficient fine-tuning of large language models,” IEEE Transactions on Mobile Computing, 2025. [16] J. Liu, R. Li, H. Xu, Q. Ma, J. Yan, and L. Huang, “Fedquad: Adaptive layer-wise lora deployment and activation quantization for federated fine-tuning,” IEEE Transactions on Mobile Computing, 2025. [17] Y. Zou, Z. Wang, Y. Shi, and Y. Zhou, “Joint rank optimization and bandwidth allocation for heterogeneous federated lora fine-tuning,” IEEE Transactions on Vehicular Technology, 2025. [18] H. Wu, X. Chen, and K. Huang, “Resource management for low-latency cooperative fine-tuning of foundation models at the network edge,” IEEE Transactions on Wireless Communications, vol. 24, no. 6, pp. 4839– 4852, 2025. [19] X. Zhang, N. Yan, Y. Su, Y. Deng, and T. Mahmoodi, “Communicationaware knowledge distillation for federated llm fine-tuning over wireless networks,” in GLOBECOM 2025-2025 IEEE Global Communications Conference. IEEE, 2025, pp. 1956–1961. [20] L. Fu, H. Zhang, G. Gao, M. Zhang, and X. Liu, “Client selection in federated learning: Principles, challenges, and opportunities,” IEEE Internet of Things Journal, vol. 10, no. 24, pp. 21 811–21 819, 2023. [21] B. Zhang, D. Wang, Y. Zhu, and Z. Han, “Data divergence-aware client selection via knowledge graph for federated llm fine-tuning,” IEEE Transactions on Mobile Computing, 2025. [22] Y. Zhao, Y. Xiu, C. Dai, N. Wei, and D. Niyato, “Movable antenna enhanced federated fine-tuning of large language models via hybrid client selection optimization,” arXiv preprint arXiv:2506.00011, 2025. [23] S. N. Wadekar, A. Chaurasia, A. Chadha, and E. Culurciello, “The evolution of multimodal model architectures,” arXiv preprint arXiv:2405.17927, 2024. [24] J.-B. Alayrac, J. Donahue, P. Luc, A. Miech, I. Barr, Y. Hasson, K. Lenc, A. Mensch, K. Millican, M. Reynolds et al., “Flamingo: a visual language model for few-shot learning,” Advances in neural information processing systems, vol. 35, pp. 23 716–23 736, 2022. [25] J. Li, D. Li, S. Savarese, and S. Hoi, “Blip-2: Bootstrapping languageimage pre-training with frozen image encoders and large language models,” in International conference on machine learning. PmLR, 2023, pp. 19 730–19 742. [26] H. Liu, C. Li, Q. Wu, and Y. J. Lee, “Visual instruction tuning,” Advances in neural information processing systems, vol. 36, pp. 34 892– 34 916, 2023. [27] J. He, C. Zhou, X. Ma, T. Berg-Kirkpatrick, and G. Neubig, “Towards a unified view of parameter-efficient transfer learning,” arXiv preprint arXiv:2110.04366, 2021. [28] N. Houlsby, A. Giurgiu, S. Jastrzebski, B. Morrone, Q. De Laroussilhe, A. Gesmundo, M. Attariyan, and S. Gelly, “Parameter-efficient transfer learning for nlp,” in International conference on machine learning. PMLR, 2019, pp. 2790–2799. [29] X. L. Li and P. Liang, “Prefix-tuning: Optimizing continuous prompts for generation,” in Proceedings of the 59th annual meeting of the association for computational linguistics and the 11th international joint conference on natural language processing (volume 1: Long papers), 2021, pp. 4582–4597. [30] Z. Zhang, Y. Yang, Y. Dai, Q. Wang, Y. Yu, L. Qu, and Z. Xu, “Fedpetuning: When federated learning meets the parameter-efficient tuning methods of pre-trained language models,” in Findings of ACL, 2023, pp. 9963–9977. [31] X. Zhou, J. He, Y. Ke, G. Zhu, V. Gutiérrez-Basulto, and J. Pan, “An empirical study on parameter-efficient fine-tuning for multimodal large

language models,” in Findings of the Association for Computational Linguistics: ACL 2024, 2024, pp. 10 057–10 084. [32] B. Xu, X. Shu, H. Mei, G. Xie, B. Fernando, and J. Tang, “Fedmllm: Federated fine-tuning mllm on multimodal heterogeneity data,” arXiv preprint arXiv:2411.14717, 2024. [33] B. Xu, H. Mei, Z. Bai, J. Gong, R. Yan, G. Xie, Y. Yao, B. Fernando, and X. Shu, “You only communicate once: One-shot federated lowrank adaptation of mllm,” Advances in Neural Information Processing Systems, vol. 38, pp. 61 630–61 661, 2026. [34] Y. J. Cho, J. Wang, and G. Joshi, “Towards understanding biased client selection in federated learning,” in International conference on artificial intelligence and statistics. PMLR, 2022, pp. 10 351–10 375. [35] C. Li, X. Zeng, M. Zhang, and Z. Cao, “Pyramidfl: A fine-grained client selection framework for efficient federated learning,” in Proceedings of the 28th annual international conference on mobile computing and networking, 2022, pp. 158–171. [36] J. Zhao, X. Chang, Y. Feng, C. H. Liu, and N. Liu, “Participant selection for federated learning with heterogeneous data in intelligent transport system,” IEEE transactions on intelligent transportation systems, vol. 24, no. 1, pp. 1106–1115, 2022. [37] Z. Chai, A. Ali, S. Zawad, S. Truex, A. Anwar, N. Baracaldo, Y. Zhou, H. Ludwig, F. Yan, and Y. Cheng, “Tifl: A tier-based federated learning system,” in Proceedings of the 29th international symposium on highperformance parallel and distributed computing, 2020, pp. 125–136. [38] J. Shin, Y. Li, Y. Liu, and S.-J. Lee, “Fedbalancer: Data and pace control for efficient federated learning on heterogeneous clients,” in Proceedings of the 20th Annual International Conference on Mobile Systems, Applications and Services, 2022, pp. 436–449. [39] F. Lai, X. Zhu, H. V. Madhyastha, and M. Chowdhury, “Oort: Efficient federated learning via guided participant selection,” in 15th {USENIX} Symposium on Operating Systems Design and Implementation ({OSDI} 21), 2021, pp. 19–35. [40] J. Tang, X. Li, H. Li, P. Li, X. Wang, and V. C. Leung, “Joint classbalanced client selection and bandwidth allocation for cost-efficient federated learning in mobile edge computing networks,” IEEE Transactions on Mobile Computing, vol. 24, no. 7, pp. 5681–5698, 2025. [41] J. Xu and H. Wang, “Client selection and bandwidth allocation in wireless federated learning networks: A long-term perspective,” IEEE Transactions on Wireless Communications, vol. 20, no. 2, pp. 1188– 1200, 2020. [42] H. Wang, Z. Kaplan, D. Niu, and B. Li, “Optimizing federated learning on non-iid data with reinforcement learning,” in Proc. IEEE INFOCOM. IEEE, 2020, pp. 1698–1707. [43] S. Bouaziz, H. Benmeziane, Y. Imine, L. Hamdad, S. Niar, and H. Ouarnoughi, “Flash-rl: Federated learning addressing system and static heterogeneity using reinforcement learning,” in 2023 IEEE 41st international conference on computer design (ICCD). IEEE, 2023, pp. 444–447. [44] S. Moon and Y. Lim, “Client selection for federated learning in vehicular edge computing: A deep reinforcement learning approach,” IEEE Access, vol. 12, pp. 131 337–131 348, 2024. [45] P. Sunehag, G. Lever, A. Gruslys, W. M. Czarnecki, V. Zambaldi, M. Jaderberg, M. Lanctot, N. Sonnerat, J. Z. Leibo, K. Tuyls et al., “Value-decomposition networks for cooperative multi-agent learning,” arXiv preprint arXiv:1706.05296, 2017. [46] T. Rashid, M. Samvelyan, C. S. De Witt, G. Farquhar, J. Foerster, and S. Whiteson, “Monotonic value function factorisation for deep multiagent reinforcement learning,” Journal of Machine Learning Research, vol. 21, no. 178, pp. 1–51, 2020. [47] A. Huang, Y. Wang, R. Liu, H. Zou, and X. Zhou, “Qvf: Incorporating quantile value function factorization into cooperative multi-agent reinforcement learning,” Pattern Recognition, vol. 161, p. 111323, 2025. [48] H. Van Hasselt, A. Guez, and D. Silver, “Deep reinforcement learning with double q-learning,” in Proceedings of the AAAI conference on artificial intelligence, vol. 30, no. 1, 2016. [49] F. Alam, F. Ofli, and M. Imran, “Crisismmd: Multimodal twitter datasets from natural disasters,” in Proceedings of the international AAAI conference on web and social media, vol. 12, no. 1, 2018. [50] F. Ofli, F. Alam, and M. Imran, “Analysis of social media data using multimodal deep learning for disaster response,” arXiv preprint arXiv:2004.11838, 2020.

Related documents

• Just Talk Once: Communication-Efficient Split Federated LLM Fine-Tuning on Edge Devices#821496
• FlashCP: Load-Balanced Communication-Efficient Context Parallelism for LLM Training#267627
• Distributed Generative Inference of LLM at Internet Scales with Multi-Dimensional Communication Optimization#126492
• Federated Mixture-of-Experts Alignment on Mobile Edge Networks under Data Heterogeneity#1217420
• A multi-objective approach for communication reduction in federated learning under devices heterogeneity constraints#983903