FedLAB: Traceable Semantic Codebooks for Federated Multimodal Graph Foundation Learning Zekai Chen*, Kairui Yang*, Xuaner Chen, Xunkai Li, Xun Wu, Rong-Hua Li, Guoren Wang
arXiv:2606.32016v1 [cs.LG] 30 Jun 2026
Beijing Institute of Technology, Beijing, China
Abstract Multimodal graph foundation models aim to learn reusable knowledge from graphs enriched with text, images, attributes, and relational topology, thereby supporting diverse graphcentric and modality-centric tasks. In practice, however, such multimodal graphs are often distributed across decentralized clients, where raw contents and local structures cannot be centrally shared due to privacy constraints. This motivates federated multimodal graph foundation learning, which requires not only transferable representation learning but also intrinsic semantic traceability under strict data isolation. Existing methods usually exchange or store knowledge through parameters, prototypes, embeddings, or compact codebooks, which support optimization and transfer but do not explicitly expose how modality evidence, node semantics, and topology context jointly support predictions. To bridge this gap, we propose FedLAB, a traceable semantic codebook framework that organizes multimodal graph knowledge into typed hierarchical codebooks for modality evidence, node semantics, and topology context. FedLAB further refines these trace units through federated semantic barycenter pre-training while keeping raw multimodal contents and graph structures local. Extensive experiments on 10 benchmarks and 6 downstream tasks show that FedLAB improves over state-of-theart baselines by up to 7.53%, while preserving a native semantic trace interface.
1
Introduction
Multimodal graph foundation models (MM-GFMs), as multimodal extensions of graph foundation models (GFMs), aim to learn reusable knowledge from graphs enriched with text, images, attributes, and relational topology, thereby supporting diverse graph-centric and modality-centric tasks (Hou et al. 2022; Zhu et al. 2025; He et al. 2025). In practice, largescale multimodal graphs are often distributed across decentralized clients, where nodes, edges, attributes, labels, and multimodal contents cannot be centrally aggregated. Federated graph learning (FGL), especially multimodal FGL (MM-FGL), provides a practical learning paradigm for this setting (Li et al. 2024; 2026). This motivates federated multimodal graph foundation learning (Fed-MMGFM), whose goal is not merely task-specific federated optimization, but learning a reusable multimodal graph backbone that generalizes across clients, modalities, and downstream tasks under strict data isolation.
wicket text
Hot Wheels Star Wars Wicket Character Car
text codebook
toys car boys
visual codebook
Prediction label toy car
image
Figure 1: A traceable modality-evidence interface. FedLAB decomposes a multimodal prediction into text-side and image-side evidence codes rather than only producing a fused representation.
For MM-GFMs, semantic traceability refers to the ability of a foundation backbone to expose an inspectable support path from learned knowledge to task predictions, rather than only producing opaque outputs. This property is particularly important in Fed-MMGFM, where the same backbone is reused across heterogeneous clients and shifting data distributions; without traceable support, erroneous transfer, modality bias, and shortcut-driven predictions are difficult to diagnose. However, existing federated and foundationstyle graph learning methods typically exchange knowledge through parameters, gradients, prototypes, embeddings, or compact codebooks (Chen et al. 2026a; 2026b; Wu et al. 2025). Although effective for optimization and transfer, these mechanisms do not explicitly preserve how modality signals, node semantics, and topology jointly support predictions. Post-hoc graph explainers can identify influential features or subgraphs for a trained model (Ying et al. 2019; Luo et al. 2020), but they remain instance-specific and are not learned as reusable trace interfaces during federated pretraining. Therefore, Fed-MMGFM calls for an intrinsic trace interface jointly learned with the backbone and consistently maintained across clients. We define this missing capability as the semantic traceability gap in Fed-MMGFM. In this setting, a traceable foundation backbone should expose prediction support through three complementary interfaces. ❶ ModalityEvidence Traceability: which modality-specific signals are
2
Preliminaries and Problem Formalization
Federated Traceable MMGFM Objective. Following the federated multimodal graph setting in MM-OpenFGL (Li et al. 2026), we consider a server and K clients, where each client k owns a private multimodal graph Gk = (Vk , Ek , {Xkr }r∈M , Yk ) and a modality-availability mask Mk . Raw multimodal contents, labels, and graph structures remain local, while the server coordinates collaborative training. FedLAB aims to learn a federated multimodal graph foundation model that returns both a task prediction and an intrinsic semantic trace. The trace interface is instantiated by typed codebooks B = {B r }r∈M , B s , B t , (1) Figure 2: A traceable topology-context interface. FedLAB maps noisy graph neighborhoods to topology context codes and exposes the structural trace behind the prediction.
responsible for a prediction. ❷ Node-Semantic Traceability: which reusable node-level semantic units are consistently activated across heterogeneous clients. ❸ TopologyContext Traceability: how local structural contexts modulate or refine node semantics. Therefore, the central challenge is not only to learn a shared federated multimodal representation, but to endow it with a unified mechanism that makes modality evidence, node semantics, and topology context explicitly traceable and reusable. To address this problem, we propose FedLAB, a traceable semantic codebook framework for Fed-MMGFM. FedLAB instantiates the three trace interfaces with typed hierarchical codebooks: modality evidence codebooks capture modality-specific support signals, a node semantic codebook abstracts reusable node-level concepts, and a topology context codebook models structure-induced semantic variations. For each prediction, FedLAB outputs a semantic trace path composed of evidence codes, a node semantic code, a topology context code, and contribution scores. These trace units are refined across clients through federated semantic barycenter pre-training, where clients keep raw multimodal contents and graph structures local while only sharing model updates and aggregated posterior code statistics. Our Contributions. (1) New Challenge. We identify the semantic traceability gap in Fed-MMGFM, highlighting the lack of intrinsic, training-time mechanisms to trace modality evidence, node semantics, and topology context behind predictions. (2) New Framework. We propose FedLAB, a traceable semantic codebook framework that organizes multimodal graph knowledge into modality evidence, node semantic, and topology context codes, and refines them via federated semantic barycenter pre-training without sharing raw multimodal contents or local graph structures. (3) SOTA Performance. Across 10 multimodal graph datasets and 6 graph-centric and modality-centric tasks, FedLAB consistently outperforms state-of-the-art baselines by 4.38% on average while preserving a native semantic trace interface.
which correspond to modality evidence, node semantics, and topology context, respectively. For node v on client k, let Mk,v = {r ∈ M | Mk (v, r) = 1} denote its available modalities. FedLAB predicts ŷk,v and produces Tk (v) = {crk,v }r∈Mk,v , csk,v , ctk,v , Πk,v , (2) where crk,v , csk,v , and ctk,v are modality-evidence, nodesemantic, and topology-context trace units, while Πk,v records their contribution scores. The objective is to improve downstream utility while making these trace units reusable across clients, with only model updates and aggregated posterior code statistics shared with the server.
3
Related Work
Multimodal and Federated Graph Foundation Learning. GFMs shift graph learning from task-specific architectures toward reusable backbones learned through pre-training and adaptation (Hou et al. 2022). For multimodal graphs, recent studies further integrate relational topology with text, vision, or attributes through language-graph alignment, unified embedding spaces, or discrete graph tokenization (Zhu et al. 2025; He et al. 2025; Wang et al. 2025). In federated settings, MM-FGL and federated GFM methods learn transferable knowledge under data isolation through alignment signals, prototypes, anchors, or global codebooks (Li et al. 2026; Chen et al. 2026a; 2026b; Zhu et al. 2026; Wu et al. 2025). These methods improve optimization and transfer, but their shared knowledge is mainly represented as hidden states, embeddings, prototypes, or task-agnostic tokens, rather than as reusable semantic trace interfaces. Graph Explainability and Semantic Traceability. Posthoc graph explainers identify influential nodes, subgraphs, or features for trained GNNs (Ying et al. 2019; Luo et al. 2020). However, they are usually applied after training and provide instance-level explanations instead of reusable semantic units shared across clients. FedLAB targets a different goal in Fed-MMGFM: it learns typed trace units during federated multimodal foundation pre-training, so that modality evidence, node semantics, and topology context become inspectable and reusable parts of the backbone itself.
4 4.1
Methodology
4.3
Overview
We present FedLAB, a traceable semantic codebook framework for Fed-MMGFM. As shown in Fig. 3, FedLAB learns a shared backbone from private client-side multimodal graphs and returns both task representations and semantic trace paths. It replaces monolithic fused embeddings with typed semantic references for modality evidence, node semantics, and topology context. Clients generate hierarchical traces locally, while the server refines the backbone and trace vocabulary using model updates and aggregated posterior code statistics without exposing raw data.
4.2
Shared Trace Vocabulary
Motivation. In Fed-MMGFM, trace units must be locally faithful while remaining comparable across clients. However, heterogeneous modality distributions and graph structures make client-local traces difficult to align, and raw contents or topology cannot be shared. FedLAB therefore introduces a shared trace vocabulary as a common semantic reference space. Typed trace operator. FedLAB represents the shared vocabulary as typed codebooks: B = B r ∗ r ∈ M, B s , B t , B a = ba1 , . . . , b ∗ La a , (3) where a ∈ M ∪ s, t. The three codebook types correspond to modality-evidence, node-semantic, and topology-context trace units, respectively. For a type-a representation h, FedLAB first computes a posterior code assignment: q a (h) = arg min ⟨q, da (h)⟩ + τa Ω(q), q∈∆La
(4)
P where dai (h) = |h − bai | ∗ 22 , Ω(q) = i qi log qi , and τa controls assignment smoothness. Based on this posterior, FedLAB first obtains the relaxed trace representation and selected index: z̄ a (h) =
La X i=1
qia (h)bai ,ℓa (h) = arg max qia (h). i
(5)
It then produces the differentiable trace representation and selected trace unit: h i z a (h) = sg baℓa (h) − z̄ a (h) + z̄ a (h), ca (h) = baℓa (h) . (6) Here q a (h) records posterior code usage, z a (h) is used for differentiable optimization, ca (h) is the selected semantic trace unit, and sg[·] denotes stop-gradient (Van Den Oord, Vinyals, and others 2017). For simplicity, we denote the whole typed readout process as Qa (h; B a ) = (z a (h), ca (h), q a (h)).
(7)
At each communication round, the server broadcasts the current backbone and typed codebooks to selected clients, enabling local trace encoding in a shared semantic coordinate system while raw client data remain private.
Hierarchical Trace Encoding
Motivation. The shared trace vocabulary provides common semantic references, but it does not specify how prediction support is formed on each private client graph. In multimodal graphs, observed modalities first provide nodespecific support, these signals are composed into reusable node semantics, and local topology further contextualizes the semantic state. If compressed into a single fused representation, these roles become entangled and difficult to inspect. FedLAB therefore performs hierarchical trace encoding, turning local forward computation into an explicit path from modality evidence to node semantics and topology context. Evidence-semantic routing. For node v on client k, let mk,v denote the modality-availability mask induced by Mk (v, r), and let Mk,v be the set of observed modalities. For each observed modality, FedLAB projects the raw feature into the shared trace space and retrieves the corresponding modality-evidence trace unit: hrk,v = pr (fr (xrk,v )), r ∈ Mk,v , (8) r r (zk,v , crk,v , qk,v ) = Qr (hrk,v ; B r ). Here Qr (·) is the typed trace readout defined in Sec. 4.2; r crk,v records modality-side support, while qk,v is used for optimization and federated codebook refinement. FedLAB then composes modality evidence into a nodesemantic trace. It computes modality support scores and normalizes them over observed modalities: ⊤ r srk,v = wm σ Wm [zk,v ; hrk,v ] , (9) ρk,v = MSoftmax (srk,v )r∈M /τm , mk,v . where MSoftmax denotes masked softmax that excludes missing modalities. The routed modality evidence is summarized and then read from the node semantic codebook: X r r r urk,v = φm ([zk,v ; hrk,v ]), h̄m = ρ k,v k,v uk,v , r∈Mk,v (10) s s s m s (zk,v , ck,v , qk,v ) = Qs (gs (h̄k,v ); B ). Therefore, the node-semantic code is the reusable semantic unit activated after modality evidence has been composed, rather than a direct opaque fused embedding. Topology-context routing. FedLAB further contextualizes the node-semantic trace through local topology. Let χk,v denote a lightweight structural descriptor of node v, such as degree or local neighborhood statistics. For each neighbor u ∈ Nk (v), FedLAB scores its topology contribution by s s ηk,v,u = [zk,v ; zk,u ; χk,v ; χk,u ], ek,v,u = wt⊤ σ(Wt ηk,v,u ), βk,v = Softmax (ek,v,u )u∈Nk (v) /τt . (11) Here βk,v,u denotes the contribution of neighbor u to the topology-context trace. FedLAB summarizes topologysupported semantics and reads the topology-context trace by X N s s N z̄k,v = βk,v,u zk,u , h̄tk,v = φt zk,v , z̄k,v , χk,v , u∈Nk (v) t t (zk,v , ctk,v , qk,v ) = Qt (gt (h̄tk,v ); Bt ).
(12)
Topology
Text
1 2
3 4
2
1
1
Visual
2
3
1
Server Side
Client Side Backward Propagation
Multimodal Graph Topology
2
3 2
4 1
1
2
3
Loss
Topology Codebook
Semantic Codebook
Modality Codebook
Visual Representation
1
Model Parameters
Modality Semantic
Codebook Statistics
Topology
Textual Representation
Codebook
Codebook
Updated Codebook
Codebook Aggregation
1
Text
Trace Path at Node
Visual
Trace Record
=
Text Evidence Toy Car
+
Image Evidence
Modality Contribution
Semantic Evidence 1
1 2
2
Semantic Contribution
Topology Evidence 1 2
1 3
2
3
Topology
Contribution
Contribution Scores
Updated Model
1 0.2 0.5 0.3
Updated Codebook
(Text Evidence, Image Evidence, Semantic Evidence, Topology Evidence, Contribution Scores)
Fedavg GNN Model
Updated Model
Model Aggregation
Figure 3: Framework overview of FedLAB. The server maintains a shared backbone and typed trace codebooks, while clients encode private multimodal graphs into hierarchical trace paths. Model updates and aggregated posterior code statistics are uploaded to refine the backbone and shared trace vocabulary without exposing raw data. Thus, βk,v exposes which neighbors support the topologycontext trace, while ctk,v records the selected structureinduced semantic context. r Trace path readout. Let cm k,v = {ck,v | r ∈ Mk,v } denote the activated modality-evidence units. FedLAB returns the prediction together with a compact semantic trace path: s t Πk,v = (ρk,v , βk,v ), Tk (v) = cm k,v , ck,v , ck,v , Πk,v . (13) Since ρk,v and βk,v directly participate in modality composition and topology routing, the trace path is part of the prediction process and connects each output to modality evidence, node semantics, and topology context.
4.4
Semantic Barycenter Pretraining
Motivation. Hierarchical trace encoding makes each client’s prediction process inspectable, but its trace units may remain client-specific due to heterogeneous modalities and graph structures. For foundation learning, these local traces must be aligned into reusable cross-client semantic references. Since raw contents and topology cannot be shared, FedLAB performs semantic barycenter pretraining: clients preserve trace faithfulness locally, while the server refines the shared trace vocabulary using aggregated posterior code statistics. Local trace preservation. For modality r of node v, r FedLAB denotes its hierarchical trace state as ξk,v = r s t [zk,v ; zk,v ; zk,v ]. The local training objective is Lkloc = Lktask + λtr Lktrace + λcb Lkcb .
(14)
Here Lktask denotes the supervised or self-supervised task objective, while Lktrace preserves the three trace interfaces: Lktrace = Lkmod + λsem Lksem + λtop Lktop .
(15)
Specifically, Lkmod reconstructs modality-side information from the hierarchical trace state, Lksem aligns modality evidence with the activated node-semantic trace, and Lktop encourages topology-context traces to preserve local structural relations. These terms keep trace paths faithful to modality contents, reusable semantic units, and topology context without exposing raw client data.
To stabilize trace units and avoid code collapse, FedLAB regularizes the typed codebooks: X X X Lkcb = ℓaq (h) + η DKL (q̄ka ∥ Unif(La )) , a h∈Ha k
a
(16) where Hka is the set of local type-a representations, q̄ka is the empirical usage distribution, and ℓaq (h) is the standard quantization commitment loss. The usage regularizer prevents a small number of trace units from dominating the shared vocabulary. Barycenter refinement. After local training, client k uploads model updates and aggregated posterior trace statistics. For type a and code index i, FedLAB collects only posterior usage counts and posterior-weighted local centers: P X qia (h)h h∈Ha a a a k nk,i = qi (h), µk,i = . (17) nak,i + ϵ a h∈Hk
The server forms a usage-weighted semantic barycenter: µ̂a,t i =
X
a αk,i µak,i ,
nak,i . a k′ nk′ ,i + ϵ
a αk,i =P
k
(18)
The global trace unit is then refined by a,t ba,t+1 = ProjS (1 − ηc )ba,t . i i + ηc µ̂i
(19)
This update makes each global trace unit follow its crossclient posterior usage, aligning local modality, semantic, and topology traces into reusable foundation-level references. The overall federated objective is LFedLAB =
K X Nk k=1
N
Lkloc ,
N=
K X
Nk .
(20)
k=1
Thus, local trace preservation keeps trace paths faithful to private multimodal graphs, while semantic barycenter refinement makes the shared trace vocabulary reusable across clients without sharing raw data.
Experiments
5.1
79.19
79.71
80.02
79.91
79.61
79.17
78.89
79.75
80.66
81.26
81.17
80.52
79.79
79.17
0.05
80.27
81.66
82.71
82.70
81.61
80.37
79.44
0.10
80.40
81.95
83.15
83.15
82.11
80.48
79.43
0.15
80.11
81.44
82.49
82.42
81.45
80.17
79.28
0.30
79.42
80.26
80.96
80.94
80.30
79.49
78.87
0.50
78.85
79.21
79.60
79.53
79.19
78.85
78.85
4
8
12
16
24
32
64
83
−2 Dataset Bili Music QB Bili Movie
−3
80
−4 0.001
0.01
0.05
81
0.16
0.3
0.5
λ (tested values)
79
Codebook Size
Figure 4: Robustness analysis under hyper-parameter variations. λ controls semantic code regularization, while edgedrop ratio and topology codebook size control topology perturbation strength and code granularity.
Experimental Setup
Datasets. We evaluate FedLAB on ten multimodalattributed graph benchmarks (Li et al. 2026) spanning diverse domains: Toys, Grocery, Bili Music, DY, KU, Bili Food, QB, Bili Cartoon, Flickr30k, and SemArt (Ni, Li, and McAuley 2019; Zhang et al. 2024; Plummer et al. 2015; Garcia and Vogiatzis 2018). We construct federated scenarios by applying the Louvain community detection algorithm (Blondel et al. 2008) to partition each benchmark into 10 non-IID clients. Baselines. We organize the compared methods into five groups. (1) FL: FedAvg is used as the generic federated optimization baseline (McMahan et al. 2017). (2) MM-GNN: Fed-MGNet and Fed-MHGAT capture multimodal graph dependencies with GNN-based architectures. (3) MM-FL: FedMVP and FedMAC are representative multimodal federated learning methods. (4) GFM: Fed-GFT and FedGraphCLIP instantiate graph foundation model baselines. (5) FGL-GFM: FedGFM+ and FedBook further introduce foundation-model-style knowledge sharing in federated graph learning. We use the corresponding representative sources for these baseline families: MM-GNN (Kong et al. 2021; Jia et al. 2023), MM-FL (Che et al. 2024; Nguyen et al. 2024), GFM (Wang et al. 2024; Zhu et al. 2025), and FGL-GFM (Zhu et al. 2026; Wu et al. 2025). Downstream Tasks. Following the multimodal federated graph benchmark protocol, the evaluation covers two families of downstream tasks. The graph-centric family consists of node classification and link prediction, where we use Accuracy and AUC as metrics. The modality-centric family consists of modality matching, modality retrieval, G2text, and G2image generation, evaluated by AUC, Recall@5, ROUGE-L, and CLIP-S, respectively. We report mean test performance with standard deviation over repeated runs.
5.2
0 0.03
82
−1
Edge Drop
In this section, we provide a comprehensive empirical evaluation of FedLAB. We begin by introducing the experimental setup, and then seek to answer the following research questions: Q1: Does FedLAB achieve competitive performance over strong federated multimodal graph learning baselines across diverse benchmarks? Q2: What is the individual contribution of modality evidence, node semantic, and topology context codebooks in FedLAB? Q3: How robust is FedLAB under different hyper-parameter choices, client conditions, and training dynamics? Q4: What computation and memory efficiency does FedLAB achieve compared with existing baselines? Q5: Can FedLAB provide faithful and reusable semantic traces for multimodal federated graph predictions?
0
Relative Drop (%)
5
Overall Performance
To answer Q1, we compare FedLAB with representative federated multimodal graph learning baselines across all downstream tasks. As shown in Table 1, FedLAB consistently achieves the best results on the evaluated benchmarks, with an average improvement of 4.38% over the strongest competing baseline and a maximum gain of 7.53%. The improvement is observed across both graph-centric tasks, such
as node classification and link prediction, and modalitycentric tasks, such as modality matching, retrieval, and generation. These results indicate that typed semantic codebooks do not merely improve a specific task type, but provide a reusable traceable representation that benefits heterogeneous multimodal graph learning under federated settings.
5.3
Ablation Study
To answer Q2, we remove each codebook from FedLAB while keeping the remaining training protocol unchanged. As shown in Table 2, removing the topology, modality, and node codebooks reduces the average performance by 6.23%, 6.91%, and 7.81%, respectively. The node codebook contributes most to cross-client semantic reuse, the topology codebook supports structural reasoning, and the modality codebook preserves fine-grained cross-modal evidence, confirming that the three codebooks provide complementary signals.
5.4
Robustness Analysis
To answer Q3, we evaluate FedLAB from three robustness perspectives: hyper-parameter sensitivity, client scalability, and training dynamics. Hyper-parameter Sensitivity. We evaluate three key hyper-parameters: λ, edge-drop ratio, and topology codebook size Lt , which control semantic code regularization, topology perturbation strength, and topology-code granularity, respectively. As shown in Fig. 4, FedLAB remains stable across a broad range of choices, and the heatmap presents a wide high-performing region, suggesting that FedLAB does not rely on a brittle manually tuned configuration. Client Scalability. We further evaluate FedLAB when federated partitions become more fragmented. As shown in Fig. 5, the performance changes mildly as the number of clients increases, indicating that the shared traceable codebooks help preserve transferable semantics under heterogeneous partitions. Training Dynamics. We finally compare convergence on graph-centric tasks. As shown in Fig. 6, FedLAB converges smoothly and reaches better final performance than baselines, showing that traceable codebook learning preserves
Table 1: Overall performance on MM-FGL (mean ± std). Best results are bold and second-best are underlined. Node Classification (Acc)
Description Method
Toys
Grocery
Link Prediction (AUC) Bili Music
DY
Modality Matching (AUC) KU
Modality Retrieval (Recall@5)
Bili Food
QB
Bili Cartoon Flickr30k
SemArt
73.51±1.78 73.24±2.49 71.47±5.21 70.36±1.32 75.15±2.03 71.95±2.64 78.59±5.33 76.26±3.89 65.31±2.57
70.12±0.15 71.34±0.37 57.40±2.35 70.19±1.23 71.37±1.34 72.03±1.69 68.56±0.12 73.27±0.94 74.12±2.68
FedAvg 78.51±0.09 80.10±0.27 65.91±0.18 65.15±0.07 54.66±0.53 54.63±1.09 83.88±1.74 Fed-MGNet 67.02±0.56 71.48±1.02 68.15±1.32 67.18±0.25 57.61±0.49 57.43±1.00 84.38±3.19 Fed-MHGAT 77.29±1.57 78.06±2.02 68.46±2.01 63.80±1.26 55.90±1.39 57.90±1.78 87.74±4.89 FedMVP 78.98±0.07 79.93±0.43 67.00±0.47 64.75±0.73 58.02±0.47 58.97±0.99 87.11±4.97 FedMAC 78.57±0.25 80.06±0.61 63.30±0.51 64.51±1.55 56.74±0.37 57.40±1.26 88.65±2.75 Fed-GFT 78.97±0.31 80.30±0.22 64.53±0.19 63.68±0.53 55.80±0.80 57.67±1.01 90.15±1.80 Fed-GraphCLIP 76.34±0.49 73.95±0.35 66.68±0.13 64.10±0.06 55.63±0.32 58.04±1.27 91.23±2.17 FedGFM+ 77.48±0.36 73.81±1.67 67.07±0.12 63.54±0.22 57.31±0.37 58.23±1.21 84.19±1.65 FedBook 80.19±0.77 76.45±0.17 66.29±0.35 62.96±0.28 57.63±0.47 58.91±0.28 77.44±3.28
FL MM GNN MM FL GFM FGL GFM Ours
G2Text G2Image (ROUGE-L) (CLIP-S)
43.38±0.32 48.34±1.35 47.56±1.18 48.39±1.24 48.55±1.31 47.91±1.17 47.51±1.26 48.88±1.22 46.57±1.33
84.14±0.21 84.20±0.15 74.32±0.09 74.71±0.07 63.50±0.13 65.60±0.93 94.49±1.74 81.48±1.75 49.33±1.12 77.93±0.39
FedLAB
Table 2: Ablation study on all datasets (mean ± std). Each variant removes one key codebook to validate its contribution. Node Classification (Acc) Link Prediction (AUC) Modal Matching (AUC) Modal Retrieval (R@5) Toys
Grocery
Bili Music
DY
KU
Bili Food
Full FedLAB 84.14±0.21 84.20±0.15 74.32±0.09 74.71±0.07 63.50±0.13 65.60±0.93 w/o Topology Codebook 74.09±0.31 77.36±0.22 60.42±0.36 60.65±0.29 60.86±0.24 62.37±1.05 w/o Modality Codebook 74.26±0.27 76.89±0.25 64.10±0.33 61.37±0.31 55.42±0.31 60.94±1.18 w/o Node Codebook 76.48±0.34 63.12±0.41 64.07±0.35 60.88±0.30 57.18±0.36 64.85±1.11
80
Node Classification 81.8
82.1
80.7 78.9
80.4
7 clients
Link Prediction
85 80
78.8
75
75 71.5
69.9 69.6
70
69.8
66.7
70 66.3
65 60
65
Toys
ery Groc
B
ili M
usic
DY
60
Figure 5: Robustness under different numbers of clients. stable federated optimization.
5.5
Efficiency Analysis
To answer Q4, we compare communication payload, computation, space cost, and wall-clock time. As shown in Table 3, FedLAB requires additional communication and memory for traceable codebooks, but remains faster than FedBook. Fig. 7 further shows that FedLAB achieves the highest average performance with a practical time cost.
5.6
FedAvg
90
Semantic Traceability Validation
To answer Q5, we assess whether FedLAB produces prediction-supporting and reusable traces for modality evidence, topology context, and node semantics. Modality Evidence Traceability. For instance v, let EvK denote the top-K traced modality evidence codes and Xv
FedMVP
(a) Toys
G2Image
Bili Cartoon Flickr30k
SemArt
81.48±1.75 79.68±1.24 80.37±1.18 80.27±1.21
49.33±1.12 77.93±0.39 47.86±1.09 72.10±0.52 45.12±1.21 69.52±0.73 44.76±1.16 68.74±0.68
Fed-GFT
FedBook
84.14
75
80
Accuracy (%)
Node Accuracy (%)
85
5 clients
Link AUC (%)
3 clients
90
QB 94.49±1.74 92.03±1.03 92.63±0.96 91.24±1.10
G2Text
60 85 80 75 70
40 20 0
AUC (%)
Variant
10
20
74.32
70 65 60 55
20
0
FedLAB
(b) Bili Music
30
30
40
50
40
50
Communication round
50
0
10
20
30
40
Communication round
50
Figure 6: Convergence comparison across communication rounds on representative datasets. denote the full multimodal input. We define trace sufficiency as p(yv | EvK ) TS@K = Ev , (21) p(yv | Xv ) which measures the prediction confidence retained by selected trace units. As shown in Fig. 8, FedLAB preserves more confidence as K increases, indicating that its modality codes capture compact prediction-supporting signals. Topology Context Traceability. We next evaluate whether topology traces capture structural support behind predictions. Let Gv denote the local topology and Rrv denote the top-r percent removed topology trace units. We measure topology trace drop as TTD(r) = Ev [p(yv | Gv ) − p(yv | Gv \ Rrv )] ,
(22)
where a larger value indicates stronger structural decision support. As shown in the upper panels of Fig. 9, removing
Performance-efficiency trade-off
72.5 70.0 67.5 65.0
FedAvg
1.0
FedMVP Fed-GFT FedMAC Fed-GraphCLIP FedGFM+ Fed-MGNet Fed-MHGAT FedBook
1.5
2.0
2.5
3.0
66
45
45
30
30 QB
FedAvg FedMVP FedMAC FedBook FedLAB
Comm. Payload Total Ops Space Cost Time (Scalars) (FLOPs) (MB) (s) 2.96 × 106 7.72 × 106 3.96 × 105 3.07 × 108 6.66 × 108
1.47 × 109 1.49 × 109 1.69 × 109 2.33 × 1010 2.40 × 1010
63.9 507.7 83.2 844.2 1583.3
74.09 117.33 113.44 254.39 188.99
FedLAB-selected topology traces causes steeper confidence drops than post-hoc or random removal baselines (Ying et al. 2019; Luo et al. 2020; Huang et al. 2022). Reusable Node Semantic Codes. Finally, we test whether node-semantic codes form reusable references. For label c, let Vc denote its nodes and TcK be the top-K most frequent semantic codes in Vc . We define semantic code concentration as 1 X 1 X s SCC@K = I[cv ∈ TcK ], (23) |Y| |Vc | c∈Y
v∈Vc
where csv is the node-semantic code assigned to v. As shown in the lower panels of Fig. 9, FedLAB achieves higher SCC@K than non-traceable and random assignment baselines, suggesting that same-label instances are covered by compact reusable codes rather than scattered local representations.
6
Conclusion
In this paper, we study Fed-MMGFM from the perspective of semantic traceability. We identify that existing federated graph learning and graph foundation models can transfer useful representations, but their predictions remain difficult to audit because modality evidence, node semantics, and topology context are entangled in opaque embeddings.
Bili Cart. Bili Food Da tas DY ets Grocery
5
KU
3
-K Top
1
71
5 3
-K Top
1
Figure 8: Modality evidence traceability. Grocery
Drop (%)
50
Bili Music
Bili Cartoon
Bili Food
25 0
SCC@K (%)
Table 3: Efficiency and cost analysis. FedLAB incurs additional trace-codebook overhead for semantic traceability, while maintaining practical wall-clock time compared with foundation-style baselines.
63
60
Da Bili Mus. tas ets Toys
77
74 75
60
3.5
Figure 7: Performance-efficiency trade-off. The x-axis reports relative time cost against FedAvg, and the y-axis reports average performance across benchmarks.
73
75
Relative time cost (vs. FedAvg)
Method
80
72
FedLAB
75.0
Trace Suff. (%)
Average score
77.5
0
25 50 Removed (%)
0
Toys
80
25 50 Removed (%)
0
25 50 Removed (%)
DY
0
QB
25 50 Removed (%)
KU
40 0
1
5 Top-K
10
1
FedLAB
5 Top-K FedMVP
10
1 FedBook
5 Top-K
10
1
5 Top-K
10
Random
Figure 9: Semantic traceability validation on topology trace drop (upper) and semantic code concentration (lower). Motivated by this observation, we propose FedLAB, a traceable semantic codebook framework that organizes federated multimodal graph knowledge into typed modality evidence, node semantic, and topology context codebooks. FedLAB constructs shared semantic references from decentralized multimodal graphs and refines them through federated foundation pre-training while keeping raw modalities and local graph structures private. Extensive experiments on ten benchmarks and six downstream tasks demonstrate that FedLAB consistently improves overall performance, preserves robustness under different client and hyper-parameter conditions, and maintains practical efficiency. Further explainability validation shows that FedLAB provides faithful and reusable semantic traces, offering a promising foundation for auditable multimodal graph intelligence in federated environments.
References Blondel, V. D.; Guillaume, J.-L.; Lambiotte, R.; and Lefebvre, E. 2008. Fast unfolding of communities in large networks. Journal of statistical mechanics: theory and experiment 2008(10):P10008.
Che, L.; Wang, J.; Liu, X.; and Ma, F. 2024. Leveraging foundation models for multi-modal federated learning with incomplete modality. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, 401–417. Springer. Chen, Z.; Wu, X.; Li, X.; Sun, Y.; Li, R.-H.; and Wang, G. 2026a. Stage: Tackling semantic drift in multimodal federated graph learning. arXiv preprint arXiv:2605.11919. Chen, Z.; Zhang, M.; Xing, J.; Li, X.; Wu, X.; Li, R.-H.; and Wang, G. 2026b. Prism: Topology-aware cross-modal imputation for modality-deficient federated graph learning. arXiv preprint arXiv:2606.09301. Garcia, N., and Vogiatzis, G. 2018. How to read paintings: semantic art understanding with multi-modal retrieval. In Proceedings of the European Conference on Computer Vision Workshops, ECCV. He, Y.; Sui, Y.; He, X.; Liu, Y.; Sun, Y.; and Hooi, B. 2025. Unigraph2: Learning a unified embedding space to bind multimodal graphs. In Proceedings of the ACM on Web Conference 2025, 1759–1770. Hou, Z.; Liu, X.; Cen, Y.; Dong, Y.; Yang, H.; Wang, C.; and Tang, J. 2022. Graphmae: Self-supervised masked graph autoencoders. In Proceedings of the 28th ACM SIGKDD conference on knowledge discovery and data mining, 594– 604. Huang, Q.; Yamada, M.; Tian, Y.; Singh, D.; and Chang, Y. 2022. Graphlime: Local interpretable model explanations for graph neural networks. IEEE Transactions on Knowledge and Data Engineering 35(7):6968–6972. Jia, X.; Jiang, M.; Dong, Y.; Zhu, F.; Lin, H.; Xin, Y.; and Chen, H. 2023. Multimodal heterogeneous graph attention network. Neural Computing and Applications 35(4):3357– 3372. Kong, Z.; Sun, L.; Peng, H.; Zhan, L.; Chen, Y.; and He, L. 2021. Multiplex graph networks for multimodal brain network analysis. arXiv preprint arXiv:2108.00158. Li, X.; Zhu, Y.; Pang, B.; Yan, G.; Yan, Y.; Li, Z.; Wu, Z.; Zhang, W.; Li, R.-H.; and Wang, G. 2024. Openfgl: A comprehensive benchmark for federated graph learning. arXiv preprint arXiv:2408.16288. Li, X.; Ai, Y.; Zhu, Y.; Lu, H.; Zhang, Y.; Fu, G.; Fan, B.; Dai, Q.; Li, R.-H.; and Wang, G. 2026. Mm-openfgl: A comprehensive benchmark for multimodal federated graph learning. arXiv preprint arXiv:2601.22416. Luo, D.; Cheng, W.; Xu, D.; Yu, W.; Zong, B.; Chen, H.; and Zhang, X. 2020. Parameterized explainer for graph neural network. Advances in neural information processing systems 33:19620–19631. McMahan, B.; Moore, E.; Ramage, D.; Hampson, S.; and y Arcas, B. A. 2017. Communication-efficient learning of deep networks from decentralized data. In Artificial intelligence and statistics, 1273–1282. Pmlr. Nguyen, M. D.; Nguyen, T. T.; Pham, H. H.; Hoang, T. N.; Le Nguyen, P.; and Huynh, T. T. 2024. Fedmac: Tackling partial-modality missing in federated learning with crossmodal aggregation and contrastive regularization. In 2024
22nd International Symposium on Network Computing and Applications (NCA), 278–285. IEEE. Ni, J.; Li, J.; and McAuley, J. 2019. Justifying recommendations using distantly-labeled reviews and fine-grained aspects. In Proceedings of the Conference on Empirical Methods in Natural Language Processing and the International Joint Conference on Natural Language Processing, EMNLP-IJCNLP. Plummer, B. A.; Wang, L.; Cervantes, C. M.; Caicedo, J. C.; Hockenmaier, J.; and Lazebnik, S. 2015. Flickr30k entities: Collecting region-to-phrase correspondences for richer image-to-sentence models. In Proceedings of the IEEE International Conference on Computer Vision, ICCV. Van Den Oord, A.; Vinyals, O.; et al. 2017. Neural discrete representation learning. Advances in neural information processing systems 30. Wang, Z.; Zhang, Z.; Chawla, N. V.; Zhang, C.; and Ye, Y. 2024. Gft: Graph foundation model with transferable tree vocabulary. Advances in neural information processing systems 37:107403–107443. Wang, L.; Hassani, K.; Zhang, S.; Fu, D.; Yuan, B.; Cong, W.; Hua, Z.; Wu, H.; Yao, N.; and Long, B. 2025. Learning graph quantized tokenizers. In International Conference on Learning Representations, volume 2025, 97239–97260. Wu, Z.; Zhu, Y.; Li, X.; Qiu, Z.; Li, R.-H.; Wang, G.; and Zhou, C. 2025. Fedbook: A unified federated graph foundation codebook with intra-domain and inter-domain knowledge modeling. arXiv preprint arXiv:2510.07755. Ying, Z.; Bourgeois, D.; You, J.; Zitnik, M.; and Leskovec, J. 2019. Gnnexplainer: Generating explanations for graph neural networks. Advances in neural information processing systems 32. Zhang, J.; Cheng, Y.; Ni, Y.; Pan, Y.; Yuan, Z.; Fu, J.; Li, Y.; Wang, J.; and Yuan, F. 2024. Ninerec: A benchmark dataset suite for evaluating transferable recommendation. IEEE Transactions on Pattern Analysis and Machine Intelligence. Zhu, Y.; Shi, H.; Wang, X.; Liu, Y.; Wang, Y.; Peng, B.; Hong, C.; and Tang, S. 2025. Graphclip: Enhancing transferability in graph foundation models for text-attributed graphs. In Proceedings of the ACM on Web Conference 2025, 2183– 2197. Zhu, Y.; Li, X.; Jia, J.; Hu, M.; Wu, D.; and Qiu, M. 2026. Towards effective federated graph foundation model via mitigating knowledge entanglement. Advances in Neural Information Processing Systems 38:64599–64628.