Reputation-driven Cooperation in Lattice-based Decentralized Federated Learning through Evolutionary Game Theory Phuc Hoang Truong Huynh1,2 , Dung Tran Vinh1,2 , Khoa Duc Anh Lam1,2 , An Nghiem Nguyen Truong1,2 , Uyen Nha Tran Bui1,2 , Khang 1,2 1,2 , Bao Nguyen Le Gia , Minh Le Nguyen Nhat1,2 , Nguyen Dinh Manh Hong Duong3 , , The Anh Han4∗ , , Thi Ai Thao Nguyen1,2∗ , , and Le Hong Trang1,2⋆ Faculty of Computer Science and Engineering, Ho Chi Minh City University of Technology (HCMUT), Ho Chi Minh City, Vietnam {hthphuc.sdh241, dung.tranvinh2005, khoa.lamgeo07, an.nghiemngtruong, uyen.buiangiang07, khang.nguyen2006siu, bao.nguyenbaobk4105, minh.lecomsci, thaonguyen, lhtrang}@hcmut.edu.vn 2 Vietnam National University Ho Chi Minh City, Vietnam. 3 School of Mathematics, University of Birmingham, Birmingham, United Kingdom 4 School of Computing, Engineering and Digital Technologies, Teesside University, United Kingdom [email protected]
arXiv:2608.01197v1 [cs.AI] 2 Aug 2026
1
Abstract. Decentralized Federated Learning (DFL) has emerged as an optimal privacy-preserving solution; however, it remains vulnerable to opportunistic behaviors due to the absence of a central coordinator. Although evolutionary game theory (EGT) serves as a powerful framework for analyzing such behaviors, existing studies often assume that agents possess perfect rationality and maintain static strategies. To address these limitations, this article proposes a novel EGT framework designed to analyze strategic evolution and improve overall system performance. The primary contributions of this work are threefold: First, we model peer-to-peer (P2P) interactions on a lattice network structure under the assumption of bounded rationality. Second, we formulate a comprehensive payoff matrix that incorporates training costs, communication overhead, and cooperative rewards, while tailoring a strategy update rule that captures the dynamics of spatial propagation. Third, we integrate a reputation-based reward-and-punishment mechanism to effectively deter free-riding behaviors. The simulation results demonstrate that the framework significantly outperforms the baseline. Specifically, it increases average accuracy from approximately 70% to 82%, elevates cooperation frequency to approach 100% (compared to below 5% in the baseline), and drops accuracy variance from around 0.40 to 0.002, thereby accelerating uniform convergence and ensuring system stability. ⋆
Corresponding author
2
Phuc Hoang et at. Keywords: Decentralized Federated Learning · Reputation Mechanism · Evolutionary Game Theory · Lattice Topology · Cooperation Incentives · Payoff Matrix · Fermi Imitation
1
Introduction
In recent years, Machine Learning (ML) has gained widespread popularity due to its versatile applications across various domains [14]. However, the rapid proliferation of data-driven models has raised severe concerns regarding data privacy and security [1]. To address these challenges, Federated Learning (FL) has emerged as an alternative paradigm [17], enabling multiple devices or agents to collaboratively train a shared model while retaining their raw data on local client devices [4]. Despite its significant advantages in data protection, traditional FL—specifically Centralized Federated Learning (CFL)—remains vulnerable to inherent limitations, particularly the risk of a single point of failure and potential information leakage at the central coordinating server [29]. To overcome these bottlenecks, Decentralized Federated Learning has surfaced as a promising solution [8]. By completely eliminating the central server and allowing agents to autonomously adjust their strategic behavior over a Peerto-Peer (P2P) network [6], DFL enhances the overall security and reliability of the system [7]. Nevertheless, this shift toward a fully decentralized architecture introduces complex behavioral challenges. DFL systems rely fundamentally on voluntary participation and mutual cooperation among agents. Within this environment, self-interested agents continuously modify their strategies to maximize individual utility, frequently giving rise to opportunistic behaviors such as “free-riding” [20] and malicious poisoning attacks [3] intended to disrupt model performance. Consequently, DFL is susceptible to model degradation, slow convergence rates, and, in severe cases, total network collapse [13]. To mitigate these behavioral and cooperative obstacles in DFL, the application of Evolutionary Game Theory (EGT) becomes indispensable [25]. Conventional game-theoretic approaches often rely on the unrealistic assumption of fully rational agents. In contrast, EGT offers a more realistic framework by modeling agents with bounded rationality whose strategies dynamically evolve over time based on accumulated payoffs [26,2]. Crucially, a vast majority of existing EGT studies in FL assume well-mixed populations with uniform random interactions. In real-world DFL scenarios, however, agents interact exclusively with their direct topological neighbors [12]. The relative scarcity of literature modeling interactions on specific spatial topographies, such as structured lattices [23], leaves a significant research gap regarding the mechanisms of spatial clustering for cooperation [22,18]. Therefore, integrating EGT with a spatial lattice topology plays a pivotal role in analyzing behavioral dynamics and enhancing the sustainable performance of DFL systems. This paper proposes an evolutionary game-theoretic framework to investigate the behavioral dynamics of agents (nodes) in DFL. The primary contributions of this study are threefold:
Reputation-Driven Cooperation in Lattice-Based EGT-DFL
3
– Modeling agent interactions in DFL: we construct a lattice-based network topology grounded in EGT. To the best of our knowledge, this is one of the first studies to explicitly model P2P interactions among agents under the assumption of bounded rationality. – Payoff function design and strategy update rules: we formulate a novel model that incorporates local training costs, communication overheads, and the mutual benefits derived from collaborative training. In addition, we refine the local strategy update rules to capture the imitation and dynamics of strategy propagation across neighboring nodes in the network. – Reputation-based incentive mechanisms: To mitigate the free-riding problem inherent in DFL networks, we incorporate a reputation-based incentive scheme to foster mutual cooperation. Based on this reputation metric, the system dynamically rewards agents maintaining high reputation scores while penalizing those with low scores. The rest of the paper is organized as follows. Section 2 introduces related work. Section 3 provides background knowledge. Section 4 proposes the methodology, including architecture, modeling and workflow. Section 5 evaluates its application and effectiveness by simulation. Finally, Section 6 concludes the paper.
2
Related Work
Federated Learning was initially introduced to address the escalating data privacy and security concerns in traditional machine learning. However, FL is highly susceptible to opportunistic behaviors, most notably the “free-riding” problem, where resource-constrained nodes skip local training but absorb updated models from others [28]. Early attempts to mitigate this issue primarily relied on traditional incentive mechanisms grounded in classical game theory, such as Stackelberg games or contract theory [27]. For instance, Xu et al. [27] introduced a Tripartite Evolutionary Game to model interactions among a central server, organizers, and collaborators to address partial free-riding. Similarly, Yang et al. [28] utilized a system dynamics-based multi-player evolutionary game to stabilize FL systems by regulating central server rewards and punishments. While effective, these early centralized approaches assumed perfect rationality and global observability, constantly facing single-point-of-failure risks and privacy bottlenecks inherent to the central coordinating server. To eliminate the dependency on a central server, the field naturally evolved toward DFL, allowing nodes to aggregate models autonomously via P2P gossip protocols. In this new decentralized context, nodes operate with bounded rationality, dynamically adjusting their strategies based on observed payoffs rather than global information. To model this behavioral shift, researchers increasingly adopted EGT. For example, Röder et al. [19] modeled DFL client interactions using the Iterated Prisoner’s Dilemma, employing Moran sampling as an evolutionary incentive mechanism to systematically exclude non-cooperative clients from the training process. Although this transition to EGT effectively addressed
4
Phuc Hoang et at.
bounded rationality, these strategies were primarily evaluated in well-mixed populations or simplistic topologies, often overlooking the physical and structural constraints of real-world edge devices. Recognizing the limitations of well-mixed assumptions, recent studies have begun to emphasize the impact of structured populations and spatial topologies on cooperative behavior. In real-world DFL, edge devices are bound to specific network structures and interact exclusively with their direct topological neighbors. Duong et al. [5] demonstrated through agent-based simulations on square lattices that local, neighborhood-based interventions significantly outperform global, population-wide schemes in terms of both social welfare and cost-efficiency. Their work highlighted a critical evolutionary phenomenon: local interactions induce spatial clustering of cooperators, enabling cooperative behaviors to survive and propagate even when local payoffs inherently favor defection [5]. Despite the rich historical progression from centralized EGT models [28] to decentralized EGT [19] and spatial cooperation [5], a comprehensive framework that explicitly models fully decentralized P2P interactions on a spatial lattice remains underexplored. Existing DFL incentive models lack the integration of localized imitation dynamics (e.g., the Fermi update rule) with practical FL expenditures, such as communication overhead and local training costs. To bridge this historical gap, this paper proposes a lattice-based Evolutionary-Game-Theoretic DFL architecture. By integrating a reputation-based incentive mechanism and neighborhood gossip averaging, our framework fosters the spatial clustering of cooperators, effectively mitigating the free-riding problem without relying on any central authority.
3
Background
3.1
Decentralized Federated Learning and the Free-Riding Problem
Unlike traditional server-centric FL, DFL eliminates the single point of failure and communication bottlenecks by enabling edge nodes to collaborate via P2P network topologies [16]. In a DFL architecture, global model consensus is achieved purely through local computation and iterative neighborhood interactions governed by Gossip Averaging protocols [12]. Formally, within our evolutionary game framework over a graph G = (V, E), (k) each node i ∈ V holds a local model weight vector wi at iteration round k. The primary objective of average consensus in DFL is for every node’s model parameters to converge toward the exact global arithmetic mean: n
w̄ =
1 X (0) w n i=1 i
(1)
In a synchronous neighborhood averaging scheme over a regular network topology - specifically, a periodic square lattice where each node i is connected to |Ni | = 4 direct neighbors—all nodes concurrently mix their local model
Reputation-Driven Cooperation in Lattice-Based EGT-DFL
5
weights with their open neighborhood. The state transition for an individual node i at iteration step k + 1 is locally computed as: X 1 (k) (k) (k+1) wj . (2) wi = wi + 5 j∈Ni
Compactly, across the entire network, the global state transition is governed by a doubly-stochastic mixing matrix W (k) ∈ Rn×n [12]: w(k+1) = W (k) w(k) ,
(3)
where the entries of W (k) for this 4-neighbor square lattice topology are defined as: ( 1 , if j = i or j ∈ Ni (k) (4) Wij = 5 0, otherwise (k)
(k)
This matrix construction naturally satisfies Wij ≥ 0, Wij = 0 for (i, j) ∈ / E, W (k) 1 = 1, and 1T W (k) = 1T . This guarantees that as iterations progress, the matrix product converges to limk→∞ W (k−1) . . . W (0) = J (where J = n1 11T ), (k) driving wi → w̄ for all nodes i [12]. However, the P2P nature and non-excludable neighborhood mixing step inherently expose DFL to the Free-riding problem [20]. Free-riders are selfinterested or resource-constrained nodes that aim to reap the benefits of a welltrained global model without contributing their fair share of computational resources. To contribute to global performance, a cooperative node (Si = C) executes local model training (e.g., stochastic gradient descent) prior to gossip aggregation, expending computational and energy cost c > 0. In contrast, a defecting node (Si = D) skips local training (c = 0) or sends stale model weights, yet (k) still receives and averages the updated state vectors wj from its neighbors via W (k) . Formally, the net fitness payoff Pi of node i is modeled as a function of the global model performance benefit B(·) derived from neighborhood aggregation minus its incurred operational cost ci : (k) Pi = B {wj }j∈Ni ∪{i} − ci (5) where ci = c if Si = C, and ci = 0 if Si = D. Because B(·) is shared locally due to the non-excludable nature of gossip protocols, a defecting node avoids the cost c while still exploiting the accuracy gains provided by its cooperative neighbors. Consequently, for equivalent neighborhood inputs, a free-riding node achieves a strictly higher net fitness payoff than its cooperative counterparts: Pi (D) > Pi (C)
(6)
6
3.2
Phuc Hoang et at.
Evolutionary Game Theory (EGT) on Network Topologies
Game Formulation and Payoff Structure. Game theory provides the analytical framework to understand and predict strategic decision-making in interactive scenarios marked by conflicting priorities among players [22,26]. For EGT, unlike classical game theory, which assumes fully rational players with complete information, EGT models boundedly rational agents who iteratively adjust their strategies based on observed payoffs over time [9]. We model the pairwise interaction between edge devices in DFL as a symmetric two-player normal-form game G = (A, AT ) [22]. Each node i adopts a strategy Si (t) ∈ {C, D} at round t, where C represents Cooperation (executing local Stochastic Gradient Descent (SGD) training) and D represents Defection (free-riding or transmitting uninformative updates). The expected payoff matrix A governing the interaction between node i and neighbor j under strategy profile (Si , Sj ) is formulated as: ! ! UCC UCD Q(2) − c Q(1) − c A= = (7) UDC UDD Q(1) 0 where Q(k) denotes the aggregated model performance gain derived from k actively contributing neighbors (k ∈ {0, 1, 2}), satisfying Q(2) > Q(1) > Q(0) = 0. The parameter c > 0 represents the local resource expenditure per training round (e.g., energy, compute power, and bandwidth). Topological Network Interactions. In traditional well-mixed populations, players interact uniformly at random. However, in DFL, edge devices are bound to structured networked topologies (e.g., regular lattices, scale-free, or smallworld networks), interacting exclusively with their direct topological neighbors. Let Ni denote the set of all neighbors of node i. The cumulative utility Pi (t) obtained by node i at round t is the sum of pairwise payoffs with its neighbors: X Pi (t) = u(Si (t), Sj (t)), (8) j∈Ni
where u(Si (t), Sj (t)) is the pairwise payoff derived from matrix A. This local interaction induces spatial clustering of cooperators, enabling mutual cooperation to survive against free-riders even when the local payoff favors defection (UDC > UCC ) [22] 3.3
Strategy Adoption and Fermi Imitation Rule
To model how edge nodes adapt their behavioral strategies over consecutive training rounds, we incorporate the pairwise imitation dynamics from spatial evolutionary game theory [23]. Under bounded rationality, a node i does not possess global information; instead, at the end of round t, it randomly selects a direct neighbor j ∈ Ni as a role model, to evaluate performance and behavioral update. Node i adopts neighbor j’s strategy Sj (t) for the subsequent round t + 1 according to the Fermi update rule borrowed from statistical physics [23]: 1
W (Si ← Sj ) = 1+e
Pi (t)−Pj (t) K
(9)
Reputation-Driven Cooperation in Lattice-Based EGT-DFL
7
where Pi (t) and Pj (t) denote the cumulative payoffs acquired by nodes i and j, respectively. The parameter K > 0 represents the selection intensity, quantifying the level of environmental noise or decision-making irrationality: – As K → 0+ , strategy adaptation becomes purely deterministic, where node i strictly imitates neighbor j if and only if Pj (t) > Pi (t). – As K → ∞, decision-making degrades into a random coin toss (W ≈ 0.5), ignoring payoff differentials due to severe noise or perception errors. By regulating K, the Fermi dynamics effectively capture how cooperative behavior (e.g., executing local SGD in federated learning) forms spatial clusters and survives against free-riders under varying environmental noise levels.
4
Methodology
4.1
Lattice-based Evolutionary-Game-Theoretic Decentralized Federated Learning
We construct a DFL system over a two-dimensional lattice network, which is composed of overlapping clusters consisting of one central node, referred to as the focal node, and its four immediate neighbors. These clusters are regarded as overlapping because each node simultaneously acts as the focal node of its own local neighborhood and as a neighbor of up to four other focal nodes. Each node maintains its own model and possesses local data as well as heterogeneous computational capabilities. The nodes collaboratively participate in training, exchange model parameters, and merge model weights with their neighbors through a peer-to-peer mechanism. Therefore, the system removes the dependency on a single centralized aggregator [15,21,10]. The exchange and merging processes propagate across clusters throughout the network and are repeated over multiple rounds until model convergence is achieved. During this process, under the influence of EGT, each node adopts an individual strategy s ∈ {C, D}, where the node decides whether to share (Cooperator : C) or not to share (Defector : D) its model parameters with neighboring nodes. This decision is made based on the benefit obtained and the cost incurred when a specific strategy is adopted. Since the conditions of individual nodes are heterogeneous, their strategic decisions may differ, which makes the problem more realistic and less idealized than the conventional formulation of DFL. All nodes participating in the DFL system seek to obtain a sufficiently good model for solving their own local learning tasks. Therefore, they are willing to perform training whenever the received model weights are improved. However, whether they contribute their own effort to the collective community depends on their strategic behavior in the game. A payoff matrix is established to represent this trade-off and to provide a basis for explaining the strategic choices of each node.
8
Phuc Hoang et at.
Figure 1: Overall architecture of the proposed lattice-based decentralized federated learning system, where nodes perform local training, peer-to-peer gossip averaging, reputation-aware payoff evaluation, and evolutionary strategy updating over a two-dimensional lattice network.
4.2
Evolutionary-Game-Theoretic Modeling
When participating in training, a focal node may face four interaction scenarios with respect to a neighboring node. 1. The focal node is a Cooperator and the neighbor is also a Cooperator (CC). This represents the most desirable bilateral interaction, in which both nodes share the model weights obtained from their own training processes. This interaction provides a substantial improvement in the quality of the shared model, represented by +∆Q. Although both nodes incur communication overhead (−O) and training cost (−T ) when retraining the model after receiving improved parameters, both of them are rewarded with positive reputation gains (R+ ).
Reputation-Driven Cooperation in Lattice-Based EGT-DFL
9
Figure 2: Payoff-based strategy assessment of a focal node when interacting with a neighboring node. The focal node compares its accumulated payoff with that of the neighbor, evaluates the potential benefits and costs associated with cooperation or defection, and probabilistically updates its strategy according to the relative payoff difference.
2. The focal node is a Cooperator whereas the neighbor is a Defector (CD). The Cooperator is a benevolent participant that is willing to share for the benefit of the community, thereby incurring the communication overhead (−O). However, such cooperation may not always be reciprocated by the neighbor. In some cases, the reward does not come from the expected interacting node, but rather from community-level recognition of cooperative behavior, represented by reputation benefit (R+ ). 3. The focal node is a Defector whereas the neighbor is a Cooperator (DC). In this case, the focal node behaves selfishly by receiving benefits without contributing back. The objective of maximizing individual utility prevents the focal node from sharing its own model parameters with the neighbor that has helped improve its model quality. This corresponds to the free-riding problem. The focal node trains for its own benefit and only returns the same parameters that it previously received. It accepts the associated costs (−T, −O) and the reputation penalty (R− ) because the benefit (+∆Q) obtained from free-riding may still be attractive. 4. The focal node is a Defector and the neighbor is also a Defector (DD). In this case, both nodes refuse to share with each other and accept reputation degradation (R− ) because they assume that the bilateral interaction does not provide sufficient benefit.
10
Phuc Hoang et at.
Payoff Matrix. EGT adapts game theory to evolving populations, where strategies are dynamically adopted based on each participant’s relative fitness, or payoff [18,24]. Accordingly, a 2 × 2 payoff matrix is constructed based on the two strategies s ∈ {C, D}, representing the strategic trade-off between sharing and not sharing model parameters for improving the collective model. In this behavioral analysis, each payoff value quantifies the effectiveness of a bilateral interaction through carefully computed numerical values, which support each node in selecting an appropriate strategy in each training round. The payoff rules governing two interacting nodes are summarized in Table 1.
Table 1: Evolutionary-Game-Theoretic Assessment of the Baseline Payoff Matrix Strategy
Cj
Dj
Ci
+ α ∆QCi Cj − TCi − OCi −OCi
Di
β [∆QCi ]+ − TDi − ODi
0
Note: Although Di doesn’t share its model weights, it still propagates the unmodified weights received from Cj (incurring −ODi ) and performs local updates (incurring −TDi ) to improve its model.
By observing the payoff matrix in Table. 1, the most attractive factor that encourages both the focal node and its neighbor to form a genuinely cooperative interaction is the improvement in the quality of the shared model, +∆Q, which is amplified by the synergy coefficient α > 1. This benefit is assigned to participants that make positive contributions according to their model-sharing ratio and their marginal contribution to the quality of the collective model. In contrast, the coefficient 0 ≤ β ≤ 1 characterizes the free-riding effect. Specifically, β represents the fraction of the model-quality benefit that a defecting focal node can retain by receiving the model parameters shared by a cooperative neighbor without reciprocally sharing its own updated parameters. A larger value of β indicates that the defector can appropriate a greater proportion of the neighbor’s contribution, whereas a smaller value indicates that the exploitable benefit is limited. Accordingly, β = 1 represents complete appropriation of the received model-quality gain, while β = 0 indicates that free-riding provides no effective model-quality benefit. Unlike α, which captures the complementary value generated through mutual cooperation, β does not represent a productive contribution; rather, it quantifies the retained benefit obtained from one-sided cooperation. Let i denote a focal node and j ∈ Ni denote one of its neighboring nodes. Let Q(w) denote the quality of model w (e.g., measured by its accuracy on a shared validation set or by the negative value of its loss). The marginal contribution of node i at round t can be measured using the leave-one-out method as follows: (t) (t) (t) ∆Qi = Q wCt − Q wCt \{i} ,
Reputation-Driven Cooperation in Lattice-Based EGT-DFL
11
(t)
where Ct denotes the set of contributing nodes at round t, wCt is the model (t)
aggregated from all nodes in Ct , and wCt \{i} is the model aggregated after excluding the contribution of node i. To prevent model updates that degrade model quality from receiving a reward, the effective quality contribution of node i is defined as h i+ n o (t) (t) (t) qi = ∆Qi = max 0, ∆Qi . (t)
At generation t, each node adopts a strategy si ∈ {C, D}, where C and D denote cooperation and defection, respectively. Let [x]+ ≜ max{0, x} denote the positive-part operator. The pairwise payoff received by focal node i from its interaction with neighbor j is defined as i+ h (t) (t) (t) (t) (t) α ∆Q − T − O , s , s = (C, C), i j Ci Cj Ci Ci (t) (t) (t) si , sj = (C, D), −OCi , (t) (t) (t) (10) πij si , sj = h i+ (t) (t) (t) (t) (t) β ∆Q − T − O , s , s = (D, C), i j Ci Di Di (t) (t) 0, si , sj = (D, D). (t)
Here, ∆QCi Cj denotes the model-quality improvement resulting from mutual (t)
cooperation between nodes i and j, while ∆QCi denotes the model-quality benefit obtained by the focal node in a DC interaction. The coefficients α and β represent the cooperative synergy and free-riding effects, respectively. Moreover, (t) (t) Ti and Oi denote the training and model-exchange costs. The accumulated payoff of focal node i is X (t) (t) (t) (t) Πi = πij si , sj , (11) j∈Ni
and its evolutionary fitness is defined as (t)
fi
(t)
≜ Πi .
(12)
Reputation. In addition to the quality of the shared model +∆Q, reputation is another important mechanism for guiding node behavior toward cooperative sharing that benefits the DFL system. The utility constrained by reputation is determined according to the reputation score accumulated over training rounds. A higher accumulated reputation score leads to a larger additional incentive, whereas a lower score reduces the benefit obtained by the node [11,31,30]. Furthermore, the integration of reputation into the payoff framework serves another purpose. In the DFL context, the stopping condition of the system is not that the entire network reaches a homogeneous strategic state, but rather that the model reaches convergence. Therefore, even when the entire network reaches the best cooperative state C, or falls into the worst conservative state
12
Phuc Hoang et at.
D, the DFL process must continue if the shared model has not yet achieved an acceptable level of consensus quality. The design of the reputation score provides a mechanism for shifting the balance in favor of a node surrounded by an absorbing state, such as an alldefector neighborhood, or for imposing an unavoidable penalty that discourages a node from remaining trapped in a conservative strategy when its utility is decreasing. Reviving a node through a sharing-oriented strategy can generate a resonant propagation effect, thereby creating favorable conditions for the model quality to continue improving gradually toward convergence. Table 2: Evolutionary-Game-Theoretic Assessment of the Reputation-based Payoff Matrix Strategy Ci Di
Cj Dj + + + RC (α ∆QCi Cj − TCi − OCi ) RC (−OCi ) i i − RD (β [∆QCi ]+ − TDi − ODi ) i
− RD i
The reputation value and its round-by-round update are defined as xij δn , x ≥ 0, R(xij ) = xij , x < 0, δn ( min{δn−1 + r, δmax } , if sij = C, δ0 = 1, δn = max{δn−1 − r, δmin } , if sij = D. +,(t)
When reputation is incorporated (Table. 2), let RCi
denote the positive repu−,(t)
tation modifier associated with cooperative behavior, and let RDi denote the reputation modifier associated with defective behavior. The reputation-adjusted pairwise payoff is defined as h i+ +,(t) (t) (t) (t) (t) (t) RCi α ∆QCi Cj − TCi − OCi , si , sj = (C, C), (t) (t) R+,(t) −O(t) , si , sj = (C, D), Ci Ci (t) (t) (t) π eij si , sj = h i+ −,(t) (t) (t) (t) (t) (t) R β ∆Q − T − O , s , s = (D, C), i j Di Ci Di Di (t) (t) R−,(t) , s , s = (D, D). i j Di (13)
Reputation-Driven Cooperation in Lattice-Based EGT-DFL
13
Equivalently, for the first three interactions, (t)
+,(t) (t)
(14)
π eij (C, D) = RCi πij (C, D),
(t)
+,(t) (t)
(15)
(t) −,(t) (t) π eij (D, C) = RDi πij (D, C).
(16)
π eij (C, C) = RCi πij (C, C),
Since the base payoff of the (D, D) interaction is zero, the proposed matrix −,(t) directly assigns the reputation-dependent term RDi to this interaction. The accumulated reputation-adjusted payoff is therefore X (t) (t) (t) e (t) = π eij si , sj , (17) Π i j∈Ni
and the corresponding evolutionary fitness is (t) e (t) . fei ≜ Π i
4.3
(18)
Evolutionary Game Theory-Based Training Workflow
To strike a balance between individual model performance improvement and dynamic strategic adaptation, the proposed ÈFL framework organizes its runtime execution into discrete iterative rounds. The system operates on a 5 × 5 grid population where each spatial node acts as both a local machine learning agent and a strategic player in an evolutionary game. Depending on its current strategy - either Cooperator (C) or Defector (D), a node determines whether to contribute its parameter updates to its immediate spatial neighborhood. The complete operational workflow of a single execution round is detailed below and illustrated in Fig. 3. Workflow Breakdown Step 1: Initialization At generation g = 1, the Simulation Coordinator sets up the network N , strategy space S ∈ {C, D}, payoff matrix M, Fermi intensity β, max generations G, and max rounds R. Each focal node i is assigned an initial strategy Si ∼ Uniform({C, D}) and a set of 4 spatial neighbors Ni . Step 2: Generation Loop Execution The evolutionary process runs for generation g = 1 . . . G or until DFL converges. At the beginning of each generation g, cumulative payoffs are reset (Πi ← 0, ∀i ∈ N ). Each generation consists of R training rounds (r = 1 . . . R): Step 3: Neighborhood Exchange Phase (Round r) For round r = 1, node i performs initial self-training to obtain weight wi . For subsequent rounds (r > 1), nodes bypass self-training and directly load the consolidated weights wi,consolidated from round r − 1. The parameter exchange strictly adheres to node strategies (Si , Sj ):
14
Phuc Hoang et at.
Figure 3: End-to-end workflow of the proposed EGT-enabled DFL architecture.
Reputation-Driven Cooperation in Lattice-Based EGT-DFL
15
– Focal Node Strategy: If focal node i is a Cooperator (C), it transmits its current weight wi to its immediate neighbor node j. Conversely, if node i is a Defector (D), it suppresses its parameters by sending returning unupdated weights. – Neighbor Response Strategy: If neighbor node j is a Cooperator (C), it combines the received weight wi with its own model parameters and returns the combined weight. If neighbor node j is a Defector (D), it refuses parameter sharing and returns the received weight unchanged. Step 4: Local Adaptive Training and Weight Consolidation Upon receiving returned weights from its spatial neighborhood Ni , focal node i executes local adaptive training to derive up to six performance evaluation averages: – Self-Training Average: Obtained from the initial self-training pass (or prior state). – Neighbor Evaluation Averages (up to 4): Node i trains separately on each weight returned from its 4 neighbors to calculate neighbor-specific performance averages and performance differentials ∆Average(Gji ). – Consolidated Average (1): All received weights are aggregated together into a unified representation. The Local ML Trainer executes a final training pass on this aggregated model, returning the updated consolidated weight wi,consolidated and final consolidated performance metric. Step 5: Game-Theoretic Payoff Accumulation Following model consolidation in P (r) round r, focal node i calculates its round payoff Ui = j∈Ni M(Si , Sj ) and updates its cumulative payoff: (r)
Πi ← Πi + Ui
(19)
Step 6: Fermi Strategy Update After completing R rounds in generation g, each Πi . Node i then updates its strategy node i computes its average fitness fi = Ractual stochastically for the next generation: – Node i uniformly samples one neighbor j ∼ Uniform(Ni ). – Node i computes the Fermi probability P (Si ← Sj ) to decide whether to adopt neighbor j’s strategy: P (Si ← Sj ) =
1 1 + e−β·(fj −fi )
(20)
where β ≥ 0 represents the selection pressure. – Node i updates its strategy Si accordingly for the next generation g + 1. Step 7: State Persistence and Convergence Check Finally, node i retains wi,consolidated as starting inputs for the next iteration. The workflow continues until DFL reaches convergence.
16
Phuc Hoang et at.
Algorithm 1 The ith generation evolutionary strategy under EGT in DFL Initialization: Node set N , strategy space S ∈ {C, D}, payoff matrix A, payoff U , fitness Π, Fermi intensity β, generations G, max rounds R. ▷ Randomly assign strategies to all nodes in the initial network. 1: for each node i ∈ N do 2: Assign random strategy Si ∼ Uniform({C, D}) and set neighborhood Ni (|Ni | = 4) 3: Calculate the pay-off value of node i in interactions with Ni neighbors 4: end for 5: for generation g = 1 to G or until DFL converges do 6: Reset accumulate fitness Πi ← 0, ∀i ∈ N 7: ▷ Calculate node’s payoff 8: for round r = 1 to R do 9: for each node i ∈ N do P (r) 10: Calculate round payoff Ui ← j∈Ni A(Si , Sj ) 11: 12: 13: 14: 15: 16: 17:
(r)
Calculate accumulate fitness Πg ← Πi + Ui end for end for
▷ Strategy evolution for each node i ∈ N do Πg Compute average fitness fi ← Ractual Choose randomly a neighbor j ∼ Uniform(Ni ) (according to social event) 18: Update strategy Si through Fermi probability to decide whether the strategy is updated in next generation: 1 P (Si ← Sj ) = 1 + e−β·(fj −fi ) 19: end for 20: end for
Reputation-Driven Cooperation in Lattice-Based EGT-DFL
17
Figure 4: Evolution of node strategies across successive generations under the Fermi imitation mechanism. At each generation, a focal node compares its accumulated payoff with that of a randomly selected neighboring node and probabilistically adopts the neighbor’s strategy according to their payoff difference, thereby driving the local transition between cooperation and defection over time.
5
Experiments and Analysis
5.1
Simulation Settings
To evaluate the proposed lattice Evolutionary-Game-Theoretic DFL framework, we construct a simulation environment in which each node acts both as a local learning agent and an evolutionary-game player. The experiment evaluates the evolution of node strategies and its effect on model performance. The DFL network consists of a n × n periodic square lattice containing n2 nodes. Each node interacts exclusively with its four immediate neighbors. At each round, a Cooperative (C) node contributes its model parameters to neighboring interactions, whereas a Defective (D) node limits its contribution while still receiving information from its neighbors. After the neighborhood exchange and local training processes, each node calculates its cumulative payoff according to the payoff matrix defined in Section 4.1. The node then randomly selects one neighboring node and applies the Fermi imitation rule to determine whether its strategy should be updated.
18
Phuc Hoang et at.
The main simulation parameters are summarized in Table 3.
Table 3: Common simulation settings Parameter
Setting
Network topology n × n periodic square lattice Lattice size n = 50 Neighborhood size 4 Initial strategy distribution Uniform random (P (C) = P (D) = 0.5) Maximum simulation rounds 50 Fermi selection parameter β 0.3 Training cost T 3 Communication cost O 0.5 Synergy coefficient δ δmax = 1.5, δmin = 0.5 Reputation reward/penalty r = 0.1 Evaluation metric Accuracy
Three primary measurements are recorded during the simulation: the cooperation ratio, overall model accuracy, and node fitness. The cooperation ratio is defined as NC (t) ρC (t) = , (21) N where NC (t) is the number of cooperative nodes at round t and N = n2 is the total number of nodes. These measurements allow the experiment to evaluate both the evolutionary behavior of the network and its resulting learning performance. Second, the model performance of the DFL system is monitored throughout the training process. The accuracy of the consolidated model is recorded after each round to determine whether the evolutionary dynamics result in improved learning performance and whether the model eventually approaches a stable state. Third, the cumulative payoff of each node is recorded as its evolutionary fitness. This metric is used to examine whether cooperative or defective behavior is favored under the proposed payoff mechanism and to establish a relationship between strategic evolution and learning performance.
Reputation-Driven Cooperation in Lattice-Based EGT-DFL
5.2
19
Results and Evaluation
Baseline Scenario
Reputation-based Scenario
Figure 5: Evolutionary process of strategy distribution with reputation and no reputation mechanism.
Figures 5 illustrate the evolutionary dynamics of strategy adoption under the Fermi imitation process. Each snapshot represents the distribution of strategies at different generations, where red denotes defectors and green denotes cooperators. Without the reputation mechanism, defectors gradually dominate the population. This behavior arises from the proposed payoff formulation, in which the reward is scaled by the model’s improvement in accuracy (∆Q). During the early stages of training, cooperation produces substantial accuracy gains, making collaborative training beneficial. However, as the models approach convergence, the marginal improvement in accuracy diminishes. Consequently, the benefit obtained from additional cooperation becomes insufficient to justify for its computational and communication costs. Defection therefore creates a higher net payoff, and under the Fermi imitation rule, neighboring players increasingly adopt the defection strategy, eventually leading the population to a fully defective state. With the reputation mechanism, the evolutionary outcome is reversed, with cooperation gradually dominating the population. Although the marginal improvement in model accuracy (∆Q) decreases as training progresses, the proposed payoff matrix incorporates each player’s reputation score as an additional
20
Phuc Hoang et at.
incentive. Cooperative players continuously accumulate reputation through repeated participation, resulting in a progressively larger payoff bonus. As training converges, the growth of the reputation reward surpasses the diminishing contribution of ∆Q to the payoff. Therefore, cooperation remains the more profitable strategy despite the reduced marginal learning benefit. Under the Fermi imitation rule, neighboring agents are increasingly likely to adopt the cooperative strategy, leading to the expansion of cooperative clusters until the population converges to an almost entirely cooperative state.
Figure 6: Comparison of strategy distribution over time with no reputation mechanism.
Figure 6 illustrates the evolutionary dynamics without the reputation mechanism. Initially, cooperation is beneficial because collaborative training produces substantial improvements in model accuracy (∆Q). However, as the global model approaches convergence, the marginal accuracy gain gradually diminishes, making the communication and computation costs of cooperation outweigh its benefits. Consequently, defection becomes the more profitable strategy, and the Fermi imitation process drives the population toward an almost entirely defective state. The decline in cooperation is reflected in the remaining performance metrics. As fewer players participate in training, the overall training cost decreases, resulting in a gradual recovery of the total fitness. However, reduced collaboration also limits knowledge sharing, leading to a lower final average accuracy. Meanwhile, the increasing accuracy variance indicates that individual models become
Reputation-Driven Cooperation in Lattice-Based EGT-DFL
21
more heterogeneous, as cooperative players continue improving while defective players largely cease updating their models.
Figure 7: Comparison of strategy distribution over time with reputation mechanism.
Figure 7 presents the evolutionary dynamics when the reputation mechanism is incorporated into the payoff formulation. Although the improvement in model accuracy (∆Q) gradually diminishes as training progresses, cooperative players continuously accumulate reputation through repeated participation. As the reputation reward eventually outweighs the diminishing contribution of (∆Q), cooperation remains the more profitable strategy. Consequently, the Fermi imitation process rapidly drives the population toward an almost entirely cooperative state. Sustained cooperation enables continuous knowledge sharing among participants, resulting in a higher final average model accuracy than the baseline. Although maintaining cooperation incurs additional communication and computation costs, the total fitness remains relatively stable due to the reputation incentive. Furthermore, the accuracy variance steadily decreases throughout the evolutionary process, indicating that the distributed models progressively converge to similar performance levels. These results demonstrate that the reputation mechanism not only improves learning performance but also promotes a more consistent and robust decentralized learning process.
22
6
Phuc Hoang et at.
Conclusion and Discussion
In this study, we model the Decentralized Federated Learning network under a lattice structure, where each node represents a participating agent seeking to maximize its own utility. By establishing a bounded-rationality agent model with spatial-cluster interactions, this research enables the intuitive observation of strategic evolution and the behavioral propagation dynamics across the system. Grounded in Evolutionary Game Theory (EGT), we analyze the strategic transition dynamics and formulate a comprehensive payoff matrix. Furthermore, to foster sustainable cooperation and deter free-riding behaviors, a reputationbased reward-and-punishment mechanism is integrated into the framework. Experimental results not only clearly illustrate the spatial propagation patterns of behavioral interactions, but also validate the superior effectiveness of the reputation-based reward-and-punishment mechanism in optimizing overall performance and suppressing the free-riding problem in DFL. In future work, we intend to extend this framework to more complex network topologies, such as scale-free and small-world networks, to assess the impact of network connectivity on system behavior. Additionally, future extensions will incorporate realistic client connectivity parameters (e.g., latency and bandwidth constraints) and explore multi-state models to accommodate the dynamic and flexible roles of individual nodes. Acknowledgments. We acknowledge Ho Chi Minh City University of Technology (HCMUT), VNU-HCM for supporting this study. TAH is supported by EPSRC (grant EP/Y00857X/1). MHD was supported by EPSRC grant EP/Y008561/1. T.A.H. acknowledges travel and accommodation support from the Ho Chi Minh City University of Technology (HCMUT), VNUHCM (Adjunct Professorship scheme HCMUT-VNUHCM). L.H.T and T.A.T.N acknowledge the Ho Chi Minh City University of Technology (HCMUT), VNUHCM for supporting this study.
References 1. Al-Rubaie, M., Chang, J.M.: Privacy-preserving machine learning: Threats and solutions. IEEE Security & Privacy 17, 49–58 (2018) 2. Bashir, A., Shamszaman, Z.U., Song, Z., Perc, M., Han, T.A.: Strategic commitments shape collective cybersecurity under ai inequality. Chaos, Solitons & Fractals 210, 118728 (2026) 3. Blanchard, P., Mhamdi, E.M.E., Guerraoui, R., Stainer, J.: Machine learning with adversaries: Byzantine tolerant gradient descent. In: Neural Information Processing Systems (2017) 4. Caldas, S., Wu, P., Li, T., Konecný, J., McMahan, H.B., Smith, V., Talwalkar, A.: Leaf: A benchmark for federated settings. ArXiv abs/1812.01097 (2018) 5. Duong, H.N., Nguyen, T.D., Nguyen, H.T., Song, Z., Duong, M.H., Trang, L.H., Han, T.A., et al.: Beyond cooperation and cost: Welfare-optimal incentives in structured populations. Chaos, Solitons & Fractals 211, 118843 (2026) 6. Ferenczi, A., Bǎdicǎ, C.: Fully decentralized privacy-enabled federated learning system based on byzantine-resilient consensus protocol. Simul. Model. Pract. Theory 136, 102987 (2024)
Reputation-Driven Cooperation in Lattice-Based EGT-DFL
23
7. Goh, E., Kim, D.Y., Lee, K., Oh, S., Chae, J.E., Kim, D.Y.: Blockchain-enabled federated learning: A reference architecture design, implementation, and verification. IEEE Access 11, 145747–145762 (2023) 8. Hallaji, E., Razavi-Far, R., Saif, M., Wang, B., Yang, Q.: Decentralized federated learning: A survey on security and privacy. IEEE Transactions on Big Data 10, 194–213 (2024) 9. Hofbauer, J., Sigmund, K., et al.: Evolutionary games and population dynamics, vol. 1. Cambridge university press Cambridge (1998) 10. Hu, C., Jiang, J., Wang, Z.: Decentralized federated learning: A segmented gossip approach. ArXiv abs/1908.07782 (2019), https://api.semanticscholar.org/ CorpusID:201124492 11. Kang, J., Xiong, Z., Niyato, D., Xie, S., Zhang, J.: Incentive mechanism for reliable federated learning: A joint optimization approach to combining reputation and contract theory. IEEE Internet of Things Journal 6, 10700–10714 (2019), https: //api.semanticscholar.org/CorpusID:203178529 12. Koskela, A., Kulkarni, T.: Differential privacy analysis of decentralized gossip averaging under varying threat models (2026), https://arxiv.org/abs/2505.19969 13. Lamport, L., Shostak, R.E., Pease, M.C.: The byzantine generals problem. In: ACM Trans. Program. Lang. Syst. vol. 4, pp. 382–401 (1982) 14. LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436–444 (2015) 15. Lian, X., Zhang, C., Zhang, H., Hsieh, C.J., Zhang, W., Liu, J.: Can decentralized algorithms outperform centralized algorithms? a case study for decentralized parallel stochastic gradient descent. In: Guyon, I., Luxburg, U.V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., Garnett, R. (eds.) Advances in Neural Information Processing Systems. vol. 30. Curran Associates, Inc. (2017), https://proceedings.neurips.cc/paper_files/paper/2017/file/ f75526659f31040afeb61cb7133e4e6d-Paper.pdf 16. Martínez Beltrán, E.T., Pérez, M.Q., Sánchez, P.M.S., Bernal, S.L., Bovet, G., Pérez, M.G., Pérez, G.M., Celdrán, A.H.: Decentralized federated learning: Fundamentals, state of the art, frameworks, trends, and challenges. IEEE Communications Surveys & Tutorials 25(4), 2983–3013 (2023). https://doi.org/10.1109/ COMST.2023.3315746 17. McMahan, H.B., Moore, E., Ramage, D., Hampson, S., y Arcas, B.A.: Communication-efficient learning of deep networks from decentralized data. In: International Conference on Artificial Intelligence and Statistics (2016) 18. Nowak, M.A., May, R.M.: Evolutionary games and spatial chaos. Nature 359, 826–829 (1992), https://api.semanticscholar.org/CorpusID:4328667 19. Röder, M., Geiger, F., Schleif, F.M.: Driving cooperation in federated learning via evolutionary game theory. In: 2025 International Joint Conference on Neural Networks (IJCNN). IEEE (2025) 20. Sagduyu, Y.: Free-rider games for federated learning with selfish clients in nextg wireless networks. pp. 365–370 (10 2022). https://doi.org/10.1109/CNS56114. 2022.9947274 21. Sun, T., Li, D., Wang, B.: Decentralized federated averaging (2021), https:// arxiv.org/abs/2104.11375 22. Szabo, G., Fath, G.: Evolutionary games on graphs. Phys Rep 446 (07 2006). https://doi.org/10.1016/j.physrep.2007.04.004 23. Szabo, G., Tőke, C.: Evolutionary prisoner’s dilemma game on a square lattice. Physical Review E 58, 69–73 (07 1998). https://doi.org/10.1103/PhysRevE.58. 69
24
Phuc Hoang et at.
24. Szabó, G., Fáth, G.: Evolutionary games on graphs. Physics Reports 446(4-6), 97–216 (July 2007). https://doi.org/10.1016/j.physrep.2007.04.004, http: //dx.doi.org/10.1016/j.physrep.2007.04.004 25. Tu, X., Zhu, K., Nguyen, C., Niyato, D., Zhang, Y., Li, J.: Incentive mechanisms for federated learning: From economic and game theoretic perspective. IEEE Transactions on Cognitive Communications and Networking 8, 1566–1593 (09 2022). https://doi.org/10.1109/TCCN.2022.3177522 26. Weibull, J.W.: Evolutionary game theory. MIT press (1997) 27. Xu, J., Zhao, Y., Li, X., Zhou, L., Zhu, K., Xu, X., Duan, Q., Zhang, R.: Teg-di: Dynamic incentive model for federated learning based on tripartite evolutionary game. Neurocomputing 621, 129259 (2025) 28. Yang, P., Zhang, H., Gao, F., Xu, Y., Jin, Z.: Multi-player evolutionary game of federated learning incentive mechanism based on system dynamics. Neurocomputing 557, 126739 (2023) 29. Yuan, L., Sun, L., Yu, P.S., Wang, Z.: Decentralized federated learning: A survey and perspective. IEEE Internet of Things Journal 11, 34617–34638 (2023) 30. Zeng, R., Zeng, C., Wang, X., Li, B., Chu, X.: A comprehensive survey of incentive mechanism for federated learning (2021), https://arxiv.org/abs/2106.15406 31. Zhan, Y., Zhang, J., Hong, Z., Wu, L., Li, P., Guo, S.: A survey of incentive mechanism design for federated learning. IEEE Transactions on Emerging Topics in Computing 10(2), 1035–1044 (2022). https://doi.org/10.1109/TETC.2021. 3063517, publisher Copyright: © 2013 IEEE.