arXiv CSopen access

Robust and Automated Reconfiguration of Byzantine Wide-Area Replication

Robust and Automated Reconfiguration of Byzantine Wide-Area Replication Rowdy Chotkan, Bulat Nasrulin, Johan Pouwelse and Jérémie Decouchant

Abstract—Distributed systems handle adversarial nodes through redundancy, which imposes a significant performance overhead. In blockchain systems, Byzantine fault-tolerant state-machine replication (BFT-SMR) is the replicated service that totally orders client transactions before execution. While prior research has primarily focused on designing novel consensus algorithms with improved performance, recent studies have shown that further gains can be achieved through configuration optimization. More precisely, replicas can monitor network latency to dynamically assign the leader role and tune voting weights, thereby improving consensus performance. However, we identify three vulnerabilities in this process that Byzantine nodes can exploit. To address these weaknesses, we propose Beware, a reconfiguration framework that filters out falsified latency reports, computes robust weight distributions, and applies machine learning to converge towards Byzantine-resilient configurations. Our evaluation shows that Beware reduces consensus latency by up to 45% compared to existing solutions. Index Terms—Self-optimization, reconfiguration, weighted voting, Byzantine fault tolerance, state machine replication.

315

Latency (ms)

arXiv:2606.16740v1 [cs.DC] 15 Jun 2026

Delft University of Technology, The Netherlands {r.m.chotkan-1, b.nasrulin, j.a.pouwelse, j.decouchant}@tudelft.nl

300 285 270 Jul-19 06:00 12:00 18:00 Jul-20 06:00 12:00 18:00 Jul-21

Time

Fig. 1: Measured round-trip time between Sydney and Dubai over a 48-hour period (WonderNetwork, July 19–21, 2020).

(ii) employing two distinct voting weights (Vmin and Vmax ); and (iii) applying optimization techniques to assign weights and roles (leader or backup) to replicas. This optimization allows I. I NTRODUCTION for smaller quorums consisting of only 2f +1 replicas with n+f +1 Blockchains rely on globally distributed replicas to reach a Vmax weight, compared to the larger quorums of ⌈ 2 ⌉ consensus. While this distribution enhances resilience and de- required in uniformly weighted schemes [8]. These self-reconfiguration methods rely on replicas themcentralization, it incurs higher, variable network latency, which selves to commit their network latencies through the main directly affects the performance of consensus protocols [1], [2], consensus algorithm, independently compute how the system [3]. Crucially, unlike in local networks, latencies in Wide Area should be reconfigured, and adopt the new configuration. This Networks (WANs) are highly variable over time, especially approach faces two significant challenges: (i) faulty nodes may across cloud regions or public network infrastructure. As report arbitrary latencies, leading the system to be reconfigured illustrated in Fig. 1, WAN link latencies can fluctuate by tens of into suboptimal configurations; and (ii) faulty nodes may gather percent within a single day (approximately 20% in this example) high voting weight and later on intentionally delay consensus due to dynamic and unpredictable factors such as routing messages, increasing the overall latency of blockchain consenchanges and network congestion. Moreover, these variations are not evenly distributed and tend to affect certain regions sus. As our experiments demonstrate, these attacks severely disproportionately. To maintain consistently high performance, compromise reconfiguration effectiveness (see Sec. VI-A). WAN systems must therefore support dynamic reconfiguration Addressing these challenges is crucial to the practical adoption of self-optimization frameworks, ensuring robust, low-latency to adapt in real-time to shifting network conditions. Several mechanisms have been proposed to allow blockchain performance even in adversarial environments. In practice, this consensus systems to improve their performance automatically, requires deeper integration of system-level telemetry and the such as optimistically reducing the system size based on a threat ability to distinguish between honest variation and adversarial detector [4] or selecting a faster leader [5]. Furthermore, Berger manipulation. We present Beware as a solution to these challenges. To et al. [6], [7] demonstrated that the latency of the seminal Practical Byzantine Fault Tolerance (PBFT) algorithm can be filter out inconsistent or malicious latency reports, Beware reduced in geo-distributed settings by: (i) incorporating addi- uses a robust Virtual Coordinate System (VCS) [9], [10] that tional nodes (i.e., using 3f +1+∆ replicas instead of 3f +1); sanitizes latency reports. VCSs are traditionally designed for large-scale, permissionless systems in which nodes rely on This work was partially funded by NWO/TKI grant BLOCK.2019.004. partial local information. Whereas, in our setting, nodes have

access to all latency reports, since they are committed through Beware outperforms the state of the art in consensus latency, the main consensus algorithm. Beware filters out malicious matrix sanitization, and resilience to adversarial networks. latency reports by leading each replica to locally build a VCS Specifically, Beware improves the average consensus latency that is iteratively constructed using a clustering method that of the PBFT algorithm by up to 45% compared to the stateis unaffected by incorrect latencies. To limit the impact of of-the-art reconfiguration framework. Byzantine nodes delaying messages on performance, we extend This paper is organized as follows. Sec. II details our system the weighted voting theory to establish sufficient and efficiently model, and Sec. III provides some necessary background knowlverifiable conditions for a set of weights to define a Byzantine edge on self-reconfiguring BFT-SMR systems. Sec. IV details quorum dissemination system (see Theorem 1, proven in conditions that can be efficiently checked to verify whether a Sec. IV) and efficiently support an arbitrary number of possible weighted quorum system is a Byzantine dissemination quorum voting weights (instead of only two in the literature [11], [6]). system, i.e., whether it is always available and consistent. Sec. V We demonstrate that supporting a more diverse set of weights presents Beware, our BFT-SMR reconfiguration framework. enables a more graceful performance degradation when faulty Sec. VI evaluates the performance of Beware and compares it nodes delay consensus messages. Lastly, to adopt a system with the state-of-the-art reconfiguration approaches. Sec. VII configuration that maintains low latency despite Byzantine discusses the related work. Finally, Sec. VIII concludes this nodes, we employ a machine-learning prediction model. This paper. model learns from past reconfigurations to predict the latency II. S YSTEM MODEL of a given configuration, ensuring optimal performance and faster convergence under adversarial conditions. We consider a system Π = {p1 , · · · , pN } of N = 3f +1+∆ nodes. The system operates under the assumption of a static As a summary, we make the following contributions: adversary controlling up to f Byzantine replicas that may • We identify three key limitations in existing self-optimizing deviate arbitrarily from a specified protocol. The N replicas BFT-SMRs that degrade the efficacy of current reconfigura- are tasked with executing a weighted version of the seminal tion algorithms. First, Byzantine nodes can prevent the iden- PBFT [13] partially-synchronous BFT-SMR algorithm. We tification of an efficient configuration by sharing poisoned define the configuration of the consensus system as the latency reports. Second, the limited number of voting weights distribution of special roles to replicas (e.g., leader or backup), employed in current weighted voting algorithms increases their voting weights, and protocol parameters. their sensitivity to Byzantine faults. Finally, Byzantine nodes We assume that replicas are interconnected via a wide-area can slow down a consensus algorithm by deviating from it, network, where latencies are heterogeneous and may vary a phenomenon that network latencies alone cannot capture. over time. In this context, the system’s configuration must For each of these three weaknesses, we demonstrate how be periodically re-evaluated (e.g., after a given number of corresponding Byzantine attacks can degrade consensus consensus decisions) to assign voting weights to each replica performance. and attribute the leader’s role. Overprovisioning the system • We formulate, for the first time, sufficient conditions for a by using ∆ additional replicas expands the set of possible weighted quorum system to be a Byzantine dissemination configurations the system can use to better adapt to network quorum system (Sec. IV). Verifying whether a candidate changes. Replicas independently monitor network latencies and quorum system with an arbitrary number of distinct weights commit them through the consensus algorithm. Faulty replicas satisfies these inequalities is low-complexity (the heaviest may report incorrect latencies or disrupt the protocol execution operation is sorting the weights). The only comparable by delaying the transmission of messages they are expected to method we are aware of is the exhaustive method, which send. For simplicity, we assume that replicas experience similar involves computing all quorums, verifying that each quorum latency distributions across all clients and therefore focus on is always available, and ensuring that the intersection of any minimizing consensus latency. In Sec. VII, we discuss how two quorums is large enough to guarantee safety. However, one could instead focus on end-to-end latency. this method suffers from exponential time complexity. III. R ECONFIGURING W EIGHTED BFT S YSTEMS • We design a BFT-SMR reconfiguration framework, Beware, that combines several technical innovations. Beware first In this section, we detail our system model, which follows the leverages a robust, clustering-based virtual coordinate sys- standard assumptions of existing reconfiguration frameworks. tem tailored to fully connected and wide-area adversarial We then describe the state-of-the-art approach to Byzantine networks. It further extends the weighted voting theory reconfiguration, and identify three problems it faces that limit to accommodate a more diverse set of weights, enabling its effectiveness in adversarial settings. more graceful performance degradation under attack. Finally, Beware incorporates machine learning to avoid configurations A. State-of-the-Art Consensus Reconfiguration where faulty nodes are likely to increase latency. A common approach to improving performance in Byzantine • We evaluate Beware using realistic simulations and a fault-tolerant systems involves dynamic reconfiguration based deployment, in which network conditions are drawn from on observed network conditions [14]. Following this approach, the WonderNetwork dataset [12]. Our results indicate that Aware [6] continuously monitors network conditions and

Lim.1

Latency
 Sanitization Lim.3

Latency Matrix

Reconfiguration

Weight Optimization

Consensus

Lim.2

Fig. 2: Illustration of limitations 1 to 3, which can undermine a reconfiguration process where replicas commit and sanitize a latency matrix via consensus, optimize voting weights and roles, and adopt the resulting configuration.

periodically adjusts the configuration of BFT-SMaRt [8], an problem using the sanitized matrices. Replicas use a latencyimplementation of the classical PBFT [15] algorithm that prediction function to simulate consensus latency for various consists of three communication phases (Propose, Write, and candidate configurations (changes in voting-weight distributions Accept) per request. More specifically, Aware distributes and/or leader positions) and rely on simulated annealing to WHEAT [11]’s voting weights and the leader role among identify the configuration that minimizes consensus latency. Triggering Reconfiguration. If the identified optimal replicas to minimize consensus latency. In a Byzantine system of n = 3f + 1 + ∆ replicas, WHEAT affects a voting weight configuration offers a performance improvement exceeding vmax = (1+∆/f ) to 2f replicas and a voting weight vmin = 1 a predetermined threshold (to avoid unnecessary, marginal to other replicas. Aware aims to assign voting weights and the reconfigurations), all replicas independently obtain the same leader role to better-connected replicas to accelerate quorum optimal configuration. They then collectively trigger a view change, allowing replicas to adopt their new voting weight and formation, with a quorum weight of 2(f + ∆) + 1. The reconfiguration process unfolds in five deterministic role, thereby completing the reconfiguration loop. steps, ensuring all correct replicas reach the same conclusion. B. Limitations Measuring Latencies. Each correct replica continuously In the following, we describe three weaknesses that can be measures the point-to-point network latencies to every other exploited to degrade the efficacy of current reconfiguration replica in the system. Measurements are one-sided to ensure frameworks, illustrated as Lim. 1–3 in Fig. 2. integrity against faulty replicas. Measurements are taken for critical consensus messages, primarily those of the Propose Limitation 1: Poisoned latency matrix. Byzantine nodes phase, where a leader broadcasts a block of transactions, and can communicate incorrect network latencies. While previous the Write and Accept phases, which contain all-to-all broadcasts approaches rely on simple sanitization methods, such as used for commitment. symmetrizing the latency matrix by replacing each value Disseminating and Synchronizing Latency Measurements. with the maximum of its mirrored counterpart across the Replicas broadcast their measured latency vectors to all others diagonal [6], we experimentally show that these methods are using the totally ordered underlying BFT-SMaRt consensus not immune to Byzantine attacks and do not ensure efficient protocol. This ensures that all correct replicas eventually system reconfiguration. Interestingly, while it was known that converge on and update their local copies of the global latency pairs of Byzantine nodes can underestimate their latency to matrices, M P (for the Propose phase) and M W (for the Write appear better connected, we discovered that Byzantine nodes and Accept phases), with the same, synchronized set of raw that overestimate their latency to other nodes can also degrade measurements. reconfiguration performance. Sanitizing Latency Matrices. After a predefined number of consensus instances, all replicas deterministically sanitize their Limitation 2: Under-explored Byzantine quorum systems. synchronized latency matrices, yielding M̂ P and M̂ W . This In a given system configuration, Byzantine nodes may fail to step is crucial to mitigate the influence of malicious (Byzantine) process incoming messages or refrain from sending messages. replicas that attempt to submit false or misleading latency As a result, the consensus protocol would achieve lower reports, thereby ensuring robust decision-making. Matrices are performance than predicted by the reconfiguration framework. sanitized by replacing each latency entry with the largest of its Unlike the emission of conflicting signed messages [16], two symmetric values (i.e., M [i, j] = max(M [i, j], M [j, i])). this type of Byzantine fault cannot be identified under Latency entries that replicas have not committed are set to the fluctuating network conditions we consider. Because +∞. current weighted voting systems consider only at most two Identifying the Optimal Configuration. Each replica possible voting weights, the impact of these deviations is severe. independently and deterministically solves an optimization

Limitation 3: Ineffective reconfigurations. A reconfigura- threshold. While previous works [11] relied on ad-hoc weight tion framework deterministically computes the configuration assignments, we generalize this concept by formally defining that should deliver the best performance, assuming that all the requirements of weighted dissemination quorum systems. nodes adhere faithfully to the consensus algorithm. However, We assume that each process Pi is assigned a voting weight when this assumption is not met, existing reconfiguration meth- wi ∈ {w1 , · · · , wN } such that w1 ≥ · · · ≥ wN . We consider ods repeatedly identify the same underperforming configuration a quorum system W, and note QT a weight threshold that and remain in it. It is therefore necessary for reconfiguration is used to define W, i.e., W is the set of all subsets of Π protocols to measure a configuration’s actual performance and whose total weight is larger than or equal to QT . We note learn from it to affect future reconfigurations. w(Q) the combined weight of the replicas in a set Q ∈ W. In the following, we establish sufficient conditions over QT for IV. G ENERALIZING W EIGHTED D ISSEMINATION Q UORUM W to be a dissemination quorum system. S YSTEMS P P Lemma 1. W is available iff. QT ≤ i∈[1,N ] wi − i∈[1,f ] wi . Before detailing Beware, our solution to the aforementioned limitations, we first present some theoretical results that enable Proof. (⇒) Let us assume that W is available, and consider Beware to more effectively explore the space of possible voting the set Q of the N − f nodes with the lowest weights. The P P weights, thereby increasing the number of system configurations value of w(Q) is i∈[1,N ] wi − i∈[1,f ] wi . By assumption, it considers. To the best of our knowledge, we are the first to Q ∈ W, because the system needs to be available if the f explicitly define formulae for the availability and consistency of nodes with the highest weights are faulty, which means that P P Byzantine weighted quorum systems and to formally define the QT ≤ w(Q). We then obtain QT ≤ i∈[1,N ] wi − i∈[1,f ] wi . P P weight requirements of a Byzantine weighted quorum. These (⇐) Let us assume that QT ≤ i∈[1,N ] wi − i∈[1,f ] wi . theoretical developments enable us to design an optimization Let us consider a set B of f faulty nodes. The total weight algorithm that generates potential voting weights for the nodes w(B) of this set is lower than or equal to the sum of the f P in a given consensus system and efficiently verifies whether largest weights i∈[1,f ] wi . The combined weight of the set P these weights satisfy the properties of a Byzantine quorum. Π \ B of the N − f correct nodes, i∈[1,NP ] wi − w(B), is P This section recalls the definition of a Byzantine weighted then larger than or equal to i∈[1,N ] wi − i∈[1,f ] wi . By dissemination quorum system. It then establishes a sufficient assumption, we then obtain that Π \ B ≥ QT , which means condition for a set of weights to constitute such a quorum that Π \ B ∈ W and that W is available. system. This condition can be verified in linear time, enabling P  P w + w /2 < QT then W Beware to efficiently explore the space of possible replica Lemma 2. If i i i∈[1,N ] i∈[1,f ] weight assignments in order to minimize consensus latency. is consistent. A. Dissemination quorum systems BFT-SMR systems rely on the concept of Byzantine quorums to reach decisions. In particular, they use dissemination quorums to ensure the self-verification of information, such as signed user transactions. Our objective is to identify Byzantine weighted dissemination quorums and latency-optimal weights for system reconfiguration. Definition 1 provides the formal definition of dissemination quorums. Definition 1. A quorum system Q is a dissemination quorum system [17] for a fail-prone system B if the following properties are satisfied. • Availability: ∀B ∈ B, ∃Q ∈ Q : B ∩ Q = ∅ • Consistency: ∀Q1 , Q2 ∈ Q, ∀B ∈ B : Q1 ∩ Q2 ⊈ B Instantiating the fail-prone system B to capture all possible Byzantine sets of size f , availability therefore implies that for any possible set of faulty replicas, there exists at least one quorum consisting solely of correct replicas. Consistency guarantees that any two quorums intersect in at least one correct replica. B. A Sufficient Condition for Weighted Dissemination Quorum Systems In the context of weighted voting, Byzantine quorums are defined by their total weight, which must exceed a weight

Proof. By contradiction, let usP assume that the implication is P false, i.e., that ( i∈[1,N ] wi + i∈[1,f ] wi )/2 < QT and that W is not consistent. Let us consider two sets Q1 , Q2 ∈ W such that |Q1 ∩ Q2 | contains f or less replicas. These sets are guaranteed to exist because W is not consistent. The union of Q P1 and Q2 contains at most all the replicas, i.e., w(Q1 ∪Q2 ) ≤ i∈[1,N ] wi , which can be rewritten as: X w(Q1 ) + w(Q2 ) ≤ w(Q1 ∩ Q2 ) + wi (1) i∈[1,N ]

Let us now make two observations. First, the total weight of Q1 ∩Q2 is lower than or equal to the combined weight of the f heaviest replicas, because Q1 ∩Q2 contains Pf or fewer replicas. This can be written as w(Q1 ∩ Q2 ) ≤ i∈[1,f ] wi . Second, QT ≤ w(Q1 ) and QT ≤ w(Q2 ) because Q1 , Q2 ∈ W. Using those two observations in Inequation 1, we obtain 2QT ≤ P P w + w i i∈[1,f ] i∈[1,N ] i , which contradicts our assumption. Theorem 1. Let Π = {p1 , · · · , pN } be a system where each node pi has a weight wi , with w1 ≥ w2 ≥ · · · ≥ wN . Let us consider a weight threshold QT (if any) that verifies   X X X X  wi + wi  /2 < QT ≤ wi − wi . i∈[1,N ]

i∈[1,f ]

i∈[1,N ]

i∈[1,f ]

The set of all subsets of Π whose total weight QT is larger than or equal to QT is a Byzantine dissemination quorum system. Proof. Immediate using Definition 1, and Lemmas 1 and 2. Note that in the egalitarian case, i.e., when every node has a unitary voting weight, Theorem 1 implies that a Byzantine quorum requires at least ⌈ n+f2 +1 ⌉ votes, which is a known formula. Similarly, given Aware’s [6] weight distribution, Theorem 1 provides 2f +2δ+1/2 as a strict lower bound for the weight threshold, which confirms the 2f +2δ+1 weight threshold of Aware. We leverage Theorem 1 to efficiently verify whether a set of weights can form a Byzantine-weighted dissemination quorum system. Note that Theorem 1 might not characterize all possible Byzantine dissemination quorum systems because Lemma 2 is not an equivalence. Example. Fig. 3 showcases an example in which considering more than two voting weights, and doing so in more flexible ways, following Theorem 1, enables new configurations that further improve performance. In this example, we consider n = 5 replicas with f = ∆ = 1. All replicas are correct and collectively report the indicated latency matrix. In the best configuration that Aware identifies, replica 3 is the leader, and the replicas are assigned weights [2, 1, 1, 2, 2], using 5 as the weight threshold to obtain a configuration where PBFT’s consensus latency equals approx. 152 ms. On the other hand, Beware is able to identify a better configuration where replica 4 would act as the leader and replicas would be given weights [5.125, 2, 1, 5.125, 5.125], and use a weight threshold QT such that 11.75 < QT ≤ 13.25 (e.g., 12), which would decrease the consensus latency down to approx. 125 ms (an 18% improvement over Aware). Note that a seemingly close configuration that Aware could have identified would select replica 4 as the leader and choose weights [2, 1, 1, 2, 2]. However, the latency of this configuration would be approx. 160 ms. While this example only considers correct nodes, the gap between Aware and Beware is sometimes even larger (up to 50%) when faulty replicas are present (cf. Fig. 7). V. B EWARE

Unlike classical VCS algorithms—decentralized and typically tailored to peer-to-peer systems—our approach is centralized but replicated. Given that all nodes that decide on a new configuration eventually need to obtain a consistent latency matrix, the sanitization process can either be executed by all nodes or delegated to 2f +1 nodes, which would then disseminate the sanitized matrix to all nodes. For ease of presentation, we focus here on the former option. Algorithm 1 details the pseudocode of our VCS-based sanitization method, which takes as input the latency matrix RT T , where RT T i,j represents the (possibly incorrect) latency reported by node i for its connection with node j. Initially, every node j is assigned random coordinates in the virtual space and an error ej , which expresses the system’s confidence in the accuracy of the node’s coordinates. This algorithm performs T iterations, adjusting the position of node i based on the forces exerted by the other nodes. The force exerted by a node j is computed based on the difference between RT T i,j , the reported latency between nodes i and j, and the distance between nodes i and j in the virtual space. It is directed along the unitary vector → − u i,j from nodes i to j (Alg. 1, l. 9). Additionally, this force is scaled based on the coordinate errors ei and ej (Alg. 1, l. 7), such that greater uncertainty in the coordinates of nodes i and j results in a reduced norm of the force. Note that if both ei and ej are 0, the computation would involve division by zero. In this edge case, node j’s influence is discarded, as it exerts no force on node i. For simplicity, these steps are omitted in the pseudocode. Two parameters, ce and cc , control the extent to which the coordinate error and node coordinates are updated during each iteration.

To filter out harmful Byzantine forces, Alg. 1 groups the forces that all nodes exert on node i into clusters using HDBSCAN [21], based on their cosine distance (Alg. 1, l. 11). Given up to f Byzantine nodes in a system of at least 3f +1 nodes, the cluster containing the majority of forces is dominated by correct nodes, while any remaining Byzantine influence that belongs in this cluster has little impact. Forces in the largest cluster are clipped to their median norm and then averaged to update node i ’s coordinate (Alg. 1, l. 17).

Beware follows the approach pioneered by state-of-the-art reconfiguration frameworks while addressing their limitations. In our implementation, we use a 3-dimensional space, with To do so, Beware combines three novel methods. Succinctly, Beware obtains a reliable view of network latencies in wide-area ej = 0.1, ce = cc = 0.25, and T = 1, 000, based on networks through a novel Byzantine-resilient virtual coordinate empirical tuning. These values strike a good balance between system, explores a more diverse set of voting weights to responsiveness and stability during convergence, yielding stable enhance performance under diverse conditions, and achieves and accurate virtual coordinates across our test scenarios. In a accurate performance predictions of a given configuration based system of N = 100 replicas, Algorithm 1 runs in approximately 160 s for T = 1, 000 iterations. Note that, based on Fig. 1, on historical configurations and their observed outcomes. this is sufficient to address real-life latency variations. In A. Sanitizing the Latency Matrix addition, the reconfiguration process is executed periodically To address Limitation 1, Beware sanitizes latency matrices and asynchronously at epoch boundaries and does not block using a Byzantine-resilient clustering defense [18], [19], [20], ongoing consensus decisions. In our current implementation originally designed for federated learning, that we integrate of Beware, this code is currently written in Python and can within a virtual coordinate system [9], [10]. therefore be optimized.

AWARE 0 44.91

82.62 65.07 23.25 13.7 0

96.48 27.08

98.12 8.19 87.69 0

Leader: 3

152.44 ms

Weights = [1, 2, 1, 2, 2]

Consensus Latency

6.92 59.07

67.94 45.73 46.55

0

6.36 44.14 99.24 12.24

BEWARE

40.03 0

Leader: 4 [5.125, 2, 1, 
 Weights = 5.125, 5.125]

125.05 ms

Fig. 3: Given identical network latencies, Beware’s leader and weight selection yields approx. 125 ms consensus latency, outperforming Aware’s approx. 152 ms. Algorithm 1: Robust centralized VCS construction using clustering and clipping of forces. 1: Input: Latency matrix RT T of size N × N 2: Output: Sanitized latency matrix 3: Parameters: 0 < cc , ce < 1 4: for iter in range(T ): 5: for i in [1, N ]: 6: for j in [1, N ] \ {i}: // forces exerted on i 7: w[j] = ei /(ei + ej )

es [j] = |||xi − xj || − RT T ij | /RT T ij → − − F [j] = w[j] · (RT T ij − ||xi − xj ||) · → u i,j

8: 9: 10: 11:

→ − (b1 , · · ·, bL ) = C LUSTER({ F j̸=i [j]}) // Largest cluster es = M EAN(es [b1 ], · · · , es [bL ]) 12: w = M EAN 13: P (w[b1 ], · · ·, w[bL ]) w[bk ]·es [bk ] P 14: ei = ce · k∈[1,L] + (1 − ce )·ei w[bk ] k∈[1,L]

15: 16:

→ − → − |F | = M EDIAN(| F [b1]|, · · · , | F [bL ]|)  → − → − xi = xi + cc · M EAN F [b1 ], · · · , F [bL ] /|F |

17: 18: 19: return {||xi −xj ||, ∀1 ≤i, j≤N } // Sanitized latency

matrix

B. Optimizing Byzantine Weighted Dissemination Quorums for Latency

quorums and verifying safety and liveness (see Def. 1), which is exponential in complexity. Instead, we derive sufficient conditions that can be verified in linear time and that are summarized in Theorem 1. Simulated Annealing extends the approach introduced by Berger et al. [6] to arbitrary weights: starting from the current configuration (a leader and a weight vector), it iteratively mutates the configuration to reduce latency while gradually cooling down. Concretely, at each SA iteration, a neighboring configuration is generated by either mutating the voting weights or by choosing a different leader. Weight mutations can involve: (i) small incremental changes (±1 to individual weights), (ii) random replacement of selected weights with values from the allowed range [1, 10], or (iii) heuristic-based adjustments that guide weights toward configurations more likely to satisfy quorum constraints. The newly proposed configuration is accepted if it yields lower latency,
or—if it increases latency— with probability exp −∆latency /T , where ∆latency is the latency increase and T is a temperature parameter updated as T ← T ·(1−θ). Early in the run, when T is high, SA can escape local minima by occasionally accepting worse configurations. As T cools down, it becomes increasingly unlikely to accept inferior solutions. While SA has the advantage of requiring only a few hyperparameters, its local mutation moves and fixed annealing schedule tend to converge slowly in high-dimensional weight spaces and often get trapped once the temperature is low.

Differential Evolution is a population-based evolutionary To find latency-efficient weighted quorum assignments and algorithm [22]. Each candidate in DE’s population of conaddress Limitation 2, Beware can explore the joint space figurations is an (n + 1)-dimensional vector comprising N of possible leader choices and node weights using one of continuous weight values and a single integer representing the two optimization approaches: Simulated Annealing (SA) and leader index. We maintain a population of 20 such candidates. Differential Evolution (DE). As a consequence, Beware At each generation, for every target vector xt in the population, supports an arbitrary number of distinct voting weights; each DE selects three distinct individuals (xa , xb , xc ) uniformly at
candidate configuration explored during optimization must be random and forms a mutant vector m = xa + F xb − xc , verified as a valid Byzantine dissemination quorum system. where the scaling factor F is typically set to 0.5. To ensure that Naively checking whether a candidate set of weights defines a every coordinate stays within the valid range (i.e., weights in Byzantine quorum system requires enumerating all possible [0.25, 2.5] and leaders in {0, 1, . . . , n − 1}), we clip the mutant

accordingly. A trial vector is then produced by performing crossover between m and xt . For each dimension, we inherit from m with probability Cr = 0.7 or from xt otherwise. After rounding the leader coordinate to the nearest valid integer, we evaluate the trial’s consensus latency and compare it with that of xt . If the trial has strictly lower latency, it replaces xt in the population; otherwise, xt survives for the next generation. Crucially, we bias the initial DE population so that weight vectors are drawn from a truncated Beta(5, 2) distribution on the interval [0.25, 2.5] rather than from a uniform distribution. This heuristic avoids extreme all-ones or all-twos vectors at the outset, thereby accelerating convergence toward promising regions of the search space while preserving sufficient diversity to explore substantially different configurations. Because DE evaluates 20 candidates in parallel each generation and relies on differential combinations of existing solutions, it naturally explores the high-dimensional space far more efficiently than SA’s single-chain, random-swap approach. Empirically, DE converges to near-optimal weight and leader configurations in far fewer consensus-round evaluations; it is also far less prone to getting stuck in local minima. For these reasons, DE is chosen as the default optimizer in all of our experiments, with SA retained purely as a baseline for direct comparison. Whenever we report “Beware (DE)” in our evaluation, it refers to the version that uses DE with Beta(5, 2) initialization; “Beware (SA)” refers to the SA baseline following Berger et al. [6]. While global optimization techniques identify latencyefficient Byzantine dissemination quorums, they inherently involve predicting the latency of candidate configurations in adversarial settings. For this purpose, Beware integrates predictive modeling into the optimization loop.

random audit with probability pcheck = 0.1 per step forces an explicit measurement even when the error stays just below εdesired , ensuring that rare but large drifts are eventually detected. The model is retrained whenever the running relative PW |ℓ̂i −ℓi | 1 error W exceeds εretrain = 3 × 10−2 . Retraining i=1 ℓi is cheap because W is capped, and XGBoost scales linearly in the number of samples. Feature representation. For a system with N replicas, we encode a configuration as an (N +1)-dimensional real vector x = [ w1 , . . . , wN , L ]⊤ , where wi ∈ [0.25, 2.5] is the weight of replica i and L ∈ {0, . . . , N −1} is the leader index (normalised to [0, 1] before fitting). We empirically found that the [0.25, 2.5] weight interval allows Beware to efficiently explore the weight space and identify better-weighted quorums in a short amount of time. This range is justified by two primary factors: first, it constrains weights to positive values while maintaining a 10:1 ratio between the maximum and minimum bounds; and second, it provides a contained search window, which facilitates the time-efficient exploration of weight configurations. XGBoost captures the non-linear interactions between weights and the leader choice; nevertheless, for transparency, we also keep an PN ordinary least-squares fit ℓ̂(x) = β0 + j=1 βj wj +βN +1 L+ε, which runs ≈ 15% less accurately than the boosted model, but serves as a simple sanity check. With W = 200, S = 10, εdesired = 0.01, εretrain = 0.03, pcheck = 0.1, the predictor reduces the number of full consensus-round evaluations by ∼ 60–70%. VI. P ERFORMANCE E VALUATION

We compare Beware against three baselines. We first consider Aware [6], a state-of-the-art BFT-SMR reconfiguration C. Predicting Latency and Converging towards the Best framework that sanitizes latency matrices by replacing each Configuration value with its symmetric counterpart across the diagonal if it Beware attaches a lightweight predictor to its optimization is larger. Then, we consider a PBFT baseline that uses the loop to avoid getting stuck in suboptimal configurations Newton [10] robust virtual coordinate system (VCS) to sanitize (Limitation 3) or having to temporarily adopt and monitor the latency matrix. We also include a vanilla PBFT reference each candidate configuration, which would be inefficient. Each measured without attacks, denoted as “No-Attack” in relevant replica keeps a fixed-size sliding window of the last W figures, which serves as an ideal (attack-free) performance reconfigurations. Every entry stores the leader identifier, the baseline. Sections VI-A–VI-D report results obtained using N voting weights, and the actual latency observed once the microbenchmark simulations that each isolate a key component configuration became active. When a new measurement arrives, of Beware and evaluate its performance. More precisely, these the oldest record is discarded, ensuring that the training set sections assess the building blocks of Beware using real-world always reflects current network conditions and does not overfit WAN latency datasets, both with and without faulty behaviors, to stale patterns. and with real latency variations. These simulations are executed The predictor is trained and periodically retrained in two across 20 nodes on the DAS cluter [24], equipped with dual phases. During a short bootstrapping phase, the optimizer 8-core processors running at 2.4 GHz, 60 GB of RAM, and explores normally and logs ⟨configuration, latency⟩ pairs; after the Rocky Linux operating system. Section VI-E reports the S = 10 examples, the first model is fit using XGBoost [23]. performance of a full Java implementation of Beware in a Subsequently, whenever the optimizer proposes a candidate deployment on the same academic cluster. We replay realx, it first asks the model for a latency estimate ℓ̂(x). An world WAN latency traces and show that Beware performs expensive performance monitoring of a candidate configuration well in an online setting. is performed only if ℓ̂(x) − ℓobs > εdesired , εdesired = 10−2 . In our experiments, we consider nodes that communicate over If the estimate is accurate, the optimizer can accept or reject network latencies randomly sampled from the WonderNetwork x immediately; otherwise the fresh ground-truth measurement dataset [12], a comprehensive and up-to-date dataset sourced ℓobs is inserted into the window, improving the model. A from a large global networking solution provider. In an

TABLE I: Latency attack impact measured as the percentage increase over baseline latency

Deflation Inflation Inflation-def.

Aware [6]

Newton [10]

Beware (this work)

9.58% 9.54% 9.52%

14.38% 14.63% 13.87%

3.35% 2.82% 2.82%

experiment, we position each node at a random site in the WonderNetwork dataset and compute the latency matrix based on the latency between sites. All code used for our simulations and deployment experiments is publicly available in an online repository.1

Average Latency (ms)

Attack

350 300 250

5

A. Consensus Latency under Latency Poisoning Attacks

0.030 0.025

Density

Inflation attack: Byzantine nodes increase all their link latencies. • Deflation attack: Byzantine pairs of nodes decrease their link latencies. • Inflation-deflation attack: Byzantine nodes increase their link latencies with correct nodes and decrease them with other Byzantine nodes.

10

15

Fault Threshold (f)

20

25

(a) Average consensus latency with f ∈ [1, 25]

Malicious nodes may report incorrect network latencies to enhance their perceived network latency and degrade consensus latency, thereby influencing the weight assignment process. We compare our sanitization method with previous ones using three attacks: •

Aware Newton Beware No-Attack

200

Aware Newton Beware No-Attack

0.020 0.015 0.010 0.005 0.000

300 400 500 600 For these experiments, we use Aware’s weight-assignment Latency (ms) procedure to isolate the effect of latency sanitization mechanisms. For the latency inflation and deflation attacks, we (b) Latency probability density function (PDF) with f = 25 use a factor of 3. In all experiments, the system size is set to Fig. 4: Consensus latency under inflation attack. N = 3f +1 + ∆, with ∆ = f , following Aware’s configuration. Fig. 4a shows the average consensus latency under the B. Latency Matrix Sanitization Accuracy inflation attack for all algorithms with f ∈ [1, 25], ∆ = f , We consider the inflation attack and evaluate the distance of n = 3f + 1 + ∆, and 100 repetitions per setting. The shaded the sanitized (attacked) matrix using several distance metrics bands show the 95% confidence interval of the mean across √ repetitions (computed as µ ± 1.96 · σ/ R with R = 100). For with f = ∆ = 25, N = 101, and 10,000 repetitions. The all values of f in [1, 25], Beware provides the most vigorous 90th-percentile error highlights the upper bound of sanitization defense against the inflation attack, with Newton being the most errors, while the number of errors exceeding 100 ms captures vulnerable baseline and Aware performing in between. This significant outliers. The L2 distance provides an overall measure observation also applies to the deflation and inflation-deflation of deviation between the sanitized reported and true latency attacks, as shown in Tab. I. For all values of f , Beware is the matrices, and the Root Mean Squared Error (RMSE) measures most robust defense against the inflation attack, limiting the the average magnitude of deviations, with greater sensitivity average latency increase to 8.23 ms, compared to 26.68 ms for to larger errors. Following the trend of the attack experiment, Beware’s Aware and 38.90 ms for Newton. A similar pattern holds for the deflation and inflation-deflation attacks, as shown in Tab. I. sanitization approach filters out malicious latencies more Fig. 4b shows the probability density function of the effectively than Aware and Newton. Interestingly, Newton consensus latency for all algorithms under an inflation attack, makes more errors above 100 ms (Fig. 5a), whilst Aware makes with f = ∆ = 25 and N = 3f + 1 + ∆ = 101. We use 10, 000 more significant errors, as indicated by the 90th percentile error repetitions. This figure uses f = 25 to maximize the number rate (Fig. 5b). Finally, the Kernel Density Error plot for L2 of Byzantine replicas and obtain a stable view of the spread distance (Fig. 5c) and the RMSE CDF plot (Fig. 5d) show and tails of the latency distribution. Beware’s distribution is that the cluster-based VCS sanitization approach significantly very close to the optimal one, while Aware and Newton’s outperforms all other mechanisms, with no overlap in results. The slight bimodality (visible in both L2 and RMSE) stems distributions have a higher mean and a larger spread. from Beware’s cluster selection algorithm: most runs identify a 1 https://data.4tu.nl/datasets/68ee956e-4144-405a-8364-64c233b1555f fully-honest cluster (low error), while a small number identify

90th Percentile Error (ms)

#Errors Above 100ms

5000 4000 3000 2000 1000 0

400 300 200 100 0

Aware

Newton

Beware

(a) Number of latency matrix entries whose absolute error is above 100 ms

Frequency

400 300 200 100 0

Newton

Beware

(b) 90th-percentile sanitization error

1.0

Cumulative Probability

Aware Newton Beware

500

Aware

0.8 0.6 0.4 0.2

Aware Newton Beware

0.0 5000

10000

15000

L2 Distance

20000

25000

(c) L2 distance between poisoned and sanitized matrices

50

100

150

RMSE

200

250

(d) RMSE between sanitized reported and true latencies

Fig. 5: Accuracy of the sanitization mechanisms of Beware, Aware, and Newton with inflation attack.

a cluster that includes some faulty forces (higher error); Aware/Newton do not have this discrete switch. Overall, these results highlight Beware’s effectiveness in filtering adversarial latencies. C. Weight Reconfiguration To determine the most efficient optimization approach, we evaluate the reconfiguration performance of four Beware variants against Aware. The variants use Simulated Annealing (SA), Differential Evolution (DE), and their ML-enhanced versions (SA+ML, DE+ML). Aware relies on SA. Experiments are run on a network with 3f + 1 + ∆ nodes (f = 10, ∆ = f ), measuring the average number of rounds needed to achieve latency improvements of 0–45%. Each setup is repeated 20 times. We evaluate the number of reconfiguration rounds required per baseline to reach a target latency improvement (in %) with f = 10 message-delaying faulty nodes. These faulty nodes follow the protocol honestly but add artificial delay to every message they send. This models an attack that targets Limitation 2, in which Byzantine nodes intentionally slow consensus. Reconfiguration frameworks cannot detect this attack from the latency matrix alone, since the nodes

report correct latencies but then underperform during actual consensus. Fig. 6 shows that all Beware variants outperform Aware, requiring sublinear growth in rounds as thresholds increase. Each round on the x-axis of Fig. 6 is one optimizer iteration (i.e., one proposed reconfiguration), and each such iteration internally uses 10 consensus rounds to measure consensus latency. SA performs worst, e.g., ≈ 500 rounds for 25% improvement and ≈ 800 for 40%, due to inefficient random swaps. SA+ML reduces rounds drastically (25% in ≈ 80 rounds, 84% faster than SA) but struggles to exceed 35% improvements. DE is more efficient than SA, achieving up to 80% savings for thresholds ≤ 30%, though it plateaus at higher levels (400–600 rounds for 40%). DE+ML delivers the best overall performance: fewer than 100 rounds for ≤ 25% improvements and ≈ 150 rounds for 40%, while maintaining efficiency across thresholds. In summary, SA scales poorly; SA+ML is fast at low thresholds but is limited; DE is stronger, but stalls at higher thresholds; and DE+ML consistently outperforms all others, combining speed and robustness. We further compare SA (baseline) with DE+ML by tracking latency improvements over rounds (Fig. 7) in a network of

Beware-SA Beware-SA+ML Beware-DE Beware-DE+ML

1000 800

600 500

600

Latency (ms)

Number of Rounds to Reach Threshold

1200

400 200 0

0

5

10

15

20

25

30

35

Latency Improvement Threshold (%)

40

Before After

400 300

45

Fig. 6: Number of rounds necessary to reach an improvement threshold compared to Aware (%) with f =10 message-delaying faulty nodes. Lower is better.

Latency Improvement (%)

t

200 No-Attack

Aware

Newton

Beware

Fig. 8: Consensus latency of all baselines before and after reconfiguration under inflation attack.

50 25

We believe its performance can be further improved through hyperparameter tuning on specific WAN latency datasets.

0 25

Beware-SA Beware-DE+ML

50 0

200

400

600

800

Round number

1000

1200

Fig. 7: Latency improvement (%) compared to Aware depending on the round number with f =10 message-delaying faulty nodes. Higher values are better. delaying replicas. SA (blue, solid) shows strong oscillations: even after 1, 000 rounds, improvements dip below 0%, and the median never surpasses +25%. In contrast, DE+ML (red, dashed) exceeds +20% within 60 rounds, reaches +35% before round 200, and then stabilizes with only mild fluctuations. These results highlight two key points: (i) DE+ML converges much faster, and (ii) it delivers stable improvements, crucial for production deployments. The negative spikes in SA stem from its annealing schedule, in which worse configurations are deliberately accepted to escape local minima, often persisting over multiple consecutive rounds. In DE+ML, the occasional negative dips are artifacts of single-round evaluation and do not indicate the adoption of a worse configuration. D. Adaptation to network changes We evaluate Beware under dynamic conditions using N =101 and f =25. Starting from an initial topology, we trigger reconfiguration by introducing inter-cluster congestion via random latency forces. We also test resilience against adversarial inflation attacks (Fig. 8). Before reconfiguration, VCSbased methods introduce minor noise. After reconfiguration, however, Beware closely tracks the optimal baseline, filtering adversarial latencies while adapting to genuine network changes. Competing approaches fail to match this robustness. Our experiments on synthetic datasets and the WonderNetwork dataset validated Beware’s performance with its current parameters.

E. Deployment Experiment To validate Beware under realistic network conditions, we deployed it on the DAS academic cluster [24] with n = 21 replicas and f = 5 Byzantine faults. As mentioned, computing nodes are equipped with dual 8-core processors running at 2.4 GHz, have 60 GB of RAM, and run the Rocky Linux operating system. Communication between nodes is facilitated by Gigabit Ethernet (GbE) and InfiniBand (IB), but we emulate network latency based on the WonderNetwork dataset [12] to emulate a wide-area network. Starting from a stable configuration, we provoke two scenarios that follow the trends observed in our simulations. The experiment lasts for 360 seconds. First, we injected network congestion at t = 90 s for 90 seconds, following the Pareto principle: 20% of the replicas experienced a threefold increase in latency. This models localized network degradation in which only a small subset suffers heavy congestion. As shown in Fig. 9, consensus latency briefly spikes above 800 ms but quickly recovers as Beware optimizes voting weights for faster replicas, stabilizing performance with minor fluctuations caused by reconfiguration. Next, we performed an inflation attack at t = 240 s for a 90second period, during which the f = 5 Byzantine replicas attempted to appear more reliable by artificially inflating reported latencies from other nodes (a factor-of-3 misreport). This manipulation initially degrades latency (peaking near 560 ms), but Beware again detects the inconsistency between reported and observed latencies, adjusts weights, and restores consensus performance. A PBFT baseline (not shown) would keep dishonest replicas influential, causing prolonged degradation. We expect Aware [6] to show a similar initial latency spike under congestion and some sensitivity to inflated reports, but—like Beware—to reweight toward honest replicas and recover. However, as our simulations indicate, Beware achieves more effective configurations under these conditions, a finding corroborated by our deployment results. Overall, these results show that

Congestion Attack

Latency (ms)

800 700 600 500 400 300 0

50

100

150

200

Time (s)

250

300

350

Fig. 9: Beware’s consensus latency under congestion and inflation attack. dynamic weighting enables Beware to sustain low latency under both congestion and inflation attacks, whereas PBFT remains vulnerable. VII. R ELATED W ORK Leader Selection in BFT SMR. Prior work on BFT StateMachine Replication (BFT SMR) has focused primarily on reducing consensus latency and/or increasing throughput [25], [26], [27], [28], [29], [6], [30], [7], often assuming trusted latency measurements from clients, third parties, or sysadmins. Several geo-replicated BFT-SMR systems reduce WAN latency/overhead through fast paths and communication reduction (e.g., SBFT [31]) or by removing the single-leader bottleneck via parallel leaders (e.g., Mir-BFT [32], FnF-BFT [33]), while bounded-delay protocols such as BFT-Mencius [34] target latency guarantees under stabilizing network conditions. For instance, ARCHER [35] selects leaders using clientmeasured end-to-end response times (typically optimizing a target percentile), obtained via probe requests that run the full BFT protocol and complete when the client receives f +1 matching replies. It mitigates manipulation by requiring probes to be verifiable (preventing replicas from replying early by skipping steps) and by limiting the influence of faulty clients through robust aggregation of client reports. In contrast, Prime [5], Aardvark [36], and RBFT [37] rely on replica-side measurements of primary connectivity. In this work, we assumed that clients are uniformly distributed across the network and focused on reducing consensus latency. To minimize client end-to-end latency with more general client latency distributions, future work could integrate ARCHER’s approach into our methods. The end-toend latency of a configuration would then be the average time until clients receive f + 1 replies from replicas. Unlike systems that primarily optimize client request ordering, we focus on replica-to-replica communication, which drives consensus latency and enables robust optimizations even when malicious replicas attempt to create bottlenecks. Kauri [30] instead pipelines requests to lower latency, and other systems [29], [35] pursue similar optimizations but remain vulnerable to the attacks we study. Accountability-focused work [38], [39]

detects faulty replicas through conflicting signed messages, but fails to capture those that delay consensus. In contrast, our system predicts the impact of faulty replicas on latency, limiting their influence to preserve performance. Related work has also shown that Byzantine replicas can satisfy safety and liveness while severely degrading performance via targeted slowdowns and protocol abuse (e.g., slow primary attacks [40]), motivating defenses that explicitly address performance under adversarial behavior [41]. While we apply our framework to PBFT, it could also be used with other consensus algorithms, e.g., to optimize the leader rotation scheme in HotStuff [42]. Membership and Protocol Reconfiguration. Early work like Rampart [43], [44] established that BFT systems can manage their membership by having their members vote on it. More recently, ComChain [45] implemented reconfiguration for permissioned blockchains on top of DBFT [46]. BMS [47] introduced a decentralized membership service applicable to both permissioned and permissionless environments, which active replicas can update via consensus and which clients can contact to identify the latest membership. However, Saltini [48] demonstrated that improper reconfiguration timing can compromise liveness. Their protocol, BigFooT, mitigates this by strictly constraining adoptable memberships. In contrast, Beware prioritizes optimizing role distribution and voting weights over changes to active membership. Incorporating a secure membership adaptation scheme that supports weighted voting into Beware, for example, following Heydari et al.’s approach [49], is future work. Such an extension would further amplify performance gains by effectively integrating additional, geographically diverse replicas. Similarly, Beware might be extended to dynamically select a different consensus algorithm to execute [50], [51], [52], [53]. Weighted Voting Reconfiguration. In classical BFT, each node has one vote, with quorums defined as ⌈ n+f2 +1 ⌉ nodes [17]. Weighted voting [54], [55], [56] instead assigns different weights to replicas. Garg and Bridgman [57] defined the weighted Byzantine agreement problem, showing that consensus is possible with more than n/3 failures, provided the total weight of the failures is below 1/3. WHEAT [11] extends BFT-SMR with extra replicas and weighted quorums, but restricts weights to two values used at fixed frequencies. Aware [6] optimizes latency by dynamically assigning WHEAT weights, with follow-ups for HotStuff [58] and workloadadaptive systems [59]. Developed concurrently with our work, Cabinet [60] and OptiLog [61] also employ dynamic voting weights to accelerate consensus. Cabinet [60] is designed for crash-tolerant environments, while our work addresses the more complex challenges of Byzantine faults. In Cabinet, the reweighting process is centralized: the leader unilaterally computes and distributes weights based on a geometric sequence. In contrast, our reconfiguration process is hardened against manipulation and

supports a broader range of voting weights. Our theoretical VIII. C ONCLUSION framework for weighted dissemination quorums (see Sec. IV) This paper addresses key challenges in optimizing Byzantine can be adapted to crash-tolerant systems of 2f + 1 + ∆ fault-tolerant state-machine replication (BFT-SMR) systems to nodes, provided the weights P and the quorum weight QT reduce consensus latency while maintaining resilience against Pn Pf n w − w satisfy 12 i=1 wi < QT ≤ i=1 i i=1 i . Notably, adversaries. Building on existing reconfiguration frameworks, Cabinet specifically targets systems of 2f + 1 nodes.P In such we propose a robust approach that leverages virtual coordinate a deployment, the simple majority quorum weight ( wi /2) systems, extended weighted voting schemes, and machine utilized by Cabinet is correct. However, in larger systems where learning models to predict and adopt configurations that mini∆ > 0, which we target for higher latency P improvements, the mize latency. Our contributions include identifying sufficient threshold used must be strictly larger than wi /2 for safety, conditions for Byzantine-weighted dissemination quorums, as indicated by our formulas. introducing advanced methods for handling Byzantine attacks Like Beware, OptiLog [61] is designed for Byzantine environ- on latency matrices, and providing an efficient framework ments and shares several of its objectives. However, it focuses that learns from past performance to identify configurations on assigning specific roles (e.g., root or internal nodes in a that would achieve the highest performance despite Byzantine tree-based protocol like Kauri [30]) to replicas. OptiLog uses nodes attempting to degrade it. Through extensive simulations Aware’s latency sanitization matrix, which we have shown to across diverse system sizes, network conditions, and attacks, be vulnerable to latency poisoning attacks, while Beware relies we demonstrated that our methods significantly improve on a VCS-based latency matrix sanitization. Beware replicates performance and resilience compared to existing solutions. the reconfiguration process, whereas OptiLog relies on a leader- By integrating a predictive model into the optimization loop, based reconfiguration and provides accountability through a we further enhanced adaptability to network changes and global log that must be replayed for verification. Nodes that do mitigated the impact of adversarial behaviors. Future work may not behave as expected, based on their communication latency, explore additional machine learning techniques and consider are eventually eliminated in Beware thanks to its use of machine automatically reconfiguring alternative consensus algorithms, learning. Instead, OptiLog uses a suspicion mechanism. such as DAG-based algorithms. Extending Beware to support throughput-oriented optimization is an interesting future direction. This would R EFERENCES require the system to be bandwidth-aware. However, throughput [1] D. De Sensi, T. De Matteis, K. Taranov, S. Di Girolamo, T. Rahn, and cannot be sanitized as latency can. We envision developing T. Hoefler, “Noise in the clouds: Influence of network performance variability on application scalability,” POMACS, vol. 6, no. 3, pp. 1–27, specialized distributed auditing mechanisms to produce reliable 2022. throughput matrices. Once sanitized, these metrics could be [2] A. Uta and H. Obaseki, “A performance study of big data workloads in integrated into a multi-criteria objective function, for instance, cloud datacenters with network variability,” in ICPE Companion, 2018, pp. 113–118. by maximizing a linear combination of throughput and inverse [3] A. Iosup, N. Yigitbasi, and D. Epema, “On the performance variability latency. Dynamic reweighting or stake-based weights have of production cloud services,” in CCGrid. IEEE, 2011, pp. 104–113. also been studied in permissionless Byzantine settings, e.g., [4] D. S. Silva, R. Graczyk, J. Decouchant, M. Völp, and P. EstevesVerissimo, “Threat adaptive byzantine fault tolerant state-machine by FlashConsensus [7], Heydari et al. [49], and Swiper [62], replication,” in SRDS, 2021. and it might be interesting to extend Beware accordingly. [5] Y. Amir, B. Coan, J. Kirsch, and J. Lane, “Prime: Byzantine replication Robust Virtual Coordinate Systems. Virtual Coordinate Systems (VCS) assign nodes virtual coordinates such that distances approximate network latencies, often using partial measurements, making them useful in large-scale P2P systems. Vivaldi [9], for example, models links as virtual springs and seeks a global equilibrium. These springs resist contraction or extension, respectively, when their end nodes are too close or too far apart, with a force proportional to the difference between their resting length (actual latency) and their current length (predicted latency). However, VCSs are vulnerable to data poisoning, where malicious nodes misreport latencies to launch inflation, deflation, or oscillation attacks [63], [64], [65]. To improve robustness, systems such as Veracity [66] validate coordinate updates via voting, whereas Newton [10] applies physics-inspired invariants to counter Byzantine manipulation. These defenses, however, are tailored to peer-to-peer environments and assume that faults occur only after a benign operation. We instead consider a stronger adversary that can misreport latencies from the very start.

under attack,” TDSC, vol. 8, no. 4, pp. 564–577, 2010. [6] C. Berger, H. P. Reiser, J. Sousa, and A. N. Bessani, “AWARE: Adaptive wide-area replication for fast and resilient Byzantine consensus,” IEEE TDSC, 2020. [7] C. Berger, L. Rodrigues, H. P. Reiser, V. Cogo, and A. Bessani, “Chasing lightspeed consensus: Fast wide-area byzantine replication with mercury,” in Middleware, 2024. [8] A. Bessani, J. Sousa, and E. Alchieri, “State machine replication for the masses with BFT-SMART,” in DSN, 2014. [9] F. Dabek, R. Cox, F. Kaashoek, and R. Morris, “Vivaldi: A decentralized network coordinate system,” CCR, vol. 34, no. 4, pp. 15–26, 2004. [10] J. Seibert, S. Becker, C. Nita-Rotaru, and R. State, “Newton: securing virtual coordinates by enforcing physical laws,” ToN, vol. 22, no. 3, pp. 798–811, 2013. [11] J. Sousa and A. Bessani, “Separating the wheat from the chaff: An empirical design for geo-replicated state machines,” in SRDS, 2015. [12] P. Reinheimer and W. Roberts, “Wondernetwork,” https://wondernetwork.com/. [13] M. Castro and B. Liskov, “Practical byzantine fault tolerance,” in OSDI, 1999. [14] L. Lamport, D. Malkhi, and L. Zhou, “Reconfiguring a state machine,” ACM SIGACT News, vol. 41, no. 1, pp. 63–73, 2010. [15] M. Castro and B. Liskov, “Practical byzantine fault tolerance,” in OSDI, 1999. [16] P. Civit, S. Gilbert, and V. Gramoli, “Polygraph: Accountable byzantine agreement,” in ICDCS, 2021.

[17] D. Malkhi and M. Reiter, “Byzantine quorum systems,” Distributed computing, vol. 11, no. 4, pp. 203–213, 1998. [18] T. D. Nguyen, P. Rieger, R. De Viti, H. Chen, B. B. Brandenburg, H. Yalame, H. Möllering, H. Fereidooni, S. Marchal, M. Miettinen et al., “FLAME: Taming backdoors in federated learning,” in USENIX Security, 2022. [19] H. Fereidooni, A. Pegoraro, P. Rieger, A. Dmitrienko, and A.-R. Sadeghi, “Freqfed: A frequency analysis-based approach for mitigating poisoning attacks in federated learning,” arXiv preprint arXiv:2312.04432, 2023. [20] B. Cox, A. Mălan, L. Y. Chen, and J. Decouchant, “Catalyst: Asynchronous byzantine-robust federated learning,” in BigData, 2025. [21] R. J. G. B. Campello, D. Moulavi, and J. Sander, “Density-based clustering based on hierarchical density estimates,” in PAKDD, 2013. [22] R. Storn and K. Price, “Differential evolution–a simple and efficient heuristic for global optimization over continuous spaces,” Journal of global optimization, vol. 11, pp. 341–359, 1997. [23] T. Chen and C. Guestrin, “XGBoost: A scalable tree boosting system,” in SIGKDD, 2016. [24] H. Bal, D. Epema, C. De Laat, R. Van Nieuwpoort, J. Romein, F. Seinstra, C. Snoek, and H. Wijshoff, “A medium-scale distributed system for computer science research: Infrastructure for the long term,” Computer, vol. 49, no. 5, pp. 54–63, 2016. [25] L. Lamport, “Generalized consensus and paxos,” Microsoft, Tech. Rep., 2005. [26] P. Sutra and M. Shapiro, “Fast genuine generalized consensus,” in SRDS, 2011. [27] I. Moraru, D. G. Andersen, and M. Kaminsky, “There is more consensus in egalitarian parliaments,” in SOSP, 2013. [28] J. Du, D. Sciascia, S. Elnikety, W. Zwaenepoel, and F. Pedone, “Clockrsm: Low-latency inter-datacenter state machine replication using loosely synchronized physical clocks,” in DSN, 2014. [29] S. Liu and M. Vukolić, “Leader set selection for low-latency georeplicated state machine,” IEEE TPDS, vol. 28, no. 7, pp. 1933–1946, 2016. [30] R. Neiheiser, M. Matos, and L. Rodrigues, “Kauri: Scalable bft consensus with pipelined tree-based dissemination and aggregation,” in SOSP, 2021. [31] G. G. Gueta, I. Abraham, S. Grossman, D. Malkhi, B. Pinkas, M. Reiter, D.-A. Seredinschi, O. Tamir, and A. Tomescu, “SBFT: A scalable and decentralized trust infrastructure,” in DSN, 2019. [32] C. Stathakopoulou, T. David, M. Pavlovic, and M. Vukolic, “Mir-BFT: Scalable and robust BFT for decentralized networks,” Journal of Systems Research, vol. 2, no. 1, p. 3, 2022. [33] Z. Avarikioti, L. Heimbach, R. Schmid, L. Vanbever, R. Wattenhofer, and P. Wintermeyer, “Fnf-bft: Exploring performance limits of bft protocols,” arXiv preprint arXiv:2009.02235, 2020. [34] Z. Milosevic, M. Biely, and A. Schiper, “Bounded delay in byzantinetolerant state machine replication,” in SRDS, 2013. [35] M. Eischer and T. Distler, “Latency-aware leader selection for georeplicated byzantine fault-tolerant systems,” in DSN-W, 2018. [36] A. Clement, E. Wong, L. Alvisi, M. Dahlin, M. Marchetti et al., “Making byzantine fault tolerant systems tolerate byzantine faults,” in NSDI, 2009. [37] P.-L. Aublin, S. B. Mokhtar, and V. Quéma, “RBFT: Redundant byzantine fault tolerance,” in ICDCS, 2013. [38] P. Sheng, G. Wang, K. Nayak, S. Kannan, and P. Viswanath, “BFT protocol forensics,” in CCS, 2021. [39] P. Civit, S. Gilbert, V. Gramoli, R. Guerraoui, and J. Komatovic, “As easy as ABC: Optimal (a)ccountable (b)yzantine (c)onsensus is easy!” JPDC, vol. 181, p. 104743, 2023. [40] G. S. Veronese, M. Correia, A. N. Bessani, and L. C. Lung, “Spin one’s wheels? Byzantine fault tolerance with a spinning primary,” in SRDS, 2009. [41] Y. Amir, B. Coan, J. Kirsch, and J. Lane, “Byzantine replication under attack,” in DSN, 2008. [42] M. Yin, D. Malkhi, M. K. Reiter, G. G. Gueta, and I. Abraham, “HotStuff: BFT consensus with linearity and responsiveness,” in PODC, 2019. [43] M. Reiter, “Distributing trust with the rampart toolkit,” Communications of the ACM, vol. 39, no. 4, pp. 71–74, 1996. [44] M. K. Reiter, “A secure group membership protocol,” IEEE TSE, vol. 22, no. 1, pp. 31–42, 2002. [45] G. Vizier and V. Gramoli, “ComChain: A blockchain with Byzantine fault-tolerant reconfiguration,” Concurrency and Computation: Practice and Experience, vol. 32, no. 12, p. e5494, 2020.

[46] T. Crain, V. Gramoli, M. Larrea, and M. Raynal, “DBFT: efficient leaderless byzantine consensus and its application to blockchains,” in NCA, 2018. [47] S. Steinhoff, C. Stathakopoulou, M. Pavlovic, and M. Vukolić, “BMS: Secure decentralized reconfiguration for blockchain and BFT systems,” arXiv preprint arXiv:2109.03913, 2021. [48] R. Saltini, “BigFooT: A robust optimal-latency BFT blockchain consensus protocol with dynamic validator membership,” Computer Networks, vol. 204, p. 108632, 2022. [49] H. Heydari, G. Silvestre, and A. Bessani, “How hard is asynchronous weight reassignment?” in ICDCS, 2023. [50] P.-L. Aublin, R. Guerraoui, N. Knežević, V. Quéma, and M. Vukolić, “The next 700 bft protocols,” ACM Transactions on Computer Systems (TOCS), vol. 32, no. 4, pp. 1–45, 2015. [51] J.-P. Bahsoun, R. Guerraoui, and A. Shoker, “Making bft protocols really adaptive,” in IPDPS, 2015. [52] C. Wu, H. Qin, M. Javad Amiri, B. Thau Loo, D. Malkhi, and R. Marcus, “Towards truly adaptive byzantine fault-tolerant consensus,” ACM SIGOPS Operating Systems Review, vol. 58, no. 1, pp. 15–22, 2024. [53] C. Wu, H. Qin, M. J. Amiri, B. T. Loo, D. Malkhi, and R. Marcus, “BFTBrain: Adaptive BFT consensus with reinforcement learning,” in NSDI, 2025. [54] D. K. Gifford, “Weighted voting for replicated data,” in SOSP, 1979. [55] H. Garcia-Molina and D. Barbará, “How to assign votes in a distributed system,” J. ACM, vol. 32, no. 4, pp. 841–860, 1985. [56] J. Pâris, “Voting with witnesses: A constistency scheme for replicated files,” in ICDCS, 1986. [57] V. K. Garg and J. Bridgman, “The weighted byzantine agreement problem,” in IPDPS, 2011. [58] M. Nischwitz, M. Esche, and F. Tschorsch, “Raising the awareness of bft protocols for soaring network delays,” in LCN, 2022. [59] J. Köstler, H. P. Reiser, F. J. Hauck, and G. Habiger, “Fluidity: Locationawareness in replicated state machines,” in SAC, 2023. [60] G. Zhang, S. Zhang, M. Bachras, Y. Zhang, and H.-A. Jacobsen, “Cabinet: Dynamically weighted consensus made fast,” VLDB, vol. 18, no. 5, pp. 1439–1452, 2025. [61] H. Gogada, C. Berger, L. Jehl, H. P. Reiser, and H. Meling, “OptiLog: Assigning roles in byzantine consensus,” in EuroSys, 2026. [62] A. Tonkikh and L. Freitas, “Swiper: a new paradigm for efficient weighted distributed protocols,” in PODC, 2024. [63] M. A. Kaafar, L. Mathy, T. Turletti, and W. Dabbous, “Real attacks on virtual networks: Vivaldi out of tune,” in SIGCOMM LSAD Workshop, 2006. [64] S. Beckery, J. Seibert, D. Zage, C. Nita-Rotaru, and R. State, “Applying game theory to analyze attacks and defenses in virtual coordinate systems,” in DSN, 2011. [65] S. Becker, J. Seibert, C. Nita-Rotaru, and R. State, “Securing applicationlevel topology estimation networks: Facing the frog-boiling attack,” in RAID, 2011. [66] M. Sherr, M. Blaze, and B. T. Loo, “Veracity: Practical secure network coordinates via vote-based agreements,” in Usenix ATC, 2009.