1
Graph-based Hierarchical Deep Reinforcement Learning for Deliverable Block Propagation with Optimal Hybrid Cost in Web 3.0
arXiv:2604.12920v1 [cs.NI] 14 Apr 2026
Shi Chen, Jinbo Wen, Jiawen Kang, Senior Member, IEEE, Tenghui Huang, Maomao Zhang, Tao Zhang, Member, IEEE, and Dong In Kim, Life Fellow, IEEE
Abstract—Web 3.0 is envisioned as a decentralized paradigm, where blockchain serves as a core technology for transparent and tamper-proof data management. Among various blockchain architectures, consortium blockchains have emerged as the preferred platform for enterprise-grade Web 3.0. For consortium blockchains, newly generated blocks are generally propagated to all consensus nodes for validation through the gossip protocol. However, gossip-based propagation may introduce substantial message redundancy and tail latency. Moreover, the consensus nodes exhibit heterogeneous availability patterns, and existing block propagation schemes often overlook such temporal constraints. Therefore, the joint optimization of propagation timeliness and delivery coverage remains an open problem. In this paper, we propose a deliverable block propagation optimization framework for consortium blockchain-enabled Web 3.0. We first propose a delivery-aware timeliness metric called Age of Validated Block (AoVB), which excludes block receptions occurring outside the availability window of each consensus node, thereby measuring only actionable synchronization latency. This metric is unified with the block arrival rate into a hybrid cost objective that balances timeliness against delivery. To solve this complex optimization problem, we propose a Graph-based Hierarchical Deep Reinforcement Learning (GHDRL) method, which comprises a graph isomorphism network-based assignment module and a graph attention network-based propagation module. The two modules are optimized jointly under a two-stage training strategy. Numerical results show that GHDRL consistently outperforms all compared schemes across network scales from 50 to 500 peers and cluster counts K ∈ {5, 10}, achieving up to 19.2% lower hybrid cost than the best-performing neural baseline at N = 200. Moreover, the model generalizes from 100-peer training instances to 500-peer deployments without retraining. Index Terms—Web 3.0, block propagation optimization, AoVB, GHDRL, graph neural networks.
As a core enabling technology for transparent and tamperproof data management, blockchain has driven the rapid adoption of Web 3.0 decentralized applications [1]–[4], such as decentralized finance and real-world asset tokenization. These applications impose stringent requirements on the timeliness and reliability of the underlying infrastructure. Consortium blockchains have emerged as the preferred architecture for enterprise-grade Web 3.0, as their permissioned governance mechanisms effectively coordinate decentralization with regulatory compliance [5]. In consortium blockchains, transactions are sequenced into blocks by the ordering service [6]. Subsequently, the newly generated blocks are propagated to all consensus nodes for validation via the gossip protocol [7]. However, as the consortium blockchain deployment expands to hundreds of geographically distributed Internet of Things nodes, the efficiency of gossip-based block propagation among consensus nodes has become a critical performance bottleneck [7], [8]. For instance, the flooding nature of the gossip protocol generates substantial message redundancy and severe tail latency [9], which directly degrades block propagation efficiency. Although significant efforts have been conducted to block propagation optimization in public blockchains [10]–[12], these schemes are fundamentally predicated on the assumptions of permissionless participation and always-on node availability, rendering them inapplicable to consortium settings where nodes exhibit intermittent availability. Several challenges still hinder the implementation of efficient and reliable block propagation in consortium blockchains: Challenge I. Complexity of Propagation Modeling under Node Availability Heterogeneity: In consortium blockchains, nodes affiliated with diverse organizations exhibit heterogeneous and time-varying availability patterns [13], distinguishing them from the homogeneous, always-on nodes typical of public blockchains. This availability heterogeneity introduces substantial complexity into block propagation modeling, as it necessitates the simultaneous characterization of both block propagation timeliness and delivery coverage. These two objectives are inherently coupled [14], and their interaction creates a dual-objective optimization landscape that existing timeliness metrics do not capture [12], [15]. • Challenge II. Difficulty of Coupled Combinatorial Optimization: As consortium blockchain deployments scale to hundreds of nodes across multiple organizations and ge-
•
I. I NTRODUCTION Web 3.0 is envisioned as a decentralized paradigm, which enables users to independently control their identities, data, and digital assets without relying on centralized intermediaries [1]. S. Chen, J. Kang, and T. Huang are with the School of Automation, Guangdong University of Technology, Guangzhou 510006, China (e-mails: [email protected], [email protected], [email protected]). J. Wen is with the Department of Computer Science, City University of Hong Kong, Hong Kong (e-mail: [email protected]). M. Zhang is with the School of Physics and Electronic Engineering, Fuyang Normal University, Fuyang 236037, China (e-mail: [email protected]). T. Zhang is with the School of Cyberspace Science and Technology, Beijing Jiaotong University, Beijing 100044, China (e-mail: [email protected]). D. I. Kim is with the Department of Electrical and Computer Engineering, Sungkyunkwan University, Suwon 16419, South Korea (e-mail: [email protected]). Corresponding author: Jiawen Kang.
2
ographic regions, the block propagation optimization problem naturally couples two NP-hard combinatorial subproblems, namely peer-to-cluster assignment and intra-cluster sequential path planning, whose interdependence renders the joint optimization computationally intractable [16]. Although neural combinatorial optimization methods have shown promise for common routing optimization problems [17], existing approaches face significant scalability limitations when applied to this coupled problem structure. Monolithic neural encoder-decoder architectures struggle with prohibitive computational overhead at practical network scales, while decomposition-based strategies sacrifice the joint optimality of partitioning quality and routing efficiency [12], [14], [18].
the inherent conflict between latency minimization and coverage maximization under node availability constraints. To the best of our knowledge, this is the first work to jointly capture propagation timeliness and delivery coverage for block propagation in consortium blockchains. • Graph-based Hierarchical Deep Reinforcement Learning Method: We propose the GHDRL method to decouple the joint optimization problem into peer-to-cluster assignment and intra-cluster sequential path planning. The GHDRL method integrates a GIN-based encoder that produces expressive graph-level representations for cluster partitioning through cluster-dedicated MHA, with a GATbased autoregressive decoder that constructs availabilityaware propagation paths through a feasibility pruning mechanism. This pruning mechanism enforces availability To address the above challenges, we propose a deliverable constraints without modifying the neural network archiblock propagation optimization framework for consortium tecture. We further adopt a two-stage training strategy blockchain-enabled Web 3.0. Within this framework, we first to isolate propagation learning from the non-stationary propose an Age of Validated Block (AoVB) metric as a deliverycluster configurations, ensuring stable co-optimization of aware refinement of Age of Information (AoI). Unlike standard both modules. (for Challenge II) AoI, AoVB excludes block receptions that fall outside the The rest of this paper proceeds as follows. Section II availability window of each peer, thereby quantifying only surveys the related literature. Section III describes the proposed actionable synchronization delay. We further unify AoVB framework. Section IV presents the optimization problem for with the Block Arrival Rate (BAR) into a single hybrid deliverable block propagation. Section V presents the GHDRL cost objective that balances propagation timeliness against method for solving the formulated problem. In Section VI, delivery coverage. To jointly optimize propagation efficiency we describe the training procedure of the GHDRL method. and delivery reliability under node availability constraints, we Section VII reports the numerical results. Finally, Section VIII decouple the global optimization into two tightly coordinated concludes this paper. subproblems [14], namely peer-to-cluster assignment and intracluster path planning, and propose a Graph-based HierarchiII. R ELATED W ORK cal Deep Reinforcement Learning (GHDRL) method, which consists of two end-to-end learnable modules, i.e, a Graph Iso- A. Blockchain-enabled Web 3.0 morphism Network (GIN)-based peer assignment module with Recent research has explored blockchain-enabled Web 3.0 cluster-dedicated Multi-Head Attention (MHA) for partitioning, from various perspectives [4], [6], [12]. For instance, the authors and a Graph Attention Network (GAT)-based propagation in [2] provided a systematic review of decentralized finance module equipped with a feasibility pruning mechanism that protocols built on public blockchain platforms. In [3], the auenforces availability constraints without modifying the neural thors investigated the architecture and fundamental technologies network architecture. The two modules are jointly optimized of Web 3.0. Consortium blockchains have attracted particular through a two-stage training strategy [14]. Our contributions research attention for enterprise-grade Web 3.0 deployments [5]. are summarized as follows: Specifically, the authors in [6] designed Hyperledger Fabric, • Availability-Aware Block Freshness Metric: We pro- which is a permissioned blockchain platform that organizes pose a delivery-aware timeliness metric called AoVB nodes into distinct organizations managed by a Membership Serto evaluate block freshness in consortium blockchain- vice Provider (MSP), enabling fine-grained access control and enabled Web 3.0. We derive the per-hop delay model under identity management across institutional boundaries. In [19], the structured sequential propagation specific to consortium authors provided a systematic survey of blockchain theories and blockchains, where the inter-arrival process and queueing architectures, highlighting the governance advantages of consorsemantics differ fundamentally from those of public tium blockchains in balancing decentralization with enterpriseblockchains [12]. The AoVB metric incorporates a per- grade performance and trust requirements. As consortium node validity indicator to accumulate per-hop propagation blockchain deployments scale to hundreds of nodes distributed delay exclusively at peers for which reception falls across multiple organizations and geographic regions [8], the within their availability windows, thereby quantifying only efficiency and reliability of inter-node communication, particuactionable synchronization latency. (for Challenge I) larly block propagation efficiency, have become increasingly • Hybrid Cost Objective for Deliverable Block Prop- critical to the overall system performance [7]. However, most agation: We further integrate AoVB with the BAR, existing studies on blockchain-enabled Web 3.0 focus on public which measures the fraction of peers that successfully blockchain scenarios, in which nodes operate continuously receive the block during their operational periods, into a under homogeneous conditions [12], [15], overlooking the hybrid cost objective that balances propagation timeliness organizational heterogeneity and intermittent node availability against delivery coverage. This formulation addresses that characterize consortium deployments [13]. As a result, the
3
Extract the peer network
challenges of block propagation optimization in consortium blockchains remain inadequately addressed.
GHDRL Method
Refine assignment strategy
Assignment Module Block
Assign Leader Peer
B. Block Propagation Optimization Block propagation is a performance-critical process in blockchain systems, whose efficiency directly affects transaction confirmation latency and ledger consistency [7], [15]. However, the stochastic nature of the gossip protocol generates substantial message redundancy and tail latency [9], motivating extensive optimization research [7]–[9]. For instance, the authors in [9] designed an accelerated gossip protocol with incentive mechanisms to encourage cooperative forwarding among peers. In [7], the authors provided a comprehensive survey of broadcast protocols in blockchain networks, categorizing existing approaches by their communication patterns and scalability characteristics. However, these protocol-level optimizations are inherently static in their design, as they are typically tailored to fixed network assumptions, which fail to accommodate the dynamic and heterogeneous patterns of node participation prevalent in consortium blockchains. Beyond protocol-level improvements, quantifying block propagation efficiency requires appropriate timeliness metrics. AoI [15] has emerged as a well-established timeliness metric in status-update systems, quantifying the elapsed time since the latest received update was created at the source [15]. Motivated by the effectiveness of AoI, the authors in [12] introduced the Age of Block (AoB) metric, which represents each relay hop on a sequential propagation path as an M/M/1 queue governed by a first-come-first-served policy. Building upon AoB, they further developed a GAT-based block propagation optimization method to minimize total propagation delay in public blockchains. However, most existing works on block propagation fail to account for the time-varying availability of nodes throughout the propagation process. Moreover, no existing metric jointly captures propagation timeliness and delivery coverage under a unified optimization objective.
Consortium Blockchain
Optimize propagation policy
Propagation Module
Record transactions
Block propagation optimization
Evaluate timeliness and coverage Hybrid Cost
Assignment Module
Age of Validated Block
Web 3.0
Block Arrival Rate
Unify propagation timeliness with delivery coverage
Fig. 1: The deliverable block propagation optimization framework for consortium blockchain-enabled Web 3.0, where the GHDRL method serves as the core intelligent module that partitions peers into clusters and optimizes intra-cluster propagation paths, guided by a hybrid cost objective unifying the AoVB with the BAR.
making it particularly suitable for tasks that require expressive graph-level representations, such as cluster assignment tasks. However, few works have been conducted on jointly optimizing cluster assignment and intra-cluster routing under temporal availability constraints. On one hand, monolithic encoderdecoder architectures incur high attention complexity, leading to performance degradation for problems beyond moderate scales [12]. On the other hand, while divide-and-conquer strategies [14], [27] can alleviate this bottleneck, existing implementations typically decouple the assignment and routing stages heuristically, thereby sacrificing end-to-end co-optimization of partitioning quality and routing efficiency [18]. In light of the above research gaps, we propose the GHDRL method for deliverable block propagation in consortium blockchains. III. F RAMEWORK D ESIGN
C. GNNs for Combinatorial Optimization
A. Framework Overview
As illustrated in Fig. 1, the proposed deliverable block For routing optimization, attention-based encoder-decoder architectures have emerged as a dominant framework [17]. propagation optimization framework employs a two-layer Subsequent studies have further extended this framework architecture, composed of an application layer and a blockchain to address more complex problem variants [14], [20], [21]. layer. The application layer encompasses diverse Web 3.0 For instance, the authors in [20] proposed a dual-aspect decentralized applications, such as decentralized autonomous collaborative transformer to learn improvement heuristics organizations, non-fungible tokens, and real-world asset tofor routing problems. By explicitly modeling the relational kenization [3], [4]. These application-level transactions are structure among nodes, Graph Neural Networks (GNNs) have recorded on the blockchain layer. further strengthened the representational capability of neural The blockchain layer supports the operation of upper-layer combinatorial solvers [22], [23]. In particular, since they can applications through consortium blockchains with diverse orlearn adaptive edge weights through attention mechanisms, ganizational structures. Within this layer, prospective members GATs [24] have been widely adopted for routing problems. For (i.e., permissioned peers) register with certificate authorities by instance, the authors in [23] proposed a residual edge-graph submitting their spatio-temporal profiles [6]. Subsequently, a attention network to address combinatorial optimization. In [25], designated leader peer is elected through the consensus protocol the authors introduced graph gated talking-heads attention to coordinate transaction ordering and block proposal. Upon networks for the TSP. From a theoretical perspective, GIN [22] block generation, the leader peer collects spatial-temporal has been shown to be as powerful as the Weisfeiler-Lehman information of the peer network and employs the GHDRL graph isomorphism test in distinguishing graph structures [26], method to obtain an optimal block propagation strategy.
4
B. Structured Propagation Protocol In the proposed framework, we design a structured block propagation protocol that leverages the organizational decomposability inherent to consortium blockchain networks [6]. Specifically, upon block generation, the elected leader peer partitions the remaining non-leader peers into multiple disjoint clusters based on their spatial-temporal attributes. This decomposition serves two primary purposes: (i) reducing the computational complexity of global optimization by exploiting its separable structure [27], thereby decoupling the joint optimization problem into peer-to-cluster assignment and intracluster sequential path planning; and (ii) enabling parallel intracluster propagation, thereby reducing end-to-end latency. The leader peer initially propagates the block to a designated entry node within each cluster, after which intra-cluster propagation proceeds independently along optimized sequential paths. In this protocol, the cluster count K determines the number of initial transmissions from the leader peer, one per cluster, which is analogous to the push fan-out parameter in gossipbased block dissemination [8]. We set K ∈ {5, 10}, consistent with the typical fan-out configurations in practical blockchain deployments. For example, Hyperledger Fabric employs a default push fan-out of 3 [6], and Bitcoin maintains 8 outbound relay connections [28]. Our work focuses on designing propagation paths that ensure block delivery to all peers within their specified availability windows, thereby jointly optimizing block propagation timeliness and delivery coverage. Furthermore, the security of the proposed protocol is guaranteed by the permissioned governance mechanism inherent to consortium blockchains, where all peers are authenticated, and all messages are cryptographically signed [8], [9].
IV. P ROBLEM F ORMULATION FOR D ELIVERABLE B LOCK P ROPAGATION In this section, we first introduce the BAR to quantify delivery coverage under node availability constraints. Then, we propose the AoVB metric to characterize block propagation timeliness. Finally, we unify BAR and AoVB into a hybrid cost objective that balances timeliness with coverage. In this paper, we consider a consortium blockchain network N = {1, . . . , N } consisting of N peer nodes, with a single leader node ℓ elected by the MSP. The block propagation process at each hop is modeled as an M/M/1 queuing system governed by a firstcome-first-served policy [12].
the time instant at which peer i completes block reception and validation. The validity indicator of peer i is expressed as ( 1, if ti ∈ [τis , τie ], vi = (1) 0, otherwise. To quantify delivery coverage, we introduce the BAR, which measures the proportion of peers that successfully receive the block within their availability windows. For a propagation path πk = (π1 , . . P . , πmk ) containing mk peers in cluster k, we mk denote n+ i=1 vi as the number of successful receptions k = within cluster k. Thus, the overall BAR is expressed as K
η=
1 X n+ k . K mk
(2)
k=1
However, a fundamental trade-off exists between maximizing delivery coverage and minimizing propagation delay. For example, a latency-optimal strategy preferentially routes through geographically proximate peers while bypassing those with nearly expired availability windows. Such a routing strategy improves delay performance but sacrifices coverage. Conversely, a coverage-prioritizing strategy detours to temporally constrained peers, which increases cumulative delay [8]. Therefore, effectively balancing these two objectives is critical for achieving both efficient and reliable block propagation in consortium blockchains. B. Age of Validated Block AoI [15] is a well-established timeliness metric in statusupdate systems, but it treats all reception equally, without distinguishing whether a delivery falls within the availability window of receivers. AoB [12], as the blockchain-specific AoI variant, inherits this availability-agnostic limitation. To this end, we propose the AoVB, which is a delivery-aware refinement that accounts only for receptions falling within the availability window of each peer, thereby penalizing only actionable propagation delay. In the structured propagation protocol, newly generated blocks produced by the designated ordering component are forwarded directly along intra-cluster sequential paths without the inv/getdata handshake of public blockchains [12]. Let the inter-block production interval be denoted by Xi ∼ Exp(λ), where λ represents the block generation rate. The per-hop delivery time Di = ti − ti−1 measures the elapsed duration between successive block receptions, given by [12] Di = Qi + Li , +
(3)
where Qi = (Di−1 − Xi ) is the queueing delay arising when consecutive blocks contend for the same forwarding link, Li is the link transmission time, and ti−1 denotes the block reception A. Block Arrival Rate time at the preceding peer along the propagation path. The The frequent joining and departure of peers in distributed transmission time is modeled as Li ∼ Exp(1/γi ), where γi networks introduce inherent uncertainty into the system, denotes the mean one-hop transmission delay from peer (i−1) complicating consensus maintenance and authentication [29]. to peer i.Following the distance-based channel model in [23], γi Since each peer is reachable only during a limited operational is determined by the Shannon capacity between two peers [12], period [13], we formalize this intermittent connectivity by [30], which is given by associating every non-leader peer i ∈ N with an availability B block −ε , γi = (4) window [τis , τie ]. During this window period, each node can ρs c0 di−1, i b log 1 + 2 N0 b successfully receive and validate new blocks. Let ti denote
5
where Bblock is the block size, b is the channel bandwidth, ρs is the transmit power, c0 is the path-loss constant, ε is the path-loss exponent, N0 is the noise power spectral density, and −ε di−1,i is the Euclidean distance between consecutive peers, which is expressed as [23] loc di−1, i = ∥xloc i−1 − xi ∥2 ,
(5)
where xloc denotes the location coordinates of peer i, which i is recorded in the spatio-temporal profiles submitted upon registration. Thus, the average AoB at hop i is given by [12]
preferentially routes through geographically proximate peers while bypassing those whose availability windows are nearing expiration [14], thereby sacrificing coverage. Conversely, a coverage-prioritizing strategy inflates cumulative delay through detours to temporally constrained peers. Therefore, we formulate a per-cluster hybrid cost that unifies the cumulative AoVB and the per-cluster BAR, given by Jk = Ak + ρ · (1 − ηk ),
(10)
where ηk is the per-cluster BAR from (2) and the penalty coefficient ρ > 0 governs the trade-off between propagation 1 ∆i = γi + + , (6) timeliness and delivery coverage. It is worth noting that a λ 1 − λγi sufficiently large ρ prevents the trivial solution of minimizing λ2 γi3 where 1−λγi represents the queueing penalty incurred when latency by simply skipping hard-to-reach peers [14]. By substituting (6) and (8) into (10), the explicit form is given by successive blocks contend for the same transmission link. mX Based on the per-hop AoB in (6), we now incorporate the k −1 3 λ2 γi+1 n+ 1 k validity indicator to restrict timeliness accounting to actionable Jk = vi+1 γi+1 + + +ρ 1− . λ 1 − λγi+1 mk deliveries. Thus, the single-hop AoVB from peer i to peer i+1 i=1 (11) is expressed as In (11), the first term accumulates the per-hop propagation Ai→i+1 = vi+1 ∆i+1 , (7) delay exclusively over peers that successfully receive the block within their availability windows (i.e., vi+1 = 1), thereby where vi+1 is the validity indicator from (1). When peer i + capturing the timeliness cost of actionable deliveries, while 1 falls outside its availability window, vi+1 = 0 eliminates the second term imposes a coverage penalty proportional to the corresponding latency contribution. In this way, AoVB the fraction of peers that fail to receive the block during their measures only actionable propagation delay, i.e., delay incurred operational periods. Hence, the overall hybrid cost across K in successfully synchronizing the ledger of a reachable peer. clusters is given by To characterize the end-to-end timeliness of an entire cluster, K we aggregate the single-hop AoVB over all consecutive peers 1 X ¯ Jk . (12) J = along the propagation path. For a propagation path πk = K k=1 (u1 , . . . , umk ) within cluster k, the cumulative AoVB is Let the leader peer ℓ divide the remaining N −1 peers mX k −1 P = N \ {ℓ} into K disjoint clusters {Ck (a)}K k=1 via an Ak = vi+1 ∆i+1 . (8) assignment mapping a : P → {1, . . . , K}, where each i=1 intra-cluster propagation path πk = (π1 , . . . , πmk ) forms a The cumulative AoVB in (8) is coupled through the sequential Hamiltonian path on Ck (a). Finally, the joint optimization nature of propagation,P namely the completion time at node πt problem is formulated as t−1 satisfies tπt = tπ1 + j=1 Dj , where tπt denotes the block K reception time at the t-th peer along path πk and tπ1 denotes 1 X ¯ Jk (πk , Ck (a)) min J= the block arrival time at the source node of the propagation K (13) a, {πk }K k=1 k=1 path. Thus, the validity indicator at each node depends on the s e s.t. τπt ≤ tπt ≤ τπt , cumulative system times of all upstream hops, given by ( P t−1 where the constraint in (13) ensures that every peer along the 1, if τπst ≤ tπ1 + j=1 Dj ≤ τπet , (9) propagation path receives the block before its availability winvπ t = 0, otherwise, dow expires. Problem (13) couples two NP-hard combinatorial e where τπt is the corresponding availability window deadline. subproblems, i.e., cluster partitioning and intra-cluster path planning, whose interdependence precludes straightforward decomposition. Given that conventional optimization methods C. Optimization Objective struggle to efficiently solve such coupled NP-hard problems, To ensure that validated block receptions along each prop- we propose the GHDRL to solve Problem (13). agation path achieve high timeliness, i.e., peers synchronize the latest ledger state as promptly as possible, the optimization V. GHDRL M ETHOD FOR D ELIVERABLE B LOCK objective is to minimize the cumulative AoVB Ak for each P ROPAGATION O PTIMIZATION cluster. To simultaneously guarantee that the block is delivered to as many peers as possible within their availability windows, A. Peer Assignment Module i.e., achieving high delivery coverage as required by the We first design a peer assignment module to partition (N −1) deliverable propagation criterion, the optimization objective is non-leader peers into K clusters based on their spatial and also to maximize the per-cluster BAR ηk . However, these two temporal attributes. To enable end-to-end learning, we cast objectives are inherently conflicting. A latency-optimal strategy this module as a single-step Markov decision process [14]. In λ2 γi3
6
Assignment Module
Propagation Module
2. Multi-Head Attention Cluster Embedding
3. Centralized Allocation Layer
4. GAT Encoder
1. Graph-based Representation Cluster 1 3-Layer GIN Cluster 2
Layer 1
…
…
Layer 2
Cluster k
Input I
Layer 3
Sampling
… +
?
…
SingleHeed Attention
…
6. Feasibility Pruning
…
…
·
…
…
Assignment Probabilities
:
Graph Pooling
5. GAT Decoder
Peers Embeddings
Unacceptable peer
High
Med
Low
Fig. 2: Architecture of the proposed GHDRL method, comprising an assignment module with a GIN encoder, cluster-dedicated MHA, and a centralized allocation layer for peer-to-cluster partitioning, and a propagation module with a GAT-based encoder-decoder for autoregressive path construction and a feasibility pruning mechanism for enforcing availability window constraints.
consortium blockchain networks, the positions and operational states of peer nodes are inherently dynamic, continuously evolving in response to changing network conditions. To capture this stochastic nature, we define a probability distribution p(I) over the space of all possible network configurations, where each instance I ∼ p(I) represents a specific realization of the network topology at a given time. Given a consortium blockchain node distribution instance I sampled from the distribution p(I), we represent the current state sI as a fully connected graph GI = (VI , EI ) [31], where the node set VI is defined as VI = P ∪ {ℓ},
(14)
where P = {p1 , p2 , . . . , pN −1 } denotes the set of ordinary peers, and ℓ represents the leader peer. The edge set is EI = {(u, v) | ∀u, v ∈ VI }, where each edge (u, v) is associated with the communication latency between nodes u and v. We define the action asI as the assignment of each node to one of the K clusters. To establish the connection between the two modules, the reward function is expressed as K
R(asI , sI ) = −
1 X Jk , K k=1
(15)
which has been theoretically shown to be as powerful as the Weisfeiler-Lehman graph isomorphism test in distinguishing graph structures [26]. Given the input graph GI , we stack L GIN layers to progressively refine the node embeddings. The node embedding update rule at layer l is expressed as X (l) S(l−1) , (16) S(l) (1 + ϵ(l) ) · S(l−1) + v = MLP v u u∈N (v) (l)
where Sv ∈ Rdh denotes the embedding of node v at layer l, N (v) represents the set of neighboring nodes of v, ϵ(l) is a learnable parameter that adjusts the weight of the central node’s self-information, and MLP(l) (·) is a multi-layer perceptron that introduces non-linear transformation capacity. After L layers of message passing, the GIN produces the (L) final node embeddings: the leader embedding Sℓ and (N − 1) (L) N −1 peer embeddings {Spi }i=1 . To obtain a holistic graph-level representation that summarizes the global network state, we apply an attentive graph pooling mechanism [32] over the peer node embeddings. Specifically, the attention score for each peer node is given by epi = W2⊤ · tanh W1 S(L) + b (17) 1 , pi
which represents the negative average hybrid cost fed back where W1 ∈ Rdh ×dh , b1 ∈ Rdh , and W2 ∈ Rdh are learnable from the block propagation module. This design enables the parameters. The normalized attention scores are obtained as peer assignment module to learn cluster configurations that exp(epi ) facilitate efficient intra-cluster propagation. αpi = PN −1 . (18) We parameterize the policy πθA (asI |sI ) with trainable j=1 exp(epj ) parameters θA . As illustrated in the upper part of Fig. 2, we design the policy network with three key components: graph- Based on the normalized attention scores, we aggregate all peer based state representation, MHA for cluster embedding, and a embeddings into a single vector that summarizes the global network state. The graph-level representation SG is given by centralized allocation layer. Each component is as follows: 1) Graph-based representation: To obtain expressive node N −1 X representations that capture both local neighborhood strucSG = αpi · S(L) (19) pi . tures and global graph properties, we adopt the GIN [22], i=1
7
This weighted aggregation ensures that structurally important peers, which receive higher attention scores, contribute more prominently to the global representation. To provide the subsequent cluster embedding module with comprehensive contextual information that encompasses both the global network state and the perspective of the leader, the context embedding is obtained as (L)
Sc = [SG ∥ Sℓ ], Sc ∈ R2dh ,
(20)
3) Centralized allocation layer: Based on the cluster embed(L) N −1 dings {Sk }K k=1 and peer embeddings {Spi }i=1 , we design a centralized allocation layer to perform the final cluster assignment. Following the assignment policy design in neural combinatorial optimization [14], we adopt a single-head selfattention structure to model the centralized allocation policy, (k) which computes the assignment probability βi for assigning peer pi to cluster k. We employ a sequential assignment strategy. At each step, the peer with the highest attention score is assigned to a cluster and then masked from subsequent selections. This process continues until all (N − 1) peers are assigned to one of the K clusters, yielding K disjoint cluster subsets {Ck }K k=1 . We then train the policy πθA (asI |sI ) using the RL algorithm [17] by maximizing X EI∼p(I) (R(asI , sI ) − b(I)) πθA (asI |sI ) , (24)
where [·∥·] denotes the concatenation operation. Incorporating the leader embedding alongside the global graph summary provides the downstream modules with both a macro-level view of network connectivity and the local perspective of the propagation source. The context embedding Sc , together with (L) −1 the (N − 1) peer embeddings {Spi }N i=1 , are subsequently fed into the MHA mechanism for cluster embedding. asI 2) MHA for cluster embedding: To generate differentiated cluster representations for assigning the (N − 1) peer nodes where b(I) is a baseline function introduced for variance into K clusters, we propose an MHA mechanism where reduction following [17]. The resulting cluster assignments each attention head is dedicated to one specific cluster [33]. are then passed to the block propagation module for intraIndependent parameterization enables each head to develop specluster path optimization. cialized attention patterns tailored to cluster-specific spatial and temporal characteristics, preventing collapse into homogeneous representations. B. Block Propagation Module For the k-th attention head corresponding to cluster k, we Once the peer assignment module partitions nodes into K compute the query vector from the context embedding Sc and clusters, we decompose Problem (13) into K independent intra(L) −1 key-value pairs from the peer embeddings {Spi }N i=1 : cluster subproblems. For each cluster k, our block propagation module determines the optimal path πk = (π1 , . . . , πmk ) that (k) (k) (k) (k) (k) q(k) = WQ Sc , ki = WK S(L) = WV S(L) pi , vi pi , (21) minimizes the hybrid cost Jk , where π1 denotes the designated source node and πt ̸= πt′ for all t ̸= t′ . (k) (k) (k) dk ×dh where WQ , WK , WV ∈ R are the cluster-specific As shown in the lower portion of Fig. 2, given that projection matrices, and dk = dh /K denotes the dimension of GATs effectively capture inter-node relationships via attention each head. After projecting the context and peer embeddings mechanisms, we design the propagation module using a GATinto the cluster-specific subspace, we measure the relevance of based encoder–decoder architecture [12]. The encoder extracts each peer to the corresponding cluster via scaled dot-product structural features from the cluster subgraph, while the decoder attention. The attention score between the context and each generates the propagation path autoregressively. We also design peer node is obtained as [33] a feasibility-pruning mechanism to ensure that the generated path satisfies all availability-window constraints. √ (k) ⊤ (k) 1) Encoder architecture: The encoder extracts structural exp (q ) ki / dk (k) βi = P . (22) features of each intra-cluster subgraph and produces expressive √ N −1 (k) )⊤ k(k) / d k j j=1 exp (q node-level and graph-level representations for the subsequent decoder. The encoder comprises three components: (i) an input Based on the attention scores, we compute the cluster embed- representation that constructs the cluster subgraph and extracts ding Sk for cluster k as a weighted aggregation of the peer peer features, (ii) an initial embedding layer that projects raw value vectors, which is given by features into a high-dimensional hidden space, and (iii) a stack of L graph attention layers that progressively refine N −1 X node embeddings through MHA, Batch Normalization (BN), (k) (k) Sk = βi · v i . (23) and FF sublayers. Specifically, i=1 • Input representation. For each cluster k output by the This process yields K cluster embeddings {Sk }K , each peer assignment module, we construct its corresponding k=1 encoding the aggregated information of peer nodes relevant to membership view Gk = (Vk , Ek ), represented as a fully its corresponding cluster. The independent parameterization of connected graph with a node set Vk = {1, . . . , mk } each attention head ensures that the K embeddings capture and an edge set Ek . For each node i, we construct a distinct spatial-temporal patterns, thereby providing the subsefeature vector xi = (xi , yi , τis , τie ) ∈ Rdx containing quent allocation layer with differentiated representations for location coordinates (xi , yi ) and availability window assignment decisions. [τis , τie ], where dx = 4.
8
•
Initial embedding layer. We map the raw node features to a dh -dimensional hidden space through a learnable linear projection [12], which is given by (0)
Si
= Wx xi + bx ,
(25)
2) Decoder architecture: We design the decoder to generate the propagation path in an autoregressive manner. At each decoding step t, the current node selects the next propagation target from its neighbors and appends it to the path sequence. To represent the current decoding state, we construct a context vector ct that captures both global structural information and local path history. According to [17], the context vector is expressed as if t = 1, [S̄Gk , p1 , p2 ], (L) ct = [S̄Gk , p2 , Sπ1 ], (33) if t = 2, (L) (L) [S̄Gk , Sπt−2 , Sπt−1 ], if t > 2,
where Wx ∈ Rdh ×dx and bx ∈ Rdh , with dh = 128. • Graph attention layer. After obtaining the initial em(0) k beddings {Si }m i=1 , we update them through L stacked GAT layers. Each layer comprises three sublayers: an MHA sublayer, a BN sublayer, and a Feed-Forward (FF) sublayer, connected via residual connections [12]. For the h-th attention head (h ∈ {1, . . . , H}), the query, key, and value vectors are given by where p1 , p2 ∈ Rdh are learnable placeholders. We incorporate the graph embedding S̄Gk to enable the decoder to exploit (l) (l−1) (l) (l−1) (l) (l−1) qih = WhQ Si , kih = WhK Si , vih = WhV Si , global structural information. When t > 2, we replace the place(26) holders with the embeddings of the two previously selected where WhQ , WhK ∈ Rdk ×dh and WhV ∈ Rdv ×dh , with nodes, thereby enabling AoVB-aware sequential decisions. The dk = dv = dh /H. To enforce the topological structure query, key, and value vectors are given by [17] of the cluster subgraph, we restrict the attention compu(d) (L) (L) qt = WQ ct , ki = WK Si , vi = WV Si . (34) tation to pairs of nodes that are directly connected. The compatibility score between nodes i and j is given by [23] To prevent repeated selection of nodes that have already (l) ⊤ (l) received the block, we apply a masking operation to their (q ) k ih √ jh , if i is adjacent to j, (l) compatibility scores, which are denoted as uijh = (27) dk ! −∞, (d) ⊤ otherwise. C · tanh (qt√ ) kj , if j ̸= π ′ , ∀t′ < t, t (35) ũt,j = dk By setting the compatibility of non-adjacent nodes to −∞, −∞, otherwise, we effectively prevent block propagation between disconnected nodes. The attention weights are then obtained as where C represents a clipping constant to stabilize training [33]. Finally, the selection probability is obtained as
(l)
exp(uijh ) (l) aijh = Pm . (l) k j ′ =1 exp(uij ′ h )
(28)
After computing the attention weights across all H heads independently, we concatenate their weighted outputs through learned projection matrices WhO . The MHA output is expressed as [17] mk H X X (l) (l) (l) MHAi = WhO aijh vjh . (29) h=1
j=1
The complete GAT layer combines the three sublayers through residual connections with BN [12], which is expressed as (l) (l−1) (l) Ŝi = BN(l) Si + MHAi , (30) (l)
Si = BN(l)
(l) (l) Ŝi + FF(l) (Ŝi ) ,
(31)
where FF(l) (·) = Wf 2 ReLU(Wf 1 (·) + bf 1 ) + bf 2 . (L)
k Finally, the encoder outputs the node embeddings {Si }m i=1 together with a global graph embedding, which is given by [17]
S̄Gk =
mk 1 X (L) S . mk i=1 i
(32)
Both outputs are subsequently consumed by the decoder to construct the intra-cluster propagation path.
exp(ũt,i ) . pθP (πt = i | Gk , π1:t−1 ) = Pmk j=1 exp(ũt,j )
(36)
3) Feasibility pruning: Due to availability window constraints, the initial path πk = (π1 , . . . , πmk ) generated by the decoder may contain nodes whose deadlines expire before block reception. We apply a feasibility pruning mechanism that maps the decoder output to a guaranteed feasible solution πk∗ [27], separating hard constraint satisfaction from the learning problem without modifying the core network architecture. The cumulative arrival time at the t-th node along the path is denoted as Γt =
t−1 X
γπi ,πi+1 , t ∈ [2, mk ],
(37)
i=1
with Γ1 = 0 for the source node, where γπi ,πi+1 is the mean one-hop transmission delay from (4). A node πt is feasible if and only if Γt ≤ τπet , i.e., the block arrives before the availability window deadline of that peer. The pruning proceeds sequentially. Let p index the last retained node, initialized as p = 1. For each subsequent node πt , we compute the updated arrival time Γ′t = Γ′p +γπp ,πt from the last retained node πp . If Γ′t ≤ τπet , node πt is retained and p ← t; otherwise, πt is removed and Γ′ remains unchanged. Let F ⊆ {1, . . . , mk } collect the indices of all retained nodes. The feasible path is given by πk∗ = (πt )t∈F .
(38)
9
VI. M ODULE T RAINING OF GHDRL A. Two-Stage Training Strategy Problem (13) involves two coupled decisions: partitioning N peer nodes into K clusters and determining the propagation path πk within each cluster. Joint optimization via a single policy gradient estimator is impractical because the two modules operate over fundamentally different action spaces, leading to dynamically varying cluster sizes that render the propagation subproblems non-stationary during training [14], [17]. For any fixed assignment a, the per-cluster propagation cost J (πk |Gk (a)) depends solely on the topology and node attributes of Gk (a), independent of the assignment mechanism that produced it. Hence, the nested formulation is expressed as K
min a
1 X min J (πk | Gk (a)), πk K
(39)
k=1
where the inner propagation subproblems can be solved independently once a is fixed. Therefore, we adopt a two-stage training strategy [23]. In Stage 1, the propagation module is trained on fixed-size subproblems sampled from representative distributions, yielding a generalizable path optimization policy with parameters θP∗ . In Stage 2, the propagation parameters are frozen at θP∗ , and the assignment module is trained with the average hybrid cost J¯ as the learning signal. This isolation ensures training stability by preventing the non-stationary cluster configurations from interfering with propagation learning [14], while preserving architectural modularity that allows substitution of alternative path planners without retraining. B. Propagation Module Training (Stage 1) The propagation module is trained in Stage 1 using policy gradient methods. Given a subgraph Gk , the propagation policy pθP autoregressively constructs a candidate path πk , which is then processed by the feasibility pruning mechanism to obtain a feasible path πk∗ = ϕ(πk ). The training objective minimizes the expected hybrid cost of the feasible path, given by L(θP |Gk ) = EpθP (πk |Gk ) J ϕ(πk )|Gk . (40)
Algorithm 1: Propagation Module Training (Stage 1) Input: Training set S, batch size BP , number of epochs E, and the significance level α Output: Trained parameters θP∗ bl 1 Initialize θP , θP ← θP ; 2 for epoch = 1, . . . , E do 3 for each batch of BP instances from S do 4 for i = 1, . . . , BP do 5 Sample a candidate path: (i) (i) πk ← SampleRollout(Gk , pθP ); 6 Greedy baseline path: (i),bl (i) πk ← GreedyRollout(Gk , pθPbl ); 7 Apply pruning and evaluate cost: (i) (i) J (i) ← J ϕ(πk ), Gk ; (i),bl (i) 8 J (i),bl ← J ϕ(πk ), Gk ; 9 10 11 12
Estimate the policy gradient through (41); Update parameters: θP ← Adam(θP , ∇L); if OneSidedPairedTTest(pθP , pθPbl ) < α then θPbl ← θP ;
θP∗ ← θPbl ; ∗ 14 return θP ;
13
pairwise attention scores over mk nodes with hidden dimension dh , yielding per-layer complexity O(m2k dh ). The autoregressive decoder generates a path of at most mk steps, with each step requiring O(mk dh ) for attention computation. Therefore, the computational complexity of Algorithm 1 can be expressed as O(Lm2k dh ). C. Assignment Module Training (Stage 2) With the propagation parameters frozen at θP∗ from Stage 1, the assignment module is trained to optimize the cluster partition. The training objective minimizes the expected average hybrid cost across all clusters, which is expressed as " # K 1 X ∗ L(θA |I) = EpθA (asI |sI ) J (πk |Gk ) , (42) K
We optimize the loss via the RL algorithm [17] with a rollout k=1 baseline b(Gk ) for variance reduction, which is expressed as where θ denotes the assignment module parameters, I is the A h i ∗ ∗ input instance, and J (π | G ) is computed by the frozen ∇L(θP |Gk ) = E J (πk |Gk ) − b(Gk ) ∇ log pθP (πk |Gk ) . k k propagation module. (41) Similarly, we adopt RL with a rollout baseline b(I) for Note that the cost is evaluated on the pruned path ϕ(πk ) while variance reduction. The policy gradient is given by the gradient is computed with respect to the original path πk . This formulation remains unbiased by the score function K h 1 X property of the RL estimator and is standard practice for ∇L(θA |I) = E J (πk∗ |Gk ) − b(I) K (43) handling hard constraints via deterministic post-processing [27]. k=1 i The baseline is computed by greedy decoding under a · ∇ log pθA (asI |sI ) , separate set of parameters θPbl , updated via the conservative strategy of [17]. At each epoch boundary, θPbl ← θP is with the same conservative baseline update rule. During Stage 2, executed only if a one-sided paired t-test at significance level the frozen propagation module employs greedy decoding, which α = 5% confirms improvement. The training procedure of the selects the highest-probability node at each autoregressive step, propagation module is given in Algorithm 1. The computational to deterministically evaluate each candidate cluster, ensuring complexity of Algorithm 1 is analyzed below. Specifically, the that the cost feedback Jk is a deterministic function of the GAT encoder consists of L attention layers, each computing assignment action a alone. Training instances are dynamically
10
Algorithm 2: Assignment Module Training (Stage 2) Input: Instance generator D, batch size BA , number of epochs E ′ , frozen parameters θP∗ , and the significance level α ∗ Output: Trained parameters θA bl 1 Initialize θA , θA ← θA ; ′ 2 for epoch = 1, . . . , E do 3 for each batch of BA instances from D do 4 for j = 1, . . . , BA do 5 Sample assignment: a(j) ← AssignmentPolicy(I (j) , pθA ); 6 Baseline assignment: a(j),bl ← AssignmentPolicy(I (j) , pθAbl ); 7 Partition peers into K clusters: (j) (j) 8 {Gk }K k=1 from a ; (j),bl K 9 {Gk }k=1 from a(j),bl ; 10 for k = 1, . . . , K do 11 Evaluate cluster k with frozen θP∗ : (j) (j) 12 πk ← GreedyRollout(Gk , pθP∗ ); (j) (j) (j) 13 Jk ← J ϕ(πk ), Gk ; (j),bl (j),bl 14 πk ← GreedyRollout(Gk , pθP∗ ); (j),bl (j),bl (j),bl ; 15 Jk ← J ϕ(πk ), Gk Compute the average hybrid cost: PK (j) 1 J¯(j) ← K k=1 Jk ; P (j),bl K 1 J¯(j),bl ← K ; k=1 Jk
16 17
18 19 20 21
Estimate the policy gradient through (43); Update parameters: θA ← Adam(θA , ∇L); if OneSidedPairedTTest(pθA , pθAbl ) < α then bl θA ← θA ;
∗ bl θA ← θA ; ∗ 23 return θA ; 22
VII. N UMERICAL R ESULTS A. Experimental Setup Following the common settings in DRL for combinatorial optimization, we randomly generate membership view instances of varying scales [17]. Specifically, we select the number of nodes N from {50, 100} for training and basic testing, and from {150, 200, 300, 500} for evaluating generalization capability. The number of clusters K is set to 5 and 10 following the fanout analysis in Section III-B, where K = 5 represents a more challenging scenario due to a heavier load per cluster. Node locations follow a uniform distribution (xi , yi ) ∈ U[0, 1]2 [34]. The availability windows are configured as τis ∼ U[0, 3] with τie = τis + 3 [14]. The leader peer ℓ is fixed at (0.5, 0.5) with availability window [0, 10]. For the communication model, we set the block size to Bblock = 1 MB, the transmit power of peers to ρs = 23 dBm, and the noise power density to N0 = −174 dBm/Hz [1], [35], [36]. We set the path-loss coefficient to ε = 3.38 and the unit channel gain to c0 = −30 dB, while the channel bandwidth between adjacent miners is selected from the set 180 kHz, 22 MHz, 100 MHz [12], [37]. We fix all hyperparameters after tuning on instances with N = 50 and K ∈ {5,√10}. The√parameters θ are initialized uniformly within (−1/ dh , 1/ dh ) [17], where dh = 128. For the propagation module, training is conducted over 100 epochs using dynamically generated data, with 3000 batches of 512 instances processed per epoch. The encoder employs a 3-layer GAT [24] with H = 8 attention heads and FF dimension dF = 512, following the standard multi-head attention configuration [17], [33]. For the assignment module, we train for 500 iterations with 128 dynamically generated instances per iteration, using a 3-layer GIN with ϵ(l) = 0 [22] whose Multi-Layer Perceptron (MLP) contains 2 hidden layers of dimension 32. The penalty coefficient is set to ρ = 1000 to ensure sufficiently high delivery coverage. Both modules use the Adam optimizer with a learning rate 10−4 . During inference, we train on 100-node instances and directly apply the model to all larger scales (N ≥ 100) without retraining, following the common generalization protocol [17], [34]. For N = 50, we use the model trained on 50-node instances. The entire set of experiments is performed using an NVIDIA GeForce RTX 3080Ti.
generated to ensure diversity, enabling generalization to unseen network configurations. The training procedure of the assignment module is given in Algorithm 2. For its computational complexity, the GIN encoder in the assignment module processes the full graph with N nodes over L′ layers, leading to O(L′ N 2 dh ) [22], and the MHA mechanism for cluster embedding causes O(K 2 dh ) [33]. Thus, B. Comparison Results the computational complexity of Algorithm 2 is O(L′ N 2 dh + We compare our GHDRL method against four baseline K 2 dh ). Combining Algorithms 1 and 2, for K clusters with an schemes: i) Random, which randomly selects neighbors for average size m̄ = N/K, the overall computational complexity block propagation without any optimization strategy; ii) Greedy, of the GHDRL method is O((L′ +L/K)N 2 dh +N 3 dh /K 2 ). In which always selects the nearest neighbor, representing distancecomparison, a monolithic approach that optimizes propagation based optimization; iii) GAT [12], which is an attention-based over all N nodes without decomposition incurs O(N 3 dh ) GNN model, processing the entire network via O(N 2 ) selfdue to the quadratic self-attention cost at each of the N attention without decomposition; iv) Cross Attention-based autoregressive decoding steps [33]. Our decomposition can Transformer (CAT) [27], which employs a lightweight crossreduce the dominant cubic term by a factor of K 2 . While larger attention mechanism that propagates node embeddings through K reduces per-cluster subproblem complexity, it proportionally a fixed number of representative nodes, achieving O(N ) linear increases the number of initial transmissions from the leader complexity for large-scale routing. peer. Furthermore, since the K propagation subproblems are Figure 3 shows the performance comparison among different independent, parallel execution reduces the complexity of wall- baselines under varying network scales and cluster counts. From clock propagation to O(N 3 dh /K 3 ). Fig. 3a, we observe that at small scales (N ∈ {50, 100}), our
11