arXiv:2605.27027v1 [cs.LG] 26 May 2026
SQARL: A Size-Agnostic Reinforcement Learning approach for Circuit Allocation in Distributed Quantum Architectures
Vı́ctor Carballo 1
Júlia López-Closa 2
Mario Martin 1
Abstract
1. Introduction
The scaling of quantum processors is currently limited by technical challenges such as decoherence and crosstalk. As the number of qubits grows, interference increases the computational noise. Distributed quantum computing addresses these limitations by interconnecting smaller, easier-to-handle quantum processors (cores), but it introduces the challenge of minimizing slow, errorprone inter-core communication. The task of distributing quantum circuits across cores while minimizing communication costs is known as the Qubit Allocation problem. This work focuses on developing a deep learning approach to this problem, emphasizing flexibility to quantum hardware topology and improving state-of-the-art performance.
Quantum computing has the potential to revolutionize numerous fields, including chemistry (Orobator et al., 2025), cryptography (Mavroeidis et al., 2018), computer science (Shor, 1999), and many others. This promise, however, is constrained partly by the number of qubits a device can effectively control, which, even in the most advanced quantum computers available today, is on the order of a thousand (Gambetta, 2023; Atom Computing). Reaching the number of qubits required by real-world applications, on the order of millions (Preskill, 2018), is challenging due to decoherence, a phenomenon characterized by random fluctuations in qubit states and cross-talk between them. In traditional single-chip processors, scaling qubit counts is constrained by the growing complexity of control circuitry and wiring required to maintain low error rates (Mohseni et al., 2025). As a response, multi-core architectures that aim to reduce cross-talk and interference while preserving the benefits of quantum computing have recently gained popularity (Vandersypen et al., 2017; Jnane et al., 2022). These devices consist of multiple interconnected quantum processors, also known as cores. The program to be executed is now distributed across smaller, more manageable quantum devices. Yet, multi-core architectures introduce their own challenges; in particular, reduced fidelity in nonlocal communications and additional time overhead when applying gates across cores (Rodrigo et al., 2021a). It is therefore imperative to distribute the quantum programs so as to minimize the number of expensive inter-core communications. Fig. 1 shows an example of circuit allocation.
Heuristic and non-learning algorithms, such as the Hungarian Qubit Allocation (HQA), currently represent the state of the art. Reinforcement Learning (RL) approaches leverage learned allocation policies but often lack flexibility, requiring retraining when hardware configurations change, and they fall short of the solution quality achieved by non-learning methods. However, learning mechanisms could outperform human-crafted heuristics. To overcome these limitations, this work proposes a flexible, transformer-based architecture that can handle arbitrary numbers of qubits and cores without retraining. Results show that the trained policy consistently outperforms the previous RL state of the art and narrows the gap between RL and HQA for the most common circuits. It achieves a 33% reduction in allocation cost relative to the HQA for the Cuccaro Adder and 25% on average for random circuits. These findings show that learningbased approaches can effectively match the performance of hand-crafted heuristics, a crucial step towards their application in real-world scenarios.
Quantum programs, also known as circuits, are a high-level representation that specifies a sequence of operations on the qubits. Given a quantum circuit, qubit allocation is the mapping of the circuit’s qubits (logical qubits) to the actual qubits in the device’s cores (physical qubits), accounting for the device’s hardware topology. Finding a mapping that minimizes communication is known to be NP-complete for single-chip processors (Siraichi et al., 2018; Botea et al., 2021); similarly, minimizing inter-core operations in multicore architectures leads to a combinatorial optimization problem. Hence, the existing literature often reformulates
1
Computer Science Department, Universitat Politècnica de Catalunya - BarcelonaTech (UPC) 2 High Performance Artificial Intelligence group, Barcelona Supercomputing Center. Preprint. Under review.
1
SQARL: A Size-Agnostic RL approach for Circuit Allocation in Distributed Quantum Architectures
it as other well-known combinatorial problems, such as graph partitioning (Baker et al., 2020), quadratic minimization (Bandic et al., 2023), or resource allocation (Escofet et al., 2023), and then solves it using classical optimization algorithms.
near–state-of-the-art performance on most of the remaining ones. To demonstrate these claims, we benchmark SQARL against the current SOTA classical algorithm, the Hungarian Qubit Allocation, proposed in (Escofet et al., 2023), and the best-performing existing RL-based approach, Russo’s algorithm from (Russo et al., 2024). Notably, while the algorithm is trained exclusively on small circuits (less than 20 qubits and 16 time slices), later benchmarks on much larger circuits (100 qubits and hundreds of time slices) show consistent performance on unseen problem sizes. Overall, this work makes the following contributions: 1. We compare the allocation cost of non-learning and state-of-the-art RL methods. 2. We propose a novel, problem-size-agnostic policy architecture that can adapt to varying problem sizes without retraining. 3. We propose a novel allocation procedure (sequential and parallel, as described in Sec. 4.3) that improves the algorithm’s performance. 4. We leverage a combination of REINFORCE and GRPO as training algorithms so as to enhance stability during training.
Figure 1. Illustrative example of circuit allocation. The x axis represents the time steps required to execute the circuit. The y axis represents each one of the actual qubits in the quantum hardware (physical qubits). The different processors in the distributed system are divided by dotted lines. The qubits in the original circuit (logical qubits) are labeled and color-coded; gates are drawn as black, vertical lines joining qubits.
The work is structured as follows. In Sec. 2 we provide an introduction to quantum computing, the problem of qubit allocation, and RL with a special focus on combinatorial optimization. Then, in Sec. 3 we benchmark the state of the art in non-learning and RL algorithms, and discuss the performance gap and what needs to be improved in RL methods applied to this problem for them to be competitive. Next, in Sec. 4 we present our approach to quantum allocation via RL, SQARL. Sec. 5 covers the training procedure, hyperparameter selection, and benchmarks with both methods discussed in Sec. 3. In Sec. 6, we discuss the results from the previous section. Finally, in Sec. 7 we present our conclusions and future work.
Inspired by recent breakthroughs in deep learning, more recent works explore applying reinforcement learning (RL) to the problem (Pastor et al., 2024; Russo et al., 2024). RL methods train a policy to output a probability distribution over the assignment of qubits to cores. Despite the success of RL in fields like robotics control or table games, the solution quality of RL methods lags behind that of classical solvers such as the Hungarian algorithm. Furthermore, although not inherent to RL, existing approaches require retraining from scratch whenever the number of qubits or cores changes. This is a crucial limitation to their applicability in real-world scenarios.
The complete implementation is public and can be found in the project repository https://github.com/ Vicara12/SQARL, along with the benchmarking circuits in JSON format (Carballo, 2026).
In this paper, we present SQARL (Scalable Qubit Allocation via Reinforcement Learning), an RL-based approach that greatly narrows the gap in solution cost relative to classical optimization methods. In contrast to previous approaches, SQARL is unconstrained by circuit size and hardware configuration, thanks to its qubit- and core-agnostic architecture, enabling generalization across arbitrary hardware topologies and qubit counts without requiring retraining. One trains the model once, and it can be used for circuits of any size, which we define as flexibility. At the same time, SQARL achieves state-of-the-art performance on many quantum circuits and
2. Preliminaries RL approaches to quantum allocation require concepts from quantum computing and artificial intelligence. In this section, we provide the necessary background to understand the rest of the work. 2.1. Quantum Computing Quantum programs specify how quantum information is initialized, transformed, and measured in order to implement 2
SQARL: A Size-Agnostic RL approach for Circuit Allocation in Distributed Quantum Architectures
an algorithm. In the circuit model, this corresponds to a finite sequence of operations applied to the qubits prior to measurement. Analogous to logic gates in conventional digital circuits, quantum gates are operators represented as unitary matrices that act on qubits by transforming their states. A quantum circuit is executed by applying a sequence of quantum gates to a set of qubits over time. Quantum circuits are usually designed in an architectureagnostic manner. As a result, they cannot be executed directly on a specific quantum device without accounting for hardware constraints. For instance, the set of possible quantum gates is infinite; yet, due to the challenges of gate calibration and control over many qubits, practical quantum hardware supports only a finite subset. From this universal gate set, any quantum circuit can be approximated with arbitrary accuracy. Consequently, multi-qubit operations are compiled into sequences of one- and two-qubit gates supported natively by the target hardware.
Figure 2. An annotated example of a quantum circuit.
at the same time. For instance, consider the circuit in Fig. 3, which can be divided in three different time slices.
Another constraint is limited qubit connectivity: two-qubit gates can only be applied to physically connected qubits. The arrangement of qubits, also known as the qubit topology, can limit the execution of a circuit. In practice, quantum computers rely on inserting SWAP operations (i.e., information exchanges between two physical qubits) to relocate quantum states and enable the required interactions. These SWAPs, typically a sequence of three two-qubit CNOT gates, can only be applied to adjacent qubits in a chip; hence, such operations are inherently intra-core. In contrast, intercore communications use the quantum teleportation mechanism, which has 5x to 100x longer latencies and incurs error rates 10x to 100x worse than two-qubit gates (Baker et al., 2020; Rodrigo et al., 2021b). Therefore, it is imperative to reduce the number of expensive non-local communications when mapping a circuit to the hardware.
Figure 3. An example of circuit slicing.
2.2. The Problem of Qubit Allocation Given a quantum circuit and a quantum hardware configuration, the problem of qubit allocation is to distribute the qubits across the different quantum cores for each circuit time slice in such a way that the total cost of the inter-core communications is minimized.
This mapping of a quantum program to a specific gate set and connectivity is handled by a quantum compiler. More generally, the quantum compiler is responsible for all processes required to prepare a circuit for execution, including circuit synthesis and optimization, transpilation, qubit routing, and qubit allocation (Bandic et al., 2023). Efforts to minimize the effects of decoherence generally focus on one or more of these processes.
A time-sliced quantum circuit G (refer to Sec. 4.1 for the slicing procedure) is a list of T time slices St , t ∈ {0, ..., T − 1}. Each time slice St is a list of gates, with no repeated qubits. A gate is a two-element set {qa , qb } that contains the qubits that participate in the gate. The quantum hardware is defined by the number of cores C, the vector with the number of qubits each core can hold c, and a matrix of inter-core communication costs F. The entry F(i, j) indicates the cost of moving a qubit from core i to core j in some defined metric (time, error probability, etc). Although in real setups not all cores may be connected, we consider a fully connected cost matrix F; if two cores are not directly connected, the corresponding entry in F contains the most efficient path through other cores. The resulting allocation is a matrix of size T × Q over the set of cores {0, . . . , C − 1} that contains which core a qubit is assigned to in a specific time slice, that is R ∈ {0, . . . , C − 1}T ×Q . R(i, j) = k
Quantum circuits are represented as a set of horizontal lines representing qubits, with symbols or vertical lines denoting the quantum gates applied to them. The circuit’s execution proceeds in a left-to-right, chronological order. Figure 2 shows an example of a quantum circuit. Quantum gates that do not have qubits in common can be executed simultaneously. A time slice is a contiguous group of gates that do not share any qubits and can thus be executed 3
SQARL: A Size-Agnostic RL approach for Circuit Allocation in Distributed Quantum Architectures
function T deterministically updates the state based on the chosen action at , while R evaluates the constructed solution through rewards on each state transition rt = R(at , st ). Rewards are positive in most RL setups, whereas in cost optimization, they are typically set to the negative cost of actions. Thus, reward maximization is equivalent to cost minimization. The parameter γ is a discount factor applied to rewards in future states; in optimization problems, it is usually set to 1.
indicates that at time slice i, the qubit j is at core k. For the allocation to be valid, any pair of qubits participating in a gate within a given time slice St must be assigned to the same core, so that a physical interaction between them is available. We do not consider non-local communications during the execution of a time slice St ; thus, qubit relocation is restricted to transitions between successive slices. Another restriction is that any qubit allocation must respect each core’s capacity.
In combinatorial optimization, RL policies are often parameterized using an encoder–decoder architecture, in which a problem instance x is first mapped to an embedding space h = f (x), and the decoder defines the conditional action distribution πθ (at |st , h) used to construct the solution autoregressively (Berto et al., 2025).
Formally, we define the qubit allocation problem as follows: Definition 2.1 (Qubit Allocation Problem). Given a quantum circuit G over Q qubits, consisting of T time slices, and a hardware {c, F} over C cores, find a logical qubit to core per time slice assignment, that is, a matrix over the set of cores R ∈ {0, . . . , C − 1}T ×Q that minimizes the total inter-core communication cost X X F(R(i − 1, k), R(i, k)) (1)
The primary goal of this RL approach is to find the optimal parameters θ∗ for the policy π that maximizes the expected cumulative reward across all sampled problem instances. This objective is mathematically defined as: " " T −1 ## X ∗ t θ = arg max Ex∼P (x) Eπ(a|x) γ R(st , at ) (4)
0<i<T 0≤k<Q
subject to the following constraints:
θ
1. Qubits in the same gate are in the same core: ∀{qa , qb } ∈ St : R(t, a) = R(t, b).
From Definition 2.1, the qubit allocation problem admits C T Q possible assignments, which for a 100-qubit circuit with 10 slices on a 10-core architecture is about 101000 . Many of these solutions violate problem constraints and are therefore infeasible. Hence, RL autoregressive methods often enforce feasibility through action masking (Atom Computing). The state-dependent action set A(st ) ⫅ A is formed by all feasible actions at at state st .
(2)
2. Core capacities are respected: ∀St , j ∈ {0, . . . , C − 1} : |{k | 0 ≤ k < Q, R(t, k) = j}| ≤ cj .
t=0
(3)
2.3. Reinforcement Learning In recent years, many works have explored Reinforcement Learning for combinatorial problems formulated as sequential decision processes (Bello et al., 2017; Kool et al., 2019; Berto et al., 2025). Previous studies have also examined supervised (or imitation) learning for such problems (Drakulic et al., 2023; Luo et al., 2023). However, obtaining labeled optimal solutions is infeasible in practice, since exact methods do not scale to large instances. On the other hand, heuristic methods designed to alleviate computational complexity often yield lower-quality solutions than exact methods; a model trained with imitation learning would then learn suboptimal behaviour. Instead, RL learns heuristics directly from the solution cost; that is, the neural network parameters are updated based on the outcome of each trial. This enables exploration of the state space and yields models that generate higher-quality solutions.
3. Related Work Given that qubit allocation is a fundamental issue in distributed quantum systems, numerous works have proposed solutions with varying allocation costs. First approaches focused on applying well-known discrete optimization techniques to this new problem. Among these, we can find graph partitioning methods such as FGP-rOEE (Baker et al., 2020), quadratic optimization techniques such as QUBO (Bandic et al., 2023), and resource allocation approaches such as HQA (Escofet et al., 2023). Benchmarks of these nonlearning algorithms (Escofet et al., 2025) reveal that HQA consistently outperforms the other methods in terms of allocation cost, and is therefore the state-of-the-art. More recent works explore the application of learned allocation heuristics via RL. The state of the art in this domain is Russo’s algorithm from (Russo et al., 2024). It leverages graph neural networks for circuit codification, an attention mechanism for core assignment, and learning via practicing allocations. However, some components of the architecture make it non-flexible with respect to hardware topology:
The construction process for a given instance x can be formulated as a Markov Decision Process (MDP) described by a tuple (S, A, T , R, γ). The state st ∈ S represents the partial solution at time step t, and the action space A consists of all possible construction decisions. The transition 4
SQARL: A Size-Agnostic RL approach for Circuit Allocation in Distributed Quantum Architectures
before training, one selects a number of qubits and core configurations, and any change to either requires retraining the entire model from scratch. This is an important limitation when applying allocation algorithms to real quantum systems.
world applications. This section introduces SQARL (Scalable Qubit Allocation via Reinforcement Learning), a novel algorithm for qubit allocation in distributed architectures. First, in Sec. 4.1 we discuss the division of the circuit into time slices and the encoding of its information. Then, in Sec. 4.2, we cover the policy model itself, its inputs, and the transformer-based architecture. Following this, in Sec. 4.3, we present the two strategies we propose for circuit allocation — sequential and parallel allocation. Finally, in Sec. 4.4, we conclude with a description of the random circuit sampling method used for data generation during training, exploration techniques, and the training procedure. 4.1. Circuit Encoding The unsliced circuit G′ is received as an ordered list of twoqubit gates {qa , qb }, with qa ̸= qb . However, the procedure expects a time-sliced circuit G, formed by a list of T time slices St , t ∈ {0, . . . , T − 1}. Each time slice is a list of two-qubit gates {qa , qb }, without qubit repetitions within each slice. Thus, we need to group the consecutive gates from G′ that can be executed simultaneously.
Figure 4. Allocation cost benchmark between the non-learning and RL state-of-the-art methods. Hardware consists of 10 quantum cores, each with 10 qubits. The intercommunication cost between any pair of cores is one unit.
We use a simple, fast slicing method, shown in Alg. 1. Gates are aggregated into a slice until one of the qubits interacts for a second time. The gate containing that qubit forms a new time slice. The set U keeps track of which qubits have already been used in the current time slice.
No direct performance comparison in the literature exists between the non-learning and the RL state-of-the-art. We present such a benchmark in Fig. 4, where both algorithms optimize a series of well-known quantum circuits, as well as the average cost over 64 random circuits, generated as described in Sec. 4.4. The cost is reported as the number of inter-core communications per gate (excluding gates in the first time slice, as those would never require qubit relocation), to normalize values and facilitate easier comparison across circuits on the same scale. There is a consistently large gap in the allocation costs between the two methods across all circuits, further impeding the use of RL techniques in real-world scenarios.
Algorithm 1 Circuit Slicing input An unsliced circuit G′ . output Sliced circuit G. G ← [] St ← [] U ←∅ for Gate {qa , qb } in G′ do if qa ∈ U ∨ qb ∈ U then G.append(St ) St ← [] U ←∅ end if St .append({qa , qb }) U ← U ∪ {qa , qb } end for G.append(St ) return G
Despite the performance gap, we still consider RL a promising approach to qubit allocation. The learning paradigm enables exploration-based improvement: the model can achieve better results by practicing different allocations. In addition, RL has achieved state-of-the-art results in other domains, particularly in table games and robotics.
Some components of the algorithm do not use the list of gates and time slices directly; instead, they are encoded as a set of circuit features. These aim to capture the same information in a form digestible to the model. The first circuit feature is the circuit embedding, inspired by the lookahead weights from FGP-rOEE, proposed in (Baker et al., 2020). It contains, for each time slice, all qubit interactions from that slice until the end of the circuit. A time slice St can be encoded as an adjacency matrix At ∈ {0, 1}Q×Q , where Q
4. Methodology In this work, we adopt an RL approach. By addressing its greatest disadvantages, the large cost gap relative to state-of-the-art non-learning methods and the size-flexibility constraints, RL algorithms could have practical use in real5
SQARL: A Size-Agnostic RL approach for Circuit Allocation in Distributed Quantum Architectures
is the number of qubits in the circuit, as ( 1 if {qi , qj } ∈ St . At (i, j) = 0 otherwise
current slice, position [b, ci , qj , 2] is 1 if qubit qj is allocated to core ci in this time slice and 0 if not or it has not been allocated yet.
(5)
The fourth feature encodes the number of free spots in the ith core at this specific allocation step, ci . It is encoded as 1/(ci + 1). The fifth feature captures the total cost of moving qa and qb , the qubits allocated, to core ci , or just qa for single qubit allocations. Analogously to core capacities, it is encoded as 1/(f + 1), where f = F(c′a , ci ) + F(c′b , ci ), and c′a (c′b ) the core to which qa (qb ) was allocated in the previous time slice, or f = F(c′a , ci ) for single qubit allocations. Both feature encoding styles as 1/(x + 1) enable the mapping of the range [0, +∞) to the bounded space [1, 0).
The circuit embedding for slice t, Et , is a RQ×Q matrix defined as X 1 Ak (i, j). (6) Et (i, j) = k−t+1 2 t≤k<T
These embeddings capture all the information in the circuit (the original set of gates can be reconstructed up to floating-point precision of Et ). However, the exponentially decaying sum of adjacency matrices can hinder the model’s ability to capture relationships in distant qubit interactions. This motivates a second circuit feature, the next interaction matrices. These consist of a matrix RQ×Q for each circuit slice, defined as T −k Ak (i, j). t≤k<T T − t
Nt (i, j) = max
The sixth feature is qubit-to-core attraction, a concept first introduced in (Escofet et al., 2023). The attraction between qubit qj and core ck at the tth time slice is defined as X Jt (qi , ck ) · Et (qj , qi ), (8) attrt (qj , ck ) = 0≤i<Q
(7)
where Jt (qi , ck ) = 1 if qubit qi is in core ck at time slice t, 0 otherwise, and Et is the circuit embedding as defined in Eq. 6. The seventh and eighth features are the circuit embedding items, as defined in Eq. 6, at positions Et (qj , qa ) and Et (qj , qb ), respectively. If the allocation is for a single qubit, then Et (qj , qb ) is replaced with 0. Finally, the ninth and tenth features are the next interaction elements at positions Nt (qj , qa ) and Nt (qj , qb ). Analogously, the tenth feature is replaced with 0 in single-qubit allocations.
Nt (i, j) contains a linearly decaying value that indicates how close the next interaction of qubits qi and qj is — 1 if it happens in this slice and 0 if they no longer interact. 4.2. The Policy Model The policy model, πθ , is the neural network component of the algorithm. It is used during the allocation procedure to determine to which core a qubit is assigned. The allocation procedure is discussed in-depth in Sec. 4.3, but in essence, qubits are allocated in each time slice using the policy. If two qubits participate in a gate, they are allocated together; otherwise, they are allocated individually. The policy provides a probability distribution over the set of cores at each allocation step, and qubits are allocated to cores according to this distribution. Once the policy is trained, allocating qubits to high-probability cores yields lower-cost solutions.
To summarize, when running the policy to allocate qubits qa and qb , the feature tensor contains the following vector of elements at position [b, ci , qj ]: 1. 1 if qj ∈ {qa , qb }, 0 otherwise. 2. 1 if qj was allocated to core ci in the previous time slice, 0 otherwise. 3. 1 if qj is allocated to core ci in the current time slice, 0 otherwise.
The policy model must receive all necessary information to determine this probability distribution as effectively as possible. This information is condensed in a vector of features for each of the C hardware cores and Q qubit combinations, producing a tensor of shape [B, C, Q, h], where B is the batch size and h is the number of features, in our case ten.
4. 1/(ci + 1), with ci the number of free places in core i. 5. 1/(f + 1), with f the cost of moving qa and qb to core i. 6. Average attraction of qa and qb to core i, as defined in Equation 8.
Let qa and qb be the two qubits being allocated in a time step. The first feature is a one-hot encoding indicating which qubits are being allocated. The tensor element at position [b, ci , qj , 0] equals 1 if qj = qa or qj = qb , zero otherwise.
7. Et (qj , qa ). 8. Et (qj , qb ) or 0 for single-qubit allocations. 9. Nt (qj , qa ). 10. Nt (qj , qb ) or 0 for single-qubit allocations.
The second feature indicates whether qubit qj was allocated to core ci in the previous time slice: 1 if so, 0 if not. Similarly, the third feature provides information on current allocations. Considering all qubits allocated thus far in the
As noted previously, the features are analogous for single qubit allocations, considering the qubit qa alone. 6
SQARL: A Size-Agnostic RL approach for Circuit Allocation in Distributed Quantum Architectures
Figure 5. Policy model’s architecture.
The previous tensor contains features for all Q qubits. However, we can focus on the qubits being allocated in this step by selecting and extracting the relevant slices along the qubit dimension of the tensor, resulting in a new tensor of shape [B, C, 2, h], which contains feature vectors only for these qubits and all cores at this allocation step. For single-qubit allocations, the tensor corresponding to that qubit is concatenated with zeros in the qubit dimension to get the desired shape. We refer to the tensor over all qubits as the key tensor, and to the one over the qubits being allocated alone as the query tensor. These are fed into the architecture shown in Fig. 5. Both the query and the key are projected from a feature space of size h to another of size H, a hyperparameter of the model, using distinct linear projections. The two vectors of features corresponding to the two qubits being allocated are also combined into a single one. Then, the projected key features attend to one another through a transformer encoder block. Next, the query and key features are combined using a Multi Head Attention (MHA) block. In this step, the attention between the query and key tensors aggregates the information of all qubits in each core, producing a new tensor of features for the cores of shape [B, C, H]. Lastly, the core features are allowed to attend to one another via another transformer encoder block, and are finally converted into a scalar via a linear projection. The output of this model is a tensor of shape [B, C], corresponding to the logits for the probability distribution over the set of cores for each element in the batch.
qubits. This prevents configurations in which no core has enough space for two qubits that must be allocated together. Thus, the only illegal action that remains to be masked is assigning a qubit to a core withs no free spots left. Slices are allocated in order; however, qubits do not have an intrinsic order in which they must be allocated. Qubit allocation order could affect the algorithm’s performance — some qubits have stronger preferences for assignment than others. We propose two allocation orders: a sequential order, in which a heuristic determines which qubits are allocated first; and a parallel order, in which the policy itself determines allocation priority. In sequential allocation, the qubits are pre-ordered according to a heuristic that prioritizes more active or earlierinteracting qubits. Qubits that belong to some gate in the current time slice, allocated in pairs, are ordered according to their respective value in the circuit embedding tensor (i.e. Et (qa , qb ) for qubits {qa , qb } that form a gate in the tth time slice) in descending order. Free qubits, those that do not participate in any gate in the current time slice, are sorted in descending order according to the maximum element in the embedding tensor row: for qubit qa and the tth time slice, we have max0≤i<Q Et (qa , qi ). Fig. 6 shows an example in which this heuristic improves allocation cost. Qubits 0 and 1 have been allocated; if qubit 2 were to come next, a naive policy could place it in the first core, ignoring that qubit 3 would greatly benefit from this spot, as it interacts with qubit 0 in the next time slice. By allocating qubits that interact sooner first, these preferences are more easily taken into account: qubit 3 would be allocated before qubit 2.
4.3. The Allocation Procedure Our algorithm uses an autoregressive approach to circuit allocation. Slices are allocated sequentially, and within each slice we place one qubit at a time, or two if these form a gate. Not all qubit-to-core assignations are valid solutions, as these must respect the hardware restrictions mentioned in Sec. 2.2. The probabilities of actions that would lead to illegal states need to be masked to prevent such solutions. To minimize the number of situations requiring an intervention, we propose two conditions for the allocation order. First, qubits that form a gate are allocated together. This prevents them from being allocated to different cores. Second, qubits that belong to gates are allocated first, and then unpaired
Parallel allocation aligns more closely with RL: replacing human-made heuristics with learned policies. Thus, we decide the allocation order by using the policy itself. All qubits to be allocated in a given slice are fed to the policy in batch. The policy produces logits for all cores and qubits, which are then flattened into a single vector and fed to a softmax. This produces a probability distribution over all qubit-to-core assignments; sampling from this probability distribution means allocating first the qubits with the highest core preferences. When a qubit-core pair is selected, the given qubit is allocated to the core and taken out of the
7
SQARL: A Size-Agnostic RL approach for Circuit Allocation in Distributed Quantum Architectures
The first step is producing random circuits and hardware configurations. The hardware-sampling procedure randomly selects a number of cores C and a vector of core capacities c in a given range. The model is trained with a uniform interconnection cost matrix F, where moving a qubit between any two cores has a cost of one unit. This is not a requirement, and new versions of the model could be trained on randomly sampled patterns of inter-core communication costs. Analogously to hardware sampling, we sample from a uniform distribution over all possible circuits to achieve a circuit-agnostic policy. On each iteration, a number of qubits Q and time slices T is selected randomly. Then, gates {qa , qb }, a, b ∈ {0, . . . , Q − 1} ∧ a ̸= b, are sampled and fed to Alg. 1 until the desired number of time slices T is reached.
lq 0 lq 1 lq 2 lq 3 (a)
(b)
Figure 6. A scenario where the free qubits’ order of allocation influences final cost. (a) Partial allocation diagram for hardware with two cores, with three and one qubit respectively, before assigning a core to logical qubit 2, first time slice only. (b) Circuit used in the example.
A crucial point in any RL algorithm is that of balancing exploration and exploitation. This algorithm includes two exploration mechanisms. The first is a vector of random positive values that is added to the probabilities produced by the policy. Precisely, we have that
allocation pool. The process is repeated until no qubits remain to be allocated. Note, however, that paired qubits (those that belong to gates) are allocated first, and then free qubits. The process is executed twice per time slice — first exclusively for qubit pairs, then for free qubits. Fig. 7 shows an example of this process applied to the allocation of two pairs of qubits and two free qubits to a three-core hardware.
(a)
(b)
p′ = α · x + (1 − α) · p,
(9)
where p is the original vector of probabilities, x ∼ U(0, 1)n a vector of random values sampled from a uniform distribution, and α a parameter that controls the noise-to-signal ratio. This parameter decreases slightly with each iteration. The second exploration mechanism is that of sampling distributions. When executing SQARL at test time, the different distributions are not sampled; the highest-probability value is always selected, resulting in a greedy allocation. This is to minimize allocation cost and perform deterministic allocations. However, during training, it is beneficial to trade off some performance to obtain variety in the results. The variety in actions and costs is used to learn better allocation strategies.,
(c)
Figure 7. Qubit allocation process. (a) Paired qubits are allocated together and before free qubits. The policy is applied to pairs {q0 , q1 } and {q2 , q3 } and, after sampling, the pair {q2 , q3 } is assigned to the middle core. (b) After pair {q2 , q3 } is taken out of the allocation pool, the policy is run again on the remaining pairs. (c) Once there are no more paired qubits left, the same process is repeated with the free qubits.
Action sampling, unlike greedy selection, is also relevant for another component of the training algorithm — advantage calculation. Some implementations of the REINFORCE algorithm use a previous version of the algorithm as a baseline to measure advantage. However, in combinatorial optimization problems, where small differences in allocations can yield large differences in value, this can lead to high variance and less stable training. Thus, we propose an approach to advantage calculation borrowed from GRPO (Shao et al., 2024): a circuit is optimized several times, sampling actions from the policy’s probability distributions, and the individual advantage of each allocation is obtained relative to the rest. Specifically, the costs of all allocations are gathered into a vector Ac , which is then normalized as
As will be shown in Sec. 5, sequential allocation does best in some circuits, whereas parallel does better in others. Thus, we propose allocating each circuit using both methods and selecting the allocation that yields a lower overall cost. 4.4. Training Procedure The policy is trained by practicing allocations: a random circuit and hardware are generated, the circuit is optimized several times, and actions that lead to lower allocation costs are incentivized, whereas those that result in higher costs are penalized. This process is repeated until the policy converges.
Ãc =
Ac − µ , σ
(10)
where µ and σ are the mean and standard deviation of Ac . 8
SQARL: A Size-Agnostic RL approach for Circuit Allocation in Distributed Quantum Architectures
This relative cost is then used in a training setup similar to the REINFORCE algorithm (Williams, 1992).
with 2 to 8 cores and circuits of up to 20 qubits and 4 to 16 time slices. The size of the circuits and hardware used for training is relatively small; later, we will evaluate the policy on quantum circuits with 100 qubits and hundreds of time slices on hardware with 5 and 10 cores. These samples fall outside the problem sizes the algorithm has seen during training, thereby testing its ability to scale to problem sizes beyond those encountered during training. The group used in the GRPO advantage calculation is of size 32. The invalid move penalty (β in Alg. 2) is set to 0.3 for the first 16000 iterations and to 0.5 for the last 12100. The initial noise injected into the policy’s probabilities (α in Eq. 9) is set to 0.2 with a decay factor of 0.999. The noise ratio α was raised to 0.05 at iteration 16000. The policy was trained with the sequential allocation policy, as it is more stable than parallel allocation for training. The training lasted 124 total hours of computing.
Another key aspect of RL is how to deal with illegal actions. In Sec. 4.3, we presented an allocation sequence that minimizes the amount of masking required. However, experiments show that when masking is used during training, the model is unable to learn which actions are legal. In the late stages of training, most of the probability mass is assigned to illegal actions. Thus, it is important to teach the model which actions are possible, so that the probability of performing illegal actions is minimized and training does not diverge towards attempting illegal actions. During training, the policy is allowed to perform illegal actions (allocating qubits to cores that lack empty slots, even if this leads to an invalid solution), but it is penalized. Illegal actions are always masked outside training, despite these having very little probability mass once the policy is trained.
Validation runs are executed every 25 iterations. The average normalized cost during training is shown in Fig. 8. The gray dotted line indicates the point at which noise was added to the policy to incentivize further exploration.
The training procedure is shown in Alg. 2. The parameter β controls the impact of the illegal action loss. Algorithm 2 Reinforcement Learning Training input Policy model πθ with parameters θ while not converged do {c, F} ← randomHardwareSampler() G ← randomCircuitSampler() {R0 , R1 , ..., RN −1 } ← optimizeNTimes(G, c, F, θ) {Ãc0 , ...} ← getNormalizedCostVector({R0 , ...}) for n ∈ {0, . . . , N − 1} do for allocation step si and action ai in Rn do if legal(ai , si ) then loss ← loss + (1 − β)An log(πθ (ai |si )) else loss ← loss + β log(πθ (ai |si )) end if end for end for θ ← updateParameters(loss,θ) end while
5. Experiments and Results
Figure 8. Normalized allocation cost on the validation set of circuits during training.
This section begins by reviewing the selected hyperparameters for the model and the training procedure. Next, the trained policy is tested on a selection of circuits for sequential and parallel allocation. Finally, we present a benchmark of the performance with the state of the art. The code was executed in BSC’s Marenostrum 5, with an Intel Xeon Platinum 8460Y+ CPU and a NVIDIA H100 GPU.
In Sec. 4.4, we noted that during training, the policy is allowed to take illegal actions but is penalized for doing so. If the invalid movement penalty is not set high enough, the policy could collapse into taking mostly illegal actions, as the performance gains from them outweigh the loss penalty. Fig. 9 shows the ratio of valid moves as training progresses. The policy rapidly learns which actions are illegal and takes mostly legal actions. Furthermore, because of probabilistic action sampling during training, illegal actions may still be taken even when their probability is very small. All illegal actions are masked when the model is not being trained.
Several hyperparameters need to be fixed before training the architecture described in Sec. 4.2. Specifically, we set the hidden embedding size H to 64, the number of layers per transformer encoder block to 2, and the number of heads to 2, yielding a total of 119617 trainable parameters. We trained the model for 28100 iterations on quantum hardware
For benchmarking, we have sampled 64 random circuits, 9
SQARL: A Size-Agnostic RL approach for Circuit Allocation in Distributed Quantum Architectures
the execution time is the sum of both. Circuit Name Cuccaro Adder Deutsch Jozsa Draper Adder Graph State QFT QNN Quantum Volume Random Avg
50 Qubits Seq. Par. 52 50 16 16 332 348 418 476 323 424 671 848 1000 874 195 163
100 Qubits Seq. Par. 74 76 36 38 841 919 1990 2079 1586 2506 3349 4182 4680 4241 348 273
Table 1. Comparison of the allocation cost for the two versions of the algorithm, sequential and parallel mode, in a selection of circuits with 50 and 100 qubits.
Tab. 2 presents a cost-performance benchmark of our algorithm against the RL state of the art, Russo’s algorithm, and the overall state of the art, HQA.
Figure 9. Ratio of valid moves selected by the policy during training. Data shown with a Gaussian filter (σ = 10) to smooth noise.
(a) Inter-core Comms. Improvement Russo HQA SQARL vs Russo vs HQA Cuccaro Adder 143 50 50 +65.03% +0.00% Deutsch Jozsa 48 16 16 +66.67% +0.00% Draper Adder 691 347 332 +51.95% +4.32% Graph State 546 341 418 +23.44% -22.58% QFT 1146 309 323 +71.82% -4.53% QNN 573 546 671 -17.10% -22.89% Quantum Volume 1204 841 874 +27.41% -3.92% Random Avg 208 208 163 +21.88% +21.79%
generated as described in Sec. 4.4, with 50 time slices (whose costs are averaged under the category “Random Avg”) and 7 relevant circuits in the field of quantum computing. The Cuccaro Adder is a popular circuit for adding quantum registers. It is highly efficient and requires few auxiliary qubits. Deutsch-Jozsa is a quantum algorithm of great historical importance, as it was the first to show exponential quantum advantage over non-quantum algorithms. It is used to tell whether a function is constant or balanced. The Draper Adder is a circuit that performs addition in the phase basis using the quantum Fourier transform. A Graph State is a highly entangled state represented by a graph, in which the vertices correspond to qubits and the edges represent entanglement. It is the foundation of measurement-based quantum computing. The Quantum Fourier Transform (QFT) is the quantum analogue of the discrete Fourier Transform and is used in some of the most famous quantum algorithms. Quantum Neural Networks (QNN) are variational circuits with tunable parameters. It is the primary candidate for near-term applications of quantum computers. Quantum Volume is a benchmark protocol that runs random square circuits of increasing size on a quantum computer. It is the industry standard for measuring hardware performance.
(b) Inter-core Comms. Improvement Russo HQA SQARL vs Russo vs HQA Cuccaro Adder 517 110 74 +85.69% +32.73% Deutsch Jozsa 363 36 36 +90.08% +0.00% Draper Adder 2631 955 841 +68.03% +11.94% Graph State 2892 1485 1990 +31.19% -34.01% QFT 4887 1149 1586 +67.55% -38.03% QNN 3549 2116 3349 +5.64% -58.27% Quantum Volume 5289 3908 4241 +19.81% -8.52% Random Avg 500 361 273 +45.46% +24.43% Table 2. Performance comparison with SOTA. Results show the normalized allocation cost and cost improvement of our method with RL state of the art (Russo) and with the overall state of the art (HQA) for circuits with (a) 50 qubits and (b) 100 qubits.
We first compare the sequential and parallel allocation modes in Tab. 1 to determine which performs best. Results are shown for two groups of circuits: one version with 50 qubits, optimized on a quantum computer with 5 cores, each with 10 qubits, and the other with 100 qubits, optimized on a quantum computer with 10 cores, each with 10 qubits. No allocation mode consistently outperforms the other; we propose optimizing each circuit with both modes and selecting the solution with the lower cost in an ensemble-like manner. Thus, the cost of SQARL is the minimum of the two, and
6. Discussion Tab. 1 indicates that no allocation mode consistently outperforms the other; sequential seems to perform better on structured circuits such as the Draper Adder, QFT, or QNN, whereas parallel outperforms on less structured circuits, such as the set of random circuits. It is uncertain why the parallel allocation mode cannot outperform sequential allocation across all circuits; one potential reason is that training the model exclusively in sequential mode hinders parallel 10
SQARL: A Size-Agnostic RL approach for Circuit Allocation in Distributed Quantum Architectures
allocation in some scenarios.
from Sec. 5 indicate that future iterations of SQARL (or other RL techniques) could push the allocation cost below that of HQA for the remaining quantum circuits.
Regarding the inter-core communication cost benchmark shown in Tab. 2, SQARL consistently outperforms the previous RL state-of-the-art (Russo) across almost all benchmarks. The cost reduction is significant, particularly for structured circuits such as the Cuccaro Adder, DeutschJozsa, Draper Adder, and QFT. We also match or outperform HQA for half of the circuit types, particularly for random circuits. For medium-sized circuits (50 qubits), our method remains notably close to HQA. These results show that our RL approach can learn policies that approach the efficiency of hand-crafted heuristics. In random circuits, SQARL outperforms the overall state of the art with a cost reduction of 21.90% for 50 qubits and 24.48% for 100 qubits. Regarding scalability, as the problem size doubles from 50 to 100 qubits and the core count doubles, the performance gap between our method and Russo’s widens. This is notable, as the maximum number of qubits during training is 20, which highlights that, in addition to being flexible with respect to problem size, our method can maintain the best performance, even for circuit dimensions it has never encountered before. QNN and Graph State appear to be the most challenging benchmarks.
Future work could focus on improved circuit generation for training. The performance improvement of SQARL is most notable in random circuits, the type of circuit with which the policy was trained. Although it represents a uniform distribution over all possible circuits, the results indicate that real circuits likely lie in a specific subspace rarely sampled during training. Another aspect is that of accelerating the allocation process. Most qubits do not change core from one slice to the other. A system that detects which qubits require reallocation and executes the policy exclusively on those qubits could substantially reduce execution times. Finally, circuit encoding relied on a set of hand-crafted features. Methods like GNNs or topological ML could be leveraged to extract sets of learned circuit features that improve upon those proposed in Sec. 4.2.
Code availability The complete implementation is publicly available at https://github.com/Vicara12/SQARL, along with the benchmarking circuits in JSON format (Carballo, 2026).
The average advantage of 21.79% and 24.43% over the set of 64 random circuits shows a solid performance gain in these types of circuits. Although they do not form a specific category of circuits (they represent samples from the space of all possible circuits), the vast majority are much less structured than the quantum circuits in the literature. The latter type of circuits frequently exhibits specific gate patterns and sequence repetition that seldom appear in the former. Thus, RL algorithms rarely have a chance to train on these types of circuits, which dominate real-world applications of quantum computing. If more “realistic” circuit generation techniques were available, RL methods could potentially outperform non-learning ones in the other types of circuits as well.
CRediT authorship contribution statement Vı́ctor Carballo: conceptualization, software, formal analysis, validation, investigation, visualization, methodology, writing - original draft, writing - review & editing. Júlia López-Closa: conceptualization, investigation, methodology, writing - original draft, writing - review & editing. Mario Martin: conceptualization, resources, supervision, funding acquisition, project administration, writing - review & editing.
Declaration of competing interest 7. Conclusion and Future Work
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
RL methods for qubit allocation lagged behind the overall state of the art in terms of allocation cost and flexibility to the number of qubits and core interconnection topology, as discussed in Sec. 3. However, the results from Sec. 5 show consistent improvement over the RL state of the art in allocation cost and highly competitive overall performance, with inter-core communication costs that improved or matched HQA’s on half of the benchmark circuits. Notably, in unstructured and random circuits, there is up to 25% improvement in allocation cost with respect to the best method and up to 46% with the previous best RL algorithm. In addition, the policy is hardware-agnostic and does not require retraining if the quantum hardware changes. The results
Acknowledgments This work has been funded by the Ministry of Economic Affairs and Digital Transformation of the Spanish Government through the QUANTUM ENIA project call – Quantum Spain project, and by the European Union through the Recovery, Transformation and Resilience Plan – NextGenerationEU within the framework of the Digital Spain 2026 Agenda.
11
SQARL: A Size-Agnostic RL approach for Circuit Allocation in Distributed Quantum Architectures
References
Escofet, P., Ovide, A., Bandic, M., Prielinger, L., van Someren, H., Feld, S., Alarcon, E., Abadal, S., and Almudever, C. Revisiting the mapping of quantum circuits: Entering the multi-core era. ACM Transactions on Quantum Computing, 6(1), January 2025. https://doi.org/10.1145/3655029.
Atom Computing. AC1000. https:// atom-computing.com/ac1000/. Atom Computing, accessed 2026-02-12. Baker, J. M., Duckering, C., Hoover, A., and Chong, F. T. Time-sliced quantum circuit partitioning for modular architectures. In In proceedings of the 17th ACM International Conference on Computing Frontiers, pp. 98–107. ACM, 2020. https://doi.org/10.1145/3387902.3392617.
Gambetta, J. M. The hardware and software for the era of quantum utility is here. https://www.ibm.com/ quantum/blog/quantum-utility, December 2023. IBM Quantum blog, accessed 2026-02-12.
Bandic, M., Prielinger, L., Nüßlein, J., Ovide, A., Rodrigo, S., Abadal, S., Van Someren, H., Vardoyan, G., Alarcon, E., Almudever, C. G., et al. Mapping quantum circuits to modular architectures with QUBO. In 2023 IEEE International Conference on Quantum Computing and Engineering (QCE), volume 1, pp. 790–801. IEEE, 2023. https://doi.org/10.1109/QCE57702.2023.00094.
Jnane, H., Undseth, B., Cai, Z., Benjamin, S. C., and Koczor, B. Multicore quantum computing. Phys. Rev. Appl., 18:044064, October 2022. https://doi.org/10.1103/PhysRevApplied.18.044064. Kool, W., van Hoof, H., and Welling, M. Attention, learn to solve routing problems! In Proceedings of the 7th International Conference on Learning Representations, 2019. https://openreview.net/forum?id= ByxBFsRqYm.
Bello, I., Pham, H., Le, Q. V., Norouzi, M., and Bengio, S. Neural combinatorial optimization with reinforcement learning. In Proceedings of the 5th International Conference on Learning Representations, 2017. https: //openreview.net/forum?id=rJY3vK9eg.
Luo, F., Lin, X., Liu, F., Zhang, Q., and Wang, Z. Neural combinatorial optimization with heavy decoder: Toward large scale generalization. In Proceedings of the 37th International Conference on Neural Information Processing Systems, Red Hook, NY, USA, 2023. Curran Associates Inc. https://openreview.net/forum? id=RBI4oAbdpm.
Berto, F., Hua, C., Park, J., Luttmann, L., Ma, Y., Bu, F., Wang, J., Ye, H., Kim, M., Choi, S., Zepeda, N. G., Hottung, A., Zhou, J., Bi, J., Hu, Y., Liu, F., Kim, H., Son, J., Kim, H., Angioni, D., Kool, W., Cao, Z., Zhang, Q., Kim, J., Zhang, J., Shin, K., Wu, C., Ahn, S., Song, G., Kwon, C., Tierney, K., Xie, L., and Park, J. RL4CO: An extensive reinforcement learning for combinatorial optimization benchmark. In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining, volume 2, pp. 5278–5289. ACM, 2025. https://doi.org/10.1145/3711896.3737433.
Mavroeidis, V., Vishi, K., Zych, M. D., and Jøsang, A. The impact of quantum computing on present cryptography. International Journal of Advanced Computer Science and Applications, 9(3), 2018. https://doi.org/10.14569/IJACSA.2018.090354. Mohseni, M., Scherer, A., Johnson, K. G., Wertheim, O., Otten, M., Aadit, N. A., Alexeev, Y., Bresniker, K. M., Camsari, K. Y., Chapman, B., Chatterjee, S., Dagnew, G. A., Esposito, A., Fahim, F., Fiorentino, M., Gajjar, A., Khalid, A., Kong, X., Kulchytskyy, B., Kyoseva, E., Li, R., Lott, P. A., Markov, I. L., McDermott, R. F., Pedretti, G., Rao, P., Rieffel, E., Silva, A., Sorebo, J., Spentzouris, P., Steiner, Z., Torosov, B., Venturelli, D., Visser, R. J., Webb, Z., Zhan, X., Cohen, Y., Ronagh, P., Ho, A., Beausoleil, R. G., and Martinis, J. M. How to build a quantum supercomputer: Scaling from hundreds to millions of qubits. arXiv, 2025. https://doi.org/10.48550/arXiv.2411.10406.
Botea, A., Kishimoto, A., and Marinescu, R. On the complexity of quantum circuit compilation. In Proceedings of the International Symposium on Combinatorial Search. AAAI, 2021. https://doi.org/10.1609/socs.v9i1.18463. Carballo, V. SQARL. https://github.com/ Vicara12/SQARL, March 2026. [software]. Drakulic, D., Michel, S., Mai, F., Sors, A., and Andreoli, J.-M. BQ-NCO: Bisimulation quotienting for efficient neural combinatorial optimization. In Proceedings of the 37th International Conference on Neural Information Processing Systems, Red Hook, NY, USA, 2023. Curran Associates Inc.
Orobator, E. T., Njei, T. T., and Kanu, I. Applications of quantum computing in drug development. Journal of Pharma Insights and Research, 3(3):242–252, 2025. https://doi.org/10.69613/m089dd39.
Escofet, P., Ovide, A., Almudever, C. G., Alarcón, E., and Abadal, S. Hungarian qubit assignment for optimized mapping of quantum circuits on multi-core architectures. IEEE Computer Architecture Letters, 22(2):161– 164, 2023. https://doi.org/10.1109/LCA.2023.3318857.
Pastor, A., Escofet, P., Ben Rached, S., Alarcón, E., Barlet-Ros, P., and Abadal, S. Circuit partitioning 12
SQARL: A Size-Agnostic RL approach for Circuit Allocation in Distributed Quantum Architectures
for multi-core quantum architectures with deep reinforcement learning. In IEEE International Symposium on Circuits and Systems, pp. 1–5. IEEE, 2024. https://doi.org/10.1109/ISCAS58744.2024.10557956. Preskill, J. Quantum computing in the NISQ era and beyond. Quantum, 2:79, August 2018. ISSN 2521-327X. https://doi.org/10.22331/q-2018-08-06-79. Rodrigo, S., Abadal, S., Almudéver, C. G., and Alarcón, E. Modelling short-range quantum teleportation for scalable multi-core quantum computing architectures. In Proceedings of the Eight Annual ACM International Conference on Nanoscale Computing and Communication, New York, NY, USA, 2021a. ACM. https://doi.org/10.1145/3477206.3477461. Rodrigo, S., Bandic, M., Abadal, S., van Someren, H., Alarcón, E., and Almudéver, C. G. Scaling of multicore quantum architectures: a communications-aware structured gap analysis. In Proceedings of the 18th ACM International Conference on Computing Frontiers, pp. 144–151, New York, NY, USA, 2021b. ACM. https://doi.org/10.1145/3457388.3458674. Russo, E., Palesi, M., Patti, D., Ascia, G., and Catania, V. Attention-based deep reinforcement learning for qubit allocation in modular quantum architectures. arXiv, 2024. https://doi.org/10.48550/arXiv.2406.11452. Shao, Z., Wang, P., Zhu, Q., Xu, R., Song, J., Bi, X., Zhang, H., Zhang, M., Li, Y., Wu, Y., et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv, 2024. https://doi.org/10.48550/arXiv.2402.03300. Shor, P. W. Polynomial-time algorithms for prime factorization and discrete logarithms on a quantum computer. SIAM review, 41(2):303–332, 1999. https://doi.org/10.1137/S0097539795293172. Siraichi, M. Y., Santos, V. F. d., Collange, C., and Pereira, F. M. Q. Qubit allocation. In Proceedings of the 2018 International Symposium on Code Generation and Optimization, pp. 113–125, New York, NY, USA, 2018. ACM. ISBN 9781450356176. https://doi.org/10.1145/3168822. Vandersypen, L. M. K., Bluhm, H., Clarke, J. S., Dzurak, A. S., Ishihara, R., Morello, A., Reilly, D. J., Schreiber, L. R., and Veldhorst, M. Interfacing spin qubits in quantum dots and donors—hot, dense, and coherent. npj Quantum Information, 3(1):34, 2017. https://doi.org/10.1038/s41534-017-0038-y. Williams, R. J. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine learning, 8(3):229–256, 1992. https://doi.org/10.1007/BF0099269. 13