Conceptio › Archive › arXiv CS

arXiv CSopen access

Equivariant Reinforcement Learning for Clifford Quantum Circuit Synthesis

2026 · arxiv_cs

arXiv CS · Papers · License: Open Access · 2026

Open Source ↗Direct PDF ↓

neural-networks

machine learning, deep learning, neural networks

Equivariant Reinforcement Learning for Clifford Quantum Circuit Synthesis

Richie Yeung1,2

Aleks Kissinger1

Rob Cornish3

arXiv:2605.10910v1 [quant-ph] 11 May 2026

Department of Computer Science, University of Oxford 2 Quantinuum 3 College of Computing and Data Science, Nanyang Technological University, Singapore

Abstract We consider the problem of synthesizing Clifford quantum circuits for devices with all-to-all qubit connectivity. We approach this task as a reinforcement learning problem in which an agent learns to discover a sequence of elementary Clifford gates that reduces a given symplectic matrix representation of a Clifford circuit to the identity. This formulation permits a simple learning curriculum based on random walks from the identity. We introduce a novel neural network architecture that is equivariant to qubit relabelings of the symplectic matrix representation, and which is size-agnostic, allowing a single learned policy to be applied across different qubit counts without circuit splicing or network reparameterization. On six-qubit Clifford circuits, the largest regime for which optimal references are available, our agent finds circuits within one two-qubit gate of optimality in milliseconds per instance, and finds optimal circuits in 99.2% of instances within seconds per instance. After continued training on ten-qubit instances, the agent scales to unseen Clifford tableaus with up to thirty qubits, including targets generated from circuits with over a thousand Clifford gates, where it achieves lower average two-qubit gate counts than Qiskit’s Aaronson-Gottesman and greedy Clifford synthesizers.

Introduction

Quantum programs are executed as circuits: sequences of elementary gates acting on qubits. For any given high-level operation of interest, there are usually many different circuits that can implement it. A quantum compiler finds a circuit that does so while optimizing for some cost function, typically related to the error rates of the gates used in the circuit. A general quantum operation involving n qubits is described by a 2n × 2n complex matrix, and so finding an optimal circuit for a given operation is intractable in general. As such, practitioners have focussed on important special cases of quantum circuits that admit more concise representations. In this work, we consider the task of synthesizing Clifford circuits, a general and important class of quantum circuits which can be represented tractably by 2n × 2n binary matrices known as stabilizer tableaus [1]. Figure 1 shows a four-qubit Clifford circuit together with its corresponding tableau representation. The synthesis task is to implement a target Clifford operation as an explicit circuit using as few two-qubit (or entangling) gates as possible, since these are typically much more expensive and error-prone than single-qubit gates [4, 29]. Existing Clifford synthesis methods sit at two ends of a tradeoff between runtime and solution quality. At one end, there exist polynomial-time algorithms such as Aaronson–Gottesman [1], which scale efficiently but often produce many more entangling gates than necessary. At the other end, stronger optimization and exact-synthesis methods produce much shorter circuits, but at heavy computational cost. For example, template-based optimization can take hours per circuit [8], while the optimal six-qubit database of [10] required 2.1 TB of storage and over 300,000 CPU-hours to generate. 1 Correspondence to [email protected].

Preprint.

H H H

S H

H S

H H

Figure 1: A four-qubit Clifford circuit with 5 CZ gates and its corresponding stabilizer tableau, shown as a 2n × 2n binary matrix with black entries for 1 and white entries for 0. Appendix Fig. A3 shows how successive gate applications locally update this tableau until it reaches the identity. In this work, we pursue a middle ground between these two extremes, by learning a reusable neural heuristic for Clifford synthesis that is much lighter than exact or search-heavy optimization, but achieves substantially better entangling-gate counts than polynomial-time baselines. Our approach also generalises to out-of-distribution targets larger and harder than those seen during training. We approach this task via reinforcement learning, combining curriculum learning together with a novel neural network architecture that respects the symmetries of the problem. For the six-qubit benchmark of Bravyi et al. [8], which is currently the largest regime where exact references are available, our policy solves every instance, reaches a maximum gap of one CZ gate across the full suite in 21 seconds, and with extended search matches the exact optimum on 995/1003 circuits (99.2%) in three hours. In comparison, the prior state-of-the-art method recovers 982/1003 optima (97.9%) after 217 hours; across the full reported time-limit sweep, the same 21 unrecovered circuits consume 576 hours without reaching optimum [8, 9]. Our neural network architecture is size-agnostic, which allows its reuse across different qubit counts without circuit splicing or network reparameterization. We exploit this to go beyond the exact regime of [8]. In particular, after training on six-qubit instances, we continue training the same policy on ten-qubit instances. The resulting model is able to synthesize Clifford circuits up to 30 qubits while using fewer CZ gates than Qiskit’s standard Clifford synthesizers, Aaronson–Gottesman and the Bravyi et al. greedy method [1, 24, 23]. To our knowledge, our proposal is the first reinforcement learning method for synthesis of fully connected Clifford circuits, and the first learned policy to produce near-optimal synthesis on six-qubit circuits and moreover transfer to much larger targets. Our paper aims to be accessible to machine learning researchers with no prior background in quantum computing. In what follows, Section 2 reviews Clifford synthesis from first principles; Section 3 covers related work; Section 4 describes our reinforcement-learning formulation and equivariant architecture; Sections 5 cover our experiments, and Section 6 discusses limitations.

Background on Clifford Synthesis

In full generality, a quantum computation involving n qubits requires a 2n × 2n unitary matrix to represent. This becomes intractably large very quickly as n grows. Accordingly, practitioners have developed more concise representations for important special cases of quantum circuits. A key example is Clifford circuits [21, 1]. By the Gottesman-Knill theorem [21], these correspond to stabilizer tableaus [1], whose essential phase-free content can be described succinctly as a 2n × 2n binary symplectic matrix.1 A 2n × 2n binary matrix M is symplectic if it satisfies 0 In T M ΩM = Ω, where Ω = . In 0 Here multiplication is meant over the field F2 (i.e. integers modulo 2), and In denotes the n × n identity matrix. Below we denote the set of symplectic 2n × 2n binary matrices as Sp(2n, F2 ). At a high level, the (i, j)-component of each of the four n × n quadrants of M encodes a certain interaction between the i-th and j-th qubits of the corresponding Clifford circuit (see [1] for details). It is standard to show that Sp(2n, F2 ) forms a group under matrix multiplication. Moreover, its group multiplication structure respects the composition of Clifford circuits. In other words, the Clifford 1 The precise correspondence is described in Appendix D.

circuit obtained by “plugging in” the output of one Clifford circuit into another corresponds to the symplectic matrix obtained by multiplying the tableaus of those two circuits. The symplectic group admits a collection of generators G ⊆ Sp(2n, F2 ), which we denote below as G := {Hi : 1 ≤ i ≤ n} ∪ {Si : 1 ≤ i ≤ n} ∪ {CZi,j : 1 ≤ i < j ≤ n} . (1) | {z } | {z } Single-qubit gates

Two-qubit gates

In other words, every Clifford circuit corresponds to a product of these generators, and every product of these generators gives a valid Clifford circuit. Informally, each index i and j refers to a qubit; each Hi is obtained by applying a certain column swap to the identity matrix; and each Si and CZi,j are obtained by applying certain column additions to the identity matrix (see Figure 2 for an illustration). Exact definitions of these generators are given in Appendix D. In physical terms, the generators Hi , Si , and CZi,j correspond to the native instruction set of a quantum computer. In order to execute a Clifford circuit on hardware, it must therefore be decomposed into a sequence of these generators. By using the symplectic matrix representation of Clifford circuits, the basic task of Clifford synthesis therefore becomes as follows: Given Mtarget ∈ Sp(2n, F2 ), find G1 , . . . , Gk ∈ G such that Mtarget = G1 · · · Gk . (2) In general, there are many different sequences Hi Si CZi, j G1 · · · Gk of different lengths that produce the X X X Z Z Z X X X Z Z Z X X X Z Z Z X X X same overall tableau Mtarget . As such, for practiX X X cal purposes, it is desirable to solve (2) in a way X X X Z Z Z that is in some sense efficient. In particular, it is Z Z Z Z Z Z especially useful to find circuits with fewer twoZi Zi Xj X Z Z Z X i i i i i qubit gates CZi,j , as these are typically far more Zj Zj Xi error-prone than the single-qubit gates Hi and Si . In this way, Clifford synthesis becomes a Figure 2: Symplectic generator matrices Hi , Si , natural search problem, and therefore amenable and CZi,j . Right-multiplying any symplectic to techniques from reinforcement learning, as matrix by one of these generators applies the highlighted column operations, leaving the other we explore in this paper. columns untouched. k

Related Work

Why Clifford synthesis? Clifford operations are a core part of almost every quantum algorithm. A classical result shows that every quantum computation can be expressed up to arbitrary precision using only Clifford gates (H, S and CZ) and T gates [6, 37], an additional kind of gate for which two consecutive applications are equivalent to an S gate. In the context of fault-tolerant quantum computation, the implementation of the T gate requires a costly procedure known as magic state distillation [7, 28]. Recently work on magic state cultivation [20] reduces this overhead to roughly match the overhead of a fault-tolerant two-qubit gate such as the CZ. It is therefore practically relevant to reduce CZ gate counts in Clifford circuits as we do here. Non-neural synthesis. Most prior work on Clifford synthesis has focused on traditional, non-neural methods. These typically involve a sharp trade-off between solution quality and runtime. At one end of the spectrum, a variety of efficient polynomial-time synthesis procedures have been proposed [1, 8], which run quickly but often produce circuits far from the optimum. As an example, Bravyi et al. [8] demonstrate that the widely-used Aaronson–Gottesman algorithm produces Clifford circuits with up to 8 times more CZ gates when compared to the output of their peephole optimization procedure, although this procedure can use up to 36 hours per circuit. On the other hand, various search procedures have also been considered, which can achieve near-optimal synthesis but at a much higher computational cost, which limits their applicability to smaller circuits. For example, while SAT-based methods can produce certifiably optimal solutions, they have only been applied successfully on 5 qubit Clifford circuits [33, 38] and with partial success on 6–7 qubit instances [38] using a 3-hour timeout. A∗ search has also been used to achieve optimal two-qubit counts for random 5 qubit Clifford tableaus but is outperformed by a greedy heuristic beyond 16 qubits [46], both in terms of runtime and two-qubit gate count. Our work targets the regime between these two extremes: we learn a reusable heuristic that is much lighter than exact or search-heavy optimization, but achieves substantially better entangling-gate counts than polynomial-time baselines. 3

Neural methods. Reinforcement learning has been applied to various related problems in quantum compilation. AlphaTensor [17] was adapted to T -count optimization, improving state-of-the-art results on quantum arithmetic circuits [35]. Most closely related to our setting, Kremer et al. [27] apply reinforcement learning to the problem of constrained Clifford synthesis, a setting where entangling gates can only be applied to nearest-neighbor qubits. In contrast, we consider the fully unconstrained setting with all-to-all qubit connectivity. This gives an O(n2 ) action space compared to the constrained setting of [27], whose action space is only O(n). The general Clifford synthesis task also admits stronger polynomial-time, heuristic, and exact baselines than the constrained setting, making it a more challenging and informative benchmark for learning-based methods. Beyond these examples, reinforcement learning has also been applied to various tableau synthesis problems, including CNOT synthesis [42, 27, 12], stabilizer state preparation [47, 14], and Paulinetwork synthesis [15], all of which can be viewed as special cases of our setting with additional constraints on the target Clifford tableau. Closely related work also applies learning and reinforcement learning to rewriting graphical representations such as ZX diagrams, which encode Clifford and nonClifford circuit structure [11, 32, 34, 30]. However, current graph-based ZX simplification techniques [16, 26] mostly rewrite Clifford structure around fixed non-Clifford gates [40, 43]; reducing T count requires separate optimization techniques [3, 22, 13, 35]. Compared to these related tasks, our work is the first to demonstrate robust generalization across unseen circuits of different sizes. In contrast, Kremer et al. [27] train 31 models on 31 device architectures of up to 11 qubits for their task. Since our method improves synthesis directly in the unrouted Clifford setting, the same symmetry-aware policy ideas may also be useful for these neighboring stabilizer-structured compilation problems.

Method

We approach the problem (2) via reinforcement learning rather than supervised learning. This is due to the combinatorial nature of the problem, which means exact optimal supervision is only available up to the six-qubit regime (i.e. the 1003 exact references of [8, 10]), whereas we would like to scale beyond this to larger circuit sizes. 4.1

Reverse-reduction formulation

We begin by reformulating (2) in a way that is better suited to reinforcement learning. By a standard result, the generators (1) are involutive, so we have G−1 = G for each G ∈ G. This allows us to write (2) equivalently as follows: Given Mtarget ∈ Sp(2n, F2 ), find G1 , . . . , Gk ∈ G such that Mtarget G1 · · · Gk = I2n . Given a solution to this latter problem, involutiveness then implies that Mtarget = Gk · · · G1 (i.e. the reversed sequence) solves the original problem. Figure 1 gives an example for a four-qubit tableau, and Appendix Fig. A3 shows the full reduction of that tableau to the identity. This “reverse-reduction” viewpoint is well suited to reinforcement learning because it allows us to define each episode with the same goal state I2n , rather than having this be a general tableau that changes across episodes. As we discuss in Section 4.5, it also provides a natural curriculum, where the initial state of each episode is obtained via a random walk away from the identity, and the walk length controls the overall difficulty of that episode. 4.2

Reinforcement learning setup

We formulate Clifford synthesis as a deterministic Markov decision process (S, A, T , R), where S is the state space, A is the action space, T : S × A → S is the transition function, and R : S × A → R is the reward function. To simplify notation, we formulate everything with respect to a fixed qubit count that we denote by n, although we emphasize that the policy we will train is size-agnostic and can be applied across different qubit counts without reparameterization. We take the state space to be the set of binary symplectic matrices of size 2n×2n, i.e. S := Sp(2n, F2 ). The action set is then defined as A := G, where G is the set of generator matrices from (1). There are n(n+3) n actions in total. The transition function is 2 two-qubit actions and hence |A| = 2 T (M, G) := M G. 4

so each step applies one Clifford generator to the current tableau by right multiplication. In our implementation, an episode terminates whenever the identity matrix I2n is reached, which corresponds to a successful synthesis, or when a step cap is reached, which corresponds to a failed rollout. Recall that our goal is to synthesize Clifford circuits with as few gates as possible (and in particular, as few two-qubit gates as possible). Accordingly, we define our reward function as follows: ∥M G − I2n ∥0 R(M, G) := − 0.01 I(G = Hi or Si ) − I(G = CZi,j ) + 25 I(M G = I2n ) − , 2 {z } | {z } | | 8n {z } | {z } single-qubit gate penalty

two-qubit gate penalty

success bonus

progress shaping

where I denotes an indicator function and ∥ · ∥0 counts the number of nonzero entries in a matrix. The single- and two-qubit gate penalty terms of the reward function correspond to the actual cost of the gates on an actual quantum computer. The success bonus term encourages the agent to avoid the undesirable local optimum of only ever applying single-qubit gates. Finally, the last term computes the Hamming distance of the current matrix from the identity, which provides a dense notion of progress throughout an episode. Other reward weights and shapes may be possible here, but we found empirically that this particular choice is robust and works well in practice. Overall, our goal is to find a policy π to maximise the usual expected sum of discounted rewards over PT the episode, i.e. Eπ [ t=0 γ t Rt ], where here Rt denotes the reward received at time t; the variable T denotes the length of the episode (capped at a maximum number of steps); and γ ∈ [0, 1) is a discount factor. We allow π to be stochastic, so that formally π : S → PA, where PA denotes the set of all probability distributions over A. In our experiments, we train π via PPO [36], and at evaluation time either greedily decode a fixed policy or run policy-guided rollouts (see Appendix E). 4.3

Clifford synthesis symmetries

The Clifford synthesis problem admits a natural notion of symmetry. At a high level, this is because the underlying physical qubits are interchangeable, and so relabeling the qubits in a tableau should correspondingly relabel the qubits in the optimal action sequence. In turn, these symmetries also translate into symmetries of the reinforcement learning problem itself, as we explain now. Denote by Sym(n) the group of permutations of n elements, i.e. its elements are bijections of the form σ : {1, . . . , n} → {1, . . . , n}. This group acts on the state space S as follows: P 0 σ · M := Πσ M ΠTσ , where Πσ := σ , 0 Pσ and Pσ is the permutation matrix associated with σ. Intuitively, this says that relabeling qubits simultaneously permutes the corresponding rows and columns of the block tableau, while preserving the 2 × 2 block structure structure within each qubit. Sym(n) also acts on the action space A by σ · Hi := Hσ(i) ,

σ · Si := Sσ(i) ,

σ · CZi,j := CZσ(i),σ(j) .

Intuitively, this says that after relabeling the qubits, the same physical gate is represented by the correspondingly relabeled qubit indices. In turn, this leads to an action of Sym(n) on the space of distributions PA via the pushforward (i.e. (σ · P )(a) := P (σ −1 · a) for P ∈ PA). It is now straightforward to show that the transition function T and reward function R are respectively equivariant and invariant to these actions, in the sense that T (σ · M, σ · G) = σ · T (M, G),

R(σ · M, σ · G) = R(M, G)

always holds. By a standard symmetry result for finite MDPs [48], this leads to the following result. (See also Appendix A for a proof in our notation.) Proposition 4.1. There exists an optimal policy π ⋆ : S → PA whose corresponding value function V ⋆ : S → R has the following equivariance and invariance properties respectively: π ⋆ (σ · M ) = σ · π ⋆ (M ), 4.4

V ⋆ (σ · M ) = V ⋆ (M ).

Network Architecture

We solve our reinforcement learning problem using PPO [36], which requires us to parameterize a neural network S → PA × R that maps states to pairs of action distributions and value estimates. In 5

light of Proposition 4.1, we would like these outputs to be respectively equivariant and invariant to the symmetries of the problem. To achieve this, we design a novel neural network architecture that respects these symmetries by construction. At a high level, the architecture we use is as follows: Aggregate

Embed

S −−−→ |Rn×n×h {z } −−−−−→ Edge features

Message passing

n×h n×n×h R | {z } × |R {z } −−−−→

Node features

Edge features (unchanged)

Readout

n×n×h n×h |R{z } × |R {z } −−−−→ PA×R

Node features (updated)

Edge features (unchanged)

We explain each of the layers in turn now. Embedding. Recall that tableaus are 2n × 2n binary matrices. Given an input tableau Min ∈ S, we begin by reshaping it to an n×n×4 binary tensor M according to its four n×n quadrants (see Figure 3). From a physical perspective, each entry Mij ∈ {0, 1}4 then encodes the directed interactions from the i-th to the j-th qubit in the corresponding Clifford circuit. We then map each Mij to an embedding in Rh using an embedding dictionary with 25 = 32 entries, where the dictionary key is given by the four bits in Mij together with a binary indicator I(i = j) for whether the entry encodes a self-interaction. We denote the resulting embeddings by eij ∈ Rh . (0)

Aggregation. To initialize our message passing, we compute an aggregate embedding qi ∈ Rh for each qubit by mean and max pooling the embeddings eij over the rows and columns of the tableau. We give precise details of this pooling in Appendix C. It is also possible to initialise message passing (0) more simply with qi = 0, although we found that aggregation performed better empirically. Message passing. After aggregation, we perform L rounds of message passing, where in each (k) round we compute ordered-pair messages mi←j from qubit j to the updated qubit i as follows: (k)

(k)

mi←j := ϕmsg (qi , qj , eij , eji , I(i ̸= j)) ∈ Rh , where ϕmsg is a shared MLP. Here the indicator I(i ̸= j) allows the message function to treat diagonal entries differently from off-diagonal ones, which has physical significance since the former encode self-interactions of qubits rather than interactions between distinct qubits. Given these messages, we then update the qubit embeddings as follows: (k+1) (k) := LayerNorm qi(k) + ϕupd qi(k) , meanj m(k) qi , i←j , maxj mi←j , ηi,1 where ηi,1 denotes additional rank-based count features (see Appendix C for a definition). The latter are motivated by work by [46], who show high correlation between these features and the optimal 2-qubit gate count of the related problem of state preparation. (L)

Readout. After L rounds of message passing, we get final qubit embeddings qi := qi for each ′ qubit i. We then form a global summary g ∈ Rh by computing permutation invariant summary statistics of these final embeddings and their associated count features (see Appendix C for details). To obtain the final action distribution, we then compute xi := ϕH (qi , eii , g) ∈ R

yi := ϕS (qi , eii , g) ∈ R

zij := ϕCZ (qi , qj , eij , eji , g) ∈ R,

where ϕH , ϕS , and ϕCZ are MLPs. We then take the collection of every xi , yi , and zij to be the logits for the policy over A. This guarantees that the same logit is produced for CZi,j and CZj,i , as required by the symmetry of these actions. For the value function estimate, we return ϕV (g), where ϕV is another MLP. Equivariance and invariance. Apart from the value head of the readout layer, each of the individual layers above is equivariant with respect to permutations of the qubits. For example, it is straightforward to check that relabelling the qubits in the input tableau results in a corresponding relabeling of the embeddings eij , which is then preserved by the aggregation and message-passing layers, and finally results in a corresponding relabeling of the action logits. Since the value head only depends on the global features g, it is itself invariant to qubit relabeling. 6

Binary Tableau XX ZX

Reshape

XZ ZZ

Embed

Aggregate

e11

e12

e13

q10

e21

e22

e23

q20

e31

e32

e33

q30

Message Passing

Readout q1(L)

q1i

q2(L) q2i

q3i

q3(L)

Value Head H, S Heads CZ Heads

Figure 3: Architecture of the permutation-equivariant policy used for Clifford synthesis. The input tableau is reshaped and embedded into an n × n grid of qubit-pair edge features, aggregated into n qubit tokens, updated by edge-conditioned message passing on the complete qubit graph, and passed to equivariant policy heads for H, S, and CZ actions, alongside an invariant value head. Size-agnosticism. The same design is also size-agnostic: the only learned weights are in the embedding dictionary and the MLPs, and nothing in the way these are applied depends on the number of qubits n. Accordingly, the same learned weights can be reused across different qubit counts. 4.5

Curriculum

We have not yet defined how the initial tableau Mtarget for each episode is generated. The number 2 Qn i Θ(n2 ) of valid tableaus grows as |Sp(2n, F2 )| = 2n [10]. As such, if we i=1 (4 − 1) = 2 obtained Mtarget by sampling uniformly, we would obtain very difficult targets with high probability, which would lead to very sparse rewards early in training. We therefore use curriculum learning [5, 31] inspired by reverse curriculum generation for sparse-reward goal-reaching problems [18] and DeepCubeA’s reverse-from-goal training for combinatorial puzzles [2]. Kremer et al. also use curriculum learning for constrained Clifford synthesis [27]. In our approach, each episode target is generated by a random walk from the identity of some fixed length. The walk length therefore controls the distance from the goal and hence the difficulty. Training starts with short walks and advances to longer walks once rollout success at the current level reaches 100%. Precisely, for n ∈ N and d ≥ 0, we define Pn,d to be the distribution over Sp(2n, F2 ) generated by a random walk from I2n of length L = ⌊d⌋ + B,

where B ∼ Bernoulli(d − ⌊d⌋).

This means that E[L] = d, and moreover when d is an integer, it reduces to a d-step random walk. This interpolation gives more granular control over the curriculum, especially for small d, and shifts the distribution smoothly. For n ≥ 3, this random walk converges to uniform sampling from Sp(2n, F2 ) as d → ∞ (see Theorem B.1 in Appendix B for a proof). In addition to d, our size-agnostic agent also allows us to build a curriculum across different qubit counts n. In our experiments, we exploit this by training initially on six-qubit circuits until convergence, before continuing to train on ten-qubit circuits.

Experimental Results

Using the setup described above, we optimized the parameters of our model via PPO [36] using PufferLib [41]. Full training hyperparameters are listed in Appendix Table A4. At evaluation time, we then use our policy as a heurstic for Clifford synthesis, either by greedily following the action with highest probability at each step, or by sampling actions according to the policy until a solution is found or a step cap is reached (see Appendix E for details). 5.1

Recovering Optimal Six-Qubit Clifford Circuits

We applied our approach to the benchmark of 1003 optimal six-qubit tableaus considered by Bravyi et al. [8]. (These reference tableaus were themselves taken from the exhaustive database of optimal six-qubit Clifford circuits generated by Bravyi–Latone–Maslov [10].) However, we emphasize that our agent is trained according to the methodology described in Section 4, without any access to those 1003 tableaus or their optimal solutions. Even the easiest benchmark targets are extremely 7

unlikely to be seen during training, since there are about 1.3 × 1014 optimal six-qubit Cliffords with five CZ gates [10]. As a baseline, we compared against the symbolic peephole optimizer of [8], which matches the optimal entangling-gate count for 982/1003 circuits after 217 hours, with the 21 remaining circuits consuming 576 hours without reaching the optimum [9].

Optima recovered

On this task, our approach solves each of the 1003 in995 1000 stances and matches the optimum on 995/1003 circuits 982 (99.2%). The remaining eight circuits are each off by one CZ gate. Greedily following the trained policy takes 750 0.83 seconds for the whole suite and already recovers 507/1003 optimal solutions. A brief policy-guided search pass brings all the circuits to within one CZ gate from optimal in 21 seconds total, with 610/1003 of those be500 0 60 120 180 ing optimal. The longer policy-guided search reaches the Search time (min) previous state-of-the-art 982/1003 in 22 minutes, reaches 990/1003 after 52 minutes, and reaches 995/1003 after 183 minutes, outperforming the previous state-of-the-art of Bravyi et al. [8] by a significant margin. 5.2

Generalizing Beyond the Training Size

We also evaluated our approach on larger circuits beyond six qubits. Since there are no optimal references available in this context, we compare our agent’s performance against existing synthesis algorithms that run in polynomial time. Specifically, we compare against the two main polynomialtime Clifford synthesizers from Qiskit [1, 24, 23], a quantum computing library with over a million monthly downloads at the time of writing. We do not compare against the neural methods discussed in Section 3, as they do not address our specific task, and moreover are not size-agnostic and hence would require training separate models for each qubit count in our sweep. To test size generalization and the cross-size curriculum, we benchmarked two checkpoints: one trained only on six-qubit circuits, and one for which we subsequently continued training on ten-qubit circuits also. For each qubit size n and number of initial gates d, we generated 100 held-out targets by applying d uniformly sampled Clifford gates from G to the identity tableau. We swept across n = 7, . . . , 30 and d ∈ {16, 32, 64, 128, 256, 512, 1024, ∞}, using the same fixed target tableaus for the learned agent and both Qiskit baselines. The ∞ circuits are actually sampled uniformly [25] from Sp(2n, F2 ) as that is the limit distribution of the random walk (see Theorem B.1). The results of this experiment are given in Figure 4 (see also Appendix A4 and A7 for the full sweeps). The model trained on ten qubits produces much shorter circuits when it succeeds. Across the finite-difficulty settings, the model returns lower average CZ counts than the Bravyi et al. greedy and Aaronson–Gottesman algorithms. At 30 qubits and 1024 initial Clifford gates, where the learned synthesizer still solves all targets, it uses 323.3 CZ gates on average, 124.2 fewer than Qiskit’s Bravyi et al. greedy synthesizer and 460.1 fewer than Aaronson–Gottesman. At the ∞ endpoint, the ten-qubit checkpoint starts to lose reliability beyond about 24 qubits: solve rate falls from 99% at 24 qubits to 59% at 30 qubits. The learned model still beats both Qiskit baselines in CZ count after restricting all methods to its solved targets. The appendix gives the full sweeps (Figures A5 and A6). Interestingly, these scale-transfer experiments expose a reliability–quality tradeoff between the two checkpoints. The six-qubit trained model is more reliable: it solves every larger-circuit target in our sweep, including the ∞ endpoint. Its circuits are worse, however, especially on tableaus corresponding to larger and deeper circuits. See Appendix D.1 for additional discussion. 5.3

Architecture Ablation

We ablate the policy representation by progressively removing qubit-level structure. Ours uses the message-passing architecture from Section 4; RelTransformer replaces message passing with relationaware attention [39]; Transformer uses ordinary self-attention [45]; MLP keeps equivariant per-qubit processing but removes communication between qubits; and FlatMLP uses an unconstrained MLP over the flattened tableau. 8

Mean CZ count

Qiskit-greedy

Qiskit-AG

10 qubits

100 80 60 40 20

Qiskit on RL-solved

15 qubits

400

150 100

200

50 16

128

256

512 1024

24 qubits

600

200

128

256

512 1024

99%

128

256

512 1024

Initial Clifford gate count

Figure 4: CZ-count comparison for the ten-qubit-trained model at 10, 15, and 24 evaluation qubits. Each panel fixes the qubit count and sweeps target difficulty from 16–1024 initial Clifford gates through the ∞ endpoint defined in Section 5.2. Solid Qiskit curves show all-target means; dotted same-color markers and shaded gaps show the corresponding Qiskit means restricted to targets solved by the agent when these differ. The percentage annotation gives the agent solve rate at the ∞ endpoint. Appendix Figure A4 shows the full evaluation on 7–30 qubits. CZ Count

Family Ours RelTransformer [39] Transformer [45] MLP FlatMLP

13.49 ± 0.01 13.47 ± 0.06 14.12 ± 0.28 15.88 ± 0.05 15.19 ± 0.16

Table 1: Architecture ablation on 100 uniformly sampled six-qubit Clifford targets. Entries are mean CZ counts over solved targets under rollout decoding, reported as mean ± standard deviation over three seeds. All models solve every target. The full sweep is in Appendix Table A2.

Table 1 shows model performance on uniformly random Cliffords. Since all models solve all targets, the comparison is entirely in circuit quality. The communication-based equivariant models perform best: Ours and RelTransformer are essentially tied, ordinary attention is weaker, removing communication is worse, and the flat unstructured baseline is also substantially behind. The best communication-based models use more than two fewer CZ gates than MLP and about two fewer than FlatMLP. This suggests that the important ingredient is not the exact message-passing update, but the combination of tableau-block features, qubit equivariance, and information flow between qubit tokens.

Discussion, Limitations, and Future Work

We presented a neural synthesis algorithm for Clifford circuits based on reinforcement learning over stabilizer-tableau reduction. The central architectural idea is to build permutation symmetry directly into the policy so that the model can scale across qubit sizes. We have shown that this setup is robust, giving optimal circuits at six qubits and improving over existing polynomial-time algorithms for larger qubit systems. The symmetries and two-dimensional learning curricula we study here exist in related problems, and it is an open question whether these techniques developed in this work would yield significant improvements in those areas as well. Future work can also explore the optimal way to perform this two-dimensional curriculum learning. The learned policy is small and relatively fast, and can be used on consumer devices without a GPU. In the future, this can be packaged into a lightweight library for neural Clifford synthesis. A limitation of our current formulation is that one-step action selection becomes expensive for large n: the number of possible CZi,j actions grows as O(n2 ), and the resulting rollouts can require O(n2 ) gates. This makes naive inference scale poorly even when the network itself is size-agnostic. Future work could reduce this cost with a factorized action space, hierarchical actions that choose structured multi-gate moves, or policy-guided planning methods such as Monte Carlo tree search during training and inference. 9

See y-richie-y.github.io/clifford/ for a live browser demo and recovered Bravyi et al. benchmark circuits, including new circuits for previously unrecovered optima.

Acknowledgements We thank Christopher Mingard, Jasmine Brewer, and Ciarán Ryan-Anderson for useful feedback on the manuscript. We thank Alexander Koziell-Pipe for interesting discussions and for exploring Monte Carlo tree search for circuit synthesis. RY thanks Dr Simon Harrison for generous support through the Wolfson Harrison UKRI Quantum Foundation Scholarship. This work is supported by the Engineering and Physical Sciences Research Council grant number EP/Z002230/1, (De)constructing quantum software (DeQS). The research of RC is supported by the National Research Foundation, Singapore, under its National Research Foundation Fellowship in Artificial Intelligence (Award No.: NRFFIAI1-2024-0014).

References [1] Scott Aaronson and Daniel Gottesman. Improved simulation of stabilizer circuits. Physical Review A, 70(5), 2004. doi: 10.1103/PhysRevA.70.052328. URL https://doi.org/10. 1103/physreva.70.052328. [2] Forest Agostinelli, Stephen McAleer, Alexander Shmakov, and Pierre Baldi. Solving the rubik’s cube with deep reinforcement learning and search. Nature Machine Intelligence, 1 (8):356–363, 2019. doi: 10.1038/s42256-019-0070-z. URL https://doi.org/10.1038/ s42256-019-0070-z. [3] Matthew Amy and Michele Mosca. T-count optimization and reed–muller codes. IEEE Transactions on Information Theory, 65(8):4771–4784, 2019. doi: 10.1109/TIT.2019.2906374. URL https://doi.org/10.1109/tit.2019.2906374. [4] Unai Aseguinolaza, Nahual Sobrino, Gabriel Sobrino, Joaquim Jornet-Somoza, and Juan Borge. Error estimation in current noisy quantum computers. Quantum Information Processing, 23(5), 2024. doi: 10.1007/s11128-024-04384-z. URL https://doi.org/10.1007/ s11128-024-04384-z. [5] Yoshua Bengio, Jérôme Louradour, Ronan Collobert, and Jason Weston. Curriculum learning. In Proceedings of the 26th Annual International Conference on Machine Learning, pages 41–48, 2009. doi: 10.1145/1553374.1553380. URL https://doi.org/10.1145/1553374. 1553380. [6] P. Oscar Boykin, Tal Mor, Matthew Pulver, Vwani Roychowdhury, and Farrokh Vatan. A new universal and fault-tolerant quantum basis. Information Processing Letters, 75(3):101–107, 2000. doi: 10.1016/S0020-0190(00)00084-3. URL https://doi.org/10.1016/s0020-0190(00) 00084-3. [7] Sergey Bravyi and Alexei Kitaev. Universal quantum computation with ideal Clifford gates and noisy ancillas. Physical Review A, 71(2), 2005. doi: 10.1103/PhysRevA.71.022316. URL https://doi.org/10.1103/physreva.71.022316. [8] Sergey Bravyi, Ruslan Shaydulin, Shaohan Hu, and Dmitri Maslov. Clifford circuit optimization with templates and symbolic Pauli gates. Quantum, 5:580, 2021. doi: 10.22331/ q-2021-11-16-580. URL https://doi.org/10.22331/q-2021-11-16-580. [9] Sergey Bravyi, Ruslan Shaydulin, Shaohan Hu, and Dmitri Maslov. Data to accompany Clifford Circuit Optimization with Templates and Symbolic Pauli Gates, 2021. URL https://github.com/rsln-s/Clifford_Circuit_Optimization_with_ Templates_and_Symbolic_Pauli_Gates. GitHub repository, accessed 2026-05-03. [10] Sergey Bravyi, Joseph A. Latone, and Dmitri Maslov. 6-qubit optimal Clifford circuits. npj Quantum Information, 8(1), 2022. doi: 10.1038/s41534-022-00583-7. URL https://doi. org/10.1038/s41534-022-00583-7. 10

[11] Francois Charton, Alexandre Krajenbrink, Konstantinos Meichanetzidis, and Richie Yeung. Teaching small transformers to rewrite ZX diagrams. In 3rd MATH-AI Workshop at NeurIPS’23, 2023. URL https://mathai2023.github.io/papers/34.pdf. [12] Jacopo Cossio, Daniele Lizzio Bosco, Riccardo Romanello, Giuseppe Serra, and Carla Piazza. AlphaCNOT: Learning CNOT minimization with model-based planning, 2026. URL https: //arxiv.org/abs/2604.13812v1. [13] Niel de Beaudrap, Xiaoning Bian, and Quanlong Wang. Fast and effective techniques for T-Count reduction via spider nest identities. In 15th Conference on the Theory of Quantum Computation, Communication and Cryptography (TQC 2020), volume 158, pages 11:1–11:23. Schloss Dagstuhl – Leibniz-Zentrum für Informatik, 2020. doi: 10.4230/LIPIcs.TQC.2020.11. URL https://drops.dagstuhl.de/entities/document/10.4230/LIPIcs.TQC.2020. 11. [14] Michael Doherty, Matteo Puviani, Jasmine Brewer, Gabriel Matos, David Amaro, Ben Criger, and David T. Stephen. Fast stabilizer state preparation via AI-optimized graph decimation, 2026. URL https://arxiv.org/abs/2603.17743v1. [15] Ayushi Dubal, David Kremer, Simon Martiel, Victor Villar, Derek Wang, and Juan CruzBenito. Pauli network circuit synthesis with reinforcement learning, 2025. URL https: //arxiv.org/abs/2503.14448v1. [16] Ross Duncan, Aleks Kissinger, Simon Perdrix, and John van de Wetering. Graph-theoretic simplification of quantum circuits with the ZX-calculus. Quantum, 4:279, 2020. doi: 10.22331/ q-2020-06-04-279. URL https://doi.org/10.22331/q-2020-06-04-279. [17] Alhussein Fawzi, Matej Balog, Aja Huang, Thomas Hubert, Bernardino Romera-Paredes, Mohammadamin Barekatain, Alexander Novikov, Francisco J. R. Ruiz, Julian Schrittwieser, Grzegorz Swirszcz, David Silver, Demis Hassabis, and Pushmeet Kohli. Discovering faster matrix multiplication algorithms with reinforcement learning. Nature, 610(7930):47–53, 2022. doi: 10.1038/s41586-022-05172-4. URL https://doi.org/10.1038/s41586-022-05172-4. [18] Carlos Florensa, David Held, Markus Wulfmeier, Michael Zhang, and Pieter Abbeel. Reverse curriculum generation for reinforcement learning. In Conference on Robot Learning, pages 482–495, 2017. URL http://proceedings.mlr.press/v78/florensa17a/ florensa17a.pdf. [19] Craig Gidney. Inverting clifford tableaus, 2020. URL https://algassert.com/post/2002. Blog post, accessed 2026-04-07. [20] Craig Gidney, Noah Shutty, and Cody Jones. Magic state cultivation: growing t states as cheap as CNOT gates, 2024. URL https://arxiv.org/abs/2409.17595v1. [21] Daniel Gottesman. The heisenberg representation of quantum computers, 1998. URL https: //arxiv.org/abs/9807006v1. [22] Luke E Heyfron and Earl T Campbell. An efficient quantum compiler that reduces T count. Quantum Science and Technology, 4(1):015004, 2018. doi: 10.1088/2058-9565/aad604. URL https://doi.org/10.1088/2058-9565/aad604. [23] IBM Quantum and Qiskit contributors. GreedySynthesisClifford, 2026. URL https://quantum.cloud.ibm.com/docs/en/api/qiskit/2.2/qiskit.transpiler. passes.synthesis.hls_plugins.GreedySynthesisClifford. Qiskit 2.2 API documentation, accessed 2026-05-07. [24] IBM Quantum and Qiskit contributors. DefaultSynthesisClifford, 2026. URL https://quantum.cloud.ibm.com/docs/en/api/qiskit/2.2/qiskit.transpiler. passes.synthesis.hls_plugins.DefaultSynthesisClifford. Qiskit 2.2 API documentation, accessed 2026-05-07. [25] IBM Quantum and Qiskit contributors. random_clifford, 2026. URL https://quantum. cloud.ibm.com/docs/en/api/qiskit/2.2/quantum_info. Qiskit 2.2 API documentation, accessed 2026-05-07. 11

[26] Aleks Kissinger and John van de Wetering. Reducing the number of non-Clifford gates in quantum circuits. Physical Review A, 102(2), 2020. doi: 10.1103/PhysRevA.102.022406. URL https://doi.org/10.1103/physreva.102.022406. [27] David Kremer, Victor Villar, Hanhee Paik, Ivan Duran, Ismael Faro, and Juan Cruz-Benito. Practical and efficient quantum circuit synthesis and transpiling with reinforcement learning, 2024. URL https://arxiv.org/abs/2405.13196v2. [28] Daniel Litinski. Magic state distillation: Not as costly as you think. Quantum, 3:205, 2019. doi: 10.22331/q-2019-12-02-205. URL https://doi.org/10.22331/q-2019-12-02-205. [29] Daniel Litinski and Felix von Oppen. Lattice surgery with a twist: Simplifying Clifford gates of surface codes. Quantum, 2:62, 2018. doi: 10.22331/q-2018-05-04-62. URL https: //doi.org/10.22331/q-2018-05-04-62. [30] Alexander Mattick, Maniraman Periyasamy, Christian Ufrecht, Abhishek Y. Dubey, Christopher Mutschler, Axel Plinge, and Daniel D. Scherer. Optimizing quantum circuits via ZX diagrams using reinforcement learning and graph neural networks, 2025. URL https://arxiv.org/ abs/2504.03429v1. [31] Sanmit Narvekar, Bei Peng, Matteo Leonetti, Jivko Sinapov, Matthew E. Taylor, and Peter Stone. Curriculum learning for reinforcement learning domains: A framework and survey. Journal of Machine Learning Research, 21(181):1–50, 2020. URL http://jmlr.org/papers/ volume21/20-212/20-212.pdf. [32] Maximilian Nägele and Florian Marquardt. Optimizing ZX-diagrams with deep reinforcement learning. Machine Learning: Science and Technology, 5(3):035077, 2024. doi: 10.1088/ 2632-2153/ad76f7. URL https://doi.org/10.1088/2632-2153/ad76f7. [33] Tom Peham, Nina Brandl, Richard Kueng, Robert Wille, and Lukas Burgholzer. Depth-optimal synthesis of Clifford circuits with SAT solvers. In 2023 IEEE International Conference on Quantum Computing and Engineering (QCE), pages 802–813, 2023. doi: 10.1109/QCE57702. 2023.00095. URL https://doi.org/10.1109/qce57702.2023.00095. [34] Jordi Riu, Jan Nogué, Gerard Vilaplana, Artur Garcia-Saez, and Marta P. Estarellas. Reinforcement learning based quantum circuit optimization via ZX-calculus. Quantum, 9:1758, 2025. doi: 10.22331/Q-2025-05-28-1758. URL https://doi.org/10.22331/q-2025-05-28-1758. [35] Francisco J. R. Ruiz, Tuomas Laakkonen, Johannes Bausch, Matej Balog, Mohammadamin Barekatain, Francisco J. H. Heras, Alexander Novikov, Nathan Fitzpatrick, Bernardino RomeraParedes, John van de Wetering, Alhussein Fawzi, Konstantinos Meichanetzidis, and Pushmeet Kohli. Quantum circuit optimization with AlphaTensor. Nature Machine Intelligence, 7(3): 374–385, 2025. doi: 10.1038/s42256-025-01001-1. URL https://doi.org/10.1038/ s42256-025-01001-1. [36] John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms, 2017. URL https://arxiv.org/abs/1707.06347v2. [37] Peter Selinger. Efficient Clifford+T approximation of single-qubit operators. Quantum Information and Computation, 15(1&2):159–180, 2015. doi: 10.26421/qic15.1-2-10. URL https://doi.org/10.26421/qic15.1-2-10. [38] Irfansha Shaik and Jaco van de Pol. CNOT-optimal Clifford synthesis as SAT, 2025. URL https://drops.dagstuhl.de/entities/document/10.4230/LIPIcs.SAT.2025.28. [39] Peter Shaw, Jakob Uszkoreit, and Ashish Vaswani. Self-attention with relative position representations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), pages 464–468, New Orleans, Louisiana, 2018. Association for Computational Linguistics. doi: 10.18653/v1/N18-2074. URL https://aclanthology.org/N18-2074/. [40] Will Simmons. Relating measurement patterns to circuits via Pauli flow. Electronic Proceedings in Theoretical Computer Science, 343:50–101, 2021. doi: 10.4204/EPTCS.343.4. URL https://doi.org/10.4204/eptcs.343.4. 12

[41] Joseph Suarez. PufferLib: Making reinforcement learning libraries and environments play nice, 2024. URL https://arxiv.org/abs/2406.12905v1. [42] Arianne van de Griend. Constrained quantum CNOT circuit re-synthesis using deep reinforcement learning. Master’s thesis, Radboud University, 2019. URL https://theses.ubn.ru. nl/handle/123456789/10713. [43] John van de Wetering, Richie Yeung, Tuomas Laakkonen, and Aleks Kissinger. Optimal compilation of parametrised quantum circuits. Quantum, 9:1828, 2025. doi: 10.22331/ q-2025-08-27-1828. URL https://doi.org/10.22331/q-2025-08-27-1828. [44] Elise van der Pol, Daniel E. Worrall, Herke van Hoof, Frans A. Oliehoek, and Max Welling. MDP homomorphic networks: Group symmetries in reinforcement learning. In Advances in Neural Information Processing Systems, volume 33, pages 4199–4210, 2020. URL https://proceedings.neurips.cc/paper/2020/file/ 2be5f9c2e3620eb73c2972d7552b6cb5-Paper.pdf. [45] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Information Processing Systems, volume 30, 2017. URL https://proceedings.neurips.cc/ paper/2017/hash/3f5ee243547dee91fbd053c1c4a845aa-Abstract.html. [46] Mark Webster, Stergios Koutsioumpas, and Dan E Browne. Heuristic and optimal synthesis of CNOT and Clifford circuits, 2025. URL https://arxiv.org/abs/2503.14660v1. [47] Remmy Zen, Jan Olle, Luis Colmenarez, Matteo Puviani, Markus Müller, and Florian Marquardt. Quantum circuit discovery for fault-tolerant logical state preparation with reinforcement learning. Physical Review X, 15(4):041012, 2025. doi: 10.1103/gqpr-dgz7. URL https://doi.org/ 10.1103/gqpr-dgz7. [48] Martin Zinkevich and Tucker R. Balch. Symmetry in Markov decision processes and its implications for single agent and multiagent learning. In Proceedings of the Eighteenth International Conference on Machine Learning, pages 632–640. Morgan Kaufmann, 2001. URL https://www.cs.cmu.edu/~maz/publications/symmetry7.pdf.

Equivariant Optimal Policies

This appendix records the symmetry justification for using an equivariant policy class in the Clifford synthesis MDP, following the standard symmetry theory of finite MDPs [44, 48]. The statement is not specific to Clifford circuits; it applies to any stationary discounted MDP whose dynamics and rewards commute with a group action. Proposition. Let (S, A, T , R) be a finite deterministic MDP with stationary transition and reward functions and discount factor γ ∈ [0, 1). Let a group Γ act on both S and A. Suppose that, for every σ ∈ Γ, M ∈ S, and G ∈ A, T (σ·M, σ·G) = σ·T (M, G),

R(σ·M, σ·G) = R(M, G).

Then there exists an optimal stationary stochastic Markov policy π ⋆ that is equivariant: π ⋆ (σ·G | σ·M ) = π ⋆ (G | M ) for all states M , actions G, and group elements σ. Proof.

Let V ⋆ be the unique solution to the discounted Bellman optimality equation V ⋆ (M ) = max [R(M, G) + γV ⋆ (T (M, G))] . G∈A

For any fixed σ ∈ Γ, define Wσ (M ) = V ⋆ (σ·M ). Then Wσ (M ) = V ⋆ (σ·M ) = max [R(σ·M, G′ ) + γV ⋆ (T (σ·M, G′ ))] ′ G ∈A

= max [R(σ·M, σ·G) + γV ⋆ (T (σ·M, σ·G))] G∈A

= max [R(M, G) + γV ⋆ (σ·T (M, G))] G∈A

= max [R(M, G) + γWσ (T (M, G))] . G∈A

In the second equality we reindexed the maximization using the bijection G 7→ σ·G. Thus Wσ also satisfies the Bellman optimality equation. By uniqueness, Wσ = V ⋆ , so V ⋆ (σ·M ) = V ⋆ (M ). Define the optimal action-value function Q⋆ (M, G) = R(M, G) + γV ⋆ (T (M, G)). The invariance of V ⋆ , together with the equivariance of T and invariance of R, gives Q⋆ (σ·M, σ·G) = Q⋆ (M, G). Therefore the optimal action set A⋆ (M ) = arg max Q⋆ (M, G) G∈A

satisfies A⋆ (σ·M ) = σ·A⋆ (M ). Let π ⋆ (· | M ) be the uniform distribution over A⋆ (M ). This policy is optimal because it assigns probability only to optimal actions. It is also equivariant, since the optimal action set at σ ·M is exactly the relabeling of the optimal action set at M . Hence π ⋆ (σ·G | σ·M ) = π ⋆ (G | M ). □ 14

Application to Clifford synthesis. In our Clifford synthesis MDP, the symmetry group is the qubit permutation group Sn . A permutation σ ∈ Sn acts on tableaus by simultaneously relabeling the row and column qubit blocks, and acts on generators by Hi 7→ Hσ(i) ,

Si 7→ Sσ(i) ,

CZi,j 7→ CZσ(i),σ(j) .

Right multiplication by generators commutes with this relabeling, so T (σ·M, σ·G) = σ·T (M, G). The reward is also invariant: the single-qubit and two-qubit costs depend only on the gate type, the identity tableau is fixed by every qubit relabeling, and the Hamming distance to the identity is unchanged by simultaneously permuting rows and columns. Therefore the proposition applies, and there exists an optimal stationary stochastic Markov policy for the discounted Clifford synthesis objective that is equivariant under qubit relabeling. If a hard rollout cap is included as part of the objective, one can augment the state with the remaining step budget and let Sn act trivially on that extra coordinate; the same argument then gives an optimal equivariant policy on the augmented state space.

Random-Walk Limit Distribution

This appendix justifies the use of uniformly sampled Clifford targets as the ∞ endpoint of the random-walk curriculum. Theorem B.1. Fix n ≥ 3, write Γ = Sp(2n, F2 ), and let K be the transition matrix for the random walk on Γ that right-multiplies by a uniformly sampled generator from G in (1). Then the d-step distribution from the identity converges to the uniform distribution on Γ: 1 K d (I2n , M ) −→ for every M ∈ Γ. |Γ| For n = 1 and n = 2, the corresponding walk has period 2 and therefore does not converge to the uniform distribution at fixed walk length. Proof. We use the standard ergodic theorem for finite Markov chains: an irreducible and aperiodic finite chain converges to its unique stationary distribution. Because G generates Γ, the associated Cayley graph is connected, so the chain is irreducible. Figure A1 shows a local ball in the corresponding generator graph for the two-qubit case, where random-walk steps move along generator-labeled edges from the identity. The uniform distribution is stationary: if M is uniform on Γ and G ∈ G is fixed, then M G is still uniform, because right multiplication by G only permutes the elements of Γ. For convergence to this stationary distribution we also need aperiodicity. Each generator is an involution in the binary symplectic representation, so G2 = I2n gives a closed walk of length 2. For n ≥ 3, there is also an odd closed walk: H1 CZ1,2 H1 CZ1,3 H1 CZ2,3 CZ1,2 H1 CZ1,3 = I2n . Figure A2 draws this nine-step identity circuit. Equivalently, the identity is verified by multiplying the generator matrices defined in Appendix D; it acts nontrivially only on qubits 1, 2, 3, so the same relation embeds in every larger n. Thus the period of the identity divides both 2 and 9, and is therefore 1. Irreducibility implies every state has the same period, so the whole chain is finite, irreducible, and aperiodic. The standard convergence theorem for finite Markov chains then gives 1 K d (I2n , M ) −→ for every M ∈ Γ, |Γ| which is exactly uniform sampling from Sp(2n, F2 ). For n = 1 and n = 2, the walk generated by {Hi , Si , CZi,j } has period 2, so the fixed-length distributions oscillate between the two parity classes of the Cayley graph rather than converging. Concretely, for n = 1 the generators are only H1 and S1 , and the two parity classes are {I, H1 S1 , S1 H1 }

and

{H1 , S1 , H1 S1 H1 },

the elements reachable at even and odd times, respectively. For n = 2: Sp(4, F2 ) has 720 elements, split into two parity classes of size 360 under this generator walk.

CZ1, 2

Figure A1: Local view of the Cayley graph induced by the two-qubit Clifford generators. Vertices are tableaus reached from the identity by walks of length at most three; the legend identifies the generator for each edge.

Figure A2: Nine-gate identity circuit used in Theorem B.1.

Architecture Implementation Details

This appendix gives the implementation details behind the embedding, aggregation, message-passing, and readout stages described in Section 4. All additional features below are permutation-equivariant count-based features or permutation-invariant global summaries, so including them does not change the symmetry argument in the main text. Embedding. For each 2 × 2 block Mij of the reshaped tableau, the embedding lookup uses the four binary entries of Mij together with a diagonal indicator I(i = j), giving 32 possible block states. This produces the edge embedding eij ∈ Rh used in the main text. 16

(0)

Aggregation and count features. The initial qubit token qi is built from row-pooled and columnpooled edge embeddings, the diagonal edge embedding eii , and a count-feature vector ηi . Specifically, with a shared MLP ϕnode , we set (0)

:= ϕnode ([meanj eij , maxj eij ], [meanj eji , maxj eji ], | {z } | {z } Row features

Column features

eii |{z}

, ηi ).

(3)

Diagonal embedding

Here ηi := [ηi,1 , ηi,2 ] ∈ R11 . The choice of these features is motivated by Webster et al. [46], who show high correlation between related rank-count features and optimal two-qubit gate count for stabilizer state preparation. The first component ηi,1 ∈ R9 is row,1 row,2 col ηi,1 = di , ρrow , ρi , ρcol,1 , ρcol,2 , ρoffrow , ρoffcol , i , ρi , ρi i i i i where di indicates whether the diagonal block Mii is the 2 × 2 identity; ρrow and ρcol are the i i row,1 row,2 col,1 col,2 fractions of nonzero blocks in block row i and block column i; ρi , ρi and ρi , ρi are the corresponding fractions of rank-one and rank-two blocks; and ρoffrow , ρoffcol are the fractions i i of nonzero off-diagonal blocks in row i and column i, normalized by n − 1. The vector ηi,2 ∈ R2 records the rank of the diagonal block Mii as two indicators for rank one and rank two. Message passing. The message function ϕmsg and update function ϕupd are shared across all qubit indices and all message-passing rounds. As in the main text, the update for qubit i uses the current (k) token qi , mean-pooled incoming messages, max-pooled incoming messages, and the rank-count component ηi,1 . (L)

Readout. Let qi = qi be the final qubit token after L message-passing rounds. The implementation first forms invariant statistics s := [meani qi , maxi qi , stdi qi , c̄] , col offrow c̄ := meani di , meani ρrow , meani ρoffcol , meani ηi,2 , i , meani ρi , meani ρi i where stdi is the elementwise standard deviation over qubit tokens. Thus the invariant readout statistics have dimension 3h + 7. The implementation maps these statistics through a global MLP to obtain the learned summary g. For the one-qubit action heads, the readout can be written as the separate MLPs ϕH and ϕS used in the main text. Concretely, the implementation realizes these heads with a local MLP conditioned on an action-type one-hot vector: (qi , eii , g, ηi , τa ), where τa ∈ R2 is the action-type one-hot vector. The two-qubit CZ head receives the symmetric pair context (qi + qj , qi ⊙ qj , |qi − qj |, eij + eji , eij ⊙ eji , g). Here ⊙ denotes elementwise multiplication. The value head is the MLP ϕV (s) applied directly to the invariant statistics s.

Clifford Circuits and Tableaus

This appendix briefly reviews the circuit model and the stabilizer formalism underlying the synthesis problem studied in the main text. In the standard circuit model, a quantum algorithm is expressed as a sequence of elementary gates acting on qubits. A universal gate set such as Clifford+T can approximate arbitrary quantum computations to arbitrary precision. In this paper we focus on the Clifford subset, generated by H, S, and CZ, because it admits a compact exact representation unavailable for general circuits. For a more complete treatment of Clifford circuits and stabilizer tableaus, see Aaronson and Gottesman [1]. The Pauli observables correspond to the canonical measurement bases of a qubit. Their matrix representatives are 1 0 0 1 0 −i 1 0 I= , X= , Y = , Z= . 0 1 1 0 i 0 0 −1 17

H 1

S 2

H 6

⇒

⇒1

CZ3,4

⇒

CZ2,4

⇒

CZ1,4

⇒

⇒4

CZ2,3

CZ1,3

⇒

Figure A3: Full tableau reduction sequence for the four-qubit target tableau shown in Fig. 1. Starting from the Fig. 1 target tableau, each arrow is labeled by the next elementary gate applied by the displayed circuit, and the sequence ends at the identity tableau. The Clifford generators are the one-qubit gates 1 1 1 H=√ , 2 1 −1

1 0

0 , i

and the two-qubit controlled-Z gate  1 0 CZ =  0 0

0 1 0 0

0 0 1 0

 0 0 . 0 −1

A Clifford circuit can be characterized equivalently as a circuit that maps Pauli operators to Pauli operators under conjugation. This can be verified directly on the generators: HXH = Z,

HZH = X,

†

SZS † = Z,

SXS = Y, and for the two-qubit gate CZ, CZ(X ⊗ I)CZ† = X ⊗ Z,

CZ(Z ⊗ I)CZ† = Z ⊗ I, 18

CZ(I ⊗ X)CZ† = Z ⊗ X,

CZ(I ⊗ Z)CZ† = I ⊗ Z.

Since conjugation by each generator stays within the Pauli family, any circuit built from these generators does as well. It follows that a Clifford operation can be specified exactly by how it transforms the Pauli generators X1 , . . . , Xn , Z1 , . . . , Zn under conjugation. Those generator images determine the full Clifford action up to global phase. The only consistency requirement is that these images preserve the original commutation relations: Xi commutes with Xj , Zi commutes with Zj , and Xi commutes with Zj if and only if i ̸= j; on the same qubit, Xi and Zi anticommute. The standard bookkeeping device for these generator images is the stabilizer tableau. To obtain its binary form, we encode each Pauli string by two bits per qubit: for qubit k, the pair (xk , zk ) ∈ F22 represents I, X, Z, Y via (0, 0), (1, 0), (0, 1), (1, 1), respectively, up to an overall sign. An n-qubit Pauli string is therefore represented by a length-2n binary vector (x1 , . . . , xn | z1 , . . . , zn ). The product of two phase-free Pauli strings corresponds to addition of these binary vectors over F2 : for instance, on one qubit, XZ is represented by (1, 1), which is the encoding of Y up to phase. This is the reason the signs of Pauli strings can be separated from the binary tableau. The full stabilizer tableau includes additional phase bits, but the synthesis problem in this paper uses only the phase-free action on Pauli labels. This encoding preserves exactly the structure relevant for Clifford synthesis. On one qubit, the Pauli matrices satisfy X a Z b X c Z d = (−1)ad+bc X c Z d X a Z b ,

a, b, c, d ∈ {0, 1}.

Thus the exponent ad + bc, computed modulo two, records whether the two Pauli operators commute or anticommute. For n qubits this contribution is summed over qubits, so if u and v are the binary encodings of two Pauli strings, then the value of uT Ωv over F2 determines whether the underlying Pauli strings commute or anticommute, where 0 I Ω= . I 0 Choosing the ordered generator basis X1 , . . . , Xn , Z1 , . . . , Zn then gives a 2n × 2n binary matrix describing how a Clifford circuit acts on those generators. In our convention, row r records the binary Pauli string obtained by conjugating the r-th basis generator. For example, on one qubit, H exchanges X and Z, so 0 1 MH = , 1 0 where the first row is the image of X and the second row is the image of Z. Similarly, since SXS † = Y and SZS † = Z, and since Y has phase-free encoding (1, 1), 1 1 MS = . 0 1 For a two-qubit example, use the ordered basis X1 , X2 , Z1 , Z2 . The conjugation rules for CZ1,2 give X1 7→ X1 Z2 ,

X2 7→ Z1 X2 ,

Z1 7→ Z1 ,

Z2 7→ Z2 ,

and hence  1 0 MCZ1,2 =  0 0

0 1 0 0

0 1 1 0

 1 0 . 0 1

These examples illustrate how to read a tableau: each row is one generator image, columns 1, . . . , n record X-support, and columns n + 1, . . . , 2n record Z-support. This matrix is the binary part of the stabilizer tableau. The full tableau also includes phase bits that recover the final Pauli signs, but those signs are not part of the binary symplectic state used by our 19

agent. Because a Clifford operation must preserve the Pauli commutation relations, this binary matrix must satisfy the symplectic condition M T ΩM = Ω, over F2 . There are no further constraints on the binary part of the tableau: valid binary Clifford actions are precisely the elements of the symplectic group Sp(2n, F2 ). In other words, the binary part of the tableau is exactly the symplectic matrix used in the main text, written in the standard Pauli-generator basis. The identity circuit corresponds to the identity tableau. This convention also explains why circuit composition becomes matrix multiplication. If U and V are two Clifford circuits and P is a Pauli operator, then (U V )P (U V )† = U (V P V † )U † . Thus the Pauli-coordinate map for the circuit U V is obtained by composing the Pauli-coordinate maps for V and then U , which is represented by multiplying their tableaus in the corresponding order. Throughout this appendix we use the same convention as the main text: appending a generator on the right updates the tableau by right multiplication. Since rows record generator images, this means that the same local coordinate update is applied to every row of the tableau. Equivalently, if one inspects any fixed tableau row, the local binary coordinates in that row transform as follows. For the one-qubit generators, the corresponding binary updates affect only the coordinates of the acted-on qubit: Hi : (xi , zi ) 7→ (zi , xi ),

Si : (xi , zi ) 7→ (xi , xi ⊕ zi ),

with all other qubit coordinates unchanged. For the two-qubit generator CZi,j , the binary update couples only the coordinates of the acted-on qubits: CZi,j : zi 7→ zi ⊕ xj , zj 7→ zj ⊕ xi , with xi and xj unchanged and all other coordinates unchanged. Equivalently, appending CZi,j adds the X-support column of qubit j into the Z-support column of qubit i, and vice versa. These local rules are the mechanics behind tableau reduction: every action available to the agent is an indexed generator, and applying that action performs an exact sparse update of the current binary state. In this phase-free binary representation, all three generator tableaus are involutions, including MS2 = I over F2 ; this should be distinguished from the underlying quantum gate, where S 2 = Z rather than the identity. Reaching the identity tableau therefore solves the group-reduction problem described in the main text, after which reversing the applied generators recovers a circuit for the original target. D.1

Six-qubit Checkpoint Extrapolation

Figure A7 and Table A1 isolate the zero-shot behavior of the checkpoint trained only on six-qubit targets. Under the same 6n2 no-loop rollout decoder used elsewhere, this checkpoint solves every larger-circuit target in the canonical sweep: all random-walk batches from 7 to 30 qubits and all fully random Clifford batches from 7 to 30 qubits. The cost is circuit quality rather than reliability. The six-qubit checkpoint remains strong on smaller and easier transfer cells, often beating both Qiskit baselines in CZ count, but its CZ count rises sharply on the widest high-depth and fully random targets. In contrast, the continued ten-qubit checkpoint gives much lower CZ counts when it succeeds, but begins to fail on fully random targets beyond about 24 qubits.

Decoding Details

Given a trained policy and a target tableau Mtarget , greedy decoding repeatedly applies the generator with largest policy probability to the current tableau, stopping when the identity is reached or when the rollout budget is exhausted. The resulting reduction sequence is reversed to obtain a circuit for Mtarget , as in the reverse-reduction formulation of Section 4. We also use the inverse-tableau trick of −1 Gidney [19]: decode both Mtarget and Mtarget , convert the inverse-tableau solution back to a circuit for Mtarget , and keep the shorter successful circuit. For sampled policy-guided rollouts, we repeat the same reduction procedure under a fixed compute budget and retain the shortest solved circuit found. In the larger-qubit sweeps, rollouts use a no-loop safeguard that rejects actions returning to a tableau already visited in the current trajectory when an alternative action is available. 20

20 10

350 300 250 200 150 100 50 0

25q

150

20q

400 350 300 250 200 150 99% 100 50 0

99% 100 0

26q

500

400

300 90%

200

400

300

92% 300

200

64 12 8 25 6 51 2 10 24

100

17q

18q 300 250 200 150 100 50

22q

23q

500

400

300

200

99% 200

97% 200

100

99%

28q

29q

30q

800

600 73%

Initial Clifford gate count

24q

600

500

800 700 600 500 400 83% 300 200 100 0

600

500

400

27q

700

600

500

300

200

700

600

100

400

300

150

100

500

400

200

21q

250

150

100

300

200

64 12 8 25 6 51 2 10 24

19q

16q

250

200

160 140 120 100 80 60 40 20

15q

120

14q

200 175 150 125 100 75 50 25

12q 140

100

13q

64 12 8 25 6 51 2 10 24

11q

120

600

400

74%

400

200

59%

Qiskit on RL-solved 10q

64 12 8 25 6 51 2 10 24

Qiskit-AG 100

Qiskit-greedy 9q

80 70 60 50 40 30 20 10

64 12 8 25 6 51 2 10 24

8q 60

Figure A4: Full CZ-count sweep for the ten-qubit-trained model, expanding the main-text view in Figure 4 to every evaluation width from 7 to 30 qubits. Each panel fixes the evaluation qubit count and sweeps the displayed random-walk gate counts 16 through 1024, followed by the ∞ endpoint representing uniform Clifford sampling, under the same 6n2 rollout budget with the no-loop decode safeguard. Solid Qiskit curves show all-target means; dotted same-color markers and shaded gaps show the corresponding Qiskit means on the learned-solved subset when these differ. Evaluation subset Std. 7–20 q. Std. 21–30 q., d ≤ 128 Std. 21–30 q., d ≥ 256 Fully random

Cells

Succ.

6q model CZ

AG CZ

Greedy CZ

98 1.000 40 1.000 30 1.000 24 1.000

52.97 41.43 383.56 448.68

114.06 134.81 479.53 388.72

65.14 46.59 263.19 229.33

6q model lower CZ than AG / greedy 98/98 40/40 23/30 17/24

/ / / /

93/98 40/40 10/30 10/24

Table A1: Zero-shot extrapolation of the six-qubit-trained checkpoint on larger circuits. Each cell is one matched setting of target family, qubit count, and difficulty from the canonical evaluation package. Entries report mean CZ count over solved instances, and the six-qubit trained model solves all targets. The final row uses the fully random Clifford target family, which is tracked separately from the standard random-walk sweep. All learned-policy targets and success checks use the phase-free binary symplectic tableau. For the ∞ target family generated with Qiskit’s random_clifford, we store and evaluate only the returned symplectic_matrix; Pauli signs are not part of the target state. The Qiskit baselines are compared on the same target batches at the level of two-qubit cost: CZ and CX gates each count as one entangling gate, SWAP counts as three, and single-qubit, Pauli, and phase-correction gates are excluded from the reported CZ-equivalent counts. For the six-qubit Bravyi et al. benchmark in Section 5, Table A3 gives the decoding settings behind the headline exact-recovery curve. The greedy pass evaluates the target and inverse tableaus once and keeps the shorter successful circuit. The longer search uses the same target/inverse comparison, a rollout cap of 512 generator applications, the native CUDA rollout backend, and 2048 parallel rollout environments. It uses a four-arm union of temperature schedules: t40, t40→greedy, t40→t25, and t40→t12, where t40 samples at fixed temperature 4.0, t40→t12 samples with temperature linearly decreasing from 4.0 to 1.2, and t40→greedy decreases from 4.0 to 0.05. The fixed-temperature t40 arm alone reaches 990/1003 exact recoveries in the recorded first-hit trace; the annealed arms contribute the final five recoveries. Each arm runs 4096 trials per circuit for each of the target and inverse tableaus, so one arm corresponds to 1003 × 4096 × 2 = 8,216,576 sampled reductions. The reported milestone times are first-hit times from the union trace: once an optimal circuit is found for 21

800

0.8

600

0.6

RL success

Mean CZ count

1.0

RL Qiskit-greedy Qiskit-AG Qiskit on RL-solved

400

0.4

200

0.2

Number of qubits

0.0

Figure A5: Uniform-target CZ-count comparison for the ten-qubit-trained checkpoint under the same 6n2 rollout decoder with the no-loop safeguard. The benchmark uses the separate fully random Clifford target family, with 100 held-out targets at each displayed qubit count from 10 to 30 qubits, and compares the learned policy against Qiskit’s implementation of the Bravyi et al. greedy synthesizer and Aaronson–Gottesman on the same target sets. Learned means are computed over solved instances only, learned-policy markers are colored by solve rate, solid Qiskit curves show all-target means, and dotted same-color segments show Qiskit means on the learned-solved subset. a benchmark instance, later samples can still improve other instances, but the first-hit timestamp for that instance is fixed.

Reproducibility, Compute, and Assets

The accompanying code and data release contains the code, trained checkpoints, fixed target batches, recorded result tables, figure-generation scripts, and a README with exact commands for the local evaluator, ablation validation, ensemble validation, the 1003-circuit benchmark, and the basic PPO training entrypoint. The bundled evaluator can run on CPU, with optional GPU acceleration for larger sweeps. The headline six-qubit search timings are recorded in the supplied first-hit trace and summary tables; the larger-qubit evaluation tables record per-setting elapsed time for learned rollouts and Qiskit baselines. A live browser demo for the released six-qubit model, together with the recovered Bravyi et al. benchmark circuits, is hosted at y-richie-y.github.io/clifford/. The work uses synthetic Clifford targets and public benchmark data from Bravyi et al.; no humansubject or scraped personal data are used. Existing software and benchmark assets are cited where used, and the code release identifies the bundled source trees, checkpoint files, and benchmark CSVs needed to reproduce the reported experiments. The expected positive impact is improved Clifford synthesis, which may reduce two-qubit gate counts in quantum-compilation workflows. Possible negative impacts are indirect: better circuit synthesis can marginally reduce the cost of quantum experiments, including experiments whose downstream applications are outside the scope of this paper. The released assets are specialized to Clifford synthesis and do not contain sensitive data or general-purpose generative models.

Difficulty

Family

Success rate

Mean CZ

Seeds

d = 20 d = 20 d = 20 d = 20 d = 20

Ours RelTransformer [39] Transformer [45] MLP FlatMLP

1.000 1.000 1.000 1.000 1.000

6.03 ± 0.01 6.04 ± 0.01 6.06 ± 0.03 6.19 ± 0.01 6.14 ± 0.01

3 3 3 3 3

d = 100 d = 100 d = 100 d = 100 d = 100

Ours RelTransformer [39] Transformer [45] MLP FlatMLP

1.000 1.000 1.000 1.000 1.000

11.99 ± 0.04 12.06 ± 0.03 12.50 ± 0.03 13.89 ± 0.12 13.62 ± 0.16

3 3 3 3 3

d = 1024 d = 1024 d = 1024 d = 1024 d = 1024

Ours RelTransformer [39] Transformer [45] MLP FlatMLP

1.000 1.000 1.000 1.000 1.000

13.71 ± 0.06 13.79 ± 0.15 14.37 ± 0.17 16.46 ± 0.09 15.67 ± 0.15

3 3 3 3 3

d=∞ d=∞ d=∞ d=∞ d=∞

Ours RelTransformer [39] Transformer [45] MLP FlatMLP

1.000 1.000 1.000 1.000 1.000

13.49 ± 0.01 13.47 ± 0.06 14.12 ± 0.28 15.88 ± 0.05 15.19 ± 0.16

3 3 3 3 3

Table A2: Full six-qubit architecture ablation over three seeds on fixed 100-target validation sets per difficulty. Finite d is the initial Clifford gate count used to generate each random-walk target; d = ∞ uses uniformly sampled Clifford targets. Entries report mean CZ count over solved instances under rollout decoding, as mean ± standard deviation across seeds; all completed-model success rates are 1.000. Decoder stage Greedy prefilter Brief search pass First reaches prior SOTA First reaches 990 First reaches final count

Exact / total

Time

Cumulative reductions

Step cap

507/1003 610/1003 982/1003 990/1003 995/1003

0.83 s 21 s 21.6 min 51.9 min 183.1 min

2,006 – 3,410,999 8,170,010 29,143,574

512 512 512 512 512

Temperature schedules from 4.0 greedy sampled policy rollouts 4.0 4.0 four schedules: 4.0, greedy, 2.5, 1.2

Table A3: Decoding settings for the six-qubit 1003-circuit benchmark. “Reductions” counts complete sampled reduction attempts, including both target and inverse tableaus when inverse comparison is enabled. Timings are total suite time, not per-instance time, from the recorded native CUDA rollout traces on a single NVIDIA A100 80GB GPU. The brief search pass is the short max-gap-one pass reported in the main text; the full first-hit trace is recorded for the four-arm run. Hyperparameter Optimizer Training steps Parallel environments Rollout length Curriculum range Curriculum advance threshold Learning rate Batch size Epochs per update Discount factor γ GAE parameter λ Policy clip coefficient Value clip coefficient hamming_left scale Single-qubit gate cost Terminal bonus

Value PPO 1.5 × 109 2048 256 difficulties 1–1000 success = 1.00 2.5 × 10−4 8192 5 0.99 0.95 0.15 0.2 0.5 0.01 25

Table A4: Training hyperparameters for the main 6-qubit Clifford run.

64 initial gates

128 initial gates

256 initial gates

Mean CZ count

400 300 200 100 0

9 11 13 15 17 19 21 23 25 27 29

Number of qubits

9 11 13 15 17 19 21 23 25 27 29

Number of qubits

512 initial gates

9 11 13 15 17 19 21 23 25 27 29

Number of qubits

1024 initial gates

Mean CZ count

800 600 400 200 0

9 11 13 15 17 19 21 23 25 27 29

Number of qubits

0.0

9 11 13 15 17 19 21 23 25 27 29

Number of qubits

0.2

0.4

RL success

0.6

Number of qubits

0.8

1.0

Figure A6: CZ-count comparison for the ten-qubit-trained model across the harder displayed randomwalk gate counts 64, 128, 256, 512, and 1024, together with the ∞ setting for uniform Clifford targets, under the same 6n2 rollout budget with the no-loop decode safeguard. Each panel plots mean CZ count against evaluation qubit count on the full sweep from 7 to 30 qubits. Solid Qiskit curves show all-target means; dotted same-color markers and shaded gaps show learned-solved-subset means where the learned policy fails.

50 40 30 20 10

350 300 250 200 150 100 50 0

25q

20q

400 350 300 250 200 150 100 50 0

26q

600

22q

200 150 100 50

23q

400

300

200

100

300 200 100 0

28q

29q

1000

1200

1400

1750

800

1000

1200

1500

1000

1250

800

1000

600

750

400

500

800 600

400

400 0

64 12 8 25 6 51 2 10 24

250

200

64 12 8 25 6 51 2 10 24

200

Initial Clifford gate count

24q

500

400

300

0 600

500

200

27q

250

200

64 12 8 25 6 51 2 10 24

400

600 400

100

18q 300

150

100

500

300

17q

300

200

21q 400

800

700 600 500 400 300 200 100 0

250

150

100

19q

16q

200 150

250

200

100

160 140 120 100 80 60 40 20

15q

14q

200 175 150 125 100 75 50 25

120

12q 140

100

64 12 8 25 6 51 2 10 24

13q

11q

120

64 12 8 25 6 51 2 10 24

Qiskit-AG 10q

100

Qiskit-greedy

RL 9q

80 70 60 50 40 30 20 10

30q

8q 60

64 12 8 25 6 51 2 10 24

Figure A7: Full CZ-count sweep for the six-qubit-trained model, using the same layout as Figure A4. Each panel fixes the evaluation qubit count and sweeps the displayed random-walk gate counts 16 through 1024, followed by the ∞ endpoint representing uniform Clifford sampling, under the same 6n2 rollout budget with the no-loop decode safeguard. The checkpoint solves every target shown; the figure therefore directly compares mean CZ count against the two Qiskit baselines on the same target batches. 24

Record · ID 175243 · SHA-256 f16bd765c9088589
Retrieved via Conceptio — every document is proof-bundled with source, license, and retrieval metadata.