Toward Quantum Software Automation: A Quantum-Aware Harness for LLM-Guided Evolution

Toward Quantum Software Automation: A Quantum-Aware Harness for LLM-Guided Evolution Lily Jiaxin Wan, Deming Chen, Klara Nahrstedt, and Bo Chen University of Illinois Urbana-Champaign Urbana, Illinois, USA

arXiv:2609.38883v1 [cs.SE] 30 Sep 2026

Quantum software is critical for improving the efficiency and reliability of scarce quantum hardware. However, its design still relies heavily on ad-hoc, handcrafted heuristics that are often suboptimal and quickly become obsolete as quantum hardware evolves. LLM-guided evolutionary search offers a promising way to automatically explore complex software designs, but existing search frameworks lack the quantumspecific support needed for efficient evolution: verification is expensive, feedback is sparse, and heterogeneous quantum programs require different optimization objectives. In this paper, we present QSA, a quantum-aware harness for LLM-guided evolutionary search toward automating quantum software design. QSA equips the search with three forms of quantum-specific guidance: an evolution-hardness-guided coreset and approximate scoring to reduce verification cost, static and snapshot analyses to provide fine-grained execution context, and task-specific rewards for compiler passes and runtime policies. We evaluate QSA on the IBM Quantum platform across three benchmark suites. For multiprogramming, QSA improves QPU utilization by 4.2%–9.5% and Hellinger fidelity by 15.2%–19.5% over the state of the art. For error mitigation, QSA reduces mitigation time by at least 96.8% while achieving comparable or better fidelity. These gains require only $6.9 in LLM API cost over 11.3 hours.

1

Introduction

Advances in quantum computing have enabled quantumas-a-service through cloud platforms [2, 25, 26, 36, 47, 49]. As shown in Figure 1, the scale of quantum hardware, measured by the number of physical quantum bits (qubits), has grown rapidly in recent years. For instance, IBM increased its physical qubit count by two orders of magnitude from 2021 to 2023. These advances bring quantum applications closer to reality [28]. A quantum system provides such a service by executing quantum circuits and returning classical outcomes (e.g., bitstring probabilities) through sampling. Figure 2 illustrates a 3-qubit GHZ circuit [22] and its output. Quantum software is crucial to modern quantum computing. It spans compiler passes that transform quantum circuits before execution and runtime policies that determine how circuits are scheduled on QPUs. In this work, we study two representative components: error mitigation as a compiler pass and multiprogramming as a runtime policy. Designing such software, however, is costly. First, the design space is

103

Qubit Count

Abstract

1121

102 53 27

IBM

Google 105 IonQ 36 80 56 64 RGTI 32 20 QTM

2020

2022 2024 Year

2026

Figure 1. Physical qubit count of major providers. enormous. For example, multiprogramming must decide how circuits are grouped for concurrent execution. Partitioning (n) quantum programs into pairs, where (n) is even, already yields 2𝑛/2𝑛!𝑛 ! possible combinations; even ten programs ad(2) mit 945 different groupings. Second, evaluating a design typically requires costly quantum simulation or hardware execution, making the design space difficult to model and explore efficiently. Finding a good design therefore requires extensive trial and error. As a result, existing quantum software often relies on ad-hoc, handcrafted heuristics (e.g., brute-force search and empirical hyperparameter choices), which can be substantially suboptimal. For instance, our study shows that an ad-hoc multiprogramming approach [19] achieves 45.7% lower fidelity than the offline optimal. Moreover, rapid hardware evolution (Figure 1) requires heuristic retuning. A natural direction is to automate this design process through parameter search. Traditional optimization techniques, such as Bayesian optimization [18, 54], reinforcement learning [37, 65], and program synthesis [23, 55], typically operate over predefined search spaces, limiting structural code exploration. Recent LLMs provide a more flexible search operator because they can directly generate and revise source code [12, 33, 42]. This has led to LLM-guided evolutionary search [11, 39, 51], where an LLM repeatedly mutates a program, verifies the generated candidate on the target system, and uses the observed reward to guide subsequent mutations. By incorporating system feedback into later iterations, the search can progressively refine both program structure and decision logic. However, existing frameworks provide only a generic evolutionary loop and lack the quantum-specific mechanisms needed to interact effectively with quantum systems, making them inefficient. This paper presents QSA, a quantum-aware harness for LLM-guided evolutionary search for quantum compiler and runtime optimization. QSA equips the search agent with quantum-specific mechanisms that determine what to verify and how to score it, what execution context to expose, and how

Qubit

∣0⟩ :

1-qubit Gate ∣0⟩ : 2-qubit Gate

∣0⟩ :

H X X

Measurement

Lily Jiaxin Wan, Deming Chen, Klara Nahrstedt, and Bo Chen Bitstring

Prob.

000

0.5

001

0

…

…

110

0

111

0.5

Quantum Circuit

Error Mitigation

Quantum Software Stack Multiprog Layout ramming Mapping

Heterogeneous Classical Resources

Figure 2. An example GHZ [22] quantum circuit. The circuit consists of three qubits, one 1-qubit gate, and two 2-qubit gates. The measurement module produces classical outcome of 3-bit bitstrings with different probability.

Classical Outcome Interface

Neutral Atoms

Error Correction

Heterogeneous Quantum Resources

Supercon Trapped ducting Ions Quantum Hardware Stack

Photonic

Figure 3. Quantum hybrid computational model. to reward heterogeneous quantum programs. Specifically, QSA addresses the following three challenges. Challenge 1: Expensive quantum-system verification (§4). Verification lies on the critical path of evolutionary search: each candidate program must be evaluated across multiple quantum circuits. Unfortunately, key metrics such as Hellinger fidelity [7] require costly QPU execution or quantum-circuit simulation, making verification cost prohibitive. Our key idea is to reduce both what to verify and how to score it. First, we develop evolution-hardness-guided coreset extraction. It repeatedly evolves programs using uniformly sampled verification sets, evaluates the resulting programs across circuit scales, and identifies the scale with the lowest post-evolution reward. We then sample circuit types at this challenging scale to construct the coreset. Second, we replace expensive fidelity evaluation during evolution with lightweight proxy scores derived from circuit properties. These techniques substantially reduce verification overhead while preserving effective guidance for evolution. Challenge 2: Sparse feedback for LLM-guided search (§5). Generic evolutionary search provides the LLM with program code and final rewards, but little quantum-specific context about the workload or how the program affects it during execution. Such sparse feedback makes it difficult for the LLM to understand why a mutation succeeds or fails and to generate targeted improvements. Our key idea is therefore to expose quantum-specific execution context. First, static analysis summarizes quantum-circuit characteristics before execution. Second, snapshot analysis captures intermediate system states as the evolved program executes. These signals connect circuit characteristics and program decisions to execution outcomes, enabling more informed mutations and more efficient search. Challenge 3: Heterogeneous programs in quantum software (§6). Quantum software programs can be broadly categorized according to how they interact with quantum circuits. Compiler passes directly transform the circuit, whereas runtime policies leave the circuit unchanged and instead optimize its execution on quantum hardware. Although quantum metrics such as fidelity can directly evaluate compiler passes, they are inadequate for runtime policies because they capture policy quality only indirectly. Our key idea is therefore to tailor how to reward different quantum programs. For compiler passes, we derive rewards directly from quantum-system

metrics. For runtime policies, we design a policy reward that jointly captures Pareto optimality and global ranking qualities. We measure Pareto optimality across conflicting objectives using ranks from non-dominated sorting [15], and generalizability using the correlation between the policy’s predicted ranks and the corresponding ground-truth ranks. We implement QSA on a Linux server with an AMD Ryzen 9 CPU and evaluate it using the IBM Quantum platform and public LLM APIs. For multiprogramming, QSA improves QPU utilization by 4.2%–9.5% while increasing fidelity by 15.2%–19.5% over the state of the art. For error mitigation, QSA reduces execution time by at least 96.8% while achieving comparable or better fidelity. The evolved programs further maintain consistent performance across circuit types and scales. These gains incur only $6.9 in LLM API cost over 11.3 hours. Overall, we make the following contributions: 1. We present QSA, a quantum-aware harness that enables LLM-guided evolutionary search to automatically optimize quantum compiler passes and runtime policies. 2. We design three quantum-specific harness mechanisms: an evolution-hardness-guided coreset and approximate scoring for efficient verification, a quantum-aware prompter for fine-grained search feedback, and task-specific rewards for heterogeneous quantum programs. 3. We evaluate QSA on the IBM Quantum platform and demonstrate significant improvements in QPU utilization, fidelity, and mitigation time over SOTA systems.

2

Background and Motivation

2.1

Quantum Background

We introduce the quantum circuit and the quantum hybrid computational model, followed by the key metric. Quantum circuit. A quantum circuit, often used to represent a quantum program, is a fundamental computational abstraction executed on quantum hardware. For example, Figure 2 shows a GHZ circuit operating on three qubits, the quantum counterparts of classical bits. Unlike a classical bit, which takes a value of either 0 or 1, a qubit can be prepared in a superposition of |0⟩ and |1⟩. In this circuit, a Hadamard (𝐻 ) gate [52] places the first qubit into superposition. The subsequent two-qubit controlled-NOT (CNOT or CX) gates [44] propagate this superposition across the remaining qubits,

347.7

101

100 12

Qubits

100

1.11 0.64

3.40

GV WC

10−1 15

24

Figure 4. Fidelity results.

60.06

101

20

Figure 5. Cut time analysis.

𝑖

𝐹𝐻 (𝑝, 𝑞) ∈ [0, 1], where 𝐹𝐻 = 1 indicates perfect agreement. This metric captures the cumulative impact of noise, gate errors, and decoherence on quantum hardware. We refer to it as fidelity for brevity. Why Quantum Software Design Matters?

Quantum software is critical for addressing two major limitations of modern quantum computing: resource scarcity and computation noise.

QOS MP Oracle

60

70

0.8

80

90

100

Normalized Utilization (%)

Circuit Size (# of Qubits)

creating a three-qubit entangled state. Measurement maps the quantum state to a classical bitstring, with each possible outcome occurring according to its measurement probability. Repeated circuit executions, commonly called shots, are used to estimate the resulting probability distribution. Ideally, the √ GHZ circuit prepares (|000⟩ + |111⟩)/ 2; therefore, only 000 and 111 have nonzero probabilities, each equal to 0.5. More generally, the state of an 𝑛-qubit system is represented in a 2𝑛 -dimensional state space, allowing quantum algorithms to exploit superposition, interference, and entanglement for certain computational problems. Hybrid computational model. Modern quantum computing follows a hybrid execution model that tightly couples quantum hardware with quantum software, as illustrated in Figure 3. Quantum hardware is realized using four major technologies: neutral atoms [50], trapped ions [9], superconducting circuits [30], and photonic systems [53]. It executes a circuit by applying a sequence of quantum gates to qubits through physical control operations, evolving the quantum state until measurements produce classical outputs. Quantum software compiles programs by mapping circuits onto the target QPU layout [29], and mitigates noise either before execution [5, 19, 56] or after execution [32, 58] using classical computing resources such as FPGAs, GPUs, and CPUs. In this hybrid computational model, a user submits the design of a quantum circuit, the quantum hardware and software collaborate to output classical outcomes. Quantum system metric. We evaluate circuit execution quality on NISQ hardware using Hellinger fidelity, which measures the similarity between the ideal output distribution and the noisy distribution observed from hardware execution. Let 𝑝 = {𝑝𝑖 } denote the ideal distribution and 𝑞 = {𝑞𝑖 } the measured distribution. The Hellinger fidelity is defined as !2 ∑︁ √ 𝐹𝐻 (𝑝, 𝑞) = 𝑝 𝑖 𝑞𝑖 . (1)

2.2

0.75 0.50 0.25 0.00

Fidelity

Frozen CutQC QOS

Fidelity

102

Search Time (s)

Relative Fidelity

Toward Quantum Software Automation: A Quantum-Aware Harness for LLM-Guided Evolution

0.6 0.4

45.7%

QOS MP Oracle

80-85 85-90 90-95 95-100

Normalized Utilization (%)

Figure 6. Existing multiprogramming methods achieve poor trade-offs of fidelity and utility. Resource scarcity. Due to their high cost and system complexity, QPUs remain extremely scarce resources. Today’s public quantum clouds (e.g., IBM Quantum [25], AWS Braket [2], IonQ [26], and Quantinuum [47]) expose only a limited number of QPUs and typically execute one circuit per QPU at a time to reduce interference. This practice constrains the throughput of quantum services. Our measurements on IBM Quantum further reveal severe queueing: for example, the 99.9th-percentile queue length of IBM_fez reaches 39,985 jobs. Detailed QPU availability and queue statistics are provided in Appendix A. To improve throughput, recent works explore multiprogramming [10, 14, 19, 57], which bundles compatible circuits from the same workload pool and executes them concurrently by spatially partitioning a shared QPU. Although circuits within a bundle remain logically independent, concurrent execution can substantially improve hardware utilization. Computation noise. Quantum circuits execute on Noisy Intermediate-Scale Quantum (NISQ) devices [46], which are inherently susceptible to noise from quantum gates, measurements, and decoherence. Our measurements further show that these error rates vary across QPUs; detailed backend statistics are provided in Appendix A. To improve reliability, error mitigation techniques transform noisy circuits into executions that are less susceptible to hardware errors [5, 19, 56]. For example, circuit-cutting-based methods decompose a large circuit into smaller fragments, reducing interference and shortening individual QPU executions. Figure 4 compares FrozenQubits, CutQC, and QOS, normalized to Qiskit. Boxplots show medians and quartiles, with circles marking outliers. QOS achieves the highest fidelity, improving relative fidelity by up to 347.7×, demonstrating the importance of effective error-mitigation software. 2.3

Pitfalls in Quantum Software Design

Due to the high design cost, quantum software is often adhoc and built around handcrafted heuristics. These heuristics restrict how the large design space is explored and can lead to either suboptimal decisions or excessive search overhead. Multiprogramming. Multiprogramming determines which circuits are bundled for concurrent execution on a shared QPU [14, 19]. Existing methods rank feasible bundles using manually designed scoring functions whose components and weights approximate the fidelity–utilization trade-off.

Lily Jiaxin Wan, Deming Chen, Klara Nahrstedt, and Bo Chen

Because this fixed scoring rule explores only a restricted set of trade-offs, the selected bundles can remain far from the offline optimum. Figure 6 compares MP [14] and QOS [19] with an offlineoptimal Oracle on IBM_Torino. Normalized utilization characterizes how effectively a bundle uses QPU resources across space and time [19]. In Figure 6 (left), the top 10% bundles selected by each method for 24-qubit workloads are highlighted. Oracle selects bundles near the top-right corner, whereas MP favors central bundles and QOS improves utilization at the cost of fidelity. Figure 6 (right) further reports the average fidelity in each utilization range. At 85%–90% utilization, QOS achieves 45.7%, lower fidelity than Oracle, demonstrating the suboptimality of handcrafted heuristics. Error mitigation. A key design choice in circuit-cuttingbased error mitigation is the maximum fragment size [38, 43]. Smaller fragments can improve fidelity, but increase both partitioning cost and the number of QPU executions. Existing methods therefore sweep candidate fragment sizes until finding one that satisfies the execution budget. Because each candidate requires solving a costly partitioning problem, only a limited portion of the design space can be explored within a practical time budget. Figure 5 shows the circuit-cut search time of gate virtualization (GV) [38] and wire cutting (WC) [43] for QAOA-R3 as the input circuit grows, with the target fragment size fixed at 7 as in QOS [19]. WC takes 1.11 seconds at 16 qubits, but reaches the 60-second search limit at 18 qubits and remains capped thereafter. GV grows more gradually, but its search time increases from 0.64 seconds at 20 qubits to 3.40 seconds at 22 qubits and over 10 seconds at 24 qubits. This rapidly increasing cost makes handcrafted fragment-size exploration inefficient for larger circuits. Listing 1. LLM-based Evolution Engine codebase. add (𝜃 0 , 𝑟 0 ) while not converged : Θ = codebase. sample () prompt = prompter. build (Θ) 𝜃 = llm. generate ( prompt ) r = verifier. execute (𝜃 ) codebase. add (𝜃 , r)

Preliminary Study. We first apply this generic workflow to error-mitigation software using OpenEvolve [51] and Gemini-3-Flash [20]. Across 100 iterations, evolution discovers three reward-improving programs, reducing runtime by up to 19.9%, demonstrating the potential of LLM-guided evolution for quantum software. However, directly applying the generic loop exposes three limitations discussed in §1: verifying each mutation requires costly quantum simulation or execution, the LLM receives little quantum-specific context beyond final rewards, and different quantum software components require different reward formulations. These limitations motivate QSA’s designs introduced next.

3

Figure 7 illustrates how QSA augments a generic LLM-guided evolutionary search agent and interfaces with a hybrid quantum– classical cloud. The quantum-aware harness provides domainspecific mechanisms for verification, prompting, and reward design, while the underlying evolution agent performs generic program selection and mutation. The optimized quantum software is then integrated into the quantum software stack to control quantum execution and return classical outcomes to user applications. Within the evolution loop, the harness provides three forms of quantum-specific support. First, the verifier determines what to verify and how to score it: coreset extraction reduces the circuits used for verification (§4.1), while proxy scoring replaces expensive fidelity evaluation with lightweight proxy scores derived from circuit properties (§4.2). Second, the prompter determines what context to expose: static analysis summarizes circuit properties (§5.1), while snapshot analysis captures intermediate system states during execution (§5.2). Third, task-specific rewards determine how to evaluate heterogeneous quantum programs, using quantum metrics for circuit optimization (§6.1) and policy-specific rewards for runtime policies (§6.2). Together, these components form a closed feedback loop: the verifier provides lowcost measurements, the prompter exposes quantum-specific execution context, and the rewards convert evaluation results into guidance for subsequent mutations.

4 2.4

LLM-Guided Evolutionary Search

Recent LLM-guided evolutionary frameworks use LLMs as mutation operators to iteratively generate and refine program code [39, 51]. As shown in Listing 1, a generic evolution loop consists of a codebase, a prompter, an LLM, and a verifier. Starting from an initial program 𝜃 0 , each iteration samples prior programs from the codebase, constructs an evolution prompt, generates a candidate program 𝜃 , and executes it to obtain reward 𝑟 , which is added back to the codebase to guide subsequent search.

System Design

Efficient Quantum Verifier

We reduce quantum verification cost along two dimensions: selecting a compact verification set through evolution-hardnessguided coreset extraction, and replacing expensive fidelity evaluation with lightweight approximate scoring. 4.1

Evolution-Hardness-Guided Coreset Extraction

Ideally, every candidate program should be verified on the complete test suite. However, full-suite verification is prohibitively expensive and substantially slows evolution. We

Toward Quantum Software Automation: A Quantum-Aware Harness for LLM-Guided Evolution

Efficient Quantum Verifier §4.1 Coreset What to verify Extraction §4.2 Proximate Scoring

Quantum-aware Prompter §5.2 Snapshot §5.1 Static Analysis Analysis Circuit Property System State

Execution Feedback Hybrid QuantumClassical Cloud

Generic LLM-guided Evolution Codebase

Prompt Composition

LLMs

Candidate Program

Execute/ Verify Reward Guidance Task-specific Reward §6.1 Circuit §6.2 Policy Pareto Rank/ Reward Reward Correlation

How to score Quantum Metrics

Optimized Quantum Software

User Frontend

Quantum Hardware

Quantum Application

Quantum Software

Classical Outcome

Figure 7. QSA’s quantum-aware harness around generic LLM-guided evolution, integrating efficient verification, quantumaware prompting, and task-specific rewards into a closed evolution loop. Reward

Reward

0.36 0.25 -0.08 0.00 −0.25

0 −1 −2

Sampled Evolution

Full Evaluation

12 14 16 18 20 22 24 # Qubits

Figure 8. Random selection performance.

Figure 9. Post-evolution reward across circuit scales.

therefore use a compact verification set to score candidate programs during evolution, and evaluate only the final evolved program on the complete suite. A natural baseline is to uniformly sample circuits from the full suite. However, such a sample may underrepresent cases that remain difficult for evolution. To illustrate, we conduct an error-mitigation experiment in which uniformly sampled circuits are used throughout evolution with Gemini3-Flash [20]. We defer the reward design to §6. As shown in Figure 27, the final program achieves a reward of 0.36 on the sampled verification set, but only −0.08 on the complete test suite. This gap indicates that evolution can overfit to the sampled circuits. Our key insight is to use the behavior of evolution itself to identify challenging circuit scales. Specifically, we uniformly sample circuits from the full suite and use the sampled set throughout an evolution run. We then evaluate the evolved program separately at each qubit count. We repeat this process across random samples to reduce sampling variance. We call the resulting scale-wise difficulty evolutionary hardness: a lower post-evolution reward indicates that uniformly guided evolution performs more poorly at that scale, making it a more challenging target for verification. Formally, let S (𝑘 ) denote the 𝑘-th uniformly sampled verification set, and let 𝑅𝑞 (𝜃 ) denote the reward of program 𝜃 on circuits with qubit count 𝑞. We select

𝑞 ∗ = arg min 𝑞

𝐾  1 ∑︁  𝑅𝑞 Evolve(S (𝑘 ) ) , 𝐾 𝑘=1

(2)

where 𝐾 is the number of repeated random samples. As shown in Figure 9, the evolved programs achieve the lowest reward on 22-qubit circuits, identifying this scale as the most evolutionarily challenging. We then construct the final coreset by sampling from all available circuit types at the selected qubit count. Unlike selecting a few globally hardest circuits, this group-level design preserves circuit diversity while focusing verification on a challenging scale. As shown in Figure 19, focusing evolution on this hard scale does not sacrifice performance across other circuit types or qubit counts. 4.2

Evolution-Oriented Approximate Scoring

Coreset extraction reduces the number of verification cases, but evaluating each case remains expensive when fidelity must be obtained through simulation or real QPU execution. As shown in Figure 10 (top), simulating a 12-qubit circuit takes less than 10 seconds, whereas a 24-qubit circuit can be nearly three orders of magnitude slower. Real QPU execution additionally incurs monetary cost and access delays. Our key insight is that evolution does not require exact fidelity for every candidate program. It instead needs a low-cost signal that distinguishes promising candidates and guides the search. We therefore replace exact fidelity evaluation during evolution with lightweight proxies derived from circuit properties. For individual circuits, we use depth and CNOT count: depth captures the critical-path length, while CNOT count reflects the use of error-prone two-qubit gates. Prior work has shown both to be useful fidelity indicators [5, 24, 63]. Figure 11 further shows a moderate negative correlation between depth and fidelity (−0.605). Although these proxies do not predict fidelity exactly, they are inexpensive to compute (Figure 10, bottom) and provide effective feedback for evolution. For bundles that execute multiple circuits concurrently, absolute depth or CNOT count alone does not capture how work is distributed between the sub-circuits. We therefore use their relative balance as a bundle-level proxy. For a circuit property 𝑣 (e.g., depth or CNOT count), we define

Lily Jiaxin Wan, Deming Chen, Klara Nahrstedt, and Bo Chen

0.5

CDF

1.0 0.5

Simulation 12-qubit 24-qubit

Proxy Calculation 12-qubit 24-qubit

104 10−2 100 102 Simulation/Calc Time (s)

Fidelity

CDF

1.0

0.5

0.0 Pearson r = -0.605

0

100

200 Depth

300

Figure 10. Simulation using Figure 11. Correlation of depth and fidelity. Qiskit.

𝑟 (𝑣 1, 𝑣 2 ) = min{𝑣 1, 𝑣 2 }/max{𝑣 1, 𝑣 2, 1},

(3)

where 𝑣 1 and 𝑣 2 are the property values of the two bundled circuits. The score lies in [0, 1], with a smaller value indicating greater imbalance. These lightweight, bundle-aware proxies replace expensive fidelity evaluation and substantially accelerate verification during evolution.

5

Quantum Circuit Static Analysis

Circuit layouts alone provide limited information about their execution characteristics. We therefore statically analyze each circuit and summarize it as a compact metadata profile. The profile combines basic statistics from Qiskit [1] with DAG-derived features from SupermarQ [59], including circuit width, depth, and CNOT count. Width captures the number of required qubits, depth approximates the critical execution path, and CNOT count reflects the use of noisesensitive two-qubit operations [6]. For example, the GHZ circuit in Figure 2 has width 3, depth 3, and two CNOT gates. These features provide the LLM with a concise description of circuit scale, structure, and noise sensitivity. By relating the metadata to program rewards and execution behavior, the LLM can identify workload-dependent patterns and develop more effective optimization heuristics. Appendix B lists the complete feature set. 5.2

Snapshot Enabler Atomized Operators

System State Call

State Collection

Snapshot-enabled Program to Eval Operators

Figure 12. Snapshot Analysis We address this limitation through snapshot analysis. As illustrated in Figure 12, we instrument key operators that modify quantum-related metrics and record the program state after each operator completes. When an evolved program invokes these operators, the collected snapshots expose intermediate circuit properties and execution outcomes at multiple points in the workflow. This step-by-step context allows the LLM to associate reward changes with specific program decisions, localize ineffective transformations, and generate targeted mutations.

Quantum-Aware Prompter

The base prompt in OpenEvolve [51] provides the LLM with program code and reward feedback, but little information about the quantum workloads being optimized or how a program changes them during execution. This semantic gap makes it difficult for the LLM to diagnose failures and generate meaningful mutations. Our key insight is to expose both what circuits the program operates on and how their states evolve during execution. We therefore augment the prompt with two complementary forms of quantum-aware context: static circuit profiles and execution snapshots. 5.1

Snapshot

Qiskit QOS CutQC FrozenQubits

Quantum Software Snapshot Analysis

Static profiles describe the input circuits, but do not reveal how an evolved program transforms them. A final reward only indicates whether a program performs well but not where execution begins to degrade or which operation causes the change.

6

Task-Specific Reward Design

We select the reward formulation according to how the target program interacts with quantum circuits. 6.1

Circuit-level Reward

The circuit-level reward evaluates programs that directly modify quantum circuits (e.g., error mitigation
and compilation). Let 𝑀 ( 𝑗) = 𝑚 1 ( 𝑗), 𝑚 2 ( 𝑗), . . . , 𝑚𝑛 ( 𝑗) denote the measured system metrics of job 𝑗, such as fidelity, runtime, or cost. Because these metrics have different scales and optimization directions, we first normalize each metric into a dimensionless utility 𝑚˜ 𝑖 ( 𝑗), where a larger value consistently indicates better performance. We then aggregate the normalized utilities using a weighted reward: 𝑛 ∑︁ 𝑟 circuit ( 𝑗) = 𝑤𝑖 𝑚˜ 𝑖 ( 𝑗), (4) 𝑖=1

where Í 𝑤𝑖 ≥ 0 specifies the relative importance of metric 𝑖 and 𝑖 𝑤𝑖 = 1. 6.2

Policy Reward

The policy reward evaluates programs that optimize how circuits are executed on hardware (e.g., multiprogramming and scheduling). We design the policy reward using two components: Pareto rank, which captures Pareto optimality of conflicting objectives, and policy correlation, which encourages generalization of the policy across system settings. Pareto Rank. The Pareto optimality ensures the optimality across different trade-offs among conflicting objectives. In multiprogramming, for example, fidelity and utilization may be weighted differently depending on system load: under overload, the scheduler may prioritize utilization to sustain throughput, whereas under light load, it may favor fidelity to improve user experience. A fixed weighted score is often too rigid to capture such varying preferences.

Toward Quantum Software Automation: A Quantum-Aware Harness for LLM-Guided Evolution

We therefore evaluate each candidate job using Pareto dominance. Let J denote the set
of candidate jobs, and let 𝑀 ( 𝑗) = 𝑚 1 ( 𝑗), 𝑚 2 ( 𝑗), . . . , 𝑚𝑛 ( 𝑗) denote the metric vector of job 𝑗 ∈ J (higher the better). Job 𝑗 ∗ Pareto dominates job 𝑗 if it is no worse in every objective and strictly better in at least one (i.e., ∀𝑖 ∈ [1, 𝑛], 𝑚𝑖 ( 𝑗 ∗ ) ≥ 𝑚𝑖 ( 𝑗) and ∃𝑘 ∈ [1, 𝑛], 𝑚𝑘 ( 𝑗 ∗ ) > 𝑚𝑘 ( 𝑗)). Based on this relation, we compute Pareto fronts by iterative non-dominated sorting. Ð𝑡 −1
The 𝑡-th front is defined as F𝑡 = Front J \ 𝑖=1 F𝑖 , where F1 contains all non-dominated jobs, F2 contains the non-dominated jobs after removing F1 , and so on. We then define the Pareto-rank reward of job 𝑗 as 𝑟 pareto ( 𝑗) = 𝑡,

if 𝑗 ∈ F𝑡 .

(5)

Lower rank is better. By favoring jobs with smaller Pareto rank, the evolved policy is encouraged to remain close to the non-dominated frontier. Policy Correlation. Pareto optimality alone is insufficient: the policy should also generalize across scheduling settings. In practice, a policy selects only a subset of jobs from a candidate pool, so rewarding only the final outcome can overfit the policy to a particular workload composition. Our key insight is to evaluate how well the policy’s decisions align with the ground truth. Let 𝑝 ( 𝑗) denote the measure assigned by the policy to job 𝑗 (e.g., its predicted Pareto rank), and let 𝑔( 𝑗) denote the corresponding ground-truth measure (e.g., its offline-optimal rank). We define the policycorrelation reward as
𝑟 corr = corr 𝑝 ( 𝑗), 𝑔( 𝑗) , (6) where corr(·, ·) is the Pearson correlation over all candidate jobs in J . A correlation close to 1 indicates that the policy closely reproduces the ground-truth ordering. Unlike outcome-based rewards computed only over selected jobs, this provides a denser signal that encourages generalizable policy decisions. Finally, we combine Pareto rank and policy correlation: 1

𝑟 policy = 𝛼

+ (1 − 𝛼)𝑟 corr,

(7)

𝑟¯pareto where 𝑟¯pareto is the average Pareto rank of the selected jobs and 𝛼 ∈ [0, 1] controls the relative importance of Pareto optimality and generalization. We use 1/¯𝑟 pareto ∈ (0, 1] to convert the lower-is-better Pareto rank into a higher-is-better reward term. The resulting reward favors policies that select jobs near the Pareto frontier while preserving decision orderings intended to generalize across workload conditions.

7

Implementation

Evolution Engine. We run the evolution loop on a Linux server with 32-core AMD Ryzen 9 CPU @ 4.95 GHz based on OpenEvolve [51]. LLM inference is performed via external API calls, and verification runs single-threaded. To ensure

system efficiency in the error mitigation, we enforce a 60second execution deadline per circuit. We conduct three independent 100-iteration trials per setting. The reported performance is averaged across the three runs, and the error bars represent ± one standard deviation. Reward. We tailor the reward function to each task’s requirements. For multiprogramming, we use the policy reward (Equation 7), averaging the Pareto-rank term over the top 1% candidate bundles. We set 𝛼 = 0.5 and use the depth ratio as a lightweight fidelity proxy when computing Pareto ranks. For error mitigation, we instantiate the circuit-level reward (Equation 4) using depth, CNOT count, and runtime. Specifically, 𝑚˜ 𝑑 , 𝑚˜ 𝑐 , and 𝑚˜ 𝑡 denote the normalized depth, CNOT-count, and runtime metrics, respectively, each averaged across benchmarks and direction-aligned so that larger values are better. We use relative weights 1 : 1 : 14 for 𝑚˜ 𝑑 : 𝑚˜ 𝑐 : 𝑚˜ 𝑡 , prioritizing circuit-complexity reduction while still rewarding lower runtime. Prompter. To profile program behavior, we extract static features from Qiskit [1] and Supermarq [59], encompassing eight structural metrics (e.g., depth, gate counts) and six communication-centric properties (e.g., entanglement ratio, parallelism). The snapshot analysis includes 1) the cost search trace for 22-qubit circuits in error mitigation and 2) the Pareto rank and system metrics of top 1% circuit bundles from the pool for multiprogramming. See Appendix C for complete prompt examples.

8

Evaluation

Our evaluation answers the following research questions: 1. How effectively can QSA automate quantum software design in system metrics? (§8.2) 2. How robust is QSA? (§8.3) 3. Where does QSA’s performance gain come from? (§8.4) 4. How does each harness component contribute? (§8.5) 8.1

Experimental Setup

Quantum Hardware. Experiments are conducted on IBM Heron processors [25] to evaluate circuit performance. We evaluate QSA on the 133-qubit Torino and 156-qubit Marrakesh QPUs using 8,192 measurement shots per execution. Metrics. We evaluate the following metrics: (1) Hellinger Fidelity: A score from 0 to 1 measuring the similarity between experimental results and the noise-free ideal (Equation 1). Relative fidelity represents the raw fidelity normalized by Qiskit results. (2) Normalized Effective Utilization: Following QOS [19], normalized utilization is the combined spatial and temporal utilization of a bundled execution. It accounts for how many qubits are occupied and how long they are occupied when bundled circuits have different runtimes. For brevity, we refer to this metric as normalized utilization. (3) Mitigation Time: The latency incurred during the preprocessing of large circuits into smaller ones for improving

0.8 0.7

QOS MP QSA Oracle

0.6 0.5 85

90

95

Relative Gain (%)

B

0.9

Fidelity

r

e ett

100

30 vs QOS Fid. 25 vs MP Norm. Util. +19.5 20 +16.6 +15.9 +13.7 15 +15.2 +13.3 +9.5 +10.3 10 +5.9 +4.2 +3.7 5 +1.3 0 8 16 24

Mitigation Multiprog

0

Cost (USD)

Time (hours)

10

10.0 7.5

Mitigation Multiprog

5.0 2.5

i Pro dex t 4.6 s 4.6 lash min ini 3 i 3 F GPT-5 -5.3 Co Sonne e Opu Gem Gemin GPT laude Claud C

0.0 i Pro dex t 4.6 s 4.6 lash min ini 3 ini 3 F GPT-5 -5.3 Co Sonne e Opu m e G GPT laude Claud Gem C

(a) Time Cost.

(b) Monetary Cost.

24-qubit Circuits

QSA QOS CutQC Frozen Be tte r

QSA 97.0% lower 1.0

Normalized Utilization (%) Total Qubit Count Figure 13. Multiprogramming result. 20

12-qubit Circuits

60 50 40 30 20 10 0

1.1

1.2

1.3

Relative Fidelity

1.4

Mitigation Time (s)

1.0

Mitigation Time (s)

Lily Jiaxin Wan, Deming Chen, Klara Nahrstedt, and Bo Chen

QSA QOS CutQC Frozen Be tte r

60 40 20 0

QSA 96.8% lower 0

100

200

300

Relative Fidelity

Figure 14. Error mitigation result. Mutation Operator. Since the evolution requires an LLM engine for mutation, we consider a diverse selection of commercial LLM families: (1) Gemini 3 Pro [21], (2) Gemini 3 Flash [20], (3) GPT-5-mini [40], (4) GPT-5.3-Codex [41], (5) Claude Sonnet 4.6 [4], (6) Claude Opus 4.6 [3]. Unless otherwise specified, the results are derived from Gemini 3 Pro, which has the best overall performance.

Figure 15. Time and monetary cost across LLM APIs. fidelity. (4) Quantum Overhead: The expansion factor (×) in the total number of circuit fragments. (5) Static Properties: The CNOT gate count (CNOT) and circuit depth (Depth), which indicate circuit complexity and expected error rates. Baselines. We compare QSA to two sets of baselines to comprehensively evaluate its performance. For multiprogramming, we compare against MP [14], which enables concurrent execution through interference-aware scheduling, and QOS [19], which uses utility/fidelity-aware heuristics to place compatible workloads on shared QPU resources. We exclude HyperQ [57] and QVM [60] because they primarily provide execution abstractions rather than bundle-ranking policies. For error mitigation, we compare against Qiskit [1], CutQC [56], FrozenQubits [5], and QOS [19], representing compiler optimization, circuit cutting, qubit freezing, and an integrated mitigation pipeline, respectively. Workloads. Following QOS [19], we evaluate error mitigation using nine representative NISQ algorithms from the Supermarq [59], MQT-Bench [48], and QASM-Bench [31] suites: GHZ, W-State, Bernstein-Vazirani (BV), Hamiltonian Simulation (HS), Quantum-enhanced Support Vector Machine (QSVM), Two-Local Ansatz (TL), Variational Quantum Eigensolver (VQE-n), and Approximate Optimization Algorithm (QAOA-R/P). The algorithms’ circuits can be scaled by the number of qubits, providing comprehensive coverage of criteria for evaluating contemporary quantum workloads. For multiprogramming, we use dual-program workloads that pairs the above heterogeneous circuits across scales, following prior works [14, 19]. We enforce a fixed target utilization when creating the candidate pool for scheduling: we pair circuits such that their combined qubit count remains identical across all policies. This ensures a fair evaluation while isolating the impact of qubit numbers.

8.2

Overall Performance

Multiprogramming (Fidelity vs. Utilization). We evaluate the efficacy of QSA’s multiprogramming module using fidelity and normalized utilization. As shown in Figure 13, QSA outperforms both QOS and MP in both fidelity and normalized utilization, while exhibiting a different trade-off from the Oracle, i.e., higher utilization but lower fidelity. The error bars show variance across total qubit count (8, 16, and 24). Specifically, compared to QOS, QSA improves fidelity by 15.2%–19.5% and normalized utilization by 4.2%–9.5% across different target utilization settings. Compared to MP, QSA improves fidelity by 1.3%–16.6% and normalized utilization by 3.7%–13.7%. Overall, the gains become more pronounced when the system aims to use more qubits concurrently. Error Mitigation (Fidelity vs. Mitigation Time). We evaluate the error mitigation performance of QSA using fidelity and mitigation time. Figure 14 shows the trade-off between mitigation time and relative fidelity for small-scale (12-qubit) and medium-scale (24-qubit) circuits. QSA achieves fidelity comparable to QOS while reducing mitigation time by at least 96.8%, yielding the best trade-off between fidelity and mitigation time. We observe a slight fidelity drop on small-scale circuits, but a slight fidelity improvement on medium-scale circuits. These results show that QSA maintains strong performance across different circuit scales and suggest that its benefits may become more pronounced for larger circuits. Time and Monetary Cost. Figure 15 reports the runtime and monetary cost of evolving error mitigation and multiprogramming across different LLMs. We estimate monetary cost using the input/output token usage reported by the API together with the official token pricing. On average, error mitigation evolution costs 10.7–19.2 hours and $0.5–$10.8, while multiprogramming costs 0.2–2.5 hours and $0.4–$5.5. Overall, the monetary cost is modest compared to the human

Claude Opus 4.6

5.8

12-qubit

5.9

Δ Normalized Utilization (%)

24-qubit

0.0

12

14

16

18 20 Qubits

22

24

10

1

QOS QSA

100

1 1 E 3 V Z 1 1 -R B GH HS- A-P SVMTL- QE- TAT O Q V -S OA W QA QA

Time (RoM)

1.12

1.19

1.00

1.03

1.01

1.00

1.12

1.08

1.01

0.99

1.01

1.00

0.68

0.98

0.80

0.87

0.80

0.81

0.66

0.90

0.81

Gemini 3 Pro Gemini 3 Flash GPT-5 mini

Time (MoR)

GPT-5.3 Codex Claude Sonnet 4.6 Claude Opus 4.6

Depth

CNOT

Figure 18. Error mitigation comparison across LLMs. Mitigation Time (s)

Depth CNOT Time (MoR) Time (RoM)

0.5 0.0

Time (s)

Relative Value

Figure 16. Quantum overhead Figure 17. Multiprogramming across circuit scales. comparison across LLMs. 1.0

1.00

0.5

Mitigation 11.96

10

Network

1

4.56

QPU

4.57 5.53 3.07

1.10

10

0

12 qubit

24 qubit

QSA 13.75

QOS

0.60

MP

0.50

10.83

10 5.59

0

0.17

-0.03

Pareto Rank

0.75

0.25 0.00

Correlation

Gemini 3 Flash

0.10

Pareto Rank

QOS QSA

1.0 QOS

0.04

Frozen CutQC

Gemini 3 Pro

GPT-5 mini Claude Sonnet 4.6

0.04

101

0.11

Relative Values

GPT-5.3 Codex

102

Δ Fidelity

Quantum Overhead

Toward Quantum Software Automation: A Quantum-Aware Harness for LLM-Guided Evolution

Correlation

Figure 19. QSA performance Figure 20. QSA performance Figure 21. QSA time break- Figure 22. Pareto rank and across circuit scales. across circuit topologies. down. correlation analysis. effort required to manually design and tune quantum software. Error mitigation incurs substantially higher runtime because its verification is much more expensive than that of multiprogramming. Among the evaluated models, Gemini 3 Flash [20] and GPT-5-mini [40] achieve the lowest monetary cost, whereas Claude Opus 4.6 [3] incurs the highest. Quantum Overhead. Figure 16 shows the quantum overhead of error mitigation. Across different circuit scales, QSA incurs quantum overhead comparable to that of QOS. Both QSA and QOS exhibit higher overhead than CutQC and FrozenQubit because they partition circuits more aggressively. However, this additional overhead can be amortized through multiprogramming or QPU-level parallelism. Overall, QSA translates automated software design into 4.2–9.5% higher QPU utilization and 15.2–19.5% higher fidelity for multiprogramming, while removing at least 96.8% of mitigation preprocessing time without sacrificing fidelity, enabling more efficient use of scarce QPU resources. 8.3

Robustness Across Settings

LLM Models. Figure 17 and Figure 18 show how QSA generalizes across modern LLM APIs for both error mitigation and multiprogramming. For multiprogramming, Figure 17 reports the improvements in fidelity and normalized utilization relative to QOS. All evaluated LLMs achieve better trade-offs between fidelity and normalized utilization than QOS. Gemini 3 Pro and GPT-5.3-Codex form the Pareto frontier, with Claude Opus 4.6 attaining the worst performance. For error mitigation, Figure 18 compares the relative time, depth, and CNOT across different LLM APIs, all normalized to QOS. For depth and CNOT, we report the circuit-wise mean ratio to QOS. For time, we report both the mean of ratios (MoR) and the ratio of means (RoM), with the horizontal line indicating QOS. Overall, most LLMs substantially reduce mitigation time, with Gemini 3 Pro and Gemini 3 Flash lowering RoM to 0.04 (i.e., a 96% reduction). The main

exception is GPT-5-mini, which fails to improve the reward during evolution. For depth and CNOT, Gemini and GPT models keep the relative change within 3%, whereas Claude increases them by 8%–19%, suggesting slight overfitting in this setting. Overall, QSA generalizes well across modern LLM APIs, except that GPT-5-mini fails to improve mitigation time and Claude performs worse on circuit properties. Circuit Scales. Figure 19 evaluates QSA on quantum circuits ranging from 12 to 24 qubits, with all metrics normalized to those of QOS. Across circuit sizes, QSA preserves circuit quality comparable to QOS: the relative depth and CNOT count remain within 3% of QOS in nearly all cases, except for the CNOT count of the 18-qubit circuit. The errormitigation time under RoM remains consistently below 10% of that of QOS across all circuit sizes. Under MoR, the relative error-mitigation time generally decreases as the circuit size increases and exceeds that of QOS only for the 14-qubit circuit. Overall, QSA maintains consistent performance across circuit scales, preserving circuit quality while substantially reducing error-mitigation time. Moreover, its proxy-based verifier avoids the costly simulations required by QOS, thereby reducing verification overhead. Circuit Topologies. Figure 20 shows the mitigation time distribution across different topologies of 24-qubit quantum circuits. QOS fails to maintain low latency for 6 out 9 circuit topologies (e.g., BV, GHZ). While the error mitigation time of QSA is slightly higher than that of QOS for QAOA-R3 and QSVM circuits, QSA remains a consistent and low error mitigation time. 8.4 Performance Breakdown This subsection explains where QSA’s performance gains come from. Additional evolution traces and case studies are provided in Appendix §D and Appendix §E. Error-Mitigation Breakdown. Figure 21 decomposes endto-end execution into error mitigation, networking/scheduling, and per-circuit QPU computation. As the circuit size

95.5 000 001 010 100 111

96.0

0.770 95.0 0.765 Fidelity Norm. Util.

0.760 0

0.2

0.5

0.8

Weight of Rank

94.0 1

0.80

(a) Runtime

Instr.

Depth Sim. Fid. NL Measure CNOT

Fidelity

96.0

Fidelity

96.5 96.3 0.7796.30.77 0.7796.3

0.75 0.70

0.775

Norm. Util. (%)

Fidelity

0.80 0.7696.10.7896.1

Fidelity Norm. Util.

Norm. Util. (%)

Lily Jiaxin Wan, Deming Chen, Klara Nahrstedt, and Bo Chen

0.75 92

94

QOS Rand.

55.5

-0.013

BO

55.2

-0.041

RL

18.5

QSA

96

Normalized Utilization (%)

(b) Fidelity

22.6

-0.011

1.1

0

20

+0.001

40

60 −0.04

Mitigation Time (s) ↓

0.00

Δ Fidelity ↑

Figure 23. Impact of prompt Figure 24. Impact of score Figure 25. Impact of proxy on Figure 26. Impact of optimizacomposition. weights. fidelity and utilization. tion strategies.

8.5

Ablation Study

This subsection quantifies the contribution of QSA’s harness components and examines its key design choices. Additional ablation studies can be found at Appendix G. Impact of Approximation. We evaluate various approximation of real fidelity in multiprogramming evolution using the relative-balance ratio (§4.2) of: instruction count (Instr.), depth (Depth), non-local gates (NL), measurements (Measure), and CNOTs (see Appendix H for details). We also include simulatted fidelity (Sim. Fid.) for reference. Figure 25 reveals that Instr., Depth, and NL collectively form the Pareto front of the utilization-fidelity trade-off. While Instr. maximizes fidelity, Depth offers a superior balance, achieving > 96% utilization with minimal fidelity loss. At comparable utilization, Depth outperforms NL and CNOT as it more accurately reflects decoherence time, the dominant error source in NISQ devices. Despite its higher cost, simulated fidelity does not improve the final fidelity–utilization trade-off. Impact of Prompt Composition. Figure 23 compares different prompt compositions in multiprogramming evolution. We consider three components: 1) the Pareto ranks of the currently selected bundles (§5.2), 2) the system metrics of the ground-truth bundles (§5.2), and 3) the circuit properties of the ground-truth bundles (§5.1). We use a three-digit code to indicate whether each component is included. Without static

Sampled/evolution 0.4

Reward

increases from 12 to 24 qubits, mitigation time grows only from 1.10 s to 3.07 s, while QPU computation increases by 21.3%. The key gain comes from the evolved mitigation logic: QSA bounds the fragment-size search, applies binary search and caching, and avoids unnecessary expensive cutting probes, substantially reducing preprocessing overhead (Appendix §E). Detailed depth and CNOT results are provided in Appendix §F. Multiprogramming Policy Analysis. Figure 22 explains the multiprogramming gain through the quality of the evolved scheduling policy. QSA achieves an average Pareto rank of 5.59, substantially lower than QOS and MP, and a policy correlation of 0.60, compared with 0.17 for QOS and −0.03 for MP. Thus, QSA’s gain comes from learning a policy that selects bundles closer to the Pareto frontier while better preserving the ground-truth ordering, leading to the improved fidelity–utilization trade-off observed in Figure 13.

0.36

Full-suite 0.36

0.12

0.2 -0.08

0.0 −0.2 Random coreset

Hardness-guided coreset

Figure 27. Impact of coreset selection on evolution reward. or snapshot analysis (000), both fidelity and normalized utilization are lower than the full design (111). Partial prompt (001, 010, 100) improves over 000 and can approach 111, but is less stable. Overall, the full composition (111) achieves the most stable and strongest performance, delivering high fidelity, maximum utilization, and minimal variance. Impact of Policy Weight. In Figure 24, we assess the sensitivity of the system to the parameter 𝛼 in the policy reward (Equation 7). Keeping other settings fixed, we ablate 𝛼 ∈ {0, 0.2, 0.5, 0.8, 1.0} in multiprogramming evolution. When 𝛼 = 1, i.e., policy correlation is disabled, performance degrades noticeably in both fidelity and utilization. When 𝛼 = 0, i.e., Pareto rank is disabled, fidelity also drops slightly. Overall, the best performance is achieved by balancing Pareto rank and policy correlation, with 𝛼 = 0.5 yielding the best evolution outcome. Impact of Optimization Strategies. Figure 26 compares QSA with Random, Bayesian Optimization (BO), and Reinforcement Learning (RL) for error-mitigation design. All methods target the same output decisions (target fragment size and cutting method (GV or WC)) under a 60 s search deadline. These decisions follow the optimization interface of the QOS workflow. Unlike conventional baselines that operate within predefined search or policy classes, QSA evolves the decision program itself. Appendix I details their configurations. We report mitigation time and fidelity change relative to QOS, averaged across circuit types and scales (12–24 qubits). Random and BO frequently hit the deadline and are slower than QOS, while RL provides only a modest speedup with a fidelity reduction of 0.011. The broader program-level search enables QSA to discover decision strategies beyond the predefined search or policy classes of conventional baselines. Thus, QSA reduces average mitigation time from 22.58 s to 1.06 s (21.3×) while preserving fidelity, achieving the best overall trade-off.

Toward Quantum Software Automation: A Quantum-Aware Harness for LLM-Guided Evolution

Impact of Coreset Selection. Figure 27 compares uniform random coreset selection with our evolution-hardnessguided coreset under the same evolution setting and verification budget. Both approaches achieve the same reward on the sampled verification set (0.36), but their full-suite performance differs substantially. The random coreset yields a full-suite reward of −0.08, whereas the hardness-guided coreset improves it to 0.12. This result shows that uniform sampling can provide a misleading optimization signal, while focusing verification on an evolutionarily challenging circuit scale produces a more effective coreset for guiding evolution.

9

Discussion

This section discusses the automation scope of QSA and its generalizability. More discussions are at Appendix §J. Automation Scope. QSA is semi-automated: engineers provide a one-time task specification, including the target software component, an initial implementation, and the optimization objectives. Given this specification, the quantumaware harness automatically drives the subsequent evolution process by generating mutations, verifying candidate programs, collecting quantum-specific execution context, and providing rewards to guide further mutations. Full automation could further let the LLM formulate the initial implementation and optimization specification, but reliably exploring such an unconstrained software design space remains challenging for current LLMs [35, 64]. QSA therefore constrains the search to a specified software component while automating its iterative optimization. Hardware Generalizability. QSA is not tied to a specific quantum backend (e.g., the IBM Quantum platform). Its harness relies on observable circuit properties, execution states, and system metrics rather than assumptions about a particular hardware architecture. Therefore, QSA can be applied to different or future quantum platforms as long as the metrics required by the optimization objective can be measured and the execution stack exposes sufficient feedback for verification and prompting. As quantum hardware exposes more software-configurable controls (e.g., circuit layering or pulselevel parameters), the same evolutionary approach could further extend to lower-level system optimizations.

10

Related Work

Error Mitigation. Many techniques mitigate NISQ noise across different abstraction layers. Zero-Noise Extrapolation estimates noise-free expectations by scaling noise and extrapolating to the zero-noise limit [32, 58], while Probabilistic Error Cancellation reconstructs ideal outcomes through quasiprobability sampling [17, 58]. At the pulse and circuit levels, Dynamical Decoupling [13, 45] and Pauli Twirling [61] reduce decoherence and coherent noise. Readout Error Mitigation corrects measurement noise through hardware calibration [8]. At the system level, recent frameworks such as

QOS incorporate noise-aware mapping and scheduling based on hardware calibration. QSA automates the discovery of error mitigation strategies through compiler-pass evolution, uncovering better fidelity–latency trade-offs. Multiprogramming. Quantum multiprogramming evolved from rigid physical isolation to suppress crosstalk [14] toward flexible mapping techniques like X-SWAP [34] that improve spatial utilization by relaxing program boundaries. QOS [19] uses admission control for fidelity-resource balancing, while QVM [57] leverages virtualization to enforce execution isolation. However, these systems rely on static heuristics that struggle to navigate complex trade-offs between fidelity and utilization as hardware scales increase. QSA bridges this gap by abstracting decision logic into evolvable policies, autonomously synthesizing scheduling rules that approach the Pareto-optimal frontier. LLM-Based System Optimization. Traditional approaches optimize systems through Bayesian optimization [18, 54], RL-based fine-tuning [37, 65], and program synthesis [23, 55]. However, these approaches typically require predefined parameter spaces, action spaces, or synthesis constraints, making them less suitable for optimizing complex structural code [12]. Recent work therefore increasingly leverages LLMs to directly generate and revise system code [12, 33, 42]. LLM-guided evolutionary search further enables iterative refinement by repeatedly generating, evaluating, and improving candidate programs. Evolutionary discovery of bivariate bicycle codes [11] applies this paradigm to quantum code discovery, but focuses on a specific quantum artifact. In contrast, QSA targets reusable quantum software components, including compiler passes and runtime policies. More importantly, existing evolutionary search frameworks provide only a generic search loop and lack the quantum-specific mechanisms required for efficient verification, informative execution feedback, and heterogeneous reward design. QSA addresses this gap with a quantum-aware harness that equips generic LLM-guided evolutionary search with these domainspecific mechanisms.

11

Conclusion

Rapid advances in quantum hardware call for automated software design to keep pace with evolving systems and move beyond handcrafted heuristics. We present QSA, a quantumaware harness that enables LLM-guided evolutionary search to automatically optimize quantum compiler passes and runtime policies. By providing quantum-specific mechanisms for efficient verification, informative execution feedback, and task-specific reward design, QSA makes generic evolutionary search practical for quantum software. Our evaluation shows that QSA consistently outperforms SOTA baselines across fidelity, QPU utilization, and execution time, demonstrating the potential of quantum-aware evolutionary search for automated quantum software engineering.

Lily Jiaxin Wan, Deming Chen, Klara Nahrstedt, and Bo Chen

References [1] Gadi Aleksandrowicz et al. 2019. Qiskit: An Open-source Framework for Quantum Computing. doi:10.5281/zenodo.2562111 [2] Amazon Web Services. 2025. Amazon Braket: A Fully Managed Quantum Computing Service. https://aws.amazon.com/braket/. Accessed: 2026-03-17. [3] Anthropic. 2025. Introducing Claude 4.5 Opus. https://www.anthropic. com/news/claude-opus-4-5. Accessed: 2026-03-17. [4] Anthropic. 2025. Introducing Claude 4.5 Sonnet. https://www. anthropic.com/news/claude-sonnet-4-5. Accessed: 2026-03-17. [5] Ramin Ayanzadeh, Narges Alavisamani, Poulami Das, and Moinuddin Qureshi. 2023. Frozenqubits: Boosting fidelity of qaoa by skipping hotspot nodes. In Proceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 2. 311–324. https://dl.acm.org/doi/10.1145/3575693. 3575741 [6] Adriano Barenco, Charles H Bennett, Richard Cleve, David P DiVincenzo, Norman Margolus, Peter Shor, Tycho Sleator, John A Smolin, and Harald Weinfurter. 1995. Elementary gates for quantum computation. Physical review A 52, 5 (1995), 3457. https://journals.aps.org/ pra/abstract/10.1103/PhysRevA.52.3457 [7] Anil Bhattacharyya. 1943. On a measure of divergence between two statistical populations defined by their probability distribution. Bulletin of the Calcutta Mathematical Society 35 (1943), 99–110. https: //www.jstor.org/stable/25047882 [8] Sergey Bravyi, Sarah Sheldon, Abhinav Kandala, David C Mckay, and Jay M Gambetta. 2021. Mitigating measurement errors in multiqubit experiments. Physical Review A 103, 4 (2021), 042605. https://journals. aps.org/pra/abstract/10.1103/PhysRevA.103.042605 [9] Colin D Bruzewicz, John Chiaverini, Robert McConnell, and Jeremy M Sage. 2019. Trapped-ion quantum computing: Progress and challenges. Applied physics reviews 6, 2 (2019). doi:10.1063/1.5088164 [10] Mark Crovella, Prakash Das, Cezary Dubnicki, Thomas LeBlanc, and Evangelos Markatos. 1991. Multiprogramming on multiprocessors. In Proceedings of the third IEEE Symposium on Parallel and Distributed Processing. IEEE, 590–597. https://dl.acm.org/doi/10.1109/SPDP.1991. 218246 [11] Juan Cruz-Benito, Andrew W. Cross, David Kremer, and Ismael Faro. 2026. Evolutionary Discovery of Bivariate Bicycle Codes with LLMGuided Search. arXiv:2606.02418 [quant-ph] doi:10.48550/arXiv.2606. 02418 [12] Chris Cummins, Volker Seeker, Dejan Grubisic, Baptiste Roziere, Jonas Gehring, Gabriel Synnaeve, and Hugh Leather. 2024. Meta large language model compiler: Foundation models of compiler optimization. arXiv preprint arXiv:2407.02524 (2024). https://arxiv.org/abs/2407. 02524 [13] Poulami Das, Swamit Tannu, Siddharth Dangwal, and Moinuddin Qureshi. 2021. Adapt: Mitigating idling errors in qubits via adaptive dynamical decoupling. In MICRO-54: 54th Annual IEEE/ACM International Symposium on Microarchitecture. 950–962. https://dl.acm.org/ doi/abs/10.1145/3466752.3480059 [14] Poulami Das, Swamit S Tannu, Prashant J Nair, and Moinuddin Qureshi. 2019. A case for multi-programming quantum computers. In Proceedings of the 52nd Annual IEEE/ACM International Symposium on Microarchitecture. 291–303. https://dl.acm.org/doi/abs/10.1145/ 3352460.3358287 [15] Kalyanmoy Deb, Amrit Pratap, Sameer Agarwal, and TAMT Meyarivan. 2002. A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE transactions on evolutionary computation 6, 2 (2002), 182–197. https://ieeexplore.ieee.org/document/996017 [16] Matthew DeCross, Eli Chertkov, Megan Kohagen, and Michael FossFeig. 2023. Qubit-reuse compilation with mid-circuit measurement and reset. Physical Review X 13, 4 (2023), 041057. https://journals.aps. org/prx/abstract/10.1103/PhysRevX.13.041057

[17] Suguru Endo, Simon C Benjamin, and Ying Li. 2018. Practical quantum error mitigation for near-future applications. Physical Review X 8, 3 (2018), 031027. https://journals.aps.org/prx/abstract/10.1103/ PhysRevX.8.031027 [18] Peter I Frazier. 2018. A tutorial on Bayesian optimization. arXiv preprint arXiv:1807.02811 (2018). [19] Emmanouil Giortamis, Francisco Romão, Nathaniel Tornow, and Pramod Bhatotia. 2025. {QOS}: Quantum operating system. In 19th USENIX Symposium on Operating Systems Design and Implementation (OSDI 25). 429–447. https://www.usenix.org/system/files/osdi25giortamis.pdf [20] Google DeepMind. 2025. Gemini 3 Flash Preview. https://deepmind. google/models/gemini/flash/. Accessed: 2026-03-17. [21] Google DeepMind. 2025. A new era of intelligence with Gemini 3. https://blog.google/products-and-platforms/products/gemini/ gemini-3/. Accessed: 2026-03-17. [22] Daniel M Greenberger, Michael A Horne, and Anton Zeilinger. 1989. Going beyond Bell’s theorem. In Bell’s theorem, quantum theory and conceptions of the universe. Springer, 69–72. https://link.springer.com/ chapter/10.1007/978-94-017-0849-4_10 [23] Sumit Gulwani, Oleksandr Polozov, and Rishabh Singh. 2017. Program synthesis. Foundations and Trends in Programming Languages 4, 1-2 (2017), 1–119. [24] Patrick Hopf, Nils Quetschlich, Laura Schulz, and Robert Wille. 2025. Improving figures of merit for quantum circuit compilation. In 2025 Design, Automation & Test in Europe Conference (DATE). IEEE, 1–7. https://ieeexplore.ieee.org/document/10992761 [25] IBM Quantum. 2025. IBM Quantum Platform. https://quantum.cloud. ibm.com/. Accessed: 2026-03-08. [26] IonQ. 2025. IonQ Quantum Computing Systems. https://ionq.com/. Accessed: 2026-03-17. [27] Hanru Jiang. 2024. Qubit recycling revisited. Proceedings of the ACM on Programming Languages 8, PLDI (2024), 1264–1287. https://dl.acm. org/doi/10.1145/3656428 [28] Youngseok Kim, Andrew Eddins, Sajant Anand, Ken Xuan Wei, Ewout Van Den Berg, Sami Rosenblatt, Hasan Nayfeh, Yantao Wu, Michael Zaletel, Kristan Temme, et al. 2023. Evidence for the utility of quantum computing before fault tolerance. Nature 618, 7965 (2023), 500–505. https://www.nature.com/articles/s41586-023-06096-3 [29] Oğuzcan Kırmemiş, Francisco Romão, Emmanouil Giortamis, and Pramod Bhatotia. 2025. Weaver: A Retargetable Compiler Framework for FPQA Quantum Architectures. In Proceedings of the 23rd ACM/IEEE International Symposium on Code Generation and Optimization. 299– 316. https://dl.acm.org/doi/abs/10.1145/3696443.3708965 [30] Morten Kjaergaard, Mollie E Schwartz, Jochen Braumüller, Philip Krantz, Joel I-J Wang, Simon Gustavsson, and William D Oliver. 2020. Superconducting qubits: Current state of play. Annual Review of Condensed Matter Physics 11, 1 (2020), 369–395. doi:10.1146/annurevconmatphys-031119-050605 [31] Ang Li, Samuel Stein, Sriram Krishnamoorthy, and James Ang. 2023. Qasmbench: A low-level quantum benchmark suite for nisq evaluation and simulation. ACM Transactions on Quantum Computing 4, 2 (2023), 1–26. https://dl.acm.org/doi/10.1145/3550488 [32] Ying Li and Simon C Benjamin. 2017. Efficient variational quantum simulator incorporating active error minimization. Physical Review X 7, 2 (2017), 021050. https://journals.aps.org/prx/abstract/10.1103/ PhysRevX.7.021050 [33] Hongyu Lin, Yuchen Li, Haoran Luo, Kaichun Yao, Libo Zhang, Zhenghong Lin, Mingjie Xing, Yanjun Wu, and Carl Yang. 2025. Byos: Knowledge-driven large language models bring your own operating system more excellent. arXiv preprint arXiv:2503.09663 (2025). https://arxiv.org/abs/2503.09663

Toward Quantum Software Automation: A Quantum-Aware Harness for LLM-Guided Evolution

[34] Lei Liu and Xinglei Dou. 2021. Qucloud: A new qubit mapping mechanism for multi-programming quantum computing in cloud environment. In 2021 IEEE International symposium on high-performance computer architecture (HPCA). IEEE, 167–178. https://ieeexplore.ieee. org/document/9407180 [35] Yecheng Jason Ma, William Liang, Guanzhi Wang, De-An Huang, Osbert Bastani, Dinesh Jayaraman, Yuke Zhu, Linxi Fan, and Anima Anandkumar. 2023. Eureka: Human-level reward design via coding large language models. arXiv preprint arXiv:2310.12931 (2023). https: //arxiv.org/abs/2310.12931 [36] Microsoft. 2025. Azure Quantum: Cloud Quantum Computing Platform. https://learn.microsoft.com/en-us/azure/quantum/overviewazure-quantum. Accessed: 2026-03. [37] Azalia Mirhoseini, Hieu Pham, Quoc V Le, Benoit Steiner, Rasmus Larsen, Yuefeng Zhou, Naveen Kumar, Mohammad Norouzi, Samy Bengio, and Jeff Dean. 2017. Device placement optimization with reinforcement learning. In International conference on machine learning. PMLR, 2430–2439. [38] Kosuke Mitarai and Keisuke Fujii. 2021. Constructing a virtual twoqubit gate by sampling single-qubit operations. New Journal of Physics 23, 2 (2021), 023021. https://iopscience.iop.org/article/10.1088/13672630/abd7bc [39] Alexander Novikov, Ngân Vũ, Marvin Eisenberger, Emilien Dupont, Po-Sen Huang, Adam Zsolt Wagner, Sergey Shirobokov, Borislav Kozlovskii, Francisco JR Ruiz, Abbas Mehrabian, et al. 2025. Alphaevolve: A coding agent for scientific and algorithmic discovery. arXiv preprint arXiv:2506.13131 (2025). https://arxiv.org/abs/2506.13131 [40] OpenAI. 2025. GPT-5. https://openai.com/gpt-5/. Accessed: 2026-0317. [41] OpenAI. 2026. Introducing GPT-5.3 Codex. https://openai.com/index/ introducing-gpt-5-3-codex/. Accessed: 2026-03-17. [42] Sujin Park, Mingyu Guan, Xiang Cheng, and Taesoo Kim. 2025. Principles and methodologies for serial performance optimization. In 19th USENIX Symposium on Operating Systems Design and Implementation (OSDI 25). 357–373. https://dl.acm.org/doi/10.5555/3767901.3767921 [43] Tianyi Peng, Aram W Harrow, Maris Ozols, and Xiaodi Wu. 2020. Simulating large quantum circuits on a small quantum computer. Physical review letters 125, 15 (2020), 150504. https://journals.aps.org/ prl/abstract/10.1103/PhysRevLett.125.150504 [44] JH Plantenberg, PC De Groot, CJPM Harmans, and JE Mooij. 2007. Demonstration of controlled-NOT quantum gates on a pair of superconducting quantum bits. Nature 447, 7146 (2007), 836–839. [45] Bibek Pokharel, Namit Anand, Benjamin Fortman, and Daniel A Lidar. 2018. Demonstration of fidelity improvement using dynamical decoupling with superconducting qubits. Physical review letters 121, 22 (2018), 220502. https://journals.aps.org/prl/abstract/10.1103/ PhysRevLett.121.220502 [46] John Preskill. 2018. Quantum computing in the NISQ era and beyond. Quantum 2 (2018), 79. https://doi.org/10.22331/q-2018-08-06-79 [47] Quantinuum. 2025. Quantinuum Quantum Computing Systems. https: //www.quantinuum.com/. Accessed: 2026-03-17. [48] Nils Quetschlich, Lukas Burgholzer, and Robert Wille. 2023. MQT Bench: Benchmarking software and design automation tools for quantum computing. Quantum 7 (2023), 1062. https://quantumjournal.org/papers/q-2023-07-20-1062/ [49] Rigetti Computing. 2025. Rigetti Quantum Cloud Services (QCS). https://www.rigetti.com/. Accessed: 2026-03. [50] Mark Saffman. 2016. Quantum computing with atomic qubits and Rydberg interactions: progress and challenges. Journal of Physics B: Atomic, Molecular and Optical Physics 49, 20 (2016), 202001. doi:10. 1088/0953-4075/49/20/202001 [51] Asankhaya Sharma. 2025. OpenEvolve: an open-source evolutionary coding agent. https://github.com/algorithmicsuperintelligence/ openevolve

[52] Dan J Shepherd. 2006. On the role of Hadamard gates in quantum circuits. Quantum Information Processing 5, 3 (2006), 161–177. https: //link.springer.com/article/10.1007/s11128-006-0023-4 [53] Sergei Slussarenko and Geoff J Pryde. 2019. Photonic quantum information processing: A concise review. Applied physics reviews 6, 4 (2019). doi:10.1063/1.5115814 [54] Jasper Snoek, Hugo Larochelle, and Ryan P Adams. 2012. Practical bayesian optimization of machine learning algorithms. Advances in neural information processing systems 25 (2012). [55] Armando Solar-Lezama. 2008. Program synthesis by sketching. University of California, Berkeley. [56] Wei Tang, Teague Tomesh, Martin Suchara, Jeffrey Larson, and Margaret Martonosi. 2021. Cutqc: using small quantum computers for large quantum circuit evaluations. In Proceedings of the 26th ACM International conference on architectural support for programming languages and operating systems. 473–486. https://dl.acm.org/doi/10. 1145/3445814.3446758 [57] Runzhou Tao, Hongzheng Zhu, Jason Nieh, Jianan Yao, and Ronghui Gu. 2025. Quantum virtual machines. In 19th USENIX Symposium on Operating Systems Design and Implementation (OSDI 25). 411–428. https://www.usenix.org/conference/osdi25/presentation/tao [58] Kristan Temme, Sergey Bravyi, and Jay M Gambetta. 2017. Error mitigation for short-depth quantum circuits. Physical review letters 119, 18 (2017), 180509. https://journals.aps.org/prl/abstract/10.1103/ PhysRevLett.119.180509 [59] Teague Tomesh, Pranav Gokhale, Victory Omole, Gokul Subramanian Ravi, Kaitlin N Smith, Joshua Viszlai, Xin-Chuan Wu, Nikos Hardavellas, Margaret R Martonosi, and Frederic T Chong. 2022. Supermarq: A scalable quantum benchmark suite. In 2022 IEEE International Symposium on High-Performance Computer Architecture (HPCA). IEEE, 587–603. https://ieeexplore.ieee.org/document/9773202 [60] Nathaniel Tornow, Emmanouil Giortamis, and Pramod Bhatotia. 2025. QVM: Quantum Gate Virtualization Machine. Proceedings of the ACM on Programming Languages 9, PLDI (2025), 993–1018. https://dl.acm. org/doi/10.1145/3729290 [61] Joel J Wallman and Joseph Emerson. 2016. Noise tailoring for scalable quantum computation via randomized compiling. Physical Review A 94, 5 (2016), 052325. https://journals.aps.org/pra/abstract/10.1103/ PhysRevA.94.052325 [62] Robert Wille, Aaron Lye, and Rolf Drechsler. 2014. Optimal SWAP gate insertion for nearest neighbor quantum circuits. In 2014 19th Asia and South Pacific Design Automation Conference (ASP-DAC). IEEE, 489–494. https://ieeexplore.ieee.org/abstract/document/6742939 [63] Ellis Wilson, Frank Mueller, Lindsay Bassman Oftelie, and Costin Iancu. 2021. Empirical evaluation of circuit approximations on noisy quantum devices. In Proceedings of the international conference for high performance computing, networking, storage and analysis. 1–15. https://dl.acm.org/doi/10.1145/3458817.3476189 [64] Chunqiu Steven Xia, Yinlin Deng, Soren Dunn, and Lingming Zhang. 2024. Agentless: Demystifying llm-based software engineering agents. arXiv preprint arXiv:2407.01489 (2024). https://arxiv.org/abs/2407. 01489 [65] Yanqi Zhou, Sudip Roy, Amirali Abdolrashidi, Daniel Wong, Peter Ma, Qiumin Xu, Hanxiao Liu, Phitchaya Phothilimtha, Shen Wang, Anna Goldie, et al. 2020. Transferable graph optimizers for ml compilers. Advances in Neural Information Processing Systems 33 (2020), 13844– 13855.

A

IBM Quantum Platform Analysis

This section provides additional characterization of resource availability and hardware noise on the IBM Quantum platform, complementing the discussion in §2.2.

0.0

IBM

S AW

1

re

Figure 28. Publicly accessible QPUs across major cloud providers. fez

100 0

i uum IonQ Rigett ntin Qua

Azu

1014

9605

39985 201

101

95th

99th Percentile

99.9th

Figure 29. High-percentile pending-job counts on IBM QPUs. marrakesh

torino

104

Pending Jobs

0.0035

Error

2

fez

2-qubit Gate

103

0.0030

0.00036

marrakesh

torino

1-qubit Gate

0.05

0.00034

0.04

0.00032

0.03

0.00030

Readout

0.02

0.00028

0.0025

0.01 05 09 13 17 21 25

05 09 13 17 21 25

UTC Day

05 09 13 17 21 25

UTC Day

UTC Day

Figure 31. Variation of IBM QPU error metrics over time. Best Reward So-far

2

2.5

102

212

5.0

fez mar tor

103 113

7

3670

104 7

22

10

7.5

25

10.0

Pending Jobs

Number of QPUs

Lily Jiaxin Wan, Deming Chen, Klara Nahrstedt, and Bo Chen

Gemini

100

GPT

Claude

GPT-5 mini GPT-5.3 Codex

Claude Sonnet 4.6 Claude Opus 4.6

10−1 0

Gemini 3 Pro Gemini 3 Flash

−10−1 −10

0

−101 0

102

50

Iteration

100 0

50

Iteration

100 0

50

Iteration

100

Figure 32. Learning curves for error mitigation. 101 100 03-05 00:00

03-09 00:00

03-13 00:00

03-17 00:00

UTC Time

03-21 00:00

03-25 00:00

Figure 30. Pending jobs on IBM QPUs throughout March 2026. Backend

2Q Gate (𝜇)

1Q Gate (𝜇)

Readout (𝜇)

Torino Marrakesh Fez

2.564 × 10−3

3.172 × 10−4

2.632 × 10−3

3.201 × 10−4

2.626 × 10−3

2.846 × 10−4

2.961 × 10−2 1.160 × 10−2 1.462 × 10−2

Table 1. Representative error metrics of the evaluated IBM QPUs. Resource Availability. Figure 28 shows that public quantum clouds expose only a limited number of QPUs. To further characterize resource contention, we collect the number of pending jobs on IBM_torino, IBM_marrakesh, and IBM_fez every 10 minutes throughout March 2026. Figure 29 reports the 95th, 99th, and 99.9th percentiles of the observed queue lengths. The long-tail queueing behavior is substantial; for example, the 99.9th-percentile queue length of IBM_fez reaches 39,985 jobs. Figure 30 further shows how queue lengths vary over time. IBM_fez consistently experiences heavier contention than the other observed QPUs, while all three exhibit substantial temporal variation. Hardware Noise. Table 1 summarizes representative 1qubit, 2-qubit, and readout error rates for the three IBM QPUs. The backends exhibit different error characteristics: IBM_torino has the highest readout error, while IBM_fez has the lowest 1-qubit gate error among the measured systems. Figure 31 shows that these error metrics also vary over time. The 2-qubit gate error rates are broadly comparable across the observed QPUs, although IBM_marrakesh exhibits greater temporal variation. IBM_fez generally has the lowest 1-qubit gate error, whereas IBM_torino exhibits the highest

readout error. Together, these measurements highlight both resource contention and backend-dependent noise variation in today’s quantum cloud environments.

B

Static Feature Details

Table 2 describes the 14 static features extracted through static analysis and their physical meanings. It combines structural size/complexity features (e.g., depth, qubit count, instruction/CNOT/nonlocal-gate counts), graph/connectivity features (e.g., connected components), and executionshape features (e.g., parallelism, liveness, measurement intensity, entanglement ratio, critical depth). These features give a compact physical profile of each circuit, so the optimizer can prioritize candidates that are likely easier to mitigate while preserving fidelity and runtime efficiency.

C

OpenEvolve Configurations

Listings 2 and 5 present the OpenEvolve configurations for error mitigation and multiprogramming, respectively. In addition to the static and snapshot analysis of our design, they include four basic components from OpenEvolve [51]: (1) Format requirement, which specifies editable code regions and function signatures to ensure syntactic correctness; (2) Verifier specification, which defines the reward formulation (e.g., a single metric or a weighted sum of multiple metrics); (3) Parent program, selected via a stochastic policy from topquality candidates (exploitation) or random candidates (exploration); and (4) Historical programs, where top programs guide optimization toward high-quality solutions, diverse programs broaden exploration, and inspiration programs provide reusable patterns for generation. We summarize them in Table 3.

D

Evolution Analysis

Overall. We analyze all evolution runs that achieve at least one improvement within 100 iterations for both multiprogramming and error mitigation. Figure 33 shows the CDFs

Toward Quantum Software Automation: A Quantum-Aware Harness for LLM-Guided Evolution

Static Features

Physical Meaning

depth num_qubits num_clbits num_nonlocal_gates

Number of sequential gate layers in the circuit. Number of qubits used by the circuit. Number of classical bits used by the circuit. Number of nonlocal gates, i.e., a two-qubit gate between qubits that are not directly connected on the hardware coupling graph, so SWAPs [62] must first move their states together. Number of disconnected components in the qubit-interaction graph. Total number of instructions in the circuit. Number of measurement operations. Number of CNOT (controlled-X) gates. Degree of interaction or dependency among different parts of the circuit. Degree to which qubits remain active throughout circuit execution. Degree to which operations can be executed concurrently. Relative amount or intensity of measurement operations in the circuit. Fraction of entangling gates among all gates. Length of the longest dependency chain in the circuit.

num_connected_components number_instructions num_measurements num_cnot_gates program_communication liveness parallelism measurement entanglement_ratio critical_depth

Table 2. Physical meaning of static features used in the evolution target. Content

Format Req Verifier Spec Parent Program History Program + Static Analysis + Snapshot Analysis

Editable region and function signature. Scoring formula description. Parent implementation to mutate. Top/Diverse/Inspirational programs. Extracted features through analysis. High-level scores and per-case traces.

Best Score So Far

Gemini

Block

0.5

0.0

Multiprog Mitigation

20

40

First Improve Iter

1.0

CDF

CDF

CDF

1.0

0.5

0.0

Multiprog Mitigation

50

100

Last Improve Iter

0.5 Multiprog Mitigation

0.0

5

10

Improve Count

Figure 33. Improvement analysis. of the first improvement iteration, the last improvement iteration, and the total number of improvements. The first improvement typically occurs within 27 iterations, suggesting that improvement tends to happen early or not at all. For multiprogramming, the first improvement appears within just 2 iterations, as it is an easier target to optimize. In contrast, the last improvement may occur at any point within the 100 iterations, highlighting the stochastic nature of evolution. The number of improvements is broadly distributed between 0–8 for multiprogramming and 0–15 for error mitigation, indicating sparse reward-improving mutations. Detailed learning curves are explained in the following paragraph. Learning Curves. Figure 32 illustrates the learning curve for Error Mitigation using different LLM APIs. Gemini models are the first to improve the best reward. Claude models are the second and GPT models are the last. Notably, GPT-5-mini can hardly improve the reward. In terms of the reward value, Claude achieves the highest score. Unfortunately, it fails to generalize well as described in Figure 18.

Claude

GPT-5 mini GPT-5.3 Codex

Claude Sonnet 4.6 Claude Opus 4.6

0.80 0.60 Gemini 3 Pro Gemini 3 Flash

0.40 0.20

0

50

100 0

50

Iteration

100 0

50

100

Figure 34. Learning curves for multiprogramming.

Table 3. Prompt description. 1.0

GPT

1.00

Figure 34 shows the learning curve for Multiprogramming using different LLM APIs. Most of the reward gain is obtained in the early iterations, while later iterations mainly provide marginal refinements. Gemini 3 Pro converges the fastest, reaching near-final performance almost immediately. In addition, Multiprogramming exhibits earlier and more frequent improvements than Error Mitigation. It suggesting that the search space in the former setting is easier for LLM-driven optimization.

E

Case Study

E.1

Best Case Analysis.

Our evolution using Gemini 3 Pro results in a sophisticated design for error mitigation but a simple-yet-effective one for multiprogramming. For error mitigation, QSA derives a bounded cost-search policy: it does binary search over target size, caches probe results, prefers gate virtualization first, and only pays for wire cutting checks when likely useful. This approach gives a strong runtime cut while preserving circuit properties (depth/CNOT). For multiprogramming, QSA distills a minimalist heuristic by discarding 14+ features in favor of a two-variable model: utilization and depth ratio. This selection minimizes idle-time decoherence while ensuring near-zero scheduling overhead. More details can be found at Appendix E.3.

Lily Jiaxin Wan, Deming Chen, Klara Nahrstedt, and Bo Chen

Depth

75

random_seed: 42 max_tasks_per_child: 1

Permutation Combination

50 25 0

diff_based_evolution: false max_code_length: 30000

Default 1

5

10

15

Method ID

20

24

Figure 35. Analysis of combination and permutation. E.2

Failed Case Analysis.

Effective abstraction is critical for successful policy evolution. We initially had an hypothesis that a different combination or permutation of techniques [5, 16, 27, 38, 43] adopted by QOS [19] may yield a better performance. We then abstract the details of these techniques (including circuit cutting that should have exposed the cost search logic) and tasked the LLM with exploring various permutations and combinations. This approach yielded no measurable improvements because operator ordering has negligible impact on final mitigation efficacy. As analyzed in Figure 35, neither combination nor permutation outperform the default setting. Overall, successful evolution requires precise abstraction driven by solid weakness analysis results. E.3

Initial and Best Programs

Listing 3 and Listing 4 show the initial program and the best program (generated by Gemini 3 Pro) for the error mitigation task. The best program performs well because it turns cost-search into a bounded, efficient decision process: it uses metadata to set a valid search range, applies a binary search to find the smallest feasible target size under budget, and avoids unnecessary expensive probes (especially wire cutting) during the search. That usually yields smaller feasible fragment sizes (helping depth/CNOT proxies) while reducing search overhead. It also includes caching and a final verification step, which improves stability and keeps runtime predictable across circuits. Listing 6 and Listing 7 show the initial program and the best program (generated by Gemini 3 Pro) for the error mitigation task. The evolved program identifies a minimalist, principle-driven heuristic. It condenses 14+ architectural features into a weighted sum of effective utilization (0.65) and depth similarity (0.35). This shift validates a key system insight: quantum resource management is fundamentally a temporal synchronization problem. By prioritizing the depth ratio, the logic enforces temporal alignment. Such policy parsimony ensures near-zero scheduling overhead and robust generalization across diverse hardware backends. Listing 2. OpenEvolve Configuration for Error Mitigation max_iterations: 100 checkpoint_interval: 10 log_level: "INFO" log_dir: null

early_stopping_patience: null convergence_threshold: 0.001 early_stopping_metric: "combined_score" llm: api_base: "${OPENAI_API_BASE}" api_key: "${OPENAI_API_KEY}" primary_model: "${OPENAI_MODEL}" temperature: 0.7 top_p: 0.9 max_tokens: 4096 timeout: 600 retries: 3 retry_delay: 15 prompt: system_message: | You are optimizing ONLY the function evolved_cost_search (self, q, size_to_reach, budget) in qos/error_mitigator/evolution_target.py. OUTPUT FORMAT (CRITICAL): - Return the full file contents of qos/error_mitigator/ evolution_target.py. - Output code only: no explanations, no markdown fences, no extra text. GENERALIZATION REQUIREMENT (CRITICAL): - The evaluator samples random (circuit_type, num_qubits ) pairs each run. - Qubit counts are constrained to [12,24] and must exist in the benchmark set. - Your logic must generalize across circuit families ( QAOA, BV, GHZ, VQE, QSVm, etc.) and sizes. - Do NOT branch on q.get_name(), bench strings, or any per-benchmark constants. FUNCTION CONTRACT (MUST FOLLOW EXACTLY): - Do NOT change the function signature. It MUST remain: evolved_cost_search(self, q, size_to_reach, budget) - It MUST return EXACTLY 2 values: (size_to_reach: int, method: str) where method is "GV" or "WC". - If you violate signature/return arity, evaluation will fail. CONTEXT: The input q is a Qernel. It contains metadata computed elsewhere, accessible via: input_meta = q.get_metadata() Useful fields may include: depth, num_qubits, num_clbits, num_nonlocal_gates, num_connected_components, number_instructions, num_measurements, num_cnot_gates, program_communication, liveness, parallelism, measurement, entanglement_ratio, critical_depth.

Toward Quantum Software Automation: A Quantum-Aware Harness for LLM-Guided Evolution

COST HELPER APIS (from qos/error_mitigator/run.py): - compute_gv_cost(q, size_to_reach, timeout_sec=0) - compute_wc_cost(q, size_to_reach, timeout_sec=0) - Each call returns (cost: int, timed_out: bool). - timeout_sec is per-call; GV/WC can use different values. - timeout_sec=0 means no per-call timeout. - On timeout, cost is set high and timed_out=True. Inputs/Outputs: - Inputs: * q: the Qernel (quantum circuit container) whose metadata informs the decision. * size_to_reach: the current target circuit size for cutting/virtualization search. * budget: the cost threshold used to decide whether GV/WC is acceptable. - Outputs: * size_to_reach: the chosen target size to use for the cutting/virtualization of the target circuit. * method: "GV" or "WC", which determines which technique is applied at that size. - Cost logging and traces are handled outside this file (in qos/error_mitigator/run.py). IMPORTANT PRACTICAL CONSTRAINTS: - `budget` is only meaningful for your cost-search logic (i.e., your own GV/WC cost queries). Do not assume downstream will enforce budget. - If size_to_reach >= num_qubits, the decomposition can become a no-op. Use that ONLY as an explicit fallback when costs indicate everything else is too expensive. Prefer to clamp sizes into a meaningful range (e.g., [2, num_qubits-1]) for real action. EFFICIENCY / GENERALIZATION GUIDELINES (HIGH IMPACT): - Minimize gv/wc cost calls. Avoid long +/-1 stepping loops. Prefer a small number of probes + bounded search (e.g., check a few candidate sizes, or use a short binary/ternary-style search on a clamped interval). - Use cheap metadata heuristics first; only call compute_*_cost when needed. - Keep behavior monotone and stable: similar circuits should yield similar decisions. NOTE: "QOS" refers to the baseline implementation, and " QOSE" refers to the evolved program. METRICS: - qose_depth: average depth ratio (QOSE/QOS) - qose_cnot: average CNOT gate ratio (QOSE/QOS) - qose_overhead: average circuit-count ratio (QOSE/QOS) - avg_run_time: average runtime ratio (QOSE/QOS) - combined_score (maximize): evaluator score used by evolution ranking. combined_score = - (qose_depth + qose_cnot + 1./4 * avg_run_time) NOTE:

- Depth/CNOT ratios are proxies for fidelity. The goal is to improve fidelity, reduce cost-search time, and avoid depth/CNOT regressions. We provide the execution outputs of the last program executed by the evaluator. Use them to diagnose decisions. Details of the execution outputs are as follows: EXECUTION OUTPUTS PROVIDED BY EVALUATOR: - Artifacts are organized as: * artifacts["summary"] (optional): aggregate/runlevel metrics * artifacts["cases"] (optional): list of per-(bench, size) case metrics EXECUTION OUTPUTS PROVIDED BY EVALUATOR (artifacts[" summary"]): - qose_budget: budget used for all cases - qose_run_sec_avg: average runtime of evolved mitigator - qos_run_sec_avg: average runtime of baseline QOS - gv_cost_calls_total: total GV cost calls (evolved) - wc_cost_calls_total: total WC cost calls (evolved) - qos_gv_cost_calls_total: total GV cost calls ( baseline) - qos_wc_cost_calls_total: total WC cost calls ( baseline) - qose_depth_avg: average absolute depth of evolved circuits - qose_cnot_avg: average absolute CNOT gate count of evolved circuits - qose_overhead_avg: average absolute number of circuits generated by evolved method - qos_depth_avg: average absolute depth of baseline circuits - qos_cnot_avg: average absolute CNOT gate count of baseline circuits - qos_overhead_avg: average absolute number of circuits generated by baseline EXECUTION OUTPUTS PROVIDED BY EVALUATOR (artifacts[" cases"] per case): - input_features: static circuit features (depth, qubits, gates, etc.) num_connected_components, number_instructions, num_measurements, num_cnot_gates, program_communication, liveness, parallelism, measurement, entanglement_ratio, critical_depth - qose_input_size: initial target size before costsearch adjustments - qose_depth, qos_depth, qose_cnot, qos_cnot, qose_run_sec, qos_run_sec: paired QOSE vs baseline QOS metrics - qose_output_size: final target size chosen by the algorithm - qose_method: chosen method ("GV" or "WC") - qose_gv_cost_trace, qose_wc_cost_trace: cost estimates per probe step - qose_gv_time_trace, qose_wc_time_trace: time per probe step POSSIBLE LEVERS TO EXPLORE:

Lily Jiaxin Wan, Deming Chen, Klara Nahrstedt, and Bo Chen A) Decide to use WC or GV selectively instead of always using both. B) Explore size_to_reach efficiently instead of using two while loops iteratively. C) Explore early stopping rules for WC and GV. D) Skip certain size_to_reach values based on heuristics. evaluator_system_message: "You are a strict code reviewer. Do not add commentary." num_top_programs: 3 num_diverse_programs: 2 use_template_stochasticity: true # Keep artifact injection enabled by default. include_artifacts: true max_artifact_bytes: 65536 artifact_security_filter: true database: db_path: null in_memory: true log_prompts: true population_size: 100 archive_size: 20 num_islands: 3 migration_interval: 30 migration_rate: 0.12 elite_selection_ratio: 0.05 exploration_ratio: 0.65 exploitation_ratio: 0.30 feature_dimensions: - "complexity" - "diversity" feature_bins: 12 diversity_reference_size: 30 evaluator: timeout: 12000 max_retries: 1 cascade_evaluation: false parallel_evaluations: 1 use_llm_feedback: false llm_feedback_weight: 0.0 evolution_trace: enabled: false format: "jsonl" include_code: false include_prompts: false output_path: null buffer_size: 10 compress: false

3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47

Listing 3. Initial Program for Error Mitigation.

2

from qos.error_mitigator.run import compute_gv_cost, compute_wc_cost from qos.types.types import Qernel

# OE_BEGIN gv_timeout_sec = 0 wc_timeout_sec = 0 gv_cost, _gv_timed_out = compute_gv_cost(q, size_to_reach, timeout_sec=gv_timeout_sec) wc_cost, _wc_timed_out = compute_wc_cost(q, size_to_reach, timeout_sec=wc_timeout_sec) while (gv_cost <= budget or wc_cost <= budget) and size_to_reach > 2: size_to_reach = size_to_reach - 1 gv_cost, _gv_timed_out = compute_gv_cost( q, size_to_reach, timeout_sec=gv_timeout_sec ) wc_cost, _wc_timed_out = compute_wc_cost( q, size_to_reach, timeout_sec=wc_timeout_sec ) while gv_cost > budget and wc_cost > budget: size_to_reach = size_to_reach + 1 gv_cost, _gv_timed_out = compute_gv_cost( q, size_to_reach, timeout_sec=gv_timeout_sec ) wc_cost, _wc_timed_out = compute_wc_cost( q, size_to_reach, timeout_sec=wc_timeout_sec ) method = "GV" if gv_cost <= wc_cost else "WC" # OE_END return size_to_reach, method

Listing 4. Best Program by Gemini-3-Pro for Error Mitigation 1

1

def evolved_cost_search(self, q: Qernel, size_to_reach: int, budget: int): metadata = q.get_metadata() depth = metadata.get("depth", 0) num_qubits = metadata.get("num_qubits", 0) num_clbits = metadata.get("num_clbits", 0) num_nonlocal_gates = metadata.get("num_nonlocal_gates", 0) num_connected_components = metadata.get(" num_connected_components", 0) number_instructions = metadata.get("number_instructions", 0) num_measurements = metadata.get("num_measurements", 0) num_cnot_gates = metadata.get("num_cnot_gates", 0) program_communication = metadata.get(" program_communication", 0.0) liveness = metadata.get("liveness", 0.0) parallelism = metadata.get("parallelism", 0.0) measurement = metadata.get("measurement", 0.0) entanglement_ratio = metadata.get("entanglement_ratio", 0.0) critical_depth = metadata.get("critical_depth", 0.0)

2 3 4

from qos.error_mitigator.run import compute_gv_cost, compute_wc_cost from qos.types.types import Qernel def evolved_cost_search(self, q: Qernel, size_to_reach: int, budget: int):

Toward Quantum Software Automation: A Quantum-Aware Harness for LLM-Guided Evolution

5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59

# OE_BEGIN # Optimization Strategy: # 1. Goal: Minimize 's' (subcircuit size) such that Cost (s) <= budget. # Smaller 's' -> more cuts -> better fidelity proxy ( lower depth/CNOTs). # 2. Search: Binary search on [2, num_qubits]. # 3. Efficiency: # - Prioritize GV (usually faster). # - Skip WC during search if GV is clearly feasible (<= 0.7 * budget). # - Use tight timeouts (0.2s) during search to fail fast on bad cuts. # - Use relaxed timeouts for final verification to ensure stability. metadata = q.get_metadata() num_qubits = metadata.get("num_qubits", 0) # Valid range: [2, num_qubits] n = max(2, int(num_qubits)) # Heuristic: Very small circuits rarely benefit from cutting overhead. if n <= 4: return n, "GV" # Cache: size -> (gv_cost, wc_cost) cache = {} def get_costs(s, strict=False): if s in cache: # If we need strict but cached result has 'inf' for WC, we might ideally re-run. # But for speed, we rely on the cache unless manually cleared/handled. return cache[s]

60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96

# Timeouts: Tight for search, longer for final check t_out = 0.5 if strict else 0.2 # 1. Check GV g_res = compute_gv_cost(q, s, timeout_sec=t_out) if isinstance(g_res, tuple): g_val = float(g_res[0]) g_to = bool(g_res[1]) else: g_val = float(g_res) g_to = False if g_to: g_val = float('inf') # 2. Check WC # Optimization: Skip WC in search mode if GV is clearly feasible. w_val = float('inf') check_wc = True if not strict and g_val <= (float(budget) * 0.7): check_wc = False if check_wc: w_res = compute_wc_cost(q, s, timeout_sec=t_out) if isinstance(w_res, tuple): w_val = float(w_res[0])

97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117

w_to = bool(w_res[1]) else: w_val = float(w_res) w_to = False if w_to: w_val = float('inf') cache[s] = (g_val, w_val) return g_val, w_val # Binary search for Minimum Feasible Size low = 2 high = n best_feasible_size = n # Pre-check max size to ensure we have a valid fallback. # Max size is usually feasible (cost ~0 or 1). g_n, w_n = get_costs(n, strict=False) if min(g_n, w_n) > budget: return n, ("WC" if w_n < g_n else "GV") while low <= high: mid = (low + high) // 2 g, w = get_costs(mid, strict=False) if min(g, w) <= budget: # Feasible. Try smaller. best_feasible_size = mid high = mid - 1 else: # Infeasible. Need larger. low = mid + 1 final_size = best_feasible_size # Final Decision Phase # Retrieve costs. If we skipped WC or timed out during search, check properly now. g_final, w_final = cache.get(final_size, (float('inf'), float('inf'))) # Heuristic: When to re-check WC? # If we skipped it (inf) AND GV is expensive (> 50% budget). # If GV is cheap, WC is unlikely to be significantly better to justify the cost. recheck_wc = False if w_final == float('inf') and g_final > (float(budget) * 0.5): recheck_wc = True if recheck_wc: w_res = compute_wc_cost(q, final_size, timeout_sec =0.5) if isinstance(w_res, tuple): val = float(w_res[0]) if not w_res[1]: w_final = val else: w_final = float(w_res) # Select best method based on cost final_method = "GV" if w_final < g_final and w_final <= budget: final_method = "WC" elif g_final <= budget:

Lily Jiaxin Wan, Deming Chen, Klara Nahrstedt, and Bo Chen

118 119 120 121 122 123 124

final_method = "GV" else: # Fallback if both over budget final_method = "WC" if w_final < g_final else "GV"

You may receive evaluator artifacts from previous iterations. Use them to infer what kinds of pairs your current scoring function is selecting and how those selections align with strong Pareto rank. - rank_distribution_csv: Shows how often each Pareto rank appears overall and within your selected Top-K pairs. Use this to see whether your score is concentrating mass on strong ranks. - top_pairs_metrics_csv: Shows the Top-K pairs chosen by your score together with effective utilization, Pareto metric values, Pareto rank, and score. Use this to understand which pair characteristics your current formula is rewarding. - top_rank_pairs_all_columns_csv: Expanded pair-level table for top-ranked pairs. Use this to inspect which metadata patterns tend to appear in strong candidates.

# OE_END return final_size, final_method

Listing 5. OpenEvolve Configuration for Multiprogramming max_iterations: 100 checkpoint_interval: 100 log_level: "INFO" log_dir: null random_seed: 42 max_tasks_per_child: 1

CONTEXT: The goal is to evolve a scoring function that maximizes the evaluator's objective (avg_rank, corr, or combined) by selecting optimal circuit pairs for multiprogramming based on effective_utilization and fidelity. num_top_programs: 1 num_diverse_programs: 1 use_template_stochasticity: true include_artifacts: true

diff_based_evolution: false max_code_length: 30000 early_stopping_patience: null convergence_threshold: 0.001 early_stopping_metric: score llm: api_base: "${OPENAI_API_BASE}$" api_key: "${OPENAI_API_KEY}$" primary_model: "${OPENAI_MODEL}$" temperature: 0.7 top_p: 0.9 max_tokens: 16384 timeout: 450 retries: 0 retry_delay: 5 reasoning_effort: low prompt: system_message: You are optimizing ONLY the function get_matching_score( self, q1, q2, backend, weighted=False, weights=[]) in evaluation/openevolve_pairing/target.py. OUTPUT FORMAT (CRITICAL): * Return the full file contents of evaluation/ openevolve_pairing/target.py. * Output code only: no explanations, no markdown fences, no extra text. FUNCTION CONTRACT: * Do NOT change the function signature. It MUST remain: get_matching_score(self, q1, q2, backend, weighted= False, weights=[]) * It MUST return a single float score (higher is better). * Keep it deterministic (no randomness or external state ). METADATA KEYS (from q1.get_metadata()/q2.get_metadata()) : * depth, num_qubits, num_clbits, num_nonlocal_gates, num_connected_components, number_instructions, num_measurements, num_cnot_gates, program_communication, liveness, parallelism, measurement, entanglement_ratio, critical_depth. ARTIFACTS:

database: db_path: null population_size: 100 archive_size: 20 num_islands: 3 migration_interval: 30 migration_rate: 0.12 exploration_ratio: 0.65 exploitation_ratio: 0.3 evaluator: timeout: 4800 max_retries: 1 parallel_evaluations: 1 use_llm_feedback: false

Listing 6. Initial Program for Multiprogramming. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

def get_matching_score(self, q1, q2, backend, weighted: bool = False, weights = []) -> float: """Replacement␣for␣Multiprogrammer.get_matching_score."" " meta1 = q1.get_metadata() meta2 = q2.get_metadata() depth1 = meta1.get("depth", 0) depth2 = meta2.get("depth", 0) qubits1 = meta1.get("num_qubits", 0) qubits2 = meta2.get("num_qubits", 0) nonlocal1 = meta1.get("num_nonlocal_gates", 0) nonlocal2 = meta2.get("num_nonlocal_gates", 0) cnot1 = meta1.get("num_cnot_gates", 0) cnot2 = meta2.get("num_cnot_gates", 0) meas1 = meta1.get("num_measurements", 0)

36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70

depth_ratio = min(depth1, depth2) / max(depth1, depth2, 1) qubit_ratio = min(qubits1, qubits2) / max(qubits1, qubits2, 1) nonlocal_ratio = min(nonlocal1, nonlocal2) / max( nonlocal1, nonlocal2, 1) cnot_ratio = min(cnot1, cnot2) / max(cnot1, cnot2, 1) meas_ratio = min(meas1, meas2) / max(meas1, meas2, 1) instr_ratio = min(instr1, instr2) / max(instr1, instr2, 1) instr_diff = abs(instr1 - instr2) instr_sum = instr1 + instr2 instr_diff_norm = instr_diff / max(instr_sum, 1) cc_ratio = min(cc1, cc2) / max(cc1, cc2, 1) crit_ratio = min(crit1, crit2) / max(crit1, crit2, 1e-9) liveness_avg = (liveness1 + liveness2) / 2.0 prog_comm_avg = (prog_comm1 + prog_comm2) / 2.0 parallel_avg = (parallel1 + parallel2) / 2.0 meas_ratio_avg = (meas_ratio1 + meas_ratio2) / 2.0 ent_ratio_avg = (ent_ratio1 + ent_ratio2) / 2.0 depth_sim = self.depthComparison(q1, q2) util_eff = self.effective_utilization(q1, q2, backend) util_eff_norm = util_eff / 100.0 entanglementDiff = self.entanglementComparison(q1, q2) measurementDiff = self.measurementComparison(q1, q2) parallelismDiff = self.parallelismComparison(q1, q2) # OE_BEGIN if weighted and sum(weights) > 0: return ( weights[0] * util_eff + weights[1] * entanglementDiff + weights[2] * measurementDiff + weights[3] * parallelismDiff ) # OE_END base_score = (util_eff_norm + entanglementDiff + measurementDiff + parallelismDiff) / 4.0 return base_score

Listing 7. Best Program by Gemini-3-Pro for Multiprogramming

Relative to Qiskit

QAOA-R3 BV

GHZ HS-1

QAOA-P1 QSVM

0.44 0.64 0.90

(a) Depth - 24 qubits

1.0

0.39

0.5 0.0

QSA QOS CutQC Frozen

1.0

TL-1 VQE-1

W-STATE

(b) CNOT - 24 qubits 0.36

0.42 0.64 0.92

0.5 0.0

QSA QOS CutQC Frozen

Figure 36. Static metrics comparison for error mitigation on 24-qubit circuits. Lower is better. 1 2 3 4 5 6 7

def get_matching_score(self, q1, q2, backend, weighted: bool = False, weights = []) -> float: """Replacement␣for␣Multiprogrammer.get_matching_score."" " d1 = q1.get_metadata().get("depth", 0) d2 = q2.get_metadata().get("depth", 0) util = self.effective_utilization(q1, q2, backend) / 100.0 depth_ratio = min(d1, d2) / max(d1, d2, 1) return util * 0.65 + depth_ratio * 0.35

F

QSA’s Impact on Circuit Properties

Figure 36 compares the relative depth and CNOT values of QSA and the baselines across circuits of different scales. Compared to the SOTA QOS, QSA consistently reduces both relative depth and relative CNOT by up to 0.06. These reductions help explain why QSA is able to maintain, and in some cases even improve, fidelity.

G

Additional Ablation Studies 0.658 0.631 0.650

1.00

0.428

0.75 0.50 0.25 0.00

0.041

1111 0111 0011 0001 0000

Figure 37. Impact of artifact composition (similar depth/cnot).

0.679

1.00 Time Ratio

35

meas2 = meta2.get("num_measurements", 0) instr1 = meta1.get("number_instructions", 0) instr2 = meta2.get("number_instructions", 0) cc1 = meta1.get("num_connected_components", 0) cc2 = meta2.get("num_connected_components", 0) liveness1 = meta1.get("liveness", 0.0) liveness2 = meta2.get("liveness", 0.0) prog_comm1 = meta1.get("program_communication", 0.0) prog_comm2 = meta2.get("program_communication", 0.0) parallel1 = meta1.get("parallelism", 0.0) parallel2 = meta2.get("parallelism", 0.0) meas_ratio1 = meta1.get("measurement", 0.0) meas_ratio2 = meta2.get("measurement", 0.0) ent_ratio1 = meta1.get("entanglement_ratio", 0.0) ent_ratio2 = meta2.get("entanglement_ratio", 0.0) crit1 = meta1.get("critical_depth", 0.0) crit2 = meta2.get("critical_depth", 0.0)

Time Ratio

16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34

Relative to Qiskit

Toward Quantum Software Automation: A Quantum-Aware Harness for LLM-Guided Evolution

0.75 0.50 0.25 0.00

0.041

0.040

Low

Medium

High

Figure 38. Impact of thinking levels.

Ablation of Prompt Composition for Error Mitigation Evolution. In Figure 37, we compare different prompt compositions on Gemini-3-Flash-Preview, the most cost-effective one. We use four digits to represent the availability of 1) seed program, 2) snapshot (per-circuit), 3) snapshot (summary), 4) static analysis. When part of the prompt disappears, the mitigation time increases. The mitigation time plateaus when the seed program and the snapshot (per-circuit) are removed. This analysis showcases the significance of the prompt components. Impact of Thinking Level on Error Mitigation Evolution. We analyze the impact of thinking levels on evolution using Gemini 3 Flash by configuring the thinking level in the API. It turns out that a higher thinking level does

Lily Jiaxin Wan, Deming Chen, Klara Nahrstedt, and Bo Chen

0.76

96.2

0.75

Fidelity Low

96.2

Norm. Util.

Medium

Thinking Level

0.9

Fidelity Norm. Util.

0.8

98

0.7

Mean

High

100

8

16

Total Qubit Count

24

96

Norm. Util. (%)

96.3

Fidelity

Fidelity

0.77

Norm. Util. (%)

not lead to a better result. Figure 39 shows that the fidelity and normalized utilization decreases with the thinking level. Figure 38 shows that medium thinking level achieves comparable time ratio as the low thinking level while the high thinking level drastically increases the time ratio (lower the better). We speculate the reason is that the model produces over-complicated solutions at higher thinking levels that fail to make improvements. In contrast, a low thinking level makes incremental but steady progress in each iteration, which benefits more from the evolution framework.

Figure 39. Analysis of Impact Figure 40. Analysis of Impact of Thinking Level. of Utils Used in Evolution.

96.3

0.77

Norm. Util.

Fidelity

0.78

96.2

0.77 0.76 1% 5%

Fidelity 96.1 Norm. Util. 10% 15% 20%

Top Percentage

Cost (USD)

Impact of Utilizations Used in Multiprogramming Evolution. In Figure 40, we compare the qubit counts (8,16,24) we use for evolution. It will impact both the calculation of the target optimization score, and the artifacts. While single-utilization targets (e.g., 30%) prioritize specific metrics, adopting the Mean across all levels yields the most robust generalization and balanced trade-off between fidelity and utilization. 1.0 0.5 $0.48 0.0

1%

$0.58 $0.72

$0.91

$1.11

5% 10% 15% 20% Top Percentage

(a) Ablation Studies across the (b) The cost of using different percentage used for evolution. percentage for evolution.

Figure 41. Analysis of impact on the percentage used for the evolution. Impact of Percentage Used in Multiprogramming Evolution. We compare the percentage used to compose both the target optimization score, and artifacts during the code evolution. As Figure 41 shows, top 1% and 5% percentage can achieve the best fidelity and normalized utilization, while the cost of 1% is only half the price of 5% using Gemini 3 Flash. In conclusion, from the monetary efficiency point of view, we would pick up the option of 1% percentage used in evolution.

H

Fidelity Approximation Details

Direct fidelity estimation is often computationally infeasible for complex circuits. Consequently, these approximation metrics serve as objective functions to surrogate for fidelity. Table 4 summarizes these fidelity approximation along with

their corresponding physical rationales. We evaluate them as the optimization target in OpenEvolve on Figure 25 in the main paper.

I

Optimization Baseline Details

We compare QSA against three conventional optimization strategies to determine whether its gains can be achieved by optimizing the same error-mitigation decisions without evolving program logic. All methods choose from the same output decision space: the target circuit fragment size and the cutting method (GV or WC). They are evaluated on the same workloads using qiskit quantum simulator, under a 60 s search deadline. Random. Random search uniformly samples valid fragmentsize and cutting-method configurations and retains the best configuration observed within the deadline. Bayesian Optimization. BO treats fragment size and cutting method as optimization variables and sequentially selects configurations using a constrained Gaussian-process surrogate. After evaluating each configuration, the observed reward is used to update the surrogate and select the next candidate until the deadline is reached. Reinforcement Learning. RL learns a feature-based policy that maps circuit characteristics to a fragment size and cutting method. We train the policy using the same circuit coreset used by QSA, so that both methods receive comparable workload coverage during optimization. At deployment, the learned policy directly selects an action from this fixed feature–action representation. Ours. Unlike these baselines, QSA does not optimize only the final configuration. Instead, the LLM mutates the underlying error-mitigation program and can change the control logic used to explore fragment sizes and cutting methods. For example, the evolved program can alter the search order, reuse intermediate results, or terminate unproductive searches early, while remaining subject to the same execution deadline. This comparison therefore isolates the benefit of program-level evolutionary optimization from conventional search over predefined decisions.

J

Additional Discussions

Robustness and limitations. QSA’s robustness to hardware calibration drift depends on the metrics. Mitigation time and normalized utilization are mainly determined by the evolved software policy running on classical computers and are therefore less sensitive to calibration changes. In contrast, fidelity may vary with the hardware noise level, and higher the noise, lower the fidelity gain. Regarding new workloads, our experimental results show that QSA can generalize to existing topologies with varying scales. Extension to quantum error correction. For quantum error correction, QSA can evolve decoder implementations or decoding schedules. Weakness analysis can identify the code

Toward Quantum Software Automation: A Quantum-Aware Harness for LLM-Guided Evolution

Approximation Metric

Rationale

Circuit Depth Instruction Count CNOT Count Non-local Gates Measurement

Represents critical path length; increased depth correlates with decoherence and gate errors. Denotes total gate count, reflecting the overall computational load of the quantum circuit. Quantifies error-prone entangling gates, which are primary noise sources in NISQ devices. Dictates the requirement for qubit routing and SWAP [62] insertions, impacting execution efficiency. Reflects measurement operation count, contributing to readout errors and temporal overhead.

Table 4. Summary of Performance Proxies in QSA. paths that dominate logical error rate or decoding latency, while the reward can directly combine these metrics with resource overhead. The evolution loop can then search for

policies that improve decoding quality without violating real-time latency constraints.

Related documents

• Toward Quantum Software Automation: A Quantum-Aware Harness for LLM-Guided Evolution#1143484
• DreamQAS: Learning a Decision-Useful World Model for VQE-Efficient Quantum Architecture Search#422252
• Quantum computing machine learning module — Accenture Global Solutions Limited (US11803772B2)#607109
• Qlippy: A Retrieval-Augmented GenAI Assistant for Reproducible Quantum Workflows and Experiment Tracking#971994
• Benchmarking API Drift in LLM-Generated Quantum Code Across Successive SDK Versions#343565