Conceptio › Archive › arXiv CS

arXiv CSopen access

A semantic mutation metric for metamorphic relation adequacy in scientific computing programs

2026 · arxiv_cs

arXiv CS · Papers · License: Open Access · 2026

Open Source ↗Direct PDF ↓

software-architecture software-engineering testing

software engineering, software architecture, testing

Highlights A semantic mutation metric for metamorphic relation adequacy in

arXiv:2605.17437v1 [cs.SE] 17 May 2026

scientific computing programs Meng Li, Xiaohua Yang, Jie Liu, Shiyu Yan • Five semantic mutation operators degenerate to classical MS in a proven limit • 12 PUTs, 60 cells: 5.14% AST overlap between LLM and cosmic-ray mutants • HP, SI, TF classes unreachable under default first-order syntactic tools • Pre-registered large-effect threshold not met; observed effect is mediumsized • Cross-source LLM pooling does not appreciably shift Cliff’s δ

A semantic mutation metric for metamorphic relation adequacy in scientific computing programs Meng Lia,b,c,∗, Xiaohua Yanga,b,c , Jie Liua,b,c , Shiyu Yana,b,c a

School of Computing, University of South China, Hengyang, 421001, China Hunan Engineering Research Center of Software Evaluation and Testing for Intellectual Equipment, Hengyang, 421001, China c CNNC Key Laboratory on High Trusted Computing, Hengyang, 421001, China

Abstract Context. Metamorphic Testing addresses the test-oracle problem in scientific computing, but classical Mutation Score operates on syntactic AST mutations and misses domain semantics. Objective. We propose Semantic Mutation Score (SMS), built on five domain-semantic operators (Conservation Erosion, Operator Substitution, Hyperparameter, Trajectory Flip, Structural Injection); SMS degenerates almost everywhere to MS in a characterised limit, so any SMS-based conclusion remains consistent with prior mutation-testing literature in the classical regime. Method. A 12-PUT × 5-MP design over four single-output float → float classes (numeric / probabilistic / surrogate / ML) is paired with a three-layer attribution classifier separating true semantic faults from tolerance, OOD, statistical, and artefact categories. A same-source / cross-source ablation under an identical prompt isolates the LLM-source-diversity contribution. LLM-generated mutants are compared against a default-configuration cosmic-ray syntactic pool at the ∗

Corresponding author. Email address: [email protected] (Meng Li)

AST-normalised level. Results. The pre-registered large-effect threshold for Cliff’s δ (Romano 2006) is not met under the point-estimate criterion; the observed effect lies in the medium-effect range, and we frame the corresponding hypothesis as an underpowered exploratory contribution to be re-examined at larger sample size in a follow-up study. Cross-source pooling under an identical prompt does not appreciably shift δ, indicating that LLM identity is not the lever on the aligned-versus-cross effect size within this design. AST-level overlap between LLM-generated and default cosmic-ray syntactic mutants is small; the Hyperparameter, Structural Injection, and Trajectory Flip classes are unreachable under default first-order syntactic configurations, while higher-order compositions (Jia and Harman 2008) are not refuted and are reserved as a residual threat. Conclusion. LLM-source diversity under an identical prompt does not appreciably move δ within this design; the strong-sense test (per-LLM differential prompts and a larger PUT sample) is deferred to follow-up work. The first-order unreachability evidence is independent of the effect-size question. SMS is a backward-compatible adequacy metric for domain-semantic metamorphic-relation sets in scientific computing. Keywords: metamorphic testing, mutation testing, semantic mutation operators, metamorphic relation adequacy, LLM-generated mutants, Cliff’s delta, scientific computing kernels 1. Introduction 1.1 Motivation and scope Metamorphic Testing (MT) addresses the test-oracle problem in scientific computing software: instead of checking outputs against a known-correct ref2

erence, MT checks metamorphic relations (MRs), invariants connecting outputs of related inputs. While semantic mutation testing has been named since Clark, Dan, and Hierons (2010) and tooled for C (Dan and Hierons 2012) and for UML state-machine behaviour (Derezińska and Zaremba 2019), and while MR-adjacent data-semantic mutation (Sun et al. 2016, 2024; Zhu et al. 2020; Chan and Keung 2024) and recent LLM semantic-invariance MT (Curtò and Zarzà 2025) extend the mutation target to test data, relations, and task-level semantics, the adequacy of an MR set against domain-specific code-level faults, conservation laws, monotonicity, convergence order, trajectory shape, fidelity ordering, has lacked a backward-compatible metric tied to the classical Mutation Score (MS) lineage. Jia and Harman’s (2011) MS is defined over syntactic Abstract Syntax Tree (AST) mutations and does not capture these domain semantics. Recent Large Language Model (LLM) generated mutants (Tip, Bell, and Schäfer 2024; Humbatova, Jahangirova, and Tonella 2021) do produce semantically richer mutants, but do not separate the contribution of LLM source diversity from the contribution of MR design. Throughout this paper we use the term four representative classes of single-output scientific computing kernels (numeric, probabilistic, surrogate, and machine-learning), and avoid the ambiguous software-engineering term paradigm. The scope of our claims is strictly bounded to single-output float → float kernels: each Program Under Test (PUT) is under 2 KB, and broader industrial transfer is reserved for the P3 and P5 companion papers.

1.2 Three-layer methodological framework We organise the contribution as a three-layer framework around domainsemantic mutation operators. • Layer 1, Definitional (§3.2). A mutation is semantic if it satisfies at least one of three necessary conditions: (a) it crosses a functioncall or module-import boundary, (b) it depends on domain knowledge for legality, or (c) it changes the algorithmic class. The five metaoperator classes, Conservation Erosion (CE, also written mut_C), Operator Substitution (OS, mut_M), Hyperparameter (HP, mut_G), Trajectory Flip (TF, mut_T), and Structural Injection (SI, mut_F), specialise these conditions across the four PUT classes. • Layer 2, Operational (§2.3, §4.3). An equivalence judgement E1 ∧ E2, where E1 is Automated Verification Pipeline (AVP) coherence and E2 is output equivalence on K_eq = 1000 samples, gives the conservative instantiation of the Layer-1 conditions. The trade-off against E1-alone and E2-alone variants is in Appendix A.3. • Layer 3, Applied (§3.5). AST-normalised empirical traceability across all 12 PUTs: comparing 292 v4 mutants against 1,250 cosmicray syntactic mutants, we show empirically that the v4 pool is not a subset of the syntactic-mutant pool. The 60-cell empirical audit reported in Section 5 is one demonstration following this backbone, not the paper’s main contribution. SMS is positioned as a strict generalisation of classical MS: under the degenerate limit L = L_equiv ∧ L_killed ∧ L_mut formalised in §2.6, SMS reduces almost everywhere to MS. 4

Figure 1: Three-layer methodological framework: Layer 1 (definitional necessary conditions), Layer 2 (operational E1 ∧ E2 judgement), Layer 3 (applied AST-normalised traceability across 12 PUTs).

1.3 Related work and roadmap This paper is P2 in a five-paper roadmap. P1 audits MR meta-patterns on the same 12-PUT infrastructure (Meng Li et al., Progress in Nuclear Energy, under review). P4 is the unified-theory companion (minimal MRsubset existence and three-pillar coupling). P5 and P2-CN cover regulatory and engineering transfer. Three lines of prior work bracket the contribution of this paper. (a) Classical mutation testing. Jia and Harman (2011) and Papadakis et al. (2019) survey syntactic mutation. The Coupling Effect Hypothesis (CPH), that tests detecting simple faults also detect complex faults, underpins the use of MS as a fault proxy (DeMillo, Lipton, and Sayward 1978; Andrews, Briand, and Labiche 2005; Just et al. 2014). We argue in §3.5 that CPH holds within syntactic mutation but does not automatically extend across the syntactic-versus-domain-semantic boundary, even when it couples simple syntactic faults to complex syntactic faults. The §3.5 empirical evidence (5.14% AST overlap on 12 PUTs, with HP, SI, and TF at 0/0/0) is the empirical witness for this boundary. (b) Higher-order mutation. Jia and Harman (2009) and Kintis et al. (2018) study compositions of first-order syntactic mutants. We explicitly exclude any equivalence claim about Higher-Order Mutation (HOM) and list HOM as residual threat R12 (Appendix F.1). (c) LLM-generated mutants. Tip, Bell, and Schäfer (2024) introduce LLMorpheus, which uses single-LLM JavaScript mutants. Humbatova, Jahangirova, and Tonella (2021) introduce DeepCrime for deep-learning realfault mutation. Moradi Dakhel et al. (2024) extend LLM-driven mutation

testing to test generation on Java PUTs, providing an Information and Software Technology anchor for the LLM-mutant lineage. To our knowledge no prior LLM-mutant work isolates LLM source diversity from MR design in the contribution to effect size; the three-stage ablation in §4.2 closes this gap. On the equivalent-mutant problem, Delgado-Pérez and Chicano (2020) emphasise the importance of the equiv term in the MS denominator; we extend that classical bitwise-equivalence definition to a semantic-class equivalence E1 ∧ E2 in §2.3. Zhang et al. (2021) demonstrate MT validation in class-integration test ordering, a non-scientific-computing domain, and the present paper specialises MT to scientific-computing PUTs and couples it to a domain-semantic mutation operator framework. (d) Semantic mutation operators across testing communities. Semantic mutation as a research line is older than the LLM-mutant lineage. Clark, Dan, and Hierons (2010) introduce semantic mutation testing as a path distinct from syntactic mutation; Dan and Hierons (2012) tool the approach for C; Derezińska and Zaremba (2019) apply semantic mutation operators to UML state-machine behaviour by perturbing semantic-variation-point interpretations rather than syntactic structure. Within MT specifically, Sun et al. (2016, 2024) propose data mutation operators and the µMT methodology to drive MR acquisition through controlled data-semantic perturbation; Zhu et al. (2020) formalise datamorphic testing, separating datamorphisms (test-case-to-test-case mappings) from metamorphisms (test-case-toBoolean mappings) over data semantics; Chan and Keung (2024) extend generic data mutation operators to unsupervised software-defect-prediction validation. Most recently, Curtò and Zarzà (2025) formalise semantic in-

variance in LLM MT under paraphrase, fact-reorder, expand/contract, and context-shift transformations. These works establish that mutation can target data, relation, task, and behavioural semantics. The contribution of this paper is the first to formalise a domain-specific Semantic Mutation Score (SMS) for MR adequacy in scientific computing kernels, with a proven backward-compatible reduction to classical MS (Theorem 9.1) and a code-level operator framework (Conservation Erosion, Operator Substitution, Hyperparameter, Trajectory Flip, Structural Injection) targeting kernel-specific faults. A numerical-coincidence note. Petrović and Ivanković’s (2018) ~20% “productive-mutant” rate at Google is numerically close to our LRCAcalibrated C1_share of 0.20. The two constructs differ, developer survey (subjective) versus three-layer classifier (output), so the agreement is a contextual numerical coincidence rather than mechanism validation, as we discuss in §6.1. The SMS metric is conceptually complementary to the code-verification scope of ASME V&V 20-2009 §3, which targets numerical-solver correctness. SMS targets MR fault-detection adequacy; we make no normative compliance claim (Appendix E.3 records this as a long-term aspiration only). 1.4 Research questions • RQ1 Distributions of inst_rate, equiv_rate, C1_share, survive_rate over 60 cells. • RQ2 SMS difference structure between operator-MP aligned (j=k) and cross (j̸=k) slices. • RQ3 Cross-class consistency across 4 program classes × 5 operators. 8

• RQ4 Empirical relationship between SMS and Pattern Coverage (descriptive only at n = 12; no formal test; pre-registered as a P4 hypothesis-generating observation). 1.5 Hypotheses • H1. At least 4 of the 5 operators produce ≥ 5 non-equivalent mutants on at least 9 of the 12 PUTs. • H2. The aligned-SMS to cross-SMS odds ratio is ≥ 3.0, and Cliff’s δ is ≥ 0.474. • H3.

A within-class sign test gives 4/4 across the 4 classes, and

CV(∆SMS) < 0.5. • H4. Mean suspect_share is ≤ 0.20 across the 60 cells. 1.6 Boundary between P2 and the companion papers P2 contributes the tool and the empirical report on 12 PUTs and 60 cells. P4 will contribute formal theorems (minimal MR-subset existence, reachable adequacy, three-pillar coupling). The 12 PUTs are deliberately Numerical Recipes–style toy kernels: they provide a verifiable minimum working example for the three-layer backbone, and industrial-scale transfer is reserved for P5. 2. Notation and Equivalence Judgement 2.1 Vocabulary inheritance Seven classical mutation-testing concepts are inherited verbatim from Jia & Harman (2011) and Ammann & Offutt (2008) - PUT (S_i), mutation operator (mut_j ∈ MUT), mutant (s' ∈ mut_j(S_i)), equivalent mutant (equiv_{i,k,j}), killed mutant (killed_{i,k,j}), surviving mutant 9

(survive_{i,k,j}), and Mutation Score. The SMS formula structure is preserved, with extension applied only to the internal definitions of mut, equiv, and killed. Three-state decomposition (mutually exclusive and exhaustive):

mutj (Si ) = equivi,k,j ∪ killedi,k,j ∪ survivei,k,j SMSi,k,j :=

(disjoint)

The complete notation table is in Appendix A.1; index sets and tolerance parameters (ε_eq, ε_AVPˆk, K_eq = 1000, N = 20) follow that table. 2.2 Operator signatures Operator family and alignment:

MUT = {mutC , mutM , mutG , mutT , mutF } (open, extensible) mutj : Programs → 2Programs align(j) = j

(design choice, not theorem)

Table 1: Operator signatures.

Operator

Failure semantics

Aligned MP

mut_C

Conservation-breaking

MP_1 Conservation

mut_M

Monotonicity-breaking

MP_2 Monotonicity

mut_G

Convergence-breaking

MP_3 Convergence

Table 1: Operator signatures.

Operator

Failure semantics

Aligned MP

mut_T

Trajectory-distorting

MP_4 Trajectory

mut_F

Fidelity-order-breaking

MP_5 Partial-order

Per-PUT specialisation tables (mut_C/M/G/T/F across 12 PUTs and 4 classes) are deferred to Appendix B.2. 2.3 Equivalence judgement E1 ∧ E2 The equivalence judgement is the executable instantiation of the Layer-1 necessary conditions (§3.2.0). For each candidate mutant s': • E1 (AVP-coherent): ∀ mr ∈ MR_{i,k}: AVP(S_i, mr) = AVP(s’, mr). • E2 (Output-equivalent):

∀ x in K_eq=1000 samples ~ D_S:

∥S_i(x) − s’(x)∥ ≤ ε_eq. E1 ∧ E2 is the conservative complete instantiation: false-equiv requires both AVP coherence and numerical agreement on K_eq samples to fail simultaneously, which is rare; false-non-equiv requires only one of E1, E2 to fail, biasing SMS slightly high. The trade-off table against E1-alone and E2-alone is in Appendix A.3. Under the §2.6 degenerate limit L, E1 ∧ E2 reduces almost everywhere to classical bitwise equivalence (Lemma 9.1). The killed determination preserves the classical OR-aggregation:

killed(s′ , MRi,k ) ⇐⇒ ∃ mr ∈ MRi,k : AVP(Si , mr) = pass∧AVP(s′ , mr) = fail 11

2.4 LRCA engineering attribution layer LRCA annotates every killed mutant with one of five likely root causes: C1 true semantic failure, C2 numerical-tolerance perturbation, C3 out-ofdistribution (OOD) trip, C4 statistical-assumption violation, and C5 mutator artefact. The classifier is a three-layer decision tree. Layer 1 checks tolerance robustness over N = 20 repeats with a fail-ratio cutoff of 0.80. Layer 2 triages OOD on the surrogate and machine-learning classes. Layer 3 checks statistical-assumption baselines on the probabilistic and machine-learning classes using Wilcoxon and Dynamic Time Warping. A final pass rechecks for mutator artefacts. Multiple labels are resolved by priority C5 > C4 > C3 > C2 > C1. The output quantities are C1_share (share of C1 among killed mutants) and suspect_share := 1 − C1_share. LRCA does not modify the SMS formula: the killed set is never filtered by suspect status. LRCA is a descriptive overlay that tells the reader whether a given kill is likely a true semantic-failure detection or an artefact. Appendix A.2 gives the full decision tree, the 9-grid threshold calibration, and the engineering rationale for the priority ordering. 2.5 Backward-compatibility declaration The seven core concepts (PUT, mutation operator, mutant, equivalent mutant, killed mutant, surviving mutant, mutation score) and the SMS formula |killed| / (|mut| − |equiv|) are aligned item-for-item with Jia and Harman (2011). We extend only the internal definitions: mut shifts from syntactic to domain-semantic operators, equiv shifts from bitwise behavioural equivalence to the semantic-class equivalence E1 ∧ E2, and killed 12

shifts from an equality oracle to a meta-pattern AVP. We introduce no new formula terms and no new state classifications. Under the degenerate limit L defined in §2.6, SMS reduces almost everywhere to classical syntactic MS, equivalent mutants degenerate to classical behavioural equivalence, killed mutants degenerate to bitwise difference detection, and the LRCA layer trivialises, C2 through C5 cannot fire when L1 ∧ L2 ∧ L3 hold. Any SMS-based empirical conclusion is therefore structurally consistent with the existing mutation-testing literature in the classical syntactic-mutation regime, so there is no metric-level semantic fragmentation. The skeleton of eleven concepts, the three-state decomposition, the SMS formula, and the AVP interface are fixed; the contents of MUT, MP, C, and cls remain open to future extension. 2.6 SMS → MS degeneration: formal statement The §2.5 backward-compatibility claim is formalised as a degeneration theorem. The full notation cross-reference is Appendix G.1; the joint conditions and lemma proofs are in Appendix G.2-G.4. We state only the main theorem and corollary here. Theorem and lemma labels (Theorem 9.1, Corollary 9.1, Lemma 9.1-9.3) are preserved as stable identifiers for crossreference with Appendix G. Definition (Degenerate limit). L = L_equiv ∧ L_killed ∧ L_mut, where each joint condition is a pair of paired axes acting on one layer of the SMS formula: • L_equiv = L1 ∧ L2: ε_eq → 0 ∧ K_eq → ∞ (controls equiv layer; Lemma 9.1). 13

• L_killed = L3 ∧ L4: ε_AVP → 0 ∧ MP set = {MP_eq} (controls killed layer; Lemma 9.2). • L_mut = L5 ∧ L6: mut_j switches to rule-based syntactic operators (Mothra-style) ∧ PUT class ⊆ imperative deterministic programs (controls mut layer; Lemma 9.3). Theorem 9.1 (SMS → MS degeneration). In the degenerate limit L, almost everywhere with respect to the input distribution D_S, |killedclassic | i,j SMSi,k,j − → MSi,j = syntax | |mutj (Si )| − |equivclassic i,j L

where the right-hand side is the classical Mutation Score of Jia & Harman (2011), killedˆclassic is the difference-detection set, equivˆclassic is the behavioural-equivalence set, and mutˆsyntax is the syntactic-mutant set. Proof sketch. By Lemma 9.1, equiv_{i,k,j} → equiv_{i,j}ˆclassic almost everywhere with respect to D_S; the measure-zero qualifier accommodates floating-point pathological points and Not-a-Number propagation. By Lemma 9.2, killed_{i,k,j} → killed_{i,j}ˆclassic, and the MP index k degenerates because L4 collapses MR_{i,k} to {MP_eq}. By Lemma 9.3, mut_j(S_i) → mut_jˆsyntax(S_i). Substituting into the SMS formula yields MS_{i,j}. Full lemma proofs are in Appendix G.3 and the full theorem proof in Appendix G.4. Corollary 9.1 (LRCA trivialisation). Under L, the likely-root-cause inventory C = {C1, . . . , C5} degenerates to {C1}, so suspect_share → 0 and SMS becomes a single-layer metric consistent with the engineering attribution structure of Jia & Harman (2011).

Empirical consistency. Theorem 9.1 + Corollary 9.1 jointly guarantee that any SMS-based empirical conclusion (§5 Cliff’s delta, Friedman chiˆ2, Spearman rho) is structurally consistent with existing mutation-testing literature in the classical syntactic-mutation regime, and does not constitute metric-level semantic fragmentation. 3. Experimental Subjects and Operator Framework 3.1 PUT selection Table 2: PUT selection. Mathematical Class

PUT

Name

structure

LOC

A Numeric

Lorenz ODE

Nonlinear

~150

integration

ODE

LU decom-

Linear

position

algebra

FDM 1D

Parabolic

heat

PDE

~80

~200

conduction B Proba-

bilistic

Beta-

Analytic

Binomial

posterior

~60

conjugate B2

MCMC

Markov

Metropolis-

chain

Hastings

~250

Table 2: PUT selection. Mathematical Class

PUT

Name

structure

LOC

Monte Carlo

Importance

~100

integration

sampling

Gaussian

Kernel

Process

methods

C Surrogate

~300

Regression C2

Polynomial

Orthogonal

Chaos

basis

~250

Expansion C3

D ML

Neural-net

MLP

~400

surrogate

substitution

Multi-Layer

Backpropagation ~350

Perceptron D2

Support

Convex

Vector

optimisation

~200

Machine D3

Logistic

Maximum

Regression

likelihood

~120

The 12 PUTs are inherited from P1 (Meng Li et al., under review) but independently justified along four dimensions: library-stack coverage (numpy 2.4.4, scipy 1.17.1, scikit-learn 1.8.0); mathematical-structure coverage (8

of 12 chapters of Numerical Recipes); overlap with existing mutation-testing benchmarks (DeepCrime, Defects4J, and the mutmut and cosmic-ray demos); and signature-simplification trade-offs. The full coverage argument is in Appendix B.1. The program(x: float) → float signature is a substantive constraint (§7) that bounds the upper limit of mutant semantic complexity; transfer to industrial multi-output PUTs is reserved for the P3 and P5 papers. 3.2 Necessary conditions for semantic mutation Definition (Semantic mutation criteria).

A mutant s' =

mut_j(S_i) is a semantic mutation if and only if it satisfies at least one of the following three conditions. (a) Cross-function-boundary replacement. The AST node operated on crosses at least one function-call or module-import boundary. For example, np.linalg.det(M) → np.sum(np.diag(M)). (b) Carries domain knowledge. The legality of the mutation depends on mathematical, physical, or statistical knowledge of the program’s domain, not on purely syntactic type preservation. For example, changing the Gaussian Process Regression noise_level from 1e-4 to 1e-1. (c) Changes algorithmic class. The mutation alters the algorithmic class implemented. For example, replacing RK4 with Euler changes the integration order. A mutation that satisfies none of (a) – (c) is purely syntactic: AST-local, domain-agnostic, and class-preserving. The five meta-operator classes (CE, OS, HP, TF, SI) are specialisations of (a) – (c): 17

Table 3: Necessary conditions for semantic mutation.

Operator class

(a)

(b)

(c)

Primary condition

CE Constant perturbation

△

partial (b); weakest

OS API replacement

✓

△

(a)+(b)

HP Hyperparameter

✓

△

(b)+partial(c)

TF Numerical transform

△

✓

(b)+(c)

SI / CF Structural injection

△

✓

(b)+(c)

Only CE partially satisfies the necessary conditions (it is a semantic / syntactic boundary class); OS, HP, TF, SI strongly satisfy at least one of (a), (b), (c). 3.3 60-cell instantiation matrix PUT

MP_1 cons.

MP_2 mono.

MP_3 conv.

MP_4 traj.

MP_5 p-ord.

A1 Lorenz

••

•

••

◦

A2 LU

••

◦

•

••

A3 FDM

••

•

••

◦

B1 BetaBin

••

•

◦

•

B2 MCMC

•

••

•

B3 MC

••

◦

••

•

◦

C1 GPR

•

••

•

••

C2 PCE

••

•

••

•

••

C3 NN-Surr

•

••

PUT

MP_1 cons.

MP_2 mono.

MP_3 conv.

MP_4 traj.

MP_5 p-ord.

D1 MLP

••

•

••

D2 SVM

•

••

•

◦

••

D3 LR

••

•

◦

••

Legend: •• substantial (30 cells); • moderate (24 cells); ◦ vacant (6 cells; P1 H6). Each PUT averages 24.3 mutants in the cross-source pool (range 10-30); full 60-cell × N=20 = 292 mutant instantiations × 20 AVP repetitions, with K_eq = 1000 input samples for E2 evaluation. Detailed pool counts and per-class operator specialisations are in Appendix B.2. The cell density notation distinguishes ••substantial (30 cells; aligned slice + strong off-diagonal cells where MR coverage is dense and mutant detection is expected), •moderate (24 cells; cross slice with non-trivial expected detection rate), and ◦ vacant (6 cells; inherited from P1 H6, historically empty cells where no MR is exercised). The 12 aligned (j = k) cells are precisely the on-diagonal cells; the 48 cross (j ̸= k) cells partition into the 24 •moderate + 18 ••off-diagonal substantive + 6 ◦ vacant cells. The •moderate / ••substantive distinction does not change SMS computation; it tracks expected MR coverage density per P1 documentation and informs the §6.2 R_sem / R_kill decoupling discussion. Per-class operator specialisations (illustrative).

Each meta-

operator (mut_C / M / G / T / F = CE / OS / HP / TF / SI in their dual form) requires PUT-class-specific specialisation: • mut_C Conservation-breaking: A1 Lorenz adds ε_drift to RHS 19

(slow Hamiltonian drift); A2 LU decomposition omits the (k+1)-th row multiplier; B1 Beta-Bin posterior omits normalisation; C1 GPR covariance omits positive-definite diagonal term; D1 MLP backprop omits one gradient term. • mut_M Monotonicity-breaking: A3 FDM ∆t occasionally negative; B2 MCMC acceptance min(1, r) → min(0.95, r); C2 PCE highorder coefficient sort inserts inversion; D2 SVM decision-function sign flips near boundary. • mut_G Convergence-breaking: A1 Lorenz RK4 → 1.5-order hybrid; A3 FDM 2nd-order difference → 1st-order; B3 MC doubling sample size does not 1/N-reduce variance; C3 NN-Surr training-epoch truncation. • mut_T Trajectory-distorting:

A1 Lorenz state-vector y / z

swap; B2 MCMC inserts independent-sampling segment; C3 NN-Surr training-target slow phase shift; D1 MLP hidden-layer periodic-mask activation. • mut_F Fidelity-order-breaking: A2 LU partial pivoting degrades to no pivoting; C1 GPR length-scale switches to coarse prior; C2 PCE high-order term randomly retains low-order; D3 LR regularisation occasionally large. The per-class HP / OS / TF substitution rules give similarly differentiated specialisations across PUT classes (e.g., HP on class a = tolerance / max_iter, on class c = GPR noise_level / length_scale, on class d = MLP hidden_dim / dropout; OS on class a = numerical-linalg API swap det ↔ sum(diag), on class b = sampling-API swap, on class c = surrogate-class 20

swap GPR ↔ RBF ↔ NN). The full PUT-class × operator specialisation grid is in Appendix B.2. 3.4 Primary MP convention The diagonal cells j = k form the H2-aligned slice, the off-diagonal cells form the cross slice, and the vacant cells ◦ are not formally adjudicated. The c-class primary MP is held at the P1 pre-registered choice (MP5) throughout this paper. All H1, H2, H3, and H4 verdicts are rendered under this configuration on v3 (same-source) and v4 (cross-source under an identical prompt). An earlier exploratory data-driven primary-MP shift was considered but withdrawn from the analysis after a permutation null over fully exchangeable c-class (PUT, MP) cell SMS values showed the resulting δ inflation to be statistically indistinguishable from random reselection of the c-class primary MP; the corresponding pre-registration of a primaryMP-selection rule on a fresh dataset is deferred to a follow-up study. 3.5 P2 vs syntactic mutants: 12-PUT empirical Experimental tants at

For each PUT, P2 cross-source mu-

design.

(data/mutants/${PUT}_pool_v4/,

the

AST-normalised

level

include_attributes=False)) tants

(1,250

total;

292

total)

are

compared

(ast.dump(annotate_fields=False,

with

cosmic-ray

default-operator

mu-

scripts/p2_vs_syntactic_ast_diff_batch.py).

Source: data/results/cosmic_ray_12put_ast_diff.json. Aggregate results.

Figure 2: Per-class AST overlap rate between 292 P2 mutants and 1,250 cosmic-ray syntactic mutants across 12 PUTs (overall 5.14%). HP, SI, TF (54.5% of P2 pool) are categorically unreachable (0/72, 0/33, 0/54); CE 7.81%, OS 11.67%, CF 33.33% (CF is a class-specialisation, not in main 5).

Table 5: P2 vs syntactic mutants: 12-PUT empirical.

Metric

Value

Total P2 mutants (12 PUTs)

292

Total cosmic-ray syntactic mutants (12 PUTs)

1,250

AST-normalised overlap

Overall overlap rate

5.14%

Per-operator-class breakdown (12-PUT aggregate). Table 6: P2 vs syntactic mutants: 12-PUT empirical.

Class

n_p2

n_overlap

Rate

Interpretation

0.000

Structurally unreachable

0.000

Structurally unreachable

0.000

Structurally unreachable

0.078

Boundary class (§3.2 partial (b))

0.117

Partial incidental hits

0.333

b2 only, n=9

Interpretation: refuting the “post-classification copy” challenge. • HP, SI, and TF (159 of 292 = 54.5% of v4 mutants) are unrepresentable under default first-order configurations of cosmic-ray’s AST-local operators such as BinOp, Compare, and NumberReplacer. The unreachability is structural under first-order operators; whether 23

HOM-based compositions (Jia and Harman 2008; not refuted) can close the gap is residual threat R12, addressed conceptually in §3.6(b). • The CE class shows 7.81% incidental overlap, mostly LLM-generated half-step perturbations on a2 and b3 such as _RHO=28.0 → 27.5. The remaining 92.19% of CE mutants are AST-disjoint, because the LLMs prefer domain-aware perturbations over integer ±1 changes. • An earlier draft labelled the OS row “× tool-inexpressible”. The 12PUT data refines this categorical claim to “△ 88.33% disjoint, 11.67% incidental hits”. The systematic-versus-incidental argument in §3.6 (with details in Appendix B.1.5) clarifies why those incidental hits are stochastic byproducts rather than systematic semantic mutation. • 94.86% of the v4 mutants cannot be reproduced by cosmic-ray defaults. The two pools occupy systematically distinct mutant spaces. A multi-tool cross-comparison is reserved for P4: mutpy is incompatible with Python 3.10+, and mutmut’s operator set overlaps strongly with cosmicray’s. The DeepSeek, Claude, and GPT contributions to the 15 overlap files are uneven (DeepSeek 11, Claude 4, GPT 0). Appendix B.3 covers the cosmic-ray a1 single-PUT pre-12-PUT pilot, and Appendix F discusses the LLM-source distributional-shift threat R8. 3.6 Preventive-defence framing Scope of the claim. The preventive-defence claim below is conditional on a first-order syntactic baseline; HOM-based syntactic compositions are an open question (R12) and are not refuted by the §3.5 evidence. The HP, SI, and TF zero-overlap result, combined with the §3.2 necessary24

conditions argument, amounts to a preventive-defence argument: semantic mutation operators address a class of fault hypotheses that lie beyond the reach of default first-order syntactic configurations (HOM not refuted; see §3.6(b)). Three points sharpen this framing. (a) Systematic versus incidental. Satisfying conditions (a), (b), or (c) is sufficient for a single semantic mutation, but only when satisfaction reflects design intent, not stochastic byproduct, does it constitute a systematic semantic mutation method. A syntactic tool that occasionally hits (a) or (c) with its 12 default operators does so at a non-zero probability, but the hits are not repeatable and they carry neither of the two engineering goods we want. First, designing a semantic mutator like OS det → sum(diag) requires knowing that these expressions are equivalent on diagonal matrices but not on general matrices, a deepening of source-code understanding the syntactic tool does not exhibit. Second, domain-semantic faults, wrong physical constants, unit conversions, boundary conditions, hyperparameter semantics, numerical-method order, are not AST-local; the syntactic-tool design goals (operator typos, off-by-one errors, negation flips) do not target them. Systematic semantic mutation therefore requires (a), (b), or (c) to be design intent. Appendix B.1.5 records this argument in full. (b) HOM caveat. HOM (Jia and Harman 2009; Kintis et al. 2018) could in principle compose syntactic mutants, for example, an Arithmetic Operator Replacement combined with a Statement Deletion, to partially simulate the effects of OS or HP on some PUTs. The §5 empirics did not run a HOM comparison; we list HOM equivalence testing as residual threat R12 (Appendix F.2) and confine our “tool unreachability” claim to first-order syntactic tools

(mutmut and cosmic-ray default configurations). Conceptual analysis of HOM reachability (Round-8 review response). Higher-order mutation (HOM; Jia and Harman 2008, 2009; Kintis et al. 2018) composes multiple first-order operators on the same source location. The strongest reviewer challenge to this paper’s preventive-defence framing is whether HOM compositions of cosmic-ray’s 13 default operators or mutmut’s 14 default operators can reach the §3.2 necessary conditions (a) cross-function-boundary, (b) domain-knowledge-dependent, or (c) algorithmic-class-change. We address this conceptually without empirical HOM testing. For (a): HOM compositions remain over first-order operators that are AST-local within their target node; chaining AOR ◦ Statement-Deletion within a single function does not introduce a cross-module substitution like det(M) → sum(np.diag(M)), which presupposes that np.diag is in scope and semantically equivalent on diagonal matrices but not on general matrices. HOM can reach a kindred destination only if both operators happen to coincide with a meaningful cross-boundary swap, which is exponentially unlikely under the default operator menus. For (b): HOM has no awareness of which constants are load-bearing in the PUT class. A two-operator chain such as Number-Replacer (1e-4 → 1e-3) ◦ Boolean-Swap on a Gaussian-process kernel does not “know” that 1e-4 is the regularization knob; it modifies the literal numerically without crossing the domain-semantics boundary. The Hyperparameter operator class (HP, mut_G) is by construction parameter-aware and class-specific, a property HOM compositions do not exhibit by default.

For (c): Algorithmic-class change (e.g. polynomial → spline basis; Lorenz RK4 → 1.5-order hybrid) requires structural code rewrites at the level of function bodies, library imports, or iteration schemes. HOM compositions over default first-order operators are bounded above by AST-local edits and so cannot reach algorithmic-class change without being effectively equivalent to a manual rewrite. We therefore assert that the §3.5 0/0/0 unreachability for HP / SI / TF, while strictly empirical only for first-order configurations, is unlikely to be refuted by default-operator HOM compositions on combinatorial grounds. A direct empirical HOM falsification (generating mutmut second-order mutants and re-running the AST overlap analysis) remains residual threat R12 and is reserved for P4. We thank the reviewer for prompting this conceptual analysis. A multi-syntactic-tool cross-comparison (mutmut and mutpy) is reserved for P4: mutpy is incompatible with Python 3.10+, and mutmut’s operator set strongly overlaps cosmic-ray’s. The full operator-class cross-reference between cosmic-ray (13 default operators) and mutmut (14 default operators), including which §3.2 necessary conditions each operator family reaches, is in Appendix B.6. The aggregate of the two default sets (~21 distinct firstorder AST-local operator classes) collectively fails all three §3.2 necessary conditions. (c) Source distributional shift. The 15 cosmic-ray AST overlaps are not evenly distributed across the three LLMs (DeepSeek 11/15, Claude 4/15, GPT 0/15; see cosmic_ray_12put_ast_diff.json). DeepSeek tends to generate syntactically simpler mutations. This LLM-source bias is discussed

as R8 (Appendix F.2) but does not affect the systematic-vs-incidental argument, which is based on categorical AST-locality, not hit frequency: the HP / SI / TF zero-overlap result is 0/0/0 across all three sources. 4. Experimental Procedure 4.1 Procedure overview Pipeline order: §3.1 PUTs → §4.2 Mutant generation → LRCA L0 prescreen → §4.4 Equivalence (E1 ∧ E2 ) → AVP → killed/survive → LRCA three-layer diagnosis → SMS + C1 _share report.

Each (i, k, j) cell is reported with inst_count, equiv_count, killed_count,

survive_count,

pect_share_{i,k,j},

SMS_{i,k,j},

C1_share_{i,k,j},

sus-

and the three rates inst_rate / equiv_rate /

survive_rate corresponding to RQ1. 4.2 Cross-source protocol To isolate the contributions of LLM same-source bias and MR-MP alignment design to Cliff’s delta, we ran a three-stage ablation: Table 7: Cross-source protocol. c-class primary Version

Mutant pool

Use

Same-source

MP5 (P1 legacy)

H2 baseline

(Claude Opus

(pre-registered

4.6)

primary)

Table 7: Cross-source protocol. c-class primary Version

Mutant pool

Use

Cross-source

MP5 (held at

Isolate

(Claude Opus

pre-registered)

LLM-source

4.6 + GPT-5.4 +

diversity

DeepSeek chat)

The v4 protocol (scripts/cross_source_campaign.py) runs each (PUT, operator) pair on Claude / GPT / DeepSeek with K=3 trials under an identical prompt template (temperature 0.7, V1-V4 mechanical-validation gate). Cross-source pool capacity: 37 operators × 3 sources × 3 trials = 333 attempts, 89% V1-V4 pass, 298 confirmed mutants distributed nearly equally (Claude 101, GPT 98, DeepSeek 99). Declared confound: protocol asymmetry (R13). v3 used the original Phase-1 dual-blind reviewer protocol (Claude generation + GPT-5.4 review + DeepSeek arbitration); v4 passes V1-V4 mechanical gates only and does not invoke a reviewer LLM. A fraction of the v3 → v4 source-axis contrast may therefore reflect a slight quality shift rather than LLM-source diversity per se. The follow-up study will rerun dual-blind on the v4 grid; full per-LLM token / latency / cost figures, V1-V4 specifications, and the full differential-prompt protocol are in Appendix C.1.

4.3 Mutant-pool prescreen and equivalence detection Pool prescreen (LRCA L0). Each candidate mutant passes three gates: 1. Static lint and type checking. 2. A unit self-test on simple inputs to confirm that the PUT loads, runs, and returns a finite output. 3. A double-blind review sign-off under the Phase-1 protocol. In Phase-1, the generator LLM is Claude Opus and the reviewer LLM is GPT-4o; the two roles are isolated, and the reviewer sees only the PUT and the mutant code and outputs a (syntactic, executable, fault-injected) triple. Inconsistent reviewer outputs go to a manual arbitration queue (no more than 10% of cases); double-confirmed mutants enter the pool. We then randomly sample 20% of the pool for manual review by scientific-computing researchers, and downgrade the entire batch to manual review if manual– reviewer inconsistency exceeds 10%. Each cell yields 30–50 candidates and retains 10–15 mutants. The v4 cross-source pool does not invoke the reviewer LLM, for cost and speed reasons; a fraction of the v3 → v4 source-axis shift may therefore reflect a quality decline rather than a source-diversity contribution. We declare this protocol asymmetry as threat R13 (§6.1; full justification in Appendices C.2 and C.3). Equivalence detection (E1 ∧ E2). For each mutant s' in mut_j(S_i): 1. E2 step. Sample K_eq = 1000 inputs x ~ D_S; compute ∥S_i(x) − s'(x)∥; if ≤ ε_eq for all samples, mark E2-pass. 2. E1 step. For every mr ∈ MR_{i,k}, compare AVP(S_i, mr) with AVP(s', mr); if all consistent, mark E1-pass. 30

3. Conjunction. E1 ∧ E2 → enter equiv_{i,k,j}, exclude from SMS denominator. Otherwise pass to AVP-killed determination. E2 is run before E1 because E2 is computationally cheaper (K_eq scalar evaluations) and quickly discards numerically distinct mutants; the remaining candidates are screened by AVP for MR-relation equivalence. The joint condition is conservative: false-non-equivalent (E1 ∨ E2 fail) is much easier than false-equivalent (both must mis-pass simultaneously), biasing SMS slightly high, an explicit conservative engineering choice (Appendix A.3). Killed determination. With OR-aggregation across mr in MR_{i,k}:

killed(s′ , MRi,k ) ⇐⇒ ∃ mr ∈ MRi,k : AVP(Si , mr) = pass∧AVP(s′ , mr) = fail Each (i, k, j) cell is sampled N = 20 times (statistical replicate) to compute the per-cell SMS, with mutant-level fail ratios feeding the LRCA L1 decision. AVP version is pinned to the P1 commit hash (<P1-AVP-vX.Y>); the P2 reproducibility package embeds the complete AVP source so that P2 remains self-consistent under any P1 evolution. 4.4 LRCA three-layer diagnosis For each killed mutant the three-layer decision tree applies in order: • L1 tolerance robustness (all classes). N=20 replicates with a failratio cutoff at 0.80; a kill that survives less than 80% of the 20 replicates is labelled C2 (numerical-tolerance perturbation) and exits. • L2 OOD triage (C / D classes only). Sample from D_Sˆvalid; if the mutant fails only in the OOD region, label C3 (out-of-distribution) and exit. 31

• L3 statistical-assumption baseline (B / D + Wilcoxon / DTW only). Pre-check IID / stationarity on the PUT’s own repeated samples; if the AVP statistical assumption is itself violated, label C4 and exit. • Artefact recheck. An external reviewer re-examines mutant code + prompt history; if LLM / artefact evidence (e.g., mutator overinjection) is found, label C5; otherwise label C1. Multi-label priority C5 > C4 > C3 > C2 > C1 takes the earliest confirmed non-semantic cause (decision-tree control flow). Threshold calibration over a 9-grid (ood_band ∈ {0.02, 0.05, 0.10} × tolerance_multiplier ∈ {3.0, 10.0, 30.0}, repeats fixed at 20) lifts H4 from 10/60 to 12/60 cells; the best combination (ood_band = 0.02, tolerance_multiplier = 3.0) is reported as primary, with default-threshold results retained as control (lrca_60cell_v3.json). The calibration ceiling (12/60 = 20%) remains far below the 80% pre-registered threshold; H4 is unattainable on this dataset as an intrinsic property of LLM-mutant pools, not a calibration issue. Detailed grid table, L0-L3 sub-protocols, and decision-tree pseudocode are in Appendix A.2 + C.4. 5. Statistical Analysis and Empirical Results 5.1 Statistical pipeline The primary statistical reporting follows a pre-registered hierarchy:

Figure 3: 60-cell heatmap of mean SMS over 12 PUTs × 5 MPs (v4 cross-source primary). Diagonal cells (operator-MP aligned) show concentrated mass; off-diagonal mass reflects bleed into adjacent MPs. Empty rows = PUT-level zero-mass cohort (PUTs A1, B1, D2; see §6.2).

Table 8: Statistical pipeline.

Primary statistics

Reporting format

RQ1

inst_rate, equiv_rate,

60-cell heatmap +

C1_share,

4-class marginals

survive_rate RQ2

RQ3

aligned-SMS vs

Cliff’s delta + odds

cross-SMS; sparse ◦ vs

ratio + 95% bootstrap

dense ••equiv_rate

∆SMS_c (c ∈

sign test (df = 3) +

{A,B,C,D});

descriptive forest plot

CV(∆SMS) RQ4

Spearman ρ + Kendall

scatter + dual-metric

τ (SMS vs PC)

ranking comparison

Cell-level multiple comparisons across the 60 cells use BenjaminiHochberg FDR control at alpha_FDR = 0.05. Bootstrap 95% CIs use 1,000 iterations as default (B = 10,000 for the headline H2 delta CI per R-12 response). Mixed-effects modelling (sms ~ C(class) * C(operator) + (1 | put)) failed with Singular matrix at N = 60 (insufficient column rank for the class × operator interaction; 11-d fixed-effects + 12 PUT random intercepts exceed the 60-observation budget). The Friedman test is the non-parametric formal alternative. The five hypotheses are pre-registered: H1 (operator implementability) requires ≥ 4 of 5 operators producing ≥ 5 non-equivalent mutants on ≥ 9/12 PUTs; H2 (aligned-vs-cross) requires odds ratio ≥ 3.0 34

and Cliff’s delta ≥ 0.474 (Romano 2006 large-effect threshold); H3 (crossclass consistency) requires sign-test 4/4 across 4 classes plus CV(∆SMS) < 0.5; H4 (LRCA mass) requires mean suspect_share ≤ 0.20. SMS variants (SMS, SMS_unfiltered) and construction lemmas are in Appendix D.1. 5.2 RQ1: 60-cell distribution Table 9: RQ1: 60-cell distribution.

Metric

Value

Number of cells

Mean SMS

0.104

Median SMS

0.000

Std SMS

0.213

Cells with SMS = 0

45 / 60

Mean mutants / cell (v4)

24.3 (range 10-30)

The zero-mass dominance (45/60 = 75%) is concentrated in the crossMP slice: ~88% (42/48) of cross cells are zero, ~25% (3/12) of aligned cells are zero. Cliff’s delta is well-defined under this distribution but its inference is effectively dominated by n_aligned = 12 + n_cross_nonzero ≈ 6 = 18, not the surface n=60. Median odds ratio is formally infinite (median(cross) = 0); we report “aligned median > 0 = cross median” as auxiliary qualitative evidence. Effective-n note. The surface n_aligned = 12 and n_cross = 48 mask an effective-n constraint at n_eff ≈ 18, which explains the wide 95% boot35

strap CI [0.127, 0.740] (upper/lower ratio ≈ 5.83, consistent with known liberal tendency of percentile bootstrap at small n_eff). The implications for power and the H2 verdict direction are quantified jointly with the stipulatedalternative analysis in §5.4. PUT-class diversification at n ≥ 30 (P4) is a testable route to relaxing the effective-n constraint. Power and effect-size disambiguation are treated jointly with the stipulated-alternative analysis in §5.4 (avoiding repetition). Consistency with LLM-mutant literature. Tip et al. (2024) LLMorpheus reports high cross-MP failure proportions on JavaScript scientificcomputing PUTs (specific numbers not listed in cited literature); zero-mass dominance appears to be a shared characteristic of LLM-mutant + existing MR-MP alignment designs, not a quirk of this paper’s PUT selection. 5.3 RQ2: Aligned vs cross Table 10: RQ2: Aligned vs cross.

Slice

Mean SMS

Median SMS

aligned (j = k)

0.275

0.267

cross (j ̸= k)

0.061

0.000

Two delta point-estimates under the pre-registered primary-MP convention: • v3 (same-source, pre-registered): delta = 0.323, 95% CI [0.017, 0.622]

• v4 (cross-source, c-class held at MP5): delta = 0.314, 95% CI [0.014, 0.622]. The cross-source pool aggregates Claude / GPT / DeepSeek under an identical prompt; the c-class primary is held at the pre-registered MP5 so that the v3 → v4 contrast isolates the LLMsource-diversity axis without inheriting any post-hoc primary-MP selection. H2 verdict. Neither delta crosses the Romano (2006) large-effect threshold 0.474; both lie in the medium-effect range (Romano 2006 medium 0.330). The contrast delta_v4 − delta_v3 = −0.009 (95% CI covers zero) supports a null reading on the LLM-source-diversity axis under an identical prompt: replacing a same-source pool with a three-LLM cross-source pool does not appreciably shift Cliff’s delta within this design. We frame H2 as an underpowered exploratory contribution and defer full verification to a follow-up study with a larger PUT sample. This medium-effect placement is consistent with Tip et al. (2024) LLMorpheus’s medium-effect range on JavaScript LLM mutants, a contextual literature observation (estimand caveat: their delta compares “LLM vs traditional mutants on fault detection”, ours compares “aligned vs cross MP slice within one pool”; the numerical similarity is not substantive support). 5.4 RQ2: Stipulated-alternative power The §5.2 effective-n constraint motivates an explicit power analysis. A plug-in bootstrap (5,000 replications, seed = 42) samples with replacement from the observed (n = 12, n = 48) v4 SMS distributions. The plug-in power table is: 37

Table 11: RQ2: Stipulated-alternative power.

Threshold

Interpretation

Power at (12, 48)

δ > 0.000

Any effect

0.997

δ > 0.147

Small

0.966

δ > 0.330

Medium

0.759

δ > 0.474

Large (H2)

0.423

The plug-in result answers the question “given the observed distribution, how often do we exceed the threshold?” A stipulated-alternative simulation answers a more pointed question: “if the truth equals the H2 boundary 0.474, how often does the (12, 48) design return δ ≥ 0.474?” SMS distributions have heavy ties at zero (45 of 60 cells), so a raw shift is discontinuous: any ε > 0 jumps δ from 0.314 to 0.74. We therefore use a mixture, drawing the aligned sample with probability w from (observed_aligned + 0.001) and with probability 1 − w from observed_aligned, calibrating w = 0.094 to realise E[δ] = 0.4746. Table 12: RQ2: Stipulated-alternative power.

Stipulated truth

Criterion

Power

0.474

δ ≥ 0.474 (point estimate)

0.491

0.474

Lower 95% CI bound > 0 (any effect)

0.868

Even when the truth equals the H2 boundary, this design returns “not met” verdicts in roughly half of replications. This supports the framing in 38

§5.3: the H2 verdict is a factual statement about the point estimate failing to clear the threshold, not a claim that the effect is necessarily smaller than 0.474. Increasing the sample size narrows the confidence interval but cannot lift the point estimate. The plug-in sample-size sweep (n_aligned ∈ {6, 12, . . . , 60}; n_cross = 4 × n_aligned; power for δ > 0 reaches 0.974 at n_aligned = 6 and 0.996 at 12, then plateaus) is in Appendix D.3. Symmetric reading of the same power. The 49.1% stipulated power is also the relevant power for the source-axis contrast ∆δ = −0.009 (CI covers zero): if the true source-diversity effect on δ were as large as 0.474, this design would correctly reject the null in roughly half of replications. The −0.009 null shift is therefore consistent with a wide range of true source-diversity effects, and we explicitly do not read it as evidence that source diversity is inert. The strong-sense test is deferred to a follow-up study. Note on monotone-transformation invariance. Cliff’s δ is rankbased: a function of U / (n1 ·n2 ), so applying a logit transform to SMS gives δ_logit ≡ δ_raw by construction. This is consistent with the rankinvariance theorem and does not constitute additional robustness evidence. The genuine robustness threats to the H2 verdict come from the zero-mass dominance (§5.2), not from any metric-scale choice. 5.5 RQ3: Cross-class consistency and Friedman Table 13: RQ3: Cross-class consistency and Friedman.

Class

Mean SMS (v3)

Mean SMS (v4)

a (numeric)

0.067

0.067 39

Table 13: RQ3: Cross-class consistency and Friedman.

Class

Mean SMS (v3)

Mean SMS (v4)

b (probabilistic)

0.156

0.148

c (surrogate)

0.047

0.089 (+91.5%)

d (ML)

0.081

0.112 (+38.3%)

Sign test (within-class aligned mean − cross mean, sign = +): 3 / 4 (partial) under both v3 (same-source) and v4 (cross-source); the b-class is the inverted cell. H3 primary verdict: partial (3/4). Cross-source pooling does not flip the b-class inversion. The mixed-effects primary model sms ~ C(class) * C(operator) + (1 | put) returned Singular matrix; the fallback sms ~ C(class) + C(operator) + (1 | put) had PUT random-intercept variance hit boundary 0 (degenerate). We therefore use Friedman as the non-parametric formal alternative: • PUT (n=12) × MP (k=5): chiˆ2 = 15.30, p = 0.0041 (significant). • MP rank means: 2.92, 2.58, 2.08, 3.08, 4.33. • Per-class Friedman with Bonferroni × 4 correction (R1 W4 adversarial review): a 1.000 / b 0.116 / c 1.000 / d 1.000, no per-class result remains significant after correction. Kendall’s W effect sizes: a 0.333 (small) / b 0.898 (large concordance, but caveat: N=3 per class makes this label nominal only) / c 0.333 / d 0.417. 40

Caveat. Friedman tests “are MP rank differences present” (averaged over PUTs); H3 tests “is direction consistent across 4 classes”. These are logically independent. H3 stands on the §5.5 sign test; per-class Friedman is descriptive only (small N=3 per class). 5.6 RQ4: SMS vs Pattern Coverage Status. RQ4 is reported as a descriptive observation, not a hypothesis test, because n = 12 places the 95% Spearman CI at roughly [−0.5, +0.6], so the test cannot distinguish zero, moderate-positive, or moderate-negative correlation. The numbers below are recorded so that P4 (n ≥ 30 PUTs) can pre-register a directional hypothesis. Pattern Coverage (PC) per PUT = #triggered (MP_k, R_outcome) cells / 10. Range [0.500, 1.000], mean 0.733. Pairing with mean SMS over 5 MPs: Spearman rho = 0.163 (p = 0.613); Kendall tau = 0.136 (p = 0.568); n = 12. Statistical-power caveat first. At n = 12, Spearman’s 95% CI is approximately [-0.5, +0.6]; the test cannot distinguish “zero correlation” from “moderate positive correlation” or “moderate negative correlation”. p = 0.61 / 0.57 does not support the strong claim that “SMS is independent of PC” or the strong claim that “SMS is strongly correlated with PC”. The conservative finding is no detectable correlation at n = 12 ; orthogonality is a hypothesis for P4 (n ≥ 30 PUTs or refined PC operationalisation; full PC operationalisation in Appendix D.5). 5.7 H4: LRCA mass

Table 14: H4: LRCA mass.

Metric

Value

Mean C1_share (default threshold)

0.164

Mean C1_share (calibrated best, ood_band = 0.02)

0.209

Mean suspect_share (calibrated best)

0.791

Cells meeting H4 (calibrated best)

12 / 60 = 20.0%

H4 was pre-registered before pool characteristics were known; the dense cutoff sweep below shows the verdict is intrinsic to LLM-mutant pools and not a calibration artefact, which is itself a finding worth reporting. H4 verdict: not met. A dense cutoff sweep (R-14 response, Appendix D.2) shows H4 pass-ratio is flat at 20% over cutoffs ∈ [0.05, 0.40]; no cutoff pushes it past 80%. H4 is intrinsic data property, independent of cutoff choice; v4’s suspect_share distribution is severely bimodal (median 1.0, with 48 cross cells near 1 and 12 aligned cells near 0; the [0.20, 0.80] interior is nearly empty). 6. Discussion 6.1 Cross-source contributes mutant quality, not effect size Going from same-source to cross-source under the pre-registered primaryMP convention shifts Cliff’s δ by only −0.009 (95% CI covers zero). Crosssource pooling raises mean C1_share from 0.164 to 0.209 (a 27% relative increase), class-c mean SMS by +91.5%, and class-d mean SMS by 38.3%. Under an identical prompt template, three LLMs converge on near-identical

distributions for the aligned-vs-cross question. This inversely falsifies our initial hypothesis that LLM same-source bias is the dominant factor in the H2 ceiling: cross-source pooling improves mutant quality (LRCA C1 share, class-mean SMS) without moving the aligned-vs-cross effect size. The strongsense source-diversity test, with per-LLM differential prompts (V_persona, V_cot), is deferred to a follow-up study. Appendix C.1 records the full protocol. The §3.5 evidence (5.14% AST overlap, HP/SI/TF at 0/0/0) confirms that the medium-effect ceiling is not an artefact of LLM-pool overlap with the syntactic-mutant space. 94.86% of v4 mutants are AST-disjoint from cosmic-ray defaults, and the three classes unreachable under default firstorder configurations (HP, SI, TF; 159 of 292 mutants; HOM not refuted, see §3.6(b)) lie outside that space by construction. An effect-size breakthrough therefore requires substantive MR-design refinement (a P4 task), not a larger sample. A numerical-coincidence note. Petrović and Ivanković’s (2018) ~20% productive-mutant rate at Google is close to our calibrated C1_share of 0.20. Their construct is a developer survey (subjective usefulness), and LRCA’s C1 is the output of a three-layer classifier; the agreement between the two is contextual, not mechanism validation. 6.2 Decoupling between R_sem and R_kill The operator-level pilot in Appendix C.4.1 reveals a sharp pattern: HP, TF, and OS operators on the c-class (surrogate) and d-class (machine learning) PUTs reach R_sem ≈ 1.0 (semantic feasibility, the LLM successfully produces a syntactically valid, executable, intent-bearing mutant) but R_kill 43

= 0 under the PUT’s primary MP (the AVP fails to detect the mutant under that MR). Concretely, six cells show R_sem ≥ 0.9 with R_kill = 0: c1_CE1 (GPR noise_level 1e-4 → 1e-1), c2_OS1 (polynomial → spline basis), c3_HP1 (relu → tanh), c3_TF1 (max_iter 1000 → 5), d1_HP1 (MLP α 1e-4 → 1.0), and d3_HP1 (LR C 1.0 → 1e-4). In each, the mutant is a valid hyperparameter-semantic injection, but the primary MP, MP5 asymptotic for the c-class and MP2 monotonicity for the d-class, is an asymptotic or statistical relation, and the parameter deviation it tolerates is wide enough to absorb the mutant. The cell-level evidence in §5.2 reproduces this pattern: 75% of cells are zero, concentrated in the cross slices. Under v4 cross-source, mean C1_share rises from 0.164 to 0.209 (+27%): among the few killed mutants, the crosssource pool reduces LRCA mislabelling. Source diversity therefore improves the quality of the kill set without expanding its coverage. The engineering insight is that operator-MP alignment in MR design is necessary for strong SMS signals; merely enlarging the semantically feasible mutant pool dilutes the C1 proportion without lifting the kill rate. Two questions follow for P4: can we use SMS to infer which MP class is missing for which PUT class, and can we calibrate LRCA so that the threshold separating true semantic faults from artefacts is class-specific? This decoupling motivates the source-axis null reading in §5.3: the v3 → v4 micro-shift of −0.009 attributes to LLM source diversity under a fixed prompt, while MR-design adequacy at the operator-MP level (rather than LLM identity) governs the kill rate.

6.3 H3: partial across same-source and cross-source All four class means are positive in v3 (same-source) and v4 (cross-source). Inter-class balance improves under cross-source (c +91.5%, d +38.3%), confirming that c / d classes have higher mutant-diversity demand than a / b. Mixed-effects unavailability (Singular) is a sample-size constraint at N = 60 / 12 PUTs, not evidence absence. H3 verdict: partial (3/4) under both v3 and v4; the b-class is the inverted cell. The Friedman main effect (chiˆ2 = 15.30, p = 0.0041) speaks to MP differentiation, not H3 direction. 6.4 Stakeholder analysis Scope. All deployment claims in this subsection are bounded to singleoutput float → float kernels under 2 KB, the 12 PUTs of this paper. They do not apply to industrial-scale, multi-module, or multi-output scientific computing software, which lies in the P5 / P2-CN scope. Test engineers can read SMS as a per-MR scalar adequacy score. When SMS sits well below the aligned baseline of about 0.275 (§5.3), the LRCA labels, C2 tolerance, C3 OOD, C4 statistical, point to specific repair paths. Air-gap incompatibility is a hard limitation: the workflow depends on external LLM API calls and is incompatible with most regulated air-gapped Verification and Validation (V&V) environments (IEC 60880, DO-178C, IEC 62304, ISO 26262). Pre-generated mutant pools can be reproduced offline, but generating new mutant pools requires LLM access. Self-hosted openweight LLMs and offline-cached pools are P5 mitigations. Appendix E.1 gives the full air-gap justification and the standards catalogue. MR designers can use offline batch SMS runs as a quantifiable designfeedback metric. We recommend a quarterly batch audit (about 0.5 person45

day per quarter; detailed cost breakdown in Appendix E.2; estimates based on observed timings during this paper’s 12-PUT campaign) rather than perpull-request gating, because LLM API latency, cost non-determinism, and air-gap incompatibility together rule out the per-pull-request style. Appendix E.2 gives the quarterly-audit workflow with resource-cost table. V&V documentation can carry SMS as research-grade supplementary evidence alongside code coverage and MR lists. We make no normative claim toward IEC, ISO, or ASME standards. An earlier draft proposed acceptance thresholds (aligned-cell SMS ≥ 0.20 or 0.30 plus C1_share ≤ 0.20); we removed them in revision because they have no normative backing and could be misread as enforcement-ready. Appendix E.3 records the conceptual complementarity with the code-verification scope of ASME V&V 20-2009 §3. All three stakeholder classes consume the same single source of truth (paper_numbers_v4.json and lrca_60cell_v4.json) to avoid documentation fragmentation. 7. Threats to Validity

Table 15: Threats to Validity. Mitigation #

Threat

Class

summary

LLM

Internal

Reproducibility Appendix

generation

package;

reproducibil-

identical

ity

prompt +

Detail

F.1

seed → ≥ 90% pool overlap R2

K_eq =

Appendix

probabilistic

1000 +

F.1

approxima-

Hoeffding

tion

bound;

Construct

sweep deferred to P4 R3

Circular

Internal

AVP version

Appendix

dependency

pinned to P1

F.1

P1 ↔ P2

commit hash; embedded source

Table 15: Threats to Validity. Mitigation #

Threat

Class

summary

Detail

LRCA

Internal

Decision-tree

Appendix

multi-label

priority +

F.1

boundary

multi-label cooccurrence table

Inherits P1;

Appendix

representa-

Appendix

F.2

tiveness

B.1 coverage

12-PUT

External

argument; P3 scaling R6

Cross-class

Conclusion

H3 framed

Appendix

statistical

exploratory;

F.2

power

Friedman fallback; mixed-effects unavailable

LLM

Construct

3-LLM

Appendix

homogeneity

rotation +

F.2

bias

20% manual sampling

Table 15: Threats to Validity. Mitigation #

Threat

Class

summary

Detail

LLM-source

External

DeepSeek

Appendix

distribu-

11/15 of

F.2

tional shift

overlaps; argument is categorical, not frequency

Mutant-pool

Internal

size

Pool

Appendix

expanded to

F.1

17.4 mean; effect intrinsic, not pool-dilution R10

LLM non-

Internal

determinism

De-dup, K =

Appendix

10/20

F.1

repeats, raw-response store

Table 15: Threats to Validity. Mitigation #

Threat

Class

summary

Detail

R12

HOM

External

Confined

Appendix

claim to

F.2

equivalence

first-order syntactic tools; HOM testing deferred R13

Quality not

Appendix

protocol

down

F.1

asymmetry

(C1_share

(dual-blind

0.164 →

reviewer)

0.209);

v3 vs v4

Internal

follow-up to rerun dual-blind on v4

Final limitations. We list eight known limitations. (1) Equivalence determination is a probabilistic approximation. Sampling K_eq = 1000 inputs is an engineering implementation of the undecidable equivalence problem, not a theorem-based decision. We 50

give a Hoeffding-style upper bound on the false-equivalence probability in §2.3, but we did not execute a K_eq sweep over {500, 1000, 2000} for this submission. P4 will run that sweep. (2) LLM generation homogeneity bias.

Even with three-LLM

cross-source pooling, training-data overlap across Claude, GPT, and DeepSeek may produce similar blind spots. Three-LLM rotation and 20% manual sampling reduce but do not eliminate this risk. Future work should add a fourth, independently-trained LLM family, a selfhosted open-weight model is the natural candidate. (3) Limited statistical power for cross-class consistency. The fourclass sign test has df = 3, and the mixed-effects model returns Singular matrix at N = 60 over 12 PUTs. We treat H3 as exploratory and present RQ3 in four pieces: class means, sign test, Friedman, and a forest plot. (4) LRCA reports “likely” root causes. The decision-tree priority C5 > C4 > C3 > C2 > C1 is an engineering choice. The reproducibility package preserves the multi-label co-occurrence table for every killed mutant, not only the priority-winning root cause; root_cause is best read as a likely cause rather than a definitive causal attribution. (5) The AVP is reused from P1. The P2 reproducibility package embeds the AVP source code so that P2 stays self-consistent under P1 evolution, but interface semantics may shift if P1 undergoes a major revision. (6) Epistemological scope versus engineering scope. SMS measures semantic detection capability in the epistemological sense; engineering 51

value is the specific subject of P2-CN and P5. (7) Signature simplification. The float → float single-output signature is a substantive constraint, not a purely engineering trade-off, and it bounds the upper limit of mutant semantic complexity. P3 and P5 will validate SMS portability on industrial-grade multi-output PUTs. (8) Air-gap incompatibility. The mutant-generation workflow calls external LLM APIs and is incompatible with most regulated air-gapped V&V environments (IEC 60880, DO-178C, IEC 62304, ISO 26262). Pre-generated mutant pools can be reproduced offline, but generating new pools requires LLM access. Self-hosted open-weight LLMs and offline-cached pools are P5 mitigations. 7.1 Statistical-modelling capacity at N = 60 cells / 12 PUTs The mixed-effects primary model sms ~ C(class) * C(operator) + (1 | put) returned a Singular matrix at N = 60; the fallback sms ~ C(class) + C(operator) + (1 | put) had the PUT random-intercept variance hit boundary 0 (degenerate). Both failures are not numerical accidents; rather, they are evidence that 60 observations across 12 PUTs cannot identify an 11-dimensional fixed-effects structure plus 12-PUT random intercepts. This is a hard limit of the present design: the experiment cannot, in principle, fit a hierarchical model with this fixed-effect dimensionality at this sample size. Friedman tests serve as the non-parametric formal alternative, but they answer a structurally different question. P4’s expansion to n_PUT ≥ 30 is the only path to a properly identified hierarchical model.

7.2 PUT-level zero-mass cohort Figure 2 and §6.2 document that PUTs A1, B1, and D2 produce all-zero rows across the five MPs in the cross-source pool; that is, for 25% of the PUTs, SMS gives zero signal at every operator-MP cell. This is a substantive limitation of SMS as an adequacy metric on this PUT cohort: when the LRCA C1 mass collapses to zero across all aligned and cross slices for a PUT, SMS cannot distinguish strong from weak MR sets on that PUT. The phenomenon coexists with R_sem ≈ 1.0 (mutants are valid; §6.2), so the issue is not mutant generation but MR-design adequacy at the PUT level. P4’s MR-design refinement (per-PUT MP discovery) and the proposed P4pre-registered c-class primary-MP rule on a fresh dataset are the principal mitigation paths. We declare this as residual threat R14. 7.3 R13 protocol-asymmetry magnitude estimate §4.2 declared protocol asymmetry R13: v3 used the Phase-1 dual-blind reviewer protocol (Claude generation + GPT-5.4 review + DeepSeek arbitration), while v4 passes V1-V4 mechanical gates only. To quantify the potential δ-shift attributable to this asymmetry without dual-blind, we offer a rough order-of-magnitude estimate: in the Phase-1 audit, dual-blind review filtered approximately 5-10% of LLM-generated mutants as semantically inconsistent; if these filtered mutants were systematically less effective at killing under MR_aligned (a plausible but unverified premise), the upperbound δ-shift contribution from removing the reviewer step is bounded above by approximately ±0.03 to ±0.05 on Cliff’s δ at n_aligned = 12. This bound is an order of magnitude larger than the observed −0.009 sourceaxis shift and therefore prevents reading the source-axis null as evidence of 53

LLM-source-diversity inertness; a direct dual-blind v4 rerun (deferred to a follow-up study) would give a tighter quantification. 8. Conclusion 8.1 Findings summary The 60-cell empirical demonstration produces five findings. (i) The H2 large-effect threshold is not met under the pre-registered point-estimate criterion (δ_v3 = 0.323; observed effect lies in the medium-effect range of Romano 2006). The 49.1% stipulated power at the large-effect boundary clarifies that “not met” is a statement about the point estimate, not about the effect size; we frame the finding as an underpowered exploratory contribution and defer full verification to a follow-up study with a larger PUT sample. (ii) Holding the c-class primary metamorphic relation at the pre-registered MP5, three-LLM cross-source pooling under an identical prompt shifts Cliff’s δ by only −0.009 (95% CI covers zero); within this design, LLM identity under an identical prompt is not the lever on the aligned-vs-cross effect size. A strong-sense source-diversity test with per-LLM differential prompts is deferred to a follow-up study. (iii) Cross-source pooling raises mutant quality (mean C1_share +27%, class-c mean SMS +91.5%, class-d mean SMS +38.3%) without raising the aligned-vs-cross effect size. Source diversity therefore improves the kill set at the LRCA-classification level without expanding the operator-MP coverage that drives δ. (iv) The Friedman main effect on MP differentiation is χ2 = 15.30, p = 0.0041. 54

(v) The Spearman correlation between SMS and Pattern Coverage is 0.163 at n = 12. Orthogonality is a hypothesis, not a finding. 8.2 Methodological contributions This paper contributes a three-layer framework for domain-semantic mutation in single-output scientific computing kernels. • Layer 1 (§3.2). Formal necessary conditions for semantic mutation, cross-function-boundary substitution, dependence on domain knowledge, change in algorithmic class, instantiated as five meta-operator classes: CE, OS, HP, TF, and SI (also written mut_C, mut_M, mut_G, mut_T, mut_F). • Layer 2 (§2.3). The E1 ∧ E2 equivalence judgement, the conservative complete instantiation of the Layer-1 conditions, with an explicit tradeoff against E1-alone and E2-alone variants. • Layer 3 (§3.5). AST-normalised empirical traceability across all 12 PUTs: a 5.14% overall AST overlap with cosmic-ray defaults, and HP, SI, and TF unreachable at 0/0/0 under default first-order configurations (HOM not refuted; see §3.6(b)). SMS is backward-compatible with the classical MS through §2.6 Theorem 9.1, which establishes almost-everywhere degeneration in the limit L = L_equiv ∧ L_killed ∧ L_mut. The 60-cell empirical audit reported in §5 is one demonstration following this backbone, not the paper’s main contribution. 8.3 Future work and P-series roadmap We commit six P4 follow-ups. 55

(a) A pre-registered c-class primary-MP rule on a fresh dataset. (b) A

differential-prompt

LLM-diversity

test

using

V_canonical,

V_persona, and V_cot (full protocol in Appendix C.1.1). (c) A scaling study at n ≥ 30 PUTs for SMS-vs-Pattern-Coverage orthogonality and for HOM-equivalence empirics. (d) A rerun of the dual-blind reviewer protocol on the full v4 grid, to separate protocol asymmetry (R13) from source diversity. (e) Cross-language portability work (Python → JavaScript and Julia) building on Tip, Bell, and Schäfer (2024). (f) A self-hosted open-weight LLM pilot for air-gapped industrial deployment (P5 scope). The companion P-series roadmap is as follows. • P1. MR meta-pattern audit on the same 12-PUT infrastructure (under review at Progress in Nuclear Energy and International Conference on Software Analysis, Evolution and Reengineering (SANER) 2027). • P3. Industrial-scale Java and C++ port with a second-rater inter-rater κ for LRCA. • P4. Formal theorems on minimal MR-subset existence, reachable adequacy, and three-pillar coupling, targeted at ACM Transactions on Software Engineering and Methodology. • P5. Regulatory transfer to IEC 60880, ISO 26262, and DO-178C with the conceptual complementarity argument (Chinese, in submission to Nuclear Power Engineering).

Data and code availability All

raw

data,

JSON

SSOTs

(paper_numbers_v3

v4,

rq2_cliffs_delta_v3 / v4_mp5, lrca_60cell, lrca_v4_mp5_recompute, rq2_power_stipulated, cosmic_ray_12put_ast_diff), mutant pools, AVP source, and analysis scripts will be archived on Zenodo at the time of acceptance under DOI [10.5281/zenodo.XXXXXXX] (placeholder, to be minted on acceptance). For peer review, an anonymized read-only mirror of the repository is available at [https://anonymous.4open.science/r/p2sms-anon-XXXX] (URL to be provided by the corresponding author upon Editor request, per IST guidelines).

The repository structure follows

REPRODUCIBILITY.md in the source tree.

The cosmic-ray and mutmut

operator versions referenced in §3.5, §3.6, and Appendix B.6 are pinned in requirements-frozen.txt. References DeMillo, R. A., Lipton, R. J., & Sayward, F. G. (1978). Hints on test data selection: Help for the practicing programmer. Computer, 11(4), 34-41. https://doi.org/10.1109/C-M.1978.218136 Jia, Y., & Harman, M. (2011). An analysis and survey of the development of mutation testing. IEEE Transactions on Software Engineering, 37(5), 649678. https://doi.org/10.1109/TSE.2010.62 Jia, Y., & Harman, M. (2008). Constructing Subtle Faults Using Higher Order Mutation Testing. In Proc. 8th IEEE International Working Conference on Source Code Analysis and Manipulation (SCAM), 249-258. IEEE. https://doi.org/10.1109/SCAM.2008.36

Jia, Testing.

Y.,

Harman,

(2009).

Higher

Order

Mutation

Information and Software Technology, 51(10), 1379-1393.

https://doi.org/10.1016/j.infsof.2009.04.016 Andrews, J. H., Briand, L. C., & Labiche, Y. (2005). Is mutation an appropriate tool for testing experiments? In Proc. ICSE 2005 (pp. 402411). ACM. https://doi.org/10.1145/1062455.1062530 Just, R., Jalali, D., Inozemtseva, L., Ernst, M. D., Holmes, R., & Fraser, G. (2014).

Are mutants a valid substitute for real faults

in software testing?

In Proc.

FSE 2014 (pp. 654-665).

ACM.

https://doi.org/10.1145/2635868.2635929 Papadakis, M., Kintis, M., Zhang, J., Jia, Y., Le Traon, Y., & Harman, M. (2019). Mutation testing advances: An analysis and survey. Advances in Computers, 112, 275-378. https://doi.org/10.1016/bs.adcom.2018.03.015 Kintis, M., Papadakis, M., Papadopoulos, A., Valvis, E., Malevris, N., & Le Traon, Y. (2018).

How effective are mutation testing tools?

An empirical analysis of Java mutation testing tools with manual analysis and real faults.

Empirical Software Engineering, 23(4), 2426-2463.

https://doi.org/10.1007/s10664-017-9582-5 Delgado-Pérez, P., & Chicano, F. (2020).

An experimental and

practical study on the equivalent mutant connection: ary approach.

An evolution-

Information and Software Technology, 124, 106317.

https://doi.org/10.1016/j.infsof.2020.106317 Moradi Dakhel, A., Nikanjam, A., Majdinasab, V., Khomh, F., & Desmarais, M. C. (2024). Effective test generation using pre-trained Large Language Models and mutation testing. Information and Software Technology,

171, 107468. https://doi.org/10.1016/j.infsof.2024.107468 Zhang, M., Keung, J. W., Chen, T. Y., & Xiao, Y. (2021). Validating class integration test order generation systems with Metamorphic Testing.

Information and Software Technology, 132, 106507.

https://doi.org/10.1016/j.infsof.2020.106507 Clark, J. A., Dan, H., & Hierons, R. M. (2010).

Semantic Muta-

tion Testing. In Proc. Third International Conference on Software Testing, Verification, and Validation Workshops (ICSTW), 100-109.

IEEE.

https://doi.org/10.1109/ICSTW.2010.8 Dan, H., & Hierons, R. M. (2012). tion Testing Tool for C. In Proc.

SMT-C: A Semantic Muta-

Fifth International Conference on

Software Testing, Verification and Validation (ICST), 654-663.

IEEE.

https://doi.org/10.1109/ICST.2012.155 Derezińska, A., & Zaremba, L. (2019). Mutating UML State Machine Behavior with Semantic Mutation Operators. In Proc. International Conference on Evaluation of Novel Approaches to Software Engineering (ENASE), 385-393. https://doi.org/10.5220/0007735003850393 Sun, C., Liu, Y., Wang, Z., & Chan, W. K. (2016). µMT: A Data Mutation Directed Metamorphic Relation Acquisition Methodology. In Proc. 1st International Workshop on Metamorphic Testing (MET), 12-18. ACM. https://doi.org/10.1145/2896971.2896974 Sun, C., Jin, H., Wu, S., Fu, A., Wang, Z., & Chan, W. K. (2024). Identifying metamorphic relations: A data mutation directed approach. Software: Practice and Experience, 54(3), 394-418. https://doi.org/10.1002/spe.3280 Zhu, H., Bayley, I., Liu, D., & Zheng, X. (2020).

Automa-

tion of Datamorphic Testing.

In Proc.

IEEE International Con-

ference On Artificial Intelligence Testing (AITest),

64-72.

IEEE.

https://doi.org/10.1109/AITEST49225.2020.00017 Chan,

P. Y. P.,

& Keung,

J. (2024).

Validating Unsuper-

vised Machine Learning Techniques for Software Defect Prediction With Generic Metamorphic Testing.

IEEE Access, 12, 165155-165172.

https://doi.org/10.1109/ACCESS.2024.3494044 Curtò, J. de, & Zarzà, I. de. (2025). Metamorphic Testing for Semantic Invariance in Large Language Models. IEEE Access, 13, 214772-214791. https://doi.org/10.1109/ACCESS.2025.3646270 Ammann, P., & Offutt, J. (2008). Introduction to software testing (1st ed.). Cambridge University Press. Petrović, G., & Ivanković, M. (2018). ing at Google.

In Proc.

State of mutation test-

ICSE-SEIP 2018 (pp. 163-171).

ACM.

https://doi.org/10.1145/3183519.3183521 Petrović, G., Ivanković, M., Fraser, G., & Just, R. (2021). Practical mutation testing at scale: A view from Google. IEEE Transactions on Software Engineering, 48(10), 3900-3912. https://doi.org/10.1109/TSE.2021.3107634 Tip, F., Bell, J., & Schäfer, M. (2024). LLMorpheus: Mutation testing using large language models. arXiv preprint arXiv:2404.09952. Humbatova, N., Jahangirova, G., & Tonella, P. (2021). DeepCrime: Mutation testing of deep learning systems based on real faults. In Proc. ISSTA 2021 (pp. 67-78). ACM. https://doi.org/10.1145/3460319.3464825 Just, R., Jalali, D., & Ernst, M. D. (2014). Defects4J: A database of existing faults to enable controlled testing studies for Java programs. In Proc.

ISSTA 2014 (pp. 437-440). ACM. https://doi.org/10.1145/2610384.2628055 Romano, J., Kromrey, J. D., Coraggio, J., & Skowronek, J. (2006). Appropriate statistics for ordinal level data. Annual Meeting of the Florida Association of Institutional Research, Cocoa Beach, FL. Vargha, A., & Delaney, H. D. (2000).

A critique and improve-

ment of the CL common language effect size statistics of McGraw and Wong. Journal of Educational and Behavioral Statistics, 25(2), 101-132. https://doi.org/10.3102/10769986025002101 Press, W. H., Teukolsky, S. A., Vetterling, W. T., & Flannery, B. P. (2007). Numerical Recipes: The Art of Scientific Computing (3rd ed.). Cambridge University Press. ASME V&V 20 Committee. (2009). Standard for Verification and Validation in Computational Fluid Dynamics and Heat Transfer. ASME V&V 20-2009. Hovmöller, A. (2016-).

mutmut: A Python mutation testing tool.

https://github.com/boxed/mutmut Bingham, A. (2015-). cosmic-ray: Python mutation testing. Sixty North. https://github.com/sixty-north/cosmic-ray Hałas,

(2012-).

mutpy:

Mutation

testing

for

Python.

https://github.com/mutpy/mutpy Li, M. et al. (under review). Empirical audit of metamorphic-relation meta-patterns in scientific computing software (P1). Progress in Nuclear Energy / SANER 2027. Li, M. et al. (under review). [P2-CN companion]. Nuclear Power Engineering.

A. Notation and Operator Catalogue A.1 Complete notation table Paper domain.. Paper identity: P1, P2, P4, P5. Experimental objects.. • PUT set I = {A1,A2,A3, B1,B2,B3, C1,C2,C3, D1,D2,D3}, |I| = 12. • PUT: Si (i ∈ I). • Class mapping: cls : I → {A, B, C, D} (open, extensible). • Valid input distribution: DS , sampling XKeq ∼ DS . Metamorphic testing side.. • Meta-Pattern (MP): MP = {MP1 , . . . , MP5 }, k ∈ {1, . . . , 5} (open, extensible). MP_1 Conservation, MP_2 Monotonicity, MP_3 Convergence, MP_4 Trajectory, MP_5 Partial-order. • Metamorphic Relation: MRi,k (provided by P1); mr = (r, R) ∈ MRi,k . • Automated Verification Pipeline (AVP): AVP : Programs×MRuniverse × R+ → {pass, fail}; AVP(s, mr, εkAVP ) invokes the verification method per MP_k. Mutation testing side.. • Mutation operator family: MUT = {mut1 , . . . , mut5 }, j ∈ {1, . . . , 5} (open, extensible); mut1 = mutC , mut2 = mutM , mut3 = mutG , mut4 = mutT , mut5 = mutF . 62

• Signature: mutj : Programs → 2Programs . • Mutant: s′ ∈ mutj (Si ). • Alignment: align(j) = j. Three-state decomposition (fixed).. mutj (Si ) = equivi,k,j ∪killedi,k,j ∪survivei,k,j

(disjoint, mutually exclusive, exhaustive).

• equiv determination (dual conditions): (E1) AVP-coherent: ∀mr ∈ MRi,k : AVP(Si , mr) = AVP(s′ , mr); (E2) Output-equiv: ∀x ∈ XKeq ∼ DS : ∥Si (x) − s′ (x)∥ ≤ εeq . • killed determination:

killed(s′ , MRi,k )

⇔

∃mr

∈

MRi,k

SMSi,k,j :=

LRCA engineering attribution layer (descriptive, not in SMS).. • Likely root-cause inventory: C = {C1 , . . . , C5 } (open, extensible). C1 True semantic failure; C2 Tolerance perturbation; C3 OOD; C4 Statistical-assumption violation; C5 Mutator artefact. • LRCA output: each s′ ∈ killed annotated with root_cause(s′ ) ∈ C. • Descriptive quantities: C1_sharei,k,j , suspect_sharei,k,j . 63

Slicing and cross-class.. • Slicing: aligned j = k; cross j ̸= k. • Cross-class:

∆SMSc ,

∈

{A, B, C, D};

CV(∆SMS)

std(∆SMSc )/|mean(∆SMSc )|. Tolerance and sampling.. • Tolerance: εkAVP (MP-dependent), εeq (equivalence output tolerance). • Sampling: Keq = 1000 (E2 sample size), N = 20 (statistical replicates). The notation skeleton (11 core concepts + three-state decomposition + SMS formula + AVP interface) is fixed; the content of MUT, MP, C, and cls is open to extension. A.2 LRCA decision tree and three-layer diagnostic protocol Five likely root causes (open): Table .16: LRCA decision tree and three-layer diagnostic protocol.

Code

Meaning

True semantic failure (what P2 seeks)

Numerical tolerance perturbation

Out-of-distribution (OOD)

Statistical-assumption violation

Mutator artefact

Three-layer diagnosis with L0 artefact pre-scan: 64

• L0 artefact pre-scan. Double-blind review (§4.2.4 dual-LLM crosssource + 20% manual sampling). • L1 tolerance robustness. N = 20 replicates; fail ratio ≥ 0.80 considered stable. • L2 OOD triage. For C/D classes, distinguish valid DSvalid from OOD. • L3 statistical baseline. For Wilcoxon/DTW, IID/stationarity precheck on PUT samples. Decision tree: For each s′ ∈ killedi,k,j : • L1. fail ratio < 0.80 ⇒ C2; otherwise ⇒ L2. • L2 (C/D classes). fail only in OOD ⇒ C3; otherwise ⇒ L3. • L3 (B/D classes + Wilcoxon/DTW). assumption violation ⇒ C4; otherwise ⇒ artefact recheck. • Artefact recheck. artefact evidence ⇒ C5; otherwise ⇒ C1. Multi-label priority: C5 > C4 > C3 > C2 > C1. LRCA

threshold

lrca_calibration.json).

calibration

(9-grid

scan;

Best ood_band=0.02 with any toler-

ance_multiplier (3.0 / 10.0 / 30.0) yields mean_C1_share 0.200 / H4 12/60 (20.0%); ood_band=0.05 (default) or 0.10 yields 0.164 / 10/60 (16.7%).

tolerance_multiplier has zero impact (L1 tolerance rarely

triggered); ood_band is the only discriminator. Calibration ceiling 12/60 65

= 20% remains far below 80% — H4 unattainable on this dataset, not a calibration issue (inherent SNR of LLM-mutant pools). A.3 AVP protocol specification and equivalence-judgement trade-off AVP : Programs × MR_universe × R+ → {pass, fail}.

Per-MP

verification: MP_1 tolerance equality |LHS − RHS| ≤ ε; MP_2 / MP_5 Wilcoxon signed-rank α=0.05; MP_3 convergence-order + asymptotic residual ratio; MP_4 DTW distance threshold ε_DTW. AVP version = P1 arXiv commit hash; P2 package embeds AVP source. Three-candidate equivalence trade-off. Table .17: AVP protocol specification and equivalencejudgement trade-off.

Determination

False-positive

False-negative

SMS bias

E1 alone

numerical

E1 false →

Low (smaller

coincidence +

non-equiv

denominator)

output match,

K_eq miss,

High

MR differs (rare)

full-space equal

both fail

E1 ∨ E2 false →

Slightly high

simultaneously

non-equiv

(conservative)

AVP-coherent E2 alone

E1 ∧ E2

E1 alone is fooled by insufficient AVP coverage; E2 alone by numerical coincidence on K_eq. E1 ∧ E2 is the conservative complete instantiation; under L_equiv it reduces almost-everywhere to classical bitwise equivalence 66

(Lemma 9.1). Counter-examples: E2 passes but E1 does not (rare; mutant coincides at K_eq but deviates at MR trigger points → non-equivalent); E1 passes but E2 does not (common; consistent within MR but numerical drift > ε_eq → non-equivalent). B. Experimental Subjects and Mutation-Operator Specialisations B.1 PUT selection rationale (a) Library-stack coverage. numpy 2.4.4 (A1-B3 linear algebra / arrays), scipy 1.17.1 (A1 integrate, A2 linalg, B2 stats), scikit-learn 1.8.0 (C1D3 surrogate / ML) — Python scientific-computing’s de facto foundation stack (PyPI top-50, 2026-04). Uncovered: GPU / distributed (JAX, CuPy, dask), domain-specific (BioPython, Astropy, RDKit). Reserved for P3 (R5). (b) Mathematical-structure coverage. ODE/PDE (A1, A3), direct linear algebra (A2), Bayesian analytic + MCMC + Monte Carlo (B1-B3), kernel + orthogonal polynomial + NN surrogate (C1-C3), convex optimisation + backprop + max-likelihood (D1-D3) — covers 8 of 12 chapters in Numerical Recipes (Press et al. 2007). Uncovered: advanced PDE solvers (FEM, FV, spectral); FFT; interior-point/trust-region optimisation; symbolic/CAS. Reserved for P3. (c) Comparison with benchmarks.

DeepCrime (Humbatova et

al. 2021): P2 class D topical overlap on ML kernels. Defects4J (Just et al. 2014): no overlap, only mutation-as-fault-proxy reference. mutmut / cosmic-ray demos: general Python, P2 extends scientific-computing focus. (d) Scale vs representativeness. Each PUT 50-400 LOC; signature standardised to program(x: float) → float. Substantively smaller than industrial code (typically 1-10 KLOC). 67

Limitation. The float → float signature is a substantive constraint, not engineering trade-off. Industrial inputs are typically high-dimensional (CFD grids, MD particles, FEM matrices); scalarised PUTs upper-bound mutant semantic complexity and may systematically under-estimate SMS and cross-class differences on industrial PUTs (declared §7 R5; P3 validates on industrial-grade PUTs). B.1.5 Systematic vs incidental Syntactic tools may occasionally hit §3.2 (a) (cross-function-boundary) or (c) (algorithm class), but occasionality undermines two engineering functions of semantic mutation. (a) Deepening source-code understanding. Designing OS np.linalg.det(M) → np.sum(np.diag(M)) requires knowing the two are equivalent on diagonal matrices but not on general matrices (det =

eigvals vs sum(diag) = trace =

eigvals); syntactic tool

AST traversal does not require this understanding even when similar replacements appear. (b) Revealing deep faults. Domain-semantic errors (physical-constant, unit conversion, boundary conditions, hyperparameter semantics, numerical-method order) are not AST-local; syntactic mutator design goals (operator typos, off-by-one, negation flips) have hit probability for domain errors much smaller than systematic semantic-mutator triggering, and lack repeatability. Conclusion. Syntactic tools occasionally producing mutants satisfying (a)(b)(c) is stochastic byproduct — not repeatable, not carrying engineering value. Systematic semantic mutation requires (a)(b)(c) to be design intent. B.2 Per-class operator specialisations

Table .18: Per-class operator specialisations. Representative specialisations across Operator

PUTs

mut_C Conservation-breaking

A1 Lorenz: add ε_drift to RHS (slow Hamiltonian drift); A2 LU: decomposition omits k+1-th multiplier (breaks det conservation); B1 Beta-Bin: posterior omits normalisation; C1 GPR: covariance omits positive-definite diagonal term; D1 MLP: backprop omits one gradient term.

mut_M Monotonicity-breaking

A3 FDM: ∆t coefficient occasionally negative; B2 MCMC: acceptance min(1, r) → min(0.95, r); C2 PCE: high-order coefficient sort inserts inversion; D2 SVM: decision-function sign flips near boundary.

Table .18: Per-class operator specialisations. Representative specialisations across Operator

PUTs

mut_G Convergence-breaking

A1 Lorenz: RK4 → 1.5-order hybrid; A3 FDM: 2nd-order difference → 1st-order; B3 MC: doubling sample size does not 1/N-reduce variance; C3 NN-Surr: training-epoch truncation.

mut_T Trajectory-distorting

A1 Lorenz: state-vector y/z swap; B2 MCMC: insert independent-sampling segment; C3 NN-Surr: training-target slow phase shift; D1 MLP: hidden-layer periodic-mask activation.

mut_F Fidelity-order-breaking

A2 LU: partial pivoting degrades to no pivoting; C1 GPR: length-scale switches to coarse prior; C2 PCE: high-order term randomly retains low-order; D3 LR: regularisation occasionally large.

Per-class HP / OS / TF substitution rules (illustrative). HP on class a:

tolerance / max_iter; on class c:

GPR noise_level /

length_scale; on class d: MLP hidden_dim / dropout. OS on class a: numerical-linalg API swaps (det ↔ sum(diag)); on class b: probabilitydistribution sampling swaps; on class c: surrogate-class swaps (GPR ↔ RBF ↔ NN). TF on class a: integration-order changes (RK4 → Euler); on class b: MC estimator changes. Operator-level 15 response).

cross-table

with

cosmic-ray

defaults

(R-

The 12 cosmic-ray default operators are AST-local:

NumberReplacer (Num/Constant — CE partial, △ literal values only); ReplaceArithmeticOperator

(BinOp);

ReplaceComparisonOperator

(Compare); ReplaceLogicalOperator (BoolOp); ReplaceUnaryOperator (UnaryOp); ReplaceTrueFalse (NameConstant — CE keyword, △ Bool literal); BreakContinueReplacer (Break/Continue); RemoveDecorator; RemoveExceptHandler; ZeroIterationForLoop (For); ReplaceIfBlock (If); MutateSubscript. None correspond to OS / HP / TF / SI: Table .19: Per-class operator specialisations.

P2 class

Tool support

Coverage

OS API replacement

None

△ 88.33% disjoint (incidental low-complexity hits)

× tool inexpressible

None

(0/72) TF numerical-method

× tool inexpressible

None

order

(0/54)

Table .19: Per-class operator specialisations.

P2 class

Tool support

Coverage

SI / CF structural

None

× tool inexpressible

injection

(0/33)

Operator-level conclusion. All 12 default classes remain AST-local (BinOp / Compare / BoolOp / UnaryOp / NameConstant / Subscript / If / For / Break / Continue / decorator / except). Categorically, no entry recognises sklearn / scipy hyperparameter semantics (HP), numerical-method order (TF), or control-flow intent (SI / CF). For OS, low-complexity subexpressions can be incidentally hit by BinOp; empirical 11.67% (§3.5) is consistent with 88.33% AST-disjointness lower bound. B.3 cosmic-ray a1 single-PUT pilot Before the 12-PUT generalisation (§3.5 main), a single-PUT lightweight empirical ran via scripts/run_cosmic_ray_a1.sh on a1 (file 934 B, ASTbound). Outcome: mutants_generated ~tens; ≥ 90% belong to BinOp / Compare / 12 default classes; small killed-by-test_a1.py ratio (P2 unit tests are output-shape/type sanity checks; most cosmic-ray mutants not killed) — empirical manifestation of the §3.2.6 thesis that syntactic-tool mutants are not aligned with MR-violation detection. The 12-PUT generalisation (§3.5) supersedes this pilot but corroborates the same conclusion at scale.

B.4 Expected LRCA risk profile PUT-class × LRCA-layer risk weights. A numeric: C2 dominant (⋆⋆⋆ on L1 tolerance). B probabilistic: C4 dominant (⋆⋆⋆ on L3). C surrogate: C3 dominant (⋆⋆⋆ on L2 OOD; ⋆⋆ tolerance). D ML: C3 + C4 mixed (⋆⋆⋆ L2; ⋆⋆ L3). Operator-PUT root-cause hotspots (expected). A: mut_C/G/F dominantly C2 (numerical tolerance), mut_M/T C1 (true semantic). B: dominantly C4 (statistical-assumption violations). C: mut_C/M dominantly C2/C1 on c1/c2, mut_T/F dominantly C3 on c1/c2; c3 (NN-Surr) row dominantly C5 (artefact). D: d1 row dominantly C5 (training-noise artefact); d2/d3 dominantly C1 / C3 mixed. Expected suspect_share thresholds. A: 0.10-0.20 (acceptance ≤ 0.25); B: 0.20-0.35 (≤ 0.40); C: 0.20-0.30 (≤ 0.35); D: 0.25-0.40 (≤ 0.45). 60-cell average: ≤ 0.20 (= H4; acceptance ≤ 0.25). B.5 Engineering-significance mapping j = k aligned diagonal is the H2 threshold-test slice; j ̸= k off-diagonal is control; 6 vacant cells (◦, P1 H6) are not formally adjudicated and are repurposed in §6.2 as descriptive evidence for R_sem / R_kill decoupling. The v3 → v4 micro-change (−0.009) attributes to LLM-source diversity under a fixed prompt; this is the source-axis null contrast underpinning §5.3. B.6 Mutmut vs cosmic-ray default-operator overlap Reviewer-requested

manual

operator-class

cross-reference

between

cosmic-ray’s 13 default operators (cosmic-ray 8.4.6, cosmic_ray.operators package) and mutmut’s 14 default operators (mutmut current main, 73

src/mutmut/mutation/mutators.py).

Both default-operator sets are

first-order AST-local replacements; neither implements a cross-functionboundary, domain-knowledge-dependent, or algorithmic-class-changing mutation (the three §3.2 necessary conditions for semantic mutation). Table .20: Mutmut vs cosmic-ray default-operator overlap. Reaches §3.2

P2 class

Operator class

cosmic-ray default

mutmut default

(a/b/c)?

reachability

Numeric literal

number_replacer

operator_number

(a) no, (b) no,

partial CE only

(e.g. _RHO=28.0 →

±1

27.5) Binary

binary_operator_

operator_swap_op

(a) no, (b) no,

partial OS only

arithmetic swap

replacement

(binary subset)

(incidental)

Comparison

comparison_

operator_swap_op

(a) no, (b) no,

none

swap (<, ≤, >,

operator_

(compare subset)

≥, ==, !=)

replacement

Boolean swap

boolean_replacer

operator_swap_op

(a) no, (b) no,

(+, −, ×, ÷)

(and ↔ or)

(bool subset)

Unary remove /

unary_operator_

operator_remove_

(a) no, (b) no,

swap

replacement

unary_ops,

Keyword swap

keyword_replacer

operator_keywords,

(a) no, (b) no,

operator_name

none

operator_swap_op

(True/False,

none

etc.) break ↔

break_continue

continue /

operator_keywords

(a) no, (b) no,

(subset)

(none)

(a) no, (b) no,

none

return Exception class

exception_

swap

replacer

Decorator

remove_decorator

none

(a) no, (b) no,

removal

none

Variable-binding

variable_inserter,

operator_arg_

(a) no, (b) no,

insert / replace

variable_replacer

removal, operator_

assignment

none

Table .20: Mutmut vs cosmic-ray default-operator overlap. Reaches §3.2

P2 class

Operator class

cosmic-ray default

mutmut default

(a/b/c)?

reachability

no_op insertion

no_op

(none)

(a) no, (b) no,

none

zero_iteration_

for loop

for_loop

String content

(none)

(a) no, (b) no,

none

tweak (XX

(a) no, (b) no,

none

prefix, case flip) Lambda body

(none)

operator_lambda

(none)

operator_dict_

(a) no, (b) no,

arguments

Symmetric/asymmetric (none)

operator_

(a) no, (b) no,

string-method

symmetric_string_

swap

methods_swap,

→ None/0 Dict kwarg

(a) no, (b) no,

none

name (XX

none

prefix) none

operator_ unsymmetrical_ string_methods_ swap Augmented →

(none)

simple assignment match case

operator_

(a) no, (b) no,

augmented_

none

assignment (none)

operator_match

removal

(a) no, (b) no,

none

Aggregate. Cosmic-ray ∩ mutmut: ~9 of 13 cosmic-ray operators have a near-equivalent in mutmut (numeric, binary, compare, boolean, unary, keyword, break-continue, variable, plus partial overlap on string-prefix vs XX); 4 cosmic-ray operators are unique (exception, decorator, no_op, zero-iterationfor); 6 mutmut operators are unique (string content, lambda, dict-kwarg, string-method swap, aug-assignment, match-case). The union is approxi75

mately 21 distinct first-order AST-local operator classes. None of the 21 can: • (a) Cross a function-call or module-import boundary (e.g. replace det(M) with sum(np.diag(M)) requires knowing the equivalence on diagonal matrices — outside any default-operator scope); • (b) Depend on domain knowledge for legality (e.g. a hyperparameter swap that knows noise_level=1e-4 is the load-bearing knob in a Gaussian Process Regression PUT requires PUT-class awareness); • (c) Change the algorithmic class (e.g. swap polynomial → spline basis rewrites the surrogate’s mathematical structure). The mutmut / cosmic-ray null intersection on §3.5’s HP/SI/TF classes (zero overlap) therefore reflects a structural property of first-order AST-local mutation rather than a tool-specific limitation, supporting the §3.6 preventive-defence framing. C. Experimental Procedure Details C.1 Cross-source protocol — full specification Two-stage ablation.

Table .21: Cross-source protocol — full specification. c-class primary Version

Mutant pool

Use

Single-source

MP5 (P1 legacy,

H2 baseline

(Claude Opus

pre-registered)

4.6) v4

Cross-source

MP5 (held at

Isolate

(Claude Opus

pre-registered)

LLM-source diversity

4.6 + GPT-5.4 + DeepSeek chat)

Protocol (scripts/cross_source_campaign.py). (a) For each (PUT, operator), Claude / GPT / DeepSeek run K = 3 trials with identical prompt (§4.2.2), temperature 0.7; source_tag propagated to filenames. (b) V1-V4 mechanical validation (src/p2/mutators/validation.py): V1 syntax (ast.parse), V2 executability, V3 non-triviality (|y_mutant y_original| > 1e-6), V4 signature consistency. (c) DeepSeek model: deepseek-chat (113 tokens/call, 1.8 s), not deepseek-v4-pro (340 tokens/call, 11 s) — quality-equivalent on a2_OS1 dry-run. (d) Pool capacity: 37 ops × 3 sources × 3 trials = 333 attempts; V1-V4 pass rate 89%; 298 confirmed mutants.

Sources contribute nearly equally: Claude 101

/ GPT 98 / DeepSeek 99 (Phase A engineering finding). (e) Sampling (scripts/build_pools.py, POOL_VERSION = v4): per-PUT 30 max, measured mean 24.3, range 10-30. c1 (GPR) yields only 10 because c1_HP1

/ c1_CE1 have V1-V4 near-zero pass (WhiteKernel noise_level 1e-4 → 1e-1 perturbation has minimal output impact, triggers V3 non-trivial failure — itself §6.2 evidence). Protocol-asymmetry (R13). v4 does not invoke reviewer LLM (cost / speed priority; a follow-up study will rerun dual-blind on the v4 grid, ~$58 USD); v3 used the original Phase-1 dual-blind (Claude gen + GPT-5.4 review + DeepSeek arbitration). A fraction of the v3 → v4 source-axis shift may be slight v4 quality decline rather than LLM-source diversity. C.1.1 Differential prompt protocol The v3 → v4 micro-change of −0.009 may reflect “three LLMs under identical prompt” rather than “true upper bound of LLM diversity”. Separation requires one tailored differential prompt per LLM (skeleton: scripts/run_differential_prompt.py). Design (2×3 factorial, within-(PUT, operator)).

Factor A

— prompt template, 3 levels: V_canonical (control, identical to v4), V_persona (per-LLM identity: Claude “numerical-analysis editor”; GPT “scientific-software refactorer”; DeepSeek “library-API substitution specialist”), V_cot (Claude <thinking> tags; GPT “step by step”; DeepSeek stepped-reasoning). Factor B — LLM source (Claude / GPT-5.4 / DeepSeek chat). K = 3 per cell; total 37 × 3 × 3 × 3 = 999 trials. Exit criteria. If max(delta) − min(delta) < 0.05: the v3 → v4 −0.009 source-axis null is robust. If ≥ 0.05 with prompt variant pushing delta past 0.474: H2 revised from “not met” to “prompt-conditional met”. If ≥ 0.05 but all delta < 0.474: prompt sensitivity exists but H2 unchanged. Resources. API ~$18-30 USD; wall 3-4 h at concurrency 4; V1-V4 only, 78

no reviewer LLM. C.2 Manual pilot lineage, LLM-client configuration, and L0 prescreen The original §4.2 protocol used 60% multi-LLM consensus (Claude Opus 4.6 + GPT-5.4 + DeepSeek chat unanimous) plus 40% manual injection by scientific-computing researchers. Each (mut_j, S_i) pair carried one prompt template containing PUT source, semantic intent of mut_j, prohibition against revealing specific MRs (avoids prompt leakage), and output requirements (syntactically correct, executable, single-point modification, diff < 10 lines). Reproducibility parameters: temperature 0.3 in manual-pilot stage (v4 cross-source uses 0.7 per §C.1); fixed seed; 5 candidates per prompt with 2-3 retained. Double-blind review (Scheme C). Generator LLM-G (Claude Opus) and reviewer LLM-R (GPT-4o) are cross-source with strict role separation. LLM-R sees only PUT original + mutant code, unaware of mut_j category / MR content / generator identity, outputting a triple (syntactic, executable, semantic-fault-injected). Double-confirmed mutants enter pool; inconsistencies enter manual arbitration (≤ 10%). From the double-confirmed pool, 20% is sampled for manual review; manual-vs-LLM-R inconsistency > 10% triggers full manual downgrade. Same-source bias mitigation stratifies LLMR by PUT class. Prompt-injection mitigation disables RAG / web search on the API path. L0 pool prescreen. Each mutant passes static syntax (linters / type checkers), a unit self-test (simple inputs produce finite output), and doubleblind sign-off. Target: 30-50 candidates per cell → 10-15 retained after prescreen. 79

C.4 LRCA three-layer execution Decision tree, multi-label priority (C5 > C4 > C3 > C2 > C1), and output (C1_share, suspect_share) are stated in §A.2; the full pseudocode is reproduced there. LRCA does not modify SMS; killed set is not filtered. §5.7 H4 numbers use best calibrated combination (ood_band = 0.02, tolerance_multiplier = 3.0, repeats = 20); default-threshold results retained in lrca_60cell_v3.json as control. Interface with §3.6 risk profile: L0 prescan ↔ §C.2 dual-LLM double-blind; L1 tolerance ↔ N = 20 repetition subprocess; L2 OOD ↔ input-distribution definition; L3 statistical baseline ↔ pre-check protocol. C.4.1 Pilot calibration A 2026 Q3 operator-level pilot ran 12 PUTs × 37 operators (12 is_key=True at K=20, 25 at K=10; 470 trials total). Pilot precursor quantities: R_sem (V1-V6 ∧ operator_match pass rate), D_impl (median pairwise AST + literal + identifier Jaccard distance among confirmed mutants), R_kill (proportion of confirmed mutants killed by AVP under that PUT’s primary MP). Aggregate (N=37): R_sem median 0.50 mean 0.468 (0/37 with R_sem=0); D_impl median 0.42 mean 0.392 (1/37 with D_impl ≈ 0); R_kill median 0.00 mean 0.189. All 37 operators produced ≥ 1 V1-V6 passing mutant (verifying H1). Key R_sem / R_kill decoupling pattern. R_sem ≥ 0.9 ∧ R_kill = 0 combinations all on HP / TF / OS operators in c / d classes (e.g., c1_CE1 noise 1e-4→1e-1 with R_sem 1.00, R_kill 0.00; c3_HP1 relu→tanh, c3_TF1 max_iter 1000→5, d1_HP1 MLP α, d3_HP1 LR C). R_sem-high ∧ R_kill = 1 combinations all on CE / OS operators in a / b classes (a2_CE1 LU 80

det, a2_OS1 prod→sum, b1_OS1 α/β swap, d1_TF1 label flip). This is empirical evidence for §6.2 R_sem / R_kill decoupling at cell level. D. Statistical Analysis Details D.1 SMS variants and construction lemmas SMS_unfiltered (reviewer comparison metric):

SMSunfiltered := i,k,j

(Same form as the primary metric; here killed does not distinguish C1 from C2–C5.) The appendix of the reproducibility package provides cell-by-cell difference between SMS_unfiltered and SMS; if relative difference < 5%, this confirms LRCA does not affect robustness of primary conclusions. Primary-table construction. Table .22: SMS variants and construction lemmas.

Primary statistics

Reporting format

RQ1

inst_rate, equiv_rate,

60-cell heatmap +

C1_share,

4-class marginals

survive_rate RQ2

aligned-SMS vs

Cliff’s delta + odds

cross-SMS; sparse ◦ vs

ratio + 95% bootstrap

dense ••equiv_rate

Table .22: SMS variants and construction lemmas.

Primary statistics

Reporting format

RQ3

∆SMS_c (c ∈

sign test (df=3) +

{A,B,C,D});

descriptive forest plot

CV(∆SMS) RQ4

Spearman ρ + Kendall

scatter + dual-metric

τ for SMS vs PC

ranking comparison

Multiple comparisons. Cell-level claims across 60 cells use BenjaminiHochberg FDR, alpha_FDR = 0.05. N = 20 bootstrap intervals: 1,000iteration 95% CI (B = 10,000 for the headline H2 delta CI). D.2 Cliff’s delta cutoff sensitivity and logit-transformation invariance H4 cutoff sensitivity (R-14 response). Dense grid scan (cutoff ∈ {0.05, 0.10, . . . , 0.50}, step 0.05) on v4 data (scripts/h5_sensitivity.py, data/results/h5_sensitivity_v4.json): Table .23:

Cliff’s delta cutoff sensitivity and logit-

transformation invariance.

cutoff

h5_cells_pass

h5_pass_ratio

0.05

12 / 60

20.0%

0.10

12 / 60

20.0%

0.15

12 / 60

20.0%

0.20 (paper)

12 / 60

20.0%

Table .23:

Cliff’s delta cutoff sensitivity and logit-

transformation invariance.

cutoff

h5_cells_pass

h5_pass_ratio

0.25

12 / 60

20.0%

0.30

12 / 60

20.0%

0.35

12 / 60

20.0%

0.40

12 / 60

20.0%

0.45

13 / 60

21.7%

0.50

13 / 60

21.7%

Conclusion. H4 verdict (not met) is intrinsic data property, independent of cutoff choice. v4 suspect_share distribution is severely bimodal: median 1.0, mean 0.79; 48 cross cells at suspect_share ≈ 1, 12 aligned cells in low end; the [0.20, 0.80] interior is nearly empty. Specific value 0.20 is not loadbearing. Logit-transformation invariance. Cliff’s delta is rank-based (function of U / (n1·n2)), mathematically invariant under any strictly monotone transformation (Romano 2006). Logit is strictly monotone on (0, 1), so delta_logit ≡ delta_raw is a construction result. Not additional robustness evidence; H2 robustness threats come from §5.2 zero-mass dominance, not metric-scale choice. D.3 Power analysis — plug-in and stipulated-alternative Plug-in bootstrap. With-replacement sample from observed aligned (n=12) and cross (n=48) v4 SMS pools; N_sim = 5,000, seed = 42 83

(scripts/compute_rq2_power.py, rq2_power_v4.json): Table .24: Power analysis — plug-in and stipulatedalternative.

Threshold

Interpretation

Power

delta > 0.000

Any-effect

0.997

delta > 0.147

Small

0.966

delta > 0.330

Medium

0.759

delta > 0.474

Large (H2)

0.423

Sample-size sweep (n_aligned ∈ {6, 12, . . . , 60}, n_cross = 4×): power for delta > 0 reaches 0.974 at n=6, 0.996 at 12, plateaus thereafter. RQ2 primary analysis has very low sample-size requirement; the bottleneck is the effect-size boundary itself, not sampling noise. Even at delta_truth ≈ 0.474, sample has only ~42% chance of detecting under plug-in; but this does not mean “insufficient power is the cause of H2 not being met” — observed delta_v3 = 0.323 < 0.474, observed delta_v4 = 0.314 < 0.474; increasing sample only narrows CI. Stipulated-alternative (R1 W1 adversarial review). Implementation via mixture-weight (scripts/compute_rq2_power_stipulated.py, N_sim = 2,000): SMS has heavy ties at zero (45/60 = 0), raw shifting is discontinuous (any ε > 0 jumps delta from 0.314 to 0.74). Mixture: aligned’ = (probability w) sample from (observed_aligned + 0.001) + (probability 1−w) sample from observed_aligned, calibrate w so E[delta] ≈ 0.474. Calibrated w = 0.094, realised E[delta] = 0.4746. 84

Table .25: Power analysis — plug-in and stipulatedalternative.

Stipulated truth

Criterion

Power

0.474

delta_hat ≥ 0.474 (point-estimate)

0.491

0.474

95% CI lower > 0 (any-effect)

0.868

Even at delta_truth = 0.474, the (12, 48) design returns delta_hat ≥ 0.474 in only ~49% of replications — this supports the §5.3 framing that “not met” is point-estimate, not effect-size. D.4 Friedman per-class — discussion of small-N concordance Per-class Friedman (3 PUTs × 5 MPs) with Bonferroni × 4: a chiˆ2=4.00 raw 0.406 adj 1.000, W=0.333; b chiˆ2=10.78 raw 0.029 adj 0.116, W=0.898; c chiˆ2=4.00 raw 0.406 adj 1.000, W=0.333; d chiˆ2=5.00 raw 0.287 adj 1.000, W=0.417. After correction, no per-class result remains significant at family-wise alpha = 0.05. Even b-class W = 0.898 (Cohen 1988 large concordance) is insufficient at N = 3 per class. H3 verdict rests on §5.5 sign test; per-class Friedman is sensitivity / descriptive only. Friedman main effect on full 60 cells (chiˆ2 = 15.30, p = 0.0041) confirms MP rank differences but is logically independent from H3 cross-class consistency. Mixed-effects

unavailability.

Primary

C(operator) + C(class):C(operator) + (1 | put)

sms ~ C(class) + returned

Singu-

lar (insufficient column rank for class × operator interaction; N=60 too small for 11-d fixed-effects + 12 PUT random intercepts). Fallback sms ~ C(class) + C(operator) + (1 | put) had PUT random-intercept 85

variance hit boundary 0 (degenerates to OLS). Fallback class p-values (descriptive only): b vs a 0.275 / c vs a 0.892 / d vs a 0.991. RQ3 conclusion shifts to four-piece presentation (class means + sign test + Friedman + forest plot). D.5 Pattern-coverage operationalisation Per PUT, (MP_k, R_outcome ∈ {True, False}) binary-tuple coverage: 5 MPs × 2 outcomes = 10 cells, PC = #triggered / 10. Simplest baseline (per-PUT granularity, not distinguishing mutants). PC range over 12 PUTs: [0.500, 1.000], mean 0.733 (paper_numbers_v4.json). Pairing per PUT: Spearman rho = 0.163 (p = 0.613); Kendall tau = 0.136 (p = 0.568). Power caveat. At n = 12, Spearman 95% CI ≈ [-0.5, +0.6]; cannot distinguish “zero correlation” from “moderate positive” or “moderate negative”. p = 0.61 / 0.57 means “no correlation detected at n = 12”, not “correlation does not exist”. Within-class descriptive. b2 PC = 1.0 with mean SMS = 0.067; b1 PC = 0.7 with mean SMS = 0.20. c3 PC = 1.0 with mean SMS = 0.14; c1 / c2 PC = 0.7-0.8 with mean SMS = 0.0. Within the same class, higherPC PUT has lower SMS — contradicts naive “more PC kills more mutants” assumption. Suggestive of orthogonality but n = 12 cannot confirm. P4: expand to n ≥ 30 + refine PC to incorporate mutant dimension. E. Stakeholder Deployment Considerations E.1 Air-gap incompatibility — full justification The §6.4 workflow depends on external LLM API calls (Claude / GPT / DeepSeek), incompatible with most regulated air-gapped V&V workflows: 86

Table .26: Air-gap incompatibility — full justification.

Domain

Standard

Air-gap requirement

Nuclear

IEC 60880

Air-gapped build for safety-critical software

Aerospace

DO-178C

Tool-qualified offline build for DAL-A/B; LLM API outside qualified-tool envelope

Medical

IEC 62304

Class C software lifecycle requires offline reproducible build

Automotive

ISO 26262

ASIL-D requires offline tool chain; cloud LLM non-compliant

Mitigation paths (P5). (a) Self-hosted open-weight LLMs (Llama / DeepSeek local inference) — capability gap vs API-served frontier models; quantitative impact on SMS untested. (b) Offline-cached pre-generated mutant pools with commit-hash + raw-response signature locking — adopters reproduce a published pool offline (this paper’s package supports this), but generating new pools requires external LLM access. SMS for air-gapped industrial deployment is a P5 research direction.

E.2 Quarterly batch audit workflow The original §6.5.2 per-PR GitHub Actions YAML was removed because PR-CI threshold 0.10 contradicted §6.5.3 audit threshold 0.20 and propagated stakeholder-facing CI assumptions into adopters’ pipelines. Replacement: quarterly batch audit: Table .27: Quarterly batch audit workflow.

Step

Description

Cost

Offline mutant-pool

LLM API $5-15; ~30

generation (per PUT

min

24-30 × 12 PUTs ≈ 300) sms_campaign.py

4-core laptop ~10-20

--track 2

min

run_lrca.py

< 1 min

MR-design team review

1-2 h meeting

of MRs with aligned-cell SMS significantly below historical median (0.275)

Quarterly: API ~$10 + automation 30 min + manual 1-2 h ≈ 0.5 personday. Affordable. Per-PR gating not recommended (LLM latency 5-30 s + cost + non-determinism + air-gap incompatibility); adopters needing per-PR 88

should use coverage / unit-test gates. SMS does not replace domain-expert MR physical-reasonableness judgement nor product-requirements review. E.3 V&V documentation — conceptual complementarity with ASME V&V 20-2009 §6.5.3 was retitled from “Auditors / certification bodies” in R3 adversarial review: this subsection makes no normative claim toward NRC, FDA, or ISO 26262 review teams. Within single-output kernels scope, no traceable mapping exists to current IEC 60880 / ISO 26262 / DO-178C / ASME V&V 20-2009 normative bodies. V&V documentation for scientific computing software (per ASME V&V 20-2009 §3 and similar guides) requires quantifiable test-adequacy evidence; current documentation often relies on code coverage + MR lists + SME signatures. SMS may appear as research-grade supplementary evidence alongside coverage and MR lists: (a) aligned-cell SMS per critical PUT; (b) LRCA three-layer diagnosis (C1 readout / C2-C5 attribution); (c) 60-cell matrix visualisation. Reviewers can independently run REPRODUCIBILITY.md. Original threshold recommendations (aligned-cell SMS ≥ 0.20 / 0.30 + C1_share ≤ 0.20) were removed: no normative backing in IEC / ISO / ASME; misreads as enforce-ready. SMS reported as descriptive supplementary evidence, with adequacy judgements left to V&V reviewers case-by-case. Conceptually complementary to ASME V&V 20-2009 §3 code verification (numerical-solver correctness) vs SMS (MR-set fault-detection adequacy). Substantial scale gap to multi-module CFD codes; incorporation into V&V standards body would require P5 large-scale empirics + multi-year dialogue 89

with ASME V&V 20 / IEEE 1012 / IEC 60880 committees. No advocacy that SMS enter any normative certification system in 2027. SMS does not replace SME signatures, FMECA, PIRT, system-level V&V, or ASME V&V 20 §3 numerical-solver verification — it is one link in the evidence chain, not a single-point qualification. F. Threats to Validity — Detailed Mitigation F.1 Internal threats Table .28: Internal threats.

Threat

Mitigation

Residual

LLM generation

§C.2

LLM training

reproducibility

data may be

parameters in

updated

package;

post-submission

dual-LLM cross-source + 20% manual; identical prompt+seed should yield ≥ 90% pool overlap

Table .28: Internal threats.

Threat

Mitigation

Residual

Probabilistic

§2.3 declares

K_eq sweep ∈

equiv

probabilistic

{500,1000,2000}

(K_eq=1000)

approximation;

deferred to P4

Hoeffding-style

(R1 W3

false-equiv

adversarial

bound

review)

P1 ↔ P2

AVP version

None within

circular

pinned to P1

scope

dependency

commit hash;

package embeds AVP source; §1.6 declares P1 as arXiv technical report

Table .28: Internal threats.

Threat

Mitigation

Residual

LRCA

§A.2 priority

C2-C5 may

multi-label

multi-trigger on C5>C4>C3>C2>C1;

boundary

multi-label

B/D classes

co-occurrence table in package; root_cause is “likely root cause” not definitive R9

Mutant-pool size

Pool expanded

Some PUTs < 30

to mean 17.4:

(cache-bound)

delta moved 0.321 → 0.323, CI narrowed; effect size is intrinsic ceiling, not pool dilution

Table .28: Internal threats.

Threat

Mitigation

Residual

R10

LLM

Multi-turn

Claude Opus

non-determinism

de-dup;

subscription

K=10/20

lacks seed

repetitions; raw

control

prompt+response committed for direct reuse R13

v4 used 3

Complete

implementation providers

separation

Protocol-

gap v3 vs v4

(Claude 101 /

requires a

GPT 98 /

follow-up

DeepSeek 99)

dual-blind rerun

without

on the v4 grid

dual-blind; v4 C1_share 0.209 > v3 0.164 weakly opposes “v4 quality decline”

R13 is the protocol-asymmetry threat (dual-blind vs V1-V4 only) on the v3 → v4 source-axis contrast (∆δ = −0.009). 93

F.2 External / construct / conclusion threats Table .29: External / construct / conclusion threats.

Threat

Class

Mitigation

12-PUT repre-

External

Follows P1 site

sentativeness

selection; §1.6 P-II open principle (cls extensible); P3 scaling to MD/QC/CFD

Cross-class

Conclusion

statistical power

H3 framed exploratory; mixed-effects unavailable (Singular at N=60); shifted to class means + sign test + Friedman + forest plot

Table .29: External / construct / conclusion threats.

Threat

Class

Mitigation

LLM

Construct

§C.2 three-LLM

homogeneity bias

rotation + PUT-class subdivision + 20% manual sampling; third-party LLM family validation deferred

External

LLM-source

Hit distribution

distributional

DeepSeek 11/15,

shift

Claude 4/15, GPT 0/15; §3.5 argument is categorical AST-locality, not hit frequency; HP/SI/TF zero-overlap across all 3 sources unaffected 95

Table .29: External / construct / conclusion threats.

Threat

Class

Mitigation

R12

HOM

External

Tool-

equivalence

unreachability claim confined to first-order syntactic tools; HOM empirical testing listed as P4 future work

Conclusion threats: multiple comparisons handled by BH-FDR at alpha_FDR=0.05 (§D.1); N=20 stability handled by 1,000-iteration bootstrap CI (§D.3). Construct threat — SMS measures epistemological semantic detection, not production engineering value (P2-CN scope). G. SMS → MS Degeneration Theorem — Full Proof G.1 Notation cross-reference PUT S_i, mutation operator family mut (syntactic or semantic), mutant set mut(S_i), MP set, equivalence tolerance ε_eq, equivalence sample size K_eq, AVP tolerance ε_AVP. Three-state decomposition: mut(S) = killed ∪ equiv ∪ survive (disjoint). SMS formula:

SMSi,k,j =

G.2 Degenerate-limit definition The degenerate limit L = L_equiv ∧ L_killed ∧ L_mut consists of three joint conditions (paired axes), each controlling one layer of the SMS formula. L1-L6 are not 6 independent axes; the pairing responds to dependency queries (R0 W8 / R1 §4 / R2 W3 / DA-MAJOR-3). Table .30: Degenerate-limit definition.

Joint condition

Component axes

Pairing rationale

L_equiv (Lemma 9.1)

L1: ε_eq → 0; L2:

L1 alone: equiv stays

K_eq → ∞

probabilistic (K_eq cannot cover D_S). L2 alone: bitwise equality diluted by ε_eq. Both simultaneously degenerate equiv to classical behavioural equivalence outside a D_S-measure-zero set.

Table .30: Degenerate-limit definition.

Joint condition

Component axes

Pairing rationale

L_killed (Lemma 9.2)

L3: ε_AVPˆk → 0; L4:

L3 alone: ε_AVP → 0

MP = {MP_eq},

still allows non-trivial

R(y,y’) ≡ y = y’

MP (monotonicity, convergence). L4 alone: equality still carries ε_AVP tolerance. Both simultaneously degenerate killed to classical difference detection.

L_mut (Lemma 9.3)

L5: mut_j →

L5 alone: syntactic ops

rule-based syntactic

on probabilistic/ML

(Mothra AOR/ROR/S-

may still trigger

DL/CRP); L6: cls(I) ⊆

domain-semantic

{imperative

subsets. L6 alone:

deterministic}

imperative programs still mutable by OS/HP/TF/SI. Both simultaneously degenerate mut(S) to Jia & Harman syntactic mutant set.

G.3 Lemmas — Three-state decomposition degenerates under L Lemma 9.1 (equiv degeneration; P1-3 revision with measurezero qualification). Under L_equiv (L1 ∧ L2), semantic-class equivalence (E1 ∧ E2) degenerates to classical behavioural equivalence almost everywhere w.r.t. measure D_S. Proof. E1 (type consistency) holds trivially under ε_eq → 0 (L6 makes imperative output spaces scalar/vector with static types). E2 (|S_i(x) − s’(x)| < ε_eq for K_eq samples) under L1 (ε_eq → 0) ∧ L2 (K_eq → ∞ with measure-equivalent sampling) is almost-everywhere equivalent to ∀ x ∈ D_S N: S_i(x) = s’(x), where N is a D_S-measure-zero set (continuous D_S: floating-point pathological points / NaN propagation; discrete D_S: N = ∅). This matches Jia & Harman (2011) §3 classical equivalent-mutant 99

definition under measure-zero equivalence classes. Lemma 9.2 (killed degeneration). Under L3 ∧ L4, killed degenerates to classical difference detection. Proof. L4 restricts MP to {MP_eq} with R(y, y’) ≡ y = y’. For mr = (r, R) and mutant s’, the MP_eq violation condition ∃ x: S_i(x) ̸= s’(r(x)) under L3 (ε_AVP → 0) becomes exact inequality. With r = id, violation is S_i(x) ̸= s’(x) — classical difference detection. With r ̸= id, MP_eq still requires S_i(x) = s’(r(x)) as reference oracle from original program; no new state classifications introduced. Lemma 9.3 (mut degeneration). Under L5 ∧ L6, mut_j(S_i) degenerates to the syntactic mutant set of Jia & Harman (2011). Proof. L5 switches mut_j to rule-based syntactic operators (AOR, ROR, SDL, CRP, UOI — standard Mothra/Proteum sets); L6 restricts PUTs to imperative deterministic programs, excluding triggering conditions for semantic operators on probabilistic/ML programs. Under this configuration, mut_j(S_i) is the literature-defined syntactic mutant set, independent of domain semantics. G.4 Theorem 9.1 — Detailed proof Theorem 9.1 (SMS → MS degeneration). In the degenerate limit L = L_equiv ∧ L_killed ∧ L_mut, almost everywhere w.r.t. D_S, |killedclassic | i,j SMSi,k,j − → MSi,j := syntax |mutj (Si )| − |equivclassic | i,j L

Proof. Combine Lemmas 9.1-9.3: • Numerator:

under L3 ∧ L4, killed_{i,k,j} → killed_{i,j}ˆclassic 100

(Lemma 9.2); L4 makes MR_{i,k} trivial over k (only MP_eq remains), so subscript k degenerates. • Denominator |mut_j(S_i)|: under L5 ∧ L6 → |mut_jˆsyntax(S_i)| (Lemma 9.3). • Denominator |equiv_{i,k,j}|: under L1 ∧ L2 → |equiv_{i,j}ˆclassic| (Lemma 9.1; almost-everywhere w.r.t. D_S). Substituting into the SMS formula yields MS_{i,j}. Corollary 9.1 (LRCA trivialisation). Under L = L1 ∧ L2 ∧ L3, C = {C1, . . . , C5} degenerates to {C1}. Sketch. Each of C2-C5’s triggering precondition depends on at least one L_j being violated. When L1 ∧ L2 ∧ L3 hold simultaneously, every nontrivial-space dimension (MP non-triviality, AVP tolerance non-zero, nonempty equiv set, MR-design DOF, class-mapping openness) closes; C2-C5 triggering set becomes empty (read off from §A.2 decision tree). The per-C_k to per-L_j minimum-sufficient mapping depends on §4.6 LRCA-classifier engineering thresholds; we do not claim one-to-one correspondence at formal level — readers can trace via §A.2 + §C.4. Under L, suspect_share → 0, LRCA reports only C1 — SMS degenerates to single-layer metric consistent with Jia & Harman (2011) MS engineering structure. Empirical consistency. Theorem 9.1 + Corollary 9.1 jointly guarantee any SMS-based empirical conclusion (Cliff’s delta §5.3, Friedman chiˆ2 §5.5, Spearman rho §5.6) is structurally consistent with existing Jia & Harman (2011) literature in classical syntactic-mutation scenarios — no metric-level semantic fragmentation.

101

Funding This work was supported by the National Natural Science Foundation of China (NSFC) General Program (grant no. 12575176); the Hunan Provincial Education Department Project, China (grant no. 202502000728); the Research Project on Degree and Graduate Education Reform of the University of South China (grant no. 2023JG030); the Natural Science Foundation of Hunan Province, China (grant no. 2025JJ70193); and an industry-funded research project (grant no. 230KHX060001). CRediT authorship contribution statement Meng Li: Conceptualization, Methodology, Software, Writing—original draft. Xiaohua Yang: Supervision, Formal analysis, Writing—review & editing. Jie Liu: Investigation, Validation. Shiyu Yan: Data curation, Visualization. Declaration of competing interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

102

Record · ID 200565 · SHA-256 8d88b8d77e26416b
Retrieved via Conceptio — every document is proof-bundled with source, license, and retrieval metadata.