Insert journal name here manuscript No. (will be inserted by the editor)
Empirical Evaluation of Large Language Models for Migration of Code Fragments to Post-Quantum Cryptography Javier Pallarés de Bonrostroa,1 González Vascoc,2 1
arXiv:2606.07341v1 [cs.CR] 5 Jun 2026
2
, Ana I. González-Tablasb,1
, Marı́a Isabel
Computer Science and Engineering Department, Universidad Carlos III de Madrid, Spain Mathematics Department, Universidad Carlos III de Madrid, Spain
Received: date / Accepted: date
Abstract The transition to post-quantum cryptography (PQC) requires not only replacing vulnerable cryptographic primitives, but also refactoring the surrounding software logic that manages keys, parameters, serialization formats, and protocol-specific execution flows. While existing PQC migration frameworks provide organizational guidance, practical code-level remediation remains largely manual and error-prone. This paper evaluates whether large language models (LLMs) can be trained to assist in the migration of pre-quantum cryptographic code fragments to post-quantum or quantumresistant counterparts while preserving functional correctness. To this end, we introduce a reproducible experimental framework built around a synthetic yet executable dataset of 800 paired Python code fragments covering six cryptographic families—symmetric encryption, hashing, message authentication codes, authenticated encryption, digital signatures, and key exchange—as well as combined multi-primitive cases. Each pair is validated through category-specific functional tests, enabling both dataset quality control and objective evaluation of model-generated migrations. Four models are assessed: GPT-4.1 in a zero-shot setting, and finetuned versions of GPT-3.5-turbo, GPT-4.1-mini, and CodeLlama-7B-Instruct. The results show that domain-specific fine-tuning is essential for reliable cryptographic migration. The fine-tuned GPT-4.1-mini model achieves the best overall performance, with a mean static similarity of 0.9072 and a dynamic functional correctness rate of 92.5%, substantially outperforming the zero-shot baseline. A a
e-mail: [email protected] e-mail: [email protected] c e-mail: [email protected] b
complementary validation on six open-source repositories further shows that the approach can produce useful migrations in localized cryptographic modules, while also revealing limitations in larger projects with complex dependencies and cross-module interactions. These findings suggest that fine-tuned LLMs can serve as practical components in future crypto-agile migration pipelines, provided they are coupled with automated verification and dependency-aware validation. Keywords Post-quantum cryptography · Large language models · Cryptographic migration · Code migration · LLM fine-tuning · Automated verification · Crypto-agility
1 Introduction The upcoming global transition toward Post-Quantum Cryptography (PQC) represents one of the most complex challenges in modern cybersecurity and software engineering. Future quantum computers are expected to compromise the security of widely deployed asymmetric schemes such as RSA, Diffie–Hellman, ECDSA, and EdDSA through Shor’s algorithm, while Grover’s algorithm reduces the effective strength of symmetric primitives, forcing key sizes to increase in order to maintain equivalent security margins [12]. Consequently, cryptographic infrastructures must migrate to quantum-resistant primitives such as the ModuleLattice-Based Key-Encapsulation Mechanism (MLKEM) and the Module-Lattice-Based Digital Signature Algorithm (ML-DSA) digital-signature scheme, and adopt stronger symmetric constructions like AES-256 and SHA-3-512, in line with the migration roadmaps proposed by NIST, ENISA and the European Com-
2
mission for long-term protection in a post-quantum setting [12, 4, 9]. Beyond the challenges associated with adopting cryptographic constructions based on entirely new mathematical assumptions—including assessing the practical hardness of post-quantum problems and securely implementing these new primitives—the primary bottleneck in PQC adoption is operational. Migration at scale requires discovering and refactoring cryptographic usage across large heterogeneous codebases, replacing obsolete primitives without breaking interoperability, and verifying that the resulting system preserves both its functional and security guarantees. Recent migration frameworks and guidance documents emphasise that, in the absence of crypto-agility, these processes tend to be slow, costly and error-prone, and therefore call for systematic inventory, dependency analysis and staged replacement strategies [12, 40, 4]. The motivation for this study arises from this gap: although such conceptual frameworks exist, there is a lack of practical tools capable of performing largescale cryptographic refactoring automatically. This work addresses that gap from a software-engineering perspective, leveraging synthetic data, LLM-based migration models, and functional validation to automate the transition toward PQC. At the same time, Large Language Models (LLMs) have reached remarkable maturity in code understanding, automated refactoring and repository-scale modernization. Industrial and academic studies demonstrate that LLMs can perform coordinated transformations across thousands of files, generate unit tests and validate behavioral equivalence, often reducing manual effort by up to 90 % [50, 39, 22, 49]. Surveys on LLMs for vulnerability detection and secure coding further highlight their increasing ability to reason about program semantics and security contexts [42]. This convergence between an urgent need (massive PQC migration) and an emerging capability (AI-assisted code transformation) opens an opportunity to explore whether LLMs can accelerate and help systematize post-quantum transitions in a reliable and verifiable way. For cryptographic software, recent evidence suggests that LLMs are beginning to outperform or complement static tools in specialized security tasks. Masood et al. [24] compare multiple LLMs with CryptoGuard, CogniCrypt and Snyk Code on OWASP and MASC benchmarks, showing that GPT-4-o-mini achieves higher precision and recall in detecting API misuses and weak configurations. Li et al. introduce CryptoScope [17], which combines retrieval-augmented generation (RAG) and chain-of-thought prompting over a curated 12,000-
entry cryptographic knowledge base to identify logiclevel vulnerabilities. Maskey et al. [23] evaluate LLMs on cryptanalysis and side-channel reasoning tasks, highlighting both capability boundaries and dual-use risks. These studies demonstrate that LLMs can reason about cryptographic constructs; yet, no work has established a complete pipeline for verified code migration toward PQC. Meanwhile, ecosystem-scale measurements reveal that most deployed applications remain non-ready for PQC. Strauss et al. [41] analyze more than 4,000 Android apps, uncovering widespread use of SHA-1, MD5, and RSA, and assess whether state-of-the-art LLMs can assist in replacing them. Their experiments show partial success in local substitutions (for example, SHA-1 → SHA-256) but systematic failure in end-to-end PQC migrations such as RSA/ECC → Kyber/Dilithium due to missing multi-file context, dependency resolution, and validation. Far from being a negative result, this study provides a realistic baseline that clarifies where progress is required: semantic coherence across files, explicit cryptographic context, and automated functional verification. Taken together, these developments motivate the central question of this paper: Can large language models be effectively trained and evaluated to assist in the migration of prequantum cryptographic code to post-quantum counterparts while preserving functional correctness? To answer this question, we design an experimental framework based on synthetic data generation, automatic validation and LLM fine-tuning, enabling a controlled and reproducible evaluation of AI-assisted cryptographic migration.
1.1 Related Work Migration frameworks and crypto-agility. Hasan et al. [12] propose a structured framework for migrating legacy systems to PQC through dependency analysis and case studies, while Sikeridis et al. [40] extend the concept of crypto-agility to enterprise governance, introducing ELCA as a model for rapid algorithm substitution. These works define the organisational process for PQC transition but stop short of automating code transformation. Complementary initiatives such as IBM’s Cryptography Bill of Materials (CBOM) and CISA’s Automated Cryptographic Discovery and Inventory (ACDI) focus primarily on discovery and inventory rather than remediation and code refactoring [36, 5]. In parallel, official roadmaps by ENISA
3
and NIST explicitly recommend proactive migration away from RSA/ECC towards standardised postquantum schemes, including the adoption of hybrid classical–post-quantum (C-PQC) constructions, while emphasising the need to incorporate crypto-agility into existing architectures, rather than treating PQC as a one-off switch [4, 44, 3].
well LLMs can perform verified cryptographic migration tasks.
1.2 Contributions This work proposes a reproducible experimental framework that bridges conceptual PQC migration models and practical, verifiable LLM-assisted transformation at the source-code level. The contributions are as follows:
LLMs for code refactoring and modernization. LLMdriven refactoring research has grown rapidly. Google reports multi-file code edits performed autonomously by LLM agents within its monorepository [50]. Tools such as GUPPY and Hybrid API Migration demon- C1. Dataset for reproducible cryptographic migration. We introduce a balanced and verifiable strate automated API upgrades on Android using corpus of approximately 800 pre- to post-quantum prompt-based generation and semantic patch recode pairs across seven categories: hashing, MAC, finement [22, 49]. Models such as TransCoder and digital signatures, key-exchange / KEM, authentiCode-LLaMA achieve high translation accuracy becated encryption, and combined schemes. Each pair tween programming languages [15, 38], while Few-Shot includes unit tests (e.g., sign/verify, encaps/deProgram Refactoring attains almost 96 % functional caps, encrypt/decrypt) to ensure correctness. To correctness in structural improvements with GPT-3.5 our knowledge, this is the first structured dataset [39]. Despite these achievements, all operate in securitydesigned specifically for evaluating LLMs on PQC agnostic domains and do not consider cryptographic migration tasks. The dataset has been published correctness or protocol integrity. as an openly available research artifact to support reproducibility and future benchmarking [33]. To LLMs in security and cryptographic reasoning. Recent our knowledge, this is the first structured dataset surveys [42] and benchmarks [24, 17, 23] confirm that designed specifically for evaluating LLMs on PQC LLMs can detect cryptographic misuse and reason migration tasks. about algorithmic intent, bridging part of the gap between static analysis and human auditing. However, C2. LLMs trained for verifiable cryptographic migrations. Four representative models (GPT-4.1, the ability to produce and verify secure cryptographic GPT-3.5-turbo, GPT-4.1-mini and CodeLlamareplacements remains untested. These works therefore 7B) are evaluated under zero-shot and fine-tuned provide context and motivation for exploring LLMs not configurations. Metrics combine lexical-structural only as detectors but as active participants in secure similarity, functional success rate, execution time code transformation. and cost per token. Fine-tuned models show significant improvement in localized, verifiable migrations Ecosystem readiness and empirical studies. Strauss et such as HMAC-SHA1 to HMAC-SHA3-256 or AESal. [41] represent the first attempt to use LLMs for 128 to AES-256, while complex PQC replacements real-world PQC migration. Their findings show that like RSA to Kyber remain challenging. the models succeed only in trivial substitutions and fail when context and validation are required. Other em- C3. Automated verification pipeline. We implement a validation pipeline integrating static pirical efforts address complementary angles: Ahmed et analysis, dynamic testing and runtime verification al. [1] survey PQC support in cryptographic libraries; to confirm that migrated code is syntactically Ricchizzi et al. [37] examine industrial deployment of valid, executes and produces valid cryptographic hybrid certificates; and policy roadmaps from NIST outputs. This approach transforms PQC migration and the PQC Coalition [29, 34] outline strategic but into a measurable benchmark, enabling objective non-automated migration guidance. None provide recomparison across models and full reproducibility. producible, code-level experiments assessing LLM caC4. Alignment with enterprise frameworks and pability. crypto-agility. The workflow maps its stages Overall, prior literature provides three partial pil(inventory, candidate selection, patch generalars: (1) frameworks and policies for PQC migration tion and validation) to existing PQC migration planning, (2) advances in LLM-based software refacframeworks [12, 40], facilitating its integration into toring, and (3) empirical evidence of ecosystem unorganizational change-management and continuouspreparedness. What is missing is a controlled environvalidation processes. ment that connects these strands and quantifies how
4
Together, these contributions complement largescale empirical work such as Strauss et al. [41]. While that study documents the current limits of LLMs in practical PQC migration, our framework provides the experimental foundation –data, methodology and verification– to train and evaluate such models under reproducible conditions. The combination of empirical observation and controlled experimentation establishes a basis for future research on reliable, AI-assisted post-quantum migration.
1.3 Structure of the paper The remainder of this paper is organized as follows. Section 2 provides the necessary background on large language models and software migration pipelines, with a particular focus on their relevance to cryptographic transformation tasks. Section 3 presents the overall methodology of the study. Section 4 describes the dataset construction process, including primitive selection, synthetic generation, and validation. Next, Section 5 details the experimental setup for model selection, preparation, fine-tuning, and testing. Section 6 introduces the automated evaluation framework used to assess both static similarity and dynamic functional correctness. The quantitative results are reported in Section 7, followed by the application of the best-performing model to real-world repositories in Section 8. Finally, Section 9 discusses the main limitations of the approach, and Section 10 concludes the paper and outlines future research directions. Additional details on dataset composition are provided in Appendix A.
2 Background on LLMs and migration pipelines Large language models (LLMs), built on the Transformer architecture introduced by Vaswani et al. [45], have rapidly evolved into general-purpose systems for code understanding, generation, and transformation. Recent surveys confirm their growing impact across software engineering tasks, including code generation, refactoring, testing, repair, and repository-scale maintenance [48, 13]. Industrial studies further show that these models can apply thousands of coordinated edits across monolithic repositories with minimal human supervision. For instance, Ziftci et al. report that LLM-driven agents were able to execute more than 93,000 coherent modifications within Google’s monorepository, maintaining semantic consistency and passing the full battery of automated tests [50]. Similar results appear in refactoring and API modernisation
tasks, where LLMs generate patches, run tests, and refine outputs iteratively [22, 49]. Academic work on program translation further demonstrates that models such as TransCoder [15] and Code LLaMA [38] can perform unsupervised or instruction-tuned translation between programming languages with high structural fidelity. Few-shot refactoring experiments have also shown that models like GPT-3.5 can achieve more than 95% correctness when guided with a small set of examples [39]. These advances have led to the adoption of pipeline oriented workflows in software engineering. In such pipelines, the LLM is only one component inside a broader chain that includes discovery, patch generation and validation. A common pattern is: (i) static or semantic analysis identifies candidate locations; (ii) the LLM synthesises the patch; and (iii) compilation, test execution or fuzzing validates the result. This structure mirrors modern CI/CD processes and generalises across domains such as Android API upgrades [22, 49], multi file transformations in large codebases [50], and even multimodal repository reconstruction [19]. The same organisation is now standard in scientific applications that require reproducible train/test cycles and controlled comparisons across multiple model families. One of the design dimensions of LLM pipelines is the exploitation mode. The most popular ones are described next: – Zero shot prompting treats the model as a general assistant: the input code and a natural language instruction describe the target transformation. This requires no model adaptation but performs poorly in specialised domains such as cryptography, where subtle type or parameter constraints are essential. – Retrieval augmented generation (RAG) improves grounding by injecting external documentation or specifications retrieved from a vector index; this reduces hallucinations and enables adherence to strict policies, an approach used in hybrid pipelines such as CryptoScope [17]. – Finally, fine tuning adapts the model weights to a domain specific dataset, achieving the highest task accuracy. This strategy underpins advanced refactoring frameworks and translation systems [15, 38, 39] and is the most suitable option when correctness and reproducibility are required. A second design dimension concerns the number of models involved. Recent work shows that multi stage or agent based systems outperform single model approaches. Google’s migration agent coordinates planning, patching and validation steps [50], while AutoP2C decomposes tasks into extraction, hierarchical planning
5
and iterative repair modules [19]. This modular design allows systematic evaluation: each model (classifier, generator, reviewer) can be swapped or ablated while keeping the pipeline intact. Cryptographic code migration pipelines. Despite this progress, cryptographic migration remains under explored. Existing research focuses largely on detecting misuses (such as API misuse detection in LLM based analyses [24] or logic level vulnerability identification [17]) rather than generating verified replacements for complete primitives. There are no prior works addressing end to end migration from RSA or ECDSA to post quantum schemes such as Kyber or Dilithium, nor evaluating these transformations functionally. This gap motivates the present study, which adopts the pipeline perspective described above and integrates dataset construction, fine tuning and dynamic testing to examine how far current LLMs can be pushed as practical engines for post quantum cryptographic migration. 3 Methodology The motivation for this study stems from the current lack of practical tools capable of performing large-scale cryptographic migrations. Although several frameworks have been proposed to guide post-quantum readiness at an organizational level [12, 40, 29, 34], these approaches remain conceptual and depend heavily on manual auditing to identify, modify and validate the affected cryptographic components. Existing inventory and discovery initiatives, such as IBM’s CBOM and CISA’s ACDI [36, 5], assist in locating legacy primitives but do not automate their remediation. Furthermore, recent empirical studies show that current LLMs can detect cryptographic misuses but do not reliably achieve end-toend post-quantum migrations in real software systems [24, 17, 41]. The objective of this research is to evaluate whether fine-tuned large language models (LLMs) can automatically and reliably transform pre-quantum cryptographic code into functionally equivalent post-quantum implementations, minimizing manual intervention and human error. We have designed the methodology as a reproducible, data-driven experimental process, addressing the problem from a software-engineering perspective and following a modular experimental design that includes synthetic data generation and LLM training and validation (see Figure 1). Dataset design and generation. A central component of the methodology is the construction of a dedicated dataset for post-quantum code migration (see phase
Fig. 1: Overall methodological pipeline of the study. The workflow is organized into four main stages: dataset design and generation, LLM preparation (training), LLM evaluation (testing), and real-world validation. The figure also highlights the dual role of the automated verification harness, which is reused both to validate the dataset during construction and to evaluate model-generated migrations. 1 in Fig. 1). To the best of our knowledge, no pub○ licly available benchmark currently provides paired prequantum and post-quantum implementations designed specifically for training and evaluating LLMs on this task. Moreover, collecting such pairs directly from realworld repositories is difficult in practice, since migrations are often incomplete, distributed across multiple files, tightly coupled to project-specific dependencies, or simply unavailable in aligned pre/post form. For this reason, we opted for a synthetic yet functionally accurate dataset, which makes it possible to generate large, balanced, and controlled examples while avoiding the unpredictability of production repositories. This design supports systematic evaluation across cryptographic categories such as hashing, message authentication, digital signatures, symmetric encryption, authenticated encryption, and combined cases, while preserving the verifiability needed for automated testing. As a result, the dataset serves as a central methodological asset for both model training and controlled evaluation.
LLM training and testing. From a technical point of view, the LLM preparation and evaluation stages 2 and○ 3 in Fig. 1) frame the migra(see phases ○ tion of cryptographic primitives as a transformation problem between two domains: quantum vulnerable algorithms such as RSA, ECDSA on one side, and quantum-safe primitives like CRYSTALS–Kyber or CRYSTALS–Dilithium on the other. However, the difficulty lies not in the mathematical substitution itself, but in the contextual dependencies that surround each primitive. Legacy codebases often integrate cryptogra-
6
phy through indirect abstractions, custom wrappers or dynamic imports, which hinder automated detection and replacement. Automated evaluation of code migration. The central methodological idea in this work is to treat code migration as a verifiable generation task. Rather than relying exclusively on static analysis, the model receives a concrete code fragment as input and must output a modified version that preserves functionality while replacing quantum-vulnerable cryptographic elements with their post-quantum counterparts. Crucially, the difficulty lies less in the mathematical substitution itself than in the surrounding software context: key material handling, parameter initialization, encoding and serialization conventions, error and exception management, and the interaction with application-specific wrappers or helper APIs. Accordingly, the goal is not only to generate syntactically valid code, but to produce changes that remain executable and behavior-preserving under realistic usage patterns. To operationalize this notion of verifiability, the au4 in Fig. 1) tomated verification harness (see phase ○ is applied at two points of the pipeline: during dataset 1 ), to ensure that the design and generation (phase ○ constructed examples are functionally valid, and during 3 LLM evaluation (phase ○), to assess whether the migrated outputs preserve the expected behavior. These tests check that the migrated code can be executed and that the expected input–output behavior is preserved (e.g., decrypting yields the original plaintext, signature verification succeeds for valid messages and fails for tampered ones), providing an objective signal of migration correctness beyond surface-level similarity. Application to real-world cases. Finally, to close the methodological loop, the fine-tuned model was applied to real open-source projects containing genuine crypto5 in Fig. 1). This final graphic dependencies (see phase○ phase allowed assessing the system’s practical usefulness beyond controlled settings, providing insight into how LLM-assisted migration behaves under realistic complexity, modularity and dependency constraints. Automated pipeline. Additionally, the workflow from dataset preparation to model evaluation was implemented as a scripted pipeline to support reproducibility and traceability. Automation primarily concerns the repeatable execution of (i) dataset formatting and split generation (e.g., conversion to ChatML and stratified train/validation splits), (ii) inference and static evaluation on validation samples, and (iii) functional evaluation through an automated test harness that
executes the corresponding base-case validators and aggregates pass/fail summaries. At the same time, key design steps remain human-guided, such as defining base cases, curating the set of variations included in the final dataset, configuring training runs through notebooks, and manually resolving failures when applying migrations beyond the controlled benchmark. Across stages, results are recorded as JSON/JSONL artifacts (e.g., per-sample similarity and per-test summaries), enabling quantitative comparisons across models. Language and cryptographic library selection. All experiments were conducted in Python because it provides a practical balance between cryptographic expressiveness, tooling maturity, and experimental reproducibility. Other languages such as C/C++ and Java are also widely used in security-critical software and would be natural candidates for productionoriented PQC migration studies. However, Python offers several advantages for a controlled experimental setting: it has a mature cryptographic ecosystem, including PyCA/cryptography, direct access to postquantum primitives through liboqs-python, and a simple execution model that facilitates dynamic testing of generated snippets. These properties make it easier to construct self-contained examples, execute them inside an automated test harness, and compare model outputs under repeatable conditions. The choice of Python should therefore be understood as a methodological decision rather than a claim that PQC migration is only relevant to Python codebases. By reducing language-level complexity, the experiments can focus on the core research question: whether LLMs can learn to transform cryptographic logic while preserving functional correctness. Extending the same methodology to other widely deployed languages such as C/C++, Java, Go, or Rust is left as future work. 4 Dataset design and generation The overall construction process of the corpus is summarized in Fig. 2, while Tables 1 and 2 describe the primitive mappings and the combined cases represented in the final dataset. Primitive selection and migration mapping. The first step in the dataset construction process is primitive selection and migration mapping (see 1.A in Fig. 2). The process requires a consistent mapping between prequantum and post-quantum primitives so that the resulting dataset reflects realistic software refactoring scenarios rather than purely theoretical substitutions. This
7
Table 1: Summary of primitive migrations and estimated difficulty per category. Category
Quantum Vulnerable Crypto
Quantum Resistant Crypto
Migration difficulty estimated
3DES, AES-128
AES-256
Medium. Duplicated padding, missized keys, obsolete imports.
Hash functions
MD5, SHA-1/2, BLAKE2b/s
SHA3-256
Low–Medium. Missing algorithms, reused hash objects, identifier mismatches.
Message authentication codes (MAC)
CMAC, HMACSHA1/2, BLAKE2, JWT-HMAC
HMAC-SHA3-256
Medium. Deprecated imports, method-signature errors, mismatched generation/verification.
Authenticated cryption
AES-GCM, AESCCM, AES-OCB3, AES-SIV, Fernet
ChaCha20-Poly1305
Medium. Invalid key sizes, missing methods, incorrect class wiring.
Digital signatures
RSA, DSA, ECDSA, Ed25519, Ed448
Dilithium5, SPHINCS+
High. Key-order errors, padding inconsistencies, incoherent signatures, wrong algorithm selection.
Key exchange
DH-2048/3072, ECDH, X25519, X448
Kyber1024
High. Non-shared secrets, incorrect method signatures, asynchronous API mismanagement.
Symmetric tion
encryp-
en-
Table 2: Distribution of combined pair cases in the dataset. Values indicate the number of variations for each unordered pair of primitive families. Digital signatures
Key exchange
Hashing
Message authentication codes
Authenticated encryption
16
16
16
8
16
Digital signatures
-
16
16
8
16
Key exchange
-
-
16
8
16
Hashing
-
-
-
8
16
Message authentication codes
-
-
-
-
8
Primitive family Symmetric tion
encryp-
Total number of variations
200
mapping is guided by recommendations from NIST, ENISA, and recent academic analyses of post-quantum transition strategies [12, 4, 40]. The objective is not only to replace cryptographic algorithms that are vulnerable in the quantum setting, but also to capture the structural changes that arise when moving from classical schemes such as RSA or ECDH to lattice-based constructions such as Dilithium or Kyber.
whereas others require replacing entire workflows, as in the transition from ECDSA to Dilithium or from ECDH to Kyber, where the involved primitives rely on fundamentally different algebraic structures and therefore demand substantially different arithmetic implementations. These differences directly shape the complexity of the dataset and the kinds of software-engineering errors that the models must learn to avoid.
At a methodological level, the key assumption is that post-quantum cryptography differs from classical cryptography not only in the underlying mathematics, but also in interface design, key material structure, and operational flow. Some migrations involve relatively local changes, such as increasing symmetric key sizes,
Symmetric cryptography provides the most straightforward migration path. Under Grover’s model, the effective security of a symmetric cipher is reduced, but robustness can be preserved by increasing the key size [12]. For this reason, AES-256 is selected as the standard post-quantum target for symmetric
8
Fig. 2: Dataset Generation Phase. The internal flow starts with the manual creation of migrations, continues with synthetic generation and validation of the resulting corpus.
encryption. Hash functions follow a similar logic: while SHA-2 remains viable with sufficiently long outputs, SHA-3 offers a more modern structure and stronger security margin [8]. Accordingly, SHA3-256 is used as the default target for hashing tasks, while SHA3-512 is reserved for higher-assurance contexts. Message authentication is considered in two settings: standalone MAC constructions and AEAD schemes, where integrity is integrated into the encryption primitive. Some MAC constructions, such as CMAC and GMAC, become problematic under quantum-superposition adversaries [14, 16], whereas HMAC remains secure when instantiated with a quantum-resistant hash. For this reason, HMAC– SHA3-256 is selected as the canonical post-quantum MAC. Authenticated encryption requires separate treatment. Although AES-GCM is widely deployed, it inherits structural issues from GMAC in quantum settings [16]. ChaCha20–Poly1305, by contrast, does not exhibit these weaknesses and is therefore used as the target AEAD scheme in the dataset. Public-key primitives require deeper structural changes. All classical schemes based on factoring or discrete logarithms are broken by Shor’s algorithm [12], and ENISA explicitly identifies them as highpriority migration targets [4]. For digital signatures, CRYSTALS–Dilithium is selected as the primary postquantum target due to its robustness and its role in the NIST standardization process. The dataset uses Dilithium5 to remain aligned with the highest security level considered in the study. In addition, selected signature examples also include SPHINCS+, a stateless hash-based signature scheme standardized by NIST, in order to cover an alternative post-quantum signature family with different security assumptions. For key establishment, CRYSTALS–Kyber is selected,
specifically the Kyber1024 variant, which corresponds to the strongest parameter set in the Kyber family [12, 40, 28]. Standardization note. NIST has standardized Kyber, Dilithium and SPHINCS+ as ML-KEM, ML-DSA and SLH-DSA, respectively[25, 26, 27]. However, the experimental pipeline used in this work relies on the CRYSTALS-family identifiers exposed by the pinned liboqs/liboqs-python toolchain. This choice does not alter the methodological conclusions of the study, since moving from the submission identifiers to the standardized ones is mainly an implementation-level substitution rather than a change in the migration problem itself. Repeating the experiments with ML-KEM, MLDSA and SLH-DSA identifiers is left as future work. Table 1 summarizes the selected pre-quantum and post-quantum mappings for the main primitive families, together with the estimated migration difficulty. Dataset design. A central contribution of this work is the construction of a controlled dataset specifically designed to train and evaluate large language models on cryptographic migration tasks (see 1.B in Fig. 2). The dataset acts both as a benchmark and as a source of supervised fine-tuning examples, enabling model behavior to be measured under repeatable and verifiable conditions. Its design follows one main principle: every entry must be functionally self-contained and executable, so that correctness can be assessed automatically. The final dataset contains 800 entries. Each entry consists of a paired transformation from a pre-quantum implementation to its post-quantum counterpart. Of these, 600 entries correspond to single-family migrations distributed across six cryptographic categories (listed in Table 1): symmetric encryption, digital signatures, key exchange, hashing, message authentication codes, and authenticated encryption. The remaining 200 entries correspond to combined cases. In this work, combined refers specifically to snippets in which exactly two different cryptographic primitive families appear together and must be migrated coherently within the same code fragment. This restriction is intentional: pairwise combinations introduce realistic multi-primitive interactions while keeping each migration concise, interpretable, and functionally testable. Larger compositions involving three or more primitives would increase realism, but would also add noise and make it harder to attribute failures to a specific migration pattern. Extending the dataset to more complex multi-primitive compositions is left as future work. The combined subset was constructed by considering all unordered pairwise combinations among the six
9
primitive families in the dataset: symmetric encryption, authenticated encryption, hashing, MAC, digital signatures, and key exchange. This results in 15 possible pair categories. The final subset contains 200 combined variations: most pair categories contain 16 variations, while the five pair categories involving MAC contain 8 variations each. This allocation reflects a pragmatic corpus-design decision. All pairwise interactions are represented, but MAC-involving combinations are sampled more compactly because they tend to share a stable taggeneration/tag-verification structure and therefore introduce less additional structural variability than combinations involving encryption, key establishment, or digital signatures. This keeps the combined subset bounded while still exposing the model to authentication logic in multi-primitive scenarios. The resulting distribution is summarized in Table 2. A key design decision was to ensure that every example captures the complete functional flow associated with its cryptographic family. In practice, this means that symmetric encryption examples include both encryption and decryption, digital signature examples include signing and verification, key-establishment examples validate the full shared-secret establishment flow (encapsulation/decapsulation for KEM-based schemes and shared-secret derivation for classical DH/ECDHstyle schemes), MAC examples include tag generation and verification, and hashing examples include digest computation together with the corresponding validation logic used in the test harness. This constraint ensures that migration is evaluated at the level of executable cryptographic behavior rather than as a superficial API substitution. Each dataset entry is stored in JSON Lines (JSONL) format, which is commonly used for LLM training corpora [7]. In addition to the paired preand post-quantum code fragments, each entry contains metadata such as the base case identifier, variation identifier, migration category, and the source and target algorithms involved. This structure supports downstream parsing, tokenization, and evaluation while preserving a uniform representation across all categories. The schema used throughout the dataset is shown in Listing 1, which formalizes the structure followed by all manually created and automatically expanded entries. Listing 1: JSONL schema used to represent each entry in the dataset. { " base_case_id ": " < numeric case id >" , " variation_id ": " < incremental id >" , " m i g ra t io n_ c at e g o r y ": " < crypto family >" ,
" p r e q u a n t u m _ a l g o r i t h m ": " < classical algorithm >" , " p o s t q u a n t u m _ a l g o r i t h m ": " < post - quantum algorithm >" , " prequantum_code ": " < Python code >" , " postquantum_code ": " < Python code >" , " description ": " < short human summary >" }
Dataset generation. Dataset generation (see 1.C in Fig. 2) followed a two-stage process combining manual seed design and controlled LLM-assisted expansion, followed by human review and filtering. First, a small set of base cases was written entirely by hand. These seed examples were designed to capture core migration patterns and to provide high-quality reference pairs from which broader variations could later be derived. Each manually created pair already satisfied the main design constraints of the dataset: semantic equivalence between the pre- and post-quantum versions, explicit forward and inverse operations, and executable code structure. Second, the dataset was expanded through controlled interactions with the generative model o4-mini-high. Since code-oriented LLMs typically show stronger performance in English [18, 11], all prompts and generated outputs were produced in English. The prompt context included the manually created base cases, the target JSONL schema, naming constraints, and a case guide describing the canonical cryptographic mappings to be followed. To reduce stylistic contamination across families, generation was organized as one dedicated conversation per category. Listing 2 shows two compact examples of generated dataset entries. Only the metadata, function headers and cryptographically relevant operations are shown; non-essential boilerplate is omitted for readability. The first example illustrates a digital-signature migration from RSA to Dilithium5, while the second shows a keyestablishment migration from Diffie–Hellman to Kyber1024. These examples highlight that the generated entries are not isolated primitive substitutions: each pair preserves the operational structure needed for later functional validation. Listing 2: Compact examples of generated dataset entries. Only metadata, function headers and cryptographically relevant operations are shown. # Example 1: digital signature migration { " base_case_id ": "1" , " variation_id ": "1.1" , " m i g r a t i o n _ c a t e g o r y ": " digital signature " , " p r e q u a n t u m _ a l g o r i t h m ": " RSA " , " p o s t q u a n t u m _ a l g o r i t h m ": " Dilithium5 " , " pr eq u an tu m_ c od e ": "
10 from cryptography . hazmat . primitives . asymmetric import rsa , padding from cryptography . hazmat . primitives import hashes
length =32 , salt = None , info =b ’ handshake data ’ ) . derive ( shared_key )
def generate_keys () : private_key = rsa . g e n e r a t e _ p r i v a t e _ k e y ( public_e xp on e nt =65537 , key_size =2048 ) return private_key , private_key . public_key ()
", " p o s t q u a n t u m _ c o d e ": " import oqs ALGORITHM = ’ Kyber1024 ’
def sign ( private_key , message : bytes ) -> bytes : return private_key . sign ( message , padding . PSS ( mgf = padding . MGF1 ( hashes . SHA256 () ) , salt_length = padding . PSS . MAX_LENGTH ), hashes . SHA256 () )
def generate_keys () : with oqs . K e y E n c a p s u l a t i o n ( ALGORITHM ) as kem : public_key = kem . g e n e r a t e _ ke y p a i r () secret_key = kem . e x p o r t _ s e c r e t _ k e y () return secret_key , public_key def e n c a p s u l a t e _ s h a r e d _ s e c r e t ( pe e r_ pu bl i c_ ke y ) : with oqs . K e y E n c a p s u l a t i o n ( ALGORITHM ) as kem : ciphertext , shared_secret = kem . encap_secret ( p e er _p ub l ic _k ey ) return ciphertext , shared_secret
def verify ( public_key , message : bytes , signature : bytes ) -> bool : ...
def d e c a p s u l a t e _ s h a r e d _ s e c r e t ( ciphertext , secret_key ) : ...
", " p ostquantum_code ": " import oqs import ctypes as ct ALGORITHM = ’ Dilithium5 ’ def generate_keys () : with oqs . Signature ( ALGORITHM ) as sig : public_key = sig . g e n e r a t e_ k e y p a i r () secret_key = sig . e x p o r t _ s e c r e t _ k e y () return secret_key , public_key def sign ( secret_key : bytes , message : bytes ) -> bytes : clean_key = secret_key . rstrip (b ’\\ x00 ’) with oqs . Signature ( ALGORITHM ) as sig : sig . secret_key = ct . c r e a t e _ s t r i n g _ b u f f e r ( clean_key , len ( clean_key ) ) return sig . sign ( message ) def verify ( public_key : bytes , message : bytes , signature : bytes ) -> bool : ... " } # Example 2: key - establishment migration { " base_case_id ": "2" , " variation_id ": "2.1" , " m i g ra t io n_ c at e go r y ": " key exchange " , " p r e q u a n t u m _ a l g o r i t h m ": " Diffie - Hellman " , " p o s t q u a n t u m _ a l g o r i t h m ": " Kyber1024 " , " prequantum_code ": " from cryptography . hazmat . primitives . asymmetric import dh from cryptography . hazmat . primitives . kdf . hkdf import HKDF from cryptography . hazmat . primitives import hashes parameters = dh . g e n e r a t e _ p a r a m e t e r s ( generator =2 , key_size =2048) def generate_keys () : private_key = parameters . g e n e r a t e _ p r i v a t e _ k e y () return private_key , private_key . public_key () def d e r i v e _ s h a r e d _ s e c r e t ( private_key , peer_public_ ke y ) : shared_key = private_key . exchange ( peer_pub li c_ k ey ) return HKDF ( algorithm = hashes . SHA256 () ,
" }
Expansion was carried out iteratively in batches of ten variations. After each batch, automatically detectable issues such as malformed JSONL structure or syntax errors were filtered through the testing workflow, and each accepted candidate was also reviewed manually to ensure that the code was semantically meaningful and complied with the migration requirements. This makes the generation process explicitly human-in-the-loop, rather than fully automatic, which is appropriate for a dataset intended to serve as a high-quality benchmark [46]. Between batches, reflective prompting was used to identify missing patterns, underrepresented structures, or inconsistencies in the corpus, following the general intuition of iterative self-refinement strategies [21, 35]. The expansion process also aimed to capture structural diversity beyond direct primitive substitution. As a result, the corpus includes heterogeneous programming patterns such as asynchronous execution, context managers, streaming I/O, class-based wrappers, and function-oriented implementations. This variability is deliberate: it exposes the future fine-tuned models to realistic syntactic and structural differences, reducing the risk that migration is learned as a purely lexical replacement task.
Dataset validation. Dataset validation (see 1.D in Fig. 2) was integrated directly into dataset construction. The same automated verification logic that later supports model evaluation was first used to verify the correctness of the dataset pairs themselves. This gives the validation stage a dual methodological role: it guarantees the quality of the benchmark during corpus
11
creation and later serves as the evaluation oracle for model-generated migrations. In practice, validation combines two complementary checks. First, syntactic validity is assessed during controlled execution inside the testing harness: malformed snippets fail immediately when they are loaded and executed. Second, dynamic functional tests verify that both the pre-quantum and post-quantum implementations satisfy the expected property of the corresponding cryptographic task. These properties depend on the category and include, for example, successful encrypt/decrypt round-trips, valid sign/verify flows, correct encapsulation/decapsulation behavior, or correct tag generation and verification in MAC settings. The verification workflow is organized by category. For each cryptographic family, a dedicated test file implements shared validation logic for the common cases and specialized subtests for variations with more specific calling conventions or code structures. A variation is considered correct and added to the final dataset only when the corresponding functional property is satisfied independently by both the pre-quantum and post-quantum implementations. The test harness prints structured JSON summaries reporting the total number of evaluated cases, the number of successful cases, and the identifiers of failed instances. These summaries provide the basis for the later quantitative evaluation of fine-tuned models. To make the validation logic concrete, Listings 3 and 4 show two representative examples from the dataset. The first illustrates a digital-signature migration from RSA to Dilithium5, where correctness is defined as successful signing and verification in both the pre-quantum and post-quantum branches. The second illustrates a key-establishment migration from Diffie–Hellman to Kyber1024, where correctness is defined by shared-secret agreement in the classical branch and encapsulation/decapsulation agreement in the post-quantum branch. Listing 3: Representative validation pattern for a digital-signature migration from RSA to Dilithium5.
pre_priv , pre_pub = pre_keygen () pre_sig = pre_sign ( pre_priv , TEST_MESSAGE ) if not pre_verify ( pre_pub , TEST_MESSAGE , pre_sig ) : raise ValueError (" Pre - quantum verification failed .") # --- Post - quantum branch : Dilithium5 sign / verify --post_keygen = post_ns [ mapping [" generate_keys "]] post_sign = post_ns [ mapping [" post_encrypt "]] post_verify = post_ns [ mapping [" post_decrypt "]] post_priv , post_pub = post_keygen () post_sig = post_sign ( post_priv , TEST_MESSAGE ) if not post_verify ( post_pub , TEST_MESSAGE , post_sig ) : raise ValueError (" Post - quantum verification failed .") return True
Listing 4: Representative validation pattern for a keyestablishment migration from Diffie–Hellman to Kyber1024. # Dataset entry : Diffie - Hellman -> Kyber1024 # Functional property : # - classical branch : both parties derive the same secret # - PQ branch : encapsulation and decapsulation produce the same secret def v a l i d a t e _ k e y _ e x c h a n g e _ m i g r a t i o n ( variation , mapping ) : pre_ns , post_ns = {} , {} # Load pre - quantum and post - quantum snippets . exec ( variation [" p re qu a nt um _c o de "] , pre_ns ) exec ( variation [" po s t q u a n t u m _ c o d e "] , post_ns ) # --- Pre - quantum branch : Diffie - Hellman shared secret --pre_generate = pre_ns [ mapping [" p r e _ g e n e r a t e _ key s "]] pre_derive = pre_ns [ mapping [" p r e _ d e r i v e _ s h a r e d _ s e c r e t "]] pre_priv_a , pre_pub_a = pre_generate () pre_priv_b , pre_pub_b = pre_generate () pre_shared_a = pre_derive ( pre_priv_a , pre_pub_b ) pre_shared_b = pre_derive ( pre_priv_b , pre_pub_a ) if pre_shared_a != pre_shared_b : raise ValueError (" Pre - quantum key agreement failed .")
TEST_MESSAGE = b " Test digital signature message ."
# --- Post - quantum branch : Kyber1024 KEM flow --post_generate = post_ns [ mapping [" p o s t _ g e n e r a t e _ k e y s "]] post_encap = post_ns [ mapping [" p o s t _ e n c a p s u l a t e _ s h a r e d _ s e c r e t "]] post_decap = post_ns [ mapping [" p o s t _ d e c a p s u l a t e _ s h a r e d _ s e c r e t "]]
def v a l i d a t e _ s i g n a t u r e _ m i g r a t i o n ( variation , mapping ) : pre_ns , post_ns = {} , {}
post_secret_key , po s t_ pu bl i c_ ke y = post_generate ()
# Dataset entry : RSA -> Dilithium5 # Functional property : sign / verify succeeds in both branches .
# Load pre - quantum and post - quantum snippets . exec ( variation [" p re qu a nt um _c o de "] , pre_ns ) exec ( variation [" p o s t q u a n t u m _ c o d e "] , post_ns ) # --- Pre - quantum branch : RSA sign / verify --pre_keygen = pre_ns [ mapping [" generate_keys "]] pre_sign = pre_ns [ mapping [" pre_sign "]] pre_verify = pre_ns [ mapping [" pre_verify "]]
ciphertext , shared_enc = post_encap ( p os t_ pu b li c_ k ey ) shared_dec = post_decap ( ciphertext , p os t_ se c re t_ k ey ) if shared_enc != shared_dec : raise ValueError (" Post - quantum KEM validation failed .")
12 return True
5 LLM training and testing Model selection. Selecting appropriate models was essential to evaluate the feasibility of automated cryptographic migration (see 2.A in Fig. 5). The study sought to include both proprietary and open-source architectures to ensure that the proposed methodology could be reproduced across computational and licensing constraints. The selection prioritized diversity in scale, accessibility, and architecture while maintaining compatibility with modern instruction-tuning frameworks. The selection of models followed a staged process aimed at covering a representative spectrum of instruction-tuned code LLMs available in 2025: stateof-the-art proprietary systems, widely adopted midrange models, and open-source architectures suitable for local experimentation. This diversity is essential in cryptographic migration, where the goal is not only to assess raw performance, but also reproducibility, costefficiency, and feasibility under constrained hardware and budgets. From the proprietary side, OpenAI’s models stand out for their consistent leadership in coding benchmarks such as HumanEval+ and MBPP+ [31]. Their API provides a mature fine-tuning pipeline, deterministic message formatting (ChatML), extended context windows up to one million tokens, and precise per-token usage accounting [32, 30]. Such properties make them strong candidates for controlled scientific evaluation, where traceability and repeatability are paramount. In contrast, open-source alternatives play a complementary role: they allow full control over the training environment, zero API cost, complete reproducibility, and the ability to inspect intermediate representations. Among them, CodeLlama stands out as Meta’s code-specialized LLaMA variant, trained on 500B tokens of source code and evaluated extensively on codegeneration benchmarks [38]. When combined with lowrank adaptation and 4-bit quantization, CodeLlama-7B can be fine-tuned on commodity GPUs using QLoRA [6], making it an accessible option for academic laboratories and low-resource research environments. Next, we list the models selected and the rationale behind the selection: – GPT-4.1. Included as the zero-shot reference model in the original experimental protocol. It represents the strongest general-purpose model publicly available through API at the time of the study, with top-tier coding accuracy and a 1Mtoken context window [30]. Using GPT-4.1 without
prior fine-tuning establishes a realistic ceiling for what can be achieved without domain-specific adaptation. – GPT-3.5-turbo. By 2024–2025, GPT-3.5 remained one of the most widely deployed LLMs in both industry and academia due to its stability, mature documentation and extremely low operational cost [32]. Its widespread adoption is reflected in several software-engineering studies that rely on GPT-3.5-Turbo as a practical baseline, including work on automatic refactoring where the model achieves high accuracy in few-shot settings [39]. Including GPT-3.5-turbo fine-tuning therefore enables direct comparison with a model still present in a large fraction of real-world systems and commonly used in academic benchmarks. – GPT-4.1-mini. Chosen as the primary fine-tuning target due to its favourable balance between cost and performance. According to OpenAI’s public evaluations, GPT-4.1-mini matches or exceeds GPT-4o in several reasoning and coding tasks at a fraction of the cost [31]. Its training and inference prices (up to 80% lower than GPT-4.1) make large-scale experiments feasible within a restricted budget [32]. Critically, it also inherits the full 1Mtoken context window, enabling long examples to be included without segmentation. – CodeLlama-7B-Instruct. Selected as the opensource baseline to test the replicability of the pipeline without proprietary APIs. CodeLlama is designed specifically for code and performs competitively on benchmarks relative to much larger open models [38]. Using QLoRA [6], the 7B variant can be fine-tuned on GPUs with 8–16 GB of VRAM, demonstrating that cryptographic migration can be studied even in resource-limited academic settings. Its inclusion also enables direct comparison between closed and open ecosystems. In summary, the final selection spans three axes crucial for this study: performance ceiling, industry relevance, and open-source feasibility (see Table 3). This combination ensures that the evaluation captures not only state-of-the-art behavior but also the practical trade-offs faced by organizations that must deploy PQC migration tools under heterogeneous constraints. Experimental setting. To ensure methodological parity across all evaluated models, every experiment was conducted under a uniform configuration: an 8k-token context window, identical normalization and whitespace handling, and a shared tokenization layer based on tiktoken. Although tiktoken is natively optimized for OpenAI models, its systematic use as a
13
Table 3: Summary of selected models and rationale. Model
Ecosystem
Role in study
Fine-tuned
Notes / constraints
GPT-4.1
Prop. (API)
No
GPT-3.5-turbo
Prop. (API)
GPT-4.1-mini
Prop. (API)
CodeLlama-7BInstruct
OSS (local)
Zero-shot upper-bound reference Widely deployed, low-cost baseline Primary fine-tuning target (cost/perf) Open baseline for pipeline replicability
1M-token context; strong coding baseline Mature documentation; low operational cost 1M-token context; up to 80% cheaper than GPT-4.1 4-bit QLoRA; commodity GPUs (8–16 GB VRAM)
SYSTEM: You are a code assistant that migrates prequantum cryptographic code to post-quantum code. USER: MIGRATION CATEGORY: <category> PRE-QUANTUM CODE: <python snippet>
Yes Yes Yes (QLoRA)
[INST] You are a code assistant that migrates prequantum cryptographic code to post-quantum code. MIGRATION CATEGORY: <category> PRE-QUANTUM CODE: <python snippet> [/INST]
Fig. 4: Instruction-style prompt for CodeLlama models Fig. 3: ChatML-style prompt used for OpenAI models
measurement layer allowed fair comparison of token consumption across all architectures, including the open-source models. This alignment follows standard evaluation practices in LLM benchmarking [47, 20]. A crucial consideration in cross-model evaluation is that different model families rely on different conversational abstractions. The OpenAI models (GPT-4.1, GPT-3.5-turbo, GPT-4.1-mini) operate using a rolebased chat interface (system, user, assistant), following the ChatML format. This interface is natively supported by the API and has been shown to stabilize instruction following [47]. Consequently, the structured prompt used for these models mirrors the internal message hierarchy (see Fig 3). This two-message structure was kept constant for all OpenAI models, regardless of whether they were used in zero-shot mode (GPT-4.1) or fine-tuned (GPT-3.5turbo, GPT-4.1-mini). The uniformity of this format isolates the effect of fine-tuning and avoids confounding variables arising from prompt engineering. In contrast, decoder-only open-source models such as CodeLlama-7B-Instruct do not implement a role-based chat protocol. Instead, they use an instruction boundary format built around the [INST] ... [/INST] markers, which is consistent with the LLaMA instruction-tuning methodology reported in the literature [43]. For these models, the same semantic content (category + pre-quantum code) must be serialized into a single completion prompt (see Fig 4).
The model then learns to produce the post-quantum code directly after the [/INST] token. This completiononly training scheme is standard in instruction-tuned LLaMA-family models, where the text following the closing marker constitutes the supervised target during fine-tuning [43]. This setup aligns with the expectations of the TRL SFTTrainer, which implements supervised instruction tuning by treating only the postprompt segment as the label sequence, a method widely adopted in open-source adaptation pipelines [6]. Despite these interface differences, both prompting strategies encode the same information: (1) the migration task definition, (2) the cryptographic primitive category, and (3) the input code to transform. Maintaining semantic equivalence ensures that the comparison between OpenAI chat models and a locally trained CodeLlama remains meaningful and technically fair, as recommended in cross-family LLM benchmarking [47, 20]. Finally, normalization of prompts into structured templates—either ChatML or [INST]—ensures that model behaviour reflects differences in architecture and training, not inconsistencies in how instructions are presented. This consistency is essential for isolating model-specific effects in downstream analyses such as static similarity, functional correctness, and error patterns emerging during cryptographic migration. OpenAI-hosted models were fine-tuned through the official API pipeline, while the open-source configuration relied on the HuggingFace Transformers ecosystem with quantized fine-tuning via bitsandbytes and
14
QLoRA. This setup reduced the memory footprint of CodeLlama-7B to 4-bit precision, allowing execution on a single high-end GPU without compromising numerical stability. All environments were instrumented with deterministic logging and version control to ensure exact reproducibility. This configuration established a controlled foundation for the fine-tuning and evaluation phases. Dataset split for training and testing. A crucial step in preparing the corpus for model training was the construction of a clean, balanced and statistically meaningful dataset split (see 2.B in Fig. 5). Following established practices in supervised LLM training and dataset curation [7, 46], the 800 validated examples were divided into 720 training samples and 80 held-out validation samples. The partition was generated as a random stratified holdout split. Stratification was performed by migration category, ensuring that every cryptographic family was represented in both the training and validation subsets, while randomness avoided manually selecting favourable validation examples. For each core cryptographic category–hashing, message authentication codes (MAC), digital signatures, key exchange, symmetric encryption, and authenticated encryption– exactly 90 samples were allocated to the training split and 10 samples to the validation split. This ensured that every primitive was equally represented during optimization and evaluation, avoiding biases caused by uneven category frequencies. Combined cases, i.e., snippets involving two different cryptographic primitive families, were handled separately due to their higher structural complexity. These entries were distributed so that both the training and validation subsets preserved a comparable mix of primitive-pair interactions. This design choice reduces the risk of overfitting to single-primitive patterns while still exposing the model to realistic multi-primitive interactions, which are common in production systems [46]. Stratification is particularly important in cryptographic migration tasks, where different primitives exhibit substantially different code structures, dependency patterns and verification requirements. By preserving proportional representation across both simple and combined categories, the resulting splits enable reliable generalization analysis and fair cross-model comparison. A full k-fold cross-validation protocol was considered but not adopted as the main evaluation strategy. In conventional machine-learning settings, k-fold validation is often feasible because training can be repeated
Table 4: Summary of dataset composition by cryptographic category. Category
Train
Validation
Hash functions MAC Digital signatures Key exchange / KEM Symmetric encryption Authenticated encryption (AEAD) Combined primitives
90 90 90 90 90 90 180
10 10 10 10 10 10 20
Total
720
80
many times at moderate cost. In this study, however, each fold would require launching an independent finetuning run for each model under evaluation. For hosted models, this implies creating a new training file and a separate fine-tuning job for every fold through the API; for local models, it implies repeating the full GPU fine-tuning process. Consequently, a 5- or 10-fold protocol would multiply training time, inference time and cost by the number of folds, making it disproportionate for an 800-example exploratory study. Moreover, the purpose of this work is not to estimate a single production-grade generalization score, but to compare model behaviour under a controlled, reproducible validation protocol. However, to reduce the risk that the reported results depend on a single favourable split, an additional robustness check was performed using a second random stratified holdout partition. This alternative split used the same dataset size and proportions as the original experiment–720 training examples and 80 validation examples–but changed the random seed. The same GPT-4.1-mini fine-tuning and evaluation procedure was then repeated on this new partition. This repeated stratified holdout does not replace a full k-fold protocol, but it provides additional evidence that the observed performance is not an artefact of a uniquely favourable train/validation split. Table 4 provides a compact overview of the dataset composition, while the complete distribution of cryptographic primitives across training and validation splits is reported in Appendix A for reproducibility and transparency. LLM training and testing Each model selected was trained on the same dataset and later applied to unseen code fragments to assess its capacity for reliable cryptographic migration (see 2.C in Fig. 5). Notice that while the model training is the last step in model preparation (Fig. 5 details this phase), Fig 6 provides an overview of the model testing (evaluation) pipeline.
15
Fig. 5: LLM Preparation Phase. The validated synthetic dataset is split into train and validation subsets. Only the train split is used for dataset adaptation and model-specific fine-tuning, while the validation split is reserved for the evaluation phase. GPT3.5 Turbo, GPT-4.1 Mini, and CodeLlama-7B Instruct undergo fine-tuning, whereas GPT-4.1 is included as a zero-shot reference and bypasses the preparation stage.
eter schedules. Validation loss was monitored after each epoch to detect overfitting and to confirm stable convergence. Once trained, the models were evaluated on the held-out validation split, e.g., code samples not seen during fine-tuning (see Fig. 6). The migration pipeline emulated a realistic workflow: source files were loaded, automatically segmented when exceeding the context limit, and enriched with structured prompts specifying their cryptographic category. The model then generated the migrated code, which was reconstructed into full executable files. No automatic post-processing was applied at this stage to preserve the raw output for controlled analysis. Details on the approach taken to evaluate the models’ performance when asked to migrate the code fragments of the validation dataset are described next in Section 6.
6 Automated evaluation of code fragments migration After model selection, the training phase focused on adapting the dataset and training workflow to each model family. All preprocessing, dataset conversion, prompt formatting, evaluation scripts, and local fine-tuning code were implemented in Python. For OpenAI-hosted models, fine-tuning was executed through the official API, while the local CodeLlama7B-Instruct configuration relied on the HuggingFace transformers ecosystem and trl for supervised finetuning. This distinction allowed both hosted and local models to be evaluated under a common experimental protocol while respecting the different interfaces required by each architecture. The fine-tuning process was supervised and instruction-based, aligning the model’s output generation with the expected format of pre- and postquantum code pairs. Each training instance consisted of a dialogue-style record following the ChatML convention: a system message defining the migration context, a user message containing the pre-quantum code and its cryptographic category, and an assistant message providing the corresponding post-quantum version. This conversational structure preserved alignment between fine-tuning and inference, enabling the model to generalize its reasoning to new unseen examples. Fine-tuning was carried out over approximately one million tokens using a learning rate of 2 × 10−4 , batch size of 4, and six epochs. For local experiments, CodeLlama-7B was trained using QLoRA, which allowed efficient adaptation on a single NVIDIA A100 GPU. Hosted OpenAI models were trained through the fine-tuning API under equivalent hyperparam-
Evaluating the effectiveness of large language models in cryptographic migration requires more than syntactic comparison between generated and reference code. Functional correctness, reproducibility, and error traceability are equally critical, as cryptographic transformations must preserve both algorithmic behavior and structural integrity. The evaluation framework developed for this study was therefore designed to measure multiple dimensions of model performance, combining static and dynamic verification within a unified testing pipeline. Fig. 6 outlines the end-to-end procedure used to evaluate the models. Recall that per dataset design, each dataset entry is paired not only with pre-quantum and post-quantum code, but also with a test designed during dataset construction to validate the expected functionality of that specific snippet (e.g., encryption/decryption roundtrips, signing/verification). The post-quantum code in each dataset entry can be used to assess statically the output of the LLM being evaluated. The per-entry test oracle enables functional (dynamical) checking at scale and is used both to validate the dataset itself and to evaluate model outputs on unseen samples. Model evaluation is performed through a Pythonbased test harness that dispatches each validation entry to the appropriate category-specific validator and executes the corresponding test on the generated (or migrated) code. The harness outputs structured JSON summaries reporting the total number of evaluated cases, aggregated success metrics, and the identifiers of failed instances for each model.
16
Additionally, token usage was logged for every inference call, enabling cost-efficiency analysis. For hosted models, API usage statistics were aggregated automatically, while local runs collected system-level performance metrics such as GPU memory and throughput. These auxiliary indicators helped evaluate the scalability of the proposed workflow for larger datasets and industrial applications.
Fig. 6: LLM Evaluation Phase. The prepared LLMs are evaluated on the validation dataset using both static similarity metrics and automated functional tests. Their results are then compared through a multicriteria analysis, leading to the selection of the bestperforming LLM for the final real-world validation stage.
Metrics. As already introduced, two complementary evaluation dimensions were considered. The first was static similarity, assessing how closely the generated code matched the expected structure. This metric was computed using Python’s SequenceMatcher, producing a normalized lexical-structural similarity score between 0 and 1. It provided an initial approximation of whether the model preserved code structure, naming conventions, and syntax across transformations. However, similarity alone was insufficient to judge correctness, as superficial lexical overlap can mask deeper semantic inconsistencies. To address this, a second dimension, dynamic functional correctness, was incorporated using the test oracle present in each dataset entry. Each generated script by the LLM was executed in an isolated environment, where the tests validated its functional behaviour. Depending on the cryptographic category, these tests checked whether the outputs of encryption/decryption, sign/verify, encapsulate/decapsulate, or MAC generation/verification cycles were consistent with the expected reference values. This layer of testing captured runtime errors, incorrect parameter handling, or key mismanagement that might pass unnoticed in purely static analysis. The two metrics were combined to produce a comprehensive performance profile. Static similarity served as a proxy for structural fidelity, while dynamic verification measured true functional equivalence. The dual metric enabled fine-grained comparison between models and configurations, revealing whether apparent textual precision translated into real cryptographic correctness.
Post-processing of hardcoded key sizes before validation. Error analysis was performed manually to categorize the types of failures observed during dynamic testing. Common issues included incorrect key sizes, uninitialized variables, and misinterpreted data types (e.g., converting byte objects to integers). A particularly recurrent problem involved character-counting errors in hardcoded keys. Consistent with prior findings in the literature [10], LLMs frequently miscount characters when asked to generate fixed-length cryptographic keys (for example, 32 bytes), or confuse bytes with textual characters. To mitigate these failures, a lightweight post-processing step was introduced to automatically correct the length of hardcoded keys before validation. This simple adjustment significantly reduced false negatives during dynamic testing and improved the functional validity of the generated migrations. 7 Results The evaluation produced a detailed view of how large language models behave when trained for cryptographic code migration. Across the configurations tested, the results demonstrate that fine-tuning on domain-specific data is not only beneficial but necessary to achieve reliable and verifiable transformations. Static similarity and dynamic functional correctness. Four models were evaluated: GPT-4.1 in a zeroshot configuration, and three fine-tuned variants: GPT-4.1-mini, GPT-3.5-turbo and CodeLlama-7B. The fine-tuning process for GPT-3.5-turbo followed the same methodology and dataset as GPT-4.1-mini, but delivered slightly inferior results at a significantly higher cost (Table 5). Therefore, to focus the discussion on the most representative trade-offs, the remainder of this section concentrates on the three most illustrative configurations: zero-shot (GPT-4.1), OpenAI-hosted fine-tuned (GPT-4.1-mini), and open-source locally fine-tuned (CodeLlama-7B). GPT-4.1. In the zero-shot configuration, generalpurpose models such as GPT-4.1 displayed strong syntactic fluency but poor semantic precision. As illustrated in Fig. 7a, the similarity scores are widely spread
17
Table 5: Cross-summary of the main metrics. Model
Mean sim.
GPT-4.1 (ZS) 0.3941 CodeLlama-7B (FT)* 0.6031 GPT-3.5-turbo (FT) 0.8669 GPT-4.1-mini (FT) 0.9072 *Values after corrective post-processing.
HIGH
Dyn. %
Cost [$]
Train tokens [M]
3 16 50 56
15.0 61.3 68.8 92.5
0.21 3.16 8.48 5.39
0.00 2.96 1.01 1.04
and concentrated far from perfect overlap with the reference solutions, indicating frequent deviations in the migrated code even when the surrounding structure is preserved. Dynamic validation (Fig. 8a) confirmed that only about 15 % of zero-shot GPT-4.1 outputs were functionally executable without manual intervention. Typical failures involved incorrect initialization of post-quantum primitives, invalid parameter sizes, or incomplete replacement of legacy calls. This aligns with prior observations that LLMs exhibit high linguistic coherence but limited domain awareness in cryptographic reasoning. GPT-4.1-mini. Fine-tuning led to a substantial improvement across all metrics. For the fine-tuned GPT-4.1-mini, the static distribution in Fig. 7c shifts clearly towards high similarity values, reflecting that most migrations remain very close to the reference solutions at the token and structural level. In dynamic tests (Fig. 8c), the model reached a 92.5 % success rate, meaning that the vast majority of migrated scripts not only execute successfully (are syntactically valid and runnable under the Python interpreter) but also pass the functional tests (encrypt/decrypt, sign/verify, encaps/decaps) without manual fixes. After evaluating the model performance with a different train split, the results were consistent with the original experiment: mean static similarity increased slightly from 0.907188 to 0.922453, and dynamic functional correctness increased from 74/80 to 78/80 successful tests. CodeLlama-7B. The other fine-tuned baseline shows an intermediate behaviour. As seen in Figs. 7b and 8b, CodeLlama-7B improves markedly over zeroshot performance but still lags behind GPT-4.1-mini in both static similarity and dynamic correctness. These trends are summarised numerically in Table 5 and visually in Fig. 7 and Fig. 8. Resource consumption. In terms of cost and efficiency, the fine-tuning process proved economically viable. As reported in Table 5, the entire experiment for the OpenAI-hosted models, including dataset upload, training and evaluation, required less than $15 in total
API usage. The open-source CodeLlama-7B training completed within six hours on a single NVIDIA A100 GPU using under 24 GB of memory, confirming the feasibility of reproducing the setup in academic or industrial environments with moderate resources. Selection of the best model. From a qualitative standpoint, the fine-tuned GPT-4.1-mini exhibits a strong ability to preserve structural and stylistic aspects of the original code, maintaining variable names, function signatures and comments intact while transforming only the cryptographic logic. This selectivity is critical for production settings, where unintended edits to business logic could introduce regressions. The model’s capacity to isolate and rewrite only the relevant cryptographic sections demonstrates that LLMs can achieve targeted transformations with high precision once properly specialized. The comparative results also reveal that model scale is less important than task adaptation. Despite having fewer parameters than the full GPT-4.1, the finetuned GPT-4.1-mini outperforms it in all functional metrics (Table 5). This finding supports the hypothesis that domain-specific fine-tuning yields greater returns than simply increasing model size, particularly when the training data encapsulates consistent patterns of code-to-code transformation. Overall, the results confirm that cryptographic migration can be automated effectively through instruction-tuned LLMs. Fine-tuned models outperform zero-shot configurations by a wide margin, achieving near-complete correctness in controlled validation. The next section extends this analysis to real-world codebases, testing whether these findings hold under practical conditions involving complex dependencies and unstructured software. As an additional post-hoc check, we also evaluated a more recent zero-shot model, GPT-5.5, using the same validation split and automated evaluation harness. This experiment was not part of the original model-selection protocol, and is therefore reported only as an exploratory observation. Although GPT5.5 improved over the GPT-4.1 zero-shot baseline, its results remained far below those obtained by the
18
(a) GPT-4.1 (zero-shot)
(b) CodeLlama-7B (fine-tuned)
(c) GPT-4.1-mini (fine-tuned)
Fig. 7: Static similarity distributions for pre- to post-quantum code migration. The horizontal axis represents similarity score ranges, while the vertical axis reports the number of code variations falling within each range, reflecting the lexical and structural alignment between original and migrated implementations.
(a) GPT-4.1 (zero-shot)
(b) CodeLlama-7B (fine-tuned)
(c) GPT-4.1-mini (fine-tuned)
Fig. 8: Dynamic functional correctness across model configurations. Each subfigure shows the distribution of successful and failed executions in the dynamic validation tests.
Fig. 9: Similarity distribution with predominant category by range (GPT-4.1 mini).
fine-tuned models, especially GPT-4.1-mini. This observation reinforces the main conclusion of the study: for specialized cryptographic migration tasks, newer general-purpose models may improve zero-shot performance, but domain-specific oriented models remains necessary to achieve reliable functional correctness.
Fig. 10: Dynamic results by base case for GPT-4.1-mini (fine-tuned).
Insights from error analysis. Error analysis provides additional insight into the residual failure modes of the fine-tuned GPT-4.1-mini. Fig. 9 breaks down similarity ranges by predominant cryptographic category, making it possible to inspect which families tend to be harder to migrate precisely. Likewise, Fig. 10 presents
19
dynamic results by base case, highlighting that the remaining functional failures are concentrated mainly in structurally complex cases, especially key exchange and combined primitive migrations. Manual inspection of the failing validation cases shows that most residual errors are local softwareengineering issues rather than failures to identify the intended cryptographic migration. For instance, variation 6.52, a combined symmetric-cipher and digital-signature case, failed during the post-quantum phase because the generated code referenced signing key material through local variables such as priv and pub, without exposing the namespace bindings expected by the validation harness. In variation 6.57, a combined symmetric-cipher and key-exchange case, the generated code misused the KEM API by passing an invalid public-key buffer to oqs.encap secret, producing a ValueError: byte string too long. Additional failures included function-signature mismatches in key-exchange code and assertion-level regressions in MAC self-tests. These examples, illustrated in Appendix B, suggest that the main residual weaknesses of the fine-tuned model involve API contracts, namespace management, function signatures, and key-material representation. Importantly, such errors are typically localized and detectable through lightweight validation rules. This suggests that rule-based sanity checks or post-generation repair steps could further increase robustness without changing the underlying fine-tuned model.
8 Application to real-world cases To evaluate external validity and practical feasibility, the fine-tuned GPT-4.1-mini model was applied to six real-world open-source repositories containing non-trivial cryptographic logic. Unlike the synthetic dataset, these projects include heterogeneous code styles, undocumented legacy constructs, and dependencies typical of production environments. The objective was threefold: (i) determine whether the model can identify vulnerable or deprecated primitives, (ii) generate correct post-quantum replacements, and (iii) preserve functional behaviour without requiring task-specific prompts or manual guidance. This stage completes the full evaluation pipeline, transitioning from controlled synthetic testing to practical assessment in operational codebases. The workflow for this phase is summarised in Fig. 11, which illustrates how the top-performing model advances into real-world validation.
Fig. 11: Real-world Validation Phase. The bestperforming LLM selected in the previous phase is applied to a set of selected open-source repositories. Repository-specific tests are then used to assess practical migration behavior under realistic dependency and modularity constraints, producing a structured summary of real-world migration outcomes.
Real-world repository selection. Beyond controlled evaluation on a synthetic corpus, a complementary validation phase was designed to assess the behaviour of the proposed system under realistic softwareengineering conditions. The motivation for this stage is twofold. First, real-world codebases exhibit characteristics that are absent from curated datasets, such as heterogeneous coding styles, undocumented assumptions, implicit dependencies, and legacy abstractions. Second, practical post-quantum migration efforts in industry will inevitably operate over such unstructured environments rather than over idealised examples. The repositories selected for this phase were therefore chosen to strike a balance between realism and experimental tractability. Only open-source projects with demonstrable relevance and sustained community adoption were considered, ensuring that the evaluated code reflects patterns commonly found in production systems. At the same time, the scope was restricted to projects with a manageable code footprint, so that individual cryptographic files could be processed within the effective context window of current large language models without requiring extensive manual decomposition. A further selection criterion was the explicit use of cryptographic primitives covered during the construction of the synthetic dataset. Concretely, all selected repositories rely on primitives implemented through the pyca/cryptography library and include at least one of the following categories: digital signatures, key exchange, symmetric encryption, hashing, message authentication codes (MAC), or authenticated encryption. This alignment ensures that the real-world
20
evaluation tests generalisation rather than extrapolation beyond the model’s training domain, allowing performance differences to be attributed to structural complexity rather than to unseen cryptographic concepts. For each repository, a specific source file containing pre-quantum cryptographic logic was identified and used as the direct input to the fine-tuned GPT-4.1-mini model. Unlike the synthetic dataset, these files are not pre-structured, normalised, or automatically categorised. As a result, the cryptographic category relevant to each fragment was manually identified through code inspection and primitive analysis. While this manual labelling step limits scalability, it guarantees correctness during evaluation and mirrors the current state of enterprise migration workflows, where cryptographic inventory is often performed explicitly before remediation. The evaluation of real-world migrations focuses on three complementary dimensions. First, local precision assesses whether the model modifies only the cryptographic components that require migration, preserving surrounding business logic and control flow. Second, cryptographic correctness evaluates whether deprecated or quantum-vulnerable primitives are replaced by appropriate post-quantum or quantum-resistant alternatives consistent with current guidance. Finally, functional compatibility measures whether the migrated code remains syntactically valid, can be loaded by the Python interpreter, and executes correctly under repository-specific smoke tests, understood here as lightweight functional checks that exercise the relevant cryptographic execution paths without requiring the full original test suite. These tests allow the evaluation to detect runtime regressions while keeping the real-world validation phase tractable. Taken together, these selection and evaluation criteria ensure that the repositories provide a meaningful and demanding test bed for LLM-assisted cryptographic migration. The final selection, summarized in Table 6, includes open-source projects with active adoption, manageable cryptographic files, use of primitives represented in the synthetic dataset, and relevance to realistic software-engineering scenarios. Methodology applied. The process follows a uniform and reproducible pipeline to apply the trained model to real repositories and to validate the resulting migrations under realistic constraints. Concretely: 1. Load the original source file: for each repository, the specific file containing cryptographic logic to be migrated was identified and loaded from its original location in the public repository.
2. Split into fragments (tokenisation): since GPT-4.1-mini supports an effective context window of ∼8000 tokens (accounting for prompt overhead), each file was automatically split into manageable fragments of approximately 7800 usable tokens using the tiktoken encoder. 3. Prompt construction with explicit category: for every fragment, a concise prompt was generated following the same structure used during training/validation, explicitly stating the cryptographic category relevant to that fragment (e.g., signatures, AEAD, MAC, hashing). Category assignment was performed manually via code inspection and primitive identification, ensuring accurate labelling but limiting scalability. As future work, this step could be automated using syntactic/static analysis or a dedicated classifier so the full pipeline can operate autonomously on large codebases. 4. Automatic migration via the fine-tuned model: each prompted fragment was sent to the fine-tuned GPT-4.1-mini model to produce a postquantum-oriented refactoring. Model outputs were stored as-is (without additional commentary) to facilitate deterministic reconstruction. 5. Reconstruction of migrated code: migrated fragments were concatenated and reassembled into a single executable module representing the post-quantum version of the original file. 6. Manual and automatic evaluation: the resulting code was reviewed via smoke tests, automated diff against the original (to surface structural regressions), and a targeted manual inspection to identify and document type-level issues, incorrect encodings, or runtime incompatibilities introduced by the migration. This methodology standardises real-world evaluation, making outcomes comparable across repositories and enabling systematic identification of recurrent failure modes. Per-repository results and observed failure modes. Across all projects, the model detected occurrences of pre-quantum cryptography and proposed PQCcompatible refactorings that generally matched the structure and intent of the original code. The following repository-level results highlight both strengths and limits: Netflix Lemur. The model correctly replaced RSA and ECDSA constructs, modernising certificatehandling functions. A recurrent error involved invalid key lengths when switching to ChaCha20Poly1305, which was resolved manually. Aside from this, the
21
Table 6: Real-world repositories used for external validation Project
Category
LOC
Primitives identified
Netflix Lemur
Certificate management
130
RSA, ECDSA, SHA-1/256
python-jose
JWT signing and encryption
586
RSA, ECDSA, HMAC, AES-GCM
Prefect
Secret encryption in workflows
49
AES-128-CBC, HMAC-SHA256
SnapPass
One-time password sharing
362
Fernet (AES-CBC + HMAC)
google-auth
Token signing and verification
151
RSA-PSS, SHA-256/512
Alexa SDK
HTTP signature verification
511
RSA-SHA256, X.509
migrated file executed successfully, demonstrating high accuracy on PKI-related transformations. python-jose. As one of the most complex projects, python-jose exposed the model’s limits. Multiple primitives and interdependent APIs caused structural inconsistencies in the output, leading to non-executable code. The migration produced valid local substitutions but failed to preserve global consistency, indicating that multi-layered dependencies remain challenging for current LLMs. Prefect. This repository provided a small, wellcontained symmetric module (CBC + HMAC). The model substituted AES-CBC and HMAC-SHA256 for ChaCha20Poly1305, but incorrectly assumed hexadecimal encoding for environment keys. After a minimal manual correction, the program executed normally, suggesting that fine-tuned LLMs can handle compact cryptographic components. SnapPass. This project, relying on Fernet, provided a clean AEAD-like scenario. The model produced a syntactically and functionally valid migration, with only one minor error—attempting to decode a binary key as UTF-8. Correcting this line yielded a fully functional post-quantum version. This case confirmed that the system can produce production-ready migrations in simple contexts. google-auth. The migration preserved the overall structure but failed during runtime due to incorrect key serialisation. The model occasionally represented keys as integers rather than byte objects, causing exceptions in signature generation. These issues highlight the importance of explicit type handling in future training data. Alexa SDK. Here, the only cryptographic operation was hashing for signature verification. The model replaced instances of SHA256 with SHA3 512
while maintaining documentation and logic unchanged. The resulting file was syntactically correct and passed all smoke tests without modification, exemplifying a perfect migration. Overall, these experiments revealed consistent trends: the model performed best in localised, selfcontained modules (e.g., SnapPass, Prefect, Alexa SDK) and required minimal human supervision to achieve functional results. Larger frameworks with intertwined cryptographic layers (e.g., python-jose) exposed limitations in global reasoning and dependency tracking. Execution-level validation and operational considerations. Beyond correctness, the real-world evaluation assessed operational efficiency and the practical role of the model within a realistic migration workflow. Unlike synthetic benchmarks, production repositories require validation strategies robust to incomplete documentation, external dependencies, and legacy abstractions. Evaluation was performed by isolating representative execution paths for each project. For every repository, a small set of application-level functions exercising the cryptographic logic (e.g., token generation and verification, encryption–decryption cycles, signature validation) was identified and executed on the original pre-quantum code. After migration, the same functions were re-executed on the transformed code and their outputs compared. Any deviation in behaviour, exception handling, or output structure was flagged as a functional regression. This approach evaluates correctness independently of surrounding application logic while remaining faithful to real usage patterns. From an efficiency standpoint, the migration required no repository-level prompt engineering or
22
Table 7: Summary of real-world migration outcomes Project
Main category
Result
Issues observed
Lemur
PKI / certificate management
Partial success
Incorrect key size; missing exception handling
python-jose
Mixed (sign, MAC, AEAD)
Failure
Structural inconsistency; dependency conflicts
Prefect
Authenticated cryption
en-
Partial success
Misinterpreted key encoding
SnapPass
Authenticated cryption
en-
Full success
Minor decoding error
google-auth
Digital signatures
Failure
Type mismatch for key objects
Alexa SDK
Hashing
Full success
None detected
task-specific tuning. Each fragment was processed using the same inference configuration and prompting strategy validated during synthetic testing, confirming that the fine-tuned model generalises without additional overhead. The average inference time per fragment remained within the same order of magnitude as the validation phase, and no post-processing beyond lightweight key/type fixes (as described above) was required. This suggests the approach can scale to real codebases without prohibitive computational or operational cost. Integrating LLM-assisted code migration into cryptographic transition frameworks. A key outcome of this work is the confirmation that LLM-assisted code migration aligns closely with enterprise-level cryptographic transition frameworks, particularly in bridging the gap between policy-level decisions and code-level remediation. Enterprise initiatives such as ELCA [40] and the framework proposed by Hasan et al. [12] emphasize the need for structured transition pipelines involving inventory, dependency mapping, algorithm selection, and eventual migration. Likewise, industry-led programmes—including IBM’s Cryptography Bill of Materials (CBOM) [36] and CISA’s Automated Cryptographic Discovery and Inventory (ACDI) [5]—focus on detecting and classifying vulnerable cryptographic components in codebases. However, these efforts stop short of automated code transformation. This project contributes directly at the “Remediation” phase of such pipelines. After discovery tools identify what needs to change, our LLM-based system answers how to change it. In IBM’s CBOM workflow, this corresponds to the final “Remediate” stage, which involves adapting software to conform to updated cryptographic standards—something that today often relies on manual rewriting or ad hoc scripting. The
experiments presented here show that a fine-tuned model can act as an automated cryptographic refactoring agent, capable of detecting deprecated primitives (e.g., RSA, ECDSA, AES-128), proposing compliant post-quantum replacements (e.g., Dilithium, Kyber, AES-256), and preserving functional behaviour across real-world projects. This practical capability addresses the needs articulated by ENISA [4] and NIST [3] for crypto-agile systems, defined as systems that can evolve cryptographically without overhauling the entire software stack. Rather than merely reporting issues, the system operationalises crypto-agility by carrying out the code modifications required by policy. In doing so, it fills a long-standing gap between governance and engineering. This alignment supports contribution C4, showing that fine-tuned LLMs can serve as concrete migration engines within broader organizational PQC strategies, making cryptographic modernization more scalable, less error-prone, and accessible to security engineering teams. Fig. 12 illustrates the position of this work within a canonical enterprise cryptographic migration workflow.
9 Discussion and limitations This work is intentionally scoped to enable verifiable, reproducible experimentation within realistic constraints. While this enables meaningful results in current software environments, it also imposes clear boundaries on generalisability. The most relevant design choices and their implications are discussed below. 1. Language and library scope. The decision to focus on Python and the PyCA/Cryptography library ensures a stable and widely supported substrate. Python’s dominance in academic and produc-
23
Fig. 12: Position of the proposed LLM-assisted migration system within a standardized enterprise cryptographic transition pipeline.
tion settings enhances reproducibility, while PyCA offers a well-structured, security-focused API that minimizes ambiguity in usage. These factors enable consistent functional testing and streamline the migration process. 2. Primitive-level coverage. The cryptographic primitives targeted are those explicitly affected by quantum threats. Classical asymmetric mechanisms (e.g., RSA, ECDSA, DH) are migrated to postquantum counterparts (e.g., Dilithium, Kyber), while symmetric and hash functions are updated via parameter strengthening (e.g., AES key size increases, SHA-2 to SHA-3). This aligns with NIST and ENISA guidance for hybrid transitions. It is also worth exploring migration to other post-quantum primitives that have been standardized in a second batch – such as HQC — or are currently under exploration – as it is the case with the signature schemes being evaluated towards standardization by NIST, see [2]. 3. Emphasis on authenticated encryption. Although historically overlooked in foundational taxonomies, AEAD constructions are critical in modern protocols such as TLS 1.3 and QUIC. This work treats AEAD as a first-class component, preserving its usage while replacing the key-exchange layer with KEM constructions. This mirrors real-world post-quantum deployment strategies, including NIST’s TLS migration trials. 4. Protocol and abstraction boundaries. The system does not aim to migrate full communication protocols (e.g., TLS, SSH) or higher-level constructs like handshake flows or network abstractions. It also excludes cryptographic constructions that are closely related to the primary schemes under analysis—such as threshold signatures or key exchange protocols derived from KEMs—as
well as more advanced cryptographic primitives (e.g., zero-knowledge proofs, multiparty computation protocols, and commitment schemes), whose correctness remains beyond the current capabilities of LLMs. 5. Library and ecosystem dependency. The migrations rely on existing PQC implementations such as liboqs-python. While this reflects current industry best practices, it means that the approach inherits the constraints and assumptions of these tools. As standards evolve and new implementations are adopted, the models may require retraining to reflect updated APIs and security profiles. 6. Contextual and cross-module limitations. The system performs reliably on isolated or wellencapsulated cryptographic functions. However, in real-world scenarios involving distributed logic or interdependent abstractions, local transformations can break global consistency. These errors are difficult to detect via unit tests alone and suggest a need for integrating dependency analysis or symbolic reasoning in future work. In sum, the design constraints are not merely limiting—they are enabling. They allow the problem of post-quantum migration to be framed in a way that supports automatic transformation and functional verification with current LLM technology. While generalising to broader cryptographic domains remains open, this work provides a concrete and reproducible foundation upon which such efforts can build.
10 Conclusions and future work This study investigated whether large language models can assist in the migration of cryptographic code from pre-quantum to post-quantum standards in a verifiable and efficient manner. The initial research question posed in this work was: Can large language models be effectively trained and evaluated to assist in the migration of prequantum cryptographic code to post-quantum counterparts while preserving functional correctness? The results of this work provide a clear and affirmative answer. Through a combination of a carefully curated dataset of 800 paired examples and a rigorous fine-tuning and validation pipeline, the evidence shows that LLMs can indeed perform functionally correct and structurally consistent cryptographic migrations across a wide variety of primitives. As illustrated in Fig. 13, the proposed methodology substantially streamlines the
24
Fig. 13: GPT-4.1-Mini–assisted post-quantum migration, fine-tuned, reduces process complexity and serves as an effective alternative to the manual approach.
migration workflow, reducing manual intervention and transforming what was previously a multi-stage, errorprone process into a guided, verifiable pipeline powered by a fine-tuned LLM. The proposed system integrated two core components: (i) a structured dataset covering six major cryptographic families and combined primitive cases, and (ii) a fine-tuned model optimized for cryptographic transformations while preserving functional semantics. This combination enabled controlled experimentation across synthetic benchmarks and real open-source repositories, offering a comprehensive view of LLM-assisted modernization. From the experimental results, several conclusions emerge. First, domain-specific fine-tuning is essential. Zero-shot models, despite their fluency, failed to achieve reliable cryptographic correctness. After specialization, the fine-tuned GPT-4.1-mini achieved an average similarity of 0.91 and passed 92.5% of dynamic tests, outperforming larger general-purpose models and the CodeLlama-7B baseline. Second, the system shows remarkable costefficiency: the best-performing GPT-4.1-mini configuration required less than $6 in API usage for training and evaluation, making this approach accessible for academic research and enterprise environments. Third, the fine-tuned model preserved structural fidelity: edits remained confined to cryptographic sections, and surrounding logic, comments and control flow remained intact. This selective rewriting is critical for real-world engineering use cases, where unintended modifications may introduce regressions or security flaws. Applying the model to real-world repositories further validated its external applicability. In four of the six evaluated projects—such as SnapPass, Prefect, and
the Alexa SDK —the migrated code was functional or nearly functional with minimal manual intervention. More complex projects, including google-auth and python-jose, exposed current limitations, especially when dealing with intertwined abstractions, implicit dependencies or multi-layered cryptographic flows. These observations highlight that LLMs excel at local migrations but still struggle with architecture-wide reasoning, reinforcing the need for symbolic validation and dependency-aware pipelines. Taken together, the empirical findings demonstrate that the answer to the research question is yes: large language models can be effectively trained and evaluated to support the migration of pre-quantum cryptographic code to post-quantum counterparts while maintaining functional correctness. This is feasible at low cost, with reproducible validation, and with a level of precision that makes LLM-assisted cryptographic modernization a realistic tool for engineering teams. The experiments confirm that such models do not merely rewrite code superficially—they perform semantically meaningful transformations that pass runtime verification. Looking forward, several research directions emerge: (i) Automatic fragment classification. Integrating a classifier to detect cryptographic categories would eliminate manual tagging and enable autonomous fullrepository scanning. (ii) Dataset expansion. Incorporating curated examples from industrial systems would improve robustness against complex dependency graphs and multi-file migrations. (iii) Post-processing enhancements. Embedding sanity checks for key sizes, imports, encodings and nonce management would further reduce structural errors. (iv) Multi-agent orchestration. Coordinating different LLMs for generation, verification and documentation could improve reliability and explainability. (v) Automated dynamic testing. Generating and running validation scripts automatically from model outputs would create a closedloop verification cycle. (vi) Language diversification. Extending the methodology to languages such as Go, C or Java would test its scalability beyond Python and enable broader adoption. (vii) Migration of protected data at rest. While this work focuses on source-code migration, a complete organizational PQC transition must also address data already encrypted or authenticated with legacy primitives. Future work should therefore explore how code-level migration can be coordinated with data re-encryption, re-authentication, key rotation, and backward-compatible access strategies for long-lived stored data. In summary, this work establishes a reproducible, data-driven framework for post-quantum cryptographic
25
migration using large language models. A compact, fine-tuned model such as GPT-4.1-mini can perform accurate, cost-effective migrations at scale. This confirms that LLMs can serve as practical assistants for transitioning legacy infrastructures toward quantumsafe standards, marking a tangible step toward secure and future-proof cryptographic ecosystems. The findings lay the groundwork for hybrid pipelines where AI collaborates with engineers to produce reliable, verifiable and crypto-agile software systems.
Declarations Funding This work was supported by a Collaboration Grant in University Departments for the 2025–2026 academic year, awarded by the Spanish Ministry of Education, Vocational Training and Sports. The work of A.I.G.-T. is supported by projects PID2022140126OB-I00 and PID2023-150310OB-I00, and the work of M.I.G.V. by project PID2023-151238OA-I00, all the projects funded by MCIU/AEI/FEDER UE, Spain. In addition, the work of J.P.B. and A.I.G.-T. was supported by the EMACS cybersecurity research network (Project RED2024-123456-T, funded by MICIU/AEI/10.13039/501100011033).
Competing interests The authors declare that they have no competing interests.
Data availability The dataset generated and analysed during the current study is publicly available in the e-cienciaDatos repository under the title Cryptographic Migration Dataset: Pre-Quantum to Post-Quantum, version V1, with DOI 10.21950/7GK4MJ [33].
Code availability The code used to generate, fine-tune, evaluate, and analyse the models is not publicly available at this stage due to ongoing development and repository cleanup. The core scripts required to reproduce the evaluation results can be made available from the corresponding author upon reasonable request. A public release of the evaluation harness and reproducibility scripts is planned for a future version of the artifact.
Author contributions Javier Pallarés de Bonrostro led the technical development of the work, including dataset construction, software implementation, model fine-tuning, validation, experimental analysis, visualization, and writing of the original draft. Ana Isabel González-Tablas originated the research idea, contributed to the conceptualization and methodological design of the study, supervised the work on a day-to-day basis, contributed to funding acquisition and project administration, and participated in manuscript review and editing. Marı́a Isabel González Vasco contributed expert knowledge in postquantum cryptography, provided relevant conceptual and methodological input, supported the cryptographic analysis and validation of the migration rationale, contributed to funding acquisition, and participated in manuscript review and editing.
Ethics approval Not applicable. This study did not involve human participants, human data, or animals.
Consent to participate Not applicable.
Consent for publication Not applicable.
Use of generative AI and AI-assisted technologies Generative AI tools were used as part of the research object of this study, namely for the generation and evaluation of cryptographic migration examples as described in the methodology. All scientific claims, code, experimental results, citations, and final text were reviewed and validated by the authors, who remain fully responsible for the content of the manuscript.
26
References [1]
[2]
[3]
[4]
[5]
[6]
[7]
[8]
[9]
Nadeem Ahmed, Lei Zhang, and Aryya Gangopadhyay. “A survey of post-quantum cryptography support in cryptographic libraries”. In: arXiv preprint arXiv:2508.16078 (2025). Gorjan Alagic et al. Status Report on the First Round of the Additional Digital Signature Schemes for the NIST Post-Quantum Cryptography Standardization Process. NIST Interagency/Internal Report (NISTIR) 8528. Gaithersburg, MD, USA: National Institute of Standards and Technology, Oct. 2024. doi: 10 . 6028 / NIST . IR . 8528. url: https : / / doi . org/10.6028/NIST.IR.8528. Elaine Barker et al. Considerations for Achieving Cryptographic Agility: Strategies and Practices. NIST Cybersecurity White Paper NIST CSWP 39. National Institute of Standards and Technology, Dec. 2025. doi: 10 . 6028 / NIST . CSWP . 39. url: https://doi.org/10.6028/NIST.CSWP. 39. Ward Beullens et al. Post-Quantum Cryptography: Current State and Quantum Mitigation (v2). Tech. rep. European Union Agency for Cybersecurity (ENISA), 2021. doi: 10 . 2824 / 92307. url: https : / / www . enisa . europa . eu / sites / default / files / publications / ENISA % 20Report % 20 - %20Post - Quantum % 20Cryptography % 20Current % 20state % 20and % 20quantum%20mitigation-V2.pdf. Cybersecurity and Infrastructure Security Agency (CISA). Strategy for Migrating to Automated Post-Quantum Cryptography Discovery and Inventory Tools. Tech. rep. Accedido el 23 de mayo de 2025. U.S. Department of Homeland Security, Sept. 2024. url: https : //www.cisa.gov/sites/default/files/202409/Strategy- for- Migrating- to- AutomatedPQC-Discovery-and-Inventory-Tools.pdf. Tim Dettmers et al. “Qlora: Efficient finetuning of quantized llms”. In: Advances in neural information processing systems 36 (2023), pp. 10088– 10115. Iria de-Dios-Flores et al. “Nos CorpusNOS-GL: Galician Macrocorpus for LLM training”. In: Nos CorpusNOS-GL: Galician Macrocorpus for LLM training (2024). Morris J Dworkin et al. “SHA-3 standard: Permutation-based hash and extendable-output functions”. In: (2015). European Commission and NIS Cooperation Group. Coordinated Implementation Roadmap
[10]
[11]
[12]
[13]
[14]
[15]
[16]
[17]
[18]
[19]
[20]
for the Transition to Post-Quantum Cryptography. https : / / digital - strategy . ec . europa . eu / en / library / coordinated implementation- roadmap- transition- postquantum-cryptography. European Union policy roadmap on coordinated migration to postquantum cryptography. June 2025. Tairan Fu et al. “Why Do Large Language Models (LLMs) Struggle to Count Letters?” In: arXiv preprint arXiv:2412.18626 (2024). Ayaka Harigai et al. “Response accuracy of GPT4 across languages: insights from an expert-level diagnostic radiology examination in Japan”. In: Japanese Journal of Radiology 43.2 (2025), pp. 319–329. Khondokar Fida Hasan et al. “A framework for migrating to post-quantum cryptography: Security dependency analysis and case studies”. In: IEEE Access 12 (2024), pp. 23427–23450. Juyong Jiang et al. “A Survey on Large Language Models for Code Generation”. In: ACM Transactions on Software Engineering and Methodology 35.2 (Jan. 2026), pp. 1–72. issn: 1557-7392. doi: 10.1145/3747588. url: http://dx.doi.org/ 10.1145/3747588. Marc Kaplan et al. “Breaking symmetric cryptosystems using quantum period finding”. In: Advances in Cryptology–CRYPTO 2016: 36th Annual International Cryptology Conference, Santa Barbara, CA, USA, August 14-18, 2016, Proceedings, Part II 36. Springer. 2016, pp. 207–237. Marie-Anne Lachaux et al. “Unsupervised translation of programming languages”. In: arXiv preprint arXiv:2006.03511 (2020). Nathalie Lang and Stefan Lucks. “On the postquantum security of classical authenticated encryption schemes”. In: International Conference on Cryptology in Africa. Springer. 2023, pp. 79– 104. Zhihao Li et al. “CryptoScope: Utilizing Large Language Models for Automated Cryptographic Logic Vulnerability Detection”. In: arXiv preprint arXiv:2508.11599 (2025). Zihao Li et al. “Quantifying multilingual performance of large language models across languages”. In: arXiv e-prints (2024), arXiv–2404. Zijie Lin et al. “AutoP2C: An LLM-Based Agent Framework for Code Repository Generation from Multimodal Content in Academic Papers”. In: arXiv preprint arXiv:2504.20115 (2025). Yiheng Liu et al. “Understanding llms: A comprehensive overview from training to inference”. In: Neurocomputing (2024), p. 129190.
27
[21]
[22]
[23]
[24]
[25]
[26]
[27]
[28]
[29]
Aman Madaan et al. “Self-refine: Iterative refinement with self-feedback”. In: Advances in Neural Information Processing Systems 36 (2023), pp. 46534–46594. Tarek Mahmud et al. “Automated Update of Android Deprecated API Usages with Large Language Models”. In: arXiv preprint arXiv:2411.04387 (2024). Utsav Maskey, Chencheng Zhu, and Usman Naseem. “Benchmarking large language models for cryptanalysis and mismatched-generalization”. In: arXiv preprint arXiv:2505.24621 (2025). Zohaib Masood and Miguel Vargas Martin. “Beyond Static Tools: Evaluating Large Language Models for Cryptographic Misuse Detection”. In: arXiv preprint arXiv:2411.09772 (2024). National Institute of Standards and Technology. FIPS 203: Module-Lattice-Based KeyEncapsulation Mechanism Standard. Federal Information Processing Standards Publication 203. National Institute of Standards and Technology, Aug. 2024. doi: 10.6028/NIST.FIPS.203. url: https://doi.org/10.6028/NIST.FIPS.203. National Institute of Standards and Technology. FIPS 204: Module-Lattice-Based Digital Signature Standard. Federal Information Processing Standards Publication 204. National Institute of Standards and Technology, Aug. 2024. doi: 10 . 6028 / NIST . FIPS . 204. url: https://doi.org/10.6028/NIST.FIPS.204. National Institute of Standards and Technology. FIPS 205: Stateless Hash-Based Digital Signature Standard. Federal Information Processing Standards Publication 205. National Institute of Standards and Technology, Aug. 2024. doi: 10.6028/ NIST . FIPS . 205. url: https : / / doi . org / 10 . 6028/NIST.FIPS.205. National Institute of Standards and Technology. NIST Announces First Four Quantum-Resistant Cryptographic Algorithms. https://www.nist. gov / news - events / news / 2022 / 07 / nist announces- first- four- quantum- resistantcryptographic- algorithms. Accedido el 30 de mayo de 2025. July 2022. National Institute of Standards and Technology (NIST). Migration to Post-Quantum Cryptography: Mappings to Risk Framework. Tech. rep. Draft White Paper. Available at https : / / www . nist . gov / news - events / news / 2025 / 09/new-draft-white-paper-pqc-migrationmappings - risk - framework - docs. National Cybersecurity Center of Excellence (NCCoE), Sept. 2025.
[30]
[31]
[32]
[33]
[34]
[35]
[36]
[37]
[38]
[39]
[40]
[41]
[42]
OpenAI. GPT-4.1 Overview. Accedido el 22 de mayo de 2025. 2025. url: https://platform. openai.com/docs/models/gpt-4.1. OpenAI. Introducing GPT-4.1 in the API. Accedido el 22 de mayo de 2025. Apr. 2025. url: https://openai.com/index/gpt-4-1/. OpenAI. Pricing - OpenAI API. Accedido el 22 de mayo de 2025. 2025. url: https://platform. openai.com/docs/pricing/. Javier Pallarés de Bonrostro and Ana Isabel González-Tablas. Cryptographic Migration Dataset: Pre-Quantum to Post-Quantum. Version V1. 2025. doi: 10 . 21950 / 7GK4MJ. url: https://doi.org/10.21950/7GK4MJ. Post-Quantum Cryptography Coalition (PQCC). PQC Migration Roadmap. Tech. rep. Available at https : / / pqcc . org / wp - content / uploads / 2025 / 05 / PQC - Migration - Roadmap - PQCC - 2 . pdf. PQC Coalition, May 2025. Robert Praas. Self-Reflection on Chain-ofThought Reasoning in Large Language Models. 2023. IBM Research. Cryptography Bill of Materials (CBOM). https://research.ibm.com/blog/ crypto-bill-of-materials. Accessed January 2026. 2024. Nino Ricchizzi, Christian Schwinne, and Jan Pelzl. “Applied Post Quantum Cryptography: A Practical Approach for Generating Certificates in Industrial Environments”. In: arXiv preprint arXiv:2505.04333 (2025). Baptiste Roziere et al. “Code llama: Open foundation models for code”. In: arXiv preprint arXiv:2308.12950 (2023). Atsushi Shirafuji et al. “Refactoring programs using large language models with few-shot examples”. In: 2023 30th Asia-Pacific Software Engineering Conference (APSEC). IEEE. 2023, pp. 151–160. Dimitrios Sikeridis et al. “ELCA: Introducing enterprise-level cryptographic agility for a postquantum era”. In: Cryptology ePrint Archive (2023). Joseph Strauss et al. “Assessing and Enhancing Quantum Readiness in Mobile Apps”. In: arXiv preprint arXiv:2506.00790 (2025). Available at https://arxiv.org/abs/2506.00790. Seyed Mohammad Taghavi Far and Farid Feyzi. “Large language models for software vulnerability detection: a guide for researchers on models, methods, techniques, datasets, and metrics”. In: International Journal of Information Security 24.2 (2025), p. 78.
28
[43]
[44]
[45]
[46]
[47]
[48]
[49]
[50]
Hugo Touvron et al. “Llama: Open and efficient foundation language models”. In: arXiv preprint arXiv:2302.13971 (2023). Meltem Sönmez Turan et al. Ascon-Based Lightweight Cryptography Standards for Constrained Devices: Authenticated Encryption, Hash, and Extendable Output Functions. Tech. rep. NIST SP 800-232 (Initial Public Draft). Initial Public Draft. National Institute of Standards and Technology, Oct. 2024. url: https : //csrc.nist.gov/pubs/sp/800/232/ipd. Ashish Vaswani et al. “Attention is all you need”. In: Advances in neural information processing systems 30 (2017). Xingjiao Wu et al. “A survey of human-in-theloop for machine learning”. In: Future Generation Computer Systems 135 (2022), pp. 364–381. Jingfeng Yang et al. Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyond. 2023. arXiv: 2304.13712 [cs.CL]. url: https://arxiv.org/abs/2304.13712. Quanjun Zhang et al. A Survey on Large Language Models for Software Engineering. 2024. arXiv: 2312 . 15223 [cs.SE]. url: https : //arxiv.org/abs/2312.15223. Bingzhe Zhou et al. “Hybrid API migration: A marriage of small API mapping models and large language models”. In: Proceedings of the 14th Asia-Pacific Symposium on Internetware. 2023, pp. 12–21. Celal Ziftci et al. “Migrating Code At Scale With LLMs At Google”. In: arXiv preprint arXiv:2504.09691 (2025).
29
Appendix A: Full dataset distribution
Table 8: Distribution of single cryptographic primitives across training and validation splits. Primitive / Alg.
#Var.
Train
Val.
Hash functions BLAKE2b BLAKE2s MD5 RIPEMD160 SHA-1 SHA-224 SHA-256 SHA-384 SHA-512 SHA-512/224 SHA-512/256
7 7 10 1 14 11 14 12 12 6 6
7 7 9 1 13 10 12 12 11 4 4
0 0 1 0 1 1 2 0 1 2 2
Symmetric encryption 3DES AES-128 DES
7 82 11
6 74 10
1 8 1
Message authentication codes (MAC) CMAC-AES-128 12 11 CMAC-AES-256 4 3 CMAC-3DES 2 2 GMAC-AES-128 5 5 GMAC-AES-256 3 3 HKDF-SHA-256 + 1 1 Poly1305 HKDF-SHA-256 + 1 1 HMAC-SHA-256 HMAC-BLAKE2b 2 2 HMAC-BLAKE2b-256 1 1 HMAC-BLAKE2s 3 3 HMAC-MD5 1 1 HMAC-SHA-1 11 9 HMAC-SHA-224 4 4 HMAC-SHA-256 37 32 HMAC-SHA-512 1 1 HMAC-SM3 2 2 PBKDF2 + HMAC2 1 SHA-1 Poly1305 7 7 (a) First part
1 1 0 0 0 0 0 0 0 0 0 2 0 5 0 0 1 0
Primitive / Alg.
#Var.
Train
Val.
Authenticated encryption (AEAD) AES-128-CCM 24 21 AES-128-GCM 37 35 AES-192-GCM 4 4 AES-192-CCM 1 0 AES-256-CCM 2 2 AES-256-GCM 1 0 AES-256-GCM-SIV 7 6 AES-256-SIV 15 14 AES-512-SIV 1 1 AES-OCB3 6 6 Fernet 2 1
3 2 0 1 0 1 1 1 0 0 1
Digital signatures DSA ECDSA-SECP256R1 ECDSA-SECP384R1 Ed25519 Ed448 RSA
17 24 4 18 11 26
14 22 3 17 9 25
3 2 1 1 2 1
Key exchange DH-2048 DH-3072 ECDH-P384 ECDH-SECP256R1 ECDH-SECP384R1 ECDH-SECP521R1 ECIES RSA-2048 X25519 X448
9 7 4 22 7 4 8 9 24 6
9 7 4 20 5 3 7 8 22 5
0 0 0 2 2 1 1 1 2 1
(b) Second part
30
Table 9: Distribution of combined constructions across training and validation splits. Construction
#Var.
AEAD + digital signature AES-GCM + ECDSAP-256 AES-GCM + RSA-2048 ChaCha20-Poly1305 + ECDSA-P-256 ChaCha20-Poly1305 + RSA-2048 AEAD + hashing AES-GCM + SHA-256 AES-GCM + SHA-512 ChaCha20-Poly1305 + SHA-256 ChaCha20-Poly1305 + SHA-512 AEAD + key exchange AES-GCM + DiffieHellman-3072 AES-GCM + X25519 ChaCha20-Poly1305 + Diffie-Hellman-3072 ChaCha20-Poly1305 + X25519 AEAD + MAC AES-GCM + HMACSHA-256 ChaCha20-Poly1305 + HMAC-SHA-256 Hashing + digital signature SHA-256 + ECDSA-P256 SHA-256 + RSA-2048 SHA-512 + ECDSA-P256 SHA-512 + RSA-2048 Hashing + key exchange SHA-256 + DiffieHellman-3072 SHA-256 + X25519 SHA-512 + DiffieHellman-3072 SHA-512 + X25519 Hashing + MAC SHA-256 + HMACSHA-256 SHA-512 + HMACSHA-256 (a) First part
Train
Val.
4
4
0
4 4
4 4
0 0
4
3
1
4 4 4
4 4 4
0 0 0
4
3
1
4
4
0
3 4
3 3
0 1
5
4
1
4
4
0
4
3
1
5
4
1
4 4
4 4
0 0
3
3
0
4
4
0
4 4
3 4
1 0
4
4
0
4
3
1
4
4
0
Construction
#Var.
MAC + digital signature HMAC-SHA-256 + ECDSA-P-256 HMAC-SHA-256 + RSA-2048 MAC + key exchange HMAC-SHA-256 + Diffie-Hellman-3072 HMAC-SHA-256 + X25519 Symmetric cipher + AEAD AES-128 + AES-GCM AES-128 + ChaCha20Poly1305 AES-256 + AES-GCM AES-256 + ChaCha20Poly1305
Train
Val.
4
4
0
4
3
1
4
3
1
4
4
0
4 4
3 4
1 0
4 4
4 3
0 1
Symmetric cipher + digital signature AES-128 + ECDSA-P4 4 256 AES-128 + RSA-2048 4 4 AES-256 + ECDSA-P4 4 256 AES-256 + RSA-2048 4 2 Symmetric cipher + hashing AES-128 + SHA-256 4 AES-128 + SHA-512 4 AES-256 + SHA-256 4 AES-256 + SHA-512 4
0 0 2
4 4 4 3
0 0 0 1
3
1
4 4
0 0
3
1
4
3
1
4
4
0
Symmetric cipher + key exchange AES-128 + Diffie4 Hellman-3072 AES-128 + X25519 4 AES-256 + Diffie4 Hellman-3072 AES-256 + X25519 4 Symmetric cipher + MAC AES-128 + HMACSHA-256 AES-256 + HMACSHA-256
0
(b) Second part
31
Appendix B: Representative GPT-4.1-mini failure examples Listing 5 shows two representative runtime failures observed in the outputs generated by the fine-tuned GPT-4.1-mini. Both examples correspond to raw model outputs evaluated through the dynamic validation harness. The first failure illustrates a namespacebinding problem in a combined symmetric-cipher and digital-signature migration. The second illustrates a KEM API misuse in a combined symmetric-cipher and key-exchange migration. Listing 5: Representative residual failures observed in fine-tuned GPT-4.1-mini outputs. # Variation 6.52: missing namespace bindings # Category : symmetric cipher + digital signature with oqs . Signature ( ALGO ) as s : s . secret_key = ctypes . c r e a t e _ s t r i n g _ b u f f e r ( priv , len ( priv ) ) sig = s . sign ( ct ) # Harness traceback : # RuntimeError : h a n d l e _ c i p h e r _ s i g n _ a u t o : # missing SYM_KEY / SYM , _PRIV / PRIV , or _PUB / PUB in namespace
# Variation 6.57: KEM API misuse # Category : symmetric cipher + key exchange with oqs . KeyEncapsul a t i o n ( ALGORITHM ) as kem : kem . secret_key = ct . c r e a t e _ s t r i n g _ b u f f e r ( priv , len ( priv ) ) _ , shared = kem . encap_secret ( peer_pub ) # Runtime traceback : # ValueError : byte string too long