LLM-Based Porting of Optimized C++ to CUDA Through Deoptimization and Reoptimization Daichi Mukunoki, Ryo Mikasa, Shunichiro Hayashi, Tetsuya Hoshino and Takahiro Katagiri
arXiv:2606.06063v1 [cs.DC] 4 Jun 2026
Nagoya University Furo-cho, Chikusa-ku, Nagoya 464-8601, Japan [email protected]
Abstract—When porting high-performance computing (HPC) code from CPU to GPU, CPU-oriented optimizations may obstruct LLM-based CUDA translation. We design and evaluate a Deopt-Reopt workflow that first simplifies the input C++ code and then retranslates and reoptimizes it for CUDA, comparing it against direct translation (Direct) on twelve HPC kernels with two LLMs (gpt-oss-120b (O120) and qwen-3-235b-a22b-instruct-2507 (Q235)) in Single-shot (one pass) and Iterative (repeated refinement) settings. In Singleshot, among 18 testable cases Deopt-Reopt was significantly faster among successful trials (after BH-FDR correction) in five — most clearly for conv2d, where CPU- and GPU-oriented designs diverge — but Direct was faster in three, so removing CPUspecific optimizations is not universally beneficial. An exploratory Direct-3 control that equalizes the LLM-call count left DeoptReopt ahead in only four of nineteen testable cases, with Direct3 ahead in four others. In Iterative, repeated generation and repair narrow the mode gap — markedly so for O120 — while Q235 retains large Deopt-Reopt advantages on conv2d, ddgemm, and bgemm. Deopt-Reopt’s effect on feasibility is also mixed — sharply higher for some kernels Direct rarely compiles, lower for others. Because performance is conditioned on successful trials, the benefit is conditional rather than a guaranteed end-toend gain. Overall, Deopt-Reopt is an effective but non-universal technique for LLM-based GPU porting, with gains that depend on the kernel, the model, the search budget, and the success rate. Index Terms—Large Language Models, Code Translation, CUDA, HPC, Deoptimization
I. I NTRODUCTION Scientific computing code is often developed from a simple implementation and then optimized for a specific architecture, but the simple version may be lost or never maintained. When such CPU-oriented code is ported to a GPU, handwritten optimizations such as SIMD expansion, blocking, and datalayout transformations may obscure the algorithmic structure needed for a different parallelization strategy. In such cases, deoptimization (removing architecture-specific optimizations before translation) may improve portability, although it may also discard structure that direct translation could have preserved. In recent years, the code generation capability of large language models (LLMs) has improved rapidly, and their application to automatic translation and optimization of HPC code has been actively studied [1]–[3]. Most existing work, however, focuses on direct translation from CPU code to GPU code or on one-way optimization. The effectiveness of such
deoptimization before LLM-based GPU porting has not been sufficiently investigated. This paper studies the effectiveness of deoptimization in LLM-based GPU porting of CPU code from C++ to CUDA. We compare two workflows: Direct, which translates the CPU code straight to CUDA, and Deopt-Reopt, which first deoptimizes and then translates and reoptimizes. We evaluate both in a Single-shot (one pass) and an Iterative (repeated repair and selection) setting, using two LLMs, OpenAI’s gpt-oss-120b (O120) and Alibaba Cloud’s qwen-3-235b-a22b-instruct-2507 (Q235), on twelve HPC kernels. The analysis focuses on when deoptimization helps, when reoptimization recovers performance, and when Iterative refinement narrows the gap between the workflows. We use feasibility for the property that translated code compiles, runs, and passes validation, and report performance only over successful trials (successful-trial performance), separately from feasibility. We find that deoptimization is effective but not universal: it helps most when CPU- and GPU-oriented designs diverge, yet offers little or even hurts where direct translation already preserves a usable structure, so it is best applied selectively rather than by default. Iterative refinement narrows the gap between the workflows for some models while leaving large Deopt-Reopt advantages on a few kernels, and an exploratory call-count-matched control suggests that part of the Singleshot benefit reflects the extra LLM calls rather than deoptimization itself. The rest of this paper is organized as follows. Section II reviews related work on LLM-based code translation and GPU porting. Section III describes the common evaluation setting — the porting task, hardware and software environment, models, metrics, and benchmark kernels. Section IV presents the Single-shot workflow and its results, and Section V the Iterative workflow. Section VI concludes. II. R ELATED W ORK CPU-to-CUDA porting has long been studied through compiler- and directive-based approaches such as OpenMPto-GPGPU [4] and PPCG [5], which rely on analyzable dependences or annotations rather than LLM-based translation of simplified CPU code. LLM-based GPU code generation is also active: omniCUDA [6] targets OpenMP-to-CUDA,
KernelBench [7] evaluates kernel generation, and CUDALLM [8] and CudaForge [9] use iterative generation and hardware feedback for CUDA optimization. LLMs have also been studied for HPC code translation, including Fortran→C++ translation data and multi-turn translation [3], [10], cross-language and C-to-CUDA translation corpora [2], [11], and Fortran→C++/C++→CUDA dialoguebased data generation with compilation, execution, and unittest feedback [12]. These demonstrate LLM-based porting but do not evaluate removing CPU-specific optimizations before retranslation and reoptimization. LLMs have been studied for general-purpose code refactoring [13]–[15]: they reduce code size and cyclomatic complexity on programming-exercise submissions [13] and can identify and recommend refactorings in real-world projects [14], though with notable limitations [15]; relatedly, they exhibit a natural simplification bias even when the prompt asks for the opposite [16]. LLM-driven refactoring has also been applied to parallel scientific codes for energy-aware optimization [17]. Related work on LLM-based deobfuscation [18] and input-token reduction [19] further motivates simplification before translation; we evaluate LLMbased deoptimization specifically for C++→CUDA porting. III. C OMMON E VALUATION S ETTING A. Task and Environment The task is LLM-based porting of heavily CPU-optimized C++ HPC kernels to CUDA: each input is a hand-optimized C++ implementation for aarch64/Neoverse-V2 using NEON intrinsics or other CPU-oriented optimizations, which we translate to CUDA and evaluate for feasibility and performance. Experiments were performed on a GH200 system with a 72-core Arm Neoverse-V2 CPU and an NVIDIA H100 GPU. The software stack is NVHPC 25.9, CUDA 12.6, cuDNN 9.5.1, NVPL 25.9, and ArmPL 25.07.1. CPU C++ is compiled with nvc++ (-O3 -fopenmp -tp=neoverse-v2 -std=c++17) and CUDA with nvcc (-O3 -arch=sm_90 -std=c++17). Each run uses all CPU cores with threads pinned and their memory bound to the local NUMA node. Each candidate has one warm-up and five timed runs (minimum reported); GPU timings use CUDA events after data placement and exclude host-device transfers and validation. B. Models and Inference We evaluate two open-weight LLMs independently in all workflows: gpt-oss-120b (O120) [20], a 117Bparameter MoE (∼5.1B active) with controllable reasoning, and qwen-3-235b-a22b-instruct-2507 (Q235) [21], a ∼235B-parameter MoE (∼22B active) general-purpose nonthinking instruct model. For inference, both models are served through the Cerebras inference API1 with temperature 0.5, max_completion_tokens=16384, and no fixed random 1 https://www.cerebras.ai/inference
seed (O120 uses reasoning_effort=medium, the gptoss default); serving-side numerical precision is providercontrolled and was not recorded. C. Metrics and Statistical Analysis Input and generated code are normalized by removing C/C++ comments and blank lines while preserving string literals; all reported lines-of-code (LOC) values use this normalized code. Performance differences between Direct and Deopt-Reopt are tested by a two-sided Mann-Whitney U test (α=0.05) over successful trials, separately from feasibility; cases with fewer than three successes are not tested. Unadjusted p-value marks are exploratory, while primary significance uses the Benjamini–Hochberg false discovery rate (BH-FDR) correction [22] (α=0.05) separately for Single-shot and Iterative. The Direct-3 control is auxiliary, so D+R vs Direct-3 uses unadjusted p-values only. Feasibility is reported separately, as per-mode success rates in Table II. D. Benchmark Kernels We use the twelve HPC kernels in Table I. B denotes batch size, M, N, K principal dimensions, C input channels, H, W spatial sizes, BS block size, and T iterations. Before observing results, we group kernels by how CPU-oriented structure relates to the natural GPU strategy: divergentstyle (CPU optimizations obscure a different GPU mapping), dependency/convention-dominated (constrained by algorithmic dependences or numerical conventions), and shared-style (CPU-side structural hints still help on the GPU). This is an author-defined interpretive framework, not a deterministic predictor. Table I reports LOC, input/reference performance, units, and the per-kernel validation tolerance. Input perf is the measured performance of the hand-optimized CPU C++ input (the porting source); Ref. perf is the vendor-library or hand-coded reference on the CPU and the GPU. These references are correctness oracles (and rough performance landmarks), not performance-tuned, same-precision competitors, so a reference can be slower than the hand-optimized input. For example, the DDGEMM reference runs in binary128 as a precision oracle, and the Conv2D reference builds an unvectorized im2col buffer before calling a library GEMM, rather than fusing the convolution like the hand-tuned input. Outputs are validated against vendor libraries where available (NVPL/ArmPL on the CPU; cuDNN/cuFFT/cuBLAS/cuSPARSE on the GPU) and otherwise against high-precision or hand-coded references (e.g., binary128 for DDGEMM and an FP64 reference for DFSpMM). Correctness is checked by benchmark-specific validation drivers at the benchmarking problem sizes, using perelement relative error |x − xref |/ max(|xref |, 10−12 ) against the listed tolerance; Conv2D additionally accepts a maximum absolute error of 10−4 . DFSpMM and DDGEMM use doubleword arithmetic [23], representing a value as an unevaluated sum of two native words: double-double (DD) uses two FP64 words (a ∼106-bit significand) and double-float (DF)
TABLE I B ENCHMARK KERNELS AND EVALUATION SETTINGS . G ROUPS ARE DIVERGENT- STYLE (D IV.), DEPENDENCY / CONVENTION - DOMINATED (D OM .), AND SHARED - STYLE (S HR .); ‘*’ DENOTES HAND - CODED OR DIFFERENT- PRECISION REFERENCE PERFORMANCE . T OL . IS THE PER - ELEMENT RELATIVE - ERROR TOLERANCE ; C ONV 2D ALSO ACCEPTS A 10−4 ABSOLUTE ERROR . Group
Kernel
Problem
Div.
Conv2D
Div.
BFFT
Div.
Softmax
Div.
BGEMM
2D convolution (FP32), N=16, C=32, K=64, H=W=96 Batched complex FP64 FFT, B=8192, N=1024 Row-wise FP64 softmax, M=1024, N=32768 Batched FP64 GEMM, B=1024, M=N=K=128
Dom.
DFSpMM
Dom.
FFT
Dom.
BTDMA
Dom.
DDGEMM
Shr.
Stencil
Shr.
SpMM
Shr.
GEMM
Shr.
SpMV
Sparse–dense, double-float (DF) arithmetic (48-bit significand), CSR, 32 nnz/row, M=K=24576, N=128 Single large complex FP64 FFT, N=4194304 Batched FP64 tridiagonal solve, B=32768, N=256 Dense GEMM, double-double (DD) arithmetic (106-bit significand), M=N=K=256 5-point FP64 Jacobi stencil, N=3072, T=100 Sparse–dense (FP64), CSR, 32 nnz/row, M=98304, K=524288, N=128 Dense FP64 GEMM, M=N=K=1024 FP64 BSR banded sparse matrix–vector, M=K=524288, BS=4
CPU-input structure and GPU-side interpretation Filter-transpose cache, blocking, NEON SIMD; GPU tiles with shared memory.
LOC
Unit GFlops
Input perf 2328
Ref. perf CPU GPU 474.9 24647
142
Radix-4 butterflies, bit reversal, twiddles; small FFTs remap differently on GPU. Handwritten NEON exp, SIMD, buffering; maps to warp/block reductions. NEON FP64 microkernel, blocking, prepacking; GPU uses batch/tile parallelism. Sparse access and DF arithmetic dominate; structure constrains the GPU mapping.
10−3
123
GFlops
598.9
596.4
4907
10−8
67
GB/s
185.6
210.3*
1235
10−8
210
GFlops
1935
159.4
32848
10−10
148
GFlops
17.3
735.3*
2181*
10−7
Radix-4 butterflies, bit reversal, twiddles, NEON wrapper; stage dependences dominate. Parallel Thomas, two-element elimination; forward/backward dependences dominate. DD arithmetic dominates runtime over parallelization.
519
GFlops
209.8
170.0
3154
10−8
92
GB/s
295.7
159.4
141.1
10−8
86
GFlops
44.1
4.3*
569.9*
10−22
NEON SIMD, buffering, locality optimization; grid-update strategy reusable on GPUs. p-loop unrolling, output tiling; CSR row-wise processing is a useful GPU hint.
116
GB/s
1404
970.8*
2789*
10−8
98
GB/s
414.6
203.9*
752.7
10−10
NEON FP64 microkernel, blocking, prepacking; data-movement intent stays useful. Block-sparse format, expansion, vectorization are useful GPU hints.
259
GFlops
305.5
2653
40895
10−8
63
GB/s
1508
685.6
2527
10−10
is the FP32-word counterpart. DDGEMM and DFSpMM use the operation-count definitions of their GEMM and SpMM counterparts. Except for FP32 Conv2D, interfaces are FP64. IV. S INGLE - SHOT W ORKFLOW A. Method We first use a one-pass Single-shot workflow to observe the basic effect of LLM-based deoptimization under three modes. • Deopt-Reopt (D+R): input code → deoptimization → CUDA translation → reoptimization, using three LLM calls. • Direct (D): input code → CUDA translation → reoptimization (no deoptimization), using two LLM calls. • Direct-3 (D3, control): Direct translation followed by reoptimization v1 and reoptimization v2, matching the three LLM calls of D+R; D3 trials are newly sampled and are not a subset of Direct trials. If translation does not produce a passing code (PASS), the trial stops; if v1 fails, v2 is skipped, and if v2 fails, v1 is adopted. Direct-3 tests whether a D+R successful-trial performance advantage can be explained only by the larger number of LLM
Tol.
calls; it gives Direct an extra reoptimization pass and serves as an auxiliary call-count control for successful-trial performance, not a success-rate control. Because Direct-3 spends its extra call on reoptimization rather than retrying translation, it does not test whether the same budget used for additional translation or repair attempts would close Direct’s feasibility gap. The deoptimization phase removes architecture-specific optimizations while preserving the function signature, problem sizes, and numerical results within tolerance; only code passing the same validation driver and tolerance (Table I) as the input C++ is accepted. The translation phase converts the (possibly deoptimized) C++ code to a complete CUDA implementation preserving the entry-point contract: the generated function takes device pointers and implements only deviceside computation, while host allocation and host-device transfers are handled by the benchmark harness. Reoptimization then improves the CUDA code for the target GPU without prescribing specific techniques. If a required stage fails to produce a passing adopted code, the trial is counted as a failure; when a later reoptimization stage fails, the last passing
CUDA code is retained. Prompts specify the phase objective, input code, source/target languages, contract and numerical constraints, and previous-stage code or evaluation results; target hardware information is given to the translation and reoptimization prompts but not the deoptimization prompt (a hardwareagnostic step). We run 50 independent trials per mode and model (150 per kernel-model pair, 3600 in total) and evaluate success rate and the distribution/median of final performance over successful trials; following the common evaluation setting of Section III, the three modes share the LLM, temperature, prompts, problem sizes, validation, and hardware. B. Results Figure 1 shows the Single-shot workflow results. Both this figure and Figure 2 (Iterative) use a 3×4 kernel grid; each panel places the two models side by side with boxplots over successful trials for Direct, Direct-3 (Single-shot only), and Deopt-Reopt. Success rates vary widely by model and kernel. O120 obtained successful trials for all twelve kernels, whereas Q235 had 0/50 successes in both Direct and Deopt-Reopt for spmm and gemm. At the same time, Q235 achieved nearcomplete success on spmv under Direct, with Deopt-Reopt at a high but lower rate. Feasibility thus depends more on model–kernel compatibility than on model strength alone. Accordingly, Table II should be read with success rate and successful-trial performance together, not median performance alone. Feasibility shifts are strongly model-dependent: DeoptReopt sharply raises Q235 success on most kernels, consistent with Q235 being anchored to the input CPU implementation, whereas for O120 the effect is mixed and can even lower feasibility. Across the two models, among 24 cases the Mann-Whitney U test was applied to 18 testable cases. After BH-FDR correction within the Single-shot main series, eight cases were significant: five favored Deopt-Reopt (Q235 conv2d and ddgemm, O120 conv2d, spmm, and gemm) and three favored Direct (O120 ddgemm and btdma, Q235 bfft). Magnitudes vary widely — DR/D ratios from over 40× down to ≈0.15× (Direct faster) — so the performance effect is genuinely mixed rather than uniformly favoring Deopt-Reopt. These cases span all three benchmark groups rather than concentrating in one: only conv2d favors Deopt-Reopt on both models, while ddgemm reverses between models (Deopt-Reopt for Q235, Direct for O120). The benefit is thus kernel- and modelspecific rather than a property of a single kernel class, and on some dependency-dominated kernels (O120 ddgemm and btdma) discarding the CPU structure instead hurts. To check whether the D+R successful-trial performance advantage merely reflects its extra LLM call, we ran the auxiliary Direct-3 control (three LLM calls, no deoptimization; a performance, not success-rate, control) for all 24 cases with 50 trials each, of which 19 were testable. Deopt-Reopt was significantly above Direct-3 (unadjusted) in four cases, with median DR/D3 ratios of 1.3× (O120 conv2d), 1.3× (O120
fft), 18× (Q235 conv2d), and 5.4× (Q235 ddgemm); conversely Direct-3 was faster in four others, with D3/DR ratios of 1.3× (O120 ddgemm), 1.0× (O120 dfspmm), 2.4× (O120 btdma), and 2.5× (Q235 bfft). Of the five Singleshot main-series Deopt-Reopt advantages, three (conv2d on both models and Q235 ddgemm) remained significant against Direct-3, so part of the advantage reflects the deoptimization phase itself rather than call count alone; on the other kernels the call-count-matched control no longer favors Deopt-Reopt. Full Direct-3 statistics are in the released artifact. Overall, Single-shot Deopt-Reopt most clearly helps conv2d (both models) and sharply raises Q235 feasibility, but the successful-trial performance comparison is mixed — Direct is significantly faster on several kernels (O120 ddgemm and btdma, Q235 bfft) — and Deopt-Reopt costs about 50% more LLM calls than Direct. V. I TERATIVE W ORKFLOW A. Method Single-shot is sensitive to one LLM output, so we also use an Iterative workflow that repeatedly plans, generates, evaluates, repairs, and selects candidates. This compares the two modes under a search process closer to practical HPC porting, where both receive the same repair and iteration opportunities. Each phase (deoptimization, translation, reoptimization, and direct translation) runs for up to three generations. In each generation: 1) the LLM acts as a project manager (PM) proposing a strategy from the current code and results; 2) three programmer (PG) calls generate complete implementations; 3) generated code is compiled, executed, and validated; 4) each PG may repair its code once on error; and 5) the best correct candidate seeds the next generation or the final result. Direct omits deoptimization and passes the input code directly to translation. PM, PG, and repair calls use the same LLM and parameters, with role differences expressed only in the prompt. In a single PM call, the model examines the current code, current performance, and previous-generation PG self-analyses, then emits N per-PG strategies (here N =3) that each PG follows in the same generation. Because the three PGs share one PM call, tests use the final trial-level performance value rather than treating PGs as independent samples. Repair is triggered when a PG’s code fails to compile, fails to execute, or fails validation; the failing code, the captured error message, and a brief automated self-analysis are passed back to the same PG, which produces a fixed candidate that is re-validated. For deoptimization selection, we use LOC as a simple and interpretable proxy for simplification: it reflects removal of SIMD expansion and loop unrolling, but not portability as a whole. Among candidates that pass validation, deoptimization adopts the smallest-LOC code subject to being smaller than the input CPU LOC, whereas reoptimization adopts the smallestruntime code. When no valid candidate is smaller than the
8 4
1.6 Q2 35
Q2 35
0
0.8
BTDMA TB/s
DDGEMM TFlops/s 1.0
GEMM TFlops/s
35 Q2
20
0.0 O1
35 Q2
20
O1
35 Q2
20
0.0
0.4
0.5
0.8
0.6 0.3 0.0 35 Q2
20
O1
SpMM TB/s
SpMV TB/s 2.4
8
0.8
1.6
Direct
35 Q2
20 O1
35 Q2
20
0 35 Q2
20 O1
35 Q2
20 O1
35 Q2
20
Deopt+Reopt
0.0
2.4
0.4
4
2.8
2.0 1.6 O1
35 Q2
20
O1 20
Q2 35
O1 20
Q2 35
1.5 0.0 35
O1
Stencil TB/s
4 2 0 O1
35
20
Q2
Direct-3
FFT TFlops/s
3.0
DFSpMM TFlops/s
SpMV TB/s 2.4
0 80 0 40 O1
35 Q2
20
0
2.4 O1
Direct
BGEMM TFlops/s
3.2
GEMM TFlops/s
Q2
20
0.0 O1
35 Q2
20 O1
35 Q2
20
SpMM GB/s
Softmax TB/s
0.0
0.0
0.8
4 O1 20
Q2 35
0
0 0.4
0.4
0 O1
35 Q2
DDGEMM TFlops/s
0.8
0 60 0 30
1 0 20 O1
Stencil TB/s 2.8
O1 20
BTDMA TB/s 0.8
FFT GFlops/s
Q2 35
0.0 O1 20
Q2 35
O1 20
Q2 35
2
4
0.8
0.8 0.0
0.4
3 0 O1 20
DFSpMM TFlops/s
BFFT TFlops/s
O1 20
8
Conv2D TFlops/s 1.6
6
BGEMM TFlops/s
O1
Softmax TB/s
8
BFFT TFlops/s 1.6
Conv2D TFlops/s
Deopt+Reopt
Fig. 1. Single-shot final performance (50 trials). D/D3/DR denote successful trials.
Fig. 2. Iterative final performance (50 trials). D/DR denote successful trials.
input, the input is carried forward unchanged; this occurred in only 1.0% (11/1110) of successful Iterative Deopt-Reopt trials, so the arm is essentially genuine deoptimization. If any phase fails to obtain a valid candidate, the trial is counted as a translation failure. Concretely, a phase fails only when none of its candidates passes validation across all generations and repair attempts (here, up to 3×3×2 attempts: three generations, three PG candidates per generation, and two attempts per candidate—one initial attempt plus one repair). Each Iterative trial is more expensive than Single-shot; we run 50 trials per kernel-mode for both models, evaluating translation success rate, execution performance, and LOC. The twelve kernels therefore produce 2400 Iterative trials, with DeoptReopt’s extra-phase cost included; success-rate comparisons are within-model.
were concentrated in Q235: Deopt-Reopt recovered several Q235 kernels with few or no Single-shot successes, while O120 retained successful trials for all twelve kernels. Of the 24 cases, 23 were testable; after BH-FDR correction within Iterative seven were significant — five favored Deopt-Reopt (Q235 conv2d, ddgemm, and bgemm; O120 bfft and softmax) and two favored Direct (O120 and Q235 stencil). Only the three Q235 cases carry a large effect: conv2d and ddgemm reach DR/D median ratios of 8× and 7×, and for bgemm the gap is feasibility-driven (Direct succeeds in only 7/50 trials, mostly near-failures). The remaining four significant cases are far smaller — O120 bfft at 1.3×, and O120 softmax and the two stencil cases within a few percent. For O120, then, iteration largely closes the Single-shot gap by letting Direct also adapt toward the GPU. For Q235 fft, both workflows have median performance near 1 GFlops; the outputs pass validation but stay far below the GPU reference, i.e., they do not reach high performance. A large median gap need not be significant: several sizeable median differences in Table II are not statistically significant, so medians must be read together with the test.
B. Results Results are shown in Figure 2, with performance and LOC progression in Figure 3. In Iterative, repeated generation, repair, and selection partially mitigated the Single-shot feasibility problems, which
4
100
600 200
50
0.5
0
0
0.0
75
3
50
2
25
1
0
0
DFSpMM (TFlops/s) Q235 10/41 O120 49/49
FFT (GFlops/s) Q235 38/32 O120 50/48
BTDMA (GB/s) Q235 36/43 O120 50/49
DDGEMM (TFlops/s) Q235 50/49 O120 50/49
1.5
LOC
150
1.0
100
0.5
50
0.0
0
500 400 300 200 100 0
600 400 200
125
800
100
600
75
400
50
200
25
0
0
1.0 0.8 0.6 0.4 0.2 0.0
75 50 25
IN DO1 DO2 DO3 CO OP1 OP2 OP3
IN DO1 DO2 DO3 CO OP1 OP2 OP3
IN DO1 DO2 DO3 CO OP1 OP2 OP3
0
Stencil (TB/s) Q235 47/45 O120 49/49
SpMM (GB/s) Q235 14/44 O120 50/50
GEMM (TFlops/s) Q235 1/42 O120 50/50
SpMV (TB/s) Q235 50/48 O120 50/50
200 0
Q235 Direct perf Q235 Deopt-Reopt perf
Q235 Direct LOC Q235 Deopt-Reopt LOC
2 0
O120 Direct perf O120 Deopt-Reopt perf
80 60 LOC
LOC
400
4
2.5 2.0 1.5 1.0 0.5 0.0
40 20 0 IN DO1 DO2 DO3 CO OP1 OP2 OP3
IN DO1 DO2 DO3 CO OP1 OP2 OP3
0
600
250 200 150 100 50 0
6
IN DO1 DO2 DO3 CO OP1 OP2 OP3
1
125 100 75 50 25 0
LOC
LOC
2
800
IN DO1 DO2 DO3 CO OP1 OP2 OP3
100 80 60 40 20 0
125 100
IN DO1 DO2 DO3 CO OP1 OP2 OP3
0
1000
LOC
200
LOC
2.0
3
100
250 200 150 100 50 0 IN DO1 DO2 DO3 CO OP1 OP2 OP3
0
1.0
400
4
IN DO1 DO2 DO3 CO OP1 OP2 OP3
0
150
100
IN DO1 DO2 DO3 CO OP1 OP2 OP3
50
1.5
IN DO1 DO2 DO3 CO OP1 OP2 OP3
2
200
BGEMM (TFlops/s) Q235 7/48 O120 50/50
LOC
800
LOC
150 LOC
6
Softmax (TB/s) Q235 22/43 O120 50/47
LOC
BFFT (GFlops/s) Q235 42/39 O120 50/50
LOC
Conv2D (TFlops/s) Q235 29/45 O120 50/50
O120 Direct LOC O120 Deopt-Reopt LOC
Fig. 3. Performance and LOC progression in Iterative (50 trials) over up to three generations. The horizontal axis traces the workflow stages: IN (input C++), DO1–DO3 (deoptimization generations), CO (CUDA translation), and OP1–OP3 (reoptimization generations); Direct has no DO stage. Solid line with band: performance median and Q1–Q3 (left axis); dashed line: LOC (right axis). Marker shape encodes mode (square: Direct, circle: Deopt-Reopt) and color encodes model. Each panel title lists per-model Direct/Deopt-Reopt success counts (out of 50 trials), consistent with Table II.
Zero-success Single-shot recovery is most visible for Q235, where PM-driven strategy updates and one allowed repair provide a recovery path. Deopt-Reopt markedly raises Q235 Iterative feasibility, recovering several kernels (bgemm, spmm, gemm, dfspmm) that Direct rarely compiles. Overall, Iterative narrows the mode difference markedly for O120, whereas Q235 retains substantial Deopt-Reopt advantages on conv2d, ddgemm, and bgemm. Table II summarizes the win/loss pattern; Single-shot and Iterative each use 50 trials. Because these medians are conditional on success, end-to-end performance also depends on the success rate: weighting a conditional median by success rate can reverse the favored mode, so the medians should be read together with the success columns. The three-group classification has clear exceptions — the shared-style kernel O120 spmm shows a large Single-shot Deopt-Reopt advantage, the divergent-style kernel Q235 bfft favors Direct, and on
the divergent-style O120 conv2d Direct catches up under Iterative — so it is only an interpretive framework. VI. C ONCLUSION We evaluated Deopt-Reopt for LLM-based C++-to-CUDA porting on twelve HPC kernels. In Single-shot, Deopt-Reopt was significantly faster among successful trials in five testable cases — most clearly for conv2d, where the CPU optimization structure diverges from the natural GPU strategy — but Direct was faster in three (O120 btdma and ddgemm, Q235 bfft), so removing CPU-specific optimizations is not universally beneficial; an exploratory Direct-3 control left the call-count-matched comparison roughly balanced. In Iterative, repeated generation and repair narrowed the gap, markedly for O120; Q235, however, retained large Deopt-Reopt advantages on conv2d, ddgemm, and bgemm (after BH-FDR correction, seven of 23 testable cases stayed significant, but
TABLE II C OMPARISON OF D IRECT (D) AND D EOPT-R EOPT (DR): SUCCESS RATE OVER ALL TRIALS AND MEDIAN PERFORMANCE OVER SUCCESSFUL TRIALS . A MARKER ON A MEDIAN MEANS THAT MODE IS SIGNIFICANTLY FASTER AMONG SUCCESSFUL TRIALS — ∗ UNADJUSTED P<0.05, † ALSO AFTER BH-FDR CORRECTION ; § MARKS A MEDIAN OVER FEWER THAN THREE SUCCESSFUL TRIALS ( NOT TESTED ). P ERFORMANCE UNITS FOLLOW THE U NIT COLUMN IN TABLE I. O120 Q235 Single-shot (50 trials) Iterative (50 trials) Single-shot (50 trials) Iterative (50 trials) Success (%) Median perf Success (%) Median perf Success (%) Median perf Success (%) Median perf D DR D DR D DR D DR D DR D DR D DR D DR Divergent Conv2D 64 70 1824 2704∗† 100 100 2512 2908∗ 12 96 106 4366∗† 58 90 545 4389∗† BFFT 48 52 228 164 100 100 370 467∗† 12 66 309∗† 46 84 78 319 273 Softmax 94 90 1315 1317 100 94 1322 1385∗† 2 52 20§ 18 44 86 35 34 BGEMM 88 90 552 4546∗ 100 100 3666 4485 0 94 — 354 14 96 1.79 4474∗† Dominated DFSpMM 86 82 801 797 98 98 850 860 4 88 463§ 2346 20 82 485 355 FFT 42 56 288 424 100 96 336 402 2 20 0.909§ 0.958 76 64 1.08 0.979 BTDMA 92 44 923∗† 380 100 98 940∗ 869 8 10 927 404 72 86 504 486 DDGEMM 66 82 714∗† 564 100 98 819 826 42 80 89 510∗† 100 98 119 848∗† Shared Stencil 82 58 2670∗ 2570 98 98 2849∗† 2788 36 62 2708 2799 94 90 2919∗† 2840 SpMM 66 54 51 685∗† 100 100 700 723 0 0 — — 28 88 251 130 GEMM 84 66 4416 4731∗† 100 100 4732 4748 0 0 — — 2 84 181§ 4676 SpMV 68 60 1719 1714 100 100 2216 2139 96 72 1687 1686 100 96 1711 1760 Group
Bench
only these three carry a large effect). Because performance is measured over successful trials, a lower success rate can offset a conditional gain, so the evidence supports a conditionalperformance benefit, not a guaranteed end-to-end one. In sum, Deopt-Reopt is an effective but non-universal technique for LLM-based GPU porting, with gains that depend on the kernel, the model, the search budget, and the success rate. The study is limited to two LLMs, one GPU environment, twelve kernels, a single prompt-template family, and LOC as a proxy for simplification. Correctness was validated at the benchmark problem sizes; generalization to unseen sizes, layouts, sparsity patterns, and boundary cases remains untested. A fully failure-inclusive analysis of the conditional comparison (best-of-N under a fixed call budget, bootstrap intervals) is left to future work. The small per-case samples also yield low statistical power, leaving some large median gaps nonsignificant. Future work includes larger codebases, more model families, automatic control of deoptimization degree, and simplification metrics beyond LOC; we report median ratios as a rough magnitude indicator, while formal effect-size estimation is deferred to future analysis. The experimental code and data are available at https://github.com/mukunoki/deopt_reopt. ACKNOWLEDGMENT This work was supported by the “Joint Usage/Research Center for Interdisciplinary Large-scale Information Infrastructures (JHPCN)” in Japan (Project ID: jh260065), JSPS KAKENHI JP25K24387, and the JST Next-Generation Edge AI Semiconductor Research and Development Project JPMJES2511. R EFERENCES [1] A. Dhruv and A. Dubey, “Leveraging Large Language Models for Code Translation and Software Development in Scientific Computing,” in Proceedings of the Platform for Advanced Scientific Computing Conference (PASC ’25), 2025, pp. 1–9.
[2] A. TehraniJamsaz, A. Bhattacharjee, L. Chen, N. K. Ahmed, A. Yazdanbakhsh, and A. Jannesari, “CodeRosetta: Pushing the Boundaries of Unsupervised Code Translation for Parallel Programming,” in Advances in Neural Information Processing Systems, vol. 37, 2024, pp. 100 965– 100 999. [3] L. Chen, B. Lei, D. Zhou, P.-H. Lin, C. Liao, C. Ding, and A. Jannesari, “Fortran2CPP: Automating Fortran-to-C++ Translation using LLMs via Multi-Turn Dialogue and Dual-Agent Integration,” arXiv preprint arXiv:2412.19770, 2024. [4] S. Lee, S.-J. Min, and R. Eigenmann, “OpenMP to GPGPU: A Compiler Framework for Automatic Translation and Optimization,” in Proceedings of the 14th ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming (PPoPP 2009), 2009, pp. 101–110. [5] S. Verdoolaege, J. C. Juega, A. Cohen, J. I. Gómez, C. Tenllado, and F. Catthoor, “Polyhedral Parallel Code Generation for CUDA,” ACM Transactions on Architecture and Code Optimization, vol. 9, no. 4, pp. 1–23, 2013, article 54. [6] M. Gruzewski, “Automated Transformation of OpenMP to CUDA Kernels Using AI Models,” Procedia Computer Science, vol. 270, pp. 3352– 3361, 2025. [7] A. Ouyang, S. Guo, S. Arora, A. L. Zhang, W. Hu, C. Ré, and A. Mirhoseini, “KernelBench: Can LLMs Write Efficient GPU Kernels?” in Proceedings of the 42nd International Conference on Machine Learning (ICML 2025), ser. Proceedings of Machine Learning Research, vol. 267, 2025, pp. 47 356–47 415. [8] W. Chen, J. Zhu, Q. Fan, Y. Ma, and A. Zou, “CUDA-LLM: LLMs Can Write Efficient CUDA Kernels,” arXiv preprint arXiv:2506.09092, 2025. [9] Z. Zhang, R. Wang, S. Li, Y. Luo, M. Hong, and C. Ding, “CudaForge: An Agent Framework with Hardware Feedback for CUDA Kernel Optimization,” arXiv preprint arXiv:2511.01884, 2025. [10] N. R. Ranasinghe, S. M. Jones, M. Kucer, A. Biswas, D. O’Malley, A. Most, S. L. Wanna, and A. Sreekumar, “LLM-Assisted Translation of Legacy FORTRAN Codes to C++: A Cross-Platform Study,” in Proceedings of the 1st Workshop on AI and Scientific Discovery: Directions and Opportunities. Association for Computational Linguistics, 2025, pp. 58–69. [11] Y. Wen, Q. Guo, Q. Fu, X. Li, J. Xu, Y. Tang, Y. Zhao, X. Hu, Z. Du, L. Li, C. Wang, X. Zhou, and Y. Chen, “BabelTower: Learning to Autoparallelized Program Translation,” in Proceedings of the 39th International Conference on Machine Learning (ICML 2022), ser. Proceedings of Machine Learning Research, vol. 162, 2022, pp. 23 685–23 700. [12] L. Chen, N. Xu, W. Chen, B. Lei, P.-H. Lin, D. Zhou, R. Thakur, C. Ding, A. Jannesari, and C. Liao, “Beyond Code Pairs: DialogueBased Data Generation for LLM Code Translation,” arXiv preprint arXiv:2512.03086, 2025. [13] A. Shirafuji, Y. Oda, J. Suzuki, M. Morishita, and Y. Watanobe, “Refactoring Programs Using Large Language Models with Few-Shot
Examples,” in Proceedings of the 30th Asia-Pacific Software Engineering Conference (APSEC 2023), 2023. [14] B. Liu, Y. Jiang, Y. Zhang, N. Niu, G. Li, and H. Liu, “An Empirical Study on the Potential of LLMs in Automated Software Refactoring,” arXiv preprint arXiv:2411.04444, 2024. [15] J. Cordeiro, S. Noei, and Y. Zou, “LLM-Driven Code Refactoring: Opportunities and Limitations,” in Proceedings of the 2025 IEEE/ACM 2nd International Workshop on Integrated Development Environments (IDE), 2025, pp. 32–36. [16] L. De Tomasi, C. Di Sipio, A. Di Marco, and P. T. Nguyen, “Simplicity by Obfuscation: Evaluating LLM-Driven Code Transformation with Semantic Elasticity,” in Proceedings of the 29th International Conference on Evaluation and Assessment in Software Engineering (EASE 2025), 2025. [17] M. T. Dearing, Y. Tao, X. Wu, Z. Lan, and V. Taylor, “Leveraging LLMs to Automate Energy-Aware Refactoring of Parallel Scientific Codes,” arXiv preprint arXiv:2505.02184, 2025. [18] D. Beste, G. Menguy, H. Hajipour, M. Fritz, A. E. Cinà, S. Bardin, T. Holz, T. Eisenhofer, and L. Schönherr, “Exploring the Potential of LLMs for Code Deobfuscation,” in Detection of Intrusions and Malware, and Vulnerability Assessment – DIMVA 2025, Proceedings, Part I, ser. Lecture Notes in Computer Science, vol. 15747. Springer, 2025, pp. 267–286. [19] Y. Wang, X. Li, T. N. Nguyen, S. Wang, C. Ni, and L. Ding, “Natural Is The Best: Model-Agnostic Code Simplification for Pre-trained Large Language Models,” Proceedings of the ACM on Software Engineering, vol. 1, no. FSE, pp. 586–608, 2024. [20] OpenAI, “gpt-oss-120b & gpt-oss-20b Model Card,” https: //cdn.openai.com/pdf/419b6906-9da6-406c-a19d-1bb078ac7637/ oai_gpt-oss_model_card.pdf, 2025. [21] Qwen Team, “Qwen3-235B-A22B-Instruct-2507 Model Card,” https:// huggingface.co/Qwen/Qwen3-235B-A22B-Instruct-2507, 2025. [22] Y. Benjamini and Y. Hochberg, “Controlling the false discovery rate: a practical and powerful approach to multiple testing,” Journal of the Royal Statistical Society: Series B (Methodological), vol. 57, no. 1, pp. 289–300, 1995. [23] T. J. Dekker, “A floating-point technique for extending the available precision,” Numerische Mathematik, vol. 18, no. 3, pp. 224–242, 1971.