More with Less: Efficient Parallel Scientific Code Translation via an Agent-Skill Workflow

More with Less: Efficient Parallel Scientific Code Translation via an Agent-Skill Workflow Jianuo Shen∗ , Pengyi Xu† and Bo Zhang† ∗ csjianuo.shen@mail.scut.edu.cn ∗ Department of Computer Science & Engineering, South China University of Technology, Guangzhou, Guangdong 510006 † {vxu,

bo.zhang}@mtu.edu

arXiv:2611.08947v1 [cs.DC] 6 Nov 2026

† Department of Computer Science, Michigan Technological University, Houghton, Michigan 49931

Abstract—Translating parallel scientific codes between HPC programming frameworks is a critical but labor-intensive task. Existing LLM-based approaches rely on fixed sequential pipelines with undifferentiated repair strategies and correctness criteria that do not match scientific computing practice. We present an agent-skill framework that separates translation, compilation diagnosis, and execution validation into three autonomous agents coordinated by a verdict-driven orchestrator, routing each failure to a targeted repair skill rather than retrying blindly. An automated output check drives the self-correction loop, complemented by an offline LLM-based verification that exposes false positives escaping compiler and runtime checks. We evaluate the framework on 30 HeCBench kernels translated bidirectionally among CUDA, OpenMP, and SYCL with three open-source LLMs. Our framework attains the higher verified correctness rate in all 18 model and direction combinations over a state-of-the-art baseline while consuming up to 52.4% fewer tokens, remaining consistent across models and directions without architectural modification. Index Terms—Large Language Models, Agentic Workflow, Parallel Code Translation, Agent-Skill Architecture, HPC Programming Models

I. I NTRODUCTION The ability to automatically translate code between programming languages has long been a goal of software engineering research, and recent advances in large language models (LLMs) have brought it closer to practical reality. Modern LLMs such as GPT-4 [1], DeepSeek Coder [2], and Qwen2.5-Coder [3] demonstrate strong generalization across programming languages, enabling translation pipelines that combine prompt engineering, compiler feedback, and iterative repair to achieve compelling results on general-purpose benchmarks [4], [5]. These capabilities have sparked growing interest in applying LLMs to automate the translation of real-world codebases, reducing the manual effort required to port software across language ecosystems and opening new avenues for code modernization at scale. Despite this progress, LLM-based code translation remains poorly understood in the domain of parallel scientific computing. General-purpose translation pipelines typically assess correctness through test case passing rates or code similarity metrics [6], [7], criteria that are adequate for software engineering tasks but insufficient for scientific computing workloads. Parallel scientific codes impose a stricter and more domainspecific notion of correctness: numerical outputs must be equivalent within floating-point tolerance across potentially

billions of arithmetic operations, translated code must genuinely offload computation to the target device rather than silently falling back to CPU execution, and results must be deterministic across parallel runs [8], [9]. These requirements are difficult to satisfy precisely because the semantics of parallelism, memory hierarchy, and synchronization differ fundamentally across programming frameworks such as OpenMP [10], CUDA [11], and SYCL [12], a semantic gap that has no analog in conventional language-to-language translation and that generalpurpose pipelines are not designed to bridge. Collectively, these failures remain invisible to evaluation methodologies that stop at compilation success or surface-level output matching, yet they are precisely the failures that matter most in scientific computing practice. Several systems have recently addressed aspects of this problem, but share three recurring limitations. First, existing pipelines follow fixed sequential workflows with no dynamic routing: all errors are handled identically regardless of their root cause, leading to unnecessary LLM calls and poor token efficiency. LASSI [13] exemplifies this pattern, achieving strong results on bidirectional OpenMP ↔ CUDA translation with compiler and execution feedback, but applying the same repair strategy indiscriminately to every failure type. Second, automated semantic verification remains absent or superficial. CodeRosetta [14], a purpose-built encoder-decoder transformer for C++ ↔ CUDA and C++ ↔ Fortran translation, reports 93% functional correctness on a sample of 30 kernels but does not assess numerical equivalence under floating-point arithmetic, the correctness criterion most critical for scientific workloads. LASSI similarly relies on manual output inspection rather than automated output validation. Third, existing pipelines are token-inefficient by design. LASSI and similar systems such as UniPar [15] and ParaCodex [9] apply the same LLM-based repair strategy regardless of error type, incurring unnecessary token consumption on failures that could be resolved with targeted, lightweight interventions. This indiscriminate retry behavior limits practical scalability and increases inference cost without a corresponding improvement in translation quality. Inspired by the emerging paradigm of agent-skill decomposition in autonomous AI systems [5], [16]–[18], we propose an agentic workflow for parallel scientific code translation that addresses these limitations through principled separation of concerns. Our framework comprises three collaborating agents coordinated by an orchestrator: a TranslationAgent

©2026 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses... Accepted for publication in the Proceedings of the SC’26 Workshops (AgenticAI4HPC 2026).

that encapsulates code analysis, memory-aware planning, and self-reviewing translation; a CompilationAgent that combines compilation with structured error classification to dynamically route failures to the appropriate repair strategy; and an ExecutionAgent that automatically checks output correctness between source and translated outputs and diagnoses runtime failures including race conditions and incorrect GPU offloading. Rather than treating every error as a prompt for blind retry, our orchestrator reads the verdict from each agent and selects the repair skill best suited to the diagnosed failure mode, reducing unnecessary LLM calls and improving overall token efficiency. We evaluate the framework on 30 HeCBench [19] kernels translated bidirectionally among OpenMP, CUDA, and SYCL with three recent open-source LLMs, measuring verified correctness and token consumption against a state-of-the-art baseline, and further audit the results under a stricter LLMbased complementary verification. Our results demonstrate that the agent-skill architecture achieves higher verified correctness than the baseline while reducing token consumption by up to 52.4%, remaining consistent across three open-source LLMs and all six translation directions among CUDA, OpenMP, and SYCL without architectural modification. This paper makes the following contributions: • We propose an agent-skill architecture for parallel scientific code translation that separates translation, compilation diagnosis, and execution validation into three autonomous agents. This separation enables verdict-driven routing that targets each repair skill at a specific failure mode, reducing unnecessary LLM calls while improving correctness through more precise self-correction. • We introduce an automated output check as a first-class validation step, replacing manual output inspection with a quantitative correctness signal that drives a dedicated semantic repair loop, together with an LLM-based complementary verification that audits translated artifacts offline and exposes false positives escaping the output check. • We demonstrate that our framework achieves higher translation correctness than LASSI on the same benchmark while reducing token consumption by up to 52.4%, establishing that more capable behavior does not require more expensive prompting. • We show that the framework generalizes across OpenMP, CUDA, and SYCL, supporting bidirectional translation between multiple HPC parallel programming frameworks without architectural changes. The rest of this paper is organized as follows. Section II reviews related work. Section III describes the agent-skill architecture. Section IV presents experimental setup and results. Section V discusses findings and limitations. Section VI concludes and outlines directions for future work.

distinct parallel programming frameworks to achieve peak performance [20]. The current TOP500 list [21] includes GPUs from NVIDIA, AMD, and Intel, driving a proliferation of parallel programming models including OpenMP, CUDA, SYCL, HIP [22], and Kokkos [23] that target overlapping but incompatible hardware ecosystems [24]. Manual porting is prohibitively expensive, with domain scientists estimating refactoring costs of upwards of eleven million dollars per 100,000 lines of code [25], and risks introducing subtle numerical and performance errors that are difficult to detect and reproduce. These pressures have driven growing interest in automated approaches to parallel scientific code translation, motivating the work surveyed below. A. LLM-Based Code Translation

Automated code translation has a long history in software engineering. Early approaches relied on rule-based transpilers that applied hand-coded linguistic rules to map source language constructs to target language equivalents (as in SYCLomatic [26], the production CUDA → SYCL migration tool), followed by statistical machine translation methods that applied phrase-based models originally designed for natural language to translate between programming languages such as Java and C# [27]–[29]. The emergence of neural models brought a significant shift: Roziere et al. introduced TransCoder [30], an unsupervised transformer model that leverages masked language modeling, denoising autoencoding, and back-translation to translate between Python, Java, and C++ without requiring parallel training data. Subsequent work improved upon this foundation by incorporating automated unit test execution to filter invalid translations [31] and by injecting code structure through abstract syntax trees and compiler representations [32]. With the rise of large language models, prompt-based approaches have largely supplanted purpose-built translation models, enabling zero-shot and fewshot translation across a much wider range of language pairs without task-specific training [4]. Compiler feedback and iterative repair loops have emerged as an effective complement to prompting, feeding compilation and execution errors back to the LLM for self-correction [6]. Despite these advances, general-purpose translation systems assess correctness through code similarity metrics [7] or test case passing rates [6], criteria that are insufficient for parallel scientific codes where numerical equivalence and genuine GPU offload are the relevant correctness standards. Applying these advances to parallel scientific codes has motivated a dedicated line of work in the HPC community. Lei et al. [33] introduced a dataset of manually curated OpenMP Fortran and C++ code pairs and demonstrated that fine-tuning LLMs on HPC-specific translation data significantly improves translation quality over general-purpose baselines. LASSI [13] II. R ELATED W ORK introduced a self-correcting pipeline for bidirectional CUDA ↔ The rapid diversification of HPC hardware has made parallel OpenMP translation combining self-prompting, static domain scientific code portability one of the most pressing challenges knowledge injection, and iterative compiler and execution in computational science. Leadership-class systems such as feedback, but follows a fixed sequential workflow that applies Frontier, Aurora, and El Capitan feature heterogeneous CPU- the same repair strategy to every failure type and relies GPU architectures from different vendors, each requiring on manual output inspection rather than automated output

verification; its extension LASSI-EE [34] targets energy- has also been demonstrated directly in code translation: Gray aware refactoring, without numerical equivalence verification. et al. [39] found that an agentic MATLAB → HDL translation CodeRosetta [14] proposed an unsupervised encoder-decoder pipeline with conditional tool invocation outperforms fixed transformer for C++ ↔ CUDA and C++ ↔ Fortran translation sequential workflows by over 20% points, with gains attributed with training objectives tailored to parallel code semantics, but to short prompts and on-demand tool use. These agentic softevaluates correctness on a small sample of kernels without ware engineering systems, however, assess correctness through assessing numerical equivalence under floating-point arithmetic. test suite execution, criteria that do not transfer to parallel UniPar [15] evaluated fine-tuning, hyperparameter optimization, scientific codes where numerical equivalence, genuine GPU and compiler-guided repair for translation among serial code, offload, and parallel determinism are the relevant correctness CUDA, and OpenMP, improving over off-the-shelf models standards. Our work applies the agent-skill paradigm to this while leaving numerical correctness unverified. ParaCodex [9] domain, introducing domain-specific correctness verification introduced profiling-guided agentic planning for serial and as a first-class component of the agentic workflow rather than CUDA to OpenMP GPU offload translation, using hotspot a post-hoc evaluation step. analysis and explicit data movement planning to prevent III. S YSTEM D ESIGN performance regressions, but focuses on parallelization rather than bidirectional translation between established parallel proThis section presents the design of our LLM-driven code gramming frameworks. Evaluation in this space has primarily translation framework. Translating parallel programs between relied on kernel-level benchmarks such as HeCBench [19], programming models is rarely a one-shot task: LLM-generated which provides over 500 computational kernels with CUDA, code frequently fails to compile, and code that compiles may OpenMP, and SYCL implementations. Most recently, ParEval- still produce incorrect results at runtime. Rather than treating Repo [35] extended evaluation to repository-level HPC transla- translation as a single generation step, our framework organizes tion, finding that existing approaches largely fail beyond trivial the task as a multi-agent system in which specialized agents kernel scale due to cross-file dependencies and build system collaborate through a central orchestrator and an iterative complexity. Across these systems, three recurring limitations repair loop. An LLM is invoked for code generation, rewriting, emerge: fixed sequential workflows with no dynamic error-type- analysis, and planning tasks. Meanwhile, in-loop verification is aware routing, absence of automated correctness verification, delegated to the real compiler and runtime behavior, compleand indiscriminate repair strategies that incur unnecessary token mented by an offline LLM-based complementary verification. consumption regardless of failure type. A. Overview and Design Goals B. Agentic Workflows for Software Engineering Our framework translates parallel programs between different The application of LLM-based agents to software engineering tasks has grown rapidly, driven by the observation that complex software tasks require multi-step reasoning, tool use, and iterative self-correction that single-prompt approaches cannot provide. ReAct [5] established the foundational paradigm of interleaving reasoning and acting in LLM-based agents. SWEagent [36] demonstrated that agent-computer interfaces enabling LLMs to interact with terminals, editors, and test runners substantially outperform static prompt-based approaches on real-world GitHub issue resolution. AutoCodeRover [37] combined LLMs with AST-based program analysis to autonomously localize and repair bugs, while RepairAgent [38] showed that LLM-based agents with access to compilation and test execution feedback can navigate complex repair trajectories autonomously. These systems share a common architectural insight: separating high-level decision-making from low-level execution enables more robust workflows than monolithic prompt chains. Skills as stateless execution units have gained attention for improving modularity and token efficiency. Rather than routing all tasks through a single LLM call or distributing them across communicating agents, skill-based architectures assign lightweight, targeted execution to specialized stateless components, reducing token usage compared to multi-agent alternatives [18], while reserving expensive LLM reasoning for decisions that genuinely require it [16]. The advantage of dynamic, conditional execution over fixed sequential pipelines

parallel programming models through a multi-agent system coordinated by a central orchestrator. As shown in Figure 1, the system consists of three agents: a TranslationAgent generates the target-language code, a CompilationAgent validates that the code compiles, and an ExecutionAgent runs the compiled binary and checks correctness. Each agent is composed of a fixed set of skills that it executes in sequence: the TranslationAgent runs code analysis, memory planning, and translation skills; the CompilationAgent runs compiler invocation and error classification skills; and the ExecutionAgent runs binary execution and runtime analysis skills. In addition, the framework provides specialized repair skills to handle distinct compilation and runtime failure modes. The orchestrator drives the workflow through a verdict-based self-correction loop. After each agent or repair skill executes, the orchestrator records the returned verdict and consults a routing table that maps every verdict status to the next agent, a repair skill, or a terminal state. The loop terminates upon reaching a terminal verdict, exhausting a global step limit, or exceeding a unified self-correction budget that counts only repair invocations. This verdict-driven routing separates the concerns of code generation, failure classification and repair. The overarching goal of the framework is to support translation between arbitrary parallel programming models. To make this practical, concrete translation logic is encapsulated in dynamically loaded skills rather than baked into the agent or orchestrator code, so adding a new source→target translation

Fig. 1. System overview. The orchestrator drives a verdict-based self-correction loop through the three agents, routing each diagnosed failure to a targeted repair skill until the translation passes compilation and output checking or the self-correction budget is exhausted.

pair requires only pair-specific skill implementations and prompts without modifying the system architecture. The active pair is selected at runtime through a single configuration flag, which swaps the bound skill implementations, prompt templates, and compiler choices.

successful translations are not penalized by the presence of repair machinery, while preventing runaway retry loops. C. Agent Skills and Repair Mechanisms

Each agent in the system is composed of one or more core skills that encapsulate a single capability. Repair skills are B. Task State, Verdict, and Orchestrator managed directly by the orchestrator and invoked when a stage The framework uses a shared TaskState object to represent returns a failure verdict. the evolving state of a single translation task as it moves The TranslationAgent runs three skills in sequence. First, through the orchestrator’s self-correction loop. Initialized from the AnalyzerSkill invokes an LLM to produce a frameworkthe source program and pair configuration, it tracks the current independent functional summary of the source program, detranslated code, the latest compilation and execution artifacts, scribing its computation stages, inputs, outputs, and correctness and bookkeeping information such as step history and retry criteria. The summary is stored in the shared state and later counters. A generic metadata bucket inside the TaskState allows consumed by the translator. Then, the MemoryPlanningSkill skills to share intermediate context, such as code summaries generates a memory and data-layout plan for the target and memory plans. Both agents and skills mutate this state in execution model, mapping each buffer to its intended sharing place, which simplifies the communication protocol. behavior, lifetime, and synchronization requirements. Finally, Agents and skills communicate through a structured Verdict the TranslatorSkill synthesizes the complete target-language object that encodes the outcome of the most recent step. program by combining the source code with the functional The orchestrator invokes CompilationAgent or ExecutionAgent, summary and the memory plan. which maps compiler log or runtime output to a verdict category The CompilationAgent contains two skills. The CompilerSkill according to a set of trigger conditions. The orchestrator then invokes the real target-language compiler and captures the dispatches the verdict to the corresponding repair skill or back compilation log and the resulting binary path. If compilation to TranslationAgent via a fixed routing table. The mapping succeeds, the agent returns a pass verdict and control advances from trigger conditions to verdicts and their routing targets is to the ExecutionAgent; otherwise, the ErrorClassifierSkill given in Table I. applies a set of trigger condition as shown in Table I to the The Orchestrator implements the central control loop. It compiler log as presented in Section III-B. consumes verdicts, advances the pipeline to the next agent The ExecutionAgent also contains two skills. The Runor repair skill, and enforces termination conditions. The nerSkill executes the compiled binary with test arguments orchestrator also maintains the unified self-correction budget, and records stdout, stderr, and the exit code. The configurable at construction time, whose cap on repair attempts RuntimeAnalysisSkill then maps the recorded output to a verdict applies across the entire system. Forward progress through category according to the trigger conditions in Table I. If translation, compilation, and execution does not consume this execution times out, the RuntimeAnalysisSkill automatically budget; only repair invocations do. This separation ensures that scales down numeric test arguments that exceed a configurable

TABLE I C LASSIFICATION OF COMPILATION AND RUNTIME FAILURES INTO VERDICT CATEGORIES . Agent

CompilationAgent

ExecutionAgent

Skill

Trigger condition Linker errors (undefined reference, multiple definition, ld failure) Leftover source-framework keywords (e.g., ErrorClassifierSkill __host__, __device__) Target-framework API or directive misuse (e.g., sycl::, omp target, cudaMalloc) Genuine memory-allocation failures (out of memory, invalid device pointer) Missing headers or source files C++ syntax, type, or scope errors Empty compiler log (no actionable error) Exit code 0 and no FAIL marker in stdout Exit code 0 but explicit FAIL marker in stdout RuntimeAnalysisSkill Non-zero exit code Execution timeout; args auto-scaled and rerun

Verdict PATCH

Routed to PatchSkill

PATCH

PatchSkill

PATCH

PatchSkill

MEMORY

MemoryRepair

PATCH PatchSkill PATCH PatchSkill RETRANSLATE TranslationAgent EQUIVALENT Done (terminal) SEMANTIC SemanticRepair CRASH SyncRepair REPAIR_EXECUTION RunnerSkill

TABLE II C ORE CONTENT OF SKILL PROMPTS . P LACEHOLDERS IN BRACES ARE FILLED AT RUNTIME . Skill

Core Prompt Content

AnalyzerSkill (shared)

System: You are a code analysis assistant. Describe what a program does in plain, framework-independent terms. User: Analyze the following {source language} code. Focus only on the real execution logic and numerical behavior. Include: what problem the program solves, inputs and expected outputs, computation stages, correctness criteria, and numerical concerns. Do NOT focus on {target language} API names, {target language and runtime constructs} or synchronization details. Keep the response under 250 words. System: You are an expert in {target language} parallelism. User: Given the following {source language} code, produce a memory plan for {target language} execution. List each buffer with: variable name, size, lifetime, and sharing behavior. Identify where reduction, atomic, critical, or barrier clauses are needed. Respond as a bullet list only. [BEGIN CODE SUMMARY] {code summary} [END CODE SUMMARY] [BEGIN MEMORY PLAN] {memory plan} [END MEMORY PLAN] Generate new code to refactor the following parallelized {source language} program to instead use C++ code written with {target language} directives. The following code was compiled with {compiler} {flags} and produced the following compile error: {compile log} Re-factor the above code with a fix to eliminate the stated error. System: You are a coding assistant that fixes numerical logic errors in {target language} code. User: The translated code was executed and produced incorrect or unexpected output. Translated stdout: {run stdout} Translated stderr: {run stderr} Identify and fix the numerical logic error. System: You are an expert in {target language} parallel programming synchronization. User: The following {target language} code crashes or produces non-deterministic results (exit code {run exit code}, stderr: {run stderr}). Add missing barriers, atomic operations, or reduction clauses as needed. System: You are a coding assistant that fixes memory-related compilation errors in {target language} code. User: The following code was compiled with {compiler} {flags} and produced a compile error related to memory usage: {compile log} [BEGIN MEMORY PLAN] {memory plan} [END MEMORY PLAN] Fix the memory issue.

MemoryPlanningSkill TranslatorSkill PatchSkill SemanticRepair SyncRepair MemoryRepair

threshold. The scaling factor is halved across attempts, and the modified arguments are written back to the TaskState before the binary is rerun with smaller inputs. This mechanism allows computationally intensive kernels to be validated without manual input tuning. And the total number of rescaling attempts is configurable and defaults to five.

pairs, while the TranslatorSkill and repair skills use pair-specific prompts that follow a common structure but encode different target-model constraints.

D. Compilation, Runtime, and LLM-based Verification The framework adopts an in-loop verification stage plus an LLM-based complementary verification. The in-loop verificaFour repair skills handle the failure modes exposed by tion stage relies on compiler and runtime feedback to validate compilation and execution. Two address compilation failures: translated programs and provide signals for the self-correction the PatchSkill rewrites the current code to eliminate a re- workflow. Compilation success is treated as the prerequisite ported compilation error, and the MemoryRepair specializes for execution, while runtime behavior is evaluated through in memory-related compilation errors, using the previously execution status and program outputs. A translation is accepted generated memory plan to correct allocation, mapping, or by this stage only when the generated binary exits with code sharing mistakes. The other two repair skills address runtime zero and its output contains no explicit FAIL marker (a failurefailures: the SemanticRepair fixes numerical or logical errors indicating word in the program output), corresponding to the observed at runtime, such as incorrect loop bounds, missing EQUIVALENT verdict in Table I. reductions, or precision loss, and the SyncRepair addresses Although compiler and runtime verification provide reliable crashes and non-deterministic behavior by inserting missing and reproducible signals, they cannot capture all correctness barriers, atomic operations, or synchronization constructs. Each issues in translated programs. A program may compile sucrepair skill invokes an LLM to generate corrected code and cessfully and terminate normally while still failing to execute returns a repair-compilation verdict, sending the modified code the intended workload, producing incomplete computations, or back to the CompilationAgent and closing the self-correction generating incorrect numerical results without explicit error loop. messages. Such false positives can lead to an overestimation The prompts that drive these skills are summarized in Table II. of translation success rates and obscure the actual capability The AnalyzerSkill uses a shared prompt across all translation of the translation framework.

To address these limitations, we introduce an LLM-based TABLE III complementary verification. Unlike the compiler and runtime T HE 30 H E CB ENCH BENCHMARKS USED IN OUR EVALUATION , GROUPED BY DOMAIN / PATTERN . checks that participate directly in the self-correction workflow, Domain Benchmarks the LLM verifier is applied offline to cached translation artifacts Image / computer vision background-subtract, car, distort, fhd and aims to identify defects that escape compiler and runtime Deep learning dense-embedding, p4, pointwise, rowwiseMoments / statistics / trigonometric atan2, kurtosis, langevin, michalewicz checks. The verifier receives the complete source program and Math Sorting / parallel primitives merge, quicksort, scan, bscan iso2dfd, jacobi execution outputs, and produces a structured JSON judgment Stencil / numerical methods / cryptography jenkins-hash, md5hash based on a predefined evaluation rubric. Its role is therefore Hashing Physics / simulation aidw, md, sheath, tsp Signal / data processing fft, interval, reverse2D evaluative rather than corrective: it provides a stricter postRNG / runtime / other kiss, streamPriority, contract hoc assessment of translation quality without altering the translations it audits. In practice, its confidence scores can guide of HPC and AI kernels written in CUDA, OpenMP, SYCL and users in prioritizing artifacts for manual inspection, starting HIP. It provides a diverse set of real-world GPU workloads that allow us to assess cross-model code translation in terms with the least confident ones. of correctness and token usage. In our experiments, we select The verifier follows several decision criteria: 1) The translated program must genuinely execute the intended a representative subset of 30 benchmarks from HeCBench for benchmark workload rather than bypassing computation domain coverage rather than language completeness, spanning through early exits, placeholder implementations, or no-op a wide range of application domains (Table III); requiring implementations in all three programming models would kernels; 2) Explicit failure indicators or non-zero error counts in the skew the subset toward older and simpler kernels. Six of the execution output are treated as definitive failures, regardless selected kernels lack OpenMP implementations, leaving 24 benchmarks per OpenMP-source direction. of superficial success signals; 3) Numerical discrepancies are evaluated based on semantic b) Models, directions, and baseline: We conduct expercorrectness rather than strict textual matching, allowing iments with three recent open-source LLMs (DeepSeek-V4legitimate floating-point variations; Flash [40], GLM-5.2 [41], and Kimi-K2.7-Code [42]) on all six 4) Surface-level success markers (e.g., “PASS”) are considered directed language pairs among CUDA, OpenMP, and SYCL. insufficient unless supported by meaningful execution traces We compare our framework with LASSI [13]: to keep the and consistent computational results; comparison fair, we replace the LLMs used in the original 5) Empty, truncated, or otherwise unverifiable outputs are LASSI study (GPT-4, Codestral, WizardCoder, and DeepSeekconservatively classified as failures to avoid rewarding silent Coder-V2) with the same three LLMs, and run both systems crashes or incomplete executions. on the same benchmark set under identical compilation and In addition to a binary pass/fail judgment, the verifier reports execution environments. a confidence score in the interval [0, 1]. The confidence score c) Environment and Configuration: We use g++ 8.5.0 is calibrated using a five-tier rubric that maps the strength of for OpenMP target programs, nvcc 12.8 for CUDA target available evidence to numerical ranges: programs, and icpx/dpcpp 2021.4.0 for SYCL target 1) 0.95–1.0: The execution output contains explicit verification programs. Compilation is capped at 300 seconds and execution evidence (e.g., PASS markers, zero failures, or low RMSE), at 120 seconds. The repair loop permits up to 9 iterations. and the translated program is complete; 2) 0.8–0.9: The output appears correct, but the verification B. Correctness and Token Consumption Evaluation among OpenMP, CUDA, and SYCL evidence is incomplete or partially documented; 3) 0.6–0.7: Execution succeeds, but explicit verification signals Table IV reports, for every system, model, and direction, are absent and the judgment relies primarily on program the verified correctness rate together with the total token analysis; consumption and its translation and repair split. We start with 4) 0.4–0.5: The available evidence is weak or contradictory, DeepSeek V4 Flash on the CUDA↔OpenMP pair, the most making the judgment uncertain; common translation scenario in HPC practice and also the pair 5) 0.0–0.3: The available information is insufficient for a LASSI was originally designed for. Both systems perform well reliable assessment. here: our framework translates 93.3% (CUDA→OpenMP) and By introducing this LLM-based complementary verification, 95.8% (OpenMP→CUDA) of the benchmarks correctly, against the framework reduces the risk of overestimating translation LASSI’s 86.7% and 87.5%. The two systems differ mainly quality due to limitations of compiler and runtime checks. The in how much they spend and where the tokens go. LASSI confidence-aware evaluation further encourages conservative consumes 1,327k and 1,058k tokens on the two directions, of judgments in ambiguous cases and provides a more reliable which 41% and 56% go into repairing retrieval-based drafts. assessment of generated parallel programs. Our framework consumes 718k and 835k tokens, 45.9% and 21.1% less, with only 28% and 47% spent on repair. IV. E VALUATION Beyond the CUDA↔OpenMP pair LASSI was designed A. Experimental Setup for, we add SYCL as a representative low-resource parallel a) Benchmarks: We evaluate our approach on programming model and extend the evaluation to all six HeCBench [19], a widely-used benchmark collection directions with the same model. Our framework attains the

TABLE IV C ORRECTNESS RATE AND TOKEN CONSUMPTION OF OUR FRAMEWORK AND LASSI ACROSS THREE LLM S AND SIX TRANSLATION DIRECTIONS . F OR EACH PAIR OF CORRECTNESS RATES , THE HIGHER ONE IS PRINTED IN BOLD GREEN AND THE LOWER ONE IN RED ; TOKEN FIGURES ARE UNCOLORED . T HE AVG . COLUMN AVERAGES THE SIX DIRECTIONS . correctness (%) on top / kilo-tokens (translation/repair) below

Model

LASSI

DeepSeek V4 Flash

Ours

DeepSeek V4 Flash

LASSI

GLM 5.2

Ours

GLM 5.2

LASSI Ours

Kimi K2.7 Code Kimi K2.7 Code

CUDA→OpenMP 86.7%

OpenMP→CUDA 87.5%

CUDA→SYCL 46.7%

SYCL→CUDA 83.3%

OpenMP→SYCL 33.3%

SYCL→OpenMP 53.3%

1327k (59/41)

1058k (44/56)

1690k (11/89)

1174k (48/52)

1514k (11/89)

1945k (42/58)

1451k

93.3%

95.8%

70.0%

93.3%

79.2%

100.0%

88.7%

718k (72/28)

835k (53/47)

1600k (33/67)

1191k (45/55)

1325k (35/65)

753k (73/27)

1071k

46.7%

60.0%

3.3%

53.3%

3.3%

26.7%

34.5%

1364k (50/50)

741k (55/45)

1429k (10/90)

2260k (22/78)

1258k (10/90)

1618k (44/56)

1445k

93.3%

79.2%

60.0%

90.0%

58.3%

86.7%

78.6%

648k (55/45)

609k (52/48)

967k (40/60)

828k (48/52)

1042k (32/68)

2014k (20/80)

1018k

56.7%

73.3%

50.0%

93.3%

26.7%

6.7%

54.8%

3204k (32/68)

1088k (58/42)

2058k (14/86)

1189k (60/40)

2731k (10/90)

3775k (24/76)

2341k

100.0%

100.0%

93.3%

96.7%

75.0%

100.0%

94.6%

860k (88/12)

810k (77/23)

1507k (54/46)

999k (78/22)

1628k (42/58)

887k (87/13)

1115k

100

1,000

1,500

1,200 Tokens (kilo)

Avg. 65.5%

700

50

1,100

800 400

Success rate (%)

System

700

400

0

(a) DeepSeek, OpenMP→SYCL

(b) GLM, OpenMP→SYCL

(c) Kimi, OpenMP→SYCL

Tokens (kilo)

600

1,400

500

600

400 0

3 6 Repair round

9

50

1,000

600

Success rate (%)

100 800

700

0 0

3 6 Repair round

9

0

3 6 Repair round

9

(d) DeepSeek, CUDA→OpenMP (e) DeepSeek, OpenMP→CUDA (f) DeepSeek, CUDA→SYCL Fig. 2. Repair loop effectiveness in six representative model and direction settings. In each subfigure, bars (left axis) report the cumulative token consumption in kilo-tokens and the curve (right axis) reports the cumulative success rate. Note that the left axes are truncated and do not start from zero. Round 0 is the initial translation before any repair; rounds 1 to 9 are repair invocations. Top row: OpenMP→SYCL, the most difficult direction in Table IV, under all three models; bottom row: DeepSeek V4 Flash under CUDA→OpenMP, OpenMP→CUDA, and CUDA→SYCL.

higher correctness rate on every direction and averages 88.7% against LASSI’s 65.5%. The gap is smallest on SYCL→CUDA (93.3% vs. 83.3%), where both systems can rely on the model’s familiarity with CUDA-style code, and largest on OpenMP→SYCL (79.2% vs. 33.3%), where LASSI’s retrieval corpus offers little reusable material. Runs with SYCL as the target are intrinsically harder for both systems, averaging 74.6% for our framework and 40.0% for LASSI against 96.7% and 68.3% with SYCL as the source, while the CUDA↔OpenMP pair is nearly symmetric (93.3% vs. 95.8%). Both systems read low-resource SYCL far better than they write it. The average token consumption is 26.2% lower at the same time (1,071k vs. 1,451k per direction). The only direction where our framework spends more is SYCL→CUDA (1,191k vs. 1,174k, +1.5%), and there the extra repair rounds come with a 10.0% correctness gain. Beyond these direction-level trends, the failing benchmarks show no obvious pattern, scattering across domains and kernel styles.

C. Generalization Across Models The remaining rows of Table IV repeat the experiment with GLM-5.2 and Kimi-K2.7-Code, and the key observation is that the performance of our framework is largely agnostic to the choice of LLM. Its per-model averages span only 16.0% (78.6% to 94.6%), and every direction stays from 6.7% to 33.3% across models (17.0% on average), because the verify-and-repair loop compensates for weaker first drafts instead of relying on a particular model’s strengths. LASSI, by comparison, fluctuates far more: 31.0% across its per-model averages and 38.5% per direction, with swings of 27.5% to 40.0% even on its three strongest directions. Our framework also attains the higher correctness rate in all 18 combinations. Token consumption is equally predictable, staying within 1,018k to 1,115k tokens per direction (a spread below 10%; LASSI: 1,445k to 2,341k), 29.5% to 52.4% below LASSI per model; only two of the 18 cells spend more than LASSI, and both buy large correctness gains (up to +60.0% for +24.5% tokens).

System Model

N In-loop

Ours DeepSeek V4 Flash 168 Ours GLM 5.2 168 Ours Kimi K2.7 Code 168 LASSI DeepSeek V4 Flash 168 LASSI GLM 5.2 168 LASSI Kimi K2.7 Code 159 Overall 999

88.7 78.6 94.6 65.5 34.5 57.9 70.1

Compl. ∆ FP verif. 71.4 −17.3 29 68.5 −10.1 17 86.3 −8.3 14 46.4 −19.1 32 28.0 −6.5 11 49.1 −8.8 14 58.4 −11.7 117

800

Number of artifacts

TABLE V PASS RATES (%) UNDER THE SIMPLE IN - LOOP VERIFICATION (I N - LOOP ) AND THE LLM- BASED COMPLEMENTARY VERIFICATION (C OMPL . VERIF.). ∆ IS THE CHANGE IN PERCENTAGE POINTS ; FP COUNTS ARTIFACTS PASSED BY THE FORMER BUT REJECTED BY THE LATTER . N O FALSE NEGATIVES WERE FOUND . N INE LASSI RUNS WITH K IMI -K2.7-C ODE PRODUCED NO OUTPUT AND ARE EXCLUDED .

704

600 400 150

200 15

42

<0.7

0.7–0.8

88

0 0.8–0.9

0.9–1.0

1.0

Verifier confidence score Fig. 3. Distribution of the verifier’s confidence scores over the 999 artifacts checked by the complementary verification. The median confidence is 0.95, and 79.3% of the judgments carry a confidence score of at least 0.9.

the stricter check our framework still passes 71.4%, 68.5%, and 86.3% of its artifacts with the three models, keeping D. Repair Loop Effectiveness Evaluation margins of 25.0%, 40.5%, and 37.2% over LASSI. In total, To isolate the contribution of the repair loop, Figure 2 follows 117 artifacts (11.7%) are false positives of the in-loop stage, six representative model and direction combinations through and Table VI groups them into eight escape modes led by the loop, plotting the cumulative token consumption (bars, left placeholder implementations (31 cases), numerical errors (28), axis) and the cumulative success rate (curve, right axis) after and skipped verification (25). The verifier is confident in most each round, where round 0 is the initial translation before any of its judgments: the median confidence is 0.95, 79.3% of the repair. The top row focuses on the hardest direction in Table IV, judgments score at least 0.9, and only 57 artifacts (5.7%) fall OpenMP→SYCL, whose three-model averages are the lowest below 0.8, so manual spot-checking of the uncertain remainder for both systems (70.8% for our framework and 21.1% for stays feasible. Combining the stricter pass rates with token LASSI), and shows it under all three models. All three start at consumption, our framework spends 46k to 54k tokens per or below 12.5%, and the success rate climbs steadily as the loop verified successful benchmark across the three models, against accumulates tokens, reaching 79.2%, 58.3%, and 75.0% within 112k to 184k for LASSI, a 2.1 to 3.9 times advantage in cost nine repairs: gains of 50.0% to 70.9% that come entirely from per verified success. verification-guided repair rather than from the first draft. The V. D ISCUSSION bottom row varies the direction for DeepSeek V4 Flash. On The experiments and the design process itself yield the the easier CUDA↔OpenMP pair the strong first drafts (56.7% and 33.3% solved before any repair) explain the pair’s low following findings about LLM-based translation of parallel token consumption in Table IV, yet the loop still adds 36.6% programs under verification-guided repair. and 62.5% to reach 93.3% and 95.8%; on CUDA→SYCL it 1) Skills make both translation and repair more tokenefficient. Analysis and memory-planning skills target adds 60.0% to reach 70.0%. In all six panels the success rate translation at an explicit parallelization plan rather than rises monotonically with the accumulated tokens, showing that a loosely matching retrieved example, and verificationthe repair loop, not the initial generation, accounts for most of classified feedback targets repair at the diagnosed defect the final correctness. rather than blind regeneration, keeping the repair share of E. LLM-based Complementary Verification the budget smaller across all models and directions. All success rates reported so far are judged by the simple 2) Translation difficulty is determined by the target in-loop verification stage, which compiles each artifact, runs model, not the source. Directions with the same target it, and checks its output. To assess how much these judgments cluster together in correctness, with low-resource SYCL can be trusted, we additionally applied the rigorous LLMconsistently the hardest target: current LLMs read lowbased complementary verification (Section III-D), backed by resource parallel code far better than they write it, which DeepSeek V4 Flash, to the cached artifacts of both systems. is also why retrieval offers little help on such directions. The verifier issues a pass or fail judgment together with a 3) The repair loop accounts for most of the final correctness, confidence score; Figure 3 shows the confidence distribution but returns diminish beyond roughly five rounds. Success over the 999 artifacts that produced output (1,008 runs in total, climbs monotonically from a low round-0 base on hard minus 9 LASSI runs with Kimi K2.7 Code where, due to directions and still adds large gains on easy pairs, with the complexity of the tasks, the model repeatedly failed to most gains arriving within five rounds; later rounds mostly complete its output within the API time limit). accumulate tokens. First-draft strength determines what a direction costs, not how many benchmarks eventually pass, Table V compares the pass rates under the two verification and failures that survive many rounds are systematic ones stages. The rigorous complementary verification lowers the pass that generous iteration caps cannot fix. rates of both systems, by 8.3% to 17.3% for our framework and by 6.5% to 19.1% for LASSI, and it finds no false negatives: 4) Verification and repair make the framework largely model-agnostic. Correctness and token consumption stay no artifact rejected in the loop is accepted by the verifier. Under

TABLE VI TAXONOMY OF THE 117 FALSE POSITIVES IDENTIFIED BY THE LLM- BASED COMPLEMENTARY VERIFICATION , WITH ONE REPRESENTATIVE CASE PER ESCAPE MODE . A LL EXAMPLES ARE DRAWN FROM D EEP S EEK V4 F LASH RUNS , WHICH CONTRIBUTE THE LARGEST SHARE OF FALSE POSITIVES . Escape mode Count (%) Placeholder / stub implementation 31 (26.5) Numerical error Verification skipped

28 (23.9) 25 (21.4)

Parameter mismatch Index / buffer / memory error RNG / seed issue Kernel not executed

12 (10.3) 9 (7.7) 4 (3.4) 2 (1.7)

Other

6 (5.1)

Representative case LASSI, CUDA→OpenMP, contract LASSI, OpenMP→CUDA, distort LASSI, CUDA→OpenMP, pointwise Ours, SYCL→CUDA, distort LASSI, OpenMP→CUDA, car Ours, SYCL→OpenMP, kiss LASSI, CUDA→SYCL, reverse2D —

Key evidence All-zero checksums; the kernel only zero-fills the output Per-channel maximum error of 255 PASS printed with reference validation removed Hard-coded K = 0.3 while the test argument is 0.001 RMSE of 61,218; channel dimension ignored One RNG seed per thread instead of one per element Kernel execution time reported as -nan —

in narrow bands across models while the retrieval-based baseline swings widely on both; the loop compensates for weaker first drafts, so framework-level mechanisms matter more than the choice of LLM. 5) Compiler and runtime checks are necessary but not sufficient. In-loop verification overestimates the success of both systems, with systematic escapes led by placeholder implementations, weakened validation, and silent numerical errors. The complementary verification adds no false negatives and preserves the margins between systems, and its calibrated confidence concentrates at the high end, leaving only a small uncertain tail for manual spot-checking. 6) Separating generation, failure classification, and repair keeps self-correction bounded. Verdict routing gives each repair skill a single failure mode and the context it needs, and the unified budget prevents runaway retries without penalizing clean runs; the same encapsulation makes the framework direction-agnostic.

Several challenges remain open. First, our evaluation is limited to kernel-level benchmarks; extending the framework to multi-file codebases and full applications will require crossfile analysis, build-system awareness, and translation state that persists across compilation units. Second, the LLMbased complementary verification can itself be strengthened, for example through refined evaluation rubrics, multi-verifier voting, and tighter coupling between confidence scores and targeted re-execution, so that the uncertain tail requiring manual inspection shrinks further. Third, correctness is only the first step toward practical porting: we plan to incorporate performance feedback into the repair loop so that translated programs are not only numerically correct but also performant, and to extend the direction-agnostic skill mechanism to further programming models such as HIP and Kokkos, where the lowresource difficulty observed for SYCL targets is likely to be even more pronounced.

VI. C ONCLUSION AND F UTURE W ORK

This research used the DeepBlizzard GPU HPC cluster at Michigan Technological University, which was funded by NSF Major Research Instrumentation (MRI) Grant Award Number 2215734, MRI: Acquisition of a GPU-accelerated cluster for research, training, and outreach.

This paper presented a multi-agent system for parallel scientific code translation, motivated by two gaps in existing pipelines: they verify correctness against criteria that do not match scientific computing practice, and they spend token budgets on blind, undifferentiated retries. Our framework separates translation, compilation diagnosis, and execution validation into three agents coordinated by a verdict-driven orchestrator, routes each diagnosed failure to a targeted repair skill, and treats automated output checking as a first-class validation step. On 30 HeCBench kernels translated bidirectionally among OpenMP, CUDA, and SYCL with three recent open-source LLMs, our framework attains the higher verified correctness rate in all 18 model and direction combinations over LASSI, with per-model averages of 78.6% to 94.6% against 34.5% to 65.5%, while consuming 29.5% to 52.4% fewer tokens. Under the stricter LLM-based complementary verification, it preserves margins of 25.0% to 40.5% at a 2.1 to 3.9 times advantage in cost per verified success. The experiments further show that the repair loop accounts for most of the final correctness; verificationguided repair makes the framework largely model-agnostic; and compiler and runtime checks alone systematically overestimate translation success. More capable translation behavior, in other words, comes not from more expensive prompting but from better verification scaffolding around the model.

ACKNOWLEDGMENT

R EFERENCES [1] OpenAI, “GPT-4 technical report,” March 2023. [2] D. Guo, Q. Zhu, D. Yang, Z. Xie, K. Dong, W. Zhang, G. Chen, X. Bi, Y. Wu, Y. K. Li, F. Luo, Y. Xiong, and W. Liang, “DeepSeek-Coder: When the large language model meets programming — the rise of code intelligence,” January 2024. [3] B. Hui, J. Yang, Z. Cui, J. Yang, D. Liu, L. Zhang, T. Liu, J. Zhang, B. Yu, K. Lu, K. Dang, Y. Fan, Y. Zhang, A. Yang, R. Men, F. Huang, B. Zheng, Y. Miao, S. Quan, Y. Feng, X. Ren, X. Ren, J. Zhou, and J. Lin, “Qwen2.5-Coder technical report,” November 2024, qwen Team, Alibaba Group. [4] J. Jiang, F. Wang, J. Shen, S. Kim, and S. Kim, “A survey on large language models for code generation,” ACM Transactions on Software Engineering and Methodology, vol. 35, no. 2, pp. 1–72, 2026. [5] S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. Narasimhan, and Y. Cao, “ReAct: Synergizing reasoning and acting in language models,” in The Eleventh International Conference on Learning Representations (ICLR 2023), Kigali, Rwanda, May 2023. [Online]. Available: https://openreview.net/forum?id=WE vluYUL-X [6] M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. d. O. Pinto, J. Kaplan, H. Edwards, Y. Burda, N. Joseph, G. Brockman, A. Ray, R. Puri, G. Krueger, M. Petrov, H. Khlaaf, G. Sastry, P. Mishkin, B. Chan, S. Gray, N. Ryder, M. Pavlov, A. Power, L. Kaiser, M. Bavarian, C. Winter, P. Tillet, F. P. Such, D. Cummings, M. Plappert, F. Chantzis, E. Barnes, A. Herbert-Voss, W. H. Guss, A. Nichol, A. Paino, N. Tezak, J. Tang, I. Babuschkin, S. Balaji, S. Jain, W. Saunders, C. Hesse, A. N. Carr,

J. Leike, J. Achiam, V. Misra, E. Morikawa, A. Radford, M. Knight, M. Brundage, M. Murati, K. Mayer, P. Welinder, B. McGrew, D. Amodei, S. McCandlish, I. Sutskever, and W. Zaremba, “Evaluating large language models trained on code,” July 2021. [7] S. Ren, D. Guo, S. Lu, L. Zhou, S. Liu, D. Tang, N. Sundaresan, M. Zhou, A. Blanco, and S. Ma, “CodeBLEU: A method for automatic evaluation of code synthesis,” September 2020. [8] G. Gopalakrishnan, P. D. Hovland, C. Iancu, S. Krishnamoorthy, I. Laguna, R. A. Lethin, K. Sen, S. F. Siegel, and A. Solar-Lezama, “Report of the HPC correctness summit, jan 25–26, 2017, washington, DC,” May 2017. [9] E. Kaplan, T. Bitan, L. Ghrayeb, L. Chen, T. Yotam, N. Hasabnis, and G. Oren, “ParaCodex: A profiling-guided autonomous coding agent for reliable parallel code generation and translation,” January 2026. [10] OpenMP Architecture Review Board, OpenMP Application Programming Interface Specification, Version 5.2, OpenMP Architecture Review Board, November 2021. [Online]. Available: https://www.openmp.org/wp-conte nt/uploads/OpenMP-API-Specification-5-2.pdf [11] NVIDIA Corporation. (2024) CUDA C++ programming guide. Version 12.x. [Online]. Available: https://docs.nvidia.com/cuda/cuda-c-program ming-guide/ [12] Khronos Group, SYCL 2020 Specification, Revision 9, Khronos Group, 2023, revision 9. [Online]. Available: https://registry.khronos.org/SYCL/ specs/sycl-2020/pdf/sycl-2020.pdf [13] M. T. Dearing, Y. Tao, X. Wu, Z. Lan, and V. Taylor, “LASSI: An LLMbased automated self-correcting pipeline for translating parallel scientific codes,” in 2024 IEEE International Conference on Cluster Computing Workshops (CLUSTER Workshops). IEEE, 2024, pp. 136–143. [14] A. TehraniJamsaz, A. Bhattacharjee, L. Chen, N. K. Ahmed, A. Yazdanbakhsh, and A. Jannesari, “CodeRosetta: Pushing the boundaries of unsupervised code translation for parallel programming,” in Advances in Neural Information Processing Systems, vol. 37. Curran Associates, Inc., 2024. [Online]. Available: https://proceedings.neurips. cc/paper files/paper/2024/hash/b6edb87876bec4ac2260bffa083cb992-A bstract-Conference.html [15] T. Bitan, T. Kadosh, E. Kaplan, S. Meiri, L. Chen, P. Morales, N. Hasabnis, and G. Oren, “UniPar: A unified LLM-based framework for parallel and accelerated code translation in HPC,” in 2025 IEEE High Performance Extreme Computing Conference (HPEC). Wakefield, MA, USA: IEEE, September 2025, pp. 1–9. [16] R. Xu and Y. Yan, “Agent skills for large language models: Architecture, acquisition, security, and the path forward,” February 2026. [17] T. Abuelsaad, D. Akkil, P. Dey, A. Jagmohan, A. Vempaty, and R. Kokku, “Agent-E: From autonomous web navigation to foundational design principles in agentic systems,” July 2024. [18] X. Li, “When single-agent with skills replace multi-agent systems and when they fail,” January 2026. [19] Z. Jin and J. S. Vetter, “A benchmark suite for improving performance portability of the SYCL programming model,” in 2023 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS). Raleigh, NC, USA: IEEE, 2023, pp. 325–327. [Online]. Available: https://ieeexplore.ieee.org/document/10158214 [20] P. Shu, J. Chen, Z. Liu, H. Zhao, X. Li, and T. Liu, “Survey of HPC in US research institutions,” June 2025. [21] E. Strohmaier, J. Dongarra, H. Simon, and M. Meuer. TOP500 supercomputer sites. [Online]. Available: https://www.top500.org [22] Advanced Micro Devices, Inc. (2016) HIP: C++ heterogeneous-compute interface for portability. [Online]. Available: https://github.com/ROCm/ hip [23] C. R. Trott, D. Lebrun-Grandie, D. Arndt, J. Ciesko, V. Dang, N. Ellingwood, R. Gayatri, E. Harvey, D. S. Hollman, D. Ibanez, N. Liber, J. Madsen, J. Miles, D. Poliakoff, A. Powell, S. Rajamanickam, M. Simberg, D. Sunderland, B. Turcksin, and J. Wilke, “Kokkos 3: Programming model extensions for the exascale era,” IEEE Transactions on Parallel and Distributed Systems, vol. 33, no. 4, pp. 805–817, April 2022. [24] J. H. Davis, P. Sivaraman, J. Kitson, K. Parasyris, H. Menon, I. Minn, G. Georgakoudis, and A. Bhatele, “Taking GPU programming models to task for performance portability,” in Proceedings of the 39th ACM International Conference on Supercomputing, ser. ICS ’25. Salt Lake City, UT, USA: Association for Computing Machinery, June 2025. [25] S. Matsuoka, J. Domke, M. Wahib, A. Drozd, R. Bair, A. A. Chien, J. S. Vetter, and J. Shalf, “Preparing for the future — rethinking proxy apps,” April 2022.

[26] Intel Corporation, “Syclomatic developer guide and reference,” https: //oneapi-src.github.io/SYCLomatic, 2024. [27] A. T. Nguyen, T. T. Nguyen, and T. N. Nguyen, “Lexical statistical machine translation for language migration,” in Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering, ser. ESEC/FSE 2013. Saint Petersburg, Russia: Association for Computing Machinery, August 2013, pp. 651–654. [28] S. Karaivanov, V. Raychev, and M. Vechev, “Phrase-based statistical translation of programming languages,” in Proceedings of the 2014 ACM International Symposium on New Ideas, New Paradigms, and Reflections on Programming & Software, ser. Onward! 2014. Portland, OR, USA: Association for Computing Machinery, October 2014, pp. 173–184. [29] A. T. Nguyen, T. T. Nguyen, and T. N. Nguyen, “Divide-and-conquer approach for multi-phase statistical migration for source code,” in 2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE). Lincoln, NE, USA: IEEE, November 2015, pp. 585–596. [30] B. Rozière, M.-A. Lachaux, L. Chanussot, and G. Lample, “Unsupervised translation of programming languages,” in Advances in Neural Information Processing Systems 33, ser. NeurIPS 2020. Curran Associates, Inc., 2020, pp. 20 601–20 611. [Online]. Available: https://proceedings.neurips.cc/paper/2020/hash/ed23fbf18c2cd35f8c7f8 de44f85c08d-Abstract.html [31] B. Rozière, J. M. Zhang, F. Charton, M. Harman, G. Synnaeve, and G. Lample, “Leveraging automated unit tests for unsupervised code translation,” in The Tenth International Conference on Learning Representations (ICLR 2022), Virtual Event, April 2022. [Online]. Available: https://openreview.net/forum?id=cmt-6KtR4c4 [32] M. Szafraniec, B. Rozière, H. Leather, F. Charton, P. Labatut, and G. Synnaeve, “Code translation with compiler representations,” in The Eleventh International Conference on Learning Representations (ICLR 2023), Kigali, Rwanda, May 2023. [Online]. Available: https://openreview.net/forum?id=XomEU3eNeSQ [33] B. Lei, C. Ding, L. Chen, P.-H. Lin, and C. Liao, “Creating a dataset for high-performance computing code translation using LLMs: A bridge between OpenMP Fortran and C++,” in 2023 IEEE High Performance Extreme Computing Conference (HPEC). Dedham, MA, USA: IEEE, September 2023. [34] M. T. Dearing, Y. Tao, X. Wu, Z. Lan, and V. Taylor, “Leveraging LLMs to automate energy-aware refactoring of parallel scientific codes,” arXiv preprint arXiv:2505.02184, 2025. [35] J. H. Davis, D. Nichols, I. Khillan, and A. Bhatele, “ParEval-Repo: A benchmark suite for evaluating LLMs with repository-level HPC translation tasks,” in Proceedings of the 54th International Conference on Parallel Processing, ser. ICPP ’25. Association for Computing Machinery, September 2025, pp. 94–103. [36] J. Yang, C. E. Jimenez, A. Wettig, K. Lieret, S. Yao, K. Narasimhan, and O. Press, “SWE-agent: Agent-computer interfaces enable automated software engineering,” in Advances in Neural Information Processing Systems, vol. 37. Curran Associates, Inc., 2024, pp. 50 528–50 652. [Online]. Available: https://proceedings.neurips.cc/paper files/paper/202 4/hash/5f4f82b2df4a38e69043d5024b93e6c4-Abstract-Conference.html [37] Y. Zhang, H. Ruan, Z. Fan, and A. Roychoudhury, “AutoCodeRover: Autonomous program improvement,” in Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis, ser. ISSTA 2024. Vienna, Austria: Association for Computing Machinery, September 2024, pp. 1592–1604. [38] I. Bouzenia, P. Devanbu, and M. Pradel, “RepairAgent: An autonomous, LLM-based agent for program repair,” in Proceedings of the IEEE/ACM 47th International Conference on Software Engineering, ser. ICSE ’25. Ottawa, Ontario, Canada: IEEE Press, April 2025, pp. 2188–2200. [39] H. Gray, T. Yotam, and O. Udrea, “Workflows vs agents for code translation,” in Deep Learning for Code (DL4C) Workshop, NeurIPS 2025, December 2025. [Online]. Available: https://openreview.net/forum ?id=U1emjvJ7Gi [40] DeepSeek-AI, “Deepseek-v4: Towards highly efficient million-token context intelligence,” arXiv preprint arXiv:2606.19348, 2026. [Online]. Available: https://arxiv.org/abs/2606.19348 [41] Z.ai, “Glm-5.2: Built for long-horizon tasks,” https://z.ai/blog/glm-5.2, 2026, accessed: 2026-08-04. [42] M. AI, “Kimi k2.7 code: Open-source agentic coding model,” https: //www.kimi.com/resources/kimi-k2-7-code, 2026, accessed: 2026-08-03.

Appendix: Artifact Description/Artifact Evaluation Artifact Description (AD) VII. OVERVIEW OF C ONTRIBUTIONS AND A RTIFACTS A. Paper’s Main Contributions C1 A multi-agent framework for automated HPC code translation across CUDA, OpenMP, and SYCL, comprising a TranslationAgent, a CompilationAgent, and an ExecutionAgent coordinated by a verdict-driven Orchestrator. C2 A set of repair skills (Patch, Semantic, Sync, Memory) that detect and fix compilation, runtime, and correctness errors during translation, enabling iterative self-correction without human intervention. C3 An evaluation of the framework on 30 HeCBench benchmarks across six translation directions and three LLM backends (DeepSeek V4 Flash, GLM 5.2, Kimi K2.7 Code), demonstrating higher verified correctness and lower token consumption than the LASSI baseline. B. Computational Artifacts A1 https://github.com/dinglhmds/More-with-Less-Efficient-P arallel-Scientific-Code-Translation-via-an-Agent-Skill -Workflow Artifact ID A1

Contributions Supported C1 , C2 , C3

Related Paper Elements Tables IV–VI Figures 2–3

VIII. A RTIFACT I DENTIFICATION A. Computational Artifact A1 Relation To Contributions The artifact implements the full multi-agent translation pipeline described in C1 and C2 , including the Orchestrator, all three agents, all repair skills, and the offline LLM-based complementary verifier. It also contains the LASSI baseline (configured with the same three LLMs), the batch runner, and the evaluation scripts used to produce the results in C3 . Expected Results Running the batch evaluation on the 30 HeCBench benchmarks (six translation directions, three LLMs, both systems) should produce per-benchmark verdicts (success/failure), token consumption reports, cumulative success-rate curves, and complementary-verification judgments matching Tables IV– VI and Figures 2–3 in the paper. The framework is expected to achieve a higher verified correctness rate than the LASSI baseline while consuming fewer LLM tokens on average. Expected Reproduction Time (in Minutes) Artifact Setup: approximately 30 minutes (install Python dependencies and configure LLM API access). Artifact Execution: approximately 8–12 hours for all 30 benchmarks on a GPU cluster (dominated by LLM API latency). Artifact Analysis: approximately 15 minutes (aggregate per-benchmark result.json files into summary tables and plots).

Artifact Setup (incl. Inputs) Hardware: A GPU cluster managed by Slurm with at least one NVIDIA GPU node is required for executing CUDA baselines and OpenMP/SYCL offload targets. CPUonly OpenMP execution is also supported as a fallback. The evaluation used a Slurm cluster with the mrigpu partition (one GPU per job). Software: • Python 3.9 or newer: https://www.python.org • OpenAI Python client (≥1.0): https://github.com/openai/ openai-python • PyYAML (≥6.0): https://pyyaml.org • python-dotenv (≥1.0): https://github.com/theskumar/pyt hon-dotenv • g++ 8.5.0: https://gcc.gnu.org • CUDA Toolkit 12.8 (nvcc): https://developer.nvidia.com/c uda-toolkit • Intel oneAPI DPC++/C++ Compiler 2021.4.0 (icpx/dpcpp): https://www.intel.com/content/ww w/us/en/developer/tools/oneapi/dpc-compiler.html • Clang/LLVM 18.1.8 (optional, for OpenMP GPU offload): https://clang.llvm.org • Access to an OpenAI-compatible LLM API (DeepSeek, GLM, Kimi, or equivalent) Datasets / Inputs: The evaluation uses 30 benchmarks selected from HeCBench (https://github.com/zjin-lcf/HECBench), covering image processing, deep learning, mathematical and statistical kernels, sorting and parallel primitives, stencil and numerical methods, hashing, physics simulation, and signal processing. The selected benchmark names are listed in DEFAULT_BENCHMARKS in batch_runner.py; the suite itself is obtained by cloning the HeCBench repository and passing its src directory to the batch runner via --benchmark-dir. Six of the selected kernels lack OpenMP implementations, leaving 24 benchmarks per OpenMP-source direction. No other external datasets are required. Installation and Deployment: Clone the repository, create a Python virtual environment, and install dependencies: python3 -m venv .venv source .venv/bin/activate pip install -r requirements.txt Configure the LLM API endpoint by copying .env.example to .env and setting BASE_URL, API_KEY, and MODEL_NAME. Set environment variables for compilation and execution: export CT_CUDA_PATH=/path/to/cuda/12.8 # optional, for OpenMP GPU offload: export CT_CLANG_HOME=/path/to/llvm/18.1.8 export LD_LIBRARY_PATH=\ $CT_CLANG_HOME/lib:$LD_LIBRARY_PATH export CT_SLURM_PARTITION=mrigpu export CT_SLURM_GRES=gpu:1

Artifact Execution Artifact Evaluation (AE) The experiment workflow consists of four tasks: T1 (transla- A. Computational Artifact A 1 tion) → T2 (compilation/execution) → T3 (result aggregation) → T4 (offline complementary verification). The full evaluation Artifact Setup (incl. Inputs) Access. The artifact is available at https://github.com/dingl matrix comprises two systems (our framework and the LASSI hmds/More-with-Less-Efficient-Parallel-Scientific-Code-Tra baseline), three LLMs, and six translation directions; individual runs are independent and can be distributed across Slurm jobs. nslation-via-an-Agent-Skill-Workflow. Reviewers are granted T1 : The batch runner (batch_runner.py) discovers access to an OpenAI-compatible LLM endpoint; the endpoint benchmarks under the HeCBench source directory, reads URL and an API key with sufficient quota are provided through each source program, and runs the multi-agent pipeline. the submission system. Hardware. One NVIDIA GPU node on a Slurm-managed The Orchestrator coordinates Analyzer, MemoryPlanning, and Translator skills to produce target code, then invokes the cluster (one GPU per job), as described in the AD Appendix. Software. Python 3.9+ with the packages listed in CompilationAgent and the ExecutionAgent. If compilation or execution fails, repair skills mutate the code and the pipeline re- requirements.txt (OpenAI client, PyYAML, pythonenters from the corresponding stage. Each benchmark is capped dotenv); g++ 8.5.0, CUDA Toolkit 12.8, and Intel oneAPI at 300 seconds for compilation, 120 seconds for execution, and DPC++/C++ Compiler 2021.4.0. Exact installation commands and environment variables are given in the AD Appendix a maximum of 9 repair iterations. T2 : Each source program is compiled with its native (Artifact Setup) and in the repository README.md. compiler (nvcc -O3 -std=c++17 for CUDA, g++ -O2 Artifact Execution -fopenmp -std=c++17 for OpenMP, icpx -fsycl To stay within the evaluation time budget, we define a -std=c++17 for SYCL) and executed on a Slurm reduced but representative evaluation: the CUDA→OpenMP GPU node to obtain reference outputs. Translated programs are compiled with the corresponding target com- direction with DeepSeek V4 Flash, all 30 benchmarks, run for piler (for OpenMP GPU offload, clang++ -O3 -fopenmp both our framework and the LASSI baseline. This reproduces -fopenmp-targets=nvptx64 -std=c++17) and exe- one column pair of Table IV and the bottom-left panel of Figure 2. The model is selected via MODEL_NAME in .env. cuted with the same input arguments. E1 (translation with self-correction): for our framework, run T3 : Per-benchmark results are written to output/<timestamp>/ export CT_TRANSLATION_DIRECTION=\ <benchmark>/result.json, containing the verdict, cuda_to_openmp retry counts, elapsed time, and token usage broken down by python batch_runner.py \ category and repair round. A summary script aggregates these --benchmark-dir <hecbench>/src \ files into the tables and figures reported in the paper. --output batch_cuda2omp_ae.json \ T4 : The offline LLM-based complementary verifier is applied --no-context --no-rag \ to the cached artifacts of both systems. Given the complete --use-memory-plan \ source program and the execution outputs of a translated --auto-scale-on-timeout \ program, it produces a structured JSON judgment (pass/fail plus --max-retries 5 a confidence score in [0, 1]) following the rubric described in the paper. Aggregating these judgments yields the complementary- and for the LASSI baseline, run verification pass rates in Table V, the false-positive taxonomy python experiments/lassi/\ in Table VI, and the confidence distribution in Figure 3. run_lassi30_cuda2omp_context.py Artifact Analysis (incl. Outputs) (On a Slurm cluster, the equivalent submission scripts are The expected output is a set of per-benchmark scripts/submit_framework_cuda_to_openmp_ result.json files, per-artifact verification judgments, and norag_nocontext.sh and scripts/submit_lassi a summary report. The key metrics are: (1) the verified _cuda2omp_context.sh; adapt the cluster-specific paths correctness rate (the EQUIVALENT verdict, routed to the before use.) For each benchmark this executes tasks T1 and terminal Done state), (2) token consumption aggregated T2 of the AD workflow: the verdict-driven translation loop, by direction and model, (3) the cumulative success rate as compilation with g++ -O2 -fopenmp -std=c++17, and a function of repair rounds, and (4) the complementary- execution on the GPU node, capped at 300 s compilation, 120 s verification pass rates and confidence distribution. These execution, and 9 repair iterations. E2 (aggregation): run correspond to Tables IV–VI and Figures 2–3 in the paper. python tools/\ framework_model_direction_summary.py python tools/framework_round_summary.py which aggregate all result.json files into a summary table and the cumulative success-rate data (task T3 ).

Optionally, E3 (complementary verification): run python tools/run_llm_verdict_full.py to apply the offline LLM-based verifier (task T4 ) to the cached artifacts, producing per-artifact pass/fail judgments with confidence scores. Artifact Analysis (incl. Outputs) Expected results. E1 produces 30 result.json files per system; E2 prints a table with the verified correctness rate and total token consumption per system, together with the cumulative success-rate curve over repair rounds. Evaluation methodology. Compare the summary table against the CUDA→OpenMP columns of Table IV: our framework should achieve a higher verified correctness rate than LASSI (paper: 93.3% vs. 86.7%) at lower token consumption (paper: 718k vs. 1,327k). Because LLM outputs are nondeterministic, absolute rates may shift by a few percentage points between runs; the reproduction criterion is that the ordering of the two systems and the approximate split of tokens between translation and repair are preserved. The cumulative curve should increase monotonically, matching the bottom-left panel of Figure 2. Correlation with contributions. Successful completion of the verdict-driven loop, including repair invocations, substantiates C1 and C2 ; the correctness and token comparison between the two systems substantiates C3 . If E3 is executed, the verifier’s judgments additionally provide the stricter complementary pass rates underlying Table V.

Related documents

• HPCAgent-Bench: A Framework for Sound Benchmarking of Code-Optimizing Agents, Harnesses, and Skills#1355662
• HPCAgent-Bench: A Framework for Sound Benchmarking of Code-Optimizing Agents, Harnesses, and Skills#1348926
• MiniMaxAI/OctoCodingBench#786635
• Bridging the Gap on AI-Assisted Scientific Software Development Through Transparency and Traceability#200554
• Transformed in Translation: Two-Stage Structural Uncertainty in LLM-Based Scientific Autoformalization#1084729