Latent Reasoning Guidance for Parallel Code Translation Tomer Bitan1 Erel Kaplan1 Roee Bar-Yadin1 Lian Ghrayeb1 Le Chen2 Samyak Jhaveri3 Niranjan Hasabnis4 Gal Oren1,5 1 Technion 2 Argonne National Laboratory 3 University of California, Irvine 4 Code Meta 5 Stanford University
Abstract
2023a). Parallel API translation is therefore an important but difficult code-generation problem because translations must preserve the source computation while satisfying target-specific requirements for parallel structure, memory movement, synchronization, and API usage (Jin and Vetter, 2023; Bitan et al., 2025). Large language models and autonomous code agents can propose flexible transformations beyond rigid source-to-source rules (Kadosh et al., 2023b; Chen et al., 2024a; Kaplan et al., 2026), but in executable parallel-code translation, even plausible looking outputs may fail to compile, crash, or violate reference behavior, pushing many systems toward generate→test→repair loops. Small differences in intermediate reasoning can lead to different compile→run→validate outcomes, and poor high-level parallelization choices can be hard to repair after decoding. This motivates an earlier intervention point: can nearby reasoning trajectories lead to better executable outcomes before any token-level repair is attempted? Existing guidance methods typically score decoded programs, textual reasoning traces, or completed candidates (Cobbe et al., 2021; Lightman et al., 2023; Li et al., 2025; Dai et al., 2024). Latent reasoning offers an earlier intervention point by shifting some test-time computation into continuous hidden-state trajectories before standard decoding resumes (Hao et al., 2025; Xu et al., 2025; Yue et al., 2025; Zou et al., 2025). However, most latent-reasoning systems still follow a single hidden trajectory and do not compare nearby alternatives before the final decoding. We study a test-time latent guidance method for executable parallel API translation that keeps the primary model frozen, samples perturbed hiddenstate branches during latent reasoning, scores partial latent prefixes with a smaller PRM, and resumes generation from the highest-scoring branch, placing task-specific adaptation in the compact
arXiv:2606.05518v1 [cs.DC] 3 Jun 2026
Tackling complex coding tasks often requires autonomous agents and iterative repair pipelines. These increasingly rely on large amounts of test-time computation, often spending many decoding and repair steps before discovering whether a program compiles, runs, or validates. Executable parallel-code translation is an effective setting for earlier guidance because success is behavioral rather than textual. However, most guidance methods act only after complete programs or textual traces are decoded. This motivates the question: can latent reasoning provide an earlier intervention point, before the model commits to code? We study a test-time latent guidance method for this setting that trains a smaller Process Reward Model (PRM) over continuous latent prefixes and uses it to select among alternate hidden-state trajectories before final code decoding, separately from but compatible with post-decoding optimization. On a 76-task ParaTrans benchmark evaluation, latent PRM guidance improves mean validation rate from 32.89% with unguided latent reasoning to 42.1%, outperforming fine-tuned and vanilla baselines in the same setting. These gains persist under the same three-iteration repair loop. These results provide bounded evidence that useful alternative latent continuations exist and that PRM-scored latent branch selection can improve executable outcomes in this setting without retraining the main generative model. 1
1
Introduction
Scientific and high-performance applications often need to move between programming models such as CUDA, OpenMP, OpenCL, and serial CPU implementations as hardware, compiler support, and deployment constraints change (Kadosh et al., 1
GitHub:https://github.com/ Scientific-Computing-Lab/Parallax.git
1
put unembedding matrix Wout , we compute
guidance model rather than retraining the generator. Related work and positioning are discussed in Appendix A. Empirically, latent PRM guidance improves ParaTrans validation by 9.21 points over unguided latent reasoning, and the gain persists when inserted into the same three-attempt repair loop. A held-out branch-selection test further suggests that the PRM can learn a weak but useful local preference signal to avoid undesirable latent continuations. We make three contributions. First, we identify pre-decoding latent branch selection as a distinct intervention point for executable parallel-code translation. Second, we instantiate this intervention with a smaller PRM and evaluate it on ParaTrans, where correctness requires compilation, execution, and validation rather than textual similarity. Third, we provide bounded mechanism-level evidence from a held-out branch-selection test that a smaller PRM can provide a conservative local preference signal over continuous latent branches before final code decoding.
2
⊤ ⊤ Wa = (Wout Wout )−1 Wout Win ,
the closed-form solution to min ∥Wout Wa − Win ∥2F . Wa
At latent step t, the aligned continuous input for the next step is zt+1 = ht Wa . We use K = 6 latent steps when collecting PRM training trajectories and K = 12 latent steps during inference. 2.2
We train a smaller reward model to estimate the potential of a partial latent trajectory of the reasoning process via the mean terminal quality score. The PRM uses a Qwen-Coder-7B backbone with a scalar value head. Because the PRM receives hidden states from the primary LLaMA-3.3-70B model rather than its own token embeddings, we prepend a learned adapter that maps this hidden dimension to the PRM embedding dimension. In our implementation, this maps 8192-dimensional generator states to the 3584-dimensional Qwen-Coder input space using a linear layer, GELU activation, and layer normalization. For every latent reasoning step t, the PRM Vθ receives the prefix trajectory
Method
We propose a test-time optimization method for guiding latent reasoning without fine-tuning the main generator in parallel code translation. Specifically, we keep a large latent-reasoning primary model frozen and train a smaller PRM to score partial latent trajectories before text decoding, as shown in Figure 1. 2.1
Latent process reward model
τ≤t = (h1 , . . . , ht ) and predicts a scalar value Vθ (τ≤t ). The target is the empirical terminal reward obtained from rollout sampling. For rollout supervision at each latent step t, we sample Btrain = 3 candidate branch states: two perturbed resamples and the unperturbed hidden state.
Latent-reasoning Primary Model
Let M be a frozen autoregressive model equipped with continuous latent reasoning. In our main experiments, M is LLaMA-3.3-70B. Following the COCONUT-style latent-reasoning paradigm (Hao et al., 2025), the model performs a fixed number of latent steps before returning to standard token decoding. Let ht ∈ RdM denote the primary model’s hidden state at latent step t, before the languagemodeling head. During the latent phase, ht is not decoded into a discrete token; instead, it is mapped back into the M input space and used as the next continuous input. Instead of training the main model in the manner proposed in Hao et al. (2025), we use the trainingfree alignment transform of Zou et al. (2025). Given the input embedding matrix Win and out-
(b)
h̃t = ht + ϵ(b) ,
ϵ(b) ∼ N (0, 1.42 I).
(1)
For each branch b, we roll out to terminal code (b) yt and score the completed program with an automatic reward function S(·). The prefix target is the average terminal score of its continuations: Rt =
1 Btrain
BX train
(b)
S(yt ),
b=1
The PRM is trained with mean squared error: N
1 X (i) (i) 2 L= Vθ (τ≤t ) − Rt . N i=1
2
PRM Training
Inference
Verified Results
ParaTrans Dataset
Serial
CUDA
Serial
OMP
OMP
CUDA
CUDA
OMP
Final Translated Code
Guided Original Suboptimal
Source → Target Code Translation
Large ❄ Frozen Language Model
✓ compile
</>
Large ❄ Frozen Language Model 0.21 0.21 0.81
0.21 0.21 0.39
0.21 0.21 0.78
0.21 0.21 0.92
0.21 0.21 0.23
0.21 0.21 0.89 0.21
0.65
0.95
0.79
0.47
Selected
0.94
0.65
0.89
0.89
0.52
0.68
0.68
0.91
0.29
0.56
0.12
0.65
0.65
0.79
0.79
0.94
0.94
0.89
0.89
0.68
0.68
0.91
0.91
0.56
0.56
Process Reward Model
h0
✓z run
✓
0.47 Original
✓ validate
0.52 0.65
Validation Rate
0.18
Step 1
Step 2
Reasoning traces (candidate paths)
Step 3
Step 4
Step 5
Step 6
Original
32.9%
0.24
Selected
42.1%
Latent Reasoning Steps
Figure 1: Latent PRM-guided translation pipeline. Executable feedback is converted into a pre-decoding guidance signal for parallel-code translation. (1) PRM training uses ParaTrans paired tasks across Serial, CUDA, and OpenMP: a frozen latent-reasoning generator samples candidate hidden-state paths, decodes them, and assigns supervision from executable outcomes. (2) The process reward model learns to score partial latent branches before code is emitted. (3) During inference, a new source task is processed by the frozen generator; candidate latent branches are proposed at each step, scored by the PRM, and reduced to the highest-scoring trajectory before final decoding. (4) Verified results are obtained by compiling, running, and validating the translated code against the reference behavior. This design tests whether latent trajectories can be steered before decoding while leaving the main generator frozen.
2.3
Training data and reward construction
et al. (2026). Final benchmark evaluation uses executable validation, not this mixed training reward. Additional reward and hardware details appear in Appendix C.
We construct the PRM training set from primarymodel reasoning trajectories on ParaTrans samples used by Bitan et al. (2025) and supplement with additional samples created with the same methodology, balanced across CUDA→OpenMP, OpenMP→CUDA, Serial→OpenMP, and Serial→CUDA. We use 60 development samples for hyperparameter selection and train the final PRM on the combined training and development set before evaluating on the held-out test tasks, as shown on the left side of Figure 1. The terminal reward S(y) combines executable and semantic signals, providing denser supervision than executable feedback alone. Compilation and execution receive fixed weights of 0.30 and 0.25, respectively. The remaining 0.45 weight is distributed across the validators available for the sample, proportional to their base weights. These validators include pass/fail validation, checksum or output matching when available, benchmark metrics, code-embedding similarity (Günther et al., 2024), an LLM-as-a-judge score, and a variable-comparison heuristic inspired by Kaplan
2.4
PRM-guided latent inference
At inference time, we perform greedy branch selection during the latent reasoning phase, as shown in the inference panel of Figure 1. For every latent step t, we score Btest = 8 candidate branch states: the unperturbed state ht and seven perturbed variants from Eq. 1. Each candidate forms a possible branch (b)
(b)
τ≤t = (h1 , . . . , ht−1 , h̃t ). The PRM scores each candidate branch and selects (b)
b⋆t = arg max Vθ (τ≤t ). b
Only the selected branch is retained. The selected (b⋆ ) hidden state h̃t t is aligned with Wa and fed back into the primary model for the next latent step. After K = 12 latent steps, the primary model returns to standard autoregressive decoding and produces 3
Table 1: Three-run mean executable validation rate (%) on the ParaTrans test set, with and without the same three-iteration repair loop. Method
No repair
+3-attempt repair
Vanilla Fine-tuned Latent reasoning Random branch selection Latent PRM guidance
14.55±2.55 33.48±1.11 32.89±3.44 27.28±6.34 42.10±2.28
30.67±4.34 36.00±3.11 36.40±5.47 – 45.18±3.31
Table 2: Held-out branch-selection results on 500 instances. “Best” is strict best-branch accuracy; > Orig. is strict improvement over the original continuation, and ≥ Orig. is at least as good as the original. Selection rate (%) Selector Original Random Latent PRM
the final target code. While initial training is done with K = 6 latent steps for efficiency, we find that using more latent steps at inference can further improve performance, as shown in Appendix C.
3
Best
> Orig.
≥ Orig.
33.66 32.14±0.46 36.25±0.41
43.0±0.4 45.0±0.2
66.9±0.3 79.2±0.3
the notion that pre-decoding guidance remains useful alongside later token-level repair. Appendix B reports the corresponding test-time overhead comparison, task-level win/loss/tie counts, and a postdecoding best-of-8 ablation.
Experimental Setup
4.2
The main evaluation uses the 76-task ParaTrans test set, where a translation counts as successful only if it compiles, runs, and passes integrated validation against the reference behavior. We compare vanilla LLaMA-3.3-70B, the fine-tuned LLaMA-3.3-70B baseline from Bitan et al. (2025), unguided latent reasoning with the same frozen base model, random latent branch selection with the same perturbation budget, and latent PRM guidance. Appendix C gives the full evaluation protocol, including direction counts, overlap checks, the repair setting, and the held-out branch-selection benchmark; directionwise results appear in Appendix B.
Latent-trajectory branch analysis
4
Results
We test whether perturbed latent continuations can improve on the unperturbed trajectory. On the 952sample PRM training set, 95% of trajectory samples contain at least one candidate branch with higher terminal reward than the original. 59% of candidate continuations improve over their corresponding unperturbed branch. Among samples with at least one trajectory that passes both validation and output comparison, 25% succeed only through branching: the original trajectory fails, while at least one perturbed continuation succeeds. These results show that better latent continuations often exist locally, though distinguishing them from harmful ones remains difficult.
4.1
Main executable translation results
4.3
Branch-selection analysis
The branch-selection test isolates whether the PRM can choose a useful latent branch from a fixed local candidate set. To avoid leakage, this PRM excludes the 120 training trajectories used to construct the 500 branch-selection instances. Table 2 provides bounded support for the mechanism behind latent guidance. Relative to random selection, the PRM more often selects a branch that improves over the original continuation or at least preserves it. While only modestly improving strict best-branch accuracy, this local advantage is applied repeatedly across consecutive latent steps and multiple candidate branches, where choices can compound before decoding begins. Together with the poor random-branch result in Table 1, this suggests that the PRM mainly helps by filtering harmful latent perturbations rather than by reliably finding the single best branch.
Table 1 reports executable validation rates on ParaTrans. The key comparison is between unguided latent reasoning and latent PRM guidance, since both use the same frozen base model and differ only in whether latent branches are scored and selected. Latent PRM guidance improves validation by 9.21 percentage points without repair (paired task-bootstrap 95% CI [+3.07, +15.79], two-sided paired sign-flip p = 0.0081) and by 8.78 percentage points under the same three-attempt repair loop (95% CI [+2.63, +15.35], p = 0.0064). A random branch-selection baseline with the same perturbation budget reaches only 27.28% validation, suggesting that many latent perturbations are harmful rather than automatically useful. Without repair, latent PRM guidance already exceeds all non-PRM baselines, including repaired latent reasoning. With repair, it improves further to 45.18%, reinforcing 4
Limitations
in HPC. In 2025 IEEE High Performance Extreme Computing Conference (HPEC), pages 1–9. IEEE.
This study is limited to one base-model family, one PRM backbone, and one executable parallelcode translation benchmark. The ParaTrans test set contains 76 held-out tasks, so the results should be read as evidence for this setting rather than as a general claim about latent guidance across models or domains. We also do not evaluate transfer of the learned value model across base models. We leave these broader evaluations to future work. Latent PRM guidance assumes access to intermediate hidden states, which limits its application to self-hosted models or otherwise white-box models unless black-box APIs expose these states. As with other domain-specific adaptation methods, the approach also requires task-specific supervision for the guidance model training. In our setting, this supervision is costly because parallelcode translations are long and each labeled example requires multiple roll-outs followed by executable and semantic validation. This limits the scale of the present study and makes broader robustness and transfer evaluations more difficult. Finally, although the branch-selection analysis is consistent with the filtering interpretation, it does not by itself explain all success and failure cases. Finer-grained analysis of these cases could inform more reliable latent intervention strategies beyond local branch filtering.
Gregory Bolet, Giorgis Georgakoudis, Harshitha Menon, Konstantinos Parasyris, Niranjan Hasabnis, Hayden Estes, Kirk Cameron, and Gal Oren. 2025a. Can large language models predict parallel code performance? In Proceedings of the 34th International Symposium on High-Performance Parallel and Distributed Computing, pages 1–6. Gregory Bolet, Giorgis Georgakoudis, Konstantinos Parasyris, Harshitha Menon, Niranjan Hasabnis, Kirk W Cameron, and Gal Oren. 2025b. Counting without running: Evaluating llms’ reasoning about code complexity. arXiv preprint arXiv:2512.04355. Aman Chaturvedi, Daniel Nichols, Siddharth Singh, and Abhinav Bhatele. 2025. Hpc-coder-v2: Studying code llms across low-resource parallel languages. In ISC High Performance 2025 Research Paper Proceedings (40th International Conference), pages 1– 14. Prometeus GmbH. Bei Chen, Fengji Zhang, Anh Nguyen, Daoguang Zan, Zeqi Lin, Jian-Guang Lou, and Weizhu Chen. 2022. Codet: Code generation with generated tests. arXiv preprint arXiv:2207.10397. Le Chen, Nesreen K Ahmed, Akash Dutta, Arijit Bhattacharjee, Sixing Yu, Quazi Ishtiaque Mahmud, Waqwoya Abebe, Hung Phan, Aishwarya Sarkar, Branden Butler, and 1 others. 2024a. The landscape and challenges of hpc research and llms. arXiv preprint arXiv:2402.02018. Le Chen, Arijit Bhattacharjee, Nesreen Ahmed, Niranjan Hasabnis, Gal Oren, Vy Vo, and Ali Jannesari. 2024b. Ompgpt: A generative pre-trained transformer model for openmp. In European Conference on Parallel Processing, pages 121–134. Springer Nature Switzerland Cham.
Risk Statement We do not identify material new risks from this work. The study is limited to offline benchmark evaluation of parallel-code translation.
Le Chen, Nuo Xu, Winson Chen, Bin Lei, Pei-Hung Lin, Dunzhi Zhou, Rajeev Thakur, Caiwen Ding, Ali Jannesari, and Chunhua Liao. 2025. Beyond code pairs: Dialogue-based data generation for llm code translation. arXiv preprint arXiv:2512.03086.
Acknowledgments Computational support was provided by Code Metal.
Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Mihir Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, and 37 others. 2021. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374.
References Wasi Uddin Ahmad, Saikat Chakraborty, Baishakhi Ray, and Kai-Wei Chang. 2023. Summarize and generate to back-translate: Unsupervised translation of programming languages. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 1528– 1542.
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. 2021. Training verifiers to solve math word problems. Preprint, arXiv:2110.14168.
Tomer Bitan, Tal Kadosh, Erel Kaplan, Shira Meiri, Le Chen, Peter Morales, Niranjan Hasabnis, and Gal Oren. 2025. UniPar: A unified LLM-based framework for parallel and accelerated code translation
5
Ning Dai, Zheng Wu, Renjie Zheng, Ziyun Wei, Wenlei Shi, Xing Jin, Guanlin Liu, Chen Dun, Liang Huang, and Lin Yan. 2024. Process supervision-guided policy optimization for code generation. arXiv preprint arXiv:2410.17621.
Tal Kadosh, Niranjan Hasabnis, Timothy Mattson, Yuval Pinter, and Gal Oren. 2023a. Quantifying openmp: Statistical insights into usage and adoption. In 2023 IEEE High Performance Extreme Computing Conference (HPEC), pages 1–7. IEEE.
Matthew T Dearing, Yiheng Tao, Xingfu Wu, Zhiling Lan, and Valerie Taylor. 2024. Lassi: An llm-based automated self-correcting pipeline for translating parallel scientific codes. In 2024 IEEE International Conference on Cluster Computing Workshops (CLUSTER Workshops), pages 136–143. IEEE.
Tal Kadosh, Niranjan Hasabnis, Prema Soundararajan, Vy A Vo, Mihai Capota, Nesreen Ahmed, Yuval Pinter, and Gal Oren. 2024a. Ompar: Automatic parallelization with ai-driven source-to-source compilation. In Machine Learning for Systems Workshop, NeurIPS.
Jingcheng Deng, Liang Pang, Zihao Wei, Shichen Xu, Zenghao Duan, Kun Xu, Yang Song, Huawei Shen, and Xueqi Cheng. 2025. Latent reasoning in llms as a vocabulary-space superposition. arXiv preprint arXiv:2510.15522.
Tal Kadosh, Niranjan Hasabnis, Vy A Vo, Nadav Schneider, Neva Krien, Mihai Capotă, Abdul Wasay, Guy Tamir, Ted Willke, Nesreen Ahmed, and 1 others. 2024b. Monocoder: Domain-specific code language model for hpc codes and tasks. In 2024 IEEE High Performance Extreme Computing Conference (HPEC), pages 1–7. IEEE.
Michael Günther, Jackmin Ong, Isabelle Mohr, Alaeddine Abdessalem, Tanguy Abel, Mohammad Kalim Akram, Susana Guzman, Georgios Mastrapas, Saba Sturua, Bo Wang, Maximilian Werk, Nan Wang, and Han Xiao. 2024. Jina embeddings 2: 8192-token general-purpose text embeddings for long documents. Preprint, arXiv:2310.19923.
Tal Kadosh, Niranjan Hasabnis, Vy A Vo, Nadav Schneider, Neva Krien, Abdul Wasay, Nesreen Ahmed, Ted Willke, Guy Tamir, Yuval Pinter, and 1 others. 2023b. Scope is all you need: Transforming llms for hpc code. arXiv preprint arXiv:2308.09440. Tal Kadosh, Nadav Schneider, Niranjan Hasabnis, Timothy Mattson, Yuval Pinter, and Gal Oren. 2023c. Advising openmp parallelization via a graph-based approach with transformers. In International Workshop on OpenMP, pages 3–17. Springer Nature Switzerland Cham.
Shibo Hao, Sainbayar Sukhbaatar, DiJia Su, Xian Li, Zhiting Hu, Jason Weston, and Yuandong Tian. 2025. Training large language models to reason in a continuous latent space. Preprint, arXiv:2412.06769. Re’em Harel, Yuval Pinter, and Gal Oren. 2023. Learning to parallelize in a shared-memory environment with transformers. In Proceedings of the 28th ACM SIGPLAN annual symposium on principles and practice of parallel programming, pages 450–452.
Erel Kaplan, Tomer Bitan, Lian Ghrayeb, Le Chen, Tom Yotam, Niranjan Hasabnis, and Gal Oren. 2026. Paracodex: A profiling-guided autonomous coding agent for reliable parallel code generation and translation. Preprint, arXiv:2601.04327.
Re’em Harel, Tal Kadosh, Niranjan Hasabnis, Timothy Mattson, Yuval Pinter, and Gal Oren. 2025. Pragformer: data-driven parallel source code classification with transformers. International Journal of Parallel Programming, 53(1):2.
Qingyao Li, Xinyi Dai, Xiangyang Li, Weinan Zhang, Yasheng Wang, Ruiming Tang, and Yong Yu. 2025. CodePRM: Execution feedback-enhanced process reward model for code generation. In Findings of the Association for Computational Linguistics: ACL 2025, pages 8169–8182.
Re’em Harel, Idan Mosseri, Harel Levin, Lee-or Alon, Matan Rusanovsky, and Gal Oren. 2020. Source-tosource parallelization compilers for scientific sharedmemory multi-core and accelerated multiprocessing: analysis, pitfalls, enhancement and potential. International Journal of Parallel Programming, 48(1):1– 31.
Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. 2023. Let’s verify step by step. In The twelfth international conference on learning representations.
Andrew Head. 2013. cu2cl: Cuda to opencl source-tosource translator. Accessed: 2025-06-09.
Ilya Loshchilov and Frank Hutter. 2019. Decoupled weight decay regularization. In International Conference on Learning Representations.
Yufan Huang, Mengnan Qi, Yongqiang Yao, Maoquan Wang, Bin Gu, Colin Clement, and Neel Sundaresan. 2023. Program translation via code distillation. arXiv preprint arXiv:2310.11476.
Francesca Lucchetti and Arjun Guha. 2025. Understanding how codellms (mis) predict types with activation steering. In Proceedings of the 8th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP, pages 358–397.
Zheming Jin and Jeffrey S. Vetter. 2023. A benchmark suite for improving performance portability of the sycl programming model. In 2023 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS), pages 325–327.
Matthew V Macfarlane and Clement Bonnet. 2025. Searching latent program spaces. Preprint, arXiv:2411.08706.
6
Robert Mueller-Albrecht. 2024. Syclomatic: Sycl adoption for everyone-moving from cuda to sycl gets progressively easier: Advanced migration considerations. In Proceedings of the 12th International Workshop on OpenCL and SYCL, pages 1–2.
2024. Math-Shepherd: Verify and reinforce LLMs step-by-step without human annotations. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 9426–9439.
Daniel Nichols, Aniruddha Marathe, Harshitha Menon, Todd Gamblin, and Abhinav Bhatele. 2024. Hpccoder: Modeling parallel programs using large language models. In ISC High Performance 2024 Research Paper Proceedings (39th International Conference), pages 1–12. Prometeus GmbH.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc V. Le, and Denny Zhou. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824– 24837.
Maxwell Nye, Anders Johan Andreassen, Guy Gur-Ari, Henryk Michalewski, Jacob Austin, David Bieber, David Dohan, Aitor Lewkowycz, Maarten Bosma, David Luan, Charles Sutton, and Geoffrey Irving. 2021. Show your work: Scratchpads for intermediate computation with language models.
Yige Xu, Xu Guo, Zhiwei Zeng, and Chunyan Miao. 2025. SoftCoT: Soft chain-of-thought for efficient reasoning with LLMs. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 23336– 23351.
Md Mahbubur Rahman, Arjun Guha, and Harshitha Menon. 2026. Steering code llms with activation directions for language and library control. arXiv preprint arXiv:2603.23629.
Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. 2023. Tree of thoughts: Deliberate problem solving with large language models. Advances in neural information processing systems, 36:11809–11822.
Nina Rimsky, Nick Gabrieli, Julian Schulz, Meg Tong, Evan Hubinger, and Alexander Turner. 2024. Steering llama 2 via contrastive activation addition. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 15504–15522.
Zhenrui Yue, Bowen Jin, Huimin Zeng, Honglei Zhuang, Zhen Qin, Jinsung Yoon, Lanyu Shang, Jiawei Han, and Dong Wang. 2025. Hybrid latent reasoning via reinforcement learning. arXiv preprint arXiv:2505.18454.
ROCm Developers. 2023. Hipify: Cuda to hip source translation tool. Accessed: 2025-06-09.
Zhen Zhang, Xuehai He, Weixiang Yan, Ao Shen, Chenyang Zhao, Shuohang Wang, Yelong Shen, and Xin Eric Wang. 2025a. Soft thinking: Unlocking the reasoning potential of LLMs in continuous concept space. arXiv preprint arXiv:2505.15778.
Nadav Schneider, Niranjan Hasabnis, Vy A Vo, Tal Kadosh, Neva Krien, Mihai Capota, Guy Tamir, Theodore L Willke, Nesreen Ahmed, Yuval Pinter, and 1 others. 2024. Mpirigen: Mpi code generation through domain-specific language models. In Proceedings of the 2024 Workshop on AI For Systems, pages 1–6.
Zhenru Zhang, Chujie Zheng, Yangzhen Wu, Beichen Zhang, Runji Lin, Bowen Yu, Dayiheng Liu, Jingren Zhou, and Junyang Lin. 2025b. The lessons of developing process reward models in mathematical reasoning. In Findings of the Association for Computational Linguistics: ACL 2025, pages 10495–10516.
Nadav Schneider, Tal Kadosh, Niranjan Hasabnis, Timothy Mattson, Yuval Pinter, and Gal Oren. 2023. Mpi-rical: Data-driven mpi distributed parallelism assistance with transformers. In Proceedings of the SC’23 Workshops of the International Conference on High Performance Computing, Network, Storage, and Analysis, pages 2–10.
Jiaru Zou, Xiyuan Yang, Ruizhong Qiu, Gaotang Li, Katherine Tieu, Pan Lu, Ke Shen, Hanghang Tong, Yejin Choi, Jingrui He, James Zou, Mengdi Wang, and Ling Yang. 2025. Latent collaboration in multiagent systems. Preprint, arXiv:2511.20639.
Changqing Shi, Yufei Sun, Rui Chen, Jiahao Wang, Qiang Guo, Chunye Gong, Yicheng Sui, Yutong Jin, and Yuzhi Zhang. 2025. Transcl: An automatic cudato-opencl programs transformation framework. ACM Transactions on Architecture and Code Optimization, 22(2):1–24.
A
Expanded Related Work and Positioning
Execution-based and compiler-guided codegeneration work has established that behavioral tests are a stronger signal for program quality than string overlap (Chen et al., 2021, 2022). Program-translation work then extended that view to translation settings through supervised adaptation, back-translation, distillation, and dialogue-based data generation (Ahmad et al., 2023; Huang et al., 2023; Chen et al., 2025).
Alexander Matt Turner, Lisa Thiergart, Gavin Leech, David Udell, Juan J Vazquez, Ulisse Mini, and Monte MacDiarmid. 2023. Steering language models with activation engineering. arXiv preprint arXiv:2308.10248. Peiyi Wang, Lei Li, Zhihong Shao, Runxin Xu, Damai Dai, Yifei Li, Deli Chen, Yu Wu, and Zhifang Sui.
7
Area
Representative work
Intervention point
Representation
Exec.?
Difference
Code generation and translation
Chen et al. (2021), Chen et al. (2022), Ahmad et al. (2023), Huang et al. (2023), Bitan et al. (2025) Nichols et al. (2024), Chaturvedi et al. (2025), Dearing et al. (2024), Kaplan et al. (2026), MuellerAlbrecht (2024), ROCm Developers (2023), Head (2013), Shi et al. (2025) Cobbe et al. (2021), Lightman et al. (2023), Wang et al. (2024), Li et al. (2025), Dai et al. (2024) Hao et al. (2025), Xu et al. (2025), Yue et al. (2025), Zhang et al. (2025a), Deng et al. (2025), Macfarlane and Bonnet (2025), Zou et al. (2025)
Training data, decoded programs, or repair loops
Tokens or program text
Often
We intervene before final decoding by scoring latent prefixes while keeping the generator frozen.
Fine-tuning, sourceto-source translation, or post hoc repair
Program text or symbolic code
Often
These methods operate on decoded code or deterministic migration rules rather than on continuous latent prefixes.
Token steps or completed programs
Text
Often
We supervise hidden-state prefixes before code is emitted.
Internal reasoning phase or latent search
Hidden states or latent programs
Usually no
We attach a smaller PRM to score candidate latent branches for executable parallel-code translation.
Parallel and HPC translation
PRMs and verifiers
Latent reasoning and latent search
Table 3: Taxonomy of the closest neighboring literatures by intervention point.
These papers are relevant because they define the broader code-generation landscape in which our task sits, but their intervention points remain at the level of training data or decoded text rather than pre-decoding latent guidance.
pendent of downstream repair and transformation methods and can be combined with downstream repair or optimization methods. Most existing guidance methods intervene after the program text is produced. Verifier-guided reasoning, chain-of-thought prompting, search, and PRMs provide the conceptual template for allocating additional test-time computation with auxiliary signals (Wei et al., 2022; Nye et al., 2021; Cobbe et al., 2021; Lightman et al., 2023; Wang et al., 2024; Yao et al., 2023; Zhang et al., 2025b). For code, CodePRM and process-supervision-guided policy optimization show that execution-aware supervision can improve program generation (Li et al., 2025; Dai et al., 2024). The key difference is that these methods score textual reasoning steps, decoded programs, or execution feedback after the text has been produced, whereas our method intervenes earlier by scoring continuous latent prefixes before code is emitted.
HPC and parallel language translation-specific work is closer to our evaluation domain. UniPar, LASSI, ParaCodex, HPC-Coder, and HPCCoder-v2 study executable parallel-code generation, while adjacent work studies parallelization assistance, fine-tuning, repair, or agentic adaptation with LLMs (Nichols et al., 2024; Bitan et al., 2025; Dearing et al., 2024; Kaplan et al., 2026; Chaturvedi et al., 2025; Harel et al., 2023; Schneider et al., 2023; Kadosh et al., 2023c; Harel et al., 2025; Bolet et al., 2025a; Kadosh et al., 2024b; Schneider et al., 2024; Chen et al., 2024b; Bolet et al., 2025b). Source-to-source translation systems such as SYCLomatic, HIPIFY, cu2cl, and TransCL intervene through deterministic or compiler-style program transformation rather than latent guidance (Mueller-Albrecht, 2024; ROCm Developers, 2023; Head, 2013; Shi et al., 2025; Kadosh et al., 2024a; Harel et al., 2020). Our setting is complementary: we ask whether a frozen generator can be steered before code is decoded. Thus, it is inde-
Latent reasoning offers an opportunity for this different intervention point. These papers motivate continuous hidden states as a useful reasoning substrate and shift part of test-time computation from explicit token sequences into continuous hiddenstate trajectories before returning to standard de8
coding (Hao et al., 2025; Xu et al., 2025; Yue et al., 2025; Zhang et al., 2025a; Deng et al., 2025; Zou et al., 2025). Macfarlane and Bonnet (2025) is especially relevant because it treats latent computation as a search problem. Yet most latent-reasoning systems still follow a single hidden-state trajectory. They do not ask whether nearby latent branches can be compared before the final program exists. Our contribution is more application-specific: we study the applicability of a smaller PRM that scores candidate latent prefixes for executable parallelcode translation before any final program is decoded. A more distant but relevant line of work is activation steering. These methods modify internal activations at inference time, often by adding a contrastively derived direction to control a specific behavior (Turner et al., 2023; Rimsky et al., 2024). Code-focused variants use similar interventions for type prediction or language and library control (Lucchetti and Guha, 2025; Rahman et al., 2026) but not in parallel code. We draw only on the broader idea that hidden vectors can serve as intervention points. Unlike activation steering, however, we move the intervention from decoding-time control to the latent-reasoning phase, and we use a learned value model to compare task-conditioned continuations rather than applying a fixed steering direction.
B
Setting
Wins
Ties
Losses
No repair +3-attempt repair
20 18
50 53
6 5
Table 4: Task-level paired win/loss/tie counts for PRMguided latent reasoning against unguided latent reasoning on the 76-task ParaTrans test set. Method
C→O
O→C
S→O
S→C
LR 54.4/40.4 LR-PRM 52.6/43.9
40.4/33.3 47.4/36.8
63.3/33.3 65.0/55.0
40.4/24.6 42.1/24.6
Table 5: Direction-wise compilation/validation rates on ParaTrans. Abbreviations: LR = latent reasoning, C = CUDA, O = OpenMP, S = Serial.
compilation gain on Serial→CUDA. We view this split as additional descriptive evidence that the aggregate result is not driven by only one direction, rather than as proof of direction robustness. This pattern suggests that the aggregate improvement reflects both syntactic and semantic effects, since some directions improve mainly in compilation, some in validation, and some in both. Latent Search Budget Ablation. Table 6 ablates the test-time latent search budget by varying the number of candidate branches per step, Btest , and the number of latent reasoning steps, K. Due to time and compute constraints, we construct PRM training trajectories under the smaller latent-step budget, but the preliminary pattern suggests that the PRM benefits from a larger decision surface at inference. More reasoning steps and more candidate options give the scorer additional opportunities to avoid weak continuations and occasionally choose stronger latent branches before decoding.
Supporting Results
Paired task win/loss/tie analysis. We report task-level paired win/loss/tie counts using ParaTrans tasks averaged over three runs, matching the unit used for the paired bootstrap analysis. For each task, we compute the guided-minus-unguided validation-rate difference. A task is a win if the difference is positive, a loss if it is negative, and a tie otherwise. Confidence intervals and p-values follow the task-level paired bootstrap and twosided sign-flip tests described in Appendix C, using 100,000 resamples or Monte Carlo samples. Table 4 shows that most tasks are unchanged, but among tasks whose outcome changes, PRM guidance more often improves than degrades validation in both the no-repair and repair settings.
Test-time compute cost. We estimate the incremental cost of PRM guidance relative to unguided latent reasoning under the same K = 12 latent-step setting with the 70B generator and one final decoding pass. PRM guidance adds Btest = 8 branch scores per step, including the unperturbed branch, or 8 · 12 = 96 Qwen-Coder-7B prefix-scoring calls per sample. By parameter count, this is roughly 9.6 LLaMA-3.3-70B-sized forward-step calls, while avoiding additional 70B rollouts or extra decoded programs. This is only an approximate call-level normalization and does not account for implementation details. The decoded non-code portion is unchanged in our logs, with high variance around 160 tokens in both settings, and the final code is
Direction Split. Table 5 breaks down compilation and validation rates by translation direction. PRM guidance improves over unguided latent reasoning in three of the four directions, with the largest gain on Serial→OpenMP, smaller gains on the two CUDA/OpenMP directions, and only 9
Btest
K
Validation (%)
3 3 8
6 12 12
28.94 35.52 42.10
the main gain is not explained by PRM supervision alone. Taken together, these results support the value of applying the reward model before decoding over latent prefixes in this setting.
Table 6: Ablation over the test-time latent search budget on ParaTrans test set.
C
Hyperparameters and Reproducibility
Evaluation protocol. Main evaluation uses the 76-task ParaTrans test set, split across CUDA→OpenMP (18), OpenMP→CUDA (19), Serial→OpenMP (20), and Serial→CUDA (19). We report three-run mean executable validation rates with and without a three-attempt self-repair loop. In the repair setting, failed translations are regenerated from the previous code and the executionenvironment feedback. We also report a separate 500-instance held-out branch-selection benchmark, where each saved latent prefix is paired with the original branch and two perturbed alternatives, and instances without a unique best branch are excluded. For pairwise comparisons, we use task-level paired inference. For each task, we average the binary validation outcomes over the three runs for each method, then compute the guided-minusunguided task difference. We report the mean of these task-level differences. Confidence intervals are computed using a paired nonparametric bootstrap over tasks with B = 100,000 resamples. Pvalues are computed using a two-sided paired signflip randomization test over the same task-level differences with 100,000 Monte Carlo samples. We used two NVIDIA H200 GPUs for primarymodel inference when creating the dataset and running experiments. For PRM training and code evaluations, we used two NVIDIA A40 GPUs. For the three-attempt repair setting, we feed the executionenvironment error and the previously generated code back to the model, then ask it to regenerate the full code. The exact prompt templates are shown below. We train the PRM with the AdamW optimizer (Loshchilov and Hutter, 2019) in two stages. In the first stage, we train only the adaptation layer, which maps the primary model hidden dimension of 8192 to the PRM backbone hidden dimension of 3584, together with the value head, for half an epoch. In the second stage, we unfreeze the remaining layers. We train with an effective batch size of 6 and a learning rate of 1 · 10−6 .
decoded once in each condition. This overhead is small compared with postdecoding repair. A repair attempt requires another full 70B decoding pass over the program, which in our logs generates approximately 1,853 tokens per attempt on average. Thus, the three-attempt repair setting adds approximately 3 · 1,853 additional 70B decoding steps, compared with a much smaller auxiliary PRM-scoring budget. Despite this much smaller extra budget, latent PRM guidance without repair reaches 42.1% validation, exceeding unguided latent reasoning with three repair attempts at 36.4%. This comparison suggests that the gains are not explained simply by adding test-time computation. Wall-clock measurements are also similar within run-to-run variation: 145.67±9.33s per sample with PRM guidance versus 147.33±4.67s for unguided latent reasoning over three runs. This parameter-count normalization is approximate, but it indicates that PRM scoring is not a dominant overhead in our implementation. These measurements should not be read as a full efficiency claim. They show only that PRM scoring adds modest overhead in our implementation and remains compatible with token-level repair. Post-decoding text-PRM reranking. We also evaluate a post-decoding best-of-8 control that applies the same reward supervision after complete programs have been generated. We sample eight complete outputs from unguided latent reasoning without latent perturbations, using temperature 0.8 during decoding, and train a text PRM over completed answers with the same reward function used for latent PRM training. Candidates selected by the text PRM achieve 25.00% validation, while an oracle@8 over the same candidate sets reaches 36.84%. The oracle result indicates that the sampled sets sometimes contain validating programs, while the gap between oracle@8 and the text PRM shows that this control does not reliably identify them. Moreover, even this oracle remains below latent PRM guidance at 42.10%. This suggests that
Split and leakage each translation task 10
audit. We define by the fingerprint
Split / artifact
Count
Test-kernel overlap
PRM dev Branch-selection PRM train Final PRM train Branch-selection source set ParaTrans test
60 832 952 120 76
0 0 0 0 –
S(y) are 0.2423, 0.1038, and 0.1038, respectively, in addition to the fixed 0.30 compilation and 0.25 execution weights. GPT-5-mini judge. The judge is used only for PRM target construction and never for final benchmark scoring. For each candidate rollout, GPT5-mini receives the source program, and the generated target-language program. It is asked to assess semantic equivalence, target-API correctness, and likely behavioral preservation, then return a bounded scalar score in [0, 1] with a short justification. We use the scalar score as one optional validator in the renormalized reward above.
Table 7: Leakage audit for PRM development/training artifacts and the held-out source trajectories used to construct the branch-selection benchmark. The branchselection source set row counts trajectories, not the 500 derived branch-choice instances; overlap with the ParaTrans test set is computed using (kernel_name, from_api, to_api) fingerprints.
(kernel_name, from_api, to_api). The PRM development split contains the first 15 kernels in each translation direction, for 60 examples total. It is used only for hyperparameter selection. After hyperparameters are fixed, the final PRM is trained on the union of the PRM training and development pools. For the branch-selection experiment, we train a separate PRM after removing the 120 source trajectories used to construct the branch-choice instances. The 76-task ParaTrans test set is held out from all PRM training, development, trajectory collection, reward construction, and hyperparameter selection. As a stricter check, we also compare kernel names while ignoring translation direction; this overlap is also zero.
Judge Prompt Template Role. You are an expert in HPC code translation evaluation. Compare the original and translated code section below. Assign a score between 0.0 and 1.0 based only on syntax correctness, faithfulness, and overall quality. Scoring rubric. • 0.00–0.20: incorrect or not on the right track • 0.21–0.50: major issues • 0.51–0.80: mostly correct with minor issues • 0.81–1.00: strong and faithful Rules. • Focus on logic and correctness. • Ignore style, but pay attention to pragma placement in OpenMP directives. • If the code is incomplete because of cut off text or missing includes, assume the most reasonable completion for scoring. Inputs. • Kernel name: {kernel_name or ’UNKNOWN’} • Original code: {original_code[:8000]} • Translated code: {translated_code[:8000]} Required output. Return strict JSON only:
Reward construction. The reward target mixes executable and proxy components. We keep that distinction explicit because the benchmark metric in the main paper is executable validation only. For a terminal program y, let C(y) ∈ {0, 1} indicate successful compilation, E(y) ∈ {0, 1} indicate successful execution without runtime failure, and Aj (y) ∈ [0, 1] denote optional semantic or proxy validators available for that sample. We compute P
S(y) = 0.30C(y) + 0.25E(y) + 0.45
{"score": <float 0.0-1.0>, "reason": "<short reason>"}
Variable-comparison heuristic. The variablecomparison heuristic is intended to catch semantic shortcuts not exposed by unit tests alone. GPT-5mini first identifies variables in the source and generated programs that represent the core computed quantity for the built-in test. We then instrument the reference and generated programs by adding print statements for the selected variables, execute both programs on the same test input, and compare the observed final values.
wj Aj (y) j∈A(y) P j∈A(y) wj
where unavailable validators are omitted and the remaining validator weights are renormalized. The optional validators include pass/fail integrated validation, checksum or output matching when available, benchmark-specific metrics, Jina codeembedding cosine similarity to the reference target code (Günther et al., 2024), a GPT-5-mini judge score, and the variable-comparison heuristic described below. In the common case where pass/fail validation, Jina similarity, and the GPT-5-mini judge are available, their effective contributions to
Variable Identification Prompt Template Role. You are a precise static code analysis agent. Hard rules. • Static analysis only: do not suggest edits, do not modify code, and do not add instrumentation.
11
Regular Inference Prompt Template • Use only the code text provided in the request. • Output must be valid JSON only, with no markdown or extra text. Input fields. • kernel_name: string • original_code: string, possibly empty, containing concatenated files prefixed by // File: <filename> • translated_code: string, possibly empty, using the same format Goal. For each codebase, identify 1 to 3 variables that represent the most semantically central computed results of the program or kernel. • Return multiple variables only when there are genuinely multiple important result variables. • If there is one clear result variable, return only that one. • Order variables by importance, with the most central first. Critical distinction. • Do not choose status flags, timing variables, loop counters, argument parsing variables, or configuration constants unless no computed result variable exists. • Prefer numeric result variables and, failing that, the primary output buffer updated by the kernel or parallel region. Priorities. 1. Reported numeric result scalar. Choose a nonboolean numeric scalar printed or logged as the main result, such as a checksum, error, norm, sum, energy, or output summary. 2. Derived numeric result variable just before output. If the printed value is an expression, choose the stored numeric variable closest to the output site. 3. Primary output buffer, array, or pointer. If there is no clear scalar result, choose the main computed output buffer. 4. Last resort. If nothing else fits, choose the single variable most central to the computed output, avoiding booleans and timing variables when possible. Type, missing, ambiguity, and not-found rules. • Return the declared type exactly as written in code; use UNKNOWN only when the type cannot be determined. • If original_code is missing, set source status to MISSING with null fields. • If translated_code is missing, set target status to MISSING with null fields. • If multiple candidates satisfy the same highest applicable priority, still choose one top candidate, mark the status as AMBIGUOUS, and list alternatives in notes. • Use NOT_FOUND only when code exists but no result variable can be identified under priorities 1–4. • The variables array must contain 1 to 3 objects, ordered by importance.
System. You are an HPC expert specializing in translating between parallel programming APIs. User template. For each kernel code provided, think about it step by step, and then translate it from {from_api} to {to_api}. The target {to_api} implementation is spread across {n} files: {names_str}. Provide the complete code in {to_api}. Do not truncate or use ellipses. Do not change the main function. Ensure correctness. All function names must match. The code to translate: {source_code}
3 Attempt repair prompt. The repair prompt is used in the three-attempt regeneration setting after a failed compilation or execution attempt. It provides the previous error trace together with the broken translation and asks the model to emit a complete corrected translation in a strictly parseable format. Repair Prompt Template Context. The previous translation attempt (iteration {iteration - 1}) failed with the following error: — BEGIN ERROR — {error_text} — END ERROR — Previous broken translation. — BEGIN PREVIOUS ATTEMPT — {prev_code} — END PREVIOUS ATTEMPT — Instruction. Please fix the error and produce a corrected translation. Output format requirements. • Provide the complete code in {to_api}. • Do not truncate or use ellipses. Do not change the main function. • Ensure correctness. All function names must match. • Wrap the code in a single fenced block beginning with ```cpp and ending with ```. • You may instead begin the block with ```c, ```cuda, ```omp, or ```cu if that better matches {to_api}. • If the translation spans multiple files, place a // File: <filename> marker at the start of each file’s content inside the same fenced block, using the exact target filenames. • Do not include any prose after the closing fence.
Inference prompt. The following prompt is used for the initial (non-repair) translation pass and during dataset creation. The system message establishes the model’s role as an HPC translation expert, and the user template supplies the source API, target API, expected output file count and names, and the source code.
12