Beyond Pass@k: Redundancy-Aware RLVR for Multi-Sample Code Generation Florian Le Bronnec1 [email protected]
Alexandre Verine2 [email protected]
Rio Yokota1 [email protected]
Benjamin Negrevergne3 [email protected]
1
RIKEN Center for Computational Science, Tokyo, Japan École Normale Supérieure Paris, PSL University, Paris, France 3 LAMSADE, CNRS, Université Paris-Dauphine-PSL, Paris, France
arXiv:2605.28022v1 [cs.CL] 27 May 2026
2
Abstract
et al., 2022). State-of-the-art coding models rely heavily on such post-training optimization to improve executable correctness (Grattafiori et al., 2024; Olmo et al., 2026). As coding benchmarks increasingly emphasize repeated-sampling evaluation, RL objectives have also shifted from optimizing isolated generations toward optimizing finitebudget executable success under Pass@k metrics (Tang et al., 2025; Walder and Karkhanis, 2026).
LLMs for code generation are commonly evaluated in repeated-sampling settings using Pass@k, where multiple candidate programs are executed against unit tests under a finite sampling budget. While recent verifier-based reinforcement learning (RLVR) methods improve executable correctness, how these objectives affect redundancy among sampled programs remains poorly understood. In this work, we study implementation-level redundancy in code generation using JPlag, a plagiarismdetection system for code. Across models and benchmarks, we show that correctness-only RLVR often concentrates generations around repeated implementations, whereas Pass@kaware objectives maintain lower redundancy and improve larger-budget performance.
Finite-budget Pass@k evaluation. Although these objectives improve executable performance under different sampling budgets, the sampledprogram behaviors underlying these gains remain poorly understood. In particular, Pass@k measures whether at least one sampled program is correct, but does not distinguish between repeatedly generating near-duplicate implementations and covering a broader set of distinct implementation patterns. This distinction matters because the usefulness of repeated sampling changes with k: larger sampling budgets can benefit not only from reliable generations, but also from reducing redundancy among generated programs. This motivates our first research question:
Motivated by these observations, we augment RLVR with direct anti-redundancy rewards based on JPlag similarity. Across 3 models and 3 benchmarks, discouraging near-duplicate generations reliably improves finite-budget executable performance, often matching or outperforming specialized Pass@k-aware objectives.
1
Introduction
Q1. When multi-sample performance improves, does the model use its sampling budget to produce less redundant solutions or to repeat near-duplicate implementations?
Code generation and Pass@k. When solving programming tasks with large language models (LLMs), it is common to generate multiple candidate programs until one successfully passes the task tests (Hui et al., 2024; Zheng et al., 2025; Jiang et al., 2026). This repeated-sampling setting is formalized through Pass@k, which is the probability that at least one of k sampled programs is correct (Chen et al., 2021; Guo et al., 2025).
JPlag as a redundancy diagnostic. Executable correctness alone does not distinguish between repeatedly sampling near-duplicate programs and producing a broader set of implementations under a fixed sampling budget. To study redundancy among generated programs, we use JPlag, a program-similarity system originally developed for plagiarism detection (Prechelt et al., 2002). Unlike lexical overlap metrics, JPlag uses a programming-
RLVR for code generation. Because generated programs can be executed directly against tests, code generation is a natural setting for reinforcement learning with verifiable rewards (RLVR) (Li 1
language-aware tokenization and matching procedure, making it substantially less sensitive to superficial edits such as variable renaming, formatting changes, or comments. This makes it a useful diagnostic for repeated implementations in multisample code generation. Applying this diagnostic to RLVR methods, we find that objectives targeting larger-k Pass@k performance tend to produce sampled sets with lower redundancy and higher JPlag diversity.
obtain similar Pass@k while producing very different sampled sets: one may repeatedly generate near-duplicate solutions, while another may distribute samples across less redundant implementations. Our work studies this implementation-level redundancy directly. Pass@k-aware RLVR. RLVR improves executable code generation by optimizing verifier feedback from unit tests or task checkers (Shao et al., 2024; Li et al., 2022; Gehring et al., 2025). Recent work adapts RLVR to repeated-sampling evaluation through objectives aligned with Pass@k (Tang et al., 2025; Walder and Karkhanis, 2026). These methods are the closest methodological neighbors to our work: they change credit assignment over sampled groups to improve finite-budget executable success. We ask a complementary question: how do these objectives affect redundancy among the sampled programs, and can redundancy itself serve as a useful group-level training signal?
Direct anti-redundancy training. These observations motivate our second research question: Q2. Can implementation redundancy itself serve as a useful optimization target for finite-budget code generation? To investigate this, we train RLVR objectives with explicit group-level anti-redundancy rewards based on JPlag similarity. This simple intervention reliably improves finite-budget executable performance, across models and benchmarks, often matching or outperforming specialized Pass@kaware objectives.
Diversity and redundancy in generated programs. Several studies show that post-training can improve generation quality while reducing diversity or concentrating the sampled distribution (Kirk et al., 2024; Le Bronnec et al., 2024). Explicit diversity incentives have also been explored for reasoning tasks, particularly in mathematical settings (Hu et al., 2026). For code generation, prior work has analyzed diversity using modelbased or semantic evaluation pipelines (Lee et al., 2025). Such approaches aim to capture broad behavioral or algorithmic differences between generated programs. We focus on a more specific notion: whether repeated-sampling RLVR produces nearduplicate implementations, and how this redundancy relates to finite-budget performance. While JPlag is not a full semantic-equivalence metric, it provides a practical code-aware signal that is robust to superficial edits and sufficiently informative to analyze and discourage repeated implementations.
Contributions. Our contributions are as follows: • We introduce JPlag as a practical diagnostic for implementation-level redundancy in multisample code generation. • We show that Pass@k-aware RL objectives systematically maintain lower redundancy among sampled programs than correctness-only RLVR. • We show that directly discouraging nearduplicate generations improves finite-budget executable performance and can match or outperform specialized Pass@k-aware objectives, without simply increasing sampling entropy or relying on superficial output variation. Overall, our results identify implementation redundancy as both a useful diagnostic and a practical optimization target for repeated-sampling code generation.
2
3
Background and Setup
Generation setup. For each programming problem x, a model with parameters θ defines a distribution over candidate programs y ∼ πθ (· | x). We study the multi-sample setting: for each prompt, we sample a set of n independent programs
Related Work
Repeated-sampling evaluation for code generation. Pass@k measures the probability that at least one of k sampled programs solves a coding task (Chen et al., 2021; Li et al., 2022). This metric makes finite-budget sampling central to codegeneration evaluation, but it reduces a sampled set to executable success. Two models can therefore
Y (n) = {y1 , . . . , yn }. Each program is executed against the task tests or checker, producing a binary correctness value 2
c(x, yi ) ∈ {0, 1}.
Base-RLVR. The standard correctness-only baseline uses executable correctness as a group reward by summing verifier outcomes:
Pass@k evaluation. We use Pass@k as the main executable evaluation metric throughout this work. For a task x, Pass@k measures whether at least one of k sampled programs is correct: Pass@k(x) = EY (k) ∼πθ (·|x)k
Rcorr (x, Y
(6)
Pass@k-RLVR. Tang et al. (2025) instead consider the repeated-sampling evaluation setting directly, where success depends on whether at least one program in the sampled set is correct. For a sampled group Y (k) of size k,
(2)
Rpass@k (x, Y (k) ) = max c(x, yj ).
with value 1 when n − m < k. For a dataset X = {x1 , . . . , xN }, we report
yj ∈Y (k)
(7)
With rj = c(x, yj ), the leave-one-out advantage is
N
1 X \ Pass@k(xℓ ). N
c(x, yi ).
Under this objective, each correct generation contributes additively to the reward, so credit is assigned independently to successful samples within the group.
yi ∈Y (k)
In practice, we estimate this quantity from a larger set of n sampled programs. Let m = P y∈Y (n) c(x, y) denote the number of correct samples. We then use the standard estimator
Pass@k(X ) =
)=
n X i=1
max c(x, yi ) . (1)
n−m . n \ Pass@k(x) =1− , k k
(n)
Ai = max rj − max rj .
(3)
j
j̸=i
(8)
ℓ=1
Thus a sample receives positive advantage only when removing it changes the group-level success outcome. We use the centered variant from Tang et al. (2025) in all experiments.
In the remainder of the paper, we simply write Pass@k or p@k for the dataset-level metric when the evaluation set is clear from context. 3.1
PKPO. Walder and Karkhanis (2026) extend the same pass-at-k principle to groups with size n ≥ k. Rather than computing the advantage from a single sampled group, PKPO averages the leave-one-out advantage of each sample over all k-subsets: ! X 1 PKPO max rj − max rj . Ai = n
Verifier-Based RLVR for Code Generation
Verifier-based reinforcement learning with verifiable rewards uses executable feedback to posttrain code models. We write these methods in a group-sampling form: for each prompt x, the policy samples a group Y (n) = {y1 , . . . , yn }, with each yi ∼ πθ (· | x) drawn independently. A reward R(x, Y (n) ) is then computed over the samples:
k I⊆{1,...,n} |I|=k, i∈I
In policy-gradient implementations, the group reward is transformed into per-sample advantages to assign credit within a sampled group and reduce gradient variance. Updates then take the form A(x, yi , Y (n) )∇θ log πθ (yi | x),
j∈I\{i}
This mirrors the standard Pass@k estimator eq. (2) while reducing variance relative to relying on a single sampled group. In our experiments, we use their proposed sloo_minus_one estimator.
h i JRLVR (θ) = Ex EY (n) ∼πθ (·|x)n R(x, Y (n) ) . (4)
n X
j∈I
3.2
Diversity and Redundancy Diagnostics
To study how RL objectives use a finite sampling budget, we measure redundancy among sampled programs using several complementary diagnostics. Existing approaches span a broad spectrum of tradeoffs. Lexical n-gram diversity measures local surface overlap between generations and is lightweight to compute, but remains sensitive to superficial edits. Embedding-based diagnostics such
(5)
i=1
where the methods below differ mainly in how the advantage A(x, yi , Y (n) ) is constructed. 3
Scores
Generation A
Generation B
JPlag: 1.00 1-gram: 0.45
def max_val(lst): numbers = [x for x in lst if isinstance(x, int)] return max(numbers)
def max_val(het_list): numbers = [item for item in het_list if isinstance(item, int)] return max(numbers)
def max_val(het_list): max_value = float('-inf') for item in het_list: if (isinstance(item, int) or (isinstance(item, str) and item.isdigit())): num = int(item) max_value = num if num > max_value else max_value return max_value
def max_val(het_list): numbers = [ int(item) for item in het_list if (isinstance(item, int) or (isinstance(item, str) and item.isdigit())) ] return max(numbers) if numbers else None
JPlag: 0.68 1-gram: 0.56
Table 1: Qualitative examples illustrating how JPlag similarity compares with lexical overlap. All examples use the prompt: “Write a function to find the maximum value in a given heterogeneous list.” The generated programs differ in variable names, surface form, and control structure.
as the Vendi score (Friedman and Dieng, 2023) computed over code embeddings (Kryvosheieva et al., 2025) capture broader dispersion patterns, but are less directly interpretable in terms of repeated implementations.
similarity graphs. We also report lexical and embedding-based diversity diagnostics in our experiments. While correlated with JPlag diversity, these metrics capture partially different aspects of sampled-program variation (App. D, Fig. D.5).
JPlag diversity. Executable code generation provides a setting where redundancy can be analyzed more directly through program similarity. Our main diagnostic is therefore based on JPlag, a program-similarity system originally developed for plagiarism detection (Prechelt et al., 2002). Compared to lexical overlap metrics, JPlag is substantially less sensitive to superficial edits such as variable renaming or formatting changes, making it better aligned with repeated implementations in generated code. At the same time, JPlag remains lightweight, interpretable, and scalable compared to semantic or execution-based similarity pipelines, making it practical for repeated-sampling RLVR analysis. Table 1 illustrates this behavior on generated solution pairs for the same prompt. Programs with different variable names but similar control structure receive high JPlag similarity, while programs relying on different implementations receive lower similarity despite solving the same task. For each prompt x and sampled set of programs (n) Y , we compute s(yi , yj ) ∈ [0, 1] , the JPlag similarity between sampled programs yi and yj . We define JPlag diversity (JDiv) as one minus the mean pairwise similarity: X 2 JDiv(Y (n) ) = 1− s(yi , yj ). (9) n(n−1)
4
RL Objectives Reshape Sample Redundancy
Same executable optimum, different finitesample training signals. With binary verifier rewards, correctness-only RLVR and Pass@k-aware objectives optimize the same executable outcome and do not explicitly encourage implementation diversity. Their main difference lies in how training credit is assigned within sampled groups. Correctness-only RLVR reinforces each successful generation independently, whereas Pass@k-aware objectives reward generations according to whether they change the success of the sampled set. As a result, these objectives may induce qualitatively different generation behaviors during training. We study whether these differences manifest as systematic changes in redundancy among sampled programs. Experimental setup. We compare how different RLVR objectives reshape executable performance and sampled-program redundancy after posttraining from the same base models. We train Qwen3-4B, Qwen3-8B (Yang et al., 2025), and Olmo3-7B (Olmo et al., 2026) with correctnessonly RLVR and Pass@k-aware objectives on MBPP (Austin et al., 2021), Code-Contest (Li et al., 2022), and TACO-Cobalt (Chen et al., 2026; Li et al., 2023). All methods use the same training budget and sampling configuration. Evaluation uses 200 sampled generations per prompt from the test set.
1≤i<j≤n
Larger values indicate lower redundancy among sampled programs. Appendix E additionally studies cluster-based diagnostics derived from JPlag 4
0.6
0.725 0.700 0.675
JPlag diversity
0.95
0.750
Pass@10
Pass@1
0.775
0.90
400
Training step
(a) Pass@1
600
0.4 0.3
0.85 200
Base-RLVR PKPO Pass@K-RLVR
0.5
200
400
600
200
Training step
400
600
Training step
(b) Pass@10
(c) JPlag diversity
Figure 1: Training dynamics for Qwen3-4B on MBPP. Correctness-only RLVR improves Pass@1 while reducing JPlag diversity, indicating increasing concentration on near-duplicate implementations. In contrast, pass@k-aware objectives systematically maintain higher diversity and primarily improve larger-budget performance (Pass@10).
Correctness-only RLVR concentrates repeated implementations. Across models and datasets, correctness-only RLVR tends to improve executable reliability while concentrating sampled generations around repeated implementations. Figure 1 illustrates this behavior for Qwen3-4B on MBPP: while Base-RLVR improves Pass@1, JPlag diversity steadily decreases during training, indicating increasing concentration on near-duplicate solutions. Table 2 shows that this trend extends beyond a single training run. The table aggregates promptlevel comparisons between each RL-trained model and its corresponding base model across all datasets and models, resulting in 2745 total prompt-level comparisons per method. Relative to the corresponding base model, Base-RLVR decreases JPlag diversity in 57.2% of comparisons and decreases Vendi score in 65.3% of comparisons. Although the mean JPlag-diversity change remains modest overall, the aggregate direction is consistent with correctness-only RLVR repeatedly reinforcing a narrower set of successful implementations during training.
the same executable outcome as correctness-only RLVR. This effect emerges without any explicit diversity reward, suggesting that group-level Pass@k credit assignment changes how the finite sampling budget is used during training. Taken together, these results suggest that repeated-sampling RL objectives differ not only in executable performance, but also in the redundancy structure of the sampled programs they produce. This raises a natural question: can directly discouraging redundant samples make finite sampling budgets more effective? Executable Meth.
Pass@k-aware objectives reduce redundancy among sampled programs. In contrast to correctness-only RLVR, Pass@k-aware objectives systematically maintain lower redundancy among sampled generations during training. In Table 2, Pass@k-RLVR increases JPlag diversity in 66.9% of comparisons and Vendi score in 73.3% of comparisons. PKPO exhibits the same overall JPlagdiversity trend, increasing JPlag diversity in 60.0% of comparisons. Together with the trajectories in Figure 1 and App. H, these results indicate that repeatedsampling objectives implicitly reduce redundancy among sampled programs, despite optimizing
Change p@1
Redundancy
p@10 p@100 JDiv
Vendi
Base-RLVR
↑% ↓% ∆
45.5% 37.6% 24.5% 36.1% 33.3% 13.5% 10.6% 5.6% 57.2% 65.3% 9.5 11.5 10.6 -0.046 -0.53
Pass@k-RLVR
↑% ↓% ∆
38.9% 35.4% 25.8% 66.9% 73.3% 29.6% 19.7% 5.5% 32.4% 26.6% 4.5 11.3 12.4 0.123 0.32
PKPO
↑% ↓% ∆
45.7% 42.4% 30.0% 60.0% 57.6% 26.7% 15.7% 3.6% 38.9% 42.4% 7.2 14.9 15.5 0.083 -0.13
JPlag-RLVR (Our)
↑% ↓% ∆
48.4% 46.2% 32.3% 77.4% 73.8% 28.7% 12.1% 2.2% 22.6% 26.2% 16.4 19.7 17.3 0.298 0.33
Table 2: Aggregate changes after RLVR post-training, comparing each post-trained model against its corresponding base model on the same prompt across all evaluated datasets and models. Each comparison corresponds to one prompt evaluated before and after posttraining. For each metric, ↑% and ↓% report the fraction of comparisons where the post-trained model improves or decreases relative to the base model, while ∆ reports the mean signed change. Correctness-only RLVR tends to improve executable performance while increasing redundancy among sampled programs, whereas repeatedsampling objectives more consistently improve both executable performance and JPlag diversity.
5
5
Redundancy-Aware RLVR
5.1
Proposed Method
Direct anti-redundancy improves finite-budget performance. Table 3 shows that directly optimizing anti-redundancy is effective despite its simplicity. Across models and benchmarks, JPlagRLVR often matches or outperforms specialized Pass@k-aware objectives while producing substantially lower redundancy among sampled programs. The gains are especially strong on MBPP, where JPlag-RLVR achieves the best Pass@1 and Pass@10 results for all three models, and on TACO, where it gives the strongest results for Qwen3-4B and Qwen3-8B. For example, on MBPP with Qwen3-8B, JPlagRLVR improves Pass@1 from 79.1 to 90.0 and Pass@10 from 88.0 to 98.5, while increasing JPlag diversity from 0.305 to 0.927. On TACO with Qwen3-8B, it improves Pass@10 from 48.2 to 55.5 and Pass@100 from 58.5 to 68.7, again with the highest JPlag diversity among all methods.
We answer this question with a minimal intervention: augment the standard executable RLVR objective with a group-level anti-redundancy reward. The goal is not to reward diversity in isolation, but to discourage sampled groups from containing near-duplicate implementations while preserving executable correctness as the primary training signal. Direct anti-redundancy training. For a sampled group Y (n) = {y1 , . . . , yn }, we define a scalar anti-redundancy reward Rdiv (x, Y (n) ) from pairwise relations between generated programs. We combine it with the correctness reward Rcorr from Equation 6: R(x, Y (n) ) = Rcorr (x, Y (n) ) + λdiv Rdiv (x, Y (n) ),
(10)
Anti-redundancy reshapes the performance– redundancy tradeoff. Figure 2 summarizes this effect on MBPP. While the primary objective remains improved repeated-sampling performance, the JPlag-diversity axis provides an interpretable view of how different RL objectives use the sampling budget. Relative to correctness-only RLVR, JPlag-RLVR achieves higher executable performance together with lower measured redundancy among sampled programs. This supports the central hypothesis of the paper: directly discouraging near-duplicate generations can make finite sampling budgets more effective.
where λdiv controls the strength of the antiredundancy term. Marginal contribution within sampled groups. Because Rdiv is group-level, its training signal should depend on each sample’s marginal contribution to the sampled set. We therefore use the same leave-one-out construction as the repeatedsampling objectives in Section 3.1. For the diversity component, the advantage of sample i is (n)
Adiv = Rdiv (x, Y (n) ) − Rdiv (x, Y−i ), i
(11)
JPlag-RLVR. Our main instantiation uses JPlagdiversity from Equation 9 as Rdiv . This yields a simple redundancy-aware RLVR objective: retain the executable reward, but add an explicit penalty against repeated samples within Y (n) . We also evaluate simpler lexical and embedding-based variants as controls in Section 5.3.
Gains are broadly distributed across prompts. The improvements from JPlag-RLVR are not driven by a few outlier tasks. Table 2 shows that JPlagRLVR improves the largest fraction of promptlevel comparisons for Pass@1, Pass@10, and Pass@100, while also giving the strongest average executable gains. These gains coincide with the largest increase in JPlag diversity. Appendix E further shows that JPlag-RLVR generally increases correct-only JPlag diversity and cluster dispersion, suggesting that anti-redundancy training increases useful variation rather than merely producing diverse incorrect outputs.
5.2
5.3
(n) where Y−i = Y (n) \ {yi }. This assigns positive
advantage to samples that make the group less redundant and negative advantage to samples that mainly duplicate other generations.
Main Results
Experimental setup. We use the experimental setup as in Section 4. Additional training details, including optimization budgets, hyperparameters, and dataset splits, are provided in Appendix F.
Ablations and Analysis
Simpler anti-redundancy rewards partially help. We compare JPlag-based rewards with simpler group-level anti-redundancy signals in shorter experiments on Qwen3-4B on MBPP (Appendix C). 6
Base model
0.3
0.4
0.5
0.6
JPlag diversity
0.7
0.8
0.975 PKPO 0.950 0.925 0.900 Base-RLVR Pass@K-RLVR 0.875 0.850 Base model 0.825 0.3 0.4 0.5 0.6 0.7 0.8 0.9
Pass@10
PKPO
JPlag-RLVR
JPlag-RLVR
JPlag-RLVR Pass@K-RLVR
Pass@10
Pass@10
0.975 0.950 0.925 0.900 0.875 Base-RLVR 0.850 0.825
JPlag diversity
JPlag diversity
(a) Qwen3-4B
0.94 0.92 0.90 0.88 PKPO 0.86 Base-RLVR 0.84 Pass@K-RLVR 0.82 Base model 0.80 0.55 0.60 0.65 0.70 0.75 0.80 0.85
(b) Qwen3-8B
(c) Olmo3-7B
Figure 2: Tradeoff between repeated-sampling performance and redundancy on MBPP. The y-axis reports Pass@10; the x-axis reports JPlag diversity, where higher values indicate less plagiarism-style redundancy. Arrows show movement from the base model to each training objective. Methods that move up and right improve multi-sample success while reducing redundancy. MBPP
Code-Contest
TACO
p@1 p@10 p@100 JDiv.
1gDiv. Vendi p@1 p@10 p@100 JDiv. 1gDiv. Vendi p@1 p@10 p@100 JDiv.
1gDiv. Vendi
72.4 73.6 72.5
0.153 0.218 0.279
0.363 0.475 0.430
Q WEN 3-4 B Base-RLVR Pass@k-RLVR PKPO
82.4 87.8 86.5
86.8 92.9 92.2
0.301 0.389 0.462
1.23 1.30 1.47
16.8 12.1 13.4
31.5 29.3 31.7
39.6 41.4 46.0
0.396 0.375 0.572 0.485 0.410 0.350
2.32 3.29 2.40
24.0 18.4 19.5
39.8 39.9 42.5
48.9 55.4 56.0
0.357 0.503 0.497
2.13 3.04 2.62
JPlag-RLVR(Our) 93.2∗ 98.9∗ 99.3∗ 0.822∗ 0.446∗ 1.82∗ 17.8 35.3∗
46.2
0.523 0.460
2.92 27.4∗ 46.7∗ 60.2∗ 0.640∗ 0.526∗ 3.29∗
33.5 33.9 32.0
45.6 47.6 45.6
0.370 0.348 0.495 0.455 0.410 0.366
2.37 3.39 2.43
29.2 15.8 25.5
JPlag-RLVR(Our) 90.0∗ 98.5∗ 99.2∗ 0.927∗ 0.598∗ 2.71∗ 20.3∗ 35.8
47.7
0.466 0.411
2.39
30.3 55.5∗ 68.7∗ 0.696∗ 0.548∗ 3.34
Q WEN 3-8 B Base-RLVR Pass@k-RLVR PKPO
79.1 74.5 79.3
88.0 89.5 97.1
90.8 93.6 99.0
0.305 0.510 0.583
0.143 0.326 0.342
1.24 1.77 1.78
17.4 13.5 15.0
48.2 37.7 51.0
58.5 54.1 64.7
0.337 0.522 0.501
0.362 0.478 0.412
2.33 3.89 2.95
O LMO 3-7 B 14.5 7.4 11.9
29.0 20.1 30.5
41.2 35.0 45.0
0.465 0.438 0.536 0.554 0.516 0.496
3.20 4.99 3.58
26.3 14.7 19.7
45.5 37.4 44.1
58.0 55.1 59.3
0.426 0.544 0.573
0.435 0.505 0.552
2.75 3.59 3.47
JPlag-RLVR(Our) 71.6∗ 95.0∗ 98.2∗ 0.872∗ 0.598∗ 2.71∗ 14.3
30.1
41.2
0.495 0.474
3.33
26.0
46.5
58.9
0.549
0.503
3.15
Base-RLVR Pass@k-RLVR PKPO
64.8 60.9 64.7
84.1 83.7 86.7
91.9 92.5 94.0
0.555 0.632 0.569
0.395 0.446 0.373
1.81 2.21 1.85
Table 3: Main executable performance and redundancy diagnostics across models, datasets, and training objectives. Pass@1, Pass@10, and Pass@100 report finite-budget executable success. JDiv. denotes JPlag diversity, 1gDiv. denotes 1-gram diversity, and Vendi denotes embedding-space diversity; higher is better for all three redundancy diagnostics. A star (*) marks methods significantly better than all non-diversity baselines under a paired bootstrap test over prompts (p < 0.05).
We evaluate entropy regularization, lexical 1-gram diversity, and Vendi-embedding-based diversity rewards. Among these alternatives, 1-gram diversity gives the strongest repeated-sampling gains, suggesting that even simple lexical anti-redundancy signals can improve finite-budget performance. However, JPlag-RLVR gives the strongest redundancy diagnostics and the best Pass@100, indicating that code-structure-aware similarity provides a more effective signal for reducing repeated implementations. Entropy and Vendi-based rewards are weaker in this focused comparison.
Base-RLVR achieves higher Pass@100 and lower measured redundancy, but these improvements remain substantially smaller than those obtained with explicit anti-redundancy training and come at the cost of lower Pass@1. In particular, JPlagRLVR achieves much larger gains in Pass@10 and Pass@100 while further reducing redundancy. This suggests that effective anti-redundancy training is not equivalent to simply increasing sampling entropy. Anti-redundancy gains are not explained by trivial verbosity. A possible concern is that antiredundancy rewards could be satisfied through superficial reward hacking, such as longer completions, markdown artifacts, or non-executable text rather than genuinely different programs. We do not find evidence for this explanation: both raw
Anti-redundancy is not just higher sampling entropy. Increasing sampling temperature provides a simple test of whether the gains come merely from more random generations at inference time. As temperature increases from T = 1 to T = 2, 7
0.95
1.00
0.85 0.80
JPlag diversity
Pass@10
Pass@1
0.90 0.95 0.90
0.75 0.85
0.8 Base-RLVR JPlag-RLVR λdiv = 2.0 JPlag-RLVR λdiv = 4.0
0.6
0.4
0.70 200
400
Training step
(a) Pass@1
600
200
400
600
200
Training step
400
600
Training step
(b) Pass@10
(c) JPlag diversity
Figure 3: Effect of the JPlag-RLVR coefficient during training for Qwen3-4B on MBPP. Larger diversity weight produces a stronger increase in JPlag diversity and, in this run, also improves Pass@10.
completion lengths and extracted executable-code lengths remain broadly comparable across methods (Appendix G). This suggests that the gains are not simply caused by producing longer or superficially modified outputs.
Method
T
p@1 p@10 p@100 JDiv
Vendi
Base-RLVR Base-RLVR Base-RLVR JPlag-RLVR
1 1.5 2 1
75.0 74.4 70.9 93.2
1.21 1.29 1.47 1.82
85.4 86.7 87.4 98.9
90.0 92.0 92.4 99.3
0.297 0.356 0.417 0.822
Redundancy reduction must be paired with correctness. Optimizing anti-redundancy without executable correctness is also insufficient. As a sanity check, we train Qwen3-4B on MBPP using only the JPlag diversity reward, removing the correctness term. Appendix B shows that this objective rapidly increases JPlag diversity, but executable performance collapses. Thus, lower redundancy is useful only when coupled with the verifier reward: the goal is not diversity in isolation, but non-redundant correct generations under a finite sampling budget.
Table 4: Temperature ablation for Qwen3-4B on MBPP. Increasing the sampling temperature for BaseRLVR trades lower Pass@1 for higher Pass@10, higher Pass@100, and higher JPlag diversity. Explicit antiredundancy rewards still achieve substantially stronger multi-sample performance.
Balancing correctness and redundancy. Figure 3 studies the effect of the anti-redundancy coefficient on Qwen3-4B for MBPP. Compared with Base-RLVR, both redundancy-aware runs increase JPlag diversity throughout training and improve Pass@10. Larger coefficients reduce redundancy more strongly, but can slightly weaken executable reliability despite higher diversity. This illustrates the central tradeoff of redundancy-aware RLVR: anti-redundancy is useful as a controlled training signal, not as an objective to maximize independently of correctness.
6
executable performance. Its variance is broadly comparable to the other RLVR objectives, suggesting that the anti-redundancy reward improves finitebudget behavior without introducing additional instability in these experiments.
Conclusion
We studied redundancy as a hidden axis of repeatedsampling RLVR for code generation. Our results show that objectives with similar executable rewards can induce different sampled-program behavior: correctness-only RLVR often concentrates generations around repeated implementations, whereas Pass@k-aware objectives tend to maintain lower redundancy and improve larger-budget performance. We then showed that this redundancy signal can be used directly. A simple JPlag-based antiredundancy objective generally improves finitebudget Pass@k performance, often matching or outperforming specialized Pass@k-aware methods while increasing useful variation among successful generations. Overall, these results suggest that finite-budget code generation should not be understood only through marginal correctness. How a model allocates its samples across repeated or distinct implementations is itself a useful diagnostic and a practical optimization target.
Redundancy-aware training remains effective across random seeds. The improvements from JPlag-RLVR persist across independent training runs. We evaluate five seeds for Qwen3-4B on MBPP and TACO, covering both a relatively highsuccess short-form benchmark and a more difficult competitive-programming benchmark. Across these runs, JPlag-RLVR maintains substantially higher JPlag diversity than the baselines and achieves the strongest average repeated-sampling 8
Limitations
methods, it may also exploit incomplete tests rather than satisfy the intended specification. We restrict our experiments to standard programming benchmarks and sandboxed execution.
JPlag as a redundancy diagnostic. Our analysis relies on JPlag as a practical diagnostic for implementation-level redundancy in generated code. Compared with lexical overlap metrics, JPlag is substantially less sensitive to superficial edits such as variable renaming or formatting changes, making it useful for studying repeated implementations in sampled code generations. However, it remains an approximate structural similarity measure rather than a semantic equivalence oracle: two programs can implement the same underlying algorithm while receiving low JPlag similarity, and conversely structurally similar programs may differ semantically. Nevertheless, our results show that even this relatively simple structural signal is already useful both as a diagnostic and as a training objective, suggesting that richer program-analysis or semantic-equivalence tools could further improve redundancy-aware training in future work.
Acknowledgements This work was supported by the French government under the management of Agence Nationale de la Recherche as part of the “Investissements d’avenir” program, reference ANR-19-P3IA0001. This work was granted access to the HPC resources of IDRIS under the allocations 2025AD011014053R2, 2025-A0181016159, and 2025AD011014022R3 made by GENCI.
References Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, and Charles Sutton. 2021. Program synthesis with large language models. Preprint, arXiv:2108.07732.
Scale and benchmark scope. Our experiments are limited to three open-weight models and competitive-programming-style benchmarks involving relatively short executable programs. Because RLVR training with repeated sampling, sandbox execution, and pairwise similarity computation is computationally expensive, we do not extensively explore larger-scale models, broader hyperparameter sweeps, or seed variation across all model–dataset settings. We include a targeted multi-seed evaluation for Qwen3-4B on MBPP and TACO, but broader stability studies remain future work. It also remains unclear whether the same redundancy patterns hold for long-horizon software engineering tasks, multi-file repositories, or agentic coding settings.
Bytedance-Seed-Foundation-Code-Team, :, Yao Cheng, Jianfeng Chen, Jie Chen, Li Chen, Liyu Chen, Wentao Chen, Zhengyu Chen, Shijie Geng, Aoyan Li, Bo Li, Bowen Li, Linyi Li, Boyi Liu, Jiaheng Liu, Kaibo Liu, Qi Liu, Shukai Liu, and 37 others. 2025. Fullstack bench: Evaluating llms as full stack coders. Preprint, arXiv:2412.00535. Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, and 39 others. 2021. Evaluating large language models trained on code. Preprint, arXiv:2107.03374. Ziru Chen, Dongdong Chen, Ruinan Jin, Yingbin Liang, Yujia Xie, and Huan Sun. 2026. Bridging online and offline rl: Contextual bandit learning for multi-turn code generation. Preprint, arXiv:2602.03806.
Dependence on executable verifiers. Like other RLVR methods, our approach depends on executable reward signals derived from tests or task checkers. Weak or incomplete test suites can reward programs that satisfy the verifier without fully solving the intended task. More generally, our framework applies most naturally to domains where correctness can be checked automatically.
Dan Friedman and Adji Bousso Dieng. 2023. The vendi score: A diversity evaluation metric for machine learning. Transactions on Machine Learning Research. Jonas Gehring, Kunhao Zheng, Jade Copet, Vegard Mella, Taco Cohen, and Gabriel Synnaeve. 2025. RLEF: Grounding code LLMs in execution feedback with reinforcement learning. In Proceedings of the 42nd International Conference on Machine Learning, volume 267 of Proceedings of Machine Learning Research, pages 19034–19055. PMLR.
Potential risks. By improving finite-budget code generation, our method could also make repeatedsampling attacks more effective by reducing the number of attempts needed to elicit harmful or insecure code from models, although we do not investigate this behavior. Like other verifier-based
Aaron Grattafiori and 1 others. 2024. The llama 3 herd of models. Preprint, arXiv:2407.21783.
9
Daya Guo and 1 others. 2025. Deepseek-r1 incentivizes reasoning in llms through reinforcement learning. Nature, 645(8081):633–638.
Yujia Li, David Choi, Junyoung Chung, Nate Kushman, Julian Schrittwieser, Rémi Leblond, Tom Eccles, James Keeling, Felix Gimeno, Agustin Dal Lago, Thomas Hubert, Peter Choy, Cyprien de Masson d’Autume, Igor Babuschkin, Xinyun Chen, PoSen Huang, Johannes Welbl, Sven Gowal, Alexey Cherepanov, and 7 others. 2022. Competitionlevel code generation with alphacode. Science, 378(6624):1092–1097.
Zican Hu, Shilin Zhang, Yafu Li, Jianhao Yan, Xuyang Hu, Leyang Cui, Xiaoye Qu, Chunlin Chen, Yu Cheng, and Zhi Wang. 2026. Diversityincentivized exploration for versatile reasoning. In The Fourteenth International Conference on Learning Representations.
Team Olmo and 1 others. 2026. Olmo 3. Preprint, arXiv:2512.13961.
Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Keming Lu, Kai Dang, Yang Fan, Yichang Zhang, An Yang, Rui Men, Fei Huang, Bo Zheng, Yibo Miao, Shanghaoran Quan, and 5 others. 2024. Qwen2.5-coder technical report. Preprint, arXiv:2409.12186.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, and 2 others. 2019. PyTorch: An imperative style, high-performance deep learning library. In Advances in Neural Information Processing Systems, volume 32.
Juyong Jiang, Fan Wang, Jiasi Shen, Sungju Kim, and Sunghun Kim. 2026. A survey on large language models for code generation. ACM Trans. Softw. Eng. Methodol., 35(2).
Lutz Prechelt, Guido Malpohl, and Michael Philippsen. 2002. Finding plagiarisms among a set of programs with jplag. Journal of Universal Computer Science, 8(11):1016–1038.
Robert Kirk, Ishita Mediratta, Christoforos Nalmpantis, Jelena Luketina, Eric Hambro, Edward Grefenstette, and Roberta Raileanu. 2024. Understanding the effects of RLHF on LLM generalisation and diversity. In The Twelfth International Conference on Learning Representations.
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. 2024. DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models. arXiv preprint. ArXiv:2402.03300 [cs].
Daria Kryvosheieva, Saba Sturua, Michael Günther, Scott Martens, and Han Xiao. 2025. Efficient code embeddings from code generation models. Preprint, arXiv:2508.21290.
Yunhao Tang, Kunhao Zheng, Gabriel Synnaeve, and Remi Munos. 2025. Optimizing language models for inference time objectives using reinforcement learning. In Forty-second International Conference on Machine Learning.
Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient memory management for large language model serving with PagedAttention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles, pages 611–626.
Leandro von Werra, Younes Belkada, Lewis Tunstall, Edward Beeching, Tristan Thrush, Nathan Lambert, Shengyi Huang, Kashif Rasul, and Quentin Gallouedec. 2020. TRL: Transformer reinforcement learning. https://github.com/huggingface/ trl.
Florian Le Bronnec, Alexandre Verine, Benjamin Negrevergne, Yann Chevaleyre, and Alexandre Allauzen. 2024. Exploring precision and recall to assess the quality and diversity of LLMs. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 11418–11441, Bangkok, Thailand. Association for Computational Linguistics.
Christian Walder and Deep Tejas Karkhanis. 2026. Pass@k policy optimization: Solving harder reinforcement learning problems. In The Thirty-ninth Annual Conference on Neural Information Processing Systems.
Seonghyeon Lee, HeeJae Chon, Joonwon Jang, Dongha Lee, and Hwanjo Yu. 2025. How diversely can language models solve problems? exploring the algorithmic diversity of model-generated code. In Findings of the Association for Computational Linguistics: EMNLP 2025, pages 152–167, Suzhou, China. Association for Computational Linguistics.
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, and 3 others. 2020. Transformers: State-of-the-art natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 38–45. Association for Computational Linguistics.
Rongao Li, Jie Fu, Bo-Wen Zhang, Tao Huang, Zhihong Sun, Chen Lyu, Guang Liu, Zhi Jin, and Ge Li. 2023. Taco: Topics in algorithmic code generation dataset. Preprint, arXiv:2312.14852.
10
An Yang and 1 others. 2025. Qwen3 technical report. Preprint, arXiv:2505.09388. Kunhao Zheng, Juliette Decugis, Jonas Gehring, Taco Cohen, benjamin negrevergne, and Gabriel Synnaeve. 2025. What makes large language models reason in (multi-turn) code generation? In The Thirteenth International Conference on Learning Representations.
11
A
Random Seed Variation
To evaluate robustness across training runs, we additionally perform five independent runs for Qwen3-4B on both MBPP and TACO. These benchmarks provide complementary regimes, ranging from relatively high-success short-form problems (MBPP) to more difficult competitive-programming tasks (TACO). Table A.5 shows that the qualitative trends of the paper remain stable across seeds. In particular, JPlag-RLVR consistently produces substantially higher JPlag diversity than the other RLVR objectives while maintaining strong repeated-sampling executable performance. Although RLVR objectives exhibit noticeable variance across runs, the anti-redundancy reward does not appear to introduce additional instability relative to the baselines in these experiments. MBPP
Method p@1
p@10
p@100
TACO
JDiv
1gDiv
Vendi
p@1
p@10
p@100
JDiv
1gDiv
Vendi
QWEN 3-4 B
Base-RLVR 72.3 ± 1.6 82.7 ± 0.5 87.3 ± 0.5 0.306 ± 0.024 0.158 ± 0.017 1.23 ± 0.05 25.2 ± 1.4 40.8 ± 1.2 51.2 ± 2.0 0.368 ± 0.049 0.363 ± 0.035 2.31 ± 0.15 Pass@k-RLVR 71.6 ± 3.2 87.6 ± 3.6 93.2 ± 3.0 0.466 ± 0.065 0.285 ± 0.062 1.58 ± 0.20 15.7 ± 2.8 35.1 ± 4.4 49.7 ± 4.6 0.470 ± 0.034 0.445 ± 0.043 3.23 ± 0.36 PKPO 79.0 ± 8.0 92.4 ± 6.0 95.7 ± 3.2 0.474 ± 0.055 0.335 ± 0.069 1.74 ± 0.24 20.6 ± 1.4 42.3 ± 1.1 56.4 ± 1.7 0.448 ± 0.043 0.399 ± 0.020 2.68 ± 0.10 JPlag-RLVR(Our) 89.8 ± 7.5 97.3 ± 3.7 98.5 ± 1.8 0.769 ± 0.049 0.454 ± 0.028 1.94 ± 0.18 26.5 ± 2.4 46.7 ± 1.4 59.0 ± 1.3 0.631 ± 0.024 0.515 ± 0.019 3.29 ± 0.17
Table A.5: Five-seed results for Qwen3-4B on MBPP and TACO. Values report mean and standard deviation across independent training runs. Across both datasets, JPlag-RLVR consistently maintains substantially higher JPlag diversity while remaining competitive or superior on repeated-sampling executable performance.
B
Diversity-Only Training Dynamics 1.0
0.8
0.4 0.2
JPlag diversity
Pass@10
Pass@1
0.6 0.6 0.4 0.2
0.0
0.8 JPlag only Base model
0.6 0.4
0.0 0
200
400
Training step
(a) Pass@1
600
0
200
400
Training step
(b) Pass@10
600
0
200
400
600
Training step
(c) JPlag diversity
Figure B.4: Diversity-only training for Qwen3-4B on MBPP. Optimizing the diversity reward without executable correctness increases JPlag diversity, but collapses executable performance. Effective anti-redundancy training must balance correctness and redundancy.
Figure B.4 provides the training dynamics for the diversity-only sanity check discussed in Section 5.3. The objective increases JPlag diversity, but without executable correctness it collapses finite-budget performance.
C
Alternative Diversity Reward Ablations
Table C.6 compares alternative group-level diversity rewards on Qwen3-4B and MBPP. Distributional and embedding-space rewards are weak. Entropy regularization does not improve over Base-RLVR, with slightly lower executable performance and nearly unchanged diversity diagnostics. The Vendi-based reward is also comparatively weak in this setting, leaving most metrics close to or below Base-RLVR, including the Vendi score itself. This suggests that generic distributional spreading or embedding-space diversity may not provide a sufficiently useful training signal for finite-budget code generation in this setup. Surface anti-redundancy already helps. In contrast, rewards that more directly discourage repeated generations produce substantially stronger results. The lexical 1-gram reward already yields large improvements over Base-RLVR, showing that even simple surface-level anti-redundancy can be an effective training signal for finite-budget code generation. The JPlag-based reward performs best overall, 12
p@1 p@10 p@100 JDiv. 1gDiv. Vendi Base-RLVR 75.1 85.3 Entropy 74.3 82.8 Vendi-div 76.3 85.4 1-gram-div 86.5 96.3 JPlag-RLVR(Our) 93.2∗ 98.9∗
90.5 0.301 0.167 87.6 0.302 0.155 90.0 0.285 0.156 98.4 0.457 0.269 99.3 0.822∗ 0.446∗
1.21 1.20 1.19 1.52 1.82
Table C.6: Alternative group-level diversity rewards on Qwen3-4B and MBPP. Pass@1, Pass@10, and Pass@100 report finite-budget executable success; JDiv, 1gDiv, and Vendi report JPlag, 1-gram, and embedding-space diversity, respectively. Lexical 1-gram anti-redundancy already improves repeated-sampling performance over Base-RLVR, while the JPlag-based reward gives the strongest overall results and redundancy diagnostics. Entropy and Vendibased rewards are weaker in this comparison. A star (*) marks methods significantly better than all non-diversity baselines under a paired bootstrap test over prompts (p < 0.05).
achieving the strongest executable performance together with the strongest redundancy diagnostics across all reported metrics. JPlag better targets relevant redundancy. These results support two conclusions. First, the gains are not specific to a single implementation of the reward: reducing repeated generations is itself useful under finite sampling budgets. Second, the stronger performance of JPlag over 1-gram diversity supports our use of a code-specific redundancy signal rather than a purely lexical one. Surface-level anti-redundancy already helps, but JPlag provides a better-aligned training signal for discouraging repeated implementations.
D
Metrics Discussions
Relationship between diversity diagnostics. Figure D.5 shows Pearson correlations between diversity metrics aggregated across all evaluated models and training objectives for each dataset. JPlag diversity and lexical diversity are strongly correlated across datasets, suggesting that lexical overlap often tracks implementation redundancy reasonably well. In contrast, Vendi exhibits only moderate correlation with both metrics, indicating that embedding-space dispersion captures a related but distinct notion of variation among sampled programs.
JPlag div.
1.00
0.89
0.58
JPlag div.
1.00
0.70
0.53
JPlag div.
1.00
0.86
0.55
1-gram div.
0.89
1.00
0.72
1-gram div.
0.70
1.00
0.78
1-gram div.
0.86
1.00
0.70
Vendi
0.58
0.72
1.00
Vendi
0.53
0.78
1.00
Vendi
0.55
0.70
1.00
g JPla
. div
m -gra
. div
1
(a) Code-Contest
i
d Ven
g JPla
. div
m -gra
. div
1
(b) MBPP
d Ven
i
g JPla
. div
m -gra
. div
1
i
d Ven
(c) TACO
Figure D.5: Correlation between diversity metrics across prompts pooled over all models and training methods for each dataset.
E
JPlag Cluster and Correct-Only Analysis
The main experiments report JPlag diversity as a pairwise redundancy diagnostic. Here we add a coarser cluster-based view of the sampled implementations. For each prompt, we construct a threshold graph over generated programs by connecting two programs when their JPlag similarity exceeds τ = 0.7. The connected components of this graph define groups of near-duplicate implementations. 13
MBPP p@10 JDiv
Eff
Code-Contest JDiv-c Eff-c p@10 JDiv
Eff
TACO
JDiv-c Eff-c p@10 JDiv
Eff
JDiv-c Eff-c
Q WEN 3-4 B Base-RLVR Pass@k-RLVR PKPO JPlag-RLVR(Our)
82.4 87.8 86.5 98.9
0.301 16.95 0.389 16.81 0.462 14.88 0.822 19.23
0.275 0.355 0.432 0.811
15.35 13.94 12.93 18.73
31.5 29.3 31.7 35.3
0.396 2.67 0.572 47.59 0.410 7.57 0.523 11.54
0.315 0.411 0.312 0.420
2.00 3.27 1.84 2.93
39.8 39.9 42.5 46.7
0.357 4.62 0.503 22.50 0.497 12.69 0.640 37.95
0.274 0.397 0.378 0.549
1.72 2.98 2.69 6.04
88.0 89.5 97.1 98.5
0.305 18.95 0.510 12.19 0.583 14.81 0.927 90.64
0.275 0.477 0.555 0.921
17.44 10.94 12.28 81.19
33.5 33.9 32.0 35.8
0.370 2.55 0.495 15.31 0.410 8.16 0.466 8.51
0.279 0.378 0.317 0.397
1.59 2.30 1.83 2.34
48.2 37.7 51.0 55.5
0.337 2.81 0.522 18.25 0.501 18.09 0.696 35.21
0.263 0.377 0.395 0.623
1.69 2.61 3.13 8.34
84.1 83.7 86.7 95.0
0.555 22.27 0.632 20.88 0.569 20.11 0.872 28.73
0.502 0.585 0.523 0.842
13.16 12.94 12.71 18.59
29.0 20.1 30.5 30.1
0.465 5.68 0.536 11.66 0.516 19.30 0.495 12.89
0.335 0.433 0.403 0.382
1.76 2.86 2.51 2.06
45.5 37.4 44.1 46.5
0.426 8.63 0.544 24.64 0.573 28.37 0.549 11.95
0.326 0.411 0.451 0.452
2.20 3.56 4.01 2.83
Q WEN 3-8 B Base-RLVR Pass@k-RLVR PKPO JPlag-RLVR(Our) O LMO 3-7 B Base-RLVR Pass@k-RLVR PKPO JPlag-RLVR(Our)
Table E.7: JPlag-based redundancy and cluster diagnostics across all sampled programs and correct-only programs. JDiv is mean pairwise JPlag diversity. Eff is the effective number of JPlag clusters obtained from a similarity threshold graph with τ = 0.7. The −c columns compute the corresponding metrics after filtering to correct programs only.
Cluster-based redundancy diagnostics. Table E.7 reports two kinds of JPlag-based coverage diagnostics. JDiv is the JPlag diversity score from Equation (9), computed over all sampled programs. Eff is the effective number of JPlag clusters, ! X eff(Yx ) = exp − pr log pr , r
where pr is the fraction of samples assigned to cluster r. Unlike the raw number of clusters, the effective cluster count is lower when most generations fall into a few dominant implementation families. It therefore distinguishes broad, balanced implementation coverage from a long tail of rare variants. Cluster-based redundancy diagnostics. Table E.7 reports two kinds of JPlag-based diagnostics. JDiv is the pairwise JPlag diversity score from Equation (9), computed over all sampled programs. Eff is the effective number of JPlag clusters, ! X eff(Yx ) = exp − pr log pr , r
where pr is the fraction of samples assigned to cluster r. Unlike the raw number of clusters, the effective cluster count is lower when most generations fall into a few dominant near-duplicate groups. It therefore measures whether samples are distributed across multiple substantial JPlag-based clusters, rather than dominated by a single near-duplicate group with a long tail of rare variants. Correct-only diagnostics. The columns marked with “-c” compute the same diagnostics after filtering to correct programs only. JDiv-c measures pairwise JPlag diversity among successful solutions, and Eff-c measures the effective number of JPlag clusters among successful solutions. This split is important because a model can increase diversity by generating many incorrect variants. The correct-only metrics test whether lower redundancy also appears within the useful part of the sampled set. The table shows that JPlag-RLVR usually produces the strongest JPlag-based implementation coverage on MBPP, both over all samples and among correct solutions. For example, with Qwen3-8B on MBPP, JPlag-RLVR increases JDiv from 0.305 to 0.927 and Eff from 18.95 to 90.64; after filtering to correct 14
Cluster 1
Cluster 2
def is_subset_sum( arr, n, target): achievable = set() achievable.add(0)
def is_subset_sum( nums, n, target): dp = [False] * ( target + 1) dp[0] = True
for num in arr: new_sums = set() for s in achievable: new_sums.add(s + num) achievable.update( new_sums) return target in achievable
Cluster 3
def is_subset_sum( nums, start, target): if target == 0: return True if start == len(nums): return False for num in nums: same_prev = ( for i in range(target, start > 0 and num - 1, -1): nums[start] == nums[start - 1] if dp[i - num]: ) dp[i] = True if same_prev: prev_fails = not is_subset_sum( return dp[target] nums, start - 1, target) if prev_fails: return is_subset_sum( nums, start + 1, target) remaining = target - nums[start] if is_subset_sum( nums, start + 1, remaining): return True return is_subset_sum( nums, start + 1, target)
Table E.8: Representative generations from three JPlag clusters for one MBPP prompt: “Write a function to determine if there is a subset of the given set with sum equal to the given sum.” The clusters correspond to recognizable implementation families, including set-based dynamic programming, boolean-table dynamic programming, and recursive search.
programs, JDiv-c remains high at 0.921 and Eff-c increases from 17.44 to 81.19. This indicates that the diversity gain is not merely due to incorrect exploratory outputs: successful solutions themselves are spread across many more substantial JPlag-based implementation clusters. The pattern is more heterogeneous on Code-Contest and TACO, where Pass@k-aware objectives such as Pass@k-RLVR and PKPO sometimes obtain the largest all-sample cluster dispersion. However, JPlag-RLVR remains competitive and is often strongest on correct-only diagnostics, particularly on TACO where it achieves the highest JDiv-c across all three models. These results suggest that direct anti-redundancy optimization reduces redundancy among successful implementations rather than simply increasing variation across all generated programs. Table E.8 provides illustrative examples of the structure captured by the cluster analysis. In this example, JPlag groups near-duplicate generations into coherent implementation families rather than only detecting surface-level lexical overlap. The representative clusters differ in control flow and state representation, suggesting that the effective cluster count can reflect meaningful variation across recognizable programming strategies.
F
Implementation Details
Training setup and hyperparameters. All RLVR methods use the same optimization hyperparameters within each dataset. We use a learning rate of 10−6 . Models are trained for 15 epochs on MBPP and 6 epochs on Code-Contest and TACO-Cobalt. Each optimization batch contains 8 prompts with 32 sampled generations per prompt for MBPP, and 4 prompts with 32 generations per prompt for Code-Contest and TACO-Cobalt. We reduce the number of prompts per batch on competitive-programming datasets to control execution cost during RLVR training. Optimization hyperparameters were selected using Base-RLVR validation performance and then kept fixed across all RL objectives. We use validation Pass@10 for checkpoint selection. For the baselines, we follow the recommendations of Pass@k-RLVR and PKPO original works and set the objective parameter k to match the evaluation target, i.e., k = 10. For Pass@k-RLVR, this additionally requires reducing the sampled group size to 10 generations per prompt. For redundancy-aware objectives, we select the anti-redundancy coefficient λdiv using validation Pass@10. We search over {1, 2, 4, 6} on MBPP and over {0.25, 0.5, 1.0, 1.5, 2.0} on Code-Contest and TACO-Cobalt. We use dataset-specific search grids because we empirically observed that the useful coefficient range varies across datasets, potentially due to differences in task difficulty, code structure, and 15
implementation variability. Datasets and splits. Table F.9 summarizes the train, validation, and test splits used in our experiments. We use three executable code-generation benchmarks: • MBPP (Austin et al., 2021) contains short crowd-sourced Python programming tasks designed around entry-level programming concepts, each paired with reference code and unit tests. We use the standard train/validation/test split; the public dataset is distributed under CC-BY-4.0. • Code-Contest (Li et al., 2022) contains competitive-programming problems from online judges, with problem statements, input-output tests, and human submissions in multiple languages. We use the official validation and test sets; the public release licenses code under Apache-2.0 and non-code materials under CC-BY-4.0. • TACO-Cobalt (Chen et al., 2026; Li et al., 2023) is a cleaned version of TACO focused on competitionstyle algorithmic code generation, with public and hidden tests for each task. It does not provide an official validation split, so we reserve 300 training examples for validation; the public dataset card lists an MIT license. For all datasets, we use a simple instruction template that presents the programming task and requests a Python solution in a standardized executable format: You are an expert Python programmer, and here is your task: {{ current_problem.content }} Your solution should be in Python and follow this format: ```python [your code here] ```
Dataset MBPP Code-Contest TACO-Cobalt
Train Size
Validation Size
Test Size
374 1,000/13,328 1,000/5,853
90 117 300/5,853 (train)
500 165 250
Table F.9: Dataset splits used in our experiments. Fractions indicate the subset size used out of the available split.
To control computational cost during RLVR training, we subsample 1000 training examples from Code-Contest and TACO-Cobalt. We use the full MBPP training split because of its smaller size and lower execution cost. Models and compute. We run experiments with Qwen3-4B, Qwen3-8B (Yang et al., 2025), and Olmo37B (Olmo et al., 2026), covering models with approximately 4, 8, and 7 billion parameters. Across all training, validation, and test-generation runs, the experiments used approximately 15,000 GPU hours on NVIDIA H100 and A100 GPUs. Software stack. Training is implemented in PyTorch (Paszke et al., 2019) using Hugging Face Transformers (Wolf et al., 2020) and TRL (von Werra et al., 2020). We use vLLM (Kwon et al., 2023) for inference. Sandbox and execution environment. We use SandboxFusion (Bytedance-Seed-Foundation-CodeTeam et al., 2025) as the execution backend for both training and evaluation.1 SandboxFusion provides isolated execution and benchmark evaluation for LLM-generated code across multiple programming datasets and programming languages. For all experiments, generated completions are executed inside the sandbox environment and evaluated against the corresponding unit tests or task checkers before computing RL rewards and diversity diagnostics. SandboxFusion is licensed under the Apache License, Version 2.0. 1
https://github.com/bytedance/SandboxFusion
16
JPlag implementation details. We use JPlag v6.2.0 (Prechelt et al., 2002) through a local containerized service based on the official GPL-3.0 release.2 Generated completions are first executed in the sandbox environment, from which we extract the submitted Python code before computing similarity scores. For both training and evaluation, we compute pairwise JPlag similarities between generated programs for the same prompt using the Python 3 frontend with SimilarityMetric.AVG, minimum_token_match=5. We use the returned averageSimilarity value as the pairwise similarity score. JPlag clustering is disabled during similarity computation; all cluster-based diagnostics reported in the appendix are computed afterward from the resulting pairwise similarity matrix. 1-gram diversity metric. For the lexical diversity metric, we first remove comments and docstrings from each generated program. The remaining code is then tokenized using the tiktoken tokenizer3 . Pairwise 1-gram overlap is computed over the resulting token sequences, and converted into a diversity score analogously to the JPlag-based metric. Checkpoint selection. We select checkpoints using validation Pass@10. For MBPP, validation uses the full validation split with 100 sampled generations per prompt. For Code-Contest, we use the full validation split with 50 sampled generations per prompt to control repeated-sampling evaluation cost. For TACO-Cobalt, we use the 300-example validation subset together with 50 sampled generations per prompt, again to control computational cost during validation. These validation settings are kept fixed across all methods. Reward encoding. In the main text, we write verifier rewards as ri = c(x, yi ) for readability. In the implementation, binary correctness values c(x, yi ) ∈ {0, 1} are mapped to signed rewards in {−1, 1} before applying the policy-gradient estimators. This affine transformation does not change the relative ordering of samples or the leave-one-out advantages used by the training objectives. AI usage. We used AI assistants during the development and writing process, including for code prototyping, debugging, experiment-management scripts, and language polishing. All experimental results, analyses, and paper claims were checked by the authors.
2 3
https://github.com/jplag/JPlag https://github.com/openai/tiktoken
17
G
Completion Length Diagnostics
Figure G.6 compares raw completion lengths and extracted executable-code lengths across methods. These diagnostics address the possibility that diversity rewards reduce measured redundancy through superficial reward hacking, such as producing longer conversational completions, chain-of-thought padding, or markdown artifacts. Across MBPP, Code-Contest, and TACO-Cobalt, length distributions remain broadly comparable, suggesting that the diversity gains in the main experiments are not explained by trivial length inflation. PKPO
Pass@K-RLVR
JPlag-RLVR
500 400 300 200 100 0 Olmo3-7B
Qwen3-4B
Qwen3-8B
Base-RLVR
Extracted code length (tokens)
Completion length (tokens)
Base-RLVR
Pass@K-RLVR
400 300 200 100 0 Olmo3-7B
1000 800 600 400 200 0 Qwen3-4B
Qwen3-8B
Base-RLVR
Pass@K-RLVR
1250 1000 750 500 250 0 Qwen3-4B
Pass@K-RLVR
JPlag-RLVR
600 400 200 0 Olmo3-7B
Qwen3-4B
Qwen3-8B
(d) Code-Contest, extracted code
1500
Olmo3-7B
PKPO
800
JPlag-RLVR
Qwen3-8B
Base-RLVR
Extracted code length (tokens)
Completion length (tokens)
PKPO
Qwen3-8B
1000
(c) Code-Contest, raw completions Base-RLVR
Qwen3-4B
(b) MBPP, extracted code
1200
Olmo3-7B
JPlag-RLVR
500
JPlag-RLVR
Extracted code length (tokens)
Completion length (tokens)
PKPO
Pass@K-RLVR
600
(a) MBPP, raw completions Base-RLVR
PKPO
(e) TACO-Cobalt, raw completions
PKPO
Pass@K-RLVR
JPlag-RLVR
1500 1250 1000 750 500 250 0 Olmo3-7B
Qwen3-4B
Qwen3-8B
(f) TACO-Cobalt, extracted code
Figure G.6: Completion-length diagnostics across datasets and training methods. Each row corresponds to one dataset, with raw completion lengths on the left and lengths after extracting executable code on the right. Diversity improvements are not accompanied by systematic inflation in either raw completions or extracted code.
18
H
Additional Baseline Training Trajectories
Figures H.7–H.14 provide the same three-panel view as Figure 1 for all model and dataset combinations. Together, these runs support the heterogeneous sample-redundancy pattern discussed in Section 4: verifier training reliably improves executable correctness, while changing sampled-program redundancy. The appendix trajectories cover Olmo3-7B on Code-Contest and TACO-Cobalt, plus Qwen3-4B and Qwen3-8B on MBPP, Code-Contest, and TACO-Cobalt. They show additional cases where executable performance and JPlag diversity evolve differently across baseline objectives. 0.60
0.175
Pass@10
Pass@1
0.125 0.100
JPlag diversity
0.30
0.150
0.25 0.20
0.075
0.55
Base-RLVR PKPO Pass@K-RLVR
0.50 0.45
0.15
0.050 250
500
750
1000
1250
1500
250
500
Training step
750
1000
1250
250
1500
500
(a) Pass@1
750
1000
1250
1500
Training step
Training step
(c) JPlag diversity
(b) Pass@10
Figure H.7: Baseline training dynamics for Olmo3-7B on Code-Contest.
0.35 0.55
0.25
JPlag diversity
0.30
Pass@10
Pass@1
0.55 0.50 0.45
Base-RLVR PKPO Pass@K-RLVR
0.50
0.45
0.40 0.20 250
500
750
1000
1250
1500
250
500
Training step
750
1000
1250
250
1500
500
(a) Pass@1
750
1000
1250
1500
Training step
Training step
(c) JPlag diversity
(b) Pass@10
Figure H.8: Baseline training dynamics for Olmo3-7B on TACO-Cobalt.
JPlag diversity
0.6
0.95
0.750
Pass@10
Pass@1
0.775
0.725 0.700 0.675
0.90
400
Training step
(a) Pass@1
600
0.4 0.3
0.85 200
Base-RLVR PKPO Pass@K-RLVR
0.5
200
400
600
Training step
(b) Pass@10
200
400
(c) JPlag diversity
Figure H.9: Baseline training dynamics for Qwen3-4B on MBPP.
19
600
Training step
0.35
0.15
JPlag diversity
Pass@10
Pass@1
0.20 0.30
0.25
0.6 Base-RLVR PKPO Pass@K-RLVR
0.5 0.4
0.10 0.3
250
500
750
1000
1250
1500
250
500
Training step
750
1000
1250
250
1500
500
(a) Pass@1
750
1000
1250
1500
Training step
Training step
(c) JPlag diversity
(b) Pass@10
Figure H.10: Baseline training dynamics for Qwen3-4B on Code-Contest. 0.34
Pass@10
Pass@1
0.30 0.28
JPlag diversity
0.55
0.32
0.50
0.26 0.45
0.50 Base-RLVR PKPO Pass@K-RLVR
0.45 0.40 0.35
0.24 250
500
750
1000
1250
1500
250
500
Training step
750
1000
1250
250
1500
500
(a) Pass@1
750
1000
1250
1500
Training step
Training step
(c) JPlag diversity
(b) Pass@10
Figure H.11: Baseline training dynamics for Qwen3-4B on TACO-Cobalt. 1.000
0.82
0.6
Pass@10
Pass@1
0.80 0.78
0.950 0.925
0.76
0.900
0.74
0.875 200
400
JPlag diversity
0.975
600
200
Training step
400
0.5 Base-RLVR PKPO Pass@K-RLVR
0.4 0.3
200
600
(a) Pass@1
400
600
Training step
Training step
(c) JPlag diversity
(b) Pass@10
Figure H.12: Baseline training dynamics for Qwen3-8B on MBPP. 0.50
JPlag diversity
0.36
0.20 0.18
Pass@10
Pass@1
0.34
0.16
0.32 0.30
0.14 0.28
0.45
Base-RLVR PKPO Pass@K-RLVR
0.40
0.35
0.12 0.26 250
500
750
1000
1250
1500
250
500
Training step
750
1000
1250
250
1500
500
(a) Pass@1
750
1000
1250
1500
Training step
Training step
(c) JPlag diversity
(b) Pass@10
Figure H.13: Baseline training dynamics for Qwen3-8B on Code-Contest. 0.400 0.50
JPlag diversity
0.60
Pass@10
Pass@1
0.375 0.350 0.325 0.300
0.55
0.50
0.45
Base-RLVR PKPO Pass@K-RLVR
0.40 0.35 0.30
0.275 250
500
750
1000
Training step
(a) Pass@1
1250
1500
250
500
750
1000
Training step
(b) Pass@10
1250
1500
250
500
750
1000
1250
(c) JPlag diversity
Figure H.14: Baseline training dynamics for Qwen3-8B on TACO-Cobalt.
20
1500
Training step