arXiv CS open access

Energy-Efficient Reinforcement Learning for Large Language Models

Rahul Sharma et al. · arxiv_cs

arXiv CS · Papers

cloud distributed-computing parallel-computing

Reinforcement learning for language models alternates between generating responses and training the policy. These stages place different demands on GPUs, creating opportunities to reduce clock frequency with little loss in throughput. Selecting frequencies online is difficult because response length

Conceptio Open Archive

Analyze & explore related research in Conceptio

Connect this record directly into your research workflow or AI agent. Explore citations across open repositories, query full text, and export structured proof bundles.

Explore in Search →Connect via MCP / API ↗

E NERGY-E FFICIENT R EINFORCEMENT L EARNING FOR L ARGE L ANGUAGE M ODELS Rahul Sharma * Nitin Vegesna * Yanxin Zhang † NVIDIA

arXiv:2611.05748v1 [cs.DC] 3 Nov 2026

A BSTRACT Reinforcement learning for language models alternates between generating responses and training the policy. These stages place different demands on GPUs, creating opportunities to reduce clock frequency with little loss in throughput. Selecting frequencies online is difficult because response lengths change and frequency trials themselves consume time and energy. We present EE-RL (energy-efficient RL), an online controller that selects separate frequencies for generation and training during RL execution. It compares candidate settings with a prediction of the default GPU governor on the same token workload, and keeps the most energy-efficient setting that meets a throughput threshold. The controller then monitors throughput and energy and adjusts frequencies as the workload changes. Fixed-frequency sweeps on H100 and GB200 show GPU energy savings of up to 28.2% per valid training token across synchronous and asynchronous execution, while largely preserving throughput. EE-RL turns this headroom into online savings, reducing GPU energy per valid training token by up to 16.9% at a modest throughput cost. Keywords: LLM post-training, reinforcement learning, GPU energy efficiency, dynamic voltage and frequency scaling (DVFS), runtime frequency control, phase-aware optimization, asynchronous reinforcement learning, energy-aware scheduling.

I NTRODUCTION

Reinforcement learning (RL) post-training improves a language model by repeatedly generating responses, evaluating them, and updating the model (Sheng et al., 2025; Fu et al., 2025). This process can consume substantial energy. Posttraining Olmo 3 32B as a reasoning model, for example, used 17× more datacenter energy than its instruction-tuned counterpart, largely due to rollout generation (Morrison et al., 2026). The growing electricity demand of AI data centers makes energy efficiency important for operating costs and infrastructure expansion (International Energy Agency, 2025). Reducing energy consumption without substantially extending GPU allocation time can lower electricity costs while preserving resource availability. Because GPUs consume energy while computing, moving data, and waiting, improving energy efficiency requires identifying when higher clock frequencies accelerate post-training and when they merely increase power consumption. An RL update includes response generation, log-probability computation, policy training, and weight transfer to the *

Equal contribution; authors listed in random order. Rahul Sharma: rahulsha@nvidia.com. Nitin Vegesna: nvegesna@nvidia.com. † Corresponding author: Yanxin Zhang (yanxinz@nvidia.com). Preprint.

Figure 1. Phase-average GPU power in Qwen3-30B-A3B on 32 H100s under the governor and at fixed 1650/1350 MHz generation/training frequencies. The time axis omits inter-phase gaps, which the annotated changes include.

generation engine. These phases differ in arithmetic intensity (Williams et al., 2009) and therefore place different demands on the hardware. Autoregressive generation produces responses one token at a time and can be limited by memory bandwidth, while training processes batches of complete responses and typically uses more parallel computation. Weight transfers, long responses that leave the rest of a batch waiting (Gao et al., 2026), and dependencies

Energy-Efficient Reinforcement Learning for Large Language Models

between phases introduce more waiting. Default GPU management, which we call the governor, nevertheless runs all of this work near peak clock frequency, so memory-bound work and idle waiting can consume additional power without a corresponding throughput gain.

including the cost of testing and changing frequencies.

BACKGROUND : H OW RL U SES GPU R ESOURCES

This mismatch creates headroom, an opportunity to lower GPU frequency, and with it power, while losing little endto-end throughput relative to the governor. The benefit depends on whether lower power outweighs any increase in execution time. Figure 1 illustrates the opportunity by comparing power under the governor with a lower-frequency setting. Power is lower in all phases, while their durations remain similar. The available headroom varies with the workload and how generation and training are distributed across GPUs.

Higher GPU clock frequencies can speed up computeintensive work, but offer less benefit when performance is limited by memory movement or waiting. RL includes phases with different demands, and the GPU allocation also influences their execution time and compute demands. These differences create opportunities to lower frequencies— and energy consumption—with little loss in overall performance.

Finding an efficient setting in advance is expensive. It requires repeated RL runs that consume valuable GPU hours, and the result may change with the model, batch size, or hardware allocation. A setting chosen at startup can also become less suitable as the model produces longer or shorter responses (Gao et al., 2026; Sheng et al., 2026). Selecting frequencies online avoids a separate search for every workload, but the controller must distinguish these changes in work from the effect of changing frequency. It must also save enough energy to offset the cost of testing candidate settings.

Our workloads use Group Relative Policy Optimization (GRPO), which generates several responses to each prompt and compares their rewards within the group to update the model (Shao et al., 2024). We call this repeated process an update. Generation produces the responses, and training uses them to change the model’s parameters. The updated parameters are then made available to the generation engine.

We present EE-RL (energy-efficient RL), a controller that selects separate generation and training frequencies while the RL workload runs. It measures GPU energy per valid training token and throughput in valid training tokens per second, rather than per step, because response lengths vary and steps do not represent a fixed amount of work. Valid training tokens are response tokens consumed by training that contribute to the loss. The controller tests frequency settings in short trials against the governor. It keeps the most energy-efficient setting within its throughput limit, then monitors energy and throughput to adjust frequencies as needed. The paper makes three contributions: • Evidence of energy-saving headroom. Measurements on H100 and GB200 show where lower phase frequencies save energy and where longer execution offsets the power reduction. • Online per-phase frequency control. EE-RL accounts for changing token workloads and selects the most energy-efficient tested setting that satisfies its throughput bound relative to the governor. • Controller evaluation. Matched comparisons measure energy savings and throughput over the full measured workload in synchronous and asynchronous RL,

2.1

Generation, training, and useful work

Each update also includes log-probability computation and weight refit. Log-probability computation scores response tokens for the training objective, while weight refit transfers updated policy parameters to the generation engine. These stages can have different power demands, but their shorter durations make separate frequency settings less worthwhile. To limit clock-switching overhead, we set log-probability computation to use the training frequency, while weight refit runs under both. Energy measurements include all allocated GPUs, covering generation, training, weight transfer, and waiting. 2.2

How generation and training share GPUs

We consider three execution modes. In colocated synchronous execution, generation and training take turns on the same GPUs. Both contribute directly to the time required for an update, and frequency changes follow their phase boundaries. In disaggregated synchronous execution, generation and training use separate GPU pools but remain synchronized at each update. Training waits for the generated batch, and the next generation round waits for updated policy weights. Pool sizes can differ, and the waiting pool continues to consume power. In both synchronous modes, frequency changes preserve the intended update dependencies. Each training update consumes the completed batch before the next generation round uses the updated policy. Unlike asynchronous execution, changing a phase’s duration does not change which batch is

Energy-Efficient Reinforcement Learning for Large Language Models

scheduled for that update, so frequency changes should not affect the model’s final accuracy. In asynchronous execution, generation produces responses while training consumes previously generated work. A faster pool may therefore tolerate a lower frequency without slowing overall post-training progress. Timing can also affect which responses are available when training samples a batch. We define a response’s policy age as the difference between the policy version used for generation and the version against which it is consumed for training. Bounding this age limits the staleness of consumed responses. We evaluate two weight-update schedules. With drained updates, in-progress responses finish before new weights are loaded into the generation engine. With in-flight updates, generation pauses with responses still in progress, loads new weights, recomputes their KV caches, and resumes. The policy-age limits used by each schedule are reported with the results. 2.3

Power, time, and energy

The governor serves as our reference, with observed peak clocks of 1980 MHz on H100 and 2062 MHz on GB200. EERL requests GPU core frequencies while hardware manages voltage. Dynamic power scales approximately as Pdyn ∝ 2 Vdd f (Chandrakasan et al., 1992). Reducing frequency lowers switching activity and can also permit lower voltage. Energy savings depend on execution time as well as power. Slowing one phase can keep both active and waiting GPUs drawing power for longer. Over a measurement interval of RT P duration T , we define energy as E = 0 g∈G Pg (t) dt, throughput as q = N/T , and energy per valid training token as e = E/N , where G is the allocated GPU set, Pg (t) is the power of GPU g, and N counts valid training tokens. Energy includes every allocated GPU, whether active or waiting, but excludes CPU, network, and cooling energy.

C HARACTERIZING E NERGY-S AVING H EADROOM

We now quantify this headroom. We sweep reduced generation and training frequencies and compare their energy per valid training token and end-to-end throughput with the governor across workloads, GPU allocations, and execution modes. 3.1

Where power can be reduced

Figure 1 compares the same workload and seed under the governor and a lower-frequency setting. Power falls in every phase, most during log-probability computation and training, while phase durations remain close. This example illustrates a setting where lower power outweighs the added

Table 1. GRPO headroom workloads and GPU configurations for updates 5–25. Tuples give prompts per update P , responses per prompt K, and training batch size B. GPU splits denote generation/training allocations. Checkmarks indicate tested sizes and dashes indicate sizes not included. Model

Small (S)

Medium (M)

Large (L)

Qwen Nemotron

(32, 4, 128) (16, 4, 64)

(64, 8, 512) (16, 8, 128)

(128, 16, 2048) (32, 8, 256)

Sequence limits: Qwen 4096; Nemotron 8192. (a) H100 Deployment

GPUs

Qwen3-30B-A3B 32, 64 8/32, 16/32, 8/24 16/32, 8/24 16/32, 8/24 Nemotron-Super-49B Colocated sync. 64 Disaggregated sync. 16/32 16/64 Async, drained 16/32, 16/64 Async, in-flight 16/32 16/64 Colocated sync. Disaggregated sync. Async, drained Async, in-flight

✓ ✓ ✓ ✓

✓ ✓ — ✓

✓ ✓ ✓ ✓ ✓ ✓

✓ — ✓ — — ✓

(b) GB200 Deployment Colocated sync. Disaggregated sync. Async, drained Async, in-flight Colocated sync.

GPUs

Qwen3-30B-A3B 16 ✓ 8/16 ✓ 12/24 ✓ 4/16 — 4/16 — Nemotron-Super-49B 32 ✓

✓ ✓ ✓ ✓ ✓

✓ — ✓ — —

✓

execution time. The multi-seed sweeps below examine how consistently this trade-off holds across frequency pairs and workloads. 3.2

How the opportunity varies across workloads

We vary two frequency settings, referred to as generation and training frequencies. In colocated execution, these settings follow phase boundaries on shared GPUs. In disaggregated and asynchronous execution, each applies to its corresponding GPU pool. Each setting is compared with the governor using the same workload and GPU allocation size. Table 1 summarizes these configurations. Table 2 reports representative settings for every execution mode, with energy and throughput changes relative to the governor. Among the synchronous settings, colocated execution saves 5.08–11.53% energy per valid training token across both

Energy-Efficient Reinforcement Learning for Large Language Models Table 2. Representative headroom relative to the governor, with four matched seeds per row. Energy saving measures the reduction in all-GPU energy per valid training token, and throughput measures valid training tokens per second. (a) Synchronous execution Energy saving (%)

Throughput change (%)

Colocated 1800/1500 1875/1020 1800/1500

8.29 11.53 5.08

−0.99 −0.69 −1.79

Disaggregated 1500/1200 1350/1350 1350/1500 1800/1400 1800/1400

28.23 26.14 23.75 14.31 13.27

−0.36 −1.16 −1.31 −0.44 −0.47

GPUs

Workload

Gen/train (MHz)

Qwen Qwen Nemotron

32 H100 16 GB200 32 GB200

Large Medium Medium

Qwen Nemotron Nemotron Qwen Qwen

8/32 H100 16/64 H100 16/64 H100 12/24 GB200 12/24 GB200

Large Medium Small Small Medium

Model

(b) Asynchronous execution Model

Energy saving (%)

Throughput change (%)

∆ training reward

(policy age 1) 21.85 21.13 19.89 7.84

−1.83 −0.26 −1.37 −1.90

+0.0015 −0.0038 −0.0006 +0.0005

(policy age 2) 21.69 21.27 19.91 8.91

+1.48 −1.37 −0.78 −1.24

−0.0047 +0.0052 −0.0009 +0.0015

Gen/train (MHz)

GPUs

Workload

Nemotron Nemotron Qwen Qwen

16/64 H100 16/64 H100 8/24 H100 4/16 GB200

Medium Medium Medium Medium

Drained updates 1725/810 1830/1200 1830/810 1900/1700

Nemotron Nemotron Qwen Qwen

16/64 H100 16/64 H100 8/24 H100 4/16 GB200

Small Medium Large Medium

In-flight updates 1725/1200 1725/1200 1830/810 1900/1400

Pool splits give generation/training GPU counts. Each row includes its study’s idle clock setting. All rows cover updates 5–25. ∆ training reward is the signed difference in mean training reward from the governor and serves as a learning-behavior diagnostic.

platforms, with throughput reductions below 2%. Disaggregated execution offers more headroom, because each pool waits while the other is busy and that waiting time can run at a lower clock. On H100, Qwen and Nemotron save 23.75–28.23% with at most 1.31% lower throughput. On GB200, disaggregated Qwen saves 14.31% and 13.27% on the small and medium workloads, with less than 1% lower throughput. 3.3

When generation overlaps training

Asynchronous execution also makes the relative completion times of the generation and training pools important. A pool that repeatedly finishes before the next dependency may tolerate a lower frequency without reducing end-toend throughput because its additional computation time can replace time otherwise spent waiting for the other pool. The asynchronous rows of Table 2 cover drained updates with policy age 1 and in-flight updates with policy age 2. The selected drained and in-flight settings save up to 21.85%

and 21.69% energy per valid training token, respectively, with measured throughput reductions below 2%. Drained results also lower the clock of idle pools, which the governor does not do, so their savings combine active and idle clock control. For these rows, the table also reports trainingreward differences as a learning-behavior diagnostic rather than evidence of held-out accuracy equivalence. Across all modes, the most efficient pair depends on the workload and deployment, so a pair selected for one configuration should not be assumed suitable for another.

U SING H EADROOM TO R EDUCE E NERGY

The headroom measurements show that lower frequencies can save energy, but further reductions can eventually reverse the benefit. A useful frequency choice must balance lower power against longer execution, including the energy consumed by waiting GPUs.

Energy-Efficient Reinforcement Learning for Large Language Models

4.1

Balancing power reduction against slower progress

When a phase is limited by memory movement or communication, additional core speed may do little to improve throughput. Reducing frequency can therefore lower power with little effect on completion time, consistent with the distinction between compute and bandwidth limits in the Roofline model (Williams et al., 2009). At sufficiently low frequencies, however, slower computation can extend the phase and reduce end-to-end throughput. The energy cost follows directly from the metrics in Section 2. With P denoting average power summed across the allocated GPUs, e=

E PT P = = . N N q

(1)

Energy per valid training token falls when power decreases proportionally more than throughput. A further frequency reduction becomes counterproductive when the additional execution time outweighs the power saving. The lowest available frequency is therefore not necessarily the most efficient choice. Figure 2 separates throughput loss, power reduction, and the resulting energy change. For disaggregated Qwen, the lowest-frequency setting reduces power further but slows execution enough to give back much of the saving. In contrast, the asynchronous Nemotron results show substantial energy savings with little throughput loss. These examples illustrate why frequency selection must consider both power and end-to-end throughput. 4.2

Choosing a setting under a throughput limit

We express the performance requirement as an allowed throughput loss relative to the governor. Among tested settings that meet this requirement and save energy, we select the one with the lowest measured energy per valid training token. A setting that saves more energy but exceeds the throughput-loss allowance is not eligible, and the governor remains available if no tested setting qualifies. Table 3 lists the throughput-loss allowance.

a trial fairly when the work changes from update to update (Section 5.1), and the second is how to search when the stages interact differently in each execution mode (Sections 5.2 and 5.3). 5.1

Comparing at equal work

An update with longer responses takes longer at any clock, so elapsed time alone would confuse a change in work with the effect of a frequency. EE-RL instead asks what the governor would have done with the same work. Each run begins with a calibration phase, in which the first updates run at governor clocks and the controller only observes. From governor updates only, starting with these, it fits P log ybA = β0 + k βk zk , (2) where ybA is a predicted governor time and each zk is the standardized log(1 + x) of a token count x, which is the number of generated tokens G, valid training tokens N , or tokens in the longest response L. Generation time uses G and L, since generation cannot finish before its longest response does. Training time uses N . When the stages run in sequence, update time is their sum plus remaining work such as refit and reward. When they overlap, it is the longer of the two. Figure 3 tests this adjustment by predicting later governor updates from earlier ones. Token counts lower median prediction error in every measured configuration, most for H100 Nemotron. The gain is smallest for GB200 Qwen, particularly for energy. 5.2

Choosing a frequency pair

In synchronous execution and with drained asynchronous updates, every update runs under a single clock setting, so a trial of a few updates can be judged directly against the governor prediction. The governor observations around each trial are added to the references, so the prediction draws on recent governor behavior. A candidate passes if it retains 98% of predicted throughput and lowers energy per valid training token (Algorithm 1). Search follows the execution schedule: synchronous execution searches both phase clocks together, while drained execution searches the pools sequentially (Section 6).

Choosing frequencies at runtime exploits energy-saving headroom without extensive profiling in advance. EE-RL tests lower generation and training clocks in short trials against the governor and selects the most energy-efficient setting that retains at least 98% of governor throughput. It then monitors energy and throughput to guide adjustments as the workload changes.

A setting that passes a short trial may become less suitable as the workload changes. In synchronous and drained execution, EE-RL checks a trailing window of 20 usable updates every 10 updates, comparing throughput and energy with the governor reference for the same work. A throughput shortfall prompts a clock increase, while sufficient margin permits a further reduction. Each clock change starts a fresh lookback window. Workload drift or repeated coverage failures trigger new governor measurements to refresh the reference.

Two questions shape the design. The first is how to judge

When synchronous or drained execution places generation

EE-RL: RUNTIME GPU F REQUENCY C ONTROL

Energy-Efficient Reinforcement Learning for Large Language Models Valid training tokens / time

Gen/train MHz Small

Qwen · H100 Gen8/Train32 · disaggregated -1.6 1800/1500 810/810

Medium

Energy / valid training token

-17.5

-34.6

-16.2

-40.0

-8.2

Nemotron · H100 Gen16/Train64 · drained, age 1 -0.3 1830/1200

-21.3

-21.1

-1.8

-23.3

-21.9

1725/810 Large

Mean GPU power

Qwen · 32 H100 · colocated -1.0 1800/1500 1200/1200

-9.2

-5.8 −40 −20

-8.3

-30.9 0

-26.6

−40 −20

Change from native governor (%)

−40 −20

Figure 2. Throughput, power, and energy-per-token changes relative to the governor for representative generation/training frequency pairs, as geometric means over four matched seeds.

the waiting pool to an idle clock until that phase begins. No computation runs in these gaps, so this saves energy that the governor spends on idle GPUs, without slowing the update. 5.3

Figure 3. Median absolute error in predicting governor throughput and energy for later updates, with and without token counts. Lower is better.

Algorithm 1 Choosing GPU frequencies in synchronous and drained execution

Require: Supported frequencies, governor observations 1: Fit the governor predictor on token counts and longest response 2: for each search round do 3: {Synchronous: both clocks. Drained: first pool, then second pool.} 4: repeat 5: Propose a candidate for this round 6: Run it between governor observations 7: Compare throughput and energy with the governor prediction 8: until the round’s search ends 9: Confirm the passing setting with the lowest energy per token on fresh observations 10: If confirmation fails, try the next passing setting or continue the search 11: Keep the confirmed setting fixed in later rounds 12: end for 13: Lock the confirmed setting

With in-flight updates, generation continues across updates, pausing only for each weight refit. Requests started under one clock can finish under the next, so the in-flight controller, unlike the drained one, cannot fit generation time at a given clock. EE-RL therefore treats selection as an experiment. Each trial places a block of candidate updates between governor blocks and compares their throughput and energy at matched work. Block length is set from the noise measured during calibration. As with drained updates, pools are searched one at a time, beginning with the less busy one, and the best setting is confirmed before it is locked. Because a single confirmation cannot rule out a small loss, the controller keeps one governor block for every two candidate blocks after locking, so the comparison with the governor continues.

C ONTROL ACROSS E XECUTION M ODES

A slower phase does not always mean a slower RL update. In synchronous execution, added computation time extends the update. With drained updates, it may replace time spent waiting for the other pool. In-flight responses can cross a clock change, making their cost hard to attribute to either setting. These dependencies determine how EE-RL measures trials and searches frequencies. Table 3 gives the configuration. 6.1

and training on separate GPU pools, one pool often waits for the other before its next phase can start. The controller drops

Choosing frequencies with in-flight updates

Synchronous execution

Generation and training run in sequence, so their added execution times share the same throughput allowance. EE-

Energy-Efficient Reinforcement Learning for Large Language Models

ing the rendezvous. Other candidates use the whole-update governor prediction.

Table 3. Controller configuration. Setting Allowed loss Calibration Trial Confirmation Search grid Feedback step

Sync.

Drained

In-flight

2% 15 updates 3 updates

2% 45 updates 3, 6, or 12 updates 1–2 trials

6 fresh updates 6 fresh updates

Monitoring

90 MHz 45/90 MHz 20 updates, every 10

50/100 MHz 50 MHz 20 updates, every 10

Idle clock

345 MHz

50/100 MHz — 1 governor per 2 candidate blocks —

Allowed loss is relative to governor throughput. Asynchronous search uses 50 MHz spacing for the first pool and 100 MHz for the second. Synchronous feedback uses 45 MHz steps, increasing to 90 MHz above 4% throughput loss. GPUs support only discrete clock frequencies, so applied frequency steps can differ slightly from these nominal sizes. The idle clock applies only to separate GPU pools.

RL fits a governor predictor for each phase and adds the median remaining time, including refit and reward. During a trial, it measures each phase’s time and energy relative to its governor prediction at the observed token workload. Applying these ratios to the governor reference updates estimates the candidate’s throughput and energy over a common workload, reducing sensitivity to the few updates in a trial. The phase timings identify which clock needs to rise when a pair is too slow. Search starts with both clocks reduced and raises the phase responsible for most of the added time. Once a pair passes, it explores lower settings for further energy savings. 6.2

Asynchronous execution with drained updates

Generation and training overlap but must rendezvous before each weight update, and clocks change at this rendezvous. The later arrival determines update time, so slower computation in the earlier pool may simply shorten its wait. EE-RL records both arrivals and searches the pool that usually arrives first, leaving the other on the governor. Its confirmed setting then provides the reference for lowering the second pool. The combined pair must meet the throughput requirement and use no more energy than the first setting alone, so a second lowered clock is kept only if it benefits the whole allocation. Response-length variation can obscure this waiting interval. When only training is slowed, EE-RL applies the measured slowdown to the governor references and recomputes update time from their recorded arrivals. Holding generation fixed estimates how much training can slow before delay-

6.3

Asynchronous execution with in-flight updates

A response may begin under one clock and finish under another. Measuring the responses completed within an update would therefore mix work from different settings. The runtime instead counts the valid tokens consumed by training, changes clocks during the weight-refit pause, and leaves updates carrying requests from the preceding setting out of trial decisions. The longest consumed response remains a predictor feature because it improves held-out estimates when response lengths vary, even though it no longer determines an update boundary. Neighboring updates also share work, so a short trial may contain less independent evidence than its length suggests. Calibration estimates this correlation and selects the shortest trial block meeting a 2% standard-error target for comparison with the governor. Each candidate block is bracketed by governor blocks. Without a drained rendezvous to measure waiting directly, search starts with the less busy pool, then tests the second with the first setting held. Governor blocks continue after locking to check the throughput requirement. 6.4

Runtime measurement

The energy accounting must include waiting GPUs: slowing one pool can extend the time the other remains powered. The runtime samples each GPU every 200 ms and sums interpolated NVML energy counters across the allocation (NVIDIA, 2026). Acceptance uses whole-update time and energy, including waits and clock changes.

E VALUATION

EE-RL saves energy in all eight controller comparisons after including selection and adaptation costs. We examine how the execution schedule determines the useful control target and why the savings after locking differ from those over the complete interval. 7.1

Experimental setup

We evaluate Qwen3-30B-A3B and Mistral-NeMo 12B on H100 in colocated and disaggregated synchronous execution, and with drained or in-flight asynchronous updates. Mistral-NeMo was not part of the headroom study, so it tests the controller on a model without a prior sweep. NemotronSuper-49B’s longer updates left too few steps within the allocated runtime for controller evaluation. Each comparison matches the seed, workload, and GPU topology, using separate physical allocations. Rates use summed valid training tokens, time, and all-GPU energy

Energy-Efficient Reinforcement Learning for Large Language Models Table 4. Controller performance relative to matched governor runs over the listed updates, including search and adaptation costs. Energy savings and throughput changes use valid training tokens. Accuracy change is the difference in final validation accuracy from the governor, in percentage points. Dashes indicate unreported accuracy. GPU splits give generation/training counts. Drained and in-flight runs use policy ages 1 and 2, respectively. Energy/token Throughput Accuracy Token Model / execution GPUs (P, K, B) limit Updates saving (%) change (%) change (pp) Qwen, colocated sync. Qwen, disaggregated sync. Mistral, colocated sync. Mistral, disaggregated sync. Qwen, async drained Mistral, async drained Qwen, async in-flight Mistral, async in-flight

32 H100 8/32 H100 16 H100 8/16 H100 8/24 H100 8/16 H100 16/32 H100 8/16 H100

64/8/512 64/8/512 64/8/512 64/8/512 32/4/128 64/8/512 32/4/128 64/8/512

over the same updates, including search, confirmation, correction, waiting, and synchronization. Table 4 reports every configuration. RL workloads also vary from run to run, even with matched seeds, because sampling and the policy drift apart and response lengths change. We therefore also compare each controller run with a governor model fitted on two pooled governor runs of the same workload, and report these workload-normalized results in Appendix E. 7.2

Savings survive online selection

Disaggregated synchronous Qwen achieves the largest saving: 16.9% with 1.4% lower throughput, compared with 7.0% energy saving on colocated. Mistral-NeMo shows the same ordering. Separate pools expose idle intervals in addition to headroom during computation, and the controller reduces clocks in both pools. The Mistral-NeMo results show that useful settings can be found without a prior headroom sweep. Their savings still depend on the balance between the pools and their execution schedule. 7.3

Overlap makes training the useful control target

In all four asynchronous runs, the controller tests lower generation clocks but ultimately slows only training. These choices suggest that training can run at a lower frequency while generation prepares the next batch, whereas slowing generation does not yield an accepted energy–throughput trade-off. Arrival order or utilization determines which GPU pool is searched first, but the controller’s decisions depend on how each setting affects end-to-end progress. In drained execution, this spare training capacity translates into energy savings of about 14% over the full interval and about 20% after locking. Slower training uses time that would otherwise be spent waiting at the rendezvous. Lowering clocks during idle periods adds to the savings for Qwen, but rarely triggers for Mistral-NeMo because

4096 4096 4096 4096 4096 4096 4096 4096

2–107 2–93 2–265 2–315 2–175 2–300 2–335 2–560

7.00 16.94 4.14 8.89 14.51 13.56 7.24 6.52

−1.25 −1.39 +1.62 −1.52 −1.97 +1.46 −1.08 −1.61

— — — — 0.00 +0.90 +0.18 −0.14

its idle gaps are short. The two workloads thus achieve similar savings through different combinations of active and idle frequency control, with the benefit measured across the whole GPU allocation. 7.4

Selection and monitoring have different costs

Every run saves more energy after locking than before it (Table 5). Disaggregated synchronous Qwen saves 22.4% after locking, versus 16.9% over the complete interval. Figure 4 shows this selection process in a separate colocated run, where candidate trials alternate with governor observations before confirmation. Reporting only the after-lock interval would miss these selection costs. After locking, monitoring both recovers throughput and finds further reductions. For disaggregated synchronous Mistral-NeMo, three clock increases are followed by lower throughput losses: 3.30% to 1.65%, 2.35% to 1.20%, and 2.70% to 0.34% in the next 20-update windows. These windows retain 9.06–9.66% energy savings in the controller’s work-normalized estimates. For drained Qwen, monitoring identifies sufficient throughput margin to lower the training clock from 1065 to 1005 MHz. Following confirmation, this setting achieves 19.50% energy savings with 0.56% throughput loss against the paired governor over updates 142–175. In-flight execution also pays for measurement after selection. Its full-interval savings are roughly half those of drained execution. Noisier measurements delay locking, and about a third of after-lock updates still use the governor for continued comparison. In-flight runs also omit idle dropping. The different schedules and allocations prevent attributing the gap to any one factor, but the persistent governor blocks explain why a longer run alone would not recover all of the available saving. Longer stable runs can amortize a completed search, but not a fixed fraction of time spent rechecking the choice.

Energy-Efficient Reinforcement Learning for Large Language Models Table 5. Controller performance before and after locking, relative to the paired governor over the same updates. Energy savings are measured per valid training token. All runs use H100 GPUs. GPU splits give generation/training counts. Gov denotes native governor control, and Varies denotes changing frequency settings. After-lock intervals include subsequent adaptation and, for in-flight runs, governor comparison blocks. Energy/token Throughput Gen/train Model / execution Interval Updates (MHz) saving (%) change (%) Qwen, colocated sync.

Before lock After lock

2–77 78–107

Varies 1830/1425

6.59 8.04

−1.13 −1.55

Qwen, disaggregated sync.

Before lock After lock

2–61 62–93

Varies 1695/1425

14.02 22.44

−1.27 −1.60

Mistral, colocated sync.

Before lock After lock Before lock After lock Before lock After lock

2–172 173–265 2–59 60–315 2–89 90–175

Varies 1785/1695 Varies Varies Varies Gov/Varies

3.95 4.56 8.04 9.08 9.91 19.23

−0.18 +4.99 −1.01 −1.64 −2.96 −0.94

Mistral, async drained, 8/16

Before lock After lock

2–190 191–300

Varies Gov/1185

9.32 20.87

−0.10 +4.19

Qwen, async in-flight, 16/32

Before lock After lock

2–257 258–335

Varies Gov/1125

6.45 9.82

−0.89 −1.70

Mistral, async in-flight, 8/16

Before lock After lock

2–313 314–560

Varies Gov/1425

5.32 8.03

−2.45 −0.52

Mistral, disaggregated sync. Qwen, async drained, 8/24

Generation

cannot establish equal learning quality.

Training

Clock (MHz)

2000

Confirmed at 53

1600

1695 / 1335

1200 1

Update

Figure 4. Measured phase clocks in a separate 90-update colocated Qwen run on 32 H100s. Confirmation after update 53 selects different generation and training frequencies. This trace illustrates search and is separate from the scored runs.

7.5

Interpreting throughput and learning quality

All eight full-interval comparisons lose less than 2% throughput, although two asynchronous runs exceed that loss before locking (Table 5). A locally accepted setting therefore does not guarantee a run-level bound that includes unsuccessful trials and corrections. Workload differences also matter. Synchronous MistralNeMo controller runs consume more valid tokens, and drained Mistral-NeMo has shorter longest responses after locking. These changes contribute to throughput gains and energy-per-token savings, so they should not be read as clock-induced speedups. The asynchronous validationaccuracy changes in Table 4 are all below one percentage point in magnitude, but one matched seed per configuration

D ISCUSSION AND L IMITATIONS

Controlling the final throughput cost. The difference between trial-level acceptance and complete-run throughput suggests managing performance as an accumulated budget. A controller could track estimated accumulated slowdown and use the remaining allowance to limit the duration of new trials. This would connect exploration cost to the job’s remaining horizon. Early in a run, more exploration may be economical. For production runs spanning thousands of updates, the longer horizon could justify a more extensive search: its upfront cost can be amortized over many subsequent updates, potentially allowing the controller to find more energy-efficient settings and achieve larger net savings. Near completion, retaining an accepted setting may contribute more than another search step. Workload and allocation dependence. A frequency choice depends on token counts, response lengths, the work within each stage, and the GPU allocation. The stages themselves depend on the RL algorithm. We evaluate only GRPO, while methods that train a value model, such as PPO (Schulman et al., 2017), add stages with their own frequency demands. Changes in pool size or parallelism can shift the balance between stages, and we also observed governor-performance differences across physical allocations. Resuming a checkpoint on a different allocation should therefore trigger a fresh frequency search. Our controller evaluation uses one

Energy-Efficient Reinforcement Learning for Large Language Models

matched seed per configuration. Repeated seeds and component ablations would quantify variability and separate the contributions of prediction, search, confirmation, and feedback. Replaying a fixed pair over the same horizon would directly compare adaptation with static control. Learning quality. Across the four asynchronous controller runs, final validation accuracy differs from the paired governor by between −0.14 and +0.90 percentage points. These differences are small, but each comparison uses one matched seed. Asynchronous timing can change which responses training consumes and their policy ages. Repeated runs across seeds would better characterize the effect on learning quality, and comparisons at a common held-out quality target would test whether the energy savings persist when matching achieved quality. Pool allocation and idle power. The benefit of frequency control depends on the balance between GPU pools. Slowing one pool can reduce its active power while extending the other pool’s wait. Idle control can reduce the energy consumed during that wait, but its additional benefit depends on the pool’s existing idle power. Whole-allocation energy captures both effects. Results that combine active-frequency control with idle dropping therefore measure both mechanisms together. They do not isolate the contribution of either mechanism.

R ELATED W ORK

The balance between power and progress also arises in conventional model training and in serving, where prior systems adjust GPU power, frequency, or scheduling to improve efficiency. These efforts complement work on RL systems that improves resource use through placement, parallelism, and scheduling. Together, they provide the context for EE-RL’s approach to frequency control during RL post-training. Energy-aware training. Zeus jointly tunes batch size and GPU power limit for recurring training jobs through online exploration (You et al., 2023). EnvPipe and Perseus lower the frequency of computation off the critical path of pipelineparallel training (Choi et al., 2023; Chung et al., 2024), and kernel-level DVFS selects clock frequencies for individual kernels (Spaan et al., 2026). Zeus uses short iterative profiles to estimate power and throughput, while Perseus constructs frequency schedules over a static per-iteration computation graph. In RL post-training, generation and training respond differently to frequency. Response-length changes can alter execution time independently of frequency, and slowing one stage can extend the other stage’s waiting time. EE-RL therefore accounts for response-length changes when comparing settings, measures the energy of the whole allocation, including waiting GPUs, and includes exploration costs in its savings.

Energy-aware LLM serving. DynamoLLM reconfigures instance count, parallelism, and GPU frequency for pools of similar requests under latency targets (Stojkovic et al., 2025), and throttLL’eM sets frequency from projected KV-cache usage and batch size (Kakolyris et al., 2025). With prefill and decode disaggregated onto separate GPU pools (Zhong et al., 2024), GreenLLM, VoltanaLLM, and DualScale set a separate frequency for each phase (Liu et al., 2026; Yu et al., 2025; Basit et al., 2026), and AFlex further separates attention and feed-forward frequencies (Hu et al., 2026). These systems optimize inference energy under serving latency constraints. EE-RL instead controls the coupled generation and training stages of RL post-training, targeting wholeallocation energy per valid training token under an end-toend throughput allowance. Serving-engine optimizations such as iteration-level scheduling (Yu et al., 2022) and paged KV-cache management (Kwon et al., 2023) are complementary. They improve scheduling and memory efficiency and can change the frequency settings that are beneficial. RL post-training systems. HybridFlow supports flexible placement and execution of RL dataflows (Sheng et al., 2025), ReaL reallocates parameters to adapt parallelization across the stages of an RL step (Mei et al., 2025), and HetRL schedules RL across heterogeneous GPUs (He et al., 2026). RollPacker mitigates long-tail rollouts in synchronous RL post-training (Gao et al., 2026). AReaL decouples generation from training while controlling data staleness (Fu et al., 2025), and PipelineRL and Laminar also run the two concurrently (Piché et al., 2026; Sheng et al., 2026). These systems primarily improve throughput or learning speed through placement, parallelism, and scheduling rather than GPU frequency control. Measurements of Olmo 3 32B Think attribute most of its Think-specific post-training energy to rollout generation (Morrison et al., 2026). EE-RL instead targets energy by controlling clocks within a given placement, using phase boundaries and pool balance to identify headroom. Changes to placement or parallelism can alter phase balance and therefore warrant re-evaluating the selected frequencies.

C ONCLUSION

Generation and training expose different opportunities for GPU frequency reduction during RL post-training. Measurements of GRPO workloads on H100 and GB200 show that these opportunities can reduce GPU energy per valid training token while largely preserving governor throughput. EE-RL selects frequencies during execution, accounts for response-length variation, and revisits its choices as the workload changes. Phase-aware frequency control complements RL scheduling and placement optimizations by adapting GPU speed to the work being performed.

Energy-Efficient Reinforcement Learning for Large Language Models

ACKNOWLEDGMENTS We thank Trivikram Krishnamurthy, Sridutt Bhalachandra, Terry Kong, and Bhargava Gopireddy for their support of this work.

R EFERENCES Basit, O., Liu, Y., Kong, Z. J., and Hu, Y. C. DualScale: Energy-efficient disaggregated LLM serving via phase-aware placement and DVFS. arXiv preprint arXiv:2602.18755, 2026. URL https://arxiv.or g/abs/2602.18755.

International Energy Agency. Energy and AI. Technical report, International Energy Agency, 2025. URL https: //www.iea.org/reports/energy-and-ai. Kakolyris, A. K., Masouros, D., Vavaroutsos, P., Xydis, S., and Soudris, D. throttLL’eM: Predictive GPU throttling for energy efficient LLM inference serving. In 2025 IEEE International Symposium on High Performance Computer Architecture (HPCA), pp. 1363–1378, 2025. doi: 10.1109/HPCA61900.2025.00103.

Chandrakasan, A. P., Sheng, S., and Brodersen, R. W. Lowpower CMOS digital design. IEEE Journal of Solid-State Circuits, 27(4):473–484, 1992. doi: 10.1109/4.126534.

Kwon, W., Li, Z., Zhuang, S., Sheng, Y., Zheng, L., Yu, C. H., Gonzalez, J. E., Zhang, H., and Stoica, I. Efficient memory management for large language model serving with PagedAttention. In Proceedings of the 29th Symposium on Operating Systems Principles, pp. 611–626, 2023. doi: 10.1145/3600006.3613165.

Choi, S., Koo, I., Ahn, J., Jeon, M., and Kwon, Y. EnvPipe: Performance-preserving DNN training framework for saving energy. In 2023 USENIX Annual Technical Conference (USENIX ATC 23), pp. 851–864. USENIX Association, 2023. URL https://www.usenix.o rg/conference/atc23/presentation/choi.

Liu, Q., Huang, D., Zapater, M., and Atienza, D. GreenLLM: SLO-aware dynamic frequency scaling for energyefficient LLM serving. In Proceedings of the 63rd ACM/IEEE Design Automation Conference (DAC ’26), 2026. arXiv:2508.16449.

Chung, J.-W., Gu, Y., Jang, I., Meng, L., Bansal, N., and Chowdhury, M. Reducing energy bloat in large model training. In Proceedings of the ACM SIGOPS 30th Symposium on Operating Systems Principles (SOSP ’24), pp. 144–159, 2024. doi: 10.1145/3694715.3695970. Fu, W., Gao, J., Shen, X., Zhu, C., Mei, Z., He, C., Xu, S., Wei, G., Mei, J., Wang, J., Yang, T., Yuan, B., and Wu, Y. AReaL: A large-scale asynchronous reinforcement learning system for language reasoning. In Advances in Neural Information Processing Systems, volume 38, 2025. Gao, W., Zhao, Y., An, D., Wu, T., Cao, L., Xiong, S., Huang, J., Wang, W., Yang, S., Su, W., Wang, J., Qu, L., Zheng, B., and Wang, W. RollPacker: Taming longtail rollouts for RL post-training with tail batching. In 23rd USENIX Symposium on Networked Systems Design and Implementation (NSDI 26), pp. 849–866. USENIX Association, 2026. He, Y., Zhang, S., Gai, J., Zhang, X., Han, B., Wang, B., Rangwala, H., and Karypis, G. HetRL: Efficient reinforcement learning for LLMs in heterogeneous environments. In Proceedings of Machine Learning and Systems, volume 8, 2026. Hu, C., Xu, L., Liu, T., Lyu, M., Li, Y., Wang, S., Quan, S., Yang, Y., Tang, W., Wang, Y., Yu, F., and Wu, J. Energy-efficient LLM serving via disaggregated attention– FFN and flexible frequency scaling. arXiv preprint arXiv:2608.01891, 2026. URL https://arxiv.or g/abs/2608.01891.

Mei, Z., Fu, W., Li, K., Wang, G., Zhang, H., and Wu, Y. ReaL: Efficient RLHF training of large language models with parameter reallocation. In Proceedings of Machine Learning and Systems, volume 7, 2025. Morrison, J., Smith, N. A., and Strubell, E. The hidden cost of thinking: Energy use and environmental impact of LMs beyond pretraining. arXiv preprint arXiv:2605.01158, 2026. URL https://arxiv.org/abs/2605.0 1158. NVIDIA. NVML API Reference Guide: Device Queries (nvmlDeviceGetTotalEnergyConsumption), 2026. URL https://docs.nvidia.com/deploy/nvml-a pi/api/group__nvmlDeviceQueries.html. Accessed October 30, 2026. Piché, A., Kamalloo, E., Pardinas, R., Chen, X., and Bahdanau, D. PipelineRL: Faster on-policy reinforcement learning for long sequence generation. Transactions on Machine Learning Research, 2026. arXiv:2509.19128. Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017. URL https: //arxiv.org/abs/1707.06347. Shao, Z., Wang, P., Zhu, Q., Xu, R., Song, J., Bi, X., Zhang, H., Zhang, M., Li, Y. K., Wu, Y., and Guo, D. DeepSeekMath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300, 2024. URL https://arxiv.org/abs/2402.0 3300.

Energy-Efficient Reinforcement Learning for Large Language Models

Sheng, G., Zhang, C., Ye, Z., Wu, X., Zhang, W., Zhang, R., Peng, Y., Lin, H., and Wu, C. HybridFlow: A flexible and efficient RLHF framework. In Proceedings of the Twentieth European Conference on Computer Systems (EuroSys ’25), 2025. doi: 10.1145/3689031.3696075. Sheng, G., Tong, Y., Wan, B., Zhang, W., Jia, C., Wu, X., Wu, Y., Li, X., Zhang, C., Peng, Y., Lin, H., Liu, X., and Wu, C. Laminar: A scalable asynchronous RL posttraining framework. In Proceedings of the Twenty-First European Conference on Computer Systems (EuroSys ’26), 2026. arXiv:2510.12633. Spaan, J., Chen, K.-H., and Varbanescu, A.-L. How much energy is wasted in LLM operations? Evidence from kernel-level DVFS. In Proceedings of the 23rd ACM International Conference on Computing Frontiers (CF ’26), 2026. doi: 10.1145/3801487.3801822. Stojkovic, J., Zhang, C., Goiri, Í., Torrellas, J., and Choukse, E. DynamoLLM: Designing LLM inference clusters for performance and energy efficiency. In 2025 IEEE International Symposium on High Performance Computer Architecture (HPCA), pp. 1348–1362, 2025. doi: 10.110 9/HPCA61900.2025.00102. Williams, S., Waterman, A., and Patterson, D. Roofline: An insightful visual performance model for multicore architectures. Communications of the ACM, 52(4):65–76, 2009. doi: 10.1145/1498765.1498785. You, J., Chung, J.-W., and Chowdhury, M. Zeus: Understanding and optimizing GPU energy consumption of DNN training. In 20th USENIX Symposium on Networked Systems Design and Implementation (NSDI 23), pp. 119–139. USENIX Association, 2023. URL https://www.usenix.org/conference/ns di23/presentation/you. Yu, G.-I., Jeong, J. S., Kim, G.-W., Kim, S., and Chun, B.G. Orca: A distributed serving system for TransformerBased generative models. In 16th USENIX Symposium on Operating Systems Design and Implementation (OSDI 22), pp. 521–538. USENIX Association, 2022. URL https://www.usenix.org/conference/os di22/presentation/yu. Yu, J., Taneja, A., Lin, J., and Zhang, M. VoltanaLLM: Feedback-driven frequency control and state-space routing for energy-efficient LLM serving. arXiv preprint arXiv:2509.04827v1, 2025. URL https://arxiv. org/abs/2509.04827v1. Zhong, Y., Liu, S., Chen, J., Hu, J., Zhu, Y., Liu, X., Jin, X., and Zhang, H. DistServe: Disaggregating prefill and decoding for goodput-optimized large language model serving. In 18th USENIX Symposium on

Operating Systems Design and Implementation (OSDI 24), pp. 193–210. USENIX Association, 2024. URL https://www.usenix.org/conference/os di24/presentation/zhong-yinmin.

AI U SAGE D ISCLOSURE AI-based tools assisted with manuscript writing and editing, experimental and analysis scripts, and result consistency checks. The authors take responsibility for the final content and findings.

Energy-Efficient Reinforcement Learning for Large Language Models

A PPENDIX This appendix expands the headroom, token-work prediction, and controller experiments with measurement details, the complete valid headroom inventory, and workload-normalized controller results.

E STIMATION AND S ELECTION

P P P P For seed s and setting p, form qs,p = N/ T and es,p = E/ N , where N counts valid training tokens. For each matched seed let xs = log(qs,p /qs,A ). The tables report bq = exp(x̄). R The energy ratio uses the same construction. Energy saving is 100(1 − Re ), and throughput change is 100(Rq − 1). Selection targets lower energy per token while retaining at least 98% of governor throughput.

ACCOUNTING FOR T OKEN W ORK AND R ESPONSE L ENGTH

A fixed number of RL updates does not imply a fixed amount of work. Longer responses increase the tokens processed by generation and training, while a particularly long response can delay generation even when total token counts are similar. Figure A1 shows that work, time, and energy vary even under unchanged governor control. Without accounting for this variation, a frequency trial can appear faster or slower simply because its workload differs. EE-RL therefore predicts governor performance for the work observed during each candidate trial. The synchronous and drained references both use log-transformed generated and valid training token counts, together with the maximum response length, which governs when generation finishes. The in-flight reference uses the valid training tokens and longest response of the batch that training consumes. These models fit only governor observations, keeping candidate measurements from changing their own reference. To evaluate token-count normalization, we use nine governor-only colocated runs: three seeds each for Qwen on 32 H100s, Nemotron on 64 H100s, and Qwen on 16 GB200s. Each forward test fits 15 past observations, predicts the next three, and advances by three, producing 88, 69, and 80 blocks, respectively. Compared with a constant reference, normalization reduces median absolute throughput prediction error from 1.57% to 0.59%, 5.28% to 0.31%, and 1.29% to 1.01%. Update-energy prediction errors also decrease. Configuration Qwen, 32 H100 Nemotron, 64 H100 Qwen, 16 GB200

Blocks 88 69 80

Throughput error (%) Constant Token-work 1.57 5.28 1.29

0.59 0.31 1.01

Energy error (%) Constant Token-work 0.51 3.67 0.70

0.30 0.56 0.69

A separate governor-only Mistral-NeMo 12B experiment tests the contribution of response length. Across 24 held-out three-update blocks, adding a longest-response proxy to the token-count model reduces update-time prediction RMS log error from 0.085 to 0.044. Work normalization improves the reference but does not eliminate the need for repeated measurements and continued monitoring.

Energy-Efficient Reinforcement Learning for Large Language Models

Figure A1. Workload variation under governor-only Nemotron-Super-49B execution on 64 H100s. Faint lines show individual updates, and solid lines show trailing five-update means. Values are normalized to their mean over the first ten measured updates. Elapsed time sums measured update durations.

C OMPLETE S YNCHRONOUS H EADROOM R ESULTS

Note on GB200 measurements. We observed substantial run-to-run variation on the GB200 cluster. The GB200 results in the synchronous and asynchronous tables below, as well as the GB200 rows of Table 2, should be read with this variation in mind. The tables contain 208 H100 and 85 GB200 non-governor comparisons, grouped by model and execution mode with hardware identified in each heading. Each uses a matched native-governor reference. Clock pairs are requested frequencies. Observed clocks may be slightly lower, and “Gov” leaves the phase governed. All synchronous comparisons aggregate four matched seeds. Row suffixes specify measured updates when they differ from updates 5–25. C.1

Colocated synchronous: Qwen, 32 H100

Four matched seeds per comparison. Workload / split Gen/train MHz ∆q (%) Energy saving (%) n Small Small Small Small Small Small Small Small Small Medium Medium Medium Medium Medium Medium Medium Medium Medium Large Large Large Large Large Large Large

1200/1200 1500/1500 1800/1800 1980/1980 1650/1200 1650/1350 1800/1200 1800/1350 1800/1500 1200/1200 1500/1500 1800/1800 1980/1980 1650/1200 1650/1350 1800/1200 1800/1350 1800/1500 1200/1200 1500/1500 1800/1800 1980/1980 1650/1200 1650/1350 1800/1200

-12.10 -4.84 -0.80 +0.39 -4.85 -3.24 -2.27 -2.54 -1.68 -10.17 -4.67 -1.50 -1.21 -5.43 -3.89 -4.28 -3.54 -2.28 -5.82 -1.95 -0.14 -0.10 -3.17 -2.31 -2.99

20.91 9.56 -0.13 -0.70 11.85 10.03 9.97 7.45 4.69 24.05 10.88 0.15 -1.23 14.42 12.48 12.00 9.44 5.50 26.59 12.74 0.25 -0.34 18.88 15.60 16.57

4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4

Energy-Efficient Reinforcement Learning for Large Language Models Workload / split Gen/train MHz ∆q (%) Energy saving (%) n Large Large

C.2

1800/1350 1800/1500

-2.03 -0.99

13.51 4 8.29 4

Colocated synchronous: Qwen, 64 H100

C.3

-14.16 -5.57 -1.84 -1.84 -6.59 -5.29 -4.76 -3.83 -2.93 -11.44 -4.53 -1.39 -1.50 -5.24 -4.15 -4.47 -2.86 -2.34 -7.10 -2.74 -0.36 -0.48 -3.45 -2.50 -3.02 -2.16 -1.43

18.26 8.52 -0.51 -1.71 10.05 8.33 7.63 6.20 3.35 22.15 10.14 0.06 -1.37 12.67 10.75 9.96 8.34 4.37 25.70 12.15 0.62 -0.73 17.35 14.31 14.76 11.71 7.13

4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4

Colocated synchronous: Qwen, 16 GB200

1875/1020 1875/930 1950/1020 1950/930 Gov/1020 Gov/840 Gov/885 Gov/930 Gov/975 1875/1020 1875/930 1950/1020 1950/930 Gov/1020 Gov/840 Gov/885 Gov/930 Gov/975 1875/1020 1875/930

+1.62 +1.53 +4.35 +3.43 +2.51 +2.02 +4.67 -0.98 -3.49 -0.69 +1.05 -2.19 +3.10 +11.44 +2.38 +0.03 +6.19 +0.80 -1.08 +0.46

12.61 11.98 12.62 12.02 9.88 10.40 12.12 9.12 6.95 11.53 12.82 10.36 14.55 16.99 13.71 10.41 14.64 11.90 13.44 14.46

4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4

Energy-Efficient Reinforcement Learning for Large Language Models Workload / split Gen/train MHz ∆q (%) Energy saving (%) n Large Large Large Large Large Large Large

C.4

1950/1020 1950/930 Gov/1020 Gov/840 Gov/885 Gov/930 Gov/975

+0.63 -7.65 -3.30 -3.95 -5.42 -9.00 +1.35

13.62 8.78 10.03 12.19 12.05 6.65 13.65

4 4 4 4 4 4 4

Colocated synchronous: Nemotron, 64 H100

Four matched seeds per comparison. Workload / split Gen/train MHz ∆q (%) Energy saving (%) n Small Small Small Small Small Small Small Small Small Small Small Small Small Small Small Medium Medium Medium Medium Medium Medium Medium Medium Medium Medium Medium Medium Medium Medium Medium Large Large Large Large Large Large Large Large Large Large Large Large Large Large Large

1200/1200 1500/1500 1800/1800 1980/1980 1650/1200 1650/1350 1800/1200 1800/1350 1800/1500 Gov/1500 Gov/1350 1800/Gov 1650/Gov Gov/1200 1800/1650 1200/1200 1500/1500 1800/1800 1980/1980 1650/1200 1650/1350 1800/1200 1800/1350 1800/1500 Gov/1500 Gov/1350 1800/Gov 1650/Gov Gov/1200 1800/1650 1200/1200 1500/1500 1800/1800 1980/1980 1650/1200 1650/1350 1800/1200 1800/1350 1800/1500 Gov/1500 Gov/1350 1800/Gov 1650/Gov Gov/1200 1800/1650

-11.87 -3.26 -0.41 -1.80 -4.38 -3.64 -3.64 -2.03 -1.12 -0.73 -2.52 -0.80 -2.98 -2.24 -0.63 -10.92 -3.82 -0.75 -0.77 -4.89 -4.04 -4.49 -2.21 -2.22 -1.80 -2.32 -0.80 -2.68 -3.27 -2.01 -11.38 -2.64 -0.88 -0.95 -6.95 -5.52 -6.03 -4.54 -2.07 -2.11 -4.35 -1.16 -1.90 -5.37 -1.52

21.33 8.32 0.03 -1.41 7.51 6.40 4.54 3.75 1.46 1.88 3.19 0.04 3.03 4.77 0.78 22.48 8.97 -0.10 -0.51 9.42 7.93 6.64 5.00 2.50 2.09 4.71 0.41 3.01 6.36 0.68 23.02 8.72 -0.44 -1.02 10.50 8.31 7.70 5.79 2.52 2.37 5.25 -0.19 2.66 7.67 0.17

4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4

Energy-Efficient Reinforcement Learning for Large Language Models

C.5

Colocated synchronous: Nemotron, 32 GB200

C.6

-12.99 -8.59 -3.01 -0.84 -5.00 -5.14 -2.53 -3.27 -3.52 -13.61 -4.68 -1.67 +0.32 -6.29 -5.08 -4.94 -3.91 -1.79 -14.52 -6.95 -3.99 -1.33 -8.68 -6.47 -8.80 -6.79 -8.86

7.49 5.78 2.95 0.43 6.61 4.85 6.16 4.24 4.07 7.52 7.58 2.71 -0.19 5.81 6.44 4.73 4.50 5.08 7.32 6.52 3.34 -0.27 7.00 5.27 3.30 3.88 2.76

4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4

Disaggregated synchronous: Qwen, H100 Gen8/Train32

Four matched seeds per comparison. Workload / split Gen/train MHz ∆q (%) Energy saving (%) n Small Small Small Small Small Small Small Small Small Small Medium Medium Medium Medium Medium Medium Medium Medium Medium Medium Large Large Large Large Large Large

1980/1980 1800/1500 1650/1500 1500/1500 1500/1350 1500/1200 1200/1500 1200/1350 1200/1200 810/810 1980/1980 1800/1350 1650/1350 1500/1500 1500/1350 1500/1200 1200/1500 1200/1350 1200/1200 810/810 1980/1980 1800/1200 1650/1200 1500/1500 1500/1350 1500/1200

-1.36 -1.56 -3.12 -4.88 -5.68 -5.95 -13.09 -12.56 -12.87 -34.62 -0.44 -1.36 -3.09 -4.78 -3.53 -5.24 -10.63 -10.71 -10.61 -30.71 +0.58 -0.82 -1.45 -1.41 -2.79 -0.36

11.25 16.19 17.40 18.44 21.59 23.38 18.57 22.25 24.16 8.19 11.76 21.27 22.14 19.53 23.72 25.44 20.79 24.24 26.80 15.10 13.23 24.73 25.82 21.74 25.01 28.23

4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4

Energy-Efficient Reinforcement Learning for Large Language Models Workload / split Gen/train MHz ∆q (%) Energy saving (%) n Large Large Large

C.7

1200/1500 1200/1350 1200/1200

-4.05 -3.84 -4.45

24.72 4 28.17 4 30.26 4

Disaggregated synchronous: Qwen, H100 Gen16/Train32

C.8

1980/1980 1800/1200 1650/1200 1500/1500 1500/1350 1500/1200 1200/1500 1200/1350 1200/1200 810/810 1980/1980 1800/1200 1650/1200 1500/1500 1500/1350 1500/1200 1200/1500 1200/1350 1200/1200 810/810 1980/1980 1800/1200 1650/1200 1500/1500 1500/1350 1500/1200 1200/1500 1200/1350 1200/1200 810/810

-1.73 -2.16 -5.08 -6.94 -6.65 -6.54 -14.32 -14.36 -14.31 -36.06 -0.74 -1.32 -3.31 -3.46 -4.58 -4.29 -9.57 -9.32 -10.65 -30.61 -0.00 -1.85 -2.15 -2.90 -3.48 -3.32 -6.50 -6.85 -8.02 -24.64

8.31 18.11 18.63 15.25 18.24 20.55 17.36 20.50 22.31 6.32 9.00 20.55 21.88 18.22 21.10 23.71 20.73 24.41 26.12 14.96 8.99 22.15 24.05 18.70 22.68 25.59 21.66 25.75 27.69 20.85

4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4

Disaggregated synchronous: Qwen, H100 Gen8/Train24

Four matched seeds per comparison. Workload / split Gen/train MHz ∆q (%) Energy saving (%) n Small Small Small Small Medium Medium Medium Medium Large Large Large Large Large Large

1800/1650 1800/1500 1650/1650 1650/1500 1800/1500 1800/1350 1650/1500 1650/1350 1800/1500 1800/1425 1800/1350 1800/1200 1650/1350 1650/1200

-1.84 -1.86 -2.26 -3.53 -1.27 -1.14 -2.81 -3.52 -0.96 -2.08 -2.06 -3.16 -3.36 -2.23

12.17 15.63 14.33 16.30 15.67 20.00 17.00 20.63 16.38 18.03 20.16 22.05 21.14 24.21

4 4 4 4 4 4 4 4 4 4 4 4 4 4

Energy-Efficient Reinforcement Learning for Large Language Models

C.9

Disaggregated synchronous: Qwen, GB200

Four matched seeds per comparison. Rows marked 5–15 cover updates 5–15 because at least one seed did not reach update 25 within the job’s time limit. All other rows cover updates 5–25. Workload / split Gen/train MHz ∆q (%) Energy saving (%) n S 8/16 S 8/16 S 8/16 S 8/16 M 8/16 M 8/16 M 8/16 M 8/16 L 8/16; 5–15 L 8/16; 5–15 L 8/16; 5–15 L 8/16; 5–15 S 12/24 S 12/24 S 12/24 S 12/24 M 12/24 M 12/24 M 12/24 M 12/24 L 12/24 L 12/24 L 12/24 L 12/24; 5–15

C.10

1800/1800 1400/1800 1800/1400 1400/1400 1800/1800 1400/1800 1800/1400 1400/1400 1800/1800 1400/1800 1800/1400 1900/1900 1800/1800 1400/1800 1800/1400 1400/1400 1800/1800 1400/1800 1800/1400 1400/1400 1800/1800 1400/1800 1800/1400 1400/1400

-2.68 -10.47 -3.91 -9.41 -0.62 -5.88 +1.55 -4.27 -3.98 -3.67 -7.06 +0.02 -1.15 -8.06 -0.44 -8.54 -0.73 -4.89 -0.47 -3.18 +1.43 +0.53 +1.41 -1.49

9.41 4.82 11.67 8.28 10.51 8.22 16.26 11.53 9.18 12.81 10.25 13.14 10.44 7.51 14.31 10.25 10.62 9.61 13.27 14.67 14.10 14.96 18.28 17.64

4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4

Disaggregated synchronous: Nemotron, H100 Gen16/Train64

Four matched seeds per comparison. Workload / split Gen/train MHz ∆q (%) Energy saving (%) n Small Small Small Small Small Small Medium Medium Medium Medium Medium Medium Large Large Large Large Large Large

1350/1350 1500/1500 1350/1500 1500/1350 1500/1650 1650/1500 1350/1350 1500/1500 1350/1500 1500/1350 1500/1650 1650/1500 1350/1350 1500/1500 1350/1500 1500/1350 1500/1650 1650/1500

-2.44 -1.46 -1.31 -2.07 -3.74 -3.15 -1.16 -2.52 -1.77 -2.48 -0.01 -1.83 -3.95 -1.96 -4.10 -4.45 -1.02 -2.49

26.29 20.96 23.75 23.81 17.20 18.30 26.14 18.92 21.71 23.33 17.11 18.20 23.94 17.84 18.51 22.02 14.20 16.28

4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4

Energy-Efficient Reinforcement Learning for Large Language Models

C.11

Disaggregated synchronous: Nemotron, H100 Gen16/Train32

Four matched seeds per comparison. Rows marked 5–15 cover updates 5–15 because at least one seed did not reach update 25 within the job’s time limit. All other rows cover updates 5–25. Workload / split Gen/train MHz ∆q (%) Energy saving (%) n Small Small Small Small Small Small Medium Medium Medium Medium Medium Medium Large; 5–15 Large; 5–15 Large; 5–15 Large; 5–15 Large; 5–15 Large; 5–15

C.12

1350/1350 1500/1500 1350/1500 1500/1350 1500/1650 1650/1500 1350/1350 1500/1500 1350/1500 1500/1350 1500/1650 1650/1500 1350/1350 1500/1500 1350/1500 1500/1350 1500/1650 1650/1500

-1.73 -2.72 -1.20 -3.26 -1.90 -2.64 -4.22 -3.31 -1.28 -2.20 -2.64 -1.46 -0.06 +1.17 +1.66 +0.99 +0.24 +1.06

24.98 17.89 21.53 21.20 15.50 16.01 23.50 17.08 20.22 22.26 14.25 16.63 25.20 18.87 20.57 24.31 14.36 17.76

4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4

Disaggregated synchronous: Nemotron, GB200

Four matched seeds per comparison. Rows marked 5–15 cover updates 5–15 because at least one seed did not reach update 25 within the job’s time limit. Workload / split Gen/train MHz ∆q (%) Energy saving (%) n S 8/16; 5–15 S 8/16; 5–15 M 8/16; 5–15 M 8/16; 5–15 M 8/16; 5–15 M 8/16; 5–15 M 8/16; 5–15

1400/1400 1900/1800 1400/1400 1800/1800 1400/1800 1800/1400 1800/1900

-4.99 +1.40 -0.88 -0.73 +1.98 -2.46 +0.64

13.31 15.72 16.27 9.42 16.08 13.66 10.55

4 4 4 4 4 4 4

A SYNCHRONOUS H EADROOM AND T RAINING R EWARD

The tables report 67 comparisons over updates 5–25, each with four matched seeds and a native-governor reference over the same interval: 32 drained and 31 in-flight H100 comparisons, plus two Qwen comparisons of each mode on GB200. Drained and in-flight execution use policy ages 1 and 2, respectively. No shorter-window estimates or comparisons with fewer than four paired seeds are included. Incomplete or invalid comparisons are omitted, including large Nemotron in-flight at 16/32 H100s and large Qwen drained runs. Energy saving is the reduction in whole-allocation GPU energy per valid training token, and ∆q is the change in valid training tokens per second. Splits give generation/training GPU counts, S/M/L denote workload sizes, and clock pairs give requested generation/training MHz. Each comparison retains its study’s idle-control policy. Reward differences are candidate minus governor in absolute reward units. Training reward is a diagnostic, not final validation accuracy. Table D1. Qwen, H100, asynchronous drained; updates 5–25. Four matched seeds per row. Split

Size Gen/train MHz Energy saving (%) ∆q (%) ∆ reward

8/24 8/24 8/24 8/24

S S S S

1725/810 1830/345 1830/525 1830/810

+19.36 +15.75 +17.91 +18.73

-3.29 -3.93 -2.29 -1.67

-0.0046 -0.0023 -0.0017 +0.0015

Energy-Efficient Reinforcement Learning for Large Language Models Qwen, H100, asynchronous drained; updates 5–25. (continued) Split

Size Gen/train MHz Energy saving (%) ∆q (%) ∆ reward

8/24 8/24 8/24 8/24 8/24 8/24 16/32 16/32 16/32 16/32 16/32 16/32 16/32 16/32 16/32 16/32

S M M M M M S S S S S M M M M M

1830/1200 1725/810 1830/345 1830/525 1830/810 1830/1200 1725/810 1830/345 1830/525 1830/810 1830/1200 1725/810 1830/345 1830/525 1830/810 1830/1200

+17.53 +20.95 +17.28 +19.10 +19.89 +19.12 +17.40 +14.39 +15.69 +16.30 +15.33 +18.50 +8.42 +16.32 +16.74 +15.60

-0.84 -2.66 -2.06 -1.42 -1.37 -0.85 -2.88 -2.64 -2.13 -1.51 -1.09 -2.26 -15.72 -1.73 -2.38 -2.12

+0.0007 +0.0009 +0.0011 -0.0001 -0.0006 +0.0004 +0.0028 +0.0045 +0.0020 +0.0035 +0.0007 -0.0003 -0.0010 -0.0003 -0.0003 +0.0000

Table D2. Nemotron, H100, asynchronous drained; updates 5–25. Four matched seeds per row. Split

Size Gen/train MHz Energy saving (%) ∆q (%) ∆ reward

16/32 16/32 16/32 16/32 16/32 16/64 16/64 16/64 16/64 16/64 16/64 16/64

S S S S M S S S S M M M

1725/810 1830/810 1830/1200 1830/1500 1830/1200 1725/810 1830/810 1830/1200 1830/1500 1725/810 1830/1200 1830/1500

+19.07 +17.83 +17.14 +9.11 +17.41 +21.46 +20.80 +20.16 +13.70 +21.85 +21.13 +13.91

-1.68 +0.72 +0.95 +0.14 -2.94 -2.92 -1.84 -2.12 -3.07 -1.83 -0.26 +2.41

-0.0019 -0.0017 +0.0000 +0.0045 -0.0025 +0.0000 -0.0007 -0.0045 +0.0028 +0.0015 -0.0038 -0.0025

Table D3. Qwen, GB200, asynchronous drained; updates 5–25. Four matched seeds per row. Split Size Gen/train MHz Energy saving (%) ∆q (%) ∆ reward 4/16 4/16

M M

1900/1400 1900/1700

+7.60 +7.84

-3.10 -1.90

+0.0006 +0.0005

Table D4. Qwen, H100, asynchronous in-flight; updates 5–25. Four matched seeds per row. Split

Size Gen/train MHz Energy saving (%) ∆q (%) ∆ reward

8/24 8/24 8/24 8/24 8/24 8/24 8/24 8/24 16/32 16/32 16/32 16/32 16/32

S S M M M L L L S S S M M

1830/810 1830/1200 1725/1200 1830/810 1830/1200 1725/1200 1830/810 1830/1200 1725/1200 1830/810 1830/1200 1725/1200 1830/810

+18.22 +17.06 +18.60 +19.11 +17.68 +19.00 +19.91 +17.93 +16.02 +15.39 +14.49 +17.78 +16.89

-2.00 -0.80 -2.02 -1.47 -0.89 -2.27 -0.78 -0.93 -2.12 -2.20 -0.96 -1.01 -1.16

+0.0060 -0.0008 +0.0013 -0.0007 +0.0033 -0.0002 -0.0009 +0.0001 +0.0112 +0.0086 +0.0060 +0.0009 +0.0014

Energy-Efficient Reinforcement Learning for Large Language Models Qwen, H100, asynchronous in-flight; updates 5–25. (continued) Split

Size Gen/train MHz Energy saving (%) ∆q (%) ∆ reward

16/32 16/32 16/32 16/32

M L L L

1830/1200 1725/1200 1830/810 1830/1200

+16.07 +17.37 +17.43 +16.36

-0.24 -1.92 -0.42 -0.09

-0.0004 -0.0011 -0.0003 +0.0001

Table D5. Nemotron, H100, asynchronous in-flight; updates 5–25. Four matched seeds per row. Split

Size Gen/train MHz Energy saving (%) ∆q (%) ∆ reward

16/32 16/32 16/32 16/32 16/32 16/32 16/64 16/64 16/64 16/64 16/64 16/64 16/64 16/64

S S S M M M S S M M M L L L

1725/1200 1830/810 1830/1200 1725/1200 1830/810 1830/1200 1725/1200 1830/810 1725/1200 1830/810 1830/1200 1725/1200 1830/810 1830/1200

+17.67 +10.05 +15.92 +18.12 +13.00 +16.38 +21.69 +18.64 +21.27 +15.70 +20.83 +19.23 +12.32 +18.52

-1.78 -13.26 -2.14 -4.05 -15.68 -4.21 +1.48 -5.18 -1.37 -13.57 +0.11 -7.33 -25.93 -7.35

-0.0041 -0.0024 -0.0058 -0.0035 -0.0020 +0.0012 -0.0047 -0.0037 +0.0052 -0.0093 -0.0058 +0.0009 -0.0009 +0.0012

Table D6. Qwen, GB200, asynchronous in-flight; updates 5–25. Four matched seeds per row. Split Size Gen/train MHz Energy saving (%) ∆q (%) ∆ reward 4/16 4/16

M M

1900/1400 1900/1700

+8.91 +4.85

-1.24 +0.67

+0.0015 +0.0004

W ORKLOAD -N ORMALIZED C ONTROLLER R ESULTS

Update time and energy depend on the work each update receives. Response lengths and token counts differ between runs even with the same code and seed, because sampling, node timing, and the policy itself drift apart. These differences often reach 2–3% of throughput, the same size as the clock effects we measure. A direct comparison with one governor run therefore cannot fully separate the cost of the clocks from a difference in work. We repeat each controller comparison against a governor model fitted to the work, using the controller run and two governor runs of the same configuration. This model is fitted offline and is separate from the controller’s online predictor. For each model and execution mode, we pool every update from both governor runs, starting at update 2, and fit update time and GPU energy by ordinary least squares on each update’s workload features. Only governor updates enter the fit, so the model describes only what governor clocks do. The features follow what sets update time in each mode. In synchronous execution, generation is fitted on generated tokens G and longest-response tokens L, training on valid training tokens N , and the remaining time is the governor median. With drained updates, Mistral-NeMo uses L and G and Qwen uses G and N , since nearly every Qwen update reaches the 4096-token limit and L barely varies. With in-flight updates, the fit uses the valid training tokens and longest response of the batch each update consumes, together with the previous update’s valid training tokens, because in-flight work spills into the next update. For each interval, a run’s ratio is its measured total divided by the total the model predicts for the same updates. A ratio of one means the run matched the governor on its own work. The governor reference is the mean of the two governor ratios. Throughput change is the governor ratio divided by the controller ratio, minus one, and energy saving is one minus the controller ratio divided by the governor ratio. Ratios of sums weight each update by its length, as the totals in Table 4 do. All runs are scored on the same updates, and search, transition, and clock-switch updates all count. Pooling two governor runs widens the range of work the model covers, so it predicts the controller’s updates without

Energy-Efficient Reinforcement Learning for Large Language Models

extrapolating. It also reduces the weight of any one governor run, each of which has its own nodes and small slowdowns. Because one model scores both governors, the spread of their ratios shows how much variation remains after accounting for work. Over each full run, every governor lies within 0.7% of the pooled model in time. Whole run Run Qwen, colocated sync. Qwen, disaggregated sync. Mistral, colocated sync. Mistral, disaggregated sync. Qwen, async drained Mistral, async drained Qwen, async in-flight Mistral, async in-flight

After lock

Updates Energy saving Throughput 2–107 2–93 2–265 2–315 2–175 2–300 2–335 2–560

7.1% 16.4% 3.1% 9.3% 13.9% 11.7% 7.5% 6.6%

−1.17% −1.67% −0.42% −1.65% −2.19% −1.84% −1.25% −1.18%

Updates Energy saving Throughput 78–107 62–93 173–265 60–315 90–175 191–300 258–335 314–560

8.3% 22.0% 2.5% 9.7% 18.8% 18.7% 10.3% 7.7%

−1.21% −1.71% +0.08% −1.53% −0.95% −1.02% −1.57% −0.60%

Normalization keeps every energy saving within 2 points of Table 4, and it moves most throughput changes by less than half a point. The exceptions are the Mistral-NeMo runs with throughput gains in Table 4. After accounting for differences in work, both show small losses. Drained Qwen loses 2.19% over its full run, slightly beyond the 2% limit.

Energy-Efficient Reinforcement Learning for Large Language Models

Analyze & explore related research in Conceptio

Related documents