arXiv CSopen access

NoiseTilt: Noise-Tilted Reverse Kernels for Diffusion Reward Alignment

NoiseTilt: Noise-Tilted Reverse Kernels for Diffusion Reward Alignment Jisung Hwang1 , Yunhong Min1 , Jaihoon Kim1 , I-Chao Shen2 , Minhyuk Sung1 1

Aesthetic Image

arXiv:2606.18066v1 [cs.LG] 16 Jun 2026

FLUX

2

KAIST

The University of Tokyo

FLUX

FLUX + Ours

“The fluffy cloud floated beside the fiery sunset and the misty mountain.”

“Wolf”

Spatial Alignment Layout

Counting

Text Alignment

FLUX + Ours

FLUX

FLUX + Ours

“17 eggs.”

VLM-Based Reward FLUX + Ours

“A dog and a bird sitting on a branch while an eagle is flying in the sky.”

Wan2.1

FLUX

FLUX

FLUX + Ours

“Does the overlaid red line represent the horizon of the scene?”

Preference-Aligned Video Generation

Wan2.1 + Ours

“A car moving slowly on an empty street, rainy afternoon.”

Figure 1. Inference-time reward alignment results using our Noise-Tilted Reverse Kernels (NTRK) across diverse reward-guided diffusion applications.

Abstract We introduce the Noise-Tilted Reverse Kernel (NTRK), a reward-guided diffusion sampler that injects reward gradients through the noise term, leaving the pretrained reverse kernel unchanged and requiring only a single sample per step. Reward-guided sampling at inference time has greatly expanded the versatility of pretrained diffusion models. Yet existing methods face a trade-off. Gradient-based guidance shifts the reverse mean, steering generation but pushing intermediate states outside the region that the model was trained on and degrading quality. Search-based methods preserve quality but gain no gradient signal. No prior method achieves both. NTRK resolves this by keeping the reverse mean fixed and biasing the noise term toward high reward. We introduce a whitening operator, the central mechanism behind NTRK, that makes the reward gradient safe to inject as noise without losing its guiding signal. Across various reward alignment tasks, NTRK outperforms recent state-of-the-art baselines without losing sample quality. Remarkably, on aesthetic generation, NTRK surpasses the reward of the best baseline at 500 NFEs using only 25 NFEs, a 20× reduction in compute. Keywords Noise-Tilted Reverse Kernel ; Whitening Operator ; Reward Alignment ; Inference-Time Scaling

NoiseTilt: Noise-Tilted Reverse Kernels

1 Introduction

Latent Visualization

Sampled Image

Atypical Noise

Typical Noise

Inference-time scaling has become one of the most powerful levers for expanding the capabilities of pretrained generative models, enabling alignment to user- and task-specific preferences without costly retraining. In particular, diffusion models, which underpin state-of-the-art image and video generation [6, 35, 54, 70], are well suited for inference-time scaling thanks to their iterative denoising process. When preferences can be expressed as a reward function, this iterative procedure offers a model-agnostic mechanism to reshape the sampling distribution toward highreward regions. As a result, reward-guided sampling at inference time has been applied broadly, including deblurring [7, 10, 13, 18, 57, 62, 74], super-resolution [9, 10, 62, 74], aesthetic image generation [30, 46, 75], spatial alignment [3, 75], Figure 2. Typical vs. atypical noise. Typical noise text–image alignment [29, 30], and increasingly, produces high-quality outputs, whereas atypical noise induces artifacts. controllable video generation [27, 42, 44, 50, 77]. Behind all these applications lies the same based methods preserve the latter but forgo graquestion: how to steer each denoising step toward dient guidance entirely, relying on random samhigh-reward outcomes. These methods broadly pling to find reward-improving directions. To fall into two paradigms, distinguished by how the best of our knowledge, no prior method they modify or utilize the reverse Gaussian ker- achieves both, and our work addresses this gap. nel. The first, mean-shifted (gradient guidance) To resolve this trade-off, we propose the Noisemethods, pioneered by DPS [10], augment the Tilted Reverse Kernel (NTRK), a drop-in altermean of the reverse Gaussian kernel with re- native that leaves the reverse-kernel mean enward gradients, steering the trajectory toward tirely unchanged and routes reward information high-reward regions. MCMC-based extensions through the noise term at each step. further build on this principle [3, 63, 76]. The The central challenge is that any signal insecond, search-based methods [29, 38, 46], draw jected through the noise term must still be typiK candidates from the stochastic reverse kernel cal Gaussian noise [24, 25], a multi-faceted propand select among them by a reward criterion erty encompassing norm concentration, spatial such as argmax or importance sampling, without uncorrelatedness, and collective Gaussian statisgradients or kernel modification. Several works tics. A raw reward gradient is usually structured combine both via particle filtering [30, 71, 75]. and deterministic; injecting it directly through The two paradigms represent opposite ends of the noise channel produces atypical noise and a fundamental trade-off between gradient guid- again drives states out-of-distribution, as illusance and noise-compatibility. Mean-shifted meth- trated in Figure 2. Inspired by the concept ods gain the former but introduce a critical mis- of whitening in statistical learning [26, 28], we match: adding reward gradients to the reverse introduce a whitening operator that processes mean displaces sampled states outside the noise- the reward gradient before injection, suppresscompatible regime, the narrow region where the ing structured components incompatible with the pretrained model was trained to operate, with pretrained model while preserving the directional the displacement growing as the gradient magni- information of the gradient. Whitening is theretude increases (see Figure 3). This displacement fore not an auxiliary step but the core mechanism drives intermediate states out-of-distribution and of NTRK, and reward alignment improves directly degrades generation quality. It also makes these with the quality of the whitening operator. methods prone to reward hacking [17]. SearchIn our experiments, we demonstrate the ef2

NoiseTilt: Noise-Tilted Reverse Kernels

fectiveness of NTRK, which can also be combined Table 1. Comparison of reverse-kernel guidance mechanisms. NTRK is the only approach satisfying all with multi-particle strategies such as Best-of-N three properties simultaneously. (BoN) [65]. Across a range of applications, inMean- SearchNoisecluding aesthetic image generation, text-aligned Base Hybrid Shifted Based Tilted image generation, VLM-based reward alignment, — DPS [10] SVDD [38] DAS [30] NTRK preference-aligned video generation as showcased Examples Gradient ✗ ✓ ✗ ✓ ✓ in Figure 1, we show that NTRK, paired with sim- guidance ple BoN, can outperform recent state-of-the-art Pretrained ✓ ✗ ✓ ✗ ✓ -compatible approaches that employ more complex techniques Draws / step 1 1 K K 1 such as trajectory rollback [76], particle filtering [29, 30, 38], and initial-point search [75]. In particular, for aesthetic image generation, our for inference-time alignment. Numerous extenmethod surpasses the reward achieved by the sions have since been proposed to improve samstate-of-the-art baseline with 500 NFEs using pling efficiency, accuracy, and applicability: Freeonly 25 NFEs (i.e., 5%), while maintaining im- DoM [76] incorporates additional Monte Carlo sampling, other works develop more advanced age quality. solvers [18, 57, 62, 63, 71], and the framework has been extended to latent diffusion models [55, 56] 2 Related Work enabling scalable applications in large generative We review prior work on reward alignment for systems [54], and applied across image and video diffusion models at inference time. Fine-tuning- generation [3, 12, 34, 36, 74]. As shown in Tabased approaches include direct backpropaga- ble 1, however, shifting the kernel mean pushes tion [11, 48] and reinforcement learning [5, 16, samples outside the noise-compatible Gaussian 51, 69]; we focus on inference-time methods that regime that pretrained models are trained on. operate directly on pretrained models without additional training, and can further be applied on top of fine-tuned models [37, 41]. We organize these into two threads: methods that modify the per-step reverse kernel, grouped into three categories summarized in Table 1, and methods for making reward gradients noise-compatible, which underpin our whitening operator.

Search-based methods. Rather than modifying the kernel mean, search-based methods exploit the stochasticity of the reverse diffusion process by drawing K candidates from the unmodified base kernel and selecting those with higher rewards [46, 52, 61]. These methods trade additional computation for improved sample quality while strictly preserving the native reverse kernel. SVDD [38] is a representative example that selects the highest-reward sample at each denoising step; RBF [29] improves efficiency by dynamically allocating the sampling budget across timesteps. These methods remain noisecompatible but forgo gradient guidance and require K draws per step.

2.1 Reverse Kernel Methods

Rather than manipulating attention maps [8, 19, 20, 33, 47], which tend to be task- and modelspecific, we focus on methods that modify the reverse kernel using a differentiable reward. A separate line that treats the injected noise as an optimization variable [15, 66] is covered in the Hybrid methods. A third group combines grasecond thread below. dient guidance with multi-particle sampling, gainMean-shifted reverse kernels. A foundational ing the directionality of gradient guidance and method in this line is DPS [10], which incorpo- the diversity of multiple draws. DAS [30] incorporates reward information directly into the diffu- rates reward gradients within a Sequential Monte sion sampling process by shifting the mean of Carlo (SMC) framework [14], and Ψ-Sampler [75] each reverse kernel using gradients derived from additionally reshapes the initial particle distributhe reward model via Tweedie’s formula [53]. By tion toward the reward-aligned posterior. Howleveraging the pretrained diffusion prior, this ap- ever, because these methods still shift the kernel proach can be applied to diverse tasks and has be- mean to inject gradient guidance, they inherit the come one of the most widely adopted frameworks noise-compatibility issue of mean-shifted kernels.

3

NoiseTilt: Noise-Tilted Reverse Kernels

Across all three groups, no existing approach 3 Overview simultaneously achieves gradient guidance and We formalize the reward alignment problem and noise-compatibility. analyze four design choices for the per-step reverse kernel. The central question is how reward 2.2 Noise-Compatible Perturbations information can be incorporated at each denoisOur work addresses this gap with NTRK, which ing step without violating the noise-compatible routes reward information through the stochastic regime that the pretrained model relies on. noise term rather than the mean and is the only approach satisfying all three properties in Ta- 3.1 Problem Definition ble 1. The central challenge is that the injected perturbation must remain noise-compatible, a Given a pretrained diffusion model that maps property more demanding than simply matching a source noise distribution, pT = N (0, I), to a a Gaussian norm [24, 25]. Inspired by statisti- data distribution p0 , our objective is to generate cal whitening [26, 28], our whitening operator high-reward samples x0 , a task generally known processes the reward gradient to exhibit Gaus- as reward alignment. Formally, this objective is ∗ sian statistics and spatial uncorrelatedness before formulated as finding a target distribution p0 [32, 67, 68] such that: injection. Regularization-based methods. Several prior \label {eq:reward_max_obj} \begin {aligned} p^{*}_0 = \argmax _{q}\;\mathbb {E}_{\bm {x}_0 \sim q} \left [ r(\bm {x}_0) \right ]-\beta \,\mathcal {D}_{\text {KL}} \left [ q \| p_0 \right ], \end {aligned} (1) works have studied how to encourage Gaussian typicality in vectors involved in diffusion infer- which maximizes the expected reward r(x0 ) while ence, each from a different context: earlier ap- the KL divergence acts as a regularizer, preventproaches apply norm-based regularization to keep ing deviation from the pretrained data distrioptimized noise or latent vectors near the Gaus- bution. The temperature β > 0 controls this sian concentration shell [4, 15, 58], while DNO [66], trade-off: smaller β produces stronger reward MPGR [25], and StressDream [60] additionally tilting, while larger β stays closer to the preregularize higher-order spatial statistics alongside trained distribution. norm, in the respective contexts of noise-space opThe optimal reverse kernel p∗θ (xt−1 | xt ) retimization, gradient guidance stabilization, and quired to sample from the target distribution in video world model steering. All of these impose Equation (1) can be approximated as follows: soft constraints for specific properties of Gaussian noise, however, and cannot guarantee those \label {eq:optimal_policy} \begin {aligned} p^*_\theta (\bm {x}_{t-1} | \bm {x}_{t}) \propto p_\theta (\bm {x}_{t-1} | \bm {x}_{t})\exp \left ( \tfrac {V_{t-1}(\bm {x}_{t-1})}{\beta } \right ), \end {aligned} (2) properties. where the exact value function Vt (xt ) is intractable Projection-based methods. WGNC [24] is and is approximated using Tweedie’s posterior the first to reframe noise-compatibility as a promean [53]. That is, Vt (xt ) ≈ r(x̂0|t ), where jection problem, directly mapping the reward x̂0|t := E[x0 | xt ]. The design of the per-step regradient onto a white Gaussian noise feasible set verse kernel determines how this approximation defined by hard blockwise norm constraints in the is realized in practice, and whether the resulting Fourier domain. This is the first approach in the updates remain noise-compatible. spirit of true statistical whitening, achieving typIn the following sections, we first recall the icality enforcement in a single pass. However, its base diffusion reverse kernel and then compare hard equality constraints also distort noise that two common approximation routes to reward is already noise-compatible. Our whitening opguidance: mean-shifted reverse kernels and searcherator upgrades this projection approach: by rebased selection. Lastly, in Section 3.5, we introplacing hard equality constraints with confidenceduce our proposed noise-tilted reverse kernel that interval projections, it strongly suppresses atyppreserves the base mean while injecting reward ical structure while leaving genuine Gaussian information through the stochastic term. These noise nearly unchanged (see Section B.7 for a four kernels are illustrated in Figure 3. detailed comparison).

4

NoiseTilt: Noise-Tilted Reverse Kernels

Base Kernel

Mean-Shifted

1 draw / step | Section 3.2

1 draw / step | DPS [10], Section 3.3

xt−1 = µθ (xt , t) + σt ϵt ,

xt−1 = µ̃θ (xt , t) + σt ϵt ,

ϵt ∼ N (0, I).

µ̃θ (xt , t) := µθ (xt , t) + λt ∇xt r(x̂0|t ).

Search-Based

Noise-Tilted

K draws / step | SVDD [38], Section 3.4

1 draw / step | Ours, Section 3.5

xt−1 = µθ (xt , t) + σt ϵ⋆t ,

xt−1 = µθ (xt , t) + σt ϵ̃t ,

(i⋆ )

ϵ⋆t = ϵt

(i)

,

(i)

i⋆ = arg max r(x̂0|t−1 ), i∈{1,...,K}

ϵ̃t =

(i)

√

ρt W(∇xt r(x̂0|t )) +

√ 1 − ρt ϵt .

xt−1 = µθ (xt , t) + σt ϵt . Figure 3. Reverse-kernel guidance mechanisms. The green annulus indicates the noise-compatible regime; blue dots show the induced sample distribution. Mean-shifted guidance injects reward information by shifting the reverse mean, pushing samples outside the noise-compatible regime. Search-based guidance preserves the base mean by selecting the best among K candidate noise draws. NTRK also preserves the base mean but constructs a single reward-tilted noise draw, achieving reward alignment without leaving the noise-compatible regime.

5

NoiseTilt: Noise-Tilted Reverse Kernels

3.2 Diffusion Reverse Kernel

tended, including advanced sampling schemes [18, 76], particle-based sampling [30, 75], and apA diffusion model [21, 64] progressively denoises an initial Gaussian noise by sequentially apply- plications ranging from images [3, 36, 74] to ing a Markovian reverse kernel, modeled as a videos [12, 34]. Despite its practical success, a closer look at Gaussian transition from xt to xt−1 over discrete the standardized perturbation reveals a subtle yet timesteps t = T, . . . , 0: critical mismatch with the base reverse dynamics. \label {eq:reverse_kernel} p_\theta (\bm {x}_{t-1} \mid \bm {x}_t) = \mathcal {N}\big (\bm {x}_{t-1}; \bm {\mu }_\theta (\bm {x}_t,t),\sigma _t^2\bm {I}\big ), (3) Identifying and analyzing this discrepancy is one of the main contributions of this work. To make where the mean µθ (xt , t) is parameterized using the mismatch explicit, consider the standardized a neural network, and the variance σt2 controls perturbation induced by Equation (6) relative to the amount of stochasticity injected each step. the base mean µθ (xt , t): The visualization of Equation (3) is presented in Figure 3. \label {eq:eta_mean} \bm {\eta }_t^\text {mean} = \bm {\epsilon }_t + \frac {\lambda _t}{\sigma _t}\nabla _{\bm {x}_t}r(\hat {\bm {x}}_{0|t}). (7) To generate a sample in practice, xt−1 is drawn from the distribution in Equation (3) using The deterministic reward gradient term in Equathe standard reparameterization trick: tion (7) means that the standardized perturba \label {eq:base_sampling} \begin {aligned} \bm {x}_{t-1} &= \bm {\mu }_\theta (\bm {x}_t,t)+\sigma _t\bm {\epsilon }_t,\\ \bm {\epsilon }_t&\sim \mathcal {N}(\bm {0},\bm {I}). \end {aligned}

tion is no longer a Gaussian noise assumed by the pretrained reverse kernel. Consequently, the reverse updates can drift away from the learned intermediate distributions, and repeated iterations may push samples toward regions that are insufficiently supported by the pretrained diffusion in practice. An alternative approach avoids this issue by preserving the base mean and instead biasing the stochastic perturbation, as we describe in the following subsection.

(4)

Since the reverse kernel is parameterized as a Gaussian distribution, any valid transition between consecutive latent states must satisfy the condition that the standardized perturbation ηt is a Gaussian noise: \label {eq:eta_base} \bm {\eta }_t := \frac {\bm {x}_{t-1} - \bm {\mu }_\theta (\bm {x}_t, t)}{\sigma _t}.

(5)

3.4 Search-Based Reverse Kernel

Under the base reverse process, ηt = ϵt holds trivially, so the standardized perturbation is in- To approximate the optimal reverse transition in Equation (2), search-based methods draw K canherently a standard Gaussian noise. didates from the base reverse kernel, all centered at the pretrained mean: 3.3 Mean-Shifted Reverse Kernel [10] For efficient reward alignment, first-order reward \label {eq:search_kernel} \bm {x}_{t-1}^{(i)} = \bm {\mu }_\theta (\bm {x}_t,t)+\sigma _t\bm {\epsilon }_t^{(i)}, \quad \bm {\epsilon }_t^{(i)}\sim \mathcal {N}(\bm {0},\bm {I}), (8) information can be injected directly into the reverse transition in Equation (3). Concretely, the where i = 1, . . . , K. As a representative searchmean-shifted reverse kernel [10] modifies the base based method, SVDD [38] selects the candidate reverse kernel by adding this gradient to its mean: whose predicted reward score is highest: \label {eq:search_select} i^\star = \argmax _{i\in \{1,\dots ,K\}}\,r\!\left (\hat {\bm {x}}_{0|t-1}^{(i)}\right ), \quad \bm {x}_{t-1} = \bm {x}_{t-1}^{(i^\star )}. \label {eq:grad_kernel} \bm {x}_{t-1} = \bm {\mu }_\theta (\bm {x}_t,t) + \lambda _t\nabla _{\bm {x}_t}r(\hat {\bm {x}}_{0|t}) + \sigma _t \bm {\epsilon }_t,

(9)

(6) Reward information therefore appears ⋆through (i ) the selected stochastic perturbation ϵt rather than through a deterministic mean shift. As illustrated in Figure 3, search-based guidance can therefore be interpreted as an implicit form of noise tilting. However, when high-reward samples are in low-density regions of the pretrained distribution, this becomes inefficient, requiring a prohibitively large number of samples. We

where ϵt ∼ N (0, I), λt > 0 is a guidance hyperparameter, and detailed derivations are provided in Section A.2. As illustrated in the top-right panel of Figure 3, Equation (6) injects reward guidance by shifting the reverse-kernel mean while keeping the injected noise term unchanged. Since its introduction, this mean-shifted reverse kernel has been widely adopted and ex-

6

NoiseTilt: Noise-Tilted Reverse Kernels

provide a more detailed derivation of this view to the base mean becomes in Section A.3. In the following subsection, we \label {eq:eta_ours} \bm {\eta }_t^{\text {noise}} := \frac {\bm {x}_{t-1}-\bm {\mu }_\theta (\bm {x}_t,t)}{\sigma _t} = \bm {\tilde {\epsilon }}_t. (13) propose a method that retains this advantage of preserving the base mean while reducing the sampling cost to a single draw. The kernel is thus noise-tilted in that reward guidance is expressed entirely through the noise term 3.5 Noise-Tilted Reverse Kernel (NTRK) rather than the mean, as formalized in EquaOur key idea is to preserve the pretrained re- tion (13). Crucially, Equation (12) preserves verse mean µθ (xt , t) and inject reward informa- the pretrained reverse-kernel structure, using the tion only through the noise, thereby avoiding same base mean µθ (xt , t) and the same nominal the noise-compatibility mismatch of mean-shifted noise scale σt as Equation (3). Consequently, kernels. Unlike search-based selection, which re- each step remains noise-compatible with the prequires drawing K candidates per step, our ap- trained reverse dynamics. As shown in the right panel of Figure 3, samples of xt−1 are drawn proach achieves this with a single noise draw. toward higher-reward regions while remaining The challenge is that a raw reward gradient within the regime where the pretrained model is usually structured and deterministic rather operates reliably. than Gaussian, so it cannot directly serve as A core contribution of this work is the cona noise-compatible perturbation. We therefore struction of the whitening operator W so that it introduce a whitening operator W that maps the reward gradient to a noise-compatible direction, is statistically well-founded in high dimensions and define the whitened reward direction wt := and expressive enough to preserve reward signal. In Section 4, we formalize the required statistical W(∇xt r(x̂0|t )). We formalize W in Section 4. conditions and introduce our confidence-interval To incorporate wt into the noise term while projection method to realize W in practice. preserving its Gaussian form, we rely on a basic identity: if ϵ1 , ϵ2 ∼ N (0, I) are independent, then for any ρ ∈ [0, 1], 4 Whitening Operator \label {eq:noise_add} \sqrt {\rho }\,\bm {\epsilon }_1+\sqrt {1-\rho }\,\bm {\epsilon }_2 \sim \mathcal {N}(\bm {0},\bm {I}).

(10) Our goal is to transform an arbitrary input vector into a typical standard Gaussian noise vector. AlWe define the guided injected noise by mixing the though N (0, I) has nonzero density everywhere whitened direction with unbiased stochasticity: in RN , in high dimensions almost all probability mass concentrates on a narrow typical region. \label {eq:tilde_eps} \bm {\tilde {\epsilon }}_t = \sqrt {\rho _t}\,\bm {w}_t + \sqrt {1-\rho _t}\,\bm {\epsilon }_t, (11) Pretrained generative models are trained on trajectories whose injected perturbations lie in this where ϵt ∼ N (0, I) and ρt ∈ [0, 1] controls the typical region; consequently, perturbations that guidance strength. Intuitively, the first term indrift away from it can act as out-of-distribution jects a whitened guidance component that carries inputs to the learned reverse dynamics (see Figreward-gradient information, while the second ure 2). We therefore design a whitening operator term preserves unbiased stochasticity. A heurisW : RN → RN that moves the reward gradient tic interpretation of ρt relative to search-based direction toward typical standard Gaussian noise guidance under a local linearity assumption is before it is mixed into the stochastic term of the provided in Section A.5. reverse kernel. Using Equation (11), we replace the base samThe typical set is difficult to characterize expling rule in Equation (4), resulting in our noiseactly in closed form. Instead, we approximate it tilted reverse kernel, NTRK: using a collection of high-confidence constraints induced by known statistics of the standard nor \label {eq:noise_kernel} \bm {x}_{t-1} = \bm {\mu }_\theta (\bm {x}_t,t) + \sigma _t\bm {\tilde {\epsilon }}_t. (12) mal distribution. Concretely, we precompute Most importantly, the standardized noise relative 99.99% confidence bounds and define W as a sequence of projections onto the corresponding confidence sets. A key building block is a twolevel order-statistic projection (2OS), which we 7

NoiseTilt: Noise-Tilted Reverse Kernels

2OS Projection

+

Tile-wise Statistics

+

Multiple Domains

Sampled Noise z

W(z)

Sampled Image

Latent Visualization

Input

Figure 4. Effect of our whitening operator W. As we apply the components of W to a structured latent (left to right), it becomes more similar to typical Gaussian noise with more realistic samples. For typical Gaussian noise, W(z) changes negligibly (two rightmost columns).

element to its confidence interval

describe next.

Two-level order statistics. Let x ∈ RN be Z_{r,j}\leftarrow \operatorname {clip}(Z_{r,j};L_{r,j},U_{r,j}), \label {eq:whiten_clip} (17) the vector to whiten, reshaped into a tile matrix Y ∈ RM ×D (so N = M D). The 2OS statistic and then invert the two sorting permutations to is obtained by sorting within each tile and then map the clipped Z back to the original tile layout. We prove in Section B.2 that this sort–clip–unsort sorting across tiles at each within-tile rank: operation equals the Euclidean projection onto \bm {Z}=\mathrm {sort}_0\bigl (\mathrm {sort}_1(\bm {Y})\bigr ), \label {eq:whiten_2os_def} (14) C2os , and we therefore call it the 2OS projection. The 2OS projection prevents extreme values where sort1 sorts each row of Y and sort0 sorts from concentrating in a few tiles. Because clipeach column. Intuitively, Z captures a “rank-ofping is applied to each rank pair (r, j) of the rank” summary: Zr,j is the r-th smallest value doubly-sorted statistic, each tile contains a balamong the j-th order statistics collected from all anced spread of small-to-large values consistent tiles. For standard Gaussian noise, each entry with typical standard Gaussian noise. Applying Zr,j concentrates sharply, allowing tight confi2OS additionally to tile-wise mean and energy dence bounds for each (r, j). statistics constrains block-level moments, while Confidence bounds for the 2OS statistic. repeating across orthogonally transformed doFix α = 10−4 and let Φ denote the CDF of mains captures structured correlations that the the standard normal distribution. For each rank value-domain projection alone cannot reach. lo , q hi ) The full whitening operator W is composed of pair (r, j), we compute quantile bounds (qr,j r,j from the nested order-statistic distribution, and 2OS projections applied to various tile-wise statistics across multiple transformed domains, as demap them to value-domain bounds scribed in Section B. We visualize this progression (L_{r,j},U_{r,j}) = \bigl (\Phi ^{-1}(q^{\mathrm {lo}}_{r,j}), \Phi ^{-1}(q^{\mathrm {hi}}_{r,j})\bigr ). \label {eq:whiten_2os_quantile_lo} (15) in Figure 4. Since all constraints are derived from 99.99% confidence intervals under the reference The exact Beta-quantile construction is given in distribution, typical standard Gaussian noise Section B.2. passes through with negligible modification in These bounds specify a 1 − α confidence set practice (cosine similarity > 0.99999; see the two for the doubly-sorted matrix Z: rightmost columns in Figure 4), whereas structured inputs are substantially whitened, yielding \mathcal {C}_{\mathrm {2os}} = \{\bm {Y}:L_{r,j}\le Z_{r,j}\le U_{r,j},\ \forall r,j\}. \label {eq:whiten_confset_2os} (16) a perturbation that is noise-compatible with the pretrained reverse kernel. 2OS projection via sort–clip–unsort. Given Y , we compute Z as in Equation (14), clip each 8

NoiseTilt: Noise-Tilted Reverse Kernels

Table 2. Quantitative comparison on image reward alignment. Left: aesthetic image generation (target: Aesthetic Score [59]). Right: text-aligned image generation (target: PickScore [31]). For single-particle methods we augment sampling with Best-of-N to match the total NFE, denoted with † . Dark green cells indicate the best result for each metric, light green the second best. Aesthetic Image Generation Method

NFE

Target Reward

Text-Aligned Image Generation Target Reward

Held-Out Reward

Held-Out Reward

Aesthetic Pick Image VQA Pick Aesthetic Image VQA ↑ ↑ HPSv2 ↑ ↑ ↑ ↑ ↑ HPSv2 ↑ ↑ ↑ Score -Score Reward Score -Score Score Reward Score Base [35] BoN [65] DPS† [10] FreeDoM† [76] SVDD [38] RBF [29] DAS [30] Ψ-Sampler [75]

25 500 500 533 500 500 500 500

6.0282 6.7310 6.7647 6.8406 7.1363 6.9900 6.9384 7.0116

0.2144 0.2197 0.2191 0.2185 0.2177 0.2183 0.2183 0.2188

0.2759 0.2890 0.2861 0.2853 0.2814 0.2826 0.2860 0.2847

1.0538 1.1419 1.0639 0.9941 1.0256 1.0761 1.0568 1.1235

0.9644 0.9597 0.9624 0.9635 0.9510 0.9689 0.9706 0.9737

0.2054 0.2146 0.2147 0.2133 0.2204 0.2202 0.2139 0.2120

5.4664 5.8582 5.8073 5.8492 5.8743 5.8618 5.8385 5.7329

0.2316 0.2619 0.2622 0.2572 0.2699 0.2682 0.2568 0.2551

0.1710 0.6883 0.6310 0.5354 0.7592 0.7583 0.5226 0.4590

0.8011 0.8021 0.8028 0.7990 0.8201 0.8149 0.7990 0.8145

NTRK (Ours) NTRK† (Ours)

25 500

7.4510 7.9656

0.2200 0.2197

0.2928 0.2932

1.2565 1.1669

0.9728 0.9609

0.2224 0.2327

5.7720 5.9020

0.2601 0.2817

0.5257 0.7370

0.8210 0.8017

5 Experiments

flow models, respectively. In all experiments, we fix the sampling steps to 25. As a reference, In this section, we present experimental results we also include the results of the base models demonstrating the effectiveness of NTRK com- without any guidance method applied. pared to prior baselines, with the experimental setup described in Section 5.1. We first evalu- Baselines. We compare NTRK against a range ate aesthetic image generation and text-aligned of inference-time reward-alignment algorithms image generation in Section 5.2 and Section 5.3, discussed in Section 2, including both gradientrespectively. We then extend the evaluation to based guidance methods and search-based appreference-aligned video generation in Section 5.4. proaches. Specifically, we consider DPS [10] and Finally, we show that our method can also be FreeDoM [76] as single-particle gradient-based applied on top of fine-tuned models in Section 5.5. methods, DAS [30] and Ψ-Sampler [75] as multiWe provide additional applications, including particle gradient-based methods, and SVDD [38], counting tasks and VLM-based reward alignment, RBF [29], and BoN [65] as search-based methin Sections C.3 and C.4, and alignment results us- ods. We additionally compare against DNO [66], ing a different diffusion model [6] in Section C.5. a noise-optimization method with a distinct NFE Additional quantitative and qualitative results budget structure, in Section C.2. Note that multi-particle and search methods for the main experiments are provided in Secutilize multiple samples during sampling, whereas tion C.6. single-particle methods produce only a single tra5.1 Experiment Setup jectory. For fair comparison, we fix the total Tasks. We evaluate NTRK across three reward- number of function evaluations (NFE) across all guided generation settings in the main text: aes- methods. In particular, for single-particle samthetic image generation, text-aligned image gen- pling methods such as DPS and FreeDoM, we eration, and preference-aligned video generation. augment sampling with Best-of-N (BoN) [65], For aesthetic image generation, we use 45 animal which runs multiple independent sampling proprompts from previous work, DDPO [5]. For text- cesses and selects the highest-reward output, enaligned image generation, we use 100 prompts suring that1 the overall NFE is comparable across in complex category of T2I-CompBench++ [22]. methods. For all quantitative results, methods † For preference-aligned video generation, we use augmented with BoN are marked with . For 200 prompts from VBench [23] animal and scenery NTRK, we report results both with and without BoN to isolate the effect of the proposed method. categories. 1 FreeDoM [76] incorporates additional MCMC samFor image and video generation applications, pling, which increases the computational cost. we use FLUX [35] and Wan2.1 [70] as the base

9

NoiseTilt: Noise-Tilted Reverse Kernels Base [35]

BoN [65]

DPS† [10]

SVDD [38]

Ψ-Sampler [75]

NTRK† (Ours)

6.8442

7.0671

8.5842

7.0071

7.1417

8.6239

Aesthetic Image Generation

“Sheep”

6.5477

6.6494

6.9178 “Frog”

6.0279

7.1517

6.9119

Text-Aligned Image Generation

“The yellow cone was suspended in mid-air near the orange pyramid and the green cylinder.”

0.2190

0.2227

0.2235

0.2328

0.2224

0.2372

“The soft, billowing curtains fluttered in the gentle breeze, adding a touch of elegance to the room.”

0.2070

0.2126

0.2122

0.2146

0.2055

0.2313

Figure 5. Qualitative comparison on image reward alignment. Top: aesthetic image generation (target: Aesthetic Score [59]). Bottom: text-aligned image generation (target: PickScore [31]). Scores shown in italics below each image. For single-particle methods we augment sampling with Best-of-N to match the total NFE, denoted with † .

Implementation details and hyperparameter set- spectively. Overall, NTRK achieves the best tartings are summarized in Section C.1. get reward performance across all methods, and even outperforms all baselines with only 1/20 of 5.2 Aesthetic Image Generation NFE. On held-out rewards, NTRK also delivers Evaluation Metrics. In this work, we refer to the best image quality for both ImageReward the reward used for inference-time optimization and HPSv2, while remaining comparable on textas the target reward, and to rewards not observed image alignment metrics. The qualitative exduring optimization as held-out rewards. In this amples in Figure 5 further support these trends. task, the target reward is Aesthetic Score [59]. Across different prompts, NTRK produces more As held-out rewards, we evaluate image quality visually appealing samples than the baselines, using ImageReward [73] and HPSv2 [72], and achieving the highest rewards [59]. text–image alignment using PickScore [31] and 5.3 Text-Aligned Image Generation VQA Score [39]. Evaluation Metrics. The target reward used Results. The quantitative and qualitative re- to align text-image is PickScore [31]. For heldsults are presented in Table 2 and Figure 5, re10

NoiseTilt: Noise-Tilted Reverse Kernels

Table 3. Quantitative comparison on preference-aligned video generation. The target reward is VideoReward [42], and the held-out rewards are the metrics proposed in VBench [23]. Dark green cells indicate the best result for each metric across all runs, while light green cells denote the second best. Target Reward Method

Base [70] DPS [10] FreeDoM [76] NTRK (Ours)

NFE

25 25 25 25

Held-Out Reward Motion Quality

VideoReward ↑

Visual Quality

Text Alignment

Smooth.↑ Dynamic↑ Aesthetic↑ Imaging.↑ Subject↑ Back.↑

-0.399 -0.130 -0.211 3.465

0.9629 0.9621 0.9632 0.9630

0.9300 0.9350 0.6900 0.9500

“Turkey in cage.”

0.6104 0.5867 0.5880 0.6120

0.6791 0.6738 0.6897 0.6870

0.9589 0.9398 0.9586 0.9591

0.9647 0.9473 0.9570 0.9646

“Mother whale swimming with babies.”

Base [70]

DPS [10]

FreeDoM [76]

NTRK (Ours)

Figure 6. Qualitative comparison on preference-aligned video generation using VideoReward [42]. NTRK produces videos with better text alignment and visual quality, and additional examples are provided in Figure 20.

out rewards, we evaluate text–image alignment using VQA Score [39], and image quality using Aesthetic Score [59], ImageReward [73], and HPSv2 [72].

5.4 Preference-Aligned Video Generation As done in the image generation task, we use 25 sampling steps for all methods except FreeDoM [76], for which we use 13 steps due to its additional MCMC sampling, ensuring that the total NFE remains comparable. For video generation task, we test the baselines and our method with a single particle.

Results. The quantitative and qualitative results are presented in Table 2 and Figure 5, respectively. As in the aesthetic image generation task, NTRK achieves the best target reward performance across all methods, with the same ef- Evaluation Metrics. In this task, the target ficiency trend that the 25-NFE setting already reward is VideoReward [42], which provides three outperforms all baselines. These gains also trans- component scores for Motion Quality (MQ), Vifer to held-out rewards: NTRK with 500 NFE sual Quality (VQ), and Text Alignment (TA). achieves the best Aesthetic Score and HPSv2, We use the sum of these components (MQ + while NTRK with 25 NFE attains the best VQA VQ + TA) as the target reward. For held-out Score. Qualitatively, NTRK produces samples that evaluation, we report metrics from VBench [23], better align with text prompts, particularly on grouped into six categories: Subject Consistency spatial and logical relations, compared to the and Background Consistency for text alignment, baselines. Motion Smoothness and Dynamic Degree for motion quality, and Aesthetic Quality and Imaging Quality for visual quality.

11

NoiseTilt: Noise-Tilted Reverse Kernels

Table 4. Quantitative comparison on text-aligned image generation with fine-tuned model. We compare DPS [10] and NTRK integrated with a fine-tuned model, MixGRPO [37]. For single-particle methods we augment sampling with Best-of-N, denoted with † . Dark green cells indicate the best result for each metric across all runs, while light green cells denote the second best.

Target Reward Method

Base [35] MixGRPO [37] ⌞ DPS† [10] ⌞ NTRK† (Ours)

NFE

Held-Out Reward

PickScore ↑

Aesthetic ↑ Score

HPSv2 ↑

Image ↑ Reward

VQA ↑ Score

25 25

0.2054 0.2166

5.4664 6.5245

0.2316 0.2679

0.1710 0.7605

0.8011 0.8239

500 500

0.2235 0.2545

6.6966 6.7281

0.2840 0.3224

1.0501 1.2648

0.8376 0.8498

Results. Table 3 and Figure 6 summarize the quantitative and qualitative comparisons, respectively. NTRK yields the highest VideoReward score and outperforms all the baselines. Beyond the target reward, NTRK achieves the best results on Dynamic Degree, Aesthetic Quality, and Subject Consistency, and remains marginally runner-up on the other held-out metrics. The qualitative examples further support this observation, with NTRK producing videos that better align with the text prompts. In particular, NTRK generates videos that clearly capture the elements (e.g., turkey and babies) described in the prompt.

provements across all metrics. In particular, NTRK consistently outperforms DPS on both the target reward and all held-out rewards, achieving the highest overall scores. These results confirm that NTRK is orthogonal to fine-tuning: the two can be composed to push a specific reward beyond what either approach achieves alone.

6 Conclusion

In this work, we identified a fundamental tradeoff in inference-time reward alignment: gradientbased guidance steers generation effectively but degrades sample quality, while search-based meth5.5 Integration with Fine-Tuned Models ods preserve quality but forgo gradient guidance. Fine-tuning and inference-time alignment im- We resolved this trade-off with the Noise-Tilted prove reward alignment along orthogonal direc- Reverse Kernel (NTRK), which leaves the reverse tions: the former adapts model parameters, while mean unchanged and instead biases the noise the latter guides the sampling process without term toward high reward through a whitening modifying them. Because of this orthogonality, operator that makes the reward gradient safe to NTRK can be applied on top of a fine-tuned model, inject as noise without losing its guiding signal. Across aesthetic image generation, text-aligned and the two strategies can be combined whenever the goal is to maximize a specific reward as much image generation, and preference-aligned video as possible. In this section, we integrate NTRK generation, NTRK consistently outperforms recent with MixGRPO [37], which fine-tunes the base baselines in target reward alignment without losing sample quality, and on aesthetic generation FLUX model [35]. surpasses the reward of the best baseline at 500 Evaluation Metrics. In this task, the target NFEs using only 25, a 20× reduction in compute. reward is PickScore [31], and we report the same Beyond this setting, NTRK improves reward alignheld-out rewards used in the text-aligned image ment on top of fine-tuned models, transfers to generation task described in Section 5.3. a different diffusion backbone, and extends to counting-based and VLM-based rewards. Results. As shown in Table 4, MixGRPO [37] By routing reward information through the improves both the target reward and all held-out rewards compared to the base model [35]. We noise term rather than the mean, NTRK offers a further observe that applying inference-time re- simple, broadly applicable mechanism for inferenceward alignment remains highly effective on top time reward alignment that leaves the pretrained of the fine-tuned model, yielding additional im- sampling dynamics intact. 12

NoiseTilt: Noise-Tilted Reverse Kernels

References

[9] Chung, H., Sim, B., Ryu, D., and Ye, J. C. Improving diffusion models for inverse problems using manifold constraints. In Advances in Neural Information Processing Systems, volume 35, pp. 25683–25696, 2022.

[1] Amini-Naieni, N., Han, T., and Zisserman, A. Countgd: Multi-modal open-world counting. In Advances in Neural Information Processing Systems, volume 37, pp. 48810– [10] Chung, H., Kim, J., Mccann, M. T., Klasky, 48837, 2024. M. L., and Ye, J. C. Diffusion posterior sam[2] Bai, S., Chen, K., Liu, X., Wang, J., Ge, pling for general noisy inverse problems. In W., Song, S., Dang, K., Wang, P., Wang, International Conference on Learning RepS., Tang, J., Zhong, H., Zhu, Y., Yang, M., resentations, 2023. Li, Z., Wan, J., Wang, P., Ding, W., Fu, Z., Xu, Y., Ye, J., Zhang, X., Xie, T., Cheng, [11] Clark, K., Vicol, P., Swersky, K., and J, F. D. Directly fine-tuning diffusion models on difZ., Zhang, H., Yang, Z., Xu, H., and Lin, J. ferentiable rewards. In International ConQwen2.5-VL technical report. arXiv preprint ference on Learning Representations, 2024. arXiv:2502.13923, 2025.

[3] Bansal, A., Chu, H.-M., Schwarzschild, A., [12] Daras, G., Nie, W., Kreis, K., Dimakis, Sengupta, S., Goldblum, M., Geiping, J., A., Mardani, M., Kovachki, N., and Vahand Goldstein, T. Universal guidance for dat, A. Warped diffusion: Solving video indiffusion models. In IEEE/CVF Conference verse problems with image diffusion models. on Computer Vision and Pattern RecogniIn Advances in Neural Information Processtion Workshops, 2023. ing Systems, volume 37, pp. 101116–101143, 2024. [4] Ben-Hamu, H., Puny, O., Gat, I., Karrer, B., Singer, U., and Lipman, Y. D-Flow: [13] Dou, Z. and Song, Y. Diffusion posterior Differentiating through flows for controlled sampling for linear inverse problem solving: generation. In International Conference on A filtering perspective. In International ConMachine Learning, pp. 3462–3483, 2024. ference on Learning Representations, 2024. [5] Black, K., Janner, M., Du, Y., Kostrikov, [14] Doucet, A., De Freitas, N., Gordon, N. J., I., and Levine, S. Training diffusion modet al. Sequential Monte Carlo methods in els with reinforcement learning. In Internapractice. Springer, 2001. tional Conference on Learning Representa[15] Eyring, L., Karthik, S., Roth, K., Dosovittions, 2024. skiy, A., and Akata, Z. ReNO: Enhanc[6] Cai, H., Cao, S., Du, R., Gao, P., Hoi, ing one-step text-to-image models through S., Hou, Z., Huang, S., Jiang, D., Jin, X., reward-based noise optimization. In AdLi, L., et al. Z-image: An efficient image vances in Neural Information Processing generation foundation model with singleSystems, volume 37, pp. 125487–125519, stream diffusion transformer. arXiv preprint 2024. arXiv:2511.22699, 2025. [16] Fan, Y., Watkins, O., Du, Y., Liu, H., Ryu, [7] Cardoso, G., Idrissi, Y. J. E., Corff, S. L., M., Boutilier, C., Abbeel, P., Ghavamzadeh, and Moulines, E. Monte carlo guided diffuM., Lee, K., and Lee, K. DPOK: reinforcesion for bayesian linear inverse problems. In ment learning for fine-tuning text-to-image International Conference on Learning Repdiffusion models. In Advances in Neural Inresentations, 2024. formation Processing Systems, volume 36, pp. 79858–79885, 2023. [8] Chefer, H., Alaluf, Y., Vinker, Y., Wolf, L., and Cohen-Or, D. Attend-and-excite: [17] Gao, L., Schulman, J., and Hilton, J. Scaling Attention-based semantic guidance for textlaws for reward model overoptimization. In to-image diffusion models. ACM TransacInternational Conference on Machine Learntions on Graphics, 42(4):148:1–148:12, 2023. ing, pp. 10835–10866, 2023. doi: 10.1145/3592116. 13

NoiseTilt: Noise-Tilted Reverse Kernels

[18] He, Y., Murata, N., Lai, C.-H., Takida, Y., [26] Hyvärinen, A., Karhunen, J., and Oja, E. Uesaka, T., Kim, D., Liao, W.-H., MitsuIndependent Component Analysis. John Wifuji, Y., Kolter, J. Z., Salakhutdinov, R., ley & Sons, 2001. and Ermon, S. Manifold preserving guided [27] Jang, S., Ki, T., Jo, J., Yoon, J., Kim, S. Y., diffusion. In International Conference on Lin, Z., and Hwang, S. J. Frame guidance: Learning Representations, 2024. Training-free guidance for frame-level control in video diffusion models. In Interna[19] Hertz, A., Mokady, R., Tenenbaum, J., tional Conference on Learning RepresentaAberman, K., Pritch, Y., and Cohen-Or, D. tions, 2026. Prompt-to-prompt image editing with cross attention control. In International Confer[28] Kessy, A., Lewin, A., and Strimmer, K. Optience on Learning Representations, 2023. mal whitening and decorrelation. The American Statistician, 72(4):309–314, 2018. doi: [20] Hertz, A., Voynov, A., Fruchter, S., and 10.1080/00031305.2016.1277159. Cohen-Or, D. Style aligned image generation via shared attention. In Proceedings of the [29] Kim, J., Yoon, T., Hwang, J., and Sung, M. IEEE/CVF Conference on Computer Vision Inference-time scaling for flow models via and Pattern Recognition (CVPR), pp. 4775– stochastic generation and rollover budget 4785, 2024. forcing. In Advances in Neural Information Processing Systems, volume 38, pp. 30830– [21] Ho, J., Jain, A., and Abbeel, P. Denoising 30864, 2025. diffusion probabilistic models. In Advances in Neural Information Processing Systems, [30] Kim, S., Kim, M., and Park, D. Testvolume 33, pp. 6840–6851, 2020. time alignment of diffusion models without reward over-optimization. In Interna[22] Huang, K., Duan, C., Sun, K., Xie, E., tional Conference on Learning RepresentaLi, Z., and Liu, X. T2I-CompBench++: tions, 2025. An enhanced and comprehensive benchmark for compositional text-to-image generation. [31] Kirstain, Y., Polyak, A., Singer, U., MaIEEE Transactions on Pattern Analysis and tiana, S., Penna, J., and Levy, O. Pick-aMachine Intelligence, 47(5):3563–3579, 2025. Pic: An open dataset of user preferences doi: 10.1109/TPAMI.2025.3531907. for text-to-image generation. In Advances in Neural Information Processing Systems, [23] Huang, Z., He, Y., Yu, J., Zhang, F., Si, volume 36, pp. 36652–36663, 2023. C., Jiang, Y., Zhang, Y., Wu, T., Jin, Q., Chanpaisit, N., Wang, Y., Chen, X., Wang, [32] Korbak, T., Elsahar, H., Kruszewski, G., L., Lin, D., Qiao, Y., and Liu, Z. VBench: and Dymetmant, M. On reinforcement learnComprehensive benchmark suite for video ing and distribution matching for fine-tuning generative models. In Proceedings of the language models with no catastrophic forIEEE/CVF Conference on Computer Vision getting. In Advances in Neural Information and Pattern Recognition (CVPR), pp. 21807– Processing Systems, volume 35, pp. 16203– 21818, 2024. 16220, 2022. [24] Hwang, J. and Sung, M. Gradient [33] Kumari, N., Zhang, B., Zhang, R., Shechtpreconditioning for efficient and reliable man, E., and Zhu, J.-Y. Multi-concept cusreward-guided generation. arXiv preprint tomization of text-to-image diffusion. In arXiv:2602.08646, 2026. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni[25] Hwang, J., Kim, J., and Sung, M. Momenttion (CVPR), pp. 1931–1941, 2023. and power-spectrum-based Gaussianity regularization for text-to-image models. In [34] Kwon, T. and Ye, J. C. Solving video inverse Advances in Neural Information Processing problems using image diffusion models. In Systems, volume 38, pp. 18235–18264, 2025. International Conference on Learning Representations, 2025. 14

NoiseTilt: Noise-Tilted Reverse Kernels

[35] Labs, B. F. FLUX. https://github.com/ [44] Lu, Y., Liang, Y., Zhu, L., and Yang, Y. black-forest-labs/flux, 2024. Freelong: Training-free long video generation with spectralblend temporal attention. [36] Lee, Y., Kim, K., Kim, H., and Sung, M. In Advances in Neural Information ProcessSyncdiffusion: Coherent montage via syning Systems, volume 37, pp. 131434–131455, chronized joint diffusions. In Advances in 2024. Neural Information Processing Systems, volume 36, pp. 50648–50660, 2023. [45] Luo, G., Granskog, J., Holynski, A., and Darrell, T. Dual-process image genera[37] Li, J., Cui, Y., Huang, T., Ma, Y., Fan, tion. In Proceedings of the IEEE/CVF InC., Yang, M., Zhong, Z., and Bo, L. ternational Conference on Computer Vision Mixgrpo: Unlocking flow-based grpo effi(ICCV), pp. 17972–17983, 2025. ciency with mixed ode-sde. arXiv preprint [46] Ma, N., Tong, S., Jia, H., Hu, H., Su, Y.-C., arXiv:2507.21802, 2025. Zhang, M., Yang, X., Li, Y., Jaakkola, T., [38] Li, X., Zhao, Y., Wang, C., Scalia, G., Jia, X., and Xie, S. Inference-time scaling Eraslan, G., Nair, S., Biancalani, T., Regev, for diffusion models beyond scaling denoisA., Levine, S., and Uehara, M. Derivativeing steps. arXiv preprint arXiv:2501.09732, free guidance in continuous and discrete 2025. diffusion models with soft value-based decoding. In Advances in Neural Information [47] Ma, X., Wang, Y., Chen, X., Wong, T.-T., Processing Systems, volume 38, pp. 95507– and Chen, C. Training-free stylized text-to95545, 2025. image generation with fast inference. arXiv preprint arXiv:2505.19063, 2025. [39] Lin, Z., Pathak, D., Li, B., Li, J., Xia, X., Neubig, G., Zhang, P., and Ramanan, D. [48] Prabhudesai, M., Mendonca, R., Qin, Z., Evaluating text-to-visual generation with Fragkiadaki, K., and Pathak, D. Video difimage-to-text generation. In Proceedings fusion alignment via reward gradients. In of the European Conference on Computer International Conference on Learning RepVision (ECCV), pp. 366–384, 2024. resentations, 2025. [40] Lipman, Y., Chen, R. T. Q., Ben-Hamu, H., [49] Qian, Y., Guo, Z., Deng, B., Lei, C. T., Nickel, M., and Le, M. Flow matching for Zhao, S., Lau, C. P., Hong, X., and Pound, generative modeling. In International ConM. P. T2icount: Enhancing cross-modal ference on Learning Representations, 2023. understanding for zero-shot counting. In Proceedings of the IEEE/CVF Conference [41] Liu, J., Liu, G., Liang, J., Li, Y., Liu, J., on Computer Vision and Pattern RecogniWang, X., Wan, P., Zhang, D., and Ouyang, tion (CVPR), pp. 25336–25345, 2025. W. Flow-GRPO: Training flow matching models via online RL. In Advances in Neural [50] Qiu, H., Chen, Z., Wang, Z., He, Y., Xia, M., Information Processing Systems, volume 38, and Liu, Z. Freetraj: Tuning-free trajectory pp. 40783–40818, 2025. control in video diffusion models. arXiv preprint arXiv:2406.16863, 2024. [42] Liu, J., Liu, G., Liang, J., Yuan, Z., Liu, X., Zheng, M., Wu, X., Wang, Q., Qin, W., [51] Rafailov, R., Sharma, A., Mitchell, E., ManXia, M., et al. Improving video generation ning, C. D., Ermon, S., and Finn, C. Diwith human feedback. In Advances in Neural rect preference optimization: Your language Information Processing Systems, volume 38, model is secretly a reward model. In Adpp. 82155–82192, 2025. vances in Neural Information Processing Systems, volume 36, pp. 53728–53741, 2023. [43] Liu, X., Gong, C., and Liu, Q. Flow straight and fast: Learning to generate and trans- [52] Ramesh, V. and Mardani, M. Test-time scalfer data with rectified flow. In Internaing of diffusion models via noise trajectory tional Conference on Learning Representasearch. arXiv preprint arXiv:2506.03164, tions, 2023. 2025. 15

NoiseTilt: Noise-Tilted Reverse Kernels

[53] Robbins, H. E. An Empirical Bayes Approach to Statistics. Springer, 1992.

scaling and steering of diffusion models. arXiv preprint arXiv:2501.06848, 2025.

[54] Rombach, R., Blattmann, A., Lorenz, D., [62] Song, J., Vahdat, A., Mardani, M., and Esser, P., and Ommer, B. High-resolution Kautz, J. Pseudoinverse-guided diffusion image synthesis with latent diffusion models. models for inverse problems. In InternaIn Proceedings of the IEEE/CVF Conference tional Conference on Learning Representaon Computer Vision and Pattern Recognitions, 2023. tion (CVPR), pp. 10674–10685, 2022. [63] Song, J., Zhang, Q., Yin, H., Mardani, M., [55] Rout, L., Raoof, N., Daras, G., Caramanis, Liu, M.-Y., Kautz, J., Chen, Y., and VahC., Dimakis, A., and Shakkottai, S. Solving dat, A. Loss-guided diffusion models for linear inverse problems provably via posteplug-and-play controllable generation. In rior sampling with latent diffusion models. International Conference on Machine LearnIn Advances in Neural Information Processing, pp. 32483–32498, 2023. ing Systems, volume 36, pp. 49960–49990, [64] Song, Y., Sohl-Dickstein, J., Kingma, D. P., 2023. Kumar, A., Ermon, S., and Poole, B. Score[56] Rout, L., Chen, Y., Kumar, A., Caramanis, based generative modeling through stochasC., Shakkottai, S., and Chu, W.-S. Beyond tic differential equations. In International first-order tweedie: Solving inverse problems Conference on Learning Representations, using latent diffusion. In Proceedings of the 2021. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 9472– [65] Stiennon, N., Ouyang, L., Wu, J., Ziegler, D. M., Lowe, R., Voss, C., Radford, A., 9481, 2024. Amodei, D., and Christiano, P. Learning [57] Rozet, F., Andry, G., Lanusse, F., and to summarize from human feedback. In Louppe, G. Learning diffusion priors from Advances in Neural Information Processing observations by expectation maximization. Systems, volume 33, pp. 3008–3021, 2020. In Advances in Neural Information Processing Systems, volume 37, pp. 87647–87682, [66] Tang, Z., Peng, J., Tang, J., Hong, M., Wang, F., and Chang, T.-H. Inference-time 2024. alignment of diffusion models with direct noise optimization. In International Confer[58] Samuel, D., Ben-Ari, R., Darshan, N., Maron, H., and Chechik, G. Norm-guided ence on Machine Learning, pp. 58905–58930, latent space exploration for text-to-image 2025. generation. In Advances in Neural Information Processing Systems, volume 36, pp. [67] Uehara, M., Zhao, Y., Hajiramezanali, E., Scalia, G., Eraslan, G., Lal, A., Levine, S., 57863–57875, 2023. and Biancalani, T. Bridging model-based [59] Schuhmann, C. LAION aesthetics. https: optimization and generative modeling via //laion.ai/blog/laion-aesthetics, conservative fine-tuning of diffusion models. 2022. In Advances in Neural Information Processing Systems, volume 37, pp. 127511–127535, [60] Seo, J., Veer, S., Tian, R., Ding, W., 2024. Sharma, A., Leung, K., Schmerling, E., Pavone, M., and Bajcsy, A. Stressdream: [68] Uehara, M., Zhao, Y., Black, K., HajiSteering video world models for robust polramezanali, E., Scalia, G., Diamant, N. L., icy evaluation and improvement. arXiv Tseng, A. M., Biancalani, T., and Levine, preprint arXiv:2606.00267, 2026. S. Fine-tuning of continuous-time diffusion models as entropy-regularized control. In [61] Singhal, R., Horvitz, Z., Teehan, R., Ren, International Conference on Learning RepM., Yu, Z., McKeown, K., and Ranganath, resentations, 2025. R. A general framework for inference-time

16

NoiseTilt: Noise-Tilted Reverse Kernels

[69] Wallace, B., Dang, M., Rafailov, R., Zhou, [77] Zhang, Y., Wei, Y., Jiang, D., Zhang, L., Lou, A., Purushwalkam, S., Ermon, S., X., Zuo, W., and Tian, Q. Controlvideo: Xiong, C., Joty, S., and Naik, N. Diffusion Training-free controllable text-to-video genmodel alignment using direct preference operation. In International Conference on timization. In Proceedings of the IEEE/CVF Learning Representations, 2024. Conference on Computer Vision and Pattern Recognition (CVPR), pp. 8228–8238, 2024. [70] Wan Team, A. G. Wan: Open and advanced large-scale video generative models. arXiv preprint arXiv:2503.20314, 2025. [71] Wu, L., Trippe, B., Naesseth, C., Blei, D., and Cunningham, J. P. Practical and asymptotically exact conditional sampling in diffusion models. In Advances in Neural Information Processing Systems, volume 36, pp. 31372–31403, 2023. [72] Wu, X., Hao, Y., Sun, K., Chen, Y., Zhu, F., Zhao, R., and Li, H. Human preference score v2: A solid benchmark for evaluating human preferences of text-to-image synthesis. In International Conference on Learning Representations, 2024. [73] Xu, J., Liu, X., Wu, Y., Tong, Y., Li, Q., Ding, M., Tang, J., and Dong, Y. ImageReward: Learning and evaluating human preferences for text-to-image generation. In Advances in Neural Information Processing Systems, volume 36, pp. 15903–15935, 2023. [74] Ye, H., Lin, H., Han, J., Xu, M., Liu, S., Liang, Y., Ma, J., Zou, J. Y., and Ermon, S. TFG: Unified training-free guidance for diffusion models. In Advances in Neural Information Processing Systems, volume 37, pp. 22370–22417, 2024. [75] Yoon, T., Min, Y., Yeo, K., and Sung, M. Psi-sampler: Initial particle sampling for smc-based inference-time reward alignment in score models. In Advances in Neural Information Processing Systems, volume 38, pp. 104745–104781, 2025. [76] Yu, J., Wang, Y., Zhao, C., Ghanem, B., and Zhang, J. FreeDoM: Training-free energyguided conditional diffusion model. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 23174–23184, 2023.

17

Appendix

A Reward-Guided Reverse Kernels In this section, we provide derivations and interpretations for reward-guided reverse kernels. Figure 3 and Table 1 summarize the four kernel types considered in this section: the base reverse kernel, mean-shifted reverse kernels, search-based guidance over the base kernel, and our noise-tilted reverse kernel (NTRK). Among them, mean-shifted guidance injects reward information through the deterministic term, whereas search-based guidance and NTRK act through stochastic perturbations. Before proceeding, we elaborate on the notion of noise-compatible used in Section 4. By this, we mean the regime in which the reverse update remains centered at the base transition and is perturbed by typical noise-like perturbations, so that intermediate states stay on the manifold of the learned intermediate-state distribution, rather than drifting into regions insufficiently supported by the pretrained diffusion. This shorthand captures the key intuition; it does not mean that the induced transition exactly matches the pretrained reverse-kernel distribution. With this clarification, the rest of the section is organized as follows. In Section A.1, we recall the optimal reward-tilted reverse kernel that serves as the common theoretical target and present continuous-time formulation of the base reverse kernel. In Section A.2, we derive mean-shifted reverse kernels as a standard approximation obtained by modifying the deterministic term. In Section A.3, we revisit search-based guidance, which instead draws multiple candidates from the base reverse kernel and selects favorable perturbations according to reward. In Section A.4, we interpret NTRK from both perspectives, showing how it combines the single-draw guidance advantage of mean-shifted methods with the noise-compatible-regime advantage of search-based guidance. Finally, in Section A.5, we present an intuitive interpretation of the guidance strength parameter.

A.1 Optimal Reward-Tilted and Base Reverse Kernel In this subsection, we first formulate the optimal reward-tilted reverse kernel, which serves as the target for maximizing expected rewards under a KL divergence penalty. Subsequently, we detail the continuous-time formulation of the base reverse kernel. A.1.1 Optimal Reward-Tilted Reverse Kernel Starting from an entropy-regularized objective over reverse transition policies, we derive the optimal reward-tilted reverse kernel, the associated value function, and the target marginal distribution. For a more rigorous theoretical treatment and comprehensive proofs, we refer readers to previous work [67]. We consider the sequence of reverse transition policies {q(xt−1 |xt )}Tt=1 and optimize \label {eq:reward_rl_obj_app} \max _{q}\; \mathbb {E}_{\bm {x}_{0:T}\sim q(\bm {x}_{0:T})} \left [ r(\bm {x}_{0}) - \beta \sum _{t=1}^{T} \mathcal {D}_{KL}\!\left ( q(\bm {x}_{t-1}|\bm {x}_{t}) \| p(\bm {x}_{t-1}|\bm {x}_{t}) \right ) \right ],

(18)

which maximizes the terminal reward while regularizing step-wise deviations from the pretrained base reverse kernel p(xt−1 |xt ). The temperature β > 0 controls the reward-KL trade-off.

18

NoiseTilt: Noise-Tilted Reverse Kernels

We define the expected return functional from state xt as \label {eq:return_functional_app} J_t(\bm {x}_t;q) = \mathbb {E}_{q(\bm {x}_{0:t-1}|\bm {x}_t)} \left [ r(\bm {x}_0) - \beta \sum _{\tau =1}^{t} \mathcal {D}_{KL}\!\left ( q(\bm {x}_{\tau -1}|\bm {x}_{\tau }) \| p(\bm {x}_{\tau -1}|\bm {x}_{\tau }) \right ) \right ].

(19)

The value function is the optimal value of this return functional: \label {eq:bellman_value_app} V_t(\bm {x}_t) &= \max _{q}\; J_t(\bm {x}_t;q) = J_t(\bm {x}_t;p^\star ) \nonumber \\ &= \max _{q} \left \{ \mathbb {E}_{\bm {x}_{t-1}\sim q} \bigl [ V_{t-1}(\bm {x}_{t-1}) \bigr ] - \beta \mathcal {D}_{KL}\!\left ( q(\bm {x}_{t-1}|\bm {x}_t) \| p(\bm {x}_{t-1}|\bm {x}_t) \right ) \right \}, (20) with terminal boundary condition V0 (x0 ) = r(x0 ). Optimal reward-tilted reverse kernel. Expanding the expectation and KL divergence in Equation (20) gives \label {eq:bellman_integral_app} V_t(\bm {x}_t) = \max _{q} \int q(\bm {x}_{t-1}|\bm {x}_t) \left [ V_{t-1}(\bm {x}_{t-1}) - \beta \log \frac { q(\bm {x}_{t-1}|\bm {x}_t) }{ p(\bm {x}_{t-1}|\bm {x}_t) } \right ] d\bm {x}_{t-1}.

(21)

Writing Vt−1 (xt−1 ) = β log exp(Vt−1 (xt−1 )/β) , we obtain \label {eq:bellman_single_log_app} V_t(\bm {x}_t) = \max _{q} \int q(\bm {x}_{t-1}|\bm {x}_t) \beta \log \left ( \frac { p(\bm {x}_{t-1}|\bm {x}_t) \exp (V_{t-1}(\bm {x}_{t-1})/\beta ) }{ q(\bm {x}_{t-1}|\bm {x}_t) } \right ) d\bm {x}_{t-1}.

(22)

Introducing the state-dependent normalizing constant \label {eq:partition_function_app} Z_t(\bm {x}_t) = \int p(\bm {x}_{t-1}|\bm {x}_t) \exp (V_{t-1}(\bm {x}_{t-1})/\beta ) d\bm {x}_{t-1},

(23)

and multiplying and dividing the logarithm argument by Zt (xt ) yields \label {eq:bellman_kl_form_app} V_t(\bm {x}_t) &= \max _{q} \int q(\bm {x}_{t-1}|\bm {x}_t) \beta \log \left ( \frac { Z_t(\bm {x}_t) }{ q(\bm {x}_{t-1}|\bm {x}_t) } \cdot \frac { p(\bm {x}_{t-1}|\bm {x}_t)\exp (V_{t-1}(\bm {x}_{t-1})/\beta ) }{ Z_t(\bm {x}_t) } \right ) d\bm {x}_{t-1} \nonumber \\ &= \beta \log Z_t(\bm {x}_t) - \beta \min _{q} \mathcal {D}_{KL} \left ( q(\bm {x}_{t-1}|\bm {x}_t) \bigg \| \frac { p(\bm {x}_{t-1}|\bm {x}_t)\exp (V_{t-1}(\bm {x}_{t-1})/\beta ) }{ Z_t(\bm {x}_t) } \right ). (24) Since the KL divergence is non-negative and vanishes if and only if the two distributions coincide, the optimum is uniquely attained at \label {eq:optimal_prop_app} p^\star (\bm {x}_{t-1}|\bm {x}_t) = \frac { p(\bm {x}_{t-1}|\bm {x}_t)\exp (V_{t-1}(\bm {x}_{t-1})/\beta ) }{ \int p(\bm {x}_{t-1}|\bm {x}_t)\exp (V_{t-1}(\bm {x}_{t-1})/\beta ) d\bm {x}_{t-1} }.

(25)

This is the optimal reward-tilted reverse kernel. Evaluating the optimum further yields the relation \label {eq:value_partition_app} \exp (V_t(\bm {x}_t)/\beta ) &= \int p(\bm {x}_{t-1}|\bm {x}_t)\exp (V_{t-1}(\bm {x}_{t-1})/\beta ) d\bm {x}_{t-1} \nonumber \\ &= \mathbb {E}_{p} \left [ \exp (V_{t-1}(\bm {x}_{t-1})/\beta ) \middle | \bm {x}_t \right ] = \hdots = \mathbb {E}_{p} \left [ \exp (r(\bm {x}_0)/\beta ) \middle | \bm {x}_t \right ]. (26) Value approximation. In practice, directly evaluating Equation (26) is intractable. Following previous work [38], we approximate the posterior expectation using the Tweedie estimate x̂0|t := E[x0 |xt ]: \label {eq:value_approx_app} V_t(\bm {x}_t) &= \beta \log \mathbb {E}_{\bm {x}_0\sim p(\cdot |\bm {x}_t)} \left [ \exp (r(\bm {x}_0)/\beta ) \right ] \nonumber \\ &\approx \beta \log \left ( \exp (r(\hat {\bm {x}}_{0|t})/\beta ) \right ) = r(\hat {\bm {x}}_{0|t}). (27)

19

NoiseTilt: Noise-Tilted Reverse Kernels

Target marginal distribution. Sampling from the optimal step-wise proposal induces the following target marginal at timestep t: \label {eq:target_marginal_app} p^\star (\bm {x}_t) &= \int p^\star (\bm {x}_{t:T}) d\bm {x}_{t+1:T} \nonumber \\ &= \frac { \exp (V_t(\bm {x}_t)/\beta ) }{ Z } \int p(\bm {x}_{t:T}) d\bm {x}_{t+1:T} \nonumber \\ &= \frac {1}{Z} p(\bm {x}_t)\exp (V_t(\bm {x}_t)/\beta ).

(28) Likewise, \label {eq:target_marginal_prev_app} p^\star (\bm {x}_{t-1}) &= \int p^\star (\bm {x}_{t-1}|\bm {x}_t) p^\star (\bm {x}_t) d\bm {x}_t \nonumber \\ &= \int \left ( p(\bm {x}_{t-1}|\bm {x}_t) \frac { \exp (V_{t-1}(\bm {x}_{t-1})/\beta ) }{ \exp (V_t(\bm {x}_t)/\beta ) } \right ) \left ( \frac {1}{Z}p(\bm {x}_t)\exp (V_t(\bm {x}_t)/\beta ) \right ) d\bm {x}_t \nonumber \\ &= \frac { \exp (V_{t-1}(\bm {x}_{t-1})/\beta ) }{ Z } \int p(\bm {x}_{t-1}|\bm {x}_t)p(\bm {x}_t) d\bm {x}_t \nonumber \\ &= \frac {1}{Z} p(\bm {x}_{t-1})\exp (V_{t-1}(\bm {x}_{t-1})/\beta ).

(29) Recursively applying this relation down to t = 0 recovers \label {eq:terminal_target_marginal_app} p^\star (\bm {x}_0) = \frac {1}{Z} p(\bm {x}_0)\exp (r(\bm {x}_0)/\beta ).

(30)

A.1.2 Base Reverse Kernel We connect our discrete-time formulation to the continuous-time sampling processes used in score-based generative models. Let {pt }0≤t≤T be a probability path interpolating between a tractable noise prior and the data distribution: \label {eq:stochastic_interpolant_app} \bm {x}_t = \alpha _t \bm {x}_0 + \sigma _t \bm {x}_T, (31) where αt and σt are smooth monotone schedules satisfying α0 = σT = 1 and αT = σ0 = 0. Flow-based models [40, 43] parameterize this path using a time-dependent velocity field ut : RN → RN , leading to the probability flow ODE: \label {eq:pf_ode_app} \mathrm {d}\bm {x}_t = \bm {u}_t(\bm {x}_t) \mathrm {d}t, \qquad \bm {x}_T\sim \mathcal {N}(\bm {0},\bm {I}).

(32)

Adding stochastic exploration yields the reverse-time SDE: \label {eq:reverse_sde_app} \mathrm {d}\bm {x}_t = \bm {f}_t(\bm {x}_t) \mathrm {d}t + \sigma _t \mathrm {d}\bm {w}, \qquad \bm {f}_t(\bm {x}_t) = \bm {u}_t(\bm {x}_t) - \frac {\sigma _t^2}{2}\nabla \log p_t(\bm {x}_t),

(33)

where w denotes the standard Wiener process. Discretizing Equation (33) backward in time yields the base reverse kernel: \label {eq:base_reverse_kernel_app} \bm {x}_{t-1} = \bm {\mu }_\theta (\bm {x}_t,t) + \sigma _t \bm {\epsilon }_t, \qquad \bm {\epsilon }_t\sim \mathcal {N}(\bm {0},\bm {I}).

(34)

A.2 Approach 1: Mean-Shifted Reverse Kernel A standard approach to approximating the optimal reward-tilted reverse kernel is to inject the reward information directly into the deterministic component of the reverse update. We first establish the exact relationship between the marginal velocity and the marginal score, then derive the mean-shifted reverse kernel by substituting the tilted score into the reverse SDE.

20

NoiseTilt: Noise-Tilted Reverse Kernels

Relation between velocity and score. To systematically modify this base kernel with a reward signal, we must adjust the reverse drift ft (xt ) in Equation (33). Since this drift depends on both the marginal velocity ut (xt ) and the marginal score ∇xt log pt (xt ), we first establish their exact relationship. By definition, the marginal velocity is given as: \label {eq:marginal_velocity_def} \bm {u}_t(\bm {x}_t) = \mathbb {E}[\dot {\alpha }_t \bm {x}_0 + \dot {\sigma }_t \bm {x}_T \mid \bm {x}_t] = \dot {\alpha }_t \mathbb {E}[\bm {x}_0 \mid \bm {x}_t] + \dot {\sigma }_t \mathbb {E}[\bm {x}_T \mid \bm {x}_t].

(35)

Using the standard score identity for the Gaussian transition kernel pt (xt | x0 ) = N (xt ; αt x0 , σt2 I), we can express the marginal score as: \label {eq:marginal_score_identity} \nabla _{\bm {x}_t} \log p_t(\bm {x}_t) = \mathbb {E} \left [ -\frac {\bm {x}_t - \alpha _t \bm {x}_0}{\sigma _t^2} \;\middle |\; \bm {x}_t \right ] = -\frac {\bm {x}_t - \alpha _t \mathbb {E}[\bm {x}_0 \mid \bm {x}_t]}{\sigma _t^2}.

(36)

Rearranging this expression yields the optimal denoised estimate for x0 : \label {eq:x0_posterior} \mathbb {E}[\bm {x}_0 \mid \bm {x}_t] = \frac {1}{\alpha _t}\bm {x}_t + \frac {\sigma _t^2}{\alpha _t} \nabla _{\bm {x}_t} \log p_t(\bm {x}_t).

(37)

Similarly, since the forward process is defined as xt = αt x0 + σt xT , the expected noise can be written directly in terms of the score: \label {eq:xT_posterior} \mathbb {E}[\bm {x}_T \mid \bm {x}_t] = \frac {\bm {x}_t - \alpha _t \mathbb {E}[\bm {x}_0 \mid \bm {x}_t]}{\sigma _t} = -\sigma _t \nabla _{\bm {x}_t} \log p_t(\bm {x}_t).

(38)

Substituting these posterior expectations back into the marginal velocity definition (Equation (35)), we arrive at the exact relationship: \bm {u}_t(\bm {x}_t) &= \dot {\alpha }_t \left ( \frac {1}{\alpha _t}\bm {x}_t + \frac {\sigma _t^2}{\alpha _t} \nabla _{\bm {x}_t} \log p_t(\bm {x}_t) \right ) - \dot {\sigma }_t \sigma _t \nabla _{\bm {x}_t} \log p_t(\bm {x}_t) \nonumber \\ \label {eq:marginal_velocity_score_app} &= \frac {\dot {\alpha }_t}{\alpha _t} \bm {x}_t - \underbrace {\left ( \dot {\sigma }_t \sigma _t - \sigma _t^2 \frac {\dot {\alpha }_t}{\alpha _t} \right )}_{=:C_t} \nabla _{\bm {x}_t} \log p_t(\bm {x}_t). (39)

Mean-Shifted Reverse Kernel. With this exact relationship established, we can now inject the reward signal into the reverse generative dynamics. Starting from the optimal target marginal in Equation (28), we decompose its score as follows: \label {eq:tilted_score_app} \nabla _{\bm {x}_t}\log p^\star (\bm {x}_t) &= \nabla _{\bm {x}_t} \log \left ( \frac {1}{Z} p_t(\bm {x}_t)\exp (V_t(\bm {x}_t)/\beta ) \right ) \nonumber \\ &= \nabla _{\bm {x}_t}\log p_t(\bm {x}_t) + \frac {1}{\beta }\nabla _{\bm {x}_t}V_t(\bm {x}_t). (40) This demonstrates that the optimal tilted score is the pretrained score shifted by the temperaturescaled value gradient. Next, we substitute this tilted score alongside the corresponding tilted velocity, u⋆t (xt ) = ut (xt ) − Cβt ∇xt Vt (xt ), into the reverse SDE (Equation (33)) to obtain the modified reverse drift:

\label {eq:tilted_drift_app} \bm {f}_t^\star (\bm {x}_t) &= \bm {u}^\star _t(\bm {x}_t) - \frac {\sigma _t^2}{2} \nabla _{\bm {x}_t}\log p^\star (\bm {x}_t) \nonumber \\ &= \left ( \bm {u}_t(\bm {x}_t) - \frac {C_t}{\beta } \nabla _{\bm {x}_t}V_t(\bm {x}_t) \right ) - \frac {\sigma _t^2}{2} \left ( \nabla _{\bm {x}_t}\log p_t(\bm {x}_t) + \frac {1}{\beta }\nabla _{\bm {x}_t}V_t(\bm {x}_t) \right ) \nonumber \\ &= \bm {f}_t(\bm {x}_t) - \frac {C_t + \frac {\sigma _t^2}{2}}{\beta } \nabla _{\bm {x}_t}V_t(\bm {x}_t).

(41)

21

NoiseTilt: Noise-Tilted Reverse Kernels

Because the exact value function gradient is analytically intractable, we employ the approximation from Equation (27): \label {eq:value_grad_approx_app} \nabla _{\bm {x}_t}V_t(\bm {x}_t) \approx \nabla _{\bm {x}_t}r(\hat {\bm {x}}_{0|t}). (42) We absorb the schedule-dependent coefficient Ct + σt2 /2 /β into a single guidance hyperparameter λt . Discretizing the modified reverse SDE then gives the mean-shifted reverse kernel:

\label {eq:mean_shift_kernel_app} \bm {x}_{t-1} = \bm {\mu }_\theta (\bm {x}_t, t) + \lambda _t\nabla _{\bm {x}_t} r(\hat {\bm {x}}_{0|t}) + \sigma _t \bm {\epsilon }_t, \qquad \bm {\epsilon }_t \sim \mathcal {N}(\bm {0}, \bm {I}).

(43)

While the derivation prescribes λt = Ct + σt2 /2 /β, in practice λt is treated as an independent hyperparameter. This corresponds to the mean-shifted reverse kernel introduced in Section 3.3.

A.3 Approach 2: Search-Based Reverse Kernel In contrast to modifying the deterministic drift, an alternative approximation strategy draws multiple candidate perturbations from the base reverse kernel and biases the final selection toward those yielding higher rewards. Starting from the optimal reward-tilted reverse kernel in Equation (25), we can approximate it empirically using K candidate samples drawn from the base reverse kernel:

\label {eq:importance_sampling_app} p_\theta ^\star (\bm {x}_{t-1}|\bm {x}_t) &\approx \sum _{i=1}^{K} \frac { w_{t-1}^{(i)} }{ \sum _{j=1}^{K} w_{t-1}^{(j)} } \delta _{\bm {x}_{t-1}^{(i)}}(\bm {x}_{t-1}), \\ \bm {x}_{t-1}^{(i)} &= \bm {\mu }_\theta (\bm {x}_t,t) + \sigma _t \bm {\epsilon }_t^{(i)}, \qquad \bm {\epsilon }_t^{(i)}\sim \mathcal {N}(\bm {0},\bm {I}), \nonumber \\ w_{t-1}^{(i)} &= \exp \!\left ( V_{t-1}(\bm {x}_{t-1}^{(i)})/\beta \right ). \nonumber

Here, each candidate is generated from the same base reverse kernel, and the reward tilt appears only through the selection weights. Using the value approximation in Equation (27), the weights can be approximated as \label {eq:search_weight_approx_app} w_{t-1}^{(i)} \approx \exp \!\left ( r(\hat {\bm {x}}_{0|t-1}^{(i)})/\beta \right ),

(45)

(i)

where x̂0|t−1 denotes the Tweedie estimate associated with the i-th candidate. This gives a searchbased approximation to the optimal reward-tilted reverse kernel by favoring candidates with larger estimated terminal reward. In practice, a common hard-selection variant replaces stochastic resampling by selecting the maximum-weight candidate. In particular, SVDD [38] can be interpreted as the argmax form of this search procedure: \label {eq:svdd_argmax_app} i^\star &= \underset {i\in \{1,\dots ,K\}}{\arg \max }\, r(\hat {\bm {x}}_{0|t-1}^{(i)}), \\ \bm {x}_{t-1} &= \bm {x}_{t-1}^{(i^\star )}. \nonumber

Conditioned on xt , search-based guidance samples multiple perturbations around the same base center µθ (xt , t) and selects one according to reward. This can be interpreted as inducing a biased distribution over the perturbation variable itself, while preserving the base reverse-kernel sampling form during candidate generation. Under a local linear reward model around the base reverse update, the induced perturbation bias is aligned with the reward-gradient direction, so search-based guidance may be viewed as an implicit form of noise tilting, as illustrated in Figure 3. This observation provides a natural bridge to our NTRK, which realizes reward guidance explicitly through a single tilted perturbation.

22

NoiseTilt: Noise-Tilted Reverse Kernels

A.4 Best of Both Worlds: NTRK We now interpret NTRK by connecting the two approximation routes discussed above. From the perspective of mean-shifted reverse kernels, our goal is to retain the guidance advantage of first-order reward information without modifying the deterministic term of the reverse update. From the perspective of search-based guidance, our goal is to retain the noise-compatible-regime advantage of sampling around the base update center, while avoiding the multi-draw search cost. These two viewpoints meet at the same principle: reward information should be injected through the perturbation variable rather than through a shift of the deterministic term. Accordingly, NTRK keeps the base reverse update center µθ (xt , t) and replaces the standard Gaussian perturbation by a reward-tilted perturbation: \label {eq:ntrk_kernel_app} \bm {x}_{t-1} = \bm {\mu }_\theta (\bm {x}_t,t) + \sigma _t \tilde {\bm {\epsilon }}_t,

(47)

\label {eq:ntrk_noise_app} \tilde {\bm {\epsilon }}_t = \sqrt {\rho _t}\,\mathcal {W}\!\left (\nabla _{\bm {x}_t} r(\hat {\bm {x}}_{0|t})\right ) + \sqrt {1-\rho _t}\,\bm {\epsilon }_t, \qquad \bm {\epsilon }_t\sim \mathcal {N}(\bm {0},\bm {I}).

(48)

where Here, ρt ∈ [0, 1] controls the strength of the reward-informed perturbation, and W(·) denotes the whitening operator introduced in Section 4 and detailed in Section B. The role of W(·) is to map the reward gradient to a typical noise-like vector, so that the resulting update remains in the noise-compatible regime. From the mean-shift perspective. Comparing Equations (47) and (48) with the mean-shifted reverse kernel in Equation (43), both methods exploit the same local first-order reward signal ∇xt r(x̂0|t ). However, the way this signal enters the reverse update is fundamentally different. Meanshifted guidance adds the reward gradient directly to the deterministic term, thereby changing the center of the reverse update. In contrast, NTRK preserves the deterministic term µθ (xt , t) and injects reward information only through the perturbation. Thus, NTRK may be viewed as a mean-preserving reformulation of first-order reward guidance: it retains the single-draw efficiency of mean-shifted guidance while avoiding the explicit center shift that moves the update away from the noise-compatible regime. From the search-based perspective. Comparing Equations (47) and (48) with the search-based update in Equations (44) and (46), both methods generate reward-guided updates around the same base center µθ (xt , t). Search-based guidance does so implicitly by drawing multiple perturbations from the base reverse kernel and selecting the most favorable one according to reward. As discussed in Section A.3, under a local linear assumption, this selection induces a bias over the perturbation variable aligned with the reward-gradient direction. From this viewpoint, NTRK can be interpreted as an explicit single-draw realization of that idea: instead of performing a K-candidate search and selecting a favorable perturbation afterward, we directly construct a reward-tilted perturbation through Equation (48). Therefore, NTRK combines the key advantages of both existing viewpoints. Unlike mean-shifted reverse kernels, it stays within the noise-compatible regime by preserving the base update center. Unlike search-based guidance, it achieves reward alignment with a single draw per step. In this sense, NTRK bridges the two approaches by realizing reward guidance explicitly through a single typical noise-like perturbation.

A.5 Interpretation of the Guidance Strength ρt in NTRK We now give an intuitive interpretation of the guidance strength ρt by comparing NTRK with searchbased guidance. Our goal is not to claim an exact equivalence, but to clarify how increasing ρt changes the effective strength of reward guidance relative to search-based methods. The discussion below is heuristic and relies on a local linearity assumption around the base reverse update. Although this assumption need not hold exactly, it reflects the common working belief behind 23

NoiseTilt: Noise-Tilted Reverse Kernels

gradient-based guidance methods: the reward gradient often provides a meaningful direction not only infinitesimally, but also over a practically relevant local neighborhood. Fix xt and consider perturbations around the base reverse update center µθ (xt , t). For a perturbation ϵ, define \label {eq:reverse_update_eps_app} \bm {x}_{t-1}(\bm {\epsilon }) := \bm {\mu }_\theta (\bm {x}_t,t) + \sigma _t \bm {\epsilon }, (49) and let x̂0|t−1 (ϵ) denote the Tweedie estimate obtained from xt−1 (ϵ). To connect this interpretation with the actual direction used in practice, we approximate the local reward-improving perturbation direction around ϵ = 0 by the normalized reward gradient at the current state: \label {eq:ut_def_app} \bm {u}_t := \frac {\nabla _{\bm {x}_t} r(\hat {\bm {x}}_{0|t})} {\|\nabla _{\bm {x}_t} r(\hat {\bm {x}}_{0|t})\|_2},

(50)

whenever ∇xt r(x̂0|t ) ̸= 0. Under this approximation, the reward around the base reverse update is modeled by the first-order form \label {eq:local_linearity_reward_app} r\!\left (\hat {\bm {x}}_{0|t-1}(\bm {\epsilon })\right ) \approx c_t + \kappa _t \langle \bm {u}_t, \bm {\epsilon } \rangle ,

(51)

\label {eq:ct_def_app} c_t := r\!\left (\hat {\bm {x}}_{0|t-1}(\bm {0})\right ),

(52)

where and κt > 0 is a local sensitivity coefficient. Thus, under this local approximation, candidate ranking depends only on the one-dimensional projected perturbation coordinate \label {eq:projected_coordinate_app} z := \langle \bm {u}_t, \bm {\epsilon } \rangle .

(53)

Search-based perspective. As discussed in Section A.3, search-based guidance draws K candidates from the base reverse kernel and performs importance sampling according to their estimated rewards. Under Equation (51), the importance weight of the i-th candidate satisfies \label {eq:search_linear_weight_app} w_i \propto \exp \!\left ( r\!\left (\hat {\bm {x}}_{0|t-1}(\bm {\epsilon }_t^{(i)})\right ) \right ) \approx \exp (c_t)\exp (\kappa _t z_i),

(54)

\label {eq:search_projected_coordinate_app} z_i := \langle \bm {u}_t, \bm {\epsilon }_t^{(i)} \rangle , \qquad \bm {\epsilon }_t^{(i)} \sim \mathcal {N}(\bm {0}, \bm {I}).

(55)

where Since the base perturbation is isotropic and ∥ut ∥2 = 1, each zi follows N (0, 1). Thus, search-based guidance increasingly favors candidates with larger projected coordinate zi . In the hard-selection form used by SVDD [38], this reduces to selecting \label {eq:search_projected_max_app} z_K^\star = \max _{1 \le i \le K} z_i.

(56)

\label {eq:mK_def_app} m_K := \mathbb {E}[z_K^\star ] = \mathbb {E}\!\left [\max _{1 \le i \le K} z_i\right ].

(57)

Define

Then larger K yields a larger average displacement along the reward-improving direction. For large K, the standard extreme-value approximation gives \label {eq:mK_approx_app} m_K \approx \sqrt {2\log K}.

(58)

NTRK perspective. For NTRK, the perturbation is \label {eq:ntrk_noise_rho_app} \tilde {\bm {\epsilon }}_t = \sqrt {\rho _t}\,\bm {w}_t + \sqrt {1-\rho _t}\,\bm {\epsilon }_t, \qquad \bm {\epsilon }_t \sim \mathcal {N}(\bm {0},\bm {I}),

(59)

\label {eq:ntrk_whitened_dir_app} \bm {w}_t := \mathcal {W}\!\left (\nabla _{\bm {x}_t} r(\hat {\bm {x}}_{0|t})\right ).

(60)

where

24

NoiseTilt: Noise-Tilted Reverse Kernels

Projecting onto the same reward-aligned unit direction ut gives \label {eq:ntrk_projected_coordinate_app} \tilde {z}_t := \langle \bm {u}_t,\tilde {\bm {\epsilon }}_t\rangle &= \sqrt {\rho _t}\, \left \langle \bm {u}_t, \mathcal {W}\!\left (\nabla _{\bm {x}_t} r(\hat {\bm {x}}_{0|t})\right ) \right \rangle + \sqrt {1-\rho _t}\, \langle \bm {u}_t,\bm {\epsilon }_t\rangle \nonumber \\ &= \sqrt {\rho _t}\,a_t + \sqrt {1-\rho _t}\,z_t, (61) where \label {eq:ntrk_alignment_coeff_app} a_t := \left \langle \bm {u}_t, \mathcal {W}\!\left (\nabla _{\bm {x}_t} r(\hat {\bm {x}}_{0|t})\right ) \right \rangle , \qquad z_t := \langle \bm {u}_t,\bm {\epsilon }_t\rangle .

(62)

Since ∥ut ∥2 = 1 and ϵt ∼ N (0, I), we have zt ∼ N (0, 1), and therefore \label {eq:ntrk_projected_stats_app} \mathbb {E}[\tilde {z}_t] = \sqrt {\rho _t}\,a_t.

(63)

The key quantity is thus the alignment coefficient at . It measures how strongly the whitened reward-informed perturbation projects onto the locally reward-improving direction ut . As discussed later in Section B, the whitening operator W is designed to preserve a meaningful cosine similarity with the original reward gradient. Consequently, even in very high dimension, at can remain substantial. To see this, suppose the cosine similarity between wt := W(∇xt r(x̂0|t )) and ut is around 0.1. √ When N = 65536, we typically have ∥wt ∥2 ≈ N = 256, so \label {eq:at_example_app} a_t = \langle \bm {u}_t,\bm {w}_t\rangle \approx 0.1 \times 256 = 25.6.

(64)

Thus, even a seemingly small value of ρt can already induce a strong reward-aligned bias through √ the factor ρt at . Under this interpretation, a local heuristic correspondence between search-based guidance and NTRK is obtained by matching the average projected displacement: \label {eq:rho_k_match_app} \sqrt {\rho _t}\,a_t \approx m_K.

(65)

Using the large-K approximation in Equation (58), this gives \label {eq:rho_k_match_exp_app} K_{\mathrm {eff}} \approx \exp \!\left (\frac {\rho _t a_t^2}{2}\right ).

(66)

Although this correspondence is only heuristic, it clarifies the role of ρt : increasing ρt rapidly increases the effective search strength. For example, taking the value at = 25.6 from Equation (64) gives \rho _t = 0.01 &\quad \Longrightarrow \quad \sqrt {\rho _t}\,a_t = 2.56, \qquad K_{\mathrm {eff}} \approx \exp (3.28) \approx 2.6\times 10^{1}, \nonumber \\ \rho _t = 0.10 &\quad \Longrightarrow \quad \sqrt {\rho _t}\,a_t = 8.10, \qquad K_{\mathrm {eff}} \approx \exp (32.77) \approx 1.7\times 10^{14}. \label {eq:rho_examples_app} (67) Therefore, once the preserved directional alignment is taken into account, even moderate values of ρt can correspond to an astronomically large effective search strength that would be infeasible for natural search-based methods. Although the correspondence above is only heuristic, it suggests a clear practical implication: large ρt acts like an increasingly greedy effective search over perturbations. In multi-step reverse sampling, however, stronger greediness is not always desirable. As ρt → 1, the perturbation in Equation (59) becomes nearly deterministic, and the residual exploration term vanishes. Thus, each reverse step relies more heavily on the reward-aligned component and less on stochastic exploration. While this can improve the immediate target reward, repeatedly applying such nearly deterministic updates may reduce robustness to local approximation error and gradually weaken compatibility with the noise-compatible regime, which can in turn harm held-out quality.

25

NoiseTilt: Noise-Tilted Reverse Kernels

Table 5. Ablation on the guidance strength ρt for NTRK on aesthetic image generation. We fix the sampling configuration (NFE = 25) and vary only ρt . Dark green and light green denote the best and second-best results, respectively. ρt

0.01 0.03 0.10 0.20 0.30 0.50 1.00

Target Reward

Held-out Rewards

Aesthetic ↑ Score

Pick ↑ -score

HPSv2 ↑

Image ↑ Reward

VQA ↑ Score

6.5127 6.7445 7.1128 7.3265 7.4510 7.5892 7.7571

0.2205 0.2212 0.2205 0.2204 0.2200 0.2177 0.2142

0.2979 0.2977 0.2959 0.2976 0.2928 0.2916 0.2789

1.2773 1.3135 1.2431 1.3386 1.2565 1.1592 0.9576

0.9724 0.9687 0.9730 0.9716 0.9728 0.9674 0.9492

Figure 7. Reward trade-off trajectory as ρt increases. Moderate ρt improves the target reward while preserving held-out quality, whereas overly large ρt degrades the held-out reward.

This trade-off is consistent with the ablation results in Table 5. Moderate values of ρt improve the target aesthetic score while largely preserving held-out rewards, whereas overly large values of ρt degrade held-out quality. Thus, the most effective operating regime is neither pure random sampling (ρt ≈ 0) nor fully deterministic reward chasing (ρt ≈ 1), but a moderate intermediate regime that balances reward alignment and stochastic exploration.

26

NoiseTilt: Noise-Tilted Reverse Kernels

B Whitening Operator In Section 4, we introduced the whitening operator W as the key component that enables reward guidance through the noise term while preserving the pretrained reverse-kernel mean. Its role is to transform an arbitrary input signal, such as a reward gradient, into a perturbation that remains compatible with the pretrained reverse dynamics. More precisely, our goal is to move an arbitrary vector toward the noise-compatible regime, namely, the regime of perturbations that exhibit the statistical characteristics of typical standard Gaussian noise and are therefore naturally accepted by the pretrained model. As discussed in Section 4, although N (0, I) has nonzero density everywhere in RN , in high dimensions almost all of its probability mass concentrates on a narrow typical region. The exact typical set is difficult to characterize in closed form, so instead of attempting to describe it exactly, we approximate it by intersecting a collection of high-confidence constraint sets induced by known statistics of the standard Gaussian distribution. Accordingly, the whitening operator W is implemented as a sequence of projections onto such confidence sets. Each component enforces one statistical property that typical white Gaussian noise satisfies with overwhelming probability. The resulting operator is modular, computationally efficient, and practically effective: it leaves already-typical Gaussian noise almost unchanged, while substantially suppressing structured or correlated artifacts in arbitrary inputs.

B.1 Typical Noise Accepted by a Pretrained Model The pretrained reverse kernel is calibrated under the assumption that its standardized perturbation follows standard Gaussian noise. Therefore, when we inject a reward-informed vector through the noise term, the relevant question is not merely whether the vector has the correct overall scale, but whether it lies in the regime of perturbations that the pretrained model naturally accepts. In this subsection, we clarify this regime and motivate why our whitening operator is designed through confidence-interval projections rather than simple norm matching. Let z ∈ RN follow the standard multivariate Gaussian distribution p(\bm {z}) = (2\pi )^{-N/2} \exp \!\left ( -\frac {\|\bm {z}\|_2^2}{2} \right ). \label {eq:gaussian_pdf_app}

(68)

Although this density is radially symmetric, its probability mass concentrates sharply in high dimensions. In particular, \|\bm {z}\|_2^2 \sim \chi ^2_N, \label {eq:chi_square_app} (69) √ whose mean is N and whose standard deviation is 2N . As N grows, the relative fluctuation of ∥z∥22 becomes √ small, so most samples concentrate near a thin hyperspherical shell of radius approximately N . This is the familiar norm concentration phenomenon of high-dimensional Gaussian noise. √ A natural first approximation is therefore to regard vectors on the hypersphere ∥z∥2 = N as typical. However, this is too weak for pretrained generative models. As illustrated in Figure 8, latent vectors can have the same norm while behaving very differently in a pretrained model. When the vector exhibits the fine-scale irregularity characteristic of standard Gaussian noise, the pretrained model produces valid samples. In contrast, when the vector contains structured spatial correlation or other atypical regularities, the same pretrained model can produce severe failure cases, even though the vector lies on the same hypersphere. Thus, norm concentration alone does not characterize the perturbation regime that a pretrained model reliably accepts. This observation suggests that the practically relevant object is not the hypersphere itself, but the subset of vectors whose statistics resemble those of typical standard Gaussian noise. More importantly, these are precisely the perturbations that the pretrained reverse kernel is calibrated to

27

NoiseTilt: Noise-Tilted Reverse Kernels

Typical Noise

Latent z (∥z∥2 =

√

N)

Atypical Noise

Sampled Image

Latent z (∥z∥2 =

√

N)

Sampled Image

Figure √8. Identical norm does not guarantee noise compatibility. Latent vectors with identical norm (∥z∥2 = N ) can still behave very differently under pretrained reverse dynamics. Vectors whose statistics resemble typical standard Gaussian noise produce valid samples, whereas structured or spatially correlated vectors, despite lying on the same hypersphere, lead to severe failure cases.

process: during pretraining, the model repeatedly encounters standard Gaussian samples drawn from the reference noise distribution, not arbitrary vectors on the hypersphere. Therefore, if we wish to inject a reward-informed signal through the noise term while preserving the pretrained reverse dynamics, the injected vector should mimic the statistical characteristics of those typical sampled perturbations as closely as possible. In this sense, our goal is to move an arbitrary input toward the noise-compatible regime of standard Gaussian noise. Intuitively, such vectors should not only have the correct global scale, but also avoid unnatural local structure, excessive correlation, or anomalous concentration of values and energies. The exact typical set of high-dimensional Gaussian noise is difficult to describe in a closed form that is mathematically explicit. Accordingly, rather than attempting to characterize it exactly, we construct a tractable surrogate based on a collection of high-confidence constraints derived from standard Gaussian reference statistics. Our whitening operator is built from this perspective. Each component enforces one property that standard Gaussian noise satisfies with overwhelming probability, while modifying the input as little as possible in Euclidean norm. Concretely, throughout this appendix we use 99.99% confidence intervals (CI) under the reference distribution and realize whitening as a sequence of projections onto the corresponding confidence sets. This gives a practical approximation of the noise-compatible regime: already-typical Gaussian noise is changed only minimally, while structured or correlated inputs are pushed toward the statistics of typical standard Gaussian noise. The remaining subsections make this construction explicit. We first introduce the basic two-level order-statistic (2OS) projection in Section B.2, then extend it to tile-wise mean and energy statistics in Section B.3, and finally apply the same principle in multiple orthogonal transform domains in Section B.4.

B.2 Confidence-Interval Projection via Two-Level Order Statistics We now introduce the two-level order-statistic projection (2OS), which is a key building block of our whitening operator. Its purpose is to move an arbitrary input toward typical standard Gaussian noise by projecting onto high-confidence sets derived from the reference distribution. To motivate why 2OS is needed, we first consider two simpler confidence-interval (CI) constructions. These preliminary constructions are included only to explain the design choice. They are not components of the full whitening operator. Rather, they show why we ultimately use two-level order statistics 28

NoiseTilt: Noise-Tilted Reverse Kernels

(a)

Global CI

(b)

1OS CI

(c)

2OS CI

Figure 9. Illustration of the confidence-interval hierarchy. Confidence-intervals (CI) for the standard normal distribution with confidence level 99.99% are shown. (a) The coarsest constraint is the global confidence interval for a single variable. (b) A tighter characterization is obtained by using rank-dependent confidence intervals for one-level order statistics (1OS). (c) Our two-level order-statistic (2OS) construction further refines the admissible range by conditioning on both the outer rank r and the inner rank j within each chunk.

instead of relying only on global value bounds or one-level sorted statistics. Figure 9 summarizes this build-up. Panel (a) shows the coarsest possibility: a global confidence interval for a single Gaussian variable. Panel (b) refines this by using rank-dependent confidence intervals for one-level order statistics. Panel (c) then shows the two-level construction used in our method, which additionally conditions on the inner rank within each local chunk. The key distinction is that only this final construction constrains how values are distributed inside each chunk, which is what allows it to suppress localized structure. Throughout this subsection, let H denote a continuous reference CDF and let H −1 denote its inverse CDF. In our Gaussian instantiation, we use H = Φ, the standard normal CDF, and fix the confidence level to 99.99%. A first construction: global confidence intervals (Global CI). The coarsest confidence-based construction is to require each coordinate to lie in a global high-confidence interval under the reference distribution; see Figure 9(a). Given confidence level 1 − α, define \underline {a} = H^{-1}\!\left (\frac {\alpha }{2}\right ), \qquad \overline {a} = H^{-1}\!\left (1-\frac {\alpha }{2}\right ), \label {eq:global_ci_bounds_app}

(70)

\mathcal {S}_{\mathrm {box}} = \Bigl \{ \bm {y}\in \mathbb {R}^{N} :\; \underline {a}\le y_i\le \overline {a} \;\; \forall i \Bigr \}. \label {eq:box_set_app}

(71)

and let

Proposition B.1. The Euclidean projection onto Sbox is given by elementwise clipping: \Pi _{\mathcal {S}_{\mathrm {box}}}(\bm {x}) = \bigl ( \mathrm {clip}(x_i;\underline {a},\overline {a}) \bigr )_{i=1}^{N}. \label {eq:box_projection_app}

(72)

\|\bm {y}-\bm {x}\|_2^2 = \sum _{i=1}^{N}(y_i-x_i)^2,

(73)

Proof. Because

and the constraints in Equation (71) decouple across coordinates, the minimization separates into N independent scalar projections onto the interval [a, a], which is exactly elementwise clipping. This construction preserves natural variability, since it imposes an interval rather than a deterministic equality. However, it is still far too weak for our purpose. It constrains only marginal value range, so many structured vectors can satisfy Equation (71) while remaining far from the noise-compatible regime of typical standard Gaussian noise. A second construction: one-level order statistics (1OS). A more informative construction is to constrain the sorted values themselves, as visualized in Figure 9(b). Let {xi }M i=1 be i.i.d. samples 29

NoiseTilt: Noise-Tilted Reverse Kernels

from the reference distribution with CDF H, and let x(r) denote the r-th smallest sample. By the probability integral transform, ui := H(xi ) are i.i.d. Unif(0, 1), and therefore u_{(r)} \sim \mathrm {Beta}\bigl (r,\,M-r+1\bigr ). \label {eq:orderstat_beta_app}

(74)

This yields rank-dependent confidence bounds in the quantile domain: p^{\mathrm {lo}}_{r}(M) = \mathrm {Beta}^{-1}\!\Bigl ( \frac {\alpha }{2};\, r,\, M-r+1 \Bigr ), \quad p^{\mathrm {hi}}_{r}(M) = \mathrm {Beta}^{-1}\!\Bigl ( 1-\frac {\alpha }{2};\, r,\, M-r+1 \Bigr ). \label {eq:beta_bounds_outer_app}

(75)

Mapping them back through H −1 gives the value-domain thresholds L_r = H^{-1}\!\bigl (p^{\mathrm {lo}}_{r}(M)\bigr ), \qquad U_r = H^{-1}\!\bigl (p^{\mathrm {hi}}_{r}(M)\bigr ). \label {eq:gauss_bounds_outer_app}

(76)

Let x↑ = (x(1) , . . . , x(M ) ) denote the sorted version of x, and let Px be the permutation matrix satisfying x↑ = Px x. Define \mathcal {S}_{\mathrm {os}} = \Bigl \{ \bm {y}\in \mathbb {R}^{M} :\; L_r\le y_{(r)}\le U_r, \ \ r=1,\dots ,M \Bigr \}. \label {eq:os_set_app}

(77)

Proposition B.2. A Euclidean projection onto Sos is obtained by sorting, clipping in sorted order, and undoing the sort: \Pi _{\mathcal {S}_{\mathrm {os}}}(\bm {x}) = P_{\bm {x}}^{\top }\, \mathrm {clip}\bigl ( \bm {x}_{\uparrow };\, \bm {L},\, \bm {U} \bigr ), \label {eq:os_projection_app} (78) where L = (L1 , . . . , LM ) and U = (U1 , . . . , UM ). Proof. Any y ∈ Sos can be written as y = P ⊤ y↑ for some permutation P and some nondecreasing vector y↑ satisfying Lr ≤ (y↑ )r ≤ Ur . Since the Euclidean norm is permutation-invariant, \|\bm {y}-\bm {x}\|_2^2 = \|\bm {y}_{\uparrow }-P\bm {x}\|_2^2.

(79)

For fixed y↑ , this is minimized when P x is sorted in the same order as y↑ , that is, when P = Px , by the rearrangement inequality. The remaining problem is therefore the Euclidean projection of the sorted vector x↑ onto the box constraints Lr ≤ yr ≤ Ur , which is achieved by elementwise clipping. This one-level construction is much tighter than the global interval, since it constrains each rank separately rather than only the overall value range. For example, when H = Φ, α = 10−4 , and M = 65536, the global 99.99% confidence interval is [−3.8906, 3.8906], whose width is 7.7812. In contrast, the one-level interval width Ur − Lr is only 0.0381 near the median rank and 0.0414 even at r = 16384, making it roughly 200× tighter than the global interval. Even at the most extreme rank r = 1, the width is 2.4282, which is still more than 3× tighter. Nevertheless, one-level order statistics still do not solve the main problem. Because they depend only on the globally sorted values, they are permutation-invariant with respect to the original coordinates. They constrain the global histogram, but they do not constrain how small and large values are distributed across local chunks. As a result, substantial spatial structure can still survive. This limitation is illustrated in Figure 10. Both the global CI and one-level CI projections reduce extreme values, but visible stripe-like structure remains in the latent visualization. This is precisely what motivates the two-level construction below: we need a confidence-interval projection that controls not only which values appear globally, but also how those values are distributed locally. Two-level order statistics (2OS). We now introduce the construction actually used in our whitening operator. To make the confidence-interval constraint sensitive to local organization, we partition the vector into local chunks and enforce order-statistic consistency both within each chunk and across chunks; see Figure 9(c).

30

NoiseTilt: Noise-Tilted Reverse Kernels

Global

1OS

2OS

Sampled Image

Latent Visualization

Input

Figure 10. Effect of confidence-interval (CI) projection via two-level order statistics (2OS). Global CI clipping only truncates extreme values and leaves the dominant structured pattern largely unchanged. 1OS CI further matches the global sorted histogram, but still preserves substantial local structure. In contrast, 2OS CI enforces rank consistency both within and across local chunks, forcing each chunk to contain a balanced spread of small-to-large values. This suppresses localized artifacts much more effectively and makes the latent substantially more like a typical noise. The bottom row shows the corresponding effect on the sampled image.

Specifically, reshape the vector into a matrix \bm {X}\in \mathbb {R}^{M\times D},

(80)

where each row corresponds to a chunk of size D. We first sort within each row, and then, for each within-row rank j, sort across the M rows. This yields the doubly sorted representation \bm {Z} = \mathrm {sort}_0\bigl (\mathrm {sort}_1(\bm {X})\bigr ), \label {eq:twolevel_sort_def_app}

(81)

where r indexes the outer rank across chunks and j indexes the inner rank within each chunk. Equivalently, Zr,j is the r-th smallest value among the j-th order statistics collected from all chunks. Under the i.i.d. reference model, the induced quantiles are obtained by composing Beta laws. After fixing the outer rank r, the confidence interval for the corresponding quantile is given by Equation (75). Conditioning further on the inner rank j within a chunk yields p^{\mathrm {lo}}_{r,j}(M,D) &= \mathrm {Beta}^{-1}\!\Bigl ( p^{\mathrm {lo}}_{r}(M);\, j,\, D-j+1 \Bigr ), \label {eq:beta_bounds_twolevel_lo_app} \\ p^{\mathrm {hi}}_{r,j}(M,D) &= \mathrm {Beta}^{-1}\!\Bigl ( p^{\mathrm {hi}}_{r}(M);\, j,\, D-j+1 \Bigr ), \label {eq:beta_bounds_twolevel_hi_app} (83) and therefore L_{r,j} = H^{-1}\!\bigl (p^{\mathrm {lo}}_{r,j}(M,D)\bigr ), \qquad U_{r,j} = H^{-1}\!\bigl (p^{\mathrm {hi}}_{r,j}(M,D)\bigr ). \label {eq:gauss_bounds_twolevel_app}

(84)

Now define the feasible set \mathcal {S}_{\mathrm {2os}} = \Bigl \{ \bm {Y}\in \mathbb {R}^{M\times D} :\; L_{r,j} \le \bigl (\mathrm {sort}_0(\mathrm {sort}_1(\bm {Y}))\bigr )_{r,j} \le U_{r,j}, \ \forall r,j \Bigr \}. \label {eq:twolevel_set_app}

(85)

Proposition B.3. Let Z be defined by Equation (81). Then clipping each entry of Z into its corresponding interval [Lr,j , Ur,j ] and undoing the two sorting permutations yields a Frobenius-norm

31

NoiseTilt: Noise-Tilted Reverse Kernels

projection onto S2os : \Pi _{\mathcal {S}_{\mathrm {2os}}}(\bm {X}) \in \arg \min _{\bm {Y}\in \mathcal {S}_{\mathrm {2os}}} \|\bm {Y}-\bm {X}\|_F^2. \label {eq:twolevel_projection_app}

(86)

Proof. The row-wise and column-wise sorting operations are compositions of permutation matrices and therefore preserve the Frobenius norm. Accordingly, for the purpose of Euclidean projection, we may pass to the canonical representative obtained by doubly sorting X. e satisfying Now fix a candidate doubly sorted matrix Z L_{r,j}\le \widetilde {Z}_{r,j}\le U_{r,j} \qquad \forall r,j.

(87)

e the one closest to X is obtained Among all matrices whose doubly sorted representation equals Z, by undoing the sorting permutations induced by X itself. This follows by repeated application of the rearrangement inequality: first within each row, and then across rows for each column. Consequently, the projection problem reduces to minimizing \|\widetilde {\bm {Z}}-\bm {Z}\|_F^2

(88)

subject to the box constraints above. Since these constraints decouple entrywise, the minimizer is obtained by clipping each Zr,j independently to [Lr,j , Ur,j ]. Undoing the sorting permutations then gives a Frobenius-norm minimizer in the original coordinates. The key difference from the one-level construction is that clipping is applied to every rank pair (r, j) of the doubly sorted statistic. As a result, extreme values cannot concentrate in only a few chunks. Instead, each chunk is forced to contain a balanced spread of small-to-large values consistent with typical standard Gaussian noise. In this sense, 2OS constrains not only the global histogram, but also the local distributional composition of each chunk. This is the core mechanism by which it suppresses localized structure and pushes an arbitrary vector toward typical noise.

B.3 Confidence-Interval Projection on Tile-wise Mean and Energy The 2OS projection in Section B.2 acts directly on the entries of a partitioned representation. This already suppresses localized structure by forcing each chunk to contain a balanced spread of small-to-large values. However, entrywise 2OS alone does not fully control partition-wise statistics. For example, a chunk may still contain values from all bands while having an unusually low or high mean, or it may contain values that are too concentrated around that mean, leading to an atypical centered energy. To further move the input toward typical standard Gaussian noise, we therefore apply the same confidence-interval principle to tile-wise statistics themselves. Figure 11 illustrates this effect. Starting from the output of 2OS CI on entries, we additionally constrain tile-wise statistics at multiple tile scales. This progressively removes residual low-frequency bias. This is exactly the role of tile-wise statistics in the whitening operator: they complement entrywise 2OS by directly controlling whether each partition has Gaussian-like aggregate behavior. Let x ∈ RN be partitioned into P disjoint tiles of equal size F (so N = P F ), denoted by \bm {x}^{(p)}\in \mathbb {R}^{F}, \qquad p=1,\dots ,P.

(89)

For each tile, define the tile-wise mean and centered energy \bar {x}^{(p)} = \frac {1}{F}\bm {1}^{\top }\bm {x}^{(p)}, \qquad v^{(p)} = \left \| \bm {x}^{(p)}-\bar {x}^{(p)}\bm {1} \right \|_2^2, \label {eq:block_stats_app}

(90)

where 1 ∈ RF is the all-ones vector. Under the standard Gaussian reference model, these statistics have known distributions. Indeed,

32

NoiseTilt: Noise-Tilted Reverse Kernels

+ 2 × 2 Tile Mean/Energy

2OS

+ 8 × 8 Tile Mean/Energy

Sampled Image

Latent Visualization

Input

Figure 11. Effect of confidence-interval (CI) projection on tile-wise statistics. Starting from entrywise 2OS, we additionally constrain tile-wise mean and centered energy at 2 × 2 and 8 × 8 tile scales. These tile-wise constraints further suppress residual low-frequency structure.

if x(p) ∼ N (0, IF ), then s^{(p)} := \sqrt {F}\,\bar {x}^{(p)} \sim \mathcal {N}(0,1), \qquad v^{(p)} \sim \chi ^2_{F-1}. \label {eq:block_stat_dists_app}

(91)

Thus the normalized tile-wise mean is standard normal, while the centered energy follows a chi-square law with F − 1 degrees of freedom. We now collect these quantities over all tiles. Define the tile-wise statistic vectors \bm {s}\in \mathbb {R}^{P}, \qquad \bm {v}\in \mathbb {R}^{P}, \label {eq:block_stat_fields_app}

(92)

whose p-th entries are the normalized mean s(p) and centered energy v (p) of the corresponding tile. We then apply the same 2OS projection introduced in Section B.2 to these statistic vectors, using the appropriate reference distributions. Let Hs and Hv denote the CDFs of the standard normal and χ2F −1 distributions, respectively. We define \bm {n} = \Pi ^{\mathrm {(2os)}}_{H_s}(\bm {s}), \qquad \bm {m} = \Pi ^{\mathrm {(2os)}}_{H_v}(\bm {v}), \label {eq:tile_stat_projection_app}

(93)

(2os)

where ΠH denotes the two-level order-statistic (2OS) projection of Equation (86), with confidence bounds computed from the reference CDF H. The projected statistic fields n and m specify target tile-wise mean and energy values. For each tile, we then update the tile to match those targets with minimal Euclidean modification. Specifically, let \mu _p^\star = \frac {n^{(p)}}{\sqrt {F}}, \qquad v_p^\star = m^{(p)}, \label {eq:block_targets_app}

(94)

\bm {x}^{(p)}_{\mathrm {new}} = \mu _p^\star \bm {1} + \left ( \bm {x}^{(p)}-\bar {x}^{(p)}\bm {1} \right ) \sqrt {\frac {v_p^\star }{v^{(p)}}}. \label {eq:block_mean_energy_update_app}

(95)

and define

Thus each tile is shifted to the target mean and rescaled to the target centered energy, while

33

NoiseTilt: Noise-Tilted Reverse Kernels

preserving its centered direction. Proposition B.4. Fix a tile x(p) ∈ RF and target values (µ⋆p , vp⋆ ) with vp⋆ ≥ 0. Define the feasible set \mathcal {S}(\mu _p^\star ,v_p^\star ) = \Bigl \{ \bm {y}\in \mathbb {R}^{F} :\; \frac {1}{F}\bm {1}^{\top }\bm {y}=\mu _p^\star , \ \ \bigl \| \bm {y}-\mu _p^\star \bm {1} \bigr \|_2^2=v_p^\star \Bigr \}. \label {eq:block_feasible_exact_app} (96) Let x̄(p) = F1 1⊤ x(p) and v (p) = ∥x(p) − x̄(p) 1∥22 . If v (p) > 0, then the update in Equation (95) is the Euclidean projection of x(p) onto S(µ⋆p , vp⋆ ): \bm {x}^{(p)}_{\mathrm {new}} = \Pi _{\mathcal {S}(\mu _p^\star ,v_p^\star )} \bigl (\bm {x}^{(p)}\bigr ) \in \underset {\bm {y}\in \mathcal {S}(\mu _p^\star ,v_p^\star )}{\arg \min } \|\bm {y}-\bm {x}^{(p)}\|_2^2. \label {eq:block_proj_exact_app}

(97)

When v (p) = 0, the minimizer is not unique. Proof. Write \bm {x}^{(p)} = \bar {x}^{(p)}\bm {1}+\bm {c}, \qquad \bm {1}^{\top }\bm {c}=0, \qquad \|\bm {c}\|_2^2=v^{(p)}.

(98)

Any feasible y ∈ S(µ⋆p , vp⋆ ) can be written as \bm {y} = \mu _p^\star \bm {1}+\bm {d}, \qquad \bm {1}^{\top }\bm {d}=0, \qquad \|\bm {d}\|_2^2=v_p^\star .

(99)

\|\bm {y}-\bm {x}^{(p)}\|_2^2 = \|(\mu _p^\star -\bar {x}^{(p)})\bm {1}\|_2^2 + \|\bm {d}-\bm {c}\|_2^2 = F(\mu _p^\star -\bar {x}^{(p)})^2 + \|\bm {d}-\bm {c}\|_2^2. \label {eq:block_obj_decompose_app}

(100)

By orthogonality,

The first term is fixed once µ⋆p is fixed. For the second term, \|\bm {d}-\bm {c}\|_2^2 = \|\bm {d}\|_2^2+\|\bm {c}\|_2^2-2\langle \bm {d},\bm {c}\rangle ,

(101)

so minimizing it is equivalent to maximizing ⟨d, c⟩ subject to ∥d∥22 = vp⋆ . This is achieved when d is colinear with c. Hence, when v (p) > 0, the unique minimizer is \bm {d} = \bm {c}\sqrt {\frac {v_p^\star }{v^{(p)}}},

(102)

which yields Equation (95). If v (p) = 0, then c = 0, and any feasible d with 1⊤ d = 0 and ∥d∥22 = vp⋆ attains the same minimum. The role of this construction is complementary to entrywise 2OS. The projection in Section B.2 ensures that each tile contains a balanced spread of values across different rank bands. The present construction checks whether the aggregate behavior of each tile is statistically typical. By constraining tile-wise mean and centered energy through their known reference distributions, we prevent a subset of tiles from having unusually biased averages or atypical local energy, even if their entrywise order statistics already look plausible. This is why tile-wise statistics are a natural next component of the whitening operator after 2OS.

B.4 Confidence-Interval Projection in Orthogonal Transform Domains The constructions in Sections B.2 and B.3 constrain entrywise order statistics and tile-wise statistics in the original coordinates. These components already suppress substantial local structure, but some artifacts are still more naturally exposed after an orthogonal change of basis. In particular, residual global patterns may remain diffuse in the original indexing while becoming much more explicit in a transformed domain. For this reason, our whitening operator also applies the same confidence-interval projections in additional orthogonal transform domains.

34

NoiseTilt: Noise-Tilted Reverse Kernels

2OS + Tile-wise

+ Fourier

+ Hadamard

Sampled Image

Latent Visualization

Input

Figure 12. Effect of confidence-interval (CI) projection in orthogonal transform domains. Starting from the output of 2OS and tile-wise constraints in the original coordinates, we additionally apply the same confidenceinterval projections in a compact Fourier domain and then in a Hadamard-style mixing domain. These transformed views expose complementary global structure, progressively suppressing residual artifacts and improving the sampled image.

This effect is illustrated in Figure 12. After applying 2OS together with tile-wise constraints in the original coordinates, the latent is already substantially whitened, but the sampled image still contains severe global artifacts. Applying the same confidence-interval projections in a compact Fourier domain further suppresses frequency-localized structure and yields a recognizable image, although visible artifacts remain. Applying them again in a Hadamard-style mixing domain removes much of the remaining coherent structure and produces a valid sample. Thus, orthogonal transform domains provide complementary global views that are not fully captured by coordinate-domain or tile-wise constraints alone. The validity of this strategy rests on a simple geometric fact: Euclidean projection is preserved under orthogonal changes of coordinates. Proposition B.5. Let T : RN → RN be orthogonal, i.e., T^\top T=\bm {I}.

(103)

For any nonempty closed set S ⊂ RN , define T(\mathcal {S}) = \{T\bm {y}:\bm {y}\in \mathcal {S}\}.

(104)

T\,\Pi _{\mathcal {S}}(\bm {x}) \in \Pi _{T(\mathcal {S})}(T\bm {x}), \label {eq:isometry_proj_commute_app}

(105)

\Pi _{\mathcal {S}}(\bm {x}) \in T^\top \Pi _{T(\mathcal {S})}(T\bm {x}). \label {eq:isometry_proj_commute_inv_app}

(106)

Then, for any x ∈ RN , equivalently, Hence, projecting in the transformed domain and mapping back is exactly the Euclidean projection onto the corresponding pullback set in the original domain.

35

NoiseTilt: Noise-Tilted Reverse Kernels

Proof. Since T is orthogonal, \|T\bm {u}-T\bm {v}\|_2 = \|\bm {u}-\bm {v}\|_2 \qquad \forall \bm {u},\bm {v}\in \mathbb {R}^N.

(107)

Let y ⋆ ∈ ΠS (x). Then for every y ∈ S, \|T\bm {x}-T\bm {y}^\star \|_2 = \|\bm {x}-\bm {y}^\star \|_2 \le \|\bm {x}-\bm {y}\|_2 = \|T\bm {x}-T\bm {y}\|_2.

(108)

Hence T y ⋆ is a minimizer of the distance from T x over T (S), proving Equation (105). Applying T ⊤ yields Equation (106). A second fact is that the standard Gaussian reference distribution is itself invariant under orthogonal transforms. Proposition B.6. If T ⊤ T = I and ϵ ∼ N (0, IN ), then T\bm {\epsilon }\sim \mathcal {N}(\bm {0},\bm {I}_N). \label {eq:gaussian_invariant_app}

(109)

Consequently, confidence intervals derived under i.i.d. standard Gaussian assumptions remain valid after applying T . Proof. Since T is linear, T ϵ is Gaussian with mean 0 and covariance T\bm {I}_N T^\top = \bm {I}_N.

(110)

Therefore T ϵ ∼ N (0, IN ). Compact orthogonal Fourier domain. Our first transformed view is a compact real representation of the orthonormal discrete Fourier transform. For an even-length real vector x ∈ RN , let \hat {\bm {x}} = \mathcal {F}_{\mathrm {r}}(\bm {x}) \in \mathbb {C}^{N/2+1},

(111)

where Fr denotes the orthonormal real Fourier transform. We then define the compact real transform \mathcal {T}_{\mathrm {F}}(\bm {x}) = \Bigl [ \Re (\hat {x}_0),\, \sqrt {2}\Re (\hat {\bm {x}}_{\mathrm {int}})^{\top },\, \Re (\hat {x}_{N/2}),\, \sqrt {2}\Im (\hat {\bm {x}}_{\mathrm {int}})^{\top } \Bigr ]^{\top } \in \mathbb {R}^{N}, \label {eq:compact_fourier_transform_app}

(112)

where x̂int = (x̂1 , . . . , x̂N/2−1 ) collects the interior Fourier coefficients. That is, we keep the two real boundary frequencies explicitly and pack the independent interior Fourier coefficients by √ concatenating their real and imaginary parts. The factor 2 on the interior frequencies accounts for the Hermitian symmetry of the Fourier transform of a real signal and preserves the Euclidean norm. Consequently, TF is an orthogonal linear bijection from RN to RN . Its inverse simply reconstructs the Hermitian-symmetric Fourier coefficients and then applies the inverse real FFT. The purpose of this domain is to expose structure that is more naturally described in frequency space. Periodic artifacts, low-frequency bias, and anomalous spectral concentration may remain difficult to suppress when viewed only through local statistics in the original coordinates. In the compact Fourier domain, the same CI projections from Sections B.2 and B.3 directly constrain those frequency-domain coordinates while remaining statistically consistent with the same standard Gaussian distribution. Hadamard-style mixing domain. Our second transformed view is a sparse orthogonal mixing operator based on repeated pairwise sum/difference butterflies. Let a tile of size H × W be flattened into a vector z ∈ RD with D = HW . A single butterfly step maps each adjacent pair according to (z_{2k-1},z_{2k}) \mapsto \left ( \frac {z_{2k-1}+z_{2k}}{\sqrt {2}}, \frac {z_{2k-1}-z_{2k}}{\sqrt {2}} \right ), \qquad k=1,\dots ,\frac {D}{2}. \label {eq:hadamard_butterfly_app}

36

(113)

NoiseTilt: Noise-Tilted Reverse Kernels

Applied independently across all tiles, this defines an orthogonal transform. Repeated application produces a Hadamard-style mixing effect that progressively redistributes local information across coordinates. The role of this domain is different from the Fourier domain. Whereas the compact Fourier transform makes frequency-localized anomalies explicit, the Hadamard-style mixing domain exposes coherent global structure through orthogonal linear combinations of coordinates. In practice, this complementary view is important: after Fourier-domain filtering, some residual artifacts can still remain spatially coherent, and Hadamard-style mixing makes them easier to detect and suppress by the same CI projections. In both transformed domains, the projection principle itself does not change: we transform the signal, apply the CI projections developed in Sections B.2 and B.3, and map the result back. Because the transforms are orthogonal and preserve the standard Gaussian law, these projections continue to target the same noise-compatible regime of typical standard Gaussian noise.

B.5 Component-Level Ablation of the Whitening Operator Table 6 examines the contribution of each component of W to reward alignment performance. Starting from the 2OS projection alone, we progressively add tile-wise statistics and then multidomain projections, holding the sampling configuration fixed throughout (NFE = 25). Table 6. Component-level ablation of the whitening operator W on aesthetic image generation. We fix the sampling configuration (NFE = 25) and progressively add components to the whitening operator. Dark green denotes the best target reward; light green denotes the second best. Whitening Operator

Target Reward

Held-out Rewards

Aesthetic ↑ Score

Pick ↑ -score

HPSv2 ↑

Image ↑ Reward

Base

6.0282

0.2144

0.2759

1.0538

2OS proj. + Tile-wise stat. + Multi. domain (Ours)

7.3439 7.4607 7.4510

0.2179 0.2181 0.2200

0.2891 0.2909 0.2928

1.1430 1.0585 1.2565

Figure 13. Trade-off under component ablation. Full components achieve the best trade-off.

The 2OS projection alone already provides a substantial gain in the target reward over the unguided base. Adding tile-wise statistics yields the highest target reward among the three configurations, but reduces ImageReward, reflecting a trade-off between target optimization and distributional balance across rewards. Incorporating multi-domain projections restores the held-out metrics to their highest values while preserving most of the target-reward gain, achieving the best overall trade-off. These results suggest that each component addresses a complementary aspect of noise compatibility: 2OS controls local value statistics, tile-wise statistics regulate aggregate per-tile behavior, and multi-domain projections target residual structure that is more naturally exposed in alternative orthogonal bases. Figure 13 further visualizes the target-vs-held-out trade-off as components are added.

B.6 Implementation and Optimization of the Whitening Operator The whitening operator admits many possible configurations, since it is determined by the combination of the 2OS chunk partitions in Section B.2, the tile partitions for mean and centered energy in Section B.3, and the orthogonal transform domains in Section B.4. In practice, however, the computational bottleneck is clear: the dominant cost comes from the repeated sort–clip–unsort pattern inside the 2OS projection. By contrast, the tile-wise mean and centered-energy updates are comparatively cheap, since they consist only of reductions, rescaling, and broadcasting. Accordingly, our implementation is designed to make the 2OS projections as GPU-friendly as possible.

37

NoiseTilt: Noise-Tilted Reverse Kernels

Optimization. Our first optimization is to keep the 2OS chunk sizes small. In practice, we use chunk sizes such as 4 and 64, so that the inner sorting dimension remains modest. When the chunk size is 4, the sorting can be handled essentially at thread-local scale. When the chunk size is 64, it still fits comfortably within a single GPU block and can be processed efficiently in shared memory. This matters because each 2OS projection repeatedly sorts within chunks and then across chunks at a fixed rank index. By keeping the chunk size small, we keep these repeated sorting operations efficient even when the whitening operator is applied many times. Our second optimization is to precompute all confidence bounds once at initialization. For each configuration, we cache the Gaussian and chi-square thresholds for all relevant tuples of outer-rank count, chunk size, and tile size. As a result, the online cost of each whitening call is dominated by tensor reshaping, sorting, clipping, FFTs, and orthogonal mixing, rather than by repeated evaluation of Beta, Gaussian, or chi-square quantiles. In addition, because the same sort–clip–unsort pattern is reused across many nested calls, temporary tensors and indexing patterns can also be reused efficiently on GPU. Configurations. For FLUX latents with shape [1024, 64], we use 2OS chunk sizes (2, 2) and (8, 8), tile-wise statistic sizes (1, 1), (2, 2), and (8, 8), and two Hadamard-style mixing domains: the full latent resolution and a 4 × 4 tiled mixing, repeated 32 and 8 times, respectively. For the larger Wan2.1 latent with shape [16, 13, 60, 104], reshaped as (16·13·60, 104), we use chunk sizes (2, 2) and (16, 4), tile-wise statistic sizes (2, 2) and (13, 13), and the same two mixing domains, repeated 42 and 8 times. In all cases, we use confidence level α = 10−4 . Table 7. Wall-clock overhead of the whitening operator.

Model FLUX [35] Wan2.1 [70]

Per-step Whitening Whitening DPS [10] → NTRK Model Time Time Slowdown 0.873 s 26.1 s

0.068 s 0.5 s

+7.8% +1.9%

87 s → 94 s 650 s → 663 s

Wall-clock time. These configurations were selected so that whitening accounts for less than 10% of the overall reward-alignment cost. For FLUX, one reward-guided model step takes about 0.87 s, while the whitening operator takes about 0.068 s, i.e., about 7.8% of the per-step wall-clock time. At the level of full runs, DPS [10] takes about 87 s, whereas adding whitening increases this only to 94 s. For Wan2.1, one model step takes about 26 s, while whitening takes about 0.5 s, which is roughly 2% of the runtime. Correspondingly, DPS [10] takes about 650 s, and the whitened version takes about 663 s. Thus, although the whitening operator combines several CI projections across multiple domains, its overhead remains modest relative to the underlying generative model. An important practical advantage is that this cost can be adjusted continuously through the configuration. Reducing the number of chunk sizes, tile sizes, transform domains, or mixing repetitions yields a cheaper operator, while adding them yields a stronger but slower one. This makes the whitening operator easy to adapt to models with different latent sizes and runtime budgets.

B.7 Comparison with Simpler Operators To understand why the quality of the whitening operator matters, we compare NTRK with three simpler alternatives. The key question is not merely whether an operator removes visible structure, but whether it moves an arbitrary input toward the noise-compatible regime of typical standard Gaussian noise while leaving already-typical noise nearly unchanged. We consider four whitening operators. The first is the identity map, \mathcal {W}_{\mathrm {None}}(\bm {z})=\bm {z},

38

(114)

NoiseTilt: Noise-Tilted Reverse Kernels

which corresponds to using the raw reward-informed direction without whitening. The second is norm projection, \mathcal {W}_{\mathrm {Norm}}(\bm {z}) = \sqrt {N}\,\frac {\bm {z}}{\|\bm {z}\|_2}, \label {eq:norm_projection_app} (115) √ which places the input on the high-probability hypersphere of radius N , but does not constrain its finer statistics. The third is WWGNC , based on White Gaussian Noise Constraints (WGNC) [24], which projects onto a spectral feasible set defined by hard blockwise constraints in a compact Fourier domain. This suppresses structured frequency artifacts much more strongly than norm projection, but its hard equalities remove the natural variability that genuine Gaussian noise should retain. The last is our proposed operator, WOurs . Table 8. Ablation on the whitening operator for NTRK on aesthetic image generation. We fix the sampling configuration (NFE = 25) and replace only the whitening operator. Dark green cells indicate the best result in the target reward, while light green cells denote the second best. Setting

Base [35] NTRK w/ WNone NTRK w/ WNorm NTRK w/ WWGNC NTRK w/ WOurs

Target Reward

Held-out Rewards

Aesthetic ↑ Score

Pick ↑ -score

HPSv2 ↑

Image ↑ Reward

VQA ↑ Score

6.0282 6.2392 7.1036 7.2096 7.4510

0.2144 0.2207 0.2147 0.2174 0.2200

0.2759 0.2981 0.2794 0.2901 0.2928

1.0538 1.2914 0.8908 1.1391 1.2565

0.9644 0.9690 0.9557 0.9672 0.9728

Figure 14. Trade-off under different whitening operators. Only WOurs improves the target reward without degrading the held-out quality.

Table 8 shows an ablation in which we fix the NTRK sampling setting and replace only the whitening operator. Using no whitening, i.e. WNone , improves the target reward only modestly. Norm projection, WNorm , increases the target reward further, but incurs the largest degradation in held-out quality. WWGNC provides a better trade-off than norm projection and recovers held-out quality above the base level, yet still lags behind WOurs on both target reward and held-out metrics. In contrast, WOurs achieves the highest target reward while also preserving, and in this case slightly improving, the held-out metrics. Therefore, the benefit of NTRK does not come merely from making the guidance stronger; it comes from injecting reward information through a perturbation that remains much more faithful to the statistics of typical standard Gaussian noise. Figure 15 compares the four operators on two representative inputs. In the top example of Figure 15, the input is strongly structured and leads to a severe failure case. Applying WNone or WNorm leaves the dominant structure largely intact, so the sampled image remains invalid. WWGNC removes much more of the visible structure, but still produces a noticeably distorted sample. By contrast, WOurs produces a much more typical noise and yields a realistic image. In the bottom example of Figure 15, the input is already a typical Gaussian noise. Here the desired behavior is the opposite: a good whitening operator should change the input as little as possible. Indeed, WOurs leaves the latent almost unchanged, whereas WWGNC still perturbs it visibly. Figure 15 illustrates the main difference between the two approaches: NTRK removes atypical structure when necessary, but preserves genuine noise when it is already compatible with the pretrained model. Overall, the quantitative results above and Figure 15 together show that a good whitening operator is crucial for reward alignment. Operators that are too weak fail to remove harmful structure, whereas operators that are too rigid distort even already-typical noise. The proposed whitening operator is effective precisely because it balances these two requirements: it strongly suppresses atypical structure when needed, while minimally altering inputs that already lie near the noise-compatible regime.

39

NoiseTilt: Noise-Tilted Reverse Kernels

WNorm (z)

WWGNC (z)

WOurs (z)

CosSim(z, W(z)) ∥z − W(z)∥2

1.000000 37.63

0.642806 203.35

0.480325 244.39

z (= WNone (z))

WNorm (z)

WWGNC (z)

WOurs (z)

CosSim(z, W(z)) ∥z − W(z)∥2

1.000000 1.98

0.989371 37.24

0.999996 0.77

Sampled Image

Latent Visualization

Sampled Image

Latent Visualization

z (= WNone (z))

Figure 15. Comparison with simpler whitening operators. Top: when the input latent z is strongly atypical, WNone and WNorm fail to remove the dominant structure, while WWGNC suppresses it more strongly but still yields a distorted sample. In contrast, WOurs produces a much more typical noise and a realistic image. Bottom: when z is already close to typical Gaussian noise, a good whitening operator should leave it nearly unchanged. Here, WOurs preserves the input most faithfully, whereas WWGNC still perturbs it visibly. The cosine similarity and ℓ2 distance quantify this trade-off: NTRK changes atypical inputs when needed, but minimally alters already-typical noise.

40

NoiseTilt: Noise-Tilted Reverse Kernels

C Additional Experimental Results In this section, we present additional experimental results and implementation details that complement the experiments in Section 5. We first summarize the implementation details across applications in Section C.1. We then compare with noise optimization methods in Section C.2. We further provide results on additional applications, including quantity-aware generation and VLM-based reward alignment, in Sections C.3 and C.4. We further report results with a different base model, Z-Image [6], for aesthetic and text-aligned image generation in Section C.5. Finally, we present additional results for the main experiments in Section C.6.

C.1 Implementation Details We summarize the implementation details across Table 9. Hyperparameter configurations for applications in this section. For image generation each application. tasks, we use FLUX [35] and Z-Image [6] as the Aesthetic Text- QuantityVLM Video base models for aesthetic image generation and Image Aligned Aware text-aligned image generation, and FLUX [35] for σt 0.2t 0.2t 0.2 0.2 0.2 quantity-aware generation and VLM-based reward ρt 0.3 0.3 0.5 0.5 0.5 alignment. For preference-aligned video generation, we use Wan2.1 [70] as the base flow model. Across all experiments, we fix the number of sampling steps to 25. The diffusion coefficient σt and the whitened guidance strength ρt used for each application are summarized in Table 9.

C.2 Comparison with Noise Optimization DNO [66] belongs to a distinct class of Table 10. Comparison with DNO [66] under the same methods: rather than modifying the per- NFE budget. Aesthetic image generation at 500 NFE. step reverse kernel, it treats all injected Target Reward Held-Out Reward NFE noise vectors as inference-time optimiza- Method Aesthetic Pick↑ ↑ HPSv2 ↑ tion variables and refines them through Score Score iterative Adam updates with a regulariza- Base 25 6.028 0.2144 0.2759 tion term. This design has a fundamenDNO [66] 6.772 0.2143 0.2717 tal budgetary consequence: each Adam (20 updates) 500 update requires a complete forward pass NTRK 500 7.966 0.2197 0.2932 through all T denoising steps, so generat- (20 particles) ing a single sample with K updates costs T × K NFEs. At the standard 25-NFE budget (K=0), DNO produces output identical to base sampling since no optimization has yet taken place. We compare DNO and NTRK under a matched budget of 500 NFE on the aesthetic image generation task. For DNO this corresponds to K=20 Adam updates over the full noise trajectory; for NTRK it corresponds to N =20 particles with Best-of-N. As shown in Table 10, NTRK achieves substantially higher aesthetic reward while better preserving held-out metrics, demonstrating that per-step kernel modification is a more budget-efficient strategy than trajectory-level noise optimization.

C.3 Additional Application: Quantity-Aware Generation Experiment Setup. We evaluate quantity-aware generation using 40 prompts from prior work, Ψ-Sampler [75], covering 20 object categories with randomly assigned target counts of up to 90. For all other experimental details, we follow the same setup described in Section 5.1. Evaluation Metrics. The target reward is negative smooth L1 loss computed with T2I-Count [49], which takes a generated image and the corresponding text prompt as input and returns a density map. For the held-out reward, we use alternative counting model, CountGD [1], and report MAE and counting accuracy (%), where a prediction is considered correct if the predicted count exactly 41

NoiseTilt: Noise-Tilted Reverse Kernels

Table 11. Quantitative comparison on quantity-aware generation. The target reward is T2I-Count [49]. For single-particle methods, we augment sampling with Best-of-N [65] and Ψ-Sampler [75] to match the total NFE, denoted with † and ‡ , respectively. Dark green cells indicate the best result for each metric across all runs, while light green cells denote the second best.

Target Reward Method

Held-Out Reward

NFE T2I-Count ↓

MAE ↓

Acc ↑ (%)

Image ↑ Reward

HPSv2 ↑

VQA ↑ Score

Base [35] BoN [65] DPS† [10] FreeDoM† [76] SVDD [38] RBF [29] DAS [30] Ψ-Sampler [75]

25 1000 1000 1045 1000 1000 1000 1000

16.365 2.668 2.565 5.210 1.530 1.796 2.061 1.426

16.700 4.925 4.900 7.925 3.100 4.025 3.875 3.215

5.0 22.5 15.0 20.0 17.5 30.0 27.5 30.0

0.579 0.642 0.716 0.303 0.729 0.646 0.789 0.854

0.262 0.269 0.268 0.259 0.271 0.271 0.271 0.273

0.928 0.904 0.925 0.931 0.953 0.936 0.950 0.951

NTRK‡ (Ours)

1000

0.045

2.325

37.5

0.681

0.266

0.915

matches the target count. As held-out rewards, we evaluate image quality using ImageReward [73] and HPSv2 [72], and text–image alignment using VQA Score [39]. For all methods, methods augmented with BoN [65] and Ψ-Sampler [75] are marked with † and ‡ , respectively. Results. The quantitative and qualitative results are presented in Table 11 and Figure 16, respectively. Overall, NTRK achieves the best target reward performance by a large margin over all baselines. On held-out rewards, NTRK also attains the best counting performance in terms of both MAE and accuracy, while preserving image quality relative to the base model. Qualitatively, NTRK consistently generates the desired number of objects across diverse categories and target counts, whereas the baselines often generate either fewer or more objects than the target count.

C.4 Additional Application: VLM-Based Reward Alignment In this section, we present qualitative results of applying NTRK to a VLM-based reward application: relative depth and horizon position conditioned image generation. Experiment Setup. We consider the relative depth and horizon position applications introduced in the Dual-Process framework [45]. For relative depth, the task is to enforce a specific depth ordering between two locations, specified by overlaid red dots labeled as Point A and Point B. For horizon position, the objective is to align the scene horizon with a red line overlaid on the image. To enforce these conditions, the generated image containing these visual overlays is fed into a VLM, Qwen2.5-VL [2]. The VLM is then queried with a task-specific instruction prompt (e.g., “Is Point B much closer to the camera than Point A?” or “Is the overlaid red line aligned with the horizon of the scene?”). The reward is computed as the probability of the VLM outputting the desired affirmative answer. Results. We present qualitative results in Figure 17. As shown in these examples, NTRK demonstrates the capability to align generated images with instruction prompts. Specifically, we observe that the generated scenes can successfully place the horizon along the specified red line and adjust object positioning to satisfy the target depth ordering.

42

NoiseTilt: Noise-Tilted Reverse Kernels

Base [35]

BoN [65]

DPS† [10]

SVDD [38]

Ψ-Sampler [75]

NTRK‡ (Ours)

31 (∆6)

37 (∆0)

23 (∆1)

22 (∆0)

3 (∆8)

11 (∆0)

“37 strawberries”

6 (∆31)

20 (∆17)

20 (∆17)

27 (∆10)

“22 grapes”

27 (∆5)

24 (∆2)

33 (∆11)

23 (∆1)

“11 lychees”

3 (∆8)

9 (∆2)

5 (∆6)

12 (∆1)

Figure 16. Qualitative comparison on quantity-aware generation using T2I-Count [49]. For each example, the first row shows generated images and the second row shows the corresponding heatmaps. For single-particle methods, we augment sampling with Best-of-N [65] and Ψ-Sampler [75] to match the total NFE, denoted with † and ‡ , respectively.

43

NoiseTilt: Noise-Tilted Reverse Kernels

Relative Depth Base [35]

NTRK (Ours)

“Is Point A much closer to the camera than Point B ?”

“Is Point B much closer to the camera than Point A ?” Horizon Position Base [35]

NTRK (Ours)

“Does the overlaid red line represent the horizon of the scene?”

“Does the overlaid red line represent the horizon of the scene?” Figure 17. Applications of Dual-Process [45]. Horizon position aims to align the horizontal line with the image, and relative depth to place objects with the given relative depth information. The prompts describe the instructions provided to the VLM [2], and for generation, we use the prompts provided in the Dual-Process [45].

44

NoiseTilt: Noise-Tilted Reverse Kernels

C.5 Additional Results with Different Diffusion Model In this section, we present additional diffusion reward alignment results with a different base model, Z-Image [6]. Experiment Setup. We follow the identical experimental setup described in Section 5.1, with only the base model replaced by Z-Image [6]. Specifically, for aesthetic image generation, we use 45 animal prompts from DDPO [5]. For text-aligned image generation, we use 100 prompts from the complex category of T2I-CompBench++ [22]. We fix the sampling steps to 25 for the base model. To ensure a fair comparison, we fix the total number of function evaluations (NFE) across all baseline methods, augmenting single-particle methods with Best-of-N (BoN) [65] or Ψ-Sampler [75] to match the computational budget. Aesthetic Image Generation. In this task, the target reward is Aesthetic Score [59], and the held-out rewards are ImageReward [73], HPSv2 [72], PickScore [31], and VQA Score [39]. The quantitative results are presented in Table 12. Consistent with our findings using FLUX [35], NTRK achieves the best target reward performance across all methods. Remarkably, NTRK with a single particle and only 25 NFE achieves a higher target reward (7.3500) than the best-performing multi-particle baselines (e.g., DAS [30] and SVDD [38]) evaluated at 500 NFE. When scaled to 500 NFE, NTRK further improves the target reward to 8.7860 and attains the highest scores across most held-out rewards. Text-Aligned Image Generation. For this task, the target reward is PickScore [31], while held-out rewards include VQA Score [39], Aesthetic Score [59], ImageReward [73], and HPSv2 [72]. The quantitative results are summarized in Table 13. Similar to the aesthetic image generation, NTRK consistently outperforms other baselines in target reward performance. With just 25 NFE, NTRK surpasses the performance of all baselines evaluated at 500 NFE. In addition to superior target reward alignment, NTRK generalizes effectively to held-out rewards, attaining the highest ImageReward and HPSv2 scores while remaining comparable in Aesthetic Score and VQA Score. These results confirm that NTRK can be seamlessly applied to different diffusion models without compromising fundamental image quality.

45

NoiseTilt: Noise-Tilted Reverse Kernels

Table 12. Full quantitative comparison on aesthetic image generation with Z-Image [6]. The target reward is Aesthetic Score [59]. For single-particle methods, we augment sampling with Best-of-N [65] and Ψ-Sampler [75] to match the total NFE, denoted with † and ‡ , respectively. Dark green cells indicate the best result for each metric across all runs, while light green cells denote the second best.

Target Reward

Settings

Method

Held-out Rewards

# Particles

# Steps

NFE

Aesthetic Pick ↑ ↑ Score -score

1

25

25

5.7627

0.2206

0.2949

1.2856

0.9708

4 20

25 25

100 500

5.9375 6.1749

0.2208 0.2209

0.2968 0.2962

1.3292 1.3474

0.9707 0.9709

4 20

25 25

100 500

5.9327 6.1587

0.2207 0.2208

0.2963 0.2966

1.3186 1.3784

0.9736 0.9657

3 13

25 25

123 533

5.9502 6.0753

0.2209 0.2203

0.2944 0.2942

1.3189 1.3119

0.9718 0.9678

4 20

25 25

100 500

6.0228 6.2394

0.2208 0.2209

0.2979 0.2945

1.3307 1.2996

0.9665 0.9698

4 20

25 25

100 500

6.0037 6.1600

0.2208 0.2210

0.2961 0.2941

1.3203 1.3486

0.9715 0.9692

4 20

25 25

100 500

5.9745 6.2449

0.2202 0.2214

0.2942 0.2970

1.3035 1.3174

0.9678 0.9673

10

25 + 25

500

6.1078

0.2206

0.2942

1.2843

0.9689

10

25 + 25

500

7.3139

0.2205

0.2975

1.3296

0.9669

1 2 4 20 5

25 50 25 25 100

25 100 100 500 500

7.3500 8.0537 7.5989 7.8527 8.7860

0.2212 0.2204 0.2210 0.2218 0.2208

0.3003 0.2975 0.2996 0.3009 0.2978

1.2781 1.3034 1.3185 1.3701 1.4165

0.9666 0.9706 0.9684 0.9686 0.9738

HPSv2 ↑

Image VQA ↑ ↑ Reward Score

Base [6] BoN [65]

DPS† [10]

FreeDoM† [76]

SVDD [38]

RBF [29]

DAS [30]

Ψ-Sampler [75] NTRK‡ (Ours) NTRK† (Ours)

46

NoiseTilt: Noise-Tilted Reverse Kernels

Table 13. Full quantitative comparison on text-aligned generation with Z-Image [6]. The target reward is PickScore [31]. For single-particle methods, we augment sampling with Best-of-N [65] and Ψ-Sampler [75] to match the total NFE, denoted with † and ‡ , respectively. Dark green cells indicate the best result for each metric across all runs, while light green cells denote the second best.

Target Reward

Settings

Method

Held-out Rewards

# Particles

# Steps

NFE

Pick ↑ -Score

Aesthetic ↑ Score

HPSv2 ↑

1

25

25

0.2116

5.7061

0.2561

0.6567

0.8393

4 20

25 25

100 500

0.2145 0.2162

5.7627 5.7791

0.2627 0.2654

0.7788 0.7522

0.8396 0.8299

4 20

25 25

100 500

0.2145 0.2162

5.7501 5.7422

0.2624 0.2633

0.7747 0.7380

0.8392 0.8385

3 13

25 25

123 533

0.2133 0.2155

5.7087 5.7486

0.2589 0.2625

0.7054 0.6872

0.8317 0.8435

4 20

25 25

100 500

0.2151 0.2181

5.7813 5.7910

0.2621 0.2667

0.7275 0.7893

0.8374 0.8389

4 20

25 25

100 500

0.2144 0.2165

5.7461 5.7778

0.2594 0.2628

0.6940 0.7610

0.8399 0.8445

4 20

25 25

100 500

0.2144 0.2163

5.7360 5.7657

0.2605 0.2656

0.7012 0.8306

0.8426 0.8445

10

25 + 25

500

0.2159

5.8097

0.2639

0.7419

0.8451

10

25 + 25

500

0.2158

5.7939

0.2656

0.7352

0.8336

1 2 4 20 5

25 50 25 25 100

25 100 100 500 500

0.2334 0.2436 0.2361 0.2393 0.2613

5.8487 5.8095 5.8108 5.8092 5.7398

0.2837 0.2907 0.2877 0.2886 0.2984

0.8128 0.8525 0.8224 0.8515 0.8654

0.8423 0.8446 0.8364 0.8405 0.8444

Image VQA ↑ ↑ Reward Score

Base [6] BoN [65]

DPS† [10]

FreeDoM† [76]

SVDD [38]

RBF [29]

DAS [30]

Ψ-Sampler [75] NTRK‡ (Ours) NTRK† (Ours)

47

NoiseTilt: Noise-Tilted Reverse Kernels

C.6 Additional Results of the Main Paper Experiments In this section, we present additional results for the main experiments, including expanded quantitative comparisons across diverse sampling configurations and additional qualitative examples for all three tasks in Section 5. Expanded Quantitative Results. We provide the expanded quantitative results of aesthetic image generation and text-aligned image generation with FLUX [35] in Tables 14 and 15, covering diverse settings with different numbers of particles, sampling steps, and total NFEs. Across both diffusion models and reward alignment tasks, NTRK consistently outperforms the baselines in target reward alignment while achieving competitive or superior held-out reward performance, thereby yielding the most favorable trade-offs. These results further demonstrate that the advantages of NTRK remain robust across different base models and sampling configurations.

48

NoiseTilt: Noise-Tilted Reverse Kernels

Table 14. Full quantitative comparison on aesthetic image generation with FLUX [35]. The target reward is Aesthetic Score [59]. For single-particle methods, we augment sampling with Best-of-N [65] and Ψ-Sampler [75] to match the total NFE, denoted with † and ‡ , respectively. Dark green cells indicate the best result for each metric across all runs, while light green cells denote the second best.

Target Reward

Settings

Method

Held-out Rewards

# Particles

# Steps

NFE

Aesthetic Pick ↑ ↑ Score -score

1

25

25

6.0282

0.2144

0.2759

1.0538

0.9644

4 20

25 25

100 500

6.4694 6.7310

0.2183 0.2197

0.2832 0.2890

0.9583 1.1419

0.9635 0.9597

4 20

25 25

100 500

6.4773 6.7647

0.2182 0.2191

0.2805 0.2861

0.9604 1.0639

0.9634 0.9624

3 13

25 25

123 533

6.5196 6.8406

0.2179 0.2185

0.2797 0.2853

0.8771 0.9941

0.9699 0.9635

4 20

25 25

100 500

6.5348 7.1363

0.2176 0.2177

0.2835 0.2814

1.1208 1.0256

0.9652 0.9510

4 20

25 25

100 500

6.5497 6.9900

0.2173 0.2183

0.2818 0.2826

1.1237 1.0761

0.9695 0.9689

4 20

25 25

100 500

6.5759 6.9384

0.2177 0.2183

0.2821 0.2860

1.0096 1.0568

0.9658 0.9706

10

25 + 25

500

7.0116

0.2188

0.2847

1.1235

0.9737

10

25 + 25

500

7.4180

0.2170

0.2812

1.0209

0.9656

1 1 2 4 5 10 20

25 100 50 25 100 50 25

25 100 100 100 500 500 500

7.4510 8.0538 7.8454 7.7247 8.5394 8.2081 7.9656

0.2200 0.2173 0.2160 0.2176 0.2183 0.2181 0.2197

0.2928 0.2854 0.2831 0.2835 0.2868 0.2913 0.2932

1.2565 1.1546 1.0744 0.9874 1.2613 1.1445 1.1669

0.9728 0.9682 0.9520 0.9648 0.9566 0.9643 0.9609

HPSv2 ↑

Image VQA ↑ ↑ Reward Score

Base [35] BoN [65]

DPS† [10]

FreeDoM† [76]

SVDD [38]

RBF [29]

DAS [30]

Ψ-Sampler [75] NTRK‡ (Ours) NTRK† (Ours)

49

NoiseTilt: Noise-Tilted Reverse Kernels

Table 15. Full quantitative comparison on text-aligned generation with FLUX [35]. The target reward is PickScore [31]. For single-particle methods, we augment sampling with Best-of-N [65] and Ψ-Sampler [75] to match the total NFE, denoted with † and ‡ , respectively. Dark green cells indicate the best result for each metric across all runs, while light green cells denote the second best.

Target Reward

Settings

Method

Held-out Rewards

# Particles

# Steps

NFE

Pick ↑ -Score

Aesthetic ↑ Score

HPSv2 ↑

1

25

25

0.2054

5.4664

0.2316

0.1710

0.8011

4 20

25 25

100 500

0.2103 0.2146

5.6987 5.8582

0.2463 0.2619

0.4272 0.6883

0.7767 0.8021

4 20

25 25

100 500

0.2105 0.2147

5.6926 5.8073

0.2470 0.2622

0.5169 0.6310

0.7926 0.8028

3 13

25 25

123 533

0.2099 0.2133

5.8158 5.8492

0.2458 0.2572

0.3275 0.5354

0.8041 0.7990

4 20

25 25

100 500

0.2143 0.2204

5.6745 5.8743

0.2587 0.2699

0.5536 0.7592

0.8102 0.8201

4 20

25 25

100 500

0.2150 0.2202

5.6813 5.8618

0.2603 0.2682

0.5993 0.7583

0.8031 0.8149

4 20

25 25

100 500

0.2104 0.2139

5.6601 5.8385

0.2457 0.2568

0.4134 0.5226

0.8064 0.7990

10

25 + 25

500

0.2120

5.7329

0.2551

0.4590

0.8145

10

25 + 25

500

0.2133

5.8495

0.2594

0.6226

0.8198

1 1 2 4 5 10 20

25 100 50 25 100 50 25

25 100 100 100 500 500 500

0.2224 0.2363 0.2317 0.2282 0.2439 0.2385 0.2327

5.7720 5.8397 5.8097 5.7478 5.8016 5.7252 5.9020

0.2601 0.2743 0.2720 0.2700 0.2830 0.2796 0.2817

0.5257 0.5971 0.5918 0.6399 0.6771 0.6739 0.7370

0.8210 0.8177 0.8100 0.7881 0.8205 0.8081 0.8017

Image VQA ↑ ↑ Reward Score

Base [35] BoN [65]

DPS† [10]

FreeDoM† [76]

SVDD [38]

RBF [29]

DAS [30]

Ψ-Sampler [75] NTRK‡ (Ours) NTRK† (Ours)

50

NoiseTilt: Noise-Tilted Reverse Kernels

Additional Qualitative Results. We provide additional qualitative results for aesthetic image generation, text-aligned image generation and preference-aligned video generation in Figure 18, Figure 19 and Figure 20, respectively. Across all applications, NTRK generates images and videos that better reflect the target reward, while maintaining strong visual quality and fidelity to the input text compared to the baselines. Base [35]

BoN [65]

DPS† [10]

SVDD [38]

Ψ-Sampler [75]

NTRK† (Ours)

7.5315

6.9614

7.9347

7.3300

7.0763

7.9429

"dolphin"

5.8065

6.7978

6.7873 "hedgehog"

5.8580

6.5064

6.7468

Figure 18. Additional qualitative comparison on aesthetic image generation using Aesthetic Score [59]. For single-particle methods, we augment sampling with Best-of-N to match the total NFE, denoted with † .

Base [35]

BoN [65]

DPS† [10]

SVDD [38]

Ψ-Sampler [75]

NTRK† (Ours)

“The intricate, colorful patterns of the mandala radiated outward from the center, forming a dazzling and hypnotic display.”

0.2126

0.2175

0.2140

0.2199

0.2116

0.2299

“The tall, slender birch trees swayed gently in the cool autumn breeze, their golden leaves rustling softly.”

0.2004

0.2106

0.2111

0.2231

0.2104

0.2403

Figure 19. Additional qualitative comparison on text-aligned image generation using PickScore [31]. For single-particle methods we augment sampling with Best-of-N to match the total NFE, denoted with † .

51

NoiseTilt: Noise-Tilted Reverse Kernels

“A sheep eating yellow flowers from behind a wire fence.”

“A harbour seal swimming near the shore.”

Base [70]

DPS [10]

FreeDoM [76]

NTRK (Ours)

Figure 20. Additional qualitative comparison on preference-aligned video generation using VideoReward [42]. NTRK produces videos with better text alignment, visual quality and motion quality.

52