Assessing Sample Quality in Conditional Generation under Compositional Shift

Unknown

Conceptio › Archive › arXiv CS

arXiv CSopen access

Assessing Sample Quality in Conditional Generation under Compositional Shift

Unknown · 2026 · arxiv_cs

arXiv CS · Papers · License: Open Access · 2026

Open Source ↗Direct PDF ↓

machine learning, deep learning, neural networks

Assessing Sample Quality in Conditional Generation under Compositional Shift

Berker Demirel1 , Valentino Maiorca1 , Marco Fumero1 , Theofanis Karaletsos2,3 , Francesco Locatello1

arXiv:2606.09601v1 [cs.LG] 8 Jun 2026

Institute of Science and Technology Austria (ISTA), 2 Pyramidal Inc, San Francisco, CA, USA, 3 Achira Inc, San Francisco, CA, USA

Abstract Conditional generators provide a natural tool for controllable generation, including settings where the desired condition is a new composition of observed attributes or experimental factors. In many applications, especially in scientific domains, such models are attractive to explore conditions for which real samples are rare, expensive, or not yet observed. However, this creates a circularity for evaluation: standard conditional quality metrics require a reference target distribution, but in the extrapolative regime that distribution is unavailable by definition. We address this problem with a post-hoc, per-sample trust score for assessing conditional samples using only the training distribution. The score combines two estimable quantities: global realism, measuring compatibility with the real data manifold, and attributewise faithfulness, measuring whether a sample is closer to the requested attributes than to plausible alternatives. We show that the score can recover meaningful comparisons across extrapolated generations, under a mild coverage condition on the observed attributes. These comparisons enable effective filtering, ranking, and abstention of generations and can be used directly on off-the-shelf pretrained models. In biological imaging, selected samples preserve real morphological structure better and improve downstream predictive performance, while similar gains are observed on controlled vision benchmarks. Finally, we show how the score can be applied during generation, enabling abstention before full decoding. Code is available at https://github.com/berkerdemirel/faithful-cond-gen.

Introduction

Conditional diffusion models can generate samples for user-specified conditions, including class labels [7, 29], attribute tuples [5] and biological perturbations [22]. This is increasingly appealing in scientific settings, where synthetic data are useful not because they reproduce what has already been measured, but because they may help explore conditions that are rare, expensive, or not yet observed [33, 2, 10]. For example, if we want to achieve certain phenotypic properties in a cell population, we would like to prioritize wet-lab testing of perturbations that are most likely to produce the desired outcome. A generative model can support this process by simulating experiments in silico before laboratory validation. The challenge is determining if these predictions can be trusted, as they must be both visually realistic and faithful to the requested condition, despite the target population being unseen. Evaluating conditional generation faces a circularity issue: standard distributional metrics such as Fréchet Inception Distance (FID) [11] and Kernel Inception Distance (KID) [3], along with their conditional variants, score generated samples by comparison with real samples from the target distribution. Yet conditional generators are often most useful precisely in settings where such target samples are unavailable. In this extrapolative regime, target-distribution metrics can still serve as Preprint.

Compositional setting

Measure

attributes → joint conditions

observed

attribute 2

b tri

0.29 ✓keep

F faithfulness closer to requested attributes?

extractor

requested a*

candidate x̂

T=R+F Ranked by T (low = trusted)

0.08 ✓keep

training distribution

attribute 1

Gθ

x̂

request unseen a*

filter, rank, abstain

R realism close to training distribution?

unseen target

Trust

realism + faithfulness

0.62 ~ borderline threshold

1.79 ✗reject

conditional generator

attribute 1

attribute 2 attribute 3

2.93 ✗reject

Figure 1: Pipeline for trust scoring under compositional shift. A conditional diffusion model Gθ is queried with an unseen joint condition a⋆ and produces candidate samples x̂. Features from Φ are used to compute a realism term R, which measures proximity to the real training distribution, and a faithfulness term F , which measures alignment with the requested attribute values. Their sum T = R + F provides a calibrated trust score used to rank, filter, or abstain from generated samples. controlled validation tools, but they cannot decide whether an individual generated sample should be trusted at deployment time. We therefore ask a different question: what sample quality information is identifiable from the real training distribution alone? We focus on two real-data-calibrated quantities that remain observable. A sample should be globally realistic, meaning compatible with the feature geometry of real data, and attribute-wise faithful, meaning closer to the requested attribute values than to competing values. These quantities capture separable failure modes: a sample may be realistic but wrong for the requested condition, or condition-consistent but outside the real-data manifold. Existing post-hoc reliability tools typically capture only one side of this split: out-of-distribution and one-class-style scores measure global compatibility with an in-distribution set [32, 10], while prompt- or classifierbased alignment scores measure agreement with a requested condition [20, 17, 27]. Our method combines both views in a post-hoc trust score computed in the latent space of an auxiliary feature extractor, using only the generated sample and real training-set statistics. The key idea is attribute-level decomposition: instead of matching a sample to an unseen condition as a whole, we evaluate each constituent attribute independently, checking whether its requested value is favored over plausible alternatives in observed real-data contexts. Under this coverage condition, we show that attribute-wise decisions remain identifiable even when the full joint target is not. This formalizes both what the score can recover and what remains fundamentally unidentifiable without target samples. Most notably, we demonstrate that the same real-data geometry allows our score to be evaluated before generation is complete. By aligning the diffusion model to the feature extractor space, we are able to evaluate the same trust score directly in representation space, during generation. This avoids the computational cost of repeated forward passes through the feature extractor and enables low-quality samples to be discarded before full generation. Our contributions can be summarized as follows: • Reference-free trust score. We recast per-sample quality assessment for conditional generators as a problem solvable without observing the joint target conditioning. Decomposing it into two geometric quantities, global realism and attribute-wise faithfulness, yields our post-hoc trust score computable from a pretrained generator, an encoder, and training-set statistics. • Attribute-level identifiability. We show that meaningful attribute-level comparison scores can be identified even when the full joint target is not. • Empirical validation. On RxRx1 and CelebA our score reliably filters, ranks, and curates samples under support shift under both KID-based and downstream task-based validations. • Early sample rejection. We show how the score can be lifted from a post-hoc method to the latent representations inside the generator itself via a simple geometric alignment objective, enabling quality assessment and abstention before decoding. 2

Related Work

Image quality metrics. There is active literature evaluating generative models by comparing real and generated sample sets in a feature space, including FID [11], KID [3], density/coverage [21], probabilistic precision/recall [25, 30, 13], as well as sample-level metrics closer to our setting [1]. However, these metrics require a collection of real reference samples from the target distribution being evaluated. In contrast, our method combines capabilities that no other method offers as a set: it is per-sample, post-hoc (i.e., after generation, and therefore, model agnostic), and most importantly, it is designed for compositional generalization, where no real samples from the requested target condition are available. Compositional generalization in conditional diffusion. A growing line of work studies whether conditional diffusion models can generate unseen compositions of attributes or concepts. The closest connection to our setting is [23], which analyzes when compositional abilities emerge in a synthetic discrete conditional-generation task. Related work studies the emergence of hidden compositional capabilities in concept space [24], and analyzes hierarchical compositional generalization in diffusion models through context-free grammars [8]. Other work modifies training objectives to encourage logical or compositional generation under partial support [9]. Our focus is complementary. We do not try to improve the generator or characterize when compositional generalization emerges. Instead, we ask how to evaluate and rank individual generated samples after training, especially when the requested joint condition has no real target reference set. This shifts the question from whether a model can sometimes extrapolate to whether a particular extrapolated sample should be trusted. OOD detection and support scoring. Out-of-distribution detection and feature-space support estimation provide another line of related work, but they are primarily developed for discriminative prediction rather than as trust mechanisms for synthetic generations. Mahalanobis-based detectors measure compatibility with an in-distribution feature geometry [15, 20], distance-based methods provide nonparametric alternatives [32, 28, 26, 16], and energy-based methods score whether an input lies in a high-confidence region of a learned representation space [17]. We repurpose [15] to build a trust mechanism for conditional diffusion models. For realism, we ask whether a sample looks like real data from the domain; for faithfulness, we ask whether, within a real-data-calibrated geometry, it is closer to the requested attribute value than to its competitors. This complements unconditional support scoring by incorporating the requested conditioning, even when the requested joint target is unavailable. Faithfulness and alignment scores for generated data. Conditional faithfulness is often evaluated with discriminative surrogates. In compositional-generation studies, probes can test whether generated samples contain requested attributes [23]. In text-to-image settings, CLIP-style alignment is widely used as a proxy for prompt adherence, including when assessing whether synthetic images are useful for recognition tasks [10]. Recent work has also moved alignment scoring into the denoising process, evaluating prompt–latent agreement before a final image is decoded [27]. These approaches are useful discriminative tests of whether a generated sample expresses a requested label, prompt, or attribute, but they are not directly image-quality metrics: an image can be classified as conditionconsistent while remaining unrealistic or far from the real data geometry. Our goal is to identify what conditional quality remains estimable from the training distribution alone when the requested joint target is unavailable. This yields a decomposition into global realism and reference-anchored per-attribute faithfulness, where faithfulness is not a standalone classifier score but a conditional component of a real-data-calibrated quality score.

Method

We propose a post-hoc per-sample trust score for conditional generation when the requested joint condition may be absent from the real training support. The score is computed w.r.t. any individually generated sample, relying only on the training data and a pretrained feature extractor Φ. It combines two measurable quantities: global realism (i.e., compatibility with the real-data feature geometry), and attribute-wise faithfulness, (i.e., agreement with each requested attribute value relative to competing ones). While the proposed score is applicable to any pretrained conditional generative model, in the following we assume a latent conditional diffusion model [29]. Figure 1 illustrates our pipeline. 3

Generative setting. Let A1 , . . . , AK be finite attribute-value spaces, and let a⋆ = (a⋆1 , . . . , a⋆K ) ∈ A1 × · · · × AK denote the requested joint condition. We consider a latent diffusion model Gθ with latent space Z, input space X , decoder D : Z → X , and denoising network ϵθ . Starting from Gaussian noise zT ∈ Z, the reverse process conditioned on a⋆ produces a final latent ẑ0 (a⋆ ) ∈ Z, which is decoded into the generated sample x̂(a⋆ ) = D(ẑ0 (a⋆ )) ∈ X . The joint condition a⋆ may appear in training or may lie under compositional support shift, where the specific tuple is unseen but its individual attributes appear in other real training samples. Let Φ : X → Rd denote the feature extractor. We score samples using normalized features y(x) = Φ(x)/∥Φ(x)∥2 . All means, covariance estimates, precision matrices, and calibration constants below are fit only on normalized real training features. Global realism. Let µreal and Σreal be the empirical mean and regularized covariance of real normalized features. We define the global Mahalanobis energy Ereal (y) = (y − µreal )⊤ Σ−1 real (y − µreal ), and standardize it as R(y) = (Ereal (y) − mR )/sR , where mR and sR are the mean and standard deviation of Ereal on real calibration features. Therefore, larger R indicates lower realism. Attribute-wise faithfulness. For each attribute k and value v ∈ Ak , define the real-data prototype ηk,v = E[y | ak = v], estimated by averaging the normalized features y(x) of all real training samples whose k-th attribute value is v. For each attribute k, we estimate a shared precision matrix Pk ≻ 0 from the pooled within-value covariance of real samples and define dk (y; v) = (y − ηk,v )⊤ Pk (y − ηk,v ). For requested value t ∈ Ak , the target-vs-competitor margin is Mk (y; t) = dk (y; t) − min dk (y; v). v̸=t

Thus Mk (y; t) < 0 means that y is closer to the requested value than to every competing value of attribute k. We standardize each margin on real samples with ak = t: Fk (y; t) = (Mk (y; t) − mk,t )/sk,t , where mk,t and sk,t are the corresponding real calibration mean and standard deviation. PK 1 ⋆ The total faithfulness score is F (y; a⋆ ) = K k=1 Fk (y; ak ). Larger F indicates weaker agreement with the requested attributes. The final trust score is T (y; a⋆ ) = R(y) + F (y; a⋆ ). The decomposition is interpretable: R detects samples that are globally far from the real feature distribution, while F detects samples that are realistic but mismatched to the requested attributes. Estimation details are given in Section C. 3.1

When can we assess sample quality in practice?

We remark that the same score T can be used both after generation and during generation; only the way we obtain the Φ-compatible features y changes. Figure 5 in Appendix A illustrates the difference. After generation, we decode the image and apply the feature extractor: Φ(D(ẑ0 (a⋆ ))) . y post = ∥Φ(D(ẑ0 (a⋆ )))∥2 We then evaluate T (y post ; a⋆ ). This directly scores the generated image in the feature-space geometry, but requires the VAE decoder and a Φ pass. Post-generation scoring.

During-generation scoring. While generating, we skip decoding and feature extractor encoding by mapping internal denoising representation into a Φ-compatible space. Let rℓ,τ (a⋆ ) ∈ Rdℓ denote the representation extracted at layer ℓ and denoising step τ along the inference trajectory conditioned on a⋆ . We learn a translator, parametrized by a shallow MLP, gϕ,ℓ : Rdℓ → Rd to match the pairs rℓ,τ (xi , ai ), y(xi ) obtained from real training samples xi with observed attribute condition ai . gϕ,ℓ (rℓ,τ (a⋆ )) map yℓ,τ = . ∥gϕ,ℓ (rℓ,τ (a⋆ ))∥2 map ⋆ At inference time, we evaluate T (yℓ,τ ; a ). Thus post-generation and during-generation scoring use the same real-data-calibrated metric; they differ only in whether the feature is obtained from the decoded image or from the denoising trajectory. 4

Online abstention. The mapped score enables early rejection during denoising. Given a layer-stepspecific threshold κℓ,τ , we abstain when map ⋆ T (yℓ,τ ; a ) > κℓ,τ .

This rejects samples predicted to have low trust before completing the full denoising trajectory, decoding, and post-generation encoder pass. Details of the translator training and compute accounting are given in Section F.

Theory: Identifiability under missing target distributions

If the requested condition a⋆ is missing from the real data, then the behavior of y under that condition cannot be estimated directly from the data. Proposition 1 (Informal: non-identifiability of missing targets). Let S ⊆ A1 × · · · × AK be the observed support of the conditioning variable a, and let a⋆ ∈ / S. From the observed joint distribution of (y, a) restricted to S, the conditional distribution of y at a⋆ is not identifiable without assumptions relating observed and unobserved conditions. Therefore, no post-hoc score using only the observed real distribution can certify full conditional fidelity to the missing target distribution. This motivates a weaker target: attribute-level comparisons. Instead of asking whether a sample matches the unobserved joint condition a⋆ , we ask whether it is closer to each requested attribute value than to competing values. Definition 1 (Reference coverage). Let a−k denote all attributes except ak . The observed support has reference coverage if there exists a reference condition ā such that, for every attribute k and value v ∈ Ak , P(ak = v, a−k = ā−k ) > 0. Reference coverage provides just enough conditioning variation to isolate attribute effects without relying on structural assumptions. When attributes only appear in fixed combinations, their effects are support-confounded: the data cannot tell which attribute accounts for an observed change. Reference coverage avoids this ambiguity while using the smallest possible set of observed joint conditions. We formalize this minimality result and give a support-confounding example in Sections B.1 and B.2. Moreover, under reference coverage, we can identify a fixed-context attribute comparator. Define ref the reference prototype µref k,v := E[y | ak = v, a−k = ā−k ]. For a fixed Pk ≻ 0, define dk (y; v) := ref ⊤ ref (y − µk,v ) Pk (y − µk,v ), and for target value t, ref Mkref (y; t) := dref k (y; t) − min dk (y; v). v̸=t

A negative margin means that y is closer to the requested value t than to every competitor for attribute k, in the reference context. Proposition 2 (Point identification of the reference-anchored comparator). Assume reference coverref ref age, let y be integrable, and fix Pk ≻ 0. Then µref k,v , dk (y; v), and Mk (y; t) are point-identified from the observed joint distribution of (y, a). See Section B.3 for the proof. Thus, missing target samples prevent full conditional evaluation, but not all attribute-level evaluation. Reference coverage identifies an anchored comparator. Remark: The implemented score uses pooled prototypes ηk,v := E[y | ak = v] rather than µref k,v , because pooling uses all real samples with ak = v and is more sample-efficient. Section B.4 gives sufficient conditions under which the pooled comparator makes the same attribute-wise decisions as the reference-anchored comparator, and Section B.6 reports the corresponding empirical agreement. Taken together, these results clarify what incomplete conditional support can and cannot identify. Missing joint targets rule out certification of full conditional fidelity without additional assumptions. Reference coverage provides a cardinality-minimal support condition for unconfounded attribute-level comparisons, under which the reference-anchored comparator is point-identified. The implemented pooled score is a sample-efficient surrogate: the appendix gives sufficient conditions for agreement with the identified comparator and empirically verifies this agreement. Additionally, Section B.9 shows that the comparator can still be evaluated in the full feature space without requiring a disentangled subspace, as the decisions depend only on prototype-difference discriminant directions. 5

Experiments

We evaluate the trust score in the regime it is designed for: the scorer is calibrated on real trainingsupport features and never uses real samples from the requested target condition at deployment time. See Section E for the implementation details. Datasets and shift regimes. We choose the fluorescent microscopy images dataset RxRx1 [33] as the main scientific benchmark. It contains 4 cell types and 1138 siRNA perturbations, yielding 4552 cell-type×perturbation attribute combinations (i.e.,, conditions). After filtering by support to obtain enough real samples for reliable evaluation, we obtain a test set of 25 conditions unseen at training time and 25 seen ones; full construction details are in Section H. For a more controlled setting we used a subset of CelebA [18] which is a compositional testbed with 4 binary attributes (Male, Smiling, Blond_Hair, Eyeglasses), yielding 16 conditions. In this specific setting, training sees only the all-zero reference and the four single-attribute conditions, while all 16 conditions are queried at generation time. Results for models trained with the full conditional coverage are reported in Section L. Models and scoring spaces. For each dataset and support regime, we train a vanilla conditional diffusion model (SiT-B/2 [19]) and two REPA [34] variants aligned to DINOv3 [31] or SigLIP [35]. The vanilla model is the basic deployment target of the trust score, while the REPA variants test transfer across diffusion models and provide a natural baseline for model-internal scoring due to their alignment training objective (Section 5.2). Post-generation trust scores are computed in the feature extractor representation space: DINOv3 for vanilla models and the corresponding teacher representation for REPA models. During generation, scores are computed using a translator that aligns the diffusion model features at a given layer, to SigLIP features. Additionally, we show a negative result using OpenPhenom [12] features, due to representation collapse in Section G Validation metrics. Generation quality is measured via KID, which assumes access to the real samples from the unseen target conditioning as ground truth, as opposed to our trust score. ∆KID = KID(gen, realA ) − KID(realB , realA ), always evaluated in a fixed DINOv3 mean-patch feature space. On RxRx1, we additionally validate trust selection in CellProfiler (CP) [4] morphology space, which provides a biologically interpretable per-cell measurements independent of DINOv3. 5.1

Post-generation trust as data curation

We first evaluate the score in the post-generation setting. The central question is whether a score calibrated only on real training-support data can curate generated samples for requested conditions whose target distribution is unavailable at scoring time. 5.1.1

Real-data-calibrated filtering and condition ranking

Following the convention commonly used in the out-of-distribution detection literature with the FPR95 metric [32, 6, 20, 17], we instantiate the abstention rule from Section 3.1 by setting the acceptance threshold at the 95th percentile of real-sample trust scores and comparing the accepted subset against a condition-matched random subset of the same size. Table 1 compares the ∆KID of selected generated samples based on trust score (denoted as ∆KIDtrust ) against a conditionmatched random subset of generated samples of the same size (denoted as ∆KIDbaseline ). It reports the percentage of generated samples that pass the FP95-real trust threshold, and the per-condition Spearman correlation between mean trust score and ∆KID. The trust score consistently selects better samples under support shift. On CelebA, the selected subsets improve ∆KID by 39–44% while retaining roughly 55–59% of generated samples. On RxRx1, selection is much more stringent, accepting only 4–6% of samples, but the ∆KID improvement remains strong at 39–42%. Beyond sample selection, the same table shows that per-condition mean trust provides a strong ranking of condition difficulty in both domains. Accordingly, we report the full realism/faithfulness decomposition in Section K and defer the head-to-head scorer ablation to Section L. Takeaway. Our score provides a practical support-shift filter and a useful condition-level ordering without using target real samples at scoring time. 6

Dataset

Model

Accept%

∆KIDtrust ↓

∆KIDbaseline ↓

∆%↑

ρ(T )↑

CelebA

Vanilla REPA (DINOv3) REPA (SigLIP)

58.7 55.4 56.1

0.224±.009 0.228±.010 0.239±.018

0.368±.020 0.401±.031 0.423±.020

+39.1 +43.1 +43.6

0.96 0.96 0.96

RxRx1

Vanilla REPA (DINOv3) REPA (SigLIP)

4.4 5.5 6.1

0.196±.006 0.197±.006 0.181±.003

0.323±.013 0.333±.019 0.312±.019

+39.4 +40.8 +42.0

0.90 0.90 0.87

1.2

16-way accuracy

KID (lower = better)

Table 1: Support-shift post-generation results. P95-real-threshold sample selection uses a threshold set at the 95th percentile of real-sample trust scores; ∆KID is measured in DINOv3 space and compared to a condition-matched random subset. ∆% is the relative ∆KID improvement of trustselected over random baseline (positive is better). ρ(T ) is the condition-level Spearman correlation between mean trust and ∆KID. Component correlations and full-support rows are in Section K.

1.0 0.8 0.6 0.4

0.60

0.55

Trust

0.50 0

Bin index (0=best, 9=worst)

(a) CelebA ∆KID by trust decile.

(b) CelebA downstream 16-way accuracy.

Figure 2: CelebA decile binning (REPA-DINOv3 held-out, DINOv3 scoring). ∆KID increases monotonically from bin 0 (best trust) to bin 9 (worst) (left), and correlates with downstream classification accuracy drops (right). Binning results with faithfulness and realism components, together with RxRx1 DINOv3 decile curves are reported in Section K

5.1.2

Trust rankings track sample quality and downstream utility

We next show how our score can be employed as a full ranker, rather than a filterer. We partition generated samples into class-balanced trust deciles, train a condition classifier on each decile separately, and evaluate on held-out real data. Figure 2 shows the controlled CelebA held-out worse trust deciles have worse ∆KID and yield less useful training data for downstream condition classification. On CelebA, ∆KID degrades by roughly 4× from the best to worst decile, while downstream accuracy decreases by about 10% when training on low-quality samples according to trust. This confirms that the score is ordinally meaningful in a setting where the support shift is controlled by design. In the more realistic scientific setting, the analogous RxRx1 DINOv3-space decile curves show the same ordering for KID and downstream classification; we report them in Figure 9 of Section K. Takeaway. Across all tested settings, trust score induces a sample-level ordering that tracks both image quality and downstream utility, rather than simply serving as a thresholding rule. 5.1.3

Biological validation with CellProfiler morphology

To test whether sample selection based on trust score captures biologically meaningful morphology, we evaluate RxRx1 generations in CellProfiler [4] (CP) feature space: interpretable per-cell measurements computed directly from segmented images. We use 621 CP features after variance thresholding, and a real-data |z| ≤ 5 outlier filter, then standardize on real data. The full CP pipeline is in Section I. CP-space decile utility. We repeat the decile downstream experiment of Figure 10a, on RxRx1, using CP feature space as opposed to DINOv3. Figure 3 shows results with the REPA-SigLIP diffusion model: better trust deciles yield better classifiers for both cell type classification (4 classes) 7

Accuracy (micro)

0.90 0.88 0.86 0.84 0.82

0.14 0.13 0.12 0.11

trust 0.80 0

Decile bin (0 = best trust)

Figure 3: Main RxRx1 CellProfiler validation (REPA-SigLIP marginal, SigLIP trust scoring). CP-space downstream classification by trust decile. Left: 4-way cell-type accuracy. Right: 50-way condition accuracy. Trust-ranked deciles show a clear correlation with the classification performance, showing that trust ordering improves utility in an interpretable morphology space independent of the DINOv3 validation encoder. and cell-type×perturbation targets (50 classes in total) show a similar trend. The same pattern holds for all three marginal generators and both targets in Section I. Real-spread-normalized centroid distance. For each generated sample at condition (c, s), we compute its CellProfiler distance to the real-condition centroid and standardize it by real-real variation: z(x) =

∥CP(x) − µreal,c,s ∥2 − µRR (c, s) . σRR (c, s)

Here µRR (c, s) and σRR (c, s) are estimated from random half-splits of real samples within the same condition. Thus z = 0 corresponds to typical real-real morphology spread, while z > 0 indicates excess deviation from the real morphology centroid. Table 2 shows that trust selection improves CellProfiler morphology alignment in all six model/split settings, with bootstrap confidence intervals strictly below zero (Section I, Table 11). The seen split is nearly real-like after selection: REPA-DINOv3 trust-selected samples are only 0.024σ beyond typical real morphology. On unseen conditions, generated samples drift farther from the real distribution, but trust selection still removes 18–29% of the standardized excess distance. We also provide a per-feature CP breakdown in Section I, where gains concentrate on biologically meaningful texture and shape axes: per-cell texture autocorrelation, image-level granularity, total cell-segmentation area, and cytoplasm shape regularity.

Table 2: Real-spread-normalized CP morphology distance on RxRx1 (kept-621 CP features). z(x) is the distance to the matched real-condition centroid, standardized by its real-real morphology variation. Accept% is the share of generations passing the P95-real threshold. Bootstrap 95% CIs are in Table 11. Model

Split

Accept%

z̄trust ↓

z̄baseline ↓

∆%↑

Vanilla Vanilla REPA (DINOv3) REPA (DINOv3) REPA (SigLIP) REPA (SigLIP)

seen unseen seen unseen seen unseen

66.3 21.9 61.2 21.0 65.7 23.5

0.055 0.583 0.024 0.535 0.032 0.827

0.111 0.712 0.137 0.755 0.126 1.020

+50

+18 Takeaway. On RxRx1, T -selected +83 samples are closer to real ones +29 in both calibrated CellProfiler +75 morphology space and under the +19 learned DINOv3 validation encoder. Since the metric accounts for within-condition real-real variation, gains directly reflect reductions in excess morphology deviation beyond normal biological spread.

5.2

During generation

In this section, we investigate the extent to which the score T can be evaluated at generation time before decoding so that low confidence images are never generated, directly mapping diffusion-model representations into the feature extractor space (Section 3.1). We train a lightweight translator for this mapping, with an objective that preserves the shared-covariance Mahalanobis geometry the score relies on, allowing internal-feature trust scoring without the VAE decoder or feature extractor. 8

CelebA

Accept%

∆KIDtrust ↓

∆KIDbaseline ↓

∆%↑

ρ(T )↑

REPA REPA translator translator translator

REPA (DINOv3) REPA (SigLIP) Vanilla REPA (DINOv3) REPA (SigLIP)

62.7 66.7 23.6 48.4 17.2

0.278±.015 0.293±.024 0.195±.008 0.250±.016 0.308±.015

0.404±.021 0.425±.018 0.378±.023 0.397±.017 0.409±.015

+31.2 +31.1 +48.4 +37.2 +24.8

0.95 0.95 0.83 0.92 0.87

REPA REPA translator translator translator

REPA (DINOv3) REPA (SigLIP) Vanilla REPA (DINOv3) REPA (SigLIP)

1.4 3.9 38.4 62.3 60.5

0.537±.032 0.564±.022 0.260±.003 0.246±.003 0.235±.011

0.329±.037 0.293±.023 0.346±.016 0.372±.037 0.329±.018

−63.4 −92.3 +24.9 +33.8 +28.7

0.09 0.13 0.78 0.88 0.70

Figure 4 evaluates the translator at intermediate denoising steps on CelebA held-out (250-step sampler). At k = 83 the filter already reaches 33% ∆KID improvement while saving 67% of denoising compute; by k ≈ 166 it matches the postgeneration oracle while still saving 34%. RxRx1 figure, full-support rows, decile-binning confirmation, and per-step numbers are in Section L.3, Section L.1, Section K, and Section L.2.

oracle +39.3%

100

150

200

250

per-step image ∆L2

Model

∆KID% vs. random

Feature source

RxRx1

Table 3: Trust scoring during generation. Same setup as Table 1 but the score here is evaluated from internal features mapped by a translator instead of post-generation SigLIP ones. The joint-aligned REPA baseline partly works on CelebA but fails on RxRx1; our translator restores filtering and ranking on both.

denoising step k (of 250) ∆KID% (left)

image ∆L2 (right)

Table 3 reports the support-shift held-out result. The translator gives 25–34% ∆KID improvements Figure 4: Translator scoring across denoison RxRx1 with ρ(T ) up to 0.88, and matches ing (CelebA, Vanilla SiT-B/2, 250 steps). P95post-generation filtering on CelebA. Here, real-threshold ∆KID improvement rises as the REPA-aligned features serve as a natural baseline: predicted-clean trajectory settles. REPA training aligns internal states to a pretrained teacher as part of the generator’s loss, so any REPA-trained generator provides a translator at no additional cost. This baseline partly preserves the signal on CelebA (ρ(T ) = 0.95, +31% ∆KID) but fails on RxRx1, with selection worse than random and ρ(T ) ≈ 0.Cosine distance, as used in REPA, aligns internal states to the extractor features without preserving the covariance structure needed by the Mahalanobis trust score. Thus, REPA features are not a reliable geometry for trust scoring. Takeaway. Preserving the trust geometry during generation enables early scoring; the same map further allows the score to be evaluated well before final decoding, increasing computational benefits.

Conclusion

In this work, we studied per-sample quality assessment for conditional generation under compositional shift, where real samples from the requested target condition are unavailable. We showed that full conditional fidelity to a missing target distribution is not identifiable from the observed training distribution alone, but that global realism and attribute-wise faithfulness remain estimable. We introduced a post-hoc trust score combining a pooled Mahalanobis realism term with shared-covariance attribute margins. Under reference coverage, reference-anchored attribute comparisons are identifiable. Empirically, the score supports filtering, ranking, and synthetic-data curation, improving KID, downstream utility, and RxRx1 CellProfiler morphology alignment. The same real-data geometry can also be used during generation by mapping denoising states into the pretrained-encoder feature space, enabling early abstention when the trust geometry is preserved. A limitation of the current theory is that it focuses on discrete attributes under sufficient coverage conditions. Future work should extend the framework to continuous conditioning variables, characterize weaker identifiability conditions, and use the score to drift generations toward higher-trust regions rather than only abstaining from poor ones. 9

Acknowledgements This work was supported by the Chan Zuckerberg Initiative (CZI) through its AI Residency Program. We are grateful to CZI for the opportunity to take part in the program, and to the CZI AI Infrastructure Team for providing support with the GPU cluster used to train our models. MF is supported by the MSCA IST-Bridge fellowship which has received funding from the European Union’s Horizon 2020 research and innovation program under the Marie Skłodowska-Curie grant agreement No 101034413.

References [1] Ahmed Alaa, Boris Van Breugel, Evgeny S Saveliev, and Mihaela Van Der Schaar. How faithful is your synthetic data? sample-level metrics for evaluating and auditing generative models. In International conference on machine learning, pages 290–306. PMLR, 2022. [2] Shekoofeh Azizi, Simon Kornblith, Chitwan Saharia, Mohammad Norouzi, and David J Fleet. Synthetic data from diffusion models improves ImageNet classification. Transactions on Machine Learning Research, 2023. [3] Mikołaj Bińkowski, Danica J Sutherland, Michael Arbel, and Arthur Gretton. Demystifying MMD GANs. In International Conference on Learning Representations, 2018. [4] Anne E Carpenter, Thouis R Jones, Michael R Lamprecht, Colin Clarke, In Han Kang, Ola Friman, David A Guertin, Joo Han Chang, Robert A Lindquist, Jason Moffat, et al. Cellprofiler: image analysis software for identifying and quantifying cell phenotypes. Genome biology, 7:R100, 2006. [5] Berker Demirel, Marco Fumero, Theofanis Karaletsos, and Francesco Locatello. Morphgen: Controllable and morphologically plausible generative cell-imaging. arXiv preprint arXiv:2510.01298, 2025. [6] Berker Demirel, Marco Fumero, and Francesco Locatello. Out-of-distribution detection with relative angles. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025. [7] Prafulla Dhariwal and Alexander Nichol. Diffusion models beat GANs on image synthesis. In Advances in Neural Information Processing Systems, 2021. [8] Alessandro Favero, Antonio Sclocchi, Francesco Cagnetta, Pascal Frossard, and Matthieu Wyart. How compositional generalization and creativity improve as diffusion models are trained. arXiv preprint arXiv:2502.12089, 2025. [9] Sachit Gaudi, Gautam Sreekumar, and Vishnu Boddeti. Coind: Enabling logical compositions in diffusion models. In The Thirteenth International Conference on Learning Representations, 2025. [10] Ruifei He, Shuyang Sun, Xin Yu, Chuhui Xue, Wenqing Zhang, Philip Torr, Song Bai, and Xiaojuan Qi. Is synthetic data from generative models ready for image recognition? In International Conference on Learning Representations, 2023. [11] Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. GANs trained by a two time-scale update rule converge to a local nash equilibrium. In Advances in Neural Information Processing Systems, 2017. [12] Oren Kraus, Kian Kenyon-Dean, Saber Saberian, Maryam Fallah, Peter McLean, Jess Leung, Vasudev Sharma, Ayla Khan, Jia Balakrishnan, Safiye Celik, et al. Masked autoencoders for microscopy are scalable learners of cellular biology. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11757–11768, 2024. [13] Tuomas Kynkäänniemi, Tero Karras, Samuli Laine, Jaakko Lehtinen, and Timo Aila. Improved precision and recall metric for assessing generative models. Advances in neural information processing systems, 32, 2019. 10

[14] Olivier Ledoit and Michael Wolf. A well-conditioned estimator for large-dimensional covariance matrices. Journal of Multivariate Analysis, 88(2):365–411, 2004. [15] Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin. A simple unified framework for detecting out-of-distribution samples and adversarial attacks. Advances in neural information processing systems, 31, 2018. [16] Litian Liu and Yao Qin. Fast decision boundary based out-of-distribution detector. In Forty-first International Conference on Machine Learning, 2024. [17] Weitang Liu, Xiaoyun Wang, John Owens, and Yixuan Li. Energy-based out-of-distribution detection. Advances in neural information processing systems, 33:21464–21475, 2020. [18] Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In International Conference on Computer Vision, 2015. [19] Nanye Ma, Mark Goldstein, Michael S Albergo, Julius Berner, and Yaron Lipman. Sit: Exploring flow and diffusion-based generative models with scalable interpolant transformers. In International Conference on Machine Learning, 2024. [20] Maximilian Müller and Matthias Hein. Mahalanobis++: Improving OOD detection via feature normalization. In Forty-second International Conference on Machine Learning, 2025. [21] Muhammad Ferjad Naeem, Seong Joon Oh, Youngjung Uh, Yunjey Choi, and Jaejun Yoo. Reliable fidelity and diversity metrics for generative models. In International Conference on Machine Learning, 2020. [22] Zeinab Navidi, Jun Ma, Esteban Miglietta, Le Liu, Anne E Carpenter, Beth A Cimini, Benjamin Haibe-Kains, and Bo Wang. Morphodiff: Cellular morphology painting with diffusion models. In The Thirteenth International Conference on Learning Representations, 2025. [23] Maya Okawa, Ekdeep Singh Lubana, Robert P. Dick, and Hidenori Tanaka. Compositional abilities emerge multiplicatively: Exploring diffusion models on a synthetic task. In Thirtyseventh Conference on Neural Information Processing Systems, 2023. [24] Core F Park, Maya Okawa, Andrew Lee, Hidenori Tanaka, and Ekdeep S Lubana. Emergence of hidden capabilities: Exploring learning dynamics in concept space. Advances in Neural Information Processing Systems, 37:84698–84729, 2024. [25] Dogyun Park and Suhyun Kim. Probabilistic precision and recall towards reliable evaluation of generative models. In Proceedings of the IEEE/CVF international conference on computer vision, pages 20099–20109, 2023. [26] Jaewoo Park, Yoon Gyo Jung, and Andrew Beng Jin Teoh. Nearest neighbor guidance for out-of-distribution detection. In Proceedings of the IEEE/CVF international conference on computer vision, pages 1686–1695, 2023. [27] Vasco Ramos, Regev Cohen, Idan Szpektor, and Joao Magalhaes. Beyond the noise: Aligning prompts with latent representations in diffusion models. arXiv preprint arXiv:2512.08505, 2025. [28] Jie Ren, Stanislav Fort, Jeremiah Liu, Abhijit Guha Roy, Shreyas Padhy, and Balaji Lakshminarayanan. A simple fix to mahalanobis distance for improving near-ood detection. arXiv preprint arXiv:2106.09022, 2021. [29] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. Highresolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. [30] Mehdi SM Sajjadi, Olivier Bachem, Mario Lucic, Olivier Bousquet, and Sylvain Gelly. Assessing generative models via precision and recall. Advances in neural information processing systems, 31, 2018. 11

[31] Oriane Siméoni, Huy V Vo, Maximilian Seitzer, Federico Baldassarre, Maxime Oquab, Cijo Jose, Vasil Khalidov, Marc Szafraniec, Seungeun Yi, Michaël Ramamonjisoa, et al. Dinov3. arXiv preprint arXiv:2508.10104, 2025. [32] Yiyou Sun, Yifei Ming, Xiaojin Zhu, and Yixuan Li. Out-of-distribution detection with deep nearest neighbors. In International conference on machine learning, pages 20827–20840. PMLR, 2022. [33] Maciej Sypetkowski, Morteza Rezanejad, Saber Saberian, Oren Kraus, John Urbanik, James Taylor, Ben Mabey, Mason Victors, Jason Yosinski, Alborz Rezazadeh Sereshkeh, et al. Rxrx1: A dataset for evaluating experimental batch correction methods. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 4285–4294, 2023. [34] Sihyun Yu, Sangkyung Kwak, Huiwon Jang, Jongheon Jeong, Jonathan Huang, Jinwoo Shin, and Saining Xie. Representation alignment for generation: Training diffusion transformers is easier than you think. In The Thirteenth International Conference on Learning Representations, 2025. [35] Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training. In Proceedings of the IEEE/CVF international conference on computer vision, pages 11975–11986, 2023.

Model Pipeline

Figure 5 contrasts post-generation and during-generation trust scoring. Post-generation scoring first completes denoising, decodes the final latent into an image, and applies the feature extractor Φ before evaluating the trust score. During-generation scoring instead maps an intermediate diffusion representation ht into the same Φ-compatible space using a learned translator T , bypassing decoding and feature extraction. Thus, both settings use the same calibrated trust geometry, while duringgeneration scoring enables assessment before full generation is completed. A

Post-generation

denoise → decode → extract Φ feature space

diffusion model zT unseen

z0 t=T

t=0

VAE decoder

image x̂

network layers

Φ(x̂)

training features

During-generation

fix diffusion layer → translator → bypass Φ feature space

diffusion model zT unseen

t=T

translator T

t=0

bypasses chain

network layers

T(ht)

training features

Figure 5: Post-generation and during-generation trust scoring. In the post-generation setting, the diffusion model is run to completion, the final latent is decoded into an image, and the feature extractor Φ maps the generated sample into the calibrated feature space where the trust score is evaluated. In the during-generation setting, scoring bypasses decoding and feature extraction: an intermediate denoising representation ht from a fixed diffusion layer, and selected on a possibly early diffusion timestep t, is mapped by a learned translator T into the same Φ-compatible feature space. Both pipelines therefore use the same real-data-calibrated trust geometry, but during-generation scoring enables early assessment and abstention before full generation is completed.

Additional theoretical details

B.1

Minimality for unconfounded attribute coverage

Proposition 3 (Reference coverage is minimal). Let S ⊆ A1 × · · · × AK be an observed support. Define the one-attribute contrast graph GS to have vertices a ∈ S, with an edge between two vertices if they differ in exactly one attribute. Without structural assumptions on E[y | a], suppose S permits unconfounded identification of every attribute value effect, in the sense that GS contains a connected subgraph whose vertices cover every value in each Ak . Then |S| ≥ 1 +

K X

(|Ak | − 1) = Nref .

k=1

Reference coverage attains this bound, since it contains exactly Nref joint conditionings. Hence reference coverage is cardinality-minimal. Proof of Proposition 3. Let m(a) := E[y | a] denote the conditional mean at attribute configuration a. With no structural assumptions, m(a) is an arbitrary function of the full joint attribute configuration. 13

A model-free contrast can be attributed to a single attribute only when the two conditions being compared differ in that attribute alone. If two observed conditions differ in multiple attributes, then the resulting mean difference can be decomposed in multiple ways among the changed attributes and their interactions. Thus, without structural assumptions, such a contrast is support confounded. Define the one-attribute contrast graph GS of an observed support S ⊆ A1 × · · · × AK as follows. The vertices are the observed joint conditions a ∈ S. Two vertices are connected by an edge labeled k if they differ only in coordinate k. Such an edge gives an unconfounded contrast for attribute k. Suppose an observed support identifies unconfounded effects for all attribute values. Then its oneattribute contrast graph must contain a connected subgraph whose vertices cover every attribute value: for every attribute k and every value v ∈ Ak , some vertex in the subgraph must satisfy ak = v. Otherwise, some value would either be unobserved in any one-attribute comparison or would be observed only through joint changes with other attributes, leaving its individual contribution non-identifiable. Now consider any connected subgraph T of GS that covers all attribute values. Let R be a spanning tree of T , and choose an arbitrary root condition a(0) . The root covers exactly one value of each attribute, hence K attribute values in total. Each additional vertex in the tree differs from its parent in exactly one coordinate, so it can introduce at most one previously unseen attribute value. The total number of attribute values that must be covered is K X

|Ak |.

k=1

After the root, the number of remaining attribute values that must be introduced is therefore K X

|Ak | − K =

k=1

K X

(|Ak | − 1).

k=1

Since each new observed joint condition can introduce at most one such value, any support that identifies all attribute values through unconfounded one-attribute contrasts must contain at least 1+

K X

(|Ak | − 1) = Nref

k=1

observed joint conditions. Reference coverage attains this lower bound. Indeed, for a reference condition ā, it observes the baseline condition ā and, for every attribute k and every non-reference value v ̸= āk , the one-attribute deviation (v, ā−k ). Thus it contains exactly K X 1+ (|Ak | − 1) = Nref k=1

joint conditions. These conditions form a star-shaped one-attribute contrast graph centered at ā, and the graph covers every attribute value. Therefore reference coverage is cardinality-minimal. This minimal design is not unique: other supports with Nref conditions may also cover all attribute values through unconfounded one-attribute contrasts. Reference coverage is the shared-context construction that attains the minimum. B.2

Support confounding without reference coverage

Proposition 4 (Support confounding destroys attribute attribution). There exist observed supports for which attribute-specific contributions are not determined by the observed distribution. In particular, with four binary attributes and observed support S = {0000, 1111}, the observed data distribution determines only the joint contrast between 0000 and 1111; it does not determine any unique decomposition of that contrast into attribute-specific contributions. 14

Proof of Proposition 4. It suffices to consider a scalar feature coordinate. Let the observed support be S = {0000, 1111}, and write m0 := E[y | a = 0000], m1 := E[y | a = 1111]. The observed data distribution determines m0 and m1 , hence only the joint difference m1 − m0 . Now consider an additive representation on the observed support, E[y | a] = β0 +

4 X

gj (aj ).

j=1

On the two observed cells, this implies m1 − m0 =

4 X

gj (1) − gj (0) .

j=1

Only the sum is determined; the individual differences are not. For example, the following two choices agree on the observed support but assign different attribute-1 contributions. Model A.

Set g1 (1) − g1 (0) = m1 − m0 ,

Model B.

Set gj (1) − gj (0) =

gj (1) − gj (0) = 0

m1 − m 0 4

(j = 2, 3, 4).

(j = 1, 2, 3, 4).

Both models reproduce exactly the same observed conditional means at 0000 and 1111, but they attribute different portions of the observed change to attribute 1. Hence the observed distribution on S does not determine any unique attribute-specific contribution. This is the support confounding claimed in the proposition. B.3

Identification and well-definedness of anchored objects

We collect here the basic support-side consequences used in the main text. Proof of Proposition 2. Fix an attribute k and values u, v ∈ Ak . Under reference coverage, P(ak = v, a−k = ā−k ) > 0

and

P(ak = u, a−k = ā−k ) > 0.

and

µref k,u = E[y | ak = u, a−k = ā−k ]

Hence the conditional expectations µref k,v = E[y | ak = v, a−k = ā−k ]

are functions of the observed joint distribution of (y, a) on positive-probability events, and are therefore point-identified whenever y is integrable. Their difference ref ref ∆ref k (v, u) = µk,v − µk,u

is then point-identified as well. Since Pk is fixed, the oracle distances are deterministic functions of the identified prototypes. The oracle margin is a finite minimum over identified distances, and is therefore also identified. Proposition 5 (Well-definedness under reference coverage). Assume the support has reference coverage with reference condition ā. Then for every attribute k ∈ {1, . . . , K} and every value v ∈ Ak , P(ak = v, a−k = ā−k ) > 0 and P(ak = v) > 0. Consequently, whenever y is integrable, both µref k,v := E[y | ak = v, a−k = ā−k ] 15

and

ηk,v := E[y | ak = v]

are well-defined. If in addition Pk ≻ 0, then for every y ∈ Rd , ref ⊤ ref dref k (y; v) := (y − µk,v ) Pk (y − µk,v ), and the corresponding margins

dk (y; v) := (y − ηk,v )⊤ Pk (y − ηk,v )

ref Mkref (y; t) := dref k (y; t) − min dk (y; v),

Mk (y; t) := dk (y; t) − min dk (y; v)

v̸=t

are well-defined for all targets t ∈ Ak . Proof. Reference coverage states directly that P(ak = v, a−k = ā−k ) > 0 for every k and v. Marginalizing over a−k then gives P(ak = v) ≥ P(ak = v, a−k = ā−k ) > 0. Hence both conditioning events have positive probability. Since y is integrable, the conditional ref expectations defining µref k,v and ηk,v are well-defined. If Pk ≻ 0, then the quadratic forms dk (y; v) d and dk (y; v) are finite for every y ∈ R , and since Ak is finite, the minima over v ̸= t are attained and finite. Therefore the margins are well-defined. Remark 1 (Identification versus estimation). Reference coverage is a support-side condition ensuring that the fixed-context comparison µref k,v is identified and well-defined. It does not by itself guarantee low-variance estimation in finite samples. Finite-sample issues enter through how many real samples populate each required slice, and are separate from the population-level argument. B.4

Pooled and reference prototypes

The main identifiability result concerns the reference-anchored prototypes µref k,v . The implemented score uses pooled prototypes ηk,v because they are more sample-efficient: they use all real samples with ak = v, rather than only samples in the anchored cell (ak = v, a−k = ā−k ). This appendix gives sufficient conditions under which this lower-variance pooled comparator agrees with the reference-anchored comparator. Conditional-mean decomposition and pooling assumptions. The pooled prototype for attribute k and value v is ηk,v := E[y | ak = v]. For the pooled-reference analysis, write the conditional mean as E[y | ak = v, a−k = b] = sk,v + ck (b) + δk,v (b), where sk,v is the value-associated signal, ck (b) is the remaining-context contribution, and δk,v (b) is a value-dependent interaction residual. This decomposition is only an analysis device; the score does not require estimating these terms. We use the following two assumptions: E[ck (a−k ) | ak = v] = c̄k for all v ∈ Ak , (A1) and E[δk,v (a−k ) | ak = v] = 0 for all v ∈ Ak . (A2) A1 is the substantive context-balance condition: after pooling over observed contexts, the average non-k context contribution does not depend on the value of attribute k. A2 is a centering convention rather than an additional support assumption: any value-specific average interaction bias can be absorbed into sk,v . Taking expectation over a−k | ak = v, we obtain ηk,v = sk,v + E[ck (a−k ) | ak = v] + E[δk,v (a−k ) | ak = v]. Under A1–A2, this reduces to ηk,v = sk,v + c̄k . At the reference context, µref k,v = sk,v + ck (ā−k ) + δk,v (ā−k ). Thus pooled prototypes share the same value signal sk,v as the reference prototypes, but differ by a shared context shift and a value-dependent reference leakage term. The next lemma makes this relation explicit. 16

Lemma 1 (Pooled-reference decomposition). Under A1–A2, ref ref ηk,v = µref k,v + γk − ek,v ,

where

γkref = c̄k − ck (ā−k ),

eref k,v = δk,v (ā−k ).

Proof. By the conditional-mean decomposition, ηk,v = sk,v + E[ck (a−k ) | ak = v] + E[δk,v (a−k ) | ak = v]. Using A1–A2 gives ηk,v = sk,v + c̄k . At the reference context,

µref k,v = sk,v + ck (ā−k ) + δk,v (ā−k ).

Subtracting yields ηk,v = µref k,v + c̄k − ck (ā−k ) − δk,v (ā−k ).

B.5

Pooled-reference perturbation bound

Proposition 6 (Pooled margin perturbation, value-specific split form). Assume A1–A2 and let ỹ = y − γkref . Suppose that for each v ∈ Ak ,

ref ∥eref k,v ∥Pk ≤ εk,v .

Define ref ρref k,v (y) := ∥ỹ − µk,v ∥Pk ,

ref ref ref 2 bref k,v (y) := 2εk,v ρk,v (y) + (εk,v ) .

For a target value t, define ref ref Bk,split (y; t) := bref k,t (y) + max bk,v (y). v̸=t

Then

ref Mk (y; t) − Mkref (ỹ; t) ≤ Bk,split (y; t).

Proof. By Lemma 1, ref y − ηk,v = ỹ − µref k,v + ek,v .

Therefore

ref 2 dk (y; v) = ∥ỹ − µref k,v + ek,v ∥Pk ,

while

ref 2 dref k (ỹ; v) = ∥ỹ − µk,v ∥Pk .

Define

∆v := dk (y; v) − dref k (ỹ; v).

Expanding the square gives ref ref 2 ∆v = 2 ỹ − µref k,v , ek,v P + ∥ek,v ∥Pk . k

By Cauchy–Schwarz and the value-specific leakage bound, ref ref 2 ref |∆v | ≤ 2εref k,v ∥ỹ − µk,v ∥Pk + (εk,v ) = bk,v (y).

Therefore

|∆t | ≤ bref k,t (y).

For the competitor term, using min av − min bv ≤ max |av − bv |, v̸=t

v̸=t

we obtain ref min dk (y; v) − min dref k (ỹ; v) ≤ max bk,v (y). v̸=t

v̸=t

Adding target and competitor errors gives ref ref Mk (y; t) − Mkref (ỹ; t) ≤ bref k,t (y) + max bk,v (y) = Bk,split (y; t). v̸=t

Corollary 1 (Buffered pooled-comparator decision equivalence). Under the assumptions of Proposition 6, ref Mkref (ỹ; t) > Bk,split (y; t) =⇒ sign Mk (y; t) = sign Mkref (ỹ; t). The same conclusion holds with Mk (y; t) and Mkref (ỹ; t) interchanged. Proof. Proposition 6 gives ref Mk (y; t) − Mkref (ỹ; t) ≤ Bk,split (y; t). ref Thus if either margin has magnitude larger than Bk,split (y; t), the other margin has the same sign. This proves the claim.

B.6

Empirical decision agreement (Corollary 1)

The value-specific split certificate is ref Bk,split (y; t) = bk,t (y) + max bk,v (y),

ref ref 2 bk,v (y) = 2εref k,v ∥ỹ − µk,v ∥Pk + (εk,v ) .

v̸=t

When

ref |Mkref (ỹ; t)| > Bk,split (y; t), Corollary 1 certifies that the pooled and reference-anchored margin signs agree.

We instantiate this diagnostic on held-out real samples from both datasets, where the referenceanchored comparator can be evaluated. For each attribute k, we estimate ηk,v , Pk , and µref k,v using the same calibration protocol as the trust-score experiments, and evaluate on the canonical held-out evaluation splits. For each sample, let a⋆k (y) denote the target value. We compare the deployed pooled margin Mk (y; a⋆k ) = dk (y; a⋆k ) − min⋆ dk (y; v) v̸=ak

with the corresponding reference-anchored margin ⋆ ref Mkref (ỹ; a⋆k ) = dref k (ỹ; ak ) − min⋆ dk (ỹ; v). v̸=ak

The certificate used in Table 4 is ref Bk,split (y; a⋆k ) = bk,a⋆k (y) + max⋆ bk,v (y). v̸=ak

Certified agreement. Across every row with nonzero certified support, the certified agreement rate is 1.000. This is the empirical behavior predicted by Corollary 1: whenever the reference margin exceeds the split-radius perturbation band, the pooled and reference-anchored decisions have matching signs. The certificate is strongest for CelebA Male, where 86.2% of samples are certified, and for RxRx1 cell type, where 42.1% are certified. Agreement beyond the certified regime. The certificate is sufficient, not necessary. Thus low certified coverage does not imply poor pooled/reference agreement. On CelebA, agreement remains very high for all attributes, ranging from 0.990 to 1.000, even when the certified fraction is small. RxRx1 cell type also has high agreement, 0.997, with substantial certified coverage. The main failure case is the finer-grained RxRx1 sirna decision: the median reference margin is only 1.2, while the median split bound is 16.6, so no samples are certified and overall agreement drops to 0.803. 18

ref Table 4: Empirical pooled/reference sign certification using the split bound Bk,split (y; t) = bk,t (y) + ref ref ref 2 maxv̸=t bk,v (y), bk,v (y) = 2εk,v ∥ỹ − µk,v ∥Pk + (εk,v ) . med Bsplit is the median certificate ref radius, and Pr[certified] is the fraction of samples satisfying |Mkref (ỹ; t)| > Bk,split (y; t). Agree(all) and Agree(cert.) report sign agreement before and after certification.

Dataset Attribute CelebA CelebA CelebA CelebA RxRx1 RxRx1

Male Smiling Blond_Hair Eyeglasses cell_type_id sirna_id

Neval

162770 2 162770 2 162770 2 162770 2 758 4 640 20

εref |M ref |med med Bsplit Pr[certified] Agree(all) Agree(cert.) k 0.79 0.70 0.78 0.73 0.79 1.04

74.2 22.6 21.6 13.3 14.0 1.2

53.0 44.8 49.7 46.0 15.2 16.6

0.862 0.007 0.008 < 0.001 0.421 0.000

1.000 0.998 0.994 0.990 0.997 0.803

1.000 1.000 1.000 1.000 1.000 —

Why certification is hard in low-margin regimes. The split bound is tighter than the earlier one-radius bound, but it can still be conservative when the reference margin is small relative to the leakage-weighted prototype radii. This is most visible for Smiling, Blond_Hair, and Eyeglasses, where empirical agreement is high but few samples have margins large enough to clear the certificate. In the RxRx1 sirna setting, the issue is more severe: many sirna prototypes are close in the reference geometry, so the deployed margin is often small compared with the perturbation band. This is exactly the regime where the theory predicts pooled scoring should be least reference-anchored comparator-like. B.7

Empirical A1 context-balance diagnostic

A1 (E[ck (a−k ) | ak = v] = c̄k ) is the load-bearing condition that lets pooled scoring stand in for the ref ref reference-anchored comparator. By Lemma 1, under A1+A2 the difference ηk,v − µref k,v = γk − ek,v ref ref is v-independent up to the leakage eref k,v = δk,v (ā−k ). We therefore use εk = maxv ∥ek,v ∥Pk as an ref operational A1-violation scale and compare it to the inter-prototype reference scale ∥µk,u − µref k,t ∥Pk , which estimates the value-signal gap ∥sk,u − sk,t ∥Pk in Lemma 1. A ratio ≪ 1 means that any A1 violation is small relative to the attribute signal in the reference geometry; a ratio ≳ 1 indicates that leakage is large enough to obscure at least one value-pair comparison. Estimation reuses exactly the fitting and evaluation pools of Table 4; Table 5 reports the resulting per-attribute leakage-to-gap ratios. Table 5: Empirical A1 diagnostic. “min gap” and “median gap” are the minimum and median ref pairwise Pk -Mahalanobis distances ∥µref k,u − µk,t ∥Pk over u ̸= t; Smaller ε/gap indicates greater compatibility with the A1 regime. Bold flags the regime where leakage exceeds the closest value-pair gap. Dataset

Attribute

εref k

min gap

median gap

ε/min

ε/median

CelebA CelebA CelebA CelebA

Male Smiling Blond_Hair Eyeglasses

2 2 2 2

0.79 0.70 0.78 0.73

8.81 4.99 4.86 3.84

0.090 0.141 0.160 0.190

RxRx1 RxRx1

cell_type_id sirna_id

4 20

0.79 1.04

3.96 0.90

5.28 1.64

0.199 1.157

0.149 0.635

Small-leakage regimes. On all four CelebA attributes, the leakage-to-gap ratio is below 0.20. RxRx1 cell type is also in a small-leakage regime, with ε/median gap = 0.149 and ε/min gap = 0.199. These are precisely the settings where empirical pooled-vs-comparator sign agreement is near-perfect: agreement ranges from 0.990 to 1.000 on CelebA and is 0.997 on RxRx1 cell type (Table 4). Large-leakage regime. The only regime where leakage exceeds the closest value-pair gap is RxRx1 sirna, with ε/min gap = 1.157. Its median leakage-to-gap ratio remains below one, ε/median gap = 19

0.635, but the closest sirna prototypes are sufficiently close that the perturbation band dominates many margins. This matches Table 4: RxRx1 sirna has median reference margin 1.2, median split certificate radius 16.6, no certified samples, and substantially lower pooled/reference agreement 0.803. Takeaway. The A1 diagnostic aligns with the empirical certification results. Small leakage-to-gap ratios coincide with high pooled/reference agreement and, for CelebA Male and RxRx1 cell type, substantial certified coverage. The only ratio exceeding one occurs for the closest RxRx1 sirna pair, exactly where certification fails and empirical agreement is weakest. Thus A1 is not merely a formal assumption: its violation scale is measurable and predicts when pooled scoring is a reliable reference-comparator stand-in. B.8

Exact translation equivalence

Corollary 2 (Exact translation equivalence). Suppose ηk,v = µref k,v + γ

for all v ∈ Ak .

Then, for every y and every target t, Mk (y; t) = Mkref (y − γ; t). Proof. For every v,

y − ηk,v = y − γ − µref k,v .

Therefore

dk (y; v) = dref k (y − γ; v) for every v. Taking the target distance minus the minimum competitor distance gives the result. B.9

Full-feature Mahalanobis discriminants

Proposition 7 (Pairwise decisions depend only on discriminant directions). Fix an attribute k, a positive definite matrix Pk , and prototypes {mk,v : v ∈ Ak } ⊂ Rd . Define (m)

dk (y; v) := (y − mk,v )⊤ Pk (y − mk,v ). For any two values t, u ∈ Ak , (m)

(m)

⊤ dk (y; t) − dk (y; u) = 2(mk,u − mk,t )⊤ Pk y + m⊤ k,t Pk mk,t − mk,u Pk mk,u .

Thus the sign of the pairwise preference between t and u depends on y only through the scalar projection ⊤ (Pk (mk,u − mk,t )) y. Equivalently, only the component of y along the span of the pairwise discriminant directions span {Pk (mk,u − mk,t ) : u ̸= t} can affect the target-vs-competitor comparison. Proof of Proposition 7. Fix an attribute k, prototypes {mk,v : v ∈ Ak }, and a positive definite matrix Pk . For any two values t, u ∈ Ak , expand the two Mahalanobis distances: (m)

⊤ dk (y; t) = (y − mk,t )⊤ Pk (y − mk,t ) = y ⊤ Pk y − 2m⊤ k,t Pk y + mk,t Pk mk,t ,

and (m)

⊤ dk (y; u) = (y − mk,u )⊤ Pk (y − mk,u ) = y ⊤ Pk y − 2m⊤ k,u Pk y + mk,u Pk mk,u .

Subtracting cancels the common quadratic term y ⊤ Pk y, giving (m)

(m)

⊤ dk (y; t) − dk (y; u) = 2(mk,u − mk,t )⊤ Pk y + m⊤ k,t Pk mk,t − mk,u Pk mk,u .

Thus the pairwise decision between t and u depends on y only through ⊤

(Pk (mk,u − mk,t )) y, using symmetry of Pk . Therefore components of y orthogonal to the span of the discriminant directions span {Pk (mk,u − mk,t ) : u, t ∈ Ak , u ̸= t} cannot affect any pairwise Mahalanobis decision for attribute k. This proves the proposition. B.10

Robustness details

This subsection records an additional sufficient condition under which residual context variation does not change the attribute-wise Mahalanobis decision. It is not needed for the pooled-reference identification result above, but helps explain why full-feature scoring can remain stable when residual variation is small in the relevant discriminant directions. Proposition 8 (Robustness to residual variation in discriminant directions). Fix an attribute k, a target value t ∈ Ak , and suppose the ideal pooled prototypes satisfy ηk,v = c̄k + sk,v . Let a generated sample targeted at value t have normalized feature y = c̄k + sk,t + λ, d

where λ ∈ R collects residual context dependence, imperfect disentanglement, and noise. Then for every competitor u ̸= t, dk (y; t) − dk (y; u) = 2(sk,u − sk,t )⊤ Pk λ − ∥sk,u − sk,t ∥2Pk , where ∥z∥2Pk := z ⊤ Pk z. Consequently, if 2 max (sk,u − sk,t )⊤ Pk λ < min ∥sk,u − sk,t ∥2Pk , u̸=t

u̸=t

then Mk (y; t) < 0. Proof. Since ηk,t = c̄k + sk,t , we have y − ηk,t = c̄k + sk,t + λ − (c̄k + sk,t ) = λ, and therefore

dk (y; t) = λ⊤ Pk λ.

For a competitor u ̸= t, y − ηk,u = c̄k + sk,t + λ − (c̄k + sk,u ) = sk,t + λ − sk,u . Hence dk (y; u) = (sk,t + λ − sk,u )⊤ Pk (sk,t + λ − sk,u ) = λ⊤ Pk λ − 2(sk,u − sk,t )⊤ Pk λ + ∥sk,u − sk,t ∥2Pk . Subtracting gives dk (y; t) − dk (y; u) = 2(sk,u − sk,t )⊤ Pk λ − ∥sk,u − sk,t ∥2Pk . If

2 (sk,u − sk,t )⊤ Pk λ < ∥sk,u − sk,t ∥2Pk for every u ̸= t, then each pairwise difference dk (y; t) − dk (y; u) is strictly negative. Therefore the target value t beats every competitor under the Mahalanobis comparison, so Mk (y; t) = dk (y; t) − min dk (y; u) < 0. u̸=t

Estimation details

This appendix records the finite-sample estimators used to instantiate the population-level objects from the main text. C.1

Calibration details for score normalization constants

Let Dreal denote the real-data split used to fit and calibrate the score model, and let Dcal ⊆ Dreal denote the subset used for calibration. In practice, Dcal may be a held-out calibration split or the same real-data split used for fitting when data is limited. For realism, after fitting (µreal , Σreal ), we compute breal (yi ) := (yi − µ b −1 (yi − µ E breal )⊤ Σ breal ) real

for yi ∈ Dcal ,

and estimate m̂R :=

X 1 breal (yi ), E |Dcal |

s ŝR :=

yi ∈Dcal

X 1 |Dcal | − 1

breal (yi ) − m̂R 2 . E

yi ∈Dcal

The calibrated realism score is then breal (y) − m̂R E b R(y) := , ŝR + εR where εR > 0 is a small numerical stabilizer. For faithfulness, fix an attribute k and target value t ∈ Ak . For each calibration sample yi satisfying ai,k = t, compute the fitted margin ck (yi ; t) := dbk (yi ; t) − min dbk (yi ; v). M v̸=t

Then estimate m̂k,t :=

X 1 ck (yi ; t), M cal | |Dk,t cal yi ∈Dk,t

and

v u ŝk,t := u t

X 1 ck (yi ; t) − m̂k,t 2 , M cal | − 1 |Dk,t cal yi ∈Dk,t

where

cal Dk,t := {yi ∈ Dcal : ai,k = t}. The calibrated attribute score becomes

ck (y; t) − m̂k,t M Fbk (y; t) := , ŝk,t + εF with εF > 0 a small numerical stabilizer. In practice, if a class-conditional calibration set is very small, one may use a minimum-count threshold before reporting calibrated scores, pool variance estimates across values of the same attribute, or clip the denominator away from zero. These are finite-sample stabilization choices and do not affect the population-level theory above. C.2

Shared-covariance estimation and shrinkage

For each attribute k, let ηbk,v =

nk,v i:a

i,k =v

denote the empirical prototype for value v ∈ Ak , where nk,v is the number of real samples with ai,k = v. 22

The pooled within-value covariance is b within := Σ k

X X 1 (yi − ηbk,v )(yi − ηbk,v )⊤ , Nk − |Ak | i:a =v v∈Ak

where Nk =

i,k

v∈Ak nk,v .

To improve conditioning in high dimension, we use shrinkage toward an isotropic target: b shr b within + αk τk I, Σ k := (1 − αk )Σk

τk :=

1 b within tr Σ , k d

with shrinkage coefficient αk ∈ [0, 1]. The corresponding precision estimate is b shr + εP I −1 , Pbk := Σ k where εP > 0 is a small numerical ridge added for inversion stability. The resulting fitted quadratic distance is dbk (y; v) := (y − ηbk,v )⊤ Pbk (y − ηbk,v ). The same construction can be used for the global realism model by replacing the class-conditional pooled covariance with the empirical covariance of all real normalized features. C.3

Finite-sample considerations

Several finite-sample issues are worth noting. First, reference coverage is a population support condition. In finite data, the corresponding anchored slices may exist but contain few examples. This is one reason we use pooled prototypes in the practical method even when the theory is stated relative to a reference-anchored comparator. Second, class imbalance affects both prototype estimation and calibration. For heavily imbalanced attributes, the empirical prototypes ηbk,v and calibration moments (m̂k,t , ŝk,t ) may have different variances across values. This motivates reporting class counts and, if needed, applying minimumsupport thresholds or pooled-variance stabilizers. Finally, the shrinkage level αk trades bias for conditioning. Stronger shrinkage produces more stable inverses in high dimension and small sample regimes, while weaker shrinkage preserves finer geometry when sufficient data are available.

Broader impacts

Positive impacts. The proposed trust score is intended to make conditional generative models more deployable in scientific settings where target real samples are unavailable, particularly biological imaging where wet-lab validation of in silico predictions is expensive. By providing a per-sample, per-condition reliability estimate, the score can prevent unreliable synthetic generations from being treated as evidence in downstream analyses, reducing the risk of misleading scientific conclusions and saving experimental resources by prioritizing high-trust predictions for validation. Potential negative impacts. Any post-hoc filtering rule can be misused to cherry-pick generations that look plausible without being faithful to the requested condition — our score mitigates but does not eliminate this risk, and we encourage users to report acceptance rates and per-condition trust calibration alongside any selected sample sets. Dual-use considerations. The score itself is a diagnostic and does not improve the generative capabilities of the underlying model; it does not lower the barrier to producing higher-quality synthetic faces or biological images relative to running the underlying generator alone. 23

Implementation details

Architecture. SiT-B/2 [19] backbone (12 layers, 768 hidden dim, 12 heads), patch size 2 over a 32×32 VAE latent (256 patches), stable-diffusion SD-VAE-MSE encoder/decoder [29] (frozen). Conditioning is per-attribute embedding summed with the timestep embedding. The REPA projector is a 3-layer MLP 768 → 2048 → 2048 → dh inserted at encoder depth ℓ = 8, where dh ∈ {1024, 1152, 384} for DINOv3 / SigLIP / OpenPhenom respectively. Training. AdamW with lr = 10−4 , weight decay 0, 1000 linear warmup steps, gradient clip 1.0, 400,000 optimizer steps, global batch size 128 (16 per GPU × 8 GPUs, DDP, bf16 mixed precision), EMA on the generator with decay 0.9999. Class-conditional dropout p = 0.1 for classifier-freeguidance training; for marginal/held-out runs we additionally use the compositional regularizer with weight 1.0 applied to 25% of batches at t > 0.7. Data and preprocessing. CelebA: 256×256 centre-cropped RGB, random horizontal flip, [−1, 1] normalization; conditioning on the 4 binary attributes M ALE / S MILING / B LOND _H AIR / E YE GLASSES . RxRx1: 512×512 6-channel raw fluorescence resized to 256×256 via numpy bilinear, no flips, no per-channel normalization (raw uint16 scaled to [0, 1]); conditioning on (cell_type_id, sirna_id). Sampling. 250-step DDIM-style integration of the SiT flow with tcutoff = 0.04; classifier-freeguidance scale 1.0. We draw 1000 samples per condition for CelebA (16 conditions) and 100 per condition for RxRx1 (50 conditions in the canonical subset). Trust-score calibration. Encoder features are taken as the mean over patch tokens (DINOv3 / SigLIP) or as the model’s pooled output (OpenPhenom). Covariance estimates use Ledoit– Wolf shrinkage [14] with εP = 10−5 ridge added before inversion. Calibration constants (m̂R , ŝR , m̂k,t , ŝk,t ) are estimated on the same real training-support split used for fitting; full formulas are in Section C.

During-generation translator and compute details

The translator is trained to preserve the aspects of the feature extractor representation used by the Mahalanobis trust score. A standard cosine-alignment loss is sufficient to align dominant directions, but it need not preserve the residual geometry that controls whitened distances and prototype comparisons. This can distort the relative scale of directions that are weak in cosine similarity but important after covariance normalization. We therefore train gϕ,ℓ with a whitenedgeometry objective: a tempered feature-matching term in feature-whitened coordinates, together with mean and covariance matching penalties in normalized feature space. The whitening transform is computed from the real training features, using the same covariance geometry as the trust score. This makes the mapped features compatible with the post-generation scoring rule, so that any difference between post-generation and during-generation evaluation reflects feature acquisition rather than a change in metric. Specifically, we train the translator from an internal model feature zℓ,τ (x) to the Φ feature space. Let pϕ (x) = gϕ,ℓ (zℓ,τ (x)),

p̄ϕ (x) = norm(pϕ (x)),

ȳ(x) = norm(Φ(x)).

Let µt and Σt denote the empirical mean and covariance of normalized pretrained features on real training data. We use h i ∥µp − µt ∥22 ∥Σp − Σt ∥2F 2 Lwhitgeom = λwhit Ex ∥W (p̄ϕ (x) − ȳ(x))∥2 + λgeom + , ∥µt ∥22 + ϵ ∥Σt ∥2F + ϵ where µp and Σp are the batch mean and covariance of mapped normalized features. The matrix W is a tempered whitening transform computed from the feature covariance, with exponent γ = 0.75 and condition-number cap κ = 1000. In all experiments we use λwhit = 1.0,

λgeom = 0.1. 24

Table 6: Proposition 8 robustness margin test across encoders. Violation rate = fraction of samples where noise exceeds signal in the Mahalanobis margin. N/S = noise-to-signal ratio. cell_type attribute Encoder DINOv3 SigLIP OpenPhenom

F.1

siRNA attribute

Violation

N/S

Slack

Violation

N/S

Slack

0.0% 0.0% 0.7%

0.10 0.07 0.18

56.4 63.1 32.5

69.5% 70.5% 77.8%

1.21 1.26 1.48

−4.7 −5.3 −6.3

Compute save from early abstention

Let Cden (ℓ, τ ) denote the cost of running the generator up to layer-step pair (ℓ, τ ), Cdec the decoder cost, Cenc the Φ cost, and Cgϕ the translator cost. Post-generation scoring costs Cpost = Cden (full) + Cdec + Cenc + Cscore , whereas during-generation scoring costs map = Cden (ℓ, τ ) + Cgϕ + Cscore . Cℓ,τ

Thus early abstention avoids the remaining denoising computation, the VAE decoder, and the feature extractor. F.2

Hardware, runtime, and total compute

All experiments were run on a single internal cluster node with 8×NVIDIA A100 80GB GPUs (DDP, NCCL backend, mixed-precision bf16). Generator training uses 400,000 optimizer steps with lr=1e-4, warmup_steps=1000, gradient_clip_val=1.0, and global batch size 128 (16 per GPU × 8 GPUs). Per-experiment cost. Generations are performed with the 250-step DDIM-style trajectory: 1000 samples per condition for CelebA (16 conditions ⇒ 16k samples) and 100 per condition for RxRx1 (50 conditions ⇒ 5k samples). Per-step generator FLOPs are ≈ 46 GFLOPs (SiT-B/2, 256 patches), so a full 250-step trajectory is ≈ 11.5 TFLOPs per sample; the REPA projector adds ≈ 4 GFLOPs (<0.04% overhead). Trust-evaluation cost. Once features are cached, all reported trust evaluations run on a single GPU in under an hour per (model, scoring space, regime).

OpenPhenom feature diagnostic

While OpenPhenom-aligned REPA models achieve the best generation quality on RxRx1 as measured by post-hoc DINOv3 KID (Table 16), trust scoring in the OpenPhenom-aligned feature space fails (ρ ≈ 0.05). We conducted a systematic diagnostic to understand this failure. Root cause: collapsed within-class covariance. OpenPhenom features have an effective withinclass dimensionality of ≈4 in 2304 dimensions, compared to ≈16 for DINOv3. The top principal component captures 61.5% of within-class variance, and this dominant direction is orthogonal to both cell type and perturbation identity. The precision matrix therefore has condition number ≈21M, amplifying ≈2300 near-zero variance directions. Mahalanobis scoring destroys signal. The precision-weighted signal-to-noise ratio (Mahalanobis SNR) is 0.11 for OpenPhenom vs. 0.24 for DINOv3. The Mahalanobis/Euclidean SNR ratio is 0.73 for OpenPhenom (signal-destructive) vs. 1.55 for DINOv3 (signal-enhancing). This places OpenPhenom features in the weak-robustness regime of Proposition 8: the noise term 2 maxu̸=t |(ηk,u − ηk,t )⊤ Pk λ| exceeds the signal term minu̸=t ∥ηk,u − ηk,t ∥2Pk for 77.8% of samples on the siRNA attribute (Table 6). 25

Table 7: Effect of OpenPhenom-style preprocessing on DINOv3 features. Cross-space = OP trust vs. DINOv3 KID; same-space = DINOv3 trust vs. DINOv3 KID. IN = InstanceNorm. DINOv3 preprocessing variant

ρ(cross-space)

ρ(same-space)

−0.54 −0.60 −0.65 +0.16 −0.40 +0.12

0.93 0.80 0.77 0.43 0.43 0.56

Standard (to_rgb) + InstanceNorm Dual 3ch (no IN) Dual 3ch + IN Per-channel (no IN) Per-channel + IN

Table 8: Cross-encoder comparison on RxRx1 (50 conditions, same generated images from REPAOpenPhenom model). Same-space = trust and KID in the same encoder; cross-space = encoder trust vs. DINOv3 KID. Same-space

Cross-space (vs. DINOv3 KID)

Encoder

ρ(trust)

ρ(real.)

ρ(trust)

ρ(real.)

DINOv3 SigLIP OpenPhenom

0.93 0.79 0.45

0.94 0.65 0.40

— 0.80 −0.54

— 0.84 −0.48

Preprocessing drives the geometry. An isolation experiment applying OpenPhenom-style preprocessing (InstanceNorm + per-channel encoding) to DINOv3 eliminates the cross-space anticorrelation (ρ: −0.54 → +0.12) but simultaneously destroys DINOv3’s same-space scoring power (ρ: 0.93 → 0.43); see Table 7. This confirms that the issue is a fundamental consequence of OpenPhenom’s channel-agnostic architecture: InstanceNorm removes per-image intensity variation, and per-channel processing prevents cross-channel interaction, concentrating variance into non-semantic directions. Contrastive models are robust. A control experiment with post-hoc SigLIP features on the same RxRx1 images achieves ρ = 0.80 cross-space (SigLIP trust vs. DINOv3 KID), despite SigLIP having even higher pairwise cosine similarity (0.979 vs. 0.95 for OpenPhenom). The problem is isolated to OpenPhenom’s MAE feature geometry, not to high feature similarity per se (Table 8). Feature geometry comparison. underlies the scoring failure.

Table 9 summarizes the within-class covariance geometry that

Practical implication. OpenPhenom features are discriminative for classification (η 2 ≈ 0.41 for cell type, comparable to DINOv3’s 0.36) but violate the geometric assumptions required for Mahalanobis-based distributional scoring. This highlights a key distinction: classification requires only hyperplane separability in a few discriminative dimensions, while trust scoring requires meaningful geometry across the full feature space. For trust evaluation on RxRx1, we recommend using post-hoc contrastive features (DINOv3 or SigLIP) rather than MAE-based domain-specific encoders.

RxRx1 50-condition evaluation subset

All RxRx1 results in the main text are reported on a balanced 50-condition subset (25 seen + 25 unseen, covering all 4 cell types) rather than the full 4552-condition space. This appendix documents why the subset is constructed the way it is, and why a naive “top-n by sample count, stratified by cell type” construction fails. Constraints. A usable subset must satisfy three requirements simultaneously. (i) Sample sufficiency: each retained condition must have enough real samples for a per-condition KID bootstrap to be stable. (ii) Cell-type coverage: all four cell types must appear in the subset, so that trust-vs-∆KID correlations are not driven by a single biological context. (iii) Discriminability: each condition must be distinguishable from the rest under a class-balanced linear probe on real feature space, otherwise 26

Table 9: Within-class covariance geometry across encoders (RxRx1 real features). Metric

DINOv3

SigLIP

OpenPhenom

Feature dimension Effective dim (within-class) Top-1 PC fraction Precision condition number Euclidean SNR Mahalanobis SNR Mahal/Euclid ratio

1024 16.0 0.19 5.1M 0.15 0.24 1.55

1152 12.2 0.23 1.5M 0.11 0.28 2.52

2304 4.0 0.62 21.1M 0.15 0.11 0.73

trust scoring has nothing to rank, and apparent correlations are dominated by class-imbalance artifacts rather than genuine per-condition structure. The held-out training regime (Section 5) adds a fourth constraint: the test set should include enough unseen conditionings for it to be stress-testing, the 100 most-frequent cell type × perturbation pairs removed from training. Failure mode of the naive construction. A first-pass subset was built by selecting the top-n conditions by real-sample count, stratified across cell types (7/6/6/6 seen across ct=0/1/2/3 and 9/8/8 unseen across ct=0/1/2). A cross-validated class-balanced linear probe on real DINOv3 features, run on this subset, exposed three problems. 1. Filler rows. 13 of the 25 seen conditions (all ct=0 and ct=2 non-controls) had only n=14 real samples each. This is below any stable per-condition KID bootstrap. Under a class-balanced probe on the 50-class subproblem, these rows collapse to top-1 ≈0—the probe cannot distinguish them from each other or from the rest of the catalog. They contribute no usable ranking signal. 2. No unseen functional diversity. The unseen part of the test set contained only control wells, even though several non-control held-out perturbations were available in held-out set of the diffusion training. The support-shift test therefore covered a single class of perturbations rather than the diversity the held-out set was designed to provide. 3. Imbalance-probe confound. A class-imbalanced probe resulted in a class-frequency artifact: controls dominate the per-class sample counts. Under a class-balanced probe restricted to the 50-class subproblem, the controls collapse and the genuinely discriminable rows are the non-control siRNAs together with a smaller subset of controls. Any subset that does not prioritize these rows is measuring imbalance, not trust Corrected construction. The subset used throughout the paper is built to satisfy all four constraints by explicitly targeting discriminable rows under a class-balanced probe, rather than largest-n rows. • Seen arm (25 conditions, n ≥ 23, median n=32). 20 ct=1 non-control siRNAs selected by real-sample count (all at n=32), plus 5 ct=3 controls at n ≈ 22–24. The ct=1 non-controls carry the per-condition signal identified by the class-balanced probe; the ct=3 rows satisfy the cell-type-coverage constraint (see below). • Unseen arm (25 conditions, n ≥ 27, median n ≈ 52). 8 ct=0 controls (largest by n), 7 ct=1 non-control held-out siRNAs (349, 350, 351, 352, 363, 364, 401), 2 ct=1 controls (largest by n), and 8 ct=2 controls (largest by n). This is the first configuration in which the unseen arm contains non-control perturbations, giving the support-shift evaluation functional rather than purely positional variety. Cell-type coverage notes.

Two cell-type asymmetries are worth making explicit.

• ct=0 and ct=2 appear only in the unseen arm. Their non-control conditions cap at n=14, which fails the sample-sufficiency constraint, and all of their high-n controls are listed in RXRX1_HELDOUT_PAIRS, so they cannot populate the seen arm. Subset-level cell-type coverage (each cell type appears somewhere in the 50 conditions) is therefore retained, even though arm-level coverage is not—the alternative would require reinstating the 13 filler rows. • ct=3 is always retained. ct=3 has no held-out pairs at all (so it cannot populate the unseen arm) and its non-control wells max out at n=6 (unusable). The only ct=3 data with sufficient sample counts are the 31 control wells (sirna_id 1108–1138, max n=24). The seen arm 27

therefore includes the 5 largest ct=3 controls unconditionally, as the only way to honor the all-cell-types constraint. Relationship to the naive subset. 29 of the 50 final conditions are also present in the naive construction (18 in the unseen arm, 11 in the seen arm), and no condition migrates between arms. The corrected subset is therefore a targeted sharpening of the original—dropping filler rows and adding discriminable rows—rather than a rebuild, so results under the two constructions remain directly comparable on their shared conditions.

CellProfiler validation details on RxRx1

This appendix details the CellProfiler (CP) feature pipeline used for the morphology-space validation in Section 5.1.3 and reports the per-feature correlation breakdown and uninformative-feature sanity check. Decile-binning headline numbers. Table 10 summarizes the curves of Figure 6 as the bin-0 vs. bin-9 micro-accuracy (the headline contrast referenced in the main text), per (model, target). Table 10: CP-space decile downstream classification on RxRx1 (Figure 6 headline numbers; 50condition subset, marginal models, SigLIP scoring). Micro-accuracy of bin-0 vs. bin-9 classifiers trained on per-decile gen samples and tested on the canonical-50 real pool. “Trust spread” = bin 0 − bin 9; positive means higher-trust deciles produce more useful training data. Random-baseline micro-accuracy is flat (std ≤ 0.005) and matches the bin-0/bin-9 mean. Model

Target

bin 0

bin 9

Trust spread

Random mean

Vanilla Vanilla REPA (DINOv3) REPA (DINOv3) REPA (SigLIP) REPA (SigLIP)

celltype combo celltype combo celltype combo

0.900 0.127 0.905 0.148 0.897 0.146

0.841 0.109 0.814 0.107 0.803 0.117

+0.058 +0.018 +0.091 +0.041 +0.094 +0.029

0.893 0.130 0.887 0.132 0.887 0.131

Per-sample centroid distance (full table). Table 11 reproduces Table 2 from the main text with the trust-accepted sample count ntrust and the bootstrap 95% CI on ∆d (2000 reps, stratified). Negative ∆d with a CI bounded above by zero is read as “trust-selected samples are morphologically closer to real”; the vanilla-unseen row is the only cell whose CI grazes zero. Table 11: Full version of Table 2: per-sample centroid distance on RxRx1 (kept-621 CP features) with trust-accepted counts and bootstrap CIs. Random pool is condition-matched (combo-stratified) and matched in size to ntrust per arm. Model

Arm

ntrust

Accept%

d¯trust

d¯baseline

∆d

95% CI

Vanilla Vanilla REPA (DINOv3) REPA (DINOv3) REPA (SigLIP) REPA (SigLIP)

seen unseen seen unseen seen unseen

1657 547 1530 524 1640 588

66.3 21.9 61.2 21.0 65.7 23.5

18.63 20.92 18.31 20.19 18.38 22.13

18.93 21.52 18.90 21.21 18.88 22.97

−0.300 −0.601 −0.586 −1.018 −0.496 −0.844

[−0.570, −0.020] [−1.211, +0.028] [−0.854, −0.314] [−1.663, −0.394] [−0.750, −0.241] [−1.489, −0.167]

Feature pipeline. Starting from the raw Image.csv produced by CellProfiler over the canonical-50 real subset, we apply the following deterministic pipeline. (i) Drop metadata columns (regex matches against metadata, filename, pathname, imagenumber, objectnumber, executiontime, moduleerror, series, frame, group_*, url), leaving 2467 numeric features. (ii) Drop NaN rows and restrict to the canonical 50 cell×siRNA pairs (2061 real samples). (iii) Apply a variance threshold (var < 10−5 ) and an outlier filter (drop any column with a real-data |z| > 5), giving the kept-621 set used throughout the centroid and per-feature correlation analyses. (iv) Real-fit StandardScaler is reused across all readouts. 28

Accuracy (micro)

0.90

0.88

0.86

0.16

0.14

0.12

trust

0.84 0

Decile bin (0 = best trust)

Decile bin (0 = best trust) (a) Vanilla marginal. 0.16

0.90

Accuracy (micro)

0.92

0.88 0.86 0.84 trust

0.82 0

0.14

0.12

0.10 2

Decile bin (0 = best trust)

(b) REPA-DINOv3 marginal.

Accuracy (micro)

0.90 0.88 0.86 0.84 0.82

0.14 0.13 0.12 0.11

trust 0.80 0

Decile bin (0 = best trust)

Figure 6: Full CP-space decile downstream classification on RxRx1 (kept-621 features, SigLIP trust scoring). Each row is one marginal model; left: 4-way celltype accuracy by decile, right: 50-way combo accuracy. Solid blue: trust-ranked decile. The REPA-SigLIP row is highlighted in the main text. For the uninformative-feature sanity check we use a less aggressive variant — unfiltered — which keeps all 2415 columns that survive metadata removal and zero-variance pruning, so genuinely noise-only features can surface. For each (model, arm, feature f ), rtrust (f ) = Pearson µtrust (f ), µreal (f ) , rrand (f ) = Pearson µrand (f ), µreal (f ) ,

Per-feature correlation construction.

where µ• (f ) is the per-combo mean of f over the corresponding gen subset (trust-accepted, or combo-stratified random averaged over 5 seeds), and the Pearson correlation is taken across the arm’s combos. Higher r means the feature’s per-combo profile in gen tracks real well; ∆(f ) := rtrust (f ) − rrand (f ) > 0 means trust selection sharpens cross-combo correspondence on that feature. Top-correlated features under trust (kept-621). Across all six (model, arm) cells the same families recur at the top of rtrust (r ≥ 0.90): per-cell and per-cytoplasm 29

Table 12: Top-5 features by rtrust on REPA-DINOv3, RxRx1 unseen arm (kept-621). ∆ = rtrust − rrand . Feature

rtrust

rrand

∆

Median_Cells_Texture_Correlation_OrigGray_5_02_256 Median_Cells_Texture_Correlation_OrigGray_10_02_256 Median_Cytoplasm_Texture_Correlation_OrigGray_5_02_256 Mean_Cells_Texture_Correlation_OrigGray_5_03_256 Mean_Cells_Texture_Correlation_OrigGray_5_01_256

+0.937 +0.935 +0.934 +0.934 +0.932

+0.926 +0.836 +0.863 +0.893 +0.921

+0.011 +0.099 +0.071 +0.041 +0.011

Table 13: Bot-10 features by |rrand | on Vanilla, RxRx1 unseen arm (unfiltered 2415-feature set). Random gen has no per-combo correspondence with real on these features (|rrand | ≈ 0), consistent with their interpretation as uninformative under any selection. Feature

rtrust

rrand

Family

Mean_IdentifyPrimaryObjects_Children_.._Count StDev_IdentifyPrimaryObjects_Children_.._Count StDev_Nuclei_Texture_Variance_OrigGray_5_01_256 Median_Nuclei_RadialDistribution_RadialCV_.._4of4 Median_Cells_Granularity_15_OrigGray StDev_Nuclei_AreaShape_Zernike_9_7 Median_Cytoplasm_Texture_InfoMeas1_OrigGray_10_01_256 Median_Cytoplasm_AreaShape_FormFactor Mean_Cells_AreaShape_Zernike_7_1 StDev_Cells_Neighbors_AngleBetweenNeighbors_5

−0.000 −0.000 −0.086 −0.047 −0.262 −0.269 +0.052 +0.127 +0.282 +0.080

−0.000 −0.000 +0.001 +0.001 −0.001 −0.002 −0.004 −0.004 −0.004 +0.005

tautological count tautological count texture variability outermost-shell radial CV very-coarse granularity high-order Zernike texture moment shape regularity high-order Zernike neighbour angle

texture autocorrelation at scales 3/5/10 (Cells_Texture_Correlation_OrigGray_*), image-level granularity (Granularity_2/4_OrigGray), total cell-segmentation area (AreaOccupied_AreaOccupied_IdentifySecondaryObjects), and cytoplasm shape regularity variance (StDev_Cytoplasm_AreaShape_FormFactor). Table 12 gives a representative example for the REPA-DINOv3 unseen arm. Uninformative-feature sanity check (unfiltered). Repeating the analysis on the unfiltered 2415feature set surfaces features for which even random gen has no per-combo correspondence with real. Sorting by |rrand | ascending isolates these. Table 13 shows the bot-10 for the vanilla unseen arm; the same pattern appears across all six (model, arm) cells. The recurring uninformative families across all six (model, arm) cells are: tautological segmentation counts (every primary object has exactly one secondary child after segmentation, so the column is constant), very-coarse-scale granularity (dominated by well-level lighting), high-order Zernike moments (sample-noise-dominated rotational shape), outermost-ring radial CVs, and absolute object orientation (intrinsically arbitrary). None of these should correlate with biology, and rrand ≈ 0 confirms they don’t. A side observation: a handful of features that are uninformative under random become strongly anti-correlated under trust (rtrust ≪ 0 while |rrand | ≈ 0). The clearest example is Mean_Cytoplasm_Location_MaxIntensity_Y_OrigGray on REPA-SigLIP unseen (rrand = +0.005, rtrust = −0.475): image-level position features that random gen treats as noise become systematically biased by trust selection. This is the same pose/position bias visible in the kept-621 anti-correlation list.

Graded compositional shift on CelebA

A natural follow-up question to the post-generation ordering result is whether unseen conditionings should be treated as uniformly unreliable. CelebA held-out gives this question a sharp form: each queried condition has a natural notion of shift severity given by its minimum Hamming distance to the observed training support, and we can ask whether the per-condition mean trust score follows the same gradient. 30

Figure 7 shows the answer. The five seen (single-attribute) conditions cluster at low ∆KID and low trust; unseen conditions spread outward as the Hamming distance from support grows. Crucially, this is not a binary seen-vs.-unseen effect: some unseen conditions at small Hamming distance achieve quality comparable to seen ones, and the score correctly assigns them correspondingly favorable trust. Conversely, larger-distance compositions are both harder to generate and assigned worse trust. This behavior matters for the intended use case. Under compositional generalization, the relevant question is not simply whether a request is outside training support, but how severe that shift is and whether the resulting sample should still be trusted. The score captures this graded structure directly, rather than collapsing all unseen combinations into a single failure category. 1.6

ρ = 0.96 n = 16

1.4

∆KID

1.2 1.0 0.8 0.6 0.4 0.2

Seen (5) Unseen (11)

0.0 0

Mean trust score

Figure 7: CelebA stress-test (REPA DINOv3, DINOv3 scoring): per-condition trust vs. ∆KID. Each point is one of 16 conditions. The 5 seen (single-attribute) conditions cluster at low ∆KID; unseen conditions with more active attributes drift further from training support and exhibit higher ∆KID. Trust scores track this degradation (ρ = 0.96). Table 14 compares our calibrated Mahalanobis trust score (Section 3) against three alternative scorers on the same DINOv3 mean-patch feature space, under the identical P95-real-threshold selection protocol used for Table 1. Linear probe fits one logistic regression per P attribute on real DINOv3 features and scores a generated sample by the summed free energy − a logsumexp(za (x)). kNN (per-attr.) scores by the sum, over attributes, of cosine distance from the sample to its 5-th nearest neighbour inside the target-class real subset. CLIP (CelebA-only) scores by − cos(image, prompt) over a fixed template across the 16 attribute combos. The picture is consistent with the takeaway in Section 5.1.1. All three alternative scorers produce near-degenerate P95-real thresholds — the generated trust scores sit almost entirely below the 95 th-percentile real threshold, yielding ≈ 100% acceptance — so they are unusable as abstention rules even when their ranking happens to be informative. kNN recovers a Mahalanobis-comparable condition-level correlation but cannot translate it into a calibrated filter; the linear probe and CLIP alignment collapse on both axes. Our Mahalanobis score is the only construction here that yields both non-degenerate acceptance and substantial KID improvement, which is what the deployment-time abstention story requires. The Mahalanobis rows in Table 14 are reproduced from Tables 1 and 15; held-out rows match the main text and full-support rows are kept here as sanity checks.

Table 14: Scorer ablation: P95-real-threshold sample selection across four alternative trust scorers on the same DINOv3 meanpatch feature space. Threshold, KID computation (k=500 shuffled subsampling), and random baseline are identical to Table 1. Mahalanobis (ours) is the global-energy + per-attribute margin scorer in Section 3. ρ(T,∆KID) is the Spearman correlation between percondition mean trust score and per-condition ∆KID (n=16 for CelebA, n=50 for RxRx1). All three ablation scorers produce near-degenerate P95-real thresholds (gen scores sit almost entirely below the 95 th-percentile real threshold), yielding ≈ 100% acceptance; kNN still recovers Mahalanobis-level ranking correlation, while the linear probe and CLIP alignment collapse on both axes. Mahalanobis held-out rows are reproduced from Table 1; full-support rows are kept here as sanity checks. Model

Setting

Accept%

KIDtrust ↓

KIDbaseline ↓

∆%↑

ρ(T)↑

85.6 58.7 87.9 55.4 85.2 56.1

0.266±.005 0.224±.009 0.163±.005 0.228±.010 0.225±.009 0.239±.018

0.288±.012 0.368±.020 0.194±.004 0.401±.031 0.265±.008 0.423±.020

+7.5 +39.1 +15.6 +43.1 +15.1 +43.6

+0.87 +0.96 +0.83 +0.96 +0.88 +0.97

CelebA — Linear probe (per-attribute energy) Vanilla full 100.0 0.292±.011 Vanilla held-out 100.0 0.350±.012 REPA (DINOv3) full 100.0 0.192±.008 REPA (DINOv3) held-out 100.0 0.388±.015 REPA (SigLIP) full 100.0 0.256±.013 REPA (SigLIP) held-out 100.0 0.391±.016

0.294±.008 0.343±.033 0.194±.003 0.409±.020 0.256±.013 0.417±.033

+0.9 −2.2 +1.0 +5.1 +0.0 +6.2

−0.29 −0.04 −0.27 +0.21 −0.29 +0.34

CelebA — kNN (per-attr., 5-th NN summed) Vanilla full 99.9 0.292±.013 Vanilla held-out 99.8 0.357±.023 REPA (DINOv3) full 99.9 0.188±.007 REPA (DINOv3) held-out 99.7 0.403±.012 REPA (SigLIP) full 99.9 0.263±.011 REPA (SigLIP) held-out 99.8 0.424±.020

0.302±.012 0.358±.021 0.187±.010 0.396±.026 0.263±.010 0.411±.024

+3.2 +0.3 −0.5 −1.7 +0.2 −3.0

+0.89 +0.92 +0.83 +0.88 +0.91 +0.87

CelebA — CLIP alignment (16 joint-combo prompts) Vanilla full 100.0 0.649±.013 Vanilla held-out 100.0 0.729±.016 REPA (DINOv3) full 100.0 0.625±.012 REPA (DINOv3) held-out 100.0 0.770±.020 REPA (SigLIP) full 100.0 0.631±.025 REPA (SigLIP) held-out 100.0 0.733±.011

0.656±.008 0.720±.024 0.603±.015 0.775±.034 0.613±.011 0.751±.030

+1.0 −1.3 −3.6 +0.7 −2.8 +2.5

−0.55 +0.60 −0.48 +0.77 −0.65 +0.73

RxRx1 (50-condition subset) — Mahalanobis (ours) Vanilla full 14.9 0.202±.003 Vanilla held-out 4.4 0.196±.006 REPA (DINOv3) full 21.7 0.175±.004 REPA (DINOv3) held-out 5.5 0.197±.006 REPA (SigLIP) full 18.4 0.190±.002 REPA (SigLIP) held-out 6.1 0.181±.003

0.307±.018 0.323±.013 0.276±.013 0.333±.019 0.301±.007 0.312±.019

+34.2 +39.4 +36.7 +40.8 +37.1 +42.0

+0.87 +0.90 +0.83 +0.90 +0.84 +0.87

RxRx1 — Linear probe (per-attribute energy) Vanilla full 100.0 0.304±.015 Vanilla held-out 100.0 0.329±.020 REPA (DINOv3) full 100.0 0.278±.019 REPA (DINOv3) held-out 100.0 0.351±.019 REPA (SigLIP) full 100.0 0.303±.011 REPA (SigLIP) held-out 100.0 0.330±.025

0.312±.015 0.330±.023 0.274±.019 0.354±.013 0.311±.011 0.318±.018

+2.4 +0.5 −1.3 +0.8 +2.7 −3.9

−0.09 +0.08 +0.15 +0.10 +0.01 +0.08

RxRx1 — kNN (per-attr., 5-th NN summed) Vanilla full 100.0 0.304±.015 Vanilla held-out 100.0 0.329±.020 REPA (DINOv3) full 100.0 0.278±.019 REPA (DINOv3) held-out 100.0 0.351±.019 REPA (SigLIP) full 100.0 0.303±.011 REPA (SigLIP) held-out 100.0 0.330±.025

0.312±.015 0.330±.023 0.274±.019 0.354±.013 0.311±.011 0.318±.018

+2.4 +0.5 −1.3 +0.8 +2.7 −3.9

+0.91 +0.88 +0.92 +0.91 +0.90 +0.89

CelebA — Mahalanobis (ours) Vanilla full Vanilla held-out REPA (DINOv3) full REPA (DINOv3) held-out REPA (SigLIP) full REPA (SigLIP) held-out

Detailed results

This appendix collects the auxiliary results referenced from Sections 5.1.1, 5.1.2 and 5.2: full-support sanity-check rows for the P95-real-threshold selection, condition-level Spearman correlations of the trust score with its realism and faithfulness components, the trust/realism/faithfulness decile decomposition on CelebA, the analogous DINOv3 decile binning on RxRx1, and the during-generation decile binning under the translator. Full-support sanity checks. Table 15 extends Table 1 with the full-support training-distribution rows. Relative KID gains shrink under full support, where the generator already covers the target conditions, but remain positive on both datasets and all three generators (+7–16% on CelebA, +34– 37% on RxRx1). The held-out rows are the deployment-relevant ones and are reproduced compactly in Table 1. Table 15: Complete P95-real-threshold post-generation sample selection (DINOv3 / SigLIP trust scoring). Full-support rows are sanity checks; held-out rows are the main deployment-relevant setting and are reproduced compactly in Table 1. Threshold set at the 95th percentile of real-sample trust scores; for marginal models, only seen-combo real samples are used for calibration. KID measured in DINOv3 space via shuffled subsampling (k=500, repeated for CIs). Random baseline: random subset of the same size, each compared against its condition-matched real pool. ∆% = relative KID improvement of trust-selected over random (positive = better). Model

Setting

Accept%

KIDtrust ↓

KIDbaseline ↓

∆%↑

CelebA Vanilla Vanilla REPA (DINOv3) REPA (DINOv3) REPA (SigLIP) REPA (SigLIP)

full held-out full held-out full held-out

85.6 58.7 87.9 55.4 85.2 56.1

0.266±.005 0.224±.009 0.163±.005 0.228±.010 0.225±.009 0.239±.018

0.288±.012 0.368±.020 0.194±.004 0.401±.031 0.265±.008 0.423±.020

+7.5 +39.1 +15.6 +43.1 +15.1 +43.6

RxRx1 (50-condition subset) Vanilla full Vanilla held-out REPA (DINOv3) full REPA (DINOv3) held-out REPA (SigLIP) full REPA (SigLIP) held-out

14.9 4.4 21.7 5.5 18.4 6.1

0.202±.003 0.196±.006 0.175±.004 0.197±.006 0.190±.002 0.181±.003

0.307±.018 0.323±.013 0.276±.013 0.333±.019 0.301±.007 0.312±.019

+34.2 +39.4 +36.7 +40.8 +37.1 +42.0

Component-level condition correlations. Table 16 reports the per-condition Spearman correlations of trust, realism, and faithfulness against the held-out ∆KID. Trust is the strongest predictor across both datasets and all three generators (ρ(T ) ≥ 0.88). The decomposition shows that on RxRx1 the realism signal carries most of the predictive power, consistent with the held-out perturbations producing manifold drift that the generator cannot reach; on CelebA both faithfulness and realism contribute, with faithfulness being the slightly stronger of the two on the controlled compositional split. Table 16: Condition-level Spearman correlations under the harder support-shift regime: CelebA held-out and RxRx1 held-out (DINOv3/SigLIP scoring, marginal models). ρ(T ) = trust, ρ(R) = realism, ρ(F) = faithfulness. The main text reports only ρ(T ) in Table 1. CelebA (n=16)

RxRx1 (n=50)

Model

ρ(T )↑

ρ(R)↑

ρ(F)↑

ρ(T )↑

ρ(R)↑

ρ(F)↑

Vanilla REPA (DINOv3) REPA (SigLIP)

0.96 0.96 0.96

0.69 0.71 0.76

0.84 0.86 0.84

0.90 0.90 0.88

0.88 0.88 0.86

0.68 0.75 0.75

Trust/realism/faithfulness decile decomposition (CelebA). Figure 8 re-plots the main-text Figure 2 with all three score components. Realism alone does not track downstream condition accuracy, 33

Trust Realism Faithfulness

1.2 1.0

Trust

0.8

16-way accuracy

KID (lower = better)

consistent with realism being a manifold-quality rather than a condition-aware signal; trust combines the two and drives the ordering on both panels.

0.8 0.6 0.4

Realism

Faithfulness

0.7 0.6 0.5 0.4

Bin index (0=best, 9=worst)

(a) CelebA ∆KID by decile.

(b) CelebA downstream 16-way accuracy.

Figure 8: Trust / realism / faithfulness decomposition for the main-text Figure 2. Same setting as the main figure (REPA-DINOv3 held-out, DINOv3 scoring), but plotting all three score components. Realism does not track downstream condition accuracy, which is consistent with realism not being a condition-aware signal; trust drives the ordering on both panels.

Trust Realism Faithfulness

1.5

50-way accuracy

KID (lower = better)

RxRx1 DINOv3 decile binning. Figure 9 shows the analogous DINOv3-space decile binning on the RxRx1 50-condition subset. Better trust deciles produce both lower ∆KID and stronger downstream classifiers, mirroring the CelebA ordering. The main-text discussion emphasizes the CellProfiler morphology validation in Section 5.1.3 because CP features are independent of the learned DINOv3 validation encoder used for KID.

1.0

0.5

Trust Realism Faithfulness

0.15

0.10

0.05 0

Bin index (0=best, 9=worst)

(a) RxRx1 ∆KID by trust decile.

(b) RxRx1 downstream classification.

Figure 9: RxRx1 DINOv3 decile binning (REPA-DINOv3 held-out, DINOv3 scoring, 50condition subset). These learned-encoder trends support the same ordering story as CelebA, but the main text emphasizes the CellProfiler morphology validation because it is independent of the learned DINOv3 validation encoder.

During-generation decile binning. Figure 10 confirms that the translator features reproduce the same monotonic trust ordering on both CelebA and RxRx1 without decoding the sample, complementing the support-shift evidence in Table 3.

Scorer-design ablation

L.1

Complete during-generation table

Table 17 reports the full during-generation table, including full-support sanity-check rows that are omitted from the main text. The main text keeps only the held-out rows because those are aligned with the missing-target evaluation setting. 34

KID (lower = better)

Trust Realism Faithfulness

0.6

0.5

0.4

Trust Realism Faithfulness

0.6 0.5 0.4 0.3

Bin index (0=best, 9=worst)

(a) CelebA ∆KID by decile.

(b) RxRx1 ∆KID by decile.

Figure 10: During-generation decile binning after translation. The translator features recover the monotonic trust trend without decoding the sample and re-encoding it through the feature extractor.

Table 17: Complete non-DINOv3 trust scoring: model-internal REPA features and our translator. Left: P95-real-threshold selection (KID in DINOv3 space, random baseline with per-draw conditionmatched real). Right: condition-level Spearman correlations. Full-support rows are sanity checks; held-out rows are summarized in Table 3. P95-real Selection Model

KIDtrust ↓

KIDbaseline ↓

∆%↑

ρ(T )↑

ρ(R)↑

ρ(F)↑

CelebA — REPA aligned_mean (during generation) REPA (DINOv3) full 97.8 0.187±.013 REPA (DINOv3) held-out 62.7 0.278±.015 REPA (SigLIP) full 97.2 0.254±.008 REPA (SigLIP) held-out 66.7 0.293±.024

0.195±.010 0.404±.021 0.258±.012 0.425±.018

+4.1 +31.2 +1.6 +31.1

0.73 0.95 0.81 0.95

0.74 0.64 0.80 0.56

0.29 0.91 0.53 0.89

CelebA, translator (during generation) Vanilla full 32.6 Vanilla held-out 23.6 REPA (DINOv3) full 89.2 REPA (DINOv3) held-out 48.4 REPA (SigLIP) full 87.2 REPA (SigLIP) held-out 17.2

0.254±.015 0.195±.008 0.188±.010 0.250±.016 0.248±.009 0.308±.015

0.299±.007 0.378±.023 0.197±.005 0.397±.017 0.263±.011 0.409±.015

+15.2 +48.4 +4.4 +37.2 +5.7 +24.8

0.61 0.83 0.78 0.92 0.56 0.88

0.29 0.74 0.64 0.66 0.29 0.83

0.57 0.82 0.71 0.91 0.52 0.86

RxRx1 — REPA aligned_mean (during generation) REPA (DINOv3) full 26.4 0.378±.018 REPA (DINOv3) held-out 1.4 0.537±.032 REPA (SigLIP) full 20.6 0.628±.029 REPA (SigLIP) held-out 3.9 0.564±.022

0.283±.005 0.329±.037 0.315±.006 0.293±.023

−33.4 −63.4 −99.5 −92.3

−0.01 0.09 −0.13 0.13

0.10 0.05 −0.08 −0.11

−0.27 0.19 −0.38 0.55

RxRx1, translator (during generation) Vanilla full 37.0 Vanilla held-out 38.4 REPA (DINOv3) full 82.7 REPA (DINOv3) held-out 62.3 REPA (SigLIP) full 80.3 REPA (SigLIP) held-out 60.5

0.290±.017 0.346±.016 0.281±.016 0.372±.037 0.316±.016 0.329±.018

+4.7 +24.9 +1.9 +33.8 +8.1 +28.7

0.70 0.78 0.18 0.88 0.65 0.70

0.36 0.31 0.67 0.47 0.68 0.31

0.67 0.76 −0.17 0.80 0.00 0.68

L.2

Setting

Accept%

Correlations

0.277±.007 0.260±.003 0.276±.017 0.246±.003 0.290±.015 0.235±.011

Fine-grained timestep (full table)

Table 18 gives the full per-k numbers summarized by Figure 4 in the main text. 35

Scoring step

ρtrust

KIDtrust ↓

KIDbaseline ↓

∆KID%↑ (accept)

Image ∆ L2

Steps saved

k=0 (t≈1.00) k=27 (t≈0.90) k=55 (t≈0.79) k=83 (t≈0.68) k=110 (t≈0.58) k=138 (t≈0.47) k=166 (t≈0.36) k=193 (t≈0.26) k=221 (t≈0.15) k=248 (t≈0.04)

0.21 0.08 0.37 0.58 0.65 0.73 0.75 0.79 0.78 0.79

0.290 0.309 0.263 0.240 0.234 0.223 0.208 0.207 0.202 0.210

0.358 0.365 0.356 0.357 0.355 0.346 0.342 0.367 0.356 0.347

+18.8% (19%) +15.5% (19%) +26.1% (19%) +33.0% (19%) +34.1% (21%) +35.5% (21%) +39.1% (21%) +43.6% (22%) +43.1% (23%) +39.4% (24%)

– 74.5 33.7 24.2 19.3 17.9 16.7 15.6 14.7 12.3

≈100% ≈89% ≈78% ≈67% ≈56% ≈45% ≈34% ≈23% ≈12% ≈1%

Post generation

0.96

0.221

0.365

+39.3% (59%)

–

—

Table 18: Fine-grained timestep ablation for cheap internal trust scoring on CelebA held-out (Vanilla SiT-B/2, 250-step sampler). At each scoring step k, the predicted clean latent x̂0 is passed through the translator into SigLIP space (no VAE decode, no encoder pass) before Mahalanobis scoring. Spearman ρtrust is the condition-level correlation against ∆KID. KIDacc and KIDbaseline are computed on the P95-real-threshold-accepted set and random matched-size subsets, respectively. “Image ∆ L2” is the mean L2 distance between consecutively captured VAE-decoded x̂0 images, quantifying how much the predicted clean image still moves between capture steps (undefined at k=0). “Steps saved” is the fraction of sampler steps avoided by abstaining at k. The final row is the post-generation DINOv3 oracle (reproduced from Table 14, Vanilla held-out). The ranking signal grows monotonically with k, and P95-real-threshold ∆KID matches or exceeds the post-generation oracle from k≈166 onward while still saving ≥34% of generation compute.

The same fine-grained timestep ablation on RxRx1 held-out is shown in Figure 11 for the Vanilla SiT-B/2 generator with the translator scoring path and the same 250-step sampler used for CelebA. The qualitative picture matches Figure 4: the trust signal is already useful well before the final decoded sample, and the per-step L2 change in the predicted-clean image x̂0 falls monotonically as the trajectory settles (∼ 7.0 at k = 27 down to ∼ 4.6 at k = 248).

oracle +44.0%

40 6

0 0

100

150

200

denoising step k (of 250)

per-step image ∆L2

RxRx1 timestep ablation

∆KID% vs. random

L.3

250

Two differences are worth flagging. First, on ∆KID% (left) image ∆L2 (right) RxRx1 the during-generation ∆KID saturates very early — by k = 27 the FPR95-selected subset already reaches roughly +21% over a Figure 11: RxRx1 timestep ablation (Vanilla SiT condition-matched random subset, and subse- B/2, 250-step sampler, translator features). Red quent steps oscillate in the +20% to +27% squares: P95-real ∆KID% of the FPR95-accepted band without a further monotonic trend. Sec- subset against a condition-matched random subset, ond, the during-generation score does not match evaluated on intermediate predicted-clean latents the post-generation DINOv3 oracle (+44.0%, x̂0 (k) projected through the translator. Blue cirdashed line): on this harder support-shift regime cles: per-step L2 change in the VAE-decoded x̂0 . the mapped internal feature captures roughly Dashed red line: post-generation DINOv3 oracle 60% of the discriminative power that the full (+44.0%). Compared to Figure 4, the during genfeature extractor pass recovers after decod- eration signal saturates earlier (k ≈ 27) but plateaus ing. This gap is consistent with the regime below the post-generation oracle, reflecting the in which Mahalanobis-based scoring is intrin- harder support shift on RxRx1. sically harder on RxRx1 (smaller per-attribute margins on the sirna axis, Tables 4 and 5); the during-generation extension preserves a useful trust ordering but does not recover the full precision of the post-generation oracle. The practical compute trade-off is therefore sharper than on CelebA: abstaining as early as k = 27 saves ≈89% of the denoising trajectory while retaining most of the during-generation ∆KID signal, but full post-generation scoring remains the higher-precision choice when compute is not the bottleneck. 36

Record · ID 267669 · SHA-256 d957c4fbe7b993b3
Retrieved via Conceptio — every document is proof-bundled with source, license, and retrieval metadata.