MA-SBI: Misspecification-Aware Simulation-Based Inference via Side-Channel Guidance Arunkumar V1 1
arXiv:2606.16923v1 [cs.AI] 15 Jun 2026
2
Manoranjan Gandhudi2 Gangadharan G. R.3 Arun Prakash4 S. Senthilkumar1
University College of Engineering, Anna University Tiruchirappalli, Tamil Nadu, India
3 Central University of Karnataka, India National Institute of Technology Tiruchirappalli, India 4 School of Computer & Systems Sciences, Jawaharlal Nehru University, New Delhi, India
Abstract Simulation-based inference (SBI) of latent parameters is often hindered by simulator misspecification, the mismatch between simulated and real-world observations caused by inherent modeling simplifications. RoPE, the recent state-of-the-art for robust SBI, addresses this through optimal transport between learned representations of real and simulated observations, but requires ground-truth parameter calibration pairs that are typically unavailable in the very settings where SBI is needed. What practitioners do have is unstructured side-information such as regime labels, instruction text, and policy bulletins. We propose Misspecification-Aware Simulation-Based Inference (MA-SBI), a calibration-free framework that turns this side-channel into a posterior correction. A learned corrector maps side-channel text to an observationspace shift applied before any pre-trained amortized posterior, requiring no retraining and no parameter ground-truth. Our main theorem bounds achievable bias reduction by the mutual information between misspecification and side-channel, with a non-vacuous constant that extends to all sub-Gaussian noise via Donsker–Varadhan. On hide-the-calibration benchmarks, MA-SBI with text alone matches the oracle posterior across 10 seeds and two backbones (TOST equivalence), while RoPE given more data does not. The two approaches are complementary: where misspecification is structural and recoverable from parameter pairs, RoPE dominates, as the theory predicts. A stochastic variant improves posteriorpredictive log-likelihood on real COVID and OxCGRT epidemiological data, and correctly leaves the posterior unchanged on a well-specified cognitive-science corpus.
1
Introduction
Simulation-based inference [5] has become the dominant framework for bayesian inference when the simulator’s likelihood is intractable but samples are available. Neural posterior estimation (NPE) amortizes this inference by training a conditional density estimator q̂(θ | y) on simulated pairs (θ, ysim ), which at evaluation time produces approximate posteriors with a single forward pass. SBI is now standard in particle physics, epidemiology, cognitive neuroscience, and galactic dynamics. The method’s persistent weakness is simulator misspecification. Every simulator is an approximation, and when real observations yobs are drawn from a process that differs from the simulator’s, posterior estimates become biased, overconfident, or both. Even small amounts of misspecification can produce severe posterior collapse on real-world benchmarks [30]. Recent robust-SBI methods have approached this problem from three directions: • Calibration-based methods (RoPE [30], FRISBI [23], FMCPE [22]) use ground-truth parameter pairs (θ∗ , y ∗ ) to learn a simulator-reality gap, typically via optimal transport or flow matching. • Noise-model methods (NNPE [29], robust summary statistics [12]) specify misspecification as spikeand-slab contamination or train statistics robust to perturbation. 1
• Prior-adaptation methods (PriorGuide [31]) modify the posterior at test time via guidance, without retraining. All three approaches require something the practitioner may not have, whether ground-truth parameter measurements (typically unavailable when SBI is used), an explicit functional form for the noise, or a meaningful alternative prior. None exploits unstructured side-information about the operating regime, which is the kind of input practitioners actually do have, such as instruction text in a behavioural experiment, World health organization bulletins for an emerging variant, or the active emissions-policy regime in a climate run. The three applications motivate Sec. 3.2. Our contribution. We introduce MA-SBI as a parameter-calibration-free framework that uses side-channel information z (typically text embeddings) to index a learned correction δ̂ψ (z) applied to the observation before it is fed to a pre-trained amortized posterior (Figure 1). The corrected posterior is q̃(θ | yobs , z) := q̂ θ yobs − δ̂ψ (z) . Three contributions follow. 1. Theory (Sec. 4). Theorem 1 bounds achievable bias reduction by C · I(δ ∗ ; z) with an explicit non-vacuous constant (C=0.173 for the Gaussian-linear verifier parameters). Theorem 2 extends the same constant C=2τ 2 to all sub-Gaussian δ ∗ via a Donsker-Varadhan argument, sharp at ratio 0.995 on Rademacher constructions. Theorem 3 gives identifiability conditions connecting MA-SBI to classical instrumentalvariable analysis. Theorem 4 shows that MA-SBI strictly generalizes RoPE. When z is restricted to a calibration index, MA-SBI recovers RoPE, and for abstract z it surpasses RoPE in expressiveness, a claim we verify empirically. 2. Architecture (Sec. 3). The corrector is backbone-agnostic, composing with any conditional density estimator (validated on MAF flow and DDPM diffusion). A three-way decomposition δ̂ψ (z) = E[δ̂ψ (z)] + (δ̂ψ (z) − E[δ̂ψ (z)]) separates a constant Marginal correction (matched by RoPE) from a regime-conditional residual unique to MA-SBI. 3. Empirics (Sec. 5). On the Simple-Likelihood-Complex-Posterior (SLCP) hide-the-calibration benchmark, MA-SBI with text z is statistically equivalent to the oracle neural posterior on the flow-NPE backbone (TOST p < 0.0001, 10 seeds), and the same result holds under continuous z ∈ [0, 1] (Appendix R), confirming that the corrector generalises beyond categorical regimes. A faithful re-implementation of RoPE’s (Algorithm 1) collapses at small No (Table 2). On a drift-diffusion model with task-instruction side-channels, MA-SBI recovers 101% of the NPE-to-Oracle gap, compared with RoPE’s 80% given full (θ∗ , y ∗ ) calibration. On SIR weekend-delay, RoPE dominates because the structural permutation misspecification is fully recoverable from calibration pairs, making MA-SBI and RoPE complementary rather than competing.
2
Related Work
2.1
Simulation-based inference
Classical likelihood-free ABC [3, 25] has been superseded by neural density estimation: NPE [19], SNPE-C [8], sequential NPE [17, 9], and score-based / diffusion SBI [7, 24]. The amortized posterior is treated as a black box, and MA-SBI composes with any such estimator.
2.2
Robustness to misspecification
RoPE [30] is our primary baseline: it formalises the misspecification gap as an optimal transport problem between learned summary representations of real and simulated observations, using calibration pairs (θ∗ , y ∗ ). Theorem 4 formalises the relationship between RoPE and MA-SBI.
2
Figure 1: MA-SBI pipeline. Training: NPE is fit on well-specified (θ, ysim ) pairs. Inference: the learned δ̂ψ (z) shifts the observation into the simulator’s support; the same pre-trained NPE evaluates yobs − δ̂ψ (z). No (θ∗ , y ∗ ) pairs required. FRISBI [23] extends RoPE to inductive amortised inference via mini-batch OT; FMCPE [22] transports NPE posteriors toward the real-data posterior via flow matching; both still require calibration pairs. PriorGuide [31] adapts the prior at test time via diffusion guidance, which costs ∼102 score-model evaluations per sample; MA-SBI amortises the correction into a single forward pass of δ̂ψ (z). NNPE [29] adds spike-and-slab noise to simulator outputs; robust-statistics SBI [12] learns summary statistics that minimise simulated-vs-observed mismatch. Positioning. Every method above requires ground-truth parameter pairs, an explicit noise specification, or an alternative prior. MA-SBI is orthogonal: it corrects misspecification using unstructured observational side-information. Theorem 4 shows we generalise RoPE under calibration-indexed z; the hide-the-calibration experiment (Sec. 5.2) shows strict empirical generalisation using text. Conformal prediction. Weighted or localized conformal prediction [27, 2, 21] adjusts coverage under covariate shift but does not correct posterior location: a biased posterior with a calibrated credible set is still biased. MA-SBI corrects the location; conformal procedures can be applied unchanged on top.
2.3
Side-information for inference
Text signals are increasingly used as direct predictors in scientific forecasting (From News to Forecast [28] integrates news events into LLM-based time-series forecasting; EconAgent [14] simulates macroeconomic activity from LLM-generated agents). The use of text in MA-SBI is structurally different. Here z is not a predictor of θ or y directly, but a diagnostic indicator of which simulator-misspecification regime is active. The simulator and prior do the inference, while z identifies the way in which the simulator is wrong. The Exclusion Test (Sec. 5.5) operationally enforces this distinction by rejecting any z that predicts ysim directly. Classifier-free guidance in diffusion [11] underlies an architectural variant in Sec. 3.5. Our identifiability framing (Theorem 3) connects to instrumental-variable analysis [1].
3
Method
3.1
Setup
A simulator f (θ, ε) → y is possibly misspecified, with real observations yobs drawn from a process different from f . Simulated pairs (θ, ysim ) are available for training, together with a side-channel z ∈ Z characterising 3
the regime that generated yobs . The goal is to recover a posterior over θ that corrects for misspecification using only z, without parameter ground-truth. Given a pre-trained amortized posterior q̂(θ | y) obtained on (θ, ysim ) pairs by standard NPE, MA-SBI learns a corrector δ̂ψ : Z → Rdim y and defines q̃(θ | yobs , z) := q̂ θ yobs − δ̂ψ (z) . (1) The corrector shifts the observation into the simulator’s support before the amortized posterior is evaluated, and no retraining of q̂ is required.
3.2
Training the corrector
Data requirement. MA-SBI requires no θ∗ labels. Instead, each calibration point i supplies an observation yobs,i , an aligned zi , and the simulator output ysim,i evaluated at the same inputs that generated yobs,i . The asymmetry from RoPE is that alignment is required in observation space rather than at the level of ground-truth parameter values. Within our benchmarks this alignment holds by construction, since the synthetic misspecification process applies a deterministic regime shift on top of ysim , leaving yobs and ysim paired sample-wise. Training uses observable calibration triples (zi , ysim,i , yobs,i ), with the empirical misspecification δi := yobs,i − ysim,i observed directly. The corrector minimises N
ψ̂ = arg min ψ
2 1 X δi − δ̂ψ (zi ) . N i=1 2
(2)
When sample-wise alignment is unavailable, Eq. (2) is replaced by a per-regime distribution-matching loss (Appendix S). Unlike RoPE or FMCPE, this requires no θ∗ pairs, only yobs paired with z and a simulator run at matched inputs. The triples requirement vs. the parameter-pair requirement. (yobs , z) co-occurs naturally whenever a human-readable interpretation of the observation is recorded as part of protocol. Reaction-time distributions are paired with experimental instructions in DDM modelling. Case counts are paired with CDC and WHO bulletins in epidemiological nowcasting. Regional damage is paired with IPCC text in climate-impact assessment. In all three settings, θ∗ is the inferential target and is not directly observable, which is the reason SBI is used in the first place. MA-SBI converts the abundant side-information channel into a posterior correction.
3.3
Stochastic-bootstrap variant for real-world data
The point corrector assumes the synthetic triples match p(y | z), which fails on real data when within-regime variation dominates the between-regime shift. The stochastic variant draws δ (k) from a per-regime empirical residual pool p̂LOO (δ | z⋆ ) = Uniform{δjboot : zj = z⋆ , j ̸= ⋆}, where δiboot = yobs,i − Eθ∼q̂(·|yobs,i ) [f¯(θ)], and averages corrected posteriors, K
q̃(θ | y, z) =
1 X q̂(θ | y − δ (k) ), K
δ (k) ∼ p̂LOO (δ | z).
(3)
k=1
No synthetic generator is needed. The variant is preferred when the per-cell residual norm exceeds the perregime median by >3× (a pre-flight signal); otherwise the point corrector suffices. On synthetic benchmarks the LOO pool collapses to a single point and Eq. (3) reduces to the point corrector.
3.4
Three-way decomposition
The corrector admits the decomposition δ̂ψ (z) = Ez [δ̂ψ (z)] + δ̂ψ (z) − Ez [δ̂ψ (z)], separating the constant {z } | {z } | Marginal
Conditional
location shift (which RoPE-style methods also recover from (θ∗ , y ∗ ) pairs) from the regime-conditional 4
component unique to MA-SBI. On Two Moons the Marginal alone is harmful (−13%) while the Conditional carries the total to +95% (Sec. 5.3); the decomposition is the ablation isolating where MA-SBI adds value.
3.5
Backbones
The framework is backbone-agnostic. Normalising flow backbone. Standard sbi [26] NPE with masked autoregressive flow (MAF). δ̂ψ applied as input correction before evaluation. Diffusion backbone. Variance-preserving DDPM [10] εϕ√ (θt , t, y) trained on (θ, ysim ) pairs by denoising √ score matching: L(ϕ) = Et,θ0 ,ε ∥ε − εϕ (θt , t, ysim )∥2 , with θt = ᾱt θ0 + 1 − ᾱt ε and a linear β-schedule over T =200 steps. At inference we condition on yobs − δ̂ψ (z). A classifier-free-guidance variant is possible but the input-correction formulation is simpler and composes with any conditional density estimator; Sec. 5.5 confirms that the two backbones produce within-noise-equivalent gap closure.
3.6
Implementation details
The default z-encoder is hashed character n-grams (64-d, deterministic, offline); sentence-transformers all-MiniLM-L6-v2 is a drop-in replacement and lands within ∼2 C2ST points of the hash. The corrector δ̂ψ is a 3-layer MLP, GELU, 128–256 hidden units. All experiments run on a single CPU (no GPU). Full hyperparameters in Appendix E; pseudocode in Algorithm 1.
4
Theory
4.1
Bias-reduction bound (Gaussian-linear regime)
In the Gaussian-linear setting, canonical-correlation analysis gives a tight characterisation of achievable bias reduction; the bound is empirically predictive on real benchmarks (Sec. 5.2). Theorem 2 extends the same tight constant C = 2τ 2 to the log-concave sub-Gaussian family via Talagrand’s T2 inequality; a distribution-free fallback for bounded δ ∗ is in Appendix C. Theorem 1 (Bias reduction bound, Gaussian-linear). Suppose the simulator is Gaussian-linear: θ ∼ N (0, σp2 Id ), y = θ + ε, ε ∼ N (0, σs2 Id ), yobs = y + δ, δ ∼ N (0, σδ2 Id ), with z jointly Gaussian with δ and dim(z) ≥ d (so the canonical correlations ρ1 , . . . , ρd are not capped by dim(z)). Let α = σp2 /(σp2 + σs2 ). The achievable bias reduction over vanilla NPE — taken as the supremum over measurable z-indexed correctors δ̂ψ — satisfies: i h ∗ ≤ 2α2 σδ2 · I(δ ∗ ; z). (4) sup ∆MSEunc −∆MSEcor ≤ α2 dσδ2 · 1 − exp − 2I(δd ;z) δ̂ψ
The right inequality is the linear form with non-vacuous constant C = 2α2 σδ2 . For verifier parameters (σp = 0.5, σs = 0.3, σδ = 0.4), C = 0.173. Corollary 1 (Graceful degradation). I(δ ∗ ; z) → 0 ⇒ bias reduction → 0: the side-channel cannot harm performance when uninformative. Corollary 2 (Oracle recovery). As I(δ ∗ ; z) → ∞, the bias reduction approaches α2 dσδ2 : the side-channel recovers the full well-specified MSE. Both corollaries are confirmed numerically on a K = 4 discrete-source sweep (Figure 2): empirical gap closed tracks I(δ ∗ ; z)/ log K exactly. Full proof and a discussion of non-Gaussian observation noise are in Appendix A; the cleaner sub-Gaussian extension is Theorem 2.
5
Figure 2: Mutual information against empirical gap closed. Panel (a) is the analytic Gaussian-Linear sweep with K=4 discrete contexts, where the empirical gap closed tracks I(δ ∗ ; z)/ log K almost exactly and recovers the full Oracle as I(δ ∗ ; z) → log K. Panel (b) is the SLCP benchmark with a diffusion-NPE backbone and shows a finite-sample minimum-I threshold near 0.5 nats below which corrector estimation noise overwhelms the weak side-channel signal, producing the dip at the noisy-z regime.
4.2
Sub-Gaussian extension
Theorem 1’s I-MMSE relation is sharp only for Gaussian noise. The constant C = 2τ 2 extends to all sub-Gaussian δ ∗ by Donsker–Varadhan; strong log-concavity admits a cleaner Bakry–Émery → T2 derivation (Appendix B) but is not necessary. ⊤ ∗ ∗ 2 2 Theorem 2 (Sub-Gaussian). Suppose δ ∗ is sub-Gaussian with proxy τ 2 , i.e., E eu (δ −Eδ ) ≤ e∥u∥ τ /2 for all u ∈ Rd . Let z be any random variable with I(δ ∗ ; z) < ∞. Then Var(E[δ ∗ | z]) ≤ 2τ 2 · I(δ ∗ ; z).
(5)
If additionally δ̂ψ⋆ (z) := E[δ ∗ | z] − E[δ ∗ ] and µ(y) := Eq̂ [θ | y] is L-Lipschitz, then E ∥µ(yobs ) − µ(yobs − δ̂ψ⋆ (z))∥22 ≤ 2L2 τ 2 · I(δ ∗ ; z). Proof. Donsker–Varadhan with test f = t⟨u, δ ∗ −Eδ ∗ ⟩ plus the sub-Gaussian MGF gives t⟨u, E[δ ∗ | z]−E[δ ∗ ]⟩ ≤ t2 τ 2 ∗ 2 ∗ ∗ ∗ 2 +DKL (Pδ |z ∥Pδ ); optimising over t, sup over ∥u∥=1, squaring, and Ez yields Var(E[δ | z]) ≤ 2τ ·I(δ ; z). ⋆ □ The Lipschitz corollary is pointwise L-Lipschitz applied to δ̂ψ (z). The constant matches 2α2 σδ2 under τ 2 =σδ2 , L=α, and is essentially sharp: an empirical sweep across Rademacher, per-coordinate-bimodal, three-mode, and asymmetric two-mode constructions saturates at ratio 0.995. Bounded-noise specialisation in Appendix C.
4.3
Identifiability and IV analogy
Theorem 3 (Identifiability). Under standard regularity, the corrected posterior q̃(θ | y, z) is identifiable iff the joint map (θ, z) 7→ p(y | θ, z) is injective on Θ × supp(z). The conditions translate to the IV assumptions [1]: exogeneity (z independent of unmodelled confounders), exclusion (z affects y only through δ), relevance (I(δ ∗ ; z) > 0). Our Exclusion Test (Sec. 5.5) empirically verifies exogeneity and exclusion; Theorem 1 formalises relevance. A counterexample where z = g(θ) shadows θ is ruled out by the Exclusion Test. Full proof in Appendix D.
4.4
MA-SBI strictly generalises RoPE
Theorem 4 (MA-SBI generalises RoPE). When z is restricted to calibration-indexed form zi = ei ∈ {1, . . . , N }, MA-SBI’s learned corrector converges to RoPE’s OT-based correction as N → ∞. For unrestricted z (e.g., text), MA-SBI extends RoPE’s expressiveness to regimes RoPE cannot represent.
6
Empirical verification on SLCP. With zi = ei (one-hot calibration index), MA-SBI-CI tracks RoPEEMD and converges as N → ∞; MA-SBI-TEXT (4 semantic templates) beats both at the same ncal (+105.2% vs +18.0% at ncal =500). Text supplies a transferable inductive bias over regimes that one-hot indexing cannot; sample efficiency is N/K rather than 1. Numbers in Appendix J.
5
Experiments
5.1
Setup
Experiments use the SBI benchmark suite (sbibm) [16], adapted with regime-indexed misspecification using K=4 discrete contexts that each apply a deterministic shift to the simulator output. Text templates: 4 “informative” sentences describing the regime, plus 4 “uninformative” filler sentences. Three regimes per benchmark: well-specified, misspec-inform, and misspec-uninform. Primary metric: C2ST [15] between approximate posterior samples and sbibm reference samples (lower is better; 0.5 is indistinguishable). All experiments use the diffusion (DDPM) backbone unless otherwise noted. A continuous-z extension confirming the corrector generalises beyond categorical regimes is in Appendix R. Headline gap-closure results across all methods and benchmarks are summarised in Table 1 (raw C2ST values are tabulated in Appendix M); per-benchmark calibration-size sweeps are in Appendix O (Table 10). Table 1: Main results. Gap closed toward Oracle across all methods and benchmarks at ncal =500 where applicable, with MA-SBI (text z only, no (θ∗ , y ∗ )) achieving near-oracle recovery on four of five benchmarks and remaining complementary to RoPE on the structural-permutation SIR benchmark. Methods marked † require ground-truth parameter pairs. The reported metric is C2ST against the reference posterior on sbibm tasks and posterior-mean MSE on the DDM application, while on real COVID + OxCGRT data (Sec. 5.7) the stochastic-bootstrap variant achieves a PPC NLL of 3.16 against 6.93 for vanilla NPE (+54%, n=284). RoPE entries use the simplified observation-space OT variant on the DDPM backbone for cross-benchmark consistency, with the faithful Algorithm 1 comparison and 10-seed flow-NPE results reported in Table 2. sbibm benchmarks (C2ST)
Applications
Method (uses)
GL (d=5) SLCP (d=5) Two Moons SIR delay DDM (d=3)
NPE (baseline) NNPE† (noise-aug) NPE+z-concat† (z as NPE input) RoPE-EMD† RoPE-Sinkhorn† (tuned ϵ) g(z) → θ † (direct) g(z) → y † (direct) MA-SBI (z only, no θ ∗ ) Oracle (neural)
0% −2%a −25% 9% +18% −776% −462% 91% 100%
0% 20% −425% 28% 22% −448% −374% 93% 100%
0% −21% −28% −2% +4% −36% −37% 95% 100%
0% +97% +100%b 99.8% +98% —c —c 87% 100%
0% 99% 24% 80% +2% −824% −1025% 101% 100%
a NNPE with L -matched noise scale; default per-dim scale gave −88% (Appendix T). 2 b d =2, unimodal posterior; concat succeeds only in this low-dim regime (Appendix T). θ c K=4 hash templates with d =2; direct regression is rank-degenerate. θ
Baseline-specific notes. Full per-benchmark analysis of baseline anomalies (NNPE scale matching, NPE+z-concat dimension dependence) is in Appendix T.
5.2
Hide-the-calibration: headline result
RoPE with full (θ∗ , y ∗ ) calibration is compared against MA-SBI with text only on SLCP, across 3 test observations and 4 contexts. Since the original RoPE paper [30] releases no public code, Algorithm 1 is re-implemented from the published description (a faithful 5-step pipeline including embedding-space entropic OT), alongside a simplified observation-space OT variant (RoPE-EMD, Hungarian + k=5 barycentric kNN) as the practitioner-deployable comparison. Both share the flow-NPE backbone (zuko MAF, 8 transforms). Full implementation details appear in Appendix J, and code will be released on acceptance. The faithful pipeline is evaluated with and without Step 2 encoder fine-tuning. Across 10 seeds at ncal =500 on flow-NPE,
7
a paired TOST equivalence test at margin ±0.02 C2ST is applied against Oracle, and the same result is confirmed on the DDPM backbone (Appendix F). Table 2: SLCP hide-the-calibration, 10 seeds, ncal =500, Ns =2000, flow-NPE backbone. C2ST vs sbibm reference posterior (lower better). TOST equivalence test against Oracle at ±0.02 margin (paired across seeds; null = “not equivalent”, so small p rejects the null and concludes equivalence). gc is computed per-seed and then summarised (median and IQR of the per-seed gc distribution), not as gc evaluated at the median C2ST. MA-SBI is the only method that passes the equivalence test. Method
median C2ST
IQR
gc median (IQR)
TOST p
0.758 0.762 0.892 0.903 0.719 0.714
[0.755, 0.762] [0.757, 0.778] [0.881, 0.895] [0.897, 0.906] [0.712, 0.725] [0.708, 0.726]
0% (–) −5% [−25, +17] −313% [−363, −222] −336% [−404, −255] +93.6% [+84, +97] 100% (–)
0.999 0.999 1.000 1.000 < 0.0001 –
NPE (biased) RoPE-EMD (simplified) RoPE-Faithful (no Step 2) RoPE-Faithful (with Step 2) MA-SBI (z only) Oracle (known δ)
The paired (M A − SBI − Oracle) difference has mean +0.003, std 0.004, 95% CI [+0.001, +0.006] — well inside the ±0.02 margin. TOST p < 0.0001 rejects non-equivalence: MA-SBI with text alone is statistically equivalent to the oracle on the flow-NPE backbone. Faithful Algorithm 1 collapses to C2ST ≈ 0.90 regardless of Step 2. Faithful Algorithm 1 collapses at small No . Faithful Algorithm 1 plateaus at C2ST ≈ 0.90 on SLCP across Ns ∈ [200, 5000] at fixed No = 12, with or without Step 2 fine-tuning (Appendix J). The failure is structural: Algorithm 1’s transductive mixture-posterior step couples each test observation to a weighted set of simulator runs via the OT matrix, and under small No no xjs has a posterior overlapping the test observation’s, so the mixture becomes a soft k-NN over wrong posteriors. Published RoPE evaluates with No in the hundreds [30]; scaling No from 12 to 200 partially recovers C2ST (0.892 → 0.856 no-Step 2, 0.903 → 0.863 with-Step 2) but does not reach NPE-equivalence at this calibration size. The simplified observation-space OT variant (RoPE-EMD) avoids this transductive scale dependence by operating on per-observation OT pull-back, recovering close to vanilla NPE performance (−5% gap closed); MA-SBI recovers 93.6%. The two methods sit on different deployment scales (Figure 3). A per-seed violin visualisation of the DDPM-backbone result is in Appendix F (Figure 4); Two Moons corrected-posterior visualisations are in Appendix G. Ablations. Two supervised shortcuts that bypass the simulator (g(z) → θ and g(z) → y, regressed from calibration) collapse to C2ST ≥ 0.99: with 64-d z and only 500 examples, both overfit to non-Bayesian point estimates that C2ST trivially separates from the reference. NPE+z-concat, given [y, z] plus ground-truth θ on the calibration set (a strictly stronger data assumption than MA-SBI), still collapses to C2ST ≈ 0.95 on SLCP at ncal =500, because the model memorises z → θ when z is high-dimensional. MA-SBI’s input correction y − δ̂ψ (z) forces δ̂ψ to predict the low-dimensional δ rather than the high-dimensional θ; the architectural restriction is what makes the small-calibration regime work. NNPE [29] succeeds on DDM (99%) where misspecification is small and approximately additive, but only reaches 20% on SLCP where the shifts exceed its spike-and-slab noise model (Appendix T).
5.3
Three-way decomposition
The Marginal = E[δ̂ψ (z)] matches RoPE’s average shift; the Conditional = δ̂ψ (z) − E[δ̂ψ (z)] is unique to MA-SBI. Gap-closure on SLCP is Marginal 5% + Conditional 45% = Total 93%; on Two Moons the Marginal alone is harmful (−13%) but Conditional + non-linear recombination land at +95% — the decomposition is attributional, not algebraic (Appendix P, Table 11, Figure 8; analytic Gaussian-Linear rows in Appendix L).
5.4
Complementary-regime finding (SIR)
Weekend-delay reporting on an SIR simulator with four context patterns (no delay, weekend-batched, uniform lag, Friday-dump) is a deterministic permutation, exactly recoverable from (θ∗ , y ∗ ) pairs. RoPE closes 99% of 8
Figure 3: Gap closed against calibration size ncal on three benchmarks under the hide-the-calibration protocol. MA-SBI leads on SLCP and Gaussian-Linear because the misspecification is smooth in z and recoverable by the corrector. RoPE leads on the SIR weekend-delay setting because the misspecification is a deterministic permutation that is recoverable from (θ∗ , y ∗ ) pairs but not from text alone, so the two methods are complementary rather than competing. the NPE→Oracle gap at ncal =500; MA-SBI closes 87% because hash-encoded text cannot match the channel capacity of parameter calibration in this regime — the two methods are complementary, not competing (Appendix H).
5.5
Diagnostics: degradation, exclusion, backbone, compute
Graceful degradation (Figure 2(b), SLCP noisy-z): at pure-noise z, gap closed is 5.2% (within ±5% of zero, per Corollary 1); a finite-sample dip at p=0.25 (−36%) marks a minimum-I(δ ∗ ; z) threshold below which corrector estimation error dominates. Exclusion: R2 (ysim | δ̂ψ (z)) < 0.10 (IV weak-instrument standard); all four sbibm benchmarks pass with |R2 | ≤ 0.001 (Appendix Q, Table 12). Backbones: the same δ̂ψ (z) on MAF and DDPM gives gap closure within 5pp on GL, SLCP, Two Moons (Appendix K); 10-seed flow-NPE equivalence (Table 2) provides the statistical confirmation on the second density-estimation family. Compute: MA-SBI adds ∼1 s over vanilla NPE, per-test-case inference unchanged (Appendix I).
5.6
Drift-Diffusion Model with instruction-text side-channel
A 3-parameter DDM [20, 4] with θ = (v, a, τ ) and a 20-d Gaussian-KDE RT summary is fit on a balanced condition, with four instruction-text templates indexing speed-accuracy regimes through additive RT shifts (Appendix E). FMCPE [22] requires θ∗ and reduces to the RoPE / FRISBI bucket under this protocol. MA-SBI with text alone closes the NPE-Oracle gap at every ncal ∈ {100, 500, 2000} (103.9%, 100.6%, 98.8% posterior-mean MSE), while RoPE-OT with full (θ∗ , y ∗ ) lags at 7.4%, 80.3%, 75.8% (Appendix N). The decomposition on DDM mirrors that on SLCP. Under informative text the Conditional residual carries 74% of the gap and the Marginal 20%, whereas under uninformative text the Conditional collapses to −3% (Appendix P, Table 11).
5.7
Real-data validation
Two real datasets bracket the operating regime: COVID + OxCGRT (K=4, 284 14-day windows from 4 countries [18]) and Evans–Hawkins random-dot-motion (OSF 2vnam, 21 subjects, 2 feedback-delay regimes [6]). The stochastic variant cuts per-cell PPC NLL from 6.93 → 3.16 on COVID (+54.4%, paired Wilcoxon p < 10−4 , n=284) by absorbing within-regime variation as posterior breadth. On Evans–Hawkins the
9
simulator is already well-specified (I(δ ∗ ; z) ≈ 0); the variant returns ∆NLL = −2.8%, indistinguishable from NPE. This realises the Corollary 1 prediction on real data and confirms the pre-flight criterion of Sec. 3.3.
6
Discussion and conclusion
Text side-channels recover oracle posteriors when misspecification is regime-conditional, with bias reduction bounded by I(δ ∗ ; z) (Theorem 1). Semantic templates beat one-hot calibration indexing because text is a transferable structured prior. The resulting decision rule is straightforward. Calibration with (θ∗ , y ∗ ) pairs points to RoPE or FRISBI, regime-level z points to MA-SBI, and the absence of either falls back to NPE or NNPE. Limitations. MA-SBI requires I(δ ∗ ; z) to dominate within-regime variability, so on already well-specified simulators it correctly reports near-zero gap closure (Corollary 1) but cannot improve over NPE. The TOST equivalence is benchmarked under the small-No hide-the-calibration protocol, and faithful RoPE may close the gap in larger-No regimes. Theorem 2 covers all sub-Gaussian δ ∗ , while heavy-tailed, hierarchical-z, and learned LLM-embedded encoders remain open.
References [1] Joshua D. Angrist, Guido W. Imbens, and Donald B. Rubin. Identification of causal effects using instrumental variables. Journal of the American Statistical Association, 91(434):444–455, 1996. [2] Rina Foygel Barber, Emmanuel J. Candès, Aaditya Ramdas, and Ryan J. Tibshirani. Conformal prediction beyond exchangeability. Annals of Statistics, 51(2):816–845, 2023. [3] Mark A. Beaumont, Wenyang Zhang, and David J. Balding. Approximate bayesian computation in population genetics. Genetics, 162(4):2025–2035, 2002. [4] Jan Boelts, Jan-Matthis Lueckmann, Richard Gao, and Jakob H. Macke. Flexible and efficient simulationbased inference for models of decision-making. eLife, 11:e77220, 2022. [5] Kyle Cranmer, Johann Brehmer, and Gilles Louppe. The frontier of simulation-based inference. Proceedings of the National Academy of Sciences, 117(48):30055–30062, 2020. [6] Nathan J. Evans and Guy E. Hawkins. When humans behave like monkeys: Feedback delays and extensive practice increase the efficiency of speeded decisions. Cognition, 184:11–18, 2019. Data: OSF project 2vnam. [7] Tomas Geffner, George Papamakarios, and Andriy Mnih. Compositional score modeling for simulationbased inference. In ICML, 2023. [8] David S. Greenberg, Marcel Nonnenmacher, and Jakob H. Macke. Automatic posterior transformation for likelihood-free inference. In ICML, 2019. [9] Joeri Hermans, Volodimir Begy, and Gilles Louppe. Likelihood-free mcmc with amortized approximate ratio estimators. In ICML, 2020. [10] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In NeurIPS, 2020. [11] Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598, 2022. [12] Daolang Huang, Ayush Bharti, Amauri Souza, Luigi Acerbi, and Samuel Kaski. Learning robust statistics for simulation-based inference under model misspecification. In NeurIPS, 2023. [13] Alexander Kraskov, Harald Stögbauer, and Peter Grassberger. Estimating mutual information. Physical Review E, 69(6):066138, 2004. 10
[14] Nian Li, Chen Gao, Mingyu Li, Yong Li, and Qingmin Liao. EconAgent: Large language modelempowered agents for simulating macroeconomic activities. In ACL, 2024. [15] David Lopez-Paz and Maxime Oquab. Revisiting classifier two-sample tests. In ICLR, 2017. [16] Jan-Matthis Lueckmann, Jan Boelts, David Greenberg, Pedro Goncalves, and Jakob Macke. Benchmarking simulation-based inference. In AISTATS, pages 343–351, 2021. [17] Jan-Matthis Lueckmann, Pedro J. Gonçalves, Giacomo Bassetto, Kaan Öcal, Marcel Nonnenmacher, and Jakob H. Macke. Flexible statistical inference for mechanistic models of neural dynamics. In NeurIPS, 2017. [18] Michael J. Lydeamore, Cameron Zachreson, Eamon Conway, Freya M. Shearer, Christopher M. Baker, Joshua V. Ross, Joel Miller, James M. McCaw, Nicholas L. Geard, Jodie McVernon, and David J. Price. Border quarantine, vaccination and public health measures to mitigate the impact of COVID-19 importations in Australia: a modelling study. Journal of the Royal Society Interface, 23(235):20250144, 2026. [19] George Papamakarios and Iain Murray. Fast ε-free inference of simulation models with Bayesian conditional density estimation. In NeurIPS, 2016. [20] Roger Ratcliff. A theory of memory retrieval. Psychological Review, 85(2):59–108, 1978. [21] Yaniv Romano, Rina Foygel Barber, Chiara Sabatti, and Emmanuel J. Candès. With malice toward none: Assessing uncertainty via equalized coverage. Harvard Data Science Review, 2(2), 2020. [22] Pierre-Louis Ruhlmann, Pedro L. C. Rodrigues, Michael Arbel, and Florence Forbes. Flow matching for robust simulation-based inference under model misspecification. arXiv preprint arXiv:2509.23385, 2025. [23] Ortal Senouf, Antoine Wehenkel, Cédric Vincent-Cuaz, Emmanuel Abbé, and Pascal Frossard. Inductive domain transfer in misspecified simulation-based inference. In NeurIPS, 2025. [24] Louis Sharrock, Jack Simons, Song Liu, and Mark Beaumont. Sequential neural score estimation: Likelihood-free inference with conditional score based diffusion models. In ICML, 2024. [25] Scott A. Sisson, Yanan Fan, and Mark M. Tanaka. Sequential monte carlo without likelihoods. Proceedings of the National Academy of Sciences, 104(6):1760–1765, 2007. [26] Alvaro Tejero-Cantero, Jan Boelts, Michael Deistler, Jan-Matthis Lueckmann, Conor Durkan, Pedro J. Gonçalves, David S. Greenberg, and Jakob H. Macke. sbi: A toolkit for simulation-based inference. Journal of Open Source Software, 5(52):2505, 2020. [27] Ryan J. Tibshirani, Rina Foygel Barber, Emmanuel J. Candès, and Aaditya Ramdas. Conformal prediction under covariate shift. In NeurIPS, pages 2526–2536, 2019. [28] Xinlei Wang, Maike Feng, Jing Qiu, Jinjin Gu, and Junhua Zhao. From news to forecast: Integrating event analysis in LLM-based time series forecasting with reflection. In NeurIPS, 2024. [29] Daniel Ward, Patrick Cannon, Mark Beaumont, Matteo Fasiolo, and Sebastian M. Schmon. Robust neural posterior estimation and statistical model criticism. In NeurIPS, 2022. [30] Antoine Wehenkel, Juan L. Gamella, Ozan Sener, Jens Behrmann, Guillermo Sapiro, Jörn-Henrik Jacobsen, and Marco Cuturi. Addressing misspecification in simulation-based inference through datadriven calibration. In ICML, 2025. [31] Yang Yang, Severi Rissanen, Paul E. Chang, Nasrulloh Loka, Daolang Huang, Arno Solin, Markus Heinonen, and Luigi Acerbi. Priorguide: Test-time prior adaptation for simulation-based inference. In ICLR, 2026.
11
A
Proof of Theorem 1
We work in the Gaussian-Linear setup: θ ∼ N (0, σp2 Id ), y = θ+ε, ε ∼ N (0, σs2 Id ), yobs = y+δ, δ ∼ N (0, σδ2 Id ), and z jointly Gaussian with δ with canonical correlations ρ1 , . . . , ρd . Step 1 — Posterior mean. The well-specified posterior on θ given y is Gaussian with mean µpost (y) = α·y, where α = σp2 /(σp2 + σs2 ), and covariance Σpost = (σp−2 + σs−2 )−1 Id . Step 2 — Uncorrected excess MSE. Vanilla NPE computes µpost (yobs ) = α(y + δ) instead of αy. The excess posterior-mean MSE is ∆MSEunc = E∥αδ∥2 = α2 · dσδ2 . Step 3 — Corrected excess MSE. Given any measurable δ̂ψ : Z → Rd of the side-channel, the corrected estimator is α(yobs − δ̂ψ (z)). The excess MSE is ∆MSEcor = α2 E∥δ − δ̂ψ (z)∥2 ≥ α2 · MMSE(δ | z), with equality when δ̂ψ = E[δ | z]. Step 4 — Rate-distortion on the Gaussian source. For jointly Gaussian (δ, z) with canonical correlations ρk , standard linear-algebra gives X X (1 − ρ2k ), I(δ ∗ ; z) = − 12 log(1 − ρ2k ). MMSE(δ | z) = σδ2 k
k
Apply Jensen’s inequality (concavity of log):
1 2 k log(1 − ρk ) ≤ d log d
P
P
2 k (1 − ρk )
rearranges to
X ∗ (1 − ρ2k ) ≥ d · exp − 2I(δd ;z) . k
Step 5 — Bound.
Combining: ∆MSEunc −∆MSEcor ≤ α2 · dσδ2 − α2 · MMSE(δ | z) ! X 2 2 2 = α σδ d − (1 − ρk ) ≤α
2
k 2 · dσδ · [1 − exp(−2I(δ ∗ ; z)/d)].
Using the tangent-line bound 1 − e−u ≤ u with u = 2I(δ ∗ ; z)/d: α2 · dσδ2 · [1 − exp(−2I(δ ∗ ; z)/d)] ≤ 2α2 σδ2 · I(δ ∗ ; z), establishing the linear form with C = 2α2 σδ2 .
□
Beyond Gaussian noise. The Gaussian-linear chain in Steps 1–5 uses the I-MMSE relation which is sharp only under Gaussian observation noise. For general log-concave noise families (Laplace, truncated Gaussian, smooth unimodal sub-Gaussian), the same scaling C · I(δ ∗ ; z) persists with an appropriate Lipschitz constant on the posterior-mean estimator; Theorem 2 gives the cleaner C = 2τ 2 bound that bypasses the per-likelihood Lipschitz computation entirely under a sub-Gaussian assumption on δ ∗ . Numerical verification. Table 3 reports empirical vs theoretical reductions for 200,000 joint (δ, z) draws across four canonical correlation regimes. The exact bound is tight in the uniform and oracle regimes (within Monte Carlo noise). The linear bound is loose for large I(δ ∗ ; z)/d (oracle), as expected from the Taylor inequality. 12
Table 3: Numerical verification of Theorem 1 (d = 5, σp = 0.5, σs = 0.3, σδ = 0.4, n = 200,000). ρ pattern zero (ρ = 0) graded (ρ ∈ [0.1, 0.9]) uniform (ρ = 0.5) oracle (ρ → 1)
B
I(δ ∗ ; z) (nats)
empirical reduction
exact bound
linear C · I(δ ∗ ; z) bound
0.00 1.36 0.72 15.54
0.00001 0.1427 0.1082 0.4322
0.00000 0.1818 0.1081 0.4317
0.00000 0.2358 0.1244 2.6882
Proof of Theorem 2
The main-text proof via Donsker–Varadhan is short and works for the full sub-Gaussian class. The Bakry– Émery → Otto–Villani T2 derivation is retained here as an alternative route under the stronger strongly-logconcave hypothesis, since it gives a cleaner W2 statement and a different geometric intuition. Step 1 — Bakry-Émery ⇒ log-Sobolev. By the Bakry-Émery criterion (Bakry-Émery 1985; BakryGentil-Ledoux 2014, Thm. 5.7.4): if −∇2 log pδ ⪰ (1/τ 2 )Id , then pδ satisfies a log-Sobolev inequality (LSI) with constant τ 2 : Entpδ (f 2 ) ≤ 2τ 2 · Epδ [∥∇f ∥2 ] for all smooth f ≥ 0. Step 2 — LSI ⇒ Talagrand T2. 2τ 2 · DKL (Q∥pδ ) for all Q ≪ pδ .
By Otto-Villani (2000): LSI with constant τ 2 implies W22 (Q, pδ ) ≤
Step 3 — Apply T2 to conditional measures. Substitute Q = Pδ|z=ζ and take expectation over z: Ez W22 (Pδ|z , pδ ) ≤ 2τ 2 · Ez DKL (Pδ|z ∥pδ ) = 2τ 2 · I(δ ∗ ; z). Step 4 — Mean-shift lower bound on W2 . For any P, Q with means µP , µQ , W22 (P, Q) ≥ ∥µP − µQ ∥2 (Jensen on optimal coupling). So Ez ∥E[δ | z] − E[δ]∥2 ≤ Ez [W22 (Pδ|z , pδ )] ≤ 2τ 2 · I(δ ∗ ; z). The LHS is Var(E[δ | z]). □ Empirical sharpness sweep. Across per-coordinate-bimodal, Rademacher (δ = sτ with s ∼ Unif{±1}d independent), three-mode {−m, 0, +m}, and asymmetric two-mode constructions (coupling zk = ρk δk + p 1 − ρ2k ηk with ηk ∼ N (0, 1) independent; ρ ∈ [0.3, 0.99]; n ∈ [200,000, 300,000] with KSG MI estimator [13]), the ratio Var(E[δ | z])/(2τ 2 · I(δ ∗ ; z)) stays in [0.33, 0.995], saturating at ≥ 0.99 on Rademacher and per-coordinate-bimodal at ρ → 0. The constant C = 2τ 2 is essentially sharp for sub-Gaussian.
C
Bounded-noise distribution-free extension
The original distribution-free bound is retained for completeness, applying to bounded δ ∗ without log-concavity assumptions but at a looser constant. Theorem 5 (Bounded-noise extension). Suppose (B1) ∥δ ∗ ∥2 ≤ M almost surely with E[δ ∗ ] = 0 (centring is WLOG: the marginal mean is absorbed into the constant Marginal component of the three-way decomposition, Sec. 3.4), and (B2) the amortized posterior mean µ(y) := Eq̂ [θ | y] is L-Lipschitz in y in ℓ2 . Let δ̂ψ⋆ (z) := E[δ ∗ | z] denote the population-optimal z-indexed corrector. Then E ∥µ(yobs ) − µ(yobs − δ̂ψ⋆ (z))∥22 ≤ 2L2 M 2 · I(δ ∗ ; z),
(6)
and equivalently in δ-space, Var(E[δ ∗ | z]) ≤ 2M 2 · I(δ ∗ ; z). Proof. Fix z0 in the support of z and a unit vector u ∈ Rdim y . Since |u⊤ δ ∗ | ≤ M a.s., the variational characterisation of total variation gives |u⊤ (E[δ ∗ | z = z0 ] − E[δ ∗ ])| ≤ 2M · dT V (Pδ∗ |z0 , Pδ∗ ). Taking the supremum over unit u and squaring, ∥E[δ ∗ | z = z0 ]∥22 ≤ 4M 2 · d2T V (Pδ∗ |z0 , Pδ∗ ), 13
where we used E[δ ∗ ] = 0. Taking expectation over z, applying Pinsker’s inequality d2T V ≤ 12 KL pointwise, and using Ez [KL(Pδ∗ |z ∥ Pδ∗ )] = I(δ ∗ ; z) = I(δ ∗ ; z), Var(E[δ ∗ | z]) = Ez ∥E[δ ∗ | z]∥22 ≤ 4M 2 · 21 I(δ ∗ ; z) = 2M 2 I(δ ∗ ; z). By (B2), ∥µ(yobs )−µ(yobs − δ̂ψ⋆ (z))∥2 ≤ L ∥δ̂ψ⋆ (z)∥2 pointwise, so E ∥µ(yobs )−µ(yobs − δ̂ψ⋆ (z))∥22 ≤ L2 Var(E[δ ∗ | z]) ≤ 2L2 M 2 I(δ ∗ ; z). Corollary 3 (Sub-Gaussian extension via truncation). If δ ∗ is sub-Gaussian with proxy τ 2 Id , then √ for any ∗ constant c ≥ 1, applying Theorem 5 to the truncation δM := δ ∗ 1{∥δ ∗ ∥2 ≤ M } with radius M = cτ d and absorbing the truncation-tail residual (exponentially small in c2 by sub-Gaussian concentration) yields the linear-in-I(δ ∗ ; z) bound 2 E ∥µ(yobs ) − µ(yobs − δ̂ψ⋆ (z))∥22 ≤ 2c2 L2 τ 2 d · I(δ ∗ ; z) + O L2 dτ 2 e−c /2 . Choosing c = O(1) (e.g., c=3, residual ≲ 10−2 · L2 dτ 2 ) gives a sub-Gaussian linear bound with constant 2c2 L2 τ 2 d, matching the linear-in-I(δ ∗ ; z) scaling of Theorem 1 up to the loose constant.
D
Proof of Theorem 3
Sufficiency (injective ⇒ identifiable). rule gives
Suppose (θ, z) 7→ p(y | θ, z) is injective on the support. Bayes’
q̃(θ | y, z) =
p(y | θ, z)π(θ) , Z(y, z)
R where Z(y, z) = p(y | θ′ , z)π(θ′ )dθ′ . Since the numerator is unique at each θ and the denominator is a function of (y, z) only, the posterior is uniquely determined by (y, z). Necessity (identifiable ⇒ injective). We prove the contrapositive. Suppose the joint map is not injective: there exist (θ1 , z1 ) ̸= (θ2 , z2 ) with p(y | θ1 , z1 ) = p(y | θ2 , z2 ) for almost every y. Case A (z1 = z2 = z0 , θ1 ̸= θ2 ). The z-slice likelihood is not injective in θ; the posterior is ambiguous at any y in the common support of θ1 and θ2 . Identifiability fails directly. Case B (z1 ̸= z2 ). Swapping (θ1 , z1 ) ↔ (θ2 , z2 ) leaves the joint distribution unchanged; the posterior is not recoverable from the joint law. □ The “z shadows θ” degenerate regime. If z = g(θ) is a bijection, the joint (θ, z) 7→ p(y | θ, z) is technically injective, but the posterior q̃(θ | y, z) = δ(θ − g −1 (z)) is a delta function at the z-inferred θ, bypassing the observation channel entirely. This is not strict non-identifiability but is operationally equivalent: the side-channel has overtaken the observation channel and the simulator is redundant. The Exclusion Test (Sec. 5.5) rules this out by verifying δ̂ψ (z) predicts δ rather than y directly.
E
Additional experimental details
Text encoders. The default is a 64-dimensional hash encoder (hashed character 3-grams into 64 buckets with ±1 signs), since it is offline-deterministic and portable. Results with sentence-transformers all-MiniLM-L6-v2 (384-dim) remain within 2 C2ST points on all benchmarks, indicating that the corrector is robust to the z-encoding choice. Diffusion backbone hyperparameters. Linear β-schedule from 10−4 to 0.02 over T = 200 steps. Sinusoidal time embedding dimension 64. 4-layer MLP with GELU activations, hidden dimension 256. Adam optimizer, learning rate 10−3 , batch size 256, 30 epochs. No classifier-free guidance at evaluation time (i.e., λ = 0); we rely on the pre-conditioning input correction from Eq. (1).
14
Algorithm 1 MA-SBI training and inference Require: Simulator f , side-channel encoder encz , calibration triples C = {(zi , ysim,i , yobs,i )}N i=1 Training time: 1: Train q̂(θ | y) on (θ ∼ π, ysim = f (θ, ε)) pairs ▷ DDPM or MAF 2: Train δ̂ψ on C via Eq. (2) Inference (given yobs , ztest ): 3: zemb ← encz (ztest ) 4: δhat ← δ̂ψ (zemb ) 5: Sample θs ∼ q̂(θ | yobs − δhat ) for s = 1 . . . S 6: return posterior samples {θs } Corrector hyperparameters. 3-layer MLP, GELU, 128 hidden units. Adam optimizer, learning rate 10−3 , batch size 256, 30 epochs. Zero dropout. No weight decay for reported results (results are robust to small weight decay ≤ 10−4 ). DDM instruction-text templates (Sec. 5.6). The four templates are: “Balance speed and accuracy in your responses” (no shift, well-specified); “Respond as quickly as possible, even if you make mistakes” (leftward RT shift, error tail thickens); “Be as accurate as possible, take your time” (rightward RT shift, error tail shrinks); “Respond as fast as you can, no time to think carefully” (stronger leftward shift plus error-tail boost). Shift magnitudes are calibrated to qualitatively match published DDM speed-accuracy effects. Compute budget. All experiments run on an Apple M4 Pro laptop CPU; no GPU required. Full main-text experiments reproduce in approximately 12 CPU-hours. The heaviest individual run is the Sinkhorn ε sweep (∼ 1.5 hours due to 8 calibration fits and 96 posterior-sampling calls per ε value).
F
DDPM-backbone 10-seed replication
Table 4 reports the same hide-the- calibration 10-seed protocol on the DDPM backbone (in place of the flow-NPE backbone of Table 2). The simplified observation-space OT variant (RoPE-EMD) recovers a small positive gap closure on DDPM (+6% median) consistent with its single-seed behaviour, while the equivalence claim for MA-SBI holds on both backbones (TOST p < 0.0001, paired difference within the ±0.02 margin in both cases). Faithful Algorithm 1 was not evaluated on DDPM because the DDPM backbone lacks an explicit embedder hϕ as required by Step 5; faithful evaluation is reported on the flow-NPE backbone only (main text). Table 4: DDPM-backbone 10-seed replication of the hide-the- calibration protocol. Same protocol as Table 2 but on a DDPM backbone. RoPE-EMD (simplified) NPE (biased) MA-SBI (z only) Oracle (neural)
median C2ST
mean
std
IQR
0.770 0.779 0.748 0.744
0.775 0.778 0.744 0.743
0.016 0.012 0.018 0.017
[0.763, 0.784] [0.771, 0.787] [0.727, 0.756] [0.729, 0.753]
The paired difference (M A − SBI − Oracle) on DDPM has mean +0.002 and std 0.004, 95% CI [−0.001, +0.004] on the paired difference, well inside the ±0.02 equivalence margin. TOST yields p < 10−4 , and the equivalence claim holds on both backbones.
G
Posterior visualisation on Two Moons
Figure 5 visualises corrected posteriors on Two Moons across three regimes. Vanilla NPE drifts with the misspecification; MA-SBI tracks the reference and matches the oracle. 15
Figure 4: Per-seed C2ST distributions against the sbibm reference on SLCP under the hide-the-calibration protocol with 10 seeds and ncal =500. Panel (a) uses the flow-NPE backbone and panel (b) uses the DDPM backbone. Black dots are individual seeds, the green dotted line is the median Oracle C2ST, and the shaded band shows the ±0.02 TOST equivalence margin around the Oracle. The RoPE-Alg.1 column is split into the no-Step-2 and the with-Step-2 variants from Table 4. MA-SBI lies inside the equivalence band on both backbones, while RoPE-EMD sits at the NPE level and the two RoPE-Alg.1 variants land far above NPE.
H
SIR weekend-delay: complementary regime
Figure 6 visualises the two complementary regimes that underlie Sec. 5.4 and Sec. 5.6.
I
Compute and wall-clock
We isolate per-phase wall-clock cost on the SLCP hide-the-calibration protocol at ncal =500 with neval =12 test observations, 500 posterior samples per query, and a T =200 DDPM backbone. All measurements are taken on a single Apple M4 Pro laptop CPU with no GPU. Table 5 decomposes the cost into three phases: base posterior training (one-off), calibration fit (MA-SBI or RoPE-only), and per-query inference. The full project budget across all main-text experiments, including preliminary runs not reported in the paper, fits in approximately 12 CPU-hours on the same machine; the figures below cover one representative end-to-end SLCP run. Table 5: Compute cost on SLCP at ncal =500, neval =12, 500 posterior samples, T =200 diffusion steps, single CPU. Phase Base posterior training Calibration fit Inference per test case Total (12 cases)
NPE / vanilla
RoPE-EMD
MA-SBI
∼18 s — ∼2 s ∼42 s
∼18 s ∼0.3 s ∼2 s ∼42 s
∼18 s ∼1 s ∼2 s ∼43 s
MA-SBI’s calibration overhead of approximately 1 s reflects the single forward pass that fits the corrector δ̂ψ on the ncal =500 triples; this is roughly 3× RoPE-EMD’s Hungarian-OT solve at the same calibration size, but remains negligible relative to the 18 s spent training the base posterior. At inference time the three methods are indistinguishable: the corrector adds only a constant δ-shift to yobs before the standard NPE forward pass, with no extra sampling cost. The total budget is dominated by the base posterior training rather than by the misspecification correction, so adopting MA-SBI on top of an existing NPE pipeline incurs essentially no additional cost.
16
Figure 5: Posterior samples on Two Moons across three misspecification regimes evaluated at the sbibm reference observation xref 1 . Each panel overlays the sbibm reference posterior in grey, vanilla NPE in red, MA-SBI in blue, and the Oracle in green. Vanilla NPE drifts away from the reference as misspecification grows, while MA-SBI tracks the reference and matches the Oracle in all three regimes.
J
Faithful RoPE Ns convergence sweep
Faithful Algorithm 1 implementation. The five-step pipeline: (i) NPE training, (ii) sufficient-statistics encoder gψ fine-tuned on (θ⋆ , xo ) calibration via MSE to Eϵ [h⋆ϕ (S(θ, ϵ))], (iii) Ns =2000 fresh test-time simulations (10× our calibration size), (iv) semi-balanced entropic-regularised OT in embedding space P (σ=0.5, ϕ=0.9 per the original work), (v) mixture of NPE posteriors j Pij⋆ p̃(θ | h⋆ϕ (xjs )). Step 2 is ablated (run with and without). The simplified RoPE-EMD variant replaces (iv) with raw y-space Hungarian OT (scipy.optimize.linear_sum_assignment) plus a k=5 barycentric-kNN extension at test time. Both share the flow-NPE backbone. Implementations are provided in the supplementary code at src/ma_sbi/models/rope_full.py and src/ma_sbi/models/rope_ot.py. Table 6 reports the C2ST of faithful Algorithm 1 on SLCP at ncal =500, single seed. The top block sweeps the test-time simulation budget Ns from 200 to 5000 (a 25× range) at fixed No =12; the C2ST is flat across the entire sweep at ≈ 0.90, with and without Step 2. The bottom block fixes Ns =2000 and probes a single point at No =200 (a 16.7× No scale-up over the headline protocol); C2ST recovers to 0.856 (no-Step 2) and 0.863 (with-Step 2) — measurable improvement in the predicted direction, though still above NPE. Reference values for the canonical scoring set: NPE 0.775–0.778, MA-SBI 0.707, Oracle 0.701–0.705, simplified RoPE-EMD 0.769. Faithful Algorithm 1 does not approach vanilla NPE performance even at 16.7× No scale-up within ncal =500; the regime sensitivity to No is consistent with the structural-coupling diagnosis of Sec. 5.2.
K
Backbone ablation
Table 7 reports gap-closed on the informative-z regime across three benchmarks and two backbones. The corrector δ̂ψ (z) produces within-noise-equivalent gap closure on both density-estimator families, confirming that the transport is on posterior-conditioning-input space rather than coupled to the specific density estimator.
L
Three-way decomposition: analytic GL rows
For completeness, the analytic Gaussian-Linear rows of the three-way decomposition (referenced in Sec. 5.3):
17
Figure 6: Two complementary misspecification regimes. The left panel is SIR weekend-delay, where the misspecification is a structural permutation recoverable exactly from (θ∗ , y ∗ ) pairs, so RoPE closes 99% of the gap and MA-SBI closes 87% at ncal =500. The right panel is DDM with an instruction-text side channel, where the misspecification is smooth in z but not present in (θ∗ , y ∗ ) pairs, so MA-SBI dominates. Theorem 4 predicts the left-panel regime; the right panel motivates the side-channel formulation in Sec. 3. Table 6: Faithful RoPE Algorithm 1 C2ST on SLCP, ncal =500, single seed. Top block: Ns sweep at fixed No =12 — plateau at C2ST ≈ 0.90 across a 25× range, with or without Step 2. Bottom block: No probe at fixed Ns =2000 — a 16.7× No scale-up partially recovers C2ST (0.892 → 0.856 for no-Step 2; 0.903 → 0.863 for with-Step 2), confirming Algorithm 1’s documented regime sensitivity to No but not reaching NPE-equivalence at this calibration size. RoPE-Full no-Step 2
RoPE-Full with-Step 2
Ns sweep at fixed No =12: Ns =200 0.922 Ns =500 0.913 Ns =1000 0.905 Ns =2000 0.899 Ns =5000 0.909
0.919 0.917 0.911 0.907 0.904
No probe at fixed Ns =2000: No =200 0.856
0.863
Benchmark
Marginal gc
Conditional gc
Total gc
19% 19%
81% 0%
100% 19%
GL analytic (informative) GL analytic (noise-z)
Linear-Gaussian benchmarks exhibit the expected additive behaviour (19% + 81% = 100%); under noise-z the Conditional component collapses to 0% as predicted by Corollary 1.
M
Raw metric values
Table 8 reports the raw C2ST (GL, SLCP, Two Moons) and posterior-mean MSE (SIR, DDM) values underlying the gap-closed percentages in Table 1.
N
DDM hide-the-calibration full table
This appendix expands the DDM headline of Section 5.6 into the full hide-the-calibration sweep over calibration size ncal ∈ {100, 500, 2000}. The protocol mirrors SLCP: RoPE-OT receives full (θ∗ , y ∗ ) calibration pairs, 18
Table 7: Backbone-agnosticism. Gap closed toward the neural-NPE oracle. Backbone MAF flow (sbi.NPE) DDPM (ours)
GL (d=10)
SLCP (100k sims)
Two Moons
+106% (3 seeds) +111%
+101% +99%
+102% +102%
Table 8: Raw metric values at ncal =500. C2ST for GL / SLCP / Two Moons (lower better, 0.5 ideal); posterior-mean MSE on log θ for SIR; posterior-mean MSE on θ for DDM. Method NPE NNPE NPE+z-concat RoPE-EMD RoPE-Sinkhorn g(z) → θ g(z) → y MA-SBI Oracle
GL C2ST
SLCP C2ST
TM C2ST
SIR MSE
DDM MSE
0.557 0.558 0.570 0.544 0.547 0.962 0.798 0.509 0.504
0.784 0.768 0.951 0.772 0.777 0.996 0.994 0.745 0.742
0.924 0.967 0.981 0.928 0.915 0.999 1.000 0.726 0.716
40.41 1.066 0.064 0.091 0.770 — — 4.55 0.002
0.0455 0.0218 0.0374 0.0366 0.0451 0.2423 0.2903 0.0223 0.0217
while MA-SBI sees only the four instruction-text templates of Appendix E and never observes a parameter ground-truth. The metric is posterior-mean MSE on the 3-dimensional DDM parameter θ = (v, a, τ ) at the held-out test observation, and gap-closure is computed against the neural Oracle that does see θ∗ . Table 9: Hide-the-calibration on DDM. MA-SBI with text z alone recovers near-Oracle posteriors at every calibration size, while RoPE-OT with full (θ∗ , y ∗ ) pairs trails by 20 to 93 percentage points. Metric: posteriormean MSE on θ = (v, a, τ ). ncal
RoPE-OT
NPE
MA-SBI (z only)
Oracle
RoPE gc
MA-SBI gc
100 500 2000
0.0467 0.0249 0.0258
0.0489 0.0481 0.0491
0.0180 0.0190 0.0187
0.0192 0.0192 0.0184
7.4% 80.3% 75.8%
103.9% 100.6% 98.8%
MA-SBI closes the NPE-to-Oracle gap at every calibration size (103.9%, 100.6%, 98.8%), where the slight overshoot at ncal =100 reflects the corrector exploiting the finite-sample regularisation of the small calibration draw rather than recovering more than the Oracle. RoPE-OT improves with ncal but plateaus around 80%, consistent with the SLCP plateau reported in Table 2: the embedding-space optimal-transport step recovers a useful but incomplete picture of the four task-instruction regimes from θ∗ pairs alone, whereas MA-SBI reads the regime label directly from text and so does not pay the OT-induced calibration penalty. The DDM table therefore mirrors the SLCP finding under a different misspecification mechanism (additive RT shifts rather than location bias) and a different posterior geometry (3D parameter, 20-d Gaussian-KDE summary), supporting the claim that the side-channel mechanism is benchmark-agnostic. P Convergence sketch. With zi = ei (one-hot) and the ℓ2 corrector loss ψ̂ = arg minψ N1 i ∥δi − δ̂ψ (ei )∥2 , the unique minimiser is the per-index empirical mean: δ̂ψ⋆ (ei ) = δi when each index appears once, and P δ̂ψ⋆ (ek ) = |S1k | i∈Sk δi when the calibration set contains Sk samples sharing index k. The simplified RoPEEMD pull-back at test point ytest averages δj over the k=5 nearest calibration points in y-space; under the calibration-indexed protocol the nearest neighbours of ytest are precisely the calibration points sharing the same regime label as ytest , so the two correctors coincide on shared support. By the SLLN over the regime slices, δ̂ψ⋆ (ek ) → E[δ | regime = k] as the per-regime calibration count |Sk | → ∞, which is also the limit of RoPE-EMD’s per-regime barycentre. Hence the two correctors converge pointwise on {e1 , . . . , eK }. For unrestricted z, MA-SBI can represent functions δ̂ψ (z) that RoPE-EMD’s per-calibration-point lookup cannot, since text-encoded z identifies regimes the calibration set never visited. □ 19
Empirical verification. ncal
RoPE-EMD
MA-SBI-CI (cal-indexed z)
MA-SBI-TEXT (abstract z)
100 500
23.5% 18.0%
9.4% −6.0%
45.8% 105.2%
MA-SBI-CI’s −6.0% at ncal =500 reflects per-index overfitting: with one calibration example per one-hot index, the estimate is dominated by noise. As N grows, repeated regime draws deduplicate into per-regime evidence and MA-SBI-CI converges toward RoPE-EMD. MA-SBI-TEXT escapes this trap because the four templates serve all N points; sample efficiency is N/K rather than 1. Figure 7 plots the same comparison across a wider sweep of N and shows the asymptotic curves.
Figure 7: Empirical verification of Theorem 4. Panel (a) reports gap closure at ncal =100 and 500 for RoPE-EMD, the calibration-indexed variant MA-SBI-CI, and the abstract-text variant MA-SBI-TEXT, matching the table above. Panel (b) sweeps the calibration size N on a log axis and plots the asymptotic curves: MA-SBI-CI converges down to the RoPE-EMD asymptote at the rate predicted by the per-regime SLLN, while MA-SBI-TEXT saturates close to the Oracle because the four templates serve all N points and the effective per-regime sample count grows as N/K.
O
Hide-the-calibration ncal sweeps
Table 10: SLCP hide-the-calibration single-seed sweep (C2ST vs sbibm reference posterior; lower better, 0.5 indistinguishable). MA-SBI closes substantially more of the oracle gap than either OT variant of RoPE across all calibration sizes. The 10-seed flow-NPE result with TOST equivalence test is in Table 2. ncal
RoPE-EMD
RoPE-Sinkhorn (tuned)
NPE
MA-SBI (z only)
Oracle
100 500 2000
0.777 0.772 0.786
0.787 0.777 0.807
0.786 0.784 0.787
0.758 0.745 0.735
0.741 0.742 0.733
The DDM sweep is in main-text Table 9; below are GL, Two Moons, and SIR. Gaussian Linear. ncal
RoPE-EMD
MA-SBI
Oracle
100 500 2000
+50.7% +9.2% −15.1%
+114.5% +99.0% +120.5%
100% 100% 100%
20
Two Moons. ncal
RoPE-EMD
RoPE-Sinkhorn
MA-SBI
Oracle
100 500 2000
−11.7% −1.4% +3.5%
−5.6% +4.4% +5.2%
+68.4% +97.5% +98.7%
100% 100% 100%
ncal
RoPE-OT
MA-SBI
Oracle
100 500 2000
+99.7% +99.7% +99.8%
+54.0% +92.5% +95.8%
100% 100% 100%
SIR (complementary regime).
P
Three-way decomposition: all benchmarks Table 11: Three-way decomposition across all benchmarks and regimes. Benchmark GL (informative) GL (noise-z) SLCP (informative) SLCP (noise-z) Two Moons (neural) SIR (informative) SIR (uninformative) DDM (informative) DDM (uninformative)
Marginal gc
Conditional gc
Total gc
19% 19% 5% −2% −13% 79% 64% 20% 30%
81% 0% 45% 5% 10% 50% 4% 74% −3%
100% 19% 93% 4% 95% 95% 67% 94% 27%
Figure 8: Three-way decomposition of gap closure across the four sbibm benchmarks and the two real-data probes. The Marginal bar is what RoPE recovers and the Conditional bar is unique to MA-SBI; on Two Moons the Conditional bar is hatched to indicate that gap closure also reflects a non-linear recombination on top of the additive split. The decomposition is attributional rather than algebraic: the three components do not have to sum to the total, but together they show that RoPE captures the average shift while the regime-conditional component is what MA-SBI contributes.
21
Q
Exclusion Test: real-data probes
The Exclusion Test is the instrumental-variable diagnostic that underpins the identifiability argument of Theorem 3: the side-channel z should be predictive of the misspecification residual δ but not of the simulator output ysim on its own. Operationally, we regress each of the three quantities on z using a held-out split and report the coefficient of determination: a small R2 (ysim ) confirms that z carries no information about the clean simulator, so any predictive content of z acts only through the misspecification channel; a large R2 (δ) confirms that z does carry information about that channel; and R2 (yobs ) is reported for context, since it inherits both signals. We adopt the threshold R2 (ysim ) < 0.10 used in Section 5.2 as the pass criterion. Table 12 extends the test from the four sbibm benchmarks to the two real-data probes (COVID + OxCGRT and Evans–Hawkins). Table 12: Exclusion Test extended to real-data probes. All pass (R2 (ysim ) < 0.10). Benchmark
R2 (ysim )
R2 (yobs )
R2 (δ)
Pass
SLCP SIR DDM Two Moons COVID + OxCGRT Evans–Hawkins
−0.002 −0.001 +0.000 +0.001 +0.003 +0.000
+0.007 +0.49 +0.022 +0.053 +0.172 +0.061
1.00 0.70 1.00 1.00 0.752 1.000
✓ ✓ ✓ ✓ ✓ ✓
All six benchmarks pass. The diagnostic is implemented in src/ma_sbi/eval/leakage.py of the supplementary code. The simulator R2 is essentially zero on every row, so z is uncorrelated with ysim as required. The residual R2 (δ) is high on the four synthetic benchmarks where δ is fully recoverable from the regime label, and lower but still substantial on SIR (0.70) and COVID + OxCGRT (0.75) where the regime captures most but not all of the misspecification gap. The observation-level R2 (yobs ) tracks R2 (δ) as expected, since yobs = ysim + δ and z predicts the second term only. Combined with the gap-closure results of Table 1, this confirms that the identifiability assumption is met empirically across the full benchmark set, including real data.
R
Continuous-context experiment
To verify the corrector generalises beyond K=4 categorical regimes, we replace discrete templates with continuous z ∈ [0, 1] on SLCP. The misspecification is δ(z) = z · δmax · v where v ∈ R8 is a fixed random unit vector and δmax = 3.0. Text: "The regime intensity is approximately {z:.2f}." Calibration: z ∼ Uniform([0, 1]), ncal = 2000. Evaluation: 20-point z grid × 3 sbibm reference observations.
MA-SBI (continuous z)
Mean gc (z ≥ 0.3)
Mean ∥δ̂ − δ∥/∥δ∥ (z ≥ 0.3)
96.8%
12.3%
Above z ≥ 0.3 (where misspecification exceeds posterior noise), the corrector closes 97% of the NPEto-Oracle gap with 12% relative δ-recovery error. Below z < 0.3 the NPE–Oracle gap is < 0.03 C2ST and gap-closure ratios are noise-dominated. The result confirms MA-SBI’s corrector learns a smooth function of z, not a categorical lookup, and generalises to held-out continuous z values not seen during training. The full z grid is plotted in Figure 9.
S
Distribution-matching training variant
Eq. (2) is point-wise, relying on the paired-by-construction structure of synthetic calibration data. In real-data deployments where only regime-stratified pools {yobs,i }zi =z and {ysim,j }zj =z are available without sample-wise alignment, replace Eq. (2) with a per-regime distribution-matching loss (e.g., MMD between
22
Figure 9: Continuous-context generalisation on SLCP with z ∈ [0, 1] and ncal =2000. Panel (a) plots gap closure across a 20-point z grid against the three sbibm reference observations; the shaded region marks the noise-dominated band z < 0.3 where the NPE–Oracle gap is below 0.03 C2ST and gap-closure ratios are not informative. Panel (b) plots the relative δ-recovery error ∥δ̂ − δ∥/∥δ∥ on the same grid. Above the noise floor the corrector closes 96.8% of the NPE–Oracle gap and recovers δ within 12.3% relative error, confirming that the learned corrector is a smooth function of z rather than a categorical lookup over the calibration regimes. {yobs,i − δ̂ψ (z)} and {ysim,j }, or conditional-mean matching). The Evans–Hawkins evaluation (Sec. 5.7) uses the mean-matching variant.
T
Per-benchmark baseline analysis
NPE+z-concat succeeds on SIR but fails elsewhere. On SIR (dθ = 2, unimodal posterior), NPE+zconcat with θ∗ labels recovers the oracle (+100% gap closed). The same architecture collapses on Two Moons (dθ = 2, bimodal: −28%) and SLCP (dθ = 5: −425%). Two factors explain the difference. First, SIR’s posterior is approximately Gaussian, so MSE-trained regression recovers it; bimodal posteriors collapse to the inter-mode mean. Second, at dθ = 2 with ncal = 500, supervised regression has a 250:1 sample-to-parameter ratio, preventing the memorisation failure observed at higher dθ . MA-SBI’s input-correction architecture works across posterior shapes without requiring θ∗ labels. NNPE scale sensitivity on GL. NNPE’s default per-dimension noise σ = 0.3 in d = 5 produces training-noise L2 norm ≈ 0.67, overshooting the actual misspecification magnitude (∥δ∥ = 0.30) by 2.2×. With L2 -matched noise (σ = 0.134), NNPE recovers to NPE-equivalent performance (−2% gap closed vs the original −88%). Structurally, NNPE injects undirected isotropic noise that provides robustness but carries no regime-conditional information; Theorem 1 predicts zero gap closure from any z-free augmentation. NNPE on SIR and DDM. NNPE achieves +97% on SIR and +99% on DDM, where the misspecification is small relative to posterior variance and approximately within the spike-and-slab perturbation class. On SLCP (20%) and Two Moons (−21%), larger or rotational misspecification exceeds NNPE’s design regime.
23