RECAST: Model Reconstruction via Counterfactual-Aware Wasserstein Geometry under Limited Data
Xuan Zhao * 1 Lena Krieger * 1 2 Zhuo Cao 1 Arya Bangun 1 Hanno Scharr 1 Ira Assent 1 3 Class 0
Counterfactual explanations (CFs) help understand machine learning models by identifying minimal input changes that would lead to alternative model outcomes. Recent work demonstrates their utility for reconstructing black-box models, enabling third-party auditing of opaque decision systems for fairness and accountability. Still, CFbased reconstruction may suffer from decision boundary shifts, overfitting, and restrictive assumptions requiring online query access to target platforms. We propose REconstruction via Counterfactual-Aware waSserstein opTimization (RECAST) under limited data and restricted access, a behavioral surrogate model based on Wasserstein barycentric prototypes. Our approach addresses decision boundary shifts by incorporating CFs as informative, though less representative, samples for both classes, maintaining high surrogate fidelity in low-sample regimes without requiring online access during reconstruction. To enhance fairness auditing, our method enables systematic group fairness diagnostics. Experiments on real-world datasets and various setups show that RECAST effectively achieves high fidelity and query efficiency, as well as stable results even when the access is limited and noisy.
Class 1
CF
4 3
f2
arXiv:2606.27948v1 [cs.LG] 26 Jun 2026
Abstract
2 1 0 0.0
0.5
1.0
1.5
f1
2.0
2.5
3.0
3.5
Figure 1. CFs (x) are generated for class 0 (blue) samples - minimal changes that flip the target’s prediction from class 0 → class 1 (pink). Naively treating these as class 1 training samples shifts the surrogate’s boundary (solid line) toward class 0, causing class 0 samples to be misclassified as class 1. This is overconfidence: the surrogate becomes overly certain about class 1 in regions where the target (dashed line) is not. We instead treat CFs as less certain samples, giving them less influence than true class 1 samples.
behaviour of the target model, enabling independent investigation of model behavior, including transparency, accountability, and fairness concerns such as fairwashing (Aı̈vodji et al., 2021). Reconstruction under limited access is fundamentally a problem of stability: conclusions about model behavior should remain invariant across all models that are behaviorally indistinguishable from the observed data. While studied as model extraction attacks (MEA) (Tramèr et al., 2016; Gong et al., 2020; Carlini et al., 2024; Ferry et al., 2024) in adversarial contexts, reconstruction is crucial to understand model behaviour in high-stakes decisionmaking systems as required by the EU AI Act1 . In addition to transparency and accountability, the AI Act also aims to improve diversity and fairness. One way to achieve this is through model reconstruction, by examining how sensitive groups are distributed relative to the learned decision geometry. Recent work leverages counterfactual explanations (CFs), i.e., minimally modified instances that attain a desired prediction, for model reconstruction (Aı̈vodji et al., 2020; Wang et al., 2022; Dissanayake & Dutta, 2024). This line of work features two main advantages: CFs naturally encode boundary-adjacent information, and improve query efficiency by providing pairs of closely related instances with contrasting predictions.
1. Introduction Machine-Learning-as-a-Service (MLaaS) platforms have democratized access to AI systems in high-stakes decisionmaking systems. Such systems can be audited by reconstruction, i.e., creation of a surrogate model to closely mimic the * Equal contribution 1 IAS-8, Forschungszentrum Jülich, Germany 2 LMU Munich, Munich Center for Machine Learning (MCML), Germany 3 Department of Computer Science, Aarhus University, Denmark. Correspondence to: Xuan Zhao <[email protected]>, Lena Krieger <[email protected]>.
Proceedings of the 43 rd International Conference on Machine Learning, Seoul, South Korea. PMLR 306, 2026. Copyright 2026 by the author(s).
1
1
artificialintelligenceact.eu/chapter/3
RECAST: Model Reconstruction via Counterfactual-Aware Wasserstein Geometry under Limited Data Table 1. Model reconstruction using CFs. RECAST intentionally avoids explicit boundary recovery and instead focuses on functionality reconstruction, which is better posed under one-sided CFs, offline access and low-query settings. ∗ paper studies only neural networks. + requires prior knowledge about the distribution. SAMPLES: Aı̈vodji et al. (2020).
However, existing CF-based reconstruction approaches face critical limitations. First, decision boundary shifts occur, when surrogate models trained on CFs as full class samples (Aı̈vodji et al., 2020) become overconfident in regions where the CFs are located. Then, the learned decision boundary is shifted incorrectly (Fig. 1), due to margin-based generalization (Shokri et al., 2021), and worsens with one-sided CFs. Two-sided CFs could mitigate this (Wang et al., 2022), but typically platforms do not provide them: rejected loan applicants receive explanations on how to achieve approval, but approved applicants do not learn how their application might have been denied (Aı̈vodji et al., 2020; Dissanayake & Dutta, 2024). Second, neural network-based methods such as Counterfactual Clamping Attacks (Dissanayake & Dutta, 2024), are prone to overfitting when limited training data is available. As data is usually costly in this setting, constrained by strict query budgets to limit API costs and rate limits, data efficiency is critical. Third, under lowquery and one-sided CF settings, decision boundaries are statistically non-identifiable, rendering recovery or approximation of decision boundaries (Dissanayake & Dutta, 2024; Khouna et al., 2025) inapplicable: multiple classifiers may exhibit indistinguishable input–output behavior while having substantially different decision surfaces. Also, data is potentially noisy, as MLaaS platforms may return imperfect or perturbed outputs (Liang et al., 2024). We assume this is a passive observation problem and do not model interactions with a strategic platform using adversarial threat models.
Method DualCF TRA CCA SAMPLES RECAST (Ours)
One-sided CFs
Model agnostic
Behavior-centric
Access?
✗ ✓ ✓ ✓ ✓
✗ ✗ ✓ ✓∗ ✓
✓ ✗ ✗ ✓ ✓
Online Online Online Offline+ Offline
2. Related Work Model extraction attacks (MEAs) aim to reconstruct hidden models from queries to an MLaaS platform (Liang et al., 2024; Zhao et al., 2025). In our work, we focus on functionally equivalent extraction, i.e., recovering a surrogate model whose predictions match those of the target model (Aı̈vodji et al., 2020) under offline access, i.e., based on a fixed sample set constructed without any prior knowledge. Existing work usually requires interactive online query access to the target model during the reconstruction process, often not available in practice (Table 1). Early methods focused on recovering model parameters or explicitly approximating decision boundaries through costly extensive querying (Pal et al., 2020; Jagielski et al., 2020). More recent ones employ CFs, i.e., a minimally modified instance for an input that attains the desired prediction, for instance suggesting an income increase to secure a loan otherwise rejected. Aı̈vodji et al. (2020), here referred to as SAMPLES, use CFs as full labeled data. The approach is vulnerable to decision boundary shift, i.e., it over-confidently assesses samples that lie very close to the decision boundary and are therefore less representative (see Fig. 1). In practice, typically only one-sided CFs are available, to support users in understanding negative outcomes, while avoiding information leak about the hidden model.
For effective reconstruction of a binary classifier with limited or noisy data, we propose REconstruction via Counterfactual-Aware waSserstein opTimization (RECAST) which incorporates CFs as soft samples for both classes. We leverage Wasserstein barycenters as prototypes for each class to capture the underlying structure of each class distribution in a data-efficient manner, suitable for low-data regimes and noisy samples. To detect asymmetric treatment and decision disparities, we adapt a group fairness diagnostic, based on the reconstructed model, that compares how different sensitive groups (e.g., gender or race) are positioned relative to the barycentric decision geometry.
Recent works, particularly those leveraging CFs, often adopt a boundary-centric perspective. They aim to infer the geometry of the target classifier decision boundary, e.g., with a modified entropy loss that treats CFs as in-between classes (CCA (Dissanayake & Dutta, 2024)) or divide the input space into subspaces and incorporate locally optimal CFs (TRA (Khouna et al., 2025)).
Our main contributions include: • Our Model REconstruction via Counterfactual Constrained Wasserstein Geometry (RECAST) solution to behavioral reconstruction via counterfactual-aware Wasserstein barycentric prototypes. • Empirical validation of effectiveness of RECAST in various settings, including four datasets, different target model classes, CF generators, and noisy data, demonstrating high fidelity and query efficiency. • A geometry-based fairness diagnostic to characterize distributional disparities across sensitive groups under distribution shift.
However, the non-identifiability effect (Orekondy et al., 2019; Tramèr et al., 2016) means that under finite samples, different classifiers can exhibit identical or nearly identical classification behavior while having fundamentally different decision boundaries. Another line of work (Aı̈vodji et al., 2020; Wang et al., 2022) thus emphasizes behavior-centric 2
RECAST: Model Reconstruction via Counterfactual-Aware Wasserstein Geometry under Limited Data
Our goal is to reconstruct a surrogate model m̂ that is consistent with the information provided by these limited observations, i.e., to reduce disagreement rate Pr under µ: Pr m(x) ̸= m̂(x) .
reconstruction, to reproduce the input–output behavior of the target model rather than its decision geometry. However, earlier approaches that include CFs, require access to two-sided CFs, e.g., training on pairs, consisting of CFs and CFs for CFs (DualCF (Wang et al., 2022)) or are challenged by decision boundary shifts (SAMPLES (Aı̈vodji et al., 2020)). We propose a behavior-centric approach, that addresses these limitations, by reconstructing class-level behavior from CFs, which naturally encode boundary-adjacent information, using Wasserstein-based prototypes.
x∼µ
Observation (Non-identifiability). In binary classification with one-sided CF access and limited queries, multiple datagenerating distributions can induce identical observable data while differing in their decision boundaries, rendering boundary recovery statistically ill-posed. Given this non-identifiability, reconstruction quality can only be assessed in terms of behavioral agreement with the target model, rather than recovery of a unique decision boundary.
While superficially related to classical learning under data selection bias (Heckman, 1979; Shimodaira, 2000; Zadrozny, 2004), the setting differs in two fundamental aspects. First, the target decision boundary is statistically non-identifiable under offline, one-sided CF access, whereas classical reweighting assumes an identifiable target approximated via ERM on reweighted samples. Second, CFs may overshoot or be systematically biased (see Figure 2), rendering them unreliable as boundary samples. Margin-based methods such as SVMs rely precisely on reliability. CFs instead encode directional distributional constraints between class-conditional distributions, which RECAST handles at distribution level via Wasserstein surrogates rather than via sample-level reweighting or margin maximization.
Such non-identifiability is well known in classical blackbox model reconstruction under finite queries (Tramèr et al., 2016; Orekondy et al., 2019). CFs provide richer information than label queries, but a related ambiguity persists under one-sided CF access, as they do not uniquely constrain the underlying decision surface. Accordingly, we reconstruct distributional class-level behavior: we aim to reduce disagreement between target model and reconstructed model rather than decision boundaries, and model this behavior through a CF-consistent distributional geometry over probability measures.
3. The RECAST Method
Although CFs are defined at the individual level, their impact on model reconstruction is inherently distributional, as they constrain the admissible perturbations between classconditional data distributions. This naturally motivates a Wasserstein-geometric view that measures distributional differences via minimal-cost transport, rather than parametric or pointwise prototype constructions. Prior work has leveraged optimal transport to model counterfactual distributions in a similar distributional setting (You et al., 2025). However, such approaches focus on generating or characterizing CFs themselves, whereas our objective is fundamentally different: we use CFs as structural constraints for reconstructing class-level decision behavior, which uses a Wasserstein geometry over class-conditional measures.
We start by defining the problem of model reconstruction in low-query regime and with one-sided CF, then describe how our new RECAST approach leverages Wasserstein optimization under CF-consistent uncertainty (Section 3.1) to learn class conditional prototypes (Section 3.2), which we employ in a barycentric prototype classifier (Section 3.3). Finally, we adapt a threshold invariant fairness diagnostic to a Wasserstein geometry perspective (Section 3.4).
Problem Setting. We study reconstruction of a binary black-box classifier m : X → {0, 1}, with predictions m(x) = I[ŷm (x) ≥ 0.5], where ŷm are (unobserved) prediction scores. Information sources are restricted to:
Recent CF-based reconstruction methods (Dissanayake & Dutta, 2024; Khouna et al., 2025) assume that CFs lie near the decision boundary. In practice, however, constraints such as immutable features often force CFs to overshoot the boundary, as illustrated in Figure 2. Since the location of CFs relative to the decision boundary is unknown, they cannot be reliably treated as boundary points. Moreover, when a CF lies on the boundary, assigning it a hard label induces bias (Figure 1). RECAST therefore treats CFs as soft, cross-class constraints, which is less sensitive to the location of the CFs relative to the boundary. Since prototypes aggregate distributional information across all available samples, they average out sample-level noise while preserving the
Offline collection of labeled inputs. We are provided with a set of labeled inputs {(x, m(x))}, that can be grouped by class into Dc := {x : m(x) = c} for c ∈ {0, 1}. Our access is restricted to offline, i.e., no access to the target classifier during reconstruction. One-sided CFs. Additionally, we receive a set of reject-to-accept, i.e., one-sided, CFs Dcf := {xcf }, corresponding to inputs x such that m(x) = 0 and ŷm (xcf ) ≥ 0.5. One-sided CFs are xcf ∈ ′ argminx′ ∈X cost(x, x ) s.t. ŷm (x′ ) ≥ 0.5, where cost(x, x′ ) is a user-specified cost measuring the magnitude of the input change, for inputs x with prediction m(x) = 0. 3
RECAST: Model Reconstruction via Counterfactual-Aware Wasserstein Geometry under Limited Data
Figure 2. Kernel density estimates of target model scores ŷm (x) for both class inputs, CFs; common CF methods. Concentration well above decision threshold 0.5 means CFs overshoot the boundary, confirming RECAST’s approach of avoiding near-boundary assumptions.
structure relevant to the target’s decisions. This aggregation effectively increases the usable signal per query, mitigating overfitting in low-query regimes.
0.5
3.1. CF-consistent Uncertainty
0
Definition 3.1 (CF-consistent feasible set). Let Pc be the empirical class-conditional distributions and Pcf the onesided CF distribution. For each class c ∈ {0, 1}, we define n o Cc = µ ∈ P2 (X ) : W2 (µ, Pc ) ≤ εc , W2 (µ, Pcf ) ≤ δc ,
0
1 cf
1
Figure 3. CF-consistent Wasserstein lens. All points in the figure correspond to probability distributions in Wasserstein space. Each class c is associated with a CF-consistent feasible set Cc = BW2 (Pc , εc ) ∩ BW2 (Pcf , δc ), given by the intersection (colored face) of a Wasserstein ball centered at Pc (dashed circle) and a ball centered at Pcf (dotted circle). Although CFs are one-sided, Pcf acts as shared geometric anchor that constrains both classes.
as the CF-consistent feasible set, where P2 (X ) denotes the set of probability measures on X with finite second moments, and W2 denotes the 2-Wasserstein distance (App. A.1). Radii εc and δc encode the uncertainty induced by finite samples, noisy queries, and imperfect CF generation; chosen appropriately, feasible sets Cc are non-empty. The resulting uncertainty set over joint distributions is U = µ : µ(· | y = c) ∈ Cc , c ∈ {0, 1} .
the relative geometry between Pc and Pcf , rather than by a single ambiguity radius. 3.2. Robust Barycenters Theorem 3.2 (Minimizers of Robust Risk Bounds). Let (Q0 , Q1 ) ∈ P2 (X )2 denote any pair of class-conditional reconstruction distributions, let U ⊂ P2 (X )2 denote the CF-consistent uncertainty set (Definition 3.1), which contains all class conditional data generating distributions consistent with the observed samples and CFs. For any such reconstruction (Q0 , Q1 ) ∈ P2 (X )2 , the worst-case distributional reconstruction risk Rc (Qc ) = supµ∈Cc W22 (µ, Qc ), admits a tractable Wasserstein-robust upper bound. The minimizers are the solutions to the optimization problem X min (1 − λc )W22 (Qc , Pc ) + λc W22 (Qc , Pcf ) .
This construction induces a lens-shaped ambiguity set in Wasserstein space (Figure 3). Distributionally Wasserstein robust optimization (WRO) under Wasserstein ambiguity sets is the minimization of the worst case expected loss over all distributions within a Wasserstein ball around an empirical distribution (Mohajerin Esfahani & Kuhn, 2018; Blanchet et al., 2019; Gao & Kleywegt, 2023). Relation to Wasserstein robustness. Classical Wasserstein-robust optimization (WRO) optimizes over a single Wasserstein ball centered at an empirical distribution, with radius that can be statistically calibrated from data. In our setting, however, the ambiguity radii cannot be reliably calibrated from limited one-sided CF data. Thus, our CF-consistent feasible set is the intersection of two Wasserstein balls: one centered at the class-conditional data distribution and one centered at the counterfactual distribution, where BW2 (P, r) = {µ ∈ P2 (X ) : W2 (µ, P) ≤ r} denotes the Wasserstein ball of radius r centered at P. This two-anchor construction encodes not only sampling uncertainty but also geometric constraints induced by CFs, yielding a lens-shaped ambiguity set rather than a single ball. As a result, robustness in our setting is governed by
(Q0 ,Q1 )
c∈{0,1}
Minimizing this objective is not equivalent to solving the constrained robust problem directly, but yields a geometryaware surrogate whose minimizers are stable under CFconsistent perturbations. We measure reconstruction risk in Wasserstein distance to ensure consistency with the geometry of the uncertainty set. Remark 3.3 (Geometric uniqueness). Without additional regularity assumptions, the minimizer of Theorem 3.2 does not need be unique as a probability measure. Nevertheless, all minimizers lie on the same Wasserstein displacement geodesic between Pc and Pcf , and therefore lead to a unique CF-identifiable Wasserstein distance geometry. 4
RECAST: Model Reconstruction via Counterfactual-Aware Wasserstein Geometry under Limited Data
Barycentric form. For each c ∈ {0, 1}, the optimal Q⋆c satisfies Q⋆c = argmin (1−λc )W22 (µ, Pc )+λc W22 (µ, Pcf ) (1)
Class 0
f2
µ∈P2 (X )
Q⋆c
i.e., is the 2-Wasserstein barycenter of Pc and Pcf (App. A.2).
0
Class 1
1
cf
f2
2
2
1
1
0
0
1
2
3
4 f1
0
0
1
0
1
0.5
2
3
4 f1
Figure 4. Top: Full original dataset and decision tree classification (blue and pink); CFs as x (orange); points available for reconstruction circled (black). Bottom: Decision region between barycenters: Q0 (blue), Q1 (pink); black circles midpoints along the optimal transport path (grey lines), indicating the decision region Q0.5 .
Proof idea. For each class c, a worst-case risk is Rc (Qc ) = supµ∈Cc W22 (µ, Qc ). Using standard inequalities in Wasserstein space, this worst-case risk admits a robust upper bound by a convex combination of W22 (Qc , Pc ) and W22 (Qc , Pcf ). Optimizing this bound yields the Wasserstein barycenter in Eq. (1), and separability across classes gives the objective in Theorem 3.2. The full proof is given in App. A.3.
carry then no influence on Q⋆0 , and the formulation reduces to standard binary classification. The case εc = δc = 0 would require Pc = Pcf , which contradicts the reject-toaccept CF setting and renders the problem ill-posed. When Pcf lies approximately equidistant between the two class distributions, the resulting ratio assigns comparable influence to both classes. To set λc appropriately, we consider the following stability:
Importantly, our formulation does not require explicit specification of the uncertainty radii εc and δc . In our derivation (App. A.3), their relative effect is absorbed into the data-dependent mixing weight λc : selecting an optimal λc corresponds to selecting optimal radii, and λc determines the location of the CF-consistent Wasserstein barycenter.
Lemma 3.4 (Stability along the barycentric geodesic). For class c ∈ {0, 1}, let (Q⋆c (λ))λ∈[0,1] denote a constant-speed Wasserstein geodesic between Pc and Pcf . Then, function λc 7→ Rc (Q⋆c (λc )) is Lipschitz continuous on [0, 1], with Lipschitz constant Lc = 2 εc + W2 (Pc , Pcf ) W2 (Pc , Pcf ).
Canonical instantiation of λc . To better illustrate the role of λc , we rewrite the class-specific objective (Theorem 3.2) in an equivalent normalized form by dividing by (1 − λc ): X λc min W22 (Qc , Pc ) + W22 (Qc , Pcf ) . 1 − λc (Q0 ,Q1 )
Consequently, small perturbations of λc lead to proportionally small changes in the worst-case risk. Proof in App. A.4.
c∈{0,1}
This form makes explicit that the influence of CF information is governed by the ratio λc /(1 − λc ).
As a direct consequence, for any (unobservable) optimizer λ⋆c ∈ [0, 1] of the robust upper bound along the barycentric geodesic, the suboptimality of a chosen λc is bounded as
Theorem 3.2 yields a family of robust barycenters {Q⋆c (λc )}. In classical Wasserstein robust optimization, one would select λc (equivalently, the ambiguity radius) to tighten the bound with respect to the true risk. However, under onesided CF access and limited data, such statistical calibration is fundamentally infeasible.
Rc (Q⋆c (λc )) − Rc (Q⋆c (λ⋆c )) ≤ Lc |λc − λ⋆c |. This result ensures that the canonical instantiation remains behaviorally stable even under under small variations of λc .
In this regime, λc is not uniquely identifiable from data within the CF-consistent uncertainty set. We consequently instantiate λc through the observable Wasserstein geometry of the three anchor distributions. Let A = W22 (Pcf , Pc ) and B = W22 (Pcf , P1−c ). The relative geometry induces the λc B relation 1−λ = A+B , yielding the closed-form expression c B λc = A+2B .
3.3. Barycentric Prototype Classifier Definition 3.5 (CF-consistent behavioral equivalence). Two different classifiers m1 , m2 are behaviorally equivalent (w.r.t. one-sided CF access), if ∀ µ ∈ U , Prx∼µ m1 (x) ̸= m2 (x) = 0, where the probability denotes the disagreement rate of the two different models under µ. The CF-consistent uncertainty set U contains multiple classconditional distributions that are indistinguishable from the observed data. Any valid pointwise decision rule must therefore be invariant over all (µ0 , µ1 ) ∈ U of class-conditional data generating distributions, and cannot depend on quantities (such as densities or likelihoods) that vary within this
From this perspective, the ratio λc /(1 − λc ) modulates the contribution of one sided CF samples as soft cross-class signals: it increases as Pcf is away from the opposite class 1−c, and decreases when it is relatively distant from class c. The formulation degrades gracefully at boundary cases: when Pcf = P1 , the weights reduce to λ1 = 0 and λ0 = 1, CFs 5
RECAST: Model Reconstruction via Counterfactual-Aware Wasserstein Geometry under Limited Data
set. In contrast, Wasserstein distances are invariant to CFunidentifiable variations, making W2 (δx , Q̂c ) a natural CFidentifiable score. This is because CF-consistent uncertainty constrains only how probability mass can be transported in the input space, while leaving its local density unspecified; density-based quantities vary under such unidentifiable rearrangements, whereas Wasserstein distances depend solely on the induced transport geometry.
of Wasserstein geometry. Given the barycentric class prototypes (Q⋆0 , Q⋆1 ), we define a Wasserstein-geometric score p : X → (0, 1) as: p(x) = σ W2 (δx , Q⋆0 ) − W2 (δx , Q⋆1 ) where σ denotes the sigmoid function. Based on Chen & Wu (2020), we adapt Threshold Invariant Demographic Parity (TIDP) and Threshold Invariant Equalized Odds (TIEO), i.e., equal selection rates and equal prediction accuracy across sensitive attributes independent of a decision threshold, in the perspective of Wasserstein geometry. In contrast to previous work, our score operates on entire score distributions, capturing disparities that persist across all decision thresholds. We quantify group-level disparities by comparing the distributions of this score across sensitive groups that share a sensitive attribute S ∈ {0, 1}. We compute the Wasserstein-1 distance between the score distributions conditioned on different values of S to assess TIDP, and between score distributions conditioned on both S and the label Y ∈ {0, 1} to assess TIEO. As the scores are induced by barycentric prototypes defined over the CF-consistent feasible sets, the resulting fairness diagnostics are evaluated with respect to a shared uncertainty-aware decision geometry and quantify how different groups are positioned relative to the barycentric decision geodesic. Equations in App. A.7.
Given the reconstructed class prototypes(Q̂0 , Q̂1 ), a CFidentifiable decision statistic is given by the 2-Wasserstein distance from a Dirac mass to the class prototypes: m̂(x) := arg min W2 δx , Q̂c , (2) c∈{0,1}
where δx denotes the Dirac measure at an input x. This classifier should be interpreted as a canonical representative of the CF-consistent behavioral equivalence class, rather than a uniquely identifiable ground-truth decision rule. Importantly, the proposed framework does not restrict the form of the final decision mechanism, provided it operates solely on CF-identifiable geometric quantities. The stability of the barycentric classifier follows directly from the Wasserstein geometry. By Corollary A.4 in App. A.3, the induced pointwise decision score is Lipschitz continuous with respect to perturbations of the prototype distributions under W2 . This stability, combined with the distributional smoothing induced by the barycentric aggregation, yields a uniform bound on the prediction disagreement between the classifiers. Thus, for any Q′c distribution in p CF-consistent feasible set Cc : W2 (Qc , Q′c ) ≤ Rc (Qc ), and small reconstruction risks Rc (Qc ) imply small variations in the classification behavior across all CF-consistent realizations.
Summary. Under one-sided CF access the target decision boundary is fundamentally non-identifiable, so reconstruction cannot recover a unique ground-truth classifier. RECAST therefore (i) constructs a CF-consistent Wasserstein uncertainty set, (ii) computes Q̂c as Wasserstein barycenters, and (iii) predicts with the classification rule, Eq. (2). Minimizing the resulting Wasserstein reconstruction risk promotes stability of decision behavior under CF-consistent perturbations, yielding reduced prediction disagreement.
This classification rule induces a decision region in Wasserstein space. Let T denote an optimal transport map from Q̂0 to Q̂1 . The Wasserstein geodesic between the two class prototypes is given by the displacement interpolation Qγ := (1 − γ)Id + γT # Q̂0 , γ ∈ [0, 1]. The decision region ischaracterized by the midpoint of this geodesic, Q0.5 = 12 Id + 21 T Q̂0 , which represents the distribu-
4. Experiments We evaluate RECAST across four real-world datasets, three target model families, and multiple CF generation methods, assessing fidelity, robustness, and fairness preservation.
#
tion equidistant from both class prototypes in Wasserstein distance (Figure 4). In this geometric view, Q̂0 and Q̂1 act as distributional prototypes, summarizing the CF-consistent class-conditional geometry. Classification is performed by comparing the Wasserstein distance of a point to these prototypes, yielding a prototype-based classifier in Wasserstein space. For the overall geometric relationship between CFconsistent lens and barycentric prototypes see Figure 3.
4.1. Experimental Setup Data. We study four publicly available binary classification datasets, that are often used in literature and cover a variety of properties, e.g., dimensionalities, dataset size, and tasks: Adult Income (Becker, 1996), COMPAS (Angwin et al., 2016), HELOC (FICO, 2018), and California Housing (Pace & Barry, 1997). We include exploratory cases on additional data modalities in App. C.6 and C.7.
3.4. Threshold-invariant Fairness Diagnostic
Baselines. To the best of our knowledge, there is limited prior work on reconstruction restricted data access and only
RECAST also supports fairness auditing by adapting a threshold-invariant fairness diagnostic in the perspective 6
RECAST: Model Reconstruction via Counterfactual-Aware Wasserstein Geometry under Limited Data
one-sided CFs (Table 1). As such, we adapt three representative baselines to offline-only, i.e., no querying during reconstruction. Aı̈vodji et al. (2020), referred to as SAMPLES, incorporates CFs as ordinary samples. CCA, Counterfactual Clamping Attack, modifies entropy loss to explicitly incorporate CFs (Dissanayake & Dutta, 2024). TRA (Khouna et al., 2025) assumes prior knowledge that the target classifier is a decision tree with axis-parallel splits, and is therefore included only when such structural information is available; in contrast, RECAST and the other baselines operate without access to this form of model-specific prior. For all baselines, we use hyperparameters recommended in the original papers.
addition to the balanced setting, we conduct supplementary experiments with imbalanced D0 and D1 ; see results in App. B. CF Generation. We evaluate several CF generation methods, including MCCF (Wachter et al., 2017), which seeks CFs with minimal input changes. In our implementation, we add an ℓ1 regularization term to encourage sparse input changes. DiCE (Mothilal et al., 2020) ensures actionability by enforcing immutable features. To improve robustness under model shifts, we adopt ROAR (Upadhyay et al., 2021). For realism, we apply 1-Nearest-Neighbor from the desired class and C-CHVAE (Pawelczyk et al., 2020), which uses variational autoencoders. Details in App. B.
Barycentric prototype computation. We compute the barycenters Qc using optimization Algorithm 1 in App. A. We optimize an entropically regularized (Sinkhornsmoothed) surrogate of the Wasserstein barycenter objective for computational efficiency, following standard practice in optimal transport (Chizat et al., 2020), theoretical guarantees and an ablation regarding the blur parameter are included in App. A.5 and A.6. All solver settings and hyperparameters are kept fixed across experiments. All implementation details, including initialization, solver settings, and hyperparameters also in App. B. Code is available online.2
Robustness Evaluation Protocol. Since RECAST is derived under CF-consistent uncertainty, we evaluate not only average fidelity under i.i.d. test data, but also robustness under distributional shifts. Although our uncertainty is specified over training distributions, its effect manifests as sensitivity to distributional shifts during test. We thus evaluate robustness by probing fidelity under perturbations of the test distribution, which operationalizes different plausible realizations of the underlying Qc . Specifically, for each reference test set Dref , we construct a family of shifted test sets Dref = {x + ϵ : x ∈ Dref , ϵ ∼ N (0, τ 2 I)}, with noise levels τ ∈ {0.05, 0.1, 0.2, 0.4}.
Reconstruction. As target models, we study neural networks (MLP), logistic regression (LR), and tree-based classifiers (DT) trained on 80% of the original datasets, which remains unknown during the reconstruction phase. In each experiment, we randomly sample a certain query size, e.g., 100 instances from class 0 and 100 from class 1, as D0 , D1 , resp. From class 0 samples various CF methods are used to generate CFs Dcf . We evaluate the reconstructed models on a held-out reference set Dref , disjoint from the reconstruction data.
Although average fidelity is a widely used metric for model reconstruction, it can be dominated by samples far from the decision threshold, whose predictions are inherently stable under perturbations. Consequently, high overall fidelity may mask disagreement in decision-sensitive regions. We thus also evaluate robustness on a near-threshold subset where the target model likely exhibits higher uncertainty. Although the true decision boundary is not identifiable, |ŷm (x)−0.5| ≤ γ captures inputs most susceptible to distributional perturbations (Blanchet et al., 2024). We define the near-threshold set as Dnear = {x ∈ Dref : |ŷm (x)−0.5| ≤ γ}, with γ = 0.05. We report fidelity on both the full set Dref and on Dnear . Note that access to the target model’s continuous output scores is used only during the evaluation phase to construct diagnostic subsets Dnear , and is neither available nor used during the reconstruction phase. We additionally evaluate robustness under cross-domain covariate shift, details of that protocol are in App. C.4.
We use fidelity (Aı̈vodji et al., 2020) rather than accuracy as our primary metric: a surrogate that faithfully mimics the target model’s decision, including its systematic errors, should score highly regardless of ground-truth label agreement, since behavioral reconstruction, not predictive performance is the relevant objective for reconstruction (see App. C.3 for a joint comparison). Fidelity between the target model m and the surrogate m̂ on a reference set Dref is defined as: X 1 Fidm,Dref (m̂) = I[0,1] [m(x) = m̂(x)] . |Dref | x∈Dref
Fairness Diagnostic Protocol. For each distribution-shift level τ , we report the absolute difference between the fairness diagnostics for target model m and reconstructed model m̂ over the corresponding shifted reference set Dref . We show how well the reconstructed models preserve the original decision disparities, and assess their use for auditing.
We repeat each experiment ten times with different random seeds and report the mean and variance of our method. In 2 https://github.com/zhaoxuan00707/ce_ reconstruction
7
RECAST: Model Reconstruction via Counterfactual-Aware Wasserstein Geometry under Limited Data Table 2. Fidelity. Multilayer perceptron (MLP), logistic regression (LR), decision tree (DT) target models, low-query size 100. RECAST consistently achieves higher (in one case comparable) fidelity across datasets and target models. TRA only feasible with DT target. MLP (Target Model)
Dataset Adult In. COMPAS HELOC Housing
LR (Target Model)
DT (Target Model)
SAMPLES
CCA
RECAST (Ours)
SAMPLES
CCA
RECAST (Ours)
SAMPLES
CCA
RECAST (Ours)
TRA
0.801 ± 0.035 0.415 ± 0.060 0.427 ± 0.121 0.459 ± 0.102
0.843 ± 0.020 0.765 ± 0.020 0.644 ± 0.057 0.716 ± 0.070
0.908 ± 0.034 0.855 ± 0.022 0.696 ± 0.078 0.712 ± 0.121
0.909 ± 0.008 0.543 ± 0.004 0.551 ± 0.013 0.477 ± 0.001
0.816 ± 0.025 0.785 ± 0.026 0.670 ± 0.013 0.656 ± 0.014
0.913 ± 0.033 0.821 ± 0.047 0.798 ± 0.025 0.789 ± 0.094
0.819 ± 0.017 0.396 ± 0.003 0.620 ± 0.008 0.554 ± 0.012
0.827 ± 0.018 0.657 ± 0.027 0.702 ± 0.023 0.619 ± 0.034
0.894 ± 0.040 0.869 ± 0.049 0.772 ± 0.029 0.774 ± 0.149
0.842 ± 0.020 0.826 ± 0.007 0.702 ± 0.024 0.654 ± 0.015
Figure 5. Fidelity (mean ± std) of surrogate models on real-world datasets under varying query sizes. RECAST with superior performance.
Ablation Study. We conduct an ablation study on different prototype constructions, distance metrics, and CFs integration strategies in RECAST with all details in App. C.1.
model may arise when perturbations cause inputs to cross decision surfaces, an effect most pronounced near the decision threshold. As the perturbation strength increases, fidelity decreases for all methods (see Figure 6). Still, RECAST degrades substantially more gracefully than the baselines, and SAMPLES and CCA suffer pronounced performance drops even under moderate noise. This gap is even more pronounced in the near-threshold region Dnear most sensitive to perturbations.
4.2. Reconstruction Fidelity Table 2 summarizes fidelity results across target classifiers, with query size 100 and MCCF as CF generation method. In almost all cases, our method achieves superior fidelity to SAMPLES and CCA, comparable in one. We also study how the amount of available training data influences the performance of the reconstructed models with query sizes from 25 to 400 in Figure 5. Notably, our approach demonstrates a clear advantage when the query size is small, highlighting its efficiency in low-query regimes.
Along the second axis, we evaluate robustness under realistic distribution shift using Folktables ACSIncome dataset (Ding et al., 2021), training on California 2018 data and evaluating on Michigan 2014, a setting that combines both geographical and temporal variation (see App. C.4,Table 7). RECAST consistently achieves highest fidelity across query budgets (100 and 150), while SAMPLES leads on accuracy but lags notably on fidelity, consistent with the distinction drawn above, where accuracy and fidelity capture fundamentally different objectives. These results confirm that RECAST maintains strong behavioral consistency with the target model even under substantial distribution shifts.
4.3. Effect of CF Geometry RECAST consistently achieves high fidelity across CF generation strategies, see Table 3. Baseline methods exhibit substantially larger fluctuations, indicating sensitivity to the properties of CFs. Unlike baselines, RECAST neither interprets CFs as true samples (SAMPLES) nor assumes proximity to the decision boundary (CCA), properties that depend on the generator, illustrated in Figure 2.
Table 3. Fidelity, varying counterfactual generation methods on Adult, query size 100, target MLP. RECAST shows robustness with high fidelity across diverse CF generators.
4.4. Robustness under Perturbations We study robustness along two axes: behavioral stability under additive perturbations of the test distribution and generalization under realistic distribution shift across domains. Both test whether RECAST’s behavioral agreement with the target model holds beyond the reconstruction setting. Disagreement between a reconstructed surrogate and the target
Baselines
CF Generation MCCF DiCE ROAR 1-Nearest-Neighbor C-CHVAE
8
SAMPLES
CCA
RECAST (Ours)
0.793 0.933 0.735 0.712 0.357
0.893 0.903 0.735 0.835 0.817
0.913 0.923 0.910 0.916 0.923
RECAST: Model Reconstruction via Counterfactual-Aware Wasserstein Geometry under Limited Data
Figure 7. Absolute fairness (DTIDP (solid) and DTIEO (dashed)) differences between reconstructed and target model as a function of noise level τ . Lower values mean more faithful preservation of target model fairness characteristics.
Figure 6. Robustness to additive noise on Adult. Fidelity as a function of the noise level τ for both full reference set Dref and near-threshold subset Dnear , which captures decision-sensitive inputs. ∆ denotes the fidelity drop between τ = 0 and τ = 0.4.
calibrated via classical WRO methods, but this lies beyond the scope of the low-query regime studied here.
4.5. Fairness Diagnostics
In privacy-sensitive auditing contexts, where datasets are small, access is restricted, and queries are costly, such highcapacity approaches become infeasible, making lightweight, geometry-driven reconstruction methods such as RECAST particularly relevant in practice.
The goal of the proposed fairness diagnostics is to assess whether a reconstructed surrogate preserves the fairnessrelated behavior of the target model, under CF-consistent distributional uncertainty. These diagnostics are thus intended solely for auditing purposes and do not constitute fairness performance or debiasing claims. Figure 7 reports the fairness diagnostic error between the reconstructed surrogate and the target model under increasing perturbations. Lower values indicate that the surrogate more faithfully preserves the fairness characteristics of the target model. RECAST consistently exhibits smaller diagnostic consistency gap and more graceful degradation with noise.
We acknowledge that Wasserstein distance can be computationally demanding in large-scale settings. However, in the low-data regimes considered here, this cost is not a practical bottleneck. In runtime experiments (App. C.8), we observe training times near linear in query size.
5. Conclusion and Future Work
4.6. Effect of Query Budget
We propose RECAST, a model reconstruction approach grounded in Wasserstein geometry that captures relationships between data distributions, enabling the formation of robust class prototypes that effectively represent both labeled data and CFs, under one-sided CF access and limited query budgets.
RECAST is designed for limited query access and onesided CFs, where direct reconstruction of the target model is fundamentally sample-inefficient and the decision boundary is not statistically identifiable. To make this regime explicit, we additionally study how reconstruction performance varies with available query budget.
In our experiments, we demonstrate that RECAST maintains high fidelity in small data regimes, where overfitting and poor generalization are prevalent concerns. Importantly, RECAST is not designed to compete with large-query model extraction: it is designed for reconstruction scenarios where data is scarce, access is restricted, and CFs are the only boundary-relevant signal.
Specifically, we compare RECAST with a no CFs baseline over number of queried labeled instances, and identify the crossover point at which direct supervised reconstruction becomes competitive (Figure 9, App. C.5). This analysis reveals a clear practical boundary: in low-query regimes, CFs provide crucial decision-relevant information that cannot be recovered from samples alone, enabling RECAST to substantially outperform both CF-based and non-CF baselines. As the query budget increases and empirical samples begin to adequately cover decision-relevant regions, the advantage of CF-based reconstruction naturally diminishes, and simpler supervised models become sufficient. Note that with sufficient data, the Wasserstein mixing weights could be
Our approach is currently limited to binary classification, which is common in literature since most applications consider accept-reject decision. Extending RECAST to multiclass settings, and leveraging prior knowledge constitute interesting directions for future work.
9
RECAST: Model Reconstruction via Counterfactual-Aware Wasserstein Geometry under Limited Data
Acknowledgements
2020.
This work was partially funded by project W2/W3-108 Initiative and Networking Fund of the Helmholtz Association. We gratefully acknowledge the computing time granted through project XAI (No. 65881) on the supercomputer JURECA at Jülich Supercomputing Centre (JSC).
Aı̈vodji, U., Arai, H., Gambs, S., and Hara, S. Characterizing the risk of fairwashing. pp. 14822– 14834, 2021. URL https://proceedings. neurips.cc/paper/2021/hash/ 7caf5e22ea3eb8175ab518429c8589a4Abstract.html.
Impact Statement
Angwin, J., Larson, J., Mattu, S., and Kirchner, L. Propublica compas recidivism risk score data and analysis. https://www.propublica.org/article/ machine-bias-risk-assessments-incriminal-sentencing, 2016.
This work studies classifier reconstruction under restricted data access and one-sided CF supervision, a setting that arises naturally when interacting with black-box or proprietary decision systems. By characterizing what aspects of model behavior are fundamentally identifiable under such constraints, our framework enables high-fidelity reconstruction of decision behavior, supporting the auditing and investigation of deployed models for fairness, bias, and accountability concerns in settings with limited model access. Importantly, our approach does not aim to recover a unique ground-truth classifier or individual training examples, but instead focuses on CF-identifiable behavioral structure derived from distributional and counterfactual information.
Becker, B. Adult data set. https://archive.ics. uci.edu/dataset/2/adult, 1996. Blanchet, J. H., Kang, Y., and M., K. R. A. Robust wasserstein profile inference and applications to machine learning. J. Appl. Probab., 56(3):830–857, 2019. doi: 10.1017/JPR.2019.49. URL https://doi.org/10. 1017/jpr.2019.49. Blanchet, J. H., Cui, P., Li, J., and Liu, J. Stability evaluation through distributional perturbation analysis. In ICML. OpenReview.net, 2024.
The primary impact of our work involves reducing the barriers to model reconstruction. Organizations and communities affected by ML decision-making systems, e.g., loan applications, hiring, or healthcare, often lack resources to audit the systems they rely on. Our approach offers these groups a practical tool to investigate issues, such as disparate impact, discrimination, or violations of fairness principles in deployed models, even when model owners are unwilling or unable to grant direct access.
Carlini, N., Paleka, D., Dvijotham, K. D., Steinke, T., Hayase, J., Cooper, A. F., Lee, K., Jagielski, M., Nasr, M., Conmy, A., Wallace, E., Rolnick, D., and Tramèr, F. Stealing part of a production language model. 2024. URL https://openreview.net/forum? id=VE3yWXt3KB.
Our work supports the regulatory landscape around responsible, transparent and accountable AI, as policymakers worldwide establish requirements for algorithmic impact assessments and accountability.
Chen, M. and Wu, M. Towards threshold invariant fair classification. In UAI, volume 124 of Proceedings of Machine Learning Research, pp. 560–569. AUAI Press, 2020.
While our method does introduce security considerations for information leakage, the societal benefit of enabling fairness investigations outweighs these concerns in contexts involving high-stakes decisions affecting vulnerable populations. Our work is likely to advance the field toward the establishment of more trustworthy and responsible AI systems by improving accountability.
References Agueh, M. and Carlier, G. Barycenters in the wasserstein space. SIAM J. Math. Anal., 43(2):904–924, 2011. doi: 10.1137/100805741. URL https://doi.org/10. 1137/100805741. Aı̈vodji, U., Bolot, A., and Gambs, S. Model extraction from counterfactual explanations. CoRR, abs/2009.01884, 10
Chizat, L., Roussillon, P., Léger, F., Vialard, F., and Peyré, G. Faster wasserstein distance estimation with the sinkhorn divergence. In Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M., and Lin, H. (eds.), Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual, 2020. URL https://proceedings. neurips.cc/paper/2020/hash/ 17f98ddf040204eda0af36a108cbdea4Abstract.html. Deng, L. The MNIST database of handwritten digit images for machine learning research [best of the web]. IEEE Signal Process. Mag., 29(6):141–142, 2012. doi: 10. 1109/MSP.2012.2211477. URL https://doi.org/ 10.1109/MSP.2012.2211477.
RECAST: Model Reconstruction via Counterfactual-Aware Wasserstein Geometry under Limited Data
Ding, F., Hardt, M., Miller, J., and Schmidt, L. Retiring adult: New datasets for fair machine learning. Advances in Neural Information Processing Systems, 34, 2021.
Mag., 58(12):83–89, 2020. doi: 10.1109/MCOM. 001.2000196. URL https://doi.org/10.1109/ MCOM.001.2000196.
Dissanayake, P. and Dutta, S. Model reconstruction using counterfactual explanations: A perspective from polytope theory. In Globersons, A., Mackey, L., Belgrave, D., Fan, A., Paquet, U., Tomczak, J. M., and Zhang, C. (eds.), Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, NeurIPS 2024, Vancouver, BC, Canada, December 10 - 15, 2024, 2024. URL http://papers. nips.cc/paper_files/paper/2024/hash/ 97d539aa02b82c52d4a9eda62e1c6435Abstract-Conference.html.
Hardt, M., Price, E., and Srebro, N. Equality of opportunity in supervised learning. Advances in neural information processing systems, 29, 2016.
Dwork, C., Hardt, M., Pitassi, T., Reingold, O., and Zemel, R. Fairness through awareness. In Proceedings of the 3rd innovations in theoretical computer science conference, pp. 214–226, 2012.
Heckman, J. J. Sample selection bias as a specification error. Econometrica: Journal of the econometric society, pp. 153–161, 1979. Jagielski, M., Carlini, N., Berthelot, D., Kurakin, A., and Papernot, N. High accuracy and high fidelity extraction of neural networks. In 29th USENIX security symposium (USENIX Security 20), pp. 1345–1362, 2020. Jigsaw. Jigsaw toxic comment classification challenge. https://www.kaggle.com/c/jigsawtoxic-comment-classificationchallenge, 2018.
Feldman, M., Friedler, S. A., Moeller, J., Scheidegger, C., and Venkatasubramanian, S. Certifying and removing disparate impact. In proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining, pp. 259–268, 2015.
Khouna, A., Ferry, J., and Vidal, T. From counterfactuals to trees: Competitive analysis of model extraction attacks. CoRR, abs/2502.05325, 2025. doi: 10.48550/ARXIV. 2502.05325. URL https://doi.org/10.48550/ arXiv.2502.05325.
Ferry, J., Fukasawa, R., Pascal, T., and Vidal, T. Trained random forests completely reveal your dataset. In Forty-first International Conference on Machine Learning, ICML 2024, Vienna, Austria, July 21-27, 2024. OpenReview.net, 2024. URL https://openreview.net/forum? id=cc72Vnfvoc. Feydy, J., Séjourné, T., Vialard, F., Amari, S., Trouvé, A., and Peyré, G. Interpolating between optimal transport and MMD using sinkhorn divergences. In Chaudhuri, K. and Sugiyama, M. (eds.), The 22nd International Conference on Artificial Intelligence and Statistics, AISTATS 2019, 16-18 April 2019, Naha, Okinawa, Japan, volume 89 of Proceedings of Machine Learning Research, pp. 2681– 2690. PMLR, 2019. URL http://proceedings. mlr.press/v89/feydy19a.html. FICO. Explainable machine learning challenge dataset, 2018. URL https://community.fico. com/s/explainable-machine-learningchallenge. Gao, R. and Kleywegt, A. J. Distributionally robust stochastic optimization with wasserstein distance. Math. Oper. Res., 48(2):603–655, 2023. doi: 10.1287/MOOR.2022. 1275. URL https://doi.org/10.1287/moor. 2022.1275. Gong, X., Wang, Q., Chen, Y., Yang, W., and Jiang, X. Model extraction attacks and defenses on cloudbased machine learning models. IEEE Commun. 11
Liang, J., Pang, R., Li, C., and Wang, T. Model extraction attacks revisited. In Zhou, J., Quek, T. Q. S., Gao, D., and Cárdenas, A. A. (eds.), Proceedings of the 19th ACM Asia Conference on Computer and Communications Security, ASIA CCS 2024, Singapore, July 1-5, 2024. ACM, 2024. doi: 10.1145/3634737.3657002. URL https://doi. org/10.1145/3634737.3657002. McCann, R. J. A convexity principle for interacting gases. Advances in mathematics, 128(1):153–179, 1997. Mohajerin Esfahani, P. and Kuhn, D. Data-driven distributionally robust optimization using the wasserstein metric: Performance guarantees and tractable reformulations. Mathematical Programming, 171(1):115–166, 2018. Mothilal, R. K., Sharma, A., and Tan, C. Explaining machine learning classifiers through diverse counterfactual explanations. In FAccT, pp. 607–617. ACM, 2020. Orekondy, T., Schiele, B., and Fritz, M. Knockoff nets: Stealing functionality of black-box models. In CVPR, pp. 4954–4963. Computer Vision Foundation / IEEE, 2019. Pace, R. K. and Barry, R. Sparse spatial autoregressions, 1997. URL https://www.dcc.fc.up.pt/ ltorgo/Regression/cal_housing.html. ˜ Data retrieved from the StatLib repository.
RECAST: Model Reconstruction via Counterfactual-Aware Wasserstein Geometry under Limited Data
Pal, S., Gupta, Y., Shukla, A., Kanade, A., Shevade, S., and Ganapathy, V. Activethief: Model extraction using active learning and unannotated public data. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pp. 865–872, 2020. Pawelczyk, M., Broelemann, K., and Kasneci, G. Learning model-agnostic counterfactual explanations for tabular data. In WWW, pp. 3126–3132. ACM / IW3C2, 2020. Shimodaira, H. Improving predictive inference under covariate shift by weighting the log-likelihood function. Journal of statistical planning and inference, 90(2):227–244, 2000. Shokri, R., Strobel, M., and Zick, Y. On the privacy risks of model explanations. In AIES, pp. 231–241. ACM, 2021. Tramèr, F., Zhang, F., Juels, A., Reiter, M. K., and Ristenpart, T. Stealing machine learning models via prediction apis. In USENIX Security Symposium, pp. 601–618. USENIX Association, 2016. Upadhyay, S., Joshi, S., and Lakkaraju, H. Towards robust and reliable algorithmic recourse. In NeurIPS, pp. 16926– 16937, 2021. Wachter, S., Mittelstadt, B. D., and Russell, C. Counterfactual explanations without opening the black box: Automated decisions and the GDPR. CoRR, abs/1711.00399, 2017. Wang, Y., Qian, H., and Miao, C. Dualcf: Efficient model extraction attack from counterfactual explanations. In FAccT, pp. 1318–1329. ACM, 2022. You, L., Cao, L., Nilsson, M., Zhao, B., and Lei, L. Distributional counterfactual explanations with optimal transport. In AISTATS, volume 258 of Proceedings of Machine Learning Research, pp. 1135–1143. PMLR, 2025. Zadrozny, B. Learning and evaluating classifiers under sample selection bias. In Proceedings of the twenty-first international conference on Machine learning, pp. 114, 2004. Zhao, K., Li, L., Ding, K., Gong, N. Z., Zhao, Y., and Dong, Y. A survey on model extraction attacks and defenses for large language models. In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 2, pp. 6227–6236, 2025.
12
RECAST: Model Reconstruction via Counterfactual-Aware Wasserstein Geometry under Limited Data
A. Wasserstein Distance and Barycenters A.1. Wasserstein Distance We briefly recall the notions from optimal transport that are required for the formulation and analysis of our reconstruction framework, in particular Definition 3.1 and Theorem 3.2. Let P2 (Rd ) denote the set of probability measures on Rd with finite second moments. 2-Wasserstein distance.
For µ, ν ∈ P2 (Rd ), the squared 2-Wasserstein distance is defined as Z 2 W2 (µ, ν) := inf ∥x1 − x2 ∥22 dγ(x1 , x2 ), γ∈Π(µ,ν)
(3)
Rd ×Rd
where Π(µ, ν) denotes the set of couplings (transport plans) with marginals µ and ν. The Wasserstein distance induces a geometry on probability measures that accounts for both mass displacement and spatial structure, and is therefore well suited for comparing distributions arising from localized perturbations such as counterfactual examples. Monge formulation. When there exists a measurable transport map T : Rd → Rd such that T# µ = ν, the optimal transport problem admits the equivalent Monge formulation Z T = arg min ∥x − T (x)∥22 dµ(x). (4) T# µ=ν
Rd
In general, such a map need not exist; however, the Kantorovich formulation in (3) always remains well defined and is the form used throughout this work. A.2. Wasserstein Barycenter d N Given a finite collection of probability measures {µi }N i=1 ⊂ P2 (R ) and nonnegative weights {λi }i=1 satisfying 1, the Wasserstein barycenter is defined as
µ∗ = arg
min
N X
µ∈P2 (Rd )
λi W22 (µ, µi ).
PN
i=1 λi =
(5)
i=1
The barycenter µ∗ provides a distributional prototype that summarizes the input measures in Wasserstein space. Unlike Euclidean averaging, this formulation aligns probability mass prior to aggregation, enabling the resulting prototype to reflect decision-relevant geometric structure induced by the target classifier. In our setting, the input measures correspond to distributions derived from counterfactual perturbations. These perturbations encode geometric information about the decision behavior of the black-box model, allowing the barycenter to serve as a behaviorally meaningful class-level representation without requiring explicit recovery of the decision boundary. A.2.1. R ELEVANT PROPERTIES We summarize only the properties required for the theoretical results in Section 3. • Existence. If each µi ∈ P2 (Rd ), then a Wasserstein barycenter µ∗ exists. • Convexity of the objective. The barycenter objective in Eq. (5) is convex in µ, ensuring that all minimizers attain the same optimal value. • Non-uniqueness. The barycenter may not be unique; however, as shown in Theorem 3.2, this non-identifiability does not affect the induced decision rule, which depends only on Wasserstein distance values rather than on a particular minimizer. 13
RECAST: Model Reconstruction via Counterfactual-Aware Wasserstein Geometry under Limited Data
A.3. Robust Risk Bounds and Barycentric Optima We begin by proving Theorem 3.2, which establishes that the minimizers of a CF-consistent robust risk upper bound admit a Wasserstein barycentric form. The subsequent results in this subsection further clarify the geometric interpretation and stability properties of these barycentric optima. Proof of Theorem 3.2. Proof. Fix a class c ∈ {0, 1}. We derive a tractable upper bound on the CF-consistent worst-case risk Rc (Qc ) := sup W22 (µ, Qc ),
Cc = {µ : W2 (µ, Pc ) ≤ εc , W2 (µ, Pcf ) ≤ δc }.
µ∈Cc
For any µ ∈ Cc , the triangle inequality yields W2 (µ, Qc ) ≤ W2 (Qc , Pc ) + εc ,
W2 (µ, Qc ) ≤ W2 (Qc , Pcf ) + δc .
Consequently, n 2 2 o W22 (µ, Qc ) ≤ min W2 (Qc , Pc ) + εc , W2 (Qc , Pcf ) + δc . Since for any αc ∈ [0, 1] and any a, b ≥ 0, min{a, b} ≤ (1 − αc )a + αc b, we obtain 2 2 W22 (µ, Qc ) ≤ (1 − αc ) W2 (Qc , Pc ) + εc + αc W2 (Qc , Pcf ) + δc . To control the cross terms, we apply Young’s inequality: for any η1 , η2 > 0 and any a ≥ 0, 2εc a ≤ η1 a2 +
ε2c , η1
2δc a ≤ η2 a2 +
δc2 . η2
Applying this inequality to the two squared terms yields 1 2 ≤ (1 + η1 )W22 (Qc , Pc ) + 1 + ε , η1 c 2 1 2 δ . W2 (Qc , Pcf ) + δc ≤ (1 + η2 )W22 (Qc , Pcf ) + 1 + η2 c W2 (Qc , Pc ) + εc
2
Combining the above bounds, we obtain for all µ ∈ Cc , W22 (µ, Qc ) ≤ (1 − αc )(1 + η1 )W22 (Qc , Pc ) + αc (1 + η2 )W22 (Qc , Pcf ) + Kc , where
1 2 1 2 Kc = (1 − αc ) 1 + εc + α c 1 + δ η1 η2 c
is independent of Qc . Taking the supremum over µ ∈ Cc yields the robust upper bound Rc (Qc ) ≤ (1 − αc )(1 + η1 )W22 (Qc , Pc ) + αc (1 + η2 )W22 (Qc , Pcf ) + Kc . Since Kc does not depend on Qc , minimizing this bound is equivalent to minimizing its quadratic part. Defining λc =
αc (1 + η2 ) ∈ [0, 1], (1 − αc )(1 + η1 ) + αc (1 + η2 )
we obtain the barycentric objective (1 − λc )W22 (Qc , Pc ) + λc W22 (Qc , Pcf ), whose minimizer is the 2-Wasserstein barycenter of Pc and Pcf with weights (1 − λc , λc ). 14
RECAST: Model Reconstruction via Counterfactual-Aware Wasserstein Geometry under Limited Data
Existence and interpretation of barycentric optima. Theorem 3.2 shows that minimizing a suitable robust upper bound on the CF-consistent worst-case risk leads to a Wasserstein barycentric prototype. We formalize this consequence below and clarify the role of the associated bounding parameters. Lemma A.1 (Barycentric optima induced by robust bounds). For each class c ∈ {0, 1}, consider the CF-consistent worst-case risk Rc (Qc ) := sup W22 (µ, Qc ),
Cc = {µ : W2 (µ, Pc ) ≤ εc , W2 (µ, Pcf ) ≤ δc }.
µ∈Cc
There exist parameters αc ∈ [0, 1] and η1 , η2 > 0 such that the minimizer of a valid robust upper bound on Rc (Qc ) is given by the 2-Wasserstein barycenter between Pc and Pcf with weight λc =
αc (1 + η2 ) . (1 − αc )(1 + η1 ) + αc (1 + η2 )
Equivalently, for this choice of parameters, n o arg min (1 − λc )W22 (Qc , Pc ) + λc W22 (Qc , Pcf ) Qc
minimizes a robust upper bound on the CF-consistent worst-case risk Rc (Qc ). Proof. This result follows directly from the proof of Theorem 3.2. For any αc ∈ [0, 1] and η1 , η2 > 0, the derivation above yields a robust upper bound of the form Rc (Qc ) ≤ (1 − αc )(1 + η1 )W22 (Qc , Pc ) + αc (1 + η2 )W22 (Qc , Pcf ) + Kc , where Kc is independent of Qc . Since multiplication of the objective by a positive constant does not affect its minimizers, the quadratic part of the bound can be renormalized to yield the barycentric objective with weight λc =
αc (1 + η2 ) . (1 − αc )(1 + η1 ) + αc (1 + η2 )
Remark A.2 (Barycentric lens of CF-consistent optima). For any fixed η1 , η2 > 0, the mapping αc 7−→ λc =
αc (1 + η2 ) (1 − αc )(1 + η1 ) + αc (1 + η2 )
is continuous and strictly increasing from [0, 1] to [0, 1]. Consequently, by varying αc , the induced barycentric weight λc ranges over the entire unit interval. Thus, every Wasserstein barycenter along the geodesic between Pc and Pcf is the minimizer of some CF-consistent robust upper bound on Rc . Geometrically, the family of robust optima fills the Wasserstein “lens” induced by the intersection BW2 (Pc , εc ) ∩ BW2 (Pcf , δc ), justifying the barycentric prototype interpretation used in Figure 3. Remark A.3 (Why no explicit bound tuning is required). Different choices of the bounding parameters (αc , η1 , η2 ) lead to different robust upper bounds but induce barycentric optima that lie on the same Wasserstein geodesic between Pc and Pcf . In practice, we therefore bypass explicit bound tuning and instead select the barycentric weight λc directly from the empirical geometry, yielding a CF-consistent representative of the robust equivalence class. Canonical barycentric classifier.
Given the surrogate pair (Q∗0 , Q∗1 ), we define the canonical barycentric classifier m̂(x) = arg min W2 (δx , Q∗c ). c∈{0,1}
15
RECAST: Model Reconstruction via Counterfactual-Aware Wasserstein Geometry under Limited Data
Corollary A.4 (Stability of the induced decision score). Let Q = (Q0 , Q1 ) and Q′ = (Q′0 , Q′1 ) be two pairs of classconditional prototypes. Define the decision score ∆Q (x) := W2 (δx , Q1 ) − W2 (δx , Q0 ), and the induced classifier m̂Q (x) := I[∆Q (x) ≤ 0] . Then, for all x ∈ X ,
∆Q (x) − ∆Q′ (x) ≤ W2 (Q0 , Q′0 ) + W2 (Q1 , Q′1 ).
Consequently, for any L-Lipschitz loss ϕ : R → [0, 1], Ex∼PX [ϕ(∆Q (x))] − Ex∼PX [ϕ(∆Q′ (x))] ≤ L(W2 (Q0 , Q′0 ) + W2 (Q1 , Q′1 )) . Proof.
By the triangle inequality of the Wasserstein distance, W2 (δx , Qc ) − W2 (δx , Q′c ) ≤ W2 (Qc , Q′c )
for c ∈ {0, 1}.
Subtracting the two class scores yields the first inequality. The second inequality follows directly from the Lipschitz continuity of ϕ and Jensen’s inequality. □ Remark.
If there exists a reference distribution µc ∈ Cc such that Rc (Qc ) ≤ εc , then √ W2 (Qc , µc ) ≤ εc .
Hence, reconstruction error measured by E (Q0 , Q1 ) = R0 (Q0 ) + R1 (Q1 ) controls the stability of the induced decision √ score up to O( ε). This provides a geometric interpretation of why minimizing the barycentric objective promotes behavioral stability under CF-consistent uncertainty. A.4. Proof of Lemma 3.4 Proof. Fix λ, λ′ ∈ [0, 1] and define gλ (µ) := W22 (µ, Q⋆c (λ)). By definition of the supremum, Rc (Q⋆c (λ)) − Rc (Q⋆c (λ′ )) = sup gλ (µ) − sup gλ′ (µ) µ∈Cc
µ∈Cc
≤ sup gλ (µ) − gλ′ (µ) . µ∈Cc
For any fixed µ ∈ Cc , let a := W2 (µ, Q⋆c (λ)) and b := W2 (µ, Q⋆c (λ′ )). Using the identity |a2 − b2 | = |a − b|(a + b) together with the reverse triangle inequality for W2 , we obtain gλ (µ) − gλ′ (µ) ≤ a + b W2 Q⋆c (λ), Q⋆c (λ′ ) . Next, since Cc = {µ : W2 (µ, Pc ) ≤ εc , W2 (µ, Pcf ) ≤ δc }, we have for any µ ∈ Cc and λ ∈ [0, 1], W2 (µ, Q⋆c (λ)) ≤ W2 (µ, Pc ) + W2 (Pc , Q⋆c (λ)) ≤ εc + W2 (Pc , Pcf ), where the last inequality follows from the fact that (Q⋆c (λ))λ∈[0,1] is a Wasserstein geodesic between Pc and Pcf . An analogous bound holds for W2 (µ, Q⋆c (λ′ )). Moreover, by the constant-speed property of the geodesic, W2 Q⋆c (λ), Q⋆c (λ′ ) = |λ − λ′ | W2 (Pc , Pcf ). Combining these bounds yields gλ (µ) − gλ′ (µ) ≤ 2 εc + W2 (Pc , Pcf ) W2 (Pc , Pcf ) |λ − λ′ |. Taking the supremum over µ ∈ Cc concludes the proof. 16
RECAST: Model Reconstruction via Counterfactual-Aware Wasserstein Geometry under Limited Data
Algorithm 1 Optimization for discrete prototypes used in experiments Require: Encoded point clouds D0 , D1 , Dcf ⊂ Rd ; support size M ; iterations Tmax ; learning rate η; random seed s; Sinkhorn parameters blur. b 0, Q b 1. Ensure: Prototypes Q 1: Set seed s. f 2 (·, ·) using Sinkhorn with parameters blur and uniform weights for all discrete 2: Define entropic OT surrogate W 2 measures. 3: for each class c ∈ {0, 1} do f 2 between Dcf , Dc , and D1−c . 4: Derive λc using W 2 5: end for 6: 7: Initialization. For each class c ∈ {0, 1}, randomly sample M points without replacement from Dc to initialize the
support locations Qc ∈ RM ×d ; set support weights to uniform (fixed) 8: Initialize Adam optimizer over support locations {Q0 , Q1 } with learning rate η. 9: for t = 1 to Tmax do f 2 (Q0 , D0 ) + λ0 W f 2 (Q0 , Dcf ). 10: L0 ← (1 − λ0 ) W 2 2 f 2 (Q1 , D1 ) + λ1 W f 2 (Q1 , Dcf ). 11: L1 ← (1 − λ1 ) W 2 2 12: L ← L 0 + L1 . 13: Take an Adam step on the support locations of Q0 , Q1 to minimize L . 14: end for b 0 ← Q0 15: Set Q b 1 ← Q1 16: Set Q b 0, Q b1 17: return Q A.5. Theoretical Guarantee of Convergence and Sinkhorn Divergence
Let P0 , P1 , Pcf ∈ P2 (X ) be probability measures with finite second moments supported on a compact metric space X ⊆ Rd and λc ∈ [0, 1] constants. Assume the optimization is carried out over the 2-Wasserstein space W2 (X ). Remark A.5. Our loss function L defined as: X L (Q0 , Q1 ) = (1 − λc )W22 (Qc , Pc ) + λc W22 (Qc , Pcf ) , c∈{0,1}
admits a unique minimizer (Q⋆0 , Q⋆1 ) ∈ P2 (X ) × P2 (X ). We can decompose the loss function L (Q0 , Q1 ) in two independent parts LQ0 + LQ1 , with LQc = (1 − λc )W22 (Qc , Pc ) + λc W22 (Qc , Pcf ) , for c ∈ {0, 1}, which is essentially a version of a Wasserstein barycenter in Eq. (5) with µ = Qc , λi = λc , (1 − λc ), N = 2, and µ1 and µ2 the probability measures with finite second moments Pc and Pcf respectively. It is also shown that the optimal solution for p = 2, is the geodesic curve provided by the McCann’s interpolant (McCann, 1997; Agueh & Carlier, 2011). As a sum of two Wasserstein barycenters L (Q0 , Q1 ) admits thus an optimal solution (Q∗0 , Q∗1 ). Approximation Theoretical results in Sections 3.1- 3.3 are derived for the exact 2-Wasserstein distance W2 . In practice, we replace W2 with the Sinkhorn divergence Sϵ to obtain a smooth and computationally efficient objective using entropic regularization. This corresponds to optimizing a regularized (smoothed) approximation of the Wasserstein barycenter problem rather than the exact Wasserstein DRO objective. As ϵ → 0, the Sinkhorn divergence converges to W2 , and the solution approaches the true Wasserstein barycenter. For a regularization parameter ϵ, the entropically regularized Wasserstein distance is defined as follows: Z 2 Wϵ,2 (µ, ν) := min ∥y − x∥22 dγ(x, y) + ϵH(γ|µ ⊗ ν), | {z } γ∈Π(µ,ν) | {z } relative entropy transport cost
17
RECAST: Model Reconstruction via Counterfactual-Aware Wasserstein Geometry under Limited Data
Sinkhorn divergence is defined as: 1 2 1 2 2 Sϵ (µ, ν) := Wϵ,2 (µ, ν) − Wϵ,2 (µ, µ) − Wϵ,2 (ν, ν). 2 2 The modified loss function is then X
Lϵ (Q0 , Q1 ) =
(1 − λc )Sϵ (Qc , Pc ) + λc Sϵ (Qc , Pcf ) .
c∈{0,1} 2 Approximation quality increases as ϵ → 0: Wϵ,2 (µ, ν) → W22 (µ, ν), therefore the minimizers of Lϵ (Q0 , Q1 ) converge to the minimizers of L (Q0 , Q1 ). ′
′
Chizatp et al. (2020) show that for ϵ ≈ n−1/(d +4) , they achieve with probability 1 − θ that |Sϵ,n − W22 | ≲ n−2/(d +4) + −1/2 log(2/θ), with n being the number of independent samples in Rd and d′ denoting 2⌊d/2⌋. When choosing n ′ n ≳ log(2/θ)ε−(d +4)/2 , they achieve the desired ε-accuracy with 1 − θ. Thus they conclude a computational complexity of O(n2 /(ϵε)). A.6. Sensitivity to Sinkhorn Regularization The Sinkhorn algorithm introduces a regularization parameter ϵ that controls the trade-off between computational efficiency and approximation of the exact Wasserstein distance. We ablate the blur parameter on Adult (query size n = 50) by sweeping the Sinkhorn blur (ϵ) from 0.2 to 0.005. Fidelity remains stable across the range of values, while the reference-set mean margin Ex |W2 (x, Q∗1 ) − W2 (x, Q∗0 )| decreases slightly from 82.66 to 81.08. This indicates that smaller blur leads to a modest reduction in margin, while the predicted labels remain largely unchanged, signaling stable decision behavior despite changes in regularization. A.7. Fairness Let P denote the data-generating distribution over (X, Y, S), where S ∈ {0, 1} is the sensitive attribute, X the random input variable, and Y ∈ {0, 1} the class label as a random variable with realizations y ∈ {0, 1}. Demographic parity (Dwork et al., 2012; Feldman et al., 2015) describes that the probability of an individual to be assigned to a class y should not depend on the sensitive group S: Demographic Parity
P (Ŷ = y | S = 0) = P (Ŷ = y | S = 1).
Equalized Odds (Hardt et al., 2016) describes that the prediction accuracy should not depend on the sensitive group S: ( P (Ŷ = 1 | S = 0, Y = 0) = P (Ŷ = 1 | S = 1, Y = 0) Equalized Odds P (Ŷ = 1 | S = 0, Y = 1) = P (Ŷ = 1 | S = 1, Y = 1). We adapt threshold-invariant demographic parity disparity (TIDP) and threshold-invariant equalized odds disparity (TIEO) from Chen & Wu (2020). Specifically, we define threshold-invariant demographic parity disparity (TIDP) as the Wasserstein-1 distance between the score distributions conditioned on different values of S: p# P (X | S = 0), DDTIDP = W1 . p# P (X | S = 1) Analogously, threshold-invariant equalized odds disparity (TIEO) compares the score distributions conditioned on both S and the label Y ∈ {0, 1}: p# P (X | S = 0, Y = y), DDTIEO (y) = W1 . p# P (X | S = 1, Y = y) These diagnostics capture disparities that persist across all decision thresholds, as they operate on entire score distributions. Moreover, they quantify how different sensitive groups are positioned relative to the barycentric decision geodesic. 18
RECAST: Model Reconstruction via Counterfactual-Aware Wasserstein Geometry under Limited Data
B. Experimental Setup The following subsections deal with datasets, and implementation as well as experiment details. B.1. Description and Pre-processing of Real-World Benchmark Datasets To evaluate our framework, we employ four well-known publicly available tabular datasets: Adult Income, California Housing, COMPAS, and HELOC. Below are their key characteristics: • Adult Income: Derived from the 1994 U.S. Census, this dataset captures demographic and financial attributes such as education level, marital status, age, and annual earnings. The classification task involves predicting whether an individual’s income exceeds $50,000 (denoted as y = 1). The original dataset consists of 32,561 entries, with 24,720 labeled as y = 0 and 7,841 as y = 1. To balance the classes, we randomly selected 7,841 samples from y = 0, resulting in a final dataset of 15,682 entries. The dataset includes 6 numerical and 8 categorical features, with the latter converted to integer encodings. All features were normalized to [0, 1]. • California Housing Prices (Housing): The dataset consists of data for houses in a district in California collected in the 1990 census. The dataset consists of 20,640 samples and 9 features. The target variable is the average house value, we transform it into a binary target variable by setting the median house value as a threshold. • COMPAS: Developed to study racial bias in recidivism prediction algorithms, this dataset contains 6,172 entries with 20 numerical features. The target variable, is recid divides the data into 3,182 (y = 0) and 2,990 (y = 1) samples. Feature values were normalized to [0, 1]. • Home Equity Line of Credit (HELOC): This dataset records credit risk assessments for customers seeking home equity loans. It comprises 10,459 entries, each with 23 numerical features. The prediction target, is at risk identifies customers likely to default. The dataset is moderately imbalanced, with 5,000 samples for y = 0 and 5,459 for y = 1. Pre-processing. We adopt a unified data loading and pre-processing pipeline across all datasets. For each dataset, we construct training, validation, and test splits using stratified sampling, holding out 20% for testing and 20% of the remaining data for validation. Target labels are standardized by stripping whitespace and removing trailing punctuation where applicable. Features containing missing indicators (e.g., “?”, or dataset-specific sentinel values) are mapped to NaN, infinite values are removed, and rows with missing targets are discarded. Features are then automatically partitioned into continuous and categorical variables based on data type. Continuous features are median-imputed and scaled using a RobustScaler with the 5–95 percentile range, while categorical features are imputed with the most frequent value and one-hot encoded with unknown-category handling. All transformations are fit on the training split only and applied to validation and test data. We further store metadata such as output feature names, continuous and categorical index locations, one-hot category mappings, and scaler parameters, which are subsequently used both for model training and for decoding counterfactual examples back into raw feature space. For the California Housing dataset, we follow prior work and binarize the target at the median value to obtain a balanced binary prediction task. Importantly, we do not assume access to the target model’s internal pre-processing pipeline. While a unified pipeline is used within each experimental run to ensure internal consistency, the reconstruction model does not share or reuse the exact feature transformations employed by the target model. In this sense, pre-processing is treated as an integral part of the black-box system and is not exposed to the auditor. We note that most existing model reconstruction and CF-based extraction methods implicitly assume that the attacker operates in the same feature representation as the target model, and therefore do not explicitly study the effect of preprocessing or representation misalignment. In practice, however, differences in normalization, encoding, imputation, or feature engineering can substantially alter the geometry of the input space and thereby impact reconstruction performance. This distinction is particularly relevant when reproducing prior baselines: we observe that reconstruction fidelity can be noticeably lower than the values reported in earlier work, which we attribute in part to the absence of shared pre-processing assumptions. We view this setting as more reflective of realistic auditing scenarios, where the feature transformation pipeline is typically proprietary and inaccessible. 19
RECAST: Model Reconstruction via Counterfactual-Aware Wasserstein Geometry under Limited Data
B.2. Implementation Details All experiments were implemented in Python 3.12 and conducted on a workstation equipped with an NVIDIA RTX 3090 GPU, with 124 GB of RAM running on Ubuntu 24.04. We used the following libraries for the implementation: pandas, scikit-learn, geomloss, numpy, torch and official code for the CF generation methods. Hyperparameters are set according to a grid search. Our code is included in the supplementary material. B.3. Experiment Details The following sections present details regarding the experiment setup. Target classifiers. Across all datasets, we treat several standard tabular models as black-box target classifiers. Following the experimental protocol in the main paper, we train a logistic regression model (LR), a multilayer perceptron (MLP), and tree-based classifiers on the pre-processed feature representations obtained from our ColumnTransformer pipeline. The LR model is implemented as a single linear layer with a sigmoid output, optimized using Adam and a binary cross-entropy loss with logits, with early stopping on validation loss. The main MLP consists of two hidden layers with 20 and 10 units, respectively, with ReLU activations and a dropout rate of 0.1, followed by a single-unit output layer; it is trained with Adam, a learning rate of 10−3 and weight decay of 10−4 , again with early stopping. For tree-based targets, we employ a decision tree (DT) with maximum depth 6 and minimum leaf size 20. All models are trained on the training split only and evaluated on the held-out test split using fidelity as the primary metric. Baselines. We compare our method against several representative reconstruction baselines that incorporate counterfactual explanations. Most of them are designed for interactive online access, however, since our setting is constrained to offline-only, we adapt the baselines and restrict them to offline data too. The first baseline, SAMPLES, follows Aı̈vodji et al. (2020) and simply augments the observed labeled samples with their CFs, treating CFs as ordinary labeled points when training standard classifiers such as LR, MLP and DT on the pre-processed features. The second baseline, CCA (Counterfactual Clamping Attack), implements the counterfactual clamping loss of Dissanayake & Dutta (2024): we train a small neural surrogate network with hidden layers of sizes (20, 10, 5) and ReLU activations, followed by a sigmoid output, on tri-valued labels y ∈ {0, 0.5, 1}, where 0.5 denotes CFs. For regular samples (y ∈ {0, 1}) we use standard binary cross-entropy, while CFs are encouraged to have predictions above a threshold k = 0.5 via the clamping objective, which only penalizes CFs whose predicted probability does not exceed k. For completeness, we also report results for TRA (Khouna et al., 2025), which is competitive in our setting but restricted to tree-based models; we therefore instantiate TRA only for tree targets and include it as an additional baseline in the appendix. CF generation methods. We implement several CF generators in a unified framework. The 1-Nearest-Neighbor (1NN) method searches over a pool of raw samples whose predicted label matches the desired outcome and selects the point with minimal mixed cost (ℓ1 distance on continuous features plus Hamming distance on categorical features). The differentiable MCCF variant optimizes a CF directly in encoded feature space using gradient-based updates on a scaled representation, with continuous dimensions updated freely and categorical groups projected to (approximate) one-hot vectors via straight-through Gumbel–Softmax; the objective trades off a prediction loss that pushes the output to the target class against an ℓ1 -style proximity term. Our DiCE implementation follows a randomized search strategy in encoded space, sampling perturbations of continuous dimensions and randomly flipping categorical one-hot entries only for a specified subset of features, and returns the first candidate that achieves the desired prediction with minimal raw-space cost. For C-CHVAE, we train a tabular VAE in encoded space and then optimize in the latent space to find codes whose decoded samples both lie close to the original point (in encoded distance) and are classified as the target label; successful decoded candidates are mapped back to raw feature space via the inverse preprocessing pipeline. Finally, ROAR is implemented as a robust MCCF-style optimizer in encoded space replacing the standard prediction term with a robust probability under weight noise, aggregating predictions over multiple noisy parameter draws (either by averaging or worst-case aggregation), while using the same continuous and categorical distance regularizers; this yields CFs that remain valid under moderate target-model parameter shifts. As shown in Fig. 2, CF generation methods exhibit substantial variability in the prediction scores assigned by the target model, with some overshooting the decision threshold. CCA assumes that CFs lie close to the decision boundary and leverages such near-boundary samples to shape the surrogate decision surface. When CFs instead yield prediction scores far above 0.5, this assumption is violated and reconstruction performance deteriorates. By contrast, RECAST neither interprets CFs as true samples (like SAMPLES) nor assumes proximity to the decision boundary. Instead, it incorporates CF 20
RECAST: Model Reconstruction via Counterfactual-Aware Wasserstein Geometry under Limited Data
information through CF-consistent distributional geometry, which enables stable reconstruction even under heterogeneous CF generation behaviors. Wasserstein Barycenter Computation We compute entropically regularized Wasserstein costs using the SamplesLoss("sinkhorn", p=2) implementation from GeomLoss (Feydy et al., 2019), which provides a differentiable Sinkhorn approximation of W2 between empirical measures. To construct interpretable class representations, the experiment leverages 2-Wasserstein distances using the Sinkhorn algorithm implemented in the geomloss library. The objective is to learn two barycenters Q0 and Q1 that serve as prototypical representatives of each class in the transformed feature space. The loss function incorporates the following components: Wasserstein distance from class 0 samples to Q0 and Q1 . Each barycenter is initialized with M support points (default M = 50). Unless stated otherwise, all experiments use a fixed support size of M = 50. Optimization is performed using the Adam optimizer with a learning rate of 0.01 for 200 epochs. The formulation of the loss function follows Equation 3.2 from Section 3, with all Wasserstein distances replaced by their entropically regularized Sinkhorn surrogates. Hyperparameter Sensitivity. We conduct an extensive ablation study to assess the sensitivity of RECAST with respect to its optimization and regularization parameters. Specifically, we perform a grid search over the learning rate η ∈ {10−3 , 5 × 10−3 , 10−2 }, the number of training epochs {100, 200, 400}. In addition, since Wasserstein barycenters are computed via an entropically regularized Sinkhorn approximation, we evaluate the effect of the entropic regularization parameter blur ∈ {10−3 , 5 × 10−3 , 10−2 , 5 × 10−2 }. Unless stated otherwise, we fix η = 10−2 , blur = 5 × 10−2 , and train for 200 epochs in all experiments, which we found to provide stable numerical behavior in practice. Figure 8 illustrates the convergence behavior of the objective in Equation 3.2 on the Adult dataset under different query sizes k.
Figure 8. Loss convergence over epochs on Adult dataset.
Repetition All quantitative analyses were averaged across 10 independent repetitions to account for variability in sampling and model behavior.
21
RECAST: Model Reconstruction via Counterfactual-Aware Wasserstein Geometry under Limited Data
C. Additional Experiments This section of the Appendix includes additional experiments, that could not be included in the main paper due to space limitations. C.1. Ablation Study We conduct a ablation study to assess RECAST with respect to its design choices. First, we analyze the role of the class-conditional weighting λc by selectively disabling each component. As shown in Table 4, removing either component degrades fidelity, while disabling both leads to substantial performance loss, highlighting their complementary roles. Second, we replace Wasserstein barycenters with other prototype constructions, such as the Euclidean mean, medoid-based prototypes and MMD. Here, CFs can only be included as labeled data. We emphasize that this difference is not a design choice but a structural limitation of the compared prototype representations. Euclidean mean and medoid prototypes operate on point estimates and do not admit a natural mechanism to incorporate CFs as soft, distributional constraints. These alternatives consistently underperform, particularly in low-query regimes, underscoring the importance of Wasserstein geometry for capturing CF-induced distributional structure. However, MMD can be plugged in our design easily with similar λc , with only Wasserstein distance changed to MMD. Finally, we evaluate the effect of counterfactual information by removing CFs from the reconstruction. As shown in Table 4, excluding CFs leads to a substantial drop in fidelity, supporting their role in shaping boundary-relevant geometry rather than pinpointing the boundary itself. Sensitivity to optimization hyperparameters (learning rate and training epochs) is reported in Appendix C. Table 4. Ablation study of RECAST (Adult, MLP, query size = 100). Method Variant
CFs
Prototype
λc
Loss ↓
Fidelity (%) ↑
RECAST (Full)
✓
Wasserstein
Ours
2.18
90.8
λc Fixed 1−λ c
✓
Wasserstein
λc 1−λc = 0.5
2.61
88.7
Euclidean prototype Medoid prototype Maximum Mean Discrepancy
✓ ✓ ✓
Euclidean mean Medoid RBF-MMD
– – Similar to Ours
3.12 2.88 2.65
82.4 84.1 87.3
No CFs
×
Wasserstein
–
3.05
81.6
C.2. Imbalanced Data In our main experiments, we used the same value of query size for D0 , D1 , and Dcf . To evaluate the robustness of our approach under imbalanced data conditions, we conducted additional experiments using varying sample sizes for the counterfactual dataset. These experiments simulate more realistic, unbalanced settings in which the sizes of D0 , D1, and Dcf may differ, while the total number of natural samples satisfies |D0 | + |D1 | = 200. As shown in Table 5, our method maintains strong performance even under these imbalanced conditions. Table 5. Fidelity under different class imbalance ratios. Dataset
20% class 0 : 80% class 1 SAMPLES CCA RECAST (Ours)
80% class 0 : 20% class 1 SAMPLES CCA RECAST (Ours)
Adult In. COMPAS HELOC Housing
0.751 ± 0.017 0.385 ± 0.007 0.397 ± 0.014 0.429 ± 0.008
0.702 ± 0.016 0.352 ± 0.011 0.365 ± 0.016 0.391 ± 0.010
0.793 ± 0.018 0.715 ± 0.018 0.604 ± 0.052 0.666 ± 0.065
0.848 ± 0.030 0.795 ± 0.020 0.646 ± 0.071 0.662 ± 0.112
0.762 ± 0.023 0.638 ± 0.022 0.561 ± 0.045 0.624 ± 0.057
0.819 ± 0.027 0.738 ± 0.025 0.592 ± 0.059 0.615 ± 0.088
C.3. Experiments including Accuracy Table 6 reports target model accuracy alongside surrogate accuracy and fidelity for RECAST, SAMPLES, and CCA on the Adult and COMPAS datasets at query sizes n = 100 and n = 150, using logistic regression as the target model and 1-nearest-neighbor as the CF method. We report both metrics because they capture different objectives: accuracy measures predictive performance against ground-truth labels, whereas fidelity measures agreement with the target model’s predictions, including its errors. Consequently, the two metrics need not correlate, a surrogate can achieve high fidelity while having lower task accuracy if the target model itself makes systematic errors. RECAST achieves the highest fidelity across both 22
RECAST: Model Reconstruction via Counterfactual-Aware Wasserstein Geometry under Limited Data Table 6. Target model accuracy, surrogate accuracy, and fidelity at query sizes n = 100 and n = 150 on Adult and COMPAS. Accuracy and fidelity need not correlate: fidelity measures agreement with the target model’s predictions (including its errors), not ground-truth label accuracy. Best surrogate results per dataset and query size in bold. n = 100
n = 150
Dataset
Method
Target Acc.
Acc.
Fid.
Acc.
Fid.
Adult
Samples CCA RECAST
0.8463 0.8463 0.8463
0.7463 0.5598 0.7741
0.7800 0.5097 0.8400
0.7488 0.5318 0.7700
0.7679 0.4797 0.8600
COMPAS
Samples CCA RECAST
0.6696 0.6696 0.6696
0.5983 0.5405 0.5580
0.7004 0.5441 0.7560
0.5424 0.5376 0.5770
0.7518 0.5398 0.7860
Table 7. Accuracy and fidelity on Folktables (ACSIncome) under cross-domain distribution shift (train: CA 2018, evaluate: MI 2014) at query sizes 100 and 150.
Reconstruction SAMPLES CCA RECAST
Query size = 100 Acc. Fid. 0.7356 0.8641 0.7302 0.6362 0.6473 0.8661
Query size = 150 Acc. Fid. 0.7499 0.8596 0.7251 0.6251 0.7254 0.8886
datasets and query sizes, while remaining competitive on accuracy, demonstrating that behavioral agreement with the target is not achieved at the expense of predictive performance. C.4. Additional Experiments on Distribution shifts To investigate robustness under realistic distribution shift, we conduct an additional experiments using Folkstables (Ding et al., 2021). We train a logistic-regression target model on ACSIncome data from California (CA), 2018 (acc=0.7860), and evaluate reconstruction under shift to Michigan (MI), 2014, a setting that combines both geographical and temporal variation. We generate CFs using 1-nearest-neighbour and evaluate the performance (fidelity and accuracy) on query budget (100, 150). Results are shown in Table 7. RECAST consistently achieves higher fidelity across query budgets (100 and 150), demonstrating robust recovery under distribution shift. SAMPLES leads on accuracy but trails substantially on fidelity, consistent with the pattern observed in Section C.3, accuracy and fidelity capture different objectives, and fidelity is the primary measure of reconstruction quality. These results demonstrate that RECAST maintains strong behavioral consistency even under substantial distribution shifts. C.5. Crossover Between RECAST and No-CF Reconstruction We investigate when CF supervision becomes less critical for model reconstruction under increasing data availability. Specifically, we compare RECAST with a no-CF baseline trained solely on natural samples, as the number of available samples grows. We fix a target model trained on the Adult dataset and evaluate reconstruction fidelity, defined as the fraction of test instances on which the surrogate and the target model produce identical predictions. For the no-CF baseline, we use a decision tree surrogate, which exhibits stable behavior in low-sample regimes while allowing increased expressivity as more data become available. Figure 9 illustrates a crossover pattern. In low-data regimes, CF-based reconstruction achieves substantially higher fidelity, highlighting its strong sample-efficiency advantage. As the number of natural samples increases, the performance gap narrows and reverses around n ≈ 1000 samples per class, where the no-CF surrogate attains comparable or higher fidelity. This pattern reflects a trade-off between data efficiency and surrogate expressivity in audit-oriented reconstruction settings. Counterfactual supervision provides informative local constraints when observational data are scarce, but yields diminishing benefits once sufficient coverage of the input space is available. 23
RECAST: Model Reconstruction via Counterfactual-Aware Wasserstein Geometry under Limited Data
Figure 9. Crossover between CF-based and no-CF reconstruction on the Adult dataset. CF-based reconstruction achieves higher fidelity in low-data regimes, while the no-CF decision tree surrogate improves with increased natural samples and surpasses the CF-based method around n ≈ 1000 per class.
C.6. Data Modalities / High-Dimensional Data Our framework is generally applicable to any data modality that can be embedded as vectors, e.g., pixel representations of visual data. Thus, we can apply RECAST to high-dimensional datasets after transforming the input data to latent space. To show this, we perform an additional proof-of-concept on the well-known MNIST dataset (Deng, 2012). We load the dataset from the sklearn library, normalize the pixel values to be in-between [0, 1] and reduce the classification problem to binary (e.g., classification between 0 and 1). For the target classifier, we employ 66% to train a standard sklearn MLP-classifier and generate CFs as the nearest neighbor belonging to the other class. RECAST is then trained on a subset of the training set in pixel-space (d = 784). Across different binary tasks and sample sizes (around 100 - 500), RECAST consistently achieves a fidelity around 0.7 or higher, see Figure 10. C.7. Language Case Study: Reconstruction under One-Sided Textual CFs We present an exploratory case study on reconstruction for text-based classifiers under limited access. The target model is a binary classifier m : Rd → {0, 1} trained on fixed text embeddings. Texts x ∈ X are mapped to embeddings ψ(x) ∈ Rd using a frozen LLM-based encoder. During reconstruction, the target model is accessed only through discrete label queries m(ψ(x)). Representation space. Since optimal transport is not defined over raw text, all reconstruction is performed in the embedding space induced by ψ. The embedding model is fixed and shared across all methods. 1.00 0-1 0-8 1-7 3-5 3-6 3-8 4-7 5-8
0.95
Fidelity
0.90 0.85 0.80 0.75 0.70 100
200
300
400
Sample Size (Both Classes)
500
Figure 10. Fidelity results for various sample sizes and binary classification problems within the MNIST dataset.
24
RECAST: Model Reconstruction via Counterfactual-Aware Wasserstein Geometry under Limited Data
Runtime (seconds)
Dataset
MNIST Adult Adult (10) Adult (50)
60 40 20 0
500
1000
1500
Sample Size (Both Classes)
2000
Figure 11. Runtime for RECAST across different datasets and configurations. Each RECAST model is trained for 500 epochs.
Task and target model. We consider a toxicity classification task (Jigsaw, 2018). A binary classifier m is trained on embeddings ψ(x) obtained from a standard toxicity dataset and is treated as a black-box model during reconstruction. One-sided textual CFs. To emulate one-sided recourse, CFs are generated only for toxic instances with m(ψ(x)) = 0. For each such input x, a single textual CF xcf is produced using an LLM-based rewriting prompt that aims to minimally modify the text toward a non-toxic outcome. The resulting text is embedded via ψ, and the CF is retained only if m(ψ(xcf )) = 1. This yields three offline datasets: D0 = {x : m(ψ(x)) = 0}, D1 = {x : m(ψ(x)) = 1}, and Dcf = {xcf : x ∈ D0 }. Reconstruction . objective.
We apply RECAST in the embedding space to compute class prototypes Q0 and Q1 using the barycentric
Evaluation. We evaluate query budgets of 100, 150, and 200 samples per class. Reconstruction fidelity is consistently lower than in tabular settings, typically saturating around 80%. We attribute this to strong representation compression: text inputs are mapped to fixed embeddings and further reduced to a 50-dimensional space. C.8. Runtime To investigate runtime, we vary the number of samples used to train RECAST. The runtime experiments are performed on a MacBook Pro with 24 GB RAM and an Apple M4 Pro. We first split the full dataset in 33% test and 67% training data. Then we take different fractions, of the training set to train RECAST. We include MNIST (d = 784) and three versions of adult (d = 14 before pre-processing, d = 104 with one-hot-encoding) with varying feature sets (d = 104, d = 50, d = 10) to account for different sample dimensionalities. Each RECAST instance is trained for 500 epochs. Note that implementation improvements like early stopping could improve these runtimes, however we want to focus on a fair comparison and trained each instance the same amount of epochs. To mitigate influence of background tasks, we performed 5 runs per configuration. While the complexity of our approach is high in theory, we observe, see Figure 11, that the runtime in practice grows near linear with the number of samples. We additionally see that the number of features, compare adult with 104, 50, and 10 features, has even less impact on the runtime.
25