M ODEL S TEALING T HROUGH THE L ENS OF M ODEL M ULTIPLICITY
Eliott Baltz§ , Satoshi Hara‡ , and Ulrich Aïvodji§
arXiv:2606.15493v1 [cs.LG] 13 Jun 2026
‡
The University of Electro-Communications : Tokyo § ÉTS, Mila : Montréal
A BSTRACT Model stealing attacks, where adversaries create high-fidelity surrogate models, are a significant threat to the intellectual property of machine learning services. Conventional wisdom suggests these surrogates could provide adversaries with economic leverage comparable to the original service providers. This paper challenges this assumption by evaluating model stealing attacks beyond mere fidelity to the target model. Because query-based extraction provides only partial supervision of the target’s input-output behavior, the surrogate is not uniquely identified: many near-optimal surrogates can achieve comparable fidelity while differing in deployment-relevant properties. Instead of performing a classic learning-based model stealing attack, we compute the Rashomon Set (i.e., the set of almost-equally-accurate models) of surrogate models, and evaluate its diversity using multiplicity metrics (ambiguity, discrepancy, and Rashomon Capacity) and group fairness metrics. Across tabular, medical imaging, and NLP tasks, our experiments on real-world datasets reveal that despite exhibiting similar fidelity to the target model, surrogate models can display significant variances in other critical performance metrics. These findings cast doubt on the presumed equivalence between high-fidelity surrogates and the target model in practical deployment scenarios.
1
Introduction
Machine learning models have become integral to various applications, yet they remain vulnerable to a range of attacks that threaten their integrity and the business models built around them [Shokri et al., 2017, Fredrikson et al., 2014, Ateniese et al., 2015, Carlini et al., 2019, Tramèr et al., 2016, Kumar et al., 2020]. Among these, model stealing attacks [Tramèr et al., 2016] pose a significant challenge, particularly in the context of Machine Learning as a Service (MLaaS). Adversaries exploit query access to a target model to create a surrogate model that mimics its performance, potentially compromising the intellectual property of the original service provider. Traditionally, the success of model stealing attacks has been gauged by the fidelity of the surrogate model, defined as its label agreement with the target model on a reference dataset. High-fidelity surrogates have therefore been interpreted as evidence of a significant intellectual-property loss, suggesting that the adversary has effectively replicated the target model’s capabilities. However, recent advancements in our understanding of model multiplicity, particularly through the lens of Rashomon Sets [Semenova et al., 2022], challenge this simplistic interpretation. The Rashomon Set, consisting of models that perform approximately as well as a reference model, has been shown to include models with different practical performance profiles. This diversity manifests in conflicting predictions on individual samples [Marx et al., 2020] and varying performance under distribution shifts [D’Amour et al., 2022]. Similarly, research on post-hoc explanations has revealed that high-fidelity explainers, which can be viewed as surrogate models of a different function class, often display disparate performance in fairness metrics, facilitating explanation manipulation attacks such as fairwashing [Aïvodji et al., 2021]. In light of these findings, we pose a critical question: Does high-fidelity model stealing truly produce surrogate models with comparable deployment behavior? Our investigation suggests that claims of intellectual property loss based only on fidelity may be incomplete, and that new metrics are necessary to characterize the similarity between
Rashomon Set of surrogate models
x₁
ŝ ₂
t(x₁) ŝ ₃
Reconstruct
Evaluate
xₙ ŝ ₄
t(xₙ)
ŝ ₅
ŝ ₁
Target model (t)
Adversary extraction
Multiplicity and fairness metrics ŝ ₙ
Figure 1: Overview of extraction-induced multiplicity. An adversary queries the target model t on auxiliary inputs and uses the returned predictions to reconstruct a surrogate model s. Rather than treating this surrogate as a unique clone of the target, we study the surrounding Rashomon Set of near-equivalent surrogate models that can arise from the extraction process. We then evaluate this set using fidelity, multiplicity metrics, and group fairness metrics. high-fidelity surrogates and their target models. Given the existence of a sizable Rashomon Set, a firm attempting to “steal” another firm’s model may recover a model that matches overall fidelity yet disagrees with the source model at the instance level. In those contexts, “successful” theft judged by fidelity alone can fail to replicate downstream behavior, redistribute errors across populations, and alter individual recourse, even when fidelity is high. Accordingly, evaluating surrogates demands metrics beyond accuracy and fidelity, including ambiguity, discrepancy, Rashomon Capacity, and fairness metrics, since fidelity is an incomplete proxy for functional equivalence. Viewed through this lens, model stealing is an identification problem under multiplicity: query access constrains the surrogate only on a finite region of the input space, leaving a potentially large set of near-equivalent predictors consistent with the observed query-response pairs. Our evaluation, therefore, targets the structure of this extractioninduced Rashomon Set rather than treating a single high-fidelity surrogate as representative. In this paper, we focus on fidelity extraction. Figure 1 summarizes this setting and highlights the step studied in this work: after reconstructing a surrogate, we evaluate the surrounding extraction-induced Rashomon Set rather than treating the stolen model as a unique clone of the target. We study this phenomenon across three domains: tabular prediction tasks, medical image classification, and NLP sentiment classification. This lets us test whether extraction-induced multiplicity is specific to one model family or whether it persists across neural networks, convolutional architectures, and transformer-based language models. This paper makes the following key contributions: • We introduce a method for generating dropout-based Rashomon Set sets around extracted surrogate models, allowing us to study model stealing as a multiplicity problem rather than as a single-model fidelity problem. • We evaluate extraction-induced multiplicity across tabular data, medical imaging, and NLP tasks. • We characterize the diversity of extracted Rashomon Set sets using ambiguity, discrepancy, Rashomon Capacity, and group fairness metrics. • Our empirical results demonstrate that surrogate models can exhibit high fidelity while still displaying different behavior, including conflicting sample-level predictions, attack-dependent multiplicity, and variable fairness values on tabular tasks. These findings cast doubt on the prevailing interpretation of high-fidelity model extraction attacks and their implications for model exploitation. By highlighting the nuanced relationship between fidelity and practical performance across multiple domains, our work paves the way for more comprehensive evaluations of model stealing attacks and their potential impact on MLaaS.
2
Background & Related work
We focus on classification tasks. Let X ∈ X ⊂ Rn denote an input, Y ∈ Y its label, and t : X → Y the target model from a model class H. 2
2.1
Multiplicity
Multiplicity can arise at several stages of the machine learning pipeline. At the dataset and preprocessing level, multiple plausible datasets or preprocessing choices may represent the same phenomenon due to noise, missingness, measurement error, or structural bias, leading to different sets of near-optimal predictors [Meyer et al., 2023]. At the training level, predictive multiplicity occurs when several near-optimal models achieve similar performance while disagreeing on individual predictions [Marx et al., 2020]. Related work on underspecification shows that different training runs can converge to models with similar in-distribution performance but different downstream behavior [D’Amour et al., 2022], while representation-level perturbations can also generate diverse families of near-optimal predictors [Eerlings et al., 2026]. At the explanation level, a single predictive model may admit multiple explanations that differ substantially in feature attributions or fairness narratives [Aïvodji et al., 2021, Hwang et al., 2026]. In standard learning pipelines, multiplicity reflects ambiguity in the data, preprocessing, or optimization process. In our setting, multiplicity is induced by the extraction process itself. Because query-based supervision provides only partial information about the target model, the stolen surrogate is identified only up to the constraints imposed by the queried examples. Model stealing, therefore, gives rise to an extraction-induced Rashomon Set: a family of surrogates that exhibit high label agreement with the target yet may diverge elsewhere in individual predictions, fairness, or robustness. 2.2
Characterization of multiplicity
We focus on three multiplicity metrics: ambiguity, discrepancy, and Rashomon Capacity. Ambiguity measures how often models in the Rashomon Set disagree on individual predictions; discrepancy measures the worst-case disagreement with a reference model; and Rashomon Capacity measures the dispersion of predictive scores across the set. Formal definitions are given in Section 3.1. 2.3
Model stealing attacks
In model stealing attacks [Tramèr et al., 2016], an adversary uses black-box access to a target model t to train a surrogate model s. The attacker first queries t on an auxiliary dataset, collects its outputs, and then trains s on these query-response pairs. Since the attack set depends on the query strategy, sampling distribution, and budget, model stealing can be viewed as a data acquisition pipeline whose outputs may induce different surrogate solution sets even for the same target model. Model stealing attacks are commonly divided into task-accuracy extraction, where the surrogate matches the target’s task performance, and fidelity extraction, where it reproduces the target’s decision behavior [Jagielski et al., 2020]. The fidelity of a surrogate model s with respect to a target model t on a reference set Xref is fidelity(s) =
X 1 I(s(x) = t(x)). |Xref | x∈Xref
2.4
Rashomon Set and Predictive Multiplicity
The Rashomon Effect refers to the existence of multiple models that explain the same data equally well [Breiman, 2001]. In machine learning, this motivates the study of the Rashomon Set, the set Sϵ (h0 ) of all models whose empirical risk lies within ϵ of that of a baseline model h0 [Marx et al., 2020]: Sϵ (h0 ) := {h ∈ H : R̂(h) ≤ R̂(h0 ) + ϵ}.
(1)
Predictive multiplicity exists when some h ∈ Sϵ (h0 ) disagrees with h0 on at least one input [Marx et al., 2020]. 2.5
Approximation of the Rashomon Set
Enumerating the full Rashomon Set is infeasible for complex model classes such as multilayer perceptrons. A practical alternative is to use Dropout at inference time to sample models likely to belong to the Rashomon Set [Hsu et al., 2024]. Compared with repeated retraining [Breiman, 2001] or adversarial weight perturbation [Hsu and Calmon, 2022], dropout-based sampling is much cheaper computationally and allows us to generate large sets of near-equivalent models efficiently. 3
2.6
Related work
To the best of our knowledge, this work is the first to study the fragility of fidelity in model reconstruction attacks through the lens of multiplicity. Prior work has shown that models within a Rashomon Set can differ in fairness or interpretability even when accuracy is similar [Black et al., 2022], but this question has not been examined in the context of model stealing. More broadly, the limits of fidelity have already been noted in interpretable machine learning, where high-fidelity explanations can still be misleading or strategically manipulated, for example, in fairwashing settings [Aïvodji et al., 2019, Lakkaraju and Bastani, 2020]. Our work extends this concern to model extraction by asking whether high-fidelity stolen models are genuinely behaviorally equivalent to their targets.
3
Evaluation metrics
We evaluate the implications of model stealing attacks using multiplicity and fairness. ACSIncome
(a) Knockoff Nets (Orekondy et al., 2019)
0
9
0.1
8
0.0
7
Dropout Rate
0.0
6
0.0
5
0.0
4
0.0
1 0.0
0
9
0.1
8
0.0
7
Dropout Rate
0.0
6
0.0
5
0.0
4
0.0
3
0.0
2
0.0
0.0
0.0
1
0.92
3
0.94
0.0
0.96
2
Fidelity
Fidelity
0.98
1.00 0.98 0.96 0.94 0.92 0.90 0.88 0.86 0.84
0.0
1.00
ACSMobility
0.0
ACSEmploy
(b) Prediction-API model stealing (Tramèr et al., 2016)
Figure 2: Fidelity for Rashomon Set of size 5000 for each dropout rate for both attacks on Folktables datasets.
3.1
Multiplicity
We quantify predictive multiplicity within the Rashomon Set using both label-based and score-based metrics. The notions of discrepancy and ambiguity are introduced to capture conflicting predictions within the Rashomon Set [Marx et al., 2020]. More precisely, given a Rashomon Set Sϵ (h0 ), the corresponding discrepancy δϵ (h0 ) and ambiguity αϵ (h0 ) are defined as follows: n
1X δϵ (h0 ) := max 1 [h (xi ) ̸= h0 (xi )] h∈Sϵ (h0 ) n i=1
(2)
n
αϵ (h0 ) :=
1X max 1 [h (xi ) ̸= h0 (xi )] n i=1 h∈Sϵ (h0 )
(3)
Discrepancy quantifies the worst-case disagreement rate between the reference model h0 and any other model within the Rashomon Set, while ambiguity captures the average pointwise disagreement between h0 and the Rashomon Set. On one hand, high ambiguity means that across many input instances, there is at least some disagreement between h0 and other models in the Rashomon set, whereas low ambiguity reflects broad consensus (i.e., most models tend to agree with h0 on most data points). On the other hand, a high discrepancy indicates the existence of at least one model whose predictions diverge significantly from those of h0 , suggesting potential extremes in model behavior. In contrast, a low discrepancy implies that all competing models make predictions that closely align with h0 . These complementary metrics characterize how impactful the choice of model can be for individuals. In high-stakes domains, this shows that even if a model performs well in terms of accuracy, other models may exist that perform equally well while producing substantially different outcomes for some people. This means that an attacker may achieve high-fidelity extraction without reproducing the actual decision rule of the target model. 4
Beyond label-based disagreement, we also evaluate score-level multiplicity through Rashomon Capacity, which measures how widely predictive probabilities can vary across models in the Rashomon set while maintaining near-optimal performance [Hsu and Calmon, 2022]. For a sample xi , the Rashomon Capacity is defined as mC (xi ) := 2C(Mϵ (xi )) , where ϵ ≥ 0 is the Rashomon parameter, Mϵ (xi ) denotes the ϵ-multiplicity set of xi (that is, the set of predictions for xi induced by models in the Rashomon Set), and C(Mϵ (xi )) is the capacity of that multiplicity set. Intuitively, larger values of Rashomon Capacity indicate greater dispersion of predictive probabilities across near-optimal models, even when these models exhibit similar overall performance. In our setting, this metric complements ambiguity and discrepancy by capturing variability in confidence scores rather than only variability in predicted labels. 3.2
Fairness
For fairness evaluation, we consider the case of binary classifiers (i.e., Y = {0, 1}) and focus on statistical notions of fairness. These metrics require a model to exhibit approximate parity for some statistical measure across the different demographic groups defined by a sensitive attribute G (e.g., age, sex). In particular, we assume binary sensitive attribute (i.e., G = {0, 1}) for simplicity and consider four metrics, namely statistical parity (∆SP ) [Dwork et al., 2012], predictive equality (∆PE ) [Corbett-Davies et al., 2017], equal opportunity (∆EOpp ) and equalized odds (∆EOdds ) [Hardt et al., 2016]. Table 1 presents a summary of the definition of these metrics. Table 1: Summary of the different statistical notions of fairness considered. Metric
Definition
∆SP ∆PE ∆EOpp
|P (Ŷ = 1|G = 0) − P (Ŷ = 1|G = 1)| |P (Ŷ = 1|Y = 0, G = 0) − P (Ŷ = 1|Y = 0, G = 1)| |P (Ŷ = 1|Y = 1, G = 0) − P (Ŷ = 1|Y = 1, G = 1)| |P (Ŷ = 1|Y = 1, G = 0) − P (Ŷ = 1|Y = 1, G = 1)|
∆EOdds
+|P (Ŷ = 1|Y = 0, G = 0) − P (Ŷ = 1|Y = 0, G = 1)|
Beyond global fidelity, fairness metrics show that a stolen model can match the victim’s outputs on average while redistributing errors across demographic groups in harmful ways. An attacker optimizing only fidelity may land on a clone whose decisions differ precisely in regions or subpopulations where margins are small. Multiplicity metrics reveal where disagreements are possible, but fairness metrics tell us who bears them.
4
Experiments Prediction-API model stealing
0.16
0.7
0.9
0.14
0.6
0.8
0.12
0.7
0.10
0.6 0.5
0.08 0.06
0.4
0.04
0.3
0.02
0.2
0.00
ACSEmploy
ACSIncome ACSMobility
Rashomon Capacity
1.0
Discrepancy
Ambiguity
Knockoff Nets
0.5 0.4 0.3 0.2 0.1
ACSEmploy
ACSIncome ACSMobility
0.0
ACSEmploy
ACSIncome ACSMobility
Figure 3: Multiplicity results for dropout-based Rashomon Set sets on Folktables datasets. Each box summarizes the metric across dropout rates p ∈ {0.01, 0.02, . . . , 0.10}, for each dropout rate, 5000 stochastic surrogate models are sampled. We evaluate whether high-fidelity model extraction is sufficient to guarantee behavioral equivalence between a victim model and its stolen surrogate. Across all experiments, we ask if dropout-based Rashomon Set around extracted surrogates reveals predictive multiplicity hidden by aggregate fidelity. 5
Knockoff Nets
0.175
0.10 0.08 0.06 0.04 0.02
0.20
0.150
Equalized Odds
Equal Opportunity
0.12
Statistical Parity
Base model
0.200
0.14
0.00
Prediction-API model stealing
0.125 0.100 0.075 0.050 0.025
ACSEmploy ACSIncome ACSMobility
0.000
ACSEmploy ACSIncome ACSMobility
0.15 0.10 0.05 0.00
ACSEmploy ACSIncome ACSMobility
Figure 4: Fairness metrics for Rashomon Set for both attacks. Each box summarizes the metric across dropout rates p ∈ {0.01, 0.02, . . . , 0.10}, for each dropout rate, 5000 stochastic surrogate models are sampled. (Predictive equality in Figure 8). We study this question across three settings: tabular prediction, medical image classification, and NLP sentiment classification. This allows us to test whether extraction-induced multiplicity is specific to a particular architecture or data modality, or whether this phenomenon generalizes across different prediction tasks. 4.1
Extraction pipeline
All datasets are divided into partitions of 60% training, 20% attack (used to generate the surrogate model), and 20% test data. For these experiments, we train one victim model and perform one extraction attack per dataset. The Rashomon Set is then generated by applying dropout to the extracted surrogate rather than by retraining multiple victim models. We use dropout values ranging from 0.01 to 0.1 in increments of 0.01, and sample 5,000 stochastic models for each dropout value, yielding 10 dropout-based Rashomon Set sets of size 5,000 per dataset. We consider two complementary black-box extraction attacks, which will be reused across all subsequent experiments. Knockoff Nets We use the Knockoff Nets extraction procedure [Orekondy et al., 2019], which steals model functionality through black-box interactions alone. We use ART’s (Adversarial Robustness Toolbox) [Nicolae et al., 2018] PyTorchClassifier interface for both networks and run Knockoff Nets with adaptive query sampling and a lossbased reward. To improve stability, the attack pool is balanced using the victim’s predicted classes before extraction. Prediction-API stealing We follow the standard black-box model stealing setting [Tramèr et al., 2016, Jagielski et al., 2020], in which an adversary queries the target model on an auxiliary dataset, collects the returned posterior outputs, and uses these outputs as supervision to train a stolen model that approximates the target model’s behavior. We instantiate this setting using the model extraction pipeline provided by the amuletml [Duddu et al., 2024] package. 4.2
Experiment 1: Extraction of neural networks for tabular data
Motivation We study whether predictive multiplicity occurs on tabular data when the surrogate is obtained via blackbox extraction rather than being trained directly from labeled data, and whether this phenomenon holds across distinct extraction frameworks. Datasets We conduct our experiments on three widely used benchmark datasets from Folktables [Ding et al., 2021] employed in the fair machine learning literature. All datasets are derived from the 1994 US Census survey. Those datasets are: ACSIncome (income superior to 50,000$), ACSMobility (home did not change since a year ago) and ACSEmploy (employed). To compute the fairness metrics, we define majority and minority groups using the race attribute RAC1P [Ding et al., 2021], which is included among the predictive features in all three tasks. We use White alone as the majority group and Black or African American alone as the marginalized group. Experimental setup For each dataset, both the victim and extracted surrogate are feed-forward neural networks with one hidden layer of 1,000 ReLU units. 6
OCTMNIST
(a) Knockoff Nets (Orekondy et al., 2019)
0
9
0.1
8
0.0
7
Dropout Rate
0.0
6
0.0
5
0.0
4
0.0
1 0.0
0
9
0.1
8
0.0
7
Dropout Rate
0.0
6
0.0
5
0.0
4
0.0
3
0.0
2
0.0
0.0
0.0
1
0.75
3
0.80
0.0
0.85
2
Fidelity
Fidelity
0.90
0.96 0.94 0.92 0.90 0.88 0.86 0.84 0.82 0.80
0.0
0.95
PneumoniaMNIST
0.0
BreastMNIST
(b) Prediction-API model stealing (Tramèr et al., 2016)
Figure 5: Fidelity for Rashomon Set of size 5000 for each dropout rate for both attacks on MedMNIST datasets. Results Figure 2 first shows that the extracted surrogates operate in a high-fidelity regime across the three tabular tasks. Under Knockoff Nets, fidelity remains above 0.91 for all datasets, with ACSMobility close to perfect agreement and ACSEmployment around 0.95. Under prediction-API stealing, fidelity is also high for ACSMobility and ACSEmployment, remaining above 0.94, while ACSIncome stays mostly above 0.85 despite a more visible decline as the dropout rate increases. These results indicate that the extracted surrogates retain strong aggregate agreement with the victim model across attacks and datasets. Figure 3 shows the distribution of ambiguity, discrepancy, and Rashomon capacity across all dropout-based Rashomon sets. We observe high ambiguity across all datasets, ranging from 20% to 100%. This means that between 20% and 100% of individuals receive conflicting predictions across models in the dropout-based Rashomon sets. For example, in the low-dropout regime (≤ 0.03), where fidelity is very high, ambiguity ranges from 0.28 to 0.65 for ACSEmploy, from 0.20 to 0.74 for ACSIncome, and from 0.40 to 1.00 for ACSMobility as highlighted by Figure 9. Figure 10, instead, shows that the discrepancy is more moderate in the low-dropout regime. For ACSIncome, we observe discrepancy values between 8% and 10%. This means that, even among high-fidelity surrogates, one can find a competing surrogate that assigns conflicting predictions to up to 10% of individuals. Discrepancy ranges from 0.038 to 0.05 for ACSEmploy and remains close to 0.0 for ACSMobility. Rashomon capacity is largest for ACSIncome under prediction-API stealing, increasing from 0.2204 to 0.6915 (see Figure 11). It remains lower for ACSMobility, suggesting that ACSIncome exhibits stronger score-level variation across near-equivalent stolen models. Overall, these results show that high-fidelity extraction does not identify a unique behavioral clone of the victim model: many individuals may receive different predictions depending on which near-equivalent surrogate is selected, even when all surrogates preserve similar aggregate fidelity. Figure 4 shows that this predictive multiplicity also affects group fairness estimates. Fairness remains relatively concentrated for ACSMobility, but varies more strongly for ACSIncome and ACSEmploy, including shifts in equal opportunity and statistical parity under prediction-API stealing. 4.3
Experiment 2: Extraction of neural networks for medical imaging and NLP
Motivation We next examine whether the predictive multiplicity observed after extracting tabular models also appears in high-dimensional domains. Medical imaging and NLP provide two complementary settings: the former uses deep convolutional models over visual inputs, while the latter uses transformer-based language models over text. Together, these experiments test whether dropout-based Rashomon Set remain informative after black-box extraction beyond tabular neural networks. Datasets For medical imaging, we use three datasets from MedMNIST [Yang et al., 2023]: BreastMNIST (2 classes), OCTMNIST (4 classes), and PneumoniaMNIST (2 classes), covering breast ultrasound, optical coherence tomography, and chest X-ray classification. For NLP, we use FST, a binary financial sentiment classification task derived from financial phrase-bank [Malo et al., 2014]. Experimental setup For the medical imaging experiments, both the victim and extracted surrogate are ResNet-50 models [He et al., 2016] initialized with ImageNet-pretrained weights. For the NLP experiment, both the victim and surrogate are BERT sequence classifiers [Devlin et al., 2018]. Because the NLP experiments are more computationally 7
Knockoff Nets
0.25
0.8
Discrepancy
0.6 0.5
Rashomon Capacity
0.20
0.7
Ambiguity
Prediction-API model stealing
0.15 0.10
0.4 0.3
0.8 0.6 0.4 0.2
0.05 BreastMNIST OCTMNIST PneumoniaMNIST
BreastMNIST OCTMNIST PneumoniaMNIST
BreastMNIST OCTMNIST PneumoniaMNIST
Figure 6: Multiplicity results for Rashomon Set for both attacks on MedMNIST datasets. Each box summarizes the metric across dropout rates p ∈ {0.01, 0.02, . . . , 0.10}, for each dropout rate, 5000 stochastic surrogate models are sampled. expensive, we use the same dropout grid from 0.01 to 0.10 but sample 500 epsilon-filtered stochastic models per dropout value. Dropout is applied throughout the extracted BERT surrogate by perturbing all linear layers. Medical imaging results Figure 5 highlights that PneumoniaMNIST is the clearest high-fidelity case: both attacks remain close to 0.96 fidelity across the entire dropout range, not only for small dropout values. The extracted surrogate appears highly faithful under the usual agreement metric. Nevertheless, Figure 6 shows that ambiguity rises from 0.3029 to 0.7201 under Knockoff Nets and from 0.3294 to 0.7372 under prediction-API stealing, while discrepancy remains low, below 0.0435 (see Figure 13). Under prediction-API stealing for OCTMNIST, fidelity is high at low dropout, ranging from 0.9267 to 0.9162 for dropout values up to 0.03. Yet, the corresponding Rashomon Set already presents high ambiguity (0.7259–0.7891) and substantial Rashomon Capacity (0.3713–0.4820) in the low-dropout regime as shown on Figures 12–14. This indicates that the sampled surrogates do not merely flip a small number of isolated labels: their predictive scores vary substantially across the Rashomon Set, so the extracted model leaves a broad local region of plausible probability assignments. Finally, for BreastMNIST, fidelity is around 0.81 for both attacks, yet ambiguity still reaches above 0.80 by dropout 0.03. Overall, the image experiments show that high fidelity, when it is achieved, can coexist with substantial local multiplicity, and that different attacks can produce different Rashomon Set. NLP results Figure 7 shows that the extracted BERT surrogates remain highly faithful to the victim on FST. Under Knockoff Nets, fidelity stays between 0.9308 and 0.9233 across the dropout range, while prediction-API stealing remains slightly higher, between 0.9448 and 0.9307. Thus, unlike settings where multiplicity could be attributed to poor extraction, the NLP experiment remains firmly in a high-fidelity regime for both attacks. The multiplicity metrics reveal a different behavior: ambiguity grows rapidly even while fidelity changes only mildly. In the low-dropout regime (≤ 0.03), Figure 15 shows that ambiguity already increases from 0.1421 to 0.3223 under Knockoff Nets and from 0.1523 to 0.3223 under prediction-API stealing. Rashomon Capacity follows the same gradual augmentation. In the low-dropout regime, it rises from 0.0986 to 0.2037 under Knockoff Nets and from 0.0758 to 0.1436 under prediction-API stealing (Figure 15). At higher dropout values, this effect becomes much stronger, with ambiguity reaching 0.8680 for Knockoff Nets and 0.7716 for predictionAPI stealing. Overall, the NLP experiment shows that even a consistently high-fidelity extracted language model can contain a progressively expanding Rashomon Set of alternative predictions.
5
Discussion
Across tabular data, medical imaging, and NLP, our experiments show that the main risk in model extraction is not captured by fidelity alone. A stolen model can agree with the victim on aggregate while still belonging to a Rashomon Set set with substantial predictive variation. This is visible even in high-fidelity regimes: PneumoniaMNIST and FST preserve strong agreement with the victim, yet still exhibit large ambiguity under dropout sampling. Thus, extraction 8
Knockoff Nets Prediction-API model stealing
0.095
0.8
0.090 0.085
0.6
Discrepancy
0.935
0.5 0.4
0.930
6
7
8
9
0
0.0
0.0
0.0
0.1
4 0.0
5
3 0.0
0.0
2 0.0
0.0
1 0.0
0.925
Dropout Rate
0.5
0.4
0.7
Ambiguity
Fidelity
0.940
0.9
0.080 0.075
0.3
0.070
0.2
0.065 ets koff N
Knoc
PI
tion-A
Predic
Rashomon Capacity
0.945
0.3
0.2
0.1 ets koff N
Knoc
PI tion-A
Predic
ets koff N
Knoc
I
P tion-A
Predic
Figure 7: Results for Rashomon Set for both attacks on FST dataset. Each box summarizes the metric across dropout rates p ∈ {0.01, 0.02, . . . , 0.10}, for each dropout rate, 500 stochastic surrogate models are sampled.
success should not be interpreted as behavioral equivalence. For downstream use, auditing, or attack transfer, a surrogate that appears faithful globally may still differ from the victim on specific inputs. The structure of this multiplicity depends on both the dataset and the extraction pipeline. Knockoff Nets and predictionAPI stealing do not merely produce different fidelity values; they can induce different local neighborhoods around the extracted surrogate. In some cases, such as OCTMNIST, lower fidelity is accompanied by much larger Rashomon Capacity, while in others, high-fidelity extraction still leaves substantial ambiguity. This suggests that two attacks with similar aggregate performance may yield surrogate families that disagree in different ways and on different examples. The tabular experiments further show that predictive multiplicity can affect fairness performances. Even when a surrogate appears to approximate the victim well, the fairness profile of models sampled around that surrogate can vary, especially for ACSIncome and ACSMobility. This suggests that claims of faithful copying should be evaluated not only through aggregate fidelity, but also through whether the surrogate preserves the victim’s group-level performance patterns across populations. This work has several limitations. First, our Rashomon Set sets are approximated through dropout-based sampling rather than exhaustive enumeration, so the reported variability captures a local neighborhood around the extracted surrogate. However, because our approximate Rashomon Set sets already reveal substantial multiplicity, they are sufficient to demonstrate a concrete risk; the approximation may underestimate the full space of near-equivalent surrogates, but the observed variability is already large enough to challenge claims of faithful copying. Second, each domain uses a fixed architecture family: neural networks for tabular data, ResNet-50 for medical imaging, and BERT for NLP. This corresponds to a strong-adversary setting in which the surrogate architecture is well aligned with the victim architecture. Relaxing this assumption by considering heterogeneous victim and surrogate architectures would likely enlarge the space of near-equivalent surrogates and may induce even greater multiplicity. Such experiments would also provide a broader picture of the extent to which supposedly faithful copies can diverge in deploymentrelevant performance across populations. Overall, our results should therefore be viewed as a conservative estimate of extraction-induced multiplicity. Even under favorable conditions for producing a faithful copy, high fidelity does not eliminate substantial variation in downstream behavior. Finally, while we focus on predictive multiplicity and tabular fairness, other behavioral notions remain open, including explanation agreement, robustness, calibration, and adversarial transferability.
6
Conclusion
In this paper, we revisited the standard interpretation of model stealing success by showing that high fidelity does not, by itself, imply that a stolen model is practically interchangeable with its victim. By studying extraction through the lens of the RASHOMON effect, we showed that query-based supervision can leave the surrogate underdetermined, yielding families of near-equivalent extracted models rather than a unique faithful clone. Across Knockoff Nets and prediction-API model stealing, and across tabular, medical imaging, and NLP tasks, extracted surrogates exhibited substantial multiplicity even when fidelity remained high. 9
Our results show that this multiplicity appears at several levels. At the prediction level, extracted Rashomon Set can contain models with large ambiguity, discrepancy, and Rashomon Capacity. At the fairness level, tabular surrogates with similar fidelity can induce different group-level outcomes. These findings suggest that fidelity should be viewed as only one dimension of extraction quality, rather than as a sufficient proxy for functional equivalence. The practical consequences of model stealing depend not only on how closely a surrogate matches the victim on average, but also on the structure of the extraction-induced Rashomon Set surrounding it. This has direct implications for attack evaluation and defense design in MLaaS settings: assessing model extraction risk requires measuring predictive multiplicity, and when relevant fairness variability, alongside conventional agreement-based metrics. More broadly, our work places model stealing within the wider literature on multiplicity by showing that extraction itself constitutes a source of non-uniqueness at deployment time.
Acknowledgements The authors thank the Digital Research Alliance of Canada for computing resources. Ulrich Aïvodji is supported by the Fonds de recherche du Québec – Nature et technologies (FRQNT) Research Support for New Academic grant (342675).
References Ulrich Aïvodji, Hiromi Arai, Olivier Fortineau, Sébastien Gambs, Satoshi Hara, and Alain Tapp. Fairwashing: the risk of rationalization. In International Conference on Machine Learning, pages 161–170. PMLR, 2019. Ulrich Aïvodji, Hiromi Arai, Sébastien Gambs, and Satoshi Hara. Characterizing the risk of fairwashing. Advances in Neural Information Processing Systems, 34:14822–14834, 2021. Giuseppe Ateniese, Luigi V Mancini, Angelo Spognardi, Antonio Villani, Domenico Vitali, and Giovanni Felici. Hacking smart machines with smarter ones: How to extract meaningful data from machine learning classifiers. International Journal of Security and Networks, 10(3):137–150, 2015. Emily Black, Manish Raghavan, and Solon Barocas. Model Multiplicity: Opportunities, Concerns, and Solutions. In 2022 ACM Conference on Fairness Accountability and Transparency, pages 850–863. ACM, 2022. ISBN 978-14503-9352-2. doi: 10.1145/3531146.3533149. URL https://dl.acm.org/doi/10.1145/3531146.3533149. Leo Breiman. Statistical modeling: The two cultures (with comments and a rejoinder by the author). Statistical science, 16(3):199–231, 2001. Nicholas Carlini, Chang Liu, Úlfar Erlingsson, Jernej Kos, and Dawn Song. The secret sharer: Evaluating and testing unintended memorization in neural networks. In 28th USENIX Security Symposium (USENIX Security 19), pages 267–284, 2019. Sam Corbett-Davies, Emma Pierson, Avi Feller, Sharad Goel, and Aziz Huq. Algorithmic decision making and the cost of fairness. In Proceedings of the 23rd acm sigkdd international conference on knowledge discovery and data mining, pages 797–806, 2017. Alexander D’Amour, Katherine Heller, Dan Moldovan, Ben Adlam, Babak Alipanahi, Alex Beutel, Christina Chen, Jonathan Deaton, Jacob Eisenstein, Matthew D Hoffman, et al. Underspecification presents challenges for credibility in modern machine learning. Journal of Machine Learning Research, 23(226):1–61, 2022. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: pre-training of deep bidirectional transformers for language understanding. CoRR, abs/1810.04805, 2018. URL http://arxiv.org/abs/1810.04805. Frances Ding, Moritz Hardt, John Miller, and Ludwig Schmidt. Retiring adult: New datasets for fair machine learning. Advances In Neural Information Processing Systems, 34:6478–6490, 2021. Vasisht Duddu, Sebastian Szyller, and N. Asokan. SoK: Unintended Interactions among Machine Learning Defenses and Risks, April 2024. Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel. Fairness through awareness. In Proceedings of the 3rd innovations in theoretical computer science conference, pages 214–226, 2012. Gijs Eerlings, Bram Zoomers, Jeroen Liesenborgs, Guillermo Rovelo Ruiz, and Kristof Luyten. Diverse: Disagreement-inducing vector evolution for rashomon set exploration. In International Conference on Learning Representations, 2026. Matthew Fredrikson, Eric Lantz, Somesh Jha, Simon Lin, David Page, and Thomas Ristenpart. Privacy in pharmacogenetics: An {End-to-End} case study of personalized warfarin dosing. In 23rd USENIX security symposium (USENIX Security 14), pages 17–32, 2014. 10
Moritz Hardt, Eric Price, and Nati Srebro. Equality of opportunity in supervised learning. Advances in neural information processing systems, 29, 2016. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016. Hsiang Hsu and Flavio Calmon. Rashomon capacity: A metric for predictive multiplicity in classification. Advances in Neural Information Processing Systems, 35:28988–29000, 2022. Hsiang Hsu, Guihong Li, Shaohan Hu, and Chun-Fu (Richard) Chen. Dropout-based rashomon set exploration for efficient predictive multiplicity estimation. International Conference on Learning Representations, 2024. Hanjin Hwang, Seonghyeon Lee, Lucas Rosenblatt, Steven Euijong Whang, and Julia Stoyanovich. Explanation multiplicity in shap: Characterization and assessment. arXiv preprint arXiv:2601.12654, 2026. Matthew Jagielski, Nicholas Carlini, David Berthelot, Alex Kurakin, and Nicolas Papernot. High accuracy and high fidelity extraction of neural networks. In 29th USENIX security symposium (USENIX Security 20), pages 1345– 1362, 2020. Ram Shankar Siva Kumar, Magnus Nyström, John Lambert, Andrew Marshall, Mario Goertzel, Andi Comissoneru, Matt Swann, and Sharon Xia. Adversarial machine learning-industry perspectives. In 2020 IEEE security and privacy workshops (SPW), pages 69–75. IEEE, 2020. Himabindu Lakkaraju and Osbert Bastani. " how do i fool you?" manipulating user trust via misleading black box explanations. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society, pages 79–85, 2020. Pekka Malo, Ankur Sinha, Pekka Korhonen, Jyrki Wallenius, and Pyry Takala. Good debt or bad debt: Detecting semantic orientations in economic texts. Journal of the Association for Information Science and Technology, 65(4):782–796, 2014. ISSN 2330-1635, 2330-1643. doi: 10.1002/asi.23062. URL https://asistdl. onlinelibrary.wiley.com/doi/10.1002/asi.23062. Charles Marx, Flavio Calmon, and Berk Ustun. Predictive multiplicity in classification. In International Conference on Machine Learning, pages 6765–6774. PMLR, 2020. Anna P. Meyer, Aws Albarghouthi, and Loris D’Antoni. The dataset multiplicity problem: How unreliable data impacts predictions. In Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency, pages 193–204, 2023. Maria-Irina Nicolae, Mathieu Sinn, Minh Ngoc Tran, Beat Buesser, Ambrish Rawat, Martin Wistuba, Valentina Zantedeschi, Nathalie Baracaldo, Bryant Chen, Heiko Ludwig, Ian Molloy, and Ben Edwards. Adversarial robustness toolbox v1.2.0. CoRR, 1807.01069, 2018. URL https://arxiv.org/pdf/1807.01069. Tribhuvanesh Orekondy, Bernt Schiele, and Mario Fritz. Knockoff nets: Stealing functionality of black-box models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4954–4963, 2019. Lesia Semenova, Cynthia Rudin, and Ronald Parr. On the existence of simpler machine learning models. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, pages 1827–1858, 2022. Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. Membership inference attacks against machine learning models. In 2017 IEEE symposium on security and privacy (SP), pages 3–18. IEEE, 2017. Florian Tramèr, Fan Zhang, Ari Juels, Michael K Reiter, and Thomas Ristenpart. Stealing machine learning models via prediction {APIs}. In 25th USENIX security symposium (USENIX Security 16), pages 601–618, 2016. Jiancheng Yang, Rui Shi, Donglai Wei, Zequan Liu, Lin Zhao, Bilian Ke, Hanspeter Pfister, and Bingbing Ni. MedMNIST v2 – A large-scale lightweight benchmark for 2D and 3D biomedical image classification. Scientific Data, 10 (1):41, January 2023. ISSN 2052-4463. doi: 10.1038/s41597-022-01721-8.
11
Appendix
This appendix provides complementary plots that decompose the aggregate results reported in Section 4.
Additional fairness results. Figure 8 reports predictive equality for the tabular experiments. It complements the fairness results in Figure 4 by showing how another group fairness metric varies across the dropout-based Rashomon Set.
0.09
Knockoff Nets Prediction-API model stealing Base model
0.08
Predictive Parity
0.07 0.06 0.05 0.04 0.03 0.02 ACSEmploy
ACSIncome
ACSMobility
Dataset
Figure 8: Predictive Equality for Rashomon Set of size 5000 for both attacks.
Tabular multiplicity results. Figures 9, 10, and 11 provide the individual ambiguity, discrepancy, and Rashomon Capacity results for the Folktables datasets. These plots expand the summary shown in Figure 3 and show how each metric evolves across dropout rates and extraction attacks.
ACSIncome
1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2
ACSMobility
1.0
Ambiguity
0.9 0.8 0.7 0.6 0.5
(a) Knockoff Nets (Orekondy et al., 2019)
0
9
0.1
8
0.0
7
Dropout Rate
0.0
6
0.0
5
0.0
4
0.0
3
0.0
2
0.0
1
0.0
0.0
0
9
0.1
8
0.0
7
Dropout Rate
0.0
6
0.0
5
0.0
4
0.0
3
0.0
0.0
0.0
0.0
2
0.4
1
Ambiguity
ACSEmploy
(b) Prediction-API model stealing (Tramèr et al., 2016)
Figure 9: Ambiguity for Rashomon Set of size 5000 for each dropout rate for both attacks on Folktables datasets. 12
ACSIncome
0.02
0.0 3 0.0 4
Dropout Rate
0.0 1 0.0 2
0.0 9 0.1 0
0.0 7 0.0 8
0.0 5 0.0 6
0.0 3 0.0 4
0.0 1 0.0 2
0.00
(a) Knockoff Nets (Orekondy et al., 2019)
0.0 9 0.1 0
0.04
0.16 0.14 0.12 0.10 0.08 0.06 0.04 0.02 0.00
0.0 7 0.0 8
0.06
Discrepancy
Discrepancy
0.08
ACSMobility
0.0 5 0.0 6
ACSEmploy
Dropout Rate
(b) Prediction-API model stealing (Tramèr et al., 2016)
Figure 10: Discrepancy for Rashomon Set of size 5000 for each dropout rate for both attacks on Folktables datasets.
ACSEmploy
ACSIncome
ACSMobility
0.8
0.3
Rashomon Capacity
0.2 0.1 0.0
0.6 0.4 0.2
(a) Knockoff Nets (Orekondy et al., 2019)
0
0.1
9
8
0.0
Dropout Rate
0.0
7
6
0.0
0.0
5
4
0.0
3
0.0
2
0.0
1 0.0
0
9
0.1
8
0.0
7
Dropout Rate
0.0
6
0.0
5
0.0
4
0.0
3
0.0
2
0.0
0.0
0.0
1
0.0
0.0
Rashomon Capacity
0.4
(b) Prediction-API model stealing (Tramèr et al., 2016)
Figure 11: Rashomon Capacity for Rashomon Set of size 5000 for each dropout rate for both attacks on Folktables datasets.
Medical imaging multiplicity results. Figures 12, 13, and 14 report the corresponding multiplicity metrics for the MedMNIST experiments. They provide a more detailed view of the image-domain results summarized in Figure 6. OCTMNIST
PneumoniaMNIST
0.8
0.8
0.7
0.7
Ambiguity
Ambiguity
BreastMNIST
0.6 0.5 0.4
0.6 0.5 0.4
(a) Knockoff Nets (Orekondy et al., 2019)
0
0.1
9
8
0.0
Dropout Rate
0.0
7
6
0.0
0.0
5
4
0.0
0.0
3
2
0.0
1
0.0
0.0
0
9
0.1
8
0.0
7
Dropout Rate
0.0
6
0.0
5
0.0
4
0.0
0.0
3
2
0.0
0.0
0.0
1
0.3
(b) Prediction-API model stealing (Tramèr et al., 2016)
Figure 12: Ambiguity for Rashomon Set of size 5000 for each dropout rate for both attacks on MedMNIST datasets. 13
BreastMNIST
OCTMNIST
0.25
0.225 0.200
Discrepancy
0.20 0.15 0.10
0.175 0.150 0.125 0.100 0.075
0
8
9
0.1
Dropout Rate
0.0
7
0.0
6
0.0
5
0.0
4
0.0
3
0.0
0.0
1 0.0
0
9
0.1
8
0.0
7
0.0
6
0.0
5
0.0
4
0.0
3
0.0
2
0.0
1
0.0
0.0
Dropout Rate
(a) Knockoff Nets (Orekondy et al., 2019)
2
0.050
0.05
0.0
Discrepancy
PneumoniaMNIST
(b) Prediction-API model stealing (Tramèr et al., 2016)
Figure 13: Discrepancy for Rashomon Set of size 5000 for each dropout rate for both attacks on MedMNIST datasets. OCTMNIST
PneumoniaMNIST
1.0
0.6
0.8
0.5
Rashomon Capacity
0.6 0.4 0.2 0.0
0.4 0.3 0.2 0.1
(a) Knockoff Nets (Orekondy et al., 2019)
0
9
0.1
0.0
8
7
Dropout Rate
0.0
0.0
6
0.0
5
0.0
4
0.0
3
0.0
2
1 0.0
0
9
0.1
0.0
8
7
Dropout Rate
0.0
0.0
6
5
0.0
0.0
4
0.0
3
0.0
2
0.0
0.0
1
0.0
0.0
Rashomon Capacity
BreastMNIST
(b) Prediction-API model stealing (Tramèr et al., 2016)
Figure 14: Rashomon Capacity for Rashomon Set of size 5000 for each dropout rate for both attacks on MedMNIST datasets. NLP multiplicity results. Figure 15 reports ambiguity, discrepancy, and Rashomon Capacity for the FST sentiment classification experiment. These results complement Figure 7 by showing the individual multiplicity trends behind the aggregate NLP results. Knockoff Nets
Prediction-API model stealing
FST 0.095
0.8
0.075 0.070
0.2
0
9
0.1
8
0.0
7
Dropout Rate
0.0
6
0.0
0.0
5
4
0.0
0.0
3
0.0
2
0
0.1
9
8
0.0
Dropout Rate
0.0
7
6
0.0
0.0
5
0.0
4
3
0.0
2
0.0
0.0
1
0.0
0.0
0
9
0.1
8
0.0
Dropout Rate
0.0
7
6
0.0
5
0.0
4
0.0
3
0.0
2
0.0
0.0
0.0
1
0.065
0.4
0.0
0.2
0.080
0.6
1
0.4
0.085
0.0
0.6
Rashomon Capacity
0.090
Discrepancy
Ambiguity
0.8
Figure 15: Ambiguity, Discrepancy and Rashomon Capacity for Rashomon Set of size 500 for each dropout rate for both attacks on FST dataset.
14