arXiv:2605.22746v1 [cs.LG] 21 May 2026
Plug-in Losses for Evidential Deep Learning: A Simplified Framework for Uncertainty Estimation that Includes the Softmax Classifier Berk Hayta TU Munich [email protected] Simon Mittermaier Infineon Technologies
Hannah Laus TU Munich & MCML
Felix Krahmer TU Darmstadt, TU Munich & MCML
Abstract Real-world sensor-based learning systems require uncertainty estimation that is both reliable and computationally efficient. Evidential Deep Learning (EDL) provides single-pass uncertainty estimation by modeling the class probabilities via Dirichlet distributions, where the Dirichlet parameters are predicted by a learned neural network mapping. However, this approach can lead to computational challenges, as Dirichlet expected objectives are more complex than standard supervised learning losses, complicating their analysis and implementation. We address this issue by approximating the objective of the first-order empirical risk minimization problem induced by EDL with a plug-in loss evaluated at the Dirichlet mean and show that, under mild assumptions, the approximation error decays with growing evidence for a broad class of loss functions, including mean-squared error and cross-entropy loss. As a special case, our analysis provides justification for the use of softmax in the context of uncertainty estimation, since under a particular evidence-to-Dirichlet mapping, our framework includes the standard softmax classifier. We validate the proposed simplified objectives on the Google Speech Commands dataset and show that they achieve predictive accuracy and selective prediction performance comparable to classical EDL, while being simpler to implement using standard deep learning losses and training pipelines. To the best of our knowledge, this empirical analysis is the first to obtain coverage-accuracy trade-offs for speech recognition tasks through EDL.
1
Introduction
Deep neural networks are increasingly deployed in resource-constrained and safety-critical settings, where predictive accuracy alone is often insufficient. Overconfident predictions may lead to undesirable or unsafe system behavior, motivating methods that quantify predictive uncertainty and enable selective prediction. At the same time, many established approaches for uncertainty estimation come with significant computational overhead, e.g., Monte Carlo dropout (Gal and Ghahramani [2016]), Bayesian neural networks (Neal [2012], Blundell et al. [2015]) and Deep ensembles (Lakshminarayanan et al. [2017], Wilson and Izmailov [2020]). This can make them unsuitable for real-time applications such as embedded sensing, where uncertainty estimation methods should add only minimal inference-time overhead; moreover, many of these methods require specific training procedures, making them less straightforward to integrate into existing pipelines. Evidential Deep Learning (EDL) (Sensoy et al. [2018]) aims to address these issues by enabling uncertainty estimation in a single forward pass while remaining compatible with standard neural Preprint.
network training. The idea of EDL is not to learn only a single prediction, but rather to predict parameters of a Dirichlet distribution modeling the class probabilities. Despite promising empirical results for controlled benchmark settings (Sensoy et al. [2018], Deng et al. [2023]), there remains a gap between the theoretical foundations of this uncertainty-aware learning strategy and the algorithmic implementation. In particular, the theory-based loss function of EDL is based on an explicit formula for the expected loss over the unknown Dirichlet distribution to be estimated. Compared with ordinary supervised losses, the resulting objectives are more complex because they depend on both the projected class probabilities and the total Dirichlet concentration. This additional structure can make the optimization landscape harder to interpret and may contribute to sensitivity in hyperparameter choices, slow convergence, or convergence to poor local minima. This motivates a closer analysis of the empirical-risk structure induced by EDL and the plug-in simplifications developed below. In this work, we propose a remedy for these concerns by working with an approximation to the EDL loss function that is closer to the loss without EDL, while also retaining the uncertainty information provided by its minimizer. Thus, the resulting uncertainty-aware learning problem can be optimized using simpler supervisedlearning objectives with efficient implementations and well-understood training behavior. To illustrate this, we validate the resulting simplified objectives on the Google Speech Commands v1 dataset (Warden [2018]). As we demonstrate in theoretical derivations and numerical studies, the resulting uncertainty-aware learning methods exhibit comparable operational metrics to EDL. More precisely, we proceed via a Taylor expansion of the Dirichlet expected loss around the Dirichlet mean. The first-order term vanishes in expectation, and higher terms decay with growing evidence, which establishes that evaluating the loss at the Dirichlet mean yields a good approximation of the EDL objective. As a special case, the softmax classifier belongs to this class of simplified evidential classifiers. For the Google Speech Commands v1 dataset, we estimate uncertainty distributions for correct and incorrect predictions and evaluate the resulting coverage–accuracy trade-off. By thresholding an uncertainty score, the classifier can withhold predictions for uncertain samples, increasing accuracy on the accepted samples at the cost of reduced coverage. Both EDL and the proposed approximate EDL objectives, including the softmax case, provide such uncertainty scores. Contributions.
The contributions of this work are as follows:
• In this paper, we derive that the evidential deep learning framework (EDL) can be simplified by working with plug-in losses that approximate the EDL objective proposed in Sensoy et al. [2018]. • We rigorously show that, for growing evidence, the EDL objective is well approximated by the proposed plug-in loss and quantify the approximation error. • The resulting classification framework includes a variant of the classical softmax classifier, hence our analysis provides a first step towards understanding the performance of softmax for uncertainty estimation in the EDL framework. • We apply the resulting uncertainty estimation framework for a realistic keyword spotting task on Google Speech Commands v1 and find that it performs comparably to the original EDL formulation, despite its simpler implementability. • We illustrate the benefits of the resulting uncertainty estimates by evaluating operational uncertainty through fixed high-reliability operating points, coverage–accuracy trade-offs, and uncertainty distributions for correct and incorrect predictions. To the best of our knowledge, this is the first uncertainty analysis in the spirit of EDL in the speech recognition context.
2
Related work
Evidential Deep Learning Evidential deep learning was introduced by Sensoy et al. [2018] for classification and is inspired by Dempster-Shafer theory (Dempster [1968], Shafer [1990, 1976]) and 2
subjective logic (Jøsang [2001], Jsang [2018]). In recent years many refinements and extensions of EDL have been proposed (Ulmer et al. [2023], Gao et al. [2025]). For example, R-EDL (Chen et al. [2024]) replaces the fixed prior-weight choice in the subjective-logic parametrization of classical EDL with a tunable hyperparameter, and separately simplifies the original EDL mean-squared-error objective by removing its variance term. I-EDL (Deng et al. [2023]) adds an extra Fisher regularization term to the loss. Re-EDL (Chen et al. [2025]) and Shen et al. [2024] both argue that for improved out-of-distribution detection one can leave out the KL-divergence term. Besides advancements in EDL, there is also a line of work criticizing EDL (Bengs et al. [2022], Jürgens et al. [2024], Shen et al. [2024]). Jürgens et al. [2024] shows that EDL does not model epistemic uncertainty in the formal Bayesian sense, since its uncertainty does not necessarily vanish with infinite training data and is highly dependent on hyperparameter choices. On the other hand, Shen et al. [2024] shows that EDL can perform well for out-of-distribution detection, even if it should not be interpreted as Bayesian uncertainty quantification. These observations motivate a direct study of the Dirichlet-expected losses used in classification EDL. In particular, our work analyzes the relation between losses evaluated under the Dirichlet expectation and losses evaluated at the Dirichlet mean. In a parallel line of work, Malinin and Gales [2018] introduced Prior Networks which also use Dirichlet-output classifiers. The method of Malinin and Gales [2018] requires OOD samples for training which is not available in many applications but other works in that line also get away without OOD samples (Tsiligkaridis [2021], Haussmann et al. [2020]). Furthermore, there are works on EDL for regression (Amini et al. [2020], Malinin et al. [2020]). In this work, we focus on classification EDL as introduced in Sensoy et al. [2018], while noting that related Dirichlet-output classifiers have also been studied in the Prior and Posterior Network literature (Ulmer et al. [2023]). Softmax confidence and deterministic uncertainty Holm et al. [2023] analyze the empirical uncertainty-estimation properties of softmax confidence compared to MC dropout for text classification and find that it performs competitively. In contrast, Ovadia et al. [2019] presents a large-scale empirical study showing that deterministic confidence measures, such as softmax, fail on certain tasks under distribution shifts and can display misleading calibration behavior. Speech command recognition Speech Command Recognition, often also called Keyword Spotting or Wake-Word Detection, is a subtask within the field of Speech Recognition with a focus on detecting certain words or phrases. The most common benchmark for this task is derived from the Google Speech Commands v1 dataset, described in Warden [2018]. In practice, this task is often performed with limited compute and power resources, often in battery-powered settings. Majumdar and Ginsburg [2020] presents a model architecture optimized not only for prediction accuracy but also for efficient inference. We use this model as the basis for our experiments and, to the best of our knowledge, are the first to showcase evidential deep learning in this realistic and impactful application.
3
Preliminaries
3.1
Problem setup
We consider a supervised multiclass classification problem with K classes. Let X ⊆ Rd denote the input space and let Y = {1, . . . , K} denote the label space. Data are drawn from an unknown distribution PXY over X × Y, and we observe an i.i.d. sample {(xi , yi )}ni=1 . Let fθ : X → RK be a parametric model with parameters θ ∈ Θ, and let z = fθ (x) ∈ RK denote the output logits. Predictions are obtained by composing fθ with a mapping p̂ : RK → ∆K−1 into the probability simplex. Throughout, class labels may be identified with their one-hot representations when convenient. Given a loss function ℓ : ∆K−1 × Y → R+ , the population and empirical risks are defined as R(θ) = E(X,Y )∼PXY ℓ(p̂(fθ (X)), Y ) ,
3
n X b n (θ) = 1 R ℓ(p̂(fθ (xi )), yi ). n i=1
(1)
b n (θ), The goal of a learning problem is to minimize the empirical risk, i.e., θ ⋆ ∈ arg minθ∈Θ R which means that model predictions are learned by minimizing a loss over the training data. This is called empirical risk minimization ([Mohri et al., 2018, Chapter 4.2]). 3.2
Evidential Deep Learning
We consider the Evidential Deep Learning (EDL) framework for classification introduced by Sensoy et al. [2018]. Throughout, we refer to the original formulation as classical EDL to distinguish it from the simplified formulations introduced later. For a loss ℓ : ∆K−1 × Y → R+ , the classical Dirichlet-expected EDL objective is h i E(X,Y )∼PXY Eπ∼Dir(αθ (X)) ℓ(π, Y ) .
(2)
The neural network enters this objective through the Dirichlet-parameter map x 7→ αθ (x). Concretely, it outputs logits z = fθ (x) ∈ RK , which are mapped to componentwise evidence values e = τ (z), where τ is typically monotone increasing (e.g., exponential or softplus function). The evidence is then mapped to Dirichlet parameters α = ϕ(e). The classical choice is αi = ei + 1, while we allow K componentwise maps ϕ : RK ≥0 → R>0 . Predictions are obtained from the Dirichlet mean p̂(z) = Π(α(z)) =
α(z) , α0 (z)
α0 (z) =
K X
αj (z).
(3)
j=1
The hard decision satisfies ŷ(z) = arg maxi p̂i (z) = arg maxi αi (z) = arg maxi ei (z). Since classical EDL is usually specified through the training objective, we separate out the underlying logits-to-Dirichlet-parameter map and call this map the classical evidential classifier. Definition 1 (Classical evidential classifier). Using the mappings above, a classical evidential classifier is the logits-to-Dirichlet-parameter map z 7→ α(z) = ϕ(τ (z)).
(4)
We use two classical EDL losses from Sensoy et al. [2018]. Let y ∈ {0, 1}K denote the one-hot encoded target label. The cross-entropy objective is written as the Dirichlet expectation " K # K X X EDL LCE (α, y) = Eπ∼Dir(α) − yk log πk = yk ψ (0) (α0 ) − ψ (0) (αk ) , (5) k=1
k=1
where ψ
(0)
denotes the digamma function. The evidential mean-squared-error loss is LEDL MSE (α, y) =
K X
2
(yk − p̂k ) +
k=1
K X p̂k (1 − p̂k ) k=1
α0 + 1
.
(6)
Classical EDL is commonly trained with an additional KL regularizer that penalizes evidence assigned to incorrect classes. Following Sensoy et al. [2018], the regularized loss is ℓreg EDL (α, y) = ℓEDL (α, y) + λt KL(Dir(α̃) ∥ Dir(1)) ,
4
Simplified Evidential Deep Learning
4.1
A first-order ERM view of classical EDL
α̃ = y + (1 − y) ⊙ α.
(7)
Before simplifying classical EDL, we first show that it can be viewed within the standard firstorder ERM framework. Classical EDL objectives are commonly written as nested expectations. However, the inner Dirichlet distribution is fully determined by the model output through the mapping z 7→ α(z). In practical implementations, the inner expectation Eπ∼Dir(α(z)) ℓ(π, y) is reduced to a closed-form expression before training (e.g. Eq. 5 or Eq. 6), yielding the scalar loss used for gradientbased optimization. Accordingly, the Dirichlet expectation defines a deterministic transformation of the base loss ℓ, yielding the induced loss ℓEDL on the model output. 4
Definition 2 (Induced evidential loss). Let ℓ : ∆K−1 × Y → R+ be a base loss. Then the induced evidential loss associated with classical EDL is defined by ℓEDL (z, y) := Eπ∼Dir(α(z)) ℓ(π, y) . (8) Because the Dirichlet parameters are fully determined by the model output, optimization remains over the model parameters θ alone. The inner expectation therefore does not introduce an additional optimization variable, but instead defines the scalar loss ℓEDL used in empirical risk minimization. Classical EDL can therefore be analyzed within the standard first-order empirical risk minimization framework using the induced loss ℓEDL . 4.2
Simplifying Dirichlet-expected evidential losses
Following the first-order ERM view in Section 4.1, we study the per-sample losses induced by the inner Dirichlet expectation and show that they are well approximated by plug-in losses evaluated at the projected probabilities. Definition 3 (Plug-in evidential loss). Let ℓ : ∆K−1 × Y → R+ be a classification loss on the probability simplex. The corresponding plug-in evidential loss is defined by ℓplug (α, y) = ℓ(Π(α), y),
Π(α) =
α , α0
α0 =
K X
αj .
(9)
j=1
Thus, the parameters α affect the objective only through the projected probabilities Π(α). Consequently, for a parametric model x 7→ αθ (x), the resulting expected-risk objective is E(X,Y )∼PXY ℓ(Π(αθ (X)), Y ) , (10) replacing the Dirichlet-expected loss used in classical EDL. For example we can write the crossentropy loss and the mean squared error loss in its plug-in formulation ℓCE plug (α, y) = − log Π(α)y ,
2 ℓMSE plug (α, y) = ∥Π(α) − y∥2 .
(11)
The plug-in cross-entropy objective also appears in Sensoy et al. [2018] through a Type-II maximum likelihood derivation, while the plug-in MSE corresponds to the classical EDL MSE objective (Eq. 6) without its Dirichlet variance term, as also considered in R-EDL (Chen et al. [2024]). In our formulation, both arise from the same construction: replacing the Dirichlet-expected loss by a classification loss on the probability simplex evaluated at the projected probabilities. More generally, any loss on the probability simplex induces a simplified evidential objective via composition with Π, avoiding explicit Dirichlet expectations while preserving the classifier output Π(α). 4.3
Simplified evidential classifiers
To make predictions explicit, we define a simplified evidential classifier whose decisions depend only on p̂(z), in contrast to the classical evidential classifier in Definition 1. Definition 4 (Simplified evidential classifier). A simplified evidential classifier is the induced probability map z 7→ p̂(z) = Π(α(z)) = Π(ϕ(τ (z))),
(12)
used with learning objectives that depend only on p̂(z), rather than directly on the intermediate parameter vector α(z). Interpretation. The distinction between Definitions 1 and 4 is not the predictive map itself, but the role assigned to the intermediate vector α during training. In classical EDL, α enters the objective through Dirichlet-based losses or regularization terms. Under plug-in objectives, α acts only as a deterministic parametrization whose projection p̂(z) determines both prediction and optimization. Thus, for plug-in objectives, the optimization problem can be studied through the induced probability map without relying on the full Dirichlet-expected loss structure. 5
Remark 1. Our definition of simplified evidential classifiers allows the additive constant c ≥ 0. The case c = 0 is included intentionally. Since the evidence map satisfies ei > 0, the parameters αi = ei still define a valid Dirichlet distribution. This case should not be confused with the Subjective Logic interpretation associated with classical EDL ( Sensoy et al. [2018], Chen et al. [2024]). Classical EDL uses αi = ei + 1, for which K/α0 corresponds to the usual vacuity mass. When c = 0, the fixed additive prior mass is absent, and K/α0 is no longer guaranteed to be bounded by one. We therefore interpret K/α0 for c = 0 as an inverse-concentration diagnostic rather than as standard Subjective-Logic vacuity. 4.4
Softmax models as Simplified evidential classifiers
Simplified evidential classifiers are defined through their logits-to-probabilities map. More broadly, any output layer that maps logits to positive class scores and then normalizes them fits this definition. The softmax classifier is a canonical example. Theorem 1 (Softmax as a simplified evidential classifier). Any neural network equipped with a softmax output layer defines a simplified evidential classifier in the sense of Definition 4. Proof. Let z ∈ RK denote the network logits. Choose τ (z) = exp(z) and ϕ(e) = e. Then i) αi (z) = exp(zi ), and therefore Π(α(z))i = PKαi (z) = PKexp(z , which is exactly the α (z) exp(z ) j=1
j
j=1
j
softmax mapping. Theorem 1 is structural: it concerns only the classifier parametrization and makes no assumption on the training loss. Thus, whenever optimization depends only on the predicted probabilities, standard softmax networks belong naturally to the simplified evidential framework. 4.5
Plug-in approximation for smooth losses
We can now derive bounds showing that the discrepancy between ℓEDL (α, y) and the plug-in loss ℓplug (α, y) is controlled by the concentration parameter α0 . This provides theoretical support for simplified evidential objectives depending only on projected probabilities. We consider sufficiently smooth losses that are twice continuously differentiable on the relevant region of the simplex, and we obtain the plug-in approximation by a first-order expansion around the Dirichlet mean. Theorem 2 (Explicit first-order expansion). Assume that for each label y, the loss ℓ(·, y) : ∆K−1 → R is twice continuously differentiable on a neighborhood of the relevant region of the simplex, with bounded and locally Lipschitz Hessian. Let π ∼ Dir(α), and let p̂ = Π(α). Then ℓEDL (α, y) = ℓplug (α, y) + R(α, y), where the remainder satisfies R(α, y) = O (α0 + 1)−1 .
(13)
Proof Sketch. We first apply the second-order Taylor bound and then apply Theorem 3 to bound the remainder term. After applying the definition of thecovariance for Dirichlet random variables, one obtains that the remainder is of size O (α0 + 1)−1 . Further, one can see that the first-order term vanishes in expectation and the second-order term is also of size O (α0 + 1)−1 by restructuring it and using the definition of the covariance for Dirichlet random variables. Proof details and explicit non-asymptotic bounds are provided in Appendix A. The preceding results show that Dirichlet-expected evidential losses can be replaced, up to concentration-controlled correction terms, by plug-in losses evaluated at projected probabilities: ℓEDL (α, y) ≈ ℓplug (α, y).
(14)
This yields simplified evidential objectives that depend only on Π(α), eliminating the need to explicitly evaluate Dirichlet-expected losses while retaining their dominant first-order behavior. Remark 2. For losses that are only Lipschitz continuous on the simplex, one can still approximate the Dirichlet-expected evidential loss by the plug-in loss, but with a weaker remainder O((α0 + 1)−1/2 ). See Appendix B. 6
The Taylor-based approximation in Theorem 2 requires uniformly bounded curvature on the relevant region of the simplex. This condition is natural for smooth losses such as the mean-squared error loss, but it is not globally valid for cross-entropy: for ℓ(π, y) = − log πy , the derivatives become singular as πy → 0. Since cross-entropy is central in classification, we state a separate result based on the Dirichlet logarithmic moment, under an interiority condition on the projected class probability. Lemma 1 (Cross-entropy plug-in correction). Let π ∼ Dir(α), p̂ = Π(α) = α/α0 , and α0 = PK K be a one-hot target vector, and let y ∈ {1, . . . , K} denote its target j=1 αj . Let y ∈ {0, 1} class index, so that yj = 1 if and only if j = y. Consider the cross-entropy loss ℓ(π, y) = PK − j=1 yj log πj = − log πy . Writing py = αy /α0 , if py ≥ δ > 0, then Eπ∼Dir(α) ℓ(π, y) = − log py + Oδ (α0−1 ). (15) Proof Sketch. Dirichlet logarithmic moment gives E[− log πy ] = ψ(α0 ) − ψ(αy ), where ψ denotes the digamma function. Rewriting αy = py α0 separates the plug-in term − log py from two digamma correction terms. The standard asymptotic bound |ψ(t) − log t| = O(t−1 ) gives an error of order O(α0−1 )+O(αy−1 ). Since py ≥ δ, we have αy ≥ δα0 , and hence the total correction is Oδ (α0−1 ). Remark 3. Our bounds should be read as high-concentration approximations, corresponding to the high-evidence regime that evidential training is intended to reach for well-supported predictions. In this regime, Dirichlet-expected losses and their plug-in counterparts have the same leadingorder behavior. Since plug-in losses depend on α only through Π(α), they isolate the projectedprobability component of the objective from the concentration scale. The concentration scale remains a separate modeling degree of freedom; Section 5 empirically shows, through matched no-KL and KL-regularized variants, that explicit regularization can substantially change concentration-based selective prediction. 4.6
Interpretation of the KL regularizer
Classical EDL objectives often include a KL-divergence regularizer between a Dirichlet distribution induced by the network output and a non-informative Dirichlet prior. From the first-order ERM perspective adopted in this work, this term is interpreted as a deterministic regularization function of the evidential parameters α. The Dirichlet distribution provides an analytic form for the penalty, but the resulting objective remains an ordinary deterministic training objective optimized by standard gradient-based methods. In the classical construction, the KL term is applied after replacing the target-class component by its prior value. Operationally, this removes the target-class evidence from the regularizer, so the penalty mainly discourages evidence assigned to incorrect classes. The KL term is therefore not required to define the simplified evidential classifier or the plug-in objectives, but it can affect the learned evidence scale and the induced class probabilities. Consequently, it may influence both concentrationbased uncertainty scores, such as vacuity, and probability-based scores, such as predictive entropy. This motivates the KL and no-KL variants included in the experimental comparison.
5
Experiments
We evaluate classical evidential classifiers (Definition 1) and simplified evidential classifiers (Definition 4) using both Dirichlet-expected losses and their plug-in counterparts (Definition 3). Our goal is to test whether the simplified framework preserves predictive accuracy and the operational usefulness of uncertainty for selective prediction. Motivated by Theorem 1, we also include softmax-style parameterizations and evaluate all variants under a common selective-prediction protocol. 5.1
Experimental setup
We evaluate all models on the Google Speech Commands v1 dataset (Warden [2018]) using the full 30-class classification task and the official train, validation, and test splits. All experiments use the same MatchboxNet backbone (Majumdar and Ginsburg [2020]) and follow the NVIDIA NeMo MatchboxNet preprocessing and augmentation pipeline (Kuchaiev et al. [2019]). They also use the same base optimizer, learning-rate schedule, and number of training epochs, except for the 7
objective-specific regularization choices summarized in Table 1. Thus, the comparisons isolate the effect of the evidential parameterization and training objective rather than differences in architecture, data processing, or optimization setup. Full configuration details are provided in Appendix C.1. Table 1: Model variants used in the experiments. Here ei = τ (zi ), α = ϕ(e), and p̂ = Π(α). For KL-regularized models, λt = min(1, t/T ), where t denotes the epoch, and the KL term is computed using shifted Dirichlet parameters αiKL = ei + 1. EDL-CE
EDL-CE no KL
EDL-MSE
softplus ei + 1 Dir. CE –
softplus ei + 1 Dir. MSE+KL 600
τ (zi ) softplus ϕ(ei ) ei + 1 Loss Dir. CE+KL T 400
Plug-in CE Plug-in MSE Softmax Softplus Softmax+KL Softmax+EDL-CE softplus ei + 1 CE(p̂) –
softplus ei + 1 MSE(p̂) –
exp softplus exp ei ei ei CE(p̂) CE(p̂) CE(p̂) + KL – – 400
exp ei Dir. CE –
The Softmax model recovers the standard softmax classifier by Theorem 1. The Softplus model is the analogous c = 0 projected classifier using a softplus evidence map instead of an exponential. 5.2
Selective-prediction protocol
We evaluate selective prediction using two uncertainty scores: PK − j=1 p̂j log p̂j K uvacuity = (16) , uentropy = α0 log K PK where α0 = j=1 αj . All scores are computed for every model under the common prediction parametrization introduced above and evaluated using the same thresholding protocol. For models with c = 0, we compute vacuity using the shifted concentration αis = ei + 1, so that α0s = PK j=1 ej + K; this preserves the uncertainty ordering. Given an uncertainty score and threshold t, predictions with uncertainty above t are withheld. With c nc , nf , and nw denoting correct, incorrect, and withheld samples, we report Accth = ncn+n , f n +n
c f Acctotal = nc +nnfc+nw , and Coverage = nc +n . The main tables report Acctotal at operating f +nw points selected by target Accth values of 99.0%, 99.5%, and 99.9%. Details of the operating-point selection rule are given in Appendix C.2.
5.3
Main results
Table 2 reports total accuracy at selective-prediction operating points selected to match target thresholded accuracies, using entropy and vacuity as uncertainty scores. Under entropy-based selection, the c = 0 simplified evidential classifiers trained with CE(p̂), namely Softmax and Softplus models, achieve the best or near-best total accuracy across most operating points. For example, at the 99.9% target, Softmax and Softplus models achieve total accuracies of 88.41% and 87.64%, respectively. This shows that entropy remains a strong uncertainty score for selective prediction on GSC V1, and that the simplified evidential classifiers trained with plug-in losses provide a strong baseline for both prediction and selective prediction. The plug-in variants closely track their classical EDL counterparts across both CE and MSE losses. Plug-in EDL-CE achieves nearly the same base accuracy and total accuracy at the selective operating points as EDL-CE under both entropy- and vacuity-based thresholding. The same pattern is observed for EDL-MSE and Plug-in EDL-MSE. These results are consistent with the approximation results in Section 4: replacing the Dirichlet-expected loss by the corresponding loss evaluated at the projected Dirichlet mean preserves the main predictive and operational uncertainty behavior on this benchmark. The vacuity-based results highlight that Dirichlet-expected losses alone do not make vacuity competitive with entropy. Without KL regularization, vacuity still contains some reliability information, since thresholding by vacuity improves thresholded accuracy. However, the no-KL variants degrade strongly at stricter operating points. In particular, Softmax outperforms both EDL-CE no KL and Softmax+EDL-CE under vacuity-based selection: at the 99.9% target, Softmax obtains 62.00% total accuracy, compared with 47.14% for EDL-CE no KL and 50.81% for Softmax+EDL-CE. Thus, replacing the standard CE(p̂) objective by the Dirichlet-expected CE loss does not by itself make 8
vacuity more effective. In contrast, adding KL regularization to the Softmax model raises the corresponding vacuity-based total accuracy to 80.36%, while entropy-based selection remains broadly similar despite visible changes in the KDE plots in Figure 1. This suggests that the KL regularizer substantially improves the reliability ranking induced by vacuity. Overall, the experiments indicate that the proposed simplified EDL framework, based on plug-in losses and simplified evidential classifiers, captures the main predictive and selective-prediction behavior of the classical EDL framework on this benchmark. The results also show that KL regularization mainly affects the usefulness of vacuity, rather than entropy, for selective prediction. It should be noted that in some cases the two uncertainty metrics lead to almost exactly the same coverage because the accepted sets selected by entropy and vacuity largely overlap. In other cases, the rankings induced by the two metrics differ substantially, leading to different selective-prediction behavior. Table 2: Total accuracy under entropy- and vacuity-based selective prediction on the GSC V1 test split. Values are means over 5 runs with ±2σ. Columns report Acctotal at the largest-coverage operating point satisfying the indicated Accth target up to numerical tolerance. Corresponding entropy-based curves are shown in Figure 2. Entropy Model
Base Acc.
99.0%
99.5%
Vacuity 99.9%
99.0%
Softmax 97.21±0.18 96.47±0.59 94.50±1.23 88.41±2.39 Softplus 97.07±0.16 96.14±0.46 94.81±0.63 87.64±6.41 Softmax + KL 96.84±0.18 95.79±0.35 94.19±0.80 84.53±3.12 Softmax + EDL-CE 97.03±0.32 96.12±0.71 94.79±0.70 87.76±6.57 EDL-CE 96.88±0.37 95.76±0.75 93.61±1.26 81.61±5.80 EDL-CE no KL 96.68±0.26 94.81±1.17 92.40±1.26 75.58±14.11 Plug-in EDL-CE 96.84±0.38 95.68±0.67 93.39±1.57 83.55±6.58 EDL-MSE 96.55±0.19 94.91±0.37 92.87±0.95 80.93±5.09 Plug-in EDL-MSE 96.55±0.10 94.88±0.31 92.46±0.62 82.53±6.08
175
25
Correct Incorrect
150
Correct Incorrect
20
50 25 0.2
0.0
0.2
Vacuity
Softmax
0.4
0.6
10
Correct Incorrect
10 8
15
Density
75
15
Density
Density
Density
100
99.9%
95.54±1.38 86.57±2.93 62.00±17.09 63.93±6.19 58.75±4.13 52.87±6.31 95.71±0.35 94.13±0.75 80.36±7.60 94.45±2.24 84.79±3.67 50.81±24.50 95.73±0.78 93.55±1.23 81.62±5.81 89.28±3.83 81.09±6.57 47.14±15.60 95.68±0.68 93.40±1.56 83.55±6.58 94.89±0.39 92.87±0.95 80.93±5.09 94.86±0.26 92.46±0.62 82.53±6.08
20
125
0
Correct Incorrect
99.5%
10
6 4
5
5
0
0
0.0
0.2
Vacuity
0.4
0.6
Softmax + KL
2
0.0
0.2
0.4
Vacuity
EDL-CE no KL
0.6
0
0.25 0.00
0.25
0.50
Vacuity
0.75
1.00
1.25
EDL-CE
Figure 1: Vacuity KDEs for correctly and incorrectly classified test samples on GSC V1. The selected pairs compare KL-regularized models with their no-KL counterparts; additional KDE plots are provided in Appendix C.4.
6
Conclusion and future directions
In this work, we showed that classical EDL objectives can be approximated by a class of simplified plug-in objectives. Empirically we demonstrated that these approximations perform as well as their classical counterparts for predictive and selective prediction on the Google Speech Command v1 dataset. These results suggest that there is a strong practical value of evidential methods when output parametrization, plug-in losses, and potentially regularization are combined. These findings motivate future work on the simplified EDL framework, consisting of plug-in losses (Definition 3) and simplified evidential classifiers (Definition 4), together with explicit uncertaintyaware regularization. In particular, softmax-style models, viewed as simplified evidential classifiers by Theorem 1, provide a natural starting point for studying decision-relevant uncertainty while retaining simple single-pass inference. 9
A limitation of the present study is that the empirical evaluation is restricted to a single keywordspotting benchmark and in-distribution selective prediction. Future work should therefore evaluate the same framework on a wider range of datasets, architectures, and distribution-shift settings. This is especially important because selective prediction can improve deployment reliability by identifying uncertain situations in which additional safety measures or human review may be needed. A natural follow-up question would be to understand when the uncertainty ordering induced by vacuity and entropy does or does not agree locally or globally, and how to best combine these two uncertainty metrics through a two-threshold selective-prediction rule.
Acknowledgments and Disclosure of Funding FK gratefully acknowledges the support of the German Science Foundation (DFG) in the context of the priority program Theoretical Foundations of Deep Learning (project KR 4512/6-1). HL and FK gratefully acknowledge the support of the Munich Center for Machine Learning (MCML).
References Alexander Amini, Wilko Schwarting, Ava Soleimany, and Daniela Rus. Deep evidential regression. Advances in Neural Information Processing Systems, 33:14927–14937, 2020. Viktor Bengs, Eyke Hüllermeier, and Willem Waegeman. Pitfalls of epistemic uncertainty quantification through loss minimisation. Advances in Neural Information Processing Systems, 35: 29205–29216, 2022. Charles Blundell, Julien Cornebise, Koray Kavukcuoglu, and Daan Wierstra. Weight uncertainty in neural network. In International Conference on Machine Learning, pages 1613–1622. PMLR, 2015. Mengyuan Chen, Junyu Gao, and Changsheng Xu. R-edl: Relaxing nonessential settings of evidential deep learning. In Proceedings of the International Conference on Learning Representations, 2024. Mengyuan Chen, Junyu Gao, and Changsheng Xu. Revisiting essential and nonessential settings of evidential deep learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025. doi: 10.1109/TPAMI.2025.3583410. Arthur P Dempster. A generalization of bayesian inference. Journal of the Royal Statistical Society: Series B (Methodological), 30(2):205–232, 1968. Danruo Deng, Guangyong Chen, Yang Yu, Furui Liu, and Pheng-Ann Heng. Uncertainty estimation by fisher information-based evidential deep learning. In International Conference on Machine Learning, pages 7596–7616. PMLR, 2023. Yarin Gal and Zoubin Ghahramani. Dropout as a bayesian approximation: Representing model uncertainty in deep learning. In International Conference on Machine Learning, pages 1050–1059. PMLR, 2016. Junyu Gao, Mengyuan Chen, Liangyu Xiang, and Changsheng Xu. A comprehensive survey on evidential deep learning and its applications. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025. doi: 10.1109/TPAMI.2025.3625258. M. Haussmann, S. Gerwinn, and M. Kandemir. Bayesian evidential deep learning with pac regularization. In Advances in Approximate Bayesian Inference Symposium, 2020. Andreas Nugaard Holm, Dustin Wright, and Isabelle Augenstein. Revisiting softmax for uncertainty approximation in text classification. Information, 14(7):420, 2023. Audun Jøsang. A logic for uncertain probabilities. International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems, 9(03):279–311, 2001. Audun Jsang. Subjective Logic: A formalism for reasoning under uncertainty. Springer Publishing Company, Incorporated, 2018. 10
Mira Jürgens, Nis Meinert, Viktor Bengs, Eyke Hüllermeier, and Willem Waegeman. Is epistemic uncertainty faithfully represented by evidential deep learning methods? In Proceedings of the 41st International Conference on Machine Learning, pages 22624–22642, 2024. Oleksii Kuchaiev, Jason Li, Huyen Nguyen, Oleksii Hrinchuk, Ryan Leary, Boris Ginsburg, Samuel Kriman, Stanislav Beliaev, Vitaly Lavrukhin, Jack Cook, Mariya Popova, Jocelyn Huang, and Jonathan M. Cohen. Nemo: a toolkit for building ai applications using neural modules. arXiv preprint arXiv:1909.09577, 2019. Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and scalable predictive uncertainty estimation using deep ensembles. Advances in Neural Information Processing Systems, 30, 2017. Somshubra Majumdar and Boris Ginsburg. Matchboxnet: 1d time-channel separable convolutional neural network architecture for speech commands recognition. In Proc. Interspeech 2020, pages 3356–3360, 2020. Andrey Malinin and Mark Gales. Predictive uncertainty estimation via prior networks. Advances in Neural Information Processing Systems, 31, 2018. Andrey Malinin, Sergey Chervontsev, Ivan Provilkov, and Mark Gales. Regression prior networks. arXiv preprint arXiv:2006.11590, 2020. Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalkar. Foundations of machine learning. MIT press, 2018. Radford M Neal. Bayesian learning for neural networks, volume 118. Springer Science & Business Media, 2012. Yurii Nesterov. Introductory lectures on convex optimization: A basic course, volume 87. Springer Science & Business Media, 2013. Yaniv Ovadia, Emily Fertig, Jie Ren, Zachary Nado, David Sculley, Sebastian Nowozin, Joshua Dillon, Balaji Lakshminarayanan, and Jasper Snoek. Can you trust your model’s uncertainty? evaluating predictive uncertainty under dataset shift. Advances in neural information processing systems, 32, 2019. Murat Sensoy, Lance Kaplan, and Melih Kandemir. Evidential deep learning to quantify classification uncertainty. In Advances in Neural Information Processing Systems, volume 31, pages 3183–3193, 2018. Glenn Shafer. A Mathematical Theory of Evidence, volume 42. Princeton University Press, 1976. Glenn Shafer. Perspectives on the theory and practice of belief functions. International Journal of Approximate Reasoning, 4(5-6):323–362, 1990. Maohao Shen, Jongha Jon Ryu, Soumya Ghosh, Yuheng Bu, Prasanna Sattigeri, Subhro Das, and Gregory W Wornell. Are uncertainty quantification capabilities of evidential deep learning a mirage? Advances in Neural Information Processing Systems, 37:107830–107864, 2024. Theodoros Tsiligkaridis. Information aware max-norm dirichlet networks for predictive uncertainty estimation. Neural Networks, 135:105–114, 2021. Dennis Thomas Ulmer, Christian Hardmeier, and Jes Frellsen. Prior and posterior networks: A survey on evidential deep learning methods for uncertainty estimation. Transactions on Machine Learning Research, 2023. Pete Warden. Speech commands: A dataset for limited-vocabulary speech recognition. arXiv preprint arXiv:1804.03209, 2018. Andrew G Wilson and Pavel Izmailov. Bayesian deep learning and a probabilistic perspective of generalization. Advances in Neural Information Processing Systems, 33:4697–4708, 2020.
11
A
Explicit expansion of Dirichlet-expected losses
This appendix provides an explicit asymptotic expansion of Dirichlet-expected losses around the Dirichlet mean. The result characterizes how Dirichlet-based objectives differ from plug-in losses evaluated at the projected probabilities and shows that the additional contribution enters as a varianceinduced correction term arising from the Dirichlet covariance. This expansion clarifies the structural role of projected probabilities in evidential classifiers and supports the classifier definitions adopted in the main text. Theorem 3 (Second-order Taylor bound for twice continuously differentiable Lipschitz functions [Nesterov, 2013, Lemma 1.2.4]). Let U ⊂ RK be open and let f : U → R be twice continuously differentiable. Assume that ∇2 f is Lipschitz on U with constant M ≥ 0 in operator norm, i.e. ∥∇2 f (u) − ∇2 f (v)∥op ≤ M ∥u − v∥2 ,
∀ u, v ∈ U.
Then for all x, y ∈ U such that the line segment [x, y] ⊂ U , we have ∇f (y) − ∇f (x) − ∇2 f (x)(y − x) 2 ≤
M ∥y − x∥22 , 2
1 M f (y) − f (x) − ∇f (x)⊤ (y − x) − (y − x)⊤ ∇2 f (x)(y − x) ≤ ∥y − x∥32 . 2 6
(17) (18)
Given the above we can restate our main result Theorem 2. Theorem 4 (Explicit expansion of Dirichlet-expected losses). Let π ∼ Dir(α),
α ∈ RK +,
α0 :=
K X
αi ,
p̂ := Π(α) =
i=1
α ∈ ∆K−1 . α0
(19)
Fix a label y. Assume that ℓ(·, y) is twice continuously differentiable on ∆K−1 , with Hessian Hℓ (q, y) := ∇2q ℓ(q, y).
(20)
Assume further that there exist constants M ≥ 0 and G ≥ 0 such that, for all q, u, v ∈ ∆K−1 , ∥Hℓ (q, y)∥op ≤ M,
(21)
∥Hℓ (u, y) − Hℓ (v, y)∥op ≤ G ∥u − v∥2 .
(22)
Then 1 L1 (p̂, y) + R(α, y) α0 + 1 = ℓplug (α, y) + O (α0 + 1)−1
ℓEDL (α, y) = Eπ∼Dir(α) [ℓ(π, y)] = ℓ(p̂, y) +
(23) (24)
where L1 (p̂, y) :=
1 tr Hℓ (p̂, y) Diag(p̂) − p̂p̂⊤ . 2
(25)
Proof. Fix y and set h := π − p̂.
(26)
Since π, p̂ ∈ ∆K−1 and the simplex is convex, the line segment between p̂ and π is contained in ∆K−1 . Applying Taylor’s theorem with Lipschitz Hessian to q 7→ ℓ(q, y) along this segment gives 1 ℓ(π, y) = ℓ(p̂, y) + ∇ℓ(p̂, y)⊤ h + h⊤ Hℓ (p̂, y)h + r(π, y), 2
(27)
Given Theorem 3 we obtain the following bound of the remainder term |r(π, y)| ≤
G G 3 3 ∥h∥2 = ∥π − p̂∥2 . 6 6 12
(28)
Taking expectations and using E[π − p̂] = 0, we obtain 1 E[ℓ(π, y)] = ℓ(p̂, y) + E h⊤ Hℓ (p̂, y)h + E[r(π, y)]. 2 The quadratic term satisfies E h⊤ Hℓ (p̂, y)h = tr(Hℓ (p̂, y) Cov(π)) .
(29)
(30)
For π ∼ Dir(α), Cov(π) =
1 Diag(p̂) − p̂p̂⊤ . α0 + 1
(31)
Therefore, E[ℓ(π, y)] = ℓ(p̂, y) +
1 L1 (p̂, y) + R(α, y), α0 + 1
(32)
here R(α, y) := E[r(π, y)] and L1 as defined in Eq. 25. To bound L1 (p̂, y), note that Diag(p̂) − p̂p̂⊤ is symmetric positive semidefinite. By the trace bound, 1 ∥Hℓ (p̂, y)∥op tr Diag(p̂) − p̂p̂⊤ 2 1 2 = ∥Hℓ (p̂, y)∥op (1 − ∥p̂∥2 ) 2 1 2 ≤ M (1 − ∥p̂∥2 ). 2
|L1 (p̂, y)| ≤
(33) (34) (35)
2
Since ∥p̂∥2 ≥ 1/K for every p̂ ∈ ∆K−1 , it follows that 1 1 |L1 (p̂, y)| ≤ 1− M. 2 K
(36)
We still need to bound the remainder by the Taylor remainder bound, |R(α, y)| = |E[r(π, y)]| ≤ E|r(π, y)| ≤
G 3 E ∥π − p̂∥2 . 6
(37)
Since π, p̂ ∈ ∆K−1 , we have ∥π − p̂∥2 ≤ 2,
3
2
∥π − p̂∥2 ≤ 2 ∥π − p̂∥2 .
(38)
Consequently, 2
1 − ∥p̂∥2 . α0 + 1
(39)
G 1 − ∥p̂∥2 G G 1 − ∥p̂∥2 ·2 = ≤ . 6 α0 + 1 3 α0 + 1 3(α0 + 1)
(40)
3
2
E ∥π − p̂∥2 ≤ 2E ∥π − p̂∥2 = 2 tr(Cov(π)) = 2 Therefore, 2
|R(α, y)| ≤
2
Thus, R(α, y) = O (α0 + 1)−1
(41)
which proves the theorem. Next, we will restate the cross-entropy specific plug-in correction (Lemma 1) and prove it. Lemma 2 (Cross-entropy plug-in correction). Let π ∼ Dir(α),
α ∈ RK +,
α0 =
K X i=1
13
αi ,
p̂ = Π(α) =
α . α0
(42)
Let y ∈ {0, 1}K be a one-hot target vector, and let y ∈ {1, . . . , K} denote its target class index, so that yj = 1 if and only if j = y. Equivalently, py =
αy . α0
(43)
For the one-hot cross-entropy loss ℓ(π, y) = − log πy , if py ≥ δ > 0, we have E[− log πy ] = − log py + ∆CE (α, y) ≤ − log py + Oδ (α0−1 ),
(44)
∆CE (α, y) := ψ(α0 ) − log α0 − ψ(αy ) − log αy ,
(45)
where
and ψ denotes the digamma function. Proof. For a Dirichlet random vector, the standard log-moment identity gives E[log πy ] = ψ(αy ) − ψ(α0 ).
(46)
E[− log πy ] = ψ(α0 ) − ψ(αy ).
(47)
Therefore,
α
Since py = αy0 the plug-in cross-entropy loss is αy = log α0 − log αy . − log py = − log α0
(48)
Subtracting the plug-in loss from the Dirichlet-expected loss gives E[− log πy ] − (− log py ) = ψ(α0 ) − ψ(αy ) − log α0 + log αy = ψ(α0 ) − log α0 − ψ(αy ) − log αy = ∆CE (α, y).
(49) (50)
This proves the exact decomposition. Using the standard bound |ψ(t) − log t| ≤
1 , t
t > 0,
(51)
we obtain |∆CE (α, y)| ≤ |ψ(α0 ) − log α0 | + |ψ(αy ) − log αy | 1 1 ≤ + . α0 αy
(52) (53)
Since αy = α0 py , this can be written as 1 1 + . α0 α0 py
(54)
1 1 1 + δ −1 + = . α0 α0 δ α0
(55)
E[− log πy ] = − log py + Oδ (α0−1 ).
(56)
|∆CE (α, y)| ≤ If py ≥ δ > 0, then |∆CE (α, y)| ≤ Thus,
14
B
Lipschitz plug-in approximation
In this section, we will state the plug-in approximation for the class of Lipschitz functions, which in contrast to Theorem 2 does not require the loss to be differentiable. Lemma 3 (Lipschitz plug-in approximation). Assume that for each label y, the loss ℓ(·, y) : ∆K−1 → R is L-Lipschitz with respect to the Euclidean norm. Let π ∼ Dir(α) and denote p = Π(α). Then s 1 − ∥p∥22 L ℓEDL (α, y) − ℓplug (α, y) ≤ L . (57) ≤√ α0 + 1 α0 + 1 Proof. By the Lipschitz property of ℓ, ℓ(π, y) − ℓ(p, y) ≤ L∥π − p∥2 .
(58)
Taking expectations and applying Jensen’s inequality, q ℓEDL (α, y) − ℓplug (α, y) = Eπ∼Dir(α) [ℓ(π, y)] − ℓ(p, y) ≤ L E∥π − p∥2 ≤ L E∥π − p∥22 . (59) Since p = E[π], E∥π − p∥22 = tr(Cov(π)).
(60)
pk (1 − pk ) , α0 + 1
(61)
For a Dirichlet random vector, Var(πk ) = hence tr(Cov(π)) =
K X pk (1 − pk ) k=1
α0 + 1
=
1 − ∥p∥22 . α0 + 1
(62)
Substituting into the previous bound gives (57). The final inequality follows from ∥p∥22 ≥ 0.
C
More experimental results and details
C.1
Experimental details
We use the public Google Speech Commands v1 dataset ( Warden [2018]), released under the Creative Commons Attribution 4.0 license. Experiments are implemented with NVIDIA NeMo ( Kuchaiev et al. [2019]), released under the Apache 2.0 license, and use a MatchboxNet backbone following Majumdar and Ginsburg [2020]. Hardware and software. Experiments are run on a workstation with an AMD Radeon RX 7900 XTX GPU and an AMD Ryzen 7 7700 CPU under Windows 11 using WSL with Ubuntu 24.04. The training environment uses ROCm 7.2.2 and NVIDIA NeMo for the MatchboxNet implementation, preprocessing, and training pipeline. Training setup. Following the MatchboxNet optimization setup (Majumdar and Ginsburg [2020]), all models are trained for 200 epochs using NovoGrad with β1 = 0.95, β2 = 0.5, maximum learning rate 0.05, and minimum learning rate 10−3 . We use a warmup-hold-decay learning-rate schedule with 5% warmup, 45% hold, and second-order polynomial decay for the remaining steps. Final runs use batch size 256 on a single GPU. Weight decay is set to 10−3 for models without KL regularization and to 0 for models trained with the KL regularizer, avoiding an additional weight decay penalty on top of the explicit Dirichlet regularizer. 15
Preprocessing and augmentation. We follow the MatchboxNet preprocessing and augmentation pipeline (Majumdar and Ginsburg [2020]). Each waveform is converted into 64 MFCC features computed from 25 ms windows with 10 ms stride, and the temporal dimension is symmetrically zero-padded or cropped to 128 feature vectors. During training, we apply random time shifts in [−5, 5] ms, additive white noise with magnitude in [−90, −46] dB, SpecAugment with two time masks of width up to 25 time steps and two frequency masks of width up to 15 frequency bands, and SpecCutout with five rectangular masks using the same time and frequency dimensions. Runtime. A single 200-epoch training run took approximately 1 hour on the workstation with an AMD Radeon RX 7900 XTX GPU and an AMD Ryzen 7 7700 CPU. The final experimental suite consisted of 45 independent training runs and was completed in approximately 22–24 hours of wallclock time by running up to three jobs in parallel on the same GPU. Evaluation, selective-prediction curve computation, and plotting were inexpensive compared with training. C.2
Selective-prediction operating point selection
For each trained model and uncertainty score, we compute a selective-prediction curve by sorting test samples in increasing uncertainty and evaluating prefixes of this ordering. Each point on the curve corresponds to a thresholded classifier that accepts samples below an uncertainty threshold and withholds the remaining samples. For a target thresholded accuracy a⋆ , we select the operating point with maximum coverage among all thresholds satisfying Accth ≥ a⋆ − ε, ε = 10−6 . The selected operating point is τ ⋆ ∈ arg max Coverage(τ ) τ
subject to
Accth (τ ) ≥ a⋆ − ε.
This asymmetric rule reflects the operational selective-prediction setting: for a required reliability level, the goal is to retain as many samples as possible. The tolerance ε is used only to avoid numerical rounding effects. For every selected operating point, we also verify the identity Acctotal = Accth · Coverage, up to numerical precision. C.3
Accuracy vs coverage
Figure 2 reports entropy-based selective-prediction threshold curves for all model variants. For each run, the normalized predictive-entropy threshold is varied, and the corresponding coverage and thresholded accuracy are computed. Each panel therefore shows directly how the operating point changes with the entropy threshold. Stricter thresholds reject more samples, reducing coverage while typically increasing the accuracy among the retained samples. The plots provide a visual summary of this trade-off and of the variability across runs, whereas the tables report the quantitative comparisons at selected operating points. Solid lines denote the mean over five runs and shaded regions denote ±2σ. C.4
Additional uncertainty distributions
Figures 3 and 4 show additional KDE plots of the uncertainty scores for correctly and incorrectly classified test samples. The entropy plots are shown for all model variants, while the vacuity plots include the variants not shown in the main text. These figures are intended as diagnostic visualizations of how each uncertainty score separates correct and incorrect predictions; the selective-prediction tables provide the quantitative comparison at fixed operating points.
16
0.8
0.6
0.985 0.980
0.4 Thresholded accuracy ±2 std. acc. Coverage ±2 std. cov.
0.2
0.4
0.6
0.8
Entropy threshold
1.0
0.975
Thresholded accuracy ±2 std. acc. Coverage ±2 std. cov.
0.970 0.965
0.0
0.4
0.0
0.2
0.990
0.6
0.985 0.980
0.965
0.4 Thresholded accuracy ±2 std. acc. Coverage ±2 std. cov.
0.0
0.2
0.4
0.6
0.8
Entropy threshold
1.0
0.2 0.0
0.6
0.985 0.980
0.4 Thresholded accuracy ±2 std. acc. Coverage ±2 std. cov.
0.2
0.4
0.6
0.970 0.965
0.0
0.2
Entropy threshold
0.8
Plug-in EDL-CE
1.0
0.2 0.0
0.4
0.6
0.8
Entropy threshold
1.0
0.2 0.0
Softmax + KL 1.000
1.0
0.995
0.8
0.995
0.8
0.990
0.6
0.985 0.980
0.4
0.975
Thresholded accuracy ±2 std. acc. Coverage ±2 std. cov.
0.970 0.0
0.2
0.4
0.6
Entropy threshold
0.8
1.0
0.2 0.0
0.990
0.6
0.985 0.980
0.4
0.975
Thresholded accuracy ±2 std. acc. Coverage ±2 std. cov.
0.970 0.965
0.2
0.4
0.6
0.8
Entropy threshold
1.0
0.2 0.0
EDL-CE no KL
1.000
1.0
1.000
1.0
0.995
0.8
0.995
0.8
0.990
0.6
Coverage
0.990
0.0
Thresholded accuracy ±2 std. acc. Coverage ±2 std. cov.
1.000
0.965
Thresholded accuracy
0.8
Coverage
Thresholded accuracy
0.995
0.965
0.0
0.4
0.975
EDL-CE 1.0
0.970
1.0
0.980
1.0
Softmax + EDL-CE 1.000
0.975
Entropy threshold
0.8
Coverage
0.8
Thresholded accuracy
0.995
Coverage
Thresholded accuracy
1.000
1.0
0.970
0.6
0.6
0.985
Softplus Proj.
Softmax
0.975
0.4
0.2
0.8
0.990
Coverage
0.0
0.980
1.0
0.995
0.985 0.980
0.4
0.975
Thresholded accuracy ±2 std. acc. Coverage ±2 std. cov.
0.970 0.965
0.0
0.2
0.4
0.6
Entropy threshold
EDL-MSE
0.8
1.0
0.2 0.0
0.990
0.6
Coverage
0.965
0.985
Thresholded accuracy
0.970
0.2
0.6
Thresholded accuracy
0.975
0.990
Coverage
0.990
1.000
Coverage
1.0
0.995
Thresholded accuracy
1.000
0.8
Thresholded accuracy
1.0
0.995
Coverage
Thresholded accuracy
1.000
0.985 0.980
0.4
0.975
Thresholded accuracy ±2 std. acc. Coverage ±2 std. cov.
0.970 0.965
0.0
0.2
0.4
0.6
Entropy threshold
0.8
1.0
0.2 0.0
Plug-in EDL-MSE
Figure 2: Entropy-based selective-prediction threshold curves for all model variants on GSC V1. Each panel shows thresholded accuracy and coverage as functions of the normalized predictiveentropy threshold, averaged over five runs. The curves illustrate the selective-prediction trade-off: stricter uncertainty thresholds reject more samples, reducing coverage while typically increasing the accuracy among accepted samples. Solid lines show the mean across runs and shaded regions denote ±2σ. Quantitative operating-point comparisons at target thresholded accuracies are reported in the tables.
17
0.25 0.00 0.25 0.50 0.75 1.00 1.25
Correct Incorrect
40 35 30 25 20 15 10 5 0
8 6 4 2 0
0.25 0.00 0.25 0.50 0.75 1.00 1.25
Entropy Softplus Proj.
8
Correct Incorrect
Density
Density
Density
4 3
Correct Incorrect
7
0.0
0.4
0.6
Entropy EDL-CE
0.8
1.0
3
4 3
2
1
1
0.4
0.6
0.8
1.0
Entropy Plug-in EDL-CE
0
1.2
0.8
1.0
1.2
Correct Incorrect
3
1 0.2
0.6
Entropy EDL-CE no KL
4
2 0.0
0.4
5
2
0
0.2
6
Density
4
0.0
7
5
5
Correct Incorrect
0
1.2
Correct Incorrect
6
Density
Density
0.2
7
6
1.25
2
Entropy Softmax + EDL-CE
8
1.00
4
0
0.25 0.00 0.25 0.50 0.75 1.00 1.25
0.75
6
1 0
0.50
Entropy Softmax + KL
8
5
2
10
0.25
10
6
20
0.25 0.00
12
Correct Incorrect
7
30
Correct Incorrect
10
Entropy Softmax
40
12
Density
Correct Incorrect
Density
Density
40 35 30 25 20 15 10 5 0
0.0
0.2
0.4
0.6
Entropy EDL-MSE
0.8
1.0
0
1.2
0.0
0.2
0.4
0.6
0.8
Entropy Plug-in EDL-MSE
1.0
1.2
Figure 3: Entropy KDE plots for all model variants on GSC V1. Each plot shows the distribution of normalized predictive entropy for correctly and incorrectly classified test samples. These plots are intended as diagnostic visualizations. Correct Incorrect
12
800
8
Density
Density
10 6 4
8
600 400
0.6
0.7
0.8
Vacuity Softplus Proj.
0.9
0
1.0
10
0.05
0.00
0.05
0.10
Vacuity Softmax + EDL-CE
0.15
0
0.20
0.25 0.00 0.25 0.50 0.75 1.00 1.25
Vacuity Plug-in EDL-CE
10
Correct Incorrect
8
Density
Density
2
6 4
6 4 2
2 0
6 4
Correct Incorrect
8
Correct Incorrect
10
200
2 0
12
Correct Incorrect
Density
14
0.25 0.00
0.25
0.50
0.75
Vacuity EDL-MSE
1.00
0
1.25
0.25 0.00
0.25
0.50
0.75
Vacuity Plug-in EDL-MSE
1.00
1.25
Figure 4: Additional vacuity KDE plots for model variants not shown in the main text. Each plot shows the distribution of vacuity for correctly and incorrectly classified test samples. Together with the main-text vacuity KDEs, these plots provide a visual diagnostic of how concentration-based uncertainty separates correct and incorrect predictions.
18