Robust Risk Under Evolving Uncertainty: A Wasserstein Counterpart of the Entropic Value-at-Risk

Robust Risk Under Evolving Uncertainty: A Wasserstein Counterpart of the Entropic Value-at-Risk

Deep Ganguly* 1

Jan Křetínský1,2

1

arXiv:2608.19073v1 [cs.AI] 19 Aug 2026

2

Technical University of Munich, Germany Masaryk University, Brno, Czech Republic

Abstract An agent still learning its environment should be cautious while ignorant and bold once confident. The entropic value-at-risk captures this through a robust-optimization identity—a confidence level fixes the radius of a relative-entropy ball of alternative models—but that ball cannot reach catastrophes the nominal deems impossible, precisely what a safe agent must hedge. We instead use an optimal-transport ball and study the coherent risk measure it induces, the Wasserstein entropic valueat-risk. It has a variational dual mirroring the entropic formula (an inverse temperature becomes a transport price), occupies a definite place in the risk hierarchy, and provably accounts for the reachable catastrophes the entropic measure ignores; we verify both dualities numerically. Driving the transport radius by belief entropy then yields a closed-form robust dynamic-programming operator whose caution contracts as the belief sharpens, with a certified safety sandwich and a sharp safety switch.

1

RISK UNDER EVOLVING UNCERTAINTY

Consider a drone crossing a canyon under uncertain wind. A high-cruise policy is fast when the air is calm but is thrown into the cliffs by a sudden gust; a hover-and-creep policy is safe in any wind but slow. The wind regime—a hidden type z drawn once—is never observed directly; the drone sees only noisy signals and must infer z as it flies. Two reflexes each fail. A controller that assumes the worst case forever (robust control [Iyengar, 2005, Nilim and El Ghaoui, 2005, Wiesemann et al., 2013]) keeps hovering even after * Funded by the Deutsche Forschungsgemeinschaft (DFG) re-

search training group ConVeY (GRK 2428).

the air has visibly calmed: safe, but so conservative that operators switch it off. A controller that commits before its belief sharpens (expected-value or Bayesian planning [Kaelbling et al., 1998, Ghavamzadeh et al., 2015]) cruises on a hunch, and a single wrong guess about z is fatal—an expected return that blends a benign mode with a catastrophic one is a number nobody actually attains. What is wanted is a controller whose caution evolves with its evidence— maximal under ignorance, vanishing once the environment is identified—together with a safety bound that holds at every stage of learning. The agent’s risk attitude must therefore be a function of its evolving epistemic state, and the entropy of its belief is a live readout of how much it should distrust its own model. The entropic value-at-risk [Ahmadi-Javid, 2012] turns this intuition into algebra. It is the unique single-parameter coherent family that sweeps from the mean to the essential supremum, and it has an exact distributionally robust representation: its confidence level α equals the radius − ln α of a relative-entropy ball of alternative laws. Ganguly et al. [2025] further show the induced optimization is convex and admits well-posed, convergent estimation. This robust reading is what we want for evolving uncertainty: “how conservative should I be?” becomes “how large should my ambiguity set be?”, and the latter is answered by the agent’s own uncertainty. But the relative-entropy geometry has a blind spot that is decisive for safety. A ball {Q : KL(Q∥P ) ≤ ρ} contains only laws absolutely continuous with respect to the nominal P : if P assigns zero probability to a catastrophic transition, no member assigns it positive probability, since KL is infinite off the support of P . As the agent learns and its nominal kernel concentrates away from rare disasters, the relative-entropy adversary loses the ability even to represent those disasters, so the safety check becomes vacuous precisely when overconfidence is most dangerous. The optimaltransport (Wasserstein) geometry has no such blind spot: it prices a perturbation by how far mass must physically move, so a verifier can still ask “what if it is a storm?” at a cost

Accepted for the 2nd Workshop on Safe AI at UAI (SafeAI 2026).

proportional to the distance to that outcome—reachable but expensive [Mohajerin Esfahani and Kuhn, 2018, Gao and Kleywegt, 2023, Blanchet and Murthy, 2019]. Related work. Risk-averse dynamic programming [Ruszczyński, 2010], risk-sensitive control and reinforcement learning [Howard and Matheson, 1972, Chow et al., 2015, Hau et al., 2023], percentile and parameteruncertainty MDPs [Delage and Mannor, 2010], constrained and safe RL [Altman, 1999, García and Fernández, 2015, Sui et al., 2015, Brunke et al., 2022], and adversarially- or Wasserstein-robust RL [Pinto et al., 2017, Abdullah et al., 2019] all add caution to sequential decisions—but with a fixed attitude. Our contribution is to let the ambiguity radius, hence the risk attitude, be read off the agent’s evolving belief, and to give it the optimal-transport geometry safety requires [Kuhn et al., 2019]. Contributions. We keep the entropic value-at-risk’s robust formulation and its guarantees, and replace its ball. • We define the Wasserstein entropic value-at-risk (WEVaR) and prove a Kantorovich–Rubinstein variational dual (Thm. 1) that mirrors the entropic formula term for term, is convex and well-posed (the transport analogue of the guarantee of Ganguly et al., 2025), and has a closed “mean-plus-Lipschitz” form. • We show it is coherent and place it in the risk hierarchy (Thm. 2): it is sandwiched against the entropic measure by a transport–entropy inequality and strictly accounts for zero-nominal-probability catastrophes the entropic measure ignores. • We verify both robust-optimization problems, their dualities, and the comparison numerically with Gurobi (§4), confirming strong duality to solver tolerance. • Driving the transport radius by belief entropy gives a closed-form robust dynamic-programming operator (Thm. 3) and a computable safety switch under evolving belief (§5).

2

THE ENTROPIC VALUE-AT-RISK AND ITS ROBUST FORMULATION

Let X be a bounded loss on a finite space S with reference law P , and let d : S × S → R≥0 be a ground metric with diameter D = maxs,s′ d(s, s′ ). A risk measure is coherent [Artzner et al., 1999] if it is monotone, translation-invariant, positively homogeneous and subadditive; the conditional value-at-risk [Rockafellar and Uryasev, 2000] and the entropic value-at-risk are its canonical instances. The latter has the dual/primal pair
EVaRα (X) = inf 1t ln EP [etX ] − ln α (1) t>0  = sup EQ [X] : KL(Q∥P ) ≤ − ln α , (2)

where the inverse temperature t in (1) is the multiplier on the relative-entropy constraint in (2), and the worst-case ⋆ law is the exponential tilt Q⋆s ∝ Ps et Xs [Ahmadi-Javid, 2012, Ganguly et al., 2025]. Two facts we carry forward from Ganguly et al. [2025]: (G1) the reparametrized objective is convex, so (1) is a well-posed one-dimensional convex program with a unique solution; (G2) the tilt Q⋆ is supported on supp(P ). Property (G2) is the blind spot: EVaRα (X) is independent of the value of X on any zeronominal-probability state, for every α.

3

SWAPPING THE BALL: A WASSERSTEIN ROBUST RISK

We keep the robust template (2) and replace the relativeentropy ball P by a 1-Wasserstein ball, W1 (Q, P ) = minγ∈Γ(Q,P ) s,s′ d(s, s′ )γ(s, s′ ). Definition 1 (WEVaR). For radius ε ≥ 0,  WEVaRε (X) = sup EQ [X] : W1 (Q, P ) ≤ ε .

(3)

Theorem 1 (Variational dual, convexity, closed form). For bounded X and ε ≥ 0,  WEVaRε (X) = inf λε + EP [ Xλc ] , (4) λ≥0

where Xλc (s) = maxs′ (X(s′ ) − λ d(s′ , s)) is the ctransform of X. The objective is convex in λ, so (4) is a well-posed one-dimensional convex program (mirroring (G1)); the optimal price satisfies λ⋆ ≤ Lipd (X); ε 7→ WEVaRε (X) is concave and nondecreasing; and WEVaRε (X) = EP [X] + ε Lipd (X)

(5)

for ε below a saturation threshold (in particular, exactly for two-point supports), where Lipd (X) = maxs̸=s′ |X(s) − X(s′ )|/d(s, s′ ). Proof sketch. Strong duality for Wasserstein DRO [Gao and Kleywegt, 2023, Blanchet and Murthy, 2019] gives (4); the c-transform Xλc is the transport counterpart of the cumulant 1t ln EP etX in (1). Convexity in λ holds because Xλc is a pointwise maximum of affine functions of λ, hence convex, and EP and the λε term preserve convexity. For λ ≥ Lipd (X) the maximum is attained at s′ = s, so the bracket equals EP [X] and the objective is EP [X] + λε, minimized at λ = Lipd (X); this gives (5) until transporting all displaceable mass to the maximizer of X saturates the bound, after which the value rises concavely toward maxs X(s). Equations (1) and (4) are the same template—inf over a one-dimensional dual variable of a smoothed expectation plus radius-times-multiplier—with an entropic smoothing for the relative-entropy ball and a transport (inf-convolution)

KL ball (red ) is trapped on the support edge; W1 ball (blue) reaches the Fail vertex

smoothing for the Wasserstein ball. The temperature t and the transport price λ are the same Lagrangian object on two ambiguity geometries.

Fail

Proposition 1 (Coherence). WEVaRε is a coherent risk measure for every ε ≥ 0. Proof sketch. It is the support function of the convex, compact set {Q : W1 (Q, P ) ≤ ε}, hence positively homogeneous and subadditive; P lies in the set, giving monotonicity; and adding a constant to X shifts every EQ [X] equally, giving translation invariance. Theorem 2 (Hierarchy and relation to the entropic measure). On a finite metric space with diameter D: (i) WEVaR0 (X) = EP [X] and WEVaRε (X) ↑ ess sup(X) as ε ↑ D, so WEVaR sweeps the full hierarchy; (ii) (sandwich) EVaRα (X) ≤ WEVaR D√− 1 ln α (X) for all

Pb Goal

Caution

2

α ∈ (0, 1]; (iii) (catastrophe) if P (s⋆ ) = 0 and X(s⋆ ) > EP [X], then EVaRα (X) is independent of X(s⋆ ) for every α, whereas WEVaRε (X) strictly increases in X(s⋆ ) once ε > distd (s⋆ , supp P ). Proof sketch. (i) is immediate p from finiteness. (ii): Pinsker gives total variation ≤ KL/2 and W1 ≤ D · TV, so the relative-entropy ball of radius q − ln α is contained in

the Wasserstein ball of radius D − 12 ln α; take suprema [Bobkov and Götze, 1999, Boucheron et al., 2013]. (iii) is (G2) versus the transport reach: no finite-radius relativeentropy ball contains a Wasserstein ball, and the gap is exactly the reachable-but-zero-probability catastrophes.

4

EMPIRICAL GUARANTEES

We verify both robust programs and their duals with Gurobi 12 on a five-state metric space (X = [0, 2, 4, 8, 100], d(s, s′ ) = |s − s′ |). For the entropic measure we solve the relative-entropy-constrained program (2) (a nonlinear program) and match it against the convex primal (1); for WEVaR we solve the transport linear program (3) and match it against the Kantorovich–Rubinstein dual (4). Strong duality holds to solver tolerance: the entropic primal (1) and the relative-entropy program (2) agree to 1.2 × 10−4 , and the transport linear program agrees with its Kantorovich–Rubinstein dual to 9 × 10−7 . The closed form (5) is exact in the unsaturated regime (here ε ≤ 0.1) and is otherwise upper-bounded by the exact one-dimensional dual, which the solver confirms. Across a sweep of radii both measures rise from the mean toward the worst case, with EVaRα ≤ WEVaR at the Pinsker-matched radius (Thm. 2(ii)). The catastrophe blind spot is decisive: with a zero-nominal-probability disaster whose loss we grow from 50 to 1000, the entropic value-at-risk changes by exactly 0

Figure 1: Why Wasserstein, geometrically. On the next-state simplex at a confident belief—the nominal P̄b places zero mass on FAIL—the relative-entropy ball (red) is trapped on the support edge, blind to the catastrophe, while the 1-Wasserstein ball (blue) reaches the FAIL vertex; the worstcase adversary (×) sits on its boundary. This is Thm. 2(iii) drawn on ∆(S). at fixed confidence, while WEVaR at a fixed radius changes by 807.5—the gap of Thm. 2(iii), shown geometrically in Fig. 1 and quantitatively across radii in Fig. 4 (App. E).

5

EVOLVING UNCERTAINTY: BELIEF ENTROPY AS THE RADIUS

We now let the radius evolve. An environment has a hidden type z ∈ Z drawn once; the agent maintains a Bayesian belief b ∈ ∆(Z) with update ψ,P forms the belief-weighted nominal kernel P̄b (· | s, a) = z b(z)P (· | s, a, z), and sets the transportP radius to the belief entropy, ε(b) = β H(b) with H(b) = − z b(z) ln b(z) and sensitivity β > 0, giving the ambiguity set U(b) = {Q : W1 (Q, P̄b ) ≤ βH(b)}. As observations sharpen b, the entropy H(b) ↓ 0, the ball contracts and drifts toward the nominal (Fig. 2), and via Thm. 1 the agent’s risk attitude slides from worst-case to risk-neutral—a dial driven by epistemic state. Since the controller minimizes value, the relevant functional is the infimal twin of Def. 1, whose closed form is EP̄b [V ] − βH(b) Lipd (V ). Theorem 3 (Closed-form robust update; contraction; safety). For the operator, with discount γ ∈ (0, 1] and belief update b′ = ψ(b, s, a, s′ ),   (TV )(s, b) = max r(s, a) + γ inf EQ V (s′ , b′ ) : a

Q∈U (b)

(i) the inner problem has the closed form of Thm. 1—no coupling linear program—reducing the per-(s, a, b) cost to

(a) Safety sandwich

U(b) contracts as belief sharpens Fail

5

b = 0.97, = 0.20 Fail

V(s0, )

b = 0.50, = 1.04

Pb

0 5

Pb Caut.

Goal

0.0

Caut.

Figure 2: The entropy-modulated ambiguity set U(b) on the next-state simplex. As the belief sharpens the radius ε = βH(b) contracts (1.04 → 0.20) and the nominal P̄b drifts toward the goal; the worst-case adversary (×) loses its reach toward the failure vertex—robustness that vanishes with uncertainty (the geometric content of Thm. 3). O(|S|2 ); (ii) T is a γ-contraction in ∥ · ∥∞ when γ < 1, and a contraction on proper MDPs when γ = 1; either way it has a unique fixed point V ⋆ [Bertsekas and Tsitsiklis, 1996]; ⋆ (iii) V ⋆ obeys a safety sandwich Vwc (s, b) ≤ V ⋆ (s, b) ≤ ⋆ Ez∼b [Vopt (s, z)] between the always-maximally-cautious value and the type-aware oracle, and as H(b) → 0 the ⋆ radius vanishes and V ⋆ (s, b) → Vopt (s, z ⋆ ). Since H(b) ≤ ln |Z|, the worst-case radius is β ln |Z|; requiring one action to remain viable under maximal ignorance gives the calibration β < D/ ln |Z|, the transport analogue of choosing a confidence level.

0.2 0.4 0.6 0.8 1.0 belief b(calm) (sharpens ) (b) Geometric contraction

101 Vk V

Goal

oracle (knows z) V (ours) max-caution floor

zeros high low

100

k

10 1 10 2

0

10

20 30 VI step k

40

Figure 3: Computed guarantees on a five-state corridor with a hidden slip type (γ = 0.9, Gurobi-checked). (a) The adaptive value V ⋆ (s0 , b) stays inside the safety sandwich between the always-maximally-cautious floor Vwc and the type-aware oracle ceiling; the price of robustness shrinks as the belief sharpens. (b) From arbitrary initializations, value iteration with the closed-form operator collapses geometrically onto V ⋆ along the γ k envelope, certifying the bound at every iterate.

Evolving safety switch. On the Ambiguous Bridge (γ = 1), S PRINT reaches the goal (+100) under a benign type but falls to catastrophe (−1000) under an adversarial type, while C RAWL is always safe but costly. With belief b = [α, 1 − α] and d(SG , SF ) = 1, so Lipd (V ) = 1100, Thm. 3(i) gives in closed form Q(S PRINT) = −1+1100(α−βH(α))+ −1000 and Q(C RAWL) = 80 − 1100 min(βH(α), 1), which a Gurobi transport solve reproduces to 10−13 . Equating them (the entropy terms cancel) yields a safety switch independent of β: the agent C RAWLs until α ≥ α⋆ = 1081/1100 ≈ 0.983, then S PRINTs. The threshold is set purely by the reward asymmetry. The same creep-then-cruise switch governs the canyon of §1: the drone hovers until its confidence that the air is calm crosses ≈ 0.63, then cruises (Fig. 6, App. E).

to 4.4 once the belief reaches b(calm) = 0.95, and to zero at identification. Value iteration with the closed-form operator converges from arbitrary initializations at empirical rate 0.90 = γ, and the closed-form inner update matches a Gurobi transport solve at every sampled belief (the underlying duality is verified to solver tolerance in §4), so a valid safety bound is available at every iterate, not only at convergence. The belief-simplex manifolds (Fig. 5) and the safety-vs-efficiency rollouts (Fig. 7) appear in App. E.

Guarantees, illustrated. On a five-state corridor whose forward action slips to a failure state only under a hidden “storm” type (Fig. 3), the safety sandwich of Thm. 3(iii) holds at every belief: the adaptive value never drops below the always-maximally-cautious floor and never exceeds the type-aware oracle. The price of robustness is temporary—the ceiling gap falls from 8.4 at the uniform belief

The entropic and Wasserstein robust risks are the relativeentropy and optimal-transport faces of one idea—a coherent risk whose ambiguity radius is the agent’s epistemic uncertainty—bridged by entropic optimal transport [Cuturi, 2013, Peyré and Cuturi, 2019]; scaling the closed-form operator to continuous spaces via Lipschitz critics and to active information gathering are the natural next steps.

6

DISCUSSION

References Mohammed Amin Abdullah, Hang Ren, Haitham Bou Ammar, Vladimir Milenkovic, Rui Luo, Mingtian Zhang, and Jun Wang. Wasserstein robust reinforcement learning. arXiv preprint arXiv:1907.13196, 2019. Amir Ahmadi-Javid. Entropic value-at-risk: A new coherent risk measure. Journal of Optimization Theory and Applications, 155(3):1105–1123, 2012. Eitan Altman. Constrained Markov Decision Processes. Chapman & Hall/CRC, 1999. Philippe Artzner, Freddy Delbaen, Jean-Marc Eber, and David Heath. Coherent measures of risk. Mathematical Finance, 9(3):203–228, 1999. Dimitri P. Bertsekas and John N. Tsitsiklis. Neuro-Dynamic Programming. Athena Scientific, 1996. Jose Blanchet and Karthyek Murthy. Quantifying distributional model risk via optimal transport. Mathematics of Operations Research, 44(2):565–600, 2019. Sergey G. Bobkov and Friedrich Götze. Exponential integrability and transportation cost related to logarithmic Sobolev inequalities. Journal of Functional Analysis, 163 (1):1–28, 1999. Stéphane Boucheron, Gábor Lugosi, and Pascal Massart. Concentration Inequalities: A Nonasymptotic Theory of Independence. Oxford University Press, 2013. Lukas Brunke, Melissa Greeff, Adam W. Hall, Zhaocong Yuan, Siqi Zhou, Jacopo Panerati, and Angela P. Schoellig. Safe learning in robotics: From learning-based control to safe reinforcement learning. Annual Review of Control, Robotics, and Autonomous Systems, 5:411–444, 2022. Yinlam Chow, Aviv Tamar, Shie Mannor, and Marco Pavone. Risk-sensitive and robust decision-making: A CVaR optimization approach. In Advances in Neural Information Processing Systems (NeurIPS), 2015. Marco Cuturi. Sinkhorn distances: Lightspeed computation of optimal transport. In Advances in Neural Information Processing Systems (NeurIPS), 2013.

Javier García and Fernando Fernández. A comprehensive survey on safe reinforcement learning. Journal of Machine Learning Research, 16:1437–1480, 2015. Mohammad Ghavamzadeh, Shie Mannor, Joelle Pineau, and Aviv Tamar. Bayesian reinforcement learning: A survey. Foundations and Trends in Machine Learning, 8(5–6): 359–483, 2015. Jia Lin Hau, Marek Petrik, and Mohammad Ghavamzadeh. Entropic risk optimization in discounted MDPs. In International Conference on Artificial Intelligence and Statistics (AISTATS), 2023. Ronald A. Howard and James E. Matheson. Risk-sensitive Markov decision processes. Management Science, 18(7): 356–369, 1972. Garud N. Iyengar. Robust dynamic programming. Mathematics of Operations Research, 30(2):257–280, 2005. Leslie Pack Kaelbling, Michael L. Littman, and Anthony R. Cassandra. Planning and acting in partially observable stochastic domains. Artificial Intelligence, 101(1–2):99– 134, 1998. Daniel Kuhn, Peyman Mohajerin Esfahani, Viet Anh Nguyen, and Soroosh Shafieezadeh-Abadeh. Wasserstein distributionally robust optimization: Theory and applications in machine learning. In Operations Research & Management Science in the Age of Analytics (INFORMS TutORials), pages 130–166. INFORMS, 2019. Peyman Mohajerin Esfahani and Daniel Kuhn. Data-driven distributionally robust optimization using the Wasserstein metric: Performance guarantees and tractable reformulations. Mathematical Programming, 171:115–166, 2018. Arnab Nilim and Laurent El Ghaoui. Robust control of Markov decision processes with uncertain transition matrices. Operations Research, 53(5):780–798, 2005. Gabriel Peyré and Marco Cuturi. Computational optimal transport. Foundations and Trends in Machine Learning, 11(5–6):355–607, 2019.

Erick Delage and Shie Mannor. Percentile optimization for Markov decision processes with parameter uncertainty. Operations Research, 58(1):203–213, 2010.

Lerrel Pinto, James Davidson, Rahul Sukthankar, and Abhinav Gupta. Robust adversarial reinforcement learning. In International Conference on Machine Learning (ICML), 2017.

Deep Ganguly, Sarthak Girotra, Sirish Sekhar, and Ajin George Joseph. Risk-seeking reinforcement learning via multi-timescale entropic value-at-risk optimization. Transactions on Machine Learning Research, 2025.

R. Tyrrell Rockafellar and Stanislav Uryasev. Optimization of conditional value-at-risk. Journal of Risk, 2:21–42, 2000.

Rui Gao and Anton J. Kleywegt. Distributionally robust stochastic optimization with Wasserstein distance. Mathematics of Operations Research, 48(2):603–655, 2023.

Andrzej Ruszczyński. Risk-averse dynamic programming for Markov decision processes. Mathematical Programming, 125(2):235–261, 2010.

Yanan Sui, Alkis Gotovos, Joel Burdick, and Andreas Krause. Safe exploration for optimization with Gaussian processes. In International Conference on Machine Learning (ICML), 2015. Cédric Villani. Optimal Transport: Old and New. Springer, 2009. Wolfram Wiesemann, Daniel Kuhn, and Berç Rustem. Robust Markov decision processes. Mathematics of Operations Research, 38(1):153–183, 2013.

APPENDIX: EXTENDED PROOFS ′ Throughout, S is finite with |S| = n; d is a metric on S with diameter s,s′ d(s, s ); P ∈ ∆(S) is the reference P D = max ′ ′ law; X : S → R is a bounded P loss; and W1 (Q, P ) = minγ∈Γ(Q,P ) s,s′ d(s, s ) γ(s, s ) with Γ(Q, P ) the couplings of Q and P . We write EQ [X] = s Q(s)X(s).

A. THEOREM 1 (a) Duality. Since W1 (Q, P ) is itself a minimum over couplings, the worst-case expectation is the value of a single linear program over the coupling γ ≥ 0 whose first marginal is fixed to P : X X X WEVaRε (X) = max γ(s, s′ ) X(s′ ) s.t. γ(s, s′ ) = P (s) ∀s, d(s, s′ ) γ(s, s′ ) ≤ ε, (6) γ≥0

s,s′

s′

s,s′

P P where the perturbed law is the second marginal Q(s′ ) = s γ(s, s′ ) (automatically in ∆(S) because s,s′ γ = s P (s) = 1), and the objective equals EQ [X]. The program is feasible (take γ(s, s′ ) = P (s)1[s′ = s], of transport cost 0 ≤ ε) and bounded (X is bounded), so LP strong duality applies. Attaching a free multiplier u(s) to each marginal equality and λ ≥ 0 to the budget, the Lagrangian is X X
P (s)u(s) + λε + γ(s, s′ ) X(s′ ) − u(s) − λ d(s, s′ ) . P

s,s′

s

Its supremum over γ ≥ 0 is finite iff X(s′ ) − u(s) − λ d(s, s′ ) ≤ 0 for all s, s′ , i.e. u(s) ≥ maxs′ X(s′ ) − λ d(s, s′ ) = P Xλc (s), in which case the supremum equals s P (s)u(s) + λε (attained at γ = 0). Minimizing over feasible u sets u(s) = Xλc (s), giving n o  P WEVaRε (X) = min λε + s P (s) Xλc (s) = inf λ≥0 λε + EP [Xλc ] , λ≥0

which is (4). (For general Polish spaces this is Kantorovich–Rubinstein/Wasserstein-DRO strong duality, Villani, 2009, Gao and Kleywegt, 2023, Blanchet and Murthy, 2019.)
(b) Convexity and λ⋆ ≤ Lipd (X). For each fixed s, λ 7→ XλcP (s) = maxs′ X(s′ ) − λ d(s, s′ ) is a pointwise maximum of functions affine in λ, hence convex; therefore EP [Xλc ] = s P (s)Xλc (s) is convex (nonnegative combination), and ϕ(λ) := λε + EP [Xλc ] is convex on [0, ∞), so (4) is a well-posed one-dimensional convex program. Next, if λ ≥ Lipd (X) then for every s, s′ , X(s′ ) − X(s) ≤ Lipd (X) d(s, s′ ) ≤ λ d(s, s′ ), so X(s′ ) − λ d(s, s′ ) ≤ X(s) with equality at s′ = s; hence Xλc (s) = X(s) and ϕ(λ) = λε + EP [X], which is strictly increasing in λ. A convex ϕ that is increasing on [Lipd (X), ∞) attains its minimum at some λ⋆ ≤ Lipd (X). (c) Monotonicity and concavity in ε. The feasible set {Q : W1 (Q, P ) ≤ ε} grows with ε, so WEVaRε (X) is nondecreasing. By (4) it is an infimum over λ of the maps ε 7→ λε + EP [Xλc ], each affine in ε; an infimum of affine functions is concave.
(d) Closed form Let T (s) ∈ arg maxs′ X(s′ ) − Lipd (X) d(s, s′ ) be a steepest-ascent target and set  and saturation. 
P ε̄(X) := P EP d(s, T (s)) = s P (s) d s, T (s) . The right derivative of the convex function EP [Xλc ] at λ = Lipd (X) equals − s P (s) d(s, T (s)) = −ε̄(X) (each active term contributes the negative source–target distance, inactive terms contribute 0). Hence the left derivative of ϕ at Lipd (X) is ε − ε̄(X), which is ≤ 0 iff ε ≤ ε̄(X); in that regime the minimizer is λ⋆ = Lipd (X) and c WEVaRε (X) = Lipd (X) ε + EP [XLip ] = EP [X] + ε Lipd (X). d (X)

For |supp(P ) ∪ {T (s)}| = 2 there is a single transport channel, ε̄(X) = W1 (δarg max X , ·) exhausts only at a vertex, and the formula is exact up to that point. □ B. PROPOSITION 1 (COHERENCE) Write ρ(X) = WEVaRε (X) = supQ∈B EQ [X] with B = {Q : W1 (Q, P ) ≤ ε}, a nonempty (P ∈ B), convex, compact subset of ∆(S). Monotonicity: if X ≤ Y pointwise then EQ [X] ≤ EQ [Y ] for every Q ∈ B (as Q ≥ 0), so

ρ(X) ≤ ρ(Y ). Translation invariance: for c ∈ R, EQ [X + c] = EQ [X] + c for all Q, hence ρ(X + c) = ρ(X) + c. Positive
homogeneity: for t ≥ 0, ρ(tX) = supQ∈B t EQ [X] = t ρ(X). Subadditivity: ρ(X + Y ) = supQ∈B EQ [X] + EQ [Y ] ≤ supQ∈B EQ [X] + supQ∈B EQ [Y ] = ρ(X) + ρ(Y ). These are the axioms of Artzner et al. [1999]; B is the risk envelope of the induced dual representation. □ C. THEOREM 2 (i) Sweep from mean to worst case. W1 is a metric on ∆(S), so W1 (Q, P ) = 0 ⇐⇒ Q = P and WEVaR0 (X) = • E ε (X) ≤ maxs X(s). Let s ∈ arg maxs X(s). Then W1 (δs• , P ) = PP [X]. For any•Q, EQ [X] ≤ maxs X(s), so WEVaR P • • s P (s) d(s, s ) ≤ D, so δs ∈ B once ε ≥ s P (s)d(s, s ), whence WEVaRε (X) = maxs X(s). Combined with monotonicity (Thm. 1(c)), WEVaRε (X) ↑ maxs X(s) as ε ↑ D. P Lemma (transport vs. total variation). For all Q, P , W1 (Q, P ) ≤ D · TV(Q, P ), where TV(Q, P ) = 12 s |Q(s) − P (s)|. Proof: P let m(s) = min(Q(s), P (s)); the coupling that keeps mass m in place (γ(s, s) ≥ m(s)) and transports the residual mass s (Q(s) − m(s)) = TV(Q, P ) arbitrarily incurs cost ≤ D · TV(Q, P ), and W1 is the minimal cost. ⋄ (ii) Sandwich. Fix α ∈ (0, 1] and put ρ = − ln α. For any Q with KL(Q∥P ) ≤ qρ, Pinsker’s inequality gives TV(Q, P ) ≤ p p p KL(Q∥P )/2 ≤ ρ/2, so by the Lemma W1 (Q, P ) ≤ D ρ/2 = D − 12 ln α. Therefore {Q : KL(Q∥P ) ≤ q − ln α} ⊆ {Q : W1 (Q, P ) ≤ D − 12 ln α}, and taking the supremum of EQ [X] over the larger set, EVaRα (X) =

sup

EQ [X] ≤

KL(Q∥P )≤− ln α

sup√

W1 (Q,P )≤D

EQ [X] = WEVaR D√− 1 ln α (X). 2

− 12 ln α

(iii) Catastrophe domination. Assume P (s⋆ ) = 0 and X(s⋆ ) > EP [X]. Entropic measure is blind. EP [etX ] = P tX(s) omits the term s⋆ , so it—and hence EVaRα (X) = inf t>0 1t (ln EP [etX ] − ln α)—does not depend s:P (s)>0 P (s)e ⋆ on X(s ), for every α. (Dually, any Q with KL(Q∥P ) < ∞ satisfies Q ≪ P , so Q(s⋆ ) = 0 and EQ [X] ignores X(s⋆ ).) ⋆ ⋆ Transport measure reacts.
Let δ = distd (s , supp P ) = mins:P (s)>0 d(s, s ) and s0 an achieving state. For ε > δ choose η0 = min P (s0 ), ε/δ > 0 and the feasible law Qη0 = P − η0 δs0 + η0 δs⋆ , for which W1 (Qη0 , P ) ≤ η0 δ ≤ ε. Then
WEVaRε (X) ≥ EQη0 [X] = EP [X] + η0 X(s⋆ ) − X(s0 ) , P which is strictly increasing in X(s⋆ ). Moreover WEVaRε (X) = maxQ∈B s Q(s)X(s) is convex in the vector X, and by Danskin’s theorem its partial right-derivative in X(s⋆ ) equals max{Q(s⋆ ) : Q optimal}; for X(s⋆ ) large enough any optimal Q must place positive mass on s⋆ (else Qη0 strictly improves), so the derivative is positive and WEVaRε (X) strictly increases in X(s⋆ ) whenever ε > δ. □ D. THEOREM 3
Fix (s, a, b) and write the continuation vector W (s′ ) = V s′ , ψ(b, s, a, s′ ) and radius ε(b) = βH(b), nominal P̄b . (i) Closed form of the inner minimization. Applying Theorem 1 to the loss −W and using (−W )cλ (s) = maxs′ (−W (s′ ) − λd(s′ , s)) = − mins′ (W (s′ ) + λd(s′ , s)), n o  
W (s′ ) + λ d(s, s′ ) , inf EQ [W ] = −WEVaRε(b) (−W ) = sup EP̄b Wλc,− − λ ε(b) , Wλc,− (s) = min ′ Q∈U (b)

λ≥0

s

a one-dimensional concave maximization, with first-order/unsaturated value EP̄b [W ] − βH(b) Lipd (W ) (Thm. 1(d) applied to −W ). Evaluating Lipd (W ) costs O(n2 ) and the dual is a scalar program, versus an O(n2 )-variable transportation LP (6). (ii) Contraction. We use two properties of T. Monotonicity: if V ≤ V ′ then W ≤ W ′ pointwise, so inf Q EQ [W ] ≤ inf Q EQ [W ′ ] and, taking maxa , TV ≤ TV ′ . Scaled constant shift: for c ∈ R, adding c to V adds c to every continuation, and inf Q EQ [W + c] = inf Q EQ [W ] + c, so T(V + c1) = TV + γc1. For γ < 1, monotonicity and the scaled shift are exactly the hypotheses of the contraction lemma [Bertsekas and Tsitsiklis, 1996, Ch. 2]: TV − TV ′ ≤ γ∥V − V ′ ∥∞ 1 and symmetrically, so ∥TV − TV ′ ∥∞ ≤ γ∥V − V ′ ∥∞ , a γ-contraction. For γ = 1 the shift is exact and gives nonexpansiveness;

under properness—every policy together with every kernel selection in the sets U(·) reaches the terminal set T in uniformly bounded expected time—the standard stochastic-shortest-path argument [Bertsekas and Tsitsiklis, 1996, Ch. 3] upgrades this to a contraction in a weighted supremum norm ∥ · ∥w : there exist m ≥ 1, κ ∈ [0, 1) with ∥Tm V − Tm V ′ ∥w ≤ κ∥V − V ′ ∥w . Either way T has a unique fixed point V ⋆ . (iii) Safety sandwich. Upper bound. Since P̄b ∈ U(b), inf PQ∈U (b) EQ [W ] ≤ EP̄b [W ], so TV ≤ TBayes V pointwise, where TBayes is the non-robust operator with kernel P̄b = z b(z)P (· | s, a, z); by monotone iteration V ⋆ ≤ VBayes . ⋆ Committing to one policy under belief b cannot beat knowing the type, so VBayes (s, b) ≤ Ez∼b [Vopt (s, z)] (nonnegative ⋆ value of information), giving the upper bound. Lower bound. Let Vwc be the fixed point of the operator Twc obtained by replacing the radius βH(b) with its maximum β ln |Z| at every (s, b), keeping the same nominal P̄b . Since βH(b) ≤ β ln |Z|, the ambiguity ball in T is contained in that of Twc , so its inner infimum is no smaller; hence TV ≥ Twc V pointwise ⋆ and, by monotone iteration, V ⋆ (s, b) ≥ Vwc (s, b). This is the bound used in Fig. 3 and requires no nested-set condition. ⋆ Convergence. As b → δz , H(b) → 0 and ε(b) → 0, so U(b) → {P (· | s, a, z ⋆ )}; by the Lipschitz dependence of the value ⋆ on the radius (Thm. 1(c): 0 ≤ WEVaRε − WEVaR0 ≤ εLipd ), V ⋆ (s, b) → Vopt (s, z ⋆ ). □ E. SUPPLEMENTARY EXPERIMENTS AND FIGURES All figures are reproduced by the released code (a Colab-ready package). The body shows the geometric core (Figs. 1, 2) and the certified guarantee (Fig. 3); the remaining plots are collected here.

100

(a) Hierarchy & sandwich

80 60 40 20

EVaR (KL ball) WEVaRD /2 (W ball) P[X] ess supX

risk (worst-case loss)

risk (worst-case loss)

Quantitative verification of the dualities. Figure 4 is the solver-side companion to §4: across a sweep of radii the entropic and Wasserstein measures both rise from the mean to the worst case, with EVaRα ≤ WEVaR at the Pinsker-matched radius (Thm. 2(ii)); and a zero-nominal-probability disaster leaves the entropic measure flat while WEVaR reacts (Thm. 2(iii)).

0.00 0.25 0.50 0.75 1.00 1.25 KL radius = ln

(b) Zero-prob. catastrophe

200

WEVaR (W ball) EVaR (KL ball, any )

150 100

disaster becomes reachable

50 0

0

1 2 3 Wasserstein radius

4

Figure 4: Gurobi-verified comparison on a five-state metric space. (a) hierarchy and the Pinsker sandwich; (b) the zeroprobability catastrophe: the entropic value-at-risk is invariant to the disaster’s magnitude, WEVaR is not. Manifolds over the belief simplex. For a three-type environment the belief lives on a 2-simplex, and the controller reads three scalar fields off it (Fig. 5): the radius ε(b) = βH(b) (an entropy bowl, maximal at the centroid and zero at the vertices), the robust value V ⋆ (s0 , b), and the policy regions. Safety switches. The belief-dependent policy flips from the safe to the optimal action at a computable threshold (Fig. 6): on the Ambiguous Bridge at α⋆ ≈ 0.983 (set by the reward asymmetry, independent of β), and on the stormy-drone canyon at b(calm) ≈ 0.63. Safety-vs-efficiency rollouts, and an honest caveat. Figure 7 compares the three planners on the canyon. WEVaR Pareto-dominates the static-robust (M AXIMIN) planner—comparable safety at higher return, the “un-freezing” effect. In this calibrated-belief setting the expected-value (Bayesian) planner is already near-optimally cautious under a severe catastrophe, so the empirical separation between WEVaR and Bayesian is small; the regime in which belief-scaled caution measurably reduces failures is one of miscalibrated belief or distribution shift, where the belief is learned rather than computed from a known model. The body’s contribution is therefore the coherent measure, its geometry, and the certified guarantee; this rollout is illustrative.

Manifolds over the belief simplex (Z) storm

calm

policy region (blue=cruise, orange=creep)

robust value V (s0, b)

radius (b) = H(b)

1.12 0.96 0.80 0.64 0.48 0.32 0.16 0.00

breezy

storm

storm

calm

breezy

15 10 5 0 5 10 15 20

calm

breezy

Figure 5: Manifolds over the belief simplex ∆(Z), |Z| = 3 (calm/breezy/storm): ambiguity radius, robust value, and policy region (cruise only near the calm vertex).

0 250 500 750 1000

safety switch

robust action value

robust action value

Stormy drone: evolving safety switch

0.983

Qrob(Sprint) Qrob(Crawl)

0.00 0.25 0.50 0.75 1.00 belief = b(zben) (sharpens )

15 10 5 0 5

Q(Cruise) Q(Creep)

0.0

0.2 0.4 0.6 0.8 1.0 belief b(calm) (sharpens )

Figure 6: Robust action values vs. belief. Left: the Ambiguous Bridge (S PRINT vs. C RAWL, switch at α⋆ ≈ 0.983). Right: the stormy-drone canyon (C RUISE vs. C REEP, switch at b(calm) ≈ 0.63).

catastrophe rate

0.025

Stormy drone: safety vs. efficiency wevar maximin

0.020

wevar (ret 15.2) maximin (ret 15.2) bayesian (ret 16.0)

0.015 0.010 0.005 0.000

bayesian

3.6

3.7 3.8 3.9 4.0 time-to-goal (steps)

Figure 7: Catastrophe rate vs. time-to-goal on the stormy-drone canyon (4000 episodes). WEVaR dominates the static-robust M AXIMIN baseline; the Bayesian planner is already cautious here (see caveat).

Related documents

• Bayesian Confidence Recalibration and Research-Equilibrium Criticality: Temporal Support in Robust Portfolios#926632
• The Variance Preservation Premium: Why Independent Sources of Novelty Can Increase Resilience Under Model Uncertainty#687756
• An Entropic Factor Model for Robust Portfolio Replication#926461
• Knowledge Under Threat#702206
• Evolutionarily Optimal Risk Aversion.#122995