Chance-constrained selection of sequential intervention strategies from counterfactual estimates

Chance-constrained selection of sequential intervention strategies from counterfactual estimates Minkyoung Kima , Beakcheol Janga,∗ a Graduate School of Information, Yonsei University, 50 Yonsei-ro, Seodaemun-gu, Seoul, 03722, Republic of Korea

ABSTRACT

Keywords: Analytics Prescriptive analytics Causal inference Dynamic treatment regimes Chance-constrained programming

Many operational decisions are sequences of interventions under a cumulative resource limit, such as a maintenance schedule within a crew-hour budget. Choosing among them calls for the outcome and the cumulative cost each would produce, counterfactual quantities identified from observational data. Two strategies with the same expected cost can exceed the budget at very different rates, so constraining the mean does not bound how often an overrun occurs. Prior two-step architectures, recently extended to continuous doses, constrain the mean cost rather than its tail and allocate at a single decision point. Methods that do bound a cost tail take its distribution from a specified model rather than identifying it from data. We present a predictthen-optimize framework. In the prediction step, any estimator returning an outcome value and a cost distribution supplies what the decision rule consumes, so the predictor is interchangeable. In the optimization step, a chance-constrained selection over a finite candidate set bounds the probability that the cumulative cost exceeds the budget. That tail does not decompose across stages, so each strategy is scored whole. Sweeping the tolerated violation probability traces a safety-utility frontier, and distribution-free finite-sample bounds cover violation and outcome shortfall. Four of five environments, spanning clinical treatment and equipment maintenance, supply exact counterfactual ground truth; the fifth carries real outcomes from a digital-health micro-randomized trial. Across them, the rule holds the budget where a point-estimate rule overruns it, at an outcome cost the frontier makes explicit. All code is available at https: //github.com/mfriendly/counterfactual-chance-selection.

arXiv:2608.13209v1 [stat.ME] 13 Aug 2026

ARTICLE INFO

1. Introduction

Many operational decisions are not single choices but sequences of interventions carried out under a cumulative resource budget. A maintenance planner schedules overhauls across an equipment fleet, a firm times retention offers to its customers, and a clinician administers treatments to a patient over a stay. In each setting, the cost of a strategy is a random variable, so a constraint on its mean leaves the probability of an overrun uncontrolled. Past interventions were chosen in response to the state, and the cost is observed only for the strategy actually followed. Choosing among the rest turns on counterfactual quantities identified from observational data. Table 1 sets out these settings and the form the budget takes in each. In each case, the resource a strategy consumes accumulates over the horizon. That resource need not be monetary. It may be a physical quantity such as crew-hours, or a cumulative exposure such as a lifetime dose limit. The interventions a strategy administers depend on the trajectory the unit follows under it. Two strategies with the same expected cost can therefore exceed the budget at very different rates. The experiments show this directly on a set of strategies whose mean costs all lie inside the budget. Several of these strategies place more than a fifth of their cost mass above the budget. The operational question is which strategy maximizes the outcome while holding the probability of an overrun within a tolerated level. Each intervention changes the state that determines the next one. A criterion on the tail, accumulation over a horizon, and identification from observational data all follow from that single fact rather than from three independent modeling choices. Budget-constrained prescriptive analytics has posed this decision at a single point in time. De Vos et al. (2026) allocate continuous treatment doses under a budget on total treatment cost, separating a prediction step from an optimization step. They name the extension to sequential decisions and the relaxation of their deterministic-cost ∗ Corresponding author

minky@yonsei.ac.kr (M. Kim); bjang@yonsei.ac.kr (B. Jang)

ORCID (s): 0009-0001-3530-0885 (M. Kim); 0000-0002-3911-5935 (B. Jang)

M. Kim and B. Jang: Preprint submitted to Elsevier

Page 1 of 43

Chance-Constrained Selection from Counterfactual Estimates Table 1 Sequential intervention settings in which a cumulative treatment cost is constrained by a budget. Setting

Outcome

Intervention

Cumulative cost

Source of spread

Maintenance Retention Clinical Digital health

Availability Customer retained Survival Engagement

Overhaul Offer Treatment Notification

Crew-hours Offer spend Cumulative exposure Notifications sent

Time-to-failure heterogeneity Response heterogeneity Stochastic patient trajectory Randomization over the horizon

Three of these settings are instantiated in Section 5.

assumption as the directions their framework leaves open. Both describe the setting above, in which the horizon is multi-stage and the cost accumulated over it is random. Two further lines of work address parts of this setting, and neither covers all of it. One takes its cost distribution from a known model; the other constrains the mean rather than the tail. Sequential decision making under a resource constraint has a risk-neutral form and a risk-sensitive form. The risk-neutral form is the constrained Markov decision process, which bounds the expected cumulative cost of a policy (Altman, 1999). The risk-sensitive form replaces the expected cost with a chance constraint or a conditional-value-at-risk constraint (Chow et al., 2018). That work assumes an online or simulator-based regime in which the transition dynamics are known or can be sampled. Here the cost distribution must instead be identified from observational trajectories in which treatment is confounded with prognosis. Policy learning under a resource constraint couples estimation with a downstream budgeted decision (Sakaguchi, 2025; Athey and Wager, 2021; Liu et al., 2024a). Its budgeted instances, however, are either single-stage or, when sequential, constrain the expected cost rather than its tail. In the taxonomy of the contextual-optimization survey of Sadana et al. (2025), that literature is organized around an uncertain objective evaluated under risk neutrality. The decision here places the uncertainty in the constraint and the criterion on the tail, both of which the survey identifies as open. That leaves a cumulative-budget chance constraint on the tail of the realized cumulative cost, in a sequential setting with costs identified counterfactually. The expectation of a cumulative cost decomposes across stages while its tail probability does not, so the stage-wise machinery these methods build on does not carry over. The constraint has to be read off the full-horizon cost distribution of a strategy taken whole. Such an evaluation is possible strategy by strategy but not over a parameterized policy class, so the decision is restricted to a finite set of candidates. That restriction is also what admits a guarantee holding across the set at once, at a price growing only with its logarithm. Such a decision requires two estimates per candidate strategy, the outcome and the cumulative-cost distribution, and counterfactual prediction under time-varying treatments supplies both (Lim et al., 2018; Bica et al., 2020; Li et al., 2021; Melnychuk et al., 2022; Xiong et al., 2024; Wu et al., 2024). These models predict what each strategy would do and leave the choice among strategies to the analyst. The chance-constraint rule selects the outcome-maximizing strategy subject to a constraint on the probability that the cumulative cost exceeds the budget. It consumes one outcome value and one cost distribution per candidate strategy and nothing else. Any estimator returning those two quantities can therefore supply them, and either step can be replaced without touching the other. In the taxonomy of Sadana et al. (2025), this predict-then-optimize architecture is sequential learning and optimization, estimating first and optimizing after, rather than decision rule optimization or integrated learning and optimization. The constraint sits on the resource axis rather than the outcome. Imposing risk aversion on the outcome is degenerate when the outcome is a binary terminal event such as survival. By contrast, the probability of a budget overrun is a quantity an operator can state in advance and be held to. Sweeping the tolerated violation probability traces a safety-utility frontier, and the decision maker selects an operating point on it by setting how much overrun is acceptable. This paper makes the following advances. • We formulate the selection of a sequential treatment strategy under a cumulative budget as a chance-constrained decision problem over counterfactual outcome and cost distributions. To the best of our knowledge no established method constrains the upper tail of a cost that accumulates over a horizon and is identified counterfactually from observational data.

M. Kim and B. Jang: Preprint submitted to Elsevier

Page 2 of 43

Chance-Constrained Selection from Counterfactual Estimates

• We show that the tail probability of a cumulative cost does not decompose across stages (Proposition 1). The rule therefore scores each candidate strategy on its full-horizon cost distribution and consumes one outcome value and one cost distribution per strategy. It traces a safety-utility frontier as the tolerated violation is swept. • We establish two distribution-free finite-sample guarantees under independent cost and value samples. The first bounds the overrun probability of every admitted strategy. The second bounds each admitted strategy’s outcome shortfall against the best strategy feasible at a tightened tolerance. The same estimation gap equates the rule with its distributionally robust counterpart, and both bounds grow with the candidate set only through its logarithm. • We evaluate by decision quality in five environments drawn from the settings of Table 1. Four provide exact counterfactual ground truth, and the fifth provides real outcomes from a micro-randomized trial. We characterize how the decision degrades as predictor error grows and how large a cost underestimate the violation guarantee withstands. The remainder of the paper is organized as follows. Section 2 places the problem against the lines of work that bear on it. Section 3 formalizes the decision problem and Section 4 develops the chance-constrained selection rule and its guarantees. Section 5 describes the experimental design and Section 6 reports the results. Section 7 concludes and states the limitations and the directions they indicate.

2. Sequential Intervention under a Cumulative Budget 2.1. The problem and the two axes In a budget-constrained sequential intervention problem, interventions are taken over a horizon and interact, so an action taken now changes the state the next decision must respond to. The resource each strategy consumes accumulates over that horizon into a random total, and an outcome is to be maximized subject to a ceiling on that total. Two axes organize the work that bears on such a problem. The first axis is the functional of the cumulative cost the risk criterion constrains, its mean or its upper tail. The second is whether the distribution that functional is taken over is supplied by a known model or identified counterfactually from observational trajectories. Table 2 places the closest work along both axes. Relevant work comes from both sides of the prediction-to-decision pipeline and from operations research. Counterfactual prediction under time-varying treatments supplies the estimates, and methods that turn such predictions into a chosen strategy consume them. A budgeted decision has been posed before in operations research, as a risk constraint over a sequential decision process and as an allocation of a limited intervention budget. Each is reviewed in turn, and the axes return at the close of the section to account for the cell no method occupies.

2.2. Counterfactual prediction and the step to decision A mature line of work estimates the counterfactual trajectories that would follow a given sequence of treatments. Recurrent marginal structural networks (Lim et al., 2018), the counterfactual recurrent network (Bica et al., 2020), G-Net (Li et al., 2021), and the Causal Transformer (Melnychuk et al., 2022) established neural g-computation and balancing as the standard tools for this setting. The G-Transformer (Xiong et al., 2024) extended g-computation to dynamic treatment regimes with a transformer architecture. These methods supply the prediction step. A g-computation model of this family can supply, for each candidate strategy, not a point prediction but the full distribution of survival and of cumulative treatment. Where this literature reports forecast accuracy, the method treats accurate forecasting as a prerequisite and shifts the evaluation to decision quality. Several recent methods move from prediction toward decision but stop short of selecting a strategy under a resource constraint. The generative approach of Wu et al. (2024) produces high-dimensional counterfactual outcome distributions for time-varying treatments and positions itself as a tool for decision-making. It leaves the choice among treatment sequences to the analyst. DistDeD (Killian et al., 2023) learns return distributions with implicit quantile networks and uses a conditional-value-at-risk signal to flag dead-end states. The framework is an early-warning tool and does not learn a policy. Our optimization step requires the predictor to expose a distribution instead of a mean, and places that distribution inside an explicit budget-constrained selection rule. The AI Clinician (Komorowski et al., 2018) learns and evaluates sequential treatment policies but relies on off-policy estimation rather than counterfactual ground truth. That reliance is a source of evaluation difficulty, and the field has developed off-policy estimators to mitigate it (Dudík et al., 2011; Thomas et al., 2015; Thomas and Brunskill, 2016; Kallus and Zhou, 2022). Maximizing an estimated value to obtain a treatment rule is itself well established (Zhao et al., 2012). M. Kim and B. Jang: Preprint submitted to Elsevier

Page 3 of 43

Chance-Constrained Selection from Counterfactual Estimates

The predict-then-optimize line of work spans prescriptive analytics that maps covariates to decisions (Bertsimas and Kallus, 2020) and decision-focused losses that train a predictor for its downstream optimization (Elmachtoub and Grigas, 2022). Sadana et al. (2025) survey these methods. Within that survey’s taxonomy, the uncertainty here sits in the constraint instead of the objective. The criterion is risk-averse on the cost tail instead of risk-neutral on the mean. Both placements are among the directions the survey identifies as open beyond its core of uncertain objectives under risk neutrality. Finite-sample and distributionally robust guarantees for predict-then-optimize have been established for uncertain objectives, through covariate sample-average approximation (Kannan et al., 2025) and bootstrap-robust prescription (Bertsimas and Van Parys, 2022). The guarantee of Section 4 is of the same finite-sample kind but controls the tail of the constraint. Separating estimation from decision follows the distinction between estimating a causal effect well and making a good decision from it (Fernández-Loría and Provost, 2022). The decision-quality claims are anchored to environments in which the counterfactual optimum is computable, so that regret and budget violation can be measured exactly.

2.3. Risk- and chance-constrained sequential decision-making A separate line in operations research constrains risk over a sequential decision process instead of a single choice. The constrained Markov decision process bounds the expected cumulative cost of a policy and is solved through occupation measures and linear programming (Altman, 1999). A risk-sensitive branch replaces the expected cost with a tail functional. Chow et al. (2018) constrain the conditional value at risk of the cumulative cost, or its probability of exceeding a threshold. Bäuerle and Ott (2011) minimize the average value at risk of the discounted cost through state augmentation. Haskell and Jain (2015) minimize the expected cumulative cost of a Markov decision process subject to a chance constraint on that same cost. They reformulate the problem as a linear program over occupation measures. Angelotti et al. (2026) share the offline selection structure of this problem and choose among a fixed set of candidate policies. However, they maximize a risk-aware objective over a Bayesian posterior on the transition model instead of constraining a resource. The risk they control is therefore epistemic and contracts as data accumulate. By contrast, the tail we constrain belongs to the cost distribution itself and persists under exact knowledge of the dynamics. These methods constrain the same upper-tail object we target, and the chance-constrained formulation of Haskell and Jain (2015) is the nearest neighbor on the constraint. The difference is the regime in which the cost distribution is obtained. In this literature, the transition dynamics are known. The cost distribution of a policy is therefore generated by rolling it out in the environment or by solving the model through its occupation measure. Here the cost distribution is instead identified counterfactually from observational trajectories in which treatment is confounded with prognosis, under the assumptions of Section 3.1. The constraint is then enforced in an optimization step that is agnostic to the predictor producing it, not inside a policy-optimization procedure. Approximate and feasibility-oriented treatments of constrained risk-sensitive Markov decision processes also appear in this operational-research setting (Kumar et al., 2023). Closer to the guarantee we establish, Rahimian and Pagnoncelli (2023) study chance-constrained programs with the constraint imposed on a distribution conditional on observed covariates and estimated from data. They establish finite-sample feasibility of the sample problem solved at a tightened risk level. Proposition 3 is of that kind, with the conditioning on an intervention instead of on a covariate. It reaches the shifted level by the same route, a uniform deviation over the candidate set carried by a union bound. The distribution the constraint is evaluated on is therefore counterfactual and rests on the identification assumptions rather than on observed conditioning alone. A separate simulation-optimization line shares the discrete structure of this problem more closely than the policyoptimization one does. Chance-constrained selection of the best chooses, among a finite set of alternatives, the one maximizing a primary performance measure subject to a probabilistic constraint on secondary measures (Hong et al., 2015). Recent procedures achieve a guaranteed probability of correct selection without indifference-zone assumptions (Zhong et al., 2026). That work poses the same discrete decision, maximizing a value over a finite candidate set subject to a probabilistic constraint, and they differ in what is scarce. There the alternatives can be simulated at will and the question is how much simulation effort each requires, so the guarantee concerns the sampling procedure. Here they cannot be run at all: the primary and secondary quantities of every strategy but the one actually followed are counterfactual. The guarantee concerns the constraint evaluated on distributions identified from observational trajectories. The term chance-constrained Markov decision process is also used for a different problem. There, the constraint is placed on uncertain cost or reward parameters rather than on the realized cumulative cost. Tail objectives of this kind recur across operational-research decision problems, from feature-based conditional-value-at-risk minimization in the M. Kim and B. Jang: Preprint submitted to Elsevier

Page 4 of 43

Chance-Constrained Selection from Counterfactual Estimates

newsvendor setting (Liu and Zhu, 2024) to robust utility maximization over chance constraints (Wang et al., 2025). The setting studied here takes the constraint as given rather than developing the optimization theory those methods contribute.

2.4. Prescriptive analytics with budgets Prescriptive analytics turns estimated treatment effects into intervention decisions, and a line of operationalresearch work couples effect estimation with a downstream allocation under a resource limit. De Vos et al. (2026) give the architecture we share. A prediction step estimates dose responses off the shelf, and an optimization step allocates doses subject to a budget on total treatment cost. They evaluate in regret against a full-information optimum computed on known counterfactuals. Their separation of the two steps is deliberate and is defended on the same grounds we adopt. The separation admits pre-trained or substituted predictors and allows constraints to be revised without retraining. Caljon et al. (2026) likewise optimize treatment allocation under a resource limit when treatment of one entity affects the outcomes of others. They select which entities in a network to target under a cardinality budget. Both allocate at a single decision point under a constraint on the total taken across units. Carried to a multi-stage horizon, that architecture meets a cumulative cost that is a random variable. The constraint falls on the upper tail of that cost rather than on its total. A separate line of prescriptive analytics has used heterogeneous treatment effects to allocate a fixed intervention budget across individuals, constraining an aggregate or single-decision budget. The framework of McFowland III et al. (2021) converts estimated heterogeneous treatment effects into individual-level targeting decisions through an integer program and is the closest prior work in spirit. It allocates treatment across individuals under a budget at a single decision point rather than over a sequential horizon. Vanderschueren et al. (2024) allocate limited resources across uncertain tasks under a capacity that is itself random with a known distribution. They absorb that randomness by maximizing the expected payoff under it rather than constraining its tail. They report that a model trained directly on the allocation objective outperforms a two-stage pipeline when capacity is scarce. This result holds at a single decision point where that objective is computable from observed labels. Sun (2026) extends empirical welfare maximization to a budget with a cost that must itself be estimated. The constraint is placed on the population expected cost of an eligibility policy at a single decision point. Under such a constraint, no statistical rule is uniformly welfare-efficient and feasible at once. Where the constraint binds exactly, the welfare loss does not vanish with sample size. The same obstacle governs the benchmark against which Section 4 states its objective bound. Closest on the constrained-decision axis, Athey and Wager (2021) learn treatment-assignment policies from observational data under application-specific constraints, a budget among them, encoded as restrictions on the admissible policy class. They give regret guarantees against the best policy in that class, for a single-decision assignment. A parallel line constrains risk within the dynamic treatment regime itself. Liu et al. (2024b) learn optimal regimes subject to stagewise controls on the risk of adverse events, enforced inside a weighted classification estimator. Their decomposition into single-stage problems rests on an acute-risk condition under which the risk at a stage depends only on the most recent treatment. They identify a cumulative count of costly treatments as a case the condition excludes. Liu et al. (2024a) take the constraint to the horizon total, maximizing the expected reward subject to a ceiling on the expected cumulative risk. A constraint on the total blocks the backward algorithms the stagewise problem admits, since the control depends on future-stage rules still to be estimated. They recover those algorithms by converting the constrained problem through a Lagrange function into a single reweighted outcome, the decision maker stating the ceiling on the expected risk. Laber et al. (2018) bound the expected value of a harm cost defined over the whole trajectory in a dosing regime estimated from observational data. They sweep that bound to present an efficacy-againstrisk curve on which a clinician sets a tolerability limit. All three constraints are placed on an expectation, of a per-stage risk, of the horizon total, and of a trajectory-level maximum. By contrast, the constraint here is placed on the upper tail of the horizon total. In econometrics, Sakaguchi (2025) brings budgets into the sequential setting directly, extending empirical welfare maximization to dynamic treatment regimes under intertemporal budget and capacity constraints. It gives finite-sample distribution-free bounds on both the welfare regret and the excess of the estimated regime’s mean cost over the budget. The constraint there bounds a weighted sum, across stages, of the expected fraction of the population treated at each stage, with stage weights representing relative treatment costs. It therefore acts on population averages. The distribution of the cost an individual accumulates over the horizon does not enter the formulation. The constraint here is placed on the upper tail of that individual cumulative cost. Even with the cost distribution of every strategy known exactly, the cumulative treatment a strategy consumes is a random variable across units. This is because the treatments administered M. Kim and B. Jang: Preprint submitted to Elsevier

Page 5 of 43

Chance-Constrained Selection from Counterfactual Estimates Table 2 Sequential methods with a cumulative cost, by the functional the risk criterion constrains and by where the cost distribution comes from. No entry occupies the lower-right cell. Known model, simulator, or online interaction

Observational trajectories, counterfactual identification

Mean or aggregate resource use

Altman (1999)

Sakaguchi (2025); Liu et al. (2024a)

Upper tail of the cumulative cost

Chow et al. (2018); Bäuerle and Ott (2011); Haskell and Jain (2015)

Budgeted allocation at a single decision point (De Vos et al., 2026; McFowland III et al., 2021; Vanderschueren et al., 2024; Caljon et al., 2026) constrains a total across units, not across time. Liu et al. (2024b) constrain a per-stage adverse-event probability where its companion in the table constrains the horizon total. Chance-constrained ranking and selection (Hong et al., 2015; Zhong et al., 2026) shares the discrete selection form. Its constrained measure, however, is a secondary simulation output, and the resource spent is simulation effort rather than a decision cost.

depend on the trajectory each unit follows. The probability that the realized cost exceeds the budget can therefore be positive, and 𝜀 is the tolerated value of that tail probability. A strategy can hold its expected cost within budget while leaving a heavy tail above it, so the functionals do not coincide even with unlimited data.

The two axes together The two axes of Table 2 are not independent, and the lower-right cell is not reached by moving along either one alone. The expected cumulative cost is additive across stages and, in the Markov-decision-process formulation, linear in the occupation measure of the policy. That allows a mean constraint to be estimated by stage-wise sample means and imposed inside backward-induction or mixed-integer-programming machinery over a parameterized policy class. A tail probability is a functional of the distribution of the full-horizon sum and decomposes in neither way, as Section 3.2 makes precise. It is a static quantile constraint, and static quantile constraints are not time consistent. A choice that is optimal at the start of the horizon need not remain optimal in continuation. The constraint therefore admits no Bellman recursion unless the state is augmented with the cost accumulated so far (Bäuerle and Ott, 2011; Haskell and Jain, 2015). The methods in the left column obtain that augmented recursion from a specified transition model, at the price of an extra continuous dimension. When the cost distribution is instead identified counterfactually, the model that would supply the recursion is missing. The horizon distribution of each strategy therefore has to be evaluated as a whole. The optimization step then scores a finite set of candidate strategies and evaluates the constraint on the distribution of each, rather than optimizing over a parameterized policy class.

3. Problem Formulation Figure 1 summarizes the two steps. The decision problem is formalized first, and Section 4 develops the optimization step that solves it. The setting is any sequential intervention with a scalar outcome to be maximized and a cumulative resource cost to be constrained. Section 5 instantiates it on clinical, operations, and behavioral objectives. Table S1 of the supplementary material collects the notation.

3.1. Problem setup We adopt the sequential potential-outcome framework for time-varying treatments, in the formulation used by the g-computation and balancing literature (Robins, 1986; Robins et al., 2000; Bica et al., 2020; Melnychuk et al., 2022; Li et al., 2021). A unit is observed over discrete time steps 𝑡 ∈ {0, 1, … , 𝐾}. At each step, we observe time-varying covariates 𝐿𝑡 , a treatment decision 𝐴𝑡 , and an outcome that is a component of the next-step covariate vector. The observed history up to step 𝑚 is 𝐻𝑚 = (𝐿̄ 𝑚 , 𝐴̄ 𝑚−1 ), where 𝐿̄ 𝑚 = (𝐿0 , … , 𝐿𝑚 ) and 𝐴̄ 𝑚−1 = (𝐴0 , … , 𝐴𝑚−1 ) denote the covariate and treatment histories. A dynamic treatment strategy 𝑔 = (𝑔𝑚 , … , 𝑔𝐾 ) is a sequence of decision rules, as in the literature on dynamic treatment regimes (Murphy, 2003; Chakraborty and Moodie, 2013). Each rule maps the history available at that step to a treatment, 𝑔𝑡 ∶ 𝐻𝑡 ↦ 𝐴𝑡 . For a strategy 𝑔 we write 𝐿𝑡 (𝑔) for the covariate vector at step 𝑡 that would be observed if treatment were assigned according to 𝑔 from step 𝑚 onward. We write 𝐿̄ >𝑡 (𝑔) for the path the strategy generates after step 𝑡. The outcome under 𝑔 is the corresponding component of 𝐿𝑡 (𝑔). M. Kim and B. Jang: Preprint submitted to Elsevier

Page 6 of 43

Chance-Constrained Selection from Counterfactual Estimates

V(g)

counterfactual prediction

stage m

stage m+1

stages comonotone

Fg

max V(g) subject to Pr(C(g) > B) ≤ ε

g′

g

stage m

stage m+1

stages independent

B

B

outcome

candidate strategies

identical stage marginals

observed trajectories

ε cumulative cost C(g)

One value and one cost distribution per strategy

cumulative cost C(g′)

Pr(C(g) > B) > Pr(C(g′) > B)

(a) Prediction

(b) The cost tail does not decompose across stages

selected strategy

(c) Chance-constrained selection

Figure 1: The two steps and the fixed contract between them, one outcome value and one cost distribution per strategy. The center panel shows two strategies with identical per-stage cost distributions and identical mean cost, and cumulative costs that exceed the budget with different probability.

Identification of the counterfactual quantities below rests on three standard assumptions of the framework. Assumption 1 (Sequential ignorability). At each step, treatment is independent of the future potential covariate path given the observed history, 𝐴𝑡 ⟂ 𝐿̄ >𝑡 (𝑔) ∣ 𝐻𝑡 . Assumption ( 2 (Positivity). Every ) treatment a strategy may assign has positive probability given any history that can occur, Pr 𝐴𝑡 = 𝑔𝑡 (ℎ𝑡 ) ∣ 𝐻𝑡 = ℎ𝑡 > 0 whenever Pr(𝐻𝑡 = ℎ𝑡 ) > 0, so that the counterfactual is supported by data. Assumption 3 (Consistency). On the event that the realized treatment history agrees with what the strategy assigns, the potential covariates equal the observed ones, 𝐿𝑡 (𝑔) = 𝐿𝑡 on {𝐴̄ 𝑡−1 = 𝑎̄𝑡−1 } with 𝑎𝑠 = 𝑔𝑠 (ℎ𝑠 ). These assumptions are the sequential analogues of the ignorability (no unmeasured confounding), overlap, and consistency conditions familiar from cross-sectional treatment-effect estimation. Under them, the g-computation formula identifies the distribution of the covariate path under a strategy from observational quantities (Robins, 1986), 𝐾 ( ) ∏ ( )| 𝑝 𝑙̄𝑚∶𝐾 ∣ 𝑔 = 𝑝 𝑙𝑡 ∣ 𝑙̄𝑡−1 , 𝑎̄𝑡−1 | , |𝑎𝑠 =𝑔𝑠 (ℎ𝑠 )

(1)

𝑡=𝑚

where each factor is a conditional law of the observed data and the treatments are set to those the strategy would assign. S3 of the supplementary material carries the assumptions through to the cost distribution. Both quantities the decision rule consumes are functionals of Equation (1). The outcome value is an expectation under it. The cost distribution is the law of the accumulated treatment along the same path, [𝐾 ] ( ) ∑ ( ) ∑ ( ) 𝐹𝑔 (𝑏) = Pr 𝐶(𝑔) ≤ 𝑏 = 𝑝 𝑙̄𝑚∶𝐾 ∣ 𝑔 𝟙 𝑐 𝑔𝑡 (ℎ𝑡 ) ≤ 𝑏 . (2) 𝑙̄𝑚∶𝐾

𝑡=𝑚

The budget constraint is therefore imposed on a counterfactual object: 𝐹𝑔 is never observed for any strategy other than the one the data-generating policy followed. Equation (2) makes it estimable.

3.2. Counterfactual quantities under a strategy The method scores a candidate strategy by two counterfactual quantities rather than by a single point prediction. The first is the strategy’s outcome value, the probability that the unit reaches the horizon in the favorable terminal state under the strategy, ( ) 𝑉 (𝑔) = Pr favorable terminal state ∣ 𝑔 , (3)

M. Kim and B. Jang: Preprint submitted to Elsevier

Page 7 of 43

Chance-Constrained Selection from Counterfactual Estimates

which is the utility the decision maker wishes to maximize. The second is the strategy’s cumulative treatment cost over the horizon, 𝐶(𝑔) =

𝐾 ∑ ( ) 𝑐 𝐴𝑡 ,

𝐴𝑡 ∼ 𝑔,

(4)

𝑡=𝑚

where 𝑐(⋅) counts the treatments administered at a step, so 𝐶(𝑔) is the cumulative quantity of treatment a strategy consumes. It is a random variable, not a fixed number, because the treatments administered depend on the trajectory the unit follows under the strategy. The method therefore works with the full distribution of 𝐶(𝑔), not only its mean. Which functional of that distribution the budget constrains determines what machinery is available. By linearity of expectation, the mean separates across stages, 𝐾 ] ∑ [ ] 𝔼 𝐶(𝑔) = 𝔼 𝑐(𝐴𝑡 ) ,

[

(5)

𝑡=𝑚

so a constraint on 𝔼[𝐶(𝑔)] is a sum of per-stage terms, each estimable from the visits to stage 𝑡 alone. That permits a mean-budget constraint to be imposed inside backward induction or written as a linear constraint over occupation ∑ measures. The tail probability admits no such separation. Writing 𝑆𝑡 = 𝑡𝑠=𝑚 𝑐(𝐴𝑠 ) for the partial sum, ( ) ( ) Pr 𝐶(𝑔) > 𝐵 = Pr 𝑆𝐾 > 𝐵 = 1 − 𝐹𝑆𝐾 (𝐵),

(6)

a functional of the distribution of the full-horizon sum. It depends on the joint law of (𝑐(𝐴𝑚 ), … , 𝑐(𝐴𝐾 )), not on the per-stage marginals separately. Proposition 1 (Non-decomposability of the cost tail). There exist strategies for which the per-stage cost) distri( butions coincide at every stage while the budget-violation probabilities differ. Consequently Pr 𝐶(𝑔) > 𝐵 is not a functional of the stage-wise marginal laws of 𝑐(𝐴𝑚 ), … , 𝑐(𝐴𝐾 ), and a constraint on it cannot be assembled from per-stage estimates. Equation (5), by contrast, writes the mean as a sum of per-stage terms under every dependence structure. The construction is a two-stage horizon, 𝐾 = 𝑚 + 1, with per-stage cost valued in {0, 1}. Under 𝑔, a single Bernoulli(1∕2) draw 𝑍 sets both stage costs, 𝑐(𝐴𝑚 ) = 𝑐(𝐴𝑚+1 ) = 𝑍, so the stages are comonotone; under 𝑔 ′ they are independent Bernoulli(1∕2) draws. Every stage marginal is Bernoulli(1∕2) under both, so by Equation (5) the mean costs agree at 𝔼[𝐶(𝑔)] = 𝔼[𝐶(𝑔 ′ )] = 1. The totals do not: 𝐶(𝑔) takes the values 0 and 2 with probability 1∕2 each. The total 𝐶(𝑔 ′ ) takes 0, 1, 2 with probabilities 1∕4, 1∕2, 1∕4. At any budget 𝐵 ∈ [1, 2), ( ) Pr 𝐶(𝑔) > 𝐵 = 21 ,

( ) Pr 𝐶(𝑔 ′ ) > 𝐵 = 14 ,

(7)

so two strategies indistinguishable stage by stage, in marginal law and in mean, differ in violation probability by a factor of two. Both are realizable in the framework of Section 3.1. A binary state variable is drawn at the first stage and carried forward in the covariates. The strategy 𝑔 treats at both stages or at neither, according to that variable. The strategy 𝑔 ′ draws its treatment indicator independently at each stage. The gap widens with the horizon: extending the pair to ℎ stages leaves every stage marginal Bernoulli(1∕2). The comonotone strategy violates a budget 𝐵 ∈ [ℎ − 1, ℎ) with probability 1∕2, and the independent one violates it with probability 2−ℎ . The phenomenon is the time inconsistency of a static quantile constraint, known in the risk-sensitive control literature. There it forces the state to be augmented with the cost accumulated so far before a Bellman recursion becomes available (Bäuerle and Ott, 2011). The tail probability depends on the dependence between stages, which leaves the mean unchanged. It is therefore reachable either through that augmented recursion, which needs a transition model to recurse through, or by estimating the law of the full-horizon sum directly. Where the cost distribution is identified counterfactually rather than specified, the second route is the one that remains, and the chance-constraint rule takes it. Both 𝑉 (𝑔) and the distribution of 𝐶(𝑔) are identified under Assumptions 1 to 3, and we compute them by simulating trajectories under 𝑔. They lie on different axes, the first a property of the outcome and the second of the treatment the strategy consumes. A constraint that moved with the outcome would never bind whereas a treatment budget trades off against it and shapes the decision. M. Kim and B. Jang: Preprint submitted to Elsevier

Page 8 of 43

Chance-Constrained Selection from Counterfactual Estimates

4. The Optimization Step: Chance-Constrained Selection The optimization step turns the two counterfactual quantities of Section 3.2 into a recommendation. We state the decision rules and separate them by the functional of the cost distribution each one constrains. An operating point is then selected on the frontier the advocated rule traces. The violation and the outcome shortfall of the selected strategy are bounded in finite samples.

4.1. The decision rules The decision maker seeks the strategy that maximizes the outcome while respecting a treatment budget 𝐵. The choice is over the finite set  of candidate strategies that Section 2 arrives at. Section 5.5 describes how we obtain  in each evaluation environment, where it must be constructed rather than taken as given so that the counterfactual optimum is computable. The rules below differ in their robustness to the fact that, in deployment, the outcome and cost ̂ of a strategy are estimated rather than known. Let 𝑉̂ (𝑔) denote the estimated outcome value, 𝐶(𝑔) the estimated mean cost, and 𝜎̂ 𝐶 (𝑔) the estimated standard deviation (s.d.) of the cost. The first rule acts on the estimated mean cost directly, ∗ 𝑔naive = arg max 𝑉̂ (𝑔) 𝑔

̂ subject to 𝐶(𝑔) ≤ 𝐵.

(8)

This point-estimate rule is the implicit one whenever a strategy is selected by comparing expected costs to a budget. ̂ Its weakness is an instance of the optimizer’s curse (Smith and Winkler, 2006). Even when 𝐶(𝑔) is unbiased for every ̂ strategy separately, restricting attention to those that pass the test 𝐶(𝑔) ≤ 𝐵 selects preferentially among the realizations ̂ ∗ ) − 𝐶(𝑔 ∗ )] < 0, and a strategy with true expected cost above in which the cost was underestimated. Hence 𝔼[𝐶(𝑔 naive naive the budget can appear feasible and be recommended. The second rule guards against this by holding out a margin proportional to the estimated cost uncertainty, ∗ 𝑔ub = arg max 𝑉̂ (𝑔) 𝑔

̂ subject to 𝐶(𝑔) + 𝜅 𝜎̂ 𝐶 (𝑔) ≤ 𝐵,

(9)

where 𝜅 ≥ 0 controls the conservatism of the margin. This rule reduces violations, but 𝜅 carries no units in which a decision maker can reason. A margin of one standard deviation corresponds to no stated tolerance for exceeding the budget, and the value that performs well depends on the cost distributions at hand. The third rule, which is the one we advocate, constrains the probability that the cost exceeds the budget directly, the functional Equations (5) and (6) distinguish, ( ) ∗ = arg max 𝑉̂ (𝑔) subject to Pr 𝐶(𝑔) > 𝐵 ≤ 𝜀, (10) 𝑔cc 𝑔

where 𝜀 ∈ [0, 1] is the violation probability the decision maker is willing to tolerate. The constraint in Equation (10) is a chance constraint in the sense of Charnes and Cooper (1959). Writing 𝐹𝑔 for the distribution function of 𝐶(𝑔), it restricts the budget to lie above the corresponding quantile, ( ) ( ) Pr 𝐶(𝑔) > 𝐵 ≤ 𝜀 ⟺ 𝐹𝑔 (𝐵) ≥ 1 − 𝜀 ⟺ VaR1−𝜀 𝐶(𝑔) ≤ 𝐵, (11) so the rule admits a strategy only if the upper tail of its cost distribution stays within the budget with the required probability. Replacing the value at risk by the conditional value at risk gives a constraint that is not equivalent but safe. Because CVaR1−𝜀 (𝐶) ≥ VaR1−𝜀 (𝐶), every strategy it admits satisfies Equation (10). A strategy with a cost that exceeds the budget rarely but by a wide margin can satisfy Equation (10) and fail the conditional-value-at-risk constraint. That constraint is convex in the cost distribution and is the least conservative of the generator-based safe convex approximations of a chance constraint (Rockafellar and Uryasev, 2000; Nemirovski and Shapiro, 2006). It is evaluated on the sampled distribution as the average of the worst ⌈𝜀𝑛⌉ realizations, ( ) ̂ 1−𝜀 𝐶(𝑔) = CVaR

⌈𝜀𝑛⌉

1 ∑ 𝐶 (𝑔), ⌈𝜀𝑛⌉ 𝑗=1 (𝑛−𝑗+1)

(12)

where 𝐶(1) ≤ ⋯ ≤ 𝐶(𝑛) are the ordered cost samples of 𝑔. As 𝜀 approaches one that average covers the whole sample and the constraint reduces to the mean-budget test of Equation (8). The two rules coincide in that limit. Chance constraints M. Kim and B. Jang: Preprint submitted to Elsevier

Page 9 of 43

Chance-Constrained Selection from Counterfactual Estimates

are in general intractable to optimize, and the standard remedy is a tractable convex approximation (Nemirovski and Shapiro, 2006). Here the cost distribution is available in sampled form from the predictor. The risk measure sits on the constraint, not on the objective, since the lower tail of a binary terminal outcome does not separate strategies. Each rule constrains a different functional of the cost distribution: the mean, the mean plus a margin, the upper tail quantile, and the mean of that tail. The rules therefore differ even where the cost distribution is known exactly. The last two bound the frequency and the severity of the same overrun and Section 6.3 reports how they compare. They would coincide only if the cost were deterministic. The regret the chance-constraint rule incurs under exact counterfactuals is therefore the intrinsic price of bounding the tail instead of the mean, separate from any estimation error. Under estimation, the naive rule is additionally unreliable, admitting strategies with a mean that only appears to satisfy the budget. Sweeping 𝜀 traces the safety-utility frontier, on which it positions the recommendation between admitting overruns and sacrificing outcome. Worst-case and multi-objective alternatives fit the budget less well as it is usually given; S16 sets out both. The rule is evaluated on the sampled cost distribution of each strategy, the sample-average-approximation form of the constraint (Luedtke and Ahmed, 2008; Nemirovski and Shapiro, 2006). For each candidate the predictor supplies an outcome value and 𝑛 cost samples, the empirical tail probability is the fraction of those samples above the budget. The rule returns the highest-outcome strategy with tail probability at most 𝜀, or reports infeasibility when none is; Algorithm S1 in the supplementary material states the procedure. Selection is 𝑂(||𝑛) given the outcome value and the 𝑛 cost samples of each candidate, and a sweep over 𝜀 reuses the same per-strategy tail counts. The entire frontier therefore costs one pass over the sampled costs, and the budget or the tolerated violation can be revised without re-estimating anything.

4.2. Selecting an operating point Equation (10) and the frontier it traces still leave open which operating point the decision maker should adopt. A recommender that returns the whole curve defers that choice instead of supporting it. Under exact counterfactuals, the frontier is an ordered trade-off by construction. Proposition 2 (Frontier monotonicity). Fix the budget 𝐵, and let 𝑉 ∗ (𝜀) = max{𝑉̂ (𝑔) ∶ Pr(𝐶(𝑔) > 𝐵) ≤ 𝜀} be the achievable outcome at tolerated violation 𝜀. Then 𝑉 ∗ is non-decreasing in 𝜀 on [0, 1]. Write (𝜀) = {𝑔 ∈  ∶ Pr(𝐶(𝑔) > 𝐵) ≤ 𝜀} for the admissible set. For 𝜀 ≤ 𝜀′ the defining inequality is weaker on the right, so (𝜀) ⊆ (𝜀′ ) and hence 𝑉 ∗ (𝜀) ≤ 𝑉 ∗ (𝜀′ ), a maximum over a larger set being no smaller. The frontier therefore exchanges tolerated violation for outcome monotonically. This monotonicity makes a target on either axis resolve to a single recommendation. Its curvature is another matter: Section 6.7 shows it is too irregular for a curvature-based knee to be stable or consistent across environments. The operating point is set by operational target instead. Given a maximum tolerated tail probability 𝜌, the method returns the admissible strategy of greatest outcome. Given a minimum acceptable outcome, it returns the admissible strategy with the smallest tail probability. The decision maker therefore states a target on whichever axis is operationally meaningful instead of choosing the constraint level 𝜀 directly. The selection assumes nothing about the shape of the frontier, since it optimizes one objective over the swept strategies subject to a bound on the other. The realized tail probability of a strategy can exceed its nominal tolerance once the outcome and cost are estimated rather than known. The target is then applied to the realized tail probability.

4.3. Finite-sample guarantees The chance-constraint rule of Equation (10) is stated in terms of the true cost distribution but in deployment is applied to an estimate of it. That substitution must not silently inflate the budget violation the rule is meant to control. Relating the violation of a sampled chance constraint to the sample size in this way is standard (Luedtke and Ahmed, 2008). Let 𝐹𝑔 (𝑏) = Pr(𝐶(𝑔) ≤ 𝑏) be the true cumulative distribution of the cost of strategy 𝑔. Let 𝐹̂𝑔 be the empirical distribution formed from 𝑛 independent samples of 𝐶(𝑔). The rule admits 𝑔 when 1 − 𝐹̂𝑔 (𝐵) ≤ 𝜀. Proposition 3 (Feasibility under cost estimation). If sup𝑏 |𝐹̂𝑔 (𝑏) − 𝐹𝑔 (𝑏)| ≤ 𝛿, then every strategy admitted by the empirical chance-constraint rule satisfies ( ) Pr 𝐶(𝑔) > 𝐵 ≤ 𝜀 + 𝛿. (13) When the cost samples are independent, the Dvoretzky-Kiefer-Wolfowitz inequality with the tight constant of Massart 2 (1990) gives Pr(sup𝑏 |𝐹̂𝑔 (𝑏) − 𝐹𝑔 (𝑏)| > 𝛿) ≤ 2𝑒−2𝑛𝛿 for each strategy. A union bound over  carries it to all of them

M. Kim and B. Jang: Preprint submitted to Elsevier

Page 10 of 43

Chance-Constrained Selection from Counterfactual Estimates

at once, ( ) 2 | | Pr ∃ 𝑔 ∈  ∶ sup |𝐹̂𝑔 (𝑏) − 𝐹𝑔 (𝑏)| > 𝛿 ≤ 2||𝑒−2𝑛𝛿 , | | 𝑏

(14)

√ so setting the right-hand side to 𝜂 gives 𝛿 = log(2||∕𝜂)∕(2𝑛) = 𝑂(𝑛−1∕2 ), at which Equation (13) holds with probability at least 1 − 𝜂 simultaneously for every admitted strategy. Evaluating the uniform bound at the budget and chaining it with the admission rule gives 𝐹𝑔 (𝐵) ≥ 𝐹̂𝑔 (𝐵) − 𝛿 ≥ (1 − 𝜀) − 𝛿,

(15)

where the second inequality is the admission condition 𝐹̂𝑔 (𝐵) ≥ 1 − 𝜀; rearranging is Equation (13). Because the control is uniform over the threshold, it certifies the budget evaluation at every 𝐵 at once. The union bound is therefore needed only over the finite strategy set, not over the continuum of budgets. Sections 6.2 and 6.7 sweep both and use this uniformity. The two quantities in Equation (13) are named for use throughout. The certified slack is 𝛿, the uniform deviation the premise allows. The certified ceiling is 𝜀 + 𝛿, the bound the proposition places on the realized tail probability of an admitted strategy. The certified slack is an estimation term that vanishes as the cost distribution is estimated from more data. The finite-sample term in the mean-cost budget bound of Sakaguchi (2025) plays the same role. It is distinct from the tolerated violation 𝜀, which is the nominal tail-risk level fixed in the constraint. The two settings differ in the constrained functional, the tail instead of the mean. Evaluating the constant of Equation (14) in the real-outcome environment of Section 5.4, with 𝑛 = 349 independent participants, || = 17 candidate strategies and 𝜂 = 0.05, gives 𝛿 ≈ 0.10. A nominal tolerance 𝜀 = 0.2 therefore carries a certified ceiling of about 0.30 on the realized tail probability.1 The same translation applies to the conditional-value-at-risk operating point, with a constant that depends on the cost support. The chance-constraint form is stated because it admits the explicit Dvoretzky-Kiefer-Wolfowitz constant. The uniform deviation in Proposition 3 also connects the rule to its distributionally robust counterpart. Constraining the worst case over a Kolmogorov ball of radius 𝑟 around 𝐹̂𝑔 gives the same rule at tolerance 𝜀 − 𝑟 (S16). Taking 𝑟 = 𝛿 recovers Proposition 3 and its certified ceiling of 𝜀 + 𝛿. Proposition 3 controls feasibility but leaves open how close the outcome of the selected strategy is to the budgetfeasible optimum. A bound against 𝑔 ∗ (𝜀), the best strategy truly feasible at the nominal tolerance, cannot hold uniformly. A strategy with cost tail exactly at 𝜀 is excluded by the empirical rule whenever sampling error pushes its estimated tail above the threshold. The empirical feasible set therefore need not contain the true optimum. Accordingly, the comparison is made against a benchmark defined at a tightened tolerance. This parallels the analysis of sampled chance constraints, where solving the sample problem at one risk level certifies a bound relative to a slightly different level (Luedtke and Ahmed, 2008, Theorem 3). The same obstacle appears in the econometric treatment of budgeted policy choice, where no statistical rule attains the highest welfare and satisfies the budget uniformly at once. The welfare loss fails to vanish with sample size exactly where the constraint binds (Sun, 2026). The tightened benchmark is thus forced by the structure of the problem, not adopted for convenience. Write 𝑔 ∗ (𝜀′ ) = arg max{𝑉 (𝑔) ∶ Pr(𝐶(𝑔) > 𝐵) ≤ 𝜀′ } for the best strategy truly feasible at tolerance 𝜀′ , with 𝑉̂ (𝑔) formed as an average over 𝑛 independent units. Proposition 4 (Finite-sample regret at a tightened benchmark). Suppose the per-unit contributions to 𝑉̂ (𝑔) lie in an interval of length 𝑅 for every 𝑔 ∈ . Suppose also that the cost samples and the value samples are each independent across units. Fix 𝜂 ∈ (0, 1) and set √ √ log(4||∕𝜂) log(4||∕𝜂) 𝛿= , 𝜖𝑉 = 𝑅 . (16) 2𝑛 2𝑛 Let 𝑔̂ be the strategy returned by the empirical chance-constraint rule at tolerance 𝜀. Then, with probability at least 1 − 𝜂, simultaneously ( ) ( ) Pr 𝐶(𝑔) ̂ >𝐵 ≤𝜀+𝛿 and 𝑉 𝑔 ∗ (𝜀 − 𝛿) − 𝑉 (𝑔) ̂ ≤ 2𝜖𝑉 . (17) 1 The independent unit is the participant, so Monte Carlo redraws of the treatment realization do not enlarge 𝑛, and a distribution-free bound is conservative at a few hundred units. In the exact-counterfactual environments, the cost distribution comes from simulation, so 𝑛 can be taken large and 𝛿 made negligible. The regret and violation reported there are therefore not subject to this inflation.

M. Kim and B. Jang: Preprint submitted to Elsevier

Page 11 of 43

Chance-Constrained Selection from Counterfactual Estimates

The proof intersects two uniform deviation bounds: a Dvoretzky-Kiefer-Wolfowitz bound on the cost distributions and a Hoeffding bound (Hoeffding, 1963) on the value estimates. Each is taken over  at confidence 𝜂∕2. On that intersection, the feasibility half follows as in Proposition 3, and 𝑔 ∗ (𝜀 − 𝛿) is admitted by the empirical rule. The rule then maximizes 𝑉̂ over a set containing it, and the regret half follows. S1 of the supplementary material gives the argument. The constants carry log(4||∕𝜂) where Proposition 3 carries log(2||∕𝜂) because the confidence budget is split between the two bounds; the numerical check above is unaffected to two decimals. Both terms depend on the candidate set only through log||, so enlarging it is inexpensive at a computational cost Section 6.7 measures. A sampled chance constraint over a finite candidate set exhibits the same logarithmic dependence (Luedtke and Ahmed, 2008, Theorem 5). Comparing against 𝑔 ∗ (𝜀 − 𝛿) rather than 𝑔 ∗ (𝜀) concedes whatever outcome lies between the two tolerances. The concession is small where the frontier is flat near 𝜀, larger where a high-outcome strategy sits just inside it, and shrinks at the rate of the feasibility slack. The constant 𝑅 belongs to the value estimator rather than the decision rule and scales with the largest importance weight for an inverse-probability-weighted one. The certified margin is therefore loose at these sample sizes in the way a distribution-free bound is expected to be. Two benchmarks are in play and measure different things: Equation (17) bounds the shortfall against 𝑔 ∗ (𝜀 − 𝛿), feasible under the same tail constraint at a tightened tolerance. The regret of Section 6 is measured instead against the risk-neutral oracle of Section 5.6. That oracle is constrained on the mean cost and therefore admits strategies the chance constraint excludes. The reported regret then carries the intrinsic price of bounding the tail. Propositions 2 to 4 take the cost distribution to be estimated directly from independent cost samples, as it is in the real-outcome environment, not produced by an estimated outcome predictor. Its error would propagate through the dynamics into the cost tail and be coupled across stages by the sequential structure. They also hold for a finite candidate set. Extending them to a parameterized policy class would therefore require a complexity measure in place of the log|| of Equation (16). Rahimian and Pagnoncelli (2023) carry a finite-set feasibility bound to a compact decision set along those lines. Both extensions are left to future work.

4.4. Predictor and evaluation The decision rule consumes the outcome value and the cost distribution of each strategy but does not prescribe how they are produced. An estimator must supply the distribution of the cumulative cost, not only its mean. A g-computation model estimated from observational data is the natural fit: simulating trajectories under a strategy yields as many draws of Equations (3) and (4) as are wanted. Recurrent and Transformer forms are available (Li et al., 2021; Xiong et al., 2024). Inverse probability weighting also meets the requirement, and the real-outcome environment of Section 5.4 uses it. Weighting estimators, however, recover the upper tail less reliably than the mean. Any estimator honoring the same contract, one outcome value and one cost distribution per strategy, can be substituted without changing the rules. Whether a useful safety-utility trade-off exists, and how a chance constraint traces it, is a property of the decision problem and the environment rather than of any particular estimator. It is therefore established under exact counterfactuals, where the outcome and cost distribution of each strategy can be computed without estimation error (Section 5). The dependence of the decision on predictor accuracy is a separate question. We address it by injecting controlled estimation error into those exact quantities, since committing to one learned predictor would tie the errors to its architecture and training. Because a predictor fit to observational data does not err symmetrically, the injected error includes a systematic cost underestimate as well as mean-zero noise. Section 6.5 reports how large an underestimate the violation guarantee withstands.

5. Experimental Design We use five environments, ordered from fully synthetic to fully real. In four of them, the counterfactual outcome and treatment cost of every strategy can be computed exactly. Table 3 summarizes them by data provenance and by the cost and outcome each one measures, and Table S3 gives their scale. The evaluation measures decision quality against a known optimum rather than the out-of-sample cost that contextual-optimization studies typically report (Sadana et al., 2025). The environments exercise the features that distinguish the problem: a sequential horizon, a cumulative cost with a genuine upper tail, and identification of that cost from observational trajectories. A single-stage benchmark with uncertainty in the objective would test none of them. The environments also vary one axis at a time away from fully controlled dynamics. The simulators fix the dynamics entirely. The semi-synthetic environment replaces the covariates with real ones while keeping the outcome M. Kim and B. Jang: Preprint submitted to Elsevier

Page 12 of 43

Chance-Constrained Selection from Counterfactual Estimates Table 3 The five evaluation environments, ordered by increasing realism, with the source of each outcome and cost. Scale and range are in Table S3.

Covariates Outcome Exact oracle Counterfactuals Cost unit Outcome measure

Sepsis

Tumor

MIMIC-IV

C-MAPSS

Drink Less

synthetic synthetic yes exact treatments survival

synthetic synthetic yes exact doses tumor control

real synthetic yes exact doses survival

real real degradation yes exact (specified) crew-hours no failure

real real (observed) no (off-policy) IPW prompts engagement

Every synthetic component is an established benchmark or a standard modeling approach from the counterfactual-inference and reliability literatures. In the maintenance environment, the source trajectories run to failure and contain no maintenance actions. The overhaul effect is therefore a specified model component, not one identified from the benchmark.

model that makes the oracle available. The maintenance environment replaces the clinical objective with an operations one on real degradation data. The comparison to a mean-budgeted approach is internal to every result. The naive rule of Equation (8) imposes the expected-cost constraint, and its budget violations measure the cost of managing the mean instead of the tail. An external mean-budgeted method would optimize over a parameterized policy class, so the comparison would confound the constrained functional with the policy space. Holding the policy space fixed at the candidate set used here reduces a mean-budget constraint to Equation (8) exactly. The rules therefore share a candidate set, a predictor, and an injected error realization, and differ only in the functional of the cost distribution each one constrains.

5.1. Simulation environments (sepsis and tumor) The sepsis simulator of Oberst and Sontag (2019) is a discrete Markov decision process over 720 physiological states and 8 actions. It is adopted as a benchmark for counterfactual and off-policy reasoning in sequential clinical decision making. A latent diabetes indicator drawn once per patient and held fixed makes treatment confounded with prognosis in the observational trajectories used later. The simulator is the primary setting because its transition and reward structure is fully specified over a finite state space. We compute the survival probability and the cumulative treatment cost of any strategy exactly by value iteration and rollout, which supplies the oracle decision quality is measured against. The tumor-growth model of Geng et al. (2017) is the de facto benchmark for time-varying treatment-effect work. In it, chemotherapy and radiotherapy reduce tumor volume over a horizon. A candidate strategy is a volume-threshold rule, the cost is the cumulative treatment count, and the outcome is a final tumor volume below a fixed level. All three are computed exactly by simulating the dynamics. Its uncertainty arises from patient heterogeneity in the growth and treatment-response parameters rather than from stochastic transitions. The cost-distribution spread on which the chance constraint operates therefore has a different origin in each of the two. Both are specified in full in S5.

5.2. Real-covariate semi-synthetic environment (MIMIC-IV) The semi-synthetic environment pairs the real covariate distributions of an intensive-care cohort with a synthetic outcome with counterfactuals known by construction. This retains the exact-regret measurement while removing the artificiality of simulator covariates. We draw the covariates from MIMIC-IV (Johnson et al., 2023), 3,000 patient trajectories of up to twenty-four hourly steps from a sepsis cohort extracted by a standard pipeline. They therefore carry the severity structure of a real intensive-care population. The outcome follows the benchmark model of the Causal Transformer (Melnychuk et al., 2022), a standard construction for semi-synthetic time-varying-treatment benchmarks on intensive-care data. Being a deterministic function of the covariates and the treatment sequence, it makes the outcome and cost of any strategy exactly computable, with no off-policy estimation. The candidate strategies are an escalation ladder indexed by a single aggressiveness parameter. Because the synthetic outcome is a design choice, we vary the outcome threshold and the physiological group driving the outcome. Section 6.6 reports the decision-rule behavior under each. The cohort composition, the outcome construction and the ladder are given in S5. We average all reported quantities over seeds that vary both the sampled cohort and the random components of the outcome. M. Kim and B. Jang: Preprint submitted to Elsevier

Page 13 of 43

Chance-Constrained Selection from Counterfactual Estimates

5.3. Cross-domain environment: predictive maintenance (C-MAPSS) The maintenance environment carries the objective outside clinical care. We build it from the NASA C-MAPSS turbofan benchmark (Saxena et al., 2008), a standard collection of run-to-failure sensor trajectories for a fleet of engines under a single operating condition. In a deployment, a remaining-useful-life predictor fit to such trajectories (Lei et al., 2018; Sateesh Babu et al., 2016; Zheng et al., 2017; Li et al., 2018) would play the role the g-computation model plays in the clinical environments, with the optimization step again agnostic to it. We summarize each engine’s condition by a scalar health index rising from zero at installation toward one at failure, and a candidate strategy is an overhaul threshold. The cost is the cumulative number of maintenance crew-hours consumed over a fixed operating horizon, a hard budget that cannot be traded against the benefit. The outcome is availability, completion of the horizon without an unplanned failure. The benchmark records engines run to failure and contains no maintenance actions, so the overhaul effect is specified rather than estimated from it. This environment tests whether the optimization step transfers to an operations objective under a hard resource limit. It does not test whether the identification assumptions of Section 3.1 can be met from degradation data. The spread that makes the chance constraint informative arises from heterogeneity in time-to-failure across engines. The index construction, the overhaul reset and the averaging over sampled fleets are given in S5.

5.4. Real-outcome environment with known propensities (Drink Less) This environment replaces modeled outcomes with observed ones, at the price of the oracle. We build it from the Drink Less micro-randomized trial (Bell et al., 2023). In this design, the intervention is randomized repeatedly within each participant at every decision point (Klasnja et al., 2015). A behavior-change app for alcohol reduction either did or did not send a daily engagement notification, with a probability fixed by the trial design and so known exactly. We treat the notification as the action and its presence as the per-step cost. The budgeted cost is the cumulative number of notifications over the horizon. The outcome to be maximized is engagement with the app in the hour after the decision point. Candidate policies are global rates and state-aware rules that notify more often where the trial shows the notification to be most effective. Because the randomization probabilities are known, we estimate the value of a candidate without outcome-model assumptions, by per-decision inverse probability weighting of the proximal outcome (Bao et al., 2025). This estimator avoids the trajectory-length weight degeneracy that afflicts importance sampling for long-horizon outcomes. We obtain the cost distribution directly, by accumulating a policy’s notifications over each participant’s observed state sequence. The chance constraint then acts on the tail of the realized cost induced by its randomization over the horizon. Both quantities are evaluated along the histories the trial generated, the standard form of micro-randomized-trial analysis (Bao et al., 2025). For the global policies the cost is exact, since their notifications do not depend on the state. Elsewhere the estimates coincide with the counterfactual quantities of Section 3.2 up to the effect of a notification on the subsequent state. With no specified model there is no exact oracle, so Sections 5.6 and 5.7 do not apply and regret is not reported. We evaluate the decision rules instead on the estimated value and the estimated violation probability of the recommended policy, summarized over bootstrap resamples of the participants. The policy family and the softening are given in S5.

5.5. Candidate strategies The rule selects among a set of candidate strategies rather than optimizing over a parameterized policy class. The finite set is a property of the problem the framework is posed for. First, a cumulative-budget decision of this kind is usually made over a small number of standing alternatives. Examples are a treatment escalation ladder, a maintenance schedule at a few inspection thresholds, and a notification policy at a few intensities. In a deployment, those alternatives are given, fixed by clinical guideline, operating procedure, or regulation. The question put to the analyst is which of them to run under the budget, not what the space of all measurable regimes contains. Second, restricting to a finite set keeps the constraint exactly the object the method is about. The cost tail is evaluated on each strategy’s own cumulative-cost distribution. Optimizing over a policy class would require a stage-wise decomposition, and the tail does not admit one. Third, the price of the restriction is bounded, since the guarantees of Section 4 depend on the set only through log||, so enlarging it costs little. Extending to a parameterized class instead would require a complexity measure in place of that term. The evaluation cannot take the candidate set as given, because measuring regret against a computable optimum requires the counterfactual outcome and cost of every candidate. A set drawn from practice would not come with those. We therefore construct the set in each environment. The construction is an experimental design choice instead of a component of the method. S12 reports what turns on it. Three constructions that place the candidates differently M. Kim and B. Jang: Preprint submitted to Elsevier

Page 14 of 43

Chance-Constrained Selection from Counterfactual Estimates

along the cost axis leave the exchange the chance constraint makes intact. In the micro-randomized trial of Section 5.4, the candidates are not solved for but written down from the trial’s own randomization structure. They are fixed-rate and state-aware notification policies. This is closer to how a deployed set would arise. In the simulators, we generate this set by solving for a range of treatment intensities; the semi-synthetic environment instead uses the escalation ladder of Section 5.2. For the sepsis simulator we apply value iteration with a per-step penalty 𝜆 on the quantity of treatment administered. Sweeping 𝜆 produces a family of deterministic strategies running from aggressive at 𝜆 = 0 to conservative at large 𝜆, each a well-defined dynamic treatment regime. The solver settings and the ways the candidates can be spaced along the cost axis are given in S12.

5.6. Exact counterfactual computation For each candidate strategy we compute its outcome value and its cumulative-treatment-cost distribution by rolling the strategy out in the environment. The resulting outcome probability and cost distribution are exact up to Monte Carlo error, which we control by using a large number of rollout episodes. The outcome value is the fraction of episodes ending in discharge, and the cost distribution is the empirical distribution of cumulative treatments over episodes. These exact quantities define the oracle the evaluation uses. They give the oracle optimum at each budget, the outcomemaximizing strategy with expected cost within the budget, against which the regret of every rule is measured, [ ] ( ) 𝑔 orc (𝐵) = arg max 𝑉 (𝑔) subject to 𝔼 𝐶(𝑔) ≤ 𝐵, Reg(𝑔) = 𝑉 𝑔 orc (𝐵) − 𝑉 (𝑔). (18) 𝑔∈

The word oracle is used in two ways below, both referring to this computation. The oracle cost distribution of a strategy is the exact counterfactual distribution it returns. The oracle at a budget is the mean-feasible optimum of Equation (18) computed from those exact quantities. The constraint in Equation (18) is on the mean cost instead of on its tail. The oracle at a budget is therefore risk-neutral and admits strategies the chance-constraint rule excludes. The regret of that rule against it then carries the intrinsic price of bounding the cost tail, not only the effect of estimation error. S6 reports the regret at zero injected error as a separate column for that reason. A rule can also record a negative regret under Equation (18). It does so by selecting a higher-outcome strategy that the mean-feasible optimum excludes, at the price of budget violations. The exact quantities also give the true expected cost of a recommended strategy, against which budget violations are checked.

5.7. Standing in for a predictor In deployment, the outcome and cost of a strategy would be produced by an estimator returning the distribution of the cumulative cost, not only its mean. The decision rules differ only when they act on such estimates instead of on exact quantities. To study this regime without tying the conclusions to a particular learned model, we inject controlled estimation error into the exact quantities. We perturb the oracle outcome value of each strategy by relative Gaussian noise with standard deviation 𝜀𝑠 . We perturb the oracle cost samples by relative Gaussian noise with standard deviation 𝜀𝑐 . Each decision rule is then applied to the perturbed quantities, with regret and violation measured against the unperturbed oracle. This injected error is a relative perturbation of the estimated value and cost, distinct from the intrinsic spread of the cost distribution on which the chance constraint operates. We apply symmetric mean-zero perturbations, a systematically biased perturbation that underestimates the cost, and a rescaling of each strategy’s cost samples about their own mean by a common factor. The rescaling holds every expected cost fixed and alters only how far the cumulative cost disperses around it. Varying the magnitude measures how the decision degrades as the predictor becomes less accurate. Perturbing the outcome and cost components separately attributes regret and violation to the error sources (Section 6.5).

5.8. Decision rules and metrics We compare the decision rules of Section 4: the naive point-estimate rule of Equation (8), the upper-bound rule of Equation (9) with margin parameter 𝜅, and the chance-constraint rule of Equation (10) with tolerated violation probability 𝜀. We additionally report the conditional-value-at-risk rule, which admits a strategy when CVaR1−𝜀 (𝐶(𝑔)) ≤ 𝐵, the safe convex approximation of Equation (10). We evaluate it on the same cost samples and at the same 𝜀. They then differ only in which functional of the upper tail the budget constrains, the tail mean instead of the tail quantile. The naive rule serves as the plug-in benchmark, since it inserts the point estimates into the budget feasibility check. Constrained-Markov-decision-process and safe reinforcement-learning methods are not run as baselines, since they require online interaction or known transition dynamics and so address a different regime M. Kim and B. Jang: Preprint submitted to Elsevier

Page 15 of 43

Chance-Constrained Selection from Counterfactual Estimates

(Section 2). We compute the chance-constraint probability from the perturbed cost samples of each strategy and sweep 𝜀 to trace the safety-utility frontier. The outcome regret is Reg of Equation (18), reported in percentage points (pp), with lower values better. The budget violation rate is the fraction of recommendations for which the true expected cost, evaluated under the oracle, exceeds the budget, also lower-is-better. The realized tail probability of a recommendation is Pr(𝐶(𝑔) > 𝐵) for the strategy it selects, computed on the true cost samples of that strategy. It is the quantity Equation (10) constrains. It separates from the budget violation rate whenever a strategy is feasible in expectation while carrying an upper tail beyond the budget. The oracle match rate is the fraction of budget-and-seed combinations for which the selected strategy is the one the risk-neutral oracle of Section 5.6 selects at that budget. Constrained ranking and selection measures this quantity as the probability of correct selection (Hong et al., 2015). Those procedures assume the best candidate is separated from the rest by a stated margin and introduce an acceptable set where it is not. We read the rate alongside regret: when several candidates attain nearly the same outcome, the rate can be low while regret stays small. Because the oracle is constrained on the mean cost, the rate measures how closely a rule recovers the risk-neutral choice. We report it for the point-estimate rule alone. The chance-constraint rule refuses strategies the oracle admits whenever their cost tail exceeds the tolerance, so its agreement is not informative. The safety-utility frontier reports the achievable outcome against the tolerated violation as 𝜀 varies, and the budget frontier reports the oracle-optimal outcome as the budget varies. Counterfactual forecasting error is not reported, since what is claimed is a better decision from a given set of counterfactual estimates, not a better estimate.

6. Results The evaluation is outcome regret against the oracle optimum and the budget violation rate, and summary values are unweighted averages over each environment’s budget grid. A budget enters an average only where every rule that average reports is defined in all 10 seeds. A quantity comparing two rules is therefore taken over the budgets both admit at. A quantity reporting one rule alone is taken over the budgets that rule admits at. Per-budget tables are in S6. Sections 6.1 to 6.4 establish the frontier and the decision rules under controlled error. Section 6.5 replaces controlled error with estimated predictors. Sections 6.6 to 6.8 test transfer across environments, operating-point selection, and real outcomes.

6.1. The exact budget-outcome frontier For each budget 𝐵 we compute the strategy maximizing the outcome among those for which the expected cumulative cost does not exceed it. The computation uses the exact distributions obtained by rolling each candidate out in the simulator. These numbers therefore involve no estimation and characterize the trade-off itself. Table S9 reports the result. The frontier is monotone in the budget, as it must be, and its curvature is the informative part. The outcome gained per unit of cost rises to 8.6 percentage points over the step from 𝐵 = 6 to 𝐵 = 8. It then falls to 3.2 by 𝐵 = 12, to 1.9 by 𝐵 = 20, and to 0.2 once the budget is removed. The value of an additional unit of budget is therefore not constant, and a recommender reporting only the best strategy at a given budget does not convey it.

6.2. Decision rules under estimation uncertainty The oracle quantities are perturbed as Section 5.7 sets out, and regret and violation are measured against the oracle optimum at the same budget. Table S5 reports the rules at every budget at an injected error of 15% and 𝜀 = 0.2, since what each rule gives up varies sharply with the budget. The naive rule is never far from the oracle. Its regret stays between 1.4 and 3.5 percentage points across the whole grid, well below the 9.3 to 26.6 points the chance-constraint rule gives up. From 𝐵 = 4 upward, the naive rule selects strategies with true expected cost above the budget, in 18.7% of cases at 𝐵 = 4 and in 4.1% at 𝐵 = 20. The chanceconstraint rule violates the budget at no budget in the grid, with no seed-to-seed spread. The comparison is therefore not which rule attains more outcome. It is what the guarantee costs. The violation is also unstable where it is largest, averaging 18.7% with a standard deviation of 19.7 at 𝐵 = 4. A few seeds overrun in most cases while others never do, so a decision maker deploying the rule once would not know which draw had been taken. The realized cost tail locates where the naive rule’s outcome advantage comes from. At 𝐵 = 12, the strategy it selects places 42.7% of its cumulative-cost mass above the budget, against 12.6% for the strategy the chance-constraint M. Kim and B. Jang: Preprint submitted to Elsevier

Page 16 of 43

Chance-Constrained Selection from Counterfactual Estimates Table 4 Chance-constraint frontier in the sepsis environment, swept over 𝜀 at budget 𝐵 = 12. 0.02

0.05

0.10

0.20

0.30

0.40

0.50

0.70

1.00

46.4 ± 0.5 0.0 ± 0.0

42.0 ± 0.6 0.0 ± 0.0

36.4 ± 0.7 0.0 ± 0.0

26.6 ± 0.5 0.0 ± 0.0

16.9 ± 1.0 0.0 ± 0.0

8.9 ± 0.6 0.1 ± 0.1

2.5 ± 0.4 19.4 ± 3.4

−12.9 ± 0.5 97.3 ± 0.4

−15.1 ± 0.4 99.2 ± 0.2

𝜀 Outcome regret (pp) Budget violation (%)

Mean ± s.d. over 10 seeds at 15% injected error. At 𝜀 = 1, the constraint is vacuous.

λ = 0.656

30

λ = 0.214 λ = 0.100

20

Outcome regret (pp)

λ = 0.086 λ = 0.062 λ = 0.060 λ = 0.052 λ = 0.046 λ = 0.031 λ = 0.029

Chance constraint (ε sweep) CVaR (ε sweep) Naive Upper bound

0.05 0.1 0.2 0.3

10

feasible optimum

0

0.5

−10 0.7

−20 −30

λ = 0.026 λ = 0.024

Budget B = 12 0

5

10

15

20

25

30

35

Cumulative treatment cost

(a)

−40

1

0

20

40

60

80

100

Budget violation (%)

(b)

Figure 2: Sepsis environment at budget 𝐵 = 12. (a) Cumulative-cost distributions of the 12 of 21 candidates feasible in expectation, with the five refused at 𝜀 = 0.2 in orange; (b) safety-utility frontier over 𝜀 at 15% injected error, 10 seeds, naive and upper-bound rules as reference points.

rule selects. That fraction grows with the budget for the naive rule, from 20.5% at 𝐵 = 1 to 47.7% at 𝐵 = 20. For the chance-constraint rule, it never exceeds the tolerance of 0.2 at any budget. Figure 2(a) shows why the two rules can disagree while both respect the budget in expectation. Twelve of the twenty-one candidates are feasible in expectation at 𝐵 = 12. Five of those twelve place more than a fifth of their cost mass above it, and the chance constraint refuses exactly those five. The two tightest budgets qualify the picture. At 𝐵 = 1 and 𝐵 = 2, the naive rule violates nothing and gives up less than the chance-constraint rule. The reason is that only two or three strategies are feasible in expectation and the constraint removes options without averting an overrun. Its selection at 𝐵 = 1 nonetheless carries a cost tail of 20.5%, which the budget violation rate does not register because the true mean stays inside the budget. Section 6.3 evaluates the rules on the realized tail for that reason. The price of the guarantee is not constant, largest at 𝐵 = 8 and 𝐵 = 12 at 25.6 and 26.6 points and smallest at 𝐵 = 4 at 9.3. A decision maker cannot know in advance which budget will be in force. The same estimation noise that leaves the naive rule mean-feasible at 𝐵 = 2 makes it overrun at 𝐵 = 4. The chance-constraint rule, by contrast, converts a stated tolerance into a strategy regardless of which budget applies. Sweeping the tolerance traces that exchange directly. Table 4 reports the sweep at 𝐵 = 12, the budget at which the naive rule’s regret is largest. Figure 2(b) plots regret against violation with the naive and upper-bound rules as reference points. The exchange is monotone over the 21 tolerances in the grid, with no reversal at any step. Regret falls from 46.4 points at 𝜀 = 0.02 to −15.1 at 𝜀 = 1 while violation rises from zero to 99.2%. Violation stays at exactly zero up to 𝜀 = 0.33, by which point half the outcome available on this frontier has been recovered. It stays under a tenth of a percent through 𝜀 = 0.40. Beyond 𝜀 = 0.5 the rule gains outcome almost entirely by overrunning. The separation is not an artifact of how densely the candidate set is populated. S12 repeats the comparison on the practice-sized set of seven a deployment would more plausibly enumerate. The naive rule violates under both constructions, at 4.6% against 8.0% under 21. The chance-constraint rule holds at zero under both. A denser set supplies more strategies with cost tail near the budget. The opportunity to misjudge feasibility from a point estimate therefore grows with the candidate count rather than shrinking. M. Kim and B. Jang: Preprint submitted to Elsevier

Page 17 of 43

Chance-Constrained Selection from Counterfactual Estimates

Outcome regret (pp)

Sepsis

Tumor

Semi-synthetic

Maintenance

30

30

30

20 20

20

20

10

10

10 10 0 0.1

0.2

0

0.3

0.1

Realized tail probability

0.2

0.1

0.2

0.3

Realized tail probability

Realized tail probability

Chance constraint

CVaR constraint

0.05

0.10

Realized tail probability

Figure 3: Outcome regret against realized tail probability, chance constraint (blue) against conditional value at risk (orange), by environment, 10 seeds.

Table 5 Ratio of the selected strategy’s realized tail probability to the tolerance that produced it. Sepsis

Tumor

Semi-synthetic

Maintenance

Tolerance

chance

CVaR

chance

CVaR

chance

CVaR

chance

CVaR

𝜀 = 0.10 𝜀 = 0.20 𝜀 = 0.50 maximum

0.32 0.48 0.79 0.89

0.08 0.12 0.20 0.31

0.24 0.53 0.87 0.91

0.02 0.07 0.20 0.25

0.48 0.58 0.74 0.75

0.16 0.17 0.23 0.31

0.30 0.30 0.64 0.90

0.05 0.06 0.08 0.10

Ten seeds. A value of one means the realized tail equals the tolerance exactly; values below one are conservative, the direction Proposition 3 permits. The maximum is over the tolerances below one, since at 𝜀 = 1 the constraint is vacuous. The upper-bound rule has no tolerance to form the ratio against.

6.3. Comparison with conservative decision rules The chance constraint bounds how often the cumulative cost exceeds the budget. A conditional-value-at-risk constraint bounds the average of the costs that do exceed it. The upper-bound rule bounds neither. The comparison cannot be read on the budget violation rate. The budget violation rate counts a recommendation as violating when the true expected cost exceeds the budget. Both conservative rules admit only mean-feasible strategies by construction, so their rate is zero at every tolerance short of one. Matching on it would measure one rule against a criterion the other satisfies as an identity. We evaluate each rule on the realized tail probability of Section 5.8, the quantity the chance constraint is stated over. Table 5 reports the ratio of the selected strategy’s realized tail probability to the tolerance that produced it, over the whole tolerance grid. Under the chance constraint, the fraction rises with the tolerance in every environment and reaches between 0.75 and 0.91 at the loose end of the grid. A decision maker who raises the tolerated violation obtains a recommendation with an overrun probability that moves with it. Under the conditional value at risk, it saturates below 0.31 everywhere and below 0.10 in the maintenance environment. That tolerance is a conservatism setting whose realized effect is three to ten times smaller than its nominal value and varies threefold across environments. The upper-bound rule cannot be entered in the table at all. Its 𝜅 is a multiple of a standard deviation, and no tolerance exists against which a realized tail could be read as a fraction. Choosing between 𝜅 = 0.25 and 𝜅 = 0.5 requires measuring decision quality against the true optimum. A deployment has no such optimum. The same ordering appears in how often each rule declines to recommend. The conditional-value-at-risk rule declines more often at every tolerance in the environments where either rule declines; S8 gives the rates. Declining is informative rather than a failure, since it reports that the budget is infeasible at the stated risk appetite, which a mean-constrained rule cannot express. A declined recommendation is excluded from that rule’s averages rather than counted as zero. No rule is therefore charged for the outcome it forgoes, and these rates are read alongside the regret. Proposition 3 bounds the realized tail probability at each budget separately, so the check is made at that level. Across the four environments, 21 tolerances and every budget, the mean realized tail probability of the admitted strategy stays M. Kim and B. Jang: Preprint submitted to Elsevier

Page 18 of 43

Chance-Constrained Selection from Counterfactual Estimates Table 6 Mean absolute outcome margin of each conservative rule against the chance-constraint rule at matched realized tail probability, in percentage points. CVaR

Upper bound

0.21 0.58 0.04 4.9

0.24 0.76 0.12 5.3

Sepsis Tumor Semi-synthetic Maintenance

At a matched realized tail probability of 0.15 the chance-constraint and conditional-value-at-risk rules both reach 12.9 points of regret in sepsis and 12.8 in the semi-synthetic environment.

at or below the tolerance. This holds in all 520 combinations at which the rule admits a strategy. At the level of the individual recommendation, the realized tail exceeds the nominal tolerance in up to 47.9% of cases. It exceeds the certified ceiling 𝜀 + 𝛿 of Equation (13) in at most 0.40%, against the 5% the Dvoretzky-Kiefer-Wolfowitz bound allows. Those exceedances sit entirely at tolerances of 0.22 and above. The rule therefore approaches the ceiling only where the constraint is loose enough that it operates at the boundary. Table 6 reports the outcome margin of each conservative rule against the chance constraint, read at a common realized tail probability (Figure 3). The rules are built on different functionals, one on the tail mean and one on a multiple of the standard deviation. They nonetheless trace close to the same efficient set as the chance constraint in these environments.

The maintenance environment At a matched realized tail probability, both conservative rules reach less regret than the chance-constraint rule there. The margin is 4.9 percentage points for the conditional value at risk and 5.3 for the upper-bound rule. One cause accounts for both margins. The outcome takes three levels across the 25 candidates, and 23 tie at the top. At budgets of four and above, the regret is therefore near zero whichever strategy is selected. At 𝐵 = 2, the chance constraint admits no strategy until the tolerance reaches 0.13, and the conditional value at risk none until 0.30. The comparison is a single binary choice at 𝐵 = 3, where choosing wrongly costs 56.24 points. The averaged regret is that one error spread over the budgets the average retains. Regret counts only the outcome. A rule that declines an overrunning strategy is therefore charged for what it gives up and credited with nothing for the overruns it avoids. At that budget, the chance constraint refuses the overrunning strategy and accepts a realized tail of 0.006. The conditional value at risk at a vacuous tolerance accepts 0.238. The tumor environment shares the structure at a smaller scale, with 20 of 23 candidates tied. S8 works through the arithmetic and the per-budget breakdown. The two constraints answer different questions and the framework accommodates either. Where an overrun is an administrative event with cost largely independent of its size, the frequency is the quantity to bound. A crew-hour budget exceeded by one shift or by three is an overrun of this kind. The chance constraint states the frequency in units a decision maker can set. Where the harm grows with the size of the overrun, as for a cumulative-exposure limit, the average severity is the quantity to bound. The conditional value at risk is then the appropriate choice, at the cost of a tolerance that must be calibrated environment by environment.

6.4. Finite-sample feasibility across sample sizes Proposition 3 bounds the realized tail probability of an admitted strategy by 𝜀 + 𝛿(𝑛) once the cost distribution is estimated from 𝑛 independent samples. The bound is distribution-free, so how much of it the data uses is an open question at deployment sample sizes. We apply the empirical rule to resamples of every candidate’s oracle cost distribution; Table S10 reports the protocol and the result. The observed deviation is a roughly constant fraction of 𝛿(𝑛) across a fortyfold range of sample size, running from 0.65 to 0.72 depending on the environment. The bound is therefore correct in its dependence on 𝑛 and loose by about a third in its constant. Premise and conclusion both hold inside their nominal levels. These deviations are the sampling term alone: the cost distributions are the exact ones, and 𝐹̂𝑔 differs from 𝐹𝑔 only because it is built from finitely many draws. Section 6.5 measures the same quantity when the cost distribution is itself estimated from observational trajectories, which is the model term. A deployment carries both, and Proposition 3 then applies with 𝛿 replaced by their sum. M. Kim and B. Jang: Preprint submitted to Elsevier

Page 19 of 43

Chance-Constrained Selection from Counterfactual Estimates CC violation Naive violation

CC regret Naive regret

5

5

10

75 0

50 25

−10

0 0

10

20

30

Injected noise (%)

−40

−20

Cost bias (%)

0

0

Outcome regret (pp)

10

100

Budget violation (%)

10

Outcome regret (pp)

15

0

Regret, outcome error Regret, cost error

(b)

15

Budget violation (%)

Outcome regret (pp)

(a)

CC violation Naive violation

Violation, outcome error Violation, cost error

4 (c)

14

3

12

2

10 8

1

Budget violation (%)

CC regret Naive regret

6 0

10

20

30

Injected error (%)

Figure 4: Sensitivity of the decision to estimation error, sepsis environment, averaged over the budget grid. (a) Mean-zero relative noise; (b) one-sided cost underestimate; (c) outcome-only against cost-only error, naive rule. Means over 10 seeds; we report seed-to-seed spread per budget in S6.

6.5. Estimation error and an estimated predictor The decision rests on estimates of outcome and cost, so its behavior as those degrade determines what a deployment can rely on. We inject controlled error into the exact quantities and average the response over the budget grid. Figure 4 reports mean-zero noise, a one-sided cost underestimate, and the decomposition of the injected error into its outcome and cost components. Under mean-zero noise, the chance-constraint violation is exactly zero at every injected-error level from zero to 30%, on every seed and every budget. Its regret rises from 14.3 to 18.0 points. Degraded estimates cost outcome without ever breaching the budget. The naive rule moves the other way on both axes, its violation falling from 12.8% to 6.8% as its regret rises from 0.5 to 3.9 points. Outcome noise redirects the selection away from the boundary strategy its own sampling error would otherwise admit. At zero injected error, its violation is that sampling effect alone. The chance-constraint rule’s exposure does not depend on the noise level at all. A trained predictor does not err symmetrically. Because units in worse states receive more intervention in observational data, a predictor fit to such data can underestimate cost. Underestimation is the error mode most consequential for a tail constraint. We inject a one-sided cost bias on top of the mean-zero noise and sweep it past what the estimated predictors below exhibit. The sweep therefore locates the breaking point rather than an operating point. Figure 4(b) reports it. The chance-constraint violation stays at zero through a 10% underestimate, reaches 0.70% at 30%, then 9.2% at 40% and 33.1% at 50%. The naive rule runs at 28.9% to 88.7% under the same bias, a factor of 87 apart at a 30% underestimate. A modest bias moves the chance-constraint rule’s selection to a different strategy that is still truly feasible. The violation rises only once the bias pulls a truly infeasible strategy into the set the biased estimate judges feasible. The upper-bound rule absorbs more bias, holding zero through 30%, with higher regret at every level. The level at which the guarantee fails is a requirement on deployment rather than a property of the rule, here between a 30% and a 40% underestimate. We read that band on the aggregate bias, and the estimated predictors below include one that satisfies it and breaks the rule anyway. The error sources act on different objectives and in opposite directions, as Figure 4(c) shows. Outcome error raises the regret from 0.5 to 3.9 points and lowers the violation from 12.8% to 6.3%. Cost error does the reverse, leaving the regret at 0.5 points and raising the violation to 14.0%. Accuracy on the cost distribution and on the outcome are therefore not interchangeable, and no single forecast-accuracy figure determines decision quality. A predictor can place the expected cost correctly and still understate how widely the cost disperses around it, which no perturbation above reaches. Rescaling each strategy’s cost samples about their own mean by a common factor 𝑠 holds every expected cost fixed. The point-estimate rule’s admitted set is therefore identical across the sweep, and everything the chance-constraint rule does across 𝑠 is attributable to the spread alone. Understating it raises the outcome and the realized tail together. As 𝑠 falls from one to one half, the chance-constraint regret falls by 5.7, 5.5 and 5.3 outcome percentage points in sepsis, tumor and the semi-synthetic environment. This is because an understated tail admits strategies the correct one refuses. The realized tail probability of the selected strategy meanwhile rises to 0.184, 0.154 and 0.206 against a nominal tolerance of 0.2. The largest of these exceeds the tolerance, because samples with M. Kim and B. Jang: Preprint submitted to Elsevier

Page 20 of 43

Chance-Constrained Selection from Counterfactual Estimates Table 7 Function approximation at 20,000 trajectories, chance-constraint rule at 𝜀 = 0.2. Deviation

Violation (%)

Environment

Estimator

Cost bias (%)

Largest

>𝛿

Point estimate

Chance

Tumor

ensemble ridge ensemble ridge

−14.30 +8.86 +2.51 −5.30

0.252 0.103 0.147 0.261

6∕23 0∕23 4∕25 16∕25

10.0 0.0 6.7 96.7

0.0 0.0 0.0 11.7

Semi-synthetic

The certified slack is 𝛿 = 0.131 in both environments. The deviation column is the largest per-strategy deviation over seed means, as in Tables S16 and S17, which report all sample sizes.

an understated spread no longer satisfy the premise of Proposition 3. The budget violation moves far less, staying at zero in two environments and rising only from 0.02% to 0.08% in the tumor environment. The effect of an understated spread is therefore recorded mostly on the realized tail probability, not on the budget violation rate. S9 reports the sweep, the maintenance environment, and the region of 𝑠 it excludes.

An estimated predictor end to end The injected error above is a controlled substitute for a predictor. We replace it

with an estimated predictor. The sepsis state space is finite. The transition kernel can therefore be estimated by counting transitions rather than by function approximation, without any tuning choice that would confound the reading. We generate observational trajectories under a behavior policy that conditions on severity, so treatment is confounded with prognosis. We fit the kernel by categorical maximum likelihood and roll each of the 21 candidate strategies forward under it. State-action pairs the behavior policy never visits receive a uniform transition over states. The estimated outcome value and cost distribution go to both rules unchanged, and the oracle enters only to score the result. S14 records the validation of the rollout against the simulator’s own implementation. Estimation and decision quality move together between 1,000 and 5,000 trajectories. The uniform deviation falls from 0.294 to 0.097, the point-estimate rule’s violation from 18.6% to 2.9% and its regret from 4.85 to 2.89 points. The fraction of evaluation episodes traversing a state-action pair the behavior data never visited falls from 26.5% to 7.7%. The last is the mechanism: where the observational data does not cover the region a strategy traverses, the uniform fallback supplies the transition. The estimated cost distribution is then wrong in the tail. At 1,000 trajectories the point-estimate rule overruns the budget in 18.6% of recommendations. The chanceconstraint rule overruns in none, at that sample size and at every larger one. Its regret is the price, 16.37 points against 4.85. The regrets then behave differently as the estimate improves. The point-estimate rule’s regret falls from 4.85 to 0.57 points, which is estimation error dissolving. The chance-constraint rule’s stays between 15.36 and 16.37, because it is not estimation error but the outcome conceded by refusing strategies the mean-feasible oracle admits. A deployment cannot tell from a single run which sample-size regime it is in. The exposure of the point-estimate rule depends on that regime and the exposure of the chance-constraint rule does not. The estimator errs in the level of cost, not in its order; S13 reports the rank correlations and the bias. A rule selecting on cost ranking alone would be unaffected. The chance constraint, however, compares an estimated cost distribution against an absolute budget, so the level governs it. Its realized cost tail stays between 0.579 and 0.697 of the nominal tolerance across the four sample sizes. The rule is therefore conservative against its own stated level under model error as well as under sampling error.

Function approximation Counting is viable only because the sepsis state space is finite.

On the tumor environment, a gradient-boosted ensemble and a ridge regression of deliberately different flexibility err in opposite directions (Table 7; S10). Neither bias contracts as the sample grows, which places it in the model rather than in the sampling. The bias is residual confounding the fitted model does not remove. That changes what the point-estimate rule is exposed to. In the sepsis environment, its violation fell to zero as the estimate improved, so its exposure was a small-sample condition. Here it does not fall at any of four sample sizes spanning two orders of magnitude (Table 7). Under the ridge, it is zero throughout, not because the rule is safe but

M. Kim and B. Jang: Preprint submitted to Elsevier

Page 21 of 43

Chance-Constrained Selection from Counterfactual Estimates

because that estimator happens to err upward. The safety of a rule reading only the mean is a property of which way its predictor errs. The chance-constraint rule holds the violation at zero under both estimators at every sample size. It does so even where the premise of Proposition 3 fails. The guarantee is a sufficient condition on the deviation, not a necessary one, and the rule is observed to hold on the far side of it. The regret is 8.1 to 10.3 points under the ensemble and 13.9 to 14.4 under the ridge. Regret is small for either rule in absolute terms because 20 of the 23 candidates tie at an outcome of 99.9%. The semi-synthetic environment has the tightest cost-underestimate band of the four, at 6.6% violation for a 30% underestimate against 0.70% in sepsis. We fit the same estimators there, in S10.2. Both aggregate biases are small, the ensemble overestimating the cost by 2.5% and the ridge underestimating it by 5.3%. The band therefore admits either with a wide margin. The aggregate bias conceals the ridge’s per-strategy errors. These run from −28% on four adjacent strategies to +79% on one and cancel in the mean. A band read on the aggregate does not detect this, and the deviation does. The ridge exceeds the certified slack 𝛿 = 0.131 at 16 of the 25 strategies, at a largest per-strategy value of 0.261. The chance-constraint rule does not hold the budget here. This is the only configuration of Table 7 in which it fails. It admits a violating strategy in 11.7 to 15.0% of recommendations, and the realized tail of what it selects reaches 0.324 against a nominal tolerance of 0.2. The certified ceiling still holds. Proposition 3 bounds the realized tail probability at 𝜀 + 𝛿 = 0.331 under its premise, and the realized tail stays under that at every sample size, at 0.323 to 0.329. The point-estimate rule on the same estimator overruns in 93 to 97% of recommendations, at a negative regret of about −5.7 points. The rule gains outcome the mean-feasible optimum excludes, with the corresponding overruns. The cause is the estimator, not the covariates. S10.2 reports the coverage correlations and the overlap between the estimators’ failure sets. The contrast with the spread sweep above locates the mechanism. There the mis-statement is common to every strategy, and almost everything the widened admitted set takes in is still feasible in expectation. The violation therefore stays below one percent. Here it differs across strategies by more than a hundred percentage points, and the rule admits infeasible strategies in more than one recommendation in ten. Uniform mis-statement of the spread is the benign form and dispersion of it across strategies is the damaging one. The requirement therefore transfers, and the number does not. A predictor supplying the cost distribution must keep its deviation inside the certified slack. The deviation column of Table 7 checks this, and a bias band read on the aggregate is not a substitute.

The share of the regret that is estimation error The exact counterfactual distribution of each strategy bounds what any predictor could deliver to the rule. Running the same rule on that distribution at 20,000 trajectories and comparing within seed separates the price of bounding the tail from the price of estimating it. S17 reports the three arms, the paired intervals and the scale factors. In sepsis, the estimated predictor is ahead of the exact-distribution arm at every tolerance up to 0.3. The sign follows from the benchmark. The oracle at a budget is mean-feasible, so a predictor reading the tails exactly refuses strategies the oracle admits, while one understating a tail admits them. Under function approximation, both estimators leave a gap. The gap reaches 9.95 outcome percentage points for the ridge at a tolerance of 0.2, with every paired interval excluding zero. Correcting the level does not recover that part. A single pooled rescaling to the correct mean leaves one estimator’s selection unchanged and moves the other the wrong way. The estimators misstate the shape of the upper tail rather than its position, and a scalar applied after the fact does not reach it.

6.6. Generalization across environments The remaining environments differ in mechanism, in the source of their uncertainty, and in one case in domain. In the tumor and maintenance environments, the budget stops binding early and the frontier is flat thereafter; in sepsis and the semi-synthetic environment it binds throughout the grid. Where the frontier saturates, every budget above the saturation point admits a strategy at the top outcome level and the decision is trivial there. That makes the regret comparisons in those two environments turn on a single step. Full per-budget frontiers are in S6. Panel (a) of Table 8 reports the naive and chance-constraint rules averaged over each environment’s budget grid, the two the violation axis separates. Section 6.3 places the conservative rules on the axis that separates them. The naive rule overruns in every environment, between 5.3% and 15.2% of the time. The chance-constraint rule holds the violation at or below 0.21% at a regret between 9.5 and 16.9 points. Sweeping the tolerance reproduces the exchange of outcome for violation in each of them; S11 plots the frontiers. The chance-constraint regret rises and falls with the steepness of the oracle frontier. It peaks at 24.6 points at the single step where the tumor outcome rises and at 56.3 at the corresponding step in the maintenance environment. The semi-synthetic environment has no frontier step and stays within a 12.6 to 17.8 point band. Where a strategy with a M. Kim and B. Jang: Preprint submitted to Elsevier

Page 22 of 43

Chance-Constrained Selection from Counterfactual Estimates Table 8 Decision rules at 𝜀 = 0.2, 10 seeds. (a) Exact-counterfactual environments; (b) real-outcome environment. (a) Exact counterfactuals Naive

(b) Real outcomes, Drink Less

Chance constraint

Environment

Regret

Viol.

Regret

Viol.

Sepsis Tumor Semi-synthetic Maintenance

2.5±0.7 0.5±0.3 3.5±0.4 0.4±0.3

8.0±3.5 15.2±2.5 7.3±3.2 5.3±1.2

16.9±0.7 11.1±0.8 14.9±0.7 9.5±0.2

0.00±0.00 0.02±0.01 0.00±0.00 0.21±0.13

Rule Naive Upper-bound Chance-constraint

Value

Est. tail

2.66±0.16 2.28±0.16 2.63±0.18

0.149±0.081 0.099±0.005 0.115±0.010

Panel (a) averages over each environment’s budget grid at 15% injected error, regret in outcome percentage points and violation in percent. The semi-synthetic environment is built on MIMIC-IV and the maintenance one on C-MAPSS. Panel (b) is at 𝐵 = 15 and reports value, since no oracle exists there.

materially better outcome first becomes affordable, the constraint must refuse its cost tail, and the guarantee is expensive there. The naive rule meanwhile overruns at nearly every budget rather than at isolated ones. It reaches 18.7% in sepsis, 22.8% in the tumor environment, 9.9% in the semi-synthetic environment and 17.0% in the maintenance environment. The sepsis simulator’s uncertainty is in its transitions, and in the others it is in the units. The behavior is therefore not tied to a particular mechanism, to synthetic covariates, or to the setting in which exact evaluation is most readily available. The semi-synthetic outcome is a design choice, so we vary the benchmark construction through the outcomethreshold quantile and the real covariates across disjoint physiological groups. The chance-constraint rule holds the violation at zero in every configuration and the naive rule overruns in between 8.4% and 13.2% of cases (Table S12). The success criterion in the tumor environment and the outcome threshold in the semi-synthetic environment qualify this. They are modeling choices, not intrinsic terminal events as the outcome is in the sepsis simulator. Still, a volume target is clinically meaningful, and we report the robustness to the threshold above. The value of the chance constraint is also not a simple function of how widely the candidate cost distributions are dispersed within an environment. That dispersion and the naive-versus-chance gap were not monotonically related. The claim is therefore restricted to the consistency of the behavior across environments rather than to a single quantitative driver.

6.7. Structure of the frontier and operating-point selection The frontier is monotone in the tolerance, as the operating-point selection of Section 4 requires. Sweeping 𝜀 over the 21 tolerances of the grid, the outcome regret is non-increasing and the budget violation is non-decreasing in every one of the four environments. There is no reversal at any step. Relaxing the tolerance can therefore only gain outcome and can only raise exposure. The curvature of the frontier carries no comparable regularity: the second difference of regret changes sign 8 to 12 times depending on the environment. The frontier is not convex in any of them. That is expected rather than incidental. The constrained functional is a quantile, and a quantile constraint does not deliver a convex efficient set even where the feasible set at a fixed tolerance is convex. An operating point is therefore selected by reading the frontier, not by locating a knee. The density of the frontier follows the spread of the candidate outcomes. Where candidates realize outcome values across the whole range, as in sepsis and the semi-synthetic environment, regret falls at every step of the tolerance grid. Where most are tied at one value, as in the tumor and maintenance environments, the frontier is sparse. A tightened target may then land on a conservative operating point that a slightly looser one would improve on. Table S13 reports the operating point the target-based selection returns at two values of the target 𝜌. At a target of 10%, it recommends a strategy with regret between 0.6 and 4.9 outcome percentage points in three environments. The realized tail probability is below the target in each. Tightening the target to five percent costs between 1.7 and 2.8 further points. The decision maker states a rate and receives a strategy, without choosing 𝜀 directly. The maintenance environment is the exception. Relaxing the target there gains 0.02 points: all of its regret comes from the single binary choice at 𝐵 = 3. That choice does not turn over until the tolerance passes 0.43. Where a frontier is that sparse, a target expressed as a violation rate has little to select over.

M. Kim and B. Jang: Preprint submitted to Elsevier

Page 23 of 43

4.0 (a)

Global rate policy State-aware policy

3.5

Estimated value (engagements)

Estimated value (engagements)

Chance-Constrained Selection from Counterfactual Estimates

3.0 2.5 2.0 1.5 1.0

5

10

15

20

(b)

ε = 1.00

3.50 3.25 3.00

ε = 0.40

2.75

ε = 0.20

2.50

ε-swept frontier Naive Upper-bound Chance-constraint

2.25 2.00

25

Notification budget (cumulative cost)

ε = 0.05

0.0

0.2

0.4

0.6

0.8

Realized tail probability Pr(C(g) > B)

1.0

Figure 5: Drink Less real-outcome environment, bootstrap mean±s.d. (a) Estimated value against notification budget, global-rate policies as a banded line and state-aware policies as markers; (b) safety-utility frontier over 𝜀 at 𝐵 = 15.

The certified slack grows only in log||, so densifying the candidate set is cheap in the guarantee. A sweep over set size on the tumor environment measures what it costs in computation. From 50 to 10,000 candidates, the slack rises by a factor of 1.30 at 𝜂 = 0.05 and the budget violation stays at zero at every size. Generating a candidate’s cost distribution is a rollout and takes 0.18 seconds however many there are. The optimization step takes 0.12 seconds plus 0.22 milliseconds per candidate. At 10,000 candidates, a rollout pass therefore runs for half an hour and the selection for two seconds. The optimization step accounts for 0.13 percent of the runtime, at unchanged peak memory. The outcome the chance constraint attains rises from 0.861 to 0.900 as the grid fills in. Its margin over the point-estimate rule narrows from 7.9 to 4.8 percentage points, most of that by 1,000 candidates. We measure regret against the meanfeasible optimum of the set being swept, so it moves with that set and is not read across sizes.

6.8. Real outcomes under known propensities On the Drink Less environment, the framework operates on real engagement outcomes. The per-decision weighting stays well conditioned: the effective sample size ranges from about 4,650 to the full 10,470 decision points. No importance weight exceeds 2.375, the largest ratio the softened candidate policies can form against the trial’s randomization probabilities. The value comes directly from the trial data without an oracle, and per-policy diagnostics are in S15. The estimated value increases monotonically with the notification budget, from about 1.2 engagements over the horizon to about 3.7. This traces a real-outcome budget frontier with the shape seen on the model-based environments. The best state-aware policy attains 0.22±0.04 more engagements than the global policy of the same cost, as Figure 5(a) shows. The three decision rules separate on real outcomes as the controlled study predicts, in panel (b) of Table 8. At a budget of 15 notifications, the naive rule attains the highest value, but the estimated tail probability of its selection is large and highly variable across resamples. The chance-constraint rule attains nearly the same value while holding the estimated tail probability to a tightly controlled level; the upper-bound rule is the most conservative. Sweeping the tolerance traces the frontier directly, as Figure 5(b) shows. The estimated value rises from 1.90±0.13 to 3.66±0.26 as 𝜀 relaxes from 0.05 to 1, while the realized tail probability rises from 0.007±0.001 to one. The pattern is unchanged when the outcome is engagement within the following day rather than the following hour. Because this environment has no oracle, the quantities are estimated value and estimated violation rather than regret against a known optimum. Its evidence is therefore of a different kind from the preceding sections and complements them. Selecting a policy by its estimated value and reporting that estimate would inflate the value if the selection were chasing sampling noise. Over 50 random splits of the participants, we choose the recommended policy on one half and re-estimate it on the other. The split estimates sit at or above the single-sample values, and the chance-constraint rule selects the same policy under every split. The half-sample standard deviations are 0.08 to 0.12 on a value scale spanning 1.2 to 3.7. They bound any optimism at roughly a tenth of an engagement rather than excluding it.

7. Conclusion, Limitations, and Further Research We presented a predict-then-optimize framework for sequential intervention selection under a cumulative budget. In the prediction step, any estimator returning an outcome value and a cumulative-cost distribution for each candidate M. Kim and B. Jang: Preprint submitted to Elsevier

Page 24 of 43

Chance-Constrained Selection from Counterfactual Estimates

strategy supplies what the decision rule consumes. Both are counterfactual quantities identified from observational trajectories. In the optimization step, the rule returns the outcome-maximizing strategy subject to a chance constraint on the probability that the cost exceeds the budget. That tail does not decompose across stages. The constraint is therefore evaluated on the full-horizon distribution of each candidate, and the decision is a selection over a finite set. Distribution-free finite-sample bounds accompany the rule on both sides. They limit how far an admitted strategy can exceed the tolerated violation and how far its outcome can fall short of what is achievable at a tightened tolerance. The method traces the budget-outcome frontier and locates where an added unit of budget is worth most. It eliminates the budget violations the naive rule incurs, at a quantified outcome cost that varies with the budget. The decision is robust to estimation error in the predictor, including moderate systematic cost bias. The chance constraint does not improve on the naive rule at every budget, and we report the exchange directly. It costs outcome where a high-outcome strategy carries a heavy cost tail and controls violation where the naive rule overruns. The method returns the safety-utility frontier, and a stated operational target selects a single recommendation on it. The frontier shows the outcome achievable at each level of tolerated budget violation. Its single control, the tolerated violation probability, has an operational rather than statistical meaning. It sets, at the level of policy, not of the individual case, how often a cumulative-toxicity limit or a maintenance-hour budget may be overrun across a population. The per-case recommendation is computed within that envelope. The level can therefore be set against the operational cost of an overrun in the domain at hand. It is tightened where an overrun is expensive and relaxed where outcome is at a premium. Where to invest in a better model follows from the error decomposition of Section 6.5. Outcome error and cost error act on different objectives, regret and violation respectively. Budget overruns observed in deployment reflect the cost distribution the predictor implies. In a g-computation model, that distribution is a functional of the same simulated trajectory that produces the outcome. The check that matters is calibration of that distribution against the absolute budget, not accuracy on the outcome alone. Several limitations bound the scope of the claims. First, the decision rules and the frontier are established against exact or known outcomes. The four exact-counterfactual environments supply a ground-truth optimum, and the microrandomized trial supplies real outcomes under known randomization probabilities. Those four span real covariates and real degradation data, so the consistency across them is not an artifact of one setting. None supplies a clinical realoutcome deployment in which the outcome is a long-horizon terminal event and the propensities must be estimated. That case is harder for a tail-constrained method specifically. Importance-sampling estimators recover the upper tail of a cumulative cost far less reliably than its mean once a strategy departs from the behavior policy. Such a study is therefore better suited to a qualitative check that the conservative rules reduce overruns than to a precise reconstruction of the frontier. It is the natural next step. Second, the frontier and the decision rules are established under exact counterfactuals, and the effect of estimation error is studied by injecting controlled noise. This isolates the optimization step from the accuracy of any one predictor. Where the state space is finite the substitution can be checked directly, and Section 6.5 does so on the sepsis environment. A continuous-state environment requires function approximation. The same section repeats the check with estimators of opposite cost bias. The bias is architecture-dependent, and the violation the chance constraint admits is not. A deployment would still inherit the accuracy of whatever predictor it fits. Where the cost distribution is fit by function approximation rather than counted, a measurable part of the regret is estimation error rather than the price of the constraint. Recalibrating the estimated distribution to the correct mean recovers none of it. Closing that requires a predictor that gets the upper tail right. Third, like any counterfactual method, the validity of the recommendations rests on the identification assumptions, in particular sequential ignorability. Sequential ignorability is not testable from observational data alone and would have to be argued from domain knowledge about what drives treatment decisions. Fourth, the method selects a strategy once at the start of the horizon and does not re-select as the remaining budget is consumed. Fifth, the candidate sets used here are constructed rather than drawn from practice, since measuring regret against a computable optimum requires counterfactuals for every candidate. S12 shows the reported exchange survives both a practice-sized and a dense construction, but a deployment would inherit whatever set its guidelines supply. How the method behaves on a set chosen for reasons other than cost spacing is not something these experiments can settle. These limitations indicate the directions the work should take. Integrating an end-to-end predictor is the first, and the cost estimate on which the violation guarantee depends is the component to validate. The deviation against the certified slack, not an aggregate bias band, is the check a deployment should run. A rolling-horizon rule that updates the recommendation on the realized cost so far would address the single-selection limitation. Settings with several M. Kim and B. Jang: Preprint submitted to Elsevier

Page 25 of 43

Chance-Constrained Selection from Counterfactual Estimates

simultaneous resource or toxicity limits would call for a vector of budgets and a joint chance constraint. Where the dynamics are estimated rather than known, the strategy set could be derived from operating guidelines or learned directly, in place of the sweep across treatment intensities used here. This would change how the frontier is populated without changing the decision component. Finally, which features of the history drive a switch between strategies is a natural complement to the decision-quality evaluation and would address the interpretability expectations of highstakes deployment.

CRediT authorship contribution statement Minkyoung Kim: Conceptualization, Methodology, Software, Formal analysis, Investigation, Data curation, Visualization, Writing – original draft. Beakcheol Jang: Conceptualization, Supervision, Funding acquisition, Writing – review and editing.

Declaration of competing interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Funding This study was supported by the National Research Foundation of Korea Fund (grant number RS-2023-00273751).

Data availability This study uses four publicly available sources and one credentialed-access database. The environments are built on the Gumbel-max sepsis simulator (Oberst and Sontag, 2019), the tumor-growth model of Geng et al. (2017), the NASA C-MAPSS turbofan degradation dataset (Saxena et al., 2008), the Drink Less micro-randomized trial data (Bell et al., 2023), available at https://osf.io/w3szp/, and MIMIC-IV (Johnson et al., 2023), accessed through PhysioNet under credentialed access. S5 gives the construction of each environment. The code implementing the evaluation environments, the decision rule, and all reported analyses is available at https://github.com/ mfriendly/counterfactual-chance-selection.

Declaration of generative AI and AI-assisted technologies in the writing process During the preparation of this work the authors used Claude in order to refine the phrasing of the manuscript. After using this tool, the authors reviewed and edited the content as needed and take full responsibility for the content of the published article.

References Altman, E., 1999. Constrained Markov Decision Processes. Chapman and Hall/CRC. Angelotti, G., Drougard, N., Chanel, C.P.C., 2026. An offline risk-aware policy selection method for Bayesian Markov decision processes. Artificial Intelligence 354, 104519. doi:10.1016/j.artint.2026.104519. Athey, S., Wager, S., 2021. Policy learning with observational data. Econometrica 89, 133–161. doi:10.3982/ECTA15732. Bao, Y., Bell, L., Williamson, E., Garnett, C., Qian, T., 2025. Estimating causal effects for binary outcomes using per-decision inverse probability weighting. Biostatistics 26, kxae025. doi:10.1093/biostatistics/kxae025. Bäuerle, N., Ott, J., 2011. Markov decision processes with average-value-at-risk criteria. Mathematical Methods of Operations Research 74, 361–379. doi:10.1007/s00186-011-0367-0. Bell, L., Garnett, C., Bao, Y., Cheng, Z., Qian, T., Perski, O., Potts, H.W.W., Williamson, E., 2023. How notifications affect engagement with a behavior change app: Results from a micro-randomized trial. JMIR mHealth and uHealth 11, e38342. doi:10.2196/38342. Bertsimas, D., Kallus, N., 2020. From predictive to prescriptive analytics. Management Science 66, 1025–1044. doi:10.1287/mnsc.2018.3253. Bertsimas, D., Van Parys, B., 2022. Bootstrap robust prescriptive analytics. Mathematical Programming 195, 39–78. doi:10.1007/ s10107-021-01679-2. Bica, I., Alaa, A.M., Jordon, J., van der Schaar, M., 2020. Estimating counterfactual treatment outcomes over time through adversarially balanced representations, in: International Conference on Learning Representations (ICLR). Caljon, D., Van Belle, J., Berrevoets, J., Verbeke, W., 2026. Optimizing treatment allocation in the presence of interference. European Journal of Operational Research 328, 620–632. doi:10.1016/j.ejor.2025.09.015.

M. Kim and B. Jang: Preprint submitted to Elsevier

Page 26 of 43

Chance-Constrained Selection from Counterfactual Estimates Chakraborty, B., Moodie, E.E.M., 2013. Statistical Methods for Dynamic Treatment Regimes: Reinforcement Learning, Causal Inference, and Personalized Medicine. Statistics for Biology and Health, Springer, New York, NY. doi:10.1007/978-1-4614-7428-9. Charnes, A., Cooper, W.W., 1959. Chance-constrained programming. Management Science 6, 73–79. doi:10.1287/mnsc.6.1.73. Chow, Y., Ghavamzadeh, M., Janson, L., Pavone, M., 2018. Risk-constrained reinforcement learning with percentile risk criteria. Journal of Machine Learning Research 18, 1–51. URL: http://jmlr.org/papers/v18/15-636.html. De Vos, S., Bockel-Rickermann, C., Lessmann, S., Verbeke, W., 2026. Uplift modeling with continuous treatments: A predict-then-optimize approach. European Journal of Operational Research 330, 230–244. doi:10.1016/j.ejor.2025.10.025. Dudík, M., Langford, J., Li, L., 2011. Doubly robust policy evaluation and learning, in: Proceedings of the 28th International Conference on Machine Learning (ICML), pp. 1097–1104. Elmachtoub, A.N., Grigas, P., 2022. Smart “predict, then optimize”. Management Science 68, 9–26. doi:10.1287/mnsc.2020.3922. Fernández-Loría, C., Provost, F., 2022. Causal decision making and causal effect estimation are not the same and why it matters. INFORMS Journal on Data Science 1, 4–16. doi:10.1287/ijds.2021.0006. Geng, C., Paganetti, H., Grassberger, C., 2017. Prediction of treatment response for combined chemo- and radiation therapy for non-small cell lung cancer patients using a bio-mathematical model. Scientific Reports 7, 13542. doi:10.1038/s41598-017-13646-z. Haskell, W.B., Jain, R., 2015. A convex analytic approach to risk-aware Markov decision processes. SIAM Journal on Control and Optimization 53, 1569–1598. doi:10.1137/140969221. Hoeffding, W., 1963. Probability inequalities for sums of bounded random variables. Journal of the American Statistical Association 58, 13–30. doi:10.1080/01621459.1963.10500830. Hong, L.J., Luo, J., Nelson, B.L., 2015. Chance constrained selection of the best. INFORMS Journal on Computing 27, 317–334. doi:10.1287/ ijoc.2014.0628. Johnson, A.E.W., Bulgarelli, L., Shen, L., Gayles, A., Shammout, A., Horng, S., Pollard, T.J., Hao, S., Moody, B., Gow, B., Lehman, L.w.H., Celi, L.A., Mark, R.G., 2023. MIMIC-IV, a freely accessible electronic health record dataset. Scientific Data 10, 1. doi:10.1038/ s41597-022-01899-x. Kallus, N., Zhou, A., 2022. Stateful offline contextual policy evaluation and learning, in: Proceedings of the 25th International Conference on Artificial Intelligence and Statistics (AISTATS), PMLR, pp. 11169–11194. Kannan, R., Bayraksan, G., Luedtke, J.R., 2025. Data-driven sample average approximation with covariate information. Operations Research 73, 3245–3259. doi:10.1287/opre.2020.0533. Killian, T.W., Parbhoo, S., Ghassemi, M., 2023. Risk sensitive dead-end identification in safety-critical offline reinforcement learning. Transactions on Machine Learning Research (TMLR) . Klasnja, P., Hekler, E.B., Shiffman, S., Boruvka, A., Almirall, D., Tewari, A., Murphy, S.A., 2015. Microrandomized trials: An experimental design for developing just-in-time adaptive interventions. Health Psychology 34S, 1220–1228. doi:10.1037/hea0000305. Komorowski, M., Celi, L.A., Badawi, O., Gordon, A.C., Faisal, A.A., 2018. The artificial intelligence clinician learns optimal treatment strategies for sepsis in intensive care. Nature Medicine 24, 1716–1720. doi:10.1038/s41591-018-0213-5. Kumar, U.M., Bhat, S.P., Kavitha, V., Hemachandra, N., 2023. Approximate solutions to constrained risk-sensitive Markov decision processes. European Journal of Operational Research 310, 249–267. doi:10.1016/j.ejor.2023.02.039. Laber, E.B., Wu, F., Munera, C., Lipkovich, I., Colucci, S., Ripa, S., 2018. Identifying optimal dosage regimes under safety constraints: An application to long term opioid treatment of chronic pain. Statistics in Medicine 37, 1407–1418. doi:10.1002/sim.7566. Lei, Y., Li, N., Guo, L., Li, N., Yan, T., Lin, J., 2018. Machinery health prognostics: A systematic review from data acquisition to RUL prediction. Mechanical Systems and Signal Processing 104, 799–834. doi:10.1016/j.ymssp.2017.11.016. Li, R., Hu, S., Lu, M., Utsumi, Y., Chakraborty, P., Sow, D.M., Madan, P., Li, J., Ghalwash, M., Shahn, Z., Lehman, L.w.H., 2021. G-Net: a recurrent network approach to g-computation for counterfactual prediction under a dynamic treatment regime, in: Proceedings of Machine Learning for Health (ML4H), PMLR, pp. 282–299. Li, X., Ding, Q., Sun, J.Q., 2018. Remaining useful life estimation in prognostics using deep convolution neural networks. Reliability Engineering & System Safety 172, 1–11. doi:10.1016/j.ress.2017.11.021. Lim, B., Alaa, A.M., van der Schaar, M., 2018. Forecasting treatment responses over time using recurrent marginal structural networks, in: Advances in Neural Information Processing Systems (NeurIPS), pp. 7493–7503. Liu, C., Zhu, W., 2024. Newsvendor conditional value-at-risk minimisation: A feature-based approach under adaptive data selection. European Journal of Operational Research 313, 548–564. doi:10.1016/j.ejor.2023.08.043. Liu, M., Wang, Y., Fu, H., Zeng, D., 2024a. Controlling cumulative adverse risk in learning optimal dynamic treatment regimens. Journal of the American Statistical Association 119, 2622–2633. doi:10.1080/01621459.2023.2270637. Liu, M., Wang, Y., Fu, H., Zeng, D., 2024b. Learning optimal dynamic treatment regimens subject to stagewise risk controls. Journal of Machine Learning Research 25, 1–64. Luedtke, J., Ahmed, S., 2008. A sample approximation approach for optimization with probabilistic constraints. SIAM Journal on Optimization 19, 674–699. doi:10.1137/070702928. Massart, P., 1990. The tight constant in the Dvoretzky-Kiefer-Wolfowitz inequality. The Annals of Probability 18, 1269–1283. doi:10.1214/aop/ 1176990746. McFowland III, E., Gangarapu, S., Bapna, R., Sun, T., 2021. A prescriptive analytics framework for optimal policy deployment using heterogeneous treatment effects. MIS Quarterly 45, 1807–1832. doi:10.25300/MISQ/2021/15684. Melnychuk, V., Frauen, D., Feuerriegel, S., 2022. Causal transformer for estimating counterfactual outcomes, in: International Conference on Machine Learning (ICML), PMLR, pp. 15293–15329. Murphy, S.A., 2003. Optimal dynamic treatment regimes. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 65, 331–355. doi:10.1111/1467-9868.00389.

M. Kim and B. Jang: Preprint submitted to Elsevier

Page 27 of 43

Chance-Constrained Selection from Counterfactual Estimates Nemirovski, A., Shapiro, A., 2006. Convex approximations of chance constrained programs. SIAM Journal on Optimization 17, 969–996. doi:10.1137/050622328. Oberst, M., Sontag, D., 2019. Counterfactual off-policy evaluation with Gumbel-max structural causal models, in: International Conference on Machine Learning (ICML), PMLR, pp. 4881–4890. Rahimian, H., Pagnoncelli, B., 2023. Data-driven approximation of contextual chance-constrained stochastic programs. SIAM Journal on Optimization 33, 2248–2274. doi:10.1137/22M1528045. Robins, J.M., 1986. A new approach to causal inference in mortality studies with a sustained exposure period—application to control of the healthy worker survivor effect. Mathematical Modelling 7, 1393–1512. doi:10.1016/0270-0255(86)90088-6. Robins, J.M., Hernán, M.A., Brumback, B., 2000. Marginal structural models and causal inference in epidemiology. Epidemiology 11, 550–560. doi:10.1097/00001648-200009000-00011. Rockafellar, R.T., Uryasev, S., 2000. Optimization of conditional value-at-risk. Journal of Risk 2, 21–41. doi:10.21314/JOR.2000.038. Sadana, U., Chenreddy, A., Delage, E., Forel, A., Frejinger, E., Vidal, T., 2025. A survey of contextual optimization methods for decision-making under uncertainty. European Journal of Operational Research 320, 271–289. doi:10.1016/j.ejor.2024.03.020. Sakaguchi, S., 2025. Estimation of optimal dynamic treatment assignment rules under policy constraints. Quantitative Economics 16, 981–1022. doi:10.3982/QE2288. Sateesh Babu, G., Zhao, P., Li, X.L., 2016. Deep convolutional neural network based regression approach for estimation of remaining useful life, in: Database Systems for Advanced Applications (DASFAA), LNCS, pp. 214–228. doi:10.1007/978-3-319-32025-0_14. Saxena, A., Goebel, K., Simon, D., Eklund, N., 2008. Damage propagation modeling for aircraft engine run-to-failure simulation, in: International Conference on Prognostics and Health Management (PHM), pp. 1–9. doi:10.1109/PHM.2008.4711414. Smith, J.E., Winkler, R.L., 2006. The optimizer’s curse: Skepticism and postdecision surprise in decision analysis. Management Science 52, 311–322. doi:10.1287/mnsc.1050.0451. Sun, L., 2026. Empirical welfare maximization with constraints. Journal of Econometrics 253, 106169. doi:10.1016/j.jeconom.2025.106169. Thomas, P.S., Brunskill, E., 2016. Data-efficient off-policy policy evaluation for reinforcement learning, in: Proceedings of the 33rd International Conference on Machine Learning (ICML), PMLR, pp. 2139–2148. Thomas, P.S., Theocharous, G., Ghavamzadeh, M., 2015. High-confidence off-policy evaluation, in: Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence (AAAI), pp. 3000–3006. doi:10.1609/aaai.v29i1.9541. Vanderschueren, T., Baesens, B., Verdonck, T., Verbeke, W., 2024. A new perspective on classification: Optimally allocating limited resources to uncertain tasks. Decision Support Systems 179, 114151. doi:10.1016/j.dss.2024.114151. Wang, S., Mehrotra, S., Peng, C., 2025. Robust concave utility maximization over chance constraints. European Journal of Operational Research 321, 800–813. doi:10.1016/j.ejor.2024.10.007. Wu, S., Zhou, W., Chen, M., Zhu, S., 2024. Counterfactual generative models for time-varying treatments, in: Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD), pp. 3402–3413. doi:10.1145/3637528.3671950. Xiong, H., Wu, F., Deng, L., Su, M., Lehman, L.w.H., 2024. G-Transformer: counterfactual outcome prediction under dynamic and time-varying treatment regimes, in: Proceedings of the 9th Machine Learning for Healthcare Conference (MLHC), PMLR. Zhao, Y., Zeng, D., Rush, A.J., Kosorok, M.R., 2012. Estimating individualized treatment rules using outcome weighted learning. Journal of the American Statistical Association 107, 1106–1118. doi:10.1080/01621459.2012.695674. Zheng, S., Ristovski, K., Farahat, A., Gupta, C., 2017. Long short-term memory network for remaining useful life estimation, in: 2017 IEEE International Conference on Prognostics and Health Management (ICPHM), pp. 88–95. doi:10.1109/ICPHM.2017.7998311. Zhong, Y., Zhao, B., Zhang, K., Jiang, G., 2026. Chance-constrained selection of the best: An indifference-zone-free procedure. European Journal of Operational Research 335, 479–489. doi:10.1016/j.ejor.2026.05.042.

M. Kim and B. Jang: Preprint submitted to Elsevier

Page 28 of 43

Chance-Constrained Selection from Counterfactual Estimates

Supplementary material S1. Proof of Proposition 4 Proposition 4 of Section 4.3 states the following. With 𝛿 and 𝜖𝑉 set as in Equation (16), the strategy 𝑔̂ returned by the empirical chance-constraint rule at tolerance 𝜀 satisfies Equation (17) with probability at least 1 − 𝜂. The argument intersects two uniform deviation bounds, each taken at half the confidence budget.

Uniform bounds Applying the Dvoretzky-Kiefer-Wolfowitz inequality with the tight constant of Massart (1990) to each strategy separately and taking a union bound over  at level 𝜂∕2 gives | | sup |𝐹̂𝑔 (𝑏) − 𝐹𝑔 (𝑏)| ≤ 𝛿 | | 𝑏

for every 𝑔 ∈ ,

(19)

√ with 𝛿 = log(4||∕𝜂)∕(2𝑛). The per-unit contributions to 𝑉̂ (𝑔) lie in an interval of length 𝑅 by assumption, so Hoeffding’s inequality (Hoeffding, 1963) applied to 𝑉̂ (𝑔) and a union bound at the same level give |̂ | for every 𝑔 ∈ , (20) |𝑉 (𝑔) − 𝑉 (𝑔)| ≤ 𝜖𝑉 | | √ with 𝜖𝑉 = 𝑅 log(4||∕𝜂)∕(2𝑛). Each event fails with probability at most 𝜂∕2, so both hold simultaneously with probability at least 1 − 𝜂. The remainder of the argument works on that intersection, which makes the conclusions of Equation (17) simultaneous rather than separate.

Feasibility The empirical rule admits 𝑔̂ only if 1 − 𝐹̂𝑔̂ (𝐵) ≤ 𝜀. Evaluating Equation (19) at 𝑏 = 𝐵 gives 𝐹𝑔̂ (𝐵) ≥ 𝐹̂𝑔̂ (𝐵) − 𝛿 ≥ (1 − 𝜀) − 𝛿, and rearranging gives Pr(𝐶(𝑔) ̂ > 𝐵) ≤ 𝜀 + 𝛿. This is the argument of Proposition 3 at the constant of Equation (16).

Regret Write 𝑔̃ = 𝑔 ∗ (𝜀 − 𝛿) for the best strategy truly feasible at the tightened tolerance. Since Pr(𝐶(𝑔) ̃ > 𝐵) ≤ 𝜀 − 𝛿,

Equation (19) evaluated at 𝑏 = 𝐵 gives 1 − 𝐹̂𝑔̃ (𝐵) ≤ 𝜀, so 𝑔̃ is admitted by the empirical rule. The rule returns the admitted strategy of greatest estimated value, and it maximizes over a set containing 𝑔, ̃ so 𝑉̂ (𝑔) ̂ ≥ 𝑉̂ (𝑔). ̃ Applying Equation (20) to 𝑔̃ and 𝑔, ̂ ( ) ( ) 𝑉 (𝑔) ̃ − 𝑉 (𝑔) ̂ ≤ 𝑉̂ (𝑔) ̃ + 𝜖𝑉 − 𝑉̂ (𝑔) ̂ − 𝜖𝑉 ≤ 2𝜖𝑉 , (21) which is the second half of Equation (17). The tightened tolerance enters only through the admission of 𝑔: ̃ at the nominal tolerance 𝜀 a strategy with a true tail exactly at 𝜀 need not be admitted. The argument would then have no benchmark inside the empirical feasible set.

S2. Notation Table S1 collects the notation and Table S2 the terms used throughout.

S3. Identification of the counterfactual cost distribution Section 3.1 states Equation (1) and attributes it to Robins (1986). The budget constraint is imposed on the full law of the cumulative cost rather than on a summary of it. We set out the argument that carries Assumptions 1 to 3 to that law. Write 𝐿̄ 𝑚∶𝐾 (𝑔) for the covariate path generated under strategy 𝑔. Fix a path 𝑙̄𝑚∶𝐾 and let 𝑎̄𝑡−1 denote the treatment history the strategy assigns along it, 𝑎𝑠 = 𝑔𝑠 (ℎ𝑠 ) for 𝑠 < 𝑡. By the chain rule, the counterfactual path law factorizes into its step-wise conditionals, 𝐾 ( ) ∏ ( ) Pr 𝐿̄ 𝑚∶𝐾 (𝑔) = 𝑙̄𝑚∶𝐾 = Pr 𝐿𝑡 (𝑔) = 𝑙𝑡 ∣ 𝐿̄ 𝑡−1 (𝑔) = 𝑙̄𝑡−1 .

(22)

𝑡=𝑚

M. Kim and B. Jang: Preprint submitted to Elsevier

Page 29 of 43

Chance-Constrained Selection from Counterfactual Estimates Table S1 Notation used throughout. Symbol

Meaning

𝑡 ∈ {0, … , 𝐾} 𝐿𝑡 , 𝐴𝑡 𝐿̄ 𝑚 , 𝐴̄ 𝑚−1 𝐻𝑚 = (𝐿̄ 𝑚 , 𝐴̄ 𝑚−1 ) 𝐿𝑡 (𝑔) 𝑔 = (𝑔𝑚 , … , 𝑔𝐾 ) 

Discrete decision steps over the horizon Time-varying covariates and treatment decision at step 𝑡 Covariate and treatment histories up to step 𝑚 Observed history at step 𝑚 Potential covariate vector at step 𝑡 under strategy 𝑔 Dynamic treatment strategy, with 𝑔𝑡 ∶ 𝐻𝑡 ↦ 𝐴𝑡 Finite candidate set of strategies

𝑉 (𝑔), 𝑉̂ (𝑔) 𝑐(⋅) ∑ 𝐶(𝑔) = 𝑡 𝑐(𝐴𝑡 ) ∑𝑡 𝑆𝑡 = 𝑠=𝑚 𝑐(𝐴𝑠 ) ̂ 𝐶(𝑔), 𝜎̂ 𝐶 (𝑔) 𝐹𝑔 , 𝐹̂𝑔

Outcome value of strategy 𝑔, and its estimate Cost of the treatment administered at a step Cumulative cost of strategy 𝑔 over the horizon Partial cost sum through step 𝑡 Estimated mean and standard deviation of the cost True and empirical distribution function of 𝐶(𝑔)

𝐵 𝜀 𝜅 ∗ ∗ ∗ 𝑔naive , 𝑔ub , 𝑔cc (𝜀) 𝑉 ∗ (𝜀) 𝜌 𝑟 (𝐹̂𝑔 ), 𝑟

Cumulative budget Tolerated probability that the cost exceeds the budget Margin parameter of the upper-bound rule Strategies selected by the three decision rules Admissible set at tolerance 𝜀 Achievable outcome at tolerance 𝜀 Target ceiling on the realized tail probability Kolmogorov ambiguity set and its radius

𝑛 𝛿 𝜖𝑉 , 𝑅 𝜂

Number of independent samples per strategy Uniform deviation of 𝐹̂𝑔 from 𝐹𝑔 Value deviation, and the range of per-unit value contributions Failure probability of the finite-sample guarantees

𝑔 orc (𝐵), Reg(𝑔) 𝜀𝑠 , 𝜀𝑐 𝑏𝑐 𝜆

Oracle optimum at budget 𝐵, and regret against it Standard deviation of the injected outcome and cost error Relative systematic bias in the estimated cost Per-step treatment penalty used to generate candidates

Under Assumption 1, the treatment received at each step is independent of the future potential covariate path given the observed history. Conditioning further on the event that the observed treatment history agrees with what the strategy assigns therefore leaves each factor unchanged, ( ) ( ) Pr 𝐿𝑡 (𝑔) = 𝑙𝑡 ∣ 𝐿̄ 𝑡−1 (𝑔) = 𝑙̄𝑡−1 = Pr 𝐿𝑡 (𝑔) = 𝑙𝑡 ∣ 𝐿̄ 𝑡−1 = 𝑙̄𝑡−1 , 𝐴̄ 𝑡−1 = 𝑎̄𝑡−1 . (23) On that event, Assumption 3 equates the potential covariate with the observed one. This replaces the counterfactual on the right of Equation (23) by a conditional law of the observed data, ( ) ( ) Pr 𝐿𝑡 (𝑔) = 𝑙𝑡 ∣ 𝐿̄ 𝑡−1 = 𝑙̄𝑡−1 , 𝐴̄ 𝑡−1 = 𝑎̄𝑡−1 = 𝑝 𝑙𝑡 ∣ 𝑙̄𝑡−1 , 𝑎̄𝑡−1 . (24) Assumption 2 makes each conditioning event in Equations (23) and (24) occur with positive probability, so every factor is well defined. Substituting Equations (23) and (24) into Equation (22) gives Equation (1). The cumulative cost is a deterministic function of the path. Along 𝑙̄𝑚∶𝐾 the strategy assigns 𝑎𝑡 = 𝑔𝑡 (ℎ𝑡 ) at every ( ) ∑ step, so 𝐶(𝑔) = 𝐾 𝑡=𝑚 𝑐 𝑔𝑡 (ℎ𝑡 ) carries no randomness beyond that of the path itself. Its law is the image of the path law under that map, [𝐾 ] ∑ ( ) ∑ ( ) 𝐹𝑔 (𝑏) = Pr 𝐿̄ 𝑚∶𝐾 (𝑔) = 𝑙̄𝑚∶𝐾 𝟙 𝑐 𝑔𝑡 (ℎ𝑡 ) ≤ 𝑏 , (25) 𝑙̄𝑚∶𝐾

𝑡=𝑚

M. Kim and B. Jang: Preprint submitted to Elsevier

Page 30 of 43

Chance-Constrained Selection from Counterfactual Estimates Table S2 Terms used throughout, with the section that defines each. Term

Definition

Defined in

Outcome regret

Outcome of the oracle at a budget minus the outcome of the selected strategy, in percentage points Fraction of budget-and-seed combinations in which the selected strategy’s true expected cost exceeds the budget Pr(𝐶(𝑔) > 𝐵) for the selected strategy, on its true cost samples Fraction of budget-and-seed combinations in which the selected strategy is the one the oracle selects at that budget Exact counterfactual distribution of the cumulative cost of a strategy Mean-feasible optimum, the outcome-maximizing strategy with true expected cost within the budget Tolerated probability that the cumulative cost exceeds the budget Uniform deviation of the empirical cost distribution from the true one, at confidence 1 − 𝜂 𝜀 + 𝛿, the bound Proposition 3 places on the realized tail probability Candidate strategies satisfying the chance constraint at a given tolerance and budget The tolerance the decision maker adopts, and the recommendation it produces Achievable outcome against the tolerated violation as 𝜀 varies Oracle-optimal outcome as the budget varies Fraction of the discretized state-action cells a strategy traverses in which the behavior policy generated no samples

5.6

Budget violation rate Realized tail probability Oracle match rate Oracle cost distribution Oracle at a budget Tolerance 𝜀 Certified slack 𝛿 Certified ceiling Admissible set (𝜀) Operating point Safety-utility frontier Budget frontier Coverage gap

5.8 5.8 5.8 5.6 5.6 4.1 4.3 4.3 4.2 4.2 4.2 5.8 S10

which is Equation (2) once the path law is replaced by its identified form. The outcome value 𝑉 (𝑔) of Equation (3) is an expectation under the same law and is identified by the same substitution. Both quantities the decision rule consumes therefore rest on Equations (22) to (24) and on nothing further. The three assumptions deliver the whole of 𝐹𝑔 and not one of its moments. A rule that constrains 𝔼[𝐶(𝑔)] uses only the first moment of Equation (25), and Equation (5) shows that this moment is a sum of per-stage terms. The chance constraint of Equation (10) evaluates 𝐹𝑔 at the budget, which Equation (6) shows depends on the dependence between stages and not on the per-stage laws alone. The identification argument is indifferent to which functional is taken, but the estimation problem is not: the tail at a single point is estimated less accurately than a mean at the same sample size. Proposition 3 bounds the consequence of that gap for the recommendation.

S4. Selection procedure Section 4 states the chance-constrained rule. Algorithm S1 writes it out. Selection touches each candidate once, so the cost is 𝑂(||𝑛) in the size of the candidate set and the number of cost samples. A sweep over 𝜀 reuses the same per-strategy tail counts. Algorithm S1 Chance-constrained selection at a tolerated violation 𝜀. Require: candidate strategies , budget 𝐵, tolerated violation 𝜀 ∗ Ensure: recommended strategy 𝑔cc 1: for each 𝑔 ∈  do 2: obtain 𝑉̂ (𝑔) and cost samples 𝐶 (1) (𝑔), … , 𝐶 (𝑛) (𝑔) from the predictor ∑ 3: 𝑣̂(𝑔) ← 1𝑛 𝑛𝑗=1 𝟏[ 𝐶 (𝑗) (𝑔) > 𝐵 ] 4: end for 5:  ← { 𝑔 ∈  ∶ 𝑣 ̂(𝑔) ≤ 𝜀 } ∗ ← arg max ̂ 6: return 𝑔cc 𝑔∈ 𝑉 (𝑔), or infeasible if  = ∅

M. Kim and B. Jang: Preprint submitted to Elsevier

⊳ empirical tail probability 1 − 𝐹̂𝑔 (𝐵) ⊳ admissible set

Page 31 of 43

Chance-Constrained Selection from Counterfactual Estimates

S5. Environment specifications Section 5 summarizes each environment. This appendix records the mechanism in full, so that every quantity the decision rule consumes can be traced to its generative source. Table S3 gives the scale of each environment and the range each quantity covers, completing Table 3. Table S3 Scale and range of the five evaluation environments. Sepsis

Tumor

MIMIC-IV

C-MAPSS

Drink Less

Units Horizon Decision points State Actions Candidate strategies

8000 20 20 720 discrete 8 21

2000 30 30 continuous 4 23

3000 24 16.0‡ continuous 2 25

2000 500 25 continuous 2 25

349 30 30 continuous 2 17

Mean cost range Outcome range

0.00–34.03 9.9–96.5%

0.00–14.19 65.5–99.8%

1.08–15.98 35.1–90.2%

0.85–25.00 43.6–100.0%

1.51–28.52 1.23–3.64†

The cost unit and the outcome measure of each column are given in Table 3. † Drink Less values are per-participant IPW estimates of expected cumulative engagement, not an outcome probability. ‡ Mean over stays; episodes follow the real stay lengths.

S5.1. Sepsis simulator A patient state is described by four physiological variables, heart rate, blood pressure, oxygen saturation and glucose. Three binary indicators record whether each of antibiotics, vasopressors and mechanical ventilation is currently active. The simulator carries a latent binary diabetes indicator, drawn once per patient and held fixed over the episode. This indicator makes treatment confounded with prognosis in the observational trajectories of Section 6.5. At each step, the clinician chooses one of eight actions, the on or off setting of the three treatments. Policies are solved on the mixture of the two diabetes-specific transition matrices over the 720 observed states, while rollouts use the diabetes-specific dynamics. The oracle therefore follows the simulator’s generative process rather than the mixture used for planning. Following the simulator’s definition, a state is absorbing and fatal when at least three physiological variables are abnormal. It is absorbing and discharging when no variable is abnormal and no treatment is active. All other states are transient. An episode runs for at most twenty steps, with a reward of +1 for discharge and −1 for death.

S5.2. Tumor-growth simulator The environment implements the pharmacokinetic-pharmacodynamic model of Geng et al. (2017), in which tumor volume evolves under chemotherapy and radiotherapy over a fixed horizon. Growth and treatment-response parameters are drawn per patient, so two patients under the same strategy follow different volume paths and consume different amounts of treatment. That per-patient heterogeneity, rather than stochastic transitions, is the source of the cost spread the chance constraint operates on. This makes the environment complementary to the sepsis simulator. A candidate strategy is a volume-threshold rule: treatment is administered at a step whenever the tumor volume exceeds the strategy’s threshold. A lower threshold is therefore a more aggressive strategy. The cost is the cumulative count of treatments administered along the realized path. The outcome is binary success, taken as a final tumor volume below a fixed level. Because the dynamics are specified in full, the success probability and the cost distribution of every candidate are obtained by simulating the model directly. The oracle and the decision-quality metrics are therefore defined exactly as in the sepsis environment.

S5.3. Maintenance environment Each engine’s condition is summarized by a scalar health index obtained from a closed-form regression of its informative sensor channels onto the fraction of life elapsed, clipped to the unit interval and made monotone. The index therefore rises from zero at installation toward one at failure. Inspections are spaced a fixed number of cycles apart, and a candidate strategy performs an overhaul whenever the index exceeds its threshold. M. Kim and B. Jang: Preprint submitted to Elsevier

Page 32 of 43

Chance-Constrained Selection from Counterfactual Estimates

Baseline degradation is taken directly from the real run-to-failure trajectories. The effect of an overhaul is modeled as a virtual-age reset as is standard in reliability analysis. The reset restores the engine to a younger effective age instead of to new. The spread in the cost distribution is induced by a per-engine multiplier on the degradation rate. The time-to-failure heterogeneity that makes the chance constraint informative therefore comes from the fleet rather than from the maintenance policy.

S5.4. Semi-synthetic cohort The sampled cohort comprises 47,986 hourly records drawn from an extraction of 27,935 stays. Table S4 reports its composition. Table S4 Semi-synthetic cohort, mean (s.d.) over the sampled trajectories. Quantity Trajectories sampled Sequence length (steps) Age (years) Charlson comorbidity index SOFA score

Value 3,000 16.0 (5.7) 65.6 (16.2) 5.2 (3.0) 5.5 (2.7)

The covariate trajectories therefore carry the severity structure of a real intensive-care population. The cohort is large enough that the empirical cost distribution on which the chance constraint operates is stable across resamples. The benchmark remains tractable to regenerate under each random seed. The outcome follows the benchmark model of the Causal Transformer (Melnychuk et al., 2022). An untreated trajectory is the sum of an exogenous component and an endogenous component. The exogenous component is a random nonlinear function of selected real covariates, and the endogenous component is drawn from a Gaussian process with a spline trend. A designed treatment effect that decays over an effect window is applied for each of the three synthetic treatments. The terminal outcome is taken from the trajectory by a soft threshold, so that a more favorable trajectory has a higher probability of the favorable outcome. The threshold is placed at a quantile of the untreated outcome distribution that fixes the baseline difficulty of the benchmark.

S5.5. Semi-synthetic candidate ladder and benchmark variation Each candidate strategy applies the three synthetic treatments according to thresholds on the running outcome. A single aggressiveness parameter shifts all three thresholds together, from a setting that treats only the most severe trajectories to one that treats readily. Sweeping it produces a family of state-dependent regimes analogous to the penalty sweep in the simulators. The exact oracle, the injected estimation error and the decision rules of the main text apply to it without change. Because the synthetic outcome is a design choice, the benchmark is varied. The difficulty is varied by placing the outcome threshold at different quantiles of the untreated outcome distribution. The real covariates that drive the exogenous component are exchanged across disjoint physiological groups. Section 6.6 reports the decision-rule behavior under each.

S5.6. Drink Less policy family The candidate policies are written down from the trial’s own randomization structure rather than solved for. Global policies notify at a fixed rate over the horizon. State-aware policies notify more often where the trial shows the notification to be most effective. They concentrate notifications early in enrollment and on days when the participant has already engaged. Every policy is softened away from the deterministic boundary, so that no decision point receives a candidate probability of zero or one and the per-decision importance weights remain bounded. S15 reports the resulting effective sample sizes and the largest weight each policy forms against the trial’s randomization probabilities.

S6. Per-budget decision-rule tables Tables S5 to S8 report the three decision rules at every budget in each environment, the per-budget breakdown underlying the summaries of Section 6.2 and Table 8. Regret is in outcome percentage points and violation in percent, M. Kim and B. Jang: Preprint submitted to Elsevier

Page 33 of 43

Chance-Constrained Selection from Counterfactual Estimates

each averaged over 10 seeds. The summary value for a rule is the unweighted mean of its column here, so every summary value quoted in Section 6 is reproducible from these tables. A budget is included only where every seed admits a strategy under every rule; the excluded cells are shown as a dash. In the violation columns, the upper-bound rule admits a strategy only when the estimated mean plus a positive margin stays within the budget. The zeros in its violation column therefore record the admission criterion rather than a measured outcome. Table S5 Sepsis: decision rules at each budget (15% injected error, 𝜀 = 0.2; mean ± s.d., 10 seeds). The last column is the chance rule at 0% injected error. Naive

Upper bound

Chance (𝜀 = 0.2)

𝐵

Regret

Viol.

Regret

Viol.

Regret

Viol.

Regret (0%)

1 2 4 6 8 12 20

1.6±0.4 3.5±0.7 1.4±3.3 2.0±3.6 2.3±0.3 3.5±0.4 3.1±0.3

0.0±0.0 0.0±0.0 18.7±19.7 17.1±17.8 10.1±2.7 6.2±1.7 4.1±1.1

12.0±0.6 19.5±0.6 15.2±3.8 19.6±3.8 28.3±0.9 26.8±0.4 18.0±0.3

0.0±0.0 0.0±0.0 0.0±0.0 0.0±0.0 0.0±0.0 0.0±0.0 0.0±0.0

12.0±0.6 10.5±0.6 9.3±3.6 17.1±3.8 25.6±0.9 26.6±0.5 17.3±0.3

0.0±0.0 0.0±0.0 0.0±0.0 0.0±0.0 0.0±0.0 0.0±0.0 0.0±0.0

10.5 8.6 7.6 14.3 23.3 23.5 12.1

Table S6 Tumor: decision rules at each budget (15% injected error, 𝜀 = 0.2; mean ± s.d., 10 seeds). The last column is the chance rule at 0% injected error. Naive

Upper bound

Chance (𝜀 = 0.2)

𝐵

Regret

Viol.

Regret

Viol.

Regret

Viol.

Regret (0%)

1 2 3 4 5 6

2.6±0.7 −0.8±1.2 1.0±0.2 0.2±0.0 0.1±0.0 0.0±0.0

2.4±1.2 19.1±5.8 17.4±7.8 11.0±9.0 18.1±7.7 22.8±7.7

5.0±0.7 14.6±0.7 32.4±0.9 26.5±1.2 22.0±1.3 9.7±1.9

0.0±0.0 0.0±0.0 0.0±0.0 0.0±0.0 0.0±0.0 0.0±0.0

4.6±0.8 11.2±0.9 24.6±1.1 19.4±1.6 5.4±1.1 1.3±0.2

0.1±0.1 0.0±0.0 0.0±0.0 0.0±0.0 0.0±0.0 0.0±0.0

4.4 1.9 21.7 3.7 3.7 0.0

Table S7 Semi-synthetic (MIMIC-IV): decision rules at each budget (15% injected error, 𝜀 = 0.2; mean ± s.d., 10 seeds). The last column is the chance rule at 0% injected error. Naive

Upper bound

Chance (𝜀 = 0.2)

𝐵

Regret

Viol.

Regret

Viol.

Regret

Viol.

Regret (0%)

6 8 10 12 14 16

3.2±0.9 3.3±0.6 3.1±0.4 3.4±1.0 4.0±0.7 4.1±0.4

8.4±6.1 6.0±5.7 7.2±4.5 9.9±3.7 8.0±4.7 4.0±7.3

17.7±1.1 17.2±1.8 15.1±2.2 14.6±2.2 18.6±2.7 19.7±1.1

0.0±0.0 0.0±0.0 0.0±0.0 0.0±0.0 0.0±0.0 0.0±0.0

15.3±0.8 14.5±1.0 12.6±1.4 12.6±1.7 16.7±3.2 17.8±1.2

0.0±0.0 0.0±0.0 0.0±0.0 0.0±0.0 0.0±0.0 0.0±0.0

11.1 10.2 8.1 8.4 12.9 13.2

S7. Supporting results from the main text We draw 𝑛 ∈ {50, 100, 200, 500, 1000, 2000} independent samples from the oracle cost distribution of every candidate, form 𝐹̂𝑔 and apply the empirical rule, over 200 resamples at each of 10 seeds and at 𝜂 = 0.05. This gives both M. Kim and B. Jang: Preprint submitted to Elsevier

Page 34 of 43

Chance-Constrained Selection from Counterfactual Estimates Table S8 Maintenance (C-MAPSS): decision rules at each budget (15% injected error, 𝜀 = 0.2; mean ± s.d., 10 seeds). The last column is the chance rule at 0% injected error. Budgets at which no strategy is admitted are shown as a dash and excluded from the averages. Naive

Upper bound

Chance (𝜀 = 0.2)

𝐵

Regret

Viol.

Regret

Viol.

Regret

Viol.

Regret (0%)

2 3 4 5 6 7

0.0±0.0 2.1±1.6 0.1±0.0 0.0±0.0 0.0±0.0 0.0±0.0

0.0±0.0 0.0±0.0 8.7±2.6 0.8±0.4 5.2±9.7 17.0±6.6

0.0±0.0 56.3±1.1 16.8±6.9 0.2±0.1 0.1±0.1 0.1±0.1

0.0±0.0 0.0±0.0 0.0±0.0 0.0±0.0 0.0±0.0 0.0±0.0

0.0±0.0 56.3±1.1 0.6±0.3 0.0±0.0 0.0±0.0 0.0±0.0

0.0±0.0 0.0±0.0 0.1±0.1 0.1±0.1 0.4±0.8 0.7±0.4

0.0 51.4 0.0 0.0 0.0 0.0

Table S9 Oracle budget frontier (sepsis, 21 candidate strategies, 10 seeds). The last column is the outcome gained per unit of realized cost over the preceding row. Budget 𝐵

Outcome (%)

Realized cost

ΔOutcome / ΔCost

1 2 4 6 8 12 20 ∞

21.9 ± 0.6 30.5 ± 0.5 39.1 ± 3.7 52.2 ± 3.7 66.6 ± 0.6 79.9 ± 0.4 93.2 ± 0.3 96.5

0.88 ± 0.01 1.95 ± 0.03 3.46 ± 0.55 5.53 ± 0.46 7.21 ± 0.07 11.37 ± 0.09 18.49 ± 0.15 32.73

8.1 5.7 6.3 8.6 3.2 1.9 0.2

Table S10 Mean uniform deviation sup𝑏 |𝐹̂𝑔 − 𝐹𝑔 | as a fraction of the distribution-free bound 𝛿(𝑛), at 𝜂 = 0.05. Candidate-set sizes are 21, 23, 25 and 25, so 𝛿(𝑛) differs across environments only in the third decimal. 𝑛

𝛿(𝑛)

Sepsis

Tumor

Semi-synthetic

Maintenance

50 100 200 500 1,000 2,000

0.259 0.183 0.130 0.082 0.058 0.041

0.718 0.721 0.722 0.720 0.721 0.717

0.681 0.679 0.679 0.683 0.683 0.682

0.717 0.715 0.710 0.716 0.718 0.711

0.657 0.657 0.653 0.653 0.659 0.661

𝛿(𝑛) shown for the sepsis candidate set. Two thousand resamples per cell.

the uniform deviation and the realized tail probability of the admitted strategies. Both the premise and the conclusion hold well inside their nominal level. The uniform deviation falls within 𝛿(𝑛) in 97.8% to 99.5% of resamples against a nominal 95%. The realized tail probability of the admitted strategies stays within the certified excess in 99.6% to 100%. The ratio is flat in 𝑛: within each environment it varies by less than one percentage point over the fortyfold range of sample size. It runs from 0.717 to 0.722 in sepsis and from 0.653 to 0.661 in the maintenance environment. A constant ratio confirms the rate: the observed deviation shrinks at exactly the 𝑛−1∕2 rate the bound asserts, and only the constant differs between environments. The largest single deviation exceeds 𝛿(𝑛) in every environment, by factors between 1.11 and 1.39. A bound holding with probability 1 − 𝜂 produces exactly this: at two thousand resamples roughly one hundred are permitted to exceed it, and the maximum over those is above it by construction. The guarantee therefore concerns the satisfaction rate, not

M. Kim and B. Jang: Preprint submitted to Elsevier

Page 35 of 43

Chance-Constrained Selection from Counterfactual Estimates

the maximum. At the four largest sample sizes, the realized tail probability of the admitted strategies stays within the certified excess in 100% of resamples in three of the four environments. Table S11 An estimated kernel in place of the exact one, sepsis environment (mean ± s.d., 10 seeds, 7 budgets). Both rules act on the same estimated quantities and are scored against the same mean-feasible oracle; the chance-constraint rule is at 𝜀 = 0.2. sup𝑏 |𝐹̂𝑔 − 𝐹𝑔 | is the maximum over the 21 strategies and 10 seeds. Point-estimate rule

Chance-constraint rule

Traj.

Cost bias (%)

sup𝑏 |𝐹̂𝑔 − 𝐹𝑔 |

Regret (pp)

Viol. (%)

Regret (pp)

Viol. (%)

Tail∕𝜀

1,000 5,000 20,000 100,000

−9.03 ± 9.71 +2.89 ± 3.88 −0.48 ± 1.01 −0.06 ± 0.63

0.294 0.097 0.064 0.037

4.85 ± 4.45 2.89 ± 0.96 0.72 ± 1.12 0.57 ± 0.61

18.6 2.9 0.0 0.0

16.37 15.36 15.65 16.33

0.0 0.0 0.0 0.0

0.697 0.645 0.628 0.579

Table S12 Robustness to benchmark construction (MIMIC-IV semi-synthetic at budget 𝐵 = 12; 15% injected error; mean ± s.d., 10 instances). Difficulty set by the outcome-threshold quantile 𝑞; covariate rows use disjoint physiological groups. Naive

Chance (𝜀 = 0.2)

Configuration

Regret

Viol. (%)

Regret

Viol. (%)

Zero-viol. regret

𝑞 = 0.25 (baseline) 𝑞 = 0.15 𝑞 = 0.35 Covariates: RR, MAP, creatinine Covariates: SpO2 , lactate, WBC

3.4±1.0 2.9±1.1 3.3±0.7 3.5±1.1 3.7±0.8

9.9 13.2 8.4 8.6 9.3

12.6±1.7 14.0±2.2 10.4±1.3 12.6±1.8 13.0±1.4

0.0 0.0 0.0 0.0 0.0

8.9 9.9 7.5 9.0 9.5

Regret in outcome percentage points. The last column is the lowest regret reached along the tolerance frontier while the violation remains at zero.

Table S13 Target-based operating point at two violation targets 𝜌, averaged over the budget grid (mean±s.d., 10 seeds; regret in outcome percentage points, violation in percent). 𝜌 ≤ 5%

𝜌 ≤ 10%

Environment

𝜀

Regret

Viol.

𝜀

Regret

Viol.

Sepsis Tumor Semi-synthetic (MIMIC-IV) Maintenance (C-MAPSS)

0.36 0.28 0.40 0.22

7.7±0.7 2.3±0.6 5.7±0.6 9.4±0.2

1.7±1.5 4.9±1.3 0.8±0.8 1.0±0.4

0.40 0.30 0.45 0.25

4.9±0.7 0.6±0.5 3.6±0.6 9.4±0.2

6.0±4.0 9.7±1.9 8.2±4.9 5.3±1.3

S8. The maintenance environment at a budget of three Section 6.3 reports that the whole matched-tail gap in the maintenance environment comes from a single binary choice at a budget of three. The per-budget breakdown accounts for it. The averaged regret is the mean of the per-budget regrets of Table S14. At a tolerance of 0.30 the budget of three contributes 56.24 and the four larger budgets contribute 0.08 between them. Averaging over the six budgets that admit a strategy therefore gives 9.39, the observed value at that tolerance and the top of the plateau running from 9.33 to 9.39. Below 0.30 a budget of two admits nothing and five budgets are averaged instead, raising the figure to 11.27 against an observed 11.27 to 11.31. The plateau rises across that range because the residual at the larger budgets grows as the M. Kim and B. Jang: Preprint submitted to Elsevier

Page 36 of 43

Chance-Constrained Selection from Counterfactual Estimates

tolerance tightens, reaching 0.70 at a tolerance of 0.20. Setting that residual to zero and reading the average as one error divided by the number of budgets would understate it by the same amount. The tolerance at which the rule begins to admit t0.65 is displaced from its exact value by estimation noise. Under the exact cost distribution t0.65 exceeds a budget of three with probability 0.248, so a rule with perfect information would admit it above that tolerance. The observed regret instead collapses between 0.40 and 0.45. A fraction 0.371 of t0.65’s cumulative cost falls at exactly three crew-hours, where the 15% multiplicative perturbation of Section 5.7 pushes it above the budget half the time. The tail the rule estimates is therefore about 0.248 + 0.371∕2 = 0.43, and the observed transition brackets that value. Discretely valued cost concentrated at the budget makes the rule conservative, the direction Proposition 3 permits. Table S14 Per-budget regret in the maintenance environment, chance constraint at 𝜀 = 0.30 against the conditional-value-at-risk constraint at 𝜀 = 1. Budget Mean-feasible strategies Chance-constraint regret (pp) CVaR regret (pp) Realized tail, chance Realized tail, CVaR

2

3

4

5

6

7

1 0.00 0.00 0.080 0.080

2 56.24 2.14 0.006 0.238

3 0.05 0.06 0.088 0.072

4 0.02 0.03 0.168 0.106

5 0.01 0.02 0.168 0.101

5 0.01 0.01 0.225 0.160

Ten seeds. The two operating points carry almost the same averaged tail, 0.122 and 0.126, and allocate it differently across budgets.

Section 6.3 also reports how often each rule declines to recommend. In sepsis and tumor, both always recommend. In the other two, the conditional-value-at-risk rule declines more often at every tolerance, in 8.6% of cases against 2.9% at 𝜀 = 0.05 in the semi-synthetic environment and 32.6% against 20.7% in the maintenance environment.

S9. Mis-specification of the cost spread Section 6.5 perturbs the level of the estimated cost. This sweep perturbs its spread. It rescales each strategy’s oracle cost samples about their own mean by a common factor 𝑠, selecting on the rescaled samples and evaluating on the true ones. Ten seeds, 2,000 repetitions per budget, 200 cost draws per repetition, 𝜀 = 0.2, and the same budget grids, candidate sets and injected error as the level sweep. Table S15 reports the sweep for 𝑠 ≤ 1. Cost is bounded below at zero, so the rescaling is clipped there, and for 𝑠 > 1 the clip raises the realized mean by between 8 and 76 percent. The point-estimate rule’s violation does fall on that side, but it falls because the mean it reads has been inflated and not because it is insensitive to the spread. Those rows therefore do not measure what the sweep is for. The construction cannot raise the spread while preserving the mean on a non-negative cost. Overstating the spread also moves the chance-constraint rule toward conservatism, which the realized tail on that side confirms, so understatement is the direction that bears on the guarantee. Levels here are not directly comparable with Table 8. This sweep pools individual recommendations across the budget grid, so each budget carries weight in proportion to the draws at which the rule admits. The main text instead averages the per-budget means equally. The two weightings agree wherever every budget admits in every draw. That covers all four environments except the maintenance budget of two, where the chance rule admits in about half the draws. The maintenance level here is consequently about one point higher. Comparisons across 𝑠 within the table are unaffected, since the budget set and the weighting are the same at every 𝑠.

S10. Function approximation on the continuous-state environments S10.1. Tumor Two estimators of different flexibility are fit by g-computation on observational tumor trajectories, and both decision rules act on the resulting estimates. The oracle is mean-feasible and scores both. Six budgets and 10 seeds give 60 observations per cell, reported in Table S16. Cost bias is the mean over strategies of the relative error in each strategy’s estimated mean cost. M. Kim and B. Jang: Preprint submitted to Elsevier

Page 37 of 43

Chance-Constrained Selection from Counterfactual Estimates Table S15 Cost spread mis-specification, 𝑠 ≤ 1 (𝜀 = 0.2, 10 seeds). Point estimate Environment

Chance constraint

𝑠

Reg.

Viol.

Reg.

Viol.

Realized tail

Sepsis

0.50 0.70 0.85 1.00

2.50 2.50 2.50 2.50

7.91 7.91 7.91 7.91

11.24 13.73 15.56 16.92

0.00 0.00 0.00 0.00

0.184 0.136 0.112 0.095

Tumor

0.50 0.70 0.85 1.00

0.52 0.52 0.52 0.52

15.00 15.00 15.00 15.00

5.64 7.55 9.38 11.14

0.08 0.03 0.02 0.02

0.154 0.134 0.119 0.105

Semi-synthetic

0.50 0.70 0.85 1.00

3.50 3.50 3.50 3.50

7.22 7.22 7.22 7.22

9.68 11.96 13.51 14.94

0.00 0.00 0.00 0.00

0.206 0.164 0.138 0.115

Maintenance

0.50 0.70 0.85 1.00

0.39 0.39 0.39 0.39

5.23 5.23 5.23 5.23

9.39 9.41 9.44 10.38

0.96 0.48 0.31 0.24

0.075 0.070 0.066 0.061

Regret in outcome percentage points, violation in percent. The realized mean is preserved exactly at every row, so the point-estimate rule’s admitted set is identical across 𝑠 within each environment.

Table S16 Two estimators under function approximation, tumor environment (mean over 10 seeds; chance-constraint rule at 𝜀 = 0.2). The deviation column is the largest, over strategies, of each strategy’s mean deviation over seeds. Per-seed maxima are not retained for this environment, so the column understates the supremum. The certified slack is 𝛿 = 0.131. Deviation

Point-estimate rule

Chance rule

Traj.

Estimator

Cost bias (%)

Largest

>𝛿

Regret (pp)

Viol. (%)

Regret (pp)

Viol. (%)

1,000

ensemble ridge ensemble ridge ensemble ridge ensemble ridge

−12.56 +9.25 −13.24 +8.57 −14.30 +8.86 −14.73 +8.97

0.210 0.107 0.226 0.103 0.252 0.103 0.276 0.104

5∕23 0∕23 5∕23 0∕23 6∕23 0∕23 7∕23 0∕23

0.59 0.73 0.73 0.73 0.73 0.73 0.73 0.73

15.0 0.0 18.3 0.0 10.0 0.0 16.7 0.0

10.34 14.06 9.39 13.92 8.10 13.92 10.04 14.36

0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0

5,000 20,000 100,000

The rank correlation between estimated and exact mean cost is 1.000 for both estimators at every sample size. The estimators therefore disagree on the level of the cost, not on the ordering. The certified slack at this cost-sample size and candidate-set size is 𝛿 = 0.131. The ridge stays below it at every strategy. The ensemble exceeds it at five to seven of the 23, entirely at the strategies where its cost bias is largest.

S10.2. Semi-synthetic Table S17 reports the same estimators on the semi-synthetic environment, where the cost-underestimate band is the tightest of the four. The certified slack is 𝛿 = 0.131 and the certified ceiling 𝜀 + 𝛿 = 0.331. The deviation column is again the largest per-strategy value and understates the supremum. This grid runs to 20,000 trajectories rather than 100,000.

M. Kim and B. Jang: Preprint submitted to Elsevier

Page 38 of 43

Chance-Constrained Selection from Counterfactual Estimates Table S17 Two estimators under function approximation, semi-synthetic environment (mean over 10 seeds; chance-constraint rule at 𝜀 = 0.2). Realized tail is that of the strategy the chance rule selects, on oracle cost samples. Deviation

Point-estimate rule

Chance rule

Traj.

Estimator

Cost bias (%)

Largest

>𝛿

Regret (pp)

Viol. (%)

Viol. (%)

Realized tail

1,000

ensemble ridge ensemble ridge ensemble ridge

+2.17 −5.30 +2.52 −4.99 +2.51 −5.30

0.150 0.260 0.147 0.262 0.147 0.261

4∕25 16∕25 4∕25 15∕25 4∕25 16∕25

2.62 −5.73 2.43 −5.55 2.56 −5.67

6.7 95.0 6.7 93.3 6.7 96.7

0.0 15.0 0.0 13.3 0.0 11.7

0.171 0.329 0.168 0.323 0.169 0.324

5,000 20,000

The ridge aggregate bias of −5.3% is the mean of per-strategy biases running from −28% to +79%, which cancel. Its cost error does not track how well the observational data covers the states a strategy visits. The coverage gap of a strategy is the fraction of the discretized state-action cells it traverses in which the behavior policy generated no samples. The cost error of a strategy is the ratio of its estimated to its true mean cost, minus one, in absolute value. Under 𝑘-means discretization at 𝑏 = 5, the Spearman correlation between the absolute relative cost error and the coverage gap is 0.05 for the ridge and 0.75 for the ensemble. Repeating it under uniform and quantile discretization at the same cell count separates the two estimators by sign rather than by magnitude. The ensemble stays positive at 0.81 and 0.33, while the ridge moves to 0.26 and −0.20. Coverage predicts where the ensemble errs under every partition and predicts nothing about the ridge under any. The association is in rank rather than in level, since a few strategies at the sparsely covered ends of the grid account for it. The two estimators exceed the slack on different strategies, at a Jaccard index of 0.18. The ridge’s worst strategies sit in the best-covered region of the grid.

S11. Safety-utility frontiers in the remaining environments Figure S1 traces the frontier of Section 6.6 in the three environments other than sepsis. Each panel is obtained by sweeping the tolerated violation probability 𝜀 under 15% injected estimation error, with the naive and upper-bound rules placed as reference points. Tumor

Semi-synthetic

Outcome regret (pp)

40 (a)

Maintenance

(b)

(c)

30 20 10 0

Chance (ε sweep) CVaR (ε sweep) Naive Upper bound

−10 0

20

40

60

80

100

0

Budget violation (%)

20

40

60

Budget violation (%)

80

0

20

40

60

80

100

Budget violation (%)

Figure S1: Safety-utility frontiers outside the sepsis environment, traced by sweeping the tolerated violation probability 𝜀 under 15% injected estimation error, with the naive and upper-bound rules as single points for reference. The exchange is monotone in all three. The tumor and maintenance frontiers step rather than curve, because most of their candidates are tied at a single outcome value. (a) Tumor, 10 seeds; (b) Semi-synthetic, 10 benchmark instances; (c) Maintenance, 10 instances.

M. Kim and B. Jang: Preprint submitted to Elsevier

Page 39 of 43

Chance-Constrained Selection from Counterfactual Estimates

S12. Candidate-set placement The candidate sets in the simulators are solved rather than enumerated. For the sepsis simulator, value iteration is applied with a per-step penalty 𝜆 on the quantity of treatment administered. The solved policy therefore maximizes survival net of a treatment cost weighted by 𝜆. At 𝜆 = 0, the policy maximizes survival without regard to treatment cost, recovering the aggressive end of the spectrum, while large 𝜆 yields conservative policies that treat sparingly. The discount factor is 0.99, iteration runs to convergence, and absorbing states are treated as self-absorbing during value iteration. The candidates can then be spaced along the cost axis in three ways, by even spacing in 𝜆, by even spacing in target cost, or by a denser placement. The rest of this appendix reports what turns on that choice. Section 6.2 reports the decision rules on a candidate set of 21 value-iteration strategies. This appendix repeats the comparison on the seven-strategy set a deployment would more plausibly enumerate, holding the budget grid, the injected error, the tolerance and the 10 seeds fixed. The only difference is then how densely the space of strategies is populated. Table S18 reports the comparison. Table S18 Decision rules under two candidate-set constructions (sepsis, 15% injected error, 𝜀 = 0.2; mean ± s.d., 10 seeds). Regret is measured against the oracle optimum of the same construction. Construction

Strategies

Oracle outcome (%)

Naive regret

Naive viol. (%)

Chance regret

Practice-sized Dense

7 21

46.2±0.4 54.8±0.7

0.4±0.4 2.5±0.7

4.6±2.4 8.0±3.5

14.3±0.5 16.9±0.7

The chance-constraint rule violates the budget in 0.00% of cases under both constructions. The dense construction yields 21 distinct strategies from 25 penalty values.

Three readings follow. The first is that the separation between the rules survives both constructions. The naive rule overruns the budget under each, and the chance-constraint rule holds at exactly zero under each. The result of Section 6.2 is therefore not an artifact of how many strategies are enumerated. The second is that density raises the point-estimate rule’s exposure rather than lowering it. Its violation rises from 4.6% to 8.0% as the set grows. A denser set contains more strategies with a cost distribution that places its mean inside the budget and a substantial part of its mass outside. Each of those is an opportunity for a point estimate to admit a strategy that a tail constraint would refuse. The third concerns how the regret figures should be read. The chance-constraint rule’s regret rises from 14.3 to 16.9 points across the two constructions, which taken alone would suggest the rule performs worse on the denser set. It does not. Regret is measured against the oracle optimum of the same construction, and that optimum rises from 46.2% to 54.8%. In absolute terms, the strategy the chance-constraint rule selects improves from 31.8% to 37.9%. The denser set does not refine the same operating points; it contains genuinely better ones, and the rule finds them. The benchmark improves faster than the recommendation does.

S13. Estimated-kernel diagnostics, sepsis This appendix records supporting detail for the estimated-kernel block of Section 6.5. The falling unvisited fraction of Section 6.5 shows Assumption 2 failing at small samples and holding at larger ones. The assumption asks for positive probability on the treatments a strategy may assign along histories that can occur, not for coverage of the state space. At a hundred thousand trajectories 58.8% of state-action pairs remain unvisited, and only 0.04% of transitions in the evaluation rollouts are drawn from them. The unvisited region lies outside the candidate strategies’ support, and the assumption requires that and no more. The deviation column of Table S11 is the 𝛿 of Proposition 3, measured under model error rather than assumed. At the cost-sample size used throughout Section 6, the certified slack is 𝛿 = 0.1297. From 5,000 trajectories the measured deviation stays below it at every strategy and every seed, so model error does not dominate the sampling slack the guarantee already carries. At 1,000 trajectories it exceeds the slack at 17 of the 21 strategies, and the 𝜀 + 𝛿 ceiling does not cover that regime. The two terms combine rather than substitute. A deployment that estimates the cost distribution from a fitted model and then evaluates it on 𝑛 draws carries both. Proposition 3 then applies with 𝛿 replaced by their M. Kim and B. Jang: Preprint submitted to Elsevier

Page 40 of 43

Chance-Constrained Selection from Counterfactual Estimates

sum, the sampling slack of 0.1297 at 200 draws plus the measured model deviation of 0.0638 at 20,000 trajectories. Section 6.4 measures the sampling term on its own, at a mean of 0.72 𝛿 in this environment. Figure S2 shows where the error matters. Points below the identity line are conservative: the rule judges a strategy riskier than it is and refuses it, which costs outcome and breaches nothing. Only the shaded quadrant produces a violation. It contains five of the 147 strategy-budget pairs at 1,000 trajectories and none at any larger size. The five pairs sit at exact tail probabilities between 0.21 and 0.25 against estimated values between 0.13 and 0.18. The failure at small samples is therefore a shallow underestimate near the tolerance rather than a gross misordering. One comparison qualifies the injected-noise design. At 1,000 trajectories the estimated kernel biases cost by −9.03% on average, and a uniform injected underestimate of 10% leaves the point-estimate rule’s violation at zero. The estimator produces 18.6%. A fitted model errs unevenly across strategies, with a standard deviation of 9.71 points around that mean. It errs on the outcome as well, and it distorts the shape of the cost distribution rather than rescaling it. A uniform multiplicative bias of the same average magnitude is therefore an optimistic stand-in. The sweep in Figure 4(b) locates the breaking point of the rule rather than of a deployment.

Estimated tail Pr̂ (Cg > B)

1.0

(a) n = 1, 000

(b) n = 5, 000

0.8 0.6 0.4 0.2 violation zone

0.0

Estimated tail Pr̂ (Cg > B)

1.0

5 in zone

(c) n = 20, 000

0 in zone

(d) n = 100, 000

0.8 0.6 0.4 0.2 0.0 0.00

0 in zone

0.25

0.50

0.75

True tail Pr(Cg > B)

1.00

0 in zone

0.00

0.25

0.50

0.75

True tail Pr(Cg > B)

1.00

Figure S2: Estimated against exact tail probability of the cumulative cost, for each of the 21 candidate strategies at each budget, under a kernel estimated from observational trajectories. Points in the shaded quadrant (red) have an estimated tail inside the tolerance 𝜀 = 0.2 and a true tail outside it. That quadrant is the only route by which the chance constraint admits an infeasible strategy. The quadrant empties between 1,000 and 5,000 trajectories. (a) 𝑛 = 1,000; (b) 𝑛 = 5,000; (c) 𝑛 = 20,000; (d) 𝑛 = 100,000.

The estimator errs in the level of cost and not in its order: the rank correlation between estimated and exact mean cost is 0.999 at 1,000 trajectories and 1.000 at every larger size. The level is biased by −9.03%.

S14. Validation of the tabular kernel rollout The estimated-predictor experiment of Section 6.5 rolls candidate strategies forward under a transition kernel estimated by counting. Before any estimated number was produced, the vectorized kernel rollout was checked against the original object-based rollout of the simulator. Two defects were found and corrected in the course of that check, and both would have gone unnoticed in aggregate summaries. The first is the hidden mixture component. The marginal kernel formed by weighting the two component transition matrices by their mixture probabilities is not the generative process. The latent binary attribute is drawn once per episode and then held fixed. The two components differ by as much as 0.51 in transition probability. Rolling out under the marginal kernel understated the outcome value by up to 64 percentage points. M. Kim and B. Jang: Preprint submitted to Elsevier

Page 41 of 43

Chance-Constrained Selection from Counterfactual Estimates

The second is terminal scoring. The simulator terminates an episode on a nonzero reward and treats episodes that reach the horizon without entering an absorbing state as favorable. Scoring the outcome only at absorbing states misclassified every horizon-reaching episode as unfavorable. After both corrections, Table S19 compares the two implementations at three to four thousand rollouts per strategy. Table S19 Vectorized kernel rollout against the original object-based rollout. Strategy vi_l0.000 vi_l0.050 vi_l0.200

Outcome, original

Outcome, vectorized

Cost, original

Cost, vectorized

0.9637 0.6428 0.2142

0.9595 0.6295 0.2110

33.587 6.898 0.899

34.566 6.817 0.838

The outcome values agree within two standard errors for every strategy. A six-seed repeat at vi_l0.000 gives a cost difference of +0.156 against a two-standard-error width of 0.379, so the cost agreement is within noise. A residual gap of roughly half a percentage point in the outcome persists because the tabular kernel marginalizes the simulator’s factored dynamics. That is a property of the tabular representation and not of the estimator. The observational trajectories used to fit the kernel are generated under a behavior policy with treatment intensity increasing with the number of abnormal physiological variables. This reproduces the confounding pattern that motivates the identification assumptions of Section 3.1. Its mean treatment intensity is 1.601 interventions per episode.

S15. Importance-weight diagnostics on the real-outcome environment Table S20 reports, for each candidate policy in the Drink Less environment, the effective sample size and the largest importance weight of the value estimate. The randomization probabilities are fixed by the trial design and the candidate policies are softened away from the deterministic boundary. The weights are therefore bounded above by the ratio of the largest attainable candidate probability to the smallest trial probability. The observed maximum of 2.375 attains that bound, and no policy approaches degeneracy. The cost distribution is not estimated by importance weighting: it is accumulated directly from each participant’s observed state sequence under the candidate policy. The chance constraint therefore does not inherit the weight variance that the value estimate carries. The state-aware policies sit further from the trial’s randomization probabilities than the global ones at comparable cost. This lowers their effective sample size and raises their value at the same time. The policy the chance-constraint rule selects at 𝐵 = 15 retains an effective sample size of 58.9% of the decision points. The selection is stable: across the 50 selection-estimation splits of Section 6.8 it is chosen every time, with a cross-fitted value of 2.627, matching the full-sample value.

S16. Alternative formulations and the distributionally robust form A worst-case constraint would require every realized cost to stay within budget. An overrun, however, is a resource event rather than a catastrophe, and the worst realized cost is an unstable target in finite samples. A multi-objective formulation would return a Pareto set, where sweeping 𝜀 traces the same family while keeping the budget as a fixed ceiling. The distributionally robust form is closer. Suppose the constraint is imposed on the worst case over the ambiguity set 𝑟 (𝐹̂𝑔 ) = {𝐹 ∶ ‖𝐹 − 𝐹̂𝑔 ‖∞ ≤ 𝑟} of distributions within Kolmogorov distance 𝑟 of 𝐹̂𝑔 , rather than on 𝐹̂𝑔 itself. Every such 𝐹 satisfies 𝐹 (𝐵) ≥ 𝐹̂𝑔 (𝐵) − 𝑟. The bound is attained by the distribution that moves mass 𝑟 from just below the budget to just above it, so the robust constraint at tolerance 𝜀 reduces to ( ) sup 1 − 𝐹 (𝐵) = 1 − 𝐹̂𝑔 (𝐵) + 𝑟 ≤ 𝜀 ⟺ 1 − 𝐹̂𝑔 (𝐵) ≤ 𝜀 − 𝑟, (26) 𝐹 ∈𝑟 (𝐹̂𝑔 )

the empirical rule of Equation (10) at the tightened tolerance 𝜀 − 𝑟. The robust rule is therefore the same rule at a different operating point. Sweeping 𝜀 traces the robust frontier along with the nominal one, and the choice of radius becomes a choice of position on a curve the method already reports. Taking 𝑟 = 𝛿 recovers Proposition 3 of the main text and its certified ceiling of 𝜀 + 𝛿. The equivalence is specific to an ambiguity set defined in the supremum norm. M. Kim and B. Jang: Preprint submitted to Elsevier

Page 42 of 43

Chance-Constrained Selection from Counterfactual Estimates Table S20 Per-policy importance-weight diagnostics, Drink Less environment (10,470 decision points, 349 participants). ∗ marks the policy selected by the chance-constraint rule at 𝐵 = 15, 𝜀 = 0.2. Policy

Mean cost

Value

ESS

ESS (%)

Max weight

global_r0.0 global_r0.1 global_r0.2 global_r0.3 global_r0.4 global_r0.5 global_r0.6 global_r0.7 global_r0.8 global_r0.9 global_r1.0

1.5 3.0 6.0 9.1 12.1 15.1 18.0 21.0 24.0 27.0 28.5

1.23 1.36 1.63 1.90 2.17 2.44 2.70 2.97 3.24 3.51 3.64

4654 5148 6296 7623 8977 10051 10470 10050 8967 7599 6912

44.5 49.2 60.1 72.8 85.7 96.0 100.0 96.0 85.6 72.6 66.0

2.375 2.250 2.000 1.750 1.500 1.250 1.000 1.167 1.333 1.500 1.583

aware_h0.8_l0.0∗ aware_h0.8_l0.2 aware_h1.0_l0.0 aware_h1.0_l0.2 aware_h0.8_l0.4 aware_h1.0_l0.4

12.8 15.1 15.1 17.4 18.1 20.3

2.63 2.75 2.91 3.03 2.91 3.19

6168 7427 5617 6636 8985 7841

58.9 70.9 53.6 63.4 85.8 74.9

2.375 2.000 2.375 2.000 1.500 1.583

Balls defined in a transport distance do not reduce to a shift of the tolerance, because they can move mass across the budget by an unbounded amount.

S17. Predictor ceiling: the estimation share of the regret Section 6.5 reports how much of the chance-constraint rule’s regret is attributable to the estimated predictor rather than to the constraint. The comparison runs the same rule on three cost distributions per strategy and differences the results within seed. The estimated arm is the predictor’s own distribution and reproduces the reported numbers. The oracle arm is the exact counterfactual distribution, which bounds what any predictor could deliver to the rule. Nothing trained for the decision can improve on the true cost law. The calibrated arm rescales the estimated distribution by a single pooled factor per environment and seed so that its mean matches the truth. It stands in for what a decisionaware level correction could learn. The factor is pooled over candidate strategies and never fitted per strategy. A perstrategy factor would use oracle information, so the arm would no longer stand in for a learnable correction. Feasibility is evaluated against the true cost samples in every arm. A budget-by-tolerance cell is dropped from all three arms whenever any one of them admits nothing, so the arms are always compared on the same cells. In sepsis, the estimated arm is ahead of the oracle arm at every tolerance up to 0.3. Its regret is below the exact distribution’s by 0.77 points at a tolerance of 0.2, with a paired bootstrap interval of [−1.06, −0.51] over 10 seeds. At the tighter tolerances the gap is 0.29 to 1.01 points, with budget violation at zero throughout. A predictor with no error would make the rule more conservative here rather than less. Estimation therefore does not separate the rule from the benchmark in this environment, and integrated training would have nothing to recover. Under function approximation, the picture reverses. The exact distribution lowers the regret by 2.14, 4.29 and 4.76 points against the ensemble at tolerances of 0.05, 0.10 and 0.20. Against the ridge it lowers the regret by 2.40, 5.12 and 9.95, every interval excluding zero. For the ensemble, five of the thirty budget-by-tolerance cells are undefined in 9 of the 10 seeds because one of the arms admits nothing there. The ensemble comparison therefore runs over the remaining 255 cell-seed pairs. Rescaling each estimated distribution by a single pooled factor so that its mean matches the truth leaves the ridge selecting the same strategy in every cell. The recovered share is zero. It moves the ensemble the wrong way, raising its regret by 11.51 points at a tolerance of 0.2. There its bias is an underestimate of 14 percent, and the corrected distributions lead the rule to select cheaper strategies with lower outcome. The factors are 0.93 to 0.95 and 1.18 to 1.36, the sizes the measured biases imply. M. Kim and B. Jang: Preprint submitted to Elsevier

Page 43 of 43

Related documents

• AD-WM: Action-Discriminative World Models for Counterfactual Model Predictive Control#1078335
• Semiparametric Inference for Counterfactual Regression under Intervention-Driven Shift#925762
• On the Graphical Rules for Recovering the Average Treatment Effect Under Selection Bias#612760
• On the Graphical Rules for Recovering the Average Treatment Effect Under Selection Bias#611283
• A framework for maximizing the benefit from retaining regrowth on private land#1133231