The Impact of Operational-Data Fidelity when Assessing Safety-Critical Autonomous-Vehicle Software

1

The Impact of Operational-Data Fidelity when Assessing Safety-Critical Autonomous-Vehicle Software

arXiv:2608.10025v1 [cs.RO] 9 Aug 2026

Kizito Salako, Rabiu Tsoho Muhammad

Abstract—For safety–critical software, data from the software’s operational past (e.g. a sequence of success and failure events experienced by the software) can provide strong statistical support for reliability claims about the software. However, such data might not describe past software failure events in sufficient detail, and this might leave a reliability assessment (based on this data) unable to account for important features of past software failures. In this paper, by extending conservative Bayesian inference (CBI) techniques used in reliability assessment, we illustrate a principled statistical approach for checking the robustness of reliability claims derived from insufficiently detailed operational data. We demonstrate the extent to which insufficient detail in operational data can undermine software reliability claims in autonomous vehicle (AV) safety assessment scenarios. Reliability claims derived from insufficiently fine-grained data might be dangerously optimistic, despite a concerted effort by an assessor to use such data conservatively during the assessment. While these findings are consistent with previous work on the impact of statistical model fidelity in Bayesian software reliability assessments, our work clarifies why attempts to use low-fidelity data conservatively can be naı̈ve, and we give the first conservative estimates of the impact of data fidelity on assessments. Index Terms—software reliability assessment, conservative Bayesian inference, reliability bounds, operational data fidelity, autonomous vehicle safety

L IST OF A BBREVIATIONS AND S YMBOLS ADS AEB ASIL AUC AV CBI CEHSS FP, FN FTDSS GNSS IMU HARA i.i.d. ODD OOD pfc PK ROC RSS SOTIF SW

automated driving system autonomous emergency braking automotive safety integrity level area under the curve autonomous vehicle conservative Bayesian inference critical-event handling support system false positive, false negative fault-tolerant decision support system global navigation satellite system inertial measurement unit hazard analysis and risk assessment independent and identically distributed operational design domain out of distribution probability of failure per classification evidence-based prior knowledge receiver operating characteristic responsibility-sensitive safety safety of the intended functionality software

K. Salako and R. Muhammad are with the Centre for Software Reliability, City St. George’s, University of London, Northampton Square EC1V 0HB, U.K. (email: {k.o.salako,rabiu-tsoho.muhammad}@city.ac.uk)

P Q p, q Θ Ω D Si or S1 , S2 n k1 k2 k n1 n2 a b b1 l l1 l2 Li L∗i , Li∗ pi , q i Φ∗ θej , pej , qej η

unknown probability of FP classification unknown probability of FN classification generic values of the random quantities P, Q unknown probability of correct classification sample space of feasible (P, Q) pairs set of admissible prior distributions satisfying the assessor’s PK constraints regions in the partition of Ω induced by the PKs number of observed classifications number of observed FPs number of observed FNs total number of observed failures number of successes before first failure number of successes needed after first failure assessor’s prior confidence that the classifier meets the engineering target claimed upper bound on pfc; engineering target lower bound on classifier accuracy lower confidence–bound on pfc practical lower limit on P practical lower limit on Q likelihood value at (pi , qi ) ∈ Si extrema of Li over the relevant regions of Si support-point coordinates of a worst-case prior conservative posterior-confidence infimum / worst-case posterior confidence transition probabilities to success, FP, and FN from state j, j = 1, 2, 3 confidence/weight assigned to the i.i.d. assumption in the Markov-model sensitivity analysis

I. I NTRODUCTION OFTWARE is required to be very reliable if its purpose is to take actions towards ensuring safety: sufficiently reliable software will take correct actions when needed, sufficiently often. When assessing such software, an assessor can use statistical inference to draw conclusions about whether the software is sufficiently reliable. The assessor does this by applying a statistical model (of the software’s failure behavior) to data gathered from the software’s operational past. This model must be appropriate—in particular, the model’s fidelity must be compatible with the level of detail in the operational data. For example, for on-demand systems, the data can be an enumeration of the outcomes of the software’s past actions, indicating whether an action was correctly taken or the software failed. An appropriate statistical model to use with such data might be a parametric family of generalized

S

2

Bernoulli random processes, if the software’s failure behavior can be argued to be adequately modeled by such processes. Ideally, data from past software operation is gathered for a particular assessment that uses a particular statistical model. Realistically, this is often not the case. Gathering the data at the required fidelity might be too expensive (e.g. gathering the data requires an infeasible amount of human judgment, time, or is unaffordable in monetary terms), or the gathered data was intended for a different statistical model, or for a different purpose altogether, or the desired fidelity is unobtainable. Consequently, the data granularity can vary significantly, depending on the circumstances surrounding the data collection. The data might be “coarse–grained”; for instance, it only indicates when successes/failures of the software occurred, but not the nature or type of each success/failure. If the software were, say, a binary classifier that should raise an alarm when unsafe driving conditions occur while traveling in an autonomous vehicle (AV), it would be important to know what failure mode occurred in addition to knowing that a failure has occurred; for instance, whether the failure was a false positive (FP) or false negative (FN) classification of the driving event. By omitting such details, coarse–grained reliability data can leave an assessor with some uncertainty about the software’s past failure behavior; uncertainty that should be explicitly incorporated into any reliability claims (based on this data) about the software’s future failure behavior. An assessor’s uncertainty about the software’s operational past should temper any claims of the software being reliable. What is the impact—of using operational data with missing detail and the uncertainty this brings—in reliability assessments? When is there little impact: reliability claims are as trustworthy as claims made had the data been more detailed? When is the impact significant: claims are dangerously optimistic, leading to misplaced trust in unreliable software? This paper presents statistical techniques that conservatively estimate this impact. These novel extensions of conservative Bayesian inference (CBI) support evidence–based, conservative, reliability claims. They provide a means to conservatively check the robustness of reliability claims based on insufficiently detailed data. The paper’s research contributions are: 1) formally incorporating assessor uncertainty about failure data details into reliability assessments (see Section IV); 2) novel CBI techniques that account for this uncertainty and quantify the impact this has on the conservative reliability claims an assessor can justify (see Sections IV–VII); 3) four theorems, with and without failure data uncertainty, that give conservative posterior confidence bounds on a system’s probability of failure (see Section V). 4) demonstrating how attempts at conservatively using coarse operational data in assessments can lead to optimistic claims (see Sections VI, VII); 5) guidance for statistical AV–safety assurance under operational–data logging constraints (see Section VII). The paper’s outline is: Section II highlights related work, Section III describes the reference AV scenario, and Section IV details the CBI approach. Section V presents worstcase confidence bounds on classifier reliability under different degrees of uncertainty about the failure-types exhibited by the

classifier during operation. The confidence bounds are applied in Section VI. Section VII discusses practical considerations and implications, with concluding remarks in Section VIII. II. R ELATED W ORK A. AV Safety Standards The functional–safety baseline for production road vehicles is the ISO 26262 series (second edition, 2018). The standard targets hazards due to malfunctioning safety–related electrical/electronic subsystems and prescribes a risk–based safety lifecycle from conception through validation [1], [2]. Crucially, for software–intensive AV stacks (e.g. Fig. 1), ISO 26262 goes beyond system–level process guidance: it explicitly requires artifacts and evidence at the software– component level, including specification of software safety requirements, software unit verification, software integration and verification, and testing of embedded software [3]. It is reinforced by supporting process requirements for verification planning, change/configuration management, confidence in software tools and pre-existing software components (including “proven–in–use” style arguments); these emphasize explicit assessment/justification of safety–relevant software elements [4]. ISO 26262 uses hazard analysis and risk assessment (HARA) to rate hazardous events by severity, exposure, controllability, and to assign an Automotive Safety Integrity Level (ASIL) [2]. The ASIL rating sets the level of rigor and independence for development and assurance activities [5]. ISO 26262 is fault–oriented. Complementary standards address AV–relevant risks not directly attributable to component faults. ISO 21448 (SOTIF) provides an argument framework for avoiding unreasonable risk due to functional insufficiencies, and explicitly calls for design, verification and validation measures (and operational–phase activities) for automated– driving and emergency–intervention functions that depend on complex sensors and processing algorithms [6]. ISO/PAS 8800 provides a framework/guidance for building convincing safety assurance claims for artificial-intelligence (AI)-based safetyrelated systems within the vehicle. ISO/PAS 8800 tailors and extends ISO 26262 and ISO 21448 for AI elements [7]. Finally, ANSI/UL 4600 offers a deployment–facing structure for scrutinizing evidence used to justify safety claims for autonomy–relevant subsystems [8]. While not a standard, Assurance 2.0 argues that autonomous systems assurance must be more rigorous, evidence–centered, continuous/incremental, with explicit treatment of defeaters and probabilistic treatments of evidence [9]–[11]. B. (Conservative) Bayesian Methods for Assessments CBI supports conservative assessments of software reliability using operational data and evidence-based beliefs about the software’s propensity to fail/succeed [12]. This includes support for claims about fault–freeness/perfection and very small failure probabilities, using failure–free operational evidence and experience from previous similar systems [13]–[15]. Preliminary work applied CBI to binary classifier assessment [16], and to highlighting circumstances where demonstrations of desirable levels of reliability from AV operational testing are

3

Examples: obj / lane / traffic light classifiers

Examples: intent / cut-in classifiers

Examples: maneuver selection risk scoring

Core Nominal Driving channel (“Doer” channel) Sensors camera/lidar/radar GNSS/IMU, map

Perception

Prediction

Planning

detect & classify

intent & motion

behavior/trajectory

nominal setpoints

Fault-tolerant decision support system (FTDSS)

Validation Result

Monitoring channel (“Checker” channel)

selected setpoints

Decision / Selector

Control & Actuation

fallback setpoints

Fallback channel

Monitoring / safety assurance operational design domain (ODD) violation classifier; anomaly detector; action approval classifier; consistency checks; safety envelope / RSS-style checks; trajectory & setpoint validation

Fallback / minimal risk maneuver; critical–event handling support system (CEHSS); safe stop / pull-over

CriticalEvent / ODD exit

Fig. 1. An AV autonomy stack schematic indicating typical stages in the stack, applications of classifiers (i.e. white boxes) with a focus on the monitoring channel, and how a channel–wise Doer/Checker/Fallback safety architecture can gate actuation.

infeasible [17], [18]. CBI has been used to estimate the impact of erroneous modeling assumptions on assessments, such as incorrectly assuming statistically independent test outcomes during software testing [19], [20]. Bishop et al. [21] formalize “confidence bootstrapping” for AV assessment: staged deployment policies are coupled to CBI-derived lower bounds on failure–free operation over a finite future “confidence horizon”, so that risk can be contained early while evidence from safe operation tightens conservative predictions over time. Aghazadeh-Chakherlou et al. [22] address the problem that AVs evolve, extending previous CBI results for arguing post– change safety from pre–change safe operation under strong improvement assumptions, while [23] incorporates both pre– change operational evidence and partial prior confidence in fault-freeness, showing that earlier operation can materially improve post–change dependability claims when post–change data are scarce. [24] argues that extreme safety claims should explicitly account for inevitable epistemic doubt in assurance arguments, and show how conservative bounds during early operation can be strengthened by “fall–back” arguments. CBI aligns with robust Bayesian and imprecise-probability frameworks, which favor inferences that remain valid under plausible uncertainty about the inputs to inference [25]–[30]. C. Some AV Safety Assessment Approaches The infeasibility of demonstrating AV reliability from road testing alone has been argued using classical frequentist statistics [31]. Multivariate Bayesian inference (that explicitly models uncertainty about AV operating modes) has been proposed to assess AV safety using fleet operational data [32]. Scenario-based statistical testing involves sampling representative and/or challenging traffic scenarios that AVs may encounter [33]; however, Zhao et al. argue for such testing to have a stronger statistical foundation for stopping rules, residual-risk estimation, and claims about simulation fidelity [34]. Importance sampling and cross-entropy methods accelerate exposure to rare, safety-critical scenarios [35] and have been applied in end-to-end AV testing [36]. These approaches

typically require fully specified priors, high-fidelity typed data, or validated stochastic/simulation models. The present paper addresses a complementary problem: when operational logs are too coarse, what conservative claims remain justified? D. Some Binary Classifier Assessment Approaches Binary classifier assessment estimates Bernoulli error rates (e.g. FN and FP probabilities) from operational data. Empirical evaluation typically involves summary statistics of a classifier’s failure behavior; statistics including accuracy, sensitivity (true positive rate (TPR)/recall), specificity (true negative rate (TNR)), precision (positive predictive value), and F1 score (harmonic mean of precision and recall) [37]–[39]. Performance is frequently reported across thresholds via receiver operating characteristic (ROC) curves (TPR vs. false positive rate FPR) [40] and the associated area under the curve (AUC) [41]. Confusion matrices reveal trade-offs between failure rates and the types/frequency of failures/successes at different decision thresholds. However, ROC/AUC can be misleading under strong class imbalance, motivating complementary precision– recall (PR) curves and PR-focused summaries [39], [42]. Bayesian ROC methods propagate observational and distributional uncertainty into ROC/AUC trade-offs. A substantial literature treats ROC curves as random objects induced by prior distributions over class-conditional score distributions. Also, semi/non-parametric Bayesian constructions can be more robust than binormal-style assumptions while still supporting principled threshold selection and credible intervals [43]–[45]. Bayesian ROC also helps with “ground truth” uncertainty [46]. III. R EFERENCE AV S CENARIO A. From AV Driving Episodes to Operational Evidence Fig. 1 exemplifies a recursive AV driving pipeline. During a journey, the AV repeatedly senses, localizes, perceives, predicts, plans, checks, selects, controls, actuates, and then observes the consequences of its action. This perception– planning–control decomposition follows traditional AV software tasks [47]. The planning and decision-making context,

4

including safety and verification concerns, is consistent with [48]. The Doer/Checker/Fallback architecture follows [49]. A typical machine-learning (ML)-based nominal channel consumes camera, lidar, radar, global navigation satellite system (GNSS), inertial measurement unit (IMU), and map inputs; constructs fused scene representations; detects and classifies objects, lanes, drivable space, and traffic lights; predicts agent intent and motion; generates behavior or trajectory candidates; and produces nominal setpoints. In parallel, the checker channel evaluates the nominal channel’s intermediate and final outputs. The Doer/Checker/Fallback structure in Fig. 1 captures this arrangement: a monitoring subsystem and fault-tolerant selector can approve nominal setpoints, reject them, or switch to a minimal-risk fallback channel [49]. On each journey, this pipeline induces a sequence of operational episodes; e.g. an AV journey may often proceed through episodes such as: automated driving system (ADS) available? → in operational design domain (ODD)? → lane estimate reliable? → lead gap safe? → target lane-change gap acceptable? → proceed through signal? → trajectory approved? → fallback required?

For the statistical model in Section IV, a “classification” is a triggered decision by a specified classifier, in a specified ODD and operational-mode slice, with fixed software version, thresholds, sensor configuration, and ground-truth rule. An operational log row might record: event mode ODD speed stopline signal candidate classifier score output truth outcome

= intersection_approach = signalized = urban/day/dry = 11.8m/s = 42m = amber = proceed = Ctraj = 0.83 = intervene = unsafe_to_proceed = correct

From such information and counterfactual reasoning, the success/failure data required by our statistical model is obtained. B. Binary Classification in AV Safety The classifier instances in Fig. 1 are not limited to lowlevel perception labels. They include binary safety-relevant decisions embedded throughout the recursive driving process. A classifier Cj is invoked on a demand input, consisting of raw or intermediate AV state information, and outputs one of two labels: for example, safe/unsafe, in-ODD/out-of-ODD, allow/intervene, approve/reject, or nominal/fallback. Terrosi et al. explicitly characterize an automated-driving safety monitor that detects hazardous situations and triggers emergency braking as an “extended binary classifier” that classifies the system state as safe or unsafe and acts accordingly [50]. Reachabilitybased safety supervision similarly produces an activation signal indicating that the state has been deemed unsafe and that evasive action, such as braking, should be initiated [51]. At the perception–action boundary, an autonomous emergency

braking (AEB) function can be abstracted as a binary decision component—no action vs. emergency braking [52]. A runtime monitor that approves safe actions and blocks or replaces unsafe actions implements the same binary logic [53]. For our statistical model, fix the classifier Cj , the operational mode, and the interpretation of the positive label. If positive means unsafe/intervene, then an FP is an unnecessary intervention when the proposed action was actually safe, while an FN is an unsafe allowance when the proposed action should have been blocked. At each relevant ODD check, an ODD monitor outputs in-ODD/out-of-ODD. An anomaly or out-ofdistribution (OOD) detector outputs normal/OOD. An actionapproval classifier outputs OK/not-OK when the nominal channel proposes a trajectory or setpoint for execution. A notOK inhibits the proposal, after which the selector chooses an alternative (e.g. minimal-risk maneuver, controlled stop) [49]. So, a single AV journey generates several datasets of classifier outcomes on classification tasks: e.g. during ODD monitoring, lane-change attempts, intersection approaches, proposed trajectory validation, and degraded operation. In principle, these datasets yield success/failure counts for our model, after each binary classifier output has been compared with counterfactual ground truth and classified as a success, FP, or FN, ex post facto. Thus, the Fig. 1 pipeline provides enough data to classify FPs/FNs, but in practice this requires reliable ground truth and counterfactual reasoning about what would have occurred had the monitor not intervened [50], [54]. IV. M ETHODOLOGY The following statistical model is used in Section V to show the impact of data-fidelity on assessments. Unlike continuous-time software reliability models with failure events in continuous execution/calendar time (e.g. [55], [56]), the modeling framework adopted here is discrete and “demand”– indexed (cf. [17], [57]). This distinction aligns naturally with system type: continuous-time models are well suited to continuously operating systems (e.g. transaction-processing or telecommunications software), whereas safety-critical protection or decision functions—such as on-demand monitors or AV safety classifiers—are more appropriately modeled as discretedemand systems with reliability defined per actuation/decision. A. A Bayesian Model of Binary Classification A binary classifier receives a sequence of s-independent and identically distributed (i.i.d.) samples for classification. To be consistent with the i.i.d. assumption, data collection should sample event-level demands—e.g. one lane-change-gap decision, one intersection-approach decision, or one trajectoryapproval decision per triggered episode—from a stable operational profile. AV operational modes matter because mixing lane keeping, intersections, merging, degraded perception, and fallback episodes may mix different FP/FN probabilities. Even with careful sampling, the approximation is imperfect: closedloop interventions alter subsequent traffic states, and realworld logs are temporally and logically coupled1 . 1 This i.i.d. assumption should be understood to hold approximately, at best. The impact of this assumption on a given assessment’s claims can be evaluated using CBI/robust inference techniques—see Section VII.

5

The classifier labels each sample as either belonging to class “1” or class “0”. There are 4 possible outcomes on each sample: the classifier can succeed in two ways (i.e. correctly classifying the sample as belonging to class “1” or “0”) or fail in two ways (i.e. an FP failure, where a “0” sample is misclassified as “1”, or an FN failure, where a “1” sample is misclassified as “0”). Ignoring differences in success types2 , only 3 outcomes are possible—either success or one of two failure types. Which of these 3 outcomes will occur is not known before the classifier receives the sample. So, there are 3 probabilities an assessor would like to know: the probability Θ that the software succeeds, the probability P of an FP failure, and the probability Q of an FN failure. By definition, Θ + P + Q = 1, so the assessor’s uncertainty reduces to not knowing the true values of only P , Q. The quantity “P + Q” is the unknown probability of failure per classification (pfc). All of this defines an unknown categorical random process— parameterized by (P, Q)—that models the occurrence of the classifier’s successes/failures. The probabilities P , Q, Θ are unconditional; cf. conditional FP/FN rates in ROC analyses. Prior to observing the classifier in operation, the assessor forms beliefs—represented as some probability distribution P—about how likely various values of (P, Q) are. These beliefs are based on reliability evidence gathered before observing the classifier in operation (see section VII-B). Then, upon observing the classifier in operation—e.g. on n classification samples, observing that the classifier made k1 FPs, k2 FNs, and n − k1 − k2 correct classifications3 —an assessor updates their (prior) beliefs via Bayesian inference. As a consequence, this operational evidence can sway the assessor into being more/less skeptical about, say, the classifier having very good accuracy. That is, given the assessor’s beliefs P, and given the classifier failed on k1 and k2 samples, the assessor’s doubt in the classifier’s accuracy being good enough is P(Θ < 1 − b | n tasks, k1 FPs, k2 FNs)

(1)

for some sufficiently small b ∈ [0, 1] such that 1 − b is the minimum accuracy required of the classifier. Priors can often be difficult to specify or justify fully; e.g. for the assessment scenarios presented in this paper, where the prior P is a 2-dimensional joint distribution over feasible (P, Q) values. A pragmatic alternative is the assessor specifies only those aspects of P that available evidence justifies. This often means that only a partial specification of the prior is provided; e.g. specifying 90% confidence in the classifier’s accuracy being greater than 0.999. Formally, a partial specification of the prior defines a set D of all prior distributions that, each, satisfy the assessor’s beliefs. Restricting D to priors with well-defined posterior probabilities, one can determine the worst support the priors in D can give for a reliability claim, by finding the worst-value these priors can give for a posterior probability of interest. This is the CBI approach. For example, (1) quantifies an assessor’s doubt—in the classifier’s accuracy being good enough—after observing the classifier in 2 Admittedly,

this simplification makes the analyses somewhat failurecentric, which is understandable when the concern is safety. The presented techniques and analyses can be extended to cater for multiple success types. 3 The assessor notes each outcome on each sample in the sequence.

operation. However, a skeptical assessor might want to know the most doubt they can have, after seeing evidence of reliable operation. That is, rather than computing (1) using a specific prior P, the assessor seeks a prior that solves the following: sup P(Θ < 1 − b | n tasks, k1 FPs, k2 FNs).

(2)

D

The solution to (2) is conservative—the smallest amount of doubt an assessor can have (based on the operational evidence) that is at least as large as any doubt the assessor can justify! A prior that gives this conservative posterior value is a “worst-case” prior. Under the categorical process, (2) is sup D

E[L(n, k1 , k2 ; P, Q)1P +Q>b ] E[L(n, k1 , k2 ; P, Q)]

(3)

where L(n, k1 , k2 ; p, q) := pk1 q k2 (1 − p − q)n−k1−k2 is the likelihood and 1pr is an indicator function—it equals 1 when an observation satisfies predicate pr, and is 0 otherwise. If the assessor is told k failures occurred, but not which of these are FPs or FNs, the categorical process becomes a Bernoulli process and (2) is replaced by sup P(Θ < 1 − b | n tasks, k failures ) D

= sup D

E[L(n, k; P, Q)1P +Q>b ] E[L(n, k; P, Q)]

(4)

with likelihood L(n, k; p, q) := (p + q)k (1 − p − q)n−k . We focus on conservatively bounding pfc (equivalently accuracy). In practice, multiple reliability–related metrics, e.g. sensitivity and specificity, are of concern. The impact of data fidelity is most easily appreciated by focusing on a single metric, say accuracy, but the analyses presented in this paper can be extended to incorporate other metrics; see Section VII. B. Prior Beliefs about Classifier Reliability What are some beliefs our assessor might justify? The assessor might express various degrees of confidence related to how reliable the classifier is expected to be. That is, in the assessor’s informed opinion, the classifier’s development process is expected to produce very reliable classifiers. Referring to these beliefs as prior knowledge (PKs), we can state these more formally: for a, b1 , l, l1 , l2 ∈ [0, 1], PK 1. The classifier is imperfect: P(P + Q ⩾ l) = 1, where the pfc, P + Q, cannot be smaller than l, given the limits of what is achievable with current software engineering methods, available training data and hardware. PK 2. Being very confident that the classifier’s accuracy is better than required: P(Θ ⩾ b1 ) = a, for a, b1 close to 1; PK 3. A more restrictive version of PK1: P(P ⩾ l1 , Q ⩾ l2 ) = 1, where P, Q cannot be smaller than l1 , l2 in practice and the smallest possible pfc is l1 + l2 . So, l1 + l2 ⩽ 1 − b1 . C. Reliability–Target Constraints on Parameters The values of a, b1 , l, l1 , l2 are constrained by reliability targets that the software must meet. These values must be consistent with general guidance on supporting reliability

6

q

assessment scenarios we consider. Fig. 2 is the partition due to beliefs PK1 and PK2, and Fig. 3 is the partition due to PK2 and PK3. Depending on the PKs, D contains distributions that assign probabilities to subsets in Fig. 2 or Fig. 3.

1

V. C ONFIDENCE B OUNDS ON R ELIABILITY 1 − b1

S1 l

S2

0

1 − b1

l

1

p

We present four theorems that give conservative confidence in an upper bound b on the pfc P + Q. Recall, accuracy Θ is 1 − pfc. For ease of presentation, Theorems 1–4 give the assessor’s smallest “confidence” (i.e. infima), rather than the equivalent assessor’s greatest doubt (i.e. suprema) in (3), (4). In what follows, if Li is the value of the relevant likelihood at point (pi , qi ) ∈ S i of the relevant partition of Ω, define4 L∗i :=

Fig. 2.

PK1, PK2, partition the set Ω of feasible (P, Q)-values.

sup S i ∩[P +Q>b]

Li ,

Li∗ :=

inf

S i ∩[P +Q⩽b]

Li

q

A. Assessor is Unaware of Failure–Type Occurrences

1

Consider the problem (4) of determining the least confidence (in the software’s pfc being small enough), when the assessor knows k failures occurred (over n classifications) but does not know each failure’s type. Theorem 1 solves this. Theorem 1. For n, k, a, b, b1 , l, P , Q, D as already defined and constrained, we seek

1 − b1

E[L(n, k; P, Q)1P +Q⩽b ] E[L(n, k; P, Q)] s.t. PK1, PK2

S1

inf D

S2 l2

0 l1 Fig. 3.

1 − b1

1

p

PK2, PK3, partition the set Ω.

claims using statistical/operational testing [58]. Typically, b1 is relatively large, because the development team aims to develop classifiers that exhibit high accuracy (i.e. Θ). Relatedly, a is relatively large because the assessor is very confident that the development team has produced a classifier that meets reliability targets. Since 1 − b is the minimum accuracy required of the classifier, 0 < 1 − b ⩽ b1 . Moreover, by PK3 and the non-negativity of l1 and l2 , max{l1 , l2 } ⩽ min{b, 1 − b1 }. The assessor also has realistic expectations about how good the classifier can be. For non-trivial classification problems, a classifier employing ML algorithms is likely to not have 100% accuracy, primarily due to the imprecision with which the ML algorithm approximates the unknown target classification function, and due to training the algorithm on only a sample of the data to be classified. Thus, the development team cannot produce ML–based software with P, Q smaller than some very small l1 , l2 . Operational evidence is typically gathered over an observation period for which the maximum likelihood estimates of P , Q, i.e. kn1 and kn2 , are larger than the target-pfc 1 − b1 . D. The Feasible Values for FN and FP Probabilities The feasible (P, Q)-values lie within a triangle, Ω, in the unit square [0, 1]2 . Two partitions of Ω are relevant for the

(5)

Solution: Let Li := L(n, k; pi , qi ) for (pi , qi ) ∈ S i in Fig. 2. The infimum Φ∗ takes the form Φ∗ =

aL2∗ 11−b1 ⩽b aL2∗ + (1 − a)L∗1

That is, objective function (5) attains its infimum, Φ∗ , with a discrete prior distribution over Ω of the form (e.g. see Fig. 4a): ( 1 − a, if i = 1 P(P = pi , Q = qi ) = a, if i = 2 Our second theorem also solves the problem, but the assessor expresses the more detailed PK3 belief instead of PK1. Theorem 2. For n, k, a, b, b1 , l1 , l2 , P , Q, D as already defined and constrained, we seek E[L(n, k; P, Q)1P +Q⩽b ] E[L(n, k; P, Q)] s.t. PK2, PK3 inf D

(6)

Solution: Let Li := L(n, k; pi , qi ) for (pi , qi ) ∈ S i in Fig. 3. The infimum Φ∗ takes the form Φ∗ =

aL2∗ 11−b1 ⩽b aL2∗ + (1 − a)L∗1

That is, objective function (6) attains its infimum, Φ∗ , with a discrete prior distribution over Ω of the form (e.g. see Fig. 4b): ( 1 − a, if i = 1 P(P = pi , Q = qi ) = a, if i = 2 4 L∗ or L i∗ are relevant only when their “S i ∩ ...” are non-empty. i

7

q

q

1

1

k n

k n

b

b

1 − b1

1 − b1 (p1 , q1 )

(p1 , q1 )

l (p2 , q2 ) l2

0

l

1 − b1

b

k n

1

p

(p2 , q2 )

0 l1

(a) A discrete prior over Ω that solves (5) when k > nb. The black dots indicate (pi , qi ) locations (in the partition of Fig. 2) that this prior may assign non-zero probability mass to.

1 − b1

b

k n

1

p

(b) A discrete prior over Ω that solves (6) when k > nb. The black dots indicate the only (pi , qi ) locations (in the partition of Fig. 3) that this prior may assign non-zero probability mass to.

Fig. 4. Example worst-case prior distributions when the assessor has uncertainty about which failure-types have occurred.

q

q

1

1

k2 n−k1

k2 n−k1

b

b

1 − b1

1 − b1 (p1 , q1 )

l

(p1 , q1 )

(p2 , q2 ) l2

(p2 , q2 )

0

l

1 − b1

b

k1 n−k2

1

p

0 l1

(a) A discrete prior over Ω that solves (7) when k1 + k2 > nb. The black dots indicate the only (pi , qi ) locations (in the partition of Fig. 2) that this prior may assign non-zero probability mass to. Here, (p1 , q1 ) = ( kn1 , kn2 ). Whichever (p2 , q2 ) location gives the smallest likelihood value will be assigned non-zero probability a. Fig. 5.

(p2 , q2 ) 1 − b1

b

k1 n−k2

1

p

(b) A discrete prior over Ω that solves (8) when k1 + k2 > nb. The black dots indicate the only (pi , qi ) locations (in the partition of Fig. 3) that this prior may assign non-zero probability mass to. Here, (p1 , q1 ) = ( kn1 , kn2 ).

Example worst-case prior distributions when the assessor knows which failure-types have occurred.

B. Assessor is Aware of Failure–Type Occurrences Now consider problem (3) of determining the least confidence (in the pfc being small enough) an assessor can justify, when the assessor knows k1 FPs and k2 FNs have occurred in n classifications. Our third theorem is analogous to Theorem 1. Theorem 3. For n, k1 , k2 , a, b, b1 , l, P , Q, D as already defined and constrained, we seek E[L(n, k1 , k2 ; P, Q)1P +Q⩽b ] inf D E[L(n, k1 , k2 ; P, Q)] s.t. PK1, PK2

(7)

Solution: Let Li := L(n, k1 , k2 ; pi , qi ) for (pi , qi ) ∈ S i in Fig. 2. The infimum Φ∗ is 0 if k1 ⩾ 1 or k2 ⩾ 1. Otherwise, aL2∗ 11−b1 ⩽b, k1 =k2 =0 Φ = aL2∗ + (1 − a)L∗1 ∗

That is, objective function (7) attains its infimum, Φ∗ , with a discrete prior distribution over Ω of the form (e.g. see Fig. 5a): ( P(P = pi , Q = qi ) =

1 − a, a,

if i = 1 if i = 2

Like Theorem 3, the next theorem also solves CBI problem (3), but with PK1 replaced by the more detailed PK3. Theorem 4. For n, k1 , k2 , a, b, b1 , l1 , l2 , P , Q, D as already defined and constrained, we seek E[L(n, k1 , k2 ; P, Q)1P +Q⩽b ] D E[L(n, k1 , k2 ; P, Q)] s.t. PK2, PK3 inf

(8)

8

Solution: Let Li := L(n, k1 , k2 ; pi , qi ) for (pi , qi ) ∈ S i in Fig. 3. The infimum Φ∗ takes the form Φ∗ =

aL2∗ 11−b1 ⩽b aL2∗ + (1 − a)L∗1

That is, objective function (8) attains its infimum, Φ∗ , with a discrete prior distribution over Ω of the form (e.g. see Fig. 5b): ( 1 − a, if i = 1 P(P = pi , Q = qi ) = a, if i = 2 Theorem 4 is proved in Appendix A; the other theorems are proved using analogous steps. The Φ∗ solutions for (5) through (8) may have the same functional form, but not necessarily the same values; the locations (pi , qi ) and likelihood Li used for each Φ∗ differ, depending on the objective function, PKs, and parameter values. Figs. 4–5 show examples of “worst-case” prior distributions that solve (5)–(8) for particular parameter ranges. These discrete distributions are depicted “from above”, looking down on the distributions and their common domain Ω. Each black dot is a location—in the relevant partition of Ω (i.e. Figs. 2, 3)—that is assigned a probability mass consistent with the assessor’s PK beliefs stated in the relevant theorem. These results extend previous CBI solutions. In the appropriate limits, the marginals of these two dimensional worst– case priors recover the one dimensional CBI worst–case priors in [17]; e.g. when no failures are observed (see Section VI). Strictly speaking, worst-case priors may not be feasible, because they may assign probability mass to locations that violate one or more PKs. Instead, worst-case priors should be viewed as limits of feasible priors, and their preferred locations as so-called limit points of the respective S i subsets (see [59]). Each Theorem’s worst–case prior encodes the most skeptical beliefs an assessor can have; beliefs about what the unknown probabilities, P , Q, might be. Figs. 4–5 show that the “most skeptical” assessor believes the observed failure behavior of the software can arise in two ways: either the software has failure probability values (p1 , q1 ) that make this behavior as likely as possible5 when the software is not sufficiently accurate (i.e. when [P + Q > b]), or the software has failure probability values (p2 , q2 ) that make this behavior as unlikely as possible when the software is sufficiently accurate (i.e. when [P + Q ⩽ b]). This extreme skepticism depends on the software’s observed behavior, so an assessor cannot choose to be this skeptical before observing the software in operation. But, the point of these worst–case priors is not that an assessor should hold these beliefs, but that an assessor could have held these beliefs before observing the software (since these beliefs are consistent with their PKs). And, had they held these beliefs, their doubts about the software’s accuracy (after observing the software) would nevertheless be justified. VI. A PPLICATION TO AV S AFETY A SSESSMENT The following assessment scenarios adapted from [17] illustrate the impact of not accounting for multiple failure–types. 5 That is, the (P, Q)-value where the likelihood is biggest.

Fig. 6. The number of failure-free classifications needed for 95% confidence in the software accuracy satisfying Θ ⩾ 1 − b.

Scenario 1 (no failures). Consider an AV safety-monitoring function (e.g. an unsafe-driving-state detector in the monitoring channel) that contributes to an ASIL D safety goal in an ISO 26262 safety case. An assessor must be at least 95% posterior confident that the monitor’s pfc satisfies an upper bound b. The assessor’s pre-operational knowledge is encoded in PKs, including that neither failure mode can be arbitrarily unlikely (e.g. P, Q ⩾ 10−15 ) and, prior to observing operational data, the assessor is 90% confident that the pfc meets the development target 1 − b1 = 1.09 × 10−10 (equivalently, accuracy Θ ⩾ b1 = 1 − 1.09 × 10−10 ). The AV then accumulates n classifications in the ODD without failure. How many failure-free classifications are required to, conservatively, attain the required 95% confidence level? Fig. 6 illustrates how the required number of failure-free classifications is the same for different conservative assessors. The number is not affected by differences between PK1 and the “more restrictive” PK3, nor is it affected by the possibility of different types of failure occurring. This is consistent with the univariate “no-failures” CBI result in [17]. Consequently, the different theorems’ curves overlap perfectly. Scenario 2 (a single failure). Assume Scenario 1 and suppose that after n1 failure-free classifications the conservative posterior confidence meets the 95%-confidence acceptance criterion. The monitor then fails on the (n1 + 1)-th classification, causing the assessor’s confidence to drop. How many more failure-free classifications, n2 , must the software complete in order for the assessor’s confidence to return to 95%? It depends on the assessor’s prior knowledge and whether the types of failures that occur during operation are known. Fig. 7 compares the (n1 , n2 ) values for different conservative assessors. No curve is plotted for Theorem 3 because this assessor’s confidence drops to zero upon seeing

9

Fig. 7. The extra failure-free classifications needed to regain lost confidence from a single failure. For each n1 , the bound b is deduced—all four theorems give the same b under failure-free operation—from which n2 is determined. Theorem 3 is omitted because n2 = ∞ for all n1 (see main text for details).

a single failure—for example, if k1 = 1 and k2 = 0, then the posterior confidence (7) is zero, using the prior in Fig. 5a with (p2 , q2 ) = (0, l). Consequently, after a single failure, no finite amount of failure-free operational evidence will suffice for the assessor to regain their lost confidence: n2 for Theorem 3 is infinity, dwarfing the other theorems’ curves. The infinite n2 from Theorem 3 is due to PK1—it allows for the conservative belief that failures consisting of only one type are extremely unlikely, when the pfc is smaller than b. Using PK3 instead, in Theorem 4, gives finite n2 for all n1 . The larger the value of n1 , the smaller the pfc bound b that n1 ultimately justifies having 95% confidence in. So, it is unsurprising that the number n2 of required additional classifications also increases. This is apparent from Theorem 4’s monotonically increasing curve, but less apparent from Theorem 1 and 2’s curves—for all sufficiently large n1 , these two curves are monotonically-increasing and bounded. The shape of Theorem 1 and 2’s curves mirror the analogous univariate CBI problem: see Sec. III-B and App. B of [17]. The “hump” is a result of a change in the preferred location n1 →∞ b1 in S 2 (Figs. 2 and 3) at about n1 ≈ 1011 and n2 −→ 1−b1 ; see Appendix B for a proof adapted from [17]. VII. D ISCUSSION A. Impact of Uncertainty from Failure–data Fidelity Uncertainty about failure–data does not necessarily undermine conservatism in assessments—but, when it does, this can be significant. When no failures are observed, Fig. 6 shows that Theorems 1, 2, 3, 4 are all equally conservative, despite ignorance about which failure types have occurred in Theorems 1, 2. This is consistent with our analytical focus on the impact of failure-type uncertainty. However, when failures are observed, Fig. 7 depicts several orders of

magnitude differences between the theorems’ results. After a single failure, the “optimism gap” between Theorem 3 and the other three theorems is infinite, as no finite amount of additional failure–free operation can restore 95% confidence. The gap between Theorem 4 and the remaining two theorems grows in the number of extra successes n2 needed following an initial n1 observed successes and a single failure: at n1 ≈ 9 × 1011 , Theorem 4 requires n2 ≈ 1013 which is three orders of magnitude greater than the alternative theorems. These results demonstrate that not accounting for multiple failure modes can lead to concluding that confidence can be recovered after a failure and the evidence requirements to do this are modest, whereas a mode–aware conservative analysis can conclude recovery is vastly more difficult or, even, infeasible. Also note, Theorem 4’s curve is a worst-case result, so the vertical “gap” between its curve and the other curves represents posterior confidence from infinitely many non-conservative prior distributions that require more failurefree operation (when failure-types are accounted for) than the failure-free operation required by Theorems 1, 2. Although incorporating multiple failure modes can lead to more conservative conclusions, Theorems 3 and 4 have different sensitivities to observed failures. These differences arise from the relative evidential strength of PK1 and PK3: the PKs represent evidence that can, or cannot, rule out extremely undesirable failure behavior. For example, using Theorem 3, an assessor’s grounds for believing the classifier’s failure probability is bounded (i.e., PK1) remain compatible with the classifier never making, exclusively, either FN or FP errors (i.e. in Fig. 2, the Ω region has the axes as boundaries). Under this residual possibility, even a single observed FN or FP failure of the software can constitute strong evidence against that failure-type impossibility, and correspondingly strong evidence in favor of substantial unreliability—consistent with the interpretation that the successes observed (not observing the alleged impossible failure mode) are largely due to chance. This impossibility of certain failure modes is ruled out if evidence suggests the classifier’s individual FN and FP probabilities cannot be smaller than some non-zero values at the limits of what has been demonstrated to be practically achievable (e.g. based on demonstrated past performance of classifiers using similar algorithms/methods in similar applications, so Ω’s boundaries do not coincide with axes in Fig. 3). Using CBI, an assessor can question the strength of reliability evidence—a form of principled “belief refinement” in the assessment process. An assessor specifies increasingly more detailed beliefs as more evidence becomes available, with CBI constantly providing the most critical interpretation of the evidence. Conceptually, as more evidence becomes available, the assessor is narrowing the set of prior distributions in D that are consistent with the evidence. In contrast, fully specified prior distributions in traditional Bayesian inference leave no room for belief refinement, other than by observational evidence. The assessor who uses Theorems 1, 2, is attempting to be conservative, when faced with uncertainty about which failure types have occurred. Sometimes such attempts are conservative, and sometimes they aren’t. The fact—that CBI results that do not account for the extra failure modes (Theo-

10

⩽

Fig. 8.

Conservative posterior confidence guaranteed by (9)

rems 1, 2) are not necessarily conservative, compared with CBI results that take those failure modes into account (Theorems 3, 4)—is consistent with CBI’s conservatism guarantee. By design/definition, a prior distribution using the same evidence and statistical model (i.e. likelihood function) cannot yield a more conservative value for the posterior probability of interest. Nevertheless, if the evidence or statistical model differs from that used for a CBI solution, the guarantee no longer applies, in line with Popov’s findings [60]. In [60], Popov compares two Bayesian AV assessment approaches—a “white box” approach that accounts for the various operational modes of an AV system, and a “black box” approach that uses CBI but does not account for the operational modes. The “black box” CBI results were not always conservative compared with the “white box” results. The CBI results in [60] would have been conservative compared with the “white box” setting if the operational modes had been accounted for. We can exemplify an analogue in the present setting. Notice how the objective function in (5) is a convex combination of objective functions from (7): E[L(n, k; P, Q)1P +Q⩽b ] E[L(n, k; P, Q)] k X E[L(n, k1 , k − k1 ; P, Q)1P +Q⩽b ] = wk1 E[L(n, k1 , k − k1 ; P, Q)] k1 =0

( k ) E[L(n,k1 ,k−k1 ;P,Q)] where wk1 := Pkk1 k E[L(n,j,k−j;P,Q)] . Consequently, Theoj=0 ( j ) rem 1’s infimum can always be bounded below by the smallest infima given by Theorem 3 for some k1 . That is, E[L(n, k1 , k − k1 ; P, Q)1P +Q⩽b ] E[L(n, k1 , k − k1 ; P, Q)] E[L(n, k1 , k − k1 ; P, Q)1P +Q⩽b ] inf inf D k1 =0,...,k E[L(n, k1 , k − k1 ; P, Q)]

inf

inf

k1 =0,...,k D

=

E[L(n, k; P, Q)1P +Q⩽b ] D E[L(n, k; P, Q)]

inf

(9)

The interchange of infima in (9) is allowed because the priors in D and the k1 values are independent, so the iterated infima are an infimum over the Cartesian product {0, . . . , k} × D (see Appendix C). An analogous inequality holds between the infima of Theorems 2 and 4. In summary, when there is certainty about operational evidence (such as what failure modes occurred), this should be incorporated in the CBI analyses to ensure conservatism. Otherwise, if there is uncertainty, using (9) ensures the CBI–based assessment remains conservative. Inequality (9) is illustrated in Fig. 8 with k = 2 failures. Posterior confidence from Theorems 1, 2, 4 grows as the number of failure-free classifications grows. While, confidence from Theorem 3 remains at 0. Due to inequality (9), at each n, Theorem 1’s confidence curve is bounded below by “no confidence” from Theorem 3. Similarly, Theorem 2’s curve is bounded below by the minimum confidence from Theorem 4 under three pairs of (k1 , k2 ) values: (0, 2), (1, 1), (2, 0). Fig. 8 also illustrates the general ordering: for each n > 2, Theorem 2 gives the least conservative confidence, then Theorem 1, then the minimum of Theorem 4’s confidence values, and finally the minimum of Theorem 3’s confidence values. B. Practical Context and Guidance Theorems 1–4 can be applied at various stages of development, review, and deployment; see Fig. 9. First, during design and pre-integration verification and validation (V&V). Second, during certification-oriented assurance-case construction and independent assessment, theorem-derived posteriors can serve as explicit quantitative sub-claims that connect test/analysis evidence to higher-level safety claims, complementing prescriptive-process evidence with an auditable argument about what the data justifies [61]. Third, at system validation and pre-deployment trials (including scenario-based testing and shadow-mode evaluations), the theorems can be applied to operational-test datasets to obtain conservative bounds tailored to an ODD, aligning with AV-focused work that emphasizes combining operational testing with other design/verification evidence in a statistically principled way [18]. Fourth, during early staged roll-out, theorem-based updates naturally support “confidence bootstrapping” from failurefree exposure under deliberately controlled increments in AV fleet [21]. Finally, in post-deployment monitoring and change management (e.g. after software updates or distribution shift), theorem-based assessments can be re-run incrementally as new evidence arrives, consistent with Assurance 2.0 [9], [10]. Before operational testing, assessors may justify PKs using various forms of evidence. Formal expert–elicitation methods help to translate an assessor’s judgment—e.g. translating perceptions of the development team’s competence, the plausibility/robustness of implemented algorithms, and confidence in the engineering organization and toolchain—into probability statements or constrained prior sets [62], [63]. For example, experts may be presented with alternative failure/success scenarios and asked to express how surprised they would be (on an appropriate scale) to see such scenarios given the ancillary

11

CBI Theorems 1–4

PK1–PK3 from pre-operational evidence: concept, safety analysis, safety requirements, architecture/design, lab testing.

Concept & HARA

Theorems 3–4: Typed failure data: FP and FN counts (k1 , k2 ) observed. Applicable to structured SW V&V campaigns and to validation / operation when FP/FN labels are logged.

System & Technical Safety Reqs

SW safety reqs & architecture

Theorems 1–2: Coarse failure data: only total failures k in n classifications are logged, without distinguishing FP/FN. Used when validation or in-service logging records failures but not their types.

SW design, impl. & V&V

PK-generating development stages

System / vehicle validation

Operational data from tests and use: – coarse failure counts k in n classifications; – FP/FN-typed counts (k1 , k2 ) from SW V&V, system/vehicle validation, and in-service operation.

Operation & field monitoring

Data-generating development/operational stages. AV stack subsystems in operation (cf. Fig. 1)

Fig. 9. Illustration of where the CBI theorems apply across an ISO 26262-style software (SW) lifecycle (illustrated for the AV stack of Fig. 1). PK1–PK3 are justified from pre-operational evidence (e.g. concept, safety analyses, past operational data from similar software used in similar contexts, SW safety requirements, SW architecture/design/algorithms, SW V&V). During SW V&V, system/vehicle validation, and in-service operation, operational data—either coarse failure counts or FP/FN-typed counts—are combined with PKs via Theorems 1–4 to obtain conservative bounds on the classifier’s failure probability.

b = β(1−b1 ) for some bounded scaling β > 1. The sensitivity of nmin to a, b, 1 − b1 , β can be analysed using families of curves such as those in Fig. 10. For example, moving upward along a vertical line at a given engineering target-pfc 1 − b1 , either a larger nmin is needed to gain posterior confidence in smaller b bounds (with a fixed prior confidence “a” in the classifier satisfying 1 − b1 bound) or a larger nmin is needed for posterior confidence in a fixed b bound (but with less prior confidence “a” in the classifier satisfying the 1 − b1 bound). So, for a testing budget of N classification tasks, and with an elicited PK2 prior confidence, the assessor can determine the smallest bound b∗ they can achieve 95% posterior confidence in. Specifically b∗ = β ∗ (1−b∗1 ), where β ∗ is either the infimum of those β with (a, β)-curves that intersect the horizontal N budget line at/before 1 − b1 , or the supremum of those β with (a, β)-curves that intersect the budget line at/after 1 − b1 , and (1 − b∗1 ) is the engineering target at which the (a, β ∗ )-curve intersects the budget line. Fig. 10. sensitivity of failure-free evidence needed to obtain 95% confidence in a pfc bound, when the assessor is unsure about the precise value of their confidence a in an engineering target-pfc 1 − b1 .

evidence. Elicitation can be supported by Bayesian-belief networks that take into account classifier training maturity, classification algorithm choice, and verification artifacts [64], [65]. Elicitation can be carried out with a single assessor or among a group of assessors to obtain PK-parameter values that reflect a consensus view. Elicited PKs can be refined via Bayesian inference using unit/integration test evidence Sensitivity analysis can reveal whether CBI results are sensitive to PK-parameter values in a given practical scenario; if they are insensitive, the precise parameter values are unimportant over the specified range—any values from this range “will do” for assessment, as the following two examples show. Let nmin be the minimum number of failure-free classifications needed to conservatively achieve 95% posterior confidence in the classifier satisfying the pfc bound b, given the assessor’s PK2 confidence P(Θ ⩾ b1 ) = a where

The i.i.d. categorical process of Section IV assumes temporal statistical independence and stationarity of failure/success probabilities—properties that may only hold approximately in practice. Sensitivity analysis can reveal whether the i.i.d. assumption leads to relatively optimistic assessment claims compared with using a more sophisticated model (e.g. see [19], [20]). For example, conservative confidence bounds can be obtained using the stationary Markov-chain in Fig. 11 and e j ⩾ b1 ) = a; (PK3-a) these PKs: for j = 1, 2, 3, (PK2-a) P(Θ e e P(Pj ⩾ l1 , Qj ⩾ l2 ) = 1; (PK2); (PK3); (PK4) confidence e2 = Q e3 ) = η in the i.i.d. assumption P(P = Pe2 = Pe3 , Q = Q for some 0 ⩽ η ⩽ 1. This generalisation models deviations from the i.i.d. model using η—with η = 1 we recover the i.i.d. model, while η = 0 rules out the i.i.d. model completely. In Fig. 12 the conservative posterior confidence in bound b = 10−2 , from i.i.d. and stationary non-i.i.d. models, are compared. In this scenario, the i.i.d. model is the most optimistic; the weaker the i.i.d. assumption (i.e. the smaller η is), the smaller the worst-case confidence in the classifier. By accumulating failure-free evidence, the posterior confidence from all of the models initially monotonically increases. How-

12

θe1

success θe2

θe3 pe1

pe2

pe3

qe1 FN

FP

qe3

qe2 Fig. 11. Three-state stationary Markov model for classifier outcomes. The initial distribution is (θ, p, q) and the transition probabilities are (θej , pej , qej ) for j = 1, 2, 3, satisfying θ = 1 − p − q, θej = 1 − pej − qej , pe1 θ = p − pe p2 − qe p3 and qe1 θ = q − pe q2 − qe q3 .

Fig. 12. Sensitivity of conservative posterior confidence to a and η, when there are no failures. Here, l1 = l2 = 10−9 , b = 10−2 , 1 − b1 = 10−4 .

ever, the non-i.i.d. curves eventually begin to monotonically decrease at different rates. Asymptotically, all the non-i.i.d. curves converge to 0 posterior confidence. Here, the closer the values of η and a are to 1, the better the agreement between the i.i.d. and non-i.i.d. models for longer. The curves highlight when the i.i.d. model does not deviate substantially from the non-i.i.d. models. This is consistent with the findings in [19]. C. Limitations and Future Work AV operational data can be clustered by route/weather, can exhibit non–stationarity (e.g. software updates, distribution shift from changing driving conditions), and may contain correlated failures (cf. safety-critical software in general [66]). The statistical model treats outcomes as success vs. two failure modes (FP/FN), and explicitly collapses multiple success types. This “failure-centric” focus omits potentially decision– relevant structure (e.g. near–misses, degraded performance

modes, or graded severity). More broadly, “low fidelity” in (4) is primarily missing failure–type labels, whereas operational fidelity issues can also include noisy labels, uncertain counts, partial observability of reliability-related events/event–types, missing details of operational modes, more failure and success types, and scenario–dependent misclassification costs. The primary quantity pfc (i.e. P +Q) is bounded in the form of confidence statements (e.g. (1)–(4)). However, system-level safety cases often require linking such component-level rates to hazard rates, exposure, and severity (e.g. how “classifications” map to miles, ODD coverage, or risk per hour). The AV examples in Sections VI, VII do not yet provide calibrated mappings from pfc bounds to end-to-end safety metrics. The (P, Q) distribution can change due to changes in the operational environment—e.g. the AV operating in driving conditions very different from those during assessment—or due to significant AV software updates—e.g. further classifier training or a change in decision thresholds. Incorporating such “distribution drift” into conservative assessments can be done; e.g. using generalizations of CBI models that account for changes in operational context, such as the models used in [15], [18], [23]. Without such generalizations, this paper’s results must be applied only “in-between” significant software updates or “in-between” changes in operational environment. Future work can investigate supporting other forms of conservative dependability claims; e.g. CBI bounds on the timeliness of mitigating actions taken by safety subsystems. VIII. C ONCLUSION The fidelity of operational evidence places limits on what claims are justified in an assessment of safety-critical-software reliability. When assessing safety-relevant classifiers in an AV stack, this paper illustrates how an assessor’s attempts at supporting conservative claims—using “low-fidelity” evidence with a “guaranteed-to-be-conservative” statistical inference approach (i.e. CBI)—can be valid sometimes, but invalid at other times. The problem is that the CBI guarantee is conditional on the evidence, so that if some evidence is not used with CBI (such as missing information about failure-types that have occurred) the results of inference may be undesirably optimistic, compared with Bayesian inference based on a fully–specified feasible prior and “all” of the evidence. An assessor often has no choice but to work with “missing detail” in operational evidence. The CBI extensions presented in this paper aid in using such evidence conservatively. R EFERENCES [1] International Organization for Standardization (ISO), “Keeping safe on the roads: series of standards for vehicle electronics functional safety just updated,” Dec. 2018. [Online]. Available: https://www.iso. org/news/ref2358.html [2] R. Debouk, “Overview of the second edition of ISO 26262: Functional safety—road vehicles,” Journal of System Safety, vol. 55, no. 1, pp. 13– 21, Mar. 2019. [Online]. Available: https://doi.org/10.56094/jss.v55i1.55 [3] International Organization for Standardization, “ISO 26262-6:2018 road vehicles — functional safety — part 6: Product development at the software level,” Dec. 2018, edition 2. [Online]. Available: https://www.iso.org/standard/68388.html [4] ——, “ISO 26262-8:2018 road vehicles — functional safety — part 8: Supporting processes,” Dec. 2018, edition 2. [Online]. Available: https://www.iso.org/standard/68390.html

13

[5] S. Chonnad, V. Litovtchenko, and V. Sargsyan, “The Role of Confirmation Measures in ISO 26262 Functional Safety Automotive Products,” White paper, Synopsys, Jan. 2023, pub: Jan. 2023. Accessed: 2025-12-29. [Online]. Available: https://data.embeddedcomputing.com/ uploads/articles/whitepapers/16799.pdf [6] ISO 21448:2022 Road vehicles — Safety of the intended functionality, International Organization for Standardization (ISO) Std., 2022. [Online]. Available: https://www.iso.org/standard/77490.html [7] ISO/PAS 8800:2024 Road vehicles — Safety and artificial intelligence, International Organization for Standardization (ISO) Std., 2024. [Online]. Available: https://www.iso.org/standard/83303.html [8] ANSI/UL 4600 Ed. 2-2022 Standard for Safety for the Evaluation of Autonomous Products, Underwriters Laboratories (UL) Std., 2022. [Online]. Available: https://webstore.ansi.org/standards/ul/ul4600ed2022 [9] R. Bloomfield and J. Rushby, “Assurance 2.0: A manifesto,” 2020. [Online]. Available: https://arxiv.org/abs/2004.10474 [10] ——, “Assessing confidence with assurance 2.0,” 2022. [Online]. Available: https://arxiv.org/abs/2205.04522 [11] S. Chen, M. J. Elizebeth, R. Aghazadeh Chakherlou, X. Zhao, E. Barbier, S. Khastgir, and P. Jennings, “A scalable framework for safety assurance of self-driving vehicles based on assurance 2.0,” 2025. [Online]. Available: https://arxiv.org/abs/2510.00092 [12] P. G. Bishop, R. E. Bloomfield, B. Littlewood, A. A. Povyakalo, and D. R. Wright, “Toward a formalism for conservative claims about the dependability of software-based systems,” IEEE Transactions on Software Engineering, vol. 37, no. 5, pp. 708–717, 2011. [13] L. Strigini and A. A. Povyakalo, “Software fault-freeness and reliability predictions,” in Computer Safety, Reliability, and Security (SAFECOMP 2013), ser. Lecture Notes in Computer Science, vol. 8153. Springer, 2013, pp. 106–117. [14] B. Littlewood, K. Salako, L. Strigini, and X. Zhao, “On reliability assessment when a software-based system is replaced by a thoughtto-be-better one,” Reliability Engineering & System Safety, vol. 197, p. 106752, 2020. [15] K. Salako, L. Strigini, and X. Zhao, “Conservative confidence bounds in safety, from generalised claims of improvement & statistical evidence,” in 51st Annual IEEE/IFIP Int. Conf. on Dependable Systems and Networks, ser. DSN’21. Taipei Taiwan: IEEE/IFIP, 2021, pp. 451–462. [16] K. Salako, “Loss-size and reliability trade-offs amongst diverse redundant binary classifiers,” in Quantitative Evaluation of Systems, M. Gribaudo, D. N. Jansen, and A. Remke, Eds. Springer International Publishing, 2020, pp. 96–114. [17] X. Zhao, V. Robu, D. Flynn, K. Salako, and L. Strigini, “Assessing the safety and reliability of autonomous vehicles from road testing,” in the 30th Int. Symp. on Software Reliability Engineering. Berlin, Germany: IEEE, 2019, pp. 13–23. [18] X. Zhao, K. Salako, L. Strigini, V. Robu, and D. Flynn, “Assessing safety-critical systems from operational testing: A study on autonomous vehicles,” Information and Software Technology, vol. 128, p. 106393, 2020. [19] K. Salako and X. Zhao, “The unnecessity of assuming statistically independent tests in bayesian software reliability assessments,” IEEE Transactions on Software Engineering, vol. 49, no. 4, pp. 2829–2838, 2023. [20] ——, “Demonstrating software reliability using possibly correlated tests: Insights from a conservative bayesian approach,” Quality and Reliability Engineering International, vol. 40, no. 3, pp. 1197–1220, 2024. [21] P. Bishop, A. Povyakalo, and L. Strigini, “Bootstrapping confidence in future safety from past safe operation,” in 2022 IEEE 33rd International Symposium on Software Reliability Engineering (ISSRE). Charlotte, NC, USA: IEEE, 2022, pp. 97–108. [22] R. Aghazadeh Chakherlou, K. Salako, and L. Strigini, “Arguing safety of an improved autonomous vehicle from safe operation before the change: new results,” in 2022 IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW). IEEE, 2022, pp. 307– 312, presented at RAIS 2022: 2nd International Workshop on Reliability of Autonomous Intelligent Systems; ISSREW print ISBN: 978-1-66547679-9. [23] R. Aghazadeh Chakherlou and L. Strigini, “Using pre-change operational evidence for predicting post-change reliability, given prior confidence in fault-freeness,” in Advances in Reliability, Safety and Security, Part 3: ESREL 2024 Contributions, K. Kolowrocki and E. Dabrowska, Eds. Gdynia, Poland: Polish Safety and Reliability Association, 2024, pp. 9–18, electronic ISBN: 978-83-68136-02-9, print ISBN: 978-8368136-15-9. [24] P. Bishop, A. Povyakalo, and L. Strigini, “Doubt in safety claims is inevitable: What is its impact, what can be done about it?” in Computer

Safety, Reliability, and Security. SAFECOMP 2025 Workshops, ser. Lecture Notes in Computer Science, M. Törngren, B. Gallina, E. Schoitsch, E. Troubitsyna, and F. Bitsch, Eds. Springer Cham, 2025, vol. 15955, pp. 226–239, softcover ISBN: 978-3-032-02017-8. [25] J. O. Berger, “Robust bayesian analysis: Sensitivity to the prior,” Journal of Statistical Planning and Inference, vol. 25, no. 3, pp. 303–328, 1990. [26] J. Berger and L. M. Berliner, “Robust bayes and empirical bayes analysis with ϵ-contaminated priors,” The Annals of Statistics, vol. 14, no. 2, pp. 461–486, 1986. [Online]. Available: https://projecteuclid.org/journals/annals-of-statistics/volume-14/ issue-2/Robust-Bayes-and-Empirical-Bayes-Analysis-withepsilon-Contaminated-Priors/10.1214/aos/1176349933.full [27] P. Walley, Statistical Reasoning with Imprecise Probabilities. Chapman and Hall, 1991. [28] E. Moreno and J. A. Cano, “Robust bayesian analysis with ϵcontaminations partially known,” Journal of the Royal Statistical Society: Series B (Methodological), vol. 53, no. 1, pp. 143–155, 1991. [29] J. O. Berger, “An overview of robust Bayesian analysis,” Test, vol. 3, no. 1, pp. 5–124, 1994. [30] K. Salako and R. T. Muhammad, “Conservative software reliability assessments using collections of bayesian inference problems,” arXiv, 2025, accessed: 2025-12-22. [Online]. Available: https://arxiv.org/abs/ 2511.07038 [31] N. Kalra and S. M. Paddock, “Driving to safety: How many miles of driving would it take to demonstrate autonomous vehicle reliability?” Transportation Research Part A: Policy and Practice, vol. 94, pp. 182– 193, 2016. [32] P. Popov, “Dynamic safety assessment of autonomous vehicle based on multivariate bayesian inference (DyAVSA),” Journal of Reliable Intelligent Environments, vol. 11, 2025. [33] S. Riedmaier, T. Ponn, D. Ludwig, B. Schick, and F. Diermeyer, “Survey on scenario-based safety assessment of automated vehicles,” IEEE Access, vol. 8, pp. 87 456–87 477, 2020. [34] X. Zhao, R. Aghazadeh-Chakherlou, C.-H. Cheng, P. Popov, and L. Strigini, “On the need for a statistical foundation in scenario-based testing of autonomous vehicles,” in 2025 IEEE 28th International Conference on Intelligent Transportation Systems (ITSC). IEEE, 2025, pp. 3998–4005. [Online]. Available: https://doi.org/10.1109/ITSC60802. 2025.11423546 [35] D. Zhao, H. Lam, H. Peng, S. Bao, D. J. LeBlanc, K. Nobukawa, and C. S. Pan, “Accelerated evaluation of automated vehicles safety in lane-change scenarios based on importance sampling techniques,” IEEE Transactions on Intelligent Transportation Systems, vol. 18, no. 3, pp. 595–607, 2017. [36] M. O’Kelly, A. Sinha, H. Namkoong, R. Tedrake, and J. C. Duchi, “Scalable end-to-end autonomous vehicle testing via rare-event simulation,” in Advances in Neural Information Processing Systems 31. Curran Associates, Inc., 2018, pp. 9827–9838. [37] D. G. Altman and J. M. Bland, “Diagnostic tests. 1: Sensitivity and specificity,” BMJ, vol. 308, no. 6943, p. 1552, 1994. [Online]. Available: https://pubmed.ncbi.nlm.nih.gov/8019315/ [38] M. Sokolova and G. Lapalme, “A systematic analysis of performance measures for classification tasks,” Information Processing & Management, vol. 45, no. 4, pp. 427–437, 2009. [39] J. Davis and M. H. Goadrich, “The relationship between precision-recall and roc curves,” in Proceedings of the 23rd International Conference on Machine Learning (ICML ’06). ACM, 2006, pp. 233–240. [40] T. Fawcett, “An introduction to ROC analysis,” Pattern Recognition Letters, vol. 27, no. 8, pp. 861–874, 2006. [41] J. A. Hanley and B. J. McNeil, “The meaning and use of the area under a receiver operating characteristic (ROC) curve,” Radiology, vol. 143, no. 1, pp. 29–36, 1982. [Online]. Available: https://pubmed.ncbi.nlm.nih.gov/7063747/ [42] T. Saito and M. Rehmsmeier, “The precision-recall plot is more informative than the ROC plot when evaluating binary classifiers on imbalanced datasets,” PLOS ONE, vol. 10, no. 3, p. e0118432, 2015. [Online]. Available: https://journals.plos.org/plosone/article?id= 10.1371/journal.pone.0118432 [43] A. Erkanli, M. Sung, E. J. Costello, and A. Angold, “Bayesian semiparametric roc analysis,” Statistics in Medicine, vol. 25, no. 22, pp. 3905–3928, 2006. [44] A. J. Branscum, W. O. Johnson, T. E. Hanson, and I. A. Gardner, “Bayesian semiparametric roc curve estimation and disease diagnosis,” Statistics in Medicine, vol. 27, no. 13, pp. 2474–2496, 2008. [45] J. Gu, S. Ghosal, and A. Roy, “Bayesian bootstrap estimation of roc curve,” Statistics in Medicine, vol. 27, no. 26, pp. 5407–5420, 2008.

14

[46] Y.-K. Choi, W. O. Johnson, M. T. Collins, and I. A. Gardner, “Bayesian inferences for receiver operating characteristic curves in the absence of a gold standard,” Journal of Agricultural, Biological, and Environmental Statistics, vol. 11, no. 2, pp. 210–229, 2006. [47] S. D. Pendleton, H. Andersen, X. Du, X. Shen, M. Meghjani, Y. H. Eng, D. Rus, and M. H. Ang, “Perception, planning, control, and coordination for autonomous vehicles,” Machines, vol. 5, no. 1, p. 6, 2017. [Online]. Available: https://www.mdpi.com/2075-1702/5/1/6 [48] W. Schwarting, J. Alonso-Mora, and D. Rus, “Planning and decisionmaking for autonomous vehicles,” Annual Review of Control, Robotics, and Autonomous Systems, vol. 1, pp. 187–210, 2018. [49] The Autonomous Working Group (Safety & Architecture), “Safe automated driving: Requirements and architectures,” Full report, Version 1.0 (01.12.2023), Dec. 2023, accessed: 2025-12-29. [Online]. Available: https://www.the-autonomous.com/wp-content/uploads/2023/ 12/wg-safetyarchitecture-full-report-a4.pdf [50] F. Terrosi, L. Strigini, and A. Bondavalli, “Impact of machine learning on safety monitors,” in Computer Safety, Reliability, and Security (SAFECOMP 2022), ser. Lecture Notes in Computer Science, vol. 13414. Cham: Springer, 2022, pp. 129–143. [Online]. Available: https://doi.org/10.1007/978-3-031-14835-4 9 [51] S. Kojchev, E. Klintberg, and J. Fredriksson, “A safety monitoring concept for fully automated driving,” in 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC). IEEE, 2020, pp. 1–7. [Online]. Available: https://doi.org/10.1109/ITSC45102.2020. 9294307 [52] M. Yuhas and A. Easwaran, “Co-design of out-of-distribution detectors for autonomous emergency braking systems,” in 2023 IEEE 26th International Conference on Intelligent Transportation Systems (ITSC). Bilbao, Bizkaia, Spain: IEEE, 2023, pp. 1996–2003. [Online]. Available: https://doi.org/10.1109/ITSC57777.2023.10421953 [53] D. Jackson, V. Richmond, M. Wang, J. Chow, U. Guajardo, S. Kong, S. Campos, G. Litt, and N. Aréchiga, “Certified control: An architecture for verifiable safety of autonomous vehicles,” 2021. [Online]. Available: https://arxiv.org/abs/2104.06178 [54] P. Koopman and M. Wagner, “Challenges in autonomous vehicle testing and validation,” SAE International Journal of Transportation Safety, vol. 4, no. 1, pp. 15–24, 2016. [55] K. Goseva-Popstojanova and K. S. Trivedi, “Failure correlation in software reliability models,” IEEE Transactions on Reliability, vol. 49, no. 1, pp. 37–48, 2000. [56] A. L. Goel and K. Okumoto, “Time-dependent error-detection rate model for software reliability and other performance measures,” IEEE Transactions on Reliability, vol. R-28, no. 3, pp. 206–211, 1979. [57] B. Littlewood and L. Strigini, “Validation of ultra-high dependability for software-based systems,” Communications of the ACM, vol. 36, no. 11, pp. 69–80, 1993. [58] L. Strigini and B. Littlewood, “Guidelines for statistical testing,” Centre for Software Reliability, City University London, Technical Report, PASCON/WO6-CCN2/TN12, 1997, technical report prepared under ESA/ESTEC project PASCON. [Online]. Available: https://openaccess. city.ac.uk/id/eprint/254/2/StatsTesting TN12-3.1distrib2.pdf [59] K. Salako and R. T. Muhammad, “Fixed-point characterisations of extremal distributions under partial distributional constraints,” 2026. [Online]. Available: https://arxiv.org/abs/2608.04315 [60] P. Popov, “Why black-box bayesian safety assessment of autonomous vehicles is problematic and what can be done about it?” IEEE Transactions on Intelligent Vehicles, vol. 11, no. 2, pp. 311–324, 2026. [61] R. Hawkins, I. Habli, T. Kelly, and J. McDermid, “Assurance cases and prescriptive software safety certification: A comparative study,” Safety Science, vol. 59, pp. 55–71, 2013. [62] A. O’Hagan, C. E. Buck, A. Daneshkhah, J. R. Eiser, P. H. Garthwaite, D. J. Jenkinson, J. E. Oakley, and T. Rakow, Uncertain Judgements: Eliciting Experts’ Probabilities. John Wiley & Sons, 2006. [63] P. H. Garthwaite, J. B. Kadane, and A. O’Hagan, “Statistical methods for eliciting probability distributions,” Journal of the American Statistical Association, vol. 100, no. 470, pp. 680–701, 2005. [64] N. E. Fenton, M. Neil, and J. G. Caballero, “Using ranked nodes to model qualitative judgments in bayesian networks,” IEEE Transactions on Knowledge and Data Engineering, vol. 19, no. 10, pp. 1420–1432, 2007. [65] K. Salako and L. Strigini, “When does “Diversity” in development reduce common failures? insights from probabilistic modeling,” IEEE Trans. Dependable Secur. Comput., vol. 11, no. 2, pp. 193–206, 2014. [Online]. Available: https://doi.org/10.1109/TDSC.2013.32 [66] P. G. Bishop, “The variation of software survival time for different operational input profiles (or why you can wait a long time for a big

bug to fail),” in FTCS-23: The Twenty-Third International Symposium on Fault-Tolerant Computing. IEEE, 1993, pp. 98–107.

A PPENDIX A P ROOF OF T HEOREM 4 Proof. For n, k1 , k2 , a, b, b1 , l1 , l2 , P , Q, D as already defined and constrained, we seek E[L(n, k1 , k2 ; P, Q)1P +Q⩽b ] E[L(n, k1 , k2 ; P, Q)] s.t. PK2, PK3 inf D

Define C = [p+q ⩽ 1−b1 ] and G = [p+q ⩽ b]. The Fig. 3 partition is therefore just S 1 = C c ∩ Ω and S 2 = C ∩ Ω. So, every admissible prior distribution satisfies P(S 1 ) = 1 − a, P(S 2 ) = a. There is no remaining mass-allocation freedom. Under the theorem’s reliability-target ordering, 1 − b1 ⩽ b so S 2 = C ∩ Ω ⊆ G. Thus, all mass in S 2 contributes to the numerator. Meanwhile, S 1 contains points above the bline, because S 1 = [p + q > 1 − b1 ] ∩ Ω and 1 − b1 ⩽ b, so S 1 ∩ Gc ̸= ∅, up to usual boundary/closure conventions. Now, write f (p, q) = L(p, q)1G (p, q) and g(p, q) = L(p, q), where L(p, q) is shorthand for L(n, k1 , k2 ; p, q). Then, the objective is EP [f (P, Q)] . inf P∈D EP [g(P, Q)] The functions are bounded on Ω: g = L is continuous, while f = L1G is bounded piecewise continuous with possible discontinuity only on the line p+q = b. Hence, we may appeal to fixed-point arguments to solve the problem after refining S 1 , S 2 by the regions G, Gc , and the boundary p + q = b. Indeed, by Theorem 1 of [28] or Proposition 2.2 of [59], we may restrict the optimization to priors that are the weak limits of feasible discrete priors, supported by one selected location in each of S 1 and S 2 . So, to find the infimum, we use Dinkelbach-type iteration; we can do this because the infimum problem is “separable”, in that it can be recast as a weighted sum of Dinkelbach differences that each need to be minimized. For a candidate infimum value ϕ for the objective function, each Dinkelbach difference takes the form hϕ = f − ϕg. On subset G, hϕ = (1 − ϕ)L, whereas on Gc , hϕ = −ϕL. The infimum is given by appropriately allocating probability mass to (p, q) locations in S 1 and S 2 that minimize h. Since 0 ⩽ ϕ ⩽ 1, this has the following consequences for determining the h-minimizing (p, q) locations. On S 2 , we have S 2 ⊆ G. Therefore, on S 2 , minimizing hϕ = (1 − ϕ)L means minimizing L; i.e. the (p, q) location that gives the likelihood value L2∗ = inf (p,q)∈S 2 ∩G L(p, q) on S 2 , with boundary locations interpreted as limits of non-boundary locations. Analogously, on S 1 , the minimizing (p, q) location can be in S 1 ∩ G or S 1 ∩ Gc . On S 1 ∩ G, the Dinkelbach difference is non-negative, hϕ = (1 − ϕ)L ⩾ 0, while on S 1 ∩ Gc it is non-positive, hϕ = −ϕL ⩽ 0. Thus, the location lies in S 1 ∩ Gc , and there it maximizes L. That is, the location that gives the likelihood value L∗1 =

sup (p,q)∈S 1 ∩Gc

L(p, q) =

sup (p,q)∈S 1 ∩[p+q>b]

L(p, q).

15

Again, with boundary locations interpreted as limits of nonboundary locations, so the “sup” is over S 1 ∩ [p + q ⩾ b]. This pair of h-minimizing (p, q) locations are the support of an extremal prior that assigns probability 1 − a to the location in S 1 ∩ Gc and probability a to the location in S 2 ∩ G; for example, see Fig. 5b. Substituting these support values into the posterior-confidence ratio gives Φ∗ =

aL2∗ 11−b1 ⩽b . aL2∗ + (1 − a)L∗1

The indicator 11−b1 ⩽b clarifies the solution is not necessarily zero when PK2’s high-confidence region lies at or below the reliability target b-line; equivalently, 1 − b ⩽ b1 implies the indicator equals 1.

Using n − 1 = n1 + n2 and substituting (13) into the l.h.s. of (14) then gives  n +n  n  n  n 1−b 1 2 1−b 1 1−b 2 1−b 2 = =K . b1 b1 b1 b1 (15) Equating the far r.h.s. of both (14) and (15), the Ks cancel: n  1 − b1 1−b 2 = . (16) b1 b step III: (taking n1 → ∞.) By (13),   1−b ln K ln = . b1 n1

(17)

Taking logs in (16) and rearranging, A PPENDIX B n1 →∞ b1 P ROOF THAT n2 −→ 1−b1 IN F IG . 7

n2 =

Proof. In Scenario 2, for each n1 we first deduce a bound b = b(n1 ) by requiring that the failure-free evidence k = 0 over n1 classifications yields target confidence c ∈ (0, 1) (here c = 0.95): Φ0 (b; n1 ) = c. (10) Then, after observing one (untyped) failure at the (n1 + 1)-th classification, we add n2 further failure-free classifications, so the total count becomes n = n1 + 1 + n2 (k = 1), and we choose n2 so that the conservative confidence returns to c: Φ1 (b; n1 + 1 + n2 ) = c.

(11)

For Theorems 1 and 2, and for all sufficiently large n1 (i.e. to the right of the “hump” in Fig. 7), b1 lim n2 = . n1 →∞ 1 − b1

(12)

a bn1 1 . a bn1 1 + (1 − a)(1 − b)n1

Imposing (10) and rearranging yields  n 1−b 1 a(1 − c) = K, K := . b1 c(1 − a)

(18)

Since b = b(n1 ) ↓ (1 − b1 ), write b = (1 − b1 ) + δ with δ = δ(n1 ) ↓ 0. Then,     1 − b1 δ δ ln = − ln 1 + =− + o(δ) . (19) b 1 − b1 1 − b1
, so that From (17), (1 − b)/b1 = exp lnnK 1     δ 1−b ln K ln K 1 1− = = exp =1+ +o . b1 b1 n1 n1 n1 + o(1/n1 ) and So, δ = −b1 lnnK 1     1−b 1 ln K ln +o = . b1 n1 n1

(20)

Substituting (19), (20), and the expansion of δ, in (18) gives

This can be shown in three steps. step I: (deducing b = b(n1 ) from the failure-free condition.) For k = 0 in the regime b > 1 − b1 , Theorems 1 and 2 give Φ0 (b; n1 ) =

1 ln 1−b  b . ln 1−b b1

n2 =

δ + o(δ) − 1−b 1 ln K 1 n1 + o( n1 )

=

1 ln K − n−b + o( n11 ) 1 (1−b1 )

ln K 1 n1 + o( n1 )

−→

b1 1 − b1

as n1 → ∞, which proves (12). The integer asymptotic b1 supremum on n2 this implies is ⌈ 1−b ⌉. 1

(13)

When a < c, as in Scenario 2 in the main text of the paper, n1 →∞ this ensures K ∈ (0, 1). Thus, (1 − b)/b1 = K 1/n1 −→ 1, so b ↓ (1 − b1 ). step II: (recovery after one untyped failure.) For k = 1 and sufficiently large n1 , the least-favorable likelihood in S 2 is attained on [P +Q = 1−b1 ] and the most-favorable likelihood in S 1 ∩[P +Q ⩾ b] is attained on [P +Q = b], so Theorems 1 and 2 give conservative confidence a(1 − b1 )b1n−1 Φ1 (b; n) = . a(1 − b1 )b1n−1 + (1 − a)b(1 − b)n−1 Imposing (11) (with n = n1 + 1 + n2 ) and rearranging yields  n−1 1−b a(1 − c) (1 − b1 ) 1 − b1 = · =K· . (14) b1 c(1 − a) b b

A PPENDIX C P ROOF OF I NTERCHANGE OF I NFIMA IN (9) Proof. Let K := {0, 1, . . . , k} and let D denote the admissible set of priors. The elements of K and D do not depend on each other. Define the objective f : K × D → [0, 1] by f (k1 , P) is equal to the quantity being minimized in (9) (the posterior confidence expression under (k1 , k − k1 ) and prior P). Set α := inf

inf f (k1 , P),

k1 ∈K P∈D

β := inf

inf f (k1 , P).

P∈D k1 ∈K

We show α = β by proving both inequalities. First, α ⩽ β. Fix any P ∈ D. For every k1 ∈ K, inf f (k1 , P′ ) ⩽ f (k1 , P).

P′ ∈D

Taking inf k1 ∈K of both sides gives inf

inf f (k1 , P′ ) ⩽ inf f (k1 , P),

k1 ∈K P′ ∈D

k1 ∈K

16

i.e. α ⩽ inf k1 ∈K f (k1 , P). Since this holds for every P ∈ D, taking inf P∈D yields α ⩽ inf inf f (k1 , P) = β. P∈D k1 ∈K

Second, α ⩾ β. Fix any k1 ∈ K. For every P ∈ D, f (k1 , P) ⩾ inf f (k1′ , P). ′ k1 ∈K

Taking inf P∈D of both sides gives inf f (k1 , P) ⩾ inf inf f (k1′ , P), ′

P∈D

P∈D k1 ∈K

i.e. inf P∈D f (k1 , P) ⩾ β. Since this holds for every k1 ∈ K, taking inf k1 ∈K yields α = inf inf f (k1 , P) ⩾ β. k1 ∈K P∈D

Combining α ⩽ β and α ⩾ β gives α = β.

Related documents

• Development of new advanced estimation algorithms for improving the resilience of the autonomous and connected vehicle#226840
• Runtime Safety Filtering for Learned Small UAS Separation Policies under GNSS Degradation#792784
• Runtime Safety Filtering for Learned Small UAS Separation Policies under GNSS Degradation#788784
• Isolation as a First-Class Principle for LLM-Agent System Safety: Concepts, Taxonomy, Challenges and Future Directions#924112
• Isolation as a First-Class Principle for LLM-Agent System Safety: Concepts, Taxonomy, Challenges and Future Directions#919883