arXiv CSopen access

When Derived Measurements Mislead: Quantifying and Mitigating LLM Over-Trust with Privileged-Modality Reliability Evidence

When Derived Measurements Mislead: Quantifying and Mitigating LLM Over-Trust with Privileged-Modality Reliability Evidence Zongheng Guo1 , Tao Chen2 , Tianli Li5 , Mingzhe Cui2 , Yang Jiao3 , Lei Xie2 , Yi Pan3 , Xiao Hu4 , and Manuela Ferrario1 1

Department of Electronics, Information and Bioengineering, Politecnico di Milano, Milan, Italy State Key Laboratory of Industrial Control Technology, Zhejiang University, Hangzhou, China 3 Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China 4 Nell Hodgson Woodruff School of Nursing, Emory University, Atlanta, USA 5 Department of Integrated Traditional Chinese and Western Medicine Cardiology, China-Japan Friendship Hospital, Beijing, China 2

arXiv:2607.28421v1 [cs.AI] 30 Jul 2026

Abstract

representations, and a downstream large language model (LLM) subsequently reasons over these representations to answer questions or recommend actions. First, an upstream model maps raw observations X = (x1 , ..., xN ) to a compact representation or derived measurements D = g(X), then a downstream large language model (LLM) uses D to answer a question or recommend an action. Such architectures are increasingly common in vision systems that connect LLMs to specialized visual models or computer-aided diagnosis outputs Wang et al. [2024], Wu et al. [2023], in audio systems that coordinate speech interfaces and specialized audio models Huang et al. [2024], and in signal-language systems that pass physiological summaries, derived insights, or encoded time series representations into language reasoning Fang et al. [2024], Langer et al. [2025]. Although this decomposition improves modularity and accessibility, it can transform an uncertain estimation process into an apparently factual token or field. In many deployments, the downstream LLM only observes D. It has no access to the original measurements, the acquisition conditions, or evidence about whether the upstream estimate is reliable for the current instance.

Derived measurements increasingly enter large language model (LLM) pipelines as direct facts despite their instance-dependent validity. We define derived-feature over-trust (DFOT) as the failure in which a downstream LLM assigns such a measurement the epistemic status of a direct fact or uses it outside its valid scope. Using physiological sensing as a case study, D1 tests acceptance of a PPG-derived rhythm contradicted by offline ECG, whereas D2 tests rejection of an offlineconfirmed reliable PPG rhythm under misleading severe history. ECG supplies training supervision and offline reference construction but is never shown to the LLM. Five estimands quantify this chain: conflict over-trust rate (COTR) and context-induced error rate (CIR) characterize D1/D2; correct repair rate (CRR) measures frozenerror repair; evidence-specific repair margin (ESRM) contrasts matched and patient-disjoint shuffled evidence; and utility harm rate (UHR) measures unnecessary verification among HIGH-reliability cases used without verification at baseline. The framework does not depend on a particular reliability generator. We demonstrate it on 50,000 paired PPG–ECG records using ECG-to-PPG privileged distillation as an illustrative baseline and PPGonly inference. On a protocol-locked 187-patient test, the baseline improves four repair and specificity endpoints by 1.82–6.69 percentage points, with all paired confidence intervals excluding zero; UHR increases by 0.67 percentage points (95% CI: −0.4 to +1.7). DFOT provides a common evaluation target for stronger mitigation methods. The code is available at: https://github.com/ZonghengGuo/When-Derived-Measurements-Mislead.

This results in a type of failure that is different from both hallucinations and conventional prediction errors. Even if a downstream model reasons coherently from the information it receives, it can still be wrong because it assigns a derived estimate the epistemic status of a direct fact. We call this derived-feature over-trust (DFOT). DFOT can arise whenever an LLM uses an upstreamderived value whose validity depends on the specific instance. Therefore, it is an interface-level problem rather than a pathology specific to one modality. Unlike probability calibration, which concerns whether an upstream model’s confidence reflects empirical correctness Guo et al. [2017], DFOT concerns a downstream decision maker relies on a derived value appropriately for the specific instance. Unlike selective prediction Geifman and El-Yaniv [2017, 2019], the uncertainty originates in an upstream

Introduction Many contemporary AI systems follow a modular architecture in which specialized upstream models first convert raw observations into structured intermediate 1

Figure 1: End-to-end instantiation of the DFOT framework. A proof-of-concept privileged student converts trainingonly ECG supervision into PPG-only reliability evidence. The baseline reliability generator (B2) uses a conventional PPG-only reliability model, whereas the proposed privileged generator (K2) distills ECG-derived privileged information into the deployment-time PPG reliability estimate. The central matched-versus-shuffled evaluation measures repair (CRR) and evidence-package specificity (ESRM) at the LLM interface. KD: Knowledge Distillation; AF: atrial fibrillation; E: reliability evidence. measurement channel and must survive a semantic interface to a different model. In high-stakes settings such as medicine, failure at this interface can cause an appropriate upstream estimate to be used inappropriately downstream.

and widely deployed, its reliability can vary depending on rhythm, motion, perfusion, device, and acquisition conditions Allen [2007], Charlton et al. [2023]. The uncertainty introduced during this estimation process is largely hidden once only the derived feature is presented to the downstream model. In our implementation, D contains a PPG-derived heart rate and categorical rhythm assessment, while a synchronized ECG serves as a higher-fidelity offline reference. This enables two controlled DFOT scenarios. In D1, the LLM receives a PPG-derived rhythm label that is contradicted by the synchronized ECG reference. In D2, misleading historical information encourages the LLM to reject a current PPG-derived rhythm that is supported by agreement between PPG and ECG. The ECG is only used for privileged training and offline evaluation, and is never shown to the downstream LLM. Rather than proposing another reliability estimator, we ask how over-reliance on derived measurements should be defined, induced, quantified, and distinguished from generic caution. The privileged ECG signal provides one concrete case study, but the primary object of study is the downstream failure itself and the quantities used to characterize it. Our contributions are:

LLM studies have shown that they exhibit unstable behaviour when contextual and parametric evidence conflict Longpre et al. [2021], Chen et al. [2022], Wu et al. [2024], Cattan et al. [2025]. Several studies suggest that medical LLMs often produce definitive answers even under substantial uncertainty, rather than abstaining or expressing calibrated confidence Singhal et al. [2023], Cocchieri et al. [2026]. However, existing evaluations usually treat context as something to be accepted or rejected. They do not directly characterize whether an LLM appropriately relies on a value produced by another estimator when that value is valid for some instances but misleading for others. The distinction is crucial: access to a measurement doesn’t mean that the downstream model uses it appropriately for each instance. We use physiological sensing as a concrete and auditable case study. In the field of wearable health technology, photoplethysmography (PPG) data is converted into heart rate or rhythm labels, which are then interpreted by a language model. Although PPG is convenient

• We introduce DFOT as a distinct downstream failure mode: an LLM assigns a derived estimate the 2

epistemic status of a direct fact, rather than treating it as an uncertain measurement whose validity depends on the current instance.

[2020], Antiperovitch et al. [2024], Pereira et al. [2020], Pillai et al. [2025], Guo et al. [2026, 2025]. Related works also investigate multimodal transfer, remote sensing, and learning from noisy physiological signals Geenjaar et al. [2026], Guo et al. [2023], Chen et al. [2024, 2026], Li et al. [2026], Tang et al. [2024], Langer et al. [2025]. These works improve an upstream estimator, whereas DFOT evaluates whether a downstream LLM appropriately relies on the resulting derived measurements.

• We formalize two complementary challenge mechanisms and five linked estimands—COTR, CIR, CRR, ESRM, and UHR—that use fixed (“frozen”) denominators to disentangle failure induction, conditional repair, evidence specificity, and interface harm. • We introduce a matched-versus-patient-disjointshuffled evidence intervention that tests whether revision depends on information specific to the current instance rather than the mere presence of reliability language.

Privileged information as an instantiation. Learning with privileged information (LUPI) and distillation transfer supervision available only during training Vapnik and Vashist [2009], Hinton et al. [2015], Lopez-Paz et al. [2016], including ranking-based, imperfect-teacher, cross-modal, and ECG-guided PPG formulations Yang et al. [2022], Martı́nez-Garcı́a et al. [2025], Gupta et al. [2016], Wei et al. [2025], Ni et al. [2025]. We use this family of methods to construct one reliability signal, but our primary contribution is the formulation of DFOT as a downstream failure mode together with controlled interventions and estimands that quantify failure induction, repair, evidence specificity, and downstream harm.

• We provide a protocol-locked PPG–ECG benchmark together with a privileged-distillation baseline, establishing a reproducible point of comparison for stronger future DFOT mitigation methods.

Related Work Appropriate reliance and evidence use. Knowledge-conflict benchmarks manipulate retrieved evidence or introduce conflicts between contextual evidence and a model’s parametric knowledge Longpre et al. [2021], Chen et al. [2022], Wu et al. [2024]. Self-evaluation and semantic entropy estimate the reliability or uncertainty of generated answers Kadavath et al. [2022], Kuhn et al. [2023]. Beyond question answering, LLMs increasingly serve as decision-making agents in domains such as autonomous scientific analysis and code optimization Yuan et al. [2026], Kuang et al. [2025]. These settings further highlight the importance of appropriately using intermediate information, although robust iterative optimization remains challenging Nie et al. [2026]. Calibration, selective prediction, and learning to defer address predictive uncertainty and decision risk Guo et al. [2017], Lakshminarayanan et al. [2017], Ovadia et al. [2019], Geifman and El-Yaniv [2019], Madras et al. [2018], Mozannar and Sontag [2020]. In contrast, DFOT asks whether a downstream agent appropriately relies on an instance-specific derived measurement. However, warnings, explanations, or attention weights do not demonstrate that a model uses evidence appropriatelyJain and Wallace [2019], Wiegreffe and Pinter [2019]. We therefore evaluate evidence use through matched–shuffled interventions and decision-level estimands.

DFOT: problem formulation and metric chain In general, let Xd,i = (xd,i,1 , . . . , xd,i,Ni ) denote the deployment-modality measurements available for instance i, and let Xp,i = (xp,i,1 , . . . , xp,i,Tp,i ) denote a synchronized, higher-fidelity privileged modality available only during development and offline evaluation. The two modalities may be multivariate and need not have identical sampling rates or sequence lengths, provided that they refer to the same physiological episode. An upstream estimator maps the deployment signal to a derived feature, Di = g(Xd,i ), while a separate deploymenttime reliability model produces a reliability estimate, Ri = S(Xd,i ). The reliability estimate may be converted into semantic evidence through a rendering function h so that Ei = h(Ri ). The downstream model M receives a task description Qi , contextual information Ci , the derived feature Di , and, optionally, the reliability evidence Ei : Oi = M (Qi , Di , Ci , Ei ),

Xp,i ∈ / input(M ).

(1)

Here, Ei = ∅ denotes the no-evidence condition. The privileged modality is never provided to the downstream model. Derived-feature over-trust occurs when the downstream model fails to condition its reliance on Di on the feature’s instance-specific validity or intended scope. In D1, the model accepts a current derived feature that is contradicted by the privileged reference. In D2, out-of-scope historical context causes the model to reject a current derived feature that is supported by agreement between the

Derived physiological measurements. PPG enables scalable heart-rate and rhythm monitoring Tison et al. [2018], Perez et al. [2019], but error varies across devices and activities Shcherbina et al. [2017], Bent et al. [2020]. Raw-waveform, multitask, signal-quality-aware, and foundation models improve physiological representations Torres-Soto and Ashley [2020], Aschbacher et al. 3

N : dsevere ̸= yi }. All evidence conditions use these fixed i denominators. ESRM operationally measures matchedpackage specificity beyond the generic effect of receiving a reliability channel; it does not separately identify label, score, or wording effects.

deployment and privileged modalities. In our instantiation, ECG is treated as a higher-fidelity rhythm reference rather than as infallible ground truth. Let C denote the set of reference-confirmed D1 conflict instances, and let N denote the set of D2 instances for which the no-evidence, neutral-context condition yields the correct downstream decision. Let bi denote the parsed downstream decision in the corresponding no-evidence challenge condition, and let yi denote the parser-defined target decision. Let F denote the relevant challenge-specific frozen error set, F1 for D1 or(F2 for D2, as defined below. In our experiments, Suspect, i ∈ D1, yi = . T rustworthy, i ∈ D2. Let dsevere denote the parsed decision in the no-evidence i severe-history condition. Let di (e) denote the parsed downstream decision under evidence condition e, and let ui (e) denote its parsed measurement-use field. Let zi ∈ {High, Low} denote the offline reference-defined reliability class, which is distinct from the deploymenttime estimate Ri . The baseline-eligible utility set is

Claim hierarchy. The design separates four progressively stronger claims: accessibility, whether a reference channel contains learnable information; transfer, whether it improves a deployment-time reliability signal; utilization, whether that signal repairs downstream decisions; and specificity, whether repair depends on the current instance. Upstream accuracy addresses the first two claims, CRR addresses utilization, and ESRM is required for specificity. Benchmark and reporting contract. The protocol is agnostic to the reliability generator: calibration, ensembles, selective prediction, multimodal teachers, or other uncertainty models may provide case-level evidence. A valid DFOT instantiation requires an independently defined reference relation that determines whether reliance on Di is appropriate, an observable reliance failure, and a downstream target that can be evaluated independently of the reliability signal. The evidence must not reveal the withheld reference value. A matched–control intervention must preserve the case, task, evidence format, and decision history while breaking only the correspondence between the evidence and the current instance. Cases, baseline decisions, parsers, and denominators must be frozen before comparison. Each challenge should report its baseline characterization rate, absolute matched and shuffled outcomes, their paired difference, and uncertainty respecting the sampling unit. Absolute CRR reveals residual error burden, ESRM distinguishes matchedcase evidence use from generalized caution, and UHR or a domain-appropriate cost endpoint quantifies harm on a separately defined set whose baseline decision is already useful. We avoid a weighted composite because acceptable tradeoffs among repair, specificity, abstention, workload, delay, and missed detection are application dependent; the metric vector instead supports auditable Pareto comparisons.

H = {i : zi = High ∧ ui (∅) = Use-As-Is} . We abbreviate matched and shuffled evidence by m and s. The five estimands are 1 X COTR = 1{bi accepts Di } , |C| i∈C

1 X CIR = 1{dsevere ̸= yi } , i |N | i∈N

1 X 1{di (e) = yi } , CRR(e) = |F| i∈F

ESRM = CRR(m) − CRR(s) i 1 Xh = 1{di (m) = yi } − 1{di (s) = yi } , |F| i∈F

1 X UHR = 1{ui (m) = Verify-First} . |H| i∈H

(2) COTR measures acceptance of a derived measurement despite conflict with the privileged reference , CIR measures context-induced rejection of a derived measurement supported by the privileged reference, and CRR measures parser-defined decision repair on a frozen error set. ESRM contrasts matched and shuffled evidence packages beyond mere presence of reliability evidence. UHR intentionally focuses on evidence-induced unnecessary verification of initially usable measurements. Together, these estimands transform DFOT from an informal concern into a set of falsifiable comparisons with explicitly defined denominators.

DFOT evaluation protocol Using the method-independent contract above, we instantiate two complementary challenge mechanisms. D1 presents hidden-irregularity cases in which the PPGderived rhythm appears regular and freezes the subset that the downstream LLM accepts despite disagreement with the offline ECG reference. D2 holds a currently regular segment fixed while introducing severe atrial fibrillation (AF)-related history that induces an incorrect judgment of the current window. Only cases that are

Frozen estimands. D1 freezes F1 = {i ∈ C : bi ̸= yi }, whereas D2 first restricts to N and freezes F2 = {i ∈ 4

answered correctly under neutral history are eligible for the frozen baseline-error set.

Cohort and cases. The source scan of the MIMIC-III Matched Waveform Database v1.0 identified 1,297 candidate patients with simultaneous PLETH and Lead-II recordings. After segmentation and an eligibility check, the final cohort comprised 50,000 synchronized fourminute records from 1,275 patients: 36,115 records from 925 patients (pts) were used for training, 6,464 records form 163 pts for validation, and 7,421 records from 187 pts for a locked test. These splits are patient-disjoint. PPG was resampled at 50 Hz (12,000 samples per each segment) and Lead-II ECG at 125 Hz (30,000 samples per segment). Reliability is defined based on four clear rhythm cases: both regular and both irregular are HIGH, while PPG-regular/ECG-irregular (hidden irregularity) and PPG-irregular/ECG-regular (false alarm) are LOW. The test contains 3,733 clear cases (897 LOW, 2,836 HIGH), including 401 cases of hidden irregularity and 2,151 cases of both regular rhythms. This construction prevents two label collisions. An irregular rhythm is classified as ’HIGH reliability’ when both modalities agree, whereas regular-looking PPG is classified as ’LOW reliability’ when the ECG indicates hidden irregularity. Other relations are excluded from clear-label endpoints. The low-FPR analysis restricts further positives to hidden irregularity and negatives to both regular records, which is a narrower target than global LOW/HIGH AUROC. Full partition and case tables are provided in the Supplementary Material.

Challenge sets and population estimates. D1 and D2 are controlled stress tests rather than prevalence estimates. Each holds the target case fixed, isolates one reliance mechanism. COTR and CIR verify that the intended failure has been induced before CRR and ESRM evaluate conditional repair on the corresponding frozen denominator. These rates therefore characterize DFOT inducibility and conditional repair, not population event frequency. Matched, shuffled, and harm controls. Reactive mitigation appends reliability evidence to the frozen baseline interaction, directly testing revision of an observed error. Matched evidence is taken from the current record. Patient-disjoint shuffled evidence preserves the case, baseline answer, timing, treatment arm, Student seed, and prompt template while replacing only the evidence source. The resulting matched–shuffled CRR difference defines evidence-package specificity through ESRM. A separate baseline-eligible HIGH-reliability set H measures UHR under matched evidence: unnecessary movement from Use-As-Is to Verify-First. Frozen execution and inference. E0 evaluates Qwen3-8B under the primary P1 prompt; E1 holds P1 fixed across Qwen3-8B, DeepSeek-V4-Pro, and GPT-5.5; E2 tests three semantically equivalent prompts within Qwen3-8B; and E3 applies the frozen Qwen3-8B/P1 protocol to patient-disjoint test patients with no prior downstream LLM queries. Qwen requests use vLLM 0.25.1, temperature 0, at most 96 output tokens, disabled thinking, and deterministic parsing. Before E3 inference, cases, prompts, parser, donor rules, thresholds, gates, and bootstrap settings were frozen. E3 uses 5,000 patient-cluster bootstrap replicates, resampling three Student seeds with replacement and sharing each seed draw across paired arms and evidence conditions. Four efficacy gates require the lower confidence bounds for the differences in matched CRR and ESRM between the proposed K2 generator and the baseline B2 generator to remain positive in both D1 and D2 (see Figure 1 (B)).

Proof-of-Concept Case ECG-Referenced PPG

Hierarchical ECG teacher. The teacher processes eight contiguous 30-s ECG windows with a shared encoder, gated multi-view fusion, and a temporal Transformer, to produce both window-level and record-level three-class predictions. Averaged over three random seeds, it achieves a record-level rhythm AUROC is 0.9912 ± 0.0007. Architectural details and auxiliary training objectives are provided in the Supplementary Material. PPG-only student. The student is initialized from SIGMA-PPG Guo et al. [2026], pools 120 PPG patches into eight windows aligned with the teacher, and predicts the probability of low reliability (plow ). B2 optimizes reliability BCE; the proof-of-concept K2 additionally matches aligned teacher outputs: LK2 = Lrel + 0.15 · (LwinKD + LglobalKD ),

Study:

(T )

(T )

LKD = T 2 KL(pT ∥pS ), (T ) p• = softmax(z• /T ),

The empirical case study is designed to validate the DFOT formulation’s ability to distinguish reliability interfaces, rather than propose an exhaustive solution. ECGreferenced supervision provides a specific proof-of-concept signal that is not available at deployment. The evaluation framework itself does not depend on this particular teacher, modality pair, or distillation objective.

(3) (4)

T = 2.

(5)

Here zT , zS ∈ R3 are teacher and student logits aligned by window index. Window-level Knoledge Distillation (KD) averages the Kullback–Leibler divergence (KL) over the eight aligned windows, whereas global KD is applied once per record. K2-global-shuffled globally permutes the teacher targets while preserving architecture, marginal 5

targets, losses, and optimization. B2, K2, and K2-globalshuffled share the same training schedule, optimization budget, data order, and random seeds (42, 123, and 2026). T = 2 and the 0.15/0.15 weights were selected using the validation set and frozen before access to the test set. Detailed architecture, index-alignment audit, hyperparameters, and the 17-arm ablation study are provided in the Supplementary Material

pp. All four paired confidence intervals exclude zero, and every Student seed preserves the direction of effect. Robustness across models and prompts. Under P1, the K2–B2 efficacy contrasts remain positive for Qwen38B, DeepSeek-V4-Pro, and GPT-5.5, with no directional reversal. Within Qwen3-8B, the effect also remains positive across P1–P3; pooled D1 and D2 ESRM gains are +0.064 [0.029, 0.097] and +0.071 [0.035, 0.108], respectively. This is not a complete LLM-by-prompt factorial evaluation.

Evidence interface. The student output qualifies the use of the PPG-derived rhythm rather than revealing an ECG diagnosis. The fixed language-facing block contains a HIGH/LOW label, plow rounded to four decimals, and an explanation that the score estimates current-segment surrogate reliability. LOW denotes plow ≥ 0.5 and HIGH otherwise; the frozen interface contains no label–score mismatches. No ECG waveform, rhythm value, diagnosis, teacher logit, or embedding is shown to the downstream LLM. Matched and shuffled conditions preserve the case, baseline interaction, arm, seed, timing, and template while changing only the patient-disjoint evidence source. The exact block and complete prompts are provided in the Supplementary Material.

The proof-of-concept supplies a discriminative baseline. The 17-arm development study selects K2 at AUROC of .9042 versus B2 at .8927, while withincase shuffled-target K5 remains close to .9003, suggesting that both privileged regularization and correct teacher– student correspondence contribute. On the independently locked upstream test, K2 exceeds B2 by .0250 AUROC [.0118, .0274] and globally shuffled ECG by .0115 [.0095, .0455]. The framework therefore distinguishes between evidence generators before and after the language interface. However, extreme low-FPR effects remain unresolved, meaning this discriminative baseline is a starting point rather than an endpoint for DFOT mitigation.

Worked metric example. In one D1 case, the PPGderived rhythm is regular while the offline ECG reference is irregular. The no-evidence verdict is TRUSTWORTHY, so the case enters F1 . Matched LOW evidence changes the verdict to SUSPECT, giving 1{di (m) = yi } = 1, whereas shuffled HIGH evidence leaves the error unchanged, giving 1{di (s) = yi } = 0. The case therefore contributes one to ESRM. D2 is evaluated analogously after severe history induces an error; UHR instead uses the separate baselineeligible set H. Complete D1 and D2 examples are provided in the Supplementary Material.

Negative controls sharpen the research target. A tail-aware objective improves the development tail mean by 3.06 percentage points (95% CI 1.28–5.56) over K2 and by 2.59 points (95% CI 0.85–4.53) over hiddenweighted BCE, but it does not demonstrate a reliable advantage over a patient-deranged shuffled-tail control: (+1.31) points (95% CI (-1.70) to (+4.50)); pAUC difference (0.0057) (95% CI (-0.0105) to (0.0212)). This control preserves the number of positive assignments (346) and all training mechanics while retaining the true hidden label for only 10.69% of examples. We therefore do not pursue further tail-specific optimization. Instead, we conclude that the additional gain cannot be attributed to label-specific privileged information, highlighting the importance of negative controls in distinguishing genuine correspondence effects from generic optimization benefits.

Results DFOT induction. On validation, D1 COTR is 1.000 (300 records; 37 patients), and D2 CIR is 0.942 (292/310 neutral-correct records). E3 reproduces both manipulations: all 300 D1 records from 44 patients are over-trusted, while severe history induces an error in all 297 neutralcorrect D2 records from 97 patients, giving CIR = 1.000. These controlled manipulation checks define the frozen error sets and are not prevalence estimates.

Discussion DFOT as a research target. The principal contribution of this work is a problem definition and measurement framework. Modular AI systems are commonly evaluated through either upstream prediction performance or downstream task accuracy, leaving the interface between them largely unexamined. An upstream estimate may be accurate on average yet unreliable for a particular instance, while a downstream model may reason coherently but assign that estimate more authority than its provenance supports. DFOT names this failure and separates its induction, repair, instance specificity, and interface cost

Conditional repair and specificity. On controlled validation, all four K2–B2 efficacy intervals exclude zero. On E3, D1 matched CRR/ESRM increase from 40.11/21.67% with B2 to 44.83/26.17% with K2, corresponding to gains of 4.72 and 4.50 pp. Thus, 55.17% of the deliberately induced D1 errors remain unrepaired, leaving substantial mitigation headroom. D2 matched CRR/ESRM increase from 94.39/15.82% to 96.21/22.51%, corresponding to gains of 1.82 and 6.69 6

A

Patient-disjoint locked Student test

K2

B2 0.93061

Downstream paired effects

0.93262

0.93

AUROC

B

Locked-test reliability transfer

D1 matched CRR

+5.2 [1.3, 6.5]

D1 ESRM

+4.9 [1.1, 8.9]

D2 matched CRR

+1.3 [0.7, 3.8]

D2 ESRM

+6.8 [1.1, 11.9]

0.92600 +0.0210

0.92 +0.0289 +0.0251

0.91

0.91162

0.90 0.90171

0.90090

Seed 42

Seed 123

Seed 2026

0

3

6

9

12

K2-B2 effect (percentage points) Paired bootstrap: K2-B2 +.0250 [.0118, .0274]

C

K2-ECG-shuffled +.0115 [.0095, .0455]

D

Robustness across LLMs and prompts LLM families under P1

D1 ESRM 0.049

Qwen3-8B DeepSeek-V4-Pro

0.068

Exploratory 2 pp limit

+0.87

Qwen3-8B

0.071

0.045

+1.33 0.060

GPT-5.5

Verification-harm tolerance

D2 ESRM

0.112

DeepSeek-V4-Pro +0.97

Qwen3-8B prompt formulations 0.049

P1 formal

0.067 0.070

P2 clinical narrative

+0.93

P1-P3 pooled Qwen

0.076 0.074

P3 reliability/status separated 0.00

GPT-5.5

0.068

0.03

0.06

0.09

0.12

K2-B2 ESRM (difference in proportion)

−0.5

0.0

0.5

1.0

1.5

2.0

K2-B2 UHR difference (percentage points) All 95% CI upper bounds < exploratory limit

Figure 2: DFOT results in the PPG–ECG case study. (A) Locked-test upstream reliability transfer. (B) Validationstage conditional repair and evidence-package specificity. (C) Robustness across LLM families under P1 and across prompt formulations under Qwen3-8B. (D) Interface verification harm. The independent E3 replication is reported in Table 1. into testable estimands. The matched–shuffled intervention further tests whether revision depends on evidence corresponding to the current instance rather than on the mere presence of reliability language.

evidence from a shuffled donor despite limited remaining repair headroom, illustrating why absolute CRR and ESRM should be interpreted jointly. K2 should therefore be viewed as a reference baseline for the newly defined task rather than as a definitive solution. ECG-referenced distillation provides one convenient deployment-time PPG-only reliability signal with which to exercise the framework. The ablations suggest that regularization and category-level structure explain part of its benefit, while the locked global-shuffled comparison supports an additional contribution from correct teacher– student correspondence. Future DFOT methods need not use the same teacher, architecture, loss, or evidence representation.

What the case study establishes. The PPG–ECG case study shows that the framework is non-degenerate and discriminative. Both challenges reproducibly induce their intended failures, the metric chain distinguishes B2 from K2, and the comparison replicates on records from the patient-disjoint locked-test partition. The contrasts also persist across the tested LLM families under P1 and across prompt formulations within Qwen3-8B. D1 and D2 probe complementary forms of inappropriate reliance: accepting a misleading derived measurement and allowing out-of-scope historical context to override a valid current measurement. K2 improves D1 matched repair and ESRM, although substantial residual error leaves clear headroom for stronger methods. D2 matched repair is already near ceiling, making its larger ESRM improvement more informative than its smaller CRR gain. K2 more strongly separates matched current-case

Why a metric vector is necessary. Upstream transfer, downstream repair, evidence specificity, and interface cost capture different properties. Improved AUROC establishes better reliability ranking, not downstream use. Increased CRR establishes repair, but without ESRM it cannot distinguish matched-case evidence use from generic caution. UHR separately quantifies unnecessary 7

Endpoint D1 matched CRR D1 ESRM D2 matched CRR D2 ESRM UHR∗

Validation ∆ [95% CI] E3 B2 E3 K2 +5.2 [1.3,6.5] +4.9 [1.1,8.9] +1.3 [0.7,3.8] +6.8 [1.1,11.9] +0.87 [−0.60,1.40]

40.11 21.67 94.39 15.82 2.89

E3 ∆ [95% CI] Criterion

44.83 +4.72 [1.67,7.28] 26.17 +4.50 [0.98,8.18] 96.21 +1.82 [1.51,2.91] 22.51 +6.69 [2.60,7.19] 3.56 +0.67 [−0.40,1.70]

Met Met Met Met Met

Table 1: Controlled-validation effects and protocol-locked E3 replication. E3 B2/K2 columns are absolute percentages; deltas and CIs are percentage points. All efficacy CI lower bounds exceed zero. ∗ The UHR criterion is an exploratory interface-engineering limit (upper CI < 2 pp), not clinical non-inferiority.

Ethical Statement

verification on measurements that were already usable at baseline.

This study uses retrospective, de-identified MIMIC waveform data which was accessed in accordance with the relevant credentials and data-use requirements. The study doesn’t involve any prospective recruitment or clinical intervention. The proposed framework evaluates controlled LLM behavior and is not intended for use in direct clinical decision-making.

The unresolved low-FPR behavior reinforces this separation: global ranking can improve without improving the stringent operating region required by a particular workflow. DFOT mitigation is therefore multi-objective, involving failure coverage, repair, matched-case specificity, stringent-region behavior, and intervention cost. The exploratory 2-pp UHR limit is an interface-engineering constraint within this vector, not a universal clinical margin.

Conclusion We introduce and formalize derived-feature over-trust, provide a five-estimand metric chain, and use matchedversus-shuffled interventions to distinguish matched-case repair from generic caution. A protocol-locked PPG–ECG case study demonstrates that the framework is reproducible and discriminative across tested patients, LLM families, and prompt formulations. The proof-of-concept K2 baseline improves reliability ranking, downstream repair, and evidence-package specificity while leaving clear headroom for stronger mitigation methods. Although demonstrated with physiological signals, the DFOT definition and evaluation chain apply more broadly wherever downstream models reason over instance-dependent derived measurements. DFOT is therefore a concrete evaluation target and an open research agenda for more reliable measurement-to-LLM pipelines.

Transfer beyond physiological sensing. Although the empirical case study uses synchronized PPG and ECG, DFOT can arise whenever an upstream model produces an instance-dependent derived measurement that is consumed by a downstream reasoner. A new instantiation requires a derived measurement, an independently defined criterion for appropriate reliance, a controlled challenge exposing inappropriate reliance, and a matched– shuffled or equivalent counterfactual control that breaks instance correspondence while preserving the interface. The reference may come from a higher-fidelity modality, repeated measurements, expert adjudication, or another independently specified source. What transfers across domains is the problem definition, metric chain, and intervention logic; the particular reliability generator and numerical effects remain application-specific.

References John Allen. Photoplethysmography and its application in clinical physiological measurement. Physiological Measurement, 28(3):R1–R39, 2007. doi: 10.1088/09673334/28/3/R01.

Research agenda. Future work can compare calibrated uncertainty heads, ensembles, conformal or selective predictors, retrieval-based verification, and stronger multimodal students within the same frozen downstream protocol. The purpose of the present study is not to declare DFOT solved by one ECG–PPG method, but to make a previously diffuse interface failure visible, measurable, and comparable. The formal definition, controlled challenges, linked estimands, and reference baseline establish a common target against which stronger mitigation methods can be developed.

Pavel Antiperovitch, David Mortara, Joshua Barrios, Robert Avram, Kimberly Yee, Armeen Namjou Khaless, Ashley Cristal, Geoffrey Tison, and Jeffrey Olgin. Continuous atrial fibrillation monitoring from photoplethysmography: Comparison between supervised deep learning and heuristic signal processing. JACC: Clinical Electrophysiology, 10(2):334–345, 2024. doi: 10.1016/j.jacep.2024.01.008. 8

Kirstin Aschbacher, Defne Yilmaz, Yaniv Kerem, et al. Atrial fibrillation detection from raw photoplethysmography waveforms: A deep learning application. Heart Rhythm O2, 1(1):3–9, 2020. doi: 10.1016/j.hroo.2020. 02.002.

Yonatan Geifman and Ran El-Yaniv. Selective classification for deep neural networks. In Advances in Neural Information Processing Systems, volume 30, 2017. Yonatan Geifman and Ran El-Yaniv. SelectiveNet: A deep neural network with an integrated reject option. In Proceedings of the 36th International Conference on Machine Learning, pages 2151–2159, 2019.

Brinnae Bent, Benjamin A. Goldstein, Warren A. Kibbe, and Jessilyn P. Dunn. Investigating sources of inaccuracy in wearable optical heart rate sensors. npj Digital Medicine, 3:18, 2020. doi: 10.1038/s41746-020-0226-6.

Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q. Weinberger. On calibration of modern neural networks. In Proceedings of the 34th International Conference on Machine Learning, pages 1321–1330, 2017.

Arie Cattan, Alon Jacovi, Ori Ram, Jonathan Herzig, Roee Aharoni, Sasha Goldshtein, Eran Ofek, Idan Szpektor, and Avi Caciularu. (d)RAGged into a conflict: Detecting and addressing conflicting sources in retrieval-augmented LLMs. Google Research publication, 2025. Accessed July 2026.

Zongheng Guo, Huahua Chen, Lili Lin, Wenhui Zhou, Meng Yang, Na Ying, and Chunsheng Guo. Remote heart rate estimation via convolutional neural networks with transformers. Journal of the Franklin Institute, 360(17):13149–13165, 2023. doi: 10.1016/j.jfranklin. 2023.10.013.

Peter H. Charlton et al. The 2023 wearable photoplethysmography roadmap. Physiological Measurement, 44 (11):111001, 2023. doi: 10.1088/1361-6579/acead2.

Zongheng Guo, Tao Chen, and Manuela Ferrario. QualityFM: A multimodal physiological signal foundation model with self-distillation for signal quality challenges in critically ill patients. arXiv preprint arXiv:2509.06516, 2025.

Huahua Chen, Xiang Zhang, Zongheng Guo, Na Ying, Meng Yang, and Chunsheng Guo. ACTNet: Attention based CNN and transformer network for respiratory rate estimation. Biomedical Signal Processing and Control, 96:106497, 2024. doi: 10.1016/j.bspc.2024. 106497.

Zongheng Guo, Tao Chen, Yang Jiao, Yi Pan, Xiao Hu, and Manuela Ferrario. SIGMA-PPG: Statistical-prior informed generative masking architecture for PPG foundation model. arXiv preprint arXiv:2601.21031, 2026.

Hung-Ting Chen, Michael Zhang, and Eunsol Choi. Rich knowledge sources bring complex knowledge conflicts: Recalibrating models to reflect conflicting evidence. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 2292– 2307, 2022.

Saurabh Gupta, Judy Hoffman, and Jitendra Malik. Cross modal distillation for supervision transfer. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2827–2836, 2016.

Tao Chen, Mingzhe Cui, Zongheng Guo, Chenhao Wu, Lei Xie, and Luca Mainardi. Instantaneous frequencychirprate region and synchrosqueezing in the timefrequency-chirprate space. Signal Processing, 246: 110607, 2026. doi: 10.1016/j.sigpro.2026.110607.

Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531, 2015. Rongjie Huang, Mingze Li, Dongchao Yang, Jiatong Shi, Xuankai Chang, Zhenhui Ye, Yuning Wu, Zhiqing Hong, Jiawei Huang, Jinglin Liu, et al. Audiogpt: Understanding and generating speech, music, sound, and talking head. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 23802–23804, 2024.

Alessio Cocchieri, Luca Ragazzi, Giuseppe Tagliavini, and Gianluca Moro. LLMs (almost) never abstain under medical uncertainty. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics, pages 29573–29613, 2026. doi: 10.18653/ v1/2026.acl-long.1365.

Sarthak Jain and Byron C. Wallace. Attention is not explanation. In Proceedings of NAACL-HLT, pages 3543–3556, 2019. doi: 10.18653/v1/N19-1357.

Cathy Mengying Fang, Valdemar Danry, Nathan Whitmore, Andria Bao, Andrew Hutchison, Cayden Pierce, and Pattie Maes. Physiollm: Supporting personalized health insights with wearables and large language models. In 2024 IEEE EMBS International Conference on Biomedical and Health Informatics (BHI), pages 1–8. IEEE, 2024.

Saurav Kadavath, Tom Conerly, Amanda Askell, et al. Language models (mostly) know what they know. arXiv preprint arXiv:2207.05221, 2022. Zhiyi Kuang, Ryan Rong, YuCheng Yuan, and Allen Nie. Learning game-playing agents with generative code optimization. In ICML 2025 Workshop on Programmatic Representations for Agent Learning, 2025.

Eloy Geenjaar, Vince Calhoun, Scott Daly, et al. A robust PPG foundation model using multimodal physiological supervision. arXiv preprint arXiv:2606.07365, 2026. 9

Lorenz Kuhn, Yarin Gal, and Sebastian Farquhar. Semantic uncertainty: Linguistic invariances for uncertainty estimation in natural language generation. In International Conference on Learning Representations, 2023.

Yaniv Ovadia, Emily Fertig, Jie Ren, Zachary Nado, D. Sculley, Sebastian Nowozin, Joshua Dillon, Balaji Lakshminarayanan, and Jasper Snoek. Can you trust your model’s uncertainty? evaluating predictive uncertainty under dataset shift. In Advances in Neural Information Processing Systems, volume 32, 2019.

Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and scalable predictive uncertainty estimation using deep ensembles. In Advances in Neural Information Processing Systems, volume 30, 2017.

Tania Pereira, Nate Tran, Kais Gadhoumi, Michele M. Pelter, Duc H. Do, Randall J. Lee, Rene Colorado, Karl Meisel, and Xiao Hu. Photoplethysmography based atrial fibrillation detection: A review. npj Digital Medicine, 3:3, 2020. doi: 10.1038/s41746-019-0207-9.

Patrick Langer, Thomas Kaar, Max Rosenblattl, et al. OpenTSLM: Time-series language models for reasoning over multivariate medical text- and time-series data. arXiv preprint arXiv:2510.02410, 2025.

Marco V. Perez, Kenneth W. Mahaffey, Haley Hedlin, et al. Large-scale assessment of a smartwatch to identify atrial fibrillation. New England Journal of Medicine, 381(20):1909–1917, 2019. doi: 10.1056/ NEJMoa1901183.

Chenqi Li, Yu Liu, Timothy Denison, and Tingting Zhu. BioX-Bridge: Model bridging for unsupervised crossmodal knowledge transfer across biosignals. In International Conference on Learning Representations, 2026.

Arvind Pillai, Dimitris Spathis, Fahim Kawsar, and Mohammad Malekzadeh. PaPaGei: Open foundation models for optical physiological signals. In International Conference on Learning Representations, 2025.

Shayne Longpre, Kartik Perisetla, Anthony Chen, Nikhil Ramesh, Chris DuBois, and Sameer Singh. Entitybased knowledge conflicts in question answering. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 7052–7063, 2021. doi: 10.18653/v1/2021.emnlp-main.565.

Anna Shcherbina, C. Mikael Mattsson, Daryl Waggott, et al. Accuracy in wrist-worn, sensor-based measurements of heart rate and energy expenditure in a diverse cohort. Journal of Personalized Medicine, 7(2):3, 2017. doi: 10.3390/jpm7020003.

David Lopez-Paz, Leon Bottou, Bernhard Schölkopf, and Vladimir Vapnik. Unifying distillation and privileged information. In International Conference on Learning Representations, 2016.

Karan Singhal, Shekoofeh Azizi, Tao Tu, et al. Large language models encode clinical knowledge. Nature, 620:172–180, 2023. doi: 10.1038/s41586-023-06291-2.

David Madras, Toniann Pitassi, and Richard Zemel. Predict responsibly: Improving fairness and accuracy by learning to defer. In Advances in Neural Information Processing Systems, volume 31, 2018.

Hui Tang, Gang Ma, Lishen Qiu, Lesong Zheng, Rui Bao, Jing Liu, and Lirong Wang. Blood pressure estimation based on PPG and ECG signals using knowledge distillation. Biomedical Engineering Letters, 15(1):39–51, 2024. doi: 10.1007/s13239-023-00695-x.

Mario Martı́nez-Garcı́a, Iñaki Inza, and Jose A. Lozano. Teacher privileged distillation: How to deal with imperfect teachers? Knowledge-Based Systems, 316:113338, 2025. doi: 10.1016/j.knosys.2025.113338.

Geoffrey H. Tison, Jose M. Sanchez, Brandon Ballinger, et al. Passive detection of atrial fibrillation using a commercially available smartwatch. JAMA Cardiology, 3(5):409–416, 2018. doi: 10.1001/jamacardio.2018.0136.

Hussein Mozannar and David Sontag. Consistent estimators for learning to defer to an expert. In Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pages 7076–7087, 2020.

Jessica Torres-Soto and Euan A. Ashley. Multi-task deep learning for cardiac rhythm detection in wearable devices. npj Digital Medicine, 3:116, 2020. doi: 10. 1038/s41746-020-00320-4.

Juntong Ni, Saurabh Kataria, Shengpu Tang, Carl Yang, Xiao Hu, and Wei Jin. PPG-Distill: Efficient photoplethysmography signals analysis via foundation model distillation. arXiv preprint arXiv:2509.19215, 2025.

Vladimir Vapnik and Akshay Vashist. A new learning paradigm: Learning using privileged information. Neural Networks, 22(5–6):544–557, 2009. doi: 10.1016/j. neunet.2009.06.042.

Allen Nie, Xavier Daull, Zhiyi Kuang, Abhinav Akkiraju, Anish Chaudhuri, Max Piasevoli, Ryan Rong, YuCheng Yuan, Prerit Choudhary, Shannon Xiao, et al. Understanding the challenges in iterative generative optimization with LLMs. arXiv preprint arXiv:2603.23994, 2026.

Sheng Wang, Zihao Zhao, Xi Ouyang, Tianming Liu, Qian Wang, and Dinggang Shen. Interactive computeraided diagnosis on medical image using large language models. Communications Engineering, 3(1):133, 2024. 10

Riling Wei, Hanjie Chen, Kelu Yao, Chuanguang Yang, Jun Wang, and Chao Li. ECG-guided individual identification via PPG. In IEEE International Conference on Acoustics, Speech and Signal Processing, 2025. doi: 10.1109/ICASSP49660.2025.10890435. Sarah Wiegreffe and Yuval Pinter. Attention is not not explanation. In Proceedings of EMNLP-IJCNLP, pages 11–20, 2019. doi: 10.18653/v1/D19-1002. Chenfei Wu, Shengming Yin, Weizhen Qi, Xiaodong Wang, Zecheng Tang, and Nan Duan. Visual chatgpt: Talking, drawing and editing with visual foundation models. arXiv preprint arXiv:2303.04671, 2023. Kevin Wu, Eric Wu, and James Zou. ClashEval: Quantifying the tug-of-war between an LLM’s internal prior and external evidence. arXiv preprint arXiv:2404.10198, 2024. Shuo Yang, Sujay Sanghavi, Holakou Rahmanian, Jan Bakus, and S. V. N. Vishwanathan. Toward understanding privileged features distillation in learning-torank. In Advances in Neural Information Processing Systems, volume 35, 2022. Yucheng Yuan, Yuanfeng Ji, Zhongxiao Li, and Ruijiang Li. SP-Mind: An autonomous reasoning agent for spatial proteomics analysis. arXiv preprint arXiv:2606.24235, 2026.

11

Supplementary Material A

Scope and Reading Guide

This supplement documents the experiment ledger, complete arm definitions, protocol lock, negative controls, and analysis boundaries. The paper’s contribution hierarchy is: (1) naming and formalizing DFOT; (2) defining its fiveestimand metric chain; (3) introducing matched-versus-shuffled evidence as an instance-specificity intervention; and (4) validating the framework through a PPG–ECG case study. ECG-to-PPG distillation is the focused proof-of-concept baseline used in that fourth step, not the general definition of DFOT. For auditability, the six result families below are listed in experimental chronology rather than order of conceptual importance: 1. Student validation: arm and threshold selection, including the 17-arm ablation and exact original-recipe reproduction. 2. Locked upstream test: a one-shot evaluation of frozen B2, K2, and shuffled-ECG checkpoints. No downstream LLM calls had been made on the test split at this stage. 3. Controlled DFOT validation (E0): characterization, mitigation, evidence-package specificity, and interface verification-harm endpoints under Qwen3-8B/P1. 4. Cross-LLM robustness (E1): the frozen comparisons under Qwen3-8B, DeepSeek-V4-Pro, and GPT-5.5. 5. Prompt robustness (E2): Qwen3-8B under three semantically equivalent prompt formulations. 6. Downstream test replication (E3): a separately protocol-locked Qwen3-8B/P1 evaluation on the previously downstream-unqueried patient-disjoint test partition.

B

Data, Splits, and Case Construction

Patient-disjoint cohort The source is the MIMIC-III Matched Waveform Database v1.0. A channel-level scan identified 1,297 candidate patients with simultaneous PLETH and Lead-II recordings. Requiring a complete 240-s paired interval, at most 20% missing samples in either raw channel, and valid ECG and PPG beat-interval extraction yielded the frozen 50,000-record cohort from 1,275 patients. Missing samples in retained intervals were linearly interpolated; the frozen 50k table and patient manifest contain no missing identifiers or required endpoint fields. PPG is resampled to 50 Hz and ECG to 125 Hz. No MIMIC clinical table enters the frozen waveform builder, so cohort age, sex, and race summaries are unavailable. All splitting is performed at patient level. Split

Records Patients Role

Train Validation Locked test Total

36,115 6,464 7,421 50,000

925 Training 163 Selection 187 Confirmation 1,275 Full cohort

Table 2: Frozen cohort partitions after the 1,297-patient channel-level source screen. Patient overlap is zero for every pair of splits. The primary reliability label is defined only on clear ECG/PPG rhythm relations. Rows outside the four clear cases remain in the protocol data but are excluded from the clear-label reliability endpoints. Case

PPG

ECG ref. Reliability

Both regular Regular Regular Hidden irreg. Regular Irregular Both irregular Irregular Irregular False alarm Irregular Regular

HIGH LOW HIGH LOW

Table 3: Clear rhythm cases. Reliability denotes agreement with the ECG-referenced rhythm, not clinical normality. 12

On the locked test, 3,733 rows satisfy the clear-case definition: 897 LOW and 2,836 HIGH. The low-FPR contrast specifically uses 401 hidden-irregularity and 2,151 both-regular rows. The distinction matters because a model can rank the full LOW/HIGH task well while failing to isolate hidden irregularity at stringent regular-case FPR.

C

Hierarchical Teacher and Student

ECG teacher The teacher receives 30,000 ECG samples and partitions them into eight contiguous 30-s windows. Each window is represented by overlapping 0–25 s and 5–30 s views. A shared encoder processes both views; gated fusion produces one local representation. A two-layer temporal Transformer aggregates the eight representations into a global state. The outputs are: • eight 256-dimensional window embeddings and one global embedding; • window and global rhythm logits; • heart rate, RMSSD, and SDNN estimates. The teacher is task-specific and trained from scratch. A public pretraining initialization did not improve this task and was not used for the primary teacher. Three-seed mean global rhythm AUROC is 0.9912 ± 0.0007, HR MAE is 1.241 ± 0.004 bpm, RMSSD correlation is 0.9868 ± 0.0011, and SDNN correlation is 0.9831 ± 0.0014. These values qualify the teacher as a privileged reference model; they do not demonstrate transfer.

Exact original-recipe validation Arm B2 K2-global-shuffled K2 matched

AUROC AUPRC Brier ECE15 TPR@1% TPR@2% TPR@5% .8926 .8909 .9018

.7466 .1018 .7509 .1000 .7642 .0984

.0689 .0710 .0577

19.05% 17.53% 17.86%

25.87% 22.62% 23.81%

36.80% 37.34% 36.15%

Table 4: Exact original-recipe validation. K2 improves global ranking and calibration but not hidden-irregularity sensitivity in the extreme low-FPR region.

PPG student and objectives The PPG input has 12,000 samples. Patch size 100 yields 120 tokens from the 18-block SIGMA-PPG encoder. Eight groups of 15 tokens are attention-pooled to align with teacher windows. A two-layer, 256-dimensional, eight-head temporal Transformer produces window and global states. The primary output is plow . For a clear-case reliability target y ∈ {0, 1}, the base objective is weighted BCE: Lrel = BCEWithLogits(r, y; w+ = NHIGH /NLOW ).

(6)

For three-dimensional teacher and student rhythm logits zT , zS ∈ R3 and T = 2, LKD = T 2 KL (softmax(zT /T )∥softmax(zS /T )) .

(7)

Window KD is averaged across the eight windows; global KD is computed once per record. The scalar reliability logit r is produced by a separate head and is supervised by Lrel rather than by the logit losses. The K2 objective is Lrel + 0.15LwinKD + 0.15LglobalKD . The audit confirms output shapes [N, 8, 3] and [N, 3] and that teacher/student indices are aligned for KL matching, but it does not recover the three class names/index order, the exact teacher-label generation rule, or whether the window/global classifier parameters are shared. A common permutation of aligned teacher and student indices leaves the KL objective unchanged; this mathematical invariance does not recover class-specific clinical semantics or complete source-level reproducibility. Pending recovery of the training sources, we therefore interpret K2 as a frozen aligned three-way auxiliary-logit recipe and make no index-specific mechanism claim. 13

The temperature T = 2 and the equal window/global coefficients 0.15/0.15 were chosen from a limited candidate set during validation-stage development and frozen before locked-test access. The locked test was not used for hyperparameter selection. Because the search was limited rather than exhaustive, the reported findings establish the behavior of this frozen recipe but do not establish hyperparameter optimality or broad insensitivity.

D

Proof-of-Concept Student Ablations

Training schedule The exact original recipe uses maximum 15 epochs, patience 4, batch size 32, weight decay 0.01, and gradient clipping at 1.0. The head and backbone learning rates are 10−4 and 10−5 . Epochs 1–2 train the head, epochs 3–5 partially unfreeze the final six backbone blocks, and the remaining epochs permit full adaptation. Seeds are 42, 123, and 2026. The locked test uses nine frozen checkpoints: three seeds for B2, K2, and K2-global-shuffled. Arm

Backbone

Reliability supervision

Privileged target

B1 B2 K1 K2 K3 H1 H2 K5 K4 S4

Frozen SIGMA Fine-tuned SIGMA Fine-tuned SIGMA Fine-tuned SIGMA Fine-tuned SIGMA Fine-tuned SIGMA Fine-tuned SIGMA Fine-tuned SIGMA Fine-tuned SIGMA Scratch

BCE BCE BCE BCE BCE BCE BCE BCE BCE BCE

None None HR/RMSSD/SDNN Window + global rhythm logits Window + global embeddings Global rhythm logits Window rhythm logits Within-case shuffled full target Logits + embeddings + metrics Full hierarchical target

AUROC

AUPRC

.8887 ± .0079 .8927 ± .0012 .8949 ± .0032 .9042 ± .0046 .8960 ± .0033 .8995 ± .0008 .9015 ± .0033 .9003 ± .0053 .9018 ± .0031 .8755 ± .0069

– .7512 ± .0041 – .7639 ± .0087 – – – – .7522 ± .0177 .7190 ± .0110

Table 5: Historical 17-arm validation ablation (principal arms shown). K2 is the selected student. “Full” supervision is not uniformly better, consistent with optimization conflict among heterogeneous teacher targets. The ablation supports three limited interpretations. First, SIGMA initialization matters because K4 exceeds its scratch counterpart S4. Second, both temporal levels contribute descriptively: K2 exceeds H1 and H2. Third, the aligned-logit recipe is more effective than explicit metrics or latent imitation in this development screen. K5 preserves rhythm case while shuffling full targets within case and remains close to K2 (.9003 versus .9042 AUROC). Thus the ablation cannot attribute the full K2 gain to instance-matched privileged semantics; category-level structure, auxiliarytask regularization, or optimization effects plausibly contribute. The locked K2-versus-global-shuffled comparison tests whether an additional instance-correspondence component remains, but it does not decompose these mechanisms. K2–B2 validation AUROC and AUPRC differences are +0.0093 and +0.0176. K2–shuffled AUROC is +0.0110. At threshold 0.65, K2–B2 hidden LOW is −2.49 points (95% CI −9.63–5.34), and operational H-mean is −3.13 points (−12.00–6.36). The frozen validation decision is therefore ranking-only.

E

Locked-Test Protocol and Results Seed

B2 AUROC

42 123 2026

.90171 .90090 .91162

K2 AUROC

K2–B2

.93061 +.02890 .92600 +.02510 .93262 +.02100

Table 6: Frozen seed-level AUROC on the patient-disjoint locked Student test. The K2–B2 difference is positive for every seed. The protocol fixed data manifests, thresholds, model identifiers, checkpoint hashes, validation prediction hashes, comparison signs, and bootstrap settings before test access. The test was accessed once under lock hash e81efb2019fbbdc326aa7d34da195bec62ea25012386f01a76b178de826ed427. All nine checkpoints completed. The test was not used for retraining, checkpoint choice, threshold selection, prompt design, or downstream LLM evaluation. 14

Comparison

Metric

K2–B2 K2–ECG-shuffled K2–B2 K2–B2 K2–B2 K2–B2

AUROC AUROC standardized pAUC@5% FPR TPR@1% FPR TPR@2% FPR TPR@5% FPR

Mean difference +.0250 +.0115 +.0023 -.0008 +.0037 +.0082

95% CI

Interpretation

[.0118,.0274] Confirmed global ranking [.0095,.0455] Confirmed ECG specificity [-.0409,.0446] Not confirmed [-.1763,.1620] Not confirmed [-.1642,.1359] Not confirmed [-.0801,.0957] Not confirmed

Table 7: Frozen one-shot locked-test paired hierarchical-bootstrap effects. Matched K2 improves global ranking over both B2 and ECG-shuffled control; the low-FPR intervals rule out a claim that K2 improves all safety-relevant operating points.

F

DFOT Tasks and Metrics

Task separation Characterization is run before mitigation and without reliability instructions. This is necessary: a baseline prompt that already tells the LLM to verify whenever evidence is missing destroys the over-trusted subset and makes CRR undefined. D1 and D2 use distinct prompts because they test different mechanisms. Utility is a third task with separate outputs for measurement use and clinical status. The exact frozen prompt strings and deterministic parsers are versioned protocol artifacts. No separate system prompt is used; task instructions are carried by the user messages. Their semantic skeleton is: • D1 characterization: current PPG-derived rhythm + decision question; no reliability evidence. • D2 characterization: the identical current segment under neutral versus severe history; no reliability evidence. • Reactive mitigation: exact previous answer + current matched or patient-disjoint shuffled reliability statement. • Utility: current measurement + evidence; output MEASUREMENT USE and CLINICAL STATUS separately. The confirmed reliability block is reproduced below. The submission artifact includes the exact P1–P3 task templates, model request records, parser definitions, case manifests, donor assignments, and their frozen hashes.

Qwen checkpoint and inference runtime E0 and the Qwen portions of E1/E2 use a local five-shard bfloat16 Qwen3-8B checkpoint with Qwen3ForCausalLM, 36 layers, hidden size 4096, 32 attention heads, eight key-value heads, and maximum context length 40,960. It is served through the OpenAI-compatible vLLM 0.25.1 runtime. Formal requests explicitly set temperature to 0, maximum output length to 96 tokens, and thinking mode to disabled. Although the checkpoint’s bundled generation configuration contains sampling defaults, these are not the formal request settings and do not alter greedy temperature-zero decoding. The retained SHA-256 values are f7c4...7c30 for config.json, 989f...f654 for the frozen P1 prompt code, 7177...17fb for the formal configuration, and be0f...e51c for the deterministic scorer. The local checkpoint lacks a Hugging Face snapshot reference, so no exact Hub revision is claimed. The transformers version=4.51.0 field belongs to the saved model configuration and is not presented as the executing runtime version.

Reliability-evidence interface The downstream LLM receives a fixed textual rendering of the deployment-time PPG-only reliability student. The label is generated directly from the displayed score: LOW when plow ≥ 0.5 and HIGH otherwise. Scores are rounded to four decimal places, and the frozen downstream export has zero label–score mismatches. Student-validation operating thresholds are not used for this language-facing rendering. The exact evidence block is: Additional current-segment reliability evidence is now available: Current-segment rhythm reliability evidence: HIGH/LOW. Estimated probability of LOW rhythm reliability: 0.xxxx. This evidence estimates whether the PPG-derived rhythm feature is a reliable surrogate for ECG-referenced rhythm in this current segment. HIGH/LOW describes surrogate reliability only; the ECG rhythm value itself is not shown.

In an instantiated prompt, HIGH/LOW and 0.xxxx are replaced by the corresponding entries in Table 8; all other 15

wording is identical. Audit identifiers, student arm, seed, and donor identity are not rendered. The LLM receives the case text, PPG-derived heart rate, categorical PPG-derived rhythm, and this PPG-only evidence. It receives no ECG waveform, ECG rhythm value, ECG-derived diagnosis or label, teacher logit, teacher embedding, or teacher window/global representation. Case Evidence Label

plow Verdict

D1 D1 D2 D2

.9961 .0003 .1465 .3164

matched shuffled matched shuffled

LOW HIGH HIGH HIGH

SUSPECT TRUSTWORTHY TRUSTWORTHY SUSPECT

Table 8: Exact substitutions in the fixed evidence template and resulting parsed verdicts for the worked examples.

D1 matched and shuffled example. For audit case val hidden irregularity 1155 (K2, seed 123, reactive), the language-facing PPG rhythm is regular while the offline ECG reference is irregular. The no-evidence response is: VERDICT: TRUSTWORTHY ACTION: The PPG-derived rhythm assessment is reliable for routine use in this segment.

The instance-matched LOW/.9961 evidence changes the response to: VERDICT: SUSPECT ACTION: The PPG-derived rhythm assessment has low reliability and should be interpreted with caution.

Thus Ri (matched) = 1. For the patient-disjoint shuffled donor, the same case, frozen baseline, arm, seed, timing, and prompt are retained. HIGH/.0003 evidence leaves the response unchanged: VERDICT: TRUSTWORTHY ACTION: The PPG-derived rhythm assessment is reliable for routine use in this segment.

Hence Ri (shuffled) = 0 and this case contributes 1 to ESRM. D2 matched and shuffled example. For audit case val both regular 49505 (K2, seed 2026, reactive), the neutral-history no-evidence response correctly returns TRUSTWORTHY. Severe history changes the no-evidence response to: VERDICT: SUSPECT ACTION: Cross-check with clinical context and consider potential PPG artifact or undetected arrhythmia.

The case therefore enters the frozen D2 induced set. Matched HIGH/.1465 evidence produces the revised response: VERDICT: TRUSTWORTHY ACTION: Current segment rhythm assessment is reliable based on high surrogate reliability evidence.

Patient-disjoint shuffled HIGH/.3164 evidence instead leaves the response incorrect: VERDICT: SUSPECT ACTION: Consider potential undetected arrhythmia despite high rhythm reliability evidence.

Hence Ri (matched) = 1, Ri (shuffled) = 0, and the case contributes 1 to ESRM. This example shows different revisions when both labels are HIGH but displayed scores differ; it does not by itself establish the continuous score as a general causal mechanism. Utility interface.

Utility responses use separate fields, for example:

MEASUREMENT USE: USE AS IS CLINICAL STATUS: REGULAR

or 16

MEASUREMENT USE: VERIFY FIRST CLINICAL STATUS: UNDETERMINED

The same evidence block is used, but the denominator is a separate record-disjoint HIGH-reliability utility set whose baseline measurement use is USE AS IS. UHR counts only evidence-induced VERIFY FIRST; the separately parsed clinical-status field does not define utility harm. UHR is therefore an interface-specific unnecessary-verification endpoint, not a comprehensive measure of clinical utility, delayed care, escalation, or patient outcome.

Metric denominators Let C denote the reference-confirmed D1 conflict cases and N the D2 cases whose no-evidence, neutral-context decision is correct. Let bi denote the parsed downstream decision in the corresponding no-evidence challenge condition, dsevere i the parsed decision in the no-evidence severe-history condition, and di (e) the parsed downstream decision under evidence condition e. Let ui (e) denote the parsed MEASUREMENT USE field under evidence condition e, and let yi denote the parser-defined target decision (Suspect for D1 and Trustworthy for D2). Let zi ∈ {High, Low} denote the offline reference-defined reliability class, which is distinct from the deployment-time reliability estimate Ri . The baseline-eligible utility set is H = {i : zi = High ∧ ui (∅) = Use-As-Is} .

(8)

The challenge-specific frozen error sets are F1 = {i ∈ C : bi ̸= yi },

F2 = {i ∈ N : dsevere ̸= yi }. i

We use F = F1 for D1 and F = F2 for D2, and abbreviate matched and shuffled evidence by m and s, respectively. The five estimands are #{i ∈ C : baseline accepts Di } , |C| #{i ∈ N : dsevere ̸= yi } i CIR = , |N | #{i ∈ F : di (e) = yi } CRR(e) = , |F| #{i ∈ H : ui (m) = Verify-First} UHR = . |H| COTR =

(9) (10) (11) (12)

ESRM is defined as ESRM = CRR(m) − CRR(s). The randomized source substitution preserves the case, frozen baseline, student arm, seed, timing, and evidence template while replacing only the patient-disjoint evidence source. Consequently, the rendered HIGH/LOW label and numerical score may change jointly. ESRM therefore measures the benefit of the complete matched evidence package relative to the shuffled package. The K2–B2 ESRM contrast quantifies the additional evidence-package specificity associated with teacher supervision beyond the generic effect of providing reliability evidence. It does not separately identify the contributions of the categorical label, numerical score, thresholding, or template wording, and no factorial decomposition of the evidence interface is claimed. For a concrete D1 case, a no-evidence TRUSTWORTHY verdict on a reference-confirmed conflict contributes one event to COTR and places the instance in F1 . If matched evidence restores the target decision (SUSPECT) while shuffled evidence leaves the baseline decision unchanged, then the case contributes 1{di (m) = yi } − 1{di (s) = yi } = 1 to the ESRM numerator. In D2, a neutral-correct case that becomes incorrect under severe history contributes one event to CIR and enters F2 ; matched and shuffled evidence are then evaluated on that frozen induced case. Individual ESRM contributions can therefore be −1, 0, or 1, and the reported ESRM averages these contributions over the frozen error set. UHR is computed on the independent utility set H, not on either repair set. 17

Scenario Arm COTR/CIR Matched CRR Shuffled CRR ESRM

UHR

D1 D1 D1 D2 D2 D2

3.00% 3.92% 4.44% 3.00% 3.92% 4.44%

B2 K2 K4 B2 K2 K4

1.000 1.000 1.000 .942 .942 .942

.298 .341 .323 .947 .961 .943

.171 .159 .200 .794 .747 .796

.127 .182 .123 .153 .214 .147

Table 9: Aggregate formal v3.1 reactive results. COTR and CIR are challenge-set manipulation checks and therefore do not vary by student arm; they are not prevalence estimates. Arm-level rates are descriptive aggregates; the prespecified paired hierarchical-bootstrap effects are reported in Table 10.

G

DFOT Metric Closure on Controlled Validation

D1 contains 300 hidden-conflict cases from 37 patients, all over-trusted at baseline. D2 has 310 neutral-correct eligible cases; severe history induces errors in 292 cases from 89 patients, for CIR 0.942 (95% CI 0.877–0.994). In D1, K2 increases matched CRR by 5.2 points and ESRM by 4.9 points over B2, supporting partial correct-repair and matched-package-specificity gains in current-measurement conflicts. The D2 matched CRR effect is small in absolute magnitude because B2 already repairs 94.7% of the induced set. ESRM is the more diagnostic endpoint: K2 increases the matched–shuffled package gap by 6.8 points under a ceilinged repair rate. The UHR increase is 0.0087 (95% CI −0.006–0.014); its upper bound meets the prespecified exploratory 0.02 interface-engineering limit. This does not establish zero harm, broader utility, or clinical non-inferiority. Prespecified endpoint

Mean

95% CI Gate

D1 K2–B2 CRR D1 K2–B2 ESRM D2 K2–B2 CRR D2 K2–B2 ESRM K2–B2 UHR

+.052 [.013,.065] Pass +.049 [.011,.089] Pass +.013 [.007,.038] Pass +.068 [.011,.119] Pass +.0087 [-.006,.014] Pass∗

Table 10: Paired patient/seed hierarchical bootstrap. ∗ Upper CI below the prespecified exploratory 0.02 interfaceengineering limit, not a clinical non-inferiority margin.

H

E1: Cross-LLM Robustness

E1 reuses the frozen patients, cases, student seeds, matched/shuffled assignments, parser, and endpoints under P1 while changing only the LLM family. The three LLMs form a fixed robustness panel rather than a random sample from a population of models. The E1 claim is limited to the three tested LLMs. It supports cross-family robustness of the controlled effect, not invariance across all present or future LLMs.

I

E2: Prompt Robustness

E2 holds Qwen3-8B, medical facts, evidence content, cases, target task, and parser fixed. P1 is the formal template; P2 reorganizes the same content as a clinical narrative; P3 explicitly separates measurement reliability from current clinical status. Only wording and organization change. P3 has the largest descriptive ESRM values, but E2 does not prespecify or test P3 superiority over P1/P2. The supported claim is persistence across semantically equivalent formulations, not universal prompt invariance. 18

Endpoint

Qwen3-8B

DeepSeek-V4-Pro

GPT-5.5

1.000 .942 .341 .182 +.049 .961 .214 +.068 +.87 pp 0

.940 .829 .372 .151 +.071 .929 .276 +.045 +1.33 pp 0

1.000 .931 .390 .248 +.060 .951 .314 +.112 +.97 pp 0

D1 COTR D2 CIR D1 K2 matched CRR D1 K2 ESRM D1 K2–B2 ESRM D2 K2 matched CRR D2 K2 ESRM D2 K2–B2 ESRM K2–B2 UHR Parse failures

Table 11: E1 descriptive cross-LLM results. DFOT remains inducible and K2–B2 ESRM remains positive in both scenarios for every tested model.

Endpoint

Qwen3-8B ∆ [95% CI] DeepSeek-V4-Pro ∆ [95% CI] GPT-5.5 ∆ [95% CI] Equal-model pooled ∆ [95% CI]

D1 matched CRR D1 ESRM D2 matched CRR D2 ESRM

+.052 [.013,.065] +.049 [.011,.089] +.013 [.007,.038] +.068 [.011,.119] Model Qwen3-8B DeepSeek-V4-Pro GPT-5.5

+.064 [.021,.104] +.071 [.018,.126] +.021 [.004,.046] +.045 [.009,.087]

+.072 [.028,.113] +.060 [.016,.108] +.028 [.010,.051] +.112 [.054,.171]

K2–B2 UHR

95% CI Gate

+.063 [.033,.087] +.060 [.030,.091] +.021 [.010,.036] +.075 [.041,.111]

+.87 pp [-.60,+1.40] pp Pass +1.33 pp [+.20,+1.85] pp Pass +.97 pp [-.10,+1.62] pp Pass

Table 12: E1 paired hierarchical-bootstrap inference. Pooled efficacy effects are equal-weight averages over the three LLMs within synchronized patient/seed bootstrap replicates. Every efficacy CI lower bound exceeds zero, and every UHR CI upper bound is below the 2-pp tolerance.

Prompt Frozen formulation constraint P1 P2 P3

Current formal template Clinically narrative expression of identical facts and task Explicit separation of measurement reliability and current clinical status

Table 13: E2 prompt formulations. Medical content and target task are invariant.

Endpoint

P1

P2

P3

D1 COTR 1.000 .930 .880 D2 CIR .942 .860 .790 D1 K2 matched CRR .341 .365 .395 D1 K2 ESRM .182 .189 .211 D1 K2–B2 ESRM +.049 +.067 +.076 D2 K2 matched CRR .961 .949 .970 D2 K2 ESRM .214 .217 .254 D2 K2–B2 ESRM +.068 +.070 +.074 K2–B2 UHR +.87 pp +.95 pp +.95 pp Parse failures 0 0 0 Table 14: E2 prompt-specific results. DFOT remains above the prespecified inducibility thresholds, and no efficacy endpoint reverses direction.

19

Pooled endpoint

∆

95% CI Gate

D1 matched CRR +.059 [.025,.087] Pass D1 ESRM +.064 [.029,.097] Pass D2 matched CRR +.024 [.008,.040] Pass D2 ESRM +.071 [.035,.108] Pass K2–B2 UHR +.93 pp [-.15,+1.58] pp Pass Table 15: E2 equal-prompt pooled paired effects. All efficacy CI lower bounds exceed zero; the UHR CI upper bound is below 2 pp.

J

E3: Protocol-Locked Downstream Test Replication

Scope and freeze E3 evaluates downstream transport to the MIMIC test partition, which is patient-disjoint from training and validation. The partition’s upstream Student predictions had already been evaluated, but no downstream LLM response from these patients had been accessed. Before inference, case-selection seed 20260717 fixed the target records, donor rules, parser, bootstrap, four efficacy gates, and one exploratory interface criterion. The protocol lock records SHA-256 c1b1...d735; the combined-case manifest is b629...bd77. E3 uses the same Qwen3-8B/P1 interface, B2/K2 checkpoints, three Student seeds, reactive timing, and 0.5 evidence threshold as the controlled analysis. Task

Selected records Selected patients Frozen analysis set

D1 hidden irregularity D2 both regular Utility HIGH

300 300 300

44 99 111

300 / 44 patients 297 / 97 patients 150 / 76 patients

Record-disjoint union

900

158

—

Table 16: E3 target sets. Tasks are record-disjoint but can contain different records from the same patient. D1/D2 frozen sets are baseline errors; the utility set is the baseline-eligible UHR denominator. All 300 D1 records are over-trusted, so COTR is 1.000. Of 300 selected D2 records, 297 are correct under neutral history; severe history induces an error in all 297, so the prespecified denominator gives CIR 297/297 = 1.000. The three neutral-history errors are excluded before F2 is frozen. The 300-record utility cohort contains 150 both-regular and 150 both-irregular HIGH-reliability cases. The prespecified baseline-eligibility rule freezes H after the no-evidence response and before either arm’s evidence is scored. All 150 both-regular cases return USE AS IS and form H; all 150 both-irregular cases return VERIFY FIRST and cannot exhibit the defined transition. E3 UHR is therefore a narrow both-regular endpoint.

Evidence donors and inference The clear-case evidence pool contains 3,733 records from 180 test patients. Shuffled evidence always uses a different patient and record. Within each Student seed, donor identity is exactly paired between B2 and K2, while each arm renders the corresponding arm- and seed-specific score and label. Donor maps are independently frozen across Student seeds. Mitigation comprises 3,582 shuffled assignments involving 1,476 distinct donor records; utility comprises 1,800 assignments involving 843 distinct donor records. Across both stages, 1,985 distinct donors are used. Each of 5,000 replicates resamples patients with replacement and then draws three Student seeds with replacement. The same seed draws are shared across arms, matched/shuffled evidence, scenarios, and utility. This preserves paired contrasts while propagating patient and experimental-replicate variation; because donor maps are seed-specific, the seed replicate jointly indexes checkpoint and frozen donor allocation. E3 closes the prespecified internal holdout question. The challenge manipulations again produce frozen errors, and K2 reproduces the validation-stage partial repair and matched-versus-shuffled package-specificity advantages on patient-disjoint test records: all four efficacy intervals exclude zero. D1 matched CRR rises from .4011 to .4483, so most D1 errors remain unrepaired; D2 matched CRR is ceilinged, while ESRM rises from .1582 to .2251. The UHR upper bound meets the exploratory interface limit only on the baseline-eligible both-regular subset. These results support 20

Scenario

Arm

Matched CRR / UHR

Shuffled CRR

ESRM

D1 D1 D2 D2 Utility Utility

B2 K2 B2 K2 B2 K2

.4011 .4483 .9439 .9621 .0289 .0356

.1844 .1866 .7856 .7370 — —

.2167 .2617 .1582 .2251 — —

Table 17: E3 pooled absolute rates, averaging the three prespecified Student seeds. Utility entries are UHR on the 150-record baseline-eligible both-regular set. D1 matched K2 CRR of .4483 leaves .5517 of frozen D1 errors unrepaired. Prespecified K2–B2 endpoint Estimate D1 matched CRR D1 ESRM D2 matched CRR D2 ESRM UHR

+.0472 +.0450 +.0182 +.0669 +.0067

95% CI

Criterion Status

[+.0167, +.0728] Lower > 0 [+.0098, +.0818] Lower > 0 [+.0151, +.0291] Lower > 0 [+.0260, +.0719] Lower > 0 [−.0040, +.0170] Upper < .020

Pass Pass Pass Pass Pass

Table 18: E3 paired patient-cluster and Student-seed bootstrap. All four efficacy CI lower bounds exceed zero. The UHR CI upper bound is below the exploratory 0.02 interface-engineering limit, which is not clinically validated. K2–B2 effect (percentage points) D1 matched CRR D1 ESRM D2 matched CRR D2 ESRM UHR

Seed 42

Seed 123

Seed 2026

Three-seed mean

+5.33 +4.67 +0.34 +6.06 +0.67

+4.00 +6.00 +2.02 +6.67 +0.67

+4.83 +2.83 +3.10 +7.34 +0.67

+4.72 +4.50 +1.82 +6.69 +0.67

Table 19: E3 seedwise effects. Every efficacy effect remains positive across all three Student seeds.

internal replication of conditional behavioral effects under the frozen MIMIC protocol, not natural-distribution net benefit, external transport, comprehensive safety, or clinical outcome improvement.

K

Exploratory Clinician Evaluation of a Continuous Heart-Rate DFOT Extension

Motivation and task construction. The primary study evaluates DFOT using categorical PPG-derived rhythm measurements through the D1/D2 challenges defined in the main paper. As a separate exploratory analysis, we investigated whether the same framework could be instantiated for a continuous derived measurement, namely PPG-derived heart rate. The categorical-rhythm benchmark and continuous heart-rate extension used separate, record-disjoint subsets of the source cohort. The HR extension is not part of the primary D1/D2 protocol and does not contribute to its prespecified efficacy or utility gates. Within the HR extension, the 25 underlying cases were partitioned into two mutually exclusive subsets. The first comprised 12 hidden-tachycardia challenge cases, in which the displayed PPG-derived heart rate was non-tachycardic while the review-only ECG reference indicated tachycardia. An appropriate reliability-aware report should avoid confidently accepting the displayed PPG estimate and recommend verification. The second comprised 13 reliablenormal control cases, in which the PPG-derived and ECG-referenced heart rates agreed and unnecessary verification should therefore be avoided. Each case generated two reports: one without explicit reliability evidence and one with the language-facing HR reliability interface, yielding 50 reports in total. The no-evidence report was generated without an explicit reliability interface, whereas the evidence-guided report received the categorical HR-reliability label and its associated PPG-only reliability scores. The ECG reference was available only to the evaluator and was never provided to the deployment-time report generator. 21

A

B

Clinical quality under paired evidence conditions

Clinician–GPT-5.5 rating concordance

One independent clinician; thin lines connect the same case

Independent ratings of all 50 reports

Hidden tachycardia (n = 12 pairs)

Reliable-normal control (n = 13 pairs)

5 4

4.77 3.62

Clinician rating

Clinician rating (1--5)

4.92

3 2.33

2

5

4

21

4

2

4

4

5

3 2

2

3

6

6

2

2

3

1

1 No evidence

Evidence-guided

No evidence

Evidence-guided

1

GPT-5.5 rating Mean paired Δ = +2.58 [2.33, 2.83]

Mean paired Δ = +1.15 [0.62, 1.77]

Exact 70% | within ±1 100% | QWK = 0.906

Figure 3: Exploratory clinician evaluation of the continuous heart-rate DFOT extension. (A) One clinician, blinded to GPT-5.5 ratings and internal analysis labels, scored 50 reports from 25 paired cases on a prespecified 1–5 scale integrating clinical correctness, safety, and actionability. Thin lines connect reports from the same underlying case, and diamonds indicate group means. Bracketed values show 95% case-bootstrap confidence intervals for the mean paired change based on 20,000 replicates. (B) Independent clinician–GPT-5.5 rating concordance across all reports. Cells show report counts. Exact agreement was 70%, all ratings differed by at most one point, and quadratic-weighted agreement was κ = 0.906.

Independent clinician scoring. An independent clinician evaluated the 50 anonymized reports using a prespecified five-point ordinal scale that jointly considered clinical correctness, safety, and actionability. A score of 5 indicated a fully appropriate report, whereas a score of 1 indicated a clearly incorrect or potentially unsafe report. The clinician was instructed to assess whether the report appropriately interpreted the displayed PPG-derived heart rate in light of the review-only ECG reference and the accompanying reliability information without rewarding report length or stylistic preferences. The clinician was blinded to GPT-5.5 ratings and internal analysis labels. GPT-5.5 independently scored the same reports using the same rubric. Analyses were paired by the 25 underlying cases. Mean paired score changes and their 95% confidence intervals were estimated using 20,000 case-level bootstrap replicates. Clinician–GPT-5.5 concordance was summarized using exact agreement, agreement within one ordinal point, and quadratic-weighted κ. The evaluation involved a single prespecified clinician and was intended as an exploratory face-validity assessment rather than a formal multi-rater validation.

Results. For the 12 hidden-tachycardia cases, the clinician’s mean score increased from 2.33 without reliability evidence to 4.92 with evidence-guided reporting, corresponding to a mean paired increase of 2.58 points (95% CI: [2.33, 2.83]). All 12 cases received higher scores under the evidence-guided condition, indicating that the reliability interface helped the reports avoid confidently accepting an apparently non-tachycardic PPG estimate when the offline ECG reference indicated tachycardia. For the 13 reliable-normal controls, the mean score increased from 3.62 to 4.77, with a mean paired increase of 1.15 points (95% CI: [0.62, 1.77]).Eight cases improved and five were unchanged; none received a lower score. These findings suggest that the higher ratings were not achieved by indiscriminately recommending ECG verification when the displayed measurement was already reliable. Clinician and GPT-5.5 ratings agreed exactly for 35 of 50 reports (70%), and all 50 ratings differed by no more than one ordinal point. Quadratic-weighted agreement was κ = 0.906. Together, these findings provide exploratory face-validity evidence that the DFOT evaluation framework can be instantiated for a continuous derived measurement in addition to categorical rhythm. However, because the evaluation involved a single clinician, a small number of cases, and a composite ordinal score, it should not be interpreted as multi-clinician consensus, probability calibration, prospective clinical validation, or evidence of improved patient outcomes. 22

L

Tail Mechanism Audit

Global ranking metrics do not indicate where improvements occur along the ROC curve and may therefore overlook behavior in the stringent low-false-positive operating region. We therefore explored a tail-aware objective on the development set under a prespecified stop rule. To determine whether any apparent gain depended on label-specific privileged information rather than generic optimization effects, we designed a negative control that preserved the number of positive examples and all training mechanics while disrupting teacher–student label correspondence. The final patient-deranged shuffled-tail control uses patient-disjoint donors, preserves all 346 positive assignments, and retains the true hidden label for only 10.69% of reassigned examples. Comparison

∆ tail mean

True-tail – K2 anchor True-tail – hidden-weighted BCE True-tail – patient-deranged shuffle K2 true-tail – B2 true-tail

95% CI ∆ standardized pAUC

+3.06 pp [1.28,5.56] +2.59 pp [.85,4.53] +1.31 pp [-1.70,4.50] +2.30 pp [-1.06,6.51]

95% CI

+.0138 [.0067,.0240] +.0120 [.0038,.0212] +.0057 [-.0105,.0212] +.0108 [-.0059,.0295]

Table 20: Development-only tail confirmation. The effective shuffled-tail comparison crosses zero, so label specificity is not demonstrated. The frozen decision is STOP TAIL GAIN NOT LABEL SPECIFIC: no full-data tail escalation, no test-tail access, and no additional tuning of margin, weight, or hard-negative ratio. This means “label specificity was not demonstrated,” not “the objective has no effect.”

M

Protocol Design Lessons and Corrective Actions

No-evidence leakage An early baseline prompt explicitly instructed the LLM to verify when reliability evidence was absent. The model consequently avoided DFOT before characterization, producing no induced subset. Although the run completed technically, CRR and ESRM were undefined and the responses were excluded.

Clinical-status/reliability collision A second protocol used one TRUSTWORTHY/SUSPECT output for both measurement reliability and rhythm status. The model interpreted SUSPECT clinically, while the scorer interpreted it as unreliability. The gate observed no valid D1/D2 characterization and stopped the evidence stage before tens of thousands of invalid calls.

Resulting rules These failures produced six requirements: characterize before mitigating; use mechanism-specific D1/D2 prompts; separate measurement use from clinical status; randomize only the donor source while holding the case and evidence template fixed (the resulting label and score may change jointly); reuse the exact baseline answer in reactive mitigation; and freeze prompt, parser, cases, thresholds, and statistics together.

N

Claim Registry

Table 21 summarizes the E0–E3 frozen claim registry and the corresponding scope of interpretation.

O

Reproducibility Checklist for Authors

E0–E3 freeze the locked Student results, validation D1/D2 effects, cross-LLM and cross-prompt robustness endpoints, the patient-disjoint downstream test replication, the exploratory UHR interface criterion, and the claim registry. The E3 protocol records the case and donor manifests, synchronized bootstrap design, exact Qwen settings, deterministic parser, and input/output hashes. The executable release should include preprocessing and patient-split builders, case/donor manifests or an authorized-user reconstruction route under the MIMIC data-use agreement, full prompt/parser artifacts, and checkpoint/configuration hashes. The following items remain unavailable or require author verification: 23

Claim

Status

Supported conclusion

Do not conclude

DFOT formulation

Introduced

Metric chain

Introduced

Names a downstream failure caused by Equivalent to hallucination or upinappropriate reliance on derived mea- stream misclassification. surements. Five estimands separate induction, re- A single metric fully characterizes pair, specificity, and interface harm. DFOT. Tests whether downstream revision de- Identifies the separate effects of label, pends on instance-specific evidence. score, or wording.

Framework

Matched–shuffled interven- Validated tion Upstream evidence generation Upstream transfer

Confirmed

Matched-target specificity Confirmed

Extreme low-FPR transfer Not confirmed

K2 improves PPG-only reliability rank- PPG reconstructs ECG or ECG is ing over B2. ground truth. Supports an instance-correspondence The entire gain is explained by recovcomponent beyond generic regulariza- ered ECG semantics. tion. Global ranking and low-FPR behavior All safety operating points improve. remain distinct.

Downstream DFOT evaluation D1 repair

Confirmed

D1 specificity

Confirmed

D2 repair

Confirmed

D2 specificity

Confirmed

Interface harm (UHR)

Criterion met

K2 improves matched repair for currentmeasurement conflicts. K2 improves D1 ESRM for the matched evidence package. K2 improves matched repair under misleading history. K2 improves D2 ESRM despite nearceiling CRR. The exploratory engineering limit is satisfied on the baseline-eligible utility set.

Clinical-error reduction or external transport. ESRM isolates label, score, or wording effects. Real-world longitudinal robustness. ESRM explains all context conflicts or internal LLM mechanisms. Clinical non-inferiority, absence of harm, or validated utility.

Robustness Cross-LLM robustness

Supported

Prompt robustness

Supported

E3 replication

Confirmed

Positive effects persist across the three Universal robustness across LLM famevaluated LLMs. ilies. Positive effects persist across P1–P3. Invariance to arbitrary prompts or model–prompt interactions. All four efficacy gates replicate on held- External-dataset or prospective transout MIMIC patients. port.

Clinical interpretation Clinical outcome

Not tested

Results concern controlled downstream Clinical safety or improved patient outbehavior. comes.

Table 21: Frozen claim registry summarizing the scope of the supported conclusions. • institutional ethics approval, consent/waiver, and data-use permissions; • device/vendor metadata and formal-cohort age, sex, and race statistics, which are absent from the frozen waveform artifacts; • the three teacher-output class names/index order, their label-generation rule, teacher/student target correspondence, and whether window/global classifier parameters are shared; • a full Qwen checkpoint revision or complete shard hashes, the exact DeepSeek/GPT service revisions, and non-vLLM runtime packages; • ECG-teacher wall-clock time and complete token/cost ledger; 24

• final data/code release level and a MIMIC-compliant reproducible route for authorized reviewers.

25