arXiv CSopen access

Uncertainty Quantification for Flow-Based Vision-Language-Action Models

arXiv:2606.18043v1 [cs.RO] 16 Jun 2026

Uncertainty Quantification for Flow-Based Vision-Language-Action Models

Ralf Römer TU Munich

Maximilian Seeliger ETH Zurich

Saida Liu TU of Munich

Ben Sturgis TU Munich

Marco Bagatella ETH Zurich MPI IS Tübingen

Daniel Marta ETH Zurich

Andreas Krause ETH Zurich

Angela P. Schoellig TU Munich

Abstract Vision-language-action models (VLAs) combine vision-language backbones with expressive generative action heads trained via flow matching on large-scale robotic datasets. Despite their strong empirical performance in robotic manipulation, VLAs lack mechanisms to quantify confidence in their predictions and to detect when their actions may be unreliable. This presents a critical limitation for real-world deployment in non-stationary environments, where models inevitably encounter scenarios outside their pretraining distribution and may fail without warning. To address this, we derive an efficient method for quantifying epistemic uncertainty in flow-matching models by leveraging velocity-field disagreement (VFD) across a small ensemble. We successfully use this uncertainty estimate for failure detection during deployment and active fine-tuning of flow-based VLAs. To this end, we propose SAVE, a framework for uncertainty-guided active multitask fine-tuning that reduces the number of costly expert demonstrations required to adapt VLAs to new tasks. Through extensive experiments on the LIBERO benchmark, we demonstrate that VFD yields better-calibrated uncertainty estimates predictive of downstream performance, that VFD achieves strong performance in detecting failures, and that uncertainty-guided data acquisition with SAVE requires at least 22 % fewer samples than baselines. In summary, our work shows that quantifying epistemic uncertainty in flow-based VLAs improves both failure awareness and adaptation. Project website: tum-lsy.github.io/uq_vla/.

1

Introduction

The premise of vision-language-action models (VLAs) [67, 39, 54] is to bring the large-scale pretraining paradigm that has been successful in deep learning in recent years [17, 20, 6] into the physical world. Driven by the increasing availability of large-scale robotics datasets [38, 53], modern VLAs have demonstrated impressive performance in multitask learning, exhibiting remarkable robustness and emergent zero-shot capabilities [10], particularly for tasks that require a nuanced understanding of images and language. Architecturally, the state of the art has mostly converged on a powerful recipe [54, 64, 57, 10]: a large pre-trained vision-language backbone for semantic understanding, followed by an action expert trained via flow matching [45, 22] to produce smooth, continuous robot actions. Flow-based VLAs provide stable training, relatively fast inference, and the expressivity needed to model the complex, highly multimodal distributions of real-world physical demonstrations. Preprint.

Epistemic Uncertainty Estimation for VLAs Train Ensemble Use diverse dataset randomly shuffled.

Velocity Field Disagreement (VFD) ,

Compute the scaled differences between the velocities along ODE paths to estimate epistemic (model) uncertainty.

Flow-based VLA VLA ensemble

Uncertainty-Guided Active Multitask Fine-Tuning New Tasks “Put both the alphabet soup and the tomato sauce in the basket.“

Estimate Uncertainty

Selection

Query Expert

Fine-Tuning

For all candidate tasks, compute VFD uncertainties.

Prioritize most uncertain tasks and select top-K episodes.

Collect episodes starting from selected initial states.

Replay to ensure ensemble diversity.

Task weights

“Put the black bowl in the bottom drawer of the cabinet and close it.“

...

“Put the yellow and white mug in the microwave and close it.“

Low

High

Expert Demonstrations

Selected initial states Repeat for

rounds

Figure 1: Top: VFD quantifies epistemic uncertainty by measuring scaled differences between ensembled velocity fields. Bottom: SAVE prioritizes tasks by their mean VFD uncertainty and, for the most uncertain initial observations within each sampled task, requests an expert demonstration. The models are then fine-tuned using new and replay data, yielding data-efficient multitask adaptation.

VLAs are pre-trained on large-scale static datasets that cover a broad distribution of objects and tasks [38, 53, 3]. However, deploying these models in the real world inevitably exposes them to non-stationarity. Users repurpose robots for novel tasks, object appearances may change, and environments evolve. In such scenarios, pre-trained VLAs may perform well on some tasks but fail completely on others, without explicit knowledge of which tasks are fully understood from pre-training and which require additional data. The inability of these models to communicate what they do not know prevents robust self-improvement and timely failure detection. VLAs confidently execute erratic actions in out-of-distribution scenarios [2, 27, 58] rather than abstaining or asking for help when uncertain [55], raising significant safety concerns [43, 27, 59]. Crucially, robustly adapting VLAs to new domains currently requires collecting large numbers of human expert demonstrations [52], which is prohibitively expensive. To address these problems, our work focuses on uncertainty quantification for flow-based VLAs. Leveraging the generative architecture of these models, we propose an efficient method for estimating epistemic uncertainty by measuring divergence between ensembled velocity fields. Building on this estimator, we propose sample-efficient active fine-tuning via velocity-field epistemic uncertainty (SAVE) for VLAs. In summary, our main contributions are: • We introduce a mathematically grounded epistemic uncertainty estimator for flow-matching models based on velocity field disagreement (VFD). • We propose SAVE, a framework for uncertainty-guided active fine-tuning of flow-based VLAs that uses VFD to prioritize tasks and initial states for expert demonstration collection. • We empirically evaluate the effectiveness of our methods, demonstrating that VFD predicts task performance and detects deployment failures more reliably than baselines, and that SAVE reduces the amount of expert data required for multitask adaptation by 22 %.

2

Related Work

Vision-Language-Action Models. VLAs have emerged as a prevalent approach for learning manipulation policies, by fine-tuning pre-trained vision-language backbones on robotic data [39, 67, 54, 64]. 2

Earlier VLAs map vision-language observations to discrete or continuous action tokens via regression or classification heads [39, 67]. In this work, we focus on recent flow matching-based VLAs [54, 64], which provide an expressive framework for modeling the highly multimodal action distributions in expert demonstrations by treating behavior cloning as a conditional generation problem [54, 64]. However, flow-based VLAs do not natively express uncertainty. Therefore, they cannot inherently identify situations in which they lack knowledge of which action to perform [2, 42], preventing safe, autonomous real-world deployment. Fine-tuning a VLA requires collecting many expensive expert demonstrations [52]. This motivates estimating where the model is most uncertain: before deployment, to prioritize situations for querying an expert, and during deployment, to detect failures. Uncertainty Quantification for Generative Models. Uncertainty quantification in deep learning aims to estimate the trustworthiness of model outputs [1] and typically decomposes uncertainty into epistemic (model) and aleatoric (data) uncertainty [32]. While the latter is an irreducible property of the data generation process, epistemic uncertainty can be reduced by collecting new and, importantly, the right data. Depending on the model class, uncertainty quantification has been instantiated in many different ways. Test-time dropout [47] and inference on augmented inputs [7] may, for instance, be applied directly at test time and are mostly architecture-agnostic. More powerful approaches, such as deep ensembles [41] and Laplace approximations [16], require training multiple models or modifying the training procedure, in exchange for more accurate uncertainty estimates. With the rise of modern generative models [30, 45], which, despite their expressivity, often remain overconfident [51, 36], several recent works have investigated uncertainty quantification for this model class. Jazbec et al. [33] apply a last-layer Laplace approximation to diffusion models and measure variability in a semantic representation space using a Gaussian approximation, which may, however, be unsuitable for multi-modal distributions. DECU [9] trains an ensemble of latent diffusion models and performs pairwise distance estimation using the denoising means. Other recent works rely on similarity metrics in language space [24], leverage hypernetworks [13], or train explicit confidence predictors [50]. Closer to our setting, Ju et al. [34] estimate epistemic uncertainty for pre-trained VLMs through a Riemannian formulation of flow matching, but do not target action generation or active fine-tuning of VLAs. Uncertainty quantification is also an active area of research in autoregressive models [63], with domain-specific techniques spanning from differential entropy over vocabulary [44], to perplexity [56, 23], and Dirichlet evidence [48], but these methods primarily measure aleatoric uncertainty, i.e., data ambiguity. In this work, we focus on estimating epistemic uncertainty in flow-matching models, which have become the state of the art in generative modeling. Active Learning. A direct application of reliable uncertainty estimates is in active data selection, a well-established research field with deep roots in experimental design [12] and active learning [62], including Bayesian disagreement-based acquisition for deep models [26, 40]. While a large body of literature has proposed domain-agnostic approaches [5, 31], several recent works have focused on active learning in MDPs, particularly for learning policies [35, 28]. Several existing approaches operate offline [29, 2], and filter a fixed dataset based on mutual information between states and actions or influence detection in a static setting. A long-running line of research [15, 37] operates instead in an online setting, and tries to infer when a human labeling effort is required. Closer to our approach, Bagatella et al. [8] also focus on an iterative setting but propose an algorithm that queries entire expert demonstrations to maximize information gain over multi-task expert trajectories. While this approach has strong regret guarantees, it does not scale to modern VLAs. In contrast, we directly leverage the generative formulation of state-of-the-art VLAs for uncertainty-guided active fine-tuning.

3

Preliminaries

Bayesian Uncertainty Quantification. We consider a dataset D = {(x(n) , y (n) )}N n=1 and a neural network with parameters θ ∈ Θ trained to learn the relationship between y ∈ Y and x ∈ X . Bayesian modeling considers the posterior predictive distribution Z p(x | y, D) = p(x | y, θ)p(θ | D)dθ, (1) which involves the posterior p(θ | D) that is generally intractable for deep neural networks [32]. For this reason, the integral (1) has to be approximated in practice by an MC PM 1 estimate p(x | y, D) ≈ M i=1 p(x | y, θi ), θi ∼ p(θ | D), where approaches such as ensembling [41] or dropout [25] can be used to obtain θ1 , . . . , θM ∼ p(θ | D). The total uncertainty 3

in the generation  x Rfor an input y is given by the entropy of the predictive distribution (1), H p(x | y, D) = − p(x | y, D) log p(x | y, D)dx, which includes both irreducible aleatoric uncertainty and epistemic uncertainty, which can be reduced by collecting additional data. Subtracting the aleatoric component, given by the expected entropy, yields the epistemic uncertainty [32]      I(x, θ | y, D) = H p(x | y, D) − Eθ∼p(· | D) H p(x | y, θ) = Eθ∼p(· | D) [DKL (p(x | y, θ) || p(x | y, D))] ,

(2)

which is equivalent to the mutual information between x and θ and can be viewed as the expected disagreement between a model learned from the data and the true posterior (1) at a test input y. Flow Matching. The goal of generative modeling is to learn a probability distribution q from a set of (n) samples D. We first consider unconditional generation, where D = {x(n) }N ∈ Rd . Flow n=1 ∼ q, x d d matching models a time-dependent velocity field us : [0, 1] × R → R that generates a probability path (ps )0≤s≤1 transforming a source distribution p0 (e.g., a Gaussian) to the data distribution p1 = q. d This means the flow ϕs : [0, 1] × Rd → Rd solving the ODE ds ϕs (x) = us (ϕs (x)), ϕ0 (x) = x satisfies xs = ϕs (x0 ) ∼ ps for x0 ∼ p0 . As us (x) is unknown, conditional flow matching [45] constructs ps as the marginal of simpler conditional paths ps (x | x1 ), where x1 ∼ q. A common choice are optimal transport (OT) Gaussian conditional paths ps (x | x1 ) = N (x | sx1 , (1 − s)2 I), for which the conditional velocity field us (x | x1 ) has an analytical expression. A neural network vsθ is then trained to regress onto this target vector. Given a dataset D = {(x(n) , y (n) )}N n=1 ∼ q, a common training objective for learning q(x | y) is h i 2 L(θ) = Es∼Unif([0,1]),(x1 ,y)∼Unif(D),xs ∼ps (· | x1 ) vsθ (xs , y) − us (xs | x1 ) 2 . (3) New samples x ∼ p1 (· | y) from the learned distribution can be generated, for example, through Euler integration starting from x0 ∼ p0 with step size δs = 1/Ns and xs+δs = xs + vsθ (xs , y)δs. Vision-Language-Action Models. A VLA is a robot policy πθ : O → ∆(A) that maps multimodal observations ot ∈ O to distributions over actions at ∈ A. The observations typically contain the robot’s proprioceptive state st , camera images It and language instructions l, i.e., ot = (st , It , l). Flow-based VLAs [54, 57, 64, 10] are typically trained through behavior cloning from large-scale data D = {(o(n) , a(n) )}N n=1 ∼ q using the flow-matching loss (3) with x = At = (at , at+1 , . . . , at+H ) and y = ot , generating entire action chunks to increase temporal consistency and robustness against non-Markovian expert behavior [14, 10]. Problem Statement. At its core, our work focuses on uncertainty estimation to detect failures and guide data acquisition for VLAs. More formally, we consider a set of K tasks T = {Tk }K k=1 , each of which can be described by an MDP Mk = (O, A, Pk , Rk , γ), where O and A are observation and action spaces, Pk : O × A → ∆(O), Rk : O → R and ρk ∈ ∆(O) are task-specific dynamics, rewards and initial observation distributions, and γ is a discount P∞ factor. Given a policy π : O → ∆(A), its single-task performance is simply Jk (π) = Eπ,Pk ,ρk t=0 γ t Rk (ot ), and its multitask performance PK 1 is J(π) = K k=1 Jk (π). Our goal is to fine-tune a flow-based VLA πθ (pre-trained on large-scale, diverse data) to maximize its average multitask performance. For each task Tk ∈ T , we assume the availability of a pool of L candidate initial observations Ok = {okl }L l=1 ∼ ρk , where each okl in practice contains an initial robot state, camera observations and language instructions. Without any environment interaction, a subset of observations Ô ⊂ ∪K k=1 Ok needs to be selected, and an expert demonstrator π ∗ ≈ argmaxπ J(π) will return a demonstration starting from each of the selected observations: τe = (o0 , a0 , . . . ) with at ∼ π ∗ (ot ) and ot+1 ∼ Pk (ot , at ) for each o0 ∈ Ô.

4

Epistemic Uncertainty Estimation in Flow-Matching Models

Training a flow-matching model via (3) yields an estimate θ of the model parameters given the data D. We aim to quantify whether the model, for a given conditioning y, exhibits high epistemic uncertainty that could be reduced by collecting more data. A simple estimate of total predictive uncertainty is the conditional entropy H[p(x | y, θ)] [58]. As this quantity conflates epistemic and aleatoric uncertainty, we need to isolate the epistemic component (2). We denote pθi (x | y) = p(x | y, θi ) and 4

consider the MC approximation of (2) from a set of model parameters θ1 , . . . , θM ∼ p(θ | D), i.e., M

1 X DKL pθi (x | y) || p(x | y, D) M i=1   M M X 1 X 1 ≈ DKL pθi (x | y) || pθj (x | y) M i=1 M j=1

I(x, θ | y, D) ≈

M
1 X ≤ 2 DKL pθi (x | y) || pθj (x | y) , M i,j=1

(4a)

(4b)

(4c)

where (4c) follows by applying Jensen’s inequality to each term in the outer sum, using the convexity of the KL divergence in its second argument. Hence, the average pairwise KL divergence between flow-matching models trained on the same data D, evaluated for a conditioning input y, represents an approximate upper bound of the epistemic uncertainty. In principle, the KL divergence between two flow-matching models can be estimated by computing the likelihood of individual samples by solving an augmented ODE that involves the divergence of the learned velocity field. However, computing the divergence is prohibitively expensive in high dimensions. Therefore, we derive a direct relationship between the pairwise KL divergence and the learned velocity fields. For the proof, we refer to Appendix A.2. Theorem 4.1. Consider two distributions pθ1 (x | y), pθ2 (x | y) induced by velocity fields vsθ1 (x, y), s vsθ2 (x, y) of flow-matching models with OT Gaussian conditional probability paths. Define κs = 1−s , and assume that, for all s ∈ [0, 1], the marginal probability densities and velocity fields decay sufficiently fast at infinity such that pθs 1 (x | y) vsθ1 (x, y) → 0, pθs 2 (x | y) vsθ2 (x, y) → 0 as ∥x∥ → ∞. Then, the KL divergence between the two distributions is given by Z 1 h i
2 DKL pθ1 (x | y) || pθ2 (x | y)) = κs Exs ∼pθ1 (x | y) vsθ1 (xs , y) − vsθ2 (xs , y) 2 ds. (5) s

0

Note that by definition of κs , velocity differences at higher flow-matching times s, where the samples contain less noise, are more indicative of epistemic uncertainty. Theorem 4.1 allows us to efficiently approximate the KL divergence terms in (4c) by sampling x0 ∼ p0 , performing Euler integration using the learned velocity field vsθ1 and calculating the velocity field disagreement (VFD) at intermediate ODE states x0 , xδs , . . . , i.e., " N −1 # s X
1 2 DKL pθ1 (x | y) || pθ2 (x | y)) ≈ Ex ∼p κsℓ vsθℓ1 (xsℓ , y) − vsθℓ2 (xsℓ , y) 2 , (6) Ns 0 0 ℓ=0

where sℓ = ℓδs. Finally, denoting V = (vsθ1 , . . . , vsθM ), plugging (6) into (4c) and averaging over non-identical pairs, we define the VFD score for tractable epistemic uncertainty estimation as " M N −1 # s X X 2 1 θi (i) θj (i) ue (y; V) = , Ex ∼p κsℓ vsℓ (xsℓ , y) − vsℓ (xsℓ , y) M (M − 1)Ns 0 0 i,j=1 2

(7)

ℓ=0

j̸=i (i) (i) where xsℓ +δs = xsℓ + vsθℓi

(i) (i) xsℓ , y δs, x0 = x0 . In practice, we approximate the expectation

in (7) by forward integrating a small batch of B actions, leveraging GPU parallelization.

5

Uncertainty-Guided Active Multitask Fine-Tuning

Having proposed VFD for estimating epistemic uncertainty in flow-matching models, we now build on this and present SAVE for sample-efficient active multitask adaptation of flow-based VLAs. As established in Section 3, expert rollouts are unavailable for data acquisition. Instead, we must decide which demonstrations are most valuable to collect based solely on potential initial configurations for the given tasks. For this purpose, we calculate the VFD uncertainty for candidate initial observations. Higher values indicate the policy is more uncertain about how to act in a given scene, suggesting limited exposure during pre-training and a potential benefit from additional expert demonstrations. 5

Algorithm 1 Uncertainty-Based Active Fine-Tuning of Flow-Based VLAs with SAVE. 1: Input: Base VLA πb , pool of initial configurations {Ok }K k=1 , diverse dataset Dpre . 2: Output: Final policy πR . 3: Hyperparameters: Rounds R, queries per round ne , temperature τ , replay ratio λ. 4: From πb , train an ensemble Π0 on Dpre . (≤0) 5: Initialize buffer Dnew ← ∅. 6: for r = 0 to R − 1 do ▷ Active learning loop. 7: for k = 1 to K do 8: for each okl ∈ Ok do (r) 9: Compute VFD uncertainty ukl = ue (okl ; Πr ) via (7). (r) 10: Compute task uncertainty Uk via (8). 11: Compute task weights W (r) via (9). 12: for ξ = 1 to ne do ▷ Data acquisition. 13: Sample k ∼ Cat(W (r) ). 14: Select the most uncertain initial observation o∗ from task Tk via (10). (≤r) 15: Query the expert to collect a demonstration τe starting from o∗ and add τe to Dnew . (r) 16: Construct training mixture Dtrain via (11). (≤r) (r) (≤r+1) ← Dnew . 17: Πr+1 ← Fine-tune(Πr , Dtrain ), Dnew VLA Ensemble. The high computational cost of pre-training foundation models makes it prohibitively expensive to train multiple VLAs from scratch to sample from the posterior. Instead, we take a single pre-trained base VLA πb and fine-tune it M times on randomly shuffled versions of a diverse (1) (M ) dataset Dpre to obtain a VLA ensemble Π0 = (π0 , . . . , π0 ) for uncertainty estimation. Uncertainty-Guided Data Acquisition. We train the VLA ensemble over R rounds of active finetuning. At each round r, we evaluate the current ensemble Πr on all candidate initial observations (r) and compute, for all tasks k ∈ {1, . . . , K} and candidate initial observations okl ∈ Ok , ukl = ue (okl ; Πr ), where the VFD score ue (·) measuring epistemic uncertainty is defined in (7). A simple strategy would be to always select the single most uncertain candidate observation. However, this approach can overemphasize a small set of outlier scenarios, leading to poor task coverage and detrimental effects on multitask improvement. Thus, we first aggregate uncertainty at the task level using the mean score L

(r)

Uk

=

1 X (r) ukl . L

(8)

l=1

It may then be tempting to directly request a batch of demonstrations from the most uncertain task; however, this is both theoretically flawed (as mutual information would need to be updated as soon as a single new demonstration is collected) and practically suboptimal (as an excessive number of demonstrations could be collected for an easy task). While posterior uncertainty can be easily computed under strong assumptions about the model class (e.g., analytically, without even observing x in the case of Gaussian Processes [8]), its estimation with neural networks remains, in general, intractable. As collecting demonstrations in batches remains practically necessary for computational reasons, we instead introduce diversity by defining a categorical sampling distribution Cat(W (r) ) (r) (r) with a temperature parameter τ ≥ 0 and weights W (r) = (w1 , . . . , wK ) over tasks, where  τ (r) Uk (r)  τ . wk = P (9) (r) K U ′ ′ k =1 k Larger values of τ concentrate probability mass on the most uncertain tasks, and τ = 0 corresponds to uniform sampling. We perform ne expert queries per round and first sample a task index k ∼ Cat(W (r) ), effectively prioritizing tasks with high estimated uncertainty while retaining exploration over all tasks. We then select the most uncertain initial observation within the sampled task, (r)

o∗k = arg max ukl , okl ∈Ok

6

(10)

and request an expert demonstration starting from o0 = o∗k , τe = (o0 , a0 , . . . ). Repeating this (r) procedure ne times yields a batch of newly collected demonstrations Dnew . Iterative Fine-tuning. Typically, fine-tuning only on newly collected demonstrations leads to forgetting previously acquired capabilities [19, 58]. To avoid this issue, we fine-tune the VLA ensemble on a mixture of pre-training data and queried data, (r)

(≤r) Dtrain ∼ λ Unif(Dpre ) + (1 − λ) Unif(Dnew ),

(11)

(≤r)

where Dnew denotes all demonstrations collected up to round r, and λ ∈ [0, 1] controls the replay ratio. By determining which tasks currently need data most and then selecting the highest-uncertainty initial observations within those tasks, SAVE balances exploitation of uncertain scenes with exploration across tasks for sample-efficient multitask adaptation.

6

Experiments

With our experiments, we primarily aim to answer four research questions: Q1: What is the best strategy for estimating epistemic uncertainty in flow-based VLAs? Q2: How does uncertainty calibration affect the performance of active fine-tuning with SAVE? Q3: How does uncertainty-guided data acquisition compare to optimizing for diversity alone? Q4: Are uncertainty estimates informative for detecting failures during deployment? 6.1

Experimental Setup

Environments. We conduct our experiments on the LIBERO [46] benchmark, which simulates a Franka manipulator in various household environments. For active fine-tuning, we consider K = 10 long-horizon tasks from the most challenging suite, LIBERO-10. The initial VLA ensemble Π0 is trained on 30 tasks from the LIBERO-Goal, Spatial, and Object suites, and we also include three active-learning tasks in Dpre to instill prior task competence that modern VLAs increasingly possess. Implementation Details. We use SmolVLA [64] in our experiments, which attaches an action expert trained via flow matching to a pre-trained SmolVLM-2 backbone and uses both cross- and self-attention during action generation. The policy is conditioned on two camera images (third-person and wrist), the proprioceptive state, and the task language instruction, generates action chunks of length H = 50, and replans every 25 environment steps, using an ODE step size δs = 0.1. We compute VFD with a batch size B = 5, perform R = 15 rounds of iterative active fine-tuning and query ne = 5 expert demonstrations per round, using a replay ratio of λ = 0.5. All experiments are repeated across three random seeds, and we report the mean and standard deviation across seeds. Baselines. We compare VFD against six baselines for uncertainty estimation in modern generative models. Action-L2 computes the pairwise L2 distance between independently generated action chunks from the ensemble members. ACE [58] computes the conditional entropy in the action chunk distribution. We adopt DECU [9], which performs pairwise distance estimation at a “branching” ODE timestep for denoising diffusion models (DDIM), for flow matching. Generative Uncertainty (GU) [33] generates samples from all ensemble members starting from the same initial noise to approximate the posterior predictive as a Gaussian, and calculates its entropy. Entropy [49] and Perplexity [23] operate on the VLM prefix, computing average per-token entropy and the exponentiated negative sum of the language tokens’ log-probabilities, respectively. 6.2

Calibration

Our uncertainty-guided data acquisition method relies on the epistemic uncertainty for an initial observation okl being strongly negatively correlated with the probability of success. For this reason, we evaluate calibration using Spearman’s rank correlation coefficient ρ between the mean uncertainty and the success rate as the primary calibration metric; a value of −ρ = 1 indicates that the success rate decreases monotonically with increasing uncertainty. In addition, we report the Pearson correlation coefficient, measuring the strength of the linear relationship between the two quantities. We iteratively fine-tune a VLA ensemble to evaluate calibration over multiple rounds, collecting demonstrations at random each round to eliminate the effect of the data-acquisition strategy. 7

1.0

Success Rate / Uncertainty

1.0 0.8 0.6 0.4 0.2 0.0

- Spearman

0.8 0.6 0.4

M=2 M=3 M=4

0.2 0.0

2

4

6

8 10 12 14

Round r

Figure 2: The VLA ensemble size has little impact on calibration, allowing for a lightweight two-member ensemble.

P1

P2

P3

Prompt

P4

Action-L2 ACE DECU GU Entropy Perplexity VFD (ours) Success Rate

P5

Figure 3: VFD is also well-calibrated when varying only the language prompt.

Table 1: Calibration analysis. Negative Spearman rank (↑) and negative Pearson (↑) correlation between uncertainty estimates and per-task success rates, averaged across iterative fine-tuning rounds. Metric −Spearman −Pearson

Action-L2 ±0.13

0.50 0.48±0.09

ACE ±0.12

0.31 0.36±0.08

DECU ±0.13

0.31 0.23±0.15

GU ±0.00

0.62 0.65±0.02

Entropy

Perplexity

VFD (ours)

±0.12

±0.09

0.71±0.03 0.71±0.02

0.10 0.23±0.21

−0.04 0.02±0.15

Small ensembles are sufficient. Since fine-tuning VLAs is expensive, it is highly desirable to have small ensembles. Figure 2 shows the impact of the ensemble size M on calibration for all fine-tuning rounds. The VLA ensembles consistently remain well-calibrated even for M = 2, indicating that they do not collapse to a single mode of the posterior. Due to the competitive performance and computational advantage of two-member ensembles, we adopt this strategy in all subsequent experiments. VFD excels at estimating epistemic uncertainty. The overall calibration across all rounds is summarized in Table 1. Across all tasks and rounds, VFD is better calibrated than the baselines, achieving a 0.09 higher negative Spearman correlation than the second-best method, GU. These results establish VFD as a powerful uncertainty estimator to guide data acquisition in active multitask fine-tuning. VFD attends to different input modalities. Since the initial observations within a task involve only variations in visual and proprioceptive input, we investigate whether VFD can capture epistemic uncertainty induced solely by varying the language command. We generate five increasingly semantically perturbed versions of each language prompt (P1 to P5) and evaluate the success rate and uncertainty for each, using the final policy π15 after iterative random data selection. The calibration curves in Figure 3, normalized for comparability, show that VFD is also well-calibrated with respect to the language instruction. 6.3

Active Fine-Tuning

Having shown superior calibration of VFD, we evaluate whether SAVE improves multitask performance more than diversity-based and uniform selection baselines, given the same expert demonstration budget. We compare our VFD-based acquisition rule against Action-L2 and GU, which performed best in the calibration experiments (cf. Table 1), as well as a diversity-based greedy baseline [61] that selects the most visually diverse initial observations. For the uncertainty-guided approaches, we sweep over temperature values τ ∈ {1, 1.5, 2, 2.5, 3}. Further implementation details are provided in Appendix B, and additional experimental results are reported in Appendix C. SAVE with VFD uncertainty guidance yields faster improvement. Table 2 reports the number of rounds it takes for different active fine-tuning approaches to achieve certain levels of success. SAVE w/ VFD surpasses the performance of random and diversity-based selection in the last three rounds with 50 % and 45 % fewer costly expert demonstrations, respectively. Compared to other approaches for uncertainty-guided data acquisition with SAVE, VFD requires at least 22 % less data to achieve similar performance. After all rounds, SAVE with VFD-guided data acquisition achieves the highest success rate of 67 % across tasks, outperforming the other methods by 3 to 12 % given the same 8

Table 2: Sample efficiency. Number of active fine-tuning rounds (↓) to reach certain success rates (SR) and final SR (↑) for different demonstration selection strategies. Thresholds not reached within 15 rounds are marked as "—".

≥ 40 % ≥ 45 % ≥ 50 % ≥ 55 % ≥ 60 % ≥ 65 %

5.7 6.7±1.2 8.7±2.1 — — —

±0.5

4.7 7.3±0.9 9.7±1.7 — — —

±0.8

5.0 6.0±1.4 9.0±2.4 9.0±1.0 — —

±1.4

5.0 5.7±2.4 7.3±1.9 11.0±1.6 12.7±1.7 —

5.0±0.8 5.3±0.9 6.0±0.8 8.0±1.6 10.0±0.8 12.5±0.5

Final SR

54.6±0.9

54.9±1.3

56.8±7.6

64.0±2.6

67.1±3.2

Top-task uncertainty share

Task concentration

Success rate

Overall success

Round r

0.30

SAVE w/ GU

=0 = 2.5

0.25 0.20 0.15

1 3 5 7 9 11 13 15

Round r

Success rate

Diversity

±1.2

0.7 0.6 0.5 0.4 =0 = 0 uniform 0.3 = 2.5 = 2.5 uniform 0.2 init 1 3 5 7 9 11 13 15

SAVE w/ Action-L2

SAVE w/ VFD

Random

SR Threshold

0.7 0.6 0.5 0.4 0.3 0.2

Exploration vs. success =0 =1 = 1.5

Round 1

0.7

=2 = 2.5 =3

Round 15

0.8

0.9

1.0

Normalized task entropy

Figure 4: Effect of the task-sampling temperature τ on SAVE. Larger τ biases expert queries toward higher-uncertainty tasks, with uniform sampling for τ = 0. Left: Uncertainty-guidance is beneficial both for sampling tasks and initial observations. Legend entries containing uniform correspond to sampling initial observations within a sampled task uniformly instead of uncertainty-guided. Middle: Uncertainty-based sampling more rapidly reduces the fraction of uncertainty concentrated in the most uncertain task, indicating better allocation of demonstrations to underperforming tasks. Right: SAVE reduces the difference in prior knowledge about tasks; for τ ≤ 2.5, the temperature controls an exploration–exploitation trade-off between task coverage and final success rate. budget of expert demonstrations. Comparing the results to the calibration reported in Table 1, we find that the performance of uncertainty-guided active fine-tuning heavily depends on the quality of the uncertainty signals. Task and initial state selection benefit from uncertainty guidance. To isolate the effects of task-level selection from choosing the exact initial configuration for the expert demonstration, we compare both parts with and without uncertainty guidance. Figure 4 (left) shows that most of the gain over random acquisition stems from the prioritization at the task level, while uncertainty-based initial observation selection yields a minor additional improvement. A key reason for this performance gain is shown in the middle of Figure 4 and in Figure 11. While pre-trained VLAs naturally exhibit different levels of uncertainty per task, our uncertainty-guided active fine-tuning strategy can reduce these differences and improve multitask performance more quickly by allocating the demonstration budget to tasks that require it most. Further, Figure 4 (right) complements this observation by showing that over multiple rounds, SAVE tends towards high-entropy categorical distributions over tasks (i.e., more diversity in the selection). But for moderate τ ≤ 2.5, it can actually trade off this diversity for faster improvement in multitask success by focusing on high-uncertainty tasks (cf. Figure 10). 6.4

Failure Detection

Lastly, we evaluate whether high epistemic uncertainty during deployment indicates imminent task failure. For this, we roll out the final VLA policy obtained after active fine-tuning 30 times per task and calculate VFD scores using the two-member ensemble at each action-generation timestep. Following prior works [65, 58], we calibrate task-specific thresholds from 10 successful rollouts using conformal prediction [4] and report accuracy and true-positive-rate (TPR). 9

7

TPR

Accuracy

We consider three recent baselines for detecting failDetection Accuracy True-Positive-Rate ures of generative policies: ACE [58] computes 0.7 0.8 the conditional entropy of the action distribution, STAC [2] compares action distributions at consecu0.6 0.6 tive timesteps, and RND-OE [58] detects OOD obser0.4 vations. As shown in Figure 5, high VFD is a strong 0.5 0.2 signal for policy failures, achieving 67 % accuracy 0.4 and correctly predicting 79 % of all failures (TPR). ACE STAC RND-OE VFD (ours) These results demonstrate the potential of our method to improve the reliability of VLAs by enabling them to express their own confidence. Figure 5: VFD epistemic uncertainty can detect failures during deployment.

Conclusion

We present VFD, a mathematically grounded and computationally tractable method for estimating epistemic uncertainty in flow-based VLAs. On that basis, we propose SAVE, a framework for multitask active fine-tuning of VLAs that allocates the demonstration budget to tasks and initial observations that require it most. Our experiments show that VFD is better-calibrated than baselines, and that using it for active fine-tuning with SAVE reduces data collection by at least 22 % compared to uncertainty estimation baselines. These results, along with the superior failure-detection performance, demonstrate the capabilities of our methods to improve the reliability and adaptability of VLAs. Limitations. While we have shown that VFD requires only a two-member ensemble to be wellcalibrated, the reliance on training and maintaining a separate set of model weights is a limitation of our method. We use the uncertainty for the initial observation as a proxy for task difficulty. Our experiments empirically confirm this correlation, but there may be certain scenarios that “look simple” but are challenging for the policy (e.g., due to complex contact dynamics). Finally, treating uncertainty as the primary signal for determining which data to collect does not account for informational dependencies across tasks. Developing tractable methods to quantify these dependencies and maximizing expected information gain are interesting avenues for future work.

10

Acknowledgements This work was supported by the German Federal Ministry of Research, Technology and Space (BMFTR) under the Robotics Institute Germany (RIG) funded by BMFTR grant 16ME0997K, the German Research Foundation (DFG) within the RTG project ConVeY funded by grant GRK 2428, the Humboldt Professorship for Robotics and Artificial Intelligence and the Swiss National Science Foundation under NCCR Automation, grant agreement 51NF40 180545. Marco Bagatella is supported by the Max Planck ETH Center for Learning Systems.

References [1] M. Abdar et al. A review of uncertainty quantification in deep learning: Techniques, applications and challenges. Information Fusion, 76:243–297, 2021. [2] C. Agia, R. Sinha, J. Yang, Z. Cao, R. Antonova, M. Pavone, and J. Bohg. Unpacking failure modes of generative policies: Runtime monitoring of consistency and progress. In Conference on Robot Learning (CoRL), 2025. [3] AgiBot-World et al. AgiBot World Colosseo: A large-scale manipulation platform for scalable and intelligent embodied systems. In International Conference on Intelligent Robots and Systems (IROS), 2025. [4] A. N. Angelopoulos and S. Bates. Conformal prediction: A gentle introduction. Foundations and Trends in Machine Learning, 16(4):494–591, 2023. [5] J. T. Ash, C. Zhang, A. Krishnamurthy, J. Langford, and A. Agarwal. Deep batch active learning by diverse, uncertain gradient lower bounds. In International Conference on Learning Representations (ICLR), 2020. [6] M. Awais, M. Naseer, S. Khan, R. M. Anwer, H. Cholakkal, M. Shah, M.-H. Yang, and F. S. Khan. Foundation models defining a new era in vision: a survey and outlook. IEEE Transactions on Pattern Analysis and Machine Intelligence, 47(4):2245–2264, 2025. [7] M. S. Ayhan and P. Berens. Test-time data augmentation for estimation of heteroscedastic aleatoric uncertainty in deep neural networks. In Medical Imaging with Deep Learning, 2018. [8] M. Bagatella, J. Hübotter, G. Martius, and A. Krause. Active fine-tuning of multi-task policies. In International Conference on Machine Learning (ICML), 2025. [9] L. Berry, A. Brando, and D. Meger. Shedding light on large generative networks: Estimating epistemic uncertainty in diffusion models. In Conference on Uncertainty in Artificial Intelligence (UAI), 2024. [10] K. Black et al. π0.5 : a vision-language-action model with open-world generalization. In Conference on Robot Learning (CoRL), 2025. [11] R. Cadene, S. Alibert, F. Capuano, M. Aractingi, A. Zouitine, P. Kooijmans, J. Choghari, M. Russi, C. Pascal, S. Palma, et al. LeRobot: An open-source library for end-to-end robot learning. In International Conference on Learning Representations (ICLR), 2026. [12] K. Chaloner and I. Verdinelli. Bayesian experimental design: A review. Statistical science, 10: 273–304, 1995. [13] M. Chan, M. Molina, and C. Metzler. Estimating epistemic and aleatoric uncertainty with a single model. Advances in Neural Information Processing Systems (NeurIPS), 2024. [14] C. Chi, S. Feng, Y. Du, Z. Xu, E. Cousineau, B. Burchfiel, and S. Song. Diffusion policy: Visuomotor policy learning via action diffusion. Robotics: Science and Systems (RSS), 2023. [15] Y. Cui, D. Isele, S. Niekum, and K. Fujimura. Uncertainty-aware data aggregation for deep imitation learning. In IEEE International Conference on Robotics and Automation (ICRA), 2019. 11

[16] E. Daxberger, A. Kristiadi, A. Immer, R. Eschenhagen, M. Bauer, and P. Hennig. Laplace redux - effortless Bayesian deep learning. Advances in Neural Information Processing Systems (NeurIPS), 2021. [17] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), 2019. [18] J. Diquigiovanni, M. Fontana, S. Vantini, et al. The importance of being a band: Finite-sample exact distribution-free prediction sets for functional data. STATISTICA SINICA, 1:1–41, 2024. [19] S. Dohare, J. F. Hernandez-Garcia, Q. Lan, P. Rahman, A. R. Mahmood, and R. S. Sutton. Loss of plasticity in deep continual learning. Nature, 632:768–774, 2024. [20] A. Dosovitskiy et al. An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learning Representations (ICLR), 2021. [21] B. Efron. Tweedie’s formula and selection bias. Journal of the American Statistical Association, 106:1602–1614, 2011. [22] P. Esser, S. Kulal, A. Blattmann, R. Entezari, J. Müller, H. Saini, Y. Levi, D. Lorenz, A. Sauer, F. Boesel, et al. Scaling rectified flow transformers for high-resolution image synthesis. In International Conference on Machine Learning (ICML), 2024. [23] E. Fadeeva et al. Fact-checking the output of large language models via token-level uncertainty quantification. In Findings of the Association for Computational Linguistics: ACL 2024, pages 9367–9385, 2024. [24] G. Franchi, N. Belkhir, D. N. Trong, G. Xia, and A. Pilzer. Towards understanding and quantifying uncertainty for text-to-image generation. In Conference on Computer Vision and Pattern Recognition (CVPR), 2025. [25] Y. Gal and Z. Ghahramani. Dropout as a Bayesian approximation: Representing model uncertainty in deep learning. In International Conference on Machine Learning (ICML), 2016. [26] Y. Gal, R. Islam, and Z. Ghahramani. Deep Bayesian active learning with image data. In International Conference on Machine Learning (ICML), 2017. [27] Q. Gu, Y. Ju, S. Sun, I. Gilitschenski, H. Nishimura, M. Itkina, and F. Shkurti. SAFE: Multitask failure detection for vision-language-action models. In Advances in Neural Information Processing Systems (NeurIPS), 2025. [28] Z. He, Y. Cao, and M. Ciocarlie. Uncertainty comes for free: Human-in-the-loop policies with diffusion models. arXiv preprint arXiv:2503.01876, 2025. [29] J. Hejna, S. Mirchandani, A. Balakrishna, A. Xie, A. Wahid, J. Tompson, P. Sanketi, D. Shah, C. Devin, and D. Sadigh. Robot data curation with mutual information estimators. Robotics: Science and Systems (RSS), 2025. [30] J. Ho, A. Jain, and P. Abbeel. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems (NeurIPS), 33:6840–6851, 2020. [31] D. Holzmüller, V. Zaverkin, J. Kästner, and I. Steinwart. A framework and benchmark for deep batch active learning for regression. Journal of Machine Learning Research (JMLR), 24(164): 1–81, 2023. [32] E. Hüllermeier and W. Waegeman. Aleatoric and epistemic uncertainty in machine learning: An introduction to concepts and methods. Machine Learning, 110:457–506, 2021. [33] M. Jazbec, E. Wong-Toi, G. Xia, D. Zhang, E. Nalisnick, and S. Mandt. Generative uncertainty in diffusion models. In Conference on Uncertainty in Artificial Intelligence (UAI), 2025. [34] L. Ju, M. Nautiyal, A. Hellander, E. Vats, and P. Singh. Epistemic uncertainty quantification for pre-trained VLMs via Riemannian flow matching. arXiv preprint arXiv:2601.21662, 2026. 12

[35] K. Judah, A. Fern, and T. G. Dietterich. Active imitation learning via reduction to IID active learning. In Conference on Uncertainty in Artificial Intelligence (UAI), 2012. [36] R. Karczewski, M. Heinonen, and V. Garg. Diffusion models as cartoonists: The curious case of high density regions. In International Conference on Learning Representations (ICLR), 2025. [37] U. B. Karli, T. Kurumisawa, and T. Fitzgerald. Ask before you act: Token-level uncertainty for intervention in vision-language-action models. In Second Workshop on Out-of-Distribution Generalization in Robotics at RSS, 2025. [38] A. Khazatsky et al. DROID: A large-scale in-the-wild robot manipulation dataset. In Robotics: Science and Systems, 2024. [39] M. J. Kim et al. OpenVLA: An open-source vision-language-action model. In Conference on Robot Learning (CoRL), 2024. [40] A. Kirsch, J. van Amersfoort, and Y. Gal. BatchBALD: Efficient and diverse batch acquisition for deep Bayesian active learning. In Advances in Neural Information Processing Systems (NeurIPS), 2019. [41] B. Lakshminarayanan, A. Pritzel, and C. Blundell. Simple and scalable predictive uncertainty estimation using deep ensembles. Advances in Neural Information Processing Systems (NeurIPS), 2017. [42] S.-W. Lee, X. Kang, and Y.-L. Kuo. Diff-DAgger: Uncertainty estimation with diffusion policy for robotic manipulation. In IEEE International Conference on Robotics and Automation (ICRA), 2025. [43] Q. Li, B. Yin, W. Huang, R. Liu, B. Zou, R. Yu, J. Ye, W. Yu, and X. Wang. Visionlanguage-action safety: Threats, challenges, evaluations, and mechanisms. arXiv preprint arXiv:2604.23775, 2026. [44] C. Ling et al. Uncertainty quantification for in-context learning of large language models. In North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), 2024. [45] Y. Lipman, R. T. Chen, H. Ben-Hamu, M. Nickel, and M. Le. Flow matching for generative modeling. In International Conference on Learning Representations (ICLR), 2023. [46] B. Liu, Y. Zhu, C. Gao, Y. Feng, Q. Liu, Y. Zhu, and P. Stone. LIBERO: Benchmarking knowledge transfer for lifelong robot learning. Advances in Neural Information Processing Systems (NeurIPS), 2023. [47] A. Loquercio, M. Segu, and D. Scaramuzza. A general framework for uncertainty estimation in deep learning. IEEE Robotics and Automation Letters, 2020. [48] H. Ma, J. Chen, J. T. Zhou, G. Wang, and C. Zhang. Estimating LLM uncertainty with evidence. arXiv preprint arXiv:2502.00290, 2025. [49] A. Malinin and M. Gales. Uncertainty estimation in autoregressive structured prediction. In International Conference on Learning Representations (ICLR), 2021. [50] Z. Mei, T. Yin, M. Baker, O. Shorinwa, and A. Majumdar. World models that know when they don’t know: Controllable video generation with calibrated uncertainty. arXiv preprint arXiv:2512.05927, 2025. [51] E. Nalisnick, A. Matsukawa, Y. W. Teh, D. Gorur, and B. Lakshminarayanan. Do deep generative models know what they don’t know? In International Conference on Learning Representations (ICLR), 2019. [52] NVIDIA et al. GR00T N1: An open foundation model for generalist humanoid robots. arXiv preprint arXiv:2503.14734, 2025. 13

[53] A. O’Neill et al. Open X-embodiment: Robotic learning datasets and RT-X models: Open X-embodiment collaboration. In IEEE International Conference on Robotics and Automation (ICRA), 2024. [54] Physical Intelligence et al. π0 : A vision-language-action flow model for general robot control. arXiv preprint arXiv:2410.24164, 2024. [55] A. Z. Ren et al. Robots that ask for help: Uncertainty alignment for large language model planners. In Conference on Robot Learning (CoRL), 2023. [56] J. Ren, J. Luo, Y. Zhao, K. Krishna, M. Saleh, B. Lakshminarayanan, and P. J. Liu. Out-ofdistribution detection and selective generation for conditional language models. In International Conference on Learning Representations (ICLR), 2023. [57] M. Reuss, H. Zhou, M. Rühle, Ö. E. Yağmurlu, F. Otto, and R. Lioutikov. FLOWER: Democratizing generalist robot policies with efficient vision-language-flow policies. In Conference on Robot Learning (CoRL), 2025. [58] R. Römer, A. Kobras, L. Worbis, and A. P. Schoellig. Failure prediction at runtime for generative robot policies. Advances in Neural Information Processing Systems (NeurIPS), 2025. [59] R. Römer, J. Balletshofer, J. Thumm, M. Pavone, A. P. Schoellig, and M. Althoff. From demonstrations to safe deployment: Path-consistent safety filtering for diffusion policies. In IEEE International Conference on Robotics and Automation (ICRA), 2026. [60] R. Römer, Y. Zhang, and A. P. Schoellig. CLARE: Continual learning for vision-language-action models via autonomous adapter routing and expansion. arXiv preprint arXiv:2601.09512, 2026. [61] O. Sener and S. Savarese. Active learning for convolutional neural networks: A core-set approach. In International Conference on Learning Representations (ICLR), 2018. [62] B. Settles. Active learning literature survey. Technical report, University of Wisconsin–Madison, 2009. [63] O. Shorinwa, Z. Mei, J. Lidard, A. Z. Ren, and A. Majumdar. A survey on uncertainty quantification of large language models: Taxonomy, open research challenges, and future directions. ACM Computing Surveys, 58:1–38, 2025. [64] M. Shukor et al. SmolVLA: A vision-language-action model for affordable and efficient robotics. arXiv preprint arXiv:2506.01844, 2025. [65] C. Xu, T. Khuong Nguyen, E. Dixon, C. Rodriguez, P. Miller, R. Lee, P. Shah, R. Ambrus, H. Nishimura, and M. Itkina. Can we detect failures without failure data? Uncertainty-aware runtime failure detection for imitation learning policies. In Robotics: Science and Systems (RSS), 2025. [66] X. Zhai, B. Mustafa, A. Kolesnikov, and L. Beyer. Sigmoid loss for language image pre-training. In IEEE/CVF International Conference on Computer Vision (ICCV), pages 11975–11986, 2023. [67] B. Zitkovich et al. RT-2: Vision-language-action models transfer web knowledge to robotic control. In Conference on Robot Learning (CoRL), 2023.

14

Appendix Table of Contents A Theoretical Results A.1 Flow Matching Fundamentals . . . . . . . . . . . . . . . . . . . . . . . . . . . A.2 Proof of Theorem 4.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

16 16 16

B Experimental Details B.1 Computational Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B.2 Environments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B.3 Pre-Training . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B.4 Uncertainty Quantification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B.5 Calibration Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B.6 Active Fine-Tuning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . B.7 Failure Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

18 18 18 18 18 21 23 23

C Additional Results C.1 Toy Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . C.2 Additional Results for Section 6.3 . . . . . . . . . . . . . . . . . . . . . . . . . C.3 Ensembling vs. Laplace Approximation . . . . . . . . . . . . . . . . . . . . . . C.4 Iterative Fine-Tuning vs. Retraining . . . . . . . . . . . . . . . . . . . . . . . . C.5 Effect of τ on SAVE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . C.6 Detailed Results on Failure Detection . . . . . . . . . . . . . . . . . . . . . . .

23 23 24 24 26 26 26

D Broader Impact

28

15

A

Theoretical Results

A.1

Flow Matching Fundamentals

Flow matching [45] formulates generative modeling as learning a time-dependent vector field us that transports samples from a simple base distribution p0 to the data distribution p1 = q. We consider a noise distribution p0 (x0 ) = N (x0 | 0, I) and a data distribution x1 ∼ q. The OT Gaussian conditional probability path is defined as ps (x | x1 ) = N (x | sx1 , (1 − s)2 I), (12) and the corresponding flow that pushes the noise distribution p0 to ps (x | x1 ) is given by the affine map ϕs (x0 ) = (1 − s)x0 + sx1 . (13) d The conditional velocity field us (x | x1 ) defining this flow via the ODE ds ϕs (x0 ) = us (ϕs (x)), and generating ps (x | x1 ), can be derived as: d us (x | x1 ) = ϕs (x0 ) = x1 − x0 . (14) ds Marginalizing over the data distribution q(x1 ) yields the marginal probability path Z (15) ps (x) = ps (x | x1 )q(x1 )dx1 , and the marginal velocity field is defined as Z ps (x | x1 )q(x1 ) us (x) = us (x | x1 ) dx1 . ps (x) Finally, these components must satisfy the continuity equation ∂ ps (x) + div(ps (x)us (x)) = 0, ∂s which ensures probability mass conservation. A.2

(16)

(17)

Proof of Theorem 4.1

To simplify notation, we drop the fixed conditioning input y and the parameterization of the velocity fields as neural networks in the following derivations, considering two terminal distributions p1 (x) = p11 (x), p2 (x) = p21 (x) induced by velocity fields u1s (x), u2s (x) and similar base distributions p10 (x) = p20 (x) = N (0, I). In the proof of Theorem 4.1, we will use the following lemma. Lemma A.1. Let us (x) be the marginal velocity field of a flow matching model trained with Gaussian OT conditional probability paths, and let ps (x) be the marginal probability path induced by us (x). Then, for all s ∈ [0, 1), sus (x) − x ∇ log ps (x) = . (18) 1−s Proof. Applying Tweedie’s formula [21] to (12) and (15) yields ∇ps (x) ps (x) sE[x1 | xs = x] − x ⇔ ∇ log ps (x) = . (1 − s)2 Plugging the conditional velocity (14) into the marginal velocity (16) gives Z us (x) = (x1 − x0 )ps (x1 | xs = x)dx1 E[sx1 | xs = x] = x + (1 − s)2

⇔

= E[x1 − x0 | xs = x] E[x1 | xs = x] − x = 1−s E[x1 | xs = x] = x + (1 − s)us (x), 16

(19) (20)

(21) (22) (23) (24)

1 −x where (23) follows from x = (1 − s)x0 + sx1 ⇔ x1 − x0 = x1−s and taking the conditional expectation. Finally, plugging (23) into (20) yields

sx + s(1 − s)us (x) − x (1 − s)2 sus (x) − x = , 1−s

∇ log ps (x) =

(25) (26)

which concludes the proof. Proof of Theorem 4.1. Recall that we aim to express the KL divergence between two distributions p1 (x) and p2 (x) parameterized as flow matching models in terms of their time-dependent velocity fields u1s (x) and u2s (x). We write DKL (p1 (x) || p2 (x)) = DKL (p1 (x) || p2 (x)) − DKL (p10 (x) || p20 (x)) + DKL (p10 (x) || p20 (x)) (27) Z 1 ∂ = DKL (p1s (x) || p2s (x))ds, (28) 0 ∂s where the last step follows from the fact that p10 = p20 since both models use the same source distribution. Applying the product rule, we can reformulate the term in the integral as Z ∂ ∂ p1 (x) 1 2 DKL (ps (x) || ps (x)) = p1s (x) log s2 dx (29) ∂s ∂s ps (x) Z Z ∂ 1 p1 (x) ∂ p1 (x) = ps (x) log 2s dx + p1s (x) log s2 dx (30) ∂s ps (x) ∂s ps (x) ! Z Z ∂ 1 ∂ 2 p (x) p (x) ∂ 1 p1 (x) = ps (x) log 2s dx + p1s (x) ∂s 1 s − ∂s 2 s dx ∂s ps (x) ps (x) ps (x) (31) Z 1 Z Z ps (x) ∂ 2 ∂ 1 p1s (x) ∂ 1 = p (x) log 2 dx + p (x)dx − p (x)dx. ∂s s ps (x) ∂s s p2s (x) ∂s s {z } | =0

(32) R ∂

R ∂

∂ where the second term vanishes since ∂s p1s (x)dx = ∂s p1s (x)dx = ∂s 1 = 0. To reformulate the other terms in (32), we proceed as Z Z 1 ∂ ∂ 1 p1 (x) ps (x) ∂ 2 DKL (p1s (x) || p2s (x)) = ps (x) log 2s dx − p (x)dx (33) ∂s ∂s ps (x) p2s (x) ∂s s Z Z 1 p1 (x) ps (x) = − div(p1s (x)u1s (x)) log s2 dx + div(p2s (x)u2s (x))dx ps (x) p2s (x) (34) Z Z 1 1 p (x) p (x) s s = p1s (x)u1,⊤ dx − p1s (x)u2,⊤ dx s (x)∇ log 2 s (x)∇ log 2 ps (x) ps (x) (35) Z 1 p (x) = p1s (x)(u1s (x) − u2s (x))⊤ ∇ log 2s dx (36) ps (x) Z = p1s (x)(u1s (x) − u2s (x))⊤ (∇ log p1s (x) − ∇ log p2s (x))dx, (37)

where (34) follows from the continuity equation (17), and (35) follows from the divergence theorem m and the assumption that pm s (x)us (x) → 0 as ∥x∥ → ∞. Applying Lemma A.1 to the last term in the integral (37) yields s ∇ log p1s (x) − ∇ log p2s (x) = (u1 (x) − u2s (x)). (38) 1−s s 17

s Defining κs = 1−s and plugging (38) into (37) results in Z ∂ 1 2 DKL (ps (x) || ps (x)) = p1s (x)(u1s (x) − u2s (x))⊤ (∇ log p1s (x) − ∇ log p2s (x))dx ∂s Z = p1s (x)κs (u1s (x) − u2s (x))⊤ (u1s (x) − u2s (x))dx   = κs Ex∼p1s (·) ∥u1s (x) − u2s (x)∥22 .

Finally, we can insert (41) into (28) and obtain Z 1   1 2 DKL (p (x) || p (x)) = κs Ex∼p1s (·) ∥u1s (x) − u2s (x)∥22 ds,

(39) (40) (41)

(42)

0

concluding the proof. Remark A.2. Although the weighting κs in (42) diverges as s → 1, the VFD uncertainty estimate remains finite. In practice, our estimator (7) evaluates the velocity difference on a grid sℓ = ℓδs with ℓ ∈ {0, . . . , Ns − 1}, so the largest weight is κ1−δs = 1−δs δs , which is finite for δs > 0.

B

Experimental Details

B.1

Computational Resources

All experiments1 are run on a compute node with two NVIDIA RTX 4090 GPUs, 8 CPU cores, and 64 GB of RAM. With this hardware setup, one active learning experiment with 15 rounds of uncertainty quantification, episode selection, 4,000 gradient steps, and 30 evaluation rollouts per task takes approximately 12.5 hours. We run 25 different configurations for active learning, ablating temperature and uncertainty estimates, over three seeds, resulting in 75 total active learning experiment runs. This results in approximately 940 compute hours, i.e., with two GPUs in parallel, approximately 1,880 GPU hours. We estimate that about the same computing budget was used for developing the codebase and preliminary experiments. B.2

Environments

We use the LIBERO simulation benchmark [46] (MIT License) in our experiments. The environments include a Franka robotic manipulator with a parallel-yaw gripper in a kitchen environment. For each task, up to 50 human expert demonstrations are available. The observations ot consist of two RGB images with spatial resolution 256 × 256, one from a fixed workspace view and one from a camera mounted on the robot’s end-effector. Additionally, observations include the 8-dimensional proprioceptive state of the arm, comprising the end-effector position (3D), the end-effector orientation parameterized as an axis-angle (3D), and the gripper joint positions (2D). Actions at are continuous and 7-dimensional: 3D delta end-effector translation, 3D delta end-effector orientation, and 1D gripper command. B.3

Pre-Training

We pre-train a SmolVLA [64] (Apache License) model for 30,000 steps with a batch size of 32 on LIBERO-Spatial, LIBERO-Object, and LIBERO-Goal (10 tasks each), as well as tasks (0, 1, 2) from LIBERO-10 (also referred to as LIBERO-Long). B.4

Uncertainty Quantification

The computation of VFD-based epistemic uncertainty estimation is summarized in Algorithm 2 and illustrated in Figure 6. Action-L2 Implementation. Action-L2 estimates epistemic uncertainty from the disagreement between the sampler policy and an ensemble of M terminal action distributions. Given an observation 1 Our codebase builds on LeRobot [11], released under an Apache License.

18

Algorithm 2 Efficient Epistemic Uncertainty Estimation for Flow-Matching Models using Velocity Field Disagreement (VFD). 1: Input: Flow-matching model ensemble pθ1 (x | y), pθ2 (x | y) with vector fields V = (vsθ1 , vsθ2 )

trained via OT paths, conditioning input y, ODE integration step size δs = 1/Ns . 2: Output: Estimate of the epistemic uncertainty ue (y; V) at the conditioning input y. 3: Hyperparameters: Batch size B. (1,1:B)

(2,1:B)

4: Sample x0 ∼ N (0, I), x0 5: Initialize ue ← 0. 6: for ℓ = 0 to Ns − 1 do 7: Set s = ℓδs. 8: for each i ∈ {1, 2} do

∼ N (0, I). ▷ ODE integration.

2 PB (i,b) (i,b) Compute the expected VFDs d2ℓ,i = B1 b=1 vsθ1 xs , y − vsθ2 xs , y 2 . 1 s 10: Update uncertainty estimate ue ← ue + 2N d2 . s 1−s ℓ,i
(i,1:B) (i,1:B) (i,1:B) 11: Forward integrate xs+δs = xs + vsθi xs , y δs. 12: return ue (y; V) = ue . 9:

Figure 6: Visualization of the velocity-field disagreement (VFD) computation for two conditioning inputs with different levels of epistemic uncertainty. o, we draw independent Gaussian noise samples from p0 and generate C terminal action chunks from the sampler policy and from each ensemble member. Let {Asi (o)}C i=1 denote the action chunks generated by the sampler policy, and let (m)

{Aj

(o)}C j=1

denote the action chunks generated by ensemble member m ∈ {1, . . . , M }. The discrepancy between the sampler policy and ensemble member m is computed as the mean pairwise Euclidean distance between their sampled terminal action chunks: C

dm (o) =

C

1 XX (m) Asi (o) − Aj (o) . C 2 i=1 j=1 2

The Action-L2 epistemic uncertainty estimate is then the average discrepancy across ensemble members: M 1 X uAction-L2 (o) = dm (o). M m=1 ACE Implementation. ACE [58] estimates epistemic uncertainty from the dispersion of action samples generated by a single policy. Given an observation o, we draw C independent Gaussian 19

noise samples and generate C action chunks via flow-matching ODE integration. For each sampled chunk c ∈ {1, . . . , C}, we extract the first three action dimensions at each timestep k, corresponding (c) to end-effector position deltas ∆pk ∈ R3 . Starting from the current end-effector position pcurr , these deltas are integrated to obtain absolute position trajectories: t X (c) (c) pt = pcurr + ∆pk , t ∈ {1, . . . , H}. k=1 (c)

At each timestep t in the prediction horizon H, the sampled positions {pt }C c=1 are discretized into a 3D grid. The grid cell size along each coordinate axis is set to 0.03 times the empirical range of the samples along that axis. Let Bt denote a set of bins over the occupied grid at time t and let Pt (b) be the empirical probability of a sample falling into bin b ∈ Bt at time t. The ACE uncertainty estimate is the Shannon entropy of these bin counts, averaged over the prediction horizon: H

uACE (o) = −

1 XX Pt (b) log2 Pt (b). H t=1 b∈Bt

DECU Implementation. DECU [9] estimates epistemic uncertainty in diffusion models from the disagreement among ensemble members’ denoising mean at an intermediate point of the generative process. Given the close relationship between diffusion and flow matching, we adapt DECU to flow matching by replacing the denoising mean with the learned velocity field. Given an observation o, we (c) sample C independent Gaussian noise vectors x0 ∼ p0 for c ∈ {1, . . . , C}. Each ensemble member integrates the flow-matching ODE from s = 0 to a branching time sb = 0.995 to obtain intermediate (c) (c) (c) states xsb . At sb , we evaluate the velocity fields of all M ensemble members, vi = vsθbi (xsb , o) for i ∈ {1, . . . , M }. Disagreement is measured using the Pairwise-Distance Estimator (PaiDE). For each sampled chunk c, define the squared pairwise velocity-field distances (c)

(c)

Dij = vi

(c)

− vj

2

i, j ∈ {1, . . . , M }.

, 2

The score for sample c is then  M   X 1 (c) log  exp −Dij  . M i=1 M j=1

1 (c) dDECU (o) = −

M X



The DECU epistemic uncertainty estimate for observation o is the mean score over the C sampled chunks: C 1 X (c) uDECU (o) = d (o). C c=1 DECU Generative Uncertainty (GU) Implementation. As described in the main text, GU [33] approximates the posterior predictive action distribution as a Gaussian. Using our deep ensemble, we compute a raw uncertainty score for each initial observation okl , where k indexes the task and l indexes the observation within that task. For a fixed initial noise sample, let A(i) (okl ) ∈ RH×Da (i) denote the action trajectory generated by ensemble member i ∈ {1, . . . , M }, and let akl,t,d denote its action value at timestep t ∈ {1, . . . , H} and action dimension d ∈ {1, . . . , Da }. The raw GU uncertainty score is obtained by summing the log-variance of the ensemble predictions across all timesteps and action dimensions: uraw kl =

Da H X X

 h i  (i) log VarM i=1 akl,t,d + ϵvar ,

t=1 d=1

20

where ϵvar = 10−8 is added for numerical stability. Because the differential entropy of a continuous distribution can be negative, these raw scores may also be negative. To obtain non-negative uncertainty weights for use in (9), we first aggregate the observation-level scores into a raw task-level uncertainty score Ukraw . We then shift the task-level scores by   raw raw Uk = Ukraw − ′ min Ukraw + ϵ max U − min U . ′ off k′ k′ ′ ′ k ∈{1,...,K}

k ∈{1,...,K}

k ∈{1,...,K}

Here, ϵoff > 0 is a small offset coefficient. This shift removes negative values and when the task scores are not all identical, the range-dependent offset moves the minimum-uncertainty task away from zero so that it retains non-zero sampling probability after normalization. Entropy Implementation. Entropy [49] utilizes the extent to which the VLM (SmolVLA backbone) successfully predicted the next token of prefix. Given an observation o, the policy constructs a prefix sequence of length Sprefix containing images, robot states and language. The VLM predicts the next-token probability distribution ψm at each valid prefix position m ∈ {1, . . . , Sprefix }, with a single forward pass. The uncertainty score is calculated by Shannon entropy of this distribution, averaged over all valid prefix tokens:

uEntropy = −

1

Sprefix

XX

Sprefix m=1

ψm (z) log ψm (z),

z∈Z

where Z is the VLM vocabulary and ψm (z) is the predicted probability of token z at position m. Perplexity Implementation. Perplexity [23] estimates uncertainty by measuring the likelihood of the language instruction under the VLM backbone. Similar to Entropy, it evaluates the prefix in a single forward pass without action generation. We extract the predicted next-token log-probabilities for the Slang language tokens within their respective prefix. Let zm be the true language token at sequence position m. The uncertainty score u is the exponentiated negative mean log-probability of these tokens:  uPerp = exp −

Slang X



1 log ψm (zm ) Slang m=1

Diversity-Based Greedy. The diversity-based greedy baseline [61] selects initial observations to maximize visual diversity. Let Ok denote the pool of candidate initial observations, and Ô denote the set of selected observations. We extract visual features for each oi ∈ Ok using a SigLIP [66] vision encoder2 . We rank all candidates using the k-Center Greedy algorithm. At each step, we select o∗ that maximizes the minimum distance ∆ to Ô: o∗ = arg max

min ∆(oi , oj ),

oi ∈Ok \Ô oj ∈Ô

where ∆(oi , oj ) is the Euclidean distance between visual features. We update Ô ← Ô ∪ {o∗ } and repeat until all observations are ranked. In each active fine-tuning round, we select the top ne observations from this ranking. B.5

Calibration Experiments

We primarily measure calibration by comparing the uncertainty estimate for an initial observation with the success probability of rolling out the policy from there. More specifically, we calculate the mean uncertainty (8) per task and evaluate the per-task success rate by rolling out the policy multiple times for different initial states of the robot and the objects in the scene. To assess whether higher uncertainty correlates with a lower success rate, we calculate the Spearman rank correlation 2Weights downloaded from Hugging Face: huggingface.co/google/siglip-so400m-patch14-384.

21

Table 3: Prompts to evaluate calibration of the uncertainty estimators with respect to variations in the language input. Changes from the original prompt are marked in bold. Task P1 (orig.)

P2

P3

P4

P5

1

put both the alpha- put the alphabet place both the al- move both the al- transfer the soup bet soup and the soup and the phabet soup and the phabet soup and cans to the basket tomato sauce in the tomato sauce into tomato sauce in the tomato sauce to the basket the basket basket basket

2

put both the cream place both the put the cream move the cream place the dairy cheese box and the cream cheese box cheese box and cheese and the but- products in the butter in the basket and the butter in the butter into the ter into the basket basket the basket basket

3

turn on the stove switch on the stove start the stove and activate the stove power on the cookand put the moka and place the moka set the moka pot on and put the moka top and position pot on it pot on it the burner pot on top of it the coffee maker on it

4

put the black bowl put the black bowl place the black store the black move the dark in the bottom into the lower bowl in the bottom bowl in the bottom bowl to the lowest drawer of the drawer of the cabinet drawer cabinet drawer cabinet compartcabinet and close it cabinet and shut it and close it and close the ment and shut it drawer

5

put the white mug set the white mug place the white put the white cup arrange the mugs: on the left plate and on the left plate mug on the left on the left dish and white one on the put the yellow and and the yellow and plate and place the the yellow-white left plate, yellow white mug on the white mug on the yellow and white cup on the right and white one on right plate right plate mug on the right dish the right plate plate

6

pick up the book grab the book and pick up the book take the book move the book to and place it in the place it in the back and put it in the and place it in the the back compartback compartment compartment of the rear compartment back section of the ment of the orgaof the caddy caddy of the caddy caddy nizer

7

put the white mug place the white put the white cup set the white mug position the mug on the plate and put mug on the plate on the dish and on the plate and on the plate and set the chocolate pud- and put the choco- move the chocolate place the chocolate the pudding to the ding to the right of late pudding to the pudding to the right pudding to the right right of it the plate right of the plate side of the plate of the plate

8

place both the al- put both the alpha- put the alphabet move both the al- transfer the soup phabet soup and the bet soup and the soup and the cream phabet soup and and cheese box to cream cheese box cream cheese box cheese box into the cream cheese box the basket in the basket in the basket basket to the basket

9

put both moka pots place both moka put the two moka set both coffee pots move the two on the stove pots on the stove pots on the stove on the stove moka pots onto the cooktop

10

put the yellow and place the yellow put the yellow and move the yellow place the yellowwhite mug in the and white mug in white mug into and white mug into white mug in the microwave and the microwave and the microwave and the microwave and microwave and close it close it shut it close the door shut the door

coefficient ρ between the two quantities. If and only if the success rate decreases monotonically with higher uncertainty, ρ = −1. As a secondary metric, we also report the Pearson correlation coefficient, which measures the strength of the linear relationship between the two quantities. In Figure 3, we investigate whether the uncertainty scores are well-calibrated when a decrease in success rate stems solely from varying the language command. To this end, we use different language prompts for each task, listed in Table 3. 22

B.6

Active Fine-Tuning

The set of tasks T over which we want to maximize multitask performance includes all K = 10 tasks from LIBERO-10. We perform active fine-tuning for R = 15 rounds, selecting ne = 5 episodes per round, resulting in a total of 75 selected demonstrations. In each round, we warm-start the VLA ensemble’s model parameters with the final parameters from the previous fine-tuning round and perform 4,000 gradient steps with a batch size of 32. The learning rate is adapted using cosine decay, with 200 initial warmup steps followed by a decay from 5 × 1−−5 to 55 × 1−−5 over the remaining steps. We replay the ensemble training data Dpre , using a replay ratio of λ = 0.5. This means that 50 % of the gradient steps are performed on data from episodes selected during active fine-tuning, and the remaining on Dpre . This strategy avoids catastrophic forgetting [19, 60] and maintains sufficient diversity within the VLA ensemble. B.6.1

Evaluation

In each active fine-tuning round r, we evaluate the current VLA ensemble by performing 30 envi(1) ronment rollouts with the first member πr for each LIBERO-10 task. We roll out the policy for up to 520 environment timesteps, terminating early upon task success. Sample Efficiency Computation. The goal of active fine-tuning is to improve performance with as little data as possible. To quantify this capability, we measure sample efficiency as follows. Denote the success rate of method A after round r1 by SRA,r1 . Let r2 = argminr s.t. SRB,r ≥ SRA,r1 be the number of rounds required for method B to reach at least a similar success rate as SRA,r1 . Then, if r2 < r1 , method B requires (1 − rr12 ) × 100% fewer samples to achieve the same performance, 1 i.e., method B is r1r−r × 100% more sample-efficient than method A. To obtain robust results, we 2 average across r1 ∈ {R − 2, R − 1, R}. With this evaluation protocol, we find that SAVE w/ VFD requires 50 % fewer samples than Random, 45 % fewer samples than Diversity, 40 % fewer samples than SAVE w/ Action-L2, and 22 % fewer samples than SAVE w/ GU, corresponding to 101 %, 83 %, 68 %, and 29 % higher sample efficiency, respectively. B.7

Failure Detection

For our experiments, we utilize the failure prediction framework [58], released under the MIT license. We use 10 successful rollouts per task for threshold calibration, i.e., to compute a one-sided conformal prediction band [18]. At each policy inference timestep, we compute the uncertainty score ue and flag the rollout as Fail if ue exceeds its corresponding threshold. The true-positive-rate (TPR) quantifies how many failures were correctly identified as such, and the TNR quantifies how many successful rollouts were correctly not flagged as Fail. We also report the detection time for rollouts correctly flagged as Fail, which corresponds to the timestep when the uncertainty score first exceeds its threshold. The detection time is normalized by the maximum episode length. Since failure detection is a trade-off between accuracy and early detection, we also report timestep-wise accuracy (TWA), which rewards correctly detected failures more the earlier they are flagged. Following [58], we do not cherry-pick a confidence value for conformal prediction and instead average results over quantiles (0.9, 0.91, . . . , 0.99).

C

Additional Results

C.1

Toy Example

We illustrate the capabilities of our VFD method for quantifying epistemic uncertainty in flowmatching models with a simple example. Example 1. Consider a conditional generative modeling problem with x = (x1 , x2 ), y = (y1 , y2 ) ∈ R2 . The ground-truth conditional distribution is a bimodal Gaussian with input-dependent means; q(x | y) = 0.5N (x | µ1 (y), I) + 0.5N (x | µ2 (y), I), where µ1 (y) = (sin (πy1 ) + 0.5, cos (πy2 ) − 0.5) and µ2 (y) = (cos (2πy2 ) − 0.5, sin (0.5πy1 ) + 0.5). We draw a set of conditioning inputs {y (n) }N n=1 with N = 4000 from a uniform distribution over an annulus with inner radius 0.75 and outer radius 1.25 and sample x(n) ∼ q(x | y (n) ). We parameterize the velocity field using an MLP and train an ensemble of M = 5 flow-matching models 23

Training Data Density 1.5

VFD Epistemic Uncertainty

1.5 50

1.0

20

0.5

40

0.5

0.0

15

0.0

30

0.0

0.5

10

0.5

20

0.5

1.0

5

1.0

10

1.0

0

1.5

0

1.5

0.5

1.5 1

0 y1

1

1.5 1.0 0.5 0.0 0.5 1.0 1.5 y1

y2

1.0

y2

25

1.0

y2

1.5

KL Divergence (pi||q) 3.0 2.5 2.0 1.5 1.0 0.5 1.5 1.0 0.5 0.0 0.5 1.0 1.5 y1

0.0

Figure 7: Epistemic uncertainty estimation for a 2D generative modeling problem. Our velocity field disagreement (VFD) uncertainty score is high for inputs far from the training distribution, similar to the KL divergence between the learned models’ conditional distributions and the ground truth.

0.7

Success Rate

0.6 0.5 0.4 Random Diversity GU Action-L2 VFD (ours)

0.3 0.2

init 1

2

3

4

5

6

7 8 9 Round r

10 11 12 13 14 15

Figure 8: Learning curves corresponding to the active fine-tuning results reported in Table 2. with a batch size of 256 for 300 epochs using the standard loss (3). As shown in Figure 7, our VFD score assigns regions far from the training data a high level of epistemic uncertainty, which closely PM resembles the average KL divergence i=1 DKL (pθi || q) between the learned distributions and the ground truth (4a). C.2

Additional Results for Section 6.3

We provide the full learning curves obtained in the active fine-tuning experiments in Figure 8. These plots correspond to the results reported in Tables 2 and 4. C.3

Ensembling vs. Laplace Approximation

We compare training a VLA ensemble to computing the Laplace approximation [16] for sampling from the model posterior. Since computing the full Laplace approximation for the entire SmolVLA model is computationally intractable, we resort to the last-layer Laplace approximation, which the authors also use to compute GU in the original work [33]. We fit a diagonal last-layer Laplace approximation with laplace-torch [16] around each trained SmolVLA checkpoint. Concretely, we freeze the full VLA and place the posterior only on the final action projection layer, which maps 3 This corresponds to sampling tasks and initial observations uniformly at random.

24

Table 4: Success rates in percent (↑) after different active fine-tuning rounds for different demonstration selection strategies. The values reported in the summary Table 2 are marked by a * symbol. Round 5

Round 10

Round 15

±3.3

±2.3

±2.3

54.9±1.3 54.0±3.4 52.2±1.8 56.8±7.6 55.0±2.6 53.4±5.6 53.6±5.1 53.6±2.4 64.0±2.6 59.2±5.5 60.6±2.8 60.1±2.2 59.9±1.4 58.0±5.9 59.2±3.9 60.1±3.8 64.2±3.0 67.1±3.2 62.2±4.9 54.6±0.9 58.1±4.5 59.9±4.7 62.0±0.7 62.6±3.7 61.9±6.3

29.6 24.1±2.7 27.1±2.5 32.6±4.1 31.8±4.1 29.7±3.6 27.4±3.0 24.4±2.4 31.8±5.7 31.4±3.7 31.8±3.4 31.4±4.2 32.4±3.6 26.1±2.0 31.6±7.3 32.1±5.1 28.1±2.9 29.4±1.6 28.3±3.1 25.9±2.9 30.7±5.5 31.0±5.1 29.4±2.4 30.7±0.7 29.7±2.1

41.1 40.6±2.4 41.4±4.2 40.7±8.3 40.7±9.0 41.8±5.7 41.8±6.8 40.3±4.5 44.4±3.8 42.0±1.2 45.6±3.0 46.7±3.8 49.3±2.3 37.7±0.9 44.8±5.3 46.6±1.6 46.0±5.5 44.3±5.0 45.8±2.8 40.9±5.4 44.0±5.2 43.4±5.1 43.2±5.8 45.3±1.7 40.8±3.6

1.0

1.0

0.8

0.8

0.6

0.6

- Pearson

-Spearman

Diversity* Action-L2 (τ = 0) Action-L2 (τ = 1) Action-L2 (τ = 1.5)* Action-L2 (τ = 2) Action-L2 (τ = 2.5) Action-L2 (τ = 3) GU (τ = 0) GU (τ = 1)* GU (τ = 1.5) GU (τ = 2) GU (τ = 2.5) GU (τ = 3) VFD (τ = 0) VFD (τ = 1) VFD (τ = 1.5) VFD (τ = 2) VFD (τ = 2.5)* VFD (τ = 3) VFD (τ = 0, uniform)*3 VFD (τ = 1, uniform) VFD (τ = 1.5, uniform) VFD (τ = 2, uniform) VFD (τ = 2.5, uniform) VFD (τ = 3, uniform)

Round 1

0.4 0.2 0.0

2

3 Round r

4

0.4 0.2

Ensemble Laplace 1

51.1 51.1±3.4 46.9±5.2 51.8±5.7 53.0±5.7 49.0±7.4 46.4±8.3 49.6±5.7 53.0±2.2 57.4±4.3 57.2±3.6 59.3±3.4 57.6±1.7 47.1±0.8 54.8±2.7 53.8±1.4 59.0±2.1 61.1±4.1 59.2±3.2 50.4±3.3 52.6±0.9 56.2±1.1 58.2±2.1 59.3±1.5 55.4±2.9

0.0

5

Ensemble Laplace 1

2

3 Round r

4

5

Figure 9: Ensembling vs. Laplace Approximation. Calibration is measured by the negative Spearman rank correlation (left) and the negative Pearson correlation (right) between the average task success rate and VFD uncertainty. Two separate ensemble members achieve much better calibration than sampling a second model from the last-layer Laplace approximation. the action expert hidden states to the predicted flow-matching action velocity. The Laplace posterior is fit as a regression model on the same training episodes used for that active-learning round: for each calibration frame, we sample a flow-matching time s, a noise x0 ∼ N (0, I), and a target action x1 , and use the corresponding velocity x1 − x0 as the regression target. We use all selected training episodes for the round, a calibration fraction of 1.0, and a diagonal Hessian approximation. Even when limited to the last layer, the computational cost of the last-layer Laplace approximation remains considerable, requiring about 5 hours per checkpoint/round on an NVIDIA RTX 4090 GPU. This is much higher than the 30 minutes it takes to fine-tune a second ensemble member for 4,000 steps. Due to the high cost, we only fit the Laplace approximation for the first 5 rounds. As 25

0.7

Success rate

0.6 0.5

Exploration vs. success =0 =1 = 1.5 =2 = 2.5 Pareto front Round 1 Round 15

0.90

0.95

1.00 0.68 0.64 0.60

0.4

0.56

0.3 0.65 0.70 0.75 0.80 0.85 0.90 0.95 1.00

Normalized task entropy

Figure 10: Pareto front of the exploration-exploitation trade-off.

shown in Figure 9, the two-member ensemble provides much better VFD score calibration than the Laplace approximation. Given also the lower computational cost of ensembling, we consider it the preferred option for posterior sampling in VLAs for uncertainty quantification.

C.4

Iterative Fine-Tuning vs. Retraining

We also test whether our iterative active-learning loop, designed to minimize data-collection effort, requires an additional separate full fine-tuning stage to yield optimal policy performance. To this end, we compare the final-round policy π15 to a policy retrained from the base VLA using a 50/50 (≤15) split between the ensemble training data Dpre and the actively selected data Dnew . For the selection of our best performing temperature τ = 2.5, the retrained policy achieves an overall success rate of 70.6%±0.9 , which is only 3.5% higher than the iteratively fine-tuned policy. This demonstrates that uncertainty-guided acquisition produces a strong policy already during the iterative active learning process.

C.5

Effect of τ on SAVE

Figure 10 shows a detailed view of the trade-off between exploiting the uncertainty estimates and exploring the task space evenly. The exploitation is measured via the success rate of the rolled-out policy, and the diversity of the categorical task distribution is measured by its entropy. We can see that they form a Pareto front: higher temperatures increase the success rate at the cost of diversity. Naturally, this does not hold ad infinitum. In our experiments, we observe that for temperatures τ > 2.5, the success rate starts declining, as too few tasks from the pool are explored (cf. Figure 11), making it harder to maximize the success rate across all tasks.

C.6

Detailed Results on Failure Detection

We provide more detailed results that include all metrics in Figure 12. Compared with baselines specifically developed for failure detection, our general VFD uncertainty estimation method demonstrates strong performance, achieving the highest accuracy (0.67), TPR (0.79), and TWA (0.54), as well as the second lowest detection time of 0.4. These results highlight the potential of VFDbased epistemic uncertainty estimation for online monitoring of VLAs, enabling targeted human intervention or activation of safety fallbacks. 26

Selection =0 VFD =1 VFD = 1.5 VFD =2 VFD = 2.5 VFD =3 VFD = 1.5 Action-L2 =1 GU Diversity

Uncertainty share

1 5

1 5

10 1 5

10 1 5

10 1 5

10 1 5

10 1 5

10 1 5

10 1 5

10 1 5

10 1 5

10 1 5

10 1 5

10 1 5

10 1 5

10 1 5

10 1 5

10

10

No uncertainty scores 1

3

0.0

0.5

5 1.0

7

9

Round r 1.5

11 13 15 2.0

Selected episodes

2.5

3.0

1

3

5

0.00

0.05

7

9

11 13 15

0.10

0.15

0.20

Round r

Uncertainty share

0.25

Figure 11: Behavior of our VFD-acquisition rule for different temperature values τ over all active fine-tuning rounds. Left: Number of selected episodes per task. Right: Relative share of overall uncertainty per task.

27

0.7

0.8 0.6 0.4 0.2

0.9 0.8 0.7 0.6 0.5 0.4 STAC

0.4

TNR

0.5

TPR

0.6

ACE

Detect. Time

0.9 0.8 0.7 0.6 0.5 0.4 0.3 VFD (ours)

RND-OE

Timestep-Wise Acc. 0.55 TWA

True-Negative-Rate

Detection Time

True-Positive-Rate

Accuracy

Detection Accuracy

0.50 0.45

Figure 12: Full results of our failure detection experiments. VFD achieves the highest timestep-wise accuracy (TWA), which measures the capability to detect failures accurately and early.

D

Broader Impact

This work may have positive societal impact by improving the reliability of VLAs in robotic systems. In particular, calibrated epistemic uncertainty can help identify unfamiliar situations, support more data-efficient adaptation, and enable detection of runtime failures before unsafe actions are executed. At the same time, uncertainty estimates are naturally imperfect and may pose risks if deployed as the sole safety mechanism; false confidence could lead to harmful robot behavior, while overly conservative estimates could reduce usability. These risks should be mitigated through conservative threshold design, careful human oversight, safe fallback policies, and rigorous evaluation under realistic deployment conditions before real-world use.

28