1
Continual Learning for Traversability Prediction with Uncertainty-Aware Adaptation
arXiv:2609.17141v1 [cs.RO] 15 Sep 2026
Hojin Lee, Yunho Lee, Daniel A Duecker, and Cheolhyeon Kwon
Abstract—Traversability prediction is a critical component of autonomous navigation in unstructured environments, where complex and uncertain robot-terrain interactions pose significant challenges such as traction loss and dynamic instability. Despite recent progress in learning-based traversability prediction, these methods often fail to adapt to novel terrains. Even when adaptation is achieved, retaining experience from previously trained environments remains a challenge, a problem known as catastrophic forgetting. To address this challenge, we propose a continual learning framework for traversability prediction that incrementally adapts to new terrains using a generative experience recall model. A key virtue of the proposed framework is two folds: i) retain prior experience without storing past data; and ii) incorporate the uncertainty of the generated samples from the recall model, enabling uncertainty-aware adaptation. Real-world experiments with a skid-steering robot validate the effectiveness of the proposed framework, demonstrating its ability to adapt across a series of diverse environments while mitigating catastrophic forgetting. Index Terms—Continual Learning, Planning under Uncertainty, Field Robots, Machine Learning for Robot Control
I. I NTRODUCTION UTONOMOUS navigation in unstructured environments poses significant challenges due to varying terrain traversability, making traversability prediction a central research focus [1]. Traditionally, traversability prediction has been approached by crafting rules based on terrain geometry [2] and semantic context [3]. However, these methods often fail to capture complex robot-terrain interactions. Recently, learning-based methods have offered promising solutions by leveraging driving data without relying on handcrafted traversability rules [4]. While the learning-based traversability prediction methods perform well in conditions similar to the training environment, they often generalize poorly when deployed in novel terrains [5]. To address this limitation, adaptation techniques have been proposed to update traversability models when encountering novel terrain. [6]. However, such adaptation often leads to
A
This work was supported in part by the National Research Foundation of Korea (NRF) grants (RS-2024-00342930 and RS-2020-NR049569), and in part by the InnoCORE program (N10250155(00)), both funded by the Korea government (Ministry of Science and ICT). (corresponding author: Cheolhyeon Kwon) Hojin Lee, Yunho Lee, and Cheolhyeon Kwon are with the Ulsan National Institute of Science and Technology, Ulsan, 44919, Republic of Korea ([email protected], [email protected], [email protected]). Daniel A Duecker is with Munich Institute of Robotics and Machine Intelligence (MIRMI), Technical University of Munich (TUM), Germany ([email protected]).
Fig. 1: Continual adaptation across different types of terrains without access to past data. The robot incrementally updates its model while retaining experience for reliable traversability prediction.
catastrophic forgetting, where newly acquired experience overwrites prior experience, resulting in degraded performance in previously trained environments. To facilitate adaptation without catastrophic forgetting, strategies such as regularization-based continual learning and dynamic architectural methods have been explored [7]. Among these methods, experience replay-based continual learning has been widely adopted in traversability learning, as it enables models to update by reusing past data without requiring complex optimization objectives or architectural changes [8], [9]. These methods incrementally expand memory or selectively store data for experience replay. However, they are often hindered by high memory costs and/or the potential loss of valuable information during data selection. To enable adaptation without losing prior experience and without relying on memory-demanding replay mechanisms, we propose a novel continual learning framework for traversability prediction that features uncertainty-aware adaptation. This framework integrates sensory environmental information and robot state information as input modalities, effectively learning to predict both traversability and its associated uncertainty. A key attribute of the proposed framework is a generative experience recall model, which retrieves prior experience to guide continual updates of both traversability prediction model and itself. In this way, prior experience can be retained without needing memory for storing past data. The generative recall model is further empowered by an uncertainty-aware adaptation mechanism, whereby the uncertainties of recalled samples are evaluated so that trustworthy samples are used to update the recall model. Building on this foundation, the trained traversability prediction model is integrated into a model predictive control (MPC) framework to navigate a robot across a series of diverse environments. These design choices enable a continual learning framework for traversability prediction that exploits a generative experience recall model for effective (i.e., retention of prior
2
TABLE I: Comparison of Learning-based Traversability Prediction Methods with Adaptation Capability. [6] [9] [8] Proposed
Input modality Vision Vision Vision Vision, geometry, dynamics
Adaptation technique Adaptation with recent experience Experience replay Experience replay Generative experience replay
experience), efficient (i.e., no need to store past data), and reliable (i.e., uncertainty-aware) update. Real-world experiments with a skid-steering robot confirm the effectiveness of the proposed framework, showing its adaptability across diverse environments while retaining performance in previously trained environments. The main contributions are as follows: • We propose an uncertainty-aware traversability prediction method that associates terrain features with the robot dynamics to capture complex robot-terrain interactions. • We introduce a continual learning framework leveraging generative experience recall and an uncertainty-aware adaptation mechanism to retain prior experience while adapting to novel environments. • Real-world experiments using a skid-steering robot validate the proposed framework, demonstrating persistent navigation performance while learning uncertain traversability across diverse environments. II. R ELATED W ORK A. Learning-based Traversability Prediction Learning-based methods have emerged as a powerful alternative to traditional rule-based methods in traversability prediction. They rely on real or simulated data to model complex robot-terrain interactions with minimal supervision [2]. A critical design choice in prediction models is the input data modality, which directly impacts the model’s ability to comprehend underlying dynamics. Geometry features are commonly used [2], but they are insufficient to capture semantic terrain characteristics. To address this, the Learning Applied to Ground Vehicles (LAGR) program exerted pioneering effort on employing visual features for traversability prediction [3]. Learning terrain geometry and visual features may not be sufficient, since traversability is substantially influenced by the robot’s state and dynamics. Some methods attempt to exploit proprioceptive sensor data to capture this influence, such as measuring traversability in terms of vibration level [10]. However, they still fall short in fully representing traversability, making it difficult to assess the admissible states for navigating in a given terrain. To comprehend the interdependencies between the robot’s state and terrain features in traversability prediction, [11] encompasses the robot’s speed, inertial sensor data, and terrain visual information to establish a forward kinematics model. This model can predict the robot’s dynamic response on the terrain, interpreting its traversability. A major limitation of this approach, however, is the lack of uncertainty modeling, which degrades the robustness and reliability of traversability predictions. Furthermore, the aforementioned learning-based methods often fail to generalize when deployed in environments that differ from the training conditions. In response to this shortcoming, we propose a continual learningbased framework that combines terrain features and robot state
Prior-experience memory N/A Fixed-sized memory Incremental dynamic memory N/A
Uncertainty modeling Epistemic Aleatoric Aleatoric, epistemic
information for traversability prediction, while also taking into account uncertainty. B. Adaptation for Traversability Prediction Learning-based traversability prediction faces challenges in novel environments that differ from the trained environment. Adaptation methods address these challenges by updating the model with newly collected data, focusing on rapid adaptation to new environments [6]. However, they overlook retaining experience from previously trained environments during updates. This leads to degraded navigation performance when the robot is deployed in previous environments. Experience replay mechanisms have been proposed to address this issue by storing past data in a memory buffer and replaying it alongside newly acquired data [7]. [8] employs an incremental replay buffer that clusters visual features, but its memory footprint grows with environmental diversity, limiting scalability in long-term deployment. In contrast, [9] uses a fixed-size replay buffer, carrying the risk of discarding valuable information when adding new data. These methods present a fundamental trade-off between scalability and representational capacity, making it difficult to retain prior experience across diverse environments. Moreover, neither [8] nor [9] models epistemic uncertainty, which is essential for assessing prediction confidence in unfamiliar environments. They also primarily rely on terrain appearance while neglecting the influence of robot dynamics on traversability, limited in capturing the complexity of robot–terrain interactions. To address the shortcomings in experience replay mechanisms, generative model-based continual learning offers an alternative by synthesizing past data from learned representations, thereby eliminating the need to store past data and supporting scalable adaptation to diverse environments [7]. While offering notable advantages, such methods often fail to account for the uncertainty associated with generated samples, which undermines the prediction model’s updates and potentially compromises performance. To this end, we propose a framework that accounts for the uncertainty of the generated samples and facilitates reliable adaptation to new data and retention of prior experience. To provide a comparative overview, Table I summarizes key aspects of the representative traversability prediction methods with adaptation, including the proposed framework. III. P RELIMINARIES A. Robot Dynamics with Traction Parameters Let us consider a discrete-time dynamics for a ground robot described by: xk+1 = f (xk , uk , ξk ),
3
Fig. 2: Overview of the proposed framework. (a) Deployment phase: uncertainty-aware traversability prediction and navigation. (b) Continual learning phase: uncertainty-aware model adaptation to new environments via generative experience recall.
where xk represents the robot’s state, uk is the control input, and ξk denotes the traction parameter at time step k, accounting for variability in robot-terrain interactions. f is modeled by the following unicycle-based kinematic as it effectively captures the dynamics of a wide range of ground robots, including skid-steering and legged platforms: x x x cmd pk+1 ξk · vk · cos(θk ) pk pyk+1 = py + ∆t · ξkx · vkcmd · sin(θk ) , (1) k θk ξkz · ωkcmd θk+1 where the state vector xk = [pxk , pyk , θk ]⊤ represents robot’s X −Y position in global frame and its yaw angle. The control input uk = [vkcmd , ωkcmd ]⊤ consists of the linear and angular velocity commands, and ∆t > 0 denotes the sampling time. The traction parameters ξk = [ξkx , ξkz ]⊤ ∈ R2 account for the linear and angular discrepancies between the commanded and actual velocities due to uncertain robot-terrain interactions. Without loss of generality, we consider these discrepancies as a measure of traversability [12]. B. Planning with Uncertain Traversability Prediction In unstructured environments with varying traversability, the navigation problem involves determining the optimal control input sequence u∗k:k+T −1 that minimizes the robot’s navigation cost over a receding horizon of length T [12]. Given the state xk , and a navigation cost function C, the receding horizon stochastic optimal control problem can be formulated as: min E[C(x0:T , u0:T −1 )]
u0:T −1
s.t.
x0 = xk , xt+1 = f (xt , ut , ξˆt ), ∀t ∈ {0, · · · , T − 1}
(2)
where ξˆt ∼ P(ξ|xt ) represents the distribution of traction at time step t, which can be predicted by the traversability prediction model. To solve (2), we utilize a sampling-based MPC method, leveraging its parallelizability on GPUs for realtime computation [13]. C. Continual Learning Problem Formulation To reliably codify ξˆ across different environments, we tackle the problem of domain-incremental continual learning for traversability prediction [7]. We define the traversability prediction model T : Rq → R2 × R2>0 as a mapping from input features, comprising the robot’s state and terrain features, to the mean and variance of the Gaussian distributions over traction parameters ξˆ ∼ N (µ(·), σ 2 (·)), where (µ(·), σ 2 (·)) = T (·). The goal is to train a sequence of models T 1 , · · · , T i−1 , such that T i adapts to new environments encountered during the ith deployment, while retaining experience from previous (1st , · · · , i − 1th ) deployments. In this setting, the input distribution (environmental terrain characteristics) may shift over deployments, whereas the underlying task of learning (traversability prediction model) remains consistent. At each deployment phase i, the robot collects data Di in the corresponding environment, forming a sequential dataset D1:i = {D1 , . . . , Di }. Ideally, the model T i would be trained on the full dataset D1:i , but storing and accessing all the past data is impractical on resource-constrained platforms. Instead, our continual learning framework aims to update T i−1 using only Di , with the goal of producing T i that preserves performance without direct access to D1:i−1 . IV. A LGORITHM D EVELOPMENT This section presents the proposed continual learning framework for traversability prediction, empowered by uncertainty-
4
aware adaptation, as depicted in Fig. 2.
the robot’s pose, capturing terrain characteristics over a fixed window for traversability model training and prediction.
A. Self-Supervised Traversability Labeling To train the traversability prediction model, we collect samples of traction parameters ξ while navigating over different terrain environments, including both traversable and non-traversable regions. Inspired by the nonlinear movinghorizon estimator in [4], we utilize odometry to estimate the robot’s state x and use this information to estimate the traction parameters ξ. Specifically, we employ an Extended Kalman Filter (EKF) where the traction parameters are modeled as a random walk process: ξk+1 = ξk + ν. Here, ν is zeromean Gaussian process noise that accounts for gradual variations over time [14]. Using (1) along with state xk+1 , the EKF iteratively refines the traction parameter estimates ξk , establishing datasets that are used to train the traversability prediction model. B. Terrain Feature Processing We use RGB images and 3D point clouds to build a multi-modal terrain representation based on [15]. The point clouds are processed into an elevation grid map, capturing the geometric properties of the terrain. Simultaneously, visual features are extracted from images using a pre-trained DINOv2 backbone [16], producing pixel-wise feature embeddings. However, projecting these high-dimensional visual features (e.g., 384 dimensions for the DINOv2-small model) into a grid map incurs substantial computational and memory overhead, making it impractical for real-time use on mobile robots.
Fig. 3: Grid mapping of the terrain feature ϕ, based on geometric and visual information of the corresponding terrain patches beneath the robot’s four wheels.
To address this, we employ an autoencoder network to compress the high-dimensional visual feature embeddings from DINOv2 into a low-dimensional latent feature. The network is trained in an unsupervised manner on the original dense DINOv2 features from a large-scale off-road dataset by optimizing reconstruction loss. Once trained, the encoder part is deployed for real-time processing, receiving dense features from the DINOv2 backbone and generating compact, lowdimensional visual features to be integrated into the multimodal grid map. As a result, terrain features ϕ ∈ Rp , implicating both geometric and visual information, are assigned to individual grids in the map, as illustrated in Figure 3. These terrain features are stored in a local grid map centered on
C. Robot-terrain Interaction Dataset Based on the labeled traction parameter data, driving data, and terrain feature data, we construct a robot-terrain interaction dataset: D = {(ϕj , vj , ξj ) | j = 1, . . . , B} , where each sample consists of the terrain feature vector ϕj , the robot’s velocity state vj , and the labeled traction parameter ξj at time step j. B denotes the total number of samples in the dataset. The terrain feature vector ϕj = ⊤ ⊤ ⊤ ⊤ ⊤ consists of multiple terrain features ϕj,1 , ϕj,2 , ϕj,3 , ϕj,4 extracted at different contact points. For instance, in a fourwheeled robot, each ϕj,i corresponds to the terrain features at the ith wheel position (pxj,i , pyj,i ) at time step j. The robot’s ⊤ includes the linear and angular velocity state vj = vjx , wjz velocities estimated by odometry. To construct the dataset, we retrieve the stored local terrain grid maps for each recorded robot pose. For each map, we project the trajectory history within a local window and extract terrain features at the wheel contact points along the trajectory. These features are then paired with the corresponding robot velocity and traction estimates to form the dataset. Samples with missing terrain features at any wheel contact point due to being outside the sensor field of view are discarded. D. Uncertainty-aware Traversability Prediction Given the dataset D, the model T is trained to predict the traction parameter as ξˆk = T (ϕk , vk ), where ϕk represents the terrain features, and vk denotes the robot velocity state at time step k. To account for uncertainty, we model T as a probabilistic ensemble model capable of capturing both aleatoric and epistemic uncertainties [17]. Each ensemble member receives the concatenated terrain feature ϕk and velocity state vk , as input and is trained independently. All ensemble members share the same architecture, comprising a multi-layer perceptron with fully connected layers of sizes [16, 32, 32, 16], each followed by a LeakyReLU non-linear activation function. The mth ensemble model Tm outputs the traction parameter as a Gaussian distribution: 2 2 ξˆk,m ∼ N (µk,m , σk,m ), where [µk,m , σk,m ] = Tm (ϕk , vk ).
Based on predictions from M ensemble models, we compute the final prediction as: ξˆk ∼ N (µk , σk2 ), where [µk , σk2 ] = T (ϕk , vk ),
(3)
with the mean and variance computed as: M
µk =
1 X µk,m , M m=1
2 2 σk2 = σk,ale + σk,epi ,
2 2 where the variances from aleatoric σk,ale and epistemic σk,epi uncertainties are respectively given by: M
2 σk,ale =
1 X 2 σ , M m=1 k,m
M
2 σk,epi =
1 X (µk,m − µk )2 . M m=1
5
E. Uncertainty-Aware Adaptation with Experience Recalling To adapt the traversability prediction model across diverse environments, we leverage a generative model–based continual learning framework. The generative model recalls past input–output samples for experience replay without storing raw data. Before delving into the details of our continual learning framework, the initial step is established as follows. 1) Initial training of the traversability prediction model: we begin with the initial model, T 1 , trained on the first environment dataset D1 , which serves as the foundation for adaptation. This initial model T 1 is trained by minimizing the Negative Log-Likelihood (NLL) loss, denoted as LNLL , which captures the likelihood of the estimated traction parameters given the corresponding terrain features and the robot’s dynamics states, i.e., velocity. 2) Initial training of the generative experience recall model: we design a generative recall model R1 , which randomly synthesizes the prediction model’s input and output data, denoted as Drecall . As with the prediction model, the input data consists of two components: terrain features ϕrecall and robot dynamic states, i.e., v recall . Leveraging the fact that velocities are physical quantities with bounded ranges, we design a conditional recall process that first samples velocity and then generates corresponding terrain features conditioned on it. To accomplish this, we adopt a Conditional Variational Autoencoder (CVAE) [18], which consists of an encoder and a decoder, denoted as qenc (z|ϕ, v) and pdec (ϕ|v, z), respectively. Upon training of the CVAE, the decoder is used to recall features ϕrecall conditioned on a sampled velocity state v recall and latent variable z. During recall, both v recall and z are randomly sampled, where v recall respects the velocity limits defined by the robot’s hardware specifications, and z is drawn from a Gaussian prior distribution. Next, to synthesize the output data, the trained traversability prediction model T processes the recalled inputs as follows: ξ recall ∼ N (µrecall , (σ recall )2 ), where [µrecall , (σ recall )2 ] = T 1 (ϕrecall , v recall ). This prediction includes σ recall as a measure of uncertainty, reflecting the model’s confidence in its generated samples. Specifically, a higher σ recall indicates that the recalled sample likely originates from an environment that was not well represented in D1 for training T 1 . 3) Uncertainty-aware traversability prediction model update: we update the traversability prediction model from T i−1 to T i for i ≥ 2, using only the collected dataset Di and the generative recall model Ri−1 . First, we generate a batch of recalled input-output samples using Ri−1 , overwriting the dataset Drecall . These recalled samples are then combined with the newly collected dataset Di to update the model T i by minimizing the following loss function: LT i = LNLL (Di ) + λLadapt (Drecall ),
(4)
where λ is a scaling constant. The first term, LNLL (Di ), captures the experience obtained from newly acquired data. The second term, Ladapt (Drecall ), aims to retain prior experience through recalled samples that imitate past data. This is
achieved by aligning the distributions of the predicted and recalled output distributions using the Jensen–Shannon (JS) divergence as a distance measure, defined as follows: X1 KL T i (ϕrecall , v recall )|ξ recall Ladapt (Drecall ) := 2 D recall 1 + KL ξ recall |T i (ϕrecall , v recall ) , 2 where KL denotes the Kullback-Leibler (KL) divergence. This distributional alignment alleviates catastrophic forgetting of recalled samples and mitigates overfitting to unreliable samples with higher σ recall , thereby reducing the risk of erroneous updates. 4) Uncertainty-aware generative experience recall model update: similar to the prediction model update, we update the generative experience recall model from Ri−1 to Ri using the newly collected dataset Di and recalled dataset Drecall , without explicit access to past data, i.e., Dl−1 , ∀l ≤ i. Specifically, we recall data by randomly sampling input features, denoted as (ϕrecall , v recall ), using Ri−1 . The quality of the recalled samples is assessed using their uncertainty (σ recall )2 , predicted by the preceding prediction model T i−1 . These uncertainty estimates act as proxies for distributional similarity to real data, enabling us to identify and filter recalled samples that deviate from the real data distribution. To this end, we apply a variance threshold τ to exclude uncertain recalls, yielding the refined recall dataset: D̄recall := (ϕrecall , v recall ) σ recall < τ . We then define the augmented dataset as the union of the newly acquired data Di and the filtered recalled samples: Daug := Di ∪ D̄recall .
(5)
The generative experience recall model Ri is trained using Daug , ensuring that only recalled samples with sufficiently low uncertainty contribute to retaining prior experience while recently acquired data facilitates adaptation to the new environment. A comprehensive overview of the proposed continual learning framework 1 with uncertainty-aware adaptation is presented in Algorithm 1. Algorithm 1 Continual Learning Uncertainty-aware Adaptation
Framework
with
Initialize: Set iteration index i = 1. Input: Initial dataset D1 . 1: Initial train T 1 and R1 (Sections IV.E.1 and IV.E.2). 2: while New dataset D i+1 is acquired do 3: Generate recalled dataset Drecall using Ri and T i . 4: Update T i → T i+1 with (4) (Section IV.E.3). 5: Update Ri → Ri+1 using Daug in (5) (Section IV.E.4). 6: i = i + 1. 7: end while 1 Code available at Traversability-Learning.
https://github.com/HMCL-UNIST/Continual-
6
F. Navigation with Traversability Prediction Based on the updated traversability prediction model, the optimal off-road navigation can be formulated as an MPC problem. The MPC minimizes a cost function associated with predicted future states as outlined in Section III-B. Specifically, the objective cost in (2) is defined as: C(xt , µt , σt2 ) = w1 Goal(pxt , pyt ) + w2 σt2 + w3 |1 − µt |, where Goal(pxt , pyt ) encourages progress toward the goal over a finite horizon, and σt2 penalizes regions of high predictive uncertainty stemming from limited environment knowledge and/or irreducible measurement noise. The slip cost term |1−µt | encourages the predicted traction to approach 1, representing ideal alignment between the robot’s actual motion and the commanded input under the nominal dynamics. Notably, µt is not constrained to lie within [0, 1], allowing the model to capture a wider range of robot-terrain interactions. The weights w1 , w2 , w3 , ∈ R+ are tunable scaling factors. To solve for (2), which is subject to stochastic dynamics f , we adopt a mean-propagation strategy for cost approximation, following the method in [17]. Specifically, the robot dynamics are propagated using the mean estimates of uncertain traction parameters. The propagated states are then used to evaluate the cost function and optimize the control inputs. Under this strategy, the navigation problem can be expressed as the following MPC problem: T X
min
u0:T −1
C(xt , µt , σt2 )
t=0
s.t.
x0 = xk , xt+1 = f (xt , ut , µt ), ∀t ∈ {0, · · · , T − 1}
[µt , σt2 ] = T (ϕt , vt ), ∀t ∈ {0, · · · , T − 1}. Upon optimization, only the first control u0 is applied to the robot, while the remaining u1:T −1 are temporarily stored in a buffer and used as fallback actions if the subsequent MPC computation fails to meet the desired control frequency. V. F IELD E XPERIMENT A. Platform and Dataset Description The experiments are conducted on a Clearpath Jackal robot, equipped with an NVIDIA Jetson AGX Orin for onboard computation, an Ouster LiDAR, and a ZED X stereo camera to sense its surrounding environment. A Lidar-inertial odometry [19] provides high-frequency pose and velocity estimates. The ZED X camera captures both RGB images and depth point clouds, which are used to extract the terrain features. The extracted features are encoded into a multi-modal grid map with a resolution of 0.25 meters [15], and represented as latent vectors of dimension p = 3, updated at a rate of 20 Hz. An ensemble of M = 5 models performs traversability prediction. The entire system is built within the ROS 2 environment. The control horizon is set to T = 30 with a time step of ∆t = 0.1 seconds, corresponding to a 3-second prediction window. A batch of 1024 rollouts is used per optimization cycle to support a control frequency of 10 Hz.
Fig. 4: PCA projection of terrain features ϕ from five distinct environments.
The evaluation of continual learning necessitates diverse environments, which enables assessment of how well prior knowledge is retained when adapting to novel environments. In light of this requirement, we consider five environments with distinct terrain characteristics: an asphalt trail in a public park (Dataset D1 ), a gravel terrain near a river shore (Dataset D2 ), a sand area (Dataset D3 ), a bike path (Dataset D4 ), and a forest with dense vegetation (Dataset D5 ). At each site, we collected approximately 10 minutes of data by manually driving through both traversable and non-traversable areas. We apply Principal Component Analysis (PCA) to the extracted terrain features ϕ, projecting them into a two-dimensional space to characterize distributional differences among the five environments, as illustrated in Figure 4. The projected features reveal distinct domain shifts that are pertinent to a domain-incremental continual learning problem, as discussed in Section III-C. B. Evaluation Metrics The traversability prediction model is trained sequentially on datasets D1 through D5 , each corresponding to data collected from Environments 1 through 5. Continual learning performance is evaluated in terms of adaptation and memory retention. Both are computed from the prediction performance of the traction model on test datasets using the Negative LogLikelihood (NLL). Let nk,j ∈ R, k ≤ j denote the NLL on the test set of Dk after training on datasets D1:j . Adaptation performance at the environment j is given by nj,j , which measures how well the updated model T j fits Dj . A lower nj,j indicates better adaptation performance. Memory retention performance is evaluated using the Forgetting Measure (FM) [7], defined as fk,j = max(0, nk,j − mini<j nk,i ), which captures the maximum performance degradation on Dk after training on Dj . A lower FM indicates better experience retention and improved continual learning performance. C. Baseline Methods For comparative analysis, we evaluate the proposed continual learning framework against five baseline methods:
7
Complete-Memory (CM): This upper-bound baseline trains T i using the full dataset Dj , 1 ≤ j ≤ i, enabling batch retraining with complete access to past data. • Incremental-Memory (IMOST) [8]: This method updates T i−1 using an incremental dynamic memory buffer that stores new samples according to an informationexpansion criterion. • Fast Adaptation (WVN) [6]: This baseline updates T i−1 using only the recently acquired data Di , without considering experience retention. • LwF-fashion (LwF) [20]: This method uses the model T i−1 and the recently acquired data Di to retrain T i without accessing past data. Experience retention is achieved via distillation by minimizing the KL divergence between the outputs of T i−1 and T i on Di . • Naive Generative Rehearsal (NGR): A variant of our method that updates T i−1 with recalled samples Ri−1 and the recently acquired data Di without the uncertaintyaware adaptation in Section IV-E4, serving as an ablation of the threshold τ . To ensure a fair comparison, all baseline methods are interfaced to take terrain and robot states as inputs and to output probabilistic traction, as in the proposed framework. For each method, we identify the hyperparameter setting on the Pareto front with equal weighting of adaptation and memory retention. •
2) Evaluation of adaptation performance: as part of the quantitative evaluation, Table II and Table III report the mean and standard deviation of NLL and FM over ten training sessions for each continual learning scenario. The CM method consistently achieves the lowest forgetting, as it keeps access to the full dataset throughout training. Among those without direct access to past data, our proposed method yields the lowest FM, indicating improved retention of prior experience while effectively adapting to new environments. The IMOST method achieves retention performance comparable to, and occasionally exceeding, our method, but this comes at the cost of a continually growing memory requirement. This highlights the effectiveness of our uncertainty-aware adaptation for continual learning without storing past data, retaining prior experience with only the parameters of the prediction and recall models. TABLE II: Adaptation statistics during continual learning (mean ± std). Training from D 1 to D j
NLL, nj,j 1
D →D
2
1
D → D2 → D3 D1 → · · · → D4 D1 → · · · → D5
Test j
D2
D3
D4
D5
CM IMOST WVN Method LwF NGR Proposed
0.17 ± 0.15 0.50 ± 0.26 0.13 ± 0.08 0.19 ± 0.12 0.82 ± 0.34 0.71 ± 0.29
-0.61 ± 0.09 -0.46 ± 0.08 -0.57 ± 0.07 -0.55 ± 0.07 -0.49 ± 0.06 -0.25 ± 0.11
0.89 ± 0.13 1.27 ± 0.07 1.06 ± 0.01 1.03 ± 0.11 1.20 ± 0.21 0.97 ± 0.30
-0.66 ± 0.06 -0.35 ± 0.06 -0.44 ± 0.07 -0.57 ± 0.06 -0.43 ± 0.02 -0.47 ± 0.05
TABLE III: FM statistics during continual learning (mean ± std).
D. Field Experiment Results and Discussion 1) Evaluation of experience recalling: we begin by assessing the validity of recalled samples generated by the generative experience recall model Ri during adaptation. The distributions of the recalled terrain features are visualized on top of the real terrain features using PCA, as shown in Figure 5. While the proposed method produces samples that closely align with the real data distribution, the ablation method (NGR) generates dispersed samples that poorly align with real data. These results highlight that the uncertaintyaware adaptation introduced in Section IV-E4 helps the recall model produce samples that better reflect collected experience, supporting more accurate model updates during continual learning.
Fig. 5: PCA projection of recalled samples alongside real terrain features collected from Environments 1–5.
Training from D 1 to D j
FM, fk,j 1
Test k
Method
D1 → D2
D → D2 → D3
D1 → · · · → D4
D1 → · · · → D5
D1
CM IMOST WVN LwF NGR Proposed
0.60 ± 0.29 0.73 ± 0.10 1.53 ± 0.01 1.60 ± 0.12 0.80 ± 0.05 0.67 ± 0.05
0.57 ± 0.12 1.12 ± 0.12 2.08 ± 0.03 2.24 ± 0.23 1.38 ± 0.08 1.11 ± 0.07
0.70 ± 0.20 1.33 ± 0.11 1.95 ± 0.11 2.06 ± 0.14 1.42 ± 0.06 1.24 ± 0.08
0.76 ± 0.17 1.55 ± 0.24 2.56 ± 0.03 2.68 ± 0.16 1.82 ± 0.21 1.45 ± 0.13
D2
CM IMOST WVN LwF NGR Proposed
– – – – – –
0.12 ± 0.12 0.38 ± 0.25 2.17 ± 0.32 2.33 ± 0.34 0.55 ± 0.15 0.38 ± 0.27
0.20 ± 0.14 0.67 ± 0.19 2.28 ± 0.39 2.26 ± 0.30 0.68 ± 0.14 0.68 ± 0.18
0.22 ± 0.09 1.03 ± 0.51 2.29 ± 0.19 2.72 ± 0.18 1.22 ± 0.37 0.98 ± 0.23
D3
CM IMOST WVN LwF NGR Proposed
– – – – – –
– – – – – –
0.35 ± 0.17 1.24 ± 0.23 3.07 ± 0.33 2.85 ± 0.11 1.39 ± 0.14 0.93 ± 0.21
0.49 ± 0.20 1.51 ± 0.22 3.18 ± 0.25 3.50 ± 0.45 2.13 ± 0.19 1.49 ± 0.31
D4
CM IMOST WVN LwF NGR Proposed
– – – – – –
– – – – – –
– – – – – –
0.17 ± 0.12 0.36 ± 0.15 1.60 ± 0.21 1.52 ± 0.17 0.29 ± 0.11 0.25 ± 0.16
Apart from the quantitative findings, we qualitatively evaluate the traversability prediction model’s ability to retain prior experience through navigation trials. Specifically, we assess the performance of model T 5 in Environments 1 after sequential training on Environments 1 through 5 (i.e., D1 → · · · → D5 ). To probe the impact of uncertainty-aware navigation under continual learning, the grass region in the park (treated as an OoD area) was excluded from data collection, while only the asphalt region (in-distribution) was traversed during training. Figure 6 illustrates representative trajectories
8
from three trials per method, and Table IV summarizes the navigation results. Here, Goal reached denotes the number of successful goal completions out of three trials, and OoD avoidance indicates the number of trials in which the robot successfully avoided entering the grass region. The WVN method fails to reach the goal in one trial, reflecting degraded predictions due to forgetting. Other baselines, except CM and the proposed method, also fail to retain prior experience and incorrectly estimate uncertainty in the OoD region, resulting in poor avoidance performance. In contrast, the proposed method demonstrates stronger experience retention, reliably reaching the goal while avoiding OoD areas. Despite being trained without access to past data, it performs comparably to the CM method. These results demonstrate that our method effectively retains prior experience, enabling reliable navigation under continual learning scenarios.
Fig. 6: Navigation results in Environment 1 using the traversability prediction model T 5 , with the paths of each method overlaid on the field images.
TABLE IV: Navigation results across 3 trials per method. Values indicate the number of successful trials. Method Metric Goal reached OoD avoidance
CM IMOST WVN LwF NGR Proposed 3 3
3 0
2 0
3 0
3 1
3 3
VI. C ONCLUSIONS This paper presents a novel continual learning framework for traversability prediction in unstructured environments, addressing the challenges of adaptation and catastrophic forgetting without requiring explicit storage of past data. The core contribution is an uncertainty-aware adaptation strategy that selectively updates both the traversability prediction model and the generative experience recall model based on uncertainty estimates of recalled experiences. Real-world experiments with a skid-steering robot validate the effectiveness of the proposed method, demonstrating improved experience retention and reliable navigation performance. Future work will investigate more generalizable learning strategies that leverage prior knowledge, such as physics-informed foundation models, for traversability prediction. R EFERENCES [1] Christos Sevastopoulos and Stasinos Konstantopoulos. A survey of traversability estimation for mobile robots. IEEE Access, 10:96331– 96347, 2022.
[2] Juhana Ahtiainen, Todor Stoyanov, and Jari Saarinen. Normal distributions transform traversability maps: lidar-only approach for traversability mapping in outdoor environments. Journal of Field Robotics, 34(3):600– 621, 2017. [3] Dongshin Kim, Jie Sun, Sang Min Oh, James M Rehg, and Aaron F Bobick. Traversability classification using unsupervised on-line visual learning for outdoor robot navigation. In Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006., pages 518–525. IEEE, 2006. [4] Mateus V Gasparino, Arun N Sivakumar, Yixiao Liu, Andres EB Velasquez, Vitor AH Higuti, John Rogers, Huy Tran, and Girish Chowdhary. Wayfast: Navigation with predictive traversability in the field. IEEE Robotics and Automation Letters, 2022. [5] Xiaoyi Cai, James Queeney, Tong Xu, Aniket Datar, Chenhui Pan, Max Miller, Ashton Flather, Philip R Osteen, Nicholas Roy, Xuesu Xiao, et al. Pietra: Physics-informed evidential learning for traversing out-ofdistribution terrain. IEEE Robotics and Automation Letters, 2025. [6] Jonas Frey, Matias Mattamala, Nived Chebrolu, Cesar Cadena, Maurice Fallon, and Marco Hutter. Fast Traversability Estimation for Wild Visual Navigation. In Proceedings of Robotics: Science and Systems, Daegu, Republic of Korea, July 2023. [7] Liyuan Wang, Xingxing Zhang, Hang Su, and Jun Zhu. A comprehensive survey of continual learning: Theory, method and application. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024. [8] Kehui Ma, Zhen Sun, Chaoran Xiong, Qiumin Zhu, Kewei Wang, and Ling Pei. Imost: Incremental memory mechanism with online self-supervision for continual traversability learning. In 2025 IEEE International Conference on Robotics and Automation (ICRA), pages 8788–8794. IEEE, 2025. [9] Hyung-Suk Yoon, Ji-Hoon Hwang, Chan Kim, E In Son, Se-Wook Yoo, and Seung-Woo Seo. Adaptive robot traversability estimation based on self-supervised online continual learning in unstructured environments. IEEE Robotics and Automation Letters, 2024. [10] Lorenz Wellhausen, Alexey Dosovitskiy, René Ranftl, Krzysztof Walas, Cesar Cadena, and Marco Hutter. Where should i walk? predicting terrain properties from images via self-supervised learning. IEEE Robotics and Automation Letters, 4(2):1509–1516, 2019. [11] Anuj Pokhrel, Mohammad Nazeri, Aniket Datar, and Xuesu Xiao. Cahsor: Competence-aware high-speed off-road ground navigation in se(3). IEEE Robotics and Automation Letters, 2024. [12] Xiaoyi Cai, Siddharth Ancha, Lakshay Sharma, Philip R Osteen, Bernadette Bucher, Stephen Phillips, Jiuguang Wang, Michael Everett, Nicholas Roy, and Jonathan P How. Evora: Deep evidential traversability learning for risk-aware off-road autonomy. IEEE Transactions on Robotics, 2024. [13] Hojin Lee, Junsung Kwon, and Cheolhyeon Kwon. Learning-based uncertainty-aware navigation in 3d off-road terrains. In 2023 IEEE International Conference on Robotics and Automation (ICRA), pages 10061–10068. IEEE, 2023. [14] Michael Bloesch, Marco Hutter, Mark A Hoepflinger, Stefan Leutenegger, Christian Gehring, C David Remy, and Roland Siegwart. State estimation for legged robots: Consistent fusion of leg kinematics and imu. 2013. [15] Gian Erni, Jonas Frey, Takahiro Miki, Matias Mattamala, and Marco Hutter. MEM: Multi-Modal Elevation Mapping for Robotics and Learning. In 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 11011–11018, 2023. [16] Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193, 2023. [17] Hojin Lee, Taekyung Kim, Jungwi Mun, and Wonsuk Lee. Learning terrain-aware kinodynamic model for autonomous off-road rally driving with model predictive path integral control. IEEE Robotics and Automation Letters, 8(11):7663–7670, 2023. [18] Kihyuk Sohn, Honglak Lee, and Xinchen Yan. Learning structured output representation using deep conditional generative models. Advances in neural information processing systems, 28, 2015. [19] Kenny Chen, Ryan Nemiroff, and Brett T Lopez. Direct lidar-inertial odometry: Lightweight lio with continuous-time motion correction. In 2023 IEEE international conference on robotics and automation (ICRA), pages 3983–3989. IEEE, 2023. [20] Zhizhong Li and Derek Hoiem. Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence, 40(12):2935– 2947, 2017.