arXiv CSopen access

Self-Healing Visual Recovery for Autonomous Ground Vehicles Using Camera-Only Visual Odometry

Self-Healing Visual Recovery for Autonomous Ground Vehicles Using Camera-Only Visual Odometry JAKOB SOLBERG BERNTZEN1,2 , SAFIA FATIMA1,2 , AND LEON MOONEN1 (Member, IEEE) 1

2

Simula Research Laboratory, Oslo, Norway University of Oslo, Oslo, Norway

Corresponding author: Leon Moonen (e-mail: [email protected]).

arXiv:2607.11686v1 [cs.RO] 13 Jul 2026

This work was supported in part by the Research Council of Norway through the cureIT project (grant #300461).

ABSTRACT Low-cost unmanned ground vehicles are often used in indoor places like warehouses, inspection corridors, and farm rows, where painted floor lines guide the robot. Line following is useful because it only needs one camera and little computing power, but it can fail when the line is blocked or turns sharply and goes out of view. Sensor-rich platforms tolerate this through hardware redundancy (LiDAR, GPS, multiple cameras), but camera-only systems must recover at runtime with no additional infrastructure. This paper presents a lightweight, two-stage recovery approach that restores guideline tracking without LiDAR, GPS, or a GPU. When the line is lost, the robot first turns in place while slowly relaxing its color checks and waiting for confirmation across multiple frames (Stage 1). If the line is still not found, monocular visual odometry moves the robot back to saved breadcrumb positions before it tries again (Stage 2). The system uses a depth-gated HSV line tracker, a YOLOv8n obstacle detector, and a visual odometry breadcrumb mapper, and it runs at 20 Hz on CPU-only hardware. The controller embeds a complete MAPE-K loop within a single 50 ms control tick, with no external adaptation manager required. The approach is evaluated across 119 fault-injected episodes on three Webots simulation courses. The method was successful in 86.6% of cases, with a median recovery time of 3.26 seconds. These results demonstrate that reliable visual recovery is feasible on camera-only UGVs within practical cost and computational limits. INDEX TERMS Autonomous ground vehicle, self-adaptive systems, line following, visual odometry, obstacle avoidance, fault tolerance, low-cost robotics, YOLOv8, depth-gated perception, Webots simulation. I. INTRODUCTION

A

UTONOMOUS ground vehicles (UGVs) are used in a growing range of applications, including logistics, industrial inspection, and small-scale agriculture, where continuous and unattended operation is valuable [1, 2]. Falling component costs and persistent labor shortages have moved these vehicles from laboratory prototypes into routine field use for tasks such as crop monitoring, transport, and inspection, where reliable operation must be maintained under real-world conditions [1, 2]. Section III examines this demand and the case for low-cost autonomy in more detail. High-end UGVs deployed in demanding sectors such as space exploration, military reconnaissance, and precision agriculture rely on redundant sensor suites that may include LiDAR, RADAR, GPS, multiple cameras, and inertial measurement units, together with sophisticated sensorfusion architectures for localization and fault detection [2– 4]. These configurations deliver reliable navigation in dyThis work is licensed under a Creative Commons Attribution 4.0 International (CC BY 4.0) license.

namic and unstructured environments, but their substantial acquisition cost, integration complexity, and maintenance requirements place them beyond the financial reach of many small-to-medium enterprises and developing-market operators [5, 6]. Many sectors, including warehouse logistics, floorguided inspection, and small-scale agriculture, depend on repetitive, labor-intensive tasks that could benefit greatly from automation but cannot justify the cost of sensorrich platforms [5]. Guideline following is a navigation paradigm particularly well suited to these deployments, a painted or taped floor line provides a deterministic reference with minimal infrastructure cost, and a robot can track it using only a single camera and limited onboard processing [7]. Line following is therefore widely adopted in warehouse automation, conveyor-guided inspection robots, and agricultural row-following systems, where structured layouts make such guidance practical and costeffective. Copyright © 2026 held by the author(s).

1

J. S. Berntzen, S. Fatima, and L. Moonen: Self-Healing Visual Recovery for Autonomous Ground Vehicles . . .

Camera-first UGVs provide a cost-effective alternative by relying on a single RGB-D sensor alongside modern computer vision and deep learning techniques. Recent advances in lightweight object detection [8, 9], monocular depth estimation [10, 11], and visual odometry (VO) [12, 13] have made camera-only platforms increasingly practical in structured indoor environments. Even so, an important resilience gap remains: how can a camera-only system detect navigational failure, specifically the complete loss of its primary visual reference, and recover autonomously without external positioning aids? The failure modes of line-following systems arise from three principal sources: (i) partial or complete occlusion of the guide line by obstacles or debris, (ii) strong reflections, lighting changes, or worn markings that degrade color segmentation reliability, and (iii) heading deviations induced by obstacle avoidance maneuvers that carry the line beyond the camera field of view. Each failure mode demands a different recovery response, a brief heading correction may suffice for a partial occlusion, while complete line loss requires a systematic search or autonomous navigation back to a previously observed pose [14, 15]. Without an effective recovery mechanism, even a single unrecovered failure halts the robot until a human intervenes, undermining the economic justification for autonomous deployment. Self-healing systems research [16, 17] characterizes this failure mode as a runtime recovery challenge, requiring the system to detect deviations from expected behavior and apply corrective actions autonomously without human intervention. The MAPE-K (Monitor-Analyze-Plan-ExecuteKnowledge) architectural pattern [16, 18] has been adapted for robot navigation and service continuity [5, 19], but CPU-only, single-camera platforms cannot easily support the external adaptation managers or runtime environment models typically assumed by full MAPE-K implementations. Existing recovery methods rely either on costly sensors for global localization [4, 20] or on deep learning pipelines that require dedicated GPU compute [21, 22]. By contrast, lightweight alternatives that run on a single CPU remain underexplored. This is especially true for guideline following, a control paradigm widely used in warehouse automation, floor-guided inspection robots, and agricultural row-following platforms where low hardware cost is paramount. The main contributions of this paper are as follows: 1) We present a unified perception-control-recovery loop that operates at 20 Hz on CPU-only hardware, without relying on GPU acceleration. 2) We develop a depth-gated HSV line tracker that integrates online hue adaptation via exponential moving average (EMA) with per-row floor modeling and periodic SVD-based plane fitting to reduce false positives on reflective or cluttered surfaces. 3) We propose a depth-fused obstacle avoidance module that combines geometric residuals from a learned 2

floor model with YOLOv8n bounding-box detections, prioritized by median depth within a forward image corridor. 4) We introduce a two-stage visual recovery strategy: first, an in-place spin-and-search procedure with relaxed thresholds and strict multi-frame confirmation; second, VO-guided navigation to stored breadcrumb poses when the initial search fails. 5) We conduct an empirical evaluation across 119 induced line-loss episodes on three geometrically diverse courses. The rest of the paper is organized as follows. Foundational concepts are introduced in Section II. Related work is reviewed in Section III. The system design is presented in Section IV, and the experimental setup is described in Section V. Results and discussion are presented in Section VI, threats to validity in Section VII, future directions in Section VIII, and the conclusion in Section IX. II. BACKGROUND A. AUTONOMOUS GROUND VEHICLE NAVIGATION

Autonomous robots execute a continuous sense–decide–act cycle: sensors capture the environment, an onboard decision process selects an appropriate action, and actuators carry it out [23]. Because sensors are imperfect and actuation is noisy, modern robotic systems adopt a probabilistic view that models both state and observation uncertainty explicitly. Bayesian filtering provides the core mechanism, a prior belief over the vehicle state is updated at each timestep by combining a motion model p(xk | xk−1 , uk ) with a sensor observation model p(zk | xk ), yielding a posterior that degrades gracefully rather than failing catastrophically when sensor data is sparse or noisy. This probabilistic formulation is especially valuable for lowcost platforms, where individual sensors may be unreliable. For line-following UGVs, the relevant state is the lateral offset and heading relative to the guide line. When this visual reference is available, a proportional-derivative (PD) controller is sufficient for stable tracking. When the reference signal is lost (for example, in the absence of a fallback mechanism), the controller no longer receives a valid input. As a result, the vehicle may come to a stop or drift indefinitely. Self-healing behavior therefore requires the system to detect loss autonomously, execute a recovery strategy, and reacquire the reference without human intervention. B. VISUAL ODOMETRY

Visual odometry (VO) estimates incremental camera motion from a sequence of images without GPS or wheel encoders [12, 24]. A typical monocular VO pipeline consists of three stages: (i)feature detection, using detectors such as GFTT or ORB to identify salient keypoints, (ii)feature matching across frames, with KNN and Lowe’s ratio test to filter ambiguous correspondences, and (iii) motion recovery, using a five-point algorithm inside RANSAC to

J. S. Berntzen, S. Fatima, and L. Moonen: Self-Healing Visual Recovery for Autonomous Ground Vehicles . . .

solve for the essential matrix E, which encodes the relative rotation R and translation direction t̂ between frames. A key limitation is drift, as each pose estimate is conditioned on the previous one, small errors accumulate over time and push the trajectory away from the true path, especially in low-texture or repetitive environments. Full SLAM systems correct drift via loop closure and bundle adjustment [4], but these operations are too costly for CPU-only, real-time control. For short-range navigation (under 5 m), however, a lightweight GFTT along with ORB and five-point pipeline provides sufficient pose accuracy without a heavy SLAM back-end, making it well suited to the breadcrumb-based recovery strategy described in this paper. C. MAPE-K SELF-ADAPTIVE ARCHITECTURE

The MAPE-K architectural pattern [16, 17] follows runtime adaptation using four sequential stages operating over a shared knowledge base: Monitor collects system and environment observations, Analyze detects deviations from expected behavior, Plan selects a corrective strategy, and Execute applies the chosen action through effectors. In canonical deployments, these stages are implemented as a separate adaptation manager that communicates with managed components through well-defined interfaces, allowing the adaptation logic to be independently developed and verified [18]. For embedded robotics on CPU-only hardware, a separate adaptation manager adds too much latency and resource overhead. The controller in this paper instead runs the MAPE-K loop in line with the control cycle, rather than through an external manager. This gives up the modularity of a full MAPE-K deployment, but keeps latency and resource use low enough for the low-cost UGV hardware targeted here. Section IV describes how the four control phases carry out the MAPE-K loop. III. RELATED WORK A. INDUSTRIAL DEMAND AND THE CASE FOR AFFORDABLE AUTONOMY

The growth of service and industrial robotics creates strong economic pressure to reduce the cost of autonomous systems. The IFR World Robotics 2025 report puts the worldwide operational stock of industrial robots at 4.66 million units in 2024, a 9% rise over the previous year, with 542,000 new units installed [25]. In the same year, nearly 200,000 professional service robots were sold, again up 9%, of which transportation and logistics accounted for the largest share at 102,900 units (up 14%) [26]. Robot density remains high in leading markets, reaching 1,220 units per 10,000 employees in the Republic of Korea, 818 in Singapore, and 449 in Germany [27]. This expansion is driven not only by large manufacturers but increasingly by small and medium enterprises seeking to automate repetitive tasks at lower capital investment [5, 6].

High-end UGVs achieve robustness through sensor redundancy and hardware diversity, but their cost and complexity limit adoption [2, 28]. Camera-based platforms address this gap by replacing expensive sensors with computer vision and machine learning [29]. The challenge that remains is reliability, unlike sensor-rich systems that tolerate individual sensor failures through redundancy, camera-first systems must handle failure modes through algorithmic resilience rather than hardware redundancy. This motivates the design of embedded recovery strategies that can operate entirely on commodity CPU hardware. B. VISUAL ODOMETRY AND SLAM

Visual odometry (VO) estimates camera motion by tracking feature correspondences across successive frames [12, 24]. The field can be broadly divided into two families of approaches based on how they extract geometric information from images. Feature-based visual SLAM has been a central research direction in robot perception because it offers an interpretable pipeline for estimating motion and building a sparse map from images. These methods detect salient keypoints (for example, FAST corners) and compute compact descriptors (e.g., ORB) that can be matched across frames using distance metrics and ratio tests to infer interframe correspondences. A landmark system in this line of work is PTAM [30], which demonstrated real-time monocular SLAM by splitting tracking and mapping into parallel threads, enabling a stable front-end pose estimate while the back-end incrementally optimizes a keyframe map. Building on this architecture, the ORB-SLAM family [31, 32] introduced more robust relocalization, loopclosure detection using bag-of-words techniques [33], and support for monocular, stereo, and RGB-D sensing. ORBSLAM3 further extended the framework to visual–inertial configurations and a multi-map design, establishing a widely used baseline for feature-based V-SLAM [32]. Since relative pose estimates inevitably accumulate error over long trajectories, state-of-the-art feature-based systems reduce drift through global bundle adjustment and pose-graph optimization in factor-graph back-ends [4, 34]. Figure 1 summarizes the typical outputs of such a pipeline, including tracked keypoints and the resulting estimated camera trajectory. Direct methods avoid explicit keypoint extraction and instead minimize photometric alignment error over highgradient pixels, making them accurate over short baselines when the brightness constancy assumption holds [13, 35]. LSD-SLAM [35] extended this approach to large scale, semi-dense mapping, while Direct Sparse Odometry (DSO) [13] added a fully photometric calibration model and joint optimization of poses and inverse depths. Direct methods can be very precise at short ranges but are more sensitive to lighting changes and motion blur than feature based alternatives. Visual-inertial odometry (VIO) tightly couples image 3

J. S. Berntzen, S. Fatima, and L. Moonen: Self-Healing Visual Recovery for Autonomous Ground Vehicles . . .

sufficient accuracy for this purpose without the overhead of a full SLAM back-end or a GPU-dependent learned front-end. C. DISPLACEMENT RECOVERY AND FAULT-TOLERANT NAVIGATION

FIGURE 1. Feature based Visual SLAM: tracked keypoints (green) overlaid on the image and the estimated camera trajectory (blue) accumulated from matched feature correspondences across keyframes. Drift accumulates along the trajectory unless corrected by loop closure and bundle adjustment.

measurements with IMU pre-integration to stabilize motion estimates during rapid turns, aggressive maneuvers, or weak texture regions, and to recover metric scale from monocular input [36]. OKVIS [37] introduced the tightly coupled non-linear optimization formulation, while VINS-Mono [38] demonstrated robust monocular VIO on embedded platforms using a sliding-window smoother and loop closure. These systems substantially improve robustness at the cost of requiring an IMU, which adds hardware and calibration complexity. Learned front-ends replace hand crafted feature descriptors with differentiable correspondence networks. DROID-SLAM [22] formulates tracking and mapping as iterative updates to a dense flow field using recurrent neural networks, while DPVO [39] extends this to sparse patch correspondences for higher throughput. Surveys of deep V-SLAM report a growing trend to combine learned place recognition with geometric back-ends for better performance under strong appearance change [40], but the GPU requirement of these networks precludes their use on low-cost CPU-only platforms. Most V-SLAM systems separate the estimation problem into a front end and a back end [4]. The front end extracts geometric constraints, the back end optimizes a factor graph using bundle adjustment and pose-graph optimization, with incremental solvers such as iSAM2 [34] exploiting sparsity to update solutions efficiently as new observations arrive. In practice, the choice of front-end family involves a trade-off: feature based methods handle large viewpoint changes and support strong loop closure while direct methods are precise at short baselines but sensitive to lighting. VIO adds short-term stability at the cost of additional hardware and learned front-ends improve robustness at the cost of GPU inference. Two-stage recovery in this work requires only shortrange VO (typically under 5 m of travel to a stored pose). A GFTT along with ORB and five-point pipeline provides 4

Displacement recovery refers to cases in which a robot is physically moved while its internal state estimate remains unchanged, causing the belief to become confident but inconsistent with actual sensor observations [20, 23]. The system must quickly detect this inconsistency and restore a correct global pose before normal operation can resume. Probabilistic approaches address kidnapping through adaptive particle filters [20, 41] that expand or contract the particle population based on the correspondence between predicted and observed measurements. When sensor-state consistency drops below a threshold, the filter injects additional hypotheses to cover a wider region of the state space, then prunes back once a consistent pose is found. This approach gracefully handles sudden localization failures without requiring a global map query, but its scalability is limited by the number of particles needed to cover highdimensional state spaces. Appearance-based place recognition is widely used in visual SLAM to generate candidate poses for recovery after tracking failure. In bag-of-words methods such as Galvez’s approach [33], local image descriptors are quantized into a learned visual vocabulary, and keyframes are represented by histograms over these visual words. This representation enables fast retrieval of keyframes with similar appearance from large databases, providing effective relocalization hypotheses for subsequent geometric verification. FABMAP [42] extends this idea with a probabilistic generative model of visual scene appearance, which explicitly accounts for the probability of observing visual words given a particular location and improves robustness under lighting and viewpoint change. Retrieved candidates are geometrically verified using PnP estimation or a fivepoint essential matrix solver inside RANSAC before being accepted, and successful relocalization poses are fused into the SLAM back-end through pose-graph optimization [31]. Geometry-based descriptors provide an alternative when visual appearance is ambiguous or repeat-textured. ScanContext [43] encodes the spatial distribution of a LiDAR point cloud into a compact 2D array indexed by range and angle, enabling efficient place retrieval without the sensitivity to lighting that affects appearance-based methods. SegMatch [44] further segments the 3D scene into objects and matches segments across different traversals, reducing the impact of dynamic changes. These descriptors reduce the candidate search space through geometric consistency checks before expensive alignment, speeding up recovery and lowering false-positive rates in repetitive environments. Learning-based relocalization offers a complementary strategy. Regression models such as PoseNet [45] directly

J. S. Berntzen, S. Fatima, and L. Moonen: Self-Healing Visual Recovery for Autonomous Ground Vehicles . . .

predict a 6-DOF camera pose from a single image using a convolutional network trained on a specific scene, providing a coarse absolute-pose estimate in constant time without an explicit map query. Retrieval models such as NetVLAD [46] encode image content into a compact global descriptor optimized for place retrieval, producing strong candidates that are then geometrically refined. Both approaches require GPU inference and scene-specific training, limiting their applicability to the camera-only, CPU-only target of this work. Prior work has also demonstrated embedded visual recovery on resource-constrained hardware without these heavier components. Work in [14] reported 87% qualitative success and a Mean Time to Recovery (MTTR) of approximately 3.23 s on an ARM11 processor. Another system combined appearance cues with motion primitives, reducing MTTR by roughly half relative to Monte Carlo Localization [15]. More recently, deep learning has been applied to failure detection in autonomous driving, improving robustness under distribution shift while requiring GPU inference [21]. These results confirm that competitive recovery performance is achievable at lower hardware tiers, motivating the approach presented in this paper. These recovery stages form a MAPE-K loop [16–18]; Section IV describes how the control phases realize its stages. Probabilistic model checkers such as PRISM [47, 48] can verify recovery guarantees under abstracted environment models, but constructing such models is too costly for real-time control loops; this work therefore relies on empirical MTTR and success-rate metrics. D. LIGHTWEIGHT AI FOR ROBOTIC PERCEPTION

Deep learning has substantially broadened the capabilities of vision-based robotic systems by providing both semantic understanding and geometric cues at low latency. Object detection networks such as YOLOv7 and YOLOv8 formulate detection as a single grid-based regression problem, dividing the input image into a spatial grid and directly predicting bounding boxes and class probabilities for each cell [8, 9]. This unified formulation avoids the two-stage region-proposal overhead of earlier detectors and achieves real-time throughput competitive with slower, more accurate architectures. These models are typically trained on large-scale datasets such as COCO [49], which provides 80 object categories and over 330,000 images, giving the network the visual diversity needed for robust generalization. Exporting Ultralytics models to ONNX format and deploying them via the OpenCV DNN backend enables CPU-only inference on ARM and x86 platforms without requiring a dedicated GPU or proprietary acceleration library [50]. A practical demonstration of this approach is a monocular obstacle avoidance pipeline that exports a RetinaNet-50 model to ONNX and compiles it with TensorRT on Jetson-class hardware, confirming that deep detection models can

be adapted for embedded platforms under constrained resources [50]. Semantic segmentation networks parse the full image at the pixel level and are useful for identifying floor regions, free space, and obstacles simultaneously. Lightweight architectures designed for embedded deployment include ENet [51], which reduces computation through early downsampling and asymmetric convolutions; BiSeNet [52], which maintains spatial resolution through a bilateral branch structure; and Fast-SCNN [53], which achieves realtime throughput through a shared feature extraction layer and a lightweight decoder. These networks enable robots to interpret scene layout continuously without overwhelming limited onboard processing budgets, making them suitable for floor detection and free-space mapping on mobile platforms. Monocular depth estimation provides complementary range information when a dedicated depth sensor is unavailable or when its accuracy is insufficient for fine-grained tasks. Monodepth2 [10] uses self-supervised training from stereo pairs or monocular sequences with a novel masking strategy to handle moving objects, while MiDaS [11] trains on a diverse mixture of depth datasets to produce relative-depth estimates that generalize across scene types. These methods are primarily useful for tasks that require qualitative depth ordering rather than metric accuracy. In this paper, the system uses a depth sensor directly for metric range measurements, so monocular depth estimation is not needed. YOLOv8n (the nano variant) is used as a semantic complement to a geometric depth-residual obstacle detector. The two detection modalities are fused during the Analyze phase, and candidate obstacles are ranked by their median depth within the forward image corridor, giving priority to the closest detected object regardless of whether it was identified by the geometric or the semantic channel. E. SELF-ADAPTIVE SYSTEMS

Self-adaptive systems are designed to modify their own behavior at runtime in response to changes in the environment or in the system itself, without human intervention [16, 17]. The MAPE-K architectural pattern [16, 18] is the most widely adopted framework for organizing this adaptation: Monitor observes system state and environmental conditions; Analyze detects deviations from expected behavior using the monitored data; Plan selects and sequences corrective strategies; and Execute applies the chosen plan through actuators or configuration changes, all over a shared Knowledge Base that persists runtime state across loop iterations. The Rainbow framework [17] was one of the first platforms to realize MAPE-K in software systems, introducing the concept of an external adaptation manager that governs managed application components through a model-based interface. Subsequent work in the robotics domain has adapted this structure to navigation and service-continuity 5

J. S. Berntzen, S. Fatima, and L. Moonen: Self-Healing Visual Recovery for Autonomous Ground Vehicles . . .

scenarios, demonstrating that self-adaptive loops can improve predictability, enhance safety, and strengthen operational resilience during runtime faults [19]. A key challenge in applying MAPE-K to robotic platforms is latency: conventional MAPE-K deployments assume asynchronous, loosely-coupled adaptation that is unsuitable for tightlytimed control loops running at 20–50 Hz. Recovery effectiveness in self-adaptive systems is typically quantified using time-to-recovery and recoverysuccess-rate metrics [19, 54]. These capture both responsiveness (how quickly recovery occurs) and reliability (how consistently it succeeds). We use the same metrics to evaluate our approach, and discuss them in more detail in Section V-F. Probabilistic model checkers such as PRISM [47, 48] can verify recovery guarantees by abstracting the adaptive system as a Markov decision process and checking temporal logic properties over all reachable states. Because the recovery algorithm in this paper uses explicit stage budgets, timeouts, and confirmation thresholds, it is structurally well suited to such formal analysis. F. RESEARCH GAP

Existing recovery methods typically rely either on redundant sensing modalities, such as LiDAR, GPS, and RADAR [4, 20], or on GPU-class hardware for real-time inference [21, 32]. Neither requirement is well suited to the growing class of low-cost, camera-first UGVs used in logistics and inspection. Although prior camera-only systems [14] have achieved comparable recovery performance, they do not incorporate onboard depth-obstacle fusion or VO-based breadcrumb navigation, which makes them more vulnerable to complete line loss in geometrically complex environments. To the best of our knowledge, no prior work addresses all three constraints simultaneously: (i) camera-only sensing without LiDAR, RADAR, or GPS; (ii) CPU-only computation without GPU acceleration; and (iii) autonomous line-loss recovery using VO-guided return to stored poses. The proposed algorithm operates under all three constraints while achieving recovery accuracy comparable to that of heavier prior systems. Table 1 maps these gaps against representative related work, spanning a lightweight camera-only recovery system [14], a deep-learning failure detector [21], and a full visual SLAM pipeline [32]. This is a qualitative comparison of capabilities; the SLAM and deep-learning systems are included because they represent the heavier alternatives this work aims to avoid, even though they do not report line-loss recovery times. IV. PROPOSED APPROACH

The overview of the proposed system is illustrated in Figure 2. Each 50 ms control tick proceeds through four sequential phases that map directly onto a MAPE-K selfadaptive loop [16]: Sensor Ingestion (Monitor), State Estimation (Analyze), Decision Layer (Plan), and Ac6

TABLE 1. Capability analysis: proposed approach vs. relevant related work consisting of a lightweight camera-only recovery system [14], a deep- learning-based failure detector [21], and a full visual SLAM pipeline [32]. Legend: ✓= feature present, ◦ = partial, × = absent.

Proposed

[14]

[21]

[32]

✓ ✓ ✓ ✓ ✓ ✓

✓ ◦ ✓ × × ✓

× × ✓ × × ✓

◦ ✓ × × × N/A

CPU-only No LiDAR/GPS Line-loss recovery VO breadcrumbs Depth obstacle fusion >85% recovery rate

tuation (Execute). A shared Knowledge Base stores the hue model h∗ , the ground-plane estimate, the breadcrumb map, and all runtime timers across ticks. The following subsections describe each phase in detail. A. HARDWARE PLATFORM

The target hardware is a JetBot-class differential-drive UGV equipped with an Orbbec Astra RGB-D camera (top row of Figure 2; a closer view is shown in Figure 3). The Astra provides synchronized color and depth images at up to 640 × 480 resolution and 30 Hz, with a horizontal field of view of approximately 58◦ . The depth stream gives per-pixel range measurements without a separate laser or ultrasonic sensor. The controller is implemented in Python and combines OpenCV for image processing, visual odometry, object detection, and depth handling; NumPy for array operations, linear algebra, and robust statistics; and the Webots Python API for hardware abstraction. A single YOLOv8n ONNX model is loaded through cv2.dnn.readNetFromONNX and run entirely on the CPU backend. The system does not rely on a GPU, ROS, or any external middleware. B. PHASE 1: SENSOR INGESTION (MONITOR)

At the beginning of each control tick (Phase 1, Figure 2), a synchronized RGB frame and depth map are acquired from the Webots API. The depth image is resized to 640 × 480 using nearest-neighbour interpolation to preserve metric consistency across depth discontinuities. These aligned images serve as the only inputs to the downstream pipeline; no GPS, LiDAR, or wheel-encoder data are used. C. PHASE 2: STATE ESTIMATION (ANALYZE)

In Phase 2, three perception modules are executed for each new RGB-D frame pair (Figure 2). Their outputs are combined in the shared Knowledge Base, which stores the online hue model h∗ , the per-row floor depth model {gr } , the breadcrumb map line_map, and the runtime timers used by subsequent control stages. 1) Depth-Gated Line Tracker

The Depth-Gated Line Tracker (Figure 2) detects the guide line in HSV space by combining color segmentation with

J. S. Berntzen, S. Fatima, and L. Moonen: Self-Healing Visual Recovery for Autonomous Ground Vehicles . . .

Orbbec Astra RGB-D Camera

Phase 1 — Sensor Ingestion RGB frame + aligned depth map (50 ms tick)

Knowledge Base Hue model (h*) Floor model Breadcrumb map Timers & state

breadcrumbs

Depth-Fused Obstacle Detector Geometric residuals + YOLOv8n

Lightweight Visual Odometry GFTT + ORB + 5-pt RANSAC

ANALYZE

Depth-Gated Line Tracker HSV + floor-plane gating

MONITOR

JetBot UGV

lookup

P1

Obstacle Avoidance

PLAN

Phase 3 — Decision Layer P1: Obstacle → Avoidance · P2: Line visible → Following · P3: Line lost → Recovery P3

P2

Line Following PD control

Two-Stage Recovery Stage 1: Spin-and-Search Rotate in-place · relaxed thresholds

success

timeout

retry (≤ 3)

Stage 2: VO-Guided Navigation Navigate to nearest breadcrumb

Phase 4 — Actuation Wheel velocity commands to motors

Resume tracking

Stage 1 reacquires

exhausted

EXECUTE

wheel commands

Failure — stop

FIGURE 2. Overview of the proposed recovery framework. Each 50 ms tick traverses four phases (Monitor → Analyze → Plan → Execute) organized as a MAPE-K loop. Three perception modules feed a shared Knowledge Base: the Decision Layer selects one behavior per tick by priority (P1: Avoidance ≻ P2: Line Following ≻ P3: Recovery).

prove robustness, we apply a depth-based floor mask. Each image row r maintains an exponential moving average (EMA) of observed depth values: (k−1) g(k) + αg D(k) r = (1 − αg ) gr r ,

(a) Robot platform: the JetBot differential-drive UGV (top view).

(b) Orbbec Astra RGB-D camera providing color and depth at 30 Hz.

FIGURE 3. Hardware configuration. The Astra camera is front-facing, providing synchronized RGB and depth at 640 × 480, ≈ 58◦ HFoV.

a depth-aware constraint. A hue reference h∗ is initialized from a small seed region near the lower center of the frame and is then updated online using an exponential moving average: h∗k = (1 − α) h∗k−1 + α ĥk , (1) where ĥk denotes the median hue within the seed region and α controls the adaptation rate. In normal operation, the near-field mask is generated using inRange(h∗ − δh and h∗ + δh ), where δh sets the allowable hue tolerance. Raw color segmentation alone is prone to false positives on reflective floor surfaces and non-floor objects. To im-

(2)

with αg = 0.05 and a 20-frame initialization warmup. Every three frames, we recompute a full SVD-based plane fit using pixels whose depth lies within tolerance τ of the per-row model. The fitted plane n⊤ p = d is then used to gate candidate floor pixels. A pixel is classified as floor only if its residual to the plane is below 0.02 m and its vertical gradient is below 0.04. The near-field line mask is defined as: Mnear = medBlur(Mcolor ∧ Mfloor ) .

(3)

When Mnear falls below the area threshold, the controller switches to a relaxed far-field mask with a fixed hue band (area ≥ 500 px) and slows to 65% of nominal speed to preserve heading while the near-field view remains partially occluded. 2) Depth-Fused Obstacle Detector

The Depth-Fused Obstacle Detector (Figure 2) detects obstacles within a forward image corridor spanning half the image width by fusing geometric and semantic cues. 7

J. S. Berntzen, S. Fatima, and L. Moonen: Self-Healing Visual Recovery for Autonomous Ground Vehicles . . .

D. PHASE 3: DECISION LAYER (PLAN)

FIGURE 4. Simulation view with fused obstacle detection and a forward corridor.

Geometric obstacles are extracted from depth residuals. Points whose depth exceeds the per-row floor model by more than kmad · MAD + τmargin with MAD-based adaptive threshold kmad = 2 and τmargin = 0.05 m are treated as candidate obstacles. These candidates are further filtered using real-world size constraints, requiring a minimum width and height of 0.03 m, a pixel area of at least 140 px, and sufficient vertical gradient. To reduce incorrect detections, blobs must also persist for at least two consecutive frames. Semantic obstacles are detected using YOLOv8n. For each bounding box, the aligned depth image is used to compute a median depth estimate, and the resulting candidates are ranked by their distance within the forward corridor. The fused obstacle output is illustrated in Figure 4, and the ranked obstacle list is forwarded to the Decision Layer.

After state estimation, the Decision Layer determines the robot’s action for each control tick by applying three mutually exclusive priorities (Phase 3, Figure 2): P1. Obstacle Avoidance: if the nearest fused obstacle lies within the warning threshold (d ≤ dwarn = 1.40 m), the avoidance behavior takes precedence over all other actions. P2. Line Following: in the absence of an immediate obstacle threat, the robot continues tracking the guide line using PD control whenever either the near-field mask or the relaxed far-field mask is active. P3. Recovery: if neither line mask has been detected for longer than the 0.6 s grace window and no obstacle is blocking, the two-stage recovery algorithm is engaged. All timing variables associated with these priorities are maintained in the Knowledge Base, allowing state to persist across ticks without external middleware. Consequently, the complete MAPE-K loop is executed within a single 50 ms control tick, removing the need for a separate adaptation manager [16]. E. PHASE 4: ACTUATION (EXECUTE)

In Phase 3, the chosen behavior calculates the wheel speeds, limits them to safe hardware values, and sends them to the motors in Phase 4 (Figure 2) for the execution. 1) P1: Obstacle Avoidance

When the Decision Layer selects P1, the closest obstacle in the fused list controls the avoidance response. If d ≤ dwarn = 1.40 m, the robot slows and pivots toward the larger left-right depth gap. If d ≤ dnear = 0.75 m, it carries out an immediate bounded pivot (hold 1.0–3.0 s, hard timeout 3.0 s). Normal line tracking resumes once clearance is greater than 1.65 m.

3) Lightweight Visual Odometry and Knowledge Base

2) P2: Line Following (PD Control)

The Lightweight Visual Odometry module (Figure 2) produces incremental pose estimates that populate the Knowledge Base breadcrumb map. The pipeline consists of: (i) GFTT feature detection; (ii) ORB descriptor extraction; (iii) KNN matching with Lowe’s ratio test; and (iv) a five-point essential matrix solver with RANSAC. These estimates are then linked to form an incremental robot trajectory. Whenever the near-field line mask is active, and the robot has moved more than 0.10 m since the last stored pose, the current trajectory pose is appended to a FIFO list (line_map). This list captures recently observed line locations and serves as the target set for Stage-2 recovery. Because the VO system is monocular and does not use loop closure, the breadcrumb trail gradually drifts over longer runs. This remains the main limitation of Stage-2 recovery and is discussed further in Section VI-A5.

The robot follows the centroid (cx , cy ) of the largest connected component in Mnear using a PD controller:

8

ek =

cx − w/2 , w/2

uk = Kp ek + Kd

ek − ek−1 , ∆t

(4)

with Kp = 0.9, Kd = 0.1, and ∆t = 0.05 s (20 Hz). Forward speed is decreased when lateral error becomes large: vk = vbase · max(vmin , 1 − β |ek |) ,

(5)

where β determines how aggressively speed is reduced. Wheel velocities are given by: ωL = vk + uk ,

ωR = vk − uk ,

(6)

and are clipped to the hardware limits before being sent to the robot. When the far-field mask is the only cue, vk is scaled by 0.65 and uk is derived from the far centroid, which provides degraded but stable heading maintenance.

J. S. Berntzen, S. Fatima, and L. Moonen: Self-Healing Visual Recovery for Autonomous Ground Vehicles . . .

Algorithm 1 Two-stage recovery (called each tick after grace window expires)

START Main Control Loop 50 ms tick · 20 Hz

[P1] Obstacle detected? resume tracking

Yes

Obstacle Avoidance

No

Follow Line PD control

[P2] Line visible?

Yes

No

11: procedure SpinAndSearch 12: stability_count ← 0; t0 ← now 13: while now −t0 < Tspin do 14: rotate in place with relaxed thresholds 15: if line confirmed: area & bottom-of-image check

[P3] Line Lost > 0.6s

Yes

Stabilize

Found Line

1: procedure Recovery 2: if not line_lost then 3: mark lost; freeze h∗ ; record tstart ; push VO pose 4: if SpinAndSearch then ▷ Stage 1 5: if Stabilize then 6: return success 7: if attempts ≥ Nmax then ▷ budget exhausted 8: stop; return failure 9: if not NavigateToNearest then ▷ Stage 2 10: return failure

Stage 1 — Spin and Search

then Stage 1 Reaquires Yes

No (spin timeout)

Stage 2 — VO-Guided Navigation

retry (≤ 3)

No

Retry > 3

FAILURE Timeout / No candidates Stop

FIGURE 5. Two-stage recovery control flow (Phase 4, Figure 2). Stage 1 (spin-and-search) attempts reacquisition first; Stage 2 (VO-guided navigation to a Knowledge-Base breadcrumb) is escalated to only on Stage-1 failure. Up to three spin–navigate cycles are permitted before a failure episode is declared

16: 17: 18: 19: 20:

stability_count += 1 if stability_count ≥ Nconfirm then return true else stability_count ← 0 attempts += 1; return false

21: procedure NavigateToNearest 22: pick pose p∗ ∈ line_map with ∥p∗ −p∥ > 0.10 m 23: drive with angle–distance control, timeout = Tstep 24: return reached or timeout

pose: uθ = Kp,θ (θtgt − θ),

3) P3: Two-Stage Recovery Algorithm

When P3 is selected, the two-stage recovery algorithm is activated. The 0.6 s grace window requires the line cues to remain absent continuously before recovery is triggered, so a brief dropout alone does not qualify. Figure 5 presents the complete recovery control flow within Phase 4 of Figure 2. Stage 1: Spin-and-Search. Once recovery starts, h∗ is held at its latest value and the detection criteria are relaxed: δh += 8 hue units, floor-plane tolerance increases by 0.02 m, and the gradient ceiling is lifted to 0.04. The robot then rotates in place for up to Tspin = 2.0 s at an angular fraction of 0.65. A detection is accepted only after Nconfirm = 5 consecutive frames satisfy both conditions: (a) the nearfield blob area is at least 900 px; and (b) the blob centroid sits below 45% of the image height, helping filter out ceiling reflections and partial views. After a successful detection, the system pauses for 0.6 s to stabilize before resuming nominal PD control, followed by a 0.8 s relaxedthreshold window to lower the chance of immediately falling back into recovery. Stage 2: VO-Guided Navigation. If Stage 1 fails, the nearest breadcrumb pose more than 0.10 m away is selected from line_map in the Knowledge Base. A proportional angle-then-distance controller then guides the robot to that

ud = Kp,d (dtgt − d),

(7)

using an 8.0 s timeout for each step. On arrival, Stage 1 is repeated. Up to Nmax = 3 spin–navigate cycles are allowed; if the line still has not been reacquired after that, the episode is considered a failure. Algorithm 1 summarizes the Two-Stage Recovery Algorithm; the full set of controller parameters is listed in Table 4 in Appendix V. EXPERIMENTAL SETUP

The simulation environment stresses the Depth-Gated Line Tracker and Depth-Fused Obstacle Detector (Phase 2); course geometry determines which branch of the Phase3 Decision Layer is activated most frequently; and fault injection directly triggers the Two-Stage Recovery Module (Phase 4) in Figure 2. A. ROBOT PLATFORM

The target hardware is a JetBot-class differential-drive UGV representing the minimum viable configuration considered in this work: two independently driven wheels, an onboard CPU, and a single front-facing camera. The JetBot platform, originally developed by NVIDIA as an accessible research robot, has been widely adopted for embedded robotics owing to its compact form factor and modest power budget [55, 56]. These properties make it 9

J. S. Berntzen, S. Fatima, and L. Moonen: Self-Healing Visual Recovery for Autonomous Ground Vehicles . . .

The approach is evaluated on two world types constructed in the Webots simulator (Figure 8): • Factory hall: a long-corridor setting with shelving, ramps, and a reflective floor. The reflective surface challenges the SVD plane gating in the Depth-Gated Line Tracker, while the clutter frequently triggers the Depth-Fused Obstacle Detector. • Office: a simpler, structured layout used for baseline calibration of the Phase-2 detection thresholds. C. LINE COURSES

FIGURE 6. Top view of the JetBot differential-drive UGV used in all experiments. The compact chassis houses two independently driven wheels and a front-facing Orbbec Astra RGB-D camera, representing the minimum hardware configuration targeted by this work.

representative of cost-sensitive deployments in logistics, inspection, and small-scale agriculture, where rich sensor suites are not economically justifiable. A top-down view of the platform is shown in Figure 6. The depth-sensing modality is the Orbbec Astra camera introduced in Section IV-A, which pairs its RGB sensor with a structured-light depth unit [57]. Its effective indoor depth range of 0.6–8 m covers both the obstacle warning zone (dwarn = 1.40 m) and the near-field line-tracking region, so no separate laser or ultrasonic sensor is required. Camera intrinsic parameters are derived analytically from the reported field of view rather than from a physical calibration target, since the Webots simulator provides consistent sensor geometry. The horizontal focal length is approximated as fx = fy =

w , 2 tan(θHFoV /2)

(8)

where w = 640 px and θHFoV ≈ 58◦ , giving fx ≈ 617 px. The principal point is set to the image center (cx , cy ) = (320, 240). These intrinsics are used by the VO module to back-project depth pixels into 3D and to estimate the essential matrix from matched feature correspondences. B. SIMULATION ENVIRONMENT

All experiments were carried out in Webots [58], an open-source robot simulator that uses the Open Dynamics Engine (ODE) for rigid-body physics and offers realistic RGB and depth sensor models. Webots was preferred over alternatives such as Gazebo because it has lower CPU overhead, better documentation, and a more integrated Python API for rapid iteration [59]. Figure 7 shows the Webots interface used throughout all experiments, including the 3D scene editor, the robot model, and the sensor visualization panels that were used to monitor RGB and depth streams during development. 10

Three guideline courses with progressively greater geometric complexity were painted onto the floors of the simulation worlds (Figure 9). Course geometry largely determines which stage of the Phase-4 Recovery Module is triggered most often. • Course 1 (oval): A smooth, continuously curving route with no sharp corners. When the line is lost, it falls well outside the camera FoV, so the Phase-3 Decision Layer typically escalates to P3 and invokes VO-Guided Navigation (Stage 2). • Course 2 (narrow turns): An indented layout with short straight segments and abrupt direction changes. Line loss often still leaves part of the line visible, so Spin-and-Search (Stage 1) resolves most episodes within the 2 s spin budget. • Course 3 (acute-corner triangle): A triangular route with three acute corners that causes immediate, complete line loss. Stage 2 is needed frequently, but the compact geometry keeps Knowledge-Base breadcrumbs within short travel distances. D. FAULT INJECTION PROTOCOL

Line-loss episodes were induced by programmatically removing the painted line texture from the camera view during operation, simulating real-world cases such as worn floor markings, strong glare, or sudden heading deviation after obstacle avoidance. Boxes and cones were also placed near or on the line to independently stress the avoidance module. Three fault categories were evaluated in the study: 1) Visual loss: the guideline texture was temporarily removed from the camera view during operation. The near-field mask vanished immediately, while far-field cues could persist briefly when the robot remained approximately aligned with the course. This constituted the primary fault mode and accounted for all 119 evaluation episodes. 2) Dynamic obstacles: boxes or cones were introduced on or near the guide line. These disturbances activated the avoidance module and could induce secondary line loss if the resulting avoidance maneuver moved the line outside the camera field of view. 3) Pose displacement: the robot was teleported between frames to assess Stage-2 navigation under large initial

J. S. Berntzen, S. Fatima, and L. Moonen: Self-Healing Visual Recovery for Autonomous Ground Vehicles . . .

FIGURE 7. Webots simulation interface used in all experiments. The 3D scene editor (centre) displays the robot and environment model; the side panels provide real-time RGB and depth sensor feeds used during algorithm development and fault-injection testing.

when the controller first detects line loss (grace window starts) and ends when either: (a) Nconfirm = 5 consecutive frames confirm stable reacquisition; or (b) all recovery cycles are exhausted. Timestamps at both events are logged to compute TTR. Only visual-loss (type 1) episodes are included in the reported statistics. E. RESEARCH QUESTIONS (a) Office world (baseline).

(b) Factory hall world.

FIGURE 8. Simulated environments. The factory hall includes reflective flooring, shelving, and ramps that challenge the Phase-2 perception modules.

(a) Course 1: Smooth oval.

(b) Course 2: Arrow turns.

(c) Course 3: Acute triangle.

FIGURE 9. The three line courses used in our evaluation. The course shape determines the line-loss frequency and which stage of the Phase-4 Recovery Module dominates.

pose errors. This condition was used only during preliminary calibration and was not included in the 119-episode evaluation set. Experiments consisted of multiple standardized laps per course starting from fixed initial poses. An episode begins

Three research questions frame the evaluation: RQ1 What autonomous recovery strategies can enable a vision-based UGV to regain operational status after losing its navigation cue? RQ2 What are the design trade-offs between achieving high recovery resilience and maintaining low hardware cost in camera-first UGV systems? RQ3 To what extent can simulation-based fault injection be used to evaluate and refine a recovery algorithm for low-cost UGVs? F. EVALUATION METRICS

Each episode is evaluated using three metrics: 1) Recovery success rate: the share of episodes in which line reacquisition is successful. We report 95% Wilson score confidence intervals to account for finite sample sizes. 2) Time to Recovery (TTR): the elapsed time from fault onset to confirmed reacquisition in successful episodes. Results are reported as mean ± SD, median, and the p10–p90 range. 11

J. S. Berntzen, S. Fatima, and L. Moonen: Self-Healing Visual Recovery for Autonomous Ground Vehicles . . .

3) Recovery mechanism: an episode-level classification of Stage-1 (spin-only) versus Stage-2 (VOnavigation), highlighting how course geometry influences the algorithm’s behavior. VI. RESULTS AND DISCUSSION

A detailed analysis of the results is presented in this section. Table 2 and Figures 10 and 11 summarize the per-course results, while Table 3 provides cross-system context. Unlike the qualitative positioning in Table 1, this table reports quantitative recovery times, which are limited to the prior camera-only system that reports a directly comparable metric [14]. The SLAM and deep-learning systems in Table 1 do not report a line-loss recovery time and are therefore not included here. A. RQ1: EFFECTIVENESS OF TWO-STAGE RECOVERY 1) Overall Success and Speed

Across 119 induced line-loss episodes, the proposed system successfully recovered in 103 cases (86.6%). The corresponding 95% Wilson confidence interval was [0.79, 0.92] (Table 2), the lower CI bound of 0.79 indicates the result is unlikely to be a sampling artifact. The overall median Time to Recovery (TTR) was 3.26 s, with a mean of 3.48 s ± 2.55 s. The broad spread in recovery times (p10 = 0.41 s, p90 = 7.27 s) reflects the two operating modes of the staged strategy, with rapid spin-based reacquisition in some cases and slower VO-guided navigation in others. 2) Stage-1 Spin-And-Search

Stage-1 contributed 34 of the 103 successful recoveries (33%) and typically completed reacquisition in under 1 s. It has shown most effective results on Course 2, where the narrow-turn geometry often left part of the line visible after a heading deviation; as a result, 24 of 45 Course 2 successes (53%) were resolved through spin-only recovery (Figure 10b, Table 2). The pronounced sub-1 s peak in Figure 11b provides clear evidence that Stage-1 behaved as intended, delivering rapid recovery while the relaxed thresholds and 5-frame confirmation rule helped reduce false positives. 3) Stage-2 VO-Guided Navigation

When spin-only recovery was insufficient, Stage 2 guided the robot to the nearest stored VO breadcrumb before the recovery sequence was reinitiated. Stage 2 accounted for 69 of the 103 successful recoveries (67%) and was particularly effective under complete line-loss conditions, contributing to 90% of Course 1 successes (18/20, Figure 10a) and 79% of Course 3 successes (30/38, Figure 10c). Stage 2 recovery times ranged from 2 to 9 s, depending on breadcrumb proximity, which produced the pronounced right tail observed in all three histograms (Figure 11). Although Stage 2 was used frequently, Course 3 achieved a median TTR of 4.27 s because of its shorter inter-corner breadcrumb spacing, whereas Course 1 required longer navigation segments, 12

resulting in a median TTR of 3.48 s and the lowest overall success rate (76.9%, Table 2). 4) Post-Recovery Stability

A 0.6 s stabilization pause, followed by a 0.8 s relaxedthreshold window after reacquisition, prevented oscillatory re-entry into recovery in almost all episodes. Only a small number of back-to-back activations were observed, and these occurred only on Course 3, where recovery often ended near an acute corner. Overall, the robot resumed autonomous lap completion in the vast majority of episodes without operator assistance, demonstrating a high taskcontinuity rate in the sense of [19]. 5) Failure Boundary

The 16 failed episodes, distributed as 6 on Course 1, 4 on Course 2, and 6 on Course 3, defined the conditions under which the strategy was not fully effective. Three failure modes were identified across these episodes: (i) sparse breadcrumb coverage, where obstacle avoidance displaced the robot beyond the reach of any stored pose and repeated spin attempts exhausted the recovery budget; (ii) persistent occlusion, where dynamic obstacles blocked the line throughout all recovery attempts and prevented both spin-based search and post-navigation recovery from succeeding; and (iii) VO drift, where monocular visual odometry drift during pure rotation or low-texture traversal caused Stage 2 to navigate toward an incorrect target, moving the robot farther from the line. Because individual episodes often combined more than one of these effects, we characterize the failure modes qualitatively rather than assigning a single cause to each of the 16 episodes. Each mode has a corresponding mitigation in Section VIII. Stage 1 spin-based recovery handled partial-visibility failures in under 1 s (Figure 11b), whereas Stage 2 VOguided navigation handled complete-loss scenarios in 2– 9 s (Figs. 10, 11a, 11c). The post-recovery grace window was necessary to prevent oscillatory re-entry into recovery. Overall, the strategy’s effectiveness was primarily limited by breadcrumb coverage, persistent obstacle occlusion, and VO drift. Answer to RQ1: The two-stage recovery restored guideline tracking in 86.6% of 119 line-loss episodes (95% CI 0.79–0.92) at a median 3.26 s, showing that camera-only, CPU-only recovery is effective across varied course geometries. B. RQ2: CAMERA-FIRST COST-RESILIENCE TRADE-OFFS 1) Accuracy at Reduced Cost

Table 3 compared the proposed system with two published approaches in terms of recovery accuracy, mean TTR, GPU dependency, and sensor requirements. The proposed system achieved a recovery accuracy of 0.87 (103/119), comparable to the 87% qualitative success rate reported by

J. S. Berntzen, S. Fatima, and L. Moonen: Self-Healing Visual Recovery for Autonomous Ground Vehicles . . .

TABLE 2. Per-course recovery results. ‘‘Spin’’ = Stage-1 spin-only resolutions; ‘‘Nav’’ = Stage-2 VO-navigation resolutions. TTR statistics over successful episodes only. (Evidence for RQ1 and RQ3.)

Episode

Success

%

Mean±SD (s)

Median (s)

p10 (s)

p90 (s)

Spin / Nav

Course 1 Course 2 Course 3

26 49 44

20 45 38

76.9 91.8 86.4

3.99±1.92 2.51±2.57 4.36±2.42

3.48 0.85 4.27

2.64 0.33 1.21

7.34 6.93 8.33

2 / 18 24 / 21 8 / 30

Overall

119

103

86.6

3.48±2.55

3.26

0.41

7.27

34 / 69

(a) Course 1: Spin 2, Nav 18.

(b) Course 2: Spin 24, Nav 21.

(c) Course 3: Spin 8, Nav 30.

FIGURE 10. Recovery mechanism counts per course (RQ1). Stage-1 spin dominates where the line stays partially visible (Course 2); Stage-2 VO navigation dominates where it disappears completely (Courses 1 and 3). The split validates the staged escalation design.

(a) Course 1: Time to Recovery (TTR).

(b) Course 2: Time to Recovery (TTR).

(c) Course 3: Time to Recovery (TTR).

FIGURE 11. Per-course TTR histograms for successful episodes (RQ1). Course 2 shows a clear bimodal distribution: a sub-1 s peak (Stage-1 spin) and a 2–7 s tail (Stage-2 navigation). Courses 1 and 3 are dominated by Stage-2, producing unimodal distributions shifted right. The shape of each histogram directly reflects the mechanism counts in Figure 10.

Lee et al. [14]. Since these studies used different platforms, sensors, and test conditions, this comparison indicates relative performance rather than a direct benchmark. Notably, this performance was obtained using only an Orbbec Astra RGB-D camera, with an estimated retail cost of $100–150, and without relying on LiDAR, RADAR, GPS, or a GPU. In contrast, Lee et al. required additional sensors on an embedded ARM11 platform. These findings indicate that comparable recovery performance was achieved with a significantly simpler and lower-cost hardware configuration, enabled by VO-based breadcrumb navigation and a depth-gated line tracker. 2) The Residual TTR Cost and Its Geometry-Dependence

The mean Time to Recovery (TTR) of 3.48 s is slightly higher than Lee et al.’s reported ≈3.23 s (Table 3). Rather than reflecting a general slowdown, this difference is driven by Stage-2 navigation in geometrically demanding courses, where the camera-only setup is most constrained.

TABLE 3. Comparison with prior vision-based recovery systems (RQ2). Platform and protocol differences preclude direct benchmarking; results are indicative of relative performance across hardware tiers.

Metric

Proposed

[14]

Accuracy (succ./att.) 0.87 (103/119) 0.87a Mean TTR (s) 3.48 ≈3.23b GPU required No No External sensors RGB-D only RGB + extras a Qualitative success rate; not per-episode. b Global relocalization on embedded ARM11.

On Course 1, long arcs move the line completely out of the field of view, so the median TTR remains 3.48 s and Stage-2 accounts for 90% of successful recoveries (Table 2, Figure 10a, Figure 11a). By contrast, on Course 2, the narrow-turn geometry preserves partial line visibility, reducing the median TTR to 0.85 s and allowing Stage-1 spin recovery to resolve 53% of its successfull recoveries 13

J. S. Berntzen, S. Fatima, and L. Moonen: Self-Healing Visual Recovery for Autonomous Ground Vehicles . . .

(Table 2, Figure 10b, Figure 11b), which is competitive with or faster than heavier sensor-equipped systems. Overall, the performance difference is geometry-dependent and predictable rather than a uniform penalty. 3) Cost of the Camera-Only Constraint in Failure Modes

Two of the three failure modes identified under RQ1 arise directly from the camera-only, CPU-only design, reflecting the trade-off required to achieve self-healing under strict resource constraints. Sparse breadcrumb coverage could be addressed through global LiDAR-based localization, which enables relocalization without dependence on stored visual poses. VO drift remains an inherent limitation of monocular, depth-free odometry; a sensor-rich system could reduce this limitation through IMU fusion or loop closure. Collectively, these failures define the conditions under which the embedded MAPE-K loop cannot recover autonomously, and Section VIII outlines concrete extensions to the knowledge base and recovery policy. The camera-first design achieved recovery accuracy comparable to that of heavier prior systems (0.87 vs. 0.87, Table 3) while requiring substantially less hardware. The principal trade-off was a modest increase in mean TTR (3.48 s vs. ≈3.23 s), and this difference was strongly dependent on scene geometry. In environments that preserved partial line visibility, such as Course 2, the penalty was minimal, whereas fully occluded environments exposed the limitations of monocular VO. Answer to RQ2: Camera-first, CPU-only recovery matched the 0.87 accuracy of a heavier prior system at a fraction of the hardware cost, with only a modest, geometry-dependent mean TTR increase (3.48 s vs. ≈3.23 s) and no unrecoverable failures. C. RQ3: SIMULATION FOR EVALUATION AND REFINEMENT 1) Scale and Precision of Fault Injection

The Webots simulation enabled the safe, deterministic, and reproducible induction of 119 controlled line-loss episodes, which would have been impractical on physical hardware because of cost, time, and safety constraints. Because fault onset was programmatically triggered and timestamped, exact TTR values were recorded for every episode (Table 2, p10–p90 columns). In contrast, physical hardware does not allow fault onset to be measured precisely, so TTR values would be approximate and the p10– p90 distribution in Table 2 could not be reported reliably. 2) Geometry-Driven Insight Through Course Variation

The three courses were designed to evaluate how course layout influences the recovery strategy, and the results show that geometry is the main determinant of which recovery mechanism is selected. Course 2, with its narrowturn configuration, achieved 53% spin-only recoveries 14

(Figure 10b), while the smoother oval of Course 1 achieved only 10% spin-only recoveries (2/20; Figure 10a). This 43-percentage-point gap, combined with the consistent behavior observed within each course, indicates that recovery mode is strongly driven by geometry rather than random variation. Because the simulation provides precise episode-level labels, this pattern can be identified clearly; in a real-world setting, it would likely be less apparent due to confounding effects such as floor reflections, lighting variation, and sensor calibration drift. 3) Algorithm Refinement Through Rapid Iteration

Simulation enabled systematic tuning of the key thresholds, including hue tolerance, floor-plane tolerance, confirmation frame count, spin duration, and obstacle-distance parameters, through repeated fault injection without physical wear, battery constraints, or safety risk. For example, the 5-frame confirmation threshold (Nconfirm ) and the 0.6 s postrecovery stabilization window were set to their final values during repeated fault-injection runs in simulation. Their defaults are listed in Section V; a systematic ablation of these parameters is left to future work. Validating these refinements on physical hardware would have required substantially more experiments. Webots-based fault injection produced 119 precisely timestamped episodes across three geometrically diverse courses, enabling exact Time to Recovery (TTR) distributions (Table 2, Figure 11) and mechanism-level breakdowns (Figure 10) that would be difficult to obtain on physical hardware. This controlled setting isolated the geometry-driven split in recovery mechanisms and allowed rapid, safe refinement of the controller thresholds. Answer to RQ3: Simulated fault injection produced 119 controlled, timestamped episodes that enabled exact TTR distributions and safe threshold tuning beyond what physical trials allow, while not replacing real-world validation. D. CROSS-COURSE ANALYSIS

The three courses collectively illustrate a consistent relationship between course geometry and recovery mechanism. A comparison of mechanism distributions across courses (Figure 10) reveals a 43-percentage-point gap in Stage-1 spin recovery rates between the most favorable course (Course 2: 53%) and the least favorable (Course 1: 10%). This gap is not attributable to random variation but to systematic differences in how the three course layouts preserve or remove line visibility after a heading deviation. From the perspective of the MAPE-K loop, this geometry-dependence reflects the interplay between the Monitor and Plan stages. On smooth oval courses (Course 1), continuous arcs allow the line to exit the camera field of view entirely, so the Monitor stage cannot supply any positive evidence during a spin, and the Plan stage must

J. S. Berntzen, S. Fatima, and L. Moonen: Self-Healing Visual Recovery for Autonomous Ground Vehicles . . .

escalate to Stage-2 navigation. On narrow-turn courses (Course 2), abrupt direction changes produce line-loss episodes where the line remains partially visible from slightly different headings, allowing Stage-1 to confirm reacquisition within the 2 s spin budget. Course 3 sits between these extremes: acute corners cause complete line loss similar to Course 1, but the compact triangle geometry keeps stored breadcrumbs within short travel distances, producing Stage-2 recoveries that complete quickly despite the heavier navigation overhead. The TTR distributions (Figure 11) reinforce this interpretation. Course 2 shows a bimodal distribution with a sharp sub-1 s peak (Stage-1) and a 2–7 s tail (Stage-2), the two populations separated by the 2 s spin budget. Courses 1 and 3 show right-skewed unimodal distributions that peak in the Stage-2 range, confirming that Stage-1 contributes minimally when geometry forces complete occlusion. TTR therefore depends mainly on course geometry rather than on the specific fault, a finding that directly supports deployment planning for real-world line-following environments. The 95% Wilson confidence interval for the overall success rate (0.79–0.92) reflects the relatively modest episode count of 119. Widening the evaluation to a larger number of courses and fault conditions would tighten this interval. Nevertheless, the consistent within-course behavior and the clear geometry-driven mechanism split already provide strong evidence that the two-stage strategy is valid and that its performance characteristics are predictable rather than incidental. VII. THREATS TO VALIDITY

Simulation-to-reality gap. All experiments were conducted in Webots, which relies on idealized sensor models. In real deployments, cameras may be affected by motion blur, rolling shutter, glare, and calibration drift. Real depth sensors can also introduce flying pixels, quantization noise, and interference on reflective or IR-absorbing surfaces. These effects may distort the floor-plane estimate, create false obstacle detections, or hide genuine ones, which could reduce tracking and recovery performance in physical deployment. Environment assumptions. The method assumes a clearly visible painted line on a mostly flat surface. Worn markings, shadows, textured tiles, or uneven lighting may weaken the HSV line mask. The per-row EMA floor model and SVD plane fitting also work best on smooth surfaces; ramps, thresholds, or rough terrain can violate the planar assumption and lead to incorrect floor-versus-obstacle classification. VO accuracy. The lightweight monocular VO does not include loop closure, IMU fusion, or learned feature descriptors. During pure rotations and in low-texture corridors, drift can build up and shift Stage-2 navigation targets by 10–20 cm, which may be enough to miss the line on a narrow track.

Sample size. Although the 119-episode dataset yields confidence intervals that remain above the 0.79 lower bound, it is still relatively small compared with large-scale deployed fleet data. Results may differ across substantially different floor materials, line widths, or lighting conditions. VIII. FUTURE WORK

Improved localization: Replacing a lightweight monocular VO with a keyframe-based visual SLAM system (for example, ORB-SLAM3 [32]) would introduce loop closure and long-range drift correction, directly addressing failure category (iii). Visual-inertial odometry (for example, VINS Mono [38]) would further stabilize estimates during the pure-rotation phases of Stage-1 spin. Appearance-based global relocalization: Adding a compact place-recognition front end (such as Galvez’s bags or a learned global descriptor) would let Stage2 propose candidate poses globally instead of routing to the nearest stored breadcrumb. This would address failure category (i) by removing the dependence on local breadcrumb coverage. Adaptive recovery policy: Replacing fixed-duration spin and navigation timeouts with a decision policy that balances expected TTR, perception confidence, and remaining breadcrumb quality would make recovery more efficient and responsive to runtime conditions. Integration with motion planners: The current pivotbased obstacle avoidance could be replaced by exporting free-space estimates to a local cost map and connecting them to established planners (DWA, TEB) for smoother navigation, especially during Stage-2 recovery in cluttered environments. Formal self-adaptive verification: The two-stage structure, with explicit budgets, timeouts, and confirmation thresholds, is well suited to formal analysis with probabilistic model checkers such as PRISM [47, 48]. Modeling the recovery protocol as a discrete Markov decision process would make it possible to derive formal bounds on selfhealing success probability and expected TTR, then verify them against the empirical results reported here. This would strengthen confidence in deployment scenarios not covered by simulation. Learning-based adaptive recovery policy: The fixed spin and navigation timeouts could be replaced with a reinforcement learning policy trained on MAPE-K feedback signals (line confidence, breadcrumb quality, remaining budget, and course geometry). Such a policy would adapt the self-healing behavior to environment-specific failure patterns observed at runtime, moving beyond the handtuned thresholds of the current design toward a genuinely self-adaptive controller [18]. Physical validation: Deploying and validating the algorithm on a real JetBot or equivalent platform with a physical Orbbec Astra camera is the most important next step. Results on real hardware, under varied lighting, flooring materials, and line-wear conditions, would quantify the 15

J. S. Berntzen, S. Fatima, and L. Moonen: Self-Healing Visual Recovery for Autonomous Ground Vehicles . . .

sim-to-real gap and confirm whether the self-healing behavior observed in simulation transfers to physical deployment. IX. CONCLUSIONS

This paper presents a lightweight, camera-first self-healing controller for autonomous ground vehicles that recovers from a complete loss of guidelines using only an RGB-D camera and CPU-based computation. The proposed system embeds a MAPE-K loop within a single 50 ms control cycle, enabling perception, decision-making, and actuation to operate continuously without operator intervention or external adaptation infrastructure. Its design combines three tightly coupled components: a depth-gated HSV line tracker with online hue adaptation and periodic floor modeling, a depth-fused obstacle avoidance module that integrates geometric residuals with YOLOv8n detections, and a two-stage recovery mechanism that first performs in-place spin-and-search and then resorts to VO-guided navigation toward stored breadcrumb poses when immediate reacquisition fails. The experimental evaluation on 119 fault-injected episodes across three geometrically diverse Webots simulation courses demonstrated that the system recovered successfully in 86.6% of cases, with a 95% Wilson confidence interval of 0.79–0.92. The median time to recovery was 3.26 s, and the mean was 3.48 s, indicating that robust recovery can be achieved under strict sensing and computational constraints. The results further show that recovery behavior is strongly shaped by environment geometry: partial-visibility failures are typically resolved rapidly by Stage 1 spin recovery, whereas complete-occlusion cases require Stage 2 navigation and incur longer recovery times. The post-recovery stabilization step also proved important in reducing oscillatory re-entry into recovery. Overall, the findings show that reliable and repeatable visual recovery is feasible without LiDAR, GPS, or GPUclass hardware. Compared with prior heavier sensor-based approaches, the proposed method achieves competitive recovery performance while remaining low-cost, modular, and deployable on resource-constrained platforms. These results show that self-healing recovery is achievable on camera-only UGVs, and they motivate future work on realworld validation, improved localization, and more adaptive recovery policies.

TABLE 4. Key system parameters used across all experiments. Parameter

Depth-gated line tracker (Eq. 1–3) δh (recovery) +8 αg (floor EMA) 0.05 Ground warmup 20 PLANE_FIT_EVERY_N 3 Floor residual / gradient 0.02 m / 0.04 Far-mask area / speed 500 px / 0.65

Depth-fused obstacle detection and avoidance kmad 2.0 τmargin 0.05 m Min width / area / persist 0.03 m / 140 px / 2 dwarn 1.40 m dnear 0.75 m

8.0 s Max navigate-to-breadcrumb time

Two-stage recovery Nconfirm Tspin Nmax Grace window Stabilization hold Post-recovery relax

5 2.0 s 3 0.6 s 0.6 s 0.8 s

Detector YOLO input YOLO_CONF_THRESH YOLO_NMS_THRESH

Consecutive confirmation frames Max spin-search duration Max spin–navigate cycles Delay before declaring loss again Hold still after reacquisition Looser thresholds after recovery

640×640 YOLOv8n input resolution 0.35 Confidence threshold 0.45 NMS IoU threshold

References [1]

[2]

[3] [4]

[5]

[6]

[8]

[9]

APPENDIX. PARAMETER SETTINGS [10] [11]

16

MAD residual scale Extra margin over floor model Obstacle size and persistence filters Obstacle warning distance Obstacle near / pivot distance

Breadcrumb navigation (Eq. 7) Tstep

[7]

Table 4 shows the values of key system parameters used across all experiments.

Extra hue expansion in recovery Per-row floor EMA weight Init frames for floor model SVD plane-fit cadence (frames) Floor-pixel acceptance thresholds Far-mask trigger and speed scale

Control loop and steering (Eq. 4, 5) TIME_STEP / ∆t 50 ms / 0.05 s Control loop period (20 Hz) Kp 0.9 PD steering proportional gain Kd 0.1 PD steering derivative gain

ACKNOWLEDGMENT

The first author carried out this work as part of a Master’s project with the Secure and Trustworthy Intelligent Systems group at Simula Research Laboratory.

Value Description

W. Wei, M. Xiao, W. Duan, H. Wang, Y. Zhu, C. Zhai, and G. Geng. ‘‘Research Progress on Autonomous Operation Technology for Agricultural Equipment in Large Fields.’’ In: Agriculture 14.9 (2024), p. 1473. DOI: 10.3390/agriculture14091473. M. Agelli, N. Corona, F. Maggio, and P. V. Moi. ‘‘Unmanned Ground Vehicles for Continuous Crop Monitoring in Agriculture: Assessing the Readiness of Current ICT Technology.’’ In: Machines 12.11 (2024), p. 750. DOI: 10.3390/machines12110750. M. Maimone, Y. Cheng, and L. Matthies. ‘‘Two Years of Visual Odometry on the Mars Exploration Rovers.’’ In: J. Field Robotics 24.3 (2007), pp. 169–186. DOI: 10.1002/rob.20184. C. Cadena, L. Carlone, H. Carrillo, Y. Latif, D. Scaramuzza, J. Neira, I. Reid, and J. J. Leonard. ‘‘Past, Present, and Future of Simultaneous Localization and Mapping: Toward the RobustPerception Age.’’ In: IEEE Trans. Robotics 32.6 (2016), pp. 1309– 1332. DOI: 10.1109/TRO.2016.2624754. M. Misaros, O.-P. Stan, I.-C. Donca, and L.-C. Miclea. ‘‘Autonomous Robots for Services—State of the Art, Challenges, and Research Areas.’’ In: Sensors 23.10 (2023), p. 4962. DOI: 10.3390/s23104962. M. Spagnuolo, G. Todde, M. Caria, N. Furnitto, G. Schillaci, and S. Failla. ‘‘Agricultural Robotics: A Technical Review Addressing Challenges in Sustainable Crop Production.’’ In: Robotics 14.2 (2025), p. 9. DOI: 10.3390/robotics14020009. F. Bonin-Font, A. Ortiz, and G. Oliver. ‘‘Visual Navigation for Mobile Robots: A Survey.’’ In: J. Intelligent and Robotic Systems 53.3 (2008), pp. 263–296. DOI: 10.1007/s10846-008-9235-4. C.-Y. Wang, A. Bochkovskiy, and H.-Y. M. Liao. ‘‘YOLOv7: Trainable Bag-of-Freebies Sets New State-of-the-Art for RealTime Object Detectors.’’ In: IEEE/CVF Conf. Computer Vision and Pattern Recognition. 2023, pp. 7464–7475. G. Jocher, A. Chaurasia, and J. Qiu. Ultralytics YOLOv8. Url: https://github.com/ultralytics/ultralytics. 2023. C. Godard, O. Mac Aodha, M. Firman, and G. J. Brostow. ‘‘Digging Into Self-Supervised Monocular Depth Estimation.’’ In: IEEE/CVF Int’l Conf. Computer Vision. 2019, pp. 3828–3838. R. Ranftl, K. Lasinger, D. Hafner, K. Schindler, and V. Koltun. ‘‘Towards Robust Monocular Depth Estimation: Mixing Datasets

J. S. Berntzen, S. Fatima, and L. Moonen: Self-Healing Visual Recovery for Autonomous Ground Vehicles . . .

[12] [13] [14] [15]

[16]

[17] [18]

[19]

[20] [21]

[22]

[23] [24]

[25]

[26] [27]

[28]

[29]

[30]

for Zero-Shot Cross-Dataset Transfer.’’ In: IEEE Trans. Pattern Analysis and Machine Intelligence 44.3 (2022), pp. 1623–1637. doi: 10.1109/TPAMI.2020.3019967. D. Scaramuzza and F. Fraundorfer. ‘‘Visual Odometry [Tutorial].’’ In: IEEE Robotics & Automation Magazine 18.4 (2011), pp. 80– 92. doi: 10.1109/MRA.2011.943233. J. Engel, V. Koltun, and D. Cremers. ‘‘Direct Sparse Odometry.’’ In: IEEE Trans. Pattern Analysis and Machine Intelligence 40.3 (2018), pp. 611–625. doi: 10.1109/TPAMI.2017.2658577. S. Lee, S. Lee, and S. Baek. ‘‘Vision-Based Kidnap Recovery with SLAM for Home Cleaning Robots.’’ In: J. Intelligent & Robotic Systems 67.1 (2012), pp. 7–24. doi: 10.1007/s10846-011-9647-4. R. C. Luo, K. C. Yeh, and K. H. Huang. ‘‘Resume Navigation and Re-Localization of an Autonomous Mobile Robot after Being Kidnapped.’’ In: IEEE Int’l Symp. Robotic and Sensors Environments (ROSE). 2013, pp. 7–12. doi: 10.1109/ROSE.2013. 6698410. S. White, J. Hanson, I. Whalley, D. Chess, and J. Kephart. ‘‘An Architectural Approach to Autonomic Computing.’’ In: Int’l Conf. Autonomic Computing, 2004. Proceedings. 2004, pp. 2–9. doi: 10.1109/ICAC.2004.1301340. B. Cheng et al. Using Models at Runtime to Address Assurance for Self-Adaptive Systems. 2015. arXiv: 1505.00903. R. de Lemos et al. ‘‘Software Engineering for Self-Adaptive Systems: A Second Research Roadmap.’’ In: Software Engineering for Self-Adaptive Systems II: Int’l Seminar, Dagstuhl Castle, Germany, October 24-29, 2010 Revised Selected and Invited Papers. Springer, 2013, pp. 1–32. doi: 10 . 1007 / 978 - 3 - 642 35813-5_1. A. Romero-Garcés, A. Hidalgo-Paniagua, M. González-García, and A. Bandera. ‘‘On Managing Knowledge for MAPE-K Loops in Self-Adaptive Robotics Using a Graph-Based Runtime Model.’’ In: Applied Sciences 12.17 (2022), p. 8583. doi: 10 . 3390 / app12178583. S. Thrun, D. Fox, W. Burgard, and F. Dellaert. ‘‘Robust Monte Carlo Localization for Mobile Robots.’’ In: Artificial Intelligence 128.1 (2001), pp. 99–141. doi: 10.1016/S0004-3702(01)00069-8. S. Yu, F. Yan, Y. Zhuang, and D. Gu. ‘‘A Deep-Learning-based Strategy for Kidnapped Robot Problem in Similar Indoor Environment.’’ In: J. Intelligent & Robotic Systems 100.3 (2020), pp. 765–775. doi: 10.1007/s10846-020-01216-x. Z. Teed and J. Deng. ‘‘DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Cameras.’’ In: Advances in Neural Information Processing Systems. Vol. 34. Curran Associates, Inc., 2021, pp. 16558–16569. S. Thrun. ‘‘Probabilistic Robotics.’’ In: Commun. ACM 45.3 (2002), pp. 52–57. doi: 10.1145/504729.504754. D. Nister, O. Naroditsky, and J. Bergen. ‘‘Visual Odometry.’’ In: IEEE Computer Society Conf. Computer Vision and Pattern Recognition, 2004. CVPR 2004. Vol. 1. 2004, pp. I–I. doi: 10. 1109/CVPR.2004.1315094. IFR International Federation of Robotics. World Robotics 2025 Report – Industrial Robots. Url: https://ifr.org/ifr-press-releases/ news/global-robot-demand-in-factories-doubles-over-10-years. 2025. IFR International Federation of Robotics. World Robotics 2025 Report – Service Robots. Url: https://ifr.org/ifr-press-releases/ news/service-robots-see-global-growth-boom. 2025. IFR International Federation of Robotics. Robot Density Surges in Europe, Asia, and Americas. Url: https://ifr.org/ifr- pressreleases / news / robot - density - surges - in - europe - asia - and americas. 2025. O. Liu, S. Yuan, and Z. Li. ‘‘A Survey on Sensor Technologies for Unmanned Ground Vehicles.’’ In: 3rd Int’l Conf. Unmanned Systems (ICUS). 2020, pp. 638–645. doi: 10.1109/ICUS50048. 2020.9274845. P. Mittal. ‘‘A Comprehensive Survey of Deep Learning-Based Lightweight Object Detection Models for Edge Devices.’’ In: Artificial Intelligence Review 57.9 (2024), p. 1. doi: 10.1007/ s10462-024-10877-1. G. Klein and D. Murray. ‘‘Parallel Tracking and Mapping for Small AR Workspaces.’’ In: IEEE and ACM Int’l Symp. Mixed and Augmented Reality. 2007, pp. 225–234. doi: 10.1109/ISMAR. 2007.4538852.

[31]

[32]

[33] [34]

[35]

[36]

[37]

[38]

[39] [40] [41] [42]

[43]

[44]

[45] [46]

[47]

[48]

[49]

R. Mur-Artal, J. M. M. Montiel, and J. D. Tardós. ‘‘ORB-SLAM: A Versatile and Accurate Monocular SLAM System.’’ In: IEEE Trans. Robotics 31.5 (2015), pp. 1147–1163. doi: 10.1109/TRO. 2015.2463671. C. Campos, R. Elvira, J. J. G. Rodríguez, J. M. M. Montiel, and J. D. Tardós. ‘‘ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual–Inertial, and Multimap SLAM.’’ In: IEEE Trans. Robotics 37.6 (2021), pp. 1874–1890. doi: 10.1109/TRO.2021. 3075644. D. Galvez-López and J. D. Tardos. ‘‘Bags of Binary Words for Fast Place Recognition in Image Sequences.’’ In: IEEE Trans. Robotics 28.5 (2012), pp. 1188–1197. doi: 10.1109/TRO.2012.2197158. M. Kaess, H. Johannsson, R. Roberts, V. Ila, J. J. Leonard, and F. Dellaert. ‘‘iSAM2: Incremental Smoothing and Mapping Using the Bayes Tree.’’ In: The International J. Robotics Research 31.2 (2012), pp. 216–235. doi: 10.1177/0278364911430419. J. Engel, T. Schöps, and D. Cremers. ‘‘LSD-SLAM: Large-Scale Direct Monocular SLAM.’’ In: Computer Vision – ECCV 2014. Springer International Publishing, 2014, pp. 834–849. doi: 10. 1007/978-3-319-10605-2_54. C. Forster, L. Carlone, F. Dellaert, and D. Scaramuzza. ‘‘OnManifold Preintegration for Real-Time Visual–Inertial Odometry.’’ In: IEEE Trans. Robotics 33.1 (2017), pp. 1–21. doi: 10. 1109/TRO.2016.2597321. S. Leutenegger, S. Lynen, M. Bosse, R. Siegwart, and P. Furgale. ‘‘Keyframe-Based Visual–Inertial Odometry Using Nonlinear Optimization.’’ In: The International J. Robotics Research 34.3 (2015), pp. 314–334. doi: 10.1177/0278364914554813. T. Qin, P. Li, and S. Shen. ‘‘VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimator.’’ In: IEEE Trans. Robotics 34.4 (2018), pp. 1004–1020. doi: 10.1109/TRO.2018. 2853729. Z. Teed, L. Lipson, and J. Deng. ‘‘Deep Patch Visual Odometry.’’ In: Adv. Neural Information Processing Systems 36 (2023), pp. 39033–39051. M. N. Favorskaya. ‘‘Deep Learning for Visual SLAM: The State-of-the-Art and Future Trends.’’ In: Electronics 12.9 (2023), p. 2006. doi: 10.3390/electronics12092006. D. Fox. ‘‘KLD-Sampling: Adaptive Particle Filters.’’ In: Advances in Neural Information Processing Systems. Vol. 14. MIT Press, 2001. M. Cummins and P. Newman. ‘‘FAB-MAP: Probabilistic Localization and Mapping in the Space of Appearance.’’ In: The International J. Robotics Research 27.6 (2008), pp. 647–665. doi: 10.1177/0278364908090961. G. Kim and A. Kim. ‘‘Scan Context: Egocentric Spatial Descriptor for Place Recognition Within 3D Point Cloud Map.’’ In: IEEE/RSJ Int’l Conf. Intelligent Robots and Systems (IROS). 2018, pp. 4802– 4809. doi: 10.1109/IROS.2018.8593953. R. Dubé, D. Dugas, E. Stumm, J. Nieto, R. Siegwart, and C. Cadena. ‘‘SegMatch: Segment Based Place Recognition in 3D Point Clouds.’’ In: IEEE Int’l Conf. Robotics and Automation (ICRA). 2017, pp. 5266–5272. doi: 10.1109/ICRA.2017.7989618. A. Kendall, M. Grimes, and R. Cipolla. ‘‘PoseNet: A Convolutional Network for Real-Time 6-DOF Camera Relocalization.’’ In: IEEE Int’l Conf. Computer Vision. 2015, pp. 2938–2946. R. Arandjelovic, P. Gronat, A. Torii, T. Pajdla, and J. Sivic. ‘‘NetVLAD: CNN Architecture for Weakly Supervised Place Recognition.’’ In: IEEE Conf. Computer Vision and Pattern Recognition. 2016, pp. 5297–5307. R. Calinescu, L. Grunske, M. Kwiatkowska, R. Mirandola, and G. Tamburrelli. ‘‘Dynamic QoS Management and Optimization in Service-Based Systems.’’ In: IEEE Trans. Software Engineering (TSE) 37.3 (2011), pp. 387–409. doi: 10.1109/tse.2010.92. M. Kwiatkowska, G. Norman, and D. Parker. ‘‘PRISM 4.0: Verification of Probabilistic Real-Time Systems.’’ In: Computer Aided Verification. Springer, 2011, pp. 585–591. doi: 10.1007/ 978-3-642-22110-1_47. T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick. ‘‘Microsoft COCO: Common Objects in Context.’’ In: Computer Vision – ECCV 2014. Springer International Publishing, 2014, pp. 740–755. doi: 10.1007/978-3319-10602-1_48.

17

J. S. Berntzen, S. Fatima, and L. Moonen: Self-Healing Visual Recovery for Autonomous Ground Vehicles . . .

[50]

[51] [52]

[53] [54] [55] [56] [57] [58] [59]

18

A. Mallik, M. L. Gaopande, G. Singh, A. Ravindran, Z. Iqbal, S. Chao, H. Revalla, and V. Nagasamy. ‘‘Real-Time Detection and Avoidance of Obstacles in the Path of Autonomous Vehicles Using Monocular RGB Camera.’’ In: SAE International J. Advances and Current Practices in Mobility 5.2022-01-0074 (2022), pp. 622– 632. doi: 10.4271/2022-01-0074. A. Paszke, A. Chaurasia, S. Kim, and E. Culurciello. ENet: A Deep Neural Network Architecture for Real-Time Semantic Segmentation. 2016. arXiv: 1606.02147. C. Yu, J. Wang, C. Peng, C. Gao, G. Yu, and N. Sang. ‘‘BiSeNet: Bilateral Segmentation Network for Real-time Semantic Segmentation.’’ In: European Conf. Computer Vision (ECCV). 2018, pp. 325–341. R. P. K. Poudel, S. Liwicki, and R. Cipolla. Fast-SCNN: Fast Semantic Segmentation Network. 2019. arXiv: 1902.04502. B. Beyer, C. Jones, J. Petoff, and N. R. Murphy. Site Reliability Engineering: How Google Runs Production Systems. "O’Reilly Media, Inc.", 2016. isbn: 978-1-4919-5118-7. NVIDIA-AI-IOT/Jetbot. Url: https://github.com/NVIDIA- AIIOT/jetbot. 2026. AI Robot Kits from NVIDIA JetBot Partners. Url: https://www. nvidia.com/en- us/autonomous- machines/embedded- systems/ jetbot-ai-robot-kit/. Astra Series: Structured Light Cameras | Orbbec. Url: https : //www.orbbec.com/products/structured- light- camera/astraseries/. Cyberbotics: Robotics Simulation with Webots. Url: https : / / cyberbotics.com/. A. Ayala, F. Cruz, D. Campos, R. Rubio, B. Fernandes, and R. Dazeley. ‘‘A Comparison of Humanoid Robot Simulators: A Quantitative Approach.’’ In: Joint IEEE 10th Int’l Conf. Development and Learning and Epigenetic Robotics (ICDL-EpiRob). 2020, pp. 1–6. doi: 10.1109/ICDL-EpiRob48136.2020.9278116.