1
Managing Task Execution for Unknown Workloads in Batteryless IoT: A Hardware-Agnostic Evaluation
arXiv:2606.24340v1 [cs.LG] 23 Jun 2026
Samer Nasser, Henrique Duarte Moura, Ritesh Kumar Singh, Maarten Weyn, and Jeroen Famaey
Abstract—In recent years, the Internet of Things (IoT) paradigm has been shifting toward batteryless, energy-harvesting architectures. Sustaining reliable operation in these systems requires intelligent management of highly volatile stored energy. As edge applications grow in complexity, traditional energyaware schedulers struggle with unpredictable workloads due to their reliance on static execution thresholds or pre-measured, hardware-specific task profiles. To overcome this, we propose two novel, hardware-agnostic dynamic scheduling strategies treating applications as a "black box," requiring no prior energy information: a model-free Reinforcement Learning (RL) agent and an on-the-fly Approximated Prediction (AP) method. We evaluate these methods against an adaptive task rate approach (AsTAR) and optimized static thresholds using a custom-built, physically accurate simulation framework driven by real-world solar data and dynamic LoRa transmission profiles. Rather than claiming universal superiority, our analysis exposes the distinct operational trade-offs of each method: the AP approach delivers lightweight, near-oracle task throughput; the RL agent provides tunable survival-execution balancing; and AsTAR excels at execution pacing across long energy gaps. Finally, we demonstrate that while these advanced strategies provide critical resilience for severely constrained systems with small capacitors, devices with larger energy buffers can efficiently rely on simpler, less computationally expensive static policies.
I. I NTRODUCTION The Internet of Things (IoT) has grown rapidly in recent years, with deployments expected to increase from around 20 billion devices today up to 40 billion by 2030 [1]. Since most devices are battery-powered, their environmental and economic impact is becoming a major concern. Li-ion batteries contain toxic rare earth materials such as cobalt and nickel and are often improperly recycled, significantly contributing to e-waste and raising serious health concerns [2]. Moreover, rechargeable batteries have a limited number of charging cycles in their lifetime, leading to a large number of batteries needing replacement during long-term and largescale deployment. This also shines a light on the economic complications that come along with battery-powered IoT, as This research was funded by the AMBIENT-6G and CORRELATE project. The AMBIENT-6G project received funding from the Smart Networks and Services Joint Undertaking (SNS JU) of the European Union’s Horizon Europe research and innovation programme under Grant Agreement No 101192113. CORRELATE was realized in collaboration with imec, with project support from VLAIO (Flanders Innovation and Entrepreneurship). S. Nasser, H. D. Moura, R. K. Singh, M. Weyn, and J. Famaey are with the University of Antwerp - imec, IDLab Research Group, Antwerp, Belgium (e-mail: [email protected]). This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible.
these replacements amount to a loss of technical and financial resources that could be used elsewhere. In recent years, ambient IoT and batteryless IoT [3] have emerged as a response to the environmental and economic questions brought about by the rapid proliferation of IoT across all sectors of our modern society. These fields aim to significantly prolong the lifetime of battery-powered IoT devices or even eliminate the need for batteries by leveraging novel energy harvesting techniques, ultra-low-power electronics, and efficient communication protocols, paving the way for (super) capacitors as the main energy storage element. While supercapacitors typically have a higher cost per unit of energy, their extremely long cycle life and high power density make them suitable for many low-power, long-lifetime IoT applications where batteries require frequent and costly replacement [4]. However, their lower energy density compared to batteries also introduces challenges. Unlike batteries, which maintain a relatively stable voltage during discharge, a capacitor’s voltage drops proportionally to the energy it expends. This highly dynamic and non-linear behavior, coupled with the unpredictable nature of ambient energy harvesting, means that the available energy and the device’s operational capabilities are constantly fluctuating. If the capacitor voltage drops below a certain critical level, it can no longer supply the IoT device with power, causing complete system failure. Reliable operation under these dynamic energy conditions thus requires advanced energy management, only executing an application task when the capacitor is sufficiently charged. Static operational strategies, relying on a fixed voltage threshold for task execution, are often insufficient or require hardware-specific manual calibration. Furthermore, static methods typically assume the energy consumption of a task is known in advance. In practice, applications often behave as a "black box" with dynamic energy profiles (e.g., changing energy consumption for transmitting with different payloads or transmit powers), making static prediction highly unreliable. Instead, a dynamic thresholding mechanism is desired, enabling intelligent adaptation to real-time harvested power, stored energy, and unknown task demands to optimize execution and ensure system resilience. To tackle this problem, we propose the general batteryless IoT system design illustrated in Figure 1. In this architecture, a solar panel converts ambient solar energy into electrical power, and the resulting harvesting current (IH ) is directed to a supercapacitor, which serves as the primary energy storage element. The energy stored within the capacitor then powers the microcontroller unit (MCU) and its associated peripherals. To ensure reliable operation under these highly fluctuating
2
Reinforcement Learning
Approximated Prediction
Vt Ih
Execution Decision Logic
Ih
AIMD/MIAD
Solar Energy
Capacitor storage
MCU + Peripherals
Fixed Thresholding
Fig. 1. General system overview of a batteryless ambient IoT system. Harvested solar energy gets transformed to electrical energy through a solar panel, and is stored in a supercapacitor, which powers the MCU and peripherals. The task execution logic within the MCU can be governed by multiple different approaches: Using an RL-based approach, using approximated prediction of the workload, using AIMD/MIAD task rate adaptation, or using a fixed threshold.
conditions, we investigate and compare three distinct dynamic task execution methods: AsTAR [5], which implements an additive increase multiplicative decrease (AIMD)/multiplicative increase additive decrease (MIAD) approach; a novel RLbased approach; and a novel Approximated Prediction (AP)based approach, benchmarking them against an optimized fixed threshold. Crucially, these methods are designed to operate without prior knowledge of the task’s energy profile, while maintaining their adaptability across different physical capacitor sizes. Furthermore, each of these methods is based on a fundamentally different working principle, making them highly valuable for comparison. Rather than simply proving dynamic methods are superior, the core objective of this work is to expose the inherent trade-offs, such as task throughput, system recovery time, and pacing, between these approaches. We aim to identify the practical limits of dynamic thresholding and determine under which specific hardware constraints these advanced methods provide the most value compared to statically optimized baselines. To summarize, the main contributions of this paper are: Hardware-Agnostic, "Black Box" Task Scheduling: We propose two novel dynamic execution strategies capable of intelligently managing energy budgets, irrespective of capacitor size, without prior knowledge of task energy consumption: a model-free RL agent and an on-the-fly AP method. • Comprehensive Batteryless Simulation Framework: We develop a physically accurate simulation environment that couples the non-linear charging dynamics of a capacitor with real-world ambient solar harvesting data and highly variable system load profiles (specifically modeling LoRa Adaptive Data Rate (ADR) transmissions). It is openly available via https://github.com/SamerN97/batteryless-IoT-taskmanagement. • Evaluation of Operational Trade-offs: We provide an extensive comparative analysis of our proposed methods against an AIMD baseline (AsTAR), a short-term oracle, and static thresholds, explicitly detailing the inherent trade-offs between task throughput, system recovery time,
•
and execution pacing. Identification of Practical Limits: We highlight the boundaries of dynamic thresholding, demonstrating that these advanced methods provide the most significant value under severe hardware constraints (i.e., small capacitor sizes), while systems with a larger energy buffer can often rely on simpler, less computationally expensive static strategies. The remainder of this paper is organized as follows: Section II discusses existing literature on energy-aware task scheduling and energy-neutral IoT systems. Section III provides a comprehensive problem formulation and details the mathematical modeling of the capacitor behavior and energy dynamics. Section IV describes the evaluated energy-aware task execution approaches in detail. Section V outlines the experimental setup and the methodology employed for evaluation. Section VI presents and discusses the experimental results. Finally, Section VII concludes the paper and highlights future research opportunities. •
II. R ELATED W ORK The challenge of managing the dynamic energy budget in batteryless, energy-harvesting IoT devices to meet specific operational requirements spans multiple research domains, from fundamental intermittent computing architectures to advanced energy-aware scheduling algorithms. Our work intersects with several of these areas, specifically focusing on dynamic task execution without prior knowledge of energy profiles. A. Intermittent Computing and Energy Management A prominent strategy for managing unreliable power is intermittent computing, allowing systems to safely suspend and resume across frequent power failures. Early works like Mementos [6] rely on frequent state checkpointing, while subsequent models like Alpaca [7] utilize task-based programming to guarantee progress without traditional checkpoints. To mitigate power loss overhead, hardware-software co-designs like Capybara [8] introduce reconfigurable capacitor banks to prevent mid-task failures. Pushing this to the operating system level, frameworks like InK [9] introduce event-driven kernels
3
that preemptively schedule reactive microtasks based on immediate energy. However, these architectures fundamentally embrace a power failure-tolerant paradigm or require specialized programming models to manage the OFF-state. In contrast, our work treats power failures as highly undesirable, focusing on proactive energy management on standard hardware to entirely avoid the OFF-state and maintain continuous capabilities.
strict domain randomization of hardware parameters during training, our agent learns underlying energy dynamics instead of memorizing hardware-specific trajectories. Thus, it acts as an adaptable, hardware-agnostic scheduler capable of dynamically balancing task throughput and system recovery under unpredictable energy availability, without manual recalibration. III. P ROBLEM F ORMULATION AND S YSTEM M ODEL
B. Energy-Aware Task Scheduling To proactively avoid power failures, systems must align active periods with available energy. Traditional operations use static thresholds or predictive models (e.g., exponentially weighted moving average (EWMA) [10], solar forecasting [11]) to adjust duty cycles based on long-term averages, typically assuming deterministic execution costs [12]. As recently highlighted by López et al. [13], designing effective energyaware protocols requires models that accurately reflect tasklevel variability and storage constraints rather than oversimplifying them. Real-time dynamic scheduling aims to address this. At the algorithmic level, AsTAR [5] handles fluctuating, "black box" workloads using an AIMD approach to adjust task rates. While highly effective for long-term pacing, it limits immediate energy utilization. Conversely, model-based dynamic thresholding [14] recalculates the capacitor voltage to maximize immediate throughput based on instantaneous environmental changes. However, such methods still assume a deterministic workload and rely on prior task consumption profiling. Faced with non-deterministic workloads, they rely on worst-case assumptions, causing conservative, sub-optimal scheduling. Our work bridges this gap by proposing hardwareagnostic, dynamic thresholding strategies that treat the application as a highly volatile "black box." By adapting to unknown task profiles on-the-fly, our methods aim to achieve high task throughput without offline profiling or manual calibration. C. Machine Learning for Energy Harvesting IoT Given ambient energy unpredictability, RL has emerged as a promising energy management approach. Early work used tabular Q-learning to dynamically adjust wireless sensor node duty cycles based on stored energy and current harvesting rates [15], [16]. For example, RLMan [17] leverages linear function approximations to achieve an effective, lightweight power management policy. However, these methods, typically trained for a single, fixed hardware configuration, assume a relatively stable or predictable energy cost per active period, focusing primarily on broad duty-cycle modulation. Recently, Deep Reinforcement Learning (DRL) has been applied to handle the continuous state spaces in energy harvesting environments, optimizing metrics such as throughput and latency [18]. While DRL shows promise, it generally bounds its state evaluations to environmental factors and the instantaneous state of charge, relying on fixed hardware constraints. This hardware-specific formulation is less suitable for IoT deployment scenarios with unknown task costs. Our RL-based approach targets "black box" applications with volatile workloads. By using a rich temporal observation space and employing robust RL algorithms coupled with
In batteryless, energy-neutral IoT systems, task execution decisions are tightly coupled to the energy dynamics of the storage element. For such systems, the capacitor voltage V (t) serves as the primary indicator of the available energy budget, directly influencing whether the device remains operational or risks a power failure. The system’s evolution can therefore be modeled as a Markov Decision Process (MDP) in which the next state depends on the current capacitor voltage, the harvesting rate, and the decision to execute or skip a task. At each decision point, the device must choose between two actions: • Execute (E): Perform a task, consuming energy. • No Execute (NE): Go into a low-power sleep mode to accumulate energy. Considering a time difference ∆t, the capacitor’s voltage update V (t+∆t), derived from its electrical model, determines whether the system stays ON or transitions to the OFF-state. Executing a task in an unfavorable energy state can result in V (t + ∆t) < Vmin , forcing a shutdown. Conversely, skipping execution when harvesting conditions are favorable can lead to missed opportunities for useful work. The following subsections formalize this problem in two steps: 1) Capacitor Model — We present the electrical model governing V (t) evolution under harvesting and load, and derive a closed-form update equation suitable for simulation. 2) System Behavior — We describe the operational modes, possible state transitions, and decision outcomes, represented as an MDP, highlighting the consequences of each action on system availability and performance. A. Capacitor Model In a batteryless, energy-neutral system, the storage state is most conveniently tracked via the capacitor voltage V (t), since stored energy is described in Equation (1). Practically, V (t) determines when the system can execute work (e.g., perform a wireless transmission) and enforces safety margins to avoid power failures. To evaluate these decisions in simulation, we use a compact model that links harvesting current and load to the evolution of V (t). E(t) = 21 CV (t)2
(1)
Figure 2 shows an electrical model representing an energyneutral system containing three main components. The first component is the harvester, which is modeled as a current source IH . It is responsible for harvesting ambient energy, such as solar or kinetic energy, and converting it to electrical
4
energy. The second component is the capacitor C, which acts as the main energy storage device of the system. The third, and final component is the load that includes all energyconsuming elements of the system, such as the CPU, radio, and inherent capacitor leakage. They are collectively modeled as an equivalent resistance Req (t) at time t.
C
IH
+ Vt
Req
-
Harvester
Capacitor
Load
Fig. 2. Equivalent circuit of a batteryless IoT device with harvester current source, storage capacitor, and equivalent load Req .
Applying Kirchhoff’s current law (KCL) at the storage node with Ohm’s law and the capacitor i–v relation results in: IH (t) = C
V (t) dV (t) + . dt Req (t)
(2)
V (t) Intuitively, the KCL balance C dV dt = IH (t) − Req (t) shows that the capacitor charges when the harvester current IH (t) exceeds the load current, discharges when the load current exceeds the harvester current, and is in steady state when equality holds. Rearranging Equation (2) results in the following first-order linear differential equation:
dV (t) 1 IH (t) + V (t) = . dt Req (t) C C
(3)
For simulation, we assume IH (t) and Req (t) to be constant within each step ∆t, thus dropping the function notation. Solving the first-order differential equation on [t, t + ∆t] then yields the exact RC step response: − ∆t − ∆t V (t + ∆t) = IH Req 1 − e Req C + V (t) e Req C . (4) This closed-form update is numerically stable for any ∆t > 0 and avoids per-step truncation errors introduced by numerical approximations. However, it should be noted that the physical accuracy of the prediction fundamentally relies on choosing a decision interval ∆t that is sufficiently small to ensure the piecewise-constant assumption for the harvesting current IH and equivalent load Req remains valid. By mapping the continuous-time update to a discrete-time sequence where the next state Vt+1 corresponds to V (t + ∆t), this equation enables direct prediction of the storage voltage for any decision interval ∆t, harvesting current IH , and equivalent load Req . In the context of system control, Req depends on the chosen action: a higher load during task execution (E) and a lower load during sleep (NE). Furthermore, the capacitor leakage can also be simulated as part of the equivalent load. By comparing Vt+1 to the minimum operating voltage Vmin , the system can decide whether it will remain operational or risk a power failure (OFF) state. This link between the electrical model and decision-making forms the basis for representing the system
as an MDP, where each choice directly shapes the next energy state and operational mode. B. System Model The main goal of the system is to optimally utilize the harvested energy to balance high task throughput with longterm system resilience, proactively minimizing the risk of power failures. To achieve this, the device follows a dutycycled execution pattern: it resides in an ultra-low-power sleep mode between periodic wake-ups. At each wake-up, the system samples V (t) to decide whether conditions are favorable to execute a task (E), e.g., sensing, data processing or storage, transmission, reception, etc., or whether it should return to sleep mode (NE) to accumulate more energy. We assume that a task is always ready for execution, but the energy required for task completion may vary depending on the task type. The decision is therefore based on both the available energy budget and the pending task’s specific energy demand. In a dynamic energy-harvesting environment, fixed execution schedules are unreliable: executing a task without sufficient stored energy may cause the capacitor voltage to drop below Vmin , forcing the device into a power failure (OFFstate). Unlike classical intermittent computing systems, where power failures are frequent and computation resumes after state restoration, our approach aims to proactively avoid OFFstate transitions altogether through energy-aware scheduling. Residing in the OFF-state is undesirable for several reasons. First, volatile memory is lost, requiring the system to reinitialize and reload any necessary state after power-up. Second, the device cannot perform sensing, communication, or other useful work while powered off. Finally, recovering from a power failure enforces a prolonged period of mandatory downtime, as the system remains inactive until the capacitor slowly recharges past the hardware’s turn-on threshold (Vto ). Figure 3 illustrates the possible system state transitions that can be determined through Equation (4) with Req chosen according to the action (E or NE). Each state is defined by the device mode (ON or OFF) and the capacitor voltage at times t and t + 1: • Safe outcomes: The capacitor voltage at t + 1 remains above Vmin , and the device stays ON. In transition A , a task is successfully executed; in transition B , the system remains in sleep mode to conserve energy. • Failure outcomes: The capacitor voltage at t+1 falls below Vmin , leading to shutdown. In transition C , even the sleep mode is unsustainable due to insufficient harvesting current; in transition D , a task is attempted without sufficient stored energy, causing a power failure. The core challenge is to determine, at each wake-up, whether executing a task will sustain system operation or cause a power failure. While the energy-harvesting environment may exhibit some degree of predictability, the energy requirements of individual tasks often vary significantly and unpredictably, making accurate decision-making more difficult. For example, the energy required for performing a wireless transmission can change depending on the network quality, the payload, and the connection time. Furthermore, the system’s energy dynamics
5
Vt+1 > Vmin
Vt+1 > Vmin
A
B
ON, Vt+1
thresholding is most effective when the energy consumption of tasks is known in advance, such deterministic energy profiles are not always available in practice. B. AsTAR Approach
E
Decision Logic
NE
Vt+1 < Vmin
C
OFF, Vt+1
ON, Vt D Vt+1 < Vmin
Fig. 3. An overview of the possible evolutions of the system’s state transitions starting from the ON state. E stands for ’execute’ and represents the execution of a task, while NE stands for ’no execute’ and represents the sleep mode. The ovals represent the states of the system, consisting of the device state (ON or OFF) and the capacitor voltage at time t or t + 1. A and B represent desirable state transitions that keep the device in ON-state, while C and D are undesirable transitions putting the device in OFF-state. Vmin is the minimal capacitor voltage threshold for the device to maintain its ON-state, determined by the hardware’s minimal operational voltage. Note that there also exists a transition from the OFF-state to the ON-state, which is omitted from this visual overview for clarity, since it does not depend on the decision logic.
are dependent on the capacitor size. Larger capacitors result in larger energy buffers with longer charge and discharge times. The performance of the decision logic should be independent of the capacitor size. The remainder of this work aims to evaluate multiple decision logic approaches in terms of capacitor-size-agnosticism, successful task completion, OFFstate transition avoidance, and reliability. IV. D ECISION L OGIC A PPROACHES Having modeled the system’s operational behavior as an MDP, this section explores the decision-making policies used to govern task execution. First, we discuss the static thresholding approach, which simply triggers task execution based on a predefined capacitor voltage threshold. Then we move on to AsTAR [5], an adaptive execution mechanism based on AIMD. Finally, we introduce two novel adaptive approaches: the first one relying on an RL agent’s policy and the second one using real-time approximations of the load consumption. A. Static Thresholding Approach Static thresholding is the simplest policy in our evaluation. The decision logic is governed by a fixed, predefined voltage threshold, Vthresh . At each wake-up, the system compares the current capacitor voltage Vt with this fixed value. If Vt ≥ Vthresh , the task will be executed (E); otherwise, the system will return to sleep mode (NE) to preserve its energy. The approach offers extreme simplicity, but its performance is highly dependent on the chosen threshold value. If a conservative approach is chosen with Vthresh ≫ Vmin , the system might miss out on execution opportunities. However, if a more ambitious approach is chosen with Vthresh ≈ Vmin , the system is at high risk of triggering power failures. While static
AsTAR, rather than identifying an optimal voltage threshold for task execution, aims to maintain the capacitor voltage around a specific target point, Voptimal . Consequently, its primary focus is on ensuring stable, perpetual operation rather than simply maximizing the execution frequency. As described in Algorithm 1 of Yang et al. [5], this behavior is achieved by dynamically adjusting the task execution rate based on the current capacitor voltage, Vt . The adaptation logic evaluates Vt against an acceptable margin (m) and applies different control strategies accordingly: • Optimal Voltage (Voptimal − m ≤ Vt ≤ Voptimal + m): The execution rate remains unchanged. • Low Voltage (Vt < Voptimal − m): The execution rate is adjusted using the AIMD principle, relying on a comparison with the previously measured voltage, Vprev . If Vt > Vprev , the task rate (number of executions per day) is incrementally increased by 1. Conversely, if Vt < Vprev , the rate is divided by 2. This creates a steady increase but an aggressive decrease, prioritizing system safety during low-energy states. • High Voltage (Vt > Voptimal + m): The system operates according to the MIAD principle. This allows for a more aggressive increase in the execution rate to safely utilize excess energy, paired with a moderate decrease, which is well-suited for a secure high-voltage state. C. RL-based Approach Another approach is formulating the decision logic as a model-free RL problem. By treating the system’s energy dynamics as an MDP, an RL agent can learn an optimal, adaptive execution policy through continuous interaction with the environment. Unlike classical thresholding, the RL agent does not calculate a definitive voltage limit. Instead, it evaluates the holistic state of the system and infers the (long-term) consequences of executing a task versus sleeping. For this approach, we employ Proximal Policy Optimization (PPO) [19], a highly robust actor-critic algorithm, due to its clipping mechanism. This ensures training stability and prevents catastrophic policy degradation when exposing the agent to the highly volatile, randomized capacitor bounds utilized in our training environment, which is designed with the following core RL components: Observation Space: To make informed decisions, the agent receives a comprehensive, normalized state vector at each wake-up interval. To provide temporal context and allow the agent to perceive trends (e.g., a charging or discharging capacitor), the observation space includes a history window of the most recent steps for the following features: • Capacitor Voltage (V ): The primary indicator of stored energy. • Harvesting Current (IH ): The ambient current being generated.
6
Task Duration (T D): The duration of the last performed task. • Time Since Failure (T SF ): A step counter (e.g., lowpower Real Time Clock (RTC)) that tracks the elapsed time since the last power failure, functioning as a risk decay metric. • Task Parameters: The parameters defining the energy profile of the last performed (transmission) task. • Capacitor Size (C): The total capacitance of the energy storage element. • Energy level (E): The last known energy level of the device, derived from V and C through Equation (1). •
Action Space: The agent outputs a discrete action that maps directly to the MDP formulation described in Section III: Action (NE): Skip execution and go into a low-power sleep state to accumulate energy. • Action (E): Execute the task at hand. •
Reward Formulation: The RL-based approach is flexible as the reward function can be tailored depending on the desired optimization. In this work, the reward function consists of four parts: Positive Task Reward (PTR): The agent receives a positive reward for successfully completing a task execution without power failure. • Positive Inaction Reward (PIR): The agent receives a positive reward for choosing not to execute a task when it would have caused a power failure. • Negative Failure Reward (NFR): The agent receives a negative reward when it has experienced a power failure. • Negative Inaction Reward (NIR): The agent receives a negative reward for choosing not to execute a task when it would have been safe to do so. •
We showcase two specific reward formulations: The first one minimizing the time between successful task executions (inter-task interval (ITI)), and the second one minimizing the total off-time. The exact reward components are formulated as follows: Reward PTR
ITI Opt. T SF 1 + 4 T SF
max
Off-Time Opt. Enorm
PIR
1
1
NFR
max(−0.1Nof f , −1000)
max(−0.1Nof f , −1000)
NIR
−0.5(1 + 4Enorm Inorm )
−0.5Enorm Inorm
The main objective of the ITI Optimization is to enforce consistent task execution by minimizing the variance and bounding the maximum time between successful tasks. To achieve this, the PTR is scaled by the device’s survival streak (T SF with T SFmax = 1000). Its design teaches the agent that maintaining continuous, stable uptime is a necessary prerequisite for frequent execution. However, to prevent the agent from artificially inflating its T SF by simply remaining idle, the penalty for inaction (NIR) is aggressively amplified when both the stored energy and ambient harvesting conditions are high. This forces the agent to actively capitalize on
available energy rather than hoarding it, thereby closing the gaps between executions. The Off-Time Optimization, on the other hand, is designed with a more conservative, survival-first objective: minimizing the absolute time the system spends in a power failure state. Here, the agent’s rewards are tightly coupled strictly to the available energy budget. By scaling the PTR directly with the normalized capacitor energy level Enorm , the agent is incentivized to execute tasks primarily when the capacitor is well-charged, inherently buffering the system against sudden energy depletion. While the NIR still discourages the agent from wasting peak harvesting opportunities, the absence of a T SF multiplier means the agent operates without the aggressive pressure to maintain an execution rhythm, resulting in a safer, highly robust policy. To establish a baseline of system safety, both formulations share a static reward of 1 (PIR) for correctly choosing to sleep when executing a task would have caused a power failure. Furthermore, both utilize a dynamic Negative Failure Reward (NFR) that scales proportionally with the duration of the offstate (Nof f ). This directly aligns the mathematical penalty with the exact physical metric the system aims to minimize, ensuring the agent learns to avoid prolonged periods of failure. Domain Randomization for Generalization: To ensure hardware-agnostic performance, it is important to expose the agent to variability in capacitor sizes during training. This is achieved through domain randomization of the simulated hardware. The capacitor size (C) is randomly mutated across different training episodes. This forces the policy to learn the underlying physics of the capacitor’s charge and discharge behavior rather than memorizing a fixed trajectory for a specific hardware configuration. 1) Energy and Memory Overhead: While the RL agent provides a highly adaptive policy, its execution introduces an inherent computational overhead that must be accounted for within the system’s energy budget. Because the PPO model is trained offline, the MCU is strictly responsible for inference. However, the process of extracting the observation state, loading the network parameters, and executing the mathematical forward pass of the neural network requires active processing time, drawing a higher current than the device’s baseline sleep mode. Consequently, the equivalent electrical load of the agent (Ragent ) and its required execution time (tagent ) must be explicitly factored into the simulation. Furthermore, deploying the RL agent introduces specific memory overheads for the MCU. The compiled actor-network weights and biases must be persistently stored in Flash memory, while runtime RAM is required to maintain the observation history array, which provides the agent with essential temporal context. D. Approximated Prediction Approach Traditional model-based approaches, such as the one presented in [14], rely on prior knowledge of a task’s specific power consumption. Because we treat the application as a "black box" with unknown energy demands, directly applying these models is unfeasible. However, this limitation can be overcome by dynamically approximating the task’s power consumption based on measurable system parameters.
7
Looking back at Equation (4) for simulating the capacitor behavior, the only unknown variable required to predict the future voltage V (t+∆t) is the equivalent load Req . Assuming that a task’s power profile does not change drastically over a short period, we can approximate the current task’s consumption by deriving Req from the execution of the previous task. By measuring the capacitor voltage immediately before (Vt−1 ) and after (Vt ) a task completes, we capture the necessary information to infer Req . Because Req is present both inside and outside the exponential term, rearranging Equation (4) results in an implicit relationship that cannot be solved for Req directly. Therefore, we formulate this as a root-finding problem to numerically approximate Req using the Newton-Raphson method, defining our objective function f (Req ) = 0 as follows: t
f (Req ) = Req + C ln
Vt −IH Req Vt−1 −IH Req
Algorithm 1: Approximated Prediction Calibration Procedure Input: Current voltage Vt , Harvested current IH , Hardware minimum voltage Vmin , Supply voltage Vsupply CalibrateFlag ← True; Ethresh ← Initial Error Threshold; foreach wake-up interval do if CalibrateFlag == True then if Vt > Vthresh then Vt−1 ← Vt ; Execute Task; Measure Vt ; Req ← NewtonRaphson(Vt−1 , Vt , IH , t); Ethresh ← 10 × (Vsupply /Req ); CalibrateFlag ← False; else Sleep(); else
=0
(5)
While the Newton-Raphson solver provides an accurate estimate for Req , each approximation introduces a computational overhead, which translates to an added energy cost. To minimize this impact on the energy budget, we introduce a predict-and-verify execution strategy coupled with a dynamic recalibration procedure, as outlined in Algorithm 1. The system uses the most recently approximated Req to predict the future capacitor voltage Vpredict that would result from executing the pending task. The execution decision is based directly on this prediction: if Vpredict safely exceeds a padded minimum operating threshold Vmin_padded , the system executes the task. To account for estimation inaccuracies, this padding scales proportionally with the approximated task current Itask , such that Vmin_padded = Vmin + k · Itask , where the scaling constant is experimentally determined to be k = 1.5 V/A. This ensures a larger safety buffer for heavier, more power-hungry executions. If the predicted voltage is too low, the system skips the execution and returns to sleep mode to harvest more energy. Following a task execution, the system enters a verification phase to determine if a recalibration is needed. By introducing an error threshold Ethresh , which is dynamically scaled to the currently estimated task consumption (10 · Vsupply /Req ), the system evaluates the accuracy of the cached Req . A full recalibration, consisting of a new Newton-Raphson approximation and an update to Ethresh , is triggered on only two occasions. The first case is when the system undergoes a power failure as a result of a task execution. This absolute failure implies that the current Req was severely underestimated and must be recalculated upon reboot. The second case is when the absolute difference between the predicted voltage Vpredict and the actual measured voltage Vactual after task execution exceeds the error threshold Ethresh . This indicates that the current Req is drifting from the system’s estimate due to a shifting power profile, meaning the model must be recalculated to ensure future reliability. 1) Energy and Memory Overhead: While the AP approach provides a lightweight mechanism for dynamic thresholding, it introduces specific computational and memory overheads.
Vpredict ← PredictVoltage(Vt , IH , Req ); if Vpredict > Vmin_padded then Execute Task(); Vactual ← MeasureVoltage(); if Vactual ≤ Vmin then CalibrateFlag ← True; else if |Vactual − Vpredict | > Ethresh then CalibrateFlag ← True; else Skip Task and Sleep();
The energy costs occur (1) during prediction, where evaluating the exponential update equation with a cached equivalent resistance (Req ) requires brief MCU active time, and (2) during calibration, where it runs a higher cost, iteration-dependent Newton-Raphson solver. To avoid power failures, this should be explicitly modeled in the simulation, and the maximum number of operations should be bound (max 100 iterations used in this work). Regarding memory, non-volatile Flash is required to store the algorithmic solver logic and mathematical libraries (such as floating-point exponentials and logarithms). Runtime RAM requirements remain minimal, needing only enough space to cache recent Req values, pre- and posttask voltage measurements (Vt−1 , Vt ), and the dynamic error threshold (Ethresh ). E. Short-Term Oracle Finally, for the purpose of evaluation, we also introduce a Short-Term (ST) Oracle baseline. The ST Oracle serves purely as a comparative approach and cannot be implemented in realworld scenarios. It operates with perfect short-term knowledge of the outcome of an execution decision; therefore, it always executes a task as long as that specific execution will not immediately cause the device’s capacitor voltage to drop below Vmin and trigger a power failure. Note that this approach is not globally optimal, as it does not look beyond a single task execution. V. E XPERIMENTATION S TRATEGY AND PARAMETERS To evaluate the performance of our proposed adaptive decision logic approaches, we developed a comprehensive
8
simulation framework. This framework accurately models the hardware characteristics, task execution profiles, and energy harvesting dynamics of a realistic batteryless IoT device. The experiments are designed to test the robustness, energy efficiency, and hardware-agnostic capabilities of each scheduling policy under changing environmental conditions. A. System and Task Modeling Our simulated batteryless IoT node is modeled as a fully tasked sensing and communication device. It simulates a central STM32L4 MCU [20] interfacing with an SHT30 temperature and humidity sensor [21], and an SX1262 LoRa communication module [22] capable of variable payloads, spreading factors, and transmit power levels controlled through an ADR mechanism, dynamically adapting the transmission’s power profile to the current wireless channel conditions [23]. For our experiments, the base decision interval for the system wake-ups was set to ti = 30 seconds. At each wake-up, the system always measures Vt (ADC read-out) and IH (Coulomb counter read-out) and performs a temperature measurement. Subsequently, the decision logic is performed before committing to either executing a LoRa transmission or returning to deep sleep. The decision logic, therefore, only applies to the transmission task as this is the main energy consumer within the system. To ensure the physical accuracy of the MDP and the capacitor update equations detailed in Section III, the power profiles of the individual hardware components were explicitly defined. The operational voltage bounds were set to a maximum capacitor voltage Vmax = 5.5 V, a hardware turn-on threshold Vto = 2.3 V, and a minimum operational threshold Vmin = 1.8 V. Dropping below Vmin immediately triggers a power failure (OFF-state). The equivalent load resistance Req for the system dynamically shifts based on the energy consumption of the task. 1) Fixed Energy Consumption Profiles: The baseline operations of the device exhibit deterministic execution times and current draws. These are based on the electrical characteristics defined in the respective component datasheets [20]– [22] operating at a 3.3 V supply. For computational tasks, the MCU clock frequency is assumed to be 1 MHz. The RL policy is implemented with a total of 20,163 parameters. At a clock frequency of 1 MHz, the inference process requires approximately 44,000 clock cycles for multiply-accumulate operations and non-linear activations, resulting in an estimated 53 ms of active processing time when considering 20% control and data overhead. The specific energy consumption for each fixed activity is outlined in Table I. 2) Variable Energy Consumption Profiles: Unlike the baseline operations, several critical components of the system exhibit highly variable energy demands depending on the environmental state, algorithmic behavior, and hardware configuration. These dynamics are explicitly modeled to assess the adaptability of the different scheduling approaches: • LoRa Transmission (ADR): The communication task’s energy profile is highly fluctuating. To simulate realworld dynamic channel conditions, such as log-normal
TABLE I F IXED E NERGY C ONSUMPTION P ER TASK (Vsupply = 3.3V) Component
Current (mA)
Duration (s)
Energy (mJ)
Sleep (standby) Sense temp. Read Vt Read IH Read RL model Run RL model Run AP sim.
0.00065 0.691 0.311 0.091 0.091 0.091 0.091
– 0.00533 0.00005 0.00023 0.00504 0.053 0.0003
– 0.0122 0.00005 0.00007 0.00151 0.0159 0.00009
Note: Capacitor leakage energy consumption not included.
shadow fading caused by environmental changes, we generate an RSSI trace based on a normal distribution. The trace uses a baseline of -110 dBm and is bounded between -135 dBm and -90 dBm. Based on the instantaneous RSSI, the ADR mechanism adjusts the Spreading Factor (SF ∈ [7, 12]) and transmit current (15 mA to 118 mA) according to Semtech specifications [22], mapped in Table II. To simulate varying application demands, the payload (P L) size is randomized between 20 and 255 bytes on a daily basis. The reception window is fixed at 50 ms with a 12 mA current draw. To accurately simulate transmission energy, the total airtime (Tair ) is calculated based on SF and P L. Following standard LoRa specifications, Tair is defined as: Tair = (npreamble + 4.25 + npayload ) ·
2SF BW
where the bandwidth BW = 125, 000 Hz, the programmed preamble npreamble = 8, and the payload symbol count npayload is calculated as: 8P L − 4SF + 44 · CR, 0 npayload = 8 + max 4(SF − 2DE) Here, CR = 5 (for a standard 4/5 coding rate) and the Low Data Rate Optimization flag DE = 1 if SF ≥ 11 (and 0 otherwise). The total transmission energy is then obtained from the product of Tair , the 3.3 V supply voltage, and the dynamic ADR transmit current. • AP Calibration Procedure: When the AP approach triggers a calibration, it relies on a Newton-Raphson solver to estimate the equivalent load resistance. Because the execution time depends on the number of iterations required to converge, its energy consumption is variable. Operating at 1 MHz, each iteration requires approximately 3 ms at an active current draw of 0.091 mA, with the solver capped at a maximum of 100 iterations to prevent infinite loops. • Capacitor Leakage: To ensure hardware-agnostic evaluation, the simulation utilizes varying capacitor sizes. Because inherent leakage current scales with the physical size of the capacitor, a static leakage value is insufficient. Based on the datasheet specifications for the 5.5 V DGH capacitor series [24], we derived a linear regression model to dynamically calculate the leakage current Ileakage (in
9
TABLE II L O R A ADR PARAMETER M APPING BASED ON RSSI (Vsupply = 3.3V) SF
TX Power (dBm)
TX Current (mA)
≤ −125 (−125, −120] (−120, −110] (−110, −100] > −100
12 10 9 8 7
+22 +17 +14 +14 +10
118 58 45 25.5 15
Harvesting Current (A)
RSSI Range (dBm)
0.08 0.06 0.04 0.02 0.00
0
5
10
15
20
25
Time (Days)
30
35
40
Fig. 4. Overview of the solar harvesting validation data used in the experiments with an augmented sample frequency of 30 seconds.
µA) based on the capacitance C (in Farads): Ileakage = 4.7442 · C + 4.9302. B. Environmental Data and Preprocessing To simulate realistic and highly dynamic ambient energy availability, we used a dataset containing solar irradiance data collected from September 2022 until August 2023 in Hveragerði, Iceland. We then converted the irradiance data to harvesting current by considering a solar panel with size 60.1x41.3 mm and efficiency of 18.5% (in line with commercially available panels), approximated cosine loss of 25%, Power Management Integrated Circuit (PMIC) efficiency of 90%, and supply voltage of 3.3 V, resulting in a conversion factor of 0.000093942. Since the original dataset was sampled at 15-minute intervals, we applied linear interpolation to upsample the data, generating 29 intermediate points between each raw sample. This provided a high-resolution harvesting current (IH ) profile that aligns with our 30-second system decision interval. Furthermore, the data was cleaned of "midnight sun" days, chunked into 72-hour segments and randomly shuffled to eliminate seasonal pattern influences. Finally, the data was split into a training set (80%) and a validation set (20%). Comprising 45 days of solar harvesting data, the validation set serves as the final evaluation benchmark for all approaches. As illustrated in Figure 4, this dataset captures a diverse range of weather conditions and daylight hours. C. RL Training and Inference Configuration The RL agent was trained using the PPO implementation from the StableBaselines3 framework [25]. To ensure hardware-agnostic performance, the environment employs domain randomization, i.e., at the start of every training episode,
the capacitor size C is uniformly sampled from a discrete set of values ζ ranging from 0.5 F to 10 F, in increments of 0.5 F, i.e., ζ = {0.5, 1.0, . . . , 10.0} F. To provide the agent with temporal context and allow it to perceive charge/discharge trajectories, the observation space is stacked with a history window of the k = 10 most recent timesteps. The training and environment parameters were configured as follows: • Algorithm: PPO with default MlpPolicy architecture and hyperparameters • Discount factor (γ): 0.99 • Training Episode Length: 8,640 steps (equivalent to 3 physical days) • Total Training Steps: 1,000,000 A critical design choice is decoupling the physical simulation clock from the agent’s steps during power failures. When the device shuts down (Vt < Vmin ), the entire physical recovery period required to reach the turn-on threshold (Vto ) is perceived by the agent as a single, heavily penalized step. This accurately mirrors physical deployment: since the MCU is powered down during a failure, the agent cannot actively experience the passage of time. Furthermore, strict temporal boundaries are maintained: if the physical simulation clock reaches the maximum episode length during this OFF-state, the episode truncates immediately. For the evaluation phase, the inference strategy shifts from episodic training to continuous deployment. The episode length is extended to match the exact size of the validation dataset (45 days, or 129,600 steps). VI. R ESULTS AND D ISCUSSION In this section, we present and analyze the experimental results from our simulated batteryless IoT environment. We compare the performance of the different decision logic approaches across multiple criteria. The evaluation focuses on exposing the inherent trade-offs between task throughput, system reliability, and hardware agnosticism under highly variable solar energy harvesting conditions. A. Evaluation Setup and Metrics To comprehensively evaluate the robustness and adaptability of each decision logic approach, the simulation was run across the full validation dataset of solar harvesting traces. To test hardware-agnosticism, each approach was evaluated for all capacitor sizes in ζ. Furthermore, in addition to an optimized static thresholding approach, we introduced two more threshold levels to represent opposite ends of the operational spectrum. The first, Static (1.9 V), is an aggressive threshold set at 0.1 V above the hardware’s minimum operating voltage (Vmin = 1.8 V), designed to prioritize immediate task execution. The second, Static (3.45 V), uses the optimal threshold level when using a 0.5 F capacitor, and represents a highly conservative approach that prioritizes system survival by demanding a large energy buffer before allowing any task execution.
10
1000 900
700
8.5
8.0
7.5
7.0
6.5
6.0
5.5
5.0
4.5
4.0
3.5
3.0
2.5
2.0
1.5
1.0
0.5
600
Capacitor Size (Farads)
9.5 10 .0
Agent (ITI) Agent (Off-Time) Opt. Static Thresh. Static (1.9V) Static (3.45V) Approx. Pred. ST Oracle AsTAR
800
9.0
Mean Successful Transmissions per 24h
B. Execution rate vs. Reliability 1100
105 104 103 102
.0
9.5
8.0
7.5
10
Capacitor Size (Farads)
7.0
6.5
6.0
5.5
5.0
4.5
4.0
3.5
3.0
2.5
2.0
1.5
1.0
100
9.0
101
8.5
Agent (ITI) Agent (Off-Time) Opt. Static Thresh. Static (1.9V) Static (3.45V) Approx. Pred. ST Oracle AsTAR
0.5
Median Continuous ON-State Duration (Steps, Log)
Fig. 5. Mean daily successful executions by approach for capacitor sizes 0.5-10 F (0.5 F steps).
Fig. 6. Median time between off-states by approach for capacitor sizes 0.5-10 F (0.5 F steps).
The performance of each policy was assessed using five metrics: Mean daily successful executions: The average number of tasks successfully completed within a 24-hour period without causing the capacitor voltage to drop below the minimum operational threshold (Vt < Vmin ). • Median time between off-states (survival time): The median continuous duration the system maintains an operational state (Vt ≥ Vmin ) before experiencing a power failure. • Mean inter-task interval (ITI): The average elapsed time between two consecutive successful task executions across the entire evaluation period. • Median daily maximum inter-task interval (ITI): The median of the single longest interval between consecutive successful executions recorded for each 24-hour period. This metric specifically captures the system’s ability to pace its energy consumption across prolonged harvesting gaps (e.g., nighttime). • Median continuous off-state duration (recovery time): The median time the system remains non-operational after a power failure before the capacitor accumulates sufficient energy to reach the hardware turn-on threshold (Vt ≥ Vto ).
•
A central challenge in batteryless IoT is balancing the desire for a high execution rate with the necessity of system reliability. This trade-off becomes clear when comparing the mean daily executions against the median time between off-states. The results demonstrate a "Boom and Bust" principle among the more greedy methods. Figure 5 shows approaches such as the ST Oracle, Optimal Static Threshold, Static (1.9 V), and AP achieving the highest number of daily successes, already exceeding 1000 transmissions per 24 hours starting from the 1-1.5 F capacitor size. It is important to explicitly highlight the performance of the AP approach here: while the ST Oracle relies on perfect near-future knowledge and the Optimal Static Threshold requires prior optimization, AP achieves nearOracle throughput without requiring any prior knowledge of the task or capacitor size. It dynamically calculates on-thefly what the optimal static method must find through prior trial and error. However, Figure 6, indicates that an aggressive execution profile comes at a cost to reliability. These greedy methods exhibit significantly lower median times between offstates, indicating more frequent power failures. While these approaches maximize the total number of data transmissions, the resulting increase in off-time implies a loss in continuous baseline sensing capabilities (e.g. temperature monitoring), as the device is entirely inactive during these blackout periods. C. Pacing To understand how executions are distributed over time, we analyzed the mean ITI and the daily maximum ITI. Looking at Figure 7, the AsTAR approach consistently maintains the highest average time between executions across all capacitor sizes, and therefore the lowest average execution rate. However, examining Figure 8, the daily maximum ITI reveals AsTAR’s distinct advantage. For all other approaches, the median maximum ITI flatlines at approximately 103 steps (roughly 8 hours), which directly correlates to the nighttime period where solar harvesting is zero. While AsTAR executes less frequently on average, it is specifically designed to effectively bridge this nighttime gap. For capacitor sizes larger than 2.5 F, AsTAR drastically reduces the maximum daily ITI to near 102 steps (around 1 hour). It is the only evaluated approach that successfully paces its energy consumption to retain active communication capabilities throughout the night, eliminating the standard 8-hour communication blackout. In scenarios where night-time transmissions are strictly required, AsTAR therefore presents itself as the preferred approach. D. System Resilience and Recovery Times System resilience is not only defined by avoiding power failures but also by how quickly a device can recover when one occurs. Figure 9 highlights major differences in recovery behaviors. The Static (3.45 V) and the Agent (Off-Time) approaches stand out by effectively eliminating power failures completely for the vast majority of capacitor sizes, thereby maximizing total device up-time. Interestingly, in the highly constrained, low-capacitor region (0.5 F and 1 F), the RL
Fig. 7. Mean time between successful executions by approach for capacitor sizes 0.5-10 F (0.5 F steps).
9.0
8.0
7.5
7.0
6.5
6.0
5.5
5.0
4.5
4.0
3.5
3.0
2.5
2.0
100
8.5
Agent (ITI) Agent (Off-Time) Opt. Static Thresh. Static (1.9V) Static (3.45V) Approx. Pred. ST Oracle AsTAR
101
Capacitor Size (Farads)
9.5 10 .0
Capacitor Size (Farads)
9.5 10 .0
9.0
8.5
8.0
7.5
7.0
6.5
6.0
5.5
5.0
4.5
4.0
3.5
3.0
2.5
0.5
0
2.0
1
1.5
Agent (ITI) Agent (Off-Time) Opt. Static Thresh. Static (1.9V) Static (3.45V) Approx. Pred. ST Oracle AsTAR
1.5
2
102
1.0
3
103
0.5
Median Continuous OFF-State Duration (Steps, Log)
4
1.0
Mean Time Steps Between Successes
11
Fig. 9. Median of the continuous off-state duration by approach for capacitor sizes 0.5-10 F (0.5 F steps). Notice the Agent (off-time) and Static (3.45) approaches completely eliminating off-time starting from 2 F and 1 F respectively.
Median Max Daily ITI (Steps, Log)
103
Agent (ITI) Agent (Off-Time) Opt. Static Thresh. Static (1.9V) Static (3.45V) Approx. Pred. ST Oracle AsTAR
.0
9.5
9.0
8.5
8.0
7.5
10
Capacitor Size (Farads)
7.0
6.5
6.0
5.5
5.0
4.5
4.0
3.5
3.0
2.5
2.0
1.5
1.0
0.5
102
Fig. 8. Median of the daily maximum time between executions by approach for capacitor sizes 0.5-10 F (0.5 F steps).
Agents and the AP approach show specific recovery adaptations. In this range, they manage to keep off-state durations remarkably low, successfully waking up and establishing operation faster than the static or Oracle approaches. Furthermore, in the 1.5 F to 6 F range, the AP approach also manages to keep the off-state duration lower than the Optimized Static approach while generally outperforming it in terms of survival time and achieving equal performance in mean daily executions. This demonstrates that AP’s dynamic recalibration not only matches the throughput of statically optimized baselines but actively protects the system better during recovery phases. E. Hardware Agnosticism A major limitation of static thresholding is the need for manual optimization depending on the specific hardware deployment. The experimental results show that a low threshold like Static (1.9 V) struggles to keep the device alive in the 0.5 F to 5.5 F range, leading to massive drops in survival time (Figure 6). However, the survival time for Static (1.9 V) significantly increases at 5.5 F. Because the stored energy scales with capacitance (c.f. Equation (1)), 1.9 V now represents enough residual energy to prevent a drop below Vmin . Conversely, picking a highly conservative threshold like Static (3.45 V) ensures device survival across the board, but leaves large amounts of harvested energy unutilized, resulting in missed execution opportunities and a lower daily
success rate (Figure 5). The adaptive methods, particularly the RL agents and AP approach, demonstrate strong hardware agnosticism. They dynamically adjust their execution logic without requiring manual recalibration, maintaining a better balance between survival and throughput across the entire 0.5 F to 10.0 F range. Beyond its hardware agnosticism, the AP approach is particularly notable for its minimal computational footprint. While the RL agents provide excellent adaptability, their deployment requires a more significant MCU resource footprint due to the complexity of neural network inference. In contrast, the AP approach operates as a mathematically direct, lightweight solver. Although it lacks the fine-grained tunability of the RL framework, it provides a highly efficient alternative that achieves hardware-agnosticism with minimal computational and memory requirements. F. ITI vs. Off-Time Optimization The flexibility of the RL-based approach allows for tailoring the system’s behavior via different reward formulations. Comparing the ITI-optimized agent with the Off-Time-optimized agent reveals two distinct operational strategies. The ITIoptimized agent is inherently designed to trade more total offtime in exchange for smaller task spacing and higher overall daily executions by taking more aggressively driven execution decisions. On the other hand, the Off-Time-optimized agent more strongly prioritizes survival. It minimizes blackout durations and entirely avoids power failures for all capacitor sizes starting from 2 F, but this extreme safety comes at the cost of executing fewer tasks and allowing a slightly higher average ITI. This comparison validates that the RL framework can be effectively tuned according to the specific demands of the IoT application. In this work, we have explored agent optimizations for ITI and survival, but the RL-based approach is definitely not limited to these configurations. VII. C ONCLUSION AND F UTURE W ORK This paper investigated the performance trade-offs of various hardware-agnostic, dynamic task execution strategies for batteryless, energy-harvesting IoT devices. By treating the
12
application as a "black box," we evaluated a model-free RL agent, an Approximated Prediction method, an additive increase multiplicative decrease-based approach (AsTAR), and static baselines against a highly dynamic solar-harvesting environment. A key takeaway from our extensive evaluation is that no single decision logic is universally superior; rather, each approach offers distinct benefits tailored to specific operational requirements. Our findings demonstrate that assessing the system’s inherent hardware constraints is a critical first step in addressing energy volatility. When form-factor and economic constraints allow for a large energy buffer (e.g., > 5.5 F in our evaluated scenarios), simple and computationally inexpensive static thresholding policies are highly effective. In such less constrained systems, the advanced dynamic approaches evaluated in this work do not offer significant enough benefits to justify their inherent computational and memory overheads, serving as an important boundary regarding the practical utility of dynamic thresholding. However, when strict hardware limitations force the system to operate with small capacitors, dynamic execution strategies become essential for maintaining continuous operation. Under these severe constraints, our proposed AP approach provides a lightweight, real-time adaptation mechanism that closely matches optimal oracle throughput without requiring manual calibration. Alternatively, if the application requires a highly tunable balance between task execution and system survival, the RL-based approach offers a flexible, hardware-agnostic policy. Finally, for applications that specifically require consistent execution pacing to bridge long harvesting gaps (such as night-time communication), AsTAR remains the most effective strategy. Future work will focus on deploying these adaptive strategies onto physical hardware to validate the simulated energy overheads of the decision logic under real-world MCU constraints. Additionally, exploring hybrid policies that seamlessly switch between static thresholding during peak harvesting hours and dynamic evaluation during energy-scarce periods presents a promising avenue for further optimizing batteryless, energy-neutral IoT systems. R EFERENCES [1] IoT Analytics. (2024) State of iot 2024: Number of connected iot devices growing 13% to 18.8 billion globally. Accessed on 22 August 2025. [Online]. Available: https://iot-analytics.com/number-connected-iot-devices/ [2] Y. Zhao, O. Pohl, A. I. Bhatt, G. E. Collis, P. J. Mahon, T. Rüther, and A. F. Hollenkamp, “A review on battery market trends, second-life reuse, and recycling,” Sustainable Chemistry, vol. 2, no. 1, p. 11, 2021. [3] 3GPP, “Ambient iot: Redefining wireless communication for industry 4.0,” https://www.3gpp.org/technologies/anbient-iot-tsdsi, 3rd Generation Partnership Project (3GPP), Tech. Rep., 2025, accessed on 22 August 2025. [4] A. Townsend and R. Gouws, “A comparative review of lead-acid, lithium-ion and ultra-capacitor technologies and their degradation mechanisms,” Energies, vol. 15, no. 13, p. 4930, 2022. [5] F. Yang, A. S. Thangarajan, G. S. Ramachandran, W. Joosen, and D. Hughes, “Astar: Sustainable energy harvesting for the internet of things through adaptive task scheduling,” ACM Trans. Sen. Netw., vol. 18, no. 1, Oct. 2021. [Online]. Available: https: //doi.org/10.1145/3467894
[6] B. Ransford, J. Sorber, and K. Fu, “Mementos: System Support for Long-Running Computation on RFID-Scale Devices,” in Proceedings of the 16th International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS). ACM, 2011, pp. 159–170. [7] K. Maeng, A. Colin, and B. Lucia, “Alpaca: intermittent execution without checkpoints,” Proc. ACM Program. Lang., vol. 1, no. OOPSLA, Oct. 2017. [Online]. Available: https://doi.org/10.1145/3133920 [8] A. Colin, E. Ruppel, and B. Lucia, “A Reconfigurable Energy Storage Architecture for Energy-Harvesting Devices,” in Proceedings of the 23rd International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS). ACM, 2018, pp. 767– 781. [9] K. S. Yıldırım, A. Y. Majid, D. Patoukas, K. Schaper, P. Pawelczak, and J. Hester, “Ink: Reactive kernel for tiny batteryless sensors,” in Proceedings of the 16th ACM Conference on Embedded Networked Sensor Systems, ser. SenSys ’18. New York, NY, USA: Association for Computing Machinery, 2018, p. 41–53. [Online]. Available: https://doi.org/10.1145/3274783.3274837 [10] A. Kansal, J. Hsu, S. Zahedi, and M. B. Srivastava, “Power Management in Energy Harvesting Sensor Networks,” ACM Transactions on Embedded Computing Systems (TECS), vol. 6, no. 4, pp. 32–es, 2007. [11] A. Cammarano, C. Petrioli, and D. Spenza, “Pro-energy: A novel energy prediction model for solar and wind energy-harvesting wireless sensor networks,” in Proceedings of the 2012 IEEE 9th International Conference on Mobile Ad-Hoc and Sensor Systems (MASS), ser. MASS ’12. USA: IEEE Computer Society, 2012, p. 75–83. [Online]. Available: https://doi.org/10.1109/MASS.2012.6502504 [12] C. Moser, L. Thiele, D. Brunelli, and L. Benini, “Adaptive power management in energy harvesting systems,” in 2007 Design, Automation & Test in Europe Conference & Exhibition, 2007, pp. 1–6. [13] O. L. A. López, M. Ashraf, S. Nasser, G. M. de Jesus, R. K. Singh, M. C. Filippou, and J. Famaey, “Foundations for energy-aware zero-energy devices: From energy sensing to adaptive protocols,” 2025. [Online]. Available: https://arxiv.org/abs/2507.22740 [14] A. Sabovic, A. K. Sultania, C. Delgado, L. D. Roeck, and J. Famaey, “An Energy-Aware Task Scheduler for Energy-Harvesting Batteryless IoT Devices,” IEEE Internet of Things Journal, vol. 9, no. 22, pp. 23 097–23 114, Nov. 2022. [Online]. Available: https: //ieeexplore.ieee.org/document/9803046/ [15] S. Kosunalp, “A new energy prediction algorithm for energy-harvesting wireless sensor networks with q-learning,” IEEE Access, vol. 4, pp. 5755–5763, 2016. [16] R. Chaoming Hsu, C.-T. Liu, and W.-M. Lee, “Reinforcement learningbased dynamic power management for energy harvesting wireless sensor network,” in Next-Generation Applied Intelligence, B.-C. Chien, T.-P. Hong, S.-M. Chen, and M. Ali, Eds. Berlin, Heidelberg: Springer Berlin Heidelberg, 2009, pp. 399–408. [17] F. Ait Aoudia, M. Gautier, and O. Berder, “Rlman: An energy manager based on reinforcement learning for energy harvesting wireless sensor networks,” IEEE Transactions on Green Communications and Networking, vol. 2, no. 2, pp. 408–417, 2018. [18] B. Zhao and X. Zhao, “Deep reinforcement learning resource allocation in wireless sensor networks with energy harvesting and relay,” IEEE Internet of Things Journal, vol. 9, no. 3, pp. 2330–2345, 2022. [19] J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal policy optimization algorithms,” arXiv preprint arXiv:1707.06347, 2017. [Online]. Available: https://arxiv.org/abs/1707.06347 [20] STMicroelectronics, STM32L412xx Datasheet: Ultra-low-power Arm Cortex-M4 32-bit MCU+FPU, 100DMIPS, up to 128KB flash, 40KB SRAM, analog, ext. SMPS, STMicroelectronics, 2022, rev 9, Accessed: March 10, 2026. [Online]. Available: https://www.st.com/resource/en/ datasheet/stm32l412kb.pdf [21] Sensirion AG, Datasheet SHT3x-DIS: Humidity and Temperature Sensor, Sensirion AG, December 2022, version 7, Accessed: March 10, 2026. [Online]. Available: https://sensirion.com/media/documents/ 213E6A3B/63A5A569/Datasheet_SHT3x_DIS.pdf [22] Semtech Corporation, SX1261/2 Datasheet: Long Range, Low Power, sub-GHz RF Transceiver, Semtech Corporation, 2020, accessed: March 10, 2026. [Online]. Available: https://www.semtech.com/products/ wireless-rf/lora-connect/sx1262 [23] The Things Network, “Adaptive data rate (ADR),” 2026, accessed: March 10, 2026. [Online]. Available: https://www.thethingsnetwork.org/ docs/lorawan/adaptive-data-rate/ [24] Cornell Dubilier Electronics, “DGH Series Supercapacitor Datasheet,” Cornell Dubilier Electronics, Datasheet, 2024, accessed: 2026-03-16. [Online]. Available: https://www.cde.com/resources/catalogs/DGH.pdf
13
[25] A. Raffin, A. Hill, A. Gleave, A. Kanervisto, M. Ernestus, and N. Dormann, “Stable-baselines3: Reliable reinforcement learning implementations,” Journal of Machine Learning Research, vol. 22, no. 268, pp. 1–8, 2021. [Online]. Available: http://jmlr.org/papers/v22/ 20-1364.html
Samer Nasser Received his B.Sc. and M.Sc degrees in Electronics and ICT Engineering Technology from the University of Antwerp, Belgium, in 2020 and 2021, respectively. After working as an earlystage researcher in the field of environmental monitoring technology at DMR in Aalborg, Denmark, he is currently pursuing a Ph.D. in Applied Engineering at the University of Antwerp within the IDLab research group (University of Antwerp and IMEC). His work focuses on ambiently powered intelligent system design for sustainable IoT applications.
Henrique Duarte Moura is a senior researcher at the University of Antwerp and imec, Belgium. He is a member of the Perceptive Radio Systems team in the IDLab research group. He received his B.Sc degree in electrical and electronics engineering from the Universidade Federal de Minas Gerais (UFMG), Brazil, in 1991 and in system information from Universidade Estácio de Sá, Brazil, in 2012. He received his M.Sc. and Ph.D. degrees in Computer Science from UFMG, Brazil, respectively, in 2015 and 2019.
Ritesh Kumar Singh is a Principal Research Fellow at the University of Antwerp and a senior researcher at IMEC, Belgium. He is a member of the IDLab research group, where he leads the low-power portfolio. He received his M.Tech in Information Technology from IIIT, Allahabad, India in 2012 and subsequently worked in LG & TCS research labs. He obtained his Ph.D. in Applied Engineering at the University of Antwerp in 2022. His current research interests include energy-aware computing, 6G, ML for low-power devices, and sustainable IoT.
Maarten Weyn is a full professor and ViceRector of Research and Impact at the University of Antwerp. He teaches wireless communication system. His research at imec-IDLab focuses on ultra-low power sensor communication, embedded systems, sub-1 GHz communication, sensor processing, and localization. Maarten co-founded spinoffs Aloxy, CrowdScan, IoSa, and AtSharp, and contributed to 1OK and Viloc.
Jeroen Famaey is an associate professor at the University of Antwerp, Belgium, and a senior researcher at IMEC, Belgium. His current research interests include low-power distributed machine learning and wireless communications for Ambient IoT devices, as well as data-driven integrated sensing and communications. He has co-authored over 200 papers, published in international peer-reviewed journals and conference proceedings.