Received XX Month, XXXX; revised XX Month, XXXX; accepted XX Month, XXXX; Date of publication XX Month, XXXX; date of current version XX Month, XXXX.
arXiv:2607.00056v1 [cs.IT] 30 Jun 2026
Active Sensing for RIS-Aided Tracking and Power Control: A Hybrid Neuroevolution and Supervised Learning Approach George Stamatelis1 (Student Member, IEEE), Hui Chen 2 (Member, IEEE), Henk Wymeersch2 (Fellow, IEEE), and George C. Alexandropoulos1 (Senior Member, IEEE) 1
Department of Informatics and Telecommunications, National and Kapodistrian University of Athens, Panepistimiopolis Ilissia, 16122 Athens, Greece. 2 Department of Electrical Engineering, Chalmers University of Technology, 412 58 Gothenburg, Sweden. Corresponding author: George C. Alexandropoulos (email: alexandg@ di.uoa.gr).
A preliminary version of this manuscript has been presented at the IEEE ICASSP, Barcelona, Spain, May 2026 [1]. This work has been supported by the Smart Networks and Services Joint Undertaking project 6G-DISAC under the European Union’s Horizon Europe research and innovation programme under Grant Agreement No 101139130. G. Stamatelis was supported by the Hellenic Foundation for Research and Innovation (HFRI) under the 5th Call for HFRI PhD Fellowships (Fellowship Number: 21080).
ABSTRACT This paper studies energy efficient tracking of power-limited mobile users with the
assistance of a Reconfigurable Intelligent Surface (RIS). Since localization pilot transmissions dominate the energy budget of power-constrained devices, we introduce a low-overhead feedback link from the Base Station (BS) to the user to enable dynamic uplink power control. To navigate the discrete and decentralized nature of this active sensing problem, we propose a novel Dual-Agent (DA) deep learning framework that jointly optimizes the discrete RIS phase profiles and the UE’s transmit power in real time. Specifically, our approach employs a hybrid training methodology integrating the neuroevolution paradigm with supervised learning, effectively overcoming the nondifferentiability of discrete phase responses from the RIS unit elements and the strict information bottleneck of single-bit feedback messages for pilot power control. The proposed DA active sensing framework can be applied with both single- and multi-antenna BSs, the latter with only minor modifications in the structure of one NN: an additional output branch with appropriate structure is included for the latter case to select a valid digital combiner from a finite set. Extensive numerical simulations demonstrate that the proposed scheme achieves highly accurate and robust tracking across diverse target motion models, outperforming extended Kalman and particle filters, as well as, machine learning-based trackers. Furthermore, in static localization, it is shown to significantly outperform traditional fingerprinting schemes, deep reinforcement learning baselines, and standard backpropagation-based estimators. Tracking, beamforming, power control, multi-agent systems, neuroevolution, reconfigurable intelligent surface.
INDEX TERMS
I. INTRODUCTION
Precise device and object location knowledge can significantly enhance modern wireless applications, including robotic navigation [2], intelligent vehicles and traffic management [3], [4], Internet of Things (IoT) [5], [6], as well as assisted living [7]. However, accurate location
estimation in multipath environments is a challenging problem, especially when trying to localize power-limited lightweight IoT devices. To this end, Reconfigurable Intelligent Surfaces (RISs) [8] have emerged as a powerful candidate for enhancing the performance of various
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ VOLUME ,
1
Stamatelis et al.:
communication, localization, and sensing schemes [9]– [14]. An RIS consists of a large number of low-cost, passive elements that can intelligently manipulate electromagnetic waves, typically radio waves, to enhance the performance of wireless networks. When a signal is transmitted, it encounters obstacles or interference, and the RIS intelligently manipulates the reflection of these signals to enhance their propagation. By controlling the characteristics of the reflected waves, RIS can direct the signal to specific locations, improve coverage, reduce interference, and enhance overall signal quality. This adaptive manipulation of signals is achieved with lowpower consumption, making RIS an efficient solution for optimizing wireless communication in complex environments, such as urban areas or indoor spaces. Naturally, RISs are envisioned as one of the core technologies for the upcoming (6G) generation of wireless networks [15]– [17]. However, in order to harvest the full potential of this technology, careful tuning of their elements is necessary. Most practical hardware RIS implementations come with discrete and even binary RIS phase profiles [18], [19], making it practically impossible to find optimal solutions, as discrete optimization is NP-hard. To that end, various approximate solution methods for a wide variety of applications have been proposed, most of whom are based on machine learning. For instance, Deep Reinforcement Learning(DRL) [20]–[22], neural contextual bandits [23], and even NeuroEvolution (NE) [24] have been applied to adaptive RIS control problems. A. Background and Related Works a: Active Sensing and Localization
Active sensing refers to the adaptive, online reconfiguration of environmental parameters to enhance the performance of a specific estimation task. The theoretical foundations of this field were established by Chernoff in his seminal work on Active Hypothesis Testing (AHT) [25]. Chernoff introduced an information-theoretic criterion for sequentially selecting the most informative experiments within a hypothesis testing framework. While his original formulation focused on optimizing medical trials, the underlying model has since found broad applicability in wireless communications, including radar-assisted target classification [26], and sensor networks [27]. More recently, research in signal processing has successfully bridged classical theory with modern learning-based approaches, combining traditional AHT formulations with data-driven artificial Neural Network (NN) algorithms (see, for example, [28]–[31]. In position estimation problems, like the ones considered in this paper, a single User Equipment (UE) transmits pilot signals to a Base Station (BS), and the latter receives the positioning pilot signals through 2
their direct link and reflections from scatterers. To this end, RISs constitute an important infrastructure component in environments where multiple obstacles obstruct the direct UE-BS path [18], enabling dynamically programmable RIS-induced reflections. In such RISempowered systems, active sensing indicates adaptive online selection of the elements of the RIS. In fact, it has been experimentally shown to significantly outperform passive RIS-enabled localization algorithms [32]. More specifically, [32] designed an NN algorithm for jointly manipulating the RIS profiles actively (control task), and estimating the true UE location (estimation task). The algorithm, based on Long Short Term Memory (LSTM) models [33], maintains a hidden state representing important knowledge from past observation, and uses that along with the most recent observation in order to select the next RIS phase profile. Once transmissions have stopped, hidden state features are passed to another NN responsible for estimating the UE’s position. The entire parameter set (RIS selection module and estimation module) were trained as one network with backpropagation based optimizers. An extension of this algorithm for localization services with privacy risks was proposed, very recently, in [34], where the RIS was tasked to avoid sending strong reflections to problematic areas where malicious third parties are located. A Lagrangian-based optimization algorithm that balances an estimation objective with a privacy criterion was designed in order to enforce location leakage constraints. However, the aforedescribed research works are limited to idealized RIS models ignoring practical hardware impairments and limitations. In practical hardware models, RIS elements are typically limited to finite sets of phase configuration values due to hardware quantization [16], [18], [19], [35]–[37], which introduces additional challenges. To this end, at each time instance, the NN controlling the RIS has to output a discretevalued vector corresponding to the next phase profile, and future profile selections, as well as the final estimation depend on these outputs, meaning that the optimization objective becomes non-discrete (and hence non-differentiable). This feature implies that typical backpropagation optimizers cannot be applied to such objective functions. In addition to overlooking these discrete hardware constraints, existing approaches are strictly limited to static localization scenarios. To bridge this gap, in this paper, we present a novel estimation framework that extends beyond stationary positioning to enable the real-time tracking of mobile trajectories in multipath environments. The state-of-the-art assumes fixed transmission power for the localization pilot signals, which can be highly inefficient for dynamic trajectories. In practical IoT deployments, devices operate under strict energy constraints where continuous, high-power pilot transmissions rapidly
VOLUME ,
drain battery life [38]–[40]. Conversely, a fixed low-power transmission risks losing track of a UE of interest during sudden maneuvers or severe fading. To resolve this, we herein introduce dynamic power control into the active sensing loop. By leveraging the LSTM’s hidden state, which inherently encapsulates the system’s uncertainty regarding UE’s trajectory, our framework is designed to learn to adjust pilot transmission power intelligently over time. In particular, it conserves energy during predictable movement phases, and selectively boosts transmission power only when necessary to refine the location estimate or recover from severe attenuation. This joint optimization of discrete RIS configurations and temporal power allocation establishes a highly energyefficient tracking protocol, perfectly aligned with the resource-constrained requirements of 6G IoT networks. b: RIS-Assisted Tracking Algorithms
Besides static UE location estimation, RISs have been also used to improve the performance of tracking systems due to their inherent ability to dynamically improve coverage on different areas [41]–[43]. However, the reliance on unrealistic RIS response models remains a critical bottleneck in the existing tracking literature: state-ofthe-art algorithms predominantly assume continuously tunable phase responses to facilitate real-time optimization. Beyond these hardware constraints, current works typically demand computationally prohibitive operations at every time step (such as message passing on graphs, heavy matrix inversions, and iterative gradient descent), further hindering their practical applicability. Particularly, in scenarios with high UE mobility, this computational latency is unacceptable, since a fast-moving target will have already transitioned to a new state by the time the algorithm converges. Finally, the available methods are strictly tailored to fully specified analytical channel models, limiting their generalizability to complex, real-world propagation environments. Conversely, data-driven approaches offer a robust alternative; they are inherently model-agnostic and capable to learn the underlying propagation mechanics autonomously, completely bypassing the need to hardcode rigid analytical channel expressions into the NN architecture [32]. c: Learning-Based System Configuration
NE algorithms reframe the training of NN models as a stochastic global search problem, which is typically solved via evolutionary or genetic strategies. These methods maintain a population of individuals, each representing the parameters of a policy NN, and iteratively refine this population using evolution-inspired operators [44]. While the application of NE to sequential decision-making is a well-established concept [45], [46], it has recently been demonstrated that this tool can rival, or even surpass, state-of-the-art DRL algorithms [47], VOLUME ,
[48], establishing a robust benchmark in wireless communication domains [24], [31]. Notably, even very old and simple evolutionary algorithms can yield near stateof-the-art performance on popular DRL testbeds [49]. A paramount advantage of NE is its gradient-free nature, which avoids the instability issues—such as vanishing or exploding gradients—that often plague typical deep learning and DRL optimization schemes associated with backpropagation through time [46]. This characteristic is particularly advantageous for online RIS configuration control, given the large number of the constituent metamaterials with quantized electromagnetic responses; such discrete constraints create a rugged optimization landscape that is not naturally compatible with differentiation-based algorithms. Furthermore, although DRL can provide strong decision-making capabilities, its effectiveness often hinges on the availability of welldefined, dense reward functions. Solving problems with multiple conflicting constraints makes the construction of such reward signals challenging and limiting. In contrast, NE operates effectively with sparse feedback, requiring only a scalar fitness score to be evaluated at an episode’s conclusion. B. Paper’s Contribution
The contributions of this paper are summarized as follows: • Problem definition: We formulate the active uplink tracking problem for power-limited UEs aided by an RIS. Distinct from previous studies, our model strictly incorporates practical hardware and system constraints; in particular, i) discrete phase shifts for the RIS elements, and ii) a limited feedback capacity for uplink power control, restricting the BS-UE control link to messages of1 a single bit. Besides localization of static UEs considered by prior works [32], [34] we also consider position estimation for moving UEs. • Novel multi-agent algorithm: We develop a novel Dual-Agent (DA) NE framework to solve the proposed non-differentiable joint control problem of the RIS phase configuration and the uplink UE transmission power. The algorithm features two collaborating Recurrent NNs (RNNs): i) a BS agent that actively tunes the discrete RIS configuration, and ii) a UE agent that adaptively manages pilots transmit power by interpreting the history of binary feedback. By optimizing these agents via evolutionary strategies, we bypass the non-differentiability 1 As will be explained later on, the proposed uplink power control
frameowrk requires only trivial modifications to incorporate multibit feedback. However, we chose to focus on the more challenging case of single-bit feedback accommodating to ultra-low-power IoT receivers [50], [51].
3
Stamatelis et al.:
issue of discrete hardware, enabling efficient joint learning.
Extensive numerical experiments are presented revealing that the proposed algorithm strictly satisfies the required power budget constraints, while achieving superior performance compared to traditional filters [52], [53] and supervised RNN trackers. Furthermore, on static localization, our approach outperforms fingerprinting [12], purely supervised, and DRL baselines. In addition, our simulation studies demonstrate that our single-bit feedback algorithm virtually matches the performance of NE variants utilizing expensive scalar control links, and exhibits minimal performance drop-off compared to schemes using full transmission power for the pilot signals. Compared to the preliminary conference version in [1], this work extends the power-constrained active sensing framework to support the continuous tracking of mobile UEs and the configuration of multi-antenna BSs. In addition, a comprehensive performance evaluation under varying scattering conditions, including a sensitivity analysis of the NE hyperparameters, is presented. Furthermore, we investigate the design tradeoffs of the single-bit control link, demonstrating how the proposed learned collaborative protocol mitigates the information bottleneck to experimentally outperform traditional tracking filters [52], [53] as well as localization baselines [12].
C. Notation and Organization
Lower case bold letters are used to represent vectors, e.g. x, and upper case bold letters are reserved for matrices, e.g. X. The conjugate transpose of a matrix X is denoted as XH , whereas diag(x) denotes the diagonal matrix constructed by the vector x. Id represents the d-dimensional identity matrix and 0d a d-dimensional vector filled with zeros. Calligraphic letters, e.g. X , typically represent sets, and wx represents the trainable weights of a NN, stacked in a vector. E[·] denotes expectation, whereas ℜ(x) and ℑ(x) represent the real and imaginary parts of a complex number x, respectively. Finally, and the cardinality of a set X is denoted as card(X ). The rest of this paper is organized as follows. Section II presents the wireless system under consideration and the power-constrained UE tracking problem. The proposed multi-agent algorithm is described in Section III and numerically verified in Section IV. Finally, Section V concludes the paper. 4
FIGURE 1: The considered RIS-aided active UE tracking system incorporating pilot transmission power control.
II. System Modeling and Design Objective
We consider a system comprising a single-antenna2 BS receiving pilot symbols from a mobile single-antenna UE positioned in an unknown location pt ∈ R3 , with the intention to obtain an accurate estimate for pt , denoted, henceforth, as p̂t , at each time slot t. In particular, the UE is assumed to move according to a prespecified model m(·), as follows: x(t) ≜ m(x(t − 1)) + n̄(t) ∈ Rs ,
(1)
where the s-dimensional state vector x(t) collects the target coordinates pt as well as other relevant target information, such as velocities, turn rate, and acceleration. The noise vector n̄(t) is typically assumed to be zero2 mean Gaussian, i.e., n̄(t) ∼ N (0s , σue Is ), and accounts for modeling, estimation, and generalization errors. We finally use notation mprior to represent any state prior distribution available. The pilot symbols communication is assisted by an RIS whose controller is managed by the BS via a dedicated error-free control channel [54]. The RIS consists of Nris response-tunable elements, which, according to the vast majority of the currently available hardware implementations [16], [18], [19], [35]–[37], contribute an effective discrete phase shift on the impinging signal. We denote this phase configuration for each i-th RIS unit element (i = 1, . . . , Nris ) as θi , with all belonging to a finite set Θ. The static positions pbs and pris of the BS and the RIS, respectively, are assumed to be known to the estimation framework presented in this paper. As the UE follows the dynamic motion model in (1), the wireless channels are re-sampled at each time instance 2 The single-antenna BS case has been chosen, herein, for ease of
exposition and in order to be consistent with the system models of relevant prior works [32], [34]. Later on, in Section C, the system model and problem formulation will be extended to the multiantenna BS case.
VOLUME ,
t, independently reflecting the evolving geometry of the BS-UE, BS-RIS, and RIS-UE wireless links. The UE transmits a sequence of T pilot symbols to the BS over an equal number of consecutive time instances. At each t-th frame (t = 1, . . . , T ), both the UE’s transmission power P (t) and the RIS phase configuration profile, denoted as Φ(t) ≜ diag(ϕ(t)) with: ϕ(t) ≜ [ejπθ1 (t) , . . . , ejπθNris (t) ],
(e.g., walls or trees) obscuring the LoS propagation path between the UE and the BS. This attenuation is introduced to simulate challenging tracking scenarios. Note that, in the presence of a strong LoS direct path, moderate-to-high received Signal-to-Noise Ratio (SNR) levels would typically suffice for accurate UE tracking [56], rendering the active RIS configuration redundant.
(2)
constitute free design parameters that we intend to optimize for our UE tracking objective.
C. Proposed Active Sensing Protocol
Upon collecting the received signal y(t) at each t-th time Let x(t) denote the unit-power complex-valued pilot instance, the BS decides on the RIS phase configuration symbol transmitted at each t-th time instance. The for the next time instance [32], Φ(t + 1), as well as baseband received signal at the BS side during this frame on the value of a control variable b(t) to be fed back to the UE instructing it to refine its pilot symbols is modeled as follows: transmission power at the next time instance, P (t + 1). √ y(t) ≜ P (t) (hd (pt )+hbs,ris (pt )Φ(t)hris,ue (pt ))x(t)+n(t), More specifically, when the received observations are (3) informative enough for accurate tracking, the BS may where hd (pt ) ∈ C is the gain of the direct BS-UE request a lower uplink transmission power, thus, enabling channel, whereas hbs,ris (pt ) ∈ C1×Nris and hris,ue (pt ) ∈ power savings at the UE side. On the other hand, CNris ×1 represent the respective gains of the BS-RIS when the received signals are too noisy to infer pt and RIS-UE channel matrices, and n(t) ∼ N (0, σn2 ) is satisfactorily, the BS requests higher power levels from the Additive White Gaussian Noise (AWGN), whose the UE for its future pilot symbols transmissions. variance σn2 can be reliably estimated, thus, assumed To facilitate robust detection of the lightweight feedknown. To simplify the notation, in the remainder of back messages, we restrict b(t) to single-bit commands the paper, we will drop the term pt from the symbols (e.g., ‘0’ implies power reduction and ‘1’ power boosting), for the channels and related measures, implying that all leaving the UE to dynamically determine its exact of them depend on the unknown UE position. transmission power within a range [0, Pmax ]. While multibit feedback could explicitly dictate precise power levels, B. Channel Model decoding such complex messages typically requires coWe adopt a general Ricean fading model for all wire- herent demodulation at the UE, which demands powerless channels, which accounts for the presence of a hungry phase tracking hardware. Alternatively, while dominant Line-of-Sight (LoS) component alongside Non- non-coherent amplitude tracking allows for ultra-lowLoS (NLoS) multipath scattering [55]. Specifically, the power reception at the UE side (e.g., wake-up or lowchannel vector (or scalar) h ∈ {hd , hbs,ris , hris,ue } is rate control receivers [50], [51]), attempting to decode expressed as follows: multi-bit symbols non-coherently introduces a dimen(√ ) √ sionality constraint that may demand prohibitively large √ κ 1 h= β hLoS + hNLoS , (4) BS transmit power levels to maintain distinguishable κ+1 κ+1 thresholds [57]. Consequently, designing our collaborawhere β denotes the distance-dependent path loss dic- tive protocol around 1-bit feedback ensures the UE can tated by the UE position at time t, κ represents the utilize an ultra-low-power non-coherent receiver without Ricean factor, and hLoS , hNLoS denote the deterministic imposing excessive transmit power requirements on the Line of Sight (LoS) and stochastic Non-LoS (NLoS) BS. To compensate for this strict information bottleneck, components, respectively. The NLoS components are the proposed active sensing protocol relies on the UE to modeled as standard Rayleigh fading channels with intelligently integrate the entire history of its received entries drawn from a complex Gaussian distribution feedback bits. By interpreting the temporal sequence of CN (0, 1). The LoS components are defined by the binary commands rather than just the instantaneous bit, geometric array response vectors corresponding to the the UE may infer the urgency of the sensing task and Angles of Departure (AoD) and Arrival (AoA), which are adjust its power accordingly. As we will experimentally determined by the positions pt , pbs , and pris . To account demonstrate, this proposed learned protocol incurs minfor the obstruction of the direct path, the direct link imal performance degradation, allowing the proposed hd is modeled with an additional blockage attenuation lightweight algorithm to virtually match the tracking √ coefficient ρ ∈ (0, 1], such that hd = ρ hd,Ricean ; factor accuracy of policies utilizing substantially costlier, high ρ captures the penetration loss caused by obstacles capacity control links. A. Received Signal Model
VOLUME ,
5
Stamatelis et al.:
D. Problem Formulation
According to the adopted active sensing paradigm, the BS decides at each t-th time instance the next frame’s RIS phase configuration, Φ(t + 1), leading to the most favorable observations for the UE tracking objective. In addition, it also decides the transmit power level P (t + 1), which is acknowledged to the UE via the b(t) transmission in the control/feedback channel. This mode of operation implies that the RIS phase profile and the UE transmit power levels at each (t+1)-th frame depend on all past observations. Let us define this dependency through a function g(·), i.e., it holds ∀t < T : {P (t + 1), Φ(t + 1)} ≜ g (y(1), . . . , y(t)) .
and the most accurate possible estimate can be made. Denoting this estimate as p̂ = f (y(1), . . . , y(T )), OP ’s objective simplifies, in the “static” localization case, to: [ ] 2 min E ∥p̂ − p∥2 , (9) gbs (·),gue (·),f (·)
while the constraints (8b)–(8d) remain the same. III. The Proposed DA Deep Learning Method
(5)
The initializations P (0) and Φ(0) can be set to arbitrary values (e.g., P (0) = Pmax , and Φ(0) so as to illuminate a large portion of the RIS area of influence [58]) if there is lack of any relevant a priori information for the tracking objective. Function g(·) that actually determines the active sensing policy can be separated in two mappings: the BS mapping gbs (·) in charge of configuring RIS profiles and selecting the 1-bit feedback messages, and the UE mapping gue (·). Formally: {Φ(t + 1), b(t)} ≜ gbs (y(1), y(2), . . . , y(t)),
(6a)
P (t + 1) ≜ gue (b(1), b(2), . . . , b(t)).
(6b)
Finally, following the same mindset, the UE position estimation at the BS after the reception of the t pilot symbols will be a function of these symbols (i.e., the processing result upon them). Let f (·) represent this function, hence, we define: p̂t ≜ f (y(1), . . . , y(t)) .
(7)
Let Gbs , Gue denote, respectively, the set of all admissible functions gbs (·), gue (·) in (6), and F the set of all estimator functions f (·) in (7). Adopting the Euclidean distance metric (specifically, the Mean Squared Error (MSE)), we formulate the following optimization problem for our tracking objective: [ T ] ∑ 2 OP : min E ∥p̂t − pt ∥2 (8a) gbs ,gue ,f (·)
t=1
s.t. θi (t) ∈ Θ ∀i = 1, . . . , Nris , ∀t = 1, . . . , T, P (t) ∈ [0, Pmax ] ∀t = 1, . . . , T, ] [ T ∑ P (t) ≤ BP , E
(8b) (8c) (8d)
t=1
where BP represents a cumulative UE power budget constraint over the considered T -frame time horizon. It is noted that an interesting special case of the OP formulation, that has received the most attention in the relevant active sensing literature [32], [34], is the “static” localization case. In this case, the estimator f (·) is only used in the final instance of the T -instance horizon, when all received signal information has been collected, 6
FIGURE 2: Graphical illustration of the DA algorithm describing the collaborating agents at the BS and UE sides. In this section, we present our DA approach for the considered binary feedback messages enabling the UE power control in the uplink. As illustrated in Fig. 2 and detailed in the sequel, the proposed approach involves two collaborating agents: one at the BS side and one at the UE. Their policies are jointly optimized to achieve the sufficient level of coordination enabling accurate UE localization and tracking. A. Data-Driven OP Solution
Solving OP analytically is computationally prohibitive. At every time instance t, the optimal strategy requires an exhaustive search over the combinatorial space of discrete RIS phase configurations (which scales as card(Θ)Nris ), coupled with the continuous UE power allocation variable. This instantaneous optimization is further complicated by the following temporal dependency: current actions dictate the quality of future estimates, necessitating a dynamic programming approach rather than greedy selection. Formally, this falls under the umbrella of Partially Observable Markov Decision Processes (POMDPs) [59]. The partial observability arises particularly because the true system state, i.e., VOLUME ,
the unknown UE’s position pt , is hidden and must be inferred from noisy observations (following the model in (3)). Exact solutions to POMDPs generally require tracking a continuous, high-dimensional belief state (the posterior distribution of pt ), a task proven to be NPhard [60] even without our additional UE power budget constraint. In more detail, our UE tracking problem formulation essentially constitutes a Decentralized POMDP (DecPOMDP), which introduces exponential complexity [61]. Actually, this complexity stems from the asymmetric information structure: the BS observes y(t), but cannot directly set the UE pilot transmission power, while the UE controls the power but observes only the single-bit feedback message b(t). These two agents must, therefore, implicitly learn a coordination protocol to overcome this information bottleneck. To this end, inspired by prior works on related active sensing problems, e.g., [28]–[30], we devise a data-driven algorithm to approximate the intractable optimal policy. We leverage the ability of RNNs to maintain a compact hidden state that acts as a surrogate for the complex belief state. In the sequel, we parametrize the adaptive control policies gbs (·), gue (·) and the position estimator f (·) as Deep NNs (DNNs), and, to navigate the non-differentiable landscape due to discrete RIS phase responses, we propose a three-step training approach for their efficient training.
1) The BS Agent
A policy NN, P and an estimator NN, E are deployed, with the former controlling the RIS phase configuration as well as the BS’s binary feedback to the UE, and the latter being responsible for processing the observation sequence and estimating the UE location. Both NNs include LSTMs and, for simplicity, we assume that they share the same number of hidden layers, activation functions, and layer sizes. However, they differ in their subsequent Feed-Forward (FF) stacks, which are tailored to their output requirements. Let ȳ(t) ≜ [ℜ{y(t)}, ℑ{y(t)}]. The policy NN takes its hidden state vector ℓP (t) and the most recent observation ȳ(t) as inputs3 to its LSTM, producing the output op1 (t + 1). This output then passes through an additional NN of linear ReLU-activated layers, yielding the vector op2 (t+1) ∈ RNris card(Θ)+1 . Its first Nris card(Θ) elements are passed through an element-wise softmax(·) function to define a probability distribution over the configuration of the RIS elements, while the last element is transformed using a tanh(·) and a sign(·) functions to
FIGURE 3: A high-level overview of the estimator NN E at the BS: At each time instance t, the vector ȳ(t) is passed through the LSTM parameters, and the resulting hidden state is stored for the next time instance t + 1. The output oe1 (t) is further processed by a stack of linear ReLU-activated layers to generate the final position estimate p̂t . For static localization, the estimator is only evaluated at the last step t = T of the horizon.
produce the single-bit b(t)-value to be transmitted4 to the UE. On the other hand, the estimator NN, which is depicted in Fig. 3, processes ȳ(t) with its LSTM unit using the most recent hidden state ℓE (t), producing the output oe1 (t). This output is then passed through a stack of linear layers activated by Rectified Linear Unit (ReLUs), resulting in an output p̂t ∈ R3 that represents the estimation for the UE position at time instance t. In the case of localization, where pt = p is fixed, E is employed only once at the end of the episode. The observations are sequentially passed through the same LSTM weights, and the final (i.e., at t = T ) output oe1 (T ) is provided to the output stack of linear ReLU-activated layers to infer p̂. We use the following notation to highlight the parameterization of the functions gbs (·) and f (·) appearing in expressions (6a) and (7), respectively, from the respective RNNs P and EI : gbs (ȳ(t), ℓP (t); P) , f (ȳ(t), ℓEI (t); EI ) .
Notice that, instead of requiring as input the entire observation sequence (as in expressions (6) and (7)), only the most recent observation along with the hidden vector are needed, with the latter compressing all important
3 Alternatively, the received signal strengths (RSSs) (i.e., |y(t)|2
4 The proposed policy NN can be easily modified to account for
∀t [28], [29]) can be used as inputs to the policy NN at the BS. It will be shown later on, in the results’ section, that our method
multi-bit b(t)-value messages: the network’s output needs to be increased according to the total number of feedback bits, and its
performs well also with this input.
activations need to be replaced by a softmax(·) function.
VOLUME ,
7
Stamatelis et al.:
past information. For long tracking sequences, processing the entire observation history at each instance t can become computationally infeasible.
2) The UE Agent
An additional LSTM, referred to as the power NN, M, is maintained at the UE side. This NN processes the most recent control value b(t) (received through the feedback channel) along with its hidden state ℓM (t) to select the power level P (t + 1) for the next time instance t + 1. Thanks to the representational power of LSTMs, this NN can learn highly effective power control policies despite the strict constraint of the 1-bit feedback message. As we will demonstrate later on, the performance of this decentralized scheme can approach that of systems with much richer feedback. We, henceforth, represent the parameterization of the function gue (·) appearing in (6b) from the NN M as: gue (b(t), ℓM (t); M). B. Proposed Training Procedure
The training of the proposed DA approach comprises the following three stages. First, an initial estimator NN, EI , is trained on a dataset of randomly generated episodes (i.e., with random policies). Then, the policy, P , and power, M, NNs are evolved to collect UE trajectories that aid EI ’s inference capabilities. In the sequel, when the policies have been learned, a final estimator NN E is retrained using data collected under the learned policies of the BS (P ) and UE (M) agents. The first and last steps involve classic supervised learning on the MSE loss function. The key step is the intermediate stage, where the NNs P and M need to be jointly optimized. 1) Fitness Function
We proceed by treating the pair (P, M) as a single optimization variable within the NE framework [31]. More specifically, each individual in the population is represented by a concatenated parameter vector: i ≜ [wP , wM ],
(10)
and the fitness function is defined as follows: [T ] [T ] −ÊP,M ∑ P (t) , if ÊP,M ∑ P (t) > BP t=1 [ q(i) = , ]t=1 − 1 ÊP,M,E ∑T ∥p̂t − pt ∥2 , otherwise. I t=1 T (11) where ÊX [·] represents sample averaging, with respect to the random variable or set of variables X , over a large number NEP of Monte Carlo episodes. It is noted that, in our application, besides the noise n(t) in (3), the stochastic policies and the estimator affect the averaging. Intuitively, the fitness function in (11) 8
penalizes individuals that fail to satisfy the power budget constraint BP . Among those that satisfy it, individuals leading to observations of higher quality with smaller tracking errors are preferred. Note also that, in static localization objectives, the second branch of the fitness function in (11) simplifies to ÊP,M,EI [∥p̂t − pt ∥2 ]. For each evaluation episode, a random UE position p0 is sampled along with related parameters concatenated in the state x0 , and the hidden states of all NNs are initialized. At each t-th time instance, the following four operations occur: • The UE state x(t) is updated according to the motion model m(·), as described in the state model in (1). • An observation y(t) is sampled from (3) using P (t) and Φ(t) which is provided to the policy NN P . • The latter network outputs Φ(t + 1) and an 1-bit message b(t). On the UE side, the power NN M uses b(t) along with its hidden state to select P (t + 1). • The vector ȳ(t) is also provided to the initial estimator NN EI to compute the estimation error, which determines the individual’s fitness.
2) Optimization Procedure
The proposed optimization process is performed by first initializing a population of Lpop random individuals i1 , . . . , iLpop , with iℓ ≜ [wP,ℓ , wM,ℓ ] ∀ℓ = 1, . . . , Lpop . These individuals are updated over multiple optimization iterations referred to as generations. For each generation within the range 1, . . . , Ngen , the evolutionary process iterates through the following steps (see also Algorithm 1 for more details): • Evaluation: For every individual iℓ in the current population, the corresponding parameter vector is first split to reconstruct the weight matrices of the BS policy NN and the UE power NN. The individual’s fitness is calculated and stored in order to select the best performing individuals for the population update. • Selection: Once all individuals have been evaluated, the population is sorted in descending order based on their computed fitness scores. To promote the propagation of high quality traits, only the top performing fraction of the population, specifically the best ⌊Lpop /4⌋ individuals, are selected to survive and, consequently, serve as parents for the next generation. • Crossover: To replenish the population back to its full size Lpop , offspring are generated by mating the selected parents. During this operation, the weight vectors of two randomly chosen parents are merged to construct a new candidate solution. Specifically, we employ a uniform crossover strategy: the offVOLUME ,
spring’s weight vector is initialized to zero, and each individual parameter is then assigned the value of the corresponding weight from one of the two parents, selected with equal probability (i.e., a 50% chance for each parent). • Mutation: To prevent premature convergence to local optima, a mutation operator is applied to the newly generated offspring. This involves adding zero-mean Gaussian noise with standard deviation σmut to each component c of ℓ’s weight vector with a probability pmut . Formally, for the ℓ-th individual, the mutation operator applied to each component c is: { 2 iℓ [c] + N (0, σmut ) with probability pmut iℓ [c] ← iℓ [c] with probability 1 − pmut (12)
3) Computational Complexity
Let TFP denote the computational time required for a forward pass of the policy and power NNs at the BS and UE, respectively. Given their structural similarity, we assume identical inference latency for both of these two agents. Consequently, evaluating the fitness function of the entire population entails a computational complexity of O(Lpop NEP T TFP ). The associated genetic operations (i.e., crossover and mutation) scale linearly with the parameter space, contributing complexity of O(Lpop W ), where W represents the total number of learnable weights (i.e., the total number of parameters of both the policy (P ) and power (M) NNs). Additionally, ranking the population based on fitness requires O(Lpop log(Lpop )) of complexity. Aggregating these components over Ngen generations yields the final total complexity: O (Ngen (Lpop NEP T TFP + Lpop W + Lpop log(Lpop ))) . C. Extension to Multi-Antenna Base Stations
The proposed DA active sensing framework for UE localization and tracking can be easily extended to a multi-antenna BS, which will now also be capable of dynamically adjusting its beamforming operation for further boosting the estimation accuracy. To this end, the following modifications on the policy NN P at an Nbs -antenna BS are needed. Let y(t) ∈ CNbs ×1 denote the vector form of the baseband received signal at the BS side, which can be expressed as: √ y(t) ≜ P (t) (hd + Hbs,ris Φ(t)hris,ue ) x(t) + n(t), (13) where hd ∈ CNbs ×1 and Hbs,ris ∈ CNbs ×Nris represent the gains of the direct BS-UE and the BS-RIS channels, respectively. Note that, similar to (3), all involved channel vectors and matrices depend on the unknown UE coordinates pt . In addition, n(t) ∼ N (0Nbs , σn2 INbs ) VOLUME ,
Algorithm 1 Proposed DA Active Sensing Training Input: Population size Lpop , number of generations Ngen , evaluation episodes NEP , mutation probability pmut and standard deviation σmut , and horizon T . Output: Optimized policy and estimator NNs at BS as well as power NN at UE. 1: Stage 1: Initial Estimator Training 2: Create dataset Drand using random RIS phase profiles Φ(t) and power levels P (t) at each time instant t. 3: Train initial estimator NN EI on Drand for minimizing the MSE in OP . 4: Stage 2: Optimize policies Lpop . 5: Initialize population {iℓ = [wP,ℓ , wM,ℓ ]}ℓ=1 . 6: for generation 1 to Ngen do 7: for individual ℓ = 1 to Lpop do 8: Convert individual iℓ into NNs Pℓ and Mℓ . 9: for episode e = 1 to NEP do 10: Sample user location p0 ∈ P and channel realizations hed , hebs,ris , heris,ue . 11: Initialize LSTM hidden state vectors: ℓPℓ (0), ℓMℓ (0), and ℓEI (0). 12: for t = 1 to T do 13: Move UE according to eq. (1) and compute the new channels. 14: Compute y(t) according to eq. (3). 15: BS policy NN Pℓ : [Φ(t + 1), b(t)] ← gbs (ȳ(t), ℓPℓ (t); Pℓ ). 16: UE power NN Mℓ : P (t + 1) ← gue (b(t), ℓMℓ (t); Mℓ ). 17: BS Estimation NN EI : p̂t = f (ȳ(t), ℓEI (t); EI ) 18: Store the error ∥p̂t − pt ∥2 . 19: end for 20: Calculate the fitness of the e-th episode, q e (iℓ ), via (11). 21: end for 22: Compute ← ∑NEP ethe final fitness: q(iℓ ) 1 q (i ) . ℓ e=1 NEP 23: end for 24: Sort the population based on fitness values Lpop {q(iℓ )}ℓ=1 . ⌊L /4⌋ 25: Select elite parents Pelite ← {iℓ }ℓ=1pop . 26: Crossover: Refill the population by merging pairs from Pelite (uniform selection). 27: Mutate: Perturb the new individuals via eq. (12). 28: end for 29: Extract Pℓ∗ and M∗ ℓ corresponding to the individual i∗ℓ with the highest fitness from the resulting (final) population. 30: Stage 3: Fine Tuning 31: Collect dataset Dopt using evolved policies M∗ ℓ and Pℓ∗ . 32: BS Estimation NN E : Retrain final estimator NN on Dopt to minimize the MSE in OP . 33: return Pℓ∗ , M∗ ℓ , and E . 9
Stamatelis et al.:
Type I codebook [62] considers W containing the rows of the Nbs ×Nbs Discrete Fourier Transform (DFT) matrix. Following our active sensing formulation, the mapping gbs (·) of the multi-antenna BS will be now responsible for configuring the BS digital combiner and the RIS phase profiles, as well as for selecting the 1-bit feedback messages, i.e.: {Φ(t+1), w(t+1), b(t)} = gbs (ỹ(1), ỹ(2), . . . , ỹ(t)). (15)
To this end, the policy NN P is extended by adding one more output branch, as shown in Fig. 4. This branch receives as input the most recent LSTM output op1 (t + 1) and then passes it through a linear ReLU-activated stack, whose final layer is configured to output an Nbs dimensional vector. In the sequel, a softmax(·) is applied to this output in order to sample a combiner w(t + 1) ∈ W for the (t + 1)-th time instance. All other aspects of the operation of the NNs and the procedure for solving OP remain unchanged. IV. Numerical Results and Discussion
FIGURE 4: The proposed policy NN for the multiantenna BS case. An LSTM core component extracts the important temporal features from the observation sequence. The LSTM’s output is then processed by two separate FF stacks: the one stack is in charge of selecting the RIS profile and the binary message (similar to the single-antenna case), while the other second stack is configured to output an Nbs -dimensional vector. Application of the softmax(·) function to this output yields a valid beamforming selection, i.e., such that w(t) ∈ W .
denotes the AWGN. Consider that, at each time instance t, a digital combining vector w(t) ∈ CNbs ×1 is applied at BS’s baseband as follows: ỹ(t) ≜ w (t)y(t), H
(14)
providing a complex-valued scalar. This post-processing scalar can be then plugged into the previously detailed single-antenna active sensing machinery. In practice, w(t) needs to be selected from a finite beam codebook W . For example, a simplified version of the 3GPP 5GNR 10
In this section, we evaluate the performance and robustness of the proposed DA active sensing framework through extensive simulations. Specifically, the effectiveness of the presented tracking scheme is investigated across various operational scenarios, encompassing both mobile UEs under diverse dynamic motion models, as well as static localization environments. To rigorously assess the merits of our algorithm, its performance has been benchmarked against both classical estimation techniques [12], [52], [53] as well as learning-based sensing agents [32]. Before detailing the simulation results, we first present the core algorithmic hyperparameters employed throughout all our following experiments. The parameters of the CoSyNE algorithm [63] are set to pmut = σmut = 0.5, Lpop = 50, and Ngen = 100. The policy NN’s LSTM was chosen to have 2 hidden layers of 512 units, and each of the FF branches for the selection of the RIS phase profile and the 1-bit power parameter message was chosen to have a single hidden layer of 128 and 32 units, respectively. The power NN’s LSTM had also 2 hidden layers of 512 units, followed by a linear layer of 64 hidden units. The estimator NN was designed to have a similar LSTM, followed by 2 ReLUactivated hidden layers each of 128 units. Both initial and final estimators were trained on 50000 sequences. A. System Setup
A carrier frequency of 5 GHz has been considered, corresponding to a signal wavelength of λ = 6 cm. The BS was assumed located at the position [40, −40, 10] m, whereas the top-left unit element of the RIS was placed at the origin [0, 0, 0] m. The metasurface was modeled as a uniform rectangular array, placed parallel to the x-z plane, consisting of Nris = 400 elements with binary phase responses, unless otherwise indicated. Those VOLUME ,
44
0
Target Position Start End 5 10 15 X position (m) (a) ω0 = 0.2
42 40 38 36
20
25
Y position (m)
Y position (m)
Y position (m)
55.0 52.5 50.0 47.5 45.0 42.5 40.0 37.5 35.0
Target Position Start End 12 14 16 18 20 X position (m)
22
24
(b) ω0 = 0.4
40 39 38 37 36 35 34 33
Target Position Start End
14
16
18 20 X position (m)
22
(c) ω0 = 0.8
FIGURE 5: The considered example UE trajectories for different initial turn rates ω0 , considering the motion variance σue = 0.1. It can be observed that a low initial turn rate (i.e., ω0 = 0.2 in (a)) yields a relatively smooth trajectory, whereas increasing this rate to 0.4 (as in (b)) results in more aggressive maneuvers. Finally, the turn rate of ω0 = 0.8 in (c) induces highly rapid directional changes.
elements were assumed the standard half-wavelength element spacing of λ/2 = 3 cm. To model a typical indoor propagation environment, we adopted a log-distance path loss model with an exponent of 2.5; this specific value is empirically justified for sub-6 GHz indoor systems experiencing NLoS obstructions [64]. Alongside the path loss, we considered Ricean fading conditions with a κ = 10 dB factor, unless otherwise indicated, and applied an extra attenuation of 10 dB to the direct BS-UE channel. Furthermore, assuming a transmission bandwidth of 10 MHz and a background noise power spectral density of −170 dBm/Hz, the effective noise level at the BS side σn2 was set to −100 dBm [32]. Finally, the maximum UE pilot transmission power in the uplink was bounded at Pmax = 30 dBm, and the T -horizon power∑constraint in the OP formulation was T set as BP = 0.5 t=1 Pmax . A two-dimensional motion model was considered, where the UE moves along the x and y axes under a nonlinear Coordinated Turn (CT) model. Physically, this model captures the realistic maneuvering behavior of a mobile user, such as navigating a curve or altering its path. It assumes that during any short sampling interval, the user maintains a constant forward speed while turning at a steady rate. The entire state was a 5-dimensional vector x(t) = [px,t , py,t , vx,t , vy,t , ωt ]T , where px,t and py,t represent the Cartesian position, vt ≜ [vx,t , vy,t ] denotes the velocity, and ωt is the instantaneous turn rate. Depending on the turn rate, the state transition function m(x(t)) in (1) operates in two regimes. If ωt ̸= 0, the UE moves along a circular arc with a constant angular velocity over the sampling interval, and m(x(t))
VOLUME ,
is equal to: vy (t) x (t) px (t) + vω(t) sin(ω(t)∆T ) − ω(t) (1 − cos(ω(t)∆T )) vy (t) x (t) py (t) + vω(t) (1 − cos(ω(t)∆T )) + ω(t) sin(ω(t)∆T ) , vx (t) cos(ω(t)∆T ) − vy (t) sin(ω(t)∆T ) vx (t) sin(ω(t)∆T ) + vy (t) cos(ω(t)∆T ) ω(t) where the sampling interval was set to ∆T = 0.5 sec, and the initial turn rate ω0 was sampled uniformly from [−ωmax , ωmax ]. As visualized in Fig. 5, the initial turn rate ω0 significantly dictates the curvature and aggressiveness of the resulting trajectory/episode. However, this standard CT formulation introduces a mathematical singularity (division by zero) when the target stops turning and moves in a straight line. To ensure numerical stability and model continuity, we have evaluated the limit as ωt → 0; this seamlessly reduced the system to the following state update model: px (t) + vx (t)∆T py (t) + vy (t)∆T . vx (t) lim m(x(t)) = ωt →0 vy (t) ω(t) B. Results for Tracking
For the UE tracking investigations, we have also simulated a supervised tracking LSTM and two popular filters (all with random policies): the Extended Kalman Filter (EKF) [52], [65] and the Particle Filter (PF) [53], [66] with 500 particles. In addition, acknowledging DRL’s adoption for other active sensing problems (e.g., [29]) as well as for RIS phase configuration control [22], we designed a benchmark based on the popular and powerful Advantage Actor Critic (A2C) algorithm [67]. To this end, we followed a similar three-step hybrid training approach to our proposed DA active tracking framework, 11
Stamatelis et al.:
by only replacing the implementation of the second step with a single A2C agent being in charge of selecting both the RIS phase profile and the UE power level. To enforce the power constraint for the latter, a Lagrangian reward was used similar to [68]. For fairness, this agent utilized recurrent actor and critic NNs with similar structure and size as our NE-optimized policies. Training consisted of 50000 episodes using the default A2C hyperparameters of a popular open source implementation provided in [69]. It is noted that, by comparing against this method, the effectiveness of the proposed NE procedure over off-theshelf DRL algorithms can be quantified. Overall, we conducted experimental investigations with respect to the following system parameters: • Effect of motion noise: The maximum initial turn rate was set to ωmax = 0.2, the horizon to T = 20, and the motion variance σue was varied from 0.1 to 0.4. • Effect of angle rate initialization: Parameter ωmax was varied from 0.2 to 0.8 in order to verify the ability of our model to generalize to diverse turning motions. The horizon was fixed to T = 20 and the noise to σue = 0.4. • Effect of the tracking horizon: The noise was fixed to σue = 0.4, ωmax was set to 0.2, and the time horizon was varied from T = 40 to 70.
Figure 6 illustrates the Root Mean Squared Error (RMSE) across a range of UE motion dynamics and system parameters, with the provided results being averaged over 1000 independent episodes/trajectories to ensure a statistically robust and fair evaluation of each algorithm’s performance. As shown in the figure, the proposed DA active sensing scheme consistently outperforms all baseline methods. Notably, it demonstrates strong robustness against increasing motion noise σue , higher maximum turn rates ωmax , and longer tracking horizons T . It is also shown that the A2C benchmark outperforms traditional schemes with random policies, without, however, being able to match the performance of our proposed method, signifying the importance of our NE approach. Furthermore, Fig. 7 investigates the scalability of our algorithm for an increasing number of BS antennas Nbs , where, in this case, the BS was configured as a uniform linear array. It is again shown that our scheme is superior to all benchmarks, with increasing Nbs resulting in slightly boosting tracking performance. C. Results for Localization
In the localization experiments, the UE’s static position was chosen as uniformly distributed inside the cubic area [20 ± 15, 20 ± 20m, −20] m, modeling spatial uncertainty in the x and y axes [32]. The performance of our active 12
sensing scheme was compared against the following localization algorithms: • Fingerprinting [12]: The sequence of the T RIS phase profiles was predetermined, non-adaptive and random, and the UE pilot transmit power levels at each time instance t were sampled uniformly in [0, Pmax ]. Each 1 m2 block in the candidate UE location area was assigned a fingerprint sequence |y(1)|2 , . . . , |y(T )|2 , precomputed and stored in a database. During operation, a 5-nearest neighbor fingerprint classifier was employed. We also considered another variant of this scheme where the RIS phase profile was optimized using a genetic algorithm. The default implementation of genetic algorithms from a popular open source library has been used [70]. • Supervised learning: We considered a supervised NN trained on 70000 sequences/episodes with random RIS phase profiles and power levels sampled uniformly in the range [0, Pmax ] at each time instance t. We used an FF model with 4 hidden layers each with 400 units. This NN stacks the entire observation sequence into a large vector, which is then mapped into the final 3-dimensional estimate. • End-to-end backpropagation with RIS of continuous phase responses [32], [71]: A scheme comprising an intelligent active sensing agent that selects continuous-valued RIS phase profiles, and the UE always transmitting with power Pmax has been simulated. This agent was equipped with an LSTM that decides Φ(t) using the current observation y(t) and its hidden states. The hidden states were then passed through an FF branch for the final position estimation. The structure of the LSTM layers were similar to those used in the proposed DA scheme. Training was conducted using end-to-end backpropagation on the considered MSE loss [32], [71]. • Deep reinforcement learning: The second step of the proposed DA active sensing scheme, which is tasked with the joint selection of Φ(t) and P (t), was implemented an A2C agent, as in the tracking comparisons.
Three sets of experiments, as demonstrated in Fig. 8, were conducted to evaluate the robustness and scalability of the proposed algorithm across a broad range of wireless configurations. The key findings are summarized as follows: • Impact of the RIS size (Nris ): While larger RISs provide higher beamforming gain to steer signals toward the BS, they also expand the action space, increasing the number of decision variables and the complexity of the optimization (hence, requiring larger NNs). As depicted in Fig. 8a, our approach VOLUME ,
6 5
22
4 3
6
EKF Supervised Proposed PF A2C
5 4 3
2 0.10
0.15
0.20
0.25
0.30
0.35
EKF Supervised Proposed PF A2C
20
7
Trajectory RMSE
7
Trajectory RMSE
8
EKF Supervised Proposed PF A2C
0.40
0.2
0.3
Target Motion Variance σue
0.4
0.5
0.6
0.7
0.8
Trajectory RMSE
8
18 16 14 12 10 8 6 40
45
50
Maximum Initial Turn Rate ωmax
(a) The effect of motion variance σue for ωmax = 0.2 and T = 20.
(b) The effect of ωmax for σue = 0.4 and T = 20.
55
60
65
70
Horizon T
(c) The effect of tracking horizon T for σue = 0.4 and ωmax = 0.2.
FIGURE 6: Trajectory RMSE performance over various UE motion and system parameters.
8
EKF Supervised Proposed PF A2C
Trajectory RMSE
7 6 5 4 3 2 1
2
3
4
5
6
7
8
Number of BS Antennas Nbs
FIGURE 7: Trajectory RMSE performance versus the number of BS antennas Nbs , considering ωmax = 0.2, σue = 0.4, and T = 20.
scales well with larger RISs. On the contrary, the gradient-based scheme of [32] and A2C display decreasing performance with larger RISs. • Effect of scattering conditions (κ): As illustrated in Fig. 8b, lower Ricean κ-factors, representing richer scattering environments, challenge more all considered localization schemes. It is shown that our approach remains effective even in highly cluttered systems with small Ricean κ-factors, consistently outperforming all benchmarks by a substantial margin. Interestingly, fingerprinting schemes appear to be greatly affected by κ. • Performance across noise regimes (n(t)): Figure 8c showcases the superior robustness of our scheme versus noise in comparison to all considered benchmarks.
UE power budget constraint BP , as expressed in the OP formulation. In Fig. 9, we evaluate the robustness of the proposed framework by repeating the RIS size scaling experiments of Fig. 8a using RSSs (i.e., |y(1)|2 , . . . , |y(T )|2 ) instead of the complex-valued received signals in baseband. The results demonstrate that our DA algorithm consistently maintains its superior performance over all baseline methods. This confirms that the learned collaborative protocol between our BS and UE agents is highly effective at extracting spatial features, even from simplified power-based observations, reinforcing our scheme’s applicability to various receiver hardware constraints. Pilot Transmission Power Adjustments
Figure 10 illustrates the average UE transmission power allocated at each time instance in the range t ∈ [1, 10] for the simulation setting of Fig. 8a. To capture the consistent behavior of the learned policy, instantaneous power levels across 1000 independent evaluation episodes have been used for computing the average power values at each time instance t. It is shown that the proposed power NN learns an interesting strategy: at the first time instance, it transmits a message with large power and then, in subsequent transmissions, the power levels become very small, often near 0 dBm. Only in the final instance of t = 10, a very large power level is applied to ensure detection. In Fig. 11, we examine the effect that noise n(t) has on power allocations, considering the RIS size of Nris = 400. It is demonstrated that, for moderate noise levels, the two-pulse allocation of Fig. 10 remains consistent. On the other hand, for very strong noise, the policy learns uniform power allocation. Sensitivity Study
All in all, the proposed DA active sensing scheme consistently achieves superior localization accuracy than all benchmarks, while strictly adhering to the cumulative VOLUME ,
To quantify the stability of the proposed NE framework for training, we have evaluated the Coefficient of Variation (CoVar), defined as the ratio of the standard 13
Stamatelis et al.:
7
6.0
6.0 5.5
4.5 4.0 3.5
6
Average RMSE
5.0
Average RMSE
Average RMSE
5.5
5.0 4.5 4.0 3.5
3.0
3.0
2.5
2.5
5
4
3 15
20
25
Backpropagation A2C Fingerprinting
√
30
35
40
5
6
7
8
9
10
11
−80
12
Ricean factor κ
Nris
Fingerprinting optimized Supervised Proposed
(a) Effect of the RIS size Nris : the Ricean factor is set to κ = 10 dB and the power of the noise level σn is set to −100 dBm [32].
Backpropagation A2C Fingerprinting
Fingerprinting optimized Supervised Proposed
−70
−60
Backpropagation A2C Fingerprinting
(b) Effect of the Ricean factor κ: the RIS panel size is set to Nris = 400, and the power of the noise level σn is set to −100 dBm.
−50
−40
Noise level in dBm
−30
Fingerprinting optimized Supervised Proposed
(c) Effect of the observation noise σn : the RIS panel size is set to Nris = 400 and the Ricean factor to κ = 10 dB.
6.0
30
5.5
25
Transmit power in dBm
Average RMSE
FIGURE 8: Localization RMSE performance over various system parameters.
5.0 4.5 4.0 3.5 3.0 15
20
Backpropagation A2C Fingerprinting
25
√
30
35
40
Nris
Fingerprinting optimized Supervised Proposed
FIGURE 9: Localization RMSE performance versus the RIS panel size Nris , considering RSS inputs. The Ricean factor was set to κ = 10 dB and the power of the noise level σn to −100 dBm.
deviation to the mean. This dimensionless metric provides a normalized measure of dispersion, allowing for a direct comparison of sensitivity across hyperparameters with different scales [31]. In the context of stochastic optimization, a CoVar value of less than 1.0 is considered a very good result, indicating small variance or high consistency in the algorithm’s performance, since the variance is less than the mean. In our case, we have examined the sensitivity to the population size (Lpop varied from 25 to 60), mutation probability (pmut varied from 0.25 to 0.75) and standard deviation (σmut varied from 0.25 to 0.75), as well as the initialization seed (20 different seeds). The CoVars for a tracking and a localization system are reported in Figs. 12 and 13, respectively. As depicted, these coefficients are always 14
Nris = 225 Nris = 400 Nris = 900 Nris = 1600
20 15 10 5 0 2
4
6
8
10
Time instance t
FIGURE 10: The average UE power levels in the uplink of the proposed active sensing scheme at each time instance t for different RIS panel sizes Nris . The Ricean factor was set to κ = 10 dB and the power of the noise level σn to −100 dBm.
less than 1.0 indicating that the proposed NE-based optimization is robust. The initialization seed produced the lowest CoVar, confirming that our three-stage training procedure, and specifically the use of an initial estimator NN EI to guide the evolutionary process, yields a consistent and reproducible learning process. D. Impact of the Power Control Link
Having established the effectiveness of the proposed NE training scheme over conventional and learning-based benchmarks, we now finally investigate the impact of the BS-UE control link configuration on the estimation performance. In Fig. 14, we repeated the experiment of Fig. 6b, with the difference that, in this case, our scheme is benchmarked against variations of NE-based VOLUME ,
Transmit power in dBm
30
Trajectory RMSE
4.5
-80 dBm -60 dBm -30 dBm
25 20 15
3.5
Max Power 3-bit Codebook, Full scalar Proposed
3.0 2.5
10 0.2
2
4
6
Time instance t
8
10
FIGURE 11: Same as in Fig. 10, but for different observation variance levels (σn ). The Ricean factor was set to κ = 10 dB and the RIS panel size to Nris = 400.
1.0
0.8
0.3
0.4
0.5
0.6
0.7
0.8
Maximum Initial Turn Rate
5 0
Coefficient of Variation (CoVar)
4.0
0.75
0.6
0.45
0.42
0.4
0.33
FIGURE 14: Same as in Fig. 6b, but for different control link configurations.
algorithms. More specifically, we considered the following three NE single-agent methods: • Max Power: The BS is only in charge of selecting Φ(t); no power NN, M, is available. • Full scalar transmission: The BS’s power NN decides on P (t + 1) and, then, its exact value s fed back to the UE. While a full scalar link allows the BS to explicitly dictate transmission power levels at the UE side, this option imposes a heavier control communication overhead. • 3-bit codebook: The power level P (t) is encoded using a predefined 3-bit lookup table and transmitted to the UE, which also possesses the same table.
0.2
0.0
Population Size (Lpop )
Mutation Prob. (pmut )
Mutation Noise (σmut )
Initialization Seed
FIGURE 12: Sensitivity study results for tracking with T = 20, σue = 0.4, and ωmax = 0.
Coefficient of Variation (CoVar)
1.0
0.88
0.8
0.62
0.6
0.51
0.46
0.4
The same NE algorithm with the same parameters was employed by all four power control schemes. Similarly, Fig. 15 performs the same comparison for the localization scenario considered in Fig. 8c. Remarkably, the results in Figs. 14 and 15 demonstrate that our DA active sensing scheme achieves tracking as well as localization performance nearly identical to the scalar and multi-bit variants across the entire noise regime. This behavior confirms that, by interpreting the temporal history of binary commands, the UE agent can effectively compensate for the information bottleneck of the considered 1-bit power control link. Furthermore, it can be observed that the performance gap between our power-constrained agents and the “Max Power” baseline remains less than 15% in all test cases, despite consuming only half of the UE uplink transmission power.
0.2
V. Conclusions and Future Work 0.0
Population Size (Lpop)
Mutation Prob. (pmut)
Mutation Noise ( mut)
Initialization Seed
FIGURE 13: Sensitivity study results for localization with noise level nt of −60 dBm, RIS size Nris = 400, and κ = 10 dB.
VOLUME ,
In this paper, we demonstrated that active sensing can effectively enhance localization and tracking in systems with power-limited UEs and realistic RIS phase profile characteristics. A novel hybrid scheme integrating NE and supervised learning was proposed, which was shown to outperform fingerprinting baselines, constrained DRL policies, as well as backpropagation-based approaches 15
Stamatelis et al.:
Average RMSE
3.4
Max Power 3-bit Codebook, Full scalar Proposed
3.2
3.0
2.8
2.6 −80
−70
−60
−50
Noise level in dBm
−40
−30
FIGURE 15: Same as in Fig. 8c, but for different control link configurations.
in static localization across different RIS sizes, fading conditions, and noise levels. In dynamic tracking experiments, it was demonstrated that the proposed DA active sensing framework outperforms popular recursive state estimators with random RIS policies in a diverse set of motion models. Notably, the proposed scheme achieves high localization accuracy with only single-bit feedback power control messages, demonstrating that the learned collaborative protocol can effectively overcome information bottlenecks with negligible performance degradation compared to high capacity control links. Future work will extend the presents framework to multi-RIS deployments and explore location privacy guarantees. We also plan to integrate advanced DNN pruning schemes [72] to further reduce the computational overhead for lightweight IoT devices. Finally, combining our active measurement control protocol with modelbased NN tracking architectures, such as [73], [74] is a compelling direction for future investigation. REFERENCES [1] G. Stamatelis et al., “Joint active RIS configuration and user power control for localization: A neuroevolution-based approach,” in Proc. IEEE ICASSP, (Barcelona, Spain), 2026. [2] A. Georgiev and P. Allen, “Localization methods for a mobile robot in urban environments,” IEEE Trans. Robot., vol. 20, no. 5, pp. 851–864, 2004. [3] Q. Tao et al., “SeqPolar: Sequence matching of polarized LiDAR map with HMM for intelligent vehicle localization,” IEEE Trans. Veh. Technol., vol. 71, no. 7, pp. 7071–7083, 2022. [4] S. Kuutti et al., “A survey of the state-of-the-art localization techniques and their potentials for autonomous vehicle applications,” IEEE Intern. Things J., vol. 5, no. 2, pp. 829–846, 2018. [5] P. N. Beuchat et al., “Enabling optimization-based localization for IoT devices,” IEEE Intern. Things J., vol. 6, no. 3, pp. 5639–5650, 2019. [6] K. F. Bradai et al., “Survey of localization systems in internet of things,” Mobile Netw Appl, vol. 24, p. 761–785, 2019. [7] K. Witrisal et al., “High-accuracy localization for assisted living: 5G systems will turn multipath channels from foe to friend,” IEEE Signal Process. Mag., vol. 33, no. 2, pp. 59–70, 2016.
16
[8] E. Basar, G. C. Alexandropoulos, Y. Liu, Q. Wu, S. Jin, C. Yuen, O. A. Dobre, and R. Schober, “Reconfigurable intelligent surfaces for 6G: Emerging hardware architectures, applications, and open challenges,” IEEE Veh. Technol. Mag., vol. 19, no. 3, pp. 27–47, 2024. [9] Y. Liu et al., “Reconfigurable intelligent surface aided wireless localization,” in Proc. IEEE ICC, (Montreal, Canada), 2021. [10] Z. Abu-Shaban et al., “Near-field localization with a reconfigurable intelligent surface acting as lens,” in Proc. IEEE ICC, (Montreal, Canada), 2021. [11] H. Zhang et al., “Towards ubiquitous positioning by leveraging reconfigurable intelligent surface,” IEEE Commun. Let, vol. 25, no. 1, pp. 284–288, 2021. [12] C. L. Nguyen et al., “Wireless fingerprinting localization in smart environments using reconfigurable intelligent surfaces,” IEEE Access, vol. 9, pp. 135526–135541, 2021. [13] K. Keykhosravi et al., “Leveraging RIS-enabled smart signal propagation for solving infeasible localization problems: Scenarios, key research directions, and open challenges,” IEEE Veh. Technol. Mag., vol. 18, no. 2, pp. 20–28, 2023. [14] H. Chen et al., “RISs and sidelink communications in smart cities: The key to seamless localization and sensing,” IEEE Commun. Mag., vol. 61, no. 8, pp. 140–146, 2023. [15] C. Huang et al., “Reconfigurable intelligent surfaces for energy efficiency in wireless communication,” IEEE Trans. Wireless Commun., vol. 18, no. 8, pp. 4157–4170, 2019. [16] G. C. Alexandropoulos et al., “Reconfigurable intelligent surfaces for rich scattering wireless communications: Recent experiments, challenges, and opportunities,” IEEE Commun. Mag., vol. 59, no. 6, pp. 28–34, 2021. [17] Q. Wu and R. Zhang, “Towards smart and reconfigurable environment: Intelligent reflecting surface aided wireless network,” IEEE Commun. Mag., vol. 58, no. 1, pp. 106–112, 2020. [18] G. C. Alexandropoulos et al., “RIS-enabled smart wireless environments: Deployment scenarios, network architecture, bandwidth and area of influence,” EURASIP J. Wireless Commun. Netw., vol. 103, pp. 1–38, 2023. [19] K. Stylianopoulos et al., “Asymptotically optimal closed-form phase configuration of 1-bit RISs via sign alignment,” in Proc. IEEE SPAWC, (Lucca, Italy), 2024. [20] C. Huang et al., “Reconfigurable intelligent surface assisted multiuser MISO systems exploiting deep reinforcement learning,” IEEE J. Sel. Areas Commun., vol. 38, no. 8, pp. 1839– 1850, 2020. [21] K. Stylianopoulos and G. C. Alexandropoulos, “Online RIS configuration learning for arbitrary large numbers of 1-bit phase resolution elements,” in Proc. IEEE SPAWC, (Oulu, Finland), 2022. [22] G. C. Alexandropoulos et al., “Pervasive machine learning for smart radio environments enabled by reconfigurable intelligent surfaces,” Proc. IEEE, vol. 110, no. 9, pp. 1494–1525, 2022. [23] K. Stylianopoulos et al., “Deep contextual bandits for orchestrating multi-user MISO systems with multiple RISs,” in Proc. IEEE ICC, (Seoul, South Korea), 2022. [24] G. Stamatelis et al., “Evolving multi-branch attention convolutional neural networks for online RIS configuration,” IEEE Trans. Cogn. Commun. Netw., vol. 12, pp. 14–28, 2025. [25] H. Chernoff, “Sequential design of experiments,” Ann. Math. Stat., vol. 30, no. 3, pp. 755–770, 1959. [26] M. Franceschetti et al., “Chernoff test for strong-or-weak radar models,” IEEE Trans. Signal Process., vol. 65, no. 2, pp. 289– 302, 2017. [27] K. Cohen and Q. Zhao, “Active hypothesis testing for anomaly detection,” IEEE Trans. Inf. Theory, vol. 61, no. 3, pp. 1432– 1450, 2015. [28] F. Sohrabi et al., “Deep active learning approach to adaptive beamforming for mmWave initial alignment,” in Proc. IEEE ICASSP, (Toronto, Canada), 2021. [29] G. Stamatelis and N. Kalouptsidis, “Active hypothesis testing in unknown environments using recurrent neural networks and model free reinforcement learning,” in Proc. EUSIPCO, (Helsinki, Finland), 2023.
VOLUME ,
[30] F. Sohrabi et al., “Active sensing for communications by learning,” IEEE J. Sel. Areas Commun, vol. 40, no. 6, pp. 1780– 1794, 2022. [31] G. Stamatelis et al., “Single- and multi-agent private active sensing: A deep neuroevolution approach,” in Proc. IEEE ICC, (Denver, Colorado, USA), 2024. [32] Z. Zhang et al., “Localization with reconfigurable intelligent surface: An active sensing approach,” IEEE Trans. Wireless Commun., vol. 23, no. 7, pp. 7698–7711, 2024. [33] S. Hochreiter and J. Schmidhuber, “Long short-term memory,” Neural Computation, vol. 9, no. 8, pp. 1735–1780, 1997. [34] G. Stamatelis and G. C. Alexandropoulos, “Lagrangian deep learning for private RIS-aided localization: An active sensing approach,” in Proc. IEEE ICASSP, (Barcelona, Spain), 2026. [35] M. Jian et al., “Reconfigurable intelligent surfaces for wireless communications: Overview of hardware designs, channel models, and estimation techniques,” Intell. Converg. Netw., vol. 3, no. 1, pp. 1–32, 2022. [36] P. Gavriilidis et al., “MIMO communications with 1-bit RIS: Asymptotic analysis and over-the-air channel diagonalization,” in Proc. IEEE Asilomar, (Pacific Grove, USA), October 2025. [37] Y. Zhou et al., “Wideband sub-THz reconfigurable intelligent surface using planar tightly coupled dipoles,” IEEE Antennas and Wireless Propagation Letters, vol. 23, pp. 3659–3663, November 2024. [38] G. Anastasi et al., “Energy conservation in wireless sensor networks: A survey,” Ad Hoc Netw., vol. 7, no. 3, pp. 537– 568, 2009. [39] V. Raghunathan et al., “Energy-aware wireless microsensor networks,” IEEE Signal Process. Mag., vol. 19, no. 2, pp. 40– 50, 2002. [40] G. J. Pottie and W. J. Kaiser, “Wireless integrated network sensors,” Commun. ACM, vol. 43, no. 5, p. 51–58, 2000. [41] B. Teng et al., “Bayesian user localization and tracking for reconfigurable intelligent surface aided MIMO systems,” IEEE J. Sel. Top. Signal Process, vol. 16, no. 5, pp. 1040–1054, 2022. [42] B. Wang et al., “Performance limit and phase design for nearfield tracking with reconfigurable intelligent surface,” Digital Signal Process., vol. 167, p. 105426, 2025. [43] Y. Mei et al., “Multi-user tracking in reconfigurable intelligent surface aided near-field wireless communications system,” Appl. Sci., vol. 14, no. 1, 2024. [44] X. Yu and M. Gen, Introduction to Evolutionary Algorithms. Springer London, 2010. [45] D. E. Moriarty and R. Miikkulainen, “Efficient reinforcement learning through symbiotic evolution,” Mach. Learn., no. AI94-224, pp. 11–32, 1996. [46] F. J. Gomez and J. Schmidhuber, “Co-evolving recurrent neurons learn deep memory POMDPs,” in Proc. GECCO, (Washington, DC, USA), 2005. [47] F. Petroski et al., “Deep neuroevolution: Genetic algorithms are a competitive alternative for training deep neural networks for reinforcement learning,” arXiv preprint: 1712.06567, 2017. [48] T. Salimans et al., “Evolution strategies as a scalable alternative to reinforcement learning,” arXiv preprint: 1703.03864, 2017. [49] P. Chrabaszcz et al., “Back to basics: Benchmarking canonical evolution strategies for playing atari,” in Proc. IJCAI, (Stockholm, Sweden), 2018. [50] D. D. Wentzloff et al., “Ultra-low power receivers for IoT applications: A review,” in Proc. IEEE CICC, (Virtual), 2020. [51] A. Froytlog et al., “Ultra-low power wake-up radio for 5G IoT,” IEEE Commun. Mag., vol. 57, no. 3, pp. 111–117, 2019. [52] M. Gruber, “An approach to target tracking,” technical report, MIT Lincoln Laboratory, Lexington, MA, 1967. [53] C. M. Carvalho et al., “Particle learning and smoothing,” Statistical Science, vol. 25, no. 1, pp. 88–106, 2010. [54] F. Saggese et al., “On the impact of control signaling in RIS-empowered wireless communications,” IEEE Open J. Commun. Soc, vol. 5, pp. 4383–4399, 2024. [55] M. Matthaiou et al., “Analytic framework for the effective rate of MISO fading channels,” IEEE Trans. Commun., vol. 60, no. 6, pp. 1741–1751, 2012.
VOLUME ,
[56] G. C. Alexandropoulos, “Position aided beam alignment for millimeter wave backhaul systems with large phased arrays,” in Proc. IEEE CAMSAP, (Curacao, Dutch Antilles), 2017. [57] K.-H. Ngo et al., “Noncoherent MIMO communications: Theoretical foundation, design approaches, and future challenges,” IEEE Open J. Veh. Technol., vol. 7, pp. 381–401, 2026. [58] G. C. Alexandropoulos et al., “Near-field hierarchical beam management for RIS-enabled millimeter wave multi-antenna systems,” in Proc. IEEE SAM, (Trondheim, Norway), 2022. [59] V. Krishnamurthy, Partially Observed Markov Decision Processes: Filtering, Learning and Controlled Sensing. Cambridge University Press, 2nd ed., 2025. [60] C. H. Papadimitriou and J. N. Tsitsiklis, “The complexity of Markov decision processes,” Math. Operat. Research, vol. 12, no. 3, pp. 441–450, 1987. [61] D. S. Bernstein et al., “The complexity of decentralized control of Markov decision processes,” Math. Operat. Research, vol. 27, no. 4, pp. 819–840, 2002. [62] 3GPP, “NR; physical layer procedure for data (Release 15),” 2017. [63] F. Gomez et al., “Accelerated neural evolution through cooperatively coevolved synapses,” J. Mach. Learn. Res., vol. 9, no. 31, pp. 937–965, 2008. [64] S. Kaddouri et al., “Indoor path loss measurements and modeling in an open-space office at 2.4 ghz and 5.8 ghz in the presence of people,” in Proc. IEEE PIMRC, (Bologna, Italy), 2018. [65] R. E. Larson et al., “Application of the extended Kalman filter to ballistic trajectory estimation,” Stanford Research Institute, Tech. Rep., 1967. [66] I. Urtcaga et al., “Sequential monte carlo methods under model uncertainty,” in Proc. IEEE SSP, (Palma de Mallorca, Spain), 2016. [67] V. Mnih et al., “Asynchronous methods for deep reinforcement learning,” in Proc. ICML, (NY, USA), 2016. [68] A. Ray et al., “Benchmarking safe exploration in deep reinforcement learning,” arXiv preprint: 1910.01708, 2019. [69] A. Raffin et al., “Stable-baselines3: Reliable reinforcement learning implementations,” J. Mach. Learn. Research, vol. 22, no. 268, pp. 1–8, 2021. [70] A. F. Gad, “Pygad: An intuitive genetic algorithm python library,” Multimedia Tools and Applications, pp. 1–14, 2023. [71] Z. Zhang et al., “Active sensing for localization with reconfigurable intelligent surface,” in Proc. IEEE ICC, (Rome, Italy), 2023. [72] G. Stamatelis et al., “Multi-agent actor-critic with harmonic annealing pruning for dynamic spectrum access systems,” in Proc. EUSIPCO, (Palermo, Italy), 2025. [73] G. Revach et al., “KalmanNet: Neural network aided kalman filtering for partially known dynamics,” IEEE Trans. Signal Process., vol. 70, pp. 1532–1547, 2022. [74] G. Stamatelis and G. C. Alexandropoulos, “Filtering Markov jump systems with partially known dynamics: A model-based deep learning approach,” IEEE Trans. Signal Process., 2026 (to appear).
17