1
mmAlert: A Simultaneous Device Localization and Target Tracking System via Cooperative Passive Sensing
arXiv:2606.01653v1 [cs.NI] 1 Jun 2026
Chao Yu, Bojie Lv, Chunxi Chen, Jingwen Zhang and Rui Wang
Abstract—In this paper, a cooperative passive sensing system in millimeter-wave (mmWave) band for simultaneous device localization and target tracking, namely mmAlert, is proposed. Specifically, in uplink communication with at least two transmitters, the receiver receives the line-of-sight (LoS) signals and the scattered signals off a moving target, respectively. Based on the received signals of the sensing time intervals, when a passive target moves along one or multiple unknown trajectories, mmAlert could measure the angles-of-arrival (AoAs) and bistatic Doppler frequencies of the echoes from the sensing target, and then jointly estimate the locations of the transmitters and the trajectories of the target. Specifically, the transmitters’ locations and the moving target’s trajectories can be searched by minimizing the weighted mean squared error of the AoA and Doppler measurements. The optimal solution of the minimization problem is prohibitive due to the large number of variables. Hence, a low-complexity algorithm based on the alternating optimization is proposed, where the extended Kalman filter (EKF) is introduced to quickly shape the trajectories. The mmAlert is implemented in a 60GHz communication testbed. The experiment shows with the received signal spanning a single trajectory, the average localization error of the transmitters and average trajectory reconstruction error are 0.76 m and 0.29 m, respectively. The average errors are suppressed to 0.07 m and 0.2 m respectively, if the received signal spanning 50 trajectories is used. This justifies the benefit of trajectory diversity in localization and tracking.
the locations of multiple coordinated sensing devices, whereas localization of the device itself relies on either collaborative positioning using multiple receivers or wideband signals for accurate ranging. In this paper, we would like to show that the tracking of passive moving target and the localization of communication devices might facilitate each other in an mmWave communication system, when either task cannot be accomplished solely. A. Device Localization
In the existing literature, the localization of communication devices with inband sensing usually relies on the measurement of signal fingerprints in advance [13]–[15] or prior knowledge on the locations of multiple anchor devices (e.g., access points or base stations) [16]–[19]. For example, in [13], a fingerprint database of received signal strength indicator (RSSI) was constructed for device localization, where a localization and tracking algorithm based on recurrent neural networks (RNNs) was proposed. It leveraged the continuity of device movement to improve localization accuracy. Moreover, since the channel state information (CSI) contains more fine-grained environmental Index Terms—Localization, trajectory reconstruction, coopera- information, the fingerprint based on CSI is expected to yield higher localization accuracy [14], [15]. For example, in [15], a tive passive sensing novel indoor wireless localization algorithm based on a broad I. I NTRODUCTION learning system (BLS) was proposed, which achieved meterMWAVE communication is a key technology for future level localization accuracy using CSI fingerprint. However, all wireless communication systems due to its abundant the fingerprint-based methods require a significant effort of spectrum resource [1]–[3]. However, the short wavelength collecting signal features in advance. Moreover, the changes in of mmWave signals also brings inherent disadvantages, such the environment, e.g., walking persons, might severely degrade as severe signal attenuation, susceptibility to obstacles, and the localization accuracy. frequent beam alignment [3]–[5]. Fortunately, the integrated Wireless communication devices can also be localized based sensing and communication (ISAC) technology offers a promis- on measurements of angle-of-arrival (AoA) [16], [17] and timeing solution to relieve the above issues [6]–[8]. For instance, of-flight (ToF) [18]–[20] at multiple anchor devices, whose it is shown that the wireless communication systems could locations are known in advance. For example, in [16], it was locate moving obstacles [9], [10] or its devices [11], [12] via proposed to first estimate the AoAs from three multi-antenna sensing techniques. Hence, potential mmWave signal blockage access points (APs) and then infer the target position using can be predicted, and fast beam alignment can be facilitated triangulation. The algorithm achieved a median positioning via location information. In the existing literature, device error of 57 cm. In [17], a joint AoA estimation method with localization and passive target (i.e., a target that does not emit multiple APs was proposed. Considering the coupling between signals) tracking are investigated in a separate manner. The the AoA measurements of multiple APs with respect to the prerequisites of the existing techniques might be stringent. For same target, the AoAs should not be estimated independently. example, tracking of passive target requires prior knowledge on Instead, AoA estimation at all the APs was modeled as a joint parametric optimization problem to improve the localization Chao Yu, Bojie Lv and Rui Wang are with the Department of Electrical and Electronic Engineering, Southern University of Science and Technology, accuracy. The ToF of a communication signal is usually Shenzhen, China. Chunxi Chen and Jingwen Zhang are with the College of measured by the round trip time (RTT). A trilateration algorithm Semiconductors (National Graduate College for Engineers), Southern University based on the RTT was proposed in [19] for device localization. of Science and Technology, Shenzhen, China. It was shown that the average localization error was 1.2 m This work has been accepted for publication in IEEE Transactions on Wireless Communications (TWC). DOI: 10.1109/TWC.2026.3698942 when the Wi-Fi signal bandwidth was 40MHz. In addition to
M
2
the RTT, the ToF can also be measured via the phase difference C. Our Contributions of a multi-carrier system. For example, a multi-carrier ranging As discussed in the previous parts, the localization of algorithm based on orthogonal frequency division multiplexing communication devices and the trajectory tracking of passive (OFDM) signals was proposed in [20]. It was shown that targets are treated separately in the existing literature. In fact, the median ranging errors of 0.65 m and 0.98 m could be the measurements of AoA, ToF, and Doppler frequency for traachieved in line-of-sight (LoS) and non-line-of-sight (NLoS) jectory tracking strongly depend on the locations of transmitters scenarios, respectively. Nevertheless, it relied on a complicated and receivers in ISAC systems. This facilitates the joint design calibration algorithm to mitigate the effect of carrier frequency of passive target tracking and device localization. Moreover, it offset (CFO) between the transmitter and receiver. In all the is much easier to detect the AoA and Doppler frequency of a above works, prior location knowledge on multiple anchor wireless communication signal than its propagation distance, devices is necessary. Moreover, stringent requirements, such due to the limited signal bandwidth and asynchronous clocks as large signal bandwidth and fine synchronization among the between the transmitter and the receiver. Hence, it is interesting devices, should be satisfied in the measurement of ToF. In to raise the following questions: [21], the multipath relative time-of-flight (rToF) was exploited • Could the device localization and passive target tracking to estimate the triangle between the LoS path and reflected be conducted jointly? Or could the trajectory of a passive paths, such that the transmitter could be localized. In this work, target be tracked in an ISAC system without the location sufficiently large signal bandwidth (40MHz) was necessary to information of the communication devices? distinguish reflected paths from the LoS path in the channel • Could the above joint estimation be conducted with only impulse response (CIR). Although it did not rely on the prior angle and Doppler measurements? location knowledge, specific NLoS environment was required. In this paper, we would like to shed some lights on the above questions by proposing a method of simultaneous localization B. Trajectory Reconstruction and tracking (SLAT) for an mmWave ISAC system, namely There have been numerous studies investigating the trajectory mmAlert. Specifically, at least two transmitters simultaneously reconstruction of passive moving targets via half-duplex com- transmit uplink signals to one receiver in different frequency munication devices. For example, the Widar system proposed bands, and there is one moving target in the communication in [22] tracked human motion trajectories by detecting bistatic environment. The transmitters’ locations and the trajectories of Doppler frequencies through CSI. It could achieve a decimeter- the moving target with respect to the receiver are all unknown level accuracy with at least 6 receive RF chains. The IndoTrack parameters, which usually happens in communication systems. system proposed in [23] leveraged bistatic Doppler frequency In this scenario, the existing target tracking methods cannot be and AoA measurements of 6 receive RF chains for trajectory applied due to the lack of communication devices’ locations; the reconstruction, achieving an error margin within 0.48 m. The localization of communication devices suffers from sampling WiDFS system proposed in [24] utilized three antennas on a and carrier frequency offsets. Instead, the proposed mmAlert is single receiver to cancel the carrier spectrum interference, such capable of estimating the above unknown parameters with high that the bistatic Doppler frequency and AoA information could accuracy. The estimation is facilitated by the movement of the be estimated for tracking. An average tracking error of 0.72 target. Multiple trajectories of the target are sensed. During the m was verified by the experimental results. Additionally, the sensing phase, the receiver collects the LoS signal from the two Witraj system proposed in [25] relied solely on bistatic Doppler transmitters and the scattered signal off the sensing target, such frequencies in trajectory reconstruction. When the positions of that the AoA and Doppler frequency of the scattered signal can the sensing targets at the very beginning, one transmitter and be detected. Based on these detections of multiple trajectories, three receivers were known, Witraj achieved a median trajectory the SLAT design can be formulated as an optimization problem: error of 0.3 m. An ML-Track algorithm was proposed in [26], find the best locations of the transmitters and the trajectories which employed a round-trip CSI mechanism to eliminate of the sensing target to minimize the weighted summation of the impact of CFO in Doppler frequency estimation. The AoA and Doppler measurement errors. The main contributions experimental results demonstrated that under 4-link conditions, of this paper are summarized below. the algorithm attained a median error of 0.23 m. 1 To the best of our knowledge, this is the first work Passive sensing is another promising approach for trajectory exploiting the joint detection of communication devices’ tracking using half-duplex communication transceivers. For locations and multiple sensing targets’ trajectories. Alinstance, it was demonstrated in [27] that human movements though the mmAlert is implemented in an mmWave behind walls could be tracked by exploiting Wi-Fi signals in communication system, the principle could be extended passive sensing. Meanwhile, a passive sensing system based to other communication systems, e.g., the Wi-Fi system. on mmWave communication signals was proposed in [28] Compared with our prior work [30] on SLAT for a single to achieve millimeter-level handwriting tracking. A passive trajectory, the trajectory diversity is exploited in this sensing system based on long-term evolution (LTE) signals work, such that the tracking and localization accuracy is was proposed in [29], which achieved a meter-level tracking significantly improved. accuracy for a low-altitude drone flying hundreds of meters 2 The above optimization problem for joint detection has away from the base station. In all the above works, the locations not been investigated before, owing to the novelty of of transmitters and receivers are known in advance. the joint design. Due to the high dimensionality of
3
the variable space, obtaining the optimal solution is computationally prohibitive. Therefore, a suboptimal, low-complexity solution framework integrating alternating optimization (AO) and the extended Kalman filter (EKF) is proposed. Particularly, the optimization variables consist of the transmitters’ locations, the initial positions and shapes of the trajectories. Given the locations, the initial position and shape of each trajectory could be optimized, respectively; and given multiple trajectories, the transmitters could be located with a good accuracy. In the former step, given the initial position of a trajectory, the trajectory shape could be obtained via EKF; and given the trajectory shape, the gradient of the initial position could be derived analytically. 3 The performance of the proposed SLAT design is demonstrated in a 60GHz ISAC testbed, whose measurements are released for open access1 . Experimental results demonstrate that mmAlert achieves an average trajectory reconstruction error of 0.29 m only using the measurements of a single trajectory. When the measurements on multiple trajectories are available, the transmitter localization error reduces from 0.76 m to 0.07 m, and the trajectory reconstruction error decreases from 0.29 m to 0.2 m. The remainder of this paper is structured as follows. In Section II, an overview of the system is provided. In Section III, the system’s motion model and uplink transmission model are elaborated. In Section IV, the estimation methods for bistatic Doppler frequency and AoA are described. In Section V, the joint estimation problem in SLAT design is formulated, then in Section VI, a low-complexity solution algorithm is proposed. In Section VII, the experimental results and analysis are presented. Finally, the conclusion is drawn in Section VIII. II. OVERVIEW OF MM A LERT S YSTEM Reference beam Surveillance beam
Tracking NLoS Paths
Moving Target TX1
RX LoS Paths
TX2
Localization
Fig. 1: Illustration of deployment scenario of the mmAlert system. In this paper, a simultaneous localization and tracking (SLAT) system in mmWave band, namely mmAlert, is proposed. The 1 https://github.com/lasso-sustech/mmAlert
mmAlert system can be deployed in the mmWave uplink communication scenario with at least two transmitters (user equipments), one receiver (base station) and a moving target, as illustrated in Fig. 1. It enables the localization of the transmitters and the trajectory reconstruction of the moving target via communication signals. Such that potential link blockage can be predicted. With the localization and tracking results, the human motion recognition with the mmWave communication signal could also be facilitated. In mmAlert, one phased array is adopted at each transmitter for transmit beamforming, and three phased arrays 2 are adopted at the receiver for joint sensing and communication. The receiver receives signals from the two transmitters in two orthogonal frequency bands, respectively. In data transmission, the LoS path 3 from each transmitter to the receiver is dominant. Hence, the receiver aligns two of its receive beams with the LoS paths from the two transmitters, respectively. Meanwhile, the uplink signal from each transmitter may scatter at the moving target, carrying the motion information of the target. The third receive beam of the receiver rotates periodically to capture the scattered signal from the moving target in both frequency bands, yielding a NLoS propagation path. The signal-to-noise ratio (SNR) of the NLoS path is significantly weaker than that of the LoS path, especially in mmWave band. Hence, there is no benefit of exploiting the NLoS signal in data communication. However, the proposed mmAlert could utilize the weak NLoS signal to locate the transmitters and reconstruct the trajectory of the moving target. From the perspective of passive sensing, LoS paths and NLoS paths scattered off the moving target are referred to as the reference and surveillance channels, respectively, and the corresponding receive beams are referred to as the reference and surveillance beams respectively. Denote the two transmitters as transmitter 1 and 2, the two reference beams as reference beam 1 and 2, the reference and surveillance channels from the transmitter 1 and 2 as the reference channel 1 and 2, surveillance channel 1 and 2, respectively. On one hand, the bistatic Doppler frequencies (the Doppler frequencies of the surveillance channels) can be estimated by comparing the receive signals of the surveillance and the reference channels. As a remark, due to the carrier frequency offset (CFO) between the transmitters and the receiver, the bistatic Doppler frequencies could hardly be accurately measured solely via the surveillance channels. The reference channel could help to eliminate the effect of CFO [27], [28]. On the other hand, since the surveillance beam rotates periodically, its direction with the maximum echo signal power can be treated as the AoA of both surveillance channels (Note that the AoAs of both surveillance channels are identical). Without prior knowledge of the target’s initial location or the transmitters’ positions, mmAlert utilizes only short term observations of the bistatic Doppler frequencies and AoAs from two surveillance channels. It employs a low-complexity search 2 In fact, the three phased arrays can be replaced by one hybrid antenna array as in [31], which provides multiple RF chains and multiple beams in one single antenna array. 3 It should be noted that mmAlert does not require that the reference channel follows a LoS path, a strong and static NLoS path can also works.
4
algorithm to jointly estimate the positions of both transmitters and the trajectory of the moving target relative to the receiver, achieving a close match to the observations with minor error. Moreover, if multiple trajectories of the moving target could be collected in different time periods 4 , the diversity of these trajectories can be leveraged. Under the condition that the locations of the two transmitters are fixed, such diversity helps improve both the localization accuracy of the transmitters and the reconstruction accuracy of the trajectories. Finally, as a remark, although the mmAlert is designed for FDMA (or OFDMA) uplink systems, the method can also be extended to TDMA or CDMA systems, as long as the signals of the two reference channels and the two surveillance channels can be separated.
, j-th trajectory Surveillance 1 Surveillance 2 RX
TX2
III. S YSTEM M ODEL Without loss of generality, the positions of the transmitter 1, 2 and the receiver are denoted as p1 = [xTX1 , 0]T , p2 = [xTX2 , yTX2 ]T and pr = [0, 0]T , respectively, as illustrated in Fig. 2. Based on the existing multiple signal classification (MUSIC) algorithm [32], [33], which is applicable to analog MIMO architectures, the receiver can estimate the signal AoA from the two transmitters, and align two of its receive beams, namely reference beams 1 and 2, with reference channels 1 and 2, respectively. Hence, the angle difference of arrival φRX can be calculated from the two AoAs. Similarly, the angle φTX1 in Fig. 2 can also be estimated by swapping the roles of transmitter and receiver. Hence, it is assumed that the angles of φRX and φTX1 are known in this paper. As a remark, when the positions of the transmitters change, both φRX and φTX1 should be recalculated. Therefore, both φRX and φTX1 are known to the mmAlert system. According to the property of triangle, p2 can be determined by sin φTX1 cos φRX xTX2 = xTX1 sin(φTX1 + φRX ) , (1) sin φTX1 sin φRX xTX1 yTX2 = sin(φTX1 + φRX ) as long as xTX1 can be estimated. The receiver rotates its surveillance beam periodically among the directions Φ = {ϕ1 , ϕ2 , · · · , ϕQ }, as in Fig. 2. The duration that the surveillance beam stops at one direction is Tb , and the sweeping period that the surveillance beam traverses all the directions is Td = QTb . To facilitate the simultaneous localization and tracking, it is assumed that the receiver has collected the receive signals of J (J ≥ 1) trajectories, respectively. In the remaining of this section, the motion and signal models of the j-th trajectory (j = 1, 2, · · · , J) will be elaborated.
TX1
Surveillance Beam Beam 1
Beam 2 RX
RX
Beam 3 RX
RX
Beam 4
Fig. 2: Illustration of SLAT scenario with the number of surveillance beam directions Q=4. y T x sweeping period as pj,k = [xj,k , yj,k ]T and v j,k = [vj,k , vj,k ] , T respectively, where [·] denotes the vector (matrix) transpose. The trajectory of the moving target can then be expressed as
pj,k = pj,k−1 + v j,k−1 Td = pj,1 +
k−1 X
(2)
v j,n Td , k = 2, 3, · · · , Kj ,
n=1
where Kj is the total number of sweeping periods of the j-th trajectory. Moreover, the AoA ϕj,k of the echo signal scattered off the target can be written as ϕj,k = atan2(yj,k , xj,k ),
(3)
A. Motion Model Since the sweeping period is very short, it can be approximated that the target moves with a constant velocity in a sweeping period. Denote the initial position of the moving target as pj,1 = [xj,1 , yj,1 ]T , and the starting position and velocity of the moving target in the j-th trajectory and the k-th
where atan2(·, ·) denotes the four-quadrant inverse tangent. Given the above motion model, the bistatic Doppler frequency of the m-th surveillance channel (m = 1, 2) is given by pj,k − pm pj,k − pr T 1 m fj,k = − + v j,k , (4) λm ∥pj,k − pm ∥ ∥pj,k − pr ∥
4 For example, different persons walk through the communication environment in different time periods.
where λm is the carrier wavelength of the signal from the m-th transmitter. As a result, given the j-th trajectory, denoted
5
as Tj = [pj,1 , v j,1 , · · · , v j,Kj ], and the locations of the two transmitters, the ground truth of the AoAs and bistatic Doppler frequencies of all sweeping periods are given by (3) and (4), respectively. Note that the AoAs and bistatic Doppler frequencies are measured via passive sensing, the difference between the ground truth and the measurements can be evaluated. The mmAlert will tune the transmitters’ locations and the trajectories, such that the corresponding AoAs and bistatic Doppler frequencies best fit the measurements. B. Uplink Transmission Model In the k-th sweeping period (∀k), let xm,j k,q (t), t ∈ [0, Tb ], be the baseband signal of m-th transmitter (m = 1, 2) when the surveillance beam is at the direction ϕq (q = 1, 2, · · · , Q), the corresponding receive signal via the reference beam m can be written as m,j m,j m,j m,j rk,q (t) = hm,j k,q xk,q (t − τk,q ) + nk,q (t), 0 ≤ t ≤ Tb , (5) m,j where hm,j k,q and τk,q denote the baseband channel gain and delay of the LoS path (reference channel) respectively, nm,j k,q (t) denotes the superposition of noise and NLoS echoes. The power m,j m,j of hm,j k,q xk,q (t − τk,q ) is usually much stronger than that of nm,j k,q (t). Similarly, the receive baseband signal of the surveillance beam from the m-th transmitter includes the echo signals from the moving target and static scattering clusters, which can be written as m m,j m,j m,j sm,j (t) = h̃ x t − τ̃ ej2πfj,k t + k,q k,q k,q k,q Lm,j k,q
X
(6) m,j hl xm,j k,q (t − τl ) + ñk,q (t), 0 ≤ t ≤ Tb ,
l=1 m,j m where h̃m,j k,q , τ̃k,q and fj,k denote the quasi-static baseband channel gain, delay and Doppler frequency of the surveillance channel, respectively. Moreover, Lm,j k,q is the number of paths scattered from static clusters, hl and τl are the complex gain and delay of the l-th one, ñm,j k,q (t) denotes the noise. As a remark, if the target is not at the direction ϕq of the receiver, the channel gain h̃m,j k,q would be of small magnitude. Moreover, the signal from the LoS path may also be received by the surveillance beam, which can be treated as a special static scattering cluster in the second term of the above equation. Both receive signals of reference and surveillance beams are sampled at the baseband with a period Ts , respectively. The sampled signals can be expressed by m,j m,j rk,q [n] = rk,q (nTs )
and
m,j sm,j k,q [n] = sk,q (nTs ),
where n = 1, 2, · · · , Tb /Ts . Note that the second term in right hand side of (6), which is of zero Doppler frequency, may interfere the estimation of the target Doppler frequency m fj,k . The least-square-based (LS-based) clutter cancellation method elaborated in [34] is applied for suppressing the above interference. Denote the surveillance signal after clutter cancellation as s̃m,j k,q [n] (∀k, q, m, j).
IV. D OPPLER F REQUENCY & AOA E STIMATION In this section, the estimation of bistatic Doppler frequency and AoA for the j-th trajectory in the k-th sweeping period, ∀j, k, is elaborated. The bistatic Doppler frequency is estimated according to the cross-ambiguity function (CAF) between m,j the reference signal rk,q [n] and surveillance signal s̃m,j k,q [n]. Particularly, the CAF of the receive signals from the m-th transmitter in the k-th sweeping period at the q-th surveillance beam’s direction is defined as N n o∗ X m,j m Rj,k (q, f ) = max s̃m,j e−j2πf nTs k,q [n] rk,q [n − τ ] τ
n=1
(7) ∗
where {·} is the complex conjugate, | · | is the magnitude, N = Tb /Ts denotes the number of samples when the surveillance beam stops at one direction. Since we only focus on the detection of Doppler frequency, the delay τ is not considered as a parameter of the CAF. In fact, when the signal bandwidth and sampling frequency are low, the delay between m,j rk,q [n] and s̃m,j k,q [n] is negligible. It can be observed that there m m would be a peak value of Rj,k (q, f ) when f = fj,k . As a remark, note that the AoAs of both surveillance channel m 1 and 2 are identical. Denote f˜j,k and ϕ̃j,k as the estimated bistatic Doppler frequency and AoA in the k-th sweeping period, we have m ˜m m (q̃j,k , fj,k ) = arg max Rj,k (q, f ),
(8)
q,f
m . and ϕ̃j,k = ϕq̃j,k As a result, the measurement vector z j,k in the k-th sweeping period is defined as
1 2 T z j,k = [ϕ̃j,k , f˜j,k , f˜j,k ] .
(9)
The measurement matrix of the j-th trajectory is then given by Zj = [z j,1 , z j,2 , · · · , z j,Kj ],
(10)
where Kj is the total number of sweeping periods in the j-th trajectory. The aggregation of the measurement matrices of all the J trajectories is given by the set Z = {Z1 , Z2 , · · · , ZJ }. The estimated bistatic Doppler frequencies and AoAs of example trajectories according to experiment data will be illustrated in Section VII-A. As a remark, when a volunteer is considered as the moving target, the movements of different body parts generate distinct Doppler frequency components. In this case, the strongest component can usually represent the torso’s motion, due to the maximum radar cross section (RCS) of the torso compared with other body parts. Hence, we can still use the strongest Doppler frequency component as the detected Doppler frequency. Moreover, the mmAlert proposed in this paper is designed specifically for single-target scenarios. The Doppler frequency and AoA estimation methods employed in this paper are both intended for single-target cases. When multiple targets are present, the AoA-Doppler spectrum in a single sweeping period may exhibit more than one peak. Therefore, to address
6
the multi-target tracking problem, methods such as multi-target is large, leading to a significant computation complexity. tracking (MTT) [35] are required to capture the temporal Although the Gaussian Newton method [36] and the LevenbergAoA-Doppler characteristics of different targets. Although Marquardt (LM) algorithm [37], [38] are conventionally the proposed mmWave integrated sensing and communication employed to solve least squares problems, these methods are system has the potential to support multi-target sensing, this characterized by their high computational complexity, when falls beyond the scope of the algorithm presented in this paper. the number of variables is large, and sensitivity to the initial solution. In the following section, a novel low-complexity V. SLAT P ROBLEM F ORMULATION solution algorithm exploiting the structure of SLAT problem In this section, we shall formulate the simultaneous local- is proposed. ization and tracking design as an optimization problem. For VI. L OW-C OMPLEXITY S OLUTION A LGORITHM the notation convenience, we first define the j-th trajectory as Note that in problem P1, given the locations of the two matrix transmitters, the search of the J trajectories can be decoupled. Tj = [pj,1 , v j,1 , v j,2 , · · · , v j,Kj ], This motivates us an alternating optimization framework: given and the aggregation set of all the J trajectories is given by the transmitters’ locations, find the optimal J trajectories to minimize the WMSE of the measurements, respectively. Then, T = {T1 , T2 , · · · , Tj }. given the J trajectories, find the optimal transmitters’ locations Given the two transmitters’ locations M = {p1 , p2 } and to minimize the overall WMSE of all trajectories. The above the j-th trajectory Tj , the AoAs and the bistatic Doppler two optimizations are conducted iteratively, which is referred frequencies of the two surveillance channels in all the sweeping to as the outer iteration. periods are all determined. Hence, denote the aggregation Particularly, let M0 be the initial solution of the transmitters’ matrix of actual AoAs and bistatic Doppler frequencies of both locations, and Ml−1 be the solution after the (l − 1)-th outer surveillance channels given Tj and M as iteration, the optimization of the problem P1 in the l-th outer iteration, l = 1, 2, ..., consists of the following two ϕj,1 ϕj,2 ··· ϕj,Kj sub-problems: 1 1 1 fj,2 ··· fj,Kj , H(Tj , M) = fj,1 (11) l l−1 2 2 2 fj,1 fj,2 ··· fj,K P2(l, j) : Tj = argTmin gj (Tj , M , Zj ), ∀j, (14) j j
m where ϕj,k and fj,k are calculated according to (3) and (4), respectively. Note that the AoAs and bistatic Doppler frequencies of all sweeping periods are measured by the receiver. The difference between H(Tj , M) and Zj (∀j) is the matrix of measurement error of the j-th trajectory. Hence, the weighted mean squared error (WMSE) between the measurement and the true values given the transmitters’ locations M and the the j-th trajectory Tj is defined as ( 1 T tr [Zj − H(Tj , M)] W gj (Tj , M, Zj ) ≜ Kj )
· [Zj − H(Tj , M)] ,
(12)
where tr{·} denotes the matrix trace, W = diag{w1 , w2 , w3 }, w1 , w2 and w3 denote the weights of different features respectively. The simultaneous localization and tracking problem can then be conducted by searching the optimal values of Tj (∀j) and M such that the summation of WMSE gj (Tj , M, Zj ) for all trajectories is minimized. Thus, P1 :
{T ∗ , M∗ } = arg min
J X
{T ,M} j=1
gj (Tj , M, Zj ).
(13)
and P3(l) :
l
M = arg min M
J X
gj (Tlj , M, Zj ),
(15)
j=1
where Tlj denotes the j-th trajectory after the optimization in the l-th outer iteration, respectively. According to the geometric relation in (1), problem P3 contains only one optimization variable, namely the distance xTX1 between Tx1 and Rx. It can therefore be solved efficiently using a one-dimensional search algorithm. However, the problem P2 is still complicated due to the large number of optimization variables: the initial position of the j-th trajectory and the two-dimensional velocities of all sweeping periods should be optimized simultaneously. In fact, the former determines the starting point of the trajectory, and the latter determines its shape. They can be optimized alternately. On one hand, given the trajectory shape, the optimization of starting point consists of only two variables. On the other hand, given the starting point, the existing tracking method, e.g., extended Kalman filter (EKF), can be used to shape the trajectory quickly according to the Doppler and AoA measurements. The above two steps can be conducted iteratively, which is referred to as the inner iteration. Specifically, denote the operator of shape estimation via the EKF, given the initial position of the trajectory pj,1 and measurement matrix Zj , as
Since the relations between the AoAs, the bistatic Doppler frequencies and the trajectories, the transmitters’ locations are V̂j ≜ [v̂ j,1 , v̂ j,2 , · · · , v̂ j,Kj ] = fEKF (pj,1 , Zj ), (16) nonconvex, the optimization problem P1 is also nonconvex. Furthermore, when the J trajectories and two transmitters’ where v̂ j,k is the estimated velocity of the k-th sweeping l,m locations are optimized simultaneously, the number of variables period. Let pl,m be the initial position and the shape j,1 and V̂j
7
of the j-th trajectory after m-th inner iteration in solving problem P2(l, j). In the m-th inner iteration (m = 1, 2, · · · ), the optimization of problem P2(l, j) consists of the following two steps: V̂jl,m = fEKF (pl,m−1 , Zj ) (17) j,1
Moreover, the measurement model, which maps the state to the passive sensing measurements, is given by z̃ j,k = h(sj,k ) + uj,k ,
(24)
where atan2(yj,k , xj,k ) 1 pj,k − p1 pj,k − pr T − v j,k + h(sj,k ) = λ1 ∥pj,k − p1 ∥ ∥pj,k − pr ∥ T pj,k − p2 pj,k − pr 1 v j,k − + λ2 ∥pj,k − p2 ∥ ∥pj,k − pr ∥ (25)
and
P4(j, l, m) :
(18)
l,m l−1 pl,m , Zj ). j,1 = arg min gj ([pj,1 , V̂j ], M pj,1
In the following two parts, the EKF-based trajectory reconstruction in (16) according to the AoA and bistatic Doppler frequency measurements, and the solution algorithm for problem P4 are elaborated, respectively. A. EKF-based Trajectory Reconstruction In this part, the extended Kalman filter for the reconstruction of the j-th trajectory according to the initial position pj,1 = [xj,1 , yj,1 ]T and the measurement matrix Zj = [z j,1 , z j,2 , · · · , z j,Kj ] is elaborated. First, the state vector of the j-th trajectory in the k-th sweeping period is defined as y T x sj,k = [xj,k , vj,k , yj,k , vj,k ] ,
which consists of the velocity and starting position of this sweeping period. Hence, the initial state is given by y T x sj,1 = [xj,1 , vj,1 , yj,1 , vj,1 ] ,
where the initial position pj,1 = [xj,1 , yj,1 ]T is known, and the initial velocity can be calculated according to pj,1 and the Doppler frequency measurement. Particularly, y T x ˜1 ˜2 T [vj,1 , vj,1 ] = A−1 f [fj,1 , fj,1 ] ,
(19)
where pj,1 − pr T pj,1 − p1 1 − + λ 1 ∥pj,1 − p1 ∥ ∥pj,1 − pr ∥ Af = T . 1 pj,1 − pr pj,1 − p2 − + λ2 ∥pj,1 − p2 ∥ ∥pj,1 − pr ∥
(20)
Given the above definition of trajectory state, we adopt the following state transition model sj,k = Fsj,k−1 + wj,k ,
(21)
where 1 0 F= 0 0
Td 1 0 0
0 0 1 0
0 0 Td 1
denotes the nonlinear measurement function, uj,k ∼ N (0, U) is the measurement noise, the covariance matrix U = diag(σϕ2 , σf2 , σf2 ). The trajectory reconstruction via the EKF consists of state prediction and state correction. Let y x sj,k|k−1 = [xj,k|k−1 , vj,k|k−1 , yj,k|k−1 , vj,k|k−1 ]T
denote the predicted state of the k-th sweeping period given the measurement up to the (k − 1)-th sweeping period, and y x sj,k|k = [xj,k|k , vj,k|k , yj,k|k , vj,k|k ]T
denote the corrected state of the k-th sweeping period given the measurement of the k-th sweeping period, Pj,k|k−1 and Pj,k|k be the predicted covariance matrix and corrected covariance matrix for sj,k|k−1 and sj,k|k respectively. The initial covariance matrix can be given by 2 σx 0 0 0 0 σv2 0 0 x , Pj,1|1 = (26) 0 0 σy2 0 0 0 0 σv2y where σx2 , σv2x , σy2 and σv2y denote the variances of position error and velocity error in the x-coordinate and y-coordinate, respectively. Note that sj,1|1 = sj,1 , the state prediction for the k-th sweeping period can be represented by ( sj,k|k−1 = Fsj,k−1|k−1 , ∀k > 1 . (27) Pj,k|k−1 = FPj,k−1|k−1 FT + Q Moreover, the state correction can be expressed as ( sj,k|k = sj,k|k−1 + Kj,k (z j,k − h(sj,k|k−1 )) , Pj,k|k = (I − Kj,k Cj,k )Pk|k−1
(28)
where I denotes the identity matrix, the near-optimal Kalman gain is written as (22)
is the state transition matrix, and wj,k ∼ N (0, Q) is the state transition noise as defined in [39], with covariance matrix T4d σv2x /4 T3d σv2x /2 0 0 3 2 Td σvx /2 T2d σv2x 0 0 . (23) Q= 4 2 3 2 0 0 Td σvy /4 Td σvy /2 3 2 2 2 0 0 Td σvy /2 T d σv y
T −1 Kj,k = Pj,k|k−1 CT , j,k (Cj,k Pj,k|k−1 Cj,k + U)
(29)
and the Jacobian matrix C of the measurement function is given by ∂ϕj,k ∂ϕj,k 0 0 ∂xj,k ∂yj,k 1 1 1 1 ∂fj,k ∂fj,k ∂f ∂f j,k j,k Cj,k = ∂x (30) y . x j,k ∂vj,k ∂yj,k ∂vj,k ∂f 2 2 2 2 j,k ∂fj,k ∂fj,k ∂fj,k y x ∂xj,k ∂vj,k ∂yj,k ∂vj,k
8
In the above Jacobian matrix, the partial derivatives are given by ∂ϕj,k −yj,k (31a) = 2 2 ∂xj,k xj,k + yj,k ∂ϕj,k xj,k (31b) = 2 2 ∂yj,k xj,k + yj,k m ∂fj,k 1 =− ∂xj,k λm
y y y x x (rm,j,k )2 vj,k − rm,j,k rm,j,k vj,k
(∥r m,j,k ∥)3 ! y 2 x yj,k vj,k − xj,k yj,k vj,k
+
m ∂fj,k 1 =− x ∂vj,k λm m ∂fj,k 1 =− ∂yj,k λm
x rm,j,k xj,k + ∥r m,j,k ∥ ∥pj,k ∥
where hj,k , k = 1, 2, · · · , Kj , is the k-th column of H(pj,1 ), and according to (2), ∂ϕ ∂ϕ j,k
y y x x x (rm,j,k )2 vj,k − rm,j,k rm,j,k vj,k
(∥r m,j,k ∥)3
+
(31e)
!
(∥pj,k ∥)3 y rm,j,k
yj,k + ∥r m,j,k ∥ ∥pj,k ∥
∂xj,k 1 ∂fj,k ∂hj,k ∂hj,k ∂pj,k = = ∂xj,k ∂pj,1 ∂pj,k ∂pj,1 2 ∂fj,k ∂xj,k
(31d)
y x x2j,k vj,k − xj,k yj,k vj,k
m ∂fj,k 1 y =− ∂vj,k λm
∂vec(Rj (pj,1 )) ∂vec(H(pj,1 )) =− (33) ∂pj,1 ∂pj,1 " T T #T ∂hj,Kj T ∂hj,2 ∂hj,1 , ,··· , , =− ∂pj,1 ∂pj,1 ∂pj,1
Jj =
(31c)
(∥pj,k ∥)3
Lemma 1 : Let vec(·) be the column-wise vectorization operator of a matrix, i.e. vec(Rj ) ∈ R3Kj ×1 . The Jacobian Jj ∈ R3Kj ×2 of vec(Rj ) can be written as
j,k
∂yj,k 1 ∂fj,k . ∂yj,k 2 ∂fj,k ∂yj,k
(34)
1 2 1 2 ∂fj,k ∂fj,k ∂fj,k ∂ϕj,k ∂ϕj,k ∂fj,k , , , , and are ∂xj,k ∂yj,k ∂xj,k ∂xj,k ∂yj,k ∂yj,k given by (31a), (31b), (31d) and (31f), respectively. Moreover, the gradient and Hessian matrix of gj (pj,1 ) are given by
Note that,
! ,
(31f)
where y x r m,j,k = [rm,j,k , rm,j,k ]T = pj,k − pTXm
∇gj (pj,1 ) =
T
= [xj,k − xTXm , yj,k − yTXm ] . The above state prediction (27) and correction (28) are conducted iteratively, which starts from k = 2 and update as k = k + 1 after each iteration. Hence, the overall trajectory can be reconstructed when k = Kj . The estimated velocities are expressed as V̂j = v̂ j,1 , v̂ j,2 , · · · , v̂ j,Kj , where v̂ j,k = v j,k|k , k = 1, 2, · · · , Kj . B. Solution Algorithm for Problem P4 In this part, given the shape of the trajectory V̂j , the transmitters’ locations M, and the measurement matrix Zj , the initial position of the sensing target pj,1 is obtained by solving the problem P4, where the Levenberg-Marquard algorithm is adopted. If no confusion arises, the outer and inner iteration indexes l and m in (19) are omitted in this part to simplify the notations. Specifically, let H(pj,1 ) ≜ H(Tj , M) = H([pj,1 , V̂j ], M) be the matrix of ground truth in (11) given the trajectory shape V̂j and transmitters’ locations M, Rj (pj,1 ) = Zj −H(pj,1 ) ∈ R3×Kj denote the residual matrix of the j-th trajectory, which is a matrix depending on the initial position pj,1 in the problem P4, gj (pj,1 ) = gj ([pj,1 , V̂j ], M, Zj ) be the objective function, the problem P4 is rewritten as 1 arg min gj (pj,1 ) = arg min tr{RT j (pj,1 )WRj (pj,1 )}. Kj pj,1 pj,1 (32) In order to adopt the LM algorithm, we first introduce the following conclusion on the Jacobian of the residual matrix and the Hessian of the objective function.
2 {Jj }T (W ⊗ IKj )vec(Rj (pj,1 )), Kj
(35)
2 T J (W ⊗ IKj )Jj , Kj j
(36)
and Bj ≈
where ⊗ denotes Kronecker product. Proof: The Jacobian in (34) can be derived directly from the definition, the gradient (35) and approximate Hessian (36) follow the conclusion in [40]. As a result, a suboptimal solution of the problem P4 can be obtained iteratively according to the LM algorithm. Since gj (pj,1 ) is not convex, the iteration may be trapped in not-sogood local optimums. A grid-based sampling of initial candidate solutions is combined with an iterative algorithm to achieve a high-quality suboptimal solution. Particularly, let Pj ⊆ R2 be the candidate region of the initial solution as ( ) |ϕ̃0 − ϕj,1 (pj,1 )| ≤ ∆ϕ Pj ≜ pj,1 , (37) |v j,1 | = ∥A−1 (p )f˜j,1 ∥ ≤ vmax f
j,1
where ∆ϕ and vmax denote the width of the narrow beam and the maximum velocity of the target, respectively. The initial n, 0 2,0 solutions, denoted as {p1,0 j,1 , pj,1 , · · · , pj,1 }, are randomly selected from Pj . Thus, the iterative search algorithm for the problem P4 can be written as a,b −1 pa,b+1 = pa,b ∇gj (pa,b j,1 j,1 − (Bj (pj,1 ) + µI) j,1 ),
a = 1, 2, · · · , n,
(38)
b = 0, 1, 2, · · · ,
where b is the iteration index and µ is the damping factor.
9
trajectory
Minimize the WMSE of measurements.
Section V
Measurements of Doppler & AoA
In -th outer iteration:
Section VI
trajectory Measurements of Doppler & AoA
Section VI-A:
Given the transmitters’ locations , optimize the trajectories.
In
Section VI
-th
inner iteration: Section VI-B
trajectory
Given the trajectories, optimize the transmitters’ locations
Measurements of Doppler & AoA
Given the initial position and measurements , the velocity vector can be estimated by the EKF, which determines the trajectory shape. Section VI
Given the shape of the trajectory by solving problem P4.
, the initial position
is derived
According to the geometric relation in (1), there is only one optimization variable in the problem P3, and the one-dimensional search can be applied to find the solution.
Low-Complexity Solution Algorithm
Passive Sensing
Fig. 3: Framework of the proposed low-complexity solution.
As a remark, after a number of iterations, some of the intermediate solutions {pa,b j,1 |∀a, b} might be trapped in notso-good local optimums. Discarding these solutions and the following iterations can suppress the computation complexity. Finally, let pa,∗ j,1 be the solution after the above iterative a,∗ algorithm, which is initiated from pa,0 j,1 , and P̃j = {pj,1 |∀a} be the set of the iteration results, the suboptimal solution of the problem P4(j, l, m) is given by p̂l,m j,1 = arg min gj (pj,1 ).
(39)
pj,1 ∈P̃j
In summary, the overall low-complexity solution algorithm for the SLAT problem is illustrated in the flowchart of Fig.3. VII. E XPERIMENT R ESULTS AND A NALYSIS The hardware architecture of the mmAlert system is depicted in Fig. 4. Each transmitter (transmitter 1 and transmitter 2) consists of an NI USRP-2954R software defined radio (SDR) [41] and a Sivers 60GHz phased array [42]. The bandwidth of each transmit signal is 1MHz, which consists of a training sequence and an orthogonal frequency division multiplexing (OFDM) modulated data payload. The baseband signal of the transmitter 1 is up-converted by the SDR to a central frequency of 500MHz (intermediate frequency, IF). Then, the IF signal is up-converted to 60.98GHz via the Sivers module. Similarly, the central frequency of the IF signal at the transmitter 2 is 503MHz, and the carrier frequency of the RF signal is 60.983GHz. The phased array has 16 transmit antenna elements and 16 receiver antenna elements, respectively, such that both the transmit and receive beams can be adjusted. The transmit beamwidths employed at both transmitters are 90o . The receiver consists of two NI USRP-2954R SDRs and three Sivers 60GHz phased arrays. This is because one SDR can at most connect with two phased arrays. The clocks of the two SDRs are synchronized by CDA-2990 octoclock to ensure
consistent carrier frequency offset (CFO). This is important as inconsistent CFOs at different receiver SDRs may introduce significant error in Doppler measurements. Similarly, the three Sivers share the same clock signal in a cascaded manner. At the receiver, two of the phased arrays direct their reference beams to the transmitter 1 and 2, and receive reference channel signals in two distinct frequency bands, respectively. The third phased array provides the surveillance beam, which receives signals of both frequency bands simultaneously. All the receive beamwidths are approximately 10o . The surveillance beam switches periodically among Q = 4 directions with Φ = {40.6o , 28.5o , 18.5o , 5.8o }. The surveillance beam stops at one direction with a duration of Tb = 50 ms, resulting in a complete sweep period of Td = QTb = 200 ms. This enables a velocity detection resolution of 0.025 m/s. The sampling frequency at the receiver is 5MHz, and all the sampled baseband signals are stored in a server for subsequent processing. The experiment is conducted in an indoor corridor scenario. The moving target can be a Turtlebot robot or a volunteer. The robot’s true motion trajectory is recorded by its odometer, while the volunteer’s motion trajectory is recorded by a ZED depth camera [43]. The timing of both the robot and the ZED camera can be synchronized with the receiver at millisecond level. Since the motion of different parts of the human body are different in walking, we extract the trajectory information of 21 key points of the human body from the depth image data with centimeter-level accuracy, and take the trajectory of shoulder, which is of the same height as the phased array antenna, as the real trajectory of the human body. In order to analyze the sensitivity of the localization and reconstruction algorithm versus the transmitters’ locations, the transmitter 1 is fixed at one position, and the transmitter 2 is placed at three different positions. 50 random trajectories are measured in each scene (each pair of transmitter 1 and 2’s positions). Thus, 150
10
trajectories are measured in total.
USRP- 2954R
Phased Array 500Mhz IF
Band 1 500Mhz IF
Ethernet
TX1
Host (TX1)
Reference1
USB Clock1
USRP- 2954R
Surveillance 500Mhz IF
Clock2 Ethernet
measured. The mean errors of bistatic Doppler frequency measurements for the surveillance channel 1 and 2 are 4.7Hz and 6.1Hz, respectively. The mean error of AoA measurements is 3.7o . Moreover, bistatic Doppler frequency measurements for the surveillance channel 1 are generally better than that of surveillance channel 2. This is mainly due to their different locations.
B. SLAT with Single trajectory In this part, a special case, where the mmAlert system works with a single trajectory (J = 1), is investigated. Particularly, Band 2 Ethernet Reference2 the mmAlert system measures single trajectory of the moving USB USRP- 2954R Host (TX2) Phased Array target, then estimates the locations of the two transmitters and RX TX2 USRP- 2954R the trajectory according to the measurements. The scenario is referred to as SLAT-S for the elaboration convenience. The Fig. 4: Block diagram of system implementation. localization and reconstruction results for two trajectories are illustrated in Fig. 8(a) and 8(b), where a baseline with perfect knowledge of the two transmitters’ locations and the initial A. Bistatic Doppler Frequency & AoA Estimation In this part, the measurements of bistatic Doppler frequency position of the trajectory is also plotted for comparison. It and AoA are demonstrated. In Fig. 6(a), the Doppler-time plots can be observed that when the above location knowledge is of the two surveillance channels for one example trajectory are perfectly known at the receiver, the trajectory reconstruction demonstrated, respectively, where the transmitter 2 is at TX2- matches the ground truth very well, where the minor deviations S1 as in Fig. 5(b). In this figure, the color intensity represents come from the Doppler and AoA measurement errors. On the the significance of the bistatic Doppler frequency component, other hand, without the above knowledge, our proposed SLAT which is calculated according to the CAF in (7). Thus, the algorithm can still localize the transmitters and reconstruct the CAF values of bistatic Doppler frequencies from -300kHz to trajectory. It can be observed that the localization errors of 300kHz versus time (sweeping period) are illustrated. The first both transmitters for two examples are 0.31 m, 0.17 m and and second rows of the plots correspond to the surveillance 0.35 m, 0.2 m, respectively. To evaluate the reconstruction channel 1 and 2, respectively. For each surveillance channel, accuracy, the following average Euclidean distance (AED) is the surveillance beam rotates among 4 different directions, adopted as the metric: Kj hence, the four plots in each row correspond to the four 1 X directions respectively. In each plot, the detected bistatic AED = ∥p̂j,k − pj,k ∥, (40) Kj Doppler frequencies versus time can be observed from the k=1 bright regions, and the red line is the ground truth of bistatic where ∥ · ∥ denotes the L norm, p̂ 2 j,k and pj,k are the Doppler frequency versus time. It can be observed that the reconstructed position and the ground truth at the beginning of moving target is captured by the surveillance beam of different the k-th sweeping period, respectively. For the 50 trajectories directions in different time periods, which matches the moving measured when the transmitter 2 is at location TX2-S1, the trajectory of the target. For example, in this figure, the target AED of the two examples are 0.22 m and 0.17 m, respectively. is captured by the surveillance beams of the four directions in the time intervals of [1s, 4.8s], [4.8s, 7.4s], [7.4s, 9s] and [9s, C. SLAT with Multiple Trajectories As discussed in Section V, if multiple trajectories are mea10s], respectively. However, there are still estimation errors in sured by the mmAlert system, the proposed SLAT algorithm the Doppler measurements. Therefore, a smoothing algorithm is used to suppress the is able to suppress the localization errors of both transmitters, noise as in Fig. 6(b) and 6(c), which show the Doppler detection and hence, suppress the errors of trajectory reconstruction. It and the smoothed results for the two surveillance channels, is calculated that when all the 50 trajectories (J = 50) are respectively. From these two figures, it can be seen that the jointly used for SLAT (namely SLAT-M), the estimation errors smoothed bistatic Doppler frequency is closer to the true value. for xTX1 , when the transmitter 2 is at TX2-S1, TX2-S2, and Due to the 10o beamwidth, the resolution of AoA is worse TX2-S3, are 0.07 m, 0.16 m and 0.31 m respectively. Note than that of bistatic Doppler frequency. Consequently, we apply that the locations of the two transmitters can be derived with the linear interpolation and smoothing algorithm on the AoA xTX1 . Meanwhile, the multi-tone ranging algorithm proposed measurements as illustrated in Fig. 6(d). It is demonstrated in [32] was employed for distance estimation. With a 1 MHz that the AoA after linear interpolation and smoothing closely signal bandwidth and a common clock for the transmitter and aligns with the ground truth. In the following joint detection, the receiver (This is hard to implement in real communication the bistatic Doppler frequencies and AoAs after smoothing are systems), this method yields a ranging error of up to 11.6 m, which is substantially larger than the result achieved by used in the measurement matrices defined in (10). Finally, the cumulative distribution functions (CDFs) of mmAlert. measurement errors for both bistatic Doppler frequency and In Fig. 9(a), the CDF of detection errors on xTX1 , when the AoA are shown in Fig. 7, where 150 trajectories were transmitter 2 is at location TX2-S1 and SLAT-S is adopted, is Phased Array
503Mhz IF
500Mhz IF
Host (RX)
CDA - 2990
11
RX
TX2
TX1
TX2- S1 TX2- S2 TX2- S3
ZED RX
3.45m
TX1 x Sensing region
Turtlebot/ Human
y (a)
Target (b)
Fig. 5: Illustration of the experiment scenario, where (a) shows a snapshot of the experiment scenario; (b) shows the top view. In (b), the gray area is the obstacle, e.g., the wall, the green area is the sensing region of the system, and the blue circles indicate the transmitter 2’s position in the three experiment scenes.
Impact of scanning beam number and beamwidth. We simulated the performance with beamwidths of 5◦ , 10◦ , and 15◦ , respectively. In order to cover the same sensing area, larger beamwidth leads to less sensing direcctions for the surveillance beam (smaller beam number Q) and shorter sweeping period Td . Hence, the corresponding beam numbers are 12, 6, and 4, respectively. The CDFs of trajectory reconstruction errors are shown in the Fig. 11(b). Although a 5◦ beamwidth yields a higher AoA estimation accuracy, the update frequency of AoA estimation becomes low, which leads to a significant drop in overall performance. The results for 10◦ and 15◦ beamwidths are comparable, indicating that appropriately widening the beam can improve estimation performance. Impact of transmitter location. In the experiments, the transmitter 2 is placed at the three positions, as shown in Fig. 5(b). The CDFs of trajectory reconstruction errors for the three locations with SLAT-M are illustrated in Fig. 12(a). It can be observed that the worst reconstruction errors for the three scenes are 0.3 m, 0.34 m, and 0.42 m respectively in 90% of the trajectories. Impact of sensing duration. We obtained observations of trajectories with different lengths by segmenting the original D. Analysis of Tracking Error trajectory data from the TX-S1 experimental scenario. The This part presents a sensitivity analysis of the proposed segment durations are 2 s, 4 s, 6 s, and 8 s, respectively. As algorithm through both simulations and experiments. The shown in the Fig. 12(b), when the observation duration is only factors examined include coherent integration time (CIT) N Ts , 2 s, the estimation accuracy degrades significantly due to the scanning beam number Q and beamwidth, transmitter location, limited number of AoA and Doppler observations. However, and sensing duration Kj Td . The corresponding results are as the observation duration increases, both localization and discussed as follows. trajectory tracking accuracy improve substantially. Moreover, Impact of CIT. The Doppler frequency resolution is in- when the observation duration reaches 6 s, the estimation versely proportional to the CIT. However, a longer CIT leads to accuracy becomes comparable to that obtained using the full longer sweeping period Td and less sweeping periods (smaller dataset. Kj for the j-th trajectory, ∀j). To investigate the impact of CIT Impact of trajectory shape. We compared the impact of on estimation accuracy, we simulated 50 trajectories, where the different trajectory shapes on system performance. As indicated CIT takes values of 25 ms, 50 ms, and 100 ms, respectively. by the red circles in Fig. 13, the two transmitters Tx1 and Tx2 The results, as shown in the Fig. 11(a), indicate that a CIT of are deployed along the X-axis and Y-axis, respectively. The 50 ms yields the best performance. estimation results for the triangular trajectory and the circular plotted. The green line refers to the value of 0.07 m, which is the detection error of SLAT-M. Moreover, Fig. 9(b) presents the CDFs of trajectory reconstruction errors for both SLAT-M and SLAT-S. Meanwhile, the yellow line illustrates the tracking result achieved with the EKF under the condition that the device location and the target’s initial position are perfectly known. Hence, the yellow line serves as the performance upper bound. The average trajectory reconstruction error using the SLAT-M algorithm is 0.2 m, meanwhile perfect device location knowledge (yellow line) leads to an error of 0.095 m. It can be observed that, with SLAT-M, the localization errors of both transmitters are almost negligible. This is significantly better than that of SLAT-S. Moreover, the trajectory diversity is also helpful in suppressing the trajectory reconstruction error. In fact, the transmitters’ localization benefits more from multiple trajectories. In Fig. 10, one reconstructed trajectory is illustrated with the ground truth, where the result of SLAT-S is also plotted for comparison. It can be observed that the reconstructed trajectory via the SLAT-M is much closer to the ground truth than that of SLAT-S.
12
TX1: Beam 1
Doppler frequency (Hz)
300
80
100 0 -100
40
-200
300
Doppler frequency (Hz)
5
Time (s)
10
TX2: Beam 1
80
100 0 -100
40
-200 0
5
Time (s)
80
100 0
60
-100
40
-200
20 -300
10
0
TX2: Beam 2
5
10
Time (s)
TX1: Beam 4
100 300
200
60
20 -300
TX1: Beam 3
100 300
200
60
-300 0
TX1: Beam 2
100 300
200
TX2: Beam 3
100
200
80
100 0
60
-100
40
-200
20 -300
0
5
10
Time (s) TX2: Beam 4
20
70
300
70
300
70
300
200
60
200
60
200
60
200
60
100
50
100
50
100
50
100
50
0
0
40
-100
30
-200 -300 0
20
5
Time (s)
10
0
40
-100
30
-200
20
-300 0
5
Time (s)
10
40
-100
30
-200
20
-300 0
5
10
Time (s)
70
0
40
-100
30
-200
(a)
20
-300 0
5
10
Time (s)
(a) Doppler-time spectrum of the surveillance channel 1 and 2 at the four surveillance beam directions. 100
Doppler (Hz)
Doppler (Hz)
100
50
0
Raw Doppler Ground truth Doppler Smoothed Doppler
-50 0
2
4
6
8
Time (s)
50
0
-50 0
10
Raw Doppler Ground truth Doppler Smoothed Doppler
2
4
6
Time (s)
8
10
(b)
(b) Bistatic Doppler frequencies of (c) Bistatic Doppler frequencies of surveillance channel 1 surveillance channel 2
Fig. 8: Two examples of SLAT with single trajectory.
45 1
1
0.8
0.8
0.6
0.6
15 0
0
Raw AoA Ground truth Smoothed AoA
2
4
6 Time (s)
8
10
CDF
30
CDF
AoA (deg)
60
0.4
0.4
0.2
(d) AoA
0 0
Fig. 6: Illustration of Doppler and AoA measurements of one trajectory, where (a) shows the raw Doppler-time plots of surveillance channel 1 and 2; (b) compares the detected bistatic Doppler frequencies before and after smoothing with the ground truth; (c) compares the detected AoAs with the ground truth.
0.25
0.5
0.75
1
1.25
SLAT-S SLAT-M EKF
0.2
SLAT-S SLAT-M
0
1.5
0
0.1
Error of Tx1-Rx distance (m)
0.2
0.3
0.4
0.5
0.6
0.7
0.8
Trajectory reconstruction error (m)
(a)
(b)
Fig. 9: Comparison of error performance between SLAT-S and SLAT-M.
3
0.8
0.8
0.6
0.6
0
4
6
8
0
0.4
-1
Surveillance 1 Surveillance 2
0.2
-2
10
0
0.4 0.2
1
Y (m)
1
CDF
CDF
2 1
Doppler error (Hz)
(a) Bistatic Doppler frequency
0
2
4
6
AoA error (deg)
(b) AoA
Fig. 7: CDFs of measurement errors.
8
10
RX
TX1 Ground truth SLAT-S SLAT-M
TX2
-3 0
1
2
3
4
5
X (m)
Fig. 10: Reconstruction of one trajectory with SLAT-S and SLAT-M.
1
1
0.8
0.8
0.6
0.6
CDF
CDF
13
0.4
to the least diversity in the measurement angle. Hence, it has the worst localization and reconstruction errors. VIII. C ONCLUSIONS
0.4 CIT 25 ms CIT 50 ms CIT 100 ms
0.2
beamwidth 5° beamwidth 10° beamwidth 15°
0.2
0
0 0
0.2
0.4
0.6
0.8
0
0.5
Trajectory error (m)
1
1.5
Trajectory error (m)
(a)
(b)
1
1
0.8
0.8
0.6
0.6
CDF
CDF
Fig. 11: Simulation comparison of trajectory reconstruction performance with respect to (a) CIT, and (b) beamwidth.
0.4
0.4
TX2 S1 TX2 S2 TX2 S3
0.2
duration 2 s duration 4 s duration 6 s duration 8 s
0.2
0
0
0.1
0.2
0.3
0.4
0.5
0.6
0
0.5
1
1.5
2
Trajectory error (m)
Trajectory reconstruction error (m)
(a)
(b)
Fig. 12: Experimental comparison of trajectory reconstruction performance with respect to (a) three TX2 positions and (b) sensing duration.
trajectory are shown by the green lines in Fig. 13. Although both trajectories can be successfully reconstructed, the average trajectory estimation errors are 0.15 m and 0.32 m, respectively. This is because the EKF state transition model assumes uniform linear motion of the target between two adjacent sweeping periods. In other words, the triangular trajectory fits the state transition model better than the circular trajectory. 4 3
4 3
TX2
1 0
RX
0
1
2
3
X (m)
(a) triangular
4
1 0
TX1 Ground truth Initial position known Proposed
-1 -2 -1
TX2
2
Y (m)
Y (m)
2
5
RX
TX1 Ground truth Initial position known Proposed
-1 -2 -1
0
1
2
3
4
5
X (m)
(b) circular
Fig. 13: Comparison of estimation results for two different trajectory types. Moreover, as demonstrated in Table I, the average trajectory reconstruction error and average transmitter localization error are presented for the three scenes. The SLAT-M keeps the average trajectory reconstruction error below 0.25 m in all three scenes. The transmitter localization errors with SLAT-M are 0.07 m, 0.16 m, and 0.31 m in the three scenes, respectively. Thus, the accuracy of transmitters’ localization depends on the angle between the two transmitters and the target. For example, the position of TX2-S3 is the closest to transmitter 1, leading
In this paper, we propose mmAlert, to address the simultaneous localization and tracking problem in mmWave communication systems. The mmAlert is implemented in an uplink system with at least two transmitters and one receiver. Without prior knowledge on the transmitters’ locations, the proposed method could reconstruct multiple trajectories of a moving target via the passive sensing of the AoAs and bistatic Doppler frequencies, and meanwhile, estimate the locations of the two transmitters. This is because the AoAs and bistatic Doppler frequencies of the moving target depend strongly on the locations of the transmitters. In mmAlert, a low-complexity algorithm is proposed to handle the above joint estimation, which integrates the techniques of alternating optimization and extended Kalman filter. The experimental results demonstrate that mmAlert can achieve a transmitter positioning accuracy of 0.07 m with an average trajectory reconstruction error of 0.2 m when 50 trajectories are utilized. Moreover, the accuracy will decay if only one trajectory is considered in the above joint estimation. This justifies the performance gain due to trajectory diversity. Finally, the proposed method is not limited in the mmWave band, its extension to the frequency bands below 7GHz (sub-7GHz) would be investigated in the future. Moreover, mmAlert can distinguish multiple moving targets by exploiting differences in the Doppler-AoA plane. Multi-target tracking therefore constitutes another promising direction for future research. R EFERENCES [1] X. Wang, L. Kong, F. Kong, F. Qiu, M. Xia, S. Arnon, and G. Chen, “Millimeter wave communication: A Comprehensive Survey,” IEEE Communications Surveys & Tutorials, vol. 20, no. 3, pp. 1616–1653, 2018. [2] J. Li, Y. Niu, H. Wu, B. Ai, S. Chen, Z. Feng, Z. Zhong, and N. Wang, “Mobility support for millimeter wave communications: Opportunities and Challenges,” IEEE Communications Surveys & Tutorials, vol. 24, no. 3, pp. 1816–1842, 2022. [3] Y. Niu, Y. Li, D. Jin, L. Su, and A. V. Vasilakos, “A survey of millimeter wave communications (mmwave) for 5g: opportunities and challenges,” Wireless networks, vol. 21, pp. 2657–2676, 2015. [4] D. Kumar, J. Kaleva, and A. Tölli, “Blockage-aware reliable mmwave access via coordinated multi-point connectivity,” IEEE Transactions on Wireless Communications, vol. 20, no. 7, pp. 4238–4252, 2021. [5] J. Tan, T. H. Luan, W. Guan, Y. Wang, H. Peng, Y. Zhang, D. Zhao, and N. Lu, “Beam alignment in mmwave v2x communications: A survey,” IEEE Communications Surveys & Tutorials, vol. 26, no. 3, pp. 1676–1709, 2024. [6] Z. Gao, Z. Wan, D. Zheng, S. Tan, C. Masouros, D. W. K. Ng, and S. Chen, “Integrated sensing and communication with mmwave massive mimo: A compressed sampling perspective,” IEEE Transactions on Wireless Communications, vol. 22, no. 3, pp. 1745–1762, 2023. [7] Q. Xue, C. Ji, S. Ma, J. Guo, Y. Xu, Q. Chen, and W. Zhang, “A survey of beam management for mmwave and thz communications towards 6g,” IEEE Communications Surveys & Tutorials, vol. 26, no. 3, pp. 1520–1559, 2024. [8] F. Sohrabi, T. Jiang, W. Cui, and W. Yu, “Active sensing for communications by learning,” IEEE Journal on Selected Areas in Communications, vol. 40, no. 6, pp. 1780–1794, 2022. [9] Z. Wang, J. A. Zhang, M. Xu, and Y. J. Guo, “Single-target real-time passive wifi tracking,” IEEE Transactions on Mobile Computing, vol. 22, no. 6, pp. 3724–3742, 2023. [10] K. Niu, X. Wang, F. Zhang, R. Zheng, Z. Yao, and D. Zhang, “Rethinking doppler effect for accurate velocity estimation with commodity wifi
14
TABLE I: Average error of trajectory reconstruction and TXs’ localization. TX2-S1
Scenario
TX2-S2
TX2-S3
Method
Trajectories (m)
Transmitters (m)
Trajectories (m)
Transmitters (m)
Trajectories (m)
Transmitters (m)
SLAT-S
0.29
0.76
0.34
0.87
0.37
0.95
SLAT-M
0.2
0.07
0.22
0.16
0.24
0.31
devices,” IEEE Journal on Selected Areas in Communications, vol. 40, no. 7, pp. 2164–2178, 2022. [11] H. Li, X. Chen, J. Wang, D. Wu, and X. Liu, “Dafi: Wifi-based devicefree indoor localization via domain adaptation,” Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, vol. 5, no. 4, pp. 1–21, 2021. [12] X. Zhang, Y. Zhang, G. Liu, and T. Jiang, “Autoloc: Toward ubiquitous aoa-based indoor localization using commodity wifi,” IEEE Transactions on Vehicular Technology, vol. 72, no. 6, pp. 8049–8060, 2023. [13] M. T. Hoang, B. Yuen, X. Dong, T. Lu, R. Westendorp, and K. Reddy, “Recurrent neural networks for accurate rssi indoor localization,” IEEE Internet of Things Journal, vol. 6, no. 6, pp. 10 639–10 651, 2019. [14] X. Wang, L. Gao, S. Mao, and S. Pandey, “Deepfi: Deep learning for indoor fingerprinting using channel state information,” in 2015 IEEE Wireless Communications and Networking Conference (WCNC), 2015, pp. 1666–1671. [15] X. Zhu, T. Qiu, W. Qu, X. Zhou, M. Atiquzzaman, and D. O. Wu, “BLS-Location: A wireless fingerprint localization algorithm based on broad learning,” IEEE Transactions on Mobile Computing, vol. 22, no. 1, pp. 115–128, 2023. [16] J. Xiong and K. Jamieson, “ArrayTrack: A Fine-Grained indoor location system,” in 10th USENIX Symposium on Networked Systems Design and Implementation (NSDI 13), 2013, pp. 71–84. [17] S. Yang, D. Zhang, R. Song, P. Yin, and Y. Chen, “Multiple WiFi access points co-localization through joint AoA estimation,” IEEE Transactions on Mobile Computing, vol. 23, no. 2, pp. 1488–1502, 2024. [18] M. Kotaru, K. Joshi, D. Bharadia, and S. Katti, “Spotfi: Decimeter level localization using WiFi,” ser. SIGCOMM ’15. New York, NY, USA: Association for Computing Machinery, 2015, p. 269–282. [19] C. Ma, B. Wu, S. Poslad, and D. R. Selviah, “Wi-Fi RTT ranging performance characterization and positioning system design,” IEEE Transactions on Mobile Computing, vol. 21, no. 2, pp. 740–756, 2022. [20] D. Vasisht, S. Kumar, and D. Katabi, “Decimeter-Level localization with a single WiFi access point,” in 13th USENIX symposium on networked systems design and implementation (NSDI 16), 2016, pp. 165–178. [21] E. Soltanaghaei, A. Kalyanaraman, and K. Whitehouse, “Multipath Triangulation: Decimeter-level WiFi Localization and Orientation with a Single Unaided Receiver,” ser. MobiSys ’18. New York, NY, USA: Association for Computing Machinery, 2018, p. 376–388. [22] K. Qian, C. Wu, Z. Yang, Y. Liu, and K. Jamieson, “Widar: Decimeterlevel passive tracking via velocity monitoring with commodity Wi-Fi,” in Proceedings of the 18th ACM international symposium on mobile ad hoc networking and computing, 2017, pp. 1–10. [23] X. Li, D. Zhang, Q. Lv, J. Xiong, S. Li, Y. Zhang, and H. Mei, “IndoTrack: Device-free indoor human tracking with commodity WiFi,” Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, vol. 1, no. 3, pp. 1–22, 2017. [24] Z. Wang, J. A. Zhang, M. Xu, and Y. J. Guo, “Single-target real-time passive WiFi tracking,” IEEE Transactions on Mobile Computing, vol. 22, no. 6, pp. 3724–3742, 2023. [25] D. Wu, Y. Zeng, R. Gao, S. Li, Y. Li, R. C. Shah, H. Lu, and D. Zhang, “WiTraj: Robust indoor motion tracking with WiFi signals,” IEEE Transactions on Mobile Computing, vol. 22, no. 5, pp. 3062–3078, 2023. [26] F. Shi, W. Li, C. Tang, Y. Fang, P. V. Brennan, and K. Chetty, “MLTrack: Passive human tracking using WiFi Multi-Link Round-Trip CSI and particle filter,” IEEE Transactions on Mobile Computing, vol. 24, no. 6, pp. 5155–5172, 2025. [27] B. Tan, K. Woodbridge, and K. Chetty, “Awireless passive radar system for real-time through-wall movement detection,” IEEE Transactions on Aerospace and Electronic Systems, vol. 52, no. 5, pp. 2596–2603, 2016. [28] C. Yu, Y. Luo, R. Chen, and R. Wang, “Passive handwriting tracking via weak mmwave communication signals,” IEEE wireless communications letters, vol. 13, no. 3, pp. 874–878, 2024. [29] Y. Sun, C. Yu, Y. Luo, T. Xiao Han, H. Tan, R. Wang, and F. C. M. Lau, “An experimental study of passive UAV tracking with digital arrays and
cellular downlink signals,” IEEE Open Journal of the Communications Society, vol. 6, pp. 3779–3794, 2025. [30] C. Yu, B. Lv, H. Qiu, and R. Wang, “Trajectory tracking for mmwave communication systems via cooperative passive sensing,” ZTE Communications, vol. 22, no. 3, p. 29–36, 2024. [31] A. F. Molisch, V. V. Ratnam, S. Han, Z. Li, S. L. H. Nguyen, L. Li, and K. Haneda, “Hybrid beamforming for massive mimo: A survey,” IEEE Communications Magazine, vol. 55, no. 9, pp. 134–141, 2017. [32] T. Wei, A. Zhou, and X. Zhang, “Facilitating robust 60 GHz network deployment by sensing ambient reflectors,” in 14th USENIX Symposium on Networked Systems Design and Implementation (NSDI 17). Boston, MA: USENIX Association, Mar. 2017, pp. 213–226. [33] Y. Sun, J. Li, T. Zhang, R. Wang, X. Peng, X. Han, and H. Tan, “An indoor environment sensing and localization system via mmwave phased array,” Journal of Communications and Information Networks, vol. 7, no. 4, pp. 383–393, 2022. [34] D. K. Tan, H. Sun, Y. Lu, M. Lesturgie, and H. L. Chan, “Passive radar using global system for mobile communication signal: theory, implementation and measurements,” IEE Proceedings-Radar, Sonar and Navigation, vol. 152, no. 3, pp. 116–123, 2005. [35] S. S. Blackman and R. J. Popoli, Design and Analysis of Modern Tracking Systems. Norwood, MA, USA: Artech House, 1999. [36] P. E. Gill and W. Murray, “Algorithms for the solution of the nonlinear least-squares problem,” SIAM Journal on Numerical Analysis, vol. 15, no. 5, pp. 977–992, 1978. [37] K. Levenberg, “A method for the solution of certain non-linear problems in least squares,” Quarterly of applied mathematics, vol. 2, no. 2, pp. 164–168, 1944. [38] D. W. Marquardt, “An algorithm for least-squares estimation of nonlinear parameters,” Journal of the society for Industrial and Applied Mathematics, vol. 11, no. 2, pp. 431–441, 1963. [39] X. R. Li and V. P. Jilkov, “Survey of maneuvering target tracking: dynamic models,” in Signal and Data Processing of Small Targets 2000, O. E. Drummond, Ed., vol. 4048, International Society for Optics and Photonics. SPIE, 2000, pp. 212 – 235. [40] H. P. Gavin, “The levenberg-marquardt algorithm for nonlinear least squares curve-fitting problems,” Department of Civil and Environmental Engineering Duke University August, vol. 3, pp. 1–23, 2019. [41] National Instruments, “USRP - 2954,” https://www.ni.com/zh-cn/shop/ model/usrp-2954.html, accessed: [2025-5-9]. [42] Sivers Semiconductors, “Evaluation Kits (EVK) and Evaluation Boards (EVB),” https://www.sivers-semiconductors.com/ 5g-millimeter-wave-mmwave-and-satcom/wireless-products/ evaluation-kits/, accessed: [2025-5-9]. [43] Stereolabs Docs: API Reference, Tutorials, and Integration. [Online]. Available: https://www.stereolabs.com/docs