4D Point Splatting for mmWave Radar Micro-Doppler Differentiable Rendering

Preprint

4D P OINT S PLATTING FOR MM WAVE R ADAR M ICRO D OPPLER D IFFERENTIABLE R ENDERING Adnan Armouti Cornell Tech New York, NY, USA aa2546@cornell.edu

Guanyue Qian New York University New York, NY, USA gq2032@nyu.edu

Rajalakshmi Nandakumar Cornell Tech New York, NY, USA rn283@cornell.edu

arXiv:2611.08342v1 [cs.CV] 5 Nov 2026

A BSTRACT Millimeter-wave radar resolves the micrometer displacements of a moving body as micro-Doppler, enabling gait, gesture, activity, pose and vital-sign sensing. Each task relies on a downstream model trained on real data, yet radar datasets lag orders of magnitude behind vision and language. Brute-force collection will not close that gap, and existing synthesis methods (forward-only simulators, learned generators and static-scene inverse renderers) cannot fit a real capture of a moving body, leaving a substantial sim2real gap. We present 4D Point Splatting (4DPS), a physics-based differentiable renderer that fits an explicit moving scene to a real capture. Using the capture’s synchronized camera to initialize the room and the moving body, we represent the scene as oriented points carrying material, normal and velocity, and splat each point into range and Doppler bins. Our output is the raw complex return a real radar records, so we can apply the same digital signal processing (DSP) on the rendered data, and obtain any DSP representation to run on and train any SOTA downstream model. We fit 235 sequences on HuPR and 100 on RT-Pose, two datasets with different radars. Our reconstructions set the state of the art, with median correlations of 0.940 and 0.830 on range–azimuth, 0.829 and 0.695 on range–Doppler, and 0.988 and 0.966 on micro-Doppler. Our renders also set the state of the art in downstream sim2real gap. On RT-Pose, a pose model trained on them alone reaches 21.9 cm MPJPE on real data, against 16.7 cm for real training and 127.6 cm for the RF-Genesis baseline, and we show that both the rendered- and real-trained models learn nearly the same features. Finally, our custom CUDA kernels fit a sequence in 5.7 minutes on HuPR and 12.9 on RT-Pose on one H100, making dataset-scale inverse rendering of mmWave radar possible for the first time. Together, this work can enable synthetic data generation for any human-centered downstream task and model, and scene editing for data augmentation.

1

I NTRODUCTION

Millimeter-wave (mmWave) radar resolves radial motion to a fraction of a millimeter, works in darkness and through occlusion, and captures no identifiable imagery (Zhao et al., 2018; Guan et al., 2020; Adib et al., 2015). This makes it a natural sensor for privacy-preserving human sensing, including gait, gesture, activity, fall and pose recognition (Lien et al., 2016; Singh et al., 2019; Zhao et al., 2018; Lee et al., 2023). Each of these applications requires a downstream model trained on real micro-Doppler (Chen et al., 2006), the phase modulation a moving body imposes on the return. However, each is bottlenecked by the same data scarcity, because public radar data lags orders of magnitude behind vision and language corpora. Brute-force collection cannot close that gap: radar sensors are rarer than cameras, and where a camera is point-and-shoot, every radar capture needs a rig calibrated and synchronized to reference cameras or LiDAR. Synthesis is the next best alternative, but every existing method leaves a sim2real gap, because none fit to a real capture of a moving body. Motion-driven simulators (Sundar Ram & Ling, 2008; Vishwakarma et al., 2022; Chen & Zhang, 2024) carry geometry, materials and motion explicitly but run forward only, so their parameters are set by an operator and never optimized against a real capture. Learned generators such as RF-Diffusion (Chi et al., 2024) are trained on the same scarce 1

Preprint

Input

4DPS inverse rendering

Any radar product

camera-initialized scene + real radar

fit normals, materials and velocity

synthetic data for any model

fit

render ADC

CRP

RA

RD

RAE

µD

velocity rendered RAED

real RAED scene geometry

normals

materials

Applications: Synthetic Data Generation to Train Downstream Models for Any Task Scene editing

✓ position ✓ motion ✓ timing

Pose estimation

Vital signs

Activity recognition

RR

jumping jacks

HR RAED

ADC

µD

Fall detection

fall detected RD

Figure 1: 4DPS fits a moving scene to a real radar capture. Top: from a camera-initialized scene of oriented points, 4DPS fits materials, normals and velocities (color: radial velocity) until the rendered range–azimuth–elevation–Doppler tensor matches the recording. Bottom: the fitted scene can be edited (App. G) and rendered as training data for pose, vital-sign, activity and fall models.

data and keep the scene implicit in network weights, so there is nothing to fit or edit. Radar inverse renderers do fit real captures; mmIR (Armouti et al., 2026b), 3DPS (Armouti et al., 2026a) and differentiable ray tracers (Hoydis et al., 2023; Hofmann et al., 2025; Chen et al., 2025; 2023) recover explicit materials and normals and render range, azimuth and elevation. However, all currently assume a static scene and evaluate one phase per point per frame so they have no slow-time axis (the chirp-to-chirp axis along which Doppler is measured) and nothing along which velocity could be defined. What is missing is a renderer that fits an explicit moving scene to a real capture. We present 4D Point Splatting (4DPS), a physics-based differentiable renderer that fits an explicit moving scene to a real capture, with velocity as an estimated and differentiable parameter. Like prior radar renderers (Armouti et al., 2026b;a; Chen et al., 2026), we initialize geometry from another sensor, here the capture’s synchronized camera. We represent the scene as oriented points, each carrying an ITU-R P.2040 material (ITU-R, 2023), a surface normal and, for the body, a velocity, and we splat each point into range and Doppler bins. The camera fixes where each surface is present but not how strongly it reflects at millimeter wavelengths. That strength depends on the surface’s material and on the angle between its normal and the radar, and it sets the amplitude of every return. We therefore fit the materials, normals and velocities by gradient descent until the rendered return matches the recording (Fig. 1, top). On HuPR, fitting raises the clutter-removed range–azimuth correlation from 0.270 for the camera-initialized render to 0.666. Our output is the raw complex return that a real radar records, for every chirp, antenna pair and range bin. We therefore apply the same digital signal processing (DSP) to rendered and real data, and obtain any representation, from range profiles to micro-Doppler, for any downstream model (Fig. 1, bottom). We validate each representation on two datasets with different array geometries and downstream models. Introducing an explicit velocity into the forward model raises three challenges. The first challenge is the cost: existing differentiable radar renderers output one chirp per frame, so rendering Doppler would repeat that render for every chirp, a cost that grows linearly with the chirp count and rules out corpus-scale fitting. 4DPS instead splats each point into Doppler bins as well as range bins (Fig. 2). A moving point’s return across a frame’s chirps is a single tone, so its Doppler point spread function has a closed form. Each point therefore deposits into seven Doppler bins instead of being rendered 64 times. Second, ambiguity: velocity enters the forward model only through a phase term that wraps every 2π, so velocities one Doppler ambiguity apart (9 m/s on HuPR) render identically, and gradient descent from an arbitrary start lands in the nearest minimum rather than the true minimum. To address this, we initialize the joint velocities from the video’s kinematics and 2

Preprint

Table 1: Comparison of radar rendering and synthesis methods. Signal: CIR or raw ADC. Mat.: explicit materials; Vel.: velocity estimated; Doppler: from subject motion; Real: validated on real captures; Edit: re-renderable scene; Code: released. Radar ports of neural fields and Gaussian splatting are omitted, because they supervise post-FFT power of a static scene. Method

Repr. Signal Complex Mat. Vel. Doppler Real Edit Code

Differentiable radar inverse rendering mmIR (Armouti et al., 2026b) Mesh 3DPS (Armouti et al., 2026a) Points

ADC ADC

✓ ✓

✓ ✓

– –

– –

✓ ✓

✓ ✓

✓ ✓

Human micro-Doppler simulation and synthesis RF-Genesis (Chen & Zhang, 2024) Mesh ADC HybridSim (Xiong et al., 2026) Mesh ADC

✓ ✓

– ✓

– –

✓ ✓

✓ ✓

– ✓

✓ –

4DPS (Ours)

✓

✓

✓

✓

✓

✓

✓

Points

ADC

learn only a residual, bounded by each joint’s own speed. Third, supervision: one loss must train both the static scene and the person, who holds a small fraction of a frame’s energy. A loss over the whole frame lets a large relative error on the person hide behind the scene’s return, and a loss on the person alone gives the scene no gradient. We therefore compare the range–azimuth–elevation magnitude at two slow-time views: each frame’s first chirp, which carries the static scene, and its Doppler planes, which carry the motion. Each view is compared over the whole frame and over a window on the person (Eq. 4). Solving these three problems lets us fit entire corpora for the first time, with no manual input. We fit every usable sequence of both datasets, 235 on HuPR and 100 on RT-Pose, at about 5.7 and 12.9 minutes a sequence on one H100. We set the new SOTA in 4D range–azimuth–elevation–Doppler (RAED) tensor rendering and reconstruction, with median Pearson correlations of 0.940 on range–azimuth and 0.988 on micro-Doppler on HuPR, and 0.830 and 0.966 on RT-Pose (Table 2). To test whether a model can learn from these renders, we train each dataset’s official pose model on real or on rendered data, four seeds each, and test it on both (Tables 3 and 11). Trained on our renders alone, the models keep 0.59 of their realtrained accuracy on HuPR and 0.76 on RT-Pose. By contrast, models trained on RF-Genesis renders of the same motion fail on real data, at 1.8 AP on HuPR and 127.6 cm MPJPE on RT-Pose. Because our rendered-trained models score far higher on renders than on recordings, the gap that remains is realism rather than missing information. A comparison of the two models’ internal representations shows that they compute the same features from the backbone onward (Sec. 4.3). Contributions. In summary, we make the following contributions: 1) 4DPS, to our knowledge the first differentiable renderer of human micro-Doppler: oriented points carrying material, normal and a velocity estimated per joint for the first time, splatted in closed form into range and Doppler bins, rendering the raw complex return so any representation follows; 2) a range–Doppler point splatting framework, with a closed-form Doppler point spread function and Doppler-resolved supervision, that makes corpus-scale inverse rendering tractable across multiple public corpora and different array geometries for the first time; 3) the first sim2real analysis of radar inverse rendering on downstream models: the real/rendered train–test matrix for radar human pose on two datasets with the official models, and an interpretability analysis showing that rendered- and real-trained models learn the same features; and 4) release of the renderer, the fitting chain, every fitted scene and rendered cube, the mesh-derived RT-Pose labels, the retrained models and the evaluation suite, on acceptance.

2

R ELATED W ORK

Differentiable radar rendering. Radar-domain renderers fit explicit scene parameters to captures. Differentiable ray tracers (Hoydis et al., 2023; Hofmann et al., 2025; Chen et al., 2025; 2023) do so on synthetic or known-geometry scenes. mmIR (Armouti et al., 2026b) fits the raw ADC of real captures and recovers materials, normals and antenna patterns, and 3DPS (Armouti et al., 2026a) replaces mmIR’s Monte Carlo sampling with closed-form point splatting. Every one of these renderers reconstructs a static scene. Its forward model evaluates one phase per primitive per frame, so it has no slow time, the axis along which Doppler and velocity are defined. 4DPS renders that axis in closed form and estimates velocity, so it fits a moving scene (Table 1). 3

Preprint

ºtr; i

ºtr; i = Nl Nslots Tc R_ tr; i =¸

d

Range splat ! range k

Rendered RD

Real RD

(t; r) Doppler d

Doppler splat ! Doppler d

ktr; i

range k k

ktr; i = (Rt; i + Rr; i )=2¢R

Array

Amplitude ! jRDj p

Pose

Phase ! RD per (t; r)

Str (x i ) /

Gt Gr ¢­ i fi cos µt; i Rt; i

e j2¼(Rt; i + Rr; i )=¸

Rendered RAED

Real RAED

(e; d)

L

azimuth

Antenna patterns

range

Figure 2: 4DPS overview. Camera-initialized oriented point sets for a static scene and a moving person carry normals, materials and, for the person, velocities. For every antenna pair, 4DPS splats each point into its Doppler and range bins with an amplitude and a carrier phase, forming the RAED tensor that L compares with the recording. Green: forward pass; red: gradients; gray: fixed inputs. Radar data synthesis. Radar ports of neural fields and Gaussian splatting (Huang et al., 2024; Borts et al., 2024; Kung et al., 2025; Zhang et al., 2026; Rafidashti et al., 2025) supervise on post-FFT power and discard phase, so they cannot render the phase that encodes micro-Doppler. DART (Huang et al., 2024) is the only one with a Doppler axis, and its Doppler comes from egomotion, not from an articulated subject. Motion-driven simulators (Sundar Ram & Ling, 2008; Vishwakarma et al., 2022; Chen & Zhang, 2024) produce human micro-Doppler but run forward only, so their residual against a real capture is either never quantified or never minimized. mmGPE (Xue et al., 2023) narrows that residual with a learned refiner on the simulator’s output, without fitting the scene. Learned generators (Chi et al., 2024) leave the scene implicit in network weights, where nothing can be fitted or edited. Concurrent with this work, HybridSim (Xiong et al., 2026) ray traces a physics-learning hybrid over eight sequences from one single-chip radar, trained end to end with the downstream task it feeds. Solving both at once is a weaker requirement than ours, because a renderer supervised by the task’s loss need only produce features that task finds useful. We instead fit the radar return itself and score it against the recording, so our renders must stand on their own and then transfer to a downstream model that never saw them during fitting. We do not carry it as a baseline, because it releases no code and its captures, collected for mmMesh (Xue et al., 2021), are unreleased. 4DPS fits an explicit scene, velocity included, to a real capture.

3

M ETHOD

4DPS starts from two camera-initialized point sets, a static scene and a moving person (Fig. 2; RTPose’s scene also uses its LiDAR). Each scene point carries its own ITU-R P.2040 material and normal. Each person point carries its own normal, the whole person shares one material, and its points move with 14 joint velocities skinned onto them. 4DPS renders the complex return of a TDM MIMO FMCW radar, for every chirp, transmitter–receiver pair and range bin, by splatting both point sets in custom CUDA kernels. Static points are splatted along range only, and moving points along both range and Doppler. Sec. 3.1 states the differentiable radar rendering model, from which Sec. 3.2 derives the splatting and Sec. 3.3 the objective and its learned parameters. 3.1

R ADAR F ORWARD M ODEL

Consider an FMCW radar with NTX time-division-multiplexed transmitters and NRX receivers, carrier wavelength λ, range bin width ∆R, K = 256 range bins and Nl = 64 loops per frame. In each loop every transmitter fires one chirp in its own slot, so with Nslots slots and a chirp interval Tc the loop period is Nslots Tc (216 µs on HuPR, 780 µs on the cascade). The rendering equation. Following prior radar inverse renderers (Armouti et al., 2026b;a), we take the bistatic radar equation (Skolnik et al., 1980), replace the cross-section by a spatially varying BSDF f and integrate over the receiver-centered hemisphere Ωrx , giving the received power as a 4

Preprint

rendering equation, Pr =

Pt λ 2 (4π)2

Z

Gt (ω t ) Gr (ω r ) f (ω t , x, ω r ) cos θt V (x) dωr , Rt2 (x) Ωrx

(1)

with Gt , Gr the antenna gains, Rt the path length to the transmitter, cos θt = |n·ω t | and V ∈ {0, 1} visibility. We discretize the hemisphere on an oriented point set {(xi , ni , ∆Ωi , θ i , vi )}N i=1 , each point subtending solid angle ∆Ωi at the receiver and carrying a material, with a velocity vi on person points only. The Riemann sum cancels the surface Jacobian against ∆Ωi and leaves a sum of per-point amplitudes, carrier phase attached, p p Gt (ω t,i ) q Str (xi ) = C Gr (ω r,i ) ∆Ωi fi cos θt,i e j2π(Rt,i +Rr,i )/λ , (2) Rt,i with C a constant that the learned gain absorbs (App. A.1). We set ∆Ωi = 1 for every point, because a cosine solid-angle weight darkened the limbs’ oblique sides, and the learned gains and materials carry the level. Nothing is sampled. Each point contributes one path per transmitter–receiver pair. 3.2

S PLATTING S TATIC AND M OVING P OINTS

Static scene: range splat. A renderer of one chirp can push the range FFT inside the point sum, so that each point becomes the Hann window’s known response at its fractional range bin and the range profile is splatted instead of simulated (Armouti et al., 2026a), and we use it for the static scene. A static point returns the same sample in every chirp, and the real front end removes that slow-time mean before its Doppler transform, so the static return lies in no Doppler plane and is supervised through each frame’s first chirp (Sec. 3.3). Moving person: range–Doppler splat. A per-chirp renderer never forms the slow-time axis, the Nl loops of a frame along which Doppler is measured. For the person we therefore splat along slow time as well (Fig. 2), because the DFT over loops is linear in a path’s slow-time samples just as the range FFT is linear in its fast-time samples. Transmitter t samples a given path once a loop, so the path’s slow-time samples are a single tone. A point whose bistatic path length changes at the rate Ṙtr,i = vi · (ut,i + ur,i ), with ut,i , ur,i the unit vectors from the antennas to the point, therefore sits at the fractional Doppler bin νtr,i = Nl Nslots Tc Ṙtr,i /λ, because consecutive loops of one transmitter are one loop period apart. The rendered range–Doppler product of one antenna pair is a two-axis splat, RDtr [d, k] = g

N X

bi Vi Str (xi ) e jϕt,i ΦK (k − ktr,i ) ΦNl (d − νtr,i ),

(3)

i=1

with g a learned receiver gain per radar, bi the person’s learned per-joint scattering gains skinned to point i (1 for scene points), Vi the point’s visibility V in that frame, ktr,i = (Rt,i + Rr,i )/(2∆R) the fractional range bin, ϕt,i the phase the transmitter’s own slot contributes, and ΦK , ΦNl the Hann window’s exact response on the two axes. ΦNl includes the slow-time mean removal, so static points deposit nothing here. We truncate ΦK to L = 15 range taps and ΦNl to 7 Doppler taps, so a person path costs 105 accumulations instead of a 64 × 256 cube per antenna pair, and its backward pass costs the same. App. A.3 derives this closed form from the per-chirp signal it replaces. Material model. The BSDF fi is a closed-form Jones BSDF (Wei et al., 2024; Armouti et al., 2026a), evaluated from a six-parameter ITU-R P.2040 material vector (ITU-R, 2023): a coherent specular lobe and a small-perturbation lobe, blended by the ITU roughness gates and resolved in a local s/p polarization basis (App. A.2). The scene’s materials are per point, while the whole person shares one six-vector, because per-point materials let the fit darken the limbs (App. B.2). Visibility and occlusion. The binary V of Eq. (1) is a per-frame ray cast that runs in both directions. A body point is switched off when its segment to the radar is blocked by the posed mesh or by the scene. A scene point is switched off for the frames in which the body stands between it and the radar (App. B.1). This matters more here than for a static renderer, because the shadow moves with the person. An unmasked scene point renders a return that stays constant over the frame’s loops while the recording interrupts it, and the zero-Doppler plane is most sensitive to that error. 5

Preprint

Noise model. A clutter-removed range–Doppler map is mostly empty, and its empty cells sit at the receiver’s noise floor instead of zero. A renderer that models no noise predicts exactly zero there, so an L2 on linear magnitude would charge the fit for every empty cell and the fit would answer by manufacturing power. This is a Doppler problem, because a range–azimuth image is filled by the static scene while most of a resolved range–Doppler product is floor. We therefore make the floor N (d, k) part of the forward model, a log scale and a range shape over a prior taken from the recording’s own off-person power, added to the rendered power wherever magnitudes are compared. It enters the loss only through the two plane terms, which fit its level, and the unoccupied range– Doppler level of Table 2 (LU = 0.92/0.92, 0.96 on the cascade) tests whether it is right. Benefits of 4D Point Splatting. Render any DSP representation: Eq. (3) renders the raw complex return a real radar records, so the range, azimuth, elevation and Doppler FFTs of the real data’s DSP turn one fitted scene into any representation a downstream model reads (App. A.1). Differentiable velocity: νtr,i depends on the point’s velocity, so the Doppler axis carries gradient through the skinning to the joint velocities, and the radar, not the camera, settles what the limbs did. Corpusscale: that per-path cost is independent of Nl , so a whole corpus is affordable (Sec. 3.3). 3.3

O PTIMIZATION AND I MPLEMENTATION

Our objective function evaluates different projections of the linear RAED magnitude tensor: win L = LRAE,0 + LRAE,pl + Lwin RAE,0 + LRAE,pl , | {z } {z } | global

(4)

windowed on the person

a normalized L2 on |RAE| (App. B.2), taken at two slow-time views and two scopes. The firstchirp term LRAE,0 keeps the static clutter return, so it is the only term that reaches the whole static scene. Because it is a magnitude, it supervises ADC amplitude and inter-element phase without fitting absolute phase noise. The plane term LRAE,pl supervises the 63 non-DC Doppler planes, which only the moving person reaches, so a velocity error that leaves the range profile intact still moves energy between planes. The windowed pair restricts both to the person’s spatial window and normalizes over it, because a frame-wide divisor lets a large relative error on the person hide behind the scene’s energy. Those cells hold the occupied range–Doppler region and the microDoppler signature. All four terms apply FFTs sized by the radar’s virtual aperture and chirp count, irrespective of the downstream model’s beamformer, unlike HybridSim (Xiong et al., 2026), which trains its renderer end-to-end with that model. What we optimize. For the static scene we optimize per-point materials and normals. For the person we optimize per-point normals, one material six-vector and 14 joint velocities, each the video’s kinematic velocity times a learned per-joint scale plus a bounded, band-limited residual, skinned to the points so that a joint the video calls still stays still. For the radar we optimize its gains and the noise floor (Table 5). Scale and runtime. A chirp is rendered once per physical transmitter–receiver pair, so fitting HuPR is 235 × 600 × 64 × 24 chirps and RT-Pose 100 × 299 × 64 × 192, for a total of 584 million chirps. Each is fitted for 400 epochs, so the fit renders 234 billion chirps forward and as many again backward, without ever forming a slow-time cube. That is 63,000 times the data 3DPS (Armouti et al., 2026a) fits, at 43.9 GPU-hours on H100s (App. H.2).

4

E VALUATION

Sec. 4.1 fits 235 sequences from HuPR (Lee et al., 2023) and 100 from RT-Pose (Ho et al., 2025), two public human-sensing corpora that retain slow time and a colocated camera. HuPR records raw ADC from two orthogonal single-chip radars, and RT-Pose from one cascade (App. B.1). Sec. 4.2 measures sim2real with the pose model each dataset released, MSCSA-PRGCN (Lee et al., 2023) and HRRadarPose (Ho et al., 2025), retrained from scratch for four seeds at equal optimizer steps. Each model keeps its own metric, because the labels differ. HuPR’s 2D image keypoints admit OKS AP but no metric distance, and RT-Pose’s 3D joints admit MPJPE in centimeters. RT-Pose’s joints come from the mesh that drives the render (App. E). No public simulator takes a real capture as input, so RF-Genesis (Chen & Zhang, 2024) is driven forward from the same registered motion (App. C.3), and our unfit render is the floor. Sec. 4.3 compares real- and rendered-trained models. 6

Preprint

Table 2: Rendering fidelity. Medians over sequences (HuPR pools both radars) of the Pearson correlation with the recording: range–angle (ρ; ρc clutter-removed), occupied range–Doppler cells (ρM ), and micro-Doppler centroid, spread and spectrogram (ρµ , ρσ , ρS ); LU : rendered-to-real noise-floor level. Bold: best per section. App. D gives the other DSP representations. Range–angle Range–Doppler ρ↑

Method

ρM ↑ LU → 1

ρµ ↑

ρσ ↑

ρS ↑

HuPR, two single-chip radars RF-Genesis (Chen & Zhang, 2024) 0.134 0.339 0.518 4DPS (unfit) 0.878 0.270 0.537 4DPS (Ours) 0.940 0.666 0.829

0.05 0.57 0.92

0.820 0.063 0.671 0.930 0.283 0.754 0.993 0.966 0.988

RT-Pose, cascade, one-person sequences RF-Genesis 0.119 0.221 0.324 4DPS (unfit) 0.648 0.181 0.291 4DPS (Ours) 0.830 0.470 0.695

0.09 0.58 0.96

0.750 0.042 0.394 0.803 0.185 0.515 0.973 0.704 0.966

Sequence 159 Real

4DPS

Sequence 147 RF-Genesis

Real

4DPS

Sequence 166 RF-Genesis

Real

4DPS

RF-Genesis

0.97

0.46

0.97

0.52

0.98

0.40

0.78

0.30

0.83

0.44

0.72

0.32

0.86

0.17

0.88

0.22

0.87

0.09

RA

RD

micro-Doppler

ρc ↑

Micro-Doppler

Figure 3: Real, 4DPS and RF-Genesis on three RT-Pose test sequences. Rows: micro-Doppler for the full sequence; typical range–Doppler frame; and typical range–azimuth frame; the number is the render’s sequence-level correlation with the recording. Activities: 159 stand and lift a leg, 147 stand and wave, 166 walk and wave. See App. J for more details.

4.1

R ENDERING F IDELITY

Does 4DPS reproduce the real capture, on every DSP representation? The fitted render reaches 0.940 range–azimuth and 0.988 micro-Doppler correlation on HuPR, and 0.830 and 0.966 on RTPose (Table 2). Fig. 3 shows three RT-Pose sequences beside the recording and RF-Genesis, and Tables 7–10 give every column. RF-Genesis, driven by the same motion, reaches only 0.134 and 0.671 on HuPR and 0.119 and 0.394 on RT-Pose. It renders the person alone, with no room, so it misses the static scene that fills a real range–azimuth map, and it keeps the body’s bulk motion (micro-Doppler centroid 0.820 on HuPR) but loses the limbs’ spread around it (0.063 against 0.966 fitted), below even our unfit render, because its 800 body scatterers are reassigned between ray casts and nothing fits their motion to the recording. Our control is the unfit render, the same chain with the velocity field trained for zero epochs, which reaches 0.878, 0.754, 0.648 and 0.515. In a paired, onesided comparison, the control scores lower on every sequence, for both representations and datasets. The one exception is range–azimuth on one HuPR sequence’s vertical board. Camera motion alone already tracks the body’s bulk radial movement, so the control keeps a Doppler-centroid correlation of 0.930 on HuPR and 0.803 on RT-Pose. What it loses is the spread around that centroid, where the limbs’ micro-motion sits. Its spread correlation falls to 0.283 on HuPR and 0.185 on RT-Pose, against 0.966 and 0.704 for the fitted render. The radar fit therefore recovers the motion the video gets wrong. The cascade is the harder case, because it resolves elevation, the axis the scaffold reconstructs least well. The still recordings score lowest. 7

Preprint

Table 3: Pose estimation downstream model training: real vs rendered data. On RT-Pose we train HRRadarPose (Ho et al., 2025), and on HuPR MSCSA-PRGCN (Lee et al., 2023). Every row is tested on real data, so the rendered-trained rows measure sim-to-real transfer. Mean (std) over four seeds trained from scratch. Table 11 gives the full train–test matrix. RT-Pose (cm ↓)

HuPR (AP ↑)

Train

Test

MPJPE

MRPE

Abs-MPJPE

Real

Real

16.67 (0.42)

13.14 (0.87)

19.07 (0.52)

AP

AP50

AP75

65.6 (1.7) 96.9 (0.2) 78.0 (3.1)

RT-Pose

RF-Genesis Real 127.59 (48.32) 106.48 (45.00) 148.88 (24.86) 1.8 (3.5) 8.5 (16.1) 0.0 (0.1) 4DPS Real 21.86 (1.33) 17.06 (0.98) 25.45 (1.69) 38.4 (2.9) 80.6 (3.2) 30.1 (4.3)

R→R

16.7

S→R

21.9

Per keypoint 0.03 0.13 127.6

S→R 0

50

100

150

200

HuPR

MPJPE (cm) ↓ 65.6

R→R 38.4

S→R 1.8

S→R 0

20

40

60

0.0

4DPS

0.76

0.2

0.4

0.6

0.8

1.0

retention of R→R accuracy (S→R)

AP (%) ↑ trained on real

0.59

head neck shoulders elbows wrists hips knees ankles

RF-Genesis

RT-Pose

HuPR

4DPS

RF-Genesis

Figure 4: The sim-to-real gap. Left: real-data MPJPE on RT-Pose (↓) and AP on HuPR (↑) after training on real data, 4DPS or RF-Genesis (four seeds, ±1 std; bars on their true scale). Right: retention of real-trained accuracy per keypoint group (S→R over R→R, the inverse for MPJPE); on HuPR the 4DPS deficit sits in the arms (shaded). Table 3 gives every metric.

4.2

T RAINING P OSE M ODELS ON R ENDERED DATA

How much of real-trained accuracy survives training on renders? Every row of Table 3 is tested on real data. Trained on 4DPS renders alone, each model keeps 0.59 of its real-trained accuracy on HuPR (38.4 (2.9) against 65.6 (1.7) AP) and 0.76 on RT-Pose (21.86 (1.33) against 16.67 (0.42) cm). A model trained on RF-Genesis renders of the same motion transfers almost nothing. It reaches 1.8 (3.5) AP on HuPR and 127.59 (48.32) cm MPJPE on RT-Pose (Fig. 4, left). Both renderers see the same motion, so the difference lies in what each makes of it. An RF-Genesis render holds the person alone and carries little of the limbs’ micro-motion (Sec. 4.1), so a model trained on it never learns to separate a moving body from the room around it. The 4DPS renders do carry the pose information a model needs. Tested on held-out renders, a 4DPS-trained model scores 59.6 (1.4) AP and 15.71 (1.38) cm, close to the real-trained model on recordings (Table 11). Its drop on recordings therefore comes from the remaining rendered–real differences, which the model first meets at test time. Per keypoint, the drop concentrates in the arms (Fig. 4, right). On HuPR the wrists fall from 22.2 and 25.1 AP to 8.0 and 4.7, while hips, knees and the head keep 0.73 to 0.80 of their real-trained AP. The arms are the body’s smallest and fastest-moving parts, and their rendered motion comes from the video’s kinematics, corrected only within a bounded residual (Sec. 3.3). HuPR’s coarse azimuth resolution also gives the fit little signal to correct them, so they depart most from the recording. HuPR is the harder downstream task of the two, although its renders score higher on fidelity (Table 2), for three reasons. i) One sequence gives the renderer 921,600 chirps against RT-Pose’s 3,674,112, a factor of 0.25. ii) Its radars also resolve the lateral axes far more coarsely. Eight azimuth elements on a single chip put the broadside resolution near 14.3◦ , against 1.3◦ for the cascade’s 86 slots. Each HuPR board resolves one axis only, azimuth on the horizontal and elevation on the vertical, and carries 2 elevation bins where the cascade carries 8. The labels, meanwhile, are keypoints in a camera image whose pixels subtend about 0.2◦ . The model must place joints on a grid some seventy times finer than its own sensor resolves laterally. iii) Coarse angles make the 8

Preprint

Table 4: Do rendered- and real-trained models learn the same features? Each cell is the real– rendered distance over the real–real distance between seeds (1.0: as close as two real-trained models), with its 95% interval; the readout columns are error ratios. Bold: the interval lies above the 1.2× margin fixed before the analysis. † : uninformative, as an untrained network is about as close. HuPR’s middle layers are undefined (dead temporal merges); App. F gives every layer and the errorconsistency analysis. Feature similarity CKA first

CKA middle

RT-Pose 1.35, 1.17 0.98–1.10 1.15 HuPR 1.37 undefined

Linear readout

CKA backbone

CKA logits

Procrustes Rendered- Untrained every layer trained backbone

0.98† 1.08† 1.10†

0.99†

0.98–1.05

1.16

1.37 [1.08, 1.70]

[1.09, 1.28]

1.25

1.30 1.05–1.17

[1.20, 1.38]

2.50

camera-to-radar extrinsic harder to estimate as well, and our fit is sensitive to that registration, as prior radar inverse renderers are (Armouti et al., 2026b;a) (App. I). 4.3

C OMPARING R EAL - AND R ENDERED -T RAINED M ODELS

Does a model trained on renders learn the same features as one trained on real data? We compare the two models’ activations layer by layer with two standard similarity measures, orthogonal Procrustes distance (Williams et al., 2021) and debiased linear CKA (Kornblith et al., 2019; Nguyen et al., 2021; Murphy et al., 2024), because no single measure wins across settings (Klabunde et al., 2025). Each distance (Procrustes, 1−CKA) is divided by that between two of five real-trained models, which differ only in seed, so 1.0 means as similar as two real-trained models, and we fixed 1.2 as the margin before running the analysis (Table 4). The Procrustes ratio stays between 0.98 and 1.05 at every layer on RT-Pose and between 1.05 and 1.17 on HuPR, and CKA clears the margin with confidence only at RT-Pose’s first layer (1.35), where the rendered and real inputs themselves differ most. Training on renders therefore teaches the same features as training on real data, consistent with Sec. 4.2, where the rendered-trained model’s drop comes from rendered–real differences in its input. Features that look alike can still serve pose estimation differently, so we also freeze each model’s feature extractor and fit a new linear pose readout on real data. On HuPR, features learned from renders give 1.30 times the keypoint error of real-trained features, against 2.50 times for an untrained network. Therefore, they carry most but not all of the pose information. At the deepest layers even an untrained network scores about as close († ), so similarity there is no evidence either way. App. F gives every layer, the error-consistency analysis and the protocol.

5

C ONCLUSION

We presented 4DPS, a physics-based differentiable renderer that fits an explicit moving scene to a real radar capture. Camera-initialized oriented points carry an ITU-R P.2040 material, a normal and a velocity, and are splatted in closed form into range and Doppler bins on the TDM timing of a MIMO array. The closed-form Doppler splat makes the fit cheap enough to run unattended over all 235 HuPR and 100 RT-Pose sequences, and because 4DPS renders the raw complex return, the real data’s DSP turns it into any representation. 4DPS sets the state of the art on range–azimuth, range– Doppler and micro-Doppler. On HuPR it reaches median correlations of 0.666 on clutter-removed range–azimuth and 0.988 on micro-Doppler, against 0.339 and 0.671 for RF-Genesis driven by the same motion. Pose models trained on our renders alone keep 0.59 of their real-trained accuracy on HuPR and 0.76 on RT-Pose, while models trained on RF-Genesis renders do not transfer. The rendered- and real-trained models also compute the same features, so the remaining gap comes from rendered–real differences in their input. Limitations and future work. Velocity stays within a bounded residual of the video kinematics. The background is static and rendered along singlebounce paths, and we model one person per sequence. RT-Pose’s labels come from the body mesh that drives the render, and no public simulator takes a capture as input, so our baselines run forward (App. I). Scoring edited scenes against real recordings is the next step (App. G). 9

Preprint

R EPRODUCIBILITY S TATEMENT Code, configurations, the fitted checkpoints of every sequence and the scripts behind every table will be released. Both datasets are public (HuPR (Lee et al., 2023), RT-Pose (Ho et al., 2025)). The forward model is stated in Sec. 3 with every learned quantity in Table 5; the preprocessing chain, the runtime design and the downstream protocol are in the supplement (Secs. B.1, H.2, B.2, F).

U SE OF AI S TATEMENT We used large language models to implement and edit code, clean and reformat the datasets, assist in writing, and give feedback on the research methodology and experiments. The authors reviewed all AI-assisted work and take full responsibility for the content of this paper.

R EFERENCES Fadel Adib, Hongzi Mao, Zachary Kabelac, Dina Katabi, and Robert C. Miller. Smart Homes That Monitor Breathing and Heart Rate. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, CHI ’15, pp. 837–846, New York, NY, USA, 2015. Association for Computing Machinery. ISBN 9781450331456. doi: 10.1145/2702123.2702200. URL https://doi.org/10.1145/2702123.2702200. Sizhe An and Umit Y. Ogras. MARS: mmWave-Based Assistive Rehabilitation System for Smart Healthcare. ACM Trans. Embed. Comput. Syst., 20(5s), September 2021. ISSN 1539-9087. doi: 10.1145/3477003. URL https://doi.org/10.1145/3477003. Sizhe An, Yin Li, and Umit Ogras. mRI: Multi-Modal 3D Human Pose Estimation Dataset Using mmWave, RGB-D, and Inertial Sensors. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (eds.), Advances in Neural Information Processing Systems, volume 35, pp. 27414–27426. Curran Associates, Inc., 2022. doi: 10.52202/068431-1988. URL https://proceedings.neurips.cc/paper_files/paper/2022/file/ af9c9c6d2da701da5a0acf91ec217815-Paper-Datasets_and_Benchmarks. pdf. Adnan Armouti, Yixuan Gao, and Rajalakshmi Nandakumar. 3D Point Splatting for mmWave Radar Novel View Synthesis, 2026a. URL https://arxiv.org/abs/2609.11894. Adnan Armouti, Yixuan Gao, and Rajalakshmi Nandakumar. mmIR: Frequency-Space Inverse Rendering for 3D Millimeter-Wave Radar ADC Synthesis. In Paolo Favaro, Zuzana Kukelova, Atsuto Maki, Anna Rohrbach, Konrad Schindler, and Federico Tombari (eds.), Computer Vision – ECCV 2026, pp. 148–166, Cham, 2026b. Springer Nature Switzerland. ISBN 978-3-032-37359-5. David Borts, Erich Liang, Tim Broedermann, Andrea Ramazzina, Stefanie Walz, Edoardo Palladin, Jipeng Sun, David Brueggemann, Christos Sakaridis, Luc Van Gool, Mario Bijelic, and Felix Heide. Radar Fields: Frequency-Space Neural Scene Representations for FMCW Radar. In ACM SIGGRAPH 2024 Conference Papers, SIGGRAPH ’24, New York, NY, USA, 2024. Association for Computing Machinery. ISBN 9798400705250. doi: 10.1145/3641519.3657510. URL https://doi.org/10.1145/3641519.3657510. Anjun Chen, Xiangyu Wang, Shaohao Zhu, Yanxu Li, Jiming Chen, and Qi Ye. mmBody Benchmark: 3D Body Reconstruction Dataset and Analysis for Millimeter Wave Radar. In Proceedings of the 30th ACM International Conference on Multimedia, MM ’22, pp. 3501–3510, New York, NY, USA, 2022. Association for Computing Machinery. ISBN 9781450392037. doi: 10.1145/3503161.3548262. URL https://doi.org/10.1145/3503161.3548262. Chuhan Chen, Tianshu Huang, Akarsh Prabhakara, Chaithanya Kumar Mummadi, Zhongxiao Cong, Anthony Rowe, Matthew O’Toole, and Deva Ramanan. RadarSim: Simulating Single-Chip Radar via Multimodal Neural Fields. In 2026 International Conference on 3D Vision (3DV), pp. 1258– 1268, 2026. doi: 10.1109/3DV69130.2026.00124. 10

Preprint

V.C. Chen, F. Li, S.-S. Ho, and H. Wechsler. Micro-Doppler Effect in Radar: Phenomenon, Model, and Simulation Study. IEEE Transactions on Aerospace and Electronic Systems, 42(1):2–21, 2006. doi: 10.1109/TAES.2006.1603402. Xingyu Chen and Xinyu Zhang. RF Genesis: Zero-Shot Generalization of mmWave Sensing through Simulation-Based Data Synthesis and Generative Diffusion Models. In Proceedings of the 21st ACM Conference on Embedded Networked Sensor Systems, SenSys ’23, pp. 28–42, New York, NY, USA, 2024. Association for Computing Machinery. ISBN 9798400704147. doi: 10.1145/ 3625687.3625798. URL https://doi.org/10.1145/3625687.3625798. Xingyu Chen, Xinyu Zhang, Qiyue Xia, Xinmin Fang, Chris Xiaoxuan Lu, and Zhengxiong Li. Differentiable Radio Frequency Ray Tracing for Millimeter-Wave Sensing, 2023. URL https: //arxiv.org/abs/2311.13182. Xingyu Chen, Jianrong Ding, Kai Zheng, Xinmin Fang, Xinyu Zhang, Chris Xiaoxuan Lu, and Zhengxiong Li. InverTwin: Solving Inverse Problems via Differentiable Radio Frequency Digital Twin, 2025. URL https://arxiv.org/abs/2508.14204. Guoxuan Chi, Zheng Yang, Chenshu Wu, Jingao Xu, Yuchong Gao, Yunhao Liu, and Tony Xiao Han. RF-Diffusion: Radio Signal Generation via Time-Frequency Diffusion. In Proceedings of the 30th Annual International Conference on Mobile Computing and Networking, ACM MobiCom ’24, pp. 77–92, New York, NY, USA, 2024. Association for Computing Machinery. ISBN 9798400704895. doi: 10.1145/3636534.3649348. URL https://doi.org/10.1145/ 3636534.3649348. Han Cui, Shu Zhong, Jiacheng Wu, Zichao Shen, Naim Dahnoun, and Yiren Zhao. MiliPoint: A Point Cloud Dataset for mmWave Radar. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (eds.), Advances in Neural Information Processing Systems, volume 36, pp. 62713–62726. Curran Associates, Inc., 2023. doi: 10.52202/075280-2739. URL https://proceedings.neurips.cc/paper_files/paper/2023/file/ c60468eca9cd0b0083f0ff9d0aeb171a-Paper-Datasets_and_Benchmarks. pdf. Frances Ding, Jean-Stanislas Denain, and Jacob Steinhardt. Grounding Representation Similarity through Statistical Testing. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan (eds.), Advances in Neural Information Processing Systems, volume 34, pp. 1556–1568. Curran Associates, Inc., 2021. URL https://proceedings.neurips.cc/paper_files/paper/2021/file/ 0c0bf917c7942b5a08df71f9da626f97-Paper.pdf. Junqiao Fan, Yunjiao Zhou, Yizhuo Yang, Xinyuan Cui, Jiarui Zhang, Lihua Xie, Jianfei Yang, Chris Xiaoxuan Lu, and Fangqiang Ding. M4Human: A Large-Scale Multimodal mmWave Radar Benchmark for Human Mesh Reconstruction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 42836–42846, June 2026. Xiangyu Gao, Guanbin Xing, Sumit Roy, and Hui Liu. RAMP-CNN: A Novel Neural Network for Enhanced Automotive Radar Object Recognition. IEEE Sensors Journal, 21(4):5119–5132, 2021. doi: 10.1109/JSEN.2020.3036047. Robert Geirhos, Kristof Meding, and Felix A. Wichmann. Beyond Accuracy: Quantifying Trial-by-Trial Behaviour of CNNs and Humans by Measuring Error Consistency. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Advances in Neural Information Processing Systems, volume 33, pp. 13890–13902. Curran Associates, Inc., 2020. URL https://proceedings.neurips.cc/paper_files/paper/2020/ file/9f6992966d4c363ea0162a056cb45fe5-Paper.pdf. Junfeng Guan, Sohrab Madani, Suraj Jog, Saurabh Gupta, and Haitham Hassanieh. Through Fog High-Resolution Imaging Using Millimeter Wave Radar. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 11461–11470, 2020. doi: 10.1109/CVPR42600.2020.01148. 11

Preprint

Chuan Guo, Shihao Zou, Xinxin Zuo, Sen Wang, Wei Ji, Xingyu Li, and Li Cheng. Generating Diverse and Natural 3D Human Motions from Text. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5142–5151, 2022. doi: 10.1109/CVPR52688.2022. 00509. Rongxiao Guo and Qingchao Chen. DGHMesh: A Large-Scale Dual-Radar mmWave Dataset and Generalization-Focused Benchmark for Human Mesh Reconstruction, 2026. URL https:// arxiv.org/abs/2604.22827. Yuan-Hao Ho, Jen-Hao Cheng, Sheng Yao Kuan, Zhongyu Jiang, Wenhao Chai, Hsiang-Wei Huang, Chih-Lung Lin, and Jenq-Neng Hwang. RT-Pose: A 4D Radar Tensor-Based 3D Human Pose Estimation and Localization Benchmark. In Aleš Leonardis, Elisa Ricci, Stefan Roth, Olga Russakovsky, Torsten Sattler, and Gül Varol (eds.), Computer Vision – ECCV 2024, pp. 107–125, Cham, 2025. Springer Nature Switzerland. ISBN 978-3-031-73036-8. Nikolai Hofmann, Vanessa Wirth, Johanna Bräunig, Ingrid Ullmann, Martin Vossiek, Tim Weyrich, and Marc Stamminger. Inverse Rendering of Near-Field mmWave MIMO Radar for Material Reconstruction. IEEE Journal of Microwaves, 5(2):356–372, 2025. doi: 10.1109/JMW.2025. 3535077. Jakob Hoydis, Faycal Ait Aoudia, Sebastian Cammerer, Merlin Nimier-David, Nikolaus Binder, Guillermo Marcus, and Alexander Keller. Sionna RT: Differentiable Ray Tracing for Radio Propagation Modeling. In 2023 IEEE Globecom Workshops (GC Wkshps), pp. 317–321, 2023. doi: 10.1109/GCWkshps58843.2023.10465179. Tianshu Huang, John Miller, Akarsh Prabhakara, Tao Jin, Tarana Laroia, Zico Kolter, and Anthony Rowe. DART: Implicit Doppler Tomography for Radar Novel View Synthesis. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 24118–24129. IEEE, 2024. Tianshu Huang, Akarsh Prabhakara, Chuhan Chen, Jay Karhade, Deva Ramanan, Matthew O’Toole, and Anthony Rowe. Towards Foundational Models for Single-Chip Radar. In 2025 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 24655–24665, 2025. doi: 10.1109/ ICCV51701.2025.02286. ITU-R. Effects of Building Materials and Structures on Radiowave Propagation above about 100 MHz. Recommendation P.2040-3, International Telecommunication Union, aug 2023. URL https://www.itu.int/rec/R-REC-P.2040/en. Max Klabunde, Tassilo Wald, Tobias Schumacher, Klaus Maier-Hein, Markus Strohmaier, and Florian Lemmerich. ReSi: A Comprehensive Benchmark for Representational Similarity Measures. In Y. Yue, A. Garg, N. Peng, F. Sha, and R. Yu (eds.), International Conference on Learning Representations, volume 2025, pp. 18701–18740, 2025. URL https://proceedings.iclr.cc/paper_files/paper/2025/file/ 2eef1f75516b0cfd3e944345e5f88c08-Paper-Conference.pdf. Simon Kornblith, Mohammad Norouzi, Honglak Lee, and Geoffrey Hinton. Similarity of Neural Network Representations Revisited. In Kamalika Chaudhuri and Ruslan Salakhutdinov (eds.), Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pp. 3519–3529. PMLR, 09–15 Jun 2019. URL https://proceedings.mlr.press/v97/kornblith19a.html. Andrew Kramer, Kyle Harlow, Christopher Williams, and Christoffer Heckman. ColoRadar: The Direct 3D Millimeter Wave Radar Dataset. Int. J. Rob. Res., 41(4):351–360, April 2022. ISSN 0278-3649. doi: 10.1177/02783649211068535. URL https://doi.org/10.1177/ 02783649211068535. Pou-Chun Kung, Skanda Harisha, Ram Vasudevan, Aline Eid, and Katherine A. Skinner. RadarSplat: Radar Gaussian Splatting for High-Fidelity Data Synthesis and 3D Reconstruction of Autonomous Driving Scenes. In 2025 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 27596–27606, 2025. doi: 10.1109/ICCV51701.2025.02562. 12

Preprint

Shih-Po Lee, Niraj Prakash Kini, Wen-Hsiao Peng, Ching-Wen Ma, and Jenq-Neng Hwang. HuPR: A Benchmark for Human Pose Estimation Using Millimeter Wave Radar. In 2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pp. 5704–5713, 2023. doi: 10.1109/WACV56688.2023.00567. Jaime Lien, Nicholas Gillian, M. Emre Karagozler, Patrick Amihood, Carsten Schwesig, Erik Olson, Hakim Raja, and Ivan Poupyrev. Soli: Ubiquitous Gesture Sensing with Millimeter Wave Radar. ACM Trans. Graph., 35(4), July 2016. ISSN 0730-0301. doi: 10.1145/2897824.2925953. URL https://doi.org/10.1145/2897824.2925953. Teck-Yian Lim, Spencer A. Markowitz, and Minh N. Do. RaDICaL: A Synchronized FMCW Radar, Depth, IMU and RGB Camera Data Dataset with Low-Level FMCW Radar Signals. IEEE Journal of Selected Topics in Signal Processing, 15(4):941–953, 2021. doi: 10.1109/JSTSP.2021. 3061270. Matthew Loper, Naureen Mahmood, Javier Romero, Gerard Pons-Moll, and Michael J. Black. SMPL: A Skinned Multi-Person Linear Model. ACM Trans. Graph., 34(6), November 2015. ISSN 0730-0301. doi: 10.1145/2816795.2818013. URL https://doi.org/10.1145/ 2816795.2818013. Naureen Mahmood, Nima Ghorbani, Nikolaus F. Troje, Gerard Pons-Moll, and Michael J. Black. AMASS: Archive of Motion Capture as Surface Shapes. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), October 2019. Alex Murphy, Joel Zylberberg, and Alona Fyshe. Correcting Biased Centered Kernel Alignment Measures in Biological and Artificial Neural Networks, 2024. URL https://arxiv.org/ abs/2405.01012. Thao Nguyen, Maithra Raghu, and Simon Kornblith. Do Wide and Deep Networks Learn the Same Things? Uncovering How Neural Network Representations Vary with Width and Depth. In International Conference on Learning Representations, 2021. URL https://openreview. net/forum?id=KJNcAkY8tY4. Mahan Rafidashti, Ji Lan, Maryam Fatemi, Junsheng Fu, Lars Hammarstrand, and Lennart Svensson. NeuRadar: Neural Radiance Fields for Automotive Radar Point Clouds. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 2479–2489, 2025. doi: 10.1109/CVPRW67362.2025.00234. M. Mahbubur Rahman, Ryoma Yataka, Sorachi Kato, Pu Wang, Peizhao Li, Adriano Cardace, and Petros Boufounos. MMVR: Millimeter-Wave Multi-View Radar Dataset and Benchmark for Indoor Perception. In Aleš Leonardis, Elisa Ricci, Stefan Roth, Olga Russakovsky, Torsten Sattler, and Gül Varol (eds.), Computer Vision – ECCV 2024, pp. 306–322, Cham, 2025. Springer Nature Switzerland. ISBN 978-3-031-72986-7. Julien Rebut, Arthur Ouaknine, Waqas Malik, and Patrick Pérez. Raw High-Definition Radar for Multi-Task Learning. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 17000–17009, 2022. doi: 10.1109/CVPR52688.2022.01651. Ignacio Roldan, Andras Palffy, Julian F. P. Kooij, Dariu M. Gavrila, Francesco Fioranelli, and Alexander Yarovoy. A Deep Automotive Radar Detector Using the RaDelft Dataset. IEEE Transactions on Radar Systems, 2:1062–1075, 2024. doi: 10.1109/TRS.2024.3485578. Argha Sen, Anirban Das, Swadhin Pradhan, and Sandip Chakraborty. A Dataset for MultiIntensity Continuous Human Activity Recognition through Passive Sensing, 2024. URL https: //arxiv.org/abs/2407.21125. Zehong Shen, Huaijin Pi, Yan Xia, Zhi Cen, Sida Peng, Zechen Hu, Hujun Bao, Ruizhen Hu, and Xiaowei Zhou. World-Grounded Human Motion Recovery via Gravity-View Coordinates. In SIGGRAPH Asia 2024 Conference Papers, SA ’24, New York, NY, USA, 2024. Association for Computing Machinery. ISBN 9798400711312. doi: 10.1145/3680528.3687565. URL https: //doi.org/10.1145/3680528.3687565. 13

Preprint

Akash Deep Singh, Sandeep Singh Sandha, Luis Garcia, and Mani Srivastava. RadHAR: Human Activity Recognition from Point Clouds Generated through a Millimeter-Wave Radar. In Proceedings of the 3rd ACM Workshop on Millimeter-Wave Networks and Sensing Systems, mmNets ’19, pp. 51–56, New York, NY, USA, 2019. Association for Computing Machinery. ISBN 9781450369329. doi: 10.1145/3349624.3356768. URL https://doi.org/10.1145/ 3349624.3356768. Merrill Ivan Skolnik et al. Introduction to Radar Systems, volume 3. McGraw-hill New York, 1980. Tieshuai Song, Jiandong Ye, Ao Guo, Guidong He, and Bin Yang. RadarRGBD A Multi-Sensor Fusion Dataset for Perception with RGB-D and mmWave Radar, 2025. URL https://arxiv. org/abs/2505.15860. Shobha Sundar Ram and Hao Ling. Simulation of Human MicroDopplers Using Computer Animation Data. In 2008 IEEE Radar Conference, pp. 1–6, 2008. doi: 10.1109/RADAR.2008.4720816. Guy Tevet, Sigal Raab, Brian Gordon, Yoni Shafir, Daniel Cohen-Or, and Amit Haim Bermano. Human Motion Diffusion Model. In The Eleventh International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=SJ1kSyO2jwu. Alexander Vilesov, Pradyumna Chari, Adnan Armouti, Anirudh Bindiganavale Harish, Kimaya Kulkarni, Ananya Deoghare, Laleh Jalilian, and Achuta Kadambi. Blending Camera and 77 GHz Radar Sensing for Equitable, Robust Plethysmography. ACM Trans. Graph., 41(4), July 2022. ISSN 0730-0301. doi: 10.1145/3528223.3530161. URL https://doi.org/10. 1145/3528223.3530161. Shelly Vishwakarma, Wenda Li, Chong Tang, Karl Woodbridge, Raviraj Adve, and Kevin Chetty. SimHumalator: An Open-Source End-to-End Radar Simulator for Human Activity Recognition. IEEE Aerospace and Electronic Systems Magazine, 37(3):6–22, 2022. doi: 10.1109/MAES.2021. 3138948. Bruce Walter, Stephen R. Marschner, Hongsong Li, and Kenneth E. Torrance. Microfacet Models for Refraction through Rough Surfaces. In Proceedings of the 18th Eurographics Conference on Rendering Techniques, EGSR’07, pp. 195–206, Goslar, DEU, 2007. Eurographics Association. ISBN 9783905673524. Fei Wang, Yizhe Lv, Mengdie Zhu, Han Ding, and Jinsong Han. XRF55: A Radio Frequency Dataset for Human Indoor Action Analysis. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., 8(1), March 2024. doi: 10.1145/3643543. URL https://doi.org/10.1145/ 3643543. Jiangtao Wei, Yixiang Luomei, Xu Zhang, and Feng Xu. Learning Surface Scattering Parameters from SAR Images Using Differentiable Ray Tracing. IEEE Transactions on Geoscience and Remote Sensing, 62:1–15, 2024. doi: 10.1109/TGRS.2024.3459620. Alex H Williams, Erin Kunz, Simon Kornblith, and Scott Linderman. Generalized Shape Metrics on Neural Representations. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan (eds.), Advances in Neural Information Processing Systems, volume 34, pp. 4738–4750. Curran Associates, Inc., 2021. URL https://proceedings.neurips.cc/paper_ files/paper/2021/file/252a3dbaeb32e7690242ad3b556e626b-Paper.pdf. Zhi Wu, Dongheng Zhang, Chunyang Xie, Cong Yu, Jinbo Chen, Yang Hu, and Yan Chen. RFMask: A Simple Baseline for Human Silhouette Segmentation with Radio Signals. IEEE Transactions on Multimedia, 25:4730–4741, 2023. doi: 10.1109/TMM.2022.3181455. Weitao Xiong, Tianyu Liu, Peng Li, Kok Chung Chua, Toa Chean Khim, Pu Wang, and Hongfei Xue. HybridSim: A Physics-Learning Hybrid Digital Twin for mmWave Human Sensing, 2026. URL https://arxiv.org/abs/2607.15806. Hongfei Xue, Yan Ju, Chenglin Miao, Yijiang Wang, Shiyang Wang, Aidong Zhang, and Lu Su. mmMesh: Towards 3D Real-Time Dynamic Human Mesh Construction Using Millimeter-Wave. In Proceedings of the 19th Annual International Conference on Mobile Systems, Applications, and Services, MobiSys ’21, pp. 269–282, New York, NY, USA, 2021. Association for Computing 14

Preprint

Machinery. ISBN 9781450384438. doi: 10.1145/3458864.3467679. URL https://doi. org/10.1145/3458864.3467679. Hongfei Xue, Qiming Cao, Chenglin Miao, Yan Ju, Haochen Hu, Aidong Zhang, and Lu Su. Towards Generalized mmWave-Based Human Pose Estimation through Signal Augmentation. In Proceedings of the 29th Annual International Conference on Mobile Computing and Networking, ACM MobiCom ’23, New York, NY, USA, 2023. Association for Computing Machinery. ISBN 9781450399906. doi: 10.1145/3570361.3613302. URL https://doi.org/10.1145/ 3570361.3613302. Jianfei Yang, He Huang, Yunjiao Zhou, Xinyan Chen, Yuecong Xu, Shenghai Yuan, Han Zou, Chris Xiaoxuan Lu, and Lihua Xie. MM-Fi: Multi-Modal Non-Intrusive 4D Human Dataset for Versatile Wireless Sensing. In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (eds.), Advances in Neural Information Processing Systems, volume 36, pp. 18756–18768. Curran Associates, Inc., 2023. doi: 10.52202/075280-0822. URL https://proceedings.neurips.cc/paper_files/paper/2023/file/ 3baf7a39d07e9f4f1e258a412df94521-Paper-Datasets_and_Benchmarks. pdf. Lihe Yang, Bingyi Kang, Zilong Huang, Zhen Zhao, Xiaogang Xu, Jiashi Feng, and Hengshuang Zhao. Depth Anything V2. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang (eds.), Advances in Neural Information Processing Systems, volume 37, pp. 21875–21911. Curran Associates, Inc., 2024. doi: 10.52202/ 079017-0688. URL https://proceedings.neurips.cc/paper_files/paper/ 2024/file/26cfdcd8fe6fd75cc53e92963a656c58-Paper-Conference.pdf. Dongsheng Yuan, Xie Zhang, Weiying Hou, Sheng Lyu, Yuemin Yu, Luca Jiang-Tao Yu, Chengxiao Li, and Chenshu Wu. OctoNet: A Large-Scale Multi-Modal Dataset for Human Activity Understanding Grounded in Motion-Captured 3D Pose Labels. In D. Belgrave, C. Zhang, H. Lin, R. Pascanu, P. Koniusz, M. Ghassemi, and N. Chen (eds.), Advances in Neural Information Processing Systems, volume 38, Main Conference. Curran Associates, Inc., 2025. doi: 10.52202/085713-0469. URL https://proceedings.neurips.cc/paper_files/ paper/2025/file/14950adea25005f89545688fe97fe5ea-Paper-Datasets_ and_Benchmarks_Track.pdf. Bo Zhang, Zimeng Zhou, Boyu Jiang, and Rong Zheng. SUPER: Seated Upper Body Pose Estimation Using mmWave Radars. In 2024 IEEE/ACM Ninth International Conference on Internet-ofThings Design and Implementation (IoTDI), pp. 181–191, 2024. doi: 10.1109/IoTDI61053.2024. 00020. Jiarui Zhang, Zhihao Li, Chong Wang, and Bihan Wen. RF4D: Neural Radar Fields for Novel View Synthesis in Outdoor Dynamic Scenes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 15387–15397, June 2026. Mingmin Zhao, Tianhong Li, Mohammad Abu Alsheikh, Yonglong Tian, Hang Zhao, Antonio Torralba, and Dina Katabi. Through-Wall Human Pose Estimation Using Radio Signals. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018. Peijun Zhao, Chris Xiaoxuan Lu, Bing Wang, Niki Trigoni, and Andrew Markham. CubeLearn: End-to-End Learning for Human Motion Recognition from Raw mmWave Radar Signals. IEEE Internet of Things Journal, 10(12):10236–10249, 2023. doi: 10.1109/JIOT.2023.3237494.

15

Preprint

TABLE OF C ONTENTS A Radar and Rendering Theory

17

A.1 From the Radar Equation to the Rendering Equation . . . . . . . . . . . . . . . . . . . . . .

17

A.2 The BSDF as Evaluated . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

18

A.3 Closed-Form Doppler-Plane Amplitudes . . . . . . . . . . . . . . . . . . . . . . . . . . . .

19

B Implementation Details

20

B.1 Preprocessing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

20

B.2 Training . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

21

C Datasets and Baseline

24

C.1 Dataset survey . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

24

C.2 Baseline survey . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

25

C.3 RF-Genesis as a baseline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

25

D Diagnostic Fidelity Metrics

26

E Downstream Results in Full

29

F Representation Analysis Protocol

32

G Scene Editing

35

G.1 Separability of the scene and the person . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

35

G.2 Novel motions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

35

H Ablations, Runtime, and Peak Memory

40

H.1 Ablations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

40

H.2 Runtime and Peak Memory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

43

I

Limitations

46

J

Additional Qualitative Results

46

16

Preprint

A

R ADAR AND R ENDERING T HEORY

A.1

F ROM THE R ADAR E QUATION TO THE R ENDERING E QUATION

Radar equation. The power received over the single-bounce path TX → x → RX is the bistatic radar equation (Skolnik et al., 1980), Pr =

Pt Gt (ω t ) Gr (ω r ) λ2 σ , (4π)3 Rt2 Rr2

(5)

with Pt the transmit power, Gt , Gr the antenna gain patterns and σ the bistatic radar cross-section. As in prior radar inverse rendering (Armouti et al., 2026b;a), we replace σ by a spatially varying BSDF f (ω t , x, ω r ) and integrate over the receiver-centered hemisphere Ωrx , converting the surface element to receiver solid angle through dωr = |n · ω r | dA/Rr2 , which gives Eq. (1) of the main text. Transmitter t sits at pt and receiver r at pr ; for a surface point x with unit normal n, Rt (x) = ∥x − pt ∥, Rr (x) = ∥x − pr ∥, and ω t , ω r are the unit directions from x toward the transmitter and the receiver. Surface to solid Rangle. A distributed surface replaces the point target’s cross-section by a BSDF f through σ = 4π f cos θt cos θr dA, with cos θr = |n · ω r |, so that the power received from the surface element dA at x is V Pt Gt Gr λ2 f cos θt cos θr dA/((4π)2 Rt2 Rr2 ). That element subtends dωr = cos θr dA/Rr2 at the receiver, which absorbs both cos θr and Rr2 and leaves Eq. (1), the form mmIR integrates by Monte Carlo and 3DPS discretizes deterministically. Riemann sum on the oriented point set. Each point i of the set would be assigned the solid angle its surface patch subtends at the receiver. We set it to 1 for every point. For the scene, the per-point materials absorb the level. For the person, the SMPL mesh’s sampling density is not the view’s, and a cosine weight darkens every oblique surface, the limbs’ sides among them, so the P level is left to the area scale and the learned gains. The sum of per-point powers is then 2 Pr = i Pt λ2 Gt Gr fi cos θt,i ∆Ωi Vi /((4π)2 Rt,i ), whose amplitude form, the quantity Eq. (3) of the main text calls Str (xi ), is p q Gt (ω t,i ) q Str (xi ) = C Gr (ω r,i ) ∆Ωi fi (ω t,i , ω r,i ) cos θt,i Rt,i (6) × e j 2π (Rt,i +Rr,i )/λ , with the carrier phase attached and C a constant that is immaterial because the absolute level is learned. The solid angle ∆Ωi is unit for every point, because the scene’s per-point materials and the person’s area scale α and learned joint gains √ carry the level instead. The renderer √ evaluates /Rt , the per-(point, RX) factor Gr ∆Ωi , the exactly this product: p the per-(point, TX) factor Gt√ per-path factor fi cos θt,i and the constant C = Pt L λ2 /(4π) times fixed receiver and ADC scale factors, with L the fixed antenna loss (Pt = 20 mW and L = 0.40, both immaterial under the learned gain g). Back-facing paths (cos θt or cos θr at the numerical floor) and paths switched off by the visibility masks are given exactly zero weight; the renderer is double-sided, so a normal’s sign flips rather than culls. The splat. The Hann window’s exact response, the Φ of Eq. (3), is ΦK (δ) = 21 DK (δ) − 14 DK (δ − 1) − 14 DK (δ + 1),

(7) PK−1 −j2πδn/K the K-point Dirichlet kernel. Pushing the range FFT inside the with DK (δ) = n=0 e point sum (Armouti et al., 2026a) makes the complex range profile of one physical pair the coherent sum over the visible points Str [k] = g

N X

bi Vi Str (xi ) ΦK (k − ktr,i ) + Ctr [k],

(8)

i=1

where Ctr [k] is the antenna-coupling profile √ estimated from the recording, non-zero on the first 15 bins only, and bi is 1 for scene points and αγi for person points, with α the body’s area scale and γi a per-point scattering gain skinned from 14 learned joint gains. The rendered profile has the shape of the recording’s demultiplexed ADC after the range FFT, (NTX , NRX , K) with K = 256, and no 17

Preprint

channel or bin is dropped. The range FFT of a Hann-windowed chirp is, per point, the window’s response at the point’s fractional bin, Eq. (7) with K = 256, so the ADC-then-FFT pipeline and the direct scatter-add of Eq. (8) are the same computation up to the truncation to L = 15 taps. The bin index ktr,i = (Rt,i + Rr,i )/(2∆R) with ∆R = cfs /(2SK) (4.29 cm on HuPR, 4.51 cm on the cascade) is rounded only to choose the 15 receiving bins; the carrier phase and the PSF’s fractional argument use the unrounded path length, and the PSF is read from a 1024-entry table over the fractional bin by linear interpolation. Check: one point target through the real and the rendered chain agrees to 1.5 × 10−7 (float32 rounding) on range profiles, loop range–azimuth, the Doppler cube and the Doppler planes; the 15-tap truncation is the one approximation left, 7.3 × 10−4 of the peak, the same on both datasets. Products. Every product is a Fourier transform of the rendered return, taken with the shared windows of App. B.1 on the real and the rendered side alike. The range–Doppler map of one antenna pair is the windowed Nl -point DFT over loops after removing the slow-time mean, RDtr [d, k] =

N l −1 X

wD [l] S[l, t, r, k] − S̄tr [k] e−j2πd l/Nl ,

(9)

l=0

with S̄tr [k] the mean over l and wD the Doppler window, shifted so that d = Nl /2 is DC; Eq. (3) evaluates this directly, without forming S[l, t, r, k]. Range–azimuth is the angle FFT over the azimuth virtual array formed from the same cube, RA[l, k, a] = Fv {wA [v] S[l, t(v), r(v), k]}; range– azimuth–elevation adds an elevation FFT over the array’s elevation rows, two on HuPR and seven on the cascade, so every transmitter contributes; and the range–azimuth–Doppler cube RAD[d, k, a] is Eq. (9) applied along l of RA. The micro-Doppler spectrogram column of a frame is a personrange-gated power sum over channels, MD[d] =

X X

2

w[m] RDtr [d, kp + m] ,

(10)

t,r |m|≤10

with w a 21-bin Hann window and √kp the person’s range bin (the 25th percentile of its point cloud’s range); correlations are taken on MD. The trainer’s angle FFTs have 16 azimuth bins on HuPR’s 8-slot row and 86 on the cascade (2 and 8 elevation bins); the products and metrics use 64 and 128. The angle axes are untapered inside the trainer and the downstream models, because the downstream models beamform untapered, and a wall’s untapered response is far stronger than its tapered view shows. A.2

T HE BSDF AS E VALUATED

The per-path BSDF is the closed-form BSDF of Armouti et al. (2026a), unchanged, in the coherent spatially varying BSDF (CSVBSDF) formulation of Wei et al. (2024). With θ i = (ε′r , ε′′r , σh , ℓc , τ, d) mapped to their physical ranges by sigmoid (τ ) and shifted softplus (the rest), the value the kernel returns for the path (i, t, r) is fi cos θt,i =

h

i τeff |Γh |2 KKA + (1 − τeff ) |Γn |2 KSPM cos θt,i ,

(11)

where KKA = D G/(4 cos θt cos θr ) is a GGX microfacet lobe (Walter et al., 2007) with p 4πσh /λ (clamped to [0.05, 0.95]); KSPM = Smith masking and roughness αGGX = κ κ(cos ∆−1) ηε is an exponential lobe about the mirror direction, with the von Mises–Fisher 4π sinh κ e p prefactor and concentration κ = ℓc /λ/(1 + 3σh /ℓc ) clamped to [0.5, 10] and a permittivitycontrast factor ηε = clamp((|ε′r − 1| + ε′′r )/5, 0.2, 1); Γh and Γn are the multilayer-slab Fresnel coefficients (ITU-R, 2023) of thickness d and permittivity εr = ε′r −jε′′r , evaluated at the half-vector and at the macro normal respectively, each projected from the transmit polarization through the local s/p Jones basis onto the receive polarization and squared; and τeff is the learned blend τ gated by the ITU validity sigmoids in 2πℓc /λ, 2πσh /λ and the incidence angle. The scene’s normals are the third column of a per-point unit quaternion; the person’s are a free unit vector per visible point per frame. 18

Preprint

A.3

C LOSED -F ORM D OPPLER -P LANE A MPLITUDES

The slow-time cube the trainer never forms. Under TDM, transmitter t fires in slot st of every loop, so loop l of transmitter t is acquired at
tl,t = l Nslots + st Tc , (12) with Tc the chirp interval and Nslots the slots per loop (HuPR: Tc = 72 µs, 3 slots, a 216 µs loop; the cascade: 65 µs, 12 slots, 780 µs), and a person point moving with velocity vi changes its bistatic path length to (t, r) at the rate Ṙtr,i =

d Rt,i + Rr,i = vi · ut,i + vi · ur,i , dt

(13)

with ut,i = (xi − pt )/Rt,i and ur,i = (xi − pr )/Rr,i , positive when the point recedes, so its contribution to loop l carries the phase ramp e j2πṘtr,i tl,t /λ . The rendered product of a frame is therefore the complex slow-time cube X S[l, t, r, k] = g bi Vi Str (xi ) ΦK (k − ktr,i ) + Ctr [k] i∈scene

+ g

X

bi Vi Str (xi ) ΦK (k − ktr,i ) e j2πṘtr,i tl,t /λ ,

(14)

i∈person

of shape (Nl , NTX , NRX , K), exactly the shape of the recording’s demultiplexed, range-transformed ADC, for every physical TX–RX pair. Because Eq. (12) places each transmitter at its own intra-loop offset, a moving point accumulates the inter-TX phase progression that a real TDM capture must compensate before angle estimation, and Doppler is coupled to whichever lateral axes the array resolves. Range migration and the change of the BSDF within the frame’s slow-time span (13.8 ms on HuPR, 49.9 ms on the cascade) are neglected; the carrier phase uses every physical transmitter time. The trainer never forms the cube of Eq. (14). For a person path the slow-time samples are one tone: with Eq. (12), tl,t = t0,t + l Tloop , so e j2πṘtr,i tl,t /λ = e jϕ0 e jωl ,

ϕ0 = 2π λ Ṙtr,i t0,t ,

ω = 2π λ Ṙtr,i Tloop ,

(15)

and, with ν = Nl ω/2π the path’s fractional Doppler bin, the windowed, mean-removed DFT of Eq. (9) over l evaluates at unshifted bin d′ to h Ad′ (ν) = e jϕ0 21 DNl (d′ − ν) − 14 DNl (d′ − 1 − ν) i (16) − 41 DNl (d′ + 1 − ν) − md′ DNl (−ν) , the Hann Doppler PSF (Eq. (7) with K = Nl ) plus the mean removal folded into the row, md′ = 12 at d′ = 0, − 14 at d′ = ±1 and 0 elsewhere, which is what makes the rendered planes agree with a real chain that removes the mean before it windows. The loop-0 anchor plane the range–azimuth term reads is Aanchor = e jϕ0 . The rate gradient ∂Ad′ /∂ Ṙ is the same expression with the Dirichlet kernel’s derivative, and both are evaluated in float32 with ν and ϕ0 formed in float64 and reduced to an integer bin and a fraction first. Check, on both datasets, windowed and not: against a float64 reference that is itself 4 × 10−14 from the 64-loop sum and whose gradient is 8 × 10−9 from finite differences, the closed-form kernels are at 7.3 × 10−7 (amplitudes) and 5.4 × 10−7 (rate gradient). Truncation. With the window the Doppler response is compact (the first sidelobe is 2.8 × 10−2 of the peak in amplitude at 2.5 bins and the envelope falls by a factor of eight per octave, in bin units whatever Nl is), so each path deposits into the Ld planes around its own fractional bin, as the range PSF is cut at 15. Measured on the point-target check, the truncation of the planes is 7.3 × 10−4 at 15 taps, 3.5 × 10−3 at 9, 7.7 × 10−3 at 7 and 2.3 × 10−2 at 5; Ld = 7 is the default, and at 400 epochs on sequence 183 the 7-tap and exact arms agree (spectrogram +0.941 / +0.933, energy +0.884 / +0.861, range–Doppler occupied +0.399 / +0.394, the same learned floor). A path whose 7 planes fall outside a view’s rendered support deposits nothing there.

19

Preprint

B

I MPLEMENTATION D ETAILS

B.1

P REPROCESSING

HuPR scaffold. Frames 0, 150, 300 and 450 of the sequence are read from the released RGB, passed in one batch through Depth-Anything-V2 (Large, metric) (Yang et al., 2024), the annotated person’s bounding box (padded by 12%) is cut out of each depth map, and the maps are back-projected under a 52◦ horizontal field of view, estimated from the radar’s range and the released 2D keypoints, at a pixel stride of 2, voxel-downsampled at 4 cm per view and 5 cm after merging, and given PCA normals (0.18 m radius, 50 neighbors) oriented toward the radar. Two person-free real maps are formed per board: the initial material fit trains on the pixelwise median of the four sampled frames’ loop-mean |RA|, and the range alignment scores against the per-cell 25th percentile of loop-mean |RA| over every fourth frame of the sequence (the subject is the only mover and occupies any one cell in a minority of frames, so a low quantile is the static scene with the person gone). The range translation is searched on a 2 cm grid over ±2.4 m and refined at 2 mm, every candidate scored at once on the GPU (0.3 s): the score is the min–max-normalized Pearson correlation of the shifted cloud’s (range, angle) occupancy against the person-free map, mean over both boards, restricted to the cloud’s own angle bins (the camera sees ±26◦ of the radar’s ±75◦ ) and to ranges beyond 0.5 m. Lateral offset and height are not fitted because the map constrains them poorly: positions within 0.03 of the best score span a median of 1.3 m laterally and 1.0 m in height, while the range optimum has no rival within 0.03 on 233 of 235 sequences: sequence 256: −0.55 m (r 0.23 to 0.49); 38: −1.11 m (0.10 to 0.48); 102: −0.77 m (0.13 to 0.57). The point-cloud initialization of Armouti et al. (2026a) then reduces the aligned cloud to N = 20,000 points by a field-of-view restriction, cosine-weighted resampling toward boresight and farthest-point sampling, and fits per-point materials and normals to the person-free static |RA| of both boards for 75 iterations (2–3 s), the trainer refining both for its 400 epochs. RT-Pose scaffold. Eight annotated radar frames spread over the sequence supply Livox LiDAR frames, each densified by the left camera’s Depth-Anything depth rescaled to that frame’s LiDAR by the median LiDAR/predicted depth ratio at the projected LiDAR pixels; the person is cut from each frame by the registered body mesh of that frame (points within 0.25 m of a vertex, the floor kept), so that what the person touches, such as a chair, survives. The clouds are taken to the radar frame by the rig extrinsic, merged at a 3 cm voxel and given PCA normals over a 0.3 m radius (0.5 m is needed for the LiDAR alone, whose scan lines are 0.4 m apart on the far wall). The extrinsic is the identity: the radar sits at the LiDAR to within 0.03 m laterally by two independent person measurements, and a ±0.5 m translation search against the real static map cannot identify a lateral shift (correlation 0.189 to 0.235 while walking the height to the edge of its range), so it is not refitted; the annotation frame’s origin is 0.2 m to the left of the LiDAR and the board is rotated 180◦ about boresight, both verified from the data. The cascade frames are decoded with TI’s phase and amplitude calibration. Antenna coupling. Per radar, the coherent mean over every third frame and over loops of the rangewindowed profiles at the first 15 range bins, per azimuth virtual channel. Nothing else lives within those bins (0.68 m on the cascade), so the coherent mean is the board’s own static near-field return: it explains 99.98% of the near-bin power on HuPR sequence 256 and 99.95% on cascade sequence 183 (the fraction of the near-bin energy the mean explains). It is added to the rendered profiles in training and rendering, and the absolute-gain initialization leaves those bins out. Mesh recovery. GVHMR (Shen et al., 2024) runs in static-camera mode in its own environment, started with the scene stage so that its ∼30 s overlap: the frames held in memory, YOLO tracking of the largest persistent track, ViTPose and the HMR2 features in half precision on half-resolution crops, giving the 6,890 SMPL vertices (Loper et al., 2015) per camera frame. RT-Pose’s true focal length (25.65 mm full-frame equivalent) is passed; HuPR’s is GVHMR’s default. The mesh is a deterministic function of the frames (the same video twice is bitwise; a lossy re-encode moves it 3–5 mm median, a tenth of a range bin). Camera-to-radar placement. RT-Pose: camera to LiDAR by the shipped stereo calibration, LiDAR to radar by the extrinsic; the camera and LiDAR file belonging to radar frame f has stem 3f − 4 on 85 sequences and 2f − 2 on 17, a rig clock fitted on the annotation’s pairs and exact on every annotated frame. HuPR: the camera at the rig, co-boresighted, camera (right, down, forward) to radar (x, −z, y), with the same 52◦ intrinsics at 512 px for the joints’ image projection. 20

Preprint

Per-frame radar registration. For every frame, the clutter-removed range–azimuth cube (slowtime mean removed) is averaged over loops and squared-summed over angle into a range profile of moving power; its peak between bins 10 and 200, refined by a parabola over the two neighbors, is the radar’s range to the person, and the fraction of the profile’s energy within 5 bins of the peak is the detection confidence (a rig of two boards contributes the confidence-weighted mean). The per-frame ratio rf = radar range / GVHMR centroid range is interpolated across frames whose confidence is below 0.35, median-filtered over ±7 frames, clamped to [0.5, 2], flagged where it leaves the confident frames’ interquartile range widened three times (GVHMR’s track is wrong there, and the radar’s range is followed), and every unflagged run is polished by a quadratic Savitzky–Golay filter 51 frames wide, because the median filter’s steps would otherwise add spurious radial motion to the cloud. rf moves the body along its centroid’s camera ray to rf times its range; the body keeps GVHMR’s size, since GVHMR’s error is its depth, not the body’s size. The range search is confined to the camera’s angular sector, on both mirrors of the beamformer’s angle axis, and to a range window bounded by the scale’s own range. Azimuth: one registration per sequence maps the mesh’s azimuth onto the radar’s moving return, sin θradar = a + b sin θmesh , the scale b fitted only where the mesh spans azimuth (sine std ≥ 0.10, r ≥ 0.90) and a yaw alone otherwise; the offsets are 3 to 5 degrees on the RT-Pose walking sequences, and the labels inherit the session’s lateral scale. The median scale is 0.707 (IQR 0.700–0.717) on HuPR sequence 256, i.e. GVHMR 41% long; RT-Pose’s GVHMR runs about 10% short. HuPR’s 8-channel boards do not resolve this frame by frame, so HuPR applies one depth scale per sequence (the median rf over confident frames) and one rig yaw of 3.05◦ , and places the body’s ray from the released 2D keypoints. Joints, velocities, motion frames. The 14 joints in HuPR’s order come from the SMPL joint regressor; their camera-derived velocities are a first-derivative Savitzky–Golay filter, 11 frames (1.1 s at 10 Hz), quadratic. Motion frames for evaluation are those whose pose speed, the 90th percentile over the 14 smoothed joint speeds, is at least 0.15 m/s (1.5 cm per frame), the same threshold on every sequence, pre-registered. The person’s material is initialized as one shared six-parameter vector fitted for one epoch on all frames against the loop-0 |RA| with the scene model fixed, together with the body’s area scale; the trainer refines the shared vector (per-vertex materials let the fit darken the limbs), and the level calibration of App. B.2 sets the area scale after training. Visibility. For each frame the body’s 13,776 SMPL triangles are binned by their (azimuth, elevation) box as seen from the radar on a 128 × 128 grid, padded one cell either side, so a ray asks only the triangles of its cell; the hit test is unchanged, so the result is the brute-force one bit for bit. Person mask: field of view (a boresight cosine above 0.1761, range in (0.5, 9.4) m), then a Möller–Trumbore test of the open segment from the vertex (nudged 1 mm toward the radar, past its own faces) to the radar center against the body’s triangles, and against the scaffold triangles on the radar side of the body. Scene shadow mask: every scaffold point inside the body’s angular silhouette box and beyond its nearest vertex is tested against the body’s triangles; the rest are visible without a test. Both masks are computed for the whole sequence on the device and enter the renderer as a per-point gain of 0 or 1, never as a change of the point set. With the shadow mask the wall behind the subject dims on all six products as it does in the real capture. Windowing. Range and Doppler take the periodic Hann w[n] = 21 − 12 cos(2πn/N ), whose DFT is exactly three bins and whose response is Eq. (7); the symmetric hanning(N ) is not this window and would leave a 1/N mismatch between the two sides. The array axis takes hanning(N + 2)[1: − 1], because an aperture of 8 or 86 slots cannot afford to zero an element. The cascade’s 144 azimuth virtual channels (9 TX × 16 RX) land on 86 half-wavelength grid slots, 58 of them holding two channels which are averaged, then windowed and zero-padded to a 128-bin angle FFT; HuPR’s 8 fill 8 slots, padded to 64. Preprocessing runs in 10–18 s per sequence: the ray casting, voxel downsampling, PCA normals, person ranges and masks batched on the GPU, the ADC decoded once into memory and shared by the stages. B.2

T RAINING

Level calibration. The area scale α is not learned by the objective. It is set once, after training, so that the rendered person has the recording’s level in the transform the downstream pose model of Sec. 4.2 reads: on every fourth training frame the render’s Doppler-summed magnitude is formed in that model’s own beamformer, and α is the value at which the median ratio of rendered to recorded peak inside the pelvis’s window (±0.5 m, ±8◦ ) is one. This uses no labels and no downstream model 21

Preprint

Table 5: Learned and fixed quantities of the forward model. Materials and normals are parameterized as in Armouti et al. (2026a). Learning rates: velocity coefficients 10−2 , person normals 10−3 , scene materials 10−2 , scene normals 5 · 10−3 , person material 10−3 , gains 10−2 , noise floor 3 · 10−2 (Adam). Quantity

Shape

Note

Learned joint velocity scales sj , residual cjk

(14), (bands, 14, 3)

each residual component bounded by 0.75∥vjkin ∥; 3.5 Hz cap one fused parameter over the training frames one six-vector for the body, initialized by the person stage normals as quaternions; Q N = 20,000 log-parameterized; j γj = 1 on the recording’s own off-person prior

person normals ni person material θ scene materials θ i , normals receiver gain g, joint gains γj noise floor N (d, k)

(visible points, 3) per frame (6) (N,6), (N,4) (1) per radar, (14) log scale + 8 rangeDCT coeff. per radar

Fixed positions (registered per frame), skinning weights W , the area scale α (level calibration, after training), antenna patterns, array geometry, TDM schedule, windows, PSF widths (L = 15 range, 7 Doppler), visibility masks Vi , coupling Ctr

weights, it is fitted per sequence on training frames only, and it is the operation a radar engineer calls gain calibration. The velocity field. The 14 joint velocities vj (f ) of frame f are the video-derived kinematic velocities vjkin (f ) (App. B.1) times a learned per-joint scale, plus a bounded residual,
vj (f ) = sj vjkin (f ) + βj (f ) tanh rj (f )/βj (f ) , βj (f ) = 0.75 ∥vjkin (f )∥, (17) with P tanh taken per component and the residual zero where βj (f ) = 0. The residual rj (f ) = k cjk Bk (f ) lives in an orthonormal temporal DCT basis truncated at 420 of the sequence’s 600 bands (209 of 299 on RT-Pose), a 3.5 Hz cap at the 10 Hz frame rate. Each point’s velocity vi = P W ij vj is skinned from the joints by soft weights W (normalized Gaussian weights, σ = 0.25 m, j over the three nearest joints). The normalized L2 form. Every term of Eq. (4) has the same shape, the normalized L2 that prior radar inverse renderers apply to |RA| (Armouti et al., 2026b;a),
2 meanc ŷ• [c] − y• [c] , (18) L• = meanc y• [c]2 the divisor a detached constant per frame and radar. It never contains the prediction, so the terms stay plain L2 and the absolute level remains supervised, and all terms exclude the near-field range bins below 0.5 m. The targets are yRAE,0 = |RAE[0, k, a, e]|, the first loop with the static clutter kept, and yRAE,pl = |RAED[d, k, a, e]| for the 63 planes d ̸= Nl /2, each with q its own divisor and

\ 2 + N (d, k), with the learned floor entering the rendered side under the square root, ŷ = |RAED| so the plane terms are what fit the floor. Both are formed from every transmitter. The windowed pair are the same two terms restricted to the registered person’s box, the pelvis ±0.6 m in range and ±1.2 m across the line of sight at every elevation bin, and normalized over that box, because a frame-wide divisor lets a large relative error on the person hide behind the scene’s energy. A frame renders only the Doppler bins whose real power at the person’s range rows exceeds the frame’s median (κ = 1) plus the motion band (−1.13 to +0.99 m/s), and every rendered plane is supervised, including the empty bins whose zero targets keep the rendered person compact in Doppler. Both datasets use the same settings: 400 epochs on all 600 (HuPR) or 299 (RT-Pose) frames, a constant learning rate for the first half and a cosine decay to 10−2 of it over the second, the plane targets in float16, the Doppler support at κ = 1, every term at unit weight, Adam with the rates of Table 5, the trainer’s angle axes untapered, the level calibration after training in the downstream model’s view, the velocity residual bounded at 0.75 of the joint’s kinematic speed, and the objective 22

Preprint

of Eq. (4). What differs is set by the recording, not tuned: HuPR renders 24 frames a step (two boards of 3 × 4 channels, 600 training frames, 420 of 600 DCT bands), the cascade 4 frames a step (12 × 16 = 192 channels, the count set for a 24 GB card, 299 frames, 209 of 299 bands, the same 3.5 Hz cap); the motion band is the same −1.13 to +0.99 m/s on both, 13 bins on HuPR’s 216 µs loop (0.1408 m/s per bin, ±4.5 m/s unambiguous) and 53 on the cascade’s 780 µs loop (0.0390 m/s per bin, ±1.25 m/s). The plane targets are 63 × 244 × Na per frame and radar (3.9 MB in float32 on HuPR), so they are stored in float16. Every fourth frame is scored every 10 epochs and the bestscoring snapshot is reported. Peak allocated device memory is 18.7 GiB on the cascade and 5.7 GiB on HuPR (median over sequences), and the fit’s time per sequence is in App. H.2.

23

Preprint

C

DATASETS AND BASELINE

C.1

DATASET SURVEY

Table 6 records the mmWave human-sensing datasets we considered and why each was included or excluded, against six eligibility criteria that its caption defines. Two failure modes dominate. The first is the on-chip CFAR and beamforming pipeline discarding slow time before release, which removes every dataset in the table’s second section. The second is withholding camera imagery for privacy, a practice that is tightening rather than relaxing. Only RT-Pose (Ho et al., 2025) and HuPR (Lee et al., 2023) satisfy all six criteria. One terminology caveat carries into the rest of the paper. “Raw” is used inconsistently in this literature, and HIBER (Wu et al., 2023), MMVR (Rahman et al., 2025), and M4Human (Fan et al., 2026) all describe beamformed tensors as raw radar data. Only ADC or I/Q samples preserve the per-chirp phase history that micro-Doppler is computed from. Table 6: Dataset survey. ST: slow time retained, at least two chirps per frame. ADC: raw ADC or I/Q samples, not heatmaps or point clouds. Cam: colocated camera imagery released, not merely used to generate labels. Pose: human pose labels (keypoints or a body mesh). Pub: openly downloadable. Code: reference downstream model code released. ✓ satisfied, × not satisfied, ◦ partially satisfied (access by request or agreement, a withheld or blurred modality, an expired mirror, chirps per frame undocumented, a released cube already transformed along range, angle and Doppler, or model code without training scripts or only from later work). Only RT-Pose and HuPR satisfy all six. Pretrained weights are not a criterion: RT-Pose publishes none, so we train its reference model ourselves (App. E). Dataset

ST

ADC

Cam

Pose

Pub

Code

Selected RT-Pose (Ho et al., 2025) HuPR (Lee et al., 2023)

✓ ✓

✓ ✓

✓ ✓

✓ ✓

✓ ✓

✓ ✓

No slow time (human sensing) MMVR (Rahman et al., 2025) HIBER (Wu et al., 2023) mmBody (Chen et al., 2022) M4Human (Fan et al., 2026) mm-Fi (Yang et al., 2023) mRI (An et al., 2022) OctoNet (Yuan et al., 2025) MiliPoint (Cui et al., 2023) MARS (An & Ogras, 2021) RadHAR (Singh et al., 2019) mmDoppler (Sen et al., 2024)

× × × × × × × × × × ×

× × × × × × × × × × ×

× ✓ ✓ ✓ ◦ ◦ ✓ × × × ×

✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ × ×

✓ ◦ ◦ × ✓ ✓ ✓ ✓ ✓ ✓ ✓

◦ ◦ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓

No camera released CubeLearn (Zhao et al., 2023) ColoRadar (Kramer et al., 2022)

✓ ✓

✓ ✓

× ×

× ×

✓ ✓

✓ ×

Not (fully) public DGHMesh (Guo & Chen, 2026) XRF55 (Wang et al., 2024) mmMesh (Xue et al., 2021) SUPER (Zhang et al., 2024) EquiPleth (Vilesov et al., 2022)

✓ ◦ ✓ ✓ ◦

✓ ◦ ✓ ✓ ✓

✓ ✓ ✓ × ✓

✓ × ✓ ✓ ×

× ◦ × × ◦

× ✓ ✓ × ✓

Not human sensing RaDICaL (Lim et al., 2021) IQ-1M (Huang et al., 2025) RAMP-CNN (Gao et al., 2021) RadarRGBD (Song et al., 2025) RADIal (Rebut et al., 2022) RaDelft (Roldan et al., 2024)

✓ ✓ ✓ ✓ ✓ ✓

✓ ✓ ✓ ◦ ✓ ✓

✓ ✓ ✓ ✓ ✓ ✓

× × × × × ×

✓ ✓ ✓ ✓ ✓ ◦

× ✓ ✓ × ✓ ✓

24

Preprint

C.2

BASELINE SURVEY

A row of Table 2 needs a raw complex signal for our array layout and chirp, driven by the recorded motion rather than by a class label or text, from code that runs. Of the published radar simulators, only RF-Genesis (Chen & Zhang, 2024) meets all of these requirements. mmGPE (Xue et al., 2023) and HybridSim (Xiong et al., 2026) release no code, and SimHumalator (Vishwakarma et al., 2022) renders one receive channel from a fixed library of eleven motion-capture activity classes, so it cannot take our motions or our arrays. C.3

RF-G ENESIS AS A BASELINE

What was run. We ran RF-Genesis’s released code (the public repository as of 3 August 2025) on all 335 sequences that 4DPS renders, on both datasets, and no sequence failed. Each sequence is driven by the registered SMPL mesh that drives the 4DPS render, in the same radar frame, and its output is scored by the same code on the same pose-speed frames (App. D), so the two rows of Table 2 differ only in the renderer. Changes to the released code. Each change is needed for the output to describe our radar and the recorded motion, and none adds a fitted parameter: (i) the camera and the spot light that stands for the transmitter sit at the radar board, along its boresight; (ii) the registered mesh’s vertices replace the text-generated motion; (iii) the ray intersection is evaluated before the render call, which changes no value but lets the pipeline link on our driver; (iv) each chirp is timed at its slot in the recording’s TDM schedule and fires its own transmitter, where upstream spread the chirps over the whole frame and so scaled every Doppler frequency by 3.6×; and (v) the interpolator reads the depth channel where it read intensity as depth. Adapter choices. RF-Genesis interpolates motion at 30 fps, so we resample the registered mesh from the radar’s 10 Hz frames by linear interpolation of its vertices. We give it no environment, because its only environment model is a text-prompted image that is not registered with the body. Downstream training. The RF-Genesis arms of Table 3 use the models, protocol, splits and labels of the 4DPS arms. On RT-Pose one difference is needed: RF-Genesis’s units are arbitrary and HRRadarPose normalizes its input with fixed constants, so without a gain its inputs reach the network as zeros. We therefore scale them by one gain, G = 4.57×108 , fixed before any training as the median, over 8 frames of each of the 44 training sequences, of the recording’s moving-return level (the 99.9th percentile of the mean-removed cube) over RF-Genesis’s (interquartile range 2.43 × 108 to 8.55 × 108 ). HuPR needs no gain, because its preprocessing normalizes each map from the data. Limitations of the comparison. First, RF-Genesis renders no static scene, so its range–angle products hold the person alone while the recording holds the room. The range–azimuth ρ therefore also measures the missing room, and the clutter-removed columns (range–azimuth ρc , range–Doppler ρM and micro-Doppler) are the fair comparison, because they remove the static scene on both sides. Second, the released code is single-bounce diffuse ray casting, not the full system the RF-Genesis paper describes, and we run it as released apart from the changes above. Third, RF-Genesis has no absolute scale and no receiver noise, so for thePcolumns thatP compare levels we multiply its render by one constant per radar and sequence, g = ( |RAreal |2 / |RARF-Genesis |2 )1/2 over the loop-0 range–azimuth magnitude on the retained range bins of every frame, the rule that initializes 4DPS’s own gain. The constant leaves every correlation and shape column unchanged, and because no noise is simulated, RF-Genesis’s LU stays far below 1. Fourth, it runs open loop: nothing is fitted to the capture, so its row measures a forward simulator driven by the recorded motion, not a competing fitted renderer (App. I).

25

Preprint

D

D IAGNOSTIC F IDELITY M ETRICS

The main-paper tables carry the metrics that separate methods. These are the ones that diagnose how a render fails, and they are reported for 4DPS, its unfit render and, where a column is defined for it, RF-Genesis. Protocol details behind Sec. 4.1. The micro-Doppler trajectory statistics (centroid, spread, envelopes) are taken on frames whose pose speed is at least 0.15 m/s, because a frame with no motion has no trajectory to score and its centroid is set by the residual clutter rather than by the subject. The raw chain: ADC, complex range profile and range–azimuth–elevation. The same fitted scene yields the raw ADC samples, the complex range profile and the range–azimuth–elevation magnitude with no retraining, because each is a fixed transform of the per-path quantity Eq. (3) already deposits. Tables 7 and 8 score the ADC and the CRP, and Table 9 scores the RAE next to range–angle, on the same sequences, frames and row sections as the other representations, following the CRP and ADC protocol of Armouti et al. (2026a): the first loop’s azimuth virtual array, nearfield bins zeroed. The magnitude columns of the CRP and the ADC compare each side min-max normalized on its own, so they do not depend on absolute scale, and the ADC envelope is compared after the rank-1 per-channel and per-range phase correction, because a magnitude-only objective leaves that phase free. The relative-phase columns need no correction: each scores the phase step between neighboring samples (fast time t, range r, virtual channel v) against the recording’s, as the |y|-weighted mean of cos ∆ϕ, and a per-bin rotation cancels in that step. Table 7: Rendering fidelity on the ADC samples. Table 2 of the main text carries the seven columns its prose reads; this table and Tables 8–10 carry all of them, grouped by representation, over the same sequences and the same renders, with H, V and C as there. ρenv is the envelope correlation after the rank-1 phase correction; ∆ϕt is the relative-phase coherence along fast time (1 is the recording’s phase step, 0 random), with its weighted RMSE in degrees. The unfit render is not scored. Every column here is scale-free, so RF-Genesis is scored as rendered. The ADC phase depends on the scene: on RT-Pose its median ∆ϕt is 0.51 on the clean recordings and 0.23 on the cluttered ones. Bold: the better of the two methods in each column of a row section. ρenv ↑

PSNR↑

SSIM↑

RMSE↓

∆ϕt ↑

∆ϕt (◦ )↓

HuPR, horizontal single-chip radar (H) RF-Genesis (Chen & Zhang, 2024) 0.649 4DPS (Ours) 0.857

14.0 21.2

0.37 0.58

0.205 0.088

-0.21 0.55

109 63

HuPR, vertical single-chip radar (V) RF-Genesis 0.629 4DPS (Ours) 0.772

14.0 19.0

0.34 0.48

0.203 0.114

0.11 0.44

90 72

RT-Pose, cascade (C), one-person sequences RF-Genesis 0.630 4DPS (Ours) 0.667

15.1 20.0

0.28 0.32

0.179 0.100

-0.05 0.24

103 88

Method

26

Preprint

Table 8: Rendering fidelity on the complex range profile (CRP). Sequences, renders and row sections as in Table 7. ∆ϕr and ∆ϕv are the relative-phase coherences along range and virtual channel (1 is the recording’s phase step, 0 random), each with its weighted RMSE in degrees. The unfit render is not scored. Every column here is scale-free, so RF-Genesis is scored as rendered. Its negative ∆ϕr is a sampling convention: its range profiles step by π per bin against the recording’s, and multiplying them by (−1)k makes the coherence positive and changes no other column; we report it as run. Bold: the better of the two methods in each column of a row section. SSIM↑

RMSE↓

∆ϕr ↑

∆ϕr (◦ )↓

∆ϕv ↑

∆ϕv (◦ )↓

HuPR, horizontal single-chip radar (H) RF-Genesis 0.246 20.0 0.28 4DPS (Ours) 0.856 26.0 0.65

0.102 0.052

-0.63 0.66

147 56

0.52 0.75

67 48

HuPR, vertical single-chip radar (V) RF-Genesis 0.274 20.0 0.19 4DPS (Ours) 0.785 24.8 0.56

0.102 0.061

-0.52 0.55

139 65

0.31 0.47

82 71

RT-Pose, cascade (C), one-person sequences RF-Genesis 0.211 22.8 0.43 0.073 4DPS (Ours) 0.735 28.2 0.76 0.039

-0.54 0.50

137 68

0.16 0.33

94 82

Method

ρ↑

PSNR↑

Table 9: Rendering fidelity on range–angle and range–azimuth–elevation (RAE). Sequences, renders and row sections as in Table 7. PSNR is 20 log10 (peak/RMSE), the only logarithm and applied to a scalar ratio. The unfit render is not scored on the RAE family (–). Restricted to the 57 sequences of the RT-Pose split the fitted row reads 0.835 / 0.451 on range–angle ρ and ρc , so the fitted and unfit sequence sets are comparable. RF-Genesis has no absolute scale, so for SSIM, NRMSE and PSNR its render is multiplied before scoring by one constant per radar that matches its loop-0 range–azimuth energy to the recording’s, the rule that initializes 4DPS’s own gain (App. C.3); ρ and ρc are unchanged by it. Bold: the best value in each column of a row section, by the column’s arrow. Range–angle SSIM↑ NRMSE↓ PSNR↑

ρ↑

SSIM↑ NRMSE↓ PSNR↑

HuPR, horizontal single-chip radar (H) RF-Genesis (Chen & Zhang, 2024) 0.139 0.374 4DPS (unfit) 0.901 0.353 4DPS (Ours) 0.960 0.699

0.51 0.68 0.79

1.08 0.44 0.28

22.5 30.3 34.0

0.165 – 0.942

0.48 – 0.80

1.11 – 0.33

22.7 – 33.3

HuPR, vertical single-chip radar (V) RF-Genesis 0.129 0.317 4DPS (unfit) 0.863 0.234 4DPS (Ours) 0.925 0.622

0.35 0.53 0.67

1.10 0.52 0.38

21.0 27.6 30.2

0.153 – 0.904

0.36 – 0.72

1.13 – 0.40

21.8 – 30.7

RT-Pose, cascade (C), one-person sequences RF-Genesis 0.119 0.221 4DPS (unfit) 0.648 0.181 4DPS (Ours) 0.830 0.470

0.46 0.72 0.78

1.13 0.90 0.53

25.4 27.5 31.8

0.117 – 0.779

0.51 – 0.81

1.11 – 0.58

26.5 – 31.9

Method

ρ↑

ρc ↑

RAE

27

Preprint

Table 10: Rendering fidelity on range–Doppler (RD) and micro-Doppler. Sequences, renders and row sections as in Table 7. PM is the rendered-over-real power ratio on the CFAR-occupied cells, LU and σU the level and spread ratios on the unoccupied ones, ρenv the correlation of the Doppler envelopes and BW their separation, and clDice scores connectivity after independent segmentation of each signature. Restricted to the 57 sequences of the RT-Pose split the fitted row reads 0.695 / 0.968 on RD ρM and micro-Doppler ρS , so the fitted and unfit sequence sets are comparable. RF-Genesis has no absolute scale, so for PM , LU and σU its render is multiplied before scoring by one constant per radar that matches its loop-0 range–azimuth energy to the recording’s, the rule that initializes 4DPS’s own gain (App. C.3); every other column is unchanged by it. Bold: the best value in each column of a row section, by the column’s arrow. RD occupied Method

RD floor

Micro-Doppler

ρM ↑ PM → 1 LU → 1 σU → 1 ρµ ↑ ρσ ↑

ρS ↑

ρenv ↑ BW→ 1 clDice↑

HuPR, horizontal single-chip radar (H) RF-Genesis (Chen & Zhang, 2024) 0.564 4DPS (unfit) 0.584 4DPS (Ours) 0.891

1.51 0.60 0.71

0.06 0.57 0.92

1.30 0.66 0.88

0.82 0.94 1.00

0.08 0.670 0.31 0.771 0.98 0.993

0.28 0.79 0.87

4.22 1.07 0.87

0.34 0.89 0.91

HuPR, vertical single-chip radar (V) RF-Genesis 0.480 4DPS (unfit) 0.513 4DPS (Ours) 0.795

1.12 0.37 0.54

0.05 0.56 0.92

1.24 0.60 0.88

0.82 0.93 0.99

0.03 0.672 0.25 0.740 0.94 0.984

0.28 0.77 0.80

3.79 0.91 0.79

0.37 0.83 0.83

RT-Pose, cascade (C), one-person sequences RF-Genesis 0.324 4DPS (unfit) 0.291 4DPS (Ours) 0.695

0.61 0.28 0.63

0.09 0.58 0.96

1.51 0.75 0.94

0.75 0.80 0.97

0.04 0.394 0.19 0.515 0.70 0.966

0.07 0.46 0.65

1.53 0.60 0.71

0.52 0.59 0.71

28

Preprint

E

D OWNSTREAM R ESULTS IN F ULL

Table 11 gives the full real/rendered train–test matrix, of which Table 3 keeps the rows tested on real data. Tables 12, 13 and 14 carry the downstream tables behind Table 3. Table 12 gives the retention and gap summary (retention is S→R over R→R, the inverse for MPJPE; the information gap is the absolute difference between R→R and S→S, the realism gap between S→S and S→R, and the reverse gap between R→R and R→S, in the dataset’s own unit) beside the real-trained model on a 50/50 real/rendered test set, Table 13 the per-keypoint and per-joint breakdowns with the PCK scores, and Table 14 the RT-Pose scores by activity, location, occlusion and scenario. Read the HuPR per-keypoint rows against the arms’ AP: the arms lose most of what the rendered arm loses (wrists 22.2 and 25.1 real-to-real against 8.0 and 4.7 rendered-to-real), while hips, knees and the head retain 0.73 to 0.80 of their real-to-real value against the arms’ 0.19 to 0.39. The RT-Pose baseline and its labels. HRRadarPose (Ho et al., 2025) publishes no checkpoint; our reproduction on real data reaches 16.7 cm against the 9.9 cm the paper reports, and every RT-Pose number in this paper is measured under that reproduction rather than against the published figure. The released 3D labels stop before the end of each sequence, carry a per-session scale and offset against the radar, and label one sequence with another’s radar, which is why we derive the 15 joints on every radar frame from the registered body mesh that drives the render and exclude sequence 13. HuPR trains at the released sampling ratio for 200 epochs, and RT-Pose’s remaining 63 oneperson sequences are split 44/6/13. Its 37 two-person sequences are fitted against the one person who carries pose annotations and are not scored downstream. Protocol details behind Sec. 4.2. RT-Pose’s 44/6/13 split is stratified by location, scenario, activity and occlusion, so that no split is carried by one room or one action. On rendered test data the released decoder’s confidence threshold is lowered from its released value to −1, so that every joint is kept and a rendered frame is never scored on fewer joints than its real twin; the threshold never triggers on real data, so no real number in the paper depends on it. Table 11: Real and rendered as train and test domains, the full matrix. Table 3 keeps the rows tested on real data. All six metrics are pose-estimation metrics, and every cell is retrained from scratch for four seeds with the standard deviation in parentheses. The first block is the real-only reference, the second is RF-Genesis and the third the same four arms under 4DPS. Rows tested on a renderer measure how much pose information that render carries, not accuracy on real data. RFGenesis’s RT-Pose inputs carry one level gain fixed before training, because its units are arbitrary (App. C.3). RT-Pose (cm ↓)

HuPR (AP ↑)

Train

Test

MPJPE

MRPE

Abs-MPJPE

Real

Real

16.67 (0.42)

13.14 (0.87)

19.07 (0.52)

AP

AP50

AP75

65.6 (1.7) 96.9 (0.2) 78.0 (3.1)

Real RF-Genesis 28.46 (3.55) 28.56 (3.54) 38.15 (6.14) 45.9 (3.9) 97.4 (0.7) 28.5 (9.2) RF-Genesis Real 127.59 (48.32) 106.48 (45.00) 148.88 (24.86) 1.8 (3.5) 8.5 (16.1) 0.0 (0.1) RF-Genesis RF-Genesis 12.44 (0.34) 8.97 (0.74) 12.72 (0.37) 77.0 (0.4) 98.8 (0.1) 93.8 (1.0) Real + RF-Genesis Real 16.82 (0.56) 13.71 (0.88) 19.41 (0.58) 64.1 (0.9) 96.9 (0.0) 76.5 (1.6) Real 4DPS 4DPS Real + 4DPS

4DPS Real 4DPS Real

20.05 (1.50) 21.86 (1.33) 15.71 (1.38) 16.52 (0.80)

14.32 (1.07) 17.06 (0.98) 10.08 (1.00) 14.39 (0.93)

29

22.70 (1.76) 25.45 (1.69) 15.81 (1.30) 19.43 (0.35)

49.2 (0.7) 93.9 (0.8) 45.7 (1.3) 38.4 (2.9) 80.6 (3.2) 30.1 (4.3) 59.6 (1.4) 98.5 (0.1) 67.9 (4.1) 61.6 (0.4) 96.4 (0.6) 72.3 (0.2)

Preprint

Table 12: Downstream transfer summary. Retention and the three gaps are defined in the text and read in the dataset’s own unit (AP ×100 on HuPR, MPJPE in cm on RT-Pose); the last two columns score the real-trained model on a 50/50 real/rendered test set and on the real test set, mean (std) over four seeds. R→R model Dataset

Retention Information gap Realism gap Reverse gap

HuPR (AP, pts) RT-Pose (MPJPE, cm)

0.59 0.76

6.0 1.0

21.2 6.1

16.4 3.4

50/50 test

Real test

55.8 (1.1) 65.6 (1.7) 18.93 (1.44) 16.67 (0.42)

Table 13: Per-keypoint and per-joint downstream accuracy, per arm (train→test; R real, S rendered by 4DPS), same protocol as Table 12: HuPR AP ×100 per keypoint and PCKh@0.2 (top), RT-Pose MPJPE in cm per joint and PCK at 10 and 15 cm (bottom). MPJPE is taken relative to the pelvis, so the pelvis reads 0; PCK on RT-Pose is scored on real test data only (–). Mean (std) over four seeds. HuPR keypoint (AP)

R→R

S→R

R→S

S→S

R+S→R

R Hip R Knee R Ankle L Hip L Knee L Ankle Neck Head L Shoulder L Elbow L Wrist R Shoulder R Elbow R Wrist

90.1 (0.6) 84.5 (0.7) 73.3 (1.7) 90.6 (1.3) 83.7 (2.2) 77.5 (1.3) 83.1 (1.8) 81.4 (1.1) 74.7 (1.2) 41.1 (1.4) 22.2 (1.8) 76.8 (2.3) 50.3 (4.1) 25.1 (3.0)

72.1 (2.9) 65.3 (2.8) 50.2 (2.6) 71.7 (3.3) 61.1 (3.3) 63.0 (2.0) 64.7 (4.1) 63.3 (4.0) 53.3 (4.5) 16.2 (2.6) 8.0 (1.1) 52.7 (3.6) 16.1 (2.7) 4.7 (0.8)

82.7 (1.7) 76.0 (1.3) 73.9 (2.2) 83.0 (1.4) 75.7 (1.0) 66.2 (1.3) 73.2 (1.7) 70.1 (1.0) 58.0 (2.0) 19.5 (1.0) 7.1 (0.5) 63.7 (3.3) 25.2 (0.9) 9.8 (0.7)

91.5 (0.8) 87.7 (0.6) 73.6 (1.3) 90.8 (1.3) 85.2 (2.6) 80.8 (1.0) 82.0 (1.2) 79.9 (1.1) 71.6 (1.2) 27.6 (1.4) 13.3 (0.8) 73.7 (1.7) 36.8 (2.1) 16.1 (2.3)

89.4 (0.2) 83.8 (0.5) 71.8 (1.6) 88.9 (0.6) 82.1 (0.9) 75.0 (1.0) 80.4 (1.7) 78.9 (1.0) 72.4 (0.4) 37.1 (1.3) 18.1 (0.7) 74.1 (0.9) 43.6 (2.4) 19.3 (1.1)

PCKh@0.2

43.6 (1.7) 25.0 (1.8) 27.3 (2.2) 39.2 (2.2) 41.7 (1.6)

RT-Pose joint (MPJPE, cm)

R→R

S→R

Pelvis Right Hip Right Knee Right Ankle Left Hip Left Knee Left Ankle Thorax Head Left Shoulder Left Elbow Left Wrist Right Shoulder Right Elbow Right Wrist

0.0 (0.0) 9.5 (1.0) 14.2 (1.4) 17.3 (0.9) 9.5 (0.9) 13.6 (0.5) 17.0 (0.6) 10.6 (0.8) 14.0 (0.8) 13.2 (1.1) 23.4 (0.4) 32.6 (2.0) 13.1 (0.9) 24.0 (0.8) 38.1 (4.2)

PCK@10cm PCK@15cm

45.8 (2.0) 35.9 (1.5) 64.3 (1.9) 54.0 (1.7)

R→S

S→S

0.0 (0.0) 0.0 (0.0) 0.0 (0.0) 11.4 (0.9) 11.4 (1.5) 9.0 (0.4) 18.6 (0.8) 17.2 (2.4) 20.4 (14.9) 23.0 (1.1) 19.9 (1.3) 15.2 (0.5) 12.1 (0.9) 11.1 (1.7) 9.4 (0.7) 19.7 (2.1) 16.3 (1.4) 13.3 (0.7) 22.0 (1.2) 19.4 (1.2) 15.8 (1.2) 13.8 (0.7) 12.5 (1.2) 10.0 (0.8) 16.4 (1.6) 15.4 (1.4) 13.1 (0.4) 16.6 (0.8) 15.2 (1.5) 12.7 (0.8) 28.5 (1.2) 26.6 (2.0) 21.4 (0.5) 46.7 (6.5) 37.5 (4.5) 29.7 (2.4) 16.3 (0.6) 15.6 (0.9) 12.6 (0.3) 30.0 (2.1) 27.6 (1.8) 21.6 (0.4) 52.7 (7.7) 54.9 (18.8) 31.6 (3.5)

30

– –

– –

R+S→R 0.0 (0.0) 9.2 (0.8) 15.5 (1.1) 18.9 (1.0) 9.0 (0.8) 15.1 (1.2) 18.1 (1.2) 10.0 (1.0) 13.5 (0.2) 13.1 (0.6) 23.1 (0.8) 30.7 (0.8) 13.0 (0.9) 23.9 (1.5) 34.7 (1.3) 44.8 (2.3) 62.1 (2.7)

Preprint

Table 14: RT-Pose MPJPE (cm) by group: the test sequences’ activity, location, occlusion and scenario, for the three arms scored on real test data, same protocol as Table 12; mean (std) over four seeds. RT-Pose MPJPE (cm) by group activity activity activity activity activity location location occlusion occlusion occlusion scenario scenario scenario scenario

Stand and Lift Leg Stand and Wave Hand Walk Walk and Sit Walk and Wave Hand Indoor Outdoor Cardboard Nothing Plastic pad Clean Cluttered Normal Rain

31

R→R

S→R

R+S→R

9.81 (0.20) 17.08 (1.08) 21.67 (2.14) 10.43 (0.51) 14.58 (0.28) 19.40 (0.88) 12.30 (0.63) 25.66 (1.99) 14.62 (1.15) 16.87 (1.86) 16.55 (0.39) 21.11 (1.29) 12.61 (0.77) 11.06 (0.69)

14.45 (1.60) 22.03 (0.81) 25.69 (2.54) 25.28 (1.00) 17.05 (1.31) 23.89 (1.63) 18.60 (0.89) 24.13 (1.74) 20.72 (1.24) 24.68 (2.81) 22.08 (1.32) 24.98 (2.07) 19.67 (1.17) 14.36 (0.26)

9.77 (0.27) 18.66 (1.70) 19.23 (0.43) 10.77 (1.20) 14.18 (0.41) 17.92 (1.09) 14.27 (0.67) 25.31 (0.77) 13.53 (0.73) 21.16 (1.65) 15.30 (0.69) 19.49 (1.33) 15.17 (0.59) 10.69 (1.14)

Preprint

F

R EPRESENTATION A NALYSIS P ROTOCOL

We fixed this protocol, including every measure and the 1.2× margin, before running any comparison, so the analysis is pre-registered; where a measurement departs from it, we say so. HuPR follows RT-Pose’s design and changes only what the dataset forces. The protocol’s pass rule is the point estimate; we report a verdict only when the 95% interval clears the margin. Models and comparisons. We compare 13 checkpoints per dataset: five real-trained seeds and the four rendered- and four mixed-trained seeds (0–3) of Table 3. All arms of a seed start from the same weights, and a seed’s real and rendered arms see their batches in the same order. Table 15 lists the comparisons and what differs within each; every ratio is taken against the real–real comparison of the same row. The different-initialization row is complete and carries the claim. The same-initialization row has no real–real reference of its own, because we trained no second real arm per seed, so it is read against the different-initialization real–real mean, which bounds its ratio from above. An untrained network is the only control. Table 15: The cells compared, with what differs inside each. Ratios are taken against the real–real cell of the matched row. No real arm was trained twice from one initialization, so the first row has no real–real reference and its cross pairs are read against the different-initialization real–real mean, which bounds their ratio from above. Row

Cell

Pairs

Differs in

Reference

same init

realk vs renderedk

4

data

(none trained)

diff. init

reali vs realj reali vs renderedj

10 16

init, order init, order, data

itself real–real, diff. init

beside the design

mixed vs real rendered vs rendered untrained vs real

4, 16 6 5

data (and init) init, order training itself

real–real, diff. init real–real, diff. init real–real, diff. init

Frames and layers. On RT-Pose we use every annotated frame of the 13 real test sequences (3,887 frames) and the same frames of the rendered test split. The layers (taps) are the input (T0), HRNet layer 1 (T1), the parallel branches of stages 2 to 4 (T2a–b, T3a–c, T4a–d; T4a is also the backbone output) and the head’s pre-activation maps (T6). On HuPR a frame stacks eight frames of eight chirps on two boards, so error consistency and paired inputs use all 12,600 test frames, while similarity uses every tenth (1,260 frames, every sequence represented) and the readout is fitted on every tenth frame of the real training split. Its layers are each board’s input (H0), the chirp network’s output (H1), the three encoder levels (H2–H4; H4 feeds the decoder) and the decoder’s pre-sigmoid heatmap (H5); the two boards are never averaged. At every layer we subsample 1,024 spatial positions (fewer where the grid is smaller), drawn once and shared by every frame and model, and treat each position as a row of channels. Measures. Error consistency (Geirhos et al., 2020) is the correlation of per-frame error between two models and their per-joint agreement, where each model’s per-frame error at a joint is split at that model’s own median. On HuPR a keypoint is a hit at a per-keypoint OKS of at least 0.5, and Cohen’s κ over (frame, keypoint) pairs is the primary statistic. Similarity uses debiased linear CKA (Nguyen et al., 2021), computed exactly and read as the distance 1 − CKA in every ratio, and the angular orthogonal Procrustes distance (Williams et al., 2021), because Procrustes catches functional changes CKA misses (Ding et al., 2021). The fully whitened Procrustes stays inside the margin at every layer (1.00 to 1.02) but barely separates an untrained network, so we do not use it. Intervals come from 1,000 bootstrap replicates that resample frames and model pairs. Checks. Seed 0’s real-trained checkpoint re-scores to 16.652 cm against 16.651 in the run behind Table 3 on RT-Pose, and to 0.66871 against 0.66871 AP on HuPR, under the same protocol. A model against itself gives CKA 1 and Procrustes 0 at every layer, the untrained network lies above every real–real band after the input, and the rendered test split repeats the similarity picture layer for layer. Error consistency. The two models do not fail on the same frames. Relative to two real-trained models of different seeds, they disagree about which joints they get right at 1.23 on RT-Pose and 32

Preprint

Table 16: Representational similarity by tap (RT-Pose, real test frames). CKA is a similarity and the Procrustes distance is in radians; ratio is the real–rendered different-init distance over the real– real different-init distance, with its 95% interval, against the pre-registered margin of 1.2. Floor is an untrained network. T0 is the input tensor, identical for every model.

Tap

debiased CKA (similarity ↑) real–real real–rend. floor ratio

Procrustes (distance, rad ↓) real–real real–rend. floor ratio

T0 input T1 layer 1 T2a T2b T3a T3b T3c T4a (= T5) T4b T4c T4d T6 head

1.000 0.713 0.537 0.350 0.228 0.300 0.251 0.069 0.267 0.368 0.332 0.048

0.000 0.816 0.993 1.095 1.223 1.182 1.236 1.357 1.200 1.240 1.254 1.410

1.000 0.613 0.459 0.366 0.200 0.270 0.194 0.085 0.247 0.343 0.266 0.055

1.000 0.417 0.188 0.169 0.071 0.085 0.133 0.039 0.078 0.140 0.111 0.014

– 1.35 [1.21, 1.49] 1.17 [1.01, 1.35] 0.98 [0.89, 1.07] 1.04 [0.94, 1.15] 1.04 [0.92, 1.19] 1.08 [1.05, 1.11] 0.98 [0.92, 1.05] 1.03 [0.96, 1.10] 1.04 [0.97, 1.11] 1.10 [1.03, 1.16] 0.99 [0.92, 1.07]

0.000 0.857 1.038 1.087 1.238 1.202 1.275 1.359 1.222 1.266 1.302 1.385

0.000 1.013 1.213 1.261 1.345 1.338 1.340 1.397 1.353 1.343 1.355 1.478

– 1.05 [1.02, 1.07] 1.04 [1.01, 1.08] 0.99 [0.96, 1.02] 1.01 [0.97, 1.05] 1.02 [0.98, 1.06] 1.03 [1.02, 1.04] 1.00 [0.95, 1.07] 1.02 [0.99, 1.05] 1.02 [1.01, 1.04] 1.04 [1.02, 1.05] 0.98 [0.90, 1.09]

2.94 on HuPR, and their per-frame errors correlate less, at 1.13 and 2.20 (Table 17). Two realtrained models also disagree on frames, which is why each value is a ratio to that disagreement. Behind the ratios, two real-trained seeds correlate at 0.320 in per-frame MPJPE and agree on 0.690 of the per-joint indicators, against 0.234 and 0.618 across domains; on HuPR they agree on 0.936 of the keypoint hits against 0.812, and κ falls from 0.589 to 0.335. On the rendered test split the RT-Pose ratios fall to 1.01 and 1.11, and the mixed arm is indistinguishable from a real seed (ratios 0.88 to 0.99). We report this here rather than in Sec. 4.3 because it concerns which inputs each model gets wrong, not what the models learned. Table 17: Do rendered- and real-trained models fail on the same frames? Error consistency as the real–rendered value over the real–real value between seeds, with its 95% interval (1.0: as consistent as two real-trained models). Bold: the interval lies above the 1.2× margin. The cells behind each ratio follow. Frame error corr.

Joint error agr.

1.13

1.23

RT-Pose

[0.99, 1.31]

[1.21, 1.26]

2.20

2.94

HuPR

[2.07, 2.34]

[2.78, 3.11]

Table 18: Representational similarity by tap (HuPR, real test frames), as Table 16 for RT-Pose. H1 is the chirp network’s output, H4 the tensor the decoder reads and H5 its pre-sigmoid heatmap; h and v are the horizontal and vertical boards. The level-1 and level-2 encoder taps are undefined, because they carry no signal in some checkpoints (App. F, dead temporal merges). The rendered probe reproduces every computable cell to within 0.03.

Tap

debiased CKA (similarity ↑) real–real real–rend. floor ratio

H1h chirp output H2h, H3h, H2v, H3v H4h encoder level 3 H4v encoder level 3 H5 decoder logits

0.976 0.973 0.858 1.15 [0.97, 1.35] undefined: the tap is dead in some checkpoints 0.287 0.229 0.227 1.08 [0.93, 1.27] 0.164 0.084 0.044 1.10 [0.96, 1.30] 0.674 0.554 0.040 1.37 [1.08, 1.70]

Procrustes ratio 1.08 – 1.05 1.15 1.17

Mixed arm and shared initialization. The mixed arm sits between 0.92 and 1.24 and passes 1.2 only in the point estimate at T2a (1.24, interval [1.12, 1.38]). The shared initialization explains little of the agreement: the same-initialization pairs are no closer than the different-initialization ones from stage 2 onward (0.88 to 1.13). 33

Preprint

Linear readout. We freeze each backbone and fit a closed-form ridge readout on real training data to the joint coordinates (RT-Pose: T5 pooled to 4 × 8 × 16; HuPR: H4h and H4v pooled to 4 × 4), with one penalty chosen on held-out real frames and applied to every backbone, and score it with the released evaluator. The registered form, a 1 × 1 × 1 convolution trained under the released losses, converged on no backbone, the real ones included, so we use the plan’s alternative. On RT-Pose the rendered-trained models lose 31% through their own decoders and 16% through the readout (15.62 against 13.46 cm, ratio 1.16, interval [1.09, 1.28]), but an untrained backbone probes at only 1.25, so the test barely discriminates there. On HuPR the real, rendered, mixed and untrained backbones reach 8.98, 11.66, 9.64 and 22.43 px of keypoint error, and the untrained ratio of 2.50 is what gives this test its power on HuPR. Paired inputs. Every rendered frame has a real twin at the same timestamp, so we measure the feature distance between the twins inside one model, in units of the distance between neighboring real frames. To a real-trained model the render’s departure from the recording is worth 2.87 to 3.29 frames of real motion, and it does not shrink with depth. A rendered-trained model sees a smaller gap (2.54 to 3.05), so training on renders makes a model less sensitive to the domain difference without closing it. On HuPR we use all 12,600 test frames rather than the registered tenth, so that a neighbor is one frame away as on RT-Pose; there the twin distance at the chirp network’s output is 0.97 for a real-trained and 0.97 for a rendered-trained model, so the two part only after the encoder. Dead layers on HuPR. MSCSA-PRGCN merges its frames with bias-free convolutions, and training drives some of them to denormal magnitude, so the layer after such a merge carries no signal. Of the 13 checkpoints, H2h and H3h are dead in 1 and 3, and H2v and H3v in 5 and 3. A pair containing a dead layer is undefined, so we report those layers as undefined rather than substitute another.

34

Preprint

G

S CENE E DITING

4DPS fits the static scene and the person as separate explicit components. We can therefore place a fitted person in another fitted scene, or give a fitted scene a motion that was never recorded, and render again without refitting. We test the first edit against real recordings (App. G.1). The second edit has no real recording to compare against, so we show it without a score (App. G.2). G.1

S EPARABILITY OF THE SCENE AND THE PERSON

The test. We ask whether a fitted person keeps its fidelity when the scene around it changes. For two recordings A and B of the same environment, we render B’s complete fit inside A’s fitted static scene. B’s fit is its registered mesh, person material, per-frame velocity field and normals, and joint gains; A’s scene is its points, their orientations and their materials. We then score the render against B’s real recording, on B’s frames, with the evaluation of Table 2. B’s receiver gain, noise floor and antenna coupling stay with B, because they describe B’s capture and not the room. The only other change is the person’s shadow on the scene, which we cast again for A’s scene points with B’s mesh, and nothing is refitted. Rendering B’s fit in B’s own scene through the same code reproduces B’s own render. The pairs. Two recordings count as the same environment when the time-averaged zero-Doppler range–azimuth maps of their real data correlate at 0.75 or more. We correlate the maps in dB over the retained range bins, and on HuPR we take the lower of the two boards. We set this threshold on RT-Pose, where the largest correlation between two recordings of different labeled scenes is 0.74. Each pair comes from one recording session (one subject in one set of clothes), with one sequence standing and the other moving. We take 24 such pairs on RT-Pose and 20 on HuPR, evenly spaced over the candidates, whose static returns correlate at 0.77 and 0.79 or more. Rendering every pair in both directions gives 48 and 40 renders. Result. Table 19 shows that the render in the other scene keeps all of the person’s fidelity and most of the scene’s. The range–Doppler and micro-Doppler columns and LU equal B’s own fit: the median paired difference in micro-Doppler ρS is 0.000 on RT-Pose and 0.000 on HuPR. These columns are not evidence that the scene transfers, however, because they would match even if it did not. The range–Doppler and micro-Doppler products remove the zero-Doppler return, and our renderer is single-bounce, so the static scene reaches them only through the person’s shadow; the noise floor behind LU stays with B. The evidence is in the range–azimuth columns, which the scene dominates. There, A’s scene explains B’s static return at 90 % of B’s own fit on ρ on RT-Pose and 92 % on HuPR (median paired differences −0.055 and −0.079). On the clutter-removed ρc , the share is 90 % on RT-Pose and 62 % on HuPR, and HuPR’s is the largest loss in the table. The render in the other scene still scores above B’s unfit render and RF-Genesis on most renders in every column. The one exception is HuPR’s range–azimuth ρ against the unfit render, where it is higher on 48 % of renders. Fig. 5 shows three pairs per dataset. G.2

N OVEL MOTIONS

Source and placement. Data augmentation for new activities and for falls needs motions that neither dataset contains, so we render such motions in fitted scenes. We take jumping jacks, squats, kicks and one real fall, a rug pulled from under the subject, from the CMU motion-capture set in AMASS (Mahmood et al., 2019). CMU has no clean forward or backward fall, so we generate those two with MDM (Tevet et al., 2023), a text-to-motion model trained on HumanML3D (Guo et al., 2022). MDM’s samples often end in a different posture from their prompt. We therefore drew three samples per prompt and kept the one whose joints do what the prompt says. We judge that by the pelvis’s travel relative to the radar and by where the toes point at the end. Each clip is labeled by that measured outcome. We pose each clip with the body shape fitted to the scene’s own subject and resample it to the radar’s 10 Hz frame rate. We then place it on the scene’s floor at the subject’s real position, facing the radar. No scene point more than 15 cm above the floor may come within 5 cm of the body in any frame. Where the subject’s own position fails that check, we move the start to the nearest point within 1 m that passes, which moved 7 of the 6 × 5 renders by at most 0.9 m. The render then follows App. G.1: the scene, its materials, noise floor and gain, and the subject’s person material come from the fit. The velocities are the clip’s joint kinematics scaled by the subject’s fitted 35

Preprint

Table 19: Separability of the scene and the person. B’s complete fit rendered in A’s fitted static scene (“Separated”), scored against B’s real recording with the evaluation of Table 2, beside B’s own fit, B’s unfit render and RF-Genesis driven by B’s registered motion, on the same renders and frames. Medians over renders; HuPR pools its two boards, as Table 2 does, so n counts (render, board) products. Retention is the median of the separated render over the median of B’s own fit, on the same renders. RF-Genesis’s LU comes from its level-matched scoring, as in Table 2. A column undefined on a render (no frame of the evaluation’s motion subset) has a smaller n than the row’s, which the column shows. range–azimuth range–Doppler

micro-Doppler

ρ

ρc

ρM

LU

ρµ

ρσ

ρS

n

RT-Pose, 24 pairs, both directions Separated (A’s scene, B’s fit) 0.756 B’s own fit 0.843 Retention (%) 90 B’s unfit render 0.660 RF-Genesis 0.131

0.484 0.540 90 0.120 0.221

0.696 0.697 100 0.282 0.335

0.97 0.97 100 0.58 0.09

0.962 0.962 100 0.581 0.711

0.679 0.679 100 0.168 0.068

0.962 0.962 100 0.497 0.401

48 48

HuPR, 20 pairs, both directions Separated (A’s scene, B’s fit) 0.863 B’s own fit 0.942 Retention (%) 92 B’s unfit render 0.884 RF-Genesis 0.131

0.382 0.617 62 0.136 0.161

0.802 0.802 100 0.414 0.410

0.92 0.92 100 0.57 0.05

0.970 0.970 100 0.823 0.689

0.976 0.976 100 0.265 0.019

0.985 0.985 100 0.733 0.648

Person of 188 in scene of 189

Person of 142 in scene of 144

Separated Own fit RF-Genesis

Real

80 80 80 80

Person of 177 in scene of 178

Separated Own fit RF-Genesis

Real

Separated Own fit RF-Genesis

MD

Real

44 48

0.96

0.30

0.96

0.96

0.37

0.97

0.97

0.27

0.74

0.74

0.41

0.71

0.71

0.33

0.63

0.63

0.23

0.74

0.83

0.13

0.69

0.74

0.12

0.73

0.75

0.07

RA

RD

0.96

Person of 56 in scene of 46

Person of 32 in scene of 22

Separated Own fit RF-Genesis

Real

Person of 187 in scene of 186

Separated Own fit RF-Genesis

Real

Separated Own fit RF-Genesis

MD

Real

0.98

0.53

0.98

0.98

0.57

0.98

0.98

0.63

0.81

0.81

0.41

0.83

0.83

0.39

0.78

0.78

0.42

0.86

0.96

0.11

0.93

0.97

0.15

0.94

0.96

0.09

RA

RD

0.98

Figure 5: Separability, RT-Pose (top) and HuPR’s horizontal board (bottom). Each block renders the person of one recording in the fitted scene of another recording of the same environment. Columns: the recording (Real), the person’s complete fit rendered in the other scene (Separated), the person’s own fit, and RF-Genesis driven by the same registered motion. Rows, frame, normalization and scores are those of Fig. 3. We chose the pairs by a rule: the renders at the 25th, 50th and 75th percentile of the mean ratio of the separated render’s three panel scores to the own fit’s.

36

Preprint

RT-Pose, indoor (cluttered), fitted scene of sequence 142: motions with no real recording (not scored) fall, backward

fall, forward (face down)

MD

RD

RA

MD

RD

RA

MD

squats

jumping jacks

MD

RD

fall, rug pulled

RA

MD

RD

RD

RA

kicks

RA

MD

RD

RA

Figure 6: Novel motions in the fitted scene of RT-Pose sequence 142 (indoor, cluttered) (not scored). Per motion: the posed mesh at four evenly spaced frames, seen from 45◦ between the radar’s view and the side, above the rendered micro-Doppler over the whole clip (time across, radial velocity up) and the rendered range–Doppler and range–azimuth at the clip’s frame of largest pelvis speed (range across, over the person’s range ±2 m). Panels are linear magnitude normalized to their own range, as in Fig. 3. Falls: forward and backward generated by MDM (Tevet et al., 2023), the fall from a pulled rug from CMU (Mahmood et al., 2019); exercises from CMU. per-joint factors. A motion without a recording has no per-frame velocity correction to fit, so these renders carry the motion fidelity of a render without the velocity fit. What the figures show. Figs. 6–10 render 6 motions in each of 5 fitted scenes. These motions have no real recording, so we cannot score them. We show them as a capability for data augmentation, not as a result.

37

Preprint

RT-Pose, outdoor, fitted scene of sequence 159: motions with no real recording (not scored) fall, backward

fall, forward (face down)

MD

RD

RA

MD

RD

RA

MD

squats

jumping jacks

MD

RD

fall, rug pulled

RA

MD

RD

RD

RA

kicks

RA

MD

RD

RA

Figure 7: Novel motions in the fitted scene of RT-Pose sequence 159 (outdoor) (not scored). Per motion: the posed mesh at four evenly spaced frames, seen from 45◦ between the radar’s view and the side, above the rendered micro-Doppler over the whole clip (time across, radial velocity up) and the rendered range–Doppler and range–azimuth at the clip’s frame of largest pelvis speed (range across, over the person’s range ±2 m). Panels are linear magnitude normalized to their own range, as in Fig. 3. Falls: forward and backward generated by MDM (Tevet et al., 2023), the fall from a pulled rug from CMU (Mahmood et al., 2019); exercises from CMU.

RT-Pose, indoor (clean), fitted scene of sequence 186: motions with no real recording (not scored) fall, backward

fall, forward (face down)

MD

RD

RA

MD

RD

RA

MD

squats

jumping jacks

MD

RD

fall, rug pulled

RA

MD

RD

RD

RA

kicks

RA

MD

RD

RA

Figure 8: Novel motions in the fitted scene of RT-Pose sequence 186 (indoor, clean) (not scored). Per motion: the posed mesh at four evenly spaced frames, seen from 45◦ between the radar’s view and the side, above the rendered micro-Doppler over the whole clip (time across, radial velocity up) and the rendered range–Doppler and range–azimuth at the clip’s frame of largest pelvis speed (range across, over the person’s range ±2 m). Panels are linear magnitude normalized to their own range, as in Fig. 3. Falls: forward and backward generated by MDM (Tevet et al., 2023), the fall from a pulled rug from CMU (Mahmood et al., 2019); exercises from CMU.

38

Preprint

HuPR, fitted scene of sequence 183: motions with no real recording (not scored) fall, backward

fall, forward (face down)

MD

RD

RA

MD

RD

RA

MD

RD

squats

jumping jacks

MD

RD

fall, rug pulled

RA

MD

RD

RA

kicks

RA

MD

RD

RA

Figure 9: Novel motions in the fitted scene of HuPR sequence 183 (horizontal board) (not scored). Per motion: the posed mesh at four evenly spaced frames, seen from 45◦ between the radar’s view and the side, above the rendered micro-Doppler over the whole clip (time across, radial velocity up) and the rendered range–Doppler and range–azimuth at the clip’s frame of largest pelvis speed (range across, over the person’s range ±2 m). Panels are linear magnitude normalized to their own range, as in Fig. 3. Falls: forward and backward generated by MDM (Tevet et al., 2023), the fall from a pulled rug from CMU (Mahmood et al., 2019); exercises from CMU.

HuPR, fitted scene of sequence 41: motions with no real recording (not scored) fall, backward

fall, forward (face down)

MD

RD

RA

MD

RD

RA

MD

squats

jumping jacks

MD

RD

fall, rug pulled

RA

MD

RD

RD

RA

kicks

RA

MD

RD

RA

Figure 10: Novel motions in the fitted scene of HuPR sequence 41 (horizontal board) (not scored). Per motion: the posed mesh at four evenly spaced frames, seen from 45◦ between the radar’s view and the side, above the rendered micro-Doppler over the whole clip (time across, radial velocity up) and the rendered range–Doppler and range–azimuth at the clip’s frame of largest pelvis speed (range across, over the person’s range ±2 m). Panels are linear magnitude normalized to their own range, as in Fig. 3. Falls: forward and backward generated by MDM (Tevet et al., 2023), the fall from a pulled rug from CMU (Mahmood et al., 2019); exercises from CMU.

39

Preprint

H

A BLATIONS , RUNTIME , AND P EAK M EMORY

H.1

A BLATIONS

Is every term of the objective necessary? Four terms at unit weight invite the question. We therefore remove each global term alone and the two windowed terms together. The windowed pair shares one window and one scope, the person’s box, so we remove it as one unit. We expect the global loop-0 term to matter most. The plane terms exclude DC and the windowed pair reaches only the person’s box, so the global loop-0 term is the only term that reaches the whole static scene. Each variant multiplies the removed terms by zero inside the loss and re-fits the renderer once per sequence, with everything else at the default configuration (the same epochs, person inputs, scaffolds and seed). The “None” row is not a re-fit. It is the render Table 2 reports, and it reproduces Table 2 and Tables 7– 10 cell for cell. We judge each variant twice. Reconstruction is scored over Table 2’s sequences (HuPR 235 per board, RT-Pose 63 one-person). The gap to real, rendered-to-real minus real-to-real paired per sequence, is read through the four real-trained seeds that Table 3 reports, and only on the test splits (HuPR 21, RT-Pose 13 sequences), because the real-trained models are in-sample on every other sequence. We also count, per sequence and radar, how often a variant scores worse or better than the “None” row by more than a threshold. The threshold is 0.01 on bounded scores (correlations, SSIM, clDice and the phase coherences), 2% of the “None” value on the others, and 1 cm or one AP point on the gap. We fit one renderer seed, because sequences rather than seeds are the axis we generalize over. Only the medians and the paired counts are therefore interpretable, never a single sequence. Every term carries part of the fit, but not the same part. Table 20 summarizes the three variants, and Tables 21–24 report them on every metric of Tables 7–10. The global loop-0 term carries the static scene, the global per-plane term carries the noise floor and the Doppler structure, and the windowed pair carries a smaller share on the person’s cells. None of the three can be removed without a measurable cost somewhere in that metric set. The global loop-0 term carries the static scene. Without it, the median of every magnitude column of every representation falls on every radar. Range–azimuth ρ falls from 0.960 to 0.646 on H and from 0.830 to 0.141 on C, worse on 235 of 235 and 63 of 63 sequences. The CRP ρ falls from 0.856 to 0.667 on H and from 0.735 to 0.227 on C, again worse on 235 of 235 and 63 of 63. The columns scored after static clutter removal do not move. RD ρM reads 0.697 against 0.695 on C and microDoppler ρS 0.968 against 0.966, and the floor ratios match the “None” row. The two gaps differ. RT-Pose’s gap grows from 2.00 to 174.26 cm, worse on 13 of 13 test sequences, while HuPR’s reads 16.5 against 16.4 AP (9 worse, 5 better). HuPR’s released preprocessing removes static clutter over loops before its model reads the heatmaps, so the static scene this term fits never reaches the HuPR model, while HRRadarPose reads it. The global per-plane term carries the noise floor and the Doppler structure. Without it, the floor level LU falls on every radar, from 0.92 to 0.79 on H (worse on 213 of 235), from 0.92 to 0.80 on V and from 0.96 to 0.86 on C. The floor spread σU on C moves away from 1, from 0.94 to 1.33. RD ρM falls on every radar, worse on 91 of 235 sequences on H and 53 of 63 on C, and RT-Pose’s micro-Doppler ρS falls from 0.966 to 0.957 (worse on 29 of 61). SSIM on the CRP and range–azimuth maps also falls on both HuPR boards, worse on 188 and 177 of 235 sequences on H. The geometry does not move. Range–azimuth ρ reads 0.838 on C, and neither gap separates from the “None” row (HuPR 5 worse and 9 better, RT-Pose 4 and 2). The term therefore fits the floor and the Doppler planes, not the geometry. The windowed pair has the smallest effect, on the person’s cells in the scene. Without it, RTPose’s clutter-removed range–azimuth ρc falls from 0.470 to 0.444, worse on 51 of 61 sequences, and its RAE ρ from 0.779 to 0.762. HuPR’s gap widens from 16.4 to 17.4 AP (12 test sequences worse, 2 better), while its reconstruction stays at the “None” row, with range–azimuth ρ at 0.960 on H. RT-Pose’s gap does not separate (2 worse, 2 better). The effect is the smallest of the three, and it rests on paired counts from one renderer seed. The phase-coherence columns (∆ϕt , ∆ϕr , ∆ϕv ) do not separate the variants in a consistent direction, so we do not read them as evidence for or against any term. 40

Preprint

Table 20: Objective ablation, summary. Each row removes the named terms of Eq. (4) and re-fits the renderer, with everything else at the default configuration (setup in App. H.1). “None” is the render Table 2 reports, not a re-fit. ρ is the median range–azimuth correlation and ρS the median micro-Doppler spectrogram correlation, over Table 2’s sequences (HuPR both boards pooled). ∆ is the median per-sequence gap to real on the test splits, read through the real-trained seeds of Table 3. On HuPR it is real-to-real minus rendered-to-real AP, and on RT-Pose rendered-to-real minus realto-real MPJPE and MRPE in cm, so a larger ∆ is a worse render. W/B counts the test sequences whose ∆ is worse / better than the “None” row’s by more than one AP point or 1 cm (MPJPE). One renderer seed. Tables 21–24 give every metric per radar. HuPR

RT-Pose

Term removed

ρ↑

None (full objective)

0.940 0.988

ρS ↑ ∆ (AP)↓ W/B 16.4

–

0.830 0.966

ρ↑

ρS ↑ ∆ MPJPE↓ ∆ MRPE↓ W/B 2.00

0.47

–

−LRAE,0 (global, loop 0) 0.485 0.989 −LRAE,pl (global, per plane) 0.941 0.986 win −(Lwin RAE,0 + LRAE,pl ) (windowed) 0.941 0.989

16.5 16.7 17.4

9/5 0.141 0.968 5/9 0.838 0.957 12/2 0.828 0.970

174.26 2.30 1.08

262.43 1.21 0.88

13/0 4/2 2/2

Table 21: Objective ablation on the ADC samples. Columns as Table 7 (ρenv the envelope correlation after the rank-1 phase correction, ∆ϕt the fast-time relative-phase coherence with its weighted RMSE in degrees); rows as Table 20, over the same sequences (HuPR 235 per board, RT-Pose 63 one-person). Bold: the best value in each column of a row section, by the column’s arrow. PSNR↑

SSIM↑

RMSE↓

∆ϕt ↑

∆ϕt (◦ )↓

HuPR, horizontal single-chip radar (H) None (full objective) 0.857 21.2 −LRAE,0 0.742 18.4 −LRAE,pl 0.856 21.2 win −(Lwin + L ) 0.856 21.2 RAE,0 RAE,pl

0.58 0.45 0.58 0.58

0.088 0.121 0.088 0.088

0.55 0.63 0.54 0.55

63 56 64 64

HuPR, vertical single-chip radar (V) None (full objective) 0.772 19.0 −LRAE,0 0.668 16.1 −LRAE,pl 0.773 19.0 win −(Lwin 0.776 19.0 RAE,0 + LRAE,pl )

0.48 0.35 0.48 0.48

0.114 0.157 0.113 0.114

0.44 0.56 0.45 0.45

72 62 72 72

RT-Pose, cascade (C), one-person sequences None (full objective) 0.667 20.0 −LRAE,0 0.596 18.2 −LRAE,pl 0.668 20.2 win −(Lwin 0.658 20.0 RAE,0 + LRAE,pl )

0.32 0.26 0.33 0.31

0.100 0.123 0.098 0.101

0.24 0.27 0.24 0.23

88 85 87 89

Term removed

ρenv ↑

41

Preprint

Table 22: Objective ablation on the complex range profile (CRP). Columns as Table 8 (∆ϕr and ∆ϕv the relative-phase coherences along range and virtual channel, each with its weighted RMSE in degrees); rows, sequences and bold as Table 21. PSNR↑

SSIM↑

RMSE↓

∆ϕr ↑

∆ϕr (◦ )↓

∆ϕv ↑

∆ϕv (◦ )↓

HuPR, horizontal single-chip radar (H) None (full objective) 0.856 26.0 −LRAE,0 0.667 22.4 −LRAE,pl 0.850 25.9 win −(Lwin 0.854 26.0 RAE,0 + LRAE,pl )

0.65 0.44 0.62 0.65

0.052 0.076 0.052 0.052

0.66 0.64 0.65 0.64

56 58 56 57

0.75 0.66 0.75 0.75

48 57 48 48

HuPR, vertical single-chip radar (V) None (full objective) 0.785 24.8 −LRAE,0 0.491 20.6 −LRAE,pl 0.789 24.8 win 0.788 24.8 −(Lwin RAE,0 + LRAE,pl )

0.56 0.26 0.52 0.56

0.061 0.095 0.061 0.060

0.55 0.57 0.55 0.55

65 63 65 65

0.47 0.49 0.48 0.48

71 69 71 71

RT-Pose, cascade (C), one-person sequences None (full objective) 0.735 28.2 −LRAE,0 0.227 23.7 −LRAE,pl 0.739 28.4 win 0.727 27.9 −(Lwin RAE,0 + LRAE,pl )

0.76 0.52 0.75 0.73

0.039 0.066 0.038 0.041

0.50 0.53 0.51 0.51

68 66 68 69

0.33 0.27 0.33 0.33

82 85 82 82

Term removed

ρ↑

Table 23: Objective ablation on range–angle and range–azimuth–elevation (RAE). Columns as Table 9; rows, sequences and bold as Table 21. Every row is a 4DPS render, which carries an absolute scale, so SSIM, NRMSE and PSNR are scored directly, without the level matching that RF-Genesis needs. Range–angle SSIM↑ NRMSE↓ PSNR↑

ρ↑

SSIM↑ NRMSE↓ PSNR↑

HuPR, horizontal single-chip radar (H) None (full objective) 0.960 0.699 −LRAE,0 0.646 0.456 −LRAE,pl 0.960 0.696 win −(Lwin RAE,0 + LRAE,pl ) 0.960 0.690

0.79 0.67 0.77 0.79

0.28 1.60 0.28 0.28

34.0 19.0 34.0 34.1

0.942 0.628 0.941 0.942

0.80 0.66 0.78 0.80

0.33 1.87 0.33 0.32

33.3 18.7 33.2 33.3

HuPR, vertical single-chip radar (V) None (full objective) 0.925 0.622 −LRAE,0 0.337 0.126 −LRAE,pl 0.926 0.618 win −(Lwin RAE,0 + LRAE,pl ) 0.925 0.615

0.67 0.56 0.65 0.67

0.38 10.23 0.38 0.38

30.2 1.6 30.2 30.2

0.904 0.349 0.905 0.904

0.72 0.56 0.70 0.72

0.40 11.71 0.40 0.40

30.7 1.7 30.7 30.8

RT-Pose, cascade (C), one-person sequences None (full objective) 0.830 0.470 0.78 −LRAE,0 0.141 0.166 0.65 −LRAE,pl 0.838 0.473 0.77 win −(Lwin 0.77 RAE,0 + LRAE,pl ) 0.828 0.444

0.53 7.79 0.52 0.53

31.8 9.9 31.9 31.7

0.779 0.125 0.785 0.762

0.81 0.65 0.80 0.81

0.58 7.08 0.58 0.59

31.9 11.7 31.9 31.9

Term removed

ρ↑

ρc ↑

RAE

42

Preprint

Table 24: Objective ablation on range–Doppler (RD) and micro-Doppler. Columns as Table 10 (PM the rendered-over-real power ratio on the CFAR-occupied cells, LU and σU the level and spread ratios on the unoccupied ones, ρenv the Doppler-envelope correlation, BW the envelope separation, clDice the connectivity of the segmented signatures); rows, sequences and bold as Table 21. Every row is a 4DPS render, which carries an absolute scale, so PM , LU and σU are scored directly, without the level matching that RF-Genesis needs. RD occupied

RD floor

Micro-Doppler

ρM ↑ PM → 1 LU → 1 σU → 1 ρµ ↑ ρσ ↑

Term removed

ρS ↑

ρenv ↑ BW→ 1 clDice↑

0.98 0.98 0.98 0.98

0.993 0.993 0.993 0.994

0.87 0.87 0.87 0.88

0.87 0.87 0.87 0.88

0.91 0.91 0.90 0.91

0.99 0.99 0.99 0.99

0.94 0.94 0.94 0.95

0.984 0.984 0.982 0.985

0.80 0.81 0.80 0.81

0.79 0.79 0.78 0.79

0.83 0.83 0.83 0.83

0.97 0.97 0.96 0.98

0.70 0.69 0.69 0.71

0.966 0.968 0.957 0.970

0.65 0.63 0.67 0.63

0.71 0.71 0.72 0.71

0.71 0.71 0.70 0.71

HuPR, horizontal single-chip radar (H) None (full objective) 0.891 0.71 −LRAE,0 0.892 0.70 −LRAE,pl 0.878 0.69 win −(Lwin 0.71 RAE,0 + LRAE,pl ) 0.894

0.92 0.92 0.79 0.92

0.88 0.88 0.94 0.88

1.00 1.00 1.00 1.00

HuPR, vertical single-chip radar (V) None (full objective) 0.795 0.54 −LRAE,0 0.796 0.54 −LRAE,pl 0.781 0.54 win 0.54 −(Lwin RAE,0 + LRAE,pl ) 0.800

0.92 0.92 0.80 0.92

0.88 0.88 1.07 0.88

RT-Pose, cascade (C), one-person sequences None (full objective) 0.695 0.63 −LRAE,0 0.697 0.63 −LRAE,pl 0.677 0.64 win 0.60 −(Lwin RAE,0 + LRAE,pl ) 0.695

0.96 0.96 0.86 0.95

0.94 0.94 1.33 0.93

H.2

RUNTIME AND P EAK M EMORY

This subsection reports what the two experiments cost in time and in GPU memory: reconstructing every sequence (Sec. 4.1) with 4DPS, its unfit render and RF-Genesis, and training the downstream models (Sec. 4.2). Every timing in Tables 25 and 26 is a wall time from the run that produced the paper’s numbers, one sequence or one training per GPU, for 4DPS and for RF-Genesis alike. The 4DPS fits, the unfit renders and every downstream training, including those on RF-Genesis’s renders, ran on NVIDIA H100 GPUs. A 4DPS fit’s time runs from the person stage through the 400-epoch velocity and material fit to the render and its evaluation, and excludes the scene preprocessing it inherits (App. B.1). The unfit render runs the same stages with no fit. RF-Genesis’s time covers its mesh export, its ray-cast simulation, the representations computed from the simulated cube and their scoring against the recording. Table 25: Reconstruction cost (Sec. 4.1): the median wall time per sequence and the total over the corpus. RT-Pose’s unfit render covers the 57 split sequences it is scored on. The 4DPS step is the median over sequences of the training step’s median. RF-Genesis ran on RTX 4090s and 4DPS on H100s, so the two methods’ times are not a like-for-like speed comparison. Data

Method

HuPR

4DPS H100 4DPS (unfit) H100 RF-Genesis RTX 4090

235 235 235

5.7 1.5 5.3

22.3 5.9 21.0

19.4 – –

RT-Pose 4DPS H100 4DPS (unfit) H100 RF-Genesis RTX 4090

100 57 100

12.9 1.9 6.0

21.6 1.9 10.6

19.3 – –

GPU

Sequences Per sequence (min) Total (GPU-h) Step (ms)

Downstream training (Sec. 4.2). Table 26 gives the cost of one training per H100 at the protocol’s fixed schedule, for every arm of Table 3. The real arm is shared: the RF-Genesis arms were compared against the same four real trainings, which were not retrained. A training’s time is the sum over epochs of its training time, plus the per-epoch validation on HuPR. Peak memory. Table 27 gives the peak GPU memory of each stage, and its caption says which counter each cell reads. The largest 4DPS fit allocates 19.6 GiB at its peak. 43

Preprint

Table 26: Downstream training cost (Sec. 4.2): hours per training, mean (std) over the four seeds, and the total over the four seeds. Every training ran on one H100. Evaluating Table 3’s five cells over the four seeds adds 0.5 (4DPS) and 0.5 (RF-Genesis) GPU-h on HuPR, and 0.7 and 0.7 on RT-Pose. HuPR

RT-Pose

Arm

Renderer

GPU

h / training Epochs GPU-h h / training Epochs GPU-h

Real Rendered Mixed Rendered Mixed

none (real) 4DPS 4DPS RF-Genesis RF-Genesis

H100 H100 H100 H100 H100

6.8 (0.1) 6.7 (0.0) 6.7 (0.0) 6.7 (0.0) 6.7 (0.0)

200 200 200 200 200

27.1 26.8 26.9 26.7 26.7

6.5 (0.1) 8.4 (0.0) 8.2 (0.2) 7.7 (0.1) 7.9 (0.3)

50 50 50 50 50

26.2 33.7 32.9 31.0 31.5

Table 27: Peak GPU memory (GiB): the median and the maximum over sequences for reconstruction, and the maximum over the four seeds for training. Every 4DPS cell and every training cell is PyTorch’s allocator peak (max memory allocated), as the 4DPS runs record it per sequence and RT-Pose’s trainer logs it; the reserved pool is larger. The RF-Genesis reconstruction cells are instead the card-level peak from nvidia-smi, because its ray tracer allocates memory outside PyTorch (PyTorch alone records 1.1 and 1.6 GiB), so they are not the same quantity as the 4DPS cells. The cells marked † were measured separately from the runs that produced the paper’s numbers, with the same code, protocol and data: RF-Genesis as the maximum over the shortest, median and longest sequence of each dataset on an RTX 4090, and HuPR training over 300 steps and one validation pass per arm on an H100; the 5 HuPR arms are identical (21.9 GiB at card level). HuPR Stage

Method / arm

median

max

median

max

Reconstruction 4DPS (Sec. 4.1) RF-Genesis

H100 RTX 4090

5.7 –

6.7 3.3†

18.7 –

19.6 10.8†

Training (Sec. 4.2)

H100 H100 H100 H100 H100

– – – – –

14.9† 14.9† 14.9† 14.9† 14.9†

– – – – –

11.2 11.2 11.2 11.2 11.2

real 4DPS rendered 4DPS mixed RF-Genesis rendered RF-Genesis mixed

GPU

RT-Pose

44

Preprint

The training step. Three decisions make a corpus-scale fit affordable. First, the step has no datadependent shape and no host synchronization: the path count is fixed per step and the loss chain and frames are batched, so the whole training step is captured once as a CUDA graph and replayed. Second, the forward model runs in fused CUDA kernels with hand-written backward passes, and the Doppler-plane amplitudes are evaluated in closed form (Eq. (16)). Third, everything in Eq. (3) that does not read a normal or a material is cached per frame, for the scene and for the person’s paths, and the person’s paths are sorted once per frame by their target cell, so each accumulator cell is one contiguous run summed without atomics.

45

Preprint

I

L IMITATIONS

Single-bounce transport, and one mover. As in prior radar splatting (Armouti et al., 2026a; Kung et al., 2025), we render the single-bounce path from each transmitter to a point and back, so the returns that reach a receiver only after reflecting off a second surface are absent from every product. Only the subject moves: the scene is fitted once per sequence and held static, and one person is modeled, because RT-Pose annotates the pose of only one person in its multi-person sequences. Its 37 two-person sequences are therefore fitted against the annotated person alone, and the paper reports results on the one-person sequences. Motion and labels come from the video’s body model. Each point’s velocity stays within a bounded residual of the kinematics the monocular body model supplies, and the skeleton comes from that model, never from a radar label. The fit therefore corrects the video’s motion rather than recovering it independently: a limb the body model misplaces beyond the bound is a limb the radar term cannot invent, and the bound is what keeps the Doppler fit from absorbing scaffold error as spurious velocity. RT-Pose’s labels are derived on every radar frame from the same registered mesh that drives the render, with the released-label result reported beside them (App. E). Elevation is the weakest axis. The scaffold reconstructs elevation least well of the three spatial axes, because it is built from a monocular depth or LiDAR scene and a body model rather than measured by the radar. The products that resolve elevation are therefore the harder ones, which is why HuPR’s vertical board and the RT-Pose cascade score below HuPR’s horizontal board throughout Sec. 4.1. Two of HuPR’s encoder taps are also undefined in the representation analysis, because the temporal merges feeding them are denormal in some trained checkpoints (App. F). No fitted baseline exists to compare against. No public radar simulator takes a capture as input, so RF-Genesis is driven forward from the registered motion rather than fitted to the recording we score against. The comparison that carries Sec. 4.1 is therefore against our own unfit render, which isolates what the radar fit adds over the camera but cannot show what a competing fitted renderer would achieve. The survey in App. C.2 records that this is a property of the field rather than of our selection. What the equivalence measurements cannot settle. An equivalence claim needs a control that detects a difference of a known size, and the only one we trained is the untrained network. It separates cleanly at the shallowest taps and barely at all at the backbone output and the head, so the pre-registered margin means less the deeper the tap, and we read the two deepest taps as degenerate rather than as passes. We also did not train a second real arm per seed from a shared initialization, so the same-initialization row is read against the different-initialization real–real mean rather than against a reference of its own (App. F). Rendered features transfer better on one dataset than the other. Freezing each backbone and fitting a fresh linear head on real data recovers most of the rendered arm’s deficit on RT-Pose and about half of it on HuPR (Sec. 4.3). We report the asymmetry rather than explain it: the two datasets differ in radar geometry, in downstream architecture and in label provenance at once, so this evaluation cannot attribute the difference to any one of them. Most edits are not scored. Fitting an explicit scene is what makes translation, retiming and recomposition possible. One edit, rendering a fitted person in another fitted scene of the same environment, is evaluated in App. G.1; translation, retiming and new motions are not scored, because no real recording exists to compare them against. A corpus of edited sequences is the next step.

J

A DDITIONAL Q UALITATIVE R ESULTS

Figs. 11–16 show 18 sequences per dataset, six per figure, chosen by a rule fixed before rendering so that they are typical rather than the best: we rank the corpus of Table 2 (HuPR on its horizontal board) by 4DPS fidelity (the mean range–azimuth, range–Doppler and micro-Doppler correlation), keep Fig. 3’s three RT-Pose sequences and the three HuPR test sequences ranked the same way, and add 15 at evenly spaced quantiles of the rest. Columns, frames, normalization, scores and the first three rows follow Fig. 3; the others are range–azimuth–elevation (azimuth × elevation, maximum over the person’s range ±10 bins), the complex range profile (range × virtual channel) and ADC samples (fast-time sample × channel). An undefined score prints as –. 46

Preprint

4DPS

Sequence 147 RF-Genesis

Real

4DPS

Sequence 166 RF-Genesis

Real

4DPS

RF-Genesis

0.97

0.46

0.97

0.52

0.98

0.40

0.78

0.30

0.83

0.44

0.72

0.32

0.86

0.17

0.88

0.22

0.87

0.09

0.78

0.17

0.82

0.23

0.80

0.08

0.72

0.33

0.74

0.42

0.80

0.18

0.66

0.63

0.68

0.63

0.73

0.64

ADC

CRP

RAE

RA

RD

micro-Doppler

Sequence 159 Real

4DPS

Sequence 148 RF-Genesis

Real

4DPS

Sequence 160 RF-Genesis

Real

4DPS

RF-Genesis

0.98

0.36

0.96

0.39

0.97

0.46

0.80

0.38

0.73

0.30

0.75

0.33

0.87

0.28

0.89

0.06

0.86

0.16

0.80

0.29

0.82

0.05

0.78

0.14

0.78

0.50

0.69

0.17

0.72

0.28

0.70

0.64

0.61

0.62

0.66

0.62

ADC

CRP

RAE

RA

RD

micro-Doppler

Sequence 180 Real

Figure 11: Qualitative gallery, RT-Pose (1 of 3): sequences 159, 147 and 166 (top) and 180, 148 and 160 (bottom) (4DPS-fidelity ranks 8, 1, 16, 4, 9 and 13 of 63). The top three are the sequences of Fig. 3. Selection rule, rows, columns, frames and scores are given at the start of App. J.

47

Preprint

4DPS

Sequence 183 RF-Genesis

Real

4DPS

Sequence 179 RF-Genesis

Real

4DPS

RF-Genesis

0.98

0.46

0.97

0.38

0.97

0.52

0.80

0.36

0.71

0.41

0.77

0.43

0.77

0.16

0.85

0.13

0.76

0.10

0.75

0.16

0.81

0.13

0.72

0.11

0.75

0.24

0.80

0.24

0.73

0.19

0.68

0.64

0.72

0.63

0.64

0.64

ADC

CRP

RAE

RA

RD

micro-Doppler

Sequence 144 Real

4DPS

Sequence 110 RF-Genesis

Real

4DPS

Sequence 116 RF-Genesis

Real

4DPS

RF-Genesis

0.95

0.43

0.95

0.51

0.99

0.26

0.77

0.36

0.69

0.34

0.62

0.18

0.75

0.15

0.81

0.11

0.81

0.20

0.73

0.14

0.73

0.10

0.73

0.19

0.71

0.24

0.71

0.19

0.67

0.37

0.62

0.62

0.66

0.63

0.67

0.62

ADC

CRP

RAE

RA

RD

micro-Doppler

Sequence 152 Real

Figure 12: Qualitative gallery, RT-Pose (2 of 3): sequences 144, 183 and 179 (top) and 152, 110 and 116 (bottom) (4DPS-fidelity ranks 18, 22, 26, 30, 34 and 38 of 63). As Fig. 11.

48

Preprint

4DPS

Sequence 113 RF-Genesis

Real

4DPS

Sequence 178 RF-Genesis

Real

4DPS

RF-Genesis

0.94

0.43

0.94

0.40

0.96

0.46

0.70

0.20

0.62

0.21

0.66

0.36

0.76

0.16

0.81

0.11

0.73

0.11

0.74

0.16

0.74

0.10

0.74

0.12

0.72

0.23

0.71

0.20

0.73

0.20

0.68

0.63

0.69

0.63

0.63

0.64

ADC

CRP

RAE

RA

RD

micro-Doppler

Sequence 151 Real

4DPS

Sequence 10 RF-Genesis

Real

4DPS

Sequence 108 RF-Genesis

Real

4DPS

RF-Genesis

0.96

0.31

0.96

0.43

--

--

0.73

0.32

0.64

0.27

0.17

-0.24

0.58

0.07

0.64

0.08

0.81

0.13

0.63

0.07

0.68

0.08

0.75

0.12

0.74

0.12

0.72

0.14

0.72

0.21

0.65

0.66

0.63

0.62

0.69

0.65

ADC

CRP

RAE

RA

RD

micro-Doppler

Sequence 1 Real

Figure 13: Qualitative gallery, RT-Pose (3 of 3): sequences 151, 113 and 178 (top) and 1, 10 and 108 (bottom) (4DPS-fidelity ranks 42, 46, 50, 54, 58 and 62 of 63). As Fig. 11.

49

Preprint

4DPS

Sequence 41 RF-Genesis

Real

4DPS

Sequence 248 RF-Genesis

Real

4DPS

RF-Genesis

1.00

0.65

1.00

0.68

1.00

0.66

0.93

0.65

0.94

0.64

0.93

0.67

0.96

0.15

0.94

0.16

0.96

0.17

0.94

0.18

0.92

0.18

0.94

0.20

0.81

0.30

0.86

0.27

0.84

0.32

0.80

0.65

0.87

0.66

0.82

0.66

ADC

CRP

RAE

RA

RD

micro-Doppler

Sequence 247 Real

4DPS

Sequence 172 RF-Genesis

Real

4DPS

Sequence 164 RF-Genesis

Real

4DPS

RF-Genesis

1.00

0.75

1.00

0.74

0.99

0.70

0.94

0.70

0.93

0.69

0.92

0.63

0.96

0.13

0.96

0.13

0.96

0.12

0.95

0.15

0.94

0.15

0.94

0.14

0.88

0.22

0.86

0.22

0.87

0.21

0.88

0.64

0.85

0.65

0.87

0.64

ADC

CRP

RAE

RA

RD

micro-Doppler

Sequence 192 Real

Figure 14: Qualitative gallery, HuPR, horizontal board (1 of 3): sequences 247, 41 and 248 (top) and 192, 172 and 164 (bottom) (4DPS-fidelity ranks 11, 57, 13, 8, 26 and 41 of 235). The top three are the HuPR test sequences ranked like those of Fig. 3 (4DPS fidelity plus rendered-to-real OKS). A single board resolves elevation poorly, so the range–azimuth–elevation rows are coarse along that axis.

50

Preprint

4DPS

Sequence 30 RF-Genesis

Real

4DPS

Sequence 96 RF-Genesis

Real

4DPS

RF-Genesis

1.00

0.71

0.99

0.76

0.99

0.70

0.92

0.63

0.91

0.62

0.90

0.65

0.95

0.12

0.96

0.22

0.96

0.15

0.93

0.14

0.94

0.25

0.95

0.17

0.86

0.20

0.88

0.42

0.90

0.23

0.86

0.66

0.87

0.69

0.90

0.67

ADC

CRP

RAE

RA

RD

micro-Doppler

Sequence 107 Real

4DPS

Sequence 93 RF-Genesis

Real

4DPS

Sequence 27 RF-Genesis

Real

4DPS

RF-Genesis

0.99

0.62

1.00

0.66

0.99

0.75

0.92

0.58

0.89

0.58

0.86

0.49

0.94

0.14

0.96

0.14

0.98

0.13

0.91

0.16

0.94

0.17

0.97

0.15

0.81

0.24

0.88

0.24

0.94

0.21

0.83

0.65

0.87

0.65

0.94

0.66

ADC

CRP

RAE

RA

RD

micro-Doppler

Sequence 165 Real

Figure 15: Qualitative gallery, HuPR, horizontal board (2 of 3): sequences 107, 30 and 96 (top) and 165, 93 and 27 (bottom) (4DPS-fidelity ranks 58, 73, 89, 104, 120 and 135 of 235). As Fig. 14.

51

Preprint

4DPS

Sequence 252 RF-Genesis

Real

4DPS

Sequence 56 RF-Genesis

Real

4DPS

RF-Genesis

0.98

0.68

0.96

0.54

0.98

0.53

0.86

0.52

0.85

0.15

0.81

0.41

0.96

0.11

0.97

0.14

0.96

0.11

0.94

0.13

0.95

0.18

0.95

0.13

0.86

0.20

0.81

0.32

0.90

0.19

0.86

0.63

0.80

0.67

0.89

0.65

ADC

CRP

RAE

RA

RD

micro-Doppler

Sequence 194 Real

4DPS

Sequence 67 RF-Genesis

Real

4DPS

Sequence 60 RF-Genesis

Real

4DPS

RF-Genesis

0.99

0.59

0.97

0.63

0.98

0.63

0.79

0.31

0.76

0.36

0.58

0.39

0.96

0.20

0.96

0.10

0.96

0.04

0.95

0.24

0.95

0.12

0.96

0.05

0.85

0.36

0.87

0.19

0.94

0.05

0.85

0.65

0.87

0.65

0.93

0.72

ADC

CRP

RAE

RA

RD

micro-Doppler

Sequence 158 Real

Figure 16: Qualitative gallery, HuPR, horizontal board (3 of 3): sequences 194, 252 and 56 (top) and 158, 67 and 60 (bottom) (4DPS-fidelity ranks 150, 166, 181, 197, 212 and 228 of 235). As Fig. 14.

52

Related documents

• Beyond Gait: Person Identification from Millimeter-Wave Point Clouds Across Activities of Daily Living#986448
• Kapture: Capturing Cardiac Dynamics with Koopman-Governed Learning for Efficient Radar-Based Electrocardiogram Recovery#1200606
• Lightweight Gait-Based Person Identification via Transmit-Channel Differential Encoding with MIMO Millimeter-Wave Radar.#1359549
• Untangling the Geometry and Speed for RF Sensing Spectrograms#1063171
• 2Xplat: Decoupling Geometry and Appearance Modeling for Feed-Forward 3D Gaussian Splatting#635777