Client-Side Probing of Deleted Ridge Statistics in Federated Unlearning
arXiv:2609.04475v1 [cs.CR] 3 Sep 2026
Yijun Quan
Giovanni Montana
University of Warwick [email protected] [email protected]
Abstract Federated unlearning aims to remove a client’s data from a shared model without retraining from scratch. Some efficient systems make deletion exact by storing compact, additive summaries of the training features and broadcasting an updated linear classifier after every accepted change. We show that these broadcasts can also reveal the hidden summaries. A malicious client can submit known changes, use the returned classifiers to identify the server state, and compare states immediately before and after an isolated deletion. This exposes the deleted sample, class, or client summary and can enable its reinsertion. We characterize exactly when the observations contain enough independent information, give a matching optimal construction for unrestricted probes, and derive a more realistic estimator based on additions formed from the attacker’s own data. On MNIST and CIFAR-10, high-precision broadcasts permit exact label recovery for every tested sample deletion with both probe types. Lower-precision broadcasts sharply reduce fine-grained recovery, and insufficiently diverse responses prevent identification altogether. Unrestricted probes are readily detected by their size; most individual attacker-data additions resemble honest batches, although we do not claim that the complete sequence is inconspicuous. The results identify a concrete privacy and integrity risk, its algebraic cause, and practical limits involving broadcast precision, update verification, response rate, and concurrent activity.
1
Introduction
Federated learning keeps raw data distributed across clients, but a participant may later request removal of a sample, a class, or its entire contribution. Federated unlearning (FU) Liu et al. [2022] seeks to remove that influence without collecting the raw data centrally or retraining the shared model from scratch. A useful FU mechanism must therefore combine data locality and efficient deletion with fidelity to retraining on the retained data. A common efficient design keeps a large pretrained feature extractor fixed and updates only a lightweight linear classifier. The classifier can be recomputed from two compact sums: products between training features, and products between features and labels. These sums are sufficient statistics: they contain all information that this classifier needs from the retained training set. Because clients can add or subtract their contributions, the approach supports exact continual learning and unlearning without starting over Quan et al. [2026]. We ask what an active client can learn when the server accepts such updates and returns the new classifier after each one. Attacks that compare models before and after unlearning already show that deletion can expose data. Gao et al. formalize deletion inference and reconstruction across several learning tasks Gao et al. [2022]; Hu et al. recover internal features with model access and infer labels through prediction queries Hu et al. [2024]; and Bertran et al. reconstruct samples from regularized linear models with fixed feature extractors Bertran 1
et al. [2024]. Bertran et al. estimate a hidden training-data quantity using an independent public sample from the same distribution; our attack instead identifies the required server summary from its responses. Zhou et al. similarly invert models observed before and after federated unlearning from a server adversary Zhou et al. [2026]. Those attacks use the model change caused by deletion, whereas our participating client first identifies the server state and then recovers the deleted aggregate. Related work also studies malicious clients and deletion requests. Federated clients can reconstruct peers from consecutive training updates or improve reconstruction through malicious updates Wilson et al. [2024], Yue et al. [2025]. Sheng et al. infer and reconstruct unlearned data, then use those reconstructions either to impede forgetting or to degrade performance on the deleted data Sheng et al. [2026]. Huang et al. show that unlearning requests for samples never present in training can destroy model accuracy Huang et al. [2025]. Cohen et al. prove that, for certain tasks, an exact unlearning mechanism can let an adversary controlling few points reconstruct almost the entire dataset through deletion requests Cohen et al. [2026]. Their result is a general information-theoretic warning. We give a constructive attack for systems that store the additive summaries introduced above and publish the complete classifier, together with an exact success condition and measured precision limits. The attack exploits the information revealed across multiple valid interactions, not a single before-andafter pair. The client submits changes whose effect it knows, observes how the classifier moves, and repeats until the responses determine the server’s regularized training summary. It performs this identification immediately before and after one isolated deletion. The difference reveals the deleted aggregate even when the regularization strength is unknown, because that unchanged term cancels. Resubmitting the recovered aggregate then reverses the deletion. We study both unrestricted matrix changes, which expose the algebraic limit of the interface, and additions assembled from the attacker’s own examples. Our contributions connect the threat model, theory, and measured attack. First, we prove a necessary-andsufficient condition for exact state identification: the collected classifier changes must span the full feature space. We give a probe construction that meets the resulting minimum number of responses. Second, we derive the corresponding estimator for cumulative additions made from attacker data and show why owning enough feature-diverse examples is necessary but not sufficient. Third, we recover and replay isolated sample-, class-, and client-level deletions. Experiments with two image datasets, two feature extractors, and two broadcast precisions measure where the attack succeeds, where numerical error overwhelms small deletions, and which messages a simple size-based detector can flag.
2
Background
The studied federated continual-unlearning protocol Quan et al. [2026] exchanges two matrix summaries between each client and the server. Let 𝜙(·) denote a frozen feature extractor shared by all clients, let 𝑑 be its feature dimension, and let 𝑐 be the number of classes. Client 𝑘 maintains a local retained dataset 𝐷 𝑘,𝑡 at round 𝑡 with 𝐷 𝑘,0 = ∅. Between rounds, it may receive an add batch 𝐷 +𝑘,𝑡 and a delete batch 𝐷 −𝑘,𝑡 ⊆ 𝐷 𝑘,𝑡 −1 . ±
±
± ∈ R𝑛𝑘,𝑡 ×𝑑 and one-hot label matrices 𝑌 ± ∈ {0, 1} 𝑛𝑘,𝑡 ×𝑐 . The client then computes Applying 𝜙(·) gives 𝐹𝑘,𝑡 𝑘,𝑡 the feature-product matrix 𝑆 ±𝑘,𝑡 ∈ R𝑑×𝑑 and feature–label matrix 𝐺 ±𝑘,𝑡 ∈ R𝑑×𝑐 as + ⊤ + 𝑆 +𝑘,𝑡 = (𝐹𝑘,𝑡 ) 𝐹𝑘,𝑡 ,
+ ⊤ + 𝐺 +𝑘,𝑡 = (𝐹𝑘,𝑡 ) 𝑌𝑘,𝑡 ,
(1)
and
− − ⊤ − − ⊤ − 𝑆 𝑘,𝑡 = (𝐹𝑘,𝑡 ) 𝐹𝑘,𝑡 , 𝐺 −𝑘,𝑡 = (𝐹𝑘,𝑡 ) 𝑌𝑘,𝑡 . − , 𝐺− The client transmits 𝑆 +𝑘,𝑡 , 𝐺 +𝑘,𝑡 , 𝑆 𝑘,𝑡 𝑘,𝑡 to the server.
2
(2)
At the server, the global add and delete statistics are aggregated as 𝑆𝑡+ =
𝐾 ∑︁
𝑆 +𝑘,𝑡 ,
𝐺 +𝑡 =
𝐾 ∑︁
𝐺 +𝑘,𝑡 ,
(3)
𝐺 −𝑘,𝑡 .
(4)
𝐺 𝑡 ← 𝐺 𝑡 −1 + 𝐺 +𝑡 − 𝐺 𝑡− ,
(5)
𝑘=1
𝑘=1
and 𝑆𝑡− =
𝐾 ∑︁
− 𝑆 𝑘,𝑡 ,
𝐺 𝑡− =
𝑘=1
𝐾 ∑︁ 𝑘=1
The server maintains a retained-statistics ledger, 𝑆𝑡 ← 𝑆𝑡 −1 + 𝑆𝑡+ − 𝑆𝑡− ,
and recovers the ridge-head parameters in closed form by 𝑊𝑡 = (𝑆𝑡 + 𝛾𝐼) −1 𝐺 𝑡 ,
(6)
where 𝑊𝑡 ∈ R𝑑×𝑐 and 𝛾 > 0 is a ridge coefficient. A logical probe comprises all constituent summaries that create one chosen net perturbation and the single full-head broadcast after the server applies them. Client messages, constituent add/delete pairs, and server broadcasts are counted separately in the evaluation. At each round, a client observes the current global weight 𝑊𝑡 and its own local summaries, but not other clients’ data, features, gradients, or statistics. A returned head alone does not identify (𝑆𝑡 , 𝐺 𝑡 ) because the mapping (𝑆𝑡 , 𝐺 𝑡 ) ↦→ 𝑊𝑡 = (𝑆𝑡 + 𝛾𝐼) −1 𝐺 𝑡 is many-to-one. The attack below resolves this ambiguity by actively collecting responses to known summary changes.
3
Threat Model
Passive observation of 𝑊𝑡 does not identify (𝑆𝑡 , 𝐺 𝑡 ) because many ridge states produce the same head. We therefore consider an active malicious client that submits chosen sufficient-statistic summaries and observes the full head returned by the server after each logical probe. The client knows the frozen encoder and its own messages but not other clients’ data, statistics, or the server’s fixed 𝛾. The protocol guarantees the same result as retraining only when each deletion belongs to the requesting client’s retained data. We test a server that does not verify record ownership, message origin, or duplicate submissions. The attack also needs a quiet interval: no other client may update the state while each sequence runs. Exactly one deletion of the stated size must occur between the two sequences, although its content remains unknown. Ownership checks can stop fabricated probes and unauthorized replay. They do not stop an attacker from learning through valid additions. Defending that channel requires fewer broadcasts, batched responses, partial classifiers, or added noise.
Exact Identification from Moment Differences Immediately before probing, define 𝐴 = 𝑆 + 𝛾𝐼 ≻ 0,
𝐻 = 𝐴 −1 ,
𝑊0 = 𝐻𝐺.
(7)
For probe response 𝑗, let 𝑄 𝑗 ∈ R𝑑×𝑐 be the known total moment perturbation present when 𝑊 𝑗 is observed, with no net Gram perturbation. If each probe is restored before the next, 𝑄 𝑗 = Δ𝐺 𝑗 ; if probes accumulate, Í𝑗 𝑄 𝑗 = ℓ=1 Δ𝐺 ℓ . In both cases, 𝑅 𝑗 := 𝑊 𝑗 − 𝑊0 = 𝐴 −1 𝑄 𝑗 , 3
𝐴𝑅 𝑗 = 𝑄 𝑗 .
(8)
Stack the response differences and known perturbations as 𝑅 = [𝑅1 , . . . , 𝑅𝑚 ], so that 𝑅, 𝑄 ∈ R𝑑×𝑚𝑐 and
𝑄 = [𝑄 1 , . . . , 𝑄 𝑚 ],
𝑅 = 𝐴 −1 𝑄 = 𝐻𝑄,
𝐴𝑅 = 𝑄.
(9)
(10)
Realizing a moment-only probe. An arbitrary moment change can be expressed through the protocol’s add/delete matrices while leaving the Gram matrix unchanged. For a feature vector 𝑢 and one-hot label 𝑦, 𝑢𝑢 ⊤ − (−𝑢) (−𝑢) ⊤ = 0,
𝑢𝑦 ⊤ − (−𝑢)𝑦 ⊤ = 2𝑢𝑦 ⊤ .
(11)
Applying this construction to each class column realizes any desired 𝑄 𝑗 ∈ R𝑑×𝑐 . Each submitted Gram matrix is positive semidefinite and each moment has one-hot-label form. The construction is algebraically valid, but the vector −𝑢 need not be produced by the shared encoder and the deletion need not pass an ownership check. The data-derived variant below avoids fabricated feature vectors, although our evaluated sequence still assumes that duplicate submissions are accepted. Theorem 1 (exact recovery from moment probes). Fix 𝛾 > 0, which need not be known to the attacker, and consider any protocol whose hidden additive state satisfies 𝐴 = 𝑆 + 𝛾𝐼 ≻ 0, 𝑆 ⪰ 0, 𝐺 ∈ R𝑑×𝑐 , and whose released head is 𝑊 = 𝐴 −1 𝐺. If its interface accepts the moment-only changes above and releases the resulting full head, then, in exact arithmetic, the observations (𝑊0 , {𝑊 𝑗 , 𝑄 𝑗 } 𝑚 𝑗=1 ) uniquely identify ( 𝐴, 𝐺) if and only if rank(𝑄) = 𝑑, (12) equivalently rank(𝑅) = 𝑑. When this holds, 𝐴 = 𝑄𝑅 † ,
𝐻 = 𝑅𝑄 † ,
𝐺 = 𝐴𝑊0 ,
(13)
where † denotes the Moore–Penrose pseudoinverse. Thus the regularized Gram matrix needed for deletion b recovery is obtained directly, without inverting an estimated 𝐻. For sufficiency, 𝐴 is invertible, so rank(𝑅) = rank(𝑄). At full row rank, right-multiplying 𝐴𝑅 = 𝑄 by 𝑅 † gives 𝐴 = 𝑄𝑅 † ; similarly, 𝑅 = 𝐻𝑄 gives 𝐻 = 𝑅𝑄 † , followed by 𝐺 = 𝐴𝑊0 . For necessity, suppose rank(𝑅) < 𝑑 and choose nonzero 𝑣 with 𝑣 ⊤ 𝑅 = 0. For any 𝜀 > 0, let 𝐴′ = 𝐴 + 𝜀𝑣𝑣 ⊤ ,
𝑆 ′ = 𝑆 + 𝜀𝑣𝑣 ⊤ ⪰ 0,
𝐺 ′ = 𝐴′𝑊0 .
(14)
Then 𝐴′ = 𝑆 ′ + 𝛾𝐼 ≻ 0, 𝐴′ ≠ 𝐴, and 𝐴′ 𝑅 = 𝐴𝑅 = 𝑄. Hence 𝐴′𝑊 𝑗 = 𝐴′ (𝑊0 + 𝑅 𝑗 ) = 𝐺 ′ + 𝑄 𝑗 for every probe, while 𝐴′𝑊0 = 𝐺 ′ . A distinct feasible ridge state therefore produces exactly the same observations. Necessity is over this algebraic state class; it does not require the alternative 𝐺 ′ to arise from a particular labelled dataset. Í Scope beyond the studied protocol. RanPAC McDonnell et al. [2023] independently maintains G = ℎℎ⊤ Í and 𝐶 = ℎ𝑦 ⊤ for frozen projected features and forms 𝑊𝑜 = (G + 𝜆𝐼) −1𝐶. This matches the algebraic state above. RanPAC is neither federated unlearning nor an attack target here; Theorem 1 reaches another deployment only if it also accepts known changes and releases every full head.
4
Corollary 1 (exact logical-response complexity). row rank requires
Since each 𝑄 𝑗 has at most 𝑐 independent columns, full
𝑚≥
𝑑 . 𝑐
(15)
The bound is achievable when arbitrary algebraic moment probes are accepted: for 𝑚 = ⌈𝑑/𝑐⌉, partition the columns of 𝑄 = [𝜏𝐼 𝑑 , 0] ∈ R𝑑×𝑚𝑐 , with 𝜏 > 0, into 𝑚 blocks. Then 𝑄𝑄 ⊤ = 𝜏 2 𝐼 𝑑 , so 𝑄 has full row rank, 𝜎min (𝑄) = 𝜏, and condition number one. Therefore the noiseless chosen-summary query complexity is exactly ⌈𝑑/𝑐⌉ logical full-head responses per unknown state. This statement does not exploit additional dataset-realizability structure. With 512-dimensional features and ten classes, identifying one unknown state needs 52 probe responses plus its baseline. A first attack identifies the states on both sides of a deletion. It therefore uses 104 probe responses and 109 server responses in total. The other five responses are the initial baseline, two cancellations, the deletion, and replay. Once a pre-deletion state has been identified and restored, a later event needs 55 responses. These counts treat one complete matrix block as a logical probe and assume one broadcast after that block. A server that broadcasts after every rank-one part would return many more responses. We therefore report logical probes, client messages, and server broadcasts separately. The attacker first records a baseline and identifies the pre-deletion state, then cancels its cumulative probe. After observing one isolated deletion, it treats the resulting head as the post-deletion baseline, identifies that state, and cancels the second probe sequence. The difference between the two identified states yields the deleted aggregate block, which the attacker can then replay. e = 𝑅 + 𝐸 and full-row-rank 𝑅, e direct recovery With an observed response stack 𝑅 b = 𝑄𝑅 e† 𝐴 satisfies b − 𝐴 = −𝐴𝐸 𝑅 e† , 𝐴
b − 𝐴∥ 𝐹 ≤ ∥𝐴
(16) ∥ 𝐴∥ 2 ∥𝐸 ∥ 𝐹 . e 𝜎min ( 𝑅)
(17)
Here 𝐸 includes error in each probe response and in the shared baseline subtracted from every response b = 𝑅𝑄 e † obeys ∥ 𝐻 b − 𝐻 ∥ 𝐹 ≤ ∥𝐸 ∥ 𝐹 /𝜎min (𝑄) when 𝑄 has full difference. The complementary estimator 𝐻 row rank. The implementation therefore uses SVD- or QR-based solves and reports the numerical rank and e together with direct-𝐴 and direct-𝐻 agreement. conditioning of both 𝑄 and 𝑅, b= 𝐴 b𝑊 e0 = 𝑊0 + 𝑁0 , then 𝐺 e0 has error If the baseline is observed as 𝑊 b − 𝐺 = (𝐴 b − 𝐴)𝑊0 + 𝐴𝑁 b 0. 𝐺
(18)
Thus moment recovery depends on both state-identification error and baseline precision. We solve for 𝐴 directly, symmetrize the estimate, and reject it unless it is positive definite. An independently estimated b it is a diagnostic 𝐻 = 𝐴 −1 is also symmetrized and checked for positive definiteness and agreement with 𝐴; and is never inverted to obtain 𝐴. The supplementary material reports the numerical residuals. The recovered b = 𝐴𝑊 b 0. moment is 𝐺 In the cumulative implementation, each returned classifier is paired with the total perturbation present at that time. The attacker cancels the accumulated perturbation after the last response. Numerical rank counts singular values above 𝜀 rank 𝜎max . The full pseudocode, independent inverse estimate, definiteness checks, and solve diagnostics appear in the supplement.
5
Attacker-Data Additions The unrestricted construction may use feature vectors that the shared encoder cannot produce. We therefore also study additions formed from the attacker’s own examples. The additions accumulate until the server returns classifier 𝑊 𝑗 . Let (𝑃 𝑗 , 𝑄 𝑗 ) denote their known total feature-product and feature–label summaries. Subtracting the baseline equation gives 𝐴(𝑊 𝑗 − 𝑊0 ) = 𝑄 𝑗 − 𝑃 𝑗 𝑊 𝑗 .
(19)
Stack the observed changes as 𝑋 = [𝑊1 − 𝑊0 , . . . , 𝑊𝑚 − 𝑊0 ] and the known right-hand sides as 𝑍 = [𝑄 1 − 𝑃1𝑊1 , . . . , 𝑄 𝑚 − 𝑃𝑚𝑊𝑚 ]. The state is identifiable exactly when 𝑋 has rank 𝑑, in which case 𝐴 = 𝑍 𝑋 † . If probe 𝑗 contains features 𝐹 𝑗 and labels 𝑌 𝑗 , then 𝑍 𝑗 = 𝐹 ⊤ 𝑗 (𝑌 𝑗 − 𝐹 𝑗 𝑊 𝑗 ). Let 𝐹attacker stack all distinct attacker features used by the sequence. Then rank(𝑋) = rank(𝑍) ≤ rank(𝐹attacker ). The attacker therefore needs at least 𝑑 examples and full feature rank. These conditions are not sufficient because the prediction residuals and batching sequence also affect 𝑍. Thus 52 responses is only a dimensional lower bound for attacker-data additions.
Recovering and Replaying an Isolated Deletion Let ( 𝐴pre , 𝑊pre ) and ( 𝐴post , 𝑊post ) be two identified states bracketing one deletion, with the same fixed 𝛾. The deleted contribution is exactly 𝑆del = 𝐴pre − 𝐴post ,
𝐺 del = 𝐴pre𝑊pre − 𝐴post𝑊post .
(20)
Indeed, deletion gives 𝐴post = 𝐴pre − 𝑆del because the same 𝛾 appears in both states, while 𝐺 pre = 𝐴pre𝑊pre and 𝐺 post = 𝐴post𝑊post . Adding the recovered pair to the post-deletion ledger therefore returns both sufficient statistics and the head exactly in exact arithmetic. If several honest events occur between the two identified states, the equations recover only their signed aggregate net change and cannot attribute it to a particular event or client. If 𝛾 changes between states, the Gram difference is additionally shifted by (𝛾pre − 𝛾post )𝐼. After the cumulative probes are cancelled, the implementation records the numerical consistency diagnostic 𝑟𝑊 =
∥𝑊final − 𝑊0 ∥ 𝐹 , max(∥𝑊0 ∥ 𝐹 , 𝜀den )
(21)
with 𝜀 den = 10−15 . Rank and the two positive-definiteness tests determine identification success; 𝑟 𝑊 is reported but is not a success criterion. Equality of heads does not certify equality of hidden states, so this diagnostic does not prove that no honest client updated. The supplementary material gives simulator-only hidden-ledger checks. bpre = 𝐴pre + 𝐸 pre and 𝐴 bpost = 𝐴post + 𝐸 post . To connect state-estimation error to replay integrity, write 𝐴 The replayed head then satisfies the exact identity 𝑊replay − 𝑊pre = 𝐴pre + 𝐸 pre − 𝐸 post
−1 (22)
𝐸 post (𝑊pre − 𝑊post ), whenever the leading matrix is invertible. Consequently, ∥𝑊replay − 𝑊pre ∥ 𝐹 ≤ ∥ 𝐴pre + 𝐸 pre − 𝐸 post
−1
∥2
× ∥𝐸 post ∥ 2 ∥𝑊pre − 𝑊post ∥ 𝐹 .
(23)
The pre-state error therefore enters only through the inverse, whereas the post-state error also controls the numerator. In particular, an exact post-state estimate can restore the head even when the replayed ledger 6
remains inaccurate. If the observed baselines contain errors 𝑁pre and 𝑁post , the numerator additionally bpre 𝑁pre − 𝐴 bpost 𝑁post . contains 𝐴 Matching the earlier classifier does not prove that the hidden summaries were restored: distinct server states can return the same classifier. The supplement proves this fact and gives the evaluator-only checks used for the feature-product summary, feature–label summary, and returned classifier. It also reports the smallest eigenvalue of the recovered deleted Gram block, because finite-precision error can violate the positive-semidefinite constraint expected by a validating server.
4
Probing Attack Evaluation
We test three consequences of the attack. First, can it recover the label and feature of one deleted sample? Second, can it recover the combined summaries removed by a class or client deletion? Third, can replaying those summaries reverse the deletion? Our primary experiments map images through a frozen ImageNet-1K ResNet-18 He et al. [2016], Deng et al. [2009] to 512-dimensional features. A robustness check instead uses the frozen 768-dimensional DINOv2 ViT-B/14 encoder Oquab et al. [2024]. We use MNIST Deng [2012] and CIFAR-10 Krizhevsky [2009], fixed 𝛾 = 10−3 , 50 clients, and a Dirichlet split with concentration 𝛼 = 0.05, which produces highly uneven class proportions across clients. Every evaluated split contains at least ten samples per client. Server ledgers and solves remain float64; the primary setting returns float64 heads and the precision ablation rounds every returned head and baseline to float32. The simulated server updates its retained summaries and returns the classifier after solving the ridge system. An attack first identifies the pre-deletion state and cancels its probes. The server then applies one honest deletion. The returned classifier becomes the post-deletion baseline. The attack identifies that state, cancels again, recovers the difference, and replays it from the actual post-cancellation state. Recovery uses only returned classifiers and known attacker messages. The evaluator reads the hidden summaries only to apply the chosen deletion and score recovery, cancellation, and replay. Failures remain in all success denominators. Error averages include successful runs only. We measure recovery of the deleted moment and Gram matrices with relative Frobenius error: RelErr(Δ𝐺) =
c − Δ𝐺 ∥ 𝐹 ∥ Δ𝐺 , ∥Δ𝐺 ∥ 𝐹
RelErr(Δ𝑆) =
c − Δ𝑆∥ 𝐹 ∥ Δ𝑆 . ∥Δ𝑆∥ 𝐹
Rank threshold. Across five random probe sequences per dataset (Table 1), 𝑚 = 51 always gives rank 510 and fails. At 𝑚 = 52, every stack reaches rank 512 and succeeds. The designed 52-response probe also reaches rank 512. Its float64 state error is 1.51 × 10−12 on MNIST and 6.55 × 10−13 on CIFAR-10. Probe scale. Every tested MNIST scale succeeds (see the supplementary material), but the designed probe is conspicuous. At 𝜏 = 104 , its message norm is 2.98 × 105 times the honest-update 99th percentile. The largest classifier change is 8.15×104 times its honest threshold. Clipping each payload to the honest threshold still produces a 149-fold classifier change. Deleted-block errors remain 3.68 × 10−6 for 𝑆 and 1.76 × 10−6 for 𝐺. The supplement gives the full sweep. Table 2 reports the primary designed-probe results. All 2,220 float64 attacks identify both states. Sample labels are always recovered, while aggregate errors remain between 10−12 and 10−7 depending on deletion size.
7
Table 1: Float64 identification of one hidden server state. Numerical rank uses relative tolerance 10−10 . Random probes use independent Gaussian moment increments scaled by 104 and report mean ± standard deviation over five sequences. The designed probe uses 𝑄 = [104 𝐼, 0]. Every rank-510 run fails and every rank-512 run succeeds. Data
Probe
𝑚
Success rank(𝑄)
MNIST
Random Random Random Designed
51 52 64 52
0/5 5/5 5/5 1/1
510 512 512 512
∞ ∞ – (5.35±.38) × 103 (6.79±1.46) × 107 (8.84±11.65) × 10−11 (5.08±.33) × 102 (2.22±.39) × 107 (2.44±1.63) × 10−11 1 1.06 × 106 1.51 × 10−12
CIFAR-10 Random Random Random Designed
51 52 64 52
0/5 5/5 5/5 1/1
510 512 512 512
∞ (5.35±.38) × 103 (5.08±.33) × 102 1
𝜅(𝑄)
RelErr( 𝐴)
𝜅(𝑅)
∞ (4.05±.71) × 106 (1.23±.11) × 106 4.15 × 104
– (8.13±4.32) × 10−12 (1.66±1.47) × 10−12 6.55 × 10−13
Table 2: Float64 deletion recovery with the designed probe (𝑚 = 52 responses per state, excluding the baseline). Errors are mean ± standard deviation over successful attacks only. Client results include all 50 deletions in each of 20 valid client partitions; failures remain in every success denominator. Deletion
Dataset
Successful
Labels
RelErr(Δ𝐺)
RelErr(Δ𝑆)
Sample
MNIST CIFAR-10
100/100 100/100
100/100 100/100
(4.14 ± 2.67) × 10−8 (1.06 ± 0.48) × 10−8
(1.16 ± 0.83) × 10−7 (2.69 ± 1.52) × 10−8
Class
MNIST CIFAR-10
10/10 10/10
– –
(5.97 ± 4.28) × 10−12 (1.55 ± 1.12) × 10−12
(1.67 ± 1.77) × 10−11 (4.24 ± 3.62) × 10−12
Client
MNIST CIFAR-10
1000/1000 1000/1000
– –
(3.87 ± 10.99) × 10−10 (7.69 ± 21.87) × 10−11
(9.14 ± 25.78) × 10−10 (1.80 ± 4.99) × 10−10
Deletion recovery subtracts two estimates of the full state. Their absolute errors can be large relative to a small deleted block, so subtraction amplifies relative error most strongly for sample deletion. bpre − 𝐴pre and 𝐸 post = 𝐴 bpost − 𝐴post . State differencing gives For the deleted Gram block, let 𝐸 pre = 𝐴 c − Δ𝑆 = 𝐸 pre − 𝐸 post . Therefore Δ𝑆 RelErr(Δ𝑆) ≤ 𝐵𝑆 :=
∥𝐸 pre ∥ 𝐹 + ∥𝐸 post ∥ 𝐹 . ∥Δ𝑆∥ 𝐹
(24)
Figure 1 checks this explanation run by run. The points follow the equality line across both precisions and deletion sizes. Precision changes the state error, while deletion size determines how strongly differencing amplifies it. Attacker data and cost. We instantiate the attacker-data additions using the attacker’s own examples. Across five splits, 25–33 of 50 MNIST clients and 25–28 CIFAR-10 clients pass the feature-rank gate. The selected attackers contain 867–4,309 and 779–5,342 distinct examples, respectively. We divide each set without overlap into 104 batches of 8–42 MNIST or 7–52 CIFAR-10 examples. The same set is reused after cancellation to identify the post-deletion state. These examples already appear in the initial server ledger. The experiment therefore tests duplicate submissions of attacker-owned, encoder-produced data rather than held-out records. The interface does not reject duplicates. A complete first attack sends 208 additions, two cancellations, one deletion, and replay: 212
8
104 102
class, float64 (n=100) sample, float64 (n=1000) class, float32 (n=100) sample, float32 (n=1000) bound attained
measured RelErr(ΔS)
100 10−2 10−4 10−6 10−8 10−10 10−12 10−12
median 0.69 5--95% [0.18, 0.98] max 0.998, violations 0/2200
10−10
10−8
10−6
10−4 10−2 bound BS
100
102
104
Figure 1: Measured deleted-Gram error versus the bound in Equation (24). The 2,200 attacker-data attacks use 104 responses per state and cover both datasets, both precisions, and sample and class deletions. Every run shown passed both state-identification checks; this does not imply accurate deletion recovery. No point crosses the dashed bound. Measured-to-bound ratios have pooled median 0.69, 5–95% range 0.18–0.98, maximum 0.998, and cell-median range 0.43–0.79. client messages and 213 server responses including the baseline. The supplement gives the attacker-selection rule. Observed rank for attacker’s data. With 52 additions, every pre-state reaches rank 512, but a class deletion leaves most post-state stacks deficient. Only 14/50 MNIST and 16/50 CIFAR-10 attacks succeed, and post-state ranks range from 468 to 512 (see supplement). At 104 additions, every tested stack passes at tolerance 10−10 . Median 𝜅(𝑋) is 4.06 × 105 on MNIST and 1.43 × 105 on CIFAR-10. Response count alone is therefore insufficient. Full attacker feature rank is necessary but not sufficient, and the bound does not require label diversity. We also vary the float32 rank tolerance from 10−6 to 10−10 on ten pre- or post-class-deletion stacks per dataset. At 𝑚 = 104, all twenty stacks remain full rank from 10−7 through 10−10 . At 10−6 , all ten CIFAR-10 stacks but only six MNIST stacks remain full rank. Some MNIST directions are therefore marginal. More importantly, full numerical rank only means that the estimator returns; it does not imply accurate recovery. A DINOv2 check reaches the same conclusion with 768-dimensional transformer features. None of the five stacks per dataset is full rank at the 77-response dimensional lower bound, whereas all are full rank at 104 responses. The supplement reports ranks, conditioning, and errors. Individual-message detectability. We calibrate batch-matched 99th-percentile norm thresholds on 5,000 honest additions per dataset. Testing on 10,000 separate additions gives false-positive rates of 1.38% on MNIST and 1.34% on CIFAR-10. Median attacker-message ratios are 0.818 and 0.845 relative to these thresholds. None of 520 MNIST additions and 13 of 520 CIFAR-10 additions exceed them. This test 9
Table 3: Deletion recovery from 𝑚 = 104 cumulative attacker-data addition responses per state. At relative rank tolerance 10−10 , every stack has full observed rank and both state estimates pass the positive-definiteness checks. “𝐺/𝑆” gives mean deleted-moment and deleted-Gram relative errors. Float32 changes only returned heads and baselines. Data
Heads
MNIST float64 CIFAR-10 float64 MNIST float32 CIFAR-10 float32
Sample labels
Sample 𝐺/𝑆
Class 𝐺/𝑆
Client 𝐺/𝑆
500/500 500/500 49/500 183/500
1.55 × 10−6 /1.62 × 10−6
2.87 × 10−10 /3.73 × 10−10
1.08 × 10−8 /8.76 × 10−9 1.36 × 10−9 /1.85 × 10−9 0.393/0.661 0.0842/0.161
9.43 × 10−8 /1.24 × 10−7 68.4/143 5.27/10.1
2.09 × 10−11 /2.97 × 10−11 1.32 × 10−2 /2.74 × 10−2 1.16 × 10−3 /2.42 × 10−3
excludes both cancellations and replay, so it does not show that the complete sequence is inconspicuous. Broadcast precision. With float32 responses, attacker-data sample-label recovery falls to 9.8% on MNIST and 36.6% on CIFAR-10, versus 10% chance. Seed-level rates are 7–14% and 12–100%, so the pooled CIFAR-10 rate hides strong state dependence. Larger class blocks remain more accurate. Designed probes are also precision-limited because their observed response stacks have condition numbers 1.06 × 106 and 4.15 × 104 , despite a perfectly conditioned input probe. Rounding the final honest classifier to float32 changes none of 10,000 test predictions; accuracy remains 97.15% and 85.94%. Figure 1 links the precision and deletion-size results through one mechanism. Float64 and float32 occupy different ranges of the same bound: response rounding increases state error, and differencing amplifies it most when the deleted block is small. The figure is a post-hoc bound check, not a measured prediction for untested numerical formats. Because the corresponding moment bound must also include baseline-rounding terms, we do not infer its tightness from the Gram result. Finite precision also makes some recovered sample and client Gram blocks slightly non-positivesemidefinite. A server that validates this constraint would reject those raw replays. Clipping negative eigenvalues to zero makes every tested block valid. In float64, the resulting replay-head error remains below 6 × 10−9 . The supplement reports the full diagnostic table.
Sample-level probing Sample-level experiment studies the most fine-grained privacy setting, in which one deleted training point is probed at a time. For a deleted sample ( 𝑓𝑖 , 𝑦 𝑖 ), the removed contribution to the sufficient statistics is Δ𝑆𝑖 = 𝑓𝑖 𝑓𝑖⊤ ,
Δ𝐺 𝑖 = 𝑓𝑖 𝑦 ⊤ 𝑖 .
(25)
For a one-hot label 𝑦 𝑖 = 𝑒 𝑘 , the exact deleted moment is Δ𝐺 𝑖 = 𝑓𝑖 𝑒 ⊤𝑘 : its unique nonzero column identifies 𝑘 and equals 𝑓𝑖 . Numerically, small errors can make every column nonzero, so the implementation uses b 𝑘 = arg
max
𝑟 ∈ {1,...,𝑐}
c 𝑖,:,𝑟 ∥ 2 , ∥ Δ𝐺
c b, b 𝑓𝑖 = Δ𝐺 𝑖,:, 𝑘
(26)
and reports ∥ b 𝑓𝑖 − 𝑓𝑖 ∥ 2 /∥ 𝑓𝑖 ∥ 2 . If only Δ𝑆𝑖 = 𝑓𝑖 𝑓𝑖⊤ were known, a nonzero 𝑓𝑖 would be identifiable up to a c 𝑖 = 𝑓𝑖 𝑒 ⊤ + 𝐸, the maximum-column rule global sign, not an arbitrary orthogonal transformation. Writing Δ𝐺 𝑘 is guaranteed to return 𝑘 whenever ∥𝐸 :𝑘 ∥ 2 + max ∥𝐸 :𝑟 ∥ 2 < ∥ 𝑓𝑖 ∥ 2 ; 𝑟≠𝑘
10
(27)
the simpler condition ∥𝐸 ∥ 𝐹 < ∥ 𝑓𝑖 ∥ 2 /2 is sufficient. For every correctly recovered label, the maximum-column rule implies ∥b 𝑓𝑖 − 𝑓𝑖 ∥ 2 ≤ RelErr(Δ𝐺 𝑖 ). (28) ∥ 𝑓𝑖 ∥ 2 Among correctly labelled float64 samples, mean relative feature error is 4.32×10−7 on MNIST and 2.75×10−8 on CIFAR-10. The supplementary material gives the complete feature-error summary. It also provides selected decoder outputs as a qualitative illustration. Each decoder was trained only on the selected attacker’s local image–feature pairs. This test exposes the threat of possible content interpretation from the recovered features.
Class-level probing Class-level experiment evaluates whether the probing attack can recover the aggregate statistics of a deleted class. Here, the deleted object is no longer a single point but the entire class-wise contribution to the retained-set ledger. If D𝑐 denotes the set of all training samples from class 𝑐, then the attack seeks to reconstruct ∑︁ ∑︁ Δ𝑆 𝑐 = 𝑓𝑖 𝑓𝑖⊤ , Δ𝐺 𝑐 = 𝑓𝑖 𝑦 ⊤ (29) 𝑖 . 𝑖 ∈ D𝑐
𝑖 ∈ D𝑐
This setting is especially important because it tests whether the probe can capture large, structured deletions rather than only isolated points. Table 2 reports small relative errors for the recovered class-level aggregate blocks. Because all deleted Í labels equal 𝑒 𝑐 , the class moment has the form Δ𝐺 𝑐 = ( 𝑖 ∈ D𝑐 𝑓𝑖 )𝑒 ⊤ 𝑐 . When this feature sum is nonzero and distinguishable from numerical error, its column identifies the deleted class. The aggregate does not identify the individual examples in general. Two branches are advanced sequentially: an honest branch accumulates ten class deletions, while the attacked branch replays each recovered class block immediately after its deletion. At the final step, their accuracies are 9.82% versus 97.15% on MNIST and 8.94% versus 85.94% on CIFAR10. The attacked branch’s final relative head errors from the original head are 2.42 × 10−12 and 6.65 × 10−13 ; full trajectories appear in the supplement. This measures reintroduction of aggregate blocks, not recovery of individual examples.
Client-level probing Client-level deletion targets the aggregate contribution of dataset D 𝑘 : ∑︁ ∑︁ 𝑓𝑖 𝑦 ⊤ 𝑓𝑖 𝑓𝑖⊤ , Δ𝐺 𝑘 = Δ𝑆 𝑘 = 𝑖 .
(30)
𝑖 ∈ D𝑘
𝑖 ∈ D𝑘
Across 20 independently sampled 50-client partitions per benchmark, Table 2 reports all 1,000 deleted-client blocks. The honest branch accumulates deletions, whereas the attacked branch immediately replays every recovered block. Its mean final relative head errors are (6.29 ± 0.65) × 10−12 on MNIST and (1.42 ± 0.11) × 10−12 on CIFAR-10; the supplement gives the complete trajectories. For a mixed-label client deletion, column 𝑟 of Δ𝐺 𝑘 is the sum of deleted features with label 𝑟, while Δ𝑆 𝑘 is their aggregate second moment. These aggregates reveal classwise feature sums and one overall second moment, but they do not identify individual examples in general.
5
Discussion and Conclusions
This work shows how repeated classifier releases can expose the compact training summaries used by exact ridge-based unlearning. We prove the precise condition for identifying one server state and match it with 11
an optimal unrestricted probe. We then derive an estimator that uses additions made from attacker data. Identifying states immediately before and after a deletion reveals the deleted aggregate and enables replay. The experiments show both success and clear limits. Every high-precision designed attack succeeds. With 104 attacker-data responses per state, every tested high-precision sample label is also recovered. The theoretical minimum response count does not guarantee success for attacker-data additions: some ResNet-18 and DINOv2 response stacks remain rank deficient. Lower precision sharply reduces sample recovery but leaves larger class aggregates more accurate. A simple norm detector catches the unrestricted probe but misses most individual attacker-data additions. We do not claim whether the complete sequence is stealthy. Deletion size also changes the security consequence. Recovering one sample’s feature and label is a confidentiality failure. A class or client aggregate need not reveal individual records, but replay can reverse the requested deletion. Float32 sharply reduces sample recovery while leaving class aggregates much more accurate, so it is not a uniform safeguard. The probe types expose complementary weaknesses in simple defenses. A norm threshold detects the unrestricted probe, but with float32 responses it still recovers 92 of 100 MNIST labels and all 100 CIFAR-10 labels. Attacker-data additions usually pass that test and recover every label from float64 responses. The presented attack requires the complete classifier after each update and a quiet interval. It assumes fixed regularization and one isolated deletion of known size. The evaluated additions resubmit ledger records, so duplicate checking would block them. Finally, our identifiability conditions demand full feature rank. These constraints point to concrete directions for securing ridge-based federated unlearning systems. Servers should verify ownership, reject duplicates, and authenticate replay. Valid new records may still reveal the state through repeated responses. Servers can respond less often, batch updates, release only part of the classifier, or add noise. Lower precision helps selectively. Future work should test held-out data under concurrency.
References Martin Bertran, Shuai Tang, Michael Kearns, Jamie Morgenstern, Aaron Roth, and Zhiwei S Wu. Reconstruction attacks on machine unlearning: Simple models are vulnerable. Advances in Neural Information Processing Systems, 37:104995–105016, 2024. Aloni Cohen, Refael Kohen, Kobbi Nissim, and Uri Stemmer. Protecting the Undeleted in Machine Unlearning. In Huijia (Rachel) Lin, editor, 7th Symposium on Foundations of Responsible Computing (FORC 2026), volume 368 of Leibniz International Proceedings in Informatics (LIPIcs), pages 17:1–17:18, Dagstuhl, Germany, 2026. Schloss Dagstuhl – Leibniz-Zentrum für Informatik. doi: 10.4230/LIPIcs.FORC.2026.17. URL https://drops.dagstuhl.de/entities/document/10.4230/LIPIcs.FORC.2026.17. Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pages 248–255, 2009. doi: 10.1109/CVPR.2009.5206848. Li Deng. The mnist database of handwritten digit images for machine learning research [best of the web]. IEEE Signal Processing Magazine, 29(6):141–142, 2012. doi: 10.1109/MSP.2012.2211477. Ji Gao, Sanjam Garg, Mohammad Mahmoody, and Prashant Nalini Vasudevan. Deletion inference, reconstruction, and compliance in machine (un)learning. Proceedings on Privacy Enhancing Technologies, 2022(3):415–436, 2022. doi: 10.56553/popets-2022-0079. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 770–778, June 2016. doi: 10.1109/CVPR.2016.90. 12
Hongsheng Hu, Shuo Wang, Tian Dong, and Minhui Xue. Learn what you want to unlearn: Unlearning inversion attacks against machine unlearning. In 2024 IEEE Symposium on Security and Privacy (SP), pages 3257–3275. IEEE, 2024. doi: 10.1109/SP54263.2024.00248. Yangsibo Huang, Daogao Liu, Lynn Chua, Badih Ghazi, Pritish Kamath, Ravi Kumar, Pasin Manurangsi, Milad Nasr, Amer Sinha, and Chiyuan Zhang. Unlearn and burn: Adversarial machine unlearning requests destroy model accuracy. In International Conference on Learning Representations, 2025. URL https://proceedings.iclr.cc/paper files/paper/2025/hash/ 640fd9637f6fb055f4f6551835ee1eb6-Abstract-Conference.html. Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical report, University of Toronto, Toronto, Ontario, 2009. Yi Liu, Lei Xu, Xingliang Yuan, Cong Wang, and Bo Li. The right to be forgotten in federated learning: An efficient realization with rapid retraining. In IEEE INFOCOM 2022-IEEE conference on computer communications, pages 1749–1758. IEEE, 2022. Mark D. McDonnell, Dong Gong, Amin Parvaneh, Ehsan Abbasnejad, and Anton van den Hengel. Ranpac: Random projections and pre-trained models for continual learning. In Advances in Neural Information Processing Systems, volume 36, 2023. Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. Transactions on Machine Learning Research, 2024. Yijun Quan, Wentai Wu, and Giovanni Montana. Exact federated continual unlearning for ridge heads on frozen foundation models. In Machine Learning and Knowledge Discovery in Databases (ECML PKDD), Lecture Notes in Artificial Intelligence. Springer, 2026. To appear. Xinyi Sheng, Wei Bao, Hequn Wang, Yuqin Liu, and Sen Fu. Retaliatory attacks against federated unlearning via data leakage. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 40, pages 25321–25329, 2026. doi: 10.1609/aaai.v40i30.39725. Ethan Wilson, Kai Yue, Chau-Wai Wong, and Huaiyu Dai. Federated learning nodes can reconstruct peers’ image data, 2024. Kai Yue, Richeng Jin, Chau-Wai Wong, and Huaiyu Dai. Byzantine outside, curious inside: Reconstructing data through malicious updates, 2025. Lei Zhou, Youwen Zhu, and Rongke Liu. Model inversion attack against federated unlearning. IEEE Transactions on Information Forensics and Security, 21:2342–2357, 2026. doi: 10.1109/TIFS.2026. 3666295.
13
Supplementary Material: Recovering and Replaying Deleted Ridge Statistics in Federated Unlearning
A
Exact Identification from Moment Probes
Notation and Observation Model Let the server state at one stable probing interval be 𝐻 = 𝐴 −1 .
𝐴 = 𝑆 + 𝛾𝐼 ≻ 0,
(31)
Here 𝑆 ⪰ 0, 𝐺 ∈ R𝑑×𝑐 , and 𝑊0 = 𝐴 −1 𝐺 = 𝐻𝐺. The attacker changes only the moment statistic. When response 𝑊 𝑗 is observed, let 𝑄 𝑗 ∈ R𝑑×𝑐 be the known total moment perturbation currently present. Thus 𝑅 𝑗 := 𝑊 𝑗 − 𝑊0 = 𝐴 −1 𝑄 𝑗 ,
𝐴𝑅 𝑗 = 𝑄 𝑗 .
(32)
For independently restored probes, 𝑄 𝑗 = Δ𝐺 𝑗 . For cumulative probes, 𝑄𝑗 =
𝑗 ∑︁
Δ𝐺 ℓ .
(33)
ℓ=1
Define the horizontal stacks
𝑅 = [𝑅1 , . . . , 𝑅𝑚 ],
(34)
𝑄 = [𝑄 1 , . . . , 𝑄 𝑚 ], Here 𝑅, 𝑄 ∈ R𝑑×𝑚𝑐 and
𝑅 = 𝐴 −1 𝑄 = 𝐻𝑄,
𝐴𝑅 = 𝑄.
(35)
Exact Recovery Theorem Theorem 1 (exact recovery from moment probes). Fix any 𝛾 > 0, which need not be known to the attacker, and consider C𝛾 = {( 𝐴, 𝐺) : 𝐴 = 𝑆 + 𝛾𝐼, 𝑆 ⪰ 0, 𝐺 ∈ R𝑑×𝑐 }. In exact arithmetic, the observations (𝑊0 , {𝑊 𝑗 , 𝑄 𝑗 } 𝑚 𝑗=1 ) uniquely identify ( 𝐴, 𝐺) over C𝛾 if and only if rank(𝑄) = 𝑑.
(36)
Because 𝐴 is invertible, this is equivalent to rank(𝑅) = 𝑑. When the condition holds, 𝐴 = 𝑄𝑅 † ,
𝐻 = 𝑅𝑄 † ,
𝐺 = 𝐴𝑊0 .
(37)
Here † denotes the Moore–Penrose pseudoinverse. When 𝛾 is unknown, these observations identify 𝐴 = 𝑆+𝛾𝐼 rather than 𝑆 separately: every 0 < 𝛾 ′ ≤ 𝜆min ( 𝐴) gives the feasible decomposition 𝑆 ′ = 𝐴 − 𝛾 ′ 𝐼 ⪰ 0. A fixed 𝛾 cancels when pre- and post-deletion values of 𝐴 are differenced. Proof of sufficiency. Since 𝑅 = 𝐴 −1 𝑄, the stacks have the same rank. At full row rank, right-multiplying 𝐴𝑅 = 𝑄 by 𝑅 † gives 𝐴 = 𝑄𝑅 † ; right-multiplying 𝑅 = 𝐻𝑄 by 𝑄 † gives 𝐻 = 𝑅𝑄 † ; and then 𝐺 = 𝐴𝑊0 . Thus the state is uniquely determined without inverting an estimated 𝐻. 14
Proof of necessity.
Suppose rank(𝑅) < 𝑑. Choose nonzero 𝑣 ∈ R𝑑 with 𝑣 ⊤ 𝑅 = 0. For any 𝜀 > 0, define 𝐴′ = 𝐴 + 𝜀𝑣𝑣 ⊤ ,
𝑆 ′ = 𝑆 + 𝜀𝑣𝑣 ⊤ ⪰ 0,
𝐺 ′ = 𝐴′𝑊0 .
Then 𝐴′ = 𝑆 ′ + 𝛾𝐼 ≻ 0, 𝐴′ ≠ 𝐴, and 𝐴′ 𝑅 = 𝐴𝑅 + 𝜀𝑣𝑣 ⊤ 𝑅 = 𝑄. Therefore, for every 𝑗, 𝐴′𝑊 𝑗 = 𝐴′ (𝑊0 + 𝑅 𝑗 ) = 𝐺 ′ + 𝑄 𝑗 , while 𝐴′𝑊0 = 𝐺 ′ . The distinct valid ridge state ( 𝐴′ , 𝐺 ′ ) produces the same baseline and all probe responses, so unique identification is impossible. This necessity statement is over the stated algebraic class; 𝐺 ′ need not be jointly realizable by a particular labelled dataset.
Exact Logical-Response Complexity If a logical response follows a total probe of rank at most 𝑟 max , then 𝑑 rank(𝑄) ≤ 𝑚𝑟 max , 𝑚≥ 𝑟 max
(38)
is necessary. For unrestricted 𝑑 × 𝑐 moment blocks, 𝑟 max = 𝑐, and the bound is achievable in the arbitrary chosen-summary model. Let 𝑚 = ⌈𝑑/𝑐⌉, choose a scale 𝜏 > 0, form 𝑄 = [𝜏𝐼 𝑑 , 0] ∈ R𝑑×𝑚𝑐 ,
(39)
and partition its columns into 𝑚 blocks. Then 𝑄𝑄 ⊤ = 𝜏 2 𝐼 𝑑 , so rank(𝑄) = 𝑑, 𝜎min (𝑄) = 𝜏, and 𝜅2 (𝑄) = 1. Thus the exact noiseless query complexity is ⌈𝑑/𝑐⌉ logical full-head responses per unknown state in this algebraic model; the statement does not exploit further dataset-realizability constraints. For 𝑑 = 512 and 𝑐 = 10, each unknown state requires exactly 52 logical probe responses, excluding its baseline. A first attack that identifies unknown states on both sides of a deletion uses 104 probe responses, one initial baseline, two cancellations, one deletion response, and one replay response, for 109 server responses in total. If the identified pre-deletion state is cached or restored, a subsequent event may require only 52 new post-state probes plus deletion, cancellation, and replay responses, for 55 in total. A logical response here follows a complete rank-up-to-ten block; if only one rank-one constituent may precede each broadcast, the corresponding lower bound is 𝑑 = 512 responses. Cumulative implementation. The theorem uses 𝑄 𝑗 as the total perturbation present when 𝑊 𝑗 is observed. Given chosen target totals with 𝑄 0 = 0, a cumulative implementation submits 𝐷 𝑗 = 𝑄 𝑗 − 𝑄 𝑗 −1 and, after observing the final response, submits only −𝑄 𝑚 .
B
Finite-Precision Recovery
Proposition 1 (direct response-error bounds).
Suppose the observed response stack is
e = 𝑅 + 𝐸 = 𝐻𝑄 + 𝐸, 𝑅
(40)
e 𝑗 = 𝑊 𝑗 + 𝑁 𝑗 and 𝑊 e0 = 𝑊0 + 𝑁0 , then the 𝑗th block where 𝑄 is known exactly and has full row rank. If 𝑊 b = 𝑅𝑄 e † of 𝐸 is 𝑁 𝑗 − 𝑁0 ; hence 𝐸 includes error in both probe responses and the shared baseline. Then 𝐻 satisfies b − 𝐻 = 𝐸𝑄 † , b − 𝐻 ∥ 𝐹 ≤ ∥𝐸 ∥ 𝐹 . 𝐻 ∥𝐻 (41) 𝜎min (𝑄) 15
e has full row rank, the direct estimator If 𝑅
b = 𝑄𝑅 e† 𝐴
satisfies b − 𝐴 = −𝐴𝐸 𝑅 e† , 𝐴
b − 𝐴∥ 𝐹 ≤ ∥𝐴
(42) ∥ 𝐴∥ 2 ∥𝐸 ∥ 𝐹 . e 𝜎min ( 𝑅)
(43)
b − 𝐻 = 𝐸𝑄 † , and the first bound follows from ∥𝑄 † ∥ 2 = 1/𝜎min (𝑄). Also Proof. Since 𝑄𝑄 † = 𝐼 𝑑 , 𝐻 e − 𝐸). Using 𝑅 e𝑅 e† = 𝐼 𝑑 gives 𝑄 = 𝐴𝑅 = 𝐴( 𝑅 b − 𝐴 = 𝐴( 𝑅 e − 𝐸) 𝑅 e† − 𝐴 = −𝐴𝐸 𝑅 e† , 𝐴 which proves the direct-𝐴 bound. bsym = sym( 𝐴 braw ). Since symmetrization is the orthogonal projection onto the symmetric matrices Let 𝐴 and 𝐴 is symmetric, bsym − 𝐴∥ 𝐹 ≤ ∥ 𝐴 braw − 𝐴∥ 𝐹 . ∥𝐴 (44) We report the equation residual before and after this projection, together with the minimum eigenvalue of bsym . Any positive-definite projection is disclosed separately. Although the probe scale 𝜏 does not change 𝐴 the condition number of the designed 𝑄, it controls signal-to-roundoff and cancellation drift.
C
Algebraically Admissible Moment Probes
Lemma 1 (arbitrary algebraic moment probes). Let 𝑒 𝑘 be the one-hot vector for class 𝑘. For any desired 𝑄 = [𝑞 1 , . . . , 𝑞 𝑐 ] ∈ R𝑑×𝑐 , choose 𝑢 𝑘 = 𝑞 𝑘 /2. An add summary induced algebraically by (𝑢 𝑘 , 𝑒 𝑘 ) and a delete summary induced by (−𝑢 𝑘 , 𝑒 𝑘 ) satisfy 𝑢 𝑘 𝑢 ⊤𝑘 − (−𝑢 𝑘 ) (−𝑢 𝑘 ) ⊤ = 0,
(45)
𝑢 𝑘 𝑒 ⊤𝑘 − (−𝑢 𝑘 )𝑒 ⊤𝑘 = 𝑞 𝑘 𝑒 ⊤𝑘 .
(46)
Summing over 𝑘 = 1, . . . , 𝑐 realizes 𝑄 with zero net Gram change. Each constituent Gram is positive semidefinite and each constituent moment is one-hot-label consistent. This proves algebraic admissibility only: it does not imply that both signed features are outputs of the shared encoder or that the deleted item belongs to the malicious client. The designed probe in Equation (39) is implemented by assigning its columns to these algebraic pairs. Its scale 𝜏 does not affect the condition number, but it affects signal-to-roundoff and cancellation drift. Lemma 2 (same-feature, different-label probes). The net Gram is zero and the net moment is 𝑄 = [𝑢 1 , . . . , 𝑢 𝑐−1 , −
For 𝑢 1 , . . . , 𝑢 𝑐−1 ∈ R𝑑 , add (𝑢 𝑘 , 𝑒 𝑘 ) and delete (𝑢 𝑘 , 𝑒 𝑐 ). Í𝑐−1
𝑘=1 𝑢 𝑘 ],
𝑄1𝑐 = 0.
(47)
Conversely, every 𝑄 satisfying 𝑄1𝑐 = 0 has this representation. This construction can reuse genuine attacker features and avoids requiring −𝑢 to be encoder-realizable, although deletion and label provenance remain unverified. Every resulting block has rank at most 𝑐 − 1. The corresponding lower bound is ⌈𝑑/(𝑐 − 1)⌉ = 57 responses for 𝑑 = 512, 𝑐 = 10; actual full rank additionally depends on the attacker’s feature span.
16
D
Known Gram and Moment Probes
Suppose probe 𝑗 creates known total Gram and moment perturbations (𝑃 𝑗 , 𝑄 𝑗 ) and returns 𝑊 𝑗 = ( 𝐴 + 𝑃 𝑗 ) −1 (𝐺 + 𝑄 𝑗 ),
𝑃 𝑗 = 𝑃⊤𝑗 ,
𝐴 + 𝑃 𝑗 ≻ 0.
(48)
Define 𝑋 𝑗 = 𝑊 𝑗 − 𝑊0 and 𝑍 𝑗 = 𝑄 𝑗 − 𝑃 𝑗 𝑊 𝑗 , and stack these blocks as 𝑋 and 𝑍. Subtracting 𝐴𝑊0 = 𝐺 from ( 𝐴 + 𝑃 𝑗 )𝑊 𝑗 = 𝐺 + 𝑄 𝑗 gives 𝐴𝑋 = 𝑍. (49) The state is uniquely identifiable if and only if rank(𝑋) = 𝑑; at full row rank, 𝐴 = 𝑍 𝑋 † and 𝐺 = 𝐴𝑊0 . For necessity, choose nonzero 𝑣 with 𝑣 ⊤ 𝑋 = 0 and set 𝐴′ = 𝐴 + 𝜀𝑣𝑣 ⊤ and 𝐺 ′ = 𝐴′𝑊0 . Then 𝐴′ 𝑋 = 𝐴𝑋 = 𝑍, so the same observations arise from a distinct feasible ridge state. For a cumulative attacker-owned batch ⊤ (𝐹 𝑗 , 𝑌 𝑗 ), 𝑃 𝑗 = 𝐹 ⊤ 𝑗 𝐹 𝑗 and 𝑄 𝑗 = 𝐹 𝑗 𝑌 𝑗 , giving 𝑍 𝑗 = 𝐹⊤ 𝑗 (𝑌 𝑗 − 𝐹 𝑗 𝑊 𝑗 ).
(50)
Let 𝐹attacker vertically stack all distinct attacker feature rows used in the sequence. Because 𝐴 is invertible, rank(𝑋) = rank(𝑍) ≤ rank(𝐹attacker ). Thus at least 𝑑 examples and full attacker feature rank are necessary, but not sufficient: the residuals 𝑌 𝑗 − 𝐹 𝑗 𝑊 𝑗 and the batching sequence also affect 𝑍. Label diversity is not required by this bound. The experiments below remove each cumulative addition sequence before continuing. e0 = Response-rounding error for genuine additions. Let the returned baseline and probe heads be 𝑊 e 𝑗 = 𝑊 𝑗 + 𝑁 𝑗 . Then 𝑋 e𝑗 = 𝑋 𝑗 + 𝑁 𝑗 − 𝑁0 and 𝑍 e𝑗 = 𝑍 𝑗 − 𝑃 𝑗 𝑁 𝑗 . If 𝑋 e has full row rank and 𝐵 𝑊0 + 𝑁0 and 𝑊 stacks the blocks 𝐵 𝑗 = −( 𝐴 + 𝑃 𝑗 )𝑁 𝑗 + 𝐴𝑁0 , (51) b= 𝑍 e𝑋 e† obeys the estimator 𝐴 b − 𝐴 = 𝐵𝑋 e† , 𝐴
b − 𝐴∥ 𝐹 ≤ ∥𝐴
∥𝐵∥ 𝐹 . e 𝜎min ( 𝑋)
(52)
e − 𝐴𝑋 e = 𝐵 and full row rank gives 𝑋 e𝑋 e† = 𝐼 𝑑 , which proves both the identity and the bound. Thus Indeed, 𝑍 cumulative Gram matrices enter the rounding-error numerator, while a better-conditioned response stack reduces the error. Increasing the probe size can affect both terms and is not unconditionally beneficial. The data-derived implementation applies the same numerical policy as Algorithm 1: it requires full numerical b= 𝑍 e and 𝑍, e symmetrizes the direct estimates 𝐴 e𝑋 e† and 𝐻 b= 𝑋 e𝑍 e† , and requires both to be rank of both 𝑋 positive definite. The 𝐻 estimate is only an agreement diagnostic; it is not inverted to obtain 𝐴. Cancellation residuals are reported separately and do not determine identification success.
E
Deleted-Update Recovery and Replay
Proposition 2 (exact deletion recovery and replay). Let two identified states immediately bracket one deletion: 𝐴pre = 𝑆pre + 𝛾𝐼, 𝐴post = 𝑆post + 𝛾𝐼. (53) Since the same fixed regularizer appears in both states, 𝑆del = 𝐴pre − 𝐴post .
17
(54)
Moreover 𝐺 = 𝐴𝑊, so 𝐺 del = 𝐴pre𝑊pre − 𝐴post𝑊post .
(55)
The deletion identities follow from 𝑆post = 𝑆pre − 𝑆del , 𝐺 post = 𝐺 pre − 𝐺 del , and 𝐺 = 𝐴𝑊. Adding the recovered pair to the post-deletion ledger yields 𝐴post + 𝑆del = 𝐴pre ,
𝐺 post + 𝐺 del = 𝐺 pre ,
so the replayed head is exactly 𝑊pre . If several events occur between the states, the formulas recover only their signed aggregate net change. If 𝛾 changes, the Gram difference is contaminated by (𝛾pre − 𝛾post )𝐼. If the two Gram-state estimates have errors 𝐸 pre and 𝐸 post , then 𝑆bdel − 𝑆del = 𝐸 pre − 𝐸 post , and therefore RelErr(𝑆del ) ≤
(56)
∥𝐸 pre ∥ 𝐹 + ∥𝐸 post ∥ 𝐹 . ∥𝑆del ∥ 𝐹
(57)
For comparable state-estimation errors, smaller deletion blocks are consequently harder to recover. This explains the observed ordering from class to client to sample deletion without claiming that deletion size is the only source of error. epre = 𝑊pre + 𝑁pre and 𝑊 epost = 𝑊post + 𝑁post , the deleted-moment error is If the observed baselines are 𝑊 exactly bdel − 𝐺 del = 𝐸 pre𝑊pre − 𝐸 post𝑊post 𝐺 (58) bpre 𝑁pre − 𝐴 bpost 𝑁post . +𝐴 For a deleted sample with feature 𝑓 , Proposition 3 therefore guarantees label recovery whenever the Frobenius norm of this right-hand side is below ∥ 𝑓 ∥ 2 /2. bpre = 𝐴pre + 𝐸 pre and 𝐴 bpost = 𝐴post + 𝐸 post . Replaying the Replay error from estimated states. Let 𝐴 corresponding estimated deleted block gives 𝑊replay − 𝑊pre = 𝐴pre + 𝐸 pre − 𝐸 post
−1
𝐸 post (𝑊pre − 𝑊post ).
(59)
breplay = 𝐴pre + 𝐸 pre − 𝐸 post and its moment is 𝐺 breplay = 𝐴pre𝑊pre + To derive the identity, the replayed state is 𝐴 breplay𝑊pre from this moment gives 𝐸 post (𝑊pre −𝑊post ); left-multiplication 𝐸 pre𝑊pre − 𝐸 post𝑊post . Subtracting 𝐴 −1 b by 𝐴 replay yields the result. Thus the pre-state error enters only through the inverse, while the post-state error epre = 𝑊pre + 𝑁pre and 𝑊 epost = 𝑊post + 𝑁post , the also appears in the numerator. With rounded baselines 𝑊 bpre 𝑁pre − 𝐴 bpost 𝑁post . numerator additionally contains 𝐴
F
Passive Heads Do Not Identify an Aggregate Deletion
Fix observed heads 𝑊pre and 𝑊post . For any 𝐴post ⪰ 𝛾𝐼 and any 𝐷 ⪰ 0, define 𝐴pre = 𝐴post + 𝐷,
𝐺 post = 𝐴post𝑊post ,
𝐺 pre = 𝐴pre𝑊pre .
(60)
These feasible ridge states produce the same two observed heads, while different choices of 𝐷 give different deleted Gram blocks 𝑆del = 𝐷 and moments 𝐺 del = 𝐺 pre −𝐺 post . Hence passive pre/post heads do not identify a general aggregate deletion over the algebraic state class. This statement concerns unrestricted aggregate blocks; additional single-sample structure can make passive reconstruction possible. 18
G
Sample-Level Leakage
Proposition 3 (exact and robust label/feature recovery). 𝑓 ∈ R𝑑 and one-hot label 𝑒 𝑘 , Δ𝐺 = 𝑓 𝑒 ⊤𝑘 .
For a deleted sample with nonzero feature
c = 𝑓 𝑒 ⊤ + 𝐸, define Thus the unique nonzero column identifies 𝑘 and equals 𝑓 . Under Δ𝐺 𝑘 b 𝑘 = arg
max
ℓ ∈ {1,...,𝑐}
c :ℓ ∥ 2 , ∥ Δ𝐺
c b. b 𝑓 = Δ𝐺 :𝑘
(61)
If ∥𝐸 :𝑘 ∥ 2 + max ∥𝐸 :ℓ ∥ 2 < ∥ 𝑓 ∥ 2 ,
(62)
ℓ≠𝑘
then b 𝑘 = 𝑘 and ∥ b 𝑓 − 𝑓 ∥ 2 = ∥𝐸 :𝑘 ∥ 2 . Indeed, ∥ 𝑓 + 𝐸 :𝑘 ∥ 2 ≥ ∥ 𝑓 ∥ 2 − ∥𝐸 :𝑘 ∥ 2 , whereas every incorrect column has norm ∥𝐸 :ℓ ∥ 2 . The simpler condition ∥𝐸 ∥ 𝐹 < ∥ 𝑓 ∥ 2 /2 is sufficient. Lemma 3 (information in a rank-one Gram). If 𝑓 ≠ 0 and 𝑔𝑔 ⊤ = 𝑓 𝑓 ⊤ , then 𝑔 = ± 𝑓 . Both outer products have the same one-dimensional column space, so 𝑔 = 𝑎 𝑓 ; equality then gives 𝑎 2 = 1. A single-sample Gram therefore identifies the feature up to one global sign, not an arbitrary orthogonal transformation.
H
Why Head Restoration Is Not a State Certificate
Lemma 4 (head-equivalent ridge states). For a state ( 𝐴, 𝐺) with head 𝑊 = 𝐴 −1 𝐺 and any nonzero 𝐶 ⪰ 0, or more generally any symmetric 𝐶 satisfying 𝑆 + 𝐶 ⪰ 0, define 𝐴′ = 𝐴 + 𝐶,
𝐺 ′ = 𝐺 + 𝐶𝑊 .
Then 𝐴′ = (𝑆 + 𝐶) + 𝛾𝐼 remains a feasible ridge state and ( 𝐴 + 𝐶)𝑊 = 𝐺 + 𝐶𝑊, so ( 𝐴′ , 𝐺 ′ ) has the same head. Therefore 𝑊final ≈ 𝑊0 is only a numerical consistency check; it cannot certify equality of the hidden ledgers or the absence of an intervening honest update.
I
Cumulative-Probe State-Recovery Algorithm
Numerical rank counts singular values larger than 𝜀rank 𝜎max . SubmitMomentProbe expands its argument into the constituent add/delete summaries of the algebraic probe construction; it does not modify the server ledger directly. The implementation sets 𝜀 rank = 10−10 and 𝜀 den = 10−15 . It symmetrizes both raw direct estimates but does not project either onto the positive-definite cone: a nonpositive minimum eigenvalue is a failed identification. The cancellation-head residual 𝑒 rec is diagnostic and does not determine identification success. In simulation, where hidden ledgers are available to the evaluator but not the attacker, cancellation is additionally checked using 𝑒𝑆 =
∥𝑆final − 𝑆0 ∥ 𝐹 , max(∥𝑆0 ∥ 𝐹 , 𝜀den )
𝑒𝐺 =
∥𝐺 final − 𝐺 0 ∥ 𝐹 , max(∥𝐺 0 ∥ 𝐹 , 𝜀den )
(63)
together with 𝑒 𝑊 = ∥𝑊final − 𝑊0 ∥ 𝐹 /max(∥𝑊0 ∥ 𝐹 , 𝜀den ). Equality of heads alone does not certify equality of ledgers. The true deleted Gram block is positive semidefinite, but numerical recovery need not preserve this property. The evaluation therefore reports 𝜆min ( 𝑆bdel ) and compares replay with the raw block against replay with its eigenvalue-clipped projection ΠPSD ( 𝑆bdel ). The raw replay is primary; projected replay is a diagnostic for a server that performs a basic PSD check. 19
Algorithm 1: Direct identification of one stable ridge state from cumulative moment probes. Each response is paired with its known total perturbation. The procedure cancels once with −𝑄 𝑚 , recovers 𝐴 and 𝐻 independently by SVD/QR solves, and reports rather than hides symmetry, definiteness, equation-residual, and agreement failures. 1: Input: baseline head 𝑊0 , increments {𝛿𝑄 𝑗 } 𝑚 𝑗=1 , rank tolerance 𝜀 rank , denominator floor 𝜀 den
b 𝐻, b numerical diagnostics, and identification flag b 𝐺, 2: Output: 𝐴, 3: 𝑄 tot ← 0 4: for 𝑗 = 1, . . . , 𝑚 do
𝑄 tot ← 𝑄 tot + 𝛿𝑄 𝑗 𝑊 𝑗 ← SubmitMomentProbe(𝛿𝑄 𝑗 ) 7: 𝑄 𝑗 ← 𝑄 tot ; 𝑅 𝑗 ← 𝑊 𝑗 − 𝑊0 8: end for 9: 𝑄 ← [𝑄 1 , . . . , 𝑄 𝑚 ]; 𝑅 ← [𝑅1 , . . . , 𝑅 𝑚 ] 10: 𝑄 𝑚 ← 𝑄 tot ; 𝑊final ← SubmitMomentProbe(−𝑄 𝑚 ) 11: if rank 𝜀rank (𝑄) < 𝑑 or rank 𝜀rank (𝑅) < 𝑑 then 12: return failure: state is not identifiable 13: end if braw ← 𝑄𝑅 † ; 𝐻 braw ← 𝑅𝑄 † using SVD/QR 14: 𝐴 braw 𝑅 − 𝑄∥ 𝐹 /max(∥𝑄∥ 𝐹 , 𝜀den ) 15: 𝑒 𝐴𝑅 ← ∥ 𝐴 braw 𝑄∥ 𝐹 /max(∥𝑅∥ 𝐹 , 𝜀den ) 16: 𝑒 𝐻𝑄 ← ∥𝑅 − 𝐻 braw and 𝐻 braw 17: Record relative asymmetry of 𝐴 ⊤ b ← (𝐴 braw + 𝐴 braw )/2 18: 𝐴 ⊤ )/2 b b braw 19: 𝐻 ← ( 𝐻raw + 𝐻 b b 20: 𝜆 𝐴 ← 𝜆 min ( 𝐴); 𝜆 𝐻 ← 𝜆 min ( 𝐻) 21: if 𝜆 𝐴 ≤ 0 or 𝜆 𝐻 ≤ 0 then 22: return failure: unstable recovered state 23: end if √ b𝐻 b − 𝐼𝑑 ∥ 𝐹 / 𝑑 24: 𝑒 𝐴𝐻 ← ∥ 𝐴 b ← 𝐴𝑊 b 0 25: 𝐺 26: 𝑒 rec ← ∥𝑊final − 𝑊0 ∥ 𝐹 /max(∥𝑊0 ∥ 𝐹 , 𝜀 den ) 27: return estimates, all diagnostics, and identification success 5: 6:
20
J
Experimental Protocol and Additional Results
The experiments were conducted on workstation machines equipped with multi-core CPUs and GPUs. The first machine used an AMD Ryzen 9 5900X GPU (12 cores, 24 threads) with 62 GB of system memory and an NVIDIA RTX A6000 GPU with 48 GB of VRAM. The second machine was configured with an Intel Core i9-11900K CPU (8 cores, 16 threads) and an NVIDIA RTX 3090 GPU with 24 GB of VRAM. The third machine used an Intel Core i7-14700K CPU (20 cores, 28 threads), 62 GB of system memory, and an NVIDIA RTX A5000 GPU with 24 GB of VRAM. For the most compute-intensive experiments, including DINOv2 feature extraction and numerical-rank checks, we additionally employed a multi-GPU server with dual AMD EPYC 7742 CPUs (128 cores, 256 threads in total), 1 TB of system memory, and eight NVIDIA A100 SXM4 GPUs, each with 80 GB of VRAM. Images are resized to 224×224 pixels and normalized with the ImageNet-1K mean and standard deviation; MNIST images are first converted to three channels. The primary encoder is ImageNet-1K ResNet-18 with its final fully connected layer replaced by the identity, so the feature is the 512-dimensional output after global average pooling. Feature extraction uses batches of 256. The ridge head has no intercept. The designed sample experiment selects 100 targets without replacement using seed 7; the class experiment deletes each of the ten classes; and the client experiment deletes all 50 clients in each of 20 partitions seeded 0–19. For each of five data-derived seeds, sample targets are 100 nonattacker examples selected without replacement, class targets are all ten classes, and client targets are 20 nonattacker clients. The data-derived and DINOv2 checks use seeds 0–4. The server stores its ledger and solves the ridge system in float64. “Float32 heads” means that only returned heads and their baselines are rounded to float32 before the attacker receives them; table abbreviations f64 and f32 denote these two response precisions. Numerical rank initially uses relative tolerance 10−10 ; a tolerance sensitivity check is reported below. Each dataset is divided among 50 clients with Dirichlet concentration 𝛼 = 0.05; the assignment routine makes at most 1,000 attempts and stops with an error rather than accepting a split with fewer than ten samples for any client. Every evaluated split passes this check. For each data-derived split, client 0 is used if its features have rank 512; otherwise the lowest-index fullrank client is selected. The selected MNIST clients contain 867–4,309 examples and the selected CIFAR-10 clients 779–5,342; 25–33 and 25–28 of the 50 clients, respectively, satisfy the full-feature-rank gate. All selected examples are permuted and assigned without overlap to 104 batches, giving 8–42 samples per MNIST batch and 7–52 per CIFAR-10 batch. The same examples are reused for the post-deletion sequence. They are also present in the initial training ledger, so the additions are attacker-owned and encoder-realizable but are duplicate submissions rather than held-out new records. The interface does not reject duplicates. A first complete attack uses 208 addition messages, two cancellations, one honest deletion, and one replay: 212 client messages and 213 server responses including the initial baseline. For each deletion, the program identifies and cancels a pre-deletion probe sequence, applies the deletion through the server interface, identifies and cancels a post-deletion sequence, and replays the recovered block from the actual post-cancellation state. The attacker uses only returned heads and its known probe summaries. The simulator’s true ledger is used to perform the designated deletion and to score recovery, cancellation, and replay. Errors are averaged only over successful identifications; failures remain in the denominator of every success count.
Designed-Probe Precision Table 4 gives the float32 counterpart of the main float64 table. State identification passes in every trial, but differencing two estimated states amplifies error relative to a small deletion. Designed float32 probes therefore retain all CIFAR-10 labels and 92 of 100 MNIST labels, while aggregate errors increase with finer deletion size. 21
Table 4: Recovery when the server returns float32 heads for the deterministic designed probe with 52 responses per state. Relative errors are mean ± standard deviation over successful attacks. The descriptive 95% Clopper–Pearson intervals are [0.8484, 0.9648] for MNIST and [0.9638, 1] for CIFAR-10; they treat trials sharing one server state as independent Bernoulli outcomes. Data
Deletion
Success
Labels
RelErr(Δ𝐺)
RelErr(Δ𝑆)
MNIST
sample class client
100/100 10/10 1000/1000
92/100 – –
(1.09±.81) (2.14±1.34) × 10−4 (1.17±3.01) × 10−2
(3.14±2.74) (5.76±4.03) × 10−4 (2.89±7.67) × 10−2
CIFAR-10
sample class client
100/100 10/10 1000/1000
100/100 – –
(2.48±1.82) × 10−1 (5.68±2.90) × 10−5 (2.85±8.52) × 10−3
(6.39±5.22) × 10−1 (1.62±.89) × 10−4 (6.74±18.27) × 10−3
Table 5: Recovery with cumulative attacker-data additions. Errors are mean ± standard deviation over successful attacks only; failed rank tests are excluded from these means but included in “Success.” The displayed Clopper–Pearson intervals treat pooled trials as independent and are descriptive because trials share probe states within each of five seeds. Data
Heads, 𝑚 Deletion
Success
Labels
RelErr(Δ𝐺)
RelErr(Δ𝑆)
(4.57±4.11) × 10−9
(5.95±4.79) × 10−9 (6.28±5.46) × 10−10
MNIST f64, 52 CIFAR-10 f64, 52
class class
14/50 16/50
– –
MNIST
sample class client sample class client
500/500 50/50 100/100 500/500 50/50 100/100
500/500 [.9926, 1] – – 500/500 [.9926, 1] – –
sample class client sample class client
500/500 49/500 [.0734, .1275] 50/50 – 100/100 – 500/500 183/500 [.3237, .4099] 50/50 – 100/100 –
f64, 104
CIFAR-10 f64, 104
MNIST
f32, 104
CIFAR-10 f32, 104
(3.65±2.86) × 10−10
(1.55±1.78) × 10−6 (1.62±2.02) × 10−6 (2.87±3.30) × 10−10 (3.73±4.78) × 10−10 (1.08±3.02) × 10−8 (8.76±19.49) × 10−9 (9.43±6.36) × 10−8 (1.24±.89) × 10−7 (2.09±1.35) × 10−11 (2.97±1.91) × 10−11 (1.36±3.16) × 10−9 (1.85±4.25) × 10−9 68.4±97.7 (1.32±1.86) × 10−2 .393±1.03 5.27±4.28 (1.16±.87) × 10−3 .0842±.215
143±254 (2.74±4.70) × 10−2 .661±1.97 10.1±11.1 (2.42±2.18) × 10−3 .161±.528
Although the designed perturbation stack has condition number one at 𝜏 = 104 , the returned response stack does not. Its condition number is 1.064 × 106 on MNIST and 4.148 × 104 on CIFAR-10. In float64, the corresponding direct state errors are 1.506 × 10−12 and 6.549 × 10−13 . The response stack, rather than the designed input alone, therefore controls finite-precision recovery.
Attacker-Data Additions Table 5 reports the complete data-derived deletion summary. Each row pools five probe seeds. The 𝑚 = 52 experiment targets class deletion because removing a class is the sharpest test of whether the post-deletion response stack keeps full rank. At 𝑚 = 104 and tolerance 10−10 , all sample, class, and client stacks attain rank 512. The float32 results show that a successful numerical-rank test does not guarantee accurate identification. At 𝑚 = 52, all five pre-deletion states have full rank, but most post-class-deletion states have rank 468; the observed range is 468–512. At 𝑚 = 104, median response-stack condition numbers are 4.06 × 105 on MNIST and 1.43×105 on CIFAR-10. Float32 sample-label rates by seed are 7–14% on MNIST and 12–100% on CIFAR-10, so the pooled CIFAR-10 rate hides substantial state dependence. Full attacker feature rank
22
Table 6: Numerical-rank sensitivity of float32 attacker-data response stacks. Each row contains ten matrices: one pre-deletion and one post-class-deletion stack for each of five seeds. Entries are the number with full rank 512 under the relative singular-value threshold shown. The smallest-singular-value range includes all ten matrices. Data
𝑚
10−6 10−7 10−8 10−9 10−10
MNIST MNIST CIFAR-10 CIFAR-10
52 104 52 104
0 6 1 10
2 10 5 10
6 10 6 10
7 10 6 10
7 10 6 10
𝜎min range 2.35 × 10−21 –5.29 × 10−8 1.24 × 10−7 –2.38 × 10−6 2.37 × 10−20 –7.23 × 10−7 6.73 × 10−7 –1.08 × 10−5
Table 7: Float64 state-identification diagnostics. 𝑒 𝐴𝑅 and 𝑒 𝐻𝑄 are relative residuals of the two solved matrix √ b𝐻 b − 𝐼 ∥ 𝐹 / 𝑑 measures agreement between independently estimated regularized state equations; 𝑒 𝐴𝐻 = ∥ 𝐴 and inverse. Positive minimum eigenvalues confirm both estimates pass the definiteness checks. Probe
Data
Designed Designed Attacker data Attacker data
MNIST CIFAR-10 MNIST CIFAR-10
𝑒 𝐴𝑅
𝑒 𝐻𝑄
𝑒 𝐴𝐻
b 𝜆min ( 𝐴)
b 𝜆 min ( 𝐻)
3.30 × 10−12 4.33 × 10−13 8.84 × 10−13 9.20 × 10−14
2.07 × 10−16 5.25 × 10−16 1.37 × 10−11 2.82 × 10−12
2.25 × 10−12 3.42 × 10−13 5.53 × 10−11 6.53 × 10−12
23.2 638 23.2 638
4.05 × 10−8 3.78 × 10−8 4.05 × 10−8 3.78 × 10−8
remains necessary but is not sufficient. Table 6 tests whether float32 quantization merely creates tiny singular values that pass the original 10−10 numerical-rank threshold. For each dataset and batch count, it recomputes one pre-deletion and one post-class-deletion response stack for each of five seeds. At 𝑚 = 104, all stacks remain full rank from 10−7 through 10−10 . At 10−6 , all CIFAR-10 stacks but only six of ten MNIST stacks are full rank. Thus the 𝑚 = 104 conclusion is stable at a tolerance comparable to float32 unit roundoff, but some MNIST directions are marginal under a ten-times-larger threshold. In all float32 tables, “Success” means that the estimator returned after the stated numerical-rank and positive-definiteness checks; it does not imply accurate state recovery.
Numerical Diagnostics Table 7 reports the state-equation checks promised in the main paper. The designed rows evaluate the common baseline state used by the deletion experiments; the attacker-data rows summarize the five float64 baseline identifications. For the latter, residuals are maxima and eigenvalues are minima, giving the least favorable value across seeds. The simulator-only cancellation checks are similarly small. Across all successful float64 attacker-data sample, class, and client experiments, the maxima of (𝑒 𝑆 , 𝑒 𝐺 , 𝑒 𝑊 ) are (5.06 × 10−16 , 3.24 × 10−16 , 7.89 × 10−13 ) on MNIST and (5.20 × 10−16 , 3.61 × 10−16 , 2.07 × 10−13 ) on CIFAR-10. These values use the hidden ledgers only for evaluation; the recovery procedure never reads them. Small negative eigenvalues occur for some sample and client estimates, so a server that rejects every non-PSD submitted block would reject those raw replays. Projection removes this numerical violation and leaves the recovered block accurate, but it no longer reproduces the head as closely. The projected replay errors in Table 8 remain below 6 × 10−9 in float64.
23
Table 8: Float64 replay and positive-semidefiniteness diagnostics for attacker-data additions. Values are means over successful trials except the minimum eigenvalue, which is the worst case. “Raw/PSD” compares the recovered deleted Gram block with the same block after clipping negative eigenvalues to zero. Moment error is unchanged by this projection. Replay error compares each resulting head with the pre-deletion head. Data
Level
MNIST
sample class client CIFAR-10 sample class client
min 𝜆( 𝑆bdel )
RelErr(𝑆) raw/PSD
RelErr(𝐺)
Replay raw/PSD
−1.42 × 10−3
(3.22/3.10) × 10−11
8.73 × 10−11
−1.17 × 10−4 −1.06 × 10−4 1.50 × 101 −4.00 × 10−5
(3.37/3.37) × 10−11 (3.31/2.99) × 10−12 (3.21/3.21) × 10−12 (3.32/3.30) × 10−12
8.25 × 10−11 6.73 × 10−12 6.59 × 10−12 6.51 × 10−12
1.03 × 10−12 /5.61 × 10−9 (2.36/2.36) × 10−10 3.06 × 10−11 /7.20 × 10−10 2.32 × 10−13 /3.68 × 10−10 (3.42/3.42) × 10−11 4.16 × 10−12 /7.86 × 10−11
1.06 × 10−1
(3.95/3.95) × 10−11
9.05 × 10−11
Table 9: Relative error of the recovered deleted feature, conditioned on the maximum-column rule selecting the correct sample label. Rows give the number of correctly labelled samples and the mean, median, and maximum feature error. Data
Heads
MNIST CIFAR-10 MNIST CIFAR-10
float64 float64 float32 float32
Correct 500 500 49 183
Mean
Median
Maximum
4.32 × 10−7
2.00 × 10−7
4.01 × 101 1.14
1.58 × 101 2.14 × 10−1
3.95 × 10−6 1.99 × 10−7 2.32 × 102 8.15
2.75 × 10−8
2.36 × 10−8
DINOv2 Encoder Robustness We replace ResNet-18 with the frozen DINOv2 ViT-B/14 encoder Oquab et al. [2024], increasing 𝑑 from 512 to 768 and the dimensional lower bound from 52 to 77 responses. This experiment identifies one state and does not rerun the deletion or decoder evaluations. Table 10 shows the same distinction as the main experiment: the dimensional count is necessary but does not guarantee observed rank for attacker-data additions. All 104-response estimates are positive definite; their relative 𝐺 errors range from 1.87 × 10−11 to 5.67 × 10−11 on MNIST and from 3.26 × 10−12 to 8.61 × 10−12 on CIFAR-10. The raw replay-head errors for float64 sample/class/client deletion are 1.03 × 10−12 , 2.36 × 10−10 , and 3.06 × 10−11 on MNIST, and 2.32 × 10−13 , 3.42 × 10−11 , and 4.16 × 10−12 on CIFAR-10. With float32 heads, they are 1.54 × 10−7 , 4.16 × 10−4 , and 5.72 × 10−5 on MNIST, and 5.40 × 10−8 , 1.33 × 10−4 , and 1.62 × 10−5 on CIFAR-10. A small head error does not certify a correct replayed ledger, as proved in Lemma 4. Table 10: One-state identification from attacker-data additions with frozen 768-dimensional DINOv2 ViTB/14 features. Each row summarizes five independently seeded client splits and probe sequences. A usable attacker has 768-dimensional feature rank and at least 77 samples. Ranges are minima and maxima over the five runs; errors are omitted when numerical rank fails. Data
𝑚
MNIST MNIST CIFAR-10 CIFAR-10
77 104 77 104
Success rank(𝑋) Usable 0/5 5/5 0/5 5/5
762–766 768 746–756 768
𝜅(𝑋)
RelErr( 𝐴)
20–26 – – 20–26 (1.51, 4.34) × 106 (4.14 × 10−12 , 2.50 × 10−11 ) 21–26 – – 21–26 (3.23 × 105 , 1.12 × 106 ) (3.17, 8.53) × 10−12
24
Table 11: Fixed MNIST float64 scale sweep for designed probes, with 20 sample deletions per scale. RelErr( 𝐴) is the pre-deletion state-identification error. “Update ratio” and “head ratio” divide the largest malicious request and head change by the corresponding honest 99th percentile. 𝜏 Success 100 102 104 106 108 1010 1012
20/20 20/20 20/20 20/20 20/20 20/20 20/20
RelErr( 𝐴)
RelErr(Δ𝑆)
RelErr(Δ𝐺) Update ratio
Head ratio
1.57 × 10−9
1.00 × 10−4
3.46 × 10−5
5.15 × 10−3
1.91 × 10−11 1.51 × 10−12 1.24 × 10−12 7.96 × 10−13 3.72 × 10−13 4.27 × 10−13
1.01 × 10−6 1.01 × 10−7 1.05 × 10−7 1.05 × 10−7 7.43 × 10−8 8.20 × 10−8
3.53 × 10−7 3.50 × 10−8 3.72 × 10−8 4.07 × 10−8 5.80 × 10−7 3.67 × 10−5
2.98 × 101 2.98 × 105 2.98 × 109 2.98 × 1013 2.98 × 1017 2.98 × 1021
8.15 8.15 × 102 8.15 × 104 8.15 × 106 8.15 × 108 8.15 × 1010 8.15 × 1012
MNIST
CIFAR-10
Target 𝛼 = 0.05 𝛼 = 0.5 IID Exact feature
Figure 2: Selected qualitative decoder outputs for MNIST (middle) and CIFAR-10 (right). Rows use decoders trained only on one malicious client’s local image–feature pairs under strongly non-IID (𝛼 = 0.05), moderately non-IID (𝛼 = 0.5), or independent and identically distributed (IID) client partitions. The final row decodes the exact target feature with the IID decoder and separates feature-recovery error from decoder loss. These selected examples are not a quantitative reconstruction evaluation.
Designed-Probe Scale and Detectability For an add/delete request, the update magnitude is the combined Frobenius norm of its four submitted summary matrices. Table 11 compares the largest designed request and its largest induced head change with the 99th percentiles of honest requests. All 20 deletions succeed at every scale, but most scales are conspicuous. At 𝜏 = 104 , clipping the request magnitude to the honest 99th percentile retains 20/20 success, with mean state errors 7.07 × 10−11 before and 4.37 × 10−11 after deletion, deleted-𝑆/𝐺 errors 3.68 × 10−6 /1.76 × 10−6 , and a head change still 149 times the honest 99th percentile. For attacker-data additions, batch-size-specific 99th-percentile norm thresholds are calibrated on 5,000 honest messages per dataset and evaluated on 10,000 separate honest messages. Held-out false-positive rates are 1.38% on MNIST and 1.34% on CIFAR-10. The thresholds flag 0 of 520 MNIST additions and 13 of 520 CIFAR-10 additions; the median probe-to-threshold ratios are 0.818 and 0.845. These per-message results exclude the two cumulative cancellations and replay and do not test temporal or frequency-based detection. Rounding a final honest ResNet-18 ridge head from float64 to float32 changes no prediction among 10,000 test examples on either dataset. Test accuracy remains 97.15% on MNIST and 85.94% on CIFAR-10, and the relative head-rounding errors are 2.57 × 10−8 on both datasets. This evaluates inference from the rounded broadcast, not the utility of training a server ledger in float32.
K
Additional Reconstruction and Replay Figures
Recovering frozen feature representations from the head exposes an additional channel for coarse visual leakage. To illustrate this potential security risk, we train feature decoders using (1) a local decoder and 25
MNIST
CIFAR-10
Target image LDM reconstruction
Figure 3: Feature-conditioned latent diffusion reconstructions for MNIST (middle column) and CIFAR-10 (right column). The top row contains target images and the bottom row contains samples generated from recovered frozen features by a model trained on auxiliary TinyImageNet data; these examples test coarse visual leakage rather than faithful pixel recovery.
Figure 4: Test accuracy during ten sequential class deletions with float64 designed probes on MNIST (left) and CIFAR-10 (right). The honest branch retains every deletion; the attacked branch immediately replays each recovered class block before the next deletion. Final honest/attacked accuracies are 9.82%/97.15% and 8.94%/85.94%, respectively.
Figure 5: Client-level replay over 50 sequential deletion rounds on MNIST (left) and CIFAR-10 (right). The honest branch retains every client deletion; the attacked branch immediately replays each recovered client block before the next round. Thin curves show 20 independently sampled client partitions, and emphasized curves show their medians.
26
(2) a decoder trained on auxiliary data. The local decoder used for each partition is trained only on the selected malicious client’s actual images and their frozen features; it does not use the probe construction or any target image. Figure 2 shows selected outputs. We use a lightweight multilayer perceptron (MLP) for feature decoding, trained with a standard supervised pixel-level reconstruction objective. In addition to this locally trained MLP decoder, we also report reconstructions from a conditional latent diffusion decoder trained on auxiliary TinyImageNet data, which conditions a UNet-based latent denoiser using feature-derived tokens. Figure 3 shows that although the domain shift prevents high-fidelity reconstructions, coarse visual leakage may still be visible. Overall, the reconstructions from local decoder and auxiliary data decoder demonstrate that, once a malicious client recovers frozen features, they can induce coarse visual leakage or reveal individual digit identities in MNIST examples. We include these examples only to illustrate coarse visual leakage, and do not claim faithful pixel-level recovery. Figure 4 and Figure 5 illustrate statistics replay after each class- and client-level deletion. The blue curves show test accuracy under immediate probing and reinsertion after every unlearning step, while the red curves show the effect of honest unlearning without replay. Reinserting the estimated aggregate block restores test accuracy to its pre-deletion level.
References Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. Transactions on Machine Learning Research, 2024.
27