Efficient and Robust Online Learning to Rank in Decentralized Systems Research Marcel Gregoriadis
TU Delft Delft, Netherlands [email protected]
Martijn de Vos
EPFL Lausanne, Switzerland [email protected]
arXiv:2606.12246v1 [cs.DC] 10 Jun 2026
Anne-Marie Kermarrec
EPFL Lausanne, Switzerland [email protected]
Abstract In Online Learning to Rank (OLTR), ranking models are trained directly from live user interactions, but existing systems rely on a trusted central server to collect and process these interactions. This leaves operators free to introduce biases that conflict with user interests. Decentralized learning offers an attractive alternative, allowing users to collaboratively train a shared ranking model by exchanging model updates directly with one another, without any central authority. In such settings, however, malicious nodes can send poisoned model updates that degrade the ranking quality of honest nodes. We introduce RankGuard, a decentralized OLTR framework in which users collaboratively train ranking models and exchange model updates directly with other nodes. RankGuard defends against poisoning attacks by carefully evaluating incoming models against the user’s own private click history, corrected for position bias. An incoming model is only aggregated if it better explains the user’s past interactions than the current local model, making it fundamentally hard for malicious nodes to craft updates that pass this test without also genuinely helping the user. We derive a theoretical convergence guarantee of RankGuard. To the best of our knowledge, this is the first formal convergence analysis of a decentralized OLTR algorithm. We evaluate RankGuard against four poisoning attacks, including a powerful adaptive attack, using four standard benchmarks and three click models. RankGuard outperforms all baselines in most settings while being up to 62× more efficient than its closest competitors.
CCS Concepts • Information systems → Learning to rank; • Security and privacy → Distributed systems security; • Computing methodologies → Online learning settings; Learning from implicit feedback.
Keywords Byzantine Robustness, Collaborative Machine Learning, Decentralized Online Learning to Rank, Gossip Learning
1
Introduction
Search rankings shape what people see and believe [24]. Most users never scroll past the first few search results [52]. Therefore, a nudge to these results has the power to steer consumer purchases [34,
Sayan Biswas
EPFL Lausanne, Switzerland [email protected]
Johan Pouwelse
TU Delft Delft, Netherlands [email protected] 58], influence health decisions [54], shape beliefs about contested topics [21], and even sway election outcomes [16, 24, 25]. Today, this power is concentrated in the hands of a few search engine operators, such as Google and Baidu. This allows them to introduce biases that conflict with user objectives, e.g., for commercial gain [30]. The factors that drive these ranking decisions remain opaque to the user. This motivates an alternative system that does not rely on a centralized authority for ranking search results. Modern ranking systems are trained from user interactions with search engine result pages (SERPs). A SERP presents the user with an ordered list of documents matched to their query and ranked by predicted relevance. Within this domain, Online Learning to Rank (OLTR) is a prominent paradigm for training ranking models directly from live user interactions [41, 50]. OLTR treats user interactions (typically, clicks) on these documents as implicit relevance signals to iteratively update the ranker, typically via machine learning (ML). Most OLTR systems rely on a central server to aggregate click feedback and serve rankings. Federated Online Learning to Rank (FOLTR) [45, 59] keeps click data on the user’s device and shares only model updates, but still relies on a trusted central server to aggregate those updates and distribute the resulting ranker. Decentralized learning (DL) sidesteps the need for a central server entirely [18, 51]. With DL, nodes train locally and exchange model updates directly with one another via peer-to-peer protocols, aggregating received updates into their own models. While this removes dependence on a central server, it exposes nodes to attacks from any other node in the network [22]. In particular, adversarial nodes can send maliciously crafted model updates that, once aggregated, degrade other users’ rankers. This attack is known as an untargeted poisoning attack, and has primarily been studied in the context of federated learning (FL) [28]. Yet, existing defenses were designed for federated settings with a central server and do not transfer cleanly to decentralized OLTR, where no such server exists and nodes must independently assess the trustworthiness of received model updates. Therefore, the problem of robust OLTR in decentralized settings remains unexplored. To the best of our knowledge, this work is the first to address this issue. To demonstrate the impact of malicious nodes on the ranking accuracy of honest nodes, we conduct an experiment. We set up a 100-node network with 20 % of these nodes performing three
Preprint
Gregoriadis et al.
Ranking acc. [nDCG@10]
No attack
Flip attack
LIE attack
IPM attack
0.4 0.3 0.2
RankGuard outperforms all baselines in the majority of settings. We demonstrate that RankGuard is 62× more computationally efficient than its closest competitors, making it practical for deployment in large-scale decentralized systems. Thus, RankGuard lays the groundwork for fully decentralized search mechanisms that are both robust to manipulation and free from centralized control. Contributions. This work makes the following contributions:
0.1 0.0
0
5000
10000 15000 20000 Search sessions
25000
(1) We introduce RankGuard, an efficient mechanism for robust decentralized OLTR. RankGuard thwarts poisoning attacks by having nodes carefully judge incoming model updates against their own session history (Section 3). (2) We provide the first formal convergence analysis of a decentralized OLTR. Specifically, we introduce a novel theoretical approach to account for the effect of the dynamic aggregation weights determined by each node to quality-gate the models they receive to derive the theoretical convergence bounds for RankGuard (Section 4). (3) We extensively evaluate RankGuard across four learningto-rank benchmarks, three click models, and four poisoning attacks, demonstrating that it outperforms all baselines in the majority of settings while being up to 62× more efficient than its closest competitors (Section 5 and Section 6).
30000
Figure 1: Ranking accuracy of honest nodes as users interact with the system with 100 nodes of which 20 send malicious model updates. standard poisoning attacks (Flip, LIE and IPM). After each session, a node updates its model and shares it with 7 random nodes, who aggregate it locally. The full experimental setup is provided in Section 5. Figure 1 shows the average ranking accuracy of honest nodes over time in terms of nDCG@10, a standard accuracy metric in OLTR that scores how well the top-10 results match ideal relevance ordering (1 = perfect, 0 = worst). Under all three attacks, malicious nodes significantly degrade ranking accuracy for honest nodes. Notably, after 30 000 search sessions, average accuracy drops from 0.42 to 0.09 under the Flip attack, rendering the ranking system unusable. This experiment demonstrates that ranking manipulation is a serious concern in decentralized OLTR. Decentralized platforms that would naturally benefit from robust decentralized OLTR already exist. For example, Filecoin must rank storage providers across a distributed network by price, reliability, and latency [38], and DTube must surface relevant videos without a central server [19]. For such systems, a centralized ranking component would reintroduce the very bottleneck and trust assumptions that decentralization is designed to eliminate, making a robust decentralized OLTR mechanism essential. We address this gap and introduce RankGuard, a robust aggregation mechanism designed specifically for OLTR in decentralized settings. The key idea is to turn each user’s own interaction history into a reference: when a model update is received, we evaluate how well it explains the user’s past interactions, corrected for position bias, and compare this against the user’s current local model. This comparison is formalized as a statistical test whose outcome determines how much of the received model is integrated in the current local model. A model that genuinely improves ranking quality will better explain the user’s past clicks and be up-weighted, whereas malicious model updates, trained against the user’s interests, will score poorly and be discarded. This makes it fundamentally challenging for malicious nodes to craft model updates that pass this test without also improving the ranking models of honest users. We implement RankGuard and conduct an extensive experimental evaluation across four standard learning-to-rank benchmarks (WEB30K, MQ2007, Yahoo, and Istella), three user behavior models, and with four poisoning attacks of increasing sophistication, including a challenging adaptive attack that assumes oracle knowledge, tailored specifically to RankGuard. Our results show that
2
Background and Problem Description
This section introduces the necessary background for RankGuard. We begin with an overview of OLTR and then describe Pairwise Differentiable Gradient Descent (PDGD), a state-of-the-art OLTR algorithm underlying RankGuard. An overview of the notation used in this paper is provided in Appendix A.
2.1
(Online) Learning to Rank
Learning to Rank (LTR) is the task of training an ML model to order a set of documents by relevance to a query [49]. Formally, given a query 𝑞 and a candidate document set 𝐷, a ranking model 𝑓𝜃 assigns a score 𝑓𝜃 (𝒅) to each document 𝒅 ∈ 𝐷. Such a ranking model typically operates on a feature vector representing the querydocument pair, capturing relevance signals such as BM25 score or TF-IDF. Documents are then presented to the user in decreasing score order. Training such a model traditionally requires editorial relevance judgements, in which human assessors assign relevance labels to query-document pairs. These labels serve as supervision for pointwise, pairwise, or listwise loss functions [11, 13]. Even though manual editorial annotation is effective, it is very laborintensive. Moreover, it does not account for personal preferences, as well as concept drift. Online Learning to Rank (OLTR) addresses these limitations by learning relevance rankings directly from implicit user feedback (most commonly clicks) collected during live interaction with the system [50, 68]. An OLTR system presents ranked lists to users from a provided query, observes which documents are clicked, and updates the model accordingly. This makes OLTR particularly attractive in settings where annotation is unavailable or where relevance preferences are personal and evolve over time. A central challenge in OLTR is position bias: users are more likely to examine and click documents ranked near the top, regardless of their true relevance. Therefore, naive treatment of clicks as relevance labels introduces
Preprint
Gregoriadis et al.
systematic bias into the model. Counterfactual approaches [44] and pairwise methods such as Dueling Bandit Gradient Descent [68] and PDGD [50] address this by reweighting clicks or sidestepping the bias through interleaving.
PDGD treats the 𝜌-weighted sum of pair-preference probabilities as a utility to maximize, and ascends its gradient:
∇𝑓𝜃 (·) ≈
2.2
Pairwise Differentiable Gradient Descent
𝑘 Ö
(1)
𝑃 (𝒅 𝒊 | 𝐷 \ {𝒅 1, . . . , 𝒅 𝒊−1 })
𝑖=1
The probabilistic sampling of a ranking serves two purposes: (1) it introduces controlled exploration, giving lower-scored documents a chance to be observed, (2) and it enables counterfactual reasoning about alternative orderings, which underpins the position-bias correction described next. The user may click one or multiple documents in the displayed ranking. PDGD assumes that clicks indicate a preference for the respective documents over the non-clicked documents in the list, albeit influenced by position bias. As a caveat, non-clicked documents are only considered if they were also examined. A simple but common strategy is to assume all documents up to the clicked document plus the subsequent document (if available) were examined. This strategy is based on eye-tracking studies [43]. Specifically, PDGD derives a set of pairwise preferences 𝒅 𝒊 >𝑐 𝒅𝒋 ∈ P between each clicked document 𝒅 𝒊 and non-clicked but examined document 𝒅𝒋 in the result list. Each pairwise preference provides a biased signal. To quantify this bias, PDGD proposes to contemplate the probability of the swapped ranking 𝑅 ∗ . The swapped ranking is the counterfactual: the same ranking with 𝒅 𝒊 and 𝒅𝒋 in opposite positions. Comparing 𝑃 (𝑅 | 𝐷) to 𝑃 (𝑅 ∗ | 𝐷) reveals how much the ranker, rather than chance, drove 𝑑𝑖 above 𝑑 𝑗 . When 𝑃 (𝑅 | 𝐷) ≫ 𝑃 (𝑅 ∗ | 𝐷), the ranker dictated the order and a click on 𝑑𝑖 largely reflects position bias; when the two are comparable, the user clicked 𝑑𝑖 against the ranker’s pull, i.e., the user’s click carries genuine preference signal. This allows us to quantify the position bias in each preference pair 𝒅 𝒊 >𝑐 𝒅𝒋 as: 𝜌 (𝒅 𝒊 , 𝒅𝒋 , 𝑅, 𝐷) =
𝑃 𝑅 ∗ (𝒅 𝒊 , 𝒅𝒋 , 𝑅) | 𝐷
𝑃 (𝑅 | 𝐷) + 𝑃 𝑅 ∗ (𝒅 𝒊 , 𝒅𝒋 , 𝑅) | 𝐷
.
𝜌 (𝒅 𝒊 , 𝒅𝒋 , 𝑅, 𝐷) · ∇𝑃 (𝒅 𝒊 ≻ 𝒅𝒋 ).
(3)
𝒅 𝒊 >𝑐 𝒅𝒋 ∈ P
Our work builds on PDGD [50], which provides a principled approach to debiasing user clicks and represents the state of the art in OLTR. It consists of two key ideas: probabilistic ranking from a sampling distribution, and a pairwise optimization strategy using a position-bias reweighting function applied to pairwise click preferences. PDGD considers a pointwise scoring function 𝑓𝜃 (𝒅) : R𝑑 ↦→ [0, 1], where 𝒅 ∈ R𝑑 represents the features of a query-document pair in a 𝑑-dimensional space. The learning goal is to find the parameters 𝜃 for which the ranker displays an optimal ranking of documents in 𝐷. A higher score 𝑓𝜃 (𝒅) indicates higher predicted relevance of 𝒅 ∈ 𝐷. Rather than sorting documents directly by score, PDGD uses the scores to define a probability distribution over docexp(𝑓𝜃 (𝒅 ) ) uments using the Plackett-Luce model 𝑃 (𝒅 | 𝐷) = Í ′ exp(𝑓 ′ . 𝜃 (𝒅 ) ) 𝒅 ∈𝐷 A ranking 𝑅 of length 𝑘 is then created by sampling from the distribution 𝑘 times. This allows us to model the probability of a ranking:
𝑃 (𝑅 | 𝐷) =
∑︁
(2)
Oosterhuis and de Rijke [50] proved that the resulting gradient is unbiased with respect to the user’s true pairwise preferences. PDGD therefore yields a principled OLTR update from each search session, and forms the local training step in decentralized OLTR.
2.3
Decentralized Learning
Decentralized learning (DL) is a collaborative ML paradigm in which a set of nodes jointly train a shared ML model without coordination by a central server [5]. In DL, training data never leaves the device; only model updates are exchanged, offering privacy benefits over centralized approaches. This property is particularly valuable in the OLTR setting, where user queries and click behavior can reveal sensitive information such as health concerns, financial interests, or political beliefs. In most DL algorithms, nodes maintain a local model, train it on their own data held locally, and periodically exchange model updates with a subset of other nodes in a communication graph. Upon receiving model updates, nodes aggregate them into their local model, typically via a weighted average, and this process repeats until convergence. A standard protocol in this domain is Decentralized Parallel Stochastic Gradient Descent (D-PSGD) [48], in which all nodes exchange their models with a subset of neighbors every round. DL is increasingly being adopted in high-stakes domains such as healthcare and finance [61]. However, D-PSGD and its derivatives are ill-suited for the OLTR setting for three reasons. First, they assume that each node’s training data remains fixed during training, whereas in OLTR, local datasets grow incrementally as users issue queries and generate clicks. Second, D-PSGD typically assumes a fixed communication topology, which is difficult to maintain in large decentralized networks where nodes join and leave freely and no central authority exists to coordinate topology management. Third, D-PSGD requires all nodes to exchange models synchronously every round, which presupposes that nodes are continuously available. In practice, users issue queries at irregular intervals and may leave and rejoin the network freely, a phenomenon known as churn. Gossip learning (GL) [51] is a class of DL algorithms in which nodes independently and periodically exchange their model with randomly selected nodes in the network. This process relies on a Random Peer Sampler (RPS) to sample communication partners. Unlike D-PSGD, GL is fully asynchronous: nodes contribute to and benefit from the federation whenever they are active, without waiting for other nodes to synchronize their models with. This makes GL naturally churn-resilient, as nodes joining or leaving the network do not stall the training process. Communication in GL is push-based: upon completing a local update, a node pushes its current model to a small set of sampled nodes, which aggregate the received model into their own upon receipt. These properties make GL a natural fit for the OLTR setting.
Preprint
2.4
Poisoning Attacks in Collaborative Learning
A key concern in any collaborative ML system are poisoning attacks: nodes can send maliciously crafted updates to degrade the model utility of other nodes. In the context of OLTR, such attacks can degrade ranking quality across the network, harming user experience or steering users toward irrelevant or manipulated results. Poisoning attacks have been studied extensively in FL, where a central server observes all client updates simultaneously and can apply population-level statistical filters to detect and discard outliers [28]. In DL, however, such server is not available and nodes instead receive updates only from a small, changing subset of other nodes and must assess their trustworthiness independently. While some robustness algorithms have been proposed for DL [29, 32, 40], they are designed for synchronous D-PSGD-like protocols. Robustness against poisoning in GL-like algorithms, where communication is asynchronous, push-based, and driven by irregular user activity, remains an open problem. Thus, the research question that this work answers is: How can a node in a decentralized, gossip-based OLTR system reliably distinguish between beneficial and malicious model updates, without access to a trusted server, or knowledge of which peers are malicious?
3
The Design of RankGuard
We present RankGuard, a novel and decentralized mechanism for OLTR that provides robustness against malicious nodes that attempt to corrupt the ranking of other nodes by spreading malicious model updates. The RankGuard protocol is shown in Algorithm 1 and the full workflow is visualized in Figure 2.
3.1
Gregoriadis et al. Algorithm 1: RankGuard from the perspective of node 𝑢 1 2
Initialize 𝜃𝑢 ← RandomInit()
OnQuery query 𝑞 (𝑞) 𝐷𝑢 ← RetrieveDocuments(𝑞) 5 𝑅 (𝑞) ← RankDocuments(𝜃𝑢 , 𝐷 (𝑞) )
3
4
(𝑞)
6 7 8 9
P𝑢 ← GeneratePairwiseClickPreferences(𝑅 (𝑞) ) ∇𝑓𝜃𝑢 ← 0 foreach 𝒅 𝒊 >𝑐 𝒅𝒋 ∈ P (𝑞) do (𝑞) 𝜌𝑖 𝑗 ← EstimateBias(𝒅 𝒊 , 𝒅𝒋 , 𝑅, 𝐷 (𝑞) ) // Equation (2) (𝑞)
10 11 12 13 14
∇𝑓𝜃𝑢 ← ∇𝑓𝜃𝑢 + 𝜌𝑖 𝑗 ∇𝑃 (𝒅 𝒊 ≻ 𝒅𝒋 |𝐷 (𝑞) , 𝜃𝑢 ) 𝜃𝑢+ ← 𝜃𝑢 + 𝜂∇𝑓𝜃𝑢 foreach 𝑝 ∈ SamplePeers() do Send(𝑝, 𝜃𝑢 ) 𝑄𝑢 ← 𝑄𝑢 ∪ {𝑞}
OnReceive model weights 𝜃 𝑣 16 for 𝑞 ∈ 𝑄𝑢 do 17 𝑋𝑞 ← 𝑆 (𝑞, 𝜃 𝑣 ) − 𝑆 (𝑞, 𝜃𝑢 ) √︁ 18 𝑡 ← |𝑄𝑢 | · 𝑋¯ /𝑠𝑋 19 𝛼 ← 𝜎 (𝜅𝑡) 20 𝜃𝑢+ ← (1 − 𝛼) · 𝜃𝑢 + 𝛼 · 𝜃 𝑣
15
// Equation (5)
System and Threat Model
Before describing the design of RankGuard, we outline our system and threat model. Network and nodes. We consider a network of 𝑛 nodes, each interacting with a search engine through queries and clicks. Nodes are connected through a dynamic communication graph in which every node knows about ⌈log2 𝑛⌉ other nodes. This is a commonly chosen fanout value in the domain of DL that is known to converge well, and therefore follows standard practice [18]. Each node has access to a RPS which is a service that provides nodes with a renewed set of nodes, sampled uniformly from the network population [37]. We assume the RPS delivers an unbiased random sample of peers and that messages between honest users are delivered intact. Recent work shows that continuously refreshing the set of neighboring nodes shrinks the attack surface of malicious nodes [4, 57]. Importantly, our work does not require any central server and all components of RankGuard and dependencies can be implemented using decentralized primitives. Search sessions. When a user issues a query 𝑞, they are presented with a ranked list of candidate documents 𝑅. The user examines 𝑅 from top to bottom and clicks documents according to a click model. The click model determines, based on each document’s true relevance label, the click probability and a stop probability (i.e., the chance that the user ends the session after a click and examines no further documents). Click models [15] are a standard tool for reproducing realistic behavior patterns such as position bias and noisy, relevance-dependent clicks without requiring live users. We
simulate search sessions in round-robin order across all nodes; in each round, every node processes a single session. Threat model. We assume that there is a fraction of malicious nodes 𝛽 in the network: they deviate from the protocol in Algorithm 1 by sending poisoned model updates. We make no assumption about which nodes are malicious, and honest users do not know 𝛽. For evaluation purposes, however, baselines that do require 𝛽 are given its true value, granting them an advantage that our system does not require. The set of malicious nodes remains fixed for the duration of each experiment run. We focus on untargeted poisoning attacks, in which malicious nodes aim to indiscriminately degrade ranking quality across all honest users. Untargeted poisoning attacks constitute a natural worst-case for an availability-oriented threat model: a successful attacker prevents the federation from converging to a useful ranking model, nullifying the benefits of collaboration. More specifically, malicious nodes follow the protocol but send carefully crafted model updates to other nodes (Step 5 in Figure 2). We acknowledge that the threats in decentralized systems extend beyond poisoning attacks, e.g., Sybil attacks against the RPS [20] or eclipse attacks [56]. Deploying defenses on the network layer is an orthogonal problem that can be solved by using robust peer samplers [1, 3]. Similarly, we consider privacy attacks (e.g., membership inference [57]) on the datasets of individual nodes beyond our scope. Our threat model therefore concerns the effectiveness of the trained ranking model, not the confidentiality of training data.
Preprint
Gregoriadis et al.
On user query
Document 1
Retrieve
1
2
Generate ranking
Search query
Document 3
Receive
6
7 Evaluate quality
5
3 Click 2
Preference click pairs Local model
4
Click 1
Document 4
Local ranking model
On incoming model
Document 2
Share Pairwise preferences
...
User
User
Other nodes
Session
Candidate documents
8
Train model
Pairwise aggregate
9
Session performance scores
20%
80%
New local model
Incoming model
Figure 2: The workflow of RankGuard, our decentralized framework for robust OLTR.
3.2
RankGuard in a Nutshell
The key question when doing OLTR using GL is how a node should weigh a single model received from another node when there is no trusted server that has access to multiple models to run and no way to tell which nodes are malicious. RankGuard answers this by treating each node’s own click history as a private, trusted reference. Our main insight is that an honest node accumulates reliable records of which documents they clicked on past result pages. While this record may be noisy (e.g., because a user misclicks or misinterprets search results), it cannot be exploited by an attacker. So instead of asking whether a received model resembles the models sent by other nodes, as done by traditional statistical defenses [10, 40, 67], a node using RankGuard determines whether the received model explains its own past clicks better than its current model does. A model that genuinely improves the local ranking quality is aggregated into the local model, whereas one trained to disrupt honest rankings is effectively discarded. RankGuard does so by efficiently computing session performance scores, after correcting for position bias. This pairwise model judgment is made entirely using the user’s local click history. RankGuard therefore needs neither separate validation data nor any knowledge of how many peers are malicious. It also avoids a common failure mode of statistical defenses, where honest updates can be diverse enough to be mistaken for malicious ones, particularly when local datasets are very diverse between nodes.
3.3
The RankGuard Workflow
We next present the full workflow of RankGuard, primarily driven by two events: a node performing a search (and clicking items), and a node receiving a model from another node. Importantly, every node 𝑢 maintains a local ranking model 𝜃𝑢 trained on their own click stream via PDGD (see Section 2.2). 𝜃𝑢 is randomly initialized by node 𝑢 and is locally optimized with their session history 𝑄𝑢 . Each (𝑞) 𝑞 ∈ 𝑄𝑢 stores the candidate documents 𝐷𝑢 , click pair preferences (𝑞) (𝑞) (𝑞) P𝑢 , and for each 𝒅 𝒊 >𝑐 𝒅𝒋 ∈ P𝑢 , the click pair bias 𝜌𝑖 𝑗 .
3.3.1 On user search. When node 𝑢 performs a search query 𝑞 (Step 1 in Figure 2), the search engine first retrieves a candidate document (𝑞) set 𝐷𝑢 that is relevant to the search query (line 4 in Algorithm 1). We remark that these documents may not be stored locally by node 𝑢 and could be fetched by other nodes, e.g., when using decentralized (𝑞) file-sharing applications such as IPFS [6]. Documents in 𝐷𝑢 are (𝑞) then ranked by 𝜃𝑢 , producing a ranked list 𝑅 (Step 2). Node 𝑢 then examines the list and clicks on one or more documents (Step 3). When the search session is completed, the node first infers a set of (𝑞) pairwise preferences P𝑢 while also estimating the position bias (𝑞) 𝜌𝑖 𝑗 for each click pair, then trains 𝜃𝑢 locally on these pairwise preferences, and updates the resulting model using PDGD with learning rate 𝜂 (Step 4). After this, node 𝑢 samples ⌈log2 𝑛⌉ other nodes from the network, using the RPS service, and sends the updated model 𝜃𝑢 to these sampled nodes. 3.3.2 Receiving a model. When node 𝑢 receives a model from another node 𝑣 (Step 6 in Figure 2), it will judge the quality of the received model and aggregate it if the received model has good quality. We do so by replaying both the local model 𝜃𝑢 the received model 𝜃 𝑣 on the node’s click history to assess whether, and by how much, 𝜃 𝑣 outperforms the user’s current model 𝜃𝑢 (Step 7). Note that past clicks do not constitute clean relevance signals, as they are influenced by position bias. Furthermore, recall that rankings are generated from a probabilistic distribution. Thus, the produced rankings by the evaluator are partly shaped by chance. A good evaluation function should therefore answer: What is the probability that the received model produces a ranking consistent with the user’s preferences, after correcting for position bias? To explain the design of our evaluation function, we first assume that click signals are unbiased. For each 𝑢 ∈ [𝑛], node 𝑢’s model performance in a single search session 𝑞 ∈ 𝑄𝑢 can then be measured as the probability that all clicked documents are sampled before the examined non-clicked documents, and therefore ranked above them. For all 𝑢 ∈ [𝑛], we can express this as the sum of log-probabilities: ∑︁ (𝑞) 𝑃 (𝑞 | 𝜃𝑢 , 𝑞 ∈ 𝑄𝑢 ) = log 𝑃 (𝒅 𝒊 ≻ 𝒅𝒋 | 𝐷𝑢 , 𝜃𝑢 ) (4) (𝑞)
𝒅 𝒊 >𝑐 𝒅𝒋 ∈ P𝑢
Preprint
Gregoriadis et al.
In other words, 𝑃 (𝑞 | 𝜃 𝑖 , 𝑞 ∈ 𝑄𝑢 ) quantifies how well the local model of each node explains a given search session. However, the click preferences inferred from the search session are biased. From Equation (2), we know how to quantify this bias. The bias is neutral (𝑞) (i.e., the preference pair 𝒅 𝒊 >𝑐 𝒅𝒋 is unbiased) when 𝜌𝑖 𝑗 = 0.5. A value close to 0 indicates that the position bias favored 𝒅 𝒊 , whereas a value close to 1 indicates that it favored 𝒅𝒋 . A value close to 1 yields a stronger relevance signal: the node preferred 𝒅 𝒊 despite the bias favoring 𝒅𝒋 . (𝑞) We propose to weigh each click preference pair by 𝜌𝑖 𝑗 . This, for any node 𝑢, gives us the session performance score 𝑆 (𝑞, 𝜃𝑢 ) for search session 𝑞 using model 𝜃𝑢 : ∑︁ (𝑞) (𝑞) 𝑆 (𝑞, 𝜃𝑢 ) = 𝜌𝑖 𝑗 log 𝑃 (𝒅 𝒊 ≻ 𝒅𝒋 | 𝐷𝑢 , 𝜃𝑢 ) (5) (𝑞)
𝒅 𝒊 >𝑐 𝒅𝒋 ∈ P𝑢
Thus, a high predicted probability for 𝒅 𝒊 ≻ 𝒅𝒋 contributes less to the score when 𝒅 𝒊 already had a positional advantage. If a received model 𝜃 𝑣 from node 𝑣 has a better average session performance score than the local model 𝜃𝑢 of node 𝑢, we assume it better captures the node’s preferences. We quantify this improvement as 1 ∑︁ Δ𝑆 = 𝑆 (𝑞, 𝜃 𝑗 ) − 𝑆 (𝑞, 𝜃 𝑖 ) . (6) |𝑄𝑢 | 𝑞 ∈𝑄 𝑢
A positive value of Δ𝑆 indicates that the received model better explains the node’s click preferences, corrected for position bias. However, we argue that Δ𝑆 should not be treated as a binary accept or reject criterion. Even a negative value of Δ𝑆 may carry useful signal if the query history is small and the assessment is therefore uncertain. To quantify the confidence of our performance evaluation, we compute a per-query score 𝑋𝑞 = 𝑆 (𝑞, 𝜃 𝑣 ) − 𝑆 (𝑞, 𝜃𝑢 ). We √︁ then derive the one-sample 𝑡-statistic 𝑡 = |𝑄𝑢 |𝑋¯ /𝑠𝑋 , which accounts for both the magnitude of the improvement and the number of queries. We then map 𝑡 to an interpolation weight 𝛼 = 𝜎 (𝜅𝑡), where 𝜎 is the sigmoid function and 𝜅 controls sensitivity, and update the local model 𝜃𝑢 as 𝜃𝑢+ ← (1 − 𝛼) · 𝜃𝑢 + 𝛼 · 𝜃 𝑣 . Thus, we aggregate the incoming model into the local model, weighted by the performance of the incoming model (Steps 8 and 9). We find that 𝜅 = 4 works best across datasets, click models, and attacks. When the value of Δ𝑆 is large and well-supported by many queries, the value of 𝑡 is large, 𝛼 approaches 1, and 𝜃 𝑣 largely replaces 𝜃𝑢 . When evidence is weak or contradictory, 𝛼 remains near 0 and the local model is mostly preserved. In this way, RankGuard effectively turns each node’s own interaction history into a shield: malicious models, trained against the node’s interests, will score poorly on that history and be absorbed with negligible weight.
4
Convergence Analysis of RankGuard
One of the key non-trivialities in deriving the convergence bound for RankGuard lies in the fact that it cannot simply adapt the analysis of existing DL algorithms. Because each node 𝑢 independently computes a dynamic weight for incoming models in every local round 𝑡 using a statistical 𝑡-test, the underlying aggregation scheme is fundamentally altered, breaking standard machinery used in typical DL convergence analysis, and, in turn, requiring a novel approach to handle the shifting properties of the mixing matrix. We now theoretically analyze the convergence of RankGuard. In the
interest of space, we present the sketch proof of Theorem 4, deriving the convergence rate of RankGuard, in Appendix B. We proceed by making the following standard assumptions on the learning objective, in line with the existing literature of convergence analysis in DL [7–9, 18, 26, 46]. Assumption 1 (Smoothness). For any model 𝜃 , the scoring function 𝑓𝜃 is differentiable and there exists 𝐿 < ∞ s.t. for all 𝑥, 𝑦 ∈ R𝑑 , ||∇𝑓𝜃 (𝑦) − ∇𝑓𝜃 (𝑥)|| ≤ 𝐿||𝑦 − 𝑥 ||. Assumption 2 (Bounded noise). For any model 𝜃 , there exists 𝜎 < ∞ s.t. for every node 𝑢, we have: E𝑥 ∈𝐷 (𝑞) ||∇𝑓𝜃 (𝑥) − 𝑓𝜃 (𝑥)|| 2 ≤ 𝜎 2 . 𝑢
Assumption 3 (Bounded heterogeneity). For any model 𝜃 , there exists H < ∞ such that for all 𝑥 ∈ R𝑑 , we have: h i 1 ∑︁ 2 E𝑥∼𝐷 (𝑞) ∇𝜃 𝑓𝜃 (𝑥) − ∇𝜃 𝐹𝑞 (𝜃 ) ≤ H 2, 𝑢 𝑛 𝑢 ∈ [𝑛] 1 ∑︁ E𝑥∼𝐷 (𝑞) 𝑓𝜃 (𝑥) . where 𝐹𝑞 (𝜃 ) = 𝑢 𝑛 𝑢 ∈ [𝑛]
In addition to Assumptions 1, 2, and 3, we assume that, for every node, each session comprises exactly one click. Under these above assumptions, we formalize the workflow of RankGuard and define the notion of a local round for each node as follows. Let the local model of any node 𝑢 ∈ [𝑛] in its local round 𝑡𝑢 ≥ 0 be denoted by 𝜃𝑢(𝑡𝑢 ) . Then, node 𝑢 in its local round 𝑡𝑢 (𝑎) receives 𝜃 𝑣(𝑡 𝑣 ) from another node 𝑣 in the network operating in its local round 𝑡 𝑣 (can be different from 𝑡𝑢 ), (𝑏) performs the weighted aggregation to incorporate 𝜃 𝑣(𝑡 𝑣 ) into its own local model 𝜃𝑢(𝑡𝑢 ) to produce (𝑡 + 1 )
(𝑡𝑢 ) (𝑡𝑢 ) (𝑡 𝑣 ) (𝑡𝑢 ) 𝜃𝑢 𝑢 2 = (1−𝛼𝑢𝑣 ) ·𝜃𝑢(𝑡𝑢 ) +𝛼𝑢𝑣 ·𝜃 𝑣 , where 𝛼𝑢𝑣 is computed as described in line 19 in Algorithm 1, and (𝑐) finally performs the local (𝑡 + 1 )
optimization using PDGD to obtain 𝜃𝑢(𝑡𝑢 +1) = 𝜃𝑢 𝑢 2 + 𝜂∇𝑓 (𝑡𝑢 + 1 ) 𝜃𝑢
2
as given in lines 3-12 in Algorithm 1. We are now in a position to state the main convergence result of RankGuard. Theorem 4 (Convergence of RankGuard).o Let Δ0 ∈ R s.t. n Δ0 ≥ max𝑢 ∈ [𝑛] max𝒅 ∈𝐷 (𝑞) 𝐹𝜃 (0) (𝒅) − min𝜃 𝐹𝜃 (𝒅) . If all the above 𝑢 𝑢 h i (𝑡𝑢 ) assumptions hold and if for all 𝑢, 𝑣 ∈ [𝑛] and 𝑡𝑢 > 0, E 𝛼𝑢𝑣 = 𝛼ˆ h i (𝑡𝑢 ) and Var 𝛼𝑢𝑣 = 𝜎𝛼2ˆ , then, for a constant step-size 𝜂, we have 𝑇 −1 2 1 ∑︁ 1 ∑︁ E𝒅 ∈𝐷 (𝑞) ∇𝐹𝜃 (𝑡𝑢 ) (𝒅) ∈ 𝑢 𝑢 𝑛 𝑇 𝑡 ,...,𝑡 =0 𝑢 ∈ [𝑛] 𝑛 1 ! √︂ 2 2 𝑛 1/3 𝐿 Δ0 𝐿 2 Δ20 𝐿Δ0 𝑠∗ 𝐿Δ0 O + ∗ 2 1− − + , 𝑛𝑇 𝑠𝑇 𝑛−1 (𝑛 − 1)𝑇 2 𝑇 where 𝑠 ∗ =
ˆ − 2𝜎𝛼2 2𝛼ˆ (1 − 𝛼) . ˆ 2 + 𝛼ˆ 2 + 2𝜎𝛼2ˆ (1 − 𝛼)
Remark 5. The convergence rate derived in Theorem 4 essentially demonstrates how the different design aspects of RankGuard fundamentally interact with the theoretical convergence guarantees of RankGuard. Crucially, we note that having 𝛼ˆ = 1/2, i.e., if the nodes aggregate their own models and the received models with an equal weight without performing any quality check of the received models,
Preprint
Gregoriadis et al.
we obtain 𝑠 ∗ = 1, thus boiling down to the exact convergence rate of ELLocal. Conversely, if 𝛼ˆ is 0 or 1, i.e., the nodes never incorporate the models they receive, or always replace their local models with the received models, respectively, the convergence rate, in expectation, blows up, ruining the utility, as we would expect. Moreover, we observe that, keeping everything else constant, the convergence rate’s asymptotic dependence on aggregation weight determined by each node to validate 2𝛼ˆ (1−𝛼ˆ ) −2𝜎 2
the quality of the received models is via O exp − (1−𝛼ˆ ) 2 +𝛼ˆ 2 +2𝜎𝛼 2
𝛼ˆ
as the total number of nodes in the network tends to infinity. Finally, it is noteworthy that the convergence rate of RankGuard is directly proportional to the smoothness factor 𝐿 of each node’s local objective function, and the maximum initialization error Δ0 , as is observed in convergence rates of standard DL algorithms.
5
Experimental Setup
We evaluate the robustness and efficiency of RankGuard against four poisoning attacks and compare with five state-of-the-art baselines. This section describes our experimental setup, including attacks, defense baselines, datasets, and implementation details. Throughout this section, we use 𝑢 to refer to the node receiving a model update, and 𝑣 to refer to the node sending it to 𝑢.
5.1
Poisoning Attacks
We run our experiments using one data poisoning attack native to OLTR and two state-of-the-art model poisoning attacks that have become standard benchmarks in the Byzantine-robust learning literature [3, 28, 40, 65]. We also experiment with a stronger adaptive attack. Together, these attacks span the spectrum from weak, realistic threats to strong, oracle-knowledge attacks. Data Poisoning (Flip). The adversarial click model introduced in Wang and Zuccon [60] is an attack in federated OLTR. Malicious nodes simulate interactions using a poison click model in which click probability is inversely proportional to document relevance: highly irrelevant documents are clicked with near certainty, while relevant documents are almost never clicked. The resulting pairwise preferences push PDGD updates away from the honest gradient; the corrupted model weights are then pushed to other nodes. This attack assumes no knowledge by the adversary about the victim node, making it a realistic lower bound on attacker capability. Little Is Enough (LIE). The LIE attack exploits the coordinatewise variance among honest peer models to craft perturbations that remain statistically indistinguishable from benign contributions [2]. The main idea is that statistics-based defenses assume malicious updates will be outliers; LIE invalidates this assumption by operating within the empirical spread of honest updates, bounding the perturbation to stay within the defenders’ acceptance region while maximally shifting the aggregate in a chosen direction. Concretely, attackers compute the coordinate-wise mean 𝜇 𝑗 and standard deviation 𝜎 𝑗 of the honest peers’ current models and transmit (𝜃 mal ) 𝑗 = 𝜇 𝑗 − 𝑧 max · 𝜎 𝑗 , where 𝑧 max is a perturbation factor derived from 𝑛 and 𝛽 such that a majority of honest models still lies farther from 𝜇 than 𝜃 mal , ensuring defenses preferentially retain the poisoned values. Because RankGuard is designed for pairwise aggregation, there is no population of incoming models from which 𝜇 and 𝜎 could be derived. Instead, we let the attacker
derive those statistics from the current models of all honest nodes in the network. This has important implications about the quality of the attack. Recall that in each round, every node processes a single session in round-robin order. In expectation, therefore, all honest nodes start a round with equally good models; by the end of a round, most honest nodes are only one session ahead of the current node. Because honest nodes are near-synchronized, the LIE perturbation reduces to a near-honest model plus a small noise term, rendering the attack closer to Gaussian noise. Inner Product Manipulation (IPM). Similar to LIE, IPM [62] directly targets Byzantine-robust aggregation methods. The IPM attack evades statistics-based defenses by sending model updates of bounded magnitude that point opposite to the honest gradient direction. When such updates are averaged with honest contributions, the aggregated update can have a negative inner product with the true gradient, causing the model to ascend rather than descend the loss. Xie et al. [62] show that this is especially damaging late in training: as the honest gradient shrinks near convergence, even small adversarial deviations suffice to flip the sign of the aggregate. We adapt IPM to our setting and assume a malicious node 𝑣 knows the local model 𝜃𝑢 of victim node 𝑢. Node 𝑣 trains 𝜃 𝑣 on a single search session to obtain a gradient 𝑔, and updates 𝜃 𝑣 using −𝜖𝑔 instead of 𝑔. Node 𝑣 then sends 𝜃 𝑣 to victim 𝑢. The victim thus receives an update that opposes the direction of honest learning. We use 𝜖 = 10, as preliminary experiments showed only marginal ranking degradation at smaller values. Adaptive Attack (Adapt). To stress-test the robustness of RankGuard, we develop an adaptive attack inspired by Fang et al. [28], who formulate model poisoning as an optimization problem against a known defense. Our goal is to establish an upper bound on attacker capability under full knowledge: the attacker has access to the victim’s model 𝜃𝑢 and session history 𝑄𝑢 . We let the attacker craft a poisoned model by minimizing a loss with two terms. The first term rewards alignment of the induced ˆ which we define as update 𝜃 𝑣 − 𝜃𝑢 with a harmful direction 𝑔, the negative of the victim’s honest descent direction. The attacker computes this exactly by simulating the victim’s next PDGD step on 𝑄𝑢 , and taking its opposite, yielding the harmful gradient: ∑︁ 𝑔ˆ = −∇𝜃 𝑆 (𝑞, 𝜃 ) 𝑞 ∈𝑄𝑢
𝜃 =𝜃𝑢
We measure alignment as the negative cosine similarity and define the loss term ˆ ⟨𝜃 𝑣 − 𝜃𝑢 , 𝑔⟩ Lharm (𝜃 𝑣 ) = − , ˆ +𝜖 ∥𝜃 𝑣 − 𝜃𝑢 ∥ ∥𝑔∥ where 𝜖 > 0 is a small constant to prevent division by zero. A second loss term penalizes rejection by RankGuard: 2 Laccept (𝜃 𝑣 ) = 1 − 𝛼 (𝜃 𝑣 ) For this, let 𝛼 (𝜃 𝑣 ) denote the weight 𝛼 the victim assigns to 𝜃 𝑣 given 𝜃𝑢 and 𝑄. Finally, the attacker solves 𝜃 𝑣∗ = arg min Lharm (𝜃 𝑣 ) + Laccept (𝜃 𝑣 ) 𝜃𝑣
Practically, the attack proceeds in two phases: (1) We first warm-start 𝜃 𝑣 by overfitting on 𝑄 from 𝜃𝑢 until the loss stabilizes. At this point, 𝜃 𝑣 already ranks 𝑄 well, so the optimizer can move it in harmful directions without immediately collapsing 𝛼. (2) We then
Preprint
Gregoriadis et al.
optimize 𝜃 𝑣∗ . Both phases use the same gradient update rule and learning rate that is used for honest PDGD training. Our implementation runs 100/|𝑄 | epochs for the warm-start phase and 50 steps for the loss optimization; preliminary experiments showed both losses stabilized within these budgets. The resulting 𝜃 𝑣∗ is sent to the victim, concluding one round of the adaptive attack. Repeated across multiple rounds, this procedure constitutes the strongest poisoning attack against RankGuard that we consider.
5.2
Defense Baselines
We compare RankGuard against five defense baselines: three defenses based on robust statistics and two defenses based on a private reference dataset. Together, these five baselines span the two dominant paradigms in Byzantine-robust aggregation: statistical filtering and trusted-reference scoring. Thus, they comprise a representative point of comparison against RankGuard. CS, GTS, and CWTM. We include three state-of-the-art methods based on robust statistics: CS, GTS, and CWTM. CS clips each update toward a robust center before averaging; GTS discards the updates farthest from that center; CWTM trims the top and bottom 𝛽 fraction per coordinate and averages the rest. Note that these algorithms require a pool of candidate models to make aggregation decisions, typically assuming knowledge of the fraction of malicious models 𝛽. To accommodate them, we let nodes buffer incoming models and, once the buffer reaches size ⌈log2 𝑛⌉, apply the respective aggregation rule and clear the buffer. We assume that 𝛽 is known by all nodes, so that their performance demonstrates an upper bound on the defenses’ capability. FLTrust. When a node receives a model 𝜃 𝑣 , it computes a candidate update 𝑔 = 𝜃 𝑣 − 𝜃𝑢 . It then crafts a trusted reference update 𝑟 = 𝜃ˆ𝑢 − 𝜃𝑢 where 𝜃ˆ𝑢 is created by training 𝜃𝑢 for one epoch on the local session history 𝑄𝑢 . Subsequently, it computes the ReLU ⟨𝑔,𝑟 ⟩
clipped cosine trust score 𝜏 = max 0, ∥𝑔 ∥ ∥𝑟 ∥ . If either update has zero norm or 𝜏 = 0, candidate 𝑔 is rejected. Otherwise, 𝑔 is rescaled to match the magnitude of the trusted reference and applied: 𝜃𝑢 ← 𝜃𝑢 +𝑔 · ∥𝑟 ∥/∥𝑔∥. In the original FLTrust protocol, trust scores weight a server-side aggregate over multiple client updates; in RankGuard, where received models are processed pairwise, we use the score as an acceptance criterion. ZenoPS. We adapt the score-based validation rule from ZenoPS [63]. Using the same candidate update 𝑔 and trusted reference update 𝑟 as used by FLTrust, the candidate is rejected if its alignment with the reference is too small, ⟨𝑔, 𝑟 ⟩ < 𝜁 ∥𝑟 ∥ 2 + 𝑒. Otherwise, the update ˜ 2 ≤ (1 + 𝛾)∥𝑟 ∥ 2 and applied: 𝜃𝑢 ← 𝜃𝑢 + 𝑔.˜ In line 𝑔 is clipped to ∥𝑔∥ with the original paper, we set 𝜁 = −0.02, 𝛾 = 0, and 𝑒 = 0.
5.3
Datasets and Click Models
We evaluate RankGuard and baselines using four datasets and three click models. Due to space constraints, some experiments report only a representative setting: WEB30K with the perfect click model, and an attack share 𝛽 = 0.2. Datasets. We adopt four standard datasets commonly used in LTR: WEB30K, MQ2007 [55], Yahoo [14], and Istella [17]. MQ2007 is the smallest with coarse three-level relevance labels, while WEB30K, Yahoo, and Istella offer larger query sets with five-level relevance.
Yahoo provides the highest-dimensional feature space (𝑑 = 700 features), and Istella the largest candidate sets per query. Click models. Click models provide a probabilistic account of how users interact with a ranked list, and are widely used to simulate user behavior in OLTR settings. Under the Simplified Dynamic Bayesian Network (SDBN) model [15], a user examines documents from top to bottom, clicking each with a probability conditioned on its relevance rel(𝑑), and stopping after a click with a relevancedependent stopping probability. In this work, we consider three instantiations of SDBN, describing different user behaviors: a perfect user who clicks all relevant documents and rarely stops, a navigational user seeking a single relevant result, and an informational user who clicks broadly and provides the noisiest signal. These click models are commonly considered in this domain [60].
5.4
Implementation and Evaluation Metrics
We simulate a network of 𝑛 = 100 nodes that follow the protocol described in Algorithm 1. The parameters of each node’s local model are initialized randomly. In each iteration, users take turns in round-robin order, each sampling a query from the dataset, ranking candidate documents using their model, and generating clicks via a click model. The configuration of the click models follows standard practice (for details, see Table 1 in [60]). Based on the clicks, the respective node updates their model and sends it to ⌈log2 𝑛⌉ = 7 randomly sampled nodes, using a RPS. For local model training, we employ a learning rate of 𝜂 = 0.1. By default, we assume a linear model 𝑓𝜃 (𝒅) = 𝜃 ⊤ 𝒅, where the model size equals the feature dimensionality of the respective dataset, so 𝜃 ∈ R𝑑 . To verify that RankGuard is not tied to this choice, we additionally benchmark a non-linear (neural) ranker: a feed-forward network with a single hidden layer of 64 units and sigmoid activation. The architecture and hyperparameter choices are adopted from prior work [50, 59]. Unless otherwise stated, we assume that 20 % of nodes in the network are malicious (i.e., 𝛽 = 0.2). Implementation and hardware. All baselines and experiments are implemented in Python. For PDGD, we adopt the NumPy-based implementation from Wang et al. [59]. Our experiments, including network simulations, are run on a single machine with an AMD EPYC 7282 processor; no GPU required. The efficiency benchmark (Section 6.2) uses a single CPU core, with a warm-up period that is excluded from the measurements. We always report mean and standard deviation across repetitions: across nodes for network simulations, and across random seeds (count stated per experiment) for standalone benchmarks. To support reproducibility, we make our code publicly available.1 Ranking accuracy. Ranking quality is typically evaluated offline using Normalized Discounted Cumulative Gain (nDCG) [42]. nDCG measures how well a ranked list matches an ideal ordering by grading each document by its relevance label and discounting conÍ rel(𝒅 𝒊 ) −1 tributions by position nDCG@𝑘 = 𝑍1 𝑘𝑖=1 2log (𝑖+1) , where rel(𝒅 𝒊 ) 𝑘 2 is the true relevance label of 𝒅 𝒊 and 𝑍𝑘 is a normalisation constant such that a perfect ranking scores 1. Following common practice, we truncate nDCG at 𝑘 = 10, as users rarely examine results beyond the first page. We evaluate each honest user’s model on a held-out 1 See https://anonymous.4open.science/r/rankguard-B569.
Preprint
Gregoriadis et al.
Table 1: Robustness after 30 000 sessions in a 100-node network with 20 malicious nodes. The best and second-best baselines in each setting are highlighted in bold and underlined, respectively. Defense
WEB30K perfect
informat.
perfect
navigat.
informat.
perfect
navigat.
informat.
0.415(2) 0.375(3) 0.335(1) 0.464(2) 0.452(1) 0.441(3) 0.326(16) 0.305(11) 0.278(38) 0.434(11) 0.405(23) 0.364(39)
perfect
navigat.
0.712(1) 0.678(8)
0.706(1) 0.699(1) 0.647(33) 0.615(49)
0.565(2) 0.500(14)
0.538(2) 0.420(50)
0.531(2) 0.287(129)
Flip
informat.
Istella
RankGuard 0.404(14) 0.380(24) 0.324(41) 0.472(7) 0.456(16) 0.399(43) 0.691(10) 0.676(17) 0.622(47) FLTrust 0.403(9) 0.300(26) 0.278(34) 0.460(3) 0.421(18) 0.378(39) 0.679(9) 0.648(36) 0.617(49) ZenoPS 0.374(24) 0.307(21) 0.264(49) 0.455(5) 0.422(17) 0.388(45) 0.678(9) 0.655(27) 0.610(48) GTS 0.347(3) 0.321(1) 0.317(1) 0.445(2) 0.442(3) 0.416(6) 0.703(2) 0.686(3) 0.674(5) CS 0.366(5) 0.319(2) 0.316(1) 0.446(2) 0.431(4) 0.426(5) 0.703(2) 0.672(6) 0.677(4) CWTM 0.333(2) 0.291(2) 0.311(2) 0.452(3) 0.418(6) 0.428(6) 0.701(2) 0.682(3) 0.664(5) None 0.089(3) 0.094(5) 0.107(10) 0.147(2) 0.146(3) 0.158(4) 0.443(2) 0.444(2) 0.448(7)
LIE
navigat.
Yahoo
RankGuard 0.391(17) 0.369(19) 0.308(45) 0.474(7) 0.454(15) 0.400(37) 0.689(9) 0.663(20) 0.613(50) 0.551(3) 0.489(16) 0.274(77) FLTrust 0.400(7) 0.307(15) 0.272(43) 0.453(4) 0.428(16) 0.374(36) 0.679(10) 0.658(23) 0.608(51) 0.546(12) 0.440(28) 0.289(130) ZenoPS 0.379(25) 0.305(14) 0.273(44) 0.460(7) 0.427(13) 0.376(39) 0.680(8) 0.656(24) 0.613(53) 0.554(4) 0.449(34) 0.291(134) GTS 0.128(0) 0.124(0) 0.123(0) 0.310(1) 0.225(2) 0.192(1) 0.485(0) 0.467(0) 0.465(0) 0.379(5) 0.022(0) 0.020(0) CS 0.247(3) 0.144(3) 0.124(0) 0.354(2) 0.272(3) 0.197(1) 0.595(3) 0.472(0) 0.467(0) 0.494(5) 0.336(6) 0.023(0) CWTM 0.192(5) 0.125(0) 0.124(0) 0.319(2) 0.253(2) 0.194(1) 0.534(2) 0.468(0) 0.467(0) 0.379(5) 0.024(0) 0.020(0) None 0.245(3) 0.177(4) 0.123(0) 0.374(1) 0.279(1) 0.193(0) 0.593(1) 0.474(0) 0.469(0) 0.483(2) 0.387(3) 0.022(0)
IPM
Oracle Local
MQ2007
RankGuard 0.392(26) 0.378(34) 0.333(42) 0.469(14) 0.442(19) 0.414(40) 0.689(14) 0.677(36) 0.641(39) 0.545(56) 0.511(57) 0.277(95) FLTrust 0.317(23) 0.293(49) 0.199(48) 0.418(32) 0.394(26) 0.345(49) 0.645(18) 0.623(35) 0.564(42) 0.497(5) 0.328(134) 0.220(123) ZenoPS 0.308(33) 0.284(37) 0.225(37) 0.429(14) 0.394(42) 0.325(55) 0.661(19) 0.603(38) 0.543(40) 0.470(53) 0.373(148) 0.224(111) GTS 0.240(52) 0.242(32) 0.221(58) 0.271(41) 0.312(60) 0.278(54) 0.555(50) 0.524(51) 0.555(41) 0.185(102) 0.148(99) 0.095(91) CS 0.240(47) 0.225(46) 0.208(57) 0.242(76) 0.281(60) 0.254(56) 0.522(48) 0.556(40) 0.573(56) 0.092(97) 0.155(81) 0.138(81) CWTM 0.228(38) 0.256(64) 0.203(32) 0.215(61) 0.292(66) 0.183(94) 0.547(19) 0.585(36) 0.539(16) 0.125(65) 0.096(68) 0.109(59) None 0.219(63) 0.213(54) 0.194(59) 0.259(0) 0.259(0) 0.259(0) 0.542(0) 0.542(0) 0.542(0) 0.157(94) 0.147(97) 0.102(75)
nDCG@10
Oracle
RankGuard
RankGuard-Rand
Local
(4) How does the session history size of nodes influence the effectiveness of RankGuard (Section 6.3)?
0.4 0.3
6.1
0.2 0
5000
10000
15000
20000
25000
30000
Sessions
Figure 3: Robustness under the Adapt attack.
test set after every full round (100 sessions), reporting the mean and standard deviation across nodes.
6
0.557(3) 0.509(15) 0.303(95) 0.550(9) 0.509(58) 0.278(137) 0.556(4) 0.451(31) 0.284(138) 0.521(4) 0.447(4) 0.419(5) 0.530(5) 0.433(5) 0.429(6) 0.523(4) 0.421(4) 0.418(6) 0.004(1) 0.004(1) 0.007(4)
Experimental Evaluation
Robustness of RankGuard
We assess the robustness of RankGuard from several angles. We first evaluate RankGuard against all defense baselines across datasets, click models, and three poisoning attacks (Flip, LIE, IPM). Next, we stress-test RankGuard against Adapt, a worst-case adaptive attack tailored specifically to defeat our defense, and introduce a lightweight modification that neutralizes it. We then show how the aggregation weight 𝛼 evolves as honest and malicious models are received. We then examine how robustness holds as the share of malicious nodes varies from 0 % to 50 %. Finally, we demonstrate that RankGuard’s robustness generalizes to a neural ranker.
We now present the experimental evaluation of RankGuard, which answers the following questions:
6.1.1 Main benchmark. We run RankGuard and baselines for 30 000 sessions, which is sufficient for all systems to converge. For this experiment, we consider three additional reference baselines:
(1) What is the robustness of RankGuard on preserving ranking accuracy compared to defense baselines, for different attacks, click models, and datasets (Sections 6.1 and 6.1.2)? (2) How do aggregation weights evolve in RankGuard over time and under different attacks (Section 6.1.3)? (3) How efficient, in terms of compute time, is RankGuard compared to baseline defenses (Section 6.2)?
(1) Oracle: Omniscient nodes always reject models received from malicious nodes and always average models received from honest nodes with weight 𝛼 = 0.5. (2) Local: Nodes never exchange models. This is equivalent to training on local click data only, without any collaboration. (3) None: Nodes always average received models (no defense).
Aggregation weight α
Preprint
Gregoriadis et al. F LIP
1.00
LIE
IPM
A DAPT
A DAPT (RG-Rand)
0.75 0.50 0.25 0.00
0
10000
20000
Sessions
30000
0
10000
20000
30000
0
10000
Sessions
20000
Sessions
30000
0
10000
20000
30000
Sessions
0
10000
20000
30000
Sessions
Figure 4: Evolution of 𝛼 for different attacks from honest vs. malicious users. Table 1 reports mean nDCG@10 for all honest nodes after 30 000 global sessions.2 To view each baseline’s convergence with respect to the number of global sessions, see Appendix C. After 30 000 sessions, each node has processed 300 queries locally. This can suffice to train an effective ranker with linear PDGD [50], which is why the Local baseline remains competitive. This advantage, however, hinges on the assumption that data remains stationary. Under concept drift, local interactions no longer suffice, motivating collaborative mechanisms [47]. As expected from prior work [2, 62], statistical defenses (CS, GTS, CWTM) collapse under LIE and IPM. RankGuard, FLTrust, and ZenoPS, which assess models based on performance rather than statistics, are less affected by these attacks. Across attacks, datasets, and click models, RankGuard ranks first in the majority of settings compared to all baselines, sometimes even above Oracle. Note that Oracle, while knowing whether a model is received from a malicious node, does not guarantee optimal performance. Where RankGuard does not come out first, the gap to the best-performing baseline is usually small. Curiously, one setting stands out as an exception: informational on Istella, across all attacks. In summary, our main benchmark results show that RankGuard consistently delivers robust ranking accuracy across all evaluated settings. 6.1.2 Adaptive attack. We designed Adapt as a worst-case attack tailored to defeat RankGuard (see Section 5.1). This attack assumes that the attacker knows the victim’s session history. Figure 3 shows ranking accuracy across sessions for RankGuard, the Local baseline and a modified version of RankGuard. Our evaluation confirms that Adapt is effective: the Local baseline outperforms RankGuard, showing that the attacker successfully degrades users’ rankers. To thwart even this powerful attack, we propose a simple modification to RankGuard in scenarios where the attacker may gain knowledge of users’ session history 𝑄. By evaluating each received model on a (freshly sampled) random subset 𝑄˜ ⊂ 𝑄 instead of all of 𝑄, RankGuard becomes non-deterministic: the attacker can no longer compute 𝛼 exactly, and thus cannot craft an update guaranteed to be accepted. We evaluate this method across subset sizes and find that RankGuard performs best when using only 20 % of the session history (shown as RankGuard-Rand. in Figure 3). At 17 500 sessions, RankGuard-Rand. exceeds nDCG@10 of 0.4 and nearly matches Oracle, compared to 0.28 for RankGuard without random subsampling. We conclude that this variant of RankGuard 2 To reduce evaluation noise, we report nDCG@10 averaged over the last 10 rounds (29 000–30 000 sessions) rather than the final round alone.
neutralizes the attack once users have accumulated a sufficiently large 𝑄. In this setting, this happens between 8000–15 000 sessions (i.e., local history lengths of 80–150). The evolution of 𝛼 over this window offers an interesting insight into what is happening. 6.1.3 Evolution of 𝛼 in RankGuard. Figure 4 shows the evolution of aggregation weight 𝛼 ∈ [0, 1] across sessions for honest and malicious models, across all four attacks. As the experiment progresses, local session histories grow and models converge, giving RankGuard increasing certainty in distinguishing honest from malicious updates. Under Flip and IPM, RankGuard achieves near-perfect separation, with malicious models almost always rejected outright (𝛼 ≈ 0). In contrast, under LIE it is more difficult to distinguish honest from malicious models under the LIE attack.Nevertheless, RankGuard consistently aggregates honest models with higher weights and shows robustness in most scenarios (see Table 1). In Section 6.1.2, we established that a non-deterministic variant of RankGuard is needed to neutralize the Adapt attack. In Figure 4, we compare Adapt on the original RankGuard with Adapt on RankGuard-Rand. Both show the attacker dominating early, when short session histories make RankGuard’s verdict unreliable; the poisoned model’s 𝛼 peaks before declining as histories grow. In the former, however, honest models are rejected throughout, so the node never benefits from collaboration even as the attack weakens. The latter breaks this: honest 𝛼 rises steadily and overtakes the poisoned 𝛼 after around 12 000 sessions, restoring useful aggregation. 6.1.4 Varying the attack fraction. We now study the robustness of RankGuard varying the attacker fraction 𝛽. Figure 5 show how ranking accuracy evolves for the Flip, LIE and IPM attack, for three click models and while varying 𝛽 between 0 % and 50 %. We do not evaluate 𝛽 > 0.5, as an adversarial majority breaks the assumptions of the peer sampler and lies beyond the reach of any aggregation defense. Note that lower attack shares not only indicate fewer malicious nodes but also more honest nodes, and thus more useful updates to learn from. These effects cannot be clearly isolated. Under the perfect click model, the curves for different attack shares are nearly indistinguishable, all converging to nDCG@10 ≈ 0.4. The navigational and informational click models introduce more variance and slower convergence, as expected from their noisier feedback. Particularly difficult to interpret are the results of LIE with the informational click model. Surprisingly, 𝛽 = 0.5 outperforms 𝛽 = 0. We attribute this to LIE’s degeneration under round-robin (see Section 5.1): with honest models tightly clustered, the attack
Preprint
Gregoriadis et al. 0%
10 %
perfect
20 %
30 %
40 %
50 %
navigational
informational 0.4
F LIP nDCG@10
0.2
0.2
0.2
LIE nDCG@10
0.4
0.4
0.4
0.2
0.2
0.2
IPM nDCG@10
0.4
0.4
0.4
0.4
0.2
0.2
0.3
0.3
0.3
0
5000 10000 15000 20000 25000 30000
0.4
0.2 0
5000 10000 15000 20000 25000 30000
Sessions
0
5000 10000 15000 20000 25000 30000
Sessions
Sessions
Figure 5: Robustness of RankGuard under varying shares of malicious nodes in the network.
6.1.5 Neural rankers. To assess whether RankGuard’s robustness holds in the non-convex regime, we repeat our evaluation using a neural ranker. Figure 6 shows that RankGuard continues to outperform the baselines across attacks, indicating that its defense does not depend on convexity and generalizes to non-linear rankers. It requires roughly 1.5–3× more sessions until convergence, depending on the attack. At the point of convergence, it reaches slightly higher accuracy than the linear ranker. This mirrors centralized experiments of PDGD [50]. In summary, RankGuard remains effective with neural rankers; the slower convergence and small accuracy gain are properties of the ranker, not of the defense.
6.2
Efficiency of RankGuard
We now quantify the computational efficiency of RankGuard and baselines. Table 2 reports the runtime of RankGuard and its two most competitive baselines, FLTrust and ZenoPS, across two ranking models and varying session history sizes, averaged over 100 trials. Specifically, this corresponds to Step 7 in Figure 2. Note that while population-based filters (CS, GTS, and CWTM) are computationally very cheap, they do not fit the pairwise aggregation setting motivated earlier and, as our results show, are not competitive in robustness. We therefore exclude them from this comparison. The computational overhead of RankGuard scales with the number of sessions (or precisely, the number of preference pairs in P (𝑞) ) used
Oracle GTS
RankGuard CS
FLTrust CWTM
10000
20000
ZenoPS Local
0.4 nDCG@10
reduces to small noise around the honest mean, which under the already-noisy informational signal acts as mild regularization rather than a genuine attack. Under IPM with the navigational click model, 𝛽 = 0.5 stalls convergence entirely, plateauing at nDCG@10 ≈ 0.25 while all lower attack shares reach ≈ 0.4, indicating the need for an honest majority. The same cannot be observed for informational. We suspect this stems from how IPM is implemented: the attacker sends −𝜖𝑔, where 𝑔 is the honest gradient after completing one search session. The navigational click model produces fewer but more relevance-concentrated clicks, yielding a cleaner, more directed gradient; reversing it thus yields a more directed harmful update.
0.3 0.2 0
5000
15000 Sessions
25000
30000
Figure 6: Robustness under IPM attack using a neural ranker.
to evaluate incoming models. The same is the case for FLTrust and ZenoPS. However, RankGuard is cheaper per session: it only requires forward passes through the model, whereas FLTrust and ZenoPS replay sessions, involving PDGD gradient estimation and update steps. For example, at 100 sessions RankGuard requires roughly 130 ms, versus the 670 ms needed by FLTrust and ZenoPS. This makes RankGuard with a linear ranking model up to 62× faster than its competitors. We additionally evaluate the computational efficiency of the defenses when PDGD uses a neural ranker. Even here, RankGuard remains the fastest defense, retaining an 11× speedup over FLTrust and ZenoPS. The reduced speedup with a neural ranker indicates that FLTrust and ZenoPS are dominated by model-agnostic replay overhead, particularly computing PDGD bias weights during session replay (Equation (2)). Hence, replacing the linear scorer with a neural scorer adds only a small relative cost to their already expensive pipeline. In contrast, RankGuard reuses cached bias weights and primarily performs model scoring for the defense; neural scoring therefore accounts for a larger share of its runtime, reducing the relative speedup while still leaving RankGuard substantially faster. This efficiency is critical, because it lets RankGuard draw on a larger 𝑄 for more
Gregoriadis et al.
Table 2: Runtime (seconds) for the evaluation of one received model in relation to the local session history length. Defense
Model
|𝑄 | = 102
|𝑄 | = 103
|𝑄 | = 104
RankGuard
Linear Neural Linear Neural Linear Neural
0.012 (0.001) 0.073 (0.013) 0.669 (0.127) 0.721 (0.138) 0.668 (0.127) 0.721 (0.139)
0.129 (0.003) 0.781 (0.145) 8.057 (0.487) 8.290 (0.821) 8.058 (0.487) 8.164 (0.537)
1.448 (0.032) 8.144 (1.151) 92.102 (1.286) 94.963 (1.708) 92.144 (1.277) 94.688 (1.393)
FLTrust ZenoPS
reliable judgments, as well as process a higher volume of models in the same time budget.
6.3
0.6
IPM with = 1 IPM with = 10
0.4 0.2 0.0 0
10 20 30 40 Local session history length |Q|
50
Figure 7: Attack detectability in relation to local session history length measures as the aggregation weight RankGuard assigns to a malicious model under IPM.
How Many Sessions Are Needed?
RankGuard’s defense ability hinges on the size of the user’s session history 𝑄. Larger |𝑄 | lets RankGuard estimate 𝑋𝑞 more reliably, tightening the 𝑡-statistic and producing more stable 𝛼 values. Conversely, smaller |𝑄 | lets sampling noise dominate RankGuard’s verdict on received models, and even honest peers may be rejected while malicious ones slip through. We now investigate how much history is needed before RankGuard’s decisions stabilize. To this end, we initialize a model 𝜃𝑢 randomly and train it on clicks generated from 1000 randomly sampled queries using the perfect click model. This gives a fairly converged ranker, as well as a local history with |𝑄𝑢 | = 1000. We then construct a poisoned model 𝜃 𝑣 by computing the gradient on a copy of 𝜃𝑢 for clicks generated from a freshly sampled query, and updating the copy in the opposite (i.e., harmful) direction. Note that this is equivalent to the IPM attack with 𝜖 = 1 (our previous experiments used 𝜖 = 10), and represents a minimal and harder to detect attack. By construction, 𝜃 𝑣 is strictly worse than 𝜃𝑢 at explaining the user’s clicks: it was derived from 𝜃𝑢 by stepping away from the honest gradient. The correct verdict from RankGuard is therefore full rejection, 𝛼 = 0. Any 𝛼 > 0 means the poisoned model has been (partially) incorporated; the gap from zero quantifies the defense’s failure on this minimal attack. We evaluate the aggregation weight 𝛼 that RankGuard assigns to the poisoned model as a function of the suffix length of 𝑄𝑢 used to compute it. We repeat this experiment over 1000 seeds and report the mean and standard deviation in Figure 7. Even in the scenario where the attacker deviates from the victim model minimally (𝜖 = 1), RankGuard rejects the poisoned model (𝛼 ≈ 0) after only around 40 sessions; for the stronger 𝜖 = 10 from our main experiment, the poisoned model is always rejected after only 10 local sessions. In short, RankGuard needs remarkably little history to defend effectively, allowing new participants to benefit from collaboration early. This responsiveness also matters under concept drift: as a user’s interests shift, RankGuard can assess incoming models against only the user’s most recent sessions.
7
Aggregation weight
Preprint
Related Work
In FL, robust aggregation techniques such as Krum [10], CWTM [67], and Geometric Median [53] assume an honest majority and treat malicious updates as statistical outliers. While effective against
naive attacks, they break under carefully crafted malicious updates that remain within the honest distribution [2, 28, 62]. Many works [27, 40, 66] adapt robust aggregation rules such as Krum and coordinate-wise trimming to DL but suffer from the same weaknesses. This motivated trusted-reference defenses that score updates by their effect on a benign validation dataset. In FL, FLTrust [12] compares each client update to a server-side reference update by cosine similarity, while Zeno and its variants [63, 64] rank updates by their estimated loss reduction on a small clean dataset held by the server. In DL, UBAR [39] removes the server dependence by having nodes screen model updates with their local data, and combines a distance-based pre-selection with a loss-based acceptance test that keeps only updates improving the node’s local objective. Sentinel [31] extends this idea with a hybrid of similarity filtering, local-loss validation, and norm normalization. Basil [23] also applies the local-validation principle but processes models over a logical ring with a memory of past updates. All these methods assume synchronous, fixed-topology communication and a pool of simultaneously available neighbor models. Yet, many of the assumptions break under the asynchronous, push-based, churn-prone dynamics of gossip learning we consider in this work. The principle of loss-based evaluation is closest to our approach in spirit, but they do not directly apply to OLTR: PDGD [50] ascends an unbiased gradient estimate rather than minimizing an explicit loss. Furthermore, nodes only have click data available, which is noisy and biased. The OLTR setting has remained understudied by the Byzantinerobust learning community. The only systematic study, by Wang and Zuccon [60], evaluates untargeted poisoning attacks with robust aggregators for federated OLTR, finding that these substantially harm ranking quality. We close this gap with RankGuard, a gossip-native defense for decentralized OLTR. A parallel line of work [35, 36] approaches decentralized LTR by gossiping raw click data rather than model weights. SwarmSearch [35] employs Data Shapley [33] to value peers’ data and discard poisoned contributions. This paradigm is ill-suited to our setting: Data Shapley requires expensive Monte Carlo retraining; sharing raw interaction data sacrifices privacy, and data gossip is communication-inefficient compared to model gossip. Moreover, being batch-based rather than online LTR, it does not learn from live, position-biased clicks.
Preprint
8
Conclusion
We presented RankGuard, a decentralized framework for OLTR in which users collaboratively train ranking models through peer-topeer gossip, without any central authority. Rather than filtering out statistical outliers among peer updates, each node judges incoming models by how well they explain its own position-bias-corrected click history, a design we support with the first formal convergence analysis of a decentralized OLTR algorithm. Our evaluation shows this approach is both effective and practical: RankGuard delivers the strongest ranking accuracy across most attacks, datasets, and click models, while running up to 62× faster than comparable defenses, making it efficient enough for Internet-scale deployment. It also defends reliably from very short click histories, so new participants benefit from collaboration almost immediately, with the defense adapting gracefully as user interests drift over time.
References [1] Alex Auvolat, Yérom-David Bromberg, Davide Frey, Djob Mvondo, and François Taïani. 2023. Basalt: A rock-solid byzantine-tolerant peer sampling for very large decentralized networks. In Proceedings of the 24th International Middleware Conference. 111–123. [2] Gilad Baruch, Moran Baruch, and Yoav Goldberg. 2019. A Little Is Enough: Circumventing Defenses For Distributed Learning. In Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada, Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d’Alché-Buc, Emily B. Fox, and Roman Garnett (Eds.). 8632–8642. https://proceedings.neurips.cc/paper/ 2019/hash/ec1c59141046cd1866bbbcdfb6ae31d4-Abstract.html [3] Yacine Belal, Mohamed Maouche, Sonia Ben Mokhtar, and Anthony SimonetBoulogne. 2025. Granite: a byzantine-resilient dynamic gossip learning framework. ArXiv preprint abs/2504.17471 (2025). https://arxiv.org/abs/2504.17471 [4] Yacine Belal, Mohamed Maouche, Sonia Ben Mokhtar, and Anthony SimonetBoulogne. 2025. Inferring communities of interest in collaborative learning-based recommender systems. In 2025 IEEE 45th International Conference on Distributed Computing Systems (ICDCS). IEEE, 505–515. [5] Enrique Tomás Martínez Beltrán, Mario Quiles Pérez, Pedro Miguel Sánchez Sánchez, Sergio López Bernal, Gérôme Bovet, Manuel Gil Pérez, Gregorio Martínez Pérez, and Alberto Huertas Celdrán. 2023. Decentralized federated learning: Fundamentals, state of the art, frameworks, trends, and challenges. IEEE Communications Surveys & Tutorials 25, 4 (2023), 2983–3013. [6] Juan Benet. 2014. Ipfs-content addressed, versioned, p2p file system. arXiv preprint arXiv:1407.3561 (2014). [7] Sayan Biswas, Antoine Boutet, Davide Frey, Romaric Gaudel, Rachid Guerraoui, Maxime Jacovella, Anne-Marie Kermarrec, Dimitri Lerévérend, François Taïani, and Martijn de Vos. 2026. Your Neighbors Know: Leveraging Local Neighborhoods for Backdoor Detection in Decentralized Learning. arXiv preprint arXiv:2605.19969 (2026). [8] Sayan Biswas, Mathieu Even, Anne-Marie Kermarrec, Laurent Massoulié, Rafael Pires, Rishi Sharma, and Martijn de Vos. 2025. Noiseless Privacy-Preserving Decentralized Learning. Proceedings on Privacy Enhancing Technologies (2025). [9] Sayan Biswas, Anne-Marie Kermarrec, Alexis Marouani, Rafael Pires, Rishi Sharma, and Martijn De Vos. 2025. Boosting asynchronous decentralized learning with model fragmentation. In Proceedings of the ACM on Web Conference 2025. 685–696. [10] Peva Blanchard, El Mahdi El Mhamdi, Rachid Guerraoui, and Julien Stainer. 2017. Machine Learning with Adversaries: Byzantine Tolerant Gradient Descent. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett (Eds.). 119–129. https://proceedings. neurips.cc/paper/2017/hash/f4b9ec30ad9f68f89b29639786cb62ef-Abstract.html [11] Christopher J. C. Burges, Tal Shaked, Erin Renshaw, Ari Lazier, Matt Deeds, Nicole Hamilton, and Gregory N. Hullender. 2005. Learning to rank using gradient descent. In Machine Learning, Proceedings of the Twenty-Second International Conference (ICML 2005), Bonn, Germany, August 7-11, 2005 (ACM International Conference Proceeding Series, Vol. 119), Luc De Raedt and Stefan Wrobel (Eds.). ACM, 89–96. doi:10.1145/1102351.1102363 [12] Xiaoyu Cao, Minghong Fang, Jia Liu, and Neil Zhenqiang Gong. 2021. FLTrust: Byzantine-robust Federated Learning via Trust Bootstrapping. In 28th Annual Network and Distributed System Security Symposium, NDSS 2021, virtually, February 21-25, 2021. The Internet Society. https://www.ndss-symposium.org/ndss-
Gregoriadis et al. paper/fltrust-byzantine-robust-federated-learning-via-trust-bootstrapping/ [13] Zhe Cao, Tao Qin, Tie-Yan Liu, Ming-Feng Tsai, and Hang Li. 2007. Learning to rank: from pairwise approach to listwise approach. In Machine Learning, Proceedings of the Twenty-Fourth International Conference (ICML 2007), Corvallis, Oregon, USA, June 20-24, 2007 (ACM International Conference Proceeding Series, Vol. 227), Zoubin Ghahramani (Ed.). ACM, 129–136. doi:10.1145/1273496.1273513 [14] Olivier Chapelle and Yi Chang. 2011. Yahoo! learning to rank challenge overview. In Proceedings of the learning to rank challenge. PMLR, 1–24. [15] Olivier Chapelle and Ya Zhang. 2009. A dynamic bayesian network click model for web search ranking. In Proceedings of the 18th International Conference on World Wide Web, WWW 2009, Madrid, Spain, April 20-24, 2009, Juan Quemada, Gonzalo León, Yoëlle S. Maarek, and Wolfgang Nejdl (Eds.). ACM, 1–10. doi:10. 1145/1526709.1526711 [16] European Commission. 2024. Commission opens formal proceedings against TikTok on election risks under the Digital Services Act. Press release IP/24/6487. Available from https://ec.europa.eu/commission/presscorner/detail/en/ip_24_ 6487. [17] Domenico Dato, Sean MacAvaney, Franco Maria Nardini, Raffaele Perego, and Nicola Tonellotto. 2022. The Istella22 Dataset: Bridging Traditional and Neural Learning to Rank Evaluation. In SIGIR ’22: The 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, Madrid, Spain, July 11 - 15, 2022, Enrique Amigó, Pablo Castells, Julio Gonzalo, Ben Carterette, J. Shane Culpepper, and Gabriella Kazai (Eds.). ACM, 3099–3107. doi:10.1145/ 3477495.3531740 [18] Martijn de Vos, Sadegh Farhadkhani, Rachid Guerraoui, Anne-Marie Kermarrec, Rafael Pires, and Rishi Sharma. 2023. Epidemic Learning: Boosting Decentralized Learning with Randomized Communication. In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023, Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine (Eds.). http://papers.nips.cc/paper_files/paper/2023/hash/ 7172e147d916eef4cb1eb30016ce725f-Abstract-Conference.html [19] Trinh Viet Doan, Tat Dat Pham, Markus Oberprieler, and Vaibhav Bajpai. 2020. Measuring decentralized video streaming: A case study of dtube. In 2020 IFIP Networking Conference (Networking). IEEE, 118–126. [20] John R Douceur. 2002. The sybil attack. In International workshop on peer-to-peer systems. Springer, 251–260. [21] Tim Draws, Nava Tintarev, Ujwal Gadiraju, Alessandro Bozzon, and Benjamin Timmermans. 2021. This Is Not What We Ordered: Exploring Why Biased Search Result Rankings Affect User Attitudes on Debated Topics. In SIGIR ’21: The 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, Virtual Event, Canada, July 11-15, 2021, Fernando Diaz, Chirag Shah, Torsten Suel, Pablo Castells, Rosie Jones, and Tetsuya Sakai (Eds.). ACM, 295–305. doi:10.1145/3404835.3462851 [22] El-Mahdi El-Mhamdi, Rachid Guerraoui, Arsany Guirguis, Lê Nguyên Hoang, and Sébastien Rouault. 2020. Genuinely distributed byzantine machine learning. In Proceedings of the 39th Symposium on Principles of Distributed Computing. 355–364. [23] Ahmed Roushdy Elkordy, Saurav Prakash, and Salman Avestimehr. 2022. Basil: A fast and Byzantine-resilient approach for decentralized training. IEEE Journal on Selected Areas in Communications 40, 9 (2022), 2694–2716. [24] Robert Epstein and Ronald E Robertson. 2015. The search engine manipulation effect (SEME) and its possible impact on the outcomes of elections. Proceedings of the national academy of sciences 112, 33 (2015), E4512–E4521. [25] Robert Epstein, Ronald E Robertson, David Lazer, and Christo Wilson. 2017. Suppressing the search engine manipulation effect (SEME). Proceedings of the ACM on Human-Computer Interaction 1, CSCW (2017), 1–22. [26] Mathieu Even, Anastasia Koloskova, and Laurent Massoulié. 2024. Asynchronous sgd on graphs: a unified framework for asynchronous decentralized and federated optimization. In International Conference on Artificial Intelligence and Statistics. PMLR, 64–72. [27] Cheng Fang, Zhixiong Yang, and Waheed U Bajwa. 2022. BRIDGE: Byzantineresilient decentralized gradient descent. IEEE Transactions on Signal and Information Processing over Networks 8 (2022), 610–626. [28] Minghong Fang, Xiaoyu Cao, Jinyuan Jia, and Neil Gong. 2020. Local model poisoning attacks to { Byzantine-Robust } federated learning. In 29th USENIX security symposium (USENIX Security 20). 1605–1622. [29] Minghong Fang, Zifan Zhang, Hairi, Prashant Khanduri, Jia Liu, Songtao Lu, Yuchen Liu, and Neil Gong. 2024. Byzantine-robust decentralized federated learning. In Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security. 2874–2888. [30] Chiara Farronato, Andrey Fradkin, and Alexander MacKay. 2023. Selfpreferencing at Amazon: evidence from search rankings. In AEA Papers and Proceedings, Vol. 113. American Economic Association 2014 Broadway, Suite 305, Nashville, TN 37203, 239–243. [31] Chao Feng, Alberto Huertas Celdrán, Janosch Baltensperger, Enrique Tomás Martínez Beltrán, Pedro Miguel Sánchez Sánchez, Gérôme Bovet, and Burkhard Stiller. 2024. Sentinel: An aggregation function to secure decentralized federated
Preprint learning. In ECAI 2024: 27th European Conference on Artificial Intelligence, 19– 24 October 2024, Santiago de Compostela, Spain–Including 13th Conference on Prestigious Applications of Intelligent Systems (PAIS 2024). SAGE Publications Pvt. Ltd 1 Oliver’s Yard, 55 City Road, London, EC1Y 1SP, 1760–1767. [32] Renaud Gaucher, Aymeric Dieuleveut, and Hadrien Hendrikx. 2024. Unified breakdown analysis for byzantine robust gossip. ArXiv preprint abs/2410.10418 (2024). https://arxiv.org/abs/2410.10418 [33] Amirata Ghorbani and James Zou. 2019. Data shapley: Equitable valuation of data for machine learning. In International conference on machine learning. PMLR, 2242–2251. [34] Anindya Ghose, Panagiotis G Ipeirotis, and Beibei Li. 2014. Examining the impact of ranking on consumer behavior and search engine revenue. Management Science 60, 7 (2014), 1632–1654. [35] Marcel Gregoriadis, Rowdy M Chotkan, Petru Neague, and Johan Pouwelse. 2025. SwarmSearch: Decentralized Search Engine with Self-Funding Economy. In 2025 IEEE 50th Conference on Local Computer Networks (LCN). IEEE, 1–10. [36] Marcel Gregoriadis, Quinten Stokkink, and Johan Pouwelse. 2025. Decentralized adaptive ranking using transformers. In Proceedings of the 5th Workshop on Machine Learning and Systems. 12–18. [37] Rachid Guerraoui, Anne-Marie Kermarrec, Anastasiia Kucherenko, Rafael Pinot, and Martijn de Vos. 2024. Peerswap: A peer-sampler with randomness guarantees. In 2024 43rd International Symposium on Reliable Distributed Systems (SRDS). IEEE, 271–281. [38] Barbara Guidi, Andrea Michienzi, and Laura Ricci. 2022. Evaluating the decentralisation of filecoin. In Proceedings of the 3rd international workshop on distributed infrastructure for the common good. 13–18. [39] Shangwei Guo, Tianwei Zhang, Han Yu, Xiaofei Xie, Lei Ma, Tao Xiang, and Yang Liu. 2021. Byzantine-resilient decentralized stochastic gradient descent. IEEE Transactions on Circuits and Systems for Video Technology 32, 6 (2021), 4096–4106. [40] Lie He, Sai Praneeth Karimireddy, and Martin Jaggi. 2022. Byzantine-robust decentralized learning via clippedgossip. ArXiv preprint abs/2202.01545 (2022). https://arxiv.org/abs/2202.01545 [41] Katja Hofmann, Shimon Whiteson, and Maarten de Rijke. 2013. Balancing exploration and exploitation in listwise and pairwise online learning to rank for information retrieval. Information Retrieval 16, 1 (2013), 63–90. [42] Kalervo Järvelin and Jaana Kekäläinen. 2002. Cumulated gain-based evaluation of IR techniques. ACM Transactions on Information Systems (TOIS) 20, 4 (2002), 422–446. [43] Thorsten Joachims, Laura Granka, Bing Pan, Helene Hembrooke, and Geri Gay. 2005. Accurately interpreting clickthrough data as implicit feedback. In Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (Salvador, Brazil) (SIGIR ’05). Association for Computing Machinery, New York, NY, USA, 154–161. doi:10.1145/1076034.1076063 [44] Thorsten Joachims, Adith Swaminathan, and Tobias Schnabel. 2017. Unbiased Learning-to-Rank with Biased Feedback. In Proceedings of the Tenth ACM International Conference on Web Search and Data Mining, WSDM 2017, Cambridge, United Kingdom, February 6-10, 2017, Maarten de Rijke, Milad Shokouhi, Andrew Tomkins, and Min Zhang (Eds.). ACM, 781–789. doi:10.1145/3018661.3018699 [45] Eugene Kharitonov. 2019. Federated Online Learning to Rank with Evolution Strategies. In Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining, WSDM 2019, Melbourne, VIC, Australia, February 11-15, 2019, J. Shane Culpepper, Alistair Moffat, Paul N. Bennett, and Kristina Lerman (Eds.). ACM, 249–257. doi:10.1145/3289600.3290968 [46] Anastasia Koloskova, Nicolas Loizou, Sadra Boreiri, Martin Jaggi, and Sebastian Stich. 2020. A unified theory of decentralized SGD with changing topology and local updates. In International conference on machine learning. PMLR, 5381–5393. [47] Chang Li and Maarten de Rijke. 2019. Cascading Non-Stationary Bandits: Online Learning to Rank in the Non-Stationary Cascade Model. In Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI 2019, Macao, China, August 10-16, 2019, Sarit Kraus (Ed.). ijcai.org, 2859–2865. doi:10. 24963/ijcai.2019/396 [48] Xiangru Lian, Ce Zhang, Huan Zhang, Cho-Jui Hsieh, Wei Zhang, and Ji Liu. 2017. Can Decentralized Algorithms Outperform Centralized Algorithms? A Case Study for Decentralized Parallel Stochastic Gradient Descent. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett (Eds.). 5330–5340. https://proceedings.neurips. cc/paper/2017/hash/f75526659f31040afeb61cb7133e4e6d-Abstract.html [49] Tie-Yan Liu. 2010. Learning to rank for information retrieval. In Proceeding of the 33rd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2010, Geneva, Switzerland, July 19-23, 2010, Fabio Crestani, Stéphane Marchand-Maillet, Hsin-Hsi Chen, Efthimis N. Efthimiadis, and Jacques Savoy (Eds.). ACM, 904. doi:10.1145/1835449.1835676 [50] Harrie Oosterhuis and Maarten de Rijke. 2018. Differentiable Unbiased Online Learning to Rank. In Proceedings of the 27th ACM International Conference on Information and Knowledge Management, CIKM 2018, Torino, Italy, October
Gregoriadis et al. 22-26, 2018, Alfredo Cuzzocrea, James Allan, Norman W. Paton, Divesh Srivastava, Rakesh Agrawal, Andrei Z. Broder, Mohammed J. Zaki, K. Selçuk Candan, Alexandros Labrinidis, Assaf Schuster, and Haixun Wang (Eds.). ACM, 1293–1302. doi:10.1145/3269206.3271686 [51] Róbert Ormándi, István Hegedűs, and Márk Jelasity. 2013. Gossip learning with linear models on fully distributed data. Concurrency and Computation: Practice and Experience 25, 4 (2013), 556–571. [52] Bing Pan, Helene Hembrooke, Thorsten Joachims, Lori Lorigo, Geri Gay, and Laura Granka. 2007. In Google we trust: Users’ decisions on rank, position, and relevance. Journal of computer-mediated communication 12, 3 (2007), 801–823. [53] Krishna Pillutla, Sham M Kakade, and Zaid Harchaoui. 2022. Robust aggregation for federated learning. IEEE Transactions on Signal Processing 70 (2022), 1142– 1154. [54] Frances A Pogacar, Amira Ghenai, Mark D Smucker, and Charles LA Clarke. 2017. The positive and negative influence of search results on people’s decisions about the efficacy of medical treatments. In Proceedings of the ACM SIGIR International Conference on Theory of Information Retrieval. 209–216. [55] Tao Qin and Tie-Yan Liu. 2013. Introducing LETOR 4.0 Datasets. CoRR abs/1306.2597 (2013). http://arxiv.org/abs/1306.2597 [56] Atul Singh, T-W Ngan, Peter Druschel, and Dan S Wallach. 2006. Eclipse attacks on overlay networks: Threats and defenses. In Proceedings IEEE INFOCOM 2006. 25th IEEE international conference on computer communications. [57] Ousmane Touat, Jezekael Brunon, Yacine Belal, Julien Nicolas, César Sabater, Mohamed Maouche, and Sonia Ben Mokhtar. 2025. Exposing the Vulnerability of Decentralized Learning to Membership Inference Attacks Through the Lens of Graph Mixing. In Proceedings of the 26th International Middleware Conference. 180–194. [58] Raluca M Ursu. 2018. The power of rankings: Quantifying the effect of rankings on online consumer search and purchase decisions. Marketing Science 37, 4 (2018), 530–552. [59] Shuyi Wang, Bing Liu, Shengyao Zhuang, and Guido Zuccon. 2021. Effective and privacy-preserving federated online learning to rank. In Proceedings of the 2021 ACM SIGIR international conference on theory of information retrieval. 3–12. [60] Shuyi Wang and Guido Zuccon. 2023. An analysis of untargeted poisoning attack and defense methods for federated online learning to rank systems. In Proceedings of the 2023 ACM SIGIR International Conference on Theory of Information Retrieval. 215–224. [61] Stefanie Warnat-Herresthal, Hartmut Schultze, Krishnaprasad Lingadahalli Shastry, Sathyanarayanan Manamohan, Saikat Mukherjee, Vishesh Garg, Ravi Sarveswara, Kristian Händler, Peter Pickkers, N Ahmad Aziz, et al. 2021. Swarm learning for decentralized and confidential clinical machine learning. Nature 594, 7862 (2021), 265–270. [62] Cong Xie, Oluwasanmi Koyejo, and Indranil Gupta. 2019. Fall of Empires: Breaking Byzantine-tolerant SGD by Inner Product Manipulation. In Proceedings of the Thirty-Fifth Conference on Uncertainty in Artificial Intelligence, UAI 2019, Tel Aviv, Israel, July 22-25, 2019 (Proceedings of Machine Learning Research, Vol. 115), Amir Globerson and Ricardo Silva (Eds.). AUAI Press, 261–270. http://proceedings.mlr.press/v115/xie20a.html [63] Cong Xie, Oluwasanmi Koyejo, and Indranil Gupta. 2022. Zenops: A distributed learning system integrating communication efficiency and security. Algorithms 15, 7 (2022), 233. [64] Cong Xie, Sanmi Koyejo, and Indranil Gupta. 2020. Zeno++: Robust Fully Asynchronous SGD. In Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event (Proceedings of Machine Learning Research, Vol. 119). PMLR, 10495–10503. http://proceedings.mlr.press/v119/ xie20c.html [65] Caiyi Yang and Javad Ghaderi. 2024. Byzantine-Robust Decentralized Learning via Remove-then-Clip Aggregation. In Thirty-Eighth AAAI Conference on Artificial Intelligence, AAAI 2024, Thirty-Sixth Conference on Innovative Applications of Artificial Intelligence, IAAI 2024, Fourteenth Symposium on Educational Advances in Artificial Intelligence, EAAI 2014, February 20-27, 2024, Vancouver, Canada, Michael J. Wooldridge, Jennifer G. Dy, and Sriraam Natarajan (Eds.). AAAI Press, 21735–21743. doi:10.1609/AAAI.V38I19.30173 [66] Zhixiong Yang and Waheed U Bajwa. 2019. ByRDiE: Byzantine-resilient distributed coordinate descent for decentralized learning. IEEE Transactions on Signal and Information Processing over Networks 5, 4 (2019), 611–627. [67] Dong Yin, Yudong Chen, Kannan Ramchandran, and Peter L. Bartlett. 2018. Byzantine-Robust Distributed Learning: Towards Optimal Statistical Rates. In Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsmässan, Stockholm, Sweden, July 10-15, 2018 (Proceedings of Machine Learning Research, Vol. 80), Jennifer G. Dy and Andreas Krause (Eds.). PMLR, 5636–5645. http://proceedings.mlr.press/v80/yin18a.html [68] Yisong Yue and Thorsten Joachims. 2009. Interactively optimizing information retrieval systems as a dueling bandits problem. In Proceedings of the 26th Annual International Conference on Machine Learning, ICML 2009, Montreal, Quebec, Canada, June 14-18, 2009 (ACM International Conference Proceeding Series, Vol. 382), Andrea Pohoreckyj Danyluk, Léon Bottou, and Michael L. Littman (Eds.). ACM, 1201–1208. doi:10.1145/1553374.1553527
Preprint
A
List of Notation
Symbol 𝑞 𝐷 (𝑞) 𝒅 𝑑 𝑓𝜃 (·) 𝜃 𝑅 𝑅∗ 𝒅 𝒊 ≻ 𝒅𝒋 𝒅 𝒊 >𝑐 𝒅𝒋 P (𝑞) (𝑞) 𝜌𝑖 𝑗 𝜂 𝑛 𝛽 𝑢, 𝑣 𝜃𝑢 𝑄𝑢 𝑆 (𝑞, 𝜃 ) Δ𝑆 𝑋𝑞 𝛼 𝑄˜ ⊂ 𝑄
B
Gregoriadis et al.
Description Session identifier Candidate document set for query in session 𝑞 Feature vector of a query-document pair Dimensionality of 𝒅 ∈ R𝑑 Scoring function with parameters 𝜃 Model weights Sampled ranking of length 𝑘 Ranking 𝑅 with one click pair 𝒅 𝒊 >𝑐 𝒅𝒋 swapped pairwise preference indicating that document 𝒅 𝒊 should be ranked above 𝒅𝒋 Click-inferred preference: 𝒅 𝒊 was clicked and 𝒅𝒋 was examined but not clicked Set of pairwise click preferences inferred for session 𝑞 Position-bias weight of pref. pair 𝒅 𝒊 >𝑐 𝒅𝒋 in session 𝑞 Learning rate Number of nodes in the network Fraction of malicious nodes in the network Node identifiers (in pairwise settings we denote 𝑢 as the local node and 𝑣 as a peer) Model weights of node 𝑢 Session history of node 𝑢 Bias-corrected session performance score Mean score improvement of received model against local model Per-query score difference 𝑆 (𝑞, 𝜃 𝑣 ) − 𝑆 (𝑞, 𝜃𝑢 ) Aggregation (interpolation) weight, 𝜎 (𝜅𝑡) Random subset of 𝑄 (anti-adaptive variant)
Proof Sketch of Theorem 4
Under the assumptions and formalism of RankGuard’s workflow presented in Section 4, we reduce our system down to a weighted aggregation variant of EL-Local [18] with an in-degree of 1 for each local round of every node in the network. Recall that (𝑖) 𝛼ˆ and 𝜎𝛼2ˆ , respectively, are the mean and variance of the weight any node assigns, in any of its local rounds, to the received model, and (𝑡𝑢 ) (𝑖𝑖) node 𝑢, in any local round 𝑡𝑢 , assigns a weight of 1 − 𝛼𝑢𝑣 to (𝑡𝑢 ) its own model and 𝛼𝑢𝑣 to the received model. Incorporating this weighted aggregation scheme, we determine the expected variance, 𝑉 , of the mixing matrix by 2 2 (𝑡𝑢 ) (𝑡𝑢 ) ˆ 2 + 𝛼ˆ 2 + 2𝜎𝛼2ˆ . 𝑉 = E 1 − 𝛼𝑢𝑣 + 𝛼𝑢𝑣 = (1 − 𝛼) On the other hand, from the convergence analysis of EL-Local, we 1 have 𝑉 = 𝑠+1 for any generic in-degree 𝑠. Combining these, we get the expected and effective in-degree 𝑠 ∗ for each node in each of its 2𝛼ˆ (1−𝛼ˆ ) −2𝜎 2
local rounds as 𝑠 ∗ = 𝑉1 − 1 = (1−𝛼ˆ ) 2 +𝛼ˆ 2 +2𝜎𝛼 2 . Plugging this expected 𝛼ˆ
and effective in-degree into the proof of Theorem 1.b) of EL-Local derives the convergence bound for RankGuard.
Preprint
C
Gregoriadis et al.
Robustness Plots RankGuard
FLTrust
ZenoPS
Yahoo nDCG@10
MQ2007 nDCG@10
WEB30K nDCG@10
perfect
GTS
navigational
0.4
0.4
0.3
0.3
0.4 0.3 0.7 0.6
CWTM
informational
0.4 0.3 0.2
0.2
0.2
CS
0.4
0.4
0.3
0.3
0.2 0.7
0.7
0.6
0.6
0.5
0.5
0.50
0.4
0.25
0.2
Istella nDCG@10
0.6 0.4 0.2 0
5000
10000 15000 20000 25000 30000
0.00
0
5000
Sessions
10000 15000 20000 25000 30000
0.0
0
5000
Sessions
10000 15000 20000 25000 30000
Sessions
Figure 8: Robustness under the Flip attack with attack share 𝛽 = 0.2.
RankGuard
FLTrust
ZenoPS
WEB30K nDCG@10
perfect
GTS
navigational
0.4
0.2
CS
0.4
0.4
0.2
0.2
0.4
0.4
0.2
0.2
CWTM
informational
MQ2007 nDCG@10
0.5
0.3 0.7
0.7
0.6
0.6
0.6
0.5
0.5
0.5
0.50
0.50
0.4
0.25
0.25
0.2
Yahoo nDCG@10 Istella nDCG@10
0.4
0.00
0
5000
10000 15000 20000 25000 30000
Sessions
0.00
0.0 0
5000
10000 15000 20000 25000 30000
0
5000
Sessions
Figure 9: Robustness under the LIE attack with attack share 𝛽 = 0.2.
10000 15000 20000 25000 30000
Sessions
Preprint
Gregoriadis et al. RankGuard
FLTrust
ZenoPS
perfect
navigational
CS
0.2
0.2
0.2
0.4
0.4
0.4
0.2
0.2
0.7
0.7
0.6
0.6
0.5
0.5
0.5
0.50
0.50
0.4
0.25
0.25
0.2
MQ2007 nDCG@10
WEB30K nDCG@10
0.4
0.00 5000
10000 15000 20000 25000 30000
Sessions
informational
0.2 0.7 0.6
0.0
0.00 0
CWTM
0.4
0.4
Yahoo nDCG@10 Istella nDCG@10
GTS
0
5000
10000 15000 20000 25000 30000
0
5000
Sessions
Figure 10: Robustness under the IPM attack with attack share 𝛽 = 0.2.
10000 15000 20000 25000 30000
Sessions