Rain: RDMA-assisted In-Network Scheduling for Microsecond-scale Workloads
arXiv:2606.03352v1 [cs.NI] 2 Jun 2026
ZHIHUANG MA, University of Science and Technology of China, China XINGMING CUI, University of Science and Technology of China, China XIAOLIANG CHEN, University of Science and Technology of China, China ZUQING ZHU, University of Science and Technology of China, China Modern data center applications increasingly require microsecond-scale service time with strict tail latency requirements, which can hardly be realized with existing in-network task schedulers due to their inherent limitations. Specifically, software-based schedulers struggle to balance throughput and latency, while switchbased designs either lack global coordination, rely on packet recirculation heavily, or only offer limited support for large tasks. In light of these restrictions of the state-of-the-arts (SOTAs), we, in this work, propose Rain, an RDMA-assisted in-network scheduler built atop programmable switches that maintains centralized queues while bounding worker-local queues. Rain introduces a bidirectional on-switch queuing mechanism to buffer and match tasks and worker-issued tokens directly in the switch, avoiding worker-side polling and approximating the optimal behavior of join-bounded-shortest-queue without global aggregation. A switchdriven RDMA engine pre-writes arbitrarily large tasks via one-sided WRITE multicasts, keeping only compact metadata on the switch. Slice-aware scheduling further localizes decisions to more homogeneous queues, reducing dispersion-induced head-of-line blocking. Moreover, our study reveals that real-world systems can diverge from theoretical predictions: shallower worker queues do not always improve tail latency. Leveraging this insight, Rain incorporates an adaptive scheduling strategy to optimize worker queue depths and workerto-slice mappings at runtime. Evaluations with the real-world application RocksDB show that Rain achieves 1.75× higher throughput than the best-performing SOTA while satisfying the same tail latency requirement. CCS Concepts: • Networks → Data center networks; Programmable networks; In-network processing. Additional Key Words and Phrases: Request load balancing, Request scheduling, Programmable switch ACM Reference Format: Zhihuang Ma, Xingming Cui, Xiaoliang Chen, and Zuqing Zhu. 2026. Rain: RDMA-assisted In-Network Scheduling for Microsecond-scale Workloads. Proc. ACM Netw. 4, CoNEXT2, Article 22 (June 2026), 21 pages. https://doi.org/10.1145/3808670
1
Introduction
Modern data center (DC) applications, such as latency-critical key-value stores [36, 41] and transactional databases [26, 53], increasingly operate at microsecond timescales and require strict tail-latency service-level objectives (SLOs) [1, 8, 17, 31, 35, 37]. However, meeting these requirements stresses existing system designs, because systems that work well for millisecond-scale tasks [44, 52, 59] often fail to provide the newly-required responsiveness once being pushed to the high-throughput regime. To cope with the demands of microsecond-scale workloads, system architects have explored both scale-up and scale-out approaches. Scale-up approaches originally Authors’ Contact Information: Zhihuang Ma, University of Science and Technology of China, Hefei, China, zhma@mail. ustc.edu.cn; Xingming Cui, University of Science and Technology of China, Hefei, China, [email protected]; Xiaoliang Chen, University of Science and Technology of China, Hefei, China, [email protected]; Zuqing Zhu, University of Science and Technology of China, Hefei, China, [email protected].
This work is licensed under a Creative Commons Attribution 4.0 International License. © 2026 Copyright held by the owner/author(s). ACM 2834-5509/2026/6-ART22 https://doi.org/10.1145/3808670 Proc. ACM Netw., Vol. 4, No. CoNEXT2, Article 22. Publication date: June 2026.
22:2
Zhihuang Ma, Xingming Cui, Xiaoliang Chen, and Zuqing Zhu
attempted to sustain performance when facing the slowing of Moore’s Law and Dennard scaling, by adding more cores, memory, and accelerators to a single machine [9, 10, 18, 19, 23, 25]. Nevertheless, this will increase hardware complexity, cost, and power density. Consequently, DCs now adopt scale-out architectures that distribute workloads across a cluster of machines [29, 33, 51, 61], or even across multiple clusters [58], to improve scalability, elasticity, and fault tolerance [11, 38, 56, 57]. Such a scale-out scenario places a premium on effective workload scheduling, since without careful coordination, systems are prone to head-of-line (HoL) blocking, resource contention, and elevated tail latency. Meanwhile, although scheduling strategies for scale-up systems have been extensively explored [9, 10, 18, 19, 23, 25], scale-out environments have much more stringent qualityof-service (QoS) requirements on latency and scalability and thus pose a few new challenges. Specifically, cluster-level throughput is normally much higher, maintaining global and timely visibility over a cluster or multiple clusters is more difficult, and communication latency becomes a non-negligible factor. Therefore, centralized software schedulers [6, 16, 59] had to struggle to achieve both high throughput and low tail latency [44, 51], while distributed software schedulers [4, 44, 46] often made suboptimal decisions due to delayed coordination and incomplete visibility on loads [29, 51]. Moreover, adding a software scheduler will typically make the forwarding paths of requests one hop longer, which can further exacerbate the aforementioned issues. To overcome these limitations, recent efforts have begun to leverage the line-rate packet processing capability of programmable switches [21] to realize high-performance in-network scheduling for scale-out environments [29, 33, 51, 58, 61]. Although promising, they still bear several unsolved restrictions. RackSched [61] and Pallas [33] adopted a push-based scheduling model that directly pushes tasks to workers without queuing any in the switch, but this scheme would be unfavorable to tail latency under high load dispersion [29]. Even though Pallas optimistically classified tasks to reduce dispersion, its classification would become ineffective when tasks’ service time could hardly be distinguished in advance [10]. R2P2 [29] introduced in-network task caching but relied heavily on packet recirculation, which reduces the switch’s available forwarding bandwidth; it is also unsuitable for the workloads whose tasks are stateful and thus require strict ordering. In our experiments, we further observed that relying on recirculation might make scheduling performance diverge noticeably from its theoretical expectations. Although Draconis [51] supports in-network caching, its design is primarily tuned for small-size tasks, while large tasks still require an additional round-trip time (RTT) to fetch data from clients or from an external in-memory store in the cluster. The restrictions of the state-of-the-arts (SOTAs) above motivated us to propose Rain, an innetwork scheduler built on programmable switches for microsecond-scale workloads, in this work. Rain is designed to maintain tail latency according to SLO while remaining as work-conserving as possible to maximize throughput, by leveraging two key features: 1) efficient in-network dispatching through on-switch scheduling, and 2) continuous system tuning through adaptive scheduling. On-switch scheduling (§3.2). Rain performs fine-grained scheduling directly in the data plane, by maintaining centralized queues there for both tasks and admission tokens. Specifically, clientsubmitted tasks and worker-issued tokens are temporarily buffered in a programmable switch on the data path, and a task is dispatched only when it is paired with a token. This design bounds each worker’s local queue to a configured depth. For the tasks that are too large to be buffered in the switch, Rain utilizes the remote direct memory access (RDMA) based multicast to pre-write their payloads to target servers while only storing fixed-size metadata in the switch. To mitigate workload dispersion, Rain partitions both in-network queues and workers into slices so that each scheduling queue handles tasks with similar characteristics [9, 10, 33]. In contrast to Pallas [33], which relies heavily on task grouping, Rain adopts slicing as a conservative auxiliary optimization while retaining centralized scheduling to enforce shallow per-worker queues.
Proc. ACM Netw., Vol. 4, No. CoNEXT2, Article 22. Publication date: June 2026.
Rain: RDMA-assisted In-Network Scheduling for Microsecond-scale Workloads
22:3
Adaptive scheduling (§3.3). In addition to static in-network scheduling, Rain incorporates a runtime scheduling agent that continuously adjusts the depths of workers’ local queues and workerto-slice mappings. This adaptive mechanism enables Rain to react timely to workload shifts and the worker performance changes due to co-located interference [14]. Our study further uncovers an often-overlooked yet important phenomenon: in real-world systems, tail-latency behavior might deviate from theoretical predictions due to practical issues, and thus simply maintaining shallower worker queues does not necessarily lead to better tail performance. We implement Rain on a Tofino switch [21] and evaluate it using both synthetic workloads and real-world applications. The results show that Rain consistently achieves the lowest 99% tail latency while fully utilizing the system’s processing capacity. When being evaluated with real-world application RocksDB, Rain delivers 1.75× higher throughput than a SOTA in-network scheduler [33] under the same SLO. Moreover, our adaptive scheduling enables Rain to maintain low tail latency even under dynamic workloads. In all, our major contributions are as follows: • We design a bidirectional queuing mechanism to buffer and match tasks and admission tokens in a programmable switch, removing the need for worker-side task polling as well as enabling centralized control with bounded worker queues at low overhead. • We develop an RDMA-assisted technique that allows programmable switches to maintain centralized queues for tasks of arbitrary size by buffering only compact metadata. • We identify and characterize a phenomenon, namely, tail latency collapse, where reducing worker-local queue depth—a practice that is traditionally believed to be beneficial for tail latency—can instead increase tail latency under nearly-overloaded conditions. • We implement a full prototype of Rain and perform experimental comparisons with the SOTAs, demonstrating the advantages and effectiveness of Rain. This work does not raise any ethical issues. 2
Background and Motivation
2.1
In-Network Scheduling
Existing in-network scheduling approaches can be broadly classified into two categories based on their granularities: flow scheduling and task scheduling. Flow-level techniques, such as ECMP and other layer-4 load-balancer [60], operate on flows and are effective at balancing the volumes of aggregated traffic under high flow-entropy conditions [15]. However, their coarse granularity makes it infeasible to distinguish individual tasks within a flow, leading to HoL blocking and persistent imbalance when service demands vary across flows. Task scheduling encompasses both millisecond-scale and microsecond-scale regimes. Systems such as Google Borg [52], Apache Spark [59], and Sparrow [44] target at the millisecond-scale tasks whose execution time is several orders of magnitude longer than their scheduling delays, enabling sophisticated software schedulers with global coordination. At the microsecond timescale, however, the service time of tasks becomes comparable to or even shorter than a network RTT, making scheduling delay a non-negligible portion in the overall latency. For a task with 10 µs service time, a centralized software scheduler deployed on its forwarding path adds an extra intra-rack round trip plus queuing that can easily exceed one-third of the total response time [1, 49], pushing tail latency beyond SLO thresholds [8]. Distributed software schedulers avoid the extra hop but sacrifice global visibility, leading to suboptimal placement under high loads; neither approach delivers near-optimal scheduling at this timescale without prohibitive latency or host CPU overhead. Proc. ACM Netw., Vol. 4, No. CoNEXT2, Article 22. Publication date: June 2026.
22:4
2.2
Zhihuang Ma, Xingming Cui, Xiaoliang Chen, and Zuqing Zhu
Theories behind In-Network Scheduling
Scheduling behavior is largely determined by how tasks are queued and dispatched across workers. The underlying queuing model governs the trade-off between tail latency, work-conserving efficiency, and implementation complexity. Join-bounded-shortest-queue (JBSQ) [29] formalized this trade-off by combining a centralized scheduler queue with local queues on workers, and limited each worker’s queue depth by a bound parameter 𝑛. When 𝑛 = 1, the scheduling becomes fully centralized and pull-based, i.e., each worker only fetches tasks when it is idle, avoiding HoL blocking but sacrificing work-conserving efficiency. At the opposite extreme, 𝑛 = ∞ yields a fully distributed, push-based scheme that maximizes work conservation but can incur severe HoL blocking as workers’ local queues can grow unbounded. The power-of-𝑘 policy [42] provided a non-global alternative by sampling 𝑘 workers per task and selecting the one with the shortest queue occupancy. This reduces coordination overhead due to state collection, though at the cost of less precise load balancing. The simulations in [7, 29] suggested that centralized queuing combined with shallow worker queues provides the best tail latency, particularly under high service time dispersion. More broadly, schemes exploiting global queue information consistently outperformed those that rely on partial states. 2.3
In-Network Scheduling in Practice
2.3.1 On-Switch Scheduling. Programmable switches such as Intel Tofino [21] provide high throughput and predictable, low per-packet latency, making them attractive substrates for in-network scheduling. However, translating the ideal principles derived from theory—centralized queuing and globally-coordinated scheduling—into the constraints of real hardware remains challenging. Building a centralized queue requires buffering tasks of variable sizes. To access task metadata carried in packets, the switch has to parse packet fields, but packet parsers usually only support limited look-up lengths and thus cannot extract arbitrary task payload. Moreover, the capability of maintaining a queue in on-chip memory is restricted by the narrow width and fixed structure of register arrays. Global coordination is also subject to hardware constraints. Specifically, in Tofino’s feed-forward pipeline, each register array resides in a single pipeline stage and thus can only be accessed at most once per packet during a pipeline pass. This constraint precludes any iterative aggregation or comparison within the array. Consequently, when the number of queues exceeds the number of pipeline stages, scanning all the queues to find the globally shortest one cannot be done in a single pipeline pass, i.e., the queue registers are distributed across stages. Due to these limitations, existing in-network schedulers adopt various compromises. R2P2 [29] approximated centralized queuing through packet recirculation, keeping tasks in the pipeline instead of explicitly buffering them. Although this avoids generating large queues, the scheme actually consumes more switch bandwidth, increases latency, and risks packet reordering. In practice, we also observe that such recirculation causes the implemented behavior to deviate noticeably from the theoretical performance of JBSQ. Draconis [51] is optimized for small, fixed-size tasks that can be buffered directly on the switch. For larger tasks, workers still need extra time to fetch the associated data after dispatch, which adds to their end-to-end latency. RackSched [61] adopted a two-level scheduling design, in which the in-network component does not utilize centralized queuing and applies a sub-optimal power-of-𝑘 policy. 2.3.2 Tail Latency Collapse. A major discrepancy between theoretical queuing models and realworld systems lies in the cost of task retrieval. In theory, workers should be able to pull tasks from a centralized queue instantaneously, incurring no network delay or control overhead. However, in practice, each pull request has to traverse the network and compete for the processing resources on a switch or network interface card (NIC), and thus introduces non-negligible latency. Therefore, Proc. ACM Netw., Vol. 4, No. CoNEXT2, Article 22. Publication date: June 2026.
Rain: RDMA-assisted In-Network Scheduling for Microsecond-scale Workloads
99% Tail Latency (μs)
Worker queue depth = 1
Worker queue depth = 2
150
300
100
200
50
100
0
2200
2400
2600
2800
22:5
0
Load (kRPS) (a) Exponential (mean = 10 μs)
200
400
Load (kRPS) (b) Bimodal (50% 10 μs, 50% 100 μs)
Fig. 1. Tail latency versus system load under two worker queue depths.
workers might stall while waiting for new tasks, making the system non-work-conserving [9]. This inefficiency has a significant impact on stability. Specifically, as the load grows, accumulated pull delays reduce the system’s effective service capacity, causing an earlier onset of tail-latency collapse, i.e., a sharp increase in response time near saturation. As a consequence, shallow worker queues do not always improve tail latency: once the system enters the collapse region, tail latency escalates rapidly and soon outweighs the benefits of limiting queue depth. Figures 1(a) and 1(b) show the results when the service time of workloads follows exponential and bimodal distributions, respectively, where the queue depth of each worker is limited as 1 or 2 and the load is in kilo requests per second (kRPS). When the service time follows the exponential distribution, shallower queues collapse earlier than deeper ones, reflecting reduced work conservation and lower sustainable throughput. Under the bimodal distribution, this effect is less pronounced as the service time is more variable. In all, the results highlight that overly-short queues can magnify non-work-conserving behavior, making tail latency rise sharply as system approaching collapse. 2.4
Classification Alleviates but Does Not Eliminate Dispersion
Keys per SCAN
3000
600
2500 2000
400
1500 1000
200
500
Average service time (μs)
A natural approach to mitigating service time dispersion is to classify tasks and route each type of tasks to a dedicated queue or worker pool [9, 10, 33]. This stratification narrows the service time distribution within each type and enables more informed scheduling decisions, reducing the chance that an exceptionally long task blocks many short ones. However, in real-world systems, execution time depends on numerous dynamic factors that have joint effects and thus can hardly be coarsely distinguished with type labels, including input data characteristics, cache locality, interference from co-located workloads, and variability in downstream storage or external services.
32 64 128 256 512 1024
Item size (bytes)
Fig. 2. Average service time of SCAN in RocksDB as a function of item size and number of keys. Proc. ACM Netw., Vol. 4, No. CoNEXT2, Article 22. Publication date: June 2026.
22:6
Zhihuang Ma, Xingming Cui, Xiaoliang Chen, and Zuqing Zhu
For example, in RocksDB [41], the service time of operations like GET and SCAN increases with the number of keys accessed, and it also correlates with the item size [10]. Our measurements in Figure 2 further confirm that these two factors jointly affect the average service time, even when we only consider a single operation type (SCAN). Hence, considerable residual dispersion can remain within each task type, amplifying the negative impact of HoL blocking. To this end, although classification is a useful primitive, it cannot fully eliminate workload dispersion, and thus practical schedulers have to address the residual variability among homogeneous tasks. 3 3.1
Design of Rain System Overview
The system of Rain consists of two core components: a programmable switch and lightweight scheduler agents co-located with workers, as illustrated in Figure 3. Note that, we make Rain focus on in-network scheduling and thus it is designed to be agnostic to worker roles, regardless of whether a worker serves as the final task handler or as a second-level scheduler [61]. Servers Shared RDMA buffer Shared RDMA buffer Shared RDMA buffer
Programmable switch Tasks
Bidirectional queue
RDMA engine
Task dispatching
Tasks
Tokens
Workers Workers Workers Scheduler agent Scheduler agent Scheduler agent
Fig. 3. System architecture of Rain.
The programmable switch serves as the ingress for worker traffic and performs per-task scheduling directly in the data plane. We design it to have the following modules: 1) a set of centralized bidirectional queues that are maintained for both tasks and tokens, enabling precise task dispatching (§3.2.1), and 2) an RDMA engine that offloads large task payloads to workers’ memory while retaining only compact metadata in the switch (§3.2.2). The in-network queues and their corresponding workers are distributed to slices, such that tasks can be mapped to the slices according to their types, thereby reducing workload dispersion within each slice (§3.2.3). Each scheduler agent runs as a lightweight runtime module on a server, managing one or more workers. It monitors the runtime status of its workers, and adjusts their participation in scheduling accordingly, by tuning their token quotas and updating worker-to-slice mappings (§3.3). In addition, we also include a controller for Rain, which is not shown in Figure 3, to coordinate worker registration and program the switch with necessary metadata, such as the routing rules for task dispatching and RDMA configuration parameters that enable valid one-sided RDMA WRITE. 3.2
On-Switch Scheduling
3.2.1 Bidirectional Queue. Rain maintains in-switch queues for both client-submitted tasks and worker-issued tokens. When a task and a token cannot be immediately matched, the unmatched entry is buffered in the switch. At any time, only one side of the bidirectional queue, i.e., either the task queue or the token queue, can be non-empty. Figure 4 illustrates the overall workflow. Approximating JBSQ. As discussed in §2.3.1, the feed-forward architecture of programmable switches makes it difficult to efficiently enforce bounded local queues and to compute the global minimum queue length required for exact JBSQ. Rain instead approximates JBSQ through a tokendriven dispatching mechanism, which enforces bounded local queues as: each worker can hold at Proc. ACM Netw., Vol. 4, No. CoNEXT2, Article 22. Publication date: June 2026.
Rain: RDMA-assisted In-Network Scheduling for Microsecond-scale Workloads Task from clients
Token from worker enqueue dequeue
yes
decrease
Switch ingress
22:7
Token balance
Token queue
dequeued token
Task Dispatching
balance >= 0
increase
dequeue enqueue
no
Task queue
Switch egress
dequeued task
Fig. 4. In-network scheduling workflow centered on the bidirectional queue.
most 𝑛 outstanding tokens, and more lightly-loaded workers tend to retain more tokens and thus are more likely to receive new tasks. This mechanism coincides with JBSQ in two cases: 1) if task service time is constant, it performs round-robin dispatching among the least-loaded workers, and 2) when each worker’s local queue depth is limited to one, it becomes identical to JBSQ. In practice, Rain further approaches JBSQ behavior by partitioning resources into slices to reduce workload dispersion (§3.2.3), and when dispersion is unavoidable, constraining the local queue depth to one again recovers the equivalence to JBSQ (§3.3). Efficient queue operations. To compactly encode the queue state, Rain maintains a tokenbalance counter, where a positive value denotes the number of outstanding tokens, and a negative value (in absolute terms) is for the buffered tasks. This representation fits well with a feed-forward switch pipeline, as enqueue/dequeue operations are reduced to simple updates on the bidirectional queue. Hence, Rain avoids costly check-and-rollback cycles with packet recirculation [51]. Elimination of worker polling. By queuing tokens in the switch, Rain removes the need of workers actively polling for tasks. Each worker issues a token to signal its readiness for a new task. The token is either immediately consumed by an outstanding task, or cached in the switch so that the next arriving task can be dispatched to the worker instantly. This push-based design maximizes responsiveness without incurring any additional bandwidth cost due to workers’ polling. Task#n
Server
Metadata for task#n
n
n
Arbitrary length
Fixed length
Worker thread 服务端（CPU） 服务端（CPU）
...
3
2
RDMA buffer
too large 3
2
1
Slice multicast table
ITE WR
321 Task queue RDMA engine
WRITE
...
Programmable switch Server
Worker thread 服务端（CPU） 服务端（CPU）
...
3
2
RDMA buffer
Fig. 5. RDMA engine that pre-writes large tasks to worker memory via one-sided RDMA WRITE.
3.2.2 RDMA Engine. Each task-queue entry in Rain has a fixed length (e.g., 8 bytes in our implementation) to ensure feasible implementation on programmable switches. However, this makes it difficult to accommodate the tasks whose sizes are larger than the length. Meanwhile, Rain needs to make sure that all tasks participate in the same scheduling process without compromise. Therefore, we design a RDMA engine in Rain to indirectly cache oversized tasks while preserving the abstraction of a centralized queue. As exemplified in Figure 5, the tasks that cannot be directly buffered in the switch will be pre-written to the RDMA buffers in workers’ memory via one-sided RDMA WRITE operations. Then, for each of these oversized tasks, the switch only needs to store in its task queue a fixed-size metadata descriptor, which records the remote memory location, payload Proc. ACM Netw., Vol. 4, No. CoNEXT2, Article 22. Publication date: June 2026.
22:8
Zhihuang Ma, Xingming Cui, Xiaoliang Chen, and Zuqing Zhu
size, and other per-task metadata. When the task is dispatched, the designated worker retrieves the full task content from its local RDMA buffer based on this metadata reference. WRITE multicasting. Supporting large tasks within the same scheduling pipeline means that the handling worker of a task is not known at the time of pre-writing. To address this, Rain identifies all potential workers within the same scheduling slice (§3.2.3) and pre-writes payloads of oversized tasks to the RDMA buffer of each candidate worker via one-sided RDMA WRITE as follows. During initialization, each scheduler agent exposes a shared RDMA buffer and queue pair (QP) to its co-located workers. The RDMA buffer is organized as a ring, with the write offset maintained by the switch to ensure consistent wrap-around placement. Note that, this introduces no data races or synchronization overhead, as each worker accesses its buffer only after a task being dispatched to it. The switch uses its built-in multicast capability to perform RDMA WRITE multicasting. Constructing valid RDMA WRITE packets1 . To construct valid RDMA WRITE packets from scratch, Rain has to ensure the correctness of all the routing and RDMA state fields, including the destination MAC address, source and destination IP addresses, queue pair numbers (QPNs), and remote memory keys (rkeys). The packet sequence number (PSN) is also maintained continuously to preserve reliable connection semantics. Tofino’s packet replication engine creates one copy per multicast group member between the ingress and egress pipelines, and the per-destination RDMA fields are then rewritten by match-action tables keyed on the egress port, so each replica carries the correct per-worker RDMA state. The static connection state (e.g., routing information, QPNs, and rkeys) is stored in match-action tables installed by the controller during worker registration, while PSNs are dynamically tracked in a register array. We disable integrity checks (iCRC), as recomputing them in the switch pipeline is infeasible [39]. 3.2.3 Scheduling in Slices. To mitigate workload dispersion, Rain partitions both tasks and workers into slices. Both the bidirectional queue and RDMA WRITE multicasting operate at the granularity of slices. A data-plane table maps tasks to slice identifiers according to the tasks’ types, and it is maintained and updated in runtime by the Rain controller. During enqueue/dequeue operations, tasks and tokens consult this table to determine their slice. This design is based on the observation that tasks of the same type in an application tend to exhibit similar service time [33]. For example, in RocksDB, GET operations typically access only a small amount of data (e.g., fetching 10 keys costs about 12 µs), whereas SCAN requests traverse much larger ranges (e.g., scanning 5, 000 keys costs about 650 µs) [33]. Hence, slicing groups similar tasks together such that scheduling decisions can be made among workers handling relative homogeneous workloads, thereby reducing the probability of HoL blocking. Note that, slicing is intended to alleviate (rather than eliminate) workload dispersion, as in certain cases, dispersion might still remain even after slicing [10]. Note that the idea of type-based task separation has been explored before [9, 10, 33]; Rain adopts it as an auxiliary optimization to reduce workload dispersion, while the key difference is that Rain retains centralized bounded queuing within each slice, which bounds tail latency effectively. 3.3 Adaptive Scheduling Rain introduces an adaptive scheduling strategy to tune worker behaviors dynamically in runtime, i.e., continuously adjusting each worker’s token quota and worker-to-slice mapping in response to changes in workload characteristics and performance variations caused by interference [14]. Specifically, to avoid tail latency collapse (§2.3.2), we make sure that the adaptive scheduling can distinguish the rise in tail latency stemming from overload-induced collapse from that due to HoL blocking. Rain infers the root-cause by examining the ratio of worker-side queuing latency to 1 Rain targets RDMA over Converged Ethernet version 2 (RoCEv2) [55].
Proc. ACM Netw., Vol. 4, No. CoNEXT2, Article 22. Publication date: June 2026.
Rain: RDMA-assisted In-Network Scheduling for Microsecond-scale Workloads
22:9
Algorithm 1 Adaptive Scheduling 1: for each sampling interval Δ𝑡 do 2: for each slice 𝑠 do 3: for each worker 𝑤 ∈ W𝑠 do 4: sample 𝐿𝑤 , 𝐿𝑠𝑤 , 𝑇𝑤 , and 𝑄𝑠𝑤,𝑤 for worker 𝑤; 5:
+𝑇𝑤 𝑤 , 𝐿𝑤𝐿+𝐿 , and 𝑄𝑠𝑤,𝑤 ; update S𝑤 , R 𝑤 , Q𝑤 with 𝐿𝑤 +𝐿𝑇𝑠𝑤 𝑤 𝑠𝑤
6: for each control interval of 𝐾 sampling intervals do 7: for each slice 𝑠 do 8: label(𝑠) = ∅, 𝑂𝑠 = 0, 𝐼𝑠 = 0; 9: for each worker 𝑤 ∈ W𝑠 do 10: 11: 12: 13: 14: 15: 16: 17: 18: 19: 20: 21: 22: 23: 24: 25:
𝑝99
compute 99% tail slowdown 𝑆 𝑤 from samples in S𝑤 ; compute average ratio 𝑅¯𝑞 from samples in R 𝑤 ; 𝑝99 compute 99% tail switch-side queue length 𝑄𝑠𝑤,𝑤 from samples in Q𝑤 ; 𝑝99 if 𝑆 𝑤 > 𝑆 th then 𝑝99 if (𝑛 𝑤 > 1) and (𝑅¯𝑞 > 𝑅th ) and (𝑄𝑠𝑤,𝑤 = 0) then 𝑛 𝑤 = 𝑛 𝑤 − 1; else if (𝑛 𝑤 < 𝑛 max ) and (𝑅¯𝑞 < 𝑅th ) then 𝑛 𝑤 = 𝑛 𝑤 + 1; else 𝑂𝑠 = 𝑂𝑠 + 1; else measure idleness ratio of worker 𝑤 over the control interval as 𝐼 𝑤 ; 𝐼𝑠 = 𝐼𝑠 + 𝐼 𝑤 ; reset sample buffers S𝑤 , R 𝑤 , Q𝑤 ; if (𝑂𝑠 > 0) and (𝐼𝑠 > 1) then Í 𝑁𝑠 𝑁𝑠 = ; 𝑛 𝑤 , 𝑛¯ = | W 𝑠| 𝑤 ∈ W𝑠
26: 27: 28: 29: 30: 31: 32: 33: 34:
for each worker 𝑤 ∈ W𝑠 do ¯ 𝑛 𝑤 = 𝑛; else if 𝑂𝑠 > 0 then label(𝑠) = Overloaded; else if 𝐼𝑠 > 1 then label(𝑠) = Underloaded; S𝑜 = {𝑠 | label(𝑠) = Overloaded}, S𝑢 = {𝑠 | label(𝑠) = Underloaded}; while (S𝑜 ≠ ∅) and (S𝑢 ≠ ∅) do 𝑠𝑜 = arg max (𝑂𝑠 ), 𝑠𝑢 = arg max (𝐼𝑠 ); 𝑠 ∈ S𝑜
𝑠 ∈ S𝑢
35:
𝑤 ★ = arg min (𝑛 𝑤 );
36: 37:
reassign 𝑤 ★ to 𝑠𝑜 , 𝑠 (𝑤 ★) = 𝑠𝑜 , 𝑛 𝑤★ = 1; S𝑜 = S𝑜 \ {𝑠𝑜 }, S𝑢 = S𝑢 \ {𝑠𝑢 };
𝑤 ∈ W𝑠𝑢
total queuing latency, i.e., HoL blocking driven by deep worker queues can usually be identified by seeing a disproportionate increase in worker-side queuing delay. To implement this idea, we make each scheduler agent continuously collect switch-side and worker-side queuing latencies (𝐿𝑠𝑤 and 𝐿𝑤 , respectively). To avoid requiring clock synchronization Proc. ACM Netw., Vol. 4, No. CoNEXT2, Article 22. Publication date: June 2026.
22:10
Zhihuang Ma, Xingming Cui, Xiaoliang Chen, and Zuqing Zhu
between the switch and NICs on servers, the latencies are measured independently in their respective clock domains and then aggregated as durations to obtain the total queuing delay. In addition, it records processing times (𝑇𝑤 ) and switch-side queue occupancy (𝑄𝑠𝑤,𝑤 ) observed by the worker. All switch-side metrics are embedded into packets via in-band network telemetry (INT) [30]. To support adaptive worker-to-slice allocation, each worker also reports its idleness. For threads using a poll-mode driver (PMD) [13], the ratio of polling iterations that return zero packet to total iterations provides an effective measure of idleness [20]. Algorithm 1 describes the procedure of the adaptive scheduling, which is implemented by scheduler agents running alongside workers and invoked at a high frequency. The agent continuously runs a fine-grained sampling loop (Lines 1–5) that, every small sampling interval Δ𝑡 (e.g., 10 𝜇s), collects measurements {𝐿𝑤 , 𝐿𝑠𝑤 ,𝑇𝑤 , 𝑄𝑠𝑤,𝑤 } from all slices and workers and appends the derived statistics to per-worker sample buffers. Every control interval consisting of 𝐾 such sampling steps (Line 6). At the per-worker level (Lines 9–22), the agent computes from the accumulated samples 𝑝99 the 99% tail slowdown 𝑆 𝑤 (Line 10), the average ratio 𝑅¯𝑞 of worker-side latency to total queuing 𝑝99 latency (Line 11), and the 99% tail switch-side queue length 𝑄𝑠𝑤,𝑤 (Line 12). 𝑝99 When the slowdown 𝑆 𝑤 exceeds the preset threshold 𝑆 th (Line 13), the agent first determines where the queuing is dominated. If it is dominated by the worker queue (𝑅¯𝑞 > 𝑅th ) while the switch 𝑝99 queue remains empty at the 99% tail level (𝑄𝑠𝑤,𝑤 = 0) and the worker still has more than one outstanding token (𝑛 𝑤 > 1), the agent conservatively decreases the worker’s token quota 𝑛 𝑤 by one (Line 15) to mitigate HoL blocking. Symmetrically, if queuing is instead dominated by the switch queue (i.e., 𝑅¯𝑞 < 𝑅th ) and the worker has not reached its maximum quota (𝑛 𝑤 < 𝑛 max ), the agent cautiously increases 𝑛 𝑤 by one (Lines 16–17) to help drain the switch queue. Only when neither of these cases applies does the agent interpret the event as persistent overload and increment the slice’s overload counter 𝑂𝑠 (Line 19). 𝑝99 When the slowdown stays below the threshold (𝑆 𝑤 ≤ 𝑆 th ), the agent aggregates the idleness of workers in a slice by summing up their idleness ratios 𝐼 𝑤 over the entire control interval (Lines 21– 22). After checking all the workers in a slice, the agent uses 𝑂𝑠 and the total idleness 𝐼𝑠 to infer the slice state (Lines 24–31). If both overload and underutilization signals are present, it equalizes local depths by averaging 𝑛 𝑤 within the slice (Lines 25–27), smoothing intra-slice imbalance without changing the slice’s worker set. Otherwise, it labels the slice as overloaded or underloaded for potential cross-slice rebalancing; a slice is considered underloaded only when the total idleness exceeds one worker-equivalent (𝐼𝑠 > 1), so that reassigning a worker is relatively safe in terms of not overloading the slice. The global rebalancing routine proceeds greedily (Lines 32–37): in each iteration, it pairs the most overloaded slice and the slice with the largest total idleness (Line 34), migrates the least-provisioned worker from the latter to the former (Lines 35–36), and updates the candidate sets accordingly (Line 37). The migrated worker’s token quota is initialized to 𝑛 𝑤★ = 1, after which subsequent epochs adjust its depth as needed. The scheduler agent interacts with workers via shared memory: workers expose their local statistics in a shared control block, and the agent updates control knobs. Per-worker control state is laid out on separate cache lines to avoid inter-worker false sharing [54]. We use a sampling interval of Δ𝑡 = 10 𝜇s and aggregate 𝐾 = 1, 000 samples per worker before each control decision. The slowdown threshold for triggering token-quota adjustments is set to 𝑆 th = 10, with the token quota capped at 𝑛 max = 8. We will analyze the effect of ratio threshold 𝑅th , which distinguishes worker-queue-dominated slowdown from switch-queue-dominated slowdown in evaluations (§5.3). Relation to backpressure. Unlike classical backpressure, which reacts monotonically to queue buildup, Algorithm 1 first determines the root cause of tail-latency degradation via the ratio 𝑅¯𝑞 and then takes proper actions to address either HoL blocking or tail-latency collapse (§2.3.2). Proc. ACM Netw., Vol. 4, No. CoNEXT2, Article 22. Publication date: June 2026.
Rain: RDMA-assisted In-Network Scheduling for Microsecond-scale Workloads
4
22:11
Implementation
We prototype Rain using an Intel Tofino switch [21] and commodity servers equipped with NVIDIA ConnectX-5 100 GbE NICs. On the switch, our implementation consumes 11 out of the 12 pipeline stages and uses 24.1% of the available SRAM without relying on any TCAM resources. On the server side, we use the data plane development kit (DPDK) [12] to implement high-performance packet I/O, including flow steering for dispatching tasks to their designated workers. Tofino’s per-stage memory is constrained, so we maximize queue depth by striping each queue entry across pipeline stages, storing only one byte per stage. Under this layout, the in-switch payload width is bounded by the number of stages dedicated to it: we allocate eight of them, capping in-switch task data at 8 bytes per entry alongside the required metadata (e.g., the client identifier). Note that, this 8-byte cap is an implementation limitation due to our pipeline stage budget rather than a design choice driven by application requirements. Under this design, the implementation supports up to 128 K queued task entries. Although we optimize queue depth aggressively, queue overflow under heavy load with significant workload dispersion cannot be eliminated. As a safeguard, we cap the total outstanding Í requests at 𝑄 max = 𝑄𝑠𝑤 + 𝑛 𝑤 , where 𝑄𝑠𝑤 is the switch queue capacity and 𝑛 𝑤 is the token quota 𝑤
of worker 𝑤. Scheduler agents distribute each client’s share of 𝑄 max via the control path, and clients enforce admission control locally and apply backpressure to the application layer, preventing silent drops and switch-side crashes. Note that, the adaptive scheduling is not an overload-control mechanism because it only regulates worker-side queues without limiting outstanding tasks globally. 5
Evaluations
We evaluate Rain with a series of experiments designed to answer the following questions: • How does Rain compare to the SOTAs of in-network schedulers in terms of tail latency and service time slowdown under static workloads? (§5.2) • How does Rain’s adaptive scheduling perform under dynamic workloads? (§5.3) • What extra packet rate and buffer footprint does Rain’s RDMA pre-writing incur? (§5.4) 5.1
Evaluation Setup
Our testbed consists of three servers directly connected to a programmable switch based on Intel Tofino ASICs. Each server is equipped with Intel Xeon Silver 4316 CPUs and a 100-Gbps NVIDIA ConnectX-5 NIC configured in the Ethernet mode. The NIC and all CPU cores used in our experiments reside on the same non-uniform memory access (NUMA) node to avoid cross-NUMA traffic. All the servers run Linux kernel 5.15.0 and use DPDK 22.11.4 to accelerate packet I/O. Two servers act as workers and one operates as the client. Both the client and workers are implemented as multi-threaded applications. Client threads generate workloads and collect throughput and latency statistics. Worker threads follow a symmetric thread model, i.e., each maintains a first-in-first-out (FIFO) task queue and executes tasks in a run-to-completion fashion. By default, we allocate 16 CPU cores to the client and 32 CPU cores to workers totally (across the two servers). 5.2
Evaluations with Static Workloads
We first evaluate Rain under static workloads whose statistical properties remain constant, and vary only the offered load to obtain load-latency curves. Each data point is obtained by first running static workloads at the selected load for one second for warming up, followed by a measurement interval of 10 seconds. The benchmark systems are as follows: • R2P2 [29]: It implements the JBSQ(𝑛) strategy using packet recirculation, and relies on explicit queue-depth reports from workers to keep its on-switch cache updated. Specifically, Proc. ACM Netw., Vol. 4, No. CoNEXT2, Article 22. Publication date: June 2026.
22:12
Zhihuang Ma, Xingming Cui, Xiaoliang Chen, and Zuqing Zhu
each task packet traverses the switch pipeline iteratively, sampling a few workers in each pass to determine whether a worker meets the target queue depth. Tasks begin by looking for workers with a queue depth of one. If none can be found after checking all the workers, the target queue depth might be increased by one2 . • RackSched [61]: It performs on-switch scheduling using a power-of-𝑘 approach3 . Upon the arrival of a task, the switch randomly selects a subset of workers and assigns the task to the worker with the shortest reported queue depth, without imposing an upper-bound on queue length. RackSched either depends on explicit worker reports or incurs an extra recirculation pass after each worker selection, to refresh the on-switch queue-depth cache. • Draconis [51]: It buffers tasks directly in the switch and adopts a pull-based model where workers fetch tasks proactively, to approximate JBSQ(1). When workers are idle and need new tasks, they have to continuously issue pull requests to retrieve the tasks. • Pallas [33]: It isolates tasks by type and dispatches tasks in each type to a dedicated subset of workers. For queue management, it uses a simple weighted round-robin policy to select workers and directly pushes tasks without limiting worker queue depth. Our performance comparisons focus mainly on in-network scheduling, and thus all the experiments use the same symmetric worker-thread model. Note that, some baseline systems encompass more than just an in-network scheduler, e.g., RackSched also includes an intra-server scheduler. To keep the discussions focused, we use the suffix “SW” to refer specifically to their switch-side implementations throughout the rest of this section.
99% Tail Latency (μs)
RAIN
R2P2-SW
RackSched-SW
Draconis-SW
100 80
100
300
60
200
40
50 100
20 0
Pallas-SW
400
2600
2800
3000
Load (kRPS) (a) Constant (10 μs)
0
2800
3000
3200
Load (kRPS) (b) Exponential (mean = 10 μs)
0 350
400
450
500
550
Load (kRPS) (c) Bimodal (50% 10 μs, 50% 100 μs)
Fig. 6. Tail latency versus system load under static synthetic workloads.
5.2.1 Synthetic Workloads. We first evaluate Rain using synthetic workloads with constant, exponential, and bimodal service-time distributions, and the results are shown in Figures 6(a)-6(c), respectively. In this setting, we disable slice partitioning to isolate the contribution of Rain’s bidirectional queuing mechanism. We do not distinguish task types, i.e., any task can be dispatched to any worker, and the switch-side queue is shared by all the tasks. This corresponds to a scenario where slice partitioning by task types is not feasible, and thus neither Rain nor Pallas can apply task-type-aware scheduling in this evaluation. The sizes of 80% of the tasks are 8-byte, while the remaining can use a longer size. For tasks larger than 8 bytes, Rain transfers payloads via RDMA multicasting, whereas Draconis incurs an additional delay (equal to a RTT) to retrieve task data. Observation 1: Rain achieves the best performance in tail-latency. In Figure 6(a) (with constant service time), both Rain and Pallas-SW achieve the shortest tail latency, when the load does not exceed 2,800 kRPS. As workload dispersion is negligible in this case, Pallas-SW’s round-robin 2We use either JBSQ(1) or JBSQ(2), depending on the specific workloads under evaluation. 3We set 𝑘 = 2 in the evaluations.
Proc. ACM Netw., Vol. 4, No. CoNEXT2, Article 22. Publication date: June 2026.
Rain: RDMA-assisted In-Network Scheduling for Microsecond-scale Workloads
22:13
and push-based policy does not induce noticeable HoL blocking, i.e., it effectively approximates the JBSQ(1) behavior provided by Rain at relatively low loads. However, as the load further increases or workload dispersion grows, the disadvantages of Pallas-SW become evident, and HoL blocking causes its tail latency to rise much faster, making it perform the worst in Figures 6(b) and 6(c). Interestingly, although R2P2-SW and Draconis-SW also implement JBSQ(1), they both provide noticeably longer tail latency than Rain. In Figure 6(a), when the load does not exceed 2,800 kRPS, Rain provides ∼15 µs tail latency, whereas that from R2P2-SW and Draconis-SW is roughly 30 µs and 35 µs, respectively. We attribute this gap to the overheads in the respective JBSQ(1) implementations of R2P2-SW and Draconis-SW. R2P2-SW heavily relies on recirculation, which can lead to task reordering, i.e., tasks undergoing recirculation might be overtaken by later arrivals that more quickly seize the opportunity to be scheduled to a worker, causing certain tasks to experience unexpectedly large numbers of recirculations and thus prolonging tail latency significantly. For Draconis-SW, workers pull tasks continuously, and even through we maximize the pull frequency without considering bandwidth overhead, the explicit pulling still diverges from the idea behavior of JBSQ(1). Moreover, Draconis-SW incurs extra fetch time for large tasks (workers need to retrieve task data after being dispatched metadata), further widening the tail latency gap relative to Rain. Push-based systems (RackSched-SW and Pallas-SW) fall behind pull-based designs (Rain and Draconis-SW), under highly-dispersive bimodal workloads. RackSched-SW’s power-of-𝑘 policy and push-based dispatch perform well and can even surpass certain pull-based designs, only when both workload dispersion and load are low, attributing to its relatively low implementation overhead. Observation 2: Rain fully utilizes system throughput. Pull-based schedulers normally offer better tail latency, but their non-work-conserving behavior, caused by fixed worker queue depths, can make them susceptible to tail latency collapse as load increases, which will significantly offset their advantage on tail latency. Specifically, both R2P2-SW and Draconis-SW use staticallyconfigured worker queue depths, which can hardly adapt to sudden load peaks, making them the system bottleneck, even the systems are ready to deliver higher throughput with deeper queues. In contrast, Rain’s adaptive scheduling dynamically adjusts worker queue depths by monitoring signs of HoL blocking in runtime, thereby avoiding the inherent limitations of aforementioned pull-based designs. For example, for the test cases in Figure 6, Rain increases the worker queue depth to 8 to adapt to high loads in the scenario with constant service time, whereas when the service time is under the bimodal distribution, it expands the queue depth more conservatively to 2 for heavy loads. We will further analyze the behavior and impact of adaptive scheduling in §5.3. R2P2-SW 99% Tail Latency (μs)
99% Tail Slowdown
40 30 20 10 0
500
1000
Load (kRPS) (a) Overall
1500
RackSched-SW
Draconis-SW
200
99% Tail Latency (μs)
RAIN 50
150 100 50 0
500
1000
Load (kRPS) (b) GET requests
1500
Pallas-SW
500 400 300 200 100 0
500
1000
1500
Load (kRPS) (c) SCAN requests
Fig. 7. Overall 99% tail slowdown and per-class 99% tail latency (fixed number of keys per request).
5.2.2 Workloads of Real-world Application. We further evaluate Rain using a real-world application, RocksDB [41], a widely deployed key-value store. In this evaluation, we enable task classification, as Proc. ACM Netw., Vol. 4, No. CoNEXT2, Article 22. Publication date: June 2026.
22:14
Zhihuang Ma, Xingming Cui, Xiaoliang Chen, and Zuqing Zhu
RocksDB requests naturally fall in distinct categories and exhibit substantial differences in average service time. By default, the workloads consist of 90% GET and 10% SCAN requests, where each GET request packet explicitly carries 10 keys, which the worker iteratively serves via 10 point Get operations with a key size of 64 bytes and each SCAN request specifies a range of 500 keys (a starting key plus a count), which the worker serves via iterator traversal with a key size of 1,024 bytes, taking approximately 8 𝜇s and 107 𝜇s in our setup, respectively. In both Rain and Pallas-SW, GET and SCAN requests use independent on-switch queues and dedicated workers (14 and 18 workers, respectively), while all the other systems use a common pool of workers. The results are shown in Figure 7, including the 99% tail latency for GET and SCAN requests and the overall 99% tail slowdown, defined as the completion time normalized by service time. Observation 3: Scheduling in slices delivers substantial tail-latency benefits. By enabling task classification on top of bidirectional queuing, this experiment isolates the additional benefit of slice-aware scheduling. Since both Rain and Pallas-SW classify workloads to process them separately, they achieve significantly lower tail latency than those without. Specifically, they can keep the overall 99% tail slowdown below 10 even when the load reaches 1,650 kRPS, whereas the best-performing design without classification (i.e., Draconis-SW) can only maintain the slowdown below 10 when the load is below 900 kRPS. Although Draconis-SW does not perform classification, its JBSQ(1) policy allows it to maintain reasonable performance under moderate load. However, its non-work-conserving behavior, combined with the highly-dispersive workloads used in this evaluation, makes it prone to overload and tail-latency collapse. For R2P2-SW, we observe that the tail latency of its GET tasks approaches to the service time of SCAN tasks even under relatively light loads. This further supports our earlier observation that its recirculation-based JBSQ implementation introduces task reordering, which in turn causes the tail latency to deviate from its theoretical prediction. As expected, the tail latency from RackSched-SW, which relies on a push-based strategy and imposes no limit on worker queue depth, increases rapidly with load for both task types, causing a substantial rise in overall slowdown. The downside of scheduling in slices is reduced peak throughput. Since Rain and Pallas-SW divide workers across task types, their maximum achievable throughput is lower than that of systems with shared workers, which is reflected by the earlier overload point of SCAN requests shown in Figure 7(c). This reduction in peak throughput is an inherent cost of slice-based scheduling, but the benefit in overall 99% tail slowdown remains substantial, justifying its necessity clearly. RAIN
200
100
0
500
1000
Load (kRPS) (a) Overall
Pallas-SW
100
99% Tail Latency (μs)
99% Tail Latency (μs)
99% Tail Slowdown
300
80 60 40 20 0
500
1000
Load (kRPS) (b) GET requests
800 600 400 200 0
500
1000
Load (kRPS) (c) SCAN requests
Fig. 8. Overall 99% slowdown and per-class 99% tail latency (keys per request in exponential distribution).
Observation 4: Rain effectively adapts to real-world workloads. In the previous evaluation, both the number of keys and item sizes were fixed for each task type, resulting in nearly constant per-class service time. This explains why the push-based Pallas-SW performed similarly to Rain. Proc. ACM Netw., Vol. 4, No. CoNEXT2, Article 22. Publication date: June 2026.
Rain: RDMA-assisted In-Network Scheduling for Microsecond-scale Workloads
22:15
To evaluate with a more realistic scenario, we draw the numbers of keys per GET and SCAN request from exponential distributions with means of 10 and 500 keys, respectively, and repeat the tests. The results are plotted in Figure 8. Consistent with our observations in the tests with synthetic workloads, when Pallas-SW cannot rely on classification to control task dispersion, its push-based queuing strategy makes tail latency rise rapidly. In contrast, Rain keeps regulating worker queue depth to avoid HoL blocking. Therefore, Rain achieves 1.75× higher throughput than Pallas-SW before its 99% tail slowdown exceeds 10, and at the load where Pallas-SW approaches overloaded, Rain reduces the 99% tail latency by 41.10% and 37.69% for GET and SCAN requests, respectively. RAIN (Rth = 0.2) Worker Queue Depth
99% Tail Latency (μs)
RAIN w/o adaptive scheduling 300 200 100 0
2000
2250
2500
2750
3000
RAIN (Rth = 0.5)
RAIN (Rth = 0.8)
3 2 1
3250
2000
2250
2500
2750
Load (kRPS)
Load (kRPS)
(a)
(b)
3000
3250
Fig. 9. Impact of HoL-blocking threshold 𝑅th on Rain’s adaptive scheduling4 .
5.3
Evaluations with Dynamic Workloads
In this section, we evaluate Rain’s adaptive scheduling under dynamic workloads. These experiments isolate adaptive scheduling’s contribution beyond the static mechanisms above. 5.3.1 Parameter Tuning. Rain’s adaptive scheduler exposes two key parameters: the target 99% tail slowdown 𝑆 th and the threshold 𝑅th . Since the former (𝑆 th ) is typically dictated by user requirements, we fix it as 10, while the latter (𝑅th ) limits the fraction of queuing delay attributable to worker-side HoL blocking. Using the workloads with exponentially distributed service time (mean value at 10 𝜇s), we initialize the worker queue depth as 2 and gradually increase system load to test different values of 𝑅th . The results are shown in Figure 9. Observation 5: 𝑅th governs the tradeoff between tail latency and system stability. A smaller 𝑅th lets Rain tolerate less HoL blocking, making it more aggressive in reducing the worker queue depth 𝑛 to control HoL blocking, while a larger 𝑅th yields more conservative adjustments. As shown in Figure 9, when 𝑅th = 0.2, the system reduces 𝑛 from 2 to 1 at relatively low load, achieving the best tail latency when the load is below 2,300 kRPS. However, this aggressive reduction makes the system more susceptible to overload and tail-latency collapse at higher loads. In contrast, when we set 𝑅th = 0.8, the queue depth remains unchanged until the load reaches 3,000 kRPS and is increased only when the system approaches overloaded. In summary, a smaller value of 𝑅th favors lower tail latency, while increasing its value makes the system emphasize more on stability. 5.3.2 Time-varying Workloads. Finally, we consider a scenario with time-varying workloads, in which both the offered load and the dispersion of workloads can change dynamically. We set 𝑅th as 0.3 and monitor the tail latency to evaluate the effectiveness of Rain’s adaptive scheduling. Initially, the workers are evenly divided into two slices, and then their allocations are adjusted by 4 Queue depths in (b) are integer; curves offset vertically for clarity.
Proc. ACM Netw., Vol. 4, No. CoNEXT2, Article 22. Publication date: June 2026.
Zhihuang Ma, Xingming Cui, Xiaoliang Chen, and Zuqing Zhu
99% Tail Latency (μs)
22:16
RAIN w/o adaptive scheduling RAIN
3000
2000
1000
0
0
1
2
3
4
Time (s)
Fig. 10. Real-time 99% tail latency sampled every 10 ms. At 𝑡 = 1 s, the dispersion of input workloads increases for 1 s before returning to normal, and then at 𝑡 = 3 s, the system load increases for 1 s and then falls back.
the adaptive scheduling. The worker queue depth is initialized as 2. We inject workloads into one of the slices, and Figure 10 shows the results from the working slice. Observation 6: Rain’s adaptive scheduling maintains low tail latency for time-varying workloads. We begin with workloads whose service time is exponentially distributed (mean value at 10 𝜇s) at a very light system load of 160 kRPS. As the system is far from saturation and the slowdown remains small, adaptive scheduling does not intervene. At 𝑡 = 1 s, we increase the dispersion of workloads by switching to a bimodal distribution with 50% of tasks having a service time of 10 𝜇s and the service time of remaining 50% at 100 𝜇s. Hence, although the system is still underutilized, the degree of HoL blocking rises significantly, and in response, the adaptive scheduling reduces the worker queue depth from 2 to 1, thereby improving tail latency. At 𝑡 = 3 s, we increase the offered load to 1,600 kRPS, pushing the system to overloaded and causing tail latency to spike. Over the next five control intervals (each lasting 10 ms), Rain’s adaptive scheduler performs the following adjustments in sequence: it 1) restores the queue depth from 1 to 2, 2) further increases the queue depth to 3, 3) moves one worker from the idle slice to the working one, when observing that the tail slowdown remains above the target, 4) borrows the second worker from the idle slice, after which the tail latency returns to within the bound of target slowdown, and 5) reduces the queue depth to 2, when detecting that, with two additional workers, the switch queue stays at zero while the worker-side HoL blocking remains high. These adjustments enable Rain to respond promptly to workload dynamics and maintain stable tail latency. 5.4
RDMA Pre-writing Overhead Analysis
Rain pre-writes a task’s payload via RDMA multicasting only when the payload exceeds the on-switch cache capacity (e.g., larger than 8 bytes) and no free worker token is available upon arrival; otherwise, the task follows the normal forwarding path. The aggregate RDMA write rate is thus 𝑅RDMA = 𝑇 · 𝛼 · 𝛽 · 𝑚, where 𝑇 is task throughput, 𝛼 the fraction of tasks buffered in the on-switch queue, 𝛽 the fraction whose payload cannot be cached, and 𝑚 the number of candidate workers per multicast. Since 𝛽 and 𝑚 are fixed by application and deployment, 𝛼 is the only runtime knob. To stress-test it, we replay the three synthetic workloads of §5.2 with all tasks enlarged to 16 bytes (𝛽 = 1) and 𝑚 = 2, with adaptive scheduling (§3.3) enabled and queue depth ranging from 1 to 4. Figure 11 reports the measured 𝛼. Observation 7: Rain’s RDMA pre-writing incurs bounded overhead in both write rate and memory. At low-to-moderate loads, 𝛼 stays negligible across all three workloads, as arriving tasks almost always find a free token and bypass on-switch queuing. Near saturation 𝛼 rises, but adaptive scheduling counteracts this by enlarging the worker queue depth to replenish free tokens. Proc. ACM Netw., Vol. 4, No. CoNEXT2, Article 22. Publication date: June 2026.
Rain: RDMA-assisted In-Network Scheduling for Microsecond-scale Workloads
22:17
On-switch Queuing Ratio (%)
Non-trivial overhead thus arises only near saturation with a large fraction of large-payload tasks, which remains tolerable since microsecond-scale applications are typically IOPS-bounded rather than bandwidth-bounded; if tighter bounds are required, operators can cap slice size or restrict multicasting to a subset of servers per slice. The per-server memory overhead is likewise bounded, since each worker only needs to reserve a small pre-write buffer proportional to its adaptive queue depth (at most 8 in our evaluations) and the largest task payload, which is negligible compared to the memory already provisioned on modern servers. 100
100
100
80
80
80
60
60
60
40
40
40
20
20
20
0
0
2500
2750
3000
3250
Load (kRPS) (a) Constant (10 μs)
2500
3000
3500
Load (kRPS) (b) Exponential (mean = 10 μs)
0
400
500
600
Load (kRPS) (c) Bimodal (50\% 10 μs, 50\% 100 μs)
Fig. 11. On-switch queuing ratio versus system load under static synthetic workloads.
6 6.1
Discussion Failure Handling and Correctness of the RDMA Engine
Rain targets a single-rack deployment where the ToR switch also play the role of in-network scheduler, forming a controlled RDMA domain (e.g., RoCEv2); transient packet loss, NIC/QP errors, and worker or switch restarts can still occur, and our goal is to bound their impact rather than mask all failures in the data plane. For correctness, the outstanding tokens per node are capped by the RDMA ring-buffer size to prevent overrun, and each RDMA-written payload carries a copy of the request metadata so a worker executes a task only when the switch-delivered descriptor matches the buffer slot, preventing consuming stale or mismatched payloads. For recovery, a worker detecting a QP error stops issuing tokens and notifies the scheduler agent, which deregisters the worker and resets its RDMA state (QPN, PSN, rkey, buffer offset, tokens); once the QP is re-established, the agent registers the fresh state and the worker rejoins its slice. Since state is tracked per worker, a failure only affects that worker’s outstanding tokens while other workers and slices are not interrupted; orphaned tasks are recovered via end-host timeout and resubmission, and unused RDMA replicas on non-selected candidates are reclaimed naturally as the ring buffer advances. 6.2
RDMA MTU and Multi-Packet Requests
By configuring the host networking MTU to be no larger than the RDMA MTU (typically 4 KB), each RDMA WRITE fits within a single RDMA packet, and the switch never needs to split one WRITE into multiple operations. When a logical request exceeds one MTU, the end-host network stack segments it into multiple packets before transmission. To maintain request affinity, Rain lets the client send the first packet through the normal scheduling path and then directs all subsequent ones to the assigned worker based on the scheduling metadata returned in the first response. 6.3
Threats to Validity and Future Work
Scalability. Worker scaling will not be expensive: each new worker consumes only lightweight pipeline metadata (e.g., QPN). The binding constraint actually comes from the client side, as more clients can generate more outstanding requests that need to share the 128 K-entry on-switch task Proc. ACM Netw., Vol. 4, No. CoNEXT2, Article 22. Publication date: June 2026.
22:18
Zhihuang Ma, Xingming Cui, Xiaoliang Chen, and Zuqing Zhu
buffer. Scaling beyond this single-rack budget calls for a multi-rack extension, e.g., hierarchical scheduling where per-rack Rain instances serve as local schedulers coordinated by a lightweight global layer. We leave this as future work. Switch-side overflow and overload control. Switch queues are inherently shallower than host-side queues, so pull-based in-network schedulers, including Rain, are more vulnerable to overflow than push-based designs under open-loop arrivals. Our current safeguard is a global admission-control cap (§4). More sophisticated overload control is left as future work. Pipeline resource budget. Our Tofino prototype uses 11 of 12 pipeline stages. If other data-plane programs (e.g., load balancing, telemetry, or access control) need to be co-deployed, the pipeline layout has to be carefully planned to stay within the stage and SRAM limits. 7
Related Work
This section highlights the studies that are relevant to Rain but have not been commented yet. Intra-server task scheduling. Early server-level systems [2] often used hash-based techniques such as receiver-side scaling (RSS) to distribute requests across CPU cores. Beyond these randomized mechanisms, subsequent studies [9, 10, 23, 25] explored centralized software schedulers that can make global dispatch decisions to improve core utilization and reduce tail latency. Complementary efforts leveraged hardware acceleration, including multicore system-on-a-chip designs [18], FPGAbased NIC schedulers [34], and SmartNIC-assisted architectures [7]. These approaches focused on optimizing task distribution within a single server and are therefore orthogonal and complementary to the in-network scheduling problem addressed in this work. CPU scheduling. A broad class of systems dynamically adjusted CPU allocations in response to workload variations or interference, typically via user-level runtime operations that scale processing capacity on demand [3, 14, 22, 27, 40, 43, 45]. These mechanisms primarily regulated CPU resources directly. In contrast, our adaptive scheduling reacts to tail-latency behavior and modulates worker queue depth, providing a complementary control channel alongside CPU-level schedulers. RDMA in programmable switches. Integrating RDMA with programmable switches has expanded the design opportunity for high-performance distributed systems. Several efforts have demonstrated Tofino-based data planes issuing RDMA READ/WRITE requests directly [5, 24, 28, 47, 48], enabling switches to actively access remote memory. Programmable switches have also been used to extend or optimize RDMA fabric capabilities [32, 39, 50]. 8
Conclusion
This paper presented Rain, an RDMA-assisted in-network scheduler built on a programmable switch for microsecond-scale, latency-critical workloads. Rain integrates bidirectional on-switch queuing, a switch-driven RDMA engine, and slice-aware scheduling to approximate centralized JBSQ behavior while efficiently supporting large tasks. Our study further revealed that real-world systems can diverge from theoretical predictions: shallow worker queues do not always improve tail latency. Leveraging this insight, Rain incorporates an adaptive scheduler that dynamically tunes worker queue depths and worker-to-slice mappings, preserving work conservation and sustaining SLO-level tail latency across diverse workloads, and thus outperforms the SOTAs. Acknowledgments We would like to thank the anonymous reviewers and our shepherd, Gyuyeong Kim, for providing valuable feedback. This work was supported by the National Key R&D Program of China under Grant 2023YFB2903903. Proc. ACM Netw., Vol. 4, No. CoNEXT2, Article 22. Publication date: June 2026.
Rain: RDMA-assisted In-Network Scheduling for Microsecond-scale Workloads
22:19
References [1] Luiz André Barroso, Mike Marty, David A. Patterson, and Parthasarathy Ranganathan. 2017. Attack of the Killer Microseconds. Commun. ACM 60, 4 (April 2017), 48–54. [2] Adam Belay, George Prekas, Ana Klimovic, Samuel Grossman, Christos Kozyrakis, and Edouard Bugnion. 2014. IX: a Protected Dataplane Operating System for High Throughput and Low Latency. In Proc. of USENIX OSDI 2014. 49–65. [3] Adam Belay, George Prekas, Mia Primorac, Ana Klimovic, Samuel Grossman, Christos Kozyrakis, and Edouard Bugnion. 2016. The IX Operating System: Combining Low Latency, High Throughput, and Efficiency in a Protected Dataplane. ACM Trans. Comput. Syst. 34, 4 (Dec. 2016), 1–39. [4] Eric Boutin, Jaliya Ekanayake, Wei Lin, Bing Shi, Jingren Zhou, Zhengping Qian, Ming Wu, and Lidong Zhou. 2014. Apollo: Scalable and Coordinated Scheduling for Cloud-Scale Computing. In Proc. of USENIX OSDI 2014. 285–300. [5] Xinyi Chen, Liangcheng Yu, Vincent Liu, and Qizhen Zhang. 2023. Cowbird: Freeing CPUs to Compute by Offloading the Disaggregation of Memory. In Proc. of ACM SIGCOMM 2023. 1060–1073. [6] Eyal Cidon, Sean Choi, Sachin Katti, and Nick McKeown. 2017. AppSwitch: Application-Layer Load Balancing within a Software Switch. In Proc. of ACM APNet 2017. 64–70. [7] Alexandros Daglis, Mark Sutherland, and Babak Falsafi. 2019. RPCValet: NI-driven Tail-Aware Balancing of µs-Scale RPCs. In Proc. of ACM ASPLOS 2019. 35–48. [8] Jeffrey Dean and Luiz André Barroso. 2013. The Tail at Scale. Commun. ACM 56, 2 (Feb. 2013), 74–80. [9] Henri Maxime Demoulin, Joshua Fried, Isaac Pedisich, Marios Kogias, Boon Thau Loo, Linh Thi Xuan Phan, and Irene Zhang. 2021. When Idling is Ideal: Optimizing Tail-Latency for Heavy-Tailed Datacenter Workloads with Perséphone. In Proc. of ACM SOSP 2021. 621–637. [10] Diego Didona and Willy Zwaenepoel. 2019. Size-Aware Sharding for Improving Tail Latencies in In-Memory Key-Value Stores. In Proc. of USENIX NSDI 2019. 79–94. [11] Xiaoyan Dong, Xiaoliang Chen, and Zuqing Zhu. 2025. On the Risk-aware Connection Defragmentation in OCS-based Data-center Networks. IEEE Trans. Netw. Serv. Manag. 22, 5 (2025), 3909–3920. [12] DPDK. 2025. DPDK. Retrieved November 11, 2025 from https://www.dpdk.org/. [13] DPDK. 2025. Poll Mode Driver. Retrieved November 11, 2025 from https://doc.dpdk.org/guides-24.03/prog_guide/poll_ mode_drv.html. [14] Joshua Fried, Zhenyuan Ruan, Amy Ousterhout, and Adam Belay. 2020. Caladan: Mitigating Interference at Microsecond Timestamps. In Proc. of USENIX OSDI 2020. 281–297. [15] Adithya Gangidi, Rui Miao, Shengbao Zheng, Sai Jayesh Bondu, Guilherme Goes, Hany Morsy, Rohit Puri, et al. 2024. RDMA over Ethernet for Distributed Training at Meta Scale. In Proc. of ACM SIGCOMM 2024. 57–70. [16] Ionel Gog, Malte Schwarzkopf, Adam Gleave, Robert NM Watson, and Steven Hand. 2016. Firmament: Fast, Centralized Cluster Scheduling at Scale. In Proc. of USENIX OSDI 2016. 99–115. [17] Long Gong and Zuqing Zhu. 2014. Virtual Optical Network Embedding (VONE) over Elastic Optical Networks. J. Lightw. Technol. 32, 3 (2014), 450–460. [18] Jack Tigar Humphries, Kostis Kaffes, David Mazières, and Christos Kozyrakis. 2019. Mind the Gap: A Case for Informed Request Scheduling at the NIC. In Proc. of ACM HotNets 2019. 60–68. [19] Stephen Ibanez, Alex Mallery, Serhat Arslan, Theo Jepsen, Muhammad Shahbaz, Changhoon Kim, and Nick McKeown. 2021. The Nanopu: A Nanosecond Network Stack for Datacenters. In Proc. of USENIX OSDI 2021. 239–256. [20] Intel. 2023. Core Utilization in DPDK Apps. Retrieved November 11, 2025 from https://www.intel.com/content/www/ us/en/docs/vtune-profiler/cookbook/2023-0/core-utilization-in-dpdk-apps.html. [21] Intel. 2025. Intel Tofino. Retrieved November 11, 2025 from https://www.intel.com/content/www/us/en/ark/products/ series/210606/intel-tofino.html. [22] Călin Iorgulescu, Reza Azimi, Youngjin Kwon, Sameh Elnikety, Manoj Syamala, Vivek Narasayya, and Herodotos Herodotou. 2018. PerfIso: Performance Isolation for Commercial Latency-Sensitive Services. In Proc. of USENIX ATC 2018. 519–532. [23] Rishabh Iyer, Musa Unal, Marios Kogias, and George Candea. 2023. Achieving Microsecond-Scale Tail Latency Efficiently with Approximate Optimal Scheduling. In Proc. of ACM SOSP 2023. 466–481. [24] Matthias Jasny, Lasse Thostrup, Sajjad Tamimi, Andreas Koch, Zsolt István, and Carsten Binnig. 2024. Zero-sided RDMA: Network-Driven Data Shuffling for Disaggregated Heterogeneous Cloud DBMSs. ACM Manag. Data 2 (Jan. 2024), 1–28. [25] Kostis Kaffes, Timothy Chong, Jack Tigar Humphries, Adam Belay, David Mazières, and Christos Kozyrakis. 2019. Shinjuku: Preemptive Scheduling for 𝜇second-scale Tail Latency. In Proc. of USENIX NSDI 2019. 345–360. [26] Anuj Kalia, Michael Kaminsky, and David G. Andersen. 2016. sand Simple Distributed Transactions with Two-Sided (RDMA) Datagram RPCs. In Proc. of USENIX OSDI 2016. 185–201. [27] Antoine Kaufmann, Tim Stamler, Simon Peter, Naveen Kr Sharma, Arvind Krishnamurthy, and Thomas Anderson. 2019. TAS: TCP Acceleration as an OS Service. In Proc. of ACM EuroSys 2019. 1–16. Proc. ACM Netw., Vol. 4, No. CoNEXT2, Article 22. Publication date: June 2026.
22:20
Zhihuang Ma, Xingming Cui, Xiaoliang Chen, and Zuqing Zhu
[28] Daehyeok Kim, Zaoxing Liu, Yibo Zhu, Changhoon Kim, Jeongkeun Lee, Vyas Sekar, and Srinivasan Seshan. 2020. TEA: Enabling State-Intensive Network Functions on Programmable Switches. In Proc. of ACM SIGCOMM 2020. 90–106. [29] Marios Kogias, George Prekas, Adrien Ghosn, Jonas Fietz, and Edouard Bugnion. 2019. R2P2: Making RPCs First-Class Datacenter Citizens. In Proc. of USENIX ATC 2019. 863–880. [30] KREONET. 2025. In-Band Network Telemetry. Retrieved November 11, 2025 from https://www.kreonet.net/eng/R&D/ pageView/899?t=1762646400030. [31] Baojia Li and Zuqing Zhu. 2022. GNN-based Hierarchical Deep Reinforcement Learning for NFV-Oriented Online Resource Orchestration in Elastic Optical DCIs. J. Lightw. Technol. 40, 4 (2022), 935–946. [32] Wenxue Li, Junyi Zhang, Yufei Liu, Gaoxiong Zeng, Zilong Wang, Chaoliang Zeng, Pengpeng Zhou, Qiaoling Wang, and Kai Chen. 2024. Cepheus: Accelerating Datacenter Applications with High-Performance RoCE-Capable Multicast. In Proc. of IEEE HPCA 2024. 908–921. [33] Xudong Liao, Han Tian, Xinchen Wan, Chaoliang Zeng, Hao Wang, Junxue Zhang, Mengyu Ma, Guyue Grace Liu, and Kai Chen. 2025. Towards Optimal Rack-Scale 𝜇s-LevelCPU Scheduling through In-Network Workload Shaping. In Proc. of USENIX ATC 2025. 179–198. [34] Jiaxin Lin, Adney Cardoza, Tarannum Khan, Yeonju Ro, Brent E. Stephens, Hassan Wassel, and Aditya Akella. 2023. RingLeader: Efficiently Offloading Intra-Server Orchestration to NICs. In Proc. of USENIX NSDI 2023. 1293–1308. [35] Junjie Liu, Wei Lu, Fen Zhou, Ping Lu, and Zuqing Zhu. 2017. On Dynamic Service Function Chain Deployment and Readjustment. IEEE Trans. Netw. Serv. Manag. 14, 3 (2017), 543–553. [36] Redis Ltd. 2025. Redis - The Real-time Data Platform. Retrieved November 11, 2025 from https://redis.io/. [37] Ping Lu, Liang Zhang, Xiahe Liu, Jingjing Yao, and Zuqing Zhu. 2015. Highly-Efficient Data Migration and Backup for Big Data Applications in Elastic Optical Inter-Datacenter Networks. IEEE Netw. 29, 5 (2015), 36–42. [38] Qian Lv, Yuxiao Zhang, Suoning Zhang, Ruoxing Li, Ke Meng, Bowen Zhang, Fuguang Huang, Xiaoliang Chen, and Zuqing Zhu. 2025. On the TPE Design to Efficiently Accelerate Hitless Reconfiguration of OCS-based DCNs. IEEE J. Sel. Areas Commun. 43, 5 (2025), 1780–1792. [39] Zhihuang Ma, Zichen Xu, Tingyu Li, Zijiang Yang, Xiaoliang Chen, and Zuqing Zhu. 2025. Rdmax: Scalable RDMA RPC on Reliable Connection Through QP Multiplexing and In-Network Dispatching. IEEE Trans. Networking (Nov. 2025), 1–15. [40] Michael Marty, Marc de Kruijf, Jacob Adriaens, Christopher Alfeld, Sean Bauer, Carlo Contavalli, Michael Dalton, et al. 2019. Snap: A Microkernel Approach to Host Networking. In Proc. of ACM SOSP 2019. 399–413. [41] Inc. Meta Platforms. 2025. RocksDB - A Persistent Key-Value Store. Retrieved November 11, 2025 from https://rocksdb. org/. [42] Michael Mitzenmacher. 2001. The Power of Two Choices in Randomized Load Balancing. IEEE Trans. Parallel Distrib. Syst. 12, 10 (Oct. 2001), 1094–1104. [43] Amy Ousterhout, Joshua Fried, Jonathan Behrens, Adam Belay, and Hari Balakrishnan. 2019. Shenango: Achieving High CPU Efficiency for Latency-Sensitive Datacenter Workloads. In Proc. of USENIX NSDI 2019. 361–378. [44] Kay Ousterhout, Patrick Wendell, Matei Zaharia, and Ion Stoica. 2013. Sparrow: Distributed, Low Latency Scheduling. In Proc. of ACM SOSP 2013. 69–84. [45] Henry Qin, Qian Li, Jacqueline Speiser, Peter Kraft, and John Ousterhout. 2018. Arachne: Core-Aware Thread Management. In Proc. of USENIX OSDI 2018. 145–160. [46] Xiaoqi Ren, Ganesh Ananthanarayanan, Adam Wierman, and Minlan Yu. 2015. Hopper: Decentralized SpeculationAware Cluster Scheduling at Scale. In Proc. of ACM SIGCOMM 2015. 379–392. [47] Amedeo Sapio, Marco Canini, Chen-Yu Ho, Jacob Nelson, Panos Kalnis, Changhoon Kim, Arvind Krishnamurthy, Masoud Moshref, Dan Ports, and Peter Richtarik. 2021. Scaling Distributed Machine Learning with In-Network Aggregation. In Proc. of USENIX NSDI 2021. 785–808. [48] Mariano Scazzariello, Tommaso Caiazzi, Hamid Ghasemirahni, Tom Barbette, Dejan Kostić, and Marco Chiesa. 2023. A High-Speed Stateful Packet Processing Approach for Tbps Programmable Switches. In Proc. of USENIX NSDI 2023. 1237–1255. [49] Arjun Singhvi, Nandita Dukkipati, Prashant Chandra, Hassan MG Wassel, Naveen Kr Sharma, Anthony Rebello, and Henry Schuh. 2025. Falcon: A reliable, low latency hardware transport. In Proc. of ACM SIGCOMM 2025. 248–263. [50] Cha Song, Xin Khooi, Raj Joshi, Inho Choi, Jialin Li, and Mun Chan. 2023. Network Load Balancing with In-network Reordering Support for RDMA. In Proc. of ACM SIGCOMM 2023. 816–831. [51] Sreeharsha Udayashankar, Ashraf Abdel-Hadi, Ali Mashtizadeh, and Samer Al-Kiswany. 2024. Draconis: NetworkAccelerated Scheduling for Microsecond-Scale Workloads. In Proc. of ACM EuroSys 2024. 333–348. [52] Abhishek Verma, Luis Pedrosa, Madhukar Korupolu, David Oppenheimer, Eric Tune, and John Wilkes. 2015. Large-Scale Cluster Management at Google with Borg. In Proc. of ACM EuroSys 2015. 1–17. [53] Xingda Wei, Jiaxin Shi, Yanzhe Chen, Rong Chen, and Haibo Chen. 2015. Fast In-Memory Transaction Processing Using RDMA and HTM. In Proc. of ACM SOSP 2015. 87–104.
Proc. ACM Netw., Vol. 4, No. CoNEXT2, Article 22. Publication date: June 2026.
Rain: RDMA-assisted In-Network Scheduling for Microsecond-scale Workloads
22:21
[54] Wikipedia. 2025. False Sharing. Retrieved November 11, 2025 from https://en.wikipedia.org/wiki/False_sharing. [55] Wikipedia. 2025. RDMA over Converged Ethernet. Retrieved November 11, 2025 from https://en.wikipedia.org/wiki/ RDMA_over_Converged_Ethernet. [56] Xuexia Xie, Binjun Tang, Xiaoliang Chen, and Zuqing Zhu. 2025. P4INC-AOI: All-Optical Interconnect Empowered by In-Network Computing for DML Workloads. IEEE/ACM Trans. Netw. 33, 3 (2025), 1236–1251. [57] Hao Yang and Z. Zhu. 2024. Traffic-aware Configuration of All-Optical Data Center Networks based on Hyper-FleXLION. IEEE/ACM Trans. Netw. 32, 3 (2024), 2675–2688. [58] Parham Yassini, Khaled Diab, Saeed Mahloujifar, and Mohamed Hefeeda. 2024. Horus: Granular In-Network Task Scheduler for Cloud Datacenters. In Proc. of USENIX NSDI 2024. 1–22. [59] Matei Zaharia, Mosharaf Chowdhury, Michael J. Franklin, Scott Shenker, and Ion Stoica. 2010. Spark: Cluster Computing with Working Sets. In Proc. of USENIX HotCloud 2010. 10–10. [60] Chaoliang Zeng, Layong Luo, Teng Zhang, Zilong Wang, Luyang Li, Wenchen Han, Nan Chen, et al. 2022. Tiara: A Scalable and Efficient Hardware Acceleration Architecture for Stateful Layer-4 Load Balancing. In Proc. of USENIX NSDI 2022. 1345–1358. [61] Hang Zhu, Kostis Kaffes, Zixu Chen, Zhenming Liu, Christos Kozyrakis, Ion Stoica, and Xin Jin. 2020. RackSched: A Microsecond-Scale Scheduler for Rack-Scale Computers. In Proc. of USENIX OSDI 2020. 1225–1240.
Received December 2025; accepted April 2026
Proc. ACM Netw., Vol. 4, No. CoNEXT2, Article 22. Publication date: June 2026.