Partially Observable Learning for Multi-Platform Dispatch Optimization

Partially Observable Learning for Multi-Platform Dispatch Optimization Fengming Yao

Man Luo

fy283@exeter.ac.uk University of Exeter Exeter, UK

M.Luo@exeter.ac.uk University of Exeter Exeter, UK

arXiv:2608.10897v1 [cs.LG] 11 Aug 2026

Abstract Instant delivery platforms have become a critical component of urban logistics, increasingly relying on crowdsourced couriers to fulfill highly dynamic orders. In real-world systems, couriers are not exclusive to a single platform and may concurrently serve multiple platforms, while each platform can only observe its own orders and couriers’ interactions due to privacy and operational constraints. This results in a multi-platform dispatch environment with inherent partial observability. However, most existing works on dispatch optimization assume full courier observability and mandatory assignment acceptance, causing substantial performance degradation when deployed in realistic multi-platform settings. In this paper, we propose POLO, a partially observable multi-agent reinforcement learning framework for dispatching optimization in multi-platform instant delivery systems. POLO firstly models each platform-grid pair as an independent agent that learns dispatch policies solely from platform-local observations, aligning the learning process with real-world privacy and operational constraints. To support effective decision-making under incomplete and heterogeneous courier information, POLO introduces a novel attention-based policy representation that selectively aggregates inter-courier information. Moreover, we design a counterfactual reward shaping mechanism to mitigate the non-stationarity induced by joint actions across grids, leading to more stable and scalable learning. We develop a high-fidelity simulator to evaluate dispatch performance under varying numbers of platforms and system scales. Extensive experiments demonstrate that POLO consistently outperforms strong baselines in terms of platform revenue and courier travel efficiency, highlighting its robustness and effectiveness in realistic multi-platform settings. The implementation of both the simulator and the proposed model is available at https://github.com/yaofengming1999/polo-courier.git

CCS Concepts • Information systems → Decision support systems.

Keywords Instant delivery, Multi-agent reinforcement learning, Crowdsourced delivery, Multi-platform dispatch Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org. xxx ’26, © 2026 Copyright held by the owner/author(s). Publication rights licensed to ACM. ACM ISBN 978-x-xxxx-xxxx-x/YYYY/MM https://doi.org/10.1145/nnnnnnn.nnnnnnn

Figure 1: Multi-platform instant delivery. A courier may serve multiple platforms and reject the assigned order.

ACM Reference Format: Fengming Yao and Man Luo. 2026. Partially Observable Learning for MultiPlatform Dispatch Optimization. In . ACM, New York, NY, USA, 11 pages. https://doi.org/10.1145/nnnnnnn.nnnnnnn

1

Introduction

Instant delivery provides on-demand transportation of goods, food, and groceries in response to real-time customer requests and has become a critical component of urban logistics systems [8, 30]. The rapid growth of instant delivery markets has led to the coexistence of multiple delivery platforms within the same regions [5, 8, 24], such as Meituan, Ele.me, and JD.com in China, Deliveroo, Uber Eats, and Just Eat in Europe, and DoorDash and Grubhub in the United States. This multi-platform setting fundamentally reshapes how dispatch decisions are made in urban delivery systems. What’s more, couriers play a central role in this process, and a substantial fraction of them are crowdsourced workers who are not exclusively affiliated with a single platform [1, 8]. While crowdsourced couriers provide platforms with a flexible and scalable workforce [2], they also introduce limited control over courier behavior and increased uncertainty in system dynamics [21]. As illustrated in Fig. 1, multiplatform dispatch is characterized by partial observability and decentralized control. Each platform can only observe its own orders and courier interactions, while courier availability, future routes, and acceptance decisions are jointly influenced by actions taken across competing platforms. As a result, the global system state is not accessible to any single decision-maker. This partial observability introduces strong non-stationarity into the order dispatch process and significantly degrades the performance of existing methods when applied to multi-platform settings. Order dispatch is a core decision-making problem in instant delivery and has been extensively studied [23, 35]. Dispatch decisions directly affect routing feasibility, delivery timelines, customer satisfaction, and platform revenue [12, 14, 25, 27, 32]. In practice, due to

xxx ’26, 2026,

orders overlapping along courier routes, dispatch is naturally formulated as a concurrent spatio-temporal decision problem [28], where platforms must determine which orders to serve and how to assign heterogeneous couriers at each decision epoch. To address the complexity of concurrent dispatch, recent work has applied reinforcement learning (RL) to concurrent dispatch, modeling couriers or courier-order pairs as learning agents parameterized by spatial and temporal features [10, 12, 28]. While these methods demonstrate strong performance in controlled settings, they typically rely on the assumption of full observability of courier states and exclusive platform control. Such assumptions are reasonable in single-platform systems but no longer hold in realistic multi-platform environments, where couriers may serve multiple platforms simultaneously and platform visibility is inherently limited. Existing work has also studied courier acceptance behavior [3] and capacity management [21] in multi-platform delivery systems. However, these works do not explicitly address how a platform should make dispatch decisions under competition and partial observability. Only a small number of studies investigate courier dispatch under partial observation. These works primarily focus on data privacy-preserving mechanism design [6, 31] to facilitate multi-platform courier participation. Nevertheless, they still assume exclusive courier affiliation with a single platform or exclusive control, limiting their applicability to decentralized multi-platform settings. To address these challenges, we propose POLO, a Partially Observable multi-agent reinforcement Learning framework for multi-platform dispatch Optimization. POLO explicitly aligns the learning process with the decentralized and privacy-constrained nature of real-world instant delivery systems. Our main contributions are summarized as follows: • We identify and formalize the instant delivery dispatch problem under partial observability, highlighting the limitation of existing single-platform assumptions. • We propose a partially observable multi-agent RL approach in which each platform-grid pair learns dispatch policies solely from platform-local observations. An attention-based policy representation is introduced to capture heterogeneous courier information under incomplete observations. • To mitigate non-stationarity induced by shared couriers and concurrent actions across agents, we design a counterfactual reward shaping mechanism that isolates individual agent contributions. • We develop a high-fidelity simulator based on real-world data and conduct extensive experiments across varying system scales and numbers of platforms, demonstrating consistent improvements in platform revenue and courier travel efficiency over existing methods across varying system scales and numbers of platforms.

2

Problem Definition

In this section, we present all relevant definitions, and the notations are summarized in Table 1.

2.1

Preliminaries

2.1.1 Episode. Each episode corresponds to a continuous-time decision process within a specific time period.

Fengming Yao and Man Luo

Table 1: Notation Summary Symbol Description

Symbol Description

Time and Episode

Courier

Continuous time Discrete timestamp index Dispatch round index Global decision epoch index Total timestamps per episode Max dispatch rounds

𝑡 𝑡𝑑 𝑞 𝜏 𝑀 𝑇

Platform

𝑁𝑝 𝑝

Total number of platforms Platform index

C 𝑐𝑗 l𝑡𝑗 𝑣𝑗 d𝑡𝑗 I𝑡𝑗

Set of couriers Courier 𝑗 Courier location Courier moving speed Movement direction Courier income

Î𝑡𝑗

Courier average income

T𝑗𝑡 R 𝑡𝑗

Courier historical trajectory Planned future route of 𝑐 𝑗

𝑝,𝑡

Location and Grid l A 2D spatial location 𝑋 max Boundaries of the city 𝑁 Number of grid cells

R𝑗 Route of 𝑐 𝑗 observable by platform 𝑝 𝑑 ( R ) Total travel distance of route R |R| Number of stops in route R Payment and Revenue

N𝑐𝑡 Couriers in each cell at time 𝑡 N𝑜𝑡 Unassigned orders in each cell 𝑝,𝑡 N𝑜 Orders from platform 𝑝 Order

𝑟𝑝 Payment ratio to courier 𝑟𝑐 Compensation ratio for overdue 𝑅𝑝 Revenue of platform 𝑝 Agent and MDP

O 𝑜𝑖 𝑡𝑖,𝑐 𝑡𝑖,𝑝𝑑 𝑣𝑖 l𝑖,𝑝 l𝑖,𝑑 𝑝𝑖

𝑛 Agent (grid cell) index 𝑁𝑝𝑟 Pruning size 𝑝,𝜏 𝑜𝑛 Earliest order of agent 𝑛 at epoch 𝜏 𝑝,𝜏 C𝑛 Candidate courier set for agent 𝑛 𝑝,𝜏 S𝑛 Observation of agent 𝑛 𝑝,𝜏 𝑎𝑛 Action of agent 𝑛 𝑝,𝜏 𝑟𝑛 Reward of agent 𝑛 Training

Set of all orders in one episode Order 𝑖 Creation time of order 𝑜𝑖 Promised delivery time of 𝑜𝑖 Monetary value of order 𝑜𝑖 Pickup location of order 𝑜𝑖 Delivery location of order 𝑜𝑖 Platform of order 𝑜𝑖

𝑠𝑖𝑡 State of order 𝑜𝑖 at time 𝑡 O𝑝 Orders belonging to platform 𝑝 O 𝑝,𝜏 Unassigned orders at epoch 𝜏 Matching Features 𝜏 𝑑𝑝𝑖,𝑗 𝑝,𝜏

𝑑𝑡𝑖,𝑗

Pickup distance for (𝑜𝑖 , 𝑐 𝑗 )

𝜋𝜃𝑎 𝑉𝜃𝑐

Policy network Value network

Discount factor Learning rates for actor/critic TD advantage Reward coefficients

Functions and Operators 𝑝,𝜏

Detour for inserting 𝑜𝑖 into R 𝑗

Policy Network

𝛾 𝛼 𝑎 , 𝛼𝑐 𝑝,𝜏 𝛿𝑛 𝜌 𝑓 , 𝜌𝑑 Π𝑝 (·)

Platform-local route projection

I(·)

Indicator function

𝑣(R)

Value of on-time orders in route

2.1.2 Platform. We assume that there are a total of 𝑁𝑝 platforms, indexed by 𝑝 ∈ {1, . . . , 𝑁𝑝 }. A platform 𝑝 is responsible for receiving orders from customers and assigning them to available couriers. 2.1.3 Location and Grid. We use l to denote any location in the city, where l = (𝑥, 𝑦). The spatial boundaries of the city are defined by 𝑋 max and 𝑌max , such that 𝑥 ∈ [0, 𝑋 max ] and 𝑦 ∈ [0, 𝑌max ]. The city is partitioned into 𝑁 disjoint hexagonal grid cells. At time 𝑡, we record the number of couriers N𝑐𝑡 and the number of unassigned orders N𝑜𝑡 in each grid cell. Both N𝑐𝑡 and N𝑜𝑡 belong to Z𝑁≥0 . From the perspective of platform 𝑝, the observable unassigned order count 𝑝,𝑡 N𝑜 corresponds to the size of the subset of orders associated with Í 𝑝,𝑡 that platform. Accordingly, we have 𝑝 N𝑜 = N𝑜𝑡 . 2.1.4 Order. We use O to denote the set of orders that appear in a single episode. Each order 𝑜𝑖 ∈ O is represented as a tuple:
𝑜𝑖 = 𝑡𝑖,𝑐 , 𝑡𝑖,𝑝𝑑 , 𝑣𝑖 , l𝑖,𝑝 , l𝑖,𝑑 , 𝑝𝑖 , (1) here, 𝑡𝑖,𝑐 denotes the creation time of order 𝑜𝑖 , and 𝑡𝑖,𝑝𝑑 denotes its promised delivery time. The variable 𝑣𝑖 represents the order value. The pickup and delivery locations of the order 𝑜𝑖 are denoted by l𝑖,𝑝

Partially Observable Learning for Multi-Platform Dispatch Optimization

xxx ’26, 2026,

and l𝑖,𝑑 , respectively. The platform to which the order belongs is indicated by 𝑝𝑖 ∈ {1, . . . , 𝑁𝑝 }. Accordingly, we define O 𝑝 as the set of orders belonging to platform 𝑝, given by O 𝑝 = { 𝑜𝑖 ∈ O | 𝑝𝑖 = 𝑝 }. We assume that the status of an order 𝑜𝑖 at time 𝑡, denoted by 𝑠𝑖𝑡 , can take one of the following discrete values: 𝑠𝑖𝑡 ∈ {0, 1, 2, 3, 4, 5}, where the states correspond to released, assigned, picked up, ontime delivered, overdue delivered, and canceled, respectively. Here, we assume that an order is canceled by the customer if it is not assigned to any courier by its promised delivery time. 2.1.5 Courier. We use C to denote
the set of couriers. Each courier 𝑐 𝑗 ∈ C is initialized as 𝑐 𝑗 = l0𝑗 , 𝑣 𝑗 , where l0𝑗 denotes the initial location of courier 𝑐 𝑗 , and 𝑣 𝑗 denotes its moving speed. We flatten the location and direction features and use 𝑠 𝑡𝑗,(1) and 𝑠 𝑡𝑗,(2) to represent the scalar features and two-dimensional spatial features of courier 𝑐 𝑗 at time 𝑡, respectively, defined as
𝑠 𝑡𝑗,(1) = 𝑡, l𝑡𝑗 , d𝑡𝑗 , I𝑡𝑗 , Î𝑡𝑗 , 𝑑 (R 𝑡𝑗 ), |R 𝑡𝑗 |, I(|R 𝑡𝑗 | > 0) ∈ R10, (2)
𝑠 𝑡𝑗,(2) = T𝑗 𝑡 , R 𝑡𝑗 ∈ R𝐿𝑟 ×2 . (3) where d𝑡𝑗 ∈ R2 denotes the moving direction at time 𝑡, I𝑡𝑗 and Î𝑡𝑗 indicate the courier’s income and average income from all couriers. Moreover, I(·) denotes the indicator function, which equals 1 if the condition is true and 0 otherwise; here we use it to indicate if the courier is idle or not. T𝑗 𝑡 represents the movement trajectory of the courier 𝑐 𝑗 up to time 𝑡, and R 𝑡𝑗 denotes the planned future route if the courier is currently performing delivery tasks. 𝐿𝑟 denotes the combined length of the T𝑗 𝑡 and R 𝑡𝑗 . 𝑑 (R 𝑡𝑗 ), and |R 𝑡𝑗 | indicate the distance and stops in couriers’ routes. We assume that each courier is eligible to perform tasks from all platforms. Consequently, while the historical trajectory T𝑗 𝑡 is fully observable to all platforms through the courier’s mobile GPS signals, the future route plan R 𝑡𝑗 𝑝,𝑡

remains only partially observable. Let R 𝑗 denote the route plan observable by platform 𝑝, defined as   𝑝,𝑡 R 𝑗 ≜ Π𝑝 R 𝑡𝑗 , (4) where Π𝑝 (·) is a platform-local projection operator that extracts the route segments observable to platform 𝑝, whose definition is in Algorithm 1 Algorithm 1: Platform-local projection operator Π𝑝 (R) Input: Full route plan R = ⟨l1, l2, . . . , l | R | ⟩, platform id 𝑝 Output: Partially observable route plan R 𝑝 Initialize R ← [ ], L 𝑝 = {l𝑖,𝑝 |𝑜 𝑗 ∈ O 𝑝 } ∪ {l𝑖,𝑑 |𝑜 𝑗 ∈ O 𝑝 } ; for 𝑚 ← 1 to |R| do if l𝑚 ∈ L 𝑝 then Append l𝑚 to R 𝑝 ; return R 𝑝 ;

Figure 2: Global state → Platform’s observation → Agent’s observation. Each platform observes only its own orders and a platform-local partial route for each courier. The platform agent observes the earliest order and candidate couriers. rounds into a single global decision epoch index 𝜏 = 𝜏 (𝑡𝑑 , 𝑞), where 𝑞 denotes the 𝑞-th dispatch round at real timestamp 𝑡𝑑 . The global epoch index satisfies 𝜏 ∈ {0, 1, . . . ,𝑇 }, where 𝑇 denotes the maximum accumulated number of dispatch rounds within an episode. 2.1.7 Dispatch. At each decision epoch 𝜏, each platform aims to make dispatch decisions by assigning its currently unassigned orders to available couriers. Specifically, the set of unassigned orders for platform 𝑝 at decision epoch 𝜏 is O 𝑝,𝜏 = { 𝑜𝑖 ∈ O 𝑝 | 𝑠𝑖𝜏 = 0 }.

2.1.8 Payment, Compensation and Revenue. We assume that a fixed ratio 𝑟 𝑝 of the order value is paid to the courier as payment if the courier accepts the order. We have 𝑠𝑖𝑇 ∈ {3, 4, 5} denoted the terminal state of order 𝑜𝑖 . If an order is delivered but overdue, an additional compensation with a ratio 𝑟𝑐 is paid to the customer. Thus, the revenue of platform 𝑝 is defined as follows:   ∑︁ 𝑅𝑝 = I[𝑠𝑖𝑇 ≠ 5] (1 − 𝑟 𝑝 ) − 𝑟𝑐 I[𝑠𝑖𝑇 = 4] 𝑣𝑖 . (6) 𝑜𝑖 ∈ O 𝑝

2.1.9 Decision Objective. We aim to design a dispatch algorithm that makes decisions based on platform-local partial observations, with the goal of maximizing overall platform revenue while simultaneously minimizing the average courier travel distance.

2.2

Problem Formulation

2.2.1 Platform’s Partial Observation. Based on the above definitions, we replace the continuous time index 𝑡 with the global decision epoch index 𝜏 to represent the observed state at decision epoch 𝜏. As shown in Fig. 2, the observation of platform 𝑝 consists of four components. The first component is the courier state set 𝑝,𝜏 S 𝑗 , where each element corresponds to the platform-observable 𝑝,𝜏 state 𝑠 𝑗 of courier 𝑐 𝑗 . It is constructed by applying platform-local filtering to the states defined in Eqs. 2 and 3, as follows: 𝑝,𝜏

𝑠𝑗 2.1.6 Decision Epoch. We divide each episode into 𝑀 discrete timestamps, which is a common setting in existing literature and real-world applications [12, 27]. Accordingly, the timestamp is indexed by 𝑡𝑑 ∈ {1, 2, . . . , 𝑀 }. We further assume that at each timestamp 𝑡𝑑 , there are at most 𝑄𝑡𝑑 rounds to sequentially dispatch unassigned orders. For notational simplicity, we linearize all dispatch

(5)

𝑝,𝜏

𝑝,𝜏

= (𝑠 𝑗,(1) , 𝑠 𝑗,(2) ).

(7) 𝑝,𝜏

The second observation component, denoted by S𝑖 , represents the set of unassigned orders belonging to platform 𝑝 at decision epoch 𝜏. This component consists of |O 𝑝,𝜏 | unassigned orders. For each order 𝑜𝑖 , we have
𝑝,𝜏 𝑠𝑖 = l𝑖,𝑝 , l𝑖,𝑑 , 𝑣𝑖 , 𝑡𝑖,𝑝𝑑 − 𝑡𝑑 . (8)

xxx ’26, 2026,

Fengming Yao and Man Luo

Figure 3: Overview of the POLO framework. The entire decision horizon is divided into discrete timestamps. At each timestamp, agents perform multiple dispatch rounds to make assignment decisions. Each agent adopts an actor–critic architecture, which outputs a score for each candidate courier as well as a value estimate of the current state.
𝑝,𝜏 𝑝,𝜏 𝑝,𝜏 𝑝,𝜏 𝑝,𝜏 We further construct an order-courier matching feature S𝑖,𝑗 , Here, S 𝑗,𝑛 = S 𝑗,(1),𝑛 , S 𝑗,(2),𝑛 ⊆ S 𝑗 denotes the states of the 𝑝,𝜏 𝑝,𝜏 𝑝,𝜏 which captures features for each feasible order-courier pair. Each pruned candidate couriers, where 𝑐 𝑗 ∈ C𝑛 . The term 𝑠𝑖,𝑛 ⊆ S𝑖 𝑝,𝜏 𝑝,𝜏 𝑝,𝜏 element is defined as corresponds to the earliest order, where 𝑜𝑖 = 𝑜𝑛 . S𝑖,𝑗,𝑛 ⊆ S𝑖,𝑗 𝑝,𝜏
𝑝,𝜏 𝜏 represents all order-courier matching states between the earliest 𝑠𝑖,𝑗 = 𝑑𝑝𝑖,𝑗 , 𝑑𝑡𝑖,𝑗 , (9) 𝑝,𝜏 𝑝,𝜏 order 𝑜𝑛 and each candidate courier 𝑐 𝑗 ∈ C𝑛 . 𝜏 denotes the distance between the current location where 𝑑𝑝𝑖,𝑗 𝑝,𝜏 2.2.4 Action. Each agent takes its observation as input and outputs of courier 𝑐 𝑗 and the pickup location of order 𝑜𝑖 . The term 𝑑𝑡𝑖,𝑗 a score vector that represents the dispatch preference over candidate represents the incremental travel distance incurred by inserting couriers. Based on this score vector, an action is defined as order 𝑜𝑖 into courier 𝑐 𝑗 ’s route based on platform 𝑝’s observable 𝑝,𝜏 𝑝,𝜏 route R 𝑗 . 𝑎𝑛 ∈ {0, 1, . . . , 𝑁𝑝𝑟 − 1}. (12) The fourth state component captures the demand–supply inforThe action indicates the index of the selected courier within the mation under partial observability, defined as 𝑝,𝜏 candidate courier set C𝑛 . The selected courier is then assigned to 𝑝,𝜏 𝑝,𝜏
𝑝,𝜏 𝜏 S𝑐𝑜 = N𝑐 , N𝑜 . (10) serve the order 𝑜𝑛 . 2.2.2 Platforms’ Spatial Agents. We formulate the dispatch problem within a multi-agent reinforcement learning (MARL) framework. Specifically, we treat each grid cell of each platform as an agent, resulting in a total of 𝑁 × 𝑁𝑝 agents. Unassigned orders are spatially partitioned among agents according to their pickup locations. At multiple dispatch rounds, orders assigned to each agent are sorted according to their release times and processed sequen𝑝,𝜏 tially. Let 𝑜𝑛 denote the earliest order handled by the 𝑛-th agent with platform 𝑝 at decision epoch 𝜏. For each selected order, we prune the candidate courier set based on the pickup distance and 𝑝,𝜏 obtain a candidate courier set C𝑛 with size 𝑁𝑝𝑟 . We then define the four key components of a Markov decision process (MDP) for each agent, namely the observation, action, reward, and state transition, to model the dispatch decision-making process.

2.2.5 Reward. The challenge of reward design arises from two main challenges. First, accepting a new order alters the courier’s route, which may cause previously assigned orders to become overdue. Second, multiple agents may simultaneously influence the same courier in a single dispatch round. To address these issues, we adopt a counterfactual reward shaping to evaluate the contribution 𝑝,𝜏+1 of each agent’s dispatch decision. Let R 𝑗 denote the updated 𝑝,𝜏+1

route after one dispatch round and R 𝑗,−𝑛 denote the counterfactual route obtained by removing the dispatch action of agent 𝑛 from the joint decision. We have two functions, 𝑑 (·) and 𝑣 (·), which compute the total travel distance of a route and the total value of on-time delivered orders along the route, respectively. Based on these definitions, the reward for agent 𝑛 taking the dispatch action (𝑜𝑖 → 𝑐 𝑗 ) is computed as

𝑝,𝜏

2.2.3 Observation. We use S𝑛 to denote the observation of the 𝑛-th agent. It is obtained by filtering the states defined in Eqs. 7, 8, 9, 𝑝,𝜏 and 10 with respect to the earliest order 𝑜𝑛 , the pruned candidate 𝑝,𝜏 𝑝,𝜏 courier set C𝑛 , and the demand–supply state S𝑐𝑜 , as follows: 𝑝,𝜏 𝑝,𝜏 𝑝,𝜏 𝑝,𝜏 𝑝,𝜏
S𝑛 = S 𝑗,𝑛 , 𝑠𝑖,𝑛 , S𝑖,𝑗,𝑛 , S𝑐𝑜 . (11)

𝑝,𝜏

𝑝,𝜏+1

𝑟𝑛 = 𝑣 (R 𝑗

𝑝,𝜏

𝑝,𝜏+1

) − 𝑣 (R 𝑗 ) + 𝜌 𝑓 [𝑣 (R 𝑗 𝑝,𝜏+1

+ 𝜌𝑑 [𝑑 (R 𝑗

𝑝,𝜏

) − 𝑑 (R 𝑗 )].

𝑝,𝜏+1

) − 𝑣 (R 𝑗,−𝑛 )]

(13)

2.2.6 State Transition. After each agent executes a dispatch action, we apply the courier behavior model to determine whether the

Partially Observable Learning for Multi-Platform Dispatch Optimization

xxx ’26, 2026, 𝑝,𝜏

matching feature S𝑖,𝑗,𝑛 . Specifically, we construct two feature vec 𝑝,𝜏  𝑝,𝜏 𝑝,𝜏  𝑝,𝜏  tors S 𝑗,(1),𝑛 ∥ S̃𝑐𝑜 and S𝑖,𝑗,𝑛 ∥ 𝑠˜𝑖,𝑛 , which represent the comprehensive courier features and order-related features, respectively. These concatenated features are then processed by two separate linear encoders to produce the courier and order embeddings E𝑐 ∈ R𝑁𝑝𝑟 ×𝑑𝑐 and E𝑜 ∈ R𝑁𝑝𝑟 ×𝑑𝑜 . For the trajectory and route feature 𝑝,𝜏 S 𝑗,(2),𝑛 , we employ a gated recurrent unit (GRU) encoder to obtain its embedding representation E𝑟 ∈ R𝑁𝑝𝑟 ×𝑑𝑟 . 3.1.2 Attention-based Inter-Courier Aggregation Encoder. Rather than independently evaluating the matching score for each candidate courier, we introduce a novel attention-based mechanism into the policy network to jointly assess all candidates by explicitly considering the entire candidate courier set. Specifically, this module operates as follows. We first concatenate the encoded courier, order, and route embeddings to construct an individual embedding for each candidate courier, given by: Figure 4: Framework of the policy network. The policy network operates on partial observations of couriers, orders, courier–order matches, and supply–demand states (Eq. 11). It applies an intra-encoder to individual features and an interencoder to obtain aggregated representations, which are then fed into an MLP to produce the final scores. courier accepts the assigned order. If the courier accepts, the order is inserted into the courier’s current route using a detour-minimax insertion heuristic. The system then transitions to decision epoch 𝜏 ← 𝜏 + 1. If there remain unprocessed orders at the current timestamp, the physical time does not advance and the index 𝑡𝑑 remains unchanged. Once all orders at the current timestamp have been processed, the system advances the physical time by one decision interval, i.e., 𝑡𝑑 ← 𝑡𝑑 + 1, and constructs the next state using newly arrived orders in the subsequent interval.

E = [E𝑐 ||E𝑜 ||E𝑟 ] ∈ R𝑁𝑝𝑟 ×𝑑1 ,

(14)

where 𝑑 1 = 𝑑𝑐 + 𝑑𝑜 + 𝑑𝑟 . Then we apply a multi-head self-attention mechanism over the concatenated embeddings E to obtain an aggregated representation H as follows: H = MultiHeadAttn(E) ∈ R𝑑2 .

(15)

The attention mechanism enables the policy to model implicit interactions and competition among different couriers when making effective dispatch decisions. 3.1.3 Action Generation. We further repeat the aggregated representation H and concatenate it with the individual embeddings E to obtain [ H̃ ∥ E] ∈ R𝑁𝑝𝑟 × (𝑑1 +𝑑2 ) . The concatenated representations are then fed into an MLP to produce per-courier matching scores:   Z = MLP H̃ ∥ E ∈ R𝑁𝑝𝑟 . (16) Finally, we define the categorical distribution over actions as:

3

Methodology

Fig. 3 shows the overview of the proposed MARL framework. The entire decision horizon is divided into discrete timestamps. At each timestamp, agents perform multiple dispatch rounds to make assignment decisions. Each agent takes agent-specific, platform-local observations (as illustrated in Section 2.2.3) as input and outputs a score over candidate couriers for dispatch.

3.1

Policy Network

The policy network consists of three main components. First, an intra-courier encoder is used to encode the individual features of each candidate courier. Next, an inter-courier encoder aggregates information across the entire candidate courier set to selectively capture interactions and relative comparisons among candidates. Finally, a multilayer perceptron (MLP) is applied to produce the final output with the desired dimensionality. 3.1.1 Intra-Courier Encoder. The policy network takes each agent’s 𝑝,𝜏 𝑝,𝜏 observation S𝑛 (Eq. 11) as input and outputs an action index 𝑎𝑛 . 𝑝,𝜏 As illustrated in Fig. 4, the supply–demand feature S𝑐𝑜 and the 𝑝,𝜏 order feature 𝑠𝑖,𝑛 are repeated 𝑁𝑝𝑟 times and concatenated with 𝑝,𝜏 the candidate couriers’ first feature S 𝑗,(1),𝑛 and the courier–order

𝜋𝜃𝑎 (𝑎 | S) = Softmax(Z).

(17)

Actions are sampled from this distribution as: 𝑝,𝜏 𝑝,𝜏
𝑎𝑛 ∼ 𝜋𝜃𝑎 · | S𝑛 .

(18)

3.2

Training

We train the policy network with parameters 𝜃 𝑎 using an actor-critic framework. The critic network, parameterized by 𝜃𝑐 , takes the same 𝑝,𝜏 input S𝑛 as the policy network and is used to estimate the state 𝑝,𝜏 value function 𝑉𝜃𝑐 (S𝑛 ), which reflects the expected discounted long-term revenue. The value function for platform 𝑝 at decision epoch 𝜏 is defined as: "𝑇 −𝜏 # ∑︁ 𝑝,𝜏
ℓ 𝑝,𝜏+ℓ 𝑉𝜃𝑐 S𝑛 = E 𝛾 𝑟𝑛 , (19) ℓ=0

where 𝛾 ∈ (0, 1) is the discount factor. During training, we store transitions from each agent of each platform in a replay buffer D 𝑝,𝜏 𝑝,𝜏 𝑝,𝜏+1 𝑝,𝜏 in the form (S𝑛 , 𝑎𝑛 , S𝑛 , 𝑟𝑛 ). At each update step, a minibatch B of transitions is uniformly sampled from D. We define the temporal-difference (TD) advantage for each transition as:   𝑝,𝜏 𝑝,𝜏 𝑝,𝜏+1 𝑝,𝜏
𝛿𝑛 = 𝑟𝑛 + 𝛾𝑉𝜃𝑐 S𝑛 − 𝑉𝜃𝑐 S𝑛 . (20)

xxx ’26, 2026,

Fengming Yao and Man Luo

1e4

0.5

Base Median Large

Latitude

Orders

1.0

0.0 00:00 04:00 08:00 12:00 16:00 20:00

46.0 45.9 45.8 174.4

Time

Base Median Large

174.6

174.8

Longitude

175.0 174.4

Pickup Delivery

174.6

174.8

175.0

Longitude

Figure 5: Spatio-temporal distribution of generated orders shows two daily peaks. The delivery area is more spatially extensive than the pickup area. Table 2: Statistics of the instances across the three scales. Scale Base Median Large

Orders 86 201 1418

Couriers 26 52 329

Time span 17:00-18:00 17:00-18:00 17:30-18:00

Hex size 0.7 km 0.7 km 0.7 km

Grids 𝑁 16 20 120

The critic parameters are updated by minimizing the TD error over the mini-batch: ∑︁ 𝑝,𝜏 𝑝,𝜏
𝜃 𝑐 ← 𝜃 𝑐 + 𝛼𝑐 𝛿𝑛 ∇𝜃𝑐 𝑉𝜃𝑐 S𝑛 , (21) 𝑝,𝜏

( S𝑛 ,·) ∈ B

here 𝛼𝑐 is the learning rate of the critic. Similarly, the policy parameters are updated with the learning rate 𝛼𝑎 using the policy gradient: ∑︁ 𝑝,𝜏 𝑝,𝜏 𝑝,𝜏
𝜃 𝑎 ← 𝜃 𝑎 + 𝛼𝑎 𝛿𝑛 ∇𝜃𝑎 log 𝜋𝜃𝑎 𝑎𝑛 | S𝑛 . (22)

Algorithm 2: POLO Training Input: Order set O, courier set C, platforms number 𝑁𝑝 Output: Learned policy 𝜋𝜃𝑎 Initialize policy network 𝜋𝜃𝑎 and value network 𝑉𝜃𝑐 ; Initialize replay buffer D; for episode = 1 to maxEpisode do Reset simulator with randomly generated instance; for decision epoch 𝜏 = 0 to 𝑇 do for each platform 𝑝 in parallel do Partition O 𝑝,𝜏 to agents by pickup grid cell; 𝑝,𝜏 for each agent 𝑛 with order 𝑜𝑛 do 𝑝,𝜏 1. Prune courier pool to obtain C𝑛 ; 𝑝,𝜏 2. Construct S𝑛 according to Eq. 11; 𝑝,𝜏 𝑝,𝜏 3. Sample 𝑎𝑛 ∼ 𝜋𝜃𝑎 (· | S𝑛 ); 𝑝,𝜏 4. Assign 𝑜𝑛 to courier 𝑐 𝑎𝑝,𝜏 ; 𝑛

Execute actions and advance to 𝜏 + 1; for each agent 𝑛 of platform 𝑝 do 𝑝,𝜏 1. Compute reward 𝑟𝑛 according to Eq. 13; 𝑝,𝜏 𝑝,𝜏 𝑝,𝜏 𝑝,𝜏+1 2. Store (S𝑛 , 𝑎𝑛 , 𝑟𝑛 , S𝑛 ) in D;

𝑝,𝜏

( S𝑛 ,·) ∈ B

4 Evaluation 4.1 Simulator We construct a simulator based on a real-world dataset collected from a major food delivery platform in China, Meituan1 . The dataset contains detailed order information generated over eight consecutive days. To simulate the demand pattern, in each episode, the simulator randomly samples orders from the same time interval across the eight days to ensure real pickup and delivery location distribution. In the multi-platform scenario, sampled orders are further randomly assigned to different platforms. We divide the spatially separated regions into three instance scales, namely Base, Median, and Large. We visualize the spatio-temporal distribution of the generated orders in Fig. 5. The order demand exhibits two pronounced peaks during daytime hours. Spatially, pickup locations are more concentrated in the city center, while delivery locations are more dispersed toward the surrounding areas. Couriers are initialized using their first observed locations of the day in the real data. Considering the flexibility of courier working schedules, we randomly activate 𝑟 𝑎 % couriers, which is consistent with empirical findings reported in [26], where approximately 67% of couriers perform the majority of instant delivery tasks. We also adopt the courier behavior model proposed in [26]. In this model, a courier’s order rejection probability depends on the number of 1 https://www.meituan.com

for update times = 1 to maxUpdate do 1. Sample mini-batch B from D; 2. Compute TD advantage according to Eq. 20; 3. Update 𝜃𝑐 according to Eq. 21; 4. Update 𝜃 𝑎 according to Eq. 22;

on-hand orders, the pickup distance, and the remaining time of the most urgent order in the courier’s route.

4.2

Experiment Setting

For the simulator, the decision interval is set to 2 minutes. The compensation ratio is set to 𝑟𝑐 = 1, and the payment ratio is 𝑟 𝑝 = 0.2. We randomly activate 𝑟 𝑎 = 50% couriers in each episode for the base- and median-scale settings and 𝑟 𝑎 = 10% for the large-scale setting. The courier travel speed is set to 𝑣𝑐 = 0.003 𝑘𝑚/𝑠, and the size of the candidate courier set is 𝑁𝑝𝑟 = 10. For the baseand median-scale settings, each episode spans from 17:00 to 18:00, while for the large-scale setting, each episode spans from 17:30 to 18:00. For training, the replay buffer size is set to 100,000, and the batch size is 256. The policy network and value network are updated 10 times after each episode, and the models are trained

Partially Observable Learning for Multi-Platform Dispatch Optimization

xxx ’26, 2026,

Table 3: Overall performance across the three instance scales under different platform counts. Higher GMV and lower ACTD are better. Dark red bold and underline denote the best and second-best results in each column. POLO is lightly shaded, and the last row in the GMV block reports POLO’s relative gain over the second-best method.

Metric

Method 𝑁𝑝 = 1 Random 158.58±14.21 PRandom 164.04±14.72 DetGreedy 133.69±12.35 DisGreedy 164.36±14.72 GMV (↑) IBM 176.51±17.13 TCAC 220.94±7.65 FAIR 218.77±8.20 POLO 219.56±8.56 POLO Gain (%) −0.62% Random 6.28±0.37 PRandom 5.17±0.29 DetGreedy 6.33±0.30 DisGreedy 4.23±0.30 ACTD (↓) IBM 3.82±0.21 TCAC 5.39±0.27 FAIR 5.42±0.29 POLO 5.25±0.21

Base 𝑁𝑝 = 2 160.09±12.29 166.28±12.98 123.30±12.64 164.86±14.83 159.81±16.18 207.47±7.98 200.35±10.70 202.81±9.34 −2.24% 6.37±0.34 5.19±0.24 5.81±0.29 4.24±0.27 4.03±0.25 5.40±0.27 5.46±0.28 5.35±0.31

𝑁𝑝 = 3 157.37±14.51 166.69±11.62 122.04±12.18 163.35±14.94 159.83±20.93 194.42±10.61 178.36±12.36 183.82±10.28 −5.45% 6.37±0.33 5.19±0.28 5.68±0.29 4.23±0.27 4.12±0.26 5.38±0.28 5.35±0.30 5.29±0.29

6.1

6.3

9.4

4.5

28.7

4.3

TCAC-ACTD

0.2

-0.4

2.1

0.3

17.0

6.0

FAIR-GMV

8.4

11.0

3.5

8.0

0.3

7.4

15

FAIR-ACTD

0.7

-2.0

-4.4

-1.6

0.0

-0.2

10 5

7.6

9.4

6.7

5.4

22.2

11.5

POLO-ACTD

1.9

-1.1

3.8

-2.1

20.3

4.7

Median Median

Large

Large

Base

Np = 1

Base

25

6

20

Relative change (%)

POLO-GMV

0

2 Np = 2 3 Np = 1 2 Np = 2 3 Np = 1 2 Np = 2 3

Figure 6: Relative performance change as the number of platforms increases.

for a total of 1,000 episodes. All agents share network parameters, and both networks are optimized using the Adam optimizer with a learning rate of 0.0002. The discount factor is set to 𝛾 = 0.97. The entire region is partitioned into hexagonal grids with a side length of 0.7 km, and the attention module employs 4 attention heads. Under this setting, we summarize the dataset statistics and the total number of grids in Table 2.

4.3

Median 𝑁𝑝 = 2 353.49±25.21 374.96±27.18 299.80±24.61 361.33±30.83 374.64±30.07 464.48±21.15 477.54±20.41 481.99±20.68 +0.93% 8.10±0.29 6.04±0.26 6.72±0.28 5.10±0.23 4.72±0.18 6.18±0.21 6.32±0.23 6.28±0.28

ACTD

TCAC-GMV

𝑁𝑝 = 1 346.92±21.22 374.64±22.28 326.65±23.04 359.58±25.24 431.68±31.20 512.56±18.41 494.84±20.72 516.43±18.13 +0.76% 8.18±0.28 6.02±0.21 7.51±0.25 5.10±0.26 4.47±0.17 6.05±0.18 6.61±0.23 6.05±0.19

Baseline and Metrics

We compare our model with the following baselines: • Random: Orders are dispatched randomly to available couriers. • PRandom: Orders are dispatched randomly to the 𝑁 pr closest available couriers. This baseline mimics a broadcast-based dispatch mechanism, in which couriers compete to accept orders and the fastest responder is assigned the order. • DisGreedy: Orders are assigned to the closest available courier. • DetGreedy: Orders are assigned to the courier that incurs the minimum additional travel distance.

5 4

𝑁𝑝 = 3 342.66±23.83 373.69±25.55 285.16±25.92 363.88±27.01 367.63±32.11 443.60±20.49 439.18±20.62 455.92±18.14 +2.78% 8.15±0.25 6.05±0.25 6.50±0.27 5.09±0.26 4.85±0.21 6.20±0.22 6.22±0.23 6.15±0.25

better

𝑁𝑝 = 1 −261.56±73.61 2119.66±94.96 1379.78±78.93 2207.85±90.19 3847.10±121.19 4216.44±113.94 2973.70±102.78 4229.25±135.20 +0.30% 20.79±0.31 10.04±0.16 13.59±0.20 8.68±0.16 6.33±0.11 8.23±0.13 10.28±0.18 8.08±0.11

8 7 6 5

Large 𝑁𝑝 = 2 −266.88±58.22 2122.54±90.53 1206.06±77.10 2194.45±94.41 2848.89±100.91 3006.11±98.19 2964.76±92.24 3290.76±100.20 +9.47% 20.86±0.24 10.03±0.18 11.98±0.20 8.69±0.17 7.33±0.15 9.63±0.17 10.28±0.16 9.72±0.17

𝑁𝑝 = 3 −296.00±55.69 2119.99±95.74 1171.64±75.75 2212.10±94.15 2582.17±115.46 2875.70±107.49 2745.94±97.85 2911.20±107.14 +1.23% 20.88±0.29 10.04±0.20 11.42±0.23 8.71±0.18 7.73±0.17 10.21±0.19 10.26±0.19 10.18±0.16

20 15

better

10

better

150 200 300 400 500 0 2000 (a) GMV on Base Scale (b) GMV on Median Scale (c) GMV on Large Scale Random PRandom

DetGreedy DisGreedy

IBM TCAC

FAIR POLO

Figure 7: Trade-off analysis between key metrics.

• IBM [34]: Orders are iteratively assigned using bipartite graph matching to maximize the total matching weight at each decision epoch, where the weight of each order–courier pair is defined based on the order value and the corresponding detour distance. • TCAC [28]: A learning-based concurrent dispatch method that focuses on time constraints. • FAIR [12]: A learning-based concurrent dispatch method that focuses on reducing income disparities among couriers. All baseline methods focus on dispatch decision-making, while the minimax detour strategy is uniformly adopted by the simulator for route planning. The two learning-based methods, TCAC and FAIR, use the same state construction and training settings as POLO to ensure a fair comparison. We evaluate the performance of different dispatch methods using three metrics from the global perspective: GMV (Gross Merchandise Volume) and ACTD (Average Courier Travel Distance). The detailed definitions of these metrics are as follows: Í 𝑁𝑝 𝑇 ∑︁ 𝑐 𝑗 ∈ C 𝑑 (T𝑗 ) 𝐺𝑀𝑉 = 𝑅 𝑝 , 𝐴𝐶𝑇 𝐷 = |C| 𝑝=1

xxx ’26, 2026,

4.4

Overall Performance

Table 3 presents the experimental results under different problem scales and numbers of platforms. We summarize the following findings based on the results. (1) As the number of platforms increases, all learning-based methods exhibit performance degradation, reflected by reduced GMV, increased ACTD, or both. Moreover, this degradation becomes more significant in large-scale scenarios. For example, when applying the TCAC method in the base-scale setting, increasing the number of platforms from 1 to 2 reduces the GMV from 220.94 to 207.47, corresponding to only a 6.1% decrease. In contrast, under the large-scale setting, the GMV decreases from 4216.44 to 3006.11, which exceeds a 28.7% reduction. This finding indicates that partially observable environments can substantially affect decision-making performance in real-world applications, especially in large-scale multi-platform settings. Among these methods, POLO achieves superior GMV performance in most experimental settings. Although POLO obtains the second-best GMV in the base-scale scenario, its advantage becomes much more significant in large-scale scenarios. For example, under the large-scale setting with two platforms, POLO achieves approximately a 9.47% improvement in GMV compared to TCAC. (2) On the other hand, heuristic methods such as IBM and DisGreedy generally achieve lower ACTD values, as they assign orders primarily based on minimum detour distance without considering order value or delivery urgency. While this minimizes courier travel, it sacrifices revenue by ignoring high-value assignments. POLO’s reward explicitly balances order value against travel cost through the distance penalty coefficient 𝜌𝑑 , such that the higher ACTD reflects a deliberate design choice prioritizing revenue over travel minimization, rather than a routing inefficiency. Although POLO does not achieve the lowest ACTD compared with heuristic methods, it consistently attains higher GMV and lower ACTD than the other learning-based baselines. To further illustrate the trade-off between these two metrics, Figure 7 presents scatter visualizations under the two-platform large-scale setting. Ideally, methods closer to the bottom-right corner indicate better performance, corresponding to higher GMV and lower ACTD. The results show that POLO consistently occupies a more favorable position across all scales, with particularly clear advantages in large-scale scenarios.

4.5

Performance under Distribution Shifts

In real-world applications, different platforms may exhibit different preferences regarding delivery time, pricing strategies, and service quality. To evaluate the robustness of our method under distribution shift scenarios, we further report experimental results under both spatial and temporal distribution shifts. Specifically, for the spatial distribution shift, orders generated from different geographic areas are more likely to belong to different platforms. For the temporal distribution shift, we simulate heterogeneous platform preferences by assigning one platform a lower delivery price but a longer promised delivery time (i.e., lower 𝑣𝑖 and higher 𝑡𝑖,𝑝𝑑 ). The corresponding results are presented in Table 4. We observe that our model achieves the best GMV performance in most distribution shift scenarios, demonstrating its strong adaptability and robustness under realistic multi-platform environments.

Fengming Yao and Man Luo

Table 4: Sensitivity results under temporal and spatial distribution shifts with 𝑁𝑝 = 2. Temporal diff Spatial diff Scale Method GMV (↑) ACTD (↓) GMV (↑) ACTD (↓) Random 152.60±9.37 6.37±0.37 160.07±14.99 6.35±0.35 PRandom 156.28±10.65 5.24±0.31 166.22±12.52 5.18±0.30 DetGreedy 124.91±12.38 5.77±0.31 121.44±12.28 5.71±0.27 DisGreedy 152.72±13.29 4.26±0.25 166.15±15.56 4.29±0.24 Base IBM 154.12±15.51 4.08±0.27 162.76±17.28 4.00±0.23 TCAC 173.03±8.78 5.24±0.28 209.15±9.31 5.39±0.30 192.71±9.30 5.50±0.26 FAIR 175.26±7.70 5.38±0.30 POLO 177.44±8.44 5.30±0.27 196.52±13.05 5.26±0.31 Random 158.50±57.41 20.82±0.26 −266.06±67.52 20.82±0.27 PRandom 2239.22±82.25 10.05±0.20 2106.82±98.88 10.06±0.17 DetGreedy 1365.81±78.59 11.99±0.23 1195.34±74.78 11.96±0.20 DisGreedy 2261.62±93.19 8.64±0.18 2216.57±97.18 8.66±0.18 Large IBM 2884.94±110.70 7.31±0.14 2986.11±139.91 7.18±0.16 TCAC 3180.27±89.37 9.89±0.16 3154.97±109.86 9.91±0.15 FAIR 2927.71±93.80 10.29±0.18 2928.44±93.63 10.28±0.19 POLO 3258.55±77.20 9.73±0.19 3384.15±119.68 9.38±0.16

Table 5: Attention ablation study with 𝑁𝑝 = 2. Base Large GMV (↑) ACTD (↓) Time (↓) GMV (↑) ACTD (↓) Time (↓) POLO 202.81±9.34 5.35±0.31 0.1601 3290.76±100.20 9.72±0.17 4.6703 w/o Att 199.22±10.68 5.30±0.31 0.1782 3178.27±96.43 9.86±0.17 4.2383 Variant

4.6

Ablation Study

Table 5 and Table 6 further analyze the effectiveness of two critical design components in POLO. From Table 5, where the variant without the attention module is denoted as w/o Att, we observe that the attention-based inter-courier aggregation mechanism, although introducing additional computational overhead in large-scale scenarios, effectively improves performance on both GMV and ACTD. Moreover, the increase in inference time remains acceptable in practice. On the other hand, Table 6 evaluates the contributions of different components in the counterfactual reward design. The results indicate that both the counterfactual term and the distance penalty term positively contribute to the overall performance. Furthermore, similar to previous observations, the performance improvements become more significant under large-scale settings, highlighting the potential of POLO in real-world applications.

4.7

Sensitivity Analysis

Table 7 presents the performance of learning-based methods on the large-scale scenario with two platforms under different hexagon sizes. We observe that, although the hexagon size needs to be manually specified, POLO consistently achieves strong performance across different settings, demonstrating its robustness to changes in spatial partition granularity. Figure 8 presents the performance under different pruning numbers 𝑁𝑝𝑟 and courier appearance ratios 𝑟 𝑎 . Overall, POLO demonstrates robust performance across different parameter settings. The first parameter, the courier appearance

Partially Observable Learning for Multi-Platform Dispatch Optimization

xxx ’26, 2026,

Table 6: Effect of zeroing each reward coefficient in POLO. We compare the zero setting against the reference non-zero setting used in the main configuration (0.6 for counterfactual ratio and 1.0 for distance penalty ratio). We define ΔGMV = GMVref − GMV0 and ΔACTD = ACTD0 − ACTDref , so positive deltas indicate benefit from using the non-zero coefficient. Positive deltas are highlighted in dark red. Coeff. 𝜌𝑓 𝜌𝑑

Base (𝑁𝑝 = 2) Large (𝑁𝑝 = 2) 0 Ref. GMV@0 GMV@ref ACTD@0 ACTD@ref ΔGMV ΔACTD 0 Ref. GMV@0 GMV@ref ACTD@0 ACTD@ref ΔGMV ΔACTD 0.0 0.6 196.79±11.27 202.81±9.34 5.21±0.28 5.35±0.31 +6.02 −0.14 0.0 0.6 3207.47±110.31 3290.76±100.20 9.88±0.16 9.72±0.17 +83.29 +0.16 0.0 1.0 193.08±12.61 202.81±9.34 5.34±0.28 5.35±0.31 +9.73 −0.01 0.0 1.0 2974.51±111.94 3290.76±100.20 10.10±0.20 9.72±0.17 +316.25 +0.38 25

200

15.0

4000

150

12.5

3000

20

2000

15

1000

10

10.0

100

7.5

50 0 0.1

5.0 0.2 0.3 0.4 0.5 ra (a) GMV on Base Scale

200

0.1

0.2 0.3 0.4 0.5 ra (b) ACTD on Base Scale

5.8 5.6

150

5.4

100

5.2 5.0

50 04

2.5

4.8 7 10 13 16 Npr (e) GMV on Base Scale

4

7 10 13 Npr (f) ACTD on Base Scale TCAC

0 0.04 0.07 0.10 0.13 0.16 ra (c) GMV on Large Scale

3000 2500 2000 1500 1000 500 16 0 4 FAIR

50.04 0.07 0.10 0.13 0.16 ra (d) ACTD on Large Scale 11.0 10.5 10.0 9.5

7 10 13 16 Npr (g) GMV on Large Scale

9.0

POLO

4

7 10 13 16 Npr (h) ACTD on Large Scale

Figure 8: Sensitivity analysis under different pruning numbers 𝑁𝑝𝑟 and courier appearance ratios 𝑟 𝑎 . POLO achieves higher GMV and lower ACTD in most parameter settings. ratio 𝑟 𝑎 , reflects different workload levels in the system. Specifically, it simulates scenarios ranging from limited courier availability to abundant courier resources. As expected, increasing the number of available couriers generally leads to higher GMV and lower ACTD. Notably, even under scenarios with limited courier availability, POLO is still able to achieve relatively high GMV, indicating that the proposed method can identify effective dispatch strategies even in challenging environments. The second parameter, the pruning number 𝑁𝑝𝑟 , controls the number of candidate couriers considered for each order. The results show that, for POLO, the optimal pruning number is around 10 in terms of GMV performance. When 𝑁𝑝𝑟 is too small, the candidate set becomes overly restricted, potentially excluding high-quality assignment choices. In contrast, an excessively large 𝑁𝑝𝑟 increases the difficulty of decision-making and may introduce unnecessary noise into the optimization process.

4.8

Inference Efficiency

We report the inference time of all learning-based methods in Table 8, measured as the average end-to-end seconds per test instance, including feature extraction, decision-making, and simulator stepping. We find that POLO achieves the lowest inference time among all methods. This is mainly because TCAC and FAIR make decisions sequentially for each order, while POLO performs parallel order assignment across multiple grids, resulting in faster inference.

5

Related Work

Order Dispatch in On-Demand Services. Order dispatch is a fundamental problem in on-demand service systems and has been extensively studied over the past decades. Early approaches primarily relied on rule-based or heuristic strategies, such as greedy matching, collaborative filtering, or queue-based assignment [13, 22, 34]. While these methods are easy to deploy, they are often sensitive to parameter tuning and struggle to adapt to highly dynamic environments, where orders arrive continuously and supply-demand conditions evolve rapidly over time [4]. To improve adaptability, recent studies have increasingly explored learning-based methods for dispatch optimization. One line of work applies RL to optimize decision intervals or caching strategies, effectively decomposing the dynamic dispatch problem into a sequence of static optimization problems [14, 15, 29]. Another line of research directly focuses on learning dispatch policies through RL, enabling end-to-end dispatch decision-making based on system states [17, 18, 27]. Several studies combine these two directions; for example, Ma et al. [18] propose a hierarchical RL framework to jointly optimize dispatching and routing decisions in real-world supply chain systems. With the rise of instant delivery services, the concurrent order dispatch problem has been introduced to explicitly model overlapping delivery tasks [10]. To address the increased complexity, subsequent work reformulated the RL modeling paradigm by shifting the agent perspective

xxx ’26, 2026,

Fengming Yao and Man Luo

Table 7: Hex-size sensitivity on the large instance. Method TCAC FAIR POLO

0.4 GMV (↑) ACTD (↓) 3077.69±133.90 9.45±0.15 2928.82±102.35 10.31±0.17 3359.06±110.00 9.61±0.15

0.7 GMV (↑) ACTD (↓) 3006.11±98.19 9.63±0.17 2964.76±92.24 10.28±0.16 3290.76±100.20 9.72±0.17

Table 8: Inference time (seconds). Method TCAC FAIR POLO

𝑁𝑝 = 1 4.0113 7.3350 3.9215

𝑁𝑝 = 2 7.6692 9.0552 5.1377

𝑁𝑝 = 3 9.1673 8.5202 3.6518

from service suppliers to service demanders and parameterizing heterogeneous couriers within the action space [35]. Under this framework, Wang et al. [28] propose an actor-critic approach for food delivery dispatch under strict time constraints, while Jiang et al. [12] further propose a competitive multi-agent framework to address income disparity among couriers. Despite their effectiveness in single-platform settings, most existing dispatch methods rely on strong assumptions, including full observability of couriers’ states and exclusive platform control over dispatch decisions. These assumptions limit their applicability in realistic multi-platform environments, where couriers are shared across platforms and system states are only partially observable. Crowdsourced Couriers and Multi-Platform Settings. Crowdsourced couriers and the coexistence of multiple platforms have become increasingly common in modern on-demand service markets [1, 21], motivating growing research interest in the challenges arising from such settings. A substantial body of work focuses on modeling crowdsourced worker behavior, including working-time patterns [19], task acceptance decisions [3],and incentive modeling [34]. These studies provide important insights into individual courier behavior under dynamic and uncertain conditions. However, relatively little work explicitly studies order dispatch in multiplatform settings. A small number of studies [6, 7, 31] investigate dispatch under multiple platforms with partial observation, typically distinguish between local workers and cooperative workers, and focus on privacy-preserving information sharing mechanisms. While these approaches relax some observability assumptions, they generally continue to assume exclusive courier affiliation or centralized control over dispatch decisions. In contrast, real-world instant delivery systems are characterized by decentralized decision-making and shared couriers, where each platform operates with platform-local information and without access to competitors’ states. POLO directly addresses this setting by formulating multi-platform dispatch as a partially observable multi-agent learning problem, without requiring cross-platform coordination or information sharing. Partially Observable MARL and Scalable Architectures. Multiagent reinforcement learning (MARL) has been widely adopted in delivery and logistics optimization due to its ability to model the sequential, distributed decision-making inherent in large-scale dispatch systems [36]. A fundamental characteristic of MARL is

1.0 1.3 GMV (↑) ACTD (↓) GMV (↑) ACTD (↓) 3151.85±115.33 9.99±0.17 3138.91±109.00 9.98±0.17 2924.84±111.93 10.27±0.16 2930.30±115.74 10.27±0.19 3419.36±108.67 9.51±0.17 3360.17±102.54 9.58±0.18

that each agent operates under partial observability, receiving only a local observation rather than the full global state. Existing work predominantly addresses spatial partial observability, where each agent’s observation is limited to its local geographic region or neighborhood [36]. Such local observations introduce non-stationarity during training, as each agent’s environment appears non-stationary from its local perspective when other agents simultaneously update their policies [11]. To address this, the Centralized Training with Decentralized Execution (CTDE) paradigm [9, 16, 20, 33] has emerged as a principled framework for partially observable MARL: a centralized critic with access to global state is used during training, while each agent executes its policy using only local observations. However, these methods primarily treat partial observability as a spatial phenomenon and assume that a shared global state is accessible during centralized training. In our work, we identify and formalize a more restrictive form of partial observability that arises in multi-platform instant delivery: beyond spatial locality, each platform agent is subject to a platform-specific information mask, whereby it can only observe couriers and orders belonging to its own platform. This privacy-induced constraint applies equally during training, as aggregating per-platform states would require inter-platform coordination that is unavailable in practice, making the standard CTDE assumption inapplicable in this setting.

6

Conclusion

In this paper, we studied the order dispatch problem in realistic multi-platform instant delivery systems, where couriers are shared across platforms and each platform operates under partial observability. We propose POLO, a partially observable multi-agent reinforcement learning framework that enables decentralized dispatch using only platform-local observations. POLO combines an attention-based policy network to handle heterogeneous courier information with a counterfactual reward shaping mechanism to mitigate non-stationarity induced by joint actions across agents. Through extensive experiments, we demonstrated that POLO consistently improves platform revenue and reduces courier travel distance across varying system scales and numbers of platforms. This work provides a principled learning framework for multiplatform dispatch under partial observability and offers insights into the design of scalable and realistic decision-making systems for urban instant delivery.

GenAI Disclosure Statement Generative AI tools were used solely for language refinement, grammatical correction, and figure-generation code assistance in this work.

Partially Observable Learning for Multi-Platform Dispatch Optimization

References [1] Aliaa Alnaggar, Fatma Gzara, and James H Bookbinder. 2021. Crowdsourced delivery: A review of platforms and academic literature. Omega 98 (2021), 102139. [2] Aliaa Alnaggar, Fatma Gzara, and James H Bookbinder. 2024. Compensation guarantees in crowdsourced delivery: Impact on platform and driver welfare. Omega 122 (2024), 102965. [3] Rosemonde Ausseil, Marlin W Ulmer, and Jennifer A Pazour. 2024. Online acceptance probability approximation in peer-to-peer transportation. Omega 123 (2024), 102993. [4] Gerardo Berbeglia, Jean-François Cordeau, and Gilbert Laporte. 2010. Dynamic pickup and delivery problems. European journal of operational research 202, 1 (2010), 8–15. [5] Carmelo Cennamo. 2021. Competing in digital markets: A platform-based perspective. Academy of Management Perspectives 35, 2 (2021), 265–291. [6] Yurong Cheng, Boyang Li, Xiangmin Zhou, Ye Yuan, Guoren Wang, and Lei Chen. 2020. Real-time cross online matching in spatial crowdsourcing. In 2020 IEEE 36th international conference on data engineering (ICDE). IEEE, 1–12. [7] Yurong Cheng, Zhaohe Liao, Xiaosong Huang, Yi Yang, Xiangmin Zhou, Ye Yuan, and Guoren Wang. 2024. Cross Online Ride-Sharing for Multiple-Platform Cooperations in Spatial Crowdsourcing. In 2024 IEEE 40th International Conference on Data Engineering (ICDE). IEEE, 4140–4152. [8] Laetitia Dablanc, Eleonora Morganti, Niklas Arvidsson, Johan Woxenius, Michael Browne, and Neila Saidi. 2017. The rise of on-demand ‘Instant Deliveries’ in European cities. In Supply Chain Forum: An International Journal, Vol. 18. Taylor & Francis, 203–217. [9] Jakob Foerster, Gregory Farquhar, Triantafyllos Afouras, Nantas Nardelli, and Shimon Whiteson. 2018. Counterfactual multi-agent policy gradients. In Proceedings of the AAAI conference on artificial intelligence, Vol. 32. [10] Baoshen Guo, Shuai Wang, Yi Ding, Guang Wang, Suining He, Desheng Zhang, and Tian He. 2021. Concurrent order dispatch for instant delivery with timeconstrained actor-critic reinforcement learning. In 2021 IEEE Real-Time Systems Symposium (RTSS). IEEE, 176–187. [11] Pablo Hernandez-Leal, Bilal Kartal, and Matthew E Taylor. 2019. A survey and critique of multiagent deep reinforcement learning. Autonomous Agents and Multi-Agent Systems 33, 6 (2019), 750–797. [12] Lin Jiang, Shuai Wang, Baoshen Guo, Hai Wang, Desheng Zhang, and Guang Wang. 2023. FairCod: A fairness-aware concurrent dispatch system for large-scale instant delivery services. In Proceedings of the 29th ACM SIGKDD Conference on knowledge discovery and data mining. 4229–4238. [13] Der-Horng Lee, Hao Wang, Ruey Long Cheu, and Siew Hoon Teo. 2004. Taxi dispatch system based on current demands and real-time traffic conditions. Transportation Research Record 1882, 1 (2004), 193–200. [14] Minne Li, Zhiwei Qin, Yan Jiao, Yaodong Yang, Jun Wang, Chenxi Wang, Guobin Wu, and Jieping Ye. 2019. Efficient ridesharing order dispatching with mean field multi-agent reinforcement learning. In The world wide web conference. 983–994. [15] Yafei Li, Qingshun Wu, Xin Huang, Jianliang Xu, Wanru Gao, and Mingliang Xu. 2022. Efficient adaptive matching for real-time city express delivery. IEEE Transactions on Knowledge and Data Engineering 35, 6 (2022), 5767–5779. [16] Ryan Lowe, Yi I Wu, Aviv Tamar, Jean Harb, OpenAI Pieter Abbeel, and Igor Mordatch. 2017. Multi-agent actor-critic for mixed cooperative-competitive environments. Advances in neural information processing systems 30 (2017). [17] Man Luo, Bowen Du, Wenzhe Zhang, Tianyou Song, Kun Li, Hongming Zhu, Mark Birkin, and Hongkai Wen. 2023. Fleet rebalancing for expanding shared e-mobility systems: A multi-agent deep reinforcement learning approach. IEEE Transactions on Intelligent Transportation Systems 24, 4 (2023), 3868–3881. [18] Yi Ma, Xiaotian Hao, Jianye Hao, Jiawen Lu, Xing Liu, Tong Xialiang, Mingxuan Yuan, Zhigang Li, Jie Tang, and Zhaopeng Meng. 2021. A hierarchical reinforcement learning based optimization framework for large-scale dynamic pickup and delivery problems. Advances in neural information processing systems 34 (2021), 23609–23620. [19] Seyed Sina Mohri, Neema Nassir, Russell G Thompson, Patricia Sauri Lavieri, and Hadi Ghaderi. 2025. Crowd-shipping systems with public transport passengers:

xxx ’26, 2026,

Operational planning. Transportation Research Part E: Logistics and Transportation Review 194 (2025), 103916. [20] Tabish Rashid, Mikayel Samvelyan, Christian Schroeder De Witt, Gregory Farquhar, Jakob Foerster, and Shimon Whiteson. 2020. Monotonic value function factorisation for deep multi-agent reinforcement learning. Journal of Machine Learning Research 21, 178 (2020), 1–51. [21] Martin WP Savelsbergh and Marlin W Ulmer. 2022. Challenges and opportunities in crowdsourced delivery planning and operations. 4OR 20, 1 (2022), 1–21. [22] Kiam Tian Seow, Nam Hai Dang, and Der-Horng Lee. 2009. A collaborative multiagent taxi-dispatch system. IEEE Transactions on Automation science and engineering 7, 3 (2009), 607–616. [23] Jie Shi, Yuanqi Gao, Wei Wang, Nanpeng Yu, and Petros A Ioannou. 2019. Operating electric vehicle fleet for ride-hailing services with reinforcement learning. IEEE Transactions on Intelligent Transportation Systems 21, 11 (2019), 4822–4834. [24] Meirong Tan, Hao Li, Hongwei Wang, and Pei Yin. 2025. Research on e-commerce platform’s instant retail strategy considering delivery timeliness under consumer channel preferences. Asia Pacific Journal of Marketing and Logistics (2025). [25] Xiaocheng Tang, Fan Zhang, Zhiwei Qin, Yansheng Wang, Dingyuan Shi, Bingchen Song, Yongxin Tong, Hongtu Zhu, and Jieping Ye. 2021. Value function is all you need: A unified learning framework for ride hailing platforms. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 3605–3615. [26] Hai Wang, Lei Zhao, Kaile Yan, Xiaoyi Wang, Yile Liang, and Jinghua Hao. 2025. Meituan: Operational-Level On-Demand Food Delivery Data for the 2025 INFORMS TSL Data-Driven Research Challenge. Available at SSRN 5696423 (2025). [27] Jingwei Wang, Qianyue Hao, Wenzhen Huang, Xiaochen Fan, Qin Zhang, Zhentao Tang, Bin Wang, Jianye Hao, and Yong Li. 2025. Coopride: Cooperate all grids in city-scale ride-hailing dispatching with multi-agent reinforcement learning. In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 1. 1457–1468. [28] Shuai Wang, Baoshen Guo, Yi Ding, Guang Wang, Suining He, Desheng Zhang, and Tian He. 2023. Time-constrained actor-critic reinforcement learning for concurrent order dispatch in on-demand delivery. IEEE Transactions on Mobile Computing 23, 8 (2023), 8175–8192. [29] Yansheng Wang, Yongxin Tong, Cheng Long, Pan Xu, Ke Xu, and Weifeng Lv. 2019. Adaptive dynamic bipartite graph matching: A reinforcement learning approach. In 2019 IEEE 35th international conference on data engineering (ICDE). IEEE, 1478–1489. [30] Haomin Wen, Youfang Lin, Lixia Wu, Xiaowei Mao, Tianyue Cai, Yunfeng Hou, Shengnan Guo, Yuxuan Liang, Guangyin Jin, Yiji Zhao, et al. 2024. A survey on service route and time prediction in instant delivery: Taxonomy, progress, and prospects. IEEE Transactions on Knowledge and Data Engineering (2024). [31] Yi Yang, Yurong Cheng, Ye Yuan, Guoren Wang, Lei Chen, and Yongjiao Sun. 2022. Privacy-preserving cooperative online matching over spatial crowdsourcing platforms. Proceedings of the VLDB Endowment 16, 1 (2022), 51–63. [32] Zhaoxing Yang, Haiming Jin, Guiyun Fan, Min Lu, Yiran Liu, Xinlang Yue, Hao Pan, Zhe Xu, Guobin Wu, Qun Li, et al. 2024. Rethinking order dispatching in online ride-hailing platforms. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 3863–3873. [33] Chao Yu, Akash Velu, Eugene Vinitsky, Jiaxuan Gao, Yu Wang, Alexandre Bayen, and Yi Wu. 2022. The surprising effectiveness of ppo in cooperative multi-agent games. Advances in neural information processing systems 35 (2022), 24611–24624. [34] Libin Zheng, Lei Chen, and Jieping Ye. 2018. Order dispatch in price-aware ridesharing. Proceedings of the VLDB Endowment 11, 8 (2018), 853–865. [35] Zefang Zong, Tao Feng, Jingwei Wang, Tong Xia, and Yong Li. 2025. Deep Reinforcement Learning for Demand-Driven Services in Logistics and Transportation Systems: A Survey. ACM Transactions on Knowledge Discovery from Data 19, 4 (2025), 1–42. [36] Zefang Zong, Meng Zheng, Yong Li, and Depeng Jin. 2022. Mapdp: Cooperative multi-agent reinforcement learning to solve pickup and delivery problems. In Proceedings of the AAAI conference on artificial intelligence, Vol. 36. 9980–9988.

Related documents

• FedHV: Low-Overhead Hypervolume Weighting for Federated Multi-Objective Optimization#1117121
• Collate: Collaborative Neural Network Learning for Latency-Critical Edge Systems#353033
• MEDEA: A Design-Time Multi-Objective Manager for Energy-Efficient DNN Inference on Heterogeneous Ultra-Low Power Platforms#617837
• MEDEA: A Design-Time Multi-Objective Manager for Energy-Efficient DNN Inference on Heterogeneous Ultra-Low Power Platforms#617337
• Incorporating energy-efficient concrete materials in structural reinforced concrete design using multi-objective optimization#306280