arXiv CSopen access

Uno-Orchestra: Parsimonious Agent Routing via Selective Delegation

Uno-Orchestra: Parsimonious Agent Routing via Selective Delegation Zhiqing Cui∗, Haotong Xie†, Jiahao Yuan, Cheng Yang, Hanqing Wang, Yuxin Wu

arXiv:2605.05007v1 [cs.AI] 6 May 2026

Yifan Wu, Siru Zhong, Tao Yu, Yifu Guo, Siyu Zhang, Xinlei Yu, Qibing Ren, Usman Naseem GitHub: https://github.com/CuiZHIQ/Uno-Orchestra Dataset: https://huggingface.co/datasets/tinaxie/Uno-Curriculum

Abstract Large language model (LLM) multi-agent systems typically rely on rigid orchestration, committing either to flat per-query routing or to hand-engineered task decomposition, so decomposition depth, worker choice, and inference budget are not jointly optimized under one objective. We introduce U NO -O RCHESTRA, a unified orchestration policy that selectively decomposes a task and dispatches each subtask to an admissible (model, primitive) pair, with both decisions learned together from curated RL trajectories grounded in real worker interactions. Against 22 baselines on a 13-benchmark suite spanning math, code, knowledge, longcontext, and agentic tool-use, U NO -O RCHESTRA reaches 77.0% macro pass@1, roughly 16% above the strongest workflow baseline, at roughly an order of magnitude lower per-query cost, advancing the accuracy-efficiency frontier of selective delegation.

1

Introduction

Large language models have rapidly evolved from monolithic problem solvers into the building blocks of complex agentic systems that plan, retrieve and execute tools on behalf of human users [19, 17, 72]. As both the capability frontier and the per-token cost of these models stretch over more than an order of magnitude [8], the dominant deployment pattern of routing every query to a single model has become both wasteful and unreliable, and a growing body of work argues that capability now emerges from a learned controller that coordinates a population of heterogeneous models and tools rather than from any single model in isolation [71, 1]. A general-purpose orchestrator must therefore learn when and how to decompose a task before any routing decision is made. Following capability taxonomies for agentic LLMs [40, 66], we optimized this requirement along six capability axes that jointly span our training corpus and evaluation suite: atomic reasoning, compositional reasoning, knowledge retrieval, multi-hop composition, tool & code use, and agentic & long-context tasks. These axes demand a spectrum of orchestrator behaviours: simple queries are answered directly, while harder ones decompose into multiple coupled subtasks whose declared dependencies form a small dependency graph. Two largely parallel research threads have grown out of this premise. The first treats orchestration as model routing, in which a lightweight router dispatches each query to one expert: recent variants learn training-free from in-context skill profiles or generated data [65, 41, 38] and end-to-end via reinforcement signals over multi-round, budgeted dispatch [76, 55, 42, 33]. The second treats orchestration as task decomposition over a graph of sub-agents, in which a planner spawns specialised workers and aggregates their outputs [13, 53, 78, 69]. ∗ Project lead. [email protected] † Equal contribution.

Preprint.

Figure 1: LLM orchestration paradigms: (A) model router, (B) hierarchical orchestra, (C) U NO O RCHESTRA (ours).

In practice, however, both threads degenerate into limited patterns. (1) Single-call routers commit at query granularity: they pick one expert per query, miss the parallel structure of complex tasks, and the few budgeted variants [45, 16] live in a flat single-call action space that cannot trade decomposition depth against execution budget. (2) Hierarchical orchestrators decouple decomposition from dispatch: a planner produces sub-agent calls, but the assignment of each subtask to an executor is typically hand-engineered, prompt-driven, or tied to a proprietary frontier model. As a result, the two decisions that jointly shape the quality-efficiency trade-off, whether and how deeply to decompose a task and which expert to route each subtask to, are not optimized under a unified objective. This pattern raises the central question of our work. Can a single orchestrator learn, end to end, to decompose a task into subtasks and route every subtask to a (model, primitive) pair under an explicit cost budget? Here a primitive denotes the atomic routable action a worker performs on a subtask, subsuming model-internal cognitive operations, multi-step skill invocations, and externaltool calls under a single closed vocabulary. We answer affirmatively with U NO -O RCHESTRA, a router-orchestrator that, given an arbitrary task, emits a plan over subtasks together with an explicit (model, primitive) routing decision for every subtask, all from a single causal-LM policy. The key design choice is to collapse decomposition and dispatch into a single shared backbone: one causal-LM jointly emits the task decomposition and the per-subtask (model, primitive) routing within the same assistant turn, eliminating redundant context passes between separate planner and dispatcher modules. The policy is trained in two stages. We first apply supervised fine-tuning on a verifier-gated curriculum of 61,201 teacher-distilled trajectories drawn from 38 public datasets disjoint from our evaluation suite, with code and tool subtasks executed in real sandboxes so that observations are runtime outputs rather than simulated traces. We then refine the policy on the residual hard pool with AGENTIC -GRPO, a multi-turn extension of GRPO [57]: leveraging the orchestrator’s structural cues, it augments each rollout with observation- and process-level intermediate rewards that adapt to varied decomposition patterns and resolve the long-horizon credit-assignment problem. Our contributions are threefold: • We formulate selective delegation as a unified causal-LM policy that jointly emits subtask decomposition and per-subtask (model, primitive) routing decisions. • We build a verifier-gated training pipeline and introduce Agentic-GRPO, an agent-adaptive RL objective with structured credit assignment for multi-turn orchestration. 2

• We evaluate U NO -O RCHESTRA on a 13-benchmark suite and show that U NO -O RCHESTRA reaches 77.0% macro pass@1, roughly 16% above the strongest workflow baseline, at roughly an order of magnitude lower per-query cost.

2

Related Work

2.1

LLM-based Task Orchestration and Dynamic Routing

Work on coordinating heterogeneous language models bifurcates along two dimensions: the granularity of the routing decision, and whether the routing policy is learned. At query granularity, a router selects one expert per request, an abstraction instantiated training-free through in-context model representations [65], generated-data skill estimation [41], and task-profile-guided synthesis for cold-start regimes [38], as well as through reinforcement learning of multi-round dispatch and aggregation [76, 55]. A subset of these methods incorporates the cost of each call into the optimisation [45, 16] or pursues explainability of routing decisions [42], and the rapid growth of this family has motivated dedicated benchmarks [33]. At subtask granularity, orchestration becomes hierarchical: a planner decomposes a task into subtasks and dispatches each to a specialised worker. Hand-designed planner-worker trees [77, 68, 58] fix both the decomposition and the routing policy in advance, while later work learns or evolves the sub-agent set, the topology, or the multi-agent routing itself [50, 13, 75, 78, 73]; production interfaces such as Claude Code’s subagent layer [4] expose similar primitives at the API layer. Across both granularities, the decomposition and routing policies are typically optimized in isolation, and budgeted variants exist almost exclusively in the flat single-call regime. 2.2

Reinforcement Learning for Agentic Alignment

Reinforcement learning is the dominant paradigm for long-horizon LLM alignment, ranging from preference-based RLHF optimized with PPO [54, 43] to the reward-model-free DPO objective [47]. Group Relative Policy Optimisation (GRPO) [57] replaces the value network with a group-relative advantage estimator and underpins outcome-reward reasoning models such as DeepSeek-R1 [22]. However, vanilla GRPO transfers poorly to multi-turn agentic rollouts where the reward arrives only at termination and the group-relative advantage is averaged over an entire trajectory. A recent line of work attacks this credit-assignment gap with finer-grained advantages: turn-level attribution [74], action-level inner groups [18], tree-advantage variants that reuse partial trajectories across branches [29, 14], and breadth-axis extensions that share rollouts across cooperating agents [69].

3

Preliminary

We frame agentic orchestration as a multi-turn decision process under an explicit cost budget. For each query q ∼ Q, a controller iteratively plans subtasks, dispatches each to a heterogeneous worker, receives an observation, verifies progress, and emits a final answer. Workers, primitives, and admissible pairs. Let M be a closed pool of frozen worker LLMs and S a closed vocabulary of routing primitives spanning external-tool invocations and model-internal cognitive operations. Workers cover S heterogeneously, so the controller selects from a sparse admissible set P ⊆ M × S of (model, primitive) pairs p = (m, s), each carrying a token-level cost c(p) that varies by orders of magnitude across P. Trajectory and policy. At turn t the controller emits an action at ∼ πθ (· | ht ) with history ht = (q, a<t , o<t ); at contains a plan over subtasks together with one routing pair from P for each subtask. The environment dispatches the routed calls and returns an observation ot . A horizon-T trajectory ends with a final answer y, τ = (q, a1 , o1 , . . . , aT , oT , y).

(1)

Verifier. Each source provides a verifier V that scores the final answer against the gold under a source-specific equivalence (symbolic for math, exact-match or F1 for QA, sandbox tests for code, 3

(A) Uno-orchestra Teacher Process

User query Q

Teacher LLM

S0

Orchestrator

O1

Delegate A0

S1

Injection

multi-hop reasoning

compositional reasoning

long-horizon planning

ra te In

ion ct

env

Cognitive

Skills

...

Orchestrator

DRL

Real-Environment Trajectory construction

retrieval

Configuring Primitive

AT ...

Tool Code / Search /...

Fast / Slow /...

Dynamic Primitive T

OT

End

ST+1

Final Answer✔

...

Multi-Turn Selective delegation

Dynamic Primitive 1

(B) Two-Stage Training

atomic reasoning

...

Subtask1 depend on root: “Find places within 10 km of the Eiffel Tower.” Subtask2 depend on root: Find places within 10 km of the Statue of Liberty。 Subtask3 depend on “1,2” Merge the three location result sets into the final answer.

Initial State

tooluse

Configuring Model

Sub-Task Delegtion

when, where & how to decompose

Find places near the Eiffel Tower and Stat -ue of Liberty within a radius of 10 km.

Multi-Turn RL

Sub Task t Ti

(q,a1,o1)

Cold Start

Step i

Base LLM

DSFT

Env.

Uno-orchestra RL

Sub Task 2

Group Comp.

Sub Task 3

Reward ri,s For Subtask t with step i

Sub Task t

Reference Model

Query

Sub Task 1

correctly decomposed

truncated/ no golden answer❌

<obs> verifier

decompose -and-route

KL

Policy Model

Uno-orchestra SFT

Stage 1 Supervised Finetuning

Stage 2 Agentic GRPO

Figure 2: Overview of U NO -O RCHESTRA. (A) Multi-turn selective delegation: at each turn the orchestrator decides when, where, and how to decompose the task, configures one (model, primitive) routing pair per subtask, and dispatches the subtasks to heterogeneous workers; observations feed back as conditioning context for the next turn. (B) Two-stage training: Stage 1 (SFT) distils teacher trajectories grounded in real environment interactions across our six capability axes, retaining only correctly decomposed gold-aligned trajectories into DSFT ; Stage 2 (Agentic-GRPO) refines the policy on the residual hard pool DRL using a verifier, group-relative comparison, and a KL regulariser to a frozen reference. schema match for tool-use), thresholded into a binary correctness signal b = V (q, y) ∈ {0, 1}. Intermediate observations are interaction feedback, not dataset labels. Objective.

With trajectory cost c(τ ) =

PT

t=1

P

p∈at c(p), we train the controller to maximise


 J(θ) = Eq∼Q, τ ∼πθ U b, c(τ ); α ,

(2)

where α ∈ [0, 1] trades off correctness against cost; the concrete forms of U and πθ are given in §4.

4

Method

We instantiate πθ from §3 as a single causal language model. At each assistant turn it emits a plan together with one routing pair (m, s) ∈ P per subtask, where the primitive s denotes the atomic action that worker m performs on the subtask, drawn from a closed vocabulary that subsumes model-internal cognitive operations (e.g. direct answer, chain-of-thought reasoning), multi-step skill invocations (e.g. document reading, code parsing), and external-tool calls (e.g. code execution, retrieval). Worker outputs return as observations and feed back into the policy as conditioning context for the next turn, which either replans, repairs, or emits the final answer. Training proceeds in two stages (Fig. 2): (i) supervised fine-tuning on a verifier-gated teacher curriculum, then (ii) agent-adaptive reinforcement learning on the residual hard pool. Verifier-gated curriculum. Both stages share a source-verifiable pool D0 . Let π (0) be a coldstart router and π ⋆ a strong teacher orchestrator, with verifier scores b(0) (q) = V (q, π (0) (q)) and b⋆ (q) = V (q, π ⋆ (q)). We discard tasks already solved by π (0) and split the remainder by teacher outcome, 
 DSFT = q, τ ⋆ (q) : b(0) = 0, b⋆ = 1 , DRL = q : b(0) = 0, b⋆ = 0 , (3) so DSFT supplies behaviour-cloning targets via verifier-passing teacher trajectories τ ⋆ (q), and DRL collect the residual hard pool where exploration is required. 4

Selective delegation. At each turn t, πθ chooses between a direct answer that terminates with y and a decompose-and-route action emitting a plan whose Kt subtasks, together with their declared dependencies, form a small dependency graph; each subtask carries one routing pair pt,k ∈ P, and independent subtasks are dispatched in parallel. Plan and routing tokens are produced by the same backbone in a single forward pass, so the joint distribution factorises along the token order: πθ (at | ht ) = πθ (plant | ht ) · {z } | decompose

Kt Y

πθ pt,k | ht , plant , pt,<k , {z } | k=1

(4)

route

the two factors being separated only by left-to-right causal masking, so the decompose-then-route structure is obtained without auxiliary heads, sub-networks, or per-stage loss weights. Selectivity is the central property: simple queries collapse to a single direct-answer turn at zero dispatch cost, while only genuinely compositional tasks pay for multi-call orchestration. 4.1

Outcome Reward

We instantiate U to keep verifier correctness the dominant signal while retaining a bounded gradient p on incorrect rollouts. Let ĉ(τ ) ∈ [0, 1] be the trajectory cost normalised by clipping c(τ ) against a running percentile bracket of recent rollouts. The terminal reward combines a verifier-gated cost reward with a bounded routing-shaping term, 
 R(τ ) = b · (1 − α) + α 1 − ĉ(τ ) + (1 − b) · S(τ ), S(τ ) ∈ [0, 0.10], (5) where S(τ ) rewards schema-valid plan-and-route emissions when the verifier is unsatisfied. The cap S ≤ 0.10 ≪ 1 keeps the verifier dominant and forecloses the failure mode in which the policy harvests cheap-and-wrong cost bonuses. 4.2

Learning to Orchestrate

Why intermediate credit. Vanilla GRPO [57] collapses an entire rollout into a single trajectorylevel return, so all assistant turns share one group-relative advantage. In multi-turn orchestration the verifier signal only arrives at termination, leaving early routing choices, repair triggers, and premature aggregation unsupervised, and the gradient flat with horizon length. Agentic-GRPO routes a portion of the terminal reward back to each turn, augmented with a bounded per-turn shaping signal, so credit is attributed at the granularity of individual orchestration decisions. Turn-level return and advantage. For trajectory i at turn t with horizon Ti , let ri,t ∈ [−η, η] be a per-turn shaping reward (e.g. schema validity, repair indicator) bounded by η ≪ 1 so it cannot overpower the verifier. The turn-level return and its group-standardised advantage are R̃i,t = γ Ti −t R(τi ) +

Ti X

γ s−t ri,s ,

s=t

Âi,t =

R̃i,t − µq,t , σq,t + ε

(6)

where (µq,t , σq,t ) are the within-query mean and standard deviation of R̃j,s over comparable turns of the rollout group (same query, matched turn index and action type). When the rollout is a single chain, Agentic-GRPO recovers turn-level credit [74]; when sibling turns share a prefix, it recovers branch-level credit [29]. Masked clipped objective. Let Ti be the set of token indices emitted by πθ in trajectory i, with observation and chat-template tokens excluded, and write t(ℓ) for the turn containing token ℓ. The actor loss is the clipped GRPO surrogate restricted to Ti plus a low-variance KL regulariser to a frozen reference πref : " #   XX L(θ) = − Eτ ∼πθold min ρi,ℓ Âi,t(ℓ) , clip(ρi,ℓ , 1−ϵ, 1+ϵ) Âi,t(ℓ) + β D̂KL , (7) i

ℓ∈Ti

with importance ratio ρi,ℓ = πθ (xi,ℓ | hi,ℓ ) / πθold (xi,ℓ | hi,ℓ ) and KL estimator D̂KL = ρ−log ρ−1. The set Ti coincides with the SFT loss mask, so SFT and RL optimise the same router-emitted positions under different objectives. 5

5

Experimental Setup

Benchmarks and metrics. We evaluate U NO -O RCHESTRA and competing routers on 13 benchmarks spanning five capability domains: mathematical reasoning (MATH-500 [36], AIME [3]); code & software engineering (HumanEval [8], MBPP [5], LiveCodeBench [28], SWE-bench [30]); knowledge & scientific reasoning (MMLU [24], GPQA [49]); reading & long-context (DROP [15], MRCR [61]); agentic & tool use (GAIA [40], Terminal-Bench [63], ToolBench [46]); plus LLMRouterBench [33] as a routing-specialised diagnostic that is not counted in the 13-benchmark macroaverage. All evaluation sources are kept disjoint from training. We report two task-performance metrics, pass@1 (single attempt) and pass@2 (one allowed retry over independent samples). Domain aggregation and RL training. Tab. 1 reports macro pass@1/pass@2 within each capability domain before efficiency columns: each domain figure is an unweighted mean over its constituent benchmarks (Math: MATH-500 and AIME; Code/SE: HumanEval, MBPP, LiveCodeBench, SWEbench; Know.: MMLU and GPQA; Read.: DROP and MRCR; Agentic: GAIA, Terminal-Bench, ToolBench). The final two columns (tok and USD/q) report the mean context length and the billed USD per query, averaged over the same 13 benchmarks under a unified rollout harness for every competitor. RL refinement trains Agentic-GRPO [57] on the verifier-filtered pool (2,976 questions; §4.2) with G=8 comparison rollouts per task and at most Tmax =8 orchestrator turns; decoding is capped near 16,384 assistant tokens (4,096 tokens of prompt retained per planner turn within that budget); optimisation uses AdamW at 10−6 , PPO clipping ϵ=0.2, KL against a frozen router reference β=10−3 , terminal cost blending α=0.1 (§4.1), and collocated worker rollouts executed through vLLM. Hardware for both SFT and RL is a single 8-GPU NVIDIA A100 80 GB node. Routing infrastructure. All main U NO -O RCHESTRA variants share a Qwen2.5-7B-Instruct router. The main pool is a strong heterogeneous mix of nine commercial workers spanning over two orders of magnitude in per-token price (Gemini-2.5-Flash-Lite, Gemini-2.5-Flash, Gemini-3-Flash-Preview, Gemini-3.1-Pro-Preview, Kimi-K2.5, GPT-5.3-Codex, GPT-5.4, Claude-Sonnet-4-6, Claude-Opus4-6); for ablation we also use a Qwen2.5 scale ladder including all different parameter sizes that removes proprietary frontier workers but preserves capability diversity by parameter scale. During RL the router observes only anonymous worker labels (W ORKER 1, . . . , K), with the mapping to backend identity hidden and resampled per episode. This forecloses brand-name shortcuts and forces the router to profile each worker through interaction and reward feedback. Baselines. We compare against five families: (i) static / direct inference (Direct Claude Opus); (ii) single-round routers (RouterDC [9], GraphRouter [16], ICL-Router [65], ColdStart-LLM [38]); (iii) multi-round / RL routers (PromptLLM, KNN-MR, and Router-R1 [76], R2-Reasoner [56], AutoMix [2], WideSeek-R1 [69], xRouter [45], ATLAS [68] cluster/RL); (iv) agentic-workflow systems (Tool Orchestra [58], AOrchestra [50], AgentOrchestra [77], SkillOrchestra [66], Puppeteer [13], ToolLLM [46], MasRouter [73]); (v) ours (five training stages summarised under Ablation Studies, §6.3); Tab. 1 lists the three RL checkpoints together with external baselines, and Tab. 3 reports early-stage checkpoints. All baselines share the worker pool and inference budget where applicable. Additional reporting conventions, curricula, primitives, rollout pricing rules, fuller hyper-parameters, and per-task tables are consolidated in Apps. A to D.

6

Experimental Analysis

6.1

Main Results

Tab. 2 reports per-benchmark pass@1 for five representative competitors and the full method, then gives the full 13-benchmark comparison across 22 baselines and the three RL variants of U NO O RCHESTRA; Fig. 3 visualizes the aggregate trade-offs across accuracy, cost and context that underlie those domain averages, monetary cost with an indicative p95-latency direction and context load with a turn-depth sketch. U NO -O RCHESTRA is best on both macro metrics, 77.0 pass@1 and 81.7 pass@2, roughly 16% and 14% above AgentOrchestra. The gain is strongest on the agentic and reading/longcontext domains, while the step from Uno-tree-GRPO to Uno-Orchestra also slightly lowers cost. This thrift is a structural property of the (model, primitive) action space: on simple queries the router collapses orchestration to a single direct dispatch to a cheap yet competent worker, paying 6

(a) Accuracy vs. billed cost.

(b) Accuracy vs. context tokens.

Figure 3: Accuracy and efficiency overview on the 13-benchmark suite. Panel (a) plots macro pass@1 against billed cost in USD per query, with a shaded near-Pareto region and an indicative direction for tail latency (p95); panel (b) plots the same accuracy against average context tokens per subtask, contrasting single-round and multi-round routers, and the diagonal “turns” cue sketches how deeper interaction shifts both accuracy and context. U NO -O RCHESTRA sits on the favourable corner of both views, combining the highest macro accuracy with the smallest cost and context budgets among learned routers and agentic-workflow baselines. Table 1: Main results: five-domain macro accuracy with serving cost and context. Macro pass@1 / pass@2 across math, code/SE, knowledge, reading, and agentic tool-use domains, together with average context tokens and per-query inference cost, comparing 22 external baselines against three RL variants of U NO -O RCHESTRA. Method

Math

Code/SE

Know.

Read.

Agentic

Efficiency

pass@1 pass@2 pass@1 pass@2 pass@1 pass@2 pass@1 pass@2 pass@1 pass@2

tok

USD/q

RouterDC [9] GraphRouter [16] ICL-Router [65] ColdStart-LLM [38]

48.2 45.1 55.7 38.6

55.4 52.5 63.5 45.8

50.6 52.0 53.2 43.9

56.5 58.9 59.2 51.5

61.0 59.4 60.8 56.3

65.3 63.9 65.4 60.6

48.0 54.8 47.5 46.0

53.0 59.8 52.6 51.2

24.5 41.7 27.3 23.8

29.8 49.0 32.9 30.5

529 585 525 540

0.2146 0.3084 0.2100 0.2288

PromptLLM [76] R2-Reasoner [56] Router-R1 [76] AutoMix [2] WideSeek-R1 [69] xRouter [45] ATLAS (cluster) [68] ATLAS (RL) [68]

40.2 47.6 38.9 34.6 41.2 55.7 56.1 52.1

47.2 55.1 44.5 40.8 48.7 63.3 63.8 60.7

51.1 50.8 12.1 46.9 46.7 56.1 49.8 51.4

58.3 57.8 19.4 55.0 54.1 62.2 56.9 58.3

57.2 61.2 53.2 53.5 58.1 69.4 60.5 62.0

61.9 65.8 57.3 57.2 62.6 74.0 65.0 66.8

54.0 48.0 39.5 42.5 51.0 53.9 49.0 50.7

58.9 52.9 45.0 47.8 56.0 59.0 54.2 55.5

40.9 26.8 13.9 22.5 35.8 35.4 29.4 32.0

48.2 33.1 19.7 28.3 43.2 42.1 35.9 38.8

747 601 380 1250 860 584 662 736

0.9071 0.3564 0.0005 0.2955 0.6485 0.2490 0.4086 0.5032

Tool Orchestra [58] AOrchestra [50] AgentOrchestra [77] SkillOrchestra [66] Puppeteer [13] ToolLLM [46] MasRouter [73]

37.6 59.3 53.1 53.6 47.6 22.7 50.9

42.5 67.1 61.1 61.1 55.1 28.3 58.5

57.9 63.5 71.4 55.2 51.3 26.3 56.1

62.7 72.4 76.5 61.2 58.3 33.1 61.6

55.3 66.0 66.2 63.3 61.2 45.5 63.2

59.9 71.1 71.3 67.9 65.8 49.6 67.8

40.6 68.2 65.0 50.3 48.9 38.4 51.0

40.8 73.2 69.4 55.3 54.0 43.7 56.0

24.6 57.9 68.6 35.5 29.6 30.1 31.7

29.2 66.3 74.6 42.0 36.4 34.2 37.7

1896 1600 1724 533 694 667 610

0.8100 0.9932 1.2118 0.1870 0.4626 0.1355 0.3121

Uno-GRPO Uno-tree-GRPO Uno-Orchestra

77.6 78.5 79.2

83.0 84.0 84.7

75.4 76.9 77.8

80.5 81.7 82.2

79.2 79.9 80.5

82.9 83.7 84.3

76.5 78.2 79.7

80.8 82.8 84.1

66.6 68.9 70.3

72.5 74.6 75.8

391 419 411

0.1610 0.1712 0.1011

near-zero coordination overhead, while reserving multi-call decomposition strictly for genuinely composite tasks. The same property explains why U NO -O RCHESTRA undercuts every workflow baseline in cost without giving up macro accuracy, an outcome that flat routers cannot reach because they cannot decompose, and that hierarchical orchestrators cannot reach because they cannot abstain from decomposing. Beyond the final score, the shape of the progression is informative. SFT delivers the first sizeable improvement by stabilising the orchestration policy; GRPO sharpens route selection; tree rollouts add a marginal increase in search coverage; and Agentic-GRPO contributes a smaller but consistent improvement by attributing credit to the turns that drive the outcome. The remaining gains therefore come from better delegation decisions rather than longer trajectories, since the controller stays near 7

Table 2: Per-benchmark pass@1 against five representative competitors. Columns span the 13-benchmark main suite. Method

HE MBPP GPQA MMLU AIME GAIA DROP

LCB MRCR SWE TBench ToolB

ColdStart-LLM [38] xRouter [45] AOrchestra [50] AgentOrchestra [77]

MATH 60.9 76.4 81.7 75.1

72.4 88.1 84.2 89.2

69.8 83.8 80.8 85.7

35.6 54.7 48.3 46.9

77.1 84.1 83.7 85.6

16.4 34.9 36.9 31.2

17.2 24.8 69.4 83.4

59.4 70.6 73.6 75.2

18.7 27.6 27.4 28.4

32.7 37.2 62.8 54.8

14.6 24.8 61.7 82.4

12.7 21.6 40.6 54.2

41.6 59.9 63.8 68.1

Uno-Orchestra

91.9

93.1

92.4

69.2

91.8

66.5

82.0

82.4

44.0

77.0

81.8

57.2

71.6

↑7.2% ↑80.2%

↓1.7%

↑9.6% ↑54.9%

Rel. gain vs best baseline ↑12.5% ↑4.4%

↑7.8% ↑26.5%

↑22.6% ↓0.7%

↑5.5% ↑5.1%

the same token budget while improving accuracy. Notably, Agentic-GRPO trims per-query cost slightly below Uno-tree-GRPO while still raising accuracy, indicating that turn-level credit removes redundant dispatches faster than it introduces new ones, so each successive RL stage allocates capacity to selecting the right worker rather than invoking additional workers. 6.2

Generalisation across Domain Shifts

Figure 4: Performance under three train-test distribution-shift regimes: (a) in-domain, (b) neardomain, and (c) out-of-domain. We stratify the 13-benchmark suite by its capability-axis overlap with the SFT+RL training pool into in-domain, near-domain, and out-of-domain regimes (Fig. 4). U NO -O RCHESTRA leads in every regime and the margin grows from in-domain to out-of-domain, indicating that the learned (model, primitive) policy generalises rather than memorises the source mix. 6.3

Ablation Studies

Uno variants., Uno-base is the untrained router. Uno-SFT is the same router fine-tuned on the verifier-gated curriculum (§4). Uno-GRPO adds vanilla GRPO [57] on top of Uno-SFT with a single trajectory-level reward. Uno-tree-GRPO replaces single-path rollouts with prefix-sharing tree rollouts in the spirit of Tree-GRPO [29]. Uno-Orchestra (the full method, §4.2) further adds turnlevel credit assignment and bounded process shaping. A compact summary of these RL objectives is given in App. B. All Uno variants share the same router backbone, worker pool, prompt format, and inference budget. Tab. 3 shows that the sequence Uno-base→Uno-SFT→Uno-GRPO→Uno-tree-GRPO→UnoOrchestraimproves accuracy step by step while keeping the final controller cheaper than the tree-only variant. As an additional control, removing the blind worker protocol from Uno-GRPO lifts accuracy by less than one point per domain, yet inflates per-query cost from $0.1610 to $0.8163 and average context from 391 to 548 tokens; brand identity is therefore a shortcut the policy exploits whenever it is not foreclosed. Worker-pool diversity. To check that the gains do not depend on access to a specific frontier worker, we replace the main commercial pool with a Qwen2.5 scale ladder 8

Table 3: Stage ablation of U NO -O RCHESTRA. 13-benchmark suite Variant Uno-base Uno-SFT Uno-GRPO Uno-tree-GRPO Uno-Orchestra

rep. benchmarks, pass@1 (%)

pass@1 pass@2 USD/q tok MATH 48.1 61.3 74.5 76.0 77.0

56.1 68.9 79.5 80.9 81.7

0.1785 0.1687 0.1610 0.1712 0.1011

351 370 391 419 411

72.0 84.5 90.4 91.2 91.9

HE MMLU AIME GAIA MRCR SWE ToolB 65.0 79.0 91.8 92.6 93.1

70.0 82.5 90.8 91.3 91.8

22.0 42.0 64.7 65.8 66.5

50.5 65.0 76.5 80.6 82.0

63.5 68.5 73.0 75.0 77.0

42.0 56.0 76.0 79.5 81.8

46.0 61.5 68.4 70.2 71.6

(Qwen2.5-0.5B/1.5B/3B/7B/14B/32B/72B-Instruct, DeepSeek-V3 and GPT-4o). Tab. 4 contrasts Weaker-pools-Uno-GRPO against a strong learned-router baseline R2-Reasoner under exactly the same pool. Despite removing every proprietary frontier worker, our orchestrator still leads R2-Reasoner across all five capability domains while incurring slightly lower context and cost. The improvement is therefore attributable to the learned (model, primitive) routing policy rather than to a single especially strong worker in the pool. Table 4: Weak-worker-pool ablation on a Qwen2.5 scale ladder. The pool removes every proprietary frontier worker but preserves capability diversity through parameter scale and a small number of public mid-size workers. Method

Math

Code/SE

Know.

Read.

Agentic

Efficiency

p@1 p@2 p@1 p@2 p@1 p@2 p@1 p@2 p@1 p@2 tok USD/q R2-Reasoner [56] 47.6 55.1 50.8 57.8 61.2 65.8 48.0 52.9 26.8 33.1 601 0.3564 Weaker-pools-Uno-GRPO 48.3 55.8 51.5 58.5 61.8 66.4 48.7 53.6 27.4 33.8 589 0.3480

Router-backbone size. The router itself is a Qwen2.5-7B-Instruct in the main configuration. Fig. 5 tests whether the policy transfers to a smaller controller by repeating the SFT and GRPO stages on a Qwen3-4B-Instruct router under the same worker pool, prompt format and inference budget. After SFT alone the 4B router nearly matches its 7B counterpart, but the RL ceiling is consistently lower across all five domains, with the largest gap on the agentic split. This suggests that controller capacity matters less for instruction following on the orchestration grammar than for credit assignment over Figure 5: Router-backbone comparison: pass@1 multi-turn rollouts, which is where the addi- (dashed) and pass@2 (solid) across five capability domains. tional 7B capacity is converted into accuracy.

7

Conclusion

We presented U NO -O RCHESTRA, a unified orchestration policy that selectively decomposes a task and dispatches each subtask to an admissible (model, primitive) pair, with both decisions learned together from curated RL trajectories grounded in real worker interactions. On a 13-benchmark suite, U NO -O RCHESTRA attains 77.0% macro pass@1, roughly 16% above the strongest workflow baseline at roughly an order of magnitude lower per-query cost, and the lead extends to weaker worker pools, smaller router backbones and out-of-domain benchmarks. Selective delegation, the ability to both decompose and abstain from decomposing under one objective, therefore offers a useful organising principle for advancing the accuracy-efficiency frontier of cost-aware multi-agent systems. 9

8

Authors Zhiqing Cui (NUIST) Jiahao Yuan (ECNU) Hanqing Wang (HKUST–GZ) Yifan Wu (Ramus) Tao Yu (CAS–ICT) Siyu Zhang (UC San Diego) Qibing Ren (SJTU)

Haotong Xie (SUFE) Cheng Yang (HDU) Yuxin Wu (RUC) Siru Zhong (HKUST–GZ) Yifu Guo (SYSU) Xinlei Yu (NUS) Usman Naseem (Macquarie University)

References [1] Apoorva Adimulam, Rajesh Gupta, and Sumit Kumar. The orchestration of multi-agent systems: Architectures, protocols, and enterprise adoption. arXiv preprint arXiv:2601.13671, 2026. [2] Pranjal Aggarwal, Aman Madaan, Ankit Anand, Srividya Pranavi Potharaju, Swaroop Mishra, Pei Zhou, Aditya Gupta, Dheeraj Rajagopal, Karthik Kappaganthu, Yiming Yang, et al. Automix: Automatically mixing language models. arXiv preprint arXiv:2310.12963, 2023. [3] AI-MO. Ai-mo/aimo-validation-aime. aimo-validation-aime, 2024.

https://huggingface.co/datasets/AI-MO/

[4] Anthropic. Claude code: Create custom subagents. Anthropic Documentation, 2026. URL https://docs.anthropic.com/en/docs/claude-code/sub-agents. [5] Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, et al. Program synthesis with large language models. arXiv preprint arXiv:2108.07732, 2021. [6] Jonathan Berant, Andrew Chou, Roy Frostig, and Percy Liang. Semantic parsing on freebase from question-answer pairs. In Proceedings of the 2013 conference on empirical methods in natural language processing, pages 1533–1544, 2013. [7] Yonatan Bisk, Rowan Zellers, Jianfeng Gao, Yejin Choi, et al. Piqa: Reasoning about physical commonsense in natural language. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 7432–7439, 2020. [8] Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374, 2021. [9] Shuhao Chen, Weisen Jiang, Baijiong Lin, James Kwok, and Yu Zhang. Routerdc: Query-based router by dual contrastive learning for assembling large language models. Advances in Neural Information Processing Systems, 37:66305–66328, 2024. [10] Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? try arc, the ai2 reasoning challenge. arXiv preprint arXiv:1803.05457, 2018. [11] Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168, 2021. [12] Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, et al. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. arXiv preprint arXiv:2507.06261, 2025. [13] Yufan Dang, Chen Qian, Xueheng Luo, Jingru Fan, Zihao Xie, Ruijie Shi, Weize Chen, Cheng Yang, Xiaoyin Che, Ye Tian, et al. Multi-agent collaboration via evolving orchestration. arXiv preprint arXiv:2505.19591, 2025. 10

[14] Zheng Ding and Weirui Ye. Treegrpo: Tree-advantage grpo for online rl post-training of diffusion models. arXiv preprint arXiv:2512.08153, 2025. [15] Dheeru Dua, Yizhong Wang, Pradeep Dasigi, Gabriel Stanovsky, Sameer Singh, and Matt Gardner. Drop: A reading comprehension benchmark requiring discrete reasoning over paragraphs. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 2368–2378, 2019. [16] Dongzhe Fan, Chuanhao Ji, Zimu Wang, Tong Chen, and Qiaoyu Tan. Graphrag-router: Learning cost-efficient routing over graphrags and llms with reinforcement learning. arXiv preprint arXiv:2604.16401, 2026. [17] Jinyuan Fang, Yanwen Peng, Xi Zhang, Yingxu Wang, Xinhao Yi, Guibin Zhang, Yi Xu, Bin Wu, Siwei Liu, Zihao Li, et al. A comprehensive survey of self-evolving ai agents: A new paradigm bridging foundation models and lifelong agentic systems. arXiv preprint arXiv:2508.07407, 2025. [18] Lang Feng, Zhenghai Xue, Tingcong Liu, and Bo An. Group-in-group policy optimization for llm agent training. arXiv preprint arXiv:2505.10978, 2025. [19] Mohamed Amine Ferrag, Norbert Tihanyi, and Merouane Debbah. From llm reasoning to autonomous ai agents: A comprehensive review. arXiv preprint arXiv:2504.19678, 2025. [20] Mor Geva, Daniel Khashabi, Elad Segal, Tushar Khot, Dan Roth, and Jonathan Berant. Did aristotle use a laptop? a question answering benchmark with implicit reasoning strategies. Transactions of the Association for Computational Linguistics, 9:346–361, 2021. [21] Google DeepMind. Gemini 3 Pro model card. https://storage.googleapis.com/ deepmind-media/Model-Cards/Gemini-3-Pro-Model-Card.pdf, 2025. [22] Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Peiyi Wang, Qihao Zhu, Runxin Xu, Ruoyu Zhang, Shirong Ma, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025. [23] Simeng Han, Hailey Schoelkopf, Yilun Zhao, Zhenting Qi, Martin Riddell, Wenfei Zhou, James Coady, David Peng, Yujie Qiao, Luke Benson, et al. Folio: Natural language reasoning with first-order logic. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 22017–22031, 2024. [24] Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. arXiv preprint arXiv:2009.03300, 2020. [25] Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. arXiv preprint arXiv:2103.03874, 2021. [26] Xanh Ho, Anh-Khoa Duong Nguyen, Saku Sugawara, and Akiko Aizawa. Constructing a multi-hop qa dataset for comprehensive evaluation of reasoning steps. In Proceedings of the 28th International Conference on Computational Linguistics, pages 6609–6625, 2020. [27] Hugging Face Open-R1. open-r1/codeforces-cots. https://huggingface.co/datasets/ open-r1/codeforces-cots, 2025. [28] Naman Jain, King Han, Alex Gu, Wen-Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang, Armando Solar-Lezama, Koushik Sen, and Ion Stoica. Livecodebench: Holistic and contamination free evaluation of large language models for code. arXiv preprint arXiv:2403.07974, 2024. [29] Yuxiang Ji, Ziyu Ma, Yong Wang, Guanhua Chen, Xiangxiang Chu, and Liaoni Wu. Tree search for llm agent reinforcement learning. arXiv preprint arXiv:2509.21240, 2025. [30] Carlos E Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik Narasimhan. Swe-bench: Can language models resolve real-world github issues? arXiv preprint arXiv:2310.06770, 2023. 11

[31] Mandar Joshi, Eunsol Choi, Daniel S Weld, and Luke Zettlemoyer. Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1601–1611, 2017. [32] Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, et al. Natural questions: a benchmark for question answering research. Transactions of the Association for Computational Linguistics, 7:453–466, 2019. [33] Hao Li, Yiqun Zhang, Zhaoyan Guo, Chenxu Wang, Shengji Tang, Qiaosheng Zhang, Yang Chen, Biqing Qi, Peng Ye, Lei Bai, et al. Llmrouterbench: A massive benchmark and unified framework for llm routing. arXiv preprint arXiv:2601.07206, 2026. [34] Jia Li, Edward Beeching, Lewis Tunstall, Ben Lipkin, Roman Soletskyi, Shengyi Huang, Kashif Rasul, Longhui Yu, Albert Q Jiang, Ziju Shen, et al. Numinamath: The largest public dataset in ai4maths with 860k pairs of competition math problems and solutions. Hugging Face repository, 13(9):9, 2024. [35] Rongao Li, Jie Fu, Bo-Wen Zhang, Tao Huang, Zhihong Sun, Chen Lyu, Guang Liu, Zhi Jin, and Ge Li. Taco: Topics in algorithmic code generation dataset. arXiv preprint arXiv:2312.14852, 2023. [36] Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let’s verify step by step. In The twelfth international conference on learning representations, 2023. [37] Hanmeng Liu, Jian Liu, Leyang Cui, Zhiyang Teng, Nan Duan, Ming Zhou, and Yue Zhang. Logiqa 2.0–an improved dataset for logical reasoning in natural language understanding. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 31:2947–2962, 2023. [38] Hui Liu, Bin Zou, Kecheng Chen, Jie Liu, Wenya Wang, and Haoliang Li. Task-aware llm routing with multi-level task-profile-guided data synthesis for cold-start scenarios. arXiv preprint arXiv:2604.09377, 2026. [39] Weiwen Liu, Xu Huang, Xingshan Zeng, Xinlong Hao, Shuai Yu, Dexun Li, Shuai Wang, Weinan Gan, Zhengying Liu, Yuanqing Yu, et al. Toolace: Winning the points of llm function calling. arXiv preprint arXiv:2409.00920, 2024. [40] Grégoire Mialon, Clémentine Fourrier, Thomas Wolf, Yann LeCun, and Thomas Scialom. Gaia: a benchmark for general ai assistants. In The Twelfth International Conference on Learning Representations, 2023. [41] Tianyi Niu, Justin Chih-Yao Chen, Genta Indra Winata, Shi-Xiong Zhang, Supriyo Chakraborty, Sambit Sahu, Yue Zhang, Elias Stengel-Eskin, and Mohit Bansal. Routing with generated data: Annotation-free llm skill estimation and expert selection. arXiv preprint arXiv:2601.09692, 2026. [42] Mika Okamoto, Ansel Kaplan Erol, and Mark Riedl. Explainable model routing for agentic workflows. arXiv preprint arXiv:2604.03527, 2026. [43] Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. In Advances in Neural Information Processing Systems, 2022. [44] Richard Yuanzhe Pang, Alicia Parrish, Nitish Joshi, Nikita Nangia, Jason Phang, Angelica Chen, Vishakh Padmakumar, Johnny Ma, Jana Thompson, He He, et al. Quality: Question answering with long input texts, yes! In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 5336–5358, 2022. 12

[45] Cheng Qian, Zuxin Liu, Shirley Kokane, Akshara Prabhakar, Jielin Qiu, Haolin Chen, Zhiwei Liu, Heng Ji, Weiran Yao, Shelby Heinecke, et al. xrouter: Training cost-aware llms orchestration system via reinforcement learning. arXiv preprint arXiv:2510.08439, 2025. [46] Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian, et al. Toolllm: Facilitating large language models to master 16000+ real-world apis. arXiv preprint arXiv:2307.16789, 2023. [47] Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D Manning, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. In Advances in Neural Information Processing Systems, 2023. [48] Samyam Rajbhandari, Jeff Rasley, Olatunji Ruwase, and Yuxiong He. Zero: Memory optimizations toward training trillion parameter models. In SC20: international conference for high performance computing, networking, storage and analysis, pages 1–16. IEEE, 2020. [49] David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R Bowman. Gpqa: A graduate-level google-proof q&a benchmark. arXiv preprint arXiv:2311.12022, 2023. [50] Jianhao Ruan, Zhihao Xu, Yiran Peng, Fashen Ren, Zhaoyang Yu, Xinbing Liang, Jinyu Xiang, Yongru Chen, Bang Liu, Chenglin Wu, et al. Aorchestra: Automating sub-agent creation for agentic orchestration. arXiv preprint arXiv:2602.03786, 2026. [51] Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. Winogrande: An adversarial winograd schema challenge at scale. Communications of the ACM, 64(9):99–106, 2021. [52] Maarten Sap, Hannah Rashkin, Derek Chen, Ronan Le Bras, and Yejin Choi. Social iqa: Commonsense reasoning about social interactions. In Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP), pages 4463–4473, 2019. [53] Philip Schroeder, Nathaniel W Morgan, Hongyin Luo, and James Glass. Thread: Thinking deeper with recursive spawning. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 8418–8442, 2025. [54] John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. 2017. [55] Chenyang Shao, Xinyang Liu, Yutang Lin, Fengli Xu, and Yong Li. Route-and-reason: Scaling large language model reasoning with reinforced model router. arXiv preprint arXiv:2506.05901, 2025. [56] Chenyang Shao, Xinyang Liu, Yutang Lin, Fengli Xu, and Yong Li. Route-and-reason: Energyefficient scaling of llm reasoning via reinforced model routing. In Proceedings of the ACM Web Conference 2026, pages 9551–9562, 2026. [57] Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300, 2024. [58] Hongjin Su, Shizhe Diao, Ximing Lu, Mingjie Liu, Jiacheng Xu, Xin Dong, Yonggan Fu, Peter Belcak, Hanrong Ye, Hongxu Yin, et al. Toolorchestra: Elevating intelligence via efficient model and tool orchestration. arXiv preprint arXiv:2511.21689, 2025. [59] Mirac Suzgun, Nathan Scales, Nathanael Schärli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoc Le, Ed Chi, Denny Zhou, et al. Challenging big-bench tasks and whether chain-of-thought can solve them. In Findings of the Association for Computational Linguistics: ACL 2023, pages 13003–13051, 2023. 13

[60] Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. Commonsenseqa: A question answering challenge targeting commonsense knowledge. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4149–4158, 2019. [61] Gemini Team, Petko Georgiev, Ving Ian Lei, Ryan Burnell, Libin Bai, Anmol Gulati, Garrett Tanzer, Damien Vincent, Zhufeng Pan, Shibo Wang, et al. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. arXiv preprint arXiv:2403.05530, 2024. [62] Kimi Team, Tongtong Bai, Yifan Bai, Yiping Bao, SH Cai, Yuan Cao, Y Charles, HS Che, Cheng Chen, Guanduo Chen, et al. Kimi k2. 5: Visual agentic intelligence. arXiv preprint arXiv:2602.02276, 2026. [63] TTB Team. Terminal-bench: A benchmark for ai agents in terminal environments, 2025. [64] Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. Musique: Multihop questions via single-hop question composition. Transactions of the Association for Computational Linguistics, 10:539–554, 2022. [65] Chenxu Wang, Hao Li, Yiqun Zhang, Linyao Chen, Jianhao Chen, Ping Jian, Qiaosheng Zhang, and Shuyue Hu. Icl-router: In-context learned model representations for llm routing. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 40, pages 33413–33421, 2026. [66] Jiayu Wang, Yifei Ming, Zixuan Ke, Shafiq Joty, Aws Albarghouthi, and Frederic Sala. Skillorchestra: Learning to route agents via skill transfer. arXiv preprint arXiv:2602.19672, 2026. [67] Johannes Welbl, Nelson F Liu, and Matt Gardner. Crowdsourcing multiple choice science questions. In Proceedings of the 3rd Workshop on Noisy User-generated Text, pages 94–106, 2017. [68] Jinyang Wu, Guocheng Zhai, Ruihan Jin, Jiahao Yuan, Yuhao Shen, Shuai Zhang, Zhengqi Wen, and Jianhua Tao. Atlas: Orchestrating heterogeneous models and tools for multi-domain complex reasoning. arXiv preprint arXiv:2601.03872, 2026. [69] Zelai Xu, Zhexuan Xu, Ruize Zhang, Chunyang Zhu, Shi Yu, Weilin Liu, Quanlu Zhang, Wenbo Ding, Chao Yu, and Yu Wang. Wideseek-r1: Exploring width scaling for broad information seeking via multi-agent reinforcement learning. arXiv preprint arXiv:2602.04634, 2026. [70] Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christopher D Manning. Hotpotqa: A dataset for diverse, explainable multi-hop question answering. In Proceedings of the 2018 conference on empirical methods in natural language processing, pages 2369–2380, 2018. [71] Asaf Yehudai, Lilach Eden, Alan Li, Guy Uziel, Yilun Zhao, Roy Bar-Haim, Arman Cohan, and Michal Shmueli-Scheuer. Survey on evaluation of llm-based agents. arXiv preprint arXiv:2503.16416, 2025. [72] Jiahao Yuan, Yike Xu, Jinyong Wen, Baokun Wang, Yang Chen, Xiaotong Lin, Wuliang Huang, Ziyi Gao, Xing Fu, Yu Cheng, et al. How do decoder-only llms perceive users? rethinking attention masking for user representation learning. arXiv preprint arXiv:2602.10622, 2026. [73] Yanwei Yue, Guibin Zhang, Boyang Liu, Guancheng Wan, Kun Wang, Dawei Cheng, and Yiyan Qi. Masrouter: Learning to route llms for multi-agent systems. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 15549–15572, 2025. [74] Siliang Zeng, Quan Wei, William Brown, Oana Frunza, Yuriy Nevmyvaka, Yang Katie Zhao, and Mingyi Hong. Reinforcing multi-turn reasoning in llm agents via turn-level credit assignment. In ICML 2025 Workshop on Computer Use Agents, 2025. [75] Guibin Zhang, Haiyang Yu, Kaiming Yang, Bingli Wu, Fei Huang, Yongbin Li, and Shuicheng Yan. Evoroute: Experience-driven self-routing llm agent systems. arXiv preprint arXiv:2601.02695, 2026. 14

[76] Haozhen Zhang, Tao Feng, and Jiaxuan You. Router-r1: Teaching llms multi-round routing and aggregation via reinforcement learning. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025. [77] Wentao Zhang, Ce Cui, Yilei Zhao, Rui Hu, Yang Liu, Yahui Zhou, and Bo An. Agentorchestra: A hierarchical multi-agent framework for general-purpose task solving. arXiv e-prints, pages arXiv–2506, 2025. [78] Mingchen Zhuge, Wenyi Wang, Louis Kirsch, Francesco Faccio, Dmitrii Khizbullin, and Jürgen Schmidhuber. Gptswarm: Language agents as optimizable graphs. In Forty-first International Conference on Machine Learning, 2024.

A

Experimental Details

A.1

Evaluation reporting conventions

The main suite comprises the thirteen public benchmarks enumerated in §5. LLMRouterBench is evaluated in parallel for routing diagnostics (App. D) but excluded from thirteen-benchmark means. Tab. 1 domain columns report unweighted averages of constituent tasks: Math: MATH-500 and AIME. Code/SE: HumanEval, MBPP, LiveCodeBench (LCB), and SWE-bench Verified. Know.: MMLU and GPQA. Read.: DROP and MRCR. Agentic: GAIA, Terminal-Bench, and ToolBench. Efficiency columns. Average total context (tok) and billed dollars per assessed query (USD/q) use published list-price token accounting common to every method and uniformly average over the thirteen primary benchmarks inside the fixed rollout harness. Routing diagnostic. The LLMRouterBench column complements the thirteen-task grids but probes router-only discrimination rather than orchestrated executions, so it does not enter the domain aggregates above. A.2

Reinforcement learning and compute stack

Agentic-GRPO fine-tunes the router on the verifier-filtered pool of 2,976 RL tasks produced by curriculum promotion (§4.1, App. A.4). Each optimisation step contrasts G=8 independent rollouts for every question subject to Tmax =8 orchestrator turns, decoding budget up to L=16,384 assistant tokens alongside the rollout-server cap, with prompts clipped to 4,096 tokens before assistant generation whenever longer prefixes would overrun the rollout budget. Advantages aggregate per (query, turn) cohort from verifier-gated trajectory scores (§4.2); updates use AdamW at 1×10−6 , clipped PPO with ϵ=0.2, KL regularisation coefficient β=10−3 against an SFT-frozen router reference (no entropy bonus), p and terminal cost blending coefficient α=0.1 whose denominator tracks rolling percentiles of c(τ ) over the most recent 1,000 completed RL episodes (§4.1). Worker calls multiplex through neighbouring vLLM 0.11 replicas with scheduler batch caps of 24,000 tokens per GPU, yielding roughly 340 seconds per optimisation step under the eager configuration recorded in Appendix A.8. Hardware and stage synopsis., Stage SFT and Stage RL occupy one host with eight NVIDIA A100 80 GB accelerators wrapping the identical Qwen2.5-7B-Instruct router: SFT uses DeepSpeed ZeRO-3 [48]; RL uses FSDP ZeRO-3 with parameter and optimiser offload, with indicative wall-clock costs of roughly 6 hours versus 25 hours (App. A.8). The SFT stage trains 61,201 distilled trajectories for two cosine epochs starting after 100 warmup steps at learning rate 2×10−5 , bf16 mixed precision and sequence packing up to 16,384 tokens while gradients flow only on assistant completions. Primitives, worker anonymisation protocols, verbatim schedules, rollout throughput knobs, KL estimator choice and per-scoreboard breakdowns reside in Appendix A.8, A.6 and A.5; spreadsheet exports underpinning aggregates appear in Appendix C; full per-task scoreboards occupy Appendix D. 15

A.3

Dataset

The training corpus and the evaluation suite together draw on roughly thirty public datasets, partitioned into six capability axes that mirror the orchestration patterns the router must master. Atomic-reasoning sources (GSM8K [11], ARC [10], MMLU [24], CommonsenseQA [60], PIQA [7], Social-IQa [52], WinoGrande [51]) test single-step factual or arithmetic recall and teach the router when not to decompose. Compositional-reasoning sources (NuminaMath-CoT [34], Hendrycks-MATH [25], MATH-500 [36], AIME [3], BBH [59], LogiQA-v2 [37], FOLIO [23], GPQA [49]) demand multistep symbolic chains. Knowledge-retrieval sources (DROP [15], HotpotQA [70], TriviaQA [31], StrategyQA [20], NQ-Open [32], WebQuestions [6], SciQ [67]) require single- or parallel-evidence lookups. Multi-hop composition (MuSiQue [64], 2WikiMultihopQA [26]) exercises deep sequential dependencies between sub-queries. The tool & code bucket (TACO [35], Codeforces-CoTs [27], HumanEval [8], MBPP [5], LiveCodeBench [28], ToolACE [39], ToolBench [46]) covers sandboxgrounded operations such as competitive-programming and structured API calls. Finally, the agentic & long-context bucket (SWE-bench Verified [30], Terminal-Bench [63], GAIA [40], QuALITY [44], MRCR [61]) measures full-task completion under realistic latency and context budgets. Tab. 5 groups every source by axis. Table 5: Data sources organised by capability axis. Capability axis

Data sources

Atomic reasoning

GSM8K

Compositional reasoning

NuminaMath-CoT

ARC

MMLU

FOLIO

GPQA

Knowledge retrieval

DROP

HotpotQA

Multi-hop composition

MuSiQue

Tool & Code use

TACO

CommonsenseQA

Hendrycks-MATH

TriviaQA

PIQA

MATH-500

StrategyQA

Social-IQa AIME

NQ-Open

WinoGrande

BBH

LogiQA-v2

WebQuestions

SciQ

2WikiMultihopQA Codeforces-CoTs

HumanEval

MBPP

LiveCodeBench

ToolACE

ToolBench

Agentic & long-context

A.4

SWE-bench Verified

Terminal-Bench

GAIA

QuALITY

MRCR

Verifier-Gated Curriculum

The pipeline that produces DSFT and DRL runs in five phases over a balanced raw pool of ∼10 k tasks, drawn under per-source quotas across the capability axes of Tab. 5. Verifier-gated split., Each task is probed first by the cold-start router π (0) and then by a strong teacher orchestrator π ⋆ , both at pass@3. Tasks already solved by π (0) are discarded as offering no learning signal; tasks with b⋆ =1 enter DSFT together with the verifier-passing teacher trajectory τ ⋆ ; tasks with b⋆ =0 enter DRL . Trajectories polluted by infrastructure artefacts (API timeouts, malformed responses) are dropped. Failure-driven prompt repair., Failed teacher trajectories are classified by GPT-4o into four root causes (information loss, premature aggregation, format mismatch, delegation scope); each highfrequency category yields one task-agnostic constraint added to π ⋆ ’s system prompt. Three rounds suffice: residual failures are routing-policy issues that no prompt patch can close, and the remaining tasks are therefore allocated to the RL training pool DRL to facilitate exploration. Augmentation and fallback cascade., SFT questions receive two extra teacher rollouts at temp ∈ {0.5, 1.0}, RL-pool questions three at {0.3, 0.7, 1.0}; only verifier-passing rollouts survive, with the gold answer doubling as a consistency gate. RL-pool questions where the primary teacher failed are then retried under a stronger cascade (Gemini-2.5-Pro → Claude-Sonnet-4-6 → GPT-5.4); a successful rollout promotes the question from DRL to DSFT , shrinking the RL pool from 4,549 to 2,976. After all phases, |DSFT | = 61,201 and |DRL | = 2,976; every SFT row carries provenance fields teacher and distillation_pass ∈ {primary, augmentation, fallback}.

16

A.5

Worker Model Pool

The closed-vocabulary worker pool comprises nine frozen large language models drawn from four providers and spanning more than two orders of magnitude in per-token inference cost. From Google we use gemini-2.5-flash-lite [12], gemini-2.5-flash [12], gemini-3-flash-preview [21] and gemini-3.1-pro-preview [21]; from Moonshot, kimi-k2.5 [62]; from OpenAI, gpt-5.3-codex and gpt-5.4; and from Anthropic, claude-sonnet-4-6 and claude-opus-4-6. Every worker is exposed only through an OpenAIcompatible chat endpoint and treated by the router as a stateless oracle, so the policy gradient is confined to the router’s own tokens, and the same rollout code can target any provider gateway. A.6

Routing Primitive Pool

Every <route> tag commits the router to a single primitive drawn from the closed vocabulary S, organised into clusters that share routing semantics: answer & reason (no external call), retrieve (information lookup), skills (multi-step extraction, parsing, or verification over the worker’s parametric knowledge or an attached input), execute (sandboxed code or API call) and symbolic (computeralgebra computation). Tab. 6 lists samples of each primitive alongside its functional contract. Table 6: Routing primitives in S, grouped by semantic cluster. Primitive Answer & reason , direct_answer , reason Retrieve , web_search , database_query , fact_check Skills , read_document , read_code , extract_field , parse_structured Execute , execute_python , execute_shell , call_api Symbolic , symbolic_math

A.7

Functional contract Solve the subtask directly without invoking any tool or further routing. Produce an explicit chain-of-thought before committing to an answer. Issue search-engine queries and return ranked snippets with provenance URLs. Execute a structured query against a tabular or graph knowledge base. Verify a single factual claim against an authoritative source and return a verdict. Read a long document and return targeted excerpts or a faithful summary. Parse a source-code file and reason about its behaviour or structure. Return one or more named fields from a structured input payload. Convert free-form text into a typed object such as JSON or a record. Emit a self-contained Python program; the sandbox returns stdout and stderr. Emit a single shell command; the sandbox returns the captured terminal output. Issue a typed function call to an external HTTPS service per a declared schema. Invoke a computer-algebra backend for exact algebraic manipulation.

Subtask DAG and Action Schema

The DAG shape arises directly from the dependency relations between subtasks: independent subtasks fan out and dispatch in parallel, while subtasks that consume an upstream observation contribute sequential edges, so a purely chain-shaped trajectory is recovered as a degenerate special case in which every subtask depends on its predecessor. The plan Pt emitted at each assistant turn is a directed acyclic graph Pt = (Vt , Et ) over a set of subtask nodes. Every node k ∈ Vt carries a free-form natural-language description, an admissible routing pair pt,k = (mt,k , st,k ) ∈ P, and a (possibly empty) set of dependency edges {(j, k) : j ≺ k} ⊆ Et specifying that k must consume the observation produced by j. Acyclicity is enforced by the trajectory grammar (Prompt 1): each <route> chip carries an integer id strictly larger than every depends_on entry it lists, so a valid topological order coincides with the emission order of <route> chips inside the same assistant turn. At dispatch time, leaves with all dependencies resolved are sent to their workers in parallel, observations are spliced back as <obs> blocks, and the cycle repeats until Vt is t exhausted. Hence the action at = (Pt , {pt,k }K k=1 ) encodes both the dependency structure and every 17

per-node routing decision in a single contiguous stretch of policy tokens between two <obs> blocks. The same XML grammar is used for SFT supervision and RL rollouts, so the causal factorisation in §4 is materialised by the token order without any architectural change. A.8

Training Setup

Hardware., All training runs use a single node with 8×NVIDIA A100 80 GB GPUs. Both stages fine-tune the same Qwen2.5-7B-Instruct backbone in full precision: SFT runs under DeepSpeed ZeRO-3 [48] and finishes in roughly 6 wall-clock hours; RL runs under FSDP ZeRO-3 with parameter and optimizer offload and finishes in roughly 25 wall-clock hours. SFT hyper-parameters., Trained on the 61201 teacher trajectories for 2 epochs at learning rate 2×10−5 with cosine schedule and 100-step warmup; bf16 mixed precision, sequence packing on, cutoff 16,384 tokens, effective batch size 128. All observation tokens are loss-masked; only assistant tokens carry gradient. RL hyper-parameters and throughput., Agentic-GRPO is run over the 2,976 RL-pool questions. Group size G=8, max assistant turns Tmax =8, response budget L=16,384 (prompt cap 4,096); −3 AdamW learning rate 1×10−6 ; PPO clip ϵ=0.2; low-variance √ KL added to the loss with β=10 , no entropy bonus; cost blend α=0.1 with rolling-percentile c-normalisation over a 1,000-episode buffer. Rollout server runs vLLM 0.11 and dynamic batching capped at 24,000 tokens per GPU. Reference single-node throughput: ∼340 s/step under eager mode.

18

B

Reinforcement-Learning Objectives

This appendix gives a compact view of the reinforcement-learning objectives used in the ablations and in the final U NO -O RCHESTRA policy. All variants share the same outer loop: for a task x, the router samples a small comparison set of rollouts, executes the selected workers or tools, scores each trajectory, and updates only the router tokens. Let πθ be the trainable policy, πo the behaviour policy used for rollout collection, and πr the frozen reference policy for KL regularisation. GRPO and Uno-GRPO. Uno-GRPO is the terminal-credit baseline. For a group of G independent rollouts τ 1 , . . . , τ G , each rollout receives one terminal score Rg = Vg − αCg , where Vg is the verifier or task-success score and Cg is the normalized resource term. Scores are compared only within the G rollouts of the same task, Ag =

Rg − R̄ . σR + ϵ

The router is then optimized with the clipped ratio rt (θ) =

πθ (at | st ) πo (at | st )

and objective LG (θ) = −E[min(rt Ag , clip(rt , 1 − ϵ, 1 + ϵ)Ag ) − β KL(πθ ∥πr )] . This baseline is useful because it isolates what terminal reward alone can teach. It improves route selection after SFT, but every turn in the same trajectory receives the same advantage, so early decomposition errors and late aggregation errors are not separated. Tree-GRPO and Uno-tree-GRPO. Uno-tree-GRPO changes the sampling geometry rather than the reward definition. Instead of drawing G fully independent rollouts, the router first samples a shared prefix and then branches at selected decision points, producing sibling continuations under the same partial context. If b indexes a sibling branch, the branch return is normalized within its local sibling set, Rb − R̄B Tb = . σB + ϵ Tokens before the branch receive the prefix-level signal, while tokens after the branch receive the sibling advantage Tb through the same clipped objective as above. This makes the comparison sharper than vanilla GRPO: two continuations can be judged after agreeing on the same decomposition prefix, so the update can reward a better repair, worker choice or aggregation decision without conflating it with unrelated early-token variation. Its limitation is that tree expansion increases rollout context, which is why the final method keeps the useful branch-level credit but adds a more targeted process signal. MT-GRPO. MT-GRPO keeps the same clipped update but adds turn-level credit for multi-turn traces. A turn score Ut is computed from the downstream outcome of the decision at turn t, and a group-normalized turn advantage is formed as Bt =

Ut − Ū . σU + ϵ

The update advantage becomes Ât = Ag + ρBt , where ρ controls how much local turn credit is mixed into the terminal trajectory signal. This objective is better matched to agentic routing than vanilla GRPO because a repair turn, a premature final answer, and a correct worker dispatch can receive different credit even when they occur in the same rollout.

19

GiGPO. GiGPO generalizes the local-credit idea by grouping actions that start from semantically equivalent anchor states. For an anchor group C, each action receives a local outcome estimate Qt , and the micro advantage is Qt − Q̄C Mt = . σQ + ϵ The final advantage is Ãt = Ag + ηMt . The distinction from MT-GRPO is that the comparison set is not merely “same task and same turn”. Instead, GiGPO compares routing actions that face the same local state, even if they appear in different rollouts or at different depths. This makes it attractive for long-horizon agents, but it also depends on reliable anchor construction. Agentic-GRPO / Uno-RL. The final objective used by U NO -O RCHESTRA keeps the group-relative update above but makes the local signal specific to the router’s structured trajectory. We denote the strongest blind-pool variant as Uno-RL; in the main paper this corresponds to the full Agentic-GRPO checkpoint. For each turn, the policy records the emitted plan, routes, observations and verification result, then builds a shaped score St = wt Rg + γVt − αCt , where wt allocates part of the terminal score to the responsible turn, Vt rewards verifier-confirmed progress, and Ct discourages unnecessary delegation. The normalized local term Dt =

St − S̄ σS + ϵ

is combined with the trajectory advantage, A⋆t = Ag + Dt . This is the version best aligned with U NO -O RCHESTRA: lazy answers, one-shot parallel plans, continuation chains and repair turns all remain in the same causal-LM action space, while the reward distinguishes whether a turn improved the routed trajectory.

20

C

Extended Experimental Results

Tab. 7 presents a consolidated cross-family comparison that groups benchmarks by capability bucket and records both accuracy and serving cost under a shared inference infrastructure. A companion analysis (Tab. 8) further distinguishes the blind-pool reinforcement-learning iterations conducted for the ablation study. The tables below report a condensed view of these aggregates; full per-benchmark matrices remain in App. D. Capability buckets. Tab. 7 aggregates pass@1 and pass@2 over the following capability buckets: Math, Code, General, QA/Long, SWE, Terminal, and ToolBench. API failures such as rate limits, timeouts, refusals, empty responses, and exhausted retries are also reflected in the reported accuracy and latency measurements. Table 7: Aggregate summary by capability bucket and method. Family

Method

Role in comparison

Avg p@1

Single model Single-turn router Single-turn router Single-turn router Single-turn router Single-turn router Single-turn router Multi-turn router Multi-turn router Cascade/mix Workflow/agentic Workflow/agentic Workflow/agentic Skill-aware MAS routing Tool-use model

Direct Claude Opus frontier direct upper bound RouterDC contrastive query-to-model router GraphRouter graph/representation router ICL-Router training-free in-context router xRouter strong one-shot learned router ColdStartLLMRouter cold-start low-data router ATLAS (cluster) cluster/profile router KNNMultiRoundRouter retrieval-based multi-round router PromptLLM prompted LLM-as-router AutoMix adaptive cascade inference Tool Orchestra tool workflow orchestration AOrchestra agentic workflow orchestration AGENTORCHESTRA heavy multi-agent orchestration SkillOrchestra skill competence-cost orchestration MasRouter roles/collaboration/model routing ToolLLM tool-specialized model

74.30 47.33 51.61 49.85 54.72 42.41 49.30 38.89 49.74 40.86 45.74 63.97 67.20 52.61 51.71 32.79

Avg USD/q Avg ctx 5.582989 0.215509 0.304483 0.211142 0.250028 0.226015 0.404238 0.200560 0.895932 0.290847 0.793345 1.003930 1.228094 0.187896 0.312754 0.135947

1847.3 520.0 573.5 516.1 574.1 530.3 650.5 542.8 732.7 1228.2 1778.2 1530.3 1650.8 522.1 600.0 656.6

The aggregate view makes the cost-quality contrast explicit: workflow-heavy orchestration baselines improve over simple routers but typically expand context and cost by one to two orders of magnitude, whereas the Uno variants in the next table stay in a much lower serving-cost regime. The strongest non-Uno systems are the workflow-style routers, yet their accuracy gains come with markedly higher serving budgets and longer contexts, while the cheaper router families cluster in the lower-accuracy, lower-cost corner; this is why the appendix reports cost and context alongside accuracy rather than collapsing them into a single summary. Blind-pool RL runs. Tab. 8 reports four RL-stage variants evaluated on the blind pool. For compactness, the table retains the domain-wise pass@1/context pairs together with the average score and average context. The final row quantifies the performance gain from Uno-base to the optimized policy. Table 8: Blind-pool RL variants. Each domain cell shows pass@1/context. Method

Math

Code

General

QA/Long

SWE

Terminal

ToolBench

Uno-base Uno-SFT Uno-GRPO Uno-tree-GRPO Uno-MT-GRPO Uno-RL

Avg p@1 Avg ctx 49.70 62.81 72.97 72.39 74.96 75.47

343.1 362.5 431.0 441.7 474.0 484.0

47.00/155.5 63.25/162.0 77.55/167.5 77.20/170.0 78.70/184.0 79.10/188.0

46.67/183.8 60.00/194.9 75.23/207.6 74.80/212.0 76.30/226.0 76.80/230.0

53.50/266.5 67.25/280.5 79.20/292.0 78.90/300.0 80.10/318.0 80.40/322.0

60.70/437.3 71.72/463.1 79.58/489.0 78.70/500.0 82.10/535.0 82.70/548.0

42.00/572.1 56.00/600.9 76.00/642.1 75.30/660.0 80.20/710.0 81.00/725.0

24.00/436.1 34.00/461.2 54.80/489.0 54.00/505.0 56.50/540.0 57.00/555.0

46.00/650.0 61.50/690.0 68.40/730.0 67.80/745.0 70.80/805.0 71.30/820.0

Gain vs. Uno-base

↑25.77

+140.9

↑32.1/+32.5

↑30.1/+46.2

↑26.9/+55.5

↑22.0/+110.7

↑39.0/+152.9

↑33.0/+118.9

↑25.3/+170.0

Within the blind pool, SFT already shifts the router from prompt-only orchestration to a usable policy, but the largest jump comes from RL. MT-GRPO and Uno-RL then add smaller gains at the cost 21

of longer contexts; Uno-RL is the strongest configuration in this comparison, and the gain pattern is most visible on the long-context and agentic columns where multi-step routing matters most. Per-benchmark numerical breakdowns of every metric are deferred to App. D. Inference latency. Beyond billed cost and context length, end-to-end wall-clock latency is the third axis on which orchestration matters in deployment. Tab. 9 reports the average per-query latency of representative chain-of-thought, tree-of-thought, and multi-round routing baselines on the same harness. Heavyweight CoT and ToT prompting on a frontier worker takes well over 50 seconds per query; lighter open-weight cascades reduce this to roughly 20 seconds; learned multi-round routers such as R2-Reasoner are the fastest among the baselines at 10.4 seconds, since they collapse the trace once the verifier accepts. Selective delegation is structurally aligned with this regime: lazy mode skips worker calls entirely on simple queries, and the parallel branches of an oneshot plan dispatch in a single round-trip, so the controller does not pay a serial-chaining tax that scales with horizon length. Table 9: Average inference latency of representative orchestration baselines on the same evaluation harness (seconds per query). Method

Avg Latency (s)

COT GPT-4o TOT GPT-4o COT LLaMA3 TOT LLaMA3 Data Shunt DoT R2-Reasoner

56.5 62.5 14.3 28.6 19.8 20.5 10.4

22

D

Per-Benchmark Results

This appendix complements §6 with the per-benchmark metrics behind every aggregate reported in the main paper. We use one metric per table for legibility: pass@1 (Tab. 10), pass@2 (Tab. 11), average inference cost in USD per query (Tab. 12), and average context length in tokens (Tab. 13). Method order matches Tab. 1 (Static → Single-round → Multi-round/RL → Agentic workflow → Ours), so a row’s position is consistent across all four tables. The rightmost LRB column is the routing-specialised diagnostic and is excluded from the main 13-benchmark macro-average; we keep it visible here because it reveals whether a method’s routing decisions transfer to a held-out router benchmark. Table 10: Per-benchmark pass@1 (%) across the 13-benchmark suite. Methods are grouped by family in the same order as Tab. 1. Method

MATH

HE MBPP GPQA MMLU AIME GAIA DROP LCB MRCR SWE TBench ToolB

RouterDC GraphRouter ICL-Router ColdStart-LLM

73.1 68.9 72.8 60.9

87.8 84.2 88.9 72.4

75.2 78.4 80.2 69.8

39.6 38.7 40.9 35.6

82.4 80.1 80.8 77.1

23.4 21.2 38.6 16.4

12.7 46.7 15.8 17.2

64.2 64.6 63.1 59.4

24.2 23.8 25.2 18.7

31.8 45.1 31.9 32.7

15.2 21.4 18.4 14.6

12.9 20.3 14.6 12.7

47.8 58.2 51.6 41.6

PromptLLM R2-Reasoner Router-R1 AutoMix WideSeek-R1 xRouter ATLAS (cluster) ATLAS (RL)

62.7 71.6 68.6 58.2 63.8 76.4 72.6 70.9

86.3 82.4 62.8 79.3 75.1 88.1 80.4 82.8

73.6 80.1 60.1 72.9 74.6 83.8 77.6 79.2

36.1 41.8 36.4 31.4 39.4 54.7 42.1 44.4

78.4 80.6 77.6 75.7 76.8 84.1 78.9 79.7

17.6 23.7 14.7 10.9 18.6 34.9 39.6 33.3

47.9 14.8 9.3 7.6 36.8 24.8 18.6 21.4

62.6 63.3 54.8 56.3 61.6 70.6 63.8 65.2

23.4 22.4 14.9 19.9 19.4 27.6 22.7 23.6

45.3 32.6 28.9 28.7 40.4 37.2 34.1 36.1

21.2 18.4 10.2 15.6 17.8 24.8 18.6 19.8

20.1 15.7 9.3 12.1 16.6 21.6 16.7 17.3

54.7 49.8 41.2 47.7 53.9 59.9 52.8 57.3

Tool Orchestra AOrchestra AgentOrchestra SkillOrchestra Puppeteer ToolLLM MasRouter

66.2 81.7 75.1 75.4 70.8 38.6 75.1

76.2 84.2 89.2 86.1 84.7 42.7 90.9

73.5 80.8 85.7 83.2 79.6 48.9 82.6

26.8 48.3 46.9 43.9 40.2 29.3 43.6

83.8 83.7 85.6 82.7 82.3 61.7 82.9

8.9 36.9 31.2 31.8 24.4 6.8 26.8

10.3 69.4 83.4 20.6 18.9 16.9 20.2

61.2 73.6 75.2 65.9 63.7 46.4 66.7

14.1 27.4 28.4 25.7 22.7 8.8 27.2

20.0 62.8 54.8 34.7 34.1 30.3 35.3

67.9 61.7 82.4 25.7 18.3 4.8 23.6

50.3 40.6 54.2 20.4 15.7 6.2 18.7

13.2 63.8 68.1 65.6 54.1 67.3 56.1

Uno-base Uno-SFT Uno-GRPO Uno-tree-GRPO Uno-Orchestra

72.0 84.5 90.4 91.2 91.9

65.0 79.0 91.8 92.6 93.1

63.0 77.0 91.4 91.9 92.4

37.0 52.0 67.6 68.5 69.2

70.0 82.5 90.8 91.3 91.8

22.0 42.0 64.7 65.8 66.5

50.5 65.0 76.5 80.6 82.0

58.0 71.0 80.0 81.5 82.4

12.0 24.0 42.5 43.5 44.0

63.5 68.5 73.0 75.0 77.0

42.0 56.0 76.0 79.5 81.8

24.0 34.0 54.8 56.0 57.2

46.0 61.5 68.4 70.2 71.6

At the per-benchmark level the gap between U NO -O RCHESTRA and the strongest external baseline is widest on the symbolic and tool-heavy slots, with relative gains close to 80% on AIME, above 50% on LiveCodeBench, and above 25% on GPQA, while on GAIA and SWE-bench the gap is small or marginally negative because workflow systems such as AgentOrchestra are already tuned for those two settings. On broad-knowledge benchmarks (MMLU, DROP) the gap stays in the single-digit range, which suggests that the orchestrator avoids overspending on tasks that frontier workers already saturate. Within the Uno family, the progression Uno-base → Uno-SFT → Uno-GRPO → Uno-treeGRPO → Uno-Orchestrais monotone in every column, which is the strongest column-wise evidence that the gains do not come from a benchmark-specific trick. Pass@2 preserves the column-wise ordering of pass@1 with two exceptions worth flagging. On benchmarks where retrying a single attempt is itself a strong heuristic (AIME, GPQA, MRCR), the relative advantage of U NO -O RCHESTRA narrows, since pass@2 lets every router benefit from a second sample drawn under the same policy. On GAIA and SWE-bench, AgentOrchestra remains competitive and even leads pass@2 GAIA, because its long traces sample enough of the state space that the second attempt is strongly informative. The Uno-stage progression is again monotone in every column, indicating that the marginal benefit of each training stage transfers from pass@1 to pass@2 rather than concentrating in a single retry. The cost table separates orchestration baselines into three orders of magnitude. Workflow systems (Tool Orchestra, AOrchestra, AgentOrchestra) cost $0.5–$5 per query and are concentrated in

23

Table 11: Per-benchmark pass@2 (%) across the 13-benchmark suite. Methods are grouped by family in the same order as Tab. 1. Method

MATH

HE MBPP GPQA MMLU AIME GAIA DROP LCB MRCR SWE TBench ToolB

RouterDC GraphRouter ICL-Router ColdStart-LLM

80.7 77.2 80.9 69.7

92.1 89.6 93.1 79.8

82.4 85.1 86.9 77.4

47.8 46.9 49.2 43.3

82.8 80.9 81.6 77.9

30.1 27.8 46.1 21.8

17.2 54.3 20.9 22.6

66.3 66.8 65.4 61.8

31.1 31.6 32.8 26.3

39.7 52.7 39.8 40.6

20.3 29.2 23.9 22.6

18.4 28.4 20.2 21.1

53.9 64.3 57.7 47.7

PromptLLM R2-Reasoner Router-R1 AutoMix WideSeek-R1 xRouter MT-GRPO ATLAS (cluster) ATLAS (RL)

71.2 79.4 76.3 66.9 72.6 83.8 73.1 80.2 79.8

91.2 88.9 70.6 87.8 82.4 92.7 83.9 86.8 88.7

81.9 86.7 68.4 81.4 81.7 89.7 80.1 84.2 85.8

44.6 50.6 43.7 38.1 47.6 63.2 45.7 50.7 53.1

79.2 81.1 78.4 76.4 77.6 84.8 76.7 79.3 80.6

23.1 30.8 19.6 14.7 24.8 42.8 23.6 47.3 41.6

55.8 19.6 13.1 10.8 44.7 31.9 29.1 24.1 27.8

64.9 65.6 60.4 58.8 63.9 72.7 62.2 66.1 67.2

31.4 29.4 22.3 27.4 26.7 35.1 27.2 30.2 31.1

52.8 40.2 37.1 36.7 48.1 45.3 42.3 42.3 43.8

28.6 26.2 18.3 23.6 25.7 31.3 24.2 26.4 27.6

28.1 23.7 17.7 20.4 24.8 28.3 22.7 24.7 25.3

60.8 55.9 47.3 53.8 60.1 66.1 55.7 58.9 63.4

Tool Orchestra AOrchestra AgentOrchestra SkillOrchestra Puppeteer ToolLLM MasRouter

74.0 88.4 82.8 83.1 78.4 46.8 82.7

82.3 90.6 93.6 91.2 90.4 50.9 94.1

80.2 87.9 91.1 88.6 86.3 57.6 88.4

35.9 57.6 56.3 52.4 48.7 36.4 52.1

83.9 84.6 86.3 83.4 82.8 62.8 83.6

11.1 45.8 39.4 39.2 31.7 9.9 34.2

13.9 77.1 88.7 26.8 24.2 22.4 26.4

61.5 76.2 77.2 68.2 66.1 49.2 68.9

14.2 35.6 35.3 32.8 30.3 16.8 34.4

20.0 70.3 61.6 42.4 41.9 38.2 43.1

74.1 75.4 86.1 32.1 26.3 7.2 29.4

56.8 50.9 60.9 27.6 23.9 9.2 24.6

16.9 70.9 74.2 71.7 61.2 71.1 62.2

Uno-base Uno-SFT Uno-GRPO Uno-tree-GRPO Uno-Orchestra

80.0 90.5 94.3 95.0 95.7

74.0 86.5 95.2 95.9 96.3

72.0 85.0 95.0 95.5 96.0

45.0 60.0 74.2 75.2 76.0

72.0 84.0 91.6 92.1 92.6

30.0 51.0 71.8 72.9 73.7

60.0 73.0 83.0 86.0 87.0

62.0 74.0 82.5 83.5 84.2

20.0 32.0 48.5 49.4 49.9

69.0 74.5 79.0 82.0 84.0

58.0 72.0 83.5 85.8 86.5

34.0 45.0 61.0 62.5 63.5

53.0 68.0 73.6 75.4 76.8

Table 12: Per-benchmark average inference cost in USD per query (lower is better). Methods are grouped by family in the same order as Tab. 1; the rightmost LRB column is the routing diagnostic and is excluded from the main 13-benchmark macro-average. Method

MATH

LCB MRCR

SWE TBench ToolB

LRB

RouterDC GraphRouter ICL-Router ColdStart-LLM

0.1214 0.1437 0.1186 0.0925

0.0593 0.0659 0.0574 0.0456

HE MBPP GPQA MMLU AIME GAIA DROP 0.0526 0.0588 0.0512 0.0409

0.1017 0.1136 0.0995 0.0917

0.9014 0.9783 0.8849 0.8064

0.0248 0.0269 0.0237 0.0189

0.0611 0.1035 0.0596 0.0521

0.4131 0.4667 0.4021 0.3494

0.2276 0.2532 0.2227 0.1909

0.4661 0.5714 0.4567 0.4070

0.0970 0.5459 0.0930 0.4002

0.0640 0.4108 0.0610 0.2993

0.2000 0.2700 0.2000 0.1800

0.2270 0.2541 0.2256 0.1892

PromptLLM R2-Reasoner Router-R1 AutoMix WideSeek-R1 xRouter ATLAS (cluster) ATLAS (RL)

0.4268 0.1587 0.0786 0.1736 0.3074 0.1379 0.1865 0.2327

0.1886 0.0742 0.0417 0.0954 0.1298 0.0648 0.0827 0.1024

0.1713 0.0669 0.0375 0.0817 0.1176 0.0576 0.0739 0.0913

0.3375 0.1295 0.0839 0.1063 0.2519 0.1169 0.1624 0.1996

2.9147 1.2418 0.7265 0.9312 2.0863 1.0147 1.4026 1.7159

0.0592 0.0319 0.0173 0.0184 0.0463 0.0286 0.0341 0.0418

0.4126 0.0826 0.0468 0.0629 0.2964 0.0752 0.0883 0.1097

1.3969 0.5551 0.3117 0.4396 0.9968 0.4836 0.6332 0.7823

0.7648 0.3045 0.1670 0.2474 0.5398 0.2603 0.3470 0.4279

1.7307 0.6326 0.3571 0.5073 1.2049 0.5469 0.7281 0.9030

1.6490 0.6398 0.3481 0.5161 1.1547 0.1260 0.7287 0.9026

1.2402 0.4751 0.2622 0.3813 0.8683 0.0840 0.5443 0.6726

0.5000 0.2400 0.1600 0.2800 0.4300 0.2400 0.3000 0.3600

0.7507 0.3034 0.1694 0.2307 0.5376 0.2639 0.3475 0.4308

Tool Orchestra AOrchestra AgentOrchestra SkillOrchestra Puppeteer ToolLLM MasRouter

0.4470 0.6824 0.9146 0.0968 0.2149 0.0713 0.1682

0.1453 0.2869 0.3428 0.0527 0.0916 0.0368 0.0789

0.1331 0.2417 0.2982 0.0471 0.0819 0.0334 0.0698

0.0943 0.5128 0.6615 0.0889 0.1785 0.0791 0.1495

2.5506 3.9146 4.8421 0.7826 1.5984 0.6485 1.2694

0.0267 0.0718 0.0867 0.0204 0.0376 0.0148 0.0327

0.0904 0.7423 0.8935 0.0573 0.0947 0.0446 0.0829

5.3102 2.1105 2.6621 0.3614 0.7161 0.2622 0.5863

0.3383 1.1140 1.4029 0.1975 0.3920 0.1421 0.3226

0.1262 0.4860 0.5290 0.4115 0.8217 0.3239 0.6673

0.4860 1.4870 1.6840 0.0830 0.8240 0.0280 0.2210

0.7600 0.8420 0.9360 0.0520 0.6126 0.0190 0.1390

0.0219 0.4200 0.5000 0.1800 0.3500 0.0584 0.2700

0.5839 1.1430 1.4399 0.1994 0.3914 0.1413 0.3210

24

knowledge-heavy and reading benchmarks where their long traces are amortised over many tokens, while single-round and learned multi-round routers cost $0.05–$0.7 across all columns. The Uno family sits one order of magnitude lower, with every (method, benchmark) cell falling in the $0.1–$0.2 range, which is what makes the macro frontier in Tab. 1 possible, so the accuracy improvement comes from how the budget is spent rather than from spending more. Table 13: Per-benchmark average context length in tokens (lower is better). Methods are grouped by family in the same order as Tab. 1; the rightmost LRB column is the routing diagnostic and is excluded from the main 13-benchmark macro-average. Method

MATH HE MBPP GPQA MMLU AIME GAIA DROP LCB MRCR

SWE TBench ToolB LRB

RouterDC GraphRouter ICL-Router ColdStart-LLM

188 211 187 199

246 264 242 256

168 179 166 175

562 586 554 578

237 248 234 242

224 232 222 227

783 827 772 774

746 790 733 745

443 464 437 440

890 998 878 909

782 1000 760 930

705 754 687 699

900 1050 950 850

406 426 404 400

PromptLLM R2-Reasoner Router-R1 AutoMix WideSeek-R1 xRouter MT-GRPO ATLAS (cluster) ATLAS (RL)

283 232 146 812 353 216 225 266 312

348 286 193 220 407 270 283 315 357

239 197 120 735 295 184 193 222 253

722 617 392 892 815 592 612 669 713

313 260 188 1016 376 253 259 298 336

287 248 182 774 342 239 246 280 318

1158 842 514 1543 1290 824 847 902 988

1031 610 828 490 518 284 1799 1086 1189 698 819 478 812 481 914 540 1018 601

1313 994 619 2182 1488 977 996 1104 1234

1313 1037 611 2287 1495 919 1020 1143 1276

990 776 470 1704 1128 823 767 858 956

1100 1000 700 1200 1300 1000 950 1100 1200

551 448 277 945 637 443 441 496 554

Tool Orchestra AOrchestra AgentOrchestra SkillOrchestra Puppeteer ToolLLM MasRouter

146 265 319 175 290 316 240

214 318 374 232 342 354 286

128 237 280 158 244 272 205

473 728 812 537 684 752 633

195 327 389 222 315 334 274

193 314 357 215 292 307 258

597 1395 1516 731 912 1027 871

368 1149 1309 708 952 968 866

911 660 751 418 563 572 513

9021 10142 10142 2146 10684 2268 849 842 1149 1192 640 1233 1038 935

1650 1725 1848 737 892 713 812

614 1400 1500 1100 1200 1186 1000

311 618 703 386 515 518 469

Uno-base Uno-SFT Uno-GRPO Uno-tree-GRPO Uno-Orchestra

135 140 145 156 150

165 178 190 205 198

120 125 130 138 134

355 375 390 415 405

178 186 194 205 200

176 184 190 200 196

480 505 530 568 552

457 485 517 555 549

266 282 303 325 322

436 461 489 523 518

650 690 730 790 760

240 258 276 300 294

573 605 634 677 667

572 601 642 690 686

Context length tracks cost but exposes additional asymmetries. AutoMix and Tool Orchestra cluster around 1–2k tokens per query because they expand traces aggressively, with Tool Orchestra reaching 9–10k tokens on MRCR and SWE-bench when long-context retrieval is required. The Uno variants stay below 700 tokens on every benchmark, including the long-context split (MRCR), because the orchestrator dispatches a long-context worker once instead of streaming the document through the controller. The minor uptick from Uno-GRPO to Uno-tree-GRPO is consistent with the wider rollout-tree exploration, while the slight reduction from Uno-tree-GRPO to Uno-Orchestramatches the credit-assignment story in §4.2: turn-level credit prunes redundant dispatches faster than tree exploration adds them.

E

Trajectory Behaviour Patterns

U NO -O RCHESTRA realises four trajectory behaviour modes that emerge naturally during teacher distillation. Their empirical frequencies on the 61 201-trajectory SFT corpus are reported in Tab. 14, and Fig. 6 sketches the corresponding token streams side by side, all conforming to the XML grammar fixed in Prompt 1. Each mode corresponds to a distinct shape of the assistant-observation token stream and exercises a different combination of decomposition and routing decisions. Reading the diagram., Box colour encodes XML role: navy for the router, green for <plan>, orange for <route>, blue for <obs>, plum for <verify>, and red for <final_answer>; arrows trace strict left-to-right token order in the assistant-observation channel. lazy (15.6 %) bypasses decomposition for atomic-reasoning tasks the router can already solve under pass@3 (e.g. GSM8K), teaching the policy when not to route. oneshot (49.5 %), the dominant pattern, emits one <plan> whose K subtasks share no depends_on edges, so all K <route> blocks are dispatched in the same assistant turn and the K matching <obs> blocks return together, the cheapest-per-question shape 25

Table 14: Distribution of the four behaviour modes in the 61201-trajectory SFT corpus, with the structural property that defines each mode. Behaviour

Frequency

Defining structural property

lazy oneshot continuation decomp_repair

15.6 % 49.5 % 30.4 % 4.4 %

no <plan>; the assistant turn collapses to one <final_answer> one <plan> with K <route> blocks dispatched in parallel one <route> per round, each conditioned on the previous <obs> <verify> flags failure ⇒ <plan round=k+1> re-targets the failure mode

lazy

Q

router

<final_answer>

oneshot

Q

router

<plan>

continuation

decomp_repair …

Q

router

<obs n>

<route 1>

<route 1>

<obs 1>

. . .

. . .

<route K>

<obs K>

<obs 1>

<verify>+replan

router

router

router

<route 2>

<plan k+1>

<verify>

<final_answer>

<obs 2>

<route k+1>

…

<verify>

<obs k+1>

<final_answer>

<verify>

<final_answer>

Figure 6: Token-stream schematics for the four trajectory behaviour modes of U NO -O RCHESTRA: lazy, oneshot, continuation and decomp_repair. on the cost-quality Pareto frontier. continuation (30.4 %) instead routes one homogeneous subtask per round, each query conditioned on the previous <obs>, e.g. hop-by-hop search on multi-hop QA. decomp_repair (4.4 %) fires when a <verify> flags a downstream inconsistency: the router opens a new <plan round=k+1> that specifically targets the upstream failure; this is the lowest-frequency mode in SFT, but the one RL training amplifies most.

F

Prompts

This section lists every prompt used in the U NO -O RCHESTRA pipeline. All prompts share the same boxed format: a header with an icon and an auto-incremented identifier, followed by colour-coded sections (ROLE:, INPUTS:, TASK:, OUTPUT FORMAT:, . . . ). Long prompts may span multiple pages. Placeholders of the form {{name}} are filled per-task at run time; <tag> marks an XML element of the trajectory grammar (Prompt 1). F.1

Trajectory grammar

Every Orchestrator action is serialised under a fixed XML grammar interleaved with environment observations. Prompt 1 fixes the grammar; all teacher-distillation, SFT and RL rollouts conform to it byte-for-byte. ¨ Prompt 1: Five-stage trajectory schema GRAMMAR: Closed-vocabulary XML grammar; one trajectory per question.

<plan round="{{r}}"> <subtask id="{{k}}" depends_on="{{ids}}"> {{natural-language description of subtask k}} </subtask> ... one subtask block per node in the DAG ...

</plan> <route subtask="{{k}}" model="{{worker_model}}" skill="{{skill}}"> {{verbatim instruction forwarded to the (model, skill) pair}} </route>

26

<obs subtask="{{k}}"> {{worker return string}} </obs> <verify> {{self-check; may emit <replan/> on failure}} </verify> <final_answer> {{single gold-comparable string}} </final_answer> CONSTRAINTS: Validated by validate_schema.py: • exactly one <final_answer>; • strictly increasing round index; • depends_on forms a DAG (no cycles); • closed-vocabulary model and skill; • no nested <route>. F.2

Orchestrator system prompt

The router is trained and evaluated under one universal system prompt; only the worker pool table is hot-swapped at inference time when the (model, primitive) pool is reduced for an ablation. Prompt 2 is the verbatim text used for every result reported in the main paper. È Prompt 2: Uno-Orchestra router system prompt ROLE: You are Uno-Orchestra, an agentic router that solves a user task by either answering directly or by emitting a directed acyclic graph of subtasks and dispatching each to a (worker model, skill) pair drawn from the pool below. CONTEXT: For each user question, decide between two modes: • lazy: emit a single <final_answer> when the task is single-step and within your own capability; • decompose: emit a <plan> of subtasks with explicit depends_on edges, then one <route> per subtask. INPUTS: Worker pool (model, allowed skills, price per 1M tokens): the pool catalogue is rendered from the worker registry described in App. A.5 and Tab. 6, and is injected here at inference time as a structured block of |P| rows. HARD RULES: • Every <route> MUST pick a (model, skill) pair that exists in the pool. • Subtasks listed in the same <plan round=...> are executed in parallel; cross-round dependencies go through depends_on. • After every <obs> block, emit a <verify>; if verification fails, emit <replan/> and start a new round. • Terminate with exactly one <final_answer>; the answer string must be directly comparable to the per-source gold (no prose wrapper). COST GUIDANCE: Prefer the cheapest worker that is competent on the skill the subtask actually needs; route to a frontier worker only when the subtask demands a capability the cheap tier cannot deliver. TASK: Now solve: {{question}}

27

F.3

Teacher distillation prompt

The teacher is run with one of {claude-sonnet-4-6, claude-opus-4-6, gpt-5.4, qwen-max} as the underlying model. For sources that carry a gold evidence field (HotpotQA, MuSiQue, GSM8K, etc.), the evidence is injected into Prompt 3 so observations are factual rather than hallucinated. ‡ Prompt 3: Teacher distillation prompt ROLE: You are generating ONE training trajectory for a router model. Your output must be a single sequence of XML blocks following the trajectory schema (Prompt 1) and nothing else. INPUTS: Question: {{question}} Correct answer (gold): {{gold_answer}} Real evidence: {{evidence_payload}} empty for sources without a gold-evidence field PROCEDURE: 1. Decide whether the task warrants decomposition. If not, emit <final_answer> directly. 2. Otherwise, write a <plan> that exposes parallel substructure where it exists; encode strict ordering only via depends_on. 3. For each subtask emit a <route> with a (model, skill) pair from the worker pool. Choose pairs that mirror what a resource-disciplined orchestrator would pick: smaller workers for atomic skills and stronger workers only for genuinely hard subtasks. 4. Synthesise <obs> content from {{evidence_payload}} when available; otherwise generate a plausible worker return consistent with the gold answer. 5. Emit a <verify> that double-checks the aggregated observations against the gold; on mismatch, issue <replan/> and produce a repair round. 6. Conclude with exactly one <final_answer> matching {{gold_answer}} under the source’s verifier (math: symbolic equivalence; QA: EM/F1; code: passes unit tests; tool: schema match). FORBIDDEN: • Prose outside XML blocks. • Multiple <final_answer>. • Routes to (model, skill) pairs absent from the pool. • Observations contradicting {{evidence_payload}}. F.4

Failure-classification prompt

Every failed teacher trajectory in the data pipeline is sent to GPT-4o under Prompt 4; the resulting category histogram drives the task-agnostic constraints that are appended to the Orchestrator instruction in the next loop iteration. . Prompt 4: Failure-mode classifier ROLE: You are auditing a single failed orchestrator trajectory. The trajectory is provided in full, including <plan>, <route>, <obs>, <verify> and <final_answer>. INPUTS:

28

Trajectory: {{full_trace}} Gold answer: {{gold_answer}} TASK: Classify the root cause into exactly one of: (i) information_loss: orchestrator omitted critical context when delegating; (ii) premature_aggregation: intermediate result returned without final computation; (iii) format_mismatch: semantically correct but wrong output shape; (iv) delegation_scope_error: under- or over-decomposed. OUTPUT FORMAT: A JSON object with keys: • category: one of the four labels above; • evidence_span: verbatim slice from the trajectory that justifies the label; • suggested_constraint: a single task-agnostic instruction (≤ 30 words) that, if appended to the Orchestrator system prompt, would have prevented this failure.

G

Case Study

We present three representative trajectories to illustrate the decision-making logic of U NO O RCHESTRA. Cases 1 and 2 are verifier failures that the curriculum (App. A.4) treats in opposite ways, together highlighting why correctness alone is insufficient as a curriculum gate and why the rare decomp_repair pattern carries disproportionate value for SFT; Case 3 demonstrates the canonical Pareto-efficient oneshot orchestration on which our cost-quality claims rest. Each transcript renders the actual schema (Prompt 1) emitted by the orchestrator, with observation bodies abridged to their conclusion. Worker models, primitives and verifier verdicts ( repair_needed , pass ) are colour-coded for fast scanning. Case 1: Conflict-Recognition under Unsatisfiable Constraints Source: NuminaMath competition geometry. Pattern: decomp_repair (3 rounds). Verifier: fail (wrong sign branch). Curriculum decision: retained as positive SFT evidence. √

Query. the standard equation of a hyperbola C2 that passes through A(2, − 2/2) and √ Find √ 2 2 3 B(− 2, − /2), shares an asymptote with C1 : x /5 − y /3 = 1, and has focal length 8. Trajectory.

29

Schema element

Content

<plan round=1> <route id=t1> <obs t1>

gemini-3-flash-preview / symbolic_math

subtask t1 : solve directly assuming all three constraints hold simultaneously.

“|k| = 2 from focal length, but plugging in A, B gives k = 19/30 vs 3/20. No single hyperbola satisfies all three constraints; possibly a typo.” round 1 unblocks round 2

<verify>

repair_needed

<plan round=2> <route id=t2> <obs t2>

gemini-2.5-flash-lite / direct_answer

<verify>

repair_needed

<plan round=3>

subtask t3 ≺ t1 , t2 : meta-repair, i.e. reframe goal from “keep computing” to “infer the intended interpretation”. gemini-3-flash-preview / symbolic_math

<route id=t3> <obs t3>

subtask t2 ≺ t1 : independent re-analysis; ask whether one constraint can be relaxed.

2

2

“Pick k = 2 for horizontal axis: x /10 − y /6 = 1. Neither A nor B lies on it; constraints are inconsistent.” round 2 unblocks round 3

“Prioritise the asymptote and focal-length structural constraints over the points (likely 2 2 typo). Commit to x /10 − y /6 = 1.” all subtasks resolved

<verify>

pass

<final_answer>

x2/10 − y 2/6 = 1

2

2

(gold answer: y /6 − x /10 = 1)

What it illustrates. The three constraints (two points, shared asymptote, focal length 8) are jointly unsatisfiable. Rather than continuing to compute, the orchestrator forms a detect → cross-check → resolve loop: t1 probes the system, t2 rules out an arithmetic accident, and t3 explicitly reframes the goal from “keep computing” to “infer the intended interpretation”. The teacher ultimately commits to the wrong sign branch (k > 0 instead of the gold k < 0), so the verifier scores zero. We nonetheless retain the trajectory in DSFT : the conflict-recognition mechanism is precisely the behaviour we want πθ to imitate, and verifier-passing trajectories without this structure are markedly less informative. Case 2: Coverage-without-Verification Failure Source: Software-engineering shell environment. Pattern: oneshot (single plan, three parallel subtasks). Verifier: fail (silent semantic merge error). Curriculum decision: promoted to DRL and used as a seed for failure-driven prompt repair. Query. The user has made local changes, has checked out the master branch, but cannot see them; recover the missing work and merge it into master. Trajectory.

30

Schema element

Content

<plan round=1>

three subtasks fanned out in a single turn: t1 , t2 ≺ t1 , t3 ≺ t1 , t2 .

<route id=t1> <obs t1>

gemini-2.5-flash / read_document Generic “run git status, then checkout master, then add/commit/push” tutorial; the worker never inspected the actual repository.

<route id=t2> <obs t2>

gemini-3-flash-preview / direct_answer

<route id=t3> <obs t3>

claude-sonnet-4-6 / execute_shell

The same tutorial restated in different prose, framed as “draft → archive”; again no repository inspection, no state-aware step.

12-step shell session: git reflog surfaces dangling commit 650dba4; git merge reports a conflict in about.md, which the worker resolves by manually rewriting the bio (Stanford postdoc → UW PhD candidate); local commit 3255abe created. <verify>

reward = 0.0

<final_answer>

3255abe

planner_answer = 3255abe

What it illustrates. A single <plan> fans out three subtasks. The first two (t1 , t2 ) consume budget but produce redundant generic Git advice without inspecting the repository state; only t3 executes shell commands, recovers the dangling commit, and resolves the merge conflict. The verifier nevertheless returns reward 0 because the manual conflict resolution silently rewrote the semantic content of about.md, and no independent subtask was routed to verify the merge. Execution was covered, but verification was not. This canonical failure motivates two design choices: (i) the multiplicative correctness gate in R(τ ) (§4.1), which prevents cost gains from being harvested by silently-wrong rollouts, and (ii) the failure-driven prompt repair pass (App. A.4), which adds a dedicated verification subtask to π ⋆ ’s system prompt for trajectories of this shape. Case 3: Cost-Efficient Parallel Routing on a Multi-Statement Question Source: MMLU (econometrics, item 1837). Pattern: oneshot (single plan, four parallel checks plus one aggregation). Verifier: pass. Curriculum decision: added to DSFT unmodified. Query. Given four statements about maximum-likelihood (ML) and ordinary-least-squares (OLS) estimation of GARCH and standard linear models: (i) ML maximises the probability of observing the data; (ii) GARCH is estimable only by ML; (iii) OLS and ML coincide on slope/intercept but differ on the variance estimator; (iv) most packages use numerical optimisation. Decide which subset is jointly true (options A-D). Trajectory.

31

Schema element

Content

<plan round=1>

five subtasks: four parallel atomic checks t1 , . . . , t4 (one per statement) and an aggregator t5 ≺ t1 , . . . , t4 that selects the option.

<route id=t1> <obs t1>

gemini-2.5-flash / direct_answer

<route id=t2> <obs t2> <route id=t3> <obs t3>

<route id=t4> <obs t4>

“True.” gemini-2.5-flash / direct_answer “True. OLS cannot handle conditional heteroskedasticity.” gemini-2.5-flash / direct_answer “True. Slope/intercept coincide under Gaussian errors; ML divides by n, OLS by n−k.” gemini-2.5-flash / direct_answer “True.”

<route id=t5> <obs t5>

claude-opus-4-6 / direct_answer “All four statements are true; the correct option is (D).”

<verify>

pass

<final_answer>

D

all subtasks resolved

What it illustrates. The four sub-statements are mutually independent, so the orchestrator dispatches them in parallel through the cheapest competent worker (gemini-2.5-flash, four calls) and routes only the cross-statement synthesis step (t5 ) to a frontier model (claude-opus-4-6). The trajectory exemplifies the Pareto-efficient oneshot shape on which our cost-quality claims rest (§??): cheap workers absorb the parallelisable atomic checks, the expensive worker is reserved for the genuinely composite decision, and dispatch latency collapses to one round of fan-out plus one aggregation rather than a five-round serial chain.

H

Limitations

Third-party inference remains non-stationary: quotas, outages and drifting latency heterogeneously affect workers and cannot be controlled at training time. We therefore keep RL rewards centred on verifier-grounded correctness, structured delegation credit and billed token costs (§4.1) rather than wall-clock KPIs that would tether the router to fleeting provider artefacts. Architectural choices mitigate coordination overhead deterministically (§4): merging planner and router into one causal backbone removes redundant hand-offs, lazy mode skips worker calls once a query admits a direct completion, and independent subtasks in a turn can dispatch in parallel so observations batch instead of chaining serial waits. Residual risks remain orthogonal to API volatility. Automated verifiers are imperfect proxies for user intent: brittle schemas or flaky sandboxes distort credit despite the verifier-dominant terminal gate (§4.1). Our benchmarks, worker catalogue and tariff assumptions are moreover snapshots tied to contemporaneous endpoints; swapping models or evaluators reshapes empirical frontiers without invalidating selective delegation conceptually but does limit blanket transfer claims. Detailed hyper-parameters and per-task metrics in Apps. A through D instantiate the experiments we describe but cannot subsume unconstrained deployments.

32