T HINK S HORT, D EFER S MART, ACT, AND R EPEAT: C ALIBRATED R EASONING AND U NCERTAINTY-AWARE D EFERRAL FOR E DGE LLM AGENTS Amirmohammad Farzaneh Osvaldo Simeone Institute for Intelligent Networked Systems (INSI) Northeastern University London London, UK {a.farzaneh,o.simeone}@nulondon.ac.uk
arXiv:2607.26865v1 [stat.ML] 29 Jul 2026
A BSTRACT LLM agents following the ReAct paradigm (Yao et al., 2023) are promising enablers of complex multi-step tasks, including multi-hop question answering, code generation, and control of physical AI systems. Yet, when deployed at the edge, they must tightly manage their reasoning budget while remaining reliable and deferring to a cloud-side model only when local uncertainty is too high to act safely. We propose Think Short, Defer Smart (TSDS), a framework that synergistically integrates a lightweight convergence probe, which halts on-device reasoning once the intended action has stabilized, with a perplexity-based deferral rule that escalates uncertain actions to a cloud-side model. Both mechanisms are jointly calibrated on end-to-end episode trajectories via a multi-objective Learn-Then-Test (LTT) procedure, providing simultaneous finite-sample guarantees on expected episode reward and cloud-call rate. We evaluate TSDS on four ReAct benchmarks spanning arithmetic reasoning (GSM8K), multi-hop question answering (HotpotQA), code generation (MBPP), and multi-step embodied planning (household robot), and compare against thought-calibration-only and calibrated-deferralonly standalone baselines. TSDS reduces per-episode thinking compute by 43%– 73% over deferral-only baselines across HotpotQA, MBPP, and the household robot task, while maintaining certified reward and cloud-call rate guarantees.
1
I NTRODUCTION
Autonomous agents powered by large language models (LLMs) are increasingly deployed in resource-constrained settings, from robotic manipulation and autonomous navigation (Ahn et al., 2022; Shridhar et al., 2021) to edge network management and industrial process control (Xu et al., 2024; Chevalier-Boisvert et al., 2023). In these settings, agents must operate on local hardware with limited compute, memory, and power, while remaining reliable. ReAct (Yao et al., 2023) agents, which alternate free-text thoughts with executable actions, serve as the backbone of such agentic systems (Shinn et al., 2023; Zhou et al., 2024; Sun et al., 2023). Thought generation consumes the majority of inference compute per step (Wu et al., 2025; Guo et al., 2025). However, even a wellreasoned action from an edge-side model can be wrong, and errors compound over the episode, making deferral to a cloud-based larger model necessary (Piatrashyn et al., 2026). In settings where cloud queries carry latency and bandwidth costs, an agent must therefore maintain reliability, while minimizing wasted reasoning compute and cloud-based calls. As outlined in Fig. 1, our work addresses this problem by building on the following observations. First, in reasoning models, thinking may continue well beyond the point where a decision on intended action has converged. Esakkiraja et al. (2026) show that the committed action is encoded by hidden representations well before the end of the reasoning trace, implying that many generated tokens are pure post-convergence waste. To make matters worse, as exemplified in Fig. 1, excessive thinking may lead the model to become increasingly overconfident in the selection of a suboptimal action. This complicates cloud deferral mechanisms, which are often based on the edge model’s internal uncertainty signals (Shridhar et al., 2021; Chevalier-Boisvert et al., 2023). Overall, a joint 1
Probe φ( ⋅ ) Don’t overthink!
Probe φ
Don’t overthink!
Action
Low Low
Action
Action
High
Action
High
Edge agent π
Don’t overthink!
edge Probe φ
Edge agent π edge
LowObservation
Cloud model π
cloud
Action
Cloud model π cloudAction
(a) High
Environment Environment
Observation Ot
Prompt X
PPL Cloud model π cloud Edge agent π edge Probe φ Observation Generate No Take action Thought Yes thought token Uncertain? edge converged?
At
τt,l
No
Environment
Environment St
Reward r
Yes
Thought convergence edge Edge agent πprobe Thought convergence threshold λT
Uncertainty
Thought convergence probe
Deferral threshold λD Thought convergence Thought convergence probe threshold λ
Large model π cloud
Take action Decision A cloudmade! t
about (b) Uncertain decision
T
T
Thought convergence threshold λL Uncertainty Uncertainty
Decision made!
Decision made!
Uncertain about decision
Uncertain about decision
Deferral threshold λD
Deferral threshold λD
LT
Certain about decision (but wrong) Thought tokens Certain about decision (but wrong) Certain about decision
(but wrong)
Thought tokens
(c)
Figure 1: TSDS pipeline as a high-level illustration (a) and detailed pipeline (b): An edge agent π edge interacts with the environment across a number of time steps t = 1, 2, . . . At each step t, an action is taken after a reasoning phase, obtaining an observation from the environment. In the proposed TSDS, a convergence probe φ monitors the hidden reasoning state and halts generation once the intended action has stabilized as gauged via a threshold λL . Then, an uncertainty measure on the chosen action, such as the PPL, is calculated: if it is higher than a threshold λD , the decision or Acloud , on the action is deferred to a larger cloud model π cloud . The chosen action, namely Aedge t t is executed in the environment, and an observation Ot is collected. A final reward r is produced at the end of an episode. (c) Illustration of the interplay between thought convergence and uncertaintybased deferral rule for a setting in which the agent selects a suboptimal action: The probe score φ (top) crosses the thought convergence threshold λL at τ ∗ tokens, showing that the edge agent’s action has already converged. At this point, the uncertainty of the chosen action (bottom) is larger than the deferral threshold λD , correctly triggering deferral to the cloud agent. If reasoning were allowed to continue past token τ ∗ , the edge agent would grow increasingly overconfident, suppressing the deferral mechanism and letting a suboptimal action through unchallenged (see Appendix H for empirical evidence).
calibration of reasoning and deferral mechanisms has the potential to reduce resource consumption in terms of number of edge tokens, while also enhancing the efficiency of cloud deferral. Our contribution: We propose Think Short, Defer Smart (TSDS), a novel reasoning-based agentic framework that integrates and jointly calibrates the two synergetic mechanisms illustrated in Fig. 1: 1. Thought calibration: At each interaction step with the environment, following the ReAct framework (Yao et al., 2023), the edge agent reasons to determine an optimized next action. In order 2
to avoid overthinking, a lightweight convergence probe monitors the hidden representations of the edge agent’s model during reasoning. The probe is trained to detect the convergence of the reasoning trace towards a given decision (Wu et al., 2025). 2. Uncertainty-Aware Deferral: Once reasoning has converged and the edge action has been extracted, an uncertainty score over the resulting action token distribution is computed, and deferral to a cloud agent is triggered selectively by comparing the uncertainty score against a calibrated threshold λD (Piatrashyn et al., 2026; Chen et al., 2023). Deciding when to stop thinking and when to defer entails a trade-off between the agent’s performance in the environment, which is measured by the task-specific reward signal r, and resource consumption, which encompasses number of thinking steps, deferral rate, and number of environment interactions. Furthermore, as illustrated in Fig. 1(c), thought calibration and cloud deferral are strongly intertwined, as excessive reasoning can produce over-confident uncertainty signals, causing the edge agent to select suboptimal actions, instead of correctly deferring to the cloud agent. Conversely, early thought truncation may produce an uncertainty signal higher than warranted by the actual confidence of the edge agent, unduly increasing deferral frequency. Joint calibration of reasoning and cloud deferral is carried out via Learn-Then-Test (LTT) (Angelopoulos et al., 2021; Farzaneh & Simeone, 2026), which formulates hyperparameter selection as a multiple hypothesis testing problem, providing distribution-free finite-sample guarantees.
2
R ELATED W ORK
LLM agents: ReAct (Yao et al., 2023) established a by-now canonical pattern of alternating freetext reasoning with tool calls. Subsequent work added verbal self-reflection (Shinn et al., 2023), adaptive planning (Sun et al., 2023), and lifelong skill libraries (Wang et al., 2023). A fundamental limitation shared by all these systems is that each thought trace runs until a positional stop string, and hence inference cost scales with the trace length budget rather than with the actual difficulty of the reasoning step. Uncertainty quantification and deferral: Information-theoretic scores such as perplexity (PPL) provide uncertainty estimates that correlate with actual model errors (Malinin & Gales, 2021; Fomicheva et al., 2020). However, translating these raw scores into reliable deployment-level guarantees requires explicit calibration (Angelopoulos et al., 2021; Frankel et al., 2025). Model cascades (Chen et al., 2023; Yue et al., 2023; Ong et al., 2024; Hou et al., 2026) route single-turn queries across models of different capacities, focusing on independent requests without multi-step dependencies. ReDAct (Piatrashyn et al., 2026) applies PPL-guided deferral inside an agentic loop, but the deferral threshold is chosen heuristically. Test-time compute and thought calibration: Thought calibration (Wu et al., 2025) trains a convergence probe on offline single-turn trajectories and uses LTT (Angelopoulos et al., 2021) to certify a stopping threshold. Early-exit methods (Wang et al., 2025) and conformal prediction wrappers (Frankel et al., 2025; Chen et al., 2024) also aim at an adaptive use of inference resources.
3
P ROBLEM D EFINITION
Edge agent and environment: As shown in Fig. 1, we consider an edge agent π edge that interacts with a physical or virtual environment over discrete time steps t = 1, . . . , T to carry out a task described by a natural language prompt X. At each step t, the agent maintains a history Ht = X, (O1 , T1 , A1 ), . . . , (Ot−1 , Tt−1 , At−1 ), Ot , where (Tt , At ) are the thought trace and action at step t, and Ot is the observation of the environment upon taking action At−1 . Actions may include tool calls (e.g., Search, Lookup), code execution, free-form text, or motor commands in a physical environment (Ahn et al., 2022; Shridhar et al., 2021; Chevalier-Boisvert et al., 2023). At episode termination, i.e., after time t = T , the environment issues a scalar reward r(X, A1 , . . . , AT ) ∈ [0, 1] that depends on the task prompt X, which specifies both the goal and the evaluation criterion, and on the trajectory of actions taken. Examples include 3
exact-match for question answering (Yang et al., 2018; Cobbe et al., 2021) and binary pass/fail for goal-oriented tasks (Austin et al., 2021; Shridhar et al., 2021). At each time step t, given history Ht , the edge agent generates a thought trace Tt = (τt,1 , . . . , τt,Lt ). Each token τt,i ∼ π edge (· | Ht , τt,1 , . . . , τt,i−1 ) is drawn autoregressively using the edge agent’s (ℓ) model π edge . For future reference, we denote as ht,i ∈ Rd the latent activation of model π edge for token i at layer ℓ. After Lt reasoning steps, the edge agent may take an action Aedge or defer to the t edge edge edge cloud. The action Aedge is produced by model π as A ∼ π (· | H , τ , . . . , τt,i−1 ). t t,1 t t Cloud agent and cascading: A more capable cloud model π cloud is available to the edge agent as a fallback. If a cloud deferral decision is made at time t, the full history Ht is transferred to the cloud, and the model π cloud generates an alternative action Acloud ∼ π cloud (· | Ht ), replacing the t edge edge agent’s action At for that step. We write as Dt ∈ {0, 1} the indicator for whether the cloud model is called (Dt = 1) or not (Dt = 0). Thinking convergence and deferral: The edge agent’s operation is governed by two thresholdparameterized mechanisms. A thinking convergence rule decides at each thought position i whether to halt reasoning and extract an action Aedge and an uncertainty score Ut . To this end, the rule t (ℓ) applies a threshold λL to a convergence signal Ci,t extracted from the latent activations ht,i of the edge model as: stop if Ct,i > λL and continue otherwise. A deferral rule decides whether to execute the edge action or hand the step to π cloud using a threshold λD as: defer if Ut > λD and not defer otherwise. Accordingly, the executed action At is ( Aedge if Ut ≤ λD t At = (1) Acloud if Ut > λD , t with action Aedge produced at thinking step t Lt = min{arg min{i : Ct,i > λL }, Lmax }, i
(2)
where Lmax is the maximum allowed number of thinking steps. Design objectives: We define the expected episode reward R(λ) = E[r(λ)], where the expectation is over task prompts and environment stochasticity, and we have highlighted the dependence on the vector λ = (λL , λD ) of hyperparameters. The deferral cost, i.e., the expected number of cloud calls per episode, and the thinking cost, i.e., number of reasoning tokens per episode, are given by " T # " T # X X CD (λ) = E Dt and CL (λL ) = E Lt , (3) t=1
t=1
respectively. Another important metric is the episode-length cost CS (λ) = E[T ], where T is the (random) number of agent steps until episode termination. The overall design objective is to minimize thinking cost CL (λL ), while keeping average reward R(λ) above a floor Rmin and cloud max deferral rate CD (λ) within a budget CD , i.e., min CL (λL ) λ
4
subject to
max R(λ) ≥ Rmin and CD (λ) ≤ CD .
(4)
T HINK S HORT, D EFER S MART
This section introduces TSDS, a framework that addresses problem (4) by proposing concrete instantiations of the thinking convergence rule (Sec. 4.1) and the deferral rule (Sec. 4.2), as well as by incorporating the calibration of their joint threshold vector λ via multi-objective LTT (Sec. 4.3). 4.1
C ONVERGENCE P ROBE AND T HOUGHT C ALIBRATION
The goal of the thinking convergence rule is to detect, from the edge agent π edge ’s internal representations, the moment at which its reasoning has committed to a stable action. We follow the general 4
approach of (Wu et al., 2025) of training a lightweight probe on offline trajectories to certify a stopping threshold, tailoring the methodology to agents that reason to act in a multi-step interactive loop. (ℓ)
As illustrated in Fig. 1, define the convergence signal as Ct,i = φθ (ht,i ) ∈ R, thus map(ℓ) ping the hidden state ht,i at reasoning step i and layer ℓ to the scalar convergence score Ct,i .
The layer ℓ is treated as a hyperparameter. The probe φθ is trained offline on a dataset Dtr = (ℓ),(n) (n) tr {(ht,i , yt,i )}N n=1 , where index n = 1, . . . , Ntr runs over training episodes. The dataset Dtr is collected by running the model π edge with full thought generation (no early stopping, no deferral) (ℓ) and by recording for each position i in a set I the latent state ht,i . Position i is labelled positive, (n)
i.e., yt,i = 1, if the early-stopped action at every subsequent probe position i′ ≥ i within the same step agrees with the full-thought action, i.e., Y (n) (n) (n) yt,i = 1 Ai′ ,t = At ∈ {0, 1}, (5) i′ ∈I:i′ ≥i (n)
(n)
where At is the full-thought reference action and Ai,t is the early-stopped action obtained by truncating the trace at position i and injecting token </think>. 4.2
C LOUD D EFERRAL
Once the edge action Aedge is extracted, the uncertainty score Ut is obtained as the PPL of the t , following (Piatrashyn et al., 2026). Alternative scores are defined in edge model for action Aedge t Appendix B and can be substituted without modifying the calibration procedure. 4.3
M ULTI -O BJECTIVE C ALIBRATION
In this section, we address the multi-objective optimization problem (4) via LTT (Angelopoulos et al., 2021; Farzaneh & Simeone, 2026). For each candidate reasoning threshold λL , we run the TSDS system with thought calibration only, obtaining the held-out calibration dataset as n o (n) Ncal (n) Dcal (λL ) = r(n) (λL ), {Ut (λL )}Tt=1 , (6) n=1
(n) where r(n) (λL ) is the terminal reward of episode n, and Ut (λL ) is the uncertainty score at step t. (n) (n) (n) Given the scores Ut (λL ), the per-step deferral indicator Dt (λD ) = 1[Ut (λL ) > λD ] can be
evaluated post-hoc for any candidate threshold λD . (i)
(j)
Following LTT, we construct a finite grid Λ = {(λL , λD )} of |Λ| candidate pairs and follow the LTT procedure summarized in Appendix C. The procedure returns a hyperparameter vector λ̂ ∈ Λ with the following guarantee. Proposition 1 (Joint risk control guarantee (Angelopoulos et al., 2021)). Assume that calibration data Dcal is drawn i.i.d. from the same distribution underlying the generation of the test episode. Then, the hyperparameter λ̂ returned by LTT satisfies the constraints in problem (4) with probability larger than 1 − δ, i.e., max P R(λ) ≥ Rmin and CD (λ) ≤ CD ≥ 1 − δ, (7) where δ is a user-defined probability.
5
E XPERIMENTS
We evaluate TSDS on three benchmarks: code generation (MBPP, Sec. 5.2), multi-step question answering (HotpotQA, Sec. 5.3), and simulated household-robot planning (Sec. 5.4). An additional experiment on single-step arithmetic reasoning (GSM8K) can be found in Appendix G.2. The edge model DeepSeek-R1-Distill-Qwen-7B and the Exponential Moving Average (EMA) convergence probe are adopted throughout. All experiments implement LTT with one-sided binomial 5
Algorithm 1 Think Short, Defer Smart (step t) Require: Certified pair λ = (λL , λD ) ∈ Λ̂; probe φθ ; π edge ; π cloud ; history Ht . 1: Generate first token τt,1 with π edge ; i ← 1 2: while i < Lmax do 3: Retrieve hidden state ht,i 4: if φθ (ht,i ) ≥ λL then break ▷ Thought converged 5: else generate next token τt,i+1 ; i ← i + 1 6: end if 7: end while edge 8: Inject </think>; generate At ▷ Action extraction edge 9: ut ← U (At | Ht , τt,1:i , </think>) 10: if ut ≤ λD then 11: At ← Aedge ▷ Confident: accept t 12: else 13: Acloud ∼ π cloud (· | Ht ) ▷ Uncertain: defer t 14: At ← Acloud t 15: end if 16: Execute At ; observe Ot+1 ; update Ht+1
p-values with Bonferroni correction at level δ = 0.10 (see Appendix C). The target deferral rate is max = 0.70. The candidate grid Λ for LTT is formed by considering a uniform sweep of the unit CD interval with 5 levels for threshold λL and 4 uniformly spaced quantiles of the per-episode maximum uncertainty on dataset Dtr for threshold λD . Benchmark-specific settings and parameters are given in each subsection. All reported metrics are means over 50 independent calibration–test splits. 5.1
BASELINES
We compare TSDS against the following policies, which isolate the contribution of different components. • E-ReAct (Edge ReAct): As in the original ReAct baseline (Yao et al., 2023), the model π edge runs with full thought traces (λL = ∞), and there is no deferral (λD = ∞). • Cloud ReAct: The cloud model π cloud is used for every step, providing a performance ceiling at maximum cloud deferral cost. • E-ReAct-TC (Edge ReAct with Thought Calibration): Thought calibration is active, but deferral is disabled (λD = +∞). This isolates the contribution of thought truncation alone, without the safety net of the deferral mechanism. • ReDAct (Piatrashyn et al., 2026): The edge model π edge generates full thought traces (λL = ∞), deferring whenever the PPL exceeds a fixed threshold λ̂D , set to the median PPL on Dtr . • ReDAct-CD (ReDAct with Calibrated Deferral): The edge model π edge generates full thought traces (λL = 0), setting Lt = Lmax in (2), and the deferral threshold λD is selected via singleobjective LTT on dataset Dcal to satisfy the reward constraint R(λ) ≥ Rmin with probability 1−δ. This isolates the contribution of calibrated deferral without thought calibration. 5.2
C ODE G ENERATION : MBPP
We evaluate TSDS on MBPP (Austin et al., 2021), a benchmark of mostly basic Python programming problems. Each problem specifies a Python function in natural language with unit tests, and reward is 1 if all unit tests pass and 0 otherwise. Actions are complete Python functions, with a mean of 1166 thought tokens per problem. This is a single-step setting, i.e., T = 1. Setup: The cloud model is DeepSeek-R1-Distill-Qwen-32B, and the thought caps for both the edge and cloud models is set to Lmax = 2048. The probe φθ is trained at layer ℓ = 5 on 100 MBPP train-split episodes. We evaluate on 257 test problems with 50 random 60/40 splits. The reasoning-threshold grid is λL ∈ {0.1, 0.2, . . . , 0.9}, and the deferral grid 6
0.70 0.65
S C D ct ct eA eA ct-T t-C TSD E-R loud R -ReA eDAc R E C
0.8
[CL ]
0.75
CDmax
1.0
Average thinking cost
0.80
[CD ]
R min
Average deferral rate
[R]
Average reward
0.85
0.6 0.4 0.2 0.0
S C D ct ct eA eA ct-T t-C TSD E-R loud R -ReA eDAc R E C
1250 1000 750 500 250 0
S C D ct ct eA eA ct-T t-C TSD E-R loud R -ReA eDAc R E C
Figure 2: Test results for MBPP, shown for five policies and averaged over 50 random 60/40 calibration–test splits (ntest = 103 per split). Error bars show one standard deviation across splits. Left: Average reward per policy. The dashed horizontal line marks the minimum reward floor Rmin = 0.69. Center: Average deferral rate to the cloud model. The dashed line marks the max maximum allowed deferral fraction CD = 0.70. Right: Average thinking cost (tokens). Hatched bars denote uncalibrated reference policies.
λD ∈ {0.084, 0.092, 0.106, 0.117, 0.141} [nat] is drawn from PPL percentiles on Dtr , giving |Λ| = 45 candidate pairs. The minimum reward floor is Rmin = 0.69. Fig. 2 reports average reward, deferral rate, and thinking cost for all policies, averaged over 50 calibration–test splits. E-ReAct runs full thought traces without deferral, establishing the uncalibrated edge baseline. Cloud ReAct achieves higher reward but routes every problem to the cloud model, exhausting the deferral budget. E-ReAct-TC is not able to certify any hyperparameters to guarantee reward above the floor Rmin due to not having access to the cloud model. ReDAct-CD improves reward through selective cloud escalation, and satisfies the cloud deferral budget on average, but at the cost of requiring full thought traces for the edge agent. TSDS clears the reward floor and respects the deferral budget while using 64% fewer thought tokens than E-ReAct and ReDAct-CD and calling the cloud model on less than half the rate of ReDAct-CD. 5.3
M ULTI -S TEP S ETTING : H OTPOT QA
We evaluate TSDS on HotpotQA (Yang et al., 2018), a multi-hop QA benchmark where the agent takes up to Tmax = 7 steps, with action space including the tools Search[·], Lookup[·], and Finish[·], to answer questions requiring two supporting passages. Reward is a binary terminal exact match indicator. Setup: The cloud model is DeepSeek-R1-Distill-Qwen-14B. λR = 0.3 Edge and cloud thought caps are set to λR = 0.1 λR = 0.5 Lmax = 384 and 1024, respectively. The λR = 0.7 probe φθ is trained at layer ℓ = 5 on 50 λR = 0.9 train-split episodes. We evaluate on 250 validation problems with 50 random 75/25 λR = 0.3 λR = 0.7 λR = 0.1 splits. The reasoning-threshold grid is λL ∈ {0.1, 0.3, 0.5, 0.7, 0.9}, and the deferral λR = 0.9 λR = 0.5 grid λD ∈ {0.258, 0.318, 0.355, 0.392} [nat], corresponds to percentiles {40, 60, 75, 85} of the per-episode maximum uncertainty on dataset Dtr , giving |Λ| = 20 candidate Figure 3: Reward vs. thinking cost on HotpotQA hyperparameters. The minimum reward floor across five λ grid values. The blue dashed curve L is Rmin = 0.22. shows thought calibration only, and the solid Fig. 3 shows the average reward as a function of green curve shows TSDS at the certified deferral the average thinking cost, as obtained by vary- threshold λD = 0.318. All values are means over ing the threshold λL for a fixed deferral thresh- the full pool of 250 validation episodes. old λD = 0.318. Thought calibration is seen 7
to significantly reduce the thinking cost relative to E-ReAct, but the average reward is negatively affected. In contrast, TSDS uniformly outperforms E-ReAct, illustrating the benefit of combining thought calibration and cloud deferral. Additionally, ReDAct-CD only marginally improves over EReAct, confirming that untruncated reasoning induces overconfidence in the edge model, resulting in fewer actions being deferred to the cloud model. Fig. 4 reports average reward, deferral rate, and thinking cost for all policies, averaged over 50 calibration–test splits. ReDAct-CD improves reward through selective cloud escalation, though at the cost of the full reasoning budget. The pattern mirrors Fig. 2. ReDAct-CD and TSDS are the only two policies clearing the reward floor and respecting the deferral budget, while TSDS does so with substantially fewer thought tokens than ReDAct-CD. In the multi-step setting of this experiment this advantage is further reinforced, as per-step thought calibration concentrates cloud deferral on precisely the steps where the edge model is most uncertain. 5.4
S IMULATED H OUSEHOLD -ROBOT P LANNING
To evaluate TSDS in a physical AI setting, we introduce a text-based simulated householdmanipulation benchmark for evaluating multi-step physical-AI agents without requiring vision or physical hardware. Inspired by SayCan (Ahn et al., 2022) and ALFWorld (Shridhar et al., 2021), the environment places an agent in a four-room setting (bedroom, kitchen, bathroom, living room) and asks it to complete tasks drawn from four families, namely fetch, place, clean, and heat, by issuing five action primitives: go to <room>, pick up <object>, put down <object>, heat <object>, and clean <object>, within at most Tmax = 6 steps. Reward is binary and is equal to 1 if the task is completed by episode termination, and to 0 otherwise. The environment layout is illustrated in Appendix G.5 (Fig. 12). Setup: The same 7B/14B model pair as HotpotQA is used with Lmax = 512 for the edge agent and 1024 for the cloud agent. The probe is retrained at layer ℓ = 10 on 50 household episodes (see Appendix G.5 for an ablation study). We evaluate on 200 held-out episodes with 50 random 75/25 splits. The reasoning-threshold grid is λL ∈ {0.1, 0.3, 0.5, 0.7, 0.9}, and the deferral grid λD ∈ {0.287, 0.313, 0.343, 0.384} [nat] corresponds to percentiles {40, 55, 70, 85} of the perepisode maximum uncertainty on Dtr , giving |Λ| = 20. The minimum reward floor is Rmin = 0.52.
0.3 0.2
S C D ct ct eA eA ct-T t-C TSD E-R loud R -ReA eDAc R E C
CDmax
1.0
[CL ]
0.8
Average thinking cost
0.4
[CD ]
R min
[R]
Average reward
0.5
Average deferral rate
Fig. 5 reports average reward, deferral rate, and thinking cost for all policies, averaged over 50 calibration–test splits. The overall pattern follows Figs. 2 and 4, with ReDAct-CD and TSDS being the only two policies that satisfy both constraints, while TSDS does so at 64% lower thinking cost than ReDAct-CD and E-ReAct. One notable difference is that ReDAct-CD achieves lower reward than TSDS here, unlike in Figs. 2 and 4. This can be further evidence that full thought traces allow the edge model to grow overconfident in multi-step plans, suppressing the perplexity signal at the
0.6 0.4 0.2 0.0
S C D ct ct eA eA ct-T t-C TSD E-R loud R -ReA eDAc R E C
1250 1000 750 500 250 0
S C D ct ct eA eA ct-T t-C TSD E-R loud R -ReA eDAc R E C
Figure 4: Test results for HotpotQA, shown for all five policies and averaged over 50 random 75/25 calibration–test splits. Error bars show one standard deviation across splits. Left: Average reward per policy. The dashed horizontal line marks the minimum reward floor Rmin = 0.22. Center: Average deferral rate to the cloud model. The dashed line marks the maximum allowed deferral max fraction CD = 0.70. Right: Average thinking cost (tokens). Hatched bars denote uncalibrated reference policies. 8
0.6 0.5
S C D ct ct eA eA ct-T t-C TSD E-R loud R -ReA eDAc R E C
CDmax
1.0
[CL ]
0.8
Average thinking cost
0.7
[CD ]
R min
Average deferral rate
[R]
Average reward
0.8
0.6 0.4 0.2 0.0
S C D ct ct eA eA ct-T t-C TSD E-R loud R -ReA eDAc R E C
1500 1000 500 0
S C D ct ct eA eA ct-T t-C TSD E-R loud R -ReA eDAc R E C
Figure 5: Test results for the household robot task, shown for five policies and averaged over 50 random 75/25 calibration–test splits. Error bars show one standard deviation across splits. Left: Average reward per policy. The dashed horizontal line marks the minimum reward floor Rmin = 0.52. Center: Average deferral rate to the cloud model. The dashed line marks the maximum max allowed deferral fraction CD = 0.70. Right: Average thinking cost (tokens). Hatched bars denote uncalibrated reference policies.
steps where cloud escalation is most needed, while TC-induced truncation preserves uncertainty and directs deferral to the genuinely hard steps.
6
C ONCLUSION
We introduced Think Short, Defer Smart (TSDS), which composes thought calibration and uncertainty-aware deferral into a jointly calibrated pipeline for LLM agents. A convergence probe halts on-device reasoning once the intended action has stabilized, and a perplexity-based deferral rule escalates uncertain actions to a cloud model. To ensure reliability, a multi-objective LearnThen-Test procedure certifies that reward and cloud-call rate remain within user-specified limits, selecting the operating point that minimizes thinking compute. Across all four benchmarks, TSDS simultaneously satisfies the certified reward floor and the cloudcall budget, while consuming substantially fewer thought tokens than deferral-only baselines. This outcome highlights two complementary insights. First, thought calibration and deferral to a cloud model are synergistic, as truncated reasoning leads to detecting uncertainty signals at their most meaningful state. Second, the necessity of deferral varies sharply with task complexity. On simpler single-step tasks, truncation alone nearly suffices and deferral provides a targeted safety net, while on multi-step tasks such as the household robot task, thought calibration without deferral collapses performance severely, and cloud escalation is essential for recovery. Future directions: Immediate extensions include richer uncertainty signals, e.g., semantic entropy, to narrow the deferral gap toward the oracle; multi-tier cascades of agents with multiconstraint LTT certification; and extending thought calibration to models without an explicit reasoning delimiter by inferring action convergence directly from hidden-state trajectories (Esakkiraja et al., 2026).
ACKNOWLEDGMENTS This work was supported by the European Research Council (ERC) under the European Union’s Horizon Europe Programme (grant agreement No. 101198347). The work of O. Simeone was also supported by an EPSRC Open Fellowship (EP/W024101/1) and by the EPSRC project (EP/X011852/1).
R EFERENCES Michael Ahn, Anthony Brohan, Noah Brown, et al. Do as I can, not as I say: Grounding language in robotic affordances. arXiv preprint arXiv:2204.01691, 2022. 9
Anastasios N. Angelopoulos, Stephen Bates, Emmanuel J. Candès, Michael I. Jordan, and Lihua Lei. Learn then test: Calibrating predictive algorithms to achieve risk control. arXiv preprint arXiv:2110.01052, 2021. Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie J. Cai, Michael Terry, Quoc V. Le, and Charles Sutton. Program synthesis with large language models. arXiv preprint arXiv:2108.07732, 2021. Jiechen Chen, Sangwoo Park, and Osvaldo Simeone. Knowing when to stop: Delay-adaptive spiking neural network classifiers with reliability guarantees. IEEE Journal of Selected Topics in Signal Processing, 19(1):88–102, 2024. Lingjiao Chen, Matei Zaharia, and James Zou. FrugalGPT: How to use large language models while reducing cost and improving performance. arXiv preprint arXiv:2305.05176, 2023. Maxime Chevalier-Boisvert, Bolun Dai, Mark Towers, Rodrigo de Lazcano, Lucas Willems, Salem Lahlou, Suman Pal, Pablo Samuel Castro, and Jordan Terry. Minigrid & miniworld: Modular & customizable reinforcement learning environments for goal-oriented tasks. In Advances in Neural Information Processing Systems, volume 36, 2023. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168, 2021. Olive Jean Dunn. Multiple comparisons among means. Journal of the American Statistical Association, 56(293):52–64, 1961. Nishant Esakkiraja et al. Therefore: Actions are encoded in hidden representations before text generation. arXiv preprint arXiv:2604.01202, 2026. Amirmohammad Farzaneh and Osvaldo Simeone. Statistically valid hyperparameter selection: From tuning to guarantees. arXiv preprint arXiv:2606.25601, 2026. Marina Fomicheva, Shuo Sun, Lisa Yankovskaya, Frédéric Blain, Francisco Guzmán, Mark Fishel, Nikolaos Aletras, Vishrav Chaudhary, and Lucia Specia. Unsupervised quality estimation for neural machine translation. Transactions of the Association for Computational Linguistics, 8: 539–555, 2020. Max Frankel et al. Conformal prediction for large language model inference. arXiv preprint arXiv:2502.09535, 2025. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. DeepSeek-R1: Incentivizing reasoning capability in LLMs via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025. Qiushuo Hou, Sangwoo Park, Matteo Zecchin, Yunlong Cai, Guanding Yu, Osvaldo Simeone, and Tommaso Melodia. Reliable llm-based edge-cloud-expert cascades for telecom knowledge systems. IEEE Transactions on Communications, 2026. Subhash Kantamneni. Attention probes, 2025. Cited via Kramár et al. (2026). János Kramár, Joseph Engels, Senthooran Wang, Bilal Chughtai, Rohin Shah, Neel Nanda, and Arthur Conmy. Building production-ready probes for Gemini. arXiv preprint arXiv:2601.11516, 2026. Andrey Malinin and Mark Gales. Uncertainty estimation in autoregressive structured prediction. arXiv preprint arXiv:2002.07650, 2021. Isaac Ong, Amjad Almahairi, Vincent Wu, Wei-Lin Chiang, Tianhao Wu, Joseph E. Gonzalez, M. Waleed Kadous, and Ion Stoica. RouteLLM: Learning to route LLMs with preference data. arXiv preprint arXiv:2406.18665, 2024. 10
Aliaksei Piatrashyn et al. ReDAct: Reasoning-driven action selection under uncertainty for language model agents. arXiv preprint arXiv:2604.07036, 2026. Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: Language agents with verbal reinforcement learning. In Advances in Neural Information Processing Systems, volume 36, 2023. Mohit Shridhar, Xingdi Yuan, Marc-Alexandre Côté, Yonatan Bisk, Adam Trischler, and Matthew Hausknecht. ALFWorld: Aligning text and embodied environments for interactive learning. In International Conference on Learning Representations, 2021. Haotian Sun, Yuchen Zhuang, Lingkai Kong, Bo Dai, and Chao Zhang. AdaPlanner: Adaptive planning from feedback with language models. In Advances in Neural Information Processing Systems, volume 36, 2023. Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. Voyager: An open-ended embodied agent with large language models. arXiv preprint arXiv:2305.16291, 2023. Hao Wang et al. EAT: Efficient adaptive thinking for LLMs via early exit. arXiv:2503.05061, 2025.
arXiv preprint
Ruixuan Wu, Weijia Shi, and Luke Zettlemoyer. arXiv:2505.18404, 2025.
Thought calibration.
arXiv preprint
Jianing Xu et al. On-device language models: A comprehensive review. arXiv:2409.00088, 2024.
arXiv preprint
Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William W. Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. HotpotQA: A dataset for diverse, explainable multi-hop question answering. In Proceedings of EMNLP, 2018. Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. ReAct: Synergizing reasoning and acting in language models. In International Conference on Learning Representations, 2023. Murong Yue, Jie Zhao, Min Zhang, Liang Du, and Ziyu Yao. Large language model cascades with mixture of thought representations for cost-efficient reasoning. arXiv preprint arXiv:2310.03094, 2023. Andy Zhou, Kai Yan, Michal Shlapentokh-Rothman, Haohan Wang, and Yu-Xiong Wang. Language agent tree search unifies reasoning, acting, and planning in language models. arXiv preprint arXiv:2310.04406, 2024.
11
A
C ODE AVAILABILITY
The full implementation of TSDS, including rollout collection, convergence probe training, LTT calibration, and all experiment scripts, is available at: https://github.com/amirfar76/think-short-defer-smart The Household Robot Task benchmark environment introduced in Sec. 5.4 is released as a standalone package at: https://github.com/amirfar76/household-robot-bench
B
U NCERTAINTY M EASURES
The uncertainty score ut in the deferral rule (Sec. 3) can be any information-theoretic quantity derived from the token-level probabilities of the generated action sequence y = (τ1 , . . . , τL ) given context x, with joint probability p(y | x, θ) =
L Y
p(τi | x, τ<i , θ).
(8)
i=1
All three measures below are available as free by-products of the autoregressive decoding pass, requiring no additional forward pass (Malinin & Gales, 2021). Sequence Probability (SP): USP (y | x) = −
L X
log p(τi | x, τ<i , θ).
(9)
i=1
The negative log-likelihood of the entire sequence. SP is length-sensitive: longer sequences accumulate higher values even at the same per-token confidence. Perplexity (PPL): L
UPPL (y | x) = −
1X log p(τi | x, τ<i , θ). L i=1
(10)
SP normalized by sequence length, measuring per-token average surprise and removing the length bias. High perplexity signals that the model was repeatedly uncertain about its own tokens. PPL is the primary score recommended by (Piatrashyn et al., 2026) for deferral and is used in all our experiments. Mean Token Entropy (MTE): L
UMTE (y | x) =
1X H(τi | x, τ<i , θ), L i=1
(11)
where H(τi | ·) is the Shannon entropy of the token distribution at position i. While SP and PPL examine the probability of the generated token, MTE measures the spread of the full distribution, capturing uncertainty that SP and PPL may miss when the model happens to generate a high-probability token despite wide disagreement across the vocabulary (Malinin & Gales, 2021).
C
L EARN -T HEN -T EST C ALIBRATION P ROCEDURE
For each candidate λk ∈ Λ, we define two null hypotheses corresponding to constraint violations: the reward null HkR : R(λk ) ≤ Rmin , i.e., that the reward constraint is violated, and the cost null max HkC : CD (λk ) ≥ CD , i.e., that the deferral budget is exceeded. With φθ and Λ fixed before C R C Dcal is observed, we compute one-sided p-values pR k and pk for the null hypotheses Hk and Hk , 12
respectively. For each candidate λk , the p-values are computed from the pre-collected calibration (i) data for that specific λL value (Sec. 3), ensuring independence between the data and the threshold under test. When rewards are binary, the exact one-sided binomial p-value min pR ) ≤ ⌊N r̄k ⌋ , (12) k = Pr Binomial(N, R where r̄k is the sample mean reward, is the probability that N i.i.d. Bernoulli trials with success probability Rmin produce at most as many successes as observed. An analogous one-sided test R C gives pC k from the sample mean deferral rate c̄k . Since the statistics pk and pk are valid p-values C for their respective nulls (Angelopoulos et al., 2021), the maximum pk = max(pR k , pk ) is a valid R C p-value for the union null Hk ∪ Hk (Angelopoulos et al., 2021). The certified set is then Λ̂ = A {pk }, δ , (13) where A is any FWER-controlling procedure (Dunn, 1961). We select the final hyperparameter λ̂ from the certified set Λ̂ via lexicographic minimization over Λ̂. We first minimize the thinking cost CL (λL ) (3), and in multi-step settings, break ties by minimizing the episode-length cost CS (λ). Any remaining ties are broken by minimizing CD (λ). We adopt Bonferroni correction (Dunn, 1961) as the default choice for A as it requires no assumptions on the structure of Λ, and provides unconditional FWER control in any setting.
D
T HOUGHT C ALIBRATION : S TOPPING RULE I LLUSTRATION
Fig. 6 illustrates the stopping rule from Section 4.1 on a single interaction step. The edge model generates thought tokens τt,1 , τt,2 , . . . sequentially; at each strided position i ∈ Pt , the convergence (ℓ) probe φθ evaluates the hidden state ht,i and returns a scalar score. The score rises as the model’s intermediate action stabilises: early in the thought, the model is still exploring reasoning paths and its committed action fluctuates (score near zero); as reasoning consolidates, the score climbs toward one. Stopping fires the first time the probe score crosses the certified threshold λL . In the example shown, this occurs at position i = 5: tokens τt,6 and τt,7 are never generated, saving two of the seven allocated positions. The delimiter </think> is injected immediately after τt,1:5 , and the edge model generates Aedge from the truncated trace. The action is then passed to the deferral mechanism t (Section 4.2): its PPL score is compared to λD , and if the action is insufficiently confident the cloud model is invoked instead.
E
P ROBE A RCHITECTURE D ETAILS
We compare seven candidate instantiations of the convergence probe φθ , all trained by minimizing the binary cross-entropy loss with soft targets using AdamW (learning rate 10−3 , weight decay 10−3 , full-batch, early stopping on validation loss with patience 20 out of a maximum 200 epochs). Window probes use the W = 16 most recent strided probe positions left-padded with zeros and a binary mask. (ℓ)
• Linear: logistic regression φ(h) = w⊤ h + b on the single hidden vector ht,p . (ℓ)
• MLP: two-layer MLP with hidden width 100 and ReLU activations on the single vector ht,p . • Mean-linear: per-position linear score uj = w⊤ xj + b averaged over unmasked window slots (Kramár et al., 2026, §3.1.1). • EMA: trained identically to Mean-linear; at inference, per-position scores are passed through an EMA (ej = αuj + (1 − α)ej−1 , α = 0.5) and the probe output is maxj ej (Kramár et al., 2026, §3.1.2). • Attention probe: per-position MLP transform followed by softmax-weighted aggregation across K = 4 heads, aggregated by a linear projection (Kantamneni, 2025). • MultiMax probe: same per-position MLP transform but with hard max over the window for K = 4 heads, aggregated by a linear projection (Kramár et al., 2026, §3.2.1). 13
not generated
¿6
¿1
¿2
¿3
¿4
¿7
< =think >
¿5
Atsmall
p^ t = 5
¾('(ht; p ))
1
¸T
0 p=1
2
3
4
5
Figure 6: Illustration of the thought calibration stopping rule. The probe signal σ(φθ (ht,i )) rises as the intended action stabilizes. Generation halts at the first position crossing the certified threshold λL (here i = 5), leaving τt,6 , τt,7 ungenerated. The delimiter </think> is injected and the model generates Aedge from the partial trace. t • Rolling-mean attention probe: per-position MLP transform, attention-weighted mean over each sub-window of width w′ = 4 strided positions, maximum over K = 10 sub-window scores. The architecture identified as strongest across a broad range of probing tasks in (Kramár et al., 2026, §3.2.2).
F
P ROOF OF P ROPOSITION 1
C R C The p-values pR k (12) and pk are valid one-sided p-values for Hk and Hk respectively (AngelopouR C los et al., 2021); hence pk = max(pk , pk ) is a valid p-value for the joint null HkR ∪ HkC . Because A controls the FWER at level δ, the probability that any true joint null is rejected is at most δ, which is equivalent to (7).
G
A DDITIONAL E XPERIMENTAL D ETAILS
G.1
C ONVERGENCE P ROBE A BLATION
We compare seven probe architectures on Nep = 100 GSM8K train-split problems with DeepSeek-R1-Distill-Qwen-7B (28 layers, d = 3584, greedy, bfloat16), yielding 1076 probe samples at stride 16 (cap Lmax = 384). The probe target is the stable-run label, Y yt,p = 1 Ap′ ,t = At , (14) p′ ∈Pt , p′ ≥p
which is 1 iff the early-stopped action agrees with the full-thought action at position p and at every subsequent probe position in step t, directly aligned with the irrevocability of the stopping rule (Sec. 4.1). To isolate hidden-state signal from the global trend in positive rate across the thought trace, we stratify by relative position p/Lt and report per-quartile AUROC AUROCQq = AUROC {(ŝi , yi ) : pi /Lti ∈ [(q−1)/4, q/4)} , q ∈ {1, 2, 3, 4}. (15) Windowed probes outperform single-state probes: the EMA probe leads with pooled AUROC 0.815 at ℓ⋆ = 5 (vs. ≤ 0.673 for single-state), generalizing consistently across all four quartiles. Best probing layers are shallow (ℓ ≈ 5 for windowed probes; ℓ = 16 for single-state), suggesting actionrelevant information concentrates early in the transformer stack and benefits from temporal smoothing across the window. The non-monotone quartile pattern (Q2 highest at 0.857, Q3 dip at 0.748, 14
(ℓ)
Figure 7: Test AUROC of φθ (ht,p ) vs. probing layer ℓ on GSM8K under the stable-run label (14). Solid lines: seed-averaged test AUROC; shaded bands: ±1 standard deviation. Dashed line: chance (0.5). The EMA and rolling-mean attention probes dominate at ℓ ≈ 5. Q4 recovery to 0.812 for EMA) reflects class-balance shifts: Q1 has a low positive rate (stability requires all subsequent probe positions to agree, rare for p ≪ Lt ); Q2 shows peak discrimination because the model encodes the action before committing irrevocably; Q4 recovers as the model converges toward its natural stopping string. Table 1: Probe ablation on GSM8K with DeepSeek-R1-Distill-Qwen-7B (Nep = 100 problems, 1076 probe samples, 3 seeds) under the stable-run label (14). For each probe architecture we report the best layer ℓ⋆ and its seed-averaged pooled AUROC, plus per-quartile AUROC AUROCQ1 , . . . , AUROCQ4 (defined in (15)). Bold marks the best entry in each column. Probe
ℓ⋆
pooled AUROC
Q1
Q2
Q3
Q4
Linear MLP Mean-linear Attention MultiMax Rolling-mean attn. EMA
16 16 5 5 26 5 5
0.637 0.673 0.686 0.733 0.626 0.762 0.815
0.494 0.496 0.608 0.672 0.556 0.672 0.817
0.688 0.727 0.687 0.727 0.700 0.770 0.857
0.606 0.606 0.663 0.708 0.725 0.733 0.748
0.603 0.713 0.758 0.823 0.554 0.874 0.812
Reasoning model and data collection: We use DeepSeek-R1-Distill-Qwen-7B with 28 transformer layers and hidden size d = 3584. For each of the 100 GSM8K train-split problems, we decode the thought trace greedily until </think> or Lmax = 384 tokens. A single forward pass (ℓ) with hidden-state output provides ht,p at every thought position and every layer at zero marginal cost. For each strided position p ∈ Pt , we splice </think> after τt,1:p and decode to obtain Ap,t . The convergence label yt,p is then computed via (14). Ablation grid and metrics: We train each of the seven architectures at every layer ℓ ∈ {1, . . . , 28} from three random initializations with the 60/20/20 problem-level split fixed, giving 7×28×3 = 588 configurations. The formal AUROC is the U-statistic AUROC = Pr ŝI > ŝJ yI = 1, yJ = 0 , I, J ∼ Uniform{1, . . . , n} independently, (16) equal to 0.5 for a probe whose scores are independent of the label. Position-only floor: A logistic regression on the relative thought position p/Lt alone achieves AUROC = 0.530. This position-only floor quantifies the trivial signal available from late-thought 15
Figure 8: Per-quartile AUROC at the best layer ℓ⋆ for each probe architecture on GSM8K under the stable-run similarity score (14). The EMA probe leads on Q1 , Q2 , and Q3 , while the rolling-mean attention probe attains the highest late-thought AUROC (AUROCQ4 = 0.874). The dashed grey line marks chance performance (AUROC = 0.5).
samples having higher positive rates. Only 22% of problems are stable throughout their thought; the median first-stable position falls at 91% of thought length, and the mean flip count is 2.8 per step. G.2
GSM8K E ND - TO -E ND TSDS
We evaluate TSDS on GSM8K (Cobbe et al., 2021), a benchmark of grade-school arithmetic word problems. The task is single-step (T = 1): the agent generates a complete reasoning trace followed by a final numerical answer in a boxed format, and reward is a binary exact-match indicator. Setup: We use DeepSeek-R1-Distill-Qwen-7B as the edge model π edge (Lmax = 384, mean thought length 159 tokens) and DeepSeek-R1-Distill-Qwen-14B as the cloud agent π cloud (Lmax = 1024, mean thought length 741 tokens). The EMA probe is set at layer ℓ = 5 and trained on 100 GSM8K train-split problems. We evaluate on 1300 held-out problems averaged over 50 random 85/15 calibration–test splits (Ntest = 195 per split); full-thought accuracies are 84.5% for π edge and 90.2% for π cloud . Calibration: The reasoning-threshold grid is λL ∈ {0.1, 0.2, . . . , 0.9}. The deferral grid λD ∈ {0.053, 0.063, 0.073, 0.088, 0.127} [nat] is set to quantiles 30–95 of the PPL distribution on Dtr , following the general procedure described in Sec. 5.3. This gives |Λ| = 45 candidates. We use exact one-sided binomial p-values with Bonferroni correction at δ = 0.10, Rmin = 0.855, and max CD = 0.70. Table 2: GSM8K test-set results (Dtest ), means over 50 random splits; each policy evaluated at its own LTT-selected λ̂. R̂: empirical reward; ĈD : deferral rate; ĈL : mean thinking tokens. Hatched entries fail the joint LTT constraints. Bold: lowest ĈL among policies satisfying the joint constraints. Policy
R̂
ĈD
ĈL (tok) min
GSM8K (ntest =195); λ̂=(0.10, 0.088), R E-ReAct 0.852 0.000 Cloud ReAct 0.906 1.000 E-ReAct-TC (λL =0.10) 0.852 0.000 ReDAct-CD (λL =0) 0.864 0.082 TSDS @ λ̂ 0.890 0.255
16
max =0.855, CD =0.70 173 — 18 173 18
0.906
Ĉ D, test
R̂ test
0.864 0.852
0.86
0.6 0.4
0.84
0.26
0.2
0.82
R min Act
d CloAu ct eAct-TC eDAct-CD R Re E-R
TSDS
0.08
Act
E-Re
d CloAu ct eAct-TC eDAct-CD R Re E-R
125 100 75 50 25
0.00
0.0
Ĉ T (thought tokens)
0.8
0.890
0.90
E-Re
173
150
0.92
0.88
173
175
CDmax
1.00
1.0
0.94
TSDS
18 0
0
Act
E-Re
d CloAu ct eAct-TC eDAct-CD R Re E-R
TSDS
Figure 9: GSM8K test-set results (50-seed averages ± std, ntest =195 per split, δ = 0.10). Across 50 independent 85/15 splits of 1300 held-out problems, each method’s operating point λ̂ is selected by LTT on the calibration portion; all metrics are measured on the held-out test portion. Left: test-set reward; dashed line marks Rmin = 0.855. A bar above this line reflects empirical test performance, max not the LTT guarantee. Center: test-set deferral rate, CD = 0.70 dashed. Right: mean thinking tokens. E-ReAct-TC fails the LTT reward constraint in all 50 splits (hatched, shown for reference); TSDS and ReDAct-CD satisfy the joint constraints in their respective splits.
Results: TSDS reaches R̂test = 0.890, a 0.026 improvement over ReDAct-CD, while using just 18 thought tokens per problem, one-tenth of ReDAct-CD’s cost. ReDAct-CD retains the full thought trace (173 tokens) and defers on only 8% of problems, achieving R̂test = 0.864. E-ReAct-TC does not certify: without a deferral safety net, truncation-induced errors have no recovery path and the reward lower bound fails to clear the floor in all 50 splits. The elevated deferral rate of TSDS (26%) relative to ReDAct-CD (8%) reflects the thought-calibration–deferral synergy: aggressively truncated thoughts produce higher PPL, directing more problems to π cloud , with that deferral concentrated on the problems where truncated reasoning is most fragile. Cloud ReAct achieves R̂test = 0.906 but max defers unconditionally (CD = 1.0), violating CD = 0.70. Rollout quality: On 1300 test-split problems, full-thought accuracy is 84.5% for π edge and 90.2% for π cloud . The two models agree on 1026 problems, are both wrong on 54, only π edge is right on 73, and only π cloud is right on 147. An oracle deferral catching exactly the 147 “only-large-right” problems would lift accuracy from 84.5% to 95.8%. Baseline details: • E-ReAct (λL = λD = +∞): full thought, no deferral. • Cloud ReAct (λD = −∞): always defer; no Thought Calibration. • E-ReAct-TC (λD = +∞, λL = 0.10): truncate but never defer; excluded from Λ̂ because R̂cal = 0.839 < Rmin . • ReDAct-CD (λL = 0, λD = 0.088): full thought; λD selected via single-objective LTT; certifies in the 50-split analysis. G.3
H OTPOT QA: BAR C HART AND PARETO F RONTIER n.c.
Pareto frontier: Within each Pareto curve the frontier slopes downward: more aggressive truncation (lower ĈL ) produces more uncertain actions and therefore more deferral to π cloud , which max solves problems in fewer environment steps (3.11 vs. 4.70 for π edge ). At CD = 0.90 the budget is so permissive that all λL values converge on the same aggressive-deferral optimum, collapsing the frontier to a single point. G.4
C ODE G ENERATION : MBPP
Additional details for Sec. 5.2. Table 3 reports per-policy means. 17
Deferral rate
0.280
0.30
Ĉ D, test
R̂ test
R min = 0.22, CDmax = 0.70
0.305
0.35 0.226
0.25
0.6
0.48 0.33
0.4 0.2
R min Act
E-Re
d CloAu ct eAct-TC eDAct-CD R Re E-R
0.446
0.0
TSDS
Act E-Re
Ĉ D, test
R̂ test
d CloAu ct eAct-TC eDAct-CD R Re E-R
743
800 600 400
0
TSDS
1.00
0
Act E-Re
d CloAu ct eAct-TC eDAct-CD R Re E-R
0.243
0.226 0.185
0.2
0.6
0.49
0.4 0.2
0.1
R min d CloAu ct eAct-TC eDAct-CD R Re E-R
TSDS
0.0
0.12 0.00
Act E-Re
TSDS
1310
1200 1000 733
800
691
600 400 200
0.00
d CloAu ct eAct-TC eDAct-CD R Re E-R
1310
1400
CDmax
0.8
0.297
Act
1000
n.c.
0.4
E-Re
1312
1200
200 0.00
1.0
0.3
1310
1400
CDmax
0.8
0.40
0.20
R min = 0.12, CDmax = 0.70
Thinking cost
1.00
1.0
Ĉ T (thought tokens)
Reward
Ĉ T (thought tokens)
0.446
0.45
TSDS
0
0
Act E-Re
d CloAu ct eAct-TC eDAct-CD R Re E-R
TSDS
Figure 10: HotpotQA results under two constraint regimes, averaged over 50 random splits. Each panel shows test-set means and ±1 std across 50 independent 75/25 splits of 250 episodes; the max lines mark the bounds derived from Dcal , not thresholds on test-set values. Upper Rmin and CD min (R = 0.22): E-ReAct-TC (R̂cal =0.202) and E-ReAct (R̂cal =0.218) fall below the reward floor; max ; TSDS and ReDAct-CD satisfy the joint constraints. Lower (Rmin = Cloud ReAct exceeds CD 0.12): E-ReAct-TC also satisfies the reward constraint; TSDS selects a more aggressive λL =0.10, further reducing thinking cost with the deferral safety net. Columns: R̂, ĈD , ĈL . Hatched bars fail the joint constraints. Table 3: MBPP test-set results averaged over 50 random 60/40 calibration–test splits (ntest = 103 per split). R̂ is the empirical test reward, ĈD is the fraction of problems deferred to the cloud model, and ĈL is the mean number of thinking tokens per problem. Policies that violate either LTT constraint are shown with hatching; the bold entry is the LTT-selected policy with the fewest thinking tokens. R̂
ĈD
ĈL (tok)
0.719 0.810 0.620 0.779 0.714
0.000 1.000 0.000 0.786 0.346
1183 — 422 1206 422
Policy E-ReAct Cloud ReAct E-ReAct-TC (λL =0.80) ReDAct-CD (λL =0) TSDS @ λ̂
Multi-seed evaluation methodology: Because LTT controls the average over random calibration/test partitions, we report results averaged over Nseeds = 50 independent random 60/40 splits of the 257 MBPP test episodes. For each split we rerun the LTT procedure (fixing all other hyperparameters) and record whether TSDS certifies and its operating-point metrics. Reported means and standard deviations are across splits. TSDS certifies in 22/50 splits; across those 22 splits, mean R̂test = 0.714 ± 0.026 and mean ĈL = 422 ± 118 tokens. Non-certified splits contribute to the baseline averages reported in Fig. 2. Rollout details: Both π edge and π cloud are run with max thought= 2048. The 7B model achieves mean thought length 1166 tokens; the 32B model produces substantially longer traces. The probe is EMA at ℓ = 5, retrained on 930 probe samples from 100 MBPP train-split episodes. Uncertainty scores (PPL) are computed from the token probabilities of the generated Python function body. 18
CDmax = 0.65 CDmax = 0.45 CDmax = 0.25 CDmax = 0.90
5.00
Ĉ S (environment steps)
4.75 4.50 4.25 4.00 3.75 3.50 3.25 3.00
650
700
750
800
850
900
Ĉ T (thought tokens) max Figure 11: Pareto frontiers in (ĈL , ĈS ) space on HotpotQA for four deferral budgets CD (Rmin ≈ 0.10 throughout), averaged over 200 random 75/25 calibration/test splits. For each split and each certified λL , we record (ĈL , minλD ∈Λ̂(λL ) ĈS ); plotted points are means across splits; shaded bands max shifts the frontier upward and rightward: fewer allowed deferrals are ±1 std of ĈS . Tightening CD force more environment steps and exclude aggressive truncation from the certified set.
Baseline details: • E-ReAct: 119/154 wins on the seed-0 partition, binomial p-value 0.013 ≫ δ/|Λ| = 0.00222. • E-ReAct-TC (λL = 0.80, λD = +∞): 111/154 wins, p-value 0.21. max = 0.70. • ReDAct-CD (λL = 0, λD = 0.084): certifies on reward but ĈD,cal = 0.786 > CD max • Cloud ReAct: ĈD,cal = 1.000 > CD . • TSDS (λ̂ = (0.80, 0.084)): 123/154 wins, p-value 0.0018 < 0.00222, satisfying both constraints and receiving LTT certification. G.5
H OUSEHOLD ROBOT: E NVIRONMENT L AYOUT AND D IAGNOSTICS
Probe layer: On the household robot task, an ablation over ℓ ∈ {5, 8, 10, 14, 20} on Dtr shows the EMA probe reaches its best AUROC at ℓ = 10 (0.863) versus 0.830 at ℓ = 5. This differs from the other three benchmarks where ℓ = 5 is optimal, and confirms that the best probing layer is task-dependent rather than a fixed architectural default. Fig. 12 shows the floor plan of the simulated household environment used in Section 5.4. The environment consists of four rooms (living room, kitchen, bedroom, bathroom), each containing a fixed set of surfaces and appliances on which objects are placed at the start of each episode. The agent navigates between rooms via go to <room> and interacts with objects via pick up, put down, heat, and clean actions, aiming to complete one of four task families (fetch, place, clean, heat) within at most six steps.
H
T HOUGHT C ALIBRATION –D EFERRAL I NTERPLAY: P ROBE S IGNAL AND PPL ACROSS THE T HOUGHT T RACE
Fig. 13 offers empirical evidence of the thought-calibration–deferral interplay introduced in Section 1 and depicted schematically in Fig. 1(c), drawn from the Dtr episodes of the GSM8K experi19
Living Room
Kitchen
sofa
counter stove
coffee table
microwave refrigerator
bookshelf sink
Bedroom
Bathroom shelf bathtub
bed
dresser
sink nightstand
Surface / furniture
Appliance
Figure 12: Floor plan of the simulated household environment. Each room contains a set of surfaces and appliances (see legend); objects are placed on surfaces at episode initialisation. All four rooms are mutually accessible via the go to <room> action in a single step, irrespective of the agent’s current location. ment (Appendix G.2). We focus on the 22 episodes whose probe score at the first strided position is below 0.10, the “slow-converging” subset in which Thought Calibration and deferral interact most visibly, because both the convergence signal and the action uncertainty evolve gradually across the thought trace rather than snapping to their final values within the first few tokens. The top panel shows the EMA probe score rising from near zero to ≈0.87 over 192 thought tokens. Had Thought Calibration been applied with threshold λL =0.50, it would have halted reasoning at ≈90 tokens. The bottom panel reveals the consequence: at that crossing, mean action PPL is still ≈0.076 [nat], above λD =0.073, so the deferral rule would fire and the cloud model would be invoked. Allowing reasoning to continue to 192 tokens reduces mean PPL to ≈0.064 [nat], below λD , at which point the edge model’s action is confident enough to execute without deferral. This is precisely the tension that motivates joint calibration of λ (Section 4.3): reasoning thresholds that are too aggressive shorten thoughts but inflate the deferral rate, while overly conservative thresholds waste reasoning compute on problems the edge model could already handle confidently. The LTT procedure selects λ̂ to minimise thinking cost subject to both the reward floor and the deferral budget simultaneously, automatically trading off these competing pressures.
20
1.0
probe score
0.8
λT = 0.5
mean probe score
0.6 0.4 0.2
crossing at ≈ 90 tok
0.0
mean action PPL (smoothed) λD = 0.073 nats
0.12
action PPL (nats)
n = 22 episodes
0.10 0.08 0.06 0.04
PPL ≈ 0.076 > λD ⇒ defers
25
50
PPL ≈ 0.064 < λD ⇒ no deferral
75
100
125
thought tokens elapsed
150
175
200
Figure 13: Thought calibration–deferral interplay on the 22 slow-converging GSM8K probe-training episodes (those where the EMA probe score at the first strided position, p=16 tokens, is below 0.10), using the EMA probe at ℓ=5. Top: mean EMA probe score (±1 s.e.) as a function of absolute thought-token count. The score rises from near zero at 16 tokens, crosses λL =0.50 (dashed orange) at ≈90 tokens (grey dotted vertical line), and reaches ≈0.87 at 192 tokens. Bottom: mean PPL of the early-stopped action Ap,t (±1 s.e.) with a three-point moving-average overlay. At the crossing (≈90 tokens), mean PPL ≈0.076 [nat] is above the deferral threshold λD =0.073 (dashed red line): Thought Calibration early-stopping would trigger deferral at this point. By 192 tokens, mean PPL has fallen to ≈0.064 [nat], below λD , showing that had reasoning continued, the small model’s action would have been confident enough to avoid deferral entirely. The dotted vertical line marks this crossing: early-stopped thoughts that look fragile to the deferral mechanism would have been fine if allowed to run to completion.
21