T RUST THE R IGHT T EACHER : Q UALITY-AWARE S ELF -D ISTILLATION FOR GUI G ROUNDING Jingyuan Huang1,2∗ Zuming Huang2 Yucheng Shi3 Tianze Yang1 Xiaoming Zhai1 Wei Chu2 Ninghao Liu4 1 University of Georgia 2 INFLY Tech 3 Tencent AI Lab 4 The Hong Kong Polytechnic University
arXiv:2606.18101v1 [cs.AI] 16 Jun 2026
A BSTRACT Graphical user interface (GUI) grounding requires vision-language models (VLMs) to identify small target elements in high-resolution screenshots and predict precise screen coordinates. On-policy self-distillation (OPSD) is a promising post-training approach for this coordinate-sensitive task, since it provides dense token-level teacher signals beyond hard coordinate labels. However, naive OPSD is not well suited to GUI grounding: OPSD evaluates the teacher on studentgenerated prefixes, the quality of coordinate-token teacher signals can degrade when the prefix has already deviated from the target coordinate, leading to unreliable teacher signal. To mitigate this, We propose quality-aware self-distillation for VLM-based GUI grounding, which improves coordinate-token teacher-signal quality through soft correctness-aware gating and teacher-probability scaling. The soft correctness-aware gate checks whether the teacher’s current coordinate-token prediction can still be completed into the ground-truth box under the studentgenerated prefix. If not, the corresponding teacher signal is down-weighted. Teacher-probability scaling then uses the teacher’s confidence as a lightweight factor to further calibrate the strength of the gated supervision. A key empirical finding is that neither component alone improves overall performance, whereas combining them consistently improves performance. This suggests that the two mechanisms play complementary roles: correctness-aware gating suppresses unreliable coordinate-token supervision, while teacher-probability scaling calibrates the strength of the remaining signals. Experiments across six GUI grounding benchmarks show that our method consistently improves the base model and outperforms strong baselines.
1
I NTRODUCTION
GUI grounding is a fundamental capability for VLMs and agents that operate computers, mobile devices, and web applications (Cheng et al., 2024; Hong et al., 2024; Gou et al., 2025; Wu et al., 2024b). Given a screenshot and an instruction, the model must identify the intended interface element and output its screen coordinates (Cheng et al., 2024; Gou et al., 2025; Park et al., 2025). This task is especially challenging in high-resolution screenshots and complex GUI scenes, where target elements can be small, visually similar, and densely arranged (Hong et al., 2024; Li et al., 2025; Park et al., 2025). Existing post-training methods provide limited supervision for this coordinatesensitive task (Park et al., 2025; Zhou et al., 2025; Tang et al., 2025). SFT is simple and stable, but it treats the annotated coordinate as a hard target, providing little information beyond the final answer (Park et al., 2025). It does not exploit the teacher’s uncertainty or other “dark knowledge” over plausible coordinate tokens (Hinton et al., 2015), limiting the richness of supervision for finegrained localization. Reinforcement learning methods such as GRPO optimize task outcomes, but they require multiple rollouts and rely on sparse rewards (Shao et al., 2024; Zhao et al., 2026a). Such outcome-only supervision is costly and provides weak guidance for fine-grained localization (Tang et al., 2025; Park et al., 2025). On-policy self-distillation (OPSD) is a promising alternative to both GRPO and SFT. By training on teacher distributions along student-generated trajectories, OPSD provides dense token-level teacher ∗
This work was done during an internship at INFLY Tech.
1
signals without requiring the large number of rollouts used by GRPO-style methods (Zhao et al., 2026a). In principle, such soft supervision can carry richer information than hard-label SFT, including preferences among plausible locations (Hinton et al., 2015; Zhao et al., 2026a). However, a naive instantiation of OPSD is not well suited to GUI grounding. The effectiveness of OPSD depends critically on the quality of teacher signals. OPSD queries the teacher on prefixes generated by the student (Zhao et al., 2026a). Since GUI coordinates are produced autoregressively, an incorrect student prefix can already encode a wrong spatial hypothesis. Conditioned on such a prefix, the teacher’s subsequent logits may become a plausible continuation of the wrong coordinate rather than a useful teacher signal toward the true target. Thus, directly applying OPSD to GUI grounding can lead to unreliable coordinate-token teacher signals. To improve the quality of teacher signals, we propose quality-aware self-distillation for VLMbased GUI grounding, which calibrates coordinate-token supervision by combining two complementary components: soft correctness-aware gating and teacher-probability scaling. Specifically, soft correctness-aware gating exploits a special property of GUI grounding: coordinate predictions are spatially verifiable against the ground-truth bounding box (Cheng et al., 2024; Wu et al., 2024b; Tang et al., 2025). Under the current student-generated prefix, we regard a coordinate-token teacher signal as reliable if the teacher’s current coordinate-token prediction can still fall inside the groundtruth bounding box, and as unreliable if it can no longer fall inside the box. Soft correctness-aware gating assigns the full gate value to reliable signals and down-weights unreliable ones rather than discarding them. In addition, teacher-probability scaling uses the teacher probability of the top coordinate-token prediction as a lightweight scaling factor to further refine the gated teacher signal, assigning larger distillation weights to higher-probability teacher signals and softening lowerprobability ones. This combination makes coordinate-token supervision both correctness-aware and certainty-aware, preserving useful distributional information while reducing the negative impact of unreliable teacher signals. The contributions of this paper are as follows: • We propose quality-aware self-distillation for GUI grounding, a teacher signal quality-aware method that calibrates coordinate-token teacher signals according to their reliability and improves GUI grounding performance. • We use GUI grounding as a spatially verifiable setting to empirically study teacher signal reliability in on-policy self-distillation, especially how unreliable teacher signals should be treated during training. • We conduct comprehensive experiments on six GUI grounding evaluation sets and show that our method consistently improves the base model and outperforms strong post-training baselines, with ablation studies verifying the complementary effects of gating and scaling.
2
R ELATED W ORK
GUI Grounding. GUI grounding requires vision-language models to localize target interface elements and output precise screen coordinates given a screenshot and a natural-language instruction. Recent work has improved GUI grounding through post-training on GUI-specific data. Supervised fine-tuning methods train models with annotated instruction-coordinate pairs, as in SeeClick, CogAgent, UGround, OS-Atlas, and RVLM (Cheng et al., 2024; Hong et al., 2024; Gou et al., 2025; Wu et al., 2024b; Park et al., 2025). More recent reinforcement-learning-based methods further optimize GUI grounding with verifiable reward signals(Yang et al., 2026b), including GUI-G1 and GUI-G2 (Zhou et al., 2025; Tang et al., 2025). Some works further design distance-aware or continuous spatial rewards for GRPO-style GUI grounding, providing denser feedback according to how close the predicted coordinate is to the target region (Shi et al., 2026; Zeng et al., 2026; Tang et al., 2025; Zhao et al., 2026b). Self-Distillation and Teacher-Signal Reliability. OPSD trains the student on trajectories sampled from its own policy, while a teacher model provides token-level teacher signals on the same studentgenerated prefixes (Zhao et al., 2026a; Yuan et al., 2026). Prior On-Policy Distillation (OPD)/OPSD analyses point out that teacher signals are not uniformly reliable (Zhu et al., 2026; Zheng et al., 2026; Ke et al., 2026). In particular, when the teacher is conditioned on student-generated prefixes, these prefixes may be imperfect or distributionally mismatched, causing the teacher signal to become noisy or less informative (Zhu et al., 2026; Zheng et al., 2026). Other work further observes that 2
Figure 1: Overview of our proposed method. The signal acquisition process is simplified in this illustration; in practice, the signal is obtained by computing the reverse KL divergence between the probability distributions induced by the student and teacher model logits. directly exposing the full correct answer can make the privileged teacher signal overly sharp or neardeterministic, weakening the benefit of soft distillation (Tan & Hong, 2026; Zhang et al., 2026a). Methods to Improve Teacher-Signal Reliability. To mitigate these problems, several reliabilityaware OPD/OPSD methods improve teacher signals through proxy-based weighting or privileged teacher inputs. For example, entropy-based self-distillation methods use teacher uncertainty to adjust token-level update weights (Ke et al., 2026; Zhang et al., 2026a); perplexity-based OPD methods down-weight teacher guidance that appears unreliable (Zheng et al., 2026); and methods that combine self-distillation with verifiable feedback use outcome correctness to anchor update directions or route supervision paths (Yang et al., 2026a; Zheng et al., 2026). In vision-language settings, Vision-OPD constructs a crop-conditioned teacher to provide more focused teacher signals for a full-image student (Yuan et al., 2026). GUI-SD further adapts this idea to GUI grounding by constructing a visually enriched privileged teacher input with layout-preserving regional masking, while using coordinate-token weighting and entropy-based scaling to improve GUI self-distillation (Zhang et al., 2026a). These works establish an important point: naive self-distillation should not treat all teacher signals as equally trustworthy. However, existing criteria still mainly rely on indirect proxies to improve teacher signals’ quality. Indirect proxies such as entropy, teacher probability, or perplexity may correlate with signal reliability on average, but they do not provide a direct guarantee that the selected or emphasized signals are actually reliable. GUI grounding provides a natural opportunity to bridge this gap, because coordinate predictions are spatially verifiable. Our work leverages this structure and uses the ground-truth box as a direct training-time reliability criterion for coordinate-token teacher signals. Instead of blindly imitating unreliable teacher signals, our method softly down-weights and further scales their loss contribution before distillation.
3
M ETHODOLOGY
3.1
P RIVILEGED I NFORMATION C ONSTRUCTION
For each training example, we are given a GUI instruction x, a screenshot I, and a ground-truth bounding box B. The student is conditioned on the original input (x, I). During training, follow3
ing GUI-SD’s visually privileged input design, we additionally construct a privileged teacher input (x+ , I + ), where I + is obtained by layout-preserving regional masking: the target region is preserved and visually highlighted, while task-irrelevant regions are suppressed(Zhang et al., 2026a). The teacher-only prompt x+ indicates that the answer lies in the highlighted region and asks the teacher to answer the question. At inference time, only the original input (x, I) is available. 3.2
C ONSTRUCTING Q UALITY- AWARE T EACHER S IGNALS
The student samples an on-policy response y, and both student and teacher distributions are evaluated on the same student-generated prefix: y ∼ πθ (· | x, I), PSt = πθ (· | x, I, y<t ), PTt = sg πθ (· | x+ , I + , y<t ) . (1) Here, sg[·] denotes stop-gradient, so the privileged teacher distribution is used only as a teacher signal. The core of our method is to combine soft correctness-aware gating with teacher-probability scaling for coordinate-token supervision. This mechanism is applied only to coordinate tokens, i.e., coordinate digit tokens in the model response. Other response tokens, such as formatting tokens or non-coordinate text tokens, are distilled normally. This design focuses on coordinate-token supervision, which directly determines GUI grounding accuracy. Soft correctness-aware gating. For a coordinate-token position t, we first apply softmax to the privileged teacher’s logits produced under the current student-generated prefix, obtaining the teacher distribution PTt . Let D denote the set of coordinate digit tokens. We define d⋆t as the coordinate digit token with the highest teacher probability: d⋆t = arg max PTt (d). d∈D
(2)
The binary compatibility indicator ht then verifies whether this top coordinate-token prediction remains feasible for the corresponding coordinate axis under the current student-generated prefix. For a coordinate-token position t, let a(t) ∈ {x, y} denote its axis, and let Ba(t) denote the interval of the ground-truth bounding box B on axis a(t). We append d⋆t to the current axis-specific prefix induced by y<t , and check whether the remaining digits can still be completed into a valid coordinate value within Ba(t) . Thus, x-coordinate tokens are checked only against the x-axis interval of B, and y-coordinate tokens only against the y-axis interval. We define 1, if such a completion exists, ht = (3) 0, otherwise. This prefix-aware binary indicator identifies whether the teacher’s strongest coordinate-token prediction is compatible with the target region under the current student-generated prefix. Instead of using hard correctness-aware gating, which would discard failed-gate coordinate-token signals entirely, we convert the binary compatibility indicator into a soft correctness-aware gate: gt = α + (1 − α)ht .
(4)
Thus, compatible coordinate-token predictions receive gate value gt = 1, while incompatible coordinate-token predictions receive gate value gt = α rather than being discarded. In our main method, we set α = 0.5, so failed-gate coordinate-token signals are down-weighted by half. This soft correctness-aware gating strategy preserves potentially useful teacher signals while reducing the influence of unreliable coordinate-token supervision. Teacher-probability scaling. However, correctness-aware gating alone is still insufficient. The gate provides a prefix-aware judgment of spatial compatibility, but it does not measure the teacher’s uncertainty. Two teacher predictions may both pass the gate, while their distributions can have very different quality: a higher teacher probability usually reflects a clearer preference, whereas a lower teacher probability may be closer to a decision boundary and more likely to be affected by visual clutter, occlusion, or similar distractors (Hendrycks & Gimpel, 2018; Guo et al., 2017; Zheng et al., 4
2026; Ke et al., 2026). Therefore, for coordinate-token positions, we further scale the distillation strength by the teacher probability. We define pt = PTt (d⋆t ),
(5)
where pt is the probability assigned by the privileged teacher to its top-1 coordinate-token prediction. A larger pt indicates that the teacher assigns higher probability to the current coordinate token, so the corresponding distillation term should contribute more strongly; a smaller pt indicates higher uncertainty, so the teacher signal is softened even when it is spatially compatible with the target box. 3.3
W EIGHTED R EVERSE -KL O BJECTIVE
Combining the coordinate-token indicator, the soft correctness-aware gate, and the teacherprobability scaling term, we define the token-level distillation weight as wt = (1 − rt ) + rt gt λ pt .
(6)
Here, rt indicates whether position t is a coordinate token. If t is not a coordinate token, then rt = 0 and wt = 1, so the token is distilled normally. If t is a coordinate token, then rt = 1, and the loss contribution is controlled by the soft correctness-aware gate gt and the teacher probability pt . The scaling coefficient λ, a fixed scalar, further calibrates the overall contribution of coordinatetoken supervision. Since reliability-based weighting can reduce the aggregate loss mass assigned to coordinate tokens, λ prevents these decisive tokens from becoming under-emphasized in the training objective. λ is set as 3 in our main experiment. Equivalently, for coordinate and non-coordinate tokens, Eq. equation 6 gives rt = 0, 1, wt = λpt , rt = 1 and ht = 1, αλpt , rt = 1 and ht = 0.
(7)
Finally, we train the student with the resulting weighted reverse-KL objective over response tokens: X 1 Lours (θ) = Ey∼πθ (·|x,I) wt DKL PSt ∥ PTt . (8) |R(y)| t∈R(y)
Here, R(y) denotes the set of response-token positions. Prompt tokens are excluded from the loss. The expectation is over the on-policy student response y sampled from the student policy, as defined in Eq. equation 1. Eq. equation 6 yields three behaviors: (i) ordinary response tokens are distilled with weight 1; (ii) compatible coordinate tokens are distilled with weight λpt ; and (iii) incompatible coordinate tokens are down-weighted to αλpt . In our main method, we set α = 0.5 and λ = 3. The soft correctness-aware gating and teacher-probability scaling mechanism only modulates the loss contribution of each token-level KL term, thereby reducing blind imitation of unreliable coordinate-token supervision while still preserving useful teacher signals.
4
E XPERIMENTS
4.1
E XPERIMENTAL S ETUP
All experiments use Qwen3.5-9B as the backbone model. The training data follows the GUI-SD’s data construction, which is built based on ScaleCUA(Zhang et al., 2026a; Liu et al., 2025). We report results on six GUI grounding benchmarks: ScreenSpot-Pro (SSP) (Li et al., 2025), ScreenSpotv2 (Wu et al., 2024a), UI-Vision Element Grounding (UIEG) (Nayak et al., 2025), OSWorld-G, OSWorld-G-R (Xie et al., 2025), and MMBench-GUI L2 Element Grounding (MMG) (Wang et al., 2025). UIEG and MMG are GUI grounding subsets of their corresponding benchmarks (Nayak et al., 2025; Wang et al., 2025). 5
4.2
M AIN R ESULTS
Table 1 shows that our method achieves the best results across six benchmarks among all compared methods. Our method reaches 72.23 macro-average accuracy, outperforming the strongest baseline, the GUI-SD baseline, by 2.16 points. These results demonstrate that improving the quality and reliability of the teacher signal is an effective direction for GUI grounding OPSD. Table 1: Main results on six GUI grounding evaluation sets. Avg denotes macro-average accuracy, computed as the arithmetic mean over the six evaluation sets. Method
SSP
ScreenSpot-v2
UIEG
OSWorld-G
OSWorld-G-R
MMG
Avg
Qwen3.5-9B GRPO SFT Naive-OPSD GUI-SD Ours
63.00 63.50 65.15 66.41 64.71 68.37
91.75 91.90 93.32 93.87 94.97 95.68
26.92 28.36 33.33 41.87 42.29 43.24
61.35 61.52 62.06 59.75 63.65 66.49
67.73 68.26 71.10 65.60 70.04 72.34
80.41 81.64 83.59 85.98 84.76 87.26
65.19 65.86 68.09 68.91 70.07 72.23
Compared with the GUI-SD baseline, the key difference lies in how the teacher signal is weighted. GUI-SD strengthens coordinate-token supervision through digit-position weighting and entropybased scaling(Zhang et al., 2026a). However, these weights are not explicitly correctness-aware: a teacher signal can still be amplified even when it is inconsistent with the target coordinate or potentially harmful to the student. Our method instead calibrates the teacher signal according to signal reliability. By reducing the influence of unreliable teacher signals and emphasizing more reliable coordinate-token supervision, our method provides a higher-quality teacher signal for GUI grounding. Our ablation studies will further elaborate this. Our method also substantially outperforms SFT and GRPO baselines, improving the macro-average accuracy by 4.14 and 6.37 points, respectively. Compared with SFT, which mainly learns from hard target labels under teacher forcing, our self-distillation objective leverages teacher logits as soft supervision. Such soft targets contain richer “dark knowledge” beyond one-hot labels and can provide more informative training signals for the student (Hinton et al., 2015). Moreover, since our method supervises the student on its own generated prefixes, it better matches the autoregressive inference process and helps mitigate the exposure-bias problem, where training on ground-truth prefixes but testing on model-generated prefixes may lead to error accumulation (Bengio et al., 2015; Zhao et al., 2026a; Zhang et al., 2026b). Compared with GRPO, our method provides dense tokenlevel supervision from the teacher distribution, whereas GRPO mainly relies on sparse outcomelevel reward feedback (Shao et al., 2024; Zhang et al., 2026b). This dense supervision in our method is particularly beneficial for GUI grounding, where accurate coordinate prediction requires finegrained token-level learning signals. 4.3 4.3.1
A BLATION S TUDIES H IGH - LEVEL COMPONENT ANALYSIS
Table 2 studies the individual and combined effects of our two core components: soft correctnessaware gating and teacher-probability scaling. Starting from the Vision-PV-Only baseline, where the teacher is provided with only visual privileged information during self-distillation training, the model achieves 70.43 macro-average accuracy. Adding soft correctness-aware gating alone obtains 69.97 macro-average accuracy, while adding teacher-probability scaling alone obtains 70.19. These two single-component variants do not bring stable improvements across the benchmarks, and even decrease the overall macro-average accuracy compared with the Vision-PV-Only baseline. When both components are combined, the macro-average accuracy increases to 72.23, outperforming the Vision-PV-Only baseline by 1.80 points. A more detailed comparison on SSP reveals why the two components need to be combined. The Vision-PV-Only baseline obtains 67.49 on SSP. Adding only soft correctness-aware gating decreases the score to 67.11, and adding only teacher-probability scaling also decreases it to 67.24. This 6
Table 2: High-level component analysis. Method
Gating Scaling
SSP
Vision-PV-Only +Soft Gating +Probability Scaling +Gating & Scaling (Ours)
None Soft None Soft
67.49 67.11 67.24 68.37
None None 3pt 3pt
ScreenSpot-v2 UIEG OSWorld-G OSWorld-G-R MMG 94.10 94.50 94.18 95.68
41.74 41.58 41.19 43.24
63.65 61.88 63.65 66.49
70.04 69.68 70.74 72.34
85.57 85.05 84.16 87.26
Avg 70.43 69.97 70.19 72.23
suggests that using either component alone can introduce a mismatch: gating alone may downweight teacher signals that are still useful for training, while teacher-probability scaling alone may incorrectly amplify unreliable teacher signals. In contrast, our method reaches 68.37 on SSP and achieves the best overall macro-average accuracy. This indicates that soft correctness-aware gating and teacher-probability scaling are complementary: gating first reduces the influence of erroneous teacher signals, allowing teacher-probability scaling to emphasize reliable teacher signals with a lower risk of amplifying unreliable teacher signals. 4.3.2
E FFECT OF GATING STRENGTH
Table 3 studies the effect of gating strength while keeping the teacher-probability scaling rule fixed as 3 × pt . Without gating, the teacher-probability scaling-only variant achieves 70.19 macro-average accuracy. Hard correctness-aware gating, which removes failed-gate signals entirely, improves the result to 71.46. Our soft correctness-aware gating variant achieves the best macro-average accuracy, reaching 72.23. Table 3: Effect of gating strength. Method
Gating
Teacher-Probability Scaling Only None +Soft Correctness-Aware Gating(Ours) Soft, α = 0.5 +Hard Correctness-Aware Gating Hard, α = 0
Scaling
SSP
3pt 3pt 3pt
67.24 68.37 67.87
ScreenSpot-v2 UIEG OSWorld-G OSWorld-G-R MMG 94.18 95.68 95.05
41.19 43.24 43.11
63.65 66.49 64.54
70.74 72.34 72.52
84.16 87.26 85.67
Avg 70.19 72.23 71.46
These results suggest that effective teacher-signal filtering should not be purely binary. When no gating is applied, teacher-probability scaling adjusts the strength of teacher signals according to teacher probability, but this scaling is not ground-truth-aware, and may therefore incorrectly amplify teacher signals that are unreliable. Hard correctness-aware gating addresses this issue by removing failed-gate signals, but this strategy can be too aggressive. A key advantage of OPD-style training is that the teacher can provide corrective token-level feedback on prefixes generated by the student, thereby mitigating exposure bias and error accumulation in autoregressive generation (Arora et al., 2023; Agarwal et al., 2024; Zhao et al., 2026a). In our setting, once the student predicts an incorrect coordinate prefix such that no subsequent tokens can bring the final coordinate back to the target region, the following teacher signals will be judged as unreliable by the gating criterion. However, completely discarding these signals would remove the teacher’s corrective guidance on erroneous student states, even though they may still help the student learn how to recover from, or avoid, similar mistakes. Soft correctness-aware gating therefore provides a better compromise by downweighting, rather than discarding, failed-gate signals. This preserves potentially useful corrective information while reducing the influence of unreliable teacher signals. 4.3.3
E FFECT OF TEACHER - PROBABILITY SCALING
Table 4 evaluates the contribution of teacher-probability scaling while keeping soft correctnessaware gating and the fixed scaling coefficient unchanged. With the fixed scaling coefficient, the model achieves 71.12 macro-average accuracy. Further introducing teacher-probability scaling improves the result to 72.23, yielding a gain of 1.11 points. Table 4: Effect of teacher-probability scaling. Method
Gating
Fixed λ = 3 Soft, α = 0.5 + Probability scaling (Ours) Soft, α = 0.5
Coordinate-Token Weight
SSP
3(P ass)/1.5(F ail) 3pt (P ass)/1.5pt (F ail)
67.74 68.37
7
ScreenSpot-v2 UIEG OSWorld-G OSWorld-G-R MMG 94.73 95.68
41.03 43.24
65.43 66.49
71.99 72.34
85.81 87.26
Avg 71.12 72.23
This result shows that, even when soft correctness-aware gating and the fixed scaling coefficient are already applied, using teacher probability to further modulate the teacher-signal strength remains beneficial. Soft correctness-aware gating controls the reliability of teacher signals at a coarse level by down-weighting failed-gate cases, while the fixed scaling coefficient preserves the importance of coordinate tokens. However, the retained teacher signals can still vary in teacher probability and quality. Teacher-probability scaling provides an additional fine-grained calibration, based on the observation that higher teacher probability values are generally correlated with higher-quality teacher signals(Ke et al., 2026). Therefore, teacher signals with higher teacher probability receive larger distillation weights, while teacher signals with lower teacher probability are down-weighted. 4.3.4
E FFECT OF THE SCALING COEFFICIENT
Table 5 studies the effect of the scaling coefficient λ under the same soft correctness-aware gating setting. When λ = 1, the method achieves 71.20 macro-average accuracy. Increasing λ to 2 slightly improves the result to 71.32, and the best overall macro-average accuracy is obtained at λ = 3, reaching 72.23. Further increasing λ to 4 decreases the macro-average accuracy to 71.80. Table 5: Effect of the scaling coefficient. λ
SSP
ScreenSpot-v2
UIEG
OSWorld-G
OSWorld-G-R
MMG
Avg
1 2 3 (Ours) 4
67.49 67.68 68.37 69.07
94.97 95.05 95.68 95.52
42.31 42.07 43.24 42.76
64.72 65.25 66.49 65.78
71.45 71.99 72.34 71.81
86.25 85.87 87.26 85.88
71.20 71.32 72.23 71.80
These results indicate that the overall strength of coordinate-token supervision plays an important role in GUI grounding. Since soft correctness-aware gating and teacher-probability scaling suppress unreliable or low-confidence teacher signals, an additional coefficient is needed to preserve sufficient supervision on reliable coordinate tokens. However, the coefficient must be carefully calibrated. Although λ = 4 further improves the accuracy on SSP, it reduces the overall macro-average accuracy, suggesting that an overly large coefficient may harm the model’s general grounding ability. Notably, this variant even surpasses the best entry among models with fewer than 12B parameters on the official ScreenSpot-Pro leaderboard1 . In our experiments, λ = 3 provides the best trade-off between maintaining effective coordinate-token supervision and preserving robust performance across benchmarks.
5
D ISCUSSION AND L IMITATIONS
GUI grounding provides a concrete setting for examining how teacher signals should be used in onpolicy distillation when their quality can be explicitly verified. Unlike general token prediction tasks, GUI grounding has a spatially checkable structure: under a given decoding prefix, a coordinatetoken prediction can be tested by whether it remains possible to complete it into the ground-truth target region. This allows teacher supervision to be assessed not only through indirect proxies such as confidence or uncertainty, but also through its compatibility with the target constraint. Under this view, unreliable teacher signals should not be treated in a purely binary manner. A signal that is incompatible with the ground-truth region should not be imitated as strongly as a compatible one, since doing so may reinforce incorrect spatial predictions. However, such signals are not necessarily devoid of useful information; they may still reflect local preferences or distributional structure learned by the teacher. Therefore, our study explores a soft way of using teacher supervision under verifiable reliability: target compatibility is used to adjust the trust placed in the teacher signal, while probability calibration controls the strength of the supervision. This provides an initial attempt to make on-policy distillation more reliability-aware in GUI grounding, where imperfect teacher signals are weakened and reshaped rather than simply discarded. 1 https://huggingface.co/datasets/likaixin/ScreenSpot-Pro?leaderboard_ max_params=12B
8
Our method also has limitations. First, the correctness-aware gate relies on ground-truth bounding boxes during training, so it is most directly applicable when spatial annotations are available. Second, the current reliability criterion is designed for coordinate-token prediction in GUI grounding. Extending the same idea to tasks without explicit spatial coordinates may require different forms of verifiable teacher-signal assessment. Future work could also study whether similar reliability-aware self-distillation strategies transfer across model scales and other visually grounded agent tasks.
6
C ONCLUSION
We presented quality-aware self-distillation for GUI grounding, aiming to improve the reliability of coordinate-token teacher signals in on-policy self-distillation. Our method uses soft correctnessaware gating to down-weight teacher predictions that are incompatible with the target region under the student-generated prefix, and further applies teacher-probability scaling to refine the strength of coordinate-token supervision. Experiments on six GUI grounding benchmarks show that the proposed method consistently improves the base model and outperforms strong post-training baselines, including SFT, GRPO, naive OPSD, and GUI-SD. These results highlight spatial verifiability as an effective signal for improving teacher-signal reliability in GUI grounding self-distillation.
R EFERENCES Rishabh Agarwal, Nino Vieillard, Yongchao Zhou, Piotr Stanczyk, Sabela Ramos, Matthieu Geist, and Olivier Bachem. On-policy distillation of language models: Learning from self-generated mistakes, 2024. URL https://arxiv.org/abs/2306.13649. Kushal Arora, Layla El Asri, Hareesh Bahuleyan, and Jackie Chi Kit Cheung. Why exposure bias matters: An imitation learning perspective of error accumulation in language generation, 2023. URL https://arxiv.org/abs/2204.01171. Samy Bengio, Oriol Vinyals, Navdeep Jaitly, and Noam Shazeer. Scheduled sampling for sequence prediction with recurrent neural networks, 2015. URL https://arxiv.org/abs/1506. 03099. Kanzhi Cheng, Qiushi Sun, Yougang Chu, Fangzhi Xu, Yantao Li, Jianbing Zhang, and Zhiyong Wu. Seeclick: Harnessing gui grounding for advanced visual gui agents, 2024. URL https: //arxiv.org/abs/2401.10935. Boyu Gou, Ruohan Wang, Boyuan Zheng, Yanan Xie, Cheng Chang, Yiheng Shu, Huan Sun, and Yu Su. Navigating the digital world as humans do: Universal visual grounding for gui agents, 2025. URL https://arxiv.org/abs/2410.05243. Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q. Weinberger. On calibration of modern neural networks, 2017. URL https://arxiv.org/abs/1706.04599. Dan Hendrycks and Kevin Gimpel. A baseline for detecting misclassified and out-of-distribution examples in neural networks, 2018. URL https://arxiv.org/abs/1610.02136. Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network, 2015. URL https://arxiv.org/abs/1503.02531. Wenyi Hong, Weihan Wang, Qingsong Lv, Jiazheng Xu, Wenmeng Yu, Junhui Ji, Yan Wang, Zihan Wang, Yuxuan Zhang, Juanzi Li, Bin Xu, Yuxiao Dong, Ming Ding, and Jie Tang. Cogagent: A visual language model for gui agents, 2024. URL https://arxiv.org/abs/2312. 08914. Junlong Ke, Zichen Wen, Weijia Li, Conghui He, and Linfeng Zhang. Respecting self-uncertainty in on-policy self-distillation for efficient llm reasoning, 2026. URL https://arxiv.org/ abs/2605.13255. Kaixin Li, Ziyang Meng, Hongzhan Lin, Ziyang Luo, Yuchen Tian, Jing Ma, Zhiyong Huang, and Tat-Seng Chua. Screenspot-pro: Gui grounding for professional high-resolution computer use, 2025. URL https://arxiv.org/abs/2504.07981. 9
Zhaoyang Liu, Jingjing Xie, Zichen Ding, Zehao Li, Bowen Yang, Zhenyu Wu, Xuehui Wang, Qiushi Sun, Shi Liu, Weiyun Wang, Shenglong Ye, Qingyun Li, Xuan Dong, Yue Yu, Chenyu Lu, YunXiang Mo, Yao Yan, Zeyue Tian, Xiao Zhang, Yuan Huang, Yiqian Liu, Weijie Su, Gen Luo, Xiangyu Yue, Biqing Qi, Kai Chen, Bowen Zhou, Yu Qiao, Qifeng Chen, and Wenhai Wang. Scalecua: Scaling open-source computer use agents with cross-platform data, 2025. URL https://arxiv.org/abs/2509.15221. Shravan Nayak, Xiangru Jian, Kevin Qinghong Lin, Juan A. Rodriguez, Montek Kalsi, Rabiul Awal, Nicolas Chapados, M. Tamer Özsu, Aishwarya Agrawal, David Vazquez, Christopher Pal, Perouz Taslakian, Spandana Gella, and Sai Rajeswar. Ui-vision: A desktop-centric gui benchmark for visual perception and interaction, 2025. URL https://arxiv.org/abs/2503.15661. Joonhyung Park, Peng Tang, Sagnik Das, Srikar Appalaraju, Kunwar Yashraj Singh, R. Manmatha, and Shabnam Ghadar. R-vlm: Region-aware vision language model for precise gui grounding, 2025. URL https://arxiv.org/abs/2507.05673. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. Deepseekmath: Pushing the limits of mathematical reasoning in open language models, 2024. URL https://arxiv.org/abs/2402. 03300. Yucheng Shi, Wenhao Yu, Jingyuan Huang, Wenlin Yao, Wenhu Chen, and Ninghao Liu. Towards trustworthy gui agents: A survey, 2026. URL https://arxiv.org/abs/2503.23434. Zhiquan Tan and Yinrong Hong. Paint: Partial-solution adaptive interpolated training for selfdistilled reasoners, 2026. URL https://arxiv.org/abs/2604.26573. Fei Tang, Zhangxuan Gu, Zhengxi Lu, Xuyang Liu, Shuheng Shen, Changhua Meng, Wen Wang, Wenqi Zhang, Yongliang Shen, Weiming Lu, Jun Xiao, and Yueting Zhuang. Gui-g2 : Gaussian reward modeling for gui grounding, 2025. URL https://arxiv.org/abs/2507.15846. Xuehui Wang, Zhenyu Wu, JingJing Xie, Zichen Ding, Bowen Yang, Zehao Li, Zhaoyang Liu, Qingyun Li, Xuan Dong, Zhe Chen, Weiyun Wang, Xiangyu Zhao, Jixuan Chen, Haodong Duan, Tianbao Xie, Chenyu Yang, Shiqian Su, Yue Yu, Yuan Huang, Yiqian Liu, Xiao Zhang, Yanting Zhang, Xiangyu Yue, Weijie Su, Xizhou Zhu, Wei Shen, Jifeng Dai, and Wenhai Wang. Mmbench-gui: Hierarchical multi-platform evaluation framework for gui agents, 2025. URL https://arxiv.org/abs/2507.19478. Zhiyong Wu, Zhenyu Wu, Fangzhi Xu, Yian Wang, Qiushi Sun, Chengyou Jia, Kanzhi Cheng, Zichen Ding, Liheng Chen, Paul Pu Liang, and Yu Qiao. Os-atlas: A foundation action model for generalist gui agents, 2024a. URL https://arxiv.org/abs/2410.23218. Zhiyong Wu, Zhenyu Wu, Fangzhi Xu, Yian Wang, Qiushi Sun, Chengyou Jia, Kanzhi Cheng, Zichen Ding, Liheng Chen, Paul Pu Liang, and Yu Qiao. Os-atlas: A foundation action model for generalist gui agents, 2024b. URL https://arxiv.org/abs/2410.23218. Tianbao Xie, Jiaqi Deng, Xiaochuan Li, Junlin Yang, Haoyuan Wu, Jixuan Chen, Wenjing Hu, Xinyuan Wang, Yuhui Xu, Zekun Wang, Yiheng Xu, Junli Wang, Doyen Sahoo, Tao Yu, and Caiming Xiong. Scaling computer-use grounding via user interface decomposition and synthesis, 2025. URL https://arxiv.org/abs/2505.13227. Chenxu Yang, Chuanyu Qin, Qingyi Si, Minghui Chen, Naibin Gu, Dingyu Yao, Zheng Lin, Weiping Wang, Jiaqi Wang, and Nan Duan. Self-distilled rlvr, 2026a. URL https://arxiv.org/ abs/2604.03128. Tianze Yang, Yucheng Shi, Ruitong Sun, Jingyuan Huang, Ninghao Liu, and Jin Sun. Tron: Targeted rule-verifiable online environments for visual reasoning rl, 2026b. URL https://arxiv. org/abs/2606.01599. Qianhao Yuan, Jie Lou, Xing Yu, Hongyu Lin, Le Sun, Xianpei Han, and Yaojie Lu. Visionopd: Learning to see fine details for multimodal llms via on-policy self-distillation, 2026. URL https://arxiv.org/abs/2605.18740. 10
Xiangjian Zeng, Wenjing Li, Qingqiang Wu, and Liang Zhang. Fdc-ground: Improving grpo for gui grounding via exponential rewards and fact-aligned pruning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 40, pp. 28122–28130, 2026. Yan Zhang, Daiqing Wu, Huawen Shen, Can Ma, and Yu Zhou. Learn where to click from yourself: On-policy self-distillation for gui grounding, 2026a. URL https://arxiv.org/abs/ 2605.00642. Yan Zhang, Daiqing Wu, Huawen Shen, Can Ma, and Yu Zhou. Learn where to click from yourself: On-policy self-distillation for gui grounding, 2026b. URL https://arxiv.org/abs/ 2605.00642. Siyan Zhao, Zhihui Xie, Mengchen Liu, Jing Huang, Guan Pang, Feiyu Chen, and Aditya Grover. Self-distilled reasoner: On-policy self-distillation for large language models, 2026a. URL https://arxiv.org/abs/2601.18734. Yu Zhao, Wei-Ning Chen, Huseyin Atahan Inan, Samuel Kessler, Lu Wang, Lukas Wutschitz, Fangkai Yang, Chaoyun Zhang, Pasquale Minervini, Saravan Rajmohan, and Robert Sim. Learning gui grounding with spatial reasoning from visual feedback, 2026b. URL https://arxiv. org/abs/2509.21552. Binbin Zheng, Xing Ma, Yiheng Liang, Jingqing Ruan, Xiaoliang Fu, Kepeng Lin, Benchang Zhu, Ke Zeng, and Xunliang Cai. Scope: Signal-calibrated on-policy distillation enhancement with dual-path adaptive weighting, 2026. URL https://arxiv.org/abs/2604.10688. Yuqi Zhou, Sunhao Dai, Shuai Wang, Kaiwen Zhou, Qinglin Jia, and Jun Xu. Gui-g1: Understanding r1-zero-like training for visual grounding in gui agents, 2025. URL https: //arxiv.org/abs/2505.15810. Siqi Zhu, Xuyan Ye, Hongyu Lu, Weiye Shi, and Ge Liu. The many faces of on-policy distillation: Pitfalls, mechanisms, and fixes, 2026. URL https://arxiv.org/abs/2605.11182.
11
A
P ROMPT T EMPLATES , P RIVILEGED V ISUAL C UES , AND T RAINING TARGETS
A.1
S HARED S YSTEM P ROMPT Listing 1: Shared system prompt used by both the teacher and the student.
You may call one or more functions to assist with the user query. You are provided with function signatures within <tools> ... </tools> XML tags: <tools> {"name": "computer_use", "description": "Use a mouse to interact with a computer.", "notes": "Click with the cursor tip centered on targets ; avoid edges unless asked. Do not use other tools (type, key, scroll, left_click_drag). Only left_click are allowed.", "parameters ": {"type": "object", "required": ["action"], "properties": {"action ": {"type": "string", "enum": ["left_click"], "description": "The action to perform."}, "coordinate": {"type": "array", "description": "(x, y): pixels from left/top. Required for action=left_click."}}}} </tools> For each function call, return a JSON object with function name and arguments within <tool_call> ... </tool_call> XML tags: <tool_call> {"name": "<function-name>", "arguments": <args-json-object>} </tool_call>
A.2
S TUDENT AND T EACHER U SER P ROMPTS Listing 2: Student user prompt template.
<image> {original GUI instruction / query}
Listing 3: Teacher user prompt template with privileged hint. <image> {original GUI instruction / query} Hint: The answer is located within the green rectangle.
Listing 4: Example teacher user prompt. <image> 5:20 PM Hint: The answer is located within the green rectangle.
A.3
T RAINING TARGET F ORMAT
The model is trained to output a structured tool call. In our main experiments, the target response contains no additional natural-language reasoning or rationale. The canonical output format is: Listing 5: Canonical training target format. <tool_call> {"name": "computer_use", "arguments": {"action": "left_click", " coordinate": [x, y]}} </tool_call>
12
(a) Teacher input with visual privileged information.
(b) Student input without visual privileged information.
Figure 2: Visualization of privileged and non-privileged inputs. The teacher receives an augmented image in which the target region is marked by a green rectangle, together with a textual hint. The student receives only the original GUI image and the original user instruction.
B
C ONSTRUCTION OF V ISUAL P RIVILEGED I NFORMATION
B.1
T EACHER V ISUAL P RIVILEGED I NFORMATION
Following GUI-SD (Zhang et al., 2026a), we provide the teacher with target-aware visual privileged information during training. Given the original GUI screenshot I and the ground-truth target bounding box b, we construct a Gaussian soft mask around the target region. Let db (u, v) denote the Euclidean distance from pixel (u, v) to the bounding box b, where pixels inside the box have distance 0. The masked image is computed as db (u, v)2 α(u, v) = exp − , Imask (u, v) = α(u, v)I(u, v). 2σ 2 Here, σ controls the spatial decay of the Gaussian mask. This operation keeps the target region fully visible while softly suppressing background regions farther away from the target. In addition, we draw a green rectangle around the ground-truth target region and append a short textual hint indicating that the answer is located inside the rectangle. These visual and textual cues are used only for the teacher during training. The student always receives the original GUI screenshot and the original instruction, without any privileged visual cue. B.2
V ISUALIZATION OF P RIVILEGED AND N ON -P RIVILEGED I NPUTS
Figure 2 shows an example of the teacher and student inputs. The teacher image contains the visual privileged information, while the student image remains the original GUI screenshot.
C
T RAINING D ETAILS
We provide the training hyperparameters used for our self-distillation experiments in Table 6. We perform online self-distillation using an EMA teacher. The teacher is initialized from a legacy model and updated after each successful optimizer step according to θteacher ← 0.95 θteacher + 0.05 θstudent . Distillation is applied over the full vocabulary with weight α = 1.0. C.1
BASELINE -S PECIFIC T RAINING D ETAILS
Unless otherwise specified, we evaluate all methods after one epoch of training using the final checkpoint. Under our training setup, one epoch of supervised training corresponds to 62 optimization steps. In contrast, GRPO requires substantially more updates and reaches over 400 training steps per epoch. For a fair comparison under a similar training budget, we report the GRPO result using 13
Table 6: Training hyperparameters for self-distillation. Hyperparameter
Value
Training setup Number of GPUs(H800) Number of nodes Total epochs Final evaluated step Train batch size PPO mini-batch size
8 1 1 62 112 112
Optimization Optimizer Learning rate Learning-rate scheduler Learning-rate warmup steps Adam betas Weight decay Gradient clipping
AdamW 1.25 × 10−6 Constant 8 (0.9, 0.999) 0.01 1.0
Self-distillation Full-logit distillation Distillation weight α Distillation top-k Distillation tail correction Distillation clipping Teacher source Teacher always on Teacher regularization Teacher EMA decay Student mixing coefficient Teacher update rule Teacher update frequency Teacher prompt mode Include environment feedback Do not reprompt on self-success Maximum reprompt length Teacher entropy weighting
True 1.0 None; full vocabulary True None Legacy model True EMA 0.95 0.05 θteacher ← 0.95 θteacher + 0.05 θstudent After each successful optimizer step None False True 10,240 None
the checkpoint at step 62 in the main results. For Naive-OPSD, the teacher is provided with the ground-truth text bounding box as privileged information during OPSD training. For GUI-SD, we directly adopt the training hyperparameters reported in its original paper.
14