UXBench: Measuring the Actionability of LLM-Generated UX Critiques Wenjie Wang1 , Yue Huang1 , Zipeng Ling2 , Han Bao1 , Hang hua3 , Xiaonan Luo1 , Yu Jiang1 , Shiyi Du4 , Yuexing Hao5 , Xiaomin Li6 , Yuchen Ma7 , Dianzhuo Wang6 , Yanfang Ye1 and Xiangliang Zhang*1 1 University of Notre Dame, 2 University of Pennsylvania, 3 University of Rochester, 4 Carnegie Mellon
University, 5 Massachusetts Institute of Technology, 6 Harvard University, 7 LMU Munich
arXiv:2606.16262v1 [cs.SE] 15 Jun 2026
* Corresponding author: [email protected].
§ Code
Project Page
Abstract Large language models (LLMs) are increasingly deployed as UX judges that inspect interfaces, diagnose usability problems, and propose repairs. Yet no controlled benchmark measures whether the resulting critiques are reliable and actionable across heterogeneous product surfaces. We introduce UXBench, a benchmark for evaluating LLMs as interaction-grounded UX judges. UXBench comprises local-first runnable web fixtures spanning ten product-surface families, paired with coverage-gated browser exploration that forces models to collect interaction evidence before reporting. Each judge model produces a structured UX report over seven rubric dimensions; report quality is measured by whether a fixed downstream repair agent can improve the interface based on the critique. We evaluate eight frontier models under both an automated repair-lift protocol and a blind human validation study. Results show that UX judging is neither saturated nor onedimensional: models differ meaningfully in report actionability, exhibit distinct rubric-level repair signatures, vary in fixture-level reliability, and trade leadership across surface categories.
1 Introduction LLMs are increasingly used not only to generate code and interface mockups but also to review existing interfaces, diagnose usability problems, and propose repairs (Beltramelli, 2017; Si et al., 2025; Wu et al., 2024b; Duan et al., 2024). This shift makes UX judgment a natural target for model evaluation (Liu et al., 2023; Zheng et al., 2023). Recent work further extends this direction from static UI generation and critique to LLM-assisted usability testing, simulated user studies, and automated UX flaw detection (Lu et al., 2025b; Calvano et al., 2025; Gao et al., 2026). A useful judge should browse a page, recognize when users cannot infer the current state or next action, identify missing feedback or recovery paths, and communicate these issues in a form developers can act on (Parlakkılıç, 2021; Shneiderman, 1983). Yet it remains unclear whether current frontier models are reliable UX judges, or whether models that appear similar in general capability produce equally actionable critiques. As illustrated in Figure 1, we study UX judging as an interaction-grounded evaluation problem: both human and LLM judges must inspect the running interface, exercise controls, observe feedback, and turn the resulting evidence into actionable critique.
1 / 30
UXBench
Evaluating this capability is difficult beHuman Judge LLM Judge cause web UX is not fully visible from a screenshot or a static DOM snapshot. Many usability failures are interactional: a disabled control may have no explanation, a form may silently reject input, a destructive action may hide its consequences, or a mobile layout may collapse into an unusable state. Recent studies of digital forms and responsive interfaces show that usability depends on input organization, vali- Figure 1. Interaction-grounded UX judging: like hudation behavior, editing flow, and cross-device man reviewers, LLM judges should inspect the runlayout behavior, rather than only on the first ning interface, exercise controls, and ground critiques rendered view (Iftikhar et al., 2020; Breitmayer in observed feedback rather than static impressions. et al., 2024; Parlakkılıç, 2021). Accessibility work similarly treats labels, error identification, focus behavior, status messages, and dynamic content changes as properties that must remain perceivable during interaction (World Wide Web Consortium (W3C), 2024; Mehralian et al., 2025). A judge that only inspects the first rendered view can therefore produce fluent but weakly grounded criticism. Reliable UX judging requires evidence collection: the model must exercise controls, observe state transitions, test error and recovery paths, and tie each finding to concrete behavior in the running interface. Existing benchmarks only partially cover this setting. Web and GUI agent benchmarks typically measure whether an agent can complete user tasks in a browser or operating-system environment (Deng et al., 2023; Zhou et al., 2024; Koh et al., 2024; He et al., 2024; Xie et al., 2024; Cheng et al., 2024), while UI generation benchmarks evaluate whether models can synthesize interfaces from textual or visual specifications (Beltramelli, 2017; Si et al., 2025; Yun et al., 2024). Screenshotlevel preference and design-evaluation setups can compare visual appeal or layout quality, or region-level design critiques (Wu et al., 2024a; Duan et al., 2024), but they do not isolate whether a model can produce interaction-grounded UX reports whose recommendations lead to better interfaces. As a result, the community lacks a controlled benchmark for measuring the actionability and reliability of LLM-generated UX critique across the heterogeneous product surfaces that users encounter in practice. We introduce UXBench, a benchmark for evaluating LLMs as interaction-grounded UX judges. UXBench consists of local-first runnable web fixtures implemented as static HTML/CSS/JavaScript bundles, removing live-site drift from A/B tests, personalization, backend failures, and third-party outages. The benchmark spans ten surface families—from landing pages and checkout flows to dashboards, chatbot interfaces, and mobile micro-UIs. Within each family, UXBench pairs realproduct anchors with independently authored synthetic siblings: anchors preserve recognizable interaction patterns, while siblings vary branding, text, layout, and visual identity so that models must judge the interface in front of them rather than recall memorized impressions of familiar products. Each UXBench run evaluates a judge model through a coverage-gated browser trajectory. The model prescans the fixture, forms an exploration plan, then repeatedly observes the rendered page, takes user-like actions, and inspects the resulting feedback. A coverage gate prevents premature termination: if the model attempts to stop before collecting sufficient evidence, the environment returns the unmet conditions and resumes exploration. After the trajectory, the model produces an Click
2 / 30
Type
Scroll
UXBench
evidence-grounded UX report over seven rubric dimensions. To measure whether the critique is actionable rather than merely plausible, UXBench passes the report to a fixed repair agent that edits the fixture while preserving the original product intent, brand identity, and interaction semantics. The repaired interface is then scored under a fixed evaluator, turning report actionability into a comparable signal. We evaluate eight frontier models with two complementary protocols: an automated sweep and a blind human validation study. Across both, UXBench reveals that UX judging is neither saturated nor one-dimensional: repair lift varies meaningfully across models, rubric-level signatures diverge, fixture-level reliability differs even among similarly ranked models, and the leading model changes across product-surface categories. Our contributions are threefold: 1 We formulate LLM-based UX judging as an interaction-grounded, report-conditioned repair evaluation problem, where the quality of a judge is measured by whether its evidence-backed critique supports downstream interface improvement. 2 We introduce UXBench, a local-first suite of runnable web fixtures spanning ten productsurface families, with real anchors, synthetic siblings, coverage-gated browser exploration, and evidence-grounded UX reporting. 3 We evaluate eight frontier models under automated and human protocols, showing that UX judge models differ in report actionability, rubric-level strengths, fixture-level reliability, and surface-conditioned competence.
2 Related Work 2.1 Web and GUI Agent Benchmarks Recent web and GUI agent benchmarks evaluate whether language and multimodal agents can perceive interface state, act in interactive environments, and complete user-specified tasks. Early controlled platforms such as World of Bits and MiniWoB/MiniWoB++ established web interaction as a measurable learning problem, while later benchmarks extend the setting to realistic websites, visual web environments, enterprise software, desktop operating systems, and mobile devices. Collectively, Mind2Web, WebArena, VisualWebArena, WebVoyager, SeeClick, OSWorld, WorkArena, BrowserGym, WebLINX, Android in the Wild, AndroidWorld, and GUIOdyssey measure abilities such as instruction following, multi-step navigation, visual grounding, desktop control, conversational web interaction, and cross-app mobile operation (Shi et al., 2017; Liu et al., 2018a; Deng et al., 2023; Zhou et al., 2024; Koh et al., 2024; He et al., 2024; Cheng et al., 2024; Xie et al., 2024; Drouin et al., 2024; Chezelles et al., 2025; Lù et al., 2024; Rawles et al., 2025, 2023; Lu et al., 2025a). These benchmarks make interaction a central evaluation object, but their success criteria are usually task completion, final-state correctness, or action accuracy. In contrast, UXBench studies a different capability: the model is not asked to complete a user task, but to inspect a running interface, gather evidence about usability failures, and produce findings that can support downstream interface repair.
2.2 UI Generation and Automated Design Another line of work studies interface understanding, UI generation, and automated design evaluation. Large-scale UI datasets and design-search systems such as Rico, learned mobile design semantics, WebUI, Gallery D.C., and VINS support component understanding, semantic annota3 / 30
UXBench
tion, retrieval, visual search, and data-driven design assistance (Deka et al., 2017; Liu et al., 2018b; Wu et al., 2023; Chen et al., 2019; Bunian et al., 2021). UI generation methods and benchmarks, including pix2code, Design2Code, Web2Code, UICoder, and DCGen, further evaluate whether models can synthesize interface code from screenshots, webpages, text specifications, or multimodal prompts (Beltramelli, 2017; Si et al., 2025; Yun et al., 2024; Wu et al., 2024b; Wan et al., 2025). Closer to evaluation, UIClip and UICrit assess visual design quality, screenshot–description alignment, or region-level design critiques (Wu et al., 2024a; Duan et al., 2024). This work shows that models can reason over UI structure, appearance, and code, but the evaluation target is often the generated artifact or a static visual/design judgment. In contrast, UXBench evaluates critiques produced after browser interaction: missing feedback, silent validation, hidden consequences, weak recovery paths, and other UX failures are treated as interactional evidence that should be observed, reported, and tested through report-conditioned repair.
2.3 LLM-Assisted Usability Testing Recent work has begun to use LLMs and multimodal agents for usability testing, heuristic evaluation, accessibility testing, and simulated user studies. Earlier automated-usability-evaluation research surveyed how software tools can support usability inspection, testing, inquiry, analytical modeling, and simulation, while also emphasizing the difficulty of automating context-sensitive interface judgment (Ivory and Hearst, 2001). Newer LLM-based systems revisit this problem with stronger language and UI-understanding capabilities: UXAgent simulates usability testing with LLM agents, AXNav replays natural-language accessibility tests through assistive-technology navigation, and recent studies explore LLM-based UX testing, heuristic evaluation, and comparison with human experts (Lu et al., 2025b; Taeb et al., 2024; Hsueh et al., 2024; Guerino et al., 2025). These works are closest to UXBench because they treat usability evaluation as an interactive or expert-like process rather than a purely visual judgment. UXBench differs mainly in its evaluation framing: instead of using LLMs as open-ended testing assistants, it fixes the fixtures, exploration protocol, repair agent, and scorer to compare judge models by whether their evidence-grounded reports produce measurable interface improvement.
3 Benchmark Construction Figure 2 provides an overview of the UXBench construction and evaluation pipeline. The benchmark is built around four connected components: local-first fixture construction, coverage-gated browser exploration, evidence-grounded UX reporting, and report-conditioned repair.
3.1 Fixture Design and Catalog Local-First Fixtures. UXBench consists of local-first web fixtures implemented as static HTML/CSS/JavaScript bundles, each servable from a local file server without backend dependencies. Pinning the interface in this way removes the major sources of drift that confound live-site evaluation (e.g., A/B tests, personalization, and third-party outages), so that any score difference between models reflects the models themselves rather than the environment in which they were evaluated. Surface Categories. Rather than building one or two deep applications, UXBench prioritizes breadth across the kinds of product surfaces web users routinely encounter (e.g., marketing, transactional, operational, and mobile surfaces; the full taxonomy is given in Table 3). Different surfaces stress qualitatively different UX competences: a judge that handles one family well (e.g., 4 / 30
UXBench
1
Data Construction
2
Anchor (Real Product)
Agent Execution Flow Observe - Act - Inspect
Trace Linked Findings
Map structure, goals, and risks
Execute interactions guided by plan & coverage gates
Detailed report inspected from trajectories
Products Acme Plan
Trajectory Example
$29 / mo
Pro Plan $59 / mo
Team Plan
Synthetic Siblings
$99 / mo
Checkout [email protected] Select
Card number
4242 4242 4242 4242
Select
...
3
10 Surface Families Pricing
Onboarding
Booking
Dashboard
Docs
Privacy
Visualize
Chatbot
Mobile
Reporting Agent
Confirm
Rubric
Goal-state Navigation Action Clarity Scent Feedback
Unclear CTA on pricing
Select
Pay $29.00
Landing
4
Prescan & Plan
You’re all set! A receipt has been sent to [email protected].
Go to dashboard
Success message buried below fold Missing inline validations on card number
Flow Error Efficiency Recovery
Trust Transp.
Scanability Accessibility
Evidence Grounded Reporting and Repair
Unclear next step on pricing page No validation for card number Delete consequence not surfaced Success messages hidden below the fold
...
✓ Card number looks good
Original
Repaired
Repair Agent
Figure 2. Overview of UXBench. Local-first real anchors and synthetic siblings are evaluated through coverage-gated browser exploration. Trace-linked findings are converted into evidence-grounded UX reports, which are then used for report-conditioned repair and scored with the same seven-dimensional UX rubric.
a static marketing page) may still misread another (e.g., a transactional flow whose failures only emerge mid-interaction). Anchors and Synthetic Siblings. Within each surface category, UXBench pairs one or more real-product anchors with several independently authored synthetic siblings (Table 3). The anchor grounds the category in a recognizable interaction pattern (e.g., the structure of a familiar checkout or dashboard), while the siblings preserve that interaction model but vary the surface (e.g., re-skinning the visual identity and rewriting the textual content). The anchor supplies external validity, while the siblings test whether the judge evaluates the interface currently in front of it rather than memorized impressions of well-known products. More details are provided in Appendix B.
3.2 Agent Execution Flow Overview. A single UXBench run proceeds in four stages: 1 the judge agent executes a lightweight prescan and produces an exploration plan; 2 it enters a closed-loop observe–act trajectory in a real browser; 3 on termination it emits an evidence-grounded UX report; and 4 the report is consumed by a fixed code-editing agent to perform localized repair on the fixture source. Prescan and Planning. Each run begins with a prescan that compiles a brief structural summary of the site (e.g., visible pages and salient controls) and converts it into an explicit exploration plan stating what the agent intends to verify and when it may stop. The plan is adaptive, not scripted: the agent re-routes when feedback is ambiguous (e.g., switching to a recovery path or revisiting under a mobile viewport) rather than following a fixed click sequence. Browser-Based Trajectory. Following the prescan and plan, the agent evaluates the runnable interface by repeatedly observing the current page state, taking a user-like action, and inspecting the resulting feedback. Interactive evaluation is necessary because many UX failures only emerge during interaction (e.g., silent form validation or a disabled control with no visible response). Formally, at step t the agent receives an observation ot of the current page (combining textual, structural, and visual cues, e.g., rendered text, a screenshot, and any runtime warnings) and selects an action at (e.g., a click, a text entry, or a viewport switch), yielding an interaction trajectory τ = ( o0 , a0 , o1 , . . . , o T ). 5 / 30
UXBench
Algorithm 1 Coverage-Gated Exploration Require: fixture profile P (HTML pages, prescan controls), plan Π (phases, target pages), step budget B, depth d∈{quick, std, deep, exh}, viewport mode v∈{desk, mob, both} 1: open target URL; o0 ← Observe(); τ ← [o0 ] 2: for t = 0, . . . , B − 1 do 3: C ← CoverageState( P, Π, τ, ot , d, v) 4: at ← Brain(ot , τ, Π, C ) ▷ judge model proposes action 5: if at = finish and C.stop_allowed=⊥ then ▷ retry with unmet coverage 6: at ← Brain(ot , τ, Π, C, must_continue=C.reason) 7: end if 8: if at = finish and C.stop_allowed=⊥ then ▷ deterministic continuation 9: at ← Fallback(C ): open an unvisited page, else exercise an unexplored control 10: end if 11: if at = finish then break 12: end if 13: execute at ; ot+1 ← Observe() 14: append at and ot+1 to τ 15: end for 16: C ← CoverageState( P, Π, τ, LastObs(τ ), d, v) 17: return τ, EvidenceConfidence(τ, C ) ∈ {low, med, high}
UXBench treats this trajectory as the primary evidence source: reported issues are expected to correspond to concrete interaction events (e.g., a state transition that occurred without feedback) rather than to unsupported visual impressions.
3.3 Evidence-Grounded Reporting and Repair After exploration, UXBench produces a final UX report using the accumulated trajectory, coverage summary, and final browser state. Each metric-level judgment must be supported by concrete trace evidence, so a finding is considered valid only when it can be linked to an observed interaction event. This makes the report auditable rather than merely rhetorical. To further test whether the critique is actionable, UXBench performs report-conditioned repair. Given the ranked findings and the fixture source code, we invoke Claude Code as an independent code-editing agent with a restricted file-editing toolset. The original interface, brand identity, and interaction intent are specified as invariants, ensuring that any improvement in repaired UX quality reflects the actionability of the generated report rather than an unconstrained redesign.
3.4 Coverage-Gated Termination The trajectory loop terminates only when the collected evidence is substantive enough to back the report. UXBench enforces this with a coverage gate: at each step, the system recomputes a compact coverage state that operationalizes evidential sufficiency over the interaction trace and the current plan (e.g., whether the agent has actually exercised the salient controls of the page, not merely visited it). If the judge proposes to stop before this threshold is reached, the environment rejects the finish decision and returns the unmet conditions, pushing the agent back toward under-explored regions. The gate therefore constrains only evidence collection, not the downstream report or repair stage, and its role is to ensure that final UX findings are grounded in observed interaction consequences rather than in superficial browsing. Algorithm 1 summarizes this loop; detailed algorithm is provided in Appendix A. 6 / 30
UXBench
Overall
Per-dimension (∆)
Base.
Repaired
∆
Goal
Nav
Fdbk
Flow
Err
Trust
Scan
GPT 5.4
3.27
3.49
+0.22
+0.13
+0.15
+0.36
+0.05
+0.36
+0.21
+0.26
Kimi k2.5
3.27
3.48
+0.21
+0.12
+0.17
+0.44
+0.05
+0.12
+0.27
+0.29
Sonnet 4.6
3.27
3.45
+0.19
+0.15
+0.05
+0.35
+0.03
+0.28
+0.15
+0.35
GPT Mini
3.27
3.45
+0.19
+0.13
+0.13
+0.38
+0.05
+0.18
+0.13
+0.33
GLM 5.1
3.27
3.45
+0.17
−0.02
+0.17
+0.32
+0.05
+0.20
+0.20
+0.32
GPT Nano
3.27
3.44
+0.17
+0.07
+0.05
+0.34
+0.10
+0.20
+0.05
+0.37
Qwen 3.6
3.27
3.42
+0.15
+0.02
+0.07
+0.24
+0.12
+0.07
+0.12
+0.39
Gemini 3.1
3.27
3.41
+0.14
+0.08
+0.03
+0.24
+0.08
+0.11
+0.11
+0.34
Model
Table 1. Overall and per-dimension repair lift across the eight evaluated models, sorted by ∆. Bold indicates the column maximum. For compactness, tables abbreviate Claude-Sonnet-4.6 as Sonnet 4.6, GPT-5.4-Mini as GPT Mini, GPT-5.4-Nano as GPT Nano, Qwen-3.6-Plus as Qwen 3.6, and Gemini-3.1-Pro as Gemini 3.1.
4 Experimental Setup 4.1 Protocols UXBench evaluates judge models using two complementary protocols: an automated LLM-asjudge sweep and a blind human validation study. Automated sweep. Each evaluated model drives the same coverage-gated exploration agent over the runnable fixtures in Table 3 under a fixed exploration budget, producing one complete evaluation run per model–fixture pair. Each run is converted into an evidence-grounded UX report and scored by a fixed scoring judge, GPT-5.4-Mini. The fixed scoring judge is blinded to the identity of the judge model that produced each repair report. Human evaluation. To validate whether autoReal anchors (IQR) Synthetic siblings (IQR) Mean mated repair-lift rankings align with human UX judgment, we conduct a blind expert review of the repaired candidate interfaces. We recruit A11y expose six participants with UX or front-end design Coupling experience. For each fixture, reviewers inspect the rendered repaired webpage produced from an anonymized judge report and rate the inter- IA breadth face on the same seven rubric dimensions, with 0 20 40 60 80 100 model identities, presentation order, and autoScore mated scores hidden throughout. Reviewers Figure 3. Structural characterization of real anchors may revisit reports within the same group be- and synthetic siblings in UXBench. A11y denotes fore submission, allowing local recalibration accessibility, and IA denotes information architecture. across candidates.
4.2 Scoring Rubric UXBench scores each fixture along the seven default dimensions in Table 5. The rubric reuses constructs measured by the standard family of validated UX questionnaires (e.g., SUS, UEQ,
7 / 30
UXBench
Model
Land.
Price
Onbd.
Book.
Dash
Docs
Privacy Visual. Chatbot Mobile
GPT 5.4
0.667
0.714
0.375
0.411
0.583
0.738
0.589
0.333
0.417
0.446
Kimi k2.5
0.637
0.518
0.429
0.661
0.381
0.637
0.464
0.786
0.643
0.286
Sonnet 4.6
0.476
0.196
0.536
0.393
0.589
0.690
0.250
0.381
0.851
0.411
GPT Mini
0.452
0.661
0.521
0.536
0.292
0.286
0.643
0.571
0.500
0.536
GLM 5.1
0.411
0.393
0.543
0.536
0.625
0.560
0.554
0.595
0.417
0.714
GPT Nano
0.429
0.804
0.664
0.768
0.476
0.250
0.518
0.262
0.482
0.536
Qwen 3.6
0.548
0.500
0.461
0.482
0.363
0.315
0.571
0.524
0.381
0.571
Gemini 3.1
0.381
0.214
0.471
0.214
0.690
0.524
0.411
0.548
0.310
0.500
Table 2. Category-level pairwise win rates across the ten UXBench surface families. Land., Price, Onbd., Book., and Dash denote Landing, Pricing, Onboarding, Booking, and Dashboard, respectively; Visual. denotes visual design. Each cell reports a model’s mean pairwise win rate against the other seven evaluated models within the same surface family. Bold indicates the highest win rate in each category.
and NASA-TLX; details in Appendix C), but re-grounds each construct as a criterion that can be answered from the interaction trace rather than from a post-task self-report. Each metric is rated on a 1–5 ordinal scale (5 indicates no meaningful issue, 1 a blocking failure), and the rubric score is the equally weighted mean of its metrics.
4.3 Dataset Characterization. UXBench contains 41fixtures: 11 real anchors and 30 synthetic siblings. Figure 3 characterizes UXBench fixtures along three structural axes. The clearest separation appears in accessibility exposure: synthetic siblings have a higher interquartile range and mean than real anchors, suggesting that they introduce more localized implementation-level risks such as missing labels, weak accessibility cues, or placeholder interactions. In contrast, real anchors show higher informationarchitecture breadth, with both the mean and upper range shifted slightly upward, reflecting broader navigation structures and more system-like page organization. Cross-page coupling does not clearly separate real and synthetic fixtures: both groups span almost the full score range and have highly overlapping distributions. This overlap suggests that coupling is not primarily determined by whether a fixture is real or synthetic, but instead reflects within-benchmark heterogeneity. Overall, the figure shows that UXBench is not organized along a single difficulty axis; it combines fixtures that stress global reasoning over navigation and state consistency with fixtures that stress local inspection of feedback, accessibility, and recovery behavior.
5 Results Judge models differ in their ability to produce repair-actionable UX reports. Table 1 shows that UXBench exposes measurable differences among judge models under a paired evaluation protocol. Since every model is evaluated on the same fixture set, repaired by the same fixed agent, and scored by the same evaluator, the remaining variation reflects differences in the reports produced by the judge models. Although all models improve over the unrepaired baseline, the size of the improvement differs across judges: GPT-5.4 obtains the largest repair lift (+0.22), while Gemini-3.1-Pro obtains the smallest (+0.14), yielding an 0.08-point spread on the 1–5 rubric scale. This gap indicates that UX report generation is not saturated among current frontier models: 1
8 / 30
UXBench
Fixed Score Robustness
GPT 5.4 Kimi K2.5 Sonnet 4.6 GPT Mini GLM 5.1 GPT Nano Qwen 3.6 Gemini 3.1 3.0
Delta vs Site Robustness
3.49
+0.22
3.48
+0.21
3.45
+0.19
3.45
+0.19
3.45
+0.17
3.44
+0.17
3.42
+0.15
3.41
+0.14
3.5
4.0
4.5
Fixed score across sites (1-5)
-1.0
-0.5
0.0
0.5
1.0
Delta vs site baseline (1-5)
Figure 4. Robustness across judge models. Per-fixture repaired scores and uplift ∆ relative to the site baseline. Models with higher mean lift exhibit wider site-level spreads, separating the sweep along an axis the aggregate column cannot expose.
models are not interchangeable as UX judges, even when downstream repair and scoring are held fixed. At the same time, the narrow absolute spread suggests that adjacent ranks should be interpreted cautiously and analyzed together with fixture-level variation rather than treated as a definitive global ordering. Appendix G reports site-paired significance tests, bootstrap confidence intervals, and effect sizes for repair-lift estimates. 2 Judge models exhibit distinct rubric-level repair signatures. The per-dimension deltas GPT-5.4 Kimi Sonnet GPT-Mini GLM GPT-Nano Qwen Gemini in Table 1 show that models differ not only in 0.650 0.550 0.500 0.500 0.500 0.650 0.650 GPT-5.4 the amount of repair lift they produce, but also in the UX dimensions where that lift appears. 0.600 0.600 0.600 0.500 0.700 0.650 Kimi 0.350 GPT-5.4 obtains the strongest gain on error re0.400 0.400 0.450 0.500 0.400 0.600 Sonnet 0.450 covery, Kimi-K2.5 leads on feedback and trust transparency, Claude-Sonnet-4.6 is strongest on GPT-Mini 0.500 0.400 0.600 - 0.500 0.400 0.650 0.800 goal-state clarity, and Qwen-3.6-Plus reaches the 0.400 0.550 0.500 0.550 0.700 0.800 GLM 0.500 largest gains on flow and scanability/accessibility. These model-specific signatures indicate that UX GPT-Nano 0.500 0.500 0.500 0.600 0.450 - 0.550 0.650 report quality is not a one-dimensional ability: 0.300 0.600 0.350 0.300 0.450 0.550 Qwen 0.350 two models with similar aggregate scores may still support different kinds of downstream re- Gemini 0.350 0.350 0.400 0.200 0.200 0.350 0.450 pair. 3 Judge models vary in fixture-level reliability. 0.00 0.25 0.50 0.75 1.00 Figure 4 shows that models also differ in how Figure 6. Pairwise model wins. Overall pairwise consistently their reports transfer across individ- win-rate matrix across evaluated models. ual fixtures. GPT-5.4 achieves the highest mean repaired score and mean lift, but its site-level interval is wide, indicating uneven performance across the benchmark. Some lower-mean models show tighter site-level distributions, suggesting more stable but less peak-performing behavior. This variation matters in practice: a judge can produce highly actionable reports on some interfaces while yielding weak repair outcomes on others. 9 / 30
UXBench
Landing Page
Pricing Page
Onboarding
Booking
Dashboard
avg 3.38 | max 3.54
avg 3.57 | max 3.82
avg 3.60 | max 3.66
avg 3.41 | max 3.54
avg 3.21 | max 3.38
Docs
avg 3.66 | max 3.90 GPT-5.4
Privacy
avg 3.54 | max 3.61 Claude-Sonnet-4.6
Kimi-K2.5
Visualization
Chatbot
avg 3.36 | max 3.52
avg 3.33 | max 3.61
GLM-5.1
Gemini-3.1-Pro
GPT-5.4-Mini
Mobile UI
avg 3.36 | max 3.50 Qwen-3.6-Plus
GPT-5.4-Nano
Figure 5. Category-level repaired-score profiles across the eight evaluated models. Each panel reports the category mean and the best repaired score. The leading model changes across surface types, and the gap between category averages and best-model scores varies noticeably across surfaces.
UX repair decomposes into surface-conditioned subcompetences. Figure 5 shows that surface families differ in both the leading model and the headroom they expose. Documentation and pricing pages reach relatively high repaired scores, whereas dashboard and chatbot/agent interfaces remain lower on average, suggesting that some surfaces are easier to improve once actionable issues are identified, while others require harder diagnosis of dense state, interaction history, or conversational feedback. Table 2 provides a complementary pairwise view: the strongest model in one category often fails to dominate in another, and winning margins vary across columns rather than following the aggregate ranking. Thus, category-level variation is not merely a visualization artifact of mean scores; it reflects surface-conditioned differences in what UX evidence a judge can collect, prioritize, and translate into actionable findings. UXBench should therefore be read as a multi-granularity benchmark that separates global report quality from surface-specific diagnostic competence. 5 Pairwise win rates expose the stability of model separation. Mean repaired scores summarize average report actionability, but they do not show whether a model wins broadly across fixtures or is driven by a few large gains. Figure 6 addresses this by converting overall performance into paired fixture-level comparisons. The matrix reveals a top cluster rather than a single dominant judge: GPT-5.4, Kimi-K2.5, and GLM-5.1 each average about 0.57 pairwise win rate against the other models, while several head-to-head cells among nearby models remain at or near 0.5. This means that aggregate leaders are separated, but not by a strict total order. Conversely, Gemini-3.1-Pro shows a broadly weak pairwise profile, losing to most alternatives even though it can still lead a specific surface family such as dashboards. Pairwise win rates therefore refine the aggregate results: UXBench does not merely rank models by mean repair lift, but reveals whether those advantages persist across heterogeneous interfaces or depend on localized strengths. 6 Human evaluation confirms the broad signal but sharpens the top-tier separation. Figure 7 reports blind human ratings of the repaired interfaces produced from different judge-model reports. The human results broadly support the automated protocol at the coarse level: interfaces repaired from stronger judge reports are generally perceived as more usable by expert reviewers. However, the human ratings also make the separation less like a strict leaderboard and more 4
10 / 30
UXBench
Goal State Clarity Sonnet 4.6
Navigation Scent
3.99
GPT 5.4
3.98
GLM 5.1 3.87
3.65
3.85
3.64
GPT Mini
3.84
3.59
GPT Nano
3.83
3.59
3.79
3.92 Mean score GPT-5.4
4.05 3.54
3.71 Mean score
Claude-Sonnet-4.6
3.54
3.45
3.87
3.31
Kimi-K2.5
3.36
3.49 Mean score
3.67
GLM-5.1
3.69
3.30
3.70
3.29
3.69
3.80 Mean score
3.27
GPT-5.4-Mini
3.19
3.36 Mean score Gemini-3.1-Pro
3.82 3.81 3.78
3.68
3.24
3.92
3.85
3.71
3.76
3.72
3.85
3.71
3.75 3.73
3.36
3.90
3.78
3.37
3.80
3.51
Scanability and Accessibility
3.80
3.42
3.80
3.45
Trust Transparency
3.47
3.83
3.57
3.67
3.86
Kimi K2.5
Error Recovery
3.86
3.59
3.70
Qwen 3.6
Flow Efficiency
3.60
3.76
3.91
Gemini 3.1
Action Feedback
3.81
3.78
3.67
3.53
3.63
3.75 Mean score
Qwen-3.6-Plus
3.71
3.86
3.67
3.81 Mean score
3.96
GPT-5.4-Nano
Figure 7. Human-rated UX profiles of repaired interfaces produced from each judge model’s report. Each panel shows the mean human score for one rubric dimension under blind model identity.
like a top-tier cluster. GPT-5.4 and Claude-Sonnet-4.6 form the clearest upper group, with small dimension-level differences rather than a uniformly dominant model across all criteria. Meanwhile, some models that are competitive under automated repair lift do not translate that advantage into equally strong perceived interface quality. This distinction is important because automated repair lift measures whether a report helps a fixed repair agent improve rubric scores, whereas human evaluation captures whether the resulting webpage feels clearer, easier to navigate, more responsive, and more trustworthy to reviewers. Thus, the human study calibrates UXBench by supporting its actionability signal while showing that close model comparisons should be interpreted through human-perceived interface quality rather than automated scores alone. More detailed human–LLM comparison results are provided in Appendix E.
6 Conclusion We proposed UXBench, a benchmark for evaluating LLMs as interaction-grounded UX judges. UXBench combines runnable fixtures, coverage-gated exploration, evidence-grounded reports, and report-conditioned repair to test whether model critiques can drive real interface improvements. Our results show that UX judging remains unsaturated and multi-dimensional, with models differing across actionability, reliability, rubric dimensions, and product surfaces.
11 / 30
UXBench
Limitations This work has several limitations. First, UXBench uses local-first static fixtures to ensure control and reproducibility, but this design cannot fully capture the dynamics of live production systems, such as personalization, backend failures, or long-term user behavior. Second, report actionability is measured through a fixed repair agent and fixed scorer, so the results reflect how well reports support this controlled repair pipeline rather than all possible developer workflows. Finally, our human validation uses expert reviewers but remains limited in scale; broader studies with more diverse users and deployment contexts would further clarify how automated UX-judge signals translate to real-world UX improvement.
Ethical Considerations This work evaluates LLMs as UX judges in controlled, local-first web fixtures rather than on live user-facing services. The benchmark does not require user accounts, collect personal user data, or interact with production systems. Human validation is conducted through blind review of anonymized model reports, with model identities and automated scores hidden from reviewers. A potential risk is that automated UX judges may be over-trusted or used as substitutes for humancentered design practice. We emphasize that UXBench measures report actionability under a controlled repair pipeline and should complement, rather than replace, human usability studies, accessibility audits, and domain-specific expert review.
12 / 30
UXBench
References Anthropic. Introducing claude sonnet 4.6. https://www.anthropic.com/news/ claude-sonnet-4-6, February 2026. Accessed: 2026-05-26. Tony Beltramelli. pix2code: Generating code from a graphical user interface screenshot, 2017. URL https://arxiv.org/abs/1705.07962. Marius Breitmayer, Lisa Arnold, Michael Winter, and Manfred Reichert. Exploring user editing preferences in electronic forms: an empirical study. Inf. Syst. E-Bus. Manag., 22(1):171–207, January 2024. ISSN 1617-9846. doi: 10.1007/s10257-023-00662-y. URL https://doi.org/10. 1007/s10257-023-00662-y. John Brooke. Sus: A quick and dirty usability scale. Usability Eval. Ind., 189, 11 1995. Sara Bunian, Kai Li, Chaima Jemmali, Casper Harteveld, Yun Fu, and Magy Seif El-Nasr. Vins: Visual search for mobile user interface design, 2021. URL https://arxiv.org/abs/2102.05216. Miriana Calvano, Antonio Curci, Rosa Lanzilotti, Antonio Piccinno, and Azzurra Ragone. Leveraging large language models for usability testing: a preliminary study. In Companion Proceedings of the 30th International Conference on Intelligent User Interfaces, pages 78–81. ACM, 03 2025. doi: 10.1145/3708557.3716341. Chunyang Chen, Sidong Feng, Zhenchang Xing, Linda Liu, Shengdong Zhao, and Jinshui Wang. Gallery d.c.: Design search and knowledge discovery through auto-created gui component gallery. Proc. ACM Hum.-Comput. Interact., 3(CSCW), November 2019. doi: 10.1145/3359282. URL https://doi.org/10.1145/3359282. Kanzhi Cheng, Qiushi Sun, Yougang Chu, Fangzhi Xu, Yantao Li, Jianbing Zhang, and Zhiyong Wu. Seeclick: Harnessing gui grounding for advanced visual gui agents, 2024. URL https: //arxiv.org/abs/2401.10935. Thibault Le Sellier De Chezelles, Maxime Gasse, Alexandre Drouin, Massimo Caccia, Léo Boisvert, Megh Thakkar, Tom Marty, Rim Assouel, Sahar Omidi Shayegan, Lawrence Keunho Jang, Xing Han Lù, Ori Yoran, Dehan Kong, Frank F. Xu, Siva Reddy, Quentin Cappart, Graham Neubig, Ruslan Salakhutdinov, Nicolas Chapados, and Alexandre Lacoste. The browsergym ecosystem for web agent research, 2025. URL https://arxiv.org/abs/2412.05467. Biplab Deka, Zifeng Huang, Chad Franzen, Joshua Hibschman, Daniel Afergan, Yang Li, Jeffrey Nichols, and Ranjitha Kumar. Rico: A mobile app dataset for building data-driven design applications. In Proceedings of the 30th annual ACM symposium on user interface software and technology, pages 845–854, 2017. Xiang Deng, Yu Gu, Boyuan Zheng, Shijie Chen, Samuel Stevens, Boshi Wang, Huan Sun, and Yu Su. Mind2web: Towards a generalist agent for the web, 2023. URL https://arxiv.org/abs/ 2306.06070. Matthew Dixon, Karen Freeman, Nicholas Toman, et al. Stop trying to delight your customers. Harvard business review, 88(7/8):116–122, 2010.
13 / 30
UXBench
Alexandre Drouin, Maxime Gasse, Massimo Caccia, Issam H. Laradji, Manuel Del Verme, Tom Marty, Léo Boisvert, Megh Thakkar, Quentin Cappart, David Vazquez, Nicolas Chapados, and Alexandre Lacoste. Workarena: How capable are web agents at solving common knowledge work tasks?, 2024. URL https://arxiv.org/abs/2403.07718. Peitong Duan, Chin yi Chen, Gang Li, Bjoern Hartmann, and Yang Li. Uicrit: Enhancing automated design evaluation with a uicritique dataset, 2024. URL https://arxiv.org/abs/2407.08850. Alice Gao, Weixi Tong, Rishab Vempati, Katharina Reinecke, R. Benjamin Shapiro, Tianyi Zhang, and Jason Wu. Training computer use agents to assess the usability of graphical user interfaces, 2026. URL https://arxiv.org/abs/2604.26020. Google. Gemini 3.1 pro: A smarter model for your most complex tasks. https://blog.google/ innovation-and-ai/models-and-research/gemini-models/gemini-3-1-pro/, February 2026. Accessed: 2026-05-26. Guilherme Guerino, Luiz Rodrigues, Bruna Capeleti, Rafael Ferreira Mello, André Freire, and Luciana Zaina. Can GPT-4o Evaluate Usability Like Human Experts? A Comparative Study on Issue Identification in Heuristic Evaluation, page 381–402. Springer Nature Switzerland, September 2025. ISBN 9783032050052. doi: 10.1007/978-3-032-05005-2_20. URL http://dx.doi.org/10. 1007/978-3-032-05005-2_20. S. G. Hart and Lowell E. Staveland. Development of nasa-tlx (task load index): Results of empirical and theoretical research. Advances in psychology, 52:139–183, 1988. URL https: //api.semanticscholar.org/CorpusID:15252590. Hongliang He, Wenlin Yao, Kaixin Ma, Wenhao Yu, Yong Dai, Hongming Zhang, Zhenzhong Lan, and Dong Yu. WebVoyager: Building an end-to-end web agent with large multimodal models. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors, Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 6864–6890, Bangkok, Thailand, August 2024. Association for Computational Linguistics. doi: 10.18653/v1/2024.acl-long.371. URL https://aclanthology.org/2024.acl-long.371/. Nien-Lin Hsueh, Hsuen-Jen Lin, and Lien-Chi Lai. Applying large language model to user experience testing. Electronics, 13:4633, 11 2024. doi: 10.3390/electronics13234633. Aleeha Iftikhar, Raymond R. Bond, Victoria Mcgilligan, Stephen J. Leslie, Khaled Waleed Younis Rjoob, Charles Knoery, Ciara Quigg, Ryan Campbell, Kyle Boyd, Anne McShane, and Aaron J. Peace. Comparing single-page, multipage, and conversational digital forms in health care: Usability study. JMIR Human Factors, 8, 2020. URL https://api.semanticscholar.org/CorpusID: 234503933. Melody Y. Ivory and Marti A Hearst. The state of the art in automating usability evaluation of user interfaces. ACM Comput. Surv., 33(4):470–516, December 2001. ISSN 0360-0300. doi: 10.1145/503112.503114. URL https://doi.org/10.1145/503112.503114. Kimi Team. Kimi k2.5: Visual agentic intelligence, 2026. URL https://arxiv.org/abs/2602.02276. Jing Yu Koh, Robert Lo, Lawrence Jang, Vikram Duvvur, Ming Lim, Po-Yu Huang, Graham Neubig, Shuyan Zhou, Russ Salakhutdinov, and Daniel Fried. VisualWebArena: Evaluating 14 / 30
UXBench
multimodal agents on realistic visual web tasks. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors, Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 881–905, Bangkok, Thailand, August 2024. Association for Computational Linguistics. doi: 10.18653/v1/2024.acl-long.50. URL https://aclanthology. org/2024.acl-long.50/. Bettina Laugwitz, Theo Held, and Martin Schrepp. Construction and evaluation of a user experience questionnaire. In Proceedings of the 4th Symposium of the Workgroup Human-Computer Interaction and Usability Engineering of the Austrian Computer Society on HCI and Usability for Education and Work, USAB ’08, page 63–76, Berlin, Heidelberg, 2008. Springer-Verlag. ISBN 9783540893493. doi: 10.1007/978-3-540-89350-9_6. URL https://doi.org/10.1007/978-3-540-89350-9_6. James R. Lewis. Psychometric evaluation of an after-scenario questionnaire for computer usability studies: the asq. ACM SIGCHI Bull., 23:78–81, 1991. URL https://api.semanticscholar.org/ CorpusID:24344824. James R. Lewis. Ibm computer usability satisfaction questionnaires: psychometric evaluation and instructions for use. Int. J. Hum.-Comput. Interact., 7(1):57–78, January 1995. ISSN 1044-7318. doi: 10.1080/10447319509526110. URL https://doi.org/10.1080/10447319509526110. James R. Lewis. Psychometric evaluation of the pssuq using data from five years of usability studies. International Journal of Human–Computer Interaction, 14:463 – 488, 2002. URL https: //api.semanticscholar.org/CorpusID:263890198. James R. Lewis, Brian S. Utesch, and Deborah E. Maher. Umux-lite: when there’s no time for the sus. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’13, page 2099–2102, New York, NY, USA, 2013. Association for Computing Machinery. ISBN 9781450318990. doi: 10.1145/2470654.2481287. URL https://doi.org/10.1145/2470654. 2481287. Evan Zheran Liu, Kelvin Guu, Panupong Pasupat, Tianlin Shi, and Percy Liang. Reinforcement learning on web interfaces using workflow-guided exploration, 2018a. URL https://arxiv. org/abs/1802.08802. Thomas F. Liu, Mark Craft, Jason Situ, Ersin Yumer, Radomír Měch, and Ranjitha Kumar. Learning design semantics for mobile apps. Proceedings of the 31st Annual ACM Symposium on User Interface Software and Technology, 2018b. URL https://api.semanticscholar.org/CorpusID:52980013. Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, and Chenguang Zhu. G-eval: NLG evaluation using gpt-4 with better human alignment. In Houda Bouamor, Juan Pino, and Kalika Bali, editors, Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 2511–2522, Singapore, December 2023. Association for Computational Linguistics. doi: 10.18653/v1/2023.emnlp-main.153. URL https://aclanthology.org/2023.emnlp-main.153/. Quanfeng Lu, Wenqi Shao, Zitao Liu, Lingxiao Du, Fanqing Meng, Boxuan Li, Botong Chen, Siyuan Huang, Kaipeng Zhang, and Ping Luo. Guiodyssey: A comprehensive dataset for cross-app gui navigation on mobile devices, 2025a. URL https://arxiv.org/abs/2406.08451.
15 / 30
UXBench
Yuxuan Lu, Bingsheng Yao, Hansu Gu, Jing Huang, Jessie Wang, Yang Li, Jiri Gesi, Qi He, Toby Jia-Jun Li, and Dakuo Wang. Uxagent: A system for simulating usability testing of web design with llm agents, 2025b. URL https://arxiv.org/abs/2504.09407. Xing Han Lù, Zdeněk Kasner, and Siva Reddy. Weblinx: Real-world website navigation with multi-turn dialogue, 2024. URL https://arxiv.org/abs/2402.05930. Forough Mehralian, Ziyao He, and Sam Malek. Automated accessibility analysis of dynamic content changes on mobile apps. In Proceedings of the IEEE/ACM 47th International Conference on Software Engineering, ICSE ’25, page 2689–2701. IEEE Press, 2025. ISBN 9798331505691. doi: 10.1109/ICSE55347.2025.00039. URL https://doi.org/10.1109/ICSE55347.2025.00039. OpenAI. Introducing gpt-5.4. https://openai.com/index/introducing-gpt-5-4/, March 2026a. Accessed: 2026-05-26. OpenAI. Introducing gpt-5.4 mini and nano. https://openai.com/index/ introducing-gpt-5-4-mini-and-nano/, March 2026b. Accessed: 2026-05-26. Alaattin Parlakkılıç. Evaluating the effects of responsive design on the usability of academic websites in the pandemic. Education and Information Technologies, 27, 07 2021. doi: 10.1007/ s10639-021-10650-9. Qwen Team. Qwen3.6-plus: Towards real world agents. https://qwen.ai/blog?id=qwen3.6, April 2026. Accessed: 2026-05-26. Christopher Rawles, Alice Li, Daniel Rodriguez, Oriana Riva, and Timothy Lillicrap. Android in the wild: A large-scale dataset for android device control, 2023. URL https://arxiv.org/abs/ 2307.10088. Christopher Rawles, Sarah Clinckemaillie, Yifan Chang, Jonathan Waltz, Gabrielle Lau, Marybeth Fair, Alice Li, William Bishop, Wei Li, Folawiyo Campbell-Ajala, Daniel Toyama, Robert Berry, Divya Tyamagundlu, Timothy Lillicrap, and Oriana Riva. Androidworld: A dynamic benchmarking environment for autonomous agents, 2025. URL https://arxiv.org/abs/2405.14573. Jeff Sauro. Supr-q: a comprehensive measure of the quality of the website user experience. Journal of Usability Studies archive, 10:68–86, 2015. URL https://api.semanticscholar.org/CorpusID: 16552794. Jeff Sauro and Joseph S. Dumas. Comparison of three one-question, post-task usability questionnaires. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’09, page 1599–1608, New York, NY, USA, 2009. Association for Computing Machinery. ISBN 9781605582467. doi: 10.1145/1518701.1518946. URL https://doi.org/10.1145/1518701. 1518946. Tianlin Tim Shi, Andrej Karpathy, Linxi Jim Fan, Jonathan Hernandez, and Percy Liang. World of bits: an open-domain platform for web-based agents. In Proceedings of the 34th International Conference on Machine Learning - Volume 70, ICML’17, page 3135–3144. JMLR.org, 2017. B. Shneiderman. Direct manipulation: A step beyond programming languages. Computer, 16(8): 57–69, August 1983. ISSN 0018-9162. doi: 10.1109/MC.1983.1654471. URL https://doi.org/10. 1109/MC.1983.1654471. 16 / 30
UXBench
Chenglei Si, Yanzhe Zhang, Ryan Li, Zhengyuan Yang, Ruibo Liu, and Diyi Yang. Design2Code: Benchmarking multimodal code generation for automated front-end engineering. In Luis Chiruzzo, Alan Ritter, and Lu Wang, editors, Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 3956–3974, Albuquerque, New Mexico, April 2025. Association for Computational Linguistics. ISBN 979-8-89176-189-6. doi: 10.18653/v1/2025.naacl-long.199. URL https://aclanthology.org/2025.naacl-long.199/. Maryam Taeb, Amanda Swearngin, Eldon Schoop, Ruijia Cheng, Yue Jiang, and Jeffrey Nichols. Axnav: Replaying accessibility tests from natural language. In Proceedings of the CHI Conference on Human Factors in Computing Systems, CHI ’24, page 1–16. ACM, May 2024. doi: 10.1145/ 3613904.3642777. URL http://dx.doi.org/10.1145/3613904.3642777. Yuxuan Wan, Chaozheng Wang, Yi Dong, Wenxuan Wang, Shuqing Li, Yintong Huo, and Michael Lyu. Divide-and-conquer: Generating ui code from screenshots. Proceedings of the ACM on Software Engineering, 2(FSE):2099–2122, June 2025. ISSN 2994-970X. doi: 10.1145/3729364. URL http://dx.doi.org/10.1145/3729364. World Wide Web Consortium (W3C). Web content accessibility guidelines (wcag) 2.2. https:// www.w3.org/TR/WCAG22/, December 2024. W3C Recommendation, World Wide Web Consortium (W3C), 12 December 2024. Jason Wu, Siyan Wang, Siman Shen, Yi-Hao Peng, Jeffrey Nichols, and Jeffrey P. Bigham. Webui: A dataset for enhancing visual ui understanding with web semantics, 2023. URL https://arxiv. org/abs/2301.13280. Jason Wu, Yi-Hao Peng, Amanda Li, Amanda Swearngin, Jeffrey P. Bigham, and Jeffrey Nichols. Uiclip: A data-driven model for assessing user interface design, 2024a. URL https://arxiv. org/abs/2404.12500. Jason Wu, Eldon Schoop, Alan Leung, Titus Barik, Jeffrey Bigham, and Jeffrey Nichols. UICoder: Finetuning large language models to generate user interface code through automated feedback. In Kevin Duh, Helena Gomez, and Steven Bethard, editors, Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 7511–7525, Mexico City, Mexico, June 2024b. Association for Computational Linguistics. doi: 10.18653/v1/2024.naacl-long.417. URL https://aclanthology.org/2024.naacl-long.417/. Tianbao Xie, Danyang Zhang, Jixuan Chen, Xiaochuan Li, Siheng Zhao, Ruisheng Cao, Toh Jing Hua, Zhoujun Cheng, Dongchan Shin, Fangyu Lei, Yitao Liu, Yiheng Xu, Shuyan Zhou, Silvio Savarese, Caiming Xiong, Victor Zhong, and Tao Yu. Osworld: Benchmarking multimodal agents for open-ended tasks in real computer environments, 2024. URL https://arxiv.org/ abs/2404.07972. Sukmin Yun, Haokun Lin, Rusiru Thushara, Mohammad Qazim Bhat, Yongxin Wang, Zutao Jiang, Mingkai Deng, Jinhong Wang, Tianhua Tao, Junbo Li, Haonan Li, Preslav Nakov, Timothy Baldwin, Zhengzhong Liu, Eric P. Xing, Xiaodan Liang, and Zhiqiang Shen. Web2code: A large-scale webpage-to-code dataset and evaluation framework for multimodal llms, 2024. URL https://arxiv.org/abs/2406.20098. 17 / 30
UXBench
Z.AI. Glm-5.1: Towards long-horizon tasks. https://z.ai/blog/glm-5.1, April 2026. Accessed: 2026-05-26. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. Judging llm-as-a-judge with mt-bench and chatbot arena, 2023. URL https://arxiv.org/abs/ 2306.05685. Shuyan Zhou, Frank F Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Sridhar, Xianyi Cheng, Tianyue Ou, Yonatan Bisk, Daniel Fried, et al. Webarena: A realistic web environment for building autonomous agents. In International Conference on Learning Representations, volume 2024, pages 15585–15606, 2024.
18 / 30
UXBench
A Details of Coverage Gate Algorithm Algorithm 2 Coverage-Gated Exploration Require: fixture profile P with target pages U and prescanned controls K Require: exploration plan Π, step budget B Require: depth d ∈ {quick, std, deep, exh} Require: viewport mode v ∈ {desk, mob, both} Ensure: exploration trace τ and evidence confidence q ∈ {low, med, high} 1: V ← Viewports(v) 2: G ← CoverageGoals( P, Π, d, V ) 3: open the first target page in Π under the first viewport in V 4: o0 ← Observe(); τ ← [o0 ] 5: for t = 0, . . . , B − 1 do 6: compute Pseen , Vseen , Kdone , and Sseen from τ 7: C.unmet ← (Gpage \ Pseen ) ∪ (Gviewport \ Vseen ) ∪ (Gcontrol \ Kdone ) ∪ (Gstate \ Sseen ) 8: C.stop_allowed ← (C.unmet = ∅ ∧ NoBlockingFailure(τ )) 9: at ← Brain(ot , τ, Π, C ) 10: if at = finish and C.stop_allowed = ⊥ then 11: at ← Brain(ot , τ, Π, C, must_continue = C.unmet) 12: end if 13: if at = finish and C.stop_allowed = ⊥ then 14: at ← Fallback(C ) 15: end if 16: if at = finish then 17: break 18: end if 19: if Unsafe( at , P) then 20: at ← Fallback(C ) 21: end if 22: execute at 23: ot+1 ← Observe() 24: append ( at , ot+1 ) to τ 25: end for 26: recompute all coverage sets and C from τ 27: if C.stop_allowed = ⊤ then 28: q ← high 29: else if Gpage ⊆ Pseen and Gviewport ⊆ Vseen then 30: q ← med 31: else 32: q ← low 33: end if 34: return τ, q
B Surface Categories Fixture Design and Catalog. UXBench is built from local-first web fixtures implemented as static HTML/CSS/JavaScript bundles. Each fixture can be served from a local file server and evaluated without accounts, backend services, third-party APIs, or live-site dependencies. This design makes the benchmark reproducible: model comparisons are not affected by A/B tests, personalization, network failures, or changes in production websites. At the same time, the fixtures 19 / 30
UXBench
#
Category
Real anchor
Synthetic siblings
1 2 3 4 5 6 7 8 9 10
Landing Page Pricing Page Onboarding Booking Dashboard Docs Privacy Visualization ChatBot Mobile UI
notion slack shopify, govuk-passport booking cloudflare-radar stripe-docs microsoft-privacy owid-population chatgpt ridenow
pelagic, meadowos, stratabox codekite, lattice, vaultkey solstice-bank, greengrove, civicport orbitride, moonlight-tickets, tablerose fleetatlas, aeroiq, pulsegrid tessera, weaveapi, runeforge-docs privacy-dashboard, aurora-network, meadowid fred-unrate, climate-almanac, migration-atlas lumen-research, forge-coder, atlas-tutor brewlog, harborwallet, larkfit
Table 3. UXBench site catalog. Real anchors ground each category in recognizable public-product interaction patterns, while synthetic siblings preserve the core interaction model with independently authored branding, content, layout, and visual identity.
remain interactive rather than screenshot-only, allowing judge models to click controls, enter text, switch viewports, observe feedback, and ground their reports in actual interface behavior. As shown in Table 3, UXBench spans ten product-surface families, covering common web experiences such as landing pages, pricing pages, onboarding flows, booking interfaces, dashboards, documentation, privacy/settings pages, data visualizations, chatbot interfaces, and mobile micro-UIs. Within each family, we pair real-product anchors with independently authored synthetic siblings. The real anchors ground each category in recognizable public-product interaction patterns, while the synthetic siblings preserve the same broad interaction model but vary branding, copy, layout, and visual identity. This anchor–sibling structure balances external validity with control: anchors expose models to realistic UX conventions, while siblings test whether a judge evaluates the current interface rather than relying on memorized impressions of familiar products.
C Rubric Construction UXBench derives its rubric from established usability and user-experience instruments, but does not directly administer these instruments as questionnaires. Instead, as summarized in Table 5, we use them as construct references and translate recurring notions such as clarity, navigation support, feedback, effort, recovery, trust, and scanability into browser-grounded evaluation questions. This lets the rubric preserve links to prior UX measurement practice while remaining answerable from interaction evidence collected in a live web interface. System Usability Scale (SUS). SUS provides a compact reference for perceived usability, learnability, consistency, and user confidence (Brooke, 1995). In UXBench, these constructs mainly support goal-state clarity and trust/consequence transparency. User Experience Questionnaire (UEQ). UEQ covers both pragmatic and hedonic aspects of user experience, including perspicuity, efficiency, dependability, stimulation, and attractiveness (Laugwitz et al., 2008). We primarily use its pragmatic dimensions to motivate clarity, flow efficiency, and scanability.
20 / 30
UXBench
Model
Provider
GPT-5.4 Kimi-K2.5 Claude-Sonnet-4.6 GPT-5.4-Mini GLM-5.1 GPT-5.4-Nano Qwen-3.6-Plus Gemini-3.1-Pro
OpenAI Moonshot AI Anthropic OpenAI Zhipu AI OpenAI Alibaba Google
Open ✗ ✓ ✗ ✗ ✓ ✗ ✗ ✗
Table 4. Judge models used in the UXBench benchmark sweep. ✓ indicates open-weight availability; ✗ indicates API-only access.
Post-Study System Usability Questionnaire (PSSUQ). PSSUQ emphasizes system usefulness, information quality, and interface quality (Lewis, 1995, 2002). These constructs inform our treatment of navigation cues, action feedback, recovery guidance, and task-supporting information. Standardized User Experience Percentile Rank Questionnaire (SUPR-Q). SUPR-Q is especially relevant to website-level evaluation because it covers usability, trust, appearance, and loyalty (Sauro, 2015). We use it to ground navigation scent, trust transparency, and scanability for web interfaces. UMUX-Lite. UMUX-Lite reduces perceived usability to whether the system meets user requirements and is easy to use (Lewis et al., 2013). These two judgments align with goal fit, discoverability, and confidence in continuing. Single Ease Question (SEQ). SEQ captures the perceived difficulty of a completed task (Sauro and Dumas, 2009). We use it as a task-local reference for flow efficiency, especially when a path requires unnecessary steps, repeated input, or backtracking. After-Scenario Questionnaire (ASQ). ASQ focuses on satisfaction with task completion, time required, and supporting information (Lewis, 1991). It motivates our use of action feedback, flow efficiency, and error recovery as interaction-path properties. Customer Effort Score (CES). CES highlights the effort users must expend to accomplish a goal or resolve a problem (Dixon et al., 2010). In UXBench, it supports our attention to friction, repeated work, hidden requirements, and avoidable detours. NASA Task Load Index (NASA-TLX). NASA-TLX provides vocabulary for workload, effort, time pressure, and frustration (Hart and Staveland, 1988). We use it to capture cognitive burden in dense flows, unclear states, and difficult recovery paths.
21 / 30
UXBench
#
UXBench metric
In-browser evaluation question
Reference Operationalized scales constructs
1
Goal-state clarity
Can users quickly understand the page purpose, current state, available options, and most sensible next action?
UEQ, PSSUQ, SUS, UMUXLite
Perspicuity; information quality; perceived ease of use; confidence in using the system
2
Navigation scent
Do labels, menus, tabs, search, and filters provide reliable cues toward the right content or next step?
SUPR-Q, PSSUQ, UEQ
Ease of navigation; ease of finding information; clarity of organization; learnability cues
3
Action feedback
Are user actions followed by clear ASQ, feedback for selection, input, PSSUQ, loading, validation, success, and NASAfailure states? TLX
Ease of task completion; support information; system-status visibility; error feedback; frustration
4
Flow efficiency
Can users complete multi-step or SEQ, cross-page tasks without ASQ, unnecessary detours, repetition, CES, waiting, or backtracking? UEQ
Task ease; time satisfaction; perceived effort; efficiency; cognitive burden
5
Error recovery
Does the interface prevent likely mistakes and provide clear ways to correct, undo, retry, or return when problems occur?
PSSUQ, ASQ, NASATLX
Error prevention; recovery from mistakes; helpful guidance; support information; effort and frustration
6
Trust transparency
Before committing, can users understand costs, permissions, privacy-relevant choices, and consequences of sensitive actions?
SUPR-Q, SUS, PSSUQ, UMUXLite
Trust; confidence transacting; confidence using the system; clarity of consequences; information quality
7
Scanability and accessibility
Is the page easy to scan, visually prioritized, readable across screen sizes, and operable with basic accessibility cues?
SUPR-Q, UEQ, PSSUQ
Appearance; attractiveness; readability; organized information; responsive usability; basic accessibility support
Table 5. UXBench default scoring rubric and construct alignment. The seven metrics operationalize observable interface qualities that recur across established UX instruments, including SUS, UEQ, PSSUQ, SUPR-Q, UMUX-Lite, SEQ, ASQ, CES, and NASA-TLX, but restate them as browser-grounded questions answerable from interaction evidence rather than post-task self reports.
22 / 30
UXBench
Model
Land.
Price
Onbd.
Book.
Dash
Docs
Privacy Visual. Chatbot Mobile
GPT 5.4
0.696
0.804
0.886
0.732
0.696
0.696
0.482
0.786
0.554
0.821
Kimi k2.5
0.393
0.304
0.614
0.339
0.429
0.196
0.464
0.452
0.321
0.321
Sonnet 4.6
0.607
0.607
0.443
0.732
0.857
0.643
0.768
0.905
0.571
0.714
GPT Mini
0.286
0.429
0.371
0.446
0.429
0.732
0.339
0.429
0.714
0.589
GLM 5.1
0.446
0.268
0.400
0.339
0.446
0.482
0.554
0.524
0.857
0.554
GPT Nano
0.500
0.643
0.543
0.304
0.232
0.286
0.375
0.333
0.375
0.339
Qwen 3.6
0.518
0.464
0.257
0.357
0.321
0.393
0.375
0.262
0.196
0.286
Gemini 3.1
0.554
0.482
0.486
0.750
0.589
0.571
0.643
0.310
0.411
0.375
Table 6. Category-level human pairwise win rates across the ten UXBench surface families. Land., Price, Onbd., Book., and Dash denote Landing, Pricing, Onboarding, Booking, and Dashboard, respectively; Visual. denotes visual design. Each cell reports a model’s mean pairwise win rate against the other evaluated models within the same surface family. Bold indicates the highest human-preferred model in each category.
D Evaluated Judge Models UXBench evaluates eight frontier LLMs as UX judge models: GPT-5.4, GPT-5.4-Mini, GPT-5.4Nano (OpenAI, 2026a,b), Claude-Sonnet-4.6 (Anthropic, 2026), Gemini-3.1-Pro (Google, 2026), Kimi-K2.5 (Kimi Team, 2026), GLM-5.1 (Z.AI, 2026), and Qwen-3.6-Plus (Qwen Team, 2026). These models span major closed API systems and competitive open or open-weight-oriented model families, covering recent progress in agentic reasoning, coding, multimodal understanding, and computer-use capabilities. In our benchmark, each model is used only as a judge/report generator: it inspects the same web fixtures, produces UX reports under the same rubric, and is compared through the same downstream repair-and-scoring protocol. Table 4 summarizes the evaluated models and their providers.
E LLM-Human Judge Comparison F Human Evaluation Details We provide additional details on the human evaluation interface and procedure. The evaluation instructions is shown in Figure 10, and interface is shown in Figure 12, Figure 11, Figure 13. We recruited six participants with UX or front-end design experience. The evaluation was conducted through a custom web interface that presented each webpage together with its README, which described the project context and intended interaction scenario. Participants were asked to read the README before inspecting the webpage, so that their judgments were grounded in the intended use case rather than in visual appearance alone. Before submitting ratings, participants explored each webpage directly. They were instructed to interact with salient reachable controls, inspect navigation behavior, check visible state changes and feedback messages, and test failure or recovery paths when applicable. After this exploration step, participants opened the scoring panel and rated the candidate webpage on the same seven UX dimensions used in the main evaluation. For dimensions involving mobile experience, participants switched the rendering device in the evaluation interface and evaluated the corresponding mobile layout and interaction behavior. Model identities were hidden throughout the study. Candidate webpages were shown under
23 / 30
UXBench
n r = 0.59 rho = 0.61 MAE = 0.29 ff = +0.29
3.8
GPT-5.4
+0.30
+0.11
+0.37
+0.36
+0.60
+0.02
+0.14
+0.63
+0.44
+0.65
Kimi
+0.28
-0.01
+0.25
+0.07
+0.56
-0.12
+0.21
+0.34
+0.19
+0.47
Sonnet
+0.42
+0.36
+0.19
+0.39
+0.57
+0.07
+0.42
+0.71
+0.01
+0.55
GPT-Mini
+0.39
+0.00
+0.18
+0.23
+0.59
+0.38
+0.11
+0.46
+0.45
+0.30
GLM
+0.37
+0.15
+0.17
+0.18
+0.47
+0.07
+0.20
+0.38
+0.64
+0.32
GPT-Nano
+0.48
-0.07
+0.15
+0.11
+0.47
+0.26
+0.16
+0.57
+0.11
+0.26
Qwen
+0.36
+0.10
+0.17
+0.18
+0.50
+0.24
+0.04
+0.37
+0.28
+0.29
Gemini
+0.56
+0.40
+0.22
+0.44
+0.35
+0.12
+0.22
+0.41
+0.33
+0.04
g
. rd
g
3.9
in nd
La
-0.75
icin
Pr
a bo On
g
in ok
Bo
sh Da
cs Do
rd
a bo
-0.38
0
cy
iva
Pr
l.
ua
Vis
+0.38
at
Ch
t bo
e
bil
Mo
+0.75
Figure 8. Human–LLM score gaps across model–surface pairs. Each cell shows the difference between blind human ratings and automated LLM-based scores; positive values indicate cases where human reviewers rated the repaired webpage higher than the LLM evaluator.
anonymous labels such as A, B, and C, and participants could not access model names, automated scores, or aggregate rankings. After all ratings were collected, we restored the hidden label–model mapping and aggregated the human scores across participants, webpages, and rubric dimensions. 1 Human pairwise judgments break the GPT-5.4 Kimi Sonnet GPT-Mini GLM GPT-Nano Qwen Gemini automated top-cluster ambiguity. The automated LLM-based pairwise results suggest a GPT-5.4 - 1.000 0.700 0.900 0.900 1.000 1.000 0.900 relatively flat top group, where several strong 0.100 0.350 0.450 0.650 0.700 0.400 Kimi 0.000 models remain close in head-to-head comparisons. In contrast, the human pairwise matrix 0.900 0.700 0.900 0.850 1.000 0.850 Sonnet 0.300 in Figure 6 produces a much sharper hierarchy. GPT-5.4 wins nearly all human head-to-head GPT-Mini 0.100 0.650 0.300 - 0.600 0.650 0.900 0.600 comparisons with large margins, while Claude0.550 0.100 0.400 0.650 0.800 0.300 GLM 0.100 Sonnet-4.6 becomes the only model that consistently challenges it. This pattern is also re0.350 0.150 0.350 0.350 0.750 0.350 GPT-Nano 0.000 flected in the human category-level pairwise results in Table 6: GPT-5.4 leads several task- and 0.300 0.000 0.100 0.200 0.250 0.200 Qwen 0.000 conversion-oriented surfaces, whereas Claude0.600 0.150 0.400 0.700 0.650 0.800 Sonnet-4.6 leads multiple information-dense or Gemini 0.100 trust-sensitive surfaces. Thus, human evaluation does not merely reproduce the automated 0.00 0.25 0.50 0.75 1.00 LLM ranking; it separates models whose re- Figure 9. Overall human pairwise win-rate matrix. ports lead to similarly score-improving repairs into those whose repaired webpages are more clearly preferred by expert reviewers.
24 / 30
UXBench
Model GPT-5.4 Kimi-K2.5 Claude-Sonnet-4.6 GPT-5.4-Mini GLM-5.1 GPT-5.4-Nano Qwen-3.6-Plus Gemini-3.1-Pro Spearman’s ρ = 0.635
Automated Lift
Human Mean
Automated Rank
Human Rank
+0.216 +0.205 +0.174 +0.172 +0.171 +0.162 +0.148 +0.136
3.84 3.70 3.81 3.75 3.75 3.69 3.67 3.71
1 2 3 4 5 6 7 8
1 6 2 3.5 3.5 7 8 5
Kendall’s τb = 0.546
n = 8.
Table 7. Agreement between automated repair lift and blind human ratings. Automated Lift denotes the model-level repair-score improvement measured automatically. Human Mean denotes the average blind human rating. Automated Rank and Human Rank denote the corresponding rankings under the two evaluation protocols. Rank correlations are computed across the eight model-level aggregates.
Human–LLM gaps reveal where automated scoring underestimates perceived UX quality. Figure 8 compares human ratings with automated LLM-based scores at the model–surface level. The differences are not uniform: human ratings are especially higher on visually dense, stateful, or mobile-facing surfaces such as dashboard, visualization, chatbot, and mobile UI, while some simpler surfaces show smaller gaps or even weak reversals for particular models. This indicates that the automated scorer captures a useful repair-actionability signal, but it is less sensitive to some qualities that human reviewers perceive directly in the repaired webpage, such as visual coherence, interaction legibility, and surface-level polish. The comparison therefore calibrates UXBench rather than contradicting it: automated repair lift is effective for broad model screening, but human evaluation remains necessary for interpreting close ranks and for judging whether repaired interfaces actually feel better to users. 3 Human validation supports automated repair lift as a coarse actionability signal. Table 7 compares automated repair lift with blind human ratings at the model-aggregate level. The two signals are positively associated, with Spearman’s ρ = 0.635 and Kendall’s τb = 0.546 across the eight evaluated models, indicating that reports producing larger automated repair gains generally also lead to repaired interfaces that human reviewers perceive as more usable. However, the agreement is not a strict ranking match: GPT-5.4 remains the strongest model under both protocols, but several middle-ranked models shift noticeably between automated and human evaluation. This pattern suggests that automated repair lift captures a useful report-actionability signal, while human ratings capture additional perceptual qualities of the repaired webpage, including clarity, polish, and interaction legibility. Thus, the human study calibrates UXBench rather than replacing the automated protocol: automated scores are effective for broad model screening, but close model comparisons should be interpreted through human-perceived interface quality. 2
G Statistical Validation of Repair Lift and Human Alignment We validate the repair-lift signal with site-paired statistical tests, bootstrap confidence intervals, and model-level human–automatic rank correlations. For both automated and human protocols, lift is computed by pairing each repaired score with the corresponding fixture’s unrepaired
25 / 30
UXBench
Model
n
Mean Lift
95% boot. CI
t-test p
Wilcoxon p
dz
GPT-5.4 Kimi-K2.5 Claude-Sonnet-4.6 GPT-5.4-Mini GLM-5.1 GPT-5.4-Nano Qwen-3.6-Plus Gemini-3.1-Pro
39 41 40 39 41 41 41 38
+0.209 +0.205 +0.188 +0.188 +0.171 +0.162 +0.148 +0.137
[0.104, 0.310] [0.103, 0.301] [0.083, 0.294] [0.083, 0.287] [0.072, 0.267] [0.065, 0.259] [0.051, 0.245] [0.031, 0.244]
< 0.001 < 0.001 0.001 0.001 0.002 0.002 0.005 0.018
< 0.001 < 0.001 0.002 0.002 0.002 0.004 0.007 0.027
0.629 0.628 0.546 0.570 0.531 0.504 0.462 0.400
Pooled model–site rows
320
+0.176
[0.140, 0.212]
< 0.001
–
0.538
Table 8. Automated repair lift under site-paired comparison against each fixture’s unrepaired baseline. Scores are on the 1–5 rubric scale. Model GPT-5.4 Claude-Sonnet-4.6 GPT-5.4-Mini GLM-5.1 Gemini-3.1-Pro Kimi-K2.5 GPT-5.4-Nano Qwen-3.6-Plus
n
Mean Lift
95% boot. CI
t-test p
Wilcoxon p
dz
41 41 41 41 41 41 41 41
+0.346 +0.318 +0.258 +0.253 +0.214 +0.211 +0.199 +0.173
[0.244, 0.451] [0.245, 0.395] [0.178, 0.340] [0.158, 0.350] [0.097, 0.320] [0.131, 0.294] [0.129, 0.272] [0.098, 0.250]
< 0.001 < 0.001 < 0.001 < 0.001 < 0.001 < 0.001 < 0.001 < 0.001
< 0.001 < 0.001 < 0.001 < 0.001 < 0.001 < 0.001 < 0.001 < 0.001
1.015 1.282 0.967 0.798 0.585 0.786 0.846 0.691
Table 9. Human-validation lift under site-paired comparison against each fixture’s unrepaired baseline. Scores are blind expert ratings on the same 1–5 rubric scale.
baseline. This paired design asks whether a judge model’s report leads to consistent improvement on the same site, rather than merely achieving a higher unpaired average. Confidence intervals are computed using percentile bootstrap resampling. We report paired t-test and Wilcoxon signedrank p-values, together with paired Cohen’s dz as an effect-size measure. Table 8 and Table 9 show that repair lift is consistently positive under both au- Statistic Estimate p 95% boot. CI tomated and human protocols. In the auSpearman’s ρ 0.635 0.091 [−0.190, 1.000] tomated protocol, the pooled lift across Kendall’s τb 0.546 0.061 [−0.091, 1.000] model–site rows is +0.176 on the 1–5 scale, with a 95% bootstrap confidence interval Table 10. Model-level rank correlation between automated of [0.140, 0.212] and a paired effect size of repair lift and human mean score across the eight evaluated models. dz = 0.538. Each individual model also has a positive confidence interval excluding zero. Human validation shows the same direction with larger effects: all repaired interfaces improve over their corresponding site baselines under blind expert ratings, with mean lift ranging from +0.173 to +0.346 and all paired tests significant. These results support the interpretation that evidence-grounded reports contain actionable information for downstream interface repair. Table 10 compares automated repair lift with human-rated interface quality at the model level.
26 / 30
UXBench
The association is positive but uncertain: Spearman’s ρ = 0.635 and Kendall’s τb = 0.546, but both confidence intervals are wide and include zero. This uncertainty is expected because the rankcorrelation analysis contains only eight model-level points. We therefore interpret the automated protocol as a useful actionability signal that is directionally aligned with human judgment, while treating close model rankings as suggestive rather than definitive.
27 / 30
UXBench
UXBench Human Evaluation Instructions Please use the assigned evaluation system to complete the UX evaluation tasks. Your goal is to evaluate each webpage variant based on direct interaction with the webpage, while keeping the evaluation blind and objective.
1. Login and Task Selection • Open the assigned evaluation system. • Enter your username and log in. • Select the batch assigned to you. • Complete the evaluation for all webpages and all variants within that batch.
2. Webpage Exploration After entering a webpage variant, first read the README. The README provides the project background, goals, and relevant context for understanding the webpage. Before scoring, please explore the webpage as thoroughly as possible: • Browse the main page content. • Click all clickable buttons, links, tabs, menus, cards, and other interactive elements. • Try filling out forms, switching states, and triggering error or success messages. • Check whether the webpage has UX issues such as insufficient feedback, unclear flow, ambiguous state, or layout problems. • For dimensions that require mobile experience evaluation, switch to a mobile rendering device before making your judgment. Please do not evaluate only the first screen or a static view. Explore the webpage as a real user would.
3. Scoring After sufficiently exploring the webpage, click the evaluation button to open the scoring interface. Rate the current webpage variant on each UX dimension based on your actual experience. Please remain objective when scoring: • Do not base your judgment on model names or personal preference. • The system displays anonymous variants such as A, B, and C by default. • Do not actively infer, reveal, or record the corresponding model names. • Scores should be based on your actual exploration of the current webpage. • If an issue appears only under a specific interaction state, page state, or mobile state, include it in the corresponding dimension rating.
4. Submission and Saving • After completing the scoring for one webpage variant, click submit. • Once all variants of a webpage have been scored, the system will automatically save the results. • After finishing the current variant, continue with another variant until all webpages and all variants in the batch are completed.
5. Important Notes • Explore each webpage thoroughly; do not rely only on the first screen or a static screenshot. • Focus on whether real users can understand the current state, the next action, the reason for errors, and the result of their actions. • Check both desktop and mobile experiences when relevant, especially for the mobile experience rating dimension. • Maintain blind evaluation. Do not actively display, search for, infer, or record model names. • Make sure every webpage and every variant in your assigned batch is fully evaluated. 15 Figure 10. Human Eval Instructions.
28 / 30
UXBench
Figure 11. README panel shown alongside each fixture. It describes the project context and intended interactions, helping reviewers understand the use case before evaluating the webpage.
Figure 12. Human rating panel used after exploration. Reviewers score each candidate webpage on the seven UX dimensions using a 1–5 scale, with model identities and automated scores hidden.
29 / 30
UXBench
Figure 13. Mobile evaluation view. Reviewers switch the rendering device to inspect responsive layout and mobile-specific interaction behavior before assigning mobile-related ratings.
30 / 30