WebCompass: Towards Multimodal Web Coding Evaluation for Code Language Models Xinping Lei† , Xinyu Che† , Junqi Xiong† , Chenchen Zhang† , Yukai Huang† , Chenyu Zhou† , Haoyang Huang, Minghao Liu, Letian Zhu, Hongyi Ye, Jinhua Hao, Ken Deng, Zizheng Zhan, Han Li, Dailin Li, Yifan Yao, Ming Sun, Zhaoxiang Zhang, Jiaheng Liu∗ Nanjing University † Equal contribution.
Kuaishou Technology
∗ Corresponding author.
arXiv:2604.18224v1 [cs.SE] 20 Apr 2026
Abstract Large language models are rapidly evolving into interactive coding agents capable of end-to-end web coding, yet existing benchmarks evaluate only narrow slices of this capability—typically text-conditioned generation with static-correctness metrics— leaving visual fidelity, interaction quality, and codebase-level reasoning largely unmeasured. We introduce WebCompass, a comprehensive, multimodal benchmark that provides a unified lifecycle evaluation of web engineering capability. Recognizing that realworld web coding is an iterative cycle of generation, editing, and repair, WebCompass spans three input modalities (text, image, and video) and three tightly coupled task types (generation, editing, and repair), yielding seven complementary task categories that closely mirror professional workflows. Through a multi-stage, human-in-the-loop pipeline, we curate high-quality instances covering 15 generation domains, 16 editing operation types, and 11 repair defect types, each annotated at Easy/Medium/Hard difficulty levels. On the evaluation side, we adopt a checklist-guided LLM-as-a-Judge protocol for editing and repair, and propose a novel Agent-as-a-Judge paradigm for generation that autonomously executes generated websites in a real browser, explores interactive behaviors via the Model Context Protocol (MCP), and iteratively synthesizes targeted test cases—closely approximating human acceptance testing. We evaluate a diverse set of representative closed-source and open-source models and observe that: (1) closed-source models remain substantially stronger and more balanced; (2) editing and repair exhibit distinct difficulty profiles, with repair preserving interactivity better but remaining execution-challenging; (3) aesthetics is the most persistent bottleneck, especially for open-source models; and (4) framework choice materially affects outcomes, with Vue consistently challenging while React and Vanilla/HTML perform more strongly depending on task type. All benchmark data1 , evaluation code2 , and project page3 are publicly available.
1
Introduction
Large Language Models (LLMs) have rapidly evolved from passive code assistants into interactive coding agents capable of implementing substantial software changes from natural-language instructions (Yang et al., 2024b; Wang et al., 2024; Cognition AI, 2024). This progress is especially evident in web development, where outputs can be directly executed, visually inspected, and iteratively refined. A growing body of work has proposed benchmarks that span different task types and input modalities for web coding (Table 1). Yet evaluating web coding is fundamentally different from evaluating traditional code generation. Success depends not only on functional correctness, but also on visual fidelity, interaction behavior, responsiveness, accessibility, and overall user experience. These aspects are difficult to capture with standard code-centric metrics such as pass@k on HumanEval (Chen et al., 2021) or unit-test pass rates on SWE-Bench (Jimenez et al., 2023), which focus on algorithmic correctness or repository-level bug fixing rather than interactive front-end applications. To address this gap, we introduce WebCompass, a unified multimodal benchmark and evaluation framework for web coding. WebCompass spans text, image, and video inputs, covers generation, editing, and repair tasks, and adopts task-aware evaluation tailored to each setting. For editing and repair, we use a checklist-guided LLM-as-a-Judge protocol (Zheng et al., 2023), which is well suited to 1 https://huggingface.co/datasets/NJU-LINK/WebCompass 2 https://github.com/NJU-LINK/WebCompass 3 https://nju-link.github.io/WebCompass/
1
Radial Comparison of Generation/Edit/Repair
7 43.2 8 46.8 .71 65 0 .0 72
6 55.7 0.22 6 68 .95
74.05 75.38
77.18 61.26 50.39
350
58
42
.62
25
Ge
n-S
55.92 7 64.0 62.26 47.06 34.56
F ir-R
Gen-DSQ
F
Repair-ITI
Edit
CT ir-R
-ITG
a Rep Edit-S
TC
60.83 58.16 56.63
2 26 7.74 .96
66.9 7 69.5 71.8 2 6
0 62.7 .14 65 2 65.8
GPT-5.2 Gemini-3-Pro-Preview Claude-4.5-Opus
TI Edit-F
Easy Medium Hard
250
25.48 26.62
22 23.5.78 3
7 24.6 0 27.3 5 48.4 6 4 54.1 41.2
PI
Number of Tasks
Gen-RUN
pa Re
61.9 68.87 3 85.54 87.30 79.33
300
Editing
Generation
.14
.70
Generation Edit Repair
Qwen3-VL-235B-A22B-Instruct Qwen3-VL-32B-Instruct
200 150
123
109
42
50
48
30
33
34
37
Text-Guided Generation
Vision-Guided Generation
Video-Guided Generation
Figure 1: Radar chart of model performance across all seven task types in WebCompass.
300
68
68
114
114
300
300
90
90
120
120
90
90
Diagnostic Repair
Visual-Diagnostic Repair
94
100
0
Repair
300
45
44
118
118
Text-Guided Editing
Vision-Guided Editing
Figure 2: Difficulty distribution of WebCompass.
Table 1: Comparison with prior web coding benchmarks. WebCompass is the first to support all three task types across text, image, and video modalities. Gen.=Generation, Edit=number of supported editing categories, Rep.=number of supported repair categories, Multi-page=project-level multi-page testing, Interact.=interactive functionality evaluation, Visual=aesthetics and visual fidelity evaluation, Agentic=Agent-as-a-Judge dynamic testing (using LLM agents to interact with browsers and synthesize tests), Reverse=reverse-engineered deterministic repair tasks. A red cross indicates that the task family is not supported. Data sizes are reported as the number of tasks or question-answer pairs. Benchmark
Size
Gen.
Edit (#)
Rep. (#)
Multi-page
Interact.
Visual
Agentic
Reverse
✓ ✓ ✓ ✓ ✗ ✓ ✓
✓ ✓ ✗ ✗ ✓ ✓ ✗
✗ ✗ ✗ ✗ ✗ ✗ ✗
✗ ✗ ✗ ✗ ✗ ✗ ✗
Input Modality
Generation-Only Benchmarks Interaction2Code (Wan et al., 2024) FronTalk (Wu et al., 2025) Web-Bench (Xu et al., 2025) FrontendBench (Zhu et al., 2025) WebApp1K (Cui, 2024) IWR-Bench (Chen et al., 2025) WebGen-Bench (Lu et al., 2025)
504 1000 1000 148 1000 113 101
✓ ✓ ✓ ✓ ✓ ✓ ✓
✗ ✗ ✗ ✗ ✗ ✗ ✗
✗ ✗ ✗ ✗ ✗ ✗ ✗
✗ ✓ ✓ ✗ ✓ ✓ ✓
Image Text
Image
Text
Image
Text Text Video Text
Multi-Task Benchmarks SWE-bench MM (Yang et al., 2024a) DesignBench (Xiao et al., 2025)
517 900
✗ ✓
3 6
4 6
✓ ✓
✗ ✗
✗ ✓
✗ ✗
✗ ✗
Image
WebCompass (Ours)
1526
✓
16
11
✓
✓
✓
✓
✓
Text
Text
Image
Image
Video
patch-based outputs with constrained solution spaces. For generation, we propose an Agent-as-a-Judge protocol (Zhuge et al., 2024), in which an autonomous agent launches the generated website in a real browser, explores it through MCP, synthesizes targeted test cases, and scores the result based on execution. This design reflects the differing nature of web coding tasks. Editing and repair are localized and checklistaligned, making diff-level inspection and before/after screenshots sufficient for reliable evaluation. Generation, by contrast, is open-ended and long-horizon, with correctness often depending on multi-step runtime behavior that static inspection cannot capture. By combining multimodal task coverage with execution-based evaluation, WebCompass provides a more realistic and scalable benchmark for assessing web coding agents. Contributions. (1) Unified lifecycle coverage. Unlike prior benchmarks that target isolated tasks or modalities (Table 1), WebCompass jointly evaluates generation, editing, and repair across text, image, and video inputs, enabling cross-task and cross-modality comparisons within a single framework. (2) Rigorous and deterministic task construction. We refine underspecified queries into structured design documents for generation, synthesize context-consistent requirements without leaking implementation details for editing, and provide exact search/replace annotations mapping buggy code to clean targets for repair, ensuring reproducible evaluation. (3) Task-aware evaluation paradigms. We introduce an Agent-as-a-Judge protocol that combines real-browser interaction with iterative test-case synthesis for open-ended generation tasks, complementing checklist-guided LLM-as-a-Judge for constrained patch-based tasks.
2
Figure 3: Overview of WebCompass. The benchmark supports three input modalities (text, image, video) and three task types (generation, editing, repair), resulting in seven complementary task categories that cover the full lifecycle of web development.
2
WebCompass
2.1
Overview
WebCompass supports three input modalities (text, image, and video) and three types of web coding tasks (generation, editing, and repair), resulting in seven task categories: Text-Guided Generation (text-conditioned web generation), Vision-Guided Generation (image-conditioned web generation), VideoGuided Generation (video-conditioned web generation), Text-Guided Editing (text-instructed web editing via patches), Vision-Guided Editing (image-grounded web editing via patches), Diagnostic Repair (textdescribed web repair via patches), and Visual-Diagnostic Repair (image-grounded web repair via patches). Each task is designed to closely reflect real-world development scenarios. We define each task as follows: 1. Text-Guided Generation. The input is a textual specification of a target web page, consisting of three aspects: (i) page content, (ii) interaction behaviors, and (iii) visual appearance. The model is required to output a complete web code repository that satisfies the specification. 2. Vision-Guided Generation. The input comprises multiple screenshots of a web page. Beyond presenting content, layout, and visual styling, the screenshots are also intended to capture interactive functionalities. Depending on the data source, we consider two types of screenshot sets: (i) a collection covering the main page and its subpages, and (ii) a sequence capturing page state changes during browsing. The model is required to reproduce a web code repository whose visual appearance and functionality match the screenshots. 3. Video-Guided Generation. The input is a screen-recorded browsing video containing multiple user interactions. The model is required to generate a web code repository whose appearance and functionality are consistent with those demonstrated in the video. 4. Text-Guided Editing. The input includes a web code repository and a text-based editing instruction. The model is required to output a code patch that edits the repository such that the updated web page meets the instructions. 5. Vision-Guided Editing. The input includes a screenshot of the current web page, the corresponding web code repository, and an editing instruction. The model is required to output a code patch that modifies the repository so that the edited web page satisfies the instruction. 3
6. Diagnostic Repair. The input includes a web code repository and a textual description of the existing issues. The model is required to output a code patch that repairs the repository and resolves the described problems. 7. Visual-Diagnostic Repair. The input includes a screenshot of the current web page, the web code repository, and a description of the existing issues. The model is required to output a code patch that repairs the repository and resolves the described problems. Taken together, WebCompass serves as a comprehensive benchmark to evaluate the capabilities of multimodal models in realistic web engineering scenarios. Beyond basic code generation, it rigorously assesses a model’s proficiency across several critical dimensions: (1) Nuanced User Intent Understanding, encompassing layout structure, aesthetic design styles, and complex interaction logic; (2) Fine-grained Cross-modal Reasoning, requiring precise alignment between visual inputs (images/videos) and code implementations; (3) Repository-level Context Awareness, testing the ability to maintain consistency within existing codebases during editing and repairing; and (4) Diagnostic & Problem-Solving Skills, specifically for identifying and fixing semantic or visual anomalies.
2.2
Data Collection
To ensure the benchmark reflects real-world scenarios, we employ a multi-stage, human-in-the-loop pipeline to construct a high-quality benchmark covering all seven task types. Figure 4 illustrates the overall process. 2.2.1
Text-Guided Generation.
We design the Text-Guided Generation set to (i) contain realistic and actionable requirements and (ii) cover diverse web page types. We therefore collect initial queries from multiple complementary sources: WebGen-Bench (Lu et al., 2025) (manually constructed queries), ArtifactsBench (Zhang et al., 2025) (diverse page categories with rigorous filtering), BigCode Arena (real user requests), and high-quality web showcases from V0 (an AI IDE for web coding). These sources form our initial query pool. To reduce redundancy, we embed queries using BGE-M3 and perform k-means clustering to obtain a deduplicated candidate set. We then use an LLM to assign category and difficulty labels to each query (five independent annotations per query), taking the majority vote as the final label. Finally, we perform stratified sampling across categories and difficulties to obtain 123 text-guided generation queries. However, we observe that queries from everyday usage scenarios are often underspecified, leading to large variations in generated pages across models. While such low-constraint queries can test a model’s creativity, they hinder automated evaluation because creativity and implicit-intent matching are subjective and difficult to judge automatically—it is unclear whether the model is being “overly clever” or truly aligned with user intent. To address this, we prompt an LLM to act as a product manager and elaborate each underspecified request into a structured web design document covering (1) page content, (2) interaction behaviors, and (3) visual appearance. 2.2.2
Vision-Guided Generation.
Although many existing datasets include webpage screenshots, most contain relatively simple UIs that are insufficient to challenge modern models. We observe that WebRenderBench provides a large number of visually complex webpages, but typically only includes a single screenshot per website. We thus perform data augmentation: we parse the subpage URLs referenced in index.html, randomly select two, and use Playwright to capture their screenshots. To further test whether models can reproduce multi-page websites and their dependency relationships, we inject a JavaScript overlay into the main-page screenshot to highlight the positions of subpage URLs with colored bounding boxes. Due to network instability and dynamic content loading, screenshots may contain artifacts. We therefore conduct multiple rounds of LLM-based verification as an initial filter, followed by manual inspection. In addition, most existing datasets only provide static screenshots and lack dynamic webpage content. Although Interaction2Code (Wan et al., 2024) supplies multiple images to convey certain interaction information, it still cannot adequately represent animations and complex interaction patterns. To fill this gap, we browse diverse webpages from V0 and Figma and manually extract keyframes capturing critical state changes. These two components—augmented multi-page screenshots and dynamic keyframe sequences—together constitute the Vision-Guided Generation test set.
4
Generation Text-Guided Generation
1. web content 2. Interaction 3. Visual Design
ArtifactsBench BigCode Arena
Prototypes Collection for Editing & Repair Task
Web Design Doc
WebGen-Bench
Query Pool
Multi-Stage Filtering Length Check
Filtering Classifi- Query cation Enhancement Sampling
V0 showcase
Dataset: Text
Vision-Guided Generation
Code Review WebRenderBench Prototypes
DOM Parsing Via JavaScript Injection
· Complex components · Interactions/Animations · Application Scenarios
Screen Capture
Interaction Recording
Web Prototypes ...
Dataset: Video
Web Prototypes Multi-page Website
• • • • •
Dataset: Image
Expansion
11 Repair Types • • • • •
Aggregated Requirements • UI updates • Interaction flows • state feedback
Data Table Rich Text Editor Drag & Drop Interface Parallax Scrolling ......
Occlusion Crowding Text Overlap Alignment ......
LLM
Defect Types Faulty Website
Repair
Dynamic websites (V0, Figma)
Select 50 high-quality prototypes
16 Editing Types
Visual-based Page State Change Detection
Video-Guided Generation Path Planning
Human Curation
Requirements
Single-page Website
Editing
Real interactive Interaction Mode webpages Recognition
Repair
Editing
Figure 4: Data construction pipeline for WebCompass. Top: prototypes are collected through multi-stage filtering, manual selection, and page-level expansion. Bottom: each prototype is converted into editing tasks (left, green) or repair tasks (right, red) following task-type–specific procedures. 2.2.3
Video-Guided Generation.
Compared to text and images, videos can more clearly convey dynamic effects such as animations and multi-step interactions. To emphasize this advantage, we manually select webpages from V0 and Figma with rich dynamic behaviors across different categories, browse them, and record interaction videos. Annotators are instructed to first explore each webpage, plan a comprehensive exploration path, and then conduct the final recording to ensure thorough coverage of all interactive features. 2.2.4
Editing & Repair Task Data Collection Pipeline
Prototype Collection for Editing & Repair. Both editing and repair tasks share a common pool of high-quality web prototypes (Figure 4, top). We construct these prototypes from the WebRenderBench test set via a three-stage pipeline: length filtering → automatic quality scoring → human curation, then expand each selected prototype into single-page and multi-page variants. • Stage 1: Length filtering. We constrain the total character count across all code files to 32k–64k, with each individual file no longer than 48k characters. These bounds approximate the multi-file coordination complexity of medium-to-large front-end projects, while avoiding overly small instances (lacking difficulty) or overly large ones (inducing context truncation and unstable evaluation). • Stage 2: Quality scoring. For candidates satisfying the length constraints, we use GPT-4o to perform a code review on a 10-point scale and retain those scoring ≥ 9, yielding 81 candidates. • Stage 3: Human curation and expansion. We manually select 50 high-quality prototypes. Each prototype is kept as a S INGLE - PAGE website and additionally extended into a M ULTI - PAGE website by adding extra pages, inter-page navigation, and shared resources. Together, the two variants constitute the Web Prototypes used for all downstream task construction. Text-Guided and Vision-Guided Editing. Starting from each web prototype as the executable source website, we create editing instances by introducing new or enhanced requirements aligned with 16 predefined high-level task types covering complex components (e.g., data tables, rich-text editors, dragand-drop interfaces), interaction/animation effects (e.g., parallax scrolling), and holistic application scenarios (Figure 4, bottom-left). For every task type, we aggregate requirements that specify what to change—including UI updates, interaction flows, and state feedback—while deliberately omitting implementation details (e.g., class names, selectors, or CSS values) to ensure fairness and realism. The resulting requirements, paired with the source website, form the editing instances; Vision-Guided variants additionally supply a reference screenshot in lieu of (or alongside) the textual instruction. 5
Diagnostic and Visual-Diagnostic Repair. Repair tasks are constructed in a verifiable reverse manner (Figure 4, bottom-right). We treat a clean web prototype as the destination and use an LLM to inject explainable, observable front-end defects drawn from 11 repair types, producing the faulty source website. The model is then required to repair the source back to the destination. The injected defects span three dimensions: • Visual layout: occlusion, crowding, text overlap, misalignment, insufficient contrast, overflow, and distorted proportions. • Semantics & structure: incorrect semantic/nesting structures and missing attributes. • Interaction usability: broken interactions and loss of interactivity. We then generate natural-language repair instructions that provide vague hints about potential defect types or underlying issues, rather than a complete description of the problem, ensuring no implementation details are leaked. To guarantee determinism and support automatic evaluation, each repair instance includes an exact text-level modification annotation (search/replace) that is the strict inverse of the defect-injection edits. This design ensures (i) a uniquely correct, runnable solution, (ii) reproducible transformation from source to destination, and (iii) automated verification and error localization. Throughout, we enforce contextual consistency and the “specify goals, not methods” principle. Ecological validity of injected defects. The 11 defect categories are not arbitrarily chosen. They are the product of a systematic analysis of over 200 real-world community submissions on V0 and corresponding GitHub Issues, from which we identified the most frequently occurring front-end anti-patterns. Each category (e.g., Occlusion, Overflow, Loss of Interactivity) corresponds to a high-frequency failure mode observed in practice. By grounding our synthetic defects in this empirical distribution, we ensure ecological representativeness—models are tested on the kinds of bugs they are most likely to encounter in real-world web development, rather than on artificial corner cases.
2.3
Quality Control
We apply a multi-layered quality assurance process across all task types: Automated checks. Before human review, every instance passes through a suite of automated validators: (i) all code repositories must compile and render without fatal errors in a headless Chromium environment; (ii) editing and repair patches must apply cleanly to their respective source repositories; and (iii) repair search/replace annotations are verified to be the exact inverse of the defect-injection edits, guaranteeing a unique, deterministic solution. LLM-assisted screening. We use an LLM to perform multi-round quality checks on generated requirements and screenshots. For Vision-Guided Generation, the LLM verifies that screenshots are complete (no blank regions, missing assets, or broken layouts caused by network issues). For edit and repair tasks, the LLM checks that natural-language instructions are unambiguous, do not leak implementation details, and are consistent with the underlying code changes. Human curation. All instances undergo a final round of expert human review. Annotators verify (i) the correctness and completeness of task descriptions, (ii) the visual quality of screenshots and videos, (iii) the appropriateness of difficulty labels (Easy/Medium/Hard), and (iv) the alignment between requirements and ground-truth patches. Instances that fail any criterion are revised or discarded.
2.4
Dataset Statistics
We propose a fine-grained taxonomy for the generation, editing, and repair tasks, as detailed in Table 2. The generation task encompasses 15 distinct domains: “E-commerce & Fintech”, “Enterprise & Productivity”, “Social & Communication”, “Data Science & Analytics”, “Content Creation & Multimedia”, “Entertainment & Streaming”, “Game Development & Gaming”, “Education & Learning”, “Simulation & Scientific Modeling”, “Infrastructure & System Management”, “DevTools & Engineering”, “Logic & Workflow Visualization”, “Location Services & Transit”, “Information & Personal Branding”, and “Lifestyle & Niche Utilities”. The editing task consists of sixteen operation types: Data Table, Rich Text Editor, Drag & Drop Interface, Tree View, Real-time Dashboard, Infinite Scroll, Async Form Validation, File Upload with Progress, Parallax Scrolling, Page Transitions, Particle Effects, Skeleton Loading, Shopping Cart, User Authentication, Multi-step Wizard, and Notification Center. The repair task addresses eleven types of front-end defects spanning visual, semantic, and interactive dimensions: Occlusion, Crowding, Text Overlap, Alignment, Color & Contrast, Overflow, Sizing/Proportion, Loss of Interactivity, Semantic Error, Nesting Error, and Missing Attributes.
6
Table 2: Detailed taxonomy of Generation, Editing, and Repair tasks in WebCompass. Generation covers 15 application domains; Editing defines 16 modification operations; Repair addresses 11 front-end defect types spanning visual, semantic, and interactive dimensions. Generation (15 Types) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
E-commerce & Fintech Enterprise & Productivity Social & Communication Data Science & Analytics Content Creation & Multimedia Entertainment & Streaming Game Development & Gaming Education & Learning Simulation & Scientific Modeling Infrastructure & System Mgmt. DevTools & Engineering Logic & Workflow Visualization Location Services & Transit Information & Personal Branding Lifestyle & Niche Utilities
Editing (16 Types) 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
Data Table Rich Text Editor Drag & Drop Interface Tree View Real-time Dashboard Infinite Scroll Async Form Validation File Upload with Progress Parallax Scrolling Page Transitions Particle Effects Skeleton Loading Shopping Cart User Authentication Multi-step Wizard Notification Center
Repair (11 Types) 1 2 3 4 5 6 7 8 9 10 11
Occlusion Crowding Text Overlap Alignment Color & Contrast Overflow Sizing/Proportion Loss of Interactivity Semantic Error Nesting Error Missing Attributes
Our benchmark comprises a total of 1526 tasks, distributed as follows: 123 for Text-Guided Generation, 109 for Vision-Guided Generation, 94 for Video-Guided Generation, 300 for Text-Guided Editing, 300 for Vision-Guided Editing, 300 for Diagnostic Repair, and 300 for Visual-Diagnostic Repair. Each task is annotated with a difficulty level (Easy, Medium, or Hard) based on the complexity of the required functionality, the number of interactive components, and the sophistication of the visual design. A detailed breakdown of per-category counts is provided in Figure 2.
2.5
Task Type Descriptions
To comprehensively evaluate models across a wide spectrum of real-world web development challenges, WebCompass defines 15 generation application domains, 16 diverse editing task types, and 11 repair defect types. Table 2 provides an overview, and the following subsections detail each editing and repair task type. 2.5.1
Editing Task Types
The 16 editing task types span from low-level UI components to full business workflows, ensuring broad coverage of frontend engineering skills. They are organized into four categories: Complex Components. This category includes Data Table (sortable, paginated, filterable table with row selection and inline editing), Rich Text Editor (WYSIWYG editor with formatting toolbar, link/image insertion, and form-synced output), Drag & Drop Interface (draggable items with drop-zone feedback, cross-container reordering, and state persistence), and Tree View (nested expand/collapse tree with cascading selection and search filtering). Frontend–Backend Integration. This category covers Real-time Dashboard (live-updating metric cards with animated counters and sparkline charts), Infinite Scroll (scroll-triggered lazy loading with skeleton placeholders and end-of-content handling), Async Form Validation (debounced server-side validation with inline status indicators and submit gating), and File Upload with Progress (drag-and-drop upload with per-file progress bars, queue management, and cancel support). Advanced Animations. This category encompasses Parallax Scrolling (multi-layer differential scroll speeds with viewport-triggered fade/scale effects), Page Transitions (coordinated enter/exit animations
7
such as fade, slide, and zoom between SPA content views), Particle Effects (canvas-based particle system with physics, cursor interaction, and connection lines), and Skeleton Loading (shimmer-animated placeholders matching content structure with smooth reveal). Business Scenarios. This category includes Shopping Cart (full cart flow with quantity controls, real-time totals, and localStorage persistence), User Authentication (login, registration, and password-recovery forms with validation and auth state management), Multi-step Wizard (step indicator with per-step validation, cross-step data persistence, and review summary), and Notification Center (notification dropdown with unread badge, categorized alerts, and mark-as-read actions). 2.5.2
Repair Defect Types
The 11 repair defect types cover visual, semantic, and interactive failure modes commonly encountered in frontend development, organized into three dimensions: Visual Layout. This dimension includes seven defect types: Occlusion (one element covers another due to incorrect z-index stacking), Crowding (spacing between elements is removed or collapsed, causing visual clutter), Text Overlap (text overflows its container and overlaps with adjacent content), Alignment (elements are offset from their expected grid or sibling alignment), Color & Contrast (text color is too close to the background, reducing readability), Overflow (content exceeds a fixed-size container without proper overflow handling), and Sizing/Proportion (elements are given extreme or distorted dimensions). Semantic Correctness. This dimension includes Semantic Error (semantic HTML tags replaced with non-semantic equivalents, e.g., <h1> replaced by <div>) and Nesting Error (invalid HTML nesting, e.g., <a> inside <a>, or <div> inside <p>). Interactive Usability. This dimension includes Loss of Interactivity (interactive elements disabled or blocked via pointer-events: none) and Missing Attributes (accessibility or functional attributes removed, e.g., alt, aria-label).
3
Evaluation Methodology
We adopt task-specific evaluation paradigms tailored to the output characteristics of each task family. For Editing & Repair, where models produce localized code patches, we use LLM-as-a-Judge (§3.1). For Generation, where correctness depends on end-to-end runtime behavior, we use Agent-as-a-Judge (§3.2). Both paradigms score along three axes—executability, functional, and visual—whose operationalization is task-dependent. For Generation: Runnability (build and launch success), Spec Implementation (functional behavior matches the design document), and Design Quality (visual polish). For Editing: Instruction Targeting (patch applies and targets the instruction’s required locations), Feature Integrity (original interactions preserved and new components functional), and Style Conformance (visual edit landed and unchanged regions consistent). For Repair: Root-Cause Targeting (patch applies and targets the defect’s root cause), Interaction Integrity (interactions preserved and interactive-class defects repaired), and Reference Fidelity (visual match to the ground-truth fixed screenshot). We select the judge model based on highest agreement with human annotations (§4.3.1).
3.1
LLM-as-a-Judge for Editing & Repair
For each instance, we apply the predicted patches to the source repository, discard blocks that fail to apply, and launch the modified project in a headless Chromium browser to capture screenshots (Figure 5). The judge receives the original task requirement, the source repository, the model-generated patch, build and runtime logs after patch application, and before/after screenshots captured in headless Chromium. For Repair tasks, it additionally receives the defect description, the ground-truth modifications, and the reference fixed screenshot. It scores checklist items independently along the three task-specific dimensions (0–10 each), produces evidence-grounded structured JSON output, and aggregates the resulting dimension-wise scores into the final task score.
3.2
Agent-as-a-Judge for Generation
Traditional evaluation approaches for web generation fall into two camps, each with a critical blind spot. Pure test-based methods (e.g., unit tests or DOM assertions) can programmatically verify functional correctness—whether a button triggers the right callback or a form validates inputs—but cannot assess 8
(a) Context and Evidence Collection Headless Chromium Task Instance
Capture Screenshots
Models Being Tested
Generated Patches
Apply Patches Filter out Mismatches
Source Repository
Before Fix
Modified Project
After Fix
Ground Truth
(b) Task-Specific Prompt Construction Prompt Construction
LLM as a Judge
Edit Task Context
System Role: You are a Senior
Repair Task Context Defect Descriptions
Natural-Language Instructions
Software Architect. Your goal is to evaluate the quality of a code patch ... Task Context: The button doesn't work ... Evaluation Evidence: <Generated Code Modification>,<Original UI Screenshot>,<Modified UI Screenshot> ....
Generated & Ground-Truth Code Modifications
Generated Code Modifications
Before Fix Screenshot
Original UI Screenshot
After Fix Screenshot
Modified UI Screenshot
Ground-Truth Screenshot
(c) Multi-Dimensional Scoring Mitigate Judge Bias Evidence-Grounded Response
Final Evaluation Report
Judge Model Selection
• Detailed Reasoning
• Compare Models
• Visual Comparison
• Highest Human Agreement
• • •
Multi-Dimensional Score
Runnability (RUN) Spec Implementation (SPI) Design Quality (DSQ)
Final Average Score
Figure 5: Illustration of the LLM-as-a-Judge evaluation pipeline.
(a) Context ＆ Evidence Collection Standardized Evaluation Checklist
Task Instance
Atomic items 1.Task Name 2.Reference Action Sequence 3.Expected Outcome 4.Evaluation Criteria 5.Point Value
Source Repository
Models Being Tested
Checklist Generation Model
Manual Review
(b) Dynamic Interaction & Iterative Code Verification Iterative Code Analysis Loop
Safeguards
(1)Selector-only Adaptation (2)Checklist-anchored Assertions (3)Hard-evidence Grounding
Judge Agent
Synthesize Test Cases (Javascript)
Perform User Interactions
<[DOM selector', ID : I.. '] ..(‘>
Analyze Code for Selector Match
Model Context Protocol(MCP)
Execute Tests via MCP
Chrome_dev
Open Browsers
(c) Checklist-Guided Scoring Auditable Evidence Collection
Evidence-Grounded Checklist Scoring
Click Capture Elements Screenshots
Structured Final Evaluation Report
Final Score • • • • • Repair • Edit
Interactive Screenshots
Test Case Results
Console Before/After Comparison Logs Screenshots
Minimizing Biases (e.g., via anchored rubrics)
...
Instruction Targeting (ITG) Feature Integrity (FTI) Style Conformance (STC) Root-Cause Targeting (RCT) Interaction Integrity (ITI) Reference Fidelity (RFF)
Justification with Citable Evidence
Figure 6: Agent-as-a-Judge evaluation pipeline. The MCP bridge enables bidirectional communication: the agent sends interaction commands to the browser and receives DOM snapshots, console logs, and screenshots as evidence.
9
visual fidelity, layout harmony, or aesthetic quality. Conversely, screenshot-based comparison methods can capture visual appearance but struggle to verify multi-step interactive behaviors, state transitions, and dynamic content that only manifest through real user interaction. Human acceptance testing naturally combines both capabilities: a QA engineer can inspect the UI visually and write ad-hoc test scripts to probe edge cases, switching fluidly between the two modes. To approximate this dual capability in an automated setting, we adopt Claude Code as the evaluation orchestrator paired with the Model Context Protocol (MCP) for browser control. This architecture is deliberately chosen because it endows the judge agent with two complementary verification channels: (1) as a code agent, it can dynamically synthesize and execute JavaScript test cases that programmatically inspect DOM states, CSS properties, and functional logic with deterministic precision; and (2) through the MCP bridge to a real browser, it can simulate authentic user interactions—clicking, scrolling, typing, navigating—while capturing screenshots and console logs as auditable evidence. Neither channel alone suffices: scripted tests miss visual quality, and browser interaction alone cannot efficiently verify complex state invariants. Their combination enables a unified evaluation loop that closely mirrors how a human tester would accept or reject a web application. Figure 6 illustrates our Agent-as-a-Judge pipeline. A Code Agent, augmented with the Model Context Protocol (MCP) for real-browser control, evaluates each generated website in four stages: (1) Checklist generation: an LLM produces a structured evaluation checklist defining tasks, interaction sequences, expected outcomes, and score values; this checklist remains fixed throughout to prevent circular reasoning. (2) Browser interaction: the agent launches the website in headless Chromium, executes checklist interactions (clicking, typing, scrolling, navigation), and records DOM snapshots, console logs, and screenshots as auditable evidence. (3) Adaptive code verification: the agent synthesizes executable JavaScript test cases for each checklist item, programmatically verifying DOM states, CSS properties, and functional behaviors. Crucially, when implementation details differ from expectations, the agent adapts only DOM locators (e.g., element selectors and IDs) while keeping all behavioral assertions unchanged—ensuring that evaluation criteria remain anchored to the original specification rather than drifting toward the model’s output. Failed tests trigger an iterative debugging loop in which the agent inspects the actual code, diagnoses the mismatch, and re-attempts verification before assigning a score. (4) Evidence-grounded scoring: the agent scores each item along Runnability, Spec Implementation, and Design Quality with structured justifications; scores lacking auditable evidence (screenshots, test results, or console logs) are discarded. Three safeguards prevent evaluation bias: checklist immutability (no new criteria after Stage 1), selectoronly adaptation in Stage 3, and mandatory hard-evidence grounding for every score. All experiments are conducted on a Linux server with per-task execution timeouts to prevent infinite loops or hanging processes. For generation evaluation, we use Claude Code (v2.0.67) as the evaluation orchestrator together with the Chrome DevTools MCP Server (v0.19.0), which provides headless Chromium rendering, DOM inspection, and browser automation capabilities.
3.3
Scoring and Failure Handling
Scoring formula. For each task instance, let {s1 , s2 , . . . , sn } denote the individual checklist item scores and {m1 , m2 , . . . , mn } their corresponding maximum scores. Each item’s normalized score is ri = si /mi . To prevent a single zero-scored item from collapsing the entire task score, we apply a smoothing constant ϵ = 1 to any item where si = 0, replacing its normalized score with ϵ/mi . The task-level score is then computed as the harmonic mean of the normalized item scores: stask =
n ∑in=1 r1i
(1)
We choose the harmonic mean over the arithmetic mean because it penalizes imbalanced performance: a web artifact that excels on some criteria while completely failing on others should not receive a high overall score. This score is computed separately for each of the three per-task evaluation dimensions. Handling cascading failures. Web generation tasks frequently produce outputs that fail at different stages of the build–render–interact pipeline, and a naïve application of the scoring formula could yield misleading results. We define explicit fallback strategies for three failure scenarios: 1. Complete build failure (the project does not compile or launch): the functional and visual dimensions are set to 0; only the executability dimension contributes a meaningful score. 2. Partial rendering failure (the project launches but some pages or components fail to render): the executability dimension is penalized proportionally; 10
the visual dimension is evaluated on the rendered portion (or set to 0 if nothing is visible); the functional dimension is evaluated only on reachable components. 3. Runtime crash (the project renders initially but crashes during interaction): the executability and visual dimensions are scored on the initial render; the functional dimension is scored only on the testable subset, with untestable items receiving 0. These fallback rules ensure that cascading failures degrade scores gracefully rather than producing undefined or inflated results, faithfully reflecting the progressive nature of web application quality.
4
Experiments
This section describes our experimental setup and provides a comprehensive overview of results. We organize experiments by task type (generation / editing / repair) and input modality (text / image / video). Beyond the main results (§4.2), we report several focused analyses: judge model selection (§4.3.1), framework-based subset evaluation (§4.3.2), difficulty-level analysis (§4.3.4), and impact of thinking mode (§4.3.8).
4.1
Evaluated LLMs and Frameworks
We report main benchmark results for ten models from both closed-source and open-source families. All selected models natively support text, image, and video inputs, allowing us to use the same model set across all modalities. Full model details, including auxiliary comparison variants used in later analyses, are provided in Appendix A.4. We employ Claude Code (v2.0.67) as the evaluation orchestrator and the Chrome DevTools MCP Server (v0.19.0) for browser rendering, DOM inspection, and automated interaction verification in a headless Chromium environment. This setup enables an agent-based evaluation pipeline that programmatically assesses the functional correctness and UI consistency of generated web applications.
4.2
Main Results
Table 3 presents the overall and per-task-type scores for all evaluated models. Table 3: Comparison of models across different task types. Green bold indicates the best score in each column; blue underline indicates the second best. Each task has three evaluation dimensions: Generation uses Runnability (RUN), Spec Implementation (SPI), and Design Quality (DSQ); Editing uses Instruction Targeting (ITG), Feature Integrity (FTI), and Style Conformance (STC); Repair uses Root-Cause Targeting (RCT), Interaction Integrity (ITI), and Reference Fidelity (RFF). Overall is the arithmetic mean of all nine dimension scores. Generation
Models
RUN.
SPI.
DSQ.
Editing ITG.
FTI.
Repair STC.
Overall
RCT.
ITI.
RFF.
48.45 54.16 53.18 41.24 40.44
85.54 87.30 86.84 79.33 80.63
65.71 72.00 71.65 58.70 61.31
67.40 66.68 65.42 61.90 57.12
68.87 68.74 61.93 51.85 50.71
46.88 46.28 43.27 31.31 31.35
41.14 38.78 35.20 29.58 28.08
Closed-Source Large Language Models Claude-Opus-4.5 Gemini-3-Pro-Preview Gemini-3-Flash-Preview GPT-5.2 Claude-Sonnet-4.5
77.18 74.05 74.87 75.38 65.30
68.95 55.76 54.32 60.22 50.37
62.26 64.07 62.42 55.92 56.78
71.86 69.52 65.95 66.97 60.06
65.82 65.14 62.35 62.70 53.71
60.83 58.16 57.21 56.63 45.51
Qwen3-VL Series Open-Source Large Language Models 235B-A22B-Instruct 235B-A22B-Thinking 32B-Instruct 30B-A3B-Thinking 30B-A3B-Instruct
61.26 63.86 50.39 47.37 41.79
42.14 35.02 25.62 20.87 20.80
47.06 45.21 34.56 37.47 29.28
27.74 22.15 26.96 19.82 20.57
25.48 21.67 26.62 21.21 20.97
23.53 19.06 22.78 18.20 17.93
27.30 27.02 24.67 18.08 19.32
Several key patterns emerge. Model ranking and the closed–open gap. Claude-Opus-4.5 and Gemini-3-Pro-Preview achieve the highest Overall scores (67.40 and 66.68, respectively) with complementary strengths: Claude leads 11
Generation RUN (77.18) and Editing ITG (71.86), while Gemini leads Repair RCT (54.16) and Repair RFF (72.00). The closed–open gap is substantial: the best open-source model (Qwen3-VL-235B-A22B-Instruct) reaches an Overall of 41.14, trailing the top closed-source model by over 26 points. Smaller open models (30B variants) fall further, reaching less than half the top closed-source scores. Task-type patterns. For closed-source models, Generation and Editing consistently follow the ordering executability > functional > visual (e.g., Claude-Opus-4.5: RUN 77.18 > SPI 68.95 > DSQ 62.26 on Generation). Repair shows a different pattern: ITI ≫ RFF > RCT (e.g., Gemini-3-Pro-Preview: 87.30 > 72.00 > 54.16). This ordering is explained by the task structure: Interaction Integrity trends high because 9 of 11 defect types are visual or semantic—the interactive layer is rarely affected, so preservation is nearly automatic; the 2 interactive-class defects (Loss of Interactivity, Missing Attributes) are localized enough that models can usually repair them. Reference Fidelity is mid-range because matching the gold fixed screenshot is nontrivial. Root-Cause Targeting is lowest because correctly locating the defect’s root cause without introducing new errors remains the hardest part of repair. Note that the functional and visual axes measure different capabilities across tasks: Editing’s Feature Integrity tests both preservation and new-component functionality, whereas Repair’s Interaction Integrity is primarily regression safety; similarly, Editing’s Style Conformance evaluates edit outcome fidelity, while Repair’s Reference Fidelity measures closeness to a gold reference. Editing is especially challenging for open-source models, where scores fall to 18–28 across dimensions, revealing a major gap in context-aware code modification. Visual quality as the persistent bottleneck. Across all ten models, the visual dimension is the lowestscoring axis in Generation and Editing (Design Quality and Style Conformance, respectively). Even Gemini-3-Pro-Preview, the strongest model on this axis, reaches only 64.07 on Generation DSQ. The gap is wider for weaker models and consistent across task types. Notably, Gemini-3-Pro-Preview and Gemini-3-Flash-Preview outperform GPT-5.2 on the visual axis despite comparable executability scores, indicating that visual fidelity and functional correctness do not scale in lockstep.
4.3
Further Analysis
4.3.1
Judge Model Selection
To validate automated evaluation reliability, we compare three Claude-family judge models (Opus-4.5, Sonnet-4.5, Haiku-4.5) against human judgments on a 200-sample subset. As shown in Table 4, ClaudeOpus-4.5 achieves the highest human agreement (Pearson r of 0.93–0.96 across tasks), and is adopted as the default judge. Notably, all judges show higher agreement on edit/repair tasks than on generation, consistent with the more constrained solution space of patch-based tasks. As shown in Figure 7, a comparison of full model rankings between the agent-based evaluator and human annotators further confirms strong alignment, with most rank differences being zero or at most one, validating the automatic evaluation protocol as a reliable proxy for human judgment. Table 4: Judge model comparison. We report human agreement (Pearson r) and estimated cost per sample. Green bold: best; blue underline: second best. Judge Model Claude-Opus-4.5 Claude-Sonnet-4.5 Claude-Haiku-4.5
Generation
Editing
Repair
Cost Analysis
r
r
r
Cost
0.93 0.88 0.76
0.94 0.90 0.79
0.96 0.89 0.81
$4.66 $2.34 $1.02
This comparison also reveals a clear cost–quality trade-off. The cost column in Table 4 reports the average API token expenditure (in USD) for evaluating a single task instance. Claude-Sonnet-4.5 is cheaper but consistently trails Opus in agreement, while Haiku shows a substantial drop in alignment despite the lowest cost. We therefore choose Opus-4.5 as the default judge because judge reliability is foundational to benchmark validity, and the additional evaluation cost is justified by the stronger agreement with human assessment. 4.3.2
Subset Evaluation on Different Front-End Frameworks
To assess how framework choice affects model performance, we evaluate four representative models (GPT-5.2, Gemini-3-Pro-Preview, Claude-Opus-4.5, Qwen3-VL-235B-A22B-Instruct) on a 180-task subset (60 per task category), each completed in React, Vue, and Vanilla (plain HTML/CSS/JS). Figure 8 presents the overall scores; per-dimension breakdowns are in Appendix A.3.
12
Rank (lower is better )
Generation 5
5 4
4 3
3 2
2
Repair
5
5
4
4
3
3
2 1
1
5
5
4
4
3
3
2
1
1
ude
Cla
Editing
Agent Rank Human Rank
2
2
5
4
3
2
1
1
1
-1
+0
+0
+0
+0
+1
-1
+0
+0
-1
+1
+0
+0
+0
-4.5
-Pro ni-3
-5.2 GPT
35B 3-2
30B
-4.5 pus
-Pro
-5.2 GPT
35B
30B
-4.5 pus
-Pro
-5.2 GPT
35B
30B
us -Op
+1
i Gem
Qw
en
Qw
en3
-O
ude
Cla
ini-
Gem
3
-2
en3 Qw
-
en3 Qw
-O
ude
Cla
ini-
Gem
3
-2
en3 Qw
-
en3 Qw
Figure 7: Comparison of model rankings between agent-based automatic evaluation and human evaluation across three tasks. In most cases, the rank difference is zero or at most one, indicating strong agreement between the automatic evaluator and human annotators.
Generation
Editing
Repair
80
Overall Score (%)
70 60 50 40 30 20 10 0
React
Vue
Vanilla Claude-4.5-Opus
React
Vue
Gemini-3-Pro-Preview
Vanilla GPT-5.2
React
Vue
Vanilla
Qwen3-VL-235B-Instruct
Figure 8: Overall scores across front-end frameworks for four representative models on Generation, Editing, and Repair tasks. Scores are computed as the harmonic mean of the three per-task evaluation dimensions. Three key findings emerge. (1) Vanilla dominates Generation and Editing, but not Repair. Across all four models, framework-free code consistently yields the highest scores in Generation and Editing. In Repair, however, the Vanilla advantage diminishes: for instance, GPT-5.2 achieves its best Repair score with React. We attribute this to a structural difference between task types: Generation and Editing require producing substantial new code, where Vanilla’s absence of build toolchains, framework-specific syntax (e.g., JSX, template directives), and component lifecycle conventions reduces the surface area for errors. Repair, by contrast, demands precise localization and modification of existing defects, where React’s explicit component boundaries and unidirectional data flow may help models isolate faulty code regions more effectively than unstructured Vanilla codebases. (2) Vue consistently underperforms. Vue yields the lowest scores in the majority of model–task combinations. A plausible contributing factor lies in Vue’s single-file component (SFC) format, which interleaves three heterogeneous syntax modes—HTMLlike templates with custom directives (v-if, v-for, @click), JavaScript/TypeScript logic, and scoped CSS—within a single file. This demands simultaneous coordination across markup, logic, and styling, increasing the likelihood of cross-block inconsistencies. By comparison, React’s JSX keeps rendering logic within standard JavaScript, and Vanilla avoids framework abstractions entirely. (3) Open-source models share the same framework sensitivity pattern (peaking on Vanilla for Generation/Editing) but at a uniformly lower performance ceiling, suggesting that the observed framework preferences are primarily driven by inherent task–framework interactions rather than model-specific factors. 4.3.3
Task-Type Breakdown
To reveal where strong models succeed and fail, we further decompose Edit and Repair into fine-grained subtask categories for the three best closed-source models. Figures 9 and 10 report the harmonic-mean score for each subtask type. Editing: animation-heavy operations are the hardest. A clear difficulty hierarchy emerges across editing operation types (Figure 9): Business Scenario tasks such as Shopping Cart and Multi-step Wizard are consistently the easiest, followed by Real-time & Async tasks, then Interactive Components, with
13
GPT-5.2
100
Claude-Opus-4.5
Gemini-3-Pro-Preview
Overall Score (%)
90 80 70 60 50 40
r s s e g d n n d ll rt w or ce ns ng ect nte res tio abl Ca zar oar cro llin dit tio rfa atio Vie adi Ce Eff rog Wi tica tE lid cro nsi ta T shb nte ee ing Lo te S n P n a e x I a r a i p S p l r o e e n e V T n i h c p p Da D i x o st ti to e ro eT uth hT cat rm wit Inf Sh alla ltiele Par Fo tifi Ric Pag ad -tim &D er A Sk Par Mu nc No eal plo Us ag y r R s U D A e Fil
Figure 9: Overall score breakdown for editing tasks across 16 operation types. Scores are computed as the harmonic mean of Instruction Targeting, Feature Integrity, and Style Conformance per subtask, averaged over all instances containing that operation type. GPT-5.2
100
Claude-Opus-4.5
Gemini-3-Pro-Preview
Overall Score (%)
90 80 70 60 50 40
on
usi
cl Oc
ap erl
g
din
w Cro
Te
v xt O
ent
gnm Ali
lo
Co
ast
ntr
o rC
w
n
rtio
flo
er Ov
Siz
ing
po Pro
f In
ss o
Lo
ac ter
ty
i tiv
Sem
ic ant
or
Err
Ne
g stin
or
e but
Err
ng
ssi
Mi
ri Att
s
Figure 10: Overall score breakdown for repair tasks across 11 defect categories. Scores are computed identically to Figure 9. Advanced Animation tasks such as Parallax Scrolling, Page Transitions, and Particle Effects forming the hardest category. This ordering is stable across all three models, suggesting that editing difficulty scales with the degree of visual dynamism and cross-component coordination required. Repair: semantic defects remain the main bottleneck. A similar difficulty gradient appears in repair (Figure 10). Structural and interactive defects such as Loss of Interactivity, Nesting Error, and Text Overlap are reliably fixed, as they often manifest in localized DOM structures or event handlers. Semantic-level defects, however, prove substantially harder: Semantic Error elicits the lowest scores across all models, followed by Crowding and Missing Attributes. These categories require reasoning about design intent and implicit visual constraints that go beyond pattern-matching on code structure. Consistency matters more than isolated wins. An instructive discrepancy emerges for editing: GPT-5.2 outperforms Gemini-3-Pro-Preview on 13 of 16 subtask types when averaged per category, yet trails on instance-level scores in Table 3. This reversal stems from our harmonic-mean aggregation—GPT-5.2 exhibits higher cross-subtask variance, and the harmonic mean penalizes low outliers steeply. This highlights that multi-requirement evaluation rewards consistency, not just peak subtask performance. No such reversal occurs for repair, where Gemini-3-Pro-Preview leads on all 11 defect categories, reflecting genuinely superior repair capability. 4.3.4
Difficulty-Level Analysis
Each WebCompass instance is annotated as Easy, Medium, or Hard according to required functionality, number of interactive components, and visual sophistication. This stratification lets us examine how model quality degrades with task complexity.
14
To further investigate how model capabilities scale with task complexity, we break down the evaluation results across three difficulty levels (Easy, Medium, Hard) for each task category. Figure 11 presents an overview across all three task families. Across all task families and evaluation dimensions, model scores consistently decrease as difficulty increases (Figures 12, 13, and 14). This degradation is particularly striking in generation on the Spec Implementation dimension (Figure 12), where Hard tasks require implementing more complex user flows, multi-step state transitions, and richer dynamic behavior. For example, Gemini-3-Pro-Preview drops from 89.83 on Easy generation tasks to 37.64 on Hard ones, suggesting that faithfully implementing the full functional spec becomes disproportionately challenging as task complexity grows. Cross-task observations. As shown in Figure 11, Qwen3-VL-235B-A22B-Instruct consistently ranks last across all task–difficulty settings, with its weakest performance appearing most clearly on Editing. In contrast, GPT-5.2, Claude-4.5 Opus, and Gemini-2.5 Pro remain substantially stronger across the board, although their relative advantages vary by task. Overall, performance drops as difficulty increases for all models, while the relative ranking is largely preserved, suggesting that harder front-end tasks degrade performance broadly rather than uniformly widening the gap between models.