UIBenchKit: A unified toolkit for design-to-code model evaluation Chinh T. Le
Trevor Ong Yee Siang
arXiv:2605.13141v1 [cs.SE] 13 May 2026
Singapore Management University Singapore [email protected]
Yuxuan Wan
Yintong Huo∗
The Chinese University of Hong Kong China [email protected]
Singapore Management University Singapore [email protected]
Abstract Recent years have seen substantial progress in automated designto-code generation, with many methods proposed for generating HTML and CSS from webpage screenshots. However, the absence of a standardized evaluation platform makes it difficult to compare these methods fairly, limiting both practical adoption and systematic research progress. To bridge this gap, we introduce UIBenchKit, an open-source, integrated toolkit designed to unify the evaluation of design-to-code tasks. UIBenchKit abstracts the complexities of environment setup, model inference, and code rendering, offering researchers a plug-and-play architecture to compare various methods under consistent settings. In addition, it offers an analytical interface for comparison across multiple metrics. Using UIBenchKit, we conduct a benchmarking study of existing tools and derive several findings that highlight directions for future improvement. By providing a streamlined environment for both experimentation and evaluation, UIBenchKit aims to accelerate future benchmarking and innovations in web engineering. The evaluation platform and toolkit are available at the project page https://www.uibenchkit.com/.
Keywords Multi-modal Large Language Model, Code Generation, User Interface, Web Development, Toolkit, Benchmark ACM Reference Format: Chinh T. Le, Trevor Ong Yee Siang, Jingyu Xiao, Yuxuan Wan, and Yintong Huo. 2026. UIBenchKit: A unified toolkit for design-to-code model evaluation. In . ACM, New York, NY, USA, 5 pages. https://doi.org/10.1145/ nnnnnnn.nnnnnnn
1
Jingyu Xiao
The Chinese University of Hong Kong Singapore Management University China Singapore [email protected] [email protected]
Introduction
The translation of visual user interface (UI) mockups into functional front-end implementations (referred to as UI2Code or designto-code), sits at the intersection of design and web development. This step remains challenging and costly, as it demands visual fidelity and correct implementation while coordinating designers and developers. Recently, Multimodal Large Language Models (MLLMs) have demonstrated remarkable capabilities in directly generating structured HTML and CSS from visual inputs, catalyzing a wave of ∗ Yintong Huo is the corresponding author
both research and commercial interest in this area. In the commercial space, dedicated UI2Code products including Vercel’s v0, Lovable, Bolt.new, and Figma, have collectively attracted millions of active users [4]. On the research front, a growing body of work has emerged on UI2Code, spanning methods that improve generation quality [2, 3, 10, 11], benchmarks that evaluate model performance [8], and recent efforts that extend UI2Code generation from static UI mockups to interactive webpages [12]. Despite this rapid progress, there is a lack of a standardized evaluation toolkit for UI2Code research. Individual evaluations are tightly coupled to specific LLM backbones, configurations, and prompts [8, 10, 11]. Moreover, there is also no unified evaluation metric. Without a standardized pipeline, it is unclear whether reported improvements reflect stronger methods or simply differences in prompts or evaluation protocols. As a result, this makes it difficult for practitioners and researchers to choose and compare tools. We believe that a reliable evaluation platform for UI2Code must satisfy several key requirements. First, it must be modularized to isolate the effects of different factors, thereby enabling controlled and reproducible experiments. Second, it has to be extensible to incorporate new benchmarks and methods. Third, given the complexity of UI2Code evaluation, which spans visual, structural, and textual evaluation metrics [8, 11], the platform should provide a data analytical interface to help practitioners compare tools. To address these needs, we build UIBenchKit, a unified and extensible toolkit for developing, reproducing, and benchmarking design-to-code methods. UIBenchKit abstracts the end-to-end workflow into modular components. It encompasses integration with various MLLMs, method execution, and evaluation. This allows users to test and compare multiple methodologies through a simple interface. We envision that UIBenchKit enables more transparent and reproducible evaluation of UI2Code frameworks in future research. In summary, UIBenchKit makes three main contributions. (1) A unified execution framework for UI2Code research. UIBenchKit provides a standardized pipeline for evaluating design-to-code tools across datasets, backbone LLMs, and methodologies. It handles dataset preparation, method execution, and output generation through a unified interface. (2) An analytical interface for multi-dimensional comparison. UIBenchKit provides a reporting and visualization interface that helps users inspect generated outputs and compare methods across multiple dimensions.
Conference’17, July 2017, Washington, DC, USA
Chinh T. Le, Trevor Ong Yee Siang, Jingyu Xiao, Yuxuan Wan, and Yintong Huo
Input
Execution Pipelines Pre-Processing Experiment Configuration
Playwright Renderer
Method Dispatcher
Dataset Manager
Evaluation Suite Run results
Run Orchestrator
Custom Input
Generated Webpage code
Design-To-Code Approaches LLM Model Generation Method
Hugging Face Datasets
Direct Prompting
Output
Post-Processing and Evaluation
Blockbased
Divide and Conquer
Layout Aware Generation
Hierarchical Code Generation
Code-Level Metrics
Model Evaluation
Screenshot generated Webpage
FineGrained Metrics
Artefacts and metadata
Computation Cost
Visual Metrics
Figure 1: UIBenchKit System Architecture Design (3) A large-scale benchmarking study. Using UIBenchKit, we benchmark 16 models, 5 methodologies, and 2 datasets, which total to 832 instances under a unified setting, yielding a comprehensive view of current performance in terms of visual fidelity, structural accuracy, and computational overhead.
2 The Design of UIBenchKit 2.1 System Architecture Figure 1 illustrates UIBenchKit’s end-to-end evaluation system. The evaluation begins with the Dataset Manager, which prepares benchmark datasets or custom screenshot folders into a shared input format. This ensures that all methods operate on the same input structure and preprocessing conventions. After the inputs are normalized, the Run Orchestrator records the experiment configuration, tracks the progress for each instance, and stores the generation outputs and metadata. This provides reproducibility and supports recovery for long-running or failure-prone benchmark runs. During execution, the Method Dispatcher route prepared inputs to various UI2Code models. Although the methods differ in decomposition, generation, and assembly strategies, UIBenchKit exposes them through a common execution interface. For each LLM call made by a selected method, UIBenchKit abstracts provider-specific inference details such as input formatting, API calls, backend routing, and token accounting. This allows a method to be easily evaluated across different MLLM families. Once the method produces HTML, the Post-processing and Evaluation layer renders the generated code into screenshots using Playwright and applies an extensible set of evaluation metrics.
allows users to upload screenshots or benchmark-style folders, select models and methods, inspect generated outputs, and review evaluation results.
2.3
UIBenchKit supports reproducible experiments through a shared CLI and REST API workflow. Users specify an input source, target model, and generation method, then submit the run through a unified interface. During execution, UIBenchKit exposes status polling and stores intermediate artifacts, allowing users to monitor progress and recover incomplete runs. Upon completion, it returns a consolidated report containing configurations, generated HTML, rendered screenshots, token statistics, and evaluation scores.
2.4
Tool Implementation
Our implementation of UIBenchKit consists of two main modules: • Backend API: The core execution engine is implemented in Python using Flask [7]. It serves as the entry point for run submission, status tracking, and artifact access. • Web Frontend: For interactive use, UIBenchKit provides a browserbased frontend implemented as a Single Page Application using React [5], TypeScript [6], and Tailwind CSS [9]. This interface
Data Analytical Interface
UIBenchKit provides an analytical reporting module for interpreting evaluation results across models and methods. Upon run completion, the toolkit aggregates data into a unified report supporting multi-dimensional comparisons across models and methods. Users can compare model-method combinations across visual similarity, structural accuracy, fine-grained UI matching, and computational overhead. Crucially, the interface links quantitative metrics directly to underlying artifacts, such as input screenshots, generated HTML, and rendered webpages. This traceability allows users to determine not just which method excels, but why specific failures occur. The interface module complements the execution framework and delivers actionable insights for researchers and practitioners.
3 2.2
Reproducing Benchmark Runs
User Demo of UIBenchKit
To complement the CLI and API workflows for large-scale experiments, UIBenchKit provides a lightweight web interface tailored for quick exploratory analysis and qualitative inspection. This GUI facilitates an interactive demo on targeted UI examples through following workflow: (1) Upload UI examples. The user uploads one or more target UI screenshots. (2) Select comparison settings. The user selects specific methods and target LLMs for comparison.
UIBenchKit: A unified toolkit for design-to-code model evaluation
EXAMPLE EVALUATION OUTPUT
UPLOAD IMAGE AND REFERENCE CODE
VIEW RESULTS CONFIGURATION
Conference’17, July 2017, Washington, DC, USA
• LaTCoder [3], a Layout-as-Thought method that decomposes a UI into layout-aware blocks, generates code for each block using CoT-based prompts, and assembles the final page with dynamic selection; • LayoutCoder [11], a layout-guided method that builds a UI layout tree from element relations and recursive block projection, then fuses MLLM-generated atomic-region code snippets into the final webpage; • UICopilot [2], a hierarchical method that predicts a coarse DOM tree with bounding boxes, generates HTML/CSS for leaf-node regions, and refines global styles while preserving the structure.
Figure 2: UIBenchKit Graphical User Interface
4.2 (3) Run generation. UIBenchKit processes the selected combinations through the same backend generation and rendering pipelines as the CLI and API. (4) Inspect results. The interface presents the input screenshot alongside the generated HTML, rendered output, and evaluation metrics. This enables a rapid case-to-case comparison.
4
Evaluation
To demonstrate the practicality of UIBenchKit, we utilized the framework to conduct a large-scale empirical study across diverse models and methodologies. This study includes 16 leading LLMs and 5 open-source methodologies on two popular datasets. The evaluation was orchestrated using the tool’s API entry point, ensuring standardized preprocessing and identical rendering environments across all test runs.
4.1
Experimental Setup
(1) Datasets: We evaluated the generated frontend code on two widely recognized benchmarks: Design2Code [8] (484 real-world webpage mockups) and DCGen [10] (348 distinct webpage designs). (2) Models: Rather than limiting our evaluation to a single architecture, we benchmark 16 distinct Multimodal Large Language Models. These span state-of-the-art proprietary models (e.g., GPT, Claude, and Gemini families) and open-source ones (e.g., Qwen and LLaMA families). (3) Methods: We evaluate five representative UI2Code methodologies supported by UIBenchKit. We reproduce the open-source methodologies using their original implementations. • Direct prompting We evaluate the basic design-to-code capability of each LLM by directly prompting them. The unified prompt is: "Here is a prototype image of a webpage. Return a single piece of HTML and Tailwind CSS code to reproduce exactly the website. Use "placeholder.png" to replace the images. Pay attention to things like size, text, position, and color of all the elements, as well as the overall layout. Respond with the content of the HTML+Tailwind CSS code." ; • DCGen [10], a divide-and-conquer approach that segments the input UI into regions, generates region-level code, assembles them into a complete layout, and refines the result through iterative rendering and candidate selection;
Evaluation Metrics
For our experimental setup, we utilize screenshots of live webpages to simulate the input design mockups 𝐼𝑜 , relying on their underlying HTML and CSS as the ground-truth code 𝐶𝑜 . We measure generation performance across three primary dimensions: high-level similarity, fine-grained element matching, and resource efficiency. • High-Level Similarity: We measure the overall closeness of the outputs through two metrics: Structural Code Similarity, utilizing Normalized Levenshtein Distance (S = 1 - distance/(𝑙 1 + 𝑙 2 )) [1] between 𝐶𝑔 and 𝐶 0 , and Visual Image Similarity via CLIP Score [8], employing CLIP-ViT-B/32 embeddings to score the resemblance of 𝐼𝑔 to 𝐼𝑜 . • Fine-Grained Element Matching: To capture detailed performance nuances missed by macro-level evaluations, we adopt the element-matching suite from Si et al [8]. This approach detects and aligns visual blocks between 𝐼 0 and 𝐼𝑔 , assessing accuracy across four specific dimensions: Block-match (ratio of matched to total block sizes), Text similarity (Sørensen-Dice coefficient), Color similarity (CIEDE2000 formula), and Position similarity (block center alignment). • Resource Consumption: Because inference cost has emerged as a primary bottleneck limiting the practical scalability and commercial adoption of LLMs [13], we systematically track and evaluate the token consumption for each method. Specifically, we calculate the average number of text and vision tokens consumed by the respective MLLMs per generation, providing a transparent benchmark for computational overhead.
4.3
Evaluation Results
Due to space constraints, the full evaluation results are available on the project website at https://www.uibenchkit.com/. Figure 3 illustrates the leaderboard view used to compare benchmark results across models and methods. Our experimental findings show that decomposition-based UI-to-code frameworks can improve layout fidelity on more complex webpages, but their gains are not uniform across all evaluation metrics. For example, LaTCoder performs strongly on structural metrics, particularly in foreground block matching and positioning, whereas direct prompting remains competitive on simpler cases. We also observe that global similarity metrics, such as CLIP, can obscure layout errors, so fine-grainedlevel metrics provide a clearer picture of practical UI fidelity.
Conference’17, July 2017, Washington, DC, USA
Chinh T. Le, Trevor Ong Yee Siang, Jingyu Xiao, Yuxuan Wan, and Yintong Huo
relationships between UI elements. UICopilot [2] not only produces image segments but also uses a Transformer-based decoder to infer the DOM tree of a webpage. We believe that this is a direction that has potential for exploration. We hope that UIBenchKit will continue to provide key insights that will inspire subsequent research efforts in UI2Code methods in the same way we gained insights through our experiments.
6
Figure 3: Evaluation result available on the project website
5
Lesson Learnt
While running our standardized evaluation pipeline, we observed that many UI2Code methods, despite their distinct designs, share a common segmentation-generation pattern: they first decompose screenshots into visual or structural units before generating and assembling code. To better understand recurring failures, we analyzed cases where the generated webpage was incomplete or invalid, leading to two key insights. • Optimization: While decomposition-based pipelines improve scalability, they shift the problem from generation complexity to intermediate representation accuracy, making overall performance highly sensitive to early-stage errors. Its benefits include a reduced context length for LLM-based code generation, which simplifies LLM reasoning of local UI elements. The modularity of UI elements also enables parallel code generation, which reduces generation time. However, our experiments also showed that this optimization introduces a systemic trade-off. Errors in early stages, such as inaccurately identifying bounding boxes are amplified downstream. The framework either fails when reaching the threshold number of bounding boxes allowed or it wastefully generates redundant code chunks. • Image segmentation: Image segmentation is the primary source of failure, as current methods rely heavily on low-level visual cues that do not reliably capture the semantic structure of modern UIs. We have come to realize that there is no "one size fits all" image segmentation algorithm. Approaches like the one proposed in LayoutCoder [11] assumes that elements are xy-axis aligned. This means that the algorithm struggles to segment angled elements accurately. LatCoder [3] on the other hand relies heavily on identifying strong visual boundaries, leading to decorative lines, borders, and even text at times being over-segmented. The issues identified here stems from the fact that images do not encode the visual hierarchy that is inherently obvious to us humans. Our research has found that LLM code generation capabilities have improved tremendously, aided by frameworks which segments images into smaller visual blocks. However, the most significant hurdle in UI2Code generation remains the limitations of existing image segmentation techniques. Future image segmentation systems should attempt to incorporate structure-aware representations of the image segments. In fact, UICopilot’s [2] assembly stage makes a good attempt at incorporating both structural references and semantic
CONCLUSION
In this paper, we introduce UIBenchKit, an extensible and unified framework designed to address the current fragmentation in automated design-to-code research. We demonstrated the platform’s robustness by seamlessly integrating 5 distinct generation methodologies and 16 diverse Multimodal Large Language Models (MLLMs), culminating in a comprehensive empirical benchmark. By providing both a RESTful orchestration API and a high-throughput CLI, UIBenchKit significantly lowers the barrier to entry for reproducing state-of-the-art results and conducting fair, standardized evaluations. In our future works, we will focus on expanding support for dynamic web components and integrating modern frontend frameworks like React to further bridge the gap between multimodal AI research and practical software engineering workflows.
References [1] Max Bachmann. 2026. Levenshtein Distance — RapidFuzz Documentation. https: //rapidfuzz.github.io/Levenshtein/levenshtein.html Accessed: 2026-02-26. [2] Yi Gui, Zhen Li, Zhongyi Zhang, Yao Wan, Dongping Chen, Hongyu Zhang, Yi Su, Bohua Chen, Xing Zhou, Wenbin Jiang, and Xiangliang Zhang. 2025. UICopilot: Automating UI Synthesis via Hierarchical Code Generation from Webpage Designs. arXiv:2505.09904 [cs.SE] https://arxiv.org/abs/2505.09904 [3] Yi Gui, Zhen Li, Zhongyi Zhang, Guohao Wang, Tianpeng Lv, Gaoyang Jiang, Yi Liu, Dongping Chen, Yao Wan, Hongyu Zhang, Wenbin Jiang, Xuanhua Shi, and Hai Jin. 2025. <scp>LaTCoder:</scp> Converting Webpage Design to Code with Layout-as-Thought. In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2 (KDD ’25). ACM, 721–732. doi:10.1145/ 3711896.3737016 [4] Jason Lemkin. 2025. How Vercel Hit $9.3B and Replit Hit $3B. . . After a Decade: The Long Paths to AI Overnight Success. https://www.saastr.com/how-vercelhit-9-3b-and-replit-hit-3b-after-a-decade-the-long-paths-to-ai-overnightsuccess/. SaaStr. Reports v0 reaching 3.5 million users at the time of Vercel’s Series F in September 2025. Accessed: 2026-04-28. [5] Meta Open Source. [n. d.]. React Documentation. https://react.dev/. Accessed: 2026-05-10. [6] Microsoft. [n. d.]. TypeScript Documentation. https://www.typescriptlang.org/ docs/. Accessed: 2026-05-10. [7] Pallets Projects. [n. d.]. Flask Documentation. https://flask.palletsprojects.com/. Accessed: 2026-05-10. [8] Chenglei Si, Yanzhe Zhang, Ryan Li, Zhengyuan Yang, Ruibo Liu, and Diyi Yang. 2025. Design2Code: Benchmarking Multimodal Code Generation for Automated Front-End Engineering. arXiv:2403.03163 [cs.CL] https://arxiv.org/abs/2403. 03163 [9] Tailwind Labs. [n. d.]. Tailwind CSS Documentation. https://tailwindcss.com/docs. Accessed: 2026-05-10. [10] Yuxuan Wan, Chaozheng Wang, Yi Dong, Wenxuan Wang, Shuqing Li, Yintong Huo, and Michael Lyu. 2025. Divide-and-Conquer: Generating UI Code from Screenshots. Proceedings of the ACM on Software Engineering 2, FSE (June 2025), 2099–2122. doi:10.1145/3729364 [11] Fan Wu, Cuiyun Gao, Shuqing Li, Xin-Cheng Wen, and Qing Liao. 2025. MLLMBased UI2Code Automation Guided by UI Layout Information. Proceedings of the ACM on Software Engineering 2, ISSTA (June 2025), 1123–1145. doi:10.1145/ 3728925 [12] Jingyu Xiao, Yuxuan Wan, Yintong Huo, Zixin Wang, Xinyi Xu, Wenxuan Wang, Zhiyao Xu, Yuhang Wang, and Michael R Lyu. 2025. Interaction2code: Benchmarking mllm-based interactive webpage code generation from interactive prototyping. In 2025 40th IEEE/ACM International Conference on Automated Software Engineering (ASE). IEEE, 241–253. [13] Boqin Zhuang, Jiacheng Qiao, Mingqian Liu, Mingxing Yu, Ping Hong, Rui Li, Xiaoxia Song, Xiangjun Xu, Xu Chen, Yaoyao Ma, and Yujie Gao. 2025. Beyond
UIBenchKit: A unified toolkit for design-to-code model evaluation
Benchmarks: The Economics of AI Inference. arXiv:2510.26136 [cs.AI] https:
Conference’17, July 2017, Washington, DC, USA
//arxiv.org/abs/2510.26136