AnimWeb-Bench: Benchmarking Animated Webpage Code Generation from Videos

arXiv:2611.09769v1 [cs.SE] 6 Nov 2026

AnimWeb-Bench: Benchmarking Animated Webpage Code Generation from Videos Zhongyi Zhang

Jingyu Xiao∗

The Chinese University of Hong Kong Hong Kong, China zyzhang01@link.cuhk.edu.hk

The Chinese University of Hong Kong Hong Kong, China jyxiao@link.cuhk.edu.hk

Shuqing Li

Michael R. Lyu

The Chinese University of Hong Kong Hong Kong, China sqli21@cse.cuhk.edu.hk

The Chinese University of Hong Kong Hong Kong, China lyu@cse.cuhk.edu.hk

Abstract

Keywords

Animation is pervasive on the Web, the CSS animation property alone appears on over three-quarters of webpages. Yet existing webpage code generation benchmarks target static screenshots, user-triggered interactions, or isolated motion graphics, leaving the ability of multimodal large language models (MLLMs) and coding agents to reconstruct complete animated webpages largely unexplored. To fill this gap, we introduce AnimWeb-Bench, the first benchmark for animated webpage code generation from videos. It comprises 124 real-world webpages spanning 7 application domains and 33 animation types, with 91.9% of pages combining multiple types, each paired with a reference video and the static assets of the original webpage. We further propose a human-aligned, two-dimensional evaluation protocol: Static Appearance Fidelity measures structure, text, color, and visual similarity on aligned keyframes, while Animation Fidelity tracks UI components across frames and compares their position, size, velocity, and appearance with those in the reference video. Evaluating 14 MLLMs and three coding agents reveals: (1) reproducing webpage animations is substantially more difficult than reproducing static appearance; (2) explicit frame inspection and render feedback improve the reconstruction quality of coding agents; and (3) rapid, continuous changes are harder to reproduce than discrete state transitions. Our code and dataset are available at starry1001.github.io/AnimWeb-Benchwebsite.

Multimodal Large Language Models, Code Generation, Web Development, Web Animation

CCS Concepts • Software and its engineering → Automatic programming; • Computing methodologies → Artificial intelligence. ∗ Jingyu Xiao is the corresponding author.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org. Conference acronym ’XX, Woodstock, NY © 2018 Copyright held by the owner/author(s). Publication rights licensed to ACM. ACM ISBN 978-1-4503-XXXX-X/2018/06 https://doi.org/XXXXXXX.XXXXXXX

ACM Reference Format: Zhongyi Zhang, Jingyu Xiao, Shuqing Li, and Michael R. Lyu. 2018. AnimWebBench: Benchmarking Animated Webpage Code Generation from Videos. In Proceedings of Make sure to enter the correct conference title from your rights confirmation email (Conference acronym ’XX). ACM, New York, NY, USA, 13 pages. https://doi.org/XXXXXXX.XXXXXXX

1

Introduction

Motion is part of the Web’s basic visual vocabulary. The CSS animation and transition properties appear on 77% and 85% of the nearly 8 million mobile pages crawled by the HTTP Archive [20], and about half of all pages ship a prefers-reduced-motion query to adapt that motion to users’ needs [21]. On these pages, animation is functional rather than decorative: it signals system feedback, preserves continuity across state changes, and directs attention [9, 19]. At the same time, front-end code is increasingly written by Multimodal Large Language Models and coding agents: 84% of developers use or plan to use AI tools in their development process [44, 58], and generating a webpage from a visual reference has become a prominent use case targeted by a growing line of benchmarks [16, 42]. Several benchmarks have been proposed to evaluate MLLMs on front-end code generation, but most of them focus on static webpage reconstruction from screenshots [16, 27, 42, 56]. A few studies extend the evaluation to user-triggered interactions. Interaction2Code [51] represents each interaction with screenshots captured before and after a user action, while IWR-Bench [10] and WebVR [11] use demonstration videos to evaluate the reconstruction of interactive webpages. However, these benchmarks primarily target discrete state changes triggered by user actions rather than animations that unfold continuously over time. This raises a question that existing evaluations leave open: can AI systems reconstruct not only the visual states of a webpage, but also the continuous dynamics connecting them? Answering this question requires moving beyond static appearance and discrete interactive functionality to evaluate the continuous temporal behavior of visual elements. This poses additional challenges: a model must infer motion trajectories, timing, easing, and synchronization from the reference video and express them in executable code. Animation2Code [23] took a first step answering

Conference acronym ’XX, June 03–05, 2018, Woodstock, NY

Zhang et al.

Input

Reference Video

Assets

Generated Code

MLLMs & Coding Agents Animation Fidelity

Static Appearance Fidelity

Position Size Velocity

Structure

AnimWeb -Bench

Text Color

CLIP

CLIP

<!DOCTYPE html> <html> <style> .hero { animation: move 1s ease-out; } @keyframes move { from { transform: translateY(24px); } to { transform: translateY(0); } } </style> <main class="hero">...</main> <script> document.querySelector(".hero").animate( { opacity: [0, 1] }, { duration: 600, delay: 300, fill: "both" } ); </script> </html>

render Generated Video

Output

Figure 1: Overview of the AnimWeb-Bench task and evaluation. Given a reference video and the static assets of a real webpage (top), an MLLM or coding agent generates a single HTML file, which is rendered and recorded in a browser (bottom). Static Appearance Fidelity compares aligned keyframes. Animation Fidelity compares the trajectories of matched UI components. it, reconstructing web animation code from videos, but on isolated motion graphics scored within a cropped region. Real webpages are harder. Many components move in sequence with distinct timing and easing, while coexisting with the page’s layout, text, and media. Reconstruction requires inferring which components move, when, and how fast. Evaluation requires judging the motion of each component, which frame-level similarity cannot see, region-pooled tracks cannot attribute, and binary rubrics cannot grade. To bridge this gap, we introduce AnimWeb-Bench, to our knowledge the first benchmark for reconstructing complete real-world webpages together with their time-driven animations from videos. AnimWeb-Bench comprises 124 real-world webpages spanning 7 application domains and 33 animation types. As a comparison, an AnimWeb-Bench page contains 1,196 HTML tags on average, versus 71 for an Animation2Code sample. As illustrated in Figure 1, each task pairs a reference video with the original webpage’s static assets. To evaluate the generated webpages, we design a human-aligned protocol with two complementary dimensions: Static Appearance Fidelity, which compares structure, text, color, and visual similarity on aligned keyframes, and Animation Fidelity, which tracks UI elements across frames and compares their position, size, velocity, and appearance with the reference. All eight metrics correlate with 400 blind pairwise human judgments (Spearman’s 𝜌 = 0.67–0.74). We evaluate 14 MLLMs and 3 coding agents on AnimWeb-Bench. Beyond overall performance, we investigate whether coding agents improve over their underlying models and what factors affect their

performance, and analyze how different animation types affect reconstruction. Our main contributions are as follows: • We introduce AnimWeb-Bench, a benchmark for animated webpage code generation from videos. It comprises 124 real-world webpages spanning 7 application domains and 33 animation types, each paired with a reference video and the original webpage’s static assets. • We design a human-aligned evaluation protocol that separately measures how a generated page looks and how it moves. • Our comprehensive evaluation of 14 MLLMs and 3 coding agents reveals three key findings: (1) reproducing webpage animations remains challenging for current systems; (2) explicit frame inspection and render feedback improve the reconstruction quality of coding agents; (3) rapid, continuous changes are harder to reproduce than discrete state transitions.

2 Background 2.1 Task Definition The Animated Webpage Code Generation task aims to reconstruct executable webpage code by observing the animation effects shown in a reference video. Given a reference video 𝑉 𝑟 = {𝐼 1, 𝐼 2, . . . , 𝐼𝑛 } and a set of static assets (e.g., icons, images and embedded videos): 𝐴 = {𝑎 1, 𝑎 2, . . . , 𝑎𝑚 }, a model 𝑀 generates a single executable HTML file 𝐶 𝑔 = 𝑀 (𝑉 𝑟 , 𝐴), with CSS and JavaScript embedded inline. Rendering and recording 𝐶 𝑔 in a browser produces a video 𝑉 𝑔 = 𝑅(𝐶 𝑔 , 𝐴) (Figure 1). The generated video 𝑉 𝑔 should closely approximate 𝑉 𝑟 in both visual appearance and animation dynamics.

AnimWeb-Bench: Benchmarking Animated Webpage Code Generation from Videos

Table 1: Overview of existing UI-to-code benchmarks. Benchmark Pix2Code [7] Web2Code [56] WebSight [27] Design2Code [42] WebCode2M [16] CC-HARD [17] ScreenBench [24] Interaction2Code [51] WebVR [11] IWR-Bench [10] Animation2Code [23]

Input

Image Synthesized Image Synthesized Image Synthesized Image C4 Image Common Crawl Image Common Crawl Image Live Websites Image Pairs C4 & GitHub Video Synthesized Video Live Websites Video CodePen

AnimWeb-Bench (Ours) Video

2.2

Data Source

Live Websites

Conference acronym ’XX, June 03–05, 2018, Woodstock, NY

1

Webpage Sourcing & Initial Curation “Animation” tag

Samples Target Domain 1742 Static Layouts 60K Static Layouts 2M Static Layouts 484 Static Layouts 2.56M Static Layouts 128 Static Layouts 1,000 Static Layouts 127 Interactive Webpages 175 Interactive Webpages 113 Interactive Webpages 1,069 Motion Graphics

UI Code Generation Methods. Recent advances in multimodal large language models (MLLMs) have spurred a range of methods for generating webpage code from visual inputs. DCGen [47] adopts a divide-and-conquer strategy, DeclarUI [59] integrates segmentation with page transition graphs, and UICopilot [18], LayoutCoder [48], and LaTCoder [17] enhance generation through hierarchical or layout-aware modeling. Instruct4Edit [12] provides a scalable and transparent foundation for natural language-based web editing. ComUICoder [50] promotes component reuse, while EfficientUICoder [54] reduces redundant visual and code tokens. More recently, Video2Code [55] and InterWebCoder [49] extend generation to interactive webpages, focusing on functional correctness and action-triggered state transitions. TTDev [46] applies test-driven development method for full-stack website generation and VisualRepair [53] proposes dynamic tool calling and region focusing for front-end bug repair. Our work complements these efforts by evaluating the reconstruction of continuous webpage animations from videos. UI Code Generation Benchmarks. Early benchmarks focus on static webpage reconstruction from screenshots. WebSight [27] and Web2Code [56] study large-scale HTML generation using synthetic webpages, while Design2Code [42] and WebCode2M [16] introduce real-world data at increasing scales. MRWeb [45] extends evaluation to multi-page websites with external resources, and DesignBench [52] covers generation, editing, and repair across multiple web frameworks. UIBenchkit [28] provides a unified toolkit for UI2Code methods evaluation. However, screenshot-based benchmarks cannot evaluate temporal behavior. Interaction2Code [51] represents individual interactions with before-and-after screenshots, whereas IWR-Bench [10] and WebVR [11] use demonstration videos to evaluate event-driven interactions and stateful workflows. These benchmarks primarily capture discrete transitions triggered by user actions, while web animations require recovering continuous dynamics such as trajectories, timing, speed, and easing. Animation2Code [23] directly studies video-to-animation-code generation but focuses on isolated motion graphics rather than complete webpages. To our knowledge, AnimWeb-Bench is the first benchmark for reconstructing complete animated webpages from videos. Table 1 summarizes its differences from existing benchmarks.

Video Recording & Asset Collection

Domain-Balanced Crawling

Animation Annotation & Quality Control

Preliminary Annotation

1920 × 1080

30 FPS

473 Webpage URLs

3

Puppeteer

Awwwards

2–15 s H.264 / MP4

Remove Duplicates

Remove Inaccessible

Human Review

Interaction Filtering

Completeness & Quality

Exclude Clicks & Scrolls

Full Animation Cycles & Clean Assets

Domain Balance 7 Domains

Animation Taxonomy 33 Animation Types

Static Assets

452 Candidates

124 Animated Webpages

Related Work

2

124 Tasks

Figure 2: Construction pipeline of AnimWeb-Bench.

3 AnimWeb-Bench 3.1 Dataset Construction Figure 2 shows the construction pipeline of AnimWeb-Bench. Scope. AnimWeb-Bench focuses on time-driven, page-embedded animations that unfold automatically during the initial page load. We consider animations implemented through DOM elements using CSS or JavaScript, where motion is integrated with the webpage’s layout, text, and media. Webpage sourcing. We draw webpages from Awwwards [5], a web-design showcase whose entries are reviewed by a jury of professional designers and developers. Starting from sites carrying the Animation tag, we crawled up to 100 sites per application domain, collecting 473 URLs of sites submitted between 2025 and 2026. Removing duplicate and inaccessible sites left 452 candidates. Recording and asset collection. We load each page in headless Chromium (v151.0.7922.34) driven by Puppeteer [36] with a 1920 × 1080 viewport and record it at 30 fps (H.264, MP4) from navigation start. Each recording lasts 2–15 s (mean 5.91 s) and covers the page’s initial animation sequence, or one full cycle of a looping animation. We also crawl and deduplicate the static assets needed for reconstruction, i.e., images, icons, and embedded videos (46.1 per page on average). Annotation and quality control. We first use GPT-5.6 Sol to propose a domain and a set of animation-type labels for each recording from the video and the page source. Three PhD students with front-end development experience then review every instance: they correct the labels, supplement missing assets, and verify that the recording captures the complete animation.

3.2

Data Statistics and Diversity

Domain Diversity. As shown in Figure 3, AnimWeb-Bench covers 7 application domains, the largest groups are technology and digital services (25.8%), media and community (22.6%), and design and business services (18.5%). Animation Type Diversity. Based on the preliminary annotations and subsequent manual review, we identify 33 animation types, as detailed in Table 2. The most frequent are translation (64), background transition (50), appearance (49), masking (32), and fade-in (30). In total, the benchmark contains 404 animation-type annotations, 3.3 per webpage on average and between one and eight per webpage. In particular, 91.9% of webpages contain more

Conference acronym ’XX, June 03–05, 2018, Woodstock, NY

Zhang et al.

Table 2: Distribution of animation types in AnimWeb-Bench. Type

Number

Translation Background Transition Appearance Masking Fade In Text Animation Scale Up Color Transition Line Growth

64 50 49 32 30 24 22 14 13

Type

Number

Fade Out Rotation Numeric Counter Animation Scale Down Text Substitution Image Stack Animation Typewriter Animation Vertical Stripe Reveal Appearance & Opacity Changes

12 11 10 8 7 5 5 5 5

Type

Number

Blur Transition Icon State Animation Shake 3D Depth Motion Grid Tile Transition Icon State Transition Text Scramble Twinkling Card Flip

k1: frame 0 0.00 s

Sports 4.0%

Technology & Digital Services 25.8%

Translation

Architecture 6.5%

Education & Healthcare Masking 6.5% Consumer Commerce & Finance Design Services & 16.1%

Media & Community 22.6%

Business Services 18.5%

Figure 3: Domain distribution of AnimWeb-Bench.

than one animation type, reflecting the complexity of combining multiple effects within a single webpage.

3.3

Evaluation Metrics

Our evaluation protocol compares the generated video 𝑉 𝑔 with the reference video 𝑉 𝑟 in two complementary dimensions: static appearance and animation. We first identify keyframes and animation segments in the reference video and align them with the generated video (Section 3.3.1). We then evaluate static appearance on pairs of matched keyframes (Section 3.3.2) and animation on the segments between consecutive matched keyframes (Section 3.3.3). 3.3.1 Keyframes and Animation Segments. A webpage animation is a continuous process, but treating every video frame independently leads to redundancy and obscures the underlying state transitions. To obtain a compact and structured representation, we decompose each video into stable states and the animation phases that connect them. This decomposition separates the visual appearance of each state from the dynamic behavior that transforms one state into the next, which our two evaluation dimensions assess separately. We define a keyframe as a frame where the page has settled into a stable state, and an animation segment as the interval between two consecutive keyframes that contains the animation leading from one state to the next. To locate keyframes, we measure the degree of visual change around each frame. For each frame 𝐼𝑡 of a

0

Type

Number

Font Weight Animation Horizontal Image Slide Vertical Image Slide Logo Morphing Radial Expansion Animation Text Fragmentation

k2: frame 45 1.50 s

20

40

2 2 2 1 1 1

k3: frame 141 4.70 s

60

Frame index 80

100

120

140

k4: frame 179 5.97 s

160

1.0 0.9 0.8

Similarity

AnimWeb -Bench

4 4 4 3 3 3 3 3 2

0.7 0.6 0.5 0.4 0.3 0

1

2

3

4

5

Time (s)

Figure 4: Keyframe extraction on an example from AnimWebBench. The last two detected keyframes have nearly identical content and are therefore merged into 𝑘 4 . video, we compute the similarity between two frames separated by a short temporal window: 1 ∑︁ |𝐼𝑡 +ℎ (𝑝) − 𝐼𝑡 −ℎ (𝑝)| , (1) 𝑆 (𝑡) = 1 − |Ω| 𝑝 ∈Ω 255 where 𝐼𝑡 (𝑝) ∈ [0, 255] denotes the grayscale intensity of pixel 𝑝 in frame 𝑡, Ω is the set of pixels of the downsampled frames, and ℎ is the number of frames spanning 0.2 seconds. The second term is the mean absolute intensity difference between the two frames, normalized to [0, 1], so 𝑆 (𝑡) approaches 1 when the page is unchanged and decreases as the change grows. Comparing frames across a window rather than adjacent frames ensures that slow animations, whose per-frame changes are small, remain visible in the similarity curve. As illustrated in Figure 4, 𝑆 (𝑡) stays close to 1 while the page is stable and drops while an animation is in progress, so each valley of the curve corresponds to an animation phase. Between two consecutive valleys, the curve returns to a plateau, and we take the first frame on this plateau, i.e., the moment at which the page settles, as a keyframe. Together with the first and last frames of the video, which capture the initial and final states, these frames form the keyframe sequence 𝐾 = {𝑘 1, . . . , 𝑘 𝑁 }, in which adjacent keyframes with nearly identical content are merged. For each keyframe 𝐼𝑘𝑟 extracted from the reference video 𝑉 𝑟 , we locate its counterpart in the generated video 𝑉 𝑔 by scoring every 𝑔 generated frame 𝐼𝑡 with 𝑔
𝑔 𝑔 𝑠 (𝐼𝑘𝑟 , 𝐼𝑡 ) = 𝜆 Vis(𝐼𝑘𝑟 , 𝐼𝑡 ) + (1 − 𝜆) 1 − |𝑡ˆ𝑘𝑟 − 𝑡ˆ𝑡 | , (2) where Vis is a low-level visual similarity, 𝑡ˆ is the timestamp normalized by the video duration, and 𝜆 balances appearance against

AnimWeb-Bench: Benchmarking Animated Webpage Code Generation from Videos

temporal position. An order-preserving alignment then selects at most one generated frame for each reference keyframe (Appendix C). This yields matched keyframe pairs and the corresponding animation segments between consecutive pairs. 3.3.2 Static Appearance Fidelity. We compare aligned reference and generated keyframes using Structure, Text, and Color Similarity as fine-grained metrics, and Visual Similarity as a high-level metric. Each metric is averaged over all reference keyframes. Structure Similarity. We detect UI components in the reference and generated keyframes with GPA-GUI-Detector [41], obtaining box sets 𝐵𝑟 and 𝐵𝑔 , and match them one-to-one with the element matching algorithm of ComUICoder [50], yielding the matched pairs M. For each pair (𝑏 𝑟 , 𝑏𝑔 ) ∈ M, we compute 𝑠 (𝑏 𝑟 , 𝑏𝑔 ) = 0.35 𝑠 iou + 0.30 𝑠 center + 0.25 𝑠 area + 0.10 𝑠 aspect,

(3)

where the four terms measure box overlap, center proximity, area ratio, and the ratio of aspect ratios on frame-normalized boxes (Appendix C). The structure similarity is Í 𝑟 𝑔 (𝑏𝑟 ,𝑏𝑔 ) ∈ M 𝑠 (𝑏 , 𝑏 ) 𝑆 str = , (4) |𝐵𝑟 | + |𝐵𝑔 | − |M| where the denominator counts each matched pair once, so missing or redundant components lower the score. Text Similarity. We recognize the text in each keyframe with EasyOCR [22], concatenate it in reading order, and compute the normalized Levenshtein similarity [29] between the two strings. Color Similarity. We treat the components detected in the structure step as foreground and the remaining regions as background. Following prior work [42, 45], we use the CIEDE2000 formula [31] to measure the perceptual color difference of the background and of each matched foreground component. The final score averages the background and foreground results. Visual Similarity. We use the CLIP score [39] to measure the visual similarity between the reference and generated keyframes. 3.3.3 Animation Fidelity. We compare aligned reference and generated segments, each spanning two consecutive keyframes, using Position, Size, and Velocity Similarity as geometric motion metrics and the CLIP score as an appearance metric. Each metric is averaged over all reference segments. Component Trajectories. We apply GPA-GUI-Detector [41] to every frame of a segment and link detections across adjacent frames into component trajectories (Stage 1 of Algorithm 1). A detection 𝑏 is linked to a trajectory 𝜏 according to an association score 𝑎𝜏𝑏 , which combines the IoU, appearance similarity (App), and center proximity (Ctr) between 𝑏 and the latest box of 𝜏, with weights 𝛼 = 0.5, 𝛽 = 0.3, and 𝛾 = 0.2 (Appendix C). For a component 𝑖, let (𝑥𝑖 (𝑡), 𝑦𝑖 (𝑡)) and (𝑤𝑖 (𝑡), ℎ𝑖 (𝑡)) be the center and size of its box at time 𝑡, normalized by the frame width and height. We define its dynamic state as:  ⊤ z𝑖 (𝑡) = 𝑥𝑖 (𝑡), 𝑦𝑖 (𝑡), 𝑤𝑖 (𝑡), ℎ𝑖 (𝑡), 𝑥¤𝑖 (𝑡), 𝑦¤𝑖 (𝑡) ∈ R6, (5) where the velocity v𝑖 = [𝑥¤𝑖 , 𝑦¤𝑖 ] ⊤ is estimated from the positions p𝑖 = [𝑥𝑖 , 𝑦𝑖 ] ⊤ in adjacent frames: p𝑖 (𝑡 + Δ𝑡) − p𝑖 (𝑡) , Δ𝑡 with Δ𝑡 denoting the interval between adjacent frames. v𝑖 (𝑡) =

(6)

Conference acronym ’XX, June 03–05, 2018, Woodstock, NY

Algorithm 1 Trajectory Construction and Matching Input: Reference segment V 𝑟 Input: Generated segment V 𝑔 Input: GPA-GUI-Detector D Input: Thresholds 𝜃 𝑎 , 𝜃 𝑡 Output: Trajectory sets T 𝑟 , T 𝑔 Output: Matched trajectory pairs A Stage 1: Within-video trajectory construction 1: for 𝑚 ∈ {𝑟, 𝑔} do 2: T𝑚 ← ∅ 3: for each frame 𝐼𝑡 ∈ V𝑚 do 4: 𝐵𝑡 ← D (𝐼𝑡 ) 5: for each 𝜏 ∈ T 𝑚 and 𝑏 ∈ 𝐵𝑡 do 6: 𝑎𝜏𝑏 ← 𝛼 · IoU + 𝛽 · App + 𝛾 · Ctr 7: end for 8: Match T 𝑚 and 𝐵𝑡 one to one with 𝑎𝜏𝑏 > 𝜃 𝑎 9: Extend matched 𝜏; add unmatched 𝑏 to T 𝑚 10: end for 11: end for Stage 2: Cross-video trajectory matching 𝑔 12: A ← pairs with Text(𝜏𝑖𝑟 , 𝜏 𝑗 ) > 𝜃 𝑡 𝑔 13: for each unmatched 𝜏𝑖𝑟 ∈ T 𝑟 and 𝜏 𝑗 ∈ T 𝑔 do 14: 𝑑𝑖 𝑗 ← 𝑤 𝑝 (1 − 𝑃𝑖 𝑗 ) + 𝑤𝑠 (1 − 𝑆𝑖 𝑗 ) + 𝑤 𝑣 (1 − 𝑉𝑖 𝑗 ) + 𝑤𝑐 (1 − 𝐶𝑖 𝑗 ) 15: end for 16: A ← A ∪ Hungarian({𝑑𝑖 𝑗 }) 17: return T 𝑟 , T 𝑔 , A

Position, Size, Velocity Similarity, and CLIP Score. For a 𝑔 reference trajectory 𝜏𝑖𝑟 and a generated trajectory 𝜏 𝑗 , we compare their states at corresponding time steps 𝑡𝑘 . For brevity, we omit the indices 𝑖, 𝑗, and 𝑡𝑘 below. The position similarity is based on the normalized center distance: ∥p𝑟 − p𝑔 ∥ 2 𝑃𝑘 = 1 − . (7) √ 2 The size similarity compares the width and height ratios: √︄ min(𝑤 𝑟 , 𝑤 𝑔 ) min(ℎ𝑟 , ℎ𝑔 ) · . (8) 𝑆𝑘 = max(𝑤 𝑟 , 𝑤 𝑔 ) max(ℎ𝑟 , ℎ𝑔 ) Following the temporal correlation of Animation2Code [23], the velocity similarity measures agreement in both motion direction and speed:    1 v𝑟 · v𝑔 min(∥v𝑟 ∥ 2, ∥v𝑔 ∥ 2 ) 𝑉𝑘 = max 0, + . 2 ∥v𝑟 ∥ 2 ∥v𝑔 ∥ 2 max(∥v𝑟 ∥ 2, ∥v𝑔 ∥ 2 )

(9)

We set 𝑉𝑘 = 1 if both components are static and 0 if only one is. The CLIP score 𝐶𝑘 [39] compares the appearance of the two components on the image regions enclosed by their boxes. The pair scores 𝑃𝑖 𝑗 , 𝑆𝑖 𝑗 , 𝑉𝑖 𝑗 , and 𝐶𝑖 𝑗 are obtained by averaging 𝑃𝑘 , 𝑆𝑘 , 𝑉𝑘 , and 𝐶𝑘 over all time steps at which either component is present, where a time step with only one of them present scores zero. Trajectory Matching. We then match the reference and generated trajectories one to one (Stage 2 of Algorithm 1). Trajectories 𝑔 whose OCR text similarity Text(𝜏𝑖𝑟 , 𝜏 𝑗 ) exceeds 𝜃 𝑡 are matched first. Following InTraGen [30], the remaining trajectories are matched

Conference acronym ’XX, June 03–05, 2018, Woodstock, NY

Zhang et al.

by the Hungarian algorithm [26] under a cost 𝑑𝑖 𝑗 that combines their position, size, velocity, and CLIP dissimilarities with weights 𝑤 𝑝 = 0.4, 𝑤𝑠 = 0.2, 𝑤 𝑣 = 0.1, and 𝑤𝑐 = 0.3. 𝑔 Score Aggregation. For segment 𝑙, let T𝑙 𝑟 and T𝑙 be the reference and generated trajectory sets, and let A𝑙 be the matched pairs. For each 𝑄 ∈ {𝑃, 𝑆, 𝑉 , 𝐶}, the segment score is Í 𝑄𝑙 =

(𝑖,𝑗 ) ∈ A𝑙 𝑄 𝑖 𝑗 𝑔

|T𝑙 𝑟 | + |T𝑙 | − |A𝑙 |

,

(10)

So unmatched trajectories lower the score above. The final score averages over all 𝐿 reference segments: 𝐿

𝑄=

1 ∑︁ 𝑄𝑙 . 𝐿

(11)

𝑙=1

4

Experiments

• (RQ1) Overall Performance. How well do state-of-the-art MLLMs reconstruct animated webpages? • (RQ2) Coding Agents. How effectively do coding agents reconstruct animated webpages, and what affects their performance? • (RQ3) Animation Types. How does the performance of current systems vary across different animation types? • (RQ4) Human Evaluation. How do humans assess the reconstructed animated webpages?

4.1

Experimental Setup

Models. We evaluate 14 MLLMs on AnimWeb-Bench, including 10 proprietary and 4 open-weight models, as listed in Table 3. For each task, models are provided with the reference video and a set of composite images containing all processed static assets of the original webpage. Models with native video support (marked with † in Table 3) take the reference video directly. To accommodate models without native video support, we uniformly sample frames from the video at 2 frames per second and input them as an image sequence. The task is to generate a single self-contained HTML file with inline CSS and JavaScript that reproduces both the appearance and the animations of the reference webpage. Each generated page is then rendered and recorded using the same configuration as the reference videos, capturing its own animation sequence in full or one complete cycle if it loops. All other inference parameters follow the default settings recommended by the model providers. Coding Agents. Beyond single-pass generation, we further evaluate 3 coding agents, each powered by one of the MLLMs above: Codex [32] with GPT-5.6 Sol, Claude Code [2] with Claude Opus 5, and Qwen Code [37] with Qwen3.8 Max (0902), all with medium reasoning effort. Each agent operates in a sandboxed workspace that contains the task inputs, where it can execute commands, create and edit files, and inspect the inputs, but cannot access outside this workspace or the network. Unlike MLLMs, which receive inputs directly in their context, agents access the inputs as files and can decide how to process them, for example by extracting frames from the video. To examine how input format affects performance, we evaluate each agent under two settings: Video, which provides the reference video, and Frames, which provides the same 2 fps

frames given to the MLLMs. In both settings, the workspace contains the composite asset images. The agent returns a single HTML file, which is evaluated in the same way as the MLLM outputs. Human Evaluation. While automatic metrics provide a finegrained assessment of model performance, evaluating how users perceive the generated webpages is also essential. Following Design2Code [42], we conduct a pairwise human evaluation. (1) Participants. We recruit five annotators through a local university mailing list. All annotators hold at least a B.S. degree in computer science or software engineering and have more than two years of webdevelopment experience. (2) Procedure. We randomly sample 50 webpages from AnimWeb-Bench and use Qwen3.8 Max (0902) as the baseline. We select eight models and pair the output of each with that of the baseline on every sampled webpage. Throughout the evaluation, annotators follow a guideline [1] that defines the evaluation criteria for static appearance and animation and provides instructions for handling ambiguous cases. Annotators view the reference video together with the recordings of the two generated pages, and judge each pair separately on static appearance and animation by selecting “Example 1 better”, “Example 2 better”, or “Tie”. Annotators are blind to which model produced each page, and pairs are presented in randomized order. The final label of each pair is determined by majority voting among the five annotators.

4.2

RQ1: Overall Performance

Table 3 reports the results of the 14 MLLMs on AnimWeb-Bench. Systems Look Right but Move Wrong. Animation fidelity is low across the board. Even GPT-6 Astra, the strongest model, scores below 0.46 on every animation metric, and ten of the remaining thirteen models stay below 0.35. For every model, each animation score falls below each static score, and Velocity is the lowest of all eight metrics. Because the two dimensions are computed differently, their absolute values are not directly comparable. Two further observations, however, show that the gap reflects motion rather than scale. Animation metrics respond to the kind of motion on a page while static metrics do not (Section 4.4), and human annotators prefer every evaluated model over the baseline less often on animation than on static appearance (Section 4.5). Current models fail to infer and encode component-level dynamics in three ways. First, some animated components are missing from the generated page altogether, and their unmatched trajectories lower all four animation scores (Eq. (10)). Second, even when a component is animated, its trajectory or manner of change may deviate from the reference. For example, sliding along a different path or scaling instead of sliding, which lowers Position and Size. Third, and most subtle, is the temporal behavior of motion, namely when and how fast each component moves. Components may move faster, slower, or in a different direction than in the reference, which lowers Velocity. They may also appear at the wrong time: elements that the reference reveals gradually or one after another may appear all at once in the generated page, leaving time steps at which only one of the matched components is present and thus lowering all four animation scores. Appendix E illustrates each of these failures on two webpages. Faithfully recovering how a webpage moves thus remains an open challenge for current models.

AnimWeb-Bench: Benchmarking Animated Webpage Code Generation from Videos

Conference acronym ’XX, June 03–05, 2018, Woodstock, NY

Table 3: Performance comparison. All metrics are higher-is-better. Models marked with † receive the native video, and the others receive frames sampled at 2 fps. The best and second-best scores are marked in bold and underlined, respectively. Static Appearance Fidelity

Model

Structure ↑

CLIP ↑

Position ↑

Size ↑

Velocity ↑

CLIP ↑

0.4532 0.3735 0.3577 0.3928 0.3123 0.3170 0.3016 0.2313 0.2301 0.2060

0.4193 0.3359 0.3164 0.3495 0.2710 0.2561 0.2446 0.1790 0.1761 0.1586

0.3043 0.2401 0.2312 0.2653 0.2048 0.2133 0.2026 0.1456 0.1530 0.1238

0.4303 0.3511 0.3355 0.3660 0.2922 0.2960 0.2793 0.2148 0.2117 0.1908

0.3463 0.3063 0.2953 0.1702

0.2978 0.2543 0.2390 0.1294

0.2246 0.1983 0.1960 0.1170

0.3218 0.2836 0.2733 0.1584

GPT-6 Astra [33] GPT-6 Sol [34] GPT-5.6 Sol [35] Claude Fable 5.1 [3] Claude Opus 5 [4] Grok 4.7 [43] Qwen3.8 Max (0902)† [38] Gemini 3.1 Pro Preview† [14] Gemini 3.8 Flash† [15] Doubao-Seed-2.1-pro† [8]

0.6974 0.5754 0.5598 0.5952 0.5183 0.4575 0.4273 0.3453 0.3324 0.3378

Kimi K3† [25] DeepSeek V4.1 Flash [13] GLM 5.3 Flash† [57] Qwen3 VL 235B A22B Thinking† [6]

0.5373 0.4393 0.4293 0.2395

Open-Weight MLLMs 0.6775 0.6564 0.8204 0.5820 0.5854 0.7530 0.5707 0.5750 0.7441 0.4032 0.4337 0.5770

Structure↑

Agent: Frames

Agent: Video

Text↑

0.8

Agent: Video + FI

Agent: Video + FI + RF

Color↑

0.8

0.6

0.6

0.4

0.4

0.2

0.2

CLIP↑

1.0

0.6

0.8 0.6

0.4

0.4 0.2

0.0

Codex

Claude Code

Qwen Code

Position↑

0.0

Codex

Claude Code

Qwen Code

Size↑

0.4

0.0

Codex

Claude Code

Qwen Code

0.0

Qwen Code

0.0

Codex

Velocity↑

Claude Code

Qwen Code

CLIP↑ 0.4 0.3

0.15

0.2

0.10

0.1

0.1

Claude Code

0.20

0.2

0.2

Codex

0.25

0.3

0.3

0.0

0.2

0.30

0.4

Animation

Color ↑

Proprietary MLLMs 0.7804 0.7452 0.8796 0.6726 0.6557 0.8084 0.6503 0.6604 0.7898 0.7101 0.7014 0.8516 0.6240 0.6464 0.7907 0.6384 0.6169 0.7776 0.5797 0.5589 0.7643 0.5212 0.5449 0.7046 0.5061 0.5220 0.6794 0.4695 0.5149 0.6127

Single-pass MLLM

Static Appearance

Text ↑

Animation Fidelity

0.1

0.05 Codex

Claude Code

Qwen Code

0.00

Codex

Claude Code

Qwen Code

0.0

Codex

Claude Code

Qwen Code

Figure 5: Comparison of coding agents with their underlying MLLMs in single-pass generation. Codex is additionally evaluated with frame inspection (FI) and render feedback (RF). Proprietary Models Lead, but Open-Weight Models Are Closing In. GPT-6 Astra leads all eight metrics, and Claude Fable 5.1 ranks second on every metric. Among open-weight models, Kimi K3 ranks third in static Text and CLIP, surpassing GPT-6 Sol, and fifth across all animation metrics, outperforming Claude Opus 5 and approaching GPT-5.6 Sol. DeepSeek V4.1 Flash and GLM 5.3 Flash perform on par with the proprietary Grok 4.7 and Qwen3.8 Max (0902), and all four outperform the two Gemini models and Doubao-Seed-2.1-pro across all metrics.

4.3

RQ2: Coding Agents

Figure 5 compares coding agents with their single-pass MLLMs. Coding Agents Do Not Consistently Outperform Their Underlying Models. We compare each agent with its own underlying MLLM, and find that the benefit of agentic generation depends on both the agent and the input format. Qwen Code outperforms Qwen3.8 Max (0902) on all eight metrics under both settings, with relative gains of 4.3%–24.7% under Frames and 9.7%–36.1% under Video. Claude Code shows a reversal: it falls below Claude Opus 5 on all eight metrics under Frames, yet surpasses it on all eight under Video, with relative gains of 7.9%–21.7%. Codex, by contrast, trails

Conference acronym ’XX, June 03–05, 2018, Woodstock, NY

GPT-5.6 Sol on six metrics under Frames and seven under Video, with the remaining metrics changing by at most 1.4%. Comparing the two input formats, Video yields higher scores than Frames on all eight metrics for both Claude Code and Qwen Code, suggesting that video is generally the more informative input. Unlike the fixed 2 fps frames, a video allows an agent to choose its own sampling rate, for example, sampling densely around transitions, and can thus reveal intermediate states and timing that sparse frames miss. For Codex, however, Video outperforms Frames on only three metrics, indicating that this flexibility does not automatically translate into better reconstruction. The way in which an agent perceives visual input is therefore critical to its performance. Frame Inspection and Render Feedback Improve Agentic Reconstruction. To understand why Codex lags behind, we examine its execution traces under Video. We observe that Codex often extracts frames from the video, tiles them into contact sheets (e.g., nine frames per image), and relies on these sheets rather than on the individual frames (Appendix F). Such reduced tiles may obscure fine visual details and subtle changes between frames. We therefore introduce two additional Video settings for Codex. In Video + Frame Inspection (Video+FI), the prompt instructs Codex to also inspect each source frame individually at its original resolution whenever it builds a contact sheet. Video + Frame Inspection + Render Feedback (Video+FI+RF) further provides a recording tool that renders the generated page into a video, allowing Codex to compare its output with the reference and revise the code iteratively. The two enhancements improve Codex in different ways. Compared with Video, Video+FI improves all eight metrics by 7.3%–22.0% on the static metrics and 11.6%–15.6% on the animation metrics, and outperforms GPT-5.6 Sol on every metric. Building on Video+FI, Video+FI+RF further raises all four animation metrics by 4.0%–7.0%, whereas the static metrics change by only −0.3% to 2.5%. Frame inspection thus improves overall fidelity, while render feedback mainly benefits animation, likely because comparing the rendered page with the reference video exposes motion errors that are hard to detect from the code alone. With both enhancements, Codex outperforms GPT-6 Sol on all eight metrics and Claude Fable 5.1 on six, second only to GPT-6 Astra on most metrics. These results suggest that the bottleneck of agentic reconstruction lies not only in code generation but also in perception and verification: an agent improves substantially once it observes the reference closely and checks how its own page actually moves. Because the frame-inspection instruction was derived from Codex’s traces on this benchmark, we treat Video+FI and Video+FI+RF as diagnostic interventions that locate the bottleneck rather than as tuned methods.

4.4

RQ3: Animation Types

To compare performance across animation types, we divide the 124 webpages into three mutually exclusive groups based on animation type labels. (1) Spatial Motion contains at least one animation that moves components in space, such as translation, rotation, shake, or card flip. (2) Property Change contains no spatial motion but includes changes beyond visibility, such as scaling, color transitions, text animations, or numeric counters. (3) Visibility Only contains only animations in which components appear or fade in. The top part of Table 4 reports the average scores of the 14 MLLMs on each group.

Zhang et al.

Table 4: Average scores of the 14 MLLMs on webpages grouped by animation category and temporal behavior. Static Appearance

Group

Struct.

By Animation Category Spatial Motion Property Change Visibility Only

Text

Color

CLIP

Animation Pos.

Size

Vel.

CLIP

0.4615 0.6110 0.6049 0.7523 0.2911 0.2445 0.1855 0.2703 0.4654 0.5702 0.5919 0.7505 0.3190 0.2701 0.2141 0.2988 0.4811 0.6715 0.6284 0.8056 0.4325 0.3797 0.3291 0.4095

By Temporal Behavior Rapid, Continuous Change 0.3761 0.4972 0.5299 0.6883 0.2379 0.1969 0.1307 0.2228 Discrete State Transition 0.4886 0.6305 0.6414 0.7761 0.4194 0.3591 0.3004 0.3940

Win

Tie

Lose

60% 10% 30% 60% Doubao-2.1 28% 12% 60% 10% 40% 50% Gemini-3.1 20% 20% 40% 18% 42% 30% 34% 36% GLM-5.3 48% 16% 36% 28% 32% 40% DeepSeek-V4.1 56% 18% 26% 52% 22% 26% Grok-4.7 68% 12% 20% 58% 24% 18% Kimi-K3 78% 10% 12% 72% 14% 14% Claude-5.1 94% 4% 92% 6% GPT-6 0 20 40 60 80 100 0 20 40 60 80 100

(a) Static appearance

(b) Animation

Figure 6: Human evaluation results against Qwen3.8 Max (0902) on 50 sampled webpages. Spatial Motion Is the Hardest to Reproduce. Animation fidelity declines as the animations involve more motion. On animation metrics, Visibility Only scores the highest, followed by Property Change, while Spatial Motion scores the lowest, with a Velocity only 56% that of Visibility Only. This ordering reflects the demands of each group. Visibility changes only require models to determine when elements appear, and property changes alter the size, color, or content of components with limited motion, whereas spatial motion requires recovering how each element travels over time. The static metrics behave differently: between Spatial Motion and Property Change, they differ by −0.8% to +7.2% with no consistent direction, whereas all four animation metrics drop by 8.7%–13.4%. Animation categories thus mainly affect how well models reproduce motion rather than appearance, which shows that Animation Fidelity captures differences that the static metrics miss. Rapid, Continuous Changes Are Harder than Discrete State Transitions. Among individual animation types, the most challenging ones change the page rapidly and continuously, often across many elements at once, such as Card Flip, Twinkling, Text Scramble, Text Fragmentation, Shake, and Numeric Counter Animation. Their intermediate states shift at nearly every frame, making the timing difficult to infer from the reference video and to express precisely in code. The easiest types, such as Font Weight Animation, Icon State Transition, and Text Substitution, are instead discrete transitions between two clearly visible states, requiring models only to identify the start and end states and connect them with a simple transition. This contrast is also reflected in the bottom part of Table 4, where webpages with rapid, continuous changes score lower on all metrics than those with discrete state transitions.

4.5

RQ4: Human Evaluation

Figure 6 shows the human evaluation results, where higher win rates and lower loss rates indicate better reconstruction.

AnimWeb-Bench: Benchmarking Animated Webpage Code Generation from Videos

Conference acronym ’XX, June 03–05, 2018, Woodstock, NY

Human Judgments Align with Automatic Metrics. In both dimensions, human evaluators rank the models in nearly the same order as our automatic metrics. GPT-6 Astra is preferred over the baseline on 94% of the webpages for static appearance and 92% for animation, followed by Claude Fable 5.1, Kimi K3, and Grok 4.7, whose win rates decline in the same order as their automatic scores. DeepSeek V4.1 Flash and GLM 5.3 Flash perform close to the baseline, with comparable win and loss rates, consistent with their automatic scores differing from those of Qwen3.8 Max (0902) by less than 0.03 on every metric. Gemini 3.1 Pro Preview and DoubaoSeed-2.1-pro lose to the baseline on at least half of the webpages in both dimensions, again matching their low automatic scores. All eight metrics correlate positively with human preferences, with Spearman’s 𝜌 ranging from 0.667 to 0.737 (Appendix D). Animations Are Harder for Humans to Distinguish. Compared with static appearance, every model obtains a lower win rate and a higher tie rate on animation. The rise in ties is particularly pronounced for weaker models: the tie rates of DeepSeek V4.1 Flash, GLM 5.3 Flash, Gemini 3.1 Pro Preview, and Doubao-Seed-2.1-pro rise from 12%–20% to 30%–40%. When neither animation closely follows the reference, annotators find little basis to prefer one over the other, echoing our finding in Section 4.2 that reproducing animation remains challenging.

interactive computing systems. 1–6. [8] ByteDance Seed. 2026. Seed2.1 Officially Released: Advancing AI Productivity. https://seed.bytedance.com/en/blog/seed2-1-officially-released-advancingai-productivity [9] Bay-Wei Chang and David Ungar. 1993. Animation: From Cartoons to the User Interface. In Proceedings of the 6th Annual ACM Symposium on User Interface Software and Technology (UIST). 45–55. doi:10.1145/168642.168647 [10] Yang Chen, Minghao Liu, Yufan Shen, Yunwen Li, Tianyuan Huang, Xinyu Fang, Tianyu Zheng, Wenxuan Huang, Cheng Yang, Licheng Wen, et al. 2026. IWRBench: Can LVLMs reconstruct interactive webpage from a user interaction video?. In International Conference on Learning Representations, Vol. 2026. 52158– 52189. https://arxiv.org/abs/2509.24709 [11] Yuhong Dai, Yanlin Lai, Mitt Huang, Hangyu Guo, Dingming Li, Hongbo Peng, Haodong Li, Yingxiu Zhao, Haoran Lyu, Zheng Ge, Xiangyu Zhang, and Daxin Jiang. 2026. WebVR: Benchmarking Multimodal LLMs for WebPage Recreation from Videos via Human-Aligned Visual Rubrics. arXiv:2603.13391 [cs.CV] https: //arxiv.org/abs/2603.13391 [12] Truong Hai Dang, Jingyu Xiao, and Yintong Huo. 2025. Envisioning Future Interactive Web Development: Editing Webpage with Natural Language. In 2025 2nd IEEE/ACM International Conference on AI-powered Software (AIware). IEEE, 61–66. [13] DeepSeek-AI. 2026. DeepSeek-V4.1-Flash. https://huggingface.co/deepseekai/DeepSeek-V4.1-Flash [14] Google. 2026. Gemini 3.1 Pro. https://blog.google/innovation-and-ai/modelsand-research/gemini-models/gemini-3-1-pro/ [15] Google. 2026. Gemini 3.8 Flash and Gemini 3.8 Flash Cyber. https: //blog.google/innovation-and-ai/models-and-research/gemini-models/3-8flash-and-3-8-flash-cyber/ [16] Yi Gui, Zhen Li, Yao Wan, Yemin Shi, Hongyu Zhang, Bohua Chen, Yi Su, Dongping Chen, Siyuan Wu, Xing Zhou, Wenbin Jiang, Hai Jin, and Xiangliang Zhang. 2025. WebCode2M: A Real-World Dataset for Code Generation from Webpage Designs. In Proceedings of the ACM on Web Conference 2025 (Sydney NSW, Australia) (WWW ’25). Association for Computing Machinery, New York, NY, USA, 1834–1845. doi:10.1145/3696410.3714889 [17] Yi Gui, Zhen Li, Zhongyi Zhang, Guohao Wang, Tianpeng Lv, Gaoyang Jiang, Yi Liu, Dongping Chen, Yao Wan, Hongyu Zhang, et al. 2025. Latcoder: Converting webpage design to code with layout-as-thought. In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 2. 721–732. [18] Yi Gui, Yao Wan, Zhen Li, Zhongyi Zhang, Dongping Chen, Hongyu Zhang, Yi Su, Bohua Chen, Xing Zhou, Wenbin Jiang, and Xiangliang Zhang. 2025. UICopilot: Automating UI Synthesis via Hierarchical Code Generation from Webpage Designs (WWW ’25). Association for Computing Machinery, New York, NY, USA, 1846–1855. doi:10.1145/3696410.3714891 [19] Jeffrey Heer and George G. Robertson. 2007. Animated Transitions in Statistical Data Graphics. IEEE Transactions on Visualization and Computer Graphics 13, 6 (2007), 1240–1247. doi:10.1109/TVCG.2007.70539 [20] HTTP Archive. 2022. The 2022 Web Almanac: CSS Chapter. HTTP Archive Web Almanac. https://almanac.httparchive.org/en/2022/css [21] HTTP Archive. 2025. The 2025 Web Almanac: Accessibility Chapter. HTTP Archive Web Almanac. https://almanac.httparchive.org/en/2025/accessibility [22] JaidedAI. 2020. EasyOCR. https://github.com/JaidedAI/EasyOCR. [23] Anya Ji, Abhijith Varma Mudunuri, David M. Chan, and Alane Suhr. 2026. Animation2Code: Evaluating Temporal Visual Reasoning in Video-to-Code Generation. arXiv:2606.28593 [cs.CV] https://arxiv.org/abs/2606.28593 [24] Yilei Jiang, Yaozhi Zheng, Yuxuan Wan, Jiaming Han, Qunzhong Wang, Michael R Lyu, and Xiangyu Yue. 2025. Screencoder: Advancing visual-to-code generation for front-end automation via modular multimodal agents. arXiv preprint arXiv:2507.22827 (2025). [25] Kimi Team. 2026. Kimi K3: Open Frontier Intelligence. Technical Report. Moonshot AI. https://github.com/MoonshotAI/Kimi-K3 [26] Harold W Kuhn. 1955. The Hungarian method for the assignment problem. Naval research logistics quarterly 2, 1-2 (1955), 83–97. [27] Hugo Laurençon, Léo Tronchon, and Victor Sanh. 2024. Unlocking the conversion of Web Screenshots into HTML Code with the WebSight Dataset. arXiv:2403.09029 [cs.HC] doi:10.48550/arXiv.2403.09029 [28] Chinh T Le, Trevor Yee Siang Ong, Jingyu Xiao, Yuxuan Wan, and Yintong Huo. 2026. UIBenchKit: A unified toolkit for design-to-code model evaluation. In Proceedings of the 41st IEEE/ACM International Conference on Automated Software Engineering. 3445–3449. [29] Vladimir I. Levenshtein. 1966. Binary Codes Capable of Correcting Deletions, Insertions, and Reversals. Soviet Physics Doklady 10, 8 (1966), 707–710. [30] Zuhao Liu, Aleksandar Yanev, Ahmad Mahmood, Ivan Nikolov, Saman Motamed, Wei-Shi Zheng, Xi Wang, Lei Sun, Luc Van Gool, and Danda Pani Paudel. 2024. InTraGen: Trajectory-controlled video generation for object interactions. arXiv preprint arXiv:2411.16804 (2024). [31] M Ronnier Luo, Guihua Cui, and Bryan Rigg. 2001. The development of the CIE 2000 colour-difference formula: CIEDE2000. Color Research & Application: Endorsed by Inter-Society Color Council, The Colour Group (Great Britain), Canadian

5

Conclusion

We presented AnimWeb-Bench, a benchmark for animated webpage code generation from videos, together with a human-aligned evaluation protocol that measures both static appearance and animation fidelity. Our systematic evaluation of MLLMs and coding agents shows that while current systems reproduce static appearance reasonably well, they still struggle to capture the temporal dynamics of webpage animations. Our experiments further demonstrate the benefits of careful visual inspection and render feedback for coding agents. We hope AnimWeb-Bench will support future research on temporal visual understanding and code generation toward more faithful reconstruction of animated webpages.

References [1] 2026. AnimWeb-Bench Human Evaluation Guidelines. Accessed: 202611-01. https://github.com/starry1001/AnimWeb/blob/main/docs/human_ evaluation_guideline.md [2] Anthropic. 2026. Claude Code: An Agentic Coding Tool. https://github.com/ anthropics/claude-code [3] Anthropic. 2026. Claude Fable 5.1 and Claude Mythos 5.1. https://www.anthropic. com/claude-fable-and-mythos-5-1 [4] Anthropic. 2026. Introducing Claude Opus 5. https://www.anthropic.com/news/ claude-opus-5 [5] Awwwards. [n. d.]. Animated Websites. https://www.awwwards.com/websites/ animations/. Accessed: 2026-10-21. [6] Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, Wenbin Ge, Zhifang Guo, Qidong Huang, Jie Huang, Fei Huang, Binyuan Hui, Shutong Jiang, Zhaohai Li, Mingsheng Li, Mei Li, Kaixin Li, Zicheng Lin, Junyang Lin, Xuejing Liu, Jiawei Liu, Chenglong Liu, Yang Liu, Dayiheng Liu, Shixuan Liu, Dunjie Lu, Ruilin Luo, Chenxu Lv, Rui Men, Lingchen Meng, Xuancheng Ren, Xingzhang Ren, Sibo Song, Yuchong Sun, Jun Tang, Jianhong Tu, Jianqiang Wan, Peng Wang, Pengfei Wang, Qiuyue Wang, Yuxuan Wang, Tianbao Xie, Yiheng Xu, Haiyang Xu, Jin Xu, Zhibo Yang, Mingkun Yang, Jianxin Yang, An Yang, Bowen Yu, Fei Zhang, Hang Zhang, Xi Zhang, Bo Zheng, Humen Zhong, Jingren Zhou, Fan Zhou, Jing Zhou, Yuanzhi Zhu, and Ke Zhu. 2025. Qwen3-VL Technical Report. arXiv:2511.21631 [cs.CV] https://arxiv.org/abs/2511.21631 [7] Tony Beltramelli. 2018. pix2code: Generating code from a graphical user interface screenshot. In Proceedings of the ACM SIGCHI symposium on engineering

Conference acronym ’XX, June 03–05, 2018, Woodstock, NY

Society for Color, Color Science Association of Japan, Dutch Society for the Study of Color, The Swedish Colour Centre Foundation, Colour Society of Australia, Centre Français de la Couleur 26, 5 (2001), 340–350. [32] OpenAI. 2026. Codex: A Lightweight Coding Agent that Runs in Your Terminal. https://github.com/openai/codex [33] OpenAI. 2026. GPT-6 Astra: A New Generation of Intelligence. https://openai. com/index/gpt-6-astra/ [34] OpenAI. 2026. Introducing GPT-6 Sol and Luna. https://openai.com/index/ introducing-gpt-6-sol-and-luna/ [35] OpenAI. 2026. Previewing GPT-5.6 Sol: A Next-Generation Model. https:// openai.com/index/previewing-gpt-5-6-sol/ [36] Puppeteer Contributors. [n. d.]. Puppeteer: JavaScript API for Chrome and Firefox. https://github.com/puppeteer/puppeteer. Accessed: 2026-10-21. [37] Qwen Team. 2026. Qwen Code: An Open-Source AI Agent for the Terminal. https://github.com/QwenLM/qwen-code [38] Qwen Team. 2026. Qwen3.8-Max: A New Bar for Coding and Cowork. https: //qwen.ai/blog?id=qwen3.8 [39] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. In International conference on machine learning. PmLR, 8748–8763. [40] Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language Models are Unsupervised Multitask Learners. Technical Report. OpenAI. https://cdn.openai.com/better-language-models/language_ models_are_unsupervised_multitask_learners.pdf [41] Salesforce AI Research. 2026. GPA-GUI-Detector. https://huggingface.co/ Salesforce/GPA-GUI-Detector. [42] Chenglei Si, Yanzhe Zhang, Ryan Li, Zhengyuan Yang, Ruibo Liu, and Diyi Yang. 2025. Design2Code: Benchmarking Multimodal Code Generation for Automated Front-End Engineering. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), Luis Chiruzzo, Alan Ritter, and Lu Wang (Eds.). Association for Computational Linguistics, Albuquerque, New Mexico, 3956–3974. doi:10.18653/v1/2025.naacl-long.199 [43] SpaceXAI. 2026. Introducing Grok 4.7. https://x.ai/news/grok-4-7 [44] Stack Overflow. 2025. 2025 Developer Survey: AI. https://survey.stackoverflow. co/2025/ai [45] Yuxuan Wan, Yi Dong, Jingyu Xiao, Yintong Huo, Wenxuan Wang, and Michael R Lyu. 2024. MRWeb: An Exploration of Generating Multi-Page Resource-Aware Web Code from UI Designs. arXiv preprint arXiv:2412.15310 (2024). doi:10.48550/ arXiv.2412.15310 [46] Yuxuan Wan, Tingshuo Liang, Jiakai Xu, Jingyu Xiao, Yintong Huo, and Michael R. Lyu. 2026. From Runnable Code to Shippable Applications: Test-Driven Development for Full-Stack Web Application Generation. In Proceedings of the 41st IEEE/ACM International Conference on Automated Software Engineering (Munich, Germany) (ASE ’26). Association for Computing Machinery, New York, NY, USA, 726–738. doi:10.1145/3832783.3834388 [47] Yuxuan Wan, Chaozheng Wang, Yi Dong, Wenxuan Wang, Shuqing Li, Yintong Huo, and Michael Lyu. 2025. Divide-and-Conquer: Generating UI Code from Screenshots. Proc. ACM Softw. Eng. 2, FSE, Article FSE094 (June 2025), 24 pages. doi:10.1145/3729364 [48] Fan Wu, Cuiyun Gao, Shuqing Li, Xin-Cheng Wen, and Qing Liao. 2025. MLLMBased UI2Code Automation Guided by UI Layout Information. Proc. ACM Softw. Eng. 2, ISSTA, Article ISSTA050 (June 2025), 23 pages. doi:10.1145/3728925 [49] Jingyu Xiao, Dasen Dai, Yuxuan Wan, Jen-tse Huang, Yintong Huo, Hongyu Zhang, and Michael R Lyu. 2026. InterWebCoder: Learning Interactive Webpage Generation from Executable Tests via Reinforcement Learning. arXiv preprint arXiv:2610.28610 (2026). [50] Jingyu Xiao, Jiantong Qin, Shuoqi Li, Man Ho Lam, Yuxuan Wan, Jen-tse Huang, Yintong Huo, and Michael R Lyu. 2026. Component-based Reusable UI Code Generation for Complex Websites via Semantic Segmentation and Fine-grained Feedback. In Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 2. 5638–5649. [51] Jingyu Xiao, Yuxuan Wan, Yintong Huo, Zixin Wang, Xinyi Xu, Wenxuan Wang, Zhiyao Xu, Yuhang Wang, and Michael R. Lyu. 2025. Interaction2Code: Benchmarking MLLM-based Interactive Webpage Code Generation from Interactive Prototyping. In 2025 40th IEEE/ACM International Conference on Automated Software Engineering (ASE) (Seoul, Korea, Republic of). IEEE Press, 241–253. doi:10.1109/ASE63991.2025.00028 [52] Jingyu Xiao, Ming Wang, Man Ho Lam, Yuxuan Wan, Junliang Liu, Yintong Huo, and Michael R Lyu. 2025. Designbench: A comprehensive benchmark for mllm-based front-end code generation. arXiv preprint arXiv:2506.06251 (2025). doi:10.48550/arXiv.2506.06251 [53] Jingyu Xiao, Zhongyi Zhang, Haoran Hou, Yuxuan Wan, Yuan Jiang, Yintong Huo, and Michael R Lyu. 2026. VisualRepair: Dynamic Tool Calling and Region Focusing for Visual Software Issue Repair. arXiv preprint arXiv:2607.14075 (2026). [54] Jingyu Xiao, Zhongyi Zhang, Yuxuan Wan, Yintong Huo, Yang Liu, and Michael R Lyu. 2026. EfficientUICoder: A Bidirectional Token Compression Framework for

Zhang et al.

Efficient MLLM-Based UI Code Generation. Proceedings of the ACM on Software Engineering 3, FSE (2026), 2396–2418. [55] Mingde Xu, Zhen Yang, Yan Wang, Yu Wang, Xijun Liu, Zijun Dou, Wenyi Hong, Xiaotao Gu, Bin Xu, and Jie Tang. 2026. Video2Code: Generating Interactive Webpages from UI Videos via Action-Aware Revisit. arXiv preprint arXiv:2606.20711 (2026). [56] Sukmin Yun, Haokun Lin, Rusiru Thushara, Mohammad Qazim Bhat, Yongxin Wang, Zutao Jiang, Mingkai Deng, Jinhong Wang, Tianhua Tao, Junbo Li, Haonan Li, Preslav Nakov, Timothy Baldwin, Zhengzhong Liu, Eric P. Xing, Xiaodan Liang, and Zhiqiang Shen. 2024. Web2Code: a large-scale webpage-to-code dataset and evaluation framework for multimodal LLMs. In Proceedings of the 38th International Conference on Neural Information Processing Systems (Vancouver, BC, Canada) (NeurIPS ’24). Curran Associates Inc., Article 3560, 24 pages. [57] Z.ai. 2026. GLM-5.3-Flash. https://huggingface.co/zai-org/GLM-5.3-Flash [58] Xuanle Zhao, Qiushi Sun, Jingyu Xiao, Xuexin Liu, Haoyue Yang, Qiaosheng Chen, Xianzhen Luo, Jing Huang, Yufeng Zhong, Lei Chen, et al. 2026. Beyond NL2Code: A Structured Survey of Multimodal Code Intelligence. arXiv preprint arXiv:2606.15932 (2026). [59] Ting Zhou, Yanjie Zhao, Xinyi Hou, Xiaoyu Sun, Kai Chen, and Haoyu Wang. 2025. DeclarUI: Bridging Design and Development with Automated Declarative UI Code Generation. Proc. ACM Softw. Eng. 2, FSE, Article FSE011 (June 2025), 23 pages. doi:10.1145/3715726

AnimWeb-Bench: Benchmarking Animated Webpage Code Generation from Videos

A

Conference acronym ’XX, June 03–05, 2018, Woodstock, NY

Table 5: Video and asset statistics of AnimWeb-Bench.

Data Copyright and Usage

The webpages in AnimWeb-Bench were collected from Awwwards [5], and all webpages, recordings, and assets remain the property of their respective owners. To respect their ownership, we retain the source URL of each instance to credit the original webpage, and release AnimWeb-Bench solely for non-commercial research purposes. We will also promptly remove any content found to be used inappropriately.

B

Dataset Statistics

Table 5 reports the statistics of AnimWeb-Bench. Each video lasts 5.91 s and contains 2.83 animation segments and 35.4 tracklets on average, and each sample comes with 46.1 asset files on average. Following Design2Code [42], we measure code complexity with four metrics: Length is the token count of the HTML code under the GPT-2 tokenizer [40], Tags is the number of HTML tags, DOM Depth is the maximum depth of the DOM tree, and Unique Tags is the number of distinct tags. As shown in Table 6, an Animation2Code [23] sample contains only 71 tags on average, with a DOM depth of 5 and 7 distinct tags, since each sample is a standalone animation snippet rather than a complete webpage. In contrast, AnimWeb-Bench pages contain 1,196 tags, a DOM depth of 18, and 33 distinct tags on average, exceeding all the compared webpage benchmarks.

C

Implementation Details

D

Metric

Min

Max

Mean ± Std

Duration (s) Segments Tracklets Asset files

2.0 1 2 2

15.0 11 120 422

5.91 ±2.47 2.83 ±1.84 35.4 ±24.1 46.1 ±53.6

Correlation Between Automatic Metrics and Human Judgments

We quantify the agreement between our automatic metrics and human judgments using Spearman’s rank correlation coefficient 𝜌. The human evaluation in Section 4.5 yields 400 pairwise comparisons (8 models × 50 webpages), each between a model and the baseline. For each comparison, we encode the majority-vote human preference as 1 if the model is preferred, 0 for a tie, and −1 if the baseline is preferred, and compute the difference between the model’s and the baseline’s automatic scores on that webpage. Each metric is correlated with the human preferences in its own dimension, i.e., static metrics with static appearance judgments and animation metrics with animation judgments. Ties are retained and assigned average ranks. As shown in Table 7, all eight metrics correlate positively with human preferences (𝜌 = 0.667–0.737), supporting their alignment with human judgments.

This section specifies the terms and thresholds used in the evaluation metrics of Section 3.3.

E Failure Case Studies E.1 Case 1: Rotating Gallery Hero

Keyframe alignment. On downsampled frames, Vis in Eq. (2) is computed as

In the reference (Figure 7(a)), the navigation bar is present from the start. Two photo clusters at the top-right and bottom-left corners then rotate in depth, the paragraph at the bottom right appears, and the headline is finally revealed line by line through a mask. Claude Fable 5.1 merges the two clusters into a single diagonal strip spanning the page, which lowers Structure. The strip fades in and drifts toward the bottom left instead of rotating in place, so its tiles deviate in Position, Size, and animation CLIP, and move in a different direction and at a different speed, lowering Velocity. The headline fades in rather than emerging from a mask, so its box appears at full size instead of growing as it is unveiled, further lowering Size. GPT-6 Sol reproduces the masked headline but not the rotation: its photos become small, static fans of cards, lowering Size and Velocity. Its timeline is also reordered: the photos appear before the navigation bar, and the paragraph only after the headline. Components shown in the reference are therefore missing from the generated page for many time steps, which lowers all four animation metrics.

Vis = 0.15 𝑠 rgb +0.10 𝑠 gray +0.10 𝑠 hist +0.10 𝑠 edge +0.10 𝑠 cos +0.45 𝑠 layout, where 𝑠 rgb and 𝑠 gray are one minus the mean absolute pixel differences in RGB and grayscale, 𝑠 hist is the RGB histogram intersection, 𝑠 edge and 𝑠 cos are the cosine similarities of Canny edge maps and of RGB images, and 𝑠 layout compares the connected regions of white, colored, and foreground masks. We set 𝜆 = 0.85. Keyframes are aligned by order-preserving dynamic programming, where a pair can be matched only if 𝑠 ≥ 0.2, and each unmatched reference keyframe incurs a penalty of 0.12. Structure similarity. All boxes are normalized by the frame width and height. For a matched pair (𝑏 𝑟 , 𝑏𝑔 ) in Eq. (3), 𝑠 iou is the intersection over union of the two boxes; 𝑠 center is the center proximity computed as in Eq. (7), applied to the centers of 𝑏 𝑟 and 𝑏𝑔 ; 𝑠 area is the ratio of the smaller box area to the larger one; and 𝑠 aspect is the ratio of the smaller aspect ratio to the larger one. Trajectory construction and matching. In Algorithm 1, IoU is the intersection over union between a detection 𝑏 and the latest box of trajectory 𝜏, App is the cosine similarity between the CLIP image features [39] of the two box regions, and Ctr is the center proximity computed as in Eq. (7). We set 𝜃 𝑎 = 0.35 and 𝜃 𝑡 = 0.82.

E.2

Case 2: Curtain-Reveal Portfolio

In the reference (Figure 7(b)), a loading icon animates on a dark screen. A curtain-like background transition follows, in which brown and orange layers sweep upward. The name is then revealed letter by letter from left to right, and the four photos fade in place one after another.

Conference acronym ’XX, June 03–05, 2018, Woodstock, NY

Zhang et al.

Table 6: Code complexity of AnimWeb-Bench and existing benchmarks (Mean ± Std). Metric

Design2Code

WebCode2M

MRWeb

Animation2Code

AnimWeb-Bench

Length (tokens) Tags DOM Depth Unique Tags

31,216 ±23,902 158 ±100 13 ±5 22 ±6

5,366 ±2,393 184 ±77 15 ±5 24 ±6

113,724 ±139,761 543 ±768 15 ±6 22 ±8

5,427 ±13,678 71 ±247 5 ±3 7 ±2

239,303 ±487,624 1,196 ±894 18 ±4 33 ±7

(a) Case 1: Rotating Gallery Hero Claude Fable 5.1

Reference

(b) Case 2: Curtain-Reveal Portfolio

GPT-6 Sol

Reference

GLM 5.3 Flash

Gemini 3.1 Pro Preview

Figure 7: Failure cases on two webpages. Frames are ordered from top to bottom in time. Table 7: Spearman’s rank correlation 𝜌 between automatic metrics and human preferences over 400 pairwise comparisons against the baseline. Dimension

Metric

Spearman’s 𝜌

Static Appearance Fidelity

Structure Text Color CLIP

0.7365 0.7113 0.6951 0.7172

Animation Fidelity

Position Size Velocity CLIP

0.7138 0.7350 0.6667 0.7160

GLM 5.3 Flash omits the loading screen and the transition entirely. The first reference keyframe and segment therefore have no counterpart, lowering all static and animation metrics. The name

fades in as whole words instead of being revealed letter by letter, lowering all four animation metrics. The photos rise from below instead of fading in place, lowering Position and Velocity. Gemini 3.1 Pro Preview keeps the loading screen but not its animated icon. Its transition fades the background color instead of sweeping the layers upward, so the moving layers find no counterpart, lowering all four animation metrics. The two words of the name swap their vertical order, and the text is misplaced, lowering Structure and static CLIP. The four photos slide up from below all at once instead of fading in one after another, lowering all four animation metrics. They also settle enlarged at the wrong positions, further lowering Structure and static CLIP.

F

Coding Agent Execution Trace

Figure 8 shows a simplified Codex execution log on one webpage across the three settings. The settings are cumulative: FI adds frame inspection to video input, and RF further adds render feedback. With video input alone, the agent perceives the animation only through contact sheets of downscaled frames (lines 5–8) and returns its first

AnimWeb-Bench: Benchmarking Animated Webpage Code Generation from Videos

Video

Conference acronym ’XX, June 03–05, 2018, Woodstock, NY

+ Frame Inspection (FI)

+ Render Feedback (RF)

1

Video

agent: "I'll inspect the supplied animation and asset files to match the composition and timing precisely."

# states its plan

2

Video

$ find /inputs -maxdepth 3 -type f | sort

# reference.mp4, stitched_image_{1,2}.jpg

3

+RF

$ sed -n '1,240p' tools/README.md

# reads how to use the recorder

4

Video

$ ffprobe -show_entries stream=width,height,r_frame_rate,nb_frames reference.mp4

# 1920×1080, 30 fps, 180 frames, 6.0 s

5

Video

$ ffmpeg -i reference.mp4 -vf "fps=2,scale=480:-1,tile=5x4" -frames:v 1 contact.jpg

# one sheet of 480-px thumbnails

6

+FI

$ ffmpeg -i reference.mp4 -vf "select='eq(n,0)+eq(n,15)+…+eq(n,179)'" -vsync 0 ref_frames/f%02d.png

# 13 frames at full 1920×1080

7

Video

$ for t in 0 0.5 … 5.5; do ffmpeg -ss $t -i reference.mp4 -vf scale=640:-1 "$t.png"; done; montage … timeline.jpg

# fails: montage: command not found

8

Video

$ ffmpeg -i reference.mp4 -vf "fps=2,scale=640:-1,tile=3x4" -frames:v 1 timeline2.jpg

# falls back to another contact sheet

9

+RF

agent: "The reference is a six-second intro: a centered “p” mark appears after a one-second black hold, eight portfolio tiles stream in and stack, the wordmark resolves, …"

# describes the animation before coding

10

Video

agent: ```html <!DOCTYPE html><html lang="en"><head> <title>Pellmell</title><style>:root{--black:#101010; …

# initial page; final output for Video / +FI

11

+RF

$ ./tools/record.sh --html index.html --duration 6 --width 1920 --height 1080 --output preview.mp4

# round 1: records the generated page

12

+RF

$ for t in 2 2.5 … 5.9; do ffmpeg -ss $t -i preview.mp4 -frames:v 1 ours/t$t.png; done

# extracts frames to compare with reference

13

+RF

edit

# revises the code

14

+RF

…

15

+RF

agent: ```html <!doctype html><html lang="en"><head> <title>Pellmell</title><style>*{box-sizing:border-box} …

index.html

round 2, 3, …, n: record.sh → extract frames → edit index.html

# same loop, details omitted # final page

Figure 8: Simplified Codex execution log on one webpage of AnimWeb-Bench across three settings. Lines tagged +FI or +RF are steps added by that setting. page without ever viewing it (line 10). Frame Inspection supplies the same moments at full resolution (line 6), exposing details lost in the thumbnails, but the generated page still goes unseen. Render Feedback closes this loop: the agent records its own page (line 11), extracts frames to compare with the reference (line 12), and

revises the code (line 13), repeating this cycle before returning the final page (lines 14–15). Only under Render Feedback does the agent observe the output it generates, allowing errors in timing and motion to be corrected. Received 20 February 2007; revised 12 March 2009; accepted 5 June 2009

Related documents

• IChart2Code: Benchmarking Multimodal Large Language Models for Interactive Chart Code Generation#1116744
• What Comes Next? Omni-StoryBench for Evaluating Story-Grounded Omnimodal Generation#1141360
• CULTURESCORE: Evaluating Cultural Faithfulness in Video Generation Models#266217
• VISTA-Bench: Benchmarking Multilingual Image Translation with Image-Specific Rubrics#1141330
• Does Understanding Inform Generation in Unified Multimodal Models? From Analysis to Path Forward#1104916