ScreenProof: Where Is This Screen Wrong? Finding UI Faults Without a Reference

ScreenProof: Where Is This Screen Wrong? Finding UI Faults Without a Reference Huiming Sun1 Yue Li2 Kangning Yang1 Ruineng Li1 Jie Cai1 Lan Fu1 Yunwan Fu3 Jinlong Li1 Jinglin Shen1 1

OPPO US Research Center

2

3

Cleveland State University

Shanghai Jiao Tong University

Where is this screen wrong?

Finding problems, cheaply

one ColorOS 17 page, three faults · share of each type found

consistency test, no target 0.5

Activity card from Material 3

ProofJudge (ours) 0.48

component: from another design system Gemini 4 Argon

43%

gpt-6.1-sol

30%

ProofJudge (ours)

20%

Text runs off the screen

0.4 Gemini 4 Argon 0.37 0.3

ProofJudge finds the most

general: visible without a target Gemini 4 Argon

38%

gpt-6.1-sol

45%

ProofJudge (ours)

59%

Sol 0.30

F1 ↑

arXiv:2611.10234v1 [cs.AI] 6 Nov 2026

huiming.sun1@innopeaktech.com

Gemini 3.8 Flash 0.26 Luna 0.22

open VLMs ≤ 0.15

0.2

0.1

One row with square corners local style, corner radius: unlike its siblings Gemini 4 Argon

53%

gpt-6.1-sol

47%

ProofJudge (ours)

41%

0.0

$0.03

$0.1

$0.3

$1

$3

$10

no cost data

← cheaper · $ per 1,000 pages

Figure 1. Where is this screen wrong, and which judge finds it (consistency test, 2,765 screenshots, no target). Left: one generated ColorOS 17 page with three faults the data engine made in its code: an activity card from a Material 3 Expressive page (a component from another design system), text that runs off the screen (general) and one list row with square corners (local style). Bars give the share of all test faults of that type each judge finds. Right: F1 against cost per 1,000 pages; point area is an open model’s size, diamonds are API judges. ProofJudge (9B) reaches 0.48 for $0.04, the highest F1 of all judges: above Gemini 4 Argon (0.37, $4.42), gpt-6.1-sol (0.30) and every open VLM (F1 at most 0.15, $0.07–1.96). GPU cost assumes $2 per GPU-hour; gpt-6.1-sol and GPT-6 Luna report no token counts.

Abstract

frontier models flag 18–62% of clean real screenshots and 18–82% of clean generated pages, find 47–72% of text layout faults, and rarely find a foreign icon (5–19%). Our small critic ProofJudge (Qwen3.5-9B with LoRA), trained on codemade pages with one to four faults each and keeping a finding only when it is confident, reaches F1 0.48, the highest of all judges (Gemini 4 Argon 0.37; open models at most 0.15; 0.46 even without the confidence cut), and is the only faultfinding judge within 15% false alarms on clean pages (2% real, 9% generated), the property such a first pass needs, at about a hundredth of Argon’s cost. It finds the most general faults (59%) but fewer local style faults than Argon (25% against 41%) and fewer text layout faults than the API judges (43%; Gemini 4 Argon 47%). Against a named brand the task is harder still: models know a design language about

A phone screen can be wrong in ways that need no reference: a label that runs off the screen, letters that touch, a line that breaks one character before its end, one button with sharp corners among round ones. We ask whether a model can box every such fault on a screenshot, given no target design and no reference image. Such a critic could serve as a cheap first pass over generated pages before a designer reviews them, but only if it stays silent on clean pages. ScreenProof poses this target-free critique over three families of faults: general faults, local style faults (one element breaks the style its page sets) and text layout faults (letter spacing, line height, line breaks). A data engine makes each fault in page code and records the changed element’s box, and hard negatives change every sibling alike. No frontier judge is reliable:

1

cost of an API judge (Fig. 1), and is the only fault-finding judge that stays silent on at least 85% of clean real and generated pages, the property a pre-filter needs. It recalls more general faults than any other judge (59%; API judges 31–49%), yet finds fewer local style faults than Gemini 4 Argon (25% vs. 41%), few foreign icons (5%) and 43% of text layout faults. Exact labels are costly to annotate, so we synthesize them: a data engine injects faults in page code, recording exactly which element changed, and hard negatives change every sibling alike so that only a difference from the page counts. An external benchmark of real screenshots (OwlEye) tests whether a critic’s skill transfers beyond our pages. A named brand makes the task harder still: models know a design language about as well as its public documentation states it (Sec. 5; Sec. D).

as well as its public documentation states it.

1. Introduction Many faults on a screen need no reference to be seen. Some are wrong in any design: a label that runs off the screen, two cards that overlap, a button too small to tap. Some break the style the page itself sets: one button with sharp corners among round ones, one icon from another set, one component lifted from another design system. And some are faults of text layout: letters that touch, lines that crowd, a word broken in the middle, a line that ends on a single character. Designers see these at a glance, but a useful review must say where: which element is wrong and why. The question is newly pressing in generation. Multimodal models now turn screenshots and text into front-end code [24, 29, 32, 33], and their pages place text worse than they copy it [10, 24], while edits change parts nobody asked to change [17, 31]. A generated page usually comes with no reference image and no specification, so the check must use the screenshot alone. Such a critic has a direct use. It could check every generated page before a designer sees it, pass the clean ones, and send back only the flagged ones with the element and the reason; the box can also guide a code fixer or a generator. To save review time, the critic must stay silent on clean pages: one that flags most clean pages sends every page back to a human. We seek one critic that boxes every such fault on a screenshot, given no target design and no reference. ScreenProof measures this and shows that no model can do it yet, and why. Critics cannot tell a clean page from a faulty one. Frontier judges flag 18–62% of clean real screenshots and 18–82% of clean generated pages (Tab. 4); none stays within 15% on both. One element off its own page often escapes every judge. Asked whether any element breaks the style set by the rest of the page, frontier judges find a changed colour or surface material most often, but catch a single foreign icon in only 5–19% of cases; many icon faults look normal on their page and are foreign only by a design language’s conventions (Fig. 6). Text layout is found unevenly. Frontier judges find 47– 72% of text layout faults (Tab. 5): nearly every word broken in the middle, but few labels whose characters touch, and many pages where every sibling shares the same wide spacing are flagged. Our critic addresses the first failure, false alarms. Training narrows the gap without closing it: our small critic ProofJudge (Qwen3.5-9B with LoRA, Fig. 3), trained on pages with one to four faults each and keeping only confident findings, reaches the highest F1 of all judges (0.48; 0.46 without its confidence cut; Gemini 4 Argon 0.37) at a fraction of the

Contributions. • Benchmark. ScreenProof: target-free critique of phone screenshots over general, local style and text layout faults, with exact boxes, clean pages and hard negatives. It is built from public sources and released in full. Brand tracks (knowledge, critique, design): Sec. D. • Data engine. Faults are injected in page code, not pasted onto pixels, so the edit is the label: general faults, one element against its own page (radius, material, colour, icon, or a component from another design system), and one text element’s spacing, line height or line breaks. • Diagnosis and a baseline. Why the task remains unsolved, measured on our tests and an external benchmark, and ProofJudge, a lightweight critic trained on multi-fault pages, the highest F1 on our consistency test and the only judge within the false-alarm gate, at about $0.04 per 1,000 pages; it shows what training a small critic does and does not fix.

2. Related Work VLMs as judges. FOCUS [16] perturbs image-to-text and text-to-image outputs to measure how often VLM judges miss errors, using perturbations only to benchmark judges without covering UI. Visual manipulation inflates LVLM judge scores [13], rubrics aid aesthetic judges [14], and MobileForge [29] admits a judge only if it ranks a hidden original first. UI2CodeN [33] fine-tunes a judge for wholepage comparison. Instead, we train a judge on exact-label UI faults to locate the faulty element. Target-free critique and false alarms. A critic without a reference must decide both whether a page has a fault and where it is. Judge studies measure the first half: evaluators miss perturbed errors [16] and can be pushed to inflated scores [13], and UI defect benchmarks report recall on faulty

2

ScreenProof at a glance GENERAL FAULT

LOCAL STYLE FAULT

LOCAL STYLE FAULT

TEXT LAYOUT FAULT

CLEAN PAGE

Text off the screen

One radius off

A foreign list row

Word broken inside

Nothing to find

2,765 screenshots · consistency test 920 general · 576 local style · 662 components · 607 clean

513 pages · text layout test 367 text layout faults · 75 clean · 71 hard negatives

17,285 training rows · ScreenProof-Train iOS chat page marked: the exact label

iOS files page marked: the exact label

ColorOS files page marked: the exact label

HarmonyOS weather page marked: the exact label

HarmonyOS chat page a critic should stay silent

2,060 general · 1,609 local style · 2,110 components · 972 text layout · 3,341 multi-fault or designed pair · 7,193 clean

Figure 2. ScreenProof at a glance. Left: one test example per fault family, each from another page: the faulty area enlarged on top, and the whole screen below with the data engine’s exact label as corner marks; a clean page has nothing to find, and a critic should stay silent on it. Right: the two test sets and the released training set, split by fault family in the same colours (Secs. 3 and 4). Every fault is made in page code, so its box is exact.

screens [18, 30]. How often a critic boxes something on a page with nothing wrong is rarely reported; we score it on clean pages and hard negatives, and it is where frontier judges fail.

reference-free faults of a single screen with exact labels and hard negatives. UI understanding and grounding. Whereas UI datasets [3, 7], screen models [1, 34] and GUI agents [6, 9, 11] locate elements matching a description, we locate elements that break a design language.

UI quality and design evaluation. UI-Lens [30] evaluates general MLLMs on UI display defects with expert labels: nine MLLMs reach an average F1 of 22.19 on text overflow and 11.44 on text that differs across screens. OwlEye [18] detects display issues such as overlapping and missing text in human-built apps, and design-to-code benchmarks find that models copy text content well but place it badly (GPT-4o: 98.2 on text similarity, 85.5 on position) [24]. UICrit [8] collects designer critiques for automated evaluation. Designsystem compliance of LLM-generated UI has been measured via specification checklists and an LLM judge [4]. We are not aware of work that boxes elements inconsistent with their own page, and scores false alarms on clean pages and on hard negatives, without a reference.

UI generation and self-correction. Screenshot-to-code methods [25, 28] and rendered-feedback loops [32, 33] reproduce a reference, while MultiRef [5] conditions generation on role-annotated references. Generated pages place text worse than they copy it [10, 24], and edits alter unrequested parts [17, 31]. Since self-correction without external feedback is unreliable [12, 19], our fix test gives the fixer a judge’s findings, a judge’s gate or no hint (Sec. 7). Table 1 compares these works feature by feature.

3. ScreenProof Task. The input is a screenshot S and nothing else: no target design, no reference image, no specification. The output is a verdict y ∈ {clean, faulty} and a set of findings {(Bi , ei , ki , ci )}: a box Bi on S, the element type ei , the fault family ki ∈ {general, local style, text layout} (Tab. 2), and a short reason ci . For local style faults the page is its own reference: an element is wrong because it differs from its siblings, so no brand knowledge is needed. Because only pixels are given, the task is independent of how the screen was built: S may come from a web page, a native app or a design-tool export; the task definition does not depend on which.

Benchmarks for UI generation and critique. Design2Code [24] scores screenshot-to-HTML by visual similarity, WebGen-V [26] section-level visual design, DesignBench [31] generation, editing and repair across frameworks, MobileForge [29] multi-screen apps by build, navigation and fidelity, and EvoGenUI-Bench [21] multi-turn generative UI. Closest to us, TaoD2C-Bench [23] goes beyond visual fidelity with 97,652 expert annotations of required versus permitted implementation constraints on 2,861 production designs; whereas its constraints belong to one product’s code and are checked in generated code, ours are faults of the rendered screen, checked from pixels alone. For critique, WiserUI-Bench [15] asks which of two A/B-tested designs won, UXBench [20] poses usability questions on mobile screenshots, a second UXBench [27] scores critique actionability, and PerceptUI [2] simulates users. None boxes

Test sets. The consistency test (Fig. 2) holds 2,765 screenshots, 607 of them clean (120 real, 487 generated), with 920 general faults (display faults included) and 1,238 local style 3

Table 1. Positioning. Entries follow the papers’ own descriptions. Kind

UI

Target

Locates elements

Names convention

Judge trained / admitted on faults

Design2Code [24] UI2CodeN [33] ComUICoder [32] MobileForge [29] MultiRef [5] DS-Compliance [4] FOCUS [16] UI-Lens [30]

generation benchmark generation generation generation benchmark generation generation study judge benchmark judge benchmark

yes yes yes yes – yes – yes

one screenshot one screenshot pages screens references with roles design-system spec task output display quality

block matching – per element (detector) – – – – boxes

– – – – – checklist – defect categories

– fine-tuned whole-page judge – positive anchor only – – perturbations, evaluation only –

ScreenProof + ProofJudge (ours)

judge + benchmark

yes

none (named brand: Sec. D)

boxes

yes

code-level faults

Table 2. Fault taxonomy. The main task gives no target; a finding is general, local style or text layout (brand faults, which need a named target: Sec. D). Engine fault: how the data engine makes it (Sec. 4). Aspect

What goes wrong

General: visible without a design target Layout overflow, overlap, order, missing item, two selected Readability contrast, tap size, typeface Display broken image, null value, needless ellipsis

Table 3. ScreenProof-Train: training rows and images by kind. A row is an image with its labels; an image can appear in several rows.

Engine fault each, in code each, in code each, in code

Local style: one element unlike the rest of its page Style radius, material, accent, icon one element Component a control from another design one transplant system Text layout: one text element Spacing characters touch, spacing unlike siblings Lines lines touch, line height unlike siblings Breaks short label wraps, lone last word, word broken, needless ellipsis, CJK punctuation at line start

Kind

Rows

Images

Clean page Designed difference (faulted twin) General fault Local style fault Transplanted component Text layout fault Page structure (dropped item, reordered sections) Two to four faults of the kinds above

7,193 630 1,542 1,609 2,110 972 518 2,711

2,605 591 1,427 1,505 1,484 847 444 2,521

All

17,285

11,424

the fault copies the data engine makes from them. It is released in full, images included, with the source, date and terms of each item. Section C describes the collection, with sample screenshots and counts (Figs. S1 and S2).

one text element one text element one text element

4. Data Engine The engine operates on HTML pages. HTML lets us make exactly one change, read the element’s rendered box from the DOM, and re-render the page identically on any machine; a judge sees only the screenshot, though judges still flag generated and real screens at different rates (Tab. 4). The engine turns a clean page into copies with known faults. Because each fault is injected in code, the engine knows the changed element, its rendered box and its kind in Tab. 2; these form the label. Copies whose change is not visible on the first screen are discarded.

faults (transplanted components included), one per faulty page; the text layout test holds 367 faults on held-out page requirements, 75 clean pages and 71 hard negatives in which every sibling changes alike (Sec. 4). Every page in both sets is scored without a target and with the same prompt. Metrics. We report localization precision and recall, where findings and labelled elements are matched one-to-one at box IoU ≥ 0.5 (so a small box inside a large faulty element does not count), alongside scores at IoU ≥ 0.3 (Tab. 4; Tab. S8), box F1, and the share of clean pages and hard negatives a judge flags. Results are reported per family and type.

Base pages. For each of 14 page requirements, a generator writes an English base page in each of eight phone design languages and marks each component root with its kind (switch, tab bar, top bar, button, list row, search, chip, segmented control, slider, dialog, sheet, floating button), in light and dark mode (224 planned). Three requirements per design language are held out for testing. A base page serves as a clean page only if two judges find it on-spec item by item; 80% of the 223 generated pages pass (Sec. D).

Data. ScreenProof comprises pages collected from the web: phone screenshots from the vendors’ own pages (product pages, user guides, design guidelines), screens from public UI datasets, pages built with open-source UI libraries, generated base pages in eight phone design languages, and

4

Page + prompt Prompt · no target Which elements break this page’s own style?

Vision

Qwen3.5-9B

Vision encoder frozen

Exact engine label LoRA

Frozen layer

LoRA

element: workout title

Frozen layer

LoRA

convention: text runs past

Every LM linear layer Vision and LM weights frozen

verdict: inconsistent

Frozen layer

box: [244, 754, 1000, 779] kind: general the screen edge

SFT on label JSON, 1–4 faults per page

Figure 3. Overview of ProofJudge. A data-engine page and prompt pass through Qwen3.5-9B with its vision encoder and weights frozen; one LoRA on every linear layer of the language model is trained by SFT on the label JSON across pages with one to four faults each. The prompt gives no target and asks which elements break the page’s own style. The label is exact: the engine made the title overflow, a general fault under any design.

General faults. Display faults (a cropped or missing image, a null value, a needless ellipsis), overlap, text overflow, low text contrast and a shrunk tap target are visible in any design; a dropped item, reordered sections, two selected items in a single-choice group and a block set in another typeface are general faults that break the structure the page sets. The consistency test holds 920 such faults: display 347, overlap 115, two selected 111, typeface 77, contrast 66, reordering 65, tap target 48, overflow 47 and dropped item 44.

every sibling, so the page stays consistent (Sec. J). ScreenProof-Train. The engine also makes a released training set: 17,285 rows on 11,424 images of 990 pages (542 generated, 448 public web pages), 42% clean, with 591 designed differences each paired with a faulted twin (Tab. 3; Sec. O); it trains the paper’s ProofJudge.

5. What Judges Find We test five frontier models from three companies, nine open VLMs and the open base Qwen3.5-9B on the target-free tests, all with the same prompt.

Local style faults. One element of a kind that appears at least twice on the page takes another corner radius, material, accent or icon, while its siblings keep the page’s own value; for example, one button’s radius drops from 28 to 9 px while all cards keep 28 (Sec. N). Because the rest of the page serves as the reference, the fault can be judged without a target. The consistency test holds 576 such faults (icon 188, accent 179, material 123, radius 86).

Knowing is not finding. Asked in words, frontier models answer every brand-free design question they parse (28–31 of 31), and 41–59 of 70 when they must name the defect in a red box on a screenshot; training lifts Qwen3.5-9B from 33 to 60 of 70 on the latter (ProofJudge v5). These probes suggest a gap between knowing what a defect is and finding it on an unmarked page without false alarms.

Transplanted components. One component is replaced by the same kind lifted from a page of another design system (Fig. 4), with all computed styles inlined and its ::before/::after content materialized so it renders as on its source page; its post-layout box is the label. Nothing is pasted onto pixels, so no compositing trace can act as a cue. The consistency test holds 662 transplanted components.

Critics find much, and much that is not there. Frontier judges flag 18–62% of clean real screenshots and 18–82% of clean generated pages (Tab. 4; Fig. 7), and four of the nine open VLMs flag nearly every clean page. Gemini 4 Argon comes closest to the gate (18% and 18%); only our trained ProofJudge passes it among fault-finding judges (Sec. 6). Are the labels right? One of us checked 481 test items by hand, the engine’s change against its page and clean pages for anything wrong, and found 85 mislabelled: a clean page with a fault, or a change that is hidden or still looks fine. With them left out, every F1 changes by at most 0.01 (ProofJudge 0.49, Gemini 4 Argon 0.38) and the order of the judges stays. Most false alarms are thus the judges’, though some label noise remains.

Text layout faults. Each fault changes the CSS of one text element of a held-out base page; the label is the box of that element’s text after layout. Seven rules are wrong on their own: the characters of a label touch, the lines of a paragraph touch, a short control label wraps, one word or character stands alone on the last line, a word is broken inside, Chinese punctuation starts a line, and a label is cut with an ellipsis although its row has room. Two rules are wrong only against the siblings: one of at least three like labels gets wider letter spacing, or one of at least three paragraphs a larger line height; for these, a hard negative applies the same change to

One element against its own page. Without a target (Fig. 6), the two strongest API judges find a changed corner 5

Reference

Brand faults

Original generated page ColorOS 17

iOS 26 tab bar from its own dark-mode page

General faults

M3 button Material 3 Expressive

iOS typeface font stack with fallback face

Overflow text runs out of its card

Overlap card covers the title

Small target button shrunk to 45%

Low contrast “426” almost invisible

Figure 4. Faults made by the data engine on one base page (a generated ColorOS 17 page); each copy changes one thing in the page code and carries an exact label (amber corner marks). A component can be transplanted, with all its styles, from a page of another design system (the iOS 26 tab bar keeps the dark glass of its own dark-mode page); general faults are wrong in any design. Table 4. Consistency test (2,765 screenshots, 607 of them clean: 120 real and 487 generated; no target; same prompt; local models greedy without thinking, API judges at default reasoning). Consistency recall: swapped components and local style faults. The gate: at most 15% of real and of generated clean pages flagged; only ProofJudge with the cut passes it among fault-finding judges. No answer: unparsable, scored as finding nothing. Cut: a finding is kept only when the verdict’s probability of a clean page is below a threshold chosen on validation. Seconds per page: ProofJudge (rank-16 adapter) on one A100-80GB, open VLMs on A100-40GB, Gemini via API. Tinted: over half of the clean pages flagged. Clean pages flagged Judge

real ↓

generated ↓

API judges Gemini 4 Argon 18% 18% gpt-6.1-sol 38% 36% Claude Opus 5.5 (medium) 62% 82% Gemini 3.8 Flash 38% 66% GPT-6 Luna 50% 45% ProofJudge (ours, Qwen3.5-9B + LoRA) ProofJudge (ours) 2% 9% Qwen3.5-9B base 83% 89% Open VLMs, zero-shot Qwen3.8-27B 98% 88% InternVL3.5-38B 95% 99% 64% 25% GLM-4.6V (106B MoE) InternVL3.5-8B 93% 94% Mistral-Small-3.2-24B 99% 100% Qwen3-VL-32B 79% 36% Qwen3-VL-30B-A3B 27% 28% Qwen3-VL-8B 23% 0% UI-TARS-1.5-7B (GUI agent) 8% 0%

Recall

Speed

general ↑

consistency ↑

Precision ↑

F1 ↑

F1.3 ↑

No answer ↓

s/page

GPUs

38% 45% 49% 40% 31%

41% 32% 49% 26% 17%

35% 25% 22% 22% 20%

0.37 0.30 0.31 0.26 0.22

0.43 0.36 0.35 0.29 0.27

0 3 0 0 1

6.1 – – 20.6 –

– – – – –

59% 21%

25% 33%

62% 2%

0.48 0.04

0.50 0.05

0 282

1.0 –

1 –

29% 19% 14% 13% 2% 26% 8% 2% 0%

20% 17% 6% 27% 4% 28% 8% 0% 0%

11% 3% 5% 2% 0% 3% 2% 3% 0%

0.15 0.05 0.07 0.04 0.01 0.05 0.03 0.02 0.00

0.17 0.06 0.08 0.05 0.02 0.06 0.04 0.02 0.00

0 16 10 182 7 199 629 14 4

4.83 4.93 0.71 7.45 6.55 8.64 4.24 1.67 0.76

2 4 8 1 2 4 2 1 1

Text layout. On the text layout test (Tab. 5), Gemini 4 Argon finds 47% and gpt-6.1-sol 54% of the faults: nearly every mid-word break and needless ellipsis, but few labels with touching characters, while flagging 56% and 68% of the hard negatives, where all siblings share the same wide spacing. Both find most broken words (86%, 100%) but few lone last words (22%, 25%) and few labels whose characters

radius (Gemini 4 Argon 53%, gpt-6.1-sol 47%) at least as often as a general fault (38%, 45%), but rarely a foreign icon (15%, 11%). Many local faults are subtle: a colour fault changes a thin accent inside a large control (a median 2.6% of the box’s pixels, ∆E 1.0 over the box), and many icon faults look normal on their page, foreign only by a design language’s conventions (Sec. I).

6

engine label: the fault

ProofJudge

Gemini 4 Argon

gpt-6.1-sol

One material off Google Material 3 Expressive (Android 16) · no target given

Shrunk tap target Apple iOS 26 (Liquid Glass) · no target given

Local fault: one element's material (blur, shadow, surface) differs from the same elements on its page

Small tap target: a control too small to tap

Original

ProofJudge

Gemini 4 Argon

gpt-6.1-sol

Original

ProofJudge

Gemini 4 Argon

gpt-6.1-sol

1 fault

✓ found · 1 box

✗ missed · 0 boxes

✗ missed · 0 boxes

1 fault

✓ found · 1 box

✗ missed · 0 boxes

✗ missed · 0 boxes

Clean real screenshot OPPO ColorOS 17 · no target given

Missed by all Google Material 3 Expressive (Android 16) · no target given

Clean page: nothing to find, so every box is a false alarm

Local fault: one element's material (blur, shadow, surface) differs from the same elements on its page

Original

ProofJudge

clean, real

✓ silent · 0 boxes

Gemini 4 Argon

gpt-6.1-sol

✗ false alarm · 2 boxes ✓ silent · 0 boxes

Original

ProofJudge

Gemini 4 Argon

gpt-6.1-sol

1 fault

✗ missed · 0 boxes

✗ missed · 0 boxes

✗ missed · 0 boxes

Figure 5. Where ProofJudge succeeds and fails on the consistency test (no target given): four hand-picked cases, three of its strengths and one fault every judge misses; Tab. 4 gives the overall picture, where ProofJudge has the highest F1 but Gemini 4 Argon finds more local style faults. Under each title, one line names the fault; each example shows the original screenshot with the engine’s exact label (amber corner marks), then ProofJudge and two API judges with only their own boxes. ProofJudge alone finds the room card whose surface differs from its sibling’s and a control shrunk below tap size (IoU ≥ 0.5), whereas the API judges’ boxes, if any, lie elsewhere; on a clean real screenshot it stays silent where Gemini 4 Argon boxes a timeline and an empty area. No judge finds the air-quality card that has lost the surface its sibling cards keep. Twenty cases chosen by a fixed rule, including failures, are in Sec. R.

Against a named brand. Naming a target brand makes the task harder (Sec. D): models know a design language about as well as its public documentation states it (91–99% on Material 3 Expressive, 14–32% on MagicOS 11), and frontier critics with a target flag 54–89% of clean pages.

touch (17%, 27%); at the weakest strengths they find almost none (characters touch at −0.08 em: 3%, 9%). Claude Opus 5.5 at medium effort finds the most (72%, characters touching 73%) but boxes almost every page: 80% of the clean pages and 96% of the hard negatives. ProofJudge finds 43%: more needless ellipses than any API judge (96%) and most broken words (94%), but few labels whose characters touch (5%); it flags 15% of the clean pages and 13% of the hard negatives, fewer than any API judge. Scores at IoU ≥ 0.3 change little (Argon 51%, Sol 56%).

6. ProofJudge: a Small Consistency Check Model and training. ProofJudge is Qwen3.5-9B (thinking disabled) with a LoRA of rank 128 on all linear languagemodel layers and a frozen vision encoder (Fig. 3). It outputs JSON with a verdict and findings, each giving a box in 0– 1000 screen coordinates, the element, the fault family and

7

Easy: one colour or material off. Hard: one icon off. faults found on the consistency test (no target), kinds from easiest to hardest

Colour

50% ours 41%

local style · 179 faults

Material

54% ours 44%

local style · 123 faults

General fault

45% ours 59%

general · 920 faults

Radius

53% ours 41%

local style · 86 faults

Foreign component

43% ours 20%

component · 662 faults

Icon, visible

26% ours 16%

local style · 19 faults

Icon, looks normal

local style · 43 faults

0%

ours 0%

0%

25%

50%

best API judge

75%

ProofJudge (ours)

100%

Figure 6. What is easy and what is hard (consistency test, 2,765 screenshots, no target). Each row is one kind of fault, ordered by the mean share the four API judges find, easiest first; the bar is the share the best of them finds (Gemini 4 Argon, gpt-6.1-sol, Gemini 3.8 Flash, GPT-6 Luna), the dot the share ProofJudge finds. One element with another colour, material or corner radius is found by the best API judge about half the time; a foreign icon is hard for every judge, and none of the four nor ProofJudge finds an icon that looks normal on its page (Claude Opus 5.5, which flags most clean pages, finds 3 of 43; Tab. S6). ProofJudge finds the most general faults (59%). Every judge’s numbers are in Tab. S6, the split of icon faults in Sec. I. Table 5. Text layout test (held-out page requirements, no target, IoU ≥ 0.5): recall per rule, and pages flagged among 75 clean pages and 71 hard negatives (every sibling changed alike). 367 faults: characters touch 66, letter spacing unlike siblings 46, lines touch 20, line height unlike siblings 17, short label wraps 23, lone last word or character 73, word broken inside 50, needless ellipsis 72. Spacing: characters touch, letter spacing wider than the siblings’; Lines: lines touch, line height larger than the siblings’; Breaks: a short label wraps, a lone last word or character, a word broken inside, a needless ellipsis; Flagged: clean pages and hard negatives flagged. The base pages are English, so the Chinese line-start punctuation rule has no test item; “–”: not run. Judge

Gemini 4 Argon gpt-6.1-sol Claude Opus 5.5 (medium) ProofJudge (ours)

Spacing ↑

Lines ↑

Breaks ↑

All ↑

touch

wide

touch

tall

wrap

lone

word

ellipsis

17% 27% 73% 5%

41% 59% 80% 46%

60% 70% 70% 5%

18% 35% 35% 0%

43% 26% 43% 26%

22% 25% 55% 14%

86% 100% 98% 94%

81% 81% 82% 96%

47% 54% 72% 43%

Flagged ↓ clean

neg.

32% 49% 80% 15%

56% 68% 96% 13%

a reason, requiring no detection head. Our main model receives no target. It is trained by SFT on the 17,285 rows of ScreenProof-Train (42% clean; Tab. 3), with one to four independent faults per page (general, display, local style and text layout); no test or validation page requirement is in its data (Sec. P). Two runs that differ only in the seed land up to 0.04 F1 apart on validation, so we train twice and average the two adapters (a weight average of their updates), each at its final step. A finding is kept only when the model’s probability that the page is clean, read from its verdict, is below 0.01099; the cut was chosen on held-out validation screens, never on the test (tested variants: Sec. P). Earlier runs with a brand target (v5) or one fault per page (v6) are in Secs. M and N.

Table 6. Capacity and seeds (v7f data, two epochs, each run at its final step 2,570; LoRA α = 2 × rank). Validation: F1 with each model’s own cut on the validation split. Pair: the average of the two seeds’ adapters. Test: the 2,765-page consistency test with the validation cut. Rank 64 was tested and rank 256 validated once on the training machine; their raw answers are not released. † The rank-16 model tested is one run, step 2,500, checkpoint and cut chosen on validation (the paper’s model before this one).

A consistency check. A 9B model does not learn eight design languages from public pages, but it can compare an element against its own page, so ProofJudge asks only which

elements are wrong on the page itself. On the consistency test (Tab. 4 and Fig. 5) it reaches F1 0.48, the highest of all judges, ahead of Gemini 4 Argon (0.37; difference +0.109,

8

LoRA rank

seed 42

seed 7

pair

test F1

95% interval vs rank 16

16 64 128 (ours) 256

0.556 0.561 0.586 0.605

0.517 0.550 0.598 0.576

0.550 0.571 0.607 0.599

0.405† 0.445 0.481 –

– [+0.025, +0.055] [+0.060, +0.091] –

real screenshots gate 15% 18% 18%

Gemini 4 Argon

38% 36%

gpt-6.1-sol

62%

Claude Opus 5.5 38%

Gemini 3.8 Flash

Judge

66%

2%

83% 89% 64%

25%

Qwen3.8-27B 0%

88%

25%

50%

75%

F1 ↑

Normal flagged ↓

API judges Gemini 4 Argon 0.86 28% Gemini 3.8 Flash 0.75 64% gpt-6.1-sol 0.76 63% Claude Opus 5.5 (medium) 0.79 53% ProofJudge (ours) ProofJudge (ours) 0.72 25% Qwen3.5-9B base 0.64 97% Open VLMs, zero-shot GLM-4.6V 0.74 66% Qwen3-VL-8B 0.73 33% Qwen3.8-27B 0.66 100% six other open VLMs (7B–38B): F1 0.05–0.66

9%

Qwen3.5-9B base GLM-4.6V

OwlEye

82%

50% 45%

GPT-6 Luna ProofJudge (ours)

Table 7. External benchmark (consistency prompt unchanged). OwlEye: 600 Android screenshots, half with display issues; flagging all scores F1 0.67. Full table: Tab. S5.

generated pages

98%

100%

clean pages flagged (lower is better)

Figure 7. Clean pages flagged on the consistency test (no target; 607 clean pages: 120 real, 487 generated), for each judge one bar for clean real screenshots (dark) and one for clean generated pages (light). The dashed line is the 15% gate; among judges that find faults, only ProofJudge stays within it on both.

Accuracy ↑ 84% 68% 68% 73% 73% 48% 66% 72% 50%

Cost and speed. On one A100-80GB, ProofJudge (timed with an earlier rank-16 adapter) takes 1.0 s per page and, at 16 concurrent requests and an assumed $2 per GPU-hour, costs about $0.04 per 1,000 pages, versus 6.1 s and $4.42 for Gemini 4 Argon (Fig. 8), while keeping private files in house.

95% interval [+0.086, +0.132] over 10,000 paired page resamples), Claude Opus 5.5 (0.31) and gpt-6.1-sol (0.30), and far above its 9B base (0.04) and the open VLMs (0.00–0.15). It is the only fault-finding judge within the 15% gate, flagging 2% of clean real screenshots and 9% of clean generated pages (Argon: 18% and 18%), with the highest precision (62%; Argon 35%). It finds the most general faults (59%; API judges 31–49%) but fewer local style faults than the strongest API judges (25%; Argon 41%), and it stays ahead at IoU ≥ 0.3. The cut keeps it quiet on clean pages: without it the same model flags 25% and 58% of clean pages, yet its F1 (0.46) is still above every other judge, so the lead does not come from the cut alone. Capacity does the rest: the same data with a rank-16 LoRA (one run, its checkpoint and cut chosen on validation) flags 3% and 8% with the cut and reaches F1 0.40; F1 rises from rank 16 to 128 and stops there (Tab. 6).

A cheap first check. In front of an API judge, ProofJudge removes most false alarms (Fig. 9; consistency test). When gpt-6.1-sol looks only at the 42% of pages that ProofJudge flags, box F1 rises from 0.30 to 0.32 and precision from 25% to 36%, while recall falls from 37% to 28%; clean pages flagged fall from 38% to 1% of real screenshots and from 36% to 4% of generated pages, with 416 instead of 1,000 API calls per 1,000 pages. Behind the same gate Gemini 4 Argon flags 1% and 2% of clean pages at $1.88 instead of $4.42 per 1,000 pages, but its F1 falls from 0.37 to 0.33, below ProofJudge alone. As a second opinion, ProofJudge finds 328 faults that Argon misses, most of them general: together they reach recall 55% (Argon 40%) and F1 0.44, above Argon but below ProofJudge alone, and flag 20% of clean real screenshots and 24% of clean generated pages.

External benchmark. On OwlEye’s real Android screenshots [18] (Tab. 7), where flagging all scores F1 0.67, ProofJudge reaches F1 0.72 and flags 25% of normal screenshots (accuracy 73%), fewer than any API or open judge with recall above 50%; Gemini 4 Argon leads (F1 0.86).

7. Find, then Fix A critic is useful if its findings help a generator repair the page. We test this where a repair can be checked exactly (Sec. Q): 85 consistency-test pages with one local fault whose right value the page itself shows (one corner radius, accent or surface unlike its siblings), and 40 clean generated pages, on which a false finding should change nothing.

Do engine-made faults predict real ones? Over all judges scored on both (Spearman ρ), a judge’s rank on the consistency test predicts its rank on OwlEye’s real screenshots: ρ = 0.58 between the two F1 scores over 21 judges and variants, and ρ = 0.91 between the shares of clean real screenshots and of OwlEye’s normal screens flagged. Engine-made faults rank critics as real screens do on whether a page is clean.

Fixer and hints. gpt-6.1-sol gets the page’s HTML, its screenshot and one hint: none (fix whatever does not match 9

(a) Consistency F1 ↑

(b) Clean pages flagged ↓

0.6

100%

0.5

ProofJudge (ours) · F1 0.48

0.4

Gemini 4 Argon · F1 0.37

1 s/page · $0.04/1k

75%

6.1 s/page · $4.43/1k

0.3

50%

Gemini 3.8 Flash · F1 0.26

20.6 s/page · $12.4/1k

0.2

25%

0.1 0.0 $0.01

$0.1

$1

Gemini 3.8 Flash · gen. 66%

$10

real 38% Gemini 4 Argon · real 18%, gen. 18% gate: both ≤15%

0% $0.01

$100

$/1,000 pages · log scale

ProofJudge (ours) · real 2%, gen. 9%

$0.1

$1

$10

$100

$/1,000 pages · log scale

Figure 8. Cost against accuracy, the same cost scale in both panels. (a) F1 on the consistency test (2,765 pages, no target given), with time per page and cost per 1,000 pages. (b) Clean pages flagged: real screenshots (filled) and generated pages (open); a checkpoint is usable only below the dashed line on both. ProofJudge is the cheapest by two orders of magnitude and flags the fewest clean pages, real and generated. ProofJudge’s time was measured on v6, with the same base model and a smaller adapter (rank 16). ProofJudge’s cost assumes $2 per A100-hour; the API costs are list prices on 2026-10-25.

gpt-6.1-sol on every page

box F1 ↑ 0.32

0.30

ProofJudge first, Sol on the 42% it flags

API calls per 1,000 pages ↓

A judge’s findings help only as far as they are right: under “change nothing else” a finding on the wrong element keeps the fixer from the real fault, so Gemini 4 Argon’s findings lower the fix rate to 55%. ProofJudge is quiet on these subtle local faults (Fig. 6): its findings let the fixer repair 25% and harm 0% of clean pages (Argon: 15%). As a gate, Gemini 4 Argon sends 56% of the pages and 72% of the faults are fixed; ProofJudge sends 22%, harms 5% of clean pages, and lets only 28% of the faults be fixed (Sec. Q). The value of a critic for repair therefore lies both in staying silent on clean pages and in finding the small faults, and no judge does both yet.

clean pages flagged ↓

1,000 38%

36%

416 15%

1% recall 37% → 28%

precision 25% → 36%

real

4% generated

dashed: the 15% gate

Figure 9. ProofJudge as a cheap first check (consistency test, no target). gpt-6.1-sol runs either on every page or only on the 42% of pages ProofJudge flags: box F1 0.30 to 0.32, precision 25% to 36%, recall 37% to 28%, and clean pages flagged 38% to 1% (real) and 36% to 4% (generated; dashed line: the 15% gate). Sol reports no token counts, so the saving is shown in API calls; the same gate with Gemini 4 Argon is in the text.

Cost. Per 1,000 pages, fixing every page costs about $80, an Argon gate $49 and a ProofJudge gate $17 (a fixer call about $0.080), paid for by the faults each gate misses; without code (a screenshot or an installed app) the judge’s location matters more (untested).

the rest of the page, or return it unchanged), the true finding, or a judge’s findings with the instruction to fix the reported problems that are real and change nothing else. As a gate, a judge only decides whether a page goes to the fixer, which then works with no hint. A fault is fixed when the pixels inside its box that differ from the clean page fall to a quarter or less; a page is harmed when more than 1% of the pixels outside the box, or anywhere on a clean page, change.

8. Limitations and Conclusion Limitations. We study single phone screens without interaction or motion. Clean labels rest on generated base pages that two judges found on-spec. Scores come from model judges, and most earlier ProofJudge variants were trained once (the capacity runs and the paper’s model use two seeds). The text layout test uses English pages only, and in the fix test the fixer reads the page’s code, which a screenshot or an installed app does not offer.

Repair is easy; knowing when to repair is not. With the page’s code and no hint, the fixer fixes 95% of the faults and never harms the rest of a faulty page, but it also changes 12% of the clean pages. Only the true finding gives both a high fix rate (95%) and no observed harm to the 40 clean pages (0%).

Conclusion. No critic yet boxes every fault on a screen without a reference: critics flag many clean pages, an ele-

10

ment that breaks its own page’s style often escapes every judge, and text layout faults are found unevenly. A small trained critic has the highest F1 on the consistency test and is the only fault-finding judge quiet on clean pages, yet it misses many local style and text layout faults and lags on real screens. Its potential use is a cheap first pass over generated pages, before a human review: it passes most clean pages and points to the element to fix. Whether this saves designers time is untested; a study with designers is future work.

compatibility in MLLM-generated webpages. arXiv preprint arXiv:2608.12518, 2026. 2, 3 [11] Wenyi Hong, Weihan Wang, Qingsong Lv, Jiazheng Xu, Wenmeng Yu, Junhui Ji, Yan Wang, Zihan Wang, Yuxuan Zhang, Juanzi Li, Bin Xu, Yuxiao Dong, Ming Ding, and Jie Tang. CogAgent: A visual language model for GUI agents. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024. 3, 13 [12] Jie Huang, Xinyun Chen, Swaroop Mishra, Huaixiu Steven Zheng, Adams Wei Yu, Xinying Song, and Denny Zhou. Large language models cannot self-correct reasoning yet. In Proceedings of the International Conference on Learning Representations (ICLR), 2024. 3 [13] Yerin Hwang, Dongryeol Lee, Kyungmin Min, Taegwan Kang, Yong-il Kim, and Kyomin Jung. Fooling the LVLM judges: Visual biases in LVLM-based evaluation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), 2025. 2 [14] Amit Jadhav, Shaurya Beriwala, and Beomjin Kim. Feed the panel dimensions, not verdicts: Rubric-decomposed fusion of vision-language aesthetic judges. arXiv preprint arXiv:2609.27110, 2026. 2 [15] Jaehyun Jeon, Min Soo Kim, Janghan Yoon, Sumin Shim, Yejin Choi, Hanbin Kim, Dae Hyun Kim, and Youngjae Yu. Do MLLMs capture how interfaces guide user behavior? A benchmark for multimodal UI/UX design understanding. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL), pages 44269–44294, 2026. 3 [16] Mohammed Safi Ur Rahman Khan, Sanjay Suryanarayanan, Tushar Anand, and Mitesh M. Khapra. Seeing isn’t believing: Uncovering blind spots in evaluator vision-language models. arXiv preprint arXiv:2604.21523, 2026. 2, 4 [17] Qiming Li, Shujie Hu, Haohan Liu, Xiaocheng Feng, Songxiang Liu, and Guanglu Wan. MT-Web2Code: Benchmarking coding agents on multi-turn regional reconstruction and localized modification. arXiv preprint arXiv:2608.03474, 2026. 2, 3 [18] Zhe Liu, Chunyang Chen, Junjie Wang, Yuekai Huang, Jun Hu, and Qing Wang. Owl eyes: Spotting UI display issues via visual understanding. In Proceedings of the IEEE/ACM International Conference on Automated Software Engineering (ASE), pages 398–409, 2020. 3, 9 [19] Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Shashank Gupta, Bodhisattwa Prasad Majumder, Katherine Hermann, Sean Welleck, Amir Yazdanbakhsh, and Peter Clark. Self-refine: Iterative refinement with self-feedback. In NeurIPS, 2023. 3 [20] Ruichao Mao, Zhou Fang, Teng Guo, Hao Yang, Yaping Li, Shaohua Peng, Maji Huang, Xiaoyu Lin, Shuoyang Liu, Xuepeng Li, Yuyu Zhang, and Hai Rao. Reasoning for mobile user experience with multimodal LLMs: Task, benchmark, and approach. In Findings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2026. arXiv:2606.13192. 3 [21] Yue Peng, Lanke Xia, Zihan Wang, Jiahao Ye, Ke Ning, and Hongyi Wen. EvoGenUI-Bench: Evaluating LLMs

References [1] Gilles Baechler, Srinivas Sunkara, Maria Wang, Fedir Zubach, Hassan Mansoor, Vincent Etter, Victor Cărbune, Jason Lin, Jindong Chen, and Abhanshu Sharma. ScreenAI: A visionlanguage model for UI and infographics understanding. In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI), 2024. 3, 13 [2] Nicolas Bougie, Xiaotong Ye, Gian Maria Marconi, and Narimasa Watanabe. PerceptUI: LLM agents as humanaligned synthetic users for UI/UX evaluation. arXiv preprint arXiv:2606.05697, 2026. 3 [3] Sara Bunian, Kai Li, Chaima Jemmali, Casper Harteveld, Yun Fu, and Magy Seif El-Nasr. VINS: Visual search for mobile user interface design. In CHI, 2021. 3, 13 [4] Seungeon Cha, Sunghyun Jo, Jongho Shin, and Kyoungwon Seo. Design system-compliant user interface generation with LLM agents: A comparative study of context engineering strategies. In Extended Abstracts of the CHI Conference on Human Factors in Computing Systems, 2026. 3, 4 [5] Ruoxi Chen, Dongping Chen, Siyuan Wu, Sinan Wang, Shiyun Lang, Petr Sushko, Gaoyang Jiang, Yao Wan, and Ranjay Krishna. MultiRef: Controllable image generation with multiple visual references. In Proceedings of the ACM International Conference on Multimedia (Datasets Track), 2025. 3, 4 [6] Kanzhi Cheng, Qiushi Sun, Yougang Chu, Fangzhi Xu, Yantao Li, Jianbing Zhang, and Zhiyong Wu. SeeClick: Harnessing GUI grounding for advanced visual GUI agents. In ACL, 2024. 3, 13 [7] Biplab Deka, Zifeng Huang, Chad Franzen, Joshua Hibschman, Daniel Afergan, Yang Li, Jeffrey Nichols, and Ranjitha Kumar. Rico: A mobile app dataset for building datadriven design applications. In Proceedings of the ACM Symposium on User Interface Software and Technology (UIST), 2017. 3, 13 [8] Peitong Duan, Chin-yi Chen, Gang Li, Bjoern Hartmann, and Yang Li. UICrit: Enhancing automated design evaluation with a UI critique dataset. In Proceedings of the ACM Symposium on User Interface Software and Technology (UIST), 2024. 3 [9] Boyu Gou, Ruohan Wang, Boyuan Zheng, Yanan Xie, Cheng Chang, Yiheng Shu, Huan Sun, and Yu Su. Navigating the digital world as humans do: Universal visual grounding for GUI agents. In Proceedings of the International Conference on Learning Representations (ICLR), 2025. 3, 13 [10] Ziyun Guo, Jingyu Xiao, Yuqiang Sun, and Yintong Huo. Does it render everywhere? A study of cross-environment

11

as multi-turn generative UI assistants. arXiv preprint arXiv:2608.29387, 2026. 3 [22] Zhiliang Peng, Wenhui Wang, Li Dong, Yaru Hao, Shaohan Huang, Shuming Ma, and Furu Wei. Kosmos-2: Grounding multimodal large language models to the world. arXiv preprint arXiv:2306.14824, 2023. 13 [23] Chengwei Shi, Yunnong Chen, Tingting Zhou, Qiang Lu, Shiyu Yue, Xinyuan Hu, Jianfang Ru, and Liuqing Chen. TaoD2C-Bench: Benchmarking MLLMs for industrial UI code generation beyond visual fidelity. arXiv preprint arXiv:2610.10374, 2026. 3 [24] Chenglei Si, Yanzhe Zhang, Ryan Li, Zhengyuan Yang, Ruibo Liu, and Diyi Yang. Design2Code: Benchmarking multimodal code generation for automated front-end engineering. In Proceedings of the Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics (NAACL), 2025. 2, 3, 4 [25] Yuxuan Wan, Chaozheng Wang, Yi Dong, Wenxuan Wang, Shuqing Li, Yintong Huo, and Michael R. Lyu. Automatically generating UI code from screenshot: A divide-andconquer-based approach. In Proceedings of the ACM International Conference on the Foundations of Software Engineering (FSE), 2025. 3 [26] Kuang-Da Wang, Zhao Wang, Yotaro Shimose, Wei-Yao Wang, and Shingo Takamatsu. WebGen-V Bench: Structured representation for enhancing visual design in LLM-based web generation and evaluation. arXiv preprint arXiv:2510.15306, 2025. 3 [27] Wenjie Wang, Yue Huang, Zipeng Ling, Han Bao, Hang Hua, Xiaonan Luo, Yu Jiang, Shiyi Du, Yuexing Hao, Xiaomin Li, Yuchen Ma, Dianzhuo Wang, Yanfang Ye, and Xiangliang Zhang. UXBench: Measuring the actionability of LLMgenerated UX critiques. arXiv preprint arXiv:2606.16262, 2026. 3 [28] Fan Wu, Cuiyun Gao, Shuqing Li, Xin-Cheng Wen, and Qing Liao. MLLM-based UI2Code automation guided by UI layout information. In Proceedings of the ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA), 2025. 3 [29] Fan Wu, Cuiyun Gao, Yiming Huang, Yang Xiao, Yujia Chen, and Qing Liao. Looks right, works right: A project-level benchmark for multi-screen mobile app generation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), 2026. arXiv:2607.28645. 2, 3, 4 [30] Wei Xiang, Yexinrui Wu, Xinli Chen, Xinran Li, and Shi Chen. UI-Lens: Assessing general MLLMs’ potential to automate UI display quality assurance. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 25882–25892, 2026. 3, 4 [31] Jingyu Xiao, Ming Wang, Man Ho Lam, Yuxuan Wan, Junliang Liu, Yintong Huo, and Michael R. Lyu. DesignBench: A comprehensive benchmark for MLLM-based front-end code generation. arXiv preprint arXiv:2506.06251, 2025. 2, 3 [32] Jingyu Xiao, Jiantong Qin, Shuoqi Li, Man Ho Lam, Yuxuan Wan, Jen-tse Huang, Yintong Huo, and Michael R. Lyu. ComUICoder: Component-based reusable UI code generation

for complex websites via semantic segmentation and elementwise feedback. arXiv preprint arXiv:2602.19276, 2026. 2, 3, 4 [33] Zhen Yang, Wenyi Hong, Mingde Xu, Xinyue Fan, Weihan Wang, Jiale Cheng, Xiaotao Gu, and Jie Tang. UI2CodeN : UIto-code generation as interactive visual optimization. arXiv preprint arXiv:2511.08195, 2025. 2, 3, 4 [34] Keen You, Haotian Zhang, Eldon Schoop, Floris Weers, Amanda Swearngin, Jeffrey Nichols, Yinfei Yang, and Zhe Gan. Ferret-UI: Grounded mobile UI understanding with multimodal LLMs. In ECCV, 2024. 3, 13

Supplementary Material A. A short primer on the tasks This section explains the benchmark for readers outside UI design; the main paper gives the formal definitions (Sec. 3). Design languages. Every phone maker publishes, or keeps in house, a design language: the rules that make its apps look like one family. They fix how controls look (a switch, a tab bar, a search field), the accent colour and how it is used, the corner radius of cards and buttons, the material of surfaces (blur, glass, shadow), the icon set and the type scale. The rules change between versions, so a target is always a brand and a version, for example iOS 26 (Liquid Glass) or ColorOS 17. A screen that ignores them still works, but looks foreign, like an Android switch inside an iOS app. We cover eight languages: Google’s Material 3 Expressive and the systems of seven phone makers (Fig. S1). Three kinds of errors. Three families of faults need no target. A general fault is wrong under any design language: text that runs off the screen, two elements that overlap, a button too small to tap, text too faint to read. A local style fault is visible without knowing any brand: one element differs from the elements of the same kind on its own page, for example one button with sharp corners on a page of round ones, or a component comes from another design system. A text layout fault breaks a rule of type setting, such as a word broken inside or a needless ellipsis (Tab. 2). A brand fault needs a named target and is only in Sec. D. Task 1, knowledge: does the model know the rules? Without any screenshot, the model answers multiple-choice questions written from the official guidelines, such as which brand prescribes this corner radius for cards? (one of eight) or which of these four statements is true for MagicOS 11? (one of four). Picture questions then ask the same on screenshots: which brand’s rule a screenshot shows, which brand a red-boxed element comes from, and which design dimension of a page breaks a given target. A model that cannot answer these cannot be expected to judge a page. 12

Task 2, critique: can the model point at what is wrong? The model gets one screenshot and no target. For every problem it draws a box around the element, names its family (general, local style or text layout) and says what is wrong. This is what a design reviewer does. Because the faults are made by our data engine in the page’s code, we know exactly which element is wrong and where, so the answer is scored automatically: a box counts as found when it is matched one to one to a true element with intersection over union ≥ 0.5 (≥ 0.3 as a lenient variant, Sec. K). Recall is the share of true faults found, precision the share of drawn boxes that are right, F1 combines the two, and a false alarm is a “faulty” verdict or any box on a clean page, which has nothing to find. A useful critic needs both: a model that boxes everything finds every fault and is useless to a designer.

UI understanding and visual grounding. UI datasets such as Rico [7] and VINS [3] provide screens and component boxes, and screen models such as ScreenAI [1] and Ferret-UI [34] understand and ground UI elements. GUI agents ground instructions to screen locations [6, 9, 11], building on grounded MLLMs [22]. They locate an element that matches a description; we locate elements that are wrong on their own page or under any design, without a reference, and name the fault.

C. Dataset Public part: brand screenshots. We searched each brand’s own sites for phone screenshots: product and release pages, user guides, newsroom images and design-guideline sites. Every image found is recorded in a manifest with its source page, date of collection, terms of use, size and hash. An image is kept only if it is a single portrait phone screen; composites of several screens or devices, landscape images and images whose content is not a phone UI are set aside. Each kept screenshot is labelled with the system version that its page title or URL names (for example a version-pinned iPhone User Guide URL); when the page names no version, the screenshot is kept with version unknown and is not used where a version is required. In the snapshot of 23 October 2026 (from a frozen manifest copy identified by its SHA256; collection continues) the manifest held 8,755 images from brand sites, design guides, open-source repositories and videos, of which 458 were accepted, 352 with a named version (Fig. S2); 4,980 were composites, 1,852 were not portrait, 1,325 were rejected and 140 were still pending at the snapshot. Most accepted screenshots come from official sites (320) and design guides or developer documents (81); 24 are video frames, all of vivo’s and Honor’s among them. Figure S1 shows one accepted screenshot per brand at the benchmark’s target version.

Task 3, design: can the model build to the rules? The model sees a real screenshot of a brand’s app and rebuilds it as a web page, with only the brand name or also with a short public summary of the brand’s rules (a spec card). Two judge models score the rebuild item by item: is every element of the original there, in the right place and style (likeness), and does the page follow the card (specification), quoting the card for every violation. The data engine and the quality gate. Labelling problems on real screenshots by hand is slow and uncertain. We instead start from good pages and break exactly one thing in their code: transplant a component from a page of another design system, change every card’s radius, make one title overflow, or give one button another radius than its siblings (Fig. 4). The change is the label. Before a page is used, two judges check that it follows its brand’s card; pages that fail this quality gate are dropped, because a page that is already off-brand cannot be a clean example. Data. All data (pages we generate and screenshots collected from the web) are released.

Public part: other sources. Screens from public UI datasets are recorded in the same manifest with their licences (3,327 files at the time of writing, mostly from iOS1K, Design2Code and Sketch2Code); spec cards summarise each version’s official design guidelines in seven fixed fields (shape, colour, material, navigation, controls, type, spacing), each fact with the URL of the official page it comes from and nothing filled in from memory or press; a field the official pages do not state is marked not specified (4 such items for iOS 26, 1 for Material 3 Expressive, 8 for ColorOS 17, whose only public specification covers home-screen cards); the other five cards are written the same way, and since vivo, Xiaomi and Honor publish only partial official documents, most of their fields are not specified; pages built with opensource UI libraries (e.g. Framework7) and their fault copies are generated by us.

ProofJudge. ProofJudge is our small baseline critic: a 9-billion-parameter open vision-language model (Qwen3.59B) that we fine-tune on engine copies to answer the critique task in the same box format (Fig. 3). It is meant to be cheap enough to run on every page in-house; the main paper reports where it is and is not good enough. The paper’s model is v7f, trained by SFT without a target (Sec. P).

B. Positioning against prior work Table 1 compares the works discussed in the related work of the main paper, feature by feature.

13

iOS 26

Material 3E

HarmonyOS 7

One UI 9

HyperOS 4

OriginOS 7

ColorOS 17

MagicOS 11

world clock apple.com

clock material.io

weather bilibili video

quick settings youtube video

controls mi.com

lock screen bilibili video

assistant oppo.com

lock screen bilibili video

Figure S1. One accepted web screenshot per design language at the benchmark’s target version. Each label gives the version, the screen and its source; frames from videos are marked. Screens keep the language they were published in.

target version

iOS Material HarmonyOS One UI

older versions

58 5

17

77

10

HyperOS

38

7

OriginOS

3

ColorOS

24

MagicOS

7

151

64

33

36

16

brand words removed), recognises them on real screenshots (name the brand and version; find the odd screen out of four), and applies them to images (which brand’s rule a screenshot shows; which brand a red-boxed swapped element follows; which design dimension of a page breaks a target). A general probe asks the same of brand-free design knowledge, in words and on a red-boxed defect. Critique is the localization task of the main paper with a named target added, on engine copies with exact labels. Design asks a generator to restore a real screenshot of a brand’s current version as HTML, given the brand name only or the public card; two judges score likeness to the original (content, layout and style, checked against an element inventory) and to the specification (card fields followed, each violation quoting the card). Every task reports general and brand errors separately. The test sets hold 1,316 text questions from 658 facts of the official guidelines, 680 picture-and-text items and 101 general-design items for knowledge; 643 screenshots (147 clean) for critique against a named brand and 2,765 (607 clean; the first version of this test had 576 pages) for the consistency question; and 47 real screens for design.

9 80

54 26

3

version unknown

51 43 3

32

59

Figure S2. Accepted web screenshots per design language (snapshot of 23 October 2026): the target version, older versions, and screenshots whose page names no version. 458 in all.

Release. ScreenProof is released in full, images included, with the source, date and terms of use of each item.

D. Brand conformance The main paper asks the target-free question. This section reports the brand tracks of ScreenProof: a judge or generator is given a named target T = (b, v), a brand and its version, e.g. iOS 26 Liquid Glass or ColorOS 17, and a brand error is wrong only with respect to that target. A useful brand review must say which convention of which design language an element breaks; the split into general and brand errors yields a testable property: when the same screenshot is judged against a different brand, general findings should stay and brand findings should change.

Targets. A brand target always specifies a brand and version, because a brand’s versions differ in materials, corner radii, navigation and accent use; Fig. S3 shows how much a single page changes across targets. The benchmark covers seven main brands: Apple iOS, Huawei HarmonyOS, OPPO ColorOS, vivo OriginOS, Xiaomi HyperOS, Samsung One UI and Honor MagicOS, each in its latest released version (iOS 26, HarmonyOS 7, ColorOS 17, OriginOS 7, HyperOS 4, One UI 9 and MagicOS 11), alongside Material 3 Expressive.

Three tasks. Knowledge asks what a model knows of each design language, without a page to judge: it states conventions (text-only multiple choice built from public spec cards,

Kinds of errors. Main Table 2 splits target-free faults into general, local style and text layout faults; brand faults, which 14

One requirement

+ a public style card → the target page iOS 26

Today 426 kcal · 38 min 7,812 steps

Liquid Glass

navigation

from the content

Aa

SF Pro 17 pt

44 pt

colour

type

touch

Material 3 Expressive

Google

ColorOS 17

Aa

48 dp

not stated

type

touch

navigation

Roboto 16 sp

navigation

colour

OPPO

Aa

not stated

type

touch

OPPO Sans

colour

Start Workout

Recent Morning Run Strength Training Cycling

Home

Plans

Profile

fitness home, given as text; drawn here as a plain wireframe

Figure S3. One requirement (text, drawn as a wireframe) plus a public style card gives a target page in that design language (generated pages). The ColorOS 17 card cannot state the navigation shape or selected state, as OPPO’s public pages do not, and the page still draws a selected pill.

iOS 26

Material 3 Expr.

ColorOS 17

Shape

no radius values

corner scale

card spec only

Colour

no hex values

palette tokens

card spec, no hex

Material

Liquid Glass

blur not specified

marketing only

Navigation

no selected-tab look switch, search, lists

bar tokens

marketing only

control tokens

card buttons only

Type

type scale

type scale

card text only

Spacing

no margins

Controls

stated

partly stated

16 and Material, and the three reviewing judges missed the difference too. Quality gate. A base page is a positive only if it follows its card: two judges check it item by item, as in the design task, and admit only pages with a specification score of at least 6.0. 80% of the 223 checked pages pass (one Material 3 Expressive dark page, the file browser, was not generated): all iOS 26 and Material 3 Expressive pages, 93% of ColorOS 17 and One UI 9, 68% of HarmonyOS 7, 64% of OriginOS 7 and HyperOS 4, and 57% of MagicOS 11, matching the order of the knowledge results. The gate scores only what the card states and cannot vouch for undocumented conventions, so weakly documented brands also have the most uncertain positives.

margins, targets card margins only not stated

Figure S4. How much each brand’s official public pages specify per field of our spec cards. Partly stated: the card marks part of the field as unspecified. For ColorOS 17, the only public specification covers home-screen cards; material and navigation appear only in marketing text.

Type swaps (brand, type). The page’s font family and heading weights are replaced with another brand’s. Because proprietary fonts cannot be redistributed (e.g. SF Pro), publicpart type faults use only open fonts or real screenshots.

need a named target, are reported only here. General UI errors are wrong under any design language and are labelled independently of the brand. Brand errors are wrong only with respect to the target brand and version, and fall into six common visual design dimensions: component, colour, radius, material, icon and type; spacing is a general error. Layout (alignment, spacing, margins, density, tap targets) is general knowledge: its rules are shared across brands.

Style dimensions (brand: colour, radius, material, icon). One visual dimension is measured on another brand’s page for the same requirement and applied to the target page: the accent of coloured controls, the corner radius of each component kind, the blur, shadow and background of cards and bars, or the icons in corresponding positions. Each modified element becomes a labelled box, and any box with unchanged pixels is dropped. These page-wide faults leave the page internally consistent, making them purely brand questions. The brand test holds 352 such boxes (accent 129, icon 103, material 91, radius 29) and 52 with a page-wide typeface.

Documentation. The knowledge gap follows public guidelines: official pages leave 1 spec-card item unstated for Material 3 Expressive, 4 for iOS 26, and 8 for ColorOS 17 (Fig. S4). Given only the name “OPPO ColorOS 17”, a frontier generator drew the bottom navigation of ColorOS 15

Table S1. Knowledge probes. State: 1,316 text questions from 658 facts of the official guidelines, half asking which brand a fact belongs to (8-way) and half which of four facts is the brand’s; the 124-question card pilot is in Sec. F. Apply v2: 680 items on test-split screenshots. Apply: which brand’s rule a real screenshot shows (no brand given), which brand a red-boxed swapped element follows, and which design dimension of a page breaks a given target. General: brand-free design knowledge in words (say) and on a red-boxed defect (see); “–”: not run. State

Apply (picture + card text)

General design

Model

which brand / fact ↑

whose rule ↑

boxed source ↑

broken dimension ↑

say ↑

see ↑

Chance

19%

13%

13%

20%

25%

17%

gpt-6.1-sol gpt-6-astra Gemini 4 Argon Gemini 3.1 Pro Gemini 3.8 Flash

1000/1316 983/1316 925/1315 816/1316 755/1316

229/412 242/412 285/408 209/408 221/408

33/163 39/163 53/163 25/163 32/163

37/105 33/105 22/105 25/105 25/105

28/28 31/31 31/31 31/31 31/31

46/69 48/70 59/70 41/70 50/70

Qwen3.5-9B base ProofJudge v5, earlier, brand target

303/1316 295/1316

77/410 117/412

18/162 17/163

22/105 27/105

28/31 27/31

33/70 60/70

Frontier generation is clean but off-brand. Looking only for general defects, two independent API judges find one on 2 of 24 first-round generated pages (8%, 0.08 per page) and on 7–9 of 64 generated base pages (11–14%, 0.16–0.23 per page, mostly overlap); ProofJudge v5 finds none on the former. Rebuilds recover 96–98% of a screen’s text. Brand styling is the gap: 20% of generated base pages fail the specification gate, and 32–43% for HarmonyOS 7, HyperOS 4, OriginOS 7 and MagicOS 11; three judges identify the intended brand on only 34% of them (iOS 89%, Material 100%, the others 6–35%); and rebuilds carry about two offspec elements per page (Tab. S2). Frontier generators make general UI faults on a minority of pages; conformance to a specific design language is the larger gap.

General design knowledge is high in words. Every probed frontier model answers each brand-free text question it parses (28–31 of 31), and 41–59 of 70 when naming the defect in a red box; training lifts Qwen3.5-9B from 33 to 60 of 70 on the latter (ProofJudge v5), while its brand knowledge stays flat. Critique against a named brand. On defect test v2 (Tab. S11), frontier models recall 25–50% of general faults and 15–27% of brand faults, yet every model raises false alarms on over half the clean pages (54–89%), drawing 2 to 6 boxes per page. Precision therefore stays at 4–13%, and box F1 at 0.07–0.17. Design: likeness separates, the specification does not. Rebuilding 47 real screens of six brands as HTML, the GPT models place and style almost every element (layout 98– 99%, style 96–98%) while Gemini 3.1 Pro and 3.8 Flash misplace about 30% and Gemini 4 Argon about 10%, yet frontier rebuilds score about as high on the specification as the real screens (6.9–7.7 against 7.4; Sec. E).

Knowledge follows public documentation. On 1,316 text questions built from 658 facts in the official guidelines (Tab. S1; per brand in Fig. S5), every frontier model answers 91–99% of the Material 3 Expressive questions and 67–99% of the iOS 26 ones, but only 40–46% for ColorOS 17 and 14–32% for MagicOS 11; asked which brand a MagicOS fact belongs to, models are right 0–10% of the time. Even the number of facts the guidelines yield follows documentation, from 298 questions for iOS to 74 for HyperOS 4. Knowing a rule is not applying it to pixels (680 picture-and-text items on test-split screenshots): given the target and a page with one broken dimension, models pick the broken dimension on only 21–35% of 105 items (chance 20%), and name whose rule a ColorOS 17 screenshot shows on 9–27% of 96, against 94–100% for iOS. Recency matters too: the Gemini judges always chose a brand’s oldest version (Sec. F). The open 9B models stay near chance: Qwen3.5-9B answers 23% and ProofJudge v5 (earlier, brand target) 22%, against 76% for gpt-6.1-sol, and LoRA training runs stay within one point of their base.

E. Design task: restore scores In the design task a generator rebuilds a real screenshot of a brand’s current version as HTML, with the brand name only or with its public spec card (47 screens of six brands; OriginOS 7 and MagicOS 11 publish no app-page screenshots). Two judges score every page item by item (Tab. S2): likeness checks content, layout and style against an inventory of the original screen, and the specification score counts card fields followed, every violation quoting the card verbatim. The GPT models place and style almost every element (layout 98–99%, style 96–98%), Gemini 3.1 Pro and 3.8 Flash misplace about 30% (layout 69–74%) and Gemini 4 Argon about 10%, and the open base loses most of the screen. 16

Table S2. Restore track (47 screens): itemised likeness and specification scores (1–10), mean of two judges (Gemini 3.1 Pro, gpt-5.6-sol); Viol. is the number of card violations per page. Tinted: below 75%. Generator gpt-6-astra gpt-6.1-sol Gemini 4 Argon Gemini 3.8 Flash Gemini 3.1 Pro Qwen3.5-9B base Real screenshots

Ctx

n

Content

Layout

Style

Likeness

Spec.

Viol.

card name card name card name card name card name card name

47 47 47 47 47 47 47 46 47 47 38 36

100% 100% 100% 100% 98% 98% 99% 99% 97% 96% 81% 54%

98% 99% 99% 99% 89% 90% 74% 72% 69% 69% 46% 23%

98% 96% 97% 97% 95% 95% 87% 86% 88% 88% 43% 22%

9.9 9.8 9.9 9.9 9.4 9.4 8.7 8.6 8.5 8.4 5.7 3.3

7.5 7.3 7.4 7.3 7.4 7.3 7.5 7.4 7.7 6.9 7.0 5.4

1.8 2.0 1.8 1.9 1.9 2.0 1.6 2.0 1.8 2.1 2.2 2.4

–

47

–

–

–

–

7.4

1.7

Knowing brand conventions

Frontier rebuilds score about as high on the specification as the real screens (7.3–7.7 against 7.4; Gemini 3.1 Pro with the name only 6.9), so that score is no ceiling and does not separate them; the card raises it for Gemini 3.1 Pro (6.9 to 7.7) and the open base (5.4 to 7.0) but hardly for the GPT models. Qwen3.5-9B did not finish 20 of its 94 rebuilds; its rows average the finished ones.

% correct, 1,316 questions from official guidelines

Judges differ in strictness. Scored as whole pages rather than item by item, the real screenshots of the design task average 8.0 under Gemini 3.1 Pro and 5.4 under gpt-5.6sol on the same 1–10 scale, and the scorers of an earlier generation pilot agreed only moderately (rank correlation 0.47–0.81), so all models are scored by the same judges.

Material

iOS

gpt-6.1-sol

98

99

HarmonyOS One UI OriginOS HyperOS ColorOS MagicOS

71

95

58

81

44

31

gpt-6-astra

98

99

68

93

57

65

46

32

Gemini 4 Argon

99

94

76

70

56

51

43

17

Gemini 3.1 Pro

95

74

73

38

64

45

40

14

Gemini 3.8 Flash

91

67

66

52

45

24

41

25

Qwen3.5-9B base

54

23

13

16

23

3

19

17

ProofJudge v5

34

26

16

20

21

8

22

23

Figure S5. Knowing brand conventions, per design language: % correct on 1,316 text questions from 658 facts of the official guidelines. Frontier models reach 91–99% on Material 3 Expressive but 40–46% on ColorOS 17 and 14–32% on MagicOS 11 (green: the two least documented); the open 9B models, the base and ProofJudge v5, stay near chance. Brands at their latest versions: iOS 26, Material 3 Expressive, HarmonyOS 7, One UI 9, OriginOS 7, HyperOS 4, ColorOS 17, MagicOS 11.

F. Knowledge: recency or documentation? On real screenshots, judges named the brand among eight (chance 1/8) and, given the brand, the version among three (1/3); a few unparsed answers lower some denominators. ColorOS is hard to recognise even in versions released before the judges’ cutoffs, which points to thin documentation; the Gemini judges always answer the oldest version and gpt-6-astra (cutoff April 2026) names iOS 26 but not iOS 27, so recency matters too.

G. Name only vs. public card Round-0 pages of an English generation pilot (four page requirements: fitness home, new event, gallery multi-select, music player) were generated twice, from the brand name only and from the name plus the public spec card (12 pages each: 4 requirements × 3 brands), and each set was scored twice by Gemini 3.1 Pro and Gemini 3.8 Flash (mean), once with the name only and once with the card in the judge’s prompt. Only 4 pages per brand: a preliminary sign, not a result. Under both judge contexts the card pages score higher, so the pages improve and not only the standard; the mixed difference (name-only pages scored without the card, card

Knowledge probe (state). Without images, multiplechoice questions built from the public cards (two per brand and stated field, 124 in all) were answered perfectly for HarmonyOS 7 (16/16 for all six frontier judges) and nearly so for Material 3 Expressive (17–18/18), but only 4/18 (Gemini 3.8 Flash), 4/18 (Gemini 3.1 Pro), 9/18 (gpt-6-astra), 7/18 (gpt-6.1-sol), 5/18 (Claude Opus 5.5) and 6/18 (Gemini 4 Argon) for ColorOS 17, and 1/12, 1/12, 7/12, 7/12, 4/12 and 1/12 for MagicOS 11. The open base Qwen3.5-9B answers 33 of the 124 (27%). 17

Table S3. Brand and version recognition on real screenshots. ColorOS 15 has only 9 screenshots. Tinted: below half. iOS 18

iOS 26

iOS 27

ColorOS 15

ColorOS 16

ColorOS 17

Brand among eight Gemini 3.8 Flash Gemini 3.1 Pro gpt-6-astra Qwen3.5-9B base ProofJudge v1 (earlier)

30/30 30/30 30/30 29/29 30/30

30/30 30/30 30/30 25/25 29/30

30/30 30/30 30/30 30/30 28/30

4/9 3/9 1/9 1/9 1/9

13/16 5/16 7/16 0/15 1/16

5/24 10/24 5/24 2/23 1/24

Version, brand given Gemini 3.8 Flash Gemini 3.1 Pro gpt-6-astra Qwen3.5-9B base ProofJudge v1 (earlier)

30/30 30/30 30/30 30/30 30/30

0/30 0/30 27/30 0/30 0/30

0/30 0/30 0/30 0/30 0/30

9/9 9/9 2/9 4/9 8/9

0/16 0/16 6/16 1/16 3/15

0/24 0/24 1/24 5/24 2/24

By brand (latest versions)

By card dimension

S iO

S yO l S S S S I on ria rO nO rO icO e m r eU pe lo igi l at ag M Ha On Hy Or Co M Al

ur lo Co

pe Ty

d Ra

gpt-6-astra

100

100

100

100

93

71

50

58

85

90

93

93

gpt-6.1-sol

100

100

100

100

71

64

39

58

80

90

71

Claude Opus 5.5

100

100

100

57

57

57

28

33

69

90

71

Gemini 4 Argon

100

100

100

79

50

43

33

8

67

100

Gemini 3.1 Pro

89

94

100

57

29

21

22

8

56

Gemini 3.8 Flash

72

94

100

50

21

36

22

8

Qwen3.5-9B base

44

61

31

7

0

21

11

25

ProofJudge v3

39

56

25

14

14

43

11

25

ius

g cin

ion ls ics ial at ro er ph vig nt at a M N Co

s on Ic

a Gr

75

88

75

94

79

75

86

81

88

67

81

79

75

71

62

75

58

62

64

67

71

79

56

69

50

62

57

67

90

57

64

56

56

42

38

50

58

53

70

50

71

50

62

58

44

36

42

27

30

29

29

31

31

33

19

14

25

29

40

29

43

31

44

25

19

14

17

a Sp

% correct. Grey: at or above half; orange: below half. Two questions per brand and stated field; 124 in all. Open models below the line.

Figure S6. State probe (text only): accuracy of every model by brand and by card dimension, two questions per brand and stated card field, 12–18 per brand and 124 in all. Each brand at its latest version: iOS 26, Material 3 Expressive, HarmonyOS 7, One UI 9, HyperOS 4, OriginOS 7, ColorOS 17, MagicOS 11. Small and preliminary. Table S4. Brand scores (1–10) of round-0 pages by generation context and judge context. Judge context

Name-only pages

Card pages

Difference

Name only Name + card

5.88 6.17

6.29 7.00

+0.41 +0.83

By brand, name-only pages → card pages (name-only judge / card-aware judge) iOS 26 6.25 → 5.75 / 6.88 → 7.50 Material 3 Expressive 7.25 → 8.00 / 7.38 → 7.88 ColorOS 17 4.12 → 5.12 / 4.25 → 5.62

I. How visible the local test faults are

pages scored with it) would be about 1.1. ColorOS 17 gains most, in line with Fig. S4. Judges without the card see half the gain, and for iOS 26 they prefer the name-only pages.

We measured each consistency-test fault against its clean page: the box’s share of the screen, the share of its pixels that visibly change, and the mean CIEDE2000 colour difference over the box (medians; the same measures on the training faults beside them). Colour. A test colour fault boxes a large control (2.6% of the screen) in which only 2.6% of the

H. OwlEye: all judges Table S5 lists every judge on OwlEye, including the six open VLMs summarised in one row of the main table.

18

Table S5. OwlEye, all judges (Tab. 7): 600 real Android screenshots, 300 with display issues (null values, missing images, occlusion, overlapping text, blur) and 300 normal; screenshot-level verdicts with the consistency prompt. FA: normal screenshots flagged. Images used for evaluation only. Flagging every screenshot scores F1 0.67 and accuracy 50% on this balanced set, so read F1 with FA and accuracy; unanswered screenshots count as not flagged. Judge

Recall

FA

pages of the first 576 test pages, blind. On 13 pages two judges boxed the same place (IoU ≥ 0.5), for example a floating button over list rows or a send button cut off at the screen edge; 6 more rest only on a clipped last column of a sideways-scrolling row, a usual scroll hint, and stay clean. With the faulty pages removed, Gemini 4 Argon flags 16% of the clean generated pages, gpt-6.1-sol 39% and ProofJudge 7% (4% of the clean real ones); F1 on these pages stays at 0.31 for ProofJudge and 0.31 for Gemini 4 Argon, and only ProofJudge passes the gate. Swaps. A test button swap changes 28% of its box (training: 65%), and test segmented controls and chips are a fifth and two fifths of the training size; navigation-bar swaps are alike in training and test, yet ProofJudge finds few of them. The pictures are 720×1600, within the model’s pixel limit, so nothing is lost to resizing.

Prec. F1 Acc. No ans.

API judges Gemini 4 Argon 96% 28% 77% 0.86 84% Gemini 3.8 Flash 99% 64% 61% 0.75 68% gpt-6.1-sol 100% 63% 61% 0.76 68% Claude Opus 5.5 (medium) 99% 53% 65% 0.79 73% ProofJudge (ours) ProofJudge (ours) 71% 25% 74% 0.72 73% ProofJudge v6 (b2-950) 72% 37% 66% 0.69 68% ProofJudge v6 (c-2229) 81% 64% 56% 0.66 59% ProofJudge v5 (final) 97% 87% 53% 0.68 55% Qwen3.5-9B base 93% 97% 49% 0.64 48% Open VLMs, zero-shot GLM-4.6V 98% 66% 60% 0.74 66% Qwen3-VL-8B 76% 33% 70% 0.73 72% Qwen3.8-27B 99% 100% 50% 0.66 50% Qwen3-VL-32B 89% 80% 53% 0.66 55% Mistral-Small-3.2 98% 99% 50% 0.66 50% InternVL3.5-38B 98% 100% 50% 0.66 49% InternVL3.5-8B 93% 93% 50% 0.65 50% Qwen3-VL-30B-A3B 52% 57% 48% 0.50 48% UI-TARS-1.5-7B 3% 1% 67% 0.05 51%

0 0 0 0 0 0 0 0 30

J. Text layout test The text layout test asks the consistency question (no target) about faults that are wrong under any design language and need no brand knowledge. Each fault changes the CSS of one text element of a held-out base page; the label is the box of that element’s text after layout, and a fault that changes no visible pixel or lies below the first screen is dropped. Seven rules are wrong on their own: the characters of a label touch (letter spacing −0.08 or −0.12 em), the lines of a paragraph touch (line height 0.8 or 0.9 of the font size), a short control label wraps onto two lines, one word or character stands alone on the last line, a word is broken inside, Chinese punctuation starts a line, and a label is cut with an ellipsis although its row has room (to 50 or 65% of its width). Two rules are wrong only against the siblings: one of at least three labels of the same size, weight and colour gets wider letter spacing (+0.06 to +0.3 em), or one of at least three paragraphs a line height of 1.8 or 2.2. For these two, a hard negative applies the same change to every sibling, so the page stays consistent. On 367 faults (Tab. 5), Gemini 4 Argon finds 47% and gpt-6.1-sol 54% at IoU ≥ 0.5 (51% and 56% at IoU ≥ 0.3). Both find most broken words (86%, 100%) and needless ellipses (81%, 81%), but few lone last words (22%, 25%) and few labels whose characters touch (17%, 27%); at the weakest strengths they find almost none (characters touch at −0.08 em: 3%, 9%; wider spacing at +0.06 em: 0%, 8%). They flag 32% and 49% of the clean pages and more of the hard negatives (56%, 68%): spacing that is wide but the same on every sibling is often called a fault. Claude Opus 5.5 at medium effort finds the most (72%, characters touching 73%), but it boxes almost every page: 80% of the clean pages and 96% of the hard negatives. ProofJudge finds 43%: more needless ellipses than any API judge (96%) and most broken words (94%), but few labels whose characters touch (5%); it flags 15% of the clean pages and 13% of the hard negatives, fewer than any API judge. The test uses

1 46 1 50 10 6 41 207 5

pixels change (∆E 1.0 over the box): a thin accent inside the control. A training colour fault boxes a small element (0.75%) that changes a third of its pixels (∆E 5.8), so the two are different tasks. Icons. We labelled each of the 62 icon faults of the first 576 test pages (the full test has 188) from crops of the faulty page: visible when a viewer of that page alone can see the icon does not match it (another stroke weight, fill or colour, a duplicate or wrong-role glyph, an empty placeholder), brand only when the icon looks normal on the page and only the brand’s conventions mark it as foreign (a back arrow for a chevron, near-identical search glyphs, a menu icon for a map icon). Three AI judges (a Claude agent, Codex and Gemini 4 Argon) answered blind, from the faulty page only, with no model answer and no label shown; 20 unchanged icons were mixed in, and none of them was called wrong (0 of 20). By majority 19 are visible and 43 brand only; the judges agree pairwise with Cohen’s κ 0.51–0.73, and their majority matches a first label by one annotator on 51 of 62. Gemini 4 Argon is both a judge here and a scored model. Neither the four API judges of Tab. S7 nor ProofJudge finds a brand-only icon. A check by designers is future work. Clean pages. The same three judges boxed anything wrong on the 75 clean generated

19

Table S6. Faults found by kind on the consistency test, every judge (no target, IoU ≥ 0.5); columns from easiest to hardest by the mean of the four API judges, as in Fig. 6; the two icon columns cover the 62 icon faults labelled for visibility (all icons: Sec. I). Faults per kind: colour 179, material 123, general 920, radius 86, foreign comp. 662, icon, visible 19, icon, looks normal 43. ∗ The base model boxes about eight elements per page, so its recall is not comparable. Judge

Colour

Material

General

Radius

Foreign comp.

Icon, visible

Icon, looks normal

Gemini 4 Argon gpt-6.1-sol Gemini 3.8 Flash GPT-6 Luna Claude Opus 5.5 (medium) ProofJudge (ours) ProofJudge rank 16 (earlier) Qwen3.5-9B base∗

50% 40% 46% 32% 65% 41% 37% 48%

45% 54% 33% 27% 71% 44% 30% 28%

38% 45% 40% 31% 49% 59% 51% 21%

53% 47% 12% 21% 66% 41% 24% 45%

43% 30% 24% 13% 48% 20% 14% 32%

26% 21% 21% 5% 32% 16% 11% 16%

0% 0% 0% 0% 7% 0% 0% 44%

Table S7. Icon faults found on the first 576 pages of the consistency test (62 labelled icon faults; no target, IoU ≥ 0.5), split by whether the page shows them. On all 188 icon faults of the full test, Gemini 4 Argon finds 15% and gpt-6.1-sol 11%. ∗ The base model draws about eight boxes per page, so its recall is not comparable. Judge Gemini 4 Argon gpt-6.1-sol Gemini 3.8 Flash GPT-6 Luna ProofJudge (ours) Qwen3.5-9B base∗

All icons ↑

Visible ↑

Brand only ↑

8% 6% 6% 2% 5% 35%

26% 21% 21% 5% 16% 16%

0% 0% 0% 0% 0% 44%

M. ProofJudge v5 against a named brand On defect test v2 (Tab. S11), ProofJudge v5 scores box F1 0.19 (Gemini 4 Argon: 0.17) with the highest precision (20%), drawing about one box per page; it recalls 54% of the general faults but only 11% of the brand faults. It still flags 65% of the clean pages: 28% of the real screenshots but all 75 generated ones (the supervised start: 15% and 57%). It also reports about one finding per page: on the 85 pages with two to six faults it finds 4% and never all (gpt-6.1-sol: 25%, all on 9 pages). v7 trains on pages with one to four faults (Sec. P). Two causes add up (Sec. N): the counterfactual training rows label the same clean generated picture follows 515 times and does not follow 1,030 times under other targets, which a model without brand knowledge cannot tell apart, and the GRPO reward pays a guessed box whenever the policy’s belief that a page is clean is below 0.36. GRPO leaves box F1 where SFT left it (0.18 to 0.18–0.19 over steps 50–375), trading false alarms for general recall, with brand recall at 8–10% on these v5 data, LoRA of rank 16 or 64 and full fine-tuning all land at 0.18–0.21, and three real screens of the target hurt every critic: the bottleneck is brand knowledge, not the training method. v5 writes the target brand and version into the prompt. Its training data have 7,822 rows from about 770 gated base pages over 34 page requirements (27% positives), split by page, mixed with brand-free critiques the base wrote on training screenshots against forgetting. We fine-tune the LoRA (rank 16) on the label JSON (0.7 hours on eight A100 GPUs), then continue with GRPO and serve checkpoint 150 merged into one rank-32 adapter (the served v5 model; rewards in Sec. N).

English pages only; Chinese pages are future work.

K. Two IoU thresholds A finding matches a labelled element when their boxes overlap with IoU ≥ 0.5 (main rule); Tab. S8 gives every localisation score at both IoU ≥ 0.5 and IoU ≥ 0.3. Neither rule counts a small box whose centre merely lies inside a large faulty element. The lower threshold raises every judge’s score, mostly for general faults with small true boxes, and leaves the order of the judges almost unchanged.

L. Thinking effort of the API judges Thinking effort is not a hidden factor in our API results (Tab. S9). On a fixed 138-page subset of the consistency test (20 clean real, 30 clean generated, 20 swaps, 48 local faults, 20 general faults), we ran gpt-6.1-sol at five effort levels and Gemini 4 Argon at three, and repeated the default level of each. The two runs of the same setting already differ by 0.00–0.08 F1, and within that noise no level clearly beats medium (GPT) or default (Gemini), while higher levels are 2–5× slower. The one difference that holds in both runs is that swapped components are found more often at medium or default (20–50%) than at any other level (5–10%). Every judge in the paper therefore runs at medium or default.

Training methods. All runs use the v5 data and are scored on defect test v2 (643 screenshots, 147 clean) and the general text probe (31 items).

20

Table S8. Localisation at both IoU thresholds (same answers scored twice). Consistency: 2,765 screenshots, no target; defect test v2: 643 screenshots with a named brand. –: not run. Consistency F1 ↑ Judge

Defect v2 F1 ↑

IoU ≥ .5

IoU ≥ .3

IoU ≥ .5

IoU ≥ .3

0.37 0.30 0.26 0.48 – 0.04 0.15 0.07

0.43 0.36 0.29 0.50 – 0.05 0.17 0.08

0.17 0.10 0.09 – 0.19 0.06 – –

0.21 0.12 0.12 – 0.20 0.08 – –

Gemini 4 Argon gpt-6.1-sol Gemini 3.8 Flash ProofJudge (ours) ProofJudge v5 (final) Qwen3.5-9B base Qwen3.8-27B GLM-4.6V

Table S9. Thinking effort on 138 pages of the first 576-page consistency test (no target given). FA: clean pages flagged; Swap and Local: recall. Medium and default come from the full run (no timing). Judge

Effort

FA real

FA gen.

Swap

Local

F1

s/page

Gemini 4 Argon

low default default, rerun high

10% 15% 5% 15%

20% 30% 30% 30%

5% 50% 40% 5%

27% 33% 25% 29%

0.24 0.33 0.25 0.22

4 – 5 19

gpt-6.1-sol

low medium medium, rerun high xhigh max

40% 35% 45% 40% 30% 35%

27% 43% 50% 17% 27% 27%

10% 20% 20% 10% 10% 10%

33% 29% 35% 35% 35% 40%

0.24 0.19 0.19 0.22 0.19 0.22

14 – 14 27 53 70

Table S10. Training methods for ProofJudge on the v5 data (FT: fine-tuning; LM: language model). GRPO checkpoints from step 50 to 375 score F1 0.18–0.19 with brand recall 8–10%; the served model merges step 150 into one rank-32 adapter. Model Qwen3.5-9B base ProofJudge v5 ProofJudge v5 ProofJudge v5 ProofJudge v5 ProofJudge v5 ProofJudge v5

Method no training LoRA r16 (SFT) LoRA r64 full FT (LM) full FT (LM + vision) LoRA r16 + GRPO 150 same, served merged (final)

False alarms

General recall

Brand recall

Defect F1

General text

88% 37% 62% 65% 64% 56% 65%

20% 57% 63% 59% 64% 58% 54%

26% 6% 12% 12% 10% 10% 11%

0.06 0.18 0.21 0.21 0.19 0.19 0.19

90% 87% 90% 74% 71% 90% 87%

N. ProofJudge: rewards and the move to a consistency check

over half the screen matches only an equally large true box. Before any run the reward was checked against degenerate policies (report nothing, a 12-box grid, one whole-screen box, one wrong box, an unparsable answer), which must all score below the true labels. GRPO starts from the merged v5 SFT adapter and trains a new LoRA r16 on 6,004 prompts (32% clean; 8 answers per prompt, T 1.0, KL β 0.04, lr 5×10−6 , 8×A100-80GB).

The reward. Every training fault is made by the data engine, so the reward is computed by code against its labels, with no model judge in the loop. SFT teaches the answer format; GRPO then optimises a box-matching score (IoU ≥ 0.3 or centre within 50 of 1000; the paper scores at IoU ≥ 0.5). A parsable answer gets +0.1 and an unparsable one −2.0. On a clean page, reporting nothing gets +1.0, the first box costs c1 and each further box 0.25; on a faulty page the score is box F1 minus the share of faults missed, minus 0.05 per unmatched box and minus s for silence; a small bonus (+0.1× the share) rewards the right kind, and a box

Three reward versions. Only c1 and s changed. A guessed box on a page the policy believes clean with probability p pays when p < p∗ = (s − 0.05)/(c1 + s − 0.05). R1 (c1 1.5, s 0, p∗ 0) made silence cheaper than one wrong box:

21

Table S11. Critique on defect test v2 (643 screenshots, 147 clean; base pages that fail the quality gate are removed, since their clean label cannot be trusted; one prompt and one matching rule for all). ProofJudge v5 is the LoRA with GRPO checkpoint 150, served as one merged adapter. Tinted: false alarms on more than half of the clean pages. Clean false alarms ↓

General recall

Brand recall

Precision

F1

Boxes / page

Gemini 4 Argon Gemini 3.8 Flash gpt-6-astra gpt-6.1-sol Gemini 3.1 Pro

54% 89% 80% 75% 60%

40% 32% 47% 50% 25%

21% 15% 26% 27% 16%

13% 6% 6% 6% 4%

0.17 0.09 0.10 0.10 0.07

2.0 3.3 5.1 6.0 4.3

Qwen3.5-9B base ProofJudge v5, earlier, brand target

88% 65%

20% 54%

26% 11%

4% 20%

0.06 0.19

7.5 1.0

Critic

• Data. Out: counterfactual rows, whole-page typeface faults and old-version pages (all self-consistent, only brand knowledge tells them), and the page-wide style faults, which restyle every card or every accent and so leave the page consistent with itself. In: local style faults. One element of a kind that the page shows at least twice takes another corner radius, material, accent or icon, and its siblings keep the page’s own value. For example, one button’s radius drops from 28 to 9 px while the cards keep 28 (Fig. S7). Swapped components and general faults stay. The page-wide faults remain in the benchmark as brand questions for the large models. • Reward and selection. R3; clean prompts repeated to 40%; a checkpoint is usable only if its false alarms on real and on generated clean pages are both at most 15%, and F1 decides only among usable checkpoints. No v5 model passes this gate. The large judges already find this kind of fault without brand knowledge: on swapped components gpt-6.1-sol reaches 36% recall and gpt-6-astra 33%, against 23% for ProofJudge final; on page-wide colour and icon faults even gpt6.1-sol stays at 8–12%.

clean-page false alarms fell from 37% to 18%, but general recall fell from 57% to 40% and F1 from 0.18 to 0.15. R2 (c1 1.0, s 0.6, p∗ 0.36) keeps F1 at 0.18–0.19 (Tab. S10; the served step 150, merged into one r32 adapter, reaches 0.19 with 54% general recall), but a box pays whenever the policy is unsure: it reports about one box per page and flags almost every clean generated page. R3 (c1 1.5, s 0.3, p∗ 0.14) is v6’s. Why generated pages. Two causes add up. (1) In the v5 SFT data the same clean generated picture is labelled follows 515 times and does not follow 1,030 times: the counterfactual rows ask about it under another brand’s target. A 9B model that does not know the brands (22–23% on the fact set, Tab. S1) cannot use the target, so it learns that a generated page is wrong; the SFT model already flags 57% of them and 15% of real screenshots. (2) R2 then rewards a guessed box whenever p < 0.36, and pushes that prior to its limit. Part of the 100% may be correct: only about a third of the generated pages are recognised as their brand by the base-page gate, and every external judge also flags most of them (Gemini 4 Argon 88%). But a critic that flags every generated page is of no use to a designer, whatever the reason.

v6 results. Table S12 lists the v6 runs on the consistency test (576 pages, 128 clean; 19 clean pages on which both outside judges box the same place were left out when the test was built, separately from the 13 pages of Sec. I). The first run (v6-a) still learned that a generated page is faulty: every faulty training image was generated, while real screenshots appeared only clean, so it flags 4% of clean real screens but 49% of generated ones. The second run (v6-b) repeats clean generated pages until they match the faulty ones and keeps a validation split for checkpoint choice; its chosen step 950 matches Gemini 4 Argon on false alarms but finds few local faults. The third run (v6-c) trains on full-size pages (the old pixel budget shrank 720×1600 screenshots by 7% per side), with local faults as half of the faulty rows and a GRPO bonus per found local fault; its recall carries over to the three unseen test page types, but it flags almost every clean generated test page.

v6: a consistency check, not a brand check. These results change the task, not only the reward. A 9B model does not learn eight brands’ guidelines, but it does not need to: what goes wrong on a generated page is rarely the design itself (8–14% of generated pages have a general defect, Sec. D) and mostly one part whose style does not match the rest of the page. v6 therefore asks without a target brand: find each element that does not match the style of the rest of this page, or has a general problem. The page itself is the reference, so a clean page has one label only. • Prompt. No target line; finding kinds are common and inconsistent (with its aspect); the look-alike brand and the card rule leave the answer, since both need brand knowledge.

22

Table S12. ProofJudge v6 runs on the consistency test. FA: clean pages flagged (real screenshots, generated pages); Swap, Local, Radius: recall. † Boxes kept only when P (follows) < 0.005; the threshold was read off the test, so this row only shows what calibration could reach, not a result. FA real FA gen. Swap Local Radius

F1

Gemini 4 Argon gpt-6.1-sol

15% 36%

20% 41%

36% 24%

32% 28%

83% 83%

0.31 0.20

v6-a, step 750 v6-b, step 950 (chosen) v6-b, step 1300 v6-c, step 2229 v6-c, threshold 0.005†

4% 4% 9% 13% 4%

49% 20% 72% 90% 13%

16% 8% 19% 19% –

5% 7% 13% 21% 16%

4% 4% 12% 50% –

0.22 0.21 0.24 0.25 –

Two lessons. (1) Validation must look like the test. The first validation split held two page types, and on one of them (a media player with large round buttons, 67 of its 91 local faults) a changed radius is easy to see: v6-b step 950 found 15 of 24 local radius faults there but 1 of 24 on the test, although the changed elements (mostly buttons) and the size of the change (median 13–15 px) are the same in both. v6d and v7 validate on six held-out page types. (2) Confidence does not yet separate. The verdict token’s probability ranks clean generated pages below faulty pages only weakly (AUC 0.63 for v6-c, 0.62 for v6-b), so a threshold buys quiet with recall. v6d therefore supervised the comparison itself: each finding states what differs from its siblings (“17 px corners; the other buttons 50 px”), and its threshold was chosen on the validation split (six held-out page types; a later split adds 89 held-out real pages; the numbers here are on the current split). On that split v6d reaches F1 0.33 with 30% consistency recall, but flags 55% of the clean generated pages (3% of the real screenshots), so it fails the gate and is not tested.

Figure S7. A local style fault: the Start Workout button’s corner radius changes from 28 to 9 px; the cards and the navigation keep the page’s own shapes. The page, not a brand, is the reference.

P. ProofJudge v7f: the paper’s model v7f. The paper’s model is trained on the v7f data with a LoRA of rank 128 (α 256), twice with different seeds; the two adapters are averaged and a cut is put on the verdict. Its data follow v7b’s recipe (below) with three changes. (1) Validation screens leave the training set: an earlier split held rows of four validation page types, which made validation too easy; the fixed split holds six held-out page types and five more generated screens, and no training row comes from them. (2) Sixteen new training screens, none of them a test or validation screen, are drawn in eight design languages by three generators; their pages that pass the quality gate get engine faults (swaps, general and local style faults), and each of their clean pages is repeated at most four times. (3) v7b’s real HTML pages (design-system templates and kit examples) are not used. In all, the 17,285 rows of ScreenProof-Train (Sec. O; 42% clean) and 743 auxiliary rows. Training uses v7b’s settings below except the rank and the length: two epochs (2,570 steps) instead of three. Seeds and averaging. Two SFT runs on the same data that differ only in the seed land up to 0.04 F1 apart on validation (Tab. 6), more than most data changes we tried. We therefore train two seeds (42 and 7) and average their adapters exactly: the update 12 (B1 A1 + B2 A2 ) is one LoRA of rank 256 with [B1 /2, B2 /2] and [A1 ; A2 ]. Each run uses its final step, so validation chooses no checkpoint. Cut. The model’s verdict

O. ScreenProof-Train ScreenProof-Train is the training set the data engine makes for target-free critics, released with the benchmark (Tab. 3). It holds 17,285 rows on 11,424 images of 990 pages: 542 generated base pages of 38 page requirements that are not test requirements, and 448 public web pages rendered at phone size, from which 790 rows come. 42% of the rows are clean pages, so that a critic learns to stay silent. 591 designed differences pair a clean row with a faulted twin on the same element (a selected tab, a badge, a primary button, a floating button over content, a sticky bottom bar, a light placeholder): the clean row shows a difference a designer intends, the twin one that breaks the page. The paper’s ProofJudge (v7f) is trained on this set (Sec. P).

23

gives a probability that the page is clean; a finding is kept only when that probability is below 0.01099, chosen on the validation split, first by the gate (at most 15% of clean real and of clean generated pages flagged), then by F1. Capacity, not data. At rank 16, sixteen more training screens, more fault rows, GRPO and averages of up to five runs all stayed within seed noise on validation (F1 0.55–0.56). Raising the rank did not: averaged pairs reach 0.550 (rank 16), 0.571 (64), 0.607 (128) and 0.599 (256) (Tab. 6). Each rank was chosen on validation before its test run. On the v5 data, rank 16 and 64 had scored alike (Tab. S10); with the v7f data, the adapter size became the limit. Three epochs instead of two (averaged pair 0.619, +0.012) stay within the noise and are not used. Votes over runs. Instead of averaging weights, four rank-128 runs (two seeds on the v7f data, two on the v7h data, which adds sixteen generated screens) can vote: a finding stays when at least three runs box the same place (IoU ≥ 0.5), and the cut is put on their mean verdict probability. Fixed on validation (F1 0.654 against 0.607 for the paper’s model) and tested once, the vote raises F1 on the main test from 0.48 to 0.535 (paired bootstrap +0.054, 95% interval [+0.041, +0.067]) at 8% and 12% flagged clean real and generated pages, but it flags 18% of the clean generated pages of the first 576-page test and 21% of the clean text layout pages, beyond the gate, and falls on OwlEye (F1 0.66 against 0.72), at four answers per page. We therefore keep the single averaged model, although the vote is higher on validation. Every test run. The main test was scored for 7 variants of the v7 line (Tab. 6): v7e with GRPO (F1 0.37), v7f at rank 16 (0.40, the paper’s model before this one) and with GRPO (0.41), the v7h pair (0.42), the rank64 pair (0.45), the rank-128 pair (0.48, ours) and the vote (0.535). Each was fixed on validation before its test run, but the choice among the tested variants saw their test scores. Two rank-128 variants score slightly higher on validation, within seed noise, and were not tested (three epochs 0.619, the v7h data 0.612). Test. On the 2,765-page consistency test (Tab. 4) it reaches F1 0.48 and flags 2% of clean real and 9% of clean generated pages; without the cut, F1 0.46 with 25% and 58%. On the first 576 pages of the test, used for the earlier runs below, it reaches F1 0.31 (Gemini 4 Argon 0.31) and flags 4% and 11%.

faults of the kinds UI-Lens and OwlEye label (a null value, a missing image, a cropped element, a wrong ellipsis) in our own page code, and no UI-Lens or OwlEye page is used. 19 real HTML pages (design-system templates and kit examples) join as faulted copies and as clean pages. In all, 11,544 rows from 543 pages, 43% clean; faulted rows hold one (3,772), two (1,330), three (909) or four (582) faults. v7b is LoRA r16 (α 32), lr 10−4 , batch 14 on seven A100 GPUs, 3 epochs (2,646 steps). Step 1600 was chosen on six held-out page types under an earlier match rule that also counted a box whose centre lay inside the true box grown by 5% of the screen; rescored with the main rule, its validation F1 is 0.43, and the best checkpoint that passes the gate (step 250, F1 0.14, 11% of the clean generated pages) scores 0.08 on the first 576 test pages. On the main test v7b reaches F1 0.33 and flags 32% of clean real and 46% of clean generated pages (on the first 576 pages, F1 0.25; v6: 0.21). A failed run: clean real screens only. v7c adds 2,592 real screenshots (ColorBench, iOS-1K, MMBench-GUI and web pages) to the data of v7, all labelled clean, and trains for two epochs. Validation picks step 650 by the gate (0% of clean real and 11% of clean generated pages flagged), but the model has learned that a real screenshot is clean: it flags 0% of OwlEye’s faulty screens and reaches consistency recall 2% and F1 0.14 on the first 576 pages of the consistency test. No real screen in its training data carried a fault, so real screens need faulted copies as well as clean ones.

Q. Find, then fix Does a judge’s finding help a generator repair the page? We test it where the repair can be checked exactly. Pages. 85 consistency-test pages with one local fault whose right value the page itself shows (24 corner radius, 13 accent colour, 48 surface material: one element differs from its siblings), and 40 clean generated pages, on which a false finding should change nothing. Icon and swap faults are left out: the original icon or component cannot be known from the page, so agreement with the clean original would be the wrong measure. The engine keeps screenshots only, so each faulty page is rebuilt from its record and kept only if its render matches the stored screenshot (125 of 128). The fault is written as an ordinary rule in the page’s own stylesheet (a structural selector at a random place among the rules), so that the code does not give it away. A first run that left it as an inline !important style is discarded: that style gave the fault away, and the fixer found it with no hint as often as with the true finding.

v7b, the earlier model. v7 keeps v6’s target-free prompt, with a short why field per finding, and changes the data in three ways. Clean generated training pictures that Gemini 3.8 Flash and Gemini 4 Argon both box in the same place are dropped (500 of 1,831 pictures, 928 rows; 10 of 75 clean generated test pages are disputed in the same way). New copies carry one to four independent faults of different kinds with boxes that do not overlap, each checked to change visible pixels; besides overflow, overlap, small targets, low contrast and the local style faults, the engine makes display

Fixer and hints. gpt-6.1-sol (Codex, medium effort) gets the page’s HTML, its screenshot and one hint: none (fix

24

Table S13. Find, then fix: gpt-6.1-sol repairs 85 pages with one local fault and is also given 40 clean generated pages. Sent: pages that reach the fixer. ours: the paper’s model; r16 + cut: the same data with a rank-16 LoRA (the previous model); v7b: an earlier model; v6-c and v6-b: earlier one-fault runs. Hint

Fixed Radius Colour Material Harm, faulty Harm, clean

Sent

None True finding

95% 95%

100% 100%

69% 77%

100% 98%

0% 0%

12% 0%

100% 100%

Gemini 4 Argon findings ProofJudge (ours) findings ProofJudge r16 + cut (earlier) findings ProofJudge v7b (earlier) findings ProofJudge v6-c (earlier) findings ProofJudge v6-b (earlier) findings

55% 25% 16% 42% 39% 13%

79% 29% 8% 67% 46% 4%

38% 8% 8% 8% 8% 8%

48% 27% 23% 40% 44% 19%

11% 1% 0% 6% 8% 2%

15% 0% 2% 8% 8% 0%

100% 100% 100% 100% 100% 100%

Gate: Gemini 4 Argon Gate: ProofJudge (ours) Gate: ProofJudge r16 + cut (earlier) Gate: ProofJudge v7b (earlier) Gate: ProofJudge v6-c (earlier) Gate: ProofJudge v6-b (earlier)

72% 28% 21% 72% 94% 24%

92% 38% 17% 88% 100% 12%

38% 15% 15% 23% 62% 23%

71% 27% 25% 77% 100% 29%

0% 0% 0% 0% 0% 0%

10% 5% 0% 12% 12% 2%

56% 22% 17% 64% 96% 19%

Limits. The faults are injected, and a stray rule with a long structural selector may still be easier to spot in code than an inconsistency a generator writes itself. Without code (a screenshot, a design-tool frame or an installed app) the fixer cannot read the page, and the judge’s location matters more; we did not test that setting. Each condition ran once.

whatever does not match the rest of the page, or return it unchanged); the true finding (box, kind, aspect and what differs, e.g. “7 px corners; the other buttons 20 px”); or a judge’s findings on the consistency test, with the instruction to fix the reported problems that are real and change nothing else. A judge that reports nothing leaves the page as it is. As a gate, the judge only decides whether a page goes to the fixer, which then works with no hint.

R. More detection examples Figures S8 to S17 extend Fig. 5 with twenty more test screenshots (ProofJudge v7f, no target): four of each kind on the consistency test (general faults, local style faults, components from another design system, clean pages) and four on the text layout test. Unlike the hand-picked cases of the main figure, every case here is the first test page of its kind, with one brand and one page per case where possible and no condition on any model’s answer, so successes and failures appear in their real proportion. Two kinds of page are skipped, because a judge’s box there may mark a real problem the label does not hold: clean pages that are disputed (two API judges box one place) or that the blind check found faulty (Sec. I), and every page made from a base page with such a problem, or on any copy of which both API judges box one place that touches no labelled fault (for example a send button cut off at the screen edge). Each case names its fault in one line and enlarges the faulty area on the right; every box is drawn as four corner marks just outside its region: amber marks are the engine’s exact label, mint marks ProofJudge’s boxes and violet marks the API critics’. The captions state each case’s outcome; figures and captions are generated by code from the stored answers.

Scores. Against the clean original rendered the same way: a fault is fixed when the pixels inside its box that differ from the clean page fall to a quarter or less; harm is a change of more than 1% of the pixels outside the box (faulty pages) or anywhere (clean pages). Reading. Repair is not the bottleneck: with the page’s code and screenshot, the fixer finds and fixes 95% of these faults unprompted and never damages the rest of a faulty page. Run on every page, however, it also changes 12% of clean pages. Only the true finding gives both a high fix rate and no harm. A judge’s findings help only as far as they are right: under “change nothing else” a finding on the wrong element keeps the fixer from the real fault, so even Gemini 4 Argon’s findings lower the fix rate from 95% to 55%. As a gate, the paper’s ProofJudge sends only 22% of the pages and lets 28% of the faults be fixed, harming 5% of clean pages (the rank-16 model: 17%, 21%, none), while Gemini 4 Argon sends 56% and 72% are fixed; the earlier v7b sends 64% (72% fixed); the earlier v6-c sends 96% of the pages, and v6-b protects clean pages (2% harmed) but lets only 24% of the faults through.

25

Text overflow OPPO ColorOS 17 · no target given Text overflow: text runs past its container or the screen edge Engine label

ProofJudge

Gemini 4 Argon

gpt-6.1-sol Fault, enlarged

1 fault

1 box · 1/1 found

1 box · 0/1 found

2 boxes · 1/1 found

Overlap Xiaomi HyperOS 4 · no target given Overlap: two elements cover each other Engine label

ProofJudge

Gemini 4 Argon

gpt-6.1-sol Fault, enlarged

1 fault

1 box · 1/1 found

1 box · 1/1 found

1 box · 1/1 found

Figure S8. General faults on the consistency test (no target given, IoU ≥ 0.5). Text overflow: found by ProofJudge, gpt-6.1-sol; Overlap: found by ProofJudge, Gemini 4 Argon, gpt-6.1-sol. Low contrast Google Material 3 Expressive (Android 16) · no target given Low contrast: text too faint to read Engine label

ProofJudge

Gemini 4 Argon

gpt-6.1-sol Fault, enlarged

1 fault

2 boxes · 1/1 found

2 boxes · 1/1 found

1 box · 0/1 found

Tap target too small Huawei HarmonyOS 7 · no target given Small tap target: a control too small to tap Engine label

ProofJudge

Gemini 4 Argon

gpt-6.1-sol Fault, enlarged

1 fault

0 boxes · 0/1 found

0 boxes · 0/1 found

0 boxes · 0/1 found

Figure S9. General faults on the consistency test (no target given, IoU ≥ 0.5). Low contrast: found by ProofJudge, Gemini 4 Argon; Tap target too small: found by no judge.

26

One icon off Samsung One UI 9 · no target given Local fault: one element's icon differs from the same elements on its page Engine label

ProofJudge

Gemini 4 Argon

gpt-6.1-sol Fault, enlarged

1 fault

0 boxes · 0/1 found

1 box · 0/1 found

1 box · 0/1 found

One colour off OPPO ColorOS 17 · no target given Local fault: one element's colour differs from the same elements on its page Engine label

ProofJudge

Gemini 4 Argon

gpt-6.1-sol Fault, enlarged

1 fault

0 boxes · 0/1 found

0 boxes · 0/1 found

0 boxes · 0/1 found

Figure S10. Local style faults on the consistency test (no target given, IoU ≥ 0.5). One icon off: found by no judge; One colour off: found by no judge. One material off Xiaomi HyperOS 4 · no target given Local fault: one element's material (blur, shadow, surface) differs from the same elements on its page Engine label

ProofJudge

Gemini 4 Argon

gpt-6.1-sol Fault, enlarged

1 fault

0 boxes · 0/1 found

1 box · 1/1 found

1 box · 1/1 found

One radius off Huawei HarmonyOS 7 · no target given Local fault: one element's corner radius differs from the same elements on its page Engine label

ProofJudge

Gemini 4 Argon

gpt-6.1-sol Fault, enlarged

1 fault

0 boxes · 0/1 found

2 boxes · 0/1 found

1 box · 0/1 found

Figure S11. Local style faults on the consistency test (no target given, IoU ≥ 0.5). One material off: found by Gemini 4 Argon, gpt-6.1-sol; One radius off: found by no judge.

27

Foreign top bar OPPO ColorOS 17 · no target given Foreign component: the title bar comes from a page of another design system Engine label

ProofJudge

Gemini 4 Argon

gpt-6.1-sol Fault, enlarged

1 fault

0 boxes · 0/1 found

1 box · 0/1 found

1 box · 0/1 found

Foreign button Apple iOS 26 (Liquid Glass) · no target given Foreign component: the button comes from a page of another design system Engine label

ProofJudge

Gemini 4 Argon

gpt-6.1-sol Fault, enlarged

1 fault

0 boxes · 0/1 found

0 boxes · 0/1 found

0 boxes · 0/1 found

Figure S12. Components from another design system on the consistency test (no target given, IoU ≥ 0.5). Foreign top bar: found by no judge; Foreign button: found by no judge. Foreign list row Huawei HarmonyOS 7 · no target given Foreign component: the list row comes from a page of another design system Engine label

ProofJudge

Gemini 4 Argon

gpt-6.1-sol Fault, enlarged

1 fault

0 boxes · 0/1 found

0 boxes · 0/1 found

0 boxes · 0/1 found

Foreign tab bar vivo OriginOS 7 · no target given Foreign component: the tab bar comes from a page of another design system Engine label

ProofJudge

Gemini 4 Argon

gpt-6.1-sol Fault, enlarged

1 fault

0 boxes · 0/1 found

1 box · 1/1 found

0 boxes · 0/1 found

Figure S13. Components from another design system on the consistency test (no target given, IoU ≥ 0.5). Foreign list row: found by no judge; Foreign tab bar: found by Gemini 4 Argon.

28

Clean real screenshot Xiaomi HyperOS 4 · no target given Clean page: nothing to find, so every box is a false alarm Engine label

ProofJudge

Gemini 4 Argon

gpt-6.1-sol No enlarged view: nothing to find on a clean page.

clean, real

silent ✓

silent ✓

1 false box

Clean real screenshot OPPO ColorOS 17 · no target given Clean page: nothing to find, so every box is a false alarm Engine label

ProofJudge

Gemini 4 Argon

gpt-6.1-sol No enlarged view: nothing to find on a clean page.

clean, real

silent ✓

silent ✓

silent ✓

Figure S14. Clean pages on the consistency test (no target given, IoU ≥ 0.5). Clean real screenshot: false boxes from gpt-6.1-sol (1); Clean real screenshot: every judge stays silent. Clean generated page Huawei HarmonyOS 7 · no target given Clean page: nothing to find, so every box is a false alarm Engine label

ProofJudge

Gemini 4 Argon

gpt-6.1-sol No enlarged view: nothing to find on a clean page.

clean, generated

silent ✓

silent ✓

silent ✓

Clean generated page Honor MagicOS 11 · no target given Clean page: nothing to find, so every box is a false alarm Engine label

ProofJudge

Gemini 4 Argon

gpt-6.1-sol No enlarged view: nothing to find on a clean page.

clean, generated

silent ✓

silent ✓

silent ✓

Figure S15. Clean pages on the consistency test (no target given, IoU ≥ 0.5). Clean generated page: every judge stays silent; Clean generated page: every judge stays silent.

29

Characters touch OPPO ColorOS 17 · no target given Text layout: the characters of a label touch Engine label

ProofJudge

Gemini 4 Argon

gpt-6.1-sol Fault, enlarged

1 fault

0 boxes · 0/1 found

0 boxes · 0/1 found

1 box · 1/1 found

Word broken inside Huawei HarmonyOS 7 · no target given Text layout: a word is broken inside across two lines Engine label

ProofJudge

Gemini 4 Argon

gpt-6.1-sol Fault, enlarged

1 fault

1 box · 1/1 found

1 box · 1/1 found

1 box · 1/1 found

Figure S16. Text layout faults on the text layout test (no target given, IoU ≥ 0.5). Characters touch: found by gpt-6.1-sol; Word broken inside: found by ProofJudge, Gemini 4 Argon, gpt-6.1-sol. Lone last character Apple iOS 26 (Liquid Glass) · no target given Text layout: one word or character stands alone on the last line Engine label

ProofJudge

Gemini 4 Argon

gpt-6.1-sol Fault, enlarged

1 fault

0 boxes · 0/1 found

0 boxes · 0/1 found

0 boxes · 0/1 found

Hard negative: every label spaced alike Xiaomi HyperOS 4 · no target given Hard negative: every label has the same wider spacing, so nothing is wrong Engine label

ProofJudge

Gemini 4 Argon

gpt-6.1-sol No enlarged view: nothing to find on a clean page.

clean, generated

1 false box

silent ✓

11 false boxes

Figure S17. Text layout faults on the text layout test (no target given, IoU ≥ 0.5). Lone last character: found by no judge; Hard negative: every label spaced alike: false boxes from ProofJudge (1), gpt-6.1-sol (11).

30

Related documents

• Accident#1291406
• Reliable hierarchical operating system fingerprinting via conformal prediction.#351493
• Reliable hierarchical operating system fingerprinting via conformal prediction#345912
• Emergency#1291405
• Computer Misuse Act 1990#4240