PaperFit: Vision-in-the-Loop Typesetting Optimization for Scientific Documents Bihui Yu1∗ , Xinglong Xu1∗ , Junjie Jiang1∗ , Jiabei Cheng3∗ , Caijun Jia1 , Siyuan Li2 , Conghui He2 , Jingxuan Wei1 , Cheng Tan2 1 University of Chinese Academy of Sciences, 2 Shanghai Artificial Intelligence Laboratory,
arXiv:2605.10341v1 [cs.AI] 11 May 2026
3 School of Automation and Intelligent Sensing, Shanghai Jiao Tong University
A LaTeX manuscript that compiles without error is not necessarily publication-ready. The resulting PDFs frequently suffer from misplaced floats, overflowing equations, inconsistent table scaling, widow and orphan lines, and poor page balance, forcing authors into repetitive compile-inspect-edit cycles. Rule-based tools are blind to rendered visuals, operating only on source code and log files. Text-only LLMs perform open-loop text editing, unable to predict or verify the two-dimensional layout consequences of their changes. Reliable typesetting optimization therefore requires a visual closed loop with verification after every edit. We formalize this problem as Visual Typesetting Optimization (VTO), the task of transforming a compilable LaTeX paper into a visually polished, page-budget-compliant PDF through iterative visual verification and source-level revision, and introduce a five-category taxonomy of typesetting defects to guide diagnosis. We present PaperFit, a vision-in-the-loop agent that iteratively renders pages, diagnoses defects, and applies constrained repairs. To benchmark VTO, we construct PaperFit-Bench with 200 papers across 10 venue templates and 13 defect types at different difficulty. Extensive experiments show that PaperFit outperforms all baselines by a large margin, establishing that bridging the gap from compilable source to publicationready PDF requires vision-in-the-loop optimization and that VTO constitutes a critical missing stage in the document automation pipeline. Date: May 12, 2026 Correspondence: Cheng Tan, [email protected] Code
1
Dataset
Introduction
The past decade has witnessed remarkable progress in document automation. Format conversion tools such as Pandoc [139] enable structural transformation from Word and Markdown to LATEX. Document understanding models [20, 190, 49] can reconstruct LATEX source code from PDF files. Recent large language models (LLMs) can generate complete LATEX document frameworks directly from natural descriptions [163, 218]. We refer to this stage collectively as structural formatting, whose primary objective is to produce compilable .tex files. However, compilation success does not guarantee visual quality. A syntactically valid LATEX project may still produce PDFs with misplaced floats, overflowing equations, inconsistent table scaling, widow and orphan lines, and poor page balance [144, 103]. The final page may contain excessive white space that makes the content appear incomplete, or spill into an extra half page that violates strict conference page limits. Currently, resolving these issues relies entirely on manual effort: researchers repeatedly compile the source, inspect the rendered PDF, identify visual defects, adjust the .tex file, and recompile. This compile–inspect–edit cycle, particularly intense in the final hours before submission deadlines, depends almost exclusively on visual judgment that no existing tool fully automates [94]. *Equal contribution.
1
PaperFit: Vision-in-the-Loop Typesetting Optimization for Scientific Documents
Existing approaches fail to automate this process due to three fundamental limitations (Figure 1): (i) incomplete observability. Rule-based tools and compilation logs provide only one-dimensional, code-level signals (Figure 1a). They can detect overfull hbox warnings but cannot judge whether a minor overflow is visually significant, how figure placement affects reading flow, or how white space is distributed across a page. Typesetting quality is inherently a two-dimensional, spatial judgment that source code and logs alone cannot support. (ii) unconstrained repair space. When a model identifies a problem, it faces an enormous action space in which most options are pseudo-fixes: commands such as \vspace, \resizebox, and \newpage produce compilable output but violate implicit typesetting norms by distorting typography, masking issues, or shifting defects elsewhere. Template files define formatting rules for fonts, margins, and headings, yet encode none of the repair preferences that distinguish a legitimate fix from a cosmetic workaround. (iii) unverified cascading effects. LATEX edits are highly non-local: a small change in figure width can trigger page-break rearrangements across the entire document. Text-only LLMs operate in an open loop (Figure 1b), modifying source without rendering or inspecting the result, and thus cannot confirm whether an edit improves or degrades global layout. These challenges characterize typesetting as a closed-loop control problem requiring visual sensing, constrained action, and global verification after every edit. The advancement of vision-language models (VLMs) [88, 185, 221] has made it feasible to automate this closed loop: a model that can both interpret rendered pages and generate LATEX modifications can replicate the human compile–inspect–edit workflow. Naively providing page images to a VLM across multiple rounds is insufficient; without structured diagnosis, constrained repair, and gated validation, the model tends to introduce new defects or ignore page-budget constraints [140, 166]. Based on this insight, we formalize Visual Typesetting Optimization (VTO) as the task of transforming a compilable LATEX paper into a visually polished, page-budget-compliant PDF through iterative visual verification and source-level revision, and introduce a five-category defect taxonomy covering space utilization, float placement, typographic consistency, overflow, and cross-template migration. We position VTO as a critical missing stage between structural formatting and final publication. We present PaperFit, a vision-in-the-loop agent that closes the sense–act–verify loop for typesetting optimization (Figure 1c). It addresses the three challenges above through three design components: multi-source evidence integration fuses source, log, PDF, and page-image signals into structured defect records, resolving incomplete observability; a constrained repair policy explicitly defines permitted operations, forbidden pseudo-fixes, and protected content, taming the unconstrained repair space; and checklist-gated multi-round validation recompiles, re-renders, and re-inspects the full document after every edit, catching cascading effects before they propagate. To benchmark VTO, we construct PaperFit-Bench with 10 venue templates, 200 papers, and 13 defect types at three difficulty levels, and design six baselines that incrementally add capabilities from rule-only to multi-round visual repair. PaperFit achieves perfect compilation and rendering success, the highest visual quality and page-budget compliance, and substantially outperforms all baselines. The most informative comparison is against a naive multi-round visual agent sharing the same page images but lacking structured diagnosis, constrained repair, and gated validation: PaperFit surpasses it by a large margin in both visual quality and page-budget satisfaction, confirming that visual feedback is necessary but not sufficient. These results establish VTO as a critical missing stage in the document automation pipeline and highlight the decisive role of structured visual closed-loop control in producing publication-ready documents.
2
Related Work
2.1
Document Layout Analysis and Automated Formatting
Recent research in document automation primarily emphasizes structural formatting. Early foundational work in sequence modeling [76] and automatic evaluation [154] established the building 2
PaperFit: Vision-in-the-Loop Typesetting Optimization for Scientific Documents
From Code-Level Formatting to Visual Closed-Loop Typesetting Optimization Rule-based tools LaTeX source \LaTeX{ \malfetx{vcont} beginz{consitive resolution{ mase-lonfigure{ \scompile{
Text-only LLM
PaperFit
PDF preview
PDF preview
visual detector
pdflatex or
overflowing
\\sails{consmesresolution{ mansiorizations{ endi} fondelle \LaTeX}
.LaTeX source
edited .tex
no visual feedback
semastic polish
PDF preview
Recompile Render PDF and verify vision-language model Repair Diagnose LaTeX defects semantic
Visual Blindness
Structural Formatting (a) Blind to Rendered Visuals
Inspect visual layout
Compiled-success PDF (b) Open-Loop Code-level Formatting
polish
repair engine
quality gate
Publication-ready PDF (c) Vision-in-the-Loop Optimization
Figure 1: Comparison of typesetting optimization approaches: (a) Rule-based tools are blind to visuals; (b) Text-only LLMs operate in an open loop and cannot predict rendering outcomes; (c) Our PaperFit system establishes a visual closed-loop agent that mimics the iterative human workflow. blocks for later document understanding systems. VTLayout [110] represents a significant milestone by improving content block recognition through the integration of deep and shallow visual features with textual information. This integrated approach is further demonstrated by the LayoutLM series [214, 215], DocFormer [11], and the OCR-free DONUT [100]. More recent efforts have extended document layout analysis to handle complex perturbations [37], generate diverse large-scale layouts [4, 97], and enable global-to-local adaptive perception [2]. These models excel at extracting structure from document images, but their output is a recognized layout or reconstructed markup rather than a visually optimized source file. A parallel line of work focuses on generating compilable LATEX documents from scratch. LLM-driven generators such as Rxiv-Maker [164] produce complete paper frameworks from natural descriptions, cross-lingual formatting systems [218] preserve layout across languages, and agentic writing tools [135, 205] can draft entire manuscripts including LATEX source. Recent systems such as FlexDoc [3] further address document adaptation and compilation efficiency. However, all of these systems treat successful compilation as the terminal goal.
2.2
Vision-Language Models for Visual Code Editing
VLMs have significantly improved the mapping of visual signals to code, particularly in extracting structured representations from documents. Nougat [20] demonstrates this advancement by using a Swin Transformer to convert academic PDFs into markup language, thereby bridging the gap between human- and machine-readable formats. The process of converting images to LaTeX is further supported by benchmarks such as Im2Latex-100K [96] and advanced visual reasoning models like A2 R2 [1]. Additional tools, including Math2LaTeX [150] and Vision-RWKV [55], have expanded the capabilities for mathematical and structural recognition. Nevertheless, a key limitation persists: most models treat LaTeX as a static translation target. LATTE [94] introduced an iterative refinement framework for tables and formulae using visual feedback. Other studies have explored high-fidelity conversion through reinforcement learning for complex table images [126, 90].
2.3
Iterative Self-Refinement and Agentic Frameworks
The development of multi-agent systems has enabled autonomous document optimization through collaborative pipelines. For example, PaperTalker [5] employs a coordinated suite of agents for content parsing, slide generation, and virtual avatar rendering to convert papers into presentation videos. Similar agentic frameworks include Paper2Poster [153], which automates academic poster synthesis, and AutoFigure-Edit [125], which generates editable scientific illustrations. LaTeXAgent 3
PaperFit: Vision-in-the-Loop Typesetting Optimization for Scientific Documents
[56] provides stateful editing capabilities. Recent studies also examine structured translation via multi-agent coordination [253] and domain-specific review feedback [136]. A persistent challenge is establishing a reliable evaluation-optimization loop. Seeing is Improving (VFLM) [71, 70] uses visual rewards to guide iterative text layout refinement, directly addressing readability issues that are invisible at the code level. ReLook [117] applies vision-grounded reinforcement learning to web code generation, and SimpleDoc [89] integrates visual verification into multi-modal document understanding. DocReward [130] proposes learned reward models that score rendered document quality, providing an automated proxy for human visual judgment.
3
The PaperFit-Benchmark
3.1
Overview
We introduce PaperFit-Bench, a benchmark for evaluating automated LaTeX layout repair. Unlike existing benchmarks that assess compilation success or content correctness, PaperFit-Bench operationalizes evaluation as visual layout restoration from systematically perturbed sources. Each instance pairs a perturbed LaTeX source with its original compilable version as ground truth, enabling deterministic evaluation across five defect categories (Class A–E) and three difficulty tiers. The benchmark comprises 200 instances spanning 10 venues and both single- and double-column formats.
3.2
Dataset Construction
Data Collection. LaTeX source code of published papers was retrieved from arXiv, covering multiple subfields of artificial intelligence including nature language processing, computer vision, and reinforcement learning. This diversity mitigates evaluation bias toward any single typesetting style. As shown in Table 1, the resulting corpus spans 10 venue templates covering both single-column formats and double-column formats, with page limits ranging from 7 to 14. Each sample contains an average of 6.3 figures and 5.3 tables, providing substantial floating-element density that exercises the full range of layout repair capabilities. This venue diversity ensures that evaluation is not biased toward any single layout style or page constraint. Table 1: Benchmark papers statistics by conference. Conference
# Papers
Columns
Page Limit
Avg. Figures
Avg. Tables
AAAI ACM MM CVPR/ICCV ECCV ICLR ICML IEEE Trans IJCAI IJCV NeurIPS
20 20 20 20 20 20 20 20 20 20
2 2 1 1 1 2 2 2 2 1
7 8 8 14 9 8 12 7 10+ 9
5.60 7.25 5.25 5.65 6.30 5.55 4.60 5.80 10.75 5.95
5.30 6.40 4.70 4.35 4.85 5.05 3.85 4.70 9.20 4.40
Preprocessing. A standardized compilation test is applied in a controlled build environment; samples that fail compilation or depend on private macro packages are excluded. Appendix sections are uniformly removed. A dual quality-control mechanism combining manual verification ensures that each sample contains at least three figures and at least two tables. Perturbation Design and Difficulty Tiers. We adopt thirteen perturbation strategies organized into five categories aligned with our VTO defect taxonomy (Figure 2): space utilization (Class A), float placement (Class B), table width (Class C), overflow (Class D), and cross-template migration (Class E). 4
PaperFit: Vision-in-the-Loop Typesetting Optimization for Scientific Documents
Figure 2: Perturbation distribution and category composition. The inner ring shows proportions of five perturbation categories (Class A–E); the outer ring shows specific perturbation. A key design principle of PaperFit-Bench is that it prioritizes realism over simplicity. PaperFit-Bench is a mixed-disturbance benchmark rather than a collection of one-defect toy examples. Each case is generated from an academic paper project and is associated with a case metadata record and a disturbance manifest. The benchmark contains three difficulty buckets: easy, medium, and hard. These buckets should be interpreted as empirical difficulty groups, not as deterministic recipes. A hard case, for example, may combine template-transfer pressure, table overflow, and page-budget drift, while an easy case may still contain a nontrivial local table or float issue. These five active disturbance families cover the main visual typesetting optimization failure modes considered in this work. Space-utilization disturbances create widows, orphans, trailing whitespace, column imbalance, or intra-column voids. Float disturbances move figures or tables away from their natural reading position, shrink graphics, or enlarge graphics beyond the available width. Table disturbances create underutilized or overwide tables. Overflow disturbances introduce long unbreakable tokens or single-line equations that exceed the line width. Template-transfer disturbances create width mismatches or page-budget shifts after changing the surrounding template constraints. A complete listing of perturbation strategies, including their implementation details, validation status, and adoption frequencies, is provided in Table 2. Beyond defining the perturbation types themselves, our benchmark construction methodology includes an important additional layer of documentation. The benchmark construction records both the intended perturbation and its concrete source-level realization. This is important because the same high-level defect can appear in different LaTeX forms. For example, an overwide figure may arise from an explicit width larger than \linewidth, while a page-budget shift may arise from template transfer together with a text-height change. The evaluation therefore treats the manifest as the source of disturbance intent, and the compile/render outputs as evidence of the actual realized failure. Each instance is assigned a difficulty tier by the number of co-occurring perturbations: Easy (1–2), Medium (3–4), and Hard (5–8), distributed in a 3:4:3 ratio (Table 3). Cross-template perturbations (E1,
5
PaperFit: Vision-in-the-Loop Typesetting Optimization for Scientific Documents
Table 2: Summary of perturbation strategies. “Validated” column: ✓ = post-compilation semantic verification confirms the defect manifests (e.g., overfull log, page shift, or layout inspection), with results recorded in disturbance manifest.json; × = standard compilation check only, without defectlevel verification. Perturbation ID
Category
Defect
Implementation
Validated
Frequency
A1 widow orphan
A
A1
✓
59
A2 trailing whitespace
A
A2
×
72
A4 column imbalance
A
A4
✓
16
A5 column void
A
A5
✓
13
B1 float to page
B
B1
×
50
B2 oversize graphics
B
B2
×
48
B2 shrink graphics
B
B2
×
51
C1 table resizebox
C
C1
×
73
C2 table oversize
C
C2
×
79
D1 long token overflow
D
D1
×
68
D2 long formula
D
D2
✓
80
E1 template mismatch
E
E1
✓
76
E2 template page shift
E
E2
Force widow/orphan lines via truncated short paragraphs Inject trailing whitespace before bibliography Inject vertical gaps on double-column final pages Insert vertical voids within body text columns Push selected floats to dedicated float-only pages Enlarge graphics beyond available column width Shrink graphics to noticeably undersized width Wrap tables in \resizebox to undersize Widen tables beyond available column width Append unbreakable tokens to trigger line overflow Insert ultra-wide formulas to trigger display overflow Cross-template migration with unreasonable image widths Cross-template migration with reduced \textheight
✓
76
E2) become increasingly prominent in harder instances. Assembly and Finalization. Perturbed sources are assembled into complete problem instances and undergo final quality verification to ensure compilation succeeds and visual perturbations are realized. The final benchmark contains 200 instances. Having completed the description of our benchmark construction pipeline, we now compare PaperFitBench against representative existing document processing benchmarks to highlight its unique characteristics. As summarized in Table 4, PaperFit-Bench fills an important gap in the literature. It is the only benchmark that simultaneously supports systematic perturbation injection, visual evaluation based on rendered page outputs, multi-modal evidence integration, and iterative full-document repair workflows—all essential features for evaluating modern AI-powered LaTeX layout optimization agents.
6
PaperFit: Vision-in-the-Loop Typesetting Optimization for Scientific Documents
Table 3: Distribution of perturbation difficulty levels and most frequently used perturbation types. Difficulty
Samples
Easy
60
Medium
80
Hard
60
Common Perturbation Types A2 (trailing whitespace), C1 (table resizebox), C2 (table oversize), D1 (long token overflow), D2 (long formula), E1 (template mismatch), E2 (template page shift) A2 (trailing whitespace), C1 (table resizebox), C2 (table oversize), D2 (long formula), E2 (template page shift) A2 (trailing whitespace), C2 (table oversize), D1 (long token overflow), D2 (long formula), E1 (template mismatch), E2 (template page shift)
Table 4: Comparison with representative benchmarks. PaperFit-Bench is the only benchmark that combines systematic perturbation injection, visual evaluation from rendered pages, multi-modal evidence chains, and full-document iterative repair. Benchmark
Task
Perturbation
Im2Latex-100K TeXpert RoDLA DocReward LATTE
Formula reconstruction LaTeX code generation Layout robustness Quality assessment Element-level refinement
– – Limited – –
PaperFit-Bench
Visual typesetting repair
13 strategies
4
Method
4.1
Preliminaries
Visual Eval
Multi-Modal
Iterative
✗ ✗ Partial ✗ ✗
✗ ✗ ✓ ✗ ✓
✗ ✗ ✗ ✗ ✓
✓
✓
✓
Let x denote a compilable LATEX project, τ the target template, and b an optional page budget. Executing the compile-render pipeline produces log evidence ℓ, a PDF P (upon successful compilation), rendered page images I, and a page count p. Visual Typesetting Optimization (VTO) seeks a revised source x ∗ that minimizes residual visual defects under hard constraints: x ∗ = arg min ∑d∈D( x′ ) wc(d) s(d) + λe ∆( x, x ′ ) x′ | {z }
(1)
visual defect score
s.t.
C OMPILE ( x ′ , τ ) = success, ′
(2)
R ENDER ( x , τ ) = success,
(3)
′
(4)
C ONTENT ( x ) ⊇ C ONTENT ( x ), ′
| PAGES ( x , τ )| = b (when b is specified),
(5)
where D( x ′ ) is the set of visual defects detected in the rendered pages of x ′ under template τ, each characterized by its category c(d) and severity s(d); wc(d) weights defect categories according to the VTO taxonomy; ∆( x, x ′ ) measures source-level edit distance to encourage minimal, auditable changes; and λe balances edit conservatism against visual improvement. The hard constraints enforce that x ′ compiles and renders under template τ (Eqs. 2–3), preserves all scientific content including figures, tables, captions, labels, citations, and bibliography entries (Eq. 4), and meets the page budget when specified (Eq. 5). Constraints are prioritized in strict order: content preservation > compilation/rendering > page budget > visual quality > edit minimality. Because the 7
PaperFit: Vision-in-the-Loop Typesetting Optimization for Scientific Documents
objective is observable only after compiling and rendering and because even minor source edits can trigger non-local layout cascades, VTO cannot be solved by single-pass generation. We formulate it as an iterative, evidence-driven search with visual verification after every edit.
4.2
Sense: Multi-Source Evidence Integration
No single evidence source reliably captures all typesetting defects. A table may compile without warnings, use a standard tabular environment, and land on the correct page—yet overflow the column boundary. Only the page-image layer reveals this defect; only the source layer can localize the repair target. PaperFit therefore fuses four complementary evidence layers: Source-layer signals (.tex). The source layer provides document structure, template configuration, macro definitions, float environments, table structure, and counts of protected objects such as figures, tables, captions, labels, citations, and bibliography commands. This layer identifies editable regions, safeguards key scientific objects, and reveals structural mismatches resulting from template migration. Log-layer signals (.log). Compilation logs offer deterministic execution evidence, including compile failures, undefined control sequences, unresolved references, missing citations, overfull or underfull warnings, and template-compatibility errors. When the input fails to compile or render, this layer serves as the primary evidence for restoring an executable state. PDF-layer signals (.pdf). The compiled PDF provides document-level outcomes, including final page count, page order, and float landing behavior. This layer helps determine whether the page budget is met and whether floats have drifted far from their first citation. Page-image-layer signals. Rendered pages reveal two-dimensional visual defects that source code or logs cannot reliably detect, such as sparse final pages, double-column column-void artifacts, float stacking, oversized tables, local whitespace, cross-page imbalance, and visual inconsistency. The diagnosis stage converts the collected evidence into structured defect records. d = (c, o, r, e),
(6)
where c ∈ {A, B, C, D, E} is the defect category, o is the location (page and spatial region), r ∈ {blocking, degrading, cosmetic} is the severity, and e is the supporting evidence. These records form the interface between diagnosis and repair: every subsequent edit is traceable to explicit multi-source evidence, and the severity field determines repair priority in the next stage.
4.3
Act: Constrained Repair Policy
Given the defect set Dt , PaperFit must select repair actions from an enormous space, most of which are pseudo-fixes: technically compilable but typographically harmful. We control this space through a repair preference profile π that encodes action tiers, defect-category-specific strategies, forbidden operations, and protected content. 4.3.1
Repair Action Tiers
We categorize all LATEX repair actions into three tiers based on their side-effect risk: • Layout-native (preferred): float re-anchoring ([htbp] parameter adjustment), equation splitting into multiline forms (align, multline), table restructuring with width-aware environments (tabularx, table*), and figure width normalization to template-safe values. These operations address the root cause of the defect without side effects. • Spacing-manipulative (restricted): local \vspace adjustment, \setlength modification, and columnbreak hints are permitted only with explicit local justification and must pass re-verification.
8
PaperFit: Vision-in-the-Loop Typesetting Optimization for Scientific Documents
Figure 3: Overview of the PaperFit pipeline. PaperFit diagnoses layout defects from source, log, PDF, and page-image evidence, applies repairs under a repair preference profile, and validates outputs through a checklist-gated multi-round loop. • Pseudo-fix (forbidden as primary repair): \resizebox on tables, \newpage/\pagebreak for budget control, \scalebox for graphics, and content deletion. These commands may temporarily mask a defect but distort typography, violate template norms, or shift defects to other pages. 4.3.2
Defect-Aware Repair Selection
The repair profile π specifies a priority ordering across defect categories and preferred strategies: • Compile errors (highest): restore compilation via log-guided source repair. • Overflow (D): split long equations; break unbreakable tokens. • Float placement (B): re-anchor floats near first citation; normalize figure widths. • Table consistency (C): replace \resizebox with tabularx; restructure overwide tables. • Space utilization (A): adjust float positions and parameters to eliminate widows/orphans and whitespace. • Cross-template (E): reconcile width/height mismatches from template migration. At each round, the system selects the highest-priority unresolved defect from Dt and applies the top-ranked layout-native strategy for that category. If layout-native options are exhausted, spacingmanipulative actions may be attempted under the restricted policy. 4.3.3
Content Preservation and Semantic Polish Fallback
Before applying any repair, the system snapshots the count and location of all protected objects (figures, tables, captions, labels, citations, and bibliography entries). After repair, it verifies that no protected object has been deleted, displaced across section boundaries, or had its caption altered. Violations trigger automatic rollback to the pre-repair state. When layout-native repairs have been exhausted but minor page-budget gaps, widows/orphans, or sparse final pages persist, PaperFit permits bounded semantic polishing: minimal wording adjustments (e.g., tightening a verbose sentence, replacing a long phrase with a concise equivalent) that do not alter claims, results, numbers, citations, or factual meaning. This fallback is invoked only after layout-native 9
PaperFit: Vision-in-the-Loop Typesetting Optimization for Scientific Documents
options fail and remains subject to the content preservation guards. It serves as a last-resort mechanism rather than a primary repair strategy.
4.4
Verify: Checklist Quality Control
A single post-repair compilation check cannot ensure global layout because LATEX edits are highly non-local: a small change in float width can cascade into page-break rearrangements across the entire document. PaperFit recompiles, re-renders, and re-inspects the complete document after every edit:
St = ( xt , ℓt , Pt , It , Dt , Ht , at ),
(7)
where xt is the current source, ℓt is compile-log evidence, Pt is the PDF, It is the rendered page set, Dt is the structured defect report, Ht is hard-constraint signals, and at stores next actions. Each round follows six steps: (1) compile and collect logs; (2) parse deterministic signals (errors, references, overfull boxes); (3) render all pages; (4) build structured defect records from multi-source evidence; (5) apply constrained repairs per defect category and repair preference profile; (6) recompile/rerender and let the gatekeeper decide. The gatekeeper outputs one of three decisions: DONE (all constraints pass, no blocking residual defects), CONTINUE (safe but issues remain), or BLOCKED (repair is unsafe or infeasible). The DONE checklist requires successful compilation, rendering, page-level visual inspection, absence of blocking defects, page-budget satisfaction, and preservation of protected content.
5
Experiment
5.1
Experimental Setting
We evaluate on PaperFit-Bench (Section 3.2). Each method receives the same LaTeX project and target page budget; outputs are compiled, rendered, and scored with both programmatic checks and VLM-based visual evaluation. 5.1.1
Baselines.
We compare six baselines with PaperFit, spanning three feedback paradigms. Rule-based: Perturbed (unmodified input) and RuleLog (deterministic rule/log repair). Text-only: TextST (single-turn source edit) and TextMR (multi-round source-plus-log edit). Visual: VisualST (single-turn source-plus-image edit) and VisualMR (multi-round visual agent with fixed rounds). These baselines are systematically constructed to isolate the incremental value of each core capability in the visual typesetting optimization pipeline. They differ only in the evidence sources they can access, the number of repair iterations allowed, and whether they incorporate PaperFit’s structured repair machinery: Perturbed: perturbed input. Perturbed is the unmodified disturbed project. It measures the raw difficulty of the benchmark after perturbation and provides the visual reference for VLM pairwise comparison whenever the perturbed input can be rendered. RuleLog: deterministic rule/log repair. RuleLog applies deterministic repair rules driven by source and compile-log signals. It does not use model-based visual feedback. Its role is to test how far simple execution and log repair can go without page-level visual evidence.
10
PaperFit: Vision-in-the-Loop Typesetting Optimization for Scientific Documents
TextST: single-turn text-only model repair. TextST sends the LaTeX source to a model in a single repair turn. It does not inspect rendered page images. This baseline tests whether source-only reasoning can repair layout defects without observing the final pages. TextMR: multi-round text/log repair. TextMR extends TextST with multiple text/log feedback rounds. It can react to compilation errors and logs across rounds, but it still does not use page images as visual evidence. VisualST: single-turn visual repair. VisualST receives the LaTeX source and rendered page images, then performs one visual repair turn. If compilation or rendering fails, the failure is accounted for by the same evaluation protocol rather than being removed from the denominator. VisualST isolates the value and limitation of adding page images without closed-loop visual iteration. VisualMR: naive multi-round visual agent baseline. VisualMR is a fixed-round visual agent baseline. It can inspect source, logs, and page images over a small fixed number of rounds and can directly repair compile, render, and layout issues. It does not use PaperFit’s defect taxonomy, structured diagnosis records, constrained repair policy, repair-plan artifacts, rollback-aware gatekeeper, or PaperFit runtime. This makes VisualMR the closest baseline for testing whether multi-round visual feedback alone is sufficient. PaperFit. PaperFit uses the same basic input project and target page budget, but adds structured multi-source diagnosis, a repair preference profile, and checklist-gated validation. The distinction between VisualMR and PaperFit is therefore not whether a model sees page images, but whether the multi-round process is organized around explicit defects, constrained repairs, and acceptance gates. 5.1.2
Evaluation protocol.
We evaluate all methods using a dual-metric framework that combines programmatic correctness checks and human-aligned visual assessment, ensuring outputs are both technically valid and publication-ready. We report four primary binary metrics: compile success, render success, Page hit (exact page-budget match), and Win rate (fraction of cases judged visually better than the Perturbed baseline). For quantitative composite evaluation, we use two complementary scores: - Program: A 0–5 composite of non-visual execution reliability and content fidelity - VLM: A gated 0–5 visual quality score based on rendered page assessment We report both scores because Visual Typesetting Optimization (VTO) requires outputs to simultaneously satisfy hard technical constraints and subjective visual quality standards. A method that produces visually appealing pages but fails to preserve content or meet page budgets is not acceptable for publication, just as a technically correct but visually defective document fails to meet the core goal of typesetting optimization. Program Score. The Program score summarizes non-visual execution and fidelity signals on a 0–5 scale, computed as the average of five equally weighted dimensions, each normalized to [0, 1]: Program = 5 ·
1 5 sk . 5 k∑ =1
The five dimensions are: • compile reliability: Whether the candidate compiles and renders into usable pages 11
PaperFit: Vision-in-the-Loop Typesetting Optimization for Scientific Documents
• content integrity: Whether protected scientific objects (figures, tables, captions, labels, citations, bibliography entries) are fully preserved • reference quality: Whether all references resolve correctly and no severe log errors remain • page precision: Whether the output satisfies the target page budget, with a penalty for excessive source rewriting • content embedding similarity: Semantic similarity between the original and final LaTeX sources Program is intentionally not a visual score. A method can receive a high Program score by producing a compilable, faithful, page-controlled document even if its final layout still contains visible whitespace or float-quality issues. Conversely, an output that looks acceptable in a rendered screenshot will be penalized by Program if it loses protected content, violates page budget, or leaves unresolved references. VLM Visual Score. The visual evaluation uses rendered page images and produces a gated 0–5 score. It operates in two modes: 1. Pairwise comparison mode: When the Perturbed baseline renders successfully, the evaluator compares the perturbed input and candidate output side-by-side 2. Renderrescue mode: When the Perturbed baseline cannot be rendered, a renderable candidate receives credit for recovering from a non-renderable state The raw VLM score combines three weighted components: VLMraw = 0.35 Sabs + 0.40 Srepair + 0.25 Sfinal . Here: - Sabs measures absolute repair-oriented quality, including defect resolution, constraint alignment, visual quality, new-defect avoidance, and publication readiness - Srepair measures pairwise repair quality relative to Perturbed (when renderable) or render-rescue quality (when Perturbed is not renderable) - Sfinal measures final-paper aesthetics, including professionalism, space utilization, float placement, typographic consistency, and visual balance The final reported VLM score applies strict constraint gates to the raw score: - Non-renderable candidates are capped at the minimum score - Compile-dirty but renderable outputs are penalized and capped - Page-budget failure, unresolved references, or major newly introduced visual defects also trigger score capping The Win rate reported in the main results is the fraction of cases in which a method is judged better than the Perturbed baseline under this full visual evaluation protocol.
5.2
Main Quantitative Results
Table 5: Main results. VLM is the visual evaluation score; Program is the 0–5 composite programmatic score. All other quantities are rates in [0, 1]. Method
Compile ↑
Render ↑
VLM ↑
Win ↑
Program ↑
Page hit ↑
Perturbed RuleLog TextST TextMR VisualST VisualMR PaperFit
0.5800 0.5200 0.5850 0.6100 0.6250 0.9750 1.0000
0.8200 0.7600 0.5850 0.6100 0.6250 0.9750 1.0000
1.8275 2.1838 1.8522 2.1601 1.8741 2.8006 3.3907
0.0000 0.3800 0.2800 0.4250 0.2950 0.6500 0.8950
3.6344 3.3401 2.5738 2.7433 2.7681 4.5789 4.5790
0.3750 0.4444 0.4530 0.6230 0.4560 0.5487 0.8050
Neither text/log feedback nor single-turn visual feedback is sufficient. RuleLog, TextST, TextMR, and VisualST represent progressively richer feedback signals, from compile logs to multi-round text to rendered page images. Yet none exceeds a VLM score of 2.19 or a Win rate of 0.43 (Table 5). Text/log 12
PaperFit: Vision-in-the-Loop Typesetting Optimization for Scientific Documents
methods cannot judge two-dimensional layout failures such as excessive white space or float cascades, while single-turn visual editing often fails to handle non-local cascades. Naive multi-round visual repair improves usability but remains weak on page control. VisualMR reaches 0.975 on both compile and render success, confirming that multi-round visual and log feedback can remove most execution failures. However, its Page hit is only 0.549 and its Win rate is 0.650. Without explicit planning, constrained repair, and gatekeeper validation, multi-round visual editing still struggles to satisfy page budgets and avoid newly introduced visual defects. PaperFit gives the best trade-off between visual quality and constraint satisfaction. PaperFit achieves perfect compile and render success (1.000), the best VLM score (3.391), Win rate (0.895), and Page hit (0.805), with a Program score of 4.579 essentially tied with VisualMR. All methods maintain high content embedding similarity (>0.97), confirming that these gains come from layout-structure repair rather than semantic drift.
5.3
Capability Boundary Comparison
Rather than running additional external systems as direct experimental baselines—which would require substantial engineering adaptation to our specific task—we use recent papers and widely adopted open-source projects as external capability anchors. This choice avoids conflating method capability with engineering adaptation: existing external systems target related but different problems. Some systems specialize in parsing PDFs or document structure, some reconstruct local LaTeX objects from images, and some edit code repositories through command-line feedback. None of these system families directly targets full-paper visual typesetting repair for an existing LaTeX project. Table 6: External capability boundary matrix. System families: DocParser denotes PDF/document parsers, including MinerU, Marker, and Nougat [190, 49, 20]; LocalRecon denotes local LaTeX reconstruction systems, including LATTE, Table2LaTeX-RL, and LaTeX-OCR [94, 126, 19]; CodeAgent denotes general coding agents, including OpenHands, Aider, and SWE-agent [195, 66, 222]; B5-VisualAgent is our general-purpose visual coding-agent baseline; PaperFit is our full system. Capability abbreviations: MSI = multi-source input; Edit = LaTeX/code generation or editing; Loop = execution feedback loop; PVD = full-paper visual diagnosis from rendered page images; Layout = float/table/page-level layout repair; Gate = page-budget, template, and checklist-gated validation. ✓indicates full coverage, ✗indicates no coverage, and △ indicates partial coverage. System family
MSI
Edit
Loop
PVD
Layout
Gate
DocParser [190, 49, 20] LocalRecon [94, 126, 19] CodeAgent [195, 66, 222] B5-VisualAgent PaperFit
✓ ✓ ✓ ✓ ✓
✗ ✓ ✓ ✓ ✓
✗ △ ✓ ✓ ✓
✗ ✗ ✗ △ ✓
✗ ✗ ✗ △ ✓
✗ ✗ ✗ ✗ ✓
In Table 6, multi-source input means that a system can process at least one task-relevant input modality, such as PDFs, page images, local object images, code repositories, or textual instructions. DP systems extract text, equations, tables, and layout structure from PDF or document inputs, but their objective is document understanding or PDF-to-markup conversion rather than source-level repair. LR systems recover local LaTeX objects from formula or table images, but object-level image-to-LaTeX reconstruction is not equivalent to full-paper layout repair. CA systems can edit code repositories and iterate with command-line feedback, making them the closest external capability class to PaperFit; however, their feedback loop is usually organized around software task success or test passing, not visual diagnosis over page images rendered from compiled PDFs. 13
PaperFit: Vision-in-the-Loop Typesetting Optimization for Scientific Documents
VisualMR instantiates the general-purpose visual coding-agent capability class in the controlled setting. It can inspect files, run commands, compile and render the project, view page images, and edit LaTeX over fixed rounds. However, VisualMR is denied PaperFit’s VTO taxonomy, structured repair plans, constrained repair policy, runtime state management, and checklist-gated validation. VisualMR is therefore marked as partial for full-paper visual diagnosis and layout repair, and as absent for page-budget, template, and gatekeeper constraints. The capability matrix shows that external systems cover different local segments of the PaperFit capability chain, but no external family simultaneously covers multi-source input, LaTeX/code editing, execution feedback, page-image-based full-paper diagnosis, float/table/page-level repair, and pagebudget/template/gatekeeper constraints. PaperFit’s contribution is not any single input parser, code editor, or local LaTeX recognizer; it is the integration of these capabilities into a full-paper visual typesetting optimization loop for existing LaTeX projects.
5.4
Model Backend Comparison
To isolate the role of the language-model backend, we run the same PaperFit workflow with four diverse LLMs on 20 representative cases. Table 7 and Figures 4–5 show three consistent patterns. Table 7: Model comparison on 20 representative cases (6 easy, 8 medium, 6 hard). LLM Backend
Compile ↑
Render ↑
VLM ↑
Win ↑
Page hit ↑
GPT-5.4 [149] Claude Opus 4.6 [10] DeepSeek-V4 Pro [50] MiMo-v2.5-pro [210]
100.00% 100.00% 95.00% 100.00%
100.00% 100.00% 100.00% 100.00%
3.656 3.548 3.521 3.652
95.00% 90.00% 95.00% 100.00%
95.00% 100.00% 100.00% 95.00%
Aggregate performance is stable across backends. All backends obtain high VLM scores (3.52–3.66), strong win rates (90–100%), and near-perfect compile/render reliability. The overall VLM spread is only 0.14 points, far smaller than the 0.59-point gap between PaperFit and VisualMR in Table 5, suggesting that the main improvement is from PaperFit rather than a particular model. Backend differences reflect repair style rather than a single dominant model. Figure 4(a) shows that MiMo-v2.5 has the strongest repair-oriented profile, leading in defect resolution (3.90), visual quality (3.85), and publication readiness (3.80). GPT-5.4 instead leads in new-defect avoidance (4.30) and remains competitive on constraint alignment, which explains why its gated visual score slightly exceeds MiMo despite a lower raw visual score. The residual bottleneck is visual balance, not execution reliability. Figure 4(b) shows that DeepSeekV4 leads in space utilization (3.50), float placement (3.90), and visual balance (3.20), while MiMo-v2.5 has the highest overall professionalism (3.85). However, across all backends, typographic consistency is consistently high, whereas space utilization and visual balance remain the weakest dimensions. Venue-level results in Figure 5 further show that no backend dominates uniformly across templates. Difficulty-split results. Table 8 reports the difficulty-split VLM scores for the four LLM backends. The VLM spread remains ≤0.14 within each difficulty tier, and no single backend dominates across all three levels—GPT-5.4 leads on easy and medium cases while DeepSeek-V4 Pro achieves the highest score on hard cases. This cross-over pattern confirms that the ranking reflects stochastic variation rather than a systematic backend advantage.
5.5
Human–VLM Evaluation Correlation
To assess alignment between human judgments and automated scores, the Spearman correlation coefficient (r) is computed between VLM scores and average human ratings across all methods. As shown in 14
PaperFit: Vision-in-the-Loop Typesetting Optimization for Scientific Documents
Table 8: Difficulty-split VLM scores for the LLM comparison (20 cases). All four backends remain effective across difficulty levels, with score spread ≤ 0.14 on each split. LLM Backend GPT-5.4 [149] Claude Opus 4.6 [10] DeepSeek-V4 Pro [50] MiMo-v2.5-pro [210]
Easy (n=6)
Medium (n=8)
Hard (n=6)
3.821 3.598 3.163 3.648
3.792 3.442 3.509 3.778
3.310 3.638 3.893 3.486
(a) Repair and Constraint Dimensions
(b) Final Aesthetic Dimensions
Figure 4: Fine-grained VLM scores for the LLM backend comparison. Panel (a) reports repair and constraint dimensions, and panel (b) reports final aesthetic dimensions. Figure 6, the overall correlation is exceptionally high (r = 0.8571), confirming that the automated metric closely reflects human-perceived quality and faithfully captures model performance trends in the typesetting repair domain.
5.6
Qualitative Case Study
Figure 7– 10 presents the qualitative cases spanning distinct VTO modes.