arXiv CSopen access

Quality-Assured Fuzz Harness Generation via the Four Principles Framework

arXiv:2605.21824v1 [cs.CR] 20 May 2026

Quality-Assured Fuzz Harness Generation via the Four Principles Framework Ze Sheng

Dmitrijs Trizna

Luigino Camastra

Texas A&M University USA [email protected]

Aisle USA [email protected]

Aisle USA [email protected]

Zhicheng Chen

Qingxiao Xu

Jeff Huang

Texas A&M University USA [email protected]

Texas A&M University USA [email protected]

Texas A&M University USA [email protected]

Abstract

CCS Concepts

Fuzz testing is the dominant technique for finding memory-safety vulnerabilities in C/C++ software, yet its effectiveness hinges on the quality of fuzz harnesses—the programs that bridge fuzzers and library APIs. A growing body of tools now automate harness generation, but none systematically ensures the correctness of produced harnesses: logic errors, API misuse, and lifecycle violations go undetected at the source level, producing false-positive crash reports at rates as high as 94%. As LLM-driven generation scales harness creation, uncontrolled quality turns scale into a liability. We present QuartetFuzz, an autonomous harness-generation system that systematically improves correctness throughout the generation process. At its core is the Four Principles framework— Logic Correctness (P1), API Protocol Compliance (P2), Security Boundary Respect (P3), and Entry Point Adequacy (P4)—the first source-level definition of harness correctness with mathematical specifications and implementable checks. We operationalize these principles in an autonomous LLM agent that produces harnesses satisfying P1–P4 through a generate–check–fix loop before any fuzzing begins. Deployed on 23 open-source projects spanning C/C++, Java, and JavaScript, the system submits 42 bug reports, of which 29 are fixed or confirmed upstream (including 3 CVEs) and only 2 are rejected (4.8% FP rate). During generation, the built-in P1/P2 checks automatically intercepted 58 harness-induced crashes that would otherwise have been false positives. Applied as a quality auditor to 586 existing production harnesses across 70 projects, the system identifies 53 violations (45 confirmed, 35 fixed). We release a dataset of 100 labeled harnesses for reproducible evaluation. Code and dataset are available at https://github.com/OwenSanzas/ QuartetFuzz.

• Security and privacy → Software security engineering; • Software and its engineering → Software testing and debugging.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]. Conference’17, Washington, DC, USA © 2026 Copyright held by the owner/author(s). Publication rights licensed to ACM. ACM ISBN 978-x-xxxx-xxxx-x/YYYY/MM https://doi.org/10.1145/nnnnnnn.nnnnnnn

Keywords fuzz testing, fuzz harness, harness quality, LLM agent, vulnerability discovery ACM Reference Format: Ze Sheng, Dmitrijs Trizna, Luigino Camastra, Zhicheng Chen, Qingxiao Xu, and Jeff Huang. 2026. Quality-Assured Fuzz Harness Generation via the Four Principles Framework. In . ACM, New York, NY, USA, 22 pages. https://doi.org/10.1145/nnnnnnn.nnnnnnn

1

Introduction

Fuzz testing [34] is one of the most effective techniques for discovering memory-safety vulnerabilities in C/C++ software, but for libraries its success depends on the quality of the fuzz harness: the code that translates raw fuzzer bytes into library API calls. A harness does more than make the target compile. It decides which functionality is exercised, how deeply the fuzzer can reach, and whether observed crashes reflect real library bugs or mistakes in the harness. A wrong harness undermines the campaign’s effectiveness; the crashes it produces become hard to interpret. This problem has become more severe as automated harness generation has improved. Traditional systems such as FuzzGen [19], FUDGE [5], Hopper [7], and AFGen [30], and more recent LLMbased systems such as OSS-Fuzz-Gen [28], PromptFuzz [32], CKGFuzzer [45], PromeFuzz [29], and HarnessAgent [48] all aim to reduce the effort of writing harnesses and to improve fuzzing reach. Yet they still evaluate quality largely through proxy signals such as build success, crash counts, and code coverage; the only prior pregeneration checker, Scheduzz [24], validates types, not API protocol. Those signals are necessary, but they are not sufficient: a harness can compile, run, and even reach substantial code coverage while still violating API contracts, masking deep code paths, or producing false-positive crashes—at rates as high as 94% in prior work [27]. As LLMs make harness generation easy to scale, uncontrolled harness quality turns that scale into a liability.

Conference’17, July 2017, Washington, DC, USA

24% 8% 10%

58%

Ze Sheng, Dmitrijs Trizna, Luigino Camastra, Zhicheng Chen, Qingxiao Xu, and Jeff Huang

Receptive (12) Concerned (4) Declined (5) No Reply (29)

Figure 1: Developer responses when contacted about AIassisted harness generation across 50 OSS-Fuzz projects. 58% never responded; among respondents, reactions ranged from receptive (24%) to outright refusal (10%), with concerns centered on false-positive overload from prior automated tools.

We saw this concern directly in practice. When we contacted maintainers of 50 OSS-Fuzz projects about collaborating on AIgenerated harnesses and bug reports, the dominant concern was not generation speed or raw coverage but trust: maintainers had prior experience with automated fuzzing workflows that produced large numbers of false-positive reports whose sanitizer traces looked legitimate, but whose root cause was a defect in the harness rather than the library (Figure 1). In other words, the cost of poor harness quality is not only wasted CPU time; it is shifted triage burden for maintainers. This motivates a different question from most prior work: not merely whether a harness runs, but whether it is correct. To investigate, we manually audited 586 production OSS-Fuzz harnesses across 70 C/C++ projects (§5.2); informal reviews of harnesses from LLM4FDG [52], PromptFuzz [32], and PromeFuzz [29] showed the same patterns. From fuzzers exhibiting abnormal coverage, false-positive crashes, or runtime errors, we identified four correctness principles a harness must satisfy: P1, Logic Correctness—the harness itself must be free of bugs such as stale state, resource leaks, or incorrect data handling; P2, API Protocol Compliance—the harness must call the target library in a valid order with valid object lifecycles and parameter constraints; P3, Security Boundary Respect—the harness should exercise public interfaces rather than bypassing validation through internal-only code; and P4, Entry Point Adequacy—the harness should target meaningful, attack-surface-relevant entry points rather than incidental helper logic. We call these the Four Principles of fuzz harness quality. Guided by these principles, we build QuartetFuzz, an LLM-based harness generation system that checks P1–P4 during generation, before any fuzzing campaign begins. The system operates in four stages: (1) an entry-selection agent uses the call graph to enumerate candidate fuzzing targets, ensures each can reach its core implementation, prefers public APIs, and ranks them by a static danger score; (2) an API-research agent collects protocol information (call order, object lifecycle, parameter constraints) for the selected targets; (3) a harness-generator agent drafts a fuzzer source in full and runs a

bounded build loop; and (4) an adversarial-validation gate forces the harness through reach- and run-check probes, routing any crash through P1/P2/P3/P4 triage before submission. Prior generators— including LLM-based ones—target individual functions in isolation, reducing the LLM to a wrapper writer despite its demonstrated ability to comprehend code semantics across files [38]. Our system instead organizes fuzzing around Logic Groups—functional units (e.g., “parse a PNG image,” “complete a TLS handshake”) that capture stateful interactions across related APIs. Our results show that harness quality is not a secondary nicety but a practical bottleneck. Applied as a quality auditor to production harnesses, QuartetFuzz identifies 53 quality violations; 45 have been confirmed by maintainers and 35 have been fixed or merged upstream. Repairing those defects exposed 2 latent library bugs the harnesses had been masking, including a stack-buffer-overread in OpenSSL’s DES implementation that had been latent for over 25 years. On a dataset of 100 gold-standard harnesses curated from the audit (39 projects, all verified P1–P4 clean), the generator matches human-written harnesses on coverage (TOST ±2pp equivalence, 𝑝<10−10 on both line and branch) while outperforming OSS-Fuzz-Gen by 6.9–8.3pp and PromeFuzz by 4.1–5.2pp across line and branch coverage. Deployed on 23 open-source projects spanning C/C++, Java, and JavaScript, QuartetFuzz submitted 42 bug reports: 29 are fixed or confirmed upstream (including 3 CVEs) and only 2 were rejected as false positives (4.8% FP rate). Across audit and generation, the same P1/P2 checks blocked 58 harnessinduced crashes (14 in audited production harnesses, 44 in generated ones) before they could become false-positive bug reports. The 35 merged repairs and four follow-up adoptions support the paper’s central claim: harness quality should be checked before fuzzing begins, not inferred afterward from crash logs. In summary, this paper makes the following contributions. The Four Principles framework with Adversarial Probing. We define harness correctness as source-level conditions (P1–P4), operationalised by Adversarial Probing (P1/P2 runtime sub-checks) and static call-graph reachability (P3/P4 boundary sub-checks). To our knowledge, no prior LLM-based generator turns the agent on its own output before deployment. End-to-end pipeline producing deliverable harnesses. We present a four-stage LLM-agent pipeline (Logic Group selection, API protocol research, static-driven build, adversarial validation) that produces quality-assured harnesses ready for direct upstream merge. Applied as a post-hoc auditor on 586 production OSS-Fuzz harnesses, the same module identified 53 quality violations of which 45 are confirmed and 35 merged upstream—each is a deliverable harness running in or queued for production OSS-Fuzz. A curated evaluation dataset. From the 586 audited production harnesses, we select 100 P1–P4-clean harnesses across 39 projects as a gold-standard dataset for evaluating harness generation against real-world developer-written fuzzers. The dataset and evaluation pipeline are released for reproducible comparison. Real-world bug discovery. Deployed on 23 open-source projects spanning C/C++, Java, and JavaScript, the pipeline yielded 42 bug reports (29 fixed or confirmed, 3 CVEs, 4.8% FP); audit repairs additionally exposed 2 latent library bugs, including an OpenSSL DES stack-buffer-overread latent for over 25 years, and 4 maintainer teams integrated our harnesses or derived artifacts.

Quality-Assured Fuzz Harness Generation via the Four Principles Framework

2 Background 2.1 Fuzz Testing and Fuzz Harnesses Coverage-guided fuzz testing [22] repeatedly feeds mutated inputs to a program, using code coverage feedback to explore new execution paths. Fuzzers such as libFuzzer [31] and AFL++ [12], typically paired with sanitizers such as AddressSanitizer [36] and LeakSanitizer [17], are standard tools for discovering memory-safety vulnerabilities in C/C++ software. For standalone programs, the fuzzer can target the main entry point directly. For library code, which exposes APIs but lacks a standalone entry point, a fuzz harness (also called a fuzz driver or fuzz target) must be written. A libFuzzer harness implements the following interface: 1 2 3 4 5

int LLVMFuzzerTestOneInput ( const uint8_t * data , size_t size ) { // parse data , call library APIs , clean up return 0; }

The fuzzer engine calls this function repeatedly with different byte sequences. The harness is responsible for (1) parsing raw bytes into typed arguments, (2) invoking library APIs in a meaningful sequence, (3) handling errors without crashing, and (4) releasing all allocated resources before returning. Google’s OSS-Fuzz platform [15] continuously fuzzes over 1,000 open-source projects with handwritten harnesses. We focus on C/C++ library harnesses in OSS-Fuzz throughout.

2.2

Automated Harness Generation

Fuzz harness generation can be formulated as a program synthesis problem. The inputs are a codebase L with API set A = {𝑎 1, . . . , 𝑎𝑛 }, and a target specification 𝜏 at a chosen granularity—a single API function, a co-called API group, or a higher-level functionality description (e.g., “PNG decoding”). The task is to synthesize a program 𝐻 that implements the fuzzer interface, maps raw bytes to typed arguments, and composes a call sequence over a subset 𝑆 ⊆ A that exercises 𝜏. A valid harness must satisfy 𝐻 |= C(L, 𝜏), where C is the set of conditions 𝐻 must pass to be accepted as a correct harness for 𝜏 in L. Both the granularity at which 𝜏 is specified and the contents of C vary by methodology. Existing tools fix the granularity of 𝜏—single API in OSS-FuzzGen [28] and HarnessAgent [48], API clusters in PromptFuzz [32], CKGFuzzer [45], PromeFuzz [29], and Scheduzz [24], internal target functions in pre-LLM tools [5, 19–21, 50]—and run a generate– compile–fix–validate loop with C = {Ccall, Cfuzz, Ccov }, where Ccall requires 𝐻 to invoke the target function [48], Cfuzz requires shortduration fuzzing (30 s in OSS-Fuzz-Gen [28], 60 s in LLM4FDG [52]) not to surface crashes traceable to harness bugs, and Ccov requires 𝐻 ’s execution to increase coverage on the target.

3

System Design

QuartetFuzz primarily targets C/C++ open-source projects, producing LibFuzzer harnesses; Logic Group ranking consumes a precomputed call graph of the project. The pipeline separates LLM reasoning from this call-graph input cleanly enough that the same flow applies to languages where the call graph is unavailable: §5.6

Conference’17, July 2017, Washington, DC, USA

extends to Java and JavaScript with LLM-only Logic Group ranking. The four principles in §3.1 characterize harness quality; the four-stage pipeline (§3.4) checks all four before fuzzing.

3.1

The Four Principles

We extend the validation set C from §2.2 with four source-level conditions, P1–P4, checked statically and dynamically before deployment fuzzing. Each yields a pass/fail verdict; a harness is accepted only when all four pass. Setup and notation. P1 and P2 are runtime properties of 𝐻 . P3 and P4 are call-graph properties (§3.3). 𝐻 targets a specification 𝜏, given as a Logic Group with entry set 𝐸𝜏 ⊆ A and core set 𝐶𝜏 ⊆ A. Let A be the library API set and B the space of fuzz blobs. A fuzz blob se® := 𝐻 (𝑏 1 ); . . . ; 𝐻 (𝑏 𝑁 ) in quence 𝑏® = (𝑏 1, . . . , 𝑏 𝑁 ) ∈ B 𝑁 drives 𝐻 (𝑏) one shared process (the libFuzzer model). For each 𝑏𝑖 , 𝐻 produces a trace 𝑇𝐻 (𝑏𝑖 ) = ⟨(𝑎 1, args1 ), . . . , (𝑎𝑘 , args𝑘 )⟩, the prefix of calls (possibly empty) issued before 𝐻 crashes or returns. direct(𝑇 ) replays 𝑇 in a fresh process with the exact arguments the library originally observed and no harness-level parsing or state. direct is undefined when args 𝑗 depends on harness-internal state; §4 approximates it via per-iteration sub-process isolation. P1 uses two fault oracles. 𝑂 single detects faults that show up within a single trace; 𝑂 seq detects faults that show up only across iterations. We instantiate 𝑂 single as AddressSanitizer [36] and 𝑂 seq as AddressSanitizer + LeakSanitizer [17]. crash𝑂 (·) is true when oracle 𝑂 reports a fault on the given execution. In this work we assume execution is deterministic, so flaky crashes are out of scope. Both direct and 𝑂 single serve as semantic references and are not literally executed; 𝑂 seq is realized by libFuzzer with ASan and LSan in the build–fix loop. Principle 1: Logic Correctness (P1). P1 requires
∀𝑏® : ∀𝑖 ∈ {1, . . . , 𝑁 } : ¬crash𝑂 single (direct(𝑇𝐻 (𝑏𝑖 ))) (1) ® =⇒ ¬crash𝑂 seq (𝐻 (𝑏)). The implication separates harness faults from per-call API behavior; this requires asymmetric oracles, since an end-of-process oracle (LSan) in 𝑂 single would vacuously violate the antecedent on any leaking trace. Faults that propagate through API arguments escape the mathematical property and are caught by the syntactic checks P1.1–P1.8 (§3.8). Principle 2: API Protocol Compliance (P2). Each library function 𝑓 has a usage protocol 𝑃 𝑓 with language 𝐿(𝑃 𝑓 ) over its protocolrelated operations deps(𝑓 ) ⊆ A (initialisers, parameter allocators, paired cleanups, return consumers). calls(𝑇 ) and 𝑇 |𝑆 denote the standard call set and order-preserving projection. P2 requires ® ∀𝑖, ∀𝑓 ∈ calls(𝑇𝐻 (𝑏𝑖 )) : 𝑇𝐻 (𝑏𝑖 )| deps( 𝑓 ) ∈ 𝐿(𝑃 𝑓 ). ∀𝑏,

(2)

𝐿(𝑃 𝑓 ) is partial. §3.6 approximates it from headers, comments, and source. §3.8 realizes membership as eight checks P2.1–P2.8 (init, parameter construction, lifecycle, return handling, cleanup, API existence, co-call, prerequisite state). P2 is a property of the trace, independent of crashes; when a trace violates both P1 and P2, we attribute to P2. Principle 3: Security Boundary Respect (P3). Let Pub ⊆ A be the public API surface and HPub the set of P2-respecting public-only harnesses. crash𝑂 (𝑋, 𝐴) holds when oracle 𝑂 fires at library code

Conference’17, July 2017, Washington, DC, USA

Ze Sheng, Dmitrijs Trizna, Luigino Camastra, Zhicheng Chen, Qingxiao Xu, and Jeff Huang

site 𝐴 (function, line). P3 requires every 𝐻 crash to be reproducible by some public-only harness at the same site: ® 𝐴 : crash𝑂 (𝐻 (𝑏), ® 𝐴) ∀𝑏, =⇒ ∃𝐻 ′ ∈ HPub, 𝑏®′ : crash𝑂 (𝐻 ′ (𝑏®′ ), 𝐴).

(3)

A P3 violation is a crash reachable only via internal entries that bypass library defenses; no public-only harness can reproduce it, so it falls outside the threat model. P3 is not statically decidable; static reachability gives a necessary condition, and §3.5 details the fallback when no public entry reaches 𝐶𝜏 . Principle 4: Entry Point Adequacy (P4). Let ⇝ denote reachability in the call graph and unsafeReach(𝑒) a predicate that holds when 𝑒’s reach closure contains memory-safety-relevant operations. P4 requires every fuzz-consuming entry to reach the target core along a path that exposes unsafe operations: ∀𝑒 ∈ 𝐸𝜏 : 𝑒 ⇝ 𝐶𝜏 ∧ unsafeReach(𝑒).

(4)

The first conjunct rules out entries that cannot reach the target; the second rules out entries whose reach closure contains no memorysafety-relevant operations (e.g., wrappers into logging or accessors). We instantiate unsafeReach(·) as danger(·) > 0 using the depthdiscounted unsafe-operation score from §3.5 (Eq. 5); other scorings (e.g., static taint, sanitizer allow-lists) realize the same property. The four principles are ordered by scope. P1 concerns the harness, P2 the harness–library interface, P3 the public versus internal boundary, and P4 the link to real-world usage. The generator applies them in reverse (P4 first, P1 last) because checking code correctness on the wrong entry is wasted. P1 and P2 (runtime) are operationalised by Adversarial Probing (§3.2); P3 by static callgraph reachability (§3.5); P4 by static reachability and a Stage 4 reach check (§3.8).

𝐿(𝑃 𝑓 ) is only partially knowable from sources. We bridge this gap with Adversarial Probing (AP): two probe operations on input blobs (Figure 2)—a reach check (does the blob drive execution into the target API?) and a run check (does the harness crash under ASan/LSan?). The agent supplies adversarial blobs by writing code that generates a blob designed to trigger a suspected violation of a P1.x / P2.x sub-check; libFuzzer also feeds inputs during the build–fix loop. Four P1/P2 sub-checks resist probing (a single input cannot trigger them) and are decided statically (Table 8). AP fires only in Stage 4 (§3.8); Stages 1–3 use source-level analysis only, mirroring how a human author first studies an API before testing. P1.x and P2.x are therefore not LLM-as-checklist box-ticking but specific targets that probes attempt to violate. The agent reads its harness, selects from Table 8 the sub-checks the code looks most likely to violate, and gives each one attack attempt; the Stage 4 submission gate requires every selected probe to pass (no harness-bug crash) before the harness is eligible. To our knowledge, no prior LLMbased generator inspects its harness’s call sequence or constructs adversarial inputs against itself during generation.

3.3

Logic Group: Transliteration rule processing (ICU) Entry functions: Transliterator::createFromRules — create transliterator from rule string

Adversarial Probing: Two Probe Operations

Core functions: TransliterationRule — parse and compile rules TransliterationRuleParser — tokenize rule syntax UnicodeSet::applyPattern — resolve character classes

Given an input blob, AP runs the harness in one of two ways: Probe 1 — Reach check blob → run binary under GDB with breakpoint at target API → hit/miss + functions actually called Targets: P2 sub-checks needing observed call/lifecycle

Description: Converts text between scripts using user-supplied transliteration rules; parses untrusted rule strings with complex syntax.

Probe 2 — Run check blob → run binary under libFuzzer with ASan/LSan → pass/fail + sanitizer trace on fault Targets: P1 logic-correctness sub-checks Adversarial framing: the agent reads its own harness, picks the P1.x / P2.x sub-checks (e.g., P1.4 input flow, P1.6 size guard, P2.1 init order, P2.4 return value, P2.8 prerequisite state) the code looks most likely to violate, and writes a blob aimed at triggering each. Stage 4 admits the harness only if every selected probe passes.

Result: UAF vulnerability found (confirmed and fixed upstream), ranked #1 by danger score among 5 candidates.

Figure 4: An example Logic Group generated by QuartetFuzz for ICU.

3.4 Figure 2: The two probing operations of Adversarial Probing.

3.2

Adversarial Probing

The mathematical definitions of P1 and P2 resist direct execution: direct is undefined when args 𝑗 depends on harness state, and

Logic Groups

We organize fuzzing around features rather than single functions. A Logic Group LG = (name, 𝐸, 𝐶, desc) captures a feature. 𝐸 is the entry set of library APIs receiving fuzzer-controlled bytes. 𝐶 is a curated set of core functions transitively called from 𝐸 that implement the feature. name and desc label the feature and describe its security relevance. We write LG𝜏 with 𝐸𝜏 , 𝐶𝜏 when emphasising a target 𝜏. Figure 4 shows an example.

System Overview

Figure 3 shows the pipeline. Stage 1 identifies 5–10 candidate Logic Groups, ranks them by danger, and selects the top 5. Stage 2 collects API-protocol information for each selected LG. Stage 3 (static-driven build) uses only source reading and call-graph queries to produce a compilable binary. Stage 4 (adversarial validation) forces the harness through reach- and run-check probes plus crash triage before

Quality-Assured Fuzz Harness Generation via the Four Principles Framework

Conference’17, July 2017, Washington, DC, USA

Figure 3: System overview. Stage 1: the Logic Group agent explores the project, identifies candidate functionalities, and ranks entry points by danger score (P3/P4 checked). For each selected LG, Stage 2 researches API protocol from headers and callers (P2). Stage 3 produces a compilable binary via an LLM-drafted source plus a bounded build loop. Stage 4 puts that binary through libFuzzer and agent-selected adversarial probes; any crash is triaged as harness bug (fix), boundary issue (re-select entry), or real vulnerability (submit upstream).

submission. Stages 3 and 4 share a single Harness Generator agent; the agent invokes its build, fuzz, and AP-probe tools without a strict step ordering—we do not enforce a fixed tool sequence between Stage 3 and Stage 4 (full tool list in Table 5, Appendix C).

3.5

Stage 1: Logic Group Discovery

The Logic Group agent reads the project source and generates 5–10 candidate Logic Groups, then selects the top 5. P3 and P4 are the primary checks in this stage. The pipeline runs in five steps. Step 1: Existing coverage. If the project already has fuzzers, the agent reads them and reconstructs the LGs they cover, Gexist , so subsequent candidates target uncovered ground. Step 2: Candidate identification. The agent identifies 5–10 candidate features different from Gexist . For each candidate it sketches a tentative 𝐶 0 of 3–5 core functions (Figure 4). Step 3: Entry selection (P3/P4). For each candidate, the agent refines 𝐶 0 into 𝐶 by using SAST queries and source reading, dropping members that are not internal or central and adding adjacent helpers. The agent then performs a reverse caller search (after AFGen [30]) to find public APIs reaching 𝐶: 𝐸 pub = {𝑓 ∈ Apub | 𝐶 ∩ callees∗ (𝑓 ) ≠ ∅}. Equivalently, the agent walks callers backward from each 𝑐 ∈ 𝐶 and keeps the public ancestors. Apub is built coarsely from path markers (include/, public/, api/) and refined by LLM judgment for ambiguous symbols (no clear marker—the agent reads the declaring header to decide). If 𝐸 pub ≠ ∅, P3 holds and the agent picks the entry with the shortest path to 𝐶 (P4 preference). If 𝐸 pub = ∅, no

public API reaches 𝐶; the agent browses the call graph and picks an internal entry that reaches 𝐶 while preserving as many boundaries as possible (best-effort P3). Step 4: Assembly and dedup. The agent assembles the LG and semantically deduplicates it against Gexist and previously-emitted LGs from prompt context. When two new LGs share an entry, the LLM keeps both if they exercise distinct cores. Step 5: Ranking. We rank the remaining LGs by a static danger score with 1/𝑑 depth discount, after IntelliGen [53]: ∑︁ unsafe(𝑔) danger(𝑓 ) = (5) 𝑑 (𝑓 , 𝑔) 𝑔 ∈ reachable( 𝑓 , 𝐷 )

where reachable(𝑓 , 𝐷) is the set of functions reachable from 𝑓 within 𝐷 hops on the static call graph (𝐷=20; 𝐷=20 sits on a stable plateau in our sensitivity analysis, Appendix B), 𝑑 (𝑓 , 𝑔) is the shortest-path distance from 𝑓 to 𝑔, and unsafe(𝑔) counts pointer dereferences and memory operations (memcpy, malloc, free, strcpy, sprintf, etc.) in 𝑔’s implementation. By design, the 1/𝑑 factor concentrates the ranking on the directly-reachable attack surface—close unsafe operations that empty-corpus fuzzing can exercise without depending on a seed corpus. When a Logic Group has multiple entry functions, its score is danger(LG) = max 𝑓 ∈𝐸 danger(𝑓 ). Stage 1 outputs the top-5 ranked Logic Groups for processing in Stages 2 through 4. Why not rank by score alone? We avoid pure danger-score ranking for three reasons. (1) It wastes the LLM’s code comprehension. The model identifies parsers, protocol handlers, and state machines that no static metric captures. (2) Greedy ranking biases toward

Conference’17, July 2017, Washington, DC, USA

Ze Sheng, Dmitrijs Trizna, Luigino Camastra, Zhicheng Chen, Qingxiao Xu, and Jeff Huang

large complex functions and skips simpler but security-critical code. (3) Scoring all functions per project is slow.

against the binary, aggregate the per-blob results, and return the aggregated output to the agent, which reviews all outcomes together. Every outcome falls into one of four classes:

3.6

(1) Crash, harness bug. Sanitizer fires; LLM judges the cause as a P1/P2 violation in the harness itself. Violation — back to the build-fix loop. (2) No crash, reach miss. The blob did not drive execution into the target API. Violation — back to the build-fix loop. (3) Crash, real library bug. Sanitizer fires; LLM judges the cause as a defect in the library (P1–P4 clean on the harness). Not a violation — submit sanitizer report and harness artifact upstream. (4) No crash, reach OK. The harness handled the attack as intended. Not a violation — continue.

Stage 2: API Research

P2 violations (wrong call order, missing init, incorrect cleanup) are the dominant failure mode of both human and LLM harnesses (§5.2). Most prior systems give the LLM only partial protocol context: API correlation metrics over documentation [29], function signatures from project introspection [28], or post-hoc crash-derived constraints [30]. We instead run a dedicated API Research agent that pre-collects protocol information before the generator writes code. With full read access to the project source, the agent works like a human library expert—performing semantic analysis over the codebase rather than relying on structural heuristics or signature introspection—and decides what to read on its own. For each entry, the agent (1) reads the public header for signature and preconditions, (2) walks the call graph to find 2–3 production callers and reads their source, (3) inspects internal dependencies through call-graph callees, and (4) searches tests and examples. Its output is a structured P2 protocol report covering eight subchecks (P2.1–P2.8 defined in Table 8); each entry carries a claim (the agent’s project-specific finding) and evidence (file path and line). Grounding every claim in a verifiable source location keeps the agent from inventing protocol rules. Figure 5 shows a concrete instance for ICU’s createFromRules.

3.7

Stage 3: Static-Driven Build

Stages 3 and 4 run inside one Harness Generator agent: Stage 3 produces a binary that compiles, Stage 4 then attacks that binary with AP probes. The agent first reads the project’s build configuration (build script, Dockerfile, existing fuzzers, license) to settle on the right source style (.c, .cc, .cpp) and to match the conventions any existing fuzzers in the project follow. It then reads the library source, consults the Stage 1 Logic Group and the Stage 2 P2 report, and drafts the fuzzer source. While drafting, it reviews every P1/P2 sub-check via LLM semantic reasoning over the draft. No binary exists yet, so all 16 sub-checks are evaluated as source-level inspections rather than by their runtime oracles in Table 8. It then calls build_harness; on failure it reads the compiler tail, edits the source, and retries. No AP probes and no fuzz run in Stage 3; the output is a binary that compiles, at which point Stage 4 begins.

3.8

Stage 4: Adversarial Validation

Stage 4 is an Adversarial Probing (AP)-driven loop that decides whether the binary is ready to submit. The agent attacks its own harness to verify P1/P2 sub-checks. For brevity we reuse the term build-fix loop from Stage 3, but Stage 4’s build-fix loop additionally has access to AP probes and get_coverage (after the 600 s ASan run). The steps below detail the loop body. Step 1: Adversarial probing. The agent reads its own harness together with the relevant library source, and selects from Table 8 any P1.x / P2.x sub-checks the code is most likely to violate (no fixed count), writing one adversarial blob per selected check. AP_run_check / AP_reach_check execute the blobs one by one

Step 2: 600 s run + coverage unlock. Once every blob lands in case 4, QuartetFuzz performs a single 600 s ASan/LSan run on the binary. get_coverage then rebuilds the binary with coverage instrumentation and runs llvm-cov [23] over the 10-minute corpus to produce per-case line and branch coverage numbers. Step 3: Submit-eligible probing. The agent loops back to Step 1’s probing pattern with two new affordances: get_coverage and submit_harness. Submission is gated by a hard constraint and a soft check. The hard constraint is both line and branch coverage > 0 and the entry function is dynamically reached; we use the entry rather than the LG core because Stage 4 has no seed corpus, so core functions deeper in the stack may remain unreached. The soft check is the LLM’s own judgment that the harness is ready, made on the basis of additional AP probes and the coverage numbers. If the LLM declines to submit, it implicitly signals that some check still fails, and the loop continues with another build-fix iteration.

4

Implementation

The system runs on one server with 32 CPU cores and 62 GB RAM. Ten workers run in parallel. We implement QuartetFuzz in Python (∼4,500 lines of new code) on top of a shared BaseAgent abstraction that pairs an LLM with an MCP tool server [3]. Three agents (Logic Group, API Research, Harness Generator) operate over four tool categories: code_view (source navigation) and SAST (call-graph queries) are shared by all three agents; DAST (build / fuzz / GDB execution) is exposed only to the Harness Generator, which is the only agent that produces and runs binaries; each agent additionally has its own terminator tool that ends its turn loop with the agent’s final output. Table 5 (Appendix C) details per-agent inputs, outputs, models, turn caps, and how each DAST tool maps to the AP probes from §3.2. Each agent runs a turn loop: render prompt, call LLM, dispatch tool calls via FastMCP, repeat. QuartetFuzz has no built-in staticanalysis backend; it consumes a call graph in a JSON schema that any SAST tool can produce. This paper uses Joern [46] to generate the call graph, with a tree-sitter [6] fallback for projects Joern does not parse. Harness build, fuzzing, and coverage collection all use OSS-Fuzz infrastructure; Adversarial Probing (§3.2) is built on top.

5

Evaluation

We evaluate QuartetFuzz through five research questions:

Quality-Assured Fuzz Harness Generation via the Four Principles Framework

Conference’17, July 2017, Washington, DC, USA

P2 Protocol Report: Transliterator::createFromRules (ICU) P2.1 Init Sequence Claim: No global init required. Call the static factory directly. UErrorCode must be set to U_ZERO_ERROR before call. Evidence: > i18n/unicode/translit.h:1109: @stable ICU 2.0 > test/intltest/transapi.cpp:671: direct call P2.2 Parameter Construction Claim: rules (UnicodeString, semicolon-separated) carries fuzz input; dir is UTRANS_FORWARD or REVERSE; status must be U_ZERO_ERROR. Evidence: > i18n/unicode/translit.h:1103–1108: param docs P2.3 Object Lifecycle Claim: 1. Create: returns Transliterator*; caller owns. 2. Use: t->transliterate(text). 3. Destroy: delete t (virtual destructor). Evidence: > i18n/translit.cpp:1062–1063: return nullptr on failure > i18n/rbt_rule.cpp:185–191: dtor frees 5 ptrs P2.4 Return Value Handling Claim: Success: valid pointer, must delete. Failure: nullptr. Evidence: > i18n/translit.cpp:1062–1063: return nullptr on U_FAILURE

P2.5 Cleanup Sequence Claim: delete t on success; no cleanup on failure. Evidence: > test/intltest/transapi.cpp:84: delete t after use P2.6 API Existence Claim: All APIs verified to exist in ICU4C. Evidence: > translit.h:1109: createFromRules (public) > translit.h:652: transliterate (public) > translit.h:611: ˜Transliterator (virtual) P2.7 Co-call Constraints Claim: createFromRules/delete are paired; check U_FAILURE before using pointer. Evidence: > i18n/translit.cpp:1062: early return before alloc P2.8 Prerequisite State Claim: None. No global init, locale, or I/O required. Evidence: > i18n/translit.cpp:1051–1065: in-memory parser Note: the UAF vulnerability was in the destructor path (P2.3) — rbt_rule.cpp:185 deletes uninitialized pointers when the constructor returns early. This report enabled the generator to write a harness that triggered the bug.

Figure 5: Condensed P2 protocol report for ICU’s createFromRules API, generated by the API Research agent. Each entry carries a project-specific claim grounded in source-cited evidence (file:line). Sub-check definitions (P2.1–P2.8) are tabulated in Table 8. • RQ1 (Real-World Harness Quality Audit): Can our P1/P2 checks detect and repair quality violations in real-world production harnesses? • RQ2 (Logic Group Quality): Given a natural-language functionality prompt, can the pipeline identify entry functions (P3/P4) and produce a Logic Group that matches the human-selected target? • RQ3 (Generation vs. Baselines): How do our generated harnesses compare against the human-written gold standard and two LLM-based generators (OSS-Fuzz-Gen and PromeFuzz)? • RQ4 (Ablation Study): What is the contribution of each pipeline component—the P2 research stage, the static call-graph tools, and the dynamic build/coverage loop? • RQ5 (Real-World Deployment & Vulnerability Discovery): Does the full system discover real vulnerabilities in real-world projects?

5.1

Experimental Setup

Environment and fuzzing. All experiments run on one server (32 cores, 62 GB RAM) with ten parallel workers. For RQ5, projects are built and fuzzed inside their OSS-Fuzz Docker images. Fuzzing uses libFuzzer with ASan and empty seed corpora throughout. LSan is linked at build time and runs by default alongside ASan, so we use “ASan” as shorthand for the combined ASan+LSan instrumentation. Per-harness budget: RQ1 fuzzes each evaluated harness for a single 60 s run; RQ3 and RQ4 use 10 × 600 s and report the per-harness median across the 10 runs; RQ5 uses a single 10 h run per harness. Metrics. Following [22], coverage uses LLVM’s llvm-cov [23] line and branch (%). Productive rate (Tables 3, 4): unlike prior work’s build-success rate, we count a harness as failed if it fails to build

or builds with zero on both coverage metrics—a stricter criterion. Unproductive harnesses contribute 0 to the cross-case coverage mean (not N/A), so reported averages directly penalise failure rather than hiding it in the denominator. Models. Production configuration is Opus 4.6 for LG/API research and Sonnet 4.6 for harness generation; RQ5 deploys this in full, while RQ1 and RQ3 evaluate harness generation only and use Sonnet 4.6. RQ2 compares Opus 4.6 vs. Gemini 3.1 Pro on the LG task; RQ4 swaps in Gemini 3.1 Flash to probe model-capability sensitivity. All production runs use temperature 0; max_tokens uses each SDK’s default. The RQ4 Skill experiment runs Sonnet 4.6, Opus 4.6, Gemini 3.1 Pro, Gemini 3.1 Flash, and GPT-5.4 at each model’s default temperature. Baselines. We compare against two LLM-based generators. OSSFuzz-Gen (OFG) [28] is Google’s production system; its input format (project + target function) is closest to ours. PromeFuzz [29] (CCS ’25) is the latest academic system and already directly compared against PromptFuzz [32], CKGFuzzer [45], OSS-Fuzz-Gen, and hand-written OSS-Fuzz harnesses. To enable entry-by-entry comparison, we modified both OFG and PromeFuzz so that all three systems share the same I/O, model, and OSS-Fuzz project pinning; the modified code is included in our open-science artifact. Scheduzz [24] and HarnessAgent [48] are the systems closest to ours, but Scheduzz is not open-source and HarnessAgent is a concurrent work that is still under review. We compare their designs and headline numbers in §7 and Table 11. Dataset. We use 100 harnesses from 39 C/C++ OSS-Fuzz projects, drawn from the 586 in RQ1. Three researchers manually selected cases meeting all of: (i) the gold harness has stable coverage in OSSFuzz Introspector [1], indicating long-running production exposure

Conference’17, July 2017, Washington, DC, USA

on ClusterFuzz [16]; (ii) clean build without complex configuration changes; (iii) > 1% line coverage and no crash on a 600 s ASan empty-corpus run; (iv) entry function is a public API. RQ2–RQ4 use this dataset (full list in Table 9). During generation, the gold harness is replaced with a stub (return 0) so the agent cannot read it; the original is restored for coverage comparison.

5.2

RQ1: Real-World Harness Quality Audit

To validate that our P1 and P2 checks assess harness quality, we apply them directly to production harnesses written by experienced developers, reviewed through pull requests, and continuously executed on Google’s ClusterFuzz [16] for years. We deliberately do not check P3 (security boundary) or P4 (entrypoint adequacy) on these human-written harnesses because their authors are the library’s own developers and can knowingly cross boundaries or call internal functions, taking responsibility for those choices. A third-party AI generator has no such authority, so P3/P4 apply only at generation time (RQ2). Project selection. We select 70 C/C++ projects from OSS-Fuzz based on four criteria: (1) impact: high GitHub star counts, indicating wide adoption; (2) language: C or C++, where memory-safety vulnerabilities are most prevalent; (3) active maintenance: at least one commit within the past month; and (4) accessible issue tracker (e.g., GitHub/GitLab issues or JIRA): to facilitate upstream reporting. From these 70 projects we collect 586 production harnesses from OSS-Fuzz (listed in Table 10). Methodology. The audit reuses Stage 4 (§3.8) directly. Only the input changes. We feed Stage 4 the source plus its built binary as Stage 3’s output. Stage 1 is bypassed because the entry has already been chosen by the original author. Stage 2 is also bypassed; P2 protocol research instead runs inline at the start of Stage 4, without producing a standalone protocol report. Stage 4 then proceeds as designed, driven by Adversarial Probing (AP, §3.2). The agent uses Claude Sonnet 4.6 at temperature 0, with the P1.x/P2.x checklist injected into the prompt; static_analysis is disabled since the entry is already fixed. Each repair iteration rewrites the source and re-runs Stage 4’s gate. For RQ1 we fuzz each version (original and repaired) once for 60 s (default is 600 s; shortened to keep 586 cases tractable) under ASan/LSan with an empty corpus and compare LibFuzzer edge-coverage counters. The repair is submitted upstream iff its coverage is not weaker than the original’s, demonstrating to maintainers that the P1/P2 fix does not regress their existing fuzzing. If five repair attempts cannot meet this bar, the agent treats its own initial judgment as a false positive and abandons the case. Results. The audit produces two distinct outputs, both legitimate paths through Stage 4’s crash triage. (A) Harness-quality violations. The P1/P2 review identifies 53 violations (9.0% of 586) across 28 projects; the remaining 42 projects are P1/P2-clean, corroborating that our checks are not over-triggered on well-maintained harnesses. All 53 are submitted upstream. Of these, 45 (85%) are confirmed, 35 (66%) fixed or merged, and the rest await review. One additional P1 report was withdrawn after the maintainer noted the API already handled the edge case. This is our only P1 false positive and harmless to existing fuzzing.

Ze Sheng, Dmitrijs Trizna, Luigino Camastra, Zhicheng Chen, Qingxiao Xu, and Jeff Huang

(B) Latent library vulnerabilities. In two cases, repairing a P2 violation unmasked a latent library bug (Table 14, audit-fix rows). For openssl, the original harness had a wrong call order. After we fixed it, the harness triggered a stack-buffer-overread in OpenSSL’s DES implementation that had been latent for over 25 years. For tidy-html5, the original harness was missing a required call. After we added it, the harness triggered a memory leak. In both cases, crash triage classified the crash as a real library bug (not a coverage regression) and the bug was fixed upstream. This unmasking is a direct consequence of P2-driven repair. Misused harnesses silently mask bugs that hide on the correct usage path. (C) Coverage gain is not the goal. Our objective is to fix correctness defects, not to maximize coverage. Large gains occur when the original is dead code (e.g., opencv/filestorage, libyaml/emitter). Small gains occur when the harness already exercised the target but had a non-blocking P1/P2 violation (e.g., jq/parse_stream). In such cases the coverage figure mainly demonstrates that the fix does not regress existing testing. Figure 11 in the appendix shows representative P1 and P2 before/after code diffs. Table 1 reports each violation’s coverage impact. We use LibFuzzer edge coverage by default; three rows use a different metric (cairo/raster and harfbuzz/hb_set use LLVM line coverage, jq/parse_stream uses function-execution count). Notable gains include opencv/filestorage at +986%, libyaml/emitter at 14×, and openssl/quic_server at +65.7%. 14 violations also produced false-positive ASan/LSan crashes that the repair eliminated. Without the audit these 14 would have surfaced as false-positive bug reports.

5.3

RQ2: Logic Group Quality

To evaluate P3 and P4, we test whether the LG Agent’s entry selection matches human expert choices. Stage 1 (§3.5) only emits entries that satisfy the static P3/P4 conditions (public-API reach to core, danger > 0); RQ2 therefore tests whether the agent’s choices among qualifying entries align with human experts’. If the LG selects the wrong functions, the generated harness fuzzes the wrong code paths regardless of how well P1 and P2 are enforced downstream. We use the 100-case curated gold dataset (Table 9) as ground truth. The dataset labels each harness’s target function, the API that actually consumes the fuzz-byte input. A case is a match if this target appears in the LG’s entry set. Formally, a gold target 𝑔 matches the LG’s entry set E iff

∃𝑒 ∈ E. bn(𝑒) = bn(𝑔) ∨ ∃𝑒 ∈ E. 𝑒 ∈ callees𝐺 (𝑔) , | {z } | {z } direct

wrapper

where bn(·) strips namespace prefixes and callees𝐺 is the singlehop callee set from the project call graph. For each case we craft a functionality prompt from the fuzzer’s upstream pull-request description, documentation, or source code, split into two groups: 60 prompts that name the target function explicitly (w/ target), and 40 that contain only a natural-language feature description (w/o target). This contrast separates prompt-driven matching (the target name is given) from code-driven matching (the agent must locate the entry from source alone). The pipeline runs a single pass per case. The LG Agent (Claude Opus 4.6) proposes a candidate entry set E using code_view and static_analysis tools, then the Generator writes a first-draft

Quality-Assured Fuzz Harness Generation via the Four Principles Framework

Conference’17, July 2017, Washington, DC, USA

Table 1: All 35 fixed (merged upstream) violations with coverage impact; another 10 are confirmed but not yet merged (Table 10). Each row reports a single 60 s coverage comparison between the repaired harness and the original, empty corpus, ASan/LSan. #

Project

Fuzzer

Princ.

Cov. Impact

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35

apache-httpd apache-httpd binutils boost botan bzip2 bzip2 cairo gdk-pixbuf gdk-pixbuf gdk-pixbuf ghostscript harfbuzz jq lcms lcms lcms libarchive libpcap libpng libpng libssh2 libyaml ndpi njs opencv openssl openssl openvpn openvpn tidy-html5 tidy-html5 unbound wamr zlib

parse uri ranlib filesystem gcd bzip2_fd decompress raster cons file scale xpswrite hb_set parse_stream cgats dict transform_ext linkify rserver readapi transforms ssh2_client emitter is_stun script filestorage provider quic_server packet_id verify_cert general parse_file parse_packet mutator uncompress3

P1 P1 P1 P1 P1 P1,P2 P1 P2 P1,P2 P1,P2 P1 P1 P1 P1 P1 P1,P2 P1 P1 P2 P2 P1,P2 P1 P1 P1 P1 P2 P2 P1 P1 P1 P1,P2 P1,P2 P1 P2 P1

+11.2% +2.3% +11.7% +21.9% +29.7% +52.4% +18.6% +14.0% fixes FP crash fixes FP crash fixes FP crash +3.1% +29× +47.0% fixes FP crash fixes FP crash fixes FP crash fixes FP crash +25.0% fixes FP crash fixes FP crash +16.6% +14× +18.6% fixes FP crash +986% +51.4% +65.7% fixes FP crash fixes FP crash +33.5% +57.7% fixes FP crash fixes FP crash +4.8%

Audit cost: 586 harnesses, ∼10 min each, 10 parallel workers, ∼$720 total.

harness. Neither stage builds or fuzzes the harness. To compare across model families, we repeat the experiment with Gemini 3.1 Pro using the same pipeline and tools. Table 2 reports the results. With Opus 4.6, the LG stage and the first-draft harness both achieve 91% match with gold targets. The w/o target group, with zero function names in the prompt, reaches 88% LG match and 85% harness match, only 5–10pp below the w/ target group. Entry selection is therefore driven by code structure, not by the prompt content. Gemini 3.1 Pro achieves 84% LG match and 81% harness match, 7–10pp below Opus 4.6, with a wider gap on the harder w/o target split. Both models benefit from the same static-analysis tools; the gap reflects differences in code comprehension ability.

Prompt Example: libyaml/loader w/ target: “Write a fuzzer for libyaml that targets yaml_parser_load. Feed arbitrary bytes as YAML documents to test parser robustness.” w/o target: “Write a fuzzer for libyaml’s YAML document loading functionality. Feed arbitrary malformed YAML input to exercise the parsing and document construction paths.”

Figure 6: Example prompt pair for the same case. Table 2: RQ2: entry-function selection quality on 100 cases. w/ target: prompt names function (60 cases); w/o target: feature description only (40 cases). Stage

Prompt

Opus 4.6

Gemini 3.1 Pro

LG

w/ target w/o target All

56/60 35/40 91/100

52/60 32/40 84/100

Harness

w/ target w/o target All

57/60 34/40 91/100

51/60 30/40 81/100

LG = gold target ∈ entry set; Harness = gold target called in code.

The 9 misses (Opus) span three patterns: (1) five cases where target names differ across language bindings or require template resolution beyond our static analysis; (2) three C++ OO cases where both factory and instance methods are valid entries and the LG agent chose the opposite from gold; (3) one case (freerdp) covering 8/9 codec decompressors but missing the rarest.

5.4

RQ3: Generation vs. Baselines

RQ3 asks whether QuartetFuzz produces harnesses that match human-written OSS-Fuzz harnesses in coverage. All systems take the same input (project + target entry function) and run for 10×600 s under LibFuzzer (empty corpus, ASan; median coverage over 10 runs). All LLM-based systems use Claude Sonnet 4.6 (parameters in §5.1) with an identical build-retry budget of 5. We compare QuartetFuzz against Gold (the human-written OSSFuzz harness), OFG [28], and PromeFuzz [29]. Table 3 summarises; per-case data in Table 12. QuartetFuzz coverage statistically matches Gold. Across the 100 cases, the per-case difference (QF−Gold) averages +0.03pp on line and −0.07pp on branch. To check whether this gap is small enough to count as a match, we run a paired TOST equivalence test [22] within a ±2pp tolerance band; it accepts equivalence on both metrics (line 𝑝=1.6×10−11 , branch 𝑝=2.1×10−12 ). Cliff’s 𝛿 [35] measures the actual size of the gap independent of sample size, and is negligible: −0.015 on line, −0.022 on branch. Per-case breakdown vs. Gold. The match also holds at the case level: of 100 per-case differences (QF−Gold), 72/78 cases (line/branch) fall within ±2pp, with 17/10 wins and 11/12 losses. The

Conference’17, July 2017, Washington, DC, USA

Ze Sheng, Dmitrijs Trizna, Luigino Camastra, Zhicheng Chen, Qingxiao Xu, and Jeff Huang

Table 3: RQ3 coverage on 100 cases. Each harness: 10 × 600 s LibFuzzer, empty corpus, ASan; per-harness median across 10 runs; table is mean over 100 medians.

Avg. line (%) Avg. branch (%) Productive rate Avg. cost

Gold

OFG

PromeFuzz

QuartetFuzz

17.7 17.6 100/100 –

10.8 9.2 64/100 $2.43

12.5 13.4 74/100 $1.98

17.7 17.5 96/100 $1.65

17 line wins split into four: (i) multi-API drivers (7 cases) that exercise post-call paths gold’s bare-parse driver omits; (ii) configurationvarying drivers (4 cases) that route fuzzer bytes through library options; (iii) multi-strategy drivers (2 cases) that dispatch between distinct init paths from the first byte; (iv) initialisation fixes (4 cases) that add library setup gold’s harness omits. The 11 line losses split into four: (i) unproductive harnesses (4 cases) producing zero line coverage (analysed below); (ii) gold’s wider input grammar (3 cases) covering more API breadth than ours; (iii) suboptimal driver decisions (3 cases) where our input handling under-uses the entry; (iv) one marginal case at the threshold. Figure 12 shows one representative win and loss: wabt/wasm2wat (config-varying) and harfbuzz/subset (gold’s input grammar). Comparison with baselines. The two baselines fall significantly short. Per-case ranking across the four systems shows QuartetFuzz finishes first on 42/100 cases and first or second on 73/100, ahead of Gold (29 / 81), OFG (5 / 13), and PromeFuzz (24 / 33). OFG produces working harnesses for only 64/100 cases; its average coverage trails by 6.9–8.3pp. PromeFuzz succeeds on 74/100 cases but still trails by 4.1–5.2pp; even after we extract from its per-project pool the fuzzer calling the gold target (§5.1), the lack of per-function steering shows—it occasionally achieves high coverage on trivial targets (e.g., apache-httpd) but underperforms on the rest. QuartetFuzz reaches a 96/100 productive rate, against 64 for OFG and 74 for PromeFuzz. This margin reflects the practical value of P1/P2 checks and AP. Most baseline failures are build errors or zero-coverage harnesses that simple retry cannot fix, but our P1/P2guided pipeline catches and repairs them before fuzzing begins. The 4 unproductive cases are agent-introduced harness bugs (wrong include path, wrong API surface, wrong linkage convention, wrong build-include order); all 4 exhaust the 5-attempt build cap in Stage 3 without producing a binary, so none of them reaches Stage 4. To confirm QuartetFuzz’s gap over the two baselines is significant, we run a paired Wilcoxon signed-rank test on the per-case coverage differences. Both gaps come out significant: vs. OFG, 𝑝<10−11 on both metrics with Cliff’s 𝛿 medium; vs. PromeFuzz, 𝑝=1.6×10−4 on line and 𝑝=0.013 on branch (𝛿 small). QuartetFuzz also costs the least at $1.65/harness, below PromeFuzz ($1.98) and OFG ($2.43). Our P1/P2 checks catch most issues before the build loop, whereas OFG and PromeFuzz exhaust the 5-attempt build cap retrying broken harnesses; OFG still leaves 33/100 cases failing to build. Adversarial Probing analysis. Every one of the 96 productive cases invokes AP at least once, 362 AP calls in total (mean 3.77, median 3, max 12). 22 of these 96 cases need six or more probes, evidence that the agent is constructing attack blobs and reach probes

Table 4: RQ4 component ablations on the 100-case dataset, run identically to Table 3. Full reproduces the QuartetFuzz row from Table 3; the remaining columns each ablate one piece: w/o AP disables the adversarial probe gate; w/o P2 removes the API protocol report; w/o Static removes call-graph tools; Flash swaps the generator model to Gemini 3.1 Flash.

Avg. line (%) Avg. branch (%) Productive rate Avg. cost

Full

w/o AP

w/o P2

w/o Static

Flash

17.7 17.5 96/100 $1.65

16.9 16.6 84/100 $1.04

16.2 15.9 83/100 $1.35

17.5 17.3 88/100 $1.14

17.5 17.3 89/100 $1.64

rather than just reading the code. AP’s overall importance is shown by the ablation in §5.5.

5.5

RQ4: Ablation Study

We isolate the contribution of each component by disabling it and re-running all 100 cases under the same conditions as RQ3. Table 4 reports component ablations on Sonnet 4.6; per-case data is in Table 13. To test portability, we evaluate an Agent-Skill-style prompt-only configuration on five frontier models (Figure 7). Without Adversarial Probing. Productive rate drops 96→84 (−12pp); line −0.8pp, branch −0.9pp; cost falls $0.61/case (the AP overhead). The 12 lost cases all compile and run but never reach the target—without AP, no signal catches this and they slip past the build–fix loop. Without the P2 report. Removing the API-protocol report drops productive rate from 96 to 83 (−13pp) and line coverage by 1.5pp. Cost falls modestly to $1.35 because the saved P2 stage tokens are partly offset by the agent burning extra turns rediscovering protocol details on its own. The net effect is the largest quality regression of any single ablation. Without static analysis. Removing call-graph tools drops productive rate from 96 to 88 (−8pp) and line coverage by 0.2pp (−0.2pp branch). Cost falls to $1.14 because the agent has fewer tools to invoke; the LG and harness agents recover most structural context from code_view alone, so the pipeline degrades gracefully—static analysis is the least critical of the three components. Model swap (Gemini 3.1 Flash). Swapping in Gemini 3.1 Flash gives comparable coverage (−0.2pp line) but productive rate drops to 89 and cost is $1.64 (essentially unchanged from Full at $1.65); the pipeline still recovers 89 working harnesses on a weaker model, demonstrating that the principles transfer across model families even when raw capability differs. Skill-style portability. Inside the same pipeline, we keep only code_view and a prompt-only P1–P4 checklist (with and without the build–fix loop), approximating an Anthropic Agent Skill [4] setup on five frontier models. The build–fix loop helps every model (+6–14pp; Figure 7); stronger models do not substitute for it (Sonnet 4.6: 73 → 79; Opus 4.6: 67 → 76). The best Skill-style setup reaches 79/100, 17pp below the full pipeline’s 96, quantifying the infrastructure contribution.

Quality-Assured Fuzz Harness Generation via the Four Principles Framework

Conference’17, July 2017, Washington, DC, USA

Productive rate (% of 100 cases)

80

Claude Opus 4.6 (76)

75

Claude Sonnet 4.6 (73)

115 Logic Groups (23 × 5) → 115 fuzzers generated

Gemini 3.1 Pro (72)

70 65

RQ5 deployment outcomes: 115 fuzzers → 42 reports (40 TP + 2 FP)

Claude Sonnet 4.6 (79)

GPT-5.4 (69)

81 fuzzers with crashes

Claude Opus 4.6 (67)

Gemini 3.1 Flash (64) Gemini 3.1 Pro (65)

Model (color) Anthropic

Claude Opus 4.6 Claude Sonnet 4.6

60 Gemini 3.1 Flash (56) GPT-5.4 (55)

55 0.00

0.25

Build fix loop (marker)

w/o build fix loop with build fix loop effect of build fix loop

0.50 0.75 1.00 Cost per harness (USD)

34 fuzzers no crashes

44 harness issues (fixed → harness w/o crash)

37 reported as real vulnerabilities

OpenAI

harness without crash: 44 + 34 = 78

GPT-5.4

Gemini

Gemini 3.1 Pro Gemini 3.1 Flash

1.25

upstream review: 24 TP + 2 FP

1.50

select 14 in 4 collab projects (mongoose 3, fwupd 4, simdutf 5, flatbuffers 2)

with seed generation tool: +5 TP (mongoose 1 fuzzer → 2 bugs)

Figure 7: Four Principles under a prompt-only Skill-style setup: cost vs. productive rate across five models, with and without the build–fix loop.

42 reports total = 40 TP + 2 FP

29 fixed or confirmed (incl. 3 CVEs) | 11 awaiting | 2 rejected

5.6

RQ5: Real-World Deployment & Vulnerability Discovery

Fuzz harnesses exist to find bugs [51]. We therefore deploy QuartetFuzz on 23 real-world open-source projects (C/C++, Java, JavaScript) to evaluate the full pipeline and its vulnerability-discovery ability. Vulnerability Discovery. The 23 projects come from two channels: (i) 12 projects (10 C/C++, 1 Java, 1 JavaScript; bold in Table 14) where we established direct collaboration (Figure 1) with maintainers or were asked by our sponsor, and (ii) 11 additional projects following the same selection criteria as RQ1. The full project list was fixed before deployment and collaborations were established before any harness was generated and run; no project was selected post-hoc based on whether it produced a crash. ghidra and graaljs are not in OSS-Fuzz upstream; we wrote our own harnesses and build scripts. Figure 8 shows the full deployment funnel. The LG agent generates 5–10 Logic Groups (LGs) per project and keeps the top-5 by danger score, yielding 115 LGs. These 115 LGs feed full-pipeline harness generation. Adversarial Probing and the Stage 4 600 s gate test run together produced 81 sanitizer crashes. Crash triage classified 44 as harness-side P1/P2 violations (fixed in place, not reported) and 37 as real library bugs reported upstream. On 4 collaboration projects, we re-ran 14 fuzzers with no crash (mongoose 3, fwupd 4, simdutf 5, flatbuffers 2) using a targeted seed generator [39], finding 5 additional real bugs (mongoose’s HTTP fuzzer alone yielded 2). Total LLM cost: approximately $300 for the 115-LG full pipeline and $500 for seed-generation reruns across the 4 collaboration projects (∼$100/project). In total QuartetFuzz submitted 42 reports (40 real + 2 FPs): 20 fixed, 9 confirmed (including 3 CVEs), 11 awaiting review, 2 rejected—a 4.8% FP rate, the lowest among LLM-based generators (Table 7). Of the 42 reports, 24 are from collaboration projects (16 fixed, 6 confirmed, 2 FP) and 18 from random projects (4 fixed, 3 confirmed, 11 awaiting); per-project bug-finding is comparable (2.0 vs. 1.6 reports/project), and the higher acknowledge rate on collaboration projects is mostly maintainer latency. All collaboration-project bugs were independently reviewed by our sponsor and earned a $120,000 bounty. Figure 9a shows the LG-rank distribution and Figure 9b the vulnerability-type distribution of the 40 real bugs; the rank distribution concentrates at the top, confirming that the LG ranking prioritises vulnerability-prone targets.

Figure 8: RQ5 deployment funnel: 115 fuzzers (23×5 LGs) → 42 reports (40 TP + 2 FP) → 29 fixed or confirmed.

Rank 4--5 10%

Rank 3 20%

40

Rank 1 40%

bugs

Rank 2 30%

(a) LG rank.

heap overflow NULL deref stack overflow UAF integer overflow OOM memory leak assertion uncaught exception type confusion divide-by-zero out-of-bounds read

2 2 2

1 1 1

0

2

4

3 3 3

4

12

6

6 8 count (n = 40)

10

12

(b) Vulnerability types.

Figure 9: Distribution of the 40 real bugs (2 FPs excluded): 40% at rank 1, 70% in top 2.

Cross-language deployment. With LLM-only LG generation and ranking, PDFBox (Java) yielded 3 vulnerabilities (all fixed) and GraalJS (JavaScript) yielded 2 (one fixed, one confirmed)—all 5 without false positives. Controlled validation of LG ranking. Since we only deployed rank-1–5, “100% in rank 1–5” is true by construction. On the 5 projects where rank-1–5 found bugs and the rank comes from a real danger score (not Static-Analysis Fallback (SAF); see Table 14)— mongoose, fwupd, libwebp, opc-ua, rapidjson—we deploy 25 additional rank-6–10 harnesses under the same 10 h budget and dedupe (ASan top-3 frame). Ranks 6–10 yield zero new memory-safety crashes (only 2 resource-exhaustion artifacts; Table 6, Appendix A)— the ranking saturates the practically findable bug set. Upstream adoption. Beyond accepting bug reports, 4 maintainer teams produced new fuzzing artifacts from our work: OpenSSL adopted our quic-server rewrite (2.64× branch coverage); fwupd generalized our PoC into a generic FuDevice fuzzing interface; pdfbox committed our crash inputs as JUnit regression tests; libwebp enhanced their own fuzzer using ours as reference.

Conference’17, July 2017, Washington, DC, USA

6

Discussion

Mathematical specifications vs. practical implementation. In the abstract we defined P1–P4 as mathematical specifications. However, the actual implementation does not match the mathematical definitions because the mathematical predicates are not decidable over source code alone. We approximate them with syntactic sub-checks (P1.1–P1.8, P2.1–P2.8; Table 8), AP probes, and call-graph reachability with an LLM fallback. We are fully aware this gap reduces certainty in our claims and could place the paper in a vulnerable position. We chose to publish anyway because, as §1 documents, harness quality has grown into a serious engineering problem as automated harness generation scales, and to our knowledge no prior work has defined what correctness means for an LLM-generated harness. Our framework offers reasonable approximations grounded in actual audit and generation practice, with every check tied to a concrete artifact (source line, call-graph edge, sanitizer fire). We treat it as a starting definition, not a final proof, and hope future work builds on this foundation. Metric limitations. Prior automated harness-generation work evaluates systems with line coverage, branch coverage, and build success rate. These remain necessary but are no longer sufficient. Producing a harness that compiles and reaches some coverage is now within easy reach of an LLM agent. In the LLM era the bar should move from “did it build and run” to “is the harness correct”. Throughout this work, our target is to make the system produce harnesses an experienced developer would have written, not ones that maximise coverage on a benchmark. We propose P1–P4 as that next-level criterion. Because we do not yet have a fully deterministic checker, we did not use P1–P4 as the primary evaluation metric in RQ3 and RQ4. But this is exactly why RQ1, RQ2, and RQ5 matter. RQ1 validates P1/P2 on production harnesses with experienced developer review, RQ2 validates P3/P4 against humanselected targets, and RQ5 validates the full system in real-world deployment. Generated vs. human-written harnesses. P1 and P2 apply to both. A leak or misused API is a bug whoever wrote the harness; we audit human harnesses against both in §5.2. P3 and P4 do not. A library maintainer can knowingly cross a boundary or call an internal helper; an LLM that has never touched the project cannot. We therefore enforce P3 and P4 only on our generator. P3 is a preference, not a hard constraint: when no public entry reaches the target, the generator falls back to the most direct internal entry that preserves as many boundaries as possible. Seeds. RQ1–RQ4 use empty corpora throughout, and RQ5 also defaults to empty corpora. In practice seeds matter, since an empty corpus cannot produce a valid HTTP request or PNG. On 4 of the 23 RQ5 projects we additionally paired harnesses with a targeted seed generator [39], which yielded 5 of the 40 bugs. Limitations and future work. (1) Our pipeline depends on OSSFuzz, LibFuzzer, ASan/LSan, and C/C++; pdfbox (Java), graaljs (JS), and non-OSS-Fuzz collaboration projects required pipeline and architecture adjustments. Future work will extend to mainstream fuzzers (AFL++, HonggFuzz) and other languages (Rust, Go, Python). (2) RQ3 and RQ4 use line/branch coverage and productive rate. The LLM era calls for stronger metrics such as core-function or core-file coverage, which combined with P1–P4 and our 100-case dataset

Ze Sheng, Dmitrijs Trizna, Luigino Camastra, Zhicheng Chen, Qingxiao Xu, and Jeff Huang

can form a deterministic benchmark for LLM-generated harnesses. (3) All experiments use empty corpora; high-quality per-case seeds that drive execution into the core function would surface deeper bugs. (4) Joern fails on some projects (heavy C++ templates, vtable indirection) and the tree-sitter fallback gives a basic but lowerprecision call graph. Stronger pointer/alias analysis (e.g., SVF [41]) would further improve P3/P4 accuracy.

7

Related Work

Traditional harness generation. Consumer-based approaches extract API usage from existing code: FuzzGen [19], FUDGE [5], WINNIE [21], APICraft [50], libErator [43], and WildSync [44]. They depend on consumer-code availability. Constraint-based approaches enforce API protocols at runtime: Hopper [7] (93.52% API coverage but 51% spurious crashes), AFGen [30] (precision 46.97%→94.55% via constraint tracing), NEXZZER [27] (filters 93.96% of crashes as API misuse), GraphFuzz [18], DAISY [54], and Rubick [49]. SyzGen [8] targets the analogous problem for OS syscalls. None defines correctness a priori at the source level. LLM-based harness generation. PromptFuzz [32] treats generation as a fuzzing loop over prompts (1.61× branch coverage over OSS-Fuzz on 14 libraries). CKGFuzzer [45] uses a CodeQL [14]+Treesitter [6] knowledge graph; 84.4% of its 199 crashes stem from API misuse. PromeFuzz [29] builds structured knowledge bases over AST metadata, API docs, and call sequences; a DeepSeek-R1 sanitizer raises precision from 2.7% to 89.7%. OSS-Fuzz-Gen [28] integrates LLM generation into Google’s pipeline with Fuzz Introspector and found CVE-2024-9143 in OpenSSL. HarnessAgent [48] is a toolaugmented agent with LSP [33] and Tree-sitter [6]; reports 87%/81% three-round success but no re-runnable artifact. Scheduzz [24] is the only prior tool applying constraints before generation (Prolog over type/LLM-extracted API dependencies); no public artifact. LLM4FDG [52] reports a 34% API-misuse rate; FDFactory [55], TitanFuzz [10] extend to deep-learning libraries; position work on reliable LLM-driven fuzzing [9] reaches a similar conclusion. Every prior tool relies on behavioural proxies rather than source-level correctness checked before fuzzing (Table 11). Adjacent LLM-forsecurity work [11, 25, 26, 37, 42, 47] targets vulnerability detection rather than harness synthesis. Harness quality assessment. To our knowledge, no prior work audits production harnesses systematically or defines correctness a priori. AFGen’s Constraints Tracer [30] checks API constraints postcrash; OGHarn’s three oracles [40] verify compilation, execution, and coverage; deepSURF [2] treats true positives as crashes from contract-respecting use, mapping to our P2. We define quality as source-level conditions checked before fuzzing and validate on 586 production harnesses (45 confirmed, 35 fixed). We also release 100 annotated harnesses as the first labeled dataset for harness quality.

8

Conclusion

We define the Four Principles, four source-level conditions (P1– P4) that a harness must satisfy before fuzzing begins, and build QuartetFuzz, an LLM-agent generator that enforces them through a four-stage pipeline of Logic Group selection, API protocol research, static-driven build, and adversarial validation. Our audit of 586 production OSS-Fuzz harnesses flagged 53 violations (45

Quality-Assured Fuzz Harness Generation via the Four Principles Framework

confirmed, 35 fixed) and exposed 2 long-latent library bugs. QuartetFuzz matched human gold coverage on 100 cases (TOST ±2pp, 𝑝<10−10 ), beat OSS-Fuzz-Gen by 6.9–8.3pp and PromeFuzz by 4.1– 5.2pp, and shipped 42 bug reports across 23 projects with a 4.8% FP rate (29 fixed or confirmed, 3 CVEs). Across audit and generation, the same P1/P2 checks intercepted 58 harness-induced crashes that would have been false-positive reports. Harness quality is the binding constraint on fuzzing effectiveness, and the practical way to enforce it is to bake source-level checks into the generator before fuzzing begins.

References [1] Adalogics and Google. 2022. Fuzz Introspector: A Tool for Analyzing and Visualizing Fuzz Coverage. https://github.com/ossf/fuzz-introspector. [2] Georgios Androutsopoulos and Antonio Bianchi. 2025. deepSURF: Detecting Memory Safety Vulnerabilities in Rust Through Fuzzing LLM-Augmented Harnesses. arXiv:2506.15648 [cs.CR] https://arxiv.org/abs/2506.15648 [3] Anthropic. 2024. Model Context Protocol. https://modelcontextprotocol.io. [4] Anthropic. 2025. Agent Skills. https://docs.claude.com/en/docs/agents-andtools/agent-skills/overview. [5] Domagoj Babić, Stefan Bucur, Yaohui Chen, Franjo Ivančić, Tim King, Markus Kusano, Caroline Lemieux, László Szekeres, and Wei Wang. 2019. FUDGE: Fuzz Driver Generation at Scale. In Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. [6] Max Brunsfeld. 2018. Tree-sitter: An Incremental Parsing System for Programming Tools. https://tree-sitter.github.io/tree-sitter/. [7] Peng Chen, Yuxuan Xie, Yunlong Lyu, Yuxiao Wang, and Hao Chen. 2023. Hopper: Interpretative Fuzzing for Libraries. In Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security. 1600–1614. [8] Weiteng Chen, Yu Wang, Zheng Zhang, and Zhiyun Qian. 2021. SyzGen: Automated Generation of Syscall Specification of Closed-Source macOS Drivers. In Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security. 749–763. [9] Yiran Cheng, Hong Jin Kang, Lwin Khin Shar, Chaopeng Dong, Zhiqiang Shi, Shichao Lv, and Limin Sun. 2025. Towards Reliable LLM-Driven Fuzz Testing: Vision and Road Ahead. arXiv preprint arXiv:2503.00795 (2025). [10] Yinlin Deng, Chunqiu Steven Xia, Haoran Peng, Chenyuan Yang, and Lingming Zhang. 2023. Large Language Models Are Zero-Shot Fuzzers: Fuzzing DeepLearning Libraries via Large Language Models. In Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis. 423–435. [11] Zhangyin Feng, Daya Guo, Duyu Tang, Nan Duan, Xiaocheng Feng, Ming Gong, Linjun Shou, Bing Qin, Ting Liu, Daxin Jiang, and Ming Zhou. 2020. CodeBERT: A Pre-Trained Model for Programming and Natural Languages. In Findings of the Association for Computational Linguistics: EMNLP 2020. 1536–1547. [12] Andrea Fioraldi, Dominik Maier, Heiko Eißfeldt, and Marc Heuse. 2020. AFL++: Combining Incremental Steps of Fuzzing Research. In 14th USENIX Workshop on Offensive Technologies (WOOT 20). [13] FIRST. 2019. Common Vulnerability Scoring System Version 3.1: Specification Document. https://www.first.org/cvss/v3-1/. [14] GitHub. 2019. CodeQL: Variant Analysis for Code Security. https://codeql.github. com/. [15] Google. 2016. OSS-Fuzz: Continuous Fuzzing for Open Source Software. https: //github.com/google/oss-fuzz. [16] Google. 2019. ClusterFuzz: Scalable Fuzzing Infrastructure. https://google.github. io/clusterfuzz/. [17] Google Sanitizers. 2013. LeakSanitizer: A Memory Leak Detector built on top of AddressSanitizer. https://github.com/google/sanitizers/wiki/ AddressSanitizerLeakSanitizer. [18] Harrison Green and Thanassis Avgerinos. 2022. GraphFuzz: Library API Fuzzing with Lifetime-aware Dataflow Graphs. In Proceedings of the 44th International Conference on Software Engineering. 1070–1081. [19] Kyriakos K. Ispoglou, Daniel Austin, Vishwath Mohan, and Mathias Payer. 2020. FuzzGen: Automatic Fuzzer Generation. In Proceedings of the 29th USENIX Conference on Security Symposium (SEC’20). USENIX Association, USA, Article 128, 17 pages. [20] Bokdeuk Jeong, Joonun Jang, Hayoon Yi, Jiin Moon, Junsik Kim, Intae Jeon, Taesoo Kim, WooChul Shim, and Yong Ho Hwang. 2023. UTopia: Automatic Generation of Fuzz Driver using Unit Tests. In 2023 IEEE Symposium on Security and Privacy (SP). IEEE, 2676–2692. [21] Jinho Jung, Stephen Tong, Hong Hu, Jungwon Lim, Yonghwi Jin, and Taesoo Kim. 2021. WINNIE: Fuzzing Windows Applications with Harness Synthesis and Fast Cloning. In Proceedings of the 2021 Network and Distributed System Security

Conference’17, July 2017, Washington, DC, USA Symposium (NDSS 2021). [22] George Klees, Andrew Ruef, Benji Cooper, Shiyi Wei, and Michael Hicks. 2018. Evaluating Fuzz Testing. In Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security. 2123–2138. doi:10.1145/3243734.3243804 [23] Chris Lattner and Vikram Adve. 2004. LLVM: A Compilation Framework for Lifelong Program Analysis & Transformation. In Proceedings of the International Symposium on Code Generation and Optimization (CGO). 75–86. doi:10.1109/ CGO.2004.1281665 [24] Yan Li, Wenzhang Yang, Yuekun Wang, Jian Gao, Shaohua Wang, Yinxing Xue, and Lijun Zhang. 2025. Scheduzz: Constraint-based Fuzz Driver Generation with Dual Scheduling. arXiv:2507.18289 [cs.SE] https://arxiv.org/abs/2507.18289 [25] Ziyang Li, Saikat Dutta, and Mayur Naik. 2025. IRIS: LLM-Assisted Static Analysis for Detecting Security Vulnerabilities. In International Conference on Learning Representations, Vol. 2025. 35735–35758. [26] Zhen Li, Deqing Zou, Shouhuai Xu, Xinyu Ou, Hai Jin, Sujuan Wang, Zhijun Deng, and Yuyi Zhong. 2018. VulDeePecker: A Deep Learning-Based System for Vulnerability Detection. In Proceedings of the 2018 Network and Distributed System Security Symposium (NDSS 2018). doi:10.14722/ndss.2018.23158 [27] Jiayi Lin, Qingyu Zhang, Junzhe Li, Chenxin Sun, Hao Zhou, Changhua Luo, and Chenxiong Qian. 2025. Automatic Library Fuzzing through API Relation Evolvement. In Proceedings of the 2025 Network and Distributed System Security Symposium (NDSS 2025). [28] Dongge Liu, Oliver Chang, Jonathan Metzman, Martin Sablotny, and Mihai Maruseac. 2024. OSS-Fuzz-Gen: Automated Fuzz Target Generation via Large Language Models. https://github.com/google/oss-fuzz-gen. [29] Yuwei Liu, Junquan Deng, Xiangkun Jia, Yanhao Wang, Minghua Wang, Lin Huang, Tao Wei, and Purui Su. 2025. PromeFuzz: A Knowledge-Driven Approach to Fuzzing Harness Generation with Large Language Models. In Proceedings of the 2025 ACM SIGSAC Conference on Computer and Communications Security. 1559–1573. [30] Yuwei Liu, Yanhao Wang, Xiangkun Jia, Zheng Zhang, and Purui Su. 2024. AFGen: Whole-Function Fuzzing for Applications and Libraries. In 2024 IEEE Symposium on Security and Privacy (SP). IEEE, 1901–1919. [31] LLVM Project. 2015. libFuzzer: A Library for Coverage-Guided Fuzz Testing. https://llvm.org/docs/LibFuzzer.html. [32] Yunlong Lyu, Yuxuan Xie, Peng Chen, and Hao Chen. 2024. Prompt Fuzzing for Fuzz Driver Generation. In Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security. 3793–3807. [33] Microsoft. 2022. Language Server Protocol Specification 3.17. https://microsoft. github.io/language-server-protocol/. [34] Barton P. Miller, Lars Fredriksen, and Bryan So. 1990. An empirical study of the reliability of UNIX utilities. Commun. ACM 33, 12 (Dec. 1990), 32–44. doi:10. 1145/96267.96279 [35] Jeanine Romano, Jeffrey D. Kromrey, Jesse Coraggio, and Jeff Skowronek. 2006. Appropriate Statistics for Ordinal Level Data: Should We Really Be Using t-Test and Cohen’s d for Evaluating Group Differences on the NSSE and Other Surveys?. In Annual Meeting of the Florida Association for Institutional Research. Cocoa Beach, FL. [36] Konstantin Serebryany, Derek Bruening, Alexander Potapenko, and Dmitriy Vyukov. 2012. AddressSanitizer: A Fast Address Sanity Checker. In 2012 USENIX Annual Technical Conference (USENIX ATC 12). USENIX Association, Boston, MA, 309–318. https://www.usenix.org/conference/atc12/technical-sessions/ presentation/serebryany [37] Ze Sheng, Zhicheng Chen, Shuning Gu, Heqing Huang, Guofei Gu, and Jeff Huang. 2025. LLMs in Software Security: A Survey of Vulnerability Detection Techniques and Insights. Comput. Surveys 58, 5, Article 134 (2025). doi:10.1145/3769082 [38] Ze Sheng, Fenghua Wu, Xiangwu Zuo, Chao Li, Yuxin Qiao, and Lei Hang. 2024. LProtector: An LLM-driven Vulnerability Detection System. arXiv preprint arXiv:2411.06493 (2024). [39] Ze Sheng, Qingxiao Xu, Jianwei Huang, Matthew Woodcock, Heqing Huang, Alastair F. Donaldson, Guofei Gu, and Jeff Huang. 2025. All You Need Is A Fuzzing Brain: An LLM-Powered System for Automated Vulnerability Detection and Patching. arXiv:2509.07225 [cs.CR] https://arxiv.org/abs/2509.07225 [40] Gabriel Sherman and Stefan Nagy. 2025. No Harness, No Problem: Oracle-Guided Harnessing for Auto-Generating C API Fuzzing Harnesses. In 2025 IEEE/ACM 47th International Conference on Software Engineering (ICSE). 165–177. [41] Yulei Sui and Jingling Xue. 2016. SVF: Interprocedural Static Value-Flow Analysis in LLVM. In Proceedings of the 25th International Conference on Compiler Construction. 265–266. [42] Yuqiang Sun, Daoyuan Wu, Yue Xue, Han Liu, Wei Ma, Lyuye Zhang, Yang Liu, and Yingjiu Li. 2024. LLM4Vuln: A Unified Evaluation Framework for Decoupling and Enhancing LLMs’ Vulnerability Reasoning. arXiv preprint arXiv:2401.16185 (2024). [43] Flavio Toffalini, Nicolas Badoux, Zurab Tsinadze, and Mathias Payer. 2025. Liberating Libraries through Automated Fuzz Driver Generation: Striking a Balance without Consumer Code. Proceedings of the ACM on Software Engineering 2, FSE, Article FSE095 (June 2025), 23 pages.

Conference’17, July 2017, Washington, DC, USA

[44] Wei-Cheng Wu, Stefan Nagy, and Christophe Hauser. 2025. WildSync: Automated Fuzzing Harness Synthesis via Wild API Usage Recovery. Proceedings of the ACM on Software Engineering 2, ISSTA (2025), 963–984. [45] Hanxiang Xu, Wei Ma, Ting Zhou, Yanjie Zhao, Kai Chen, Qiang Hu, Yang Liu, and Haoyu Wang. 2025. CKGFuzzer: LLM-Based Fuzz Driver Generation Enhanced By Code Knowledge Graph. In 2025 IEEE/ACM 47th International Conference on Software Engineering: Companion Proceedings (ICSE-Companion). IEEE, 243–254. [46] Fabian Yamaguchi, Nico Golde, Daniel Arp, and Konrad Rieck. 2014. Modeling and Discovering Vulnerabilities with Code Property Graphs. In 2014 IEEE Symposium on Security and Privacy. IEEE, 590–604. doi:10.1109/SP.2014.44 [47] Chenyuan Yang, Zijie Zhao, and Lingming Zhang. 2025. KernelGPT: Enhanced Kernel Fuzzing via Large Language Models. In Proceedings of the 30th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 2. 560–573. [48] Kang Yang, Yunhang Zhang, Zichuan Li, Guanhong Tao, Jun Xu, and Xiaojing Liao. 2025. HarnessAgent: Scaling Automatic Fuzzing Harness Construction with Tool-Augmented LLM Pipelines. arXiv:2512.03420 [cs.CR] https://arxiv.org/abs/ 2512.03420 [49] Cen Zhang, Yuekang Li, Hao Zhou, Xiaohan Zhang, Yaowen Zheng, Xian Zhan, Xiaofei Xie, Xiapu Luo, Xinghua Li, Yang Liu, and Sheikh Mahbub Habib. 2023. Automata-Guided Control-Flow-Sensitive Fuzz Driver Generation. In 32nd USENIX Security Symposium (USENIX Security 23). 2867–2884. [50] Cen Zhang, Xingwei Lin, Yuekang Li, Yinxing Xue, Jundong Xie, Hongxu Chen, Xinlei Ying, Jiashui Wang, and Yang Liu. 2021. APICraft: Fuzz Driver Generation for Closed-source SDK Libraries. In 30th USENIX Security Symposium (USENIX Security 21). 2811–2828. [51] Cen Zhang, Younggi Park, Fabian Fleischer, Yu-Fu Fu, Jiho Kim, Dongkwan Kim, Youngjoon Kim, Qingxiao Xu, Andrew Chin, Ze Sheng, Hanqing Zhao, Brian J. Lee, Joshua Wang, Michael Pelican, David J. Musliner, Jeff Huang, Jon Silliman, Mikel Mcdaniel, Jefferson Casavant, Isaac Goldthwaite, Nicholas Vidovich, Matthew Lehman, and Taesoo Kim. 2026. SoK: DARPA’s AI Cyber Challenge (AIxCC): Competition Design, Architectures, and Lessons Learned. arXiv preprint arXiv:2602.07666 (2026). [52] Cen Zhang, Yaowen Zheng, Mingqiang Bai, Yeting Li, Wei Ma, Xiaofei Xie, Yuekang Li, Limin Sun, and Yang Liu. 2024. How Effective Are They? Exploring Large Language Model Based Fuzz Driver Generation. In Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis. 1223– 1235. [53] Mingrui Zhang, Jianzhong Liu, Fuchen Ma, Huafeng Zhang, and Yu Jiang. 2021. IntelliGen: Automatic Driver Synthesis for Fuzz Testing. In 2021 IEEE/ACM 43rd International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP). 318–327. [54] Mingrui Zhang, Chijin Zhou, Jianzhong Liu, Mingzhe Wang, Jie Liang, Juan Zhu, and Yu Jiang. 2023. Daisy: Effective Fuzz Driver Synthesis with Object Usage Sequence Analysis. In 2023 IEEE/ACM 45th International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP). IEEE, 87–98. [55] Tianming Zheng, Fanchao Meng, Ping Yi, and Yue Wu. 2026. Automating fuzz driver generation for deep learning libraries with large language models. Cybersecurity 9 (1 2026). doi:10.1186/s42400-025-00532-9

9

Open Science

We release the artifacts that support the paper’s claims through three repositories covering the system, the dataset, and the two re-runnable baselines: • QuartetFuzz: system + dataset. LG agent, API Research agent, Harness Generator, and P1–P4 checkers; the 100-case goldstandard dataset with P1–P4 labels (Table 9); and run_subset.sh reproducing RQ3 and RQ4 end-to-end. https://github.com/ OwenSanzas/QuartetFuzz • Modified PromeFuzz baseline. Our shared-I/O patch over PromeFuzz [29] that aligns model, OSS-Fuzz project pinning, and per-target input handling with QuartetFuzz, plus a matching run script. https://github.com/OwenSanzas/PromeFuzz • Modified OSS-Fuzz-Gen baseline. Our shared-I/O patch over OSS-Fuzz-Gen [28]; same alignment, README, and run script. https://github.com/OwenSanzas/oss-fuzz-gen Subset and embargo. The full live-rerun artefact (10 × 600 s traces, OSS-Fuzz images) exceeds 230 GB and is infeasible to host on GitHub; run_subset.sh targets 25 lightweight cases chosen as

Ze Sheng, Dmitrijs Trizna, Luigino Camastra, Zhicheng Chen, Qingxiao Xu, and Jeff Huang

a representative RQ3 slice (runnable in ∼30 min). We retain the right to re-run the full 100 on request. Withheld until camera-ready: actual CVE numbers (anonymised in Table 14), 11 embargoed RQ5 reports, and PoCs / sanitizer traces for unpatched vulnerabilities.

10

Ethical Considerations

Scope. 42 RQ5 reports across 23 projects (29 fixed/confirmed with 3 CVEs, 11 awaiting, 2 FPs) plus 53 P1/P2 violations from the 70project RQ1 audit (45 confirmed, 35 merged). 12 of 23 RQ5 projects are collaborations with prior maintainer or sponsor consent; the remaining 11 plus the 70 RQ1 projects are in OSS-Fuzz and receive submissions as standard PRs or advisories. All targets public; no human subjects, no user data; the disclosure protocol below was established before any audit or generation run. Stakeholders. Maintainers receive reproducible PoCs and root causes; end users gain patched dependencies via the 42 RQ5 fixes; the 35 P1/P2 fixes repair production harnesses already deployed but silently broken; the community gains a 100-case labelled benchmark. Affected parties—users of unpatched libraries (mitigated by embargo and CVE anonymisation) and uncontacted maintainers in 28 audit projects (mitigated by submitting only fixes that can be declined)—face minimal residual exposure. Adversary prerequisites (frontier-model access, source, OSS-Fuzz format) are project-ownerside, favouring defensive use. Risks and mitigations. Net defensive value (64 upstream fixes, 58 harness-induced false positives blocked pre-triage) against three risks: information leak before patch, dual-use uplift, maintainer burden. Every report uses the maintainer’s designated channel with root-cause and fix, held until patched; CVE numbers and embargoed reports are anonymised in Table 14. Released artefacts are limited to the system, the 100-case benchmark, and baseline scripts—no exploit catalog. Collaboration bugs were independently re-validated by our anonymised sponsor before bounty payout, on the same protocol as unsponsored projects. Limitations. No external ethics consultation; our stakeholder list may omit downstream commercial users and supply-chain organisations. Long-term effects on OSS maintainer practice—whether sustained automated quality auditing shifts the review burden onto maintainers—are out of scope. We will incorporate reviewer feedback into the camera-ready.

11

Generative AI Usage

System implementation. ∼90% of the framework (agent loop, MCP tools, build / runner / coverage glue, evaluation scripts) was written and maintained by the authors; Claude (Opus / Sonnet 4.6) generated only auxiliary workflows (small data-processing snippets, plotting scripts), each admitted only after standard peer code review. Writing assistance. The authors wrote paper structure and prose by hand; Claude (Opus 4.6) polished the manuscript, reviewed crosssection consistency, and assisted with figures and tables. Every Claude-suggested edit was author-reviewed before commit.

Quality-Assured Fuzz Harness Generation via the Four Principles Framework

Conference’17, July 2017, Washington, DC, USA

Table 5: QuartetFuzz MCP agents (top) and the tools each category exposes (bottom). Agent

Input → Output

Model

Tool categories

Logic Group API Research Harness Generator

source repo → top-5 LGs one LG → P2 protocol report LG + P2 report → AP-passing binary

Opus 4.6 Opus 4.6 Sonnet 4.6

code_view, SAST, terminator code_view, SAST, terminator code_view, SAST, DAST, terminator

50 30 50

Category

Tools

code_view

read_file, list_directory, search_files, list_existing_fuzzers

Role source navigation, available to all agents

SAST tool

get_callers, get_callees, find_definition, forward_reach, reverse_reach, public_entries_for, public_entries_for_batch, is_public_api

call-graph queries (forward + reverse, with public-API filtering for the core-first LG search), available to all agents

build_harness, get_coverage

build harness in OSS-Fuzz Docker; coverage measurement

AP_run_check

AP Probe 2 (§3.2); P1.x dynamic check

DAST tool

terminator

A

Turn cap

AP_reach_check

AP Probe 1 (§3.2); P2.x dynamic check

submit_logic_group / submit_p2_report / submit_harness

per-agent terminator tools (one each, ends the agent’s turn loop with its final output)

Controlled Saturation of LG Ranking

Both rank-6–10 OOMs pass our P1.5 buffer-safety check (libwebp harness caps fuzz input at 222 B, rapidjson at 220 B); the out-ofmemory states originate from unbounded library-internal allocation under bounded fuzz input, classified as resource-exhaustion artifacts rather than memory-safety crashes. Both have been reported to the respective upstreams. Table 6: LG-ranking saturation on 5 projects: rank-1–5 (deployed) vs rank-6–10, matched 10 h LibFuzzer/ASan budget. Bug counts deduplicated by ASan top-3 frame. Project

Rank 1–5 bugs

Rank 6–10 unique new

mongoose fwupd libwebp opc-ua rapidjson

3 2 2 2 1

0 0 1 (OOM) 0 1 (OOM)

Total

10

2 (resource exhaustion only)

Table 7: False-positive rate comparison. Crash and FP-rate values for prior tools are taken verbatim from the cited papers; the QuartetFuzz row reflects the 42 reports submitted in this work and the 2 maintainer-rejected cases (Table 14). Tool NEXZZER [27] UTopía [20] Hopper [7] AFGen (no constraints) [30] AFGen (full) [30] PromptFuzz [32] PromeFuzz [29] QuartetFuzz

Crashes

FP Rate

7,291 1,167 51 660 660 44 29 42

93.96% 41.6% 51.0% 53.03% 5.45% 13.6% 10.3% 4.8%

B

Danger Score Sensitivity

The danger score (Eq. 5) traverses the call graph up to depth 𝐷, default 𝐷=20. SAF projects rank LGs by LLM judgment rather than the formula, so 𝐷 does not apply to them; we sweep 𝐷 ∈ {10, 15, 20, 25, 30} on all RQ5 projects whose deployment ranking is formula-driven (SAF projects are excluded because indirect dispatch or unsupported source language prevents the call graph from producing a meaningful danger value). No project changes its rank-1 LG or top-3 LG set at any 𝐷 vs 𝐷=20; per-project Spearman 𝜌 is 1.000 for 𝐷 ∈ {15, 25, 30} and ≥ 0.9 for 𝐷=10 (mean 0.994), with the only deviation a single adjacent swap at ranks 4–5 in one project. 𝐷=20 sits on a stable plateau in [10, 30]; we adopt it as a conservative default that captures transitive reach in deeper call graphs without affecting top selections in shallower ones.

C

MCP Agent and Tools

Table 5 is keyed to §4. The two AP probes in the DAST row are already specified in §3.2 (Figure 2). The SAST row uses semantic names whose intent may not be obvious at a glance; Figure 10 fixes the input/output for each.

Conference’17, July 2017, Washington, DC, USA

SAST tools: input → output find_definition function name → definition site(s): file, line, language, external flag, complexity forward_reach function 𝑓 , depth 𝐷 (default 20) → functions reachable from 𝑓 within 𝐷 hops reverse_reach function 𝑓 , depth 𝐷 → functions that reach 𝑓 within 𝐷 hops, each tagged public / internal / unknown public_entries_for an internal core function → public APIs from which the core is reachable (candidate harness entries for that core) public_entries_for_batch list of core functions → per-core public APIs (in one tool call) is_public_api function name → public (defined under include/, public/, api/) / internal (under internal/, private/, src/, core/, util/, common/, impl/, detail/) / unknown (path matches neither; agent reads the header to decide)

Figure 10: Input/output of the six non-trivial SAST tools.

Ze Sheng, Dmitrijs Trizna, Luigino Camastra, Zhicheng Chen, Qingxiao Xu, and Jeff Huang

Quality-Assured Fuzz Harness Generation via the Four Principles Framework

Conference’17, July 2017, Washington, DC, USA

Table 8: The four principles: per-sub-check definitions and verification. P1 and P2 each have eight sub-checks; P3 cascades through three (structural → LLM-fallback → LLM-only when no call graph); P4 has two (structural + LLM-only when no call graph). Of the 16 P1/P2 sub-checks, 12 are operationalised by Adversarial Probing (AP) and 4 by static review. ID

Name

Purpose (it checks)

Probe input

Oracle/Tool

Fail signal

error-path input free-then-use input two distinct inputs in seq arbitrary input + marker bytes oversized input

LSan ASan ASan/LSan GDB

Sanitizer crash report Sanitizer crash report Sanitizer crash report∗ API call site silent

ASan

Sanitizer crash report API breakpoint fires flags signed-overflow / unaligned-cast / null-deref pattern LLM detects local copy

Principle 1: Logic Correctness — 8 sub-checks on the harness source P1.1 P1.2 P1.3 P1.4

Resource leaks Use-after-free Stale state Input flow

release of every alloc / fd / lock / handle on every exit path no pointer read or pass after free no static or global state across fuzz iterations fuzz-byte flow to target API (call not driven by constants)

P1.5

Buffer safety

P1.6 P1.7

Size checks Undefined behaviour

bounded fuzz-buffer reads; null-terminated C strings; length-checked indexing early-return when input is below API minimum no out-of-bounds / null-deref / signed-overflow / unaligned-cast

undersized input —

GDB LLM semantic†

P1.8

No reimplementation

real library code, not a local stub or copy

—

LLM semantic

Principle 2: API Protocol Compliance — 8 sub-checks on the harness–library interface P2.1

Init sequence

required predecessors in order before target call

arbitrary input

GDB

P2.2 P2.3

Parameter construction Object lifecycle

right type / range / owner / lifetime per parameter create → configure → use → destroy lifecycle on opaque objects

invalid parameters arbitrary input

ASan GDB

init breakpoint silent before target Sanitizer crash report any of create/use/destroy missing error branch silent Sanitizer crash report not under public-header path LLM detects mismatched pair prerequisite silent before target

P2.4 P2.5 P2.6 P2.7 P2.8

Return value handling Cleanup sequence API existence Co-call constraints Prerequisite state

return-code check; error-branch run; gated output use release of every resource on all exit paths in API order every called function exported by the library at pinned build paired APIs co-occurring; mutex APIs not external state (fds, sockets, env, threads) set up before and torn down after

error-return input early-exit input — — arbitrary input

GDB LSan static call graph LLM semantic GDB

public-API reach to target core (call graph) boundary respect by chosen internal entry (LLM, when P3.1 fails)

— —

static call graph LLM semantic

𝐸 pub = ∅

boundary respect over source (LLM, when call graph unavailable)

—

LLM semantic

LLM rejects boundary respect

— —

static call graph LLM semantic

reach = ∅ or danger (𝑒 ) = 0 LLM declares entry inadequate

Principle 3: Security Boundary Respect — public vs. internal entry boundary, three cascading checks P3.1 P3.2 P3.3

Boundary respect Boundary respect (P3.1 fail) Boundary respect (SAF)

LLM rejects boundary respect

Principle 4: Entry Point Adequacy — entry reaches a security-relevant target P4.1 P4.2

Entry adequacy Entry adequacy (SAF)

entry reach to core ∧ danger (𝑒 ) > 0 (call graph) entry adequacy on security-relevant target (LLM, when call graph unavailable)

SAF = static-analysis fallback (e.g., non-C/C++ projects without a precomputed call graph); the LLM takes over the corresponding judgment semantically. ∗ For P1.3, either oracle (ASan or LSan) firing on the two-input sequential probe counts as a violation; matches 𝑂 seq in §3.1. † LLM semantic = the agent autonomously explores source via code_view tools (read_file / list_directory / search_files) and judges the property by reasoning; no mechanical verdict from a sanitizer or call-graph computation.

Original

Original

P1 violation

Fixed

merged upstream

int LLVMFuzzerTestOneInput( const uint8_t *data, size_t n){ ... const T &inst = reinterpret_cast<T&>(data); // reads &data (stack addr) ... }

int LLVMFuzzerTestOneInput( const uint8_t *data, size_t n){ ... const T &inst = reinterpret_cast<T&>(*data); // reads fuzz input ... }

reads pointer address, not input Line cov: 0.05% (15 lines)

one-character fix: add dereference Line cov: 1.47% (486 lines; 29×)

P2 violation

Fixed

merged upstream

raster_t *src = raster_create( acquire_cb, release_cb, ...); surface = image_surface_from_png();

raster_t *src = raster_create( acquire_cb, release_cb, ...); surface = image_surface_from_png();

// PDF APIs on image surface: pdf_set_page_label(surface, buf); pdf_set_metadata(surface, KEYS, buf);

// PDF calls removed

cr = cairo_create(surface); cairo_set_source(cr, src); cairo_paint(cr); // short-circuits // acquire callback never fires

cr = cairo_create(surface); cairo_set_source(cr, src);

PDF APIs poison surface; paint dies Line cov: 0.57% (callback dead)

remove 2 calls; raster path live Line cov: 0.65% (+14%)

cairo_paint(cr); // proceeds // acquire callback fires

Figure 11: Representative P1 and P2 violations identified and repaired by the agent. Top: harfbuzz hb_set_fuzzer (P1)—a reinterpret_cast on a pointer variable reads its stack address instead of the fuzz input; one-character fix ((data)→(*data)) restores input flow and raises line coverage 29×. Bottom: cairo raster_fuzzer (P2)—two PDF-only APIs called on an image surface set CAIRO_STATUS_SURFACE_TYPE_MISMATCH, causing cairo_paint() to short-circuit and the acquire/release callbacks to never fire; removing the two calls makes the raster-source path reachable (merged upstream).

Conference’17, July 2017, Washington, DC, USA

Ze Sheng, Dmitrijs Trizna, Luigino Camastra, Zhicheng Chen, Qingxiao Xu, and Jeff Huang

Table 9: Evaluation dataset: 100 gold-standard harnesses from 39 OSS-Fuzz C/C++ projects, all verified P1–P4 clean. Target = the API that consumes the fuzz-byte input. Ver = project commit hash at experiment time (April 2026). LOC = lines of code in the gold harness source. #

Project

Fuzzer

Target

Ver

LOC

#

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50

apache-httpd apache-httpd binutils binutils boost boost boost boost boost brotli curl curl draco draco draco draco fftw3 freerdp glslang harfbuzz harfbuzz hwloc icu icu icu icu icu imagemagick imagemagick iperf jq jq libcoap libcoap libcoap libgit2 libgit2 libical libical libjxl libjxl libjxl libjxl libjxl libpcap libpcap libplist libplist libplist libplist

fuzz_addr_parse fuzz_tokenize fuzz_as fuzz_disassemble boost_graph_graphv.. inforead_fuzzer iniread_fuzzer jsonread_fuzzer xmlread_fuzzer decode_fuzzer curl_fuzzer_ftp fuzz_url mesh_decoder_fuzzer mesh_decoder_witho.. pc_decoder_fuzzer pc_decoder_without.. fftw3_fuzzer TestFuzzCodecs compile_fuzzer hb-shape-fuzzer hb-subset-fuzzer hwloc_fuzzer calendar_fuzzer date_time_pattern_.. normalizer2_fuzzer number_format_fuzzer unicode_string_cod.. encoder_mvg_fuzzer ping_fuzzer cjson_fuzzer jq_fuzz_parse jq_fuzz_parse_exte.. get_asn1_tag_target oscore_conf_parse_.. split_uri_target objects_fuzzer patch_parse_fuzzer libical_fuzzer libicalvcard_fuzzer cjxl_fuzzer color_encoding_fuz.. fields_fuzzer icc_codec_fuzzer set_from_bytes_fuz.. fuzz_both fuzz_filter bplist_fuzzer jplist_fuzzer oplist_fuzzer xplist_fuzzer

apr_parse_addr_port apr_tokenize_to_argv perform_an_assembl.. disassembler read_graphviz pt::read_info pt::read_ini pt::read_json pt::read_xml BrotliDecoderDecom.. fuzz_handle_transfer curl_url_set draco::Decoder::De.. draco::Decoder::De.. draco::Decoder::De.. draco::Decoder::De.. fftw_plan_dft_1d xcrush_decompress glslang::TShader s.. hb_shape hb_subset_or_fail hwloc_encode_to_ba.. icu::Calendar::cre.. icu::DateTimePatte.. icu::Normalizer2::.. icu::NumberFormat:.. icu::UnicodeString Magick::Image::read Magick::Image::ping cJSON_Parse jv_parse jv_parse_custom_fl.. get_asn1_tag coap_new_oscore_conf coap_split_uri git_object__from_raw git_patch_from_buf.. icalparser_parse_s.. vcardparser_parse_.. EncodeJpegXl jxl::ParseDescript.. jxl::Bundle::Read jxl::UnpredictICC jxl::SetFromBytes pcap_compile pcap_compile plist_from_bin plist_from_json plist_from_openstep plist_from_xml

1504691 1504691 9b9cbb0 9b9cbb0 1a80576 1a80576 1a80576 1a80576 1a80576 ab685df 70a1595 70a1595 77e616e 77e616e 77e616e 77e616e 6caf8ce 0c31662 aa8e19e e8fbf40 e8fbf40 cfe0433 0d84e02 0d84e02 0d84e02 0d84e02 0d84e02 5e318be 5e318be 896cc42 fb59f14 fb59f14 86b7781 86b7781 86b7781 1f34e2a 1f34e2a 276b8bd 276b8bd 6553831 6553831 6553831 6553831 6553831 44aa24f 44aa24f dddb76d dddb76d dddb76d dddb76d

37 34 77 110 57 46 46 46 64 63 584 55 29 30 29 30 40 467 32 375 356 36 136 58 83 93 83 121 50 38 21 36 17 46 8 49 40 44 45 260 36 107 115 77 113 44 32 32 32 32

51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100

Project

Fuzzer

Target

Ver

LOC

libxslt libxslt libyaml libyaml llamacpp mbedtls mbedtls mbedtls mbedtls ndpi ndpi ndpi ndpi ndpi ndpi ndpi ndpi ndpi ndpi ndpi opencv openexr openexr openssh openssh openssh openssh openssh openssl openssl openssl openssl php php php pjsip pjsip pjsip pjsip pugixml pugixml quickjs quickjs strongswan strongswan wabt yajl-ruby zlib zlib zopfli

xpath xslt libyaml_loader_fuz.. libyaml_scanner_fu.. fuzz_json_to_grammar fuzz_pkcs7 fuzz_x509crl fuzz_x509crt fuzz_x509csr fuzz_community_id fuzz_dga fuzz_ds_tree categories category fuzz_filecfg_config malicious_ja4 malicious_sha1 protocols risk_domains fuzz_process_packet imread_fuzzer openexr_exrcheck_f.. openexr_exrcoreche.. authopt_fuzz kex_fuzz privkey_fuzz sig_fuzz sshsigopt_fuzz acert asn1parse cms v3name fuzzer-json fuzzer-unserialize fuzzer-unserialize.. fuzz-crypto fuzz-dns fuzz-sip fuzz-srtp fuzz_parse fuzz_xpath fuzz_compile fuzz_eval fuzz_crls fuzz_ids wasm2wat_fuzzer json_fuzzer compress_fuzzer zlib_uncompress2_f.. zopfli_deflate_fuz..

xsltFuzzXPath xsltFuzzXslt yaml_parser_load yaml_parser_scan json_schema_to_gra.. mbedtls_pkcs7_pars.. mbedtls_x509_crl_p.. mbedtls_x509_crt_p.. mbedtls_x509_csr_p.. ndpi_flowv4_flow_h.. ndpi_check_dga_name ndpi_tsearch load_categories_fi.. load_category_file.. load_config_file_fd load_malicious_ja4.. load_malicious_sha.. load_protocols_fil.. load_risk_domain_f.. ndpi_detection_pro.. cv::imread checkOpenEXRFile checkOpenEXRFile sshauthopt_parse ssh_init sshkey_private_des.. sshkey_verify sshsigopt_parse d2i_X509_ACERT ASN1_parse_dump i2d_CMS_bio GENERAL_NAME_cmp php_json_yyparse php_var_unserialize php_var_unserialize pj_base64_encode pj_dns_parse_packet pjsip_parse_msg pjmedia_transport_.. pugi::xml_document.. evaluate_node_set JS_Eval JS_Eval lib->creds->create identification_cre.. ReadBinaryIr yajl_parse compress2 uncompress2 ZopfliDeflate

35323d6 35323d6 840b65c 840b65c cf8b0db 391af7c 391af7c 391af7c 391af7c 315a705 315a705 315a705 315a705 315a705 315a705 315a705 315a705 315a705 315a705 315a705 9f101a1 53cfa83 53cfa83 45b30e0 45b30e0 45b30e0 45b30e0 45b30e0 087bddc 087bddc 087bddc 087bddc e07d066 e07d066 e07d066 2965cff 2965cff 2965cff 2965cff e56134e e56134e d7ae12a d7ae12a 60f4c86 60f4c86 77a95c8 6501652 f9dd600 f9dd600 ccf9f05

21 22 51 51 29 21 40 40 40 58 46 99 24 24 24 24 42 24 24 41 16 16 15 33 454 20 62 29 48 45 55 45 61 68 82 181 78 489 201 14 46 93 49 41 34 26 104 99 16 45

Quality-Assured Fuzz Harness Generation via the Four Principles Framework

Conference’17, July 2017, Washington, DC, USA

Gold harness

QuartetFuzz harness

+8.0pp

LLVMFuzzerTestOneInput(data, size) {

LLVMFuzzerTestOneInput(data, size) {

Gold harness

FuzzedDataProvider fdp(data, size);

ReadBinaryOptions options;

Features features;

Errors errors;

#define WABT_FEATURE(v, ...) \ Module module;

if (fdp.ConsumeBool())

ReadBinaryIr("dummy filename",

features.enable_##v();

data, size, options,

#include "wabt/feature.def"

&errors, &module);

ReadBinaryOptions options(features, ...); auto bytes = fdp.ConsumeRemainingBytes();

/* discard module */

ReadBinaryIr("dummy", bytes.data(), return 0;

bytes.size(), options, ...);

+4.2pp

QuartetFuzz harness

/* apply_extended_ops, called by LLVMFuzzerTestOneInput */ while (p < end) { uint8_t op; if (!read_value(p, end, &op)) return; switch (op) { case OP_SET_FLAGS: hb_subset_input_set_flags(input, ...); case OP_KEEP_EVERYTHING: hb_subset_input_keep_everything(input); case OP_SET_CLEAR / SET_INVERT: hb_set_clear / hb_set_invert(set); case OP_SET_ADD/DEL_RANGES: hb_set_add_range / del_range(...); case OP_TEXT_ADD / TEXT_DEL: ... case OP_AXIS_PIN_ALL_TO_DEFAULT: hb_subset_input_pin_all_axes(...); case OP_AXIS_SET: hb_subset_input_set_axis_range(...); } /* + 4 more ops, 10+ APIs */ }

LLVMFuzzerTestOneInput(data, size) { blob

= hb_blob_create(data, size, HB_MEMORY_MODE_READONLY, ...);

face

= hb_face_create(blob, 0);

input = hb_subset_input_create_or_fail(); if (input) { hb_subset_input_keep_everything(input); result = hb_subset_or_fail(face, input); hb_face_destroy(result); hb_subset_input_destroy(input); } /* destroy face/blob; return 0 */

}

}

bare ReadBinaryIr; default options

randomises every WABT feature flag

op-code grammar drives 10+ hb_subset_* APIs

single keep_everything pass

Line cov: 51.4%

Line cov: 59.4%

Line cov: 7.3%

Line cov: 3.1%

}

Figure 12: Representative RQ3 coverage win and loss vs. gold on shared entries. Left: wabt/wasm2wat (+8.0pp)—QF wraps the ReadBinaryIr call with a FuzzedDataProvider that randomises every WABT feature flag, exercising parser paths gold’s default-options call never hits. Right: harfbuzz/subset (−4.2pp)—gold parses an op-code grammar that drives 10+ hb_subset_* APIs (axis pinning, set clear/invert, range add/del, . . .); QF runs a single keep_everything pass, missing the breadth. Table 10: RQ1 audit corpus: 70 OSS-Fuzz C/C++ projects, 586 unique fuzzer source files. Stars = GitHub stars; Fz = fuzzer source files audited; P1/P2 = principle violations identified; Sub = issues submitted upstream; Conf/Fix = confirmed / fixed by maintainers. Bold project names indicate at least one issue reported upstream. #

Project

Stars

Fz

P1

P2

Sub

Cf/Fx

#

Project

Stars

Fz

P1

P2

Sub

Cf/Fx

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35

apache-httpd assimp binutils boost botan brotli bzip2 c-ares cairo curl dnsmasq draco duckdb elfutils expat ffmpeg fftw3 file freerdp gdk-pixbuf ghostscript glslang harfbuzz hwloc icu igraph imagemagick iperf jq json-c lcms libarchive libcoap libgit2 libheif

3.9k 12.9k — 8.4k 3.2k 14.7k 56 2.1k — 41.3k — 7.2k 37.6k — 1.3k 59.1k 3.1k 1.6k 13.1k 33 218 3.5k 5.6k 692 3.5k 2.0k 16.2k 8.4k 34.5k 3.3k 714 3.5k 903 10.4k 2.2k

7 7 15 14 34 1 4 4 4 3 5 4 1 3 2 6 1 3 7 5 17 1 6 1 33 26 9 2 6 4 15 25 9 8 4

2 0 1 2 1 0 2 0 0 0 0 0 0 1 0 0 0 0 0 3 1 0 1 0 5 0 0 0 1 0 5 1 0 0 0

0 0 0 0 0 0 1 0 1 0 0 0 0 1 0 0 0 0 0 2 0 0 0 0 4 0 0 0 0 0 2 0 0 0 0

2 0 1 2 1 0 2 0 1 0 0 0 0 2 0 0 0 0 0 3 1 0 1 0 7 0 0 0 1 0 5 1 0 0 0

2/2 0/0 1/1 1/1 1/1 0/0 2/2 0/0 1/1 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 0/0 3/3 1/1 0/0 1/1 0/0 7/0 0/0 0/0 0/0 1/1 0/0 4/3 1/1 0/0 0/0 0/0

36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70

libical libjxl libmicrohttpd2 libpcap libplist libpng libraw libssh2 libxslt libyaml llamacpp mbedtls ndpi nettle njs opencv openexr openssh openssl openthread openvpn php pjsip pugixml quickjs ruby strongswan tidy-html5 unbound wabt wamr woff2 yajl-ruby zlib zopfli

348 3.5k — 3.1k 616 1.6k 1.5k 1.5k 71 1.1k 105.0k 6.6k 4.4k — 1.6k 87.2k 1.8k 3.8k 30.0k 3.9k 13.5k 40.0k 2.6k 4.5k 10.6k 23.5k 2.8k 2.9k 4.4k 8.0k 5.9k 1.8k 1.5k 6.8k 3.6k

3 11 9 5 4 5 1 1 2 9 8 8 61 7 1 9 2 11 30 6 12 10 17 2 3 10 8 6 5 5 2 2 1 12 2

0 0 0 0 0 1 0 1 0 1 0 0 1 0 1 2 0 1 1 0 2 0 0 0 0 0 0 1 1 0 0 0 0 4 0

0 0 0 1 0 2 0 0 0 0 0 0 0 0 0 1 0 0 1 0 0 0 0 0 0 0 0 1 0 0 1 0 0 0 0

0 0 0 1 0 2 0 1 0 1 0 0 1 0 1 3 0 1 2 0 2 0 0 0 0 0 0 2 1 0 1 0 0 4 0

0/0 0/0 0/0 1/1 0/0 2/2 0/0 1/1 0/0 1/1 0/0 0/0 1/1 0/0 1/1 1/1 0/0 0/0 2/1 0/0 2/2 0/0 0/0 0/0 0/0 0/0 0/0 2/2 1/1 0/0 1/1 0/0 0/0 3/2 0/0

Total: 70 projects (28 in bold with upstream reports), 586 harnesses, 53 P1/P2 reports submitted; 45 confirmed, 35 fixed.

One additional report (elfutils fuzz-dwfl-core) was withdrawn after a maintainer noted that dwfl_end and elf_end already handle NULL gracefully, making the added NULL checks unnecessary—our only P1 false positive, not counted in the 53 above.

Conference’17, July 2017, Washington, DC, USA

Ze Sheng, Dmitrijs Trizna, Luigino Camastra, Zhicheng Chen, Qingxiao Xu, and Jeff Huang

Table 11: Quality assurance comparison of LLM-based harness generators for C/C++ libraries. Rust-targeting work (deepSURF [2]) is discussed in §7. Correctness Tool

Verification

Timing

Venue

Analysis Scope

Compil.

Semantic

API

Entry Pt.

Runtime

Coverage

Pre

Post

Code

PromptFuzz [32] PromeFuzz [29] CKGFuzzer [45] OSS-Fuzz-Gen [28]

CCS ’24 CCS ’25 ICSE-C ’25 Google

Headers; AST/CFG AST + consumer call graph + RAG Tree-sitter + CodeQL KG Headers, Introspector

✓ ✓ ✓ ✓

✗ ✗ ✗ ✗

✗ ✗ ✗ ✗

✗ ✗ ✗ ✗

✓ ✓ ✓ ✓

✓ ✓ ✓ ✓

✗ ✗ ✗ ✗

✓ ✓ ✓ ✓

✓ ✓ ✓ ✓

HarnessAgent [48] Scheduzz [24]

arXiv ’25 arXiv ’25

LSP, Tree-sitter Headers; Prolog solver

✓ ✓

✗ ✗

✗ ✓

✗ ✗

✓ ✓

✓ ✓

✗ ✓

✓ ✓

✗† ✗

Full codebase, call graph

✓

✓

✓

✓

✓

✓

✓

✓

✓

QuartetFuzz

Compil.: language correctness and build script validity. Semantic: pre-fuzz analysis of the harness source for logic errors, resource management, and harness-introduced bugs (post-fuzz crash root-cause classification is captured by the Post column). Tools that only check API existence (anti-hallucination) or argument types pre-fuzz, without reading the harness body for logic/resource/lifecycle issues, are marked ✗. API: explicit runtime call-order/lifecycle/protocol verification (Scheduzz uses Prolog-based constraint solving; ours uses LLM-driven evidence-grounded protocol reports). Tools that retrieve API context (signatures, definitions, headers, RAG) without verifying call protocol are marked ✗. Entry Pt.: security boundary respect and attack-surface adequacy. Pre: checked before fuzzing. Post: checked after fuzzing. Code: artifact publicly available for empirical comparison at our submission time. † HarnessAgent provides an anonymous artifact at submission via 4open.science but is not packaged for re-execution on our benchmark.

Table 12: Full coverage comparison (line, branch %) vs. three baselines. G = Gold, O = OFG, P = PromeFuzz, Q = QuartetFuzz; AP = QuartetFuzz Adversarial Probing calls per case (0 if the case never produced a Stage-3 binary; Average is over the 96 productive cases). Each harness fuzzed under LibFuzzer for 10 × 600 s (empty corpus, ASan); per-cell value is the median across the 10 runs of llvm-cov line/branch. Line

Branch

Line

#

Case

G

O

P

Q

G

O

P

Q

AP

#

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50

ap-h/addr_parse ap-h/tokenize binutils/as binutils/disasm boost/graphviz boost/inforead boost/iniread boost/jsonread boost/xmlread brotli/decode curl/curl-ftp curl/url draco/mesh draco/mesh_nodq draco/pc draco/pc_nodq fftw3/fftw3 freerdp/Codecs glslang/compile harfbuzz/shape-fuzzer harfbuzz/subset hwloc/hwloc icu/calendar icu/dt icu/normalizer2 icu/num-fmt icu/ustr-cp imgmk/encoder_mvg imgmk/ping iperf/cjson jq/parse jq/prs-ext libcoap/asn1 libcoap/oscore libcoap/uri libgit2/objects libgit2/patch libical/libical libical/vcard libjxl/cjxl libjxl/col-enc libjxl/fields libjxl/icc libjxl/frm-byt libpcap/both libpcap/filter libplist/bplist libplist/jplist libplist/oplist libplist/xplist

1.0 1.1 2.3 18 37 56 54 62 60 77 5.0 4.0 11 11 3.6 4.2 14 14 23 19 7.3 15 16 14 13 14 9.0 1.0 4.9 24 3.5 7.2 1.1 1.8 1.2 2.3 1.5 12 8.5 21 81 25 7.2 15 20 24 9.9 12 10 11

1.6 1.4 0 17 34 0 0 0 0 77 0 4.0 0 0 0 0 21 13 0 19 8.3 6.2 0 0 0 0 0 0 0 24 3.1 4.2 0 0 0 1.5 1.2 11 14 0 0 0 0 0 18 25 6.4 6.6 1.3 11

98 98 0.5 0 0 0 0 0 0 23 2.7 0.5 12 9.8 11 10 84 0 0.3 1.0 1.5 7.4 4.2 4.2 4.2 4.2 4.2 1.8 1.8 27 3.3 0 0.3 0.3 0.3 10 0 15 16 25 23 0 25 23 0.8 0.9 24 22 22 7.2

1.6 1.4 0 8.6 37 55 54 62 59 80 0 0.4 20 7.3 3.4 3.0 21 14 21 19 3.1 14 0 14 16 14 13 0.8 5.1 24 3.4 6.4 0.5 1.8 1.8 2.3 1.5 13 0 21 80 25 7.2 17 19 24 9.7 12 10 11

0.5 0.5 1.9 16 33 79 60 85 68 72 3.9 3.2 9.7 10 2.5 3.7 31 10 21 23 4.6 86 12 11 11 11 6.6 0.6 3.7 27 3.5 7.1 0.3 1.1 0.8 1.8 1.0 11 9.1 15 97 45 17 15 22 27 8.3 10 8.0 8.4

0.9 0.9 0 15 32 0 0 0 0 69 0 3.1 0 0 0 0 32 9.6 0 23 5.6 17 0 0 0 0 0 0 0 26 3.2 4.6 0 0 0 1.1 0.9 9.8 13 0 0 0 0 0 20 26 5.3 4.8 0.7 9.1

57 57 0.5 0 0 0 0 0 0 17 0.9 0.3 13 11 13 12 58 0 0.1 0.5 0.7 73 2.3 2.3 2.3 2.3 2.3 1.4 1.4 29 3.1 0 0.1 0.1 0.1 8.1 0 15 16 32 30 0 32 28 0.4 0.5 21 19 19 5.0

0.9 0.9 0 7.0 33 79 60 85 64 75 0 0.4 19 6.3 3.0 2.1 32 10 19 23 2.0 63 0 11 16 11 10 0.6 4.0 26 3.5 6.4 0.3 1.1 1.4 1.8 1.0 12 0 15 96 45 18 17 21 27 8.1 11 8.0 8.4

7 6 0 1 2 2 4 2 2 3 0 7 2 3 2 2 3 2 3 2 3 3 0 3 5 3 6 2 4 7 2 3 4 4 4 10 10 2 0 2 2 1 3 2 3 2 2 3 2 2

51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100

Average Productive rate Avg. cost Model

17.7 100 -

17.7 96 $1.65

17.6

9.2

13.4

17.5

3.77

10.8 12.5 64 74 $2.43 $1.98 claude-sonnet-4-6

Branch

Case

G

O

P

Q

G

O

P

Q

AP

libxslt/xpath libxslt/xslt libyaml/loader libyaml/scanner llama/js-grm mbedtls/pkcs7 mbedtls/x509crl mbedtls/x509crt mbedtls/x509csr ndpi/cm-id ndpi/dga ndpi/ds_tree ndpi/fc-cat ndpi/fc-cty ndpi/fc-cfg ndpi/fc-mj ndpi/fc-ms ndpi/fc-pro ndpi/fc-rsk ndpi/proc-pkt opencv/imread openexr/exrcheck openexr/corecheck openssh/authopt openssh/kex openssh/privkey openssh/sig openssh/sshsigopt openssl/acert openssl/asn1parse openssl/cms openssl/v3name php/json php/unseria php/unsr-h pjsip/crypto pjsip/dns pjsip/sip pjsip/srtp pugixml/parse pugixml/xpath quickjs/compile quickjs/eval sswan/crls sswan/ids wabt/wasm2wat yajl-ruby/json zlib/compress zlib/uncomp2 zopfli/deflate

16 9.8 78 71 3.4 1.0 1.5 1.7 1.4 1.2 16 1.0 6.3 6.3 6.0 5.8 1.1 7.6 6.7 48 4.4 5.3 5.1 4.2 9.5 6.1 5.4 1.3 2.7 2.2 2.4 2.6 5.0 4.8 4.9 54 20 14 22 13 42 26 25 11 8.0 51 69 55 53 86

1.3 7.8 74 70 3.3 0.8 1.5 1.7 1.4 0.5 16 0.5 6.0 6.0 5.7 5.5 5.5 7.5 6.4 47 0 3.5 3.9 3.6 20 4.9 2.9 0 2.1 2.2 2.7 1.8 0 0 0 46 13 1.3 14 0 42 26 25 0 5.9 0 69 55 53 83

1.0 6.5 21 69 0 1.9 1.6 1.9 1.6 4.8 9.5 0 4.5 4.5 4.5 4.5 4.5 4.5 4.5 4.8 0 6.5 6.5 3.6 0 5.7 2.9 0 3.0 0 0 0 0 0 0 49 38 49 49 15 22 18 18 0 0 0 2.7 52 53 72

16 6.8 78 71 3.8 3.1 3.7 3.8 3.6 0.9 16 0.6 6.3 6.3 6.0 5.8 5.9 7.9 6.7 47 4.5 5.3 5.0 7.9 9.1 6.3 5.4 1.3 2.6 2.2 2.4 2.8 5.0 4.9 5.0 57 20 12 22 15 43 28 27 11 7.9 59 78 51 54 86

14 7.8 66 63 3.6 0.6 1.3 1.5 1.2 0.5 7.3 0.3 2.4 2.7 1.7 1.9 0.9 3.5 2.3 39 3.2 4.3 4.3 4.4 10 4.8 4.3 1.4 2.1 1.5 1.8 2.1 1.2 1.1 1.2 60 9.5 7.9 11 13 36 23 22 7.0 4.2 59 53 45 50 70

0.4 6.2 60 62 3.4 0.6 1.3 1.5 1.2 0.3 7.2 0.3 2.0 2.2 1.3 1.5 1.5 3.2 1.8 38 0 2.8 3.3 3.5 20 4.1 2.3 0 1.5 1.6 2.1 1.3 0 0 0 50 6.2 1.0 7.2 0 35 22 21 0 3.1 0 52 45 43 68

0.5 3.6 10 65 0 1.5 1.5 1.8 1.6 1.6 3.2 0 1.5 1.5 1.5 1.5 1.5 1.5 1.5 1.5 0 25 25 3.5 0 4.6 2.3 0 2.3 0 0 0 0 0 0 72 80 72 72 14 22 56 56 0 0 0 0.6 42 43 54

14 5.2 67 64 3.8 2.4 3.0 3.0 2.9 0.5 7.3 0.4 2.5 2.7 1.7 1.9 2.0 3.8 2.3 40 3.4 4.3 4.2 7.7 8.9 4.9 4.3 1.4 1.9 1.6 1.8 2.0 1.3 1.2 1.2 64 9.9 6.5 11 16 37 24 24 6.9 5.3 68 69 42 49 70

2 1 3 3 3 1 8 5 2 7 6 11 9 7 6 9 9 8 9 7 2 2 3 1 2 2 3 3 2 3 2 2 3 3 4 7 2 1 2 2 5 3 3 2 3 2 2 12 7 2

Quality-Assured Fuzz Harness Generation via the Four Principles Framework

Conference’17, July 2017, Washington, DC, USA

Table 13: RQ4 ablation: per-case coverage (%) under four configurations. Each harness fuzzed under LibFuzzer for 10 × 600 s (empty corpus, ASan); per-cell value is the median across the 10 runs (llvm-cov line/branch). AP = without adversarial probe gate; P2 = without API protocol report; St = without call-graph tools; Gm = full pipeline with Gemini 3.1 Flash. All four are ablations of the full pipeline whose aggregate appears in Table 3. Line

Branch

Line

#

Case

AP

P2

St

Gm

AP

P2

St

Gm

#

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50

ap-h/addr_par ap-h/tokenize binutils/as binutils/disassem boost/graph_gr boost/ptree_in boost/ptree_in boost/ptree_js boost/ptree_xm brotli/decode_ curl/curl__ft curl/url draco/draco_me draco/draco_me draco/draco_pc draco/draco_pc fftw3/fftw3_ freerdp/TestFuzz glslang/compile_ harfbuzz/hb-shape harfbuzz/hb-subse hwloc/hwloc_ icu/calendar icu/date_tim icu/normaliz icu/number_f icu/unicode_ imgmk/encoder_ imgmk/ping_ iperf/cjson_ jq/jq_parse jq/jq_parse libcoap/get_asn1 libcoap/oscore_c libcoap/split_ur libgit2/objects_ libgit2/patch_pa libical/libical_ libical/libicalv libjxl/cjxl_ libjxl/color_en libjxl/fields_ libjxl/icc_code libjxl/set_from libpcap/both libpcap/filter libplist/bplist_ libplist/jplist_ libplist/oplist_ libplist/xplist_

1.6 1.4 0 8.3 37 0 54 62 59 78 0 0.4 4.8 8.4 15 15 21 14 21 19 3.1 14 16 14 16 15 18 0.8 5.5 24 3.5 6.5 0.5 1.9 2.0 2.3 1.5 13 6.0 21 81 25 7.2 16 27 25 9.9 12 11 13

1.6 1.4 0 8.9 37 57 54 62 0 77 0 0.4 0 8.4 3.5 5.1 21 14 21 19 3.2 14 16 12 16 15 13 0.8 5.4 25 3.5 6.5 0 0 0.8 2.5 1.5 12 8.5 21 80 25 7.3 19 24 21 14 12 11 14

1.6 1.4 2.4 8.9 37 57 54 62 59 77 5.0 0.4 6.7 4.8 5.0 3.6 21 14 21 19 2.9 14 16 14 16 16 11 0.8 5.0 25 3.5 7.3 0 0 0 2.3 1.5 12 7.9 21 80 25 7.2 18 22 25 9.9 12 10 11

1.6 1.4 0 8.4 37 55 54 62 62 78 3.6 0.4 6.9 18 7.2 2.9 21 13 22 19 3.0 14 16 12 16 16 16 0.8 5.0 25 3.5 6.4 0 0 0 2.3 1.5 14 6.0 21 80 25 7.2 21 19 25 10 12 10 13

0.9 0.9 0 6.7 33 0 60 85 64 73 0 0.4 4.3 7.3 13 15 32 10 19 23 2.0 61 12 11 16 12 15 0.7 4.3 26 3.6 6.4 0.3 1.1 1.6 1.9 1.0 11 6.1 15 97 45 17 16 30 27 8.3 11 8.3 9.8

0.9 0.9 0 7.2 33 81 60 86 0 72 0 0.4 0 8.1 2.9 3.8 33 10 19 23 2.1 62 12 8.5 17 12 9.8 0.7 4.2 28 3.6 6.5 0 0 0.8 2.1 1.0 11 9.2 15 96 45 18 17 27 25 12 10 8.3 12

0.9 0.9 2.0 7.2 33 81 60 85 66 73 3.9 0.4 5.5 3.4 4.0 3.1 32 10 19 23 1.7 63 12 11 17 13 9.5 0.7 3.9 27 3.6 7.2 0 0 0 1.8 1.0 11 8.3 15 96 45 18 17 24 28 8.3 11 8.1 7.9

0.8 0.9 0 6.8 33 79 60 85 73 71 2.4 0.4 6.0 17 6.7 2.0 32 10.0 19 23 1.9 62 12 8.5 17 13 12 0.7 3.9 27 3.6 6.3 0 0 0 1.8 1.0 12 6.3 15 96 45 18 20 22 28 8.4 11 8.1 10

51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100

16.9 84 $1.04

16.2 83 $1.35

17.5 88 $1.14

17.5 89 $1.64

16.6

15.9

17.3

17.3

Avg Prod. Cost

Branch

Case

AP

P2

St

Gm

AP

P2

St

Gm

libxslt/xpath libxslt/xslt libyaml/libyaml_ libyaml/libyaml_ llama/s/json_t mbedtls/pkcs7 mbedtls/x509crl mbedtls/x509crt mbedtls/x509csr ndpi/communit ndpi/dga ndpi/ds_tree ndpi/filecfg_ ndpi/filecfg_ ndpi/filecfg_ ndpi/filecfg_ ndpi/filecfg_ ndpi/filecfg_ ndpi/filecfg_ ndpi/process_ opencv/imread_ openexr/openexr_ openexr/openexr_ openssh/authopt_ openssh/kex_fuzz openssh/privkey_ openssh/sig_fuzz openssh/sshsigop openssl/acert openssl/asn1pars openssl/cms openssl/v3name php/-json php/-unseria php/-unseria pjsip/crypto pjsip/dns pjsip/sip pjsip/srtp pugixml/parse pugixml/xpath quickjs/compile quickjs/eval sswan/crls sswan/ids wabt/wasm2wat yajl/json_ zlib/compress zlib/zlib_unc zopfli/zopfli_d

16 8.6 78 71 3.8 0 0 0 0 0.9 16 0.6 6.3 6.3 6.0 5.8 0.4 7.8 6.7 47 4.7 5.1 5.2 0 0 0 0 0 2.1 2.2 2.4 2.2 0 0 0 56 19 0 22 15 41 28 27 11 9.3 60 79 51 53 86

16 0 78 71 3.7 0.8 0 0 0 0.8 16 0.6 6.3 6.3 6.1 5.8 0.4 8.0 6.7 44 4.1 5.2 4.8 0 0 0 0 0 2.2 2.2 2.4 2.8 5.2 0 0 17 15 18 22 15 42 28 27 11 8.1 61 70 51 53 86

15 9.8 78 71 3.7 1.0 1.8 1.7 1.4 0.8 16 0.6 6.3 6.3 6.0 5.8 0.4 8.0 6.7 50 4.2 5.3 4.9 0 9.5 6.0 0 0 2.2 2.2 2.4 0 0 0 5.0 57 15 12 22 15 41 28 26 11 9.4 56 78 51 54 86

16 9.4 78 71 3.5 0 0 0 3.7 0.8 16 0.6 6.3 6.3 6.0 5.8 0.4 8.0 6.7 45 4.7 5.0 4.8 3.2 7.2 4.5 0 0 2.1 2.2 2.4 2.7 0 4.9 0 56 15 16 16.8 15 39 28 26 11 7.1 56 79 48 54 86

14 7.0 67 63 3.9 0 0 0 0 0.5 7.2 0.4 2.4 2.7 1.7 1.9 0.9 3.8 2.3 41 3.5 4.3 4.5 0 0 0 0 0 1.5 1.6 1.8 1.7 0 0 0 63 12 0 11 15 35 24 24 7.0 5.7 68 69 41 49 70

13 0 66 64 3.7 0.5 0 0 0 0.5 7.2 0.4 2.4 2.7 1.8 1.9 0.9 3.9 2.3 37 2.9 4.2 4.0 0 0 0 0 0 1.6 1.6 1.8 2.2 1.4 0 0 6.9 8.5 9.7 11 16 35 24 24 7.0 5.4 70 64 41 49 71

13 8.0 66 63 3.8 0.8 1.5 1.5 1.2 0.5 7.2 0.4 2.4 2.7 1.7 1.9 0.9 3.9 2.3 43 3.1 4.3 4.1 0 10 4.7 0 0 1.6 1.5 1.8 0 0 0 1.2 65 8.7 7.2 11 15 35 25 23 6.9 5.8 64 69 42 50 70

13 7.7 66 63 3.6 0 0 0 3.0 0.5 7.2 0.5 2.4 2.7 1.8 2.1 0.9 3.8 2.3 38 3.5 4.1 4.1 3.4 7.8 3.5 0 0 1.5 1.6 1.8 2.1 0 1.2 0 61 8.7 9.2 8.5 15 34 24 23 7.0 4.4 64 69 38 50 70

AP = w/o adversarial probe gate; P2 = w/o P2 report; St = w/o static analysis; Gm = Gemini 3.1 Flash. Prod. = productive (harness builds and exercises non-zero target-library lines).

Conference’17, July 2017, Washington, DC, USA

Ze Sheng, Dmitrijs Trizna, Luigino Camastra, Zhicheng Chen, Qingxiao Xu, and Jeff Huang

Table 14: All 44 submitted vulnerability reports across 24 projects: 42 from RQ5 deployment (B–D) and 2 from RQ1 audit-fix (A, latent library bugs surfaced while repairing harness violations). (A) RQ1 audit-fix; (B) C/C++ generation; (C) Java/JS (LLM-only LG ranking); (D) rejected by maintainers. Danger (Rank) = per-LG danger score (Eq. 5) and rank among 5 candidates. SAST : J = Joern, TS = tree-sitter; SAF = both backends fail on indirect dispatch (vtable / fn ptr), rank by LLM judgment. Status: fixed, confirmed, submitted, fp. #

Project

SAST

Danger (Rank)

Bug-finding LG (entry)

Bug Type

CVSS

Status

stack overread memory leak

7.5 (H) 5.5 (M)

fix fix

7.5 (H) 3.3 (L) 7.5 (H) 5.5 (M) 5.5 (M) 5.5 (M) 7.5 (H) 7.5 (H) 5.5 (M) 5.5 (M) 5.5 (M) 5.5 (M) 7.5 (H) 5.5 (M) 9.1 (C) 7.5 (H) 5.5 (M) 5.5 (M) 7.5 (H) 5.5 (M) 5.5 (M) 7.5 (H) 7.5 (H) 5.5 (M)

fix fix conf fix fix sub sub sub fix (CVE-2026-xxxxx) fix (CVE-2026-xxxxx) sub sub sub fix sub sub conf fix sub fix fix sub fix sub

7.5 (H) 7.5 (H) 7.5 (H) 7.5 (H) 5.5 (M) 5.5 (M)

conf conf conf conf fix conf

(A) Bugs found by fixing existing harness violations (RQ1) 1 2

openssl tidy-html5

– –

– –

P2 fix: call-order repair in provider fuzzer (EVP_EncryptInit_ex2) P2 fix: added missing tidyCleanAndRepair call

(B) C/C++ projects — bugs found by directly running generated harnesses 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32

ICU opencv openh264 ghidra vorbis capnproto imagemagick★ ★

tidy-html5

fwupd freetype libheif opc-ua libvpx mongoose net-snmp rapidjson libwebp

libaom libvpx flatbuffers

J J J J J J J J J J J J J J J J J J J J J J J J

121.9 (1) 94.2 (1) 20.7 (1) 14.6 (1) 14.6 (1) 8.5 (1) 8.5 (1) 7.9 (1) 13.2 (2) 13.2 (2) 5.8 (1) 5.8 (2) 5.8 (1) 4.0 (2) 1.0 (3) 2.1 (3) 3.3 (1) 3.3 (1) 14.2 (1) 28.6 (2) 2.8 (4) 18.0 (4) 5.1 (5) 2.8 (2)

Transliteration rule compilation (Transliterator::createFromRules) YAML/XML FileStorage parsing (cv::FileStorage::open) H.264 bitstream decoding (ISVCDecoder::DecodeFrame2) C++ ABI demangling (cplus_demangle) Rust v0 demangling (rust_demangle) Ogg/Vorbis header parsing (vorbis_synthesis_headerin) Vorbis residue decoding (vorbis_synthesis_blockin) Cap’n Proto text codec (capnp::TextCodec::decode) MSL script decoding (ReadMSLImage) MSL script decoding (ReadMSLImage) Document clean/repair (tidyCleanAndRepair) Error buffer accumulation (tidyErrorSummary) Generated-doc cleanup (TidyGDocClean) Logitech RDFU firmware parsing (fu_logitech_rdfu_write_firmware) Glyph bitmap copy (FT_Bitmap_Copy) HEIF image crop (heif_image_crop) PubSub JSON configuration decode (UA_PubSub_decodeJson) EventFilter parsing (UA_EventFilter_parse) Y4M input parsing (y4m_input_open) HTTP pattern matching (mg_match) VACM config parsing (vacm_parse_config_group) JSON Schema regex validation (SchemaValidator::Validate) WebP container assembly (WebPMuxAssemble) SharpYuv conversion (SharpYuvConvert, FixedPointInterpolation)

TS TS TS TS TS TS

2.3 (3) SAF (3) SAF (3) SAF (4) 5.7 (2) 1.0 (3)

AV1 rate control layer-context restore (AV1RateControlRTC::ComputeQP) VP9 complexity-adaptive quantisation (vpx_codec_encode) VP9 encoder midstream reconfiguration (vpx_codec_enc_config_set) FlexBuffers accessor traversal (flexbuffers::Reference::ToString) Reflection verifier field deref (flatbuffers::Verify, GetFieldT) Binary code generation (GenerateBinary)

UAF heap overflow heap overflow OOM OOM memory leak divide-by-zero stack overflow NULL deref stack overflow memory leak OOM UAF stack overflow UAF heap overflow assertion NULL deref integer overflow heap overflow NULL deref assertion NULL deref out-of-bounds read heap overflow integer overflow heap overflow heap overflow heap overflow NULL deref

HTTP request parser (mg_match, s.buf[j] path) HTTP request parser (mg_match, write path) UTF-16 → UTF-8 safe convert (convert_utf16_to_utf8_safe) JSON manifest loader (fwupd_json_parser_load_array) FBS schema parser (FlatBufferBuilder::Finish)

heap overflow heap overflow heap overflow stack overflow NULL deref

5.5 (M) 5.5 (M) 7.5 (H) 5.5 (M) 5.5 (M)

fix (CVE-2026-xxxxx) fix fix fix conf

type confusion heap overflow integer overflow uncaught exception uncaught exception

5.5 (M) 5.5 (M) 5.5 (M) 5.5 (M)

fix fix fix fix

5.5 (M)

conf

– –

FP FP

Bugs below additionally required seed generation: 33 34 35 36 37

mongoose★ simdutf fwupd flatbuffers

J J TS J TS

28.6 (2) 28.6 (2) SAF (3) 4.0 (2) 7.8 (1)

(C) Non-C/C++ projects (Java and JavaScript, LLM-only Logic Group generation & ranking) 38 39 40 41

pdfbox

graaljs

42

– – – –

SAF (1) SAF (2) SAF (3) SAF (1)

Inline-image decode array (PDInlineImage.getDecode) CMap parser (CMapParser.increment) PFB font parser (PfbParser.parsePfb) Intl locale validation (IntlUtil.validateAndCanonicalizeLanguageTag)

–

SAF (2)

RegExp char-class parse (RegexLexer.parseCharClassAtomCodePoint)

J TS

32.5 (1) 4.4 (3)

MQTT property iteration (mg_mqtt_next) MGMT TLV parsing (mgmt_tlv_list_load_from_buf)

(D) False-positive reports F1 F2

mongoose bluez

heap overflow heap overflow

Summary: 44 total = 42 from RQ5 deployment (20 fixed, 9 confirmed, 11 awaiting, 2 FP) + 2 from RQ1 audit-fix (both fixed). CVSS v3.1: (C)rit, (H)igh, (M)ed, (L)ow.

SAST = static-analysis backend used to derive the danger score: J = Joern (preferred for C/C++), TS = tree-sitter fallback (template-heavy C++, Java, JavaScript). SAF (rows 28–29) = the encoder LG’s reach is truncated by C++ vtable indirect dispatch that neither Joern nor tree-sitter resolves; rank tied at danger = 0, decided by LLM judgment. SAF (rows 38–42, Java/JS) = the C/C++ unsafe-keyword set in Eq. 5 does not transfer; rank by LLM only. “–” = n/a. All CVSS scores are self-estimated by the authors per CVSS v3.1 [13]. ★ Assigned CVE. Bold = direct maintainer collaboration.