Wake 5: A Python Framework for Solidity Testing, Fuzzing, and Static Analysis

Wake 5: A Python Framework for Solidity Testing, Fuzzing, and Static Analysis Josef Gattermayer

1,2

and Michal Převrátil1

1

arXiv:2611.09307v1 [cs.SE] 6 Nov 2026

2

ack3 (ACK3 LTD), ack3.ai Faculty of Information Technology, Czech Technical University in Prague November 2026

Abstract Smart-contract fuzzers differ by an order of magnitude in throughput and by design in how they construct inputs, yet published comparisons rarely separate the two. We present Wake 5, an open-source Python framework for Solidity testing, fuzzing and static analysis, and use it to locate the binding constraint of stateful smart-contract fuzzing. Wake 5 replaces the JSON-RPC connection to an external node with an in-process EVM (revm) and exposes its static analysis to language-model agents through a Model Context Protocol (MCP) server. On the Daedaluzz maze benchmark (153 reachable assertions, guard constants below 232 ), Wake 5 executes 29,931 transactions per second, on par with Foundry (30,186) and 11.5× faster than Echidna (2,600). In eight-hour campaigns, however, the three tools plateau at medians of 75, 71 and 70 assertions, and their union reaches 93. We show that Wake’s integer generator places 51% of its probability mass below 232 and that, once shallow assertions are exhausted, the generator prior rather than throughput bounds every tool. A harness written by a language-model agent, which derives inputs with a constraint solver, reached all 153 assertions in each of five campaigns executed by Wake, in 33 to 79 minutes. A complementary ablation in an agentic audit of a production Go codebase found that static-analysis tools improved recall for none of three models, raised precision for all three, and increased language-model cost by up to 12.7%. We discuss correlated oracles and non-deterministic completion, and argue that the fuzzer’s role is shifting from search to execution.

Keywords: smart contracts, Solidity, fuzzing, property-based testing, static analysis, languagemodel agents, benchmarking

1

Introduction

Smart contracts on the Ethereum Virtual Machine (EVM) [51] are immutable once deployed and directly custody value, which makes pre-deployment testing unusually consequential. A companion dataset of 135 decentralised-finance (DeFi) incidents from the first half of 2026 found that, among the 68 incidents with identified pre-incident audit history, 94% of reported losses came from attack paths outside every identified pre-incident audit scope [26]. Fuzzing is the dominant dynamic technique for finding such paths before an attacker does: Echidna [27], Foundry [23], Medusa [47], Harvey [54] and ItyFuzz [43] all generate transaction sequences and check assertions or invariants, and Daedaluzz [16, 53] was introduced to compare them on a common footing. Two questions about these tools remain poorly separated in the literature. The first is architectural: does the execution engine matter? Python-based frameworks such as Brownie [20] and Ape [9] drive an external node over JSON-RPC, while Foundry and Echidna execute in-process; a study of eleven contract fuzzers found throughput and seed quality pivotal and a standalone 1

EVM the main throughput lever, but left property-based tools such as Echidna and Foundry outside its scope [52]. The second is methodological: once a fuzzer is fast, what limits the bugs it finds? Coverage-guided greybox fuzzing [12] answers this for C programs, but stateful contracts with arithmetic guards over multi-transaction state behave differently, and a third participant has entered the workflow: language-model agents that write the harness [55, 57]. This paper describes Wake 5 and uses it to measure both questions. Wake is an open-source framework in which tests, fuzz harnesses and static analyses are written in Python against types generated from the Solidity compiler’s output [1]. The framework has been used in commercial security reviews since 2023 and its fuzz tests are credited with critical and high-severity findings in several public audit reports [2, 3, 5]. Wake 5 is the first release whose default execution engine is an EVM embedded in the Python process rather than an external node reached over a socket. Contributions. 1. We describe the design of Wake 5: a version-normalising intermediate representation over the Solidity abstract syntax tree (AST), an in-process EVM (revm [11]) bound to Python through PyO3 [40], a bounded block-history window that makes multi-hour campaigns memory-stable, manually guided fuzzing (MGF) as the harness model, a two-phase stateful shrinker, and a Model Context Protocol (MCP) server [8] that exposes static analysis to agents (Section 3). 2. We measure Wake 5, Echidna 2.3.3 and Foundry 1.7.1 on an L32 maze set generated with Daedaluzz (guard constants below 232 ): 75 eight-hour campaigns plus 45 twenty-minute throughput runs, one physical core each. Wake 5 matches Foundry’s throughput, all three tools end within five assertions of one another, and an exact analysis of Wake’s integer generator shows why the prior rather than the engine sets the hit rate on constant-guarded code (Sections 4–5.3). 3. We report five agent-written harness campaigns that complete the benchmark in 33 to 79 minutes by moving guard satisfaction into a satisfiability modulo theories (SMT) solver and navigation into breadth-first search, and we analyse what changed and what did not (Section 5.4). 4. We report an ablation of static-analysis tools in an agentic security review of a production Go codebase (three models, 134 paired cases: recall unchanged, precision up, cost up) and identify three failure modes of agent-written harnesses observed in practice: correlated oracles, non-deterministic completion and priority inversion (Sections 5.5 and 6).

2

Background and Related Work

Smart-contract fuzzers. Echidna [27] is a Haskell property-based fuzzer that derives an input grammar from the contract’s application binary interface (ABI), maintains a coverage-guided corpus and mines constants from the bytecode into a value dictionary. Foundry’s forge [23] executes Solidity-written fuzz and invariant tests on revm with typed random inputs and value dictionaries. Medusa [47] is a parallel, coverage-guided mutational fuzzer with Solidity properties. Harvey [54] adds input prediction to greybox search; ItyFuzz [43] snapshots EVM state to reach deep states; Smartian [14] steers generation with static and dynamic data-flow information; sFuzz [36] combines AFL-style fuzzing with a multi-objective adaptive strategy for hard-to-cover branches; ContractFuzzer [28] targets known vulnerability oracles. Wake differs in where the harness lives: flows, invariants and the differential model (a Python re-implementation of the expected behaviour that serves as the test oracle) are Python code, and the engine is a dependency of the test process rather than the other way round. 2

Benchmarks. Fuzzle [30] generates C programs from maze puzzles so that reachability requires satisfying chains of constraints. Daedaluzz [16, 53] applies the idea to Solidity: a contract keeps a position on a grid across transactions, moves are guarded by arithmetic conditions over eight uint64 parameters, and the score is the number of distinct assertion violations; its runners cover Echidna, Foundry, Harvey, Hybrid-Echidna and ItyFuzz. Olympia [13] is a second Solidity maze generator on which ItyFuzz, Echidna, Medusa and Foundry were compared at one-hour timeouts, ItyFuzz solving the most mazes. Klees et al. [29] and Schloegel et al. [42] set the methodological bar we follow: multiple seeds, long campaigns, per-time series rather than endpoints, and statistical tests when comparing. Evaluations of language-model agents on contract security have recently appeared [24, 49], as have large-scale evaluations of language-model-written fuzz drivers for C libraries [57]; none that we are aware of puts mainstream EVM fuzzers and an agent-written harness on one benchmark with the harness executed by one of the engines. Property-based testing and shrinking. Wake’s flow model descends from QuickCheck [15] and Hypothesis [33]: generators produce inputs, properties are checked after every step, and failing sequences are reduced. Reduction of stateful sequences is a delta-debugging problem [41, 56] with the added constraint that the Python-side model must be replayed alongside the chain. Static analysis. Slither [21] compiles Solidity to an SSA-form intermediate representation and runs pattern detectors. Wake keeps the compiler’s AST shape and adds cross-version normalisation, control-flow and data-dependency graphs, and a detector and printer API; the same representation serves a language server [35] and, since Wake 5, an MCP server [8]. Agents that write fuzzers. Fuzz4All [55] and TitanFuzz [18] use language models to generate inputs or harness code for compilers and deep-learning libraries, and Zhang et al. [57] measure language-model-written fuzz drivers at scale. For contracts, LLM4Fuzz [44] has a language model prioritise what a fuzzer explores, while PropertyGPT [32] and SmartInv [50] have it write properties and invariants that a prover or fuzzer then checks, which is the oracle question Section 6 returns to. Echidna’s agents preview exposes a running campaign to language models over MCP [19], and slither-mcp [46] exposes Slither’s analyses; Wake’s server exposes static facts only (Section 3.5). The harness in Section 5.4 is closer to a program-synthesis result: the agent produced a solver-backed navigator rather than random inputs, and the fuzzer’s role reduced to executing and replaying candidates.

3

The Design of Wake 5

Wake is organised around one intermediate representation (IR) built from the AST emitted by the Solidity compiler (solc), and several consumers of it (Figure 1). The testing framework generates Python types (pytypes) from the IR and executes tests against an EVM; the static-analysis framework, the language server and the MCP server query the IR directly.

3.1

Intermediate representation

Solidity couples source to compiler version: a file declaring pragma solidity 0.8.4 must be compiled with solc 0.8.4, ASTs differ across versions in node attributes and known defects [45], and one project routinely spans several versions with the same library file in several compilation units. Wake’s IR keeps the AST’s node types and structure but merges all compilation units into one model, resolves references across units, corrects known version-specific defects (missing referenced declarations of enum values before 0.8.2, mislabelled hexadecimal string literals before 0.7.0), and records the compiler version ranges of each source unit. A file compiled in several units 3

Language server (wake lsp) solc (one process per version)

IR (wake.ir) version-normalised AST

Detectors and printers control-flow and data-dependency graphs MCP server (wake mcp)

pytypes generated Python classes

Tests and fuzz harnesses pytest, FuzzTest, flows, invariants

Anvil / Hardhat wake_rs (Rust, PyO3) over JSON-RPC (optional) accounts, Address, ABI, RNG, snapshots

revm in-process EVM

Figure 1: Wake 5 architecture. Blue boxes denote Python components; orange boxes denote Rust components. Transactions use the in-process wake_rs–revm path by default, or the JSON-RPC path to Anvil or Hardhat.

gets one canonical AST into which the other units index their nodes. Analyses therefore traverse one tree per source file without version-specific code. The IR is pickled to disk and recompiled incrementally, which is what lets the language server answer within the editing loop.

3.2

Hybrid Python/Rust execution

A testing framework has three layers: the test specification, the framework core, and the EVM. Foundry writes all three in Solidity and Rust; Hardhat [37] uses TypeScript for the first two; Brownie and Ape use Python for the first two and reach an external node for the third. Solidity as a test language lacks floating-point arithmetic and numerical libraries, which matters for differential models of financial protocols, and it invites circular dependency between implementation and test. Python as a test language has those libraries and is represented in StarCoder’s filtered training corpus at roughly seventy times the volume of Solidity [31]. What Python-based frameworks lacked was throughput: every transaction crossed a socket. Wake 5 embeds revm [11] through a Rust extension module (wake_rs) built with PyO3 [40], the Rust binding layer for the Python interpreter, and the Alloy Ethereum libraries [7]. Transactions execute synchronously in the test process; there is no node, no RPC endpoint and no serialisation of calldata over a socket. The module also implements the Address type, account management including Trezor hardware-wallet signing, ABI encoding, an internal seeded random-number generator (RNG) for chain-level values such as block hashes, and chain state snapshots. Fuzz-test values are still drawn from a seeded Python generator; both generators are captured by snapshots so that shrinking replays deterministically (Section 3.4). The JSON-RPC path is retained for Anvil and Hardhat; Ganache support was dropped. Two engineering decisions were necessary for eight-hour campaigns. First, Wake 4 retained every block and transaction for the whole run; under revm a twenty-minute campaign retained roughly 1.6 GB per million transactions and exhausted the container’s memory. Wake 5 keeps only the newest block_history blocks (default 256, with a small hysteresis margin before pruning) and their transactions. Access outside the window raises HistoryPrunedError; transaction objects that the test keeps a reference to remain valid, while call traces and console logs, which re-execute the transaction, require the block to be retained. Second, reverted-error objects held reference cycles through the chain; breaking them was required for the memory profile to stay flat.

4

3.3

Manually guided fuzzing

Wake’s harness model is manually guided fuzzing (MGF). A harness (a fuzz test in Wake’s vocabulary) is a Python class deriving from FuzzTest. Its methods decorated with @flow are atomic steps, each usually one transaction, chosen at random according to weights and preconditions; methods decorated with @invariant are checked after every flow or every k flows. The invariants and the differential model are the harness’s test oracles, the parts that decide whether a sequence passes; we write oracle when either is meant. A sequence is one independent run of a fixed number of flows from a fresh deployment; chains are reset between sequences, and pre_sequence deploys and initialises the target before each one. Flow arguments are generated from Python type hints (uint64, Address, pytypes structs and enums), and the harness keeps a differential model of the contract in Python state and compares it with on-chain state in its invariants. Listing 1 shows the shape on an illustrative bank contract. The human supplies the transaction vocabulary, value strategy, differential model and assertions; Wake randomises within those choices. Listing 1: Manually guided fuzz test for a bank contract. class BankFuzz(FuzzTest): def pre_sequence(self): self.user = random_account() self.bank = Bank.deploy() self.expected = 0 @flow(weight=3) def deposit(self): amount = random_int(1, 10**18) self.bank.deposit(from_=self.user, value=amount) self.expected += amount @flow(precondition=lambda s: s.expected > 0) def withdraw(self): amount = random_int(1, self.expected) self.bank.withdraw(amount, from_=self.user) self.expected -= amount @invariant() def balance_matches(self): assert self.bank.balanceOf(self.user) == self.expected

Three features distinguish MGF from black-box or property-only fuzzing. (i) Revert expectations are explicit. A flow wraps a call in a may_revert or must_revert context manager that names the errors it tolerates; any other revert (a transaction that aborts and rolls back its effects) fails the test. Property-based fuzzers typically treat a reverted transaction as an uninteresting input, whereas an unexpected revert is how denial-of-service and stuck-state bugs surface. (ii) The oracle is a model, not only an invariant. Differential models in Python have found rounding and accounting errors that no single-transaction invariant expresses [2]. (iii) The method is iterative. A harness grows over many develop–run–debug cycles; bugs are typically found while the harness is being built, before any long campaign runs. Section 5.4 treats the agent as a participant in exactly this loop. Flow arguments of integer type are drawn from a prior that is not uniform. With probability 0.05 the generator returns one of five edge values {0, 1, 2n − 1, 2n−1 , 2n−1 − 1} for an unsigned n-bit type (signed types add two negative edges); otherwise it draws a bit length b uniformly from {1, . . . , n} and then a uniform value in [0, 2b ), that is, a value of at most b bits. Section 5.3 quantifies the consequences.

5

3.4

Stateful shrinking

When a sequence fails, Wake shrinks it, that is, it searches for a shorter sequence that still fails, in two phases. Phase 1 groups flows by kind, orders kinds by call count, and attempts to remove every call of a kind at once, re-running the sequence after each attempt. Phase 2 walks the remaining sequence from the start and removes flows one at a time, restoring a flow whenever the failure disappears. Both phases must keep the Python-side model consistent with the chain, so Wake snapshots the chain state and deep-copies the harness object at checkpoints, and restarts replay from the nearest checkpoint rather than from the beginning. A shortcut rule replaces the failing suffix with an earlier occurrence of the same error when one is found during replay. In Wake 5 the shrinker was reworked for the revm engine: each checkpoint captures the Python random state and the engine’s internal RNG state together with the chain snapshot, so that replay reproduces the same generated values, and a performance gap between chain snapshots and Python state copies was closed. Command-line switches select whether the reduced sequence must fail in the same flow with the same exception or merely fail. Unlike Echidna, Wake does not simplify argument values; unlike Foundry, it does not stop at transaction removal only.

3.5

Static analysis, language server and MCP server

The static-analysis framework provides a visitor API over the IR, per-function control-flow graphs, data-dependency information, and 26 built-in detectors and 19 printers (5.0.0rc3). Detectors range from local patterns (unchecked return values, tx.origin authentication, unsafe delegatecall) through known compiler defects to graph-based queries such as reentrancy, detected as state written on a control-flow path after an external call. Findings can be suppressed inline, exported as JSON and filtered against a database of known deployed contracts, and the language server surfaces them, together with navigation, hover and rename, through the Language Server Protocol [35]. Wake 5 adds an MCP server (wake mcp) that exposes the same static facts to language-model agents [8]: contracts and functions, definitions and references, C3 linearisation (the methodresolution order across inheritance), storage layout, state changes performed by a function, lookups by four-byte function selector, and external-call sites. Every tool answers from the compiled IR as compact plain text; on one representative response the plain-text form used 83 tokens against 199 for JSON (o200k_base tokenizer). Section 5.5 measures whether tools of this kind change audit outcomes.

4

Evaluation Methodology

We ask five questions. RQ1 (throughput): how many transactions per second does Wake 5 execute compared with Echidna and Foundry on the same contracts and hardware? RQ2 (coverage): how many distinct reachable assertions does each tool violate in eight hours, and how do the curves evolve? RQ3 (prior): how much of the input space does Wake’s integer prior concentrate on, and what does that imply for constant-guarded assertions? RQ4 (agent): what does an agent-written harness achieve on the same benchmark, and where did the work move? RQ5 (static-analysis tools): do static-analysis tools that return deterministic code-structure facts change recall, precision or cost in an agentic security review?

4.1

Benchmark

Daedaluzz [16] generates Solidity contracts that hold a position (x, y) on a 7×7 grid across transactions. Four external functions (moveNorth, moveSouth, moveEast, moveWest) each take eight uint64 parameters p0 , . . . , p7 , move the position, and evaluate a cell-specific chain of arithmetic guards; some cells contain an assert(false) reachable only when the guards hold, 6

Table 1: Tool versions and benchmark configurations Tool

Version

Notes

Echidna

2.3.3

Foundry (forge)

1.7.1 (commit 4072e487)

Wake Agent-written harness

5.0.0rc2 (commit d33810e) five campaigns

exploration mode, coverage-guided corpus, sequence length 100; patched to timestamp the first coverage of each assertion line invariant tests, one per assertion; solc pre-fetched into the image revm engine; first build with bounded block history Claude Opus 4.8 (xhigh); GPT-5.5 (high) researcher configured but unavailable in campaign 5; Python harness executed by Wake

some cells are walls, and some guards are unsatisfiable. Arithmetic is unchecked, so products and sums wrap modulo 264 . The score of a campaign is the number of distinct assertions violated. Listing 2 shows one cell from maze 0. Reaching its assertion requires five transactions (four transit moves and the triggering move), with inputs satisfying an inequality and an exact relation between two 64-bit values. Listing 2: Guard conditions and assertion at cell (0, 5) of L32 maze 0. if (x == 0 && y == 5) { if (p6 != uint64(uint64(2398633357) + p1)) { if (p4 == uint64(p5 * p5)) { emit AssertionFailed("5"); assert(false); } } return 5; }

The published Daedaluzz mazes draw guard constants from [0, 64], which favours any generator biased towards small values. We therefore generated an L32 set: five mazes from fixed seeds 0–4 with the generator’s constant bound raised to 232 − 1 while inputs remain uint64. Solving every guard chain with Z3 [17] (each parameter a 64-bit bit-vector, wrap-around preserved) and replaying each satisfying model against a fresh deployment established that exactly 153 assertions are reachable: 36 + 24 + 32 + 27 + 34 across the five mazes. That count is the denominator throughout. All contracts were compiled with solc 0.8.19, optimizer enabled with 200 runs.

4.2

Tools and harnesses

Table 1 lists the tools. Each tool used its own idiomatic harness: Echidna in exploration mode on the stock contract, an assertion counting as violated when its emit AssertionFailed line is first covered (the event itself is discarded by the revert), which is how the benchmark’s own runner scores it; Foundry with the generator’s .foundry.sol invariant variant, one invariant function per assertion; Wake with a FuzzTest of four flows, one per move function, each taking eight uint64 flow arguments and wrapped in may_revert(), and an invariant with period 100 that polls the contract’s found mapping and prints the first-seen time of every newly violated assertion. The Wake harness is deliberately unguided: no value strategy, no model, no preconditions. It measures the engine and the default generator, not the MGF method. Neither Echidna’s symbolic-execution mode, off by default since 2.3.0 [19], nor Foundry’s corpus-guided invariant mode, opt-in since 1.3.0 [22], was enabled; Foundry ran with the benchmark’s dictionary settings (storage and push-byte dictionaries, invariant depth 100, 500 runs per invocation, reverts tolerated). Both tools therefore ran as random generators with value dictionaries, and the comparison is between random-input fuzzers, not between solver-assisted ones.

7

4.3

Campaigns and measurement

Coverage campaigns (RQ2): 5 seeds × 5 mazes × 3 tools = 75 campaigns of eight hours. Each campaign is a Docker container pinned to one physical core of a 16-core AMD Ryzen AI MAX+ 395 workstation, with simultaneous multithreading siblings left idle; seeds 0–2 ran in August 2026 and seeds 3–4 in October 2026 from the same pinned images. A three-seed cut of these campaigns (medians 73, 72 and 71; union 90) was reported in a research note [39]; seeds 3 and 4 were added for this paper and every number here is recomputed over five seeds. Per-run series record the cumulative number of distinct violated assertions every ten seconds. The reported endpoint of a tool is the median across seeds of the per-seed sum over the five mazes. Throughput runs (RQ1): a separate set of 3 seeds × 5 mazes per tool with a twenty-minute target, 15 runs per tool, one core per run. Transaction counts come from each tool’s own reporting: Wake counts flows in the harness, Echidna prints its total call count, and Foundry’s per-invariant call counts are summed over invocations. Active time excludes compilation and setup. We report medians over the 15 runs. Throughput is machine-specific; the ratios, not the absolute rates, are the result. Agent campaigns (RQ4): five end-to-end runs of the agent on the same five mazes, each one process covering harness construction, solving and execution. For the August 2026 campaign the milestones are the wall-clock times at which the harness’s replayed reproductions reached 150 and 153 distinct assertions; for the three campaigns of 5 October 2026 they are the first-hit timestamps of the recorded findings, one of which also records from-scratch reproduction. For the campaign of 26 October 2026, run under a fixed protocol after the first draft of this paper (byte-identical L32 fixtures, Wake at the measured commit, finding records, harness and per-conversation token accounting retained), they are the earliest first-visit timestamp per assertion across its fuzz passes. Time is measured from the agent’s start, so it includes harness construction and solver work. Every campaign used Claude Opus 4.8 at xhigh, with GPT-5.5 at high configured as a research sub-agent in the same runner. In the fifth campaign, both retained research-sub-agent calls failed with a model-availability error; the main agent completed the harness without a successful researcher response. The agent has no stopping rule tied to the benchmark: three campaigns ended on their own after further work on a target with nothing left to find, 59, 123 and 170 minutes after their last assertion, and two were stopped by the operator (Table 4). Wall time and language-model cost are therefore reported per campaign as end-to-end totals that include that overrun; the completion minute is the comparable quantity. The finding records from 5 October were inspected at the time and not retained: the agent’s working directory is cleared between campaigns so that no run can reuse an earlier run’s findings, and the session step logs are the surviving record. Five runs show the spread of completion times without estimating it. Statistics: differences between tools in the per-seed eight-hour endpoints are tested with the exact two-sided Mann–Whitney U test and reported with the Vargha–Delaney Â12 effect size [48], following the guidelines of Arcuri and Briand [10]; with five seeds per tool the smallest attainable p is 0.008. Paired comparisons of binary outcomes in the ablation (RQ5, Section 5.5) use exact McNemar tests [34]. p For the paired change d = (b − c)/n, we report normal-approximation 95% intervals d ± 1.96 [(b + c)/n − d2 ]/n, where b and c count cases correct only with tools on and only with tools off, respectively; n is 55 for recall and 79 for correct rejection.

5

Results

5.1

RQ1: Throughput

Table 2 and Figure 2 give the twenty-minute throughput runs. Wake 5 executed a median 29,931 transactions per second, Foundry 30,186 and Echidna 2,600. Two tools that embed revm run within one percent of each other; the Haskell-hosted Echidna is 11.5–11.6× slower. This is the 8

Table 2: Transaction throughput and assertion discovery on L32 mazes Tool Foundry Wake 5 Echidna

Transactions

Tx/s

vs. Echidna

Distinct assertions

Assertions / 106 tx

36,655,640 35,895,579 3,119,791

30,186 29,931 2,600

11.6× 11.5× 1.0×

12 13 11

0.32 0.37 3.36

Medians over 15 runs per tool, with a twenty-minute target and one core per run. Asserts counts distinct assertions violated; Asserts/106 tx uses these short runs, not the eight-hour campaigns.

(b) Assertions per million transactions

(a) Throughput 30, 000

30,186

29,931

assertions / 106 tx

transactions / s

4

20, 000 10, 000 2,600

0

3.36

3 2 1 0.32

0.37

Foundry

Wake 5

0 Foundry

Wake 5

Echidna

Echidna

Figure 2: Transaction throughput and assertion discovery on the L32 mazes. Panels show medians over 15 runs per tool with a twenty-minute target: (a) transactions per second; (b) distinct assertions violated per million transactions.

architectural result: moving the EVM into the test process removes the gap between Python-hosted and Rust-hosted fuzzing. Per transaction the ranking inverts. Echidna violated 3.36 distinct assertions per million transactions against 0.37 for Wake and 0.32 for Foundry. Coverage guidance makes each transaction more informative; the revm-based tools compensate with volume. In twenty minutes the outcome is nearly the same: the median run violated 11, 13 and 12 assertions on its maze.

5.2

RQ2: Eight-hour coverage

Figure 3 shows the median curves and Table 3 the per-seed endpoints. After eight hours the medians are Wake 75, Foundry 71 and Echidna 70 of 153 reachable assertions; the fifteen per-seed sums range from 67 to 75. The ordering is consistent across seeds: Wake exceeds Foundry in four of five seeds and Echidna in all five (Mann–Whitney p = 0.040, Â12 = 0.90, and p = 0.016, Â12 = 0.98), whereas Foundry and Echidna are not separated (p = 0.25, Â12 = 0.76). The effect is nonetheless small in absolute terms, four assertions of 153. Most of each total arrives within the first hour (Wake 68, Foundry 61, Echidna 61 at one hour), after which the curves add a tail of seven to ten assertions over seven hours. The findings overlap heavily: the union of assertions found by any tool in any campaign is 93, leaving 60 that no campaign of the three tools reached. An 11.6× spread in throughput therefore bought at most five assertions of eight-hour coverage, and the two tools with equal throughput differ by four. Once the shallow assertions are exhausted, every tool waits on the same event: a generated input vector that satisfies a remaining guard chain. That probability is set by the generator, not the engine.

9

distinct assertion violations (sum over 5 mazes)

Benchmark maximum: 153 reachable assertions

153 120 90 60 30 0

0

1

2

3 4 5 campaign time (hours)

Wake 5 (median, 5 seeds) Echidna (median, 5 seeds) Four further campaigns, completion

6

7

8

Foundry (median, 5 seeds) Agent-written harness, August campaign

Figure 3: Distinct assertions violated on the five L32 mazes. Solid lines show medians over five seeds; shaded bands show the seed range. The dashed magenta step shows the August agent campaign; triangles mark completion of the four October campaigns. The benchmark maximum is 153 reachable assertions across all five mazes. Agent times include harness construction and solving. Table 3: Assertion coverage across the five L32 mazes Seed Tool

0

1

2

3

4 Median

Wake 5 Foundry Echidna

73 75 72 75 75 74 72 71 70 71 72 71 70 69 67

75 71 70

Agent-written harness 5×153, 33–79 min

153

Fuzzer entries are eight-hour totals per seed, out of 153 reachable assertions. All five agent campaigns reached 153 in 33–79 minutes.

5.3

RQ3: The generator prior

Let X be an unsigned 64-bit flow argument drawn by Wake’s generator (Section 3.3) and let S = {0, 1, 264 − 1, 263 , 263 − 1}. With κ(x) = max(1, ⌈log2 (x + 1)⌉) the bit length of x, Pr[X = x] = 0.01 · 1[x ∈ S] +

64 0.95 X 2−b , 64 b=κ(x)

Pr[X < 2k ] = 0.01 S ∩ [0, 2k ) +

64  X 0.95  k+ 2k−b . 64 b=k+1

(1)

(2)

Thirty-two of the 64 bit-length choices produce values below 232 , and longer draws can land there too, so Pr[X < 232 ] = 0.50984: half of all draws fall in a range that uniform sampling reaches with probability 2−32 ≈ 2.3 × 10−10 (Figure 4). For an exact 32-bit constant c, Pr[X = c] = 6.9 × 10−12 against 5.4 × 10−20 under uniform sampling, a factor of 1.3 × 108 . The prior is general Wake behaviour for type-hinted flow arguments, not benchmark code. 10

L32 constant bound

Pr[X < 2k ]

10−2

10−8

10−14

Wake 5 generator, Eq. (2) uniform uint64 10−20

0

8

16

24

32

40

48

56

64

k (bits) Figure 4: Cumulative probability that an unsigned 64-bit value is below 2k , for Wake and uniform sampling. Changes in slope at k = 1, 63 and 64 reflect the generator’s edge values.

The prior explains why Wake’s unguided harness matches and, over five seeds, slightly exceeds Echidna’s constant dictionary on L32, and it predicts where it will not: guards such as p0 ̸= 46 are easier under uniform draws, and guards that require overflow (p4 > p4 · p6 ) are never satisfied by small values. A supplementary eight-hour run on an L64 maze set with full-width constants, listed in Appendix A, is consistent with this prediction. The benchmark fork ships an experiment harness that isolates the prior by re-implementing it as an input arm alongside uniform, small-value and dictionary arms; those runs are ongoing and not reported here.

5.4

RQ4: The agent-written harness

A language-model agent was given the five L32 mazes and Wake and asked to find the assertions. The harness it produced over several develop–run–debug iterations is a Python FuzzTest, executed by Wake 5 like any other, whose input strategy is no longer random: 1. Navigation from parsed source. The agent parsed each maze’s step function into a per-cell table of guard conditions, walls and assertion cells, and navigates by breadth-first search using transit inputs known to satisfy no assertion guard. 2. Guard inputs from an SMT solver. For each assertion cell the conjunction of guards is handed to Z3 with p0 , . . . , p7 as 64-bit bit-vectors, preserving unchecked wrap-around. Queries that return unknown are retried through a simplify, bit-blast, sat tactic, which rewrites the query to propositional satisfiability. Every satisfying model is independently re-evaluated in Python against the parsed conditions before it is retained. 3. Replay as the oracle. Each candidate is executed against a fresh deployment; only the replayed Panic(0x01), the EVM panic code of a failed assert, counts. The solver result is a hypothesis, the execution is the evidence. The August campaign reached 150 of 153 assertions at minute 37 and all 153 at minute 79 (Figure 3). Its first 37 minutes were spent parsing and solving; the remaining assertions were those whose queries needed the bit-blasting fallback. Three further campaigns on 5 October 2026, on the same fixtures, also reached all 153 and completed at minutes 33, 39 and 79. Two of them emitted every finding within a second of each other once the harness ran, the solver having finished before execution started; the third repeated the August profile in two stages, 79 assertions by minute 34 and 146 by minute 56. A fifth campaign on 26 October 2026, run under a fixed protocol with its finding records, harness and token accounting retained, reached all 153 at 11

Table 4: Completion time and cost of agent-written harness campaigns

1 2 3 4 5

Date

153 at Ended

Aug 2026 5 Oct 2026 5 Oct 2026 5 Oct 2026 26 Oct 2026

79 min 33 min 79 min 39 min 34 min

on its own operator operator on its own on its own

Wall In, cached 2 h 18 min 33 min 2 h 20 min 3 h 29 min 2 h 38 min

39.09 5.40 25.47 51.13 44.63

In Out 0.46 0.15 0.80 0.95 0.56

0.28 0.10 0.22 0.41 0.33

Cost $33.41 $7.28 $26.89 $45.94 $36.01

153 at: time to the last reachable assertion. Wall: total elapsed time, including work after the last assertion. Token counts are millions of Claude Opus tokens; In excludes cached input. Cost uses list prices and includes research-sub-agent usage where available; campaign 5 records failed researcher calls. Counts for operator-stopped campaigns end at the stop. Campaign 5 used a fixed protocol; its counts include developer and reviewer conversations, checked against their transcripts.

minute 34: eight assertions at minute 30.6 in a first fuzz pass that aborted after eleven seconds on a harness error and sent the agent into debugging, and the remaining 145 at minute 34.2 in the third pass; 151 came from replaying solver-computed solutions and two from the harness’s random walk. Across the five campaigns completion took 33 to 79 minutes (median 39), and the spread lies in harness construction and solving, not in execution. Completion is not the end of a campaign: with every assertion found, the agent kept proposing further work on a target that had nothing left to find. Three campaigns ended on their own 59, 123 and 170 minutes after their last assertion and two were stopped by the operator, so wall time ranged from 33 minutes to 3.5 hours and language-model cost from $7 to $46 (Table 4), against a fixed eight hours per fuzzer campaign. A stopping rule is part of what the harness-construction workflow still lacks (Section 6). Random generation was replaced exactly where it is weakest, constructing inputs for arithmetic guards, and random walks were replaced by search over a parsed map; the runtime still deployed, transacted, checked and replayed at scale. The results are five campaigns of a different workflow, not a second engine measurement (Section 7).

5.5

RQ5: Static-analysis tools in an agentic review

To test whether deterministic code-structure facts help an agent, we ran an ablation in a different setting: a security review of an anonymised production proof-of-stake consensus client written in Go, with a Go-analysis MCP server that mirrors the surface of Wake’s server (workspace diagnostics, symbols, definitions, references, implementations, imports, direct callers and callees; eleven tools; no taint, reachability or vulnerability query) [38]. Three GPT-5.6 models (Sol, Terra, Luna) at the vendor’s highest reasoning-effort setting (xhigh) each processed 168 cases, 128 questions naming a potential issue and 40 open-ended prompts, once with the tools available (tools on) and once with the server, tool declarations and prompt references removed (tools off ). A supporting Luna instance at the high setting answered code-research sub-questions in every arm. The main comparison uses the 134 cases for which every model returned a usable verdict in both arms: 55 known bugs and 79 cases with no real bug; all 34 exclusions are non-bug cases. We report recall (the share of the 55 known bugs found), correct rejection (the share of the 79 non-bug cases dismissed, that is, false positives avoided), precision (the share of reported issues that were real), and the cost of the language-model calls (LLM cost). Recall did not improve for any model (Table 5); Sol and Terra found one and two more known bugs, respectively, with the tools off. Correct rejection of non-bug cases rose for all three, and with it precision, by 1.3 to 6.1 percentage points. No paired comparison reaches p < 0.05; the smallest p is 0.219 for Luna’s rejections. Tool use varied widely (Table 6): Luna called the tools in 96.4% of runs, Terra in 32.1%, so Terra’s ablation is close to comparing an arm against itself. LLM cost rose with tool use, by 12.7% for Luna, 8.1% for Sol and 0.0% for Terra, whose runs rarely called the tools; the static-analysis server ran locally and had no fee. A proof-of-concept arm (ten pre-registered bugs, the same three models, executable Go demonstrations) and an open-ended 12

Table 5: Static-analysis tool ablation on 134 paired cases Model

Metric

Tools on

Tools off

On − off

p

95% interval

Sol Terra Luna

recall recall recall

38/55 (69.1%) 32/55 (58.2%) 32/55 (58.2%)

39/55 (70.9%) 34/55 (61.8%) 32/55 (58.2%)

−1.8 pp −3.6 pp 0.0 pp

1.000 0.727 1.000

[−9.8, +6.1] [−13.7, +6.4] [−8.7, +8.7]

Sol Terra Luna

correct rejection correct rejection correct rejection

70/79 (88.6%) 70/79 (88.6%) 67/79 (84.8%)

69/79 (87.3%) 66/79 (83.5%) 63/79 (79.7%)

+1.3 pp +5.1 pp +5.1 pp

1.000 0.289 0.219

[−4.3, +6.8] [−1.9, +12.0] [−0.9, +11.0]

Sol Terra Luna

precision precision precision

38/47 (80.9%) 32/41 (78.0%) 32/44 (72.7%)

39/49 (79.6%) 34/47 (72.3%) 32/48 (66.7%)

+1.3 pp +5.7 pp +6.1 pp

– – –

– – –

Recall uses 55 known bugs; correct rejection uses 79 non-bug cases. Changes and normal-approximation 95% intervals are for tools on minus tools off, in percentage points (pp). The p-values are from exact McNemar tests. Precision is descriptive: its denominator is the number of confirmed verdicts, which differs between arms.

Table 6: Static-analysis tool use and change in language-model cost Model

Runs using tools

Calls per run

Median calls

LLM cost change

Sol Terra Luna

128/168 (76.2%) 54/168 (32.1%) 162/168 (96.4%)

8.01 1.25 10.11

7.0 0.0 9.5

+8.1% +0.0% +12.7%

Tool-use statistics cover 168 tools-on runs per model. Calls / run is the mean; Median is the median number of calls. Cost changes compare tools on with tools off, averaged over the 134 paired cases in Table 5; they include supporting Luna calls. Prices verified 2026-08-04.

arm (a smaller open-weight model prompted only to perform a deep security analysis, ten runs per setting, blind-judged against 45 known bugs) were flat as well: 11/30 versus 12/30 failing demonstrations and 6 versus 5 bugs recovered, both p = 1.000. The reading is that code-structure facts helped the models verify (reject false positives) but not discover. The tools had no query for the questions an audit turns on: whether untrusted input reaches a release path, which state changes precede an external call, whether an attacker-controlled implementation can satisfy an interface. A follow-up should test security-specific deterministic outputs, taint paths, attacker reachability, state-transition summaries and candidate invariants, against cases where each has a clear route to the answer.

5.6

A datapoint from a real audit

One anonymised engagement provides a non-synthetic reference for harnesses written by agents (Table 7). The full human report contained 7 critical, 3 high and 4 medium findings. The manually guided fuzzing portion, 17 person-days of auditor time, accounted for 6 critical, 0 high and 1 medium of them. Two independent agent runs of the same fuzzing task, each writing and iterating a Wake harness, reported 4/1/1 in 84 elapsed hours and 5/3/2 in 28 elapsed hours. Effort and elapsed time are different units, so the rows are context; the controlled observation is the divergence between the two agent runs, where the run that took three times longer reported fewer findings in every severity.

6

Discussion

Throughput is a prerequisite, not the constraint. Wake 5 closes the throughput gap between a Python-hosted framework and Rust-native fuzzers (RQ1), and the eight-hour curves

13

Table 7: Findings and time in one anonymised audit Execution Human report (all methods) Human MGF Agent run 1 Agent run 2

Critical

High

Medium

7 6 4 5

3 0 1 3

4 1 1 2

Time not recorded 17 person-days 84 h elapsed 28 h elapsed

Human MGF is the manually guided fuzzing subset of the full human report. Agent rows are independent runs of the same fuzzing task. Person-days measure human effort; elapsed hours measure agent runtime and are not directly comparable.

show that the gap was not what separated tools on this benchmark (RQ2). On guard chains built from equalities against constants, every random generator waits on the same low-probability event, and the generator prior (RQ3) matters more than an order of magnitude of throughput. Speed still matters in two places: it shortens the develop–run–debug loop of harness construction, and it makes replay-based verification of thousands of solver candidates cheap. Both are on the path the agent took. The fuzzer’s role is moving from search to execution. The agent-written harness (RQ4) did not replace the fuzzer; it replaced the part of the workflow that decides what the fuzzer should try. That is the MGF division of labour of Section 3.3 with the manual part automated, and it is why we consider a Python harness language and an in-process engine the right substrate for agents: the harness is ordinary code the agent can read, run and revise, and each revision executes at native speed. What an agent-written harness inherits. Three failure modes appeared in the surrounding practice and are not measured by the benchmark. Correlated oracles: a differential model derived from the implementation reproduces the implementation’s defects. If a contract credits msg.value * 2 by mistake and the agent translates that rule into the model, the equality assertion passes precisely where the contract is wrong; the oracle fails silently. An independent invariant, such as credited balances never exceeding assets held, exposes what translation cannot, and whether an agent reliably derives such invariants rather than restating source behaviour is open. Nondeterministic completion: independent runs of the same task produce different harnesses with different differential-model depth and different assertions (Section 5.6); a harness that builds, runs and passes carries no evidence that its security model is adequate, so a production workflow needs an explicit stopping rule and a quality bar separate from task completion. Priority inversion: given a high-value, high-effort task (replace a mock with an integration test against forked live-chain state) and a low-value, self-contained one (implement a fee setter), agents were observed to start with the latter and defer the former behind a plausible rationale; priority adherence has to be evaluated separately from polished status reporting. Deterministic helpers for non-deterministic reasoning. The ablation (RQ5) is a caution against inferring value from tool use: the models called the tools, the calls cost more, and recall did not move. Deterministic facts are a stable input to a variable reasoning process, not a guarantee of a better outcome. Wake’s MCP server is designed on that reading: it answers structural questions compactly and leaves the security question to whoever, or whatever, asks it.

14

7

Threats to Validity

Construct validity. Distinct assertion violations are Daedaluzz’s own score; they measure reachability under a fuzzer, not bug severity or the cost of a miss. Transactions per second come from each tool’s own counting and from active time (Section 4.3), so throughput transfers as ratios only. In the ablation, precision is descriptive because its denominator differs by arm, and recall is measured against a fixed set of known bugs rather than against all bugs present. Internal validity. Each tool ran its own idiomatic harness rather than a shared one, so harness shape is not fully separated from engine; the Wake harness was deliberately left unguided to keep the MGF method out of the measurement. The measurements used a release-candidate commit of Wake. The agent-written harness has five campaigns: completion ranged from 33 to 79 minutes, which shows the spread without estimating it, and three of the five record first-hit times during the fuzz run rather than replayed reproductions, with only their step logs retained, while the fifth retains its findings, harness and token accounting; two campaigns were stopped by the operator and the three that ended on their own did so long after completion, so the campaigns do not establish a stopping time, and the token counts of the stopped campaigns are as logged at the stop (Section 4.3). The comparison is of a workflow, not an engine. Daedaluzz is public, so the agent’s training data may include the benchmark family, although the L32 constants are freshly generated. The ablation has one run per case (four repeated cases changed one verdict). External validity. Daedaluzz mazes are synthetic: unchecked arithmetic, capped constants and guard chains over eight parameters reward exactly the constraints an SMT solver dispatches, and no L32 campaign measured a real contract. Three fuzzers were run; Medusa, which Daedaluzz never wired in, and ItyFuzz, which led the Olympia comparison [13] but is no longer actively developed, were not, and neither was any solver-assisted mode (Echidna’s symbolic mode, HybridEchidna, ItyFuzz’s concolic executor), so the agent-written harness’s advantage is not separated from that of a constraint solver as such. The generator-prior result (RQ3) is a statement about random input generation, not about fuzzers that solve constraints during the campaign. The ablation covers one Go codebase, one tool surface and three models from one vendor; it says nothing about security-specific static analysis or about Wake’s own MCP server on Solidity. The real-audit rows mix effort with elapsed time and cover one engagement. Conclusion validity. Five seeds per tool are too few for narrow confidence intervals; we report the per-seed spread (67–75 across all tools), exact rank tests and effect sizes (Section 5.2), and treat a difference of a few assertions as small even where it is consistent across seeds. No paired comparison in the ablation reaches p < 0.05, and with 55 known bugs the study is not powered to detect small recall effects; the conclusion that recall did not improve is a statement about this sample.

8

Conclusion

Wake 5 brings an in-process EVM to a Python-native testing framework and, on the Daedaluzz benchmark, matches the throughput of a Rust-native fuzzer. The same benchmark shows that on constant-guarded stateful contracts the binding constraint of random-input fuzzing has moved from throughput to input construction: an 11.6× throughput spread bought at most five assertions in eight hours, while a harness that derives inputs with a solver completed, in 33 to 79 minutes across five campaigns, a suite that eight-hour campaigns left more than half unfinished. A static-analysis tool ablation in an agentic review points the same way: code-structure facts helped agents verify, not discover. We read these results as a shift in the fuzzer’s job description, from searching to

15

executing what a human or an agent has decided to try, and we have designed Wake 5, its MGF harness model and its MCP server for that role. What remains open is a larger sample of agent runs under one measurement protocol, stopping rules for harness construction, and whether agents can derive oracles independent of the implementation under test.

Data and Software Availability Wake is available under the ISC licence at https://github.com/ack3-ai/wake. The measurements use commit d33810e (5.0.0rc2); the framework description covers commit b46f5b5 (5.0.0rc3). Benchmark code is available at https://github.com/ack3-ai/daedaluzz-benchmark. Supplementary material provides the measured framework source, benchmark data, retained harnesses and analysis scripts. Underlying audit engagement records are withheld under client confidentiality; aggregate statistics are included. Finding records for three agent campaigns were not retained (Section 4).

Acknowledgements We thank Arman Bašović for the analysis and repair of the shrinking implementation on the revm engine, ConsenSys Diligence for Daedaluzz, and the organisers of Web3 Security Summit Belgrade and ETHBelgrade 2026, where the measurements in Sections 5 and 5.5 were first presented. AI assistance. Anthropic Claude Code and OpenAI Codex assisted the preparation and revision of manuscript text throughout this paper. Codex also assisted citation checking, LaTeX formatting, figure-caption revision, edits to the plotting code for Fig. 3, and preparation of the replicationpackage code and documentation. The agent-generated fuzzing harnesses are experimental objects described in Sections 4 and 5.4; they are distinct from assistance with preparing this manuscript. The authors remain responsible for the accuracy, originality and integrity of the paper.

References [1] ack3. Wake: Python-based Solidity development, testing and fuzzing framework. https: //github.com/ack3-ai/wake, release candidate 5.0.0rc3, 2026. Documentation: https: //ack3.ai/wake/docs/5.0.0rc3/. [2] Ackee Blockchain. IPOR protocol security audit report. https://github.com/Ackee-Blo ckchain/public-audit-reports/blob/master/2023/ackee-blockchain-ipor-protoco l-report.pdf, 2023. [3] Ackee Blockchain. PWN protocol v1.3 security audit report. https://github.com/PWNDAO/ pwn_audits/blob/main/protocol/pwn-v1.3-ackee.pdf, 2023. [4] Ackee Blockchain. Python testing frameworks benchmark. https://github.com/Ackee-B lockchain/python-testing-frameworks-benchmark, 2023. [5] Ackee Blockchain. Lido community staking module v2 audit report. https://github.com/l idofinance/audits/blob/main/Ackee%20Blockchain%20Community%20Staking%20Modul e%20v2%20Audit%20Report%2009-2025.pdf, 2025. [6] Hayden Adams, Noah Zinsmeister, Moody Salem, River Keefer, and Dan Robinson. Uniswap v3 core. Technical report, Uniswap, 2021.

16

[7] Alloy contributors. Alloy: High-performance Ethereum types and providers in Rust. https: //github.com/alloy-rs/alloy, 2026. [8] Anthropic. Model context protocol. https://modelcontextprotocol.io/, 2024. [9] ApeWorX. Ape framework. https://apeworx.io/, 2026. [10] Andrea Arcuri and Lionel Briand. A hitchhiker’s guide to statistical tests for assessing randomized algorithms in software engineering. Software Testing, Verification and Reliability, 24(3):219–250, 2014. doi: 10.1002/stvr.1486. [11] bluealloy contributors. revm: Rust Ethereum virtual machine. https://github.com/bluea lloy/revm, 2026. Version 41.0 used in this work. [12] Marcel Böhme, Van-Thuan Pham, and Abhik Roychoudhury. Coverage-based greybox fuzzing as Markov chain. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security (CCS), pages 1032–1043. ACM, 2016. doi: 10.1145/2976749.2978 428. [13] Jana Chadt, Christoph Hochrainer, Valentin Wüstholz, and Maria Christakis. Olympia: Fuzzer benchmarking for Solidity. In Proceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering (ASE), pages 2362–2365. ACM, 2024. doi: 10.1145/3691620.3695352. [14] Jaeseung Choi, Doyeon Kim, Soomin Kim, Gustavo Grieco, Alex Groce, and Sang Kil Cha. Smartian: Enhancing smart contract fuzzing with static and dynamic data-flow analyses. In Proceedings of the 36th IEEE/ACM International Conference on Automated Software Engineering (ASE), pages 227–239. IEEE, 2021. doi: 10.1109/ASE51524.2021.9678888. [15] Koen Claessen and John Hughes. QuickCheck: A lightweight tool for random testing of Haskell programs. In Proceedings of the 5th ACM SIGPLAN International Conference on Functional Programming (ICFP), pages 268–279. ACM, 2000. doi: 10.1145/351240.351266. [16] ConsenSys Diligence. Daedaluzz: Benchmark generator for smart-contract fuzzers. https: //github.com/ConsenSysDiligence/daedaluzz, 2023. Accessed 2026-09-29. [17] Leonardo de Moura and Nikolaj Bjørner. Z3: An efficient SMT solver. In Tools and Algorithms for the Construction and Analysis of Systems (TACAS), volume 4963 of LNCS, pages 337–340. Springer, 2008. doi: 10.1007/978-3-540-78800-3_24. [18] Yinlin Deng, Chunqiu Steven Xia, Haoran Peng, Chenyuan Yang, and Lingming Zhang. Large language models are zero-shot fuzzers: Fuzzing deep-learning libraries via large language models. In Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA), pages 423–435. ACM, 2023. doi: 10.1145/3597926.3598067. [19] Echidna contributors. Echidna 2.3.0 release notes (symbolic execution mode) and 2.3.2-agentspreview-1 (mcp interface for fuzzing campaigns). https://github.com/crytic/echidna/ releases/tag/v2.3.0; https://github.com/crytic/echidna/releases/tag/v2.3.2-a gents-preview-1, 2026. Released 2025-12-15 and 2026-01-20. [20] eth-brownie contributors. Brownie: A Python-based development and testing framework for smart contracts. https://github.com/eth-brownie/brownie, 2023. [21] Josselin Feist, Gustavo Grieco, and Alex Groce. Slither: A static analysis framework for smart contracts. In Proceedings of the 2nd International Workshop on Emerging Trends in Software Engineering for Blockchain (WETSEB), pages 8–15. IEEE, 2019. doi: 10.1109/WE TSEB.2019.00008. 17

[22] Foundry contributors. Foundry v1.3.0 release notes: coverage-guided invariant fuzzing via corpus_dir. https://github.com/foundry-rs/foundry/releases/tag/v1.3.0, 2025. Released 2025-07-31. [23] Foundry contributors. Foundry: A blazing fast, portable and modular toolkit for Ethereum application development. https://github.com/foundry-rs/foundry, 2026. Version 1.7.1 used in this work. [24] Josef Gattermayer. Benchmarking frontier models on unpublished audits. ack3 research journal, https://ack3.ai/research/benchmarking-frontier-models-on-unpublished -audits/, 2026. Published 2026-08-03. [25] Josef Gattermayer, Michal Převrátil, and Jan Kuběna. Evaluating testing frameworks for Ethereum blockchain applications: Introducing Wake. Technical report, Ackee Blockchain, 2023. https://ackee.xyz/Wake_testing_framework_Whitepaper_draft_1.pdf. [26] Josef Gattermayer, Jan Kalivoda, and Arman Bašović. The ack3 H1 2026 DeFi incident dataset: Audit scope across 135 security incidents. arXiv preprint arXiv:2608.13792, 2026. [27] Gustavo Grieco, Will Song, Artur Cygan, Josselin Feist, and Alex Groce. Echidna: Effective, usable, and fast fuzzing for smart contracts. In Proceedings of the 29th ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA), pages 557–560. ACM, 2020. doi: 10.1145/3395363.3404366. [28] Bo Jiang, Ye Liu, and W. K. Chan. ContractFuzzer: Fuzzing smart contracts for vulnerability detection. In Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering (ASE), pages 259–269. ACM, 2018. doi: 10.1145/3238147.3238177. [29] George Klees, Andrew Ruef, Benji Cooper, Shiyi Wei, and Michael Hicks. Evaluating fuzz testing. In Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security (CCS), pages 2123–2138. ACM, 2018. doi: 10.1145/3243734.3243 804. [30] Haeun Lee, Soomin Kim, and Sang Kil Cha. Fuzzle: Making a puzzle for fuzzers. In Proceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering (ASE). ACM, 2022. doi: 10.1145/3551349.3556908. [31] Raymond Li, Loubna Ben Allal, Yangtian Zi, Niklas Muennighoff, Denis Kocetkov, Chenghao Mou, Marc Marone, Christopher Akiki, Jia Li, Jenny Chim, et al. StarCoder: May the source be with you! Transactions on Machine Learning Research, 2023. arXiv:2305.06161. [32] Ye Liu, Yue Xue, Daoyuan Wu, Yuqiang Sun, Yi Li, Miaolei Shi, and Yang Liu. PropertyGPT: LLM-driven formal verification of smart contracts through retrieval-augmented property generation. In Proceedings of the Network and Distributed System Security Symposium (NDSS). Internet Society, 2025. doi: 10.14722/ndss.2025.241357. [33] David R. MacIver, Zac Hatfield-Dodds, and many other contributors. Hypothesis: A new approach to property-based testing. Journal of Open Source Software, 4(43):1891, 2019. doi: 10.21105/joss.01891. [34] Quinn McNemar. Note on the sampling error of the difference between correlated proportions or percentages. Psychometrika, 12(2):153–157, 1947. doi: 10.1007/BF02295996. [35] Microsoft. Language server protocol specification. https://microsoft.github.io/langua ge-server-protocol/, 2024.

18

[36] Tai D. Nguyen, Long H. Pham, Jun Sun, Yun Lin, and Quang Tran Minh. sFuzz: An efficient adaptive fuzzer for Solidity smart contracts. In Proceedings of the 42nd International Conference on Software Engineering (ICSE), pages 778–788. ACM, 2020. doi: 10.1145/3377 811.3380334. [37] Nomic Foundation. Hardhat: Ethereum development environment for professionals. https: //hardhat.org/, 2026. [38] Michal Převrátil. Static analysis MCP tools in an AI audit benchmark. ack3 research journal, https://ack3.ai/research/ablating-static-analysis-from-an-ai-security-bench mark/, 2026. Presented at ETHBelgrade 2026 as “Does static analysis actually help AI?”. [39] Michal Převrátil. Benchmarking EVM fuzzers against an agent-written harness. ack3 research journal, https://ack3.ai/research/benchmarking-evm-fuzzers-against-an-agent-w ritten-harness/, 2026. Published 2026-08-26; presented at Web3 Security Summit Belgrade, 2026-08-26. [40] PyO3 contributors. PyO3: Rust bindings for the Python interpreter. https://github.com /PyO3/pyo3, 2026. Version 0.29 used in this work. [41] John Regehr, Yang Chen, Pascal Cuoq, Eric Eide, Chucky Ellison, and Xuejun Yang. Testcase reduction for C compiler bugs. In Proceedings of the 33rd ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI), pages 335–346. ACM, 2012. doi: 10.1145/2254064.2254104. [42] Moritz Schloegel, Nils Bars, Nico Schiller, Lukas Bernhard, Tobias Scharnowski, Addison Crump, Arash Ale-Ebrahim, Nicolai Bissantz, Marius Muench, and Thorsten Holz. SoK: Prudent evaluation practices for fuzzing. In Proceedings of the 2024 IEEE Symposium on Security and Privacy (S&P), pages 1974–1993. IEEE, 2024. doi: 10.1109/SP54263.2024.00137. [43] Chaofan Shou, Shangyin Tan, and Koushik Sen. Ityfuzz: Snapshot-based fuzzer for smart contract. In Proceedings of the 32nd ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA), pages 322–333. ACM, 2023. doi: 10.1145/3597926.3598059. [44] Chaofan Shou, Jing Liu, Doudou Lu, and Koushik Sen. LLM4Fuzz: Guided fuzzing of smart contracts with large language models. arXiv preprint arXiv:2401.11108, 2024. [45] Solidity Team. Solidity documentation. https://docs.soliditylang.org/, 2026. [46] Trail of Bits. slither-mcp: Slither analyses as MCP tools. https://github.com/trailofbi ts/slither-mcp, 2025. [47] Trail of Bits. Medusa: Parallelized, coverage-guided, mutational Solidity smart contract fuzzing. https://github.com/crytic/medusa, 2026. [48] András Vargha and Harold D. Delaney. A critique and improvement of the CL common language effect size statistics of McGraw and Wong. Journal of Educational and Behavioral Statistics, 25(2):101–132, 2000. doi: 10.3102/10769986025002101. [49] Justin Wang, Andreas Bigger, Xiaohai Xu, Justin W. Lin, Andy Applebaum, Tejal Patwardhan, Alpin Yukseloglu, and Olivia Watkins. EVMbench: Evaluating AI agents on smart contract security. In Proceedings of the 43rd International Conference on Machine Learning, volume 306 of Proceedings of Machine Learning Research, pages 125067–125099. PMLR, 2026. URL https://proceedings.mlr.press/v306/wang26k.html. [50] Sally Junsong Wang, Kexin Pei, and Junfeng Yang. SmartInv: Multimodal learning for smart contract invariant inference. In Proceedings of the 2024 IEEE Symposium on Security and Privacy (S&P), pages 2217–2235. IEEE, 2024. doi: 10.1109/SP54263.2024.00126. 19

[51] Gavin Wood. Ethereum: A secure decentralised generalised transaction ledger. Ethereum Project Yellow Paper, 2014. [52] Shuohan Wu, Zihao Li, Luyi Yan, Weimin Chen, Muhui Jiang, Chenxu Wang, Xiapu Luo, and Hao Zhou. Are we there yet? Unraveling the state-of-the-art smart contract fuzzers. In Proceedings of the 46th IEEE/ACM International Conference on Software Engineering (ICSE). ACM, 2024. doi: 10.1145/3597503.3639152. [53] Valentin Wüstholz. Benchmarking smart-contract fuzzers. ConsenSys Diligence blog, https: //diligence.consensys.io/blog/2023/04/benchmarking-smart-contract-fuzzers/, 2023. Accessed 2026-09-29. [54] Valentin Wüstholz and Maria Christakis. Harvey: A greybox fuzzer for smart contracts. In Proceedings of the 28th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE), pages 1398–1409. ACM, 2020. doi: 10.1145/3368089.3417064. [55] Chunqiu Steven Xia, Matteo Paltenghi, Jia Le Tian, Michael Pradel, and Lingming Zhang. Fuzz4all: Universal fuzzing with large language models. In Proceedings of the 46th IEEE/ACM International Conference on Software Engineering (ICSE). ACM, 2024. doi: 10.1145/359750 3.3639121. [56] Andreas Zeller and Ralf Hildebrandt. Simplifying and isolating failure-inducing input. IEEE Transactions on Software Engineering, 28(2):183–200, 2002. doi: 10.1109/32.988498. [57] Cen Zhang, Yaowen Zheng, Mingqiang Bai, Yeting Li, Wei Ma, Xiaofei Xie, Yuekang Li, Limin Sun, and Yang Liu. How effective are they? Exploring large language model based fuzz driver generation. In Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA), pages 1223–1235. ACM, 2024. doi: 10.1145/3650212.3680355.

A

Supplementary measurements

In-process versus JSON-RPC. An earlier internal run (July 2025) on the published Daedaluzz mazes, twenty minutes and one CPU per campaign, summed over five mazes, gave Echidna 64, Foundry 58, Wake over Anvil via JSON-RPC 51, and Wake with the in-process engine 71 distinct assertions. Tool versions were not recorded and the run predates the history-bounding fixes; it indicates only that, within Wake, the engine change is worth the difference between last and first place in a short campaign. L64 mazes. Five eight-hour seeds on a maze set whose constants span the full 64-bit range gave per-seed sums (sorted) of Echidna 65/70/71/73/73 (median 71), Foundry 74/74/74/75/75 (median 74) and Wake 5 67/67/68/69/71 (median 68). The reachable ceiling of that set was not established with the solver, so the values are not normalised; the ordering is consistent with Section 5.3: a small-value prior loses its advantage when constants are wide, and Foundry’s bytecode-mined dictionary gains one. Historical test-suite performance. The 2023 measurement that motivated Wake’s architecture [4, 25] ran 271 Uniswap v3 [6] test cases ported to each framework on a four-core cloud VM. Mean wall time on Anvil was 3.37 s for Wake, 10.96 s for Hardhat with Ethers.js, 34.80 s for Brownie and 53.27 s for Ape; on the Ganache chain the same suite took 15.78, 118.71, 51.48 and 72.72 s. Every framework ran over JSON-RPC in that measurement; the present work removes the socket from Wake’s default path. 20

Related documents

• SoK: You Find What You Seek: Rethinking Oracles, Guidance, and Input Generation in Hardware Fuzzing#1063511
• The Art, Science, and Engineering of Fuzzing: A Survey#1091976
• FuzzAgent: Multi-Agent System for Evolutionary Library Fuzzing#187251
• PickleFuzzer: A Case Study in Fuzzing for Discrepancies Between Python Pickle Implementations#187234
• FeatureFuzz: Treating Bug-prone Features as Core Primitives for Compiler Fuzzing#1386979