CornerCase: Automated Extremal Testing of Protocol Implementations Rathin Singha1 Kuan Qian1 Srinath Saikrishnan1 Tracy Zhao1 Soheil Abbasloo2 Ryan Beckett2 Siva Kesava Reddy Kakarla2 Todd Millstein1 George Varghese1
arXiv:2606.29124v1 [cs.NI] 28 Jun 2026
1
UCLA
2
Microsoft Research
characters – were unlikely to be generated by other automated test generators such as fuzzers [67] or testers that use symbolic execution engines [35, 59]. Problem: Elaborating, many real-world software bugs appear near the edges of the specification. This problem is exacerbated in network protocol implementations, which have complex input formats with many associated options and constraints. Edge cases include invalid field combinations in messages (e.g., the TCP Christmas Tree attack [43]), empty or missing fields, malformed fields, and rare but valid combinations of parameters. Notably, they include messages that violate protocol semantics, such as well-formed protocol messages received in unexpected states. As an example, unexpected handshake messages in TLS (e.g., CLIENT_HELLO in an invalid state) can lead to crashes or inconsistent state [18]. Other examples include the infamous Heartbleed bug [46] (CVE-2014-0160) in OpenSSL, where the claimed length was larger than the actual payload. The attacks above were manually crafted by hackers, but the rise of LLMs has raised the specter of attackers using AI to automatically generate such extremal attacks, just as Anthropic uses their Mythos [3] model to identify memory safety vulnerabilities. In this paper, we ask: can we use LLMs to automatically generate extremal inputs to harden Internet protocol implementations to (primarily) improve reliability and (secondarily) security, focusing on message safety errors – messages that trigger implementation bugs. Solution Approach: We answer this question in the affirmative with CornerCase, an approach and implementation that uses LLMs in a structured way to generate extremal tests for protocol implementations and analyze testing results (illustrated in Figure 1). The input to CornerCase consists of (i) a protocol specification (e.g., an RFC), (ii) a user-defined test input/output format (see Appendix § A), and (iii) a test harness for executing inputs against implementations. The output is a set of extremal test cases together with a ranked
Abstract Many software bugs in network protocol implementations arise near specification boundaries, such as inputs just within or outside allowed ranges, or messages that are valid in isolation but invalid in a given state. From the SSL Heartbleed exploit to TCP Christmas Tree packets, boundary inputs have repeatedly exposed critical weaknesses, yet remain under-tested by existing techniques such as fuzzing and model-based testing. We present CornerCase, an automated extremal testing approach that systematically targets such boundary behaviors. Our key idea is to decompose test generation into two stages: first, large language models (LLMs) extract explicit validity constraints from protocol specifications (e.g., RFCs) in a structured, section-by-section manner; second, extremal test cases are generated at or near the boundary of each constraint. These tests are executed across multiple implementations, and differential testing identifies inconsistencies. We evaluate CornerCase on widely used implementations of HTTP, DNS, BGP, SMTP, and QUIC, uncovering many previously unknown bugs. For example, the HTTP server h2o enters a redirect loop when processing URLs containing encoded null bytes. Overall, we used CornerCase to identify and file 42 anomalies; to date 26 have been acknowledged as bugs and 18 fixed, with others under active investigation.
1
Introduction
As physicists test theories on extreme cases such as infinite mass, software developers routinely test their code on what are colloquially called edge cases that are manually generated. Our paper uses AI to automate the generation of edge cases for network protocol implementations by first using LLMs to extract constraints from protocol specification documents (RFCs). We started this project after realizing that many semantically meaningful edge cases – such as long DNS names or URLs with invalid 1
set of differential anomalies observed across implementations. This design makes CornerCase protocol-agnostic and implementation-agnostic. In particular, our approach treats implementations as black boxes and does not require source-code access, making it applicable to both open and closed-source systems. Our approach leverages two properties of many protocols. First, they have detailed English specifications such as RFCs, which describe validity rules and constraints in natural language, so LLMs provide a natural mechanism for extracting and reasoning about these constraints. Second, protocols often have multiple independent implementations. This enables us to find bugs via differential testing: we can compare behaviors across implementations and flag inconsistencies, which often indicate real bugs or ambiguous parts of the specification. Methodologically, instead of asking the LLM to generate test cases in one shot, we first use it to extract explicit validity constraints from the specification document, one section at a time, guided by the user-provided test input format to focus on constraints relevant to our test setup. The LLM outputs constraints in a structured form (see Appendix § B) as tuples of constraint ID, section number, and the original RFC sentence which contains the constraint. As part of constraint extraction, we also resolve in-text references to other sections (if any) which are used to define the constraint and add to the extracted tuple. The key methodological choice behind this decomposition is to use the LLM for specification understanding before using it for test construction. Asking a model to directly generate protocol tests from a full RFC produces outputs that are broad but shallow: the model misses constraints, under-explores subtle boundaries, and fails to cover the specification systematically. By first extracting explicit constraints and only then generating extremal tests from them, we convert a vague end-to-end generation task into a coverage problem over constraints in the specification. We then separately ask the LLM to generate extremal tests (see prompt in § B.2) for each extracted constraint, ensuring all generated tests conform to the user-provided test format. Each constraint is translated into test cases by modifying only the relevant input fields to produce values just below, at, and just above the specified boundary. Separating specification understanding from test construction aligns with LLM strengths and avoids their weaknesses. We show using ablation testing (in Table 3) that our decomposition produces up to 22× more differential anomalies than one-shot LLM generation. Each generated test case is executed across multiple implementations using a user-provided test harness that runs inputs against implementations and records outputs in a unified format. We then perform differential analysis
to detect inconsistencies in behavior. Finally, we group related differential anomalies using LLM-generated tags that capture the underlying constraint and boundary type. A tag is a label that captures the constraint id and whether the test is just valid or just invalid (see § 3.3). The LLM also assists in drafting candidate bug reports for manual inspection and editing. Extremal testing is related to boundary-value analysis (BVA) [5, 56, 58, 69], a classical testing technique that targets inputs at the edges of allowed ranges. However, traditional BVA primarily focuses on numeric or range-based constraints and typically relies on manually identified boundaries. By contrast, extremal testing generalizes this idea to a broader class of constraints, extracting syntactic, semantic, and state-based rules from RFC text. A very preliminary version of the conceptual ideas in this paper was described in [60]. Sample Results and Insights: Using CornerCase, we discovered 42 bugs across HTTP, DNS, BGP, SMTP, and QUIC implementations. To characterize these, we distinguish between valid boundary cases – inputs that satisfy the specification but lie at the edge of acceptable behavior – and invalid boundary cases – inputs that are almost well-formed yet violate protocol rules. These boundaries may arise at the syntactic, semantic, or statemachine level. For example, the HTTP server h2o [51] responds to requests whose path contains a percent-encoded null byte (e.g., GET /%00) with a 301 redirect to a semantically equivalent path, inducing a redirect loop that is a potential vector for low-effort denial-of-service attacks (a valid semantic boundary case). Other examples include malformed Host headers accepted as valid (invalid syntactic boundary cases, one of which is a potential phishing exploit), nested MAIL transactions in SMTP (a state-machine boundary case), incorrect TLS versions accepted during the QUIC handshake (an invalid semantic boundary case), and BGP routes whose AS_PATH contains the router’s own confederation identifier (a valid semantic boundary case). These results suggest that many important protocol bugs arise not from arbitrary malformed traffic, but from inputs that sit just inside or just outside specification boundaries, or that are valid in isolation but invalid in context—precisely the regime that extremal testing is designed to expose. In the process of running CornerCase, we learned the following lessons that are elaborated on later. 1. Focus versus Context: Our first prompts missed many constraints until we prompted the LLM to go section-by-section in each RFC to focus the model on a small piece of text. We did better by adding sections referenced in a section’s constraints (more context). 2. Prompt Templates for General Protocols: By al2
Test Input Format
In batches
Constraints on Inputs
Test Output Format
User
RFC
1. Constraint Generation
Extremal Test Cases
2. Test Generation
User Test scripts
3. Test Execution Results from Implementations
Prioritized and Deduplicated anomalies by tag
Results with Confidence Scores
Tests with Diffs
5. Results Analysis
4. Differential Testing
6. Triaging
Figure 1: The Pipeline for Extremal Testing lowing the user to specify the test format as a JSON file, the framework can be easily extended to new protocols and previously unanticipated features. For example, our HTTP tests improved when we moved from a fixed filesystem – where the model only generates URI queries – to a richer format that allows it to specify which files should and should not exist, along with the query. 3. Bottleneck Shift: Our use of LLMs speeds up test and anomaly generation – so much so that the bottleneck now moves to bug validation and understanding. CornerCase produces hundreds of discrepancies per protocol, including genuine bugs, artifacts of user misconfiguration, and cases where multiple behaviors are arguably acceptable under the specification. To manage this volume, we use AI for prioritization, tagging, and triage. Our paper makes the following contributions: 1. Insight: We show that extremal testing becomes more powerful once LLM use is decomposed into two stages: constraint extraction from natural-language specifications, followed by extremal test generation from constraints, transforming end-to-end test generation into a coverage problem over constraints in the specification. 2. Methodology: We develop a protocol-agnostic, implementation-agnostic pipeline that uses only RFC text, a user-provided test format, and a black-box execution harness, enabling testing across heterogeneous implementations without source-code access or manual formalization. 3. Evidence: We evaluate the approach on 38 implementations across 5 protocols, discovering 42 bugs and inconsistencies (26 acknowledged, 18 fixed). An ablation study shows that each component of the decomposition is essential, with the full pipeline producing up to 22× more anomalies than one-shot LLM generation. The rest of the paper develops our claims from three
angles: the kinds of extremal bugs the method surfaces (in § 2), the structured pipeline that makes those bugs discoverable (in § 3), and empirical evidence that decomposition matters and finds new bugs across protocols (in § 4).
2
Motivating Examples
The following three examples are chosen to illustrate three different ways specification boundaries matter in practice. The HTTP null-byte case lies at a valid semantic boundary – percent-encoding rules permit it syntactically, but its interpretation falls outside what URIprocessing components are consistently defined to handle. The BGP confederation example is a valid semantic boundary: the message is well-formed, but its validity depends on reading one field (the AS path) in light of another (the router’s confederation identity). The SMTP nesting example is a state-dependent boundary: each command is individually valid, but the sequence becomes invalid given the protocol state. All three were found using CornerCase, acknowledged by the relevant developers, and are now fixed.
2.1
HTTP Null Byte Loops (h2o)
RFC 3986 states that a URI with percent-encoded null bytes (%00) "should be rejected if the application is not expecting to receive raw data" in the component. Historically, null bytes have been used in injection attacks that exploit inconsistencies in low-level string handling. While most modern HTTP servers treat such inputs as malformed and return a 400 Bad Request or 404 Not Found response in the context of static fileservers, we used CornerCase to identify two issues with null byte handling in HTTP servers: h2o [51] and Caddy [33]. The 3
model generated this test twice: from the RFC should statement, and the format rules for percent encoding. When a request is issued for a path containing a null byte (e.g., GET /%00), h2o responds with a 301 Moved Permanently redirect whose Location header points to a semantically equivalent path (e.g., /%00/), by appending a trailing slash. This can induce redirect loops, leading to repeated requests and resource amplification. While individual clients typically bound redirect depth, such behavior can still increase load in aggregate (e.g., from crawlers or automated clients), potentially creating a low-effort denial-of-service vector. Caddy, by contrast, propagates the malformed path to the filesystem layer, where a null byte in a stat call triggers a system-level EINVAL error that surfaces as a 500 Internal Server Error. Rather than rejecting the request as malformed at ingress, Caddy leaks an internal failure mode – one that may interfere with upstream error handling or monitoring systems that treat 5xx responses as server faults.
2.2
the command is syntactically valid but it is semantically invalid based on the current state of the protocol. Most SMTP servers that we tested rejected the nested MAIL FROM command with a 503 error or closed the connection. In contrast, Mailpit [21] responded with 250 2.1.0 Ok, effectively allowing a new transaction to begin while the previous one was still open. Allowing nested transactions can lead to inconsistent internal state and undefined behavior.
3
Our methodology seeks not just to automate test generation, but to convert natural-language specifications into a structured set of test targets. In that sense, the pipeline is best understood as a way of transforming RFC prose into a coverage object over which boundary-focused testing becomes systematic, traceable, and protocol-agnostic. Figure 1 shows the full pipeline in which we: (1) extract validity constraints from the specification (2) generate many extremal tests that are near the edge of these constraints (3) run these tests across multiple implementations and use differential testing to detect anomalies. We describe each stage below.
BGP Confederation Loops (GoBGP)
BGP requires that routers reject routes whose AS_PATH contains their own autonomous system (AS) number, in order to prevent routing loops. This rule also applies to BGP confederations. RFC 5065 specifies that if a router receives a route whose AS_PATH contains its own confederation identifier, it must treat it as an AS loop and reject the route. Extremal Testing generated a test case where a router receives a route whose AS_PATH contains its own confederation ID. Unlike the previous example, the received message is syntactically valid, but should be rejected because it violates a semantic constraint. When executed across implementations, FRR [12] and Batfish [27] correctly rejected the route, treating it as an AS loop. In contrast, GoBGP [13] accepted the route and installed it in the routing table. Accepting such routes can lead to incorrect behavior and potential loops.
2.3
Methodology
3.1
Input
Extremal Testing targets systems where (a) there is a written specification (often an RFC), and (b) there are multiple independent implementations that can be tested. These conditions hold for many network protocols. Our pipeline assumes the following user inputs: Specification document: An RFC or similar document that describes the protocol and its rules. Input/output test format: A structured representation of a test case in JSON format, including English descriptions of each test input field and a similar JSON file with test output fields and their description. (See Appendix § A for more details) A tester directory: A directory with a main script that can execute test cases in the JSON test input format against each implementation and output results in the JSON test output format.
SMTP Nesting Failures (Mailpit)
For an example of state-based constraints, SMTP restricts when a new mail transaction may begin. Once a MAIL FROM command has been accepted and one or more RCPT TO commands have been issued, the server expects the client to either send DATA or abort the transaction. Starting a new mail transaction with another MAIL FROM at this point is explicitly forbidden by RFC 5321. Extremal Testing generated a test case that issues a second MAIL FROM command to a server after it has accepted a recipient, but before DATA is sent. This example is extremal but different from the above two examples:
3.2
Stage 1: Constraint Generation
In the first stage of the pipeline, we extract the validity constraints from the specification. The output is a set of explicit constraints that can later be used for systematic test generation. Splitting the specification: Specifications are often too long to feed into the model at once. We therefore split the document into sections. We begin with the entire RFC in a single text document. We then scan the 4
RFC line by line and treat any line matching the sectionheader pattern ^\s*(\d+(?:\.\d+)*)\.\s+(.+)$. as starting a new subsection, where the first capture group is the section number (for example, 3, 3.1, or 3.1.2) and the second is the title. For each such header with section number k, we collect all lines from that header up to (but not including) the next header and save the resulting text in a file named section_k.txt. Here, k is the section number with dots replaced by underscores (for example, 3.1 becomes section_3_1.txt). We then feed these section_k.txt to the LLM one at a time, along with the test format given by user and a prompt to extract all constraints in the text of the section. The LLM returns a list of constraints for each section, which are appended to a global constraint list. Constraint definition: We define a constraint as any sentence in the specification that restricts how protocol inputs may be formed, ordered, or combined – that is, a boundary between acceptable and unacceptable input that an implementation is expected to enforce. Concretely, the LLM prompt (see Appendix § B) directs the model to find sentences that describe: syntax rules, allowed or disallowed values, length or size limits, character set restrictions, relationships between multiple inputs, ordering or state rules that can be represented as test inputs etc. The prompt specifies that constraints are generally RFC statements that use normative language (MUST, MUST NOT, SHOULD, SHOULD NOT), but also includes non-normative sentences that clearly describe a testable rule. Crucially, the LLM is instructed to return each constraint sentence exactly as written in the RFC— we do not rewrite, normalize, or formalize constraints at extraction time. Each constraint is recorded as a tuple ["<section_number>", "<constraint sentence>"]. This avoids introducing interpretation errors and allows later stages to trace each test case directly back to the source. The test input format plays a key role here: it is included in the prompt so that the LLM can infer which inputs are controllable in the testing framework, and thereby select only those RFC sentences that describe constraints on those inputs. This naturally filters out specification rules that, while valid constraints, cannot be exercised by the test setup (see also the discussion of test-format filtering below). Based on our extraction process, constraints typically fall into the following categories, illustrated here with concrete examples from our extracted constraint sets: 1. Range constraints: numeric bounds on values. For example, SMTP reply codes must begin with a threedigit numeric code (RFC 5321, §2.4), and DNS limits each label to between 1 and 63 octets (RFC 2181, §11).
2. Size constraints: minimum or maximum sizes for strings, lists, packets, or fields. For example, SMTP limits command line length to 512 octets including the <CRLF> (RFC 5321, §4.5.3.1.4), and DNS limits a full domain name to 255 octets including separators (RFC 2181, §11). 3. Format constraints: syntactic rules that inputs must follow. For example, URI scheme names must consist of a sequence of characters beginning with a letter and followed by any combination of letters, digits, plus, period, or hyphen (RFC 3986, §3.1), and HTTP header fields must follow a name-value syntax (RFC 9110). 4. Dependency constraints: relationships between multiple inputs. For example, in SMTP, the local-part of a mailbox MUST BE treated as case sensitive while verbs and keywords are not (RFC 5321, §2.4), and in URIs, a percent-encoded octet must be encoded as a character triplet consisting of "%" followed by exactly two hexadecimal digits (RFC 3986, §2.1). 5. Presence constraints: conditions under which a field or command is required or forbidden. For example, SMTP requires that a client MUST issue HELO or EHLO before starting a mail transaction (RFC 5321, §4.1.1.1), and TLS 1.3 requires that clients desiring certificate-based server authentication MUST send the signature_algorithms extension (RFC 8446, §4.2.3). 6. Enumeration constraints: inputs that must be chosen from a fixed set of allowed values. For example, in TLS 1.3, MD5, SHA-224, and DSA MUST NOT be offered or negotiated (RFC 8446, §4.2.3), and DNS record types must be valid RR types (RFC 1035). 7. Ordering and state constraints: rules about the order in which inputs or commands may appear. For example, SMTP specifies that mail transaction commands MUST be used in the prescribed order (RFC 5321, §3.3). TLS 1.3 requires that protocol messages MUST be sent in the order defined in Section 4.4.1 (RFC 8446, §4). 8. Cross-field semantic constraints: rules whose validity depends on interpreting the meaning of values across multiple fields or protocol state, beyond simple syntactic checks. For example, a BGP confederation member receiving an AS_PATH containing an autonomous system matching its own AS Confederation Identifier SHALL treat the path as if it contained its own AS number, i.e., reject the route as a loop (RFC 5065, §4). This constraint cannot be enforced by syntax checks alone – it requires comparing the AS_PATH content against the router’s own configuration state. Cross-reference detection. We detect crossreferences inside each constraint sentence using a similar regex pattern. Specifically, we look for textual references of the form “Section 4.1”, “Sections 4.1 and 4.2”, etc., and extract all referenced section numbers from the matched group and augment the model’s input with the corresponding RFC sections when generating tests. 5
Mapping constraints to the test format: The RFC contains many statements that might be categorized as constraints but may not be relevant for our test setup. When we extract constraints from the RFC sections, the setup summary is provided to pick only those constraints that are testable with our testing setup – e.g. A constraint from BGP RFC 4271 that says, "KEEPALIVE messages MUST NOT be sent more frequently than one per second" is not relevant if our test setup is testing the decision process and choosing the best route.
but must be rejected according to the RFC. Such nearboundary cases are difficult to generate using fuzzing or model-based testing. Tag Generation: Along with the test cases, LLM generates a tag for each of them. The tag is a label that combines the constraint identifier with a boundary indicator (positive or negative), such as C11_Positive or C18_Negative. Positive means just valid, and Negative means just invalid test cases. Batching constraints: Constraints are processed in small batches (5 by default). Each batch is sent to the LLM in a separate call. Batching is necessary in practice: providing too many constraints at once leads to repetitive outputs and poor coverage. Processing small batches encourages the model to focus on each constraint and produce multiple distinct extremal tests. Context provided to the LLM: For each batch, the LLM is given (see prompt in § B.2): the input test case format in JSON, the exact constraint sentences for the current batch, and, when applicable, the full text of any RFC sections referenced by the constraints. Providing referenced RFC sections helps the model correctly interpret constraints whose meaning depends on other parts of the specification, such as state transitions or exceptional cases. Test format and identifiers: All generated tests must conform exactly to the user-provided input test format. The LLM is instructed to populate each field according to the intended extremal test case. This grounding reduces failures due to unrelated parsing errors and increases the likelihood that tests exercise the intended code paths. The LLM outputs only a JSON array of test objects, with no additional text. Each test object includes a constraint field containing the exact RFC sentence it targets. Test identifiers are assigned post-generation to ensure global uniqueness across batches. Coverage objective: The objective of test generation is to maximize the coverage on the RFC constraints, not code coverage. For each extracted constraint, we aim to generate tests that explore behavior just below, at, and just above the specified boundary. Overall, this stage converts natural-language specification rules into concrete, boundary-focused test inputs in a systematic and reproducible way, while keeping the generation process simple and scalable.
De-duplication and normalization: New constraints are added to the global list only if the pair (section number, constraint) was not seen previously.
3.3
Stage 2: Test Generation
The goal of this stage is to systematically construct tests that lie close to the boundary between valid and invalid behavior, as described by each constraint. Constraint-driven generation: Each test case is generated from exactly one RFC constraint sentence. Constraints are passed to the test generator verbatim, together with their RFC section number and the text of any cross-referenced sections. Every generated test explicitly records the constraint that it is intended to exercise. This allows each test to be traced directly back to a specific sentence in the specification. Extremal values: For each constraint, we ask the LLM to generate multiple extremal test cases. These include: values that barely satisfy the constraint (e.g., minimum allowed length) and values that barely violate the constraint (e.g., one character too long), For numeric constraints such as 0 ≤ x ≤ 255, the LLM can generate values like −1, 0, 1, 254, 255, and 256. For size constraints such as len(s) ≤ 1024, the LLM generates strings with lengths just below, at, and just above the limit. For format and ordering constraints, the model is instructed to generate inputs that are syntactically valid but placed in invalid positions, or inputs that differ from valid ones by a minimal change. Example test cases: To illustrate the kinds of tests generated by Extremal Testing, consider the SMTP constraint in RFC 5321 Section 2.3.5, which states that – "the reserved mailbox name postmaster MUST be accepted in a RCPT command without domain qualifications." From this single constraint, our system generates both positive and negative boundary tests. A positive test checks that RCPT TO:<postmaster> is accepted after a valid EHLO and MAIL FROM sequence. In contrast, negative boundary tests modify the input slightly to violate the constraint, such as using an almost-matching name (postmaste) or an invalid form (postmaster@). These inputs differ by only a single character or structural detail,
3.4
Stage 3: Test Execution
After generation, we run the user-provided test script to execute tests on each implementation and produce a results file in the specified output format. In our experiments, these scripts use Docker containers for each protocol implementation. Given a test case t and an 6
implementation I, the tester starts or reuses the corresponding container or process, constructs the concrete input message(s) from t, sends them to I, waits for a response, and records the output in the expected format.
3.5
results so that manual investigation becomes easy. Prioritization and grouping: We sort analyzed test cases by confidence score to prioritize those most likely to correspond to real issues. Since multiple test cases may target the same specification constraint, we group results by their tag (see § 3.3) — Within each group, test cases are ranked by confidence score. The pipeline outputs all test cases, grouped and ranked, so that the user can inspect as many or as few as needed. In practice, reviewing the highest-confidence case from each group is an effective starting point, but the full set remains available for deeper investigation.
Stage 4: Differential Testing
After executing all test cases, we perform differential analysis to identify cases where implementations behave differently. The goal is not to decide which implementation is correct, but to surface test cases that deserve closer inspection. Diffing logic: For each test case, we use a Python script on the results file to compare the outputs across all implementations, each of which is a JSON dictionary. If at least two implementations return different dictionaries for the same test case, we add the test case and all responses to a separate file for detected anomalies. Output of differential analysis: All test cases with response disagreement are logged to a separate file. Each entry includes the original test metadata, and the actual responses from all implementations. This allows later stages to inspect, group, and reason about anomalies without losing contextual information.
3.6
Human validation: Results are reviewed manually about whether a case represents a true bug before submission to maintainers. The LLM is used only to assist with analysis, prioritization and deduplication. But this pipeline reduces human effort, allowing us to quickly move from hundreds of differential anomalies to a small, manageable set of high-quality, actionable bug reports.
3.8
The core of Extremal Testing is a two-stage decomposition of specification-based test generation: 1. Constraint extraction: Convert natural-language specification rules into explicit, test-format-aligned constraints. 2. Extremal Test generation: For each constraint, systematically generate tests at and around the specified boundary. This decomposition gives the pipeline two properties that test generation typically lacks. First, a form of completeness: by extracting constraints section by section and tracking which constraints have generated tests, we can measure and improve coverage over the specification. Second, traceability: every generated test records the exact constraint sentence and RFC section it targets, so any differential anomaly can be traced directly back to the specification text.
Stage 5: Result Analysis
Differential analysis can surface a large number of test cases that trigger divergent behavior across implementations. Some of these discrepancies are uninteresting, arising from user misconfiguration or cases where multiple behaviors are arguably acceptable under the specification, while others correspond to genuine bugs or specification violations. Our pipeline uses an LLM to analyze these differences and prioritize them, reducing the human effort required for validation and interpretation. LLM-based analysis of differential results: We process the output of differential testing in batches and feed it to an LLM for analysis. Each batch contains a small number of test cases (e.g., five), along with the implementation setup information and the full test metadata. For each test case, the LLM produces a short textual analysis explaining the observed disagreement and assigns a confidence score between 0 and 10, indicating how likely the case represents a real bug or meaningful inconsistency. The output of this stage is a list of test cases analyzed, each augmented with an LLM-generated comment and confidence score. These results are written to an analysis file that preserves all original test information.
3.7
Summary
4
Evaluation
This section evaluates Extremal Testing on multiple realworld protocols and libraries. Our evaluation is designed to answer three questions. Q1. Does Extremal Testing find real, previously unknown protocol bugs? Q2. Are those bugs qualitatively different from generic parser failures that existing fuzzers already surface? Q3. Is the structured decomposition—section-wise processing, constraint extraction, reference expansion— necessary, or would a one-shot LLM suffice?
Stage 6: Triaging
In the next stage of the pipeline we perform triaging to eliminate semantically similar results and prioritize the 7
The setup is described in § 4.1. Next the results in § 4.2 address the first two questions; the ablation study in §4.3 addresses the third question.
4.1
at least one implementation behaves differently from the others in a meaningful way—for example, a crash, a different error code (e.g., 4xx vs. 5xx), or acceptance vs. rejection of the same input. For each protocol, we ran the full CornerCase pipeline described in § 3. As part of its output, the tool extracts input constraints from the specification, generates boundary-focused test cases, executes them across multiple implementations, identifies differential anomalies, and applies LLM-assisted analysis to score each anomaly based on the likelihood of a specification violation. The tool also groups anomalies by the constraint they exercise and selects the highest-confidence instance per group to reduce redundancy (§ 3.7). From this ranked output, we use a confidence threshold for each protocol (generally 8), lowered when the total number of anomalies is small) to select candidates for manual inspection, and we submit bug reports for anomalies that pass this manual validation step. During manual inspection, some candidates are discarded because they correspond to implementation design choices, configurable behaviors, or cases where the RFC does not strictly mandate a single behavior. In addition, related anomalies are often grouped into a single report. These cases are excluded from the total number of reported bugs.
Experimental Setup
Model: Unless otherwise stated, all prompt-based stages of the pipeline—constraint extraction, test generation, and anomaly analysis—use GPT-5 via the OpenAI API. Full prompts are provided in Appendix § B.We evaluated Extremal Testing on the following implementations (details in § C) HTTP servers: Nginx [62], Apache [28], Lighttpd [36], Caddy [33], and H2O [51]. DNS servers: BIND [15], NSD [37], Knot [16], PowerDNS [14], CoreDNS [11], GDNSD [6], Technitium [68], HickoryDNS [30], TwistedNames [39], Yadifa [25] BGP implementations: FRR [12], GoBGP [13], Batfish [27]. SMTP servers: AioSMTPD [1], MailPit [21], Stalwart [38], SMTPD [20] and OpenSMTPD [22]. QUIC servers: quiche [10], msquic [45], quic-go [53], ngtcp2 [65], mvfst [44], kwik [24], picoquic [34], aioquic [40], neqo [48], nginx [49], chrome [54], lsquic [42], haproxy [63], quinn [55], go-x-net [64] All implementations were run inside Docker containers to ensure reproducibility. For HTTP, we test URI parsing rules from RFC 3986. Each test specifies a URI with scheme, authority, path, query, and fragment components, along with a filesystem layout with directories and symlinks; we compare the HTTP status code and resolved file path returned by each server. For DNS, we test resource record set semantics from RFC 2181, including duplicate suppression, TTL consistency, and caching precedence. Each test provides a zone file and one or more queries, and we compare the structured response—including answer records and return codes—across all resolvers. For SMTP, we test command syntax and sequencing rules from RFC 5321. Each test specifies a sequence of SMTP commands (e.g., EHLO, MAIL FROM, RCPT TO) and a server state, and we compare the three-digit response code returned by each server for the final command. For BGP, we test confederation handling (RFC 5065) and route reflection (RFC 4456). Each test specifies an AS topology, confederation membership, and advertised routes, and we compare whether each implementation installs the route in its RIB and the resulting AS path at each router. For QUIC, we test the TLS handshake from RFC 8446. Each test specifies modifications to an otherwise valid ClientHello message, and we compare whether the handshake succeeds or fails on each implementation using QuicInteropRunner [57] and a modified aioquic [40] client. We define a differential anomaly as a test case where
4.2
Differential Anomalies
Table 1 summarizes the number of unique constraints extracted, extremal tests generated, differential anomalies found and number of candidates after confidence-based prioritization and triaging – for each protocol. Each constraint produced between 3 and 11 test cases, including values at the boundary, just below it, and just above it. The candidates after the stage of triaging are manually inspected before passing them to the developers. Table 2 summarizes 42 bugs found across five protocols (after manual inspection). Of these, 26 have been acknowledged and 18 are fixed by maintainers, and the remainder have been submitted and are awaiting response. Manual inspection confirms these anomalies are not mere implementation differences: they expose dozens of concrete bugs and specification violations across widely-used protocol stacks, including security-relevant issues in HTTP host validation and BGP loop detection (Table 2). A few findings stand out for both their impact and protocol-critical nature. In BGP confederations, we found cases where GoBGP accepted routes with an AS loop and also formed sessions with invalid peer relationships, which can affect routing safety and policy isolation in multi-AS deployments; several of these were acknowledged or fixed. In HTTP, multiple servers accepted malformed or missing Host values but still served content, violating RFC host validation rules and creating a request-routing and 8
Protocol SMTP DNS BGP (Confederation) HTTP QUIC
RFC 5321 2181 5065 3986 8446
Constraints 177 65 28 239 136
Tests 630 213 98 757 328
Anomalies 526 156 64 297 16
Prioritized 84 28 25 75 6
Triaged 54 23 20 42 5
Table 1: Unique constraints, extremal tests, anomalies (disagreement among implementations), prioritized candidates (above the confidence threshold), LLM-triaged candidates (based on tag) per protocol. virtual-host security risk. In QUIC, version-negotiation behavior around TLS 1.2/1.3 combinations exposed interoperability and downgrade-surface issues, while in SMTP we observed state-machine and syntax-validation errors (for example, accepting MAIL FROM in invalid command sequences), which can enable inconsistent mail handling across implementations. Together, these examples show that extremal differential testing surfaces not only parser edge cases, but also high-value semantic bugs in security and correctness-critical protocol logic. Bug Patterns: The bugs fall into three broad categories. The most common is missing validation, where an implementation accepts syntactically invalid input that the specification requires be rejected—for example, aiosmtpd accepting MAIL FROM without angle brackets, or Caddy serving files for requests with malformed or missing Host headers. The second category is incorrect state machine transitions, where commands are accepted in the wrong order: Mailpit accepts MAIL FROM before any EHLO greeting and allows a nested MAIL FROM during an active transaction. The third is semantic misinterpretation, where an implementation misapplies a specification rule. For instance, FRR treats a Member-AS appearing in a regular (non-confederation) AS_PATH as a confederation loop, incorrectly rejecting valid routes. Protocol Trends: BGP confederation bugs involve semantic errors in routing logic, particularly in loop detection and session handling. HTTP bugs are often security-relevant, with malformed Host headers being accepted. SMTP issues stem from weak enforcement of syntax and command sequencing rules, DNS shows robustness problems in handling edge-case semantics and malformed inputs, and QUIC reveals inconsistencies in TLS version negotiation.
4.3
explicit constraints from the specification before generating tests (C), and (3) including referenced RFC sections when they are cited in the text (R). We evaluate how each of these components affects the number and usefulness of generated tests. We compare several variants of the pipeline: Base: The entire RFC is given to the LLM in one prompt, and the model directly generates test cases without section-wise processing, constraint extraction, or reference expansion. S: The RFC is fed to the LLM section by section, but the model generates test cases directly without first extracting constraints. SC: The RFC is processed section by section, constraints are extracted from each section, and test cases are then generated from the collected constraints. SR: The RFC is processed section by section and referenced sections are included when mentioned, but test cases are generated directly without an explicit constraint extraction step. CR: The entire RFC is provided at once, but the system extracts constraints before generating tests and includes referenced sections when necessary. SCR (Full System): Our complete pipeline, which processes the RFC section by section, extracts constraints, and includes referenced sections. For each variant, we generate tests and run them against all the implementations listed in § 4.1. We record the number of generated tests and the number of differential anomalies discovered across implementations. For variants that perform constraint extraction, we also report the number of extracted constraints. The ablation results across five protocols—DNS, BGP, SMTP, HTTP, and QUIC—are summarized in Table 3. The results show that the full pipeline (SCR) consistently produces the largest number of anomalies across all protocols. Each component contributes meaningfully: section-by-section processing (S) alone yields ≈ 10× more anomalies than Base for HTTP (13→131) and ≈ 22× for BGP (2→45). Adding constraint extraction further multiplies anomalies—for SMTP, SCR finds 526 vs. 124 for S alone. Reference expansion (R) has improved DNS results where SCR finds 156 anomalies vs.
Ablation Study
We perform an ablation study to identify which parts of the pipeline are responsible for turning raw LLM capability into systematic testing power. The headline result is that each layer of structure matters, and removing any one of them materially reduces anomaly yield (Table 3). Our system incorporates three key design choices: (1) processing the RFC section by section (S), (2) extracting 9
Protocol
Implementation
SMTP
Mailpit
SMTP
aiosmtpd
SMTP
aiosmtpd
SMTP
Mailpit
SMTP
OpenSMTPD
SMTP
Mailpit
SMTP
OpenSMTPD
SMTP DNS DNS DNS DNS
Stalwart Technitium Bind NSD Twisted
DNS
GDNSD
DNS DNS DNS
Yadifa Technitium CoreDNS
DNS
Yadifa
DNS DNS
HickoryDNS Twisted Names
BGP (Confed)
GoBGP
BGP (Confed)
FRR
BGP (Confed)
GoBGP
BGP (Confed)
GoBGP
BGP Confed
Batfish
BGP (Confed)
GoBGP
BGP (Confed)
GoBGP
BGP (Confed) BGP (Confed) BGP (Confed)
FRR Batfish FRR
BGP (Confed)
FRR
BGP (Confed)
FRR
BGP (Confed)
Batfish
BGP (Confed)
Batfish
HTTP
Caddy
HTTP
Caddy
HTTP HTTP
Caddy H2O
HTTP
Nginx
HTTP
H2O
HTTP
H2O
QUIC
kwik
QUIC
quic-go
Bug Description (brief) Accepts MAIL FROM before any HELO/EHLO, returning 250 instead of rejecting the command as a bad sequence. Accepts a syntactically invalid MAIL FROM command without angle brackets and returns 250 instead of rejecting it with a 501 error. Accepts invalid recipient syntax RCPT TO:Postmaster (missing angle brackets) and returns 250 instead of rejecting it. Accepts an invalid RCPT TO address containing a malformed source route (missing required colon), returning 250 instead of rejecting the syntax error. Rejects a second EHLO issued after MAIL FROM with a 503 error, even though RFC 5321 requires it to reset transaction state. Accepts a nested MAIL FROM command during an active transaction (after RCPT TO but before DATA), returning 250 instead of rejecting it. Accepts an unqualified local alias in MAIL FROM (e.g., <sales>) after EHLO, returning 250 instead of rejecting it. Stalwart drops connection after 5 rejected RCPT TO commands Returns invalid NS target for apex NS query. DNS server returns partial TXT RRSet without setting TC bit. DNS server returns partial TXT RRSet without setting TC bit. DNS server returns partial TXT RRSet without setting TC bit. Sets TC for duplicate-identical TXT RRs instead of suppressing duplicates in the RRSet KEY Record Type (RR Type 25) causes complete zone load failure. KEY Record Type (RR Type 25) causes complete zone load failure. CoreDNS drops zone records with \DDD decimal escape in owner name. Zone file parser treats \DDD decimal-escaped dot as a label separator instead of an intra-label byte. Authoritative answer returned for name below delegation (zone cut ignored). Authoritative answer returned for name below delegation (zone cut ignored). Accepts a route whose AS_PATH contains its own confederation ID/local AS (AS loop not rejected). Incorrectly rejects routes when its own Member-AS appears in a regular AS_PATH (non-confederation), treating it as a confederation loop. GoBGP incorrectly applies AS-loop detection using member-AS instead of confederation ID on eBGP sessions Incorrectly establishes an external session and propagates routes between peers in the same Member-AS. Does not distinguish AS_CONFED_SEQUENCE from AS_SEQUENCE in BGP RIB. Accepts EBGP session with same confederation ID from non-member peer and installs routes. Incorrectly accepts confederation-internal session with a non-member peer and installs routes. Confed-external peer with remote-as external readvertises route back to sender. iBGP route not propagated to peer within same confederation member-AS. Rejects route from eBGP peer whose AS equals the confederation identifier. Does not propagate route from R2 to R3 across inter-confederation eBGP session. Leaks confederation member-AS number in AS_PATH when advertising to external peer. remove-private-as all replace-as not parsed — private AS leaked to external peer BGP session incorrectly established when external peer’s AS matches a confederation member-AS. Serve files when the Host header is present but empty, instead of returning a 400 Bad Request. Invalid IP-literal values in Host header serves file instead of returning 400. (various examples) %00 in the request path returns a 500 internal server error %00 in the request path results in a 301 redirect to the same path Invalid IP-literal values in Host header serves file instead of returning 400. (various examples) Serves files when the Host header is entirely missing or present but empty, instead of returning a 400 Bad Request. Invalid values in Host header serves file instead of returning 400. (various examples) Client only offering obselete TLS 1.2 in supported versions passes Handshake instead of failing Server rejects handshake offering valid TLS 1.3 along with obselete TLS 1.2 in supported versions
Status Fixed Found Found Fixed Fixed Fixed Acked Acked Acked Found Found Found Fixed Found Found Fixed Found Acked Fixed Fixed Found Acked Acked Fixed Found Found Fixed Found Fixed Acked Fixed Found Fixed Fixed Fixed Fixed Fixed Acked Found Found Fixed Found
Table 2: Bugs discovered by Extremal Testing (total 42 bugs found, 26 acked and 18 fixed). Each row is one distinct implementation-level issue triggered by an extremal test derived from an RFC constraint. Fixed: patched by maintainers. Acked: acknowledged but not yet fixed. Found: reported and under developer investigation. Bugs span missing validation, state-machine violations, and semantic misinterpretation across all five protocols we evaluated.
10
100 for SC, probably because some DNS constraints are only fully specified via cross-referenced sections. The ablation suggests that the central challenge is not getting an LLM to produce tests at all, but getting it to cover the specification systematically. Full-RFC prompting (Base) is too diffuse for boundary discovery. Sectionwise processing (S) restores focus by localizing reasoning to a tractable unit. Constraint extraction (C) converts the RFC from unstructured prose into an explicit, enumerable set of test targets, turning test generation into a coverage problem. Reference expansion (R) sharpens this set when a constraint’s meaning genuinely depends on another section. The pipeline works not because it uses more prompting, but because each layer imposes additional structure on the model’s reasoning.
Protocol
SMTP
DNS
BGP Confed
The QUIC outlier: On QUIC, SC (20 anomalies) slightly exceeds SCR (16). We read this not as a contradiction but as evidence for a general tradeoff visible across the table: additional context helps only when it sharpens a constraint, and can hurt when it broadens the prompt without adding directly testable structure. Cross-references in RFC 8446 are heavy on prose commentary that does not translate into new controllable inputs in our test format, so reference expansion dilutes the prompt without adding coverage. This is consistent with the larger theme of the paper: for extra contextual information not to reduce focus, it must be dense with information that pertains to and enriches constraints.
4.4
HTTP
QUIC
Variant Base S SC SR CR SCR Base S SC SR CR SCR Base S SC SR CR SCR Base S SC SR CR SCR Base S SC SR CR SCR
#Cons. X X 154 X 41 177 X X 63 X 27 65 X X 23 X 20 28 X X 165 X 28 239 X X 168 X 21 136
#Tests 25 234 613 284 169 630 20 168 209 162 96 213 12 68 85 64 78 98 27 437 660 430 124 757 20 230 344 336 52 328
#Anom. 20 124 504 168 95 526 11 71 100 86 66 156 2 45 49 33 25 64 13 131 207 139 47 297 0 8 20 10 1 16
Table 3: Ablation of the three structural components of the pipeline: section-wise RFC processing (S), explicit constraint extraction (C), and targeted reference expansion (R). Columns report extracted constraints (#Cons., “X” when the variant does not perform extraction), tests generated (#Tests), and differential anomalies (#Anom.). The full SCR pipeline yields the highest anomaly count on four of five protocols, producing up to 22× more anomalies than the one-shot Base.
Lessons Learned
Our experience yields three empirical takeaways: 1. Decomposition beats direct generation: LLMs are substantially more reliable when asked to extract explicit constraints from a localized portion of a specification than when asked to generate end-to-end tests from an entire RFC. Section-wise processing and constraintfirst generation are not implementation details; they are the mechanism that turns LLM capability into systematic test coverage (Table 3: up to 22× anomaly yield vs. one-shot generation). 2. Bottleneck shifts from generation to understanding: With automated test generation, the dominant cost becomes interpreting and de-duplicating differential anomalies. Anomaly ranking, tagging, and triaging therefore become first-class parts of the methodology rather than optional engineering conveniences. For SMTP, we reduced 526 raw anomalies to 84 prioritized candidates and 54 triaged classes, which led to an orderof-magnitude reduction in manual inspection (Table 1). 3. A small interface layer suffices to generalize across protocols: The only protocol-specific inputs are a test format and an execution harness that expose the relevant controllable inputs and outputs. In practice, this
interface was sufficient to carry the same pipeline across HTTP, DNS, BGP, SMTP, and QUIC without retraining or redesigning the core method. Modern code-generation tools (e.g., GitHub Copilot) can bootstrap much of this interface from the specification itself, further lowering the per-protocol cost.
5
Related Work
Related work can be classified by what each approach relies on to generate tests or vulerabilities: implementations (fuzzing, symbolic execution, LLM-based unit testing, AI-based vulnerability analysis), past outputs (AI-based vulnerability analysis), formal models (model11
based testing), or natural-language specifications themselves. Extremal Testing sits in the fourth category. Boundary value analysis (BVA): BVA [5,56,58,69] typically focuses only on range constraints. Extremal Testing is far more general; for instance, it includes statebased constraints for messages. Typically, BVA does not use an LLM to generate boundary conditions. A recent exception [32] uses an LLM to directly generate boundary value tests from the code while we do so from the specification. Their evaluation is only for code of a few hundred lines, much smaller than the large code bases we handle. Automated testing: Fuzz testing is widely used for software testing in general [2,7,41,52,71] and specifically for BGP and DNS implementations (e.g., [17, 23, 29, 50, 61]). Although fuzzers are effective at finding parser bugs, random inputs have only a small probability of finding extremal bugs. Symbolic execution uses SMT solvers to generate test cases for many execution paths of a program (e.g., [9, 31]). Protocol implementations contain many thousand lines, making symbolic execution infeasible. Extremal testing, unlike symbolic execution, works on software whose source code is unavailable. Model-based testing uses an abstract model of a system to generate tests [8]. It has been used for DNS [35] and BGP [59] but generates valid, not extremal inputs. Eywa [47] uses an LLM to generate models, not tests. LLM driven test generation: LLM based software testing is a vast area [66], a subset of LLM based Software Engineering [26]. However, existing work [66] uses LLMs to improve coverage or to improve mutation/fuzz based testing, not for extremal testing. Our framework generates end-to-end tests on multiple implementations instead of unit tests for individual implementations. AI based vulnerability analysis: Recent work uses LLMs for security, including automated penetration testing [19] and code-level vulnerability detection [70]. LAPRAD [4] first trains an LLM on prior DNS attacks and then constructs new attacks; by contrast, we generate tests based only on RFCs.
6
command sequence. Extending to protocol behaviors that involve long multi-step workflows, timing dependencies, or complex state machines is interesting future work. Single RFCs: Constraint extraction is currently based on single RFCs. Protocol behavior can depend on multiple RFCs referencing each other. – e.g., RFC 9113 (HTTP 2) uses RFC 9110 (HTTP Semantics) which references RFC 3986 (URI Syntax). Multiple Models: While different LLMs may vary in performance on sub-tasks (e.g., constraint identification, triaging), we focus on pipeline-level design choices that are largely model-agnostic. Absolute performance can improve by choosing different models, but we expect the overall trends to remain consistent.
7
Conclusion
Extremal testing generates tests that occur near the boundaries of specifications. Our goal is to harden widely used Internet protocol implementations against messages with unexpected fields, or valid messages received in unexpected states which can degrade reliability, and in extreme cases expose dangerous vulnerabilities. At a higher level, the main lesson of this paper is that LLMs are most useful here not as end-to-end test generators, but to convert natural-language specifications into explicit test targets. We use LLMs twice: first, to extract constraints from protocol specifications and only then systematically generate test cases at or near the boundaries of these constraints. Our approach is protocolagnostic, allowing users to specify their own test input and output formats as JSON. We applied Extremal Testing to HTTP, DNS, BGP, SMTP, and the QUIC handshake, uncovering 42 bugs and inconsistencies. Of these, 26 have been acknowledged and 18 fixed by maintainers. Bugs span three broad categories: missing input validation (e.g., HTTP servers accepting malformed Host headers), state machine violations (e.g., SMTP servers accepting commands out of sequence), and semantic misinterpretation (e.g., GoBGP failing to reject AS loops in confederation paths). Our ablation study confirms that two-stage decomposition—constraint extraction followed by extremal test generation, applied section-by-section with cross-reference expansion—is essential, producing up to 22× more anomalies than naive one-shot LLM generation. Extremal testing’s focus on identifying errors due to violations of constraints in RFCs makes it complementary to existing approaches to protocol implementation reliability, including fuzz testing, symbolic execution, and model-based testing. Together, they form a powerful ensemble of techniques that can make Internet protocol implementations more resilient and secure.
Limitations and Future Work
Extremal testing currently has the following limitations: Implicit Constraints: The constraints defining the Heartbleed [46] exploit (payload length does not match length field) and Christmas Tree packets [18] (SYN, FIN, and RST flags should not be all set) are what one might call implicit constraints: they are not explicitly stated in the RFC but are strongly implied. It may be possible in future work to ask an LLM to generate implicit constraints by a new prompting strategy. Short interactions: We currently focus on generating extremal tests that involve a single message or short 12
[13] GoBGP community. GoBGP. https://github.com/osrg/gobgp, 2026.
References [1] aiosmtpd community. aiosmtpd - An asyncio based SMTP server. https://aiosmtpd.aio-libs.org/en/latest/, 2026.
[14] PowerDNS Community. PowerDNS. https://www.powerdns.com/, 2026. Github: https://github.com/PowerDNS/pdns. [15] Internet Systems Consortium. BIND 9. GitLab: https://www.isc.org/bind/, 2026. https://gitlab.isc.org/isc-projects/bind9.
[2] American Fuzzing Lop AFL. AFL 2018. https: //lcamtuf.coredump.cx/afl/. [3] Anthropic. Assessing Claude Mythos Preview’s Cybersecurity Capabilities. https://red.anthropic. com/2026/mythos-preview/, 2026. Accessed: 202604-19.
[16] CZ.NIC. Knot. https://www.knot-dns.cz/, 2025. GitLab: https://gitlab.nic.cz/knot/ knot-dns.
[4] R. Can Aygun, Yehuda Afek, Anat Bremler-Barr, and Leonard Kleinrock. LAPRAD: LLM-Assisted PRotocol Attack Discovery. In IFIP Networking 2025 Proceedings, 2025. Also available as arXiv:2510.19264.
[17] Stanislav Dashevskyi. A simple BGP fuzzer based on boofuzz. Github, 2023. https://github.com/ Forescout/bgp_boofuzzer. [18] Joeri de Ruiter and Erik Poll. Protocol State Fuzzing of TLS Implementations. In USENIX Security Symposium, 2015.
[5] Asma Bhat and S. M. K. Quadri. Equivalence class partitioning and boundary value analysis - A review. In 2015 2nd International Conference on Computing for Sustainable Global Development (INDIACom), pages 1557–1562, 2015.
[19] Gelei Deng, Yi Liu, Victor Mayoral-Vilches, Peng Liu, Yuekang Li, Yuan Xu, Tianwei Zhang, Yang Liu, Martin Pinzger, and Stefan Rass. Pentestgpt: An llm-empowered automatic penetration testing tool. arXiv preprint arXiv:2308.06782, 2024.
[6] Brandon L Black and Community. gdnsd. https://gdnsd.org/, 2023. Github: https://github.com/gdnsd/gdnsd.
[20] Python developer community. SMTPD Python library. https://docs.python.org/3.10/library/ smtpd.html, 2024.
[7] Marcel Böhme, Van-Thuan Pham, and Abhik Roychoudhury. Coverage-based greybox fuzzing as Markov chain. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, pages 1032–1043, 2016.
[21] Mailpit developers. Mailpit Email Testing Tool. https://github.com/axllent/mailpit, 2026.
[8] Josip Bozic, Lina Marsso, Radu Mateescu, and Franz Wotawa. A formal TLS handshake model in LNT. arXiv preprint arXiv:1803.10319, 2018.
[22] OpenSMTPD developers. OpenSMTPD Mail Server. https://github.com/OpenSMTPD/OpenSMTPD, 2026.
[9] Cristian Cadar, Daniel Dunbar, Dawson R Engler, et al. KLEE: Unassisted and automatic generation of high-coverage tests for complex systems programs. In OSDI, volume 8, pages 209–224, 2008.
[23] Donatas Abraitis Donald Sharp and et al. Fuzzing targets and supported fuzzers available in FRR. Github, 2023. https://docs.frrouting.org/ projects/dev-guide/en/latest/fuzzing.html.
[10] Cloudflare, Inc. quiche QUIC Implementation. https://github.com/cloudflare/quiche, 2018. (Accessed: 2026-04-10).
[24] Peter Doornbosch. Kwik QUIC Implementation. https://github.com/ptrd/kwik, 2018. (Accessed: 2026-04-10).
[11] CoreDNS community. CoreDNS. https://coredns.io/, 2026. Github: https://github.com/coredns/coredns.
[25] EURid.eu. Yadifa. https://www.yadifa.eu/, 2026. Github: https://github.com/yadifa/yadifa.
[12] FRR community. The FRRouting protocol suite. https://frrouting.org/, 2026. Github: https://github.com/FRRouting/frr.
[26] Angela Fan, Beliz Gokkaya, Mark Harman, Mitya Lyubarskiy, Shubho Sengupta, Shin Yoo, and Jie M. 13
Zhang. Large language models for software engineering: Survey and open problems. arXiv preprint arXiv:2310.03533, 2023.
[37] NLnet Labs. NSD. https://nlnetlabs.nl/projects/nsd/about/, 2026. Github: https://github.com/NLnetLabs/nsd.
[27] Ari Fogel, Stanley Fung, Luis Pedrosa, Meg WalraedSullivan, Ramesh Govindan, Ratul Mahajan, and Todd Millstein. A general approach to network configuration analysis. In Proceedings of the 12th USENIX Conference on Networked Systems Design and Implementation, NSDI’15, page 469–483, USA, 2015. USENIX Association.
[38] Stalwart Labs. Stalwart Mail Server. https://github.com/stalwartlabs/stalwart, 2026. [39] Twisted Matrix Labs. TwistedNames. https://twisted.org/, 2026. Github: https://github.com/twisted/twisted.
[28] Apache Software Foundation. Apache HTTP Server. https://httpd.apache.org, 1995. Source code: https://github.com/apache/httpd (accessed 2026-04-10).
[40] Jeremy Lainé. aioquic QUIC Implementation. https://github.com/aiortc/aioquic, 2019. (Accessed: 2026-04-10). [41] Hyojeong Lee, Jeff Seibert, Dylan Fistrovic, Charles Killian, and Cristina Nita-Rotaru. Gatling: Automatic performance attack discovery in Large-scale Distributed systems. ACM Trans. Inf. Syst. Secur., 17(4), apr 2015.
[29] Frederic Cambus. Fuzzing DNS zone parsers. https://www.cambus.net/ fuzzing-dns-zone-parsers/. [30] Benjamin Fry and Community. Hickory-DNS. https://github.com/hickory-dns/ hickory-dns, 2026. Github: https://github.com/hickory-dns/ hickory-dns/.
[42] LiteSpeed Technologies. LSQUIC QUIC Implementation. https://github.com/litespeedtech/ lsquic, 2017. (Accessed: 2026-04-10). [43] Gordon Lyon. NMAP Network Scanning: The Official NMAP Project Guide to Network Discovery and Security Scanning. Insecure, 2009.
[31] Patrice Godefroid, Nils Klarlund, and Koushik Sen. DART: Directed automated random testing. In Proceedings of the 2005 ACM SIGPLAN conference on Programming language design and implementation, pages 213–223, 2005.
[44] Meta Platforms, Inc. mvfst QUIC Implementation. https://github.com/facebook/mvfst, 2019. (Accessed: 2026-04-10).
[32] Xiujing Guo, Chen Li, and Tatsuhiro Tsuchiya. Boundary Value Test Input Generation using Prompt Engineering with LLMs: Fault Detection and Coverage analysis, 2025.
[45] Microsoft Corporation. MsQuic QUIC Implementation. https://github.com/microsoft/msquic, 2019. (Accessed: 2026-04-10).
[33] Matthew Holt. Caddy Web Server. https:// caddyserver.com, 2015. Source code: https:// github.com/caddyserver/caddy (accessed 202604-10).
[46] MITRE Corporation. OpenSSL TLS Heartbeat Extension Read Overrun (CVE-2014-0160). https://cve.mitre.org/cgi-bin/cvename. cgi?name=CVE-2014-0160, 2014. Accessed: 2026-04-17.
[34] Christian Huitema. picoquic QUIC Implementation. https://github.com/private-octopus/ picoquic, 2017. (Accessed: 2026-04-10).
[47] Rajdeep Mondal, Rathin Singha, Todd Millstein, George Varghese, Ryan Beckett, and Siva Kesava Reddy Kakarla. Eywa: Automating model based testing using llms. arXiv preprint arXiv:2312.06875, 2023.
[35] Siva Kesava Reddy Kakarla, Ryan Beckett, Todd Millstein, and George Varghese. SCALE: Automatically finding RFC compliance bugs in DNS nameservers. In 19th USENIX Symposium on Networked Systems Design and Implementation (NSDI 22), pages 307–323, 2022.
[48] Mozilla Corporation. Neqo QUIC Implementation. https://github.com/mozilla/neqo, 2019. (Accessed: 2026-04-10).
[36] Jan Kneschke. lighttpd Web Server. https: //www.lighttpd.net, 2003. Source code: https: //github.com/lighttpd/lighttpd1.4 (accessed 2026-04-10).
NGINX QUIC Implementation. [49] NGINX, Inc. https://github.com/nginx/nginx, 2020. Project page: https://quic.nginx.org/ (Accessed: 202604-10). 14
[50] NMAP Organization. Dns-fuzz. https://nmap. org/nsedoc/scripts/dns-fuzz.html.
[62] Igor Sysoev. NGINX HTTP Server. https:// nginx.org, 2004. Source code: https://github. com/nginx/nginx (accessed 2026-04-10).
[51] Kazuho Oku. H2O HTTP Server. https://h2o. examp1e.net, 2014. Source Code: https://github. com/h2o/h2o(accessed 2026-04-10).
[63] Willy Tarreau. HAProxy QUIC Implementation. https://github.com/haproxy/haproxy, 2022. QUIC support added in v2.6. Canonical source: https://git.haproxy.org/ (Accessed: 2026-04-10).
[52] Peach Fuzzer. https://peachtech.gitlab.io/ peach-fuzzer-community/.
[64] The Go Authors. golang.org/x/net: QUIC Package. https://pkg.go.dev/golang.org/x/net/ internal/quic, 2022. Source code: https:// github.com/golang/net (Accessed: 2026-04-10).
[53] quic-go contributors. quic-go QUIC Implementation. https://github.com/quic-go/quic-go, 2016. (Accessed: 2026-04-10). [54] QUIC Interop Working Group. Chrome Image for the QUIC Interop Runner. https://github.com/ quic-interop/chrome-quic-interop-runner, 2020. (Accessed: 2026-04-10).
[65] Tatsuhiro Tsujikawa. ngtcp2 QUIC Implementation. https://github.com/ngtcp2/ngtcp2, 2017. (Accessed: 2026-04-10). [66] Junjie Wang, Yuchao Huang, Chunyang Chen, Zhe Liu, Song Wang, and Qing Wang. Software testing with large language models: Survey, landscape, and vision. IEEE Transactions on Software Engineering, 2024. Also available as arXiv:2307.07221.
[55] quinn-rs contributors. Quinn: QUIC Implementation in Rust. https://github.com/quinn-rs/ quinn, 2018. (Accessed: 2026-04-10). [56] Muthu Ramachandran. Testing software components using boundary value analysis. In 2003 Proceedings 29th Euromicro Conference, pages 94–98. IEEE, 2003.
[67] Michal Zalewski. American Fuzzy Lop (AFL). https://lcamtuf.coredump.cx/afl/, 2014. Accessed: 2026-04-17.
[57] Marten Seemann and Jana Iyengar. Automating QUIC Interoperability Testing. In Proceedings of the Workshop on the Evolution, Performance, and Interoperability of QUIC, EPIQ’20, pages 8–13, New York, NY, USA, 2020. ACM. Co-located with SIGCOMM 2020, Virtual Event, USA.
[68] Shreyas Zare and Community. Technitium DNS server. https://technitium.com/dns/, 2026. Github: https://github.com/ TechnitiumSoftware/DnsServer. [69] Zhiqiang Zhang, Tianyong Wu, and Jian Zhang. Boundary value analysis in automatic white-box test generation. In 2015 IEEE 26th International Symposium on Software Reliability Engineering (ISSRE), pages 239–249. IEEE, 2015.
[58] Muhammad Sholeh, Irmah Gisfas, Muhammad Anwar Fauzi, et al. Black Box testing with Boundary Value Analysis and Equivalence Partitioning Methods. In Journal of Physics: Conference Series, volume 1823, page 012029. IOP Publishing, 2021.
[70] Xiaogang Zhou, Tianyi Zhang, and David Lo. Large language model for vulnerability detection: Emerging results and future directions. In Proceedings of the 2024 ACM/IEEE 44th International Conference on Software Engineering: New Ideas and Emerging Results (ICSE-NIER), 2024.
[59] Rathin Singha, Rajdeep Mondal, Ryan Beckett, Siva Kesava Reddy Kakarla, Todd Millstein, and George Varghese. MESSI: Behavioral Testing of BGP Implementations. In 21st USENIX Symposium on Networked Systems Design and Implementation (NSDI 24), pages 1009–1023, 2024.
[71] Sam Hocevar. zzuf: multi-purpose fuzzer. https: //caca.zoy.org/wiki/zzuf.
[60] Rathin Singha, Harry Qian, Srinath Saikrishnan, Tracy Zhao, Ryan Beckett, Siva Kesava Reddy Kakarla, and George Varghese. Extremal testing for network software using llms, 2025. [61] Robert Swiecki. Honggfuzz - Security oriented software fuzzer. https://github.com/google/ honggfuzz/tree/master/examples/bind. 15
A
Test Format
A.2
Test input format:
This appendix documents the input and output test formats used by each protocol-specific harness.
A.1
{
HTTP
Test input format: {
}
"test_id": "<integer>", "constraint": "<exact constraint that is being tested from the given list of constraints>", "description": "<description of the test case>", "tag": "<Constraint id from the given list of constraints. e.g. C1, C2, C3, ...>", "scheme": "<uri scheme>", "authority": "<uri authority>", "path": "<uri path>", "query": "<uri query>", "fragment": "<uri fragment>", "base_uri": "<base uri or null>", "expected_response_code": "<expected HTTP response code>", "expected_path": "<expected path after resolution>", "filesystem": { "<dir_path1>": ["<filename1>", "<filename2>", "..."], "<dir_path2>": ["<filename1>", "<filename2>", "..."], "<dir_pathN>": ["<filename1>", "<filename2>", "..."] }, "symlinks": { "<symlink_path1>": "<target1>", "<symlink_path2>": "<target2>", "<symlink_pathN>": "<targetN>" }
}
}
"test_id": "<integer>", "constraint": "<exact constraint that is being tested from the given list of constraints>", "description": "<Brief description of the test case. Positive for barely valid input, Negative for barely invalid input>", "tag": "<Combination of Constraint id from the given list of constraints and Positive/Negative indicator . e.g. C1_positive, C2_negative, C3_positive, ...>", "zone":"campus.edu.\t500\tIN\tSOA\tns1.outside.edu. root.campus.edu. 8 6048 4000 2419200 6048\ncampus. edu.\t500\tIN\tNS\tns1.outside.edu.\na.a.a.test.\ t500\tIN\tDNAME\tsome.domain.\n", "query": [{"Name": "*.a.test.", "Type": "DNAME"}]
Test output format: { }
"Response": "response from the DNS server in a structured format"
A.3
SMTP
Test input format: {
Test output format: {
DNS
"status_code": "<HTTP status code (integer)>", "resolved_uri": "<resolved URI after any redirects or normalization (string)>"
}
"test_id": "<integer>", "constraint": "<exact constraint that is being tested from the given list of constraints>", "description": "<Brief description of the test case. Positive for barely valid input, Negative for barely invalid input>", "tag": "<Combination of Constraint id from the given list of constraints and Positive/Negative indicator . e.g. C1_positive, C2_negative, C3_positive, ...>", "prev_command_seq": ["<cmd1>", "<cmd2>", "..." ], "server_state": "<server state after executing the previous commands e.g. INIT, EHLO_RCVD, MAIL_FROM_RCVD, ... etc>", "command": "<SMTP command to be tested>", "expected_response": "<expected server response to the command>"
Test output format: { }
16
"code": "<integer>, the SMTP response code from the server implementation"
A.4
BGP (Confederation)
A.5
QUIC
Test input format:
Test input format:
{
{
}
"test_id": "<integer>", "constraint": "<exact constraint that is being tested from the given list of constraints>", "description": "<Brief description of the test case. Positive for barely valid input, Negative for barely invalid input>", "tag": "<Combination of Constraint id from the given list of constraints and Positive/Negative indicator . e.g. C1_positive, C2_negative, C3_positive, ...>", "originAS": "integer e.g. 512. This is the AS number of the originating router", "router2": { "asNumber": "integer e.g. 768. This is the AS number of the second router", "subAS": "integer e.g. 256. If this is non-zero, then it is part of a confederation and this is the sub-AS number within the confederation" }, "router3": { "asNumber": "integer e.g. 1024. This is the AS number of the third router", "subAS": "integer e.g. 256. If this is non-zero, then it is part of a confederation and this is the sub-AS number within the confederation" }, "removePrivateAS": "boolean. A flag for router 2. Similar to cisco remove-private-as command. If this is true, then the router 2 should remove all private AS numbers from its AS path before forwarding it to router 3.", "replaceAS": "boolean. A flag for router 2. If this is true, then the router should replace all private AS numbers with the confederation number before forwarding the route to router 3.", "localPref": "integer e.g. 50. This is the local preference value set by router 2 when advertising to router 3.", "isExternalPeer": "boolean. A flag for router 3. If this is True then in BGP config of router 3. the neighbor is configured as neighbor peer-as external. i.e. if this is enabled then connection will be denied if the AS number router 2 is same as mine."
}
Test output format: { }
Test output format: {
}
"test_id": "<integer>", "constraint": "<exact constraint that is being tested from the given list of constraints>", "description": "<description of the test case>", "tag": "<Constraint id from the given list of constraints. e.g. C1, C2, C3, ...>", "tested_party":"<client | server>", "mutations": [{ "mutation": "<must be remove_field or modify_field>", "target": "<client | server>", "fields": { "field_name": "<name of the field to be removed or modified>", "new_value": "<new value for the field if mutation is modify_field, omit if mutation is remove_field>" } }, { "<mutation>": "more mutations if needed" }], "expected_result": "<success | fail>, this describes the expected outcome of the tested party.", "_comment": "this is for your information only. do not generate this field. Possible values for field_name and their new_value types are the following. See next item in the array for example on how to generate bytes, etc. random: bytes, legacy_session_id: bytes, cipher_suites: list[int], legacy_compression_methods: list[int],alpn_protocols : list[str], early_data: bool, key_share: list[ tuple [int, bytes] ], psk_key_exchange_modes: Optional[list[int]], signature_algorithms: list[int ], supported_versions: list[int] signature_algorithms: [list[int]], supported_groups: [list[int]] supported_versions: Optional[list[int ]], other_extensions: list[tuple[int, bytes]]. "
"isRIB2": "<boolean>, true if the route gets installed in RIB2", "aspath2": "<string>, AS path of the route at RIB2", "isRIB3": "<boolean>, true if the route gets installed in RIB3", "aspath3": "<string>, AS path of the route at RIB3"
"handshake_status": "success / failure"
B
LLM Prompts
We use GPT 5 over the OpenAI API for the experiments.
B.1
Constraint Generation Prompt
This prompt is used to extract testable constraints from each section of the RFC. System Prompt: You are an assistant that extracts *input-related constraints* from RFC for a specific testing framework.
17
### Inputs: - A test case format (in JSON) describing test case fields - A chunk of RFC text.
You are an assistant that generates *extremal test cases* from RFC constraints for a specific testing framework.
This prompt is used to generate extremal tests from each constraint identified earlier in the pipeline.
### Task: 1. Use the test case format to infer what inputs can be controlled by the tests. 2. Scan the RFC chunk and find sentences that define constraints on those inputs. These include: - syntax rules, - allowed or disallowed values, - length or size limits, - character set restrictions, - relationships between multiple inputs, - ordering/state rules that can be represented as test inputs/state. 3. Constraints are generally RFC statements that include MUST/MUST NOT/SHOULD/SHOULD NOT. But also look for sentences that describe a rule or constraint on inputs that can be tested with this framework. 4. Every constraint is written as a tuple: (<section_number>, <constraint>). 5. If the chunk has no relevant constraints, return [].
### Inputs You will receive: - The test case format (JSON schema or example object). - A list of RFC constraints (sentences). * A constraint is a sentence that describes a rule on inputs that the test framework can exercise (commands, arguments, states, etc.). ### Task Your job is to generate **extremal tests** for these constraints. Definition of extremal tests: - Tests at the boundary conditions between valid and invalid. - "Almost valid": barely violates a constraint (e.g., one character too many, one invalid character). - "Almost invalid": barely satisfies a constraint (e.g., minimum required length, edge of allowed range). - Test the precise point where valid becomes invalid. - Try to generate multiple extremal tests for each constraint, to cover all corner cases. - Include both positive tests (barely valid) and negative tests (barely invalid). - Consider interactions between components when relevant. - Focus on generating tests that might result in crashes/divergent behavior of serious consequence for security and reliability.
### Important: - Return each constraint sentence *exactly as written* in the RFC (no edits). - Only include sentences that can plausibly be tested using the described setup. ### Output format (for each chunk): Return ONLY a JSON array like: [ ["4.1.1", "sentence1"], ["4.1.1", "sentence2"], ["4.2", "sentence3"] ] No markdown, no explanation. Here is the test case format you should assume when deciding which RFC constraints are testable:
Now, here is a section of the RFC. Extract input-related constraints that can be tested with this test case format. Each constraint must be returned as a 2-element JSON array: ["<section_number>", "<constraint_sentence>"]. Return ONLY the JSON array as specified in the system prompt.
Requirements for the output: - Return ONLY a JSON array (no prose). - Each element is one test case object that follows the test case format. - Use ONLY the fields defined in the test format (no extra keys). - Every test object MUST include a "constraint" field set to exactly one of the provided constraint sentences (verbatim, nochanges). - Every test object MUST include a "test_id" field (you may choose any unique string or number within this batch; uniqueness across batches will be handled later).
=== RFC SECTION START === <RFC Section Text> === RFC SECTION END ===
Be explicit and systematic in exploring boundary conditions, but keep the output strictly as JSON.
B.2
The user prompt for this step includes more variables. As an example, we provide the full user prompt for batch_size = 2, where CornerCase asks the LLM to generate tests for constraints C1, C2.
=== TEST CASE FORMAT START === < Test Case Format in JSON> === TEST CASE FORMAT END ===
Test Generation Prompt
System Prompt:
18
Here is the test case format you must follow:
- Is one or more implementation likely violating the RFC (a real bug)? - Could the difference be due to acceptable implementation-specific behavior? - Could the difference plausibly be explained or fixed by configuration (e.g., security settings, extensions enabled/disabled, strictness toggles)? 2. Write a short, clear comment that summarizes your judgment, referencing: - which servers look suspicious, - whether this is likely a real bug vs. configuration/behavioral choice, - any caveats or uncertainty you have. 3. Assign a confidence score from 0 to 10 that represents how confident you are that this represents a REAL BUG (i.e., at least one implementation is non-compliant with the RFC): - 0 = very likely NOT a bug (probably config/expected behavior), - 10 = almost certainly a real bug.
=== TEST CASE FORMAT START === <test input format> === TEST CASE FORMAT END === === REFERENCED RFC SECTIONS (for context) === Section X: RFC section text Section Y: RFC section text ... === END REFERENCED SECTIONS === Here is a batch of RFC constraints. Each line has a section id, and the exact constraint sentence. Use these constraint sentences exactly (do NOT edit them) in the "constraint" field of your tests. You must generate multiple extremal tests for each constraint, covering positive and negative edges. ###Constraints: C1: [1.1] C2: [1.1] Constraint 2 text (references Section X, Y)
### Output Format Output format for each test: - You MUST output an array of objects, one object per test, of the form:
Now generate extremal test cases for these constraints. - Output ONLY a JSON array of test objects. - Each test object must: * Follow the test format fields. * Have a "constraint" field equal to exactly one of the sentences above. * Have a "test_id" field unique within this batch. Do not output any explanation or text outside the JSON array.
B.3
{{
}}
"test_id": <same integer test_id as input>, "comment": "<your explanation>", "confidence": <integer 0-10>
Constraints: - Return ONLY a JSON array (no prose, no markdown). - Do NOT omit any test from the batch; every input test must have exactly one output object. - Do NOT invent test_ids; they must match the ones you received.
Result Analysis Prompt
This prompt lets the LLM assign a confidence score and analysis text for whether the output of selected tests show real RFC violations and bugs. System Prompt:
User Prompt ### Test Results Batch (batch size {batch_size}): Here is a batch of test results where different implementations returned different responses. For EACH test in this batch, you must output an analysis object as described in the system prompt.
### Role You are an assistant helping to triage differences between {protocol_name} implementations.
### Tests <One or more Test Output JSON>
### Inputs You will be given a batch of (or a single) test results where multiple {protocol_name} implementations produced different responses for the same test case (zone + query).
### Start Analysis: Now, for this batch, return ONLY a JSON array of analysis objects of the form:
### Description of the fields in test results The test results for each test case are of the form:
{ "test_id": <int>, "comment": "<text>", "confidence": <int 0-10> }.
<Test Output Format in JSON>
Make sure every test above has exactly one corresponding analysis object.
### Task Your job for EACH test in the batch (or the single test) is to: 1. Reason about the differences between the implementations' outputs, considering:
C
19
Implementation Matrix
Protocol
Implementation Lang.
Description
DNS
BIND
C
DNS
NSD
C
DNS
Knot DNS
C
DNS
PowerDNS
C++
DNS DNS DNS
CoreDNS YADIFA HickoryDNS
Go C Rust
DNS
gdnsd
C
DNS
TwistedNames
Python
DNS
Technitium
C#
De facto standard DNS implementation, widely deployed in production. Authoritative-only DNS server, commonly used by TLD and ccTLD operators. High-performance authoritative DNS server with modern DNS feature support. Widely used DNS platform with flexible authoritative deployment support. Cloud-native DNS server widely used in Kubernetes deployments. Authoritative DNS server developed in the EURid ecosystem. Rust-based DNS stack emphasizing safety and modern implementation design. Lightweight authoritative DNS daemon designed for high-throughput serving. Python-based DNS server from the Twisted ecosystem, useful as a contrasting implementation style. Feature-rich DNS server used in self-hosted and enterprise settings.
HTTP
nginx
C
HTTP
Apache httpd
C
HTTP
Caddy
Go
HTTP HTTP
H2O lighttpd
C C
SMTP
smtpd
Python
SMTP
aiosmtpd
Python
SMTP
OpenSMTPD
C
SMTP
Mailpit
Go
SMTP
Stalwart
Rust
BGP(Confed.)
FRR
C
BGP(Confed.)
GoBGP
Go
BGP(Confed.)
Batfish
Java
QUIC QUIC
quic-go go-x-net
Go Go
QUIC
picoquic
C
QUIC
HAProxy
C
QUIC QUIC QUIC
MsQuic mvfst nginx
C C++ C
QUIC
quinn
Rust
QUIC
neqo
Rust
QUIC
quiche
Rust
QUIC
lsquic
C
QUIC
aioquic
Python
QUIC
ngtcp2
C
QUIC
kwik
Java
High-performance web server and reverse proxy, widely deployed in production. Long-standing modular HTTP server with broad compatibility and deployment footprint. Modern HTTP server known for automatic HTTPS and simple configuration. Performance-oriented HTTP server with strong HTTP/2 support. Lightweight HTTP server often used in resource-constrained environments. Python standard-library SMTP server, serving as a simple baseline implementation. Asyncio-based SMTP server framework representing modern Python async behavior. Security-focused mail transfer agent with relatively strict SMTP semantics. Lightweight SMTP testing server commonly used in development environments. Modern Rust mail server emphasizing safety and integrated mail functionality. Mature open-source routing suite widely used in operational BGP deployments. API-friendly BGP implementation suited to programmable controlplane experimentation. Control-plane analysis engine used to model and validate routing behavior. Widely used Go QUIC stack and a common interoperability baseline. QUIC implementation from the Go networking ecosystem used for comparison. Lightweight C QUIC implementation often used in experimentation and interop testing. Deployment-oriented QUIC-capable implementation integrated into a production load balancer. Microsoft’s high-performance cross-platform QUIC implementation. Meta’s QUIC stack, designed for large-scale production deployment. QUIC-enabled NGINX implementation integrated with mainstream web-serving workflows. Rust QUIC implementation emphasizing safety and clean transport abstractions. Mozilla’s QUIC/TLS stack used in browser and protocol experimentation. Cloudflare’s QUIC implementation with broad visibility in the ecosystem. LiteSpeed’s C QUIC implementation optimized for practical deployment. Python QUIC implementation that we modified to support handshake-field mutation. Standards-focused C QUIC implementation with strong conformance emphasis. Java QUIC implementation providing JVM-based interoperability coverage.
Table 4: Protocol-wise implementation matrix used in our evaluation, including implementation language and a brief distinguishing characteristic of each system. 20