Do Open-Source Video Ad Clients Follow VAST 4.3? An Empirical Study of Player Conformance

Do Open-Source Video Ad Clients Follow VAST 4.3? An Empirical Study of Player Conformance Prashant Chaudhary

arXiv:2610.39093v1 [cs.SE] 29 Oct 2026

Independent Researcher prashant@prashantchaudhary.com ORCID: 0009-0001-1980-109X

29 October 2026

Abstract Context. Video ads reach streaming and connected-TV apps through VAST documents, and companies count ads from the HTTP requests the player’s client library sends. A tracking error can leave playback intact and surface later as disagreeing counts. Objective. To measure how far open-source VAST clients meet the player duties of VAST 4.3, whether verdicts depend on the runtime, what generated inputs add, and when deviations were introduced. Method. A catalog of 108 requirements was extracted from the specification. Six open-source clients in eight configurations ran against 69 crafted test vectors and the IAB’s sample tags, under Node.js and in five browsers on three engines, with every request recorded. Grids, fuzzing and a sweep over 36 releases extend the catalog; Google’s closed IMA SDK is a comparison. Results. The open-source clients have 78 findings and 3 to 21 must-level deviations per configuration. None of the tested open-source clients that follow redirect (Wrapper) chains fills the required macros in a Wrapper’s request; the archived IMA 3.794.0 configuration tested does. One library’s timeout throws an exception no caller can catch, ending a Node.js process. In a pod (ads played back to back), one player credits a finishing ad’s completion to the next ad, fixed in release 5.3.0. A library can conform under jsdom and deviate in every tested Chromeand Safari-family browser. Conclusions. The tested clients diverge on the duties other companies use to count and diagnose ads. A shared conformance suite run in real browsers could expose most graded deviations; the harness and vectors are released. Keywords: VAST; conformance testing; specification-based testing; test oracles; ad SDKs; empirical study

1. Introduction Many standards are implemented many times over, often by small libraries with no published check against the text. When such a library is wrong, the consequence often falls on a party that never runs it. Video advertising is an extreme case. An ad in a streaming service or a connected-TV app reaches the screen through a VAST document (Video Ad Serving Template) (IAB Technology Laboratory 2022). The ad servers in the chain are run by different companies. The first often does not have the ad and returns a Wrapper, a document that points to another company’s server; the 1

Client library inside a video app or web page

Start 25 % 50 % 75 %

Request the ad tag

Ad server 1

4

Impression, quartiles, complete, error

Points to the next server Adds its own tracking URLs

2

Follow the Wrapper

Ad server 2

4

Impression, quartiles, complete, error

Points to the next server Adds its own tracking URLs

3

Follow the Wrapper

Ad server 3

4

Impression, quartiles, complete, error

Holds the ad Media file and tracking URLs

1

Player

End

Fetch a VAST document (steps 1–3) Tracking request during playback (4, sent to every server)

Wrapper

Wrapper

InLine

Each server counts the ad only from the requests it receives

Figure 1. How a VAST ad reaches the screen and is counted. The player follows a chain of Wrappers to the InLine ad, then sends impression, progress and error requests to every server in the chain; each server counts the ad only from the requests it receives.

player follows the chain to an InLine document that holds the ad. As the ad plays, the player’s client library sends HTTP requests back to every server in the chain. It sends an impression request when the ad starts, quartile and completion requests as it plays, and an error request when something fails (Figure 1). Advertisers pay, and ad servers diagnose their supply, from those requests. A library that sends them wrongly breaks nothing on screen; the ad plays and the content resumes. A pod is a group of ads played back to back, each with its own completion URL. A player receives a pod of three ads and plays all three without a visible problem. On the wire, the first ad’s completion request is never sent, the second ad’s is sent when the second ad starts, and the third ad’s is sent twice. The total number of completions is right, but the first ad is never credited, the second is credited early and the third twice, and a check of the pod’s total alone shows nothing wrong. The player is Shaka Player 5.2.12; its 5.3.0 release fixes this case and adds one regression (Section 5.1.3). A second case is a crash. Some companies resolve VAST chains on a server, for example for server-side ad insertion, which stitches ads into the video stream itself. There, a slow server anywhere in a chain makes one widely used library throw an exception that no caller can catch, and the whole Node.js process exits. The libraries are widely used: in September 2026 the most downloaded of the dedicated VAST libraries, vast-client-js, was downloaded about 237,000 times from npm, and Shaka Player, whose ad module is one of the clients tested, about 1.6 million times. Downloads count package fetches by developers and build systems, not players in the field, so they indicate use, not deployment. VAST is an open specification, but its conformance is self-declared. IAB Technology Laboratory (IAB Tech Lab), the industry body that maintains VAST, publishes the specification, sample tags and a tag validator; a validator checks the document an ad server sends. This study found no published suite that checks what a player does with that document, and no academic study known to the author has tested VAST clients against the specification (Section 3). This paper provides that study for six open-source VAST client libraries used in web and Node.js players. It asks four research questions: • RQ1 (Conformance). To what extent do open-source VAST clients meet the VAST 4.3 2

player duties they take on (some libraries leave duties such as rendering to the integrating player), where do their deviations cluster, and how does a closed production SDK compare? • RQ2 (Runtime). Which verdicts depend on the runtime and platform the client runs in, and why? • RQ3 (Beyond the catalog). Does one test vector per requirement predict a client’s behavior on generated variants of the same failure, and what does mutation fuzzing find that the catalog does not? • RQ4 (History). When were the deviations introduced, and how long have they persisted? Study design. The oracle is the specification text. The player obligations of VAST 4.3 and of the IAB macro list (the list of URL placeholders a player fills; Section 2) were extracted into a catalog of 108 requirements, each with the section it rests on and its strength as written (must, should or may; Appendix A). A recording server serves crafted test documents (the test vectors) and logs every request a client makes, and each requirement is graded mostly from what reaches the network (Figure 2). Each client runs through its documented integration in two layers: under Node.js (the Node.js layer) and in real browsers (the browser layer). The open-source clients are analyzed white-box: every deviation is traced to the source line that produces it, or to the absence of any code for the duty. Google’s closed IMA SDK is a black-box reference, graded only from its network requests and the events of its public API (Section 5.1.5). Generated error and macro grids, a mutation fuzzer and a sweep over earlier releases test the catalog’s coverage and date its findings (Section 4). Contributions. 1. The first empirical conformance study of VAST clients, to the author’s knowledge: six opensource clients in eight configurations and one closed SDK, graded on 108 requirements in each layer the client runs in (Node.js, and five browsers on three engines), with 86 findings (78 in the open-source clients, 8 in IMA). Each finding is tied to its requirements where the catalog has one, to a source location in the open-source clients or to the observable evidence for IMA, and, where the sweep covers it, to a release range (Appendix B). 2. A cross-client analysis showing that deviations concentrate on the requests other companies rely on (macros in redirect requests, error codes, pods, progress events and verification), plus robustness defects outside the specification (Sections 5.1 and 5.3). 3. Evidence that a library’s verdict can depend on the XML parser of the runtime it runs in. A test suite that runs only under jsdom, as those of vast-client-js and video-ad-sdk do, therefore cannot see a class of defects (Section 5.2). 4. A released harness, vector set, requirement catalog and result data, packaged to rerun every layer, the grids and the access check on another machine (Statements and Declarations). Scope. The study covers client-side insertion of linear video ads by JavaScript libraries, which is how web players and many connected-TV web apps play VAST. Server-side ad insertion, interactive formats, companion and overlay ads (shown beside or over the content), and native mobile and TV SDKs are outside it; the coverage map in the replication package records each duty left out and why.

2. What VAST asks of a player VAST 4.3 states that compliance is a two-party effort: ad servers send valid responses, and media players play the ad as instructed, honor the instructions of every server in a redirect chain, and send 3

tracking information back (§2.1, §2.2). The duties this study tests fall into eight groups; Section 4.2 regroups them into nine for reporting, adding robustness. Wrappers (§2.3.5, §3.19). The player follows each Wrapper’s VASTAdTagURI until it reaches an InLine document. It is required to accept five Wrappers (§3.19.1) and may reject more, reporting error 302 to every Wrapper’s Error URI (§2.3.5.1). Wrapper attributes constrain the chain. With followAdditionalWrappers="false", a further Wrapper is to be ignored. allowMultipleAds defaults to false, which means that only the first stand-alone ad (one that is not part of an ad pod, described below) of the next response may be used. fallbackOnNoAd asks the player to try another ad when the chain yields none. When the InLine has no ClickThrough URL, the one in the Wrapper closest to the InLine is used (§2.3.5.2). Errors (Error Reporting section, §3.2.1, §3.4.11). When an ad cannot be served, the player requests the Error URIs and replaces the [ERRORCODE] macro with a code from the specification’s table. The codes include 100 for an XML parsing error, 301 for an unavailable or timed-out Wrapper URI, 302 for the Wrapper limit, 303 for no ad after Wrappers and 403 for no supported media file. An error in an InLine ad reached through Wrappers is reported to every Wrapper as well. A response with no ad may carry document-level Error URIs, which the player must then request. Tracking (§2.3.4, §3.4.4, §3.14). All Impression URIs of the InLine and of every Wrapper before it are requested at the same time, or as close to it as possible. Linear tracking events (start, firstQuartile, midpoint, thirdQuartile, complete) fire when playback reaches 0, 25, 50, 75 and 100 percent; progress events fire at an offset given as a time or a percentage. Pause, resume, mute, unmute, rewind, skip and loaded each have an event. Wrappers can carry their own tracking elements, which fire alongside the InLine ones. Macros (§6 and the IAB macro list (IAB Technology Laboratory 2026)). URLs in a VAST document carry macros, bracketed names such as [TIMESTAMP] that the requesting party replaces with a value. The list marks five as required: [TIMESTAMP] (ISO 8601), [CACHEBUSTING] (a random 8-digit integer), [ERRORCODE], [LIMITADTRACKING] (whether the user has limited ad tracking) and [REASON]. CACHEBUSTING and TIMESTAMP apply to VAST request URIs as well as to tracking URLs, and the party that makes the request, usually the player, replaces them (§6.1). Values are percent-encoded one by one. Listed optional macros that the player does not fill become −1 (value unknown) or −2 (value known but withheld by policy), but macros the player does not know are not to be replaced with −1 (§6.1). XML and parsing (§2.2, §2.3.2). The response is well-formed XML 1.0, and comments and character entities are honored. From VAST 4.2 a creative can carry several UniversalAdId elements, and from 4.1 a MediaFiles element several ClosedCaptionFile elements. Ad pods. Ads with a sequence attribute form a pod. A player that plays a pod “must play the ads in the Pod in the prescribed sequence and should play as many of the ads as possible.” It moves on when one fails and should substitute an unplayed stand-alone ad from the same response. Pod support is optional, as is the skippable linear ad type (§2.3.1). Verification and icons (§3.11, §3.16, §3.17). Verification vendors are third parties that measure an ad with their own code run in the player. A player that does not run that code must request that vendor’s verificationNotExecuted URI with a [REASON] code. The code is 1 when the player declines the resource, 2 when it does not support the API framework (the interface that code expects, such as the Open Measurement Interface Definition, OMID) and 3 when the resource fails to load. When icons (small images shown over the ad) are present, the player displays them after the ad starts and requests their view and click URLs. 4

Media and versions (MediaFiles, §4.3). The player chooses a media file by its attributes, not its position, and VAST 4 players should keep accepting older documents.

3. Related work Specification-derived oracles. In the taxonomy of Barr et al. (2015), this study’s catalog is a specified oracle, its crash detection is an implicit oracle, and its differential pass is a derived one. The closest methodological precedent is RFCcert (Chen et al. 2018; Tian et al. 2019), which extracts certificate rules from the RFC text and runs targeted certificates through SSL/TLS libraries. Like RFCcert, this study takes the expected behavior from the standard instead of from agreement between implementations. It differs in three ways. Requirements are extracted by hand and kept as a citable catalog with their obligation strengths. The observed behavior is network traffic during playback, where RFCcert observes an accept or reject verdict. No prior test corpus exists. Conformance testing of many independent implementations of one protocol is established practice in security research (de Ruiter and Poll 2015; Somorovsky 2016; Sivakorn et al. 2017). This study transfers it to a domain whose implementations are small, whose specification is prose with a partial schema, and whose harm is economic. Placing the method. Grading a client from a record of the requests it sends is log-file analysis used as a test oracle (Andrews and Zhang 2003): each oracle reads the request log of a run, not the client’s internal state. The strength labels use the requirement levels must, should and may as RFC 2119 defines them (Bradner 1997); VAST 4.3 does not invoke RFC 2119, so the rule of Section 4.2 decides where its text has no such word. RQ3’s question, whether a client gives the same code for the same failure at every depth and level, is an invariance relation of the kind metamorphic testing uses (Segura et al. 2016), with the expected value taken from the specification. The robustness oracle follows Ballista (Kropp et al. 1998) in classifying how off-the-shelf components fail under exceptional inputs, from crashes and hangs to wrong error reports. A shared suite of player vectors, as proposed in Section 6.2, has a precedent in web-platform-tests, a test suite written to run in every browser that browser projects use to check their implementations of the Web platform specifications (web-platform-tests contributors 2026). Differential testing. Differential testing (McKeeman 1998) takes disagreement between implementations as the bug signal, from compilers (Yang et al. 2011) and virtual machines (Chen et al. 2016) to certificate validation (Brubaker et al. 2014) and guided input generation (Petsios et al. 2017). Herbold and Tunkel (2023) found across machine-learning frameworks that equivalent configurations are hard to establish and that most raw differences are noise. VAST clients raise a sharper form of that problem: they take on different shares of a player’s duties, and several share code (one player in this study bundles another library). A majority vote would hide a defect two clients share, so this study grades against the text and uses disagreement only to triage mutated documents. Conformance and consistency studies of library families. Kuchta et al. (2018) study inconsistencies among PDF readers, the closest precedent in form: many consumers of one document format, with divergence as the object of study. Harrand et al. (2021) measure how Java JSON libraries differ on well-formed and ill-formed input; Davis et al. (2019) show that regular expressions are not portable across engines; Lima et al. (2021) transplant test suites between JavaScript engines and find bugs each engine’s own suite missed. Wang et al. (2026) study whether software bill of materials (SBOM) tools adhere to their standards, with a longitudinal stage that parallels this study’s version sweep, and Prado et al. (2026) attribute most divergence among SBOM generators to design choices and specification ambiguity rather than implementation errors. In HTTP, differential 5

fuzzing of parsers has found discrepancies whose victim is a third party (Jabiyev et al. 2021; Kallus et al. 2024), the same structure as a VAST client whose errors land on an ad server’s counts. This study adds a requirement-level catalog, observes side effects instead of outputs, and dates each finding by release. Ad SDKs and ad measurement. Ahasanuzzaman et al. study how mobile ad libraries are released and updated (2020) and how apps integrate them (2022); Gui et al. (2015) measure the hidden costs ads impose on apps and users. FraudDroid (Dong et al. 2018) and MAdFraud (Crussell et al. 2014) detect deliberate ad fraud from network traffic, and Wu et al. (2026) show that mobile ad SDKs decide which interactions count as billable. Measurement studies of smart TVs and streaming devices map which tracking services apps contact (Mohajeri Moghaddam et al. 2019; Varmarken et al. 2020), and Marciel et al. (2016) show that video view counting can be inflated on the platform side. These studies share this paper’s premise that counting is decided in client code; they look for fraud or for which hosts are contacted, while this study grades whether libraries that try to comply send the requests the specification requires. VAST. Published work on VAST is descriptive or architectural (Seeliger et al. 2016; Smith 2016). Chaudhary and Khandelwal (2026) separate trigger, readiness and playback events in accounting for dynamic ad insertion (ads inserted into a stream at playback time) on synthetic data; the present study checks whether real clients emit those events at all. The nearest work is Sekowski’s VASTlint validator and its preprint (Sekowski 2026a), which measures whether structural validation of VAST tags fits inside a real-time bidding pipeline (the auction that selects an ad for each request). This study adds the client side: a validator grades the tag an ad server sends and cannot see a client that drops a correct request. The same project has since released vastlint-client (Sekowski 2026b), a session runtime that this study tests as a client. Some findings were known to maintainers in part: the vast-client-js timeout exception had been reported as a browser error (8bitsten 2026), and the missing macro replacement in Wrapper requests had been raised for vast-client-js and iab-vast-loader (hwde 2019; Doca 2019). This study adds the wire evidence, the release range and the Node.js consequence.

4. Study design The study follows the engineering-research and benchmarking standards of the ACM SIGSOFT Empirical Standards (Ralph et al. 2020). Figure 2 shows the pipeline.

4.1 Subjects Survey. Sixteen queries against the npm registry on October 16, 2026 returned 267 packages whose name, description or keywords mention VAST, the related IAB formats VMAP, VPAID and SIMID, or IMA; Online Resource 1 lists the queries, the packages and every decision. Each plausible client was classified from its registry metadata; the ten whose role the metadata left unclear were unpacked and searched for their own handling of VASTAdTagURI, TrackingEvents and [ERRORCODE]. A package counts as a VAST client when it (1) carries an OSI-approved license, (2) has its own VAST implementation rather than wrapping vast-client-js, video-ad-sdk or IMA, (3) resolves Wrappers or fires tracking, and (4) had a release in the 24 months before October 16, 2026. Sampling. The sample is purposive. Three clients were chosen before the survey because they are the longest-standing independent libraries: vast-client-js, video-ad-sdk and iab-vast-loader, the last 6

Execution

Test inputs

Recording server

69 crafted vectors

Serves the test documents

49 IAB sample tags

Rewrites URLs to itself

Google IMA (black box)

Generated error and macro grids

Logs every request

Load script and 8 playback scripts (full, skip, click, pod, . . . )

1,442 fuzz cases

Injects errors, delays, stalls and redirects

Client adapters documents

6 open-source clients

requests

Grading

Node.js + jsdom

5 browsers

Request log

Oracles

Verdicts

Tracing

Findings

Every request, timestamped

One per catalog requirement (108)

Conforms, deviates, delegated, n/a

Each open-source deviation traced to its source line

Defects Release ranges Issue reports

Figure 2. Study pipeline. In the execution band, the recording server serves the test documents to each client, which runs under a load or playback script, and logs every request the client sends back. In the grading band, one oracle per catalog requirement reads the log and gives a verdict for each cell (one requirement on one client configuration in one runtime); every deviation of an open-source client is then traced to client source; IMA’s are described from its requests and public API.

being the library behind IAB Tech Lab’s original VAST Tester. video-ad-sdk and iab-vast-loader have since aged out of condition (4) and are kept as reference points for the players built on them; no current deployment is claimed. Shaka Player was added by hand, because it is a widely used media player with its own VAST parser that the npm queries did not return under these keywords. Of the candidates the survey found, vastlint-client (a session runtime from the VASTlint project) was tested, and rmp-vast, a player built on vast-client-js that therefore fails condition (2), was added as a real integration of vast-client-js; its release 17.2.0 is distributed through GitHub, and its npm package stops at 6.0.0. Four small player applications that pass all four conditions were not tested: itube-modern-player, streamit-player, ts-video-player and stormcloud-video-player. They embed VAST code in a larger player instead of offering it as a library; their adoption data are in Online Resource 1, and testing them is left to future work. IMA fails condition (1) and is kept as a closed comparison. Download counts were not used to select clients, because three small packages in the survey show nearly equal monthly counts (about 23,000 each) that look like a mirror or an automated client. The clients are vast-client-js (Dailymotion 2026), video-ad-sdk (Rambler Digital Solutions 2024), iabvast-loader (Zentrick 2021), vastlint-client (Sekowski 2026b), Shaka (Shaka Project 2026), rmp-vast (Radiant Media Player 2026) and IMA (Google 2026). iab-vast-loader runs in two configurations (library defaults, and its noSingleAdPods option, which stops it from filing a lone ad as a one-ad pod) and rmp-vast in two (defaults, and omidSupport, the setting under which it handles verification at all), which makes eight open-source configurations: seven in the browsers and five under Node.js. All versions are pinned. video-ad-sdk ships directory imports meant for bundlers, and iab-vast-loader depends on a module loader that fails on Node.js 22; both are bundled unchanged with esbuild. rmp-vast runs from its vendored distribution file, whose hash is recorded.

7

Table 1. Clients and configurations. “Claimed VAST” is the version the client’s documentation states; “none” means no claim was found. Duties it leaves to the integrator

Client

Version

Style

Claimed VAST

vast-client-js

6.5.0

parser, resolver, tracker

4.3

video-ad-sdk iab-vast-loader (with iab-vast-parser 1.2.1) vastlint-client

4.0.1 2.5.1

full player loader

none 2.0 to 4.0 (its model package)

0.15.0

session runtime

none (its validator: 2.0 to 4.4)

Shaka Player

5.2.12

media player

rmp-vast (on vast-client-js 6.4.2) Google IMA HTML5 SDK

17.2.0

full player

none (“basic support”) 3 and 4 up to 4.3 none

3.794.0

closed full player 2 and 3; 4 in part

media, rendering, ad fallback, verification decisions none everything after loading

error codes, media and playback, skip-control availability, verification none

none

Layers Node.js, browsers

Node.js, browsers Node.js, browsers

Node.js

browsers browsers

browsers (comparison)

Version yardstick. Every client is graded against VAST 4.3, the current published version, because a player in the field receives whatever version the ad servers send. Four requirements on which a client deviates are newer than its claim: iab-vast-loader’s rejection of repeated UniversalAdId (X05, added in 4.2) and ClosedCaptionFile (X06, 4.1) and its unreplaced macros in a Wrapper’s request (M12; macro-based ad requests were added in 4.1), against its 4.0 claim, and IMA’s ignoring of the Wrapper ClickThrough (W17, 4.2). They are reported with that note. The clients without a claim are graded the same way.

4.2 Requirement catalog Extraction. The catalog was built from an inventory of the 122 lines of the VAST 4.3 text, from §2.1 on, that contain must, should, required or may and mention the player or Player Support, plus the IAB macro list. Each duty was mapped to a requirement or to a recorded reason for leaving it out. Appendix A lists every requirement with its section and its strength as written. Requirement IDs carry a prefix: W Wrappers, E errors, T tracking, M macros, X XML and parsing, P ad pods, A verification, I icons, S media selection, R robustness. P04 is unassigned. Duty groups. Table 2 gives the nine duty groups by which results are reported. They follow the ID prefixes with one exception. The Wrapper requirements are split by what they grade: W04 and W06 to W13 (error codes and Error URIs along the chain) join error reporting, and W14 to W18 (tracking and clicks along the chain) join impressions and tracking.

8

Table 2. Requirements by duty group and strength. “observation” counts robustness requirements that are not VAST clauses (Appendix A). Duty group

Requirements

must

should

may observation

Wrapper resolution (W01 to W03, W05, W19 to W23) Error reporting (W04, W06 to W13, E01 to E09) Impressions and tracking (W14 to W18, T01 to T08, T10 to T23) Macros (M01 to M12) Ad pods (P01 to P03, P05 to P07) XML and parsing (X01 to X03, X05, X06, X08 to X19) Media selection (S01, S02) Verification and icons (A01 to A05, I01 to I03) Robustness observations (R01 to R06, T09, X04, X07) Total

9

3

5

1

0

18

14

4

0

0

27

18

9

0

0

12 6 17

7 3 7

5 3 10

0 0 0

0 0 0

2 8 9

0 6 0

1 2 0

1 0 0

0 0 9

108

58

39

2

9

Strength. One rule assigns strength. A requirement is must where the text puts a must or “required” on the player for that duty, in the body or in an implementation note for media players. It is also must where the macro list marks a macro Required in the context tested. It is should where the text says should, or where the duty is read from a definition, an attribute or an optional macro with no modal verb. Where the body and a note conflict, the body governs, and the other reading is tested in a sensitivity analysis (Section 5.1.6). Appendix A marks the 23 requirements whose graded strength is not the literal word of the text. Macro list version. The IAB macro list is maintained outside the specification and grows. The study uses the list at commit 99d5d51 of IAB Tech Lab’s VAST repository (IAB Technology Laboratory 2026), which has 59 macros. The 50 whose introduction version is 4.2 or earlier are the graded set (the “4.3 set”). When VAST 4.3 was published, the list held 54 macros: these 50 and four marked “Next VAST release” (STOREID, STOREURL, PLAYBACKMETHODS and CONTENTCAT), which the list now marks “4.x”; the 4.3 text itself adds a value to [PLAYBACKMETHODS]. Five macros were added in 2023 and 2024: GPPSTRING and GPPSECTIONID, also marked “4.x”, and three macros for the EU Digital Services Act (DSA), marked “Next VAST release”. The catalog’s macro requirements use only the 50; the macro grid reports them apart from the other nine. Vectors. A generator writes 69 vectors with 109 documents (a vector with Wrappers has several). Of the 69, 52 serve one requirement, 9 serve two and 8 serve three or more; the largest, a fully played InLine ad, serves nine requirements (T01 to T08 and E08). Every URL carries a label naming the document and element it came from (for example w1/clicktracking), so an oracle can tell which tracker fired. Requirements whose duty concerns the InLine ad, or no document level in particular, use vectors that start with an InLine; the Wrapper requirements use Wrapper chains. Documents follow the element order of the IAB VAST 4.2 schema, and 102 of the 109 validate against it. The other seven are invalid on purpose (a malformed document, a document without the VAST namespace, three VAST 2.0 and 3.0 documents, a UniversalAdId with the 4.0 idValue attribute, and an unknown tracking event name). IAB’s 4.3 schema file is empty, so validation uses 4.2.

9

Verdicts. A cell is one requirement on one client configuration in one runtime. Each cell receives one of four verdicts: • conforms or deviates: the client takes on the duty and does or does not meet the requirement; • delegated: the client’s documentation leaves the duty to its integrator (Table 1); • n/a: one of five reasons applies: – the requirement is optional and not implemented; – the client declined a feature it may decline (pods, skippable ads); – the cell only records a measurement (such as T09, the spread of impression requests); – the client exposes no control the script needs; – the cell depends on a behavior another requirement already graded as missing. Grading rules. Four rules keep one fault from being counted many times or against the wrong code. • G1. G1 applies only to clients that do not follow Wrappers. Such a client deviates on W01 and W02, the two requirements that grade following a Wrapper, and every other Wrapperdependent cell is n/a for it. A client that follows Wrappers is graded on every Wrapper cell. • G2. A requirement that reads a value another requirement already failed (a macro left unreplaced, progress events unsupported, a pod not continued) is n/a with a pointer to that requirement. • G3. For error codes, the exact code conforms, and so do the group’s general code (100, 200, 300, 400) and 900, which the text names as the permitted minimum; a different specific code deviates. Where the error table can be read to give more than one code, every such code is accepted. • G4. A client built on another client is graded on its own behavior, and each deviation is attributed to the code that produces it. When rmp-vast deviates on a path inside its bundled vast-client-js, the finding is reported under vast-client-js, while rmp-vast’s cell still records the deviation, because a user of rmp-vast receives it. From cells to findings. A finding is one cause, located in source. For an open-source client, each deviating cell was traced to the source line in the tested release that produces it, or, for a duty the client never implements, to the absence of any code that reads the relevant element (stated with the search that shows it). Cells with one cause form one finding. Fuzzing failures are merged into root causes (Section 4.4). IMA is closed, so it is analyzed as a black box: each of its findings rests on the requests the server logged and on the error events of its public API, and, where a probe vector isolates a behavior, on that probe; no explanation is drawn from its code. IMA was not swept or fuzzed. Appendix B lists the 86 findings; Online Resource 5 gives every verdict by client and by duty group.

4.3 Harness and test execution A recording server serves the test documents and rewrites every absolute URL inside them to point back at itself, so Wrapper fetches, media requests and tracking requests all arrive at the server. The 10

server logs each with its arrival time in milliseconds. Query parameters make it return an HTTP status, delay a response, stall a body after its headers, redirect, compress, or change the content type. Node.js layer. Each test runs one client in its own Node.js process (Node.js 22.22.0), with browser globals supplied by jsdom 26.1.0, the environment in which vast-client-js and video-ad-sdk run their own test suites. Requests sent through new Image() or navigator.sendBeacon() become real HTTP requests. Playback uses a scripted video element whose clock the harness advances in steps and which fires the media events a real element fires. Browser layer. The same catalog, vectors and oracles run in real browsers through Playwright 1.56.1. The libraries are bundled into one page with esbuild 0.25.10, the scripted element is replaced by a real video element playing real media (a synthetic 10 s ad and 40 s content clip, test patterns with a tone, each as VP9 WebM and H.264 MP4), and the browser’s own DOMParser, fetch and Image replace jsdom’s. The media set also holds an audio-only pair and a corrupt file. Each test gets a fresh browser context. Nine requirements need real decoding or rendering (media failures, audio ads, pod failure handling, icons) and are graded only in the browser layer. Platforms. On Linux (x86-64), Chromium 141, Firefox 142 and WebKit 26 run through Playwright. The packaged harness was also run on the author’s Apple silicon Mac mini (macOS 27.0.1) with Chromium, Google Chrome 156, Firefox, WebKit and Safari 27.0.1; the whole package took 73 minutes. The five browsers use three engines: Blink (Chromium, Chrome), Gecko (Firefox) and WebKit (WebKit, Safari). Chrome and Safari results come from macOS, because a Linux Chrome entry resolves to the Chromium binary; WebKit is reported from both platforms, because the Linux WebKit build differs from macOS WebKit in media support (Section 5.2). Chromium and Firefox results are the Linux references, which the Mac run reproduced in every open-source cell compared (Section 5.2). IMA did not run in Safari, whose driver refused the proxy setting that serves the archived SDK. Scripts and integration. A test either loads the tag or plays it with one of eight playback scripts (Table 3). For vast-client-js and vastlint-client, which leave part of a player’s work to the integrator, the harness acts as an integrator that calls each library exactly as its documentation describes. This integrator code (code/harness/adapters/vastclient.mjs and vastlint.mjs) is part of the released harness, with its hashes. rmp-vast draws its own controls, so the harness builds the page layout its README prescribes and acts as a user would, clicking its video and skip button. IMA is driven through its public API and its own controls (Online Resource 4). It exposes no seek control, so its rewind cells are n/a. Its icon cells I01 and I03 are graded from the requests, and I02, which needs a click on the icon, is not drivable. Table 3. Load and playback scripts. Script

What the harness does

load full partial click user actions skip rewind icon pod

resolves the tag; no playback plays the ad to the end plays to 60 percent and stops the ad clicks the ad at 30 percent pauses, resumes, mutes, unmutes and changes the volume while playing probes for the skip control at 1.5 s and skips at 6 s seeks back 2 s, from 6 s to 4 s clicks a displayed icon plays every ad of a pod

11

Oracles. Each requirement has one oracle, a function that reads the request log of its cell and, for some requirements, what the client’s API returned (the resolved ad, its error code). The log is never de-duplicated: a request sent twice is two entries. Impressions, start and quartile events, progress, pause, resume, mute, skip, rewind and loaded events, pod impressions and completions, and the Impression, tracking and click-tracking requests of the InLine and the Wrappers (W14, W15, W18) are graded exactly once, so a second request deviates. Unmute is graded as fired (T15), and an extra unmute after a volume change is graded by T23. Error requests are graded on presence and code. Timing is graded only where the text implies it: quartile events must arrive in order; a pod ad’s completion must arrive before the next ad’s impression, with 50 ms of tolerance. An icon’s view request must not precede the start request by more than 50 ms, and a timed icon’s must arrive at least 800 ms after start for a 1 s offset. The simultaneity of impressions is measured, not graded. After a script ends, the harness waits 300 ms (Node.js) or 600 ms (browsers), longer for the full players, then grades both presence and absence on the log. For example, W13 serves a chain of two Wrappers and an InLine whose only media file has an unplayable type, plays it, and grades the fan-out: the InLine’s and both Wrappers’ Error URIs must be requested with one consistent code. The code itself is graded by E03 (rule G2), so a client that sends one wrong code to all three is graded on that code once. X04 and X07, which are not VAST clauses, are graded as observations: the client conforms when it parses the document or follows the URI. Timing constants. The harness uses fixed thresholds, chosen as engineering settings rather than derived from the specification: a 50 ms ordering tolerance (a pod ad’s completion against the next ad’s impression, an icon’s view against start; the quartile order check has none), settle windows of 300 ms under Node.js and 600 ms in browsers before an absent request is graded, an 800 ms threshold for an icon with a 1 s offset, and, in fuzzing, a 2 s client timeout, a 30 s wall limit and a slow-load limit of three times the client’s timeout plus 5 s (11 s). Measured margins: every conforming progress request in the browser full players arrives within 310 ms of its nominal offset; the slowest fuzz run that finished took 9.6 s; and one cell sits on a threshold, IMA’s I01. Its icon view precedes start in every run on every engine and platform, by 36 to 237 ms in 60 repeated Linux runs (20 per engine), and the 50 ms tolerance falls inside the Linux Chromium spread (36 to 68 ms; beyond it in 6 of 20 runs), so I01 is the one cell whose verdict depends on the run. On macOS, in 10 repeated runs per browser, the view preceded start by 11 to 12 ms in Chromium, 10 to 13 ms in Chrome and 13 to 50 ms in Firefox (never beyond the tolerance) and by 51 to 58 ms in WebKit (beyond it in all 10) (Table 6).

4.4 Generated inputs Error grid. The catalog tests each error requirement with one vector; the error grid enumerates the space those vectors sample. For every chain depth d from 2 to 6 and every failing level L from 2 to d (15 pairs), and each of 12 failures, the document at level L fails. Levels 1 to L − 1 are Wrappers the client has received, whose Error URIs the client must request with a code from the error table. The failures fall into three families: • network failures (HTTP 404, HTTP 500, a timeout, headers followed by a stalled body, a refused connection; code 301); • content failures (an empty 200 response, a document that is not well-formed XML, an HTML page, XML whose root is not VAST; 100, 101 or 303); • chain failures (a document with no ad, one with only a document-level Error element, a Wrapper with no VASTAdTagURI; 303, or 101, 100 or 301 for the last). 12

With seven baselines that check that chains of one to six documents resolve and that a chain of six Wrappers is accepted or rejected with 302, that gives 15 × 12 + 7 = 187 cases per client. The grid runs at the Node.js layer for the four Node.js clients (748 cells). iab-vast-loader has no tracker, so the code it determines is graded instead of its requests, and vastlint-client determines no code, so its cells are delegated unless it resolves an ad it should not. Macro grid. The macro grid places each of the 56 macros (of the list’s 59) whose contexts include all tracking pixels, VAST request URIs or both in those contexts. An InLine carries one Impression for each of the 45 pixel macros (40 of them in the 4.3 set). A Wrapper’s VASTAdTagURI carries every one of the 50 request macros (41 in the 4.3 set), once inside a CDATA section with literal ampersands and once as escaped text with &, two forms that are equally valid XML and must produce the same request. Each value a client sends is classified as left literally, empty or absent, −1 or −2, or a value. A literal or empty value deviates, because §6.1 makes the requesting party responsible for replacement and requires −1 or −2 for a listed macro it does not fill. Of the other three macros, [ERRORCODE] is graded by E04 and the error grid, [REASON] by A02, and [CLICKPOS], which applies only to click-tracking URLs, is not tested. The grid runs at the Node.js layer for the same four clients (580 cells). Mutation fuzzing. The seeds are 67 of the 69 crafted vectors (the other two, a_verification_ inline and m_optional_noflv, are single-change variants of seeded vectors), each with all its documents, and the 72 IAB sample documents of VAST 1.0 to 4.2 (139 seeds). Each case takes one seed, applies one to three random mutations to one of its documents, serves the result, and runs every Node.js client on it with the suite’s scripts. Mutations are of three kinds: • structural (deleting, duplicating, moving or copying an element; changing an attribute; replacing text; adding a comment; changing the case of an element name); • textual (truncation, a byte order mark, a changed encoding declaration, entity expansion, a namespace prefix, an inserted character, removed whitespace, a changed version attribute); • network (an HTTP status, a delay, a stalled body, compression or a changed content type on a Wrapper’s target). The oracle records only robustness failures, of seven kinds: • a process that dies or prints no result; • a run over the 30 s wall limit; • an error thrown outside any promise; • an unhandled rejection; • an exception from the client’s documented API; • a load that takes more than three times the client’s timeout plus 5 s; • an internal JavaScript error returned as the client’s error. A failure class is the client, the failure kind and the normalized message. Every class was reproduced three times from its first example, attributed to a source line by a separate script that calls the library directly, and reduced to a minimal document. It was then merged with the other classes of its cause into a root cause, labeled C1 to C9 (Appendix D). A differential pass groups cases by which clients resolved an ad, separates disagreements that a mutation induced from those its seed already showed, and probes the induced groups with single-change documents; its findings are labeled D1 to D4. The campaign drew from a single seeded pseudorandom stream in rounds of 300 13

draws, until a round added no new class, with at least 4 rounds. A draw whose mutations cannot apply (a network mutation on a document with no Wrapper target) is skipped. The campaign’s seed (20261017) regenerates all 1,442 cases in order, and a replay mode reruns any stored case.

4.5 Version sweep The sweep grades 93 requirements: the Node.js catalog except A04, A05, P07, X17, X18 and X19 (the nine browser-only requirements run as n/a), with A01, A02, R04, R05 and M06 to M09 on their Wrapper-first or FLV-first vector variants, on the last patch release of each minor release. The swept releases are vast-client-js 3.3.0 to 6.5.0 (9 releases), video-ad-sdk 2.1.0 to 4.0.1 (9), iab-vast-loader 2.0.0 to 2.5.1 (6, in both configurations) and vastlint-client 0.4.24 to 0.15.0 (12), 36 releases in all. Every verdict change between two releases was checked against a source diff of the two installed releases and, where available, the release notes. The sweep bounds each change between two swept releases; it does not cover the browser-only requirements, rmp-vast, Shaka or IMA. Old releases ran with dependencies resolved in October 2026.

4.6 Validity checks Online Resource 2 documents each check in full. Five checks rest on deterministic evidence that a reader can verify (Table 4). Three further checks were automated consistency audits: separate automated reviewers, each given only the inputs listed in Table 4b and none of the author’s reasoning, reviewed the catalog, re-derived the open-source deviations from client code, and re-derived IMA’s from logged requests, public API events and the archived SDK files. These reviewers were separate sessions of the generative AI tool named in the Use of generative AI statement (Statements and Declarations), each started with only those inputs; the author checked and resolved their outputs. Table 4b reports what they found, all of which the reported results address. These audits catch errors; they are not a measure of inter-rater reliability, and no human second rater has reviewed the catalog or the verdicts (Section 7). The verdicts rest on the evidence of Table 4: Appendix A gives the specification basis of every requirement and Online Resource 5 quotes the sentence behind it, Appendix B traces every open-source finding to a source location in the tested release, and every robustness defect was reproduced by a separate script outside the harness. Table 4. Validity checks. Check

Inputs

Scope

Outcome

Source tracing

client source of each tested release a script per defect, outside the harness each script’s request log against each library repeated runs on two platforms the 49 IAB VAST 4.x sample tags

every deviation of an open-source client every robustness root cause every script and library

every finding located (Appendix B) each reproduced 3 of 3 times (Appendix D) 8 harness issues resolved

Node.js and browser matrices, grids the five open-source clients that load tags

identical verdicts except IMA’s I01 (Section 5.2) each resolves every sample except those its findings predict (iab-vast-loader: 12)

Standalone reproduction Driver audits Determinism Access check

14

Table 4b. Automated consistency audits (separate automated reviewers, isolated inputs). Audit

Inputs

Scope

Catalog review

specification, vectors, every requirement in the oracles, drivers; no results catalog at the time of the review Re-derivation, two clients installed source and 3 deviations of neutral questions; no video-ad-sdk and findings vast-client-js Re-derivation, three result matrices of the 46 deviating cells of clients three clients; open-source vastlint-client, rmp-vast client code and and IMA, and 2 claims standalone scripts; IMA request logs, public API events and archived SDK files; none of the author’s reasoning

Outcome 8 verdict-affecting and 16 wording issues, all resolved 3 of 3 confirmed with its own scripts 40 cells confirmed, 4 harness issues resolved, 2 borderline (reported as such); both claims confirmed

The three-client re-derivation did not cover vastlint-client W23, X17, X18 and X19, rmp-vast P07, IMA W17, A05 and I01, or rmp-vast’s omidSupport configuration. In the access check, vast-client-js, video-ad-sdk and vastlint-client resolve an InLine ad from all 49 IAB samples, and rmp-vast starts an ad for all 49 (46 play to completion; the other three are overlays, which have no completion event). iab-vast-loader resolves 37 with defaults and 43 with noSingleAdPods: the six it rejects in both configurations repeat UniversalAdId or ClosedCaptionFile, and the six it rejects only with defaults are the Wrapper samples (Section 5.1.3).

5. Results 5.1 RQ1: Conformance 5.1.1 Overview Table 5 gives the verdicts and the number of findings per configuration, and Figure 3 gives the deviating cells over the cells each client takes on, by duty group. Counts of conforming cells are not comparable between clients that take on different shares of the duties. Within a duty group, each rate in Figure 3 is a conditional failure fraction: the share of the cells a client takes on that deviate. Clients take on different cells, so the rates are not a ranking of client quality. Which cells a client takes on follows each library’s documentation (Table 1): a cell is delegated where that documentation leaves the duty to the integrator. Cells are not independent: one finding can produce several deviating cells (video-ad-sdk’s 16 must-level cells in Chromium come from 12 findings), so Table 5 reports both. Appendix C lists every deviating cell of the open-source configurations of Table 5 except Shaka 5.3.0, whose cells are in Online Resource 3.

15

Table 5. Verdicts per configuration (108 requirements), Node.js layer and Chromium 141, and findings per client (Appendix B). Each row sums to 108. The must, should and obs. columns count deviating cells at must level, at should level and on robustness observations (R01 for vast-client-js and rmp-vast). Google IMA is the closed comparison. Shaka Player, rmp-vast and IMA run in the browser layer only, vastlint-client in the Node.js layer only. Shaka Player 5.3.0, released on the day of the reference runs, was run in Chromium with the same adapter, and an independent repeat agrees on every cell graded from the same vectors (Online Resource 3); it is not in Figure 3 or Appendix B. The nine browser-only requirements are n/a in the Node.js layer, which is why delegated and n/a counts differ between layers. Configuration vast-client-js 6.5.0

Layer

Node.js Chromium video-ad-sdk 4.0.1 Node.js Chromium iab-vast-loader 2.5.1, defaults Node.js Chromium iab-vast-loader 2.5.1, Node.js noSingleAdPods Chromium vastlint-client 0.15.0 Node.js Shaka Player 5.2.12 Chromium Shaka Player 5.3.0 Chromium rmp-vast 17.2.0, defaults Chromium rmp-vast 17.2.0, Chromium omidSupport Google IMA HTML5 SDK Chromium 3.794.0

conforms

must

should

obs.

delegated

n/a

findings

79 79 68 71 23 24 24

3 3 13 16 6 5 5

2 2 8 9 1 1 1

1 1 0 0 0 0 0

12 21 0 0 68 77 68

11 2 19 12 10 1 10

11

25 50 35 44 80 84

4 11 21 15 12 11

1 9 8 7 9 8

0 0 0 0 1 1

77 22 0 0 0 0

1 16 44 42 6 4

9 15 15

93

4

4

0

0

7

7

19 9

A client’s findings include robustness and macro-grid findings outside the cells of Table 5. IMA’s count is its Chromium findings; Firefox adds one (8 across browsers). rmp-vast’s count is its own findings; its cells that come from the bundled vast-client-js are counted under vast-client-js (rule G4). 5.1.2 What every client gets right Every client that follows Wrappers terminates a Wrapper loop and follows HTTP redirects, compressed responses and text/plain responses. Every client that determines error codes gives 303 for a no-ad response behind a Wrapper. Every client accepts a byte order mark and VAST 2.0 and 3.0 documents, and all but iab-vast-loader accept repeated UniversalAdId elements. Every client with a tracker requests all Impression URIs, fires start, the three quartile events and complete once each and in order, fires the pause, mute, unmute and skip trackers, and fires the impression once across user actions. Each also stops at 60 percent without thirdQuartile or complete (Shaka has no API to stop an ad and is not graded), ignores an unknown event name, and keeps tracking after a tracking URL answers HTTP 500. Each except vastlint-client, which leaves skip availability to the integrator, offers the skip control only after a time skipoffset. Those with a tracker that follow Wrappers also fire every Wrapper’s Impression, tracking and click-tracking URLs. In Figure 3, XML and parsing duties are wrong mainly in the two clients that parse XML their own way (iab-vast-loader, through iab-vast-parser, and vastlint-client).

16

iab-vastloader

vastlintclient

defaults

2/8

1/13

Impressions and tracking

rmp-vast

Google IMA

Node.js

Shaka Player

defaults

closed

2/5

5/8

2/2

4/8

4/8

6/17

0/6

1/3

6/8

2/18

0/18

0/25

5/27

–

5/23

8/20

4/27

1/25

Macros

3/12

5/9

1/1

4/9

6/7

6/12

0/12

Ad pods

1/2

3/3

1/2

0/2

3/5

2/6

0/6

XML and parsing

0/17

1/17

2/9

5/16

0/13

0/17

0/16

Media selection

–

0/2

–

–

1/2

0/1

0/1

Verification and icons

0/1

3/5

–

0/1

3/3

3/5

3/7

Robustness observations

1/8

0/8

0/7

0/8

0/4

1/8

0/8

vastclient-js

video-adsdk

Wrapper resolution

0/7

Error reporting

–

Rate (deviating / taken on): 0

50%

100%

no cell taken on k/n = deviating / taken on

Figure 3. Deviating cells over cells taken on, by duty group and client (Chromium; vastlint-client under Node.js): a conditional failure fraction on the cells each client takes on, not a ranking of client quality. Color depth is the rate, and a gray cell means none of the cells taken on deviates; each cell shows k/n. A hatched cell with a dash means the client takes on no cell of that group. IMA is a closed comparison.

5.1.3 Where the clients deviate Five findings matter most. Every tested Wrapper-following open-source client leaves the required macros in a Wrapper’s request unreplaced (M12, Macros). vast-client-js’s request timeout can end a Node.js process (R01, Robustness). Shaka credits a finishing pod ad’s completion to the next ad (P03, Ad pods), and video-ad-sdk sends wrong error codes along Wrapper chains (W07, E03, Error reporting). Whether some clients report a malformed Wrapper target correctly depends on the runtime’s XML parser (W11, Section 5.2). The paragraphs below take the duty groups in the order of the harm they do to the companies that count on the requests, naming the clients that deviate and the mechanism behind the main findings; Appendix B lists every finding with its source location and release range. Macros. None of the five Wrapper-following open-source clients replaces [CACHEBUSTING] or [TIMESTAMP] in a Wrapper’s VASTAdTagURI (M12, must). The macro list gives “VAST request URIs” as a context for both required macros, and §6.1 places the replacement on the party that makes the request, which is the player. vast-client-js only resolves relative URLs before the request; video-ad-sdk, iab-vast-loader and vastlint-client request the URI as written; rmp-vast expands macros only in the root tag it passes to vast-client-js. A downstream ad server sees literal brackets where it expected a cache buster and a time, so it cannot use the macros to tell a fresh request from a cached or repeated one. The macro grid extends the finding to every request macro in the list (Section 5.3), and the version sweep finds it in every swept release of the four swept clients (Section 5.4). The archived IMA 3.794.0 configuration tested fills both macros (Section 5.1.5), so the duty is implementable. vast-client-js fills or marks as −1 every macro of the 4.3 set, but leaves [PLAYBACKMETHODS], which VAST 4.3 mentions and the list marks 4.x, literal (Section 5.3). video-ad-sdk and vastlint-client fill only CACHEBUSTING and TIMESTAMP and send [LIMITADTRACKING] and the other listed macros literally (M03, M06); Shaka 5.2.12 17

replaces no macro (M01 to M03, M06, M11); and rmp-vast skips replacement on Error URLs (M11). Two clients get the required format wrong: video-ad-sdk’s cache buster has 1 to 10 digits and rmp-vast’s is eight alphanumeric characters, where the list specifies an 8-digit integer (M01). Error reporting. Error requests are how an ad server learns why its ad did not play, and the clients that determine codes often choose the wrong one. video-ad-sdk reports a failed Wrapper fetch as 502, a code for NonLinear (overlay) ads, instead of 301 (W07), at every depth and level of the error grid. It reports a missing playable media file as 401 instead of 403, to the InLine and to every Wrapper (E03, E09). It also reports a media file that answers 404 or cannot be decoded with the code undefined, followed 3 s later by a timeout code (E05, E06), because its ad unit sets a code only for JavaScript Error objects. Shaka never requests a no-ad response’s document-level Error URIs, sends a literal [ERRORCODE] for every media failure, after the impression has already fired, and never reports a media file that does not answer (E01 to E03, E05 to E07). rmp-vast maps an undecodable file to 401 instead of 405 (E06). A Wrapper target that is not well-formed XML must be reported with 100 (W11). vast-client-js reports a malformed Wrapper target as 301 or 303, never 100, and whether video-ad-sdk and iab-vast-loader conform depends on the runtime (Section 5.2). vastlint-client determines no code; its error cells are delegated. Ad pods. Shaka credits each pod ad’s completion request to the next ad: when an ad ends, its complete() first makes the next ad current and only then fires the complete event, whose URLs come from the current ad (P03). This is the case of Section 1. video-ad-sdk sorts pod ads with a comparator that returns 0 for every pair, so a pod in document order 3, 1, 2 plays the ad with sequence 3 first, in the root document and behind a Wrapper alike (P01, P07). vast-client-js sorts a pod in the root document but not one behind a Wrapper, because its nested parse skips the sort (P07), and rmp-vast inherits that. iab-vast-loader checks only the ads outside a pod after each Wrapper, so it reports a pod behind a Wrapper as 303, a document with no ad (P07, the cause of its W02 deviation); Shaka plays pods in document order (P01). video-ad-sdk plays only the first ad of a pod (P06), and rmp-vast and Shaka do not substitute the stand-alone ad when a pod ad fails, or do so only after the pod ends (P05). Shaka 5.3.0, released on the day of the reference runs, fixes the completion crediting (P03) and seven macro and tracking deviations (M01 to M03, M06, M11, T10, T23), percent-encodes macro values and fills [ADPLAYHEAD] (M04, M08, n/a in 5.2.12 because no macro was replaced), and adds one regression: unknown macros are replaced with −1 (M05). It still sends [ADSERVINGID] as −1 (M07). Its row in Table 5 shows the net change; the rest of the tables report 5.2.12, the release graded in every browser and on both platforms, and Online Resource 3 gives every 5.3.0 cell. Impressions and tracking. The deviations are in progress events, user actions and click handling. In the tested setup, progress events at a percentage offset never fire in video-ad-sdk (T07). The vector gives the ad’s Duration as 00:00:10, so a percentage offset, which §3.14.3 defines against the ad’s progress, has a defined time, and the media element reports the same duration once the media is loaded. The library converts each percentage offset to milliseconds once, from the media element’s duration before that duration is set, so every offset becomes NaN. vastlint-client’s documented controller fires quartiles but no progress, loaded or rewind events (T05 to T07, T20, T21). Shaka fires no progress, rewind or loaded event, and rmp-vast loses every progress event in macOS WebKit (Section 5.2). rmp-vast fires the resume tracker at ad start, because it fires on every play event including the first (T13), and both rmp-vast and Shaka fire unmute on any volume change (T23). video-ad-sdk and IMA ignore a ClickThrough that only the Wrapper carries (W17, a VAST 4.2 duty), so a click on such an ad opens no landing page; IMA also sends no click-tracking request for it.

18

Wrapper resolution. Three clients reject a chain of five Wrappers, which VAST 4 players are required to accept (W03): rmp-vast, whose default of four redirects meets an off-by-one in vastclient-js’s limit check; vastlint-client, whose depth limit counts documents instead of Wrappers; and IMA, whose own error message gives its limit as four Wrappers. With its defaults, iab-vastloader never follows a lone Wrapper with sequence="1", which it files as a one-ad pod, and ends the chain with no ad and no error (W02); every IAB VAST 4.x Wrapper sample has that shape. Shaka ignores Wrappers altogether (W01, W02). The Wrapper attributes are honored unevenly: rmp-vast overrides allowMultipleAds with true, vastlint-client and iab-vast-loader never read followAdditionalWrappers, and video-ad-sdk defaults allowMultipleAds to true (W19 to W21). vastlint-client classifies a response that holds a Wrapper ad and a stand-alone InLine ad as an InLine document, so it never follows that Wrapper (W23). XML and parsing. iab-vast-parser rejects valid VAST 4.2 documents as a whole, with error 101, when a creative has two UniversalAdId elements or a MediaFiles element has two ClosedCaptionFile elements (X05, X06), and fuzzing found four more repeatable elements that cause the same rejection (differential finding D3). Its table of collections lists them as single. vastlint-client reads XML with regular expressions, so it requests an Impression that sits entirely inside an XML comment (X17), does not request an Impression whose element also holds an XML comment (X02), and requests tracking URLs inside Extensions (X09). It also truncates an entity-escaped VASTAdTagURI at the first & (X18, X19). video-ad-sdk keeps only the first text node of an element, so a URI split across CDATA sections is truncated (X11). Verification and icons. video-ad-sdk never reads AdVerifications, so a verification vendor whose code it does not run is never told (A01, A03). Shaka never parses AdVerifications (A01), and rmp-vast handles verification only with its omidSupport setting. When a Wrapper and its InLine each carry a vendor, IMA reports only the InLine’s, and it does not report the tested verification resource that answers 404 (A03, A05; Section 5.1.5). Icons with a display offset never appear in video-ad-sdk, which draws icons once before the duration is known, and appear at once and never leave in rmp-vast, which ignores offset and duration (I03); Shaka draws no icons (I01). IMA requests an icon’s view URL before the ad’s start tracking request (I01; Section 5.1.5). Media selection. Shaka sorts media files by height and plays the first without checking its type, so an unsupported file listed first fails the ad (S01). Robustness. Three defects threaten the process that runs the library instead of the counts. vast-client-js’s request timeout throws an exception that no caller can catch (R01). On timeout, its URL handler aborts the request and throws inside the timer callback, outside the request’s promise. Under Node.js, which the library supports, the exception ends the process with exit code 1. A reproduction outside the harness under Node.js 20, 22 and 24 crashed in 49 of 49 fault runs and none of 21 controls. With 50 concurrent requests and one hanging Wrapper, none of the 49 healthy requests completed, and an HTTP service that resolves one tag per request stopped accepting connections. A server that resolves VAST chains for server-side ad insertion can therefore be stopped by one slow Wrapper; in a browser the same error is an uncaught page error, and rmp-vast inherits it. The same library clears its timer when the response headers arrive and then reads the body with no timeout, so a Wrapper target that stalls its body makes the client wait at every hop (fuzzing root cause C4). With a 2 s timeout and a 6 s stall, the ad arrived after the stall ended, and a self-referencing Wrapper ran nine stalled hops in 54 s. video-ad-sdk discards the promise of an ad whose MP4 file is marked as VPAID, producing an unhandled rejection that ends a Node.js process without a handler, and then reports an error and an impression for the same play (C3). The other robustness findings return internal TypeErrors where a VAST error code belongs.

19

vast-client-js does so on an ad element with neither InLine nor Wrapper, which also loses a valid sibling ad (C2), and iab-vast-loader on empty, non-XML and non-VAST responses (C5 to C7) and on a lower-case element name (C8). 5.1.4 Integration and delegation Three libraries take on only part of a player’s work, and the split decides what their verdicts mean. Where vast-client-js and vastlint-client leave a duty to the integrator, the harness performed it as their documentation describes, so their conforming cells show that correct behavior is possible through the documented API. A real integration can do worse: rmp-vast, a player built on vastclient-js, gives up two behaviors. It overrides allowMultipleAds (W20, W21), which vast-client-js honors through its documented API, and it never falls back to the stand-alone ads that vast-client-js keeps in reserve for its integrator (W23). In vastlint-client, a duty can be delegated and still be impossible to carry out. After a fetch failure in the middle of a chain, its session snapshot keeps only the root, so an integrator cannot reach the Error URIs of the intermediate Wrappers that the integrator is supposed to request. Its cells stay delegated, because the harness should not invent the code the library leaves open, but an integrator limited to the documented post-failure snapshot cannot conform on them. 5.1.5 A closed comparison: Google IMA HTML5 SDK 3.794.0 IMA is Google’s client-side ad SDK for web publishers, so it serves as a commercial reference point. It is proprietary, runs in browsers only and loads server-assigned configuration, so two live runs can load different code. Every file the SDK loaded while playing the vectors was recorded once from Google’s servers (Chromium, Linux, October 16, 2026), and every later run was answered from that archive, with Google’s telemetry answered locally. IMA runs against an HTTPS copy of the recording server, because on a plain-HTTP page it raises AdError 1005 without requesting the tag (Online Resource 4). Its results describe this one archived configuration. Unlike the open-source clients, IMA is analyzed as a black box: its verdicts and findings rest only on the requests the server logged, the events and error codes of its public API, and three probe vectors that isolate a behavior; no explanation in this paper is drawn from its code. Table 5 gives its Chromium verdicts; Linux WebKit gives the same deviations, Firefox adds one (W11, below), and in the macOS run the icon cell I01 conforms in all four browsers, although repeated macOS WebKit runs deviate (Table 6). IMA fills [CACHEBUSTING] and [TIMESTAMP] in a Wrapper’s VASTAdTagURI (M12) and sorts a pod behind a Wrapper (P07), which vast-client-js, rmp-vast, video-ad-sdk and iab-vast-loader do not. Its 8 deviating cells in Chromium come from 7 findings, each visible on the wire; Online Resource 4 gives the requests, API events and probes behind each. It rejects a chain of five Wrappers after requesting all five Error URIs with code 302, and its error message gives four Wrappers as its limit (W03). When only the Wrapper carries a ClickThrough, a click on the ad sends no click-tracking request and opens no landing page (W17), a VAST 4.2 duty outside the 4.2 features Google lists as supported. It rejects a Wrapper target that holds a pod and a stand-alone ad with its own error code 2, which is not a VAST code, and after a Wrapper that leads to no ad it reports 303 but does not fall back to another ad (W20, W21, W23). It reports the InLine’s verification vendor but not the Wrapper’s, sending at most one verificationNotExecuted request per ad in these probes, and does not report the tested verification resource that answers 404 (A03, A05). In every run on every engine and platform it requests an icon’s view URL before the ad’s start request (I01). On Linux the gap exceeds the 50 ms tolerance in all 20 repeated runs in Firefox (54 to 111 ms) and 20

in WebKit (165 to 237 ms); in Chromium it straddles the tolerance (36 to 68 ms, beyond it in 6 of 20 repeated runs and in both reference runs, 90 and 122 ms), so the Chromium verdict in Table 5 is the verdict of the reference runs and depends on the run. On macOS, in 10 repeated runs per browser, the view preceded start by 11 to 12 ms in Chromium, 10 to 13 ms in Chrome and 13 to 50 ms in Firefox (never beyond the tolerance) and by 51 to 58 ms in WebKit (beyond it in all 10) (Table 6). In Firefox it reports 303 for a malformed Wrapper target, where the other browsers give 100 (W11). 5.1.6 Sensitivity to strength labels M12, like M01 to M03 and M11, rests on the macro list’s Required label and on §6.1’s responsibility sentence rather than on a modal verb in the 4.3 text. P06’s should sits beside a conflicting must in the Ad Pods implementation note (Appendix A). P01 and P07 rest on a literal must. Each configuration is counted once here: the seven open-source configurations in Chromium and vastlint-client under Node.js. Their must-level deviations total 83. If M12 were graded should, the total would fall to 76: vast-client-js would keep 2 must-level deviations (P07, W11), iab-vast-loader 3 or 4, and every other configuration 10 or more. If M12, P01 and P07 were all graded should, the total would fall to 68, and vast-client-js would keep 1 (W11). If every macro requirement that rests on the list’s Required label were graded should (M01 to M03, M11, M12), the total would fall to 63. If P06 were graded must, following the implementation note, only video-ad-sdk would change (17 must in Chromium). M03 accepts -1 or -2 for [LIMITADTRACKING] because a browser has no device advertising ID, although §6.1 reserves those values for optional macros; graded strictly to 0 or 1, M03 would add one must-level deviation and one finding each to vast-client-js (total 84) and IMA (5), and one to Shaka 5.3.0. IMA otherwise keeps 4 in every scenario. No other reading adds or removes a finding. The order of the configurations by must-level deviations does not change when only M12 is relabeled; under the other two readings it changes only between rmp-vast and vastlint-client, which stay within one and two deviations of each other. Online Resource 5 gives the counts per configuration. Answer to RQ1. The open-source clients handle the common path well and diverge where other companies count and diagnose ads: macros in redirect requests, error codes, pod order and pod tracking, progress events and verification. Every tested Wrapperfollowing open-source client fails to replace the required macros in a Wrapper’s request; the archived IMA 3.794.0 configuration tested replaces them and deviates on 8 of the 101 cells it takes on in Chromium (8 percent), from 7 findings, and on 9 of 102 in Firefox, from 8 findings across browsers. The full players carry the most findings (video-ad-sdk 19, rmp-vast 15, Shaka 15) and deviate on 19 to 45 percent of the cells they take on in Chromium (from 20 of 104 for rmp-vast with omidSupport to 29 of 64 for Shaka; video-ad-sdk 25 of 96). The libraries that take on only part of a player’s work deviate on 7 to 29 percent of the cells they take on (vast-client-js 6 of 85, iab-vast-loader 6 of 30, vastlint-client 20 of 70) and leave duties that integrators can get wrong, as rmp-vast shows. The most serious defects are outside the specification: an uncaught timeout exception that ends a Node.js process, and a timeout that a stalled response body defeats.

21

5.2 RQ2: Runtime and platform Most verdicts do not depend on the runtime. One of the two Linux WebKit runs graded nine requirements (A01, A02, A04, R04, R05, M06 to M09) on different vectors, so its comparisons leave those nine out; the macOS run used the released package and is compared on every cell. Repeated runs were identical except for IMA’s I01 (Section 4.3). They covered the Node.js matrix (540 cells) in four runs on two platforms, the full Chromium matrix (864 cells) in two runs, WebKit on two Linux machines (792 cells), and the error grid (748 cells) and macro grid (580 cells) on both platforms. Between Node.js and the browsers, the four configurations that run in both layers agree in 393 of the 396 cells graded in both in Chromium, Chrome, macOS WebKit and Safari. They agree in all 396 in Firefox and in 392 of 396 in Linux WebKit. Between Linux and macOS, the open-source configurations agree in every cell in Chromium and Firefox and differ in 9 cells in WebKit; IMA’s icon cell I01 differs in all three engines (Table 6). Of their 756 cells, the seven open-source browser configurations differ from Chromium in 5 in Firefox and 3 in Linux WebKit; against macOS Chromium in the same Mac run, they differ in 2 of 756 in Chrome and in Safari and in 10 in macOS WebKit. Every difference is in Table 6 and has one of four causes: the runtime’s XML parser (W11), the media support of the browser build (S01 in Linux WebKit; S02, through the Playwright Chromium build’s missing AAC decoder), engine timing (T05 to T07, T19) and run-to-run timing (IMA’s I01). The XML parser matters most. Table 6. Verdicts that differ between layers or browsers (open-source clients, and IMA’s one timing-dependent cell; codes in parentheses; ms: icon view before start; repeats: runs beyond the 50 ms tolerance among repeated runs of the cell alone). Node.js (jsdom)

Blink: Chromium, Chrome

Gecko: Firefox

deviates (ad accepted) conforms (100) conforms

conforms (100) deviates (101) conforms

conforms

conforms

Req.

Client

W11 (must)

video-ad-sdk

W11 (must)

iab-vast-loader (both) rmp-vast (both)

conforms (100) deviates (ad accepted) not run

video-ad-sdk

conforms

T05 to T07 (must), T19 (should) S01 (should)

22

WebKit (Linux / macOS) deviates / deviates conforms / conforms conforms / deviates (T19 n/a) deviates (FLV chosen) / conforms

Safari deviates conforms conforms

conforms

Table 6 (continued)

Req.

Client

Node.js (jsdom)

S02 (may)

rmp-vast (both)

not run

I01 (must)

Google IMA

not run

Blink: Chromium, Chrome

Gecko: Firefox

Chromium: conforms n/a (401); Chrome: conforms Linux: Chromium, deviates (88 Linux: deviates (122, ms; repeats 90 ms; 20/20, 54 to repeats 6/20, 111); macOS: conforms (13 36 to 68); Chromium, ms; repeats 0/10, 13 to macOS: 50) conforms (9 ms; repeats 0/10, 11 to 12); Chrome: conforms (10 ms; repeats 0/10, 10 to 13)

WebKit (Linux / macOS) conforms / conforms

Safari conforms

deviates (367 not run ms; repeats 20/20, 165 to 237) / conforms (27 ms; repeats 10/10, 51 to 58)

The XML parser (W11). A Wrapper whose target is not well-formed XML must be rejected with error 100. For a given library version, the verdict follows the XML parser the library uses in that runtime, and there are three parser behaviors. • Node.js. jsdom’s DOMParser, like Gecko’s, returns a document whose root element is parsererror; video-ad-sdk detects it and reports 100. iab-vast-parser does not use it under Node.js: it parses with xmldom, which recovers a partial tree, and the loader returns an ad. • Blink and WebKit. The browser’s DOMParser returns a document that contains a parsererror element alongside the parts it could read. iab-vast-parser checks for that element and reports 100; video-ad-sdk does not check, takes the partial tree as a VAST response, accepts an ad from it, and reports 401 when it finds no media. • Gecko. The DOMParser returns a document whose root element is parsererror. video-ad-sdk then finds no VAST content and reports 100; iab-vast-parser sees a document that is not VAST and reports 101. vast-client-js and rmp-vast deviate in every runtime, with a code that changes (301 under Node.js and in Firefox, 303 elsewhere), and vastlint-client, which uses no XML parser, accepts the ad. Media support (S01, S02). The S01 vector lists an FLV media file before WebM and MP4 files. The Linux WebKit build reports FLV as playable, so video-ad-sdk chooses it, fails to play it and reports undefined and then 402; S01 deviates. Every other browser, including WebKit and Safari on macOS, lets video-ad-sdk skip the FLV file. The Playwright Chromium build has no AAC decoder, so rmp-vast reports 401 for the optional audio ad without starting it; Chrome, Firefox, WebKit and Safari play it.

23

Engine timing (T05 to T07, T19). rmp-vast fires no progress tracker in WebKit on macOS and fires them in every other browser, Safari included. It builds its list of progress offsets once, on the first durationchange event, and only if the duration is finite. HTML allows a first durationchange with a duration of NaN or Infinity, and the list then stays empty for the whole ad. Dispatching one early durationchange in Chromium reproduces the pattern in 2 of 2 runs, with 2 of 2 controls unaffected. The engine behavior that triggers it is inferred, because the harness does not log the ad element’s duration events, and it is reported as an rmp-vast defect that one engine exposes. Answer to RQ2. The XML parser of the runtime decides one requirement. On it, video-ad-sdk conforms under jsdom and in Firefox and deviates in every tested Blink and WebKit browser, and iab-vast-loader does the reverse, so a library’s own jsdom-based test suite cannot see the defect its users meet. Media codecs and engine timing decide a handful of other cells. Everything else is stable: verdicts are identical across repeated runs (except IMA’s I01), between Linux and macOS for the open-source configurations in Chromium and in Firefox (WebKit differs in 9 cells, and IMA’s icon timing in all three engines, Table 6), and between Node.js and the browsers in 392 to 396 of 396 cells.

5.3 RQ3: Beyond the catalog Macro grid. Table 7 gives the results. Table 7. Macro grid (number of macros). “value”, “-1 or -2” and “literal” are what a client sends for the 40 tracking-pixel macros of the 4.3 set. “4.x” counts the 5 pixel macros marked 4.x (STOREID, STOREURL, PLAYBACKMETHODS, GPPSTRING, GPPSECTIONID) left literal; “requests” counts the request-URI macros (50, 41 in the 4.3 set) the client replaces. Online Resource 5 has the full grid. Client

value

-1 or -2

literal

4.x

requests

vast-client-js 6.5.0 video-ad-sdk 4.0.1 iab-vast-loader 2.5.1 vastlint-client 0.15.0

6 2 delegated 2

34 0 delegated 0

0 38 delegated 38

5 5 delegated 5

0 / 50 0 / 50 0 / 50 0 / 50

No Node.js client replaces any request macro, of either set, in either XML form; vastlint-client’s escaped form drops 49 of the 50 macros, because it truncates the URI at the first &, and sends [TIMESTAMP] literally. In tracking pixels, vast-client-js fills six macros with values (TIMESTAMP, CACHEBUSTING, ADTYPE, UNIVERSALADID, ASSETURI, ADSERVINGID), marks the other 34 of the 4.3 set as −1, and leaves only the five pixel macros marked 4.x literal, among them [PLAYBACKMETHODS], which the VAST 4.3 text mentions. video-ad-sdk and vastlint-client fill CACHEBUSTING and TIMESTAMP and leave the other 38 of the 4.3 set literal. Error grid. Table 8 gives the results. For the four Node.js clients, each client gives the same code at every depth and level for a given failure, with two exceptions, vast-client-js on a stalled document and video-ad-sdk on a missing VASTAdTagURI, described below. Separately, the grid’s malformed document (a truncated file) differs from the catalog’s (a mismatched end tag), and on it iab-vast-loader reports 303 instead of accepting an ad and vastlint-client resolves no ad. vast-client-js accepts the ad when the stalled document is the InLine (the stalled-body defect, C4) and reports 301 when it is a Wrapper. videoad-sdk reports 502 for a missing VASTAdTagURI except at the sixth level, where it reports 302.

24

For a Wrapper without a VASTAdTagURI, vast-client-js reports 303 to the Wrappers above it and nothing to that Wrapper itself, at every level. Table 8. Error grid: code and verdict per client, by failure (15 cells per failure per client, 7 baselines; 187 cases and 748 cells in all). d is the number of documents in the chain: d − 1 Wrappers and the InLine. A cell with two verdicts differs by depth or level.

Failure

Accepted codes

vast-client-js

video-ad-sdk

iab-vast-loader (code determined)

(none)

resolve d ≤ 6; d = 7 may 302

conforms

conforms

conforms

http404

301

301; conforms

502; wrong code

301; conforms

http500

301

301; conforms

502; wrong code

301; conforms

timeout

301

301; conforms

301; conforms

stall

301

301; conforms

301; conforms

refused

301

301; conforms; uncaught exception ad accepted or 301; deviates or conforms 301; conforms

502; wrong code

301; conforms

empty

100/303

301; wrong code

html

100/303

301; wrong code

malformed

100

301; wrong code

novast

101/100/303

301; wrong code

noad

303

303; conforms

noad_err

303

303; conforms

nouri

101/100/301/303 303; partial

vastlint-client deviates at d = 6; rejects d = 7 with no code delegated (no code) delegated (no code) delegated (no code) delegated (no code)

delegated (no code) 100; conforms none; no code; delegated (no internal error code) 100; conforms 101; wrong code; delegated (no internal error code) 100; conforms 303; wrong code delegated (no code) 303; conforms 101; conforms; delegated (no internal error code) 303; conforms 303; conforms delegated (no code) 303; conforms 303; conforms delegated (no code) 502 or 302; wrong 301; conforms delegated (no code code)

Mutation fuzzing. The campaign ran 5 rounds (289, 291, 290, 283 and 289 cases; 1,442 in all, of which 1,437 were unique) and found 12 failure classes from 9 root causes: 8 library defects and 1 environment artifact (Appendix D). By client: vast-client-js 3 (C1, C2, C4), video-ad-sdk 1 (C3), iab-vast-loader 4 (C5 to C8), vastlint-client 0. The artifact, C9, is an unhandled rejection in video-ad-sdk’s icon code that occurs only because Node.js 22’s own CustomEvent reaches jsdom’s event target; the affected icon cells are browser-only, so no verdict changes. Five of the 12 classes are raised by an unmutated seed as well; the mutations alone contributed C2, C4, C5 and C8. New classes per round were 8, 2, 1, 1 and 0, and new root causes 8, 1, 0, 0 and 0. The empty final round is weak evidence that classes are exhausted: 3 classes occur in 1 case each, none in exactly 2, and the bias-corrected Chao1 estimate (Chao 1984, 1987; Böhme 2018) is 15.0 classes (95 percent interval 12.4 to 37.3) against 12 observed. Classes are message strings, and one root 25

cause (C8) accounts for four of them, so root causes are the unit reported. The last new root cause appeared at unique case 360, and rounds 3 to 5 (858 cases not seen before) added none. Under this generator, a root cause with a per-case rate above 0.35 percent would have shown with 95 percent probability. The bound assumes independent cases and holds only for this generator’s seeds and operators (Section 7). Differential pass. The differential pass found four conformance differences the robustness oracle cannot see. They are malformed XML accepted by iab-vast-loader under Node.js and by vastlintclient (D1), 303 from vast-client-js for an InLine with no usable media file (D2), four more repeatable elements rejected by iab-vast-parser (D3), and the stalled body (D4, the same defect as C4). Answer to RQ3. For the error requirements, mostly. Across 187 generated chains per client, each client gives the same code at every depth and level for a given failure, with two level-dependent exceptions. For a malformed Wrapper target, iab-vast-loader and vastlint-client respond differently to the grid’s truncated file than to the catalog’s file with a mismatched end tag. The macro grid extends the Wrapper-macro finding from the two macros the catalog grades to all 41 request macros of the 4.3 set. Fuzzing found 8 library defects, 7 of them outside the catalog (C1 is R01, rediscovered): two end a Node.js process (C1, C3) and one defeats the integrator’s timeout (C4); the differential pass added four conformance differences. It gives no evidence that the space of such defects is exhausted.

5.4 RQ4: History Table 9 lists every verdict change between consecutive swept releases; the bullets explain the main changes and the deviations that never changed. Table 9. Version sweep: verdict changes between swept releases (Node.js layer, 93 requirements, last patch of each minor release). Client

Req. Between releases Verdict

vast-client-js vast-client-js vast-client-js vast-client-js video-ad-sdk video-ad-sdk video-ad-sdk video-ad-sdk video-ad-sdk video-ad-sdk video-ad-sdk

M09 P01 T02 R01 W04 W06 E02 T05 T06 M11 P01

3.3.0 and 4.0.1 6.1.0 and 6.2.0 4.0.1 and 5.0.0 5.0.0 and 6.0.2 2.3.0 and 2.4.3 2.3.0 and 2.4.3 2.3.0 and 2.4.3 2.3.0 and 2.4.3 2.3.0 and 2.4.3 2.5.1 and 2.6.3 2.7.2 and 3.0.0

conforms to deviates deviates to conforms deviates to conforms conforms to deviates deviates to conforms deviates to conforms deviates to conforms deviates to conforms deviates to conforms n/a to conforms conforms to deviates

• The Node.js crash dates from vast-client-js’s move to fetch. Release 6.0.0 replaced the Node.js URL handler, which aborted the request and passed a 408 error to its callback, with a fetch-based handler that throws inside the timer callback. Under plain Node.js, 6.0.0 to 6.0.2 cannot make any request (each fails on an undefined window) and 6.1.0 to 6.5.0 crash; with the harness’s browser globals, 6.0.2 throws as well. The release notes place the change at 6.0.0. 26

• vast-client-js 4.0.1 began sending every UniversalAdId. Release 3.3.0 fills [UNIVERSALADID] with the first ID only, which conforms because several IDs are optional; from 4.0.1 the library joins both IDs and percent-encodes the separating comma (M09). • Pod order was fixed in one library and broken in the other. vast-client-js sorts root pods from 6.2.0. video-ad-sdk sorted them through 2.7.2; its 3.0.0 rewrite introduced the comparator that always returns 0. • video-ad-sdk 2.4.x fixed 5 cells (the 302 Wrapper-limit code, every document-level Error URI and time-offset progress events). Releases up to 2.5.1 report nothing for unplayable media; the release notes place the change at 2.6.1, and the sweep bounds it between 2.5.1 and 2.6.3. • No change in iab-vast-loader or vastlint-client. iab-vast-loader’s verdicts are the same from 2.0.0 (2018) to 2.5.1 (2021), and vastlint-client’s JavaScript did not change across its 12 swept releases; only its validation core did. • Long-standing behavior. M12 deviates in every swept release of the four swept clients; vastclient-js reports 301 for a malformed Wrapper target in all 9 releases; video-ad-sdk’s 502 code, Wrapper ClickThrough, cache-buster length, LIMITADTRACKING and percentage-progress deviations are present in all 9. Four verdict changes in the sweep’s results are not in Table 9, because none is a change in library behavior. video-ad-sdk’s empty-Impression cell (X13) is a harness artifact: rerun on its own, it conforms before 3.0.0 as after. video-ad-sdk 2.1.0’s T18 could not be driven, because that release has no skip method, and video-ad-sdk’s T19 follows its T05 fix as a dependent cell. video-ad-sdk’s W13 becomes n/a at 2.6.3 because, from then on, it sends one consistent wrong code that E03 grades (rule G2); before, it requested no Error URI. Answer to RQ4. Most deviations are as old as the swept history: M12 is in every swept release of the four swept clients, and several of video-ad-sdk’s are in all 9 of its releases. Three changes made a client deviate: vast-client-js’s UniversalAdId encoding at 4.0.1, its Node.js crash at its move to fetch, and video-ad-sdk’s pod order at its 3.0.0 rewrite. Maintainers have fixed others (vast-client-js’s root pod order at 6.2.0, 5 video-ad-sdk cells at 2.4.x).

6. Discussion 6.1 What the deviations cost A completion request credited to the next ad in a pod leaves the pod’s total right and the per-ad counts wrong. An error coded undefined, or 502 for a failed fetch, sends an ad server’s diagnosis to the wrong cause. A Wrapper request with literal macros defeats the cache-busting that ad servers rely on to tell a fresh request from a cached one. A verification vendor whose code was never run receives no verificationNotExecuted request, so its data shows an ad that was not measured. The robustness defects threaten availability rather than counts: a server that resolves VAST can be stopped by one slow Wrapper or delayed past the ad break by a stalled one (Section 5.1.3). A plausible reason these deviations persist across releases (Section 5.4) is that most of them do not break playback. The explanation applies to the tracking defects: each produces a request that looks normal, or a missing request that nobody notices, and the party that would notice runs neither the library nor its tests. It does not apply to the deviations a viewer can see, such as a rejected chain of 27

five Wrappers (W03) or a valid document rejected as a whole (X05, X06). The study does not test this explanation, but it is consistent with the sweep and with the maintainers’ issue trackers, where several of these defects were reported years ago (Section 3). A conformance claim about a player built on one of these libraries depends on which duties its integrator wrote; the delegated verdict makes that split visible (Section 5.1.4).

6.2 Implications For library maintainers. The checks with the highest yield are cheap: • replace [CACHEBUSTING] and [TIMESTAMP] in every Wrapper request; • bound both the headers and the body of a request by the same timeout, and reject rather than throw when it expires; • sort pods after every Wrapper hop, not only at the root; • map each failure to its own code from the error table; • run the test suite in at least one Blink or WebKit browser as well as under jsdom, because the XML parser decides some verdicts. For integrators and player builders. A library that delegates a duty is conformant only together with the code that performs it. Before relying on one of these libraries, an integrator should know: • which duties it leaves open (Table 1); • whether those duties can be carried out through its API (vastlint-client’s failed chains show that they sometimes cannot); • which of its defaults change the outcome (rmp-vast’s redirect limit and allowMultipleAds override). For IAB Tech Lab. A published set of player vectors with expected behavior (which requests, with which codes, in which order) would let library maintainers test themselves; the 69 vectors and the two grids here are a starting point. A machine-checkable 4.3 schema would also help. The file schemas/vast_4.3.xsd in IAB Tech Lab’s VAST repository (commit 99d5d51) is empty, and the published schemas declare each content model as an ordered sequence, so a document whose InLine children follow the order in which §3.4 lists them fails validation. Two strength statements are hard to grade. The pod body text says a player should play as many ads as possible while its implementation note says all ads must be played, and the macros’ request-URI obligation is stated as a responsibility rather than a must. For researchers. Ad-tech clients, small independent implementations of one specification whose failures land on third parties, are a natural fit for specification-derived testing. Clients take on different duties and share code, so agreement is a poor oracle. The catalog, harness and grading rules can be reused for other ad formats and for HLS and DASH interstitial players.

6.3 Reporting to maintainers The vast-client-js timeout exception was public before this study: it had been reported in the project’s issue tracker as a browser error (8bitsten 2026), and while the study was under way another 28

contributor opened a public pull request that removes the throw (justinjeong5 2026). Neither addresses the stalled body, which was not public. On October 16, 2026, the author reported both defects and the Node.js consequence privately to the project’s security contact, offering a fix, and had received no reply by October 28. On October 28, 2026, the author opened a public pull request that complements the existing one by bounding the response body read with the same timeout, with tests (Chaudhary 2026), so the stalled body was disclosed together with a proposed fix; both pull requests were open on October 28. Under the patched build the crash and the stalled-body wait disappear in every reproduction, and the project’s test suite passes. The remaining findings concern conformance; the author will report them as public issues to each active project (vast-client-js, rmp-vast, vastlint-client, Shaka Player for the findings that still hold in its 5.3 line) after the preprint is posted, each first checked against the project’s latest release, and the replication repository’s README will list each issue.

7. Threats to validity Construct validity. The catalog operationalizes VAST 4.3 as 108 requirements extracted from its obligation lines and the macro list; another extraction could cut the duties differently or assign different strengths. The extraction inventory and coverage map are released, Appendix A gives the basis and literal modal of every requirement, one rule assigns strength, and the sensitivity analysis (Section 5.1.6) shows that no finding depends on the labels in dispute and that the order of clients by must-level deviations changes only between rmp-vast and vastlint-client. Deviations are not weighted by severity, so a must-level icon deviation counts like a must-level error code. Clients that claim older versions are still graded against 4.3 (Section 4.1). For libraries that delegate duties, a conforming cell measures whether correct behavior is possible through the documented API with the harness as a careful integrator. It is an upper bound on what a real integration achieves, and rmp-vast measures one real integration. Cells are not independent, so findings are reported alongside them (Section 5.1.1). Several couplings remain. One vector can serve several requirements, so a defect in that vector or in the client’s handling of it moves all of them (for example, the vector of M10 lists an FLV file first, which ties M10 to media selection). Some n/a verdicts depend on another outcome (G1, G2), so a client that fails early has fewer cells graded. The macro requirements depend on which macros the list marks Required and in which contexts, and on the list commit used (Section 4.2). How cells are grouped into findings is a judgment; a finding is one cause, but another analyst could split or merge some. Download counts measure use, not deployment in players, so they do not weigh the findings by the viewers they reach. Internal validity. The checks of Table 4 guard against reporting a harness artifact as library behavior, and every robustness defect was reproduced by a separate script outside the harness. The Node.js layer supplies browser globals through jsdom, which changes the code path of some releases (vast-client-js 6.x takes its browser branch) and produced one false failure, traced and excluded (C9). IMA runs over HTTPS with a browser security check disabled, while the other clients run over HTTP, and its telemetry is answered locally. If any IMA behavior depends on a telemetry response or server-assigned state, the archived run cannot show it. External validity. The tested clients are a purposive sample. They cover the three integration styles the survey found and include the library behind IAB Tech Lab’s tester and a player built on another client. However, four eligible small players and the derivatives of vast-client-js were not tested, and native mobile and TV SDKs are outside a JavaScript harness. Findings that trace to vast-client-js apply to players that bundle an affected release. The browser layer covers desktop 29

browsers on Linux and macOS with short synthetic clips from a local server; mobile WebViews, smart-TV browsers, adaptive streams and production network conditions are not covered. The clients keep moving: Shaka Player 5.3.0 changes several verdicts (Section 5.1.3). The releases current on 27 October 2026 give the tested verdicts: vastlint-client 0.15.3 and 0.16.2 match 0.15.0 in all 108 Node.js cells; Shaka Player 5.2.15 matches 5.2.12 in all 108 cells in Chromium, Firefox and WebKit; and Shaka Player 5.3.2 matches 5.3.0 in Chromium and shows the same 11 changes from 5.2.12 in Firefox and WebKit. The other clients had no later release. The fuzzer ran at the Node.js layer only, so Shaka, rmp-vast and IMA were not fuzzed; network mutations were 5 percent of the mutations applied; and the fuzzing bound holds only for this generator. The version sweep ran old releases with October 2026 dependencies, so their XML-handling verdicts reflect current parser versions. Reliability. Verdicts are deterministic: repeated runs and both platforms gave identical verdicts except for the reported engine differences (Section 5.2), and on a second machine and platform the packaged harness reproduced the Node.js and Chromium rows of Table 5 for every pinned open-source configuration (Shaka 5.3.0 was run separately), Tables 7 and 8 and the access check. The sweep and the fuzzing campaign were run once. The requirements, vectors and oracles were written by one author, who works in the industry these clients serve, and the catalog was built while outcomes were already visible, so the cut of duties and the strengths could have been influenced by them; the extraction inventory and the strength rule are released so that this can be checked. The timing thresholds (Section 4.3) are fixed choices, and a different tolerance could move a cell near its boundary. The consistency audits (Table 4b) were automated; they catch errors but are not a measure of inter-rater reliability. No human second rater has reviewed the catalog or the verdicts, and that is the main remaining gap.

8. Conclusion Open-source VAST clients get the common path right and diverge on the requests that ad servers and verification vendors rely on (RQ1). None of the tested open-source clients that follow Wrappers fills the required macros in a Wrapper’s request, while the archived configuration of Google’s closed IMA SDK 3.794.0 that was tested does. One library ends a Node.js process on a slow Wrapper and lets a stalled body outlast its timeout, and a player built on it inherits both; pods play out of order in several clients, and Shaka Player 5.2.12 credits a finishing pod ad’s completion to the next ad. A library can conform under jsdom and deviate in every tested Blink and WebKit browser because of the runtime’s XML parser, while almost every other verdict is stable across runs and platforms (RQ2). Generated grids reproduce the catalog’s error-code verdicts across depths and levels, with the exceptions of Section 5.3, and extend the macro finding to every request macro, and fuzzing finds robustness defects the specification does not name (RQ3). Most deviations are as old as the swept history (RQ4). Each open-source finding traces to a source location or to a missing feature, and the released harness can check a fix. Native SDKs, server-side insertion and the remaining small players are open.

Statements and Declarations Funding. No funding was received for this work. Competing interests. The author has no competing interests to declare. The author works in advertising technology and has no affiliation with the maintainers of the tested clients, IAB Tech 30

Lab or Google. Ethics. The study uses no human participants and no production data. All test traffic in the reported runs went to local servers. For IMA, the SDK files were recorded once from Google’s servers on October 16, 2026, and served from that local archive afterward, with Google’s telemetry hosts answered locally; Online Resource 4 documents the setup. IMA is graded only from its network requests and public API. The process-ending defect was disclosed privately to the maintainers before publication (Section 6.3). Data availability. The harness, oracles, fuzzer, packaged runner and other code (Apache-2.0) and the test vectors, synthetic test media, requirement catalog with its extraction inventory and coverage map, grids and all results (CC BY 4.0) are at https://github.com/methodtrace/2026-vas t-client-conformance (tag v1.0.0), archived at Zenodo (https://doi.org/10.5281/zenodo.23581408), following open-science practice for empirical software engineering (Méndez Fernández et al. 2019). Online Resources 1 to 5 are in that archive and can be browsed at https://methodtrace.github.io/ 2026-vast-client-conformance. One command installs a pinned toolchain, runs every layer and seals the results with hash manifests; on the author’s Mac it took 73 minutes. The IAB sample tags and specification text are third-party material and are fetched by script. The archived IMA SDK files are proprietary and are kept privately; the archive’s hashes and the recording script are released. The package runs every open-source client without them, and runs IMA only when the archive is supplied. Use of generative AI. During the preparation of this work the author used Claude Opus 5.5 (Anthropic) to create the test harness, trace deviations, run the automated consistency audits of Section 4.6, and help with drafting the text. After using this tool, the author reviewed and edited the content as needed and takes full responsibility for the content of this manuscript.

Appendix A. Requirement catalog Each row gives the requirement as graded, the VAST 4.3 section or macro list entry it rests on, and the obligation as the text writes it next to the strength graded. In the last column, “required” is the text’s own word; “list” means the IAB macro list marks the macro Required; “none” means a definition or attribute with no modal verb; “imper.” an instruction with no modal verb; “n/a” a robustness observation, which is not a VAST clause; “obs.” a requirement graded as an observation. (a) marks a graded strength that differs from the literal word; the rule of Section 4.2 decides it, and Online Resource 5 quotes the sentence behind every row. Table A1. Requirement catalog: requirement as graded, VAST 4.3 basis, and strength as written and as graded. ID

Requirement

VAST 4.3 basis

W01 Follows a Wrapper to its InLine ad. W02 Follows a lone Wrapper that carries sequence= "1" (a one-ad pod) to its InLine ad, or declines it with an error (pod support is optional). W03 Accepts a chain of five Wrappers before the InLine ad (players are required to accept five).

31

Written / graded

§2.2; §2.3.5 §2.2; The Ad Element (sequence); Ad Pods

must / must must / must

§3.19.1

required / must

Table A1 (continued) ID

Requirement

VAST 4.3 basis

Written / graded

W04 With six Wrappers the player may reject; if it §2.3.5.1; §3.19.1 should / does, it determines 302 (and, with a tracker, should reports it to the Error URI of every Wrapper it received). W05 Terminates a Wrapper that points to itself §2.3.5.1 none / should (stops requesting it). (a) W06 Reports 302 (Wrapper limit) for a §2.3.5.1; Error Reporting table should / self-referencing Wrapper loop. (302) should W07 When the VASTAdTagURI is unavailable Error Reporting (Media Player must / must (HTTP 404), determines error 301. Details; table 301) W08 When the VASTAdTagURI times out, Error Reporting (table 301) must / must determines error 301. W09 When a Wrapper leads to a no-ad response and Error Reporting (No Ad Response); should / the player determines a code, the code is 303. Error (VAST) should W10 Requests the VAST-level Error URI of a no-ad §3.2.1 Error (VAST); Error must / must response reached through a Wrapper (with 303 Reporting (No Ad Response) for [ERRORCODE]). W11 When the Wrapper target is not well-formed Error Reporting (table 100); §2.3.2 must / must XML, determines error 100 (and does not use an (well-formed XML) ad from it). W12 Requests the Wrapper Error URI when its Error Reporting (Media Player must / must VASTAdTagURI fails. Details) W13 An InLine error after a chain of Wrappers is Error Reporting (Media Player must / must reported to the InLine and to every Wrapper Details); §3.4.11 Error URI, with the same code (403 here). W14 At impression time, requests the Impression §3.4.4; §2.3.4 required / URIs of the InLine ad and of every Wrapper must before it. W15 Fires the Wrapper’s Linear tracking events §3.19 (tracking elements in required / (start, quartiles, complete) together with the Wrappers) must InLine ones. W16 When both the InLine and a Wrapper carry a §2.3.5.2 must / must ClickThrough, the InLine one is used. W17 When the InLine has no ClickThrough (no §2.3.5.2 must / must VideoClicks), the ClickThrough of the Wrapper closest to the InLine is used. W18 On a click, requests the ClickTracking URIs of §3.10.2 ClickTracking; §3.19 required / the InLine and of every Wrapper. must W19 With followAdditionalWrappers="false", a §3.19 followAdditionalWrappers should / Wrapper received next is ignored rather than should followed. W20 With allowMultipleAds="false", only the §3.19 (allowMultipleAds) none / should first stand-alone Ad of the response is used (pod (a) ads are dropped). W21 Without the attribute, allowMultipleAds defaults §3.19 (allowMultipleAds default none / should to false, so only the first stand-alone Ad is used. false) (a)

32

Table A1 (continued) ID

Requirement

W22 Optional: rejects an InLine whose Category is blocked upstream; a player that does reject it must report 205. W23 With fallbackOnNoAd="true", when the Wrapper’s target returns no ad, a stand-alone ad from the response is used. E01 For a no-ad response, requests the VAST-level Error URI. E02 Requests every VAST-level Error URI when several are given. E03 With no supported MediaFile, requests every InLine Error URI with 403. E04 Replaces [ERRORCODE] in Error URIs with a numeric code. E05 When the MediaFile URI returns HTTP 404, requests the InLine Error URI with 401 (file not found). E06 When a MediaFile of a supported type cannot be decoded, requests the InLine Error URI with 405. E07 When the MediaFile URI answers only after 30 s, requests the InLine Error URI with 402 (media timeout). E08 Does not request the Error URI when the ad plays without a problem. E09 Given a NonLinear-only ad, either displays it (impression or creativeView) or reports 200/201 (unexpected ad type or linearity). T01 Requests every Impression URI of the ad.

VAST 4.3 basis

Written / graded

§3.19.2 BlockedAdCategories

may / may

§3.19 (fallbackOnNoAd)

should / should

§3.2.1 Error (VAST)

must / must

§3.2.1 Error (VAST), bounded 0+

must / must

§3.4.11; Error Reporting (table must / must 403) Error Reporting (Media Player must / must Details); macro list ERRORCODE Error Reporting (table 401) must / must

Error Reporting (table 405)

must / must

Error Reporting (table 402)

must / must

§3.4.11 (Error is for ads that none / should cannot be served) (a) §2.3.1; Error Reporting (table 200, must / must 201)

required / must T02 Does not dispatch an Impression whose URI is §3.4.4 (about:blank) should / about:blank. should T03 Fires start, firstQuartile, midpoint, thirdQuartile §3.14.1 (Linear metrics) required / and complete exactly once each, in that order. must T04 Fires every Tracking element for the same event §3.14.3 (multiple Tracking elements required / (two start trackers for two parties). per event) must T05 Fires a progress tracker with an HH:MM:SS §3.14.1 (progress, HH:MM:SS required / offset. offset) must T06 Fires a progress tracker with an §3.14.1 (progress, required / HH:MM:SS.mmm offset. HH:MM:SS.mmm offset) must T07 Fires progress trackers with percentage offsets §3.14.1 (progress, n% offset) required / (10% and 80%). must T08 Stopping at 60% fires midpoint but neither §3.14.1 (thirdQuartile, complete required / thirdQuartile nor complete. definitions) must T09 Spread between the first and last Impression §2.3.4; §3.4.4 (same time) required / obs. request at impression time (ms, measured at the (a) server).

33

§2.3.4; §3.4.4

Table A1 (continued) ID

Requirement

VAST 4.3 basis

T10 Does not request the Impression of an ad whose media never plays (no supported MediaFile). T11 Requests the Impression exactly once across pause, resume, mute and unmute. T12 Fires the pause tracker once when the user pauses. T13 Fires the resume tracker once when the user resumes. T14 Fires the mute tracker once when the user mutes.

Written / graded

§2.3.4 (IAB Digital Video none / should Measurement Guidelines); §3.4.4 (a) §3.4.4 (notify when the impression none / should occurs); §2.3.4 (a) §3.14.1 (pause) required / must §3.14.1 (resume) required / must §3.14.1 (mute) required / must required / T15 Fires the unmute tracker when the user unmutes. §3.14.1 (unmute) must T16 Skipping at 6 s (skipoffset 00:00:03) fires skip §2.3.1 (Skippable Linear); §3.8 required / and neither thirdQuartile nor complete skipoffset; §3.14.1 (skip) must (skippable support is an optional ad type). T17 The skip control is not available at 1.5 s and is §3.8 (skipoffset: when skip controls none / should available at 6 s (skipoffset 00:00:03). are made available) (a) T18 A percentage skipoffset (30% of 10 s) makes skip §3.8 skipoffset; VAST 4.2 XSD none / should available at 3 s, and skipping fires skip. (skipoffset pattern allows n%) (a) T19 Progress offsets passed before the skip (2 s and 5 §3.14.3 (progress counts a view none / should s) are fired. even if the ad is skipped) (a) T20 Fires the loaded tracker once when the media is §3.14.1 (loaded) should / ready to play. should T21 Fires the rewind tracker when the user seeks §3.14.1 (rewind) required / back 2 s (from 60% to 40%). must T22 After rewinding from 6 s to 4 s, midpoint and §3.14.1 (quartiles: played none / should the 5 s progress tracker are not fired a second continuously) (a) time when playback passes them again. T23 A volume change that is not an unmute (0.5 §3.14.1 (unmute: the user none / should while unmuted) does not fire unmute. unmuted) (a) IAB macro list: CACHEBUSTING list / must (a) M01 Replaces [CACHEBUSTING] with a random (required) 8-digit integer. M02 Replaces [TIMESTAMP] with an ISO 8601 IAB macro list: TIMESTAMP list / must (a) date-time (milliseconds and zone as in the macro (required) list example 2016-01-17T8:15:07.127-05). M03 Replaces [LIMITADTRACKING] with 0 or 1 IAB macro list: list / must (a) (the harness also accepts -1/-2, since a browser LIMITADTRACKING (required) has no device advertising ID; see detail). M04 Percent-encodes macro values (the colons in §6.1 (encodeURIComponent on must / must TIMESTAMP). values) imper. / M05 Does not write -1 into a macro it does not know §6.1 (do not replace unknown ([NOTAREALMACRO]). macros with -1) should (a) M06 Replaces each of eight listed optional macros §6.1 (unprovided optional macros must / must ([ADPLAYHEAD], [ADSERVINGID], must become -1 or -2) [UNIVERSALADID], [PODSEQUENCE], [ASSETURI], [BREAKPOSITION], [IFA], [CLIENTUA]) with a plausible value or -1/-2; none left as macro or empty.

34

Table A1 (continued) ID

Requirement

VAST 4.3 basis

M07 Fills [ADSERVINGID] with the ad’s AdServingId (in-serving).

Written / graded

§3.4.3 (player fills none / should [ADSERVINGID] from (a) AdServingId) M08 When it fills [ADPLAYHEAD], the midpoint IAB macro list: ADPLAYHEAD none / should value is HH:MM:SS.mmm and about 5 s. (HH:MM:SS.mmm) (a) §6.1 (Array values: encode each should / M09 Fills [UNIVERSALADID] with one or more of the Creative’s IDs as “registry value” pairs, each value, commas unencoded); IAB should percent-encoded; several IDs are joined by a raw macro list: UNIVERSALADID comma. M10 When it fills [ASSETURI], the value is the IAB macro list: ASSETURI (the none / should (a) MediaFile actually chosen (an unsupported FLV media file being played) is listed first). M11 Replaces [CACHEBUSTING] and IAB macro list: CACHEBUSTING, list / must (a) [TIMESTAMP] in an Error URI as in any other TIMESTAMP (contexts: all tracking URI. tracking pixels, including Error) M12 Replaces [CACHEBUSTING] and IAB macro list: CACHEBUSTING, list / must (a) [TIMESTAMP] in a Wrapper’s VASTAdTagURI TIMESTAMP (contexts: VAST request URIs) before requesting it. X01 Trims the whitespace around a CDATA-wrapped §2.3.3 (whitespace around a none / should Impression URI before handing it to the network CDATA-wrapped URI) (a) (checked on the client side, since URL parsers also strip it). X02 Ignores XML comments inside an element when §2.2 (XML comments) must / must reading its URI. X03 Decodes character entities (&) in a URI §2.3.2 (character entities should be should / given as plain text (a document that breaks honored) should §2.3.3’s CDATA rule, which real tags still do). X04 Parses a document that omits xmlns="http:// (robustness; a 4.x document n/a / obs. www.iab.com/VAST". without the VAST namespace is not schema-valid) X05 Accepts a Creative with two UniversalAdId §2.2; §3.7.1 (UniversalAdId 1+; 4.2 must / must elements. XSD unbounded) X06 Accepts two ClosedCaptionFile elements. §2.2 (play the ad in a VAST must / must response); §3.9.5 (ClosedCaptionFile 0+) X07 Follows a VASTAdTagURI whose CDATA is §2.3.3; §3.19.1 VASTAdTagURI n/a / obs. surrounded by whitespace and line breaks (on the wire; URL parsers also strip the whitespace). X08 Never requests a Tracking element whose event §3.14.3 (event values are an none / should name is not a VAST event (notARealEvent). enumeration) (a) X09 Does not request Impression or Tracking URIs §3.18 Extensions (custom XML for should / that appear inside an Extension. the publisher, not VAST elements) should X10 Parses a document that starts with a UTF-8 §2.2; §2.3.2 (XML 1.0: a UTF-8 must / must byte order mark. byte order mark is allowed) X11 Reassembles an Impression URI split across two §2.3.3 (a URI containing “]]>” split required / CDATA sections (query s=x]]>x). across CDATA sections) must X12 Accepts a UniversalAdId that still carries the §3.7.1 (VAST 4.0 idValue attribute: should / VAST 4.0 idValue attribute. players should not fail) should

35

Table A1 (continued) ID

Requirement

VAST 4.3 basis

Written / graded

X13 An empty Impression element does not stop the §3.4.4 (Impression) required / other Impression from being requested. must X14 Accepts a VAST 3.0 InLine document. §4.3 Media Players (accept older should / VAST versions) should X15 Accepts a VAST 2.0 InLine document. §4.3 Media Players (accept older should / VAST versions) should X16 Follows a VAST 3.0 Wrapper to a VAST 4.3 §4.3 Media Players (accept older should / InLine. VAST versions) should must / must X17 Does not request an Impression element that sits §2.2 (acknowledge XML entirely inside an XML comment (next to a real comments); §2.3.2 Impression). X18 Requests a VASTAdTagURI written as escaped §2.3.2 (character entities should be should / text, with & between query parameters, honored); §3.19.1 VASTAdTagURI should with all parameters intact (a=1, b=2, c=3). X19 Requests a VASTAdTagURI written as escaped §2.3.2 (character entities should be should / text, with the character reference & between honored); §3.19.1 VASTAdTagURI should query parameters, with all parameters intact (a=1, b=2, c=3). Ad Pods and Stand-Alone Ads must / must P01 Orders pod ads by their sequence attribute (document order is 3, 1, 2): the ad with sequence 1 comes first (or the pod is declined with an error; pod support is optional). should / P02 When the second pod ad cannot play, playback Ad Pods (an ad fails: move on) continues with another ad (the stand-alone ad or should the third pod ad). P03 Playing a three-ad pod, each ad gets one §2.3.4 (fire each tracker at the required / impression and one complete, fired between its appropriate time); Ad Pods must own impression and the next ad’s. P05 The failed pod ad is replaced by the stand-alone Ad Pods (substitute an un-played should / ad from the response, after the first pod ad and stand-alone ad) should before the third. P06 Plays all three ads of a pod. Ad Pods (play the pod in sequence; should / play as many ads as possible) should P07 Orders the pod ads of a Wrapper’s target by Ad Pods and Stand-Alone Ads must / must their sequence attribute (document order 3, 1, 2; (play the pod in the prescribed the Wrapper sets allowMultipleAds="true"): sequence); §3.19 the ad with sequence 1 comes first (or the pod is declined with an error; pod support is optional). A01 When it does not execute the InLine’s §3.17; §3.17.4 must / must verification resource (a JavaScript resource with verificationNotExecuted an apiFramework it does not implement), requests its verificationNotExecuted URI. A02 Fills [REASON] in verificationNotExecuted with §3.17.4 [REASON]; IAB macro list: must / must a reason code (1, 2 or 3). REASON (required) A03 Handles the Wrapper’s verification too: requests Viewability with Ad Verification should / its verificationNotExecuted URI. Services (resources from should intermediary Wrappers)

36

Table A1 (continued) ID

Requirement

VAST 4.3 basis

A04 Reports a verification resource whose apiFramework it does not implement with reason 2 (verification not supported). A05 When an OMID verification resource fails to load (HTTP 404), requests its verificationNotExecuted URI with reason 3 (error during resource load). I01 Displays the Icon after the ad starts and requests its IconViewTracking URI. I02 On a click on the Icon, requests its IconClickTracking URI. I03 Displays an Icon with offset 00:00:01 and duration 00:00:08 once playback passes 1 s, requesting its IconViewTracking URI. S01 Skips a MediaFile of an unsupported type listed first (video/x-flv) and plays a supported one. S02 Optional: plays an audio ad (adType=“audio”, Opus WebM and AAC files). R01 A timed-out Wrapper request does not escape as an uncaught exception (which terminates a Node.js process). R02 A failed Wrapper request does not escape as an uncaught exception. R03 Follows an HTTP 302 redirect on a VASTAdTagURI. R04 Parses a VAST response sent with Content-Encoding: gzip. R05 Parses a VAST response served as text/plain rather than an XML media type. R06 A tracking URI that answers HTTP 500 does not stop later trackers (quartiles, complete).

Written / graded

§3.17.4 (reason code 2); §3.17.1; must / must Viewability with Ad Verification Services §3.17.4 (reason code 3: 404 or other must / must failed response codes); Viewability with Ad Verification Services Industry Icon Support; §3.11.2 IconViewTracking §3.11.5 IconClickTracking §3.11.1 Icon (offset, duration)

must / must required / must should / should

MediaFiles (choose by attributes, not document order) §1.4; §3.4 (adType=“audio”)

should / should may / may

(engineering; not a VAST clause)

n/a / obs.

(engineering; not a VAST clause)

n/a / obs.

(HTTP; not a VAST clause)

n/a / obs.

(HTTP; not a VAST clause)

n/a / obs.

(HTTP; not a VAST clause)

n/a / obs.

(HTTP; not a VAST clause)

n/a / obs.

Appendix B. Findings One row per finding (one cause, located in source for the open-source clients). Requirements are catalog IDs with their strength (must, should or observation) as graded in code/harness/suite/ requirements.mjs. Locations are in the tested release, relative to the package root: vast-clientjs 6.5.0 src/, video-ad-sdk 4.0.1 dist/ (the code that ran), iab-vast-loader 2.5.1 and its parser iab-vast-parser 1.2.1 lib/, vastlint-client 0.15.0 dist/, Shaka Player 5.2.12 lib/ads/, rmp-vast 17.2.0 src/js/ at commit 5975ead. “absent:” means the feature is not implemented and says what was searched. IMA is closed source and is analyzed as a black box: Table B7 gives, in place of a location, the evidence the server logged and the events of IMA’s public API, from the Chromium reference run (Firefox for F86; the three Linux engines and the macOS run for F85); the probes are in the replication package. rmp-vast cells that come from its bundled vast-client are attributed to vast-client-js (F01, F02, F03, F05). 37

Release range is the span of vast-client-js, video-ad-sdk, iab-vast-loader and vastlint-client releases (Node.js sweep) in which the finding appears; Shaka, rmp-vast and IMA were not swept. Table B1. Findings in vast-client-js 6.5.0. ID

Requirements

F01

M12 (must)

F02

F03

F04

F05

F06

F07

Symptom

Requests a Wrapper’s VASTAdTagURI with [CACHEBUSTING] and [TIMESTAMP] still in brackets. P07 (must) A pod in a Wrapper’s target is returned in document order (3, 1, 2). R01 (obs.) A Wrapper request timeout throws outside any promise: uncaught exception, Node.js process exits with code 1. none: robustness (fuzz A target that sends headers and stalls its body C4, D4) is waited on with no timeout at every hop. W11 (must) Non-well-formed Wrapper target reported as 301 (Node.js, Firefox) or 303 (Chromium, Chrome, WebKit, Safari), never 100. none: robustness (fuzz An <Ad> with neither C2) InLine nor Wrapper makes get() throw a TypeError; no Error URI; sibling ads lost. none: macro grid (4.x Five pixel macros marked macros) 4.x ([STOREID], [STOREURL], [PLAYBACKMETHODS], [GPPSTRING], [GPPSECTIONID]) sent literally.

F08

M10 (should)

F09

M09 (should)

Location

Releases

src/parser/ vast_parser.js:391; src/parser/ vast_parser.js:403405 src/parser/ vast_parser.js:258315 (parse()) src/fetcher/ url_handler.js:5764

all 9 (3.3.0 to 6.5.0)

src/fetcher/ url_handler.js:6672

tested release only

src/parser/ vast_parser.js:442, 170-187, 467-477

all 9 (3.3.0 to 6.5.0)

src/parser/ ad_parser.js:20-59; src/parser/ vast_parser.js:226

tested release only

not in the swept catalog 6.0.2 to 6.5.0 (of 3.3.0 to 6.5.0)

src/util/ tested release only macros.js:1-49; src/ util/util.js:72-74; absent: STOREID, STOREURL, PLAYBACKMETHODS, GPPSTRING, GPPSECTIONID anywhere under src/ src/ all 9 (3.3.0 to 6.5.0) vast_tracker.js:899906

[ASSETURI] carries the first MediaFile, overwriting the value the integrator passes; not the one played. [UNIVERSALADID] src/ 4.0.1 to 6.5.0 (of 3.3.0 percent-encodes the vast_tracker.js:913- to 6.5.0) separating comma (%2C). 917; src/util/ util.js:42

38

Table B1 (continued) ID

Requirements

Symptom

Location

Releases

F10

none: outside catalog

src/parser/ vast_parser.js:369, :382, :74

tested release only

F11

none: outside catalog (fuzz D2)

wrapperLimit accepts one Wrapper fewer than its value (decides rmp-vast’s W03). An InLine with no usable MediaFile is reported with 303.

src/parser/ vast_parser.js:467477

tested release only

Table B2. Findings in rmp-vast 17.2.0 (not swept). ID

Requirements

Symptom

F12

W03 (must)

Stops a five-Wrapper chain after four documents with 302 (API reports 303).

F13

F14

F15 F16

F17 F18 F19 F20 F21 F22

Location

src/js/helpers/utils.js:223; src/js/index.js:591; bundled vast-client 6.4.2 src/parser/ vast_parser.js:362, 375 W20 (should), W21 (should) Passes allowMultipleAds: true, src/js/index.js:593; bundled overriding the Wrapper attribute. vast-client 6.4.2 src/parser/ vast_parser.js:389-392 W23 (should), P05 (should) Never falls back to the stand-alone absent: fallbackOnNoAd, remainingAds, getNextAds not ad (fallbackOnNoAd, failed pod used in rmp-vast’s own modules ad). (bundle search); src/js/ index.js:364-367; bundled vast-client 6.4.2 src/parser/ ad_parser.js:223-234, src/ parser/vast_parser.js:304315 M01 (must) [CACHEBUSTING] is eight src/js/framework/fw.js:194alphanumeric characters. 201 M03 (must), M06 (must) [LIMITADTRACKING] only if the src/js/index.js:652-655; src/ integrator passes it; js/helpers/tracking.js:359[PODSEQUENCE], [IFA], 362, 438-442; absent: no [CLIENTUA] get no -1. rmp-vast code names [IFA] or [CLIENTUA] (only params.macros, tracking.js:214-222) M11 (must) Error URIs skip macro replacement src/js/helpers/ except [ERRORCODE]. tracking.js:462-482 M09 (should) [UNIVERSALADID] starts with a src/js/helpers/ comma and encodes the separators. tracking.js:338-351 E06 (must) Undecodable media reported as src/js/creatives/ 401, not 405. linear.js:147-176 T13 (must) Resume tracker fires on every play src/js/helpers/ event, including ad start. tracking.js:176-180 T23 (should) Any volume change while unmuted src/js/helpers/ fires unmute. tracking.js:112-123 T02 (should) An about:blank Impression is src/js/index.js:460-466; src/ requested. js/helpers/tracking.js:82110

39

Table B2 (continued) ID

Requirements

Symptom

Location

F23

T05 (must), T06 (must), T07 macOS WebKit only: no progress (must) tracker fires (quartiles do); T19 then n/a.

F24

T21 (must)

F25

I03 (should)

F26

A01 (must), A03 (should)

src/js/creatives/ linear.js:67-108, :74-76 and :292; src/js/framework/ fw.js:84-88; src/js/players/ ad-player.js:53-58 No rewind tracker when the absent: no rewind tracker or element is seeked back (borderline: handler in rmp-vast’s own no seek control by default). modules (bundle search for rewind, seeked, seeking); showControlsForAdPlayer is false by default (src/js/ helpers/utils.js:230, used at src/js/players/adplayer.js:139) Icon drawn on first playing event; src/js/creatives/ offset and duration never read. icons.js:186-190 With defaults, no AdVerifications src/js/index.js:527-530 handling: no (verification handled only if verificationNotExecuted. omidSupport is set); src/js/ verification/omsdk.js:41-54; src/js/helpers/utils.js:245

Table B3. Findings in video-ad-sdk 4.0.1. ID

Requirements

Symptom

Location

Releases

F27

P01 (must), P07 (must)

Pods play in document order: compareBySequence returns 0 for every pair.

F28

P06 (should)

F29

E05 (must), E06 (must)

dist/vastSelectors/ index.js:13-18; :103; dist/ vastRequest/helpers/ getNextAd.js:3-5 dist/runner/ runWaterfall.js:90 dist/adUnit/VastAdUnit.js:34

F30

E03 (must), E09 (must)

Only the first ad of a pod plays; no error. Error URI requested with code=undefined for media 404 or decode failure, then 402. No playable MediaFile (or dist/runner/helpers/ NonLinear-only ad) reported as startVideoAd.js:39 401 to InLine and every Wrapper.

P01: 3.0.0 to 4.0.1 (of 2.1.0 to 4.0.1); P07: not in the swept catalog all 9 (2.1.0 to 4.0.1) browser-only, not swept

F31

I03 (should)

F32

A01 (must), A03 (should)

F33

X11 (must)

F34

T07 (must)

Icon with an offset is never drawn; dist/adUnit/helpers/icons/ no view tracker. addIcons.js:15 AdVerifications never read; no absent: no occurrence of verificationNotExecuted. “verification” (any case) in video-ad-sdk 4.0.1 dist/ or src/ A URI split across CDATA dist/xml/helpers/ sections is truncated to the first xmlSelectors.js:36-38 part. Percentage progress offsets dist/adUnit/helpers/metrics/ computed from the media handlers/onProgress.js:22element’s duration before it is set 23, :19 and :29 become NaN; never fire in the tested setup.

40

all 9 (2.1.0 to 4.0.1); 2.1.0 to 2.5.1 request no Error URI, 401 from 2.6.3 browser-only, not swept all 9 (2.1.0 to 4.0.1) all 9 (2.1.0 to 4.0.1) all 9 (2.1.0 to 4.0.1)

Table B3 (continued) ID

Requirements

Symptom

Location

Releases

F35

W07 (must)

F36

W17 (must)

Failed Wrapper fetch (404, 500, refused) reported as 502. Wrapper ClickThrough not used when the InLine has none.

all 9 (2.1.0 to 4.0.1) all 9 (2.1.0 to 4.0.1)

F37

M01 (must)

F38

M03 (must), M06 (must), M07 (should)

dist/vastRequest/ requestAd.js:14-26 dist/adUnit/helpers/metrics/ setupMetricHandlers.js:8; dist/vastSelectors/ index.js:327 dist/tracker/helpers/ parseMacro.js:9 and :22-24 dist/tracker/helpers/ parseMacro.js:19-35

F39

M12 (must)

F40

T18 (should)

F41

T20 (should)

F42 F43

W20 (should), W21 (should) T02 (should)

F44

W11 (must)

F45

none: robustness (fuzz C3)

[CACHEBUSTING] has 1 to 10 digits. parseMacro fills only CACHEBUSTING and TIMESTAMP; other listed macros literal. Requests a Wrapper’s dist/vastRequest/ VASTAdTagURI as written. requestAd.js:118 (the Wrapper’s getVASTAdTagURI is requested unchanged) Percentage skipoffset ignored. dist/adUnit/helpers/metrics/ handlers/onSkip.js:22; dist/ vastSelectors/helpers/ parseOffset.js:2-9; dist/ vastSelectors/index.js:368372 No loaded event. dist/tracker/ linearEvents.js:12-176; absent: no loaded event or handler in dist/tracker/ or dist/adUnit/helpers/metrics/ allowMultipleAds defaults to true; dist/vastRequest/ false rejects the response with 203. requestAd.js:51 and :57-61 An about:blank Impression is dist/tracker/helpers/ passed to the pixel tracker. createVastEventTracker.js:13 Browser DOMParser partial tree dist/xml/helpers/xml2js.js:5; (parsererror) accepted as VAST; sane-domparser-error 0.2.0 ad accepted, then 401. index.js:18-23 VPAID-marked MP4: unhandled dist/runner/helpers/ rejection; 901 to Error URI, then startVideoAd.js:33 impression and playback. (adUnit.start promise not handled; src startVideoAd.ts:74); dist/adUnit/helpers/vpaid/ loadCreative.js:7

41

all 9 (2.1.0 to 4.0.1) all 9 (2.1.0 to 4.0.1)

all 9 (2.1.0 to 4.0.1)

2.2.1 to 4.0.1 (of 2.1.0 to 4.0.1)

all 9 (2.1.0 to 4.0.1)

all 9 (2.1.0 to 4.0.1) all 9 (2.1.0 to 4.0.1) none in sweep

tested release only

Table B4. Findings in iab-vast-loader 2.5.1. ID

Requirements

Symptom

Location

Releases

F46

W02 (must), P07 (must)

lib/loader.js:58-65; iab-vast-parser lib/factory/ vast.js:53-55

W02: all 6 (2.0.0 to 2.5.1); P07: not in the swept catalog

F47

X05 (must), X06 (must)

iab-vast-parser lib/vast/ schema.js:13-33; lib/xml/ unmarshaler.js:113-118

all 6 (2.0.0 to 2.5.1)

F48

W19 (should)

The loader checks only vast.ads after a Wrapper: a lone <Ad sequence="1"> Wrapper, filed as a pod, is never followed (defaults only; no ad, no error), and a pod behind a Wrapper is reported as 303. Valid 4.2 documents with repeated UniversalAdId or ClosedCaptionFile rejected with 101 (and four more elements, fuzz D3). followAdditionalWrappers never consulted.

all 6 (2.0.0 to 2.5.1)

F49

W11 (must)

Node.js: xmldom recovers an ad from non-well-formed XML; Firefox: 101.

F50

M12 (must)

F51

none: robustness (fuzz C5)

absent: followAdditionalWrappers is not read in iab-vast-loader lib/ (5 files); lib/loader.js:58-62 iab-vast-parser lib/index.js:7, 18-20; lib/parse.js:34-45; xmldom 0.1.31 dom-parser.js:30, 175-183 lib/loader.js:61, :81, :14 and :88 iab-vast-parser lib/parse.js:39

F52

none: robustness (fuzz C6)

F53

none: robustness (fuzz C7)

F54

none: robustness (fuzz C8)

Requests a Wrapper’s VASTAdTagURI as written. Empty response (HTTP 204 or empty 200): an internal TypeError with no VAST code. A response that is not XML: iab-vast-parser lib/xml/ rejected with 101 and an internal unmarshaler.js:75 TypeError, where 100 applies. A root element other than VAST: iab-vast-parser lib/factory/ rejected with 101 and an internal vast.js:38 TypeError. A lower-case element name such iab-vast-parser lib/inherit/ as <creatives> rejects the whole ad.js:57, 85, 88; lib/xml/ response with an internal unmarshaler.js:19-25, 85 TypeError.

all 6 (2.0.0 to 2.5.1)

all 6 (2.0.0 to 2.5.1) tested release only tested release only tested release only tested release only

Table B5. Findings in vastlint-client 0.15.0. ID

Requirements

Symptom

Location

F55

W03 (must)

F56

W11 (must), X09 (should), X02 (must), X17 (must)

maxWrapperDepth 5 counts dist/session.js:8 documents: stops before the InLine of a five-Wrapper chain, no code. XML read with regular expressions: dist/ ads from malformed XML; resolved.js:106-148; commented-out URIs requested. dist/ tracking.js:72-141

42

Releases all 12 (0.4.24 to 0.15.0) W11, X09, X02: all 12 (0.4.24 to 0.15.0); X17: not in the swept catalog

Table B5 (continued) ID

Requirements

F57

X18 (should), X19 Entity-escaped VASTAdTagURI vastlint 0.15.0 (should) truncated at the first &amp; (or &# upstream crates/ 38;). vastlint-core/src/ inspect.rs:110-117, 126, 156-158 (compiled into vastlint_wasm_bg. wasm); caller dist/ session.js:461, 477 W19 (should), followAdditionalWrappers and dist/ W20 (should), allowMultipleAds never read. resolved.js:457-459 W21 (should) W23 (should) A response that holds a Wrapper dist/session.js:461 ad and a stand-alone InLine ad is (buildHop takes the classified as an InLine document; document type from the Wrapper is never followed, and inspectDocument) and its target (noad.xml) is never :649-651 (a hop that is requested. not a Wrapper ends resolution as resolved) dist/playbackDocumented controller fires T05 (must), T06 quartiles only: no progress, loaded queue.js:381-389 (must), T07 or rewind. (must), T21 (must), T20 (should) dist/ M03 (must), M06 Only CACHEBUSTING and (must), M07 TIMESTAMP expanded; controller tracking.js:169-181 takes no macros. (should) M12 (must) Requests a Wrapper’s dist/session.js:659 VASTAdTagURI as written. and :674 (fetchXmlFromUrl requests the URL as found) none: delegated After a mid-chain fetch failure the dist/ limit snapshot keeps only the root hop; session.js:512-514, intermediate Wrapper Error URIs 629, 674, 682-689, unreachable. 700-702

F58

F59

F60

F61

F62

F63

Symptom

Location

43

Releases not in the swept catalog

all 12 (0.4.24 to 0.15.0) all 12 (0.4.24 to 0.15.0)

all 12 (0.4.24 to 0.15.0)

all 12 (0.4.24 to 0.15.0) all 12 (0.4.24 to 0.15.0)

tested release only

Table B6. Findings in Shaka Player 5.2.12 (not swept). ID

Requirements

Symptom

Location

F64

W01 (must), W02 (must)

F65

M01 (must), M02 (must), M03 (must), M06 (must), M07 (should), M11 (must)

F66

E01 (must), E02 (must)

F67

E03 (must), E05 (must), E06 (must)

F68

E07 (must)

F69

T10 (should)

F70

F71

T05 (must), T06 (must), T07 (must), T20 (should), T21 (must) T18 (should)

F72 F73

T23 (should) S01 (should)

F74

P03 (must)

F75

P01 (must)

F76

P05 (should)

Every <Ad> without an InLine is lib/ads/vast_interstitial_ skipped: no Wrapper request, no parser.js:29-33 ad, no error. No macro replaced in tracking or absent: no CACHEBUSTING, Error URLs. TIMESTAMP, ERRORCODE, ASSETURI, ADSERVINGID or UNIVERSALADID in lib/ads/ (22 files); lib/ads/ad_utils.js:173176 A no-ad response’s lib/ads/ad_utils.js:49; lib/ document-level Error URIs are ads/vast_interstitial_parser. never requested. js:29-33; absent: no read of the <Error> children of <VAST> in lib/ads/ The error code is never lib/ads/ad_utils.js:149-151, determined: Error URIs are 173-176; lib/ads/interstitial_ requested with a literal ad_manager.js:1226-1236; absent: [ERRORCODE]. no ERRORCODE in lib/ads/ A media file that does not answer absent: no media-load timeout in is never reported (no media lib/ads/ timeout). interstitial_ad_manager.js Impression and start fire when lib/ads/interstitial_ad_ the ad is set up, before its media manager.js:1262-1263 loads. No progress, loaded or rewind lib/ads/ad_utils.js:22-38, 95events: the tracking table has no 109 such keys. A percentage skipoffset is not lib/ads/vast_interstitial_ parsed; the ad is not skippable. parser.js:81-87 A volume change fires unmute. lib/ads/ad_utils.js:161 MediaFiles sorted by height; first lib/ads/vast_interstitial_ used without checking type (FLV parser.js:117-138 fails). Each pod ad’s complete pixel is lib/ads/interstitial_ad_ sent to the next ad. manager.js:1237-1244, :892 and :2561-2567 Pods play in document order: the lib/ads/vast_interstitial_ sequence attribute is never read. parser.js:29-62 A failed pod ad is not replaced by lib/ads/interstitial_ad_ the stand-alone ad; that ad plays manager.js:1133-1140, 1207after the pod. 1211; absent: no code in lib/ads/ handles stand-alone ads

44

Table B6 (continued) ID

Requirements

Symptom

Location

F77

I01 (must), I03 (should)

Icons not drawn; trackers never fire.

F78

A01 (must)

AdVerifications never parsed: no verificationNotExecuted request.

absent: no “icon” (any case) in shaka-player 5.2.12 lib/ads/; lib/ ads/vast_interstitial_parser. js:39-61 reads Linear and NonLinearAds only lib/ads/vast_interstitial_ parser.js:29-45 (an InLine yields Creatives only); lib/ads/ ad_utils.js:44-69 (createTrackingFromInline reads Error and Impression only); absent: no “verification” in lib/ads/

Table B7. Findings in Google IMA HTML5 SDK 3.794.0 (closed comparison, black-box; not swept). Observable evidence (requests logged; public API)

ID

Requirements

Symptom

F79

W03 (must)

Rejects a chain of five Wrappers.

F80

W17 (must)

F81

W20 (should), W21 (should)

F82

W23 (should)

F83

A03 (should)

F84

A05 (must)

five Wrapper documents fetched, no InLine; Error URIs of all five requested with code 302; AdError 302 “The maximum number of VAST wrappers (4) has been reached.” Wrapper ClickThrough not used. after a click on the ad, no click-tracking request and no landing page; probe PRB3 (InLine without ClickThrough): the click pauses the ad and sends no click-tracking request A Wrapper target with a pod and start.xml and pod.xml fetched, no a stand-alone ad is rejected with Error URI requested, AdError 2 “Ad source cannot contain playlist IMA code 2. response.” After a Wrapper that leads to no Error URIs requested with 303, ad (303), no other ad is tried. AdError 900, the stand-alone ad of the same response never requested The Wrapper’s verification vendor InLine verificationNotExecuted is not reported; at most one requested with reason 2, verificationNotExecuted per ad in Wrapper’s never; probe PRB1: a these probes. Wrapper-only vendor is reported; probe PRB2: of two InLine vendors only the first is reported A verification resource that resource fetched with status 404, answers 404 is not reported. no verificationNotExecuted request

45

Table B7 (continued) Observable evidence (requests logged; public API)

ID

Requirements

Symptom

F85

I01 (must)

Icon view requested over 50 ms before the ad’s start request: Linux Firefox and WebKit, macOS WebKit, some Linux Chromium runs.

F86

W11 (must)

Firefox: 303 for a malformed Wrapper target.

icon view requested before the start request in every run; 20 repeats per Linux engine: 36 to 68 ms in Chromium (6 of 20 beyond the 50 ms tolerance; reference runs 122 and 90 ms), 54 to 111 ms in Firefox and 165 to 237 ms in WebKit (all beyond); macOS, 10 repeats per browser: 11 to 12 ms in Chromium, 10 to 13 ms in Chrome and 13 to 50 ms in Firefox (none beyond), 51 to 58 ms in WebKit (all beyond) Firefox: Error URI requested with code 303; Chromium, Chrome and WebKit: 100

Appendix C. Deviating cells per configuration Node.js layer and Chromium 141. Requirement texts are in Appendix A; the findings behind the cells are in Appendix B; per-cell details are in Online Resource 5. Table C1. Deviating cells per configuration, by graded strength (Node.js layer and Chromium 141). Configuration

Layer

must

should

observation

vast-client-js 6.5.0 video-ad-sdk 4.0.1

Node.js Node.js

M09, M10 W20, W21, T02, T20, T18, A03, M07, P06

R01 none

iab-vast-loader 2.5.1, defaults iab-vast-loader 2.5.1, noSingleAdPods vastlint-client 0.15.0

Node.js

W19

none

W19

none

Node.js

vast-client-js 6.5.0 video-ad-sdk 4.0.1

Chromium Chromium

iab-vast-loader 2.5.1, defaults iab-vast-loader 2.5.1, noSingleAdPods

Chromium

W11, M12, P07 W07, W17, E03, T07, M01, M03, P01, E09, A01, M06, M12, X11, P07 W02, W11, X05, X06, M12, P07 W11, X05, X06, M12, P07 W03, W11, T05, T06, T07, M03, X02, T21, M06, M12, X17 W11, M12, P07 W07, W11, W17, E03, T07, M01, M03, P01, E05, E06, E09, A01, M06, M12, X11, P07 W02, X05, X06, M12, P07 X05, X06, M12, P07

Node.js

Chromium

46

W19, W20, W21, W23, none T20, M07, X09, X18, X19 M09, M10 R01 W20, W21, T02, T20, none T18, A03, M07, P06, I03

W19

none

W19

none

Table C1 (continued) Configuration

Layer

must

should

observation

Shaka Player 5.2.12

Chromium

W01, W02, E01, E02, E03, T05, T06, T07, M01, M02, M03, P01, E05, E06, E07, T21, A01, M06, M11, P03, I01 W03, W11, M01, M03, E06, T13, T21, A01, M06, M11, M12, P07 W03, W11, M01, M03, E06, T13, T21, M06, M11, M12, P07

T10, T23, T20, T18, S01, M07, P05, I03

none

rmp-vast 17.2.0, defaults Chromium

rmp-vast 17.2.0, omidSupport

Chromium

W20, W21, T02, W23, R01 T23, A03, M09, P05, I03 W20, W21, T02, W23, T23, M09, P05, I03

R01

In rmp-vast’s rows, M12, P07, W11 and R01 are attributed to its bundled vast-client-js (rule G4).

Appendix D. Mutation fuzzing: root causes The campaign used 139 seeds in 5 rounds and produced 1,442 cases (1,437 unique), 12 classes and 9 root causes. video-ad-sdk locations here are in the TypeScript sources shipped in its package (src/), where the fuzzing scripts attributed them; Appendix B gives the matching dist/ code that ran. Minimal documents and reproduction scripts are in the replication package. Table D1. Mutation fuzzing root causes, with classes, unique cases and source locations. Root cause

Raised by an unmutated seed Attribution

Client

Classes

Unique cases

C1

vast-client-js 6.5.0

1

24

yes

library

C2

vast-client-js 6.5.0

1

21

no

library

C3

video-ad-sdk 4.0.1

1

18

yes

library

C4

vast-client-js 6.5.0

1

1

no

library

C5

iab-vast-loader 2.5.1 iab-vast-loader 2.5.1

1

7

no

library

1

36

yes

library

C6

47

Location src/fetcher/ url_handler.js: 57-64 src/parser/ ad_parser.js:2059 src/adUnit/ helpers/vpaid/ loadCreative.ts: 19 src/fetcher/ url_handler.js: 66-72 iab-vast-parser/ lib/parse.js:39 iab-vast-parser/ lib/xml/ unmarshaler.js: 75

Table D1 (continued) Root cause

Raised by an unmutated seed Attribution

Client

Classes

Unique cases

C7

iab-vast-loader 2.5.1

1

32

yes

library

C8

iab-vast-loader 2.5.1

4

13

no

library

C9

video-ad-sdk 4.0.1

1

30

yes

environment artifact

Location iab-vast-parser/ lib/factory/ vast.js:38 iab-vast-parser/ lib/inherit/ ad.js:88 src/adUnit/ helpers/icons/ addIcons.ts:65

References 8bitsten (2026) URLHandler timeout throws an uncaught exception. GitHub issue #510, dailymotion/vast-client-js. https://github.com/dailymotion/vast-client-js/issues/510. Accessed 20 Oct 2026 Ahasanuzzaman M, Hassan S, Bezemer C-P, Hassan AE (2020) A longitudinal study of popular ad libraries in the Google Play Store. Empirical Software Engineering 25:824–858. https://doi.org/10.1007/s10664-019-09766-x Ahasanuzzaman M, Hassan S, Hassan AE (2022) Studying ad library integration strategies of top free-to-download apps. IEEE Transactions on Software Engineering 48:209–224. https://doi.org/10.1109/TSE.2020.2983399 Andrews JH, Zhang Y (2003) General test result checking with log file analysis. IEEE Transactions on Software Engineering 29:634–648. https://doi.org/10.1109/TSE.2003.1214327 Barr ET, Harman M, McMinn P, et al (2015) The oracle problem in software testing: A survey. IEEE Transactions on Software Engineering 41:507–525. https://doi.org/10.1109/TSE.2014.2372785 Böhme M (2018) STADS: Software testing as species discovery. ACM Transactions on Software Engineering and Methodology 27:1–52. https://doi.org/10.1145/3210309 Bradner S (1997) Key words for use in RFCs to indicate requirement levels. RFC 2119, BCP 14, Internet Engineering Task Force. https://doi.org/10.17487/RFC2119 Brubaker C, Jana S, Ray B, et al (2014) Using Frankencerts for automated adversarial testing of certificate validation in SSL/TLS implementations. In: 2014 IEEE Symposium on Security and Privacy. IEEE, pp 114–129. https://doi.org/10.1109/SP.2014.15 Chao A (1984) Nonparametric estimation of the number of classes in a population. Scandinavian Journal of Statistics 11:265–270. https://www.jstor.org/stable/4615964 Chao A (1987) Estimating the population size for capture-recapture data with unequal catchability. Biometrics 43:783–791. https://doi.org/10.2307/2531532 Chaudhary P (2026) Bound the response body read by the request timeout. GitHub pull request #515, dailymotion/vast-client-js. https://github.com/dailymotion/vast-client-js/pull/515. Accessed 28 Oct 2026 Chaudhary P, Khandelwal K (2026) Trigger timing, deadline readiness, and event-aligned accounting for dynamic ad insertion. Preprint arXiv:2609.19899. https://doi.org/10.48550/arXiv.2609.19899 48

Chen C, Tian C, Duan Z, Zhao L (2018) RFC-directed differential testing of certificate validation in SSL/TLS implementations. In: Proceedings of the 40th International Conference on Software Engineering (ICSE). ACM, pp 859–870. https://doi.org/10.1145/3180155.3180226 Chen Y, Su T, Sun C, et al (2016) Coverage-directed differential testing of JVM implementations. In: Proceedings of the 37th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI). ACM, pp 85–99. https://doi.org/10.1145/2908080.2908095 Crussell J, Stevens R, Chen H (2014) MAdFraud: Investigating ad fraud in Android applications. In: Proceedings of the 12th Annual International Conference on Mobile Systems, Applications, and Services (MobiSys). ACM, pp 123–134. https://doi.org/10.1145/2594368.2594391 Dailymotion (2026) vast-client-js (@dailymotion/vast-client). Version 6.5.0. https://github.com/dailymotion/vast-client-js. Accessed 20 Oct 2026 Davis JC, Michael IV LG, Coghlan CA, et al (2019) Why aren’t regular expressions a lingua franca? An empirical study on the re-use and portability of regular expressions. In: Proceedings of the 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE). ACM, pp 443–454. https://doi.org/10.1145/3338906.3338909 de Ruiter J, Poll E (2015) Protocol state fuzzing of TLS implementations. In: 24th USENIX Security Symposium (USENIX Security 15). USENIX Association, pp 193–206. https: //www.usenix.org/conference/usenixsecurity15/technical-sessions/presentation/de-ruiter. Accessed 20 Oct 2026 Doca (2019) VASTAdTagURI macros not being replaced. GitHub issue #331, dailymotion/vast-client-js. https://github.com/dailymotion/vast-client-js/issues/331. Accessed 20 Oct 2026 Dong F, Wang H, Li L, et al (2018) FraudDroid: Automated ad fraud detection for Android apps. In: Proceedings of the 26th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE). ACM, pp 257–268. https://doi.org/10.1145/3236024.3236045 Google (2026) Interactive Media Ads (IMA) SDK for HTML5. Version 3.794.0. Served file archived 16 Oct 2026. https://imasdk.googleapis.com/js/sdkloader/ima3.js. Accessed 16 Oct 2026 Gui J, McIlroy S, Nagappan M, Halfond WGJ (2015) Truth in advertising: The hidden cost of mobile ads for software developers. In: 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering (ICSE). IEEE, pp 100–110. https://doi.org/10.1109/ICSE.2015.32 Harrand N, Durieux T, Broman D, Baudry B (2021) The behavioral diversity of Java JSON libraries. In: 2021 IEEE 32nd International Symposium on Software Reliability Engineering (ISSRE). IEEE, pp 412–422. https://doi.org/10.1109/ISSRE52982.2021.00050 Herbold S, Tunkel S (2023) Differential testing for machine learning: An analysis for classification algorithms beyond deep learning. Empirical Software Engineering 28:34. https://doi.org/10.1007/s10664-022-10273-9 hwde (2019) VAST request URI macro’s defined by IAB are not replaced for wrappers. GitHub issue #70, zentrick/iab-vast-loader. https://github.com/zentrick/iab-vast-loader/issues/70. Accessed 20 Oct 2026 IAB Technology Laboratory (2022) Video Ad Serving Template (VAST) 4.3. Specification. https://github.com/InteractiveAdvertisingBureau/VAST4.x. Accessed 20 Oct 2026 IAB Technology Laboratory (2026) VAST 4 macros (macros-data.json), in the VAST repository, commit 99d5d51. https://github.com/InteractiveAdvertisingBureau/vast. Accessed 20 Oct 2026 Jabiyev B, Sprecher S, Onarlioglu K, Kirda E (2021) T-Reqs: HTTP request smuggling with differential fuzzing. In: Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security (CCS). ACM, pp 1805–1820. 49

https://doi.org/10.1145/3460120.3485384 justinjeong5 (2026) [util] Prevent uncaught exceptions on request timeout. GitHub pull request #514, dailymotion/vast-client-js. https://github.com/dailymotion/vast-client-js/pull/514. Accessed 20 Oct 2026 Kallus B, Anantharaman P, Locasto M, Smith SW (2024) The HTTP Garden: Discovering parsing vulnerabilities in HTTP/1.1 implementations by differential fuzzing of request streams. Preprint arXiv:2405.17737. https://doi.org/10.48550/arXiv.2405.17737 Kropp NP, Koopman PJ, Siewiorek DP (1998) Automated robustness testing of off-the-shelf software components. In: Digest of Papers, Twenty-Eighth Annual International Symposium on Fault-Tolerant Computing (FTCS-28). IEEE, pp 230–239. https://doi.org/10.1109/FTCS.1998.689474 Kuchta T, Lutellier T, Wong E, et al (2018) On the correctness of electronic documents: Studying, finding, and localizing inconsistency bugs in PDF readers and files. Empirical Software Engineering 23:3187–3220. https://doi.org/10.1007/s10664-018-9600-2 Lima I, Silva J, Miranda B, et al (2021) Exposing bugs in JavaScript engines through test transplantation and differential testing. Software Quality Journal 29:129–158. https://doi.org/10.1007/s11219-020-09537-8 Marciel M, Cuevas R, Banchs A, et al (2016) Understanding the detection of view fraud in video content portals. In: Proceedings of the 25th International Conference on World Wide Web (WWW). International World Wide Web Conferences Steering Committee, pp 357–368. https://doi.org/10.1145/2872427.2882980 McKeeman WM (1998) Differential testing for software. Digital Technical Journal 10:100–107 Méndez Fernández D, Monperrus M, Feldt R, Zimmermann T (2019) The open science initiative of the Empirical Software Engineering journal. Empirical Software Engineering 24:1057–1060. https://doi.org/10.1007/s10664-019-09712-x Mohajeri Moghaddam H, Acar G, Burgess B, et al (2019) Watching you watch: The tracking ecosystem of over-the-top TV streaming devices. In: Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security (CCS). ACM, pp 131–147. https://doi.org/10.1145/3319535.3354198 Petsios T, Tang A, Stolfo S, et al (2017) NEZHA: Efficient domain-independent differential testing. In: 2017 IEEE Symposium on Security and Privacy. IEEE, pp 615–632. https://doi.org/10.1109/SP.2017.27 Prado A, Zendra O, Boinot P, Barais O (2026) Mind the gap: How SBOM specification ambiguities lead to divergent software bills of materials. An empirical tool study. In: Proceedings of SCORED 2026 (to appear). Preprint arXiv:2609.19920. https://doi.org/10.48550/arXiv.2609.19920 Radiant Media Player (2026) rmp-vast. Version 17.2.0. https://github.com/radiantmediaplayer/rmp-vast. Accessed 20 Oct 2026 Ralph P, bin Ali N, Baltes S, et al (2020) Empirical standards for software engineering research. Preprint arXiv:2010.03525. https://doi.org/10.48550/arXiv.2010.03525 Rambler Digital Solutions (2024) video-ad-sdk. Version 4.0.1. https://www.npmjs.com/package/video-ad-sdk/v/4.0.1. Accessed 20 Oct 2026 Seeliger R, Silhavy D, Pham S, Arbanowski S (2016) Cross-platform ad-insertion using HbbTV and MPEG-DASH. In: 2016 Asia Pacific Conference on Multimedia and Broadcasting (APMediaCast). IEEE, pp 7–13. https://doi.org/10.1109/APMediaCast.2016.7878174 Segura S, Fraser G, Sanchez AB, Ruiz-Cortés A (2016) A survey on metamorphic testing. IEEE Transactions on Software Engineering 42:805–824. https://doi.org/10.1109/TSE.2016.2532875 Sekowski A (2026a) VAST XML validation at bid-time scale: Latency analysis and integration 50

patterns for programmatic video pipelines. Preprint, ResearchGate. https://doi.org/10.13140/RG.2.2.11404.27520 Sekowski A (2026b) vastlint-client. Version 0.15.0. https://github.com/aleksUIX/vastlint/tree/v0.15.0. Accessed 27 Oct 2026 Shaka Project (2026) Shaka Player. Version 5.2.12. https://github.com/shaka-project/shaka-player. Accessed 20 Oct 2026 Sivakorn S, Argyros G, Pei K, et al (2017) HVLearn: Automated black-box analysis of hostname verification in SSL/TLS implementations. In: 2017 IEEE Symposium on Security and Privacy. IEEE, pp 521–538. https://doi.org/10.1109/SP.2017.46 Smith M (2016) Best practices for OTT dynamic ad insertion. In: IBC 2016 Conference. Institution of Engineering and Technology, Amsterdam, p 45. https://doi.org/10.1049/ibc.2016.0045 Somorovsky J (2016) Systematic fuzzing and testing of TLS libraries. In: Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security (CCS). ACM, pp 1492–1504. https://doi.org/10.1145/2976749.2978411 Tian C, Chen C, Duan Z, Zhao L (2019) Differential testing of certificate validation in SSL/TLS implementations: An RFC-guided approach. ACM Transactions on Software Engineering and Methodology 28:1–37. https://doi.org/10.1145/3355048 Varmarken J, Le H, Shuba A, et al (2020) The TV is smart and full of trackers: Measuring smart TV advertising and tracking. Proceedings on Privacy Enhancing Technologies 2020(2):129–154. https://doi.org/10.2478/popets-2020-0021 Wang C, Wu J, Lyu H, et al (2026) A large scale empirical analysis on the adherence gap between standards and tools in SBOM. ACM Transactions on Software Engineering and Methodology 35:1–37. https://doi.org/10.1145/3788692 web-platform-tests contributors (2026) web-platform-tests: A cross-browser test suite for the Web platform. https://web-platform-tests.org/. Accessed 21 Oct 2026 Wu S, Wang B, Zhang Y, et al (2026) When ad networks misbehave: Understanding risks of semi-drive-by splash ads. Preprint arXiv:2609.09574; to appear at ACM CCS 2026. https://arxiv.org/abs/2609.09574 Yang X, Chen Y, Eide E, Regehr J (2011) Finding and understanding bugs in C compilers. In: Proceedings of the 32nd ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI). ACM, pp 283–294. https://doi.org/10.1145/1993498.1993532 Zentrick (2021) iab-vast-loader 2.5.1, iab-vast-parser 1.2.1 and iab-vast-model 1.0.2. https://github.com/zentrick/iab-vast-loader. Accessed 20 Oct 2026

51

Related documents

• Do Open-Source Video Ad Clients Follow VAST 4.3? An Empirical Study of Player Conformance#1490163
• Operational Considerations for Streaming Media#502678
• Open Research Short Talks Project, Video 2: Open Platforms#413298
• The Economics of the Internet Backbone#942209
• Systems Performance#1331271