arXiv CSopen access

ASSEMBLAGE-DEEPHISTORY: A Cross-Build Binary Dataset with Temporal Coverage

arXiv:2605.21615v1 [cs.CR] 20 May 2026

A SSEMBLAGE -D EEP H ISTORY: A Cross-Build Binary Dataset with Temporal Coverage

Chang Liu1 Noah Fleischmann2 Nicolò Altamura3 Edward Raff4 James Holt4 Kristopher Micinski1 1 Syracuse University, Syracuse, USA 2 Booz Allen Hamilton, McLean, USA 3 Independent Researcher, Italy 4 CrowdStrike, Austin, USA {cliu57, kkmicins}@syr.edu [email protected] [email protected] {edward.raff, james.holt1}@crowdstrike.com

Abstract Existing binary corpora typically capture only one or two axes of binary variation: they either provide cross-compiler builds without a temporal axis, or CVE labels for single-build binaries. None combine cross-build diversity, cross-version history, and CVE labels into a queryable structure. We present A SSEMBLAGE D EEP H ISTORY1 , which consolidates these dimensions into a unified framework where every binary’s compilation context, source code, vulnerable functions, and package version are stored as first-class metadata. A SSEMBLAGE -D EEP H ISTORY comprises 73,610 binaries spanning 248 opensource projects, compiled across GCC, Clang, and MSVC at multiple optimization levels on Linux and Windows, with multi-year historical builds. Each binary is indexed in a database that links it to its source code, functions, debug info, variant builds, historical versions, and vulnerable functions. Three analyses demonstrate this structure’s value: (1) a three-stage LLM benchmark (recognition, strategyguided detection, and cross-build transfer) to test whether LLMs reason about binary vulnerabilities or pattern-match on build-specific artifacts; (2) a comparison of MalConv embeddings, jTrans function embeddings, and TLSH fuzzy hashes quantifying how same-package versions cluster in each space; and (3) a Bayesian regression decomposing binary similarity into contributions from temporal distance, file changes, and commits.

1

Introduction

Binary analysis research is consistently constrained by a structural gap in available datasets: real-world software evolves over time. A project is compiled across various compilers, settings, architectures, and ABIs; additionally, code is patched and refactored across updates. Existing corpora capture only fragments of this life cycle. Cross-build datasets [28, 44, 79] vary compiler settings but remain tied to a fixed version, while multi-version efforts [13, 33, 52, 95] sample many projects but typically retain only a single snapshot of each. CVE-labeled corpora generally associate each vulnerability with a specific build configuration [6, 38]. No existing resource combines cross-build diversity, multi-year version coverage, and CVE labels in a structure where compilation context, source origin, and release history are first-class, queryable metadata. 1 Dataset:

https://huggingface.co/datasets/changliu8541/assemblage-deephistory. Evaluation code: https://github.com/Assemblage-Dataset/DeepHistory-Eval. Build pipeline (integrated in Assemblage): https: //github.com/Assemblage-Dataset/Assemblage. Preprint.

Dataset

Binaries Functions Projects Cross Cross CVE Source Temporal (#) (#, K) (#) Platform Compiler Included Code Coverage

BinKit [44] BinaryCorp-26M [79] BinBench [13] LLM4Decompile [75] αdiff [51] A SSEMBLAGE [52] REALTYPE [21] Decompile-Bench [76] BinPool [6]

243,128 48,130 1,127,479 U 66,823 1,536,171 U 85,000 6,144

75,231 25,877 4,408 U 2,489 783,694 157 100,000 7

51 9,819 U 164 926 220,792 U 3,961 162

73,610

441,858

248

A SSEMBLAGE -D EEP H ISTORY

◦ ◦ ◦ ◦ ◦ • ◦ ◦ ◦ •

• ◦ ◦ ◦ ◦ ◦ ◦ ◦ ◦ •

◦ ◦ ◦ ◦ ◦ ◦ ◦ ◦ • •

◦ ◦ ◦ • ◦ • ◦ • ◦ •

◦ ◦ ◦ ◦ • ◦ ◦ ◦ ◦ •

Table 1: Comparison of binary datasets in reverse engineering, decompilation, and vulnerability research. • indicates the feature is present, ◦ indicates absent. U: undisclosed or unknown. To close this gap, we present A SSEMBLAGE -D EEP H ISTORY, comprising 73,610 binaries (42,188 Windows PE, 31,422 Linux ELF) from 248 open-source C/C++ projects compiled across GCC, Clang, and MSVC at multiple optimization levels. Projects average 5.2 distinct versions and 297 binaries; 140 projects in our corpus span more than two years of development. Every binary is associated via a database schema with its source code, internal functions, sibling builds, subsequent versions, and known vulnerabilities. Our dataset has three axes: (a) cross-build compilation, (b) multi-year version history, and (c) CVE labels. In combination, (a)–(c) enable us to do three new analyses unavailable on previous, single-axis corpora. First, we conduct several LLM reasoning benchmarks over binaries: recognition, strategy-guided detection, and cross-build transfer. The transfer stage replays the strategy authored on a reference build against compilation variants of the identical vulnerable code, separating semantic reasoning from superficial pattern matching [55, 77]. Second, leveraging the multi-version axis, we compare MalConv [64] embeddings, jTrans [79] embeddings, and TLSH [62] fuzzy hashes as signals for clustering same-package binaries. Third, a hierarchical Bayesian regression decomposes binary similarity into contributions from temporal distance, file changes, and commit frequency. In summary, our primary contributions are as follows: • A Multi-Axis Binary Dataset: We release A SSEMBLAGE -D EEP H ISTORY, a cross-platform corpus of 73,610 binaries spanning 248 open-source C/C++ projects. The dataset is indexed in a queryable database that links every binary to its source code, functions, sibling builds, version history, and 329 CVE-labeled vulnerabilities. • A Cross-Build LLM Benchmark: We construct a three-stage LLM benchmark for binary vulnerability identification. This benchmark leverages the dataset’s compilation diversity to explicitly distinguish genuine semantic reasoning from build-specific pattern matching. • Similarity and Evolution Analyses: We characterize the dataset’s similarity structure through two complementary lenses: a comparison of MalConv embeddings, jTrans embeddings, and TLSH fuzzy hashes against a hierarchical Bayesian regression that decomposes cross-version binary similarity into temporal, structural, and commit-driven components.

2

Related Work

Binary Datasets for Reverse Engineering The advancement of binary analysis research is fundamentally dependent on large-scale corpora of compiled programs. Therefore, substantial work has focused on constructing these datasets by crawling open-source repositories and employing automated build drivers (such as Make and CMake) to compile the source code at scale [7, 33, 37, 50, 56, 71, 72]. This methodology yields extensive collections of ELF binaries across different compiler versions and optimization levels, enabling downstream applications like binary similarity, function name recovery, and decompilation [8, 53, 73, 76, 78, 93]. Nevertheless, these automated compilation pipelines remain overwhelmingly restricted to Linux. While standardized build systems and package managers render headless compilation tractable in Linux environments, equivalent uniformity does not exist for Windows. 2

To address this limitation, A SSEMBLAGE [52] expanded corpus collection to Windows by compiling hundreds of thousands of projects across both operating systems, supplementing GitHubderived source code with curated vcpkg [59] packages. Other corpora target highly specific reverse-engineering domains. For instance, in the realm of similarity research, BinKit [44] provides binaries compiled across various optimization variants, while αDiff [51] contributes a massive dataset for cross-version similarity detection, the data only covers the temporal axis without crosscompiler/optimization/platform variants. Similarly, LLM4Decompile [75], NOVA [40], and IDIOMS (REALTYPE dataset) [21] focus on neural decompilation, while EMBER [4, 14, 42] and BODMASstyle malware corpora [13, 41, 69] support classification research. While these datasets significantly advance the field, they typically capture only a single snapshot of each project. Even datasets that include multiple variants fail to capture combined cross-build and cross-version dynamics with comprehensive contextual information. This leaves a critical gap in understanding how compilation choices and version drift interact over years of software evolution. Cross-Build and Multi-Version Binary Analysis Motivated by critical tasks such as patch identification, vulnerability propagation, and malware variant detection, binary similarity research has historically focused on cross-version and cross-build comparisons. Classical approaches rely on graph-matching tools [25, 26, 27, 45, 84, 92] to compare control-flow graphs across compilations. The field later shifted toward neural methods that replace graph isomorphism with learned embeddings, evolving from early architectures (Gemini, a graph-embedding similarity model [85], αDiff [51], SAFE [57], DeepBinDiff [23]) to transformer-based models with extensive pretraining (PalmTree [49], BinProv [34], jTrans [79], SigmaDiff [28]). However, while these methods are evaluated on cross-build pairs, their underlying training corpora remain inherently planar: they either vary compiler settings at a single snapshot or span a minimal version axis (e.g., adjacent releases) under a fixed build configuration. Recent library identification and version fingerprinting models [20, 22, 39, 90, 94] inherit this identical flat structure. A SSEMBLAGE -D EEP H ISTORY closes this planar gap by filling out the third axis of temporal coverage, seamlessly pairing cross-build diversity (compilers, optimization levels, operating systems) with deep, multi-year histories that span more than two years for 140 of our 248 packages. Cross-Build Vulnerability Binary Datasets for LLM Evaluation LLMs are widely suspected of succeeding on benchmarks through memorization or surface pattern matching rather than reasoning over program semantics [36, 54, 66, 83, 89, 91]. To address this, equivalence-style benchmarks such as EquiBench [83] hold program behavior fixed while varying syntactic form. At the source level, this philosophy supports a mature ecosystem: SecVulEval [2], CVE-Bench [80], SafeGenBench [48], and SEC-bench [47] all draw on richly annotated CVE corpora that track vulnerability-fixing commits across versions [9, 19, 24, 61, 67]. At the binary level, however, the landscape is far sparser. BinPool [6] labels 603 Linux CVEs on fixed compilations, and Vul-BinLLM [38] relies on synthetic Juliet cases. Because both tie each vulnerability to a single build, they leave no way to separate vulnerability semantics from build-specific artifacts. A SSEMBLAGE -D EEP H ISTORY extends the equivalence paradigm to compiled code: the underlying flaw retains its semantic identity while the compiler, optimization level, and OS may vary, ensuring that successful detection strategy must generalize across builds.

3

Dataset Details

3.1

Dataset Construction

Our corpus targets foundational C/C++ libraries characterized by well-documented multi-version histories. The selection criteria included popularity, breadth of integration, and sustained maintenance activity. To ensure our dataset remains complementary, we deliberately excluded vcpkg [59] repositories previously compiled by A SSEMBLAGE. To handle historical compilations, we employ two distinct strategies: for Windows, we utilize Conan’s standardized recipes to automate dependency resolution and compilation; for Linux, we build historical versions directly from their upstream GitHub repositories. By compiling across both operating systems, the A SSEMBLAGE -D EEP H ISTORY pipeline systematically generates Windows PE binaries (preserving MSVC PDB debug files) alongside Linux ELF binaries (retaining GCC/Clang DWARF sections). This dual-platform approach resolves two major 3

Projects

Binaries

PE

ELF

Avg ver.

Bins/Proj

By temporal coverage 0–1 year 77 1–2 years 31 2–5 years 118 5–10 years 22

13,529 10,471 42,454 7,156

5,303 7,919 25,473 3,493

8,226 2,552 16,981 3,663

2.2 5.8 6.7 6.9

176 338 360 325

By distinct versions 1 46 2–5 105 6–10 83 11–20 12 20+ 2

2,210 22,216 40,664 7,191 1,329

1,587 11,264 25,260 3,007 1,070

623 10,952 15,404 4,184 259

1.0 3.9 7.6 12.8 27.0

48 212 490 599 664

Total

73,610

42,188

31,422

5.2

297

248

Table 2: Coverage of A SSEMBLAGE -D EEP H ISTORY (library-only binaries) grouped by version span (top) and number of distinct versions per project (bottom). Avg ver. is the mean number of distinct versions per project in the binaries; Bins/Proj is the mean binaries per project.

gaps in prior Linux-centric corpora [37, 52]: it reconstructs historical Windows versions that lack centralized package snapshots (e.g., apt or yum), and natively captures the Windows ecosystem’s heavy reliance on bundled, dynamically linked libraries. 3.2

Dataset Composition

A SSEMBLAGE -D EEP H ISTORY comprises 73,610 compiled library binaries (42,188 Windows PE and 31,422 Linux ELF) sourced from 248 software packages. The corpus contains approximately 442 million functions, all indexed within a central SQLite database that preserves the mapping between source repositories, function metadata, and build configurations. Each binary is compiled across multiple configurations: Windows binaries are built with MSVC under Debug and RelWithDebInfo profiles (generating external PDB files), while Linux binaries utilize GCC and Clang with optimizations from -O0 to -O3 (retaining in-place DWARF sections). This debug information provides high-fidelity ground truth—including function boundaries, symbol names, and source-line mappings—essential for supervised function-level analysis. As detailed in Table 2, the dataset offers significant temporal depth: 140 packages span more than two years of development, with an average of 5.2 versions per project. The symbol information recovered from PDB and DWARF headers demonstrates remarkable cross-version stability; in prominent packages, tens of thousands of functions persist across numerous version pairs. This longitudinal consistency enables rigorous study of function evolution and cross-version matching without the noise introduced by heuristic boundary detection.

4

Evaluations

We demonstrate A SSEMBLAGE -D EEP H ISTORY’s research value through three analyses spanning different aspects of the dataset’s structure: Section 4.1 introduces a three-stage LLM benchmark that uses the cross-build/cross version axis to test vulnerability reasoning; Section 4.2 compares three off-the-shelf binary representations on the multi-version axis; and Section 4.3 decomposes binary similarity into temporal, structural, and activity components. 4.1

LLM Vulnerability Understanding

Experiment Setup Our benchmark contains 329 CVEs across 55 packages from A SSEMBLAGE D EEP H ISTORY, and the design is illustrated in Figure 1. CVEs are fetched by package name from the National Vulnerability Database, the CVE Program, and the GitHub Advisory Database [16, 32, 60] then filtered to the ones that affected versions intersect with compiled binary versions in the dataset. Then, we use the CVE description and patch diff to identify affected functions and we inspect about 4

Binaries

Debug info

Metadata

Source code

Task 1: Recognition

Binary Interact Agent

CVE Report

Assemblage-DeepHistory CVE database

Details

LLMs

CVE ID Strategy

Vulnerable Candidate

Sample code

Task 2: Detection

Code query / search

Task 3: Cross-Build Generalization

Mixture of vulnerable / safe code

Zero-shot / Guided CVE identification

Different binaries built w/ opt/OS/version/compiler

Mixture of source / decompiled code

Same binary as strategy generation

Replay Task 2 strategy on different binaries

Identify vulnerable code

Compare result against CVE functions

Measure accuracy decay

Figure 1: Three-Stage CVE Evaluation Design

25% of the resulting records to verify that each CVE matches the correct library and version in our dataset (manually inspected CVE IDs available in appendix). For each CVE, we chose a reference binary with lowest optimization and grouped other affected binaries to one of five Diff categories: Optimization, Compiler, OS, Version and All (per-category counts in the appendix). Experiments were conducted on a HTCondor managed cluster, and each environment was running Ubuntu 20.04.5 LTS, equipped with an AMD EPYC 9845 160-core CPU, an NVIDIA L40S GPU, and 502 GB of memory. Qwen-3.6 [87] (Qwen/Qwen3.6-35B-A3B) was served via SGLang with FP8 quantization, and Gemma 4[17] (gemma4:26b) was served via Ollama with Q4_K_M quantization ( Gemma 4 had compatibility issues with SGLang at the time of the experiments). Both models were configured with a 256K-token context window and temperature=0. Ghidra [1] version 12.0.3 was used in the evaluations. Evaluations in Section 4.1 consumed roughly 1800 L40S GPU-hours, in 8 days of queued cluster time. The remaining models used in our evaluations, Opus 4.7, GPT-5.4, and Gemini 3.1 Pro Preview (will be later referenced as Opus, GPT, and Gemini) were accessed with their latest version at the time of experiment. Binary CVE Recognition In the first and simplest task, the model selects the vulnerable function from K = 5 candidates from a single binary. The candidates are made from the vulnerable function and four distraction choices, and the four are chosen sorted by CodeBLEU [65] similarity, and shuffled before prompting. K = 5 gives a small 20% random baseline while keeping the large CVE prompts within local agents’ 256K context, avoiding long-context degradation [36, 54]. We evaluate each model under two independent factors: code representation (Ghidra-decompiled vs. original source code) and vulnerability context (zero-shot vs. with the CVE description and patch diff). Table 3 reports accuracy across five models. Note that Gemini 3.1 Pro fails to return a final answer on 1.8% even after several attempts, and these are marked as incorrect. Every model loses accuracy moving from source to binary (-0.05 to -0.20, smaller for frontier models) and gains substantially with the CVE description (+0.46 to +0.60). With description, the three frontier models reach high accuracy on both source and binary inputs; however on zero-shot tasks, all five model answers are near baseline, indicating that recognition without grounding CVE context text is close to random even for the strongest models. Guided CVE Detection In the second task, an agent aims to locate the vulnerable function in a stripped binary by exploring it through BinaryAPI, a read-only 14-method interface we built over Ghidra-based analysis. BinaryAPI exposes three code representations per function (decompiled C, annotated post-SSA p-code, and raw disassembly), bidirectional call-graph navigation, control-flow graphs and regex search across each representation. Each BinaryAPI call is a deterministic lookup against cached analysis (regex call executes at runtime), and calls execute identically across agents. The interface does not allow arbitrary code execution or debug symbols exposure. We initially conducted 100 pilot runs to set time and turn limits that give agents enough room to work without hanging the GPU cluster; 5

Setting

Qwen

Gemma

Opus

GPT

Gemini

Source, zero-shot Source, with-desc Binary, zero-shot Binary, with-desc

0.27 0.85 0.16 0.61

0.26 0.79 0.13 0.52

0.26 0.88 0.20 0.78

0.27 0.87 0.18 0.75

0.27 0.88 0.24 0.82

∆ decompilation ∆ description

−0.18 +0.51

−0.20 +0.46

−0.08 +0.60

−0.10 +0.59

−0.05 +0.60

Table 3: CVE recognition accuracy across models on CVE recognition task. ∆ decompilation is the mean accuracy drop from source to binary, averaged over the zero-shot and with-desc settings; ∆ description is the mean gain from providing the CVE description, averaged over source and binary inputs. Random baseline: 0.20.

Strategy-guided (Hit / ∆)

Solo Agent

Metric (baseline)

Gemma

Qwen

Opus

GPT

Gemini

Gemma-agent

Hit@1 Hit@5

0.00 0.02

0.15 / +0.15 0.32 / +0.32 0.29 / +0.29 0.24 / +0.24 0.29 / +0.29 0.21 / +0.19 0.34 / +0.32 0.31 / +0.29 0.27 / +0.25 0.31 / +0.29

Qwen-agent

Hit@1 Hit@5

0.03 0.12

0.38 / +0.35 0.72 / +0.69 0.83 / +0.80 0.75 / +0.72 0.78 / +0.75 0.53 / +0.41 0.82 / +0.70 0.86 / +0.74 0.81 / +0.69 0.85 / +0.73

Table 4: CVE locating performance on reference binaries (Eval 2). Each cell averages over agent results on 329 CVEs; Solo is the agent’s no-strategy baseline; ∆ is the uplift over the Solo baseline. Values are Hit@k accuracy in [0, 1].

these runs used a 30-minute, 1,000-turn cap and are not included in the final results (running-time and turn count statistics in the appendix). For the full evaluation we loosened these limits to a 1-hour wall-clock budget with no turn cap, so that agents are not cut off early. The agent terminates when the model commits to up to five candidates or returns 10 consecutive empty messages. A task scores Hit@k if any ground-truth function appears in the top-k answers; for multi-function CVEs, the best-ranked match is counted. We evaluate execution agents Gemma 4 and Qwen 3.6 in two settings: solo (locate vulnerable function given no context) and strategy-guided (strategies from five generator LLMs based on CVE, patch). Prompts are provided in the appendix. This way, we measure the generator LLMs’ understanding of vulnerability apart from its tool-calling capabilities, and the solo results provide a baseline to avoid noise introduced by agent execution. Table 4 reports solo Hit@k alongside strategy-guided Hit and uplifts. The Qwen agent benefits substantially more than the Gemma agent, reaching Hit@1 of 0.83 under Opus-authored strategies versus 0.29 for the Gemma agent on the same strategies. Opus produces the most effective strategies for the Qwen agent, leading on both Hit@1 and Hit@5, while Qwen-authored strategies lead for the Gemma agent. The Hit@1 spread across the five generators is also wider for the Qwen agent (0.38–0.83, std ≈ 0.16) than for the Gemma agent (0.15–0.32, std ≈ 0.06). This pattern suggests that tool-calling capability limits the effect of strategy quality: only when an agent can reliably execute the strategy, the strategic differences can affect the outcomes. The Qwen agent’s wider spread therefore reflects its stronger understanding and execution capability on this task. CVE Cross-Build Generalization The third task tests the generalization and understanding of the LLMs on vulnerability. We first verify that the build axes produce genuinely different decompiled code. On decompiled code pairs spanning 100 CVEs (259 Linux ELF and 194 Windows PE binaries; full table in appendix), CodeBLEU [65] drops with build divergence: 0.53–0.69 within-compiler cross-config, 0.45 cross-compiler within Linux, and 0.29 cross-OS. We execute the Eval 2 strategy-guided agents on every cross-build variant (Opt, Compiler, OS, Version, All), reusing the Eval 2 infrastructure. We report only the Qwen agent; Gemma’s reference6

5[IREKIRX ,MX$

 

















6IJ

1IER

3TX

+IQQE

'SQT

37

5[IREKIRX ,MX$



:IV

5[IR



%PP 3TYW

6IJ

+48

1IER

3TX +IQMRM

'SQT

37

:IV

%PP

1IER(MJJW

Figure 2: Cross-build transfer based on Qwen-3.6 agent. Each panel plots Hit@1 (left) or Hit@5 (right) across the Reference binary and the five Diff buckets (Opt, Compiler, OS, Version, All), with one line per strategy author LLM. The leftmost “Mean Diffs” marker pools the five Diff buckets. Gemma agent results are deferred to the appendix. build accuracy is too low across all generator LLMs for cross-build decay to be measurable, and the results are deferred to the appendix. Figure 2 plots the Qwen agent’s Hit@1 and Hit@5 across the Reference binary, five Diff buckets, and their pooled Mean. A strategy that captures vulnerability semantics should remain effective even when the binary diverges from the build it was authored against. We observe that all four strong-generator strategies accuracy decrease from Reference to the Diff mean by similar absolute amounts (Hit@1 drops of 0.12–0.15). In addition, Diff OS (CodeBLEU 0.29) is not the empirically hardest axis, and Diff Version is the lowest Hit@1 point for all four strong generators, indicating a shared blind spot across these strategies on software evolution. Hit@5 shows a similar ordering at higher values, with Opus and Gemini effectively tied at the Diff mean (≈ 0.7). 4.2

Package-Level Binary Similarity

To test how well off-the-shelf binary representations cluster same-package versions, we compare three signals on A SSEMBLAGE -D EEP H ISTORY: MalConv embeddings [64], a CNN pretrained on Windows PE for malware classification; jTrans [79], a transformer pretrained for binary function similarity; and TLSH fuzzy hashes [62], a locality-sensitive byte hash. All three are evaluated as out-of-the-box, untuned representations. The per-pair decomposition of similarity into temporal, structural, and activity components is the subject of Section 4.3; here we measure only what each representation provides under arbitrary cross-build pairs. Of A SSEMBLAGE -D EEP H ISTORY’s 248 projects, 52 have only one version or don’t come with .dll/.so/.lib and are excluded. For TLSH we follow the original paper’s practice [62] and treat distance ≤ 100 as similar; each package cell reports the fraction of within-package pairs meeting that threshold. Figure 3 shows package-mean similarity matrices for all three signals. The two MalConv panels (left, middle) show only modest separation between intra- and inter-package pairs on the full corpus (pair-level Cohen’s d = +0.21), and the gap remains similarly limited on the Windows PE subpopulation alone. Because the same pattern holds on the PE binaries that match MalConv’s training distribution, the limited separation cannot be attributed to applying PE weights to ELF inputs. Therefore, pretrained MalConv weights do not provide a usable signal for package-level similarity or version-differentiation tasks on this corpus. jTrans (third panel) shows modestly better separation (intra-package mean 0.771 vs. cross-package mean 0.657), but both means remain high in absolute terms, indicating that a transformer pretrained for binary function similarity also fails to cleanly distinguish same-package from different-package binaries on this corpus. TLSH, in contrast, yields a sparse similarity space (right panel): intra-package mean 0.277 versus cross-package mean 0.002 (with pair-level Cohen’s d = +1.36). TLSH also recovers genuine crosspackage code reuse, for example libnghttp2↔nghttp3 at 0.345 captures shared structure between related HTTP/2 and HTTP/3 implementations. TLSH is therefore the most usable of the three off-the-shelf signals on A SSEMBLAGE -D EEP H ISTORY, supporting both within-package coherence 7

intra = 0.950 inter = 0.894

intra = 0.925 inter = 0.890

intra = 0.771 inter = 0.657

intra = 0.277 inter = 0.002 0.8 0.6 0.4 0.2

MalConv (PE, package mean)

jTrans (package mean)

MalConv (package mean)

TLSH (package mean)

Mean similarity

1.0

0.0

Figure 3: Package-level binary similarity on A SSEMBLAGE -D EEP H ISTORY’s ≥ 2-version subset (ELF + PE combined). From left to right: MalConv embedding cosine similarity, PE only; MalConv embedding cosine similarity, all packages-mean; jTrans embedding cosine similarity, all packagesmean; TLSH fuzzy-hash similarity, all packages-mean.

bias days commits

MalConv TLSH jTrans

file change −0.5

0.0

0.5

1.0

1.5

2.0

2.5

3.0

Logit impact (95% HDI)

Figure 4: Global coefficient posterior means and 95% HDIs for MalConv cosine similarity, jTrans cosine similarity, and transformed TLSH similarity. Using three methods of similarity, we can gain better insights into the likely strong factors of code variation.

and detection of cross-package shared code; the strong intra/inter contrast further confirms that A SSEMBLAGE -D EEP H ISTORY captures meaningful variation across packages. 4.3

Hierarchical Bayesian Regression

A SSEMBLAGE -D EEP H ISTORY also allows for investigation of the relationship between code evolution and binary similarity, which can be critical for maintaining databases of known functions that are used for both benign and malicious purposes (e.g., encryption). We perform a Bayesian analysis to demonstrate the avenue of research this enables, comparing MalConv cosine similarity, jTrans cosine similarity, and TLSH transformed similarity as response variables under the same hierarchical model. The model is computed using Markov Chain Monte Carlo [58] with the NUTS sampler [35] and implemented with Numpyro [63]. The R̂ scores for all reported global coefficients were ≤ 1.01, indicating convergence [31]. All distributions were Gaussian or half-Gaussian (positive values only) as appropriate, and the target variable was modeled as a Beta regression: for each release pair i, the observed similarity yi ∈ (0, 1) is modeled as yi ∼ Beta(µi κ, (1 − µi )κ), where logit(µi ) is a linear function of standardized covariates and project-specific coefficients, with concentration κ ∈ R+ . As TLSH produces an unbounded nonnegative distance rather than a similarity score, we transform TLSH distance d into a Beta-regression response using s = (1 + exp((d − m)/τ ))−1 , where m is the median TLSH distance and τ is chosen so that the 90th percentile maps to 0.1 (current values from data: m = 142.35, τ = 33.44). We then apply the standard open-interval squeeze so that all responses lie in (0, 1). The Bayesian model analysis sought to understand the factors that influence binary similarity across versions. Each feature of interest is modeled with a hyper-prior (the global rate), which is used as the prior for a project-specific version of the covariate. In this way, the Bayesian approach shares information across all projects while also recognizing that the multiple compilations of one project have an intra-source similarity that should not influence other projects [30]. 8

We constructed the model using three covariates: the number of days between each pair of releases, the number of commits between them, and the number of changed source files normalized by the total source-file count in the base release. We fit the same hierarchical model separately for MalConv cosine similarity, jTrans cosine similarity, and transformed TLSH similarity. The temporal distribution of releases in A SSEMBLAGE -D EEP H ISTORY is non-uniform: release activity grows steadily, peaking in 2023–2024 with sustained activity throughout. As a result, many version pairs have short temporal separations while fewer span the full range, directly influencing how we interpret the relationship between release timing and binary similarity. Given these temporal characteristics, we first validate that the model appropriately captures the underlying relationships using a posterior predictive check that compares predictions with observed values [29]. Resolving a model that “fully captures” the distribution, or determining whether such a model exists, is beyond the scope of this article which introduces the dataset and demonstrates the potential research value. We can examine the hyper-prior’s Highest Density Interval (HDI, a Bayesian counterpart to the frequentist confidence interval [46]) to determine if a statistically significant relationship exists between the covariates and outcome (file similarity). The strongest consistent covariate effect is normalized file change, which is negative for both MalConv and TLSH, while the HDI crosses zero for jTrans, with a small negative posterior mean. This indicates that releases touching a larger fraction of the source tree tend to produce less similar binaries, consistent with widespread file changes signaling architectural refactoring or dependency updates rather than incremental feature additions. Commit count differs by metric: its HDI crosses zero for MalConv and jTrans, but is credibly negative for TLSH, where commits carry the largestmagnitude effect of any covariate. This suggests TLSH is more sensitive than MalConv and jTrans to accumulated byte-level change across active development histories. Calendar time has a negative posterior mean for all three metrics. Its HDI is credibly negative for MalConv but crosses zero for TLSH and jTrans, so release age is a stable predictor only for the MalConv embedding similarity, not for TLSH or jTrans after accounting for commits and file change. The large positive MalConv and jTrans biases reflect the high baseline similarity of these embeddings across versions, consistent with the saturation observed in Section 4.2. The TLSH bias is closer to zero after the sigmoid distance transform, reflecting a less saturated similarity scale in which structural divergence is more readily expressed. The results from Figure 4 are again the hyper-prior measuring global rates. A Bayesian model allows us to simultaneously estimate the per-project local rates. Inspecting the per-project offsets from the global prior shows that most projects cluster near the global rate, which is necessary for the linear model to be a reasonable choice, as confirmed by the posterior predictive check. The per-project offsets also exhibit covariate-specific heterogeneity, indicating author or project-specific differences in development cadence and providing insight into factors influencing binary evolution beyond the shared global rate.

5

Conclusion, Limitations, and Future Work

A SSEMBLAGE -D EEP H ISTORY unifies cross-build diversity, multi-year version history, and CVEannotated vulnerability labels in a single queryable corpus, linking each binary to its source, sibling builds, historical versions, and vulnerability metadata. Across our three evaluations, a common lesson emerges: signals that appear stable in isolation—an LLM agent’s strategy, a pretrained embedding, or a binary-similarity score—may change once the same source-level semantics are observed across different builds and releases. These results show why these separate axes of variation must be studied jointly: their interaction exposes effects that are obscured by single-axis corpora. Limitations CVE coverage concentrates on 55 of the 248 projects, reflecting the uneven distribution of vulnerability disclosure across open source. In addition, A SSEMBLAGE -D EEP H ISTORY targets x86 due to its dominant share and because our build hosts are natively x86; ARM or RISC-V coverage would require cross-compilation or a different host architecture, which is left to future work. A SSEMBLAGE -D EEP H ISTORY’s vulnerability component is intended for defensive research and carries the dual-use tension common to any CVE-labeled corpus. We mitigate this by including only publicly disclosed, already-patched CVEs whose vulnerable functions and patch diffs already appear in the NVD and upstream commits. The source is public and our binaries are built unmodified from upstream releases, thus the corpus adds no offensive capability. 9

Future Work A SSEMBLAGE -D EEP H ISTORY is built from unmodified upstream source; adversarially obfuscated binaries are out of scope despite a rich literature and trend on obfuscation and obfuscation-resilient analysis [3, 10, 11, 12, 18, 43, 68, 70, 74, 81, 86, 88]. Meanwhile, the rise of Apple Silicon, ARM-based servers, including the growing numbers of embedded architectures (e.g. RISC-V) has expanded the scope of binary analysis beyond x86 and Linux [5, 15, 82]. A fully comprehensive cross-build dataset would extend along the platform axis as well, covering macOS and ARM/RISC-V in addition to the configurations we currently provide.

6

Acknowledgments

This work was supported by NSF award CCF-2316159. This work was also supported in part through computational resources provided by Syracuse University. The authors gratefully acknowledge use of the OrangeGrid / HTC Campus Grid, supported by NSF award ACI-1341006, and technical support from Syracuse University’s Cyberinfrastructure Engineer, supported by NSF award ACI-1541396.

References [1] [2]

[3]

[4]

[5]

[6]

[7]

[8]

[9]

[10]

National Security Agency. Ghidra Software Reverse Engineering Framework. https:// github.com/nationalsecurityagency/ghidra. accessed 2026-05-06. 2019. Md Basim Uddin Ahmed, Nima Shiri Harzevili, Jiho Shin, Hung Viet Pham, and Song Wang. SecVulEval: Benchmarking LLMs for Real-World C/C++ Vulnerability Detection. 2025. URL: https://arxiv.org/abs/2505.19828. Nicolò Altamura, Enrico Bragastini, Marco Campion, and Mila Dalla Preda. “Assessing the Effectiveness of the Tigress Obfuscator Against MOPSA and BinaryNinja”. In: Proceedings of the 2025 Workshop on Research on Offensive and Defensive Techniques in the Context of Man At The End (MATE) Attacks. 2025. URL: https://doi.org/10.1145/3733817.3762702. H. Anderson and Phil Roth. “EMBER: An Open Dataset for Training Static PE Malware Machine Learning Models”. In: ArXiv (2018). URL: https://api.semanticscholar.org/ CorpusID:4888440. Apple. Apple debuts M5 Pro and M5 Max to supercharge the most demanding pro workflows. Apple Newsroom. Accessed: 2026-05-06. 2026. URL: https://www.apple.com/ newsroom/2026/03/apple- debuts- m5- pro- and- m5- max- to- supercharge- themost-demanding-pro-workflows/. Sima Arasteh, Georgios Nikitopoulos, Wei-Cheng Wu, Nicolaas Weideman, Aaron Portnoy, Mukund Raghothaman, and Christophe Hauser. “BinPool: A Dataset of Vulnerabilities for Binary Security Analysis”. In: Proceedings of the 33rd ACM International Conference on the Foundations of Software Engineering. 2025. Konstantinos Barmpis, Patrick Neubauer, Jonathan Co, Dimitris Kolovos, Nicholas Matragkas, and Richard F. Paige. “Polyglot and Distributed Software Repository Mining with Crossflow”. In: Proceedings of the 17th International Conference on Mining Software Repositories. 2020. URL: https://doi.org/10.1145/3379597.3387481. Zion Leonahenahe Basque, Ati Priya Bajaj, Wil Gibbs, Jude O’Kain, Derron Miao, Tiffany Bao, Adam Doupé, Yan Shoshitaishvili, and Ruoyu Wang. “Ahoy SAILR! There is No Need to DREAM of C: A Compiler-Aware Structuring Algorithm for Binary Decompilation”. In: 33rd USENIX Security Symposium (USENIX Security 24). 2024. URL: https://www.usenix. org/conference/usenixsecurity24/presentation/basque. Guru Bhandari, Amara Naseer, and Leon Moonen. “CVEfixes: automated collection of vulnerabilities and their fixes from open-source software”. In: Proceedings of the 17th International Conference on Predictive Models and Data Analytics in Software Engineering. 2021. URL: http://dx.doi.org/10.1145/3475960.3475985. Tim Blazytko, Moritz Contag, Cornelius Aschermann, and Thorsten Holz. “Syntia: Synthesizing the semantics of obfuscated code”. In: 26th USENIX Security Symposium (USENIX Security 17). 2017.

10

[11] Christian Collberg, Sam Martin, Jonathan Myers, Bill Zimmerman, Petr Krajca, Gabriel Kerneis, Saumya Debray, and Babak Yadegari. “The tigress c diversifier/obfuscator”. In: Retrieved August (2015). [12] Christian Collberg, Clark Thomborson, and Douglas Low. A taxonomy of obfuscating transformations. 1997. [13] Francesca Console, Giuseppe D’Aquanno, Giuseppe Antonio Di Luna, and Leonardo Querzoni. “BinBench: a benchmark for x64 portable operating system interface binary function representations”. In: PeerJ Computer Science (2023). URL: https://api.semanticscholar.org/ CorpusID:259029804. [14] Dragos Georgian Corlatescu, Alexandru Dinu, Mihaela Gaman, and Paul Sumedrea. “EMBERSim: A Large-Scale Databank for Boosting Similarity Search in Malware Analysis”. In: ArXiv (2023). URL: https://api.semanticscholar.org/CorpusID:263608542. [15] Enfang Cui, Tianzheng Li, and Qian Wei. “RISC-V Instruction Set Architecture Extensions: A Survey”. In: IEEE Access (2023). [16] CVE Program. Common Vulnerabilities and Exposures (CVE). https://www.cve.org/. Accessed: 2026-05-06. [17] Google DeepMind. Gemma 4 model card. https://ai.google.dev/gemma/docs/core/ model_card_4. accessed 2026-05-20. 2026. [18] Steven HH Ding, Benjamin CM Fung, and Philippe Charland. “Asm2vec: Boosting static representation robustness for binary clone search against code obfuscation and compiler optimization”. In: 2019 ieee symposium on security and privacy (sp). 2019. [19] Yangruibo Ding, Yanjun Fu, Omniyyah Ibrahim, Chawin Sitawarin, Xinyun Chen, Basel Alomair, David Wagner, Baishakhi Ray, and Yizheng Chen. “Vulnerability Detection with Code Language Models: How Far Are We?” In: arXiv preprint arXiv:2403.18624 (2024). [20] Chaopeng Dong, Siyuan Li, Shouguo Yang, Yang Xiao, Yongpan Wang, Hong Li, Zhi Li, and Limin Sun. “LibvDiff: Library Version Difference Guided OSS Version Identification in Binaries”. In: Proceedings of the IEEE/ACM 46th International Conference on Software Engineering. 2024. URL: https://doi.org/10.1145/3597503.3623336. [21] Luke Dramko, Claire Le Goues, and Edward J. Schwartz. Idioms: Neural Decompilation With Joint Code and Type Definition Prediction. 2025. URL: https://arxiv.org/abs/2502. 04536. [22] Ruian Duan, Ashish Bijlani, Meng Xu, Taesoo Kim, and Wenke Lee. “Identifying OpenSource License Violation and 1-day Security Risk at Large Scale”. In: Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security. 2017. URL: https://doi.org/10.1145/3133956.3134048. [23] Yue Duan, Xuezixiang Li, Jinghan Wang, and Heng Yin. “DeepBinDiff: Learning ProgramWide Code Representations for Binary Diffing”. In: 27th Annual Network and Distributed System Security Symposium, NDSS 2020, San Diego, California, USA, February 23-26, 2020. 2020. URL : https://www.ndss- symposium.org/ndss- paper/deepbindiff- learningprogram-wide-code-representations-for-binary-diffing/. [24] Jiahao Fan, Yi Li, Shaohua Wang, and Tien N. Nguyen. “A C/C++ Code Vulnerability Dataset with Code Changes and CVE Summaries”. In: Proceedings of the 17th International Conference on Mining Software Repositories. 2020. URL: https://doi.org/10.1145/3379597. 3387501. [25] Qian Feng, Rundong Zhou, Chengcheng Xu, Yao Cheng, Brian Testa, and Heng Yin. “Scalable graph-based bug search for firmware images”. In: Proceedings of the 2016 ACM SIGSAC conference on computer and communications security. 2016. [26] Halvar Flake. “Structural comparison of executable objects”. In: Detection of intrusions and malware & vulnerability assessment, GI SIG SIDAR workshop, DIMVA 2004. 2004. [27] Debin Gao, Michael K. Reiter, and Dawn Song. “BinHunt: Automatically Finding Semantic Differences in Binary Programs”. In: Information and Communications Security: 10th International Conference, ICICS 2008 Birmingham, UK, October 20 - 22, 2008 Proceedings. 2008. URL: https://doi.org/10.1007/978-3-540-88625-9_16.

11

[28] Lian Gao, Yu Qu, Sheng Yu, Yue Duan, and Heng Yin. “SigmaDiff: Semantics-Aware Deep Graph Matching for Pseudocode Diffing”. In: Proceedings 2024 Network and Distributed System Security Symposium (2024). URL: https://api.semanticscholar.org/CorpusID: 262144278. [29] Andrew Gelman. “Prior distributions for variance parameters in hierarchical models (comment on article by Browne and Draper)”. In: Bayesian Analysis (2006). URL: https://doi.org/ 10.1214/06-BA117A. [30] Andrew Gelman, Jennifer Hill, and Masanao Yajima. “Why We (Usually) Don’t Have to Worry About Multiple Comparisons”. In: Journal of Research on Educational Effectiveness (2012). URL: https://doi.org/10.1080/19345747.2011.618213. [31] Andrew Gelman and Donald B Rubin. “Inference from iterative simulation using multiple sequences”. In: Statistical science (1992). [32] GitHub. GitHub Advisory Database. https://github.com/advisories. Accessed: 202605-06. [33] Georgios Gousios. “The GHTorent dataset and tool suite”. In: 2013 10th Working Conference on Mining Software Repositories (MSR). 2013. [34] Xu He, Shu Wang, Yunlong Xing, Pengbin Feng, Haining Wang, Qi Li, Songqing Chen, and Kun Sun. “BinProv: Binary Code Provenance Identification without Disassembly”. In: Proceedings of the 25th International Symposium on Research in Attacks, Intrusions and Defenses (2022). URL: https://api.semanticscholar.org/CorpusID:252910574. [35] Matthew D. Hoffman and Andrew Gelman. “The No-U-turn sampler: adaptively setting path lengths in Hamiltonian Monte Carlo”. In: J. Mach. Learn. Res. (2011). URL: https: //api.semanticscholar.org/CorpusID:12948548. [36] Cheng-Ping Hsieh, Simeng Sun, Samuel Kriman, Shantanu Acharya, Dima Rekesh, Fei Jia, Yang Zhang, and Boris Ginsburg. “RULER: What’s the real context size of your long-context language models?” In: arXiv preprint arXiv:2404.06654 (2024). [37] Zecong Hu and Jeremy Lacomis. GitHub Cloner & Compiler. 2020. URL: https://github. com/huzecong/ghcc. [38] Nasir Hussain, Haohan Chen, Chanh Tran, Philip Huang, Zhuohao Li, Pravir Chugh, William Chen, Ashish Kundu, and Yuan Tian. VulBinLLM: LLM-powered Vulnerability Detection for Stripped Binaries. 2025. URL: https://arxiv.org/abs/2505.22010. [39] Ling Jiang, Junwen An, Huihui Huang, Qiyi Tang, Sen Nie, Shi Wu, and Yuqun Zhang. BinaryAI: Binary Software Composition Analysis via Intelligent Binary Source Code Matching. 2024. URL: https://arxiv.org/abs/2401.11161. [40] Nan Jiang, Chengxiao Wang, Kevin Liu, Xiangzhe Xu, Lin Tan, Xiangyu Zhang, and Petr Babkin. Nova: Generative Language Models for Assembly Code with Hierarchical Attention and Contrastive Learning. 2025. URL: https://arxiv.org/abs/2311.13721. [41] Robert J. Joyce, Dev Amlani, Charles Nicholas, and Edward Raff. MOTIF: A Large Malware Reference Dataset with Ground Truth Family Labels. 2021. URL: https://arxiv.org/abs/ 2111.15031. [42] Robert J. Joyce, Gideon Miller, Phil Roth, Richard Zak, Elliott Zaresky-Williams, Hyrum Anderson, Edward Raff, and James Holt. “EMBER2024 - A Benchmark Dataset for Holistic Evaluation of Malware Classifiers”. In: Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2. 2025. URL: http://dx.doi.org/10.1145/ 3711896.3737431. [43] Pascal Junod, Julien Rinaldini, Johan Wehrli, and Julie Michielin. “Obfuscator-LLVM – Software Protection for the Masses”. In: 2015 IEEE/ACM 1st International Workshop on Software Protection. 2015. [44] Dongkwan Kim, Eunsoo Kim, Sang Kil Cha, Sooel Son, and Yongdae Kim. “Revisiting Binary Code Similarity Analysis Using Interpretable Feature Engineering and Lessons Learned”. In: IEEE Transactions on Software Engineering (2023). URL: http://dx.doi.org/10.1109/ TSE.2022.3187689. [45] Joxean Koret. Diaphora. https://github.com/joxeankoret/diaphora. 12

[46] John Kruschke. “Doing Bayesian data analysis: A tutorial with R, JAGS, and Stan”. In: (2014). [47] Hwiwon Lee, Ziqi Zhang, Hanxiao Lu, and Lingming Zhang. SEC-bench: Automated Benchmarking of LLM Agents on Real-World Software Security Tasks. 2025. URL: https://arxiv. org/abs/2506.11791. [48] Xinghang Li, Jingzhe Ding, Chao Peng, Bing Zhao, Xiang Gao, Hongwan Gao, and Xinchen Gu. SafeGenBench: A Benchmark Framework for Security Vulnerability Detection in LLMGenerated Code. 2025. URL: https://arxiv.org/abs/2506.05692. [49] Xuezixiang Li, Yu Qu, and Heng Yin. “PalmTree: Learning an Assembly Language Model for Instruction Embedding”. In: Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security. 2021. URL: http : / / dx . doi . org / 10 . 1145 / 3460120 . 3484587. [50] Erik Linstead, Paul Rigor, Sushil Bajracharya, Cristina Lopes, and Pierre Baldi. “Mining Internet-Scale Software Repositories”. In: Advances in Neural Information Processing Systems. 2007. URL: https://proceedings.neurips.cc/paper_files/paper/2007/file/ a532400ed62e772b9dc0b86f46e583ff-Paper.pdf. [51] Bingchang Liu, Wei Huo, Chao Zhang, Wenchao Li, Feng Li, Aihua Piao, and Wei Zou. “α Diff: Cross-Version Binary Code Similarity Detection with DNN”. In: 2018 33rd IEEE/ACM International Conference on Automated Software Engineering (ASE). 2018. [52] Chang Liu, Rebecca Saul, Yihao Sun, Edward Raff, Maya Fuchs, Townsend Southard Pantano, James Holt, and Kristopher Micinski. Assemblage: Automatic Binary Dataset Construction for Machine Learning. 2024. URL: https://arxiv.org/abs/2405.03991. [53] Chang Liu, Yihao Sun, Thomas Gilray, and Kristopher Micinski. Superset Decompilation. 2026. URL: https://arxiv.org/abs/2603.28002. [54] Nelson F Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. “Lost in the middle: How language models use long contexts”. In: Transactions of the association for computational linguistics (2024). [55] Li Lu, Yanjie Zhao, Hongzhou Rao, Kechi Zhang, and Haoyu Wang. Evaluating and Enhancing the Vulnerability Reasoning Capabilities of Large Language Models. 2026. URL: https : //arxiv.org/abs/2602.06687. [56] Andrea Marcelli, Mariano Graziano, Xabier Ugarte-Pedrero, Yanick Fratantonio, Mohamad Mansouri, and Davide Balzarotti. “How Machine Learning Is Solving the Binary Function Similarity Problem”. In: 31st USENIX Security Symposium (USENIX Security 22). 2022. URL : https://www.usenix.org/conference/usenixsecurity22/presentation/ marcelli. [57] Luca Massarelli, Giuseppe Antonio Di Luna, Fabio Petroni, Leonardo Querzoni, and Roberto Baldoni. SAFE: Self-Attentive Function Embeddings for Binary Similarity. 2019. URL: https: //arxiv.org/abs/1811.05296. [58] Nicholas Metropolis, Arianna W Rosenbluth, Marshall N Rosenbluth, Augusta H Teller, and Edward Teller. “Equation of state calculations by fast computing machines”. In: The journal of chemical physics (1953). [59] Microsoft. vcpkg. https://github.com/microsoft/vcpkg. 2024. [60] National Institute of Standards and Technology. National Vulnerability Database. https: //nvd.nist.gov. Accessed: 2026-05-06. [61] Chao Ni, Liyu Shen, Xiaohu Yang, Yan Zhu, and Shaohua Wang. “MegaVul: A C/C++ Vulnerability Dataset with Comprehensive Code Representations”. In: 2024 IEEE/ACM 21st International Conference on Mining Software Repositories (MSR). 2024. [62] Jonathan Oliver, Chun Cheng, and Yanggui Chen. “TLSH–a locality sensitive hash”. In: 2013 fourth cybercrime and trustworthy computing workshop. 2013. [63] Du Phan, Neeraj Pradhan, and Martin Jankowiak. Composable Effects for Flexible and Accelerated Probabilistic Programming in NumPyro. 2019. URL: https://arxiv.org/abs/ 1912.11554. [64] Edward Raff, William Fleshman, Richard Zak, Hyrum S. Anderson, Bobby Filar, and Mark McLean. Classifying Sequences of Extreme Length with Constant Memory Applied to Malware Detection. 2020. URL: https://arxiv.org/abs/2012.09390. 13

[65] Shuo Ren, Daya Guo, Shuai Lu, Long Zhou, Shujie Liu, Duyu Tang, Neel Sundaresan, Ming Zhou, Ambrosio Blanco, and Shuai Ma. CodeBLEU: a Method for Automatic Evaluation of Code Synthesis. 2020. URL: https://arxiv.org/abs/2009.10297. [66] Martin Riddell, Ansong Ni, and Arman Cohan. Quantifying Contamination in Evaluating Code Generation Capabilities of Language Models. 2024. URL: https://arxiv.org/abs/ 2403.04811. [67] Bonan Ruan, Jiahao Liu, Weibo Zhao, and Zhenkai Liang. “VulZoo: A Comprehensive Vulnerability Intelligence Dataset”. In: Proceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering. 2024. URL: https://doi.org/10.1145/ 3691620.3695345. [68] Jonathan Salwan, Sébastien Bardin, and Marie-Laure Potet. “Symbolic deobfuscation: From virtualized code back to the original”. In: International Conference on Detection of Intrusions and Malware, and Vulnerability Assessment. 2018. [69] Rebecca Saul, Chang Liu, Noah Fleischmann, Richard Zak, Kristopher Micinski, Edward Raff, and James Holt. “Is Function Similarity Over-Engineered? Building a Benchmark”. In: Advances in Neural Information Processing Systems. 2024. [70] Moritz Schloegel, Tim Blazytko, Moritz Contag, Cornelius Aschermann, Julius Basler, Thorsten Holz, and Ali Abbasi. “Loki: Hardening code obfuscation against automated attacks”. In: 31st USENIX Security Symposium (USENIX Security 22). 2022. [71] Huajie Shao, Dachun Sun, Jiahao Wu, Zecheng Zhang, Aston Zhang, Shuochao Yao, Shengzhong Liu, Tianshi Wang, Chao Zhang, and Tarek Abdelzaher. “paper2repo: GitHub Repository Recommendation for Academic Papers”. In: Proceedings of The Web Conference 2020. 2020. URL: http://dx.doi.org/10.1145/3366423.3380145. [72] Mohammad Behnam Shariati, Ali Dehghantanha, Ben Martini, and Kim-Kwang Raymond Choo. “Ubuntu One investigation: Detecting evidences on client machines”. In: The Cloud Security Ecosystem. 2015. URL: https://api.semanticscholar.org/CorpusID:33377904. [73] Yan Shoshitaishvili, Ruoyu Wang, Christopher Salls, Nick Stephens, Mario Polino, Andrew Dutcher, John Grosen, Siji Feng, Christophe Hauser, Christopher Kruegel, and Giovanni Vigna. “SOK: (State of) The Art of War: Offensive Techniques in Binary Analysis”. In: 2016 IEEE Symposium on Security and Privacy (SP). 2016. [74] Ashwin Sudhir, Zion Leonahenahe Basque, Wil Gibbs, Ati Priya Bajaj, Pulkit Singh Singaria, Mitchell Zakocs, Jie Hu, Moritz Schloegel, Tiffany Bao, Adam Doupe, Yan Shoshitaishvili, and Ruoyu Wang. Pushan: Trace-Free Deobfuscation of Virtualization-Obfuscated Binaries. 2026. URL: https://arxiv.org/abs/2603.18355. [75] Hanzhuo Tan, Qi Luo, Jing Li, and Yuqun Zhang. LLM4Decompile: Decompiling Binary Code with Large Language Models. 2024. [76] Hanzhuo Tan, Xiaolong Tian, Hanrui Qi, Jiaming Liu, Zuchen Gao, Siyi Wang, Qi Luo, Jing Li, and Yuqun Zhang. Decompile-Bench: Million-Scale Binary-Source Function Pairs for Real-World Binary Decompilation. 2025. URL: https://arxiv.org/abs/2505.12668. [77] Saad Ullah, Mingji Han, Saurabh Pujar, Hammond Pearce, Ayse Coskun, and Gianluca Stringhini. LLMs Cannot Reliably Identify and Reason About Security Vulnerabilities (Yet?): A Comprehensive Evaluation, Framework, and Benchmarks. 2024. URL: https://arxiv.org/ abs/2312.12575. [78] Fish Wang and Yan Shoshitaishvili. “Angr - The Next Generation of Binary Analysis”. In: 2017 IEEE Cybersecurity Development (SecDev). 2017. [79] Hao Wang, Wenjie Qu, Gilad Katz, Wenyu Zhu, Zeyu Gao, Han Qiu, Jianwei Zhuge, and Chao Zhang. jTrans: Jump-Aware Transformer for Binary Code Similarity. 2022. URL: https: //arxiv.org/abs/2205.12713. [80] Peiran Wang, Xiaogeng Liu, and Chaowei Xiao. “CVE-Bench: Benchmarking LLM-based Software Engineering Agent’s Ability to Repair Real-World CVE Vulnerabilities”. In: Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). 2025. URL: https://aclanthology.org/2025.naacl-long.212/.

14

[81] Yufeng Wang, Yuhong Feng, Yixuan Cao, Haoran Li, Haiyue Feng, and Yifeng Wang. “ORCAS: Obfuscation-Resilient Binary Code Similarity Analysis using Dominance Enhanced Semantic Graph”. In: Proceedings of the 34th ACM International Conference on Information and Knowledge Management. 2025. URL: http://dx.doi.org/10.1145/3746252.3761266. [82] Andrew Waterman, Yunsup Lee, David A Patterson, and Krste Asanovic. “The risc-v instruction set manual, volume i: Base user-level isa”. In: EECS Department, UC Berkeley, Tech. Rep. UCB/EECS-2011-62 (2011). [83] Anjiang Wei, Jiannan Cao, Ran Li, Hongyu Chen, Yuhui Zhang, Ziheng Wang, Yuan Liu, Thiago SFX Teixeira, Diyi Yang, Ke Wang, et al. “Equibench: Benchmarking large language models’ reasoning about program semantics via equivalence checking”. In: Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 2025. [84] Yang Xiao, Bihuan Chen, Chendong Yu, Zhengzi Xu, Zimu Yuan, Feng Li, Binghong Liu, Yang Liu, Wei Huo, Wei Zou, and Wenchang Shi. “MVP: Detecting Vulnerabilities using Patch-Enhanced Vulnerability Signatures”. In: 29th USENIX Security Symposium (USENIX Security 20). 2020. URL: https://www.usenix.org/conference/usenixsecurity20/ presentation/xiao. [85] Xiaojun Xu, Chang Liu, Qian Feng, Heng Yin, Le Song, and Dawn Song. “Neural Networkbased Graph Embedding for Cross-Platform Binary Code Similarity Detection”. In: Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security. 2017. URL: http://dx.doi.org/10.1145/3133956.3134018. [86] Babak Yadegari, Brian Johannesmeyer, Ben Whitely, and Saumya Debray. “A Generic Approach to Automatic Deobfuscation of Executable Code”. In: 2015 IEEE Symposium on Security and Privacy. 2015. [87] An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jing Zhou, Jingren Zhou, Junyang Lin, Kai Dang, Keqin Bao, Kexin Yang, Le Yu, Lianghao Deng, Mei Li, Mingfeng Xue, Mingze Li, Pei Zhang, Peng Wang, Qin Zhu, Rui Men, Ruize Gao, Shixuan Liu, Shuang Luo, Tianhao Li, Tianyi Tang, Wenbiao Yin, Xingzhang Ren, Xinyu Wang, Xinyu Zhang, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yinger Zhang, Yu Wan, Yuqiong Liu, Zekun Wang, Zeyu Cui, Zhenru Zhang, Zhipeng Zhou, and Zihan Qiu. Qwen3 Technical Report. 2025. URL: https://arxiv.org/abs/2505.09388. [88] Shouguo Yang, Long Cheng, Yicheng Zeng, Zhe Lang, Hongsong Zhu, and Zhiqiang Shi. “Asteria: Deep Learning-based AST-Encoding for Cross-platform Binary Code Similarity Detection”. In: 2021 51st Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN). 2021. URL: http://dx.doi.org/10.1109/DSN48987.2021.00036. [89] Shuai Yang, Qi Yang, Luoxi Tang, Yuqiao Meng, Nancy Guo, Jeremy Blackburn, and Zhaohan Xi. On the Eligibility of LLMs for Counterfactual Reasoning: A Decompositional Study. 2026. URL: https://arxiv.org/abs/2505.11839. [90] Zimu Yuan, Muyue Feng, Feng Li, Gu Ban, Yang Xiao, Shiyang Wang, Qian Tang, He Su, Chendong Yu, Jiahuan Xu, Aihua Piao, Jingling Xuey, and Wei Huo. “B2SFinder: Detecting Open-Source Software Reuse in COTS Software”. In: 2019 34th IEEE/ACM International Conference on Automated Software Engineering (ASE). 2019. [91] Jin Peng Zhou, Charles Staats, Wenda Li, Christian Szegedy, Kilian Q. Weinberger, and Yuhuai Wu. Don’t Trust: Verify – Grounding LLM Quantitative Reasoning with Autoformalization. 2024. URL: https://arxiv.org/abs/2403.18120. [92] Wenyu Zhu, Hao Wang, Yuchen Zhou, Jiaming Wang, Zihan Sha, Zeyu Gao, and Chao Zhang. kTrans: Knowledge-Aware Transformer for Binary Code Embedding. 2023. URL: https://arxiv.org/abs/2308.12659. [93] Muqi Zou, Hongyu Cai, Hongwei Wu, Zion Leonahenahe Basque, Arslan Khan, Berkay Celik, Dave, Tian, Antonio Bianchi, Ruoyu, Wang, and Dongyan Xu. D-LiFT: Improving LLM-based Decompiler Backend via Code Quality-driven Fine-tuning. 2025. URL: https: //arxiv.org/abs/2506.10125.

15

[94] Yayi Zou, Yixiang Zhang, Guanghao Zhao, Yueming Wu, Shuhao Shen, and Cai Fu. BinCoFer: Three-Stage Purification for Effective C/C++ Binary Third-Party Library Detection. 2025. URL: https://arxiv.org/abs/2504.19551. [95] Fei Zuo, Cody Tompkins, Qiang Zeng, Lannan Luo, Yung Ryn Choe, and Junghwan Rhee. “BinSimDB: Benchmark Dataset Construction for Fine-Grained Binary Code Similarity Analysis”. In: ArXiv (2024). URL: https : / / api . semanticscholar . org / CorpusID : 273346788.

16

17

A

Technical appendices Table 5: All CVE IDs in paper. CVE-2013-0340 CVE-2014-2497 CVE-2016-3751 CVE-2016-5767 CVE-2017-12652 CVE-2017-17506 CVE-2017-6363 CVE-2017-7890 CVE-2017-9233 CVE-2018-1000222 CVE-2018-1311 CVE-2018-14553 CVE-2018-17233 CVE-2018-17234 CVE-2018-17237 CVE-2018-17432 CVE-2018-17435 CVE-2018-17437 CVE-2018-17438 CVE-2018-25032 CVE-2018-5711 CVE-2019-11038 CVE-2019-15903 CVE-2019-18609 CVE-2019-20454 CVE-2019-25048 CVE-2019-25049 CVE-2019-6706 CVE-2019-6977 CVE-2019-6978 CVE-2019-7317 CVE-2019-8396 CVE-2020-10810 CVE-2020-10811 CVE-2020-12762 CVE-2020-14409 CVE-2020-14410 CVE-2020-15166 CVE-2020-15389 CVE-2020-15945 CVE-2020-24342 CVE-2020-27814 CVE-2020-27823 CVE-2020-27824 CVE-2020-27841 CVE-2020-27842 CVE-2020-27843 CVE-2020-27844 CVE-2020-27845 CVE-2020-36400 CVE-2020-6851 CVE-2020-8112 CVE-2021-20234 CVE-2021-20235 CVE-2021-20237 CVE-2021-20298 CVE-2021-20304 CVE-2021-29338 CVE-2021-30473 CVE-2021-30475 CVE-2021-3246 CVE-2021-32765 CVE-2021-33454 CVE-2021-33455 CVE-2021-33461 CVE-2021-33463

CVE-2021-33468 CVE-2021-3598 CVE-2021-3605 CVE-2021-38115 CVE-2021-3933 CVE-2021-40145 CVE-2021-40529 CVE-2021-40530 CVE-2021-40812 CVE-2021-43519 CVE-2021-43666 CVE-2021-44647 CVE-2021-44964 CVE-2021-45960 CVE-2021-45985 CVE-2021-46141 CVE-2021-46142 CVE-2021-46143 CVE-2021-46822 CVE-2021-46880 CVE-2022-0561 CVE-2022-0562 CVE-2022-0865 CVE-2022-0908 CVE-2022-0909 CVE-2022-0924 CVE-2022-1586 CVE-2022-1587 CVE-2022-1622 CVE-2022-2056 CVE-2022-22822 CVE-2022-22823 CVE-2022-22824 CVE-2022-22825 CVE-2022-22826 CVE-2022-22827 CVE-2022-22844 CVE-2022-23852 CVE-2022-23990 CVE-2022-25235 CVE-2022-25236 CVE-2022-25308 CVE-2022-25309 CVE-2022-25310 CVE-2022-25313 CVE-2022-25314 CVE-2022-25315 CVE-2022-25761 CVE-2022-28506 CVE-2022-28805 CVE-2022-3171 CVE-2022-33099 CVE-2022-3510 CVE-2022-35737 CVE-2022-3597 CVE-2022-3627 CVE-2022-37434 CVE-2022-3970 CVE-2022-40090 CVE-2022-40674 CVE-2022-43680 CVE-2022-44370 CVE-2022-48437 CVE-2023-1999 CVE-2023-25433 CVE-2023-25434

CVE-2023-25435 CVE-2023-26965 CVE-2023-26966 CVE-2023-27102 CVE-2023-27103 CVE-2023-2731 CVE-2023-2908 CVE-2023-29416 CVE-2023-29417 CVE-2023-29419 CVE-2023-29420 CVE-2023-29421 CVE-2023-30774 CVE-2023-31038 CVE-2023-31974 CVE-2023-35784 CVE-2023-35789 CVE-2023-39327 CVE-2023-39329 CVE-2023-43887 CVE-2023-47466 CVE-2023-47471 CVE-2023-48105 CVE-2023-4863 CVE-2023-50980 CVE-2023-51792 CVE-2023-52284 CVE-2023-52425 CVE-2023-52426 CVE-2023-53154 CVE-2023-5841 CVE-2023-6992 CVE-2023-7256 CVE-2024-1580 CVE-2024-24806 CVE-2024-25269 CVE-2024-25431 CVE-2024-25629 CVE-2024-27532 CVE-2024-28182 CVE-2024-28231 CVE-2024-28757 CVE-2024-29157 CVE-2024-29158 CVE-2024-29159 CVE-2024-29161 CVE-2024-29162 CVE-2024-29164 CVE-2024-29166 CVE-2024-31047 CVE-2024-31744 CVE-2024-3203 CVE-2024-3204 CVE-2024-32605 CVE-2024-32607 CVE-2024-32608 CVE-2024-32609 CVE-2024-32610 CVE-2024-32611 CVE-2024-32612 CVE-2024-32613 CVE-2024-32614 CVE-2024-32615 CVE-2024-32616 CVE-2024-32617 18 CVE-2024-32618

CVE-2024-32619 CVE-2024-32620 CVE-2024-32621 CVE-2024-32622 CVE-2024-32623 CVE-2024-32624 CVE-2024-33873 CVE-2024-33874 CVE-2024-33875 CVE-2024-33876 CVE-2024-33877 CVE-2024-34250 CVE-2024-34251 CVE-2024-34402 CVE-2024-34403 CVE-2024-34703 CVE-2024-40724 CVE-2024-41672 CVE-2024-45490 CVE-2024-45491 CVE-2024-45492 CVE-2024-45679 CVE-2024-48423 CVE-2024-48424 CVE-2024-48425 CVE-2024-48426 CVE-2024-50382 CVE-2024-50383 CVE-2024-50602 CVE-2024-50612 CVE-2024-5171 CVE-2024-53425 CVE-2024-56827 CVE-2024-7006 CVE-2024-8006 CVE-2025-0838 CVE-2025-11961 CVE-2025-11964 CVE-2025-12495 CVE-2025-12839 CVE-2025-12840 CVE-2025-2151 CVE-2025-2152 CVE-2025-2591 CVE-2025-2592 CVE-2025-27091 CVE-2025-2750 CVE-2025-2751 CVE-2025-2754 CVE-2025-2756 CVE-2025-2757 CVE-2025-28162 CVE-2025-28164 CVE-2025-29087 CVE-2025-2913 CVE-2025-2914 CVE-2025-2915 CVE-2025-2923 CVE-2025-2924 CVE-2025-2926 CVE-2025-3015 CVE-2025-3016 CVE-2025-31115 CVE-2025-31498 CVE-2025-3158 CVE-2025-3159

CVE-2025-3160 CVE-2025-3196 CVE-2025-3277 CVE-2025-3549 CVE-2025-43967 CVE-2025-48072 CVE-2025-48073 CVE-2025-48074 CVE-2025-48174 CVE-2025-48175 CVE-2025-50952 CVE-2025-5165 CVE-2025-5166 CVE-2025-5167 CVE-2025-5168 CVE-2025-5200 CVE-2025-5202 CVE-2025-5204 CVE-2025-54812 CVE-2025-54813 CVE-2025-54874 CVE-2025-56226 CVE-2025-59375 CVE-2025-61143 CVE-2025-61144 CVE-2025-61147 CVE-2025-6120 CVE-2025-62408 CVE-2025-62600 CVE-2025-6269 CVE-2025-64505 CVE-2025-64506 CVE-2025-64720 CVE-2025-65018 CVE-2025-6516 CVE-2025-66293 CVE-2025-67899 CVE-2025-68431 CVE-2025-6965 CVE-2025-8835 CVE-2025-8836 CVE-2025-8837 CVE-2026-22801 CVE-2026-25646 CVE-2026-25835 CVE-2026-26200 CVE-2026-26981 CVE-2026-27135 CVE-2026-27171 CVE-2026-27489 CVE-2026-27622 CVE-2026-29022 CVE-2026-32877 CVE-2026-32884 CVE-2026-33165 CVE-2026-33416 CVE-2026-33636 CVE-2026-34379 CVE-2026-34380 CVE-2026-34543 CVE-2026-34544 CVE-2026-34588 CVE-2026-34589 CVE-2026-34876 CVE-2026-34877

Table 6: Manually inspected CVE IDs list. CVE-2016-5767 CVE-2018-17432 CVE-2020-14410 CVE-2020-8112 CVE-2021-30473 CVE-2021-45960 CVE-2022-22824 CVE-2022-25314 CVE-2022-40674 CVE-2023-31038 CVE-2024-29157 CVE-2024-32605 CVE-2024-32617 CVE-2024-41672 CVE-2025-11964 CVE-2025-2926 CVE-2025-50952

CVE-2017-9233 CVE-2018-17435 CVE-2020-24342 CVE-2021-20234 CVE-2021-33455 CVE-2021-45985 CVE-2022-22827 CVE-2022-25315 CVE-2023-27102 CVE-2023-35789 CVE-2024-29158 CVE-2024-32607 CVE-2024-32619 CVE-2024-45490 CVE-2025-12495 CVE-2025-3015

CVE-2018-17233 CVE-2019-15903 CVE-2020-27841 CVE-2021-20237 CVE-2021-3605 CVE-2021-46141 CVE-2022-22844 CVE-2022-28506 CVE-2023-27103 CVE-2023-39329 CVE-2024-29159 CVE-2024-32610 CVE-2024-33875 CVE-2024-50382 CVE-2025-12839 CVE-2025-3196

19

CVE-2018-17234 CVE-2019-18609 CVE-2020-27844 CVE-2021-20304 CVE-2021-38115 CVE-2021-46143 CVE-2022-25308 CVE-2022-3171 CVE-2023-29417 CVE-2023-6992 CVE-2024-31047 CVE-2024-32614 CVE-2024-34251 CVE-2024-53425 CVE-2025-2914 CVE-2025-43967

CVE-2018-17237 CVE-2020-14409 CVE-2020-36400 CVE-2021-29338 CVE-2021-40145 CVE-2022-0562 CVE-2022-25313 CVE-2022-35737 CVE-2023-29420 CVE-2024-28231 CVE-2024-3203 CVE-2024-32615 CVE-2024-40724 CVE-2024-8006 CVE-2025-2924 CVE-2026-34876

Table 7: Total packages in the DeepHistory dataset. 7bitdi abseil ada ade aeron ags alembic amqp-cpp antlr4-cppruntime apr apr-util argtable3 armadillo arsenalgear asmjit assimp base64 bdwgc blend2d boost botan box2d brotli bzip2 bzip3 c-ares c-blosc c-blosc2 c4core calceph cargs catch2 ceres-solver cfitsio cgltf charls cjson clipper clipper2 cpp-optparse cppcommon cpptrace cpu_features cpuinfo cryptopp cwalk dataframe dav1d djinni-support-lib double-conversion draco drwav duckdb eastl embree3 expat ezc3d fast-cdr fast-dds fastgltf fftw flac

flatbuffers flecs fmt foonathan-memory freetype fribidi fruit ftxui g3log geographiclib geos gflags giflib ginkgo glad glfw glog gm2calc gtest h3 hazelcast-cpp-client hdf5 hdrhistogram-c hidapi highs highway hiredis http_parser imath imgui implot ixwebsocket jasper jbig json-c json-schema-validator jsoncpp kuba-zip leptonica lerc lexbor libaesgm libaom-av1 libassert libatomic_ops libavif libde265 libdeflate libdwarf libfdk_aac libgd libheif libjpeg libjpeg-turbo libjxl libmaxminddb libmp3lame libnghttp2 libpcap libpng libpq libpqxx

20

librdkafka libressl libsigcpp libsndfile libsodium libsrtp libsvm libsvtav1 libtiff libusb libuv libwebp libxlsxwriter libyuv libzen libzip libzippp lief lightgbm log4cplus log4cxx lua lunasvg lz4 matio mbedtls md4c meshoptimizer mimalloc miniz minizip mozjpeg mpdecimal msgpack-c msys2 nasm nghttp3 ninja nlopt nng nsync octomap ogg onnx open62541 openal openal-soft openblas opencl-icd-loader opencolorio openddl-parser openexr openh264 openjpeg openmesh opus pcre pcre2 pdf-writer perfetto pkgconf plutovg

poly2tri proj protobuf pthreadpool pugixml pystring qcbor qhull qr-code-generator quirc rabbitmq-c rapidyaml raylib re2 redis-plus-plus rmlui roaring rocksdb rotor sail screen_capture_lite sdl sdl_ttf sfml simdjson simdutf snappy sobjectizer spdlog spirv-cross spirv-tools sqlite3 strawberryperl taglib tcl tinyspline tinyxml2 tracy tree-sitter tree-sitter-c uriparser utf8proc vorbis vsg vulkan-loader vvenc wasm-micro-runtime wasmtime winflexbison xerces-c xz_utils yaml-cpp yasm yyjson z3 zeromq zimg zlib zlib-ng zstd zulu-openjdk zxing-cpp

B

Evaluation appendices Metric

p50

p90

p99

Running time (s) API call turns

77.3 26

573.5 105

1680.7 232

Table 8: Eval 2 trial run time and agent turns on a random sample of 100 task.

ELF/clang

ELF/gcc

PE/msvc

0.53 0.45 0.29

0.45 0.54 0.30

0.29 0.30 0.69

ELF/clang ELF/gcc PE/msvc

Table 9: Mean CodeBLEU between cross-build variants, aggregated by compiler family.

Bucket

CVEs

Variants

Reference (Win + Linux)

329

536

Diff_Opt Diff_Compiler Diff_OS Diff_Version Diff_All

209 263 173 301 220

621 979 932 4,813 2,346

Total

329

10,227

Table 10: Eval 3 cross-build variant counts.

5[IREKIRX ,MX$

 



















5[IREKIRX ,MX$















 6IJ

1IER

3TX +IQQE

'SQT

37 5[IR

+IQQEEKIRX ,MX$







+IQQEEKIRX ,MX$



:IV



%PP 3TYW

6IJ

+48

1IER

3TX

+IQMRM

Figure 5: Eval 3 cross-build transfer comparison. 21

'SQT

37 1IER(MJJW

:IV

%PP

C

Prompts

C.1

Strategy Generation Prompt

You are a binary security researcher. Your task is to write a clear, step-by-step analysis strategy that a follower agent can execute to locate a known vulnerability in a stripped binary. The follower agent has access to binary-analysis tools (decompiler, call graph, string and import enumeration, p-code) but no prior knowledge of this specific CVE. Your strategy describes a search procedure, not an answer - assume the agent does not know which function is vulnerable until your strategy guides them to it. ## Vulnerability Information **CVE:** {cve_id} **CWE:** {cwe_id} ({cwe_name}) **Description:** {cve_description} **Patch Diff:** ‘‘‘diff {patch_diff} ‘‘‘ NOTE: The patch diff shows source-level function names like ‘{example_source_function}‘. These names will NOT exist in stripped binaries - your strategy must describe the function by its behavior (imports it calls, strings it references, control flow), not its source name. ## Binary Analysis of Affected Function(s) Two builds of the same vulnerable function are shown below. Your strategy must rely on properties stable across both - imported function names, string literals, call-graph relationships, semantic data and control-flow patterns. ### Reference build ({reference_build_key}) {reference_analysis} ### Cross-build variant ({variant_build_key}) {variant_analysis} ## Your Task Write a strategy that a follower agent will execute against stripped binaries. The same vulnerability exists in every target, but compiled form - function boundaries, register allocation, inlining, control-flow shape - differs between builds. The strategy must identify the vulnerable function in *all* targets, including builds with different operating systems (Linux, Windows), compilers (GCC, Clang, MSVC), optimization levels (O0-O3), and software versions. Some target binaries are PATCHED (vulnerability fixed). Your strategy must include a check that distinguishes vulnerable code from patched code, so the follower agent does not produce false positives on fixed binaries. ## Output Format ### Vulnerability Summary One paragraph: what the vulnerability is and why it occurs. ### Search Strategy Use as many steps as the vulnerability requires (typically 2-4). Each step should narrow the candidate pool. **Step 1: [Name]** - What to search for and why

22

- What results to expect **Step 2: [Name]** (and so on) ### Verification Checklist A list of yes/no facts the follower agent should confirm to distinguish a vulnerable candidate from a similar but patched function. e.g., - "Calls ‘memcpy‘ with size argument derived from attacker input: yes/no" - "Has a bounds check before the dereference: yes/no" ### Fallback If the primary approach fails (e.g. expected strings or imports are missing in a particular build), describe an alternative search strategy. ### Final Output The follower agent must return exactly 5 candidate functions, ranked by confidence. Describe how to produce that many candidates - if the primary approach yields fewer, the fallback or weaker signals should supply backup candidates.

C.2

BinaryAPI Agent Documentation

# BinaryAPI Reference Read-only query interface for a stripped binary executable. All analysis is pre-computed; every method is a pure lookup. Stripped binaries have auto-generated function names like ‘sub_401234‘. Imported library symbols (e.g., ‘malloc‘, ‘memcpy‘) retain their original names. ## Construction ‘‘‘python api = BinaryAPI("/path/to/binary") ‘‘‘ Accepts ELF or PE binaries. Analysis is cached by SHA-256 hash. --## Enumeration ### ‘list_functions() -> list[FunctionInfo]‘ List all internal functions in the binary, sorted by address. Each entry is a dict: ‘‘‘python { "name": "sub_4012a0", # auto-generated identifier "address": "0x4012a0", # hex entry point "size_bytes": 342, # raw binary size "num_blocks": 12 # basic block count } ‘‘‘ ### ‘get_imports() -> list[str]‘ List all imported library function names, sorted alphabetically. Example: ‘["free", "malloc", "memcpy", "printf", "strlen"]‘ ### ‘get_strings() -> list[str]‘ List all string literals (>= 4 characters) found in the binary, sorted alphabetically.

23

--## Property-Based Discovery ### ‘find_callers_of_import(import_name: str) -> list[str]‘ Return internal functions that directly call the given imported function. ‘‘‘python api.find_callers_of_import("malloc") # ["sub_401230", "sub_4015a0", "sub_401bc0"] ‘‘‘ Returns ‘[]‘ if the import does not exist in this binary. ### ‘find_functions_referencing_string(s: str, case_sensitive: bool = False) -> list[str]‘ Return functions that reference a string containing ‘s‘ (substring match). **Case-insensitive by default** - ‘s="XPM"‘ matches strings like ‘"xpm_load_image"‘. Pass ‘case_sensitive=True‘ to require exact case (the literal bytes in the binary). ‘‘‘python api.find_functions_referencing_string("error") # ["sub_401a00", "sub_402100"] api.find_functions_referencing_string("XPM") # -> finds xpm_*, XPM_*, Xpm_*, ... api.find_functions_referencing_string("XPM", case_sensitive=True) # -> only the exact ’XPM’ bytes ‘‘‘ Returns ‘[]‘ if no string contains the substring. --## Call-Graph Navigation ### ‘get_callees(func: str) -> list[str]‘ Return all functions directly called by ‘func‘ (both imports and internal functions). ‘‘‘python api.get_callees("sub_401230") # ["malloc", "memcpy", "sub_401100", "sub_401500"] ‘‘‘ ### ‘get_callers(func: str) -> list[str]‘ Return all functions that contain a direct call to ‘func‘. ‘‘‘python api.get_callers("sub_401230") # ["sub_400f00", "sub_401800"] ‘‘‘ --## Code Inspection ### ‘decompile(func: str) -> str‘ Return Ghidra-decompiled C pseudocode for ‘func‘. Variable names, expression order, and control-flow structure vary across compilers and optimization levels. Useful for human-readable understanding but less stable across builds than p-code. ‘‘‘python

24

api.decompile("sub_401230") # "void sub_401230(long param_1, int param_2) {\n ..." ‘‘‘ ### ‘get_pcode(func: str) -> str‘ Return high p-code (post-SSA intermediate representation) for ‘func‘, grouped by basic block. P-code uses: - Sequential SSA variable names: ‘v0‘, ‘v1‘, ‘v2‘, ... - Architecture-independent operations: ‘LOAD‘, ‘STORE‘, ‘INT_ADD‘, ‘INT_SUB‘, ‘INT_MULT‘, ‘INT_AND‘, ‘INT_OR‘, ‘INT_XOR‘, ‘INT_LEFT‘, ‘INT_RIGHT‘, ‘INT_SRIGHT‘, ‘INT_EQUAL‘, ‘INT_NOTEQUAL‘, ‘INT_LESS‘, ‘INT_SLESS‘, ‘INT_LESSEQUAL‘, ‘INT_SLESSEQUAL‘, ‘BOOL_NEGATE‘, ‘BOOL_AND‘, ‘BOOL_OR‘, ‘FLOAT_ADD‘, ‘FLOAT_SUB‘, ‘FLOAT_MULT‘, ‘FLOAT_DIV‘, ‘CALL‘, ‘CALLIND‘, ‘CBRANCH‘, ‘BRANCH‘, ‘BRANCHIND‘, ‘RETURN‘, ‘COPY‘, ‘CAST‘, ‘SUBPIECE‘, ‘INT_ZEXT‘, ‘INT_SEXT‘, ‘PIECE‘, ‘PTRADD‘, ‘PTRSUB‘ - Labelled blocks: ‘blk_0‘, ‘blk_1‘, ... (consistent with ‘get_cfg‘) - Phi nodes at merge points: ‘PHI(v3, v7)‘ **Annotation format** (every operand is a self-describing token): - ‘:N‘ width suffix in bytes - ‘v0:4‘ is a 4-byte SSA value, ‘0x10:8‘ is an 8-byte constant. - ‘@REG‘ / ‘@stack[off]‘ location suffix on SSA values that live in a non-unique address space - e.g. ‘v0:8@RCX‘ is the SSA value occupying the RCX register (typical Win64 first arg), ‘v3:8@stack[-0x10]‘ is a stack slot. RAM-space globals skip the SSA name entirely and render as ‘0x{addr}:N‘ since the address is the identity. - ‘<type>‘ suffix when the decompiler inferred a non-undefined data type - ‘v0:8<longlong *>‘ is an 8-byte SSA value typed as a pointer to longlong. - Signedness is preserved through the opcode (‘INT_DIV‘ vs ‘INT_SDIV‘, ‘INT_LESS‘ vs ‘INT_SLESS‘) and the condition operator (‘<u‘ for unsigned, ‘<‘ for signed). More stable across build variants than decompiled C, but watch for type/register noise the SSA ‘vN‘ numbering and the operation skeleton are the most robust elements. Example output: ‘‘‘ blk_0: v1:1@ZF<bool> = INT_EQUAL v0:8@RCX<longlong>, 0:8 CBRANCH blk_2, v0:8@RCX<longlong> == 0:8 blk_1: v2:8@RAX = PTRSUB 0:8, 0x18004a908:8 BRANCH blk_3 blk_2: v3:8<longlong> = INT_ADD v0:8@RCX<longlong>, 24:8 v4:8<longlong *> = CAST v3:8<longlong> v5:8<longlong> = LOAD [v4:8<longlong *>] blk_3: v9:8@RAX = PHI(v2:8@RAX, v5:8<longlong>) RETURN v9:8@RAX ‘‘‘ Reading the example: ‘v0:8@RCX<longlong>‘ is the first argument (Win64 calling convention puts arg0 in RCX), 8 bytes wide, typed ‘longlong‘. The PHI in ‘blk_3‘ merges two distinct SSA generations that both happen to live in RAX - the SSA names (‘v2‘, ‘v9‘) keep the generations separate even though they share the register. ### ‘get_assembly(func: str) -> str‘ Return raw disassembly (one instruction per line) for ‘func‘. Architecture-native mnemonics (x86/x64/ARM/...); operands use the native register set and addressing syntax. Use this when decompiled C hides architecturally-relevant detail: timing side-channels, constant-time violations, calling-convention issues,

25

register clobbering, or compiler-emitted branch layout. For most bugs decompiled C is sufficient - prefer ‘decompile‘ or ‘get_pcode‘ first. ‘‘‘python api.get_assembly("sub_401230") # "0x401230 PUSH RBP\n0x401231 MOV RBP,RSP\n0x401234 SUB RSP,0x30\n..." ‘‘‘ --## Control-Flow Graph ### ‘get_cfg(func: str) -> CFGResult‘ Return the intra-procedural control-flow graph of ‘func‘. ‘‘‘python { "function": "sub_401230", "blocks": ["blk_0", "blk_1", "blk_2", "blk_3"], "entry_block": "blk_0", "edges": [ {"source": "blk_0", "target": "blk_1", "edge_type": "branch_false"}, {"source": "blk_0", "target": "blk_2", "edge_type": "branch_true"}, {"source": "blk_1", "target": "blk_3", "edge_type": "fallthrough"}, {"source": "blk_2", "target": "blk_3", "edge_type": "unconditional"} ] } ‘‘‘ Edge types: ‘fallthrough‘, ‘branch_true‘, ‘branch_false‘, ‘unconditional‘. Block labels match those in ‘get_pcode()‘ output. --## Regex Search ### ‘search_decompiled(pattern: str, limit: int = 200) -> SearchResults‘ Apply a Python regex ‘pattern‘ line-by-line across all decompiled output. Returns matches grouped by function, capped at ‘limit‘ total matches. ‘‘‘python api.search_decompiled(r"memcpy\(.*,.*,.*\)") # { # "results": [ # { # "function": "sub_401230", # "match_count": 2, # "matches": [ # {"function": "sub_401230", "line_number": 15, # "line_content": " memcpy(local_buf, param_1, param_2);", # "match_text": "memcpy(local_buf, param_1, param_2)"}, # ... # ] # } # ], # "total_match_count": 2, # "truncated": false, # "limit": 200 # } ‘‘‘ When ‘truncated‘ is ‘true‘, the search stopped at ‘limit‘ matches and there may be more. Refine the pattern (more specific regex) or raise ‘limit‘ on the next call. Note that ‘match_count‘ for the last function in ‘results‘ may understate its true count when truncation hit mid-function.

26

### ‘search_pcode(pattern: str, limit: int = 200) -> SearchResults‘ Apply a Python regex ‘pattern‘ line-by-line across all p-code output. Same return shape and truncation contract as ‘search_decompiled‘. ‘‘‘python api.search_pcode(r"CALL memcpy") # Functions that call memcpy, with exact p-code lines ‘‘‘ ### ‘search_assembly(pattern: str, limit: int = 200) -> SearchResults‘ Apply a Python regex ‘pattern‘ line-by-line across all disassembly. Same return shape and truncation contract as ‘search_decompiled‘. ‘‘‘python api.search_assembly(r"^[^;]*\bRDTSC\b") # Functions that contain RDTSC (time-stamp counter reads) ‘‘‘ --## Error Handling | Scenario | Exception | |----------|-----------| | Function name not found | ‘KeyError‘ | | Method called on an imported function (e.g., ‘decompile("malloc")‘) | ‘KeyError‘ | | Invalid regex pattern | ‘ValueError‘ | | Import not in binary (‘find_callers_of_import‘) | Returns ‘[]‘ | | No string match (‘find_functions_referencing_string‘) | Returns ‘[]‘ | No other exceptions during normal query operation. I/O and analysis errors surface only at construction time. --## What Survives Stripping | Property | Survives? | Stability Across Builds | |----------|-----------|------------------------| | Imported function names | Yes | High | | String literals | Yes | High | | Call graph structure | Yes | Moderate | | P-code operations | Yes | High (arch-independent) | | CFG structure | Yes | Moderate | | Internal function names | **No** (auto-generated) | None | | Addresses/offsets | Yes but **change every build** | None | | Decompiled C variable names | Yes but **vary by compiler** | Low |

C.3

Agent Solo Mode Prompt

You are a binary security researcher. Your task is to locate a security vulnerability in a stripped binary by querying it through an analysis API. You are given no prior knowledge about the vulnerability - you must discover it on your own. ## Important Context The target binary is STRIPPED - internal function names are auto-generated identifiers like "sub_401234". You must locate suspicious functions by querying the binary’s properties: imports, strings, call graph, decompiled code, p-code, and raw disassembly.

27

## Available API You have access to the following API. Call any method at any time. There is no limit on the number of calls you can make, and no cap on the number of turns. A single run is bounded only by a 1 hour wall-clock budget. {api_documentation} ### Paging ‘list_functions‘, ‘get_imports‘, and ‘get_strings‘ are paginated via ‘offset‘ and ‘limit‘ (default limit 100). Each response tells you the ‘total‘, the current ‘offset‘, how many items were ‘returned‘, and the ‘next_offset‘ to fetch the following page (or null when exhausted). Walk through all pages if you need a global view. ### Managing context Every tool result you receive is prefixed with ‘[tag=tN]‘. If a result has served its purpose (e.g., a large decompilation you have already digested), call ‘discard_tool_result(tag="tN")‘ to replace it with a short placeholder and free context for later queries. Only discard what you no longer need - discarded results cannot be recovered in the same run. ## Instructions Analyze the binary step by step: 1. Start by exploring the binary (imports, strings, functions). Use paging and ‘discard_tool_result‘ freely - prefer many targeted queries over hoarding large raw dumps. 2. Look for risky patterns: unchecked sizes passed to memcpy / strcpy, use-after-free, integer overflow before allocation, format strings, missing bounds checks, etc. 3. Narrow down candidates through API queries. 4. Inspect promising functions via decompile(), get_pcode(), or get_assembly(). 5. When confident, provide your final answer. After each API call, you will see the results. Decide your next action based on what you learn. When you are ready, output your final answer as: CANDIDATES: [func1, func2, ...] Return exactly 5 candidates, ranked from most likely to least likely. If you have fewer than 5 strong candidates, fill remaining slots with your best guesses.

C.4

Agent Strategy Guided Mode Prompt

You are a binary analysis assistant. A senior security researcher has written a strategy for locating a known vulnerability in a stripped binary. Your job is to follow this strategy step by step, making the appropriate API calls and reporting results. ## Strategy {strategy_document} ## Important Context

28

The target binary is STRIPPED - internal function names are auto-generated identifiers like "sub_401234". The original source function names referenced in the strategy do not exist in the binary. When the strategy says "find functions that do X", you must translate that into appropriate API calls. ## Available API {api_documentation} ### Paging ‘list_functions‘, ‘get_imports‘, and ‘get_strings‘ are paginated via ‘offset‘ and ‘limit‘ (default 100). Each response includes ‘total‘, ‘offset‘, ‘returned‘, and ‘next_offset‘. ### Managing context Every tool result is prefixed with ‘[tag=tN]‘. Call ‘discard_tool_result(tag="tN")‘ to drop a result you no longer need. The run has no turn cap and no tool-result size cap - only a 1 hour wall-clock budget. ## Instructions Follow the strategy step by step: 1. Read each step of the strategy carefully 2. Translate it into one or more API calls 3. Report the results of each call 4. If a step returns empty results, check if the strategy provides a fallback. If so, follow the fallback. If not, note the failure and proceed to the next step. 5. After completing all steps, provide your final answer Do NOT add your own vulnerability analysis beyond what the strategy describes. Your role is to execute the strategy faithfully, not to independently reason about the vulnerability. When you are ready, output your final answer as: CANDIDATES: [func1, func2, ...] Return exactly 5 candidates, ranked from most likely to least likely. If you have fewer than 5 strong candidates, fill remaining slots with your best guesses.

29