Needles at Scale: LLM-Assisted Target Selection for Windows Vulnerability Research
arXiv:2606.01364v1 [cs.CR] 31 May 2026
Michael J. Bommarito II∗ [email protected] May 2026
Abstract The attack surface of a modern operating system is a haystack: thousands of signed binaries and millions of functions, almost none relevant to any given vulnerability. A human analyst or an LLM agent must pick the function worth reading before analyzing it. At whole-OS scope, this target selection, not the analysis, is the binding constraint. We present SymbolicateEnrich-Sample, a low-cost batch pipeline that turns a corpus of production Windows binaries into a queryable, priority-ranked research queue. We (i) recover function-level symbols for stripped vendor binaries by auto-fetching the public symbol files and joining them to a recovered call graph; (ii) attach cheap, deterministic structural features to each named function and, conditioned on those features, use a low-cost language model to assign a reachability tier, a risk level, a bug-class hypothesis, and a rationale; and (iii) draw diverse, prioritized batches via a priority-weighted importance sampler. The contribution is a selection substrate: the prioritization layer a downstream detector or LLM agent runs on top of. Across a whole Windows image of 7,231,419 functions, the labels are markedly selective, and stacking deterministic filters on them leaves a ∼22K-function shortlist: the candidate needles, few enough for a human or agent to work through. We characterize the pipeline’s selectivity and its failure modes, describe the methodology, and report aggregate statistics; we withhold the derived dataset for legal and dual-use reasons.
1
Introduction
Finding a memory-safety bug in a closed-source operating system is two problems wearing one coat. The visible problem is analysis: reasoning about a function’s bounds, lifetimes, and reachability until one can exhibit a primitive or cite the construct that makes it safe. The prior, hidden problem is target selection: out of the millions of functions that ship in a Windows image, deciding which few hundred deserve that expensive analysis at all. Analysis has attracted enormous tooling: symbolic execution, decompilers, fuzzers. Selection is still largely done by intuition, reputation (“SMB is juicy”), and grep. This imbalance sharpens as analysis is delegated to large language model (LLM) agents, which can already find real bugs when aimed at a specific target [7]. An agent that reads one decompiled function competently still must be pointed at the right function; aimed at a random one it burns ∗
Portions of this work were prepared with assistance from large language models. The author is solely responsible for all content, including any errors or omissions. This work was conducted for defensive and authorized vulnerabilityresearch purposes; see the data-release and ethics notes in Section 4.
1
a costly context window confirming that C-runtime startup code is not an attack surface. The bottleneck moves from “can we analyze this function” to “which function, of millions, next.” Recent work makes the same diagnosis: that vulnerability research is resource-constrained prioritization, and reframes selection as information retrieval or as candidate ranking with LLMs [9, 8, 2]. We take that framing as our starting point and ask a complementary, systems question: what does a selection substrate look like when it must cover an entire production operating system (millions of real, stripped, vendor-shipped functions) cheaply enough to run end to end? Our answer is Symbolicate-Enrich-Sample, a three-stage batch pipeline whose only job is to order the expensive reads (Section 2). The contribution is not a bug detector; it is the prioritization layer a detector should run on top of, built and characterized at production Windows scale. Windows is not incidental: the Symbolicate stage depends on Microsoft’s public symbol server, which publishes function names for stripped system binaries, an unusually generous arrangement. The Enrich and Sample stages are platform-independent, so the pipeline transfers to any ecosystem with a comparable public symbol source. Linux distributions provide one through debuginfod; platforms that do not publish system symbols (notably macOS) would need a different naming step. Contributions. • A reproducible method to recover function-level symbols for stripped production Windows binaries at scale, by auto-fetching vendor PDBs and joining them to a recovered call graph (§2.1). • Feature-grounded LLM enrichment: each function is labelled with reachability, risk, bug-class hypothesis, and rationale conditioned on cheap deterministic features, at low per-function cost (§2.2). • A priority-weighted importance sampler that converts the enriched corpus into a diverse, prioritized queue (§2.3). • A selectivity and failure-mode characterization of the resulting pipeline on a whole-OS Windows corpus (§3). • A discussion of the legal and dual-use reasons we publish the method and aggregate statistics but withhold the derived dataset (§4). Related work, in brief. LLM-based vulnerability detection has largely targeted source or synthetic/open-source-compiled code at the scale of thousands to tens of thousands of labelled samples, optimizing detection accuracy [13, 1]. The prioritization turn (selection as the real bottleneck) appears in SiftRank’s information-retrieval framing [9], in scaling-oriented candidate ranking [8], and in triage-evaluation studies [2], and scalable Windows-specific hunting has been demonstrated for individual subsystems such as ALPC [11]. The stance is older than LLMs: actionable-alert identification has ranked static-analysis warnings to cut false-positive triage cost since well before LLMs [10] and remains active [6], and directed greybox fuzzing steers scarce dynamic exploration toward reachable targets [3]; we bring the same prioritization stance to symbol-level selection over production binaries at corpus scale. On the corpus side, public Windows binary indices provide metadata and download links but no function-level symbolication or enrichment [12]. To our knowledge no public artifact combines production Windows scale, PDB-symbolicated function granularity, and feature-grounded LLM risk/reachability enrichment; that combination, and its selectivity, is what we report.
2
named call graph
labeled functions
Input
Symbolicate
Enrich
Sample
Queue
stripped PE + PDB key
fetch PDB, recover call graph, join names + sigs
deterministic features + grounded LLM labels
priority score, weighted reservoir, per-binary cap
priorityranked
Figure 1: The Symbolicate-Enrich-Sample pipeline. Symbolicate recovers a named call graph from a stripped PE via its PDB; Enrich attaches deterministic structural features and featuregrounded LLM labels; Sample draws a priority-ranked queue. Only aggregate structure crosses each stage; the expensive model sees a compact feature summary, never decompiled bytes.
2
Method
Symbolicate-Enrich-Sample runs as three batch stages over a binary corpus (Figure 1), writing to a single relational store keyed by (binary_sha256, function_va). We summarize each; the pipeline is fully reproducible from binaries that are themselves publicly retrievable from Microsoft-hosted symbol and file endpoints under Microsoft’s terms.
2.1
Symbolicate
Shipping Windows PE files carry no local symbols, but each contains a CodeView (RSDS) record naming a PDB and a <GUID><age> key. We parse that record, construct the canonical symbol-server URL, and fetch and cache the PDB. A decompiler pass (glaurung1 ) recovers the function set and inter-procedural call graph; we then join Microsoft’s public function names onto recovered function addresses and join a catalog of ∼20K Win32/WDK API prototypes onto called imports to recover the callee prototypes for each call site (we do not recover full target-function type signatures). Functions with no public name remain as synthetic sub_<addr> entries and are excluded from enrichment. This stage is deterministic, cacheable, and I/O- rather than model-bound. The function names themselves are published by the vendor; recovering them is symbolication, not decompilation of secret logic.
2.2
Enrich
Enrichment attaches a grounding feature vector and then a model label to each named function. Deterministic features. From the call graph and disassembly we compute, per function: the exported flag; caller/callee counts and the PDB-resolved names of the top callers and callees; whether the function calls a risk-relevant memory or user-buffer API, separated into copy/write sinks (memcpy, 1
glaurung is our own binary-analysis tooling; the method is agnostic to the specific decompiler.
3
memmove, RtlCopy*, memset), pool allocators, and user-pointer probes; basic-block count and size; and a breadth-first reach depth from entry points (exports, dispatch routines, graph roots). These features are cheap, exact, and carry much of the real signal: a function with no observed copy/write sink is less likely to host the copy-sink bug classes this filter targets, though it may still contain other defects. Model label. We prompt a low-cost LLM, served at a discounted batch tier, with the function’s name, recovered callee prototypes, and the grounding features, and require a structured output: a reachability tier (remote, local-ipc, local-user, admin-only, kernel-internal, library-internal); a risk level (critical/high/medium/low/info); a bug-class hypothesis; a confidence; a one-line role; and a free-text rationale. These tiers are model-assigned prioritization labels, not proofs: reach depth and names are evidence, but indirect calls, RPC routing, configuration, and runtime guards can move the true boundary. The model is instructed to reason from the supplied features (citing the copy primitive, the caller names, the reach depth) rather than from the name alone. Functions matching known runtime/CRT-helper patterns are labelled by rule (forced to library-internal/info) and never sent to the model, removing a large, predictable, low-value population from the bill. The job is sharded by a hash of binary identity across workers, each batching functions per model call with bounded concurrency, so millions of functions complete in hours. The expensive component only ever sees a compact structured summary, never raw decompiled bytes.
2.3
Sample
The enriched store is queried by a priority-weighted importance sampler. We score each function with a hand-tuned priority score (a heuristic ordering, not a calibrated probability) priority = wreach · wrisk · wconf · bonus,
(1)
where the weights map the model’s tiers to monotone numeric values and bonus multiplicatively rewards deterministic evidence the model cannot fabricate (actually calling a copy/write sink, being exported, sitting near an entry point, a parser-sized body), while penalizing trivial stubs. Given a filter (e.g. “remote, critical-or-high, calls a copy primitive”), the sampler draws without replacement via a weighted reservoir (Efraimidis–Spirakis, key u1/priority [4]), so a session covers the high-value space with diversity rather than re-returning the same top-k; a per-binary cap prevents one module from dominating. A top-N mode serves the sharpest leads directly. The sampler reads the store read-only and is safe to run while enrichment is still in progress. Which regime fits depends on the campaign. Broad, fuzzing-style sweeps favor the weighted draw, which spreads effort across the high-value space; targeted, hypothesis-driven work favors a tight filter with top-N, concentrating on one bug class, binary, or reachability tier. These sit on a spectrum from coverage to precision, and the priority score supports other points on it: stratified quotas per binary or bug class, threshold cuts, or re-weighting the features toward an analyst’s current interest. The scoring fixes an ordering; the choice of how to draw from it is left to the caller.
3
Evaluation
We are explicit about what we can and cannot evaluate. We cannot report a true-positive rate: establishing ground truth requires the very decompile-level verification the pipeline exists to prioritize, and at 5.55M functions no exhaustive oracle exists. We can evaluate the properties that determine whether the layer usefully concentrates attention (selectivity and distributional conservatism) and what it costs. 4
Corpus quantity
Count
Share
Signed PE binaries Functions recovered PDB-named functions of which enriched Unnamed sub_* stubs
5,888 7,231,419 5,553,074 5,552,861 1,678,345
76.8% 99.996% 23.2%
Table 1: The corpus: a whole production Windows image. Share is of all recovered functions, except the indented enriched row, which is of the PDB-named functions. Only PDB-named functions are enriched; unnamed stubs carry too little signal to label and are excluded. Enrichment reached all but 213 named functions. Deterministic features
Model label
Function (binary)
exp.
call.
blk.
dep.
sink
reachability
risk
bug-class hyp.
RtlDecompressBuffer (ntdll) wcscpy (ntdll) VirtualProtect (KernelBase) memcpy (ntdll)
yes yes yes yes
0 1 0 575
6 1 7 37
0 1 0 1
none memcpy none none
local-user local-user local-user library-internal
high high low info
int-overflow oob-write none-obvious none-obvious
Table 2: Four enriched records for recognizable Win32/NT APIs. exp. = exported, call. = in-binary callers, blk. = basic blocks, dep. = reach depth, sink = copy/write sink. The labels track the evidence, not the name: wcscpy is flagged because it calls memcpy, while memcpy itself (the primitive, with 575 callers) is demoted to info; the exported VirtualProtect is demoted as a thin syscall wrapper with no copy/parse sink, whereas RtlDecompressBuffer is flagged on role (size arithmetic in decompression). Labels are model outputs, not ground truth. Corpus. We apply Symbolicate-Enrich-Sample to 5,888 x86-64 Windows 11 binaries at build 10.0.26100, the servicing base shared by the 24H2 and 25H2 feature updates, drawn from two installations at 2025 patch levels (Table 1): 7,231,419 recovered functions, of which 76.8% received public PDB names and were enriched; the rest are unnamed sub_* stubs.
3.1
Selectivity and conservatism
Over the 5,552,861 enriched functions the label distribution forms a steep, conservative pyramid rather than collapsing toward the alarming end (Figure 2). The model reserves critical for 0.18% of functions and the remote-facing tier for 1.83%; a majority land in library-internal, and manual spot checks found CRT startup, C++ exception machinery, and other boilerplate pushed to info/low. This is precisely the population an unaided researcher wastes time re-dismissing. In sampled cases the rationales appear grounded : the explanations cite the supplied features (“reach_depth 48”, “calls memcpy”, “depth 1 from the dispatch entry”) rather than generic boilerplate, which is what makes the subsequent human read fast. Table 2 makes this concrete on four recognizable Win32/NT APIs: wcscpy is flagged because it calls memcpy, while memcpy itself is demoted to info, and the exported VirtualProtect is demoted as a thin syscall wrapper rather than flagged on its name. Selectivity funnel. Stacking the model’s labels with the deterministic features compounds the reduction (Figure 3): restricting to high/critical risk and remote-or-local-IPC reachability and a call to a risk-relevant memory or user-buffer API narrows 7.23M functions to ∼22K (∼2.5 orders of magnitude), and the critical ∩ remote corner to ∼2,100. We stress these are candidate counts, 5
Label distribution over 5.55M enriched functions (critical 0.18%, remote 1.83%)
Risk level critical
Reachability
2.44M
low
1.23M
info 105
1.68M
local-user
1.50M
medium
208K
local-ipc
373K
high
104
102K
remote
10K
admin-only
5K 372K
kernel-internal
3.18M
library-internal
106
104
functions (log scale)
105
106
functions (log scale)
Figure 2: Label distribution over 5,552,861 enriched functions (log x). The classifier is conservative: critical and remote (the tiers the sampler targets) are each a small fraction of the corpus, while the bulk of functions are labelled library-internal; manual spot checks of that bulk found the expected runtime/boilerplate code. not confirmed vulnerabilities; the value of the shortlist is that it is small enough for a human or agent to work through.
3.2
Failure modes
Sampling the top tier exposes two systematic weaknesses. (1) Top-tier over-reach: within the small critical bucket the model occasionally assigns a specific bug class (use-after-free, uncheckedpointer-deref) to functions whose feature vector contains no copy or memory-write sink—it names a wound with no weapon present. (2) Reachability inflation on deep parsers: it will label a structurally parser-like function remote even at a call-graph reach depth of 40+, assuming networksourced input the features do not establish. We saw both errors in the sampled top tier; the first is partly mechanically detectable: for copy-oriented bug classes, a concrete bug-class label with no corresponding sink in the features is unsupported (for other classes it is merely weakly supported, not impossible). A simple guard (forbid a concrete copy-class label without a sink, and down-weight critical rows lacking a risky primitive in the sampler) mitigates the first failure mode without re-running the model.
3.3
Cost
Low cost is a design goal, not an afterthought. Because the model component consumes only a compact structured feature summary (not decompiled code), and a large predictable fraction of functions is excluded by rule before any model call, per-function spend is small: the full 7,231,419function corpus is enrichable with a low-cost model at batch pricing, inexpensively relative to manual triage of the same surface. The deterministic features, which carry much of the ranking signal, cost nothing beyond the decompiler pass.
6
From 7.23M functions to a ~22K-function shortlist (~2.5 orders of magnitude fewer) 7.23M
All recovered functions
5.55M
PDB-named (enriched) 383K
Risk: high or critical 81K
+ Reach: remote or local-ipc 22K
+ Calls a memory-unsafe primitive 103
105
104
106
107
surviving functions (log scale)
Figure 3: Selectivity funnel. Each successive filter is a column in the enriched store; the combination reduces the search space by ∼2.5 orders of magnitude to a ∼22K-function shortlist of candidates.
4
Discussion
Why we release the method but not the data. The enriched corpus is derived from Microsoft’s copyrighted binaries. Redistributing function-level derived analysis at scale may create copyright or license risk that a methodology description and aggregate counts do not. (The function names are independently published by Microsoft’s symbol server, but the derived risk/reachability analysis is our work about protected binaries.) The closest public artifact, a Windows binary index, is deliberately metadata-only and links back to Microsoft’s own server [12]; we adopt the same posture and therefore avoid redistributing derived function-level annotations. Anyone with the binaries (publicly retrievable from Microsoft-hosted symbol and file endpoints under Microsoft’s terms) can reproduce the corpus from Section 2. Dual use. A ranked, remotely-reachable attack-surface map is an offense–defense-symmetric artifact: published openly with no patch in hand, it uplifts attackers at least as much as defenders, a concern that sharpens as LLM agents become able to act on such targets autonomously [5]. This is an independent reason, beyond copyright, to withhold the dataset and to frame the contribution as a prioritization methodology whose intended use is to accelerate responsible vulnerability research and coordinated disclosure. Limitations and threats to validity. The labels are single-model, single-pass guesses with no verified ground truth; our evaluation measures selectivity and distributional conservatism, not precision. Symbol recovery depends on PDB availability and is incomplete for fully stripped thirdparty modules. Reach depth is computed over a statically recovered call graph and misses indirect and virtual edges, so both reachability tiers and the funnel are approximations. The priority weights are hand-tuned, not learned. Future work. The natural next step is a calibration loop: log human or agent verdicts on sampled candidates, compute true-positive rate by bug-class, reachability, and module, and use it to re-weight both the classifier prompt and the priority function, turning a static guess table into a learning prioritizer, and replacing the hand-tuned weights with a model trained on verdicts.
7
5
Conclusion
Target selection, not analysis, is a binding constraint when hunting vulnerabilities across millions of functions, especially as analysis is delegated to agents that must be pointed somewhere. We showed that a low-cost Symbolicate-Enrich-Sample pipeline produces a selective, queryable prioritization layer over a production Windows corpus: it reserves its severe labels for a small minority of functions, demotes the boilerplate floor, grounds its rationales in verifiable features, and, stacked with deterministic filters, narrows a millions-strong search space to a tractable shortlist at low per-function cost. It is not a bug detector and claims no discovered vulnerabilities; it is the substrate a detector should run on. We describe the methodology and report aggregate results, and, for legal and dual-use reasons, withhold the derived dataset. Whether the shortlist improves real bug-finding yield over uninformed search is the central open question we leave to future work. Reproducibility. The pipeline (PDB auto-fetch, call-graph and feature extraction, structured LLM enrichment, priority-weighted sampler) is specified in Section 2; all inputs are production Windows binaries publicly retrievable from Microsoft-hosted symbol and file endpoints, and the figures in this paper are generated from the aggregate counts reported herein. We do not redistribute any Microsoft binaries, PDBs, function-level annotations, or sampled queues; only aggregate counts appear in the paper.
References [1] Md Basim Uddin Ahmed, Nima Shiri Harzevili, Jiho Shin, Hung Viet Pham, and Song Wang. SecVulEval: Benchmarking LLMs for real-world C/C++ vulnerability detection, 2025. https://arxiv.org/abs/2505.19828. [2] Osama Al Haddad, Muhammad Ikram, Ejaz Ahmed, and Young Lee. Prompting the priorities: A first look at evaluating LLMs for vulnerability triage and prioritization, 2025. https: //arxiv.org/abs/2510.18508. [3] Marcel Böhme, Van-Thuan Pham, Manh-Dung Nguyen, and Abhik Roychoudhury. Directed greybox fuzzing. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security (CCS), pages 2329–2344, 2017. https://doi.org/10.1145/3133956. 3134020. [4] Pavlos S. Efraimidis and Paul G. Spirakis. Weighted random sampling with a reservoir. Information Processing Letters, 97(5):181–185, 2006. https://doi.org/10.1016/j.ipl.2005 .11.003. [5] Richard Fang, Rohan Bindu, Akul Gupta, and Daniel Kang. LLM agents can autonomously exploit one-day vulnerabilities, 2024. https://arxiv.org/abs/2404.08144. [6] Xiuting Ge, Chunrong Fang, Xuanye Li, Weisong Sun, Daoyuan Wu, Juan Zhai, Shangwei Lin, Zhihong Zhao, Yang Liu, and Zhenyu Chen. Machine learning for actionable warning identification: A comprehensive survey, 2024. https://arxiv.org/abs/2312.00324. [7] Google Project Zero (Big Sleep Team). From naptime to big sleep: Using large language models to catch vulnerabilities in real-world code, 2024. https://projectzero.google/2024/10/fro m-naptime-to-big-sleep.html.
8
[8] Caleb Gross. O(N) the money: Scaling vulnerability research with LLMs, 2025. https: //noperator.dev/posts/on-the-money/. [9] Caleb Gross. Sift or get off the PoC: Applying information retrieval to vulnerability research with SiftRank, 2025. https://arxiv.org/abs/2512.06155. [10] Sarah Heckman and Laurie Williams. A systematic literature review of actionable alert identification techniques for automated static code analysis. Information and Software Technology, 53(4):363–387, 2011. https://doi.org/10.1016/j.infsof.2010.12.007. [11] Haoyi Liu, Feng Dong, Yunpeng Tian, Mu Zhang, Xuefeng Li, Fangming Gu, Zhiniang Peng, and Haoyu Wang. Needle in a haystack: Automated and scalable vulnerability hunting in the Windows ALPC sea. In Proceedings of the 2025 ACM SIGSAC Conference on Computer and Communications Security (CCS), 2025. https://doi.org/10.1145/3719027.3765180. [12] m417z. Winbindex: An index of Windows binaries, 2021. https://winbindex.m417z.com/. [13] Xin Zhou, Sicong Cao, Xiaobing Sun, and David Lo. Large language model for vulnerability detection and repair: Literature review and the road ahead, 2024. https://arxiv.org/abs/24 04.02525.
9