arXiv:2609.08240v1 [cs.DB] 8 Sep 2026
SmartANN: Object Causal Modeling Boosts Approximate Nearest Neighbor Diagnosis and Auto-Design Yutong Zhou
Guoxin Kang∗
Lei Wang
University of Chinese Academy of Sciences (ICT,CAS) Beijing, China [email protected]
Institute of Computing Technology, Chinese Academy of Sciences Beijing, China [email protected]
Institute of Computing Technology, Chinese Academy of Sciences Beijing, China [email protected]
Xueya Zhang
Qinwei Yang
Jianfeng Zhan
University of Chinese Academy of Sciences (ICT,CAS) Beijing, China [email protected]
The University of Hong Kong Hong Kong, China [email protected]
Institute of Computing Technology, Chinese Academy of Sciences Beijing, China [email protected]
ABSTRACT Approximate Nearest Neighbor (ANN) algorithms achieve high efficiency through multiple interdependent phases across index construction and query execution. However, this tight coupling allows performance loss from an upstream phase to propagate to downstream phases, affecting both their execution behavior and measurable outputs. Existing component-level works analyze mainly isolate and compare algorithmic design choices, while end-to-end benchmarks only aggregate performance metrics. Neither traces performance-loss propagation across dependent phases, making it difficult to attribute root causes or automatically redesign. To address this challenge, we present SmartANN, a framework built on the object causal model (OCM) for ANN bottleneck attribution and automated redesign. SmartANN represents the ANN workflow as eight ordered, replaceable objects and diagnoses them through a sequential diagnose-and-replace loop. At each iteration, it identifies the first object that deviates from the expected behavior or output, which we call a test oracle, as a bottleneck. As the upstream bottleneck object will obscure the downstream bottlenecks, SmartANN replaces the former with the test oracle if it exists, or an implementation with a better outcome otherwise, and then diagnoses the downstream objects. Based on the resulting bottleneck set and failure causes, SmartANN automatically selects and composes compatible actions from a pluggable action library to cover all diagnosed bottlenecks and generate an optimized end-to-end ANN design. We instantiate SmartANN for IVF-PQ and HNSW, representing widely used partition-and-quantization and graph-based ANN families, respectively. Extensive experiments on eight real-world datasets demonstrate that SmartANN improves Recall by 0.24–74.20%, while it increases QPS by 28.8–256.5% at comparable Recall, with low diagnosis and auto-design overhead. The code is available at https://github.com/ zhouyutong20/SmartANN. PVLDB Reference Format: Yutong Zhou, Guoxin Kang, Lei Wang, Xueya Zhang, Qinwei Yang, and Jianfeng Zhan. SmartANN: Object Causal Modeling Boosts Approximate Nearest Neighbor Diagnosis and Auto-Design. PVLDB, 20(1): XXX-XXX, 2027. ∗ Corresponding author.
doi:XX.XX/XXX.XX
1
INTRODUCTION
ANN algorithms are essential for high-dimensional vector retrieval [25, 38, 44]. They are widely used in recommendation systems [12, 42], information retrieval [32], and retrieval-augmented generation for large language models [18, 34]. However, the tight interphase dependencies in ANN algorithms cause cascading performance losses, whereby upstream performance losses propagate downstream, complicating diagnosis. Existing component-level analyses mainly isolate and compare predefined design choices in graph-based algorithms [51]. Such comparisons reveal the effects of individual designs but do not trace how a performance loss propagates across dependent phases. End-to-end benchmarks such as ANN-Benchmarks [1], Big-ANN-Benchmarks [47], and VectorDBBench [66] report aggregate Recall, QPS, construction time, and memory footprint. They reveal whether an ANN algorithm performs poorly but not why it performs poorly. Consequently, neither diagnosis can identify which internal phases cause the performance bottleneck under the current dataset or determine which existing ANN optimization can address it. The lack of diagnosis also prevents automated ANN design. Existing methods optimize different phases of ANN construction and query execution. LAET [35] and DARTH [6] learn search termination conditions. Neural LSH [15] and BLISS [24] improve data partitioning. LTR-IVF [49] optimizes query routing. RPQ [57] improves vector quantization. ADSampling [16] reduces distancecomputation cost through adaptive sampling. Because performance gains on one dataset frequently fail to generalize to another, users are forced into costly trial-and-error cycles to find the right optimization for their specific observed bottleneck. Systematically diagnosing and automatically redesigning ANN pipelines presents three core challenges. This work is licensed under the Creative Commons BY-NC-ND 4.0 International License. Visit https://creativecommons.org/licenses/by-nc-nd/4.0/ to view a copy of this license. For any use beyond those covered by this license, obtain permission by emailing [email protected]. Copyright is held by the owner/author(s). Publication rights licensed to the VLDB Endowment. Proceedings of the VLDB Endowment, Vol. 20, No. 1 ISSN 2150-8097. doi:XX.XX/XXX.XX
(C1) Performance loss propagates through ANN execution. The output of an upstream phase becomes the input to downstream phases. An inefficient implementation in the upstream phases thus contaminates downstream inputs and confounds their diagnostic metrics. Consequently, independent phase inspection often fails because downstream bottlenecks are obscured by performance degradation cascading from upstream. (C2) Difficulty of Preventing the Propagation from the Upstream Phases and Exposing the Downstream Bottlenecks. To properly evaluate downstream execution, one must halt the upstream propagation of performance loss. This requires replacing the bottleneck phase with an implementation that can expose the bottlenecks in downstream phases. Because intermediate phases like data partitioning and graph construction lack theoretically exact outputs, so-called test oracles that can be computed directly, it is a major challenge to prevent the performance loss from propagating from the upstream phase to the downstream phase. (C3) Existing ANN optimizations are difficult to reuse for a specific executed phase. Existing ANN optimizations tightly couple modifications across multiple execution phases. This makes it exceedingly difficult to decompose them into modular, pluggable auto-design actions. Even when a specific bottleneck phase is diagnosed, users cannot easily extract and apply the corresponding repair. To tackle the above challenges, we present SmartANN, a framework built on the object causal model (OCM) for ANN bottleneck attribution and automated redesign. SmartANN structures ANN index construction and query execution into eight ordered, replaceable objects, each defining its mechanism, diagnostic metrics, and available actions. SmartANN diagnoses objects in execution order and records the first object that deviates from its expected behavior or output, which we call the test oracle, as a bottleneck. Because an upstream bottleneck may obscure downstream bottlenecks, SmartANN replaces its output with the test oracle, when available, or its implementation with a stronger replacement implementation otherwise, before continuing downstream diagnosis. The OCM replacement operator quantifies the downstream effect by comparing diagnostic metrics before and after the replacement. Repeating this diagnose-and-replace loop yields the complete bottleneck-object set and failure causes. Given this bottleneck set and causes, SmartANN automatically constructs candidate designs from individual optimization actions or compatible action compositions whose joint coverage spans the complete set. It instantiates and evaluates each candidate and selects the best-performing end-to-end design. Rather than proposing new optimization methods, SmartANN organizes existing ANN optimizations in a pluggable action library. We instantiate SmartANN for IVF-PQ and HNSW, representing widely used partition-and-quantization and graph-based ANN families, respectively. Contributions. This paper makes the following contributions. (1) We present an object causal model for ANN algorithms based on Evaluatology[58, 59], and instantiate it for IVF-PQ and HNSW. We define a structured representation for each object, including mechanisms, actions, and diagnostic metrics, which makes internal performance loss measurable and analyzable at the object level.
(2) We propose object-level attribution through a sequential diagnosisand-replace loop. In each iteration, SmartANN locates the first bottleneck object, replaces it with a test oracle or stronger replacement implementation, and uses the replacement operator to measure changes in downstream diagnostic metrics. Repeating this process identifies the complete bottleneck object set and failure causes. (3) We build a pluggable library of 42 auto-design actions, with design knowledge encoded in action records and implementations standardized through object interfaces. Using the diagnosed bottleneckobject set and failure causes, SmartANN automatically generates and selects compatible action compositions to produce an optimized end-to-end ANN design. (4) Extensive experiments on eight real-world datasets across IVFPQ and HNSW demonstrate that bottleneck-object replacement effectively mitigates performance-loss propagation to downstream objects. SmartANN improves Recall by 0.24–74.20% and increases QPS by 28.8–256.5% at comparable Recall compared to the baseline. Compared with VDTuner, the state-of-the-art automatic performance tuning framework, SmartANN achieves a 2.9–42.0× speedup in end-to-end latency, demonstrating low diagnosis and auto-design overhead.
2 PRELIMINARIES 2.1 Problem Definition Given a dataset 𝐷 = {𝑥 1, . . . , 𝑥𝑛 } ⊂ R𝑑 , a query set 𝑄 ⊂ R𝑑 , a distance function 𝛿, and an integer 𝑘, an approximate nearest neighbor b𝑘 (𝑞) ⊆ 𝐷 of 𝑘 vectors for each query algorithm returns a result set 𝑁 𝑞 ∈ 𝑄. Let 𝑁𝑘 (𝑞) denote the exact 𝑘 nearest neighbors of 𝑞 in 𝐷 according to 𝛿. We measure search accuracy by
Recall@k =
b𝑘 (𝑞) ∩ 𝑁𝑘 (𝑞)| 1 ∑︁ | 𝑁 . |𝑄 | 𝑞 ∈𝑄 𝑘
(1)
We measure search efficiency by queries per second (QPS) under a fixed hardware environment. Each evaluated parameter configuration produces one Recall–QPS point. The non-dominated points form the Recall–QPS Pareto frontier of an ANN algorithm. Given a baseline ANN algorithm, SmartANN diagnoses the internal execution phases responsible for its performance bottlenecks on 𝐷 and 𝑄, and automatically generates redesigned ANN algorithms. For each evaluated configuration, the parameter values remain fixed, and the redesign aims to improve Recall, QPS, or both.
2.2
Object Causal Model
The object causal model (OCM) models the causal effect propagation paths among different objects within a system [58, 59]. The output of an upstream object can affect both the execution behavior and measurable outputs of downstream objects. OCM supports object-level replacement of mechanisms and actions to isolate these causal effects. SmartANN therefore adopts OCM for modeling the cause-and-effect relationship among the objects within the complex system, and uses the replace operator the measure the downstream effects of their replacements and guide automated ANN design. 2
Figure 1: SmartANN Overview: ANN object causal modeling, bottleneck diagnosis, and automated design.
3
SMARTANN OVERVIEW
failure-cause labels, supported ANN settings, input–output contracts, resource requirements, and compatibility constraints, and is implemented through the common interface. Given a diagnosed bottleneck-object set and its failure causes, SmartANN first retains compatible action compositions that jointly cover the set and then filters them by cause-label matching. It evaluates the resulting designs on the selection split and selects either the highest-Recall design or the highest-QPS design at comparable Recall. Section 6 provides further details.
As shown in Figure 1, SmartANN is an object-level framework for replacement-guided diagnosis and auto-design. Its end-to-end workflow consists of an object causal model, an attribution diagnoser, and an auto-designer. (1) Object causal model. SmartANN instantiates an OCM for the evaluation system defined by the workflow of ANN index construction and query execution. It represents this system as eight ordered objects 𝑂 1, . . . , 𝑂 8 , with causal effects propagating from upstream objects to downstream objects through their execution behavior and measurable outputs. For a given dataset and query workload, each object defines an input-output contract, expected behavior, and expected output. The contract provides a stable interface for object-level replacement, while the expected behavior and output provide references for evaluating the resulting behavior, output, and downstream effects. A deterministic expected output serves directly as a test oracle; when no exact output can be computed, SmartANN uses a stronger replacement implementation under the same input and resource constraints. This distinction is necessary for heuristic phases such as data partitioning and graph construction. (2) Attribution diagnoser. SmartANN diagnoses the objects in execution order. The first object 𝑂𝑖 encountered along the execution path that deviates from its expected behavior and output is recorded as a bottleneck, and its output is replaced with the testoracle output, or its implementation is replaced with a stronger replacement implementation, to suppress loss propagation from 𝑂𝑖 to downstream objects. Once the expected behavior and output at 𝑂𝑖 are achieved, diagnosis resumes from 𝑂𝑖+1 . This process repeats through 𝑂 8 and returns the complete bottleneck set and failure causes. (3) Auto-designer. The auto-designer comprises a pluggable action library, candidate generation, and design selection. The library contains 42 actions, with 21 for each ANN family. During knowledge preparation, each action is registered with its target objects,
4
OBJECT CAUSAL MODEL
The object causal model (OCM) provides SmartANN with the methodological foundation for tracing performance-loss propagation and isolating root causes through controlled object replacement. To support reliable diagnosis and automated design, SmartANN treats an execution phase as a separate object only when it is measurable for diagnosis, controllable through its output during replacement, and replaceable through a fixed contract for automated design. Based on the above principles, SmartANN decomposes ANN index construction and query execution into eight ordered objects. Each object is represented as 𝑂𝑖 = ⟨𝑀𝑖 , 𝑃𝑖 , 𝐴𝑖 ⟩.
(2)
𝑀𝑖 defines its basic function. 𝑃𝑖 contains its diagnostic metrics to measure its output quality. 𝐴𝑖 contains auto-design actions. An implementation that changes several objects is represented as an action bundle. Section 4.1 follows the ANN construction and query workflow to define the eight ordered objects and their mechanisms. Section instantiates these objects for IVF-PQ and HNSW and presents their corresponding diagnostic metrics and replacement methods.
4.1
Object-level ANN Decomposition
Based on the OCM, SmartANN decomposes the ANN workflow into the following eight ordered objects and their mechanisms as shown in Figure 1. 3
(1) Representation. This object transforms base and query vectors into the representations used by the ANN algorithm and provides the corresponding distance-computation interface. (2) Unit Instantiation. This object creates the basic search units and assigns their structural attributes before candidate relations are generated. (3) Candidate-Relation Generation. This object generates candidate relations between base vectors and search units or among search units for subsequent selection and storage. (4) Relation Materialization. This object selects and stores candidate relations under structural and resource constraints. When required, it also encodes the associated vector data to produce the searchable structure. (5) Access Localization. This object ranks search units or selects entry points for a query, determining where candidate exploration begins. (6) Candidate Exploration. This object explores the selected search units and materialized relations within a search budget and produces a candidate vector set. (7) Candidate Scoring. This object computes comparable exact or approximate scores for the explored candidates and produces a scored candidate set. (8) Result Decision. This object applies deduplication, optional reranking, and top-𝑘 selection to produce the final ordered results. Our division makes each object measurable, controllable, and replaceable. Existing component-level analyses instead divide graph algorithms by design choices [51] or IVF-PQ query execution by hardware kernels [29]. Such division supports component comparison, but does not provide the diagnostic metrics, controllable outputs, and replacement implementation required by SmartANN. They therefore cannot be directly reused. We use eight objects because a coarser decomposition would mix different causes of performance loss, whereas a finer decomposition would no longer provide shared interfaces for both IVF-PQ and HNSW.
4.2
distances over the original floating-point vectors. Rank agreement measures the consistency between the candidate orders induced by the two distance functions. Exact distances over the original floating-point vectors form the test oracle. The resulting reference is determined by the distance function, vector-normalization rule, and numerical precision. (2) Unit Instantiation. This object instantiates the basic search units. IVF-PQ generates coarse centroids. HNSW assigns level attributes to nodes. For IVF-PQ, clustering error and coverage radius characterize how well the units represent the data space. The inverted-list load factor further measures load imbalance under fixed assignment and materialization rules. Let 𝐿 𝑗 denote the 𝑗th inverted list and let 𝑁 denote the number of base vectors. The load factor is max 𝑗 |𝐿 𝑗 | 𝐿𝐹 = . (3) 𝑁 /nlist A value of 𝐿𝐹 closer to one indicates more balanced search units. A larger value indicates that a small number of units contain disproportionately many vectors, which may increase query-time scanning cost. For HNSW, the level distribution and top-layer size characterize the hierarchy. SmartANN further measures the mean number of upper-layer navigation hops. Let ℎ𝑞 denote the number of upper-layer hops for query 𝑞. This metric is 1 ∑︁ 𝐻2 = ℎ𝑞 . (4) |𝑄𝑑 | 𝑞 ∈𝑄 𝑑
A smaller 𝐻 2 indicates more direct hierarchical navigation. Neither search-unit formation nor hierarchy construction has a unique correct output. SmartANN therefore selects a stronger replacement implementation under a fixed unit cardinality, construction budget, memory budget, and randomization protocol. (3) Candidate-Relation Generation. This object generates candidate relations for subsequent materialization. IVF-PQ generates candidate assignments between vectors and coarse centroids. HNSW generates candidate neighbors during node insertion. SmartANN uses candidate-relation Recall to measure the completeness of the generated relations relative to the conditional exact relations. Distance gap and distance-computation count characterize relation quality and generation cost. Let 𝐸 3 denote the generated relation set and let 𝐸 3∗ denote the reference set obtained through exhaustive distance computation under the fixed upstream state. Candidaterelation Recall is |𝐸 3 ∩ 𝐸 3∗ | 𝑅3 = . (5) |𝐸 3∗ | Once the centroids or the HNSW insertion prefix are fixed, exhaustive distance computation yields a conditional test oracle. Its output also depends on the distance function and candidate-relation cardinality. (4) Relation Materialization. This object selects and stores candidate relations under structural and resource constraints. IVF-PQ writes postings and PQ codes according to the existing assignments. HNSW materializes the selected relations as hierarchical adjacency edges. For IVF-PQ, posting coverage and quantization error measure storage completeness and encoding distortion. For HNSW, node-degree distribution, graph connectivity, reachability, and path detour characterize structural constraints, global connectivity, and graph navigability. Multiple relation-selection policies can produce
Object Instantiation, Diagnostic Metrics, and Replacement
SmartANN instantiates IVF-PQ and HNSW as eight ordered objects. For each object, SmartANN specifies its role in each index family, the diagnostic metrics used to detect deviations from expected behavior or output, and a test oracle or stronger replacement implementation for replacement-guided attribution. A test oracle is used when the object has a deterministic expected output. A stronger replacement implementation is used when no unique correct structure exists, with the input, object contract, and resource budget held fixed. Let 𝑄𝑑 denote the diagnostic query set and let 𝑁𝑘 (𝑞) denote the ground-truth top-𝑘 neighbor set for query 𝑞. All object-level quality metrics are measured under a fixed upstream state and comparison budget. (1) Representation. This object forms the vector representation and distance-computation interface used by the ANN index. IVFPQ uses residual representations and a PQ code space. HNSW uses raw vectors or an optional quantized representation. SmartANN measures distance error, rank agreement, and representationtransformation time to characterize representation distortion and its cost. Distance error compares approximate distances with exact 4
valid structures, so this object has no unique test oracle. SmartANN fixes the candidate relations, degree constraints, index size, and construction budget, then selects a stronger replacement implementation with better structural quality. (5) Access Localization. This object determines where a query starts accessing the index. IVF-PQ ranks coarse units and selects the nprobe inverted lists. HNSW selects a base-layer entry point. For IVF-PQ, true-neighbor-unit coverage measures how completely the selected units cover the ground-truth neighbors. Let 𝑆𝑞 denote the unit set selected for query 𝑞 and let 𝑢 (𝑥) denote the inverted-list unit containing vector 𝑥. The per-query coverage and its mean are 1 ∑︁ 1 ∑︁ I[𝑢 (𝑥) ∈ 𝑆𝑞 ], 𝑅¯5 = 𝑅5 (𝑞). (6) 𝑅5 (𝑞) = 𝑘 |𝑄𝑑 | 𝑞 ∈𝑄 𝑥 ∈𝑁𝑘 (𝑞)
scoring time characterize its cost. Exact distances over the same candidate identifiers form the test oracle. This reference is determined by the distance function, vector representation, and numerical precision. Exact reranking over a fixed candidate set also belongs to this object. (8) Result Decision. This object applies deduplication, tie handling, and top-𝑘 selection to the existing candidates and scores. It produces the final ordered result. Let 𝑅8 (𝑞) denote the ground-truth-neighbor coverage of the final result. Candidate-to-final loss is 𝐿8 (𝑞) = 𝑅7 (𝑞) − 𝑅8 (𝑞).
(10)
This metric measures the additional loss introduced when the output of Candidate Scoring is converted into the final result. Selection agreement, top-𝑘 agreement, and decision time characterize result consistency and decision cost. When candidates, scores, the deduplication policy, and the tie-breaking rule are fixed, deterministic top-𝑘 selection forms the test oracle. Its output is determined by 𝑘 and the tie-handling policy.
𝑑
A larger 𝑅¯5 indicates that Access Localization preserves more complete sources of ground-truth neighbors for downstream exploration. For HNSW, SmartANN measures entry-point hit rate, entrydistance rank, and ground-truth reachability under a fixed downstream search budget. Given a fixed output cardinality, exact unit ranking for IVF-PQ or the best feasible entry for HNSW forms a conditional test oracle. The concrete reference is jointly determined by the distance function, nprobe or entry cardinality, upstream index structure, and downstream search budget. (6) Candidate Exploration. This object enumerates the selected postings or traverses the HNSW base layer within a search budget. It outputs a candidate vector set 𝐶𝑞 . Its primary quality metric is candidate Recall |𝐶𝑞 ∩ 𝑁𝑘 (𝑞)| 1 ∑︁ , 𝑅¯6 = 𝑅6 (𝑞). (7) 𝑅6 (𝑞) = 𝑘 |𝑄𝑑 | 𝑞 ∈𝑄
4.3
Running example
We use GloVe as a running example throughout Sections 5 and 6 to illustrate the complete SmartANN workflow for IVF-PQ and HNSW. Given the same datasets and queries, the attribution diagnoser identifies different bottleneck-object sets and failure causes for the two ANN families. For IVF-PQ, it identifies Access Localization 𝑂 5 and Candidate Scoring 𝑂 7 , caused by insufficient true-neighbor-unit coverage and approximate-scoring loss, respectively. For HNSW, it identifies Candidate Exploration 𝑂 6 , caused by insufficient candidate coverage. The auto-designer first filters candidate plans by their coverage of the diagnosed bottleneck-object set and then matches their actions against the diagnosed failure causes. This process generates a unit-selection and reranking design for IVF-PQ and a safe search-expansion design for HNSW.
𝑑
SmartANN also measures the 10th percentile of candidate Recall, p10 denoted by 𝑅6 , and the fraction of queries whose candidate Recall falls below a target 𝜏6 1 ∑︁ 𝜌6 = I[𝑅6 (𝑞) < 𝜏6 ]. (8) |𝑄𝑑 | 𝑞 ∈𝑄
5
𝑑
The mean, 10th percentile, and below-target fraction capture overall candidate coverage, tail quality on difficult queries, and the scope of the abnormality. The numbers of scanned postings, visited nodes, candidate expansions, and distance computations, together with execution time, measure exploration cost. Under a fixed upstream output, exhaustive enumeration or reachable-set traversal provides a coverage oracle. Cost-sensitive diagnosis instead uses a budgetmatched reference. The reference depends on the selected units or entry point, graph reachability, search budget, and stopping condition. (7) Candidate Scoring. This object computes comparable exact or approximate scores over a fixed candidate set. Let 𝑅7 (𝑞) denote the fraction of ground-truth top-𝑘 neighbors retained at the scoring boundary. SmartANN defines scoring loss as 1 ∑︁ 𝐿7 (𝑞) = 𝑅6 (𝑞) − 𝑅7 (𝑞), 𝐿¯7 = 𝐿7 (𝑞). (9) |𝑄𝑑 | 𝑞 ∈𝑄
REPLACEMENT-GUIDED ATTRIBUTION DIAGNOSIS
As shown in Algorithm 1, SmartANN identifies the bottleneckobject set through a sequential diagnose-and-replace loop, in which it diagnoses the first deviating object, replaces its output using the test oracle or a stronger replacement implementation, and diagnosis continues from the subsequent object until 𝑂 8 . Step 1: Object-level diagnosis. SmartANN examines 𝑂 1 through 𝑂 8 in execution order and stops at the first object 𝑂𝑖 whose diagnostic metrics in 𝑃𝑖 deviate from their expected outputs. This object is recorded as a bottleneck. Because the output of 𝑂𝑖 is consumed by subsequent objects, its performance loss may either propagate downstream or alter downstream inputs and mask independent bottlenecks. SmartANN therefore controls 𝑂𝑖 before diagnosing subsequent objects. We return to the running example to illustrate the first round of object-level diagnosis. or IVF-PQ, 𝑂 1 –𝑂 4 remain within their expected ranges, whereas Access Localization 𝑂 5 has an average true-neighbor-unit coverage of only 𝑅¯5 = 0.644. SmartANN therefore records 𝑂 5 as the first bottleneck and suspends diagnosis of 𝑂 6 –𝑂 8 . For HNSW, 𝑂 1 –𝑂 5 remain within their expected ranges,
𝑑
A larger 𝐿¯7 indicates more severe loss during approximate distance computation or score-based ordering. Distance error, rank agreement, and order-flip rate further explain the source of this loss. The number of scored candidates, distance-computation count, and 5
Algorithm 1 Sequential object-level bottleneck diagnosis
bottleneck previously masked by incomplete access localization. Replacing 𝑂 7 then reduces its scoring loss to zero. For HNSW, replacing 𝑂 6 increases average Candidate Recall from 0.863 to 0.946 and its 10th-percentile value from 0.570 to 0.860, while reducing the fraction of below-target queries from 35% to 1%. Candidate Scoring introduces no additional loss under the updated candidate input. The downstream deviation is therefore attributed to performance-loss propagation from 𝑂 6 , rather than an independent 𝑂 7 bottleneck. Step 3: Repeated diagnosis and replacement. Once 𝑃𝑖 returns to its expected range, SmartANN retains 𝑂𝑖★ and resumes diagnosis from 𝑂𝑖+1 . Previously retained replacements prevent confirmed upstream losses from affecting subsequent diagnosis. This diagnose-and-replace loop repeats until 𝑂 8 has been examined. All objects identified through their own abnormal diagnostic metrics form the complete bottleneck-object set and failure causes. We return to the running example to illustrate how SmartANN identifies the complete bottleneck-object set and the corresponding failure causes through sequential diagnosis and replacement. For IVF-PQ, the diagnostic metrics of 𝑂 1 –𝑂 4 remain within their expected ranges. Access Localization 𝑂 5 is the first deviating object, with an average true-neighbor-unit coverage of only 𝑅¯5 = 0.644, indicating insufficient access to the units containing true neighbors. SmartANN replaces its output with the test-oracle output and re-executes 𝑂 6 –𝑂 8 . Under the updated execution, 𝑂 6 satisfies its diagnostic conditions, whereas the average scoring loss at Candidate Scoring 𝑂 7 increases to 𝐿¯7 = 0.421, exposing approximate-scoring loss as an independent failure cause. SmartANN therefore records 𝑂 7 as a bottleneck and replaces it with exact distance computation over the same candidate identifiers. The diagnosis returns BIVF = {𝑂 5, 𝑂 7 }, where 𝑂 5 exhibits insufficient true-neighbor-unit coverage and 𝑂 7 exhibits approximate-scoring loss. For HNSW, 𝑂 1 –𝑂 5 remain within their expected ranges. Candidate Exploration 𝑂 6 is the first deviating object, with an average candidate Recall of 𝑅¯6 = 0.863 and a 10th-percentile Recall of p10 𝑅6 = 0.570. These metrics indicate insufficient candidate exploration, particularly for difficult queries. After SmartANN replaces its candidate output with the test-oracle output, Candidate Scoring 𝑂 7 and Result Decision 𝑂 8 satisfy their diagnostic conditions. The diagnosis therefore returns BHNSW = {𝑂 6 }, with insufficient candidate coverage as the failure cause of 𝑂 6 .
Input Ordered objects 𝑂 1, . . . , 𝑂 8 Output Bottleneck object set B 1: B ← ∅ 2: for 𝑖 ← 1 to 8 do 3: Diagnose 𝑂𝑖 using 𝑃𝑖 4: if 𝑃𝑖 is abnormal then 5: B ← B ∪ {𝑂𝑖 } 6: if a test oracle exists for 𝑂𝑖 then 7: 𝑂𝑖★ ← TestOracle(𝑂𝑖 , 𝐴𝑖 ) 8: else 9: 𝑂𝑖★ ← Replacement(𝑂𝑖 , 𝐴𝑖 ) 10: end if 11: Replace 𝑂𝑖 with 𝑂𝑖★ 12: Re-execute 𝑂𝑖 , . . . , 𝑂 8 13: assert 𝑃𝑖 is normal 14: Retain 𝑂𝑖★ 15: end if 16: end for 17: return B
whereas Candidate Exploration 𝑂 6 has a mean candidate Recall of p10 𝑅¯6 = 0.863 and a 10th-percentile Recall of 𝑅6 = 0.570. SmartANN records 𝑂 6 as the first bottleneck and suspends diagnosis of 𝑂 7 and 𝑂 8 . These results determine the objects replaced in Step 2. Step 2: Bottleneck replacement. SmartANN selects a replacement implement for 𝑂𝑖 from 𝐴𝑖 . If 𝑂𝑖 has a deterministic expected output, SmartANN uses it as a test oracle. Examples include exact distance computation over fixed candidate identifiers and exact top-𝑘 selection over fixed scores. If 𝑂𝑖 has a deterministic expected output, SmartANN replaces its output with the test oracle. Otherwise, SmartANN replaces its implementation with a stronger replacement implementation under the same inputs and resource constraints. Among the available implementations that satisfy the expected behavior of 𝑂𝑖 , SmartANN selects the one with the least measured adverse effect on downstream objects, and treats its output as the expected output of 𝑂𝑖 . Let 𝑂𝑖★ denote the replaced object. SmartANN replaces 𝑂𝑖 with ★ 𝑂𝑖 and re-executes 𝑂𝑖 and the affected downstream object while keeping the actions of other objects unchanged. This controlled execution attributes the observed downstream changes 𝑝 𝑗 ∈ 𝑃 𝑗 to the replacement of 𝑂𝑖 . The replacement operator is defined as:
6 rep 𝑂 𝑗 ; 𝑂𝑖 → 𝑂𝑖★ = 𝑝 𝑗 (𝑂𝑖★) − 𝑝 𝑗 (𝑂𝑖 ) .
(11)
ATTRIBUTION-GUIDED AUTO-DESIGN
Given the diagnosed bottleneck-object set, the auto-designer first automatically constructs candidate designs from individual actions or compatible action compositions whose joint coverage spans the complete set. SmartANN then instantiates and evaluates each candidate and selects the best-performing design according to the specified performance objective. This automated process comprises candidate generation, candidate evaluation, and selection.
A non-zero value indicates that the replacement affects 𝑂 𝑗 . Whether the effect is beneficial or adverse is determined by the semantics and values of 𝑝 𝑗 . Changes within the measurement tolerance are treated as zero. We return to the running example to quantify replacement operator effects. For IVF-PQ, replacing 𝑂 5 increases its true-neighborunit coverage from 0.644 to 1.000, allowing more true-neighbor candidates to reach 𝑂 7 . The average scoring loss at 𝑂 7 consequently increases from 0.184 to 0.421. The replacement operator gives rep𝐿¯7 𝑂 7 ; 𝑂 5 → 𝑂 ★ 5 = |0.421 − 0.184| = 0.237. Because a lower scoring loss is preferred, this change reveals a Candidate Scoring
6.1
Pluggable Action Library
(1) Knowledge preparation. SmartANN registers an ANN optimization as an action only when the optimization can be mapped to one or more objects, and its applicability conditions can be specified 6
explicitly. Each action record declares its supported ANN family, covered objects, failure causes, action type, training requirement, index-reconstruction requirement, budget requirement, compatibility mode, and output contract. The covered objects determine which bottlenecks the action can repair. The failure cause further distinguishes optimization directions for the same object. For example, insufficient candidate Recall at Candidate Exploration 𝑂 6 calls for revised exploration. When candidate Recall already meets the target, but access cost remains excessive, adaptive termination or distance-computation optimization is more appropriate. SmartANN registers an action record for each action in the library. We illustrate the record schema using hnsw_adaptive_entry, an HNSW action bundle that jointly replaces two objects. This example shows how an action record specifies the target objects and the corresponding performance bottlenecks that the action is designed to address.
online query processing. They are therefore unsuitable as deployable redesigns. Auto-design replacement instead uses deployable design actions selected from attribution evidence. These actions improve the Recall–QPS tradeoff under the object contracts and resource constraints. SmartANN prefers higher QPS at comparable Recall and higher Recall when the remaining conditions are comparable.
6.2
Action Record Action Applicable ANN family Covered objects Label Type Training and reconstruction Budget Compatibility Output contract
Design Generation and Selection
Let 𝐵 denote the diagnosed bottleneck-object set. SmartANN uses the diagnostic metrics to determine the bottleneck cause of each object in 𝐵, and then applies two-stage filtering over the action library. The first stage retains only individual actions or compatible action compositions that jointly cover every object in 𝐵, eliminating candidates unrelated to the diagnosed bottlenecks. The second stage matches the failure cause of each bottleneck with the failure causes addressed by the retained actions. A candidate is retained if it contains at least one action that matches the diagnosed failure cause of a corresponding bottleneck object in 𝐵. This process produces a small set of relevant candidate designs. SmartANN materializes and evaluates the filtered candidates using a small, dedicated selection query set. A candidate containing actions for 𝑂 1 –𝑂 4 requires the affected index structures to be rebuilt; the corresponding library actions support parallel cluster and graph construction to reduce this overhead. A candidate containing only actions for 𝑂 5 –𝑂 8 reuses the existing index and parallelizes the selection queries. Depending on the optimization objective, SmartANN selects either the design with the highest Recall or the design with the highest QPS at matched Recall. The selected design is finally evaluated on the final evaluation queries detailed in Section. We return to the running example to illustrate the automated redesign process. The auto-designer consumes the bottleneck-object sets and failure causes produced by attribution diagnosis. Each action record specifies its target objects and a failure-cause label describing the bottleneck that the action addresses. The first stage filters candidate designs by object coverage, whereas the second stage matches the diagnosed failure causes against these registered labels. A candidate that passes the coverage filter is retained if at least one of its actions has a label matching a diagnosed failure cause. For IVF-PQ, the first stage retains only individual actions or compatible action compositions that jointly cover BIVF = {𝑂 5, 𝑂 7 }. The diagnosed cause at 𝑂 5 , insufficient true-neighbor-unit coverage, matches the corresponding label of unit-selection actions. Similarly, the approximate-scoring loss at 𝑂 7 matches the label of candidate-reranking actions. This matching retains a small set of plans containing at least one cause-matched action. After evaluation on the selection split, the selected composition combines unit selection and candidate reranking, improving Recall@100 from 0.2670 to 0.6543. For HNSW, the first stage retains the actions targeting BHNSW = {𝑂 6 }, including hnsw_safe_search_expansion, hnsw_laet, hnsw_ adsampling, and hnsw_dade. The diagnosed cause, insufficient candidate coverage, matches the failure-cause label registered for hnsw_safe_search_expansion. The remaining actions are labeled
hnsw_adaptive_entry HNSW {𝑂 5 , 𝑂 6 } Repairs poor entry quality and downstream candidate-coverage loss Data-driven Neither required Fixed baseline efSearch with all additional costs included Direct query-side replacement Entry and candidate-set boundary
(2) Action Implementation. SmartANN defines a standardized input–output interface for each object and requires all actions targeting that object to conform to it. Each action is implemented as a pluggable action, allowing heterogeneous optimizations to be interchanged and composed without modifying adjacent objects. The library contains 42 actions, of which 34 are adapted from 28 existing ANN optimization methods, while eight are implemented specifically for SmartANN. These actions are evenly divided between IVF-PQ and HNSW, with 21 for each family. By implementation type, 17 are learned, 12 are data-driven, and 13 are non-learned. SmartANN implements its action runtime in C++17 and defines a standardized input–output interface for each object. All actions targeting the same object conform to this interface and are implemented as pluggable actions. Each action provides a common prepare interface for constructing or loading action-specific models and index artifacts, and an execute interface for consuming object inputs and producing contract-compliant outputs. Learned actions are trained offline on the learned training split, and their resulting models are loaded by the corresponding C++ actions. During execution, SmartANN validates action outputs, resource usage, and evidence observed at downstream object boundaries. (3) Diagnostic replacement and auto-design actions. Diagnostic replacement and auto-design replacement serve different purposes. Diagnostic replacement uses a test oracle or a stronger replacement implementation to suppress performance-loss propagation to downstream objects. Such replacement may incur prohibitive computational cost or require information unavailable during 7
{6, 8}, 𝑛𝑝𝑟𝑜𝑏𝑒 ∈ {8, 16, 32, 64}, and exact-reranking factors from {2, 4, 8, 16}. We retain only dimension-compatible values of 𝑚. For HNSW, the evaluated values are 𝑀 ∈ {12, 16, 24}, 𝑒 𝑓 𝐶𝑜𝑛𝑠 − 𝑡𝑟𝑢𝑐𝑡𝑖𝑜𝑛 ∈ {160, 200, 240, 300}, and 𝑒 𝑓 𝑆𝑒𝑎𝑟𝑐ℎ ∈ {100, 128, 160, 192, 256, 384, 576}. Candidate-exploration and scoring costs are governed by 𝑒 𝑓 𝑆𝑒𝑎𝑟𝑐ℎ. For both ANN families, we use dataset-specific subsets of these values to cover efficiency-oriented and qualityoriented Recall–QPS regions. SmartANN generates candidate redesigns from 42 registered actions, with 21 actions for IVF-PQ and 21 actions for HNSW. (2) SmartANN. For each frozen representative point, SmartANN runs the diagnose-and-replace loop on the diagnostic split to obtain the complete bottleneck-object set and its failure causes. It filters the action library by bottleneck coverage, failure-cause matching, interface compatibility, and resource constraints. Learned actions use only the learned training split, and the resulting candidate designs are evaluated on the selection split. The selected design and its parameters are then frozen for final evaluation. (3) Random baseline. Random follows the same action-training, selection, and final-evaluation protocol but cannot access diagnostic metrics or the bottleneck-object set and failure causes. For HNSW, it samples 𝐾 = 3 distinct action–budget combinations without replacement using fixed seeds 14, 28, and 42 and budget multipliers from {0.5, 0.75, 1.0}. (4) VDTuner. VDTuner performs multi-objective black-box search over the complete parameter space on the tuning split. For a fair comparison with SmartANN’s three redesigns, we select the three VDTuner configurations with the highest Recall on the tuning split and freeze them before final evaluation.
for reducing exploration or distance-computation cost and are therefore pruned. The selected design improves Recall@100 from 0.6692 to 0.8162. This paired example shows that the same dataset and query workload lead to ANN-family-specific bottleneck-object sets, failure causes, and automated redesigns.
7
EVALUATION
This section conducts an extensive evaluation of SmartANN against the state-of-the-art methods on eight real-world datasets using IVF-PQ and HNSW. All experiments run on a server equipped with two 2.0 GHz Intel Xeon Gold 6330 processors, 56 physical cores, 112 logical cores, and 503 GiB of memory. The server runs Ubuntu 20.04. All implementations are compiled with GCC 9.4.0. All reported query experiments use 28 logical cores with hyperthreading enabled
7.1
Experimental Setup
7.1.1 Datasets and Query Splits. (1) Datasets. We evaluate SmartANN on eight real-world datasets. SIFT [30], Deep [53], GIST [30, 43], and Tiny5M [48] represent image retrieval, while GloVe [45], MSong [5], Yandex-200 [46], and Wikipedia [20, 31] contain word, audio, search, and cross-modal embeddings, respectively. HubnessGloVe, Small-Gap-Deep, Density-LID-GIST, and OOD-Wikipedia further emphasize hubness, small nearest-neighbor gaps, heterogeneous local density, and distribution shift. The datasets contain 0.48–9.99 million base vectors with 96–1,024 dimensions and cover L2, angular, and cosine distances. Because the Wikipedia slice contains many duplicate vectors, we analyze it separately. (2) Query splits. Using a fixed random seed, we partition the queries into five mutually disjoint splits. The tuning split constructs the baseline Recall–QPS Pareto frontier, the diagnostic split supports object diagnosis and replacement, the learned training split trains learned actions, the selection split selects candidate designs and configurations, and the final evaluation split reports final performance. The final evaluation queries are never used for parameter tuning, diagnosis, learned training, or candidate selection, preventing test-set contamination and ensuring an unbiased evaluation. To demonstrate that SmartANN improves both query effectiveness and efficiency beyond a single point, we select three representative points from the baseline Pareto frontier. They include the maximum-Recall point, the point maximizing Recall × QPS, and the point with the largest minimum Recall distance from the first two. These well-separated representative points serve as the starting configurations for diagnosis and automated redesign.
7.1.3 Ablation Methodology. We ablate two key designs of SmartANN, namely attribution-guided auto-design generation using the bottleneck-object set and failure causes, as well as sequential replacement for downstream attribution. (1) Bottleneck-object set. We compare SmartANN with Random under the same baseline representative point, action library, training and selection data, resource budget, and final evaluation protocol. SmartANN selects designs using the diagnosed bottleneckobject set and failure causes, whereas Random cannot access this attribution evidence and samples three distinct valid designs using fixed seeds 14, 28, and 42. This comparison isolates the benefit of attribution-guided auto-design generation. (2) Replacement Operator. To validate the effect of the replacement operator, we compare Replacement, which replaces 𝑂𝑖 and re-executes the affected downstream objects, with No-Replacement, which retains the original downstream execution, while keeping the object order, diagnostic metrics, and detection thresholds fixed. This comparison evaluates whether the operator can distinguish propagated performance loss from independent or masked downstream bottlenecks.
7.1.2 Evaluation Methodology. To the best of our knowledge, no existing work jointly supports bottleneck attribution and automated ANN redesign. We therefore compare SmartANN with VDTuner [55], a state-of-the-art ANN parameter-tuning framework. To isolate the effectiveness of bottleneck attribution, we also implement a reproducible Random baseline using fixed random seeds. (1) Parameter spaces. We construct the baseline parameter space using parameters that directly determine the index structure and query cost. Across the eight datasets, the evaluated IVF-PQ values are 𝑛𝑙𝑖𝑠𝑡 ∈ {128, 256, 512, 1024}, 𝑚 ∈ {10, 14, 16, 20, 32}, 𝑛𝑏𝑖𝑡𝑠 ∈
7.2
Performance Comparison
We first report object-level diagnosis and replacement results for IVF-PQ and HNSW. We then compare the end-to-end Recall–QPS performance of SmartANN, the original baseline, and VDTuner. We finally analyze the costs of diagnosis, training, index construction, and complete-configuration evaluation. 8
Figure 2: Sequential object diagnosis and replacement at representative operating points for IVF-PQ in panel (a) and HNSW in panel (b). Darker green indicates a better normalized primary diagnostic metric. Red borders mark newly diagnosed bottleneck objects. Yellow borders mark replacements retained from preceding iterations. 7.2.1 Diagnosis and Replacement Results. Figure 2 shows the sequential attribution process for representative IVF-PQ and HNSW operating points across all datasets. Each row represents one iteration of the diagnose-and-replace loop. Cell color indicates how closely each object’s normalized primary diagnostic metric approaches its expected range. Red borders mark the first bottleneck identified in the current iteration, whereas yellow borders mark previously replaced objects. Diagnosis terminates when no new bottleneck is found. Figure 2 (a) shows that IVF-PQ bottlenecks mainly occur at Unit Instantiation 𝑂 2 , Access Localization 𝑂 5 , and Candidate Scoring 𝑂 7 , with different workloads producing bottleneck-object sets of different sizes. On GIST, replacing 𝑂 2 reduces the inverted-list load factor from 5.547 to 1.813. Replacing 𝑂 5 then increases trueneighbor-unit coverage from 0.885 to 1.000, while the scoring loss at 𝑂 7 rises from 0.242 to 0.338, exposing a previously masked scoring bottleneck. Replacing 𝑂 7 reduces this loss to zero, yielding the bottleneck-object set {𝑂 2, 𝑂 5, 𝑂 7 }. Figure 2 (b) shows that HNSW bottlenecks mainly occur at Unit Instantiation 𝑂 2 and Candidate Exploration 𝑂 6 . On GloVe, replacing 𝑂 6 improves the mean and 10th-percentile candidate Recall from 0.863 and 0.570 to 0.946 and 0.860, respectively, while reducing the fraction of queries below the target from 35% to 1%. These results attribute the performance loss to insufficient candidate exploration across both typical and difficult queries. Tiny5M contains bottlenecks at both 𝑂 2 and 𝑂 6 . Replacing 𝑂 2 reduces the mean upper-layer navigation hops from 10.28 to 8.31, but improves mean candidate Recall only from 0.714 to 0.735. Replacing 𝑂 6 further increases it to 0.829 and reduces the fraction of below-target queries to 1%. SmartANN therefore identifies 𝑂 2 and 𝑂 6 as independent bottlenecks, while 𝑂 7 and 𝑂 8 introduce no additional quality loss. Here, all candidate Recall values are measured on the diagnostic queries rather than from final evaluation queries.
Figure 3: Recall–QPS comparison among the HNSW baseline, VDTuner, and SmartANN.
preserves Recall at 0.9747 and increases QPS from 1,506 to 1,939. SmartANN retains the original design when diagnosis finds no bottleneck. SmartANN produces more favorable Recall–QPS tradeoffs than VDTuner on most datasets and operating regions. VDTuner jointly tunes construction and query parameters through black-box search. High-performance solutions often require aggressive construction and search configurations. Every change to a construction parameter also requires index reconstruction and reevaluation. VDTuner consequently incurs a large search space and high iteration cost. SmartANN instead restricts redesign to actions that match confirmed bottleneck objects. This restriction reduces ineffective complete-design evaluations. 7.2.3 Overhead. Figure 4 compares the end-to-end tuning time of SmartANN and VDTuner. SmartANN time consists of attribution diagnosis and auto-design. Auto-design includes candidate-action training, materialization, and evaluation. VDTuner time covers the complete joint search over construction and query parameters. The vertical axis uses a logarithmic scale. SmartANN requires less tuning time than VDTuner on every dataset. For HNSW, the per-dataset speedup ranges from 2.9× to 36.4×. The aggregate speedup across eight datasets is approximately 5.0×. For IVF-PQ, the per-dataset speedup ranges from 4.7× to 42.0×. The aggregate speedup is approximately 11.6×. VDTuner repeatedly constructs indexes and executes query evaluations over its complete parameter space. SmartANN restricts candidate actions according to the diagnosed bottleneck objects. It therefore avoids ineffective complete-design evaluations and substantially reduces end-to-end tuning cost.
7.2.2 End-to-End Performance . Figure 3 and Table 1 report the end-to-end results for HNSW and IVF-PQ, respectively. When retrieval quality is the primary limiting factor, SmartANN selects a quality-oriented redesign. At IVF-PQ GloVe-P1, for example, repairing Access Localization 𝑂 5 and Candidate Scoring 𝑂 7 increases Recall from 0.2670 to 0.6543. When candidate Recall already satisfies the requirement but vector accesses or distance computations remain redundant, SmartANN selects an efficiency-oriented action. At the high-Recall HNSW operating point on Yandex-200, SmartANN selects AdSampling. It 9
Table 1: IVF-PQ QPS and Recall for the baseline, VDTuner, and SmartANN on eight datasets at three operating points. The best Recall and QPS in each row are shown in bold. Dataset
Point
Baseline
VDTuner
SmartANN
Recall
QPS
Recall
QPS
Recall
QPS
GIST
P1 P2 P3
0.3888 0.6599 0.8985
2139.8 1377.7 256.5
0.3958 0.3921 0.3847
408.2 464.5 393.8
0.3969 0.6638 0.9038
873.9 699.6 124.4
Yandex
P1 P2 P3
0.2452 0.4607 0.7250
8860.2 5109.6 1577.9
0.3258 0.3254 0.3244
472.2 183.5 372.1
0.2500 0.4780 0.7405
1907.4 1514.5 309.7
Tiny5M
P1 P2 P3
0.3753 0.6180 0.8974
649.4 521.6 73.79
0.3774 0.3766 0.3729
352.1 414.2 424.2
0.3777 0.6258 0.9123
99.14 96.04 11.64
DEEP
P1 P2 P3
0.7225 0.8353 0.9637
774.8 741.6 160.6
0.4000 0.3999 0.3992
433.0 468.6 323.8
0.9436 0.9436 0.9637
96.57 95.96 160.6
Wikipedia
P1 P2 P3
0.1590 0.1963 0.2063
2091.0 1754.8 1036.5
0.1170 0.1168 0.1160
391.3 371.1 422.6
0.8882 0.9370 0.9483
415.1 378.7 337.2
MSong
P1 P2 P3
0.6080 0.7533 0.9736
3470.8 2729.3 1158.6
0.5455 0.5454 0.5450
390.4 388.2 178.3
0.9872 0.9872 -
207.2 207.8 -
SIFT1M
P1 P2 P3
0.7537 0.8734 0.9997
12879.1 6990.6 611.7
0.6469 0.6458 0.6405
414.0 295.0 91.72
0.8387 0.9003 -
3177.1 1683.0 -
P1 P2 P3
0.2670 0.4845 0.6925
8068.4 5045.1 818.3
0.3453 0.3428 0.3379
438.5 392.5 390.2
0.6543 0.6543 0.8371
1927.7 1940.8 212.3
GloVe
7.3
Figure 4: End-to-end tuning time of SmartANN and VDTuner on HNSW and IVF-PQ. Each SmartANN bar separates attribution diagnosis from auto-design.
Figure 5: Recall–QPS comparison between attribution-guided SmartANN and Random on HNSW.
Ablation Study
We ablate two key designs of SmartANN. SmartANN versus Random isolates the effect of attribution-guided automated design, while Replacement versus No-Replacement evaluates whether sequential replacement mitigates performance-loss propagation and reveals independent downstream bottlenecks. 7.3.1 Effect of Attribution-guided Automated Design. Figures 5 and 6 compare SmartANN with Random on HNSW and IVF-PQ. Both use the same baseline operating points, action library, and resource budgets, but only SmartANN selects actions using the bottleneck-object set and failure causes. Their Recall–QPS differences therefore isolate the benefit of attribution-guided automated design. SmartANN achieves more consistent Recall–QPS tradeoffs than Random on both HNSW and IVF-PQ. On IVF-PQ GIST, SmartANN reaches 0.9038 Recall and 124.4 QPS, outperforming Random at 0.8195 Recall and 55.6 QPS. Although Random occasionally finds competitive designs, its results are sensitive to action sampling and often gain QPS by sacrificing Recall. These results show that bottleneck attribution more reliably selects actions matching the diagnosed failure causes.
Figure 6: Recall–QPS comparison between attribution-guided SmartANN and Random on IVF-PQ.
downstream objects. No-Replacement retains the downstream metrics from the original execution. We compare their downstream metrics and resulting bottleneck-object sets. (1) HNSW. Table 2 shows both propagated and independent downstream loss. On GloVe, replacing Candidate Exploration at 𝑂 6 increases Candidate Recall at 𝑂 6 from 0.642 to 0.790 and at 𝑂 7 from 0.687 to 0.790. On Yandex-200, the corresponding values increase from 0.898 and 0.884 to 0.956. Because HNSW uses exact vector scoring, the normalized 𝑂 7 metrics confirm that its previous deviation was propagated from 𝑂 6 , rather than generated by an independent bottleneck. Tiny5M and DEEP contain bottlenecks at both 𝑂 2 and 𝑂 6 . Replacing 𝑂 2 changes Candidate Recall at 𝑂 6 only from 0.714 to 0.735 on Tiny5M and from 0.864 to 0.860 on DEEP. Replacing 𝑂 6 further increases Recall to 0.829 and 0.944, respectively, confirming an independent Candidate Exploration bottleneck.
7.3.2 Effect of Replacement Operator. We compare Replacement and No-Replacement using the same baseline execution, object order, diagnostic metrics, and thresholds. After identifying a bottleneck 𝑂𝑖 , Replacement substitutes its output with the test oracle or a stronger replacement implementation and re-executes the affected 10
Table 2: HNSW replacement results on diagnosed qualityproblem points. Parentheses show the absolute change relative to the previous row of the same dataset and point (current − previous). The first row of each point, clean, unchanged, and missing entries omit Δ. O2 (hops) lower is better; O6 ¯ and O7 (scored-candidate recall) higher is better. Undiag(𝑅) nosed O2 is clean. O7 before replacement is selection-batch scored recall; after the O6 oracle it tracks post-O6 𝑅¯ (native exact scoring). O2 replacement is hierarchy preview; O6 replacement is the diagnostic-period O6 oracle.
Table 3: IVFPQ replacement results on eight datasets. Parentheses show the absolute change relative to the previous row of the same dataset (current − previous). The first row of each dataset and any clean or unchanged entry omit Δ. O2 / O7: lower is better; O5: higher is better. Dataset
replacement
5.547 1.813 (−3.734) 1.813 1.813
0.906 0.885 (−0.021) 1.000 (+0.115) 1.000
0.264 0.242 (−0.022) 0.338 (+0.096) 0.000 (−0.338)
Yandex
Initial O2 O2&O5 O2&O5&O7
5.387 4.158 (−1.229) 4.158 4.158
0.800 0.798 (−0.003) 1.000 (+0.203) 1.000
0.265 0.304 (+0.040) 0.442 (+0.138) 0.000 (−0.442)
Initial O2 O2&O5 O2&O5&O7
5.516 1.854 (−3.662) 1.854 1.854
0.924 0.944 (+0.020) 1.000 (+0.056) 1.000
0.529 0.531 (+0.006) 0.583 (+0.023) 0.000 (−0.583)
O7(not bottleneck)↑
Bottlenecks: O2 + O6 DEEP Initial O2 O2&O6
9.89 9.75 (−0.14) 9.75
0.864 0.860 (−0.004) 0.944 (+0.084)
– 0.860 0.944 (+0.084)
Initial O2 O2&O6
10.28 8.31 (−1.97) 8.31
0.714 0.735 (+0.021) 0.829 (+0.094)
– 0.663 0.829 (+0.166)
Tiny5M
Bottleneck: O6 only MSong Initial O6
clean clean
0.968 0.989 (+0.021)
0.967 0.989 (+0.021)
Bottlenecks: O5 + O7
SIFT1M
Initial O6
clean clean
0.940 0.962 (+0.022)
0.950 0.962 (+0.012)
GloVe
Initial O6
clean clean
0.642 0.790 (+0.148)
0.687 0.790 (+0.103)
GIST
Initial O6
clean clean
0.761 0.846 (+0.085)
0.811 0.846 (+0.035)
Yandex
Initial O6
clean clean
0.898 0.956 (+0.058)
0.884 0.956 (+0.072)
Wikipedia
Initial O6
clean clean
0.275 0.301 (+0.026)
0.270 0.301 (+0.031)
Tiny5M
O7(↓)
Initial O2 O2&O5 O2&O5&O7
O6↑
replacement
O5(↑)
GIST
O2↓
Dataset
O2(↓)
Bottlenecks: O2 + O5 + O7
DEEP
Initial O5 O5&O7
clean clean clean
0.939 1.000 (+0.061) 1.000
0.217 0.260 (+0.043) 0.000 (−0.260)
Wikipedia
Initial O5 O5&O7
clean clean clean
0.967 1.000 (+0.033) 1.000
0.111 0.143 (+0.032) 0.000 (−0.143)
SIFT1M
Initial O5 O5&O7
clean clean clean
0.821 1.000 (+0.179) 1.000
0.086 0.160 (+0.074) 0.000 (−0.160)
GloVe
Initial O5 O5&O7
clean clean clean
0.644 1.000 (+0.356) 1.000
0.184 0.421 (+0.238) 0.000 (−0.421)
clean clean
clean clean
0.373 0.000 (−0.373)
Bottleneck: O7 only MSong
(2) IVF-PQ. Table 3 shows that upstream loss can also mask downstream bottlenecks. On GIST, replacing 𝑂 2 reduces true-neighborunit coverage at 𝑂 5 from 0.906 to 0.885, exposing an Access Localization bottleneck. Replacing 𝑂 5 subsequently increases candidate loss at 𝑂 7 from 0.242 to 0.338, exposing an independent Candidate Scoring bottleneck. The same cause appears on GloVe and Yandex-200, where improving 𝑂 5 coverage to 1.000 increases 𝑂 7 candidate loss from 0.184 to 0.421 and from 0.304 to 0.442, respectively. Replacing 𝑂 7 reduces these losses to zero. No-Replacement either attributes propagated upstream loss repeatedly to downstream objects or misses independent bottlenecks masked by upstream loss. Sequential replacement suppresses confirmed performance-loss propagation and re-evaluates downstream objects, enabling SmartANN to identify the complete bottleneckobject set without duplicate attribution.
7.4
Initial O7
upstream loss to downstream objects and misses independent bottlenecks masked by upstream performance loss. (2) Automated redesign. Using the diagnosed bottleneck-object set and failure causes to select and compose compatible actions produce effective end-to-end ANN redesigns more consistently than the reproducible Random baseline without attribution evidence. (3) Effectiveness and efficiency. Across eight real-world datasets, SmartANN improves Recall by 0.24–74.20 percentage points or increases QPS by 28.8–256.5% at comparable Recall. Compared with VDTuner, SmartANN achieves more favorable Recall–QPS tradeoffs across most datasets and a 2.9– 42.0× speedup in end-to-end latency, demonstrating low diagnosis and auto-design overhead.
Key Findings
(1) Bottleneck attribution. SmartANN combines object-level diagnostic metrics with sequential replacement to identify the complete bottleneck-object set and failure causes to provide verifiable attribution evidence. The ablation shows that No-Replacement misattributes propagated
8
RELATED WORK
ANN search have been extensively studied. ANN indexing and optimization explains how ANN indexes perform and optimize search. 11
ANN benchmarks explain how ANN index performance is measured consistently. Component-level analysis of ANN explains the differences among component designs. SmartANN further answers which mechanisms cause performance bottlenecks under a specific data distribution and query workload and which implementations should be used for targeted replacement.
8.1
applications and shows that Recall–performance evaluation based on vector-distance ground truth may not reflect actual task performance. Filtered-ANN benchmarks extend end-to-end evaluation by comparing attribute and range filtering algorithms, controlling filter selectivity and attribute–vector correlation, and generating workloads by query difficulty [39, 62, 65]. Existing end-to-end benchmarks can compare the construction cost and retrieval performance of complete ANN indexes. They cannot analyze a specific mechanism at a fine granularity and therefore cannot locate a performance bottleneck under a real data distribution and query workload. SmartANN determines the complete bottleneck mechanism set through diagnosis and replacement. It then selects and replaces corresponding pluggable implementations to improve index performance.
ANN Indexing and Optimization
IVF-PQ [27] and HNSW [41] are state-of-the-practice indexes for space partitioning with quantization and proximity graphs. They are widely used practices in current ANN algorithms. SmartANN therefore uses both algorithms for diagnosis, replacement, and autodesign. IVF-PQ limits the search range through data partitioning and reduces storage and distance-computation cost through product quantization [28]. HNSW constructs a multilayer proximity graph and reduces vector accesses by navigating from upper-layer entries to the base layer [41]. Prior work improves representation, data partitioning, graph structure, query routing, candidate exploration, candidate scoring, and search stopping for these two base indexes. For representation, quantization, and candidate scoring, OPQ [19], ScaNN/AVQ [23], DPQ [10], RPQ [57], and RaBitQ [17] improve vector encoding and distance estimation. ADSampling [16], DADE [13], PEOs [40], and FINGER [8] reduce unnecessary exact distance computations. For data partitioning and graph structure, Neural LSH [15] and BLISS [24] learn data partitions. RoarGraph [7], SPANN [9], and Elpis [2] design index structures for cross-modal queries, memory– disk indexes, and partitioned subgraphs. Flash [33], SymphonyQG [22], VSAG [64], Starling [50], and DEG [56] optimize graph construction, graph structure, data layout, or system execution. For query routing, candidate exploration, and search stopping, Learning to Route [4], LTR-IVF [49], and LEQAT [61] improve query routing. LAET [35] and DARTH [6] determine where search should stop for different queries. FARGO [63] designs global multiprobing for maximum inner product search. MEVI [60] combines a generative model with a vector index for document retrieval. Steiner-Hardness [52] is originally a query-difficulty measure. Existing ANN optimizations usually design complete solutions for predefined problems and validate them through end-to-end performance. They cannot diagnose the bottleneck mechanisms of an ANN index under a new data distribution and query workload or select the corresponding implementations. SmartANN determines the complete bottleneck mechanism set through diagnosis and replacement, then selects corresponding implementations from a pluggable implementation library for targeted replacement.
8.2
8.3
Component-Level Analysis of ANN
Prior research has begun to move from complete-algorithm comparison to component-level analysis. Wang et al. divide graph ANN into seven components and compare different implementations of a target component while fixing the others [51]. Azizi et al. summarize graph ANN into five design categories and perform controlled analysis on selected component [3]s. Other studies analyze the effect of predefined stages. Gottesbüren et al. use exhaustive search within shards and a routing oracle to analyze data partitioning, query routing, and in-shard search [21]. FANNS [29] divides an IVF-PQ query into six stages and analyzes the execution time of each stage. Hua et al. divide graph search into two stages and diagnose and repair reachability and local graph structure [26]. Other studies provide fine-grained analyses of pruning and entry selection in filtered ANN [37], distance comparison [14], I/O components in disk ANN [36], and graph-index construction [54]. Existing component-level analyses first specify the component or problem to study, then compare its implementations or measure its effect. SmartANN does not assume the bottleneck location in advance. It decomposes an ANN index into eight mechanisms and defines the input, output, and diagnostic metrics of each mechanism. The diagnosis–replacement–continued diagnosis process determines the complete bottleneck mechanism set under the current workload, measures the recoverable performance loss of each mechanism, and guides targeted mechanism replacement.
9
CONCLUSION
This paper presents SmartANN, a framework built on the object causal model for ANN bottleneck attribution and automated redesign. SmartANN represents ANN index construction and query execution as eight ordered, replaceable objects. Its sequential diagnoseand-replace loop uses test-oracle outputs or stronger replacement implementations to mitigate performance-loss propagation and identify the complete bottleneck-object set and failure causes. SmartANN then automatically selects and composes compatible actions from a pluggable action library to generate optimized end-to-end ANN designs. Extensive experiments on eight real-world datasets across IVF-PQ and HNSW show that SmartANN improves Recall by 0.24–74.20% or increases QPS by 28.8–256.5% at comparable Recall, while achieving low diagnosis and auto-design overhead.
ANN Benchmarks
Existing ANN benchmarks mainly provide end-to-end evaluation. Their metrics include index construction time, index memory use, retrieval accuracy, and retrieval speed. ANN-Benchmarks [1] is one of the most widely used evaluation tools. Big-ANN-Benchmarks [47] extends it to billion-scale datasets. BigVectorBench [31] targets compound queries in real applications, including range-filtered queries, multi-vector queries, big queries, and multimodal queries. It evaluates vector generation quality, cost, and retrieval performance for heterogeneous data. Iceberg [11] studies downstream tasks in real 12
REFERENCES
[23] Ruiqi Guo, Philip Sun, Erik Lindgren, Quan Geng, David Simcha, Felix Chern, and Sanjiv Kumar. Accelerating large-scale inference with anisotropic vector quantization. In International Conference on Machine Learning, pages 3887–3896. PMLR, 2020. [24] Gaurav Gupta, Tharun Medini, Anshumali Shrivastava, and Alexander J. Smola. Bliss: A billion scale index using iterative re-partitioning. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD ’22, page 486–495, New York, NY, USA, 2022. Association for Computing Machinery. [25] Yikun Han, Pengjiang Li, Chunjiang Liu, and Pengfei Wang. A comprehensive survey on vector database: Storage and retrieval technique, challenge. 2023. [26] Zhiyuan Hua, Qiji Mo, Zebin Yao, Lixiao Cui, Xiaoguang Liu, Gang Wang, Zijing Wei, Xinyu Liu, Tianxiao Tang, Shaozhi Liu, and Lin Qu. Dynamically detect and fix hardness for efficient approximate nearest neighbor search. Proc. ACM Manag. Data, 3(6), December 2025. [27] Herve Jegou, Matthijs Douze, and Cordelia Schmid. Product quantization for nearest neighbor search. IEEE Transactions on Pattern Analysis & Machine Intelligence, 33(01):117–128, 2011. [28] Herve Jegou, Matthijs Douze, and Cordelia Schmid. Product quantization for nearest neighbor search. IEEE Trans. Pattern Anal. Mach. Intell., 33(1):117–128, January 2011. [29] Wenqi Jiang, Shigang Li, Yu Zhu, Johannes De Fine Licht, Zhenhao He, Runbin Shi, Cedric Renggli, Shuai Zhang, Theodoros Rekatsinas, Torsten Hoefler, and Gustavo Alonso. Co-design hardware and algorithm for vector search. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, SC ’23, New York, NY, USA, 2023. Association for Computing Machinery. [30] Herve Jégou, Matthijs Douze, and Cordelia Schmid. Product quantization for nearest neighbor search. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(1):117–128, 2011. [31] Guoxin Kang, Zhongxin Ge, Jingpei Hu, Xueya Zhang, Lei Wang, and Jianfeng Zhan. Bigvectorbench: Heterogeneous data embedding and compound queries are essential in evaluating vector databases. Proc. VLDB Endow., 18:1536–1550, 2025. [32] Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. Dense passage retrieval for open-domain question answering. In Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu, editors, Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6769–6781, Online, November 2020. Association for Computational Linguistics. [33] Nativ Levy, Michael John Cafarella, Amir Gilad, Sudeepa Roy, and Brit Youngmann. CauSumX: Summarized Causal Explanations For Group-By-Average Queries. In Proceedings of the ACM SIGMOD International Conference on Management of Data, SIGMOD ’25. Association for Computing Machinery, 2025. [34] Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. Retrieval-augmented generation for knowledge-intensive nlp tasks. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin, editors, Advances in Neural Information Processing Systems, volume 33, pages 9459–9474. Curran Associates, Inc., 2020. [35] Conglong Li, Minjia Zhang, David G. Andersen, and Yuxiong He. Improving approximate nearest neighbor search through learned adaptive early termination. In Proceedings of the 2020 ACM SIGMOD International Conference on Management of Data, SIGMOD ’20, page 2539–2554, New York, NY, USA, 2020. Association for Computing Machinery. [36] Liang Li, Shufeng Gong, Yanan Yang, Yiduo Wang, and Jie Wu. I/o optimizations for graph-based disk-resident approximate nearest neighbor search: A design space exploration. Proc. VLDB Endow., 19(7):1484–1498, July 2026. [37] Mocheng Li, Xiao Yan, Baotong Lu, Yue Zhang, James Cheng, and Chenhao Ma. Attribute filtering in approximate nearest neighbor search: An in-depth experimental study. Proc. ACM Manag. Data, 3(6), December 2025. [38] Wen Li, Ying Zhang, Yifang Sun, Wei Wang, Mingjie Li, Wenjie Zhang, and Xuemin Lin. Approximate Nearest Neighbor Search on High Dimensional Data — Experiments, Analyses, and Improvement . IEEE Transactions on Knowledge & Data Engineering, 32(08):1475–1488, August 2020. [39] Mintaek Lim, Dogeun Kim, Minwoo Kim, and Jaeyoung Do. Revisiting filtered ann benchmarks: A hardness-controlled benchmark generator for realistic evaluation. Proceedings of the VLDB Endowment (PVLDB), 14(1), 2026. [40] Kejing Lu, Chuan Xiao, and Yoshiharu Ishikawa. Probabilistic routing for graphbased approximate nearest neighbor search. In Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Felix Berkenkamp, editors, Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pages 33177–33195. PMLR, 21–27 Jul 2024. [41] Yu A Malkov and Dmitry A Yashunin. Efficient and robust approximate nearest neighbor search using hierarchical navigable small world graphs. IEEE transactions on pattern analysis and machine intelligence, 42(4):824–836, 2018. [42] Shumpei Okura, Yukihiro Tagami, Shingo Ono, and Akira Tajima. Embeddingbased news recommendation for millions of users. In Proceedings of the 23rd
[1] Martin Aumüller, Erik Bernhardsson, and Alexander Faithfull. Ann-benchmarks: A benchmarking tool for approximate nearest neighbor algorithms. Information Systems, 87:101374, 2020. [2] Ilias Azizi, Karima Echihabi, and Themis Palpanas. Elpis: Graph-based similarity search for scalable data science. Proc. VLDB Endow., 16(6):1548–1559, February 2023. [3] Ilias Azizi, Karima Echihabi, and Themis Palpanas. Graph-based vector search: An experimental evaluation of the state-of-the-art. Proc. ACM Manag. Data, 3(1), February 2025. [4] Dmitry Baranchuk, Dmitry Persiyanov, Anton Sinitsin, and Artem Babenko. Learning to route in similarity graphs. In Kamalika Chaudhuri and Ruslan Salakhutdinov, editors, Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 475–484. PMLR, 09–15 Jun 2019. [5] Thierry Bertin-Mahieux, Daniel PW Ellis, Brian Whitman, and Paul Lamere. The million song dataset. 2011. [6] Manos Chatzakis, Yannis Papakonstantinou, and Themis Palpanas. Darth: Declarative recall through early termination for approximate nearest neighbor search. Proc. ACM Manag. Data, 3(4), September 2025. [7] Meng Chen, Kai Zhang, Zhenying He, Yinan Jing, and X. Sean Wang. Roargraph: A projected bipartite graph for efficient cross-modal approximate nearest neighbor search. Proc. VLDB Endow., 17(11):2735–2749, July 2024. [8] Patrick Chen, Wei-Cheng Chang, Jyun-Yu Jiang, Hsiang-Fu Yu, Inderjit Dhillon, and Cho-Jui Hsieh. Finger: Fast inference for graph-based approximate nearest neighbor search. In Proceedings of the ACM Web Conference 2023, WWW ’23, page 3225–3235, New York, NY, USA, 2023. Association for Computing Machinery. [9] Qi Chen, Bing Zhao, Haidong Wang, Mingqin Li, Chuanjie Liu, Zengzhong Li, Mao Yang, and Jingdong Wang. Spann: Highly-efficient billion-scale approximate nearest neighborhood search. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems, volume 34, pages 5199–5212. Curran Associates, Inc., 2021. [10] Ting Chen, Lala Li, and Yizhou Sun. Differentiable product quantization for end-to-end embedding compression. In Hal Daumé III and Aarti Singh, editors, Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pages 1617–1626. PMLR, 13–18 Jul 2020. [11] Tingyang Chen, Cong Fu, Jiahua Wu, Haotian Wu, Hua Fan, Xiangyu Ke, Yunjun Gao, Yabo Ni, and Anxiang Zeng. Reveal hidden pitfalls and navigate next generation of vector similarity search from task-centric views: [experiments & analysis]. Proc. ACM Manag. Data, 4(1), April 2026. [12] Paul Covington, Jay Adams, and Emre Sargin. Deep neural networks for youtube recommendations. In Proceedings of the 10th ACM Conference on Recommender Systems, RecSys ’16, page 191–198, New York, NY, USA, 2016. Association for Computing Machinery. [13] Liwei Deng, Penghao Chen, Ximu Zeng, Tianfu Wang, Yan Zhao, and Kai Zheng. Efficient data-aware distance comparison operations for high-dimensional approximate nearest neighbor search. Proc. VLDB Endow., 18(3):812–821, November 2024. [14] Liwei Deng, Penghao Chen, Ximu Zeng, Tianfu Wang, Yan Zhao, and Kai Zheng. Efficient data-aware distance comparison operations for high-dimensional approximate nearest neighbor search. Proc. VLDB Endow., 18(3):812–821, November 2024. [15] Yihe Dong, Piotr Indyk, Ilya Razenshteyn, and Tal Wagner. Learning Space Partitions for Nearest Neighbor Search. In International Conference on Learning Representations, 2020. [16] Jianyang Gao and Cheng Long. High-dimensional approximate nearest neighbor search: with reliable and efficient distance comparison operations. Proc. ACM Manag. Data, 1(2), June 2023. [17] Jianyang Gao and Cheng Long. Rabitq: Quantizing high-dimensional vectors with a theoretical error bound for approximate nearest neighbor search. Proc. ACM Manag. Data, 2(3), May 2024. [18] Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jin Pan, Yuxi Bi, Yi Dai, Jiawei Sun, Qianyu Guo, Meng Wang, and Haofen Wang. Retrieval-augmented generation for large language models: A survey. ArXiv, abs/2312.10997, 2023. [19] Tiezheng Ge, Kaiming He, Qifa Ke, and Jian Sun. Optimized product quantization. IEEE Trans. Pattern Anal. Mach. Intell., 36(4):744–755, April 2014. [20] Rohit Girdhar, Alaaeldin El-Nouby, Zhuang Liu, Mannat Singh, Kalyan Vasudev Alwala, Armand Joulin, and Ishan Misra. Imagebind one embedding space to bind them all. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 15180–15190. IEEE, 2023. [21] Lars Gottesbüren, Laxman Dhulipala, Rajesh Jayaram, and Jakub Łącki. Unleashing graph partitioning for large-scale nearest neighbor search. Proc. VLDB Endow., 18(6):1649–1662, February 2025. [22] Yutong Gou, Jianyang Gao, Yuexuan Xu, and Cheng Long. Symphonyqg: Towards symphonious integration of quantization and graph for approximate nearest neighbor search. Proc. ACM Manag. Data, 3(1), February 2025. 13
Proc. ACM Manag. Data, 4(3), May 2026. [63] Xi Zhao, Bolong Zheng, Xiaomeng Yi, Xiaofan Luan, Charles Xie, Xiaofang Zhou, and Christian S. Jensen. Fargo: Fast maximum inner product search via global multi-probing. Proc. VLDB Endow., 16(5):1100–1112, January 2023. [64] Xiaoyao Zhong, Haotian Li, Jiabao Jin, Mingyu Yang, Deming Chu, Xiangyu Wang, Zhitao Shen, Wei Jia, George Gu, Yi Xie, Xuemin Lin, Heng Tao Shen, Jingkuan Song, and Peng Cheng. Vsag: An optimized search framework for graphbased approximate nearest neighbor search. Proc. VLDB Endow., 18(12):5017–5030, August 2025. [65] Jiaxu Zhu, Jiayu Yuan, Kaiwen Yang, Xiaobao Chen, Shihuan Yu, Hongchang Lv, Yan Li, and Bolong Zheng. An experimental evaluation of hybrid querying on vectors. Proc. VLDB Endow., 19:183–195, 2025. [66] Zilliz. VectorDBBench: A benchmark tool for vector databases. https://github. com/zilliztech/VectorDBBench, 2023.
ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’17, page 1933–1942, New York, NY, USA, 2017. Association for Computing Machinery. [43] Aude Oliva and Antonio Torralba. Modeling the shape of the scene: A holistic representation of the spatial envelope. International journal of computer vision, 42(3):145–175, 2001. [44] James Jie Pan, Jianguo Wang, and Guoliang Li. Survey of vector database management systems. The VLDB Journal, 33(5):1591–1615, July 2024. [45] Jeffrey Pennington, Richard Socher, and Christopher D Manning. Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pages 1532–1543, 2014. [46] Harsha Vardhan Simhadri, George Williams, Martin Aumüller, Matthijs Douze, Artem Babenko, Dmitry Baranchuk, Qi Chen, Lucas Hosseini, Ravishankar Krishnaswamny, Gopal Srinivasa, et al. Results of the neurips’21 challenge on billion-scale approximate nearest neighbor search. In NeurIPS 2021 Competitions and Demonstrations Track, pages 177–189. PMLR, 2022. [47] Harsha Vardhan Simhadri, George Williams, Martin Aumüller, Matthijs Douze, Artem Babenko, Dmitry Baranchuk, Qi Chen, Lucas Hosseini, Ravishankar Krishnaswamny, Gopal Srinivasa, Suhas Jayaram Subramanya, and Jingdong Wang. Results of the neurips’21 challenge on billion-scale approximate nearest neighbor search. In Douwe Kiela, Marco Ciccone, and Barbara Caputo, editors, Proceedings of the NeurIPS 2021 Competitions and Demonstrations Track, volume 176 of Proceedings of Machine Learning Research, pages 177–189. PMLR, 06–14 Dec 2022. [48] Antonio Torralba, Rob Fergus, and William T Freeman. 80 million tiny images: A large data set for nonparametric object and scene recognition. IEEE transactions on pattern analysis and machine intelligence, 30(11):1958–1970, 2008. [49] Thomas Vecchiato, Claudio Lucchese, Franco Maria Nardini, and Sebastian Bruch. A learning-to-rank formulation of clustering-based approximate nearest neighbor search. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’24, page 2261–2265, New York, NY, USA, 2024. Association for Computing Machinery. [50] Mengzhao Wang, Weizhi Xu, Xiaomeng Yi, Songlin Wu, Zhangyang Peng, Xiangyu Ke, Yunjun Gao, Xiaoliang Xu, Rentong Guo, and Charles Xie. Starling: An i/o-efficient disk-resident graph index framework for high-dimensional vector similarity search on data segment. Proc. ACM Manag. Data, 2(1), March 2024. [51] Mengzhao Wang, Xiaoliang Xu, Qiang Yue, and Yuxiang Wang. A comprehensive survey and experimental comparison of graph-based approximate nearest neighbor search. Proc. VLDB Endow., 14(11):1964–1978, July 2021. [52] Zeyu Wang, Qitong Wang, Xiaoxing Cheng, Peng Wang, Themis Palpanas, and Wei Wang. Steiner-hardness: A query hardness measure for graph-based ann indexes. Proc. VLDB Endow., 17(13):4668–4682, September 2024. [53] Artem Babenko Yandex and Victor Lempitsky. Efficient indexing of billion-scale datasets of deep descriptors. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 2055–2063, 2016. [54] Shuo Yang, Jiadong Xie, Yingfan Liu, Jeffrey Xu Yu, Xiyue Gao, Qianru Wang, Yanguo Peng, and Jiangtao Cui. Revisiting the index construction of proximity graph-based approximate nearest neighbor search. Proc. VLDB Endow., 18(6):1825–1838, February 2025. [55] Tiannuo Yang, Wen Hu, Wangqi Peng, Yusen Li, Jianguo Li, Gang Wang, and Xiaoguang Liu. Vdtuner: Automated performance tuning for vector data management systems. In 2024 IEEE 40th International Conference on Data Engineering (ICDE), pages 4357–4369. IEEE, 2024. [56] Ziqi Yin, Jianyang Gao, Pasquale Balsebre, Gao Cong, and Cheng Long. Deg: Efficient hybrid vector search using the dynamic edge navigation graph. Proc. ACM Manag. Data, 3(1), February 2025. [57] Qiang Yue, Xiaoliang Xu, Yuxiang Wang, Yi Tao, and Xuliyuan Luo. Routingguided learned product quantization for graph-based approximate nearest neighbor search. 2024 IEEE 40th International Conference on Data Engineering (ICDE), pages 4870–4883, 2023. [58] Jianfeng Zhan, Lei Wang, Wanling Gao, Chenxi Wang, Fanda Fan, Guoxin Kang, and Hongxiao Li. Evaluatology: The Science of Uncovering the Causes and Effects. BenchCouncil Press, Hong Kong, China, 2.0 edition, 2026. [59] Jianfeng Zhan, Lei Wang, Wanling Gao, Chenxi Wang, Hongxiao Li, Fanda Fan, and Guoxin Kang. Evaluatology: The Science of Uncovering the Effects. BenchCouncil Press, Hong Kong, version 1.0, first edition, 2025. [60] Hailin Zhang, Yujing Wang, Qi Chen, Ruiheng Chang, Ting Zhang, Ziming Miao, Yingyan Hou, Yang Ding, Xupeng Miao, Haonan Wang, Bochen Pang, Yuefeng Zhan, Hao Sun, Weiwei Deng, Qi Zhang, Fan Yang, Xing Xie, Mao Yang, and Bin Cui. Model-enhanced vector index. In Proceedings of the 37th International Conference on Neural Information Processing Systems, NIPS ’23, Red Hook, NY, USA, 2023. Curran Associates Inc. [61] Pengcheng Zhang, Bin Yao, Chao Gao, Bin Wu, Xiao He, Feifei Li, Yuanfei Lu, Chaoqun Zhan, and Feilong Tang. Learning-based query optimization for multiprobe approximate nearest neighbor search. The VLDB Journal, 32(3):623–645, 2023. [62] Xiang Zhang, Chao Zhang, Ju Fan, Guoliang Li, and Xiaoyong Du. Vecbench: A controllable benchmark for filtered vector search: [experiments & analysis]. 14