Arxiv Preprint Ver.
C OORDINATE H ETEROGENEITY G OVERNS B INARY Q UANTIZATION : F ROM I NFO NCE TO R ECALL
arXiv:2605.17524v1 [cs.LG] 17 May 2026
Wenxuan Xiao Changsha University [email protected]
A BSTRACT Binary quantization (BQ) compresses high-dimensional embeddings into one or two bits per coordinate, enabling nearest neighbor search at extreme speed. Yet a striking puzzle persists: BQ achieves competitive recall on contrastive embeddings but fails on others—and two leading systems adopt diametrically opposite strategies (random rotation vs. preserving coordinate axes) without a common theory explaining when each is appropriate. We resolve this puzzle by connecting the Gaussian structure recently established for InfoNCE-trained representations to a complete analytical framework for BQ quality. The key insight is that coordinate heterogeneity—the non-uniformity of per-coordinate variances—governs the key aspects of BQ performance. We derive closed-form expressions for ranking fidelity, prove that the magnitude bit carries information proportional to heterogeneity, and show that random rotation destroys precisely the signal that one paradigm exploits while creating the isotropy that the other requires. A two-parameter scaling law predicts fidelity across models and dimensions. Experiments on 13 datasets and 6 embedding families validate all predictions and provide the first principled design guide for binary quantization systems.
1
I NTRODUCTION
A curious fact has emerged from the rapid adoption of vector search: binary quantization—an extreme form of compression, retaining just the sign of each coordinate—works remarkably well on embeddings produced by contrastive learning, yet performs poorly on embeddings from other training paradigms. Systems built on this observation have proliferated. QuIVer (Xiao et al., 2026) constructs its entire graph index in two-bit space; RaBitQ (Gao & Long, 2024) applies random rotation before binarization and corrects distances with per-vector scalars. These two designs make opposite assumptions about coordinate structure, yet both report competitive recall on contrastive embeddings (Reimers & Gurevych, 2019; Nussbaum et al., 2024; Chen et al., 2024). Why does binary quantization work at all—and why do contradictory strategies both succeed? The classical answer appeals to locality-sensitive hashing (LSH): random hyperplane projections preserve angular similarity (Indyk & Motwani, 1998; Charikar, 2002; Goemans & Williamson, 1995). But this theory assumes isotropic data and analyzes worst-case performance; it cannot explain the strong dependence on training objective that practitioners observe. The missing ingredient, we argue, is the specific distributional structure that contrastive training imposes on representations. Betser et al. (2026) recently proved that InfoNCE induces approximately Gaussian coordinate distributions—a result that transforms the question from geometry (“how are points arranged on the sphere?”) to statistics (“what does the covariance matrix look like?”). Building on this Gaussian prior, we identify a single quantity that governs the key aspects of BQ quality: coordinate heterogeneity, the non-uniformity of per-coordinate variances. Heterogeneity is the reason BQ works on contrastive embeddings (their unequal variances create an implicit weighting that Hamming distance inherits), the reason two bits outperform one (the magnitude bit captures variance information that the sign bit discards), and the reason rotation helps one system 1
Arxiv Preprint Ver.
but hurts another (rotation equalizes variances, destroying the implicit weighting while creating the isotropy that linear correctors require). Concretely, we establish the following: • A closed-form Spearman fidelity via Stein’s lemma, revealing that off-diagonal covariance contributes 30–50% of ranking accuracy (§??, Theorems 1–2). • A proof that the magnitude bit is strictly more informative per dimension than the sign bit, with gain monotone in heterogeneity (§5, Theorem 6). • A rotation–fidelity duality that unifies the QuIVer and RaBitQ design philosophies (§??, Theorem 3 and Corollaries 4–5). • A two-parameter scaling law that predicts fidelity across models and dimensions from three covariance statistics (§7). We validate every theoretical prediction on 13 datasets spanning 6 embedding families and dimensions from 384 to 1024, including a non-Gaussian control (GIST-960) that confirms the necessity of the Gaussian prior. In graph-based search, the two-bit advantage is amplified 1.2–4.1× in local neighborhoods—a phenomenon our framework explains through conditional concentration. The resulting theory provides, to our knowledge, the first principled explanation for why binary quantization succeeds on contrastive embeddings and concrete guidance for choosing between rotation-based and coordinate-preserving system designs.
2
R ELATED W ORK
Binary quantization and LSH. The theoretical foundation for binary encoding was laid by locality-sensitive hashing (Indyk & Motwani, 1998) and SimHash (Charikar, 2002), which connects Hamming distance to angular similarity via Pr[h(x) = h(y)] = 1 − arccos(cos(x, y))/π. The semidefinite rounding analysis of Goemans & Williamson (1995) provides complementary guarantees. Crucially, these results assume isotropic data or analyze worst-case performance; they say nothing about how distributional structure—heterogeneous variances, non-trivial covariance— affects BQ quality. Our work shows that this structure is not merely present but is the dominant factor: ranking fidelity depends on the full covariance matrix, not just on dimension and angle. Modern BQ systems. Two recent systems illustrate the design tension that motivates our theory. RaBitQ (Gao & Long, 2024) rotates vectors to isotropy before binarization, then corrects distances with per-vector scalars, achieving an unbiased estimator with variance O(1/D). QuIVer (Xiao et al., 2026) takes the opposite approach: it preserves coordinate axes and builds the entire graph index—edge selection, pruning, navigation—natively in two-bit space. Both build on graphbased ANN indices (Malkov & Yashunin, 2020; Subramanya et al., 2019; Aumüller et al., 2020) and both achieve competitive recall, yet they make contradictory assumptions about coordinate structure. RaBitQ provides universal error bounds via a distribution-free analysis, but does not exploit the specific structure of contrastive embeddings; QuIVer demonstrates strong empirical performance in coordinate-preserving mode, but does not explain its theoretical basis. Neither identifies the role of coordinate heterogeneity. Contrastive representation structure. The distributional structure we exploit originates in the InfoNCE objective (van den Oord et al., 2018), which balances alignment of positive pairs with uniformity pressure on the hypersphere (Wang & Isola, 2020; Chen & He, 2021). This framework underlies SimCLR (Chen et al., 2020), MoCo (He et al., 2020), CLIP (Radford et al., 2021), and has been studied through downstream guarantees (Saunshi et al., 2019), spectral analysis (HaoChen et al., 2021), and identifiability (Zimmermann et al., 2021). The key theoretical result we build on is due to Betser et al. (2026), who proved that InfoNCE induces asymptotically Gaussian coordinates under alignment plateau and thin-shell concentration, formalizing observations from DINO (Caron et al., 2021) and VICReg (Bardes et al., 2022). By contrast, neural collapse (Papyan et al., 2020) produces maximally isotropic supervised features—a qualitatively different regime. Isotropypromoting methods (Ermolov et al., 2021; Bardes et al., 2022) regularize toward uniform variances but do not analyze what happens when residual heterogeneity remains. Our contribution is to show that this residual heterogeneity is not a defect but the signal that BQ exploits. 2
Arxiv Preprint Ver.
Vector quantization. Product quantization (Jégou et al., 2011), optimized PQ (Ge et al., 2014), and ScaNN (Guo et al., 2020) operate in the 4–64 bit regime with learned codebooks, implemented efficiently in FAISS (Johnson et al., 2019). Our analysis targets the extreme 1–2 bit setting where no codebook is needed and distances reduce to hardware-accelerated popcount. The mathematical tools are also different: we rely on Stein’s lemma (Stein, 1981; Liu, 1994), Hoeffding’s inequality (Hoeffding, 1963), and high-dimensional concentration (Vershynin, 2018; Boucheron et al., 2013; Diaconis & Freedman, 1987), rather than rate-distortion or codebook design theory.
3
P RELIMINARIES
3.1
B INARY Q UANTIZATION
Let x, y ∈ RD be unit-normalized embeddings. The 1-bit BQ score is S1 (x, y) =
D X
sign(xi ) sign(yi ),
(1)
i=1
which equals D − 2 · dH (sign(x), sign(y)) where dH is Hamming distance. TheP2-bit BQ score augments each sign with a magnitude bit mxi = 1[|xi | > αx ], where αx = 1 j |xj |: D D X (1 + mxi )(1 + myi ) sign(xi ) sign(yi ). (2) S2 (x, y) = i=1
3.2
F ROM F IDELITY TO R ECALL : T HE F /G D ECOMPOSITION
Search recall depends on two independent factors. The first is ranking fidelity F = ρSpearman (Sb , ⟨x, y⟩), the Spearman correlation between BQ scores and true inner products over random pairs. F measures how much noise quantization introduces into rankings. The second factor is the semantic margin structure of the data. Let q be a query, n1 its true nearest neighbor, and n2 a competitor. The true margin ∆ = ⟨q, n1 ⟩ − ⟨q, n2 ⟩ determines how hard the ranking problem is, independent of quantization. We denote this gap structure G. The two factors combine through a sub-Gaussian pairwise error bound: ! γb2 ∆2 P [pairwise misordering] ≤ exp − 2 , 2 σBQ,b
(3)
where γb (the calibration slope, a function of F ) controls how faithfully BQ scores track true inner 2 products, and σBQ,b is the BQ score noise variance. A union bound over K(N − K) candidate pairs gives P [top-K error] ≤ K(N − K) exp −cb ∆2min . (4) This decomposition clarifies the scope of our theory: we analyze F (the noise side) while treating G (the signal side) as a property of the data distribution. Improving F —through bitwidth, rotation, or system design—reduces BQ noise without requiring any change to the embedding model or data. 3.3
T HE G AUSSIAN P RIOR (H1)
Assumption 1 (Coordinate Gaussianity). Let g(x) ∈ RD denote the encoder output before normalization, and f (x) = g(x)/∥g(x)∥ the unit-normalized embedding. (H1a) For any fixed k coordinates I = {i1 , . . . , ik }, dBL (L(gI ), L(ZI )) ≤ εD where Z ∼ N (µ, Σ). (H1b) There exists a deterministic rD > 0 such that E[|∥g∥ − rD |]/rD ≤ δD and E[|∥Z∥ − rD |]/rD ≤ δD , with δD → 0. 3
Arxiv Preprint Ver.
1-bit / 2-bit rotation
§7 §4–
Fidelity F (noise) our analysis
InfoNCE training
Betser et al. Assumption H1
(Gaussian)
Covariance Σ
Recall data -giv en
Margin G (signal)
Figure 1: Causal structure of the framework. The Gaussian prior (H1) connects InfoNCE training to the covariance Σ. Recall decomposes into fidelity F (quantization noise, our focus) and semantic margin G (data-dependent, taken as given). Design choices—bitwidth and rotation—affect F through Σ. H1a is justified by Betser et al. (2026), who prove that InfoNCE induces asymptotically Gaussian coordinate distributions. H1b is a thin-shell concentration condition; for Gaussian Z, it holds whenever the effective rank tr(Σ)/∥Σ∥op → ∞. We empirically verify both conditions across 13 datasets and 7 embedding models, finding QQ-plot R2 ≥ 0.9959 and norm CV ≤ 0.09 in all cases. Roadmap. Figure 1 summarizes the logical structure of our framework. The Gaussian prior (H1) connects the training objective to the covariance matrix Σ, whose structure decomposes into heterogeneity (governing F ) and semantic margins (G). Our theorems analyze the F branch; G is treated as given.
4
R ANKING F IDELITY: A N E XACT A NALYSIS
4.1
S TEIN ’ S L EMMA AND THE O FF -D IAGONAL S TRUCTURE OF F
Theorem 1 (Linear covariance formula). Under Assumption 1, let Z ∼ N (µ, Σ), S0 = P P sign(Z ), and T = d Z i 0 i j j j . Define the Stein coefficient ai = 2φ(µi /σi )/σi , where 2
φ(t) = (2π)−1/2 e−t /2 . Then: Cov(S0 , T0 ) =
X
ai Σij dj .
(5)
i,j
In particular, F depends on the full covariance matrix Σ, not just its diagonal. The off-diagonal contribution to ranking quality is captured by the Stein squared signal: Ioff :=
X X 4 (ai Σij )2 = Σ2ij 2 φ2 (µi /σi ). σi i̸=j
(6)
i̸=j
This is not Cov(S0 , T0 ) itself, but the Frobenius energy of the off-diagonal part of the Stein coefficient matrix (ai Σij )i,j . P |A| Remark 1 (Exact closed form). Cov(pi , pj ) = ΦA (tA ; RA ) − E[pi ]E[pj ], A⊆{1,2,3,4} (−2) where ΦA denotes the |A|-dimensional Gaussian CDF. In the centered case this reduces to 2 π arcsin ρ. P 1 2 Theorem 2 (Weak correlations accumulate). Under a dispersal condition D(D−1) i̸=j ρij ≍ P κ2 /D, the off-diagonal Frobenius energy satisfies i̸=j ρ2ij ≍ κ2 D, contributing a constant fraction (30–50%) of the total Stein signal. 4
Arxiv Preprint Ver.
4.2
T HE ROTATION PARADOX
Theorem 3 (Rotation uniformizes coordinate variances). Let Q be a Haar-random orthogonal matrix. Then: (a) sign(Qx) defines a random-hyperplane LSH with P [h(x) arccos(cos(x, y))/π. p (b) maxi |qi⊤ Σqi − tr(Σ)/D| ≲ ∥Σ∥op log D/D.
=
h(y)]
=
1 −
Corollary 4 (Rotation harms heterogeneity-aware BQ). Rotation drives CV2 (σ) → 0 (Theorem 3b), eliminating the heterogeneity-dependent component ∆het of the 2-bit advantage (Theorem 6b). The residual gain ∆0 > 0 (the isotropic scalar magnitude advantage) survives, so 2-bit does not fully degrade to 1-bit; however, the data-dependent advantage that QuIVer exploits is destroyed. Corollary 5 (Rotation helps linear-corrected BQ). Uniformized SNR minimizes the estimation variance of the per-vector linear corrector dˆ = fadd + frescale × IP, since Var(dˆ − d) ∝ (1 − cos2 (r, x̄))/(D − 1) is minimized at isotropy.
5
W HY 2-B IT B EATS 1-B IT: T HE M AGNITUDE I NFORMATION G AIN
Theorem 6 (2-bit strict superiority and heterogeneity monotonicity). Under Assumption 1: (a) Strict superiority. For each coordinate i with σi > 0, the per-dimension information ratio p (2) (1) 2 2 satisfies ηi := (ai +b ) /(1+3p ) > a =: η , where a = σ 2/π, b i i i i i = 2σi φ(α/σi ), i i √ and pi = erfc(α/(σi 2)). Consequently, ρ(S2 , r) > ρ(S1 , r). (b) Heterogeneity boosts the gain. Decompose the 2-bit advantage as ∆ρ(σ) = ∆0 +∆het (σ), where ∆0 = ρ2 (σ̄1) − ρ1 (σ̄1) > 0 is the isotropic baseline gain. Then ∆het (σ) = K · CV2 (σ)+O(CV3 ) with K > 0. In words: heterogeneity provides additional magnitude-bit gain beyond the scalar quantization baseline. √ 2 2 Proof sketch. Part (a): Define fp (t) = 2e−t /2 + e−t − 3 erfc(t/ 2). We show f (t) > 0 for all 2 t > 0 via f (0) = 0, f ′ (0) = 3 2/π > 0, and the strict monotonicity of g(t) = 2t(1 + e−t /2 ). Part (b): Taylor-expand ρ2 (σ) and ρ1 (σ) around ui = σi /σ̄ = 1, yielding K ≈ 0.088 > 0. Full proof in Appendix C.
6
BQ IN G RAPH -BASED S EARCH
The F /G decomposition of §3.2 governs brute-force search, where every corpus vector is scored. Graph-based methods (HNSW, Vamana) operate differently: each navigation step ranks only ∼32– 64 neighbors, and these neighbors are angularly much closer than random corpus pairs. The relevant quantity is therefore the local fidelity Flocal —ranking accuracy within a node’s neighbor set—rather than the global F over all pairs. Because the margin ∆ is smaller in this local regime but the noise σBQ also shrinks (coordinates of nearby vectors are more correlated), the net effect is an amplification of ranking accuracy. The following theorem formalizes this. Theorem 7 (Local sub-Gaussianity and MMSE advantage). Let q be a query and {n1 , . . . , nM } be graph neighbors with margin ∆ = cos(q, n1 ) − cos(q, n2 ). (a) Assume that, conditioned on the neighbor set E, the BQ score differences Uj = Sb (q, n1 ) − 2 Sb (q, nj ) satisfy: Uj − E[Uj |E] is σlocal,b -sub-Gaussian, and E[Uj |E] ≥ γb ∆. Then union 2 bound gives P [top-1 error|E] ≤ (M − 1) exp(−γb2 ∆2 /(2σlocal,b )). (b) The optimal-calibration residual variance of 2-bit is strictly smaller than that of 1-bit: since C1 ⊆ C2 , the L2 projection yields E[(T − E[T |C2 ])2 ] ≤ E[(T − E[T |C1 ])2 ]. 5
Arxiv Preprint Ver.
Table 1: Two opposite strategies for handling coordinate heterogeneity.
Encoding Heterogeneity Distance function Extra storage Quality guarantee Best for
7
Weighted Hamming (QuIVer)
Linear-Corrected (RaBitQ)
2-bit (sign + magnitude) Exploited as signal popcount 0 bytes/vector Data-dependent Graph index (HNSW)
1-bit (sign) + float correction Eliminated by rotation fadd + frescale × IP 12 bytes/vector Universal: O(1/D) IVF brute scan
A C LOSED -F ORM S CALING L AW
Theorem 8 (Scaling law for ranking fidelity). Let r = ∥Σoff ∥F /∥diag(Σ)∥F , m = |SNR|, and v = std(|SNR|). Under a dispersal assumption on Σoff : 6 ρ0 + λr2 Am (v)2 1 m2 F = arcsin + O(ϵ), Am (v) = √ . (7) exp − π 2 1 + 2v 2 1 + 2v 2 The linearization gives F ≈ β0 + β1 z(log r) + β2 z(v) with β1 > 0 and β2 < 0 (when 2m2 < 1 + 2v 2 ).
8
D ESIGN I MPLICATIONS : T WO S TRATEGIES FOR H ETEROGENEITY
These two paradigms are not competing solutions to the same problem; they are opposite strategies for handling the same physical quantity. Theorem 3 and Corollaries 4–5 make this precise: rotation maps one regime into the other. This exposes a fundamental tradeoff in BQ design: universality (distribution-free guarantees via rotation, as in RaBitQ) versus exploitation (leveraging coordinate structure for higher fidelity on the distributions that actually arise, as in QuIVer). RaBitQ provides O(1/D) variance bounds that hold for any distribution by treating heterogeneity as noise to be eliminated; our analysis reveals that this “noise” is in fact exploitable signal. Neither strategy dominates: the appropriate choice depends on whether the Gaussian prior (Assumption 1) holds.
9
E XPERIMENTS
Our theory rests on a chain of claims, each building on the previous: the Gaussian prior (Assumption 1) enables the fidelity formula, which in turn explains the magnitude bit gain, the rotation paradox, and the scaling law. We design experiments to test this chain link by link, so that any failure point would localize the gap between theory and practice. Throughout, the non-Gaussian GIST-960 dataset serves as a negative control—a distribution where our framework should and does break down. We report representative results here; full tables for all 13 datasets appear in Appendix F. 9.1
I S THE G AUSSIAN P RIOR J USTIFIED ?
The entire framework rests on Assumption 1—that coordinates are approximately Gaussian with concentrated norms. We test this by fitting per-coordinate QQ-plots against N (µ̂i , σ̂i2 ) on 50K embeddings from each dataset, reporting the mean R2 and the norm coefficient of variation. The answer is unambiguous (Table 2): every contrastive model achieves R2 ≥ 0.9959 with norm CV below 0.09. The non-contrastive GIST-960 fails completely (R2 ≈ 0, CV = 0.36), confirming that the Gaussian prior is specific to contrastive training and not an artifact of high dimensionality. 9.2
D OES THE F ULL C OVARIANCE M ATTER ?
With the Gaussian prior confirmed, we can test the fidelity formula. A natural baseline ignores offdiagonal covariance entirely, predicting F from variances alone (Fdiag ). Our theory (Theorem 1) predicts that the full covariance Ffull should match the empirical Factual much more closely. 6
Arxiv Preprint Ver.
Table 2: Coordinate Gaussianity verification (Assumption 1). All contrastive models exhibit QQplot R2 > 0.99 and thin-shell CV < 0.1. Dataset
Model
D
QQ R2
CV
Cohere-1M BGE-M3 MiniLM Landmark CodeSearch Random
Cohere-v3 BGE-M3 MiniLM-L6 DINOv2 (SSL) Jina-v2 N/A
768 1024 384 768 768 768
0.9996 0.9989 0.998+ 0.9965 0.9975 0.9967
0.04 0.05 0.06 0.07 0.05 0.08
GIST-960
Hand-crafted
960
N/A
0.36
Table 3: Ranking fidelity F : diagonal-only vs. full-Σ prediction vs. actual. Off-diagonal covariance contributes 30–50% of the signal. Dataset
Factual
Ffull
Fdiag
Expl. ratio
Cohere Arxiv CodeSearch Random
0.681 0.897 0.823 0.907
0.688 0.886 0.837 0.899
0.474 0.546 0.542 0.560
103% 97% 105% 98%
Table 3 confirms this dramatically: the diagonal-only prediction underestimates fidelity by 0.20– 0.36, while the full-Σ prediction matches within ±0.02 (mean explanation ratio 103%). The offdiagonal correlations are individually tiny (|ρij | ≈ 0.04–0.11), but there are D2 of them, and their collective contribution accounts for 30–50% of the ranking signal (Theorem 2). This is perhaps the most surprising empirical finding: the information that makes BQ work is predominantly relational (between coordinates), not marginal (within each coordinate). 9.3
I S THE S ECOND B IT W ORTH THE S TORAGE ?
Doubling the code length from 1 to 2 bits per coordinate doubles storage. Is the information gain worth it? Theorem 6 predicts yes—and that the gain should grow with coordinate heterogeneity. Table 4 confirms both predictions. The fidelity gain ∆F is strictly positive on all six datasets (+0.049 to +0.132), and the recall improvement ranges from +0.110 to +0.210. The monotonicity with heterogeneity is approximate but not perfect, likely reflecting finite-sample noise and residual nonGaussianity. The practical message is clear: for contrastive embeddings, the second bit is not a luxury but a near-doubling of the useful information per coordinate. 9.4
W HY D OES ROTATION H ELP O NE S YSTEM BUT H URT A NOTHER ?
This is the central design puzzle: RaBitQ rotates before binarization; QuIVer explicitly avoids rotation. Both succeed. Our theory predicts that rotation uniformizes variances (Theorem 3), which helps linear correctors (Corollary 5) but destroys the implicit weighting that Hamming distance exploits (Corollary 4). Table 5 confirms this prediction through a revealing spectrum of behaviors. For GIST—a degenerate distribution where all coordinates share the same sign—rotation is transformative, injecting the sign entropy that BQ needs to function at all. For contrastive embeddings, the story reverses: rotation is neutral (MiniLM, already near-isotropic) or slightly harmful (Cohere, which has well-calibrated heterogeneity that 2-bit BQ exploits). The practical implication is precise: rotate if you will apply a linear corrector; preserve axes if you rely on Hamming distance. 7
Arxiv Preprint Ver.
Table 4: Magnitude bit information gain across datasets. ∆F > 0 holds universally; the gain correlates with coordinate heterogeneity CV(σ). Dataset
CV(σ)
∆F
∆R
MiniLM Cohere CodeSearch Landmark Arxiv Random
0.118 0.182 0.098 0.110 0.108 0.070
+0.132 +0.091 +0.088 +0.088 +0.071 +0.049
+0.174 +0.144 +0.143 +0.110 +0.159 +0.210
Table 5: Effect of Haar-random rotation on sign entropy and BQ recall. The response depends entirely on the initial variance structure. Dataset GIST Wolt-CLIP Cohere MiniLM
9.5
Entropy: before → after
∆ Recall
Regime
0.000 → 0.511 0.836 → 0.616 0.747 → 0.563 ≈const
+307% +3.2pp −0.5pp ≈0
Degenerate Over-spread Near-optimal Isotropic
W HY D OES BQ W ORK B ETTER T HAN E XPECTED IN G RAPH S EARCH ?
A puzzling observation in practice is that BQ achieves high recall in graph-based search even when its global fidelity F is modest. The resolution lies in the distinction between global and local ranking accuracy. Graph neighbors are angularly much closer than random pairs, and Theorem 7 predicts that BQ’s ranking accuracy improves in this local regime. We measure local fidelity by sampling 10K nodes from a Vamana graph (m = 32) and computing the Spearman correlation between BQ and float32 rankings of each node’s neighbor set. The local advantage is substantial: Flocal exceeds Fglobal by 1.2–4.1× across all contrastive datasets. This explains why QuIVer achieves 95% Recall@10 despite Fglobal ≈ 0.68: at each navigation step, the local fidelity is high enough to make the correct greedy choice. Full results appear in Appendix F. 9.6
C AN W E P REDICT F IDELITY W ITHOUT RUNNING S EARCH ?
The ultimate test of a theory is prediction. We fit the scaling law (Theorem 8) on 768-dimensional datasets and ask: can it predict F for models at different dimensions, without ever seeing their search results? The answer is yes, with surprising accuracy. The two-parameter model achieves R2 = 0.928 insample and LOO-R2 = 0.889. Out-of-distribution, it predicts BGE-M3 at 1024-d within 0.012 and MiniLM at 384-d within 0.041 (MAE = 0.038). The negative control is equally informative: GIST960 produces a nonsensical prediction of F = 1.037, a clear signal that the Gaussian prior has been violated. The scaling law thus serves a dual purpose: it predicts fidelity when the prior holds, and diagnoses model suitability when it does not.
10
C ONCLUSION
Binary quantization is often viewed as a lossy compression technique—a necessary evil for scaling vector search. Our analysis suggests a different perspective: BQ is a covariance probe. The sign bit detects whether a coordinate is above or below its mean; the magnitude bit detects whether its deviation is large or small. Together, they form a two-bit summary statistic that captures the first and (partially) second moments of each coordinate. When these moments carry meaningful information about inter-point distances—as they do under the Gaussian structure induced by InfoNCE—BQ preserves ranking fidelity; when they do not, BQ fails. 8
Arxiv Preprint Ver.
This lens resolves the apparent paradox of contradictory system designs. Coordinate-preserving methods (QuIVer) succeed because heterogeneous variances create an implicit importance weighting that Hamming distance inherits. Rotation-based methods (RaBitQ) succeed because isotropy is precisely the condition under which a linear scalar correction becomes unbiased. These are not competing solutions; they are dual strategies for exploiting the same underlying covariance structure. Looking forward, our framework opens several directions. The scaling law could serve as a modelselection criterion: given only the covariance statistics of an embedding, one can predict whether BQ will achieve acceptable recall—and if so, which system design is preferable—without running any search experiments. More broadly, the tight correspondence between distributional structure and quantization quality suggests that the design of training objectives and the design of compression methods should be studied jointly, rather than in isolation. Limitations. Our theory requires approximate coordinate Gaussianity (Assumption 1), a condition met by InfoNCE-trained models but not by supervised or hand-crafted features. The scaling law’s three input statistics require access to the embedding covariance, which may be expensive for very large corpora. The graph navigation bounds (Theorem 7) achieve O(D2 ) sub-Gaussian parameters unconditionally; the sharper O(D) rate requires a weak-dependence assumption whose empirical verification remains open.
R EFERENCES Martin Aumüller, Erik Bernhardsson, and Alexander Faithfull. Ann-benchmarks: A benchmarking tool for approximate nearest neighbor algorithms. Information Systems, 87:101374, 2020. Adrien Bardes, Jean Ponce, and Yann LeCun. Vicreg: Variance-invariance-covariance regularization for self-supervised learning. In International Conference on Learning Representations, 2022. Roy Betser, Eyal Gofer, Meir Yossef Levi, and Guy Gilboa. Infonce induces gaussian distribution. arXiv preprint arXiv:2602.24012, 2026. Stéphane Boucheron, Gábor Lugosi, and Pascal Massart. Concentration Inequalities: A Nonasymptotic Theory of Independence. Oxford University Press, 2013. Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. In IEEE/CVF International Conference on Computer Vision, pp. 9650–9660, 2021. Moses S Charikar. Similarity estimation techniques from rounding algorithms. In Proceedings of the 34th Annual ACM Symposium on Theory of Computing, pp. 380–388, 2002. Jianlv Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu. BGE M3-Embedding: Multi-lingual, multi-functionality, multi-granularity text embeddings through self-knowledge distillation. arXiv preprint arXiv:2402.03216, 2024. Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International Conference on Machine Learning, pp. 1597–1607, 2020. Xinlei Chen and Kaiming He. Exploring simple siamese representation learning. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 15750–15758, 2021. Persi Diaconis and David Freedman. A dozen de finetti-style results in search of a theory. Annales de l’IHP Probabilités et statistiques, 23(S2):397–423, 1987. Aleksandr Ermolov, Aliaksandr Siarohin, Enver Sangineto, and Nicu Sebe. Whitening for selfsupervised representation learning. In International Conference on Machine Learning, pp. 3015– 3024, 2021. Jianyang Gao and Cheng Long. Rabitq: Quantizing high-dimensional vectors with a theoretical error bound for approximate nearest neighbor search. Proceedings of the ACM on Management of Data (SIGMOD), 2(3):1–27, 2024. 9
Arxiv Preprint Ver.
Tiezheng Ge, Kaiming He, Qifa Ke, and Jian Sun. Optimized product quantization. IEEE Transactions on Pattern Analysis and Machine Intelligence, 36(4):744–755, 2014. Michel X Goemans and David P Williamson. Improved approximation algorithms for maximum cut and satisfiability problems using semidefinite programming. Journal of the ACM, 42(6):1115– 1145, 1995. Ruiqi Guo, Philip Sun, Erik Lindgren, Quan Geng, David Simcha, Felix Chern, and Sanjiv Kumar. Accelerating large-scale inference with anisotropic vector quantization. In International Conference on Machine Learning, pp. 3887–3896, 2020. Jeff Z HaoChen, Colin Wei, Adrien Gaidon, and Tengyu Ma. Provable guarantees for self-supervised deep learning with spectral contrastive loss. In Advances in Neural Information Processing Systems, volume 34, 2021. Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. Momentum contrast for unsupervised visual representation learning. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9729–9738, 2020. Wassily Hoeffding. Probability inequalities for sums of bounded random variables. Journal of the American Statistical Association, 58(301):13–30, 1963. Piotr Indyk and Rajeev Motwani. Approximate nearest neighbors: towards removing the curse of dimensionality. In Proceedings of the 30th Annual ACM Symposium on Theory of Computing, pp. 604–613, 1998. Hervé Jégou, Matthijs Douze, and Cordelia Schmid. Product quantization for nearest neighbor search. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(1):117–128, 2011. Jeff Johnson, Matthijs Douze, and Hervé Jégou. Billion-scale similarity search with gpus. IEEE Transactions on Big Data, 7(3):535–547, 2019. William H Kruskal. Ordinal measures of association. Journal of the American Statistical Association, 53(284):814–861, 1958. Jin S Liu. A note on stein’s lemma for multivariate elliptical distributions. Journal of Statistical Planning and Inference, 38(2):227–232, 1994. Yu A Malkov and D A Yashunin. Efficient and robust approximate nearest neighbor search using hierarchical navigable small world graphs. IEEE Transactions on Pattern Analysis and Machine Intelligence, 42(4):824–836, 2020. Zach Nussbaum, Brandon Duderstadt, John X Morris, and Andriy Mulyar. Nomic embed: Training a reproducible long context text embedder. arXiv preprint arXiv:2402.01613, 2024. Vardan Papyan, X Y Han, and David L Donoho. Prevalence of neural collapse during the terminal phase of deep learning training. Proceedings of the National Academy of Sciences, 117(40): 24652–24663, 2020. Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International Conference on Machine Learning, pp. 8748–8763, 2021. Nils Reimers and Iryna Gurevych. Sentence-bert: Sentence embeddings using siamese bertnetworks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing, pp. 3982–3992, 2019. Nikunj Saunshi, Orestis Plevrakis, Sanjeev Arora, Mikhail Khodak, and Hrishikesh Khandeparkar. A theoretical analysis of contrastive unsupervised representation learning. In International Conference on Machine Learning, pp. 5628–5637, 2019. Charles M Stein. Estimation of the mean of a multivariate normal distribution. The Annals of Statistics, pp. 1135–1151, 1981. 10
Arxiv Preprint Ver.
Suhas Jayaram Subramanya, Fnu Devvrit, Harsha Vardhan Simhadri, Ravishankar Krishnaswamy, and Rohan Kadekodi. Diskann: Fast accurate billion-point nearest neighbor search on a single node. In Advances in Neural Information Processing Systems, volume 32, 2019. Aäron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748, 2018. Roman Vershynin. High-Dimensional Probability: An Introduction with Applications in Data Science. Cambridge University Press, 2018. Tongzhou Wang and Phillip Isola. Understanding contrastive representation learning through alignment and uniformity on the hypersphere. In International Conference on Machine Learning, pp. 9929–9939, 2020. Wenxuan Xiao, Peidong Zhu, and Chengcheng Li. Quiver: Rethinking ann graph topology via training-free binary quantization. arXiv preprint, 2026. Roland S Zimmermann, Yash Sharma, Steffen Schneider, Matthias Bethge, and Wieland Brendel. Contrastive learning inverts the data generating process. In International Conference on Machine Learning, pp. 12979–12990, 2021.
A
P ROOF OF T HEOREM 1 (S TEIN ’ S L EMMA FOR BQ)
A.1
L INEAR C OVARIANCE F ORMULA (C1)
By bilinearity of covariance, Cov(S0 , T0 ) = Cov(sign(Zi ), Zj ) = ai Σij .
P
i,j dj Cov(sign(Zi ), Zj ).
It suffices to show
Since Z is jointly Gaussian, the conditional expectation is linear: E[Zj − µj | Zi ] =
Σji (Zi − µi ). σi2
(8)
Therefore, Cov(sign(Zi ), Zj ) = E[sign(Zi )(Zj − µj )] = E[sign(Zi ) · E[Zj − µj | Zi ]] Σji = 2 E[(Zi − µi ) sign(Zi )]. σi
(9)
Write Zi = µi + σi X with X ∼ N (0, 1). Then E[(Zi − µi ) sign(Zi )] = σi E[X sign(X + si )] = 2σi E[X1{X>−si } ] = 2σi φ(si ),
(10)
where we used the truncated moment identity E[X1{X>a} ] = φ(a) and the symmetry φ(−si ) = φ(si ). Substituting back into (9): Cov(sign(Zi ), Zj ) =
Σij 2φ(si ) · 2σi φ(si ) = Σij = ai Σij . 2 σi σi
A.2
O FF -D IAGONAL S QUARED S IGNAL (C2) P Substituting ai = 2φ(si )/σi into Ioff = i̸=j (ai Σij )2 yields Eq. (6) directly. 11
(11)
Arxiv Preprint Ver.
A.3
S IGN -P RODUCT M OMENT F ORMULA
For completeness, we record the exact formula. Write sign(w) = 1 − 21{w≤0} . Expanding the product and taking expectations:
E
k Y
sign(Wℓ ) =
ℓ=1
X
(−2)|A| ΦA (tA ; RA ),
(12)
A⊆[k]
where ta = −µa /σa and RA is the correlation submatrix of (Wa )a∈A . In the centered bivariate 1 case with correlation ρ, the classical quadrant probability P (W1 > 0, W2 > 0) = 14 + 2π arcsin ρ 2 gives E[sign(W1 ) sign(W2 )] = π arcsin ρ.
B
P ROOF OF T HEOREM 3 (ROTATION E FFECTS )
B.1
PART ( A ): R ANDOM -H YPERPLANE LSH
Let qi⊤ be the i-th row of a Haar-random Q ∈ O(D). Each qi is uniformly distributed on SD−1 . The event sign(qi⊤ x) ̸= sign(qi⊤ y) means the hyperplane qi⊥ separates x and y. By spherical symmetry, the separation probability equals the angle θ = arccos⟨x, y⟩ divided by π: P [sign(qi⊤ x) = sign(qi⊤ y)] = 1 − B.2
arccos⟨x, y⟩ . π
(13)
PART ( B ): VARIANCE U NIFORMIZATION
Define Vi = qi⊤ Σ qi . Since E[qi qi⊤ ] = D−1 ID , we have E[Vi ] = tr(Σ)/D. The function F (q) = q ⊤ Σ q on SD−1 satisfies |F (q) − F (q ′ )| ≤ 2∥Σ∥op ∥q − q ′ ∥, so it is Lipschitz with constant L = 2∥Σ∥op . By Lévy’s concentration inequality on SD−1 : tr(Σ) Dt2 P Vi − ≥ t ≤ C exp −c . D ∥Σ∥2op
A union bound over i = 1, . . . , D with t = C ′ ∥Σ∥op ′ 1 − D−c , max
1≤i≤D
tr(Σ) qi⊤ Σ qi − D
(14)
p (log D)/D yields: with probability ≥ r
≲ ∥Σ∥op
log D . D
(15)
To ensure CV2 (σ̃) → 0, the relative error must vanish, which requires the effective rank condition tr(Σ)/∥Σ∥op → ∞. B.3
C OROLLARY 4: ROTATION H ARMS H ETEROGENEITY-AWARE BQ p Under the effective rank condition, Part (b) gives σ̃i = tr(Σ)/D (1 + o(1)) uniformly, so 2 CV (σ̃) → 0. By Theorem 6(b), the heterogeneity-dependent gain ∆het = K·CV2 (σ̃)+O(CV3 ) → 0. The isotropic baseline ∆0 > 0 survives; hence 2-bit does not fully degrade to 1-bit, but the datadependent advantage is destroyed. B.4
C OROLLARY 5: ROTATION H ELPS L INEAR -C ORRECTED BQ
This corollary requires the external error formula from RaBitQ (Gao & Long, 2024) as input. If Var(dˆ− d) ∝ (1 − cos2 (r, x̄))/(D − 1), then isotropy maximizes cos2 (r, x̄) and thus minimizes the variance. 12
Arxiv Preprint Ver.
C
P ROOF OF T HEOREM 6 (M AGNITUDE B IT G AIN )
C.1
PART ( A ): S TRICT S UPERIORITY
√ 2 (2) (1) We need ηi > ηi , i.e., (1 + e−t /2 )2 > 1 + 3 erfc(t/ 2) for all t > 0, where t = α/σi . Equivalently, define √ 2 2 f (t) = 2e−t /2 + e−t − 3 erfc(t/ 2). (16) We show f (t) > 0 for all t > 0. Step 1: Boundary values. f (0) = 2 + 1 − 3 = 0 and limt→∞ f (t) = 0. p √ 2 d erfc(t/ 2) = − 2/π q(t). Step 2: Derivative. Let q(t) = e−t /2 . Then q ′ (t) = −tq(t) and dt So h p i f ′ (t) = q(t) 3 2/π − 2t(1 + q(t)) . (17) Step 3: Monotonicity of h(t) = 2t(1 + q(t)). We have h′ (t) = 2{1 + q(t)(1 − t2 )}. For 0 < t ≤ 1: 2 h′ (t) > 0 trivially. For t > 1: (t2 − 1)q(t) ≤ maxt>1 (t2 − 1)e−t /2p = 2e−3/2 < 1, so h′ (t) > 0. Hence h is strictly increasing from 0 to ∞, and the equation h(t) = 3 2/π has a unique root t∗ . Step 4: Conclusion. f ′ (t) > 0 for t < t∗ and f ′ (t) < 0 for t > t∗ . Combined with f (0) = 0 and f (∞) = 0: f increases then decreases, so f (t) > 0 for all t > 0. C.2
PART ( B ): H ETEROGENEITY B OOSTS THE G AIN p 2 2 Let ui = σ√ 2/π, α = σ̄τ . Define C(u) = τ u(1 + e−τ /(2u ) ) and V (u) = 1 + i /σ̄, τ = 3 erfc(τ /(u 2)). In the independent-coordinate model with uniform weighting: τ
, ρ1 (σ) = q 1 + CV2 (σ)
P D−1 i C(ui ) . ρ2 (σ) = p P P (D−1 i V (ui ))(D−1 i u2i )
(18)
Taylor-expanding around ui = 1 with s2 = CV2 (σ): ρ1 = τ − τ2 s2 + O(s4 ), h ′′ i (1) V ′′ (1) 2 1 ρ2 = ρ2,0 1 + C + O(s3 ), 2C(1) − 4V (1) − 2 s where ρ2,0 = C(1)/ where
(19) (20)
p V (1) ≈ 0.914. The difference decomposes as ∆ρ = ∆0 + Ks2 + O(s3 ),
∆0 = ρ2,0 − τ ≈ 0.116,
K = ρ2,0
C ′′ (1) V ′′ (1) 1 2C(1) − 4V (1) − 2
D
P ROOF OF T HEOREM 7 (G RAPH NAVIGATION )
D.1
PART ( A ): L OCAL S UB -G AUSSIAN TAIL
+ τ2 ≈ 0.088 > 0.
(21)
Define Uj = Sb (q, n1 ) − Sb (q, nj ) for j ≥ 2. By assumption, conditioned on E, Uj − E[Uj |E] is 2 σlocal,b -sub-Gaussian, and E[Uj |E] ≥ γb ∆. A top-1 error occurs iff ∃j ≥ 2 : Uj ≤ 0. For fixed j: ! γb2 ∆2 P [Uj ≤ 0 | E] ≤ P [Uj − E[Uj |E] ≤ −γb ∆ | E] ≤ exp − 2 . (22) 2σlocal,b Union bound over j = 2, . . . , M completes the proof. 13
Arxiv Preprint Ver.
D.2
PART ( B ): MMSE A DVANTAGE
Since 2-bit code contains sign and magnitude bits, C1 ⊆ C2 , hence L2 (C1 ) ⊆ L2 (C2 ). Conditional expectation is L2 -projection, so projection onto the larger subspace cannot increase error. By the conditional variance decomposition: E[(T − E[T |C1 ])2 ] = E[(T − E[T |C2 ])2 ] + E[(E[T |C2 ] − E[T |C1 ])2 ].
(23)
The second term is non-negative, yielding the result.
E
P ROOF OF T HEOREM 8 (S CALING L AW )
This proof requires the mean-field closure assumption (S0) in addition to Theorems 1 and 2.
E.1
S TEP 1: O FF -D IAGONAL S IGNAL FROM S TEIN ’ S F ORMULA
By Theorem 1, Cov(sign(Zi ), Zj ) = ai Σij with ai = 2φ(si )/σi . The squared off-diagonal energy is X
Ioff = 4
σj2 ρ2ij φ2 (si ) =
i̸=j
E.2
2 X 2 2 −s2i σj ρij e . π
(24)
i̸=j
S TEP 2: M EAN -F IELD C LOSURE
P 2 2 Introduce weights wij = Σ2ij /∥Σoff ∥2F . Assumption S0 states that i̸=j wij e−(si +sj ) = Am (v)2 + o(1). This decouples the off-diagonal geometry from the SNR distribution.
E.3
S TEP 3: C OMPUTING Am (v)
If |si | has empirical distribution approximated by S ∼ N (m, v 2 ): Am (v) = E[e
−S 2
]= √
1 2πv 2
Z e
−x2 −(x−m)2 /(2v 2 )
e
R
1 m2 dx = √ exp − , (25) 1 + 2v 2 1 + 2v 2
by completing the square in the exponent.
E.4
S TEP 4: P EARSON TO S PEARMAN
Absorbing diagonal contributions into ρ0 and normalization into λ: ρP = ρ0 + λr2 Am (v)2 + O(ϵ). By the Kruskal formula (Kruskal, 1958) for bivariate normals: F =
E.5
ρ 6 P arcsin + O(ϵ). π 2
(26)
S TEP 5: S IGNS OF THE L INEARIZATION C OEFFICIENTS
∂(r2 Am (v)2 )/∂(log r) = 2r2 Am (v)2 > 0, so β1 > 0. For v: ∂ log Am /∂v = 2v(2m2 − 1 − 2v 2 )/(1 + 2v 2 )2 , which is negative when 2m2 < 1 + 2v 2 , giving β2 < 0. 14
Arxiv Preprint Ver.
F
A DDITIONAL E XPERIMENTAL R ESULTS
F.1
G AUSSIANITY V ERIFICATION (P ROBE 12A)
Table 6: Marginal Gaussianity across 6 datasets. AD = Anderson–Darling test at 5%.
F.2
Dataset
|skew|
|kurt|
AD pass%
QQ R2
Cohere-768 BGE-M3-1024 wolt clip-512 MiniLM-384 Random-768 GIST-960
0.060 0.098 0.147 ∼0 0.032 1.332
0.083 0.094 0.177 ∼0 0.046 3.408
79.5 64.0 42.0 87.0 89.0 0.0
0.9996 0.9989 0.9963 0.9222 0.9967 <0
F ULL -Σ F IDELITY V ERIFICATION (P ROBE 16 D )
Table 7: F explained by full covariance Σ vs. diagonal-only. “Cov Expl.” = (Ffull − Fdiag )/(Freal − Fdiag ). Dataset
Freal
Ffull
Fdiag
Gap
Cov Expl.
Residual
cohere arxiv nomic landmark nomic coco nomic codesearch jina gooaq roberta landmark dino random
0.681 0.897 0.904 0.856 0.823 0.738 0.735 0.907
0.688 0.886 0.877 0.830 0.837 0.767 0.807 0.899
0.474 0.546 0.540 0.529 0.542 0.535 0.430 0.560
0.207 0.350 0.364 0.327 0.281 0.204 0.304 0.348
103.4% 96.8% 92.6% 92.2% 104.7% 113.8% 123.9% 97.7%
−0.007 +0.011 +0.027 +0.026 −0.013 −0.028 −0.073 +0.008
Mean Cov Explanation: 103.1%
F.3
Residual MAE: 0.024
S CALING L AW C ROSS -D IMENSIONAL VALIDATION (P ROBE 16 G )
Training on 768-d datasets, predicting on held-out dimensions. 2 0.079 z(log r) − 0.041 z(v), Rtrain = 0.928, LOO-R2 = 0.889.
Model: F
Table 8: Cross-dimensional out-of-sample prediction. Dataset
D
Freal
Fpred
Error
Note
BGE-M3 wolt clip MiniLM GIST
1024 512 384 960
0.918 0.862 0.731 0.477
0.907 0.800 0.690 1.037
+0.012 +0.062 +0.041 −0.560
non-Gaussian
MAE (excl. GIST)
0.038
15
≈ 0.820 +
Arxiv Preprint Ver.
F.4
M AGNITUDE B IT G AIN (P ROBE 18)
Table 9: Spearman F and Recall@10 gain from 1-bit to 2-bit.
F.5
Dataset
F (Spearman) 1-bit 2-bit
Recall@10 1-bit 2-bit
∆F
∆R
cohere random minilm arxiv landmark codesearch
0.704 0.929 0.783 0.887 0.833 0.860
0.437 0.446 0.521 0.525 0.506 0.671
+0.091 +0.049 +0.132 +0.071 +0.088 +0.088
+0.144 +0.210 +0.174 +0.159 +0.110 +0.143
0.795 0.977 0.915 0.958 0.920 0.948
0.581 0.656 0.695 0.684 0.616 0.814
Gain
G RAPH NAVIGATION : L OCAL A MPLIFICATION (P ROBE 19)
Table 10: Global vs. local (32-NN) F advantage and top-1 accuracy. Dataset
∆F (2-bit − 1-bit) Global Local Amp.
Top-1 Accuracy 1-bit 2-bit ∆
cohere random minilm arxiv landmark codesearch
+0.091 +0.048 +0.132 +0.071 +0.087 +0.088
0.452 0.326 0.484 0.444 0.434 0.646
+0.115 +0.200 +0.188 +0.153 +0.101 +0.142
1.3× 4.1× 1.4× 2.1× 1.2× 1.6×
0.564 0.486 0.664 0.616 0.482 0.778
+0.112 +0.160 +0.180 +0.172 +0.048 +0.132
Mean amplification: 2.0×
F.6
ROTATION PARADOX (P ROBE 8)
Table 11: Effect of Haar rotation on sign entropy and BQ recall. Dataset
Horig
Hrot
∆H
Rorig
Rrot
∆R
Cohere BGE-M3 wolt clip MiniLM GIST Random
0.747 0.702 0.836 0.987 0.000 0.981
0.563 0.671 0.616 0.991 0.511 0.981
−0.184 −0.030 −0.220 +0.004 +0.511 +0.000
0.486 0.782 0.564 0.546 0.152 0.487
0.481 0.775 0.596 0.565 0.459 0.493
−0.005 −0.007 +0.032 +0.019 +0.307 +0.006
16
Arxiv Preprint Ver.
F.7
9-DATASET G AUSSIANITY V ERIFICATION (P HASE 5, P ROBE 10)
Table 12: Anisotropic Gaussian model verification across 9 datasets (768-d). R2 : QQ-plot fit of predicted vs. observed sign entropy per coordinate. R2
MAE
|SNR|
High%
Low%
Hobs
Hpred
0.9996 0.9996 0.9992 0.9977 0.9976 0.9975 0.9967 0.9965 0.1669†
0.004 0.003 0.004 0.005 0.005 0.001 0.001 0.002 0.002
0.884 0.877 0.884 1.681 1.615 0.139 0.164 0.295 0.054
25.9 36.8 37.0 74.2 75.5 0.1 0.0 2.1 0.1
11.3 13.7 11.5 4.3 5.3 61.2 52.0 36.6 89.5
0.747 0.690 0.689 0.382 0.394 0.986 0.981 0.944 0.996
0.748 0.691 0.690 0.382 0.396 0.986 0.982 0.944 0.997
Dataset cohere ccnews nomic arxiv nomic coco nomic landmark nomic codesearch jina random landmark dino gooaq roberta
†
Low R2 due to
near-zero entropy variance (MAE is smallest); see text.
F.8
C OORDINATE S IGN VS . R ANDOM H YPERPLANE (P ROBE 3)
Table 13: Pairwise overlap probabilities: coordinate sign BQ vs. random-hyperplane LSH. GW = Goemans–Williamson theoretical value 1 − arccos(cos θ)/π.
F.9
Dataset
Coord sign
Random HP
GW theory
KL(c∥rh)
Sign entropy
Cohere BGE-M3 MiniLM GIST Random
0.651 0.677 0.508 0.9999 0.513
0.744 0.701 0.506 0.768 0.513
0.746 0.692 0.506 0.774 0.513
0.0229 0.0022 0.0021 0.2648 0.0010
0.747 0.700 0.987 0.000 0.981
G AUSSIAN C OPULA R ESIDUAL A NALYSIS (P ROBE 16 H )
Table 14: Gaussian vs. Gaussian Copula predictions of F . Copula matches all marginal moments but retains Gaussian dependence.
F.10
Dataset
Freal
Fgauss
Fcopula
Res(G)
Res(C)
Improve%
cohere arxiv nomic ccnews nomic coco nomic codesearch jina gooaq roberta landmark nomic landmark dino random
0.681 0.897 0.838 0.856 0.823 0.738 0.904 0.735 0.907
0.687 0.888 0.838 0.830 0.835 0.765 0.878 0.804 0.900
0.679 0.888 0.840 0.832 0.836 0.767 0.881 0.806 0.900
−0.006 +0.009 −0.000 +0.025 −0.012 −0.026 +0.027 −0.069 +0.007
+0.002 +0.009 −0.002 +0.023 −0.012 −0.028 +0.023 −0.072 +0.007
66 8 0 8 −5 −8 14 −3 −1
R A B IT Q L INEAR C ORRECTOR VS . H AMMING BQ (P ROBE 17)
Table 15: Recall@10 under five quantization configurations. Ham = Hamming BQ, +Rot = with Haar rotation, RaBitQ = per-vector linear corrector. Dataset cohere random gist960 arxiv nomic
Hamming
Ham+Rot
RaBitQ
RaBitQ+Rot
∆(Rot) on Ham
0.440 0.437 0.001 0.524
0.381 0.471 0.361 0.532
0.616 0.610 0.001 0.643
0.495 0.610 0.429 0.629
−0.060 +0.035 +0.360 +0.008
17