HARNESS-LM: A Three-Phase Training Recipe for Harnessing SLMs in Sponsored Search Retrieval Vipul Gupta, Shikhar Mohan, Lakshya Kumar, Pranjal Chitale, Nikit Begwani, Amit Singh, Manik Varma Microsoft AI India
arXiv:2605.23572v1 [cs.IR] 22 May 2026
Abstract
asymmetry also creates the central bottleneck for modern neural retrieval: the online query encoder is invoked in real-time for each query, so model size, inference latency, and serving costs tightly limit what can be deployed. Recent Small Language Model (SLM)-based embedding models have substantially advanced text retrieval. Works such as RepLLaMA [3], LLM2Vec [4], NV-Embed [5], GRIT-LM [6], Qwen3-Embedding [7], KaLM-Embedding-V2 [8], Llama-Embed-Nemotron-8B [9], and EmbeddingGemma [10] demonstrate that decoder-only foundation models can be adapted into strong retrievers, achieving state-ofthe-art results on multilingual benchmarks like MMTEB [11]. These advances suggest that sponsored search can benefit from SLMs’ stronger semantic representations. However, deploying such retrievers in real-time sponsored search is impractical due to their scale (billions of parameters), which imposes substantial GPU requirements and makes it challenging to meet stringent latency constraints (<15 ms) at web scale.1 At the same time, these models provide a clear quality upper bound2 and can act as effective teachers, exploiting greater capacity and richer offline signals, but cannot serve the latency-critical online query path directly. This creates a gap between what performs best offline and what can be served online. A natural solution is to fine-tune a smaller query encoder (deployed online) directly along with a larger document encoder (indexed offline), but this does not fully address the gap. In this asymmetric setup, the online query encoder must retrieve against document embeddings produced by a much stronger offline encoder. Thus, the problem is not only domain adaptation, but also embedding-space compatibility: the compact query encoder must produce representations that are well calibrated to the document space of the teacher retriever. Direct supervised training of the asymmetric architecture may improve task performance, but it couples multiple objectives like domain adaptation, representation transfer, and online efficiency into a single optimization problem, thus making it challenging to get the high-quality model in one-shot. At serving time, document representations can be precomputed offline and stored in an ANN index, whereas the query encoder must be executed online for every incoming query. This induces a fundamental asymmetry in model design: the document encoder can afford to be large and computationally intensive, while the query encoder is constrained by strict latency and cost requirements. In this work, we introduce HARNESS-LM (HLM), a training recipe explicitly designed to exploit this asymmetry. The central philosophy is to decouple offline representation quality from online serving
In the competitive landscape of sponsored search, balancing retrieval quality with production latency is a critical challenge. While large retrieval models based on Small Language Models (SLMs) such as Qwen3-Embedding-4B/8B set strong upper bounds on public benchmarks, their deployment in high-throughput, latency-sensitive environments remains impractical. In this paper, we present Harness-LM (HLM), a three-phase training framework for transferring the capabilities of large-scale retrievers into compact, cost-efficient models. The approach comprises: (1) training a high-performance reference (“teacher”) retriever by fine-tuning a billion-parameterscale SLM; (2) aligning query representations via an 𝐿2 objective to distill knowledge into a sub-600M parameter student encoder; and (3) applying a final contrastive refinement stage to optimize the student for retrieval performance. We also present a comprehensive empirical study of key design choices, including alignment objectives, embedding dimensionality, model scale, architecture, and optimization strategies, to identify configurations that are most effective in production settings. On a real-world Bing Ads evaluation benchmark, HLM recovers over 98% of the reference retriever’s precision across multiple settings, while delivering up to 27× lower online query-encoder latency and 20× higher throughput on NVIDIA A100 GPUs. Online A/B testing on Bing Ads further shows a +1% Revenue, +0.6% Impression and +0.4% Click uplift over current ensemble of retrievers running in production with the deployed 190M parameter model, clearly highlighting the practical efficacy of the HLM recipe in a real world sponsored search setting.
Keywords Dense Retrieval, LLMs, SLMs, Knowledge Distillation, Pruning, Sponsored Search, Unsupervised learning, Contrastive training
1
Introduction
Sponsored search is a key revenue model for search engines, where paid advertisements (Ads) are displayed alongside organic results in response to user queries. Advertisers bid on Ads, and these are ranked using multiple signals including bid value and relevance to the user’s intent. This lets search engines monetize high-intent traffic while delivering useful, contextually relevant Ads to users. Sponsored search systems rely on a high-throughput first-stage retriever to select a small set of advertisements from a very large corpus. This stage directly shapes downstream ranking quality and business metrics while operating under strict latency, throughput, and cost constraints. Dense retrieval with dual encoders [1] is a natural fit for this setting: document embeddings can be precomputed offline and served through Approximate Nearest Neighbor (ANN) [2] search, while only the query encoder runs online. However, this deployment
1 SLMs are “small” relative to frontier Large Language Models (LLMs), but still much
larger than conventional dense retrieval encoders, making online serving costly. 2We use “upper-bound” to denote the strongest retriever trainable within our compute budget; it is not a theoretical bound. 1
HARNESS-LM: Training Pipeline Overview Gupta et al.
Supervised CL data
Phase 1
Upper-bound (SLM query + doc encoders)
Unsupervised text data Frozen query encoder
Doc Corpus Frozen doc encoder
Offline index
Doc embeddings
ANN search
Pruneand-align (Optional)
Phase 2 ℓ₂ alignment
Compact query backbone
Online serving
Query encoder
Aligned query encoder
Supervised CL data
Phase 3
CR with frozen doc encoder
Figure 1: HLM: A three-phase training framework for developing effective and compact SLM retrievers. • We introduce HLM, a three-phase training recipe for obtaining efficient and high-quality SLMs for sponsored search retrieval. • We propose ℓ2 alignment based optimization that result into effective knowledge transfer from teacher to student. • We propose a progressive prune-and-align strategy to produce encoders based on compute-latency trade-offs. • We conduct an extensive study across multiple design dimensions, including embedding size, feature representations, model architecture, training objectives, and optimization strategies, to develop high-performance SLMs suitable for production settings. • We validate HLM through offline evaluation on Bing Ads retrieval dataset and online A/B testing, showing consistent gains over production baselines while reducing online inference cost.
efficiency, enabling high-fidelity document representations without compromising real-time query processing constraints. At a high level, the recipe first asks a quality-first question: how good can an SLM-based retriever become if we relax online deployment constraints? We answer this by training teacher models that may use larger encoders and/or richer input features. These models are not necessarily deployable, but they define the quality target for the rest of the recipe. Subsequent stages aim to transfer this quality into smaller and more deployable query-side models through alignment, distillation, and latency-aware optimizations. The HLM training recipe is illustrated in Fig. 1. Training proceeds in three phases as follows. First, we train a high-fidelity reference retriever (referred to as the teacher) using larger (4B/8B parameter) SLM encoders and/or richer offline-only features. Second, we align a compact query encoder to the teachers’ query embedding space using ℓ2 regression on unsupervised text data. Third, we refine the aligned query encoder with supervised Contrastive Learning (CL) [7], while keeping the teacher document encoder frozen, and refer to it as Contrastive Refinement (CR). This produces an asymmetric retriever in which the expensive document encoder is used offline for index construction, while the compact query encoder runs online. HLM also provides an optional and a practical compression pathway. After alignment, we progressively prune the compact query encoder by reducing transformer layers and FFN dimensions, followed by re-alignment after each pruning step. This successive prune-andalign strategy is inspired by recent structured model-compression recipes such as the cascaded pruning procedure used in Ministral 3 [12] and Minitron [13], but is adapted here to dense retrieval and asymmetric query-document serving. Unlike unstructured sparsity, our pruning removes entire layers and FFN units, yielding dense smaller models with real latency and cost benefits. In practice, pruning yields up to a 6× latency reduction (40.96 ms → 6.8 ms) with only a 1.1% drop in precision (64.3 → 63.1), demonstrating a highly favorable quality–efficiency trade-off. (refer Table 9 for details). We evaluate HLM on Bing Ads sponsored search retrieval benchmarks and validate the resulting models through online A/B testing on Bing Ads live traffic. Across offline and online evaluations, HLM model consistently improve retrieval quality over the current ensemble of models in production, with online deployed model showing significant improvements across all key business metrics: +1% Revenue, +0.6% Impression and +0.4% Click uplift without degrading quality. The main contributions of our work are as follows:
1.1
Background and Related Work
Dense retrieval and CL: Dense retrieval uses dual encoders to map queries and documents to a shared embedding space, enabling offline document indexing and low-latency online query encoding. Foundational work like DPR [1], ANCE [14], and TwinBERT [15] established contrastive training with hard negatives and asymmetric serving as key design principles. SamToNe [16] further showed that same-tower negatives improve dual-encoder training. HLM builds on these ideas but focuses on transferring high-capacity SLM retrievers to compact, production-ready retrievers. SLMs as embedding models: Recent work shows that decoderonly language models can serve as strong embedding backbones through architectural and training modifications [3–6]. Among these, Qwen3-Embedding [7] is particularly relevant as it spans multiple model sizes (0.6B–8B), supports Matryoshka representations, and achieves strong multilingual performance. HLM is a principled approach to specialize such SLM backbones for Ads retrieval while maintaining deployability in a large scale production settings like sponsored search. Compression, pruning, and alignment: Deploying SLM-based retrievers under strict latency constraints motivates compression through distillation and pruning. Ministral 3 [12] uses iterative pruning with continued training to obtain compact models. Kernel-based unsupervised alignment [17] matches representations across encoders without supervised retraining. HLM combines these ideas: we train a high-quality teacher, align a compact query encoder via ℓ2 regression, optionally prune, and refine with supervised contrastive learning against the frozen teacher document encoder. 2
HARNESS-LM: A Training Recipe for Sponsored Search Retrieval
2
2.2
HLM: Training Recipe
Notation: We consider dense retrieval for sponsored search, where the goal is to retrieve the most relevant documents (Ads) from the document corpus D for an incoming query 𝑞. We use a dual-encoder retrieval model with a query encoder 𝑓𝑄 (·) and a document encoder 𝑓𝐷 (·), which map the query and document texts to normalized 𝑑dimensional embedding vectors3 . The relevance score between a query and a document is computed using the inner product: 𝑠 (𝑞,𝑑) = ⟨𝑓𝑄 (𝑞),𝑓𝐷 (𝑑)⟩. Let 𝑓𝑄𝑇 and 𝑓𝐷𝑇 denote the teacher query and document encoders, respectively. Let 𝑓𝑄𝑆 denote the smaller student query encoder obtained after alignment with 𝑓𝑄𝑇 . Next, we describe the first stage of HLM: constructing a high-quality teacher.
2.1
Phase 1: Teacher
Alignment objectives. We explored multiple objectives to transfer the teacher query representation to the smaller query encoder. One class of objectives is score-level or contrastive distillation, inspired by recent embedding model training recipes such as KaLMEmbedding-V2, where a compact embedding model is improved using contrastive distillation with fine-grained teacher signals [8]. Another class of objectives aligns pairwise structures rather than individual embeddings. For example, the loss objective for unsupervised kernel alignment [17] aligns representations by matching kernel matrices induced by teacher and student embeddings. In our setting, we found that a simpler objective works best: direct ℓ2 alignment between teacher and student query embeddings. For an alignment corpus A = [𝑞 1,𝑞 2,···,𝑞 𝑁 ] consisting of 𝑁 pieces of texts, we minimize 𝑁 ∑︁ Lalign (𝑓𝑄𝑆 ) = ∥𝑓𝑄𝑆 (𝑞𝑖 ) − 𝑓𝑄𝑇 (𝑞𝑖 )∥ 22 . (2)
In the first phase of HLM, we train a teacher which serves two purposes. First, it tells us how much quality is available from stronger SLM backbones and richer retrieval features. Second, it provides a target for later stages, where we attempt to recover as much of this quality as possible in a compact model. We obtain teacher models by relaxing one or more production constraints. One axis is model scale: we can increase the number of parameters in the query and document encoders, for example, by using a larger SLM backbone (like Qwen3-Embedding 4B/8B). Since the document encoder is used offline for indexing, a larger document encoder is often easier to justify than a larger query encoder. However, a large query encoder may still be useful for estimating the maximum quality achievable before compression. A second axis is feature richness. Beyond raw query and document text (deployable features), teacher models may use additional context, such as offline GPT-generated expansions. These enrich semantics and improve offline retrieval, but are often unavailable or too costly at serving time. We term them oracle features; models relying on them are treated as oracle or teacher models, not deployable models. An example of oracle features for the query “change from pdf into word free” is provided in Appendix A. We train our teacher using the Qwen3-Embedding contrastive objective [7], which is a modified InfoNCE loss [18]. Unlike vanilla InfoNCE, it enriches the denominator with mined hard negatives and same-tower query–query/document–document negatives [16], while masking likely false negatives from the in-batch pool. LQwenCL (𝑖) =−log
exp(𝑠 (𝑞𝑖 ,𝑑𝑖+ )/𝜏) Í . exp(𝑠 (𝑞𝑖 ,𝑑𝑖+ )/𝜏) + 𝑢 ∈ N𝑖 exp(𝑠𝑢 /𝜏)
Phase 2: Alignment
The teacher obtained in phase-1 provides a high-quality target, but it is not directly suitable for online serving. In particular, the best teachers may use a large query encoder, large document encoder, or training-time features that are not available in the online path. We therefore next study whether a smaller student query encoder can be made compatible with the teacher embedding space. The goal of alignment is to train the student query encoder: 𝑓𝑄𝑆 , such that it can replace the teacher query encoder:𝑓𝑄𝑇 from phase-1 at serving time while still retrieving against the document embeddings produced by the encoder: 𝑓𝐷𝑇 , from phase-1. This gives an asymmetric retriever: the expensive document encoder directly coming from the teacher is used offline for index construction, while the compact query encoder is used online.
𝑖=1
The teacher query encoder is frozen during this stage, and only the student query encoder is updated. After alignment, the student query embedding is scored against the frozen teacher document embedding as 𝑠 align (𝑞,𝑑) = ⟨𝑓𝑄𝑆 (𝑞),𝑓𝐷𝑇 (𝑑)⟩. This objective is attractive for two reasons. First, it directly enforces compatibility between the embedding spaces of the student and teacher query encoders, which is exactly what is required for asymmetric retrieval when the document encoder is frozen. Second, it avoids requiring labeled query–document pairs during alignment; any large corpus of query-like text can be used to expose the student to the geometry of the teacher query space. As demonstrated in Sec. 3, when we use Qwen3-Embedding-0.6B as the student query encoder and Qwen3-Embedding-4B as the teacher query encoder, the smaller encoder is able to recover the retrieval quality of the teacher encoder using ℓ2 alignment when combined with the frozen teacher document encoder. This result is central to the HLM recipe: it shows that the quality of a large query encoder can be transferred into a smaller encoder without rebuilding the document index with the student model. In other words, alignment lets us retain the stronger offline document encoder while replacing only the online query encoder.
(1)
Here, N𝑖 contains the valid negatives after false-negative masking: in-batch query–document negatives, mined hard negatives, and same-tower query–query/document–document negatives. The score 𝑠𝑢 denotes the corresponding similarity term, and 𝜏 is the temperature (refer [7] for details). The resulting teacher defines the target for the rest of the HLM recipe. Later phases aim to retain this quality while removing the challenges that make the teacher impractical for production: large latency of online query-side inference, dependence on unavailable GPT generated features, or excessive serving cost.
4 The layer importance score 𝐼
layer (ℓ ) measures how much layer ℓ transforms the representation magnitude. Due to residual connections, ℎ ℓout = ℎ ℓin + 𝐹 ℓ (ℎ ℓin ) where 𝐹 ℓ is the layer’s transformation. A ratio near 1 indicates the layer adds little to the representation, making it a candidate for removal.
3 Throughout the paper, we assume that all encoder embeddings have unit ℓ norm. 2
3
Gupta et al.
Algorithm 1 Structured Pruning for Student Query Encoder
Algorithm 2 Progressive prune-and-align pipeline
Input: Query encoder 𝑓𝑄 with 𝐿 transformer layers, calibration corpus C, target layer count 𝐾𝐿 , target FFN dimension 𝐾𝐹 Output: Pruned query encoder 𝑓e𝑄 // Stage 1: Layer Pruning (Depth Reduction) 1: Run calibration data through 𝑓𝑄 and collect hidden states at each layer boundary. 2: For each layer ℓ ∈ {1,...,𝐿}, compute the importance score4 : 𝐼 layer (ℓ) =E𝑥∼C ∥ℎ ℓout (𝑥)∥ 2 /∥ℎ ℓin (𝑥)∥ 2
Input: Aligned student 𝑓𝑄𝑆 , teacher query encoder 𝑓𝑄𝑇 , frozen teacher document encoder 𝑓𝐷𝑇 , alignment corpus A, pruning targets {(𝐾𝐿(𝑟 ) ,𝐾𝐹(𝑟 ) )}𝑟𝑅=1 Output: Compact query encoder 𝑓b𝑄𝑆 1: Set 𝑔𝑄 ← 𝑓𝑄𝑆 . 2: for each pruning target (𝐾𝐿 ,𝐾𝐹 ) do Prune 𝑔𝑄 to 𝐾𝐿 layers and FFN dimension 𝐾𝐹 using 3: Algorithm 1. Re-align 𝑔𝑄 to 𝑓𝑄𝑇 on A by minimizing ℓ2 loss L (𝑔𝑄 ) from 4: Eq. 2. 5: end for 𝑆 6: return 𝑓b 𝑄 ←𝑔𝑄
where ℎ ℓin (𝑥),ℎ ℓout (𝑥) ∈ R𝑑 are the hidden states entering and exiting layer ℓ, respectively. 3: Retain the top-𝐾𝐿 layers ranked by 𝐼 layer (ℓ), preserving their original order. // Stage 2: FFN Pruning (Width Reduction) 4: For each retained layer ℓ, score FFN hidden unit 𝑗 ∈ {1,...,𝑑 ffn }: 𝑔 𝐼 ffn (ℓ,𝑗) =E𝑥∼C 𝜎 (𝑊ℓ ℎ ℓ (𝑥)) 𝑗 · (𝑊ℓ𝑢 ℎ ℓ (𝑥)) 𝑗
Algorithm 3 HLM training recipe Input: Supervised retrieval data Dsup , alignment corpus A, large teacher backbone, compact student backbone, optional pruning targets Output: Deployable asymmetric retriever ( 𝑓b𝑄𝑆 ,𝑓𝐷𝑇 ) // Phase 1: Train the teacher 1: Train a teacher (𝑓𝑄𝑇 , 𝑓𝐷𝑇 ) using large SLM backbone and/or richer offline features with supervised CL loss (Eq. 1). // Phase 2: Align compact query encoder 2: Initialize compact query encoder 𝑓𝑄𝑆 from a pretrained SLM checkpoint. 3: Align 𝑓𝑄𝑆 to the teacher query encoder 𝑓𝑄𝑇 on A using ℓ2 loss Lalign (𝑓𝑄𝑆 ) (Eq. 2) 4: Optionally compress 𝑓𝑄𝑆 using progressive prune-and-align (Algorithm 2). // Phase 3: Contrastive refinement 5: Freeze 𝑓𝐷𝑇 and continue supervised contrastive refinement on 𝑓𝑄𝑆 using Dsup with CL loss. 6: Set 𝑓b𝑆 ← 𝑓 𝑆 .
𝑔 where 𝑊ℓ ,𝑊ℓ𝑢 ∈ R𝑑ffn ×𝑑 are the gate and up projection matrices
of the SwiGLU FFN, and 𝜎 (·) = SiLU(·) is the Sigmoid Linear Unit activation. 5: For each layer, retain the top-𝐾𝐹 FFN units by 𝐼 ffn (ℓ,𝑗). 𝑔 6: Prune corresponding rows from 𝑊ℓ ,𝑊ℓ𝑢 and columns from 𝑑 𝑑 ×𝑑 𝑊ℓ ∈ R ffn (down projection). 7: return Dense pruned model 𝑓e 𝑄 with 𝐾𝐿 layers and FFN dimension 𝐾𝐹 . 2.2.1 Alignment under pruning. In addition to aligning a smaller encoder to a larger encoder, we use pruning to understand how far the query encoder can be compressed. For pruning, we adopt structured pruning of transformer layers and the feedforward layer within the transformer layers (described in Algorithm 1). We adopt successive pruning in the same spirit as the cascade pruning recipe used in Ministral 3 [12]: rather than pruning directly to the smallest model in one step, we prune successively and realign after each pruning stage. In our case, we start from the aligned Qwen3-Embedding-0.6B model (which has 28 transformer layers) and progressively prune the model to smaller variants, including 14-layer, 7-layer, 4-layer, and 2-layer models, while also reducing the Feedforward Neural Network (FFN) dimension. Cascade distillation avoids the knowledge shock of a direct leap by using intermediate models as stepping stones, ensuring the student inherits more stable, pre-refined representations. This allows the "prune-align-repeat" pipeline to ensure that pruning for each smaller model starts from a good aligned checkpoint rather than from a heavily damaged representation [12, 13]. The progressive pruning technique is described in Algorithm 2. This progressive prune-and-align strategy gives us a practical way to trace the quality–latency frontier. Larger capacity student models retain more of the teacher quality, while smaller models reduce inference costs (like GPU hardware) and latency. We observed that after successive pruning and re-alignment, the 4-layer pruned encoder is suitable to run on CPUs or low-end GPUs as per the online serving latency requirements at Bing Ads, substantially reducing inference cost compared with GPU-served SLM encoders. While successive pruning provides a systematic pathway to explore progressively
𝑄
𝑄
𝑆 𝑇 7: return ( 𝑓b 𝑄 ,𝑓𝐷 ).
smaller architectures, it is not a mandatory component of the alignment pipeline. It can be optionally employed to characterize the quality–latency trade-off frontier, enabling selection of a model variant that best meets target performance and deployment constraints.
2.3
Phase 3: Contrastive Refinement
The alignment phase makes the compact query encoder compatible with the teacher embedding space. However, alignment alone is not directly optimized for the downstream retrieval objective. It teaches the student query encoder to mimic the teacher query representation, but it does not explicitly train the student to separate positive documents from hard negatives under the final asymmetric retrieval setup. We therefore add a final stage of refinement using contrastive loss function and refer to it as Contrastive Refinement (CR). Starting from the aligned student query encoder 𝑓𝑄𝑆 , we freeze the teacher document encoder 𝑓𝐷𝑇 and continue training only the student 4
HARNESS-LM: A Training Recipe for Sponsored Search Retrieval
query encoder using supervised query–document pairs. We optimize the Qwen3 CL loss from Eq. 1 for CR phase. The document encoder remains frozen throughout this stage, so the online query encoder is adapted to the fixed document representation space. The HLM training recipe is illustrated in Fig. 1 and described in Algorithm 3. Empirically (as noted in Sec. 3), this final contrastive stage improves precision by a few points over the aligned checkpoint. This is because the aligned encoder 𝑓𝑄𝑆 provides a good optimization starting point by already being in the teacher-compatible embedding space. Then, the contrastive objective can focus on local refinements: increasing the margin between positives and hard negatives, correcting task-specific errors, and improving top-𝐾 precision. Additionally, we also note in Sec. 3 that the three-stage HLM recipe significantly outperforms one-shot asymmetric supervised CL model with the smaller query and larger document encoders. We hypothesize that this happens for the following reasons. First, HLM separates three coupled objectives: learning a strong document space, transferring the compact query encoder into that space, and then adapting it for sponsored-search retrieval. In contrast, one-shot asymmetric model expect the compact query encoder to learn both task discrimination and cross-encoder compatibility from only pairwise positive/negative labels; this supervision from sparse information is better used after the student has absorbed task-relevant structure from the larger model through alignment. Second, HLM prevents the document space from being constraint by the capacity-limited query encoder. When both encoders are trained together, the larger document encoder can co-adapt to representations that are convenient for the smaller query encoder, rather than striving for the high quality document representations that it can possibly learn. By freezing the teacher document encoder, HLM keeps the offline index anchored in the best available document space and forces the compact query encoder to become compatible with it.
3
We evaluate HLM on an internal Bing Ads retrieval benchmark sampled from real sponsored search data. The evaluation set contains 230K user queries from 160+ countries and 50+ languages, while the retrieval corpus contains 47 million Ads. Each model encodes queries and Ads into dense vectors; ad embeddings are stored in a DiskANN vector database [2], and the top-100 Ads are selected by inner-product similarity with the query embedding. To measure retrieval quality, we score the retrieved query–ad pairs with a highquality relevance model and report Precision@100 (P@100), defined as the fraction of top-100 retrieved Ads predicted to be relevant. We use the Qwen3 family of embedding models [7] as the backbone for all experiments for the following reasons: 1) it spans multiple capacities: 0.6B, 4B, and 8B, 2) achieves strong performance on public embedding benchmarks such as MMTEB, and 3) supports Matryoshka Representation Learning (MRL) [20], enabling flexible performance-capacity tradeoffs through embedding truncation. These properties allow us to study both the quality gains from larger encoders and the constraints of compact online query encoders within a single model family.
3.1
Teacher
We first establish quality upper bounds by training teachers using symmetric dual encoders (that is, both query and document encoders have the same architecture but no shared parameters) and comparing their performance across different backbone sizes. To understand what contributes to the upper bound, we run controlled ablations over several key design choices: model parameters, embedding dimension (𝑑), optimization function (traditional InfoNCE vs. QwenInfoNCE loss), prompt in query input (that is, with vs. without prompts), feature set (deployable vs. oracle input features), number of mined negatives, and the number of encoders (shared common encoder vs. dual-encoder architecture for query and documents). Model Parameters: In Table 1, we study the effect of model scale by training symmetric retrievers with Qwen3-Embedding backbones of size 0.6B, 4B, and 8B for both query and document encoders. We adopt two different fine-tuning settings: Full Fine-Tuning (FFT) and Low-Rank Adaptation (LoRA) [21]. For the 4B backbone, increasing the LoRA rank from 16 to 128 improves P@100 from 60.2 to 62.4, as higher rank enables a larger effective parameter budget during fine-tuning. Interestingly, FFT underperforms LoRA (56.4 vs 62.4) for Qwen-4B, which we attribute to two factors: (i) overfitting due to the large number of trainable parameters relative to task complexity, and (ii) reduced batch size (32→8) due to memory constraints, leading to fewer in-batch negatives during training which can degrade CL performance [22]. To further increase capacity, we scale both the backbone and LoRA rank, training an 8B–8B model with rank 256, which achieves the best performance of 64.8 P@1006 . The best performing Qwen-0.6B0.6B model is 4-6 points worse than its 4B/8B counterparts. Overall, these experiments establish strong upper bounds at two operating points: 4B–4B (62.4 P@100) and 8B–8B (64.8 P@100). Embedding dimension: In Table 2, we study how much of the teacher retrieval quality depends on the embedding dimension 𝑑.
Experiments & Results
Experimental Setup: Our supervised training data consists of 250M query–document pairs from Bing Ads (used in Phases 1 and 3). For phase-2, we use 2B query texts as the dataset for ℓ2 alignment (note that this is unlabeled data, and hence, easier to curate). All our datasets are global, multilingual and obtained from different query deciles5 . Unless stated otherwise, we use an embedding dimension of 𝑑 =128 and one hard negative per query–document pair for contrastive learning. Hard negatives are mined using a highquality relevance model, where documents with scores in (0.2,0.5] are treated as hard negatives and those with scores > 0.5 as positives. We train phase-1 (teacher) model on 16 NVIDIA A100 80GB GPUs, and Phases 2 and 3 on 16 NVIDIA A100 40GB GPUs. Training uses a linearly warmed-up learning rate from zero over the first 10% of data, followed by linear decay, and the peak learning rate is selected via hyperparameter search. Optimization uses Adam [19] and batch sizes are chosen to maximize GPU utilization. Unless stated otherwise, we prepend the following prompt to queries during phase-1: "Given a web search query, retrieve relevant passages that answer the query.\n {user query:}".
6We limit our exploration of upper bounds to a small set of FFT and LoRA variants,
5 Queries are classified into head, torso and tail deciles based on their Search Results
page views
as Phase 1 training is compute-intensive and constrained by GPU availability. 5
Gupta et al.
Table 1: Teacher retrieval performance for symmetric Qwen3 retrievers when varying parameters and finetuning strategy. Model
Setting
Qwen3-0.6B–0.6B Qwen3-0.6B–0.6B Qwen3-0.6B–0.6B
FFT LoRA (r=64) LoRA (r=128)
58.1 47.6 48.2
Qwen3-4B–4B Qwen3-4B–4B Qwen3-4B–4B Qwen3-4B–4B
LoRA (r=16) LoRA (r=32) LoRA (r=128) FFT
60.2 61.5 62.4 56.4
Qwen3-8B–8B
LoRA (r=256)
64.8
Table 3: Ablations on prompt, features, and loss.
P@100
Setting
Qwen-4B–4B Qwen-4B–4B
Zero-shot Zero-shot
128 2560
36.1 49.3
Qwen-8B–8B Qwen-8B–8B Qwen-8B–8B
Zero-shot Zero-shot Zero-shot
128 2048 4096
38.5 49.5 50.0
Qwen-4B–4B Qwen-4B–4B
LoRA (r=128) LoRA (r=128)
128 2560
62.4 66.8
Qwen-8B–8B Qwen-8B–8B Qwen-8B–8B
LoRA (r=256) LoRA (r=256) LoRA (r=256)
128 2048 4096
64.8 67.7 67.8
Setting
P@100
Prompt in query
Zero-shot, no prompt Zero-shot, with prompt FFT, no prompt FFT, with prompt
29.5 31.9 55.6 58.1
Feature set
Deployable features Oracle features
58.1 63.7
Loss function
Vanilla InfoNCE Qwen3 objective
45.1 58.1
reduces the effective batch size (32→12), leading to fewer in-batch negatives and a less favorable training regime [22]. For the next set of ablations, we fix the backbone to Qwen3-0.6B– 0.6B to reduce computational cost. Table 3 summarizes results across three axes: prompt usage, feature richness, and loss function. Prompt in query: We evaluate the impact of adding a retrieval prompt to the query. Such prompts can help the model better interpret the input as a retrieval task, but they also increase sequence length and hence online inference cost. Prompting consistently improves retrieval quality in both zero-shot (+2.4) and fine-tuned (+2.5) settings, suggesting that prompts act as an effective inductive bias, though at the cost of increased inference overhead. Oracle features: In addition to scaling model capacity, we study an orthogonal axis, feature richness. Specifically, we consider the oracle teacher models that augment raw query and document text with additional offline-generated context (e.g., GPT-based query/document expansions) that are not always available in the online serving path (see Appendix A for an example). Oracle features yield a +5.6 P@100 gain, showing that richer context can provide a strong teacher even with a small backbone—and that improvements can come from better features, not just larger models. In Sec. 3.2, we explain distilling these benefits into a production-feasible student. Loss function: We also study the impact of the contrastive loss, where negative construction plays a critical role. We compare vanilla InfoNCE [18], which relies on in-batch negatives, with the Qwen3 contrastive objective [7], which augments training with same-tower negatives [16], in-batch negatives, and hard-negatives along with masking false negatives. Using Qwen3 objective leads to a substantial improvement (+13.0 P@100), highlighting the importance of richer negative construction. Thus, we adopt it as the default CL objective in HLM, since it provides a stronger and more targeted training signal for retrieval than vanilla InfoNCE. Overall, our teacher analysis shows that larger backbones, richer features, and improved contrastive objectives all contribute to retrieval quality. In the 𝑑=128 setting,7 the best teachers achieve 64.8 P@100 (Qwen3-8B), 62.4 (Qwen3-4B), and 63.7 (Qwen3-0.6B Oracle), which we use for alignment and compression, as described next.
Table 2: Effect of embedding dimension. Larger dimensions yield bigger gains for zero-shot models, while improvements saturate for fine-tuned models. Model
Ablation
Dim. (𝑑) P@100
Smaller embeddings reduce index storage and retrieval cost, but may also limit the capacity of the retrieval space. Instead of retraining models for different 𝑑, we leverage MRL to obtain lower-dimensional embeddings via truncation. For reference, the maximum embedding dimensions for Qwen3-4B and Qwen3-8B are 2560 and 4096, respectively. We also consider a zero-shot setting, where the open-source model is used without any fine-tuning on proprietary data. We observe two clear trends. First, the gap between small and large𝑑 shrinks significantly after fine-tuning (e.g., +13.2 for zero-shot versus +4.4 for LoRA in 4B), suggesting that task-specific adaptation compensates for reduced embedding capacity. Second, for finetuned models, performance largely saturates beyond 𝑑 =2048, with negligible improvements up to 𝑑 = 4096, indicating that moderate embedding sizes are sufficient for this task. Number of hard negatives: We study the impact of increasing the number of hard negatives (HN) per query for a Qwen3-4B–4B model fine-tuned with LoRA (rank=128). Moving from no hard negatives to one yields a large gain (59.2 → 62.4 P@100), highlighting the importance of informative negatives for contrastive learning. However, increasing from one to five hard negatives provides negligible improvement (62.4 → 62.5). We attribute this to two factors: (i) for strong pretrained models, a small number of informative hard negatives is often sufficient, and (ii) adding more hard negatives
3.2
Query Alignment
The teacher provides a high-quality embedding space, but its query encoder is too expensive for online serving (see Table 9 for details). 7We use 𝑑=128 for deployment due to its favorable performance–latency trade-off.
6
HARNESS-LM: A Training Recipe for Sponsored Search Retrieval
Table 4: Effect of teacher quality on query encoder alignment with student backbone as Qwen3-0.6B
Qwen3-4B zero-shot Qwen3-4B LoRA (r=32) Qwen3-4B LoRA (r=128) Qwen3-8B LoRA (r=256)
ℓ2 loss, align(fQS )
Teacher
Teacher Student Gap (P@100) (P@100) 36.1 61.5 62.4 64.8
36.1 61.2 61.9 62.5
0.0 0.3 0.5 2.3
10−1
0.0
0.5 1.0 1.5 # Training points (in billions)
2.0
Figure 2: Alignment loss (Eq. 2) as a function of training points for a randomly initialized model versus a pretrained model, showing that pretraining yields significantly faster convergence and a lower final loss.
Table 5: Ablations for query alignment with fixed teacher Qwen3-4B LoRA (rank=128). Setting
Random initialization Pretrained
100
Teacher
Student
Gap
Without prompt With prompt
62.4 62.4
61.9 61.9
0.5 0.5
Table 6: Pruning of the aligned Qwen3-0.6B query encoder.
Ads queries Public text + Ads queries
62.4 62.4
61.9 61.8
0.5 0.6
Student model
Pretrained Qwen3-0.6B Random initialization
62.4 62.4
61.9 59.7
0.5 2.7
KL divergence [8] Kernel matrix alignment [17]
62.4 62.4
56.4 58.7
5.8 3.7
Next, we study whether a compact query encoder can be aligned to the teacher query encoder while retrieving against the frozen teacher document encoder. Alignment to different upper bounds: We first study how the choice of teacher affects alignment quality. Table 4 shows both the teacher performance and the aligned student performance, allowing us to measure the transfer gap. Stronger teachers produce stronger students, but the gap also grows (0.0 → 2.3), suggesting that transferring a more powerful retrieval space into a compact student becomes progressively harder. Alignment to oracle teachers: We align the student to an oracle teacher using richer, non-deployable features while restricting student inputs to deployable features only. The oracle teacher achieves 63.7 P@100; the aligned student reaches 61.7, leaving a gap of 2.0— comparable to the 8B teacher (gap of 2.3). This indicates that stronger teachers yield larger transfer gaps, and that alignment can distill some, but not all, benefit of richer offline context. For the remaining ablations, we fix the teacher to Qwen3-4B LoRA (rank=128), with teacher P@100 of 62.4, and the student to Qwen3-0.6B. Table 5 summarizes the results. Prompt in Query: Unlike supervised contrastive training, prompt usage provides no benefit during alignment. We therefore omit the prompt from the student model’s input but keep it in the teacher model during the alignment phase in HLM, which simplifies serving and avoids additional online latency. Public data: Alignment on Ads queries alone preserves taskspecific retrieval quality, but substantially degrades general embedding quality. The aligned model trained only on Ads queries achieves 23.0 NDCG@10 on MMTEB, far below the 4B–4B upper bound of 42.1. Adding public text from [23], covering over 200 languages, in a 50-50
Layers FFN dim P@100
Aligned Qwen3-0.6B Pruned + aligned Pruned + aligned Pruned + aligned Pruned + aligned
28 14 7 4 2
3072 2304 1536 1024 1024
61.9 61.6 61.0 60.4 56.5
Pruned directly + aligned
4
1024
50.2
mixture with Ads queries leaves Ads performance nearly unchanged (61.8 vs. 61.9), while improving MMTEB from 23.0 to 39.8. This shows that broader alignment data preserves general embedding quality without sacrificing Ads-task performance. Pretraining: Pretraining is critical for both alignment quality and data efficiency. Compared to a pretrained (zero-shot) student, randomly initialized student performs worse (59.7 vs. 61.9 P@100) and requires substantially more training data (2.2B vs. 0.4B), as shown in Fig. 2. This indicates that the pretrained checkpoint starts from a more favorable optimization region for alignment. Alignment loss function: One of the key contribution of our paper is the proposed ℓ2 alignment objective in Eq. 2. Although simple to implement and fully unsupervised, it outperforms more elaborate alternatives such as Kullback Leibler (KL) divergence based contrastive distillation with teacher soft signals [8] and kernel-matrix alignment of teacher and student similarities [17], as also shown in Table 5. We believe this is because direct embedding-level supervision provides a denser and more faithful transfer signal than matching softened scores or pairwise structure alone, while naturally preserving compatibility with the frozen document encoder. These loss functions are described in detail in Appendix B.
3.3
Progressive pruning
We next study whether the aligned Qwen3-0.6B query encoder can be compressed for a better performance-latency tradeoff. We adopt structured pruning of self-attention and feedforward layers, followed by re-alignment to the frozen Qwen3-4B LoRA (rank=128) teacher. Table 6 shows the impact of progressing pruning and alignment. We observe a smooth degradation as the model is progressively 7
Gupta et al.
Table 7: Effect of Contrastive Refinement (CR) on P@100. Teacher (frozen)
Student
Qwen3-4B LoRA (r=128) Qwen3-4B LoRA (r=128) Qwen3-8B LoRA (r=256)
Aligned Qwen3-0.6B Pruned & Aligned 4L Aligned Qwen3-0.6B
Table 8: HLM vs. asymmetric fine-tuning.
Before CR After CR Gain 61.9
64.2
+2.3
60.4
63.1
+2.7
62.5
64.3
+1.8
Query
Naive CL HLM
Qwen3-0.6B (FFT) Qwen3-4B (FFT) Qwen3-0.6B (FFT) Qwen3-4B (LoRA, r=128)
53.4 54.6
64.2
Qwen3-0.6B (FFT) Qwen3-8B (LoRA, r=256)
56.2
64.3
similarity between matched pairs (cos(𝑞,𝑑 + ) = 0.03), where 𝑑 + is the right document for query 𝑞. After 𝐿2 alignment in phase-2 (Fig. 3b), query embeddings shift toward their corresponding documents, raising average similarity to 0.71. Contrastive refinement in phase-3 (Fig. 3c) further tightens matched pairs while separating unrelated ones, achieving cos(𝑞,𝑑 + ) =0.78, averaged across all (𝑞,𝑑 + ) pairs. One-shot asymmetric fine-tuning: Next, we compare HLM with the standard alternative of directly fine-tuning an asymmetric dual-encoder retriever with a small query and a larger document encoder. As shown in Table 8, HLM substantially outperforms one-shot asymmetric fine-tuning, with gains of 8.1-9.6 P@100 over the direct FFT baselines. This confirms the benefit of the three-phase recipe: direct asymmetric training must learn cross-tower compatibility and task discrimination jointly, whereas HLM first transfers the compact query encoder into the teacher-compatible space and then applies task-specific refinement. Latency vs. Performance: Table 9 presents a detailed comparison of the latency–quality trade-off across query encoders obtained at different stages of the HLM recipe. As expected, larger models such as Qwen3-8B and Qwen3-4B achieve strong retrieval performance but incur significantly higher latency (186.1 ms and 117.8 ms, respectively). In contrast, the aligned and contrastively refined Qwen3-0.6B model matches the 8B model’s performance (64.2 vs. 64.8 P@100) while reducing latency by more than 4× and increasing throughput from 338 to 1,461 queries/sec. Further compression via structured pruning yields a 4-layer model with only 0.19B parameters that achieves comparable performance (63.1 P@100) at just 6.8 ms latency and over 6,800 queries/sec throughput. These results highlight that HLM effectively preserves retrieval quality while enabling substantial gains in efficiency through alignment and progressive compression.
pruned, with only modest loss up to 4 layers (61.9 → 60.4), suggesting that much of the retrieval behavior can be preserved in substantially smaller students for our task. In contrast, the 2-layer model shows a sharper drop (56.5), indicating that capacity becomes the main bottleneck at extreme compression. A key observation is that pruning must be done progressively. When we prune the zero-shot Qwen3-Embedding-0.6B model directly to 4 layers and then align, performance collapses to 50.2 even after training on 2B points. This mirrors our earlier pretraining result from Fig. 2: alignment works best when the student starts from a good pretrained checkpoint that remains close to the original model, and becomes much harder when optimization begins from a severely compressed or poorly initialized model.
3.4
Document
Contrastive refinement
We next evaluate the final phase of HLM, i.e, Contrastive Refinement (CR). Starting from the aligned student query encoder from phase2, we freeze the teacher document encoder and continue training only the query encoder with the Qwen3 contrastive loss (Eq. 1). We do not prepend a retrieval prompt during this phase as the aligned model did not use the query prompt. We consider two teacher document towers: Qwen3-4B LoRA (rank=128), with P@100 of 62.4, and Qwen3-8B LoRA (rank=256), with P@100 of 64.8. Table 7 shows the impact of CR on the aligned models. CR consistently improves retrieval quality over the aligned checkpoints. With the 4B teacher, refinement improves the aligned Qwen3-0.6B student from 61.9 to 64.2, and even the progressively pruned 4-layer student from 60.4 to 63.1. With the stronger 8B teacher, refinement improves the aligned Qwen3-0.6B student from 62.5 to 64.3. These results support our hypothesis that alignment provides a strong initialization in the teacher-compatible embedding space, after which contrastive training can focus on local ranking refinements like increasing margins between positives and hard negatives and improving compatibility with the frozen document index. An interesting finding is that the gain from CR is larger for the pruned 4-layer student than for the full 0.6B student (+2.7 vs. +2.3 with the 4B teacher). This suggests that although pruning introduces a larger gap after alignment, much of that gap is recoverable through task-specific refinement. In other words, alignment transfers compatibility, while CR recovers discrimination. Fig 3 shows a Multi-Dimensional Scaling (MDS) projection of cosine distances at different phases in HLM. In the zero-shot setting (Fig. 3a), the Qwen3-0.6B query encoder produces embeddings that lie farther away from the document embeddings, yielding near-zero
3.5
Online A/B Testing
To evaluate the effectiveness of the HLM training recipe in a realworld setting, we conducted large-scale online A/B experiments on live traffic from Bing sponsored search. We choose the pruned 4-layer model with 190 million parameters for deployment due to its good latency-quality trade-off. Table 9 for more details on the latency–quality trade-offs and architectural configurations. The HLM-based retriever was deployed in production and benchmarked against an ensemble of state-of-the-art retrieval approaches already running in production. We report key business and quality metrics, including revenue, Ad clicks, Ad impressions, Quick Back Rate (QBR), and Ad defect rate. QBR measures the fraction of ad clicks that result in users quickly returning back from the clicked Ad, serving as a proxy for poor user satisfaction. Ad Defect rate, computed using high-quality offline relevance models, captures the proportion of irrelevant Ads shown. 8
HARNESS-LM: A Training Recipe for Sponsored Search Retrieval
d +)=0.03 cos(q,cos(q, d +)=0.03 cos(q, d +)=0.03
d +)=0.71 cos(q,cos(q, d +)=0.71 cos(q, d +)=0.71
(a) Zero-shot (a) Zero-shot (a) Zero-shot (a) Zero-Shot
d +)=0.78 cos(q,cos(q, d +)=0.78 cos(q, d +)=0.78
(b)(b)Phase 2: Aligned (b) Phase 2:Phase Aligned (b) 2: Aligned Phase 2: Aligned
(c) Phase (c) Phase CR3: CR3: CR (c)(c)3:Phase Phase 3: CR
Figure 3: 2-D projection of query (stars) and document (circles) embeddings across HLM training phases. Table 9: Latency–quality trade-offs of HLM query encoders. Latency and throughput are measured on a Nvidia A100 GPU. Query Encoder
Hidden dim
FFN dim
4096 2560 1024 1024
12288 9728 3072 1024
Qwen3-8B (CL) Qwen3-4B (CL) Qwen3-0.6B (Align + CR) Pruned 4L layers (Align + CR)
Query Key-Value Number of Latency Throughput HLM Heads Heads Parameters (ms) (queries/sec) P@100 32 32 16 16
Table 10: Online A/B results of HLM on sponsored search traffic (relative change with respect to production) Revenue
Clicks
Impressions
QBR
Ad Defect
+1.0%
+0.4%
+0.38%
∼0%
∼0%
7.57B 4.02B 0.60B 0.19B
186.14 117.81 40.96 6.80
338 530 1,461 6,808
64.8 62.4 64.3 63.1
to stronger teachers and extending it to broader embedding-based tasks beyond retrieval remain important directions for future work.
Acknowledgments We thank Anshul Mittal for his help with training-data curation and index creation. We also thank Rakshith V. for his help with curating the public dataset for alignment and for evaluating the HLM models on MMTEB.
From Table 10, we see that HLM achieves consistent gains across all primary engagement and revenue metrics, while maintaining QBR and Ad defect rates at parity with the baseline. These results demonstrate that the HLM recipe enables deployable SLMs that improve both user engagement and monetization without compromising quality, all within strict production latency constraints.
4
8 8 8 8
Conclusion
We present HARNESS-LM, a three-phase training recipe that decouples representation transfer from task-specific optimization. Extensive ablations demonstrate that this decoupling is crucial, with direct asymmetric fine-tuning underperforming by over 10 absolute P@100 points. Our iterative prune-and-align procedure, followed by contrastive refinement, recovers most of the quality lost during compression (P@100 64.3 → 63.1 for a 4-layer model). The resulting 190M parameter model achieves 27× lower latency while incurring only a 1.7 P@100 drop relative to an 8B teacher. Large-scale online A/B experiments on real-world Bing Ads sponsored search traffic further validate its effectiveness, yielding consistent gains of +1% in revenue, +0.6% in impression, and +0.4% in clicks over the current production ensemble. Beyond empirical gains, our study provides practical insights across key design dimensions for effectively harnessing SLMs in real-world retrieval systems. Scaling HLM 9
Gupta et al.
References
[19] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014. [20] Aditya Kusupati, Gantavya Bhatt, Aniket Rege, Matthew Wallingford, Aditya Sinha, Vivek Ramanujan, William Howard-Snyder, Kaifeng Chen, Sham Kakade, Prateek Jain, and Ali Farhadi. Matryoshka representation learning. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems, volume 35, pages 30233–30249. Curran Associates, Inc., 2022. URL https://proceedings.neurips.cc/paper_files/paper/ 2022/file/c32319f4868da7613d78af9993100e42-Paper-Conference.pdf. [21] Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Liang Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. Iclr, 1(2):3, 2022. [22] Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International conference on machine learning, pages 1597–1607. PmLR, 2020. [23] Stephan Oepen et al. Hplt 3.0: Very large-scale multilingual resources for llm and mt. mono- and bi-lingual data, multilingual evaluation, and pre-trained models, 2025. URL https://arxiv.org/abs/2511.01066. [24] Simon Kornblith, Mohammad Norouzi, Honglak Lee, and Geoffrey Hinton. Similarity of neural network representations revisited. In Kamalika Chaudhuri and Ruslan Salakhutdinov, editors, Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 3519–3529. PMLR, 09–15 Jun 2019. URL https://proceedings.mlr.press/v97/kornblith19a.html.
[1] Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. Dense passage retrieval for open-domain question answering. In Proceedings of the 2020 conference on empirical methods in natural language processing (EMNLP), pages 6769–6781, 2020. [2] Suhas Jayaram Subramanya, Fnu Devvrit, Harsha Vardhan Simhadri, Ravishankar Krishnawamy, and Rohan Kadekodi. Diskann: Fast accurate billion-point nearest neighbor search on a single node. Advances in neural information processing Systems, 32, 2019. [3] Xueguang Ma, Liang Wang, Nan Yang, Furu Wei, and Jimmy Lin. Fine-tuning llama for multi-stage text retrieval. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 2421–2425, 2024. [4] Parishad BehnamGhader, Vaibhav Adlakha, Marius Mosbach, Dzmitry Bahdanau, Nicolas Chapados, and Siva Reddy. Llm2vec: Large language models are secretly powerful text encoders, 2024. URL https://arxiv.org/abs/2404.05961. Accepted to COLM 2024. [5] Chankyu Lee, Rajarshi Roy, Mengyao Xu, Jonathan Raiman, Mohammad Shoeybi, Bryan Catanzaro, and Wei Ping. Nv-embed: Improved techniques for training llms as generalist embedding models. In International Conference on Learning Representations (ICLR), 2025. URL https://openreview.net/forum?id=lgsyLSsDRe. Spotlight. [6] Niklas Muennighoff, Hongjin Su, Liang Wang, Nan Yang, Furu Wei, Tao Yu, Amanpreet Singh, and Douwe Kiela. Generative representational instruction tuning. In International Conference on Learning Representations (ICLR), 2025. URL https://openreview.net/forum?id=BC4lIvfSzv. Poster. [7] Yanzhao Zhang, Mingxin Li, Dingkun Long, Xin Zhang, Huan Lin, Baosong Yang, Pengjun Xie, An Yang, Dayiheng Liu, Junyang Lin, et al. Qwen3 embedding: Advancing text embedding and reranking through foundation models. arXiv preprint arXiv:2506.05176, 2025. [8] Xinping Zhao, Xinshuo Hu, Zifei Shan, Shouzheng Huang, Yao Zhou, Xin Zhang, Zetian Sun, Zhenyu Liu, Dongfang Li, Xinyuan Wei, et al. Kalm-embedding-v2: Superior training techniques and data inspire a versatile embedding model. arXiv preprint arXiv:2506.20923, 2025. [9] Yauhen Babakhin, Radek Osmulski, Ronay Ak, Gabriel Moreira, Mengyao Xu, Benedikt Schifferer, Bo Liu, and Even Oldridge. Llama-embed-nemotron-8b: A universal text embedding model for multilingual and cross-lingual tasks. arXiv preprint arXiv:2511.07025, 2025. [10] Henrique Schechter Vera, Sahil Dua, Biao Zhang, Daniel Salz, Ryan Mullins, Sindhu Raghuram Panyam, Sara Smoot, Iftekhar Naim, Joe Zou, Feiyang Chen, et al. Embeddinggemma: Powerful and lightweight text representations. arXiv preprint arXiv:2509.20354, 2025. [11] Kenneth Enevoldsen, Isaac Chung, Imene Kerboua, Márton Kardos, Ashwin Mathur, David Stap, Jay Gala, Wissam Siblini, Dominik Krzemiński, Genta Indra Winata, et al. Mmteb: Massive multilingual text embedding benchmark. In International Conference on Learning Representations (ICLR), 2025. URL https://openreview.net/forum?id=zl3pfz4VCV. Poster. [12] Alexander H Liu, Kartik Khandelwal, Sandeep Subramanian, Victor Jouault, Abhinav Rastogi, Adrien Sadé, Alan Jeffares, Albert Jiang, Alexandre Cahill, Alexandre Gavaudan, et al. Ministral 3. arXiv preprint arXiv:2601.08584, 2026. [13] Saurav Muralidharan, Sharath Turuvekere Sreenivas, Raviraj Joshi, Marcin Chochowski, Mostofa Patwary, Mohammad Shoeybi, Bryan Catanzaro, Jan Kautz, and Pavlo Molchanov. Compact language models via pruning and knowledge distillation. Advances in Neural Information Processing Systems, 37:41076–41102, 2024. [14] Lee Xiong, Chenyan Xiong, Ye Li, Kwok-Fung Tang, Jialin Liu, Paul N Bennett, Junaid Ahmed, and Arnold Overwijk. Approximate nearest neighbor negative contrastive learning for dense text retrieval. In International Conference on Learning Representations, 2020. [15] Wenhao Lu, Jian Jiao, and Ruofei Zhang. Twinbert: Distilling knowledge to twin-structured compressed bert models for large-scale retrieval. In Proceedings of the 29th ACM International Conference on Information and Knowledge Management (CIKM ’20), pages 2645–2652, 2020. doi: 10.1145/3340531.3412747. URL https://researchr.org/publication/LuJZ20-0. [16] Fedor Moiseev, Gustavo Hernandez Abrego, Peter Dornbach, Imed Zitouni, Enrique Alfonseca, and Zhe Dong. Samtone: Improving contrastive loss for dual encoder retrieval models with same tower negatives. In Findings of the Association for Computational Linguistics: ACL 2023, pages 12028–12037, 2023. [17] Shizhan Gong, Yankai Jiang, Qi Dou, and Farzan Farnia. Kernel-based unsupervised embedding alignment for enhanced visual representation in vision-language models. In Aarti Singh, Maryam Fazel, Daniel Hsu, Simon Lacoste-Julien, Felix Berkenkamp, Tegan Maharaj, Kiri Wagstaff, and Jerry Zhu, editors, Proceedings of the 42nd International Conference on Machine Learning, volume 267 of Proceedings of Machine Learning Research, pages 19912–19931. PMLR, 13–19 Jul 2025. URL https://proceedings.mlr.press/v267/gong25b.html. [18] Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748, 2018.
10
HARNESS-LM: A Training Recipe for Sponsored Search Retrieval
A
𝐵 , it forms a candidate set C consisting of its positive 𝑑 + {(𝑞𝑖 ,𝑑𝑖+ )}𝑖=1 𝑖 𝑖 together with the other in-batch positives {𝑑 +𝑗 } 𝑗≠𝑖 as hard negatives. Because the document encoder is frozen, the candidate embeddings {𝑓𝐷𝑇 (𝑑)}𝑑 ∈ C𝑖 are produced once by the teacher and reused for both the teacher and the student forward pass. The teacher and student induce two similarity distributions over C𝑖 , exp 𝑓𝑄𝑇 (𝑞𝑖 ) ⊤ 𝑓𝐷𝑇 (𝑑)/𝜏𝑇 𝑇 𝑝𝑖 (𝑑) = Í , 𝑇 ⊤ 𝑇 ′ 𝑑 ′ ∈ C𝑖 exp 𝑓𝑄 (𝑞𝑖 ) 𝑓𝐷 (𝑑 )/𝜏𝑇
Oracle Features: Example
For the query “change from pdf into word free”, the corresponding oracle input augments the raw query with multiple feature-rich fields, including query rewrites, an intent explanation 8 , and retrieved web-search titles. Table 11 highlights different features that augments the query. The query, its several rewrites, intent and websearch titles are concatenated and fed to the oracle model while training. Table 11: Example of an oracle input that augments the raw query with feature-rich fields. Field
Content
Query
change from pdf into word free
Rewrites
exp 𝑓𝑄𝑆 (𝑞𝑖 ) ⊤ 𝑓𝐷𝑇 (𝑑)/𝜏𝑆 𝑝𝑖𝑆 (𝑑) = Í , 𝑆 ′ ⊤ 𝑇 𝑑 ′ ∈ C𝑖 exp 𝑓𝑄 (𝑞𝑖 ) 𝑓𝐷 (𝑑 )/𝜏𝑆
and the loss minimizes the KL divergence from the (fixed) teacher distribution to the student, LKALM =
• change pdf format to word for free • convert pdf files to word documents free • convert pdf to word online free • free online tool to convert pdf to word • free pdf to word converter • free pdf to word online • how to change pdf to word for free • online pdf to word conversion free • pdf to word converter free download • pdf to word free software
𝐵 𝐵 1 ∑︁ 𝑇 𝑆 1 ∑︁ ∑︁ 𝑇 KL 𝑝𝑖 𝑝𝑖 = − 𝑝𝑖 (𝑑)log𝑝𝑖𝑆 (𝑑) + Λ, 𝐵 𝑖=1 𝐵 𝑖=1 𝑑 ∈ C𝑖
where Λ is a constant because the teacher term 𝑝𝑇𝑖 has no gradient, and 𝜏𝑇 and 𝜏𝑆 are the teacher and student temperatures, respectively, and are set to 𝜏𝑇 =𝜏𝑆 =0.05 following [8]. We use a batch size 𝐵=1024 so that each query sees a sufficiently rich set of hard negatives. Note that, unlike the original KaLM-Embedding-V2 recipe which jointly updates both encoders, in our setting the document encoder 𝑓𝐷𝑇 is held fixed: this is what makes the resulting student directly compatible with the Phase 1 document encoder.
Query Intent The user is looking for a product, service, or Explanation information that can help convert a PDF file into a Word document for free.
B.2
Kernel-based Unsupervised Embedding Alignment
Web Search • convert pdf to word online for free adobe acrobat Titles • pdf to word converter 100 free • pdf to word convert pdf to word online for free • PDF to WORD Convert PDF to Word online for free iLovePDF
Kernel-based Unsupervised Embedding Alignment (KUEA) [17] asks the student to reproduce the pairwise query-side similarity structure induced by the teacher over a mini-batch, without involving docu𝐵 and a kernel 𝑘 (·,·), ments at all. For a mini-batch of queries {𝑞𝑖 }𝑖=1 2 ∑︁ 1 LKUEA = 𝑘 𝑓𝑄𝑆 (𝑞𝑖 ),𝑓𝑄𝑆 (𝑞 𝑗 ) −𝑘 𝑓𝑄𝑇 (𝑞𝑖 ),𝑓𝑄𝑇 (𝑞 𝑗 ) . 𝐵(𝐵 −1) 𝑖≠𝑗
B
The teacher kernel matrix is held fixed (no gradient), and self-similarity terms (𝑖=𝑗) are excluded since they are constant. We use a polynomial kernel of degree 𝑑=3 applied directly to ℓ2 -normalized embeddings, 𝑘 (𝑢,𝑣) = (𝑢ˆ ⊤ 𝑣ˆ + 1)𝑑 , which bounds kernel values in [0,8] for both teacher and student. The original KUEA formulation [17] normalizes each kernel matrix post-hoc (Frobenius or centered [24]) to absorb scale and dimension mismatch across unrelated encoders; since we feed already-normalized embeddings of matched dimension (𝑑=128) into the kernel, this is unnecessary in our intra-family setting (Qwen3-Embedding-4B teacher, Qwen3-Embedding-0.6B student).
Alternative Loss Functions for Alignment
In this section, we describe the two alternative alignment objectives we benchmark against the proposed ℓ2 loss in Table 5: the KL-based contrastive distillation loss of KaLM-Embedding-V2 [8] and the Kernel-based Unsupervised Embedding Alignment (KUEA) loss of Gong et al. [17]. Let 𝑓𝑄𝑇 and 𝑓𝐷𝑇 be the teacher query and document encoders, respectively, from the upper-bound model of Phase 1. In our experimental setting, this is the Qwen3-4B model trained with LoRA rank 128. Let 𝑓𝑄𝑆 be the student query encoder (Qwen3-0.6B) that we are aligning to the 4B-query encoder. Also, a reminder that during the alignment phase, both 𝑓𝑄𝑇 and 𝑓𝐷𝑇 remain frozen.
B.1
Inference-time rotation alignment. Because KUEA only constrains pairwise similarities among queries, its solution is identifiable only up to an orthogonal transformation of the embedding space: any rotation 𝑅 ∈ R𝑑 ×𝑑 with 𝑅 ⊤ 𝑅 =𝐼 applied to 𝑓𝑄𝑆 leaves LKUEA unchanged but arbitrarily misaligns the student query embeddings with the frozen document embeddings 𝑓𝐷𝑇 , breaking compatibility with the precomputed offline index. To recover index compatibility without retraining or recomputing 𝑓𝐷𝑇 , we estimate a single rotation on a held-out validation set of queries V = {𝑞 𝑣 }𝑁 𝑣=1 by aligning student
KL-based contrastive distillation
In the Kullback-Leibler divergence-based loss function defined in [8], the loss function transfers the teacher’s score distribution over a candidate set to the student. Concretely, for each query𝑞𝑖 in a mini-batch 8 query rewrites and intent explanations are generated via LLM
11
Gupta et al.
the cached 𝑓𝐷𝑇 index, i.e. the scoring function becomes 𝑠 (𝑞, 𝑑) = (𝑅★ 𝑓𝑄𝑆 (𝑞)) ⊤ 𝑓𝐷𝑇 (𝑑). This adds a single 𝑑 ×𝑑 matmul (𝑑=128) to queryside serving cost and, crucially, requires no changes to the offline document encoder. The numbers reported for KUEA in Table 5 use this Procrustes post-processing.
query embeddings to the corresponding teacher query embeddings via the orthogonal Procrustes problem: ∑︁ 2 𝑅★ = arg min 𝑅𝑓𝑄𝑆 (𝑞 𝑣 ) − 𝑓𝑄𝑇 (𝑞 𝑣 ) 2 = 𝑈𝑉 ⊤, ⊤ 𝑅 𝑅=𝐼
𝑞𝑣 ∈ V
Í where 𝑈 Σ𝑉 ⊤ is the SVD of 𝑣 𝑓𝑄𝑇 (𝑞 𝑣 )𝑓𝑄𝑆 (𝑞 𝑣 ) ⊤ . At test time we apply the same 𝑅★ to every test-query embedding and retrieve against
12