Rajinder Sandhu
Di Mu
Cheng Chang
Md Shahriar Tasjid
[email protected] Layer 6 AI Toronto, ON, Canada
[email protected] Layer 6 AI Toronto, ON, Canada
[email protected] Layer 6 AI Toronto, ON, Canada
[email protected] Dalhousie University Halifax, NS, Canada
Himanshu Rai
Maksims Volkovs
Ga Wu
[email protected] Layer 6 AI Toronto, ON, Canada
[email protected] Layer 6 AI Toronto, ON, Canada
[email protected] Dalhousie University Halifax, NS, Canada
Abstract
52
Dense vector retrieval is the practical backbone of Retrieval- Augmented Generation (RAG), but similarity search can suffer from precision limitations. Conversely, utility-based approaches leveraging LLM re-ranking often achieve superior performance but are computationally prohibitive and prone to noise inherent in perplexity estimation. We propose Utility-Aligned Embeddings (UAE), a framework designed to merge these advantages into a practical, high-performance retrieval method. We formulate retrieval as a distribution matching problem, training a bi-encoder to imitate a utility distribution derived from perplexity reduction using a Utility-Modulated InfoNCE objective. This approach injects graded utility signals directly into the embedding space without requiring test-time LLM inference. On the QASPER benchmark, UAE improves retrieval Recall@1 by 30.59%, MAP by 30.16% and Token F1 by 17.3% over the strong semantic baseline BGE-Base. Crucially, UAE is over 180× faster than the efficient LLM re-ranking methods preserving competitive performance, demonstrating that aligning retrieval with generative utility yields reliable contexts at scale.
50
Keywords Retrieval-Augmented Generation, Dense Retrieval, Generative Utility, Representation Learning, Distribution Matching
1
Introduction
Dense vector similarity search remains the bedrock of real-world Retrieval-Augmented Generation (RAG) systems. By mapping queries and candidates into a shared representation space, these systems leverage efficient Approximate Nearest Neighbor (ANN) search to handle large-scale datasets with minimal latency [2, 5, 11]. However, this paradigm is increasingly criticized for its reliance on semantic similarity as a proxy for generative utility. Growing evidence suggests that passages with high semantic similarity (topical overlap) often fail to provide answer-critical information and can even introduce semantic distractors that mislead the generator, especially in long-context settings where incorrect but similar passages increase decoding uncertainty [2, 13, 27, 33]. To bridge this gap, current state-of-the-art approaches shift toward utility-based retrieval [3, 12, 28, 34], where relevance is defined by how effectively a document helps a Large Language Model (LLM) produce a correct response [11]. In practice, this is often measured via perplexity reduction: a document is considered useful if its presence as context makes the ground-truth answer more predictable
UAE+SePer
UAE+BGE-Reranker
UAE+LLM Reranker
48
Gen-F1 (%)
arXiv:2604.22722v1 [cs.IR] 24 Apr 2026
Aligning Dense Retrievers with LLM Utility via Distillation
SePer
ColBERTv2 BGE-M3 Rerank
46
UAE (Ours)
44
SPLADE V3
42
BM25
RankGPT
NV-Embed V2 E5-Mistral
Ours (Single) Ours (Multi) Base (Dense) Base (Rerank) Frontier
40 BGE-Base
38
5
10
100
Latency (ms)
1000
3000
Figure 1: Efficiency vs. Performance. UAE (Read Star) occupies the optimal sweet spot: it approaches the performance of strong baselines while being ∼100x faster.
to the model [12]. While conceptually sound, these utility-based approaches face a significant practicality wall. Relying on LLMs for query generation or post-hoc re-ranking is computationally prohibitive for large-scale deployment [12, 15, 22]. Furthermore, utility signals derived from perplexity are notoriously noisy and stochastic, sensitive to token-level variations and decoding dynamics that make them difficult to use as stable training targets [9, 28]. This necessitates complex, multi-stage architectures that improve performance at the cost of extreme inference latency and high computational overhead [19, 25]. In this work, we propose Utility-Aligned Embeddings (UAE), a framework designed to merge the efficiency of dense retrieval with the superior performance of utility-based methods. Our core insight is that generative utility should be distilled directly into the bi-encoder’s (dual-encoder dense retriever) embedding space, bypassing expensive test-time LLM inference while capturing the generator’s actual preferences. We formulate this alignment as a distribution matching problem; rather than treating noisy utility scores as direct regression targets (which can lead to unstable learning and overfitting [6, 24]) we train the bi-encoder to imitate a utility-induced target distribution via a Utility-Modulated InfoNCE objective (Figure 2). Our empirical results on the QASPER benchmark [4] demonstrate that UAE significantly outperforms standard semantic retrievers [31], improving reterival Recall@1 by 30.59% and Token F1 by 17.3%. Crucially, UAE occupies a unique Paretooptimal position: it provides substantial performance gains while
,,
Rajinder Sandhu, Di Mu, Cheng Chang, Md Shahriar Tasjid, Himanshu Rai, Maksims Volkovs, and Ga Wu
being 180× faster than the efficient LLM-based re-ranking methods [26]. Figure 1 highlights the advantage of the proposed method. By maintaining standard ANN compatibility, UAE offers a practical, scalable solution for building high-fidelity RAG systems that are both informative and computationally efficient.
2
Utility Aligned Embeddings
Our objective is to develop a dense retrieval system where the embedding space structure is supervised by generative utility rather than simple semantic overlap. However, the high variance and heavy-tailed nature of raw utility scores (Figure 3 top) make them unsuitable as direct supervision targets for a bi-encoder. To bridge this gap, we decompose the alignment process into a two-stage distillation framework (Figure 2). First, we stabilize the noisy, computationally expensive utility signals by distilling them into a parameterized reward model. Second, we align the dense retriever with this model by training it to imitate the resulting utility-induced ranking distribution through supervised distribution matching.
2.1
Reward-Guided Embeddings Training
exp(𝜆 · 𝑅𝜓 (𝑞, 𝑑𝑖 )) , 𝑑 𝑗 ∈ B exp(𝜆 · 𝑅𝜓 (𝑞, 𝑑 𝑗 ))
(3)
exp(⟨𝑓𝜙 (𝑞), 𝑓𝜙 (𝑑𝑖 )⟩/𝜏) , 𝑑 𝑗 ∈ B exp(⟨𝑓𝜙 (𝑞), 𝑓𝜙 (𝑑 𝑗 )⟩/𝜏)
(4)
𝜋 ∗ (𝑑𝑖 | 𝑞) = Í 𝜋𝜙 (𝑑𝑖 | 𝑞) = Í
Parameterized Utility Approximation
The utility of a context document 𝑑 for correctly answering a question 𝑞 is denoted as 𝑈 (𝑞, 𝑑, 𝑎), where 𝑎 represents the ground-truth answer. In practice, utility is typically estimated via perplexity: ! |𝑎 | 1 ∑︁ 𝑈 (𝑞, 𝑑, 𝑎) = exp log 𝑝𝜃 (𝑥𝑡 | 𝑥 <𝑡 , 𝑞, 𝑑) , (1) |𝑎| 𝑡 =1 where 𝑝𝜃 represents the token-level generation probability of a language model. Given that natural language allows for multiple valid expressions of the same answer 𝑎 ∈ A𝑞 , we extend this definition to the expected utility 𝑈 (𝑞, 𝑑, A𝑞 ) = E𝑎∈ A𝑞 [𝑈 (𝑞, 𝑑, 𝑎)]. Theoretically, this utility function is deterministic relative to the finite set of ground-truth answers A𝑞 . In practice, however, the estimation is highly stochastic and fragile; The instability arises from three primary factors: (1) the intractable variety of linguistic expressions for ground-truth answers; (2) non-deterministic model outputs caused by matrix parallelism and floating-point associativity errors; and (3) a high sensitivity to answer length |𝑎|. Directly using raw generative utility through regression (using standard objectives like Mean Squared Error or Huber losses) is unreliable due to the aforementioned stochasticity and heavytailed distribution. Instead, we formulate reward learning as a pairwise ranking problem, focusing exclusively on preserving the generator’s ordinal preferences. We construct training quadruplets (𝑞, A𝑞 , 𝑑𝑖 , 𝑑 𝑗 ) and optimize a parameterized reward model 𝑅𝜓 using a margin-based ranking loss: LReward = max(0, 𝛿 − (𝑅𝜓 (𝑞, 𝑑𝑖 ) − 𝑅𝜓 (𝑞, 𝑑 𝑗 ))),
2.2
The reward model 𝑅𝜓 effectively captures utility but, as a crossencoder, is computationally expensive for large-scale inference. We therefore use 𝑅𝜓 as an offline teacher to distill utility preferences into a dense bi-encoder, preserving ANN search efficiency. Rather than using reinforcement learning (which is destabilized by massive discrete action spaces and high reward variance), we adopt an imitation learning formulation. We treat the normalized reward distribution as a target expert policy 𝜋 ∗ and train the retriever 𝜋𝜙 to minimize their divergence via supervised gradients, bypassing the exploration challenges of traditional RL. Distribution Matching Objective. To align the retriever with generative utility, we define a target (teacher) distribution 𝜋 ∗ and a retriever (student) distribution 𝜋𝜙 over a set of candidate contexts B as follows:
(2)
where we enforce 𝑅𝜓 (𝑞, 𝑑𝑖 ) > 𝑅𝜓 (𝑞, 𝑑 𝑗 )+𝛿 whenever 𝑈 (𝑞, 𝑑𝑖 , A𝑞 ) > 𝑈 (𝑞, 𝑑 𝑗 , A𝑞 ). By prioritizing the relative ordering of candidates over the approximation of exact utility values, this formulation provides a more robust signal for learning utility-driven representations. Here, 𝑅𝜓 is a Transformer-based encoding model (BERT). As illustrated in Figure 3 (bottom), this learned reward model effectively discriminates between truly useful contexts and the "hard semantic negatives" that often deceive standard top-1 similarity search.
where 𝜆 is a temperature hyperparameter controlling the sharpness of the target utility distribution and 𝜏 is the retriever’s scaling temperature. The retriever is optimized by minimizing the KullbackLeibler (KL) divergence between the two distributions: LUAE (𝜙) = E𝑞 𝐷 KL (𝜋 ∗ ∥ 𝜋𝜙 ) ∑︁ ∑︁ (5) ∝− 𝜋 ∗ (𝑑 | 𝑞) log 𝜋 (𝑑 | 𝑞). 𝑖
𝜙
𝑖
𝑞 𝑑𝑖 ∈ B
This objective reshapes the embedding space to reflect the generator’s preferences: high-utility documents are pulled toward the query, while low-utility contexts are pushed away. Critically, unlike standard InfoNCE which treats all negatives as equally irrelevant, this formulation preserves the graded, ordinal structure of the utility space, allowing the retriever to distinguish between varying degrees of relevance. Utility-Aware Hard Negative Mining. Since computing the full distribution over the entire corpus is computationally intractable, we adopt the Noise Contrastive Estimation (NCE) paradigm to approximate the global distribution using a combination of gold contexts and informative negative samples. However, random negatives often provide trivial gradients, while standard semantic hard negatives may accidentally include documents that the generator finds useful. To address this, we implement a Utility-Gated Mining, where a document 𝑑 − is selected as a negative only if it satisfies two conditions: (1) it is semantically similar to the query (ranked in the top-𝑘 by a base similarity function 𝑆 sem ), and (2) it is assigned substantially lower utility than the gold evidence 𝑑 + by the reward model, satisfying 𝑅𝜓 (𝑞, 𝑑 + ) − 𝑅𝜓 (𝑞, 𝑑 − ) > 𝛿. The margin 𝛿 governs the trade-off between coverage and signal purity; a conservative margin ensures that the retriever focuses on resolving semantic distractors, documents that are proximally located in embedding space but offer negligible generative utility. This gated approach, combined with the robustness of the KL-divergence objective, prevents the propagation of noise from the reward model to the final embeddings.
Aligning Dense Retrievers with LLM Utility via Distillation
Stage B: Utility Aligned Embeddings
Stage A: Utility Distillation
q
d
Stage C: Final Inference
a
Utility Modulated Loss
Hard Negative Miner
Scalar Reward
Trained bi-encoder
Target Distribution
Semantic Similarity (base)
LLM Frozen Raw Scores
,,
Trainable Shared Encoder
Dot-product Similarity ANN/FAISS/HNSW
Similarity Distribution
pair-wise margin loss
Gradients Flow
Online Encoding
Offline / Not Used in Inference
Queries
Student Distribution
Candidate batch
Reward Model (Cross-Encoder)
Corpus (Once)
Ranked Results
Imitation Learning
Med: 3.98
Candidate
Med: 0.44
1500
0.2 0.0 l ) c LexicSaemanCtiross-EOnucrs (RM
Candidate
Hard Negative
Frequency
Reward Score
0
0
2000
4000
Pairwise Accuracy: 98.3%
60 40 20 0
1
0
1
2
3
4
0.10 0.05 0.00 l ) c LexicSaemanCtiross-EOnucrs (RM
0.6 0.4 0.2 0.0 l ) c LexicSaemanCtiross-EOnucrs (RM
Figure 4: Alignment of various retrieval models with the LLM’s utility distribution. Reward Model (RM), while not a practical retriever, significantly outperforms lexical, semantic, and cross-encoder baselines across all ranking metrics, providing a higher-fidelity supervision signal for distillation.
5
Figure 3: Raw generative utility scores (top) are heavy-tailed and highly variable, making direct regression unstable and motivating reward-based smoothing. Reward model learned (bottom) mitigates the problem.
Scalability. To ensure training efficiency, we adopt an in-batch encoding protocol similar to DPR [14]. Rather than re-encoding the entire corpus, we dynamically encode only the sampled candidate set B = {𝑑 +, 𝑑 1− , . . . , 𝑑𝑘− } for each query batch. This enables end-toend optimization of the embedding space with gradients flowing through a shared encoder, while keeping training costs independent of the total corpus size. Crucially, the expensive reward model is utilized only during this offline phase; at inference time, the corpus is indexed for standard ANN search, fully decoupling utility alignment from runtime latency.
3
0.8 0.15
6000
(d) Reward Score: Pairwise Delta
80
Med: 1.98
0.4
500
Hard Negative
Med: 4.09
0.6
1000
(c) Reward Score: Low Variability 7 6 5 4 3 2 1
0.8
Pairwise Acc@5
102 103 104
Pairwise Accuracy: 96.5%
Recall@5
0
1.0
(b) Raw Utility: Pairwise Delta 2000
NDCG@5
(a) Raw Utility: High Variability 103 102
Frequency
Utility (SymLog Scale)
Figure 2: Overview of Utility-Aligned Embeddings (UAE). Utility is distilled offline into a reward model (Stage A), which defines a target utility distribution used to align a dense retriever via distribution matching (Stage B). At inference time, the trained bi-encoder supports standard ANN retrieval without any reward model or LLM inference (Stage C).
Experiments
Datasets & Evaluation Protocol. We evaluate UAE on two distinct RAG benchmarks: QASPER [4] (long-doc scientific QA) and NewsQA [29] (short-doc news extraction). We employ a hardnegative setting where the candidate pool (𝑁 = 50) for each query is constructed via dense retrieval (BGE-Base [31]) and reward model utility. This populates the pool with the semantic distractors described in Section 2.2, rigorously testing the model’s ability to prioritize true generative utility over high-similarity non-answers.
Generation Protocol. To measure downstream utility, we utilize Llama-3-8B-Instruct [7] as the fixed generator with greedy decoding (temperature=0) for reproducibility. Dataset-specific system prompts align the generator’s output with the ground-truth format: extractive phrases for NewsQA and evidence-based summaries for QASPER. Performance is quantified using Token F1 [21] and ROUGE-L [18] to assess both informational accuracy and structural fluency. Model Configuration. We initialize the utility reward model with microsoft/deberta-v3-base [8] and the retriever with BAAI/bgebase-en-v1.5 [31]. We also employ Low-Rank Adaptation (LoRA) [10] for parameter-efficient fine-tuning across both components. Reward Model Validation. We verify the distillation fidelity of the reward model (RM) in approximating the Llama-3 generator’s utility (Eq. 1) on the NewsQA validation set. To assess cross-architecture transfer, we benchmark the DeBERTa-based RM against generalpurpose retrievers (BM25, BGE, and BGE-Reranker). As shown in Figure 4, while standard models are frequently misled by semantic distractors (NDCG@1 ≤ 0.72), RM achieves 0.86 NDCG@1 and 0.70 pairwise accuracy. This confirms RM as a high-fidelity proxy that successfully distills the heavy LLM’s preferences into a compact supervisor. We set a utility threshold of 0.1 (calibrated to the top 10% of the score distribution) for relevance. While LLM utility may diverge from human labels, a known trait in LLM-as-a-judge frameworks, we prioritize generator-specific alignment to ensure the retriever surfaces contexts that the fixed LLM can effectively utilize, which is the core objective of utility-aligned RAG.
,,
Rajinder Sandhu, Di Mu, Cheng Chang, Md Shahriar Tasjid, Himanshu Rai, Maksims Volkovs, and Ga Wu
Table 1: Comparison of UAE against baselines on QASPER and NewsQA. We report Recall@1 (R@1), Recall@3(R@3), Expected Utility @1 (ExpUtil@1), Mean Average Precision (MAP), and Generation Metrics (Gen-F1, ROUGE-L). Efficiency is measured as latency (Lat.) in milliseconds. N/A denotes not available.
26.26 38.46 6.21 36.87 40.75
49.70 61.76 18.23 61.28 64.87
0.6792 0.9836 0.0932 0.8201 0.9790
27.81 32.13 8.61 38.02 33.98
20.5 25.2 5.9 23.9 26.7
20.7 25.2 6.0 24.1 26.8
5 5 9 8 46
49.61 50.58 22.45 34.15 55.11
69.79 73.67 51.29 57.37 76.53
5.1859 5.3588 4.1819 4.7378 5.5024
52.13 53.34 32.60 43.84 58.31
41.8 44.7 25.5 38.6 47.3
42.2 45.2 25.9 39.0 47.8
6 4 9 8 35
NV-Embed V2 [17]
47.04
71.75
0.9749
38.67
27.8
27.7
52 47.35
72.06
5.3325
51.28
43.4
43.9
54
E5-Mistral [34]
40.46
65.24
0.8688
33.93
26.0
26.1
39 45.15
69.99
5.2489
47.64
41.9
42.5
38
BGE-V2-M3 reranker [1] RankGPT [26] GainRAG [12] UtilityQwen [34] SePer [3]
43.05 48.89 46.30 40.95 59.84
66.79 71.52 69.23 64.25 72.71
1.0882 1.2328 1.3057 1.096 2.3445
42.93 46.13 45.29 37.66 50.12
27.4 29.2 28.0 24.0 32.6
27.5 29.3 27.9 24.1 32.1
60.67 49.68 41.20 42.77 60.61
77.62 71.54 64.17 65.35 78.65
5.7339 4.9680 5.0329 5.013 6.1131
65.49 46.78 50.96 47.90 66.17
46.5 43.7 40.7 38.9 47.6
47.0 44.2 41.4 39.3 48.0
34 1610 2104 3194 1711
UAE (Ours)
48.15
70.56
1.3269
49.49
27.0
27.0
9 54.90
77.30
5.8184
62.31
45.6
46.1
9
UAE + BGE-reranker UAE + SePer UAE + LLM Reranker
44.45 61.91 50.48
70.71 74.11 75.70
1.1458 2.4194 1.2631
45.94 53.39 50.65
28.5 34.0 29.8
28.5 33.3 29.7
43 65.65 1704 66.43 1617 60.87
86.22 88.10 87.32
5.8727 6.2394 5.7416
72.04 73.15 67.92
49.8 50.5 49.8
50.3 51.0 50.3
33 1709 1599
Rerank Ablation
Classic Retrieval
BM25 SPLADE V3 [16] InfoNCE Tuned BGE-Base [31] ColBERTV2 [23]
LLM Dense Retriever
NewsQA (Short-doc)
R@1 R@3 ExpUtil@1 MAP Gen-F1 ROUGE-L
Multi-Stage (LLM) Rerank
QASPER (Long-doc) Method
Figure 5: Zero-shot transfer performance. The model was trained only on QASPER and evaluated on unseen datasets. UAE improves or retains performance across all domains. Main Results. Table 1 presents a comprehensive evaluation of UAE against three distinct categories of baselines: classic lexical retrievers (e.g., BM25, SPLADE), state-of-the-art dense retrievers (e.g., BGE, NV-Embed), and computation-heavy multi-stage rerankers (e.g., RankGPT, SePer). We analyze performance across three key dimensions: Retrieval Performance. UAE significantly outperforms all singlestage retrievers across both datasets. On the long-document QASPER benchmark, UAE achieves a MAP of 49.49, surpassing the strongest dense baseline (BGE-Base, 38.02) by ≈ 11 points, the late-interaction model ColBERTv2 (33.98) by ≈ 15 points and even outperforming the massive LLM-based embedder NV-Embed V2 (38.67). Notably, on NewsQA, UAE’s retrieval quality (Recall@1 54.90) exceeds that of the computationally expensive RankGPT (49.68), demonstrating that aligning embeddings with generative utility can yield rerankerlevel precision in a single retrieval step. This advantage extends to ExpUtil@1 (average utility of the top-1 context). On NewsQA, UAE (5.818) surpasses both BGE-Base (4.738) and even the computationheavy RankGPT (4.968), confirming that UAE prioritizes contexts maximally conducive to generation rather than mere semantic relevance.
Lat. R@1 R@3 ExpUtil@1 MAP Gen-F1 ROUGE-L
52 1663 2293 3200 1696
Lat.
Generation Quality. These gains in retrieval translate directly to downstream generation fidelity. On NewsQA, UAE achieves a GenF1 of 45.6, surpassing standard dense retrieval (BGE-Base: 38.6) and outperforming the multi-stage RankGPT pipeline (43.7). Similarly, on QASPER, UAE improves Gen-F1 scores over classic baselines (27.0 vs. 23.9 for BGE), confirming that the retriever selects contexts that are not merely semantically relevant, but factually sufficient for the LLM to generate correct answers. Efficiency & Compatibility. A critical barrier for deploying RAG in real-time production environments is latency; HCI research establishes that system response times must remain under 100 ms to be perceived as instantaneous by users [20]. While multi-stage rerankers like RankGPT and SePer offer strong performance, they incur prohibitive latencies (>1600 ms), rendering them unsuitable for interactive applications. In contrast, UAE maintains a latency of ≈ 9 ms, matching the speed of standard bi-encoders. Furthermore, UAE is not mutually exclusive with reranking; as shown in the ablation study, combining UAE with rerankers (e.g., UAE + SePer) yields new state-of-the-art results (MAP 53.39 on QASPER), proving that UAE serves as a superior "first-stage" retriever that enhances the entire pipeline when latency budgets permit. Zero-Shot Generalization and Robustness. A common failure mode of domain-specific fine-tuning is catastrophic forgetting, where the model loses its ability to generalize to new tasks. To evaluate this, our QASPER-trained model is tested against four out-of-domain datasets: NewsQA , SQuAD [21], HotpotQA [32], and SciFact [30]. Figure 5 summarizes the results.
4
Conclusion
We present Utility-Aligned Embeddings (UAE), a framework that bridges the gap between the efficiency of dense retrieval and the
Aligning Dense Retrievers with LLM Utility via Distillation
high performance of utility-based models through distribution matching. Our results demonstrate that UAE significantly reduces semantic distractors and improves generation quality while operating 180× faster than LLM-based re-ranking methods. By maintaining standard ANN compatibility and serving as a high-quality foundation for multi-stage pipelines, UAE provides a practical and scalable solution for utility-driven RAG systems.
References [1] Jianlv Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu. 2024. M3-Embedding: Multi-Lingual, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation. arXiv:2402.03216 [2] Florin Cuconasu, Giovanni Trappolini, Federico Siciliano, Simone Filice, Cesare Campagnano, Yoelle Maarek, Nicola Tonellotto, and Fabrizio Silvestri. 2024. The Power of Noise: Redefining Retrieval for RAG Systems. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval. [3] Lu Dai, Yijie Xu, Jinhui Ye, Hao Liu, and Hui Xiong. 2025. SePer: Measure Retrieval Utility Through The Lens Of Semantic Perplexity Reduction. In The Thirteenth International Conference on Learning Representations. [4] Pradeep Dasigi, Kyle Lo, Iz Beltagy, Arman Cohan, Noah A Smith, and Matt Gardner. 2021. A Dataset of Information-Seeking Questions and Answers Anchored in Research Papers. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 4599–4610. [5] Shahul Es, Jithin James, Luis Espinosa Anke, and Steven Schockaert. 2024. RAGAs: Automated Evaluation of Retrieval Augmented Generation. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations. Association for Computational Linguistics, 150–158. [6] Aritra Ghosh, Himanshu Kumar, and P.S. Sastry. 2017. Robust Loss Functions under Label Noise for Deep Neural Networks. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 31. [7] Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, and et al. 2024. The Llama 3 Herd of Models. arXiv:2407.21783 [8] Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. 2021. DeBERTa: Decoding-enhanced BERT with Disentangled Attention. In International Conference on Learning Representations. [9] Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. 2020. The Curious Case of Neural Text Degeneration. In International Conference on Learning Representations (ICLR). [10] Edward J Hu, yelong shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. LoRA: Low-Rank Adaptation of Large Language Models. In International Conference on Learning Representations. [11] Oz Huly, Idan Pogrebinsky, David Carmel, Oren Kurland, and Yoelle Maarek. 2024. Old IR Methods Meet RAG. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval. Association for Computing Machinery, 2559–2563. [12] Yi Jiang, Sendong Zhao, Jianbo Li, Haochun Wang, and Bing Qin. 2025. GainRAG: Preference Alignment in Retrieval-Augmented Generation through Gain Signal Synthesis. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 10746–10757. [13] Bowen Jin, Jinsung Yoon, Jiawei Han, and Sercan O Arik. 2025. Long-Context LLMs Meet RAG: Overcoming Challenges for Long Inputs in RAG. In The Thirteenth International Conference on Learning Representations. [14] Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. Dense Passage Retrieval for OpenDomain Question Answering. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics, 6769–6781. [15] Zixuan Ke, Weize Kong, Cheng Li, Mingyang Zhang, Qiaozhu Mei, and Michael Bendersky. 2024. Bridging the Preference Gap between Retrievers and LLMs. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 10438–10451. [16] Carlos Lassance, Hervé Déjean, Thibault Formal, and Stéphane Clinchant. 2024. SPLADE-v3: New baselines for SPLADE. arXiv:2403.06789 [17] Chankyu Lee, Rajarshi Roy, Mengyao Xu, Jonathan Raiman, Mohammad Shoeybi, Bryan Catanzaro, and Wei Ping. 2025. NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Models. In The Thirteenth International Conference on Learning Representations. [18] Chin-Yew Lin. 2004. ROUGE: A Package for Automatic Evaluation of Summaries. In Text Summarization Branches Out. 74–81.
,,
[19] Hao Liu, Zhengren Wang, Xi Chen, Zhiyu Li, Feiyu Xiong, Qinhan Yu, and Wentao Zhang. 2025. HopRAG: Multi-Hop Reasoning for Logic-Aware RetrievalAugmented Generation. In Findings of the Association for Computational Linguistics: ACL 2025. Association for Computational Linguistics, 1897–1913. [20] Robert B. Miller. 1968. Response time in man-computer conversational transactions. In Proceedings of the December 9-11, 1968, Fall Joint Computer Conference, Part I. Association for Computing Machinery, 267–277. [21] Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. SQuAD: 100,000+ Questions for Machine Comprehension of Text. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 2383–2392. [22] Alireza Salemi and Hamed Zamani. 2024. Evaluating Retrieval Quality in Retrieval-Augmented Generation. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval. Association for Computing Machinery, 2395–2400. [23] Keshav Santhanam, Omar Khattab, Jon Saad-Falcon, Christopher Potts, and Matei Zaharia. 2022. ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interaction. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics, 3715–3734. [24] Chenze Shao, Yang Feng, Jinchao Zhang, Fandong Meng, and Jie Zhou. 2021. Sequence-level training for non-autoregressive neural machine translation. Computational Linguistics 47, 4 (2021), 891–925. [25] Yucheng Shi, Qiaoyu Tan, Xuansheng Wu, Shaochen Zhong, Kaixiong Zhou, and Ninghao Liu. 2024. Retrieval-enhanced Knowledge Editing in Language Models for Multi-Hop Question Answering. In Proceedings of the 33rd ACM International Conference on Information and Knowledge Management. Association for Computing Machinery, 2056–2066. [26] Weiwei Sun, Lingyong Yan, Xinyu Ma, Shuaiqiang Wang, Pengjie Ren, Zhumin Chen, Dawei Yin, and Zhaochun Ren. 2023. Is ChatGPT Good at Search? Investigating Large Language Models as Re-Ranking Agents. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 14918–14937. [27] Fangzheng Tian, Debasis Ganguly, and Craig Macdonald. 2025. Is Relevance Propagated from Retriever to Generator in RAG?. In Advances in Information Retrieval. Springer Nature Switzerland, 32–48. [28] Fangzheng Tian, Debasis Ganguly, and Craig Macdonald. 2026. Predicting Retrieval Utility and Answer Quality in Retrieval-Augmented Generation. arXiv preprint arXiv:2601.14546 (2026). [29] Adam Trischler, Tong Wang, Xingdi Yuan, Justin Harris, Alessandro Sordoni, Philip Bachman, and Kaheer Suleman. 2017. NewsQA: A Machine Comprehension Dataset. In Proceedings of the 2nd Workshop on Representation Learning for NLP. 191–200. [30] David Wadden, Shanchuan Lin, Kyle Lo, Lucy Lu Wang, Madeleine van Zuylen, Arman Cohan, and Hannaneh Hajishirzi. 2020. Fact or Fiction: Verifying Scientific Claims. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics, 7534– 7550. [31] Shitao Xiao, Zheng Liu, Peitian Zhang, Niklas Muennighoff, Defu Lian, and Jian-Yun Nie. 2024. C-Pack: Packed Resources For General Chinese Embeddings. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval. Association for Computing Machinery, 641–649. [32] Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William W. Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. 2018. HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering. In Conference on Empirical Methods in Natural Language Processing (EMNLP). [33] Ori Yoran, Tomer Wolfson, Ori Ram, and Jonathan Berant. 2024. Making RetrievalAugmented Language Models Robust to Irrelevant Context. In The Twelfth International Conference on Learning Representations. [34] Hengran Zhang, Keping Bi, Jiafeng Guo, Jiaming Zhang, Shuaiqiang Wang, Dawei Yin, and Xueqi Cheng. 2025. Distilling a Small Utility-Based Passage Selector to Enhance Retrieval-Augmented Generation. In Proceedings of the 2025 Annual International ACM SIGIR Conference on Research and Development in Information Retrieval in the Asia Pacific Region. Association for Computing Machinery, 22–30.