W-RAG: Source-Aware Retrieval for Enterprise Document Generation from Heterogeneous Knowledge Bases

W-RAG: Source-Aware Retrieval for Enterprise Document Generation from Heterogeneous Knowledge Bases Hridya Dhulipala1

Rajesh Ombase2 Michael Wang3 Tien N. Nyugen1 1 The University of Texas at Dallas 2 DIGITAL MANAGEMENT, LLC (DMI) 3 Massachusetts Institute of Technology {hridya.dhulipala, tien.n.nyugen}@utdallas.edu rombase@dmigs.com wangrzm@mit.edu

arXiv:2608.22081v1 [cs.SE] 22 Aug 2026

Abstract Retrieval-Augmented Generation (RAG) enables large language models to incorporate external knowledge during generation, improving factual grounding and domain adaptability. However, existing RAG pipelines assume that evidence retrieved from multiple repositories can be ranked globally using a single similarity function. While suitable for opendomain retrieval, this assumption breaks down in enterprise document generation, where heterogeneous knowledge bases (such as policies, regulations, technical documentation, and departmental guidelines) serve distinct roles and must be jointly represented in the generated document. As a result, global ranking often produces unbalanced context dominated by a subset of sources, leading to incomplete enterprise drafts. To address this limitation, we propose W-RAG, a source-aware retrieval framework that performs ontology-guided retrieval, local ranking within each knowledge base, and source-level weighting to regulate evidence composition. We further introduce a new dataset for retrieval-grounded enterprise document generation spanning multiple document types and industry domains. Experiments show that standard RAG pipelines struggle on this task, while W-RAG significantly improves document coverage and generation quality.

1

Introduction

Retrieval-Augmented Generation (RAG) has emerged as a powerful paradigm for grounding large language models in knowledge, significantly improving factuality, transparency, and domain adaptability (Lewis et al., 2021; Gao et al., 2024; Guu et al., 2020). By combining neural generation with targeted retrieval from repositories, RAG systems help organizations synthesize policies, contracts, technical documentation, and client-facing materials with greater reliability than purely parametric models (Jadad-Garcia and Jadad, 2024).

Despite this progress, current enterprise RAG implementations still inherit structural limitations. Recent studies and experience reports suggest that naïve, similarity-only RAG pipelines often underperform on document-generation tasks such as policy creation, proposals, Request for Information documents (RFIs), and compliance reports in heterogeneous corporate settings (Bruckhaus, 2024; Hadfield and Clark, 2026; Richards, 2025; Jayarama Nettar, 2024; Brynjolfsson et al., 2024; Jadad-Garcia and Jadad, 2024). This is because existing RAG systems retrieve candidates from multiple heterogeneous knowledge bases and then rank them globally before selecting the top-k context for generation. While this design is natural in open-domain retrieval, it is problematic for enterprise document generation because it assumes that evidence from heterogeneous repositories can be compared using a single ranking function. In practice, however, enterprise knowledge bases differ in scope, authority, density, and role. Previous policies, regulations, scientific knowledge, playbooks, department notes, and technical specifications should be considered differently with regard to their influence in the drafting process. Global ranking tends to favor knowledge bases that are larger, denser, or more lexically aligned with the query, while suppressing smaller but necessary repositories. The result is often a context window dominated by one source family, with critical evidence from other departments or domains omitted. For enterprise document generation, the problem is thus not only ranking quality at the chunk level, but evidence composition at the document level. In this paper, we propose W-RAG, a retrieval framework for enterprise document generation that improves both evidence selection and source composition across multiple knowledge bases. WRAG operates in three stages. First, it uses ontology-guided topic modeling to identify passages that are not only lexically similar to the query,

but also more aligned with the thematic requirements of the target document. Second, it ranks evidence locally within each knowledge base instead of collapsing all retrieved candidates into a single global ranking. Third, it uses source-level (or user-defined) weights to regulate how much context is selected from each knowledge base, so that the final evidence set better reflects document needs rather than the dominance of any one repository. Additionally, a major obstacle in studying this problem is the lack of a dataset for retrievalgrounded enterprise document generation. Existing RAG datasets primarily evaluate question answering (Yang et al., 2018; Chen et al., 2024; Karpukhin et al., 2020; Kwiatkowski et al., 2019; Joshi et al., 2017; Rajpurkar et al., 2016; Bajaj et al., 2018; Trischler et al., 2017), summarization, or general knowledge-intensive generation (Berant et al., 2013), and do not capture the multi-source evidence composition required in enterprise writing. To address this gap, we introduce a new dataset covering multiple enterprise document types, including policies, product launch documents, M&A due diligence reports, and academic program proposals. The benchmark spans several industry domains, including AI Technology & Digital Media, Healthcare & Telehealth, Climate & Sustainabilty and Finance. It is designed so that successful generation requires retrieving and combining evidence from heterogeneous knowledge bases rather than relying on a single semantically aligned source. Using this dataset, we evaluate multiple opensource and proprietary LLM-based systems under standard RAG configurations and find that they perform poorly on enterprise document generation despite producing fluent outputs. Our analysis shows that the generated enterprise document is sensitive to the knowledge bases represented in retrieved context. When retrieval is performed through global ranking alone, it often leads to drafts that fail to cover the full set of requirements specified by the requested enterprise document. Our contributions are threefold: (1) Problem formulation. We identify enterprise document generation as a retrieval setting in which standard RAG assumptions are poorly matched to task requirements, and characterize a key failure mode of existing systems: global ranking across heterogeneous knowledge bases, which fails to preserve the source composition needed for complete document generation. (2) Method. We propose a source-aware re-

trieval framework for enterprise document generation, based on weighted local retrieval and supported by expert-authored ontological guidance, to improve cross-domain evidence selection and context composition. (3) Dataset and findings. We introduce a new dataset for retrieval-grounded enterprise document generation across multiple document types and domains, and show that existing RAG algorithms underperform on this dataset, while our source-aware retrieval approach yields substantial improvements.

2

Related Works

Retrieval-Augmented Generation RetrievalAugmented Generation (RAG) has become a cornerstone of knowledge-grounded language modeling by integrating external retrieval with generative reasoning. The original RAG framework by Lewis et al. (Lewis et al., 2021) and subsequent models such as REALM (Guu et al., 2020; Izacard and Grave, 2021), and RETRO (Borgeaud et al., 2022) showed that retrieving textual evidence before generation improves factuality and reduces hallucination. More recent works (Gao et al., 2024) extend these paradigms by improving retriever–generator interaction through adaptive evidence routing (Zheng et al., 2024; Li et al., 2026a), and reinforcement learning from human feedback (Zhang et al., 2025; Li et al., 2025). Despite these advances, most RAG pipelines still assume homogeneous and open-domain corpora, relying on dense or hybrid similarity retrieval to select support documents. Such an assumption becomes problematic in enterprise or multi-source environments, where repositories differ substantially in authority, structure, and linguistic domain (Bruckhaus, 2024; Arrieta et al., 2019). Multi-Source and Structured Retrieval A growing body of work has explored retrieval beyond single-source settings. Multi-hop and multigranularity retrieval methods (Yang et al., 2018; Tang and Yang, 2024; Lin et al., 2025) aim to combine evidence across documents, yet they generally index homogeneous, task-specific datasets such as Wikipedia or news archives. Several studies on domain-specific or vertical RAG systems have sought to address this heterogeneity. Works in legal (Kabir et al., 2025; Reuter et al., 2025; Butler and Butler, 2026), biomedical (Du et al., 2024; Öğdü et al., 2025; Matsumoto et al., 2024), and financial domains (George et al., 2025; Wang

et al., 2025a,b) introduce hierarchical or ontologybased retrieval pipelines to better reflect domain semantics. However, these approaches focus on accuracy within one tightly scoped corpus. Enterprise applications, however, often require synthesizing information dispersed across knowledge bases that play complementary roles. Empirical evaluations from industrial deployments (Bruckhaus, 2024; Jayarama Nettar, 2024) confirm that global top-k retrieval leads to skewed evidence selection, where large or lexically dominant sources overshadow smaller yet essential ones. Datasets for Knowledge-Grounded Generation Benchmark datasets for RAG typically target question answering (Karpukhin et al., 2020; Kwiatkowski et al., 2019; Joshi et al., 2017; Rajpurkar et al., 2016; Bajaj et al., 2018), fact verification (Sorodoc et al., 2025), or short-form summarization (Berant et al., 2013). Multi-document reasoning datasets like HotpotQA (Yang et al., 2018) encourage evidence chaining but still draw from single knowledge domains. Recent datasets for long-form factual generation (Zhao et al., 2024), safety policy writing (Hadfield and Clark, 2026), and organizational compliance (Jadad-Garcia and Jadad, 2024) indicate emerging interest but lack heterogeneous knowledge-source coverage. Our introduced dataset extends this direction by emphasizing cross-domain evidence composition, serving as a benchmark for evaluating retrieval-weighted RAG methods in enterprise contexts.

3

Method

3.1

Overview

W-RAG is a source-aware retrieval framework for enterprise document generation over heterogeneous knowledge bases (KBs). We use the term context allocation to refer to the token-level distribution of the final prompt context across KBs. Given an input query or seed document q, a total retrieval budget Ttotal , and M KBs, the goal is to construct a context window by selecting text chunks from each KB such that the total number of retrieved tokens does not exceed Ttotal while the retrieved evidence remains relevant to the target drafting task. At a high level, W-RAG proceeds in three steps: (1) it extracts ontology-grounded topics from the input and retrieves candidate chunks from each KB, (2) it estimates and recommends how much of the final context should come from each KB, and (3) it converts these source preferences into

token budgets and builds a single source-balanced context window for generation. 3.2

Stage 1: Ontology-Guided Candidate Retrieval

W-RAG first extracts a structured topic representation of the input using an LLM guided by a domain ontology. An ontology is a formal representation of the key concepts in a domain and the relations among them. Unlike a taxonomy, an ontology allows richer, non-hierarchical relations between entities, concepts, and categories. In our system, the ontology is used both to organize domain concepts and to map free-form input into a structured topic set that can guide retrieval. For example, in a policy drafting task, extracted topics such as data privacy, access control, and compliance reporting may be mapped to ontology categories such as security and governance. Details of ontology construction and topic-extraction prompts are provided in A.2. Given input q, the topic extractor produces T = {(ti , gi )}N i=1 , where ti is an extracted topic phrase and gi is its ontology category. Each KB j is indexed as a collection of text chunks (e.g., paragraph-level segments). We denote the k-th chunk from KB j by cjk . At indexing time, each chunk is also associated with a set of extracted ontology topics, denoted by M (cjk ). WRAG retrieves candidate chunks from each KB and scores them using a hybrid relevance function: R(cjk ) = fsim (q, cjk ) + λ font (T, cjk ),

(1)

where fsim measures semantic similarity between the input and the chunk, font measures ontologybased topic alignment, and λ ≥ 0 controls the relative influence of ontology guidance. Semantic similarity is computed as cosine similarity between dense embeddings:
fsim (q, cjk ) = cos ϕ(q), ϕ(cjk ) ,

(2)

where ϕ(·) is the embedding function used by the retriever. To compute ontology alignment, we compare each input topic against the ontology topics associated with the chunk. For an input topic ti and indexed chunk topic m ∈ M (cjk ), we use the rule-

based match score   1.0 if ti exactly matches m,    0.7 if one string contains the other, s(ti , m) =  0.3 only ontology category match,    0 otherwise. These values were selected to enforce a clear ordering of match strength (exact partial categorical none) while keeping the scoring scheme simple and interpretable. Thus, chunk-level ontology score is : N

font (T, cjk ) =

1 X max s(ti , m). N m∈M (cjk )

(3)

i=1

Thus, font is the average best-match score between the input topics and the chunk’s indexed ontology topics. In practice, chunks that mention more of the input topics—or mention them more specifically— receive higher ontology scores. In all experiments, we set λ = 0.5. We selected this value on a held-out development set by sweeping λ ∈ [0, 1] and choosing the value that improved average requirement satisfaction without degrading context relevancy. The same value is used across all domains and document types. For each KB j, we sort chunks by R(cjk ) and retain the top-m candidates: Cj = {cj1 , cj2 , . . . , cjm },

(4)

where Cj denotes the top-m candidate text chunks retrieved from KB j under the hybrid score in Eq. 1. These per-KB candidate sets are then used for source weighting and final context construction. 3.3

Stage 2: Source Weight Estimation

Stage 1 identifies thematically relevant candidates, but it does not yet determine how much each KB should contribute to the final prompt. A standard global top-k strategy would simply merge all candidates into one shared ranking, which can overconcentrate the final context on a single source. W-RAG instead estimates a source weight for each KB before constructing the final context. For each KB j, we maintain a KB-level topic index Mj , obtained by aggregating the ontology topics associated with documents in that KB. Source weighting is computed in real time by comparing the input topic set T against these KB topic indexes. The main source signal is topical overlap:

where the same exact / partial / category-only match function s(·, ·) is used as in Stage 1. Thus, j Stopic ∈ [0, 1] measures how well KB j matches the input topics overall. j We combine Stopic with two auxiliary terms. The first is a weak document-availability prior: ! j Ndoc j Sdoc = min 1, × 0.5 , 100 j where Ndoc is the number of indexed documents in KB j. This term gives a small boost to KBs with broader supporting coverage while preventing size alone from dominating the score. The second is a category bonus: ( 0.2 KB j matches at least 1 input topic, j Scat = 0 otherwise.

This term rewards coarse domain alignment even when exact topic overlap is limited. The final source score for KB j is j j j j Sfinal = 0.7Stopic + 0.1Sdoc + 0.2Scat .

The coefficients reflect the intended priority of the three signals: topical overlap is the primary driver, category alignment provides domain-level support, and document availability acts only as a weak prior. We fixed these coefficients after preliminary development-stage tuning and kept them constant across all experiments. The resulting scores are normalized into percentage-based KB proportions (termed as weights):

1 X max s(ti , m), m∈Mj N i=1

(5)

M X

Sj ×100, wj = PM final k k=1 Sfinal

wj = 100. (7)

j=1

This procedure is computed in real time: the recommender does not rely on static precomputed KB preferences, but instead compares the current input topics against the current topic indexes of the KBs. When new documents are added to a KB, they affect future recommendations only after re-indexing updates the associated topic index and document counts. 3.4

Stage 3: Weighted Context Construction

Since the generator has a fixed context window, the KB weights in Eq. 7 must be converted into token budgets. For each KB j, we define

N

j Stopic =

(6)

Tj =

wj Ttotal , 100

M X j=1

Tj = Ttotal .

(8)

Weights are defined as the proportional allocation of context tokens across knowledge bases based on their estimated relevance to the query. These budgets specify how many tokens each KB is allowed to contribute to the final prompt context. W-RAG then selects the highest-ranked chunks from each locally ranked candidate set Cj until the corresponding budget Tj is exhausted. The selected chunks are merged into a Single Context Window (SCW), and each chunk is annotated with its source KB to preserve provenance. This step is the mechanism that enforces source-level constraints on the final context window, which standard RAG pipelines do not control. In this way, W-RAG transforms source relevance estimates into a generation context whose composition reflects the informational needs of the target document rather than the dominance of a single source. The final prompt is formed as P = {Isys , q, SCW}, where Isys denotes system instructions. This prompt is then passed to the LLM for enterprise document generation.

4

Dataset

Evaluating Retrieval-Augmented Generation (RAG) for enterprise-level document generation presents a unique challenge, as no standardized dataset currently exists for long-form, requirementdriven document synthesis. To address this gap, we introduce a novel dataset designed to evaluate RAG techniques for enterprise document generation under heterogeneous knowledge base conditions. Each data sample consists of an input requirement document that specifies detailed constraints and objectives for generating a target document. A sample of an input document is shown in the A.1 The dataset contains 100 requirement input documents. Each requirement document was manually constructed and belongs to one of four enterprise document types: Policies, Product launch documents, Academic program documents & Merger and acquisitions due diligence documents. These document types were selected because they represent distinct enterprise requirements with different evidence needs, rhetorical structures, and decisionmaking purposes. Details on the how the input documents were created and authored can be found in A.1.

Across these document types, the dataset covers four high-level topical themes: AI Technology & Digital Media, Healthcare & Telehealth, Climate & Sustainabilty and Finance. These themes were chosen to ensure that the benchmark reflects realistic enterprise themes in which document generation must integrate business, technical, legal, and market-specific information. To support retrieval, we construct four heterogeneous knowledge bases (KBs), each containing approximately 500 publicly sourced documents collected from open-access web resources: (1) KB1 (Corporate): Corporate filings, company reports, operational documentation, and governance materials. (U.S. Securities and Exchange Commission, 2026a; Alpha Vantage, 2026) (2) KB2 (Scientific): Peer-reviewed journal articles, technical whitepapers, and research summaries. (arXiv, 2026; National Center for Biotechnology Information, 2026) (3) KB3 (Regulatory): Legal texts, compliance guidelines, and regulatory frameworks. (U.S. Food and Drug Administration, 2026; U.S. Securities and Exchange Commission, 2026b) (4) KB4 (Market & News): Market surveys, industry reports, and news articles. (Forbes Media LLC, 2026) The goal of the using such knowledge bases is to test RAG systems’ ability to assemble a context that reflects the different functional roles of enterprise evidence. For instance, a product launch document in healthcare AI may need corporate precedent, scientific validation, regulatory requirements, and market positioning simultaneously.

5

Experiments

5.1

Baselines & Metrics

Baselines We compare W-RAG against the following representative RAG pipelines: (1) Vanilla RAG. The standard RAG pipeline (Lewis et al., 2021) that performs dense semantic retrieval over all available knowledge bases (KBs), followed by a global ranking of retrieved documents across KBs. The top-ranked passages are then provided to the generator without any explicit modeling of domain structure or source-level preferences. (2) OG-RAG. The second baseline augments retrieval with ontology-guided topic modeling (Sharma et al., 2025). For each of the four domains, an ontology was manually curated cap-

turing the major concepts, entities, and thematic relations relevant to document drafting in that domain. Retrieved documents are first filtered and organized using this ontology-guided topic representation, after which passages are globally ranked across KBs and passed to the generator. OG-RAG was shown to demonstrate higher recall of accurate facts and response correctness compared to matrix (like RAPTOR (Sarthi et al., 2024)) and graph (like GraphRAG (Edge et al., 2025)) based RAG baselines, establishing ontology-guided retrieval as a strong foundation. Ontologies We construct one ontology for each domain–document-type pair, resulting in 16 ontologies overall. Each ontology is built using a semi-automated process, where an initial automatically generated version is manually refined and verified by domain experts. These ontologies support topic extraction and ontology-guided retrieval in OG-RAG and W-RAG. Evaluation Metrics Due to the multidimensional nature of enterprise document generation, we employ the following complementary evaluation metrics: (1) Requirement Satisfaction is defined as the extent to which a generated document satisfies the minimum essential requirements outlined in the input document. Unlike QA benchmarks, where correctness can often be judged against a single reference answer, enterprise documents are better evaluated against a requirement checklist specifying the mandatory elements that must appear in a valid draft. For each document type (e.g., policy documents, product launch briefs, M&A due diligence summaries), we construct a checklist of essential requirements in consultation with domain practitioners who routinely author such documents and each checklist contains approximately 12 core requirements. Each requirement is scored on a 2-point ordinal scale: 0: Not satisfied, 1: Partially satisfied / insufficiently specified & 2: Fully satisfied. This results in a maximum score of 24 points per document. The score is then normalized. We semiautomate the scoring process by using an LLM-asa-judge that is provided with: (1) The input document or drafting prompt, (2) The generated output and (3) The requirement checklist for the corresponding document type.

The judge is instructed to assign a score for each checklist item. To improve reliability, these scores are then manually verified by domain practitioners before being used in analysis. The standard instructions given to domain practioners to validate the scoring is provided in A.7. (2) Context Relevancy measures the proportion of retrieved passages that are genuinely relevant to the input drafting task. Given a set of N retrieved passages {pi }N i=1 , an LLM judge labels each passage as relevant or nonrelevant with respect to the input document and drafting objective. Let zi ∈ {0, 1} denote this judgment. The Context Relevancy is defined as: N

1 X zi CTS = N i=1

As with Answer Relevancy, LLM-based judgments are manually validated to ensure scoring consistency. Context Retrieval as a metric isolates retrieval quality from generation quality: a system may generate fluent text, but poor retrieval will often limit factual adequacy and source grounding. (3) Realized Knowledge Base Fidelity The goal of this metric is to estimate which knowledge base most likely supports different parts of the generated document, rather than to detect verbatim reuse of retrieved passages. Because modern LLMs typically synthesize and paraphrase retrieved evidence, attribution is performed using semantic similarity in embedding space rather than lexical overlap. Consequently, the metric measures distributional grounding in knowledge source. The fidelity is calculated in the following manner Let the generated document be segmented into sentence-level units: S = {s1 , s2 , . . . , sT }. For each segment st , we compute its embedding and compare it against the embeddings of all retrieved document chunks. The chunk with the highest cosine similarity is selected: c∗ (st ) = arg max cos(ϕ(st ), ϕ(c)), c∈C

where ϕ(·) denotes the embedding function and C is the set of retrieved chunks. If the highest similarity falls below a threshold τ (= 0.6 (Es et al., 2024)), the segment is labeled

synthesized, indicating that it cannot be confidently grounded in any retrieved source. This allows the metric to explicitly account for content generated through model synthesis rather than direct source support. Each segment is then assigned to the KB of its matched chunk. Let wi denote the total number of words in segments attributed to KB i, and let wsynth denote the total number of words labeled synthesized. The realized distribution over KB usage is:

For automatic evaluation of Requirement Satisfaction and Context Relevancy, we use Claude Sonnet 4.6 (Anthropic, 2026) as the LLM-as-ajudge; these judgments are manually validated before analysis. All prompts for the LLMs, can be found in A.3 5.2

Results

JSD(P ′ ∥ Q′ ) , ln 2 where 1 indicates perfect source fidelity and 0 maximal divergence.

Generated Document Quality and Retrieval Effectiveness Before analyzing performance, we note that the LLM-as-a-judge used for scoring was manually validated and showed strong agreement with human verification, with an accuracy of 91% for Requirement Satisfaction and 86% for Context Relevancy. This suggests that the automatic scoring is reliable enough for comparative evaluation. Details on instructions for manual verification of scores is outlined in A.7 Table 1 reports two complementary metrics: Requirement Satisfaction, which measures how much of the required content is covered in the generated enterprise document, and Context Relevancy, which measures how much of the retrieved evidence is relevant to the drafting task. Across all 16 settings, W-RAG yields substantially larger improvements: relative to OG-RAG, it improves Requirement Satisfaction by 58.1% and Context Relevancy by 39.1%; relative to Vanilla RAG, the gains are 109.2% and 96.4%, respectively. In practical terms, this means that W-RAG not only retrieves better supporting evidence, but also converts that evidence into drafts that satisfy far more of the required elements mentioned the input document. A key pattern observed is that retrieval quality alone does not guarantee better document quality. For example, in AI Technology & Digital Media product launch documents, OG-RAG improves Context Relevancy by 64.4% over Vanilla RAG, but Requirement Satisfaction increases by only 11.8%. W-RAG closes this gap, indicating that enterprise document generation depends not only on retrieving relevant passages, but also on composing evidence from multiple KBs in a sourceaware manner. A.5 details a fine grained analysis of the results.

Large Language Models To ensure that observed differences are attributable to retrieval rather than generation, all retrieval-generation pipelines were run using the same backbone model, GPT4.1 (OpenAI, 2025), accessed through the Azure AI Search (Microsoft Corporation, 2026).

Knowledge Base Influence & Source Fidelity Figure 1 provides a domain-wise comparison between the prescribed source distribution at retrieval time and the realized KB attribution in the generated document. The degree to which the realized KB distribution tracks the input KB distribution

Qi = PK

wi

j=1 wj + wsynth

andQsynth = PK

for i = 1, . . . , K wsynth

j=1 wj + wsynth

.

This yields a fine-grained estimate of how much of the generated document is actually supported by each KB. (4) Jensen–Shannon Divergence To quantify how closely realized KB usage matches prescribed weights, we compute the JS Divergence (JSD) (Li et al., 2026b) between distributions P ′ and Q′ : P ′ = [P1 , . . . , PK , 0] Q′ = [Q1 , . . . , QK , Qsynth ], where the extra dimension accounts for synthesized content (PK = 0). JSD is defined as 1 1 JSD(P ′ ∥ Q′ ) = KL(P ′ ∥ M )+ KL(Q′ ∥ M ), 2 2 1 M = (P ′ + Q′ ), 2 KL denoting Kullback–Leibler divergence. Bounded in [0, ln 2], we get a fidelity score in [0, 1]: where

Fidelity = 1 −

Document Type Policies

Product Launch

Merger & Acquisitions

Academic Programs

RAG Algorithm Vanilla RAG OG-RAG W-RAG Vanilla RAG OG-RAG W-RAG Vanilla RAG OG-RAG W-RAG Vanilla RAG OG-RAG W-RAG

AI Technology & Digital Media Requirement Context Satisfaction (%) Relevancy(%) 25 57 34 73 78 76 34 45 38 74 58 78 38 52 39 70 65 78 28 31 34 64 72 81

Climate & Sustainability Requirement Context Satisfaction(%) Relevancy(%) 38 32 68 72 76 70 35 40 56 50 74 68 42 38 54 49 83 87 40 45 40 44 78 82

Healthcare & Telehealth Requirement Context Satisfaction(%) Relevancy(%) 25 30 35 40 55 70 22 28 33 45 63 71 20 32 32 41 62 68 27 22 29 42 67 74

Finance Requirement Context Satisfaction(%) Relevancy(%) 34 41 51 48 60 79 37 40 5 52 61 79 41 45 5 56 73 82 34 40 45 52 63 7

Table 1: Comparison of RAG algorithms across domains using Requirement Satisfaction and Context Relevancy

Figure 1: Knowledge-base influence across domains. Solid lines denote prescribed KB weights at retrieval time, and dotted lines denote realized KB attribution in the generated output. Colors correspond to document types. Blue : Policies, Orange : Product Launch, Green : M&A, Yellow: Academic Programs. Weights (Y-axis) range 0 to 50%

serves as a direct visual indicator of source fidelity: when a KB’s weight specified prior to retrieval is higher (i.e., more information is being retrieved from that KB than the others), the generated document reflects this intended source composition faithfully. The close alignment between the input and realized distributions across all domains suggests that the system maintains a strong linkage between retrieval-time source weighting and the resulting document content. Consistent with the visual trends, the average JS divergence scores across domains range from 0.60 to 0.74, indicating moderate-to-strong agreement between prescribed and realized KB usage. This suggests that the generation process preserves the intended mixture of corporate, scientific, regulatory, and market knowledge when constructing enterprise documents. To move beyond visual evidence alone, Section A.5.1 presents a controlled weight-override study in which prescribed KB mixtures are manually perturbed. Use Case: Corporate Proposal Drafting. A motivating enterprise use case for our setting is corporate proposal drafting, where proposal managers, solution architects, and business development teams prepare responses to complex RFPs for U.S. civilian and DoD agencies. These documents are highly structured, and heavily dependent on retrieving evidence from heterogeneous

internal knowledge bases, including past proposals, contracts, capability decks, compliance documents, pricing artifacts, and agency-specific materials. In an internal retrieval-assisted drafting workflow used from 2025–2026, teams reported substantial productivity benefits, suggesting that proposal drafting is an important real-world testbed for multi-KB document generation. We provide a detailed case-study description in Appendix A.6.

6

Conclusion

This work introduced W-RAG, a retrieval framework designed for enterprise document generation tasks that require evidence from multiple heterogeneous knowledge bases. Our experiments show that global similarity ranking can lead to imbalanced context composition, where dominant sources overshadow others that are necessary for satisfying complex requirements. By combining ontologyguided topic extraction with source-aware context allocation, W-RAG improves both requirement satisfaction and context relevance on our benchmark dataset. These results suggest that, in multisource enterprise settings, explicitly managing how retrieved context is distributed across knowledge bases can lead to more faithful and complete document generation.

7

Limitations and Threats to Validity

Dataset and Task Scope. Our evaluation is conducted on a relatively small, synthetic enterprisestyle dataset. While designed to reflect realistic multi-source document generation scenarios, it may not fully capture the diversity, scale, and noise present in real-world enterprise environments. Evaluation Metrics. We evaluate generated outputs using task-specific metrics such as requirement satisfaction and context relevancy, which involve partial subjectivity in their definition or measurement. These metrics may not fully capture all aspects of document quality, such as factual correctness, coherence, or usability in downstream workflows.

Jonathan Berant, Andrew Chou, Roy Frostig, and Percy Liang. 2013. Semantic parsing on Freebase from question-answer pairs. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1533–1544, Seattle, Washington, USA. Association for Computational Linguistics. Sebastian Borgeaud, Arthur Mensch, Jordan Hoffmann, Trevor Cai, Eliza Rutherford, Katie Millican, George van den Driessche, Jean-Baptiste Lespiau, Bogdan Damoc, Aidan Clark, Diego de Las Casas, Aurelia Guy, Jacob Menick, Roman Ring, Tom Hennigan, Saffron Huang, Loren Maggiore, Chris Jones, Albin Cassirer, and 9 others. 2022. Improving language models by retrieving from trillions of tokens. Preprint, arXiv:2112.04426. Tilmann Bruckhaus. 2024. Rag does not work for enterprises. Preprint, arXiv:2406.04369.

Baseline Comparisons. Our comparisons are limited to standard retrieval-augmented generation baselines. We do not include heavily tuned retrieval strategies, which may reduce the observed performance gap.

Erik Brynjolfsson, Danielle Li, and Lindsey Raymond. 2024. Generative ai at work. Preprint, arXiv:2304.11771.

Generality of the Approach. W-RAG is evaluated primarily in an enterprise document generation setting with structured knowledge sources. Its effectiveness in other domains—such as opendomain question answering or less structured retrieval environments—remains to be validated.

Jiawei Chen, Hongyu Lin, Xianpei Han, and Le Sun. 2024. Benchmarking large language models in retrieval-augmented generation. Proceedings of the AAAI Conference on Artificial Intelligence, 38(16):17754–17762.

References Alpha Vantage. 2026. Alpha Vantage. https://www. alphavantage.co/. Anthropic. 2026. Claude Sonnet 4.6 Model Overview. https://www.anthropic.com/claude/sonnet. Alejandro Barredo Arrieta, Natalia Díaz-Rodríguez, Javier Del Ser, Adrien Bennetot, Siham Tabik, Alberto Barbado, Salvador García, Sergio Gil-López, Daniel Molina, Richard Benjamins, Raja Chatila, and Francisco Herrera. 2019. Explainable artificial intelligence (xai): Concepts, taxonomies, opportunities and challenges toward responsible ai. Preprint, arXiv:1910.10045. arXiv. 2026. arXiv.org e-Print Archive. https:// arxiv.org/. Payal Bajaj, Daniel Campos, Nick Craswell, Li Deng, Jianfeng Gao, Xiaodong Liu, Rangan Majumder, Andrew McNamara, Bhaskar Mitra, Tri Nguyen, Mir Rosenberg, Xia Song, Alina Stoica, Saurabh Tiwary, and Tong Wang. 2018. Ms marco: A human generated machine reading comprehension dataset. Preprint, arXiv:1611.09268.

Abdur-Rahman Butler and Umar Butler. 2026. Legal rag bench: an end-to-end benchmark for legal rag. Preprint, arXiv:2603.01710.

Yongping Du, Zikai Wang, Binrui Wang, Xingnan Jin, and Yu Pei. 2024. A novel rag framework with knowledge-enhancement for biomedical question answering. In 2024 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), pages 3188–3191. Darren Edge, Ha Trinh, Newman Cheng, Joshua Bradley, Alex Chao, Apurva Mody, Steven Truitt, Dasha Metropolitansky, Robert Osazuwa Ness, and Jonathan Larson. 2025. From local to global: A graph rag approach to query-focused summarization. Preprint, arXiv:2404.16130. Shahul Es, Jithin James, Luis Espinosa Anke, and Steven Schockaert. 2024. RAGAs: Automated evaluation of retrieval augmented generation. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations, pages 150–158, St. Julians, Malta. Association for Computational Linguistics. Forbes Media LLC. 2026. Forbes: Business, Technology, and Finance News. https://www.forbes. com/. Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yi Dai, Jiawei Sun, Meng Wang, and Haofen Wang. 2024. Retrieval-augmented generation for large language models: A survey. Preprint, arXiv:2312.10997.

Ryan George, Akshay Govind Srinivasan, Jayden Koshy Joe, Harshith M R, Vijayavallabh J, Hrushikesh Kant, Rahul Vimalkanth, Sachin S, and Sudharshan Suresh. 2025. Enhancing financial RAG with agentic AI and multi-HyDE: A novel approach to knowledge retrieval and hallucination reduction. In Proceedings of The 10th Workshop on Financial Technology and Natural Language Processing, pages 19–32, Suzhou, China. Association for Computational Linguistics. Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Ming-Wei Chang. 2020. Realm: Retrievalaugmented language model pre-training. Preprint, arXiv:2002.08909. Gillian K. Hadfield and Jack Clark. 2026. Regulatory markets: The future of ai governance. Preprint, arXiv:2304.04914. Gautier Izacard and Edouard Grave. 2021. Leveraging passage retrieval with generative models for open domain question answering. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 874–880, Online. Association for Computational Linguistics. Tamen Jadad-Garcia and Alejandro R. Jadad. 2024. The foundations of computational management: A systematic approach to task automation for the integration of artificial intelligence into existing workflows. Preprint, arXiv:2402.05142. Infosys Jayarama Nettar, Harry Keir Hughes. 2024. The challenge of information retrieval in enterprise ai. Mandar Joshi, Eunsol Choi, Daniel S. Weld, and Luke Zettlemoyer. 2017. Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension. Preprint, arXiv:1705.03551. Muhammad Rafsan Kabir, Rafeed Mohammad Sultan, Fuad Rahman, Mohammad Ruhul Amin, Sifat Momen, Nabeel Mohammed, and Shafin Rahman. 2025. Legalrag: A hybrid rag system for multilingual legal information retrieval. Preprint, arXiv:2504.16121. Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. Dense passage retrieval for opendomain question answering. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6769–6781, Online. Association for Computational Linguistics. Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming-Wei Chang, Andrew M. Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov. 2019. Natural questions: A benchmark for question answering research. Transactions of the Association for Computational Linguistics, 7:452–466.

Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2021. Retrieval-augmented generation for knowledgeintensive nlp tasks. Preprint, arXiv:2005.11401. Minghan Li, Eric Gaussier, Juntao Li, and Guodong Zhou. 2026a. Evirerank: Adaptive evidence construction for long-document llm reranking. Preprint, arXiv:2411.06254. Ruizhe Li, Chen Chen, Yuchen Hu, Yanjun Gao, Xi Wang, and Emine Yilmaz. 2026b. Attributing response to context: A jensen-shannon divergence driven mechanistic study of context attribution in retrieval-augmented generation. Preprint, arXiv:2505.16415. Yuan Li, Qi Luo, Xiaonan Li, Bufan Li, Qinyuan Cheng, Bo Wang, Yining Zheng, Yuxin Wang, Zhangyue Yin, and Xipeng Qiu. 2025. R3-rag: Learning step-bystep reasoning and retrieval for llms via reinforcement learning. Preprint, arXiv:2505.23794. Jiaen Lin, Jingyu Liu, and Yingbo Liu. 2025. Optimizing multi-hop document retrieval through intermediate representations. In Findings of the Association for Computational Linguistics: ACL 2025, pages 15798–15809, Vienna, Austria. Association for Computational Linguistics. Nicholas Matsumoto, Jay Moran, Hyunjun Choi, Miguel E. Hernandez, Mythreye Venkatesan, Zhiping Paul Wang, and Jason H. Moore. 2024. Kragen: a knowledge graph-enhanced rag framework for biomedical problem solving using large language models. Bioinformatics, 40. Microsoft Corporation. 2026. Azure AI Search Documentation. https://docs.azure.cn/en-us/ search/. National Center for Biotechnology Information. 2026. NCBI: National Library of Medicine Databases. https://www.ncbi.nlm.nih.gov/. OpenAI. 2025. GPT-4.1: OpenAI Model Overview. https://openai.com/index/gpt-4-1/. Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. SQuAD: 100,000+ questions for machine comprehension of text. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 2383–2392, Austin, Texas. Association for Computational Linguistics. Markus Reuter, Tobias Lingenberg, Rūta Liepin, a, Francesca Lagioia, Marco Lippi, Giovanni Sartor, Andrea Passerini, and Burcu Sayin. 2025. Towards reliable retrieval in rag systems for large legal datasets. Preprint, arXiv:2510.06999. David Richards. 2025. The engineering gap:why 73% of enterprise rag systems fail where they matter most.

Parth Sarthi, Salman Abdullah, Aditi Tuli, Shubh Khanna, Anna Goldie, and Christopher D. Manning. 2024. Raptor: Recursive abstractive processing for tree-organized retrieval. Preprint, arXiv:2401.18059. Kartik Sharma, Peeyush Kumar, and Yunqing Li. 2025. OG-RAG: Ontology-grounded retrieval-augmented generation for large language models. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 32962–32981, Suzhou, China. Association for Computational Linguistics. Ionut-Teodor Sorodoc, Leonardo F. R. Ribeiro, Rexhina Blloshmi, Christopher Davis, and Adrià de Gispert. 2025. Garage: A benchmark with grounding annotations for rag evaluation. Preprint, arXiv:2506.07671. Yixuan Tang and Yi Yang. 2024. Multihop-rag: Benchmarking retrieval-augmented generation for multihop queries. Preprint, arXiv:2401.15391. Adam Trischler, Tong Wang, Xingdi Yuan, Justin Harris, Alessandro Sordoni, Philip Bachman, and Kaheer Suleman. 2017. NewsQA: A machine comprehension dataset. In Proceedings of the 2nd Workshop on Representation Learning for NLP, pages 191–200, Vancouver, Canada. Association for Computational Linguistics. U.S. Food and Drug Administration. 2026. FDA.gov: Official U.S. Food and Drug Administration Website. U.S. Securities and Exchange Commission. 2026a. EDGAR: SEC Search Filings Database. https: //www.sec.gov/search-filings. U.S. Securities and Exchange Commission. 2026b. SEC.gov | Home. https://www.sec.gov/. Feng Wang, Yiding Sun, Jiaxin Mao, Xue Wei, and Danqing Xu. 2025a. Fins-pilot: A benchmark for online financial rag system. In Proceedings of the 34th ACM International Conference on Information and Knowledge Management, CIKM ’25, page 6544–6548, New York, NY, USA. Association for Computing Machinery. Jingru Wang, Wen Ding, and Xiaotong Zhu. 2025b. Financial analysis: Intelligent financial data analysis system based on llm-rag. Preprint, arXiv:2504.06279. Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William W. Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. 2018. Hotpotqa: A dataset for diverse, explainable multi-hop question answering. Preprint, arXiv:1809.09600. Hanning Zhang, Juntong Song, Juno Zhu, Yuanhao Wu, Tong Zhang, and Cheng Niu. 2025. Opengenalign: A preference dataset and benchmark for trustworthy reward modeling in open-ended, long-context generation. Preprint, arXiv:2501.13264.

Qingfei Zhao, Ruobing Wang, Yukuo Cen, Daren Zha, Shicheng Tan, Yuxiao Dong, and Jie Tang. 2024. Longrag: A dual-perspective retrieval-augmented generation paradigm for long-context question answering. Preprint, arXiv:2410.18050. Yuanhang Zheng, Peng Li, Wei Liu, Yang Liu, Jian Luan, and Bin Wang. 2024. Toolrerank: Adaptive and hierarchy-aware reranking for tool retrieval. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation, LREC/COLING 2024, 20-25 May, 2024, Torino, Italy, pages 16263–16273. ELRA and ICCL. Çağatay Umut Öğdü, Kübra Arslanoğlu, and Mehmet Karaköse. 2025. An adaptive multi-agent llmbased clinical decision support system integrating biomedical rag and web intelligence. IEEE Access, 13:167390–167404.

A

Appendix

A.1

Dataset Details

A.1.1 Input Requirement Documents Each sample in our dataset begins with an input requirement document, which serves as the drafting specification for the target enterprise document. Unlike short-form prompts or questionanswer pairs, these inputs are structured requirement artifacts that describe the organizational setting, business context, constraints, intended audience, and mandatory sections that must appear in the final output. In practice, they are designed to resemble the kinds of requirement documentation used in real enterprise drafting workflows, where a writer or policy owner typically begins from a formal brief, scope document, governance note, or internal request rather than from a single naturallanguage query. For each of the four document types—Policies, Product Launch documents, M&A due diligence reports, and Academic Program proposals—we created requirement documents that specify both the situational context and the expected structural elements of the final draft. Document design was informed by common enterprise writing patterns, public institutional documentation, and the professional conventions associated with each document type. The goal was to create representative requirement specifications that capture the kinds of constraints, cross-functional evidence needs, and section-level expectations that arise in real-world enterprise authoring tasks. Figure 2 shows a sample input document for a corporate policy in the AI Technology Domain.

A.2 Ontology Construction, Topic Extraction, and Prompting

influence retrieval and source-weight estimation.

For the ontology-aware methods, we construct one ontology for each domain–document-type pair, yielding 16 ontologies in total. Each ontology is built using a semi-automated procedure: an initial draft ontology is generated automatically from representative domain material and sample requirement documents, and is then manually reviewed, refined, and verified by domain experts. This process ensures that the final ontology captures the concepts, categories, and semantic relations most relevant to the corresponding enterprise document type. An ontology in our setting is a formal representation of domain concepts and their relationships. Unlike a taxonomy or flat classification, it allows richer semantic organization by grouping related concepts under broader functional categories. For example, in a policy drafting ontology, concepts such as data privacy, access control, and audit logging may be grouped under categories such as security, governance, and compliance. The ontology is used in two places. First, it guides topic extraction from the input document, producing a structured topic representation of the drafting request. Second, it supports ontology-aware retrieval by enabling comparison between the input topics and the topics indexed from knowledge-base documents. To support this, each document added to a KB is processed through the same topic extraction pipeline, and the resulting topics are stored as document metadata. Topic extraction is performed in two stages. In the first stage, the large language model identifies salient topic phrases from the document text. In the second stage, ontology content is supplied as additional context, so that extracted topics align more consistently with the ontology vocabulary and categories. The final output is mapped into the structured representation

A.3

T = {(ti , gi )}N i=1 , where ti is an extracted topic phrase and gi is its ontology category. When new documents are added to a KB, they do not affect retrieval immediately. They must first be re-indexed: topics are extracted from the new documents, the KB topic index is updated, and document counts and metadata are refreshed. Only after this process do newly added documents

A.3.1

LLM-as-a-Judge Requirements Satisfaction Scoring

For Requirement Satisfaction, the judge model is given three inputs: (1) the original requirement document, (2) the generated enterprise document, and (3) a document-type-specific requirement checklist prepared by a domain practitioner. The judge is asked to evaluate the generated document item by item against the checklist and assign one of three labels for each requirement: fully satisfied, partially satisfied, or not satisfied. These are then converted into numeric scores according to the scoring rubric. The checklist is designed to capture the minimum essential elements that every valid document of that type should contain, independent of domain-specific style or wording. In the policy setting, example checklist items include whether the draft clearly defines scope, identifies a governance structure, specifies roles and responsibilities, states risk controls and escalation paths, defines monitoring and compliance mechanisms, and specifies a review cadence. A sample checklist is included in Figure 2 and a prompt example shown in Fig. 3 A.3.2

Context Relevancy Scoring

For Context Relevancy, the judge model is provided with: (1) the input requirement document, (2) the retrieved passages used as generation context, and (3) instructions to label each retrieved passage as relevant or non-relevant to the drafting task. The purpose of this prompt (shown in Fig. 4) is to isolate retrieval quality from final generation quality. A passage is judged relevant if it contributes useful evidence toward satisfying the requested enterprise document requirements, even if it is not copied directly into the final output. This is particularly important in our setting, where retrieved evidence may support governance, compliance, technical safeguards, or organizational procedures in complementary ways.

Essential Item

Max Points

Scope clearly defined Governance structure identified Roles & responsibilities assigned Clinical/legal standards specified Risk controls / escalation paths Quality metrics defined Workflows & documentation rules Approved technology specified Training requirements Monitoring & compliance plan Policy review cadence

3 3 3 3 2 2 2 2 2 2 1

Total

25

Table 2: Sample requirement checklist used to score policy generation quality. Each item is scored as Full, Partial, or None, and then converted into a normalized Requirement Satisfaction score.

A.4

Emperical Evaluation Results Analysis

A.4.1

Fine-grained Analysis of Generated Document Quality and Retrieval Effectiveness

Table 1 reveals 2 robust patterns. First, W-RAG consistently outperforms both baselines on document quality, indicating that better retrieval alone is insufficient unless the retrieved evidence is also composed appropriately across KBs. Second, the magnitude of improvement is not uniform: document types that require broader synthesis across heterogeneous sources benefit the most from sourceaware retrieval. Policies. For Policies, compared with OG-RAG, W-RAG yields a 43.1% relative improvement in document quality and a 26.6% improvement in retrieval quality. The strongest domain-level gain appears in AI Technology & Digital Media, where R EQUIREMENT S ATISFACTION more than doubles relative to OG-RAG (34%→78%). This suggests that policy drafting in technically evolving domains depends heavily on retrieving the right mixture of regulatory, organizational, and operational evidence. By contrast, Climate & Sustainability presents an informative counterexample: OG-RAG slightly outperforms W-RAG on C ON TEXT R ELEVANCY (72% vs. 70%), yet W-RAG still achieves higher R EQUIREMENT S ATISFAC TION (76% vs. 68%). This gap indicates that, for policy drafting, marginal improvements in passagelevel relevance do not necessarily yield better documents unless the evidence is composed in a way that covers the required policy structure. Product Launch. For Product Launch documents, relative to OG-RAG, W-RAG observes

an improvement of 44.6% in document quality and 33.9% in retrieval quality. The Healthcare & Telehealth setting is particularly revealing: R EQUIREMENT S ATISFACTION rises from 33% to 63% and C ONTEXT R ELEVANCY from 45% to 71% when moving from OG-RAG to W-RAG. Product-launch drafting appears especially dependent on combining complementary evidence types—technical capabilities, market positioning, regulatory considerations, and organizational constraints—rather than merely retrieving thematically similar text. The relatively modest improvement of OG-RAG over vanilla RAG in R EQUIREMENT S ATISFACTION suggest that identifying more relevant passages is not enough unless the system also retrieves them in a enterpriseappropriate mixture. Merger & Acquisitions. Merger & Acquisitions is the strongest-performing document type under W-RAG and exhibits some of the largest margins over both baselines. Averaged across domains, vanilla RAG reaches 35.3% R EQUIREMENT S ATIS FACTION and 41.8% C ONTEXT R ELEVANCY; OGRAG improves to 43.8% and 54.0%; and W-RAG rises sharply to 70.8% and 78.8%. This translates into a 61.7% relative gain in R EQUIREMENT S ATISFACTION and a 45.8% gain in C ONTEXT R ELE VANCY over OG-RAG. The largest single improvement occurs in Climate & Sustainability, where W-RAG reaches 83% R EQUIREMENT S ATISFAC TION and 87% C ONTEXT R ELEVANCY, compared with 54% and 49% under OG-RAG. Academic Programs. Academic Programsprovides perhaps the clearest evidence that retrieval quality and document quality should be analyzed separately. Averaged across domains,

vanilla RAG reaches 32.3% R EQUIREMENT S ATISFACTION and 34.5% C ONTEXT R ELEVANCY ; OG-RAG improves these to 37.0% and 50.5%; and W-RAG produces a much stronger 70.0% and 76.8%. Compared with OG-RAG, W-RAG improves R EQUIREMENT S ATISFACTION by 89.2% and C ONTEXT R ELEVANCY by 52.0%. The most striking example is Climate & Sustainability, where OG-RAG shows no improvement at all over vanilla RAG in R EQUIREMENT S ATISFACTION (40% vs. 40%), yet W-RAG raises this score to 78%. This pattern suggests that academic-program drafting is highly dependent on assembling evidence into a coherent, requirement-satisfying structure, and that source-balanced retrieval is particularly important when the target document spans policy, curriculum, compliance, and market-facing rationale. Domain-level trends. Analysis across all document types reveals additional domain-specific structure. Healthcare & Telehealth is the most challenging domain overall: vanilla RAG averages only 23.5% R EQUIREMENT S ATISFACTION and 28.0% C ONTEXT R ELEVANCY, while OG-RAG improves modestly to 32.3% and 42.0%. W-RAG substantially reduces this gap, improving the domain average to 61.8% and 70.8%. Relative to OGRAG, this is a 91.5% gain in document quality and a 68.5% gain in retrieval quality, the largest relative improvement among all four domains. This suggests that healthcare drafting requires assembling fragmented and specialized evidence from multiple KBs and is therefore highly sensitive to source composition. In contrast, Climate & Sustainability attains the highest average R EQUIREMENT S ATISFACTION under W-RAG at 77.8%, followed by AI Technology & Digital Media at 68.3% and Finance at 64.3%. Notably, AI Technology & Digital Media shows the highest retrieval scores already under OG-RAG (70.3% average C ONTEXT R ELE VANCY), yet document quality remains much lower until weighting is introduced (36.3% under OGRAG vs. 68.3% under W-RAG). Retrieval quality versus document quality. A final observation is that C ONTEXT R ELEVANCY and R EQUIREMENT S ATISFACTION are correlated but not interchangeable. OG-RAG frequently improves C ONTEXT R ELEVANCY by a large margin over vanilla RAG, yet document quality often improves only slightly. For example, in AI Technology & Digital Media Product Launch, C ONTEXT R EL -

EVANCY increases from 29% (45% to 74% ) when moving from vanilla RAG to OG-RAG, whereas R EQUIREMENT S ATISFACTION rises only by 4% (34% to 38%). Similar patterns appear in Policies and Academic Programs. This divergence supports the main claim of the paper: in enterprise multi-KB drafting, retrieving relevant passages is necessary but not sufficient. Systems must also regulate how much evidence from heterogeneous sources is retrieved and used during generation. W-RAG is effective precisely because it addresses both stages jointly.

A.5 Fine-grained Analysis of Knowledge Base Influence AI Technology & Digital Media. This domain exhibits the largest mismatch between prescribed and realized KB usage, making it the most challenging setting for source control. In particular, realized attribution under weaker retrieval-generation settings appears to underuse KB3 (Regulatory) and KB4 (Market) relative to their prescribed contribution, while over- or under-shooting KB1 (Corporate) and KB2 (Scientific) depending on document type. This suggests that, in this domain, relevant evidence is more unevenly distributed across KBs, making generation more prone to source collapse. Climate & Sustainability. Climate & Sustainability shows the strongest agreement between prescribed and realized distributions. Across document types, the dotted curves largely track the shape of the solid curves, especially for KB3 (Regulatory) and KB4 (Market), which are both prominent contributors in this domain. Healthcare & Telehealth. In Healthcare & Telehealth, the realized distributions capture the broad shape of the prescribed source mixture, but some attenuation is visible, particularly for KB3 (Regulatory) and KB4 (Market), where realized attribution is lower than intended for some document types. Even so, compared to weaker baselines, the realized source usage remains substantially better aligned with the prescribed mixture. Finance. Finance displays a mixed but still informative pattern. The realized distributions generally recover the main peaks in the prescribed source weights, especially for KB1 (Corporate) and KB3 (Regulatory), but show larger deviations for KB2 (Scientific) and KB4 (Market) in some document types. This suggests that finance-related generation

is strongly influenced by corporate and regulatory evidence, while market-facing information is somewhat harder to preserve in the intended proportion. Nevertheless, the realized curves still follow the global structure of the prescribed distributions, indicating meaningful source influence rather than purely free-form generation.

A.5.1

KB Influence - Case Study

To further test whether content generation is sensitive to the proportions of content retrieved from the KBs, we conduct a controlled weight-override study on the Policies document type across all four domains. Rather than relying on the automatically recommended source weights, we manually specify four alternative input KB distributions for each example. Specifically, each document is run four times, using four permutations of the same weight set, ((40, 35, 20, 5)). In each run, a different KB is assigned the lowest weight ((5)), while the remaining three KBs receive the higher allocations ((40, 35,) and (20)). This design allows us to test whether systematically changing the prescribed source composition at retrieval time leads to corresponding changes in the realized KB usage in the generated document. For each generated document, we compute the realized KB distribution using the sentence-level attribution procedure described in Section 5, and then average the realized distributions across all policy examples. Figure5 shows the results. In each panel, the dots denote the actual prescribed and realized proportions for each KB, while the connecting lines are included only to make the distributional trend easier to compare. The key result is that changing the prescribed KB mixture leads to corresponding changes in the realized source attribution of the generated document. In other words, when the retrieval context is reweighted to emphasize or de-emphasize a particular knowledge base, the content of the final enterprise document changes accordingly. This shows that enterprise document generation is highly sensitive to the type of information drawn from heterogeneous KBs: corporate, scientific, regulatory, and market sources do not contribute interchangeably, and shifting their relative presence in retrieval changes the informational composition of the output.

A.6

Corporate Proposal Drafting as a Target Application

We highlight corporate proposal drafting as a motivating enterprise application for weighted multiKB retrieval-augmented generation. In many organizations, proposal teams prepare detailed responses to Requests for Proposals (RFPs) issued by government or commercial clients. These proposals are high-value documents: they must be persuasive, factually grounded, compliant with solicitation requirements, and tailored to the client, while also reflecting company capabilities, prior performance, pricing constraints, and organizational style. As a result, proposal drafting is both documentintensive and retrieval-dependent. Why proposal drafting is challenging. Drafting a competitive proposal is time-consuming because relevant information is distributed across many heterogeneous internal sources. Writers must identify reusable content from prior proposal responses, past contracts, compliance artifacts, whitepapers, pricing documents, company capability decks, and agency-specific materials, then adapt that evidence into a coherent response. In practice, quality depends not only on retrieving relevant passages, but also on balancing evidence from multiple sources, maintaining factual consistency, satisfying compliance constraints, and customizing the draft for the target solicitation. These properties make proposal drafting a strong real-world instance of the broader problem studied in this paper. Deployment context. The motivating workflow was observed in internal bids and proposals teams responsible for responses to U.S. civilian and Department of Defense solicitations. Users included approximately 15 internal personnel across three functional groups: proposal managers, technical or solution architects, and business development professionals. The teams worked on proposals spanning application development and modernization, IT support and service desk services, and cloud transformation and modernization. The internal knowledge sources used in this workflow included past proposal responses, prior contracts, capability decks, whitepapers and thought leadership documents, pricing materials, company compliance and regulatory documents, agency-specific materials, and graphics. Workflow. The end-to-end drafting process is human-in-the-loop. A user begins by entering an

RFP summary or proposal request. The retrieval system then identifies relevant internal materials from the available knowledge bases and drafts one or more proposal sections. The resulting draft is reviewed and edited by the proposal team, who refine language, add organization-specific positioning, adjust structure where necessary, and prepare the submission package. The system therefore functions as a drafting assistant rather than a fully autonomous proposal generator. Observed operational utility. Between 2025 and 2026, the broader retrieval-assisted proposal drafting workflow was used in an ongoing internal setting. Users reported substantial efficiency benefits, including approximately 12–18 hours of time savings per team per week, with some reports indicating up to 20–25 hours saved during beta testing. Proposal turnaround was reported to be up to 3× faster, and first-draft preparation time was reduced by as much as 80% in some cases. Draft acceptance was also high for early proposal drafts (“pink drafts”), with users reporting that 90–100% of draft content was often acceptable as a starting point for downstream refinement. In addition, users reported relatively low editing burden for sections where the system could rely heavily on vetted internal knowledge and compliance-oriented document structure. Most useful generated sections. Users found the system especially useful for sections that depend heavily on reusable institutional knowledge, including past performance, technical volumes, management volumes, and task-area content. They also reported that a complete Word document output was valuable operationally because it reduced manual setup effort associated with templates and structure selection. According to user feedback, the document structure was often highly compliant with solicitation requirements, which reduced formatting and compliance overhead during early-stage drafting. Where human intervention remained necessary. Despite these benefits, expert review remained essential. Users consistently identified executive summaries, company differentiators, win themes, and staffing plans as sections requiring substantial human input. These components depend on competitive strategy, tacit organizational knowledge, dynamic personnel planning, and other forms of information that are often weakly represented in re-

trievable document collections. In particular, users noted that strong executive summaries and differentiator sections require persuasive positioning tailored to the solicitation, while staffing plans are too dynamic to be reliably delegated to an automated system. Relation to W-RAG. It is important to distinguish the operational workflow above from the specific method proposed in this paper. The production-facing system described here was a retrieval-assisted drafting workflow, but W-RAG itself has not yet been fully deployed in production. Therefore, the operational efficiency numbers above should be interpreted as evidence that corporate proposal drafting is a high-value and practically relevant use case for retrieval-augmented generation, rather than as a direct causal evaluation of W-RAG. In this paper, we evaluate W-RAG offline on a proposal drafting dataset and show that weighted multi-KB retrieval improves generated document quality and evidence use relative to standard baselines. We view the real-world workflow described here as a motivating deployment context and our offline results as evidence that W-RAG is a promising next-step improvement for such settings. Takeaway. Overall, the corporate proposal drafting workflow illustrates why weighted multi-KB retrieval matters in enterprise document generation. Proposal writing is not merely a search problem and not merely a text generation problem: it requires integrating heterogeneous internal evidence into a compliant, customizable, and organizationspecific draft while preserving human control over final content. This makes it a natural target application for W-RAG. A.7

Manual Validation Instructions

To ensure evaluation reliability, all LLM-based Requirement Satisfaction and Context Relevancy scores were manually validated by 2 humans. Validators were instructed to review the generated document against the requirement checklist item by item, independently, using the following rubric: (1) Yes : the requirement is clearly and adequately addressed in the generated document; (2) No : the requirement is absent or insufficiently covered. Validators were asked to focus on content coverage, not writing style. They were also instructed to judge each checklist item independently and not to reward broad but vague language unless the requirement was concretely addressed.

Enterprise Generative AI Governance Policy Requirement Documentation The report must be a structured document with the following four required sections: 1. Scope, Objectives & Governance • Policy scope: covered systems (LLM-based assistants, prompt interfaces, orchestration layer), business units, and geographies • Objectives: productivity enhancement, risk reduction versus uncontrolled usage, alignment with risk appetite • Governance structure: AI Risk Committee, roles of Group Risk, IT, Data, Legal/Compliance, and Business Owners • Policy ownership, review cycle (e.g., annual or on major technology/regulatory changes), and escalation paths • Relationship to existing policies: Information Security, Model Risk, Outsourcing/Third Parties, Data Protection, Conduct & Ethics 2. Data, Privacy & Security Controls • Data classification rules for prompts and outputs (e.g., prohibitions on secrets, PII, trading strategies, client identifiers in external LLMs) • Allowed vs. disallowed data types by deployment type (on-prem vs. public cloud, single-tenant vs. shared) • Data minimization and prompt hygiene standards; redaction, masking, and synthetic data where feasible • Security controls: encryption, network segmentation, access control, logging, and monitoring • Cross-border data transfer controls, including geofencing and regional hosting requirements • Privacy and confidentiality requirements by jurisdiction (GDPR, UK GDPR, GLBA, local banking secrecy laws) 3. Model Lifecycle, Quality & Safety • Model sourcing and approval: criteria for selecting LLM providers; due diligence on training data, security, and reliability • Model Risk Management alignment: classification of generative AI tools as models vs. tools; validation expectations for high-risk use cases • Allowed vs. prohibited use categories (e.g., no unsupervised credit decisions, legal advice, investment recommendations, or regulatory reporting) • Human-in-the-loop and human-on-the-loop requirements for business-critical use cases • Hallucination / output reliability controls: mandatory citation of sources for internal knowledge tasks, disclaimers for outputs • Testing, monitoring, and performance review: pre-deployment testing, ongoing monitoring, drift/failure modes, and incident thresholds 4. Implementation, Monitoring & Compliance • Onboarding process for new generative AI use cases: risk assessment templates, approvals, and documentation • User access management: eligibility, training requirements, access revocation • Training & awareness: mandatory e-learning modules, acceptable use guidelines, role-specific training • Monitoring & reporting: centralized logging, dashboards for usage by business unit/region/use case, alerts for policy breaches • Incident management: definition and handling of AI-related incidents (data leakage, harmful output, compliance breaches) • Assurance & audit: internal audit scope, regulatory exam readiness, evidence retention, KPI/KRI definitions

Figure 2: Sample input requirement document from the Policies subset of the benchmark. The document specifies the required structure and mandatory content elements that the generated enterprise policy must cover.

Sample LLM-as-a-Judge Prompt for Requirement Satisfaction You are evaluating whether a generated enterprise document satisfies the essential requirements of the requested document type. You are given: (1) The input requirement document (2) The generated enterprise document (3) A checklist of essential requirements For each checklist item, assign one of the following labels: (1) Full: the requirement is clearly and adequately satisfied (2) Partial: the requirement is mentioned but incomplete or underspecified (3) None: the requirement is absent or not meaningfully addressed Convert these labels into scores: (1) Full = full points (2) Partial = half points (3) None = zero points Judge each checklist item independently. Focus on whether the required content is present, not on writing style or fluency. Return the item-level scores and the total score.

Figure 3: Prompt template used for LLM-based scoring of Requirement Satisfaction. Scores were manually verified by domain practitioners before analysis.

Sample LLM-as-a-Judge Prompt for Context Relevancy You are evaluating whether retrieved passages are relevant to an enterprise document drafting task. You are given: (1) The input requirement document (2) A retrieved passage Determine whether the passage is relevant to drafting the requested document. Label the passage as: (1) Relevant: the passage provides useful evidence for satisfying one or more required parts of the target document. (2) Not Relevant: the passage is only loosely related, redundant, or does not materially help with the drafting task Focus on task usefulness rather than general topical similarity. A passage may be relevant even if it supports only one section of the document.

Figure 4: Prompt template used for LLM-based scoring of Context Relevancy. Passage-level judgments were manually validated before analysis.

Figure 5: Controlled KB weight-override study on Policies across all domains. Solid denotes one prescribed input KB distribution, constructed as a permutation of (40, 35, 20, 5), and the corresponding dotted curve denotes the realized KB attribution in the generated output.

Related documents

• EAR: Entity-Aware Partitioning Approach for Retrieval-Augmented Generation Development#1000228
• BudgetMem: Training-Free Selective Memory for Cost-Efficient Long-Context Processing in Language Models#1048513
• Does task decomposition improve automatic NLG evaluation?#821111
• KBB: A Knowledge-Bundle Builder for Research Studies#373516
• Survey of Enterprise Knowledge, Legal Profiles, and Review Forecasting#696125