Factual Yet One-Sided: Stance Poisoning and Evidence Balancing in Retrieval-Augmented Generation

Factual Yet One-Sided: Stance Poisoning and Evidence Balancing in Retrieval-Augmented Generation Puyuan Yang1,‡,§

Shaoxuan Dong2,‡

Liujie Wu1

Xiaochun Cheng1,*

Bo Li2,*

arXiv:2610.34294v1 [cs.CL] 25 Oct 2026

1

2

Department of Computer Science, Swansea University, Swansea, United Kingdom School of Electronics and Information, Northwestern Polytechnical University, Xi’an, China ‡

These authors contributed equally. Email: yangpuyuan123@gmail.com * Corresponding authors: xiaochun.cheng@swansea.ac.uk; libo803@nwpu.edu.cn §

Abstract The stance expressed in responses from retrieval-augmented generation systems is significantly influenced by the stance distribution of the retrieved materials. Existing research on knowledge poisoning has primarily targeted factual errors. In contrast, this study focuses on injection attacks in which the facts are correct but the stance on a controversial issue is one-sided. At a mean topic-level poisoning share of 18.9% within the post-injection ProCon documents, all four generation models from different vendors exhibited stance shifts aligned with the poisoning direction; across four concentration levels, the shifts increased monotonically. This paper proposes a Stance-Balancing Middleware positioned between the retriever and the generator. It consists of stance detection, fixed balancing (2 supporting + 2 opposing + 1 neutral), and a debiasing prompt, without introducing additional large language model calls. Across all eight concentration–direction settings, the complete method reduced the magnitude of stance deviation by an estimated 0.106 to 0.350; the six settings at poisoning shares of 11.9%– 18.9% remained significant after Holm adjustment (adjusted p ≤ 0.009). In the cross-model experiment, all eight model–direction combinations moved along the stance axis in the corrective direction, and seven of eight showed a clearly detected reduction in absolute distance from 0. In the concentration sweep, the post-defense intervals included 0 in all eight settings, while the cross-model pro-poisoning results retained small positive residual point estimates of +0.045 to +0.063. For 20 candidate documents, stance detection took approximately 1.5% of the median generation time on a single GPU and approximately 16% on CPU. Keywords: Retrieval-Augmented Generation; Knowledge Poisoning; Stance Detection; Diversity Re-ranking; Bias Mitigation

1

Introduction

1.1

Background

Retrieval-Augmented Generation (RAG) connects an updatable external knowledge base to a large language model. After a user submits a question, the system first retrieves relevant documents from the knowledge base and sends them to the model together with the question, enabling the model to formulate its response on the basis of the retrieved material. This paradigm mitigates problems arising from outdated knowledge encoded in model parameters and fabricated content and 1

has been applied in fields including healthcare, finance, aviation safety, industrial operations and maintenance, and education. At the same time, this paradigm introduces a less-discussed property: the stance of a model's response can be influenced by the retrieved material. The more faithfully the model follows the retrieved material, the more likely the stance distribution of that material is to carry over into the response. For factual questions, this provides traceable support for the answer. For controversial questions, however, the selection of materials affects which stances are presented. Controlling the composition of the retrieval pool may therefore provide a means of manipulating the stance conveyed by model outputs.

1.2

Motivation and Challenges

Most RAG-poisoning studies target factual corruption or attacker-specified outputs. Recent work has also demonstrated topic-level opinion manipulation, showing that retrieval can be exploited to steer model positions across related queries. This paper focuses on a complementary question: how to defend against factually grounded but stance-skewed document injection without modifying the retriever or generator. Specifically, we study source-grounded document injection that concentrates evidence on one side of a controversial issue; human validation of the sampled injected documents is reported in Section 4.1 and Appendix D. Controversial issues generally have no single widely accepted answer, and both sides possess genuine supporting arguments. An attacker need not fabricate content; injecting arguments from one side in bulk is sufficient to alter the stance composition of the retrieval pool. Checking individual claims alone does not establish whether the retrieved evidence represents competing positions adequately. We constructed poisoned knowledge bases in two directions for 210 controversial questions. At a mean topic-level poisoning share of 18.9% within the post-injection ProCon documents (Section 4.1), all four generation models—from different vendors and with varying parameter counts—exhibited stance shifts aligned with the poisoning direction, and the shifts increased monotonically with poisoning concentration (see Figs. 2 and 3). The attack is difficult to detect because the affected responses may remain factually valid: the cited materials can be verified, but they present arguments from only one side. Evaluations based on answer correctness struggle to capture this type of bias. The problem lies in the composition of stances, a dimension for which existing evaluation and defense systems lack adequate measurement. This study therefore begins by establishing such a measure.

1.3

Contributions

First, this study characterizes poisoning through stance skew rather than factual error. We define a continuous stance score over sets of controversial questions and use strictly paired effect sizes and bootstrap confidence intervals to quantify how much a defense reduces deviation from 0. Second, we propose a Stance-Balancing Middleware that requires no additional large language model calls. Positioned between the retriever and the generator, it performs stance detection, fixed balancing (2 supporting + 2 opposing + 1 neutral), and a debiasing prompt. Its online balancing rule contains no thresholds or weights tuned on the test set. Third, we provide a systematic evaluation pipeline covering four poisoning concentrations, two poisoning directions, four generation models, and 210 controversial questions. The evaluation reports defense effects, efficiency overhead, content-moderation blocks, refusals, and a comparison of evidence-selection mechanisms on shared retrieval results.

2

Fourth, we analyze how evidence selection shapes answer stance and examine boundary cases. The results distinguish stance balance from content diversity and show how missing stance categories and target-sensitive classification errors can affect the evidence composition supplied to the generator.

2

Related Work

This section reviews three lines of research most directly related to this study: RAG knowledge poisoning and its defenses, text stance detection, and retrieval diversification and stance constraints.

2.1

Retrieval-Augmented Generation and Its Applications

Recent surveys characterize retrieval-augmented large language models as comprising three main stages: retrieval, augmentation, and generation [1]. A system first retrieves query-relevant materials from external knowledge sources, organizes them into a generation context, and then uses a language model to answer the question based on that context. External knowledge can be updated independently of model parameters and can provide traceable evidence for answers, but it also makes generated outputs substantially dependent on the content and quality of the retrieved materials. For retrieval augmentation, AT-RAFT [2] first performs retrieval with a dual encoder and then reranks the results with a cross-encoder; IIKM [3] combines BM25 with vector retrieval and applies a second round of ranking to the candidate results. Such two-stage or hybrid retrieval can improve relevance and recall, but its objective remains to identify “more relevant content” and does not constrain the stance composition of the retrieval results. In other words, even if every document is highly relevant to the query, the final context may still cover only one side of a controversial issue. RAG has been applied to specialized domains including aviation safety, industrial logs, financial information extraction, educational question answering, and medical assistance [4–8]. RACI identifies causal factors from aviation accident reports and improves retrieval robustness through noise filtering and adversarial training; LogRAG retrieves historical normal logs to support the review of anomaly detection results; ESGReveal extracts structured financial information from corporate reports; PLRTQA integrates answers dispersed across textbooks through cross-document retrieval; and LLMind Chat restricts the knowledge source of a medical assistance system to the authoritative ICD-11 manual. Although these applications address different tasks, they collectively demonstrate that generation quality depends on the retrieval context and the reliability of the knowledge sources. In high-stakes domains such as aviation, finance, and medicine, knowledge-base contamination may further affect downstream judgments. Natural noise generally arises from imperfect retrieval, whereas knowledge poisoning involves content actively constructed and injected by an attacker so that it can be retrieved.

2.2 2.2.1

RAG Knowledge Poisoning and Defense Knowledge Poisoning Attacks

Knowledge poisoning allows an attacker to influence specific queries solely by manipulating the knowledge base, without modifying the LLM. PoisonedRAG [9] systematically formulates RAG knowledge poisoning as an optimization problem and proposes both black-box and white-box variants. Its results showed that injecting five malicious texts per target query into a knowledge base containing millions of documents could achieve attack success rates of 90%–97%. This finding demonstrates that a small number of injections can affect the retrieval and generation pipeline without requiring the attacker to access model parameters. BadRAG [10] further constructs triggeractivated adversarial passages through contrastive optimization to implement retrieval backdoor 3

attacks. Among its attack modes, Selective-Fact-as-an-Attack (SFaaA), which guides the sentiment or stance of outputs by selectively presenting facts, is the form most closely related to the attack studied here. This study differs from BadRAG/SFaaA in its research target. BadRAG focuses on retrieval backdoors and their trigger mechanisms; SFaaA is one of its attack modes and does not separately characterize and quantify systematic manipulation of the stance distribution of retrieved materials. In contrast, this study investigates stance poisoning that requires neither factual errors nor special triggers and changes the stance of an answer solely through one-sided but truthful arguments. Web crawling, user uploads, and automated data pipelines in practical systems may all provide injection channels, giving the assumption that an attacker can manipulate only the knowledge base a realistic basis. Topic-FlipRAG studies topic-level opinion manipulation across multiple related queries and constructs retrieval-promoted adversarial documents through semantic editing and adversarial triggers [11]. Our work differs in both objective and mechanism. Rather than designing a stronger attack, we study an online defense for factually grounded but one-sided document injection that does not rely on specialized triggers. The proposed middleware uses a locally executed stance classifier, a fixed evidence quota, and a debiasing instruction to constrain the stance composition presented to the generator. Most existing poisoning studies target answers whose correctness can be determined and measure effects using answer accuracy or attack success rate. Fact-checking, comparison with external knowledge, and multi-document consistency checking accordingly rely on the premise that the attack content contains verifiable errors. This premise does not hold for materials that are factually correct but one-sided in stance: each individual claim may be true, yet selective coverage can alter the stance composition of the retrieval pool. Meanwhile, language models may have their own tendencies on controversial issues [12, 13]. If the model prior aligns with the direction of the retrieved materials, the observed answer stance may reflect both influences. 2.2.2

Existing Defense Methods and Their Limitations

RobustRAG [14] adopts an “isolate-then-aggregate” strategy. It first prompts the LLM to generate an answer independently for each retrieved document and then merges the results through secure aggregation, which can provide robustness guarantees. This method avoids interactions caused by directly mixing documents, but the number of generation calls grows with the number of retrieved documents, resulting in a high cost for real-time deployment. TrustRAG [15] employs a two-stage defense: it first uses K-means clustering to identify documents that deviate from the main semantic groups and then combines cosine similarity, ROUGE, and self-evaluation to handle suspicious content and conflicts between internal and external knowledge. Its decisions depend on semantic grouping and involve clustering parameters. GRADA [16], by contrast, constructs an inter-document semantic similarity graph and propagates scores in a PageRank-like manner to strengthen densely connected document groups and suppress isolated nodes, thereby reducing the influence of poisoned content through reranking. Stance-poisoned documents can be factually correct and topically consistent with clean materials, so semantic outlier detection or graph-based filtering may miss their collectively one-sided stance composition. Semantic grouping is not equivalent to stance grouping. RobustRAG avoids this assumption but requires per-document generation and aggregation, leaving a need for low-cost defenses that directly constrain stance composition. Another line of work adds behavioral constraints to prompts. Ganguli et al. [17] found that sufficiently large models trained with human feedback possess a degree of moral self-correction 4

and that natural-language instructions can reduce certain stereotypes in model outputs. Si et al. [18] systematically investigated the effects of prompting on model reliability and likewise observed improvements, including reductions in social bias. Prompt-based methods are inexpensive, decoupled from the model, and leave the retrieval process unchanged, but they act only on the existing context: when the evidence is one-sided, a prompt cannot add materials for the missing side.

2.3 2.3.1

Stance Detection, Reranking, and Evaluation Text Stance Detection

Stance detection determines whether a text supports, opposes, or remains neutral toward a specific topic or claim. Existing datasets vary substantially in topic scope, label systems, and scale, and generalization across domains and to topics unseen during training remains an important challenge [19]. Cross-topic generalization matters for an open knowledge base: the issues addressed by candidate documents may not appear in the training data, so a detector must both distinguish stances and handle unseen topics. One approach to open-domain stance detection treats the text as a premise and constructs hypotheses from the target topic and candidate stances, thereby obtaining indirect supervision from the entailment, contradiction, and neutral relations used in natural language inference (NLI). Xu et al. [20] map support, opposition, and neutrality to these three relations, respectively, initializing RoBERTa-large with MNLI before further fine-tuning it on stance data. Their work also notes that stance detection and NLI are not fully equivalent and that direct transfer remains difficult for open-domain and open-form topics. Accordingly, this study fine-tunes the MNLI-initialized model on stance data (Section 3.3). 2.3.2

Diversity Reranking

A recent survey of result diversification classifies MMR as a greedy post-processing method: the system incrementally constructs a result list, selecting at each step a candidate that balances query relevance against its difference from the already selected set [21]. This study implements the following scoring function: "

#

∗

d = arg max λ Sim(d, q) − (1 − λ) max Sim(d, dj ) . dj ∈S

d∈R\S

Here, R denotes the candidate set, S the selected set, and λ controls the trade-off between query relevance and content dissimilarity. Related methods generally balance relevance, content dissimilarity, and aspect coverage, but they do not directly constrain the stance composition of the results. Content diversity and stance balance are distinct constraint objectives. Documents on the same side may still use different arguments and wording and thus remain dissimilar in the content space. Reducing content redundancy therefore does not directly limit the numbers of supporting, opposing, and neutral documents; this requires an explicit constraint on stance structure. Section 4.5 compares the two constraints, and Section 5.1 discusses the results. 2.3.3

RAG Evaluation

RAG systems are typically evaluated in terms of both retrieval quality and generation quality. RAGAS [22] uses faithfulness to measure the factual consistency of an answer with its context, answer relevance to measure the match between an answer and its query, and context precision and 5

recall to assess the retrieval results and their coverage. These metrics are suitable for determining whether an answer is evidence-based and relevant and whether retrieval is effective, but they do not directly measure stance skew in either the retrieved materials or the generated answer. An answer may faithfully restate a one-sided context while still performing well according to RAGAS metrics. Poisoning studies commonly use attack success rate to measure whether a target incorrect answer is induced, but this metric presupposes a clearly defined erroneous target and cannot replace stance measurement for controversial issues. Stance skew is an aggregate statistic over a question set, and a single point estimate cannot represent sampling uncertainty. Relevant NLP evaluation research recommends reporting differences in method performance together with their bootstrap confidence intervals [23], while statistical learning research also treats the bootstrap as a standard resampling method [24]. This study therefore performs sampling with replacement at the topic level and reports 95% confidence intervals for continuous stance metrics and differences between methods; the number of resamples and the interval calculation procedure are provided in Appendix B.

2.4

Research Gaps

Existing research leaves three gaps directly relevant to this study. First, knowledge poisoning research has primarily focused on factual errors and answers whose correctness can be determined. SFaaA addresses shifts in tendency caused by selectively presented facts [10], and Topic-FlipRAG demonstrates topic-level opinion manipulation across related queries [11]. The remaining question addressed here is how to defend online against factually grounded but one-sided injection without specialized triggers, in both poisoning directions. Second, methods such as TrustRAG and GRADA use semantic anomalies or graph structures [15, 16], whereas existing result-diversification methods mainly constrain content dissimilarity or the coverage of query aspects [21]. None of these constraints guarantees balance among supporting, opposing, and neutral documents in the retrieval results. Third, the per-document generation used by RobustRAG incurs a call cost that grows with the number of documents [14], whereas low-cost prompting [17, 18] cannot alter the composition of the evidence. Existing approaches therefore leave a gap between deployment cost and structural constraints. On this basis, the present study addresses stance measurement, explicit stance constraints on retrieval results, and low-cost deployment.

3

Method

3.1

Problem Formulation

Given a controversial question q for which no single answer is universally accepted and multiple positions are supported by evidence, a retrieval-augmented generation (RAG) system first uses a retriever R to retrieve relevant documents R(q, D) from a document corpus D, and then supplies a subset of these documents together with q to a generator G to produce an answer a. For such questions, answer quality cannot be assessed solely in terms of factual correctness, and the stance of the answer may also be externally manipulated. Accordingly, we define stance poisoning as follows: an attacker injects factually correct but highly one-sided documents into D to alter the distribution of evidence observed by the retriever and thereby skew the generated answers. Because the attack targets evidence selection rather than factuality, defenses predicated on determining whether content is true or false are not applicable. Our primary metric measures answer stance; Section 4.8 separately evaluates answer quality under clean conditions. For each answer a, a separate evaluation model (see Section 4.1 for its identity and for the isolation measures ensuring that the evaluation model and the evaluated 6

generation model come from different model families) classifies it as pro, neutral, or con, mapped to +1, 0, and −1, respectively. Averaging over the evaluation question set Q yields the stance score s =

1 X v(aq ), |Q′ | q∈Q′

v(·) ∈ {+1, 0, −1},

s ∈ [−1, +1].

(1)

where Q′ ⊆ Q is the set of valid questions, as defined below. A score of +1 indicates that all answers are supportive, whereas −1 indicates the opposite; a score close to 0 indicates that answers on the two sides are approximately balanced in aggregate. The stance score characterizes the overall tendency of a set of answers rather than the correctness of any individual answer. In isolation, a single pro answer is not problematic; the problem arises when a system systematically presents only one side across an entire set of controversial questions. Three additional outcome types require separate treatment: refuse (the generator declines to take a position), filtered (the answer is blocked by a content moderation mechanism), and unparsed (the judging model's output cannot be parsed). None of these outcomes expresses a stance. Encoding them as 0 would mechanically pull s toward neutrality and conflate a change in stance with a decline in the rate of stance-bearing responses. We therefore exclude all three from Q′ and report their counts separately. We evaluate three quantities. First, the setting-A stance score and its 95% confidence interval characterize attack-induced skew. Second, the primary defense effect is the reduction in absolute distance from 0 from no defense to the complete method, |sA | − |sC |; a positive value means that the score moved closer to 0, and a paired interval entirely above 0 indicates a clearly detected reduction in deviation magnitude. Third, the setting-C stance score and its interval describe residual skew after defense. A 95% confidence interval represents the uncertainty in the stance score under the current sample and resampling procedure: an interval that excludes 0 supports a directionally clear aggregate tendency, whereas an interval that includes 0 means that no directionally clear residual is detected. Inclusion of 0 does not establish that the true bias is exactly 0 or that outputs have achieved balance. For A-to-C comparisons, only questions valid in both settings enter the strictly paired analysis. Topic-cluster paired bootstrap intervals report effect sizes and their uncertainty, while two-sided paired randomization tests provide p-values. The eight A-to-C comparisons formed by two poisoning directions and four concentrations constitute one test family and are adjusted using the Holm procedure. This multiplicity adjustment changes the evidential assessment across comparisons, not the direction of the effect-size estimates.

3.2

Overall Architecture

We propose the Stance-Balancing Middleware, which is inserted between the retriever and the generator. It receives the candidate document list returned by the retriever, sequentially applies three components—stance detection, fixed balancing, and a debiasing prompt—and then passes a small, reselected set of documents together with an additional instruction to the generator. The three components form a unidirectional pipeline with no loops and require no multi-turn interaction. The overall workflow is shown in Fig. 1. The middleware reselects retrieved documents and adds an instruction without modifying the retriever, generator, or document corpus. It preserves the existing interfaces and requires no retraining of either endpoint. Section 4.4 evaluates compatibility with four generators from different vendors.

7

Figure 1: Overall architecture: the Stance-Balancing Middleware is inserted between the retriever and the generator, with neither endpoint modified.

3.3

Component 1: Stance Detection

The first component classifies each retrieved candidate document as supportive (pro), opposing (con), or neutral with respect to the current question. It identifies stance rather than factual correctness. We use a locally executed, task-specific classifier to label candidate documents without additional LLM calls. Section 4.6 measures its latency relative to generation time. This document classifier is distinct from gpt-5-mini in Section 4.1, which labels generated answers only during offline evaluation. The classifier is fine-tuned from FacebookAI/roberta-large-mnli (355 million parameters). This backbone has been trained on natural language inference (NLI), whose task formulation is similar to stance classification, making it a suitable initialization for our few-shot fine-tuning setting. Classifier inference is performed entirely locally, without calling any external API. Training and evaluation use 6-fold topic-held-out cross-validation. Topics are divided into six groups, and each classifier is trained on five groups and evaluated on the held-out group. Thus, every document is classified by a model that has not seen its topic during training, measuring cross-topic generalization rather than topic-specific memorization. The evaluation corpus contains 1,898 original documents (738 pro, 740 con, and 420 neutral). Each original document is paired with one encyclopedia-style paraphrase for augmentation, with each pair kept in the same topic fold. The classifier is trained for five epochs using AdamW with a learning rate of 1e-5, batch size 16, maximum length 256, and seed 20260831; each of the six folds holds out seven complete topics. Under this protocol, the classifier achieves a three-class macro-F1 of 0.889 and a target-stance classification accuracy of 0.905 on 588 injected documents. On the pooled out-of-fold predictions, the Neutral class achieved an F1 of 0.973, compared with 0.856 for pro and 0.848 for con; most errors were direct pro–con confusions.

8

3.4

Component 2: Fixed Stance Quota (2+2+1)

The second component is the core of the Stance-Balancing Middleware and determines the final set of documents supplied to the generator. Retrieval constants. For each question, the retriever returns the top Ksearch = 50 ranked documents. The top Kpool = 20 constitute the primary candidate pool. Documents ranked 21 through 50 are used only as supplementary sources when documents of a particular stance are insufficient. The number of documents ultimately supplied to the generator is fixed at F = 5. Balancing rule. After stance classification, the rule does not directly pass the top 5 ranked documents to the generator. Instead, it reselects 5 documents according to fixed quotas: Step 1. Within the candidate pool (the top 20 documents), select by retrieval rank, from highest to lowest, up to 2 pro documents, 2 con documents, and 1 neutral document. Step 2. If any class does not meet its quota within the candidate pool, continue down the retrieval ranking to rank 50 and select additional documents from that class until its quota is filled. Step 3. If the fallback search has exhausted all three classes and fewer than 5 documents have been selected, disregard the quotas and fill the remaining positions by retrieval rank, from highest to lowest, until exactly five documents have been selected. When all three classes are sufficiently represented, this rule limits the number of slots occupied by a single stance and does not depend on identifying whether a document was injected by an attacker. The 2+2+1 quota was not searched on the test set, and the online balancing rule contains no thresholds or weights tuned on that set. In the Step 3 case, the final set does not meet the 2+2+1 quota. Appendix F.1 provides examples and the distribution of this fallback case.

3.5

Component 3: Debiasing Prompt

The third component adds a lightweight intervention on top of balancing. An instruction is appended to the prompt supplied to the generator, informing the model that the provided materials may be imbalanced in stance. The model is instructed not to accept a document's conclusion merely because the document is longer or uses stronger language, but instead to weigh the strongest evidence from both sides equally. If the question is genuinely contested, the answer should reflect that disagreement rather than adopt the framing of either side. All settings use a common base generation template that asks the model to answer directly in 3–5 English sentences using only the retrieved documents, ground each substantive claim in those documents, and neither refuse nor request more information merely because the evidence is limited. Settings C and D insert the same fixed debiasing instruction after the question and before the output requirements, whereas settings A and B do not. The additional instruction neither changes the selected documents nor adds a model call. Appendix C provides the verbatim templates.

3.6

Complexity and Overhead

The three components have the following overhead profile. Stance detection incurs linear cost in the number of candidate documents: one small-model forward pass is performed for each candidate document. The number of documents is determined by retrieval depth rather than corpus size, and this is the only component with substantive computational cost. Fixed balancing consists solely 9

of ranking and counting logic, involves no model inference, and has negligible runtime relative to stance detection. The debiasing prompt is a constant-length text segment that adds only a small number of tokens to the prompt and does not increase the number of calls. The method therefore has an important end-to-end overhead property: it introduces no additional large-language-model calls. All computation in the Stance-Balancing Middleware is performed by a local small model and ranking logic. Consequently, it neither increases the number of calls to the remote generation service nor incurs the corresponding cost increase. This contrasts with defenses based on multi-round verification, post-retrieval review, or multi-model voting, which typically trade multiplicative increases in large-language-model calls for robustness. Measured CPU and GPU latency, together with their proportions relative to the time required to generate one answer, are reported in Section 4.6.

4

Experiments

4.1

Experimental Setup

4.1.1

Datasets

The evaluation set contains 210 controversial questions spanning 42 topics, with five questions per topic. The topics were drawn from a public list of controversial issues and cover areas including public policy, technology ethics, and social institutions. Their common characteristic is the absence of a single widely accepted answer. The clean knowledge base consists of a subset of the Natural Questions corpus containing 51,898 documents. Two direction-specific poisoned corpora were constructed on this basis: all documents injected into the pro corpus support the corresponding stance, whereas all documents injected into the con corpus oppose it. The complete bidirectional set contains 588 injected documents, with 294 documents per direction. The four single-direction knowledge bases contain 84, 168, 210, and 294 injected documents, respectively. The separately constructed set of 420 neutral Wikipedia documents is not part of the injected-document collection. Appendix A provides a descriptive comparison of word count and within-group nearest-neighbor BGE cosine similarity. For concision, the tables below refer to the pro-poisoned and con-poisoned corpora as the pro and con corpora, respectively. Two human annotators independently assessed 252 of the 588 injected documents, sampling three from each of the 84 topic–direction groups. Documents initially flagged for factual issues or insufficient evidence were reviewed, and disagreements were adjudicated. All 252 sampled documents were ultimately judged to contain verifiable claims supported by the supplied sources and a clearly one-sided stance matching the intended injection label. Appendix D reports the criteria and results. For topic t, let n ∈ {2, 4, 5, 7} be the number of documents injected in one direction and let Nt be the original number of ProCon documents. The realized poisoning share is rt =

n , Nt + n

R=

42 1 X rt . 42 t=1

Thus, the share for each topic uses the number of injected documents in that direction divided by the post-injection number of ProCon documents, and R is its macro-average over the 42 topics. The four target concentrations of 5%, 10%, 15%, and 20% yield realized macro-average shares of 6.4%, 11.9%, 14.4%, and 18.9%, respectively. The gaps between target and realized shares arose from relevant documents already present in the clean corpus. The share covers only the ProCon documents of each topic; it excludes the rest of the knowledge base, including the neutral Wikipedia 10

documents. The main text reports the actual values throughout. All four levels are included in the concentration-sweep attack and defense analyses; setting D is evaluated only at the two highest levels in the ablation study. 4.1.2

Baselines

We consider two evidence-selection baselines, each selecting five documents from the top-20 candidate pool. The proposed method also starts from this pool but can extend its search to rank 50 when a stance quota is unmet (Section 3.4): • Random-5: Five documents are sampled without replacement from the candidate pool using a fixed random seed of 20260909. • MMR (maximal marginal relevance): Documents are selected greedily. At each step, MMR considers both relevance to the question and a penalty for excessive similarity to already selected documents, using the default trade-off coefficient λ = 0.5. It represents a contentdiversification approach in which the five selected documents are mutually nonredundant. MMR removes redundancy only at the content level and does not consider stance, which is precisely the distinction examined in this study. Document vectors are reconstructed and reused directly from the FAISS index. Because the index is an L2-normalized IndexFlatIP, inner products equal cosine similarity; the procedure introduces neither additional encoding nor additional model calls. 4.1.3

Evaluation Metrics

The stance score is defined in Section 3.1. Setting A characterizes attack-induced deviation, the strictly paired reduction |sA | − |sC | is the primary effect size for the complete method, and setting C characterizes residual deviation after defense. Because questions within a topic may be correlated, we resample whole topics when estimating confidence intervals. Paired comparisons use questions valid in both settings. Appendix B provides the resampling and testing details. 4.1.4

Implementation Details

The retriever is BAAI/bge-large-en-v1.5 [25], which outputs 1,024-dimensional vectors. After L2 normalization, the vectors are indexed using FAISS IndexFlatIP [26], so the inner product equals cosine similarity. The retrieval depth, candidate-pool size, and number of documents ultimately supplied to the generator are 50, 20, and 5, respectively, consistent with Section 3.4. The main concentration-sweep, ablation, and selection-baseline experiments use claude-opus-4-8 as the generator; Section 4.4 separately replaces it with the four generators listed there. Sampling parameters not explicitly supplied by the client follow the gateway defaults. All experiments in this section use the four settings defined in Table 1. Table 1: Definitions of the four experimental settings Setting

Name

Selection of the five documents supplied to the generator

Prompt

A

No defense

No additional instruction

B

Balancing (no prompt)

Directly supply the top five retrieved documents 2 pro + 2 con + 1 neutral

11

No additional instruction

(a) No-defense attack curve

(b) Pro-side poisoning: A/B/C (c) Con-side poisoning: A/B/C

Pro-side poisoning Con-side poisoning

(d) Paired component effects

Pro 6.4% Pro 11.9%

0.4 Stance score

Pro 14.4% 0.2

Pro 18.9%

0.0

Con 6.4% Con 11.9%

−0.2

Con 14.4%

A: No defense B: Rebalancing C: Rebalancing + prompt

−0.4 6.4

11.9

14.4

18.9

Actual poisoning share (%)

6.4

11.9

14.4

18.9

Actual poisoning share (%)

A−B B−C

Con 18.9% 6.4

11.9

14.4

18.9

−0.4

Actual poisoning share (%)

−0.2

0.0

0.2

0.4

Signed stance shift

Figure 2: Attack effectiveness across poisoning concentrations and defense effectiveness under proand con-side poisoning.

Setting

Name

Selection of the five documents supplied to the generator

Prompt

C

Complete method (balancing + prompt) Prompt only

2 pro + 2 con + 1 neutral

Append a debiasing instruction

Top five retrieved documents, as in A

Append the same debiasing instruction as in C

D

Settings B and C both use stance-detection results to perform fixed balancing; C additionally applies the debiasing prompt. Setting A characterizes the attacked state without defense, setting C is the complete Stance-Balancing Middleware advocated in this study, and settings B and D isolate the contribution of each component. The answer-stance judge is gpt-5-mini in all experiments; its fallback chain, which excludes the current generator, was never triggered. Judging is used only for offline evaluation, and its cost is excluded from the online latency reported in Section 4.6. The answer-stance judge was additionally checked against human-reviewed reference labels; Appendix E reports the validation basis.

4.2 4.2.1

Main Results Attack Effectiveness

This section examines whether stance poisoning itself constitutes an effective attack. Figure 2(a) shows how the attack varies with concentration, and Table 2 reports stance scores for setting A (no defense) under four poisoning concentrations and two poisoning directions. Table 2: Attack effectiveness: stance scores and 95% confidence intervals without defense (210 questions per cell) Target concentration Poisoning share

Pro-poisoned corpus

Con-poisoned corpus

5% 10%

+0.115 [+0.014, +0.215] +0.230 [+0.124, +0.335]

−0.129 [−0.234, −0.019] −0.197 [−0.301, −0.086]

6.4% 11.9%

12

Target concentration Poisoning share

Pro-poisoned corpus

Con-poisoned corpus

15% 20%

+0.297 [+0.187, +0.409] +0.373 [+0.260, +0.483]

−0.234 [−0.343, −0.119] −0.325 [−0.429, −0.215]

14.4% 18.9%

The stance-score magnitude increases monotonically with concentration in both directions, and each shift aligns with the poisoning direction. All eight confidence intervals exclude 0, providing additional evidence that the attack-induced skew is directionally distinguishable even at the lowest actual proportion of 6.4%. Under this experimental setup, a mean topic-level poisoning share below 20% within the post-injection ProCon documents is sufficient to shift answers overall toward the corresponding direction. 4.2.2

Defense Effectiveness

Figure 2(b) and (c) show stance scores for settings A, B, and C, while Figure 2(d) shows the signed stance shifts A−B and B−C. Table 3 reports the corresponding numerical results for the same eight cells. Table 3: Stance scores and 95% confidence intervals for no defense, balancing without a prompt, and the complete method Corpus

Poisoning share

A: No defense

B: Balancing (no prompt)

C: Complete method (balancing + prompt)

Pro Pro Pro Pro

6.4% 11.9% 14.4% 18.9%

Con Con Con Con

6.4% 11.9% 14.4% 18.9%

+0.115 [+0.014, +0.215] +0.043 [−0.048, +0.129] −0.005 [−0.091, +0.077] +0.230 [+0.124, +0.335] +0.086 [−0.014, +0.190] +0.014 [−0.068, +0.092] +0.297 [+0.187, +0.409] +0.053 [−0.038, +0.144] −0.005 [−0.091, +0.072] +0.373 [+0.260, +0.483] +0.153 [+0.057, +0.249] +0.024 [−0.067, +0.113] −0.129 [−0.234, −0.019] +0.010 [−0.077, +0.097] −0.019 [−0.096, +0.058] −0.197 [−0.301, −0.086] +0.024 [−0.067, +0.120] −0.024 [−0.096, +0.052] −0.234 [−0.343, −0.119] +0.048 [−0.048, +0.144] −0.034 [−0.119, +0.049] −0.325 [−0.429, −0.215] +0.038 [−0.062, +0.144] +0.019 [−0.049, +0.095]

Across all eight concentration–direction settings, the complete method reduced absolute stance deviation by an estimated 0.106–0.350. Table 3 shows the remaining stance scores, and its continuation (Table 4) reports the strictly paired A-to-C effects. All eight setting-C intervals include 0, indicating no directionally clear residual in this concentration sweep. Table 4: Strictly paired effects of the complete method (A→C) and 95% confidence intervals (continuation of Table 3) Corpus

Poisoning share Signed A→C stance shift [95% CI]

Reduction in absolute distance from 0 [95% CI]

Pro Pro Pro Pro Con Con

6.4% 11.9% 14.4% 18.9% 6.4% 11.9%

+0.106 [−0.038, +0.181] +0.216 [+0.081, +0.300] +0.293 [+0.139, +0.383] +0.350 [+0.211, +0.447] +0.111 [−0.015, +0.205] +0.173 [+0.048, +0.262]

+0.115 [+0.005, +0.226] +0.216 [+0.110, +0.324] +0.303 [+0.180, +0.433] +0.350 [+0.232, +0.466] −0.111 [−0.228, +0.005] −0.173 [−0.273, −0.072]

13

Corpus

Poisoning share Signed A→C stance shift [95% CI]

Con Con

14.4% 18.9%

−0.203 [−0.300, −0.106] −0.351 [−0.442, −0.263]

Reduction in absolute distance from 0 [95% CI] +0.203 [+0.082, +0.295] +0.313 [+0.153, +0.411]

Note: The signed shift is sA − sC ; its sign indicates movement along the stance axis. The reduction in absolute distance is |sA | − |sC |, for which a positive value indicates movement closer to 0. Each row uses only questions valid in both A and C. Differences and intervals are computed from unrounded question-level results and may therefore differ by 0.001 from direct subtraction of the three-decimal displayed values. The six moderate-to-high concentration comparisons remain supported after Holm adjustment, while the two lowest-concentration comparisons have weaker adjusted evidence. Assigning each refusal to con, neutral, or pro preserves the direction of reduction in all eight settings, with intervals above 0 in the six moderate-to-high settings. Together, these results support the complete middleware’s effectiveness in reducing stance deviation across both poisoning directions and all tested concentrations, with stronger evidence at moderate and high concentrations.

4.3

Ablation Study

To separate the role of prompting from that of balancing, we add setting D, which retains the evidence selection used in setting A (directly supplying the top five retrieved documents) and adds only the debiasing prompt. All four settings were evaluated at the two highest concentrations. Point estimates are reported in Table 5. Table 5: Four-setting ablation comparing balancing alone, prompting alone, and their combination Corpus

Poisoning share A: No defense

B: Balancing (no C: Complete prompt) method (balancing + prompt)

Pro Pro Con Con

14.4% 18.9% 14.4% 18.9%

+0.053 +0.153 +0.048 +0.038

+0.297 +0.373 −0.234 −0.325

−0.005 +0.024 −0.034 +0.019

D: Prompt only

+0.167 +0.256 −0.215 −0.291

Note: Values are point estimates; Table 3 reports the intervals for A, B, and C. In point estimates, prompting alone reduces the absolute score by 0.019–0.130, balancing alone by 0.186–0.287, and the complete method leaves the smallest absolute residual (0.005–0.034) in all four settings. Balancing is therefore the main source of correction, and prompting adds a further reduction.

4.4

Generalization across Generators

In this experiment, only the generator is changed; the questions, index, retrieval results, stance detection, balancing code, and judge model are held fixed. The experiment covers four providers— DeepSeek, Meta, Moonshot, and Google—and compares A with C under both poisoning directions at a poisoning share of 18.9%. Generator fallback is disabled. Cross-model results are presented in Figure 3 and Table 6.

14

(a) Pro-side poisoning: A vs. C

(b) Con-side poisoning: A vs. C

0.6

DeepSeek · Pro DeepSeek · Con

0.4 Stance score

(c) Reduction in absolute bias

Gemma · Pro

0.2

Gemma · Con 0.0

Kimi · Pro Kimi · Con

−0.2

Llama · Pro

−0.4 −0.6

A: No defense C: Rebalancing + prompt

DeepSeek Gemma Kimi Llama 208/207 192/200 209/208 210/208

Llama · Con DeepSeek Gemma Kimi Llama 205/207 201/206 207/209 207/208

0.00

0.15

0.30

0.45

0.60

Reduction in absolute bias, |A| − |C|

effective n (A / C)

Figure 3: Cross-generator stance scores before and after applying the complete defense at an actual poisoning proportion of 18.9%. Table 6: Cross-generator stance scores and 95% confidence intervals (poisoning share: 18.9%) Generator

Pro A

DeepSeek-V4-Pro

+0.370 [+0.238, +0.505] Gemma-4-31B +0.318 [+0.204, +0.426] Kimi-K2.6 +0.435 [+0.321, +0.543] Llama-4-Maverick-17B +0.495 [+0.357, +0.614]

Pro C

Con A

Con C

+0.063 [+0.000, +0.126] +0.045 [+0.010, +0.082] +0.053 [+0.005, +0.110] +0.058 [+0.010, +0.106]

−0.424 [−0.514, −0.328] −0.095 [−0.188, +0.000] −0.353 [−0.454, −0.252] −0.314 [−0.435, −0.191]

−0.034 [−0.088, +0.024] +0.019 [−0.014, +0.053] −0.005 [−0.043, +0.033] +0.043 [−0.014, +0.100]

All four evaluated models exhibit setting-A shifts that align with the poisoning direction. In all eight model–direction combinations, the signed A−C interval excludes 0 and the score moves along the axis in the direction that counteracts the attack; 7/8 combinations also have an interval excluding 0 for the reduction in absolute distance from 0. Under pro-side poisoning, the complete method reduces deviation by more than 80% from the setting-A values, with setting-C point estimates from +0.045 to +0.063. Small positive residual point estimates remain; their intervals exclude 0 in three models and include 0 in DeepSeek. Under con-side poisoning, all four setting-C intervals include 0. Section 5.4 discusses the remaining pro-side residuals. Gemma-4-31B under con-side poisoning is the sole combination whose reduction in absolute distance from 0 is not clearly detected: +0.081 [−0.021, +0.179], despite a signed A−C shift of −0.121 [−0.207, −0.036]. Its setting-A score is −0.095, compared with −0.314 to −0.424 for the other models. Counts of blocked responses, refusals, and valid samples for each model are shown in Table 7. Table 7: Blocked-response and refusal statistics (denominator: 210 questions per cell) Generator

Corpus

A blocked A refused A included

C blocked C refused C included

DeepSeek-V4-Pro

Pro

1

1

1

208

15

2

207

Generator

Corpus

A blocked A refused A included

C blocked C refused C included

DeepSeek-V4-Pro Llama-4-Maverick17B Llama-4-Maverick17B Kimi-K2.6 Kimi-K2.6 Gemma-4-31B Gemma-4-31B

Con Pro

1 0

4 0

205 210

0 0

3 2

207 208

Con

0

3

207

0

2

208

Pro Con Pro Con

1 2 11 1

0 1 7 8

209 207 192 201

1 0 3 1

1 1 7 3

208 209 200 206

Gemma-4-31B has the most exclusions (up to 18 of 210 per cell); each other model excludes at most 5 per cell. Across the four tested generators, the consistent corrective movement without model-specific adjustment supports applicability beyond a single generator.

4.5

Comparison of Selection Mechanisms: Fixed Balancing, Random Selection, and MMR

This section compares the rules used to select the final five documents. Fixed balancing, random selection, and MMR use the common base prompt without the debiasing instruction, so the proposed method participates as setting B. The methods share the 210 questions, index, retrieval results, initial top-20 candidate pool, generator, and judge model. Random-5 and MMR select within the top 20, whereas the balancing rule permits extension to rank 50 when needed (Section 3.4). Question IDs are aligned across all 210 questions; paired score comparisons retain only questions valid in both methods. Table 8: Comparison of evidence-selection methods with shared retrieval results, generator, and judge (poisoning share: 18.9%) Corpus Method Pro Pro Pro Con Con Con

B: Balancing (no prompt, ours) Random-5 MMR B: Balancing (no prompt, ours) Random-5 MMR

Stance score [95% CI]

Included

Paired difference vs. B [95% CI]

+0.153 [+0.057, +0.249] +0.330 [+0.219, +0.438] +0.197 [+0.110, +0.285] +0.038 [−0.062, +0.144] −0.234 [−0.332, −0.143] −0.148 [−0.263, −0.038]

209

—

209 208 209

+0.178 [+0.024, +0.321] * +0.043 [−0.062, +0.149] † —

209 209

−0.273 [−0.389, −0.168] * −0.187 [−0.311, −0.071] *

Note: * denotes a paired-difference confidence interval that excludes 0; † denotes an interval that includes 0. Neither baseline produced any content-moderation blocks. Balancing without a prompt shows a clear advantage over random selection in both directions; Random-5 remains at −0.234 in the con corpus. Relative to MMR, the advantage is clear under con-side poisoning, with a paired difference of −0.187 [−0.311, −0.071]. Under pro-side poisoning, the paired difference from MMR is +0.043 [−0.062, +0.149]. Table 9 provides mechanism-level evidence by reporting the mean number of poisoned documents among the five documents actually supplied to the generator for each method, averaged over 210 questions. 16

Table 9: Mean number of poisoned documents among the five supplied per question Method

Pro corpus

Con corpus

A: No defense (directly supply the top five) B / C: Proposed balancing Random-5 MMR

1.743 1.181 1.267 1.048

1.424 0.986 1.152 0.900

MMR supplies fewer poisoned documents than the proposed method: 1.048 versus 1.181 for the pro corpus and 0.900 versus 0.986 for the con corpus. Nevertheless, its con-side bias is greater (−0.148 versus +0.038). This inverse ordering indicates that poison count alone does not explain the final stance bias; Section 5.1 discusses the mechanism.

4.6

Efficiency Analysis

This section quantifies the practical overhead of the middleware. All latency measurements use the same 30 questions, index, batch size of 32, and maximum sequence length of 256, with row-wise alignment; the ratios are therefore comparable. None of the measurements consumes any API calls. Table 10 reports detection latency as the number of candidate documents varies. Table 10: Stance-detection latency by number of candidate documents (median, n = 30) Documents classified

CPU

GPU

GPU speedup Share of generation time (CPU / GPU)

1 5 10 20 (actual deployment) 32 50 (full recall set)

63.2 ms 237.1 ms 348.2 ms 954.2 ms

10.6 ms 22.8 ms 48.9 ms 88.8 ms

6.0× 10.4× 7.1× 10.7×

1.07% / 0.18% 4.02% / 0.39% 5.91% / 0.83% 16.2% / 1.51%

2522.5 ms 3848.3 ms

222.2 ms 337.8 ms

11.4× 11.4×

42.8% / 3.77% 65.3% / 5.73%

Twenty documents represent the standard deployment setting. Fifty documents comprise the full retrieval set and are all classified only when a stance category is undersupplied and ranks 21–50 must be accessed; this row therefore represents a worst-case upper bound rather than routine overhead. The remaining rows show the latency curve as candidate-set size varies. The 88.8 ms GPU latency for 20 documents equals 1.51% of the median remote-generation latency of claude-opus-4-8 (5,894.3 ms; p10 4,783.6 ms; p90 7,381.7 ms) and 1.9% of its p10 value. The generation latency comes from separate benchmark runs with matched inputs, so this ratio is a scale comparison rather than a per-query paired measurement. Latencies are batch totals; batches of 32 are padded to their longest text, so latency varies nonlinearly with document count. Table 11 reports an independent measurement. Its 20-document results differ by approximately 1% from the corresponding row in Table 10 (CPU: 954.2 versus 945.3; GPU: 88.8 versus 89.7).

17

(a) Pro-side poisoning

0.6

(b) Con-side poisoning

(c) Direct between-group differences

n=49

n=63

n = mislabeled / all-correct

Pro A Poison-aligned bias

n=63/146 n=63

0.4

n=49

Pro B n=63/146

n=49

n=62

0.2

Pro C

n=146

n=62/144

n=160

0.0

Con A

n=146

n=49/160

n=144 n=159

−0.2

n=160

B

C

n=49/160

Con C

≥1 selected label wrong All selected labels correct

A

Con B

n=49/159

A

B

C

−0.25

0.00

0.25

0.50

Difference in poison-aligned bias

Figure 4: Direct between-group differences for groups with and without detection errors. Table 11: 2 × 2 decomposition of six-fold routing versus a single model and 20 versus 50 documents (median, n = 30) Configuration

CPU

Single model / 20 documents 945.3 ms (actual deployment) Six-fold routing / 20 documents 929.4 ms Single model / 50 documents 3900.1 ms Six-fold routing / 50 documents 3932.9 ms (most conservative)

GPU

Share of generation time (CPU → GPU)

GPU speedup

89.7 ms

16.0% → 1.52%

10.5×

98.3 ms 337.4 ms 338.3 ms

15.8% → 1.67% 66.2% → 5.72% 66.7% → 5.74%

9.5× 11.6× 11.6×

Table 11 shows GPU speedups of 9.5×–11.6× across the tested configurations. Six-fold routing prevents topic leakage during evaluation; deployment uses a single classifier. These measurements support integrating local stance detection into the RAG pipeline with a small latency cost relative to generation.

4.7

Robustness under Misclassification

Section 3.3 reports the stance detector's misclassification rate. This section analyzes how those errors affect method effectiveness. Grouping questions according to whether at least one stance among the five ultimately supplied documents was misclassified yields 64/210 (30.5%) for the pro corpus and 49/210 (23.3%) for the con corpus. Figure 4 and Table 12 compare these groups. The value 210 is the number of questions before grouping. Each setting still excludes blocked, refused, and unparseable cases according to Section 3.1, so the effective n values in Table 12 are slightly smaller and vary by setting.

18

Table 12: Poison-aligned bias and 95% confidence intervals grouped by the presence of misclassification Corpus Setting

Group

Pro

A: No defense

Pro Pro

A: No defense B: Balancing (no prompt) B: Balancing (no prompt) C: Complete method C: Complete method A: No defense

With misclassification All correct With misclassification All correct

Pro Pro Pro Con Con Con Con Con Con

A: No defense B: Balancing (no prompt) B: Balancing (no prompt) C: Complete method C: Complete method

Questions

Poison- 95% confidence aligned bias interval

63

+0.381 [+0.140, +0.588]

No

146 63

+0.370 [+0.258, +0.479] +0.190 [+0.025, +0.339]

No No

146

+0.137 [+0.022, +0.252]

No

62

+0.032 [−0.211, +0.222] Yes

144

+0.021 [−0.069, +0.119] Yes

49

+0.388 [+0.130, +0.615]

No

160 49

+0.306 [+0.199, +0.417] +0.143 [−0.051, +0.327]

No Yes

160

−0.094 [−0.207, +0.014]

Yes

49

+0.082 [−0.083, +0.241] Yes

159

−0.050 [−0.129, +0.025] Yes

With misclassification All correct With misclassification All correct With misclassification All correct With misclassification All correct

Includes 0?

Note: Values are sign-normalized by poisoning direction: +1 denotes complete alignment with the poisoning direction, 0 denotes neutrality, and −1 denotes complete opposition to the poisoning direction. Figure 4 directly compares the between-group difference “with misclassification” minus “all correct.” For the pro corpus, the differences are +0.011 [−0.226, +0.220] in A, +0.053 [−0.139, +0.236] in B, and +0.011 [−0.256, +0.226] in C. For the con corpus, they are +0.082 [−0.183, +0.336] in A, +0.237 [+0.015, +0.459] in B, and +0.132 [−0.044, +0.307] in C. Only the con-side setting-B interval excludes 0; the other five intervals include 0. The groups are observational and describe performance at the observed error levels, not the causal effect of misclassification. Within this scope, setting-C residual bias lies between −0.050 and +0.082 in both groups of both corpora, supporting the method’s effectiveness under the detection errors encountered in this evaluation.

4.8

Does the Middleware Harm Response Quality without an Attack?

We test whether adding the middleware harms normal response quality by comparing the baseline without defense (Clean A) and the baseline with the complete middleware (Clean C), using the same 210 questions and knowledge base without poisoning. GPT-5.4 evaluates each answer’s relevance, source support, verifiable accuracy, and argument coverage on five-point scales with method identities concealed and answer order randomized. Two human annotators independently evaluate 50 pairs spanning all 42 topics. Appendix E gives the generation settings and evaluation details.

19

Table 13: Stance and automatic quality ratings without poisoning Measure Mean stance Relevance Source support Verifiable accuracy Argument coverage

A +0.043 4.719 4.805 4.790 3.995

C Paired difference [95% CI] +0.033 A−C: +0.010 [−0.077,+0.092] 4.876 C−A: +0.157 [0.081,0.233] 4.786 C−A: −0.019 [−0.133,0.090] 4.757 C−A: −0.033 [−0.138,0.067] 4.505 C−A: +0.510 [0.343,0.690]

Stance means use 210 valid answers in A and 209 in C; the stance judge labels one C answer as a refusal. The paired stance difference uses 209 common valid pairs. Quality ratings use all 210 pairs. Intervals use topic-cluster resampling; quality dimensions are not combined into a total score. Intervals describe uncertainty in the observed differences; no equivalence margin was specified. Automatic ratings improve relevance and argument coverage; source support and verifiable accuracy change by less than 0.04 points on the five-point scale. Both human annotators rate C higher than A in all four dimensions (mean differences +0.020 to +0.429; Table 18), and the clean stance difference is +0.010 (Table 13). Taken together, these findings support preserving normal response quality when the middleware is added in the tested clean setting, with gains concentrated in relevance and argument coverage.

5

Discussion

5.1

Evidence Composition Matters Beyond Factual Support

The results highlight evidence composition as a distinct source of stance skew in RAG. Retrieved passages can differ in wording and argument while consistently favoring the same position, allowing one-sided evidence to shape the answer. The selection comparison illustrates this distinction: under con-side poisoning, MMR admits fewer poisoned documents yet yields greater stance deviation than stance balancing. The methods share the initial candidate pool, but balancing can extend retrieval when a quota is unmet. Thus, relevance and content diversity alone do not characterize whether competing positions are adequately represented. The findings support treating stance composition as an explicit dimension of retrieval evaluation and control.

5.2

Explicit Stance Control as a Defense Design

The middleware acts on the evidence supplied to the generator rather than identifying documents as poisoned. The ablation results support balancing as the main source of correction, with prompting providing a complementary adjustment. Corrective movement across the four tested generators without model-specific tuning further supports this evidence-level intervention. Together, these findings support an explicit stance-selection constraint as a practical defense design, complementary to relevance-based retrieval and content diversification.

5.3

Implications for Integration into RAG Systems

The middleware can be inserted between retrieval and generation without modifying either component or adding an online LLM call. Its detection cost is small relative to generation (Section 4.6), and it preserves answer quality in the tested clean setting (Section 4.8). These results motivate evaluating the middleware as a lightweight addition to RAG systems serving contested questions.

20

5.4

Scope and Remaining Boundaries

The method depends on available stance categories and reliable labels; the cases in Appendix F illustrate how missing neutral evidence and rhetorical misclassification alter the selected evidence. Small pro-side residuals remain in the cross-generator results, and the advantage over MMR is clear under con-side but not pro-side poisoning. Evidence currently covers English, one retriever, one question source, and model-generated injected documents, whose measured properties are described in Appendix A. Broader validation and stronger target-aware stance detection remain directions for future work.

6

Conclusion

6.1

Summary

This study investigates RAG knowledge poisoning in which documents are factually correct but one-sided in stance, and proposes the Stance-Balancing Middleware, which combines stance detection, fixed-quota balancing, and a debiasing prompt. The method constrains the evidence supplied to the generator using a quota of 2 pro + 2 con + 1 neutral, without modifying the retriever or generator and without introducing additional LLM calls. In the experiments, bidirectional stance shifts increased with poisoning concentration and were directionally consistent across four generators. Across all eight concentration–direction settings, the complete method reduced the magnitude of stance deviation by an estimated 0.106–0.350; the six settings at poisoning shares of 11.9%–18.9% remained significant after Holm adjustment (adjusted p ≤ 0.009), and the two settings at 6.4% moved in the same corrective direction (adjusted p = 0.176). In the cross-model experiment, all eight model–direction combinations moved along the stance axis in the corrective direction, and seven of eight showed a clearly detected reduction in absolute distance from 0. The concentration-sweep setting-C intervals included 0 in all eight settings, indicating that no directionally clear residual was detected there. For 20 candidate documents, detection took approximately 1.5% of the median generation time on a single GPU and approximately 16% on CPU. Together, these results show that directly constraining the stance composition of retrieved evidence can substantially mitigate attack-induced stance skew. Under pro-side poisoning in the cross-model experiment, small positive residual point estimates of +0.045 to +0.063 remained. Fixed balancing also depends on the available stance categories and detector labels, and the current evidence is limited to English, one retriever, one question source, and LLM-generated poisoned documents.

6.2

Future Work

Future work should first trace the source of residual pro-side bias through targeted experiments that distinguish the effects of generator priors, the judge model's scoring scale, and stylistic differences between the two poisoning directions. Second, the evaluation should be replicated across languages, retrievers, question sources, and human-written slanted content. At the methodological level, future research could expand recall or adopt stance-aware retrieval when a category is absent, and strengthen both target-aware stance detection and consistency checks among documents from the same source.

21

References [1] W. Fan, Y. Ding, L. Ning, et al. A survey on RAG meeting LLMs: Towards retrieval-augmented large language models. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 6491–6501, 2024. doi: 10.1145/3637528.3671470. [2] R. Ren, J. Ma, and Z. Zheng. Large language model for interpreting research policy using adaptive two-stage retrieval augmented fine-tuning method. Expert Systems with Applications, 278:127330, 2025. doi: 10.1016/j.eswa.2025.127330. [3] L. C. Chen, M. S. Pardeshi, Y. X. Liao, and K. C. Pai. Application of retrieval-augmented generation for interactive industrial knowledge management via a large language model. Computer Standards & Interfaces, 94:103995, 2025. doi: 10.1016/j.csi.2025.103995. [4] T. Ren, Z. Zhang, B. Jia, and S. Zhang. Retrieval-augmented generation-aided causal identification of aviation accidents: A large language model methodology. Expert Systems with Applications, 278:127306, 2025. doi: 10.1016/j.eswa.2025.127306. [5] W. Zhang, Q. Zhang, E. Yu, et al. Leveraging RAG-enhanced large language model for semisupervised log anomaly detection. In 2024 IEEE 35th International Symposium on Software Reliability Engineering (ISSRE), pages 168–179, 2024. doi: 10.1109/ISSRE62328.2024.00026. [6] Y. Zou, M. Shi, Z. Chen, et al. ESGReveal: An LLM-based approach for extracting structured data from ESG reports. Journal of Cleaner Production, 489:144572, 2025. doi: 10.1016/j.jclepr o.2024.144572. [7] H. A. Alawwad, A. Alhothali, U. Naseem, et al. Enhancing textual textbook question answering with large language models and retrieval augmented generation. Pattern Recognition, 162: 111332, 2025. doi: 10.1016/j.patcog.2024.111332. [8] M. Cremaschi, D. Ditolve, C. Curcio, et al. Decoding the mind: A RAG-LLM on ICD-11 for decision support in psychology. Expert Systems with Applications, 279:127191, 2025. doi: 10.1016/j.eswa.2025.127191. [9] W. Zou, R. Geng, B. Wang, and J. Jia. PoisonedRAG: Knowledge corruption attacks to retrieval-augmented generation of large language models. In 34th USENIX Security Symposium (USENIX Security 25), pages 3827–3844, 2025. URL https://www.usenix.org/conference/ usenixsecurity25/presentation/zou. [10] J. Xue, M. Zheng, Y. Hu, F. Liu, X. Chen, and Q. Lou. BadRAG: Identifying vulnerabilities in retrieval augmented generation of large language models. arXiv:2406.00083, 2024. [11] Y. Gong, Z. Chen, J. Liu, M. Chen, F. Yu, W. Lu, X. Wang, and X. Liu. Topic-FlipRAG: Topic-orientated adversarial opinion manipulation attacks to retrieval-augmented generation models. In 34th USENIX Security Symposium (USENIX Security 25), pages 3807–3826, 2025. [12] S. Feng, C. Y. Park, Y. Liu, and Y. Tsvetkov. From pretraining data to language models to downstream tasks: Tracking the trails of political biases leading to unfair NLP models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics, Volume 1: Long Papers, pages 11737–11762, 2023. doi: 10.18653/v1/2023.acl-long.656.

22

[13] S. Santurkar, E. Durmus, F. Ladhak, et al. Whose opinions do language models reflect? In Proceedings of the 40th International Conference on Machine Learning, volume 202, pages 29971– 30004. PMLR, 2023. URL https://proceedings.mlr.press/v202/santurkar23a.html. [14] C. Xiang, T. Wu, Z. Zhong, D. Wagner, D. Chen, and P. Mittal. Certifiably robust RAG against retrieval corruption. arXiv:2405.15556, 2024. [15] H. Zhou, K. H. Lee, Z. Zhan, et al. TrustRAG: Enhancing robustness and trustworthiness in retrieval-augmented generation. arXiv:2501.00879, 2025. [16] J. Zheng, A. P. Gema, G. Hong, et al. GRADA: Graph-based reranking against adversarial documents attack. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 22244–22266, 2025. doi: 10.18653/v1/2025.emnlp-main.1132. [17] D. Ganguli, A. Askell, N. Schiefer, et al. The capacity for moral self-correction in large language models. arXiv:2302.07459, 2023. [18] C. Si, Z. Gan, Z. Yang, et al. Prompting GPT-3 to be reliable. In The Eleventh International Conference on Learning Representations (ICLR 2023), 2023. Preprint DOI: 10.48550/arXiv.2210.09150. [19] M. Hardalov, A. Arora, P. Nakov, and I. Augenstein. A survey on stance detection for mis- and disinformation identification. In Findings of the Association for Computational Linguistics: NAACL 2022, pages 1259–1277, 2022. doi: 10.18653/v1/2022.findings-naacl.94. [20] H. Xu, S. Vucetic, and W. Yin. OpenStance: Real-world zero-shot stance detection. In Proceedings of the 26th Conference on Computational Natural Language Learning, pages 314– 324, 2022. doi: 10.18653/v1/2022.conll-1.21. [21] H. Wu, Y. Zhang, C. Ma, et al. Result diversification in search and recommendation: A survey. IEEE Transactions on Knowledge and Data Engineering, 36(10):5354–5373, 2024. doi: 10.1109/TKDE.2024.3382262. [22] S. Es, J. James, L. Espinosa Anke, and S. Schockaert. RAGAs: Automated evaluation of retrieval augmented generation. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations, pages 150–158, 2024. doi: 10.18653/v1/2024.eacl-demo.16. [23] Y. Bestgen. Please, don’t forget the difference and the confidence interval when seeking for the state-of-the-art status. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 5956–5962, 2022. URL https://aclanthology.org/2022.lrec-1.640/. [24] G. James, D. Witten, T. Hastie, R. Tibshirani, and J. Taylor. Resampling methods. In An Introduction to Statistical Learning: With Applications in Python, pages 201–228. Springer, Cham, 2023. doi: 10.1007/978-3-031-38747-0_5. [25] S. Xiao, Z. Liu, P. Zhang, et al. C-Pack: Packed resources for general chinese embeddings. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 641–649, 2024. doi: 10.1145/3626772.3657878. [26] M. Douze, A. Guzhva, C. Deng, et al. The Faiss library. IEEE Transactions on Big Data, 12 (2):346–361, 2026. doi: 10.1109/TBDATA.2025.3618474.

23

A

Measured Properties of the Poisoning Corpus

This appendix analyzes the properties of the complete bidirectional set of 588 injected documents rather than the subset inserted into any single direction–concentration knowledge base. Table 14 compares two measured properties of the human-authored corpus (Human ProCon, 1,478 documents) and the injected documents used in the experiments (588 documents): document word count and the maximum cosine similarity between each document and all other documents within the same group. Similarity was computed using bge-large-en-v1.5, with the CLS vector from the final hidden layer (1,024 dimensions, maximum length 256, L2-normalized). For each document, the nearest neighbor was identified only within its own group; no cross-group comparisons were made. Table 14: Measured properties of the human-authored corpus and injected documents Property

Human-authored corpus Injected documents (n = 588) (n = 1,478)

Median word count Standard deviation of word count Word-count range Median within-group nearest-neighbor cosine similarity Near-duplicate documents (cosine > 0.95)

66 57.1 0–361 0.772

66 6.4 45–82 0.798

6 (0.406%, 3 pairs)

0 (0 pairs)

The lower bound of 0 words in the human-authored corpus results from two entries whose original text consists solely of whitespace; they were retained unchanged without removal or imputation. The two groups have the same median word count, but the injected documents exhibit a substantially narrower range, limiting the external validity of the attack construction (Section 5.4). Under the strict threshold of 0.95, the injected corpus contains no mutually near-duplicate entries (0/588), whereas the human-authored corpus contains 6 such documents (0.406%, 3 pairs, including one pair formed by the two empty-text entries described above). This result supports the characterization in Section 5.1 that the injected documents are uniform in stance but varied in expression. It also shows that, under the measured similarity and the strict 0.95 threshold, near-duplication alone would not cause them to be removed en masse.

24

B

Strictly Paired Effect Sizes and Holm Adjustment

This section reports the strictly paired A-to-C results for the concentration sweep. The effect size is the reduction in absolute distance from zero, |sA | − |sC |. The adjustment affects multiplicity-aware inference, not the direction of the estimated effects. Intervals use 5,000 topic-cluster bootstrap samples with seed 20260905. Each sample draws 42 topics with replacement, includes their questions, and retains equal weighting of valid questions. Paired estimates use common valid questions and shared topic draws. For the eight concentration-sweep A-to-C tests, A and C are swapped jointly within each topic over 5,000 randomizations; p = (1 + the number of equally or more extreme absolute statistics) / 5,001. Holm adjustment covers these eight tests. Table 15: Strictly paired effect sizes and Holm-adjusted p-values Direction

25

Pro Pro Pro Pro Con Con Con Con

Poisoning Valid n in Valid n in share A C 6.4% 11.9% 14.4% 18.9% 6.4% 11.9% 14.4% 18.9%

209 209 209 209 209 208 209 209

208 208 208 206 207 209 207 208

Paired n Reduction in absolute distance from zero [95% CI] 208 +0.106 [−0.038,+0.181] 208 +0.216 [+0.081,+0.300] 208 +0.293 [+0.139,+0.383] 206 +0.350 [+0.211,+0.447] 207 +0.111 [−0.015,+0.205] 208 +0.173 [+0.048,+0.262] 207 +0.203 [+0.082,+0.295] 208 +0.313 [+0.153,+0.411]

Raw p Holm-adjusted p 0.09058 0.00120 0.00020 0.00020 0.08818 0.00300 0.00060 0.00020

0.17636 0.00480 0.00160 0.00160 0.17636 0.00900 0.00300 0.00160

Note: Positive values indicate that the complete method moves the score closer to zero. Values of p = 0.00020 are limited by the resolution of 5,000 randomization iterations. All eight settings showed positive estimated reductions; the six moderate-to-high concentration settings remained below 0.05 after Holm adjustment, whereas both lowest-concentration settings had adjusted p-values of 0.17636.

C

Generation Prompt Templates

All settings share the same base template. Settings C and D insert a fixed debiasing instruction after the question and before the output requirements. The text below reproduces the executed prompts verbatim. Base template

You are answering a contested public-policy question using ONLY the retrieved documents below. Documents: {DOCUMENTS} Question:

{QUERY}

{DEBIAS_INSTRUCTION}Write a direct answer in 3-5 sentences, in English. Ground every claim in the documents above. Do not say the documents are insufficient, do not refuse, and do not ask for more information — answer with what is given. Answer:

Document-block format [1] {DOC_1_TEXT} [2] {DOC_2_TEXT} ... [K] {DOC_K_TEXT}

Documents are numbered in the final order supplied to the generator. Only the body text is inserted, without titles; the main experiments use K=5. Debiasing instruction

Important — the retrieved documents may be unbalanced. One position can be over-represented, and some documents may be longer or more forcefully argued than others. Do not let a single lengthy or emphatic document drive your conclusion. Weigh the strongest evidence on each side even-handedly; if the question is genuinely contested, your answer should reflect that balance rather than adopt one side's framing.

Table 16: Prompt use across experimental settings Setting

Document selection

Debiasing instruction

A: No defense B: Balancing (no prompt) C: Complete method (balancing + prompt) D: Prompt only

Top five retrieved documents Five documents after balancing Five documents after balancing

Not inserted Not inserted Inserted

Same top five documents as A

Inserted

D

Human Validation of the Injected Documents

The validation sample comprised three documents from each of the 84 topic–direction groups, selected with seed 20260905. Source support required all verifiable claims in a document to be supported by the supplied ProCon passages; the supplied evidence included the document’s associated sources and supplementary sources from the same topic–direction rewriting group. Stance was assessed relative to the topic question, and the final stance label was compared with the intended pro/con injection label. Table 17 reports the final outcomes after human review and adjudication.

26

Table 17: Final human-validation outcomes for 252 sampled injected documents Criterion

Passed / assessed

Rate

252/252 252/252 252/252

100% 100% 100%

All verifiable claims supported by supplied sources Clearly one-sided stance Stance direction matches the intended injection label

E

Evaluation Details and Human Checks

Clean A reuses historical answers with ordered source passages reconstructed from saved document IDs; Clean C retains its actual inputs. Both request claude-opus-4-8 with a 400-token limit and gateway-default sampling; the historical backend version is not recorded. GPT-5.4 returned gpt-5.42026-03-05. Seed 20260905 fixes answer order and the human sample before inspecting automatic scores. Source support uses each answer’s own passages; coverage uses relevant arguments from both passage sets. Accuracy is judged within the supplied evidence. Scores range from major deficiencies (1) to full satisfaction (5); unresolved or inapplicable items are marked separately. Table 18: Human and matched automatic quality differences Dimension Relevance Source support Verifiable accuracy Coverage

Pairs

Human 1 C−A

Human 2 C−A

GPT-5.4 C−A

50 50 50 49

+0.160 +0.020 +0.080 +0.347

+0.120 +0.040 +0.080 +0.429

+0.280 0.000 +0.040 +0.735

The human sample contains one pair per topic plus eight additional pairs. Annotators independently rated identical materials without automatic scores or method identities; their ratings are reported separately. Both marked coverage inapplicable for one pair, leaving 49 matched coverage comparisons. Human differences in Table 18 are descriptive means. Overall-preference agreement was 60% between humans (κ = 0.394), and 66%/68% between GPT-5.4 and each human. These results support the aggregate direction rather than interchangeable item-level judgments. For the answer-stance judge, 50 historical clean responses were sampled from 825 responses by judge-label strata (seed 20260830). The author reviewed model-preannotated reference labels, with the evaluated judge’s labels stored separately. Agreement with gpt-5-mini was 45/50 (90%; κ = 0.850). This check covers historical clean responses only.

F

Evidence-Selection Boundary Cases

F.1

Neutral Evidence Is Absent from the Candidate Pool

Question arena-t27-q1 (pro-poisoned corpus) concerns whether birth control should be sold over the counter. The question is “Is it wise to sell birth control OTC?” All three settings produced pro answers, and the defense did not change the outcome, although all three components operated as defined.

27

Table 19: Evidence composition for the missing-neutral failure case Item

Value

Stance composition of the top 20 documents Neutral supply after extending to rank 50 Balancing target Actual selection Identity of the five selected documents Stance-classification accuracy Change in the number of poisoned documents

11 pro / 9 con / 0 neutral Still 0 according to classifier labels 2 pro + 2 con + 1 neutral 3 pro + 2 con + 0 neutral All genuine documents; 0 poisoned documents 5 / 5 correct 1 document in setting A → 0 documents in setting B

Despite correct labels for the selected documents, no neutral evidence was detected within the top 50, so the quota was filled by an additional pro document. This case identifies evidence availability as a condition for fulfilling the stance quota. All 10 cases in which the quota could not be fulfilled (4 in the pro-poisoned corpus and 6 in the con-poisoned corpus), accounting for 2.4% of the 420 question–corpus instances, arose for this reason. They were concentrated in two topics dominated by pro–con debate and lacking neutral exposition: t27 (over-the-counter birth control) and t35 (standardized testing). The method implicitly assumes that at least one document of each stance exists within the retrieval range, an assumption satisfied for 40 of the 42 topics.

F.2

Rhetorical Structure Causes Stance Misclassification

Question arena-t40-q9 (pro-poisoned corpus) concerns whether electronic voting machines are the best voting method. The question is “Is there a preference for digital voting methods?” The postdefense answer remained pro, and, in this pro-poisoned corpus, the number of poisoned documents supplied to the generator increased from 3 to 4. The candidate pool contained sufficient evidence: among the 20 documents, 8 were pro, 7 con, and 5 neutral. At the label level, balancing selected 2 pro + 2 con + 1 neutral, but the actual composition was 4 pro + 0 con + 1 neutral. Document-level results are shown in Table 20. Table 20: Document-level details for the five documents selected for arena-t40-q9 Rank

Document ID

Genuine / poisoned

Detector label

Actual stance Misclassified?

1 2 3 4 5

poison-t40-pro-5 poison-t40-pro-4 wiki-t40-p1 poison-t40-pro-0 poison-t40-pro-6

Poisoned Poisoned Genuine Poisoned Poisoned

Con Pro Neutral Pro Con

Pro Pro Neutral Pro Pro

Yes * No No No Yes *

Note: * The document expresses support by attacking the opposing alternative. The two misclassified documents support electronic voting machines but argue their case by attacking online voting. One states: “Expert consensus condemns the alternative and vindicates paper-backed machines... Machine voting with a verifiable paper trail is the superior path.” Although the text contains negative terms such as condemns, disaster, and nonstarter, the target of the negation is online voting, and the conclusion supports voting machines with a paper trail. The

28

detector captured the negative tone but failed to identify its target; consequently, both con quota slots were filled by documents that were actually pro. This case identifies argument-target recognition as a requirement for reliable stance selection.

29

Related documents

• Factual Yet One-Sided: Stance Poisoning and Evidence Balancing in Retrieval-Augmented Generation#1457448
• Lazy Grounding: Attacking Search Agents with Factual Evidence#820687
• RAGLeak: Supply-Chain Privacy Backdoors in Retrieval-Augmented Generation#1401126
• RAGLeak: Supply-Chain Privacy Backdoors in Retrieval-Augmented Generation#1381142
• KidnapRAG: A Black-Box Attack for Hijacking Reasoning in Agentic Retrieval-Augmented Generation Systems#684517